Skip to main content
    Path to Science

    Thinking Dictionary

    Scientific models

    What it is

    A scientific model is a simplified version of something that is too big, too small, too slow, or too complicated to study directly. A model can be a map, a diagram, an equation, or a computer program that imitates how something behaves. Computer models like that are called simulations.

    A model keeps the parts that matter for a particular question and leaves out the rest. A good model makes predictions that can be checked. When the checks fail, it gets improved or replaced.

    Every model leaves something out

    The statistician George Box argued that all models are, in a sense, wrong, because every one of them simplifies reality.¹ His famous version of the idea, "all models are wrong, but some are useful," dates from 1979.²

    Leaving things out is not a flaw. It is the point. A map of an underground railway is useful because it leaves out the streets above. So the real questions are: Is the model useful for this purpose? And has it been checked against real data?

    Focus on the big factors first

    Models can't include everything, so a useful habit is to start with the factors that matter most. Physicists call the biggest influences on a result "first-order" effects and the smaller ones "second-order."

    Factors really do differ in size. A large study of health risks around the world ranked 87 of them by the deaths linked to each. High blood pressure, tobacco, and poor diet were among the leading risks.³ So when you judge a factor, ask how big its effect is, not just whether it has one. A real effect can still be tiny.⁴

    Focusing on big factors first doesn't mean ignoring small ones forever. Small effects can add up, or matter a lot when many people are affected.

    Why it matters

    We meet models all the time, often without noticing: in weather forecasts, in climate projections, and in plans for traffic or disease. If we understand what a model is, we can avoid two mistakes. One is treating its output as certain. The other is dismissing it as "just a model."

    How reliable a model is can be tested. You compare what it predicted with what actually happened.⁵

    Two real tests

    Weather forecasts. Weather forecasts come from computer models of the atmosphere. These models have improved slowly and steadily over decades. Experts describe a gain of very roughly one extra day of useful forecast for each decade of work. In other words, a forecast for six days ahead today is about as good as a five-day forecast was ten years earlier.⁶

    Climate projections. Researchers looked back at climate model projections published between 1970 and 2007. They compared them with the warming that actually happened afterwards. Most of the models did reasonably well. They matched even better once the researchers allowed for differences between the pollution levels the models had assumed and the levels that actually occurred.⁵ That kind of check, made years later, is a strong test of a model.

    In everyday life

    • A recipe. A recipe is a simple model of a dish. It leaves out your oven's quirks and the size of your eggs. It is still useful, and you adjust it when the result doesn't match.
    • Comparing two models. Imagine two planners build different models of how many people will ride a town's buses. Both match the past few years of counts. But one predicts growth and the other decline. The only way to tell which is better is to check them against new counts. Two models can fit the past equally well and still disagree about the future.¹
    • Your weather app. When the app says "rain tomorrow," that comes from a model. It is usually right about tomorrow, and less often right about next week.

    The common trap

    The trap is thinking that a model that fits the past must be right. Fitting old data is not enough. A model also needs to predict new data that it wasn't built on.

    The opposite trap is saying "it's just a model," as if that settles it. Models are not wild guesses. They are built from known relationships and tested. Whether a particular model is reliable is a question you answer with evidence.⁵

    Some ways out:

    • Ask what the model is for, and what it leaves out.
    • Ask how it has been tested against real data.
    • Ask whether its predictions have come true before.

    Try it

    Test your local weather forecast as a model. Each day for a week, write down the forecast high temperature for three days ahead. When that day arrives, write down the actual high. Then do the same for the forecast one day ahead.

    How far off was each forecast on average? You will probably find the one-day forecasts were closer than the three-day ones. Forecasts get less accurate the further ahead they look. Forecasters check their own models in much the same way.

    Related concepts

    Iteration · Fermi estimation · Probabilistic reasoning · Accuracy vs precision · Multiple lines of evidence · Scientific uncertainty · Scenario planning

    Sources

    1. Box, G. E. P. (1976). Science and statistics. Journal of the American Statistical Association, 71(356), 791–799. https://doi.org/10.1080/01621459.1976.10480949

    2. Box, G. E. P. (1979). Robustness in the strategy of scientific model building. In R. L. Launer & G. N. Wilkinson (Eds.), Robustness in Statistics (pp. 201–236). Academic Press. https://doi.org/10.1016/B978-0-12-438150-6.50018-2

    3. GBD 2019 Risk Factors Collaborators [Murray, C. J. L., et al.]. (2020). Global burden of 87 risk factors in 204 countries and territories, 1990–2019: A systematic analysis for the Global Burden of Disease Study 2019. The Lancet, 396(10258), 1223–1249. https://doi.org/10.1016/S0140-6736(20)30752-2

    4. Funder, D. C., & Ozer, D. J. (2019). Evaluating effect size in psychological research: Sense and nonsense. Advances in Methods and Practices in Psychological Science, 2(2), 156–168. https://doi.org/10.1177/2515245919847202

    5. Hausfather, Z., Drake, H. F., Abbott, T., & Schmidt, G. A. (2020). Evaluating the performance of past climate model projections. Geophysical Research Letters, 47(1), e2019GL085378. https://doi.org/10.1029/2019GL085378

    6. Bauer, P., Thorpe, A., & Brunet, G. (2015). The quiet revolution of numerical weather prediction. Nature, 525(7567), 47–55. https://doi.org/10.1038/nature14956