Skip to main content
    Path to Science

    Thinking Dictionary

    Signal vs noise

    What it is

    The signal is the real change or pattern you care about. The noise is the random up-and-down that happens anyway. Almost every single measurement contains some of both.¹ So one reading on its own rarely tells you much about the real trend. To tell signal from noise, look at many measurements over time, not one surprising number.

    Why it matters

    We react to single numbers all the time. A test score drops. A poll moves. A reading on a gadget jumps. It is natural to think something has changed. Sometimes it has. But often the change is just noise, the normal wobble that happens even when nothing real is going on.

    Noise doesn't mean the data are bad. All measurements have some. The real question is how big the noise is, compared with the change you think you see.

    Example: opinion polls

    Polls usually come with a margin of error, for example "plus or minus 3 points." It tells you how much a result could wobble just because only some people were asked, not everyone.

    Now imagine two polls, a month apart, both with a margin of plus or minus 3 points. A candidate's support goes up by 3 points. Has anything changed? You can't tell. The margin of error for the change between two such polls is much bigger, about plus or minus 8 points. A 3-point shift sits well inside the noise.¹

    What helps is looking at many polls over time. A trend that shows up again and again across many polls is far more trustworthy than a jump between two.¹

    Example: the "improvement" that was just noise

    Imagine you measure something that naturally goes up and down, like your blood pressure or how stressed you feel. You only take action on days when the reading is unusually high. Then, the next time you measure, it's lower. It looks as if your action worked.

    But unusually high readings tend to be followed by more ordinary ones, even if you do nothing. This is called regression to the mean: extreme measurements tend to be followed by measurements closer to the average.² It is a classic way to mistake noise for a signal. The Placebo effect entry explores it in more detail.

    In everyday life

    • Step counts. Imagine your phone's daily step count. It jumps around from day to day. But over a few months, a slow seasonal rise or fall may show through. The daily jumps are mostly noise. The months-long trend is the signal.
    • Your bathroom scale. One morning's reading tells you little. In a study of 80 adults who weighed themselves daily, weight tended to rise over weekends and fall again during the week, in a regular rhythm. The researchers said swings like these should be seen as normal, not as real weight gain.³ The trend over several weeks tells you more than any single day.
    • Sales, scores, and step counts. A single bad week at work, a low test score, or a lazy day on your step counter may simply be noise. Look at the pattern over time before deciding something has changed.

    The common trap

    The trap is thinking "this number changed, so something happened." Maybe it did. But first, ask how big normal variation is.

    A related trap is getting excited about outliers, the readings that sit far away from the rest. Sometimes they are the most interesting part of the data. But first check that they aren't just noise.

    Some ways out:

    • Ask how big the noise usually is. A change smaller than the usual wobble may be nothing.
    • Look for repeats. A real signal tends to show up again and again.
    • Watch out for picking extremes. If you only act when things are at their worst, things will often seem to improve anyway.²

    One more warning. Taking more measurements helps you see through random noise. But it can't fix an error that pushes every reading the same way. A poll's margin of error, for example, covers only the random wobble from asking some people rather than everyone.¹ Errors that push in one direction are the subject of the Random vs systematic error entry.

    Try it

    For two weeks, measure something that naturally varies, at the same time each day. It could be your resting heart rate, how long your commute takes, or how many steps you take. Mark each day's number on a simple chart.

    Find the biggest jump from one day to the next. Then draw a line through the middle of all the points. Which is bigger: the day-to-day jumps, or the change in your line over the two weeks? If the jumps are bigger, most of what you saw from day to day was noise.

    Related concepts

    Random vs systematic error · Error bars and confidence intervals · Placebo effect · Probabilistic reasoning · Accuracy vs precision · Sample size · Cherry-picking · Regression to the mean

    Sources

    1. Mercer, A. (2016, September 8). 5 key things to know about the margin of error in election polls. Pew Research Center. https://www.pewresearch.org/short-reads/2016/09/08/understanding-the-margin-of-error-in-election-polls/

    2. Barnett, A. G., van der Pols, J. C., & Dobson, A. J. (2005). Regression to the mean: What it is and how to deal with it. International Journal of Epidemiology, 34(1), 215–220. https://doi.org/10.1093/ije/dyh299

    3. Orsama, A.-L., Mattila, E., Ermes, M., van Gils, M., Wansink, B., & Korhonen, I. (2014). Weight rhythms: Weight increases during weekends and decreases during weekdays. Obesity Facts, 7(1), 36–47. https://doi.org/10.1159/000356147