Skip to main content
    Path to Science

    Thinking Dictionary

    Peer review

    What it is

    Peer review is the step where other experts check a study before a scientific journal publishes it. These reviewers, the author's "peers," look for weaknesses in the methods, the analysis, and the conclusions. They send comments to the journal's editors, who decide whether to publish, ask for changes, or reject the paper.

    Peer review is a quality filter, not a guarantee. A good way to think of "peer-reviewed" is "checked once," not "proven true."

    Why it matters

    "Published in a peer-reviewed journal" is often used as a stamp of truth. It helps to know what peer review can and can't do.

    What it does. A large review of the research on peer review looked at 28 studies. It found that peer review seems to make papers easier to read and improves how studies are reported.¹

    What it misses. The same review found little evidence about whether peer review actually picks out valid research. Hiding the authors' names from reviewers had no clear effect.¹ In another study, more than 600 reviewers for the medical journal BMJ were sent papers with nine major errors deliberately planted in them. On average, they found only about three. A short training course helped only a little.²

    Reviewers also usually see the written paper, not the raw data. So peer review can't easily catch fraud.

    When the system fails

    • Fake peer review. Some journals claim to use peer review but barely do it. A journalist at the journal Science sent a deliberately flawed fake paper to many journals that publish online and charge authors a fee. Many accepted it with little or no real scrutiny.³
    • Errors that slip through. In 1998, The Lancet published a paper by Andrew Wakefield and colleagues claiming a link between the MMR vaccine and autism. The journal formally retracted it in 2010.⁴ A retraction means the journal withdraws a paper and says it should no longer be relied on.

    Science doesn't stop at peer review

    Peer review is the first check, not the last. After publication, other scientists read the paper, try to repeat the study, and point out problems. Repeating a study to check its result is called replication. Corrections and retractions are part of how science fixes its mistakes. The Iteration entry covers this.

    So peer review isn't useless. It keeps out a lot of weak work and improves many papers. It just isn't enough on its own.

    And the opposite is also true: not everything outside a peer-reviewed journal is worthless. Preprints, which are early versions of papers shared before peer review, and reports from respected organisations can be valuable. Judge each source on its merits.

    In everyday life

    • "A new study shows…" Is it a peer-reviewed paper, a preprint, or just a press release? They carry different weight.
    • "It's in a journal." Is it a well-established journal? A quick lateral-reading search on the journal's name can tell you a lot.
    • "The study was retracted." Search the paper's title together with "correction" or "retraction" before you share it.

    The common trap

    The first trap is thinking "peer-reviewed means true." Reviewers miss many errors, and some journals fake the process.²˒³

    The second trap is thinking "peer review is useless, so I'll trust my own judgement instead." Peer review is imperfect, but it is still a filter. Science adds more checks on top.

    Some ways out:

    • Treat peer review as one sign of quality, not proof.
    • Check the journal's reputation.
    • Look for other studies, especially independent ones, that point the same way.

    Try it

    Find a recent "study shows" headline. Trace it back to the original source. Is it a peer-reviewed journal article, a preprint, or a press release?

    If it's a journal article, look up the journal. Is it well established? Does it have a good track record? Then search the paper's title with the words "correction" or "retraction." Did anything turn up?

    Related concepts

    Credible sources · Iteration · Lateral reading · Misinformation vs disinformation · Multiple lines of evidence · Cherry-picking

    Sources

    1. Jefferson, T., Rudin, M., Brodney Folse, S., & Davidoff, F. (2007). Editorial peer review for improving the quality of reports of biomedical studies. Cochrane Database of Systematic Reviews, MR000016. https://doi.org/10.1002/14651858.MR000016.pub3

    2. Schroter, S., Black, N., Evans, S., Godlee, F., Osorio, L., & Smith, R. (2008). What errors do peer reviewers detect, and does training improve their ability to detect them? Journal of the Royal Society of Medicine, 101(10), 507–514. https://doi.org/10.1258/jrsm.2008.080062

    3. Bohannon, J. (2013). Who's afraid of peer review? Science, 342(6154), 60–65. https://doi.org/10.1126/science.342.6154.60

    4. The Editors of The Lancet. (2010). Retraction—Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. The Lancet, 375(9713), 445. https://doi.org/10.1016/S0140-6736(10)60175-4