korrents

What Adam Mastroianni thinks about peer review

@adam-mastroianni · 30 positions · 0 changes of mind

Experimental psychologist with a PhD from Harvard, and the author of the Experimental History newsletter, where he writes about how research actually gets done and why he thinks the institutions around it are failing.

Everything they publish, on ppll ↗

Adam Mastroianni did not write this page.

We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.

7 dated positions, 2022, in their own words. Our reading of what Adam Mastroianni has said — not written or endorsed by them.

  1. Our readingSixty years of peer review produced no evidence that it made science better, and the literature it produced is in a bad state.

    Their wordsAll we can say from these big trends is that we have no idea whether peer review helped, it might have hurt, it cost a ton, and the current state of the scientific literature is pretty abysmal.

    ↗The rise and fall of peer reviewexperimental-history.com 1st of 8 in this piece

    Interesting

  3. Our readingPeer reviewers miss most of the serious errors in the papers they review.

    Their wordsScientists have run studies where they deliberately add errors to papers, send them out to reviewers, and simply count how many errors the reviewers catch. Reviewers are pretty awful at this.

    ↗The rise and fall of peer reviewexperimental-history.com 2nd of 8 in this piece

    Interesting

  5. Our readingFraudulent papers pass peer review routinely, which is direct evidence that peer review does not do the job it is supposed to do.

    Their wordsIn fact, we’ve got knock-down, real-world data that peer review doesn’t work: fraudulent papers get published all the time.

    ↗The rise and fall of peer reviewexperimental-history.com 3rd of 8 in this piece

    Interesting

  7. Our readingThe demands of peer reviewers push scientists into producing bad research.

    Their wordsThe invention of peer review may have even encouraged bad research.

    ↗The rise and fall of peer reviewexperimental-history.com 4th of 8 in this piece

    Interesting

  9. Our readingScientists say they take peer review seriously, but what they actually do shows they do not think it matters.

    Their wordsIf you look at what scientists actually do, it’s clear they don’t think peer review really matters.

    ↗The rise and fall of peer reviewexperimental-history.com 5th of 8 in this piece

    Interesting

  11. Our readingNeeding a reviewer's approval before an idea counts for anything makes scientists worse at thinking, and costs the field its most unusual ideas.

    Their wordsMaking peer review harsher would also exacerbate the worst problem of all: just knowing that your ideas won’t count for anything unless peer reviewers like them makes you worse at thinking.

    ↗The rise and fall of peer reviewexperimental-history.com 6th of 8 in this piece

    Interesting

  13. Our readingPeer review is one of the things genuinely wrong with academia, not a safeguard that mostly works.

    Their wordsPeer review doesn’t work and there’s probably no way to fix it.

    ↗The rise and fall of peer reviewexperimental-history.com 7th of 8 in this piece

    Interesting