8 korrents from this piece
Every claim below was made in this piece, quoted word for word and numbered in the order the piece makes them, so you can read it there rather than take our word for it. The sentence above each quote is our reading of the claim, not their wording. Each quote was checked against a stored copy of the page at build time; where the two differ, the quote is the fact.
Their wordsAll we can say from these big trends is that we have no idea whether peer review helped, it might have hurt, it cost a ton, and the current state of the scientific literature is pretty abysmal.
Their wordsScientists have run studies where they deliberately add errors to papers, send them out to reviewers, and simply count how many errors the reviewers catch. Reviewers are pretty awful at this.
Their wordsIn fact, we’ve got knock-down, real-world data that peer review doesn’t work: fraudulent papers get published all the time.
Their wordsThe invention of peer review may have even encouraged bad research.
Their wordsIf you look at what scientists actually do, it’s clear they don’t think peer review really matters.
Their wordsMaking peer review harsher would also exacerbate the worst problem of all: just knowing that your ideas won’t count for anything unless peer reviewers like them makes you worse at thinking.
Our reading
Peer review is one of the things genuinely wrong with academia, not a safeguard that mostly works.
Their wordsPeer review doesn’t work and there’s probably no way to fix it.
Their wordsThis extremely bad system is worse than nothing because it fools people into thinking they’re safe when they’re not.