korrents

peer review

How science checks itself, and the case that the checking mostly does not work.

What people on korrents have said about peer review, newest first — 15 positions from 7 people.

FilterEveryone, all time
Who Everyone AMAdam Mastroianni 7 JRJosé Luis Ricón 2 SRStuart Ritchie 2 AGAlexey Guzey 1 AHAnton Howes 1 CBChris Blattman 1 NFNat Friedman 1
When All time Past 1 yearPast 3 yearsPast 5 years
  1. NF

    Nat Friedman quoted

    Our readingThe replication crisis is not an aberration; many of the things we believe are wrong

    Their wordsWe know less than we think The replication crisis is not an aberration Many of the things we believe are wrong We are often not even asking the right questions

    ↗Nat Friedmannat.org

    Interesting
  2. 3 years earlier
  3. AH

    Anton Howes quoted

    Our readingHistory has a reproducibility problem worse than science’s replication crisis, because hardly anyone ever goes back to the sources to check what a historian claimed they say.

    Their wordsBut I’ve become increasingly worried that science’s replication crises might pale in comparison to what happens all the time in history, which is not just a replication crisis but a reproducibility crisis.

    ↗Age of Invention: Does History have a Replication Crisis?ageofinvention.xyz 1st of 4 in this piece

    Interesting
  4. 8 months earlier
  5. SR

    Stuart Ritchie quoted

    Our readingThe replication crisis is not an aberration; many of the things we believe are wrong

    Their wordsWe know how easily these practices can distort the scientific record and give us false-positive, nonsense results.

    ↗Science isn't storytellingsciencefictions.org 2nd of 5 in this piece

    Interesting
  6. 2 weeks earlier
  7. AM

    Adam Mastroianni quoted

    Our readingSixty years of peer review produced no evidence that it made science better, and the literature it produced is in a bad state.

    Their wordsAll we can say from these big trends is that we have no idea whether peer review helped, it might have hurt, it cost a ton, and the current state of the scientific literature is pretty abysmal.

    ↗The rise and fall of peer reviewexperimental-history.com 1st of 8 in this piece

    Interesting
  9. AM

    Adam Mastroianni quoted

    Our readingPeer reviewers miss most of the serious errors in the papers they review.

    Their wordsScientists have run studies where they deliberately add errors to papers, send them out to reviewers, and simply count how many errors the reviewers catch. Reviewers are pretty awful at this.

    ↗The rise and fall of peer reviewexperimental-history.com 2nd of 8 in this piece

    Interesting
  11. AM

    Adam Mastroianni quoted

    Our readingFraudulent papers pass peer review routinely, which is direct evidence that peer review does not do the job it is supposed to do.

    Their wordsIn fact, we’ve got knock-down, real-world data that peer review doesn’t work: fraudulent papers get published all the time.

    ↗The rise and fall of peer reviewexperimental-history.com 3rd of 8 in this piece

    Interesting
  13. AM

    Adam Mastroianni quoted

    Our readingThe demands of peer reviewers push scientists into producing bad research.

    Their wordsThe invention of peer review may have even encouraged bad research.

    ↗The rise and fall of peer reviewexperimental-history.com 4th of 8 in this piece

    Interesting
  15. AM

    Adam Mastroianni quoted

    Our readingScientists say they take peer review seriously, but what they actually do shows they do not think it matters.

    Their wordsIf you look at what scientists actually do, it’s clear they don’t think peer review really matters.

    ↗The rise and fall of peer reviewexperimental-history.com 5th of 8 in this piece

    Interesting
  17. AM

    Adam Mastroianni quoted

    Our readingNeeding a reviewer's approval before an idea counts for anything makes scientists worse at thinking, and costs the field its most unusual ideas.

    Their wordsMaking peer review harsher would also exacerbate the worst problem of all: just knowing that your ideas won’t count for anything unless peer reviewers like them makes you worse at thinking.

    ↗The rise and fall of peer reviewexperimental-history.com 6th of 8 in this piece

    Interesting
  19. AM

    Adam Mastroianni quoted

    Our readingPeer review is one of the things genuinely wrong with academia, not a safeguard that mostly works.

    Their wordsPeer review doesn’t work and there’s probably no way to fix it.

    ↗The rise and fall of peer reviewexperimental-history.com 7th of 8 in this piece

    Interesting
  20. 2 months earlier
  21. SR

    Stuart Ritchie quoted

    Our readingPublished effect sizes shrink as more studies are done, and that decline is what the replication crisis is made of.

    Their wordsThe decline effect is the idea that scientific findings get smaller over time. That is, initial findings tend to report effect sizes that then crumble away—sometimes to nothing—with subsequent research.

    ↗How growth mindset shranksciencefictions.org 2nd of 3 in this piece

    Interesting
  22. 21 months earlier
  23. JR

    José Luis Ricón quoted

    Our readingPre-grant peer review predicts which research will have impact substantially better than chance does.

    Their wordsI have reviewed before the effectiveness of peer review at figuring out "what's good" in the context of grant awards, finding that peer review as currently practiced does substantially better than chance at predicting future impact, especially for the most impactful of the papers; but at the same time is is far from perfect, leaving plenty of variance unexplained.

    ↗Fund people, not projects IV: Scientific egalitarianism and lotteriesnintil.com 1st of 3 in this piece

    Interesting
  25. JR

    José Luis Ricón quoted

    Our readingThe cost of grant peer review is dominated by the time researchers spend writing applications, not by the reviewing itself.

    Their wordsThe cost of peer review is largely dominated by applying for grants, not peer reviewing.

    ↗Fund people, not projects IV: Scientific egalitarianism and lotteriesnintil.com 3rd of 3 in this piece

    Interesting
  26. 17 months earlier
  27. AG

    Alexey Guzey quoted

    Our readingPeer review is one of the things genuinely wrong with academia, not a safeguard that mostly works.

    Their wordsSpecific parts of academia that seem to be problematic: rigid, punishing for deviation, career progression; peer review; need to constantly fundraise for professors.

    ↗How Life Sciences Actually Work: Findings of a Year-Long Investigationguzey.com 3rd of 4 in this piece

    Interesting
  28. 5 years earlier
  29. CB

    Chris Blattman quoted

    Our readingComparing randomised trials of cash against NGOs’ own claims for their programmes is not a fair comparison.

    Their wordsThe evidence they cite in favor of cash points to peer reviewed randomized trials. The evidence on better performing programs point to… NGO home pages.

    ↗Are cash transfers overrated?chrisblattman.com 5th of 5 in this piece

    Interesting