Fund people, not projects IV: Scientific egalitarianism and lotteries

José Luis Ricón · 20 Jan 2021 · nintil.com ↗

3 korrents from this piece

José Luis Ricón did not write this page.

Every claim below was made in this piece, quoted word for word and numbered in the order the piece makes them, so you can read it there rather than take our word for it. The sentence above each quote is our reading of the claim, not their wording. Each quote was checked against a stored copy of the page at build time; where the two differ, the quote is the fact.

Our reading
Pre-grant peer review predicts which research will have impact substantially better than chance does.

Their wordsI have reviewed before the effectiveness of peer review at figuring out "what's good" in the context of grant awards, finding that peer review as currently practiced does substantially better than chance at predicting future impact, especially for the most impactful of the papers; but at the same time is is far from perfect, leaving plenty of variance unexplained.

Interesting
Our reading
Reviewers can tell the obviously good from the obviously bad, but there is a messy middle where they cannot tell at all.

Their wordsThe way I see this is that reviewers can identify what's obviously not great and what is obviously great (Perhaps identifying the latter is easier than the former). But there is a "messy middle" where it is unclear what to fund.

Interesting
Our reading
The cost of grant peer review is dominated by the time researchers spend writing applications, not by the reviewing itself.

Their wordsThe cost of peer review is largely dominated by applying for grants, not peer reviewing.

Interesting