metascience
Research about research: whether published findings hold up, and what makes a field more or less reliable.
FilterEveryone, all time69 from 35 people
- SA
Samuel Arbesman quoted
Complexity scientist and writer
Their wordsOf course, this doesn’t therefore mean that everything we know is going to be swept away and that our knowledge is forever entirely uncertain. We are asymptotically approaching the truth.
↗Epistemic Humility and the Pessimistic Meta-Induction of Sciencearbesman.substack.com 2nd of 2 in this piece
- 9 days earlier
- HF
Henry Farrell quoted
Professor of international affairs at Johns Hopkins SAIS and co-author
Their wordsthe value of standard social science approaches, which attempt to discover long term regularities, is likely to depreciate in relative terms until the system settles down again.
↗International relations has a problem with the futureprogrammablemutter.com 2nd of 3 in this piece
- DL
Daniel Lemire quoted
Computer science professor who works on fast data processing
Their wordsI predict that, over time, the focus will move away from "papers as the final output."
↗AI is breaking the academic sorting machinelemire.me 3rd of 3 in this piece
- 6 days earlier
- PB
Paul Bloom quoted
Psychologist at the University of Toronto and professor emeritus at Yale
Their wordsWhen it comes to the claim that smartphones are to blame, this skepticism should be reinforced by the failure to find robust evidence in its favor.
↗Kids These Days, Part IIsmallpotatoes.paulbloom.net 1st of 3 in this piece
- TC
Tyler Cowen quoted
Economist at George Mason University
Their wordsRCTs somehow have the odd disadvantage of requiring too many things to be held constant, and so they can miss the benefits of longer-term adjustments.
↗Does AI assistance enhance or erode expertise?marginalrevolution.com
- 10 days earlier
- AA
Awais Aftab quoted
Psychiatrist and clinical assistant professor at Case…
Their wordsThere is no consistency, no respect for evidence, no clear bar for when downstream implications matter and when they don't.
↗Postpartum Psychosis and Diagnostic Sclerosispsychiatrymargins.com 2nd of 3 in this piece
- AA
Awais Aftab quoted
Their wordsI do not believe that DSM deserves the respect and authority we bestow on it.
↗Postpartum Psychosis and Diagnostic Sclerosispsychiatrymargins.com 3rd of 3 in this piece
- NF
Nat Friedman quoted
Investor and engineer; CEO of GitHub from 2018 to 2021
Their wordsWe know less than we think The replication crisis is not an aberration Many of the things we believe are wrong We are often not even asking the right questions
- 3 weeks earlier
- RK
Rohit Krishnan quoted
Writes Strange Loop Canon
Their wordsIgnorance is not a temporary defect that we will eventually eliminate. It's the permanent burden of increasing specialisation and living in a complex world.
↗Nobody Knows Anythingstrangeloopcanon.com 2nd of 2 in this piece
- 5 months earlier
- TT
Terence Tao quoted
Their wordsSo, you you can't look at any given scientific achievement purely in isolation and give it an objective grade without being aware of the context both in the the past and the future. And so it it it may never be something that you can just reinforcement learn the same way that that you can for much sort of more localized problems.
↗Terence Tao – How the world’s top mathematician uses AIyoutube.com 3rd of 24 in this recording
- 4 months earlier
- GT
Gary Taubes quoted
Investigative science and health journalist
Their wordsI do not believe that a “range of studies, from a range of places, using a range of methods, consistently showing a certain result” necessarily constitutes a strong signal. I would assume the opposite.
↗Signs of Bad Scienceuncertaintyprinciples.substack.com 1st of 5 in this piece
- GT
Gary Taubes quoted
Their wordsA critical point is that the more important the science, the more likely researchers are to lower their standards for what constitutes a “strong enough” signal. It should be the opposite.
↗Signs of Bad Scienceuncertaintyprinciples.substack.com 4th of 5 in this piece
- GT
Gary Taubes quoted
Their wordsActing on weak evidence can feel justified in the moment, but it risks degrading the norms of scientific rigor. If ambiguous data are treated as sufficient to declare harm—or safety—the entire scientific enterprise absorbs the cost.
↗Signs of Bad Scienceuncertaintyprinciples.substack.com 5th of 5 in this piece
- 7 weeks earlier
- AH
Anton Howes quoted
Historian of innovation
Their wordsOver the past few decades, journal editors and peer-reviewers have increasingly insisted that papers must present large datasets that have been treated using complex statistical methods in order to make even the mildest claims about what caused what.
- 6 weeks earlier
- DH
Dave Hone quoted
Palaeontologist and reader in zoology at Queen Mary University of London
Their wordsAnd that's the thing, it's the pool of data, and I think that's what people miss. We as paleontologists get caught up on single superlative specimens and then try and treat them as a silver bullet almost.
↗Dave Hone: T-Rex, Dinosaurs, Extinction, Evolution, and Jurassic Park | Lex Fridman Podcast #480youtube.com 15th of 24 in this recording
- 11 months earlier
- EB
Ed Barnhart quoted
Archaeologist and director of the Maya Exploration Center
Their wordsAnd as responsible scientists, we’re trying to disprove our theories. We are not supposed to be trying to prove our theories. That’s one more foot out of the science box that archeology often steps.
↗Ed Barnhart: Maya, Aztec, Inca, and Lost Civilizations of South America | Lex Fridman Podcast #446youtube.com 12th of 25 in this recording
- 5 months earlier
- SC
Sean Carroll quoted
Theoretical physicist at Johns Hopkins University
Their wordsI think that Nobel Prize has enormous problems. I think it’s probably a net good for the world because it brings attention to good science. I think it’s probably a net negative for science because it makes people want to win the Nobel Prize.
↗Sean Carroll: General Relativity, Quantum Mechanics, Black Holes & Aliens | Lex Fridman Podcast #428youtube.com 20th of 20 in this recording
- 3 months earlier
- SR
Sebastian Ruder quoted
Research scientist at Meta working on multilingual models and evaluation
Their wordsAt its best, research is a collaborative and sometimes argumentative conversation.
- 5 months earlier
- AH
Anton Howes quoted
Their wordsBut I’ve become increasingly worried that science’s replication crises might pale in comparison to what happens all the time in history, which is not just a replication crisis but a reproducibility crisis.
↗Age of Invention: Does History have a Replication Crisis?ageofinvention.xyz 1st of 4 in this piece
- AH
Anton Howes quoted
Their wordsBut it’s hard to trust when you are exposed to one of the most frightening of revelations: that hardly anyone ever bothers to check.
↗Age of Invention: Does History have a Replication Crisis?ageofinvention.xyz 3rd of 4 in this piece
- AH
Anton Howes quoted
Their wordsJust as in the sciences it is considered good practice to make one’s data available, in history it should perhaps be a requirement to upload to some public repository the photographs or transcriptions of any cited archival sources that are not otherwise freely accessible online.
↗Age of Invention: Does History have a Replication Crisis?ageofinvention.xyz 4th of 4 in this piece
- 13 days earlier
- JC
Jason Crawford quoted
Founder of the Roots of Progress Institute
Their wordsEven in that case, though, what we should trust is not the particular output of the scientific process at any given time. It can make wrong turns. Instead, we should trust that it will find the truth eventually, and that it is our best and only method for doing so.
↗What does it mean to “trust science”?rootsofprogress.org 1st of 3 in this piece
- 4 months earlier
- AM
Adam Mastroianni quoted
Their wordsThere’s no point in picking some studies that are convenient to replicate, doing ‘em over, and reporting “only 36% of them replicate!”
↗Science is a strong-link problemexperimental-history.com 3rd of 3 in this piece
- 3 months earlier
- SR
Stuart Ritchie quoted
Psychologist, formerly a lecturer at King's College London
Their wordsWe know how easily these practices can distort the scientific record and give us false-positive, nonsense results.
↗Science isn't storytellingsciencefictions.org 2nd of 5 in this piece
- SR
Stuart Ritchie quoted
Their wordsHaving a pre-existing conclusion—a “take-home message”—in mind is one of the most reality-distorting influences on our thinking.
↗Science isn't storytellingsciencefictions.org 3rd of 5 in this piece
- SR
Stuart Ritchie quoted
Their wordsThe mere act of squeezing a complex process into a few thousand words—limited by a journal-imposed word count—is itself a distortion of reality.
↗Science isn't storytellingsciencefictions.org 5th of 5 in this piece
- 2 weeks earlier
- AM
Adam Mastroianni quoted
Their wordsAll we can say from these big trends is that we have no idea whether peer review helped, it might have hurt, it cost a ton, and the current state of the scientific literature is pretty abysmal.
↗The rise and fall of peer reviewexperimental-history.com 1st of 9 in this piece
- AM
Adam Mastroianni quoted
Their wordsScientists have run studies where they deliberately add errors to papers, send them out to reviewers, and simply count how many errors the reviewers catch. Reviewers are pretty awful at this.
↗The rise and fall of peer reviewexperimental-history.com 2nd of 9 in this piece
- AM
Adam Mastroianni quoted
Their wordsIn fact, we’ve got knock-down, real-world data that peer review doesn’t work: fraudulent papers get published all the time.
↗The rise and fall of peer reviewexperimental-history.com 3rd of 9 in this piece
- 2 months earlier
- SR
Stuart Ritchie quoted
Our readingA tiny, unreplicated study should not be publicised to millions of people as though it were settled.
Their wordsMore generally, you’ve got to wonder if Dweck considered whether the results of this very small, to my knowledge unreplicated, study were really ripe for publicising to literally millions of people
↗How growth mindset shranksciencefictions.org 1st of 3 in this piece