measuring intelligence
What it would mean to measure intelligence in a machine, as opposed to measuring how well it does one task it was trained for.
FilterEveryone, all time39 from 26 people
- ZM
Zvi Mowshowitz quoted
Writes Don't Worry About the Vase
Their wordsHigh performers start out with an edge that motivates more work with increasing returns to scale, then work more and harder than everyone else, and end up acquiring a vast array of related skills as they climb the entire associated tech tree.
- 4 weeks earlier
- CD
Cory Doctorow quoted
Science fiction novelist, journalist and activist
Our readingBeing able to statistically predict what someone will say does not mean you understand them.
Their wordsthe fact that you can use a statistical lookup table to predict what words someone is going to say doesn't mean you understand them
↗Pluralistic: Textured (18 Sep 2026)pluralistic.net 2nd of 3 in this piece
- 3 days earlier
- PG
Paul Graham quoted
Co-founder of Y Combinator and Viaweb
Their wordsSomeone needs to define the unit, perhaps using a chain of increasingly hard problems, each pair of which can be solved by a single model.
- 3 days earlier
- ZM
Zvi Mowshowitz quoted
Their wordsThe jump from Sol to Astra is larger than the jump from Fable 5 to Fable 5.1. This is a big deal. Astra is the best model for what one would broadly call ‘ambitious projects,’ and likely has the highest raw intelligence factor of any model.
- 5 weeks earlier
- FC
François Chollet quoted
Creator of the Keras deep-learning library and the ARC-AGI benchmark
Our reading · no longer heldThe LLM line of research will reach a capability plateau.
Recalls holdingIn the era of base LLM scaling (2022-2024), I believed the LLM line of research would reach a capability plateau (as later seen with base LLMs). In late 2024, after the o3 test-time compute demo, I changed my views: the new models were showing genuine fluid intelligence, and with this new line of work, the LLM line of research could achieve unbounded capability scaling. "There will be no wall."
SaidIn the era of base LLM scaling (2022-2024), I believed the LLM line of research would reach a capability plateau (as later seen with base LLMs). In late 2024, after the o3 test-time compute demo, I changed my views: the new models were showing genuine fluid intelligence, and with this new line of work, the LLM line of research could achieve unbounded capability scaling. "There will be no wall."
- 6 weeks earlier
- SC
Sasha Chapin quoted
Writer and author of All the Wrong Moves
Their wordsThe planning part of your mind, the one that generates ambitions and hypothetical situations, is a small part of your overall intelligence.
↗Surrender as a non-stupid life strategysashachapin.substack.com 1st of 3 in this piece
- 4 weeks earlier
- RW
Robert Wachter quoted
Professor and chair of the Department of Medicine at UCSF…
Our readingHealthcare may be the industry where AI has the greatest positive impact on the average person.
Their wordsConversely, healthcare may be the industry where AI has the greatest positive impact on the average person - meaning that AI executives should focus more on talking about it, while talking less about insider brass rings like achieving artificial general intelligence (AGI).
↗Robert Wachterrobertwachter.substack.com 2nd of 3 in this piece
- 2 months earlier
- TT
Terence Tao quoted
Their wordsyou know, right now we're going through uh an in um an cognitive version of the Copernican revolution where we used to think that human intelligence is the center of the universe. And now we're actually seeing that there's there's very different types of intelligence um that that that are out there uh with very different strengths and weaknesses.
↗Terence Tao – How the world’s top mathematician uses AIyoutube.com 4th of 24 in this recording
- 5 weeks earlier
- DA
Dario Amodei quoted
Co-founder and CEO of Anthropic
Their wordsMy one little bit, the one little bit of of fundamental uncertainty even on long time scales is this thing about tasks that aren't verifiable. Like, planning a mission to Mars, like, uh you know, doing some fundamental scientific discovery like like CRISPR, like, you know, writing a writing a novel. Hard to hard to verify those tasks.
↗Dario Amodei — “We are near the end of the exponential”youtube.com 5th of 29 in this recording
- 3 months earlier
- IS
Ilya Sutskever quoted
Co-founder and chief scientist of Safe Superintelligence Inc.
Their wordsThe models are much more like the first student but even more because then we say okay so the model should be good at competitive programming so let's get every single competitive programming problem ever and then let's do some data augmentation so we have even more competitive programming problems
↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 2nd of 28 in this recording
- IS
Ilya Sutskever quoted
Their wordsThe thing which I think is the most fundamental is that these models somehow just generalize dramatically worse than people.
↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 8th of 28 in this recording
- 8 weeks earlier
- SF
Sadayuki Furuhashi quoted
Author of the open source projects MessagePack
Their wordsAIが解ける問題を考えて解けるような仕組みを考えないといけないところがAGIに達してない感あるなぁ。AIに「お前が解ける問題を述べよ」と言ったら出てくるようにして欲しい
- 7 days earlier
- RS
Richard Sutton quoted
Computer scientist who founded the field of reinforcement learning
Their wordsWhy are you trying to distinguish humans? Humans are animals. What we have in common is more interesting. What distinguishes us, we should be paying less attention to.
↗Richard Sutton – Father of RL thinks LLMs are a dead endyoutube.com 1st of 25 in this recording
- RS
Richard Sutton quoted
Their wordsFor me, having a goal is the essence of intelligence
↗Richard Sutton – Father of RL thinks LLMs are a dead endyoutube.com 5th of 25 in this recording
- RS
Richard Sutton quoted
Their wordsif we understood a squirrel I think we'd have a we'd be almost all the way there to understanding human intelligence. The the language part is just a a small veneer on the surface.
↗Richard Sutton – Father of RL thinks LLMs are a dead endyoutube.com 11th of 25 in this recording
- 8 days earlier
- TL
Tobias Lütke quoted
Co-founder and CEO of Shopify
Their wordsAnd you know what no sci-fi author dared to predict is that the Turing tests just passes by.
↗ACQ2: How to Live in Everyone Else's Future (with Shopify CEO Tobi Lütke)youtube.com 4th of 21 in this recording
- 7 weeks earlier
- AG
Alexey Guzey quoted
Writer and founder of New Science
Our reading · no longer heldWork on large language models is not progress toward artificial general intelligence, because the models do not understand what they read.
No longer holdsIt’s become very difficult for me to maintain the belief in the stupidity of ChatGPT when every time I laugh at it, it ends up ridiculing me 6 months later.
SaidFirst, I’m now convinced that ChatGPT understands what it reads.
- 5 days earlier
- JG
James Gleick quoted
Author and journalist, born in New York City in 1954
Their wordsThe Turing test is done. We’ve proven that we can be fooled.
- 6 weeks earlier
- SH
Samuel Hammond quoted
Senior economist at the Foundation for American Innovation
Our readingA true artificial general intelligence cannot exist without being recognized as a moral subject.
Their wordsIt follows that a true AGI with full, human-level autonomy is inconceivable in the Kantian sense without also gaining our recognition as a moral subject.
- 4 months earlier
- NL
Nathan Lambert quoted
Machine-learning researcher on open language models
Their wordsI think my personal definition of AGI is much simpler. I think language models are a form of AGI and all of this super powerful stuff is a next step that's great if we get these tools. But a language model has so much value in so many domains that it's a general intelligence to me.
↗DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters | Lex Fridman Podcast #459youtube.com 8th of 44 in this recording
- 6 weeks earlier
- FC
François Chollet quoted
Their wordsThis "memorize, fetch, apply" paradigm can achieve arbitrary levels of skills at arbitrary tasks given appropriate training data, but it cannot adapt to novelty or pick up new skills on the fly (which is to say that there is no fluid intelligence at play here.)
↗OpenAI o3 Breakthrough High Score on ARC-AGI-Pubarcprize.org
- 4 months earlier
- AG
Alexey Guzey quoted
Their wordsThis means that working on neural networks is NOT getting us closer to AGI, except indirectly.
- 4 months earlier
- SC
Sean Carroll quoted
Theoretical physicist at Johns Hopkins University
Their wordsAnd rather than trying to ask, how close is it to being a human-like intelligent, we should appreciate it for what its capabilities are, and that will both be more accurate and help us put it to work and protect us from the dangers better rather than always anthropomorphizing it.
↗Sean Carroll: General Relativity, Quantum Mechanics, Black Holes & Aliens | Lex Fridman Podcast #428youtube.com 12th of 20 in this recording
- 5 months earlier
- EM
Elon Musk quoted
Founded SpaceX in 2002
Their wordsIf an AI cannot figure out new physics, it’s clearly not equal to humans, nor has surpassed humans because humans have figured out new physics.
↗Elon Musk: War, AI, Aliens, Politics, Physics, Video Games, and Humanity | Lex Fridman Podcast #400youtube.com 11th of 23 in this recording
- 4 months earlier
- GH
George Hotz quoted
American programmer and hacker
Their wordsThe problem is if you go back to 1960 and you tell them that you have a machine that can play amazing chess, of course someone in 1960 will tell you that machine is intelligent. Someone in 2010 won’t. What’s changed? Today, we think that these machines that have language are intelligent, but I think in 20 years we’re going to be like, yeah, but can it reproduce?
↗George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God | Lex Fridman Podcast #387youtube.com 11th of 30 in this recording
- 2 years earlier
- GW
Gwern quoted
Pseudonymous essayist
Their wordshuman intelligence's g is real but is the overall 'pool' of neural resources which derives from overall body integrity because the number of neurons, their density, their myelination, resistance to damage and infection etc, is causally downstream of all body and developmental systems, creating a huge mutational target
↗May 2021 Gwern.net Newslettergwern.substack.com 3rd of 3 in this piece
- 11 months earlier
- MH
Max Hodak quoted
Co-founder and chief executive of Science
Our reading · no longer heldSkepticism about artificial general intelligence arriving is warranted
Recalls holdingOver the last few years I have gone from an AGI skeptic to being significantly worried about just how soon it will likely come to exist and how dramatically it will reshape the world.
SaidOver the last few years I have gone from an AGI skeptic to being significantly worried about just how soon it will likely come to exist and how dramatically it will reshape the world.
- 7 weeks earlier
- DA
Dario Amodei quoted
Their wordsBy contrast, humans can generally perform a new language task from only a few examples or from simple instructions - something which current NLP systems still largely struggle to do.
↗Language models are few-shot learners (with 30 co-authors)arxiv.org All 3 from this paper
- DA
Dario Amodei quoted
Their wordsUltimately, however, one-shot, or even sometimes zero-shot, seem like the fairest comparisons to human performance, and are important targets for future work.
↗Language models are few-shot learners (with 30 co-authors)arxiv.org All 3 from this paper
- 7 months earlier
- FC
François Chollet quoted
Their wordsTo make deliberate progress towards more intelligent and more human-like artificial systems, we need to be following an appropriate feedback signal: we need to be able to define and evaluate intelligence in a way that enables comparisons between two systems, as well as comparisons with humans.
↗On the Measure of Intelligencearxiv.org All 5 from this paper