On the map

At the centre Language models will not replace good ideas, only the low-level work, so what they threaten is the ability to start a creative career rather than to have one. Holds DHDan Houser Read this korrent →

• Similar wording Future language models will be smaller and loop on themselves with a retrieval system, which is what will finally let them cite their sources. Holds GHGeorge Hotz
• Similar wording Large language models are a bad way to do science, because they have been fed so much that nobody knows what they already knew. Holds RSRichard Sutton
• Similar wording Large language models start in exactly the wrong place, because they try to get by without a goal and therefore without any sense of better or worse. Holds RSRichard Sutton
• Similar wording Large language models could still plateau, and that possibility should be held open even though no evidence of it has appeared. Holds DHDavid Heinemeier Hansson
• Similar wording A language model can predict what a person would say but not what will happen, and only the second of those is a model of the world. Holds RSRichard Sutton
• Similar wording Language models have no culture — no written record they hand to one another — and that absence is one of the things holding them back. Holds AKAndrej Karpathy
• Similar wording Capability does not generalise for free: a model that will move mountains on an agentic task still tells the same bad joke it told five years ago. Holds AKAndrej Karpathy
• Similar wording Coding models are worst at exactly the thing an AI research explosion would need: code that has never been written before. Holds AKAndrej Karpathy