On the map

At the centre Language models have no equivalent of sleep: nothing takes the day's context, thinks it over, and distils it back into the weights. Holds AKAndrej Karpathy Read this korrent →

• Similar wording Work on large language models is not progress toward artificial general intelligence, because the models do not understand what they read. No longer holds AGAlexey Guzey
• Similar wording Large language models are a bad way to do science, because they have been fed so much that nobody knows what they already knew. Holds RSRichard Sutton
• Similar wording Large language models could still plateau, and that possibility should be held open even though no evidence of it has appeared. Holds DHDavid Heinemeier Hansson
• Similar wording A language model can predict what a person would say but not what will happen, and only the second of those is a model of the world. Holds RSRichard Sutton
• Similar wording Language models have no culture — no written record they hand to one another — and that absence is one of the things holding them back. Holds AKAndrej Karpathy
• Similar wording Computer-use agents had to wait for language models: without pre-trained representations the reward is too sparse to ever learn from. Holds AKAndrej Karpathy
• Similar wording Large language models are bad at chess: pattern recognition is not the same capability as searching one position very deeply. Holds ACAlbert Cheng
• Similar wording Large language models start in exactly the wrong place, because they try to get by without a goal and therefore without any sense of better or worse. Holds RSRichard Sutton