On the map

At the centre Bet on a system that is maximally learned and minimally constrained, and add structure only where it improves the scaling laws. Holds DDDmitri Dolgov Read this korrent →

• Same subject: scaling laws A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. Holds NBNoam Brown
• Same subject: scaling laws A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from. Holds DPDylan Patel
• Same subject: scaling laws After pre-training, post-training and test-time scaling, the fourth scaling law is agentic: multiplying AI by spawning agents, and the whole loop scales on one thing, compute. Holds JHJensen Huang
• Same subject: scaling laws Current AI techniques are four to six orders of magnitude away from optimal in data efficiency and test-time compute efficiency. Holds FCFrançois Chollet
• Same subject: scaling laws Large language models could still plateau, and that possibility should be held open even though no evidence of it has appeared. Holds DHDavid Heinemeier Hansson
• Same subject: scaling laws Model comparisons understate real progress, because benchmark tables do not control for how much test-time compute each answer used. Holds NBNoam Brown
• Same subject: scaling laws Models resemble each other because pre-training is the same everywhere; what differentiates labs now is RL and post-training. Holds ISIlya Sutskever
• Same subject: scaling laws One pre-trained robot model now matches or beats the specialists that were fine-tuned with reinforcement learning for the very tasks they were built for. Holds CFChelsea Finn