On the map

At the centre Value functions only make reinforcement learning faster: anything you can do with one you can also do without it, just more slowly. Holds ISIlya Sutskever Read this korrent →

• Similar wording Learning from a reward ten years away is a solved problem: a value function trained by temporal-difference learning rewards the steps along the way. Holds RSRichard Sutton
• Similar wording The goal was never good simulation; it was answering counterfactuals, and a value function does that job as well as a simulator does. Holds SLSergey Levine
• Similar wording RLHF is not a tax on capability: preference tuning also raises maths and code scores, which is why the labs keep reaching for it. Holds NLNathan Lambert
• Similar wording Humans barely use reinforcement learning for intelligence — what RL they do use goes into motor tasks, not problem solving. Holds AKAndrej Karpathy
• Similar wording The human value function is extraordinarily robust — addiction aside — which is why a person can judge their own performance at a skill they have only just started. Holds ISIlya Sutskever
• Similar wording Reinforcement learning cannot be scaled for robots the way it was for language models, because every attempt spends real robot hours instead of data-centre compute. Holds CFChelsea Finn
• Similar wording The value-versus-growth distinction does not serve investors well in a fast-changing world. Holds HMHoward Marks
• Similar wording Even the most advanced AI cannot be followed blindly in investing, where value added is zero-sum and what is widely known is therefore worth little. Holds RDRay Dalio