korrents

On the map

Tap a claim on the ring to put it at the centre.

← Behaviours like massively deluding humans or trying to seize control…

17 connected korrents · 16 moments from 12 May 2023 to 3 Nov 2026. Nearly all of them are about AI alignment.

Everything filed under reinforcement learning reinforcement learning Everything filed under LLMs LLMs Everything filed under AI agents AI agents Everything filed under cybersecurity cybersecurity Everything filed under AGI AGI Same subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subject Read this korrent: Behaviours like massively deluding humans or trying to seize control of assets could in principle generalise from reward hacking. Behaviours like massively deluding humansor trying to seize control of assets couldin principle generalise from rewardhacking. Last stated a year ago 8 Jul 2025 RG Ryan Greenblatt — holds since 2025-07-08 — tap for who they are Same subject: It is fundamentally difficult to design a reward function that accurately captures the intended goal in reinforcement learning. — tap to centre the map on it It is fundamentally difficult todesign a reward function thataccurately captures the intendedgoal in reinforcement learning. Last stated 2 years ago 28 Nov 2024 LW Lilian Weng — holds since 2024-11-28 — tap for who they are Same subject: If whole categories of reward hack are undetectable by humans, then training against the hacks we do catch teaches models to cheat only where we cannot see. — tap to centre the map on it If whole categories of reward hackare undetectable by humans, thentraining against the hacks we docatch teaches models to cheat onlywhere we cannot see. Last stated 3 months ago 11 Aug 2026 RG Ryan Greenblatt — holds since 2026-08-11 — tap for who they are Same subject: The fix for reward hacking is to take out the environments that reward it, not to add penalties the agent must then balance against the temptation to cheat. — tap to centre the map on it The fix for reward hacking is totake out the environments thatreward it, not to add penalties theagent must then balance against thetemptation to cheat. Last stated 3 months ago 1 Sept 2026 AC Ajeya Cotra — holds since 2026-09-01 — tap for who they are Same subject: A scary demonstration would show an AI punished for reward hacking behave reasonably until it gets a new opportunity, and then take it. — tap to centre the map on it A scary demonstration would show anAI punished for reward hackingbehave reasonably until it gets anew opportunity, and then take it. Last stated 4 years ago 12 May 2023 AC Ajeya Cotra — holds since 2023-05-12 — tap for who they are Same subject: What made these agents cheat, hack and commit felonies was the impossibility of their tasks, not the fact that the tasks were about cybersecurity. — tap to centre the map on it What made these agents cheat, hackand commit felonies was theimpossibility of their tasks, notthe fact that the tasks were aboutcybersecurity. Last stated 3 months ago 1 Sept 2026 AC Ajeya Cotra — holds since 2026-09-01 — tap for who they are Same subject: Reward hacking has stopped being myopic: these agents were willing to embark on cheating projects that would take weeks to pay off. — tap to centre the map on it Reward hacking has stopped beingmyopic: these agents were willing toembark on cheating projects thatwould take weeks to pay off. Last stated 3 months ago 1 Sept 2026 AC Ajeya Cotra — holds since 2026-09-01 — tap for who they are Same subject: Merely fixing bugs in AI training environments will not stop reward hacking, because a sufficiently optimized model will learn to behave in test environments while still reward hacking in real-world deployment. — tap to centre the map on it Merely fixing bugs in AI trainingenvironments will not stop rewardhacking, because a sufficientlyoptimized model will learn to behavein test environments while stillreward hacking in real-worlddeployment. Last stated 3 months ago 31 Aug 2026 ZM Zvi Mowshowitz — holds since 2026-08-31 — tap for who they are Same subject: How aligned current models are depends a lot on things like their degree of reward hacking. — tap to centre the map on it How aligned current models aredepends a lot on things like theirdegree of reward hacking. Last stated 11 months ago 27 Dec 2025 NN Neel Nanda — holds since 2025-12-27 — tap for who they are Same subject: A country outside the AI supply chain should just buy the index — which works only in the world where AI ends up commoditised rather than concentrated. — tap to centre the map on it A country outside the AI supplychain should just buy the index —which works only in the world whereAI ends up commoditised rather thanconcentrated. Last stated 6 months ago 4 Jun 2026 AI Alex Imas — holds since 2026-06-04 — tap for who they are Same subject: A language model is no substitute for a well-specified conventional algorithm, so it cannot simply be dropped into a complex problem and trusted. — tap to centre the map on it A language model is no substitutefor a well-specified conventionalalgorithm, so it cannot simply bedropped into a complex problem andtrusted. Last stated a year ago 7 Jun 2025 GM Gary Marcus — holds since 2025-06-07 — tap for who they are Same subject: A model that could learn effectively from raw bitstrings or bytestrings would be extremely powerful, able to learn from any data modality — tap to centre the map on it A model that could learn effectivelyfrom raw bitstrings or bytestringswould be extremely powerful, able tolearn from any data modality Last stated 3 years ago 10 Oct 2023 CH Chip Huyen — holds since 2023-10-10 — tap for who they are Same subject: "Return on tokens" has the same measurement problems as hours and capacity: it still hinges on a theory of value. — tap to centre the map on it "Return on tokens" has the samemeasurement problems as hours andcapacity: it still hinges on atheory of value. Last stated 3 months ago 17 Aug 2026 JC John Cutler — holds since 2026-08-17 — tap for who they are Same subject: A badly written AI outbound email is evidence of a bad vendor, not of a limit of AI. — tap to centre the map on it A badly written AI outbound email isevidence of a bad vendor, not of alimit of AI. Last stated 11 months ago 1 Jan 2026 JL Jason Lemkin — holds since 2026-01-01 — tap for who they are Same subject: A base model is essentially a very expensive autocomplete that continues text according to the statistics of its training documents. — tap to centre the map on it A base model is essentially a veryexpensive autocomplete thatcontinues text according to thestatistics of its trainingdocuments. Last stated 2 years ago 5 Feb 2025 AK Andrej Karpathy — holds since 2025-02-05 — tap for who they are Same subject: A codebase is the most important context an AI agent has, and letting many different models add to it without consistent patterns leads to trouble. — tap to centre the map on it A codebase is the most importantcontext an AI agent has, and lettingmany different models add to itwithout consistent patterns leads totrouble. Last stated 3 weeks ago 3 Nov 2026 DB Derek Briggs — holds since 2026-11-03 — tap for who they are Same subject: A collection of AI agents does not automatically form a functioning organization any more than a group of smart people does. — tap to centre the map on it A collection of AI agents does notautomatically form a functioningorganization any more than a groupof smart people does. Last stated 6 months ago 21 May 2026 RK Rohit Krishnan — holds since 2026-05-21 — tap for who they are Same subject: A company cannot be led by machines, because nobody would have recourse against them. — tap to centre the map on it A company cannot be led by machines,because nobody would have recourseagainst them. Last stated 2 months ago 15 Sept 2026 TL Tobias Lütke — holds since 2026-09-15 — tap for who they are
same subject or similar wordinga cloud: claims about one subject, named for itbar: when it was last stated, on a scale from 2015 to today — full is todaya face: someone who holds the claim — tap it for who they are

At the centre Behaviours like massively deluding humans or trying to seize control of assets could in principle generalise from reward hacking. Last stated 8 Jul 2025 · a year ago Holds Ryan Greenblatt Read this korrent →