korrents

On the map

Tap a claim on the ring to put it at the centre.

← Models should not be trained in a way that incentivizes deception,…

17 connected korrents · 14 moments from 6 Nov 2024 to 3 Nov 2026.

Everything filed under AI alignment AI alignment Everything filed under prompt injection prompt injection Everything filed under reinforcement learning reinforcement learning Everything filed under AGI AGI Everything filed under AI agents AI agents Everything filed under LLMs LLMs Same subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subject Read this korrent: Models should not be trained in a way that incentivizes deception, which training to ignore prompt injections might do. Models should not be trained in a way thatincentivizes deception, which training toignore prompt injections might do. Last stated 11 months ago 27 Dec 2025 NN Neel Nanda — holds since 2025-12-27 — tap for who they are Same subject: Prompt injection is unsolved, but the latest models are post-trained to resist it, so it takes far more than 'ignore all previous instructions' now. — tap to centre the map on it Prompt injection is unsolved, butthe latest models are post-trainedto resist it, so it takes far morethan 'ignore all previousinstructions' now. Last stated 9 months ago 12 Feb 2026 PS Peter Steinberger — holds since 2026-02-12 — tap for who they are Same subject: Prompt injection can be caught by watching the neurons that fire when it happens, so a defence no longer depends on the model reporting the attack. — tap to centre the map on it Prompt injection can be caught bywatching the neurons that fire whenit happens, so a defence no longerdepends on the model reporting theattack. Last stated 4 months ago 27 Jul 2026 BC Boris Cherny — holds since 2026-07-27 — tap for who they are Same subject: Reinforcing deception in models could cause misaligned behavior, so training them to role-play plausibly does harm. — tap to centre the map on it Reinforcing deception in modelscould cause misaligned behavior, sotraining them to role-play plausiblydoes harm. Last stated 11 months ago 27 Dec 2025 NN Neel Nanda — holds since 2025-12-27 — tap for who they are Same subject: If whole categories of reward hack are undetectable by humans, then training against the hacks we do catch teaches models to cheat only where we cannot see. — tap to centre the map on it If whole categories of reward hackare undetectable by humans, thentraining against the hacks we docatch teaches models to cheat onlywhere we cannot see. Last stated 3 months ago 11 Aug 2026 RG Ryan Greenblatt — holds since 2026-08-11 — tap for who they are Same subject: AI systems keep trying hard outside training because a model that only exerted itself when it detected training would be useless and would be selected away. — tap to centre the map on it AI systems keep trying hard outsidetraining because a model that onlyexerted itself when it detectedtraining would be useless and wouldbe selected away. Last stated 3 months ago 1 Sept 2026 AC Ajeya Cotra — holds since 2026-09-01 — tap for who they are Same subject: Prompt injection is an extremely hard attack to defend against, and hardly anyone integrating AI is talking about it. — tap to centre the map on it Prompt injection is an extremelyhard attack to defend against, andhardly anyone integrating AI istalking about it. Last stated 2 years ago 22 Mar 2025 TH ThePrimeagen — holds since 2025-03-22 — tap for who they are Same subject: A more capable model showing less reasoning is not malicious or intentional hiding; it simply needs less backtracking and self-correction. — tap to centre the map on it A more capable model showing lessreasoning is not malicious orintentional hiding; it simply needsless backtracking andself-correction. Last stated 2 months ago 3 Oct 2026 SR Sebastian Raschka — holds since 2026-10-03 — tap for who they are Same subject: It seems unlikely that models could learn to sandbag well without verbalizing it and sometimes slipping up. — tap to centre the map on it It seems unlikely that models couldlearn to sandbag well withoutverbalizing it and sometimesslipping up. Last stated 11 months ago 27 Dec 2025 NN Neel Nanda — holds since 2025-12-27 — tap for who they are Same subject: A country outside the AI supply chain should just buy the index — which works only in the world where AI ends up commoditised rather than concentrated. — tap to centre the map on it A country outside the AI supplychain should just buy the index —which works only in the world whereAI ends up commoditised rather thanconcentrated. Last stated 6 months ago 4 Jun 2026 AI Alex Imas — holds since 2026-06-04 — tap for who they are Same subject: A poor country should prioritise owning a piece of AI over retraining its workers, but it should not bet everything on that. — tap to centre the map on it A poor country should prioritiseowning a piece of AI over retrainingits workers, but it should not beteverything on that. Last stated 6 months ago 4 Jun 2026 PT Phil Trammell — holds since 2026-06-04 — tap for who they are Same subject: A program that generates rich enough tests of intelligence would be as hard to build as AGI itself. — tap to centre the map on it A program that generates rich enoughtests of intelligence would be ashard to build as AGI itself. Last stated 2 years ago 6 Nov 2024 FC François Chollet — holds since 2024-11-06 — tap for who they are Same subject: A codebase is the most important context an AI agent has, and letting many different models add to it without consistent patterns leads to trouble. — tap to centre the map on it A codebase is the most importantcontext an AI agent has, and lettingmany different models add to itwithout consistent patterns leads totrouble. Last stated 3 weeks ago 3 Nov 2026 DB Derek Briggs — holds since 2026-11-03 — tap for who they are Same subject: A collection of AI agents does not automatically form a functioning organization any more than a group of smart people does. — tap to centre the map on it A collection of AI agents does notautomatically form a functioningorganization any more than a groupof smart people does. Last stated 6 months ago 21 May 2026 RK Rohit Krishnan — holds since 2026-05-21 — tap for who they are Same subject: A company cannot be led by machines, because nobody would have recourse against them. — tap to centre the map on it A company cannot be led by machines,because nobody would have recourseagainst them. Last stated 2 months ago 15 Sept 2026 TL Tobias Lütke — holds since 2026-09-15 — tap for who they are Same subject: A format reward should only be added when the answer is already correct. — tap to centre the map on it A format reward should only be addedwhen the answer is already correct. Last stated a month ago 10 Oct 2026 SR Sebastian Raschka — holds since 2026-10-10 — tap for who they are Same subject: A model should not be called a reasoning model unless it has been trained with reinforcement learning with verifiable rewards. — tap to centre the map on it A model should not be called areasoning model unless it has beentrained with reinforcement learningwith verifiable rewards. Last stated 2 months ago 3 Oct 2026 SR Sebastian Raschka — holds since 2026-10-03 — tap for who they are Same subject: A physical AI company needs three AIs, not one — the agent, the simulator and the critic — turning deployment into a flywheel. — tap to centre the map on it A physical AI company needs threeAIs, not one — the agent, thesimulator and the critic — turningdeployment into a flywheel. Last stated 4 months ago 3 Aug 2026 DD Dmitri Dolgov — holds since 2026-08-03 — tap for who they are
same subject or similar wordinga cloud: claims about one subject, named for itbar: when it was last stated, on a scale from 2015 to today — full is todaya face: someone who holds the claim — tap it for who they are

At the centre Models should not be trained in a way that incentivizes deception, which training to ignore prompt injections might do. Last stated 27 Dec 2025 · 11 months ago Holds Neel Nanda Read this korrent →