korrents

On the map

Tap a claim on the ring to put it at the centre.

← Making context windows longer will not by itself scale to long-running multimodal tasks; new ideas are needed.

17 connected korrents · 17 moments from 22 Mar 2024 to 6 Nov 2026.

Everything filed under LLMs LLMs Everything filed under scaling laws scaling laws Everything filed under open source open source Everything filed under AI agents AI agents Everything filed under chat interfaces chat interfaces Everything filed under taste taste Everything filed under OpenAI OpenAI Same subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subject Read this korrent: Making context windows longer will not by itself scale to long-running multimodal tasks; new ideas are needed. Making context windows longer will not byitself scale to long-running multimodaltasks; new ideas are needed. Last stated 2 years ago 5 Feb 2025 AK Andrej Karpathy — holds since 2025-02-05 — tap for who they are Same subject: Every announcement of a longer context window hides the cost, which is that the model gets worse at following instructions. — tap to centre the map on it Every announcement of a longercontext window hides the cost, whichis that the model gets worse atfollowing instructions. Last stated 2 years ago 19 Jun 2024 AS Aravind Srinivas — holds since 2024-06-19 — tap for who they are Same subject: A large context window does not replace fine-tuning, because fine-tuning serves the separate purpose of instilling preferences and demonstrating a task. — tap to centre the map on it A large context window does notreplace fine-tuning, becausefine-tuning serves the separatepurpose of instilling preferencesand demonstrating a task. Last stated 3 years ago 22 Mar 2024 EG Elad Gil — holds since 2024-03-22 — tap for who they are Same subject: The scarce skill is now context switching across parallel agents, not sustained deep work. — tap to centre the map on it The scarce skill is now contextswitching across parallel agents,not sustained deep work. Last stated 9 months ago 4 Mar 2026 BC Boris Cherny — holds since 2026-03-04 — tap for who they are Same subject: A bigger context window does not give you a smarter model; the intelligence of the model is what decides how much of that window it can actually attend to. — tap to centre the map on it A bigger context window does notgive you a smarter model; theintelligence of the model is whatdecides how much of that window itcan actually attend to. Last stated 4 months ago 15 Jul 2026 DH Dex Horthy — holds since 2026-07-15 — tap for who they are Same subject: Today's text-box chat interfaces are too restricted to survive multimodal models, and will look archaic within a couple of years. — tap to centre the map on it Today's text-box chat interfaces aretoo restricted to survive multimodalmodels, and will look archaic withina couple of years. Last stated a year ago 23 Jul 2025 DH Demis Hassabis — holds since 2025-07-23 — tap for who they are Same subject: Where tabs and taskbars sit is possibly a matter of taste, but placement matters when doing agent work. — tap to centre the map on it Where tabs and taskbars sit ispossibly a matter of taste, butplacement matters when doing agentwork. Last stated 2 weeks ago 6 Nov 2026 SH Scott Hanselman — holds since 2026-11-06 — tap for who they are Same subject: Multimodal models are not a fundamental change, since audio and images can be tokenized and trained on the same way as text. — tap to centre the map on it Multimodal models are not afundamental change, since audio andimages can be tokenized and trainedon the same way as text. Last stated 2 years ago 5 Feb 2025 AK Andrej Karpathy — holds since 2025-02-05 — tap for who they are Same subject: Longer context is an engineering and inference problem, not a research problem — nothing prevents it from working. — tap to centre the map on it Longer context is an engineering andinference problem, not a researchproblem — nothing prevents it fromworking. Last stated 9 months ago 13 Feb 2026 DA Dario Amodei — holds since 2026-02-13 — tap for who they are Same subject: "Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next. — tap to centre the map on it "Scaling" was powerful because itwas one word: naming a researchdirection is what tells a wholefield what to do next. Last stated 12 months ago 25 Nov 2025 IS Ilya Sutskever — holds since 2025-11-25 — tap for who they are Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it A benchmark result should bereported under a stated budget, oras a curve against test-time compute— never as a single number. Last stated 5 months ago 26 Jun 2026 NB Noam Brown — holds since 2026-06-26 — tap for who they are Same subject: A CPU and memory crunch is coming. — tap to centre the map on it A CPU and memory crunch is coming. Last stated 4 weeks ago 26 Oct 2026 DM Dillon Mulroy — holds since 2026-10-26 — tap for who they are Same subject: Scaling the 2019-2023 recipe — same architecture, bigger model, more data — is not enough; further progress depends on new architectural ideas. — tap to centre the map on it Scaling the 2019-2023 recipe — samearchitecture, bigger model, moredata — is not enough; furtherprogress depends on newarchitectural ideas. Last stated 2 years ago 20 Dec 2024 FC François Chollet — holds since 2024-12-20 — tap for who they are GM Gary Marcus — holds since 2024-10-11 — tap for who they are Same subject: The ChatGPT moment for robotics already happened a couple of years ago; what remains is the unglamorous post-training work. — tap to centre the map on it The ChatGPT moment for roboticsalready happened a couple of yearsago; what remains is the unglamorouspost-training work. Last stated 4 months ago 26 Jul 2026 JH Jensen Huang — holds since 2026-07-26 — tap for who they are Same subject: A PHP vulnerability found with GPT-5.6 Sol will cause havoc this week. — tap to centre the map on it A PHP vulnerability found withGPT-5.6 Sol will cause havoc thisweek. Last stated a month ago 21 Oct 2026 PS Peter Steinberger — holds since 2026-10-21 — tap for who they are Same subject: A business making hundreds of millions from an open source project owes it money or contribution, and can be refused its trademarks if it gives neither. — tap to centre the map on it A business making hundreds ofmillions from an open source projectowes it money or contribution, andcan be refused its trademarks if itgives neither. Last stated 2 years ago 26 Sept 2024 MM Matt Mullenweg — holds since 2024-09-26 — tap for who they are Same subject: A codec does not really exist until everyone can decode it. — tap to centre the map on it A codec does not really exist untileveryone can decode it. Last stated 6 months ago 31 May 2026 JK Jean-Baptiste Kempf — holds since 2026-05-31 — tap for who they are Same subject: A company that will not accept a project’s terms is free to go and use a more permissive project instead. — tap to centre the map on it A company that will not accept aproject’s terms is free to go anduse a more permissive projectinstead. Last stated 2 years ago 26 Sept 2024 MM Matt Mullenweg — holds since 2024-09-26 — tap for who they are
same subject or similar wordinga cloud: claims about one subject, named for itbar: when it was last stated, on a scale from 2015 to today — full is todaya face: someone who holds the claim — tap it for who they are

At the centre Making context windows longer will not by itself scale to long-running multimodal tasks; new ideas are needed. Last stated 5 Feb 2025 · 2 years ago Holds Andrej Karpathy Read this korrent →