korrents
Sebastian Raschka

What Sebastian Raschka thinks about LLMs

@sebastian-raschka · 15 positions · 0 changes of mind

AI research engineer working on large language models. He writes the Ahead of AI newsletter and is the author of Build a Large Language Model (From Scratch).

Everything they publish, on ppll ↗

Sebastian Raschka did not write this page.

We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.

8 dated positions, 2026, in their own words. Our reading of what Sebastian Raschka has said — not written or endorsed by them.

  1. Our readingComputer-use capability will be a major focus of LLM and coding-agent development in the coming months and years.

    Their wordsI expect the upcoming months (or years) also to be an era of computer use refinement on both the LLM and the agent harness layer.

    ↗GPT-6 Astra, Looped Transformers, and Hidden Reasoningmagazine.sebastianraschka.com 1st of 3 in this piece

  3. Our readingFrontier language models will remain reasoning models trained with RLVR-style chain-of-thought for the foreseeable future.

    Their wordsGPT-6 Astra (and likely any LLM in the foreseeable future) is still a reasoning model.

    ↗GPT-6 Astra, Looped Transformers, and Hidden Reasoningmagazine.sebastianraschka.com 2nd of 3 in this piece

  4. 4 weeks earlier

  5. Our readingAI text detectors and the LLMs they try to catch are locked in a permanent cat-and-mouse arms race.

    Their wordsAI checkers are essentially a cat-and-mouse game. AI checkers may learn to detect a certain pattern that is indicative of AI-generated content. Then, the next LLM may incidentally or deliberately not exhibit that pattern and avoid detection. The AI checker then has to be updated to detect said LLM, and so forth.

    ↗Building an AI Text Detector From Scratchmagazine.sebastianraschka.com 1st of 2 in this piece

  6. 4 weeks earlier

  7. Our readingPushing an LLM's reasoning budget higher eventually yields diminishing returns and becomes uneconomical.

    Their wordsThis saturation can be seen more clearly for the GPT 5.6 Sol model, which also shows that increasing reasoning budgets can become uneconomical at some point.

    ↗Controlling Reasoning Effort in LLMsmagazine.sebastianraschka.com 2nd of 3 in this piece

  8. 3 weeks earlier

  9. Our readingWhen an AI lab builds both a model and a coding harness, the model is optimized for that lab's own harness first.

    Their wordsif an LLM developer also develops a coding harness, it is somewhat safe to assume that their model is optimized for their own harness first

    ↗Using Local Coding Agentsmagazine.sebastianraschka.com 2nd of 3 in this piece

  10. 3 weeks earlier

  11. Our readingLong-context efficiency has become a central priority in LLM development because agent harnesses require working with increasingly long contexts.

    Their wordsIn 2026, long-context efficiency is king as more and more LLMs get plugged into agent harnesses (OpenClaw etc.), which requires working with longer and longer contexts.

    ↗LLM Research Papers: The 2026 List (January to May)magazine.sebastianraschka.com

  12. 3 weeks earlier

  13. Our readingAs reasoning models and agents retain more tokens for longer, KV-cache size and attention cost become the main constraint on scaling context.

    Their wordsAs reasoning models and agent workflows keep more tokens around (for longer), KV-cache size, memory traffic, and attention cost quickly become the main constraints, and LLM developers are adding a growing number of architecture tricks to reduce those costs.

    ↗Recent Developments in LLM Architectures: KV Sharing, mHC, and Compressed Attentionmagazine.sebastianraschka.com 1st of 2 in this piece

  15. Our readingLarger language models already have enough capacity that extra per-layer embedding tricks add little benefit to them.

    Their wordsHowever, larger models already have sufficient capacity where these extra embeddings may not help that much.

    ↗Recent Developments in LLM Architectures: KV Sharing, mHC, and Compressed Attentionmagazine.sebastianraschka.com 2nd of 2 in this piece