korrents
Sebastian Raschka

Sebastian Raschka

@sebastian-raschka · 15 positions · 0 changes of mind

AI research engineer working on large language models. He writes the Ahead of AI newsletter and is the author of Build a Large Language Model (From Scratch).

Everything they publish, on ppll ↗

Sebastian Raschka did not write this page.

We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.

  1. Our reading

    Computer-use capability will be a major focus of LLM and coding-agent development in the coming months and years.

    LLMs

    Their wordsI expect the upcoming months (or years) also to be an era of computer use refinement on both the LLM and the agent harness layer.

    ↗GPT-6 Astra, Looped Transformers, and Hidden Reasoningmagazine.sebastianraschka.com 1st of 3 in this piece

  3. Our reading

    Frontier language models will remain reasoning models trained with RLVR-style chain-of-thought for the foreseeable future.

    LLMs

    Their wordsGPT-6 Astra (and likely any LLM in the foreseeable future) is still a reasoning model.

    ↗GPT-6 Astra, Looped Transformers, and Hidden Reasoningmagazine.sebastianraschka.com 2nd of 3 in this piece

  5. Our reading

    Looped/recurrent-depth transformer architectures can improve model quality at a fixed compute budget when the model is large enough.

    Their wordsSo, in short, we can say that using looped transformers can improve model quality at a fixed compute budget if the model is large enough.

    ↗GPT-6 Astra, Looped Transformers, and Hidden Reasoningmagazine.sebastianraschka.com 3rd of 3 in this piece

  6. 3 weeks earlier

  7. Our reading

    Coding something from scratch is one of the best ways to truly understand how it works.

    Their wordsSo this one will not be coding-related, but coding from scratch is actually a very, very useful technique because it really helps you understand how something is implemented.

    ↗How Claude Watermarks AI-Generated Textmagazine.sebastianraschka.com

  8. 7 days earlier

  9. Our reading

    AI text detectors and the LLMs they try to catch are locked in a permanent cat-and-mouse arms race.

    LLMs

    Their wordsAI checkers are essentially a cat-and-mouse game. AI checkers may learn to detect a certain pattern that is indicative of AI-generated content. Then, the next LLM may incidentally or deliberately not exhibit that pattern and avoid detection. The AI checker then has to be updated to detect said LLM, and so forth.

    ↗Building an AI Text Detector From Scratchmagazine.sebastianraschka.com 1st of 2 in this piece

  11. Our reading

    An AI-detector's score reflects the classifier's training distribution, not a true probability that a given text was written by AI.

    AI writing

    Their wordsthe score is the classifier's estimated probability for the AI-generated class based on its training distribution. However, we shouldn't interpreted it as a general probability that the text was written by AI.

    ↗Building an AI Text Detector From Scratchmagazine.sebastianraschka.com 2nd of 2 in this piece

  12. 4 weeks earlier

  13. Our reading

    The special reasoning-trace delimiter tokens used by reasoning models are cosmetic and not required for good reasoning performance.

    Their wordsThese <think> and </think> tags are cosmetic with respect to reasoning ability. They do not make the model reason, and they are not required to achieve good reasoning performance.

    ↗Controlling Reasoning Effort in LLMsmagazine.sebastianraschka.com 1st of 3 in this piece

  15. Our reading

    Pushing an LLM's reasoning budget higher eventually yields diminishing returns and becomes uneconomical.

    LLMs

    Their wordsThis saturation can be seen more clearly for the GPT 5.6 Sol model, which also shows that increasing reasoning budgets can become uneconomical at some point.

    ↗Controlling Reasoning Effort in LLMsmagazine.sebastianraschka.com 2nd of 3 in this piece

  17. Our reading

    Raising inference-time reasoning effort on a smaller model can match the performance of a larger model run at lower effort.

    Their wordsa smaller model at a higher reasoning effort can sometimes reach a similar score as a larger model at a lower reasoning effort

    ↗Controlling Reasoning Effort in LLMsmagazine.sebastianraschka.com 3rd of 3 in this piece

  18. 3 weeks earlier

  19. Our reading

    Proprietary AI services will become more restrictive over time, making open-weight alternatives valuable as a backup.

    Their wordsproprietary services may become more restrictive over time, and it's maybe a good idea to be comfortable with open-weight alternatives as a backup.

    ↗Using Local Coding Agentsmagazine.sebastianraschka.com 1st of 3 in this piece

  21. Our reading

    When an AI lab builds both a model and a coding harness, the model is optimized for that lab's own harness first.

    LLMs

    Their wordsif an LLM developer also develops a coding harness, it is somewhat safe to assume that their model is optimized for their own harness first

    ↗Using Local Coding Agentsmagazine.sebastianraschka.com 2nd of 3 in this piece

  23. Our reading

    In coding agents, untrusted repositories should be treated as hostile by default because they can steer the agent into risky actions via approved tools.

    coding agents

    Their wordsuntrusted repos should be treated as hostile by default because they can steer the agent toward reading files, running commands, or sending data through approved tools.

    ↗Using Local Coding Agentsmagazine.sebastianraschka.com 3rd of 3 in this piece

  24. 3 weeks earlier

  25. Our reading

    Long-context efficiency has become a central priority in LLM development because agent harnesses require working with increasingly long contexts.

    LLMs

    Their wordsIn 2026, long-context efficiency is king as more and more LLMs get plugged into agent harnesses (OpenClaw etc.), which requires working with longer and longer contexts.

    ↗LLM Research Papers: The 2026 List (January to May)magazine.sebastianraschka.com

  26. 3 weeks earlier

  27. Our reading

    As reasoning models and agents retain more tokens for longer, KV-cache size and attention cost become the main constraint on scaling context.

    LLMs

    Their wordsAs reasoning models and agent workflows keep more tokens around (for longer), KV-cache size, memory traffic, and attention cost quickly become the main constraints, and LLM developers are adding a growing number of architecture tricks to reduce those costs.

    ↗Recent Developments in LLM Architectures: KV Sharing, mHC, and Compressed Attentionmagazine.sebastianraschka.com 1st of 2 in this piece

  29. Our reading

    Larger language models already have enough capacity that extra per-layer embedding tricks add little benefit to them.

    LLMs

    Their wordsHowever, larger models already have sufficient capacity where these extra embeddings may not help that much.

    ↗Recent Developments in LLM Architectures: KV Sharing, mHC, and Compressed Attentionmagazine.sebastianraschka.com 2nd of 2 in this piece