Recent Developments in LLM Architectures: KV Sharing, mHC, and Compressed Attention

Sebastian Raschka · 16 May 2026 · magazine.sebastianraschka.com ↗

2 korrents from this piece

Sebastian Raschka did not write this page.

Every claim below was made in this piece, quoted word for word and numbered in the order the piece makes them, so you can read it there rather than take our word for it. The sentence above each quote is our reading of the claim, not their wording. Each quote was checked against a stored copy of the page at build time; where the two differ, the quote is the fact.

Our reading
As reasoning models and agents retain more tokens for longer, KV-cache size and attention cost become the main constraint on scaling context.

Their wordsAs reasoning models and agent workflows keep more tokens around (for longer), KV-cache size, memory traffic, and attention cost quickly become the main constraints, and LLM developers are adding a growing number of architecture tricks to reduce those costs.
Our reading
Larger language models already have enough capacity that extra per-layer embedding tricks add little benefit to them.

Their wordsHowever, larger models already have sufficient capacity where these extra embeddings may not help that much.