Our readingWhat is a korrent?
At the level of loss functions, language-model pre-training is remarkably simple, though running it at scale is a complete engineering feat.
Drawn from what Grant Sanderson said
What these subjects mean
simplicity Less, but better: reduction as the way to make something good, and the argument about when it goes too far.
scaling laws Whether more compute keeps buying more capability, and where the curve now bends -- pre-training, post-training, test-time.