korrents

Our readingWhat is a korrent?

Before training a base model with RL to emit think tokens, it is better to do some continued pre-training or supervised fine-tuning with those tokens present.

Drawn from what Sebastian Raschka said

What these subjects mean

scaling laws Whether more compute keeps buying more capability, and where the curve now bends -- pre-training, post-training, test-time.

reinforcement learning Training by reward: environments, verifiable tasks, value functions, and whether it works or merely beats what came before.

What Sebastian Raschka actually said

  1. Sebastian Raschka AI research engineer working on large language models
    but in practice, I mean, this would work, but it would be relatively, um, brittle because the model has not seen those during pre-training. So in an ideal case scenario, I would personally recommend taking the base model and do a bit more continued pre-training or supervised fine tuning with think tokens present.

Added to korrents 9 Nov 2026 · How quotes work · Something wrong? Tell us

Do you hold this korrent?Do you also believe this?

I hold this too

Related korrents

Shared subject first, then similar wording. They may agree or disagree.

See this claim on the map