Our readingWhat is a korrent?
Linear probes are very strong classifiers, and attention-head probes are even stronger.
Drawn from what Neel Nanda said
What these subjects mean
transformers The architecture that dropped recurrence in favour of attention, and that almost every large model since is built on.
neural networks The architecture almost all modern machine learning is built on, and what changes when it gets deeper.