The transformer was close to the last answer in architecture: nothing about the core design has changed since 2017.

transformers The architecture that dropped recurrence in favour of attention, and that almost every large model since is built on.

design How things should be made and judged: what makes a design good, who should do the designing, and what an interface is for.

neural networks The architecture almost all modern machine learning is built on, and what changes when it gets deeper.