Most growth slows as it matures; the question is whether the curve is flattening or something new is restocking it.
Michael NielsenDavid Heinemeier HanssonFrançois CholletGary MarcusIlya SutskeverJason CohenJosé Luis RicónSundar Pichai
Their wordsA given field or paradigm will yield a sigmoid-like pattern and eventually it will be exhausted.
Their wordsif somebody, you know, is standing behind the dessert table and is replenishing, restocking the desserts and keeps kind of you know, adding adding new ones in, it may turn out that you know, a little bit later a much better desserts appear.
Their wordsIf you're not changing it, if there's some kind of stagnation there, if you're not changing those external sort of circumstances, yes, like you may start to get sort of diminishing returns again. But that doesn't mean there's anything intrinsic about the situation.
Their wordsBut that's not what happens. What happens is it starts with an S-curve and then it starts sagging. Its butt starts sagging down. So, I wrote an article about this called the elephant curve, which is what I named it, right?
Their wordsI think I think there'll definitely be there'll be diminishing returns because you want you want people who think differently rather than the same. I think that if they were literal copies of me, I'm not sure how much more incremental value you'd get.
Their wordsBut this particular approach has limits that are clearer by the day.
Recalls holding
Their wordsSo we should also have the humility. As amazing as the LLMs are now, it could be that they eventually plateau. We haven't seen any evidence of it yet, and I think this is also why we're seeing this absolute gobsmacking levels of investment, because so far the scaling laws are true, and the more billions are poured in, the more intelligence comes out.
Their wordsSo I do think scaling laws are working, but it's tough to get, at any given time, the models we all use the most, this maybe a few months behind the maximum capability we can deliver because that won't be the fastest, easiest to use, et cetera.