He has said every month for the past three years that there is no technical progress left to be made in LLMs and that there is no more room in the market for inference spending to grow.
Here[0] is a fun selection of excerpts from his July 2024 post "How Does OpenAI Survive?"[1]
"I see no signs that the transformer-based architecture can do significantly more than it currently does."
Claude Code (or Codex, or OpenCode, or Pi, or Amp — whatever) can do this out of the box without any skills or special tools. The most important thing for making results like this easier to achieve (in any harness) is using the best current models. Right now that's Opus 4.8 and GPT-5.5.
IRDC if the LLMs "understand" anything. They are being used here to produce outputs that are desirable. (Neglecting the real possibility that this "survey" is complete BS, as noted elsewhere.)
Judging from the fact that the Opus 4.5 inflection point was not really anticipated, and we still don’t really know what threshold was crossed that suddenly made agentic coding accessible to so many more people, I think it’s safe to say we don’t know what the thresholds will be until they’re crossed. The fact that we don’t know exactly what they’ll be isn’t a good reason to think there won’t be any more.
I think we have quite good reason to expect more. As I said, we already know (caveat with your level of irrational skepticism toward the overwhelming evidence) that the best existing models are better than the ones publicly available.
For what it's worth, at PyCon US this year I ran into a few people with access to Claude Mythos and they confirmed that it's notably better at writing code than public Claude Opus 4.7.
> caveat with your level of irrational skepticism toward the overwhelming evidence
If you can talk about my irrational skepticism (because I said that "we don't know the future", I suppose?), can I talk about your total lack of common sense?
Because the economy has been growing in the last decades does not mean that it will keep growing for the next decades. Because LLMs have been improving in the last few years does not mean that they will keep improving in the next few years. Maybe, maybe not, your guess is as good as mine. If you know the future, put your money where you mouth is and invest everything you own in LLM companies.
Your overwhelming evidence is about the past: it has been improving in the past.
I must have thought I wrote something that I didn't actually write in the previous comment — I can't figure out what "as I said" is supposed to be about.
In any case, maybe I was too subtle. I was talking about Mythos, a model that continues the trend, but which is not available to the public yet. The "overwhelming evidence" is the testimony of the people who have used it. The irrational skepticism was people who don't believe that testimony. In other words, we do know the future, because we know that model and others like it will come out soon.
Mythos is already here, you cannot use it for predictions just because you don't have access to it...
I just have an issue with all the people saying "I predicted this 10 years ago" (implying something like "you should listen to me, I make good predictions") while conveniently forgetting all the things they predicted wrongly, or the survivorship bias.
We don't know that AIs will continue improving at the pace they have, because we don't know the future. Some people will guess right, some won't. And those who guess right will be tempted to believe that they guessed right because they are more clever. All we can say is that it is possible that it improves, and it is possible that is stops improving.
It’s not deviating up and down. It’s deviating upward. It is necessarily going to wildly overstate the previous 12 months’ revenue while wildly understating the next 12 months’ revenue. There is no way to describe exponential growth in a single number that doesn’t do this. This is why adults with a brain look at the series.
Here[0] is a fun selection of excerpts from his July 2024 post "How Does OpenAI Survive?"[1]
"I see no signs that the transformer-based architecture can do significantly more than it currently does."
[0]: https://xcancel.com/pathsnotchosen/status/206360940100129633...
[1]: https://www.wheresyoured.at/to-serve-altman/
reply