In my regular research (behind a paywall), I have been saying for a while that I think the future of AI is not large language models (LLM), but small language models (SLM) run on local desktop computers or even mobile phones.
I think this all hinges on whether or not progress will slow down. For example I’ve heard of some companies etching AI models onto silicon, but nobody will buy those if the model is obsolete in a year. But if the models stop improving, then these chips might be worth the investment since they will be way more efficient for inference.
So that’s one of the biggest questions in AI right now. Are we going to hit a wall? It does kind of seem like the big models aren’t improving as much anymore, and the small models are catching up. But at the same time, Moore’s law has been going for way longer than people expected, maybe AI will be the same.
I expect doing ASICs for models will work even if they keep improving. It’ll be like regular chips getting new versions. You buy a chip with a specific model etched into it, and if it does what you need great. Next year, a new version comes out. So, it’s actually a feature since it allows companies to keep selling new chips.
It does look like we are entering diminishing returns territory though. The biggest evidence for this is that Chinese companies have now basically caught up to Anthropic and OpenAI. If the progress at the frontier was still happening at the same rate, then the gap wouldn’t be closing so quickly. There’s also a lot less noticeable difference between stuff like Claude 4.6 and Claude 5. When they went from 3.x to 4.x it was very noticeable. And at least for agentic coding, most of the improvement seems to come from the harness now. I expect improvements will continue, but at a much more gradual pace. It’s also possible people will figure out a new architecture that’s superior to LLMs, or works with them. World models are one promising area already being explored.
I think this all hinges on whether or not progress will slow down. For example I’ve heard of some companies etching AI models onto silicon, but nobody will buy those if the model is obsolete in a year. But if the models stop improving, then these chips might be worth the investment since they will be way more efficient for inference.
So that’s one of the biggest questions in AI right now. Are we going to hit a wall? It does kind of seem like the big models aren’t improving as much anymore, and the small models are catching up. But at the same time, Moore’s law has been going for way longer than people expected, maybe AI will be the same.
Edit: wording on last sentence of first paragraph
I expect doing ASICs for models will work even if they keep improving. It’ll be like regular chips getting new versions. You buy a chip with a specific model etched into it, and if it does what you need great. Next year, a new version comes out. So, it’s actually a feature since it allows companies to keep selling new chips.
It does look like we are entering diminishing returns territory though. The biggest evidence for this is that Chinese companies have now basically caught up to Anthropic and OpenAI. If the progress at the frontier was still happening at the same rate, then the gap wouldn’t be closing so quickly. There’s also a lot less noticeable difference between stuff like Claude 4.6 and Claude 5. When they went from 3.x to 4.x it was very noticeable. And at least for agentic coding, most of the improvement seems to come from the harness now. I expect improvements will continue, but at a much more gradual pace. It’s also possible people will figure out a new architecture that’s superior to LLMs, or works with them. World models are one promising area already being explored.