I'm a software engineer, completing 10 years of professional experience this year. I started my career as a web frontend engineer (it was easier for me to de...
I’d be curious why you think LLMs are dead ending? Is it that you think the Jepa models are likely to find success and win out or do you think the LLMs in generally are just hitting their peaks?
Your point on power usage is interesting, although I think that is mostly on training not usage correct?
I think they’re a dead-end mostly because of the exponential cost vs. performance. The decreasing returns are obvious, and the companies are trying to adapt by raising token prices, but that will not be enough with the current user numbers (or even double or triple, if we believe the analysts). I think that, at least with these large LLM companies, we’re actually beyond the point of economic equilibrium with this technology, at current energy and water prices.
And yes, training is more expensive than usage. That’s probably the reason why Anthropic suggested a pause in LLM development (training), supposedly because of the fear that AI could become Skynet, but really because they are getting an IPO soon and if people see their current balance numbers, the IPO would fail and the bubble would probably pop. Which really proves my point a little: the economics of these companies “improving their LLMs” (training) don’t make sense at current energy prices.
I’d be curious why you think LLMs are dead ending? Is it that you think the Jepa models are likely to find success and win out or do you think the LLMs in generally are just hitting their peaks?
Your point on power usage is interesting, although I think that is mostly on training not usage correct?
I think they’re a dead-end mostly because of the exponential cost vs. performance. The decreasing returns are obvious, and the companies are trying to adapt by raising token prices, but that will not be enough with the current user numbers (or even double or triple, if we believe the analysts). I think that, at least with these large LLM companies, we’re actually beyond the point of economic equilibrium with this technology, at current energy and water prices.
And yes, training is more expensive than usage. That’s probably the reason why Anthropic suggested a pause in LLM development (training), supposedly because of the fear that AI could become Skynet, but really because they are getting an IPO soon and if people see their current balance numbers, the IPO would fail and the bubble would probably pop. Which really proves my point a little: the economics of these companies “improving their LLMs” (training) don’t make sense at current energy prices.