" …
What exactly is AMI building? The short answer is world models, a category of AI system that LeCun has been arguing for, and working on, for years. The longer answer requires understanding why he thinks the industry has taken a wrong turn.
Large language models learn by predicting which word comes next in a sequence. They have been trained on vast quantities of human-generated text, and the results have been remarkable, ChatGPT, Claude, and Gemini have demonstrated an ability to generate fluent, plausible language across an enormous range of subjects. But LeCun has spent years arguing, loudly and repeatedly, that this approach has fundamental limits.
His alternative is JEPA: the Joint Embedding Predictive Architecture, a framework he first proposed in 2022. Rather than predicting the future state of the world in pixel-perfect or word-by-word detail, the approach that makes generative AI both powerful and prone to hallucination, JEPA learns abstract representations of how the world works, ignoring unpredictable surface detail. The idea is to build systems that understand physical reality the way humans and animals do: not through language, but through embodied experience."
Coding is a solved problem; people with zero understanding can do it by copypasta from stack overflow, and similarly skilled LLMs can do it right now, cheaper. If you’re a “coder”, you have a lovely hobby but no career. Sorry.
If you’re a software engineer though, you have nothing to fear from current LLMs. But there is much more chance of LeCun’s models learning engineering - i.e. problem solving, in which writing code is just one of the tools, and not even the most important one - through physical experience and not just text. It is, after all, how all the software engineers today did the vast majority of their learning.