In my regular research (behind a paywall), I have been saying for a while that I think the future of AI is not large language models (LLM), but small language models (SLM) run on local desktop computers or even mobile phones.
LLM ASICs are different to NPUs. The idea is to encode a specific model’s weights (all billions of them) as a gigantic resistor network on the chip itself. Analog, where ohms law becomes the processor, and no RAM is necessary - the chip IS the model. It would pave the way for cost effective, fast, low-power local inference. Given the analog circuitry, there would be issues like interference, temperature sensitivity and so on, but we’re seeing models get good results even when heavily quantised, so the idea has legs. The big disadvantage is you’re stuck with that one model forever, so we’re likely to see these first appear in non-consumer applications like automotive, defence etc. A pretty exciting concept that solves some of the environmental issues around AI.
LLM ASICs are different to NPUs. The idea is to encode a specific model’s weights (all billions of them) as a gigantic resistor network on the chip itself. Analog, where ohms law becomes the processor, and no RAM is necessary - the chip IS the model. It would pave the way for cost effective, fast, low-power local inference. Given the analog circuitry, there would be issues like interference, temperature sensitivity and so on, but we’re seeing models get good results even when heavily quantised, so the idea has legs. The big disadvantage is you’re stuck with that one model forever, so we’re likely to see these first appear in non-consumer applications like automotive, defence etc. A pretty exciting concept that solves some of the environmental issues around AI.
More info for the curious: https://taalas.com/the-path-to-ubiquitous-ai/