In my regular research (behind a paywall), I have been saying for a while that I think the future of AI is not large language models (LLM), but small language models (SLM) run on local desktop computers or even mobile phones.
What about those small prebuilt workstations with Strix/Gorgon Halo chips and big ram pools? Chinese OEMs like Beeline and GMKTec are offering some sleek little boxes. I guess it still technically not unified on a system level, but you allot most of it to VRAM in the BIOS regardless. You can probably get comparable performance with a fraction of the cost.
One of my first posts here was asking how to make use of local LLM the best. People said to buy the most beefy laptop within my budget.
Is that really the most reasonable take?
If you absolutely need one now, a mac is probably the cheapest way to run them because of the unified memory. With any x86 solution you have to get a separate video card with at least 32gb vram to run a decent local model. However, if you wait a bit then you can probably get a dedicated chip a lot cheaper in the near future https://wccftech.com/alibabas-tsmc-built-5nm-risc-v-chip-xuantie-c950-now-runs-qwen-3-8-27b-model-natively-unlocking-massive-vertical-integration-tailwinds/
What about those small prebuilt workstations with Strix/Gorgon Halo chips and big ram pools? Chinese OEMs like Beeline and GMKTec are offering some sleek little boxes. I guess it still technically not unified on a system level, but you allot most of it to VRAM in the BIOS regardless. You can probably get comparable performance with a fraction of the cost.