In my regular research (behind a paywall), I have been saying for a while that I think the future of AI is not large language models (LLM), but small language models (SLM) run on local desktop computers or even mobile phones.
What about those small prebuilt workstations with Strix/Gorgon Halo chips and big ram pools? Chinese OEMs like Beeline and GMKTec are offering some sleek little boxes. I guess it still technically not unified on a system level, but you allot most of it to VRAM in the BIOS regardless. You can probably get comparable performance with a fraction of the cost.
What about those small prebuilt workstations with Strix/Gorgon Halo chips and big ram pools? Chinese OEMs like Beeline and GMKTec are offering some sleek little boxes. I guess it still technically not unified on a system level, but you allot most of it to VRAM in the BIOS regardless. You can probably get comparable performance with a fraction of the cost.
Possibly, I haven’t looked at how easy it is to get your hands on one of those.