Microsoft is simultaneously promising to make Windows 11 run better on 8GB PCs while pushing Windows toward more local AI. If 8GB becomes the norm again, how exactly does Microsoft expect Windows PCs to become the platform for on-device AI?
From a purely mechanical hardware side, if I’m Microsoft, I’d say that they’d need to get existing computers, mostly laptops, upgraded when memory prices drop. That won’t get you local AI compute immediately, but it’ll be the fastest route to doing so. As things stand:
Much memory is soldered to achieve higher throughput.
Computer systems generally cannot run memory at different speeds, so the lowest speed stuff bounds the perfoemance of the system. Sticking any slower memory in hurts system perfoemance.
People probably will not entirely replace laptops until the whole system ages out, which will take years. If that’s what it requires to move to higher memory systems, it’s going to
So:
Stage 1: Get existing laptops upgradeable. “CoPilot+ ready”, say.
Get CAMM2 sockets in laptops. Preferably an entirely free slot.
Have Windows tell a user when they need an upgrade to use given functionality.
Have some mechanism in Windows to direct people to a nearby PC shop or vendor that can do upgrades.
Stage 2: Get hardware and OS support for non-uniform speed memory. This will take longer.
CPUs and motherboards need to be able to handle different memory speeds on different memory chips.
Start adding functionality to Windows to heuristically make use of higher-speed memory in systems with multiple speeds.
Another option: “Local cloud”. This won’t help some users — part of what Microsoft wants to leverage is their control of the local device, versus someone like OpenAI running in the cloud. That means they provide features that need to be always available, even when no other comouting devixes are available.
But IMHO, there’s also the possibility that some people just don’t want to rely on a remote service. If you have, basically, plug-and-play support for pairing that laptop to a more-capable computer on the network or USB-attached parallel compute node, then that box could do the heavy compute lifting. My suspicion is that laptops are always going to be constrained on parallel compute because of power and heat limitations. So making it easy for Joe User endow them with that capability with an external box, the way he can add printing to his laptop by having a printer attached via USB or on the network…that’s got a lot of long-term potential for heavier compute capability. EGPUs, probably the closest current analog, don’t have phenomenal latency, but current AI compute tasks aren’t particularly latency-sensitive.
From a purely mechanical hardware side, if I’m Microsoft, I’d say that they’d need to get existing computers, mostly laptops, upgraded when memory prices drop. That won’t get you local AI compute immediately, but it’ll be the fastest route to doing so. As things stand:
Much memory is soldered to achieve higher throughput.
Computer systems generally cannot run memory at different speeds, so the lowest speed stuff bounds the perfoemance of the system. Sticking any slower memory in hurts system perfoemance.
People probably will not entirely replace laptops until the whole system ages out, which will take years. If that’s what it requires to move to higher memory systems, it’s going to
So:
Stage 1: Get existing laptops upgradeable. “CoPilot+ ready”, say.
Get CAMM2 sockets in laptops. Preferably an entirely free slot.
Have Windows tell a user when they need an upgrade to use given functionality.
Have some mechanism in Windows to direct people to a nearby PC shop or vendor that can do upgrades.
Stage 2: Get hardware and OS support for non-uniform speed memory. This will take longer.
CPUs and motherboards need to be able to handle different memory speeds on different memory chips.
Start adding functionality to Windows to heuristically make use of higher-speed memory in systems with multiple speeds.
Another option: “Local cloud”. This won’t help some users — part of what Microsoft wants to leverage is their control of the local device, versus someone like OpenAI running in the cloud. That means they provide features that need to be always available, even when no other comouting devixes are available.
But IMHO, there’s also the possibility that some people just don’t want to rely on a remote service. If you have, basically, plug-and-play support for pairing that laptop to a more-capable computer on the network or USB-attached parallel compute node, then that box could do the heavy compute lifting. My suspicion is that laptops are always going to be constrained on parallel compute because of power and heat limitations. So making it easy for Joe User endow them with that capability with an external box, the way he can add printing to his laptop by having a printer attached via USB or on the network…that’s got a lot of long-term potential for heavier compute capability. EGPUs, probably the closest current analog, don’t have phenomenal latency, but current AI compute tasks aren’t particularly latency-sensitive.