I’m not sold that unified memory is going to be the future of PCs.
Unified memory systems are convenient right now, because you have been able to run out and get relatively-affordable affordable systems, like a Framework Desktop, that have unified memory and can take a lot of memory.
Discrete GPUs, which are mostly where we’ve done parallel compute, haven’t had that much memory, because there’s little point in gaming. Sure, theoretically a video game designer could make games that make useful use of a ton of VRAM. But it’d also make gaming systems cost more, and you need mass to make a platform attractive for game developers to target. If there were enough people willing and able to buy dGPUs with 512GB of memory to play games, then we’d have them out there. But…that market doesn’t exist in 2026.
And in any event, in the short term, dGPU vendors are going to want to price-discriminate between (more price sensitive) gaming customers and (less price sensitive) AI compute. I don’t think that that’s long-term sustainable, but right now, Nvidia is the leader in compute for AI stuff. As other companies move into the market, make it more competitive, I think that that’ll change.
So…we don’t have a lot of high-RAM dGPUs today. But I’m not sold that that situation will last in the long term.
Unified memory systems are useful if you have a major need to frequently share a lot of data between the parallel compute hardware and the CPU. But…currently, AI compute stuff doesn’t really do that. You load a model, and that moves a lot of data, but then the CPU doean’t need to do much with it. Maybe that could change, and someone will come up with new systems that take advantage of tight coordination between CPU and the parallel compute hardware. But as of now, it doesn’t.
The properties of the memory that are most-useful aren’t the same. Parallel comoute is really bandwidth-intensive, whereas that may not be as critical for a CPU. The CPU cares a lot about latency, whereas that may not matter much for the parallel compute hardware—for LLMs, you often know well in advance what memory you’re going to need next, moreso than the CPU. I remember reading a paper recently talking about how current HBM memory isn’t optimized for LLM compute—for LLMs, you want more read capacity and don’t care as much about write.
Plus, having different memory pools simply means that the CPU and parallel compute hardware aren’t competing for bandwidth.
That’d all tend to suggest to me that it might be a good idea to have separate memories for the two in the long run. I think that that might be where we ultimately wind up, rather than with unified-memory systems as the norm.
Unified memory hasn’t been the norm for a long time. The stranglehold is the bus connecting the memory to the graphics. SoC designs can bypass the strangle since they put the memory and gpu on the same die or literally next to it.
Having unified memory means removing unnecessary copying of data between memory segments, saving both memory usage, cpu and I/O time, and energy, especially as integrated systems (SoC) use the same literal memory chips for both things.
Basically, memory should not need to be dedicated to any task at all, it should be available to be allocated and referenced as required by the task that needs it, but since historically gpu memory and system memory are different memorys that require data to be loaded in either or the other, we are stuck with that paradigm in apps writen with that divide in mind.
I’m not sold that unified memory is going to be the future of PCs.
Unified memory systems are convenient right now, because you have been able to run out and get relatively-affordable affordable systems, like a Framework Desktop, that have unified memory and can take a lot of memory.
Discrete GPUs, which are mostly where we’ve done parallel compute, haven’t had that much memory, because there’s little point in gaming. Sure, theoretically a video game designer could make games that make useful use of a ton of VRAM. But it’d also make gaming systems cost more, and you need mass to make a platform attractive for game developers to target. If there were enough people willing and able to buy dGPUs with 512GB of memory to play games, then we’d have them out there. But…that market doesn’t exist in 2026.
And in any event, in the short term, dGPU vendors are going to want to price-discriminate between (more price sensitive) gaming customers and (less price sensitive) AI compute. I don’t think that that’s long-term sustainable, but right now, Nvidia is the leader in compute for AI stuff. As other companies move into the market, make it more competitive, I think that that’ll change.
So…we don’t have a lot of high-RAM dGPUs today. But I’m not sold that that situation will last in the long term.
Unified memory systems are useful if you have a major need to frequently share a lot of data between the parallel compute hardware and the CPU. But…currently, AI compute stuff doesn’t really do that. You load a model, and that moves a lot of data, but then the CPU doean’t need to do much with it. Maybe that could change, and someone will come up with new systems that take advantage of tight coordination between CPU and the parallel compute hardware. But as of now, it doesn’t.
The properties of the memory that are most-useful aren’t the same. Parallel comoute is really bandwidth-intensive, whereas that may not be as critical for a CPU. The CPU cares a lot about latency, whereas that may not matter much for the parallel compute hardware—for LLMs, you often know well in advance what memory you’re going to need next, moreso than the CPU. I remember reading a paper recently talking about how current HBM memory isn’t optimized for LLM compute—for LLMs, you want more read capacity and don’t care as much about write.
Plus, having different memory pools simply means that the CPU and parallel compute hardware aren’t competing for bandwidth.
That’d all tend to suggest to me that it might be a good idea to have separate memories for the two in the long run. I think that that might be where we ultimately wind up, rather than with unified-memory systems as the norm.
Unified memory hasn’t been the norm for a long time. The stranglehold is the bus connecting the memory to the graphics. SoC designs can bypass the strangle since they put the memory and gpu on the same die or literally next to it.
Having unified memory means removing unnecessary copying of data between memory segments, saving both memory usage, cpu and I/O time, and energy, especially as integrated systems (SoC) use the same literal memory chips for both things.
Basically, memory should not need to be dedicated to any task at all, it should be available to be allocated and referenced as required by the task that needs it, but since historically gpu memory and system memory are different memorys that require data to be loaded in either or the other, we are stuck with that paradigm in apps writen with that divide in mind.