jaykrown@lemmy.world to Technology@lemmy.worldEnglish · 12 hours agoDeepSeek Permanently Reduces The Price Of Its Flagship V4 Model By 75 Percenttech.yahoo.comexternal-linkmessage-square77fedilinkarrow-up1335
arrow-up1335external-linkDeepSeek Permanently Reduces The Price Of Its Flagship V4 Model By 75 Percenttech.yahoo.comjaykrown@lemmy.world to Technology@lemmy.worldEnglish · 12 hours agomessage-square77fedilink
minus-squareTja@programming.devlinkfedilinkEnglisharrow-up1·2 hours agoHow are they running it? Doesn’t the model have to fit in (V)RAM? Does Nvidia have such huge memories in the H cards?
minus-squareboonhet@sopuli.xyzlinkfedilinkEnglisharrow-up1·edit-223 minutes agoFor self hosting it essentially needs to fit in VRAM + RAM but it’ll take a lot of CPU for the part in RAM Deepseek probably uses those big fancy H cards and not one but several together to increase VRAM.
How are they running it? Doesn’t the model have to fit in (V)RAM? Does Nvidia have such huge memories in the H cards?
For self hosting it essentially needs to fit in VRAM + RAM but it’ll take a lot of CPU for the part in RAM
Deepseek probably uses those big fancy H cards and not one but several together to increase VRAM.