jaykrown@lemmy.world to Technology@lemmy.worldEnglish · 12 hours agoDeepSeek Permanently Reduces The Price Of Its Flagship V4 Model By 75 Percenttech.yahoo.comexternal-linkmessage-square77fedilinkarrow-up1335
arrow-up1335external-linkDeepSeek Permanently Reduces The Price Of Its Flagship V4 Model By 75 Percenttech.yahoo.comjaykrown@lemmy.world to Technology@lemmy.worldEnglish · 12 hours agomessage-square77fedilink
minus-squareMwa@thelemmy.clublinkfedilinkEnglisharrow-up4·edit-210 hours agoStill gonna self host it instead (maybe)
minus-squareTja@programming.devlinkfedilinkEnglisharrow-up1·2 hours agoHow are they running it? Doesn’t the model have to fit in (V)RAM? Does Nvidia have such huge memories in the H cards?
minus-squareboonhet@sopuli.xyzlinkfedilinkEnglisharrow-up1·edit-223 minutes agoFor self hosting it essentially needs to fit in VRAM + RAM but it’ll take a lot of CPU for the part in RAM Deepseek probably uses those big fancy H cards and not one but several together to increase VRAM.
Still gonna self host it instead (maybe)
FYI the flash model is ~158 GB
How are they running it? Doesn’t the model have to fit in (V)RAM? Does Nvidia have such huge memories in the H cards?
For self hosting it essentially needs to fit in VRAM + RAM but it’ll take a lot of CPU for the part in RAM
Deepseek probably uses those big fancy H cards and not one but several together to increase VRAM.
The destiled models?