inari@piefed.zip to Technology@lemmy.worldEnglish · edit-217 hours agoDeepSeek ditches Nvidia for Huawei chips in V4 launchcybernews.comexternal-linkmessage-square44fedilinkarrow-up1221
arrow-up1221external-linkDeepSeek ditches Nvidia for Huawei chips in V4 launchcybernews.cominari@piefed.zip to Technology@lemmy.worldEnglish · edit-217 hours agomessage-square44fedilink
minus-squarebrucethemoose@lemmy.worldlinkfedilinkEnglisharrow-up7·edit-216 hours agoJust not power/cost efficiently on CPU only, is what I meant. CPUs don’t have the compute for batching (running generation requests in parallel). You need an accelerator, like Huawei’s, to be economical. It’s fine for local inference, of course.
minus-squareag10n@lemmy.worldlinkfedilinkEnglisharrow-up1·15 hours agoA whole ecosystem that can run on any hardware, efficiently or not, is a whole ecosystem developed for the Chinese market
Just not power/cost efficiently on CPU only, is what I meant. CPUs don’t have the compute for batching (running generation requests in parallel). You need an accelerator, like Huawei’s, to be economical.
It’s fine for local inference, of course.
A whole ecosystem that can run on any hardware, efficiently or not, is a whole ecosystem developed for the Chinese market