Some_Emo_Chick@lemmy.world to Technology@lemmy.worldEnglish · 1 day agoGenerative AI Is an Engineering Disaster. A shockingly inefficient trillion-dollar project.www.theatlantic.comexternal-linkmessage-square112fedilinkarrow-up1618cross-posted to: usa@midwest.social
arrow-up1618external-linkGenerative AI Is an Engineering Disaster. A shockingly inefficient trillion-dollar project.www.theatlantic.comSome_Emo_Chick@lemmy.world to Technology@lemmy.worldEnglish · 1 day agomessage-square112fedilinkcross-posted to: usa@midwest.social
minus-squareBrett@programming.devlinkfedilinkEnglisharrow-up3·23 hours agoIs that quantized? 4 bit Qwen 3.6 can get 22tps on a 1060.
minus-squareAsafum@lemmy.worldlinkfedilinkEnglisharrow-up2·21 hours agoIt’s the q4 quantization, but it requires 20+GB vram and my 5080 only has 16
minus-squareDamage@feddit.itlinkfedilinkEnglisharrow-up2·13 hours agoMy framework 13 with shared RAM runs qwen quite well
Is that quantized? 4 bit Qwen 3.6 can get 22tps on a 1060.
It’s the q4 quantization, but it requires 20+GB vram and my 5080 only has 16
My framework 13 with shared RAM runs qwen quite well