not_IO@lemmy.blahaj.zone to Lemmy Shitpost@lemmy.worldEnglish · 1 day agothe myth of consensual pc buildinglemmy.blahaj.zoneimagemessage-square16fedilinkarrow-up137
arrow-up137imagethe myth of consensual pc buildinglemmy.blahaj.zonenot_IO@lemmy.blahaj.zone to Lemmy Shitpost@lemmy.worldEnglish · 1 day agomessage-square16fedilink
minus-squarejpablo68@infosec.publinkfedilinkarrow-up5·1 day agoI’m running gpt4all on a thinkpad T470, and although is slow I rather use my own hardware for running LLMs
minus-squarejh29a@lemmy.blahaj.zonelinkfedilinkarrow-up1·14 hours agoNow you can ask the PC for consent
minus-squaredb2@lemmy.worldlinkfedilinkarrow-up5·20 hours agoThey have uses, most of which aren’t what’s being pushed right now and none of which are what the tech bro sociopaths are pushing.
minus-squareDaTingGoBrrr@lemmy.mllinkfedilinkarrow-up2·19 hours agoIf you have enough regular ram and an nvme ssd you can run huge models on modest hardware using Colibrí. I got a Lenovo mini PC with 64GB of ram and a 500GB nvme and a 38B model runs with 11 tokens per second on my machine. The 38B model fits in my ram but if I had used a 120B parameter model then it would stream the rest of the model from my nvme disk.
I’m running gpt4all on a thinkpad T470, and although is slow I rather use my own hardware for running LLMs
Now you can ask the PC for consent
Please don’t run LLMs at all…
They have uses, most of which aren’t what’s being pushed right now and none of which are what the tech bro sociopaths are pushing.
I mainly use it to practice german.
If you have enough regular ram and an nvme ssd you can run huge models on modest hardware using Colibrí.
I got a Lenovo mini PC with 64GB of ram and a 500GB nvme and a 38B model runs with 11 tokens per second on my machine.
The 38B model fits in my ram but if I had used a 120B parameter model then it would stream the rest of the model from my nvme disk.