SocialistVibes01@lemmy.ml to Linux@lemmy.mlEnglish · edit-23 months agoWhich specs are as low as reasonable possible for local LLM models? Do you recommend some distro in particular?message-squaremessage-square19fedilinkarrow-up132file-text
arrow-up132message-squareWhich specs are as low as reasonable possible for local LLM models? Do you recommend some distro in particular?SocialistVibes01@lemmy.ml to Linux@lemmy.mlEnglish · edit-23 months agomessage-square19fedilinkfile-text
minus-squareplutopos@lemmy.ziplinkfedilinkarrow-up1·2 months agoThere are no lower or upper bounds for local LLMs. Just get the best GPU/RAM combo you can with your budget. Some laptops also have NPUs that are dedicated specifically to AI iirc
minus-squareSocialistVibes01@lemmy.mlOPlinkfedilinkarrow-up1·2 months agoThanks. What about NVidia vs AMD? Should I go with AMD if cheaper?
minus-squareplutopos@lemmy.ziplinkfedilinkarrow-up1·1 month agoAMD has better Linux support if you plan on using it
There are no lower or upper bounds for local LLMs. Just get the best GPU/RAM combo you can with your budget. Some laptops also have NPUs that are dedicated specifically to AI iirc
Thanks. What about NVidia vs AMD? Should I go with AMD if cheaper?
AMD has better Linux support if you plan on using it