lemmydividebyzero@reddthat.com to Technology@lemmy.worldEnglish · 1 month agoOpensource AI Must Winopensourceaimustwin.comexternal-linkmessage-square111fedilinkarrow-up1284cross-posted to: technology@midwest.socialtechnology@lemmy.ml
arrow-up1284external-linkOpensource AI Must Winopensourceaimustwin.comlemmydividebyzero@reddthat.com to Technology@lemmy.worldEnglish · 1 month agomessage-square111fedilinkcross-posted to: technology@midwest.socialtechnology@lemmy.ml
minus-squareZephyrXero@lemmy.worldlinkfedilinkEnglisharrow-up4·1 month agoI got Qwen 3.5:9b running on my 8GB GPU the other day, and it still has some room left over
minus-squaremabeledo@lemmy.worldlinkfedilinkEnglisharrow-up2·1 month agoI was talking about combined system RAM. People often overestimate what the average system specs are.
minus-squareZephyrXero@lemmy.worldlinkfedilinkEnglisharrow-up4·1 month agoIf the model dumps over to system ram it gets super slow, you ideally want it to fit completely in your VRAM
I got Qwen 3.5:9b running on my 8GB GPU the other day, and it still has some room left over
I was talking about combined system RAM. People often overestimate what the average system specs are.
If the model dumps over to system ram it gets super slow, you ideally want it to fit completely in your VRAM