I’ve looked into self-hosted AI and decided it’s not worth the cost - both in terms of hardware and energy - when compared to the relative value to be gotten out of it. YMMV.
The models you can run on consumer hardware are still nowhere near the stuff that runs in corporate data centers. To stick with your metaphor, its like running a little steam engine at home while the big guys get to operate nuclear reactors…
I’ve looked into self-hosted AI and decided it’s not worth the cost - both in terms of hardware and energy - when compared to the relative value to be gotten out of it. YMMV.
Same, pretty much. It is possible though, which makes LLMs a more democratic technology than, say, nuclear reactors.
The models you can run on consumer hardware are still nowhere near the stuff that runs in corporate data centers. To stick with your metaphor, its like running a little steam engine at home while the big guys get to operate nuclear reactors…