Both Ubuntu and Fedora have made it official: support is coming soon for running local generative AI instances.
An epic and still-growing thread in the Fedora forums states one of the goals for the next version: the Fedora AI Developer Desktop Objective. It is causing some discontent, and at least one Fedora contributor, SUSE’s Fernando Mancera, has resigned.
I think it is good to have optional support for local models that lets people use them in an offline and private and easy way. There is a lot of non technical folks using linux nowadays and many chose it for privacy and greater control over their data.
Depending on the implementation it could hook into certain os contexts and events to actually be helpful.
Either way I don’t see the cat going back in the bag with regards to LLMs. That being said I run Debian everywhere except my work machine which is ubuntu.
In this case, the the “bag” is a sucking black hole, and venture capitalists are throwing physics-defying amounts of money in it to drag the LLMs out. As soon as they stop that, the “cat” goes back in the “bag”.
Local LLM models are an exception, but they are also atrocious by comparison. Most users will get some limited utility from an LLM if they had one, maybe, but it is being accommodated and foisted everywhere like its the invention of the mouse. It is nowhere near as paradigm shifting, but is being hyped, advertised, and marketed more aggressively than any product in history. So, the roaring hype makes everyone think that if they don’t get on board too, they’ll be left in the dust, so now well-meaning projects are getting bloated up for it too.
Many of us just want this technology to get the fuck away from us until it is worth using or dies already. Is that so very much to ask?
Using an llm mondel that isn’t super advanced is actually quite freeing in my opinion, the generated output is always mediocre at best, but it’s usually good enough for boilerplate and can be decent if you need to unstuck yourself. It also isn’t good enough to lull you into just letting the llm do all the work for you since it makes obvious mistakes.
I mean if it’s completely optional and opt-in, sure go for it. Knock yourself out. I won’t be using it (my computer isn’t powerful enough to run local LLMs)
SUSE contributes to Fedora? WTF?
not anymore. IBM decided a few years ago that they don’t want grubby fossy fingers in their pie.
kind of makes me question why anyone who supports the foss part of Linux would continue to use fedora or any IBM product.
I say this as an ex fedora fanboy.
fuck IBM.
Someone from SUSE used to contribute to Fedora
lol they already support running local models. wtf is the distro gonna do…? pre-install llama.cpp? this is so silly to me that people are resigning over this, too.
Is that the one that really whips the llama.cpp’s ass?
Now I have to install winamp and get a bunch of sick skins for it.
They’re discussing a new spin specifically for AI developers, not changing existing distros to include new stuff.
CTFD
Oh good. We can all just ignore the AI version then?
No we can’t, we must be mad, because “AI”
Well, we get to slam it and heckle its users, because, well, we need SOME joy in The world…
Yes
Shame
Shaame!
Soooo, which distro to switch to next? Or are they all gonna go down this route eventually? Maybe I’ll try a *BSD for once.
Debian or anything based directly on it
Debian or anything based directly on it
Debian can also run AI models. Pretty dumb reason to get mad about.
Every linux distro can run AI models, there is 0 reason to support distros that ship with AI bullshit by default
pretty much anything that is “you build it youself” so Arch, NixOS, Gentoo, etc
but yeah I’m also tempted to finally give BSD a go.
nixos comes with systemd
I switched from Kubuntu to CachyOS last week, after 10 years or so. CachyOS is based on Arch, and did not disappoint so far, extremely fast, makes Ubuntu look old and sluggish. It’s really impressive. The basic installation was easy. The GUI package manager isn’t as polished but works. A little bit of terminal tweaking was required to install some packages (VMM and KRDC gave me some trouble) but the documentation was ok. Absolutely can recommend.
Cachy is a nice mix of latest and stable. Still recommend Debian for work, but Cachy is great if you want something that just works
I’ve used PikaOS which is Debian based and IMHO it’s right up there with CachyOS. super fast install, fast distro, great package manager, and repos with the latest stuff. very stable too.
I don’t really understand the point? You can already do a lot on Linux using AI via CLI with bash.
I don’t really understand the point?
Correct: you don’t. I’m basing this only off what you wrote, but I’m reasonably confident of my answer. Glad I could answer the question as asked.
