open-weight models are based
I almost wish I could see into my old job just to see how they are handling this after they force LLMs into fucking everything.
Might see if their website still has a sales chatbot and talk to it about my day.
Anything that’s going to become foundational to the Internet is bound to become open source and collaborative. Right?
The majority of software/hardware components? Absolutely.
However, I wouldn’t be surprised if some things are gatekept for as long as possible so the “owners” can rent-seek their copyrights/patents.
Sure but there will always be a function of hardware and energy cost.
Today’s models will run cheaply in the distant future, but the distant future’s models could only be dreamed of today. Hopefully at some point we get to a point where quality is “good enough” on cheap hardware and low energy, but I can’t tell you when it’ll get here. I bet at minimum another decade, unless you’re ok with what you get out of today’s models on dedicated consumer GPUs.
I think the current stuff that runs on processors and normal ram is worse than useless though.
I currently run Qwen 35B 3.6 A3B on a 5070 with 12G VRAM and I find it surprisingly useful. I use it to ask questions I want answers to that may contain sensitive information or which I don’t want to feed to the data harvesters.
unless you’re ok with what you get out of today’s models on dedicated consumer GPUs
This is all I use, mostly for quickly putting together personal software and doing linux stuff, it doesn’t feel limiting and is already really powerful. A lot of the stuff those models struggle with can be overcome by giving better context and more specific instructions, and that can be automated, so they should become more useful as harness software advances, independently of advancements in the models themselves. Maybe I have a limited perspective because I just haven’t tried the frontier models, but developing a dependence on services run by malevolent companies that obviously intend to use that dependence as leverage is deeply unappealing, and I’m not sure what they could offer to make that seem worth it on top of what I can already do with my own computer.
Or they could just hire someone to do the job……
This is legit a great idea.
Deepseek api is a lot less expensive, I use it all the time. The more of that is being done, less DCs they’re gonna build locally and more cheaper electric for the rest of us.You can use the Chinese services, you’re just giving a foreign government money and data instead of your home government (unless you live in china).
I would encourage all to at least experiment with running their own locally.
You can install Lmstudio on your computer and get 75% of what you need from a model installed and run at home. It won’t be as fast nor will it have all of the features, but it will be private and under your control. Using lm studio is easy, even for a novice. As you get better, you can add your own features. If you persist, you can stop using lmstudio and go right to something like Ollama running on your hardware and tinker all you like.
It really isn’t too hard to start and you will learn a lot along the way.
I am not American or Chinese, why would I care which government gets my data? They are both shit.
