If you think long term as a company that uses AI that’s the way to go anyway, your own AI server.

But alas, nobody cares about the long term, because the cunts at the top of the AI stack always make sure to make things so volatile that the little guy can never survive past the short term.

The solution? Oh just pay the big corporations to be dependent on them and not build your own thing. Surely that will help.

You have to realize, that by your own words, AI subscription prices will skyrocket eventually. So the cost analysis of your own AI server has to take that into account too, not just the current prices and current upfront cost.

I am running qwen 3.5 locally using llama.cpp on 8gb of VRAM and 16 gigs of RAM. It works well enough with a 4B to 9B parameter model along with quantization and MTP. More optimizations are on the way with turboquant and possibly other tech.

It’s just there to assist me, not do all the work, so I am happy as long as I can self host it.

I can’t say how well my specs would work in a professional setting but for personal use a MacBook should be sufficient in my opinion.

I run this setup with 36GB (32+4). Local LLMs can be really effective BUT you are constrained by context size in a way you aren’t on cloud services.

Cline supports running a local model through lmstudio but my experience feeding it any significant tasks is it just can’t handle reading and holding the contexts to build components for enterprise scale applications.

I use Claude to write a lot of utility one-off scripts. With a maximum window of 1M tokens I can hit 30+% context just writing Python scripts. API contracts, development standards, existing reusable modules, and sometimes reading the code/documentation of the services I’m going to be calling.

My MacBook can’t handle 300k token contexts. 30k seems doable. I should see how it handles my utility script folder…

Anyway that’s still no Claude but if you need a cheaper model and you can afford for developers to spend time on it before ultimately deciding they need to spend for Claude or Codex or Gemini, then rubbing a local model on a beefy MacBook is 100% an option.

Stepping up from there to building a locally hosted LLM is probably the worst of all worlds. It will be a beefy CapEx, prone to saturation by all the users, and you will most likely still have to punt the hardest jobs to cloud AI. It can certainly be done and done well, but the best example I know runs on $250-500k worth of hardware (to service a pretty big number of users to be fair).

I mean… RAM? Don’t you need mass VRAM for this kind of thing? Or are they shared on Mac?

idk how to calculate how long it pay for itself…

You don’t… Not in this industry. You guess and hope it goes in your favor.

No calculations matter if the market can jump or drop by 300% in a few months… And that applies to programming, hardware prices, AI subscription prices, regulations between countries when Trump is in office…