Mine is that the useState hook is an actual plague on the entire React ecosystem. Regardless of your opinions of React, if you wanna use something then use it right.
Mine is that the useState hook is an actual plague on the entire React ecosystem. Regardless of your opinions of React, if you wanna use something then use it right.
I agree on their usefulness but as long as you don’t use local and open LLMs you are unfortunately part of the ecosystem that does the damage. Luckily open models are catching on and hopefully some reliable companies take up on this business model.
I can’t wait for the crash, so that we can get more sustainable AI and access to the hardware to run it.
Won’t happen unfortunately. Unless you want to run a 1U with 6 GPU, requiring a custom water-cooling system, and which sips like 10kW/h of electricity while idle. It’s all integrated hardware born to be trash. The ram, cpu and GPU won’t be usable on consumer hardware without hard modifications.
1 of those GPUs can probably run 90% of what a small team would need, so you don’t need the full thing.
That’s about 600w of draw and 150w idle. Or about the same as a 5090. You don’t need exotic power or cooling for that.
They’re all soldered together, you can’t just take one
We also don’t need frontier models. The type of models that run on $5000 Mac studios can do today what the frontier models did 1 year ago.
We just need the Mac studio class hardware to come back to $2000 where it used to be.
Also, do not underestimate the ability of Chinese BGA rework shops from being able to salvage AI chips and install them on board that will fit in standard hardware. They proved they could do it with the Crypto only chips, they will figure it out for the AI chips if the supply is there.
The main reason these units use water cooling is for deployment density.
Local models are only months behind. The primary utility of comically large networks is a wider search space for the tiny fraction of weights that really matter.
I agree with you but that doesn’t change the fact datacentre hardware is not consumer friendly like it was in the past
Was it?
Not the point - this isn’t about secondhand servers, it’s about RAM being artificially scarce. Possibly as a deliberate measure to make cloud services sound less wasteful. If a 3090 with 32 GB was only a thousand bucks then OpenAI’s revenue would be even worse.
I have an old server with lots of DDR4 which I can for the most part just plug into my PC. The CPU can be put into another motherboard, and the GPU are discrete cards. Most of that hardware can be re used by consumers.
Most new AI datacentre deployment won’t be like that. An Nvidia Blackwell server has everything soldered. Good luck desoldering the CPU or GPU, the ram is either LPDDR5X which I never saw consumer motherboard use, the GPU is HBM3e and not a discrete card anymore. Whole package is unpractical for even a enthusiast to run.
The original point was that once the AI bubble bursts, we’ll have hardware to use for local inference. My claim is that no, it’ll probably be too specialized. Unless some maniac in China figures out a way to make new consumer hardware for it, but they probably won’t since Nvidia hardware is technically illegal (both export and import at this point). Even then, I don’t see Nvidia or others go back to making consumer hardware at a level where we can do serious inference locally.
Do not underestimate the Chinese electronics industry. Their adaptors are only 1 generation behind the current socket.
https://www.aliexpress.com/item/1005012221278601.html
Again, this is not about server hardware.
RAM is hideously overpriced because these doomed companies have open orders for all industrial capacity for years into the future. When the bubble bursts, the capacity’s still right there, and the same companies will be eager to sell mundane hardware to normal people at whatever prices we’ll still pay.
We are not in any sense talking about plugging datacenter machinery into home computers. That is not how the artificial crisis will end.
Why. Who else is gonna buy their silicon, when this diversion craters? They are presently making consumer hardware at a level for serious inference locally. What “serious” means gets better month by month, as a matter of software research. There is nothing to “go back” to; it’s just gonna be the only division in the black.