If there’s the demand, and I bet that there will be, someone is going to be making plug-and-play local AI compute boxes once prices come down that eliminate a lot of the learning curve. Like, preconfigured software and all that.
Joe User is not going to be talking about memory bandwidth or compiling software or setting sampler parameters. But—as the cloud guys demonstrate—he can use it if it’s set up. And there will be more-approachable front ends and such.
Maybe, I’m not really sure that’s competitive while the principal value prop of LLM tools is mass surveillance / advertising. I suppose commodity on-prem LLMs could become a thing after the current bubble pops and we scrape an economy together from the rubble. Maybe there’s a slice of people who understand the costs of trusting cloud LLMs (which would, I hope, be higher post-pop), but also lack the expertise to set up and run their own on-prem LLM, that would be interested in something like that.
I think demand for privacy-respecting (and creator-respecting) tools can be driven up by increasing global technical literacy. That would also, hopefully, increase the number of people able to create and maintain their own free tools, and decrease the ability of data slurpers to bilk ignorant people.
I wonder if they will adapt. People who are ignorant of how computers, software, and the internet work run into massive problems now, often without even realizing it. Making computers seem esoteric and difficult to understand serves the interest of those who would use those tools to scam, deceive, or otherwise manipulate people. I know it was a dizzying amount of information back when I got my degree, and it’s only gotten more complex since then. I’m sure there are a ton of people who wish they knew more but don’t know how.
A ton of adults in the US struggle with basic arithmetic, not because they’re stupid, but because they were never taught properly. They don’t try to learn, because there aren’t a ton of resources for adults at that level and it’s very embarrassing for an adult to ask for help with 8th grade (US grade for 12-13 years olds) math.
I’m going to have to do some research on this. The better adults (and children) who had shitty schools growing up can fill the gaps in their education, the harder it will be for billionaires to trick them with fancy gadgets like LLMs. Seems like a problem worth trying to address. The only thing I know off the top of my head is Khan Academy, and it’s been years since I used that, I don’t know if it’s any good nowadays. Regardless, the current state of affairs is obviously… suboptimal.
It used to be called the blinking twelve problem, because many adults were unable or unwilling to learn how to set the clock on a VCR, resulting in it being stuck at the start up state of flashing 12:00.
Even if running local AI would be as simple as downloading an app, a bunch would still have issues.
Yeah, it’s a colossal problem. One of the cool things about the internet is it allows for the decoupling of education from government or corporate power, at least as far as the internet itself can be decoupled from government or corporate power (an ongoing and also colossal problem), but it requires that people who need education 1) have physical access to education, 2) can locate the educational resources that they need, in the order they need them, and 3) are able and willing to seek out and take advantage of the resources available to them.
I’m not even sure how to start, it’s such a massive undertaking. But I think the more we can make education freely available and accessible on the internet, the better off the earth will be.
I also think that local models are generally preferable, but everyone having a local model requires more hardware, since much of the time the local hardware to run the local model is probably going to be idle, whereas hardware in the cloud can be servicing User B when it’s not servicing User A.
In 2026 and 2027, absent some very unexpected development, memory prices are going to be elevated.
And while some people could get local hardware, we couldn’t build a comparable-to-what-someone-could-get-in-the-cloud AI compute box for everyone if everyone wanted to go local, not for years, as we don’t have the memory production capacity today. If everyone tried to get a local box for AI compute at the same time, it’d just do what happened when the cloud companies did it, but at greater scale, because it’d require even more memory. There’d be a new memory shortage. Prices on memory would keep rising, pricing would-be buyers out, until the number of buyers still willing to buy was equal to what supply was present.
It’s an interesting idea to have an AI psychiatrist or AI confessional where you can talk to it about anything at all in an air-gapped environment and then at the end you can push a button that completely wipes every trace of the conversation that’s outside of your own mind. Could be beneficial.
Not really. Ideally, a professional would tell you what you need to hear where as a clanker tells you what you want to hear. I see what you’re saying though. The tech just doesn’t work the way you would need it to and it likely never will.
Use a local model
I imagine that for most of the people in that third, using a local model would be beyond their technical abilities.
If there’s the demand, and I bet that there will be, someone is going to be making plug-and-play local AI compute boxes once prices come down that eliminate a lot of the learning curve. Like, preconfigured software and all that.
Joe User is not going to be talking about memory bandwidth or compiling software or setting sampler parameters. But—as the cloud guys demonstrate—he can use it if it’s set up. And there will be more-approachable front ends and such.
Already kinda true with the dgx sparc, and probably will be replaced with a cheaper version when they find themselves overstocked.
Maybe, I’m not really sure that’s competitive while the principal value prop of LLM tools is mass surveillance / advertising. I suppose commodity on-prem LLMs could become a thing after the current bubble pops and we scrape an economy together from the rubble. Maybe there’s a slice of people who understand the costs of trusting cloud LLMs (which would, I hope, be higher post-pop), but also lack the expertise to set up and run their own on-prem LLM, that would be interested in something like that.
I think demand for privacy-respecting (and creator-respecting) tools can be driven up by increasing global technical literacy. That would also, hopefully, increase the number of people able to create and maintain their own free tools, and decrease the ability of data slurpers to bilk ignorant people.
People will adapt - or run into massive problems anyway.
I wonder if they will adapt. People who are ignorant of how computers, software, and the internet work run into massive problems now, often without even realizing it. Making computers seem esoteric and difficult to understand serves the interest of those who would use those tools to scam, deceive, or otherwise manipulate people. I know it was a dizzying amount of information back when I got my degree, and it’s only gotten more complex since then. I’m sure there are a ton of people who wish they knew more but don’t know how.
A ton of adults in the US struggle with basic arithmetic, not because they’re stupid, but because they were never taught properly. They don’t try to learn, because there aren’t a ton of resources for adults at that level and it’s very embarrassing for an adult to ask for help with 8th grade (US grade for 12-13 years olds) math.
I’m going to have to do some research on this. The better adults (and children) who had shitty schools growing up can fill the gaps in their education, the harder it will be for billionaires to trick them with fancy gadgets like LLMs. Seems like a problem worth trying to address. The only thing I know off the top of my head is Khan Academy, and it’s been years since I used that, I don’t know if it’s any good nowadays. Regardless, the current state of affairs is obviously… suboptimal.
It used to be called the blinking twelve problem, because many adults were unable or unwilling to learn how to set the clock on a VCR, resulting in it being stuck at the start up state of flashing 12:00.
Even if running local AI would be as simple as downloading an app, a bunch would still have issues.
Yeah, it’s a colossal problem. One of the cool things about the internet is it allows for the decoupling of education from government or corporate power, at least as far as the internet itself can be decoupled from government or corporate power (an ongoing and also colossal problem), but it requires that people who need education 1) have physical access to education, 2) can locate the educational resources that they need, in the order they need them, and 3) are able and willing to seek out and take advantage of the resources available to them.
I’m not even sure how to start, it’s such a massive undertaking. But I think the more we can make education freely available and accessible on the internet, the better off the earth will be.
I also think that local models are generally preferable, but everyone having a local model requires more hardware, since much of the time the local hardware to run the local model is probably going to be idle, whereas hardware in the cloud can be servicing User B when it’s not servicing User A.
In 2026 and 2027, absent some very unexpected development, memory prices are going to be elevated.
And while some people could get local hardware, we couldn’t build a comparable-to-what-someone-could-get-in-the-cloud AI compute box for everyone if everyone wanted to go local, not for years, as we don’t have the memory production capacity today. If everyone tried to get a local box for AI compute at the same time, it’d just do what happened when the cloud companies did it, but at greater scale, because it’d require even more memory. There’d be a new memory shortage. Prices on memory would keep rising, pricing would-be buyers out, until the number of buyers still willing to buy was equal to what supply was present.
That’s the way to go.
It’s an interesting idea to have an AI psychiatrist or AI confessional where you can talk to it about anything at all in an air-gapped environment and then at the end you can push a button that completely wipes every trace of the conversation that’s outside of your own mind. Could be beneficial.
Not really. Ideally, a professional would tell you what you need to hear where as a clanker tells you what you want to hear. I see what you’re saying though. The tech just doesn’t work the way you would need it to and it likely never will.
Personally, I’m more interested in being able to process personal data locally, which is legally impossible online. #GDPR