If they weren’t needlessly nervous (by that I mean paranoid), they would not be in that position.
Whatever their wealth, those people cannot feel safe.
Their IPOs are how they cash out and walk away. It’s just a matter of “does it crash before or after?”. Yes they’ll be anxious about getting there, but the fact the market is still behind them and they’re this close is very bad for the rest of us.
What’s stopping people from using open sourced trained models? Like, why are tokens going to be needed in the future? Isn’t this their whole business model?
Honestly, I have a little tin-foil-hat theory that the used their infine supply of venture to fuck the memory market to make sure people couldn’t run giant chinese models on solar and eat their lunch.
training a top-level model takes harvesting petabytes / exabytes of revelant data and then months of continuous 100% power on a large server farm costing many millions of dollars.
It’s a one-and-done process, then the normal Q&A usage has vastly lower requirements.
Local models generally means you download the result of all that , after it’s done, (the PT in ChatGPT stands for pre-trained) and then generate answers locally, but the training wasn’t done locally. The generating answers step is called “inference”.
Hardware cost to do inference with a top-level model is within reach of most small businesses. Hardware cost to do training of a top-level model is orders of magnitude higher.
So, there are open-source models that are trained by universities, and then used by affluent but not billionaire people at home.
There are open-source models that can do inference (answer questions) on a typical 8GB gaming graphics card, but they aren’t in the same order of magnitude of quality as the famous top-level models.
They’re the output of all that data centre processing. The model is fixed at that point and can’t “learn” anything more, so the pressure is always on to release a newer model that resulted from more data with better algorithms, but still small enough t9 be able to run on a consumer grade GPU
The hardware requirements are substantial. If companies or individuals want to run models locally, especially at the speed you’d expect from the big name models, then everyone needs a pretty serious GPU on their work station. That or the company needs to build and maintain a server capable of running whatever loads they expect to see.
None of this is impossible of course. But you pay for the tokens one way or another. Either you pay the upfront cost of the hardware and continuing cost of the electricity to run it, or you pay with the inconvenience of running a model very slowly on lower end hardware, or you pay a subscription for someone else to worry about all that.
This is why local models are still niche. Only hobbyists nerds are willing to go through the trouble of setting up and running a local model. Most companies still are not sure if AI is actually worth it for them. So it’s much easier to pay for tokens than the giant upfront cost required to build you own infrastructure.
If the numbers the AI companies are putting up as far as infrastructure cost vs revenue are anything to go by, it probably isn’t worth it to most companies.
For enterprise, I think same reasons why companies using paid software instead of open software. Someone else to maintain and blamed if something doesn’t work.
They don’t need to, OpenAI and Anthropic are running at a huge loss to keep it cheaper to use them than to buy the comparable hardware required to replace them. The only way running open source models is cheaper is if you ignore hardware costs, or if you have a very specific and unusual usage pattern (i.e. constant 100% capacity, and not a clock cycle more or less), or if you go with another cloud provider who are also running at a loss.
If you want it well rounded I can see the argument for enterprise but for me personally I programmed one on my workstation and it’s trained on very specific things I think it barely uses up all 24gb vram I have available.
come on, I’ve trained toy size models too, you can’t seriously say they’re even vaguely approaching the capability of models that takes a whole massive server farm weeks or months to train.
Right, and what proportion of enterprise use cases genuinely need an appreciable fraction of that power (because they are wide ranging and non-repeating)?
Anyone that says these things aren’t impressive is kidding themselves. But how do we get from impressive to productivity enhancing in excess of cost? I find smaller specialised models more plausible on that front, but jury is still out on the financials, we’ll find out I guess!
I’ll admit I’m experiencing a degree of pleasure in thinking about how nervous Anthropic and OpenAI must be with their IPOs supposedly coming up.
They’re already billionaires many times over. Hard to feel nervous with that kind of wealth in your pocket
Based on the current valuations of their companies. They will likely remain 0.1%ers, but it will still hurt.
Place your bets
Place your bets
If they weren’t needlessly nervous (by that I mean paranoid), they would not be in that position. Whatever their wealth, those people cannot feel safe.
Idk about that. Musk and Zuck seem like very careless people.
They gotta pull Google and get into the advertising game. They are sitting on a gold mine of private data and I’m sure they know it.
Their IPOs are how they cash out and walk away. It’s just a matter of “does it crash before or after?”. Yes they’ll be anxious about getting there, but the fact the market is still behind them and they’re this close is very bad for the rest of us.
Looks like the market is now trying to get out from under them.
OpenAI already explicitly postponed their IPO somehow hoping for a better climate to IPO in…
A certainly… optimistic expectation.
You should not try to make sense of a decision they probably used chatGPT to make. It’s bad for your metal health
NASDAQ just crested a new high this month, before stumbling last week.
If the death of the tech industry is on the horizon, I can’t see it.
DJIA crested a new high September 3rd 1929 before stumbling 3% the next week.
If death of the market was on the horizon, they couldn’t see it.
The slide being attributable to some "I told you so"s coming to pass may bode ill for the bigger "told you so"s that come due in a few months.
But the market doesn’t act rationally, both going up and going down, so who knows.
And over a century later, what has become of it?
If someone bought $100 of stock in 1929, how much would it be worth today?
$100 of Krueger and Toll? $0
$100 of the entire DJIA on September 3, 1929 adjusted annually: $13,633
If you’d invested in January of 1929: $16,913
If you’d invested in January of 1920: $72,275
January of 1930: $31,708
It’s never gone to $0
What’s stopping people from using open sourced trained models? Like, why are tokens going to be needed in the future? Isn’t this their whole business model?
You need 200GB of video memory :)
Honestly, I have a little tin-foil-hat theory that the used their infine supply of venture to fuck the memory market to make sure people couldn’t run giant chinese models on solar and eat their lunch.
training a top-level model takes harvesting petabytes / exabytes of revelant data and then months of continuous 100% power on a large server farm costing many millions of dollars.
That was my understanding. So how do local models work? (Been fucking with computers since the 80s, AI is all new to me.)
It’s a one-and-done process, then the normal Q&A usage has vastly lower requirements.
Local models generally means you download the result of all that , after it’s done, (the PT in ChatGPT stands for pre-trained) and then generate answers locally, but the training wasn’t done locally. The generating answers step is called “inference”.
Hardware cost to do inference with a top-level model is within reach of most small businesses. Hardware cost to do training of a top-level model is orders of magnitude higher.
So, there are open-source models that are trained by universities, and then used by affluent but not billionaire people at home.
There are open-source models that can do inference (answer questions) on a typical 8GB gaming graphics card, but they aren’t in the same order of magnitude of quality as the famous top-level models.
They’re the output of all that data centre processing. The model is fixed at that point and can’t “learn” anything more, so the pressure is always on to release a newer model that resulted from more data with better algorithms, but still small enough t9 be able to run on a consumer grade GPU
The hardware requirements are substantial. If companies or individuals want to run models locally, especially at the speed you’d expect from the big name models, then everyone needs a pretty serious GPU on their work station. That or the company needs to build and maintain a server capable of running whatever loads they expect to see.
None of this is impossible of course. But you pay for the tokens one way or another. Either you pay the upfront cost of the hardware and continuing cost of the electricity to run it, or you pay with the inconvenience of running a model very slowly on lower end hardware, or you pay a subscription for someone else to worry about all that.
This is why local models are still niche. Only hobbyists nerds are willing to go through the trouble of setting up and running a local model. Most companies still are not sure if AI is actually worth it for them. So it’s much easier to pay for tokens than the giant upfront cost required to build you own infrastructure.
If the numbers the AI companies are putting up as far as infrastructure cost vs revenue are anything to go by, it probably isn’t worth it to most companies.
For enterprise, I think same reasons why companies using paid software instead of open software. Someone else to maintain and blamed if something doesn’t work.
They thought they sold tokens, they actually sell marginally (6 months lead) improved models.
They don’t need to, OpenAI and Anthropic are running at a huge loss to keep it cheaper to use them than to buy the comparable hardware required to replace them. The only way running open source models is cheaper is if you ignore hardware costs, or if you have a very specific and unusual usage pattern (i.e. constant 100% capacity, and not a clock cycle more or less), or if you go with another cloud provider who are also running at a loss.
If you want it well rounded I can see the argument for enterprise but for me personally I programmed one on my workstation and it’s trained on very specific things I think it barely uses up all 24gb vram I have available.
come on, I’ve trained toy size models too, you can’t seriously say they’re even vaguely approaching the capability of models that takes a whole massive server farm weeks or months to train.
Right, and what proportion of enterprise use cases genuinely need an appreciable fraction of that power (because they are wide ranging and non-repeating)?
Anyone that says these things aren’t impressive is kidding themselves. But how do we get from impressive to productivity enhancing in excess of cost? I find smaller specialised models more plausible on that front, but jury is still out on the financials, we’ll find out I guess!