The AI boom has turned the standard profit margin model on its head, according to Apollo Chief Economist Torsten Slok—and it’s making the industry’s growth unsustainable.
I agree with all of those things, but for the last one I see a big example in all of the people not taking the news of a demonstrable global workspace in AI, including models that have been around for years now, as anything big.
People have been insisting that AI can’t possibly actually be thinking or have any of the most vaunted aspects of the human mind and that LLMs are a dead end that won’t go anywhere for years. We want to be different and special and unique. We always have.
Many of us here went to school when it was still taught that animals weren’t really conscious and couldn’t be self-aware. Being unwilling to look at new information and honestly assess is hubris and insistence on holding to the thing you’ve been insisting, it’s not a logical analysis.
Over the years we’ve pointed to several things to insist that AI wasn’t special like us, and that it couldn’t possibly be because it lacked aspects of the human mind needed for that. Somehow it kept turning out that every thing we pointed to in ourselves ended up being shown to have a very surprisingly similar to nearly direct analogue in those modern LLMs. Last month was jaw dropping, but the bulk of the world has been so focused on the entire world going to hell it hasn’t gotten near the attention it deserved yet.
What does it mean to think? This is a open-ended philosophical question and has any number of answers.
I completely agree with your 2nd and 3rd paragraph.
There are similarities and there are clear differences. The reason why we say LLMs are different from us is because we DO have general intelligence, LLMs have not displayed such capabilities despite the intense pressure to prove so. THIS is why we say LLMs couldn’t possibly be like us. Because evidence shows they aren’t, despite the similarities.
I would be interested to hear what developments in the last month you’re referring to, would be exciting to see a breakthrough.
Global workspace. The leading theory of consciousness. It’s a thing we can’t show empirically in ourselves, but can now see and experiment on something that matches it’s description in AI. Not just the proprietary latest models, the years old models anyone can download from the internet. No one built it, it was something else that emerged somehow through training and went unnoticed.
Here’s someone trying to backpedal into saying the thing that really matters actually isn’t genuinely thinking, now it’s… living with your mistakes. That’s how far the goal posts have been kicked over the last 6 months ending in this one. Not genuinely being able to think, rational self-awareness. Just remembering when you messed up a few months ago. And it’s such a weird pivot because AI memory is a design aspect that can be changed, something external frameworks to enhance with databases already exists for, a large part limited in current design because the more you send in the more it costs to process so it’s always capped, and also… sort of unnatural.
Running AI models the way we do with “frozen weights” isn’t mandatory. It takes a lot more hardware to do it, but it’s possible to run AI in a way more like they run during it’s training. The model files themselves would be unfrozen and allowed to change as you communicate. It’s reportedly something that’s had issues with the models forgetting things even as big as the language you’re talking in, but also a thing that is sometimes used during the alignment process. And one of the major reasons the big companies don’t care to look in to it is because if you have 10,000 people communicating with an AI over the internet and all telling it to do different things and act different ways and all of those things can be learning on the level of the model files themselves instead of confined to external temporary context windows it’s going to go crazy.
Memory on on the actual model level isn’t nonexistent, it’s what the entire training process is based on. It just wouldn’t make a good consumer product for sale, so parts of the models are removed before they’re made available online for the ones that are.
I agree with all of those things, but for the last one I see a big example in all of the people not taking the news of a demonstrable global workspace in AI, including models that have been around for years now, as anything big.
People have been insisting that AI can’t possibly actually be thinking or have any of the most vaunted aspects of the human mind and that LLMs are a dead end that won’t go anywhere for years. We want to be different and special and unique. We always have.
Many of us here went to school when it was still taught that animals weren’t really conscious and couldn’t be self-aware. Being unwilling to look at new information and honestly assess is hubris and insistence on holding to the thing you’ve been insisting, it’s not a logical analysis.
Over the years we’ve pointed to several things to insist that AI wasn’t special like us, and that it couldn’t possibly be because it lacked aspects of the human mind needed for that. Somehow it kept turning out that every thing we pointed to in ourselves ended up being shown to have a very surprisingly similar to nearly direct analogue in those modern LLMs. Last month was jaw dropping, but the bulk of the world has been so focused on the entire world going to hell it hasn’t gotten near the attention it deserved yet.
What does it mean to think? This is a open-ended philosophical question and has any number of answers.
I completely agree with your 2nd and 3rd paragraph.
There are similarities and there are clear differences. The reason why we say LLMs are different from us is because we DO have general intelligence, LLMs have not displayed such capabilities despite the intense pressure to prove so. THIS is why we say LLMs couldn’t possibly be like us. Because evidence shows they aren’t, despite the similarities.
I would be interested to hear what developments in the last month you’re referring to, would be exciting to see a breakthrough.
Global workspace. The leading theory of consciousness. It’s a thing we can’t show empirically in ourselves, but can now see and experiment on something that matches it’s description in AI. Not just the proprietary latest models, the years old models anyone can download from the internet. No one built it, it was something else that emerged somehow through training and went unnoticed.
Here’s someone trying to backpedal into saying the thing that really matters actually isn’t genuinely thinking, now it’s… living with your mistakes. That’s how far the goal posts have been kicked over the last 6 months ending in this one. Not genuinely being able to think, rational self-awareness. Just remembering when you messed up a few months ago. And it’s such a weird pivot because AI memory is a design aspect that can be changed, something external frameworks to enhance with databases already exists for, a large part limited in current design because the more you send in the more it costs to process so it’s always capped, and also… sort of unnatural.
Running AI models the way we do with “frozen weights” isn’t mandatory. It takes a lot more hardware to do it, but it’s possible to run AI in a way more like they run during it’s training. The model files themselves would be unfrozen and allowed to change as you communicate. It’s reportedly something that’s had issues with the models forgetting things even as big as the language you’re talking in, but also a thing that is sometimes used during the alignment process. And one of the major reasons the big companies don’t care to look in to it is because if you have 10,000 people communicating with an AI over the internet and all telling it to do different things and act different ways and all of those things can be learning on the level of the model files themselves instead of confined to external temporary context windows it’s going to go crazy.
Memory on on the actual model level isn’t nonexistent, it’s what the entire training process is based on. It just wouldn’t make a good consumer product for sale, so parts of the models are removed before they’re made available online for the ones that are.