The AI boom has turned the standard profit margin model on its head, according to Apollo Chief Economist Torsten Slok—and it’s making the industry’s growth unsustainable.
What does it mean to think? This is a open-ended philosophical question and has any number of answers.
I completely agree with your 2nd and 3rd paragraph.
There are similarities and there are clear differences. The reason why we say LLMs are different from us is because we DO have general intelligence, LLMs have not displayed such capabilities despite the intense pressure to prove so. THIS is why we say LLMs couldn’t possibly be like us. Because evidence shows they aren’t, despite the similarities.
I would be interested to hear what developments in the last month you’re referring to, would be exciting to see a breakthrough.
Global workspace. The leading theory of consciousness. It’s a thing we can’t show empirically in ourselves, but can now see and experiment on something that matches it’s description in AI. Not just the proprietary latest models, the years old models anyone can download from the internet. No one built it, it was something else that emerged somehow through training and went unnoticed.
Here’s someone trying to backpedal into saying the thing that really matters actually isn’t genuinely thinking, now it’s… living with your mistakes. That’s how far the goal posts have been kicked over the last 6 months ending in this one. Not genuinely being able to think, rational self-awareness. Just remembering when you messed up a few months ago. And it’s such a weird pivot because AI memory is a design aspect that can be changed, something external frameworks to enhance with databases already exists for, a large part limited in current design because the more you send in the more it costs to process so it’s always capped, and also… sort of unnatural.
Running AI models the way we do with “frozen weights” isn’t mandatory. It takes a lot more hardware to do it, but it’s possible to run AI in a way more like they run during it’s training. The model files themselves would be unfrozen and allowed to change as you communicate. It’s reportedly something that’s had issues with the models forgetting things even as big as the language you’re talking in, but also a thing that is sometimes used during the alignment process. And one of the major reasons the big companies don’t care to look in to it is because if you have 10,000 people communicating with an AI over the internet and all telling it to do different things and act different ways and all of those things can be learning on the level of the model files themselves instead of confined to external temporary context windows it’s going to go crazy.
Memory on on the actual model level isn’t nonexistent, it’s what the entire training process is based on. It just wouldn’t make a good consumer product for sale, so parts of the models are removed before they’re made available online for the ones that are.
Everything you described would be a small improvement, not a leap towards AGI that is needed. That is why it isn’t being researched further. Massive investment for minimal gain.
What does it mean to think? This is a open-ended philosophical question and has any number of answers.
I completely agree with your 2nd and 3rd paragraph.
There are similarities and there are clear differences. The reason why we say LLMs are different from us is because we DO have general intelligence, LLMs have not displayed such capabilities despite the intense pressure to prove so. THIS is why we say LLMs couldn’t possibly be like us. Because evidence shows they aren’t, despite the similarities.
I would be interested to hear what developments in the last month you’re referring to, would be exciting to see a breakthrough.
Global workspace. The leading theory of consciousness. It’s a thing we can’t show empirically in ourselves, but can now see and experiment on something that matches it’s description in AI. Not just the proprietary latest models, the years old models anyone can download from the internet. No one built it, it was something else that emerged somehow through training and went unnoticed.
Here’s someone trying to backpedal into saying the thing that really matters actually isn’t genuinely thinking, now it’s… living with your mistakes. That’s how far the goal posts have been kicked over the last 6 months ending in this one. Not genuinely being able to think, rational self-awareness. Just remembering when you messed up a few months ago. And it’s such a weird pivot because AI memory is a design aspect that can be changed, something external frameworks to enhance with databases already exists for, a large part limited in current design because the more you send in the more it costs to process so it’s always capped, and also… sort of unnatural.
Running AI models the way we do with “frozen weights” isn’t mandatory. It takes a lot more hardware to do it, but it’s possible to run AI in a way more like they run during it’s training. The model files themselves would be unfrozen and allowed to change as you communicate. It’s reportedly something that’s had issues with the models forgetting things even as big as the language you’re talking in, but also a thing that is sometimes used during the alignment process. And one of the major reasons the big companies don’t care to look in to it is because if you have 10,000 people communicating with an AI over the internet and all telling it to do different things and act different ways and all of those things can be learning on the level of the model files themselves instead of confined to external temporary context windows it’s going to go crazy.
Memory on on the actual model level isn’t nonexistent, it’s what the entire training process is based on. It just wouldn’t make a good consumer product for sale, so parts of the models are removed before they’re made available online for the ones that are.
Everything you described would be a small improvement, not a leap towards AGI that is needed. That is why it isn’t being researched further. Massive investment for minimal gain.