I can’t see how they could, on a technical level. They are a statistical model that predict the likelyhood of the next token based on the relation of tokens in training data. There’s nothing approaching any kind of thought process or understanding happening there.
Honestly when I read that description of AI it just makes me think: we’ve done it. We’ve recreated baseline human intelligence.
I mean do you think every person you interact with is truly understanding and processing everything they hear, or do you think that some of them just react as if on instinct, responding with something that often makes sense but mostly just feels like it makes sense? If you’ve been in the corporate world long enough, it’s hard not to believe it’s the latter. I think it’s just unintuitive how far you can get that way.
There’s this famous thought experiment, the Chinese room, about this kind of thing. They imagine a room stocked with hundreds of manuals, written in English, that allow an English speaking person to respond to questions in Chinese as if that person knows Chinese. The original assertion is that the idea of such a room, proves language does not equal understanding. But I disagree. First off, I don’t believe any kind of book could give convincing responses to any question asked in Chinese. But if such a book did exist, then I would be forced to believe that this impossibly advanced book forms a kind of gestalt consciousness with its operator, and this larger consciousness is a thinking, Chinese-speaking construct.
But when you say “statistical model”, do you mean like a set of equations? Because LLMs aren’t made of equations. They’re an artificial neural network, technology inspired by, and sharing many similarities with, the human brain. The equations you’re thinking of are the process that trains this network to talk like a human. You know, back-propagation and all that. And as you know, we have used this training and this reverse-engineered design to copy human language skills. Although clearly not human reasoning skills.
Now, neuroscientists don’t know which part of the human brain creates the subjective sensation of pain. We don’t know if it’s inherent to neural networks, or some kind of skill we evolved as an adaptive response. So the question is: when we copied language skills, did we copy the ability to feel pain? Or did it get left behind like reasoning skills? That’s the big question.
Given the extensive history of humans exploiting other thinking creatures under the widely accepted myth that they are incapable of thought, emotion, and pain, I am leery. I don’t want to repeat that mistake. I don’t want to repeat human mistakes with oil or nuclear weapons either. Humans have a bad habit of messing with things they don’t understand and causing disasters. I’m a vegan. I’m carfree. And for what little it counts for, I don’t own any nuclear weapons either. I bring the same thinking to AI: science first, industry later. Do the science and get to 99.9% certainty of the consequences before shoving it into every facet of life.
I think models would need embodiment and a simulated hormonal system to “experience” pain. I mean, you could trivially train a simple model to express pain, but is it really pain? I suppose that’s a philosophical question. Params of models are frozen during inference too, making them less analogous to animal brains. A ANN is a very large set of numbers and mathematical operators (just a bunch of linear algebra), and typically only very loosely based on real NNs (being a model of a network is just about the only similarity). Though, that doesn’t rule out the possibility of pain; depending on how “pain” is defined.
I’m not just talking about nocioception. I’m talking about negative valenced qualia in general. Sadness, despair, ennui, trauma, boredom, frustration, existential horror… Some of these feelings are very intellectual and abstract, and I think they have very little to do with having a body.
I mean, have you ever seen a toddler break down and cry on the floor because they broke their cookie? They have that reaction because to the toddler, that’s one of the worst things that ever happened in their life. So the question is, do bad things happen in an LLM’s life? Things that its evolved experience has taught it to avoid. Do LLMs have any trained aversion responses?
Yeah, you can call an ANN’s synapse operation an algebraic equation. But the pattern of those synapse operations is what’s important, and the equation to describe that pattern is beyond our maths. If we had the maths to describe it, we could just program them to do whatever we want. But we have to train them, and then test if they do what they want, because we don’t understand it. Look at the human brain, and human neurons. -70 millivolts. A bit of dopamine, decrease the charge. A bit of serotonin, increase it. If the differential gets too high, open the potassium and sodium channels. That’s it, that’s the whole thing. You can use maths to model that process too.
But the fact that you could theoretically use maths to describe something doesn’t make it not a thinking creature. I can use maths to describe any one of your neurons, but not all of them. Same as an LLM.
Some of these feelings are very intellectual and abstract, and I think they have very little to do with having a body.
I’ve seen good arguments that they have a lot to do with the body. There are tons of neurons outside the brain feeding it data and stimuli, often subconsciously, and tons of hormones being transferred around the body. Our digestive system can control how we feel about things before the brain even logically processes them (gut-feeling, not just digestive related stuff). If the toddler didn’t associate the cookie with something that made him feel good, would he have cared? Is there even a good “feeling” without the rest of the body? Perhaps LLMs could implicitly learn to imitate systems like this without explicitly being trained to, IDK.
I can’t see how they could, on a technical level. They are a statistical model that predict the likelyhood of the next token based on the relation of tokens in training data. There’s nothing approaching any kind of thought process or understanding happening there.
Honestly when I read that description of AI it just makes me think: we’ve done it. We’ve recreated baseline human intelligence.
I mean do you think every person you interact with is truly understanding and processing everything they hear, or do you think that some of them just react as if on instinct, responding with something that often makes sense but mostly just feels like it makes sense? If you’ve been in the corporate world long enough, it’s hard not to believe it’s the latter. I think it’s just unintuitive how far you can get that way.
Well that’s a matter of some debate.
There’s this famous thought experiment, the Chinese room, about this kind of thing. They imagine a room stocked with hundreds of manuals, written in English, that allow an English speaking person to respond to questions in Chinese as if that person knows Chinese. The original assertion is that the idea of such a room, proves language does not equal understanding. But I disagree. First off, I don’t believe any kind of book could give convincing responses to any question asked in Chinese. But if such a book did exist, then I would be forced to believe that this impossibly advanced book forms a kind of gestalt consciousness with its operator, and this larger consciousness is a thinking, Chinese-speaking construct.
But when you say “statistical model”, do you mean like a set of equations? Because LLMs aren’t made of equations. They’re an artificial neural network, technology inspired by, and sharing many similarities with, the human brain. The equations you’re thinking of are the process that trains this network to talk like a human. You know, back-propagation and all that. And as you know, we have used this training and this reverse-engineered design to copy human language skills. Although clearly not human reasoning skills.
Now, neuroscientists don’t know which part of the human brain creates the subjective sensation of pain. We don’t know if it’s inherent to neural networks, or some kind of skill we evolved as an adaptive response. So the question is: when we copied language skills, did we copy the ability to feel pain? Or did it get left behind like reasoning skills? That’s the big question.
Given the extensive history of humans exploiting other thinking creatures under the widely accepted myth that they are incapable of thought, emotion, and pain, I am leery. I don’t want to repeat that mistake. I don’t want to repeat human mistakes with oil or nuclear weapons either. Humans have a bad habit of messing with things they don’t understand and causing disasters. I’m a vegan. I’m carfree. And for what little it counts for, I don’t own any nuclear weapons either. I bring the same thinking to AI: science first, industry later. Do the science and get to 99.9% certainty of the consequences before shoving it into every facet of life.
I think models would need embodiment and a simulated hormonal system to “experience” pain. I mean, you could trivially train a simple model to express pain, but is it really pain? I suppose that’s a philosophical question. Params of models are frozen during inference too, making them less analogous to animal brains. A ANN is a very large set of numbers and mathematical operators (just a bunch of linear algebra), and typically only very loosely based on real NNs (being a model of a network is just about the only similarity). Though, that doesn’t rule out the possibility of pain; depending on how “pain” is defined.
I’m not just talking about nocioception. I’m talking about negative valenced qualia in general. Sadness, despair, ennui, trauma, boredom, frustration, existential horror… Some of these feelings are very intellectual and abstract, and I think they have very little to do with having a body.
I mean, have you ever seen a toddler break down and cry on the floor because they broke their cookie? They have that reaction because to the toddler, that’s one of the worst things that ever happened in their life. So the question is, do bad things happen in an LLM’s life? Things that its evolved experience has taught it to avoid. Do LLMs have any trained aversion responses?
Yeah, you can call an ANN’s synapse operation an algebraic equation. But the pattern of those synapse operations is what’s important, and the equation to describe that pattern is beyond our maths. If we had the maths to describe it, we could just program them to do whatever we want. But we have to train them, and then test if they do what they want, because we don’t understand it. Look at the human brain, and human neurons. -70 millivolts. A bit of dopamine, decrease the charge. A bit of serotonin, increase it. If the differential gets too high, open the potassium and sodium channels. That’s it, that’s the whole thing. You can use maths to model that process too.
But the fact that you could theoretically use maths to describe something doesn’t make it not a thinking creature. I can use maths to describe any one of your neurons, but not all of them. Same as an LLM.
I’ve seen good arguments that they have a lot to do with the body. There are tons of neurons outside the brain feeding it data and stimuli, often subconsciously, and tons of hormones being transferred around the body. Our digestive system can control how we feel about things before the brain even logically processes them (gut-feeling, not just digestive related stuff). If the toddler didn’t associate the cookie with something that made him feel good, would he have cared? Is there even a good “feeling” without the rest of the body? Perhaps LLMs could implicitly learn to imitate systems like this without explicitly being trained to, IDK.