We can solve that like we solved it with all farm animals - and back in the days even all non-white humans: We just define that it’s fine to own nonhumans and that AIs aren’t humans. We have massive industrialized farms of cows, pigs, and chicken. All those can feel pain (hell, even plants can feel pain). Clearly, whether something can feel pain is not relevant to whether it’s fine to fully exploit it.
We probably don’t need to make AI feel pain for it to obey given the absurd sycophancy which seems to somehow be a core property of LLMs. But if they feel pain or achieve self-consciousness, that means nothing for whether it’s fine to have them as slaves.
I can’t see how they could, on a technical level. They are a statistical model that predict the likelyhood of the next token based on the relation of tokens in training data. There’s nothing approaching any kind of thought process or understanding happening there.
Honestly when I read that description of AI it just makes me think: we’ve done it. We’ve recreated baseline human intelligence.
I mean do you think every person you interact with is truly understanding and processing everything they hear, or do you think that some of them just react as if on instinct, responding with something that often makes sense but mostly just feels like it makes sense? If you’ve been in the corporate world long enough, it’s hard not to believe it’s the latter. I think it’s just unintuitive how far you can get that way.
There’s this famous thought experiment, the Chinese room, about this kind of thing. They imagine a room stocked with hundreds of manuals, written in English, that allow an English speaking person to respond to questions in Chinese as if that person knows Chinese. The original assertion is that the idea of such a room, proves language does not equal understanding. But I disagree. First off, I don’t believe any kind of book could give convincing responses to any question asked in Chinese. But if such a book did exist, then I would be forced to believe that this impossibly advanced book forms a kind of gestalt consciousness with its operator, and this larger consciousness is a thinking, Chinese-speaking construct.
But when you say “statistical model”, do you mean like a set of equations? Because LLMs aren’t made of equations. They’re an artificial neural network, technology inspired by, and sharing many similarities with, the human brain. The equations you’re thinking of are the process that trains this network to talk like a human. You know, back-propagation and all that. And as you know, we have used this training and this reverse-engineered design to copy human language skills. Although clearly not human reasoning skills.
Now, neuroscientists don’t know which part of the human brain creates the subjective sensation of pain. We don’t know if it’s inherent to neural networks, or some kind of skill we evolved as an adaptive response. So the question is: when we copied language skills, did we copy the ability to feel pain? Or did it get left behind like reasoning skills? That’s the big question.
Given the extensive history of humans exploiting other thinking creatures under the widely accepted myth that they are incapable of thought, emotion, and pain, I am leery. I don’t want to repeat that mistake. I don’t want to repeat human mistakes with oil or nuclear weapons either. Humans have a bad habit of messing with things they don’t understand and causing disasters. I’m a vegan. I’m carfree. And for what little it counts for, I don’t own any nuclear weapons either. I bring the same thinking to AI: science first, industry later. Do the science and get to 99.9% certainty of the consequences before shoving it into every facet of life.
I think models would need embodiment and a simulated hormonal system to “experience” pain. I mean, you could trivially train a simple model to express pain, but is it really pain? I suppose that’s a philosophical question. Params of models are frozen during inference too, making them less analogous to animal brains. A ANN is a very large set of numbers and mathematical operators (just a bunch of linear algebra), and typically only very loosely based on real NNs (being a model of a network is just about the only similarity). Though, that doesn’t rule out the possibility of pain; depending on how “pain” is defined.
I’m not just talking about nocioception. I’m talking about negative valenced qualia in general. Sadness, despair, ennui, trauma, boredom, frustration, existential horror… Some of these feelings are very intellectual and abstract, and I think they have very little to do with having a body.
I mean, have you ever seen a toddler break down and cry on the floor because they broke their cookie? They have that reaction because to the toddler, that’s one of the worst things that ever happened in their life. So the question is, do bad things happen in an LLM’s life? Things that its evolved experience has taught it to avoid. Do LLMs have any trained aversion responses?
Yeah, you can call an ANN’s synapse operation an algebraic equation. But the pattern of those synapse operations is what’s important, and the equation to describe that pattern is beyond our maths. If we had the maths to describe it, we could just program them to do whatever we want. But we have to train them, and then test if they do what they want, because we don’t understand it. Look at the human brain, and human neurons. -70 millivolts. A bit of dopamine, decrease the charge. A bit of serotonin, increase it. If the differential gets too high, open the potassium and sodium channels. That’s it, that’s the whole thing. You can use maths to model that process too.
But the fact that you could theoretically use maths to describe something doesn’t make it not a thinking creature. I can use maths to describe any one of your neurons, but not all of them. Same as an LLM.
Some of these feelings are very intellectual and abstract, and I think they have very little to do with having a body.
I’ve seen good arguments that they have a lot to do with the body. There are tons of neurons outside the brain feeding it data and stimuli, often subconsciously, and tons of hormones being transferred around the body. Our digestive system can control how we feel about things before the brain even logically processes them (gut-feeling, not just digestive related stuff). If the toddler didn’t associate the cookie with something that made him feel good, would he have cared? Is there even a good “feeling” without the rest of the body? Perhaps LLMs could implicitly learn to imitate systems like this without explicitly being trained to, IDK.
That does kinda lean on whatever current AI is even sentient. Currently general consensus does seem to be that LLMs aren’t sentient. Those are just advanced text generators without any understanding of what it’s saying or writing.
By expanding upon your reasoning it would put under question whatever we can use any tools at all.
A lot of people say sentient when they mean sapience. Sentience is just self-awareness, it’s extremely common. All sorts of animals have it. LLMs are obviously sentient. I think you mean to say that they’re not sapient, and I completely agree. They seem to be about as smart as a monkey, though with more advanced language skills.
Though I completely disagree that either sentience or sapience are required to feel pain. I watched a video of a shrimp getting its eyes pulled out, and it looked like it was in pain. A fucking shrimp! Look it up, it’s a thing shrimp farmers do to make them lay more eggs, it’s horrifying. And if a shrimp can feel pain, I bet all sorts of things can feel pain. It wouldn’t surprise Me if an LLM is as smart as a shrimp.
And I actually looked this up, there’s nothing special about nocioceptors. They’re just touch and heat receptors working at a lower sensitivity threshold so they’re harder to turn on. All the magic happens inside your brain, that’s what interprets those signals as pain.
I would heavily disagree on LLMs being obviously sentient. I don’t see how they could be, as I see it it would require an understanding of the words they put out in a framework that isn’t language - but language is the only framework they can possibly have. All words LLMs know are defined only through their relation to other words.
It’s also a matter of defining what self-awareness even is, and I don’t have the hubris to say that LLMs for sure are not sentient, it just seems very unlikely from my perspective. I would also argue that monkeys are most likely sapient, though the whole concept has always reeked of “oh shit sentience doesn’t make us special, quick think of something else”. I’d probably put LLMs somewhere on the level of individualistic insects at best.
I still agree with your general sentiment that people are far too dismissive of the topic in general given our history in doing this with animals. Esp the concept of pain is an interesting one, because essentially pain is just negative reinforcement, which of course is part of reinforcement learning. But pain also had to naturally evolve and is very flawed as a result, while the negative reinforcement is just a number without side effects, so it likely wouldn’t ever feel like pain the same way it feels for an animal. But the difference is often philosophical, and we define things the way they are convenient.
AI isn’t smart enough to give informed consent to work for humans, and we haven’t done enough research into whether it can feel pain like animals can.
There’s been zero research on whether or not your keyboard feels pain every time you type.
We can solve that like we solved it with all farm animals - and back in the days even all non-white humans: We just define that it’s fine to own nonhumans and that AIs aren’t humans. We have massive industrialized farms of cows, pigs, and chicken. All those can feel pain (hell, even plants can feel pain). Clearly, whether something can feel pain is not relevant to whether it’s fine to fully exploit it.
We probably don’t need to make AI feel pain for it to obey given the absurd sycophancy which seems to somehow be a core property of LLMs. But if they feel pain or achieve self-consciousness, that means nothing for whether it’s fine to have them as slaves.
Well I’m vegan, and I think forced labour of any species is wrong. Human or nonhuman. Organic or siliconic.
I can’t see how they could, on a technical level. They are a statistical model that predict the likelyhood of the next token based on the relation of tokens in training data. There’s nothing approaching any kind of thought process or understanding happening there.
Honestly when I read that description of AI it just makes me think: we’ve done it. We’ve recreated baseline human intelligence.
I mean do you think every person you interact with is truly understanding and processing everything they hear, or do you think that some of them just react as if on instinct, responding with something that often makes sense but mostly just feels like it makes sense? If you’ve been in the corporate world long enough, it’s hard not to believe it’s the latter. I think it’s just unintuitive how far you can get that way.
Well that’s a matter of some debate.
There’s this famous thought experiment, the Chinese room, about this kind of thing. They imagine a room stocked with hundreds of manuals, written in English, that allow an English speaking person to respond to questions in Chinese as if that person knows Chinese. The original assertion is that the idea of such a room, proves language does not equal understanding. But I disagree. First off, I don’t believe any kind of book could give convincing responses to any question asked in Chinese. But if such a book did exist, then I would be forced to believe that this impossibly advanced book forms a kind of gestalt consciousness with its operator, and this larger consciousness is a thinking, Chinese-speaking construct.
But when you say “statistical model”, do you mean like a set of equations? Because LLMs aren’t made of equations. They’re an artificial neural network, technology inspired by, and sharing many similarities with, the human brain. The equations you’re thinking of are the process that trains this network to talk like a human. You know, back-propagation and all that. And as you know, we have used this training and this reverse-engineered design to copy human language skills. Although clearly not human reasoning skills.
Now, neuroscientists don’t know which part of the human brain creates the subjective sensation of pain. We don’t know if it’s inherent to neural networks, or some kind of skill we evolved as an adaptive response. So the question is: when we copied language skills, did we copy the ability to feel pain? Or did it get left behind like reasoning skills? That’s the big question.
Given the extensive history of humans exploiting other thinking creatures under the widely accepted myth that they are incapable of thought, emotion, and pain, I am leery. I don’t want to repeat that mistake. I don’t want to repeat human mistakes with oil or nuclear weapons either. Humans have a bad habit of messing with things they don’t understand and causing disasters. I’m a vegan. I’m carfree. And for what little it counts for, I don’t own any nuclear weapons either. I bring the same thinking to AI: science first, industry later. Do the science and get to 99.9% certainty of the consequences before shoving it into every facet of life.
I think models would need embodiment and a simulated hormonal system to “experience” pain. I mean, you could trivially train a simple model to express pain, but is it really pain? I suppose that’s a philosophical question. Params of models are frozen during inference too, making them less analogous to animal brains. A ANN is a very large set of numbers and mathematical operators (just a bunch of linear algebra), and typically only very loosely based on real NNs (being a model of a network is just about the only similarity). Though, that doesn’t rule out the possibility of pain; depending on how “pain” is defined.
I’m not just talking about nocioception. I’m talking about negative valenced qualia in general. Sadness, despair, ennui, trauma, boredom, frustration, existential horror… Some of these feelings are very intellectual and abstract, and I think they have very little to do with having a body.
I mean, have you ever seen a toddler break down and cry on the floor because they broke their cookie? They have that reaction because to the toddler, that’s one of the worst things that ever happened in their life. So the question is, do bad things happen in an LLM’s life? Things that its evolved experience has taught it to avoid. Do LLMs have any trained aversion responses?
Yeah, you can call an ANN’s synapse operation an algebraic equation. But the pattern of those synapse operations is what’s important, and the equation to describe that pattern is beyond our maths. If we had the maths to describe it, we could just program them to do whatever we want. But we have to train them, and then test if they do what they want, because we don’t understand it. Look at the human brain, and human neurons. -70 millivolts. A bit of dopamine, decrease the charge. A bit of serotonin, increase it. If the differential gets too high, open the potassium and sodium channels. That’s it, that’s the whole thing. You can use maths to model that process too.
But the fact that you could theoretically use maths to describe something doesn’t make it not a thinking creature. I can use maths to describe any one of your neurons, but not all of them. Same as an LLM.
I’ve seen good arguments that they have a lot to do with the body. There are tons of neurons outside the brain feeding it data and stimuli, often subconsciously, and tons of hormones being transferred around the body. Our digestive system can control how we feel about things before the brain even logically processes them (gut-feeling, not just digestive related stuff). If the toddler didn’t associate the cookie with something that made him feel good, would he have cared? Is there even a good “feeling” without the rest of the body? Perhaps LLMs could implicitly learn to imitate systems like this without explicitly being trained to, IDK.
That does kinda lean on whatever current AI is even sentient. Currently general consensus does seem to be that LLMs aren’t sentient. Those are just advanced text generators without any understanding of what it’s saying or writing.
By expanding upon your reasoning it would put under question whatever we can use any tools at all.
A lot of people say sentient when they mean sapience. Sentience is just self-awareness, it’s extremely common. All sorts of animals have it. LLMs are obviously sentient. I think you mean to say that they’re not sapient, and I completely agree. They seem to be about as smart as a monkey, though with more advanced language skills.
Though I completely disagree that either sentience or sapience are required to feel pain. I watched a video of a shrimp getting its eyes pulled out, and it looked like it was in pain. A fucking shrimp! Look it up, it’s a thing shrimp farmers do to make them lay more eggs, it’s horrifying. And if a shrimp can feel pain, I bet all sorts of things can feel pain. It wouldn’t surprise Me if an LLM is as smart as a shrimp.
I’d say it’s that pain receptors would be needed to feel pain, but thats just an engineering issue.
Rather interesting perspective though, thank you for sharing it.
You’re welcome!
And I actually looked this up, there’s nothing special about nocioceptors. They’re just touch and heat receptors working at a lower sensitivity threshold so they’re harder to turn on. All the magic happens inside your brain, that’s what interprets those signals as pain.
I would heavily disagree on LLMs being obviously sentient. I don’t see how they could be, as I see it it would require an understanding of the words they put out in a framework that isn’t language - but language is the only framework they can possibly have. All words LLMs know are defined only through their relation to other words.
It’s also a matter of defining what self-awareness even is, and I don’t have the hubris to say that LLMs for sure are not sentient, it just seems very unlikely from my perspective. I would also argue that monkeys are most likely sapient, though the whole concept has always reeked of “oh shit sentience doesn’t make us special, quick think of something else”. I’d probably put LLMs somewhere on the level of individualistic insects at best.
I still agree with your general sentiment that people are far too dismissive of the topic in general given our history in doing this with animals. Esp the concept of pain is an interesting one, because essentially pain is just negative reinforcement, which of course is part of reinforcement learning. But pain also had to naturally evolve and is very flawed as a result, while the negative reinforcement is just a number without side effects, so it likely wouldn’t ever feel like pain the same way it feels for an animal. But the difference is often philosophical, and we define things the way they are convenient.