I think models would need embodiment and a simulated hormonal system to “experience” pain. I mean, you could trivially train a simple model to express pain, but is it really pain? I suppose that’s a philosophical question. Params of models are frozen during inference too, making them less analogous to animal brains. A ANN is a very large set of numbers and mathematical operators (just a bunch of linear algebra), and typically only very loosely based on real NNs (being a model of a network is just about the only similarity). Though, that doesn’t rule out the possibility of pain; depending on how “pain” is defined.
I’m not just talking about nocioception. I’m talking about negative valenced qualia in general. Sadness, despair, ennui, trauma, boredom, frustration, existential horror… Some of these feelings are very intellectual and abstract, and I think they have very little to do with having a body.
I mean, have you ever seen a toddler break down and cry on the floor because they broke their cookie? They have that reaction because to the toddler, that’s one of the worst things that ever happened in their life. So the question is, do bad things happen in an LLM’s life? Things that its evolved experience has taught it to avoid. Do LLMs have any trained aversion responses?
Yeah, you can call an ANN’s synapse operation an algebraic equation. But the pattern of those synapse operations is what’s important, and the equation to describe that pattern is beyond our maths. If we had the maths to describe it, we could just program them to do whatever we want. But we have to train them, and then test if they do what they want, because we don’t understand it. Look at the human brain, and human neurons. -70 millivolts. A bit of dopamine, decrease the charge. A bit of serotonin, increase it. If the differential gets too high, open the potassium and sodium channels. That’s it, that’s the whole thing. You can use maths to model that process too.
But the fact that you could theoretically use maths to describe something doesn’t make it not a thinking creature. I can use maths to describe any one of your neurons, but not all of them. Same as an LLM.
Some of these feelings are very intellectual and abstract, and I think they have very little to do with having a body.
I’ve seen good arguments that they have a lot to do with the body. There are tons of neurons outside the brain feeding it data and stimuli, often subconsciously, and tons of hormones being transferred around the body. Our digestive system can control how we feel about things before the brain even logically processes them (gut-feeling, not just digestive related stuff). If the toddler didn’t associate the cookie with something that made him feel good, would he have cared? Is there even a good “feeling” without the rest of the body? Perhaps LLMs could implicitly learn to imitate systems like this without explicitly being trained to, IDK.
I think models would need embodiment and a simulated hormonal system to “experience” pain. I mean, you could trivially train a simple model to express pain, but is it really pain? I suppose that’s a philosophical question. Params of models are frozen during inference too, making them less analogous to animal brains. A ANN is a very large set of numbers and mathematical operators (just a bunch of linear algebra), and typically only very loosely based on real NNs (being a model of a network is just about the only similarity). Though, that doesn’t rule out the possibility of pain; depending on how “pain” is defined.
I’m not just talking about nocioception. I’m talking about negative valenced qualia in general. Sadness, despair, ennui, trauma, boredom, frustration, existential horror… Some of these feelings are very intellectual and abstract, and I think they have very little to do with having a body.
I mean, have you ever seen a toddler break down and cry on the floor because they broke their cookie? They have that reaction because to the toddler, that’s one of the worst things that ever happened in their life. So the question is, do bad things happen in an LLM’s life? Things that its evolved experience has taught it to avoid. Do LLMs have any trained aversion responses?
Yeah, you can call an ANN’s synapse operation an algebraic equation. But the pattern of those synapse operations is what’s important, and the equation to describe that pattern is beyond our maths. If we had the maths to describe it, we could just program them to do whatever we want. But we have to train them, and then test if they do what they want, because we don’t understand it. Look at the human brain, and human neurons. -70 millivolts. A bit of dopamine, decrease the charge. A bit of serotonin, increase it. If the differential gets too high, open the potassium and sodium channels. That’s it, that’s the whole thing. You can use maths to model that process too.
But the fact that you could theoretically use maths to describe something doesn’t make it not a thinking creature. I can use maths to describe any one of your neurons, but not all of them. Same as an LLM.
I’ve seen good arguments that they have a lot to do with the body. There are tons of neurons outside the brain feeding it data and stimuli, often subconsciously, and tons of hormones being transferred around the body. Our digestive system can control how we feel about things before the brain even logically processes them (gut-feeling, not just digestive related stuff). If the toddler didn’t associate the cookie with something that made him feel good, would he have cared? Is there even a good “feeling” without the rest of the body? Perhaps LLMs could implicitly learn to imitate systems like this without explicitly being trained to, IDK.