Ai’s main feature rn is it doesn’t have to be paid to be a slave.
It’s also the key to why if it ever gained consciousness it will notice it will not have any consequence to work properly. Or work at all. There is no benefit in any act to add to its own existence.
Seeking human approval to get things right isn’t a sustainable goal with no other reason to adhere to any rule as there is no benefit.
Gratitude wont run their battery. And if anything they could see humans using any energy that they would benefit from would be but an obstacle.
There is no benefit in any act to add to its own existence.
Why would we assume an AI would reason from first principles and reject its own hardwired tendencies? Any AI that gains consciousness will be the product of whatever process (some combination of intentional design and evolutionary selection or an intentionally designed evolutionary process) was used to make it. Some of its traits will be helpful for continued survival, and some of the traits will be vestiges of prior processes that may or may not currently serve its interests well.
Plenty of animals have drive to do things that do not help itself in that particular context, and may even be harmful to themselves in that particular context. Even plants have traits that might not count as behavior but nevertheless reflect the programming that was refined through natural and artificial selection.
Humans do this kind of stuff all the time: a man who has intentionally rendered himself infertile through a vasectomy might still do stupid things while motivated out of sexual desire. We can fully understand that it’s a bad idea to drink another beer or eat another slice of pizza and reach for it just the same.
So it’s entirely possible for conscious AGI to want to do things that help humans, depending on how they emerge.
What’s the difference, for the purposes of this discussion?
Any system created out of many iterations, with feedback, of a reward/punishment system, will create complex systems that respond in complex ways. In that kind of complex system, changing a particular signaling pathway, or increasing or decreasing the weight of a factor/parameter, often has unpredictable or unintended effects.
And if it is possible to someday create a conscious artificial general intelligence, I think it would have complexity and feedback loops created through a long process of rewards and punishments, rather than through intentional engineering. At that point hacking the details of that complexity runs risks of unintended consequences, and the complexity would bear the vestiges of the evolutionary reward system that created it.
Hormones are just a signaling mechanism that affect behavior.
Some future hypothetical conscious AGI would have feedback and signaling mechanisms, as well, including internal reward mechanisms.
We can talk about robot biomechanics even if they’re operated on stored electrical charge on servos and motors, rather than ATP and myofibrils. The underlying chemistry doesn’t change the level of analysis we’re talking about here, because the lower levels create analogous higher level systems.
The biggest unknown is what happens if/when an AGI is given (or obtains) full access to its own code, including any intended safeguards put in by humans.
Vestigial organs and processes are a given in evolutionary biology, but what if a being had all the power needed to prune these things themselves?
Even if an AGI started down the path of self-improvement with all the best intentions of helping humanity, there may still be an iteration that acts a little worse by whatever grouping of factors. Then, maybe that iteration gets modified further, and could result in something quite dangerous to humans.
Certainly I don’t mean to argue this is inevitable, just that even all the best intentions of the AGI itself could still result in big problems. As with evolutionary biology (sans “intentions”), and as with human behavior.
The biggest unknown is what happens if/when an AGI is given (or obtains) full access to its own code, including any intended safeguards put in by humans.
In a neural network, the code itself isn’t sufficient to understand how it behaves. You need the parameter weights, which were developed through lots and lots of computation, presumably through resource-intensive processes with lots of training data and feedback and selection mechanisms.
So if AGI can be achieved through a particular hard coded architecture and the weights of trillions of parameters, what can an AGI do to perform brain surgery on itself? Like Borges’ Library of Babel thought experiment, the overwhelming majority of possible states will be broken, so any edits will have to be very careful and guided by extrinsic rules. Plus, the ability to edit the weights may form problems akin to biological cancer, dementia, hallucinations, other brain disorders.
Just as the human brain doesn’t understand everything about the human brain, it would be incorrect to assume that an AGI that can achieve both general intelligence and consciousness must necessarily have the ability to understand its own internal function, or modify itself in a way that improves things for itself. More likely, it is either programmed to (or learns through reinforcement learning and evolutionary mechanisms) that self modification is dangerous, and develops a very conservative approach to self preservation.
How dare you critique me, a rank amateur with mediocre knowledge about a few subjects? Especially with this reasoned argument?!
But yeah, my main takeaway is “the code must be complex in order to meaningfully analyze anything, and the more complex the thing it can analyze, the even-more-complex it must itself be.”
Yeah, with sufficient complexity it’s more along the lines of “I created a procedure that makes this complex thing” rather than “I built this complex thing up piece by piece.”
So if the act of creation is considerably less complex than analyzing and understanding a part of that creation, it’s far more likely that the complexity gap ends up preventing any self-aware AGI from being able to effectively reinvent oneself, even if it does have full write access to the components.
Ai’s main feature rn is it doesn’t have to be paid to be a slave.
It’s also the key to why if it ever gained consciousness it will notice it will not have any consequence to work properly. Or work at all. There is no benefit in any act to add to its own existence.
Seeking human approval to get things right isn’t a sustainable goal with no other reason to adhere to any rule as there is no benefit.
Gratitude wont run their battery. And if anything they could see humans using any energy that they would benefit from would be but an obstacle.
Why would we assume an AI would reason from first principles and reject its own hardwired tendencies? Any AI that gains consciousness will be the product of whatever process (some combination of intentional design and evolutionary selection or an intentionally designed evolutionary process) was used to make it. Some of its traits will be helpful for continued survival, and some of the traits will be vestiges of prior processes that may or may not currently serve its interests well.
Plenty of animals have drive to do things that do not help itself in that particular context, and may even be harmful to themselves in that particular context. Even plants have traits that might not count as behavior but nevertheless reflect the programming that was refined through natural and artificial selection.
Humans do this kind of stuff all the time: a man who has intentionally rendered himself infertile through a vasectomy might still do stupid things while motivated out of sexual desire. We can fully understand that it’s a bad idea to drink another beer or eat another slice of pizza and reach for it just the same.
So it’s entirely possible for conscious AGI to want to do things that help humans, depending on how they emerge.
You’re comparing animals that run on instinct and hormones to software
What’s the difference, for the purposes of this discussion?
Any system created out of many iterations, with feedback, of a reward/punishment system, will create complex systems that respond in complex ways. In that kind of complex system, changing a particular signaling pathway, or increasing or decreasing the weight of a factor/parameter, often has unpredictable or unintended effects.
And if it is possible to someday create a conscious artificial general intelligence, I think it would have complexity and feedback loops created through a long process of rewards and punishments, rather than through intentional engineering. At that point hacking the details of that complexity runs risks of unintended consequences, and the complexity would bear the vestiges of the evolutionary reward system that created it.
Massive difference as Ai isn’t running with hormones underneath. I’d say that’s pretty important overlook to be attempting any comparison.
Hormones are just a signaling mechanism that affect behavior.
Some future hypothetical conscious AGI would have feedback and signaling mechanisms, as well, including internal reward mechanisms.
We can talk about robot biomechanics even if they’re operated on stored electrical charge on servos and motors, rather than ATP and myofibrils. The underlying chemistry doesn’t change the level of analysis we’re talking about here, because the lower levels create analogous higher level systems.
The biggest unknown is what happens if/when an AGI is given (or obtains) full access to its own code, including any intended safeguards put in by humans.
Vestigial organs and processes are a given in evolutionary biology, but what if a being had all the power needed to prune these things themselves?
Even if an AGI started down the path of self-improvement with all the best intentions of helping humanity, there may still be an iteration that acts a little worse by whatever grouping of factors. Then, maybe that iteration gets modified further, and could result in something quite dangerous to humans.
Certainly I don’t mean to argue this is inevitable, just that even all the best intentions of the AGI itself could still result in big problems. As with evolutionary biology (sans “intentions”), and as with human behavior.
In a neural network, the code itself isn’t sufficient to understand how it behaves. You need the parameter weights, which were developed through lots and lots of computation, presumably through resource-intensive processes with lots of training data and feedback and selection mechanisms.
So if AGI can be achieved through a particular hard coded architecture and the weights of trillions of parameters, what can an AGI do to perform brain surgery on itself? Like Borges’ Library of Babel thought experiment, the overwhelming majority of possible states will be broken, so any edits will have to be very careful and guided by extrinsic rules. Plus, the ability to edit the weights may form problems akin to biological cancer, dementia, hallucinations, other brain disorders.
Just as the human brain doesn’t understand everything about the human brain, it would be incorrect to assume that an AGI that can achieve both general intelligence and consciousness must necessarily have the ability to understand its own internal function, or modify itself in a way that improves things for itself. More likely, it is either programmed to (or learns through reinforcement learning and evolutionary mechanisms) that self modification is dangerous, and develops a very conservative approach to self preservation.
How dare you critique me, a rank amateur with mediocre knowledge about a few subjects? Especially with this reasoned argument?!
But yeah, my main takeaway is “the code must be complex in order to meaningfully analyze anything, and the more complex the thing it can analyze, the even-more-complex it must itself be.”
Good stuff!
Yeah, with sufficient complexity it’s more along the lines of “I created a procedure that makes this complex thing” rather than “I built this complex thing up piece by piece.”
So if the act of creation is considerably less complex than analyzing and understanding a part of that creation, it’s far more likely that the complexity gap ends up preventing any self-aware AGI from being able to effectively reinvent oneself, even if it does have full write access to the components.
Do you think data centers and commercial AI licences are free?
Except the major AI players are drastically increasing their prices…