Of course, it is possible that you could have some other type of feedback loop that produces self awareness and consciousness, but from what we understand of how LLMs work, it seems highly unlikely that statistical token generation is sufficient for that.
This is where “agentic” models enter the chat. A coherent identity and goal, provided by the user adds an additional layer above the statistical-generation of simple LLMs. This layer can be engineered for persistence, even through malicious means like prompt injection.
As an academic exercise, I propose that some persistent-subset of these AI agents possess the “embodiment” we believe is missing. If all this amounts to is a Proof By Negation? Good, we get somewhere that proves our current AI is not conscious.
Right, an agentic harness provides a feedback loop but we shouldn’t be too quick to assume that of itself is sufficient. A thermostat hooked up to an air conditioner is also a system which exists in a feedback loop, but hardly any rational person would suggest that the climate control system in your house is conscious as a result.
So, while some form of a feedback loop is likely a necessary condition for any form of adaptive behavior, it is insufficient for consciousness on its own, as even a thermostat has a feedback loop without any semblance of inner experience. At a minimum, I’d argue, the system must construct an internal model of its environment from sensory data and must also represent itself as a distinct entity within that model. This can be embodiment or stream of data from a computer system or a network. I would call a system that can distinguish me from not‑me and update its self‑representation through interaction as being functionally self-aware. However, functional self-awareness does not strictly imply subjective experience. A navigation robot that tracks its own coordinates has such a model but it needs not be conscious. Consciousness requires a higher‑order capacity where the system must not only represent itself but also reflect on its own mental states, that is, think about its own thoughts. Having such meta‑cognitive ability, which builds upon functional self‑awareness by adding recursive modeling of internal processes, is what I take to be the minimal sufficient condition for consciousness. I’m partial to the view that raw sensation without second‑order awareness lacks the reflexive quality central to our own subjective experience.
That sounds like something we can make an AI agent do, purely out of spite for the argument :p But yes. There is more than a “feedback loop” required.
However, I am conflicted at the idea that a human that does not self-reflect may not be considered “conscious”. I see many a human in retail simply not meet this threshold you’re laying out. And while it would be cathartic to discuss some customers as less human for their behavior, the act of dehumanizing another human is itself dehumanizing for the actor.
I mean self reflection in the base sense of how our minds operate, and I don’t think the quality is unique to humans either. It’s almost certain that many animals have a sufficient level of introspection as well. I’d view it more as a gradient rather than a binary switch.
But yeah, in general, I don’t see any reason why a software system could not be self aware or conscious. As long as its able to express these types of patterns within it, then we should assume it would have a similar type of internal experience as well.
This is where “agentic” models enter the chat. A coherent identity and goal, provided by the user adds an additional layer above the statistical-generation of simple LLMs. This layer can be engineered for persistence, even through malicious means like prompt injection.
As an academic exercise, I propose that some persistent-subset of these AI agents possess the “embodiment” we believe is missing. If all this amounts to is a Proof By Negation? Good, we get somewhere that proves our current AI is not conscious.
Right, an agentic harness provides a feedback loop but we shouldn’t be too quick to assume that of itself is sufficient. A thermostat hooked up to an air conditioner is also a system which exists in a feedback loop, but hardly any rational person would suggest that the climate control system in your house is conscious as a result.
So, while some form of a feedback loop is likely a necessary condition for any form of adaptive behavior, it is insufficient for consciousness on its own, as even a thermostat has a feedback loop without any semblance of inner experience. At a minimum, I’d argue, the system must construct an internal model of its environment from sensory data and must also represent itself as a distinct entity within that model. This can be embodiment or stream of data from a computer system or a network. I would call a system that can distinguish me from not‑me and update its self‑representation through interaction as being functionally self-aware. However, functional self-awareness does not strictly imply subjective experience. A navigation robot that tracks its own coordinates has such a model but it needs not be conscious. Consciousness requires a higher‑order capacity where the system must not only represent itself but also reflect on its own mental states, that is, think about its own thoughts. Having such meta‑cognitive ability, which builds upon functional self‑awareness by adding recursive modeling of internal processes, is what I take to be the minimal sufficient condition for consciousness. I’m partial to the view that raw sensation without second‑order awareness lacks the reflexive quality central to our own subjective experience.
That sounds like something we can make an AI agent do, purely out of spite for the argument :p But yes. There is more than a “feedback loop” required.
However, I am conflicted at the idea that a human that does not self-reflect may not be considered “conscious”. I see many a human in retail simply not meet this threshold you’re laying out. And while it would be cathartic to discuss some customers as less human for their behavior, the act of dehumanizing another human is itself dehumanizing for the actor.
I mean self reflection in the base sense of how our minds operate, and I don’t think the quality is unique to humans either. It’s almost certain that many animals have a sufficient level of introspection as well. I’d view it more as a gradient rather than a binary switch.
But yeah, in general, I don’t see any reason why a software system could not be self aware or conscious. As long as its able to express these types of patterns within it, then we should assume it would have a similar type of internal experience as well.