cross-posted from: https://piefed.world/c/tech/p/1246129/anthropic-uncovered-claude-s-consciousness-like-workbench-the-mysterious-j-space-hides-h
As you read this sentence, circuits in your brain are adjusting your posture, controlling your breathing, and transforming lines and curves on the screen into recognizable words. Most of this processing is invisible to you. But some of what takes place in your brain you do have access to—an image that pops into your head, or a deliberate plan you make about where to go shopping. Neuroscientists and philosophers sometimes refer to the latter type of brain activity as “consciously accessible,” to distinguish it from all the other processing that goes on unconsciously. This activity has special properties: we can describe it, control it, and use it for deliberate reasoning, in contrast to all the automatic processing that goes on without our awareness.
In a new paper, we present evidence that a similar distinction has emerged in modern language models like Claude. We find that Claude has developed a small collection of internal neural patterns that, compared to all its other internal processing, play a special role.
We call the collection of these patterns the J-space—named after the technique we used to find them, involving a mathematical concept called the Jacobian. Each J-space pattern is linked to a particular word. But when one of these patterns lights up, it doesn’t mean the model is saying that word—just that the word is on its mind. If you’ve heard of language models having a “scratchpad” or “chain of thought”—text they write to themselves while reasoning—the J-space is something different. It operates silently, in the model’s internal neural activations, allowing the model to think about a concept without writing it down. Notably, the J-space wasn’t designed or programmed by us, but instead emerged on its own during Claude’s training process.
I finally meet someone who has really bought into the “LLMs will lead to AGI” bullshit Altman and Amodei are pushing. Cool, i know those types are rare here on lemmy, i’m gonna tag you like i would a golden Pokemon!
Lol keep putting everyone who mildly disagrees with you into a bucket and othering them, totally normal healthy behaviour that leads to good outcomes, not exactly what has lead to the dysfunctional right wing of America.
I’ve made no such claims, I just actually read articles and think about them with an open mind. You’re welcome to run around screaming thought terminating phrases in all caps all you want, but I’m gonna call you out when you do.
If you are suggesting that LLMs are sentient, then this isn’t a “mild disagreement”, it’s the San Andreas fault.
I am certain that LLMs are a step into the direction of AGI, but it’s a small piece of the puzzle. There are so many technological leaps that have to happen before AGI can become real that I am certain that neither of us will see that day.
If you have such an open mind, then please also read the paper they released, which is a lot more careful in how all of this is framed. This article is here to drive VC funding, not for scientific accuracy, and this is not the first time Anthropic has framed things in a sensationalist way that had no basis in the reality of the paper. (the blackmail-thing for example.)