cross-posted from: https://piefed.world/c/tech/p/1246129/anthropic-uncovered-claude-s-consciousness-like-workbench-the-mysterious-j-space-hides-h
As you read this sentence, circuits in your brain are adjusting your posture, controlling your breathing, and transforming lines and curves on the screen into recognizable words. Most of this processing is invisible to you. But some of what takes place in your brain you do have access to—an image that pops into your head, or a deliberate plan you make about where to go shopping. Neuroscientists and philosophers sometimes refer to the latter type of brain activity as “consciously accessible,” to distinguish it from all the other processing that goes on unconsciously. This activity has special properties: we can describe it, control it, and use it for deliberate reasoning, in contrast to all the automatic processing that goes on without our awareness.
In a new paper, we present evidence that a similar distinction has emerged in modern language models like Claude. We find that Claude has developed a small collection of internal neural patterns that, compared to all its other internal processing, play a special role.
We call the collection of these patterns the J-space—named after the technique we used to find them, involving a mathematical concept called the Jacobian. Each J-space pattern is linked to a particular word. But when one of these patterns lights up, it doesn’t mean the model is saying that word—just that the word is on its mind. If you’ve heard of language models having a “scratchpad” or “chain of thought”—text they write to themselves while reasoning—the J-space is something different. It operates silently, in the model’s internal neural activations, allowing the model to think about a concept without writing it down. Notably, the J-space wasn’t designed or programmed by us, but instead emerged on its own during Claude’s training process.
You think those companies have any shred of decency? Even if they could prove without a doubt that these statistical models are more than just that - that wouldn’t change a thing in how they were used. We can’t even agree that all humans should have the same amount of rights - how do you think an sentient program would fare?
And regarding the global economy: the only “global economy” thing i can see at the moment is a group of companies that are overvalued because of circular cash flows and artificial hype that stems from an assumption that LLMs can somehow become AGI, which would inflate the value of those companies into infinity. We are currently not even able to measure a ROI for these things, and companies worldwide are pulling emergency breaks in usage because of the costs, which are not even covering what the inference itself costs. The only “Global” thing here will be the crash, fucking up the DIY PC market for a decade or longer.
The vast majority of that had nothing to do with anything I’d said. The one thing that does being the decency and ethics of the giant corporations, which of course I don’t think they have those things. Companies don’t do ethical shit because they’re good. If doing immoral things makes them a lot more money and doesn’t come with much risk of discovery, I imagine most would jump on it.
Public outcry is the issue, normally. When we find out BP is poisoning sea life, or some singer is urinating on children, or some other awful thing is being done traditionally a lot of people would get together and make a big issue of it, that costs the evildoer much money until they stop and/or leads to prison time.
That doesn’t seem to be working properly at the moment, however, so I imagine if a paper came out showing something far to close to global workspace theory in AI for anyone to be rationally comfortable about instead people would just keep chanting that it’s not real, because they’d prefer it not be.