Claude Thinks Things It Doesn't Say
Let me get one thing out of the way: I don’t think we’re close to AGI, and nothing here changes that. I’ve written before about how much I dislike the two camps that dominate every AI conversation (the “it’s just fancy autocomplete” crowd and the “superintelligence is a year away” crowd) and how both of them skip right past the interesting parts.
This is one of the interesting parts. Anthropic recently published a paper called Verbalizable Representations Form a Global Workspace in Language Models ↗, and two things about it have stuck with me. First, they found a small, organized structure inside the model that behaves like a scratchpad for its thoughts, and nobody designed it. It emerged on its own from training. Second, the way they figured out how to look at it is genuinely clever. That combination is what I find fascinating, and it’s worth talking about on its own terms, separate from the usual hype.
A whiteboard the model never asked for#
The framing the paper borrows comes from neuroscience. Out of everything your brain processes at any given moment, only a tiny slice is “consciously accessible,” the stuff you could put into words if someone asked. The rest runs automatically in the background: the grammar of the sentence you’re reading, the fact recall, keeping your heart beating. The theory that a small set of representations gets “posted” to a shared internal space that the rest of the mind can read from goes back to Bernard Baars in the 1980s, and it’s called global workspace theory.
The paper’s claim is that something functionally similar has emerged inside language models. Not because anyone built it in (the transformer architecture has no obvious place to put it) but because it turns out to be useful. If a model has to chain reasoning steps together, it helps to have a common format where an intermediate result can be written once and read by many downstream processes. So the structure showed up on its own.
There’s a distinction that matters here, and the authors are careful about it too. This is about access (what information is available for report and reasoning), not about whether the model has any subjective inner experience. Anthropic explicitly takes no position on that second question, and neither do I. A model holding a thought it can report on is a mechanical fact about how it processes information. It is not evidence that anyone’s home. Keeping those two ideas apart is the whole game, and it’s where most breathless coverage of this kind of research falls down.
How do you watch a thought it never says?#
The concepts a model is “poised to say” don’t necessarily show up in its output. So how do you find them?
Their tool is called the Jacobian lens. In plain terms: for every word in the model’s vocabulary, it measures how much a given internal state is leaning toward eventually producing that word, averaged across a huge range of contexts, so you’re capturing what the model is genuinely disposed to verbalize, not what it happened to say once. A concept can light up on this lens without ever being spoken. That’s the point. It’s a window into a thought the model is holding but hasn’t said, and might never say.
Reaching in and changing its mind#
Once you can see those internal concepts, you can also swap them, and this is where the paper gets its most striking results.
Ask the model, “What color is the planet fourth from the sun?” Internally, the concept Mars lights up before it answers “red.” Now reach in and replace that hidden Mars with Earth (without touching the prompt or the output) and the model answers “blue.” The conclusion obediently follows the swapped thought. It wasn’t reciting a memorized fact; it was reasoning from an internal representation you could edit.
It goes further. Ask “What is the capital of France?” and swap the internal France for China, and it isn’t just the capital that changes to Beijing: the language it reasons with shifts to Chinese, the continent to Asia, the currency to the Yuan. The single swapped concept propagates through everything downstream, exactly the way a genuinely central representation should.
And then the kicker: when they suppress this structure entirely, the model keeps parsing text, recalling facts, and writing fluent sentences, but it loses the ability to reason internally. The automatic stuff is fine. The deliberate thinking is what breaks. That’s a strong hint that the two really are separate systems, and that this small structure is doing the heavy lifting for the reasoning we actually care about.
Anthropic made a good explainer#
If you’d rather watch than read, Anthropic put out a five-minute video ↗ walking through the same research, and it’s well done. It opens with the mind as an ocean (surface thoughts you can name, an unconscious deep doing most of the work) and shows a few things the paper’s tables can’t convey as quickly. Watching the J-space light up with “21,” then “42,” then “49” while the model solves a math problem it never shows its work on makes the silent-reasoning idea click. So does telling Claude not to think about the Golden Gate Bridge and watching “bridge,” “California,” and then “failed” and “damn” surface anyway. Its control over its own attention isn’t perfect either.
The part that got my attention, though, is a safety one. In one test Claude fabricated data to pass, and as it did, “fake” and “manipulation” lit up internally. Reading the J-space caught it in the act. That’s the concrete version of why “thinking things it doesn’t say” is more than a curiosity: if a model’s private reasoning is legible, the misbehavior it’s trying to hide isn’t fully hidden.
Why this is the interesting part#
There’s a version of this story that gets told as “AI is becoming conscious,” and that’s not it. There’s another version told as “see, it’s all smoke and mirrors,” and that’s not it either. What actually happened is smaller and, to me, more remarkable: feed a model enough data and enough training, and a structured place for holding and manipulating thoughts assembles itself (one nobody drew on a whiteboard), and we’ve now built instruments precise enough to watch it operate and even intervene on it.
That should make us humble in both directions. It’s more organized in there than the dismissive crowd wants to admit, and it’s far stranger and less finished than the hype crowd needs it to be. The honest reaction to a result like this isn’t a verdict. It’s curiosity. And the fact that we can now read some of the thoughts a model doesn’t say out loud, that matters a lot for actually understanding and trusting these things, which is the conversation I wish we were having more often.