HeadlinesBriefing favicon HeadlinesBriefing.com

Anthropic's J-Space: A Math Primer

Towards Data Science •
×

In Anthropic's paper "Verbalizable Representations Form a Global Workspace in Language Models," researchers introduce the J-space, an LLM analogue (demonstrated on Claude Opus 4.6) to the global workspace in the human cortex. The authors present extensive empirical evidence and demonstrate the J-space as an auditing tool for LLM alignment.

We re-examine the formal definition and mathematical properties of the J-space, providing precise justifications for Anthropic's surgical interventions. The original paper's concise style can leave key mechanics implicit; we offer a clear, step-by-step account through an applied mathematics lens.

The semantic representation of a token at position t in layer 𝓁 within the residual stream is defined. The average causal impact on the final layer L is an expectation over prompts, positions, and future contexts. Multiplying by the unembedding matrix gives J-lens vectors. These form an overcomplete frame; the J-space is the set of k-sparse non-negative linear combinations of these vectors—a union of cones.

The verbalizable component is the orthogonal projection onto the J-space, an approximate k-sparse expansion. The non-verbalizable component is the residual.