Editing Claude AI (section)

=== Mechanistic interpretability ===

Anthropic conducts and publishes extensive research in '''[[mechanistic interpretability]]''' — the attempt to reverse-engineer the internal computations of neural networks to understand what concepts and circuits underlie specific behaviors. Key published work includes:

* '''Toy Models of Superposition''' (2022) — Demonstrating that models can represent more features than they have neurons, via superposition.
* '''Features in Claude 3 Sonnet''' (May 2024) — Identifying millions of interpretable "features" (concept-correlated neuron activation patterns) using sparse autoencoders; published alongside a demonstration model ("Golden Gate Claude") in which the Golden Gate Bridge feature was strongly activated, causing Claude to integrate bridge-related associations into nearly all responses.<ref name="wiki-claude"/>
* Ongoing research into attention head circuits, multi-step reasoning traces, and emergent feature geometry.