Circuit Tracing
Circuit tracing shows the steps a model takes inside to get from a prompt to its next token. Neuronpedia makes attribution graphs for your prompts, and gives you tools to explore, label, steer, and share them.
Start here: https://neuronpedia.org/graph
Background
Anthropic's Circuit Tracing paper shows a way to trace how a model gets to its output. The method replaces the model's MLPs with transcoders, which have features that are easier to understand. It then calculates how much each feature, token, and error term adds to the next one. The result is an attribution graph.
Neuronpedia made this tool with Anthropic and the Anthropic Fellows. It uses the open source circuit-tracer library.
For a longer introduction, see:
- The Circuit Analysis Research Landscape - A post by Anthropic, Google DeepMind, Goodfire AI, EleutherAI, and Decode.
- Attribution Graphs for Dummies - A two-part video tutorial.
- The Utility of Interpretability - A Latent Space video that includes a demo of the Neuronpedia tool.
Example
The Dallas graph uses the prompt "The capital of the state containing Dallas is". Gemma 2 2B answers "Austin". The graph shows a path through the model that is approximately:
Dallas ➡️ Texas ➡️ capital ➡️ Austin
In This Section
- Generate a Graph - Make a graph from your own prompt.
- Explore a Graph - Read the graph, make subgraphs, and group features into supernodes.
- Steer from a Graph - Test your circuit by turning features up or down.
- Attention in Graphs (LORSA) - Graphs that also trace attention.
- Upload Your Graphs - Add graphs that you made with other tools.