Skip to main content

Attention in Graphs (LORSA)

Standard attribution graphs replace only the MLPs with transcoders. OpenMOSS extended this to also replace attention, which they call Complete Replacement Models (CRMs). Neuronpedia can generate CRM graphs for Qwen3-1.7B.

Example: Dallas graph for Qwen3-1.7B

A CRM graph for Qwen3-1.7B. Triangles are LORSA attention features, and circles are transcoder features.

LORSA Nodes​

CRM graphs have a new type of node for attention, called LORSA (Low-Rank Sparse Attention).

  • LORSA nodes are triangles ▲. Transcoder nodes are circles ⏺.
  • When you select a LORSA node, you see the LORSA dashboard. Hover over a top activation token to see its attention pattern.

QK Tracing​

A LORSA node also shows QK tracing results in the Node Connections panel. These are the top (query feature, key feature) pairs, and the top single features, that cause the attention. They tell you why a LORSA feature attends from one token to another token.

Limits​

  • CRM prompts can have a maximum of 10 tokens.
  • QK tracing makes generation slower.