MLP Neurons

A model's MLP neurons are the model's own units. Each MLP layer has thousands of them. You do not need an SAE or another trained dictionary to study them.

Neuronpedia has dashboards for 8.6 million MLP neurons in 10 models. Each neuron has the same kind of dashboard as an SAE feature.

Start here: neuronpedia.org/neurons

Models

Model	Layers	Neurons per layer	Inference
Qwen3 8B	36	12,288	Yes
Qwen3 14B	40	17,408	No
Qwen3 32B	64	25,600	Yes
Qwen3.5 4B	32	9,216	No
Gemma 4 12B	48	15,360	No
Gemma 4 12B IT	48	15,360	No
Gemma 4 31B IT	60	21,504	Yes
Llama 3.1 8B IT	32	14,336	Yes
Llama 3.3 70B IT	80	28,672	Yes
GPT-2 Small	12	3,072	No

Neuron IDs

The source set for neurons is neurons. A neuron's source ID is only its layer number, with no source set name after it.

For example, qwen3-8b/24/2665 is neuron 2665 in layer 24 of Qwen3 8B. See Key Concepts.

The MLP Neurons Page

The MLP Neurons page has the tools for all neuron models in one place. It starts on Qwen3 8B.

• Random Neuron - Open a random neuron from a random model and layer.
• Jump To - Go to a model, a neuron layer, or a neuron.
• Search via Inference - Find the neurons that fire the most on your text. See Search via Inference.
• Browse - Look through the neurons of one layer.
• Search Explanations - Find neurons by the text of their explanations.

Inference

For models with inference, you can test neurons with your own text, steer with them, and search for them. For example:

• Search TopK by Token on Qwen3 8B, with a Martin Luther King Jr. quote.
• Steer with an "airplanes" neuron in Qwen3 8B.

Neurons fire on about half of all tokens. So Search TopK by Token does not use a Density Threshold for neurons.

Explanations

Many neurons have explanations from other neuron research platforms, and token lists from their methods. See Community Explanations and Token Lists.