What you're looking at: the activations of every MLP neuron in every transformer layer, for the token currently selected in the text above. Each of the model's 28 layers ends its computation with a wide feed-forward network (a SwiGLU MLP) of 3072 “neurons” — this heatmap shows how strongly each one fired for this single token.
Reading it: row = layer (0 at top = the first layer the token passes through, 27 at bottom = the last), column = neuron index, brightness = activation magnitude (dark ≈ near zero, bright = large). The brightness scale is fixed across the whole run, so it's comparable between tokens and layers, not just within one row.
order: raw keeps the model's native neuron numbering (effectively random visually). clustered and by mean (when available — run compute_neuron_order.py first) reorder the columns so similar or generally more/less active neurons sit near each other, which can make broad patterns easier to spot.
Some neurons are known to fire on very specific, surprising things — a particular punctuation mark, a language, a topic. Look for columns that stay bright across many different tokens (general purpose) versus ones that light up only for this one (highly specific).
What you're looking at: at every layer, the currently-selected token computes a weighted average over every token that came before it (and itself) — the weights are “attention.” This panel shows those weights for the selected token, one row per earlier token position.
heatmap mode: row = earlier token position, column = layer, brightness = how much attention this token pays to that position at that layer. column mode: pick a single layer and see its exact weights as a bar+percentage list instead.
layer: choose one specific layer, or avg layers to average across all 28 — usually the more legible view, since individual layers can be noisy or highly specialized.
skip sink: it's extremely common for the first token in a sequence to soak up a large, mostly uninformative share of attention at every layer — a well-documented phenomenon called an “attention sink.” This option hides that one row/column and rescales the rest, so the more meaningful long tail isn't squashed to invisibility.
highlight text: paints the token strip above with a color wash matching this panel's weights, so you can see at a glance, in context, which earlier words the selection is drawing from.
What you're looking at: the “logit lens” technique — taking the model's internal, in-progress representation at some depth and feeding it straight into the final unembedding step (the same matrix that turns the last layer's output into predicted-token probabilities), as if the model had stopped computing right there. It's a way to eavesdrop on what the model “believes” long before it's actually done.
Reading it: each row/column is one depth — not just the 28 transformer layers, but also named checkpoints inside them (e.g. after attention, after the MLP) where useful. The top-5 candidate tokens and their probabilities are shown for whichever depth you're looking at.
column mode shows one depth's full top-5 as bars; heatmap mode compresses every depth into a single strip so you can watch the prediction sharpen (or flip-flop) across the whole forward pass at a glance. The sparkline above heatmap mode plots the top-1 token's probability at every depth.
A common pattern: early layers guess something generic or plainly wrong, and the correct answer only “locks in” during the last handful of layers — watch for the exact depth where that happens.
What you're looking at: the full pipeline the selected token's vector travels through, top to bottom — text → vocabulary ID → embedding vector → one transformer block's internals → the final output vector → predicted next token → text again. It's meant as a map connecting the other panels above to the actual sequence of matrix operations that produced them.
real vs. schematic: solid amber boxes hold this token's actual captured numbers (the same data the other panels use). Dashed boxes are schematic diagrams of steps this tool never captures or transmits — raw Q/K/V vectors, RoPE angles, pre-softmax scores — shown as their formula/shape only, never invented numbers.
the residual stream: the 1024-dim vector labeled “residual IN,” “residual OUT,” “embedding vector,” and “output vector” is literally the same running vector at every stage — a layer's OUT is byte-identical to the next layer's IN. Step through layers with the ◀/▶ buttons next to “Transformer block” and watch this one vector morph as it passes through all 28 layers.
Stage 3 shows a single chosen layer's internals in detail (steppable); its “top attended tokens” and “active neurons” boxes reuse the exact same numbers as the Attention Lens and Neuron Heatmap panels above, just for this one layer at a time.