The residual stream at any layer lives in the same space the output head reads at the end. So take the vector at any layer and any token position, push it through the final norm and the output head, and ask: if the model had to answer from here, which word would it say? That is the logit lens. It turns the latent state into words you can read, layer by layer, with the caveat that the early layers were never meant to be read this way.
Rows are layers (0 is the embedding itself, 32 is after the last layer). Columns are the prompt tokens. Each cell is the most likely next token from that state, coloured by its probability; hover for the top five. A green frame marks cells where the lens already predicts the token that actually comes next in the prompt.
Column by column, the five most likely next tokens at the last prompt position after each layer. A ribbon joins a token that stays in the top five from one layer to the next. Widths follow probability; early layers have no real favourite, so everything is thin.
Each attention head and each layer's feed-forward block adds a vector to the residual stream. Projecting that vector alone through the final norm's scale and the output head says how much it pushed each candidate up or down, directly. This is direct logit attribution: it credits the last writer, not the earlier parts whose work it depends on.
Source lines are pinned to llama.cpp commit 2ca15f5 (Oct 4 2026). Where each head looks during this prompt is on Attention maps; the pass itself, with its measured kernels and weights, is on Through the weights.