All tracers
llama.cppattention maps · 2ca15f5 · Llama 3.1 8B Q4_K_M · the prompt pass
Logit lens Through the weights Inference tracer

Where every head looks while reading “Why is the sky blue?”

The prompt pass of the tracer's request, all 32 layers × 32 heads, each as a 41 × 41 map: for every token (row), how much of that head's attention went to each earlier token (column). Computed on the CPU from the same Q4_K_M weights, checked against the profiled run.

All 1,024 heads

One square per head, layer down the side and head across. Colour it by what each head does, then click a square.

Worth a look

Method

Pick a head

Rows are the token doing the looking, columns the token looked at. A token can only see itself and what came before, so the top right is empty. Hover for the exact value.

This head

TRACE THE ATTENTIONPress ▶ to walk the layers, the tour or the decode steps