interp — ask your coding agent why a language model said that

A drop-in skill that turns Claude Code, Cursor, Codex, or any coding agent into a mechanistic interpretability assistant. Instead of guessing about a model's internals, your agent measures them — logit lens, direct logit attribution, activation patching, induction heads, SAE features, and steering — and writes a grounded, figure-backed answer. Every technique is validated against published circuits, and it runs on a laptop (CPU/MPS) in seconds.

You:   Why does GPT-2 complete "When John and Mary went to the store, John
       gave a drink to" with " Mary" and not " John"?

Agent: [runs `interp explain ...`]
       It's the IOI name-mover circuit. The answer is undecidable until layer 9
       and becomes the top guess at layer 10. Three heads write " Mary" over
       " John": L9H9 (+1.66), L9H6 (+1.32), L10H0 (+0.88). Causally, patching
       the swapped subject name restores the clean behavior (recovery ≈ 1.0),
       confirming the name is the load-bearing token — matching Wang et al. 2022.

---

Why this exists

LLM coding agents are great at using models and terrible at explaining them — ask one why a model predicted a token and you get plausible-sounding fiction. Mechanistic interpretability has the right tools to answer for real, but they live in research notebooks, not in a form an agent can call. interp packages the core mech-interp toolkit behind a single CLI and a SKILL.md, so any agent can answer "why?" with numbers it computed itself.

Install

As an agent skill (Claude Code, etc.): drop this folder into your skills directory (e.g. .claude/skills/interp/). The agent reads SKILL.md and learns when and how to use it. Then once:

cd interp && ./run.sh setup

As a standalone CLI / library:

pip install -e .          # or: pip install -r requirements.txt
interp explain "..." " Mary" --baseline " John" --corrupt "..."

What it does

Command	Technique	Question it answers
logit-lens	Logit lens	How does the prediction form across layers?
dla	Direct logit attribution	Which heads/MLPs write the answer? (correlational)
patch / patch-heads	Activation patching	What does the model causally depend on?
induction	Induction-head detection	Which heads do in-context copying?
attention	Attention patterns	Where does a head look?
sae	SAE features (Neuronpedia)	Which interpretable features fire on this text?
steer	Contrastive activation steering	Does moving along a direction change behavior?
explain	All of the above	Full grounded report + figures for one prediction

Add --json to any command for machine-readable output (what the agent parses).

Validated against published results

This repo's correctness contract: the tools are trusted only because they independently recover circuits other researchers documented. ./run.sh test checks this every run.

Result	Published source	What interp recovers
IOI name-mover heads	Wang et al. 2022	DLA top heads include L9H9, L9H6, L10H0
IOI causality	Wang et al. 2022	Patching the subject name gives recovery ≈ 1.0
Induction heads	Olsson et al. 2022	Top heads include L5H5, L5H1, L6H9, L7H2, L7H10
Logit-lens trajectory	nostalgebraist 2020	Answer rank: ~13k → 8 (L9) → 0 (L10)

$ ./run.sh test
......                                                  6 passed

Examples

# How a prediction forms, layer by layer
interp logit-lens "The Eiffel Tower is in the city of" --target " Paris"

# Which heads cause the indirect-object prediction (causal)
interp patch-heads \
  "When John and Mary went to the store, John gave a drink to" \
  "When John and Mary went to the store, Mary gave a drink to" \
  " Mary" " John"

# Which interpretable SAE features fire on a sentence (no GPU)
interp sae "The doctor said the patient was fine"

# Detect induction heads
interp induction

See examples/ioi.md for a full walkthrough that reproduces the IOI circuit step by step.

How it works

• Built on TransformerLens for hooks, caching, and the residual-stream decomposition.
• SAE features come from the Neuronpedia API — pre-computed features + human explanations, so no SAE training or GPU is needed.
• Default model is GPT-2 small (every circuit above was discovered on it); pass --model <name> for any TransformerLens-supported model.
• Inference-only and deterministic (CPU): figures and numbers reproduce exactly.

Honest limitations

• DLA is correlational; only patching is causal. The skill enforces this distinction and won't let an agent state a causal claim from DLA alone.
• Steering with a single mean-difference vector is coarse and frequently trades the target behavior for degenerate repetition — interp surfaces this rather than hiding it. (See the companion study steering-audit for a full reliability analysis of when steering silently fails.)
• SAE explanations are auto/crowd-generated (Neuronpedia) and are a guide, not ground truth.
• Activation patching is O(layers × positions) forward passes — fast on GPT-2, slower on large models.

Citation

If you use this in research or teaching:

@software{bano2026interp,
  author = {Bano, Azra},
  title  = {interp: mechanistic interpretability for coding agents},
  year   = {2026},
  url    = {https://github.com/azrabano23/interp}
}

Built on the shoulders of the interpretability community — TransformerLens (Nanda), Neuronpedia (Lin & Bloom), and the IOI / induction-head / logit-lens work cited above.

License

MIT — see LICENSE.