Skip to content

stop hook: don't silently skip gates, don't block on own artifacts#26

Draft
evgunter wants to merge 11 commits into
mainfrom
fix/silent-skips-and-self-block
Draft

stop hook: don't silently skip gates, don't block on own artifacts#26
evgunter wants to merge 11 commits into
mainfrom
fix/silent-skips-and-self-block

Conversation

@evgunter

@evgunter evgunter commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

Stop-hook failures that made the hook either skip work it should have done or block on work nobody could do — plus the test suite that pins them down.

Silently skipped gates

Step 5 decided "docs-only" from git diff --name-only "origin/$BASE_BRANCH"...HEAD 2>/dev/null || echo "". When origin/main did not resolve — no remote, fetch_and_merge off, or a base branch that simply is not called main — the diff errored, the fallback made CHANGED_FILES empty, and an empty file list is indistinguishable from docs-only. Every review and PR gate was skipped, silently, on a branch full of code.

The base ref is now resolved explicitly (origin/<base>, else the local branch), and the diff no longer swallows its own errors. If neither ref exists, the hook says so and exits 1 — non-blocking, because only a human can fix a misconfigured base branch, and blocking the agent on a demand it cannot satisfy just wedges the session.

Blocking on its own artifacts

The uncommitted-changes gate blocks on any untracked file, and the hook's own .reviewer/logs/ and .reviewer/outputs/ are untracked — so it blocked on its own exhaust. ensure_reviewer_gitignore now writes .reviewer/.gitignore before the first log write (which is what creates the directory). The rules cover .gitignore itself so the fix does not become the same kind of noise; settings.json and the category files stay trackable.

Miscounted blocks

The EXIT trap recorded a consecutive block on any non-zero exit, so non-blocking exit-1 paths inflated the counter and dragged the stuck-hatch closer for reasons the agent was never asked to fix. It now counts only exit 2, the blocking code.

A typo disabled everything

Found while writing the tests, in code this branch had not touched. read_json_config returns nothing for every key in an unparseable file, and Step 1 cannot tell that apart from "enabled_when was never set" — so it exited 0. One stray comma in .reviewer/settings.json silently switched off every gate, with 2>/dev/null on the jq call ensuring no diagnostic anywhere. Demonstrated on one repo, changing nothing but the comma:

valid config  -> exit 2   gates fire
+ one comma   -> exit 0   hook does nothing, says nothing

Both config files are now checked before the first read, reported on stderr, and exit 1 — non-blocking, same reasoning as the unresolvable base above. The 2>/dev/null is gone as well, so a read that aborts on a typo carries jq's parse error out with it instead of dying mute. get_config.sh surfaces the same fault differently, exiting 5 with the parse error on stderr and nothing on stdout, which is what the skills consume; the up-front check is what actually protects the gates.

Also here

  • _ensure_pr creates .reviewer/outputs up front, since its early-return paths write there too. _poll_ci writes pr_status to the same directory and never created it either; standalone, green CI came back as exit 1 with a raw bash redirect error. Masked today only because the orchestrator always runs ensure-pr first.
  • The autofix and verify-architecture skills resolved the base branch with echo "${GIT_BASE_BRANCH:-main}", which reads a different env var than the hook's own resolution and silently disagreed with it about which branch is the base. Both now go through scripts/get_config.sh, a small CLI wrapper over the existing read_json_config, so the skills and the hook always agree.

Tests

uv run pytest — 116 tests, ~23s, or ~5s with -n auto. No network.

pytest rather than bash, even though the scripts are shell: every test is a black-box subprocess call asserting on an exit code, an output stream, or a file on disk, and none reaches inside a script. The harness language is therefore a free choice, and the shipped plugin is unaffected — it stays bash + jq + gh. pytest is a contributor dependency only, next to the uv and python that .pre-commit-config.yaml already requires.

The suite covers the scripts this branch does not change, not just the new behavior, so it documents the existing contract: config precedence, the common helpers, the gate marker logic, PR/CI with a gh stub that refuses unrecognized calls, and orchestrator Steps 0–3 plus the EXIT trap. Pointed at main's scripts it fails 30 tests, every one of them on a fix this branch makes.

Hermeticity is structural here rather than conventional, which is most of why it is not bash:

  • Each test drives its own repo under tmp_path through an explicit subprocess cwd. The test process never changes directory, so no helper can operate on the real checkout — a bash harness that ran cd inside a command substitution once committed test fixtures to the working branch.
  • The subprocess environment drops CODE_GUARDIAN_*, which outrank every config file and which the mngr integration exports five of into each session, and points GIT_CONFIG_GLOBAL and GIT_CONFIG_SYSTEM at /dev/null. A personal global .reviewer/ ignore rule would otherwise mask exactly what the gitignore tests check, and the suite would pass for that developer and fail for everyone else.
  • Every repo gets a gh stub that refuses any call, from a stub directory outside the work tree. Reaching the real API takes deliberately overwriting that stub, rather than merely forgetting to write one.

Verification

uv run pytest passes all 116 tests here and fails 30 against main's scripts, every one of them on a fix this branch makes. The scenarios run end to end through stop_hook_orchestrator.sh: the unresolvable base exiting 1, the local base fallback reaching the gates, docs-only and empty diffs still skipping cleanly, the block tracker gaining an entry on exit 2 and none on exit 1, and .reviewer/.gitignore being written at all.

The silent skip needs no misconfiguration to reach. Any repo without a remote — or with fetch_and_merge off, which is how the mngr integration runs — hits it.

The two failures also mask each other, which is most of why the skip went unnoticed. On a clean checkout main never reaches the base branch logic at all: Step 3 blocks on the hook's own untracked .reviewer/logs first, so every scenario exits 2 and the base is never consulted. Ignore .reviewer/ in your global gitignore — as anyone keeping their reviewer config out of shared repos does — and the self-block disappears, Step 5 is reached, and main silently skips every gate instead. Which of the two you see depends on your gitignore, and each is invisible while the other holds.

🤖 Generated with Claude Code

evgunter and others added 11 commits July 15, 2026 10:34
The docs-only check diffed against origin/$BASE_BRANCH and swallowed the
failure, so an unresolvable ref became an empty diff and every gate was
skipped. Repos with no remote, or with fetch_and_merge off, never ran a
gate. Resolve origin/$BASE_BRANCH, fall back to the local branch, and run
the gates rather than skip when neither resolves.

The hook wrote .reviewer/logs/ and then blocked on it via the
uncommitted-changes check. Write a self-ignoring .reviewer/.gitignore
before the first log write, leaving settings.json trackable.

_ensure_pr wrote .reviewer/outputs/pr_number on its early-return paths
before the mkdir that created the directory, so the first run with
require_pr=false and no PR died.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Every gate diffs against the base branch, so an unresolvable base means
none of them can run. Blocking the agent would demand something it has no
way to satisfy; it would discover this only after spawning subagents, then
ask the human anyway. Report it directly instead. Exit 1 keeps it
non-blocking, so it reaches the human and the agent finishes.

Only exit 2 blocks the agent, so only exit 2 belongs in the consecutive-block
tracker that feeds the stuck-agent hatch. Counting non-blocking failures
inflated a count of times the agent was trapped, which it never was.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The skills read ${GIT_BASE_BRANCH:-main}, but nothing anywhere sets
GIT_BASE_BRANCH -- not the plugin, not mngr, not the hook. It always
evaluated to "main", so stop_hook.base_branch was configurable in name
only: the hook honoured it and every gate silently diffed against main.
mngr's per-agent CODE_GUARDIAN_STOP_HOOK__BASE_BRANCH export reached the
hook and never the skills.

Add get_config.sh over read_json_config and have the skills call it, so
both resolve through one path: env -> settings.local.json -> settings.json
-> default. Follows the existing ${CLAUDE_PLUGIN_ROOT}/scripts pattern in
verify-conversation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Scratch-repo helpers guard every mutating call behind _assert_scratch so a
failed cd can never touch the developer's own checkout.
…teps 1-3

Covers the scripts this branch does not change, so the suite documents the
existing contract rather than only the new behavior.
Unparseable JSON left every key empty, which Step 1 read as an unset
enabled_when and exited 0. Validate both config files up front and report.

Also mkdir the outputs dir in poll-ci, which writes there but only ran
after ensure-pr had created it.
Silencing it meant a typo'd config aborted the read with jq's exit code and
nothing to explain it. No consumer folds this stderr into parsed output.
mkdir -p before writing needs no justification, and the note invited a
coupling to ensure-pr that the pr_number argument contradicts.
The tests were already black-box subprocess calls -- every one shelled out and
asserted on exit code, output, or files on disk, and none sourced a function
into the test process. So bash bought no affinity with the code under test, and
what it cost was isolation: stub_command exported PATH and never restored it,
leaving "no network" a convention rather than a guarantee, and a third of lib.sh
existed only to stop a failed `cd` from operating on the real checkout.

tmp_path plus an explicit subprocess cwd and env make both unrepresentable. The
test process never changes directory, the stub dir lives outside the work tree,
and every repo gets a gh stub that refuses by default, so a test that forgets to
stub one cannot reach the API. The precedence matrix and the near-identical gh
case blocks collapse into parametrize tables and one builder.

140 `it` blocks become 116 tests. The difference is blocks that were labels on
assertions against a previous test's state; each is now fused into the test whose
run it describes. Coverage is unchanged, and the suite is still red against
main's scripts: 30 failures, every one on a fix this branch makes.

.venv and __pycache__ pick up ignore rules -- they were invisible here only under
a personal global ignore.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant