stop hook: don't silently skip gates, don't block on own artifacts#26
Draft
evgunter wants to merge 11 commits into
Draft
stop hook: don't silently skip gates, don't block on own artifacts#26evgunter wants to merge 11 commits into
evgunter wants to merge 11 commits into
Conversation
The docs-only check diffed against origin/$BASE_BRANCH and swallowed the failure, so an unresolvable ref became an empty diff and every gate was skipped. Repos with no remote, or with fetch_and_merge off, never ran a gate. Resolve origin/$BASE_BRANCH, fall back to the local branch, and run the gates rather than skip when neither resolves. The hook wrote .reviewer/logs/ and then blocked on it via the uncommitted-changes check. Write a self-ignoring .reviewer/.gitignore before the first log write, leaving settings.json trackable. _ensure_pr wrote .reviewer/outputs/pr_number on its early-return paths before the mkdir that created the directory, so the first run with require_pr=false and no PR died. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Every gate diffs against the base branch, so an unresolvable base means none of them can run. Blocking the agent would demand something it has no way to satisfy; it would discover this only after spawning subagents, then ask the human anyway. Report it directly instead. Exit 1 keeps it non-blocking, so it reaches the human and the agent finishes. Only exit 2 blocks the agent, so only exit 2 belongs in the consecutive-block tracker that feeds the stuck-agent hatch. Counting non-blocking failures inflated a count of times the agent was trapped, which it never was. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The skills read ${GIT_BASE_BRANCH:-main}, but nothing anywhere sets
GIT_BASE_BRANCH -- not the plugin, not mngr, not the hook. It always
evaluated to "main", so stop_hook.base_branch was configurable in name
only: the hook honoured it and every gate silently diffed against main.
mngr's per-agent CODE_GUARDIAN_STOP_HOOK__BASE_BRANCH export reached the
hook and never the skills.
Add get_config.sh over read_json_config and have the skills call it, so
both resolve through one path: env -> settings.local.json -> settings.json
-> default. Follows the existing ${CLAUDE_PLUGIN_ROOT}/scripts pattern in
verify-conversation.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Scratch-repo helpers guard every mutating call behind _assert_scratch so a failed cd can never touch the developer's own checkout.
…teps 1-3 Covers the scripts this branch does not change, so the suite documents the existing contract rather than only the new behavior.
Unparseable JSON left every key empty, which Step 1 read as an unset enabled_when and exited 0. Validate both config files up front and report. Also mkdir the outputs dir in poll-ci, which writes there but only ran after ensure-pr had created it.
Silencing it meant a typo'd config aborted the read with jq's exit code and nothing to explain it. No consumer folds this stderr into parsed output.
mkdir -p before writing needs no justification, and the note invited a coupling to ensure-pr that the pr_number argument contradicts.
The tests were already black-box subprocess calls -- every one shelled out and asserted on exit code, output, or files on disk, and none sourced a function into the test process. So bash bought no affinity with the code under test, and what it cost was isolation: stub_command exported PATH and never restored it, leaving "no network" a convention rather than a guarantee, and a third of lib.sh existed only to stop a failed `cd` from operating on the real checkout. tmp_path plus an explicit subprocess cwd and env make both unrepresentable. The test process never changes directory, the stub dir lives outside the work tree, and every repo gets a gh stub that refuses by default, so a test that forgets to stub one cannot reach the API. The precedence matrix and the near-identical gh case blocks collapse into parametrize tables and one builder. 140 `it` blocks become 116 tests. The difference is blocks that were labels on assertions against a previous test's state; each is now fused into the test whose run it describes. Coverage is unchanged, and the suite is still red against main's scripts: 30 failures, every one on a fix this branch makes. .venv and __pycache__ pick up ignore rules -- they were invisible here only under a personal global ignore.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stop-hook failures that made the hook either skip work it should have done or block on work nobody could do — plus the test suite that pins them down.
Silently skipped gates
Step 5 decided "docs-only" from
git diff --name-only "origin/$BASE_BRANCH"...HEAD 2>/dev/null || echo "". Whenorigin/maindid not resolve — no remote,fetch_and_mergeoff, or a base branch that simply is not calledmain— the diff errored, the fallback madeCHANGED_FILESempty, and an empty file list is indistinguishable from docs-only. Every review and PR gate was skipped, silently, on a branch full of code.The base ref is now resolved explicitly (
origin/<base>, else the local branch), and the diff no longer swallows its own errors. If neither ref exists, the hook says so and exits 1 — non-blocking, because only a human can fix a misconfigured base branch, and blocking the agent on a demand it cannot satisfy just wedges the session.Blocking on its own artifacts
The uncommitted-changes gate blocks on any untracked file, and the hook's own
.reviewer/logs/and.reviewer/outputs/are untracked — so it blocked on its own exhaust.ensure_reviewer_gitignorenow writes.reviewer/.gitignorebefore the first log write (which is what creates the directory). The rules cover.gitignoreitself so the fix does not become the same kind of noise;settings.jsonand the category files stay trackable.Miscounted blocks
The EXIT trap recorded a consecutive block on any non-zero exit, so non-blocking exit-1 paths inflated the counter and dragged the stuck-hatch closer for reasons the agent was never asked to fix. It now counts only exit 2, the blocking code.
A typo disabled everything
Found while writing the tests, in code this branch had not touched.
read_json_configreturns nothing for every key in an unparseable file, and Step 1 cannot tell that apart from "enabled_whenwas never set" — so it exited 0. One stray comma in.reviewer/settings.jsonsilently switched off every gate, with2>/dev/nullon the jq call ensuring no diagnostic anywhere. Demonstrated on one repo, changing nothing but the comma:Both config files are now checked before the first read, reported on stderr, and exit 1 — non-blocking, same reasoning as the unresolvable base above. The
2>/dev/nullis gone as well, so a read that aborts on a typo carries jq's parse error out with it instead of dying mute.get_config.shsurfaces the same fault differently, exiting 5 with the parse error on stderr and nothing on stdout, which is what the skills consume; the up-front check is what actually protects the gates.Also here
_ensure_prcreates.reviewer/outputsup front, since its early-return paths write there too._poll_ciwritespr_statusto the same directory and never created it either; standalone, green CI came back as exit 1 with a raw bash redirect error. Masked today only because the orchestrator always runsensure-prfirst.autofixandverify-architectureskills resolved the base branch withecho "${GIT_BASE_BRANCH:-main}", which reads a different env var than the hook's own resolution and silently disagreed with it about which branch is the base. Both now go throughscripts/get_config.sh, a small CLI wrapper over the existingread_json_config, so the skills and the hook always agree.Tests
uv run pytest— 116 tests, ~23s, or ~5s with-n auto. No network.pytest rather than bash, even though the scripts are shell: every test is a black-box subprocess call asserting on an exit code, an output stream, or a file on disk, and none reaches inside a script. The harness language is therefore a free choice, and the shipped plugin is unaffected — it stays bash +
jq+gh. pytest is a contributor dependency only, next to theuvand python that.pre-commit-config.yamlalready requires.The suite covers the scripts this branch does not change, not just the new behavior, so it documents the existing contract: config precedence, the common helpers, the gate marker logic, PR/CI with a
ghstub that refuses unrecognized calls, and orchestrator Steps 0–3 plus the EXIT trap. Pointed atmain's scripts it fails 30 tests, every one of them on a fix this branch makes.Hermeticity is structural here rather than conventional, which is most of why it is not bash:
tmp_paththrough an explicit subprocesscwd. The test process never changes directory, so no helper can operate on the real checkout — a bash harness that rancdinside a command substitution once committed test fixtures to the working branch.CODE_GUARDIAN_*, which outrank every config file and which the mngr integration exports five of into each session, and pointsGIT_CONFIG_GLOBALandGIT_CONFIG_SYSTEMat/dev/null. A personal global.reviewer/ignore rule would otherwise mask exactly what the gitignore tests check, and the suite would pass for that developer and fail for everyone else.ghstub that refuses any call, from a stub directory outside the work tree. Reaching the real API takes deliberately overwriting that stub, rather than merely forgetting to write one.Verification
uv run pytestpasses all 116 tests here and fails 30 againstmain's scripts, every one of them on a fix this branch makes. The scenarios run end to end throughstop_hook_orchestrator.sh: the unresolvable base exiting 1, the local base fallback reaching the gates, docs-only and empty diffs still skipping cleanly, the block tracker gaining an entry on exit 2 and none on exit 1, and.reviewer/.gitignorebeing written at all.The silent skip needs no misconfiguration to reach. Any repo without a remote — or with
fetch_and_mergeoff, which is how the mngr integration runs — hits it.The two failures also mask each other, which is most of why the skip went unnoticed. On a clean checkout
mainnever reaches the base branch logic at all: Step 3 blocks on the hook's own untracked.reviewer/logsfirst, so every scenario exits 2 and the base is never consulted. Ignore.reviewer/in your global gitignore — as anyone keeping their reviewer config out of shared repos does — and the self-block disappears, Step 5 is reached, andmainsilently skips every gate instead. Which of the two you see depends on your gitignore, and each is invisible while the other holds.🤖 Generated with Claude Code