Skip to content

fix(drift): shape-detect slot routing + runtime>docs PR guidance#105

Merged
paultyng merged 1 commit into
mainfrom
fix/drift-shape-detect
May 24, 2026
Merged

fix(drift): shape-detect slot routing + runtime>docs PR guidance#105
paultyng merged 1 commit into
mainfrom
fix/drift-shape-detect

Conversation

@paultyng
Copy link
Copy Markdown
Owner

Summary

PR #104's dispatch surfaced two related problems: the drift script's slot router used trial-and-error and stashed mis-shaped content in the wrong slot, and the PR body gave no triage guidance for the reviewer.

  • Shape-detect routing in scripts/upstream-drift.sh: pick the slot from top-level JSON keys, not from "which slot happened to not error." Stops corpus poisoning where hooks-shaped JSON landed in mcp slot because cursor's validateMCPFile is parse-only.

    • cursor: mcpServers → mcp.json; hooks → hooks.json; else skip.
    • claude: mcpServers + (hooks|permissions) → settings.json (real Claude Code combined-file convention); mcpServers alone → --mcp-config; hooks|permissions alone → --settings; else skip.
    • codex: single config.toml, no detection.
  • PR body template gains a "How to evaluate" block with explicit triage rules.

  • RELEASING.md gains "Validator semantics" + "CI on drift PRs" sections under the existing drift-policy heading. The validator-semantics rule is load-bearing:

    testagent <vendor> validate must match what the real vendor binary accepts, even when testagent's own runtime doesn't model every field.

    Runtime > docs when they conflict. Don't tighten the validator to reject things real claude/codex/cursor still accept.

Test plan

  • bash -n + shellcheck clean
  • Smoke-tested detect_slot against 6 representative JSON shapes (cursor mcp, cursor hooks with + without version, claude combined, claude settings-only, junk) — routes as expected
  • After merge: re-dispatch the drift workflow, confirm cursor's open-source-everything PR carries the new triage block and only ships fixtures with matching shapes

🤖 Generated with Claude Code

PR #104's live dispatch exposed corpus poisoning: cursor's
`mcp-upstream-...20921904.json` was hooks-shaped content (no
`mcpServers` key) that landed in the mcp slot because the previous
trial-and-error router took the first slot that didn't error, and
cursor's `validateMCPFile` is parse-only (any JSON that parses is
"valid" mcp).

Replace trial-and-error with shape detection driven by top-level
JSON keys:

- cursor: `mcpServers` → mcp.json; `hooks` → hooks.json; else skip.
- claude: `mcpServers` + (`hooks` | `permissions`) → settings.json
  (the real Claude Code combined-file convention); `mcpServers`
  alone → --mcp-config; `hooks` | `permissions` alone →
  --settings; else skip.
- codex: single config.toml, no detection.

A candidate whose shape doesn't match a known slot is now counted
toward SKIPPED_COUNT instead of laundered into the wrong slot.

PR body template gains a "How to evaluate" triage block, and
RELEASING.md's drift-policy section gains "Validator semantics"
and "CI on drift PRs". The validator-semantics line is the
load-bearing rule for triaging future drift PRs:

> testagent <vendor> validate must match what the real vendor
> binary accepts, even when testagent's own runtime doesn't
> model every field.

Runtime > docs when they conflict. Don't tighten the validator to
reject things real claude/codex/cursor still accept.

Smoke-tested detect_slot against six representative JSON shapes
(cursor mcp, cursor hooks with + without version, claude combined,
claude settings-only, junk); routes correctly in all cases.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@paultyng paultyng marked this pull request as ready for review May 24, 2026 14:38
@paultyng paultyng enabled auto-merge (squash) May 24, 2026 14:38
@paultyng paultyng merged commit bda5625 into main May 24, 2026
3 checks passed
@paultyng paultyng deleted the fix/drift-shape-detect branch May 24, 2026 14:40
paultyng added a commit that referenced this pull request May 24, 2026
## Summary

Re-dispatch after #105 surfaced a third bug: cursor's leg failed with
`non-fast-forward` because the branch
`chore/upstream-drift-cursor-20260524` was left behind on origin when
#104 was closed (GitHub only auto-deletes branches on merge, not close).
The dedupe guard at the top of the script only checks for *open* PRs, so
it didn't catch this.

Add an unconditional `git push --delete origin "$BRANCH"` before the
push. The dedupe guard guarantees we only get here when no open PR
exists, so any leftover remote ref is from a closed/abandoned attempt
and safe to drop.

## Test plan

- [x] `bash -n` + `shellcheck` clean
- [ ] After merge: delete the orphan
`chore/upstream-drift-cursor-20260524` branch one-time, re-dispatch,
confirm all three vendors complete and cursor opens a clean PR.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant