Skip to content

Multi-agent dev/test harness: worker agents, workflows, /pre-pr gate, per-board HIL locks#3762

Merged
hathach merged 3 commits into
masterfrom
claude/agents-workflows
Jul 13, 2026
Merged

Multi-agent dev/test harness: worker agents, workflows, /pre-pr gate, per-board HIL locks#3762
hathach merged 3 commits into
masterfrom
claude/agents-workflows

Conversation

@hathach

@hathach hathach commented Jul 9, 2026

Copy link
Copy Markdown
Owner

Summary

  • Claude Code multi-agent harness for TinyUSB development and testing: five worker agent definitions (.claude/agents/: builder, port-dev, driver-reviewer, hil-operator, pr-monitor), six deterministic workflow scripts (.claude/workflows/: validate, fanout-dev, driver-review, hil-validate, full-check, pr-babysit) plus a syntax checker, and a /pre-pr skill that derives affected boards from the branch diff and runs the composed software+hardware gate.
  • Per-board HIL locks replace the stop-the-runner discipline: test/hil/board_lock.py (background holder process, kernel flocks — auto-release on holder death, holder-signaled acquire, storm-tested) and a fail-open guard in test/hil/hil_test.py that self-locks each board for its flash+test. A locked board fails fast with board locked: <holder JSON> and is re-runnable; HIL_NO_BOARD_LOCK=1 is a user-authorized bypass. The GitHub Actions runner keeps running; CI and dev sessions share the rig per board. CI starts enforcing the guard as soon as this merges (PR CI runs hil_test.py from the merge ref).
  • Docs: design spec, implementation plan, and smoke-test records under docs/superpowers/.

Test evidence

  • validate workflow: unit 61/61 (ceedling); full example sets for stm32f407disco + raspberry_pi_pico (43 examples each); code size +0.0% vs master; PVS gate green vs origin/master.
  • board_lock.py: full lifecycle, kill-9 auto-release, 10/10 concurrency-storm rounds with exactly one winner, lost-race holder info preserved.
  • hil_test.py guard: under a held lock fails in <1 s with no flasher invocation; fail-open verified against an unusable lock dir; py_compile clean.
  • HIL smoke on the real rig (runner active throughout, svc.sh never touched): lock-conflict fail-fast with locked[] surfaced → forced run with the holder surviving → normal self-locking run; board flashed/booted 18×/run. The 5 host-mode test failures observed are a rig fixture issue (pico PIO-USB host-port peers not enumerating after the 2026-07-09 rig rework) — unrelated to this PR; all 13 device-mode tests pass.
  • fanout-dev: parallel 2-port edit, 1-line diffs, git clang-format clean, independently build-verified, reverted.
  • driver-review: 22-agent scan+adversarial-verify run re-found the known rusb2 FRDY-spin wedge class (6 confirmed unbounded busy-waits) with per-finding refutation reasoning.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Rn1AN5DsTdFhRwhugfgKZi

Copilot AI review requested due to automatic review settings July 9, 2026 16:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a Claude Code multi-agent development/test harness for TinyUSB (custom worker agents + deterministic workflow scripts + /pre-pr entry skill) and replaces the “stop the runner” HIL rig discipline with per-board kernel flock locking so CI and dev sessions can safely share hardware concurrently.

Changes:

  • Add .claude/agents/ worker definitions and .claude/workflows/ scripts to orchestrate build/verify/review/HIL/PR-triage tasks.
  • Add per-board HIL locking via test/hil/board_lock.py and a hil_test.py guard that self-locks each board for flash+test.
  • Add design/spec + implementation plan + smoke records under docs/superpowers/, plus updates to the HIL skill docs to reflect the new lock protocol.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
test/hil/hil_test.py Adds per-board lock acquisition around each board’s flash+test run.
test/hil/board_lock.py New CLI utility to hold/release/status per-board advisory locks using kernel flock.
docs/superpowers/specs/2026-07-09-claude-agents-workflows-design.md Design spec for the multi-agent harness and lock protocol.
docs/superpowers/plans/2026-07-09-smoke-results.md Smoke-test notes/results for the harness and HIL lock behavior.
docs/superpowers/plans/2026-07-09-claude-agents-workflows.md Detailed implementation plan for agents/workflows/skill + lock protocol.
.claude/workflows/check.sh Node-based syntax checker for workflow JS (wraps to allow top-level return).
.claude/workflows/validate.js Parallel software gate: unit + per-board builds + size compare + PVS.
.claude/workflows/fanout-dev.js Fan-out development workflow (port-dev per item + optional verify/review).
.claude/workflows/driver-review.js Driver audit workflow with adversarial verification pass.
.claude/workflows/hil-validate.js Serialized HIL runner workflow (supports lock-aware retry and optional force).
.claude/workflows/full-check.js Composes validate then hil-validate (when requested) into one gate.
.claude/workflows/pr-babysit.js Iterative PR triage/fix/verify/push workflow with optional auto-push authorization.
.claude/skills/pre-pr/SKILL.md /pre-pr skill instructions for deriving boards and launching full-check.
.claude/skills/hil/SKILL.md Updates HIL procedure docs to use per-board locks instead of stopping runner.
.claude/agents/builder.md Builder agent contract for example build sweeps (structured JSON output).
.claude/agents/port-dev.md Port developer agent contract (scoped edits + clang-format + build verify).
.claude/agents/driver-reviewer.md Read-only driver reviewer/verifier contract (structured findings/verdicts).
.claude/agents/hil-operator.md Rig operator agent contract (lock protocol, no runner stops, HIL runs).
.claude/agents/pr-monitor.md PR monitor agent contract (CI triage + bot finding validation + drafted replies).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .claude/workflows/validate.js Outdated
Comment on lines +60 to +63
`Compare TinyUSB code size against ${base}: python3 tools/metrics_compare_base.py -b ${args.boards[0]} -e device/cdc_msc . ` +
'The report lands in cmake-metrics/<board>/metrics_compare.md. pass=false only if the tool itself errors; ' +
'detail = the flash/RAM delta summary from the report (mention any example that grew).',
{ label: 'size', phase: 'Validate', model: 'haiku', schema: STAGE },
Comment thread .claude/workflows/pr-babysit.js Outdated
Comment on lines +100 to +104
`Post these threaded replies to review comments on PR #${args.pr}. For each reply: try ` +
`gh api repos/{owner}/{repo}/pulls/${args.pr}/comments/{commentId}/replies -f body=<body> ` +
'(valid for inline review comments); if that 404s, the id is an issue comment — post a regular PR comment instead ' +
`(gh pr comment ${args.pr} --body <quote the original point, then the reply>). ` +
`Replies: ${JSON.stringify(t.replies)}. pass=true only if every reply was posted; detail = what went where.`,
Comment thread test/hil/hil_test.py
Comment on lines +80 to +88
try:
fcntl.flock(fh, fcntl.LOCK_EX | fcntl.LOCK_NB)
except OSError:
try:
info = fh.read(500).strip()
except (OSError, UnicodeDecodeError):
info = ''
fh.close()
raise RuntimeError(f'board locked: {info or "unknown holder"}')

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 186a6876e1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .claude/workflows/validate.js Outdated

if (!skip.includes('size')) thunks.push(() =>
agent(
`Compare TinyUSB code size against ${base}: python3 tools/metrics_compare_base.py -b ${args.boards[0]} -e device/cdc_msc . ` +

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Remove stray positional arg from size check

When the validate workflow reaches the size stage, this prompt tells the worker to run metrics_compare_base.py with a trailing .. That script's argparse definition has no positional arguments, so the command exits with error: unrecognized arguments: .; because the workflow says the stage fails when the tool errors, the default /pre-pr software gate will fail even when tests and builds are clean.

Useful? React with 👍 / 👎.

Comment on lines +11 to +12
- `.claude/skills/usb-recover/SKILL.md` — only when a device/fixture is wedged or processes hang in D state.
- `.claude/skills/usb-debug/SKILL.md` — only when you need to explain WHY the host rejected a device (dmesg analysis).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Add or correct the missing HIL recovery skills

In wedged or USB-debug HIL scenarios, this agent is instructed to read .claude/skills/usb-recover/SKILL.md and .claude/skills/usb-debug/SKILL.md, but I searched the repo (rg "usb-recover|usb-debug" . and the .claude/skills tree) and those skill files are not present. As soon as hil-validate hits the documented unresponsive-board path, the operator has no recovery/debug procedure to follow, so either the files need to be added or these references should point at the existing skill(s).

Useful? React with 👍 / 👎.

Comment thread .claude/workflows/pr-babysit.js Outdated
// Post drafted replies as soon as triage produces them — decoupled from
// fixing/pushing so done/unactionable cycles still post. Outward-facing,
// so gated on autoPush like every other side effect.
if (t.replies.length > 0 && args.autoPush !== false) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Require explicit autoPush before side effects

When pr-babysit is invoked with the documented minimal args ({pr: N}), autoPush is undefined and this condition treats it as enabled, so the workflow can post public replies immediately and later commit/push fixes unless the caller remembered to pass autoPush: false. Since the workflow description says pushes are authorized only with autoPush enabled, this should require autoPush === true for outward side effects rather than making write access the default.

Useful? React with 👍 / 👎.

Comment thread test/hil/hil_test.py Outdated
Comment on lines +75 to +79
fd = os.open(os.path.join(BOARD_LOCK_DIR, f'{board_name}.lock'),
os.O_RDWR | os.O_CREAT, 0o666)
fh = os.fdopen(fd, 'r+')
except OSError:
return None # odd lock dir (perms, path collision): proceed unlocked

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve lock enforcement across Unix users

On rigs where CI and dev sessions run as different Unix users, the first created lock file is still filtered by the process umask (commonly becoming 0644 despite the 0o666 mode), so a later user cannot reopen it with O_RDWR; this except OSError path then returns None and runs HIL completely unlocked. That makes the per-board arbitration fail exactly in cross-session use unless the lock file is chmodded after creation or opened in a way that does not fail open on ordinary permission mismatches.

Useful? React with 👍 / 👎.

? 'THE USER HAS EXPLICITLY AUTHORIZED FORCING: run hil_test.py with HIL_NO_BOARD_LOCK=1 in the environment (bypasses the board lock check; do NOT release or kill the existing holder). '
: 'If the run fails because the board lock is held (a dev session or concurrent CI job), report pass=false and set detail to start EXACTLY with "board locked:" followed by the holder JSON verbatim — never force the lock. ') +
'Reserve the phrase "board locked" strictly for lock contention; describe a frozen or non-enumerating board as "unresponsive" instead. ' +
`Firmware is in examples/cmake-build-${b}. Use the config for this host (hostname first), single-board flag -b ${b}, Bash timeout >= 20 min, never cancel early. ` +

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Route htpc pre-PR HIL runs to the rig

When /pre-pr chooses boards from test/hil/tinyusb.json while running on htpc, this prompt only says to use the host config; the HIL skill defaults htpc to local test/hil/local.json and uses remote ci.lan only when explicitly requested. In that context a rig-only -b board can be absent from the local config, causing hil_test.py to select no boards and return a passing run without exercising the requested hardware, so the workflow needs to explicitly request the remote HIL path for htpc rig-roster boards.

Useful? React with 👍 / 👎.

Comment thread .claude/workflows/validate.js Outdated
Comment on lines +74 to +75
'detail = GA:1/GA:2 counts plus any diagnostics in changed files.',
{ label: 'pvs', phase: 'Validate', model: 'sonnet', effort: 'low', schema: STAGE },

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Fail PVS stage when analysis cannot run

For the default /pre-pr gate, this instruction tells the PVS worker to set pass=false only for GA:1 diagnostics in changed files, so a missing pvs-studio-analyzer, failed CMake build, or analyzer error can be reported as passing because it produces no GA:1 diagnostics. Unlike the size stage above, tool failures need to fail the stage; otherwise the software gate can go green without actually running PVS.

Useful? React with 👍 / 👎.

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown

Size Difference Report

Because TinyUSB code size varies by port and configuration, the metrics below represent the averaged totals across all example builds.

Note: If there is no change, only one value is shown.

Changes >1% in size

No entries.

Changes <1% in size

No entries.

No changes
file .text .rodata .data .bss size % diff
audio_device.c 2890 0 1259 1623 4508 +0.0%
cdc_device.c 1236 16 1092 728 1963 +0.0%
cdc_host.c 6399 487 15 971 7594 +0.0%
dcd_ch32_usbfs.c 1659 0 0 2444 4103 +0.0%
dcd_ch32_usbhs.c 1892 0 0 481 2373 +0.0%
dcd_ci_fs.c 1924 0 0 1290 3214 +0.0%
dcd_ci_hs.c 1756 0 0 1344 2534 +0.0%
dcd_da146xx.c 3067 0 0 144 3211 +0.0%
dcd_dwc2.c 4223 19 0 265 4506 +0.0%
dcd_eptri.c 2273 0 0 259 2532 +0.0%
dcd_ft9xx.c 3284 0 0 172 3456 +0.0%
dcd_khci.c 1952 0 0 1290 3242 +0.0%
dcd_lpc17_40.c 1481 0 0 648 1805 +0.0%
dcd_lpc_ip3511.c 1463 0 0 264 1683 +0.0%
dcd_mm32f327x_otg.c 1477 0 0 1290 2767 +0.0%
dcd_msp430x5xx.c 1801 0 0 176 1977 +0.0%
dcd_musb.c 2595 0 0 179 2773 +0.0%
dcd_nrf5x.c 2939 0 0 292 3231 +0.0%
dcd_nuc120.c 1096 0 0 78 1174 +0.0%
dcd_nuc121.c 1170 0 0 101 1270 +0.0%
dcd_nuc505.c 0 0 1533 157 1690 +0.0%
dcd_rp2040.c 840 0 764 653 2257 +0.0%
dcd_rusb2.c 2918 0 0 156 3074 +0.0%
dcd_samd.c 1036 0 0 266 1302 +0.0%
dcd_samg.c 1322 0 0 72 1394 +0.0%
dcd_stm32_fsdev.c 2543 0 0 291 2834 +0.0%
dfu_device.c 776 28 712 136 912 +0.0%
dfu_rt_device.c 157 0 134 0 157 +0.0%
dwc2_common.c 603 22 0 0 615 +0.0%
ecm_rndis_device.c 1067 0 1 2759 3826 +0.0%
ehci.c 2763 0 0 6274 7783 +0.0%
fsdev_common.c 180 0 0 0 180 +0.0%
hcd_ch32_usbfs.c 2491 0 0 502 2993 +0.0%
hcd_ci_hs.c 181 0 0 0 181 +0.0%
hcd_dwc2.c 5071 25 1 545 5642 +0.0%
hcd_khci.c 2443 0 0 454 2897 +0.0%
hcd_musb.c 3071 0 0 157 3228 +0.0%
hcd_pio_usb.c 262 0 240 0 502 +0.0%
hcd_rp2040.c 1996 17 4 321 2338 +0.0%
hcd_rusb2.c 2923 0 0 245 3168 +0.0%
hcd_samd.c 2220 0 0 324 2544 +0.0%
hcd_stm32_fsdev.c 3248 0 1 420 3670 +0.0%
hid_device.c 1124 44 997 119 1242 +0.0%
hid_host.c 1246 0 0 1270 2516 +0.0%
hub.c 1384 8 8 30 1419 +0.0%
midi2_device.c 3006 34 1342 561 3579 +0.0%
midi2_host.c 1802 0 0 5921 7723 +0.0%
midi_device.c 1149 0 1007 619 1765 +0.0%
midi_host.c 1339 7 7 3538 4880 +0.0%
msc_device.c 2527 108 2293 804 3331 +0.0%
msc_host.c 1633 0 0 395 2028 +0.0%
mtp_device.c 1713 22 743 589 2309 +0.0%
ncm_device.c 1766 28 821 4393 6172 +0.0%
ohci.c 1925 0 0 2503 4428 +0.0%
printer_device.c 828 0 706 560 1387 +0.0%
rp2040_usb.c 386 35 619 11 1051 +0.0%
rusb2_common.c 160 0 16 0 176 +0.0%
tusb.c 457 0 389 3 459 +0.0%
tusb_fifo.c 855 0 486 0 850 +0.0%
typec_stm32.c 1230 8 2 19 1255 +0.0%
usbc.c 500 2 20 166 688 +0.0%
usbd.c 3536 57 90 355 3955 +0.0%
usbh.c 4979 57 82 1165 6250 +0.0%
usbtmc_device.c 2261 24 69 313 2606 +0.0%
vendor_device.c 639 0 534 559 1197 +0.0%
video_device.c 4433 5 1235 480 4905 +0.0%
TOTAL 125566 1053 17222 52144 179274 +0.0%

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown

MemBrowse Memory Report

No memory changes detected across 2404 targets. View Project Dashboard →

Comment thread .claude/workflows/pr-babysit.js Outdated
// Post drafted replies as soon as triage produces them — decoupled from
// fixing/pushing so done/unactionable cycles still post. Outward-facing,
// so gated on autoPush like every other side effect.
if (t.replies.length > 0 && args.autoPush !== false) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pr-babysit performs irreversible, outward-facing actions (push + public PR comments) by default when autoPush is omitted — the opposite of the documented opt-in.

meta.whenToUse says "Invoking with autoPush enabled authorizes pushes to that branch", and the comment right above this line says outward actions are "gated on autoPush like every other side effect" — both read as opt-in (act only when autoPush is enabled).

But autoPush is never defaulted (only maxCycles is, at L12-L14), so an omitted flag stays undefined, and every gate treats undefined as enabled:

  • Post public repliesargs.autoPush !== false is true when undefined, so this agent posts to the PR (L98).
  • Commit + push — the dry-run early-return only fires on an explicit === false (L158-L162); otherwise execution falls through to "commit ALL working-tree changes … then push to the PR's remote branch" (L172-L178).

Net effect: pr-babysit({ pr }) with no autoPush — which the docs say should not authorize pushes — commits, pushes to the remote branch, and posts public review replies. That is a fail-unsafe default for hard-to-reverse, outward-facing actions.

To match the documented opt-in contract, gate on autoPush === true instead of !== false (push/post only when explicitly enabled): use args.autoPush === true at L98, and invert the L159 guard so it dry-runs whenever autoPush !== true. Since the fix spans both gates I've described it rather than attaching a committable suggestion.

@hathach hathach force-pushed the claude/agents-workflows branch from 186a687 to 2de933e Compare July 9, 2026 16:24
Add worker agents (builder, port-dev, driver-reviewer, hil-operator,
pr-monitor), deterministic workflows (validate, fanout-dev, driver-review,
hil-validate, full-check, pr-babysit) and a /pre-pr gate skill, so sessions
can fan build/test/review/PR-triage work out to tiered subagents. pr-babysit
drives a PR to green: triage CI + bot reviews, fix validated findings, verify,
push, and reply-to + resolve each inline review thread (fixed or refuted).

Replace the stop-the-runner HIL discipline with per-board flock locks:
test/hil/board_lock.py plus a fail-open guard in hil_test.py let CI and dev
sessions share the rig per board (locked boards fail fast and re-run;
HIL_NO_BOARD_LOCK=1 is a user-authorized bypass). The actions-runner is
never stopped.

Design spec, implementation plan, and real-rig smoke evidence under
docs/superpowers/.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rn1AN5DsTdFhRwhugfgKZi
@hathach hathach force-pushed the claude/agents-workflows branch from 2de933e to e3dd924 Compare July 9, 2026 16:34

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e3dd9245ef

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .claude/workflows/validate.js Outdated

if (!skip.includes('size')) thunks.push(() =>
agent(
`Compare TinyUSB code size against ${base}: python3 tools/metrics_compare_base.py -b ${args.boards[0]} -e device/cdc_msc . ` +

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Pass custom base to size comparison

When /pre-pr or full-check is invoked with a user-named base other than master, this stage still tells the worker to run metrics_compare_base.py without the script's --base-branch option (confirmed via python3 tools/metrics_compare_base.py --help: --base-branch BASE_BRANCH), so the size verdict is produced against the tool default master even though the prompt says ${base}. That can hide or invent size regressions for release/backport PRs; pass ${base} through to the command.

Useful? React with 👍 / 👎.

Comment thread .claude/workflows/pr-babysit.js Outdated
Comment on lines +87 to +88
'`gh api graphql -f query=\'query($o:String!,$r:String!,$p:Int!){repository(owner:$o,name:$r){pullRequest(number:$p){reviewThreads(first:100){nodes{id isResolved comments(first:50){nodes{databaseId}}}}}}}\' -F o=OWNER -F r=REPO -F p=' + args.pr + '` ' +
'(paginate with the endCursor if there are more than 100 threads), pick the thread whose comments contain that databaseId, then resolve it with ' +

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include pageInfo in thread pagination query

For PRs with more than 100 review threads, this recipe cannot actually follow the stated pagination path: the query caps reviewThreads(first:100) but does not request pageInfo { hasNextPage endCursor } or accept an after cursor. GitHub's GraphQL pagination docs describe endCursor and hasNextPage as coming from pageInfo (https://docs.github.com/en/graphql/guides/using-pagination-in-the-graphql-api), so comments beyond the first page will not be found or resolved.

Useful? React with 👍 / 👎.

Comment thread .claude/agents/port-dev.md Outdated

## Finish checklist (in order)

1. Format only the lines you changed: `git clang-format` (no args — formats working-tree changes vs HEAD using the repo `.clang-format`). If it reformats anything, re-check your diff still builds.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Limit formatting to the assigned scope

In the default fanout-dev path, multiple port-dev workers edit the same checkout in parallel, but git clang-format with no paths operates on all working-tree lines that differ from HEAD (confirmed via git clang-format -h). When two scopes are active, one worker can format or race over another worker's in-progress changes despite the touch nothing outside it contract, producing cross-scope edits or nondeterministic diffs; pass the assigned scope to git clang-format or force worktree isolation.

Useful? React with 👍 / 👎.

Comment on lines +185 to +187
const push = await agent(
`On the current PR branch: commit ALL working-tree changes as ONE commit (imperative message summarizing the cycle-${cycle} fixes for PR #${args.pr}, repo commit conventions), ` +
"then push to the PR's remote branch. pass=true only if commit AND push succeeded; detail = pushed SHA.",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid committing unrelated dirty work

When pr-babysit runs from a PR checkout that already has uncommitted local edits (for example after a previous dry-run or failed-verification cycle left changes behind), this push step tells the agent to commit ALL working-tree changes. The workflow never checks for a clean tree before triage, so autoPush can publish unrelated files together with the current verified fixes; abort on a dirty pre-state or commit only the files tied to verified fixes.

Useful? React with 👍 / 👎.


- `BASE` = `master` unless the user names another base.
- `git diff --name-only $(git merge-base HEAD $BASE)..HEAD`
- If NO C sources changed (only docs / `.claude/` / tools): say so, and run a minimal software-only gate — `boards = [stm32f407disco]`, no HIL — unless the user asks for more.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Exercise HIL when the HIL harness changes

When a branch changes only non-C HIL code such as test/hil/hil_test.py or test/hil/board_lock.py, this rule matches because no C sources changed and it explicitly selects a minimal software-only gate with no HIL. That skips the hardware run that would exercise the changed harness/locking behavior, and the later catch-all only covers C/CMake source changes, so these Python HIL changes never reach hilBoards.

Useful? React with 👍 / 👎.

Comment on lines +24 to +27
2. Verify with a targeted build of `device/cdc_msc` for the board named in your task (or pick one from `hw/bsp/<family>/boards/` whose family uses your scope). Use a unique build dir to survive parallel siblings:
```bash
BUILD=$(mktemp -d /tmp/portdev-<BOARD>-XXXX)
cmake -S examples/device/cdc_msc -B "$BUILD" -DBOARD=<BOARD> -G Ninja -DCMAKE_BUILD_TYPE=MinSizeRel && cmake --build "$BUILD"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Verify host-controller edits with a host build

When the assigned scope is an HCD or other host-only path, this checklist still verifies only device/cdc_msc, which can leave the changed host source uncompiled (for example RP2040 CMake puts hcd_rp2040.c in the host library while device examples link the device side). A broken host-controller edit can therefore report buildOk: true; choose a host example for host scopes or require both device and host builds when the scope is ambiguous.

Useful? React with 👍 / 👎.


## Bot review harvest

- Inline review comments: `gh api repos/{owner}/{repo}/pulls/<N>/comments --paginate` (use `gh repo view --json nameWithOwner -q .nameWithOwner` for owner/repo). Issue comments: `gh pr view <N> --comments`.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Query thread resolution before reprocessing comments

In multi-cycle pr-babysit runs, resolved inline threads are still returned by the REST PR review-comments listing used here; that endpoint lists comments rather than unresolved review threads, and the resolution state lives on GraphQL reviewThreads.isResolved. After a prior cycle replies to and resolves a bot finding, the next triage can treat the same resolved comment as unresolved again, causing repeated fix/reply work unless the harvest starts from unresolved GraphQL threads or filters resolved ones.

Useful? React with 👍 / 👎.

@claude

claude Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown

Hardware-in-the-loop (HIL) Test Report

hfp-iar

✅ 52 passed · ❌ 0 failed · ⚪ 0 skipped · blank not run

Board cdc_dual_ports cdc_msc dfu cdc_msc_throughput audio_test_freertos dfu_runtime cdc_msc_freertos hid_boot_interface msc_dual_lun hid_generic_inout printer_to_cdc midi_test mtp
stm32l412nucleo ✅ CDC 627k/389k MSC 803k/749k
stm32f746disco ✅ CDC 13.2M/9.9M MSC 15.6M/26M
stm32f746disco-DMA ✅ CDC 12.8M/9.2M MSC 12.8M/30.6M
lpcxpresso43s67 ✅ CDC 11.6M/11.4M MSC 33.2M/38.9M

hfp.json

✅ 52 passed · ❌ 0 failed · ⚪ 0 skipped · blank not run

Board cdc_dual_ports cdc_msc dfu cdc_msc_throughput audio_test_freertos dfu_runtime cdc_msc_freertos hid_boot_interface msc_dual_lun hid_generic_inout printer_to_cdc midi_test mtp
stm32l412nucleo ✅ CDC 637k/385k MSC 838k/821k
stm32f746disco ✅ CDC 13.3M/10.2M MSC 13.9M/29.7M
stm32f746disco-DMA ✅ CDC 13.5M/9.6M MSC 11.7M/24.2M
lpcxpresso43s67 ✅ CDC 10.4M/10.7M MSC 27.2M/34.8M

tinyusb.json

✅ 229 passed · ❌ 110 failed · ⚪ 11 skipped · blank not run

Board cdc_dual_ports cdc_msc dfu cdc_msc_throughput audio_test_freertos dfu_runtime cdc_msc_freertos hid_boot_interface msc_dual_lun hid_generic_inout printer_to_cdc midi_test mtp host_info_to_device_cdc cdc_msc_hid msc_file_explorer msc_file_explorer_freertos device_info hid_composite_freertos
ek_tm4c123gxl ✅ CDC 514k/512k MSC 511k/491k
espressif_p4_function_ev rd 409KB/s
espressif_p4_function_ev-DMA rd 409KB/s
espressif_s3_devkitm rd 409KB/s
espressif_s3_devkitm-DMA rd 409KB/s
feather_nrf52840_express ✅ CDC 493k/514k MSC 601k/561k
max32666fthr ✅ CDC 7M/12.4M MSC 4.1M/19.5M
metro_m4_express ✅ CDC 609k/590k MSC 644k/635k
mimxrt1015_evk ✅ CDC 20.9M/14.4M MSC 31.2M/25.6M
mimxrt1064_evk ✅ CDC 21M/11.5M MSC 31.1M/27.2M rd 1368KB/s rd 1365KB/s
lpcxpresso11u37 ✅ CDC 399k/281k MSC 589k/605k
ra4m1_ek
raspberry_pi_pico ✅ CDC 508k/483k MSC 511k/512k rd 62KB/s rd 62KB/s
raspberry_pi_pico_w rd 1106KB/s rd 1022KB/s
raspberry_pi_pico2
adafruit_fruit_jam ✅ CDC 508k/511k MSC 511k/511k rd 62KB/s rd 62KB/s
stm32f072disco
stm32f407disco
stm32f723disco ✅ CDC 654k/629k MSC 648k/605k rd 16384KB/s rd 3942KB/s
stm32f723disco-DMA ✅ CDC 508k/511k MSC 511k/511k rd 15420KB/s rd 4032KB/s
stm32h743nucleo
stm32h743nucleo-DMA
stm32g0b1nucleo
stm32l476disco
stm32u083nucleo
nanoch32v203-fsdev ✅ CDC 508k/514k MSC 511k/511k
nanoch32v203-usbfs ✅ CDC 465k/343k MSC 511k/511k
ch32v103r_r1_1v0 ✅ CDC 513k/448k MSC 517k/523k
ch582m_evt ✅ CDC 231k/205k MSC 479k/474k

Review-fix batch (owner-confirmed) on the multi-agent harness:

- board_lock: detach holder stdio so a captured `hold` cannot hang on the
  daemon's inherited pipe; probe locks by holder-pid liveness instead of a
  momentary flock, which could spuriously fail a concurrent acquirer
  (storm-tested: 1 winner in 10, 0/15 acquire failures under probe storm)
- hil_test: locked board renders a visible board-locked fail row so the
  report matches the exit code; stale marker cleared on a real re-run
- pr-babysit: autoPush now opt-in (default dry run); resolve recipe
  paginates reviewThreads; post-push resolve gets issue-comment fallback
- validate: size stage honors non-default base via --base-branch; pvs
  stage delegated to the new agent
- new static-analyzer agent (sonnet): PVS-Studio SAST+MISRA for one
  board, structured findings gated on files changed vs base

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rn1AN5DsTdFhRwhugfgKZi

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0557655afb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread test/hil/board_lock.py
if not isinstance(pid, int) or pid <= 0:
return False
try:
os.kill(pid, 0)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Treat zombie lock holders as released

When board_lock.py release SIGTERMs the detached holder, that process can remain as a zombie if PID 1 does not reap it promptly (common in containers). os.kill(pid, 0) still succeeds for zombies, so is_locked() keeps reporting the stale lock file as held even though the kernel flock was released; subsequent release reports ERROR: still locked and future hold calls refuse the board. Check the process state or the flock itself instead of treating any live PID as an active holder.

Useful? React with 👍 / 👎.

Comment thread test/hil/board_lock.py
pids.add(info['pid'])
for holder in sorted(pids):
try:
os.kill(holder, signal.SIGTERM)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid killing a reused stale holder PID

Because lock files are intentionally left behind after a holder exits, a later release can read an old JSON pid after the OS has reused it for an unrelated process. In that case is_locked() treats the board as held and this line sends SIGTERM to whatever now owns that pid, even though the flock was already released; this can terminate an unrelated job on the rig. Revalidate the flock/current holder before killing, or remove/ignore stale records rather than trusting the stored pid alone.

Useful? React with 👍 / 👎.

Confirmed by a 10-finder / 28-verifier adversarial review pass:

board_lock.py — the flock is now the sole authority: drop cmd_hold's
pid-liveness pre-gate (a live hil_test.py pool worker's stale record no
longer blocks a genuinely free board); cmd_release probes the flock and
only signals a verified holder, refuses to kill hil_test.py holders
(CI mid-test), handles PermissionError; the holder daemon truncates its
lock records on SIGTERM and keeps the success pipe clear of fds 0-2
(closed-stdio hold used to leave an orphan holder while reporting
failure); --config default resolves beside the script.

hil_test.py — truncate the lock record on per-board release (pool
workers outlive their flocks); warn instead of silently failing open
when the lock dir is unusable; error out on -b names absent from the
config (was a silent zero-test exit 0, readable as a green HIL run);
drop an emptied board row in accumulate_report (variant boards left a
blank ghost row).

workflows — remove the stray positional arg that made the validate size
stage exit 2 on every run; wrap JSON.parse(args) in all six scripts;
factor pr-babysit's drifted reply recipe into postReplyRecipe and dedup
refutation replies across cycles; validate args.pr and maxCycles;
driver-review rejects an empty dimensions list; hil-validate drops a
dead guard clause and retries diagnostics with -v -r 1.

agents/docs — port-dev scopes git clang-format to its own files
(concurrent workers reformatted each other in shared checkouts);
hil-operator/hil skill wording matches actual fail-fast output; the
implementation plan is now a DO-NOT-EXECUTE historical record (banner +
checked boxes) so plan-executing agents cannot revert shipped files.

Verified: lock storm 1-winner-in-10, stale-record hold, closed-stdio
hold, dead-pid cleanup, CI-holder refusal, ghost-row 4-scenario merge,
unknown-board exit 1, py_compile + check.sh on all six workflows,
pre-commit clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Upj4hta5TNoAbidqeC1zZ6

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ff69550b3d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

)
// attempted counts as replied: better to drop a failed reply than spam duplicates
freshReplies.forEach(r => repliedIds.add(r.commentId))
if (!posted || !posted.pass) log(`cycle ${cycle}: refuted reply/resolve incomplete — ${posted ? posted.detail : 'agent died'}`)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Fail the cycle when reply resolution fails

When autoPush is true and triage has only invalid/stale bot findings, a failed reply/resolve agent is only logged here; if t.done is also true, the immediately following branch returns pass: true even though the public reply was not posted and the inline thread was not resolved. In that scenario pr-babysit can report the PR green while review work it explicitly attempted is still incomplete, so this should fail the cycle or make the success return depend on the reply/resolve result.

Useful? React with 👍 / 👎.

@hathach hathach merged commit ac595bc into master Jul 13, 2026
327 of 328 checks passed
@hathach hathach deleted the claude/agents-workflows branch July 13, 2026 15:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants