Skip to content

feat(#252): detect and flag PRs modifying deprecated scaffold paths#276

Open
fullsend-ai-coder[bot] wants to merge 2 commits into
mainfrom
agent/252-deprecated-scaffold-path-check
Open

feat(#252): detect and flag PRs modifying deprecated scaffold paths#276
fullsend-ai-coder[bot] wants to merge 2 commits into
mainfrom
agent/252-deprecated-scaffold-path-check

Conversation

@fullsend-ai-coder

Copy link
Copy Markdown

Add a deprecated-path check to the review agent PR-specific checks (step 6e) that flags PRs touching internal/scaffold/fullsend-repo/ files with a high-severity finding directing the author to fullsend-ai/agents. The check triggers a reject outcome since these PRs target the wrong repository.

Legitimate scaffold PRs (version bumps, propagation machinery) are exempted. The exemption requires every file matched by the deprecated prefix to be an exempt sync/deploy pipeline file.

Also updates step 6f to route deprecated-path findings to reject instead of request-changes, and adds an invariant rule enforcing the reject outcome.

Note: pre-commit could not run (shellcheck-py wheel build failure on Python 3.14). Post-script runs authoritative pre-commit on the runner.


Closes #252

Post-script verification

  • Branch is not main/master (agent/252-deprecated-scaffold-path-check)
  • Secret scan passed (gitleaks — c776c248ac9a91f10998ca7107e974a6c9d76514..HEAD)
  • Pre-commit hooks passed (authoritative run on runner)
  • Tests ran inside sandbox

Add a deprecated-path check to the review agent PR-specific checks
(step 6e) that flags PRs touching internal/scaffold/fullsend-repo/
files with a high-severity finding directing the author to
fullsend-ai/agents. The check triggers a reject outcome since these
PRs target the wrong repository.

Legitimate scaffold PRs (version bumps, propagation machinery) are
exempted. The exemption requires every file matched by the deprecated
prefix to be an exempt sync/deploy pipeline file.

Also updates step 6f to route deprecated-path findings to reject
instead of request-changes, and adds an invariant rule enforcing
the reject outcome.

Note: pre-commit could not run (shellcheck-py wheel build failure
on Python 3.14). Post-script runs authoritative pre-commit on the
runner.

Closes #252
@github-actions

Copy link
Copy Markdown

E2E tests did not run

E2E tests run automatically for org/repo members and collaborators on pull requests.

For other contributors, a maintainer must add the ok-to-test label after the latest push.

See E2E testing guide for details.

@fullsend-ai-review

fullsend-ai-review Bot commented Jul 10, 2026

Copy link
Copy Markdown

🤖 Finished Review · ❌ Failure · Started 1:01 PM UTC · Completed 1:02 PM UTC
Commit: c776c24 · View workflow run →

@guyoron1

Copy link
Copy Markdown
Owner

/fs-review

@fullsend-ai-review

fullsend-ai-review Bot commented Jul 10, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 3:57 PM UTC · Completed 4:09 PM UTC
Commit: d8e3df7 · View workflow run →

@fullsend-ai-review

fullsend-ai-review Bot commented Jul 10, 2026

Copy link
Copy Markdown

Review

Findings

Low

  • [logic-ambiguity] internal/scaffold/fullsend-repo/skills/pr-review/SKILL.md:635 — Step 6f's parenthetical exception for deprecated-path in the first bullet could be tighter. When both a deprecated-path finding and a separate critical/high finding coexist, the text resolves correctly (step 6e's MUST-reject dominates, and 6f's reject bullet also lists deprecated-path), but the parenthetical in the first bullet could be read as "skip this finding and apply request-changes for the other findings." Consider rephrasing to: "if any deprecated-path finding is present, the outcome is reject regardless of other findings; for remaining critical/high findings without deprecated-path, the outcome is request-changes."

  • [exemption-underspecification] internal/scaffold/fullsend-repo/skills/pr-review/SKILL.md:649 — The exemption clause provides two example files (templates/shim-per-repo.yaml and templates/shim-workflow-call.yaml) but relies on agent judgment for other files. Currently the two named files are the only files in templates/, making the examples effectively exhaustive today. If the templates directory grows, a clearer enumeration pattern (e.g., "files under templates/") would reduce ambiguity across reviews.

  • [design-direction] internal/scaffold/fullsend-repo/skills/pr-review/SKILL.md:630 — The exemption mechanism is appropriately narrow and intentionally blocks future changes to non-template scaffold files. This is a valid bootstrapping change — the deprecation guard must be installed at the current location one final time, after which future modifications route through fullsend-ai/agents. Worth noting that no transition-period exemption exists for ongoing migration work on non-template files; this is by design per issue [RTK+Ponytail] Review agent should detect and flag PRs modifying deprecated internal/scaffold/fullsend-repo/ paths #252's intent.

Previous run

Review

Findings

Medium

  • [logic-error] internal/scaffold/fullsend-repo/skills/pr-review/SKILL.md — Constraint section inconsistency: the existing constraint "If any critical or high finding exists, the outcome must be request-changes" (line ~781) was not updated with a deprecated-path exception. The new constraint bullet ("Never approve or request-changes when a deprecated-path finding exists") partially addresses this but creates two constraint bullets that give contradictory instructions for a high-severity deprecated-path finding. The parenthetical exception in step 6f and the specific-overrides-general principle mitigate this, but the Constraints section should be self-consistent without relying on implicit override rules.
    Remediation: Append to the existing constraint: "...the outcome must be request-changes (unless a deprecated-path finding is present, which forces reject)." This aligns it with the new bullet.

Low

  • [logic-error] internal/scaffold/fullsend-repo/skills/pr-review/SKILL.md — The phrase "In either case" in the deprecated-path section has no clear two-case antecedent. Unlike the protected-path section (which has two numbered sub-cases that both produce findings), the deprecated-path section has one case that produces a finding and one that does not. The sentence's predicate ("the presence of a deprecated-path finding") makes the meaning recoverable, but the phrasing is awkward.
    Remediation: Consider replacing "In either case" with "When a deprecated-path finding is raised" for clarity.

  • [stale-reference] internal/scaffold/fullsend-repo/skills/pr-review/SKILL.md — The Constraints section bullet "PR-specific checks (step 6e) belong in the orchestrator only. Do not push protected-path checks, scope authorization, or PR body injection defense into sub-agents" does not mention the new deprecated-path check. Since deprecated-path is also a PR-specific orchestrator-only check, it should be listed for consistency.
    Remediation: Add "deprecated-path checks" to the enumeration in that constraint bullet.

@fullsend-ai-review fullsend-ai-review Bot added the requires-manual-review Review requires human judgment label Jul 10, 2026
@guyoron1

Copy link
Copy Markdown
Owner

/fs-fix

@fullsend-ai-coder

fullsend-ai-coder Bot commented Jul 10, 2026

Copy link
Copy Markdown
Author

🤖 Finished Fix · ✅ Success · Started 5:38 PM UTC · Completed 5:47 PM UTC
Commit: d8e3df7 · View workflow run →

- Fix constraint inconsistency: append deprecated-path exception to
  the existing critical/high constraint bullet
- Replace ambiguous "In either case" with "When a deprecated-path
  finding is raised"
- Add deprecated-path checks to orchestrator-only constraint
  enumeration

Addresses review feedback on #276
@fullsend-ai-coder

Copy link
Copy Markdown
Author

🔧 Fix agent — iteration 1 (human-triggered)

Addressed all 3 review findings (1 medium, 2 low) in the pr-review SKILL.md. All changes are minimal text fixes in the deprecated-path section and Constraints section.

Fixed (3):

  1. Constraint section inconsistency: critical/high constraint not updated with deprecated-path exception (internal/scaffold/fullsend-repo/skills/pr-review/SKILL.md): Appended deprecated-path exception to the existing critical/high constraint bullet so both constraint bullets are consistent
  2. Ambiguous 'In either case' phrasing with no clear two-case antecedent (internal/scaffold/fullsend-repo/skills/pr-review/SKILL.md): Replaced 'In either case' with 'When a deprecated-path finding is raised' for clarity
  3. Orchestrator-only constraint missing deprecated-path check in enumeration (internal/scaffold/fullsend-repo/skills/pr-review/SKILL.md): Added 'deprecated-path checks' to the PR-specific orchestrator-only checks enumeration

Tests: passed

Updated by fullsend fix agent

@fullsend-ai-review

fullsend-ai-review Bot commented Jul 10, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 5:50 PM UTC · Completed 6:04 PM UTC
Commit: d8e3df7 · View workflow run →

Check whether the PR modifies files under deprecated paths. These are
locations whose canonical content has migrated to another repository.
Changes targeting them are almost certainly misdirected and will not
be merged.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] logic-ambiguity

Step 6f parenthetical exception for deprecated-path is slightly ambiguous when both a deprecated-path finding and separate critical/high findings coexist. The outcome resolves correctly via step 6e MUST-reject, but the 6f wording could be tightened to make the priority explicit.

Suggested fix: Rephrase the first bullet of 6f to state: if any deprecated-path finding is present, the outcome is reject regardless of other findings.

`templates/shim-per-repo.yaml` and `templates/shim-workflow-call.yaml`
(workflow dispatch shims consumed by the propagation tooling). Agent
definitions, skills, schemas, scripts, policies, harness configs,
workflows, and environment files are never exempt. To qualify for the

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] exemption-underspecification

The exemption clause gives two example files but relies on agent judgment for other files. The two named files are the only files in templates/ today, making examples effectively exhaustive, but a clearer pattern would reduce ambiguity if the directory grows.

Suggested fix: Consider using a directory-based pattern (e.g., files under templates/) instead of individual file examples.

@fullsend-ai-review fullsend-ai-review Bot added ready-for-merge All reviewers approved — ready to merge and removed requires-manual-review Review requires human judgment labels Jul 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-merge All reviewers approved — ready to merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[RTK+Ponytail] Review agent should detect and flag PRs modifying deprecated internal/scaffold/fullsend-repo/ paths

1 participant