Skip to content

fix(skill_eval): conflict guard + descriptionOverride normalization (reworks #28)#32

Merged
antongulin merged 1 commit into
mainfrom
fix/skill-eval-conflict-guard
Jul 5, 2026
Merged

fix(skill_eval): conflict guard + descriptionOverride normalization (reworks #28)#32
antongulin merged 1 commit into
mainfrom
fix/skill-eval-conflict-guard

Conversation

@antongulin

Copy link
Copy Markdown
Owner

Reworks #28 (credit @mxl for the original report and implementation — co-authored on the commit).

What changed

  • Installed-skill conflict guard: skill_eval and skill_optimize_loop now fail fast with a clear error when a skill with the same base name is already visible to opencode (installed skill or skills.paths entry), since it can steal triggers from the synthetic eval skill and produce false negatives.
  • Unlike fix(skill_eval): add conflict guard and normalize descriptionOverride #28, the guard runs opencode debug skill through the shared runProcess helper — 10s timeout (no hang if the CLI stalls) and no Bun globals in compiled output (package.test.mjs enforces this; direct Bun.spawn was why fix(skill_eval): add conflict guard and normalize descriptionOverride #28's package tests failed after rebuild).
  • Parsing is extracted into a pure findSkillConflicts() with unit tests (conflict found, missing location, no match, invalid JSON, non-array).
  • descriptionOverride normalization: empty/whitespace-only strings are treated as omitted in both tools.
  • triggerOnly gating fix (follow-up to fix(skill_eval): add trigger-only mode #29): the early success return now respects triggerOnly: false — a triggered run that ends in a failed process throws again, matching the flag's documented behavior.
  • Rebuilt dist.

Verification

  • npm run build clean
  • npm test: 24/24 pass
  • npm run test:ts: 49/49 pass
  • grep -c '\bBun\b' dist/skill-creator.js → 0

Closes #28.

🤖 Generated with Claude Code

…escriptionOverride

Reworks #28: the conflict guard now uses the shared runProcess helper
(10s timeout, no Bun globals in compiled output) with a pure, unit-tested
parser for `opencode debug skill` output. Empty descriptionOverride strings
are treated as omitted in both skill_eval and skill_optimize_loop.

Also gates the triggerOnly early return from #29 so a triggered run that
ends in a failed process still throws when triggerOnly is false.

Co-authored-by: Michael Ledin <mledin89@gmail.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

🏹 Robin

✅ Review done. I flagged 4 things worth a look.

Push fixes whenever you like, then comment /robin for another pass.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🏹 Robin

Heads up: this is a point-in-time review. Push fixes freely, then comment /robin whenever you want another pass.

⚠️ 1 Medium | 🔵 2 Low | 💡 1 Suggestions

Summary

The diff adds a conflict guard to prevent installed skills from shadowing eval-created synthetic skills, which is a solid defensive measure. The normalizeDescriptionOverride helper and the findSkillConflicts unit tests are well done. The main concern is a behavioral change in runSingleQuery that alters the early-return condition, which could affect eval results in non-triggerOnly mode if not carefully validated.


Robin — the Robin Hood of code review. Free for every PR.

Comment thread plugin/lib/run-eval.ts
Comment thread plugin/lib/run-eval.ts
Comment thread plugin/test/conflict-guard.test.ts
Comment thread plugin/lib/run-eval.ts
@antongulin antongulin merged commit 78888a8 into main Jul 5, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant