feat: add supermodel skill command#125
Closed
jonathanpopham wants to merge 2 commits intosupermodeltools:mainfrom
Closed
feat: add supermodel skill command#125jonathanpopham wants to merge 2 commits intosupermodeltools:mainfrom
jonathanpopham wants to merge 2 commits intosupermodeltools:mainfrom
Conversation
Revised the generic skill prompt based on benchmark trace analysis. Three changes: teach the .graph naming convention so agents construct paths directly, bold the read-order directive, and tell agents to check graph files before grepping for structure. Skill v2: $0.11, 31s, 7 turns (was $0.15, 42s, 11 turns) Matches Grey's hand-crafted Django prompt: $0.12, 29s, 9 turns
Locks in the six key elements that drove benchmark results: graph extension, three section names, naming convention example, and read-order directive.
WalkthroughThis PR introduces a new Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 1 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (1 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
cmd/skill.go (1)
9-19: Keep prompt content single-sourced to avoid doc drift.
skillPromptis effectively duplicated withbenchmark/CLAUDE.skill.md. Consider enforcing exact parity (or generating one from the other) so future edits don’t silently diverge.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@cmd/skill.go` around lines 9 - 19, The string constant skillPrompt is duplicated in benchmark/CLAUDE.skill.md; consolidate to a single source of truth by loading the prompt text from one canonical file instead of hardcoding it in cmd/skill.go (or by generating benchmark/CLAUDE.skill.md from skillPrompt at build time). Update cmd/skill.go to read the canonical prompt (e.g., from benchmark/CLAUDE.skill.md) into the skillPrompt variable (or add a build step that writes the canonical file), and add a small test or CI check that verifies exact parity between skillPrompt and benchmark/CLAUDE.skill.md to prevent future drift.cmd/skill_test.go (1)
8-32: Add one behavior test for the command path, not just the constant.Right now tests only assert
skillPrompt. Add a test that executesskilland asserts output contains the prompt so command wiring regressions are caught too.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@benchmark/results/summary.md`:
- Around line 12-14: Update the stale "skill (generic)" benchmark entries in the
results table: replace the old values ($0.15, 42s, 11 turns) with the PR-stated
results ($0.11, 31s, 7 turns) in the row for "skill (generic)" and then
recompute and update the derived comparison line (the comparison column that
references those values) so the cost, duration, and turns reflect the new
numbers consistently throughout the table.
---
Nitpick comments:
In `@cmd/skill.go`:
- Around line 9-19: The string constant skillPrompt is duplicated in
benchmark/CLAUDE.skill.md; consolidate to a single source of truth by loading
the prompt text from one canonical file instead of hardcoding it in cmd/skill.go
(or by generating benchmark/CLAUDE.skill.md from skillPrompt at build time).
Update cmd/skill.go to read the canonical prompt (e.g., from
benchmark/CLAUDE.skill.md) into the skillPrompt variable (or add a build step
that writes the canonical file), and add a small test or CI check that verifies
exact parity between skillPrompt and benchmark/CLAUDE.skill.md to prevent future
drift.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: cad1440b-55c3-4210-9196-8e0c138d87b5
⛔ Files ignored due to path filters (1)
benchmark/results/benchmark_results.zipis excluded by!**/*.zip
📒 Files selected for processing (6)
benchmark/CLAUDE.skill.mdbenchmark/results/blog-post-draft.mdbenchmark/results/skill-v2.txtbenchmark/results/summary.mdcmd/skill.gocmd/skill_test.go
Comment on lines
+12
to
+14
| | Cost | $0.30 | $0.12 | $0.15 | $0.25 | | ||
| | Turns | 20 | 9 | 11 | 16 | | ||
| | Duration | 122s | 29s | 42s | 73s | |
There was a problem hiding this comment.
skill (generic) benchmark values look stale against this PR’s stated results.
Line 12-Line 14 and Line 18 still show the older run ($0.15, 42s, 11 turns). The PR objective states the updated result is $0.11, 31s, 7 turns, so the derived comparison line is also off.
📌 Suggested update
-| Cost | $0.30 | $0.12 | $0.15 | $0.25 |
-| Turns | 20 | 9 | 11 | 16 |
-| Duration | 122s | 29s | 42s | 73s |
+| Cost | $0.30 | $0.12 | $0.11 | $0.25 |
+| Turns | 20 | 9 | 7 | 16 |
+| Duration | 122s | 29s | 31s | 73s |
-**skill (generic prompt): 50% cheaper, 66% faster, 45% fewer turns vs naked**
+**skill (generic prompt): 63% cheaper, 75% faster, 65% fewer turns vs naked**Also applies to: 18-18
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@benchmark/results/summary.md` around lines 12 - 14, Update the stale "skill
(generic)" benchmark entries in the results table: replace the old values
($0.15, 42s, 11 turns) with the PR-stated results ($0.11, 31s, 7 turns) in the
row for "skill (generic)" and then recompute and update the derived comparison
line (the comparison column that references those values) so the cost, duration,
and turns reflect the new numbers consistently throughout the table.
Contributor
|
Closing in favor of a new PR from the rebased origin branch that includes Windows CI fixes (TMP/TEMP env vars and filepath separators). The code changes from this PR are preserved in supermodeltools:feat/skill-command. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
supermodel skill— prints a generic prompt teaching AI agents to use.graph.*filesBenchmark (Django, 8 failing tests)
supermodel skill(this PR)Test plan
go build ./...passesgo test ./cmd/ -run TestSkillpassesSummary by CodeRabbit
Release Notes
New Features
skillcommand that provides guidance on navigating code relationships using.graphfiles.Documentation
.graphfile format, naming conventions, and usage guidance.