Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces local skill tooling via skills-package-manager, adds a new (linked) skill entry, and begins adding an evaluation fixture for the migrate-to-rsbuild skill.
Changes:
- Add
skills-package-managerconfiguration (skills.json) and lockfile (skills-lock.yaml) - Run
skills-package-manager installduringprepareand add the dependency topackage.json - Add initial
skills-test/migrate-to-rsbuildeval inputs and a stub report
Reviewed changes
Copilot reviewed 6 out of 9 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| skills.json | Adds skills-package-manager config and skill sources/links |
| skills-lock.yaml | Introduces skills lockfile (currently for skill-creator) |
| package.json | Adds skills-package-manager devDependency and runs install in prepare |
| pnpm-lock.yaml | Locks new dependency tree for skills-package-manager and transitive deps |
| dev-skills/rstack-skill-evaluator/SKILL.md | New linked skill file (currently empty) |
| skills-test/migrate-to-rsbuild/eval.json | Adds evaluation cases for migrate-to-rsbuild |
| skills-test/migrate-to-rsbuild/report.md | Adds benchmark report file (currently placeholder) |
| .prettierignore | Ignores skills-lock.yaml (and keeps pnpm-lock.yaml ignored) |
| .gitignore | Ignores installed skills dirs and test iteration outputs |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Addressed in 002cf4b (follow-up to b233bb5). I ran |
|
@copilot 把这里关于 skills-package-manager setup 的部分单独分成一个单独的 PR,“add rstack-skill-evaluator skill ” 的部分不要 |
fi3ework
left a comment
There was a problem hiding this comment.
is the reason fixtures aren't tracked in the repository to prevent fixture rot?
by the way, need this feature to track the eval report of Rstest skill, merge merge merge.
fixture 理论上维护在 rstack-examples 仓库会降低维护成本,放在 skills 仓库的话会让这个仓库变得很大,skill 下载速度变慢 Theoretically, maintaining the fixture in the rstack-examples repository would reduce maintenance costs, while placing it in the skills repository would make that repository very large, slowing down the download speed of skills. |
shouldn't it be rspack-examples, especially since we're dealing with many migration cases? but we can indeed maintain it in another repository and directly execute |
|
@copilot rebase to the latest origin/main |
Co-authored-by: SoonIter <79413249+SoonIter@users.noreply.github.com>
Co-authored-by: SoonIter <79413249+SoonIter@users.noreply.github.com>
Co-authored-by: SoonIter <79413249+SoonIter@users.noreply.github.com>
Agent-Logs-Url: https://github.com/rstackjs/agent-skills/sessions/53d11aad-f144-41c7-97bb-6868e435a1be Co-authored-by: SoonIter <79413249+SoonIter@users.noreply.github.com>
Addressed in 4af0a5a. This branch is now rebased on the latest
Done in 4af0a5a. The branch is rebased onto the latest |
4af0a5a to
f7e2ab9
Compare
| name: rstack-skill-evaluator | ||
| description: Benchmark agent skills by generating evaluation cases, comparing skill-guided and baseline runs, and recording the resulting artifacts under skills-test/{skill-name}. | ||
| metadata: | ||
| dependencies: ['skill-creator'] |
There was a problem hiding this comment.
| dependencies: ['skill-creator'] | |
| dependencies: ['skill-creator'] | |
| internal: true |
mark as internal so avoid indexed by skills.sh
Summary
This PR adds the
rstack-skill-evaluatorskill for evaluating agent skills, and sets up theskills-package-managerinfrastructure to manage skill dependencies and metadata.Related Links
None.