Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -301,6 +301,12 @@ Once installed, OpenCode will automatically detect the skill when you ask it to

OpenCode will load the opencode-skill-creator instructions and use the plugin tools to walk through the full workflow.

## Examples

The fastest way to see the workflow end to end: start with the [Docker Compose helper example](examples/docker-compose-helper/). It includes a small complete skill, a trigger eval set in the exact `skill_eval` JSON format, and copy-pasteable prompts for `skill_eval`, `skill_optimize_loop`, and benchmark review.

Use it after install to see how description evals work before creating your own skill.

## Architecture

This project has two components:
Expand Down
72 changes: 72 additions & 0 deletions examples/docker-compose-helper/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Docker Compose Helper Example

This is a complete, lightweight example of the eval-driven skill workflow:
write a small skill, test whether its description triggers on the right
queries, optimize the description, then benchmark real task output if needed.

## Files

- `SKILL.md` - the example skill
- `evals/eval-set.json` - trigger evals consumed by `skill_eval`

The eval file is a JSON array of `{ "query": string, "should_trigger": boolean }`
items. That matches the `EvalItem` interface used by the plugin.

## Run A Trigger Eval

In OpenCode, ask:

```text
Use opencode-skill-creator to run skill_eval on examples/docker-compose-helper with examples/docker-compose-helper/evals/eval-set.json.
Use runsPerQuery 3 and triggerThreshold 0.5.
```

Equivalent tool arguments:

```text
skill_eval:
skillPath: examples/docker-compose-helper
evalSetPath: examples/docker-compose-helper/evals/eval-set.json
runsPerQuery: 3
triggerThreshold: 0.5
```

Expected output: JSON with one result per query. Positive queries pass when the
skill triggers in at least 50% of successful runs; negative queries pass when it
triggers in less than 50%. With the defaults above, each query runs 3 times.

## Optimize The Description

Ask:

```text
Use opencode-skill-creator to run skill_optimize_loop for examples/docker-compose-helper using examples/docker-compose-helper/evals/eval-set.json.
Use maxIterations 5, runsPerQuery 3, triggerThreshold 0.5.
```

Equivalent tool arguments:

```text
skill_optimize_loop:
skillPath: examples/docker-compose-helper
evalSetPath: examples/docker-compose-helper/evals/eval-set.json
maxIterations: 5
runsPerQuery: 3
triggerThreshold: 0.5
```

Expected output: JSON containing the best description found, train/test scores,
and per-query trigger rates. Copy `best_description` into the frontmatter only
if it improves held-out test behavior and still reads clearly.

## Benchmark Real Output

Trigger evals only test whether OpenCode chooses the skill. To compare task
quality, run paired task outputs with and without the skill, then ask:

```text
Use opencode-skill-creator to run skill_aggregate_benchmark for my eval workspace, then open the review viewer.
```

Expected output: `benchmark.json`, `benchmark.md`, and a review viewer showing
pass rates, timing, tokens, and side-by-side run outputs.
59 changes: 59 additions & 0 deletions examples/docker-compose-helper/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
---
name: docker-compose-helper
description: Use when the user asks to create, debug, or modify Docker Compose files, compose.yaml, docker-compose.yml, service dependencies, healthchecks, ports, volumes, networks, environment variables, or local multi-container development stacks. Use this skill even if the user says "compose file" or "local stack" instead of "Docker Compose".
---

# Docker Compose Helper

Help users create, review, and repair Docker Compose setups with the smallest
working change that fits their local development workflow.

## Workflow

1. Identify the user's goal:
- creating a new compose file
- adding or changing a service
- debugging startup, networking, volume, or environment issues
- translating a `docker run` command into compose
2. Inspect existing files before editing:
- `compose.yaml`
- `compose.yml`
- `docker-compose.yaml`
- `docker-compose.yml`
- `.env`
- Dockerfiles referenced by `build`
3. Prefer Docker Compose v2 syntax:
- omit the obsolete top-level `version`
- use `compose.yaml` if creating a new file
- keep service names lowercase and stable
4. Make the smallest useful change. Do not add orchestration, custom networks,
or named volumes unless the service needs them.

## Compose Defaults

- Use explicit image tags instead of `latest` when a stable tag is obvious.
- Use named volumes for database state.
- Use bind mounts for application source during local development.
- Put secrets and machine-specific values in `.env`, not directly in YAML.
- Add `depends_on` only for startup ordering; add `healthcheck` when readiness
actually matters.
- Expose only the host ports the user needs.

## Debugging Checklist

When debugging, check these in order:

1. YAML validity and indentation.
2. Whether the service name, image, build context, and Dockerfile path exist.
3. Port conflicts between host ports and already-running services.
4. Volume paths and whether a bind mount hides files built into the image.
5. Environment variable names, `.env` loading, and missing required values.
6. Container-to-container networking: services should use service names as DNS
hosts, not `localhost`.
7. Readiness problems: add a healthcheck and condition only when the dependent
service truly must wait.

## Output

When editing files, show the changed compose snippet or file path and explain
the reason for each non-obvious setting in one short sentence.
34 changes: 34 additions & 0 deletions examples/docker-compose-helper/evals/eval-set.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
[
{
"query": "Create a compose.yaml for a Node API, Postgres database, and Redis cache for local development.",
"should_trigger": true
},
{
"query": "My docker-compose.yml starts Postgres but the app says ECONNREFUSED to localhost:5432. Help me fix the compose networking.",
"should_trigger": true
},
{
"query": "Convert this docker run command into a Docker Compose service with a named volume and port mapping.",
"should_trigger": true
},
{
"query": "Add a healthcheck and depends_on condition so my web service waits for the database to be ready in compose.yaml.",
"should_trigger": true
},
{
"query": "Review my compose file for unnecessary ports, missing env vars, and volume mistakes before I share it with the team.",
"should_trigger": true
},
{
"query": "Write a Kubernetes Deployment and Service for this container image.",
"should_trigger": false
},
{
"query": "Explain how Docker image layers and build cache work at a high level.",
"should_trigger": false
},
{
"query": "Fix this GitHub Actions workflow that builds and pushes a container image.",
"should_trigger": false
}
]