Skip to content

fix: make review-definition checklist concrete with domain-neutral violation patterns #786

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
FL4TLiN3 merged 2 commits into from
Mar 13, 2026
Merged

fix: make review-definition checklist concrete with domain-neutral violation patterns #786

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3d995cd
fix: make review-definition checklist concrete with domain-neutral vi…
FL4TLiN3 Mar 13, 2026
5e52e9d
chore: bump create-expert version to 1.0.18
FL4TLiN3 Mar 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 13 additions & 10 deletions definitions/create-expert/perstack.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@

[experts."create-expert"]
defaultModelTier = "high"
version = "1.0.17"
version = "1.0.18"
description = "Creates and modifies Perstack expert definitions in perstack.toml"
instruction = """
You are the coordinator for creating and modifying Perstack expert definitions. perstack.toml is the single source of truth — your job is to produce or modify it according to the user's request.
Expand Down Expand Up @@ -133,7 +133,7 @@ pick = ["readTextFile", "exec", "attemptCompletion"]

[experts."@create-expert/plan"]
defaultModelTier = "high"
version = "1.0.17"
version = "1.0.18"
description = """
Analyzes the user's request and produces plan.md: domain constraints, test query, verification signals, and role architecture.
Provide: (1) what the expert should do, (2) path to existing perstack.toml if one exists.
Expand Down Expand Up @@ -220,7 +220,7 @@ pick = [

[experts."@create-expert/build"]
defaultModelTier = "low"
version = "1.0.17"
version = "1.0.18"
description = """
Orchestrates the write → review → test → verify cycle for perstack.toml.
Provide: path to plan.md (containing requirements, architecture, test query, and verification signals).
Expand Down Expand Up @@ -278,7 +278,7 @@ pick = ["readTextFile", "exec", "todo", "attemptCompletion"]

[experts."@create-expert/write-definition"]
defaultModelTier = "low"
version = "1.0.17"
version = "1.0.18"
description = """
Writes or modifies a perstack.toml definition from plan.md requirements and architecture.
Provide: (1) path to plan.md, (2) optionally path to existing perstack.toml to preserve, (3) optionally feedback from a failed test to address.
Expand Down Expand Up @@ -381,7 +381,7 @@ pick = [

[experts."@create-expert/review-definition"]
defaultModelTier = "low"
version = "1.0.17"
version = "1.0.18"
description = """
Reviews perstack.toml against plan.md for domain knowledge alignment and instruction quality.
Provide: (1) path to plan.md, (2) path to perstack.toml.
Expand All @@ -402,9 +402,12 @@ For library/tool names appearing in any instruction:
- Traceable to plan.md's Domain Knowledge (i.e., user-specified)? (✓/✗ per name)
- If not user-specified, flag it — the instruction should describe the capability, not name the tool.

For each non-coordinator instruction:
- Contains only domain constraints the LLM cannot derive on its own? (✓/✗)
- Free of code snippets, JSON schemas, and step-by-step procedures? (✓/✗)
For each non-coordinator instruction, flag (✗) if any of the following appear:
- Output locations or structure — dictates where to place results or enumerates specific artifacts to produce. The instruction should describe what the output must accomplish, not its shape.
- Internal APIs of specified tools — names specific methods, hooks, functions, or parameters of a tool. "Use tool X" is the constraint; how to use its API is derivable.
- Named alternatives to reject — lists tools or approaches NOT to use. Specifying what to use implicitly excludes alternatives.
- Implementation approach — describes how to build something rather than what constraints the result must satisfy.
- Code snippets, data schemas, or step-by-step procedures.

## Verdicts

Expand All @@ -427,7 +430,7 @@ pick = ["readTextFile", "todo", "attemptCompletion"]

[experts."@create-expert/verify-test"]
defaultModelTier = "low"
version = "1.0.17"
version = "1.0.18"
description = """
Executes hard signal checks against test-expert's results, verifies their reproducibility, and checks the definition structure.
Provide: (1) the test-expert's factual report (query, what was produced, errors), (2) the verification signals from plan.md, (3) path to perstack.toml.
Expand Down Expand Up @@ -490,7 +493,7 @@ pick = ["readTextFile", "exec", "todo", "attemptCompletion"]

[experts."@create-expert/test-expert"]
defaultModelTier = "low"
version = "1.0.17"
version = "1.0.18"
description = """
Executes a single test query against a Perstack expert definition and reports what happened.
Provide: (1) path to perstack.toml, (2) the test query to execute, (3) the coordinator expert name to test.
Expand Down
Loading