From 1252333d0f2a48546ced27b53c537d5cf4480450 Mon Sep 17 00:00:00 2001 From: HiranoMasaaki Date: Fri, 13 Mar 2026 16:37:10 +0000 Subject: [PATCH] docs: add design principles to create-expert perstack.toml MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Document the seven design principles established through iterative refinement of the create-expert pipeline: 1. Built-in Verification (verifier executes, not reviews) 2. Instruction Quality via Binary Checks (subjective checks fail) 3. Domain Agnosticism (not just coding) 4. Plan ↔ write-definition Boundary (plan = raw material) 5. Failure Conditions (hard reject rules, not inverse of success) 6. Instruction Content = Domain Constraints Only 7. Brevity = Focus (verbose dilutes LLM attention) Co-Authored-By: Claude Opus 4.6 --- definitions/create-expert/perstack.toml | 60 +++++++++++++++++++++++++ 1 file changed, 60 insertions(+) diff --git a/definitions/create-expert/perstack.toml b/definitions/create-expert/perstack.toml index 440aaaae..e5674165 100644 --- a/definitions/create-expert/perstack.toml +++ b/definitions/create-expert/perstack.toml @@ -8,6 +8,66 @@ # ├── @create-expert/test-expert — single query execution (pure executor, no evaluation) # └── @create-expert/verify-test — artifact inspection + execution + definition review # ============================================================================= +# +# ============================================================================= +# Design Principles +# +# 1. Built-in Verification +# - Verifier EXECUTES and TESTS artifacts — it is not a code reviewer. +# - One verifier per team, not one per executor. +# - Verifier must be direct child of coordinator, not nested under executor. +# - Verifier needs `exec` in pick list. Without it, verification degrades +# to file reading, which cannot catch runtime failures. +# +# 2. Instruction Quality via Binary Checks +# - Subjective self-checks ("would removing this make output worse?") +# do not work — the LLM always judges its own output as necessary. +# - Binary checks work: "code block present?" "library name present?" +# have unambiguous yes/no answers with clear remediation actions. +# - Pattern: structural checks (delegates array, pick list) have always +# been followed. Content checks must follow the same binary pattern. +# +# 3. Domain Agnosticism +# - create-expert must produce experts for ANY domain — coding, writing, +# research, design, operations — not just software development. +# - Rules and examples in the pipeline must use domain-neutral language. +# - Coding-specific vocabulary in rules biases generated experts toward +# code-only solutions even when the domain is broader. +# +# 4. Plan ↔ write-definition Boundary +# - Plan outputs role assignments and domain knowledge, NOT instruction +# content. Architecture section = tree structure only (name, one-line +# purpose, role). Domain Knowledge section = raw material for +# instructions. +# - write-definition composes instructions by selecting relevant domain +# constraints per expert. It must NOT copy plan details wholesale. +# - Without this boundary, plan bloat leaks directly into instructions. +# +# 5. Failure Conditions +# - Not the inverse of success criteria — they are hard reject rules +# derived from deeply understanding the domain. +# - Each must specify: what is wrong, which expert caused it, and +# where to restart. +# - These go into the verifier's instruction so it knows what to reject. +# +# 6. Instruction Content = Domain Constraints Only +# - An instruction should contain ONLY what the LLM cannot derive on +# its own: business rules, quality bars, domain-specific anti-patterns, +# completion criteria, priority rules. +# - Everything else — implementation techniques, library choices, file +# structures, well-known algorithms — is noise that dilutes the signal. +# - Enforced by a 6-item binary checklist (no code blocks, no library +# names, no file paths, no procedures, no technique explanations, +# ≤15 lines for non-coordinator experts). +# +# 7. Brevity = Focus +# - Verbose instructions dilute LLM attention. The more you write, the +# less each sentence weighs. Short, dense instructions outperform +# long, thorough ones because the LLM concentrates on what matters. +# - This applies at every level: plan.md, instruction fields, and this +# pipeline's own definitions. If it can be said in fewer words, it +# should be. +# ============================================================================= # ============================================================================= # create-expert — Coordinator