Skip to content

refactor: adopt Hard Signal Framework for create-expert verification#781

Merged
FL4TLiN3 merged 2 commits intomainfrom
refactor/hard-signal-verification
Mar 13, 2026
Merged

refactor: adopt Hard Signal Framework for create-expert verification#781
FL4TLiN3 merged 2 commits intomainfrom
refactor/hard-signal-verification

Conversation

@FL4TLiN3
Copy link
Contributor

Summary

  • Replace soft signal verification (LLM judgment-based) with Hard Signal Framework: Ground truth × Context separation × Determinism
  • Consolidate 3 test queries into 1 comprehensive query — coverage comes from signal design depth, not repetition
  • Add reproducibility check to verify-test: re-run same commands, diff results to confirm signals are truly deterministic
  • Simplify verdict system from PASS/SUFFICIENT/CONTINUE to PASS/CONTINUE

Test plan

  • Run create-expert against a sample domain and verify plan.md produces hard signals (commands + expected outputs)
  • Verify the build loop executes write → test → verify in single-query mode
  • Confirm verify-test's reproducibility step catches non-deterministic signals
  • Validate structural checks still detect missing delegates arrays, pick lists, etc.

🤖 Generated with Claude Code

FL4TLiN3 and others added 2 commits March 13, 2026 21:31
…pipeline

Replace soft signal verification (LLM judgment) with hard signal verification
(deterministic commands with expected outputs). Key changes:

- Design Principle 1: "Built-in Verification" → "Hard Signal Verification"
  with three conditions: Ground truth, Context separation, Determinism
- Plan: 3 test queries → 1 comprehensive query with deep signal coverage
- Build loop: sequential 3-query cycle → write → test → verify with
  reproducibility confirmation
- Verify-test: artifact inspection + semantic review → hard signal execution
  + reproducibility check + structural checks
- Verdict system: PASS/SUFFICIENT/CONTINUE → PASS/CONTINUE

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…criptions

Replace "exit codes, test results, grep matches" and similar coding-centric
examples with domain-neutral language per Design Principle 3 (Domain
Agnosticism). create-expert serves all domains, not just software development.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@FL4TLiN3 FL4TLiN3 merged commit 240409d into main Mar 13, 2026
11 checks passed
@FL4TLiN3 FL4TLiN3 deleted the refactor/hard-signal-verification branch March 13, 2026 21:39
@FL4TLiN3 FL4TLiN3 mentioned this pull request Mar 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant