Skip to content

feat: optimise SOBC command prompt via autoresearch (7.7 → 8.9)#213

Closed
tractorjuice wants to merge 1 commit intomainfrom
feat/autoresearch-sobc
Closed

feat: optimise SOBC command prompt via autoresearch (7.7 → 8.9)#213
tractorjuice wants to merge 1 commit intomainfrom
feat/autoresearch-sobc

Conversation

@tractorjuice
Copy link
Copy Markdown
Owner

Summary

  • Optimised the /arckit.sobc command prompt using the autoresearch loop (19 iterations, 5 keeps, 14 discards)
  • Baseline score: 7.7 → Final score: 8.9 (improvement: +1.2)
  • Aligned to HM Treasury Green Book 2026 methodology

Changes to prompt

Keep Score Description
Baseline 7.7 Unmodified command
iter 2 8.0 Added Appraisal Summary Table (AST) with BCR to Economic Case
iter 6 8.3 Added Cross-Case Traceability Matrix with quoted stakeholder goals
iter 12 8.6 Added inline stakeholder goal quoting throughout all five cases
iter 19 8.9 Added post-write verify-and-patch for Theory of Change, CSF Assessment Matrix, and switching values

Green Book 2026 elements now generated

  • Theory of Change (Inputs → Activities → Outputs → Outcomes → Impacts)
  • CSF Assessment Matrix (5+ CSFs × all options, RAG-rated)
  • Appraisal Summary Table (NPV, BCR, stakeholder goals %, key risks)
  • Switching values (% change in costs/benefits that breaks the recommendation)
  • Cross-Case Traceability Matrix (stakeholder goals quoted from STKE across all five cases)
  • Inline goal referencing (pervasive STKE goal quotes throughout document body)
  • Optimism bias adjustment (+40% for IT projects per Green Book)
  • Sensitivity analysis with BCR for each scenario

Test plan

  • Run /arckit.sobc 001 against a test repo with stakeholder analysis
  • Verify all Green Book elements are present in output
  • Verify traceability matrix quotes match STKE content
  • Run python scripts/converter.py (already done — 7 files updated)
  • Run markdownlint

🤖 Generated with Claude Code

Autoresearch run: 19 iterations, 5 keeps, 14 discards.

Prompt improvements (Green Book 2026 alignment):
- Add Appraisal Summary Table (AST) with BCR to Economic Case
- Add Cross-Case Traceability Matrix with quoted stakeholder goals
- Add inline goal referencing throughout all five cases
- Add post-write verify-and-patch step for Theory of Change,
  CSF Assessment Matrix, and switching values
- Add citation traceability instruction for external documents

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@tractorjuice
Copy link
Copy Markdown
Owner Author

Superseded by #265 (combined autoresearch PR for all 48 commands)

@tractorjuice tractorjuice deleted the feat/autoresearch-sobc branch March 27, 2026 17:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant