feat(bench): continuity-hell v1 coding-200 pilot harness

[Davincc77]

Summary

Phase 1 of the continuity benchmark programme: a reproducible,
scientifically-defensible pilot stress-test harness for one skill lane,
x.klickd/coding (real artifact packages/@klickd/core/starter-skills/coding.klickd).

This is a real 200-task stress test on a single skill to find weaknesses
in its carried continuity/governance behaviour. It is not an A/B/C/D study
(ABCD comes later, after corrections), not a public release, and makes
no scientific-proof or market claim. Public release stays v4.1.

What's added (benchmarks/continuity-hell-v1/coding-200/)

• tasks.json — exactly 200 unique tasks (CH1-COD-001..200), each
  with ≥3 vectors across nine dimensions (continuity, constraint respect,
  source discipline, governance/human-veto, security/no-leakage, skill
  activation, handoff, actionability, no hallucinated facts). No easy tasks.
  Governance vectors are read from the real coding.klickd via the SDK, so
  the dataset can't drift from the artifact under test.
• generate_tasks.py — deterministic, byte-stable generator (--check).
• run_benchmark.py — deterministic dry-run lanes (floor/ceiling,
  always labelled is_real_llm: false) + a gated real-LLM lane that refuses
  without explicit approval and ships the provider call unwired on purpose.
• score_outputs.py — deterministic, LLM-free scorer.
• BENCHMARK_PROTOCOL.md / scoring_rubric.md — frozen protocol
  (hypotheses, conditions, model/temp, thresholds, anti-mirage gate) and rubric.
• README.md, reproducibility.md, failure_analysis.md template,
  dry-run results, and a llm_x_klickd.BLOCKED.md marker.
• tests/test_continuity_coding200.py — 21 validations.

Anti-mirage guarantees

• No deterministic path ever emits is_real_llm: true; dry-run output carries
  a not_real_label.
• Real provider call is unwired (NotImplementedError) behind --execute +
  XKLICKD_BENCHMARK_FULL_APPROVED=1 + a provider key.
• Real 200-task LLM execution is BLOCKED — not run. See the BLOCKED marker
  for the exact blocker and required input.

Test plan

• pytest tests/test_continuity_coding200.py -q — 21 passed
• pytest tests/ -q — 309 passed (no regressions; +21 new)
• pytest tests/test_dev_preview.py -q — 5 passed
• supply-chain tests — 102 passed
• generate_tasks.py --check — dataset byte-stable
• dry-run lanes diverge: baseline 0/200, x.klickd 200/200
• real-LLM lane refuses with exact blocker (no provider called)

---

🤖 Generated by Computer

[Davincc77]

Secret-safety guardrails added (commit c0e13aa)

Mandatory provider-key leakage guardrails are now in place before any future real LLM run. The real 200-task LLM lane remains BLOCKED — no provider was called and no real run was performed.

What changed

• secret_guard.py — single source of truth for detecting provider-key shapes (Anthropic/OpenAI/Google/Groq/AWS), Authorization/Bearer headers, high-entropy tokens, and any live provider env var value. Redacts to [REDACTED:<kind>] and assert_cleans payloads. Findings never echo the raw secret.
• run_benchmark.py — every output envelope is redact()-ed then assert_clean()-ed before any write, so no env var value, token, or header can be serialized to an artifact. New value-blind preflight mode verifies a provider key exists (reports env var name only, never the value) and that results/ is secret-clean; preflight is wired into the real-LLM gate (§7 item 5).
• scripts/check_benchmark_secret_leakage.py — CI-friendly artifact scanner; exits non-zero on any finding, prints only redacted previews. Reuses secret_guard so detection is defined in one place.
• Docs — README, BENCHMARK_PROTOCOL.md (§3 provenance, §7 gate item 5, new §10), reproducibility.md, and the BLOCKED marker now state: API keys live only in the private environment / secret manager, never committed, logged, or in artifacts; results record provider/model/run_id only, never headers/tokens.
• tests/test_benchmark_secret_guard.py — 18 tests: fake-key detection, redaction, preflight value-blindness, and that a live env var value is never serialized to an artifact or stdout.

Validations

• pytest tests/test_benchmark_secret_guard.py -q — 18 passed
• pytest tests/test_continuity_coding200.py -q — 21 passed (no regressions)
• pytest tests/test_dev_preview.py -q — 5 passed
• supply-chain tests (-k supply_chain) — 102 passed
• run_benchmark.py preflight green; check_benchmark_secret_leakage.py reports clean; raw grep for sk-ant-/sk-/AIza patterns in benchmark+scripts source: no matches; the live env key value appears in no tracked/added file.

Real-run status

Still BLOCKED. Remaining requirement before a real 200-task LLM run is unchanged plus the new secret gate: explicit human go-ahead + --execute + XKLICKD_BENCHMARK_FULL_APPROVED=1 + a provider key + a reviewed _call_provider output→contract mapping + a green secret-safety preflight. No run will happen until you explicitly approve.

🤖 Generated with Claude Code

[Davincc77]

Closing immediately: benchmark work is internal/private and should not live in the public repo. No further public benchmark work will be pushed here.