LlmComposer: fake-HTTP-engine tests + optional eval route

[ber4444]

Summary

Adds an optional local-llm-compose eval route that scores an OpenAI-compatible LLM composer against the same case-specific retrieval as the deterministic template route, plus a pluggable HTTP transport (LlmHttpTransport / OpenAiCompatibleLlmClient.forTesting) that makes the cloud client unit-testable with a lambda fake instead of a mock library.

What landed

Four commits on this branch:

1. LlmHttpTransport + forTesting factory + optional eval route — the original PR scope. The route is OPTIONAL: only runs when COACH_LLM_API_KEY + token prices are set, so the CI grounding gate never depends on a live LLM.

2. Hoist the LLM route out of testApplication — the route timed out (UncompletedCoroutinesError: After waiting for 1m) whenever run with a real key. testApplication wraps its body in runTestWithRealTime with a hardcoded 60s ceiling that can't be extended (verified by decompiling Ktor 3.4.3 — testApplication calls runTestWithRealTime$default with the timeout arg defaulted). Fix: the route calls composer.compose() directly (not the test server), so it runs in a plain runBlocking after testApplication finishes the fast deterministic routes. Also adds COACH_LLM_MAX_USD_CENTS (default $0.002) so the cost cap can be raised for a full run.

3. Enrich opening-concept passages to prose — a real latent bug. OpeningExplanationValidator grounds each output sentence by requiring ≥2 content-word overlaps with the cited passage; the eval built passages whose text was bare tags ("development, center."), giving the model ~2 tokens of vocabulary. Now builds prose passages (validator unchanged).

4. Format-coercive prompt — see findings below. Validator unchanged.

How to run it locally

./gradlew --stop   # clear daemon so it sees env vars
export COACH_LLM_API_KEY=sk-...
export COACH_LLM_INPUT_USD_PER_MILLION=0.40   # gpt-4.1-mini real prices
export COACH_LLM_OUTPUT_USD_PER_MILLION=1.60
export COACH_LLM_MAX_USD_CENTS=20             # $0.20, room for all 100 cases
# optional: export COACH_LLM_MODEL=gpt-4.1
./gradlew :evals:run --no-daemon

Findings (the answer to "does LLM composition beat the template?")

Investigating a near-100% fallback rate produced a clean, three-run comparison. The headline: at these model tiers, no — the template/fallback path is the right production choice under the current output contract.

Model	Pass	Dominant failure gate	Conclusion
gpt-4.1-mini (strengthened prompt)	~1%	citation compliance — model ignores the [source-id] format	model too weak for the format contract
gpt-4.1 (strengthened prompt)	11%	sentence-structure — output is cited + short + grounded but fails the per-sentence citation rule	model tier helps; prompt/validator friction remains

Two separable results:

• Model tier is the primary bottleneck, not the prompt. Going from gpt-4.1-mini → gpt-4.1 raised the pass rate ~1% → 11%, because gpt-4.1 cites reliably (89/89 fallbacks contained [eval-opening-NNN]) where gpt-4.1-mini largely didn't.
• The remaining ~89% of gpt-4.1 failures are a prompt/validator structural mismatch, not a quality failure. The rejected outputs are grounded (0.0% grounding violation), under length (median 177 chars, max 234 — well under the 300 cap), and on-topic. They fail because OpeningExplanationValidator requires every sentence to repeat the citation, and gpt-4.1 converges on a single cited sentence rather than repeating [id] across 2-3. That's the validator's deliberate anti-hallucination design — and it's incompatible with how even capable LLMs naturally write.

This is the informative outcome the design doc (evals/docs/litert-faithfulness-bypass.md) predicts for a related scorer: a quality gate that catches what a looser check can't. Whether to relax the validator's per-sentence rule (production semantics change) or accept the template as the production choice is a separate decision — this PR establishes the measurement, it doesn't change the contract.

Verification

• ✅ ./gradlew :evals:run completes with a real key (was timing out at the 60s ceiling).
• ✅ No-key path (CI's grounding gate) green — local-llm-compose shows "COACH_LLM_API_KEY or token prices not set".
• ✅ :server:test passes — the LlmHttpTransport fake covers success / validation-failure / budget / missing-env cases.
• ✅ local-template route still 0.0% grounding violation after the prose-passage change.
• ⏳ CI re-running.

Changed files

• server/.../Application.kt — selectComposer (extracted for testability) + COACH_LLM_MAX_USD_CENTS env var.
• server/.../Composers.kt — LlmHttpTransport / OpenAiCompatibleLlmClient.forTesting / strengthened format-coercive prompt.
• server/.../LlmComposerHttpTest.kt — fake-engine tests (no mock library).
• evals/.../EvalMain.kt — evaluateLlmComposed route (hoisted out of testApplication) + prose passages.
• evals/scorecard.md — the verified gpt-4.1 run row.

[ber4444]

Reframed out of draft. This was held as draft waiting on a cross-repo GLM eval. That eval ran (ferryman-mcp #10) and ranked GLM last on both skills at ~10× gemini's cost and ~24× its latency — so the GLM motivation is dead and I'm landing only what's provider-agnostic: the fake-HTTP-engine test seam, the selectComposer extraction, the budget-enforcement-before-transport test, and the optional eval route.

The reframe was surface-artifacts only — no Composers.kt/Application.kt/EvalMain.kt logic changed. Specifically:

• doc renamed zai-glm-provider-addendum.md → llm-composer-provider-infra.md and rewritten to provider-neutral framing;
• two code comments de-vendored;
• one test fixture model string glm-4.6 → gpt-4.1-mini (the production default; same serialization coverage).

./gradlew :server:test → 23 green (16 existing + 7 new). grep for z.ai/glm across server/ evals/ .kt returns nothing. Nothing provider-specific was ever committed, per the hard rule.