test(vertical): composition eval suite over the assembled chat modules (#188)

[drewstone]

Assembles the just-merged #189 (app-auth), #190 (incident gates), #191 (chat-store), and #192 (interactions) into ONE mini chat app under tests/vertical/ and evaluates whether the modules actually compose into a product — not whether their unit tests pass. The suite runs in normal CI (in-memory sqlite + memoryAdapter, no network) and becomes the permanent composition gate for issue #188: a future subpath change that keeps its unit tests green but breaks assembly fails here.

The assembled app (tests/vertical/mini-app.ts)

createAppAuth (real better-auth, memory adapter) → guard quartet → product membership check → createChatStore over :memory: better-sqlite3 with real DDL/FKs → the three #190 gates at the turn boundary → a fake sandbox producer emitting canonical sidecar part streams (text/reasoning snapshots+deltas, tool state transitions, step-finish token receipts, interaction asks, error) pumped through /stream's normalizer into both lanes (NDJSON client stream consumed with /web-react's consumeChatStream, persisted message.parts) → createInteractionAnswerRoute over a fake sidecar broker the producer genuinely blocks on (tests/vertical/fake-sidecar.ts).

Scenario verdicts (15 evals, all user-visible outcomes)

Scenario	Verdict
Full turn → assistant persisted with text/tool/step-finish parts + flattened token receipt columns; client saw text, tool_call/result, usage, metadata	PASS
Unauthenticated turn → guard 401 auth.unauthenticated; non-member turn → 403 before any row is written	PASS
Failed turn → error synthesized into the transcript when no onErrorEvent is wired (#190), user message retained, partial output + notice part persisted	PASS
Failed turn with onErrorEvent wired → routed to the callback, no transcript duplicate	PASS
Interaction ask blocks the turn; invalid free-text on option-only select → 400 INVALID_INTERACTION_ANSWER and run stays blocked; valid answer unblocks; answered card persisted and round-trips through persistedPartToInteraction; replay → 410 INTERACTION_EXPIRED	PASS
Re-emitted duplicate question → one card client-side (content-signature dedupe) and one POST resolves both registry entries	PASS
Cross-workspace answer/list of another workspace's ask → 403, ask stays outstanding	PASS
Cross-workspace thread read → 404 indistinguishable from a missing thread (app-auth guard × chat-store, first time this composition is tested)	PASS
Second workspace never sees the first's threads; listing a non-member workspace → 403	PASS
Bulk delete spanning a denied workspace → 403 and ZERO deletion (threads + messages intact); own-workspace delete removes messages too; malformed ids → 400	PASS (after fix 1)
Oversized composed prompt → 400 with over the 40000-byte budget + section breakdown, fired pre-write	PASS
Oversized env entry → 400 naming the variable + the 131072 E2BIG bound	PASS
Oversized provision payload → 400 with Breakdown: profile=…(files=…), env=…, secrets=…	PASS (after fix 3)

Seams fixed in this branch (S-cost, all additive)

1. src/chat-store/store.ts:236 — bulkDeleteThreads checked workspaces in DB row order of random hex ids: which denial surfaced first was nondeterministic and the unit test asserting the check sequence (tests/chat-store/chat-store.test.ts:221) was flaky — it failed 2/5 local runs. Distinct workspace ids are now sorted before checking; fail-closed semantics unchanged.
2. src/interactions/contract.ts — interactionToPersistedPart/noticePart returned Record<string, unknown>; the "byte-matches chat-store's ChatInteractionPart/ChatNoticePart" claim was prose only. They now return precise InteractionPersistedPart/NoticePersistedPart types (aliases, so Record<string, unknown> consumers stay source-compatible) and src/chat-store/parts.ts enforces mutual assignability at compile time — a field added on one side without the other now fails tsc.
3. src/sandbox/index.ts:1040 — ProvisionPayloadSections claimed to be structural but typed backend.profile as the SDK's AgentProfile; even the module's own call site (ensureWorkspaceSandbox) had to cast into its own gate type, and the SDK's string profile ref was rejected. Now a structural ProvisionProfileSection (+ the string ref arm); internal cast dropped.

Filed (larger than S-cost)

• Untyped boundary between /stream and /chat-store: finalizeAssistantParts/mergePersistedPart (src/stream/stream-normalizer.ts:165,211) produce JsonRecord[] while appendMessage (src/chat-store/store.ts:70) takes ChatMessagePart[] — every product persisting a streamed turn writes as unknown as ChatMessagePart[] (see tests/vertical/mini-app.ts, persistence lane). Needs a typed projection (a ChatMessagePart-typed finalize overload, or a validating toChatMessageParts in chat-store); deciding where that boundary lives is a design call, not a one-line export.
• Normalizer scope trap: normalizePersistedPart returns null for every canonical kind other than text/reasoning/tool (src/stream/stream-normalizer.ts:87-150) — an integrator feeding all parts through it silently drops step-finish receipts and interaction parts. Documented in /chat-store/parts.ts, but a normalizePersistedPart-adjacent passthrough for the storable kinds would remove the trap.
• Ergonomic note (no change requested): guards throw Responses while createInteractionAnswerRoute.resolveConnection expects { ok: false, response } returns — every product writes the same try/catch adapter (tests/vertical/mini-app.ts, resolveConnection).

Verification

• pnpm typecheck clean; pnpm test 149 files / 2416 tests green; NODE_OPTIONS=--max-old-space-size=8192 pnpm build green on a clean pnpm install --frozen-lockfile.
• Knip delta vs main: zero new findings.
• tests/chat-store run 3× consecutively green after fix 1 (was flaky before).

🤖 Generated with Claude Code

[tangletools]

✅ Auto-approved drewstone PR — f12bea43

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

_{tangletools · auto-approval · reason: drewstone_author · 2026-07-15T18:52:13Z}

[tangletools]

✅ Auto-approved drewstone PR — f9d6530b

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

_{tangletools · auto-approval · reason: drewstone_author · 2026-07-15T18:55:27Z}