M6: Thompson Sampling reward model — arm definition, prior, exploration rate, decay

[ninan-versioning]

Milestone 6 — Bandit Reward Model

Vertical: enterprise
Revenue-Signal: retention
Customer-Type: smb-brand|mid-market|enterprise

What

Architecture names Thompson Sampling as the loop closer, but no ticket defines the actual bandit: arms, priors, reward shape, exploration rate, decay schedule. Specify it as a versioned reward-model contract in the control plane.

Scope

• Arm definition: one arm per (SKU × channel × variant_id) tuple
• Reward: weighted composite — 0.6 * normalized_ROAS + 0.3 * CVR + 0.1 * reorder_rate
• Prior: Beta(2,2) for CVR, Gamma for ROAS — cold-start uses brand-rules archetype prior
• Exploration rate: posterior sampling (true TS), no epsilon override
• Decay rate: 14-day half-life on observations — keeps the loop responsive to creative fatigue
• Boundary conditions: minimum 200 impressions before arm scoring; arms with <50 impressions never promoted
• Memory: posterior state persists per (brand × arm) in LibSQL — survives restarts
• Ground truth injection: weekly merchant-confirmed winners override TS for that arm

Why

Loop-design discipline (per user's 9-primitive spec). A bandit without a documented reward + decay rate converges to local maxima and never explains its decisions to the merchant. Also makes the "positionless" claim defensible — the math is auditable.

Acceptance criteria

• reward_model.json v1 spec committed in shoploop/policy/
• Unit tests for arm scoring at cold-start, mid-life, fatigue
• Audit endpoint: GET /brands/:id/arms returns posterior + last-update timestamp
• Decay verified — arm with no recent impressions loses confidence over 14 days

Affected repos

shoploop (primary), shoploop-core (reward consumer in brand_rules update)

Cost

$0 — pure compute, runs in existing Mistral/Codestral autoresearch budget.