Backend productionization: launch, observability, hardening, idempotency, staging

[pablopunk]

Goal

Productionize the backend: launch readiness, observability, hardening, idempotency, tests, and app-selectable backend environments — independent of individual product stages.

Merged from

• Production observability and ops hardening #9 — Production observability and ops hardening
• Cross-cutting backend hardening, idempotency, tests, and staging #12 — Cross-cutting backend hardening, idempotency, tests, and staging

To do

Production envs & compatibility (was #6)

• Set production compatibility envs: NEVERMIND_MIN_DESKTOP_VERSION, NEVERMIND_LATEST_DESKTOP_VERSION, NEVERMIND_DESKTOP_UPDATE_URL
• Confirm/rotate production Stripe envs: STRIPE_SECRET_KEY, STRIPE_WEBHOOK_SECRET, STRIPE_SUBSCRIPTION_TIERS, STRIPE_TOP_UP_PACKS, CREDIT_USD, CREDIT_MARKUP, MONTHLY_FREE_CREDITS
• Confirm Stripe webhook endpoint points at latest production deployment
• Enable/configure Stripe Customer Portal
• Run billing smoke tests (subscribe, verify credits, top-up, cancel)
• Swap WorkOS staging → production credentials
• Verify packaged Electron build against production end-to-end
• Dogfood tool-calling through the pure reverse proxy
• Repeat Google streaming AI request and confirm prod logs show non-zero tokens

Observability & ops (was #9)

• Wire Axiom or BetterStack log drain for nvm Vercel project
• Create Sentry alert rules for abuse_check, health_check tags
• Add alert/log checks for Stripe webhook failures
• Confirm Stripe webhook endpoint and Customer Portal configuration
• Create/confirm Electron Sentry project
• Add external uptime monitor for https://api.nvm.fyi/api/health
• Delete stray backend Vercel project from accidental link
• Decide on source-map upload / renderer-side Sentry init

Backend hardening & idempotency (was #12)

• Add Zod schemas on endpoints that currently parse/cast request bodies manually
• Add Idempotency-Key support on chat completions
• Add request_dedup table/logic so chat retries do not double-charge
• Add focused unit tests for: cost math, model/provider resolution, getBalances
• Add optional WorkOS magic-link/test-mode e2e coverage
• Reconcile/prevent duplicate Stripe subscription checkout sessions

Staging & app-selectable backends (was #12)

• Add app-selectable backend environments (production vs PR Preview)
• Let backend URL be switched from inside the app without rebuilding Electron
• Preserve safe defaults: packaged → https://api.nvm.fyi, dev → localhost
• Make auth/token storage scoped by backend origin
• Surface active backend/environment in account/debug UI
• Add invite-code gating table/flow if needed

Useful files

• backend/src/lib/billing.ts, backend/src/lib/cost.ts, backend/src/lib/settings.ts
• backend/src/lib/proxy.ts, backend/src/lib/limits.ts, backend/src/lib/ratelimit.ts
• backend/src/lib/log.ts, backend/src/lib/compatibility.ts
• backend/src/pages/api/**, backend/src/db/schema.ts, backend/drizzle/
• src/electron/nevermind-auth.ts, src/electron/nevermind-api.ts, src/electron/nevermind-compatibility.ts
• .agents/skills/production-debugger/

Verification

• mise exec -- pnpm test
• mise exec -- pnpm -C backend test
• Prod health: curl -fsS https://api.nvm.fyi/api/health → ok: true
• Packaged app can switch between production and PR Preview backends