diff --git a/docs/architecture-decisions/027-configuration-and-secrets-model.md b/docs/architecture-decisions/027-configuration-and-secrets-model.md new file mode 100644 index 000000000..b6cf93842 --- /dev/null +++ b/docs/architecture-decisions/027-configuration-and-secrets-model.md @@ -0,0 +1,211 @@ +# ADR-027: Configuration & Secrets — Model & Tiers + +**Status**: Proposed +**Date**: 2026-07-10 +**Deciders**: Sean Matthews + +> Proposed. Companion ADRs: **ADR-028** (provider plugin), **ADR-029** (management interface), and +> **ADR-030** (docs & adopter maturation). Open questions at the end are implementation detail and +> non-blocking. + +## Context + +Today Frigg can really only set environment variables **manually at build time**: the declarative +`appDefinition.environment` list pulls values from the deploy pipeline and injects them into the +functions via serverless-oss. Beyond that, an adopter must hand-roll a Lambda layer or already know +about the `secrets-to-env.js` (Secrets Manager) path; the SSM runtime loader is drafted and unmerged. + +That single flat env list **blends separable concerns with no distinction**: +- **Infra/runtime** config — `DATABASE_URL`, `KMS_KEY_ARN`, `STAGE` (framework-owned). +- **Framework-behavior** settings — how Frigg itself should behave. +- **Per-environment / dev flags** — feature flags, code-path toggles a dev wants per stage. +- **API-module app credentials** — the client id / secret / scopes each API module needs. + +The last category is the one that breaks. **Every adopter eventually hits "env overload,"** usually +at **10+ API modules**, each with OAuth-style credentials (client id + secret + scopes ≈ 3+ values) += **30+ "env variables"** in one undifferentiated list. + +**Correction to an earlier assumption:** the existing DB models are **instance-scoped** and are *not* +a home for app/definition-level env or module credentials: +- `Integration.config` — per-**instance** user/instance configuration. +- `Credential` — auth for a given `Entity` (rolls up to an `Integration` via relationship), + injected into the API-class instance for the `Requester` base-class fetch. Per-instance + authentication only. + +So there is currently **no first-class home** for app-level, per-API-module credentials/config — +they get dumped into the flat platform env, which is exactly what overloads. + +## Decision — a tiered model + +Configuration/secrets sort into **three scopes**, each crossed with **{config, secret}**: + +| Scope | Config | Secret | Where today | +|---|---|---|---| +| **1. Platform / app-wide** (per env) | infra, framework behavior, feature/dev flags | platform secrets | build-time `environment` → env; Secrets Manager → env (needs manual `SECRET_ARN` + Parameters&Secrets extension-layer wiring); SSM (draft) | +| **2. App-level per-API-module** (per env, shared across all connections) | module settings/scopes | **module client id/secret** | ❌ **no home — dumped into flat env (the overload)** | +| **3. Per-connection** | `Integration.config` | `Credential.data` (field-encrypted) | ✅ exists; injected on demand | + +> **Terminology:** tier 3 is **per-connection** — a `Credential` per `Entity`, i.e. the tokens a +> connection holds against an external service. The common case is a **specific end user** who +> authorizes *their* account (`User → Integration → Entity → Credential`) — what Jane gets when she +> clicks "Connect HubSpot." A second case is an **app-owner–owned global connection** (ADR-024 Global +> Entities): an admin authorizes once at deploy, the `Credential`/`Entity` carries `userId: null`, and +> every end user shares that single connection to the external service. Both are tier 3 — connection +> credentials in the DB, field-encrypted — and differ only in ownership (per-end-user vs global). +> Not to be confused with **deployment tenancy** (one Frigg deployment per adopter customer), a +> separate axis. Tier 2 is the **app-level** OAuth *application* credential (one per module per +> environment, shared by all connections) — e.g. the HubSpot `client_id`/`secret` you register once. + +**Invariant (keep):** tier-3 (per-connection) secrets — end-user *and* global — are **never** promoted +to `process.env`; they're decrypted and injected into the module instance on demand. This isolation +already holds and must stay. + +**Two planes.** Every tier has a **management plane** (where values are authored — the admin API / +CLI / UI, ADR-029) and a **runtime plane** (where the app reads them). Any provider (ADR-028) can +serve one or both: e.g. an adopter can make **1Password the system of record** for tiers 1–2 while +the runtime reads from a cloud store (materialized) or from 1Password directly. Tier 3 is authored at +**connect time** — by the end user (per-connection) or by an admin once (global entities, ADR-024) — +not through the platform-env pipeline, so it lives in the DB by default. + +``` + SCOPE MANAGEMENT PLANE STORE (routing map, ADR-029/028) RUNTIME PLANE + admin API / CLI / UI + Tier 1 platform ──write──► SSM / Secrets / 1Password / local ──► process.env (every function) + Tier 2 app-module ──write──► ModuleCredential/Config | provider ──► env of functions running + that module (scoped) + Tier 3 per-conn. (written by running app) DB Credential / Integration.config ──► injected on demand, + never in process.env +``` + +**The new concept is tier 2** — an app-level, per-module credential/config store, distinct from the +instance `Credential`/`config`. It is realized by **mirroring the existing models**: new +`ModuleCredential` / `ModuleConfig` tables/collections, scoped by **app + environment + module** (not +by `Entity`), secrets field-encrypted via the existing registry. (The rejected alternative — repurpose +`Credential`/config with a scope/reference discriminator — is in *Alternatives Considered*.) + +Either way: field-encrypt the secrets, and **inject per module at build/instantiation** rather than +flattening them into the global env — which both removes the overload and scopes each module to only +its own credentials. + +### Maturation path (free & fast → mature) — a hard requirement + +No adopter should pay for a tier they don't use; each level is opt-in and backward compatible: + +- **L0 — day one, free/fast:** build-time env via `appDefinition.environment` from the pipeline. Zero + infra. (What exists today.) +- **L1:** platform secrets via a provider store (Secrets Manager / SSM / provider), opt-in. +- **L2:** structured **app-level module credentials** (tier 2) — solves env overload — DB-backed or + provider-namespaced, managed via the admin API / CLI (ADR-029). +- **L3:** multi-provider (GCP / Azure / 1Password / Vault, ADR-028), **per-function scoping** + (least privilege), and a hosted management GUI. + +The adopter-facing narrative of this journey — plus a cross-pattern worked example — is **ADR-030**. + +### Variable scoping — deliver only what a function needs + +**Principle: a variable reaches only the bundled functions that need it.** Three scoping levels, +which Frigg can **derive from the app definition's integration→module graph** (it already knows which +integration uses which modules — no hand-authored manifest): + +- **Global** — `DATABASE_URL`, `KMS_KEY_ARN`, `STAGE`, framework-behavior settings → every function. +- **Module-scoped** — tier-2 creds like `HUBSPOT_CLIENT_ID`/`SECRET`/scopes → **only functions that + run the HubSpot module.** The Asana function never sees them *unless* it also runs HubSpot — the + rule is **usage-based, not name-based**, so the graph handles the "adopter runs HubSpot through + their Asana app" case correctly. +- **Per-connection** — tier-3 → never in env; fetched on demand for the specific connection. + +This maps onto the two runtime modes (ADR-028): +- **`materialized`** → each function's env = `global ∪ creds(modules it serves)` (a per-function env + manifest). +- **`direct`** → each function's role/identity is scoped to `global ∪ module-cred paths for modules + it serves`; modules pull creds at instantiation — which also **eliminates env-overload** (nothing + sits in env). + +**Boundary rule — derive by default, annotate the edge cases.** Tier-2 module creds are derived from +the module's auth definition + the graph (no annotation); per-connection is inherently distinct; +everything else defaults to platform-global. The app definition can override scope for +adopter-specific values the graph can't infer: + +```js +environment: { + DATABASE_URL: true, // platform (global) — inferred + MY_HUBSPOT_ONLY_FLAG: { scope: 'hubspot' }, // override: only functions running the hubspot module +} +// HUBSPOT_CLIENT_ID/SECRET/scopes (tier-2) need no annotation — derived from the module + graph. +``` + +**Confirmed against the code — the granularity is already there.** The infra builder emits +**per-integration functions**: `integration-builder.js` creates, per integration, an HTTP handler +(`functions[integrationName]`), a webhook handler (`{name}Webhook`), a queue worker, and one function +per extension binding (`{name}__{binding}`) — and packaging is **`package: { individually: true }`** +(`base-definition-factory.js`), so each is a discrete, separately-packaged Lambda. Per-function +env/IAM scoping therefore needs **no bundling restructure**. + +Env is global today only by **wiring**: the composer does +`Object.assign(provider.environment, merged.environment)` (`infrastructure-composer.js`), dumping all +vars onto the shared provider block. The fix (`materialized` mode) is to emit +`functions[name].environment` = `global ∪ creds(that integration's modules)` from the +integration→module graph instead of onto the global block; shared/non-integration functions (auth, +health, reporting, db-migrate) get `global` only. `direct` mode remains the alternative that keeps +creds out of env entirely. + +## Consequences + +### Positive +- Names and separates the four concerns currently mashed into one env list. +- Gives app-level module credentials a real home → directly kills env overload at scale. +- Preserves the (already-true) instance-secret isolation invariant. +- Free/fast default with an opt-in maturation path — no day-one tax. + +### Negative +- Introduces a new storage tier (tier 2) to design, build, and migrate onto. +- Requires reconciling three overlapping platform-env mechanisms (build-time, Secrets Manager, SSM). + +### Neutral +- Establishes scope × sensitivity as the vocabulary the CLI/API and docs are organized around. + +### Resolved so far +- **Tier-2 storage: mirror models.** New `ModuleCredential` / `ModuleConfig` + tables/collections scoped by **app + environment + module** (not by `Entity`), secrets + field-encrypted via the existing registry. Chosen for clarity/cleanliness over overloading + `Credential`/`config`. +- **Runtime modes:** support **both** `materialized` and `direct`; **`materialized` is the default** + for external managers on serverless (avoids a bootstrap token in functions). (ADR-028.) +- **Per-connection creds:** **DB by default.** A pluggable **non-Frigg credential source** (resolve + tier-3 from an adopter's own store) is a considered, overridable **extension point — deferred, not + built now.** +- **Function scoping is feasible now** (confirmed in code): functions are already per-integration and + `individually` packaged, so `materialized` per-function env/IAM is a wiring change (emit per-function + env from the graph), not a bundling restructure. +- **Terminology:** the instance tier is **per-connection** (not "tenant"). +- **Scope boundary:** **derive by default, annotate the edge cases** — tier-2 from module + graph, + per-connection inherently distinct, everything else platform-global; app definition can override a + variable's scope. +- **Precedence:** tiers are separate namespaces (most-specific wins: per-connection > app-module > + platform); the routing map fixes **one backend per (tier, env)** so there's no in-tier collision by + construction; genuine overlaps → provider/routing-map source wins, local overrides for local env, + deploy validation **warns** (opt-in strict mode fails). + +## Open questions (implementation detail, non-blocking) +- Exact **annotation syntax** for scope overrides in the app definition. +- Whether collision detection defaults to **warn** or **strict**. + +## Alternatives Considered +- **Keep the single flat env list.** Rejected: it is exactly what produces env overload at 10+ + modules and blends four separable concerns with no distinction. +- **Tier-2 by repurposing existing models.** App/definition-level rows in `Credential`/`config` + distinguished from instance rows by a scope/reference discriminator. Rejected in favor of mirrored + `ModuleCredential`/`ModuleConfig` models — a dedicated table reads more clearly and avoids + overloading instance semantics. +- **Only `direct` runtime mode.** Rejected: `direct` needs a bootstrap token in platform env for + external managers; `materialized` avoids it, so both are supported with `materialized` as default + (ADR-028). + +## Related +- [ADR-028: Secrets & Config Provider Plugin](./028-secrets-config-provider-plugin.md) +- [ADR-029: Variable & Secret Management (Admin API / CLI / GUI)](./029-variable-secret-management.md) +- [ADR-030: Configuration & Secrets — Docs & Adopter Maturation](./030-configuration-secrets-docs-and-maturation.md) +- [ADR-016: Plugins](./016-plugins.md) (provider/database/encryption/queue/scheduler plugin taxonomy), + [ADR-024: Global Entities](./024-global-entities.md) (admin-authored global tier-3 connections), + ADR-005 / ADR-010 (admin surface), field-level encryption registry + (`packages/core/database/encryption/`). diff --git a/docs/architecture-decisions/028-secrets-config-provider-plugin.md b/docs/architecture-decisions/028-secrets-config-provider-plugin.md new file mode 100644 index 000000000..25916cf2c --- /dev/null +++ b/docs/architecture-decisions/028-secrets-config-provider-plugin.md @@ -0,0 +1,143 @@ +# ADR-028: Secrets & Config Provider Plugin + +**Status**: Proposed +**Date**: 2026-07-10 +**Deciders**: Sean Matthews + +> Proposed. The "where do values physically live and how are they read" half of the Configuration & +> Secrets work (ADR-027 is the model; ADR-029 is management; ADR-030 is docs & maturation). + +## Context + +The configuration/secrets tiers in ADR-027 need concrete backends, and the reality is multi-cloud: +a live **GCP** deployment, **Azure** on the radar, plus **1Password / Vault** as secret managers +adopters already use. Today only **AWS** is implemented (Secrets Manager → env is live; SSM runtime +loading is a draft; `SsmBuilder` grants IAM but nothing reads it), and there is **no functional +implementation** for GCP Secret Manager, Azure Key Vault, 1Password, or Vault — only commented-out +provider stubs. + +There is already a hexagonal seam — `CloudProviderAdapter` (a port with an AWS impl and GCP/Azure +stubs), and **ADR-016 (Plugins)** proposes required-with-defaults plugins with typed core interfaces +(`provider | database | encryption | queue | scheduler`) selected via `appDefinition.plugins`. The +problem is **cloud-agnostic; only the transport is provider-specific.** + +## Decision + +**Extend the ADR-016 plugin taxonomy with a new `secrets` / `config` plugin type** — this is a +taxonomy extension (a new core interface added to the five existing types), after which adapters +follow ADR-016's no-core-change rule: one typed core interface, many adapters. The framework never +imports a vendor SDK directly; adopters select an adapter in the app definition. + +```js +// appDefinition (illustrative) +plugins: { + secrets: { provider: 'aws' }, // 'aws' | 'gcp' | 'azure' | 'onepassword' | 'vault' | 'database' +} +``` + +**Port (illustrative) — read *and* write, so any provider can be system-of-record and/or runtime source:** +```js +class SecretsConfigProvider { // Port — ADR-016 plugin interface + async resolve(keysOrPrefix, { scope, env }) {} // runtime read → { KEY: value } + async write(key, value, { scope, env, secret }) {} // management-plane write (ADR-029) + runtimeMode() {} // 'materialized' | 'direct' + refreshPolicy() {} // e.g. { ttlSeconds: 300 } +} +``` + +**Runtime mode (decided): support both, `materialized` is the default for external managers.** +- **`materialized`** — values are synced into the function's runtime store (cloud env/SSM/Secrets or + the DB tier) at write/deploy; the function never calls the external manager at runtime. Default for + 1Password/Vault on serverless, because `direct` needs a **bootstrap secret** (a Service-Account + token) living in platform env — chicken-and-egg — whereas cloud-native stores authorize via the + function's IAM role with no stored token. +- **`direct`** — the function reads from the provider at cold start (Connect / Service Account / SDK). + Available for adopters who want the store to *literally be* 1Password/Vault at runtime. + +Mode also selects how **variable scoping** (ADR-027) is realized: `materialized` → per-function env +manifest (`global ∪ creds(modules the function serves)`); `direct` → per-function IAM scoped to those +paths, creds pulled at module instantiation. + +``` + write (ADR-029) runtime read + │ ▲ + ▼ │ + ┌──────────────────── SecretsConfigProvider (port) ──────────────────┐ + │ resolve() write() runtimeMode() refreshPolicy() │ + └───┬──────────┬──────────┬───────────┬──────────────┬───────────────┘ + aws gcp azure 1password database + (SSM+SM) (Secret Mgr)(Key Vault)(Connect/SA) (ModuleCredential / Credential) + + materialized (default, external mgrs): write ─► sync into runtime store ─► process.env + direct: function ─► provider at cold start + (needs bootstrap token; cloud-native uses IAM) +``` + +- **AWS adapter** = the existing SSM Parameter Store + Secrets Manager work, consolidated. The draft + runtime loader collapses into **one** core loader behind the port (`parametersToEnv` / + `secretsToEnv` via the Parameters & Secrets Lambda extension); `SsmBuilder` becomes its IAM half. + This folds in the SSM runtime-loading draft (branch `feature/finish-ssm-based-env-management`) + rather than shipping it as a separate ADR. +- **GCP** (Secret Manager), **Azure** (Key Vault) adapters — turn the current stubs into real impls. +- **`database` adapter** — backs ADR-027's DB tiers (app-level module creds, instance + `Credential`/`config`); the same port, a DB transport. +- **`onepassword` / `vault`** — see below. +- **Transport is adapter-internal** (Lambda extension vs SDK vs platform reference vs DB query); the + port stays cloud-neutral. Refresh/caching is a **port policy** (per-invocation with TTL — on AWS the + TTL cache is provided by the Parameters & Secrets Lambda extension that `secrets-to-env` reads + through — so freshness/rotation works without redeploy). + +### 1Password / Vault +Two modes, both behind the same interface: +- **Source-of-truth sync (default):** secrets live in 1Password/Vault; at **deploy** the CLI/API + resolves `op://` / `vault kv` references and syncs them into the runtime provider store — runtime + stays cloud-native and fast. Answers "is it just their toolchain?" → largely **yes** for authoring. +- **Runtime adapter (optional):** resolve at runtime via 1Password Connect / Service Accounts or + Vault agent. More moving parts; only if the store should *be* 1Password/Vault. + +## Consequences + +### Positive +- One interface unlocks AWS/GCP/Azure/1Password/Vault + DB without further core changes once the type + is added (the ADR-016 promise) — adapters need no core edits. +- Collapses the three competing AWS env mechanisms into a single adapter. +- Runtime stays cloud-native even when the source of truth is 1Password/Vault. + +### Negative +- Real adapter work (GCP/Azure stubs → impls; 1Password/Vault new). +- The Lambda-extension transport has no clean cross-cloud analog — the port must not leak it. + +### Neutral +- Establishes provider selection in `appDefinition.plugins` alongside database/encryption/etc. + +### Resolved +- **Runtime mode:** support both; **`materialized` default** for external managers (see above). +- **1Password/Vault:** both modes (source-of-truth-sync **and** runtime adapter); sync is the default. +- **Per-connection (tier-3):** **DB by default.** A **pluggable non-Frigg credential source** — a + `credentialSource` adapter so an adopter can resolve tier-3 from their own store — is a considered, + overridable extension point, **deferred, not built now.** + +- **Precedence (resolved):** the routing map (ADR-029) fixes **one backend per (tier, env)** → no + in-tier collision by construction. For genuine overlaps (e.g. legacy build-time env vs the + routing-map backend), the **provider/routing-map source wins**; the **local** provider overrides + for the local env only; deploy validation **warns** (opt-in strict mode fails). + +## Open questions +- Layer/extension **default-on vs opt-in** (minor; with per-function scoping confirmed, the extension + attaches only to functions that need `direct` reads). + +## Alternatives Considered +- **Keep AWS-only and hand-roll other clouds per adopter.** Rejected: GCP is already live and Azure is + on the radar; a per-adopter fork multiplies maintenance and contradicts the cloud-agnostic reality. +- **A separate top-level plugin category outside the ADR-016 taxonomy.** Rejected: secrets/config fit + the existing typed-interface-plus-adapters model; a parallel mechanism would fragment plugin + selection (`appDefinition.plugins`). +- **`direct` runtime reads as the only mode.** Rejected: needs a bootstrap token in platform env for + external managers; `materialized` is the default and `direct` remains available. + +## Related +- [ADR-027: Configuration & Secrets — Model & Tiers](./027-configuration-and-secrets-model.md) +- [ADR-029: Variable & Secret Management](./029-variable-secret-management.md) +- [ADR-030: Configuration & Secrets — Docs & Adopter Maturation](./030-configuration-secrets-docs-and-maturation.md) +- [ADR-016: Plugins](./016-plugins.md) (plugin taxonomy this extends); `CloudProviderAdapter` port + + AWS adapter; `secrets-to-env.js`; the SSM draft on `feature/finish-ssm-based-env-management`. diff --git a/docs/architecture-decisions/029-variable-secret-management.md b/docs/architecture-decisions/029-variable-secret-management.md new file mode 100644 index 000000000..8d596700d --- /dev/null +++ b/docs/architecture-decisions/029-variable-secret-management.md @@ -0,0 +1,111 @@ +# ADR-029: Variable & Secret Management — Admin API, CLI, GUI + +**Status**: Proposed +**Date**: 2026-07-10 +**Deciders**: Sean Matthews + +> Proposed. The "how do humans/tools manage values across all tiers" half of the Configuration & +> Secrets work (ADR-027 is the model; ADR-028 is the storage/provider port; ADR-030 is docs & +> maturation). + +## Context + +Even with the tier model (ADR-027) and provider port (ADR-028), adopters need a way to *manage* +variables and secrets. Today there is **no `frigg secrets` / `frigg env` command**, no unified +surface, and management is scattered: hand-edited pipeline env, `setup-gh-env-secrets.sh` scripts, a +local-only `.env` editor in the management UI. We must not solve this by putting routing logic in the +CLI (it would diverge from any future GUI). + +## Decision + +**The management interface is a set of admin API routes; every front-end is a thin client of them.** + +- **Admin API is the one brain.** Routes (admin-authed, in the same admin-operation surface family as + ADR-005 / ADR-010; this ADR introduces its own `/api/v2/admin/variables` namespace) + accept a value plus its **category / scope / sensitivity**, and the **API decides where to store + it** based on the Frigg base app config/definition (which providers/tiers are configured per + ADR-027/028) and the target environment. Storage routing lives here, once. +- **The CLI is a client of those routes** — and, because they're plain admin API, the same calls are + **curl-able** and a **hosted GUI/UI** can drive all layers of variables and secrets through the + identical API. One API, many front-ends, no duplicated logic. +- **Smart routing** = a function of (sensitivity: config vs secret) × (scope: platform / + app-level-module / per-connection) × (environment) × (configured provider). The caller states + intent; the API places it: platform → provider store; app-level module cred → tier-2 store + (DB/provider); per-connection → `Integration.config` / `Credential`. +- **Routing map (per tier × environment → backend).** The placement rules live in a declared map so + the same intent routes differently per env, and **"unified 1Password" = set every tier's backend to + `1password`** (persona: "manage all of it in one place"). Mixed setups are just a different map. + ```js + // app definition (illustrative) + secretsRouting: { + prod: { platform: '1password', appModule: '1password', perConnection: 'database' }, + local: { platform: 'local', appModule: 'local', perConnection: 'database' }, + } + ``` +- **Local parity.** A first-class **`local` provider** (gitignored file / docker DB) is the default + backend for local envs, so `frigg start` + the CLI/UI work with **zero cloud**. Because backend is + per-environment, a dev who lives in 1Password can point `local` at it too (via `op`), ideally + `materialized` so there's no runtime dependency. +- **Scoping is emitted, not manual.** On deploy/build the API + infra derive each function's variable + set / IAM from the integration→module graph (ADR-027) — the management layer never hand-maintains + per-function manifests. + +``` + CLI curl GUI / UI (thin clients — no routing logic) + └─────────┼──────────┘ + ▼ + ┌──────────────── Admin API ─────────────────┐ + │ reads routing map (tier × env → backend); │ + │ places / resolves each value once │ + └───┬──────────────┬─────────────────┬────────┘ + platform app-module per-connection + provider store ModuleCredential/ DB Credential / + (SSM/Secrets/ provider Integration.config + 1Password/local) +``` + +**Illustrative CLI (thin wrappers over admin routes):** +``` +frigg config set FEATURE_X=on --stage prod # platform config → SSM / App Config +frigg secret set DATABASE_URL --stage prod # platform secret → Secrets Manager / KV / 1Password +frigg module cred set hubspot --client-id … --secret … --scopes … --stage prod # app-level module → tier-2 store +frigg integration config set key=val # instance config → Integration.config (DB) +``` +``` +GET/PUT /api/v2/admin/variables # curl or GUI hit the same routes the CLI does +``` + +## Consequences + +### Positive +- One place decides storage routing → CLI, curl, and a future GUI stay consistent by construction. +- Adopters manage all tiers through a uniform surface; a hosted UI becomes "just another client." +- Reuses the established admin-auth/admin-operation surface rather than a new mechanism. + +### Negative +- Requires admin write routes that mutate real secret stores — needs careful auth, redaction, audit. +- Routing rules must be well-specified so `set` is predictable. + +### Neutral +- Positions variable/secret management as part of Frigg's admin API, alongside reporting/scripts. + +## Open questions +- **Routing:** inferred (from sensitivity/scope) vs explicit subcommands/flags? +- **Auth:** reuse the admin API key (per ADR-010, which folded the earlier separate reporting key into + one admin key), or mint a dedicated management credential given these routes mutate secret stores? +- **Read-back & redaction:** can values be read back (never secrets in plaintext?), and what's audited? +- Where the **routing policy** itself is declared — app definition vs API config. + +## Alternatives Considered +- **Put routing logic in the CLI.** Rejected explicitly: it would diverge from any future GUI; the + routing decision must live once, in the admin API, so every front-end stays consistent. +- **A dedicated non-admin secrets service/endpoint family.** Rejected: reuses none of the established + admin-auth/admin-operation surface and adds a second security perimeter to harden. +- **Per-tier bespoke commands with no routing map.** Rejected: the routing map is what makes "unified + 1Password" and mixed per-env setups a config change rather than new code. + +## Related +- [ADR-027: Configuration & Secrets — Model & Tiers](./027-configuration-and-secrets-model.md) +- [ADR-028: Secrets & Config Provider Plugin](./028-secrets-config-provider-plugin.md) +- [ADR-030: Configuration & Secrets — Docs & Adopter Maturation](./030-configuration-secrets-docs-and-maturation.md) +- ADR-005 (Admin Script Runner — admin auth/surface), ADR-010 (admin operations / admin API key). diff --git a/docs/architecture-decisions/030-configuration-secrets-docs-and-maturation.md b/docs/architecture-decisions/030-configuration-secrets-docs-and-maturation.md new file mode 100644 index 000000000..ed819614c --- /dev/null +++ b/docs/architecture-decisions/030-configuration-secrets-docs-and-maturation.md @@ -0,0 +1,100 @@ +# ADR-030: Configuration & Secrets — Docs & Adopter Maturation + +**Status**: Proposed +**Date**: 2026-07-10 +**Deciders**: Sean Matthews + +> Proposed. The enablement half of the Configuration & Secrets work (model = ADR-027, provider = +> ADR-028, management = ADR-029). Covers how we document it for adopters and how a Frigg app is +> expected to grow over time. The full cross-pattern worked example is a **post-implementation doc +> deliverable**; an illustrative target-state version is included below. + +## Context + +The model (ADR-027–029) is only useful if adopters can (a) find a **free, fast on-ramp**, (b) +understand **when to reach for which tier/provider**, and (c) **grow** without rework. Today the docs +describe a single flat env list; there is no maturation story and no guidance separating the four +concerns that get mashed together (infra, framework behavior, dev/feature flags, API-module app +credentials). Adopters therefore hit "env overload" with no documented path out. + +## Decision + +Ship a dedicated **"Configuration & Secrets" guide** in the Frigg docs, organized around a +maturation journey and a cross-pattern worked example. + +### 1. The guide (pages / questions it must answer) +- **Mental model** — the tiers (platform / app-level module / per-connection) × {config, secret}, the + two planes (management vs runtime), and the isolation invariant. *(from ADR-027)* +- **When to reach for which** — the decision rule (scope first, then sensitivity). +- **Per-environment setup** — how to set values per stage; the routing map. *(ADR-029)* +- **"Only what a function needs"** — scoping derived from the integration→module graph. *(ADR-027)* +- **On-demand vs env** — why per-connection creds are always on-demand. *(ADR-027)* +- **Choosing a provider** — AWS / GCP / Azure / 1Password / Vault / local, and materialized vs + direct. *(ADR-028)* +- **Managing values** — the `frigg` CLI / admin API / GUI, and the routing map. *(ADR-029)* +- **Local development** — the local provider; zero-cloud `frigg start`. *(ADR-029)* +- **Migrating** — moving from a flat pipeline env to structured tiers without downtime. + +### 2. The adopter maturation journey +Document the growth path explicitly (tied to ADR-027's L0–L3) — the headline is **"start with all +globals set manually in your pipeline; graduate to managed config as you scale."** + +| Level | Trigger | What the adopter does | Where values live | +|---|---|---|---| +| **L0 — Manual (free/fast)** | Day one | Set global envs in the deploy pipeline (`appDefinition.environment`) | Pipeline → `process.env` (global) | +| **L1 — Platform secrets** | First real secret / rotation need | Move secrets to a provider store | Secrets Manager / SSM / provider | +| **L2 — Structured module creds** | **Env overload** (10+ modules × OAuth) | Move app-level module creds to the tier-2 store; scope per function | `ModuleCredential`/`ModuleConfig` (or provider) | +| **L3 — Managed / multi-provider** | Fleet / compliance / DX | Multi-provider (GCP/Azure/1Password/Vault), per-function least-privilege, GUI | Chosen provider(s) via the routing map | + +Each level is **opt-in and backward-compatible** — no adopter pays for a tier they don't use, and L0 +keeps working forever. + +### 3. Cross-pattern worked example +Provide **the same env set shown across adopter patterns** so an adopter can locate themselves. This +is a **post-implementation deliverable** (the real doc lands once ADR-027–029 ship); the illustrative +target-state below captures intent. + +**The shared env set:** `DATABASE_URL` (platform secret), `FEATURE_X` (platform/dev flag), +`HUBSPOT_CLIENT_ID` + `HUBSPOT_CLIENT_SECRET` (tier-2 app-level module cred), a HubSpot **access +token** (tier-3 per-connection). + +| Value | A: Day-one (L0) | B: Growth (L2, AWS) | C: 1Password-unified | D: Multi-cloud (GCP) | +|---|---|---|---|---| +| `DATABASE_URL` | pipeline env → global | Secrets Manager → global env | 1Password → materialized → global env | GCP Secret Manager → global env | +| `FEATURE_X` | pipeline env → global | SSM → global env | 1Password → materialized | GCP Runtime Config → global | +| `HUBSPOT_CLIENT_ID/SECRET` | pipeline env → **all** functions | tier-2 store → **only** HubSpot functions | 1Password (tier-2) → HubSpot functions | GCP SM (tier-2) → HubSpot functions | +| HubSpot **access token** (per-conn.) | DB `Credential`, on demand | DB `Credential`, on demand | DB `Credential`, on demand | DB `Credential`, on demand | +| Routing map | — (all pipeline) | `{platform: aws, appModule: aws, perConnection: database}` | `{platform: 1password, appModule: 1password, perConnection: database}` | `{platform: gcp, appModule: gcp, perConnection: database}` | + +Two things the example makes obvious: **per-connection creds stay in the DB across every provider +pattern above** (the isolation invariant holds regardless of which cloud/secret store backs tiers 1–2) +— note this is the provider axis; whether a connection is per-end-user or an admin-authored **global +entity** (ADR-024) is a separate adoption choice, and both still live in the DB — and **moving from +A→B→C is a routing-map + storage change, not an app-code change.** + +## Consequences + +### Positive +- Gives adopters a legible on-ramp and a growth path; directly addresses the env-overload wall. +- The worked example lets adopters self-locate and see that migration is config, not code. + +### Negative +- Docs must track ADR-027–029 as they're implemented (drift risk until they ship). + +### Neutral +- Establishes the maturation levels as the shared vocabulary for docs, CLI help, and onboarding. + +## Alternatives Considered +- **Fold this guidance into ADR-027 as a section rather than its own ADR.** Rejected: the enablement/ + maturation story and the cross-pattern worked example are substantial and adopter-facing; a dedicated + ADR keeps ADR-027 focused on the model and gives docs a single anchor to track. +- **Write the full cross-pattern worked example now.** Deferred: the faithful version depends on the + shipped shapes of ADR-027–029, so an illustrative target-state table is included and the complete + example is called out as a post-implementation deliverable. + +## Related +- [ADR-027](./027-configuration-and-secrets-model.md) (model & maturation levels), + [ADR-028](./028-secrets-config-provider-plugin.md) (providers), + [ADR-029](./029-variable-secret-management.md) (management). +- [ADR-024: Global Entities](./024-global-entities.md) (per-connection vs global adoption choice). +- Frigg docs site (`docs/`) — target home for the guide. diff --git a/docs/architecture-decisions/README.md b/docs/architecture-decisions/README.md index bc7c9bfbb..bb5fc22f2 100644 --- a/docs/architecture-decisions/README.md +++ b/docs/architecture-decisions/README.md @@ -43,6 +43,10 @@ An ADR documents a significant architectural decision made in the project, inclu | [024](./024-global-entities.md) | Global Entities | Proposed | 2024-12-18 | | [025](./025-agent-harness.md) | Agent Harness | Proposed | 2026-06-09 | | [026](./026-evals.md) | Evals | Proposed | 2026-06-09 | +| [027](./027-configuration-and-secrets-model.md) | Configuration & Secrets — Model & Tiers | Proposed | 2026-07-10 | +| [028](./028-secrets-config-provider-plugin.md) | Secrets & Config Provider Plugin | Proposed | 2026-07-10 | +| [029](./029-variable-secret-management.md) | Variable & Secret Management — Admin API, CLI, GUI | Proposed | 2026-07-10 | +| [030](./030-configuration-secrets-docs-and-maturation.md) | Configuration & Secrets — Docs & Adopter Maturation | Proposed | 2026-07-10 | ## Conventions