Broker tenancy + per-IP grant cap + fail-closed guards#91
Conversation
Managed apps whose partner account cannot be split per user (an account bound to a provider campaign) had no isolation between pilot callers: upstream every caller is the same customer, so the partner cannot tell them apart and nothing else was doing it. Any caller could reference any resource id on the shared account, and account-wide lists were returned verbatim. Tenancy (internal/broker/tenancy.go, owner.go) makes the broker the isolation layer. A resource is claimed by whoever creates it; a reference in the path, query, or body is checked against the ledger before the request is forwarded; list responses are filtered to the caller's rows, and account summaries are recomputed from the caller's ledger. Deny-by-default: a resource with no ledger row belongs to nobody and is refused to everybody. Also closes several defaults that resolved to "allow" when unset: - allow entries are method-scoped, so a registry can express reads-yes/writes-no instead of one verb implying all of them. - creditEnabled keys off the credit block's presence, not a non-zero seed, so seed_credits:0 means a zero budget rather than no metering at all. - max_identities_per_ip defaults to a cap; only an explicit negative disables it. - access keys gate apps that grant something of value; the broker refuses to boot if an app requires a key that is not configured. - store and upstream errors return a correlation id instead of internal hostnames, addresses, and driver state. Tests are written as attacks: cross-tenant use via a body reference, reads of another tenant's resources, ownership takeover, type confusion, parser- differential duplicate keys, and count fields that leak the unfiltered set.
…guards Turns the per-IP identity cap into a GRANT cap: the (N+1)th distinct identity on a source IP is recorded with a zero budget instead of being refused. It can still make free (read) calls; every priced call hits 402. This bounds how much free budget one network can mint without hard-locking a legitimate user behind a shared NAT out of the app — so the cap can be set aggressively low. - deploy/setup-broker-tls.sh: the generated nginx `location /` now sets X-Real-IP. It is the only source-IP signal the broker trusts, and the per-IP cap collapses to a single 127.0.0.1 bucket without it. This was the origin of the real-IP regression; regenerating nginx on deploy no longer reintroduces it. - registry: a classic managed app that forwards with header auth but no header name is now rejected at load instead of 502-ing on first traffic. - deploy/agentphone-tenancy.py: re-enables the spend routes (messages/calls/ numbers) now that tenancy binds each to an owned resource and the grant cap bounds exposure; keeps webhooks/reactions withheld (account-level / unownable). Tests updated to assert the grant-cap contract: a capped identity reads (200) but cannot spend (402), while a funded identity spends normally.
The broker emits one `ACCESS {json}` line per forwarded request to stdout →
journald, which the monitoring crawler ingests. This middleware was lost when
the broker was last rebuilt from a branch without it, so the pipeline went dark.
Restoring it here keeps it alongside the tenancy/security changes, so a redeploy
from this branch does not clobber one for the other.
- internal/broker/accesslog.go: WithAccessLog middleware, AccessEvent (stable
JSON contract), StdoutSink, and the X-Pilot-Method (pilotctl command capture)
and X-Pilot-Unrouted (skip unknown-app / scanner noise) header constants.
- internal/broker/broker.go: mark the two unrouted 404 paths with X-Pilot-Unrouted
so scanner traffic never counts as broker usage.
- cmd/broker/main.go: wire the middleware with a -broker-name label (defaults to
BROKER_NAME env or hostname); /gw/ health/usage paths pass through unlogged.
Reduced to the stdout→journald path (what the pilot-publish crawler reads); the
remote-push HTTPSink for the smol broker is intentionally not included here.
|
Added two operational fixes surfaced during live deployment (both prod-verified on pilot-publish):
Ledger note: pruned unused-full (credits==seed) 127.0.0.1 provision rows so the cap math is clean; spent rows and active identities kept. Verified real IPs now recorded and orthogonal seeds fresh users at $5. |
Restores internal/broker coverage above the security-critical gate: the new tenancy, access-key, and ownership code was under-covered. Adds direct tests for access-key parse/verify (incl. fail-closed with no keys and the require-key gate), the SQLite ownership ledger (claim first-writer-wins, OwnerOf/OwnedSet/ Release, Owns) and credit ledger (debit/refund/settle/clamp), and account-summary redaction (owned-count recompute + field redaction).
Summary
Makes the managed-key broker enforce per-caller resource isolation for apps whose partner account cannot be split per user (e.g.
io.pilot.agentphone, an account bound to a provider campaign), and closes several defaults that resolved to "allow" when a field was unset.Upstream, every pilot user of a shared-account app is the same customer, so the partner cannot tell them apart and nothing enforced isolation. Any caller could reference any resource id on the shared account. The broker is now the tenancy layer.
What changed
Tenancy (
internal/broker/tenancy.go,owner.go,store*.go)Per-IP grant cap (
broker.go)Fail-closed defaults
allowentries are method-scoped (a registry can express reads-yes/writes-no).creditEnabledkeys off the credit block's presence, not a non-zero seed (soseed_credits: 0means a zero budget, not "no metering").max_identities_per_ipdefaults to a cap; only an explicit negative disables it.accesskey.go) available for apps that grant something of value; broker refuses to boot if an app requires a key that isn't configured.Deploy (
deploy/setup-broker-tls.sh,deploy/agentphone-tenancy.py)location /now setsX-Real-IP(the only source-IP signal the broker trusts; without it the per-IP cap collapses to one bucket).Tests
Tests are written as attacks: cross-tenant use via a body reference, reads of another tenant's resources, ownership takeover, type confusion, parser-differential duplicate keys, count fields that leak the unfiltered set, and the grant-cap contract (capped identity reads but cannot spend).
Live verification (pilot-publish)
Deployed and exercised via a real pilot node against all four managed apps:
🤖 Generated with Claude Code