diff --git a/docs/cloud-architecture-and-scalability.md b/docs/cloud-architecture-and-scalability.md new file mode 100644 index 000000000..89ff0beac --- /dev/null +++ b/docs/cloud-architecture-and-scalability.md @@ -0,0 +1,405 @@ +# ORG2 Cloud — Architecture and Scalability Reference + +Last consolidated: 2026-07-24. This is the single living reference for the +ORG2 Cloud scalability program. It supersedes and replaces +`docs/cloud-scalability-audit-2026-07-23/` (three audit reports + priority +README) and `docs/cloud-broadcast-and-storage-design-2026-07-24/` (H4/H5 +design notes + scale-out note), both removed. Server source of truth: +`ORGII-cloud-infra/supabase/migrations/` (0001 frozen baseline + numbered +increments). Client source of truth: `src/features/Org2Cloud/**`. + +--- + +## 1. System topology + +Three layers, one managed Supabase project: + +| Layer | Role | +| --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Desktop (local SQLite) | UX source of truth. Cloud is a sync target/replica; every read the UI renders comes from local storage, populated by event-driven pulls. | +| Supabase (single project `org2_cloud` schema) | (a) Postgres behind an **RPC-only surface** — all client access goes through `security definer` RPCs; direct table SELECT/DML grants to `anon`/`authenticated` are revoked, so RLS policies on data tables are defense-in-depth, not the API. (b) **Realtime**: one private org channel `presence:org:` per active client (presence + client broadcasts + server `org-db-changed` signals), plus one `postgres_changes` subscription on the caller's own `org_memberships` row (Slice A self-eviction). (c) **Storage**: private `replay` bucket holding frozen replay segments as raw-gzip objects. | +| Vercel (`apps/org2-cloud-web`) | Auth bridge (hosted login → desktop deep link), Stripe checkout/webhook/billing portal, replay **signer** (`/api/replay/sign`), orphan **GC cron** (`/api/replay/gc`, daily 04:00 UTC), inline-segment **backfill** (`/api/replay/backfill`). Holds `SUPABASE_SERVICE_ROLE_KEY` and Stripe secrets; desktop/browser only ever see the Supabase URL, the anon key, and the user's own tokens. | + +**Capability probe.** `get_cloud_capabilities()` returns +`{broadcastSignals, storageSegments, replaySignerGrants}` (0005/0006). The +client probes once per endpoint URL and remembers the result +(`org2CloudCapabilities.ts`; success-only caching); `PGRST202` (function +missing) ⇒ legacy backend ⇒ the client keeps the old wire (postgres_changes +signal channels, inline segment payloads, no signer). The same +probe-and-remember pattern covers the 0004 paged listings (opt-in +parameters; legacy call shapes stay byte-identical). Net effect: every +server upgrade is backward compatible (old clients ignore additive keys) +and forward compatible (new clients degrade against old backends). +`ORG2_VALIDATION` errors never trigger fallback — real bugs must surface. + +--- + +## 2. Scalability model, by resource + +### Connections + +Realtime bills peak concurrent connections per billing cycle (Pro includes +500; $10/1,000 over) plus messages ($2.50/M over 5M). The client holds a +**foreground lease**: connected while focused; on blur the release is +deferred by a **45s grace** (`REALTIME_LEASE_RELEASE_GRACE_MS`; refocus +cancels the timer); hidden/pagehide release immediately. This keeps the +peak-connection billing win of release-on-blur while eliminating the +teardown/recovery burst on routine alt-tabs (and the presence +viewer-chip-vanishes regression). + +Recovery on (re)subscribe edges is **disconnect-gated** +(`org2CloudRealtimeRecovery.ts`): short disconnect (<5min, or a full +refresh within the last 30s) → cursor delta; only long disconnects pay a +full listing + a force-bumped comment refresh for the active session +(bypassing the 30s comments TTL that used to swallow short-blur comments). +Reconnects carry 0–3s jitter (`reconnectAfterMs` override); comment-fetch +retries back off exponentially (10s → 5min cap, failure count preserved +across evictions). + +Channels per client: 1 socket; post-0005 topology is 2 channels (private +org channel + Slice A postgres_changes), down from 4. `eventsPerSecond: 5`, +presence budget 5/30s. Subscription scope is O(1) per client: active org +only. + +### Realtime messages + +Signals ride **Broadcast-from-Database** (0005): every data-table write +funnels through `org2_cloud.nudge_org_signal(org_id, kind)`, which upserts +a debounce row and — iff the debounced bump fired — `realtime.send`s an +`org-db-changed {kind}` broadcast to the existing private org channel. +Authorization is evaluated once at channel join (the 0001 +`realtime.messages` policies), not per change × per subscriber as +`postgres_changes` did on the shared single-threaded WAL poller — that +per-subscriber RLS evaluation was the platform's first scalability wall +(a 20-member org at ~2k writes/day ≈ 2.4M messages/month alone). + +- **Server debounce**: one bump per **(org, kind)** per **1s** (0006 PART 8; + 0003 started at 250ms per org, 0005 kept it, 0006 reshaped the PK to + `(org_id, kind)` so planes debounce independently — no kind shadowing). +- **Client throttles**: per-plane (`sessions | comments | projects | +workItems | roster | policy`) 60s trailing-edge refresh (`SignalPlane`), + with a 5min coarse safety net for legacy backends. +- **Publication cleanup** (0006 PART 6): `org_change_signals` left the + `supabase_realtime` publication; the rows are now purely debounce clocks. +- **Slice A exception**: `org_memberships` stays published and the client + keeps one user_id-filtered postgres_changes subscription forever — a + member removed while disconnected can never be reached by join-time-auth + broadcast (`is_org_member()` already false). + +Broadcast is fire-and-forget; the loss backstop is the existing +SUBSCRIBED-edge recovery on the org channel's join edge. + +### Query cost + +- **Keyset pagination everywhere**: sessions listing pages by + `(updated_at, session_id)` (0004 PART 2, ≤500/page); collab-state pages a + unified `(updated_at, kind, id)` cursor over the projects ∪ work-items + union (0004 PART 3); replay events page by `seq` + (`cloud_get_session_events_page`, 0002, default 16 / max 64 segments, + tail only on the final page); comments patch via `p_since` deltas (0004 + PART 6) keyed on the single `state_changed_at` stamp (0006 PARTs 9–11). + All paging parameters are opt-in; parameterless calls stay + byte-identical for old clients. +- **Batched entitlements**: `list_my_orgs` resolves each row's + `entitlement` in the same round-trip via the exception-wrapped + `entitlement_state_or_null` (0004 PART 1) — replaces the 1 + N + `get_entitlement_state` fan-out per roster read. +- **O(1) quota counters**: `orgs.stored_bytes_total` is delta-maintained by + append/rewrite/erasure (0003); the quota gate is counter + delta > cap, + not a per-push scan of every session row of the org. + `reconcile_org_stored_bytes()` (service-role) recomputes from ground + truth for ops/nightly reconciliation. +- **Materialized comment counts**: `cloud_sessions.comment_count` / + `unresolved_comment_count` maintained by the comment RPCs (0003) — + the per-row COUNT LATERAL in the sessions listing is gone. + +### Locks + +- **Canonical acquisition order** (0003 header is the authority), org-level + slots: 1 advisory xact locks (per org/session plane) → 2 `cloud_sessions` + rows → 3 `usage_monthly` row → 4 `orgs` row → 5 comment rows → 6 signal + row, always statement/transaction-final. The 0003 reorder fixed the + `usage_monthly` ↔ signal-row inversion (40P01 window between a member + creating a session and a member pushing). +- **Work-item narrowing** (0004 PART 5): `cloud_upsert_work_item` takes the + owning project row lock only when the short-id allocator actually + advances; steady-state edits serialize only on the per-item advisory + lock, so N members editing N items of one project no longer serialize. +- **Commit-deferred signal triggers** (0006 PART 7): the five data-plane + signal triggers are `CONSTRAINT TRIGGER ... DEFERRABLE INITIALLY +DEFERRED` — the per-org signal-row lock window shrank from "rest of the + first writer's transaction" to the commit instant, and rollbacks emit + nothing. A **per-org advisory xact lock** inside `nudge_org_signal` + totally orders the commit-instant signal writes, making cross-kind + commit deadlocks impossible. The 3 policy RPCs' nudges stay inline + (short single-row transactions, already commit-adjacent). + +### Storage + +Frozen replay segments (`seq >= 1`, immutable) live in the private +`replay` bucket as **raw gzip** objects (no base64 +33%), named +`{org_id}/{session_id}/{epoch}/{seq}-{segment_hash}.gz` where +`segment_hash` = sha256 hex of the pre-gzip canonical bytes — +immutable-by-name, retry-idempotent, collision-safe inside the org/session +auth domain. The mutable tail (`seq = 0`) stays inline in Postgres +(`payload_gz`), keeping append OCC single-transaction; a CHECK forbids a +storage-form tail. + +Writes are **upload-blobs-first, commit-manifest-second**: the append/ +rewrite RPC verifies each claimed object exists in `storage.objects`, that +its name parses to the exact (org, session, epoch, seq) being committed +and embeds the claimed hash, and charges quotas with the **server-measured** +size from object metadata — no client attestation. Missing/mismatched +object ⇒ `ORG2_VALIDATION`, nothing committed. Read RPCs return +`storagePath` XOR `payloadGz` per segment. Members read/write objects +directly with their JWT via storage RLS policies delegating to +`can_read_replay_object` / `can_write_replay_object` (which reuse the SQL +session-access ladder verbatim); share-token guests go through the Vercel +signer (section 5). Postgres cannot delete bucket objects, so +rewrite/hard-delete orphan them by design; the `replay_orphan_objects` +view (objects >24h old with no matching segment row) feeds the daily cron +sweeper. + +### Client discipline: strictly event-driven + +No recurring timers, no polling. Sync passes fire on concrete events: +signal broadcasts, agent turn-terminal, outbox writes, boot, focus/ +visibility edges, `online`, org activation. Hidden windows still push +outbound (turn-terminal + outbox events pass the hidden gate; inbound +nudges wait for visibility). Measured: after an activation burst the app +goes to complete network silence — 0 "likely polling" entries in the cmd+5 +API panel, 0% idle CPU (section 4). + +--- + +## 3. Change ledger + +Migration policy (since 2026-07-23): **0001 is a frozen baseline** — the +live project can no longer be wiped, so schema changes ship as numbered +idempotent increments applied in order. `schema_version()` stays 1; the +file sequence is the version record. + +| Migration | What shipped | Applied | Validation | +| ------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `0001_org2_cloud_schema.sql` | Consolidated full schema (flattened from the original 0001–0014 + comment-task system later removed): orgs/memberships/invites, sessions + segments + shares, projects/work items, comments, entitlements/Stripe tables, GDPR RPCs, realtime publication + `org_change_signals`, plan seed. | Live rebuild 2026-07-07; declared frozen 2026-07-23 | Adversarially reviewed at consolidation (single-txn safe, byte-equivalent to sequential apply); live probe `schema_version()=1`, 17 tables / 55 functions at freeze-era count | +| `0002_bounded_session_event_pages.sql` | `cloud_get_session_events_page` — seq-keyed paging of replay reads (≤64 segments/page, tail on final page), same auth/summary contract (audit H2). | 2026-07-23 | Live paste; signature probe 403-not-PGRST202; client loops on `afterSeq` (note: `p_after_seq` is int4 — head-reads use 2147483647, never `MAX_SAFE_INTEGER`) | +| `0003_scalability_p0.sql` | H1 lock-order fix; double signal-bump elimination; C3 O(1) `stored_bytes_total` counter + `reconcile_org_stored_bytes`; C1 partial (250ms/org signal debounce); M4/C2 partial (materialized comment counts); M2/M3 indexes. Documents the canonical lock order. | 2026-07-23 | Offline: brew postgresql@16, literal 0001→0002→0003 apply; A/B concurrency proof that the old GDPR statement order deadlocks (40P01) and the shipped order does not. Live: zero-drift verify (reconcile `fixedOrgs:0/fixedSessions:0`, per-org counters exact) | +| `0004_roster_and_listing_scalability.sql` | 7 PARTs: B1 batched entitlements in `list_my_orgs`; C2 sessions keyset paging; H3 collab-state unified-cursor paging; H3 `gc_collab_tombstones` (90d, keeps tombstone projects referenced by live items); M1 work-item lock narrowing; M4 comments `p_since`; L2 `gc_bookkeeping`. Old 2-arg signatures dropped, not overloaded (PostgREST cannot disambiguate). | 2026-07-24 | Offline: disposable PG16 six-item matrix (fresh/delta wire byte-identical, idempotent replay, paged∪ == legacy, GC ACL). Live: 4 new-signature probes 403(42501)≠PGRST202; `list_my_orgs` 7/7 orgs with full 19-key entitlement; paged==legacy walks on the active org | +| `0005_broadcast_and_storage_offload.sql` | 11 PARTs: H4 — `nudge_org_signal` helper (debounced bump + `realtime.send` on the private org channel), trigger kind mapping, roster trigger, 3 policy RPCs onto the helper, `get_cloud_capabilities`; H5 — `storage_path` column + CHECKs, `replay` bucket bootstrap + storage RLS helpers/policies, storage-form append/rewrite with server-measured quota, `storagePath` XOR `payloadGz` reads, `replay_orphan_objects` view. Guarded DO blocks NOTICE dashboard SQL where the runner lacks `storage.objects` ownership. | 2026-07-24 | Offline: 9-item matrix (idempotency, wire A/B vs 0004, manifest-vs-missing-object, mixed inline/storage quota deltas + mixed-org reconcile zero drift, orphan view, RLS helper ladder). Live: verify tail all `t`; bucket + both policies created by the migration itself; broadcast delivery, member storage reads, retract verified end to end | +| `0006_guest_grants_and_signal_cleanup.sql` | 12 PARTs: `replay_read_grants` (sha256-at-rest, 60s expiry, service-role-only table) + `cloud_authorize_replay_read` (anon-callable mint; auth ladder verbatim from the events RPC) + `cloud_redeem_replay_grant` (service-role, atomic single-use `DELETE..RETURNING`, object list re-derived at redeem under the current epoch) + `replaySignerGrants` capability + grant GC; signal cleanup — publication drop (PART 6), commit-deferred triggers (PART 7), per-(org, kind) 1s debounce + advisory total order (PART 8); `state_changed_at` comment stamp + single-stamp `p_since` (PARTs 9–11); `cloud_ops_stats()` service-role capacity snapshot (PART 12). | Final revision (eb793d9, all 12 PARTs incl. per-kind debounce, `state_changed_at`, ops stats) applied 2026-07-24, verify all `t`. | Offline adversarial matrix incl. deferred-trigger semantics (mid-tx zero sends / one send at commit / rollback zero) and the proof that removing the advisory lock reintroduces a deterministic 40P01. Live: `/api/replay/gc` 200 with key / 401 without; `/api/replay/sign` opaque 401 on a fake grant; pg_locks A/B in section 4 | + +Client PRs (ORGII, all `src/features/Org2Cloud/**` unless noted): + +| PR | Branch | Scope | Gates | +| ---------------------------------------------------------------------------------- | ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | +| #507 (merged `f207fa035`) | `codex/reclaim-cloud-realtime-connections` | Foreground-lease reclaim epic + audit P0 fixes: 45s release grace (hidden/pagehide immediate), A2 full-payload hash seed (+`avatarUrl`), disconnect-gated delta recovery, comments force-bump on full recovery, 0–3s reconnect jitter, comments exponential backoff (incl. the later `consecutiveFailures`-across-evict fix), hidden-outbound (turn-terminal + outbox pass the hidden gate), full listings converged to the active org. | tsc ours 0, 676/676 feature tests; CI green on merge; dual-instance live matrix (section 4) | +| #509 (merged `7ae39f13c`) | `codex/batch-org-entitlement-hydration` | 0004 client: entitlement seeding from `list_my_orgs` (`seedOrgEntitlement`/`hydrateOrgEntitlements`), sessions keyset paging, collab-state unified-cursor paging (ascending keyset ⇒ mid-walk writes only move into the unread tail; 50-page fuse), PGRST202 fallback + per-endpoint memory. | Full suite 6156/6156; live fallback path verified against pre-0004 shape; post-apply cmd+5 verification (section 4) | +| #511 (merged `93056a38a`) | `codex/broadcast-and-storage-offload` | 0005 client: capability probe, Slice B/B-roster channels skipped when `broadcastSignals`, `org-db-changed` on the presence channel, recovery moved to the org-channel join edge; `org2CloudStorageClient` (raw-gzip PUT/GET), segment codec raw-bytes variant, upload-then-manifest push, `storagePath` XOR `payloadGz` decode, `ORG2_VALIDATION` never falls back. | Suite green at merge; live e2e: broadcast chain + 73-object storage push (section 4) | +| #532 (open; branch `codex/replay-guest-access-and-signal-tuning`, tip `af9eea28a`) | this branch | 0006 client: guest signed replay reads via the signer (single-flight URL cache, one re-authorization on expiry, typed signer errors + single retry), B3 listing 2s coalesce window (a full fetch is never satisfied by a delta), comments `p_since` delta (force path stays full to cover un-resolve on pre-0006 backends), policy broadcasts also bump the roster version (kind-shadowing symmetry), per-kind `SignalPlane` dispatch (60s/plane + 5min coarse safety net), Rust import-watermark incremental ingest (`imported_history/watermark.rs` — kills the ~90s boot reparse). | 941/941 → 946/946 across batches; clippy clean | + +--- + +## 4. Measured evidence + +All numbers from live dual-instance runs (accounts Neonforge98/VantaNode, +shared org "CU Vanta Shares 0721") or pg_locks instrumentation on a +disposable PG16, 2026-07-23/24. + +| Claim | Measurement | +| ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Entitlement fan-out collapsed (B1) | First activation of an org: `get_entitlement_state` exactly ×1 (measured ×7 for the same trigger pre-0004); sessions/collab/members/scopes ×1 each | +| Event-driven, zero polling | Org activation burst ≈48 calls then **complete silence**; switching back to an already-activated org = 0 remote calls; 0 "likely polling" in the cmd+5 API panel across the run; idle CPU 0%, RSS stable (142MB / 95MB, no leak) | +| Restart write storm gone (A2) | After the hash-seed fix: 0/8 pushed sessions re-upserted on restart (previously 8/8 byte-identical re-sends + broadcast) | +| Lease grace keeps live path (A1) | Blur-but-visible within grace: subscribe count stays 1 (socket kept), new work items arrive live without refocus; >2min background → refocus produces one delta recovery, items arrive | +| C1 lock window (pg_locks A/B) | Concurrent same-org writer blocked **1.54s** behind a long first transaction with mid-tx AFTER-ROW signal triggers; **0.015s** with 0006 commit-deferred triggers | +| Deferred-trigger semantics | Mid-transaction: zero `realtime.send`s; commit: exactly one debounced send; rollback: zero (send is transactional) | +| Storage offload e2e (H5) | Boot pass pushed **73 raw-gzip objects** to the `replay` bucket (epoch 1); read RPC returns `storagePath` with no `payloadGz`; **non-owner member JWT direct read 200** + gunzip + sha256 == hash embedded in the path; **anon read 400**; retract cleaned the cloud listing (the 73 objects became the orphan-view/cron design case) | +| Broadcast e2e (H4) | Raw supabase-js subscription on the private channel receives `org-db-changed {workItems}` (write → trigger → broadcast); desktop: RPC write on instance 2 lands in instance 1's SQLite on the 60s coarse trailing edge; **delete tombstone propagates** (live to the held-socket peer; on refocus delta to the released one); 2.5min of zero calls after the delete | +| Comment delivery | RPC comment → peer woke within 1s and pulled exactly one listing delta (materialized `comment_count` arrives) | +| Hidden outbound | Long streaming reply, cmd+H 1.5s after send: server received the push 20s into hidden (events 5→6) | + +--- + +## 5. Operations runbook + +### Migrations + +- Paste numbered files **in order** into the Supabase SQL editor (or + `supabase db push`). Every increment is idempotent — safe to re-paste; one + transaction, rolls back on failure. +- Statements touching objects the SQL-editor role may not own + (`storage.objects` policies, publication membership) are wrapped in + guarded DO blocks that **NOTICE the exact SQL** to run in the dashboard as + `supabase_admin` instead of failing the paste. Read the NOTICEs. +- Every migration ends with **verify tails** — SELECTs with expected + outputs (`t / t / ...`, expected row counts) — run them and check before + declaring applied. Update the ledger in `ORGII-cloud-infra/README.md`. +- **Wipe is forbidden.** Since 2026-07-23 the live project cannot be + dropped and rebuilt; 0001 is frozen; changes are additive increments + keeping legacy call shapes byte-identical unless a client change lands + with them. Signature changes DROP the old signature rather than + overloading (PostgREST cannot disambiguate named-arg subsets). + +### Vercel + +- Env vars (server-only): `SUPABASE_SERVICE_ROLE_KEY`, Stripe keys, and + `CRON_SECRET`. Vercel cron invocations automatically send + `Authorization: Bearer $CRON_SECRET`; the gc/backfill routes require + exactly that header, so manual runs pass the same bearer. Changing + `CRON_SECRET` requires a redeploy. +- `POST /api/replay/sign` — body `{grant}`. Redeems a + `cloud_authorize_replay_read` grant (single-use, 60s) via the + service-role RPC and returns `{urls: {storagePath: signedUrl}, +expiresIn: 120}` for **all** of the session's objects (complete set or + nothing). 401 with an opaque `ORG2_*` code on any redeem failure. CORS + `*` — the grant token is the entire authorization. +- `GET /api/replay/gc` — cron `0 4 * * *` (vercel.json). Reads the + service-role `replay_orphan_objects` view (objects **>24h** old with no + segment row — the 24h window means a first sweep reporting `scanned: 0` + right after orphan creation is correct, not a bug) and deletes via the + Storage API in pages of 100, max 50 pages/run. Returns + `{scanned, deleted, errors}`. +- `POST /api/replay/backfill` — body `{limit?}` (default 200, max 1000). + Moves legacy inline frozen segments (storage_path null, seq ≥ 1) to the + bucket at the canonical name and sets `storage_path`; `payload_gz` and + `byte_size` are left untouched (reversible; base64-unit accounting + preserved — see section 6). Re-run until `remaining: 0`. A **later + separate pass nulls `payload_gz`** once the backfill has soaked; run it + only after signed reads have been exercised against backfilled rows. + +### Probe recipes (service key) + +- **Function existence**: call the RPC with a role that lacks EXECUTE — a + **403/42501 proves the function exists** (permission denied), while + `PGRST202` means it does not. This is the standard post-paste signature + probe. +- **Real-user token minting** (no OAuth round-trip): auth admin + `generate_link(type: magiclink)` with the service key → + `POST /auth/v1/verify` with the hashed token → session + access/refresh tokens. Used both for RPC verification as a real member + and for recovering an expired desktop instance login + (`open "orgii-instance2://auth/callback#access_token=…"`). +- `cloud_ops_stats()` (0006, service-role only): per-org top-50 activity/ + size counts + platform totals — the input to the scale-out signals in + section 7. + +--- + +## 6. Known gaps and accepted tradeoffs + +| Item | Status | +| -------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Pre-0005 desktop clients lose coarse change signals once 0006 PART 6 drops `org_change_signals` from the publication | **Accepted.** The fleet is upgraded; Slice A eviction (org_memberships) still works for any straggler, but data-plane nudges require a 0005+ client | +| `byte_size` unit split | Legacy/backfilled rows account in **base64 units**, new storage-form rows in **raw bytes** — quotas effectively loosen ~25% on new data. Documented, deliberately not compensated (backfill keeps old units precisely to avoid usage-counter drift) | +| Guest replay availability | Storage-form guest imports depend on the **Vercel signer route** being up (members are unaffected — direct storage RLS reads). Signer failures surface as typed errors with a single retry, not silent fallback | +| Kind shadowing history | 0005 had one debounce row per org, so e.g. `cloud_set_member_sharing_floor` broadcast as `roster` (the roster trigger claimed the window); 0006's deferral alone would have inverted it to `policy`; 0006 PART 8's per-(org, kind) rows eliminate shadowing (that RPC now emits **both** `policy` and `roster`). The client handles both eras: policy and roster kinds each bump roster version **and** coarse refresh | +| Comments delta un-resolve blind spot | **CLOSED** by `state_changed_at` (0006): un-resolve clears `resolved_at` but now moves the single delta stamp; the client's force path stays full-fetch to cover pre-0006 backends | +| Visible-but-unfocused window beyond the 45s grace | Lease still releases; UI can stale until refocus (then heals via cheap delta). Mitigation judged sufficient; hold-while-visible remains an option if users report it | +| `cloud_delete_account` heavy-user erasure | 0003 shipped the user-id indexes, but execution still runs as one transaction through the caller's PostgREST connection under the 8s timeout; the service-role background batch job remains future work | +| Tail re-upload growth | A long streaming turn re-uploads the whole inline tail per 3s-debounced append — O(turn²) bytes per turn (gzipped). Monitor; cap candidates documented in the client audit's D notes | +| Comment tombstones consume the 500/thread cap permanently | Spam-then-delete can cap a thread; unaddressed (LOW) | +| Append-path tail delete+reinsert bloat (L1) | Unaddressed; monitor autovacuum on `cloud_session_segments` | +| Client localStorage push-cursor/metadata maps lack session-level GC (client C2) | Dead-org pruning exists (roster reconcile + zombie-org sweep); per-session sweep against the local registry remains future work | +| Socket rebuilt on org switch | Minor redundant `list_my_orgs`; reusing the socket and swapping channels would remove it (LOW) | + +--- + +## 7. Scale-out roadmap (beyond one Supabase project) + +With 0003–0006 applied, every audit-identified bottleneck inside the +single project is closed. Levers for the next magnitude, in order — +pre-decided so the call is not made under fire. Watch `cloud_ops_stats()` +plus the Supabase dashboard. + +1. **Read replicas** (first, cheapest). Route the read-heavy RPC allowlist + (listings, event pages, roster) to a replica. Client prerequisite + already exists: every call goes through `getCloudEndpoint()`; add a + `readReplicaUrl` to `CloudEndpoint`. Writes, realtime, storage stay + primary. +2. **Storage/CDN egress** — already offloaded (0005): replay bytes go + through the Storage CDN, not Postgres. Nothing further until + multi-region. +3. **Org sharding** (the real split). The schema is shard-friendly: every + row is org-scoped; cross-org state is only `orgs` / `org_memberships` / + `cloud_profiles`. A split is a directory service mapping org → project + (endpoint + anon key) fetched at roster load; the client already keys + caches and capability memory per endpoint URL. Realtime channels and + storage buckets shard with their org's project; grants/billing stay on + a control-plane project. +4. **Multi-region** follows sharding (orgs assigned to regional projects); + no code-shape changes beyond the directory. + +Action signals: realtime messages/month at 60% of plan → widen per-kind +debounce or narrow refetch further (replicas do not help — messages bill +per delivery); primary CPU sustained >50% or connection peak >300 → read +replicas; any single org >~20% of total write volume → first sharding +candidate. + +Explicitly rejected: cross-tenant content-addressed dedup of replay +objects (timing side channel — see SharedSessionPerformance.md); per-user +connection-pooling changes (RPC-only PostgREST keeps connections flat; the +lease bounds realtime sockets). + +**User-approved next batch**: `homeEndpoint` directory hook in the client +(step 3's client half); a painless new-project migration runbook; a BCDR +drill (backup/restore of the managed project); the `payload_gz`-nulling +pass once backfill has soaked (section 5). + +--- + +## 8. Audit findings index + +This index replaces the full 2026-07-23 audit reports. Server = schema +audit of 0001; Client = desktop usage audit; LR = adversarial +foreground-lease review. "Resolved" = shipped and verified. + +### Server schema audit + +| ID | Finding | Resolution | Status | +| --- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------- | +| C1 | `org_change_signals` = one hot row per org: FOR EACH ROW triggers + append-path double-write serialized all org writes and double-fired realtime per push | 0003 (250ms debounce + double-bump elimination) → 0006 PARTs 7–8 (commit-deferred triggers, per-(org,kind) rows, advisory total order) | Resolved (1.54s → 0.015s) | +| C2 | `cloud_list_org_sessions` unbounded + per-row comment-count LATERAL; cold starts/reconnects replay the full listing | 0003 materialized counts + 0004 PART 2 keyset paging + PR #509 | Resolved | +| C3 | `enforce_stored_bytes_quota` O(N) org-wide scan per segment push, under the signal-row lock | 0003 `orgs.stored_bytes_total` delta counter + `reconcile_org_stored_bytes` | Resolved (zero-drift verified) | +| H1 | `usage_monthly` ↔ signal-row lock-order inversion → 40P01 between create and push | 0003 reorder (quota before the sessions update) | Resolved (A/B deadlock proof) | +| H2 | `cloud_get_session_events` returns the whole replay as one jsonb; no size cap | 0002 `cloud_get_session_events_page` (seq keyset) + client loop | Resolved | +| H3 | Collab-state full pull returns every version ever, tombstones included, forever | 0004 PART 3 unified-cursor paging + PART 4 `gc_collab_tombstones` (90d) + PR #509 | Resolved | +| H4 | postgres_changes = per-change × per-subscriber RLS on a shared single-threaded WAL poller; message quota is the first platform wall | 0005 Broadcast-from-Database + 0006 PART 6 publication drop + PRs #511/#532; Slice A stays postgres_changes by design | Resolved | +| H5 | Replay bytes as base64 TEXT in primary Postgres with 100GB/org entitlements | 0005 Storage offload + 0006 guest grants + Vercel sign/gc/backfill + PRs #511/#532 | Resolved (backfill ongoing; `payload_gz` nulling pending) | +| M1 | Every work-item upsert took FOR UPDATE on the owning project row | 0004 PART 5 allocator-only lock | Resolved (pg_locks: no project-row waits) | +| M2 | `cloud_delete_account` single-txn via caller connection; missing user-id indexes | 0003 indexes shipped; background-job execution not done | Partially resolved (section 6) | +| M3 | No index for the free-plan retention-window listing predicate | 0003 `(org_id, last_activity_at desc) where deleted_at is null` | Resolved | +| M4 | Unpaginated comment listing + refetch-on-nudge herd | 0004 PART 6 `p_since` + 0006 `state_changed_at` + PR #532 | Resolved | +| L1 | Append path deletes+reinserts the tail row per push; bloat amplifier | Not addressed; monitor autovacuum | Open (accepted) | +| L2 | Forever-growing bookkeeping (`stripe_webhook_events`, repo-scope events; comment tombstones eat the thread cap) | 0004 PART 7 `gc_bookkeeping` + 0006 PART 5 grant purge; tombstone-cap consumption stands | Mostly resolved | +| L3 | Quota COUNT(\*) gates | Verified bounded/fine | No action | + +### Desktop client usage audit + +| ID | Finding | Resolution | Status | +| --- | ---------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | ------------------------ | +| A1 | Every blur→focus flip = full recovery batch (~6+N RPCs, two full listings) + socket churn; no disconnect-duration gate | PR #507: 45s release grace, disconnect-gated delta recovery, recovery routed through version bookkeeping | Resolved | +| A2 | Metadata hash seed stored stripped-payload hash but compared full-payload hash — never matched; every restart re-upserted every pushed session | PR #507: seed stores the full-payload hash (+`avatarUrl`) | Resolved (0/8 re-pushed) | +| A3 | Visible-but-unfocused window holds no socket and never recovers; UI stales silently | PR #507 grace covers short blurs; refocus heals via delta | Mitigated (section 6) | +| B1 | `get_entitlement_state` × N fan-out per roster read; coordinator deferred instead of dropping | 0004 PART 1 + PR #509 batched hydration | Resolved (7→1) | +| B2 | Zero jitter anywhere: reconnect, online herd, transport retry, flat 10s comments loop | PR #507: 0–3s reconnect jitter, exponential comments backoff (10s→5min cap, failures preserved across evict) | Resolved | +| B3 | App start runs the expensive reads twice (`list_my_orgs` ×2, sessions listing ×2, repo scopes ×2) | PR #532: 2s listing coalesce window (full never satisfied by delta); PR #507 killed the A2 upsert storm half | Resolved | +| B4 | `forceAllInbound` = full collab listings for ALL orgs on start/online/roster-change | PR #507: full listings converged to the active org; others delta/defer-to-activation | Resolved | +| C1c | Missed-event heal map: gaps for visible-unfocused, non-active-org comments, inactive orgs | Partially closed by #507 (comments force-bump on full recovery); inactive-org staleness is the stated contract | Accepted | +| C2c | Persisted localStorage push-cursor/metadata maps grow without session-level GC | Dead-org pruning + zombie-org sweep exist; per-session sweep future work | Open (accepted) | +| C3c | In-memory bounds (remote sessions 64, comments 128, roster 64, tokens 500) | Verified healthy | No action | +| D | Payload notes: O(turn²) tail re-upload; 10-min events re-hash; comments always full | Comments delta shipped (#532); tail growth + re-hash monitored | Partially resolved | +| E | Subscription scope O(1)/client verified; socket rebuilt on org switch | Channels 4→2 post-0005; socket-rebuild wart remains | Open (LOW) | + +### Foreground-lease adversarial review (ship-blockers pre-#507) + +| ID | Finding | Resolution | Status | +| --- | --------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | --------------------------------------------------- | +| LR1 | No hysteresis: every blur pays full teardown/recovery (ship-blocker) | PR #507 45s grace | Resolved | +| LR2 | "Hidden window keeps pushing" was false in code — unbounded outbound staleness (ship-blocker) | PR #507 hidden-outbound on turn-terminal + outbox events | Resolved (20s-into-hidden push proof) | +| LR3 | Presence viewer-chips vanish on every blur (regression the old code warned about) | Grace period covers it | Resolved (dual-window visual check still pending) | +| LR4 | Comments missed during short blur swallowed by the 30s TTL | PR #507 force-bump on full recovery | Resolved | +| LR5 | Reconnect thundering herd, no jitter/cooldown | PR #507 jitter + backoff | Resolved (network-fault injection only unit-tested) | +| LR6 | Non-active orgs: staleness unbounded after the recurring pass was removed | Contract stated (stale-until-activation); tombstone-on-refocus verified | Accepted | +| LR7 | Double full sessions listing per refocus | PR #532 coalesce window | Resolved | +| LR8 | Zombie connections | None found; disposal correct on all paths | No action | +| LR9 | Billing model verification | Peak-concurrent billing confirmed; grace keeps the win while deleting flap load | Verified | diff --git a/docs/cloud-broadcast-and-storage-design-2026-07-24/broadcast-change-signals.md b/docs/cloud-broadcast-and-storage-design-2026-07-24/broadcast-change-signals.md deleted file mode 100644 index 496185cd6..000000000 --- a/docs/cloud-broadcast-and-storage-design-2026-07-24/broadcast-change-signals.md +++ /dev/null @@ -1,90 +0,0 @@ -# Change Signals via Broadcast-from-Database (audit item H4) - -Date: 2026-07-24. Basis: realtime signal-chain architecture map (agent survey -of 0001/0003 server path + full client subscription topology). Companion: -`replay-storage-offload.md`. - -## Why - -`postgres_changes` evaluates RLS per change × per subscriber on a single -shared WAL poller — the platform's first scalability wall (server-schema-audit -H4). The client consumes **zero row data** from any of its three -postgres_changes subscriptions (all onChange callbacks are 0-arg edges), so -the entire contract is "something in scope changed" — a perfect fit for -fire-and-forget broadcast with join-time authorization. - -## Decision summary - -| Question | Decision | Why | -| ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------- | -------- | --------- | ------ | -------------------------------------------------------------------------------------------- | -| Topic | **Reuse the existing private org channel `presence:org:`** | Already open on every active client, already authorized by the `realtime.messages` policies (0001:5694-5706), already carries client-side broadcast events (`org-control-changed` etc.). Server-side `realtime.send` from a definer trigger needs no new policy. Channels per client drop 4 → 2. | -| New event | `org-db-changed` with payload `{kind}` | Distinct from client-sent `org-control-changed`. `kind` ∈ `sessions | comments | projects | workItems | roster | policy`derived from`TG_TABLE_NAME` / call site — the discriminator today's signal row lacks. | -| Debounce | Broadcast fires **only when the debounced signal-row bump actually fired** (the 0003 250ms `WHERE` clause) — per (org, kind)? No: per org, same as today | Identical delivery semantics to the postgres_changes transport; the client's 60s coarse throttle and 5min control TTL stay unchanged. | -| Slice A (self-roster eviction) | **Stays postgres_changes** on `org_memberships` `user_id=eq.` | A member removed while disconnected can never be reached by join-time-auth broadcast (`is_org_member()` already false). Exactly the audit's recommendation. | -| Slice B (org signal) + B-roster (org memberships) | Replaced by broadcast events on the org channel when the backend supports it | The two org-scoped postgres_changes channels disappear; SUBSCRIBED-edge recovery bookkeeping moves to the org channel's join edge. | -| Old signal row / publication | **Kept and still bumped** (org_change_signals stays in the publication) | Old clients keep working; removal is a later cleanup migration once the fleet has upgraded. | -| Capability detection | New RPC `get_cloud_capabilities()` → `{broadcastSignals: true}`; client probes once per endpoint, PGRST202 ⇒ legacy | Same probe-and-remember pattern as the 0004 paged listings. | -| Roster nudges for members | New AFTER trigger on `org_memberships` → send `kind:'roster'` to the org topic | Replaces the org-filtered memberships channel for present members; the removed user is covered by Slice A. | -| Policy nudges | The 3 RPCs with direct in-RPC bumps (`cloud_rename_org`, `cloud_set_org_sharing_floor`, `cloud_set_member_sharing_floor`) recreated with `kind:'policy'` send | They bypass the trigger today; same bypass, now with a broadcast. | - -## Server (cloud-infra 0005) - -1. Helper `org2_cloud.nudge_org_signal(p_org_id uuid, p_kind text)`: - debounced signal-row upsert (unchanged 250ms suppress) and, **iff the - bump fired**, `realtime.send(jsonb_build_object('kind', p_kind), -'org-db-changed', 'presence:org:' || p_org_id, true)`. -2. `touch_org_change_signal()` trigger fn delegates to the helper with kind - mapped from `TG_TABLE_NAME` (`cloud_projects`→projects, - `cloud_work_items`→workItems, `cloud_sessions`→sessions, - `cloud_session_comments`→comments). -3. New `AFTER INSERT OR UPDATE OR DELETE` trigger on `org_memberships` → - `nudge_org_signal(org_id, 'roster')`. -4. Recreate the 3 policy RPCs replacing their inline bump with the helper - (`kind:'policy'`). -5. `get_cloud_capabilities()` → `jsonb_build_object('broadcastSignals', -true)`, grant `authenticated`. -6. Lock-order invariant preserved: the helper is still the statement-final - signal-row acquisition (canonical slot 6); `realtime.send` inserts into - `realtime.messages` (a partitioned insert, no user-table locks) after it. -7. Verify tail: single-signature checks; trigger presence; capabilities - callable. - -Offline validation: brew-PG harness stubs `realtime.send` (records rows into -a scratch table) — assert one broadcast per debounce window per kind source, -none when suppressed, roster/policy events fire, legacy signal row byte-same. - -## Client (ORGII) - -1. `org2CloudClient` (or syncClient): `getCloudCapabilities(accessToken)` - with PGRST202→null; per-endpoint memory (`capabilitiesByEndpoint`). -2. `org2CloudRealtimeClient.joinPresence`: surface server broadcasts — - `org-db-changed` events dispatch to a new `onOrgDbChanged(kind)` option - next to the existing broadcast handlers; expose the channel's SUBSCRIBED - edge (already surfaced for presence re-track) to the caller for recovery - bookkeeping. -3. `useOrg2CloudRealtime`: when capabilities report `broadcastSignals`, - skip creating Slice B and B-roster postgres_changes channels; wire - `onOrgDbChanged`: `roster` → `bumpRosterVersion(orgId)` (+ the - rosterRealtimeConnected atom keyed to the org channel's join edge); - every other kind → `scheduleCoarseSignalRefresh()` (behavior-identical - coarse path; per-kind narrowing is a later optimization, the payload - already carries it). SUBSCRIBED-edge full/delta recovery - (`decideSubscribedEdgeRecovery`) moves onto the org channel's join edge. - Slice A unchanged. -4. Legacy backends (probe fails): topology unchanged (3 pg channels + 1 - presence). Tests: transport tests reworked in - `org2CloudRealtimeClient.test.ts`; recovery/lease/scope tests untouched. - -## Quota effect - -Broadcast messages still bill per delivery, but: no per-change × per-client -RLS evaluation, no WAL-poller serialization, and the `kind` discriminator -unlocks future per-plane refetch narrowing (today every signal shotguns all -planes after the 60s throttle). The 250ms server debounce and 60s client -throttle keep message volume identical to today's. - -## Rollout order - -0005 applies live (additive, old clients unaffected) → client PR merges → -fleet upgrades → later cleanup migration drops org_change_signals from the -publication (and eventually the table, once fleet-wide). diff --git a/docs/cloud-broadcast-and-storage-design-2026-07-24/replay-storage-offload.md b/docs/cloud-broadcast-and-storage-design-2026-07-24/replay-storage-offload.md deleted file mode 100644 index af5150c58..000000000 --- a/docs/cloud-broadcast-and-storage-design-2026-07-24/replay-storage-offload.md +++ /dev/null @@ -1,72 +0,0 @@ -# Replay Segment Payload Offload to Supabase Storage (audit item H5) - -Date: 2026-07-24. Basis: segment-lifecycle architecture map (agent survey of -0001/0002/0003 + client push/pull paths). Companion: `broadcast-change-signals.md`. - -## Decision summary - -| Question | Decision | Why | -| ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| What moves to Storage | **Frozen segments only** (`seq >= 1`) | Immutable + append-only + the bulk of bytes. No overwrite churn, no orphan risk on the hot path. | -| What stays in Postgres | **Tail (`seq = 0`) stays `payload_gz` inline** | Mutable, rewritten every push, small (recent non-terminal events). Keeps append OCC single-transaction. | -| Object encoding | raw gzip bytes (`application/gzip`) | Drops the base64 +33% leg entirely; codec already isolates the base64 step (`segmentCodec.ts`). | -| Object key | `replay/{org_id}/{session_id}/{epoch}/{seq}-{segment_hash}.gz` | Immutable-by-name; hash in key makes retry idempotent and collision-safe inside the org/session auth domain (no cross-tenant dedup by design — see SharedSessionPerformance.md side-channel note). | -| Member reads/writes | **Direct Storage REST with user JWT + storage RLS** | The whole member auth ladder (membership → retention → metadata_only → restricted-visibility grant) is already SQL (`assert_session_readable`); a `security definer` helper called from the storage policy reuses it verbatim. | -| Share-token guest reads | **Vercel signer route** (`apps/org2-cloud-web`, has `SUPABASE_SERVICE_ROLE_KEY`) | Guest tier has no `auth` role, inexpressible in storage RLS. RPC mints a short-lived grant; the route redeems it and returns signed URLs. Rare path. | -| Atomicity | **Upload blobs first, then commit manifest RPC** | RPC verifies each object exists in `storage.objects` and reads its真实 size from object metadata — server-measured bytes, no client attestation. Missing object ⇒ `ORG2_VALIDATION`, nothing committed. | -| Quota unit | `byte_size` = raw object bytes (frozen) / `octet_length(payload_gz)` (tail, unchanged) | −25% vs base64 accounting: quotas get slightly looser, matches what is actually stored. Documented, not compensated. | -| Orphan GC | SQL view `replay_orphan_objects` (objects with no matching segment row, older than 24h) + service-role sweep via Storage API (Vercel cron) | Postgres FK cascade cannot delete bucket objects; rewrite/delete-account leave objects behind by design and the sweep reaps them. | -| Rollout | Additive: `cloud_session_segments.storage_path text null`; null = legacy inline. Read RPCs return `storagePath` XOR `payloadGz` per segment. | Old clients fail closed on new-format sessions (zod `payloadGz` missing ⇒ import error, no corruption). Pre-release population is auto-updating; accepted. | - -## Server (cloud-infra migration, offline-validatable parts) - -1. `cloud_session_segments.storage_path text` + relax `payload_gz` to nullable - with `check ((payload_gz is not null) or (storage_path is not null))` and - `check (seq = 0 → payload_gz is not null)` (tail always inline). -2. Bucket `replay` (private) + storage RLS policies delegating to - `org2_cloud.can_read_replay_object(name)` / `can_write_replay_object(name)` - security-definer helpers (path parse → session ladder / owner+entitlement - gates + epoch/seq sanity). -3. `cloud_append_session_events` / `cloud_rewrite_session_events`: frozen - segment wire accepts `{seq, storagePath, eventCount, segmentHash}` as an - alternative to `payloadGz`; on the storage form the RPC looks the object up - in `storage.objects` (existence + `(metadata->>'size')::bigint` + key must - embed the claimed `segment_hash`), charges quota with the object size, and - stores `storage_path`. Legacy inline form byte-identical behavior. -4. `cloud_get_session_events(_page)`: rows return `storagePath` when set, - `payloadGz` otherwise. Guest grant RPC `cloud_authorize_replay_read` + - redeem RPC (service-role) for the signer route. -5. `reconcile_org_stored_bytes` unchanged (byte_size stays truthful). -6. `replay_orphan_objects` view + `gc_replay_orphans` bookkeeping (row side); - object deletion happens in the cron sweeper, not SQL. - -## Client (ORGII) - -- Push seam: inside `org2CloudSyncClient.appendSessionEvents`/`rewrite…` — - before the RPC, upload each frozen segment's raw gzip bytes via - `PUT /storage/v1/object/replay/{key}` (JWT), bounded concurrency (reuse - `mapSegmentsBounded` cap 4); segment wire swaps `payloadGz` → - `storagePath`. Tail keeps the base64 inline path. -- Pull seam: `decodeCloudSegments` (`org2CloudBackendAdapter.ts`) — when a - segment carries `storagePath`, `GET /storage/v1/object/replay/{key}` (JWT) - → gunzip raw bytes; integrity check unchanged (sha256 of pre-gzip bytes vs - `segmentHash`). Share-token imports call the signer route first. -- Capability fallback mirror of the 0004 pattern: on storage-form rejection - (pre-migration backend), remember per endpoint and fall back to inline - wire. Full backward compatibility both directions. - -## Explicitly out of scope - -- Migrating EXISTING inline segments (backfill mover is a later service-role - batch job; inline reads stay supported indefinitely). -- TUS/resumable uploads (objects are ≤ ~256 KiB pre-gzip; far below 6 MB). -- Cross-org content dedup (side channel, rejected in SharedSessionPerformance). - -## Validation plan - -- Offline (brew PG + storage schema stub): migration idempotency, wire A/B - (inline form byte-identical vs 0004 DB), manifest commit vs missing object, - quota deltas with mixed inline/storage segments, orphan view correctness. -- Live: real Storage bucket + policies (user pastes migration, bucket created - via dashboard or SQL insert into storage.buckets), dual-instance push/pull, - cmd+5 + CPU/RAM per standing rule. diff --git a/docs/cloud-scalability-audit-2026-07-23/README.md b/docs/cloud-scalability-audit-2026-07-23/README.md deleted file mode 100644 index debab2c2c..000000000 --- a/docs/cloud-scalability-audit-2026-07-23/README.md +++ /dev/null @@ -1,39 +0,0 @@ -# ORG2 Cloud 可扩展性审计 — 2026-07-23 - -Branch: `codex/reclaim-cloud-realtime-connections` (rebased onto develop `a14bb4def`, epic commit `fd41d508c`). -三个独立审计（服务端 schema / 客户端用量 / realtime 租约对抗审查）的完整报告见同目录三个文件；本文件是合并后的优先级视图。 - -## 高杠杆修复排序（合并三方结论） - -> **进度 2026-07-23 晚**：客户端 P0 1–5 已全部落地在本分支（`d344f3b06` 租约宽限期+hash 种子、`d5850cd40` 断连门控+评论 force+抖动+hidden outbound、`ea3344388` 全量 listing 收敛到活跃 org、`bd0b444f4` 下拉滚动条样式冲突）。服务端 P0 的 0001 就地编辑 + 线上 delta 正在起草，验证基建（disposable Postgres + A/B 协议冒烟）已就绪。 -> 另：早前的「侧栏 roster 3 vs 7」已定性为纯 UI 裁剪问题（数据管线健康），见 branch 记忆。 -> -> **进度 2026-07-24**：服务端 P0 1–4 已通过 cloud-infra `0003_scalability_p0.sql` 上线并零漂移核验；P0 5 的事件分页由用户的 `0002_bounded_session_event_pages.sql` 覆盖。后续批 `0004_roster_and_listing_scalability.sql`（cloud-infra `a45adcd`，待线上粘贴）一次性收编：B1 批量 entitlement、C2 sessions keyset 分页、H3 collab-state 统一 cursor 分页 + tombstone GC、M1 工作项稳态编辑不再锁 project 行、M4 评论 `p_since` delta、L2 bookkeeping GC。客户端配套（entitlement 种子 + sessions/collab 分页走查，全部向后兼容 PGRST202 回落）在 PR #509。仍未动的平台级方向：Broadcast-from-Database（H4）、Storage payload 外移（H5）——各需独立迁移与客户端订阅重写；`org_change_signals` 去抖后仍是 org 内写串行点（C1 残余）。 - -### P0 — 客户端（本分支范围内可修） - -1. **租约释放加宽限期（30–60s）**：blur 立即释放改为延迟释放（hidden/pagehide 仍立即释放），refocus 取消定时器。消灭 alt-tab 级别的整套 socket 拆建 + 恢复风暴（~9 RPC/次，含 2–3 个全量 listing），同时保留 peak-connection 计费收益，并顺带修复 presence 「viewer chip 每次失焦消失」的回归（旧代码注释里明确记录过这个教训）。 -2. **元数据 hash 种子永不匹配 bug**：`seedFromRemoteSummary` 存的是 stripped-payload hash，`upsertMetadataIfChanged` 比的是 full-payload hash（含 `id`/`ownerMemberId`）→ 每次重启对每个已推送 session 重发一次字节相同的 `cloud_upsert_session_metadata` + 广播。一行级修复：种子时额外存 full-payload hash。 -3. **断连时长门控恢复**：SUBSCRIBED 边沿记录断连时长，< 阈值（如 5min）走 delta（serverCursor 都在），只有长断连才走全量 listing + 免 TTL 的 comments 强制刷新（30s TTL 会吞掉短暂失焦期间错过的评论——已确认路径）。 -4. **重试加 jitter**：supabase-js `reconnectAfterMs` 无抖动 + `fetchWithTransportRetry` 零延迟立即重发 + comments 10s 平坦无上限错误循环 + `online` 全 org 全量 → 灾后惊群。全部加随机抖动/指数退避。 -5. **隐藏窗口停推**：设计声称 hidden 继续推，代码里 `scheduleActivityPass`/projects outbox 都被 hidden 门拦住 → agent 跑一小时任务队友看不到任何进度。修法：turn-terminal 和 outbox-write 这两个具体事件即使 hidden 也触发一次 pass（仍然是事件驱动，不是轮询）。 - -### P0 — 服务端（cloud-infra 仓库） - -1. **`org_change_signals` 单行热点**：per-org 单行 + FOR EACH ROW 触发器 + append 路径双写 `cloud_sessions` → org 内所有写互相串行、每次 push 两条 realtime 广播。改 statement-level 触发器 + 250ms 去抖 + 消掉 quota 函数里的第二次 update。 -2. **`cloud_list_org_sessions` 无界 + per-row LATERAL 评论计数**：付费 org 无保留窗，全量拉取扫全史 + 每行评论 COUNT。加 keyset 分页（`(org_id, updated_at)` 索引已在）+ 把评论计数物化成 `cloud_sessions` 列。 -3. **`enforce_stored_bytes_quota` O(N) 扫描**：每次 segment push 求和整个 org 的全部 session 行。改增量维护的字节计数器 + 夜间对账。 -4. **`usage_monthly` ↔ signal 行锁序倒置**：新建 session（A→B）与 push（B→A）并发即 40P01 死锁窗口。一行重排修复。 -5. **`cloud_get_session_events` 单值返回整个 replay** + segment 存 Postgres base64 text（+33%）且 team 计划 100GB 配额 → 按 seq 分页 + 中期把 payload 移到 Storage。 - -### 平台配额（最先撞到的墙） - -- Realtime `postgres_changes` 对每 change × 每 subscriber 做 RLS 评估，且全部 org 共享一个 signals 表的单线程 WAL poller；20 人 org × 2k 写/天 ≈ 2.4M msg/月，先于任何 Postgres 瓶颈撞上 message 配额。**方向：迁移到 Broadcast from Database（join 时鉴权一次）**。 -- 计费维度已核实：连接按 billing cycle 的峰值并发（Pro 含 500，$10/1000 超额），消息 $2.50/M 超 5M。租约在连接维度的收益是真实的；宽限期方案两头都保。 - -## 双机冒烟实测发现（2026-07-23） - -- 双实例构建走 `pnpm run tauri:build:fast:dual`（同 repo 双身份、独立 target，产物都在 `src-tauri/target/dev-build/bundle/macos/`）。 -- 租约 acquire 侧验证通过：instance 2 聚焦 + 云 org scope → `realtime: subscribed inbound planes` 即时出现。 -- **未决 bug（待 A/B develop 构建定位）**：instance 1 boot 后侧栏 org selector 只显示 3 个 0720 时代的云 org（服务端 `list_my_orgs` 实际返回 7 个）；管理 ORG 页自己的 picker 拉到全部 7 个，但侧栏 selector 在其 refetch 后依旧 3 个 → roster 陈旧且写不进侧栏读取的状态（疑似双 jotai store 或 boot 竞态）。instance 2 重新登录后 roster 正常。 -- instance 2 的 refresh token 过期（400）后，用 `tests/e2e/.env` 的 service key 走 admin magiclink → `orgii-instance2://auth/callback#...` deep link 注入恢复登录，无需 GitHub OAuth。 diff --git a/docs/cloud-scalability-audit-2026-07-23/client-usage-audit.md b/docs/cloud-scalability-audit-2026-07-23/client-usage-audit.md deleted file mode 100644 index dc0b21860..000000000 --- a/docs/cloud-scalability-audit-2026-07-23/client-usage-audit.md +++ /dev/null @@ -1,84 +0,0 @@ -# Desktop Client Cloud-Usage Audit - -Auditor: fable subagent, 2026-07-23. Branch `codex/reclaim-cloud-realtime-connections`, scope `src/features/Org2Cloud/**` + sidebar/panel consumers. - -## A. Critical — focus-flip and restart hot paths - -### A1. Every blur→focus flip runs a full recovery batch with no short-circuit — ~6+N RPCs, two FULL listings, plus socket churn - -- Lease releases on ANY blur (`org2CloudRealtimeLease.ts:81-85`; `windowFocus.ts:7-11` requires `document.hasFocus()`). Release nulls `activeRealtimeOrgId` (`useOrg2CloudRealtime.ts:154-155`) → connection effect dependency → socket + channels torn down on blur, rebuilt on focus. -- On focus, SUBSCRIBED edges fire: Slice A `refetchOrgs()` = `list_my_orgs` + `get_entitlement_state × N` fan-out (`org2CloudOrgsAtom.ts:348-356`); Slice B `invalidateOrgInbound(org, {full:true, pushSessions:true})` → repo-scope refetch (TTL cleared) + FULL `cloud_list_org_collab_state` (cursor bypassed) + full outbound session scan + `refreshEntitlementForOrg` + FULL `cloud_list_org_sessions`; roster bump defeats the members cache when the member filter is mounted. -- Independently the remote-sessions atom's own focus listener fires a SECOND full `cloud_list_org_sessions` (`org2CloudRemoteSessionsAtom.ts:497-514`) that never records into version bookkeeping — the Slice B bump fetches again after it. -- No minimum-disconnect-duration gate anywhere: a 2-second alt-tab pays the identical price as overnight sleep, though delta paths exist (serverCursor + `mergeRemoteSessionDelta` with tombstones; collab-state cursor). -- Fleet impact: ~9 RPCs (2 full listings) + 1 socket + 4 channel joins per flip; 50 flips/day × 10k users ≈ **4.5M RPCs/day + 500k socket connects/day** from alt-tabbing alone. - -**Fix:** (1) 30–60s grace on lease release (immediate on hidden/pagehide); (2) disconnect-duration-gated full-vs-delta recovery; (3) skip Slice A refetch < X s after last roster read; (4) route focus-recovery through the same version bookkeeping as Slice B. - -### A2. Cold-start metadata hash seed can never match — every restart re-upserts metadata for every pushed session - -- `seedFromRemoteSummary` stores the hash of the STRIPPED payload (`org2CloudSessionSync.ts:99-105`), but `upsertMetadataIfChanged` compares the hash of the FULL payload (`:161-162`), which always contains `id` and `ownerMemberId` (`collabSyncUtils.ts:247-250`) that `metadataPayloadForHash` deletes (`org2CloudSessionSync.metadata.ts:37-39`). The hashes NEVER match → the seed never suppresses the first upsert. No test covers it. -- Impact: every restart re-sends byte-identical `cloud_upsert_session_metadata × S` + `broadcastOrgControlChangedToPeers` → peer re-list amplification. 10k users × 2 restarts × 20 sessions = ~400k pointless write RPCs/day. - -**Fix:** additionally store the full-payload hash in `seedFromRemoteSummary`. One-line-class, high leverage. - -### A3. Visible-but-unfocused window: no realtime, no recovery, indefinitely stale UI - -- Lease requires keyboard focus; a second-monitor window holds NO socket; the foreground-recovery path early-returns without focus (`org2CloudRemoteSessionsAtom.ts:504-510`); no polling fallback; the hook still reports `documentVisible: true` so the UI looks live. Multi-monitor "watch the team sidebar" silently stales for hours. - -**Fix:** hold the lease while `visibilityState === "visible"` (release only on hidden) — also eliminates most of A1's churn; or add a slow visible-unfocused refresh. - -## B. High — fan-out and retry behavior - -### B1. `get_entitlement_state × N` fan-out on every roster read; coordinator defers duplicates instead of dropping - -- Both roster paths hydrate all orgs (`org2CloudOrgsAtom.ts:282-287`, `:348-356`); gated calls arm a trailing timer that fires a REAL RPC at TTL expiry (`org2CloudEntitlementCoordinator.ts:82-95`) — delayed, not dropped (~1/10s/org sustained). **Fix:** return `orgSharingFloor` inside `list_my_orgs` (one RPC replaces N+1), or hydrate only the active org eagerly. - -### B2. Outage recovery has zero jitter anywhere — synchronized herd - -- `online`: `clearAllOrgBackoffs` + `forceAllInbound/ProjectsNextPass` + immediate pass (`org2CloudSyncLifecycle.ts:100-108`) → N full collab listings per client at the instant connectivity returns. -- supabase-js rejoins on fixed schedule; every SUBSCRIBED edge fires the A1 batch, no jitter (`useOrg2CloudRealtime.ts:292-297, 373-388`). -- `useOrg2CloudOrgs` fixed [2s,5s,10s,20s]; `fetchWithTransportRetry` re-sends immediately with no delay (`org2CloudFetchRetry.ts:23-28`) — 2× amplifier during hard outage. -- Comments error retry: flat 10s loop, no growth, no cap (`commentTransforms.ts:28,109-120` + re-arm at `org2CloudSessionCommentsAtom.ts:329-337`) — 6 list RPCs/min per open surface while degraded, indefinitely. - -**Fix:** 0–5s jitter before SUBSCRIBED-edge recovery and the online pass; exponential backoff + cap for comments; randomized delay in transport retry. - -### B3. App start does most expensive reads twice - -- `list_my_orgs` ×2 (initial + Slice A SUBSCRIBED), each with ×N entitlement fan-out; `cloud_list_org_sessions` ×2 for the active org (engine cold-start summary + sidebar atom initial fetch); `cloud_get_org_repo_scopes` ×2; collab-state FULL × N at start then again for active org. Realistic start (N=3, S=20): ~40 RPCs, half redundant, plus the A2 upsert storm. **Fix:** share the cold-start summary with the remote-sessions atom; suppress SUBSCRIBED-edge full recovery within X s of engine start. - -### B4. `forceAllInbound` recovery scales O(N orgs) with FULL listings - -- Start/online/roster-change set `forceAllInboundNextPass` → full `cloud_list_org_collab_state` for ALL orgs (`org2CloudSyncEngine.ts:578-580`), incl. never-opened ones. **Fix:** full for active org only; cursor delta or defer-to-activation for the rest. - -## C. Medium - -### C1. Missed-event heal map (verified) - -Heals: SUBSCRIBED edges (active org), focus recovery, `online`, roster reconcile, explicit refresh/org switch. -Gaps: visible-unfocused (A3); comment surfaces for sessions outside the active realtime org (no signal source; TTL only gates, never schedules); inactive orgs stale-by-design until selected (unbounded now — the removed recurring pass used to bound it); outbound sessions tagged to non-active orgs not pushed until activation (deliberate, no indicator). - -### C2. Persisted localStorage maps grow without session-level GC - -- Roster reconcile prunes by dead org only (`org2CloudRosterReconcile.ts:97-133`); `org2CloudPushCursorsAtom` / `org2CloudPushedMetadataAtom` keep entries for locally deleted sessions forever; `accessSettings` + `sharingFloor` deliberately excluded from pruning → zombie org entries accumulate across backend resets. **Fix:** endpoint-scoped storage keys + session-level sweep against the local registry. - -### C3. In-memory bounds healthy (verified OK) - -Remote sessions 64, comments 128, roster cache 64, force tokens 500; engine maps pruned per pass; identity flips clear everything. - -## D. Low — payload notes - -- Tail re-upload growth: frozen line = first non-terminal event; long streaming turn ⇒ each 3s-debounced append re-uploads the whole tail — O(turn²) bytes per turn (gzipped). Consider capping tail size or gating on tail-delta bytes. -- `EVENTS_CLEAN_TTL_MS` 10-min expiry re-reads + re-hashes all events per candidate session (local cost only). -- Comments listing always full + force-refetched per peer mutation — acceptable at current thread sizes. - -## E. Subscription scope — verified O(1) per client - -Single connection, 3 postgres-changes + 1 presence channel, active org only; `eventsPerSecond: 5`; presence budget 5/30s. Minor: whole socket rebuilt on org switch (drags a redundant `list_my_orgs` + entitlement fan-out); reusing the socket and swapping channels would remove it. - -## Top 5 first - -1. A2 hash-seed fix (tiny; kills the restart write storm). -2. A1 lease grace + duration-gated recovery (~4.5M RPCs/day at 10k users). -3. A3 hold lease while visible. -4. B2 jitter + comments backoff. -5. B1/B4 batched entitlements in `list_my_orgs` + active-org-only full listings. diff --git a/docs/cloud-scalability-audit-2026-07-23/realtime-lease-review.md b/docs/cloud-scalability-audit-2026-07-23/realtime-lease-review.md deleted file mode 100644 index 66eeb5ce2..000000000 --- a/docs/cloud-scalability-audit-2026-07-23/realtime-lease-review.md +++ /dev/null @@ -1,37 +0,0 @@ -# Adversarial Review: Foreground-Lease Realtime Reclaim (fd41d508c) - -Auditor: fable subagent, 2026-07-23. Traced against `@supabase/realtime-js@2.106.2` sources and the pre-commit code via `git show`. - -## Cost of one lease flip (CONFIRMED baseline) - -`isWindowFocused()` = `document.hasFocus() && !document.hidden`; the lease has zero hysteresis by design. Every blur tears down the socket + 4 channels; every refocus rebuilds and fires: 4 channel joins (each postgres_changes join inserts a `realtime.subscription` row with RLS evaluation; private presence join runs the `realtime.messages` RLS check), `list_my_orgs`, full inbound invalidation (repo scopes + FULL collab listing + push pass + entitlement + FULL session listing), members RPC if panel open, presence rejoin, plus the remote-sessions atom's own second full listing and guest-share `validate("all")`. Net ~5–8 RPCs incl. 2–3 full listings per flip, no debounce anywhere. - -## Findings (severity-ordered) - -1. **HIGH — No hysteresis: blur-triggered release makes routine desktop interactions pay the full teardown/recovery burst (CONFIRMED).** Cmd-tab, second-monitor clicks, Spotlight, native dialogs all flip `document.hasFocus()`. 100–300 flips/day ⇒ ~400–1200 channel joins + ~500–2400 RPCs/day per client. Fetches launched by an edge are not cancelled on the next release. Fix: 30–60s grace on release only (immediate on hidden→pagehide/sign-out), cancel on refocus. - -2. **HIGH — "A hidden window keeps pushing" is false in code (CONFIRMED).** The commit removed the recurring pass; what remains is visibility-gated: `scheduleActivityPass` early-returns when hidden (`org2CloudSyncLifecycle.ts:289`); projects outbox latches but skips dispatch when hidden (`:310-321`); `invalidateOrgInbound`/`reconcileRoster` skip too (`:240, 284`). Minimized for an hour ⇒ teammates see no replay progress and no work-item changes; agent comment replies still post over HTTP (`addressCommentsRun.ts:185-195`) so replies can reference transcript content teammates cannot see. Pre-commit hidden convergence was bounded at 5 min; now unbounded. Fix: trigger one pass on agent turn-terminal and outbox-write even when hidden (concrete events, no polling). Verified NOT broken: no runner waits on a realtime signal (turn lifecycle is a local atom; work-item locks are boundary RPCs) — nothing deadlocks while hidden. - -3. **MED-HIGH — Presence viewer-chips regression (CONFIRMED).** Pre-commit code carried the explicit lesson that clearing presence on blur made every viewer chip vanish; the lease reintroduces exactly that via Slice C cleanup → `handle.leave()` on every blur (`useOrg2CloudRealtime.ts:589-606`, `org2CloudRealtimeClient.ts:505-521`). The grace period fixes this too. - -4. **MED — Comments missed during a short blur can be swallowed by the 30s TTL (CONFIRMED path).** SUBSCRIBED-edge comment recovery is the org-wide bump, deliberately non-forced; `decideSessionCommentsFetch` skips within `SESSION_COMMENTS_TTL_MS = 30_000`. Blur at T+5s, teammate comments at T+10s, refocus at T+20s → bump inside TTL → skip; TTL expiry alone schedules nothing → comment invisible until an unrelated future event. Fix: force-token the recovery bump or make it TTL-exempt. - -5. **MED — Reconnect thundering herd: no jitter anywhere (mechanics CONFIRMED).** realtime-js reconnects at [1s,2s,5s,10s] then flat 10s, no jitter, and no `reconnectAfterMs` override is passed; every transient CHANNEL_ERROR/TIMED_OUT rejoin fires the full burst (no cooldown on the true-edge); `fetchWithTransportRetry` doubles failed POSTs immediately; laptop wake `onOnline` forces full listings across every org at once. Fix: jittered `reconnectAfterMs`, 0–3s random delay before edge recovery, per-org cooldown on repeated full recoveries (<30s). - -6. **MED — Non-active orgs: recovery never fires for them; staleness now unbounded (CONFIRMED gap).** Slice B subscribes only the active org; other orgs recover only on online/roster-change/restart/activation. The removed pass used to bound this at 60s/5min. Tombstone-free absences require the full listing, so a revoked project in org B is simply wrong locally all day. State the contract or add a cursor-preserving invalidation for other orgs on refocus. - -7. **LOW — Double full `cloud_list_org_sessions` per refocus (CONFIRMED).** Focus-listener fetch starts; SUBSCRIBED-edge full bump lands mid-flight, is not recorded (in-flight early-return precedes version recording), completion wakes the effect via `entrySnapshot` → second full listing. Fix: record a full bump as satisfied when a full fetch is already in flight, or drop the hook's own focus listener. - -8. **LOW — No true zombie connections (CONFIRMED).** `dispose()` reached on all paths; disposed-client guard + channel-name sequence suffix prevent topic reuse races. Residual warts: disposing a still-CONNECTING socket is now a steady-state noise source (was boot-only); the pagehide `releaseImmediately` path is effectively cosmetic (OS socket close does the real work). - -9. **Lease correctness & billing (CONFIRMED with caveats).** State machine sound; initialHeld render/effect reconciliation correct; single `main` window so multi-window is a non-issue. Billing verified: Realtime bills the highest concurrent-connection count per cycle ($10/1,000 over quota; Pro includes 500) plus messages ($2.50/M over 5M). Releasing while hidden lowers the billed peak by the fraction of running-but-unfocused clients at the fleet's busiest instant — large for a 24/7-resident desktop fleet, so the win is real. But flapping monetizes the messages meter (~200 flips × ~10 messages/day/client) and lands un-metered Postgres RPC + subscription-RLS load. At 1,000 clients × 200 flips: ~800k channel joins + ~1–2M recovery RPCs/day vs a held connection's ~4 joins/day. The grace period keeps the billing win and deletes most of the load. - -## Verdict - -**Sound in architecture, not yet sound to ship as-is.** Ship-blockers: release-on-blur with zero grace (finding 1, plus the presence regression 3), and the hidden-push freeze contradicting the design's own contract (2). - -Three highest-leverage improvements: - -1. Release-side grace period (30–60s) in the lease controller. -2. Event-driven hidden push (turn-terminal + outbox-write trigger one pass while hidden). -3. Harden the recovery edge: force-token comments recovery, dedupe the double listing, jittered reconnect + per-org full-recovery cooldown. diff --git a/docs/cloud-scalability-audit-2026-07-23/server-schema-audit.md b/docs/cloud-scalability-audit-2026-07-23/server-schema-audit.md deleted file mode 100644 index 400089919..000000000 --- a/docs/cloud-scalability-audit-2026-07-23/server-schema-audit.md +++ /dev/null @@ -1,97 +0,0 @@ -# Server-Side Scalability Audit (org2_cloud schema) - -Auditor: fable subagent, 2026-07-23. Source of truth: `ORGII-cloud-infra/supabase/migrations/0001_org2_cloud_schema.sql` (5708 lines, main). Note: the comment-task system (0002) was removed in cloud-infra commit `f69961e`; `supabase_realtime` publication contains only `org_change_signals` + `org_memberships`. - -## CRITICAL - -### C1. `org_change_signals` is a single hot row per org — every write in the org serializes on it, and it double-fires per segment push - -- One row per org, PK `org_id` (5599–5606). Trigger fires `FOR EACH ROW` on `cloud_projects` (5654), `cloud_work_items` (5658), `cloud_session_comments` (5665), `cloud_sessions` (5673) with `ON CONFLICT DO UPDATE set last_change_at = now(), seq = s.seq + 1` (5642–5647). -- `cloud_append_session_events` updates `cloud_sessions` at 689–694 **and again** inside `enforce_stored_bytes_quota` (3205–3207) → sessions trigger fires twice per push. Same for `cloud_rewrite_session_events` (2260 + quota update). -- The ON CONFLICT row lock is held until commit → org-wide write concurrency collapses to 1; in the append path the lock is taken _before_ the O(N) quota scans (C3), so every writer in the org waits behind them. `cloud_delete_project` tombstones N items → N consecutive updates of the same hot row in one transaction. -- Every bump fans out via postgres_changes to every subscribed member (2 events per segment push). - -**Fix:** statement-level triggers with transition tables (one bump per statement per org); debounce the hot row (`where s.last_change_at < now() - interval '250 milliseconds'`); eliminate the second `cloud_sessions` update by passing the byte delta into `enforce_stored_bytes_quota` or folding `stored_bytes` into the main summary UPDATE. - -### C2. `cloud_list_org_sessions` is unbounded, with a per-row comment-count LATERAL - -- Final definition 5487–5565: no LIMIT anywhere; paid plans skip retention windowing entirely (`v_retention_days is null` 5551; pro/team/enterprise seed `replayRetentionDays: null` at 5216/5238/5267). Data is never hard-deleted (5142–5149). -- Per returned row: comment-count LATERAL over `cloud_session_comments` (5523–5534) + share-probe EXISTS (5537–5546). -- Client forces a FULL pull on every Realtime (re)subscribe true-edge, so every reconnect of every member replays the full listing. -- A team org at seeded cap (5000 synced sessions/month) accretes ~60k rows/year: full pull = 60k heap rows + up to 3×10^7 comment-index entries + one giant `jsonb_agg` (100–200MB potential) → blows the 8s authenticated statement_timeout exactly when the org gets big. Delta path (`since` + `cloud_sessions_org_updated_idx (org_id, updated_at)`, 4064) is fine; cold starts and reconnects die. - -**Fix:** keyset pagination (`p_limit`, `(updated_at, session_id) >` cursor, `limit least(p_limit, 500)`) + maintain `comment_count`/`unresolved_count` as counter columns on `cloud_sessions` (comment RPCs already serialize per session via advisory lock 5419–5421). - -### C3. `enforce_stored_bytes_quota` recomputes org storage by scanning every session row of the org on every segment push - -- 3193–3223, called from every append (698–700) and rewrite (2269–2271): sums all segments of the session AND `sum(stored_bytes)` over ALL `cloud_sessions` of the org (tombstones included, never GC'd; comment 3209–3210). Neither column is in any index. 10^5 rows read per push, per actively syncing member, while holding the org signal-row lock (C1). - -**Fix:** delta-maintained org byte counter (append: `v_upload_bytes`; rewrite: new_total − old_total, old total already loaded FOR UPDATE) + nightly reconciliation. If recompute must stay: covering indexes `(org_id, session_id) include (byte_size)` and `(org_id) include (stored_bytes)`. - -## HIGH - -### H1. Lock-order inversion between `usage_monthly` and the signal row → deadlocks - -- `cloud_upsert_session_metadata` first-insert path: `usage_monthly` upsert (2966–2971, lock A) then `insert into cloud_sessions` (2977) → trigger → signal-row lock B. -- `cloud_append_session_events`: `update cloud_sessions` (689) → lock B first, then `bump_monthly_upload_quota` → lock A (300–305, called at 703). Same in rewrite (2260 → 2274). -- Two members of one org (one creating, one pushing) = classic 40P01; frequency grows quadratically with concurrent sync. `usage_monthly (org_id, period)` is itself a second org-wide hot row. - -**Fix:** unify order — bump upload quota _before_ the `cloud_sessions` update in append/rewrite. - -### H2. `cloud_get_session_events` returns the entire replay as one jsonb value; no per-session size cap - -- 1451–1473: `jsonb_agg` of every segment's `payload_gz` for the epoch; `p_after_seq` only helps re-pulls. Org-level quotas only (1GB free / 10GB pro / 100GB team) — a single session can hold the entire org quota and becomes permanently unreadable (memory blow-up, ~1GB varlena ceiling, 8s timeout). Rewrite has the same wall (2224–2258 deletes + reinserts all segments in one txn). - -**Fix:** page by seq via PK `(org_id, session_id, epoch, seq)` (3863): `p_max_segments`/byte budget + `hasMore`; client already loops on `afterSeq`. - -### H3. `cloud_list_org_collab_state` full pull returns every project/work-item version ever, tombstones included, forever - -- 1581–1618: `since` defaults to epoch 1970; tombstones intentionally returned and never hard-deleted; no LIMIT; payloads up to 64KB each. Cold-start pull grows monotonically for org lifetime. - -**Fix:** keyset pagination on existing `(org_id, updated_at)` indexes (4078, 4022) + tombstone GC (hard-delete `deleted_at < now() - 90d` via pg_cron; clients with pre-GC cursors do a full resync). - -### H4. Realtime postgres_changes: per-subscriber RLS evaluation on every signal bump + platform quotas - -- Clients subscribe postgres_changes on `org_change_signals` (RLS `is_org_member`, 5614–5615) and `org_memberships` (REPLICA IDENTITY FULL, 5621–5622). Supabase's WAL poller is single-threaded and evaluates RLS per change × per subscriber; all orgs share one signals table so matching scales with TOTAL connected desktops. -- Quotas: concurrent connections ~200 Free / 500 Pro before add-ons; messages ~2M/mo Free, ~5M/mo Pro. A 20-member org doing ~2k writes/day ≈ 2.4M messages/mo alone — the first platform limit this schema hits. - -**Fix:** move the nudge to Broadcast from Database (`realtime.broadcast_changes` on topic `org:`): private-channel auth evaluated once at join (the `realtime.messages` policies already exist, 5694–5706). Keep postgres_changes only for the self-row `org_memberships` eviction path. - -### H5. Replay bytes live in Postgres as base64 TEXT with 100GB/org entitlements - -- `cloud_session_segments.payload_gz text` (3606); base64 +33% over gzip; all in primary Postgres volume (backups/PITR/WAL/replicas). A few team orgs near entitlement = TB-scale Postgres. - -**Fix:** move payloads to Supabase Storage keyed `(org, session, epoch, seq, hash)`; keep pointer + byte_size + hash rows; serve via signed URLs. Interim: `bytea` (−25%). - -## MEDIUM - -### M1. Every work-item upsert takes FOR UPDATE on the owning project row - -- 3026–3031 + allocator update 3110–3113. N members editing N different items of one project fully serialize (~50–100 writes/s per project, org-wide, stacked under C1). **Fix:** take the project lock only around the `next_work_item_id` advance; rely on the advisory item lock (3036–3038) otherwise, keeping project-then-item order. - -### M2. `cloud_delete_account` is one transaction through the caller's PostgREST connection - -- 789–911; web route runs with caller JWT (route.ts:40–42). Missing indexes for user-id predicates: `cloud_sessions(owner_user_id)`, `cloud_session_comments(author_user_id)`, `cloud_session_shares(grantee_user_id)/(owner_user_id)`, `subscriptions(owner_user_id)`. FK cascade deletes potentially GBs of segments under an 8s timeout → heavy users can't be erased (GDPR risk). - **Fix:** add the indexes; move execution to a service-role background job that batch-deletes segments first. - -### M3. No index serves the retention-window predicate on free-plan listings - -- Filter `s.last_activity_at > now() - make_interval(...)` (5550–5553); only PK + `(org_id, updated_at)` exist. **Fix:** `create index on cloud_sessions (org_id, last_activity_at desc) where deleted_at is null;` - -### M4. Unpaginated comment listing + refetch-on-nudge herd - -- `cloud_list_session_comments` (5312–5345) ships all rows (≤500 cap incl. tombstones, ~2MB worst case) with per-comment profiles join; every comment bumps the org signal → every open member re-pulls the full thread. **Fix:** delta via `p_since` (index 4029 exists) or scope the nudge to the session via Broadcast. - -## LOW - -- L1: append path deletes + reinserts the seq-0 tail row per push (670–687); delete predicate lacks `epoch`. Bloat amplifier; monitor autovacuum. -- L2: forever-growing bookkeeping — `stripe_webhook_events` (no pruning), `org_repo_scope_events` (reads windowed by index 4106, storage-only cost), comment tombstones consume the 500 cap permanently (spam-then-delete permanently caps a thread). -- L3: quota COUNT(\*) gates verified fine (seat counts, comment cap under advisory lock, last-admin counts — all bounded). - -## Top 5 first - -1. Redesign the signal fan-out (C1) — org write-throughput ceiling AND realtime message burner. -2. Paginate `cloud_list_org_sessions` (C2) + materialized comment counts. -3. O(1) storage-quota enforcement (C3). -4. Fix the usage_monthly ↔ signal lock-order inversion (H1) — one-line reorder. -5. Page `cloud_get_session_events` (H2) and plan the Storage migration (H5). diff --git a/src-tauri/crates/orgtrack-core/src/sources/claude_code/history.rs b/src-tauri/crates/orgtrack-core/src/sources/claude_code/history.rs index c29f6e130..55fc94c88 100644 --- a/src-tauri/crates/orgtrack-core/src/sources/claude_code/history.rs +++ b/src-tauri/crates/orgtrack-core/src/sources/claude_code/history.rs @@ -11,17 +11,19 @@ use std::path::{Path, PathBuf}; use core_types::activity::ActivityChunk; use rusqlite::Connection; -use serde::Deserialize; +use serde::{Deserialize, Serialize}; use serde_json::{json, Value}; use crate::sources::imported_history::{ self, cache as imported_cache, managed_mirror, metadata::{ ImportedHistoryCacheInput, ImportedHistoryDiscoveredRecord, ImportedHistoryImpactStats, - RoundUsage, SOURCE_CLAUDE_CODE, + RoundUsage, StoredRoundUsage, SOURCE_CLAUDE_CODE, }, - paths as imported_paths, ImportedHistoryRecentPath, ImportedHistorySessionPage, - ImportedHistorySessionRow, ImportedToolCall, + paths as imported_paths, + watermark::{ImportedParseWatermark, WatermarkedTranscriptReader}, + ImportedHistoryRecentPath, ImportedHistorySessionPage, ImportedHistorySessionRow, + ImportedToolCall, }; use super::SESSION_PREFIX as CLAUDE_CODE_SESSION_PREFIX; @@ -241,7 +243,19 @@ fn sync_claude_code_history_cache(conn: &mut Connection) -> Result<(), String> { let mut rounds = Vec::new(); let mut reparsed_ids = Vec::new(); for record in changed { - if let Some(mut meta) = parse_claude_session_meta(record)? { + let stored_watermark = imported_history::watermark::read_parse_watermark_from_conn( + conn, + SOURCE_CLAUDE_CODE, + &record.source_session_id, + )?; + let parse = parse_claude_session_meta_incremental(record, stored_watermark.as_ref())?; + imported_history::watermark::write_parse_watermark_from_conn( + conn, + SOURCE_CLAUDE_CODE, + &record.source_session_id, + &parse.watermark, + )?; + if let Some(mut meta) = parse.meta { let is_managed_history_mirror = managed_ids.contains(&meta.source_session_id); reparsed_ids.push(meta.session_id.clone()); rounds.append(&mut meta.rounds); @@ -404,57 +418,51 @@ fn claude_sessions_dir_for_session_path(session_path: &Path) -> Option }) } -fn parse_claude_session_meta( - record: &ImportedHistoryDiscoveredRecord, -) -> Result, String> { - let file = fs::File::open(&record.source_path).map_err(|err| { - format!( - "Failed to open Claude history {}: {err}", - record.source_path.display() - ) - })?; - let reader = BufReader::new(file); - - let mut created_at_ms = 0; - let mut updated_at_ms = 0; - let mut external_title = claude_session_title_for_record(record)?; - let mut ai_title = String::new(); - let mut custom_title = String::new(); - let mut first_prompt = String::new(); - let mut model: Option = None; - let mut repo_path: Option = None; - let mut branch: Option = None; - let mut input_tokens = 0; - let mut output_tokens = 0; - let mut cache_read_tokens = 0; - let mut cache_write_tokens = 0; - let mut rounds: Vec = Vec::new(); +/// Resumable accumulator for one transcript's meta scan. Every field is +/// exactly the per-file state the old single-pass loop kept in locals, so it +/// can be frozen into a parse watermark's `state_json` at a complete-line +/// boundary and resumed against only the appended suffix. +#[derive(Debug, Clone, Default, Serialize, Deserialize)] +struct ClaudeSessionMetaState { + created_at_ms: i64, + updated_at_ms: i64, + /// First transcript `summary` title; the fresh sessions-dir title + /// (external, re-read each parse) still wins. + summary_title: String, + ai_title: String, + custom_title: String, + first_prompt: String, + model: Option, + repo_path: Option, + branch: Option, + input_tokens: i64, + output_tokens: i64, + cache_read_tokens: i64, + cache_write_tokens: i64, + rounds: Vec, // One API response spans several assistant lines that repeat the same // `usage`; count each `message.id` once. - let mut seen_message_ids: HashSet = HashSet::new(); + seen_message_ids: HashSet, // Primary impact source: exact counts from tool_use_result.structuredPatch. - let mut impact = ImportedHistoryImpactStats::default(); - let mut touched_files = BTreeSet::new(); + impact: ImportedHistoryImpactStats, + touched_files: BTreeSet, // Fallback for transcripts old enough to lack structuredPatch: the coarse // old_string/new_string line count. Only used when no patch data is found. - let mut fallback_impact = ImportedHistoryImpactStats::default(); - let mut fallback_touched = BTreeSet::new(); + fallback_impact: ImportedHistoryImpactStats, + fallback_touched: BTreeSet, // Subagent transcripts (`/subagents/agent-*.jsonl`) tag every // line `isSidechain: true` and carry the spawning session's UUID in // `sessionId`. Capturing it lets us subsume the child under its parent the // same way Codex does, instead of listing it as a top-level session. - let mut parent_source_session_id: Option = None; - let mut first_user_uuid: Option = None; + parent_source_session_id: Option, + first_user_uuid: Option, +} - for line in reader.lines() { - let line = line.map_err(|err| format!("Failed to read Claude history line: {err}"))?; - let trimmed = line.trim(); - if trimmed.is_empty() { - continue; - } +impl ClaudeSessionMetaState { + fn feed(&mut self, trimmed: &str, record: &ImportedHistoryDiscoveredRecord) { let parsed: ClaudeJsonlLine = match serde_json::from_str(trimmed) { Ok(parsed) => parsed, - Err(_) => continue, + Err(_) => return, }; let line_ms = parsed .timestamp @@ -466,97 +474,105 @@ fn parse_claude_session_meta( .as_deref() .and_then(imported_history::parse_iso_to_epoch_ms_opt) { - if created_at_ms == 0 || timestamp < created_at_ms { - created_at_ms = timestamp; + if self.created_at_ms == 0 || timestamp < self.created_at_ms { + self.created_at_ms = timestamp; } - if timestamp > updated_at_ms { - updated_at_ms = timestamp; + if timestamp > self.updated_at_ms { + self.updated_at_ms = timestamp; } } - if repo_path.is_none() && !parsed.cwd.trim().is_empty() { - repo_path = Some(parsed.cwd.clone()); + if self.repo_path.is_none() && !parsed.cwd.trim().is_empty() { + self.repo_path = Some(parsed.cwd.clone()); } - if branch.is_none() && !parsed.git_branch.trim().is_empty() { - branch = Some(parsed.git_branch.clone()); + if self.branch.is_none() && !parsed.git_branch.trim().is_empty() { + self.branch = Some(parsed.git_branch.clone()); } // A sidechain line whose `sessionId` differs from this file's own stem // is a subagent pointing at its spawning session. Guard against a self // reference so a malformed line can never make a session its own parent. - if parent_source_session_id.is_none() && parsed.is_sidechain { + if self.parent_source_session_id.is_none() && parsed.is_sidechain { let candidate = parsed.session_id.trim(); if !candidate.is_empty() && candidate != record.source_session_id { - parent_source_session_id = Some(candidate.to_string()); + self.parent_source_session_id = Some(candidate.to_string()); } } // Claude Code persists the session title inside the transcript. Titles are // re-emitted as the conversation evolves, so the last write wins. match parsed.r#type.as_str() { - "summary" if external_title.is_empty() => { + "summary" if self.summary_title.is_empty() => { let summary = parsed.summary.trim(); if !summary.is_empty() { - external_title = imported_history::truncate_name(summary, 200); + self.summary_title = imported_history::truncate_name(summary, 200); } } "ai-title" => { let title = parsed.ai_title.trim(); if !title.is_empty() { - ai_title = imported_history::truncate_name(title, 200); + self.ai_title = imported_history::truncate_name(title, 200); } } "custom-title" => { let title = parsed.custom_title.trim(); if !title.is_empty() { - custom_title = imported_history::truncate_name(title, 200); + self.custom_title = imported_history::truncate_name(title, 200); } } _ => {} } // Exact diff stats come from the tool-result's structuredPatch. if let Some(result) = parsed.tool_use_result.as_ref() { - collect_claude_impact_from_tool_result(result, &mut impact, &mut touched_files); + collect_claude_impact_from_tool_result( + result, + &mut self.impact, + &mut self.touched_files, + ); } - if first_user_uuid.is_none() && parsed.r#type == "user" && !parsed.uuid.trim().is_empty() { - first_user_uuid = Some(parsed.uuid.trim().to_string()); + if self.first_user_uuid.is_none() + && parsed.r#type == "user" + && !parsed.uuid.trim().is_empty() + { + self.first_user_uuid = Some(parsed.uuid.trim().to_string()); } if let Some(message) = parsed.message { - if first_prompt.is_empty() && parsed.r#type == "user" { + if self.first_prompt.is_empty() && parsed.r#type == "user" { if let Some(text) = claude_content_text(&message.content) { // GUI-launched runs prefix the first prompt with the // exec-mode briefing; bridge-only text is no title // candidate at all. let text = imported_history::strip_orgii_exec_mode_bridge(&text); if !text.trim().is_empty() { - first_prompt = imported_history::truncate_name(text, 200); + self.first_prompt = imported_history::truncate_name(text, 200); } } } - if model.is_none() + if self.model.is_none() && !message.model.trim().is_empty() && !message.model.starts_with('<') { - model = Some(message.model.clone()); + self.model = Some(message.model.clone()); } if parsed.r#type == "assistant" { for item in claude_content_items(&message.content) { collect_claude_impact_from_item( item, - &mut fallback_impact, - &mut fallback_touched, + &mut self.fallback_impact, + &mut self.fallback_touched, ); } } // Skip repeated lines of the same API response (same message.id), // which would otherwise triple both totals and rounds. - let usage_is_new = message.id.is_empty() || seen_message_ids.insert(message.id.clone()); + let usage_is_new = + message.id.is_empty() || self.seen_message_ids.insert(message.id.clone()); if let Some(usage) = message.usage.filter(|_| usage_is_new) { // input_tokens stays cache-inclusive (fresh + both cache kinds); // the cache portion is tracked separately for the cost split. - input_tokens += usage.input_tokens + self.input_tokens += usage.input_tokens + usage.cache_read_input_tokens + usage.cache_creation_input_tokens; - output_tokens += usage.output_tokens; - cache_read_tokens += usage.cache_read_input_tokens; - cache_write_tokens += usage.cache_creation_input_tokens; + self.output_tokens += usage.output_tokens; + self.cache_read_tokens += usage.cache_read_input_tokens; + self.cache_write_tokens += usage.cache_creation_input_tokens; // One round per assistant message that reports usage. `input` // here is FRESH (round convention), cache tracked separately. if usage.input_tokens > 0 @@ -564,12 +580,9 @@ fn parse_claude_session_meta( || usage.cache_read_input_tokens > 0 || usage.cache_creation_input_tokens > 0 { - rounds.push(RoundUsage { - source: SOURCE_CLAUDE_CODE, - source_session_id: record.source_session_id.clone(), - session_id: super::canonical_session_id(&record.source_session_id), - seq: rounds.len() as i64, - model: model.clone(), + self.rounds.push(StoredRoundUsage { + seq: self.rounds.len() as i64, + model: self.model.clone(), input_tokens: usage.input_tokens, output_tokens: usage.output_tokens, cache_read_tokens: usage.cache_read_input_tokens, @@ -581,65 +594,164 @@ fn parse_claude_session_meta( } } - // Prefer the precise structuredPatch counts; fall back to the coarse - // old_string/new_string heuristic only when no patch data was present. - if touched_files.is_empty() && impact.lines_added == 0 && impact.lines_removed == 0 { - impact = fallback_impact; - touched_files = fallback_touched; - } + fn finish( + mut self, + record: &ImportedHistoryDiscoveredRecord, + external_title: String, + ) -> Option { + // Prefer the precise structuredPatch counts; fall back to the coarse + // old_string/new_string heuristic only when no patch data was present. + if self.touched_files.is_empty() + && self.impact.lines_added == 0 + && self.impact.lines_removed == 0 + { + self.impact = self.fallback_impact; + self.touched_files = self.fallback_touched; + } + self.impact.touched_files = self.touched_files.into_iter().collect(); + self.impact.files_changed = self.impact.touched_files.len() as i64; - impact.touched_files = touched_files.into_iter().collect(); - impact.files_changed = impact.touched_files.len() as i64; + if self.created_at_ms == 0 && record.source_mtime_ms == 0 { + return None; + } - if created_at_ms == 0 && record.source_mtime_ms == 0 { - return Ok(None); + let derived_title = if external_title.is_empty() { + self.summary_title + } else { + external_title + }; + let session_id = super::canonical_session_id(&record.source_session_id); + let rounds = self + .rounds + .into_iter() + .map(|round| { + round.into_round_usage(SOURCE_CLAUDE_CODE, &record.source_session_id, &session_id) + }) + .collect(); + Some(ClaudeCodeHistoryMeta { + source_session_id: record.source_session_id.clone(), + session_id, + source_path: record.source_path.to_string_lossy().to_string(), + source_record_key: record.source_record_key.clone(), + source_mtime_ms: record.source_mtime_ms, + source_size_bytes: record.source_size_bytes, + source_fingerprint: record.source_fingerprint.clone(), + // Mirror the Claude Code app's own precedence: a user-set custom title + // wins, then the AI-generated title, then the derived/summary title, + // then the first prompt, and finally the raw session id. + name: if !self.custom_title.is_empty() { + self.custom_title + } else if !self.ai_title.is_empty() { + self.ai_title + } else if !derived_title.is_empty() { + derived_title + } else if !self.first_prompt.is_empty() { + self.first_prompt + } else { + record.source_record_key.clone() + }, + created_at_ms: if self.created_at_ms > 0 { + self.created_at_ms + } else { + record.source_mtime_ms + }, + updated_at_ms: if self.updated_at_ms > 0 { + self.updated_at_ms + } else { + record.source_mtime_ms + }, + model: self.model, + repo_path: self.repo_path, + branch: self.branch, + input_tokens: self.input_tokens, + output_tokens: self.output_tokens, + cache_read_tokens: self.cache_read_tokens, + cache_write_tokens: self.cache_write_tokens, + rounds, + impact: self.impact, + parent_session_id: self + .parent_source_session_id + .map(|uuid| format!("{CLAUDE_CODE_SESSION_PREFIX}{uuid}")), + first_user_uuid: self.first_user_uuid, + }) } +} - Ok(Some(ClaudeCodeHistoryMeta { - source_session_id: record.source_session_id.clone(), - session_id: super::canonical_session_id(&record.source_session_id), - source_path: record.source_path.to_string_lossy().to_string(), - source_record_key: record.source_record_key.clone(), - source_mtime_ms: record.source_mtime_ms, - source_size_bytes: record.source_size_bytes, - source_fingerprint: record.source_fingerprint.clone(), - // Mirror the Claude Code app's own precedence: a user-set custom title - // wins, then the AI-generated title, then the derived/summary title, - // then the first prompt, and finally the raw session id. - name: if !custom_title.is_empty() { - custom_title - } else if !ai_title.is_empty() { - ai_title - } else if !external_title.is_empty() { - external_title - } else if !first_prompt.is_empty() { - first_prompt - } else { - record.source_record_key.clone() - }, - created_at_ms: if created_at_ms > 0 { - created_at_ms - } else { - record.source_mtime_ms - }, - updated_at_ms: if updated_at_ms > 0 { - updated_at_ms +struct ClaudeSessionMetaParse { + meta: Option, + watermark: ImportedParseWatermark, + #[cfg_attr(not(test), allow(dead_code))] + resumed: bool, +} + +fn parse_claude_session_meta_incremental( + record: &ImportedHistoryDiscoveredRecord, + watermark: Option<&ImportedParseWatermark>, +) -> Result { + let mut reader = WatermarkedTranscriptReader::open( + &record.source_path, + "Claude", + watermark, + CLAUDE_CODE_METADATA_PARSER_VERSION, + record.source_mtime_ms, + record.source_size_bytes, + )?; + let mut state = ClaudeSessionMetaState::default(); + let mut resumed = false; + if let Some(state_json) = reader.resume_state_json() { + match serde_json::from_str::(state_json) { + Ok(parsed) => { + state = parsed; + resumed = true; + } + Err(_) => { + reader = WatermarkedTranscriptReader::open( + &record.source_path, + "Claude", + None, + CLAUDE_CODE_METADATA_PARSER_VERSION, + record.source_mtime_ms, + record.source_size_bytes, + )?; + } + } + } + let mut tail_state: Option = None; + while let Some(line) = reader.next_line()? { + let trimmed = line.text.trim(); + if trimmed.is_empty() { + continue; + } + if line.terminated { + state.feed(trimmed, record); } else { - record.source_mtime_ms - }, - model, - repo_path, - branch, - input_tokens, - output_tokens, - cache_read_tokens, - cache_write_tokens, - rounds, - impact, - parent_session_id: parent_source_session_id - .map(|uuid| format!("{CLAUDE_CODE_SESSION_PREFIX}{uuid}")), - first_user_uuid, - })) + let mut snapshot = state.clone(); + snapshot.feed(trimmed, record); + tail_state = Some(snapshot); + } + } + let external_title = claude_session_title_for_record(record)?; + let state_json = serde_json::to_string(&state) + .map_err(|err| format!("Failed to serialize Claude parse state: {err}"))?; + let next_watermark = reader.into_watermark( + CLAUDE_CODE_METADATA_PARSER_VERSION, + record.source_mtime_ms, + record.source_size_bytes, + state_json, + ); + let meta = tail_state.unwrap_or(state).finish(record, external_title); + Ok(ClaudeSessionMetaParse { + meta, + watermark: next_watermark, + resumed, + }) +} + +#[cfg(test)] +fn parse_claude_session_meta( + record: &ImportedHistoryDiscoveredRecord, +) -> Result, String> { + Ok(parse_claude_session_meta_incremental(record, None)?.meta) } fn session_meta_to_cache_input(meta: ClaudeCodeHistoryMeta) -> ImportedHistoryCacheInput { diff --git a/src-tauri/crates/orgtrack-core/src/sources/claude_code/history_tests.rs b/src-tauri/crates/orgtrack-core/src/sources/claude_code/history_tests.rs index c74ffb448..39a5df182 100644 --- a/src-tauri/crates/orgtrack-core/src/sources/claude_code/history_tests.rs +++ b/src-tauri/crates/orgtrack-core/src/sources/claude_code/history_tests.rs @@ -666,3 +666,84 @@ fn strips_orgii_exec_mode_bridge_from_claude_title_and_replay() { std::fs::remove_file(&path).expect("remove fixture"); std::fs::remove_dir(&temp_dir).expect("remove temp dir"); } + +#[test] +fn resumes_claude_meta_parse_from_watermark() { + let temp_dir = std::env::temp_dir().join(format!( + "orgii-claude-history-watermark-test-{}", + std::process::id() + )); + std::fs::create_dir_all(&temp_dir).expect("create temp dir"); + let path = temp_dir.join("claude-watermark.jsonl"); + let prefix = r#"{"type":"user","sessionId":"w","cwd":"/tmp/project","gitBranch":"main","timestamp":"2026-04-01T07:06:46.543Z","message":{"role":"user","content":"build this"}} +{"type":"assistant","sessionId":"w","timestamp":"2026-04-01T07:06:49.000Z","message":{"id":"msg_1","role":"assistant","model":"claude-sonnet-4","content":[{"type":"text","text":"a"}],"usage":{"input_tokens":10,"output_tokens":20,"cache_read_input_tokens":5,"cache_creation_input_tokens":6}}} +"#; + std::fs::write(&path, prefix).expect("write fixture"); + + let record_for = |path: &std::path::Path| { + let (source_mtime_ms, source_size_bytes) = + imported_paths::file_metadata_signature(path, "Claude").expect("metadata"); + ImportedHistoryDiscoveredRecord { + source_session_id: "claude-watermark".to_string(), + source_path: path.to_path_buf(), + source_record_key: "claude-watermark".to_string(), + source_mtime_ms, + source_size_bytes, + source_fingerprint: String::new(), + parser_version: CLAUDE_CODE_METADATA_PARSER_VERSION, + } + }; + + let first = parse_claude_session_meta_incremental(&record_for(&path), None).expect("parse"); + assert!(!first.resumed); + assert_eq!(first.watermark.byte_offset, prefix.len() as i64); + let first_meta = first.meta.expect("first meta"); + assert_eq!(first_meta.input_tokens, 21); + assert_eq!(first_meta.rounds.len(), 1); + + let suffix = r#"{"type":"assistant","sessionId":"w","timestamp":"2026-04-01T07:07:10.000Z","message":{"id":"msg_2","role":"assistant","model":"claude-sonnet-4","content":[{"type":"text","text":"b"}],"usage":{"input_tokens":40,"output_tokens":50,"cache_read_input_tokens":7,"cache_creation_input_tokens":8}}} +"#; + std::fs::write(&path, format!("{prefix}{suffix}")).expect("append fixture"); + + let resumed = parse_claude_session_meta_incremental(&record_for(&path), Some(&first.watermark)) + .expect("parse resumed"); + assert!(resumed.resumed); + let scratch = parse_claude_session_meta_incremental(&record_for(&path), None) + .expect("parse from scratch"); + assert!(!scratch.resumed); + + let resumed_meta = resumed.meta.expect("resumed meta"); + let scratch_meta = scratch.meta.expect("scratch meta"); + assert_eq!(resumed_meta.input_tokens, 21 + 55); + assert_eq!(resumed_meta.input_tokens, scratch_meta.input_tokens); + assert_eq!(resumed_meta.output_tokens, scratch_meta.output_tokens); + assert_eq!(resumed_meta.cache_read_tokens, scratch_meta.cache_read_tokens); + assert_eq!( + resumed_meta.cache_write_tokens, + scratch_meta.cache_write_tokens + ); + assert_eq!(resumed_meta.rounds.len(), 2); + assert_eq!(resumed_meta.rounds.len(), scratch_meta.rounds.len()); + assert_eq!(resumed_meta.rounds[1].seq, 1); + assert_eq!(resumed_meta.rounds[1].input_tokens, 40); + assert_eq!(resumed_meta.name, scratch_meta.name); + assert_eq!(resumed_meta.created_at_ms, scratch_meta.created_at_ms); + assert_eq!(resumed_meta.updated_at_ms, scratch_meta.updated_at_ms); + assert_eq!(resumed.watermark.byte_offset, scratch.watermark.byte_offset); + assert_eq!(resumed.watermark.prefix_hash, scratch.watermark.prefix_hash); + + // Same-length prefix mutation invalidates the watermark: the message.id + // rewrite would double-count msg_2 usage if the resume were trusted. + let mutated = format!("{prefix}{suffix}").replace("build this", "BUILD THIS"); + std::fs::write(&path, mutated).expect("mutate fixture"); + let reparsed = + parse_claude_session_meta_incremental(&record_for(&path), Some(&resumed.watermark)) + .expect("parse mutated"); + assert!(!reparsed.resumed); + let reparsed_meta = reparsed.meta.expect("reparsed meta"); + assert_eq!(reparsed_meta.input_tokens, 21 + 55); + assert_eq!(reparsed_meta.name, "BUILD THIS"); + + std::fs::remove_file(&path).expect("remove fixture"); + std::fs::remove_dir(&temp_dir).expect("remove temp dir"); +} diff --git a/src-tauri/crates/orgtrack-core/src/sources/codex/app/index.rs b/src-tauri/crates/orgtrack-core/src/sources/codex/app/index.rs index de3471e7c..3f03561eb 100644 --- a/src-tauri/crates/orgtrack-core/src/sources/codex/app/index.rs +++ b/src-tauri/crates/orgtrack-core/src/sources/codex/app/index.rs @@ -20,7 +20,7 @@ use crate::sources::imported_history::{ use crate::store::{sqlite::SqliteRecordStore, RecordStore}; use super::meta::{ - parse_codex_session_meta, resolve_codex_transcript_for_thread_id_near_path, + parse_codex_session_meta_incremental, resolve_codex_transcript_for_thread_id_near_path, session_meta_to_cache_input, }; use super::transcript::{ @@ -294,10 +294,22 @@ fn sync_codex_app_cache(conn: &mut Connection) -> Result<(), String> { let mut reparsed_ids = Vec::new(); let now_ns = unix_epoch_now_ns(); for record in changed { - if should_defer_active_codex_metadata(&record, now_ns) { + if should_defer_active_codex_metadata(record, now_ns) { continue; } - if let Some(mut meta) = parse_codex_session_meta(record)? { + let stored_watermark = imported_history::watermark::read_parse_watermark_from_conn( + conn, + SOURCE_CODEX_APP, + &record.source_session_id, + )?; + let parse = parse_codex_session_meta_incremental(record, stored_watermark.as_ref())?; + imported_history::watermark::write_parse_watermark_from_conn( + conn, + SOURCE_CODEX_APP, + &record.source_session_id, + &parse.watermark, + )?; + if let Some(mut meta) = parse.meta { let is_managed_history_mirror = crate::sources::imported_history::managed_mirror::is_managed_source_session_id( &managed_ids, diff --git a/src-tauri/crates/orgtrack-core/src/sources/codex/app/meta.rs b/src-tauri/crates/orgtrack-core/src/sources/codex/app/meta.rs index 8e7bdb47f..3539a156b 100644 --- a/src-tauri/crates/orgtrack-core/src/sources/codex/app/meta.rs +++ b/src-tauri/crates/orgtrack-core/src/sources/codex/app/meta.rs @@ -1,11 +1,9 @@ //! Codex session-meta parsing and parent-thread resolution. use std::collections::BTreeSet; -use std::fs; -use std::io::{BufRead, BufReader}; use std::path::{Path, PathBuf}; -use serde::Deserialize; +use serde::{Deserialize, Serialize}; use serde_json::Value; use crate::sources::codex::canonical_session_id; @@ -13,8 +11,9 @@ use crate::sources::imported_history::{ self, metadata::{ ImportedHistoryCacheInput, ImportedHistoryDiscoveredRecord, ImportedHistoryImpactStats, - RoundUsage, SOURCE_CODEX_APP, + StoredRoundUsage, SOURCE_CODEX_APP, }, + watermark::{ImportedParseWatermark, WatermarkedTranscriptReader}, }; use super::impact::{collect_codex_impact_from_patch_apply_end, collect_codex_impact_from_payload}; @@ -42,98 +41,92 @@ struct CodexTurnContextPayload { model: String, } -pub(crate) fn parse_codex_session_meta( - record: &ImportedHistoryDiscoveredRecord, -) -> Result, String> { - let file = fs::File::open(&record.source_path).map_err(|err| { - format!( - "Failed to open Codex history {}: {err}", - record.source_path.display() - ) - })?; - let reader = BufReader::new(file); - - let mut created_at_ms = 0; - let mut updated_at_ms = 0; - let mut external_title = codex_session_index_title_for_record(record)?; - let mut first_prompt = String::new(); - let mut model: Option = None; - let mut repo_path: Option = None; +/// Resumable accumulator for one rollout's meta scan. Every field is exactly +/// the per-file state the old single-pass loop kept in locals, so it can be +/// frozen into a parse watermark's `state_json` at a complete-line boundary +/// and resumed against only the appended suffix. +#[derive(Debug, Clone, Default, Serialize, Deserialize)] +struct CodexSessionMetaState { + created_at_ms: i64, + updated_at_ms: i64, + /// Title carried inside the transcript's `session_meta` lines; the fresh + /// session-index title (external, re-read each parse) still wins. + transcript_title: String, + first_prompt: String, + model: Option, + repo_path: Option, // Session totals are accumulated from per-round deltas (robust to codex's // cumulative resets on /compact). `input_tokens` is cache-inclusive here to // match the imported-cache convention. - let mut input_tokens = 0; - let mut output_tokens = 0; - let mut cache_read_tokens = 0; - let mut cache_write_tokens = 0; - let mut rounds: Vec = Vec::new(); + input_tokens: i64, + output_tokens: i64, + cache_read_tokens: i64, + cache_write_tokens: i64, + rounds: Vec, // Previous cumulative `total_token_usage` for delta computation. - let mut prev_input = 0i64; - let mut prev_cached = 0i64; - let mut prev_cache_write = 0i64; - let mut prev_output = 0i64; + prev_input: i64, + prev_cached: i64, + prev_cache_write: i64, + prev_output: i64, // Primary impact source: `patch_apply_end` events, which Codex emits after // every *successful* apply with a structured `changes` map (path -> // unified_diff). This covers every edit path uniformly — the `apply_patch` - // tool, `exec`-wrapped patches, etc. The tool-call scan below is only a + // tool, `exec`-wrapped patches, etc. The tool-call scan is only a // fallback for older rollouts that predate `patch_apply_end`. - let mut impact = ImportedHistoryImpactStats::default(); - let mut touched_files = BTreeSet::new(); - let mut fallback_impact = ImportedHistoryImpactStats::default(); - let mut fallback_touched = BTreeSet::new(); - let mut parent_thread_id: Option = None; - let mut source_metadata = CodexAppSourceMetadata::default(); + impact: ImportedHistoryImpactStats, + touched_files: BTreeSet, + fallback_impact: ImportedHistoryImpactStats, + fallback_touched: BTreeSet, + parent_thread_id: Option, + source_metadata: CodexAppSourceMetadata, +} - for line in reader.lines() { - let line = line.map_err(|err| format!("Failed to read Codex history line: {err}"))?; - let trimmed = line.trim(); - if trimmed.is_empty() { - continue; - } +impl CodexSessionMetaState { + fn feed(&mut self, trimmed: &str, record: &ImportedHistoryDiscoveredRecord) { let parsed: CodexJsonlLine = match serde_json::from_str(trimmed) { Ok(parsed) => parsed, - Err(_) => continue, + Err(_) => return, }; if let Some(timestamp) = parsed .timestamp .as_deref() .and_then(imported_history::parse_iso_to_epoch_ms_opt) { - if created_at_ms == 0 || timestamp < created_at_ms { - created_at_ms = timestamp; + if self.created_at_ms == 0 || timestamp < self.created_at_ms { + self.created_at_ms = timestamp; } - if timestamp > updated_at_ms { - updated_at_ms = timestamp; + if timestamp > self.updated_at_ms { + self.updated_at_ms = timestamp; } } - if first_prompt.is_empty() { + if self.first_prompt.is_empty() { if let Some(message) = user_message_from_payload(&parsed.payload) { - first_prompt = message; + self.first_prompt = message; } } - if external_title.is_empty() && parsed.line_type == "session_meta" { + if self.transcript_title.is_empty() && parsed.line_type == "session_meta" { if let Some(title) = session_title_from_payload(&parsed.payload) { - external_title = imported_history::truncate_name(&title, 200); + self.transcript_title = imported_history::truncate_name(&title, 200); } } - if parent_thread_id.is_none() && parsed.line_type == "session_meta" { - parent_thread_id = parent_thread_id_from_session_meta_payload( + if self.parent_thread_id.is_none() && parsed.line_type == "session_meta" { + self.parent_thread_id = parent_thread_id_from_session_meta_payload( &parsed.payload, codex_thread_id_from_file_stem(&record.source_record_key), ); } if parsed.line_type == "session_meta" { - capture_subagent_source_metadata(&parsed.payload, &mut source_metadata); + capture_subagent_source_metadata(&parsed.payload, &mut self.source_metadata); } - if model.is_none() || repo_path.is_none() { + if self.model.is_none() || self.repo_path.is_none() { if let Ok(turn_context) = serde_json::from_value::(parsed.payload.clone()) { - if model.is_none() && !turn_context.model.trim().is_empty() { - model = Some(turn_context.model); + if self.model.is_none() && !turn_context.model.trim().is_empty() { + self.model = Some(turn_context.model); } - if repo_path.is_none() && !turn_context.cwd.trim().is_empty() { - repo_path = Some(turn_context.cwd); + if self.repo_path.is_none() && !turn_context.cwd.trim().is_empty() { + self.repo_path = Some(turn_context.cwd); } } } @@ -153,102 +146,206 @@ pub(crate) fn parse_codex_session_meta( // Per-field delta, treating a decrease (codex resets on /compact) // as a fresh start so totals never go negative or undercount. let delta = |cum: i64, prev: i64| if cum >= prev { cum - prev } else { cum }; - let d_input = delta(cum_input, prev_input); - let d_cached = delta(cum_cached, prev_cached); - let d_cache_write = delta(cum_cache_write, prev_cache_write); - let d_output = delta(cum_output, prev_output); + let d_input = delta(cum_input, self.prev_input); + let d_cached = delta(cum_cached, self.prev_cached); + let d_cache_write = delta(cum_cache_write, self.prev_cache_write); + let d_output = delta(cum_output, self.prev_output); let d_fresh = (d_input - d_cached - d_cache_write).max(0); if d_input > 0 || d_output > 0 { let event_ms = parsed .timestamp .as_deref() .and_then(imported_history::parse_iso_to_epoch_ms_opt) - .unwrap_or(updated_at_ms); - rounds.push(RoundUsage { - source: SOURCE_CODEX_APP, - source_session_id: record.source_session_id.clone(), - session_id: canonical_session_id(&record.source_session_id), - seq: rounds.len() as i64, - model: model.clone(), + .unwrap_or(self.updated_at_ms); + self.rounds.push(StoredRoundUsage { + seq: self.rounds.len() as i64, + model: self.model.clone(), input_tokens: d_fresh, output_tokens: d_output, cache_read_tokens: d_cached, cache_write_tokens: d_cache_write, created_at_ms: event_ms, }); - input_tokens += d_input; - output_tokens += d_output; - cache_read_tokens += d_cached; - cache_write_tokens += d_cache_write; + self.input_tokens += d_input; + self.output_tokens += d_output; + self.cache_read_tokens += d_cached; + self.cache_write_tokens += d_cache_write; } - prev_input = cum_input; - prev_cached = cum_cached; - prev_cache_write = cum_cache_write; - prev_output = cum_output; + self.prev_input = cum_input; + self.prev_cached = cum_cached; + self.prev_cache_write = cum_cache_write; + self.prev_output = cum_output; } } - collect_codex_impact_from_patch_apply_end(&parsed.payload, &mut impact, &mut touched_files); + collect_codex_impact_from_patch_apply_end( + &parsed.payload, + &mut self.impact, + &mut self.touched_files, + ); collect_codex_impact_from_payload( &parsed.payload, - &mut fallback_impact, - &mut fallback_touched, + &mut self.fallback_impact, + &mut self.fallback_touched, ); } - // Prefer the authoritative `patch_apply_end` tally; only fall back to the - // tool-call scan when no successful applies were recorded (older rollouts). - if touched_files.is_empty() && impact.lines_added == 0 && impact.lines_removed == 0 { - impact = fallback_impact; - touched_files = fallback_touched; - } + fn finish( + mut self, + record: &ImportedHistoryDiscoveredRecord, + external_title: String, + ) -> Option { + // Prefer the authoritative `patch_apply_end` tally; only fall back to + // the tool-call scan when no successful applies were recorded. + if self.touched_files.is_empty() + && self.impact.lines_added == 0 + && self.impact.lines_removed == 0 + { + self.impact = self.fallback_impact; + self.touched_files = self.fallback_touched; + } + self.impact.touched_files = self.touched_files.into_iter().collect(); + self.impact.files_changed = self.impact.touched_files.len() as i64; - impact.touched_files = touched_files.into_iter().collect(); - impact.files_changed = impact.touched_files.len() as i64; + if self.created_at_ms == 0 && record.source_mtime_ms == 0 { + return None; + } - if created_at_ms == 0 && record.source_mtime_ms == 0 { - return Ok(None); + let title = if external_title.is_empty() { + self.transcript_title + } else { + external_title + }; + let name = if !title.is_empty() { + title + } else if self.first_prompt.is_empty() { + record.source_record_key.clone() + } else { + imported_history::truncate_name(&self.first_prompt, 200) + }; + let session_id = canonical_session_id(&record.source_session_id); + let rounds = self + .rounds + .into_iter() + .map(|round| { + round.into_round_usage(SOURCE_CODEX_APP, &record.source_session_id, &session_id) + }) + .collect(); + let mut source_metadata = self.source_metadata; + source_metadata.first_prompt = + (!self.first_prompt.trim().is_empty()).then_some(self.first_prompt); + Some(CodexAppSessionMeta { + source_session_id: record.source_session_id.clone(), + session_id, + source_path: record.source_path.to_string_lossy().to_string(), + source_record_key: record.source_record_key.clone(), + source_mtime_ms: record.source_mtime_ms, + source_size_bytes: record.source_size_bytes, + source_fingerprint: record.source_fingerprint.clone(), + name, + parent_session_id: self + .parent_thread_id + .as_deref() + .and_then(|thread_id| codex_parent_session_id_for_record(record, thread_id)), + created_at_ms: if self.created_at_ms > 0 { + self.created_at_ms + } else { + record.source_mtime_ms + }, + updated_at_ms: if self.updated_at_ms > 0 { + self.updated_at_ms + } else { + record.source_mtime_ms + }, + model: self.model, + repo_path: self.repo_path, + input_tokens: self.input_tokens, + output_tokens: self.output_tokens, + cache_read_tokens: self.cache_read_tokens, + cache_write_tokens: self.cache_write_tokens, + impact: self.impact, + rounds, + source_metadata, + }) } +} - let name = if !external_title.is_empty() { - external_title - } else if first_prompt.is_empty() { - record.source_record_key.clone() - } else { - imported_history::truncate_name(&first_prompt, 200) - }; - source_metadata.first_prompt = (!first_prompt.trim().is_empty()).then_some(first_prompt); - Ok(Some(CodexAppSessionMeta { - source_session_id: record.source_session_id.clone(), - session_id: canonical_session_id(&record.source_session_id), - source_path: record.source_path.to_string_lossy().to_string(), - source_record_key: record.source_record_key.clone(), - source_mtime_ms: record.source_mtime_ms, - source_size_bytes: record.source_size_bytes, - source_fingerprint: record.source_fingerprint.clone(), - name, - parent_session_id: parent_thread_id - .as_deref() - .and_then(|thread_id| codex_parent_session_id_for_record(record, thread_id)), - created_at_ms: if created_at_ms > 0 { - created_at_ms - } else { - record.source_mtime_ms - }, - updated_at_ms: if updated_at_ms > 0 { - updated_at_ms +pub(crate) struct CodexSessionMetaParse { + pub meta: Option, + pub watermark: ImportedParseWatermark, + #[cfg_attr(not(test), allow(dead_code))] + pub resumed: bool, +} + +pub(crate) fn parse_codex_session_meta_incremental( + record: &ImportedHistoryDiscoveredRecord, + watermark: Option<&ImportedParseWatermark>, +) -> Result { + let mut reader = WatermarkedTranscriptReader::open( + &record.source_path, + "Codex", + watermark, + CODEX_APP_METADATA_PARSER_VERSION, + record.source_mtime_ms, + record.source_size_bytes, + )?; + let mut state = CodexSessionMetaState::default(); + let mut resumed = false; + if let Some(state_json) = reader.resume_state_json() { + match serde_json::from_str::(state_json) { + Ok(parsed) => { + state = parsed; + resumed = true; + } + Err(_) => { + reader = WatermarkedTranscriptReader::open( + &record.source_path, + "Codex", + None, + CODEX_APP_METADATA_PARSER_VERSION, + record.source_mtime_ms, + record.source_size_bytes, + )?; + } + } + } + let mut tail_state: Option = None; + while let Some(line) = reader.next_line()? { + let trimmed = line.text.trim(); + if trimmed.is_empty() { + continue; + } + if line.terminated { + state.feed(trimmed, record); } else { - record.source_mtime_ms - }, - model, - repo_path, - input_tokens, - output_tokens, - cache_read_tokens, - cache_write_tokens, - impact, - rounds, - source_metadata, - })) + let mut snapshot = state.clone(); + snapshot.feed(trimmed, record); + tail_state = Some(snapshot); + } + } + let external_title = codex_session_index_title_for_record(record)?; + let state_json = serde_json::to_string(&state) + .map_err(|err| format!("Failed to serialize Codex parse state: {err}"))?; + let next_watermark = reader.into_watermark( + CODEX_APP_METADATA_PARSER_VERSION, + record.source_mtime_ms, + record.source_size_bytes, + state_json, + ); + let meta = tail_state + .unwrap_or(state) + .finish(record, external_title); + Ok(CodexSessionMetaParse { + meta, + watermark: next_watermark, + resumed, + }) +} + +#[cfg(test)] +pub(crate) fn parse_codex_session_meta( + record: &ImportedHistoryDiscoveredRecord, +) -> Result, String> { + Ok(parse_codex_session_meta_incremental(record, None)?.meta) } pub(super) fn session_meta_to_cache_input(meta: CodexAppSessionMeta) -> ImportedHistoryCacheInput { diff --git a/src-tauri/crates/orgtrack-core/src/sources/codex/app/mod.rs b/src-tauri/crates/orgtrack-core/src/sources/codex/app/mod.rs index 99bcba90f..a79f17f69 100644 --- a/src-tauri/crates/orgtrack-core/src/sources/codex/app/mod.rs +++ b/src-tauri/crates/orgtrack-core/src/sources/codex/app/mod.rs @@ -43,7 +43,7 @@ pub(crate) use crate::sources::imported_history::{ #[cfg(test)] pub(crate) use index::codex_sessions_dir_candidates; #[cfg(test)] -pub(crate) use meta::parse_codex_session_meta; +pub(crate) use meta::{parse_codex_session_meta, parse_codex_session_meta_incremental}; #[cfg(test)] pub(crate) use serde_json::json; #[cfg(test)] diff --git a/src-tauri/crates/orgtrack-core/src/sources/codex/app_tests.rs b/src-tauri/crates/orgtrack-core/src/sources/codex/app_tests.rs index 76d6554f9..e991d2688 100644 --- a/src-tauri/crates/orgtrack-core/src/sources/codex/app_tests.rs +++ b/src-tauri/crates/orgtrack-core/src/sources/codex/app_tests.rs @@ -1985,3 +1985,88 @@ fn strips_ide_context_from_codex_user_text() { Some("fix the login bug") ); } + +#[test] +fn resumes_codex_meta_parse_from_watermark() { + let temp_dir = std::env::temp_dir().join(format!( + "orgii-codex-history-watermark-test-{}", + std::process::id() + )); + std::fs::create_dir_all(&temp_dir).expect("create temp dir"); + let path = temp_dir.join("rollout-watermark.jsonl"); + let prefix = r#"{"timestamp":"2026-02-11T06:16:06.458Z","type":"session_meta","payload":{"cwd":"/Users/me/project","id":"abc"}} +{"timestamp":"2026-02-11T06:16:07.000Z","type":"event_msg","payload":{"type":"user_message","message":"resume me","images":[],"local_images":[],"text_elements":[]}} +{"timestamp":"2026-02-11T06:16:09.000Z","type":"event_msg","payload":{"type":"token_count","info":{"total_token_usage":{"input_tokens":100,"cached_input_tokens":20,"cache_write_input_tokens":10,"output_tokens":30,"reasoning_output_tokens":5}}}} +"#; + std::fs::write(&path, prefix).expect("write fixture"); + + let record_for = |path: &std::path::Path| { + let (source_mtime_ms, source_size_bytes) = + imported_paths::file_metadata_signature(path, "Codex").expect("metadata"); + ImportedHistoryDiscoveredRecord { + source_session_id: "rollout-watermark".to_string(), + source_path: path.to_path_buf(), + source_record_key: "rollout-watermark".to_string(), + source_mtime_ms, + source_size_bytes, + source_fingerprint: String::new(), + parser_version: CODEX_APP_METADATA_PARSER_VERSION, + } + }; + + let first = parse_codex_session_meta_incremental(&record_for(&path), None).expect("parse"); + assert!(!first.resumed); + assert_eq!(first.watermark.byte_offset, prefix.len() as i64); + let first_meta = first.meta.expect("first meta"); + assert_eq!(first_meta.input_tokens, 100); + assert_eq!(first_meta.output_tokens, 35); + assert_eq!(first_meta.rounds.len(), 1); + + // Cumulative totals continue past the watermark; the per-round delta + // depends on prev_* carried inside the persisted state. + let suffix = r#"{"timestamp":"2026-02-11T06:17:09.000Z","type":"event_msg","payload":{"type":"token_count","info":{"total_token_usage":{"input_tokens":260,"cached_input_tokens":60,"cache_write_input_tokens":25,"output_tokens":70,"reasoning_output_tokens":10}}}} +"#; + std::fs::write(&path, format!("{prefix}{suffix}")).expect("append fixture"); + + let resumed = parse_codex_session_meta_incremental(&record_for(&path), Some(&first.watermark)) + .expect("parse resumed"); + assert!(resumed.resumed); + let scratch = + parse_codex_session_meta_incremental(&record_for(&path), None).expect("parse from scratch"); + assert!(!scratch.resumed); + + let resumed_meta = resumed.meta.expect("resumed meta"); + let scratch_meta = scratch.meta.expect("scratch meta"); + assert_eq!(resumed_meta.input_tokens, scratch_meta.input_tokens); + assert_eq!(resumed_meta.output_tokens, scratch_meta.output_tokens); + assert_eq!(resumed_meta.cache_read_tokens, scratch_meta.cache_read_tokens); + assert_eq!( + resumed_meta.cache_write_tokens, + scratch_meta.cache_write_tokens + ); + assert_eq!(resumed_meta.rounds.len(), 2); + assert_eq!(resumed_meta.rounds.len(), scratch_meta.rounds.len()); + assert_eq!(resumed_meta.rounds[1].seq, 1); + assert_eq!(resumed_meta.rounds[1].input_tokens, 105); // Δinput 160 − Δcached 40 − Δcache_write 15 + assert_eq!( + resumed_meta.rounds[1].input_tokens, + scratch_meta.rounds[1].input_tokens + ); + assert_eq!(resumed_meta.name, scratch_meta.name); + assert_eq!(resumed_meta.updated_at_ms, scratch_meta.updated_at_ms); + assert_eq!(resumed.watermark.byte_offset, scratch.watermark.byte_offset); + assert_eq!(resumed.watermark.prefix_hash, scratch.watermark.prefix_hash); + + // Same-length prefix mutation invalidates the resume. + let mutated = format!("{prefix}{suffix}").replace("resume me", "RESUME ME"); + std::fs::write(&path, mutated).expect("mutate fixture"); + let reparsed = + parse_codex_session_meta_incremental(&record_for(&path), Some(&resumed.watermark)) + .expect("parse mutated"); + assert!(!reparsed.resumed); + let reparsed_meta = reparsed.meta.expect("reparsed meta"); + assert_eq!(reparsed_meta.input_tokens, scratch_meta.input_tokens); + assert_eq!(reparsed_meta.name, "RESUME ME"); + + std::fs::remove_dir_all(&temp_dir).expect("remove temp dir"); +} diff --git a/src-tauri/crates/orgtrack-core/src/sources/imported_history/cache.rs b/src-tauri/crates/orgtrack-core/src/sources/imported_history/cache.rs index 9d6a812b7..ea0a76f30 100644 --- a/src-tauri/crates/orgtrack-core/src/sources/imported_history/cache.rs +++ b/src-tauri/crates/orgtrack-core/src/sources/imported_history/cache.rs @@ -321,6 +321,11 @@ pub fn prune_missing_records_from_conn( [source], ) .ok(); + conn.execute( + "DELETE FROM imported_history_parse_watermarks WHERE source = ?1", + [source], + ) + .ok(); return Ok(()); } @@ -345,6 +350,13 @@ pub fn prune_missing_records_from_conn( [source], ) .ok(); + conn.execute( + "DELETE FROM imported_history_parse_watermarks \ + WHERE source = ?1 AND source_session_id NOT IN \ + (SELECT source_session_id FROM imported_history_session_cache WHERE source = ?1)", + [source], + ) + .ok(); Ok(()) } @@ -842,7 +854,9 @@ fn has_newer_continuation_sibling( WHERE source = ?1 AND source_session_id != ?2 AND COALESCE(parent_session_id, '') = '' - AND json_extract(source_metadata_json, '$.{CONTINUATION_GROUP_KEY_FIELD}') = ?3 + AND CASE WHEN json_valid(source_metadata_json) + THEN json_extract(source_metadata_json, '$.{CONTINUATION_GROUP_KEY_FIELD}') + END = ?3 AND (updated_at_ms > ?4 OR (updated_at_ms = ?4 AND source_session_id > ?2)) )" diff --git a/src-tauri/crates/orgtrack-core/src/sources/imported_history/cache_tests.rs b/src-tauri/crates/orgtrack-core/src/sources/imported_history/cache_tests.rs index ec0b9e820..77df63108 100644 --- a/src-tauri/crates/orgtrack-core/src/sources/imported_history/cache_tests.rs +++ b/src-tauri/crates/orgtrack-core/src/sources/imported_history/cache_tests.rs @@ -448,6 +448,82 @@ fn canonical_lookup_skips_continuation_superseded_siblings() { assert_eq!(child.source_session_id, "child"); } +#[test] +fn canonical_lookup_tolerates_legacy_non_json_metadata_rows() { + let mut conn = fixture_conn(); + let group = continuation_group_metadata_json(Some("family-uuid")); + let mut older = input(SOURCE_CODEX_APP, "gen1", 100); + older.source_metadata_json = group.clone(); + let mut newest = input(SOURCE_CODEX_APP, "gen2", 200); + newest.source_metadata_json = group; + let keyless = input(SOURCE_CODEX_APP, "journal", 300); + upsert_imported_session_cache_from_conn(&mut conn, &[older, newest, keyless]) + .expect("upsert"); + conn.execute( + "UPDATE imported_history_session_cache SET source_metadata_json = 'not-json' \ + WHERE source = ?1 AND source_session_id = 'journal'", + [SOURCE_CODEX_APP], + ) + .expect("write corrupt metadata"); + let empty: String = conn + .query_row( + "SELECT source_metadata_json FROM imported_history_session_cache \ + WHERE source = ?1 AND source_session_id = 'gen1'", + [SOURCE_CODEX_APP], + |row| row.get(0), + ) + .expect("read gen1 metadata"); + assert!(empty.starts_with('{')); + let mut legacy_empty = input(SOURCE_CODEX_APP, "keyless", 50); + legacy_empty.source_metadata_json = None; + upsert_imported_session_cache_from_conn(&mut conn, &[legacy_empty]) + .expect("upsert legacy empty row"); + + assert!( + query_cached_session_by_session_id_from_conn(&conn, "codex_app-gen1") + .expect("superseded lookup succeeds despite corrupt sibling rows") + .is_none() + ); + let (_, winner) = query_cached_session_by_session_id_from_conn(&conn, "codex_app-gen2") + .expect("winner lookup succeeds despite corrupt sibling rows") + .expect("winner resolves"); + assert_eq!(winner.source_session_id, "gen2"); + let (_, keyless_row) = + query_cached_session_by_session_id_from_conn(&conn, "codex_app-journal") + .expect("corrupt-metadata row still resolves by id") + .expect("corrupt-metadata row present"); + assert_eq!(keyless_row.source_session_id, "journal"); +} + +#[test] +#[ignore] +fn real_db_copy_sibling_query_never_errors() { + let path = std::env::var("ORGTRACK_REAL_DB_COPY").expect("set ORGTRACK_REAL_DB_COPY"); + let conn = Connection::open(&path).expect("open real db copy"); + let session_ids: Vec = conn + .prepare("SELECT session_id FROM imported_history_session_cache") + .expect("prepare") + .query_map([], |row| row.get(0)) + .expect("query") + .collect::>() + .expect("collect"); + let mut resolved = 0usize; + let mut demoted = 0usize; + for session_id in &session_ids { + match query_cached_session_by_session_id_from_conn(&conn, session_id) + .unwrap_or_else(|err| panic!("lookup failed for {session_id}: {err}")) + { + Some(_) => resolved += 1, + None => demoted += 1, + } + } + println!( + "real-db-copy rows={} resolved={resolved} demoted={demoted}", + session_ids.len() + ); + assert_eq!(resolved + demoted, session_ids.len()); +} + #[test] fn source_cache_signature_tracks_upserts_demotions_and_prunes() { let mut conn = fixture_conn(); diff --git a/src-tauri/crates/orgtrack-core/src/sources/imported_history/metadata.rs b/src-tauri/crates/orgtrack-core/src/sources/imported_history/metadata.rs index d51e9e4af..dbf250345 100644 --- a/src-tauri/crates/orgtrack-core/src/sources/imported_history/metadata.rs +++ b/src-tauri/crates/orgtrack-core/src/sources/imported_history/metadata.rs @@ -1,6 +1,8 @@ use std::path::PathBuf; -#[derive(Debug, Clone, Default)] +use serde::{Deserialize, Serialize}; + +#[derive(Debug, Clone, Default, Serialize, Deserialize)] pub struct ImportedHistoryImpactStats { pub files_changed: i64, pub lines_added: i64, @@ -107,6 +109,41 @@ pub struct RoundUsage { pub created_at_ms: i64, } +/// Per-round usage snapshot inside a parse-watermark state blob: +/// [`RoundUsage`] minus the identity columns re-derivable from the record. +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct StoredRoundUsage { + pub seq: i64, + pub model: Option, + pub input_tokens: i64, + pub output_tokens: i64, + pub cache_read_tokens: i64, + pub cache_write_tokens: i64, + pub created_at_ms: i64, +} + +impl StoredRoundUsage { + pub fn into_round_usage( + self, + source: &'static str, + source_session_id: &str, + session_id: &str, + ) -> RoundUsage { + RoundUsage { + source, + source_session_id: source_session_id.to_string(), + session_id: session_id.to_string(), + seq: self.seq, + model: self.model, + input_tokens: self.input_tokens, + output_tokens: self.output_tokens, + cache_read_tokens: self.cache_read_tokens, + cache_write_tokens: self.cache_write_tokens, + created_at_ms: self.created_at_ms, + } + } +} + #[derive(Debug, Clone)] pub struct ImportedHistoryRecordSignature { pub source_session_id: String, diff --git a/src-tauri/crates/orgtrack-core/src/sources/imported_history/mod.rs b/src-tauri/crates/orgtrack-core/src/sources/imported_history/mod.rs index fd8aef396..bfc7d2ee0 100644 --- a/src-tauri/crates/orgtrack-core/src/sources/imported_history/mod.rs +++ b/src-tauri/crates/orgtrack-core/src/sources/imported_history/mod.rs @@ -5,6 +5,7 @@ pub mod metadata; pub mod paths; #[cfg(feature = "git")] pub mod repo_identity; +pub mod watermark; use std::collections::{BTreeSet, HashMap}; use std::path::Path; diff --git a/src-tauri/crates/orgtrack-core/src/sources/imported_history/watermark.rs b/src-tauri/crates/orgtrack-core/src/sources/imported_history/watermark.rs new file mode 100644 index 000000000..0a9899ce5 --- /dev/null +++ b/src-tauri/crates/orgtrack-core/src/sources/imported_history/watermark.rs @@ -0,0 +1,292 @@ +//! Per-file parse watermarks for incremental transcript ingestion. +//! +//! A huge imported CLI transcript that keeps growing (a live session) changes +//! its `(mtime, size)` signature on every boot, which used to force a full +//! re-parse of the whole file each time. The watermark persists, per +//! `(source, source_session_id)`, the byte offset of the last COMPLETE line +//! already folded into the parser's accumulator, a hash of exactly those +//! bytes, and the accumulator state itself (`state_json`). A later parse +//! resumes from the offset when the prefix is verifiably intact — file at +//! least as large, mtime not regressed, same parser version, prefix hash +//! match — and falls back to a full re-parse otherwise. +//! +//! Only newline-terminated lines advance the watermark: a live writer may +//! still be appending to the final unterminated line, so its effects must +//! not be frozen into the persisted state (the parser feeds it into a +//! throwaway clone of the accumulator instead). + +use std::fs::File; +use std::io::{BufRead, BufReader, Read, Seek, SeekFrom}; +use std::path::Path; + +use rusqlite::{params, Connection, OptionalExtension}; + +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ImportedParseWatermark { + pub byte_offset: i64, + pub source_size_bytes: i64, + /// Nanosecond mtime; see [`super::metadata::ImportedHistoryCacheInput::source_mtime_ms`]. + pub source_mtime_ms: i64, + pub prefix_hash: String, + pub parser_version: i64, + pub state_json: String, +} + +pub fn read_parse_watermark_from_conn( + conn: &Connection, + source: &str, + source_session_id: &str, +) -> Result, String> { + let result = conn + .query_row( + "SELECT byte_offset, source_size_bytes, source_mtime_ms, prefix_hash, + parser_version, state_json + FROM imported_history_parse_watermarks + WHERE source = ?1 AND source_session_id = ?2", + params![source, source_session_id], + |row| { + Ok(ImportedParseWatermark { + byte_offset: row.get(0)?, + source_size_bytes: row.get(1)?, + source_mtime_ms: row.get(2)?, + prefix_hash: row.get(3)?, + parser_version: row.get(4)?, + state_json: row.get(5)?, + }) + }, + ) + .optional(); + match result { + Ok(watermark) => Ok(watermark), + Err( + rusqlite::Error::InvalidColumnType(..) + | rusqlite::Error::FromSqlConversionFailure(..) + | rusqlite::Error::IntegralValueOutOfRange(..), + ) => Ok(None), + Err(err) => Err(format!("Failed to read imported parse watermark: {err}")), + } +} + +pub fn write_parse_watermark_from_conn( + conn: &Connection, + source: &str, + source_session_id: &str, + watermark: &ImportedParseWatermark, +) -> Result<(), String> { + conn.execute( + "INSERT OR REPLACE INTO imported_history_parse_watermarks ( + source, source_session_id, byte_offset, source_size_bytes, + source_mtime_ms, prefix_hash, parser_version, state_json + ) VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8)", + params![ + source, + source_session_id, + watermark.byte_offset, + watermark.source_size_bytes, + watermark.source_mtime_ms, + watermark.prefix_hash, + watermark.parser_version, + watermark.state_json, + ], + ) + .map_err(|err| format!("Failed to write imported parse watermark: {err}"))?; + Ok(()) +} + +pub fn clear_parse_watermark_from_conn( + conn: &Connection, + source: &str, + source_session_id: &str, +) -> Result<(), String> { + conn.execute( + "DELETE FROM imported_history_parse_watermarks + WHERE source = ?1 AND source_session_id = ?2", + params![source, source_session_id], + ) + .map_err(|err| format!("Failed to clear imported parse watermark: {err}"))?; + Ok(()) +} + +/// Streaming FNV-1a 64 over the processed prefix. Integrity check against +/// accidental prefix rewrites, not an adversarial digest — chosen because it +/// can keep hashing across the resume boundary (validate the stored prefix, +/// then continue over newly consumed lines) without re-reading the file. +#[derive(Debug, Clone)] +pub struct PrefixHasher(u64); + +impl Default for PrefixHasher { + fn default() -> Self { + Self(0xcbf2_9ce4_8422_2325) + } +} + +impl PrefixHasher { + pub fn update(&mut self, bytes: &[u8]) { + for &byte in bytes { + self.0 ^= u64::from(byte); + self.0 = self.0.wrapping_mul(0x0000_0100_0000_01b3); + } + } + + pub fn digest(&self) -> String { + format!("{:016x}", self.0) + } +} + +#[derive(Debug)] +pub struct TranscriptLine { + pub text: String, + /// `false` only for a final line with no trailing newline — a live + /// writer may still be appending to it, so it must not advance the + /// watermark or the persisted accumulator state. + pub terminated: bool, +} + +/// Line reader over one transcript that tracks the complete-line byte offset +/// and prefix hash, seeking past an intact watermark prefix on open. +pub struct WatermarkedTranscriptReader { + reader: BufReader, + hasher: PrefixHasher, + complete_offset: u64, + resume_state_json: Option, + buf: Vec, + error_label: &'static str, +} + +impl WatermarkedTranscriptReader { + pub fn open( + path: &Path, + error_label: &'static str, + watermark: Option<&ImportedParseWatermark>, + parser_version: i64, + current_mtime_ns: i64, + current_size_bytes: i64, + ) -> Result { + let mut file = File::open(path).map_err(|err| { + format!( + "Failed to open {error_label} history {}: {err}", + path.display() + ) + })?; + let file_len = file + .metadata() + .map_err(|err| format!("Failed to read {error_label} history metadata: {err}"))? + .len(); + + let mut hasher = PrefixHasher::default(); + let mut complete_offset = 0u64; + let mut resume_state_json = None; + if let Some(watermark) = watermark { + let eligible = watermark.parser_version == parser_version + && watermark.byte_offset >= 0 + && current_size_bytes >= watermark.source_size_bytes + && current_mtime_ns >= watermark.source_mtime_ms + && watermark.byte_offset as u64 <= file_len; + if eligible + && Self::hash_prefix(&mut file, watermark.byte_offset as u64, &mut hasher)? + && hasher.digest() == watermark.prefix_hash + { + complete_offset = watermark.byte_offset as u64; + resume_state_json = Some(watermark.state_json.clone()); + } + if resume_state_json.is_none() { + hasher = PrefixHasher::default(); + file.seek(SeekFrom::Start(0)).map_err(|err| { + format!("Failed to rewind {error_label} history: {err}") + })?; + } + } + + Ok(Self { + reader: BufReader::new(file), + hasher, + complete_offset, + resume_state_json, + buf: Vec::new(), + error_label, + }) + } + + fn hash_prefix( + file: &mut File, + prefix_len: u64, + hasher: &mut PrefixHasher, + ) -> Result { + let mut remaining = prefix_len; + let mut chunk = vec![0u8; 256 * 1024]; + while remaining > 0 { + let take = remaining.min(chunk.len() as u64) as usize; + let read = file + .read(&mut chunk[..take]) + .map_err(|err| format!("Failed to read history prefix: {err}"))?; + if read == 0 { + return Ok(false); + } + hasher.update(&chunk[..read]); + remaining -= read as u64; + } + Ok(true) + } + + pub fn resume_state_json(&self) -> Option<&str> { + self.resume_state_json.as_deref() + } + + pub fn next_line(&mut self) -> Result, String> { + self.buf.clear(); + let read = self + .reader + .read_until(b'\n', &mut self.buf) + .map_err(|err| { + format!( + "Failed to read {} history line: {err}", + self.error_label + ) + })?; + if read == 0 { + return Ok(None); + } + let terminated = self.buf.last() == Some(&b'\n'); + if terminated { + self.hasher.update(&self.buf); + self.complete_offset += read as u64; + } + let mut end = self.buf.len(); + if terminated { + end -= 1; + if end > 0 && self.buf[end - 1] == b'\r' { + end -= 1; + } + } + let text = std::str::from_utf8(&self.buf[..end]) + .map_err(|err| { + format!( + "Failed to read {} history line: {err}", + self.error_label + ) + })? + .to_string(); + Ok(Some(TranscriptLine { text, terminated })) + } + + pub fn into_watermark( + self, + parser_version: i64, + current_mtime_ns: i64, + current_size_bytes: i64, + state_json: String, + ) -> ImportedParseWatermark { + ImportedParseWatermark { + byte_offset: self.complete_offset as i64, + source_size_bytes: current_size_bytes, + source_mtime_ms: current_mtime_ns, + prefix_hash: self.hasher.digest(), + parser_version, + state_json, + } + } +} + +#[cfg(test)] +#[path = "watermark_tests.rs"] +mod tests; diff --git a/src-tauri/crates/orgtrack-core/src/sources/imported_history/watermark_tests.rs b/src-tauri/crates/orgtrack-core/src/sources/imported_history/watermark_tests.rs new file mode 100644 index 000000000..cb2e2fd07 --- /dev/null +++ b/src-tauri/crates/orgtrack-core/src/sources/imported_history/watermark_tests.rs @@ -0,0 +1,247 @@ +use std::fs; +use std::path::{Path, PathBuf}; + +use rusqlite::Connection; + +use super::*; +use crate::sources::imported_history::paths as imported_paths; + +fn fixture_conn() -> Connection { + let conn = Connection::open_in_memory().expect("open in-memory db"); + crate::store::sqlite::SqliteRecordStore::init_tables(&conn).expect("init core tables"); + crate::store::sqlite::SqliteRecordStore::init_source_cache_tables(&conn) + .expect("init source cache tables"); + conn +} + +fn temp_transcript(tag: &str, content: &str) -> PathBuf { + let dir = std::env::temp_dir().join(format!( + "orgii-imported-watermark-test-{tag}-{}", + std::process::id() + )); + fs::create_dir_all(&dir).expect("create temp dir"); + let path = dir.join("transcript.jsonl"); + fs::write(&path, content).expect("write fixture"); + path +} + +fn cleanup(path: &Path) { + fs::remove_file(path).ok(); + if let Some(dir) = path.parent() { + fs::remove_dir(dir).ok(); + } +} + +fn stat(path: &Path) -> (i64, i64) { + imported_paths::file_metadata_signature(path, "Test").expect("stat fixture") +} + +fn read_all(reader: &mut WatermarkedTranscriptReader) -> Vec<(String, bool)> { + let mut lines = Vec::new(); + while let Some(line) = reader.next_line().expect("read line") { + lines.push((line.text, line.terminated)); + } + lines +} + +#[test] +fn resume_reads_only_the_appended_suffix() { + let path = temp_transcript("resume", "alpha\nbeta\n"); + let (mtime, size) = stat(&path); + + let mut full = WatermarkedTranscriptReader::open(&path, "Test", None, 1, mtime, size) + .expect("open full"); + assert!(full.resume_state_json().is_none()); + assert_eq!( + read_all(&mut full), + vec![("alpha".to_string(), true), ("beta".to_string(), true)] + ); + let watermark = full.into_watermark(1, mtime, size, "state-1".to_string()); + assert_eq!(watermark.byte_offset, "alpha\nbeta\n".len() as i64); + + fs::OpenOptions::new() + .append(true) + .open(&path) + .and_then(|mut file| std::io::Write::write_all(&mut file, b"gamma\n")) + .expect("append"); + let (mtime_after, size_after) = stat(&path); + let mut resumed = + WatermarkedTranscriptReader::open(&path, "Test", Some(&watermark), 1, mtime_after, size_after) + .expect("open resumed"); + assert_eq!(resumed.resume_state_json(), Some("state-1")); + assert_eq!(read_all(&mut resumed), vec![("gamma".to_string(), true)]); + let next = resumed.into_watermark(1, mtime_after, size_after, "state-2".to_string()); + assert_eq!(next.byte_offset, "alpha\nbeta\ngamma\n".len() as i64); + + cleanup(&path); +} + +#[test] +fn unterminated_tail_is_returned_but_never_watermarked() { + let path = temp_transcript("tail", "alpha\npart"); + let (mtime, size) = stat(&path); + + let mut reader = WatermarkedTranscriptReader::open(&path, "Test", None, 1, mtime, size) + .expect("open full"); + assert_eq!( + read_all(&mut reader), + vec![("alpha".to_string(), true), ("part".to_string(), false)] + ); + let watermark = reader.into_watermark(1, mtime, size, "state-1".to_string()); + assert_eq!(watermark.byte_offset, "alpha\n".len() as i64); + + fs::write(&path, "alpha\npartial-done\n").expect("complete tail"); + let (mtime_after, size_after) = stat(&path); + let mut resumed = + WatermarkedTranscriptReader::open(&path, "Test", Some(&watermark), 1, mtime_after, size_after) + .expect("open resumed"); + assert_eq!(resumed.resume_state_json(), Some("state-1")); + assert_eq!( + read_all(&mut resumed), + vec![("partial-done".to_string(), true)] + ); + + cleanup(&path); +} + +#[test] +fn prefix_mutation_forces_a_full_reparse() { + let path = temp_transcript("mutated", "aa\nbb\n"); + let (mtime, size) = stat(&path); + let mut reader = WatermarkedTranscriptReader::open(&path, "Test", None, 1, mtime, size) + .expect("open full"); + read_all(&mut reader); + let watermark = reader.into_watermark(1, mtime, size, "state-1".to_string()); + + fs::write(&path, "xx\nbb\ncc\n").expect("rewrite prefix"); + let (mtime_after, size_after) = stat(&path); + let mut reopened = + WatermarkedTranscriptReader::open(&path, "Test", Some(&watermark), 1, mtime_after, size_after) + .expect("open reopened"); + assert!(reopened.resume_state_json().is_none()); + assert_eq!( + read_all(&mut reopened), + vec![ + ("xx".to_string(), true), + ("bb".to_string(), true), + ("cc".to_string(), true) + ] + ); + + cleanup(&path); +} + +#[test] +fn size_regression_and_parser_version_change_force_a_full_reparse() { + let path = temp_transcript("invalidate", "aa\nbb\n"); + let (mtime, size) = stat(&path); + let mut reader = WatermarkedTranscriptReader::open(&path, "Test", None, 1, mtime, size) + .expect("open full"); + read_all(&mut reader); + let watermark = reader.into_watermark(1, mtime, size, "state-1".to_string()); + + fs::write(&path, "aa\n").expect("truncate"); + let (mtime_shrunk, size_shrunk) = stat(&path); + let shrunk = WatermarkedTranscriptReader::open( + &path, + "Test", + Some(&watermark), + 1, + mtime_shrunk, + size_shrunk, + ) + .expect("open shrunk"); + assert!(shrunk.resume_state_json().is_none()); + + fs::write(&path, "aa\nbb\n").expect("restore"); + let (mtime_restored, size_restored) = stat(&path); + let bumped = WatermarkedTranscriptReader::open( + &path, + "Test", + Some(&watermark), + 2, + mtime_restored, + size_restored, + ) + .expect("open bumped parser version"); + assert!(bumped.resume_state_json().is_none()); + + let regressed_mtime = WatermarkedTranscriptReader::open( + &path, + "Test", + Some(&watermark), + 1, + watermark.source_mtime_ms - 1, + size_restored, + ) + .expect("open regressed mtime"); + assert!(regressed_mtime.resume_state_json().is_none()); + + cleanup(&path); +} + +#[test] +fn watermark_rows_roundtrip_and_clear() { + let conn = fixture_conn(); + let watermark = ImportedParseWatermark { + byte_offset: 42, + source_size_bytes: 50, + source_mtime_ms: 1_234, + prefix_hash: "abcd".to_string(), + parser_version: 9, + state_json: "{\"created_at_ms\":1}".to_string(), + }; + + assert_eq!( + read_parse_watermark_from_conn(&conn, "claude_code", "sess-1").expect("read empty"), + None + ); + write_parse_watermark_from_conn(&conn, "claude_code", "sess-1", &watermark).expect("write"); + assert_eq!( + read_parse_watermark_from_conn(&conn, "claude_code", "sess-1").expect("read"), + Some(watermark.clone()) + ); + assert_eq!( + read_parse_watermark_from_conn(&conn, "codex_app", "sess-1").expect("read other source"), + None + ); + + clear_parse_watermark_from_conn(&conn, "claude_code", "sess-1").expect("clear"); + assert_eq!( + read_parse_watermark_from_conn(&conn, "claude_code", "sess-1").expect("read cleared"), + None + ); +} + +#[test] +fn malformed_watermark_row_degrades_to_none_and_self_heals() { + let conn = fixture_conn(); + conn.execute( + "INSERT INTO imported_history_parse_watermarks ( + source, source_session_id, byte_offset, source_size_bytes, + source_mtime_ms, prefix_hash, parser_version, state_json + ) VALUES ('claude_code', 'sess-1', 'garbage', 50, 1234, 'abcd', 9, '{}')", + [], + ) + .expect("insert malformed row"); + + assert_eq!( + read_parse_watermark_from_conn(&conn, "claude_code", "sess-1") + .expect("malformed row reads as absent"), + None + ); + + let healed = ImportedParseWatermark { + byte_offset: 42, + source_size_bytes: 50, + source_mtime_ms: 1_234, + prefix_hash: "abcd".to_string(), + parser_version: 9, + state_json: "{\"created_at_ms\":1}".to_string(), + }; + write_parse_watermark_from_conn(&conn, "claude_code", "sess-1", &healed).expect("rewrite"); + assert_eq!( + read_parse_watermark_from_conn(&conn, "claude_code", "sess-1").expect("read healed"), + Some(healed) + ); +} diff --git a/src-tauri/crates/orgtrack-core/src/store/sqlite/schema.rs b/src-tauri/crates/orgtrack-core/src/store/sqlite/schema.rs index fa17758f2..742bfca4a 100644 --- a/src-tauri/crates/orgtrack-core/src/store/sqlite/schema.rs +++ b/src-tauri/crates/orgtrack-core/src/store/sqlite/schema.rs @@ -594,7 +594,22 @@ impl SqliteRecordStore<'_> { CREATE INDEX IF NOT EXISTS idx_imported_round_created ON imported_history_round_usage(created_at_ms DESC); CREATE INDEX IF NOT EXISTS idx_imported_round_source - ON imported_history_round_usage(source);", + ON imported_history_round_usage(source); + + -- Incremental-parse resume points: byte offset + hash of the + -- processed complete-line prefix plus the serialized accumulator + -- state, so a grown transcript parses only its appended suffix. + CREATE TABLE IF NOT EXISTS imported_history_parse_watermarks ( + source TEXT NOT NULL, + source_session_id TEXT NOT NULL, + byte_offset INTEGER NOT NULL DEFAULT 0, + source_size_bytes INTEGER NOT NULL DEFAULT 0, + source_mtime_ms INTEGER NOT NULL DEFAULT 0, + prefix_hash TEXT NOT NULL DEFAULT '', + parser_version INTEGER NOT NULL DEFAULT 0, + state_json TEXT NOT NULL DEFAULT '', + PRIMARY KEY (source, source_session_id) + );", ) } } diff --git a/src/features/Org2Cloud/CloudShareImportDialog.tsx b/src/features/Org2Cloud/CloudShareImportDialog.tsx index 455df128c..3191b9796 100644 --- a/src/features/Org2Cloud/CloudShareImportDialog.tsx +++ b/src/features/Org2Cloud/CloudShareImportDialog.tsx @@ -49,6 +49,10 @@ import { consumeOrg2CloudPendingShareAtom, org2CloudPendingShareAtom, } from "./org2CloudPendingShareAtom"; +import { + Org2CloudSignerError, + type Org2CloudSignerErrorCode, +} from "./org2CloudReplaySignedReads"; import { requireCloudShareAuthEndpoint, resolveCloudShareEndpoint, @@ -67,6 +71,7 @@ interface ResolveState { interface ImportState { attemptId: number; status: "importing" | "failed"; + signerCode?: Org2CloudSignerErrorCode; } const CloudShareImportDialog: React.FC = () => { @@ -181,6 +186,9 @@ const CloudShareImportDialog: React.FC = () => { const resolveFailed = resolveError !== null; const isImporting = currentImport?.status === "importing"; const importFailed = currentImport?.status === "failed" && !resolveFailed; + const importSignerCode = importFailed + ? (currentImport?.signerCode ?? null) + : null; const localSource = resolved?.session ? findLocalCloudShareSource(sessions, resolved.session) : null; @@ -265,12 +273,18 @@ const CloudShareImportDialog: React.FC = () => { }); openSession(result.localSessionId, resolved.session.title, localRepoPath); handleClose(); - } catch { + } catch (error) { if ( activeAttemptRef.current === currentAttemptId && importGenerationRef.current === generation ) { - setImportState({ attemptId: currentAttemptId, status: "failed" }); + setImportState({ + attemptId: currentAttemptId, + status: "failed", + ...(error instanceof Org2CloudSignerError + ? { signerCode: error.code } + : {}), + }); } } }, [ @@ -398,8 +412,13 @@ const CloudShareImportDialog: React.FC = () => {
- {t("cloud.share.incomingRetryHint")} + {importSignerCode === "unreachable" + ? t("cloud.share.incomingConnectionError") + : importSignerCode !== null + ? t("cloud.share.incomingError") + : t("cloud.share.incomingRetryHint")}
) : null} diff --git a/src/features/Org2Cloud/org2CloudBackendAdapter.test.ts b/src/features/Org2Cloud/org2CloudBackendAdapter.test.ts index d8d52c400..598e3e023 100644 --- a/src/features/Org2Cloud/org2CloudBackendAdapter.test.ts +++ b/src/features/Org2Cloud/org2CloudBackendAdapter.test.ts @@ -12,6 +12,7 @@ import { buildCloudSessionFetchClient, cloudSessionIdFromRowId, } from "./org2CloudBackendAdapter"; +import { createGuestReplayObjectReader } from "./org2CloudReplaySignedReads"; import { downloadReplayObject } from "./org2CloudStorageClient"; import type { CloudSessionEventsSnapshot } from "./org2CloudSyncClient"; import { Org2CloudSyncError, isOrg2SyncErrorCode } from "./org2CloudSyncClient"; @@ -26,9 +27,15 @@ vi.mock("./org2CloudStorageClient", async (importOriginal) => { return { ...actual, downloadReplayObject: vi.fn() }; }); +vi.mock("./org2CloudReplaySignedReads", async (importOriginal) => { + const actual = await importOriginal(); + return { ...actual, createGuestReplayObjectReader: vi.fn() }; +}); + const { getSessionEvents } = await import("./org2CloudSyncClient"); const getSessionEventsMock = vi.mocked(getSessionEvents); const downloadReplayObjectMock = vi.mocked(downloadReplayObject); +const createGuestReaderMock = vi.mocked(createGuestReplayObjectReader); function makeEvent(id: string): SessionEvent { return { @@ -63,6 +70,7 @@ describe("buildCloudSessionFetchClient", () => { beforeEach(() => { getSessionEventsMock.mockReset(); downloadReplayObjectMock.mockReset(); + createGuestReaderMock.mockReset(); }); it("maps the cloud response into the self-hosted snapshot shape", async () => { @@ -143,7 +151,7 @@ describe("buildCloudSessionFetchClient", () => { expect(tailSeg.events).toEqual(tail); }); - it("threads the pinned endpoint into storage downloads", async () => { + it("threads the pinned endpoint into member storage downloads", async () => { const stored = await toFrozenSegmentStorage({ seq: 1, events: frozen1 }); const storagePath = `org-1/agentsession-abc/1/1-${stored.segmentHash}.gz`; getSessionEventsMock.mockResolvedValue({ @@ -162,20 +170,134 @@ describe("buildCloudSessionFetchClient", () => { anonKey: "custom-anon", isOfficial: false, }; + const client = buildCloudSessionFetchClient("jwt-member", endpoint); + + await client.getSessionEventSegments({ + orgId: "org-1", + sessionRowId: "org-1:user-1:agentsession-abc", + }); + + expect(downloadReplayObjectMock).toHaveBeenCalledWith( + "jwt-member", + storagePath, + endpoint, + undefined + ); + expect(createGuestReaderMock).not.toHaveBeenCalled(); + }); + + it("reads share-token storage segments through the signed-url flow", async () => { + const stored = await toFrozenSegmentStorage({ seq: 1, events: frozen1 }); + const storagePath = `org-1/agentsession-abc/1/1-${stored.segmentHash}.gz`; + getSessionEventsMock.mockResolvedValue({ + epoch: 1, + frozenSeq: 1, + tailHash: null, + count: 2, + segments: [ + { seq: 1, storagePath, eventCount: 2, segmentHash: stored.segmentHash }, + ], + }); + const download = vi.fn(async () => stored.bytes); + createGuestReaderMock.mockReturnValue({ download }); + const endpoint = { + webOrigin: "https://app.custom.example.com", + supabaseUrl: "https://db.custom.example.com", + anonKey: "custom-anon", + isOfficial: false, + }; const client = buildCloudSessionFetchClient("jwt-non-member", endpoint); + const snapshot = await client.getSessionEventSegments({ + orgId: "org-1", + sessionRowId: "org-1:user-1:agentsession-abc", + shareToken: "t".repeat(64), + }); + + expect(createGuestReaderMock).toHaveBeenCalledWith({ + orgId: "org-1", + sessionId: "agentsession-abc", + shareToken: "t".repeat(64), + endpoint, + }); + expect(download).toHaveBeenCalledWith(storagePath, undefined); + expect(downloadReplayObjectMock).not.toHaveBeenCalled(); + expect(snapshot.segments[0].events).toEqual(frozen1); + + // Same import, second fetch: the reader (and its signed-url cache) is + // shared instead of re-minting a grant per call. await client.getSessionEventSegments({ orgId: "org-1", sessionRowId: "org-1:user-1:agentsession-abc", shareToken: "t".repeat(64), }); + expect(createGuestReaderMock).toHaveBeenCalledTimes(1); + }); + + it("falls back to the member download when the authorize RPC is missing", async () => { + const stored = await toFrozenSegmentStorage({ seq: 1, events: frozen1 }); + const storagePath = `org-1/agentsession-abc/1/1-${stored.segmentHash}.gz`; + getSessionEventsMock.mockResolvedValue({ + epoch: 1, + frozenSeq: 1, + tailHash: null, + count: 2, + segments: [ + { seq: 1, storagePath, eventCount: 2, segmentHash: stored.segmentHash }, + ], + }); + createGuestReaderMock.mockReturnValue({ + download: vi.fn(async () => { + throw new Org2CloudSyncError("Could not find the function", 404); + }), + }); + downloadReplayObjectMock.mockResolvedValue(stored.bytes); + const client = buildCloudSessionFetchClient("jwt-non-member"); + + const snapshot = await client.getSessionEventSegments({ + orgId: "org-1", + sessionRowId: "org-1:user-1:agentsession-abc", + shareToken: "t".repeat(64), + }); expect(downloadReplayObjectMock).toHaveBeenCalledWith( "jwt-non-member", storagePath, - endpoint, + undefined, undefined ); + expect(snapshot.segments[0].events).toEqual(frozen1); + }); + + it("propagates guest signed-read failures without a member fallback", async () => { + const stored = await toFrozenSegmentStorage({ seq: 1, events: frozen1 }); + const storagePath = `org-1/agentsession-abc/1/1-${stored.segmentHash}.gz`; + getSessionEventsMock.mockResolvedValue({ + epoch: 1, + frozenSeq: 1, + tailHash: null, + count: 2, + segments: [ + { seq: 1, storagePath, eventCount: 2, segmentHash: stored.segmentHash }, + ], + }); + createGuestReaderMock.mockReturnValue({ + download: vi.fn(async () => { + throw new Org2CloudSyncError("ORG2_FORBIDDEN", 403); + }), + }); + const client = buildCloudSessionFetchClient("jwt-non-member"); + + await expect( + client.getSessionEventSegments({ + orgId: "org-1", + sessionRowId: "org-1:user-1:agentsession-abc", + shareToken: "t".repeat(64), + }) + ).rejects.toSatisfy((error: unknown) => + isOrg2SyncErrorCode(error, "ORG2_FORBIDDEN") + ); + expect(downloadReplayObjectMock).not.toHaveBeenCalled(); }); it("fails closed on a segment carrying neither payloadGz nor storagePath", async () => { diff --git a/src/features/Org2Cloud/org2CloudBackendAdapter.ts b/src/features/Org2Cloud/org2CloudBackendAdapter.ts index d058aeea0..b2bd288bb 100644 --- a/src/features/Org2Cloud/org2CloudBackendAdapter.ts +++ b/src/features/Org2Cloud/org2CloudBackendAdapter.ts @@ -39,6 +39,11 @@ import { mapSegmentsBounded, } from "../TeamCollaboration/sync/segmentCodec"; import type { CloudEndpoint } from "./config"; +import { + type GuestReplayObjectReader, + createGuestReplayObjectReader, + isReplayAuthorizeRpcMissing, +} from "./org2CloudReplaySignedReads"; import { downloadReplayObject } from "./org2CloudStorageClient"; import { type CloudSegmentWire, @@ -52,10 +57,14 @@ export type CloudSessionFetchClient = Pick< "getSessionEventSegments" | "streamSessionEventSegments" >; +type SegmentObjectDownload = ( + storagePath: string, + signal?: AbortSignal +) => Promise; + async function decodeCloudSegmentEvents( segment: CloudSegmentWire, - accessToken: string, - endpoint?: CloudEndpoint, + downloadObject: SegmentObjectDownload, signal?: AbortSignal ): Promise { if (segment.payloadGz != null) { @@ -63,12 +72,7 @@ async function decodeCloudSegmentEvents( } if (segment.storagePath != null) { return decodeSegmentEventsFromBytes( - await downloadReplayObject( - accessToken, - segment.storagePath, - endpoint, - signal - ) + await downloadObject(segment.storagePath, signal) ); } throw new Error("cloud segment carries neither payloadGz nor storagePath"); @@ -77,8 +81,7 @@ async function decodeCloudSegmentEvents( async function decodeCloudSegments( segments: readonly CloudSegmentWire[], afterSeq: number, - accessToken: string, - endpoint?: CloudEndpoint, + downloadObject: SegmentObjectDownload, signal?: AbortSignal ): Promise { return mapSegmentsBounded( @@ -91,12 +94,7 @@ async function decodeCloudSegments( return { seq, isTail: seq === 0, - events: await decodeCloudSegmentEvents( - segment, - accessToken, - endpoint, - signal - ), + events: await decodeCloudSegmentEvents(segment, downloadObject, signal), eventCount: segment.eventCount, segmentHash: segment.segmentHash, }; @@ -126,12 +124,57 @@ export function cloudSessionIdFromRowId(sessionRowId: string): string { * Link imports still require a registered user's access token. The optional * `input.shareToken` — threaded by the importer from * `RemoteSessionFetchOptions` — grants that signed-in user access without - * requiring membership in the source org. + * requiring membership in the source org. Storage-offloaded segments then + * read through the signed-url flow (`org2CloudReplaySignedReads`): the guest + * JWT cannot pass the replay bucket's member RLS. The signed-url map is + * cached per (session, share token) for this client's lifetime — one import. + * A backend without the authorize RPC falls back to the member download so + * the import surfaces exactly the failure it had before the signer existed. */ export function buildCloudSessionFetchClient( accessToken: string, endpoint?: CloudEndpoint ): CloudSessionFetchClient { + const guestReaders = new Map(); + const downloadForInput = (input: { + orgId: string; + sessionRowId: string; + shareToken?: string; + }): SegmentObjectDownload => { + const shareToken = input.shareToken; + if (shareToken === undefined) { + return (storagePath, signal) => + downloadReplayObject(accessToken, storagePath, endpoint, signal); + } + const sessionId = cloudSessionIdFromRowId(input.sessionRowId); + const readerKey = `${input.orgId}\u001f${sessionId}\u001f${shareToken}`; + let reader = guestReaders.get(readerKey); + if (!reader) { + reader = createGuestReplayObjectReader({ + orgId: input.orgId, + sessionId, + shareToken, + ...(endpoint !== undefined ? { endpoint } : {}), + }); + guestReaders.set(readerKey, reader); + } + const guestReader = reader; + return async (storagePath, signal) => { + try { + return await guestReader.download(storagePath, signal); + } catch (error) { + if (isReplayAuthorizeRpcMissing(error)) { + return downloadReplayObject( + accessToken, + storagePath, + endpoint, + signal + ); + } + throw error; + } + }; + }; return { async getSessionEventSegments( input: GetSessionEventSegmentsInput @@ -160,8 +203,7 @@ export function buildCloudSessionFetchClient( const segments = await decodeCloudSegments( snapshot.segments, afterSeq, - accessToken, - endpoint, + downloadForInput(input), input.signal ); return { @@ -174,6 +216,7 @@ export function buildCloudSessionFetchClient( }, async streamSessionEventSegments(input, onPage) { const afterSeq = input.afterSeq ?? 0; + const downloadObject = downloadForInput(input); return streamSessionEvents( accessToken, input.orgId, @@ -187,8 +230,7 @@ export function buildCloudSessionFetchClient( segments: await decodeCloudSegments( page.segments, afterSeq, - accessToken, - endpoint, + downloadObject, input.signal ), }); diff --git a/src/features/Org2Cloud/org2CloudCommentsClient.test.ts b/src/features/Org2Cloud/org2CloudCommentsClient.test.ts index 22a451cc0..9b1b46e16 100644 --- a/src/features/Org2Cloud/org2CloudCommentsClient.test.ts +++ b/src/features/Org2Cloud/org2CloudCommentsClient.test.ts @@ -7,6 +7,7 @@ import { } from "./config"; import { Org2CloudCommentError, + __SESSION_COMMENTS_DELTA_INTERNALS, addSessionComment, deleteSessionComment, editSessionComment, @@ -49,6 +50,7 @@ const WIRE_COMMENT = { beforeEach(() => { vi.stubGlobal("fetch", fetchMock); fetchMock.mockResolvedValue(jsonResponse(null)); + __SESSION_COMMENTS_DELTA_INTERNALS.resetDeltaSupport(); }); afterEach(() => { @@ -431,6 +433,80 @@ describe("listSessionComments", () => { ); expect(isOrg2CommentErrorCode(error, "ORG2_RETENTION_EXPIRED")).toBe(true); }); + + it("parses the 0004 serverTime anchor on a full listing", async () => { + fetchMock.mockResolvedValueOnce( + jsonResponse({ + comments: [WIRE_COMMENT], + viewerOwnsSession: false, + serverTime: "2026-07-24T10:00:00.000Z", + }) + ); + const listing = await listSessionComments("jwt-1", "org-1", "sess-1"); + expect(listing.serverTime).toBe("2026-07-24T10:00:00.000Z"); + expect(listing.appliedSince).toBeUndefined(); + expect(lastBody()).toEqual({ p_org_id: "org-1", p_session_id: "sess-1" }); + }); + + it("sends p_since and echoes the honored cursor on a delta listing", async () => { + fetchMock.mockResolvedValueOnce( + jsonResponse({ + comments: [WIRE_COMMENT], + viewerOwnsSession: true, + serverTime: "2026-07-24T10:05:00.000Z", + }) + ); + const listing = await listSessionComments("jwt-1", "org-1", "sess-1", { + since: "2026-07-24T09:59:58.000Z", + }); + expect(lastBody()).toEqual({ + p_org_id: "org-1", + p_session_id: "sess-1", + p_since: "2026-07-24T09:59:58.000Z", + }); + expect(listing.appliedSince).toBe("2026-07-24T09:59:58.000Z"); + expect(listing.serverTime).toBe("2026-07-24T10:05:00.000Z"); + }); + + it("degrades to a full listing on a pre-delta backend and pins the endpoint", async () => { + fetchMock + .mockResolvedValueOnce( + jsonResponse({ message: "Could not find the function" }, 404) + ) + .mockResolvedValueOnce( + jsonResponse({ comments: [WIRE_COMMENT], viewerOwnsSession: false }) + ); + const listing = await listSessionComments("jwt-1", "org-1", "sess-1", { + since: "2026-07-24T09:59:58.000Z", + }); + expect(fetchMock).toHaveBeenCalledTimes(2); + expect(lastBody()).toEqual({ p_org_id: "org-1", p_session_id: "sess-1" }); + // The caller MUST see this as a full listing, not the delta it asked for. + expect(listing.appliedSince).toBeUndefined(); + + // The endpoint is pinned: the next delta request goes straight to the + // legacy signature without a probing 404 round-trip. + fetchMock.mockResolvedValueOnce( + jsonResponse({ comments: [], viewerOwnsSession: false }) + ); + await listSessionComments("jwt-1", "org-1", "sess-1", { + since: "2026-07-24T10:04:58.000Z", + }); + expect(fetchMock).toHaveBeenCalledTimes(3); + expect(lastBody()).toEqual({ p_org_id: "org-1", p_session_id: "sess-1" }); + }); + + it("does not treat a non-signature 404 as missing delta support", async () => { + fetchMock.mockResolvedValueOnce( + jsonResponse({ message: "ORG2_SESSION_NOT_FOUND" }, 404) + ); + await expect( + listSessionComments("jwt-1", "org-1", "sess-1", { + since: "2026-07-24T09:59:58.000Z", + }) + ).rejects.toMatchObject({ code: "ORG2_SESSION_NOT_FOUND" }); + expect(fetchMock).toHaveBeenCalledTimes(1); + }); }); describe("Org2CloudCommentError code extraction", () => { diff --git a/src/features/Org2Cloud/org2CloudCommentsClient.ts b/src/features/Org2Cloud/org2CloudCommentsClient.ts index 5bca51881..98615ef91 100644 --- a/src/features/Org2Cloud/org2CloudCommentsClient.ts +++ b/src/features/Org2Cloud/org2CloudCommentsClient.ts @@ -187,6 +187,12 @@ const ListCommentsResultSchema = z.object({ comments: z.array(CloudSessionCommentWireSchema).default([]), /** Viewer-derived server capability; false for imports, forks and members. */ viewerOwnsSession: z.boolean().default(false), + /** 0004 delta anchor; absent on pre-delta backends. */ + serverTime: z + .string() + .nullish() + .transform((value) => value ?? undefined) + .optional(), }); const EditCommentResultSchema = z.object({ @@ -341,17 +347,68 @@ export async function resolveSessionComment( export interface SessionCommentsListing { comments: CloudSessionComment[]; viewerOwnsSession: boolean; + /** 0004 delta anchor for the caller's next `since`; absent pre-delta. */ + serverTime?: string; + /** The `since` the server actually honored; undefined ⇒ full listing. */ + appliedSince?: string; +} + +/** supabaseUrl set of backends that rejected the p_since signature (pre-0004). */ +const commentsDeltaUnsupportedEndpoints = new Set(); + +function isCommentsDeltaSignatureUnsupported(error: unknown): boolean { + return ( + error instanceof Org2CloudCommentError && + error.status === 404 && + /could not find the function/i.test(error.message) + ); } +export const __SESSION_COMMENTS_DELTA_INTERNALS = { + resetDeltaSupport: () => commentsDeltaUnsupportedEndpoints.clear(), +}; + /** - * Full thread list for one readable session, `created_at` asc (no - * pagination — the 500-row cap bounds the response). Tombstones included. + * Thread list for one readable session, `created_at` asc (no pagination — + * the 500-row cap bounds the response). Tombstones included. With + * `options.since` a 0004 backend returns only rows stamped at or past it + * (`appliedSince` echoes the honored cursor); a pre-0004 backend rejects the + * signature once per endpoint and every listing degrades to full. Callers + * MUST treat `appliedSince === undefined` as a full listing regardless of + * what they requested. */ export async function listSessionComments( accessToken: string, orgId: string, - sessionId: string + sessionId: string, + options?: { since?: string } ): Promise { + const endpointUrl = getCloudEndpoint().supabaseUrl; + const since = + options?.since !== undefined && + !commentsDeltaUnsupportedEndpoints.has(endpointUrl) + ? options.since + : undefined; + if (since !== undefined) { + try { + const payload = await callCommentRpc( + "cloud_list_session_comments", + accessToken, + { + p_org_id: orgId, + p_session_id: sessionId, + p_since: since, + } + ); + return { + ...ListCommentsResultSchema.parse(payload), + appliedSince: since, + }; + } catch (error) { + if (!isCommentsDeltaSignatureUnsupported(error)) throw error; + commentsDeltaUnsupportedEndpoints.add(endpointUrl); + } + } const payload = await callCommentRpc( "cloud_list_session_comments", accessToken, diff --git a/src/features/Org2Cloud/org2CloudReplaySignedReads.test.ts b/src/features/Org2Cloud/org2CloudReplaySignedReads.test.ts new file mode 100644 index 000000000..afa607cd2 --- /dev/null +++ b/src/features/Org2Cloud/org2CloudReplaySignedReads.test.ts @@ -0,0 +1,417 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; + +import { + ORG2_CLOUD_OFFICIAL_ANON_KEY, + ORG2_CLOUD_OFFICIAL_SUPABASE_URL, + ORG2_CLOUD_POSTGREST_SCHEMA, +} from "./config"; +import type { CloudEndpoint } from "./config"; +import { + CLOUD_REPLAY_SIGN_PATH, + Org2CloudSignerError, + authorizeReplayRead, + createGuestReplayObjectReader, + downloadSignedReplayObject, + isReplayAuthorizeRpcMissing, + signReplayReadUrls, +} from "./org2CloudReplaySignedReads"; +import { Org2CloudStorageError } from "./org2CloudStorageClient"; +import { Org2CloudSyncError } from "./org2CloudSyncClient"; + +const fetchMock = vi.fn(); + +function jsonResponse(body: unknown, status = 200): Response { + return new Response(JSON.stringify(body), { + status, + headers: { "content-type": "application/json" }, + }); +} + +function bytesResponse(bytes: Uint8Array, status = 200): Response { + return new Response(new Uint8Array(bytes), { status }); +} + +function callAt(index: number): { url: string; init: RequestInit } { + const [url, init] = fetchMock.mock.calls[index] as [string, RequestInit]; + return { url, init }; +} + +function lastCall(): { url: string; init: RequestInit } { + return callAt(fetchMock.mock.calls.length - 1); +} + +function lastBody(): Record { + return JSON.parse(String(lastCall().init.body)) as Record; +} + +const ENDPOINT: CloudEndpoint = { + webOrigin: "https://app.custom.example.com", + supabaseUrl: "https://db.custom.example.com", + anonKey: "custom-anon", + isOfficial: false, +}; + +const SHARE_TOKEN = "t".repeat(64); + +beforeEach(() => { + vi.stubGlobal("fetch", fetchMock); +}); + +afterEach(() => { + vi.unstubAllGlobals(); + vi.useRealTimers(); + fetchMock.mockReset(); +}); + +describe("authorizeReplayRead", () => { + it("posts the share token with the anon apikey and NO Authorization header", async () => { + fetchMock.mockResolvedValueOnce( + jsonResponse({ + grant: "grant-1", + expiresAt: "2026-07-24T12:00:00.000Z", + objects: ["org-1/sess-1/2/1-h1.gz"], + }) + ); + + const grant = await authorizeReplayRead( + "org-1", + "sess-1", + SHARE_TOKEN, + ENDPOINT + ); + + const { url, init } = lastCall(); + expect(url).toBe( + `${ENDPOINT.supabaseUrl}/rest/v1/rpc/cloud_authorize_replay_read` + ); + const headers = init.headers as Record; + expect(headers.apikey).toBe("custom-anon"); + expect(headers).not.toHaveProperty("authorization"); + expect(headers["content-profile"]).toBe(ORG2_CLOUD_POSTGREST_SCHEMA); + expect(lastBody()).toEqual({ + p_org_id: "org-1", + p_session_id: "sess-1", + p_share_token: SHARE_TOKEN, + }); + expect(grant).toEqual({ + grant: "grant-1", + expiresAt: "2026-07-24T12:00:00.000Z", + objects: ["org-1/sess-1/2/1-h1.gz"], + }); + }); + + it("defaults to the official endpoint", async () => { + fetchMock.mockResolvedValueOnce(jsonResponse({ grant: "grant-1" })); + await authorizeReplayRead("org-1", "sess-1", SHARE_TOKEN); + const { url, init } = lastCall(); + expect(url).toBe( + `${ORG2_CLOUD_OFFICIAL_SUPABASE_URL}/rest/v1/rpc/cloud_authorize_replay_read` + ); + expect((init.headers as Record).apikey).toBe( + ORG2_CLOUD_OFFICIAL_ANON_KEY + ); + }); + + it("maps a rejection into Org2CloudSyncError carrying the server message", async () => { + fetchMock.mockResolvedValueOnce( + jsonResponse({ message: "ORG2_FORBIDDEN" }, 403) + ); + const error = await authorizeReplayRead( + "org-1", + "sess-1", + SHARE_TOKEN, + ENDPOINT + ).catch((caught: unknown) => caught); + expect(error).toBeInstanceOf(Org2CloudSyncError); + expect((error as Org2CloudSyncError).status).toBe(403); + expect((error as Org2CloudSyncError).code).toBe("ORG2_FORBIDDEN"); + }); +}); + +describe("isReplayAuthorizeRpcMissing", () => { + it("matches only the PGRST202-style missing-function 404", () => { + expect( + isReplayAuthorizeRpcMissing( + new Org2CloudSyncError( + "Could not find the function org2_cloud.cloud_authorize_replay_read", + 404 + ) + ) + ).toBe(true); + expect( + isReplayAuthorizeRpcMissing(new Org2CloudSyncError("ORG2_FORBIDDEN", 403)) + ).toBe(false); + expect( + isReplayAuthorizeRpcMissing(new Org2CloudSyncError("not found", 404)) + ).toBe(false); + expect(isReplayAuthorizeRpcMissing(new Error("network down"))).toBe(false); + }); +}); + +describe("signReplayReadUrls", () => { + it("posts the grant to the web origin's signer route", async () => { + fetchMock.mockResolvedValueOnce( + jsonResponse({ + urls: { "org-1/sess-1/2/1-h1.gz": "https://signed.example/1" }, + expiresIn: 600, + }) + ); + + const signed = await signReplayReadUrls("grant-1", ENDPOINT); + + const { url } = lastCall(); + expect(url).toBe(`${ENDPOINT.webOrigin}${CLOUD_REPLAY_SIGN_PATH}`); + expect(lastBody()).toEqual({ grant: "grant-1" }); + expect(signed).toEqual({ + urls: { "org-1/sess-1/2/1-h1.gz": "https://signed.example/1" }, + expiresIn: 600, + }); + }); + + it("throws Org2CloudStorageError on a signer rejection", async () => { + fetchMock.mockResolvedValueOnce(jsonResponse({ error: "expired" }, 401)); + const error = await signReplayReadUrls("grant-1", ENDPOINT).catch( + (caught: unknown) => caught + ); + expect(error).toBeInstanceOf(Org2CloudStorageError); + expect((error as Org2CloudStorageError).status).toBe(401); + }); + + it("retries ONCE after a short backoff when the POST fails at network level", async () => { + vi.useFakeTimers(); + fetchMock + .mockRejectedValueOnce(new TypeError("Load failed")) + .mockRejectedValueOnce(new TypeError("Load failed")) + .mockResolvedValueOnce( + jsonResponse({ urls: { p: "https://signed.example/1" }, expiresIn: 60 }) + ); + + const pending = signReplayReadUrls("grant-1", ENDPOINT); + await vi.advanceTimersByTimeAsync(300); + const signed = await pending; + + expect(signed.urls).toEqual({ p: "https://signed.example/1" }); + // 2 transport-level attempts inside the first fetchWithTransportRetry, + // then the single backoff retry. + expect(fetchMock).toHaveBeenCalledTimes(3); + }); + + it("maps exhausted network retries to Org2CloudSignerError 'unreachable'", async () => { + vi.useFakeTimers(); + fetchMock.mockRejectedValue(new TypeError("Load failed")); + + const pending = signReplayReadUrls("grant-1", ENDPOINT).catch( + (caught: unknown) => caught + ); + await vi.advanceTimersByTimeAsync(300); + const error = await pending; + + expect(error).toBeInstanceOf(Org2CloudSignerError); + expect((error as Org2CloudSignerError).code).toBe("unreachable"); + expect((error as Org2CloudSignerError).status).toBeNull(); + expect(fetchMock).toHaveBeenCalledTimes(4); + }); + + it("never retries an HTTP rejection: 401 maps to 'unauthorized' in one attempt", async () => { + fetchMock.mockResolvedValueOnce(jsonResponse({ error: "bad grant" }, 401)); + + const error = await signReplayReadUrls("grant-1", ENDPOINT).catch( + (caught: unknown) => caught + ); + + expect(error).toBeInstanceOf(Org2CloudSignerError); + expect((error as Org2CloudSignerError).code).toBe("unauthorized"); + expect((error as Org2CloudSignerError).status).toBe(401); + expect(fetchMock).toHaveBeenCalledTimes(1); + }); + + it("classifies an expired-grant rejection as 'expired'", async () => { + fetchMock.mockResolvedValueOnce(jsonResponse({ error: "expired" }, 401)); + const error = await signReplayReadUrls("grant-1", ENDPOINT).catch( + (caught: unknown) => caught + ); + expect(error).toBeInstanceOf(Org2CloudSignerError); + expect((error as Org2CloudSignerError).code).toBe("expired"); + expect((error as Org2CloudSignerError).message).toContain( + "replay sign request failed with 401" + ); + }); + + it("propagates the original transport error without retrying when aborted", async () => { + const controller = new AbortController(); + const transportError = new TypeError("Load failed"); + fetchMock.mockImplementationOnce(() => { + controller.abort(); + return Promise.reject(transportError); + }); + + const error = await signReplayReadUrls( + "grant-1", + ENDPOINT, + controller.signal + ).catch((caught: unknown) => caught); + + expect(error).toBe(transportError); + expect(fetchMock).toHaveBeenCalledTimes(1); + }); +}); + +describe("downloadSignedReplayObject", () => { + it("GETs the raw bytes with no extra credentials", async () => { + fetchMock.mockResolvedValueOnce(bytesResponse(new Uint8Array([1, 2, 3]))); + const bytes = await downloadSignedReplayObject("https://signed.example/1"); + expect(bytes).toEqual(new Uint8Array([1, 2, 3])); + const { url, init } = lastCall(); + expect(url).toBe("https://signed.example/1"); + expect(init.headers).toBeUndefined(); + }); + + it("throws Org2CloudStorageError with the HTTP status on failure", async () => { + fetchMock.mockResolvedValueOnce(bytesResponse(new Uint8Array(), 403)); + const error = await downloadSignedReplayObject( + "https://signed.example/1" + ).catch((caught: unknown) => caught); + expect(error).toBeInstanceOf(Org2CloudStorageError); + expect((error as Org2CloudStorageError).status).toBe(403); + }); +}); + +describe("createGuestReplayObjectReader", () => { + const PATH_1 = "org-1/sess-1/2/1-h1.gz"; + const PATH_2 = "org-1/sess-1/2/2-h2.gz"; + + function makeReader() { + return createGuestReplayObjectReader({ + orgId: "org-1", + sessionId: "sess-1", + shareToken: SHARE_TOKEN, + endpoint: ENDPOINT, + }); + } + + function mockSession( + urls: Record, + options: { expiresAt?: string; expiresIn?: number } = {} + ): void { + fetchMock + .mockResolvedValueOnce( + jsonResponse({ + grant: "grant-1", + ...(options.expiresAt !== undefined + ? { expiresAt: options.expiresAt } + : {}), + objects: Object.keys(urls), + }) + ) + .mockResolvedValueOnce( + jsonResponse({ + urls, + ...(options.expiresIn !== undefined + ? { expiresIn: options.expiresIn } + : {}), + }) + ); + } + + it("authorizes+signs once and serves every object from the cached url map", async () => { + mockSession({ + [PATH_1]: "https://signed.example/1", + [PATH_2]: "https://signed.example/2", + }); + fetchMock + .mockResolvedValueOnce(bytesResponse(new Uint8Array([1]))) + .mockResolvedValueOnce(bytesResponse(new Uint8Array([2]))); + const reader = makeReader(); + + const [first, second] = await Promise.all([ + reader.download(PATH_1), + reader.download(PATH_2), + ]); + + expect(first).toEqual(new Uint8Array([1])); + expect(second).toEqual(new Uint8Array([2])); + // 1 authorize + 1 sign + 2 GETs — the concurrent walk shares one session. + expect(fetchMock).toHaveBeenCalledTimes(4); + expect(callAt(0).url).toContain("cloud_authorize_replay_read"); + expect(callAt(1).url).toContain(CLOUD_REPLAY_SIGN_PATH); + }); + + it("re-authorizes once when the reported deadline passes mid-walk", async () => { + vi.useFakeTimers(); + vi.setSystemTime(new Date("2026-07-24T10:00:00.000Z")); + mockSession( + { [PATH_1]: "https://signed.example/1" }, + { expiresAt: "2026-07-24T10:00:30.000Z" } + ); + fetchMock.mockResolvedValueOnce(bytesResponse(new Uint8Array([1]))); + const reader = makeReader(); + expect(await reader.download(PATH_1)).toEqual(new Uint8Array([1])); + + vi.setSystemTime(new Date("2026-07-24T10:01:00.000Z")); + mockSession( + { [PATH_1]: "https://signed.example/1b" }, + { expiresAt: "2026-07-24T10:01:30.000Z" } + ); + fetchMock.mockResolvedValueOnce(bytesResponse(new Uint8Array([2]))); + expect(await reader.download(PATH_1)).toEqual(new Uint8Array([2])); + expect(lastCall().url).toBe("https://signed.example/1b"); + + // The single mid-walk re-authorization is spent: a second expiry serves + // the stale map instead of looping the grant flow. + vi.setSystemTime(new Date("2026-07-24T10:02:00.000Z")); + fetchMock.mockResolvedValueOnce(bytesResponse(new Uint8Array([3]))); + expect(await reader.download(PATH_1)).toEqual(new Uint8Array([3])); + expect(lastCall().url).toBe("https://signed.example/1b"); + }); + + it("re-authorizes once when a signed URL is rejected, then retries the GET", async () => { + mockSession({ [PATH_1]: "https://signed.example/1" }); + fetchMock.mockResolvedValueOnce(bytesResponse(new Uint8Array(), 403)); + mockSession({ [PATH_1]: "https://signed.example/1b" }); + fetchMock.mockResolvedValueOnce(bytesResponse(new Uint8Array([9]))); + const reader = makeReader(); + + expect(await reader.download(PATH_1)).toEqual(new Uint8Array([9])); + expect(lastCall().url).toBe("https://signed.example/1b"); + + // Second rejection after the spent re-authorization propagates. + fetchMock.mockResolvedValueOnce(bytesResponse(new Uint8Array(), 403)); + const error = await reader + .download(PATH_1) + .catch((caught: unknown) => caught); + expect(error).toBeInstanceOf(Org2CloudStorageError); + expect((error as Org2CloudStorageError).status).toBe(403); + }); + + it("fails closed when the signer never covers the requested path", async () => { + mockSession({ [PATH_1]: "https://signed.example/1" }); + mockSession({ [PATH_1]: "https://signed.example/1b" }); + const reader = makeReader(); + + const error = await reader + .download(PATH_2) + .catch((caught: unknown) => caught); + expect(error).toBeInstanceOf(Org2CloudStorageError); + expect((error as Org2CloudStorageError).message).toContain(PATH_2); + // One re-authorization was attempted for the uncovered path, then closed. + expect(fetchMock).toHaveBeenCalledTimes(4); + }); + + it("pins the missing-RPC rejection so every download fails without re-probing", async () => { + fetchMock.mockResolvedValueOnce( + jsonResponse({ message: "Could not find the function" }, 404) + ); + const reader = makeReader(); + + const first = await reader + .download(PATH_1) + .catch((caught: unknown) => caught); + expect(isReplayAuthorizeRpcMissing(first)).toBe(true); + + const second = await reader + .download(PATH_2) + .catch((caught: unknown) => caught); + expect(isReplayAuthorizeRpcMissing(second)).toBe(true); + expect(fetchMock).toHaveBeenCalledTimes(1); + }); +}); diff --git a/src/features/Org2Cloud/org2CloudReplaySignedReads.ts b/src/features/Org2Cloud/org2CloudReplaySignedReads.ts new file mode 100644 index 000000000..510e95a5d --- /dev/null +++ b/src/features/Org2Cloud/org2CloudReplaySignedReads.ts @@ -0,0 +1,357 @@ +/** + * Share-token signed reads for storage-offloaded replay segments. + * + * Org members download `replay` bucket objects directly with their JWT + * (org2CloudStorageClient) — storage RLS delegates to the session read + * ladder. A share-token guest is not a member and cannot pass that RLS, so + * guest reads take a three-leg path instead: + * + * 1. `cloud_authorize_replay_read(p_org_id, p_session_id, p_share_token)` — + * anon-key RPC (no Authorization header; the share token is the only + * credential, mirroring the registered-link tier) → `{grant, expiresAt, + * objects}`. + * 2. POST the grant to the deployment web app's signer route + * (`{webOrigin}/api/replay/sign`) → `{urls: {storagePath → signedUrl}, + * expiresIn}`. + * 3. Plain GET per signed URL — the URL is self-authorizing. + * + * `createGuestReplayObjectReader` caches the signed-url map for the reader's + * lifetime (one import walk) and re-authorizes at most once when the grant + * or URLs expire mid-walk. A backend without the authorize RPC surfaces the + * PGRST202-style rejection (`isReplayAuthorizeRpcMissing`) so the caller can + * fall back to the member download and its pre-0006 failure mode. + */ +import { z } from "zod/v4"; + +import { + type CloudEndpoint, + ORG2_CLOUD_POSTGREST_SCHEMA, + getCloudEndpoint, +} from "./config"; +import { + fetchWithTransportRetry, + isFetchTransportError, +} from "./org2CloudFetchRetry"; +import { Org2CloudStorageError } from "./org2CloudStorageClient"; +import { Org2CloudSyncError } from "./org2CloudSyncClient"; + +export const CLOUD_REPLAY_SIGN_PATH = "/api/replay/sign"; + +const SIGN_RETRY_BACKOFF_MS = 300; + +export type Org2CloudSignerErrorCode = + | "unreachable" + | "unauthorized" + | "expired"; + +/** + * Signer-route failure with a UI-mappable code. Extends the storage error so + * existing status-based handling keeps working; `code` is what the import + * dialog keys its user-facing copy on. + */ +export class Org2CloudSignerError extends Org2CloudStorageError { + readonly code: Org2CloudSignerErrorCode; + + constructor( + message: string, + code: Org2CloudSignerErrorCode, + status: number | null = null + ) { + super(message, status); + this.name = "Org2CloudSignerError"; + this.code = code; + } +} + +function signerRejectionCode( + status: number, + payload: unknown +): Org2CloudSignerErrorCode { + const detail = + payload && typeof payload === "object" + ? String( + (payload as { error?: unknown }).error ?? + (payload as { message?: unknown }).message ?? + "" + ) + : ""; + if (status === 410 || /expired/i.test(detail)) return "expired"; + if (status === 401 || status === 403) return "unauthorized"; + return "unreachable"; +} + +// Trailing .optional() keeps the inferred keys optional (`expiresAt?:`) — +// the protocol.ts idiom — so plain object literals stay assignable. +const CloudReplayReadGrantSchema = z.object({ + grant: z.string(), + expiresAt: z + .string() + .nullish() + .transform((value) => value ?? undefined) + .optional(), + objects: z.array(z.string()).default([]), +}); + +export type CloudReplayReadGrant = z.output; + +const CloudReplaySignedUrlsSchema = z.object({ + urls: z.record(z.string(), z.string()).default({}), + expiresIn: z + .number() + .nullish() + .transform((value) => value ?? undefined) + .optional(), +}); + +export type CloudReplaySignedUrls = z.output< + typeof CloudReplaySignedUrlsSchema +>; + +/** The authorize RPC does not exist on this backend (pre-0006 signer). */ +export function isReplayAuthorizeRpcMissing(error: unknown): boolean { + return ( + error instanceof Org2CloudSyncError && + error.status === 404 && + /could not find the function/i.test(error.message) + ); +} + +/** + * Registered-link tier signed-read authorization. Anon key only — no + * Authorization header and no anon-as-bearer; the share token in the body is + * the sole capability, so a guest without org membership (or any JWT at all) + * can call it. + */ +export async function authorizeReplayRead( + orgId: string, + sessionId: string, + shareToken: string, + endpoint: CloudEndpoint = getCloudEndpoint(), + signal?: AbortSignal +): Promise { + const response = await fetchWithTransportRetry( + `${endpoint.supabaseUrl}/rest/v1/rpc/cloud_authorize_replay_read`, + { + method: "POST", + headers: { + apikey: endpoint.anonKey, + "content-type": "application/json", + "content-profile": ORG2_CLOUD_POSTGREST_SCHEMA, + }, + body: JSON.stringify({ + p_org_id: orgId, + p_session_id: sessionId, + p_share_token: shareToken, + }), + signal, + } + ); + const text = await response.text(); + let payload: unknown = null; + try { + payload = text ? JSON.parse(text) : null; + } catch { + payload = null; + } + if (!response.ok) { + const message = + payload && typeof payload === "object" && "message" in payload + ? String((payload as { message: unknown }).message) + : `org2_cloud rpc cloud_authorize_replay_read failed with ${response.status}`; + throw new Org2CloudSyncError(message, response.status); + } + return CloudReplayReadGrantSchema.parse(payload); +} + +/** + * Exchange a read grant for the signed-url map at the deployment's signer + * route. A network-level failure of the POST gets ONE extra retry after a + * short backoff (on top of fetchWithTransportRetry's immediate dead-socket + * retry); an HTTP rejection — 401 included — is never retried and maps to a + * coded Org2CloudSignerError. + */ +export async function signReplayReadUrls( + grant: string, + endpoint: CloudEndpoint = getCloudEndpoint(), + signal?: AbortSignal +): Promise { + const url = new URL(CLOUD_REPLAY_SIGN_PATH, endpoint.webOrigin).toString(); + const init: RequestInit = { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ grant }), + signal, + }; + let response: Response; + try { + response = await fetchWithTransportRetry(url, init); + } catch (error) { + if (!isFetchTransportError(error) || signal?.aborted) throw error; + await new Promise((resolve) => setTimeout(resolve, SIGN_RETRY_BACKOFF_MS)); + if (signal?.aborted) throw error; + try { + response = await fetchWithTransportRetry(url, init); + } catch (retryError) { + if (!isFetchTransportError(retryError)) throw retryError; + throw new Org2CloudSignerError( + "replay signer unreachable after retry", + "unreachable" + ); + } + } + const text = await response.text(); + let payload: unknown = null; + try { + payload = text ? JSON.parse(text) : null; + } catch { + payload = null; + } + if (!response.ok) { + throw new Org2CloudSignerError( + `replay sign request failed with ${response.status}`, + signerRejectionCode(response.status, payload), + response.status + ); + } + return CloudReplaySignedUrlsSchema.parse(payload); +} + +/** Plain GET of one signed replay object (the URL is self-authorizing). */ +export async function downloadSignedReplayObject( + url: string, + signal?: AbortSignal +): Promise { + const response = await fetchWithTransportRetry(url, { + method: "GET", + signal, + }); + if (!response.ok) { + throw new Org2CloudStorageError( + `signed replay object download failed with ${response.status}`, + response.status + ); + } + return new Uint8Array(await response.arrayBuffer()); +} + +interface SignedReadSession { + urls: Record; + /** Epoch ms after which the grant/URLs must not be trusted; null = unknown. */ + expiresAtMs: number | null; +} + +/** A signed-URL rejection class that a fresh grant can plausibly repair. */ +function isSignedUrlRejected(error: unknown): boolean { + return ( + error instanceof Org2CloudStorageError && + error.status !== null && + [400, 401, 403, 410].includes(error.status) + ); +} + +export interface GuestReplayObjectReader { + download(storagePath: string, signal?: AbortSignal): Promise; +} + +/** + * Share-token reader over one session's storage-offloaded segments. The + * authorize+sign round-trip runs once (single-flight across the codec's + * concurrent segment workers) and its url map is cached for the reader's + * lifetime; expiry mid-walk — proactive via the reported deadline or + * reactive via a signed-URL rejection — re-authorizes at most once. A + * missing authorize RPC rejects every download with the original + * PGRST202-style error without further network attempts, so the caller's + * fallback decision stays cheap. + */ +export function createGuestReplayObjectReader(input: { + orgId: string; + sessionId: string; + shareToken: string; + endpoint?: CloudEndpoint; +}): GuestReplayObjectReader { + const { orgId, sessionId, shareToken, endpoint } = input; + let session: Promise | null = null; + let reauthorized = false; + let authorizeMissing: Org2CloudSyncError | null = null; + + const establishSession = async ( + signal?: AbortSignal + ): Promise => { + const authorized = await authorizeReplayRead( + orgId, + sessionId, + shareToken, + endpoint, + signal + ); + const signedAtMs = Date.now(); + const signed = await signReplayReadUrls(authorized.grant, endpoint, signal); + const deadlines = [ + authorized.expiresAt !== undefined + ? Date.parse(authorized.expiresAt) + : Number.NaN, + signed.expiresIn !== undefined + ? signedAtMs + signed.expiresIn * 1000 + : Number.NaN, + ].filter((deadline) => Number.isFinite(deadline)); + return { + urls: signed.urls, + expiresAtMs: deadlines.length > 0 ? Math.min(...deadlines) : null, + }; + }; + + const ensureSession = (signal?: AbortSignal): Promise => { + if (!session) { + session = establishSession(signal).catch((error: unknown) => { + session = null; + if (isReplayAuthorizeRpcMissing(error)) { + authorizeMissing = error as Org2CloudSyncError; + } + throw error; + }); + } + return session; + }; + + const refreshSessionOnce = ( + signal?: AbortSignal + ): Promise | null => { + if (reauthorized) return null; + reauthorized = true; + session = null; + return ensureSession(signal); + }; + + return { + async download( + storagePath: string, + signal?: AbortSignal + ): Promise { + if (authorizeMissing) throw authorizeMissing; + let current = await ensureSession(signal); + if (current.expiresAtMs !== null && Date.now() >= current.expiresAtMs) { + current = (await refreshSessionOnce(signal)) ?? current; + } + let url = current.urls[storagePath]; + if (url === undefined) { + const refreshed = await refreshSessionOnce(signal); + if (refreshed) url = refreshed.urls[storagePath]; + if (url === undefined) { + throw new Org2CloudStorageError( + `no signed url for replay object ${storagePath}` + ); + } + } + try { + return await downloadSignedReplayObject(url, signal); + } catch (error) { + if (!isSignedUrlRejected(error)) throw error; + const refreshed = await refreshSessionOnce(signal); + if (!refreshed) throw error; + const freshUrl = refreshed.urls[storagePath]; + if (freshUrl === undefined) throw error; + return downloadSignedReplayObject(freshUrl, signal); + } + }, + }; +} diff --git a/src/features/Org2Cloud/org2CloudSessionCommentsAtom.commentTransforms.ts b/src/features/Org2Cloud/org2CloudSessionCommentsAtom.commentTransforms.ts index 0f4272391..ff689f7f0 100644 --- a/src/features/Org2Cloud/org2CloudSessionCommentsAtom.commentTransforms.ts +++ b/src/features/Org2Cloud/org2CloudSessionCommentsAtom.commentTransforms.ts @@ -106,6 +106,78 @@ export function patchComment( ); } +/** Delta cursor safety overlap (mirrors the collab-state cursor discipline). */ +export const SESSION_COMMENTS_DELTA_OVERLAP_MS = 2_000; + +/** `p_since` for a delta refetch: the stored anchor minus the overlap. */ +export function sessionCommentsDeltaSince( + lastServerTime: string +): string | undefined { + const anchorMs = Date.parse(lastServerTime); + if (!Number.isFinite(anchorMs)) return undefined; + return new Date(anchorMs - SESSION_COMMENTS_DELTA_OVERLAP_MS).toISOString(); +} + +function commentStampMs(comment: CloudSessionComment): number { + let latest = 0; + for (const stamp of [ + comment.createdAt, + comment.editedAt, + comment.deletedAt, + comment.resolvedAt, + ]) { + if (stamp === undefined) continue; + const stampMs = Date.parse(stamp); + if (Number.isFinite(stampMs) && stampMs > latest) latest = stampMs; + } + return latest; +} + +/** + * Merge a DELTA listing into the cached list: per fetched row LWW on the + * row's own stamps (a local optimistic write newer than the overlap echo is + * kept), rows absent from the delta are untouched — absence proves nothing + * behind a `since` cursor. Server tombstones ride the delta with an empty + * body, so taking the fetched row drops the evicted body. An un-resolve + * clears `resolved_at` WITHOUT a new stamp and therefore cannot ride a + * delta; every force/full-invalidation path stays a full listing + * (`mergeFullSessionComments`), which reconciles it. + */ +export function mergeDeltaSessionComments( + existing: readonly CloudSessionComment[], + fetched: readonly CloudSessionComment[] +): CloudSessionComment[] { + let merged = [...existing]; + for (const comment of fetched) { + const current = merged.find((candidate) => candidate.id === comment.id); + if (current && commentStampMs(current) > commentStampMs(comment)) continue; + merged = insertComment(merged, comment); + } + return merged; +} + +/** + * Merge a FULL listing: the server snapshot is the base, preserving ONLY + * rows that appeared locally after the fetch was claimed (optimistic adds + * the snapshot predates — their id is not in `knownIdsAtStart`). A row that + * WAS known at start but is missing from the response was deleted + * server-side (e.g. GDPR erasure) and is dropped — merging it back would + * make it immortal. + */ +export function mergeFullSessionComments( + existing: readonly CloudSessionComment[], + fetched: readonly CloudSessionComment[], + knownIdsAtStart: ReadonlySet +): CloudSessionComment[] { + const fetchedIds = new Set(fetched.map((comment) => comment.id)); + return existing + .filter( + (comment) => + !fetchedIds.has(comment.id) && !knownIdsAtStart.has(comment.id) + ) + .reduce((list, comment) => insertComment(list, comment), [...fetched]); +} + /** * The atomic-claim decision, extracted pure so the force-vs-in-flight race * is testable: a FORCED refresh that lands while a fetch is in flight must diff --git a/src/features/Org2Cloud/org2CloudSessionCommentsAtom.test.ts b/src/features/Org2Cloud/org2CloudSessionCommentsAtom.test.ts index db994f9ef..92d0dc782 100644 --- a/src/features/Org2Cloud/org2CloudSessionCommentsAtom.test.ts +++ b/src/features/Org2Cloud/org2CloudSessionCommentsAtom.test.ts @@ -6,14 +6,18 @@ import type { CloudSessionComment } from "./org2CloudCommentsClient"; import { type CloudSessionCommentsEntry, MAX_SESSION_COMMENT_CACHE_ENTRIES, + SESSION_COMMENTS_DELTA_OVERLAP_MS, countLiveComments, decideSessionCommentsFetch, getThreadResolution, groupCommentThreads, insertComment, isThreadResolved, + mergeDeltaSessionComments, + mergeFullSessionComments, mergePresentEventIdEntries, patchComment, + sessionCommentsDeltaSince, sessionCommentsEntryForIdentity, shouldEvictSessionCommentsOnError, writeSessionCommentsEntry, @@ -338,6 +342,140 @@ describe("insertComment / patchComment", () => { }); }); +describe("sessionCommentsDeltaSince", () => { + it("subtracts the safety overlap from the stored anchor", () => { + expect(sessionCommentsDeltaSince("2026-07-24T10:00:02.000Z")).toBe( + new Date( + Date.parse("2026-07-24T10:00:02.000Z") - + SESSION_COMMENTS_DELTA_OVERLAP_MS + ).toISOString() + ); + }); + + it("degrades an unparseable anchor to a full listing (undefined)", () => { + expect(sessionCommentsDeltaSince("not-a-time")).toBeUndefined(); + }); +}); + +describe("mergeDeltaSessionComments", () => { + it("upserts stamped delta rows and keeps rows absent from the delta", () => { + const existing = [ + comment({ id: "a", createdAt: "2026-07-24T01:00:00Z" }), + comment({ id: "b", createdAt: "2026-07-24T02:00:00Z" }), + ]; + const merged = mergeDeltaSessionComments(existing, [ + comment({ + id: "b", + createdAt: "2026-07-24T02:00:00Z", + editedAt: "2026-07-24T03:00:00Z", + body: "edited remotely", + }), + comment({ id: "c", createdAt: "2026-07-24T04:00:00Z" }), + ]); + expect(merged.map((entry) => entry.id)).toEqual(["a", "b", "c"]); + // `a` was not in the delta: absence behind a cursor proves nothing. + expect(merged[0]).toEqual(existing[0]); + expect(merged[1].body).toBe("edited remotely"); + }); + + it("LWW on the row's own stamps: a newer local write beats the overlap echo", () => { + const localEdit = comment({ + id: "a", + createdAt: "2026-07-24T01:00:00Z", + editedAt: "2026-07-24T05:00:00Z", + body: "local newer", + }); + const merged = mergeDeltaSessionComments( + [localEdit], + [ + comment({ + id: "a", + createdAt: "2026-07-24T01:00:00Z", + editedAt: "2026-07-24T04:00:00Z", + body: "stale echo", + }), + ] + ); + expect(merged).toEqual([localEdit]); + }); + + it("takes the fetched row on equal stamps (server echo is authoritative)", () => { + const merged = mergeDeltaSessionComments( + [comment({ id: "a", createdAt: "2026-07-24T01:00:00Z", body: "local" })], + [comment({ id: "a", createdAt: "2026-07-24T01:00:00Z", body: "server" })] + ); + expect(merged[0].body).toBe("server"); + }); + + it("delta tombstones drop the cached body", () => { + const merged = mergeDeltaSessionComments( + [comment({ id: "a", createdAt: "2026-07-24T01:00:00Z" })], + [ + comment({ + id: "a", + createdAt: "2026-07-24T01:00:00Z", + deletedAt: "2026-07-24T02:00:00Z", + body: "", + }), + ] + ); + expect(merged[0].body).toBe(""); + expect(merged[0].deletedAt).toBe("2026-07-24T02:00:00Z"); + }); + + it("cannot observe a stamp-free un-resolve (the documented delta gap)", () => { + const resolvedLocally = comment({ + id: "a", + createdAt: "2026-07-24T01:00:00Z", + resolvedAt: "2026-07-24T03:00:00Z", + }); + // The server row after un-resolve carries no stamp newer than createdAt, + // so LWW keeps the locally cached resolved state — full listings own + // this reconciliation. + const merged = mergeDeltaSessionComments( + [resolvedLocally], + [comment({ id: "a", createdAt: "2026-07-24T01:00:00Z" })] + ); + expect(merged[0].resolvedAt).toBe("2026-07-24T03:00:00Z"); + }); +}); + +describe("mergeFullSessionComments", () => { + it("reconciles an un-resolve: the snapshot row replaces the resolved cache", () => { + const resolvedLocally = comment({ + id: "a", + createdAt: "2026-07-24T01:00:00Z", + resolvedAt: "2026-07-24T03:00:00Z", + }); + const unresolvedOnServer = comment({ + id: "a", + createdAt: "2026-07-24T01:00:00Z", + }); + const merged = mergeFullSessionComments( + [resolvedLocally], + [unresolvedOnServer], + new Set(["a"]) + ); + expect(merged).toEqual([unresolvedOnServer]); + expect(merged[0].resolvedAt).toBeUndefined(); + }); + + it("preserves optimistic adds the snapshot predates, drops server-deleted rows", () => { + const optimistic = comment({ + id: "new", + createdAt: "2026-07-24T05:00:00Z", + }); + const erased = comment({ id: "gone", createdAt: "2026-07-24T01:00:00Z" }); + const kept = comment({ id: "kept", createdAt: "2026-07-24T02:00:00Z" }); + const merged = mergeFullSessionComments( + [erased, kept, optimistic], + [kept], + new Set(["gone", "kept"]) + ); + expect(merged.map((entry) => entry.id)).toEqual(["kept", "new"]); + }); +}); + describe("sessionCommentsKey", () => { it("namespaces org and session", () => { expect(sessionCommentsKey("org-1", "sess-1")).toBe("org-1|sess-1"); diff --git a/src/features/Org2Cloud/org2CloudSessionCommentsAtom.ts b/src/features/Org2Cloud/org2CloudSessionCommentsAtom.ts index 365eec029..8168dc507 100644 --- a/src/features/Org2Cloud/org2CloudSessionCommentsAtom.ts +++ b/src/features/Org2Cloud/org2CloudSessionCommentsAtom.ts @@ -39,7 +39,10 @@ import { EMPTY_ENTRY, decideSessionCommentsFetch, insertComment, + mergeDeltaSessionComments, + mergeFullSessionComments, patchComment, + sessionCommentsDeltaSince, sessionCommentsEntryForIdentity, sessionCommentsErrorRetryDelayMs, shouldEvictSessionCommentsOnError, @@ -75,8 +78,10 @@ export type { } from "./org2CloudSessionCommentsAtom.types"; export { MAX_SESSION_COMMENT_CACHE_ENTRIES, + SESSION_COMMENTS_DELTA_OVERLAP_MS, SESSION_COMMENTS_ERROR_RETRY_MS, SESSION_COMMENTS_ERROR_RETRY_MAX_MS, + sessionCommentsDeltaSince, sessionCommentsErrorRetryDelayMs, countLiveComments, decideSessionCommentsFetch, @@ -84,6 +89,8 @@ export { groupCommentThreads, insertComment, isThreadResolved, + mergeDeltaSessionComments, + mergeFullSessionComments, mergePresentEventIdEntries, patchComment, sessionCommentsEntryForIdentity, @@ -165,6 +172,7 @@ export function useSessionComments( let claimed = false; let queuedForce = false; let knownIdsAtStart = new Set(); + let anchorAtStart: string | undefined; setEntries((previous) => { const stored = previous[targetKey]; const entry = @@ -182,6 +190,7 @@ export function useSessionComments( knownIdsAtStart = new Set( (entry?.comments ?? []).map((comment) => comment.id) ); + anchorAtStart = entry?.lastServerTime; return writeSessionCommentsEntry(previous, targetKey, { ...(entry ?? EMPTY_ENTRY), identityKey: requestIdentityKey, @@ -209,10 +218,19 @@ export function useSessionComments( dropPendingForce(requestKey); return; } - const { comments, viewerOwnsSession } = await listSessionComments( + // Force paths (manual refresh, session-signal tokens, the + // SUBSCRIBED-edge recovery) stay FULL listings — a full snapshot + // is the only read that reconciles the stamp-free un-resolve. + // TTL/org-signal refetches pull the delta behind the stored anchor. + const since = + !force && anchorAtStart !== undefined + ? sessionCommentsDeltaSince(anchorAtStart) + : undefined; + const listing = await listSessionComments( accessToken, targetOrgId, - targetSessionId + targetSessionId, + since !== undefined ? { since } : undefined ); const latestAfterFetch = authRef.current; if ( @@ -222,12 +240,10 @@ export function useSessionComments( dropPendingForce(requestKey); return; } - // MERGE, not wholesale-replace: preserve ONLY rows that appeared - // locally AFTER the fetch was claimed (optimistic adds the server - // snapshot predates — their id is not in knownIdsAtStart). A row - // that WAS known at start but is missing from the response was - // deleted server-side (e.g. GDPR erasure) and is dropped — merging - // it back would make it immortal. + // MERGE, not wholesale-replace — see mergeFullSessionComments / + // mergeDeltaSessionComments for the two paths' invariants. The + // fallback-aware `appliedSince` (not the requested `since`) picks + // the path: a pre-delta backend answers every request in full. setEntries((previous) => { const latestAuth = authRef.current; if ( @@ -237,25 +253,26 @@ export function useSessionComments( return previous; } const stored = previous[targetKey]; - const existing = - stored?.identityKey === requestIdentityKey ? stored.comments : []; - const fetchedIds = new Set(comments.map((comment) => comment.id)); - const merged = existing - .filter( - (comment) => - !fetchedIds.has(comment.id) && - !knownIdsAtStart.has(comment.id) - ) - .reduce( - (list, comment) => insertComment(list, comment), - comments - ); + const sameIdentity = stored?.identityKey === requestIdentityKey; + const existing = sameIdentity ? stored.comments : []; + const merged = + listing.appliedSince !== undefined + ? mergeDeltaSessionComments(existing, listing.comments) + : mergeFullSessionComments( + existing, + listing.comments, + knownIdsAtStart + ); + const lastServerTime = + listing.serverTime ?? + (sameIdentity ? stored.lastServerTime : undefined); return writeSessionCommentsEntry(previous, targetKey, { identityKey: requestIdentityKey, comments: merged, - viewerOwnsSession, + viewerOwnsSession: listing.viewerOwnsSession, state: "ready", fetchedAt: Date.now(), + ...(lastServerTime !== undefined ? { lastServerTime } : {}), }); }); } catch (error) { diff --git a/src/features/Org2Cloud/org2CloudSessionCommentsAtom.types.ts b/src/features/Org2Cloud/org2CloudSessionCommentsAtom.types.ts index 22bed75b8..847c457a5 100644 --- a/src/features/Org2Cloud/org2CloudSessionCommentsAtom.types.ts +++ b/src/features/Org2Cloud/org2CloudSessionCommentsAtom.types.ts @@ -28,6 +28,12 @@ export interface CloudSessionCommentsEntry { consecutiveFailures?: number; /** Epoch ms of the last completed fetch attempt (0 ⇒ never fetched). */ fetchedAt: number; + /** + * The last listing's 0004 `serverTime` anchor. TTL refetches pull the + * delta behind it (minus the safety overlap); absent — legacy backend or + * never fetched — every listing stays full. + */ + lastServerTime?: string; } export type SessionCommentsFetchDecision = "claim" | "skip" | "queue_force"; diff --git a/src/features/Org2Cloud/org2CloudSyncEngine.constants.ts b/src/features/Org2Cloud/org2CloudSyncEngine.constants.ts index 542191c7c..520ffdd09 100644 --- a/src/features/Org2Cloud/org2CloudSyncEngine.constants.ts +++ b/src/features/Org2Cloud/org2CloudSyncEngine.constants.ts @@ -14,6 +14,18 @@ export const SCOPE_HYDRATE_THROTTLE_MS = 10 * 60_000; export const SCHEMA_MISMATCH_REPROBE_MS = 5 * 60_000; /** Collab-state delta cursor safety overlap (mirrors CollabSyncEngine §9.4). */ export const CURSOR_OVERLAP_MS = 2_000; +/** + * One signal recovery fans out into several serialized passes (inbound + * invalidation, entitlement resume, apply-echo `orgii-data-changed`), each + * pulling `cloud_list_org_collab_state` for the same org within about a + * second. A listing completed inside this window is shared with a later pass + * whose request it covers (a full listing covers everything; a delta covers + * any request whose cursor is at or past its own), so a burst costs one + * network listing without touching delta-cursor semantics or dropping an + * invalidation: the cursor stays anchored on the shared listing's serverTime, + * so the next real pull still covers the window. + */ +export const COLLAB_LISTING_SHARE_WINDOW_MS = 2_000; /** * Vanished-session GC cadence per org. The sweep's suspect set is dominated diff --git a/src/features/Org2Cloud/org2CloudSyncEngine.projects.test.ts b/src/features/Org2Cloud/org2CloudSyncEngine.projects.test.ts index f3370720b..726e42e8a 100644 --- a/src/features/Org2Cloud/org2CloudSyncEngine.projects.test.ts +++ b/src/features/Org2Cloud/org2CloudSyncEngine.projects.test.ts @@ -14,6 +14,7 @@ import { import type { EngineFixture } from "./org2CloudSyncEngine.testUtils"; const { + COLLAB_LISTING_SHARE_WINDOW_MS, DATA_CHANGED_DEBOUNCE_MS, ORG2_CLOUD_ENDPOINT_OVERRIDE_STORAGE_KEY, ORG2_CLOUD_EXPECTED_SCHEMA_VERSION, @@ -105,7 +106,9 @@ describe("Org2CloudSyncEngine project and endpoint synchronization", () => { expect(store.get(org2CloudCollabStateCursorsAtom)).toEqual({ "corg-1": "2026-07-01T11:59:58.000Z", }); - // … and the next concrete Realtime invalidation pulls the delta behind it. + // … and the next concrete Realtime invalidation (past the burst share + // window) pulls the delta behind it. + await vi.advanceTimersByTimeAsync(COLLAB_LISTING_SHARE_WINDOW_MS); await engine.invalidateOrgInboundAndWait("corg-1"); expect(projectsClient.listOrgCollabState).toHaveBeenLastCalledWith( "jwt-1", @@ -137,6 +140,7 @@ describe("Org2CloudSyncEngine project and endpoint synchronization", () => { ]); await engine.runSyncPass(); + await vi.advanceTimersByTimeAsync(COLLAB_LISTING_SHARE_WINDOW_MS); projectsClient.listOrgCollabState.mockClear(); bridge.drainOutbox.mockClear(); @@ -160,7 +164,7 @@ describe("Org2CloudSyncEngine project and endpoint synchronization", () => { documentStub.visibilityState = "hidden"; engine.invalidateOrgInbound("corg-1"); - await vi.advanceTimersByTimeAsync(1_000); + await vi.advanceTimersByTimeAsync(COLLAB_LISTING_SHARE_WINDOW_MS); expect(projectsClient.listOrgCollabState).not.toHaveBeenCalled(); documentStub.visibilityState = "visible"; @@ -196,6 +200,7 @@ describe("Org2CloudSyncEngine project and endpoint synchronization", () => { it("applies snake_case tombstones from Realtime-scoped project pulls", async () => { await engine.runSyncPass(); + await vi.advanceTimersByTimeAsync(COLLAB_LISTING_SHARE_WINDOW_MS); projectsClient.listOrgCollabState.mockClear(); bridge.applyRemote.mockClear(); bridge.drainOutbox.mockClear(); @@ -237,6 +242,7 @@ describe("Org2CloudSyncEngine project and endpoint synchronization", () => { it("uses a full listing only for reconnect recovery", async () => { await engine.runSyncPass(); + await vi.advanceTimersByTimeAsync(COLLAB_LISTING_SHARE_WINDOW_MS); projectsClient.listOrgCollabState.mockClear(); await engine.invalidateOrgInboundAndWait("corg-1", { full: true }); @@ -250,6 +256,7 @@ describe("Org2CloudSyncEngine project and endpoint synchronization", () => { it("resumeOrgAndWait runs exactly one serialized pass (no dirty follow-up)", async () => { await engine.runSyncPass(); + await vi.advanceTimersByTimeAsync(COLLAB_LISTING_SHARE_WINDOW_MS); projectsClient.listOrgCollabState.mockClear(); const passesBefore = engine.startedPassCount; @@ -264,6 +271,67 @@ describe("Org2CloudSyncEngine project and endpoint synchronization", () => { ); }); + it("collapses a signal-recovery listing burst to one network pull per org", async () => { + await engine.runSyncPass(); + await vi.advanceTimersByTimeAsync(COLLAB_LISTING_SHARE_WINDOW_MS); + projectsClient.listOrgCollabState.mockClear(); + + // Real-machine burst: the SUBSCRIBED-edge full recovery, the entitlement + // resume and the delta nudge all land within a second. + await engine.invalidateOrgInboundAndWait("corg-1", { + full: true, + pushSessions: true, + }); + await engine.resumeOrgAndWait("corg-1"); + await engine.invalidateOrgInboundAndWait("corg-1"); + + expect(projectsClient.listOrgCollabState).toHaveBeenCalledTimes(1); + expect(projectsClient.listOrgCollabState).toHaveBeenCalledWith( + "jwt-1", + "corg-1", + undefined + ); + // The burst's one listing still anchored the delta cursor. + expect(store.get(org2CloudCollabStateCursorsAtom)).toEqual({ + "corg-1": "2026-07-01T11:59:58.000Z", + }); + + // No invalidation intent was dropped: the next spaced trigger issues a + // REAL delta pull behind the anchored cursor. + await vi.advanceTimersByTimeAsync(COLLAB_LISTING_SHARE_WINDOW_MS); + projectsClient.listOrgCollabState.mockClear(); + await engine.invalidateOrgInboundAndWait("corg-1"); + expect(projectsClient.listOrgCollabState).toHaveBeenCalledTimes(1); + expect(projectsClient.listOrgCollabState).toHaveBeenCalledWith( + "jwt-1", + "corg-1", + "2026-07-01T11:59:58.000Z" + ); + }); + + it("never satisfies a full-recovery request with a cached delta listing", async () => { + await engine.runSyncPass(); + await vi.advanceTimersByTimeAsync(COLLAB_LISTING_SHARE_WINDOW_MS); + projectsClient.listOrgCollabState.mockClear(); + + // Delta pull first … + await engine.invalidateOrgInboundAndWait("corg-1"); + expect(projectsClient.listOrgCollabState).toHaveBeenLastCalledWith( + "jwt-1", + "corg-1", + "2026-07-01T11:59:58.000Z" + ); + // … then a full recovery inside the window: absence can only be proven + // against the complete state, so the delta must NOT be shared. + await engine.invalidateOrgInboundAndWait("corg-1", { full: true }); + expect(projectsClient.listOrgCollabState).toHaveBeenCalledTimes(2); + expect(projectsClient.listOrgCollabState).toHaveBeenLastCalledWith( + "jwt-1", + "corg-1", + undefined + ); + }); + it("syncs the project plane even for an org with no scopes or tagged sessions", async () => { store.set(org2CloudRepoScopesAtom, {}); await engine.runSyncPass(); @@ -292,6 +360,7 @@ describe("Org2CloudSyncEngine project and endpoint synchronization", () => { // schedules and drains the independent projects/work-items control plane. await vi.advanceTimersByTimeAsync(0); await engine.runSyncPassAndWaitForDrain(); + await vi.advanceTimersByTimeAsync(COLLAB_LISTING_SHARE_WINDOW_MS); client.rewriteSessionEvents.mockClear(); projectsClient.listOrgCollabState.mockClear(); bridge.drainOutbox.mockClear(); @@ -483,6 +552,7 @@ describe("Org2CloudSyncEngine project and endpoint synchronization", () => { ); await engine.runSyncPass(); + await vi.advanceTimersByTimeAsync(COLLAB_LISTING_SHARE_WINDOW_MS); projectsClient.listOrgCollabState.mockClear(); bridge.drainOutbox.mockClear(); await engine.invalidateOrgInboundAndWait("corg-1"); @@ -514,6 +584,7 @@ describe("Org2CloudSyncEngine project and endpoint synchronization", () => { // that callback into a fire-and-forget dirty pass under full-suite load. await vi.advanceTimersByTimeAsync(0); await engine.runSyncPassAndWaitForDrain(); + await vi.advanceTimersByTimeAsync(COLLAB_LISTING_SHARE_WINDOW_MS); projectsClient.listOrgCollabState.mockClear(); bridge.drainOutbox.mockClear(); @@ -592,6 +663,7 @@ describe("Org2CloudSyncEngine project and endpoint synchronization", () => { engine.start(store); try { await engine.runSyncPass(); + await vi.advanceTimersByTimeAsync(COLLAB_LISTING_SHARE_WINDOW_MS); projectsClient.listOrgCollabState.mockClear(); bridge.drainOutbox.mockClear(); diff --git a/src/features/Org2Cloud/org2CloudSyncEngine.projectsChannel.ts b/src/features/Org2Cloud/org2CloudSyncEngine.projectsChannel.ts index 69cc4d21c..36af5f1cb 100644 --- a/src/features/Org2Cloud/org2CloudSyncEngine.projectsChannel.ts +++ b/src/features/Org2Cloud/org2CloudSyncEngine.projectsChannel.ts @@ -17,14 +17,20 @@ import type { ProjectSyncBridge } from "../TeamCollaboration/engine/projectSyncB import type { Org2CloudAuthState } from "./org2CloudAuthAtom"; import type { Org2CloudOrg } from "./org2CloudOrgsAtom"; import { ensureProjectOrgForCloudOrg } from "./org2CloudProjectOrgAlias"; -import type { CloudProjectsRpc } from "./org2CloudProjectsClient"; +import type { + CloudOrgCollabState, + CloudProjectsRpc, +} from "./org2CloudProjectsClient"; import { createCloudProjectSyncClient, isOrg2ProjectsErrorCode, toCollabOrgState, } from "./org2CloudProjectsClient"; import { org2CloudCollabStateCursorsAtom } from "./org2CloudSyncAtoms"; -import { CURSOR_OVERLAP_MS } from "./org2CloudSyncEngine.constants"; +import { + COLLAB_LISTING_SHARE_WINDOW_MS, + CURSOR_OVERLAP_MS, +} from "./org2CloudSyncEngine.constants"; import type { CloudStore } from "./org2CloudSyncLifecycle"; const log = createLogger("Org2CloudSyncEngine"); @@ -32,11 +38,21 @@ const log = createLogger("Org2CloudSyncEngine"); /** Projects/work-items RPC seam (Phase B), same fetch-free-fakes purpose. */ export type Org2CloudProjectsClientDeps = CloudProjectsRpc; +interface RecentCollabListing { + /** Cursor the listing was pulled with; null = complete listing. */ + since: string | null; + completedAtMs: number; +} + export class Org2CloudProjectsChannel { /** Cloud orgId → aliased local project-org id (ensured once per start). */ private readonly projectOrgAliasIds = new Map(); /** Orgs whose CURRENT start already pulled one COMPLETE collab-state listing. */ private readonly fullCollabStateOrgIds = new Set(); + /** Cloud orgId → the last APPLIED listing, shared within the burst window + * (see COLLAB_LISTING_SHARE_WINDOW_MS). Recorded only after the cycle and + * cursor advance succeed, so sharing never skips an unapplied delta. */ + private readonly recentListings = new Map(); constructor( private readonly getStore: () => CloudStore | null, @@ -47,6 +63,7 @@ export class Org2CloudProjectsChannel { reset(): void { this.projectOrgAliasIds.clear(); this.fullCollabStateOrgIds.clear(); + this.recentListings.clear(); } prune(currentOrgIds: ReadonlySet): void { @@ -56,6 +73,9 @@ export class Org2CloudProjectsChannel { for (const orgId of this.fullCollabStateOrgIds) { if (!currentOrgIds.has(orgId)) this.fullCollabStateOrgIds.delete(orgId); } + for (const orgId of this.recentListings.keys()) { + if (!currentOrgIds.has(orgId)) this.recentListings.delete(orgId); + } } /** Realtime full-recovery invalidation: bypass the cursor once more for a @@ -114,11 +134,25 @@ export class Org2CloudProjectsChannel { const since = isFullListing ? undefined : store.get(org2CloudCollabStateCursorsAtom)[org.orgId]; - const state = await this.projectsClient.listOrgCollabState( - auth.accessToken, - org.orgId, - since - ); + // Burst coalescing: a listing applied moments ago that COVERS this + // request (complete covers everything; an older-or-equal cursor covers a + // newer one) is shared instead of re-pulled — its rows are already + // applied and its cursor already anchored, so the shared pass runs the + // channel with an empty delta purely for the outbox-drain intent. + const recent = this.recentListings.get(org.orgId); + const sharedListing = + recent !== undefined && + Date.now() - recent.completedAtMs < COLLAB_LISTING_SHARE_WINDOW_MS && + (since === undefined + ? recent.since === null + : recent.since === null || recent.since <= since); + const state: CloudOrgCollabState = sharedListing + ? { projects: [], workItems: [] } + : await this.projectsClient.listOrgCollabState( + auth.accessToken, + org.orgId, + since + ); if (!deps.isCurrentGeneration(generation)) return; // Same channel + Rust bridge as the retired self-hosted engine; the cloud @@ -153,6 +187,11 @@ export class Org2CloudProjectsChannel { if (cycle.pushErrors.length > 0) deps.scheduleProjectPushRetry(); this.fullCollabStateOrgIds.add(org.orgId); + if (sharedListing) return; + this.recentListings.set(org.orgId, { + since: since ?? null, + completedAtMs: Date.now(), + }); // Anchor the delta cursor on the server clock minus a safety overlap so // client skew cannot skip rows (consumers are idempotent) — the // self-hosted delta-cursor discipline. diff --git a/src/features/Org2Cloud/org2CloudSyncEngine.testUtils.ts b/src/features/Org2Cloud/org2CloudSyncEngine.testUtils.ts index aa9828bbf..c76c49e99 100644 --- a/src/features/Org2Cloud/org2CloudSyncEngine.testUtils.ts +++ b/src/features/Org2Cloud/org2CloudSyncEngine.testUtils.ts @@ -63,6 +63,7 @@ import type { } from "./org2CloudSyncClient"; import { Org2CloudSyncError } from "./org2CloudSyncClient"; import { + COLLAB_LISTING_SHARE_WINDOW_MS, DATA_CHANGED_DEBOUNCE_MS, EXTERNAL_HISTORY_ACTIVITY_DEBOUNCE_MS, INACTIVE_ORG_BACKOFF_COOLDOWN_MS, @@ -360,6 +361,7 @@ export function cleanupEngineFixture(engine: Org2CloudSyncEngine): void { * are installed before the engine and its collaborators are evaluated. */ export const engineTestDeps = { + COLLAB_LISTING_SHARE_WINDOW_MS, DATA_CHANGED_DEBOUNCE_MS, EXTERNAL_HISTORY_ACTIVITY_DEBOUNCE_MS, chatPanelSelectedCloudOrgAtom, diff --git a/src/features/Org2Cloud/org2CloudSyncEngine.ts b/src/features/Org2Cloud/org2CloudSyncEngine.ts index ca93ee192..a28838988 100644 --- a/src/features/Org2Cloud/org2CloudSyncEngine.ts +++ b/src/features/Org2Cloud/org2CloudSyncEngine.ts @@ -135,6 +135,7 @@ export { } from "./org2CloudSessionSync"; export type { Org2CloudSyncClientDeps } from "./org2CloudSessionSync"; export { + COLLAB_LISTING_SHARE_WINDOW_MS, ORG_BACKOFF_COOLDOWN_MS, INACTIVE_ORG_BACKOFF_COOLDOWN_MS, } from "./org2CloudSyncEngine.constants"; diff --git a/src/features/Org2Cloud/useOrg2CloudRealtime.ts b/src/features/Org2Cloud/useOrg2CloudRealtime.ts index d670b2431..b35b1cd12 100644 --- a/src/features/Org2Cloud/useOrg2CloudRealtime.ts +++ b/src/features/Org2Cloud/useOrg2CloudRealtime.ts @@ -16,7 +16,10 @@ * broadcasts drive the normal live path; the coarse * row is rate-limited to one recovery pull per minute * so unrelated planes cannot invalidate one another - * on every write. + * on every write. Broadcast backends carry the signal + * as per-kind `org-db-changed` events on the org + * channel instead, dispatched narrowly per plane with + * the same 60s window per plane. * * Realtime is an INVALIDATION signal only: the data still arrives through the * existing RPC pull paths, so no cursor/tombstone logic is duplicated here. The @@ -67,6 +70,7 @@ import { import { ORG_CONTROL_CHANGED_EVENT, ORG_DB_CHANGED_EVENT, + type Org2CloudDbChangeKind, parseOrgControlChangeKind, parseOrgDbChangeKind, registerOrgControlBroadcaster, @@ -126,11 +130,35 @@ const CHANGE_SIGNALS_TABLE = "org_change_signals"; * The backend's durable signal is intentionally coarse. Plane-specific * Presence broadcasts provide the live path; the durable coarse row is a * secondary event source. These windows throttle event storms—they do not - * schedule polling when no signal arrives. + * schedule polling when no signal arrives. On per-kind backends every plane + * keeps its own 60s window (one `SignalPlane` stamp/timer each) instead of + * sharing a single coarse stamp. */ const COARSE_SIGNAL_THROTTLE_MS = 60_000; const CONTROL_PLANE_REFRESH_THROTTLE_MS = 5 * 60_000; +/** + * Throttle keys for the trailing-edge signal refreshes: one per narrowed + * dispatch target ("inbound" covers projects + workItems, which run the same + * action) plus "coarse" for the legacy all-planes refresh. + */ +type SignalPlane = + | "coarse" + | "sessions" + | "comments" + | "inbound" + | "roster" + | "policy"; + +const ALL_SIGNAL_PLANES: readonly SignalPlane[] = [ + "coarse", + "sessions", + "comments", + "inbound", + "roster", + "policy", +]; + function isDocumentHidden(): boolean { return ( typeof document !== "undefined" && document.visibilityState === "hidden" @@ -243,11 +271,14 @@ export function useOrg2CloudRealtime(): void { // Stable refs so the per-org effect and status callbacks read current values // without forcing the connection to rebuild. const refetchRef = useRef(refetchOrgs); - const coarseSignalHandledAtRef = useRef(0); + const planeSignalHandledAtRef = useRef(new Map()); + const planeSignalTrailingTimersRef = useRef( + new Map>() + ); const controlPlaneRefreshAtRef = useRef(0); - const coarseSignalTrailingTimerRef = useRef | null>(null); + const coarseSafetyNetTimerRef = useRef | null>( + null + ); // Disconnect-duration bookkeeping for the SUBSCRIBED-edge recovery policy: // short gaps and rejoin storms downgrade to delta pulls, long gaps run the // authoritative full recovery (see org2CloudRealtimeRecovery). @@ -370,14 +401,38 @@ export function useOrg2CloudRealtime(): void { } }, [auth?.accessToken]); + // Low-frequency control-plane guard shared by every signal path: any + // signal arms the 5-min refetchOrgs + entitlement TTL check. + const maybeRefreshControlPlane = useCallback( + (orgId: string) => { + const now = Date.now(); + if ( + now - controlPlaneRefreshAtRef.current < + CONTROL_PLANE_REFRESH_THROTTLE_MS + ) { + return; + } + controlPlaneRefreshAtRef.current = now; + void refetchRef.current(); + void refreshEntitlementForOrg(orgId).then((floorChanged) => { + if (floorChanged) org2CloudSyncEngine.resumeOrg(orgId); + }); + }, + [refreshEntitlementForOrg] + ); + // Shared by the Slice B postgres_changes path (legacy backends) and the - // Slice C org-channel broadcast path (0005 backends): identical throttle - // windows and refresh behavior regardless of transport. + // Slice C safety net (per-kind backends): identical throttle windows and + // refresh behavior regardless of transport. const runCoarseSignalRefresh = useCallback(() => { const orgId = activeRealtimeOrgId; if (!orgId) return; const now = Date.now(); - coarseSignalHandledAtRef.current = now; + const handledAt = planeSignalHandledAtRef.current; + handledAt.set("coarse", now); + handledAt.set("sessions", now); + handledAt.set("comments", now); + handledAt.set("inbound", now); // A blur/visibility event releases the connection. Ignore the tiny // event-delivery race during teardown; the next SUBSCRIBED true-edge // performs the authoritative full recovery. @@ -385,42 +440,110 @@ export function useOrg2CloudRealtime(): void { org2CloudSyncEngine.invalidateOrgInbound(orgId); bumpRemoteSessionsVersion(orgId); bumpOrgCommentsSignal(orgId); - if ( - now - controlPlaneRefreshAtRef.current >= - CONTROL_PLANE_REFRESH_THROTTLE_MS - ) { - controlPlaneRefreshAtRef.current = now; - void refetchRef.current(); - void refreshEntitlementForOrg(orgId).then((floorChanged) => { - if (floorChanged) org2CloudSyncEngine.resumeOrg(orgId); - }); - } + maybeRefreshControlPlane(orgId); }, [ activeRealtimeOrgId, bumpRemoteSessionsVersion, bumpOrgCommentsSignal, - refreshEntitlementForOrg, + maybeRefreshControlPlane, ]); + // Per-plane trailing-edge throttler (the generalized coarse scheduler): + // leading run when the plane's window is clear, otherwise one trailing + // timer at the window's end. + const schedulePlaneSignalRefresh = useCallback( + (plane: SignalPlane, refresh: () => void) => { + const now = Date.now(); + const elapsed = now - (planeSignalHandledAtRef.current.get(plane) ?? 0); + const run = () => { + planeSignalHandledAtRef.current.set(plane, Date.now()); + if (isDocumentHidden()) return; + refresh(); + }; + if (elapsed >= COARSE_SIGNAL_THROTTLE_MS) { + run(); + return; + } + const timers = planeSignalTrailingTimersRef.current; + if (timers.has(plane)) return; + timers.set( + plane, + setTimeout(() => { + timers.delete(plane); + run(); + }, COARSE_SIGNAL_THROTTLE_MS - elapsed) + ); + }, + [] + ); const scheduleCoarseSignalRefresh = useCallback(() => { - const now = Date.now(); - const elapsed = now - coarseSignalHandledAtRef.current; - if (elapsed >= COARSE_SIGNAL_THROTTLE_MS) { + schedulePlaneSignalRefresh("coarse", runCoarseSignalRefresh); + }, [schedulePlaneSignalRefresh, runCoarseSignalRefresh]); + // Slow convergence net for narrowed dispatch: one trailing full coarse + // refresh per control-plane TTL window while signals keep arriving. + const armCoarseSignalSafetyNet = useCallback(() => { + if (coarseSafetyNetTimerRef.current) return; + coarseSafetyNetTimerRef.current = setTimeout(() => { + coarseSafetyNetTimerRef.current = null; runCoarseSignalRefresh(); - return; - } - if (coarseSignalTrailingTimerRef.current) return; - coarseSignalTrailingTimerRef.current = setTimeout(() => { - coarseSignalTrailingTimerRef.current = null; - runCoarseSignalRefresh(); - }, COARSE_SIGNAL_THROTTLE_MS - elapsed); + }, CONTROL_PLANE_REFRESH_THROTTLE_MS); }, [runCoarseSignalRefresh]); + const dispatchDbChangeSignal = useCallback( + (orgId: string, kind: Org2CloudDbChangeKind) => { + armCoarseSignalSafetyNet(); + if (!isDocumentHidden()) maybeRefreshControlPlane(orgId); + switch (kind) { + case "sessions": + schedulePlaneSignalRefresh("sessions", () => { + org2CloudSyncEngine.invalidateOrgInbound(orgId); + bumpRemoteSessionsVersion(orgId); + }); + return; + case "comments": + schedulePlaneSignalRefresh("comments", () => { + bumpOrgCommentsSignal(orgId); + }); + return; + case "projects": + case "workItems": + schedulePlaneSignalRefresh("inbound", () => { + org2CloudSyncEngine.invalidateOrgInbound(orgId); + }); + return; + case "roster": + schedulePlaneSignalRefresh("roster", () => { + bumpRosterVersion(orgId); + }); + return; + case "policy": + schedulePlaneSignalRefresh("policy", () => { + void refreshEntitlementForOrg(orgId).then((floorChanged) => { + if (floorChanged) org2CloudSyncEngine.resumeOrg(orgId); + }); + bumpRosterVersion(orgId); + }); + } + }, + [ + armCoarseSignalSafetyNet, + maybeRefreshControlPlane, + schedulePlaneSignalRefresh, + bumpRemoteSessionsVersion, + bumpOrgCommentsSignal, + bumpRosterVersion, + refreshEntitlementForOrg, + ] + ); + // Recovery for missed signals on a (re)subscribed org scope. Legacy: the // signal channel's SUBSCRIBED edge. 0005: the org broadcast channel's edge. const runSignalEdgeRecovery = useCallback( (orgId: string) => { - coarseSignalHandledAtRef.current = Date.now(); - controlPlaneRefreshAtRef.current = Date.now(); + const now = Date.now(); + for (const plane of ALL_SIGNAL_PLANES) { + planeSignalHandledAtRef.current.set(plane, now); + } + controlPlaneRefreshAtRef.current = now; if (isDocumentHidden()) return; // A LONG gap forces complete listings so tombstone-free absences // (revoked projects / retention shifts) are observed; a short gap or @@ -526,9 +649,13 @@ export function useOrg2CloudRealtime(): void { delete next[orgId]; return next; }); - if (coarseSignalTrailingTimerRef.current) { - clearTimeout(coarseSignalTrailingTimerRef.current); - coarseSignalTrailingTimerRef.current = null; + for (const timer of planeSignalTrailingTimersRef.current.values()) { + clearTimeout(timer); + } + planeSignalTrailingTimersRef.current.clear(); + if (coarseSafetyNetTimerRef.current) { + clearTimeout(coarseSafetyNetTimerRef.current); + coarseSafetyNetTimerRef.current = null; } }; // Connection identity follows the same activeRealtimeOrgId key in Slice A. @@ -614,17 +741,20 @@ export function useOrg2CloudRealtime(): void { payload: initialPayload, onBroadcast: (event, payload) => { if (event === ORG_DB_CHANGED_EVENT) { - // Server-originated signal (0005 Broadcast-from-Database). Same - // downstream behavior as the legacy postgres_changes transports. - // `roster` also runs the coarse refresh: the membership trigger's - // bump wins the per-org debounce window, so a member-floor RPC in - // the same transaction broadcasts as `roster` (0005 PART 4 "kind - // shadowing") and the policy recovery has to ride this kind too. + // Server-originated signal (0005 Broadcast-from-Database), + // dispatched per kind: 0006 debounces per (org, kind) — no kind + // shadowing — and a member-floor RPC emits BOTH 'policy' and + // 'roster', so each kind maps to exactly its own plane refresh. + // A plain-0005 backend still debounces per org, where a burst can + // shadow one kind behind another and starve that plane forever + // under narrowed dispatch. There is no capability flag for the + // per-kind debounce, so every kind also arms a slow trailing full + // coarse refresh (control-plane TTL cadence): a shadowed plane + // converges within 5 minutes instead of never. if (!broadcastSignals) return; const kind = parseOrgDbChangeKind(payload); if (!kind) return; - if (kind === "roster") bumpRosterVersion(orgId); - scheduleCoarseSignalRefresh(); + dispatchDbChangeSignal(orgId, kind); return; } if (event === ORG_CONTROL_CHANGED_EVENT) { @@ -749,7 +879,7 @@ export function useOrg2CloudRealtime(): void { setCommentsSignal, bumpRemoteSessionsVersion, bumpRosterVersion, - scheduleCoarseSignalRefresh, + dispatchDbChangeSignal, runSignalEdgeRecovery, ]);