Add SupaMail v0.1 — IMAP mirror to Postgres/Supabase#5
Open
fedster99 wants to merge 11 commits into
Open
Conversation
- Encrypt IMAP passwords in Node (AES-256-GCM, versioned payload) and drop the pgcrypto pgp_sym_encrypt/decrypt SQL helpers so the master key never crosses into Postgres logs or pg_stat_statements - Validate IMAP host/port before connect: reject private/link-local/ loopback/metadata IPs, restrict ports to 143/993, require TLS unless IMAP_ALLOW_PRIVATE_HOSTS opt-in is set - Constant-time bearer comparison; structured API error contract (zod input validation, global onError mapping ZodError/host-validation/ not-found/pg unique-violation to 400/404/409); separate ADMIN_TOKEN for POST /migrate - Harden MIME parsing: depth + cycle guards on bodyStructure visitors, surrogate / out-of-range code-point rejection in entity decoder, simpleParser size + link-skip limits - Sanitize sync_state_reason / sync-run error text against credential echo and control-char log injection at every persistence site - Schema: bigint size_bytes/raw_bytes, supporting indexes for the ON DELETE SET NULL FKs and for message_id_normalized lookups, folder_path documented as the canonical key - Stream reconcile UIDs through a TEMP TABLE ON COMMIT DROP so heavy mailboxes (100k+ UIDs) don't OOM the worker - Worker: uncaughtException / unhandledRejection handlers feed graceful shutdown so the advisory lock is released cleanly; SIGTERM aborts the in-flight sync between accounts - Wrap each message + attachment upsert in its own transaction so a partial attachment insert can't leave a half-formed row - Recovery: a clean sync clears DEGRADED/BROKEN back to HEALTHY - Cleanup: replace Math.max(...uids) spread, recursive throttle acquire, dead hooks.ts module, unused ImapThrottle observability - Tests: behavioral suites for crypto round-trip, host validation (IPv4/IPv6 reserved ranges, ports, TLS requirement), and MIME edge cases (out-of-range entities, cyclic / deeply nested bodyStructure). 59 tests across 10 files. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
supabase/migrations/0001_imap_mirror.sql— seven tables underpublic.imap_*, RLS enabled with no policies (deny-all to anon/authenticated by design), session-affineDATABASE_URLrequired because reconciliation relies on advisory locks.pnpm dry-run:localruns against a fake IMAP fixture,pnpm smoke:greenmailspins a realgreenmail/standaloneDocker IMAP server and syncs through the protocol.Things to look at first (non-obvious decisions)
encrypted_passwordis AES-256-GCM ({ version | iv | tag | ciphertext }) computed insrc/crypto.ts. The migrationDROPs the legacyimap_encrypt_password/imap_decrypt_passwordSQL functions so the master key never crosses into Postgres logs orpg_stat_statements.src/host-validation.tsrejects private/link-local/loopback/metadata IPs (including IPv4-mapped IPv6), restricts ports to 143/993, and refuses plaintext IMAP unlessIMAP_ALLOW_PRIVATE_HOSTS=true. Local dev and the smoke scripts opt in via per-script config override.crypto.timingSafeEqualon equal-length buffers, not!==.POST /migratefalls back toAPI_TOKENifADMIN_TOKENis unset.TEMP TABLE … ON COMMIT DROP(markMissingMessagesFromLiveUidStream) instead of passing abigint[]parameter — needed for Gmail-scale folders (100k+ UIDs).src/locks.ts). The testrepository-safety.test.ts > "does not hold account locks with idle transactions"pins this against accidental conversion topg_advisory_xact_lock.WeakSetcycle guard;simpleParseris called withmaxHtmlLengthToParse: 1 MBand link/text skip flags;decodeHtmlEntitiesrejects out-of-range and surrogate code points to prevent a single email DoSing the worker.app.onErrormapsZodError→ 400,HostValidationError→ 400,NotFoundError→ 404, pg23505→ 409; everything else logs structured JSON and returns{ error: \"internal_error\" }.What's deferred (explicit non-goals for v0.1)
Test plan
🤖 Generated with Claude Code