Skip to content

Add self-hosted env & docker updates; Convex migration scanner/report; improve webhook delivery and tests#52

Merged
weroperking merged 3 commits intomainfrom
codex/evaluate-betterbase-improvements-lfcc9r
Mar 30, 2026
Merged

Add self-hosted env & docker updates; Convex migration scanner/report; improve webhook delivery and tests#52
weroperking merged 3 commits intomainfrom
codex/evaluate-betterbase-improvements-lfcc9r

Conversation

@weroperking
Copy link
Copy Markdown
Owner

@weroperking weroperking commented Mar 29, 2026

Motivation

  • Provide a canonical self-hosted example environment and ensure it is tracked in the repo for users deploying BetterBase on their own infrastructure.
  • Harden docker-compose configs for production/self-hosted runs by pinning images, adding healthchecks, and aligning environment variable names.
  • Ship a first-class migration tool for converting Convex projects to BetterBase with an automated compatibility scanner and human-review report to reduce migration risk.
  • Improve webhook delivery reliability and observability by persisting delivery attempts and recording failures with attempt counts.

Description

  • Add .env.self-hosted.example and update .gitignore to allow it to be checked into the repo.
  • Update docker-compose.production.yml and docker-compose.self-hosted.yml to pin MinIO images, change inngest to run in production mode (inngest start), add healthchecks for Inngest, and require/rename server env vars to BETTERBASE_JWT_SECRET, BETTERBASE_PUBLIC_URL, and CORS_ORIGINS.
  • Update docs (docs/*) and deployment guides to reference the new env names and the Inngest production command.
  • Implement a robust bb migrate from-convex command: add file/directory safety checks, convert schema and functions, scan for compatibility issues (blockers/warnings), and emit JSON/Markdown migration reports; include a compatibility ruleset and file-level conversion summaries.
  • Add unit tests for the Convex migration flow (packages/cli/test/migrate-from-convex.test.ts) and small test fixes/adjustments in other test files.
  • Improve webhook delivery in packages/server/src/lib/inngest.ts by adding getNextDeliveryAttempt, restructuring HTTP send into a retried step, logging failed and successful deliveries into betterbase_meta.webhook_deliveries with attempt counts, and better timeout/error handling.

Testing

  • Ran the CLI unit tests for the migration feature via bun test targeting packages/cli, and the new migrate-from-convex tests passed.
  • Ran core and client package test suites (packages/core and packages/client) after small test updates, and tests passed locally.
  • Performed basic smoke checks of the updated Docker compose configuration by verifying services start sequence and healthchecks (manual/local verification).

Codex Task

Summary by CodeRabbit

  • New Features

    • Added Convex migration tool with compatibility scanning and detailed migration reports
    • Enhanced webhook delivery tracking with payload hashing
  • Documentation

    • Updated deployment and setup guides with new environment variable names
    • Added self-hosted deployment configuration example
    • Updated production checklist documentation
  • Chores

    • Standardized environment variable naming across configurations
    • Enhanced Docker Compose with Inngest health checks
    • Pinned container image versions for production stability

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 29, 2026

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 700bb831-aefe-49cc-bf88-a4640e65afc9

📥 Commits

Reviewing files that changed from the base of the PR and between 1440edb and 98a2597.

📒 Files selected for processing (9)
  • BetterBase_Competitive_Plan.md
  • packages/cli/src/commands/migrate/from-convex.ts
  • packages/core/test/branching.test.ts
  • packages/server/migrations/016_webhook_delivery_payload_hash.sql
  • packages/server/src/lib/inngest.ts
  • packages/server/src/lib/webhook-dispatcher.ts
  • packages/server/src/routes/admin/inngest.ts
  • packages/server/src/routes/admin/project-scoped/webhooks.ts
  • packages/server/test/inngest.test.ts

Walkthrough

The PR establishes a self-hosted deployment template, switches Inngest from dev to start mode with health checks, renames authentication environment variables from AUTH_SECRET/AUTH_URL to BETTERBASE_JWT_SECRET/BETTERBASE_PUBLIC_URL, enhances the Convex migration CLI with compatibility scanning and report generation, refactors webhook delivery to use SHA-256 payload hashing with database-level idempotency locks instead of event-level attempt tracking, and adds corresponding tests and documentation updates.

Changes

Cohort / File(s) Summary
Environment & Deployment Templates
.env.self-hosted.example, .gitignore
Added self-hosted environment template with JWT secret, admin credentials, Postgres, MinIO, Inngest, and CORS config; negated .gitignore rule to commit the example file.
Docker Compose Infrastructure
docker-compose.production.yml, docker-compose.self-hosted.yml
Pinned MinIO/mc images, switched Inngest devstart, added HTTP healthcheck, changed server dependency from service_startedservice_healthy, renamed CORS_ORIGINCORS_ORIGINS, replaced AUTH_SECRET/AUTH_URL with BETTERBASE_JWT_SECRET/BETTERBASE_PUBLIC_URL.
Documentation Updates
docs/docker-setup.md, docs/guides/deployment.md, docs/guides/production-checklist.md, BetterBase_Competitive_Plan.md
Updated environment variable references from AUTH_* to BETTERBASE_*, documented Inngest CLI modes, added competitive strategy plan.
Convex Migration CLI
packages/cli/src/commands/migrate/from-convex.ts, packages/cli/test/migrate-from-convex.test.ts
Added MigrationIssue and ConversionStats tracking, implemented scanCompatibilityIssues (regex-based rule detection), report generation to JSON and Markdown, safe directory handling, blocker/warning/manual-review aggregation; new test suite validates report output structure.
Webhook Delivery System
packages/server/src/lib/inngest.ts, packages/server/migrations/016_webhook_delivery_payload_hash.sql, packages/server/src/lib/webhook-dispatcher.ts, packages/server/src/routes/admin/project-scoped/webhooks.ts
Removed attempt field from event data; added getPayloadHash and insertWebhookDelivery using pg_advisory_xact_lock on (webhookId:eventType:payloadHash) for idempotency; migrated to payload_hash column and composite index; refactored error/success step handling.
Test Updates
packages/core/test/branching.test.ts, packages/server/test/inngest.test.ts, packages/server/src/routes/admin/inngest.ts
Changed branching.test.ts assertions from .rejects.toThrow to .resolves with mocked implementations; formatting adjustments in webhook test payloads and admin route test data.

Sequence Diagram(s)

sequenceDiagram
    actor Client
    participant Server
    participant Inngest
    participant Database
    participant HTTP["HTTP Endpoint"]
    
    Note over Client,HTTP: Old Flow (attempt-based)
    Client->>Server: Trigger webhook event
    Server->>Inngest: send("betterbase/webhook.deliver", {attempt: 1})
    Inngest->>HTTP: POST delivery (attempt=1)
    alt HTTP succeeds
        HTTP-->>Inngest: 200 OK
        Inngest->>Database: INSERT row with attempt=1
    else HTTP fails
        HTTP-->>Inngest: Error
        Inngest->>Inngest: Retry with attempt=2
        Inngest->>HTTP: POST delivery (attempt=2)
    end
    
    Note over Client,HTTP: New Flow (payload hash + advisory lock)
    Client->>Server: Trigger webhook event
    Server->>Inngest: send("betterbase/webhook.deliver", {eventType, payload})
    Inngest->>HTTP: POST delivery
    alt HTTP succeeds
        HTTP-->>Inngest: 200 OK + response
        Inngest->>Database: pg_advisory_xact_lock(hash)
        Database->>Database: COALESCE(MAX(attempt_count),0)+1
        Database-->>Inngest: INSERT with status=success
    else HTTP fails
        HTTP-->>Inngest: Error
        Inngest->>Database: pg_advisory_xact_lock(hash)
        Database->>Database: COALESCE(MAX(attempt_count),0)+1
        Database-->>Inngest: INSERT with status=failed
        Inngest->>Inngest: Retry (Inngest handles backoff)
    end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~65 minutes

Possibly related PRs

  • Adding Ingest  #45: Modifies Inngest integration environment variables, docker-compose setup, and packages/server/src/lib/inngest.ts webhook wiring—directly overlaps webhook delivery and Inngest configuration changes.
  • Read the commits  #42: Modifies packages/cli/src/commands/migrate/from-convex.ts CLI conversion functionality—same migration command being enhanced here.
  • feat: comprehensive core tasks for March 2026 release #36: Modifies webhook delivery logging/schema and dispatcher work—overlaps payload hash storage and webhook event data structure changes.
🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 11.11% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title comprehensively covers the main changes: self-hosted env config, Docker updates, Convex migration tooling with scanning/reporting, webhook improvements, and test fixes. It is specific and reflects the pull request's scope.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/evaluate-betterbase-improvements-lfcc9r

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
docs/docker-setup.md (1)

42-63: ⚠️ Potential issue | 🟠 Major

Self-hosted bootstrap/env guidance is inconsistent and can misconfigure deployments

Line 47 still points to .env.example instead of .env.self-hosted.example, and the “Required Environment Variables” block omits BETTERBASE_PUBLIC_URL and CORS_ORIGINS. This conflicts with the self-hosted contract and can produce incorrect external URL/CORS behavior.

Proposed doc fix
 # 2. Configure environment
-cp .env.example .env
+cp .env.self-hosted.example .env
 # Edit .env with your values
@@
 # Generate a secure auth secret
 BETTERBASE_JWT_SECRET=$(openssl rand -base64 32)
+BETTERBASE_PUBLIC_URL=https://your-domain.com
+CORS_ORIGINS=https://your-frontend.com
 
 # Generate Inngest keys (optional for development)
 INNGEST_SIGNING_KEY=$(openssl rand -hex 32)
 INNGEST_EVENT_KEY=$(openssl rand -hex 16)

As per coding guidelines, “Self-hosted deployment docs must use cp .env.self-hosted.example .env … Ensure required envs match the new names … Verify CORS via CORS_ORIGINS and use BETTERBASE_PUBLIC_URL for external URL references.”

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docs/docker-setup.md` around lines 42 - 63, Update the self-hosted setup step
to copy from .env.self-hosted.example instead of .env.example and update the
"Required Environment Variables" block to include BETTERBASE_PUBLIC_URL and
CORS_ORIGINS so the docs match the self-hosted contract; specifically change the
cp command shown in the shell snippet to use .env.self-hosted.example and add
entries describing BETTERBASE_PUBLIC_URL (external base URL) and CORS_ORIGINS
(comma-separated allowed origins) alongside the existing BETTERBASE_JWT_SECRET
and INNGEST keys so external URL/CORS behavior is correctly configured.
packages/server/src/lib/inngest.ts (1)

268-273: ⚠️ Potential issue | 🟡 Minor

deliveryResult accessed without null guard despite | undefined type.

At line 157-163, deliveryResult is typed as { ... } | undefined. Although control flow ensures it's defined here (we rethrow in the catch), TypeScript's narrowing may not recognize this.

🛡️ Proposed fix
  return {
    success: true,
    webhookId,
-   httpStatus: deliveryResult.httpStatus,
-   durationMs: deliveryResult.durationMs,
+   httpStatus: deliveryResult!.httpStatus,
+   durationMs: deliveryResult!.durationMs,
  };

Or initialize deliveryResult to a sentinel and remove | undefined from the type if you prefer avoiding !.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/server/src/lib/inngest.ts` around lines 268 - 273, The return block
accesses deliveryResult which is typed as possibly undefined; update the code
around deliveryResult (the variable named deliveryResult and the function that
builds/returns the object) to guarantee non-null before use—either add a
null-check (if (!deliveryResult) throw or return an error) or initialize
deliveryResult to a sentinel default so its type is not undefined (or use a
single non-null assertion in the return if you prefer, e.g. deliveryResult!),
ensuring the properties deliveryResult.httpStatus and deliveryResult.durationMs
are only accessed when deliveryResult is proven defined.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@BetterBase_Competitive_Plan.md`:
- Around line 78-92: MD022 violations occur because each week heading (e.g.,
"### Week 1", "### Week 2", "### Week 3", "### Week 4") is immediately followed
by a list without a blank line; update the markdown so there is one blank line
after each "### Week X" heading before its list to satisfy MD022 (ensure the
headings "### Week 1"/"### Week 2"/"### Week 3"/"### Week 4" are followed by an
empty line).

In `@packages/cli/src/commands/migrate/from-convex.ts`:
- Around line 48-50: The input-path validation currently logs and returns on
failure (the existsSync/statSync branch that calls logger.error and returns),
which yields a zero exit status; change this to propagate a failure by throwing
an Error (or otherwise returning a rejected promise) with a clear message so the
CLI exits non‑zero. Locate the validation in the from-convex command handler
(the block using existsSync and statSync and calling logger.error) and replace
the plain return with throw new Error(`Input path is not a directory:
${inputPath}`) (or reject) so scripts/CI receive a failing status.

In `@packages/core/test/branching.test.ts`:
- Around line 400-414: Tests incorrectly assert that listPreviewDatabases,
previewDatabaseExists, and teardownPreviewDatabase should reject; these are
success-path APIs and should be asserted to resolve deterministically. Update
the three tests referencing dbBranching.listPreviewDatabases,
dbBranching.previewDatabaseExists, and dbBranching.teardownPreviewDatabase to
assert resolution (for example use await expect(...).resolves.toBeDefined() or
await expect(...).resolves.toBeTruthy() or simply await the call and assert
expected return) instead of using rejects.toThrow so tests don't fail when
Postgres is reachable.

In `@packages/server/src/lib/inngest.ts`:
- Around line 230-241: There is a TOCTOU race between getNextDeliveryAttempt and
the subsequent INSERT: replace the separate read-then-write with an atomic
operation by acquiring a per-webhook/event/payload advisory lock
(pg_advisory_xact_lock) or performing the attempt_count computation and insert
in a single DB transaction; specifically, in the block inside
step.run("log-failed-delivery") remove the call to getNextDeliveryAttempt and
instead run one transaction that first issues
pg_advisory_xact_lock(hashtext(concat(webhookId,eventType,JSON.stringify(payload))))
and then INSERT INTO betterbase_meta.webhook_deliveries (...) SELECT
$1,$2,$3,'failed',$4,NOW(),COALESCE(MAX(attempt_count),0)+1 FROM
betterbase_meta.webhook_deliveries WHERE webhook_id=$1 AND event_type=$2 AND
payload=$3, using the same pool.query call (and the same advisory-lock+insert
pattern for the success path referenced around step.run for the success case).
Ensure identifiers: step.run("log-failed-delivery"), getNextDeliveryAttempt, and
the INSERT into betterbase_meta.webhook_deliveries are updated accordingly.
- Around line 129-137: The Events type and dispatcher payload include an unused
attempt field; remove this dead field instead of recalculating it. Update the
Events type to drop attempt, remove any attempt: 1 from the dispatcher payload,
and remove attempt from the destructuring in the webhook handler (the block that
currently extracts webhookId, webhookName, url, secret, eventType, tableName,
payload). Also search for other uses of attempt in this module and related types
and tests and delete them so getNextDeliveryAttempt() remains the single source
of truth for attemptCount.
- Around line 28-44: The query in getNextDeliveryAttempt is causing full scans
because only (webhook_id, created_at DESC) is indexed; add a composite index
that covers (webhook_id, event_type, payload) or (better) add a payload_hash
(SHA-256 of the canonical JSON string) column, index (webhook_id, event_type,
payload_hash) and change queries to filter on payload_hash (and verify payload
equality when needed) to avoid large JSONB indexes. To prevent TOCTOU on
attempt_count in getNextDeliveryAttempt, perform the read-and-insert inside a
transaction with proper serialization (use SERIALIZABLE isolation or a SELECT
... FOR UPDATE on a per-webhook/event/payload lock row or use a unique
constraint on (webhook_id,event_type,payload_hash,attempt_count) and an
UPSERT/retry loop) so concurrent callers cannot obtain the same next_attempt;
update getNextDeliveryAttempt (and the caller that inserts deliveries) to use
the chosen approach.

---

Outside diff comments:
In `@docs/docker-setup.md`:
- Around line 42-63: Update the self-hosted setup step to copy from
.env.self-hosted.example instead of .env.example and update the "Required
Environment Variables" block to include BETTERBASE_PUBLIC_URL and CORS_ORIGINS
so the docs match the self-hosted contract; specifically change the cp command
shown in the shell snippet to use .env.self-hosted.example and add entries
describing BETTERBASE_PUBLIC_URL (external base URL) and CORS_ORIGINS
(comma-separated allowed origins) alongside the existing BETTERBASE_JWT_SECRET
and INNGEST keys so external URL/CORS behavior is correctly configured.

In `@packages/server/src/lib/inngest.ts`:
- Around line 268-273: The return block accesses deliveryResult which is typed
as possibly undefined; update the code around deliveryResult (the variable named
deliveryResult and the function that builds/returns the object) to guarantee
non-null before use—either add a null-check (if (!deliveryResult) throw or
return an error) or initialize deliveryResult to a sentinel default so its type
is not undefined (or use a single non-null assertion in the return if you
prefer, e.g. deliveryResult!), ensuring the properties deliveryResult.httpStatus
and deliveryResult.durationMs are only accessed when deliveryResult is proven
defined.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 1c44fa2f-7dd3-4324-85cd-e7d11939b869

📥 Commits

Reviewing files that changed from the base of the PR and between d45ce43 and 1440edb.

📒 Files selected for processing (13)
  • .env.self-hosted.example
  • .gitignore
  • BetterBase_Competitive_Plan.md
  • docker-compose.production.yml
  • docker-compose.self-hosted.yml
  • docs/docker-setup.md
  • docs/guides/deployment.md
  • docs/guides/production-checklist.md
  • packages/cli/src/commands/migrate/from-convex.ts
  • packages/cli/test/migrate-from-convex.test.ts
  • packages/client/test/auth.test.ts
  • packages/core/test/branching.test.ts
  • packages/server/src/lib/inngest.ts

@weroperking weroperking merged commit db24622 into main Mar 30, 2026
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant