Skip to content

development.md

Latest commit

 

History

History
169 lines (121 loc) · 4.64 KB

development.md

File metadata and controls

169 lines (121 loc) · 4.64 KB
Raw

Development

This guide covers local contributor workflows for ExtendDO. ExtendDO is a Workers project with TypeScript host code, Durable Objects, R2 bindings, and local ExtendDB Rust crates compiled to wasm.

Prerequisites

  • Node.js and pnpm matching package.json.
  • Rust with the wasm32-unknown-unknown target installed.
  • wasm-pack; the project scripts use it to build crates/wasm.
  • Wrangler from the project dev dependency. Prefer pnpm exec wrangler ... or the package scripts over a globally installed Wrangler.

Install dependencies:

pnpm install

If the wasm target is missing:

rustup target add wasm32-unknown-unknown

Workspace Layout

  • worker/src: Worker host, auth, management API, R2 import/export adapter, and Durable Object classes.
  • crates/wasm: wasm-bindgen bridge that calls local ExtendDB operation handlers and calls back into the Worker host for storage.
  • third_party/extenddb: ignored local ExtendDB core, engine, storage, and auth crates created by pnpm run setup:extenddb.
  • patches/extenddb: tracked source manifest and ExtendDO compatibility patch applied during local setup.
  • docs: operator, contributor, architecture, and parity documentation.

The ignored external-projects/extenddb checkout is reference-only. It is not a Cargo dependency and it is not used as a Git submodule.

Common Commands

Build the wasm package consumed by the Worker:

pnpm run build:wasm

Generate Cloudflare Worker binding types from the tracked example Wrangler config:

pnpm run generate:types

Run TypeScript type checking after regenerating wasm and Worker types:

pnpm run typecheck

Run Rust checks for all crates using the Worker wasm target:

cargo check --workspace --target wasm32-unknown-unknown

Run the combined Rust and TypeScript check script:

pnpm run check

Run the Worker/Vitest suite:

pnpm run test

Build and dry-run package the Worker:

pnpm run deploy:dry-run
pnpm run build

Local ExtendDB Source

ExtendDO does not track ExtendDB source files in Git. Before building, create the ignored local source tree:

pnpm run setup:extenddb

The setup script downloads the pinned upstream archive from patches/extenddb/manifest.json, extracts only crates/core, crates/auth, crates/storage, crates/engine, LICENSE, and NOTICE, then applies tracked ExtendDO compatibility patches from patches/extenddb.

To use an existing local checkout instead of downloading:

pnpm run setup:extenddb -- --source ../extenddb

To recreate from a remote Git checkout with an exact ref:

pnpm run setup:extenddb -- --repo <extenddb-git-url> --ref <tag-or-sha>

Build scripts run scripts/check-extenddb-source.mjs and fail with setup instructions when third_party/extenddb is missing. They do not download source implicitly.

When refreshing ExtendDB, update patches/extenddb/manifest.json, regenerate patches/extenddb/compatibility.patch, update docs/extenddb-reference.md, and run pnpm run setup:extenddb -- --force, pnpm run test, and pnpm run build.

Local Wrangler Development

Build wasm and generated types before starting local development:

cp wrangler.example.json wrangler.jsonc
pnpm run build:wasm
pnpm run generate:types
pnpm exec wrangler dev --config wrangler.jsonc

Wrangler local development uses local simulated bindings unless a binding is configured for remote use. The tracked wrangler.example.json and local wrangler.jsonc bind:

  • CATALOG to CatalogDO
  • TABLE to TableDO
  • TRANSACTIONS to TransactionCoordinatorDO
  • IMPORT_EXPORT_BUCKET to the configured R2 bucket

Use /healthz to confirm the Worker and wasm module are available:

curl http://localhost:8787/healthz

Generated Files

pnpm run build:wasm writes the wasm package under worker/src/generated/wasm. pnpm run generate:types writes worker/src/worker-configuration.d.ts. Regenerate both after changes to Rust wasm exports, Worker bindings, Durable Object classes, wrangler.example.json, or local wrangler.jsonc.

Test Focus

The Vitest suite exercises:

  • DynamoDB JSON request routing through wasm.
  • Durable Object SQLite persistence and alarms.
  • Table lifecycle, item operations, query/scan, secondary indexes, TTL, streams, backups, import/export, and transactions.
  • Management credentials, IAM-shaped resources, SigV4 data-plane auth, and policy evaluation through the local ExtendDB auth crate.

Run pnpm run test after changing Worker code, Rust wasm code, management API behavior, import/export handling, Durable Object schema, or docs examples that copy live command shapes.