diff --git a/AGENTS.md b/AGENTS.md index ab222c07..2b70d98f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -75,12 +75,22 @@ vcms admin reset-password --email U --password P vcms backup create [--scope instance|site] [--site ID] [--out FILE] [--no-files] [--encrypt] vcms backup list # list recorded backups vcms restore --file PATH [--scope instance|site] [--site ID] [--import-as-new] --yes +vcms service install [--user NAME] # install/enable/start as an OS service (systemd/launchd/SCM) +vcms service uninstall|status|start|stop +vcms mcp stdio # thin HTTP proxy to a running server's /mcp (for MCP clients) ``` `backup`/`restore` run offline (no HTTP server) against the configured database — the disaster-recovery path when the instance won't boot. `restore` is destructive and requires `--yes`. +`mcp stdio` opens no database, secrets, or search index of its own: it forwards +JSON-RPC between stdin/stdout and the server's `/mcp` Streamable-HTTP endpoint, +reading only `VCMS_MCP_TOKEN` (bearer) and `VCMS_MCP_URL` (default +`http://127.0.0.1:3000`). So it works even when the data is owned by the OS-service +account. `vcms service install` pins `VCMS_HOME` to a system dir so the daemon stores +everything under one owned root. + Global flags (highest precedence): `--config `, `--bind `, `--database-url `, `--log-level `. The server auto-migrates the database on every startup; there is no separate migrate command. @@ -93,29 +103,53 @@ Layers merge with precedence: **CLI flag > env var > config file > built-in defa Config file search order (first existing wins; missing is fine): 1. `--config` flag / `VCMS_CONFIG` env 2. `./vcms.toml` (current dir) -3. `~/.vcms/config.toml` (CMS home; `$VCMS_HOME/config.toml` if set) — where `vcms config init` writes +3. the platform config dir (`config.toml`; `$VCMS_HOME/config.toml` in single-dir mode) — where `vcms config init` writes 4. `/etc/vcms/config.toml` -## Data directory (CMS home) +## Data directory + +Resolution lives in `apps/backend/src/paths.rs` and has two layouts: + +**Split (default, interactive installs)** — files land in the platform-conventional +per-type directories via the `directories` crate (`ProjectDirs`): + +| File(s) | Dir | Linux | macOS | Windows | +|---------|-----|-------|-------|---------| +| `config.toml`, `secrets.toml`, `.env` | config | `~/.config/vcms` | `~/Library/Application Support/vcms` | `%APPDATA%\vcms\config` | +| `vcms.db`, `storage/`, `backups/` | data | `~/.local/share/vcms` | `~/Library/Application Support/vcms` | `%APPDATA%\vcms\data` | +| `search/` (derived, rebuildable) | cache | `~/.cache/vcms` | `~/Library/Caches/vcms` | `%LOCALAPPDATA%\vcms\cache` | +| `logs/` | state | `~/.local/state/vcms` | `~/Library/Application Support/vcms` | `%LOCALAPPDATA%\vcms\data` | -All runtime files live under one home directory: `$VCMS_HOME` if set, else `~/.vcms` -(same layout on Windows, macOS, Linux via the `directories` crate). `vcms serve` -creates it on first run. Resolution lives in `apps/backend/src/paths.rs`. +(`logs/` uses the state dir where the platform has one — Linux — else the local data dir.) + +**Single** — everything nests under one root. Chosen (in precedence order) when: +1. **`$VCMS_HOME` is set** — forces the root explicitly. +2. **the system service home dir exists** — Linux `/var/lib/vcms`, macOS + `/Library/Application Support/vcms`, Windows `C:\ProgramData\vcms`. The + `vcms service` installer creates it (and leaves it behind on uninstall), so a plain + `vcms serve`/`admin`/`backup` **follows the service's data instead of forking to a + per-user split store**. This path is defined once in `paths::system_home()` and + imported by the `service` submodules. +3. **a legacy `~/.vcms` exists** — an existing install keeps working untouched. + +Otherwise (dev/eval boxes with no service) files use the platform split dirs. ```text -~/.vcms/ - config.toml # non-secret config (vcms config init target) - secrets.toml # auto-generated HMAC_SECRET + backup key (0600 on unix) - vcms.db # default SQLite database (+ -wal / -shm) - logs/ # rolling logs when [log] output = "file" - storage/ # default filesystem storage for uploads +$VCMS_HOME/ # or system home, or ~/.vcms (legacy) + config.toml secrets.toml .env + vcms.db (+ -wal / -shm) logs/ storage/ backups/ search/ ``` -Secrets: on first `serve`/`admin`, a random `HMAC_SECRET` is generated -and persisted to `secrets.toml` (`apps/backend/src/secrets.rs`), then loaded by -every process — including `vcms mcp stdio`, which is launched from an arbitrary cwd -and so cannot rely on a cwd `.env`. Env vars still override the file. `mcp stdio` -is read-only: it never creates the home dir, database, or secrets file. +When the active home is the system service home (owned by SYSTEM/root), the +data-touching commands (`serve`/`admin`/`backup`/`restore`) require elevation — a +non-elevated invocation **fails fast** with an "Administrator/root" hint (in +`paths::ensure`'s preflight) rather than silently forking to a second store. + +Secrets: on first `serve`/`admin`, a random `HMAC_SECRET` is generated and persisted to +`secrets.toml` (`apps/backend/src/secrets.rs`), then loaded by the server processes. +`vcms mcp stdio` does **not** load secrets, the database, or any data-dir file: it is a +thin HTTP proxy to the running server's `/mcp` (see CLI), forwarding `VCMS_MCP_TOKEN` as +the bearer; the server owns all disk I/O. Env-only secrets (never read from `config.toml` by convention, omitted from `config init`): `DATABASE_URL`, `HMAC_SECRET`, `S3_ACCESS_KEY_ID`, @@ -153,8 +187,10 @@ Logging keys map to the `[log]` table: `RUST_LOG`→`log.level`, `LOG_OUTPUT`→ | Variable | Default | Description | |----------|---------|-------------| | `VCMS_CONFIG` | - | Explicit config file path (same as `--config`) | -| `VCMS_HOME` | `~/.vcms` | CMS home directory (db, config, secrets, logs, storage) | -| `DATABASE_URL` | `sqlite://~/.vcms/vcms.db` | Database URL: `sqlite:path`, `postgres://...`, `mysql://...` | +| `VCMS_HOME` | - | If set, forces single-dir mode: db/config/secrets/logs/storage all nest under this root (else files use the platform split dirs) | +| `VCMS_MCP_TOKEN` | - | `vcms_site_*` access token forwarded by `vcms mcp stdio` as the bearer credential (required for stdio) | +| `VCMS_MCP_URL` | `http://127.0.0.1:3000` | Running server's base URL that `vcms mcp stdio` proxies to (`{url}/mcp`) | +| `DATABASE_URL` | `sqlite:///vcms.db` | Database URL: `sqlite:path`, `postgres://...`, `mysql://...` | | `HMAC_SECRET` | auto | HMAC key for token lookup (required; auto-generated to `secrets.toml`, env overrides) | | `BIND_ADDRESS` | `0.0.0.0:3000` | REST API listen address | | `GRPC_BIND_ADDRESS` | `0.0.0.0:50051` | gRPC server listen address | @@ -167,7 +203,7 @@ Logging keys map to the `[log]` table: `RUST_LOG`→`log.level`, `LOG_OUTPUT`→ | `S3_PUBLIC_URL` | - | Public URL for S3 assets | | `BACKUP_ENABLED` | `true` | Run the scheduled-backup poller / allow backups | | `BACKUP_DESTINATION` | `filesystem` | Backup destination: `filesystem` or `s3` | -| `BACKUP_LOCAL_PATH` | `~/.vcms/backups` | Local backup dir (when destination is filesystem) | +| `BACKUP_LOCAL_PATH` | `/backups` | Local backup dir (when destination is filesystem) | | `BACKUP_ZSTD_LEVEL` | `12` | zstd compression level for backups | | `BACKUP_DEFAULT_RETENTION` | `7` | Default "keep last N" for new schedules | | `BACKUP_S3_BUCKET` / `_REGION` / `_ENDPOINT` / `_PUBLIC_URL` | - | S3 backup destination (non-secret parts) | @@ -175,6 +211,7 @@ Logging keys map to the `[log]` table: `RUST_LOG`→`log.level`, `LOG_OUTPUT`→ | `BACKUP_S3_SECRET_ACCESS_KEY` | - | S3 backup secret key (secret, env-only) | | `BACKUP_ENCRYPTION_KEY` | auto | AES-256 backup key (hex); auto-generated to `secrets.toml` | | `MAX_UPLOAD_SIZE_MB` | `50` | Max upload size in MB | +| `UPLOAD_TOKEN_EXPIRY_SECS` | `900` | Signed upload URL lifetime (seconds) | | `COOKIE_SECURE` | `false` | Require HTTPS cookies | | `DB_MAX_CONNECTIONS` | `10` | Max DB connections | | `DB_MIN_CONNECTIONS` | `2` | Min DB connections | @@ -185,10 +222,10 @@ Logging keys map to the `[log]` table: `RUST_LOG`→`log.level`, `LOG_OUTPUT`→ | `LOG_OUTPUT` | `stdout` | `stdout` or `file` (`[log] output`) | | `LOG_FORMAT` | `pretty` | `pretty` or `json` (`[log] format`) | | `LOG_ANNOTATIONS` | `false` | Include file + line numbers (`[log] annotations`) | -| `LOG_DIR` | `~/.vcms/logs` | Log directory when `output = file` (`[log] dir`) | +| `LOG_DIR` | `/logs` | Log directory when `output = file` (`[log] dir`) | -**Note**: `HMAC_SECRET` is auto-generated and persisted to -`~/.vcms/secrets.toml` on first run. Set it explicitly via env to override. +**Note**: `HMAC_SECRET` is auto-generated and persisted to the config dir's +`secrets.toml` on first run. Set it explicitly via env to override. ## Proto Compilation diff --git a/CLAUDE.md b/CLAUDE.md index 7fa991bf..5e942c60 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -94,30 +94,54 @@ Layers merge with precedence: **CLI flag > env var > config file > built-in defa Config file search order (first existing wins; missing is fine): 1. `--config` flag / `VCMS_CONFIG` env 2. `./vcms.toml` (current dir) -3. `~/.vcms/config.toml` (CMS home; `$VCMS_HOME/config.toml` if set) — where `vcms config init` writes +3. the platform config dir (`config.toml`; `$VCMS_HOME/config.toml` in single-dir mode) — where `vcms config init` writes 4. `/etc/vcms/config.toml` -## Data directory (CMS home) +## Data directory -All runtime files live under one home directory: `$VCMS_HOME` if set, else `~/.vcms` -(same layout on Windows, macOS, Linux via the `directories` crate). `vcms serve` -creates it on first run. Resolution lives in `apps/backend/src/paths.rs`. +Resolution lives in `apps/backend/src/paths.rs` and has two layouts: + +**Split (default, interactive installs)** — files land in the platform-conventional +per-type directories via the `directories` crate (`ProjectDirs`): + +| File(s) | Dir | Linux | macOS | Windows | +|---------|-----|-------|-------|---------| +| `config.toml`, `secrets.toml`, `.env` | config | `~/.config/vcms` | `~/Library/Application Support/vcms` | `%APPDATA%\vcms\config` | +| `vcms.db`, `storage/`, `backups/` | data | `~/.local/share/vcms` | `~/Library/Application Support/vcms` | `%APPDATA%\vcms\data` | +| `search/` (derived, rebuildable) | cache | `~/.cache/vcms` | `~/Library/Caches/vcms` | `%LOCALAPPDATA%\vcms\cache` | +| `logs/` | state | `~/.local/state/vcms` | `~/Library/Application Support/vcms` | `%LOCALAPPDATA%\vcms\data` | + +(`logs/` uses the state dir where the platform has one — Linux — else the local data dir.) + +**Single** — everything nests under one root. Chosen (in precedence order) when: +1. **`$VCMS_HOME` is set** — forces the root explicitly. +2. **the system service home dir exists** — Linux `/var/lib/vcms`, macOS + `/Library/Application Support/vcms`, Windows `C:\ProgramData\vcms`. The + `vcms service` installer creates it (and leaves it behind on uninstall), so a plain + `vcms serve`/`admin`/`backup` **follows the service's data instead of forking to a + per-user split store**. This path is defined once in `paths::system_home()` and + imported by the `service` submodules. +3. **a legacy `~/.vcms` exists** — an existing install keeps working untouched. + +Otherwise (dev/eval boxes with no service) files use the platform split dirs. ```text -~/.vcms/ - config.toml # non-secret config (vcms config init target) - secrets.toml # auto-generated HMAC_SECRET + backup key (0600 on unix) - vcms.db # default SQLite database (+ -wal / -shm) - logs/ # rolling logs when [log] output = "file" - storage/ # default filesystem storage for uploads - search/ # Tantivy full-text search index (derived; rebuildable) +$VCMS_HOME/ # or system home, or ~/.vcms (legacy) + config.toml secrets.toml .env + vcms.db (+ -wal / -shm) logs/ storage/ backups/ search/ ``` +`vcms serve`/`admin` create the dirs they need on first run; `vcms mcp stdio` creates +nothing. When the active home is the system service home (owned by SYSTEM/root), the +data-touching commands (`serve`/`admin`/`backup`/`restore`) require elevation — a +non-elevated invocation **fails fast** with an "Administrator/root" hint (in +`paths::ensure`'s preflight) rather than silently forking to a second store. + Secrets: on first `serve`/`admin`, a random `HMAC_SECRET` is generated and persisted to `secrets.toml` (`apps/backend/src/secrets.rs`), then loaded by -every process — including `vcms mcp stdio`, which is launched from an arbitrary cwd -and so cannot rely on a cwd `.env`. Env vars still override the file. `mcp stdio` -is read-only: it never creates the home dir, database, or secrets file. +the server processes. `vcms mcp stdio` does **not** load secrets, the database, or +any home-dir file: it is a thin HTTP proxy (see below) and the server it forwards to +owns all of those. Env-only secrets (never read from `config.toml` by convention, omitted from `config init`): `DATABASE_URL`, `HMAC_SECRET`, `S3_ACCESS_KEY_ID`, @@ -156,8 +180,10 @@ Logging keys map to the `[log]` table: `RUST_LOG`→`log.level`, `LOG_OUTPUT`→ | Variable | Default | Description | |----------|---------|-------------| | `VCMS_CONFIG` | - | Explicit config file path (same as `--config`) | -| `VCMS_HOME` | `~/.vcms` | CMS home directory (db, config, secrets, logs, storage) | -| `DATABASE_URL` | `sqlite://~/.vcms/vcms.db` | Database URL: `sqlite:path`, `postgres://...`, `mysql://...` | +| `VCMS_HOME` | - | If set, forces single-dir mode: db/config/secrets/logs/storage all nest under this root (else files use the platform split dirs) | +| `VCMS_MCP_TOKEN` | - | `vcms_site_*` access token forwarded by `vcms mcp stdio` as the bearer credential (required for stdio) | +| `VCMS_MCP_URL` | `http://127.0.0.1:3000` | Running server's base URL that `vcms mcp stdio` proxies to (`{url}/mcp`) | +| `DATABASE_URL` | `sqlite:///vcms.db` | Database URL: `sqlite:path`, `postgres://...`, `mysql://...` | | `HMAC_SECRET` | auto | HMAC key for token lookup (required; auto-generated to `secrets.toml`, env overrides) | | `BIND_ADDRESS` | `0.0.0.0:3000` | REST API listen address | | `GRPC_BIND_ADDRESS` | `0.0.0.0:50051` | gRPC server listen address | @@ -170,7 +196,7 @@ Logging keys map to the `[log]` table: `RUST_LOG`→`log.level`, `LOG_OUTPUT`→ | `S3_PUBLIC_URL` | - | Public URL for S3 assets | | `BACKUP_ENABLED` | `true` | Run the scheduled-backup poller / allow backups | | `BACKUP_DESTINATION` | `filesystem` | Backup destination: `filesystem` or `s3` | -| `BACKUP_LOCAL_PATH` | `~/.vcms/backups` | Local backup dir (when destination is filesystem) | +| `BACKUP_LOCAL_PATH` | `/backups` | Local backup dir (when destination is filesystem) | | `BACKUP_ZSTD_LEVEL` | `12` | zstd compression level for backups | | `BACKUP_DEFAULT_RETENTION` | `7` | Default "keep last N" for new schedules | | `BACKUP_S3_BUCKET` / `_REGION` / `_ENDPOINT` / `_PUBLIC_URL` | - | S3 backup destination (non-secret parts) | @@ -178,8 +204,9 @@ Logging keys map to the `[log]` table: `RUST_LOG`→`log.level`, `LOG_OUTPUT`→ | `BACKUP_S3_SECRET_ACCESS_KEY` | - | S3 backup secret key (secret, env-only) | | `BACKUP_ENCRYPTION_KEY` | auto | AES-256 backup key (hex); auto-generated to `secrets.toml` | | `SEARCH_ENABLED` | `true` | Build/use the Tantivy full-text index for entry search (else SQL `LIKE`) | -| `SEARCH_INDEX_PATH` | `~/.vcms/search` | Directory for the Tantivy search index | +| `SEARCH_INDEX_PATH` | `/search` | Directory for the Tantivy search index | | `MAX_UPLOAD_SIZE_MB` | `50` | Max upload size in MB | +| `UPLOAD_TOKEN_EXPIRY_SECS` | `900` | Signed upload URL lifetime (seconds) | | `COOKIE_SECURE` | `false` | Require HTTPS cookies | | `DB_MAX_CONNECTIONS` | `10` | Max DB connections | | `DB_MIN_CONNECTIONS` | `2` | Min DB connections | @@ -190,10 +217,10 @@ Logging keys map to the `[log]` table: `RUST_LOG`→`log.level`, `LOG_OUTPUT`→ | `LOG_OUTPUT` | `stdout` | `stdout` or `file` (`[log] output`) | | `LOG_FORMAT` | `pretty` | `pretty` or `json` (`[log] format`) | | `LOG_ANNOTATIONS` | `false` | Include file + line numbers (`[log] annotations`) | -| `LOG_DIR` | `~/.vcms/logs` | Log directory when `output = file` (`[log] dir`) | +| `LOG_DIR` | `/logs` | Log directory when `output = file` (`[log] dir`) | -**Note**: `HMAC_SECRET` is auto-generated and persisted to -`~/.vcms/secrets.toml` on first run. Set it explicitly via env to override. +**Note**: `HMAC_SECRET` is auto-generated and persisted to the config dir's +`secrets.toml` on first run. Set it explicitly via env to override. ## Proto Compilation @@ -298,23 +325,28 @@ is **derived** and fully rebuildable. scalar text of `data`, English-stemmed, BM25-ranked). `fields_from` re-resolves the schema when opening an existing index read-only. - `mod.rs` — `SearchService`. **Reading** needs no lock: `open_read_only` (reader - only) lets *any* process search the index, including a separate `vcms mcp stdio` - running next to the server. **Writing** requires the directory lock: `open` + only) lets *any* auxiliary process search the index alongside the server (this is a + general capability; `vcms mcp stdio` itself no longer opens the index — it proxies + over HTTP). **Writing** requires the directory lock: `open` (reader + writer) is held only by the running server. `index_doc`/`delete_doc` stage uncommitted ops; `commit` flushes; `rebuild_all`/`rebuild_site` reindex from the DB by reusing `EntryRepository::list` (covers singletons too). Write/commit on a read-only instance returns `SearchError::ReadOnly`. - `queue.rs` — `SearchQueue` over the `search_index_queue` table (migration - `20260618000000`). Content writes from *any* process enqueue here + `20260618000000`). Content writes enqueue here (`enqueue`/`dequeue_batch`/`delete_ids`); UUIDv7 ids order the queue chronologically. + The table exists purely for **durability** (crash recovery), not cross-process + signaling — all producers run in the server process (`vcms mcp stdio` is an HTTP proxy). - `indexer.rs` — the **single consumer**, spawned once by the server (it owns the writer). Drains the queue in batches (present in DB ⇒ upsert doc, absent ⇒ delete - doc — `op` is advisory), commits per batch, deletes processed rows. Wakes instantly - on a local enqueue (`Notify`) and polls every 2s to catch other processes' enqueues. + doc — `op` is advisory), commits per batch, deletes processed rows. **Purely + event-driven**: one startup drain (rows left by a crash), then sleeps until an + enqueue rings the in-process `Notify` — no polling. Wiring: `Services` holds `search: Option>` (reads) and `search_queue: Option>` (writes). `Services::new` opens the index -read-write (server); `Services::new_read_only` opens it read-only (`vcms mcp stdio`). +read-write (server); `Services::new_read_only` opens it read-only for an auxiliary +process that searches without taking the writer lock. `EntryService`/`SingletonService` **enqueue** on write; `EntryService::list_entries` queries the index — so REST, GraphQL, gRPC, and MCP all get ranked search via the existing `search` param with **no handler changes**. Indexing is asynchronous @@ -324,10 +356,9 @@ index builds on startup when empty, rebuilds after a restore, and exposes owner/operator reindex routes: `POST /api/dashboard/instance/search/reindex` and `POST /api/dashboard/sites/{site_id}/search/reindex`. -This is the cross-process model: writes from `vcms mcp stdio` (or any process) land in -the durable queue and the running server indexes them; if the server is down they -drain on its next start. The remaining hard limit is *concurrent writers* — only one -process indexes at a time. +Writes land in the durable queue and the running server indexes them; rows enqueued +before a crash drain on the next start. The remaining hard limit is *concurrent +writers* — only one process indexes at a time. ### No typo tolerance (deliberate) — and how to add it diff --git a/Cargo.lock b/Cargo.lock index ac72d5f2..5a6fd5c6 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -16,34 +16,34 @@ checksum = "320119579fcad9c21884f5c4861d16174d0e06250625266f50fe6898340abefa" [[package]] name = "aead" -version = "0.5.2" +version = "0.6.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d122413f284cf2d62fb1b7db97e02edb8cda96d769b16e443a4f6195e35662b0" +checksum = "1973cfbc1a2daf9cf550e74e1f088c28e7f7d8c1e1418fb6c9dc5184b7e84c99" dependencies = [ - "crypto-common 0.1.7", - "generic-array", + "crypto-common 0.2.2", + "inout", ] [[package]] name = "aes" -version = "0.8.4" +version = "0.9.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b169f7a6d4742236a0a00c541b845991d0ac43e546831af1249753ab4c3aa3a0" +checksum = "f1fc76eaeac4c9164506c466d4ffdd8ec9d0c5bf57ee97177c4d8eceb3a0e138" dependencies = [ - "cfg-if", - "cipher 0.4.4", - "cpufeatures 0.2.17", + "cipher", + "cpubits", + "cpufeatures 0.3.0", ] [[package]] name = "aes-gcm" -version = "0.10.3" +version = "0.11.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "831010a0f742e1209b3bcea8fab6a8e149051ba6099432c8cb2cc117dec3ead1" +checksum = "fdf011db2e21ce0d575593d749db5554b47fed37aff429e4dc50bc91ac93a028" dependencies = [ "aead", "aes", - "cipher 0.4.4", + "cipher", "ctr", "ghash", "subtle", @@ -143,9 +143,9 @@ dependencies = [ [[package]] name = "anyhow" -version = "1.0.102" +version = "1.0.103" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7f202df86484c868dbad7eaa557ef785d5c66295e41b460ef922eca0723b842c" +checksum = "2a4385e2e34eb35d6b3efe798b9eb88096925d87726c0798709bf56d9ed84af3" [[package]] name = "arbitrary" @@ -155,9 +155,9 @@ checksum = "c3d036a3c4ab069c7b410a2ce876bd74808d2d0888a82667669f8e783a898bf1" [[package]] name = "arc-swap" -version = "1.9.1" +version = "1.9.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6a3a1fd6f75306b68087b831f025c712524bcb19aad54e557b1129cfa0a2b207" +checksum = "c049c0be4daef0b145cb3555416b3b8ef5b7888a38aea1a3a155801fe7b0810b" dependencies = [ "rustversion", ] @@ -413,9 +413,9 @@ dependencies = [ [[package]] name = "aws-lc-rs" -version = "1.17.0" +version = "1.17.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "5ec2f1fc3ec205783a5da9a7e6c1509cc69dedf09a1949e412c1e18469326d00" +checksum = "4342d8937fc7e5dd9b1c60292261c0670c882a2cd1719cfc11b1af41731e32ad" dependencies = [ "aws-lc-sys", "zeroize", @@ -423,14 +423,15 @@ dependencies = [ [[package]] name = "aws-lc-sys" -version = "0.41.0" +version = "0.42.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1a2f9779ce85b93ab6170dd940ad0169b5766ff848247aff13bb788b832fe3f4" +checksum = "6d9ceb1da931507a12f4fccea479dccd00da1943e1b4ae72d8e502d707361444" dependencies = [ "cc", "cmake", "dunce", "fs_extra", + "pkg-config", ] [[package]] @@ -583,7 +584,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "62ce3946557b35e71d1bbe07ec385073ce9eda05043f95de134eb578fcf1a298" dependencies = [ "byteorder", - "cipher 0.5.2", + "cipher", ] [[package]] @@ -668,6 +669,17 @@ version = "0.4.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "4f4c707c6a209cbe82d10abd08e1ea8995e9ea937d2550646e02798948992be0" +[[package]] +name = "cfb" +version = "0.7.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d38f2da7a0a2c4ccf0065be06397cc26a81f4e528be095826eee9d4adbb8c60f" +dependencies = [ + "byteorder", + "fnv", + "uuid", +] + [[package]] name = "cfg-if" version = "1.0.4" @@ -705,24 +717,15 @@ dependencies = [ "windows-link", ] -[[package]] -name = "cipher" -version = "0.4.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "773f3b9af64447d2ce9850330c473515014aa235e6a783b02db81ff39e4a3dad" -dependencies = [ - "crypto-common 0.1.7", - "inout 0.1.4", -] - [[package]] name = "cipher" version = "0.5.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e8cf2a2c93cd704877c0858356ed03480ff301ee950b43f1cbe4573b088bfa6c" dependencies = [ + "block-buffer 0.12.1", "crypto-common 0.2.2", - "inout 0.2.2", + "inout", ] [[package]] @@ -802,18 +805,21 @@ dependencies = [ "dotenvy", "email_address", "figment", + "futures-util", "hex", "hmac", "http", "http-body-util", "image", + "infer", "json-patch", + "libc", "mime_guess", "object_store", "prost", "rand 0.10.1", "regex", - "reqwest 0.13.4", + "reqwest", "rmcp", "rust-embed", "schemars", @@ -846,6 +852,7 @@ dependencies = [ "utoipa", "utoipa-scalar", "uuid", + "windows-service", "wiremock", "zstd", ] @@ -942,6 +949,12 @@ version = "0.8.7" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b" +[[package]] +name = "cpubits" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "15b85f9c39137c3a891689859392b1bd49812121d0d61c9caf00d46ed5ce06ae" + [[package]] name = "cpufeatures" version = "0.2.17" @@ -975,6 +988,16 @@ version = "2.5.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "217698eaf96b4a3f0bc4f3662aaa55bdf913cd54d7204591faa790070c6d0853" +[[package]] +name = "crc-fast" +version = "1.10.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e75b2483e97a5a7da73ac68a05b629f9c53cff58d8ed1c77866079e18b00dba5" +dependencies = [ + "digest 0.10.7", + "spin 0.10.0", +] + [[package]] name = "crc32fast" version = "1.5.0" @@ -1046,12 +1069,11 @@ checksum = "460fbee9c2c2f33933d720630a6a0bac33ba7053db5344fac858d4b8952d77d5" [[package]] name = "crypto-common" -version = "0.1.7" +version = "0.1.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a" +checksum = "1bfb12502f3fc46cca1bb51ac28df9d618d813cdc3d2f25b9fe775a34af26bb3" dependencies = [ "generic-array", - "rand_core 0.6.4", "typenum", ] @@ -1061,16 +1083,18 @@ version = "0.2.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ce6e4c961d6cd6c9a86db418387425e8bdeaf05b3c8bc1411e6dca4c252f1453" dependencies = [ + "getrandom 0.4.3", "hybrid-array", + "rand_core 0.10.1", ] [[package]] name = "ctr" -version = "0.9.2" +version = "0.10.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0369ee1ad671834580515889b80f2ea915f23b8be8d0daa4bbaf2ac5c7590835" +checksum = "baaca1c4b237092596f64d571e9db6ce4109c4ef9742e27590f1709594461f21" dependencies = [ - "cipher 0.4.4", + "cipher", ] [[package]] @@ -1242,7 +1266,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292" dependencies = [ "block-buffer 0.10.4", - "crypto-common 0.1.7", + "crypto-common 0.1.6", ] [[package]] @@ -1501,7 +1525,7 @@ checksum = "5e139bc46ca777eb5efaf62df0ab8cc5fd400866427e56c68b22e414e53bd3be" dependencies = [ "futures-core", "futures-sink", - "spin", + "spin 0.9.8", ] [[package]] @@ -1658,9 +1682,9 @@ dependencies = [ [[package]] name = "generic-array" -version = "0.14.7" +version = "0.14.9" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "85649ca51fd72272d7821adaf274ad91c288277713d9c18820d8499a7ff69e9a" +checksum = "4bb6743198531e02858aeaea5398fcc883e71851fcbcb5a2f773e2fb6cb1edf2" dependencies = [ "typenum", "version_check", @@ -1707,11 +1731,10 @@ dependencies = [ [[package]] name = "ghash" -version = "0.5.1" +version = "0.6.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f0d8a4362ccb29cb0b265253fb0a2728f592895ee6854fd9bc13f2ffda266ff1" +checksum = "2eecf2d5dc9b66b732b97707a0210906b1d30523eb773193ab777c0c84b3e8d5" dependencies = [ - "opaque-debug", "polyval", ] @@ -1906,9 +1929,9 @@ checksum = "135b12329e5e3ce057a9f972339ea52bc954fe1e9358ef27f95e89716fbc5424" [[package]] name = "hybrid-array" -version = "0.4.12" +version = "0.4.13" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9155a582abd142abc056962c29e3ce5ff2ad5469f4246b537ed42c5deba857da" +checksum = "818356c5132c1fede50f837ca96afbe78ff42413047f4abb886217845e1b6c8c" dependencies = [ "typenum", ] @@ -1945,7 +1968,6 @@ dependencies = [ "hyper", "hyper-util", "rustls", - "rustls-native-certs", "tokio", "tokio-rustls", "tower-service", @@ -2171,19 +2193,19 @@ dependencies = [ ] [[package]] -name = "inlinable_string" -version = "0.1.15" +name = "infer" +version = "0.19.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "c8fae54786f62fb2918dcfae3d568594e50eb9b5c25bf04371af6fe7516452fb" +checksum = "a588916bfdfd92e71cacef98a63d9b1f0d74d6599980d11894290e7ddefffcf7" +dependencies = [ + "cfb", +] [[package]] -name = "inout" -version = "0.1.4" +name = "inlinable_string" +version = "0.1.15" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "879f10e63c20629ecabbb64a8010319738c66a5cd0c29b02d63d272b03751d01" -dependencies = [ - "generic-array", -] +checksum = "c8fae54786f62fb2918dcfae3d568594e50eb9b5c25bf04371af6fe7516452fb" [[package]] name = "inout" @@ -2235,6 +2257,15 @@ dependencies = [ "either", ] +[[package]] +name = "itertools" +version = "0.15.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8b4baf93f58d4425749ca49a51c50ebab072c5df6994d08fed93541c331481dc" +dependencies = [ + "either", +] + [[package]] name = "itoa" version = "1.0.18" @@ -2468,16 +2499,6 @@ dependencies = [ "cfg-if", ] -[[package]] -name = "md-5" -version = "0.10.6" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d89e7ee0cfbedfc4da3340218492196241d89eefb6dab27de5df917a6d2e78cf" -dependencies = [ - "cfg-if", - "digest 0.10.7", -] - [[package]] name = "md-5" version = "0.11.0" @@ -2578,7 +2599,7 @@ dependencies = [ "httparse", "memchr", "mime", - "spin", + "spin 0.9.8", "version_check", ] @@ -2600,6 +2621,18 @@ version = "1.0.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "650eef8c711430f1a879fdd01d4745a7deea475becfb90269c06775983bbf086" +[[package]] +name = "nix" +version = "0.31.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf20d2fde8ff38632c426f1165ed7436270b44f199fc55284c38276f9db47c3d" +dependencies = [ + "bitflags", + "cfg-if", + "cfg_aliases", + "libc", +] + [[package]] name = "no_std_io2" version = "0.9.4" @@ -2726,14 +2759,16 @@ dependencies = [ [[package]] name = "object_store" -version = "0.13.2" +version = "0.14.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "622acbc9100d3c10e2ee15804b0caa40e55c933d5aa53814cd520805b7958a49" +checksum = "765784b4390c6bcf80316e5a22f4e3661b639c9d8c83246856643c27d8ce9dbe" dependencies = [ "async-trait", + "aws-lc-rs", "base64", "bytes", "chrono", + "crc-fast", "form_urlencoded", "futures-channel", "futures-core", @@ -2742,14 +2777,15 @@ dependencies = [ "http-body-util", "humantime", "hyper", - "itertools", - "md-5 0.10.6", + "itertools 0.15.0", + "md-5", + "nix", "parking_lot", "percent-encoding", "quick-xml", "rand 0.10.1", - "reqwest 0.12.28", - "ring", + "reqwest", + "rustls-pki-types", "serde", "serde_json", "serde_urlencoded", @@ -2760,6 +2796,7 @@ dependencies = [ "walkdir", "wasm-bindgen-futures", "web-time", + "windows-sys 0.61.2", ] [[package]] @@ -2780,12 +2817,6 @@ version = "0.1.13" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "269bca4c2591a28585d6bf10d9ed0332b7d76900a1b02bec41bdc3a2cdcda107" -[[package]] -name = "opaque-debug" -version = "0.3.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "c08d65885ee38876c4f86fa503fb49d7b507c2b62552df7c70b2fce627e06381" - [[package]] name = "openssl-probe" version = "0.2.1" @@ -3007,13 +3038,12 @@ dependencies = [ [[package]] name = "polyval" -version = "0.6.2" +version = "0.7.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9d1fe60d06143b2430aa532c94cfe9e29783047f06c0d7fd359a9a51b729fa25" +checksum = "7dfc63250416fea14f5749b90725916a6c903f599d51cb635aa7a52bfd03eede" dependencies = [ - "cfg-if", - "cpufeatures 0.2.17", - "opaque-debug", + "cpubits", + "cpufeatures 0.3.0", "universal-hash", ] @@ -3118,7 +3148,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "03da047801ff44bb6a4d407d4860c05fd70bb81714e6b2f3812603d5b145b042" dependencies = [ "heck", - "itertools", + "itertools 0.14.0", "log", "multimap", "petgraph", @@ -3139,7 +3169,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b570b25f7617e43d59005d0990ccb79e950a423952cea19671b7a876da390adf" dependencies = [ "anyhow", - "itertools", + "itertools 0.14.0", "proc-macro2", "quote", "syn", @@ -3204,9 +3234,9 @@ checksum = "a993555f31e5a609f617c12db6250dedcac1b0a85076912c436e6fc9b2c8e6a3" [[package]] name = "quick-xml" -version = "0.39.4" +version = "0.40.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "cdcc8dd4e2f670d309a5f0e83fe36dfdc05af317008fea29144da1a2ac858e5e" +checksum = "2474bd2e5029e7ccb6abb2ba48cf2383a333851dedf495901544281590c7da7f" dependencies = [ "memchr", "serde", @@ -3320,15 +3350,6 @@ dependencies = [ "rand_core 0.9.5", ] -[[package]] -name = "rand_core" -version = "0.6.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ec0be4795e2f6a28069bec0b5ff3e2ac9bafc99e6a9a7dc3547996c5c816922c" -dependencies = [ - "getrandom 0.2.17", -] - [[package]] name = "rand_core" version = "0.9.5" @@ -3360,7 +3381,7 @@ dependencies = [ "built", "cfg-if", "interpolate_name", - "itertools", + "itertools 0.14.0", "libc", "libfuzzer-sys", "log", @@ -3482,48 +3503,6 @@ version = "0.8.11" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4" -[[package]] -name = "reqwest" -version = "0.12.28" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "eddd3ca559203180a307f12d114c268abf583f59b03cb906fd0b3ff8646c1147" -dependencies = [ - "base64", - "bytes", - "futures-core", - "futures-util", - "h2", - "http", - "http-body", - "http-body-util", - "hyper", - "hyper-rustls", - "hyper-util", - "js-sys", - "log", - "percent-encoding", - "pin-project-lite", - "quinn", - "rustls", - "rustls-native-certs", - "rustls-pki-types", - "serde", - "serde_json", - "serde_urlencoded", - "sync_wrapper", - "tokio", - "tokio-rustls", - "tokio-util", - "tower", - "tower-http 0.6.11", - "tower-service", - "url", - "wasm-bindgen", - "wasm-bindgen-futures", - "wasm-streams", - "web-sys", -] - [[package]] name = "reqwest" version = "0.13.4" @@ -3559,12 +3538,14 @@ dependencies = [ "sync_wrapper", "tokio", "tokio-rustls", + "tokio-util", "tower", "tower-http 0.6.11", "tower-service", "url", "wasm-bindgen", "wasm-bindgen-futures", + "wasm-streams", "web-sys", ] @@ -3590,9 +3571,9 @@ dependencies = [ [[package]] name = "rmcp" -version = "1.8.0" +version = "2.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1d1f571c72940a19d9532fe52dbea8bc9912bf1d766c2970bb824056b86f3f59" +checksum = "d52d21e5b342699bc4de690e6104fc4e43255e4e8420ff0f2cbb963aac09da6f" dependencies = [ "async-trait", "base64", @@ -3621,9 +3602,9 @@ dependencies = [ [[package]] name = "rmcp-macros" -version = "1.8.0" +version = "2.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1aad0035b69380782d78ea95b508327e6deaa2235909053e596eea8f27b5e1d5" +checksum = "6c68cec74c5b3ac73ff46375ae49e161637bda80bba70f0d5db641583bb308ee" dependencies = [ "darling 0.23.0", "proc-macro2", @@ -3712,7 +3693,6 @@ checksum = "6b92b125634d9b795e7beca796cc790df15a7fb38323bf3196fda83292d06b1f" dependencies = [ "aws-lc-rs", "once_cell", - "ring", "rustls-pki-types", "rustls-webpki", "subtle", @@ -3733,9 +3713,9 @@ dependencies = [ [[package]] name = "rustls-pki-types" -version = "1.14.1" +version = "1.15.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "30a7197ae7eb376e574fe940d068c30fe0462554a3ddbe4eca7838e049c937a9" +checksum = "764899a24af3980067ee14bc143654f297b22eaebfe3c7b6b211920a5a59b046" dependencies = [ "web-time", "zeroize", @@ -4110,6 +4090,12 @@ dependencies = [ "lock_api", ] +[[package]] +name = "spin" +version = "0.10.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d5fe4ccb98d9c292d56fec89a5e07da7fc4cf0dc11e156b41793132775d3e591" + [[package]] name = "sqlx" version = "0.9.0" @@ -4248,7 +4234,7 @@ dependencies = [ "hmac", "itoa", "log", - "md-5 0.11.0", + "md-5", "memchr", "rand 0.10.1", "serde", @@ -4437,7 +4423,7 @@ dependencies = [ "fnv", "fs4", "htmlescape", - "itertools", + "itertools 0.14.0", "levenshtein_automata", "log", "lru", @@ -4486,7 +4472,7 @@ checksum = "c57166f5bcfd478f370ab8445afb4678dce44801fa5ce5c451aaf8595583c5dc" dependencies = [ "downcast-rs", "fastdivide", - "itertools", + "itertools 0.14.0", "serde", "tantivy-bitpacker", "tantivy-common", @@ -4538,7 +4524,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8a2cfc3ac5164cbadc28965ffb145a8f47582a60ae5897859ad8d4316596c606" dependencies = [ "futures-util", - "itertools", + "itertools 0.14.0", "tantivy-bitpacker", "tantivy-common", "tantivy-fst", @@ -4619,9 +4605,9 @@ dependencies = [ [[package]] name = "time" -version = "0.3.51" +version = "0.3.53" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "85c17d80feb7334b40c484e45ed1a5273dfd8bfda537c3be2e74a06a6686f327" +checksum = "18dfaaeddcb932337b5e7866ee7d0ce9b76d2fd092997146f187ec09b4558a50" dependencies = [ "deranged", "num-conv", @@ -4639,9 +4625,9 @@ checksum = "9e1c906769ad99c88eaa54e728060edef082f8e358ff32030cb7c7d315e81109" [[package]] name = "time-macros" -version = "0.2.30" +version = "0.2.31" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "dcef1a61bdb119096e153208ec5cbec23944ce8bca13be5c7f60c634f7403935" +checksum = "c431b87111666e491a90baa837f914fb45cd5dc3c268591b0220ff5057f2085f" dependencies = [ "num-conv", "time-core", @@ -5211,12 +5197,12 @@ checksum = "7df058c713841ad818f1dc5d3fd88063241cc61f49f5fbea4b951e8cf5a8d71d" [[package]] name = "universal-hash" -version = "0.5.1" +version = "0.6.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "fc1de2c688dc15305988b563c3854064043356019f97a4b46276fe734c4f07ea" +checksum = "f4987bdc12753382e0bec4a65c50738ffaabc998b9cdd1f952fb5f39b0048a96" dependencies = [ - "crypto-common 0.1.7", - "subtle", + "crypto-common 0.2.2", + "ctutils", ] [[package]] @@ -5423,9 +5409,9 @@ dependencies = [ [[package]] name = "wasm-streams" -version = "0.4.2" +version = "0.5.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "15053d8d85c7eccdbefef60f06769760a563c7f0a9d6902a13d35c7800b0ad65" +checksum = "9d1ec4f6517c9e11ae630e200b2b65d193279042e28edd4a2cda233e46670bbb" dependencies = [ "futures-util", "js-sys", @@ -5475,6 +5461,12 @@ version = "2.1.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "998767ef88740d1f5b0682a9c53c24431453923962269c2db68ee43788c5a40d" +[[package]] +name = "widestring" +version = "1.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "72069c3113ab32ab29e5584db3c6ec55d416895e60715417b5b883a357c3e471" + [[package]] name = "winapi" version = "0.3.9" @@ -5567,6 +5559,17 @@ dependencies = [ "windows-link", ] +[[package]] +name = "windows-service" +version = "0.8.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "857224b3b211c6f3616921f081ee54721ee3ad2ace2fac6a6337e032f7b4dcf2" +dependencies = [ + "bitflags", + "widestring", + "windows-sys 0.61.2", +] + [[package]] name = "windows-strings" version = "0.5.1" diff --git a/README.md b/README.md index 7919d30b..28f87a3f 100644 --- a/README.md +++ b/README.md @@ -89,51 +89,59 @@ password after your first login.* ### MCP over stdio -Run a standalone MCP process for clients that launch local stdio servers: +For clients that launch a local stdio MCP server, run `vcms mcp stdio`. It is a +**thin proxy** to a running server's `/mcp` endpoint — it opens no database, secrets, +or search index of its own. It forwards JSON-RPC between stdin/stdout and the server +over HTTP, so it keeps working even when the data is owned by a system-service account +that the client process can't read. ```bash -VCMS_MCP_TOKEN=vcms_site_... vcms mcp stdio +VCMS_MCP_TOKEN=vcms_site_... VCMS_MCP_URL=http://127.0.0.1:3000 vcms mcp stdio ``` -The command connects to an existing CMS database and never starts HTTP/gRPC -listeners, seeds users, creates a SQLite database, or runs migrations. Run -`vcms serve` first when the database schema needs initialization or migration. -MCP protocol messages use stdout; structured process logs use stderr. - -Because the process is launched by the MCP client from an arbitrary working -directory, it does **not** rely on a `.env` in the current directory. The -database path and the `HMAC_SECRET` it needs to verify the token are -read from the CMS home directory (`~/.vcms`, see [Data directory](#data-directory)) -that `vcms serve` initialized. The client only needs to supply `VCMS_MCP_TOKEN` -(and `VCMS_HOME` if you moved the home directory): +It needs only two env vars: `VCMS_MCP_TOKEN` (a `vcms_site_*` access token, forwarded +as the `Authorization: Bearer` credential) and `VCMS_MCP_URL` (the running server's +base URL, default `http://127.0.0.1:3000`; the proxy posts to `{url}/mcp`). A `vcms +serve` instance must be running. MCP protocol messages use stdout; logs use stderr. ```jsonc // Example MCP client config { "command": "vcms", "args": ["mcp", "stdio"], - "env": { "VCMS_MCP_TOKEN": "vcms_site_..." } + "env": { "VCMS_MCP_TOKEN": "vcms_site_...", "VCMS_MCP_URL": "http://127.0.0.1:3000" } } ``` ### Data directory -All runtime files live under a single home directory so a fresh install works -from any working directory. The location is `$VCMS_HOME` when set, otherwise -`~/.vcms` (resolved cross-platform — same layout on Windows, macOS, and Linux): - -```text -~/.vcms/ - config.toml # non-secret configuration (vcms config init writes here) - secrets.toml # auto-generated HMAC_SECRET + backup key (0600 on unix) - vcms.db # default SQLite database (+ -wal / -shm) - logs/ # rolling logs when [log] output = "file" - storage/ # default filesystem storage for uploads -``` - -`vcms serve` creates this directory on first run and generates `secrets.toml` if -absent. Environment variables (`DATABASE_URL`, `HMAC_SECRET`, -`STORAGE_FS_PATH`, S3 settings, …) still override these defaults. +By default, runtime files go to the platform-conventional per-type directories +(resolved cross-platform via the `directories` crate): + +| File(s) | Linux | macOS | Windows | +|---------|-------|-------|---------| +| `config.toml`, `secrets.toml`, `.env` | `~/.config/vcms` | `~/Library/Application Support/vcms` | `%APPDATA%\vcms\config` | +| `vcms.db`, `storage/`, `backups/` | `~/.local/share/vcms` | `~/Library/Application Support/vcms` | `%APPDATA%\vcms\data` | +| `search/` (rebuildable index) | `~/.cache/vcms` | `~/Library/Caches/vcms` | `%LOCALAPPDATA%\vcms\cache` | +| `logs/` | `~/.local/state/vcms` | `~/Library/Application Support/vcms` | `%LOCALAPPDATA%\vcms\data` | + +Set **`$VCMS_HOME`** to keep everything under a single root instead. When several +locations exist, the first match wins: **`$VCMS_HOME`** → the service's system +directory (below) → a legacy `~/.vcms` (honored automatically, so upgrades don't +move your data) → the split per-type defaults above. + +The `vcms service` installer stores the daemon's data under one system dir — +`/var/lib/vcms` (Linux), `/Library/Application Support/vcms` (macOS), or +`C:\ProgramData\vcms` (Windows). **Once that directory exists, the CLI uses it too**: +a plain `vcms serve`/`admin`/`backup` — even after the service is stopped or +uninstalled — targets the *same* store rather than a separate per-user copy, so you +never end up with two sets of data. Because that directory is owned by SYSTEM/root, +run those commands from an elevated (Administrator/`sudo`) terminal; a non-elevated +run fails with a clear hint instead of silently creating a second store. + +`vcms serve` creates what it needs on first run and generates `secrets.toml` if +absent. Environment variables (`DATABASE_URL`, `HMAC_SECRET`, `STORAGE_FS_PATH`, +S3 settings, …) still override these defaults. diff --git a/apps/backend/Cargo.toml b/apps/backend/Cargo.toml index 137c47a0..1dcbb4ef 100644 --- a/apps/backend/Cargo.toml +++ b/apps/backend/Cargo.toml @@ -38,8 +38,10 @@ utoipa = { version = "5", features = ["axum_extras"] } utoipa-scalar = { version = "0.3", features = ["axum"] } dotenvy = "0.15" image = { version = "0.25", default-features = false, features = ["png", "jpeg", "webp", "gif", "avif"] } -object_store = { version = "0.13", features = ["aws"] } +object_store = { version = "0.14", features = ["aws"] } bytes = "1" +futures-util = "0.3" +infer = "0.19" http = "1" regex = "1" json-patch = "4" @@ -53,7 +55,7 @@ hmac = "0.13" hex = "0.4" cookie = "0.18" base64 = "0.22" -aes-gcm = "0.10" +aes-gcm = "0.11" rand = "0.10" dashmap = "6" url = "2" @@ -65,8 +67,9 @@ tonic-prost = "0.14.5" tonic-reflection = "0.14" tonic-health = "0.14" http-body-util = "0.1.3" -rmcp = { version = "1.6", features = ["server", "transport-io", "transport-streamable-http-server", "macros", "schemars"] } +rmcp = { version = "2.0", features = ["server", "transport-io", "transport-streamable-http-server", "macros", "schemars"] } tokio-util = { version = "0.7", features = ["rt"] } +tokio-stream = { version = "0.1", features = ["net"] } schemars = "1" clap = { version = "4.6.1", features = ["derive", "env"] } figment = { version = "0.10.19", features = ["toml", "env"] } @@ -77,6 +80,14 @@ tar = "0.4" croner = "3" tantivy = "0.26" +# Unix: effective-uid check for the `service` elevation guard. +[target.'cfg(unix)'.dependencies] +libc = "0.2" + +# Windows: native Service Control Manager integration for `vcms service`. +[target.'cfg(windows)'.dependencies] +windows-service = "0.8" + [build-dependencies] tonic-build = "0.14" tonic-prost-build = "0.14" @@ -84,5 +95,4 @@ tonic-prost-build = "0.14" [dev-dependencies] tempfile = "3" tokio-test = "0.4" -tokio-stream = { version = "0.1", features = ["net"] } wiremock = "0.6" diff --git a/apps/backend/src/cli.rs b/apps/backend/src/cli.rs index 5a0eff60..63a5c2e0 100644 --- a/apps/backend/src/cli.rs +++ b/apps/backend/src/cli.rs @@ -9,18 +9,18 @@ use clap::{Parser, Subcommand}; version, about = "Headless CMS server", long_about = "Headless CMS server.\n\n\ - All runtime files live under one home directory ($VCMS_HOME, default ~/.vcms): \ - config.toml, secrets.toml, the SQLite database, logs/, and storage/. \ - `vcms serve` creates it on first run and generates secrets if absent.", - after_help = "DATA DIRECTORY ($VCMS_HOME, default ~/.vcms):\n \ - config.toml non-secret config (written by `vcms config init`)\n \ - secrets.toml auto-generated HMAC + backup secrets (0600 on unix)\n \ - vcms.db default SQLite database (+ -wal / -shm)\n \ - logs/ rolling logs when [log] output = \"file\"\n \ - storage/ default filesystem storage for uploads\n\n\ + Runtime files default to the platform's per-type directories (config, data, \ + cache, state) via the `directories` crate. Set $VCMS_HOME to keep everything \ + under one root instead (the `vcms service` installer pins it to a system dir). \ + `vcms serve` creates what it needs on first run and generates secrets if absent.", + after_help = "DATA DIRECTORIES (defaults; set $VCMS_HOME for a single root):\n \ + config dir config.toml, secrets.toml (0600 on unix), .env\n \ + data dir vcms.db (+ -wal / -shm), storage/, backups/\n \ + cache dir search/ (derived Tantivy index, rebuildable)\n \ + state dir logs/ (when [log] output = \"file\")\n\n\ KEY ENVIRONMENT (env overrides config; CLI flags override env):\n \ - VCMS_HOME home directory [default: ~/.vcms]\n \ - DATABASE_URL sqlite/postgres/mysql URL [default: sqlite://~/.vcms/vcms.db]\n \ + VCMS_HOME force single-root layout [default: platform split dirs]\n \ + DATABASE_URL sqlite/postgres/mysql URL [default: sqlite:///vcms.db]\n \ HMAC_SECRET token-lookup HMAC key [auto-generated to secrets.toml]" )] pub struct Cli { @@ -33,7 +33,7 @@ pub struct Cli { pub bind: Option, /// Database URL, e.g. sqlite:path / postgres://… (overrides config + env) - /// [default: sqlite://~/.vcms/vcms.db]. + /// [default: sqlite:///vcms.db]. #[arg(long, global = true, value_name = "URL")] pub database_url: Option, @@ -69,6 +69,11 @@ pub enum Command { #[command(subcommand)] action: BackupAction, }, + /// Manage the OS background service (systemd / launchd / Windows SCM). + Service { + #[command(subcommand)] + action: ServiceAction, + }, /// Restore a backup artifact (runs offline; destructive — replaces data in scope). Restore { /// Path to the backup artifact (`.cmsbak`). @@ -114,6 +119,39 @@ pub enum BackupAction { List, } +#[derive(Subcommand, Debug)] +pub enum ServiceAction { + /// Install the service, enable it at boot, and start it now (requires root/admin). + /// + /// The service runs `vcms serve` as the chosen OS account; `VCMS_HOME` is pinned + /// to a system dir (Linux `/var/lib/vcms`, macOS `/Library/Application Support/vcms`, + /// Windows `C:\ProgramData\vcms`) so all runtime files live under one owned root. + Install { + /// OS account the service runs as. + /// + /// Defaults to the real invoking user (`$SUDO_USER` when run via sudo). The + /// service never runs as root. On Windows the service always runs as + /// LocalSystem; custom accounts are unsupported and passing `--user` fails. + #[arg(long, value_name = "NAME")] + user: Option, + }, + /// Stop, disable, and remove the service (requires root/admin). + Uninstall, + /// Show whether the service is installed, enabled at boot, and running. + Status, + /// Start the installed service. + Start, + /// Stop the running service. + Stop, + /// Internal entry point invoked by the Windows Service Control Manager. + /// + /// Not for direct use — the SCM launches this to host the server inside a + /// Windows service. Hidden from `--help`. + #[cfg(windows)] + #[command(hide = true)] + Run, +} + #[derive(Subcommand, Debug)] pub enum McpTransport { /// Run MCP over stdin/stdout. @@ -163,4 +201,48 @@ mod tests { }) )); } + + #[test] + fn parses_service_install_with_user() { + use super::ServiceAction; + let cli = + Cli::try_parse_from(["vcms", "service", "install", "--user", "deploy"]).expect("command should parse"); + match cli.command { + Some(Command::Service { + action: ServiceAction::Install { user }, + }) => assert_eq!(user.as_deref(), Some("deploy")), + other => panic!("unexpected command: {other:?}"), + } + } + + #[test] + fn parses_service_lifecycle_subcommands() { + use super::ServiceAction; + for (args, ok) in [ + ( + ["service", "status"], + matches!(parse_action(&["service", "status"]), ServiceAction::Status), + ), + ( + ["service", "start"], + matches!(parse_action(&["service", "start"]), ServiceAction::Start), + ), + ( + ["service", "stop"], + matches!(parse_action(&["service", "stop"]), ServiceAction::Stop), + ), + ] { + assert!(ok, "{args:?} did not parse to the expected ServiceAction"); + } + } + + #[cfg(test)] + fn parse_action(args: &[&str]) -> super::ServiceAction { + let mut full = vec!["vcms"]; + full.extend_from_slice(args); + match Cli::try_parse_from(full).expect("parse").command { + Some(Command::Service { action }) => action, + other => panic!("unexpected: {other:?}"), + } + } } diff --git a/apps/backend/src/config.rs b/apps/backend/src/config.rs index 60df4d41..b5865f0c 100644 --- a/apps/backend/src/config.rs +++ b/apps/backend/src/config.rs @@ -44,6 +44,8 @@ pub struct Config { // Upload limits pub max_upload_size_bytes: usize, + /// Lifetime of signed upload URLs (seconds). + pub upload_token_expiry_secs: i64, // Cookie security pub cookie_secure: bool, @@ -129,6 +131,7 @@ struct RawConfig { backup_encryption_key: Option, max_upload_size_mb: Option, + upload_token_expiry_secs: Option, #[serde(default, deserialize_with = "de_opt_lenient_bool")] cookie_secure: Option, @@ -185,8 +188,8 @@ impl Config { pub fn load(cli: &Cli) -> Result> { let mut figment = Figment::new(); - // Lowest-precedence secret layer: persisted HMAC secret from - // `~/.vcms/secrets.toml`. Read-only here (generation happens in + // Lowest-precedence secret layer: persisted HMAC secret from the config dir's + // `secrets.toml`. Read-only here (generation happens in // `secrets::ensure()` during serve/admin). Best-effort: a missing or // unreadable file just leaves the built-in defaults in play. Env vars // and CLI flags still override these. @@ -285,7 +288,8 @@ impl Config { s3_endpoint = {}\n\ s3_public_url = {}\n\n\ # Uploads\n\ - max_upload_size_mb = {}\n\n\ + max_upload_size_mb = {}\n\ + upload_token_expiry_secs = {}\n\n\ # Security / sessions\n\ cookie_secure = {}\n\ session_lifetime_hours = {}\n\ @@ -326,6 +330,7 @@ impl Config { opt_str(&self.s3_endpoint), opt_str(&self.s3_public_url), self.max_upload_size_bytes / (1024 * 1024), + self.upload_token_expiry_secs, self.cookie_secure, self.session_lifetime_hours, self.public_registration_enabled, @@ -357,15 +362,23 @@ impl RawConfig { // `secrets::ensure()` and `mcp stdio` guards on its presence, so this only // fires on a genuinely uninitialized instance. let hmac_secret = self.hmac_secret.ok_or_else(|| { - Box::new(figment::Error::from( - "No HMAC secret resolved; run `vcms serve` once to generate ~/.vcms/secrets.toml, \ - or set HMAC_SECRET" - .to_string(), - )) + Box::new(figment::Error::from(format!( + "No HMAC secret resolved; run `vcms serve` once to generate {}, or set HMAC_SECRET", + paths::secrets_file().display(), + ))) })?; let log = self.log.unwrap_or_default(); + let upload_token_expiry_secs = self + .upload_token_expiry_secs + .unwrap_or(crate::signed_upload::DEFAULT_UPLOAD_TOKEN_EXPIRY_SECS); + if upload_token_expiry_secs <= 0 { + return Err(Box::new(figment::Error::from(format!( + "upload_token_expiry_secs must be positive (got {upload_token_expiry_secs})" + )))); + } + Ok(Config { database_url: self.database_url.unwrap_or_else(paths::default_database_url), hmac_secret, @@ -391,6 +404,7 @@ impl RawConfig { backup_s3_public_url: self.backup_s3_public_url, backup_encryption_key: self.backup_encryption_key, max_upload_size_bytes: self.max_upload_size_mb.unwrap_or(50) * 1024 * 1024, + upload_token_expiry_secs, cookie_secure: self.cookie_secure.unwrap_or(false), session_lifetime_hours: self.session_lifetime_hours.unwrap_or(24), public_registration_enabled: self.public_registration_enabled.unwrap_or(false), @@ -441,8 +455,8 @@ pub fn config_search_paths() -> Vec { paths } -/// The user-config location for the config file: `~/.vcms/config.toml` -/// (or `$VCMS_HOME/config.toml`). +/// The user-config location for the config file: the platform config dir +/// (`config.toml`), or `$VCMS_HOME/config.toml` in single-dir mode. pub fn user_config_path() -> Option { Some(paths::config_file()) } @@ -474,7 +488,9 @@ pub fn default_config_toml() -> String { # s3_endpoint = \"https://s3.example.com\"\n\ # s3_public_url = \"https://cdn.example.com\"\n\n\ # --- Uploads ---\n\ - max_upload_size_mb = 50\n\n\ + max_upload_size_mb = 50\n\ + # Signed upload URL lifetime (seconds)\n\ + upload_token_expiry_secs = {default_upload_expiry}\n\n\ # --- Security / sessions ---\n\ cookie_secure = false\n\ session_lifetime_hours = 24\n\ @@ -498,7 +514,7 @@ pub fn default_config_toml() -> String { # --- Backups ---\n\ # Run the scheduled-backup poller and allow on-demand backups.\n\ backup_enabled = true\n\ - # Destination for backup artifacts: \"filesystem\" (default, under ~/.vcms/backups)\n\ + # Destination for backup artifacts: \"filesystem\" (default, the data dir's backups/)\n\ # or \"s3\". S3 credentials are secrets — set them via env (see below).\n\ backup_destination = \"filesystem\"\n\ # backup_local_path = \"./backups\"\n\ @@ -515,7 +531,7 @@ pub fn default_config_toml() -> String { # Build a local inverted index so entry search is ranked + tokenized.\n\ # When false, search falls back to a basic SQL LIKE match.\n\ search_enabled = true\n\ - # search_index_path = \"./search\" # defaults to ~/.vcms/search\n\n\ + # search_index_path = \"./search\" # defaults to the cache dir's search/\n\n\ # --- MCP ---\n\ mcp_enabled = true\n\ mcp_allowed_hosts = [{hosts}]\n\ @@ -527,6 +543,7 @@ pub fn default_config_toml() -> String { format = \"pretty\" # pretty | json\n\ annotations = false # include file + line numbers\n\ dir = \"logs\" # used when output = \"file\"\n", + default_upload_expiry = crate::signed_upload::DEFAULT_UPLOAD_TOKEN_EXPIRY_SECS, ) } diff --git a/apps/backend/src/database/pool.rs b/apps/backend/src/database/pool.rs index 04c59e27..997d0db0 100644 --- a/apps/backend/src/database/pool.rs +++ b/apps/backend/src/database/pool.rs @@ -49,6 +49,14 @@ impl DbPool { .min_connections(config.db_min_connections) .acquire_timeout(Duration::from_secs(config.db_acquire_timeout_secs)) .idle_timeout(Duration::from_secs(config.db_idle_timeout_secs)) + // Pin the session to UTC so NOW()/CURRENT_TIMESTAMP agree with the + // chrono::Utc timestamps the app writes and compares against. + .after_connect(|conn, _meta| { + Box::pin(async move { + sqlx::query("SET time_zone = '+00:00'").execute(&mut *conn).await?; + Ok(()) + }) + }) .connect(&config.database_url) .await?; Ok(DbPool::MySql(pool)) diff --git a/apps/backend/src/grpc/interceptor.rs b/apps/backend/src/grpc/interceptor.rs index 9ac4b810..de35e06d 100644 --- a/apps/backend/src/grpc/interceptor.rs +++ b/apps/backend/src/grpc/interceptor.rs @@ -89,7 +89,7 @@ async fn validate_auth(ctx: &AuthContext, repository: &Repository) -> Result Result, config: Arc, storage_registry: Arc, - grpc_addr: SocketAddr, + listener: tokio::net::TcpListener, + shutdown: impl Future + Send + 'static, ) -> Result<(), Box> { let collection_svc = CollectionServiceImpl::new(services.collection.clone(), repository.clone()); let entry_svc = EntryServiceImpl::new(services.entry.clone(), repository.clone()); @@ -90,7 +91,7 @@ pub async fn start_grpc_server( .add_service(file_svc) .add_service(site_svc) .add_service(webhook_svc) - .serve(grpc_addr) + .serve_with_incoming_shutdown(TcpListenerStream::new(listener), shutdown) .await?; Ok(()) @@ -101,13 +102,15 @@ pub fn spawn_grpc_server( repository: Arc, config: Arc, storage_registry: Arc, - grpc_addr: SocketAddr, + listener: tokio::net::TcpListener, + shutdown: Pin + Send>>, ) -> GrpcServerFuture { Box::pin(start_grpc_server( services, repository, config, storage_registry, - grpc_addr, + listener, + shutdown, )) } diff --git a/apps/backend/src/handlers/backup_handler.rs b/apps/backend/src/handlers/backup_handler.rs index 90c8cb65..dded2a7b 100644 --- a/apps/backend/src/handlers/backup_handler.rs +++ b/apps/backend/src/handlers/backup_handler.rs @@ -679,7 +679,7 @@ pub async fn inspect_instance_backup( Ok(b) => b, Err(r) => return r, }; - inspect_response(backup.inspect_sites(&bytes), None) + inspect_response(backup.inspect_sites(bytes).await, None) } /// Inspect an uploaded backup file, list its sites, and stage the bytes so the @@ -699,7 +699,7 @@ pub async fn inspect_instance_backup_upload( }; // Validate + read the site list before staging, so a bad file is rejected // without leaving an orphan temp object. - let (manifest, sites) = match backup.inspect_sites(&bytes) { + let (manifest, sites) = match backup.inspect_sites(bytes.clone()).await { Ok(v) => v, Err(e) => return err_response(e), }; diff --git a/apps/backend/src/handlers/dashboard_handler.rs b/apps/backend/src/handlers/dashboard_handler.rs index 1f546f94..a4014468 100644 --- a/apps/backend/src/handlers/dashboard_handler.rs +++ b/apps/backend/src/handlers/dashboard_handler.rs @@ -4,6 +4,8 @@ use axum::{ response::{IntoResponse, Response}, }; +#[cfg(feature = "embed-dashboard")] +use axum::http::HeaderMap; #[cfg(feature = "embed-dashboard")] use mime_guess::from_path; @@ -15,19 +17,61 @@ use rust_embed::RustEmbed; #[folder = "../dashboard/dist"] pub struct Assets; +/// Cache policy for an embedded asset path: Vite content-hashes everything +/// under `assets/`, so those are immutable; `index.html` (and the SPA +/// fallback) must revalidate every time; the rest (favicon etc.) get a short +/// TTL. ETags (embed-time sha256) let revalidations answer with a 304. #[cfg(feature = "embed-dashboard")] -pub async fn dashboard_handler(Path(path): Path) -> Response { +fn cache_control_for(path: &str) -> &'static str { + if path.starts_with("assets/") { + "public, max-age=31536000, immutable" + } else if path == "index.html" { + "no-cache" + } else { + "public, max-age=3600" + } +} + +#[cfg(feature = "embed-dashboard")] +fn serve_embedded(path: &str, content: rust_embed::EmbeddedFile, headers: &HeaderMap) -> Response { + let etag = format!("\"{}\"", hex::encode(content.metadata.sha256_hash())); + let cache_control = cache_control_for(path); + + if headers + .get(header::IF_NONE_MATCH) + .and_then(|v| v.to_str().ok()) + .is_some_and(|inm| inm == etag) + { + return ( + StatusCode::NOT_MODIFIED, + [(header::ETAG, etag), (header::CACHE_CONTROL, cache_control.to_string())], + ) + .into_response(); + } + + let mime = from_path(path).first_or_octet_stream(); + ( + StatusCode::OK, + [ + (header::CONTENT_TYPE, mime.as_ref().to_string()), + (header::ETAG, etag), + (header::CACHE_CONTROL, cache_control.to_string()), + ], + content.data, + ) + .into_response() +} + +#[cfg(feature = "embed-dashboard")] +pub async fn dashboard_handler(Path(path): Path, headers: HeaderMap) -> Response { let requested_path = if path.is_empty() { "index.html" } else { path.as_str() }; match Assets::get(requested_path) { - Some(content) => { - let mime = from_path(requested_path).first_or_octet_stream(); - (StatusCode::OK, [(header::CONTENT_TYPE, mime.as_ref())], content.data).into_response() - } + Some(content) => serve_embedded(requested_path, content, &headers), None => { // SPA fallback if let Some(index) = Assets::get("index.html") { - (StatusCode::OK, [(header::CONTENT_TYPE, "text/html")], index.data).into_response() + serve_embedded("index.html", index, &headers) } else { StatusCode::NOT_FOUND.into_response() } @@ -35,8 +79,43 @@ pub async fn dashboard_handler(Path(path): Path) -> Response { } } +#[cfg(all(test, feature = "embed-dashboard"))] +mod tests { + use super::*; + + #[test] + fn cache_control_hashed_assets_are_immutable() { + assert_eq!( + cache_control_for("assets/index-abc123.js"), + "public, max-age=31536000, immutable" + ); + } + + #[test] + fn cache_control_index_html_revalidates() { + assert_eq!(cache_control_for("index.html"), "no-cache"); + } + + #[test] + fn cache_control_other_files_get_short_ttl() { + assert_eq!(cache_control_for("favicon.ico"), "public, max-age=3600"); + } + + #[test] + fn serve_embedded_returns_304_on_matching_etag() { + let Some(content) = Assets::get("index.html") else { + return; // no dashboard build embedded in this test run + }; + let etag = format!("\"{}\"", hex::encode(content.metadata.sha256_hash())); + let mut headers = HeaderMap::new(); + headers.insert(header::IF_NONE_MATCH, etag.parse().unwrap()); + let response = serve_embedded("index.html", content, &headers); + assert_eq!(response.status(), StatusCode::NOT_MODIFIED); + } +} + #[cfg(not(feature = "embed-dashboard"))] -pub async fn dashboard_handler(Path(_path): Path) -> Response { +pub async fn dashboard_handler(Path(_path): Path, _headers: axum::http::HeaderMap) -> Response { ( StatusCode::SERVICE_UNAVAILABLE, [(header::CONTENT_TYPE, "text/plain")], diff --git a/apps/backend/src/handlers/file_handler.rs b/apps/backend/src/handlers/file_handler.rs index 70cb0a93..83533579 100644 --- a/apps/backend/src/handlers/file_handler.rs +++ b/apps/backend/src/handlers/file_handler.rs @@ -6,7 +6,7 @@ use axum::{ response::{IntoResponse, Response}, }; use axum_extra::extract::multipart::Multipart; -use bytes::Bytes; +use futures_util::StreamExt; use serde::Deserialize; use serde_json::json; use std::sync::Arc; @@ -24,7 +24,8 @@ use crate::models::file::{BatchFileIds, FileWithUrl}; use crate::repository::Repository; use crate::repository::traits::ListFilesParams; use crate::services::Services; -use crate::services::file::UploadFileRequest; +use crate::services::file::StreamingUploadRequest; +use crate::signed_upload::{SignedUploadError, SignedUploadToken}; use crate::storage::{StorageProvider, StorageRegistry}; #[derive(Deserialize, utoipa::IntoParams)] @@ -124,12 +125,11 @@ pub async fn list_files( security(("bearer" = []), ("access_token" = [])), tag = "files" )] -#[instrument(skip(repository, services, config, ctx, multipart, storage_registry))] +#[instrument(skip(repository, services, ctx, multipart, storage_registry))] pub async fn upload_file( ctx: RequestContext, Extension(repository): Extension, Extension(services): Extension, - Extension(config): Extension, Extension(storage_registry): Extension>, mut multipart: Multipart, ) -> Response { @@ -144,76 +144,151 @@ pub async fn upload_file( .await .unwrap_or_else(|_| "filesystem".into()); - let mut file_data: Option = None; - let mut file_name: Option = None; - let mut file_content_type: Option = None; + let storage = match get_storage_for_site(&storage_provider, &storage_registry) { + Ok(s) => s, + Err(status) => return (status, Json(json!({"error": "Storage not configured"}))).into_response(), + }; + let created_by = ctx.auth.actor.user_id().map(String::from); + + // Stream the "file" field straight into storage (constant memory); size cap + // and content sniffing are enforced by the service while streaming. + let mut uploaded: Option> = None; - while let Ok(Some(mut field)) = multipart.next_field().await { + while let Ok(Some(field)) = multipart.next_field().await { let name = field.name().unwrap_or("").to_string(); - if name == "file" { - file_name = field.file_name().map(String::from); - file_content_type = field.content_type().map(String::from); + if name == "file" && uploaded.is_none() { + let file_name = sanitize_filename(field.file_name().unwrap_or("upload")); + let content_type = field + .content_type() + .map(String::from) + .unwrap_or_else(|| "application/octet-stream".into()); - // Stream the field in chunks, enforcing the size cap incrementally so an - // oversized upload is rejected without first buffering it whole. - let max = config.max_upload_size_bytes; - let mut buf: Vec = Vec::new(); - loop { + let stream = Box::pin(futures_util::stream::unfold(field, |mut field| async move { match field.chunk().await { - Ok(Some(chunk)) => { - if buf.len() + chunk.len() > max { - return ( - StatusCode::PAYLOAD_TOO_LARGE, - Json(json!({ - "error": format!("File too large. Maximum size is {}MB", max / (1024 * 1024)) - })), - ) - .into_response(); - } - buf.extend_from_slice(&chunk); - } - Ok(None) => break, - Err(e) => { - return ( - StatusCode::BAD_REQUEST, - Json(json!({"error": format!("Failed to read file: {}", e)})), - ) - .into_response(); - } + Ok(Some(chunk)) => Some((Ok(chunk), field)), + Ok(None) => None, + Err(e) => Some((Err(Box::new(e) as Box), field)), } - } - file_data = Some(Bytes::from(buf)); + })); + + uploaded = Some( + services + .file + .upload_file_streaming( + StreamingUploadRequest { + site_id: &site_id, + file_id: None, + filename: &file_name, + content_type: &content_type, + created_by: created_by.as_deref(), + storage: storage.clone(), + storage_provider: &storage_provider, + }, + stream, + ) + .await, + ); } } - let file_data = match file_data { - Some(d) => d, - None => { - return (StatusCode::BAD_REQUEST, Json(json!({"error": "No file provided"}))).into_response(); + match uploaded { + Some(Ok(file)) => (StatusCode::CREATED, Json(file)).into_response(), + Some(Err(e)) => e.into_response(), + None => (StatusCode::BAD_REQUEST, Json(json!({"error": "No file provided"}))).into_response(), + } +} + +#[utoipa::path( + put, + path = "/api/v1/files/upload/{token}", + request_body(content = Vec, description = "Raw file bytes", content_type = "application/octet-stream"), + responses( + (status = 201, description = "File uploaded", body = FileWithUrl), + (status = 400, description = "Bad request (content mismatch or unreadable body)"), + (status = 401, description = "Invalid token"), + (status = 409, description = "Upload URL already used"), + (status = 410, description = "Upload URL expired"), + (status = 413, description = "File too large"), + ), + tag = "files" +)] +#[instrument(skip(services, config, storage_registry, headers, body))] +pub async fn upload_via_signed_url( + Path(token): Path, + headers: HeaderMap, + Extension(services): Extension, + Extension(config): Extension, + Extension(storage_registry): Extension>, + body: Body, +) -> Response { + // The token is the auth: HMAC-signed by the server, time-limited, single-use. + let token = match SignedUploadToken::verify(&token, &config.hmac_secret) { + Ok(t) => t, + Err(SignedUploadError::Expired) => { + return (StatusCode::GONE, Json(json!({"error": "Upload URL expired"}))).into_response(); + } + Err(_) => { + return (StatusCode::UNAUTHORIZED, Json(json!({"error": "Invalid upload token"}))).into_response(); } }; - let file_name = sanitize_filename(&file_name.unwrap_or_else(|| "upload".into())); - let content_type = file_content_type.unwrap_or_else(|| "application/octet-stream".into()); - let created_by = ctx.auth.actor.user_id(); + // Fast reject when the client announces an oversized body upfront. + if let Some(len) = headers + .get(header::CONTENT_LENGTH) + .and_then(|v| v.to_str().ok()) + .and_then(|v| v.parse::().ok()) + && len > config.max_upload_size_bytes as u64 + { + return ( + StatusCode::PAYLOAD_TOO_LARGE, + Json(json!({ + "error": format!("File too large. Maximum size is {}MB", config.max_upload_size_bytes / (1024 * 1024)) + })), + ) + .into_response(); + } - let storage = match get_storage_for_site(&storage_provider, &storage_registry) { + // The upload's content type was fixed when the URL was minted; a + // contradicting header is a client bug worth failing loudly on. + if let Some(ct) = headers.get(header::CONTENT_TYPE).and_then(|v| v.to_str().ok()) { + let declared = ct.split(';').next().unwrap_or(ct).trim(); + if !declared.eq_ignore_ascii_case(&token.content_type) { + return ( + StatusCode::BAD_REQUEST, + Json(json!({ + "error": format!("Content-Type '{}' does not match the upload URL's '{}'", declared, token.content_type) + })), + ) + .into_response(); + } + } + + let storage = match get_storage_for_site(&token.storage_provider, &storage_registry) { Ok(s) => s, Err(status) => return (status, Json(json!({"error": "Storage not configured"}))).into_response(), }; + let file_name = sanitize_filename(&token.filename); + let stream = Box::pin( + body.into_data_stream() + .map(|r| r.map_err(|e| Box::new(e) as Box)), + ); + match services .file - .upload_file(UploadFileRequest { - site_id: &site_id, - data: file_data, - filename: &file_name, - content_type: &content_type, - created_by, - storage, - storage_provider: &storage_provider, - }) + .upload_file_streaming( + StreamingUploadRequest { + site_id: &token.site_id, + file_id: Some(&token.file_id), + filename: &file_name, + content_type: &token.content_type, + created_by: None, + storage, + storage_provider: &token.storage_provider, + }, + stream, + ) .await { Ok(file) => (StatusCode::CREATED, Json(file)).into_response(), diff --git a/apps/backend/src/lib.rs b/apps/backend/src/lib.rs index c795463d..cbce407a 100644 --- a/apps/backend/src/lib.rs +++ b/apps/backend/src/lib.rs @@ -8,6 +8,8 @@ pub mod graphql; pub mod mcp; pub mod paths; pub mod secrets; +pub mod server; +pub mod service; pub mod signed_upload; pub mod test_helpers; pub mod tracing; diff --git a/apps/backend/src/main.rs b/apps/backend/src/main.rs index 653c364d..50774c57 100644 --- a/apps/backend/src/main.rs +++ b/apps/backend/src/main.rs @@ -1,22 +1,36 @@ use clap::Parser; use cms::cli::{AdminAction, BackupAction, Cli, Command, ConfigAction, McpTransport}; use cms::config::{self, Config}; -use cms::database::{connect_db_without_migrations, init_db_with_config}; -use cms::grpc::server::spawn_grpc_server; -use cms::middleware::auth::Actor; +use cms::database::init_db_with_config; use cms::repository::Repository; -use cms::router::create_router; -use cms::services::Services; -use cms::storage::{self, STORAGE_KIND_FILESYSTEM, STORAGE_KIND_S3, StorageRegistry}; use std::error::Error; -use std::net::SocketAddr; -use std::sync::Arc; -use tracing::{debug, info, warn}; +use tracing::info; #[tokio::main] async fn main() { + // Load env from the cwd `.env` (dev) first, then `$VCMS_HOME/.env` (installed + // services run from an arbitrary cwd). dotenvy is first-wins, so real env vars and + // the cwd file keep precedence over the config-dir file. VCMS_HOME must be a real + // env var — resolved here before the `.env` loads — never set inside that file + // (chicken-and-egg). dotenvy::dotenv().ok(); + let env_file = cms::paths::env_file(); + match dotenvy::from_path(&env_file) { + Ok(()) => {} + Err(e) if e.not_found() => {} // missing file is fine + // The home `.env` is optional. When it lives in the system service home + // (`C:\ProgramData\vcms`, `/var/lib/vcms`) it's ACL-locked to SYSTEM/root, so a + // non-elevated CLI legitimately can't read it — most notably `vcms mcp stdio`, + // a proxy that needs no home file at all. Treat "permission denied" like + // "absent" and move on: the data-touching commands still hit `paths::ensure`'s + // elevate preflight, which reports the access problem with an actionable hint. + Err(dotenvy::Error::Io(io)) if io.kind() == std::io::ErrorKind::PermissionDenied => {} + Err(e) => { + eprintln!("Error: cannot load {}: {e}", env_file.display()); + std::process::exit(1); + } + } let cli = Cli::parse(); @@ -31,10 +45,11 @@ async fn main() { import_as_new, yes, }) => run_restore(file, scope, site, *import_as_new, *yes, &cli).await, + Some(Command::Service { action }) => cms::service::run_service(action, &cli).await, Some(Command::Mcp { transport: McpTransport::Stdio, - }) => run_mcp_stdio(&cli).await, - Some(Command::Serve) | None => run_serve(&cli).await, + }) => run_mcp_stdio().await, + Some(Command::Serve) | None => cms::server::run(&cli, cms::server::shutdown_signal(), || {}).await, }; if let Err(e) = result { @@ -43,231 +58,24 @@ async fn main() { } } -async fn run_serve(cli: &Cli) -> Result<(), Box> { - cms::paths::ensure()?; - cms::secrets::ensure()?; - let config = Config::load(cli)?; - - let _guard = cms::tracing::init_tracing(&config); - - if let Err(e) = config.validate_security() { - return Err(format!("Invalid production security configuration: {e}").into()); - } - - let pool = init_db_with_config(&config).await?; - - let repository = Repository::new(&pool); - - seed_admin(&repository).await; - - let storage_registry = initialize_storage(&config); - // Build these once and share them: the REST router and the gRPC server use the - // same `Services` so the single-writer search index is opened only once. - let repository_arc = Arc::new(repository.clone()); - let config_arc = Arc::new(config.clone()); - let services = Services::new(repository_arc.clone(), &pool, &config); - - let backup_destination = cms::services::backup::build_backup_destination(&config) - .map_err(|e| format!("Failed to initialize backup destination: {e}"))?; - let backup_service = Arc::new(cms::services::backup::BackupService::new( - pool.clone(), - storage_registry.clone(), - backup_destination, - &config, - )); - - let app = create_router( - repository.clone(), - config.clone(), - storage_registry.clone(), - services.clone(), - backup_service.clone(), - ); - - // Reconcile backups/restore jobs left mid-flight by a previous process: any - // running/pending row at startup is orphaned (backups only run in-process). - match cms::services::backup::meta::fail_orphaned(&pool, &cms::services::backup::now_iso()).await { - Ok(n) if n > 0 => info!("Reconciled {n} interrupted backup job(s) to failed"), - Ok(_) => {} - Err(e) => tracing::error!("Failed to reconcile interrupted backups: {e}"), - } - - if config.backup_enabled { - let scheduler_service = backup_service.clone(); - tokio::spawn(async move { - cms::services::backup::scheduler::run(scheduler_service).await; - }); - info!("Backup scheduler started"); - } - - // The search indexer is the single writer/consumer: it rebuilds the index when - // empty (first run / wiped), then drains the cross-process queue forever. - if let (Some(search), Some(queue)) = (services.search.clone(), services.search_queue.clone()) { - let repo = repository_arc.clone(); - tokio::spawn(async move { - if search.is_empty() { - info!("Search index is empty; building from database..."); - match search.rebuild_all(&repo).await { - Ok(n) => info!("Search index built: {} entries", n), - Err(e) => tracing::error!("Search index build failed: {}", e), - } - } - cms::services::search::indexer::run(search, queue, repo).await; - }); - } - - let addr: SocketAddr = config.bind_address.parse().expect("Invalid BIND_ADDRESS"); - info!("Dashboard UI available at http://{}/dashboard", addr); - info!("REST API server running on http://{}", addr); - info!("GraphQL endpoint at http://{}/api/graphql", addr); - if config.mcp_enabled { - info!("MCP HTTP endpoint at http://{}/mcp", addr); - } - - let grpc_addr: SocketAddr = config.grpc_bind_address.parse().expect("Invalid GRPC_BIND_ADDRESS"); - info!("gRPC server running on {}", grpc_addr); - - let rest_handle = tokio::spawn(async move { - let listener = tokio::net::TcpListener::bind(addr) - .await - .expect("Failed to bind address"); - - axum::serve( - listener, - app.into_make_service_with_connect_info::(), - ) - .with_graceful_shutdown(shutdown_signal()) - .await - .expect("Server error"); - }); - - let grpc_handle = tokio::spawn(spawn_grpc_server( - services.clone(), - repository_arc.clone(), - config_arc.clone(), - storage_registry.clone(), - grpc_addr, - )); - - tokio::select! { - result = rest_handle => { - if let Err(e) = result { - tracing::error!("REST server error: {}", e); - } - } - result = grpc_handle => { - if let Err(e) = result { - tracing::error!("gRPC server error: {}", e); - } - } - } - - Ok(()) -} - -async fn run_mcp_stdio(cli: &Cli) -> Result<(), Box> { - // Read-only: never create the home dir, database, or secrets file here. The - // persisted secrets written by `vcms serve` are what make this process verify - // the site token with the same HMAC secret the server signed it with. - if cms::secrets::load()?.is_none() && std::env::var("HMAC_SECRET").is_err() { - return Err("No instance secrets found. Run `vcms serve` once to initialize \ - ~/.vcms (or set HMAC_SECRET) before `vcms mcp stdio`." - .into()); - } - - let config = Config::load(cli)?; - cms::tracing::init_stdio_tracing(&config); - - info!("Starting standalone MCP stdio process"); - debug!( - database_backend = ?cms::database::backend::DatabaseBackend::from_url(&config.database_url), - "MCP stdio configuration loaded" - ); - - if let Err(error) = config.validate_security() { - return Err(format!("Invalid production security configuration: {error}").into()); - } +async fn run_mcp_stdio() -> Result<(), Box> { + // A thin proxy to the running server's `/mcp` endpoint — it touches no disk + // (no home dir, database, secrets, or search index), so it works even when those + // belong to the privileged service account. It needs only a server URL and a + // `vcms_site_*` access token, forwarded as the bearer credential. + cms::tracing::init_proxy_tracing(); let token = std::env::var("VCMS_MCP_TOKEN").map_err(|_| "VCMS_MCP_TOKEN is required for `vcms mcp stdio`")?; - if token.trim().is_empty() { + let token = token.trim().to_string(); + if token.is_empty() { return Err("VCMS_MCP_TOKEN must not be empty".into()); } - let pool = connect_db_without_migrations(&config).await?; - info!("Existing CMS database schema is compatible; no migrations were run"); - - let repository = Repository::new(&pool); - let actor = cms::mcp::auth::verify_stdio_token(&token, &repository, &config.hmac_secret) - .await - .map_err(|error| format!("MCP stdio authentication failed: {}", error.message))?; - match &actor { - Actor::ApiKey(api_key) => info!( - site_id = %api_key.site_id, - permission = %api_key.permission, - "MCP stdio site token authenticated" - ), - Actor::User(_) => return Err("MCP stdio requires a CMS site access token".into()), - } - - let storage_registry = initialize_storage(&config); - let repository = Arc::new(repository); - let config = Arc::new(config); - // Read-only search: stdio can run alongside the server without contending for - // the writer lock; its content writes enqueue for the server to index. - let services = Arc::new(Services::new_read_only(repository.clone(), &pool, &config)); - let server = cms::mcp::server::CmsServer::new_stdio(services, repository, storage_registry, config, token); - - let result = cms::mcp::transports::stdio::serve(server).await; - match &result { - Ok(()) => info!("Standalone MCP stdio process exited cleanly"), - Err(error) => tracing::error!(error = %error, "Standalone MCP stdio process exiting after failure"), - } - result -} - -fn initialize_storage(config: &Config) -> Arc { - let mut storage_registry = StorageRegistry::new(); - - // Use an explicit filesystem path if set; otherwise default to ~/.vcms/storage - // so uploads work out of the box — unless S3 is configured and takes over. - let fs_path = match (&config.storage_fs_path, config.has_s3()) { - (Some(path), _) => Some(path.clone()), - (None, false) => Some(cms::paths::storage_dir().to_string_lossy().into_owned()), - (None, true) => None, - }; - - if let Some(fs_path) = fs_path { - match storage::FileSystemStorage::new(&fs_path) { - Ok(fs) => { - storage_registry.register(STORAGE_KIND_FILESYSTEM, Arc::new(fs)); - info!("Filesystem storage initialized at {}", fs_path); - } - Err(error) => warn!("Failed to initialize filesystem storage: {}", error), - } - } - - if config.has_s3() { - match storage::S3Storage::new( - config.s3_access_key_id.as_deref().unwrap(), - config.s3_secret_access_key.as_deref().unwrap(), - config.s3_bucket.as_deref().unwrap(), - config.s3_region.as_deref().unwrap_or("us-east-1"), - config.s3_endpoint.as_deref(), - config.s3_public_url.as_deref(), - ) { - Ok(s3) => { - storage_registry.register(STORAGE_KIND_S3, Arc::new(s3)); - info!("S3 storage initialized"); - } - Err(error) => warn!("Failed to initialize S3 storage: {}", error), - } - } - - if storage_registry.get(STORAGE_KIND_FILESYSTEM).is_none() && storage_registry.get(STORAGE_KIND_S3).is_none() { - warn!("No storage providers configured. Set STORAGE_FS_PATH or S3_* env vars."); - } + let base = std::env::var("VCMS_MCP_URL").unwrap_or_else(|_| "http://127.0.0.1:3000".to_string()); + let endpoint = format!("{}/mcp", base.trim_end_matches('/')); - Arc::new(storage_registry) + info!(%endpoint, "Starting MCP stdio proxy"); + cms::mcp::transports::stdio::serve(endpoint, token).await } fn run_config(action: &ConfigAction, cli: &Cli) -> Result<(), Box> { @@ -337,7 +145,7 @@ async fn run_backup(action: &BackupAction, cli: &Cli) -> Result<(), Box) -> Result Err(format!("unknown scope '{other}' (use instance|site)").into()), } } - -/// Email identity of the default admin account. Login is by email, so the seed -/// gate keys on this (non-unique display name "admin" would be the wrong column). -const ADMIN_EMAIL: &str = "admin@cms.local"; - -async fn seed_admin(repository: &Repository) { - debug!("Checking if admin user needs to be seeded"); - if !repository.user.exists(ADMIN_EMAIL).await.unwrap_or(false) { - info!("Seeding default admin user"); - let id = uuid::Uuid::now_v7().to_string(); - let password_hash = bcrypt::hash("admin", bcrypt::DEFAULT_COST).expect("Failed to hash password"); - repository - .user - .create(&id, "admin", ADMIN_EMAIL, &password_hash) - .await - .expect("Failed to seed admin user"); - repository - .user - .set_instance_role(&id, Some("instance_owner")) - .await - .expect("Failed to assign instance owner"); - repository - .user - .update_password(&id, &password_hash, true) - .await - .expect("Failed to require password change"); - - warn!("Seeded default admin user (admin@cms.local / admin) — CHANGE THE PASSWORD IMMEDIATELY!"); - eprintln!( - "\n\ - ============================ SECURITY WARNING ============================\n\ - A default admin account was created: email 'admin@cms.local' password 'admin'\n\ - Sign in with the email and password. Anyone who can reach this server can log\n\ - in until you change it. Run:\n\ - \n vcms admin reset-password --email admin@cms.local --password \n\n\ - or change it from the dashboard now. Do NOT expose this server until done.\n\ - =========================================================================\n" - ); - } else { - debug!("Admin user already exists, skipping seeding"); - } -} - -async fn shutdown_signal() { - let ctrl_c = async { - tokio::signal::ctrl_c().await.expect("failed to install Ctrl+C handler"); - }; - - #[cfg(unix)] - let terminate = async { - tokio::signal::unix::signal(tokio::signal::unix::SignalKind::terminate()) - .expect("failed to install signal handler") - .recv() - .await; - }; - - #[cfg(not(unix))] - let terminate = std::future::pending::<()>(); - - tokio::select! { - _ = ctrl_c => info!("Received Ctrl+C, shutting down..."), - _ = terminate => info!("Received SIGTERM, shutting down..."), - } -} diff --git a/apps/backend/src/mcp/auth.rs b/apps/backend/src/mcp/auth.rs index 3e41e443..7a4edb46 100644 --- a/apps/backend/src/mcp/auth.rs +++ b/apps/backend/src/mcp/auth.rs @@ -5,7 +5,7 @@ use axum::{ middleware::Next, response::{IntoResponse, Response}, }; -use rmcp::model::{CallToolResult, Content, ErrorCode, ErrorData}; +use rmcp::model::{CallToolResult, ContentBlock, ErrorCode, ErrorData}; use rmcp::service::{RequestContext, RoleServer}; use serde_json::json; @@ -26,11 +26,11 @@ pub fn ok_result(data: &impl serde::Serialize) -> Result) -> CallToolResult { - CallToolResult::success(vec![Content::text(message.into())]) + CallToolResult::success(vec![ContentBlock::text(message.into())]) } pub fn map_err(e: impl Into) -> ErrorData { @@ -41,7 +41,7 @@ pub fn map_err(e: impl Into) -> ErrorData /// Use this for business logic errors that the LLM can act on (not found, validation, permission). pub fn tool_error(e: impl Into) -> CallToolResult { let error = e.into(); - CallToolResult::error(vec![Content::text(error.error_message())]) + CallToolResult::error(vec![ContentBlock::text(error.error_message())]) } pub fn resolve_actor(ctx: &RequestContext) -> Result { @@ -63,12 +63,6 @@ pub fn resolve_actor(ctx: &RequestContext) -> Result Result { - verify_access_token(token, repository, hmac_secret) - .await - .map_err(|(_, Json(error))| mcp_error(ErrorCode::INVALID_REQUEST, error.message)) -} - pub async fn authenticate_mcp_request(mut request: Request, next: Next) -> Response { let repository = match request.extensions().get::>() { Some(repository) => repository.clone(), diff --git a/apps/backend/src/mcp/resources/site_schema.rs b/apps/backend/src/mcp/resources/site_schema.rs index 34100cf2..893a3967 100644 --- a/apps/backend/src/mcp/resources/site_schema.rs +++ b/apps/backend/src/mcp/resources/site_schema.rs @@ -2,8 +2,7 @@ use std::sync::Arc; use chrono::Utc; use rmcp::ErrorData as McpError; -use rmcp::model::RawResource; -use rmcp::model::{Annotated, Annotations, ListResourcesResult, ReadResourceResult, Resource, ResourceContents}; +use rmcp::model::{Annotations, ListResourcesResult, ReadResourceResult, Resource, ResourceContents}; use crate::middleware::auth::Actor; use crate::models::authorization::Action; @@ -26,19 +25,11 @@ fn resource_uri(site_id: &str, path: &str) -> String { } fn make_resource(uri: &str, name: &str, title: &str, description: &str) -> Resource { - Annotated::new( - RawResource { - uri: uri.to_string(), - name: name.to_string(), - title: Some(title.to_string()), - description: Some(description.to_string()), - mime_type: Some("application/json".to_string()), - size: None, - icons: None, - meta: None, - }, - Some(Annotations::for_resource(0.5, Utc::now())), - ) + Resource::new(uri, name) + .with_title(title) + .with_description(description) + .with_mime_type("application/json") + .with_annotations(Annotations::for_resource(0.5, Utc::now())) } fn collection_to_schema_value(c: &Collection) -> serde_json::Value { diff --git a/apps/backend/src/mcp/server.rs b/apps/backend/src/mcp/server.rs index 359f6106..23264eae 100644 --- a/apps/backend/src/mcp/server.rs +++ b/apps/backend/src/mcp/server.rs @@ -28,7 +28,6 @@ pub struct CmsServer { pub storage_registry: Arc, pub config: Arc, pub authorizer: Arc, - stdio_token: Option>, } #[tool_router] @@ -46,22 +45,9 @@ impl CmsServer { storage_registry, config, authorizer, - stdio_token: None, } } - pub fn new_stdio( - services: Arc, - repository: Arc, - storage_registry: Arc, - config: Arc, - token: String, - ) -> Self { - let mut server = Self::new(services, repository, storage_registry, config); - server.stdio_token = Some(Arc::from(token)); - server - } - #[tool(description = "Get details of a specific site by ID")] async fn get_site( &self, @@ -252,7 +238,9 @@ impl CmsServer { file::get_file(&self.authorizer, &self.services, &actor, params).await } - #[tool(description = "Create a signed upload URL for uploading a file")] + #[tool( + description = "Create a single-use signed upload URL. To upload: send an HTTP PUT to upload_url with the raw file bytes as the request body and a Content-Type header equal to content_type. The URL expires at expires_at and can be used exactly once. The PUT response body is the created file record (JSON, includes the file id and url)." + )] async fn create_upload_url( &self, ctx: RequestContext, @@ -260,7 +248,15 @@ impl CmsServer { ) -> Result { let actor = self.resolve_actor(&ctx)?; let public_base_url = self.public_base_url(&ctx); - file::create_upload_url(&self.authorizer, &self.config, &actor, public_base_url, params).await + file::create_upload_url( + &self.authorizer, + &self.services, + &self.config, + &actor, + public_base_url, + params, + ) + .await } #[tool(description = "Delete a file (soft delete)")] @@ -488,12 +484,6 @@ impl ServerHandler for CmsServer { impl CmsServer { async fn authenticate_context(&self, ctx: &mut RequestContext) -> Result { - if let Some(token) = &self.stdio_token { - let actor = crate::mcp::auth::verify_stdio_token(token, &self.repository, &self.config.hmac_secret).await?; - ctx.extensions.insert(actor.clone()); - return Ok(actor); - } - self.resolve_actor(ctx) } diff --git a/apps/backend/src/mcp/tools/file.rs b/apps/backend/src/mcp/tools/file.rs index d43a9f8a..52394de5 100644 --- a/apps/backend/src/mcp/tools/file.rs +++ b/apps/backend/src/mcp/tools/file.rs @@ -113,6 +113,7 @@ fn default_content_type() -> String { pub async fn create_upload_url( authorization: &Arc, + services: &Arc, config: &Arc, actor: &Actor, public_base_url: Option, @@ -126,11 +127,28 @@ pub async fn create_upload_url( return Ok(tool_error(e)); } - let (token, upload_path) = SignedUploadToken::generate( + // Fail fast at mint time instead of returning a URL doomed to a 400 PUT. + if !crate::utils::content_types::is_allowed(¶ms.0.content_type) { + return Ok(tool_error(crate::services::error::ServiceError::BadRequest(format!( + "Content type '{}' is not allowed. Accepted types: images, videos, audio, documents, archives", + params.0.content_type + )))); + } + + // Mint against the site's actual storage provider, not a hardcoded default. + let storage_provider = services + .file + .get_storage_provider(&site_id) + .await + .unwrap_or_else(|_| "filesystem".into()); + + let (token, upload_path) = SignedUploadToken::generate_with_storage_provider( &site_id, ¶ms.0.filename, ¶ms.0.content_type, + &storage_provider, &config.hmac_secret, + config.upload_token_expiry_secs, ); let fallback_base_url = format!("http://{}", config.bind_address); diff --git a/apps/backend/src/mcp/transports/stdio.rs b/apps/backend/src/mcp/transports/stdio.rs index 5cf831f5..9ced9474 100644 --- a/apps/backend/src/mcp/transports/stdio.rs +++ b/apps/backend/src/mcp/transports/stdio.rs @@ -1,19 +1,274 @@ -use rmcp::{ServiceExt, transport}; +//! `vcms mcp stdio` — a thin proxy between an MCP client's stdin/stdout and the +//! running server's Streamable HTTP `/mcp` endpoint. +//! +//! The stdio process runs under the *invoking* user from an arbitrary cwd, while the +//! server runs as the (often privileged) service account that owns the database, +//! secrets, and search index. Rather than open those files directly — which fails +//! when they belong to the service account — this proxy forwards JSON-RPC over HTTP +//! and lets the server do all disk I/O. It needs only a URL and a `vcms_site_*` +//! access token, which it injects as the `Authorization: Bearer` header. +//! +//! The server runs the HTTP transport in stateless, JSON-response mode (no SSE, no +//! `Mcp-Session-Id`), so a flat per-message proxy is sufficient: read a +//! newline-delimited JSON-RPC message from stdin, POST it, write the JSON reply back. -use crate::mcp::server::CmsServer; +use serde_json::Value; +use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader}; -pub async fn serve(server: CmsServer) -> Result<(), Box> { - tracing::info!("MCP stdio transport active"); - let service = server.serve(transport::stdio()).await?; +/// Run the proxy loop until stdin closes (EOF → clean exit). +/// +/// `endpoint` is the fully-qualified MCP URL (e.g. `http://127.0.0.1:3000/mcp`) and +/// `token` is the `vcms_site_*` access token forwarded as the bearer credential. +pub async fn serve(endpoint: String, token: String) -> Result<(), Box> { + let client = reqwest::Client::new(); + let mut lines = BufReader::new(tokio::io::stdin()).lines(); + let mut stdout = tokio::io::stdout(); - match service.waiting().await { - Ok(reason) => { - tracing::info!(?reason, "MCP stdio transport stopped"); - Ok(()) + tracing::info!(%endpoint, "MCP stdio proxy active"); + + while let Some(line) = lines.next_line().await? { + let message = line.trim(); + if message.is_empty() { + continue; + } + if let Some(response) = forward(&client, &endpoint, &token, message).await { + stdout.write_all(response.as_bytes()).await?; + stdout.write_all(b"\n").await?; + stdout.flush().await?; + } + } + + tracing::info!("MCP stdio proxy stopped (stdin closed)"); + Ok(()) +} + +/// Forward one JSON-RPC message to the server and return the line to write to stdout, +/// or `None` when nothing should be written (a notification, which carries no `id` +/// and expects no reply). On a transport failure for a request we synthesize a +/// JSON-RPC error so the client sees a clean failure and the proxy keeps running. +async fn forward(client: &reqwest::Client, endpoint: &str, token: &str, message: &str) -> Option { + let parsed = serde_json::from_str::(message).ok(); + // Requests and batches expect a response; a lone notification object (no `id`) + // does not. Unparseable input is forwarded so the server returns a proper + // JSON-RPC parse error. + let (needs_response, id) = match &parsed { + Some(Value::Object(obj)) => (obj.contains_key("id"), obj.get("id").cloned()), + Some(Value::Array(_)) => (true, None), // batch + _ => (true, None), // unparseable → let the server reject it + }; + + let result = client + .post(endpoint) + .header(reqwest::header::AUTHORIZATION, format!("Bearer {token}")) + .header(reqwest::header::CONTENT_TYPE, "application/json") + .header(reqwest::header::ACCEPT, "application/json, text/event-stream") + .body(message.to_string()) + .send() + .await; + + let response = match result { + Ok(response) => response, + Err(error) => { + return error_line( + needs_response, + &id, + format!("cannot reach vcms server at {endpoint}: {error}"), + ); } + }; + + let status = response.status(); + let content_type = response + .headers() + .get(reqwest::header::CONTENT_TYPE) + .and_then(|value| value.to_str().ok()) + .unwrap_or_default() + .to_string(); + + let body = match response.text().await { + Ok(body) => body, Err(error) => { - tracing::error!(error = %error, "MCP stdio transport failed"); - Err(error.into()) + return error_line( + needs_response, + &id, + format!("reading vcms server response failed: {error}"), + ); } + }; + + if !needs_response { + return None; // notification: server replies 202 with an empty body + } + + // A non-2xx is an auth/host/transport failure (e.g. 401 for a bad token), not a + // JSON-RPC reply. Wrap it so the client sees a proper JSON-RPC error envelope. + if !status.is_success() { + let detail = http_error_detail(&body); + return error_line(needs_response, &id, format!("vcms server returned {status}: {detail}")); + } + + // json-response mode returns `application/json`; defensively unwrap a single SSE + // frame if the server ever streams one back. + let payload = if content_type.contains("text/event-stream") { + extract_sse_data(&body).unwrap_or(body) + } else { + body + }; + let payload = payload.trim(); + if payload.is_empty() { + return error_line( + needs_response, + &id, + "vcms server returned an empty response".to_string(), + ); + } + + // Re-serialize compactly so the line carries no embedded newlines (MCP stdio + // framing is one message per line). Fall back to the raw payload if it isn't JSON. + Some( + serde_json::from_str::(payload) + .map(|value| value.to_string()) + .unwrap_or_else(|_| payload.replace('\n', " ")), + ) +} + +/// Build a JSON-RPC error line for a failed request, or `None` for a notification +/// (which has no `id` to respond to). +fn error_line(needs_response: bool, id: &Option, message: String) -> Option { + if !needs_response { + tracing::warn!(%message, "MCP notification forward failed"); + return None; + } + let error = serde_json::json!({ + "jsonrpc": "2.0", + "id": id.clone().unwrap_or(Value::Null), + "error": { "code": -32603, "message": message }, + }); + Some(error.to_string()) +} + +/// Concatenate the `data:` lines of an SSE payload into a single JSON string. +fn extract_sse_data(body: &str) -> Option { + let mut data = String::new(); + for line in body.lines() { + if let Some(rest) = line.strip_prefix("data:") { + data.push_str(rest.trim_start()); + } + } + (!data.is_empty()).then_some(data) +} + +/// Pull a human-readable detail out of an error response body. The server's auth +/// failures are `{"error": "..."}`; fall back to the raw trimmed body otherwise. +fn http_error_detail(body: &str) -> String { + serde_json::from_str::(body) + .ok() + .and_then(|value| value.get("error").and_then(Value::as_str).map(str::to_string)) + .unwrap_or_else(|| body.trim().to_string()) +} + +#[cfg(test)] +mod tests { + use super::*; + use axum::Router; + use axum::body::Bytes; + use axum::http::{HeaderMap, StatusCode}; + use axum::routing::post; + use std::net::SocketAddr; + + /// A stub `/mcp` that echoes the received Authorization/Accept headers back inside + /// a JSON-RPC result (for requests) and 202-accepts notifications. + async fn spawn_stub() -> SocketAddr { + let app = Router::new().route( + "/mcp", + post(|headers: HeaderMap, body: Bytes| async move { + let header = |name: &str| { + headers + .get(name) + .and_then(|value| value.to_str().ok()) + .unwrap_or_default() + .to_string() + }; + let auth = header("authorization"); + let accept = header("accept"); + let parsed: Value = serde_json::from_slice(&body).unwrap_or(Value::Null); + let response_headers = [("content-type", "application/json")]; + match parsed.get("id") { + None => (StatusCode::ACCEPTED, response_headers, String::new()), + Some(id) => { + let reply = serde_json::json!({ + "jsonrpc": "2.0", + "id": id, + "result": { "auth": auth, "accept": accept }, + }); + (StatusCode::OK, response_headers, reply.to_string()) + } + } + }), + ); + let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let addr = listener.local_addr().unwrap(); + tokio::spawn(async move { axum::serve(listener, app).await.unwrap() }); + addr + } + + #[tokio::test] + async fn forwards_bearer_and_relays_response() { + let addr = spawn_stub().await; + let endpoint = format!("http://{addr}/mcp"); + let out = forward( + &reqwest::Client::new(), + &endpoint, + "vcms_site_test", + r#"{"jsonrpc":"2.0","id":1,"method":"tools/list"}"#, + ) + .await + .expect("a request must produce a response line"); + let value: Value = serde_json::from_str(&out).unwrap(); + assert_eq!(value["id"], 1); + assert_eq!(value["result"]["auth"], "Bearer vcms_site_test"); + assert!(value["result"]["accept"].as_str().unwrap().contains("application/json")); + } + + #[tokio::test] + async fn notification_produces_no_output() { + let addr = spawn_stub().await; + let endpoint = format!("http://{addr}/mcp"); + let out = forward( + &reqwest::Client::new(), + &endpoint, + "t", + r#"{"jsonrpc":"2.0","method":"notifications/initialized"}"#, + ) + .await; + assert!(out.is_none()); + } + + #[tokio::test] + async fn unreachable_server_yields_jsonrpc_error_for_request() { + // Port 1 refuses immediately on all supported platforms. + let out = forward( + &reqwest::Client::new(), + "http://127.0.0.1:1/mcp", + "t", + r#"{"jsonrpc":"2.0","id":7,"method":"tools/list"}"#, + ) + .await + .expect("a request must produce an error line"); + let value: Value = serde_json::from_str(&out).unwrap(); + assert_eq!(value["id"], 7); + assert_eq!(value["error"]["code"], -32603); + } + + #[tokio::test] + async fn unreachable_server_swallows_notification() { + let out = forward( + &reqwest::Client::new(), + "http://127.0.0.1:1/mcp", + "t", + r#"{"jsonrpc":"2.0","method":"ping"}"#, + ) + .await; + assert!(out.is_none()); } } diff --git a/apps/backend/src/middleware/auth.rs b/apps/backend/src/middleware/auth.rs index 1dd04b9f..0d1e51dd 100644 --- a/apps/backend/src/middleware/auth.rs +++ b/apps/backend/src/middleware/auth.rs @@ -176,6 +176,34 @@ fn extract_csrf_token(parts: &Parts) -> Option { // ── HMAC / sessions ── +/// How stale a session/token "last seen" timestamp may get before we rewrite +/// it. Skipping the unconditional per-request UPDATE keeps the auth hot path +/// read-only for most requests. +pub(crate) const TOUCH_INTERVAL_SECS: i64 = 60; + +/// Lenient parse of the backend-specific timestamp texts: RFC3339, Postgres +/// `::text` (`YYYY-MM-DD HH:MM:SS[.fff]+00`), or naive UTC (SQLite/MySQL). +pub(crate) fn parse_db_timestamp(s: &str) -> Option> { + if let Ok(dt) = chrono::DateTime::parse_from_rfc3339(s) { + return Some(dt.with_timezone(&chrono::Utc)); + } + if let Ok(dt) = chrono::DateTime::parse_from_str(s, "%Y-%m-%d %H:%M:%S%.f%#z") { + return Some(dt.with_timezone(&chrono::Utc)); + } + chrono::NaiveDateTime::parse_from_str(s, "%Y-%m-%d %H:%M:%S%.f") + .ok() + .map(|naive| naive.and_utc()) +} + +/// True when a "last seen/used" timestamp is missing, unparseable, or older +/// than [`TOUCH_INTERVAL_SECS`] — i.e. it should be rewritten (safe fallback). +pub(crate) fn needs_touch(last: Option<&str>) -> bool { + match last.and_then(parse_db_timestamp) { + Some(ts) => (chrono::Utc::now() - ts).num_seconds() > TOUCH_INTERVAL_SECS, + None => true, + } +} + pub fn compute_key_hmac(key: &str, hmac_secret: &str) -> String { let mut mac = HmacSha256::new_from_slice(hmac_secret.as_bytes()).expect("HMAC can take key of any size"); mac.update(key.as_bytes()); @@ -193,7 +221,9 @@ pub async fn verify_session( .await .map_err(|_| AuthError::unauthorized("Authentication service unavailable"))? .ok_or_else(|| AuthError::unauthorized("Invalid or expired session"))?; - let _ = repository.session.touch(&session.id).await; + if needs_touch(Some(&session.last_seen_at)) { + let _ = repository.session.touch(&session.id).await; + } Ok(UserActor { user_id: session.user_id, session_id: session.id, @@ -221,7 +251,7 @@ pub(crate) async fn verify_access_token( let token_hmac = compute_key_hmac(token, hmac_secret); - for (token_id, site_id, stored_hash, stored_hmac, expires_at, revoked_at, permission) in keys { + for (token_id, site_id, stored_hash, stored_hmac, expires_at, revoked_at, permission, last_used_at) in keys { if let Some(ref stored) = stored_hmac { if stored != &token_hmac { continue; @@ -242,7 +272,9 @@ pub(crate) async fn verify_access_token( .parse::() .map_err(|_| AuthError::unauthorized("Invalid access token"))?; - let _ = repository.access_token.update_last_used(&token_id).await; + if needs_touch(last_used_at.as_deref()) { + let _ = repository.access_token.update_last_used(&token_id).await; + } Span::current().record("site_id", tracing::field::display(&site_id)); return Ok(Actor::ApiKey(ApiKeyActor { diff --git a/apps/backend/src/paths.rs b/apps/backend/src/paths.rs index 2466a24a..02bb2af5 100644 --- a/apps/backend/src/paths.rs +++ b/apps/backend/src/paths.rs @@ -1,116 +1,274 @@ -//! Resolution of the CMS home directory and the runtime files within it. +//! Resolution of where the CMS keeps its runtime files. //! -//! Everything the running instance owns lives under a single root so a fresh -//! install "just works" regardless of the current working directory: +//! There are two layouts: //! -//! ```text -//! ~/.vcms/ -//! config.toml # non-secret configuration -//! secrets.toml # auto-generated HMAC + backup encryption secrets (0600) -//! vcms.db # default SQLite database (+ -wal / -shm) -//! logs/ # rolling logs when log output = "file" -//! storage/ # default filesystem storage for uploads -//! ``` +//! * **Split** (the default for an interactive install) — files land in the +//! platform-conventional per-type directories via the `directories` crate: +//! config/secrets in the config dir, the database/uploads/backups in the data dir, +//! the (rebuildable) search index in the cache dir, and logs in the state dir. +//! On Linux this is XDG (`~/.config/vcms`, `~/.local/share/vcms`, `~/.cache/vcms`, +//! `~/.local/state/vcms`); on macOS `~/Library/Application Support/vcms` (+ Caches); +//! on Windows `%APPDATA%`/`%LOCALAPPDATA%` under `vcms`. //! -//! The root is `$VCMS_HOME` when set, otherwise `~/.vcms` resolved cross-platform -//! via the `directories` crate. This mirrors the convention used by tools like -//! `CARGO_HOME` / `PGDATA`: one predictable, overridable home. +//! * **Single** — everything nests under one root. Chosen when `$VCMS_HOME` is set +//! (the OS-service installer pins it to a system dir like `/var/lib/vcms` or +//! `C:\ProgramData\vcms`), or when a legacy `~/.vcms` already holds data (so an +//! existing install keeps working untouched). use std::path::PathBuf; -/// Environment variable that overrides the home directory location. +/// Environment variable that forces the single-directory layout at a chosen root. pub const CMS_HOME_ENV: &str = "VCMS_HOME"; -/// The CMS home directory root. -/// -/// `$VCMS_HOME` wins if set and non-empty. Otherwise `~/.vcms`. As a last resort -/// (no detectable home directory) falls back to `.vcms` in the current dir. -pub fn home() -> PathBuf { - if let Some(value) = std::env::var_os(CMS_HOME_ENV) - && !value.is_empty() +/// Where each class of file lives. Resolved fresh on each call (cheap) so tests and +/// the service can change `$VCMS_HOME` without a process restart. +enum Layout { + /// One root holding everything (`$VCMS_HOME`, a legacy `~/.vcms`, or a `.vcms` + /// fallback when no platform dirs are detectable). + Single(PathBuf), + /// Per-type platform directories. + Split { + config: PathBuf, + data: PathBuf, + cache: PathBuf, + state: PathBuf, + }, +} + +impl Layout { + fn config(&self) -> PathBuf { + match self { + Layout::Single(root) => root.clone(), + Layout::Split { config, .. } => config.clone(), + } + } + + fn data(&self) -> PathBuf { + match self { + Layout::Single(root) => root.clone(), + Layout::Split { data, .. } => data.clone(), + } + } + + fn cache(&self) -> PathBuf { + match self { + Layout::Single(root) => root.clone(), + Layout::Split { cache, .. } => cache.clone(), + } + } + + fn state(&self) -> PathBuf { + match self { + Layout::Single(root) => root.clone(), + Layout::Split { state, .. } => state.clone(), + } + } +} + +/// The fixed system directory an installed OS service owns, per platform. This is the +/// single source of truth for that path — the `service` submodules import it instead +/// of re-declaring their own const. `None` on platforms without a service integration. +pub fn system_home() -> Option { + #[cfg(windows)] { - return PathBuf::from(value); + Some(PathBuf::from(r"C:\ProgramData\vcms")) } + #[cfg(target_os = "linux")] + { + Some(PathBuf::from("/var/lib/vcms")) + } + #[cfg(target_os = "macos")] + { + Some(PathBuf::from("/Library/Application Support/vcms")) + } + #[cfg(not(any(windows, target_os = "linux", target_os = "macos")))] + { + None + } +} - directories::BaseDirs::new() - .map(|dirs| dirs.home_dir().join(".vcms")) - .unwrap_or_else(|| PathBuf::from(".vcms")) +/// Resolve the active layout: `$VCMS_HOME` → an existing system service home → legacy +/// `~/.vcms` → platform split. Gathers the real filesystem/env inputs and defers the +/// precedence decision to the pure [`resolve_layout`] (kept separate so it is testable +/// without touching real system dirs). +fn layout() -> Layout { + let env_home = std::env::var_os(CMS_HOME_ENV) + .filter(|v| !v.is_empty()) + .map(PathBuf::from); + + // A system home counts only when its dir already exists on disk: the OS-service + // installer creates it, and it survives uninstall (which leaves data behind). So a + // plain CLI invocation follows the service's data instead of forking to a second, + // per-user store. Absent (dev/eval boxes) → we fall through to the split layout. + let system = system_home().filter(|p| p.is_dir()); + + // Back-compat: a pre-existing single `~/.vcms` keeps owning everything so an upgrade + // never strands a user's database or secrets. + let legacy = directories::BaseDirs::new() + .map(|base| base.home_dir().join(".vcms")) + .filter(|p| p.is_dir()); + + let split = directories::ProjectDirs::from("", "", "vcms").map(|dirs| split_from(&dirs)); + + resolve_layout(env_home, system, legacy, split) +} + +/// Pure precedence policy. Each `Option` is a candidate the caller has already vetted +/// (env set & non-empty; system/legacy dirs confirmed to exist; split from `ProjectDirs`). +/// Falls back to a local `.vcms` blob when no platform dirs are detectable (rare). +fn resolve_layout( + env_home: Option, + system_home: Option, + legacy_home: Option, + split: Option, +) -> Layout { + if let Some(root) = env_home { + return Layout::Single(root); + } + if let Some(root) = system_home { + return Layout::Single(root); + } + if let Some(root) = legacy_home { + return Layout::Single(root); + } + split.unwrap_or_else(|| Layout::Single(PathBuf::from(".vcms"))) } -/// `~/.vcms/config.toml` — the user-level config file. +/// Map `ProjectDirs` onto our four directory classes. Logs use the state dir where +/// the platform has one (Linux), else the local data dir (macOS/Windows). +fn split_from(dirs: &directories::ProjectDirs) -> Layout { + Layout::Split { + config: dirs.config_dir().to_path_buf(), + data: dirs.data_dir().to_path_buf(), + cache: dirs.cache_dir().to_path_buf(), + state: dirs.state_dir().unwrap_or_else(|| dirs.data_local_dir()).to_path_buf(), + } +} + +/// `config.toml` — the non-secret config file (config dir). pub fn config_file() -> PathBuf { - home().join("config.toml") + layout().config().join("config.toml") } -/// `~/.vcms/secrets.toml` — auto-generated secrets file. +/// `secrets.toml` — auto-generated secrets file (config dir, 0600 on unix). pub fn secrets_file() -> PathBuf { - home().join("secrets.toml") + layout().config().join("secrets.toml") } -/// `~/.vcms/vcms.db` — the default SQLite database file. -pub fn default_db_path() -> PathBuf { - home().join("vcms.db") +/// `.env` — optional environment file loaded at startup (config dir). +pub fn env_file() -> PathBuf { + layout().config().join(".env") } -/// `~/.vcms/logs` — directory for rolling log files. -pub fn logs_dir() -> PathBuf { - home().join("logs") +/// `vcms.db` — the default SQLite database file (data dir). +pub fn default_db_path() -> PathBuf { + layout().data().join("vcms.db") } -/// `~/.vcms/storage` — default filesystem storage directory for uploads. +/// `storage/` — default filesystem storage directory for uploads (data dir). pub fn storage_dir() -> PathBuf { - home().join("storage") + layout().data().join("storage") } -/// `~/.vcms/backups` — default local destination for backup artifacts. +/// `backups/` — default local destination for backup artifacts (data dir). pub fn backups_dir() -> PathBuf { - home().join("backups") + layout().data().join("backups") } -/// `~/.vcms/search` — default location for the Tantivy full-text search index. +/// `search/` — default location for the derived Tantivy index (cache dir). pub fn search_dir() -> PathBuf { - home().join("search") + layout().cache().join("search") +} + +/// `logs/` — directory for rolling log files (state dir). +pub fn logs_dir() -> PathBuf { + layout().state().join("logs") } -/// Build the default `DATABASE_URL` (`sqlite:///vcms.db`). +/// Build the default `DATABASE_URL` (`sqlite:///vcms.db`). /// /// SQLite URLs use forward slashes, so backslashes are normalized for Windows. pub fn default_database_url() -> String { format!("sqlite://{}", default_db_path().to_string_lossy().replace('\\', "/")) } -/// Create the home directory and its subdirectories (`logs/`, `storage/`). -/// -/// Called by commands that own/initialize the instance (`serve`, `admin`). -/// Read-only commands such as `mcp stdio` must not create anything. +/// Fail fast when the active home is the system service home but this process can't +/// access it (e.g. a non-elevated CLI against the ACL-locked `C:\ProgramData\vcms` or a +/// root-owned `/var/lib/vcms`). Without this, a deep database/secrets read would blow up +/// with an opaque IO error — and, worse, we must never silently fall back to a per-user +/// store, which would recreate the very two-store split this resolution exists to kill. +fn preflight_system_home() -> std::io::Result<()> { + let Layout::Single(root) = layout() else { + return Ok(()); + }; + if system_home().as_deref() != Some(root.as_path()) || !root.is_dir() { + return Ok(()); + } + // A read probe: opening the db and secrets both need list+read on this dir. + match std::fs::read_dir(&root) { + Ok(_) => Ok(()), + Err(e) if e.kind() == std::io::ErrorKind::PermissionDenied => Err(std::io::Error::new( + std::io::ErrorKind::PermissionDenied, + format!( + "vcms data at {} requires Administrator/root; re-run in an elevated terminal.", + root.display() + ), + )), + Err(e) => Err(e), + } +} + +/// Create the directories the running instance writes to (config, data, storage, +/// logs). Called by commands that own/initialize the instance (`serve`, `admin`, +/// `backup`, `restore`); read-only commands such as `mcp stdio`, `config path/show` +/// never call this and so never create anything or trip the preflight below. pub fn ensure() -> std::io::Result<()> { - let root = home(); - std::fs::create_dir_all(&root)?; - std::fs::create_dir_all(logs_dir())?; - std::fs::create_dir_all(storage_dir())?; + preflight_system_home()?; + let layout = layout(); + for dir in [ + layout.config(), + layout.data(), + layout.state().join("logs"), + layout.data().join("storage"), + ] { + std::fs::create_dir_all(dir)?; + } Ok(()) } +/// Whether a file sits inside the home root in single-dir mode. Only meaningful for +/// `$VCMS_HOME`/legacy installs; in split mode there is no single root. +pub fn single_root() -> Option { + match layout() { + Layout::Single(root) => Some(root), + Layout::Split { .. } => None, + } +} + #[cfg(test)] mod tests { use super::*; use crate::test_helpers::with_home; + use std::path::Path; #[test] - fn cms_home_env_overrides_root() { + fn vcms_home_forces_single_layout() { let dir = tempfile::tempdir().expect("temp dir"); with_home(dir.path(), || { - assert_eq!(home(), dir.path()); + assert_eq!(single_root().as_deref(), Some(dir.path())); assert_eq!(config_file(), dir.path().join("config.toml")); assert_eq!(secrets_file(), dir.path().join("secrets.toml")); + assert_eq!(env_file(), dir.path().join(".env")); assert_eq!(default_db_path(), dir.path().join("vcms.db")); - assert_eq!(logs_dir(), dir.path().join("logs")); assert_eq!(storage_dir(), dir.path().join("storage")); + assert_eq!(backups_dir(), dir.path().join("backups")); + assert_eq!(search_dir(), dir.path().join("search")); + assert_eq!(logs_dir(), dir.path().join("logs")); }); } #[test] - fn ensure_creates_subdirectories() { + fn ensure_creates_subdirectories_in_single_mode() { let dir = tempfile::tempdir().expect("temp dir"); let root = dir.path().join("home"); with_home(&root, || { @@ -130,4 +288,93 @@ mod tests { assert!(!url.contains('\\')); }); } + + #[test] + fn split_layout_separates_file_classes() { + // Drive the pure mapping directly so the assertion is platform-independent + // (real ProjectDirs vary by OS and ambient env). + let layout = Layout::Split { + config: PathBuf::from("/cfg"), + data: PathBuf::from("/dat"), + cache: PathBuf::from("/cch"), + state: PathBuf::from("/st"), + }; + assert!(layout.config().join("config.toml").starts_with(Path::new("/cfg"))); + assert!(layout.data().join("vcms.db").starts_with(Path::new("/dat"))); + assert!(layout.cache().join("search").starts_with(Path::new("/cch"))); + assert!(layout.state().join("logs").starts_with(Path::new("/st"))); + // config / data / cache / state are genuinely distinct in split mode. + assert_ne!(layout.config(), layout.data()); + assert_ne!(layout.data(), layout.cache()); + assert_ne!(layout.cache(), layout.state()); + } + + #[test] + fn split_state_falls_back_to_local_data_when_absent() { + // When a platform lacks a state dir, logs must still resolve (we feed the + // local-data dir into `state`). Modeled here as state == data. + let layout = Layout::Split { + config: PathBuf::from("/cfg"), + data: PathBuf::from("/dat"), + cache: PathBuf::from("/cch"), + state: PathBuf::from("/dat"), + }; + assert!(layout.state().join("logs").starts_with(Path::new("/dat"))); + } + + fn split_sample() -> Option { + Some(Layout::Split { + config: PathBuf::from("/c"), + data: PathBuf::from("/d"), + cache: PathBuf::from("/ch"), + state: PathBuf::from("/s"), + }) + } + + fn single_root_of(layout: Layout) -> Option { + match layout { + Layout::Single(root) => Some(root), + Layout::Split { .. } => None, + } + } + + #[test] + fn resolve_env_home_outranks_everything() { + let out = resolve_layout( + Some(PathBuf::from("/env")), + Some(PathBuf::from("/sys")), + Some(PathBuf::from("/legacy")), + split_sample(), + ); + assert_eq!(single_root_of(out), Some(PathBuf::from("/env"))); + } + + #[test] + fn resolve_system_home_outranks_legacy_and_split() { + let out = resolve_layout( + None, + Some(PathBuf::from("/sys")), + Some(PathBuf::from("/legacy")), + split_sample(), + ); + assert_eq!(single_root_of(out), Some(PathBuf::from("/sys"))); + } + + #[test] + fn resolve_legacy_home_when_no_env_or_system() { + let out = resolve_layout(None, None, Some(PathBuf::from("/legacy")), split_sample()); + assert_eq!(single_root_of(out), Some(PathBuf::from("/legacy"))); + } + + #[test] + fn resolve_falls_through_to_split() { + let out = resolve_layout(None, None, None, split_sample()); + assert!(matches!(out, Layout::Split { .. })); + } + + #[test] + fn resolve_local_blob_when_nothing_detectable() { + let out = resolve_layout(None, None, None, None); + assert_eq!(single_root_of(out), Some(PathBuf::from(".vcms"))); + } } diff --git a/apps/backend/src/repository/mysql/access_token.rs b/apps/backend/src/repository/mysql/access_token.rs index 6c2dc25f..5df706f0 100644 --- a/apps/backend/src/repository/mysql/access_token.rs +++ b/apps/backend/src/repository/mysql/access_token.rs @@ -71,7 +71,7 @@ impl AccessTokenRepository for MysqlAccessTokenRepository { async fn find_by_prefix(&self, prefix: &str) -> Result, RepositoryError> { let rows = sqlx::query_as::<_, AccessTokenLookupRow>( - "SELECT id, site_id, token_hash, token_hmac, CAST(expires_at AS CHAR) AS expires_at, CAST(revoked_at AS CHAR) AS revoked_at, permission + "SELECT id, site_id, token_hash, token_hmac, CAST(expires_at AS CHAR) AS expires_at, CAST(revoked_at AS CHAR) AS revoked_at, permission, CAST(last_used_at AS CHAR) AS last_used_at FROM access_tokens WHERE token_prefix = ?", ) .bind(prefix) diff --git a/apps/backend/src/repository/mysql/collection.rs b/apps/backend/src/repository/mysql/collection.rs index affed4a6..13ef3847 100644 --- a/apps/backend/src/repository/mysql/collection.rs +++ b/apps/backend/src/repository/mysql/collection.rs @@ -125,6 +125,9 @@ impl CollectionRepository for MysqlCollectionRepository { entry_items: &[Entry], rename_map: &std::collections::HashMap, ) -> Result<(), RepositoryError> { + // One transaction for the whole migration: per-statement commit overhead + // dominated this loop, and a partial rename is never left behind. + let mut tx = self.pool.begin().await?; for entry in entry_items { if let Ok(mut data) = serde_json::from_str::(&entry.data) && let Some(obj) = data.as_object_mut() @@ -140,10 +143,11 @@ impl CollectionRepository for MysqlCollectionRepository { sqlx::query("UPDATE entries SET data = ?, updated_at = NOW() WHERE id = ?") .bind(&new_data_str) .bind(&entry.id) - .execute(&self.pool) + .execute(&mut *tx) .await?; } } + tx.commit().await?; Ok(()) } } diff --git a/apps/backend/src/repository/mysql/entry.rs b/apps/backend/src/repository/mysql/entry.rs index 5e158d6f..bb0222d8 100644 --- a/apps/backend/src/repository/mysql/entry.rs +++ b/apps/backend/src/repository/mysql/entry.rs @@ -334,16 +334,22 @@ impl EntryRepository for MysqlEntryRepository { .execute(&mut *tx) .await?; - for file_id in &file_ids { - sqlx::query( - "INSERT IGNORE INTO entry_file_references (entry_id, file_id, site_id) SELECT ?, id, ? FROM files WHERE id = ? AND site_id = ?", - ) - .bind(entry_id) - .bind(site_id) - .bind(file_id) - .bind(site_id) - .execute(&mut *tx) - .await?; + if !file_ids.is_empty() { + // One multi-row statement instead of a query per file id; the + // SELECT keeps the "file must exist in this site" filter. + let placeholders = vec!["?"; file_ids.len()].join(", "); + let sql = format!( + "INSERT IGNORE INTO entry_file_references (entry_id, file_id, site_id) + SELECT ?, id, ? FROM files WHERE site_id = ? AND id IN ({placeholders})", + ); + let mut query = sqlx::query(sqlx::AssertSqlSafe(sql.as_str())) + .bind(entry_id) + .bind(site_id) + .bind(site_id); + for file_id in &file_ids { + query = query.bind(file_id); + } + query.execute(&mut *tx).await?; } tx.commit().await?; diff --git a/apps/backend/src/repository/mysql/file.rs b/apps/backend/src/repository/mysql/file.rs index 4d1807dd..47caa3f5 100644 --- a/apps/backend/src/repository/mysql/file.rs +++ b/apps/backend/src/repository/mysql/file.rs @@ -329,4 +329,21 @@ impl FileRepository for MysqlFileRepository { Ok(provider.unwrap_or_else(|| "filesystem".into())) } + + async fn set_thumbnail_meta( + &self, + id: &str, + thumbnail_key: &str, + width: Option, + height: Option, + ) -> Result<(), RepositoryError> { + sqlx::query("UPDATE files SET thumbnail_key = ?, width = ?, height = ? WHERE id = ?") + .bind(thumbnail_key) + .bind(width) + .bind(height) + .bind(id) + .execute(&self.pool) + .await?; + Ok(()) + } } diff --git a/apps/backend/src/repository/postgres/access_token.rs b/apps/backend/src/repository/postgres/access_token.rs index 025dda33..72409a6a 100644 --- a/apps/backend/src/repository/postgres/access_token.rs +++ b/apps/backend/src/repository/postgres/access_token.rs @@ -71,7 +71,7 @@ impl AccessTokenRepository for PostgresAccessTokenRepository { async fn find_by_prefix(&self, prefix: &str) -> Result, RepositoryError> { let rows = sqlx::query_as::<_, AccessTokenLookupRow>( - "SELECT id, site_id, token_hash, token_hmac, expires_at::text, revoked_at::text, permission + "SELECT id, site_id, token_hash, token_hmac, expires_at::text, revoked_at::text, permission, last_used_at::text FROM access_tokens WHERE token_prefix = $1", ) .bind(prefix) diff --git a/apps/backend/src/repository/postgres/collection.rs b/apps/backend/src/repository/postgres/collection.rs index 1fbc607c..11544168 100644 --- a/apps/backend/src/repository/postgres/collection.rs +++ b/apps/backend/src/repository/postgres/collection.rs @@ -127,6 +127,9 @@ impl CollectionRepository for PostgresCollectionRepository { entry_items: &[Entry], rename_map: &std::collections::HashMap, ) -> Result<(), RepositoryError> { + // One transaction for the whole migration: per-statement commit overhead + // dominated this loop, and a partial rename is never left behind. + let mut tx = self.pool.begin().await?; for entry in entry_items { if let Ok(mut data) = serde_json::from_str::(&entry.data) && let Some(obj) = data.as_object_mut() @@ -142,10 +145,11 @@ impl CollectionRepository for PostgresCollectionRepository { sqlx::query("UPDATE entries SET data = $1::jsonb, updated_at = NOW() WHERE id = $2") .bind(&new_data_str) .bind(&entry.id) - .execute(&self.pool) + .execute(&mut *tx) .await?; } } + tx.commit().await?; Ok(()) } } diff --git a/apps/backend/src/repository/postgres/entry.rs b/apps/backend/src/repository/postgres/entry.rs index a8cb9280..cbeaca43 100644 --- a/apps/backend/src/repository/postgres/entry.rs +++ b/apps/backend/src/repository/postgres/entry.rs @@ -346,14 +346,18 @@ impl EntryRepository for PostgresEntryRepository { .execute(&mut *tx) .await?; - for file_id in &file_ids { + if !file_ids.is_empty() { + // One multi-row statement instead of a query per file id; the + // SELECT keeps the "file must exist in this site" filter. sqlx::query( - "INSERT INTO entry_file_references (entry_id, file_id, site_id) SELECT $1, id, $2 FROM files WHERE id = $3 AND site_id = $4 ON CONFLICT DO NOTHING", + "INSERT INTO entry_file_references (entry_id, file_id, site_id) + SELECT $1, id, $2 FROM files WHERE site_id = $3 AND id = ANY($4) + ON CONFLICT DO NOTHING", ) .bind(entry_id) .bind(site_id) - .bind(file_id) .bind(site_id) + .bind(&file_ids) .execute(&mut *tx) .await?; } diff --git a/apps/backend/src/repository/postgres/file.rs b/apps/backend/src/repository/postgres/file.rs index 3061b8b0..ecef75f5 100644 --- a/apps/backend/src/repository/postgres/file.rs +++ b/apps/backend/src/repository/postgres/file.rs @@ -344,4 +344,21 @@ impl FileRepository for PostgresFileRepository { Ok(provider.unwrap_or_else(|| "filesystem".into())) } + + async fn set_thumbnail_meta( + &self, + id: &str, + thumbnail_key: &str, + width: Option, + height: Option, + ) -> Result<(), RepositoryError> { + sqlx::query("UPDATE files SET thumbnail_key = $1, width = $2, height = $3 WHERE id = $4") + .bind(thumbnail_key) + .bind(width) + .bind(height) + .bind(id) + .execute(&self.pool) + .await?; + Ok(()) + } } diff --git a/apps/backend/src/repository/sqlite/access_token.rs b/apps/backend/src/repository/sqlite/access_token.rs index c169b5f5..5164c0e8 100644 --- a/apps/backend/src/repository/sqlite/access_token.rs +++ b/apps/backend/src/repository/sqlite/access_token.rs @@ -71,7 +71,7 @@ impl AccessTokenRepository for SqliteAccessTokenRepository { async fn find_by_prefix(&self, prefix: &str) -> Result, RepositoryError> { let rows = sqlx::query_as::<_, AccessTokenLookupRow>( - "SELECT id, site_id, token_hash, token_hmac, expires_at, revoked_at, permission + "SELECT id, site_id, token_hash, token_hmac, expires_at, revoked_at, permission, last_used_at FROM access_tokens WHERE token_prefix = ?", ) .bind(prefix) diff --git a/apps/backend/src/repository/sqlite/collection.rs b/apps/backend/src/repository/sqlite/collection.rs index 436a790d..147ed49d 100644 --- a/apps/backend/src/repository/sqlite/collection.rs +++ b/apps/backend/src/repository/sqlite/collection.rs @@ -127,6 +127,9 @@ impl CollectionRepository for SqliteCollectionRepository { entry_items: &[Entry], rename_map: &std::collections::HashMap, ) -> Result<(), RepositoryError> { + // One transaction for the whole migration: per-statement commit overhead + // dominated this loop, and a partial rename is never left behind. + let mut tx = self.pool.begin().await?; for entry in entry_items { if let Ok(mut data) = serde_json::from_str::(&entry.data) && let Some(obj) = data.as_object_mut() @@ -142,10 +145,11 @@ impl CollectionRepository for SqliteCollectionRepository { sqlx::query("UPDATE entries SET data = ?, updated_at = datetime('now') WHERE id = ?") .bind(&new_data_str) .bind(&entry.id) - .execute(&self.pool) + .execute(&mut *tx) .await?; } } + tx.commit().await?; Ok(()) } } diff --git a/apps/backend/src/repository/sqlite/entry.rs b/apps/backend/src/repository/sqlite/entry.rs index dcdd3c1c..3d709416 100644 --- a/apps/backend/src/repository/sqlite/entry.rs +++ b/apps/backend/src/repository/sqlite/entry.rs @@ -376,16 +376,22 @@ impl EntryRepository for SqliteEntryRepository { .execute(&mut *tx) .await?; - for file_id in &file_ids { - sqlx::query( - "INSERT OR IGNORE INTO entry_file_references (entry_id, file_id, site_id) SELECT ?, id, ? FROM files WHERE id = ? AND site_id = ?", - ) - .bind(entry_id) - .bind(site_id) - .bind(file_id) - .bind(site_id) - .execute(&mut *tx) - .await?; + if !file_ids.is_empty() { + // One multi-row statement instead of a query per file id; the + // SELECT keeps the "file must exist in this site" filter. + let placeholders = vec!["?"; file_ids.len()].join(", "); + let sql = format!( + "INSERT OR IGNORE INTO entry_file_references (entry_id, file_id, site_id) + SELECT ?, id, ? FROM files WHERE site_id = ? AND id IN ({placeholders})", + ); + let mut query = sqlx::query(sqlx::AssertSqlSafe(sql.as_str())) + .bind(entry_id) + .bind(site_id) + .bind(site_id); + for file_id in &file_ids { + query = query.bind(file_id); + } + query.execute(&mut *tx).await?; } tx.commit().await?; diff --git a/apps/backend/src/repository/sqlite/file.rs b/apps/backend/src/repository/sqlite/file.rs index c8a84704..542974ef 100644 --- a/apps/backend/src/repository/sqlite/file.rs +++ b/apps/backend/src/repository/sqlite/file.rs @@ -339,4 +339,21 @@ impl FileRepository for SqliteFileRepository { Ok(provider.unwrap_or_else(|| "filesystem".into())) } + + async fn set_thumbnail_meta( + &self, + id: &str, + thumbnail_key: &str, + width: Option, + height: Option, + ) -> Result<(), RepositoryError> { + sqlx::query("UPDATE files SET thumbnail_key = ?, width = ?, height = ? WHERE id = ?") + .bind(thumbnail_key) + .bind(width) + .bind(height) + .bind(id) + .execute(&self.pool) + .await?; + Ok(()) + } } diff --git a/apps/backend/src/repository/traits.rs b/apps/backend/src/repository/traits.rs index f0aa425f..7d9a6676 100644 --- a/apps/backend/src/repository/traits.rs +++ b/apps/backend/src/repository/traits.rs @@ -268,16 +268,25 @@ pub trait FileRepository: Send + Sync { site_id: &str, ) -> Result, RepositoryError>; async fn get_storage_provider(&self, site_id: &str) -> Result; + /// Attach thumbnail metadata generated after the upload response (background task). + async fn set_thumbnail_meta( + &self, + id: &str, + thumbnail_key: &str, + width: Option, + height: Option, + ) -> Result<(), RepositoryError>; } pub type AccessTokenLookupRow = ( - String, - String, - String, - Option, - Option, - Option, - String, + String, // id + String, // site_id + String, // token_hash + Option, // token_hmac + Option, // expires_at + Option, // revoked_at + String, // permission + Option, // last_used_at (for the touch debounce) ); /// A new access token to persist (see [`AccessTokenRepository::create`]). diff --git a/apps/backend/src/router/dashboard.rs b/apps/backend/src/router/dashboard.rs index 58739107..06e680b9 100644 --- a/apps/backend/src/router/dashboard.rs +++ b/apps/backend/src/router/dashboard.rs @@ -6,14 +6,18 @@ pub fn dashboard_routes() -> Router { Router::new() .route( "/dashboard", - get(|| async { dashboard_handler(axum::extract::Path("".into())).await }), + get(|headers: axum::http::HeaderMap| async move { + dashboard_handler(axum::extract::Path("".into()), headers).await + }), ) // Bare `/dashboard/` (trailing slash) matches neither `/dashboard` nor the // `{*file}` wildcard (which needs ≥1 segment); serve the SPA shell here too so a // refresh on a client route URL ending in `/` still loads the app. .route( "/dashboard/", - get(|| async { dashboard_handler(axum::extract::Path("".into())).await }), + get(|headers: axum::http::HeaderMap| async move { + dashboard_handler(axum::extract::Path("".into()), headers).await + }), ) .route("/dashboard/{*file}", get(dashboard_handler)) } diff --git a/apps/backend/src/router/files.rs b/apps/backend/src/router/files.rs index 8ad48968..257fac30 100644 --- a/apps/backend/src/router/files.rs +++ b/apps/backend/src/router/files.rs @@ -1,13 +1,14 @@ use axum::extract::DefaultBodyLimit; use axum::{ Router, - routing::{delete, get, post}, + routing::{delete, get, post, put}, }; use tower_http::limit::RequestBodyLimitLayer; use crate::handlers::file_handler::{ batch_delete_files, batch_permanent_delete_files, batch_restore_files, delete_file_handler, get_file, get_file_references, list_files, restore_file, serve_file, serve_file_thumbnail, upload_file, + upload_via_signed_url, }; /// Public API CRUD routes (mounted at /api/v1) @@ -36,6 +37,16 @@ pub fn file_serve_routes() -> Router { .route("/api/files/{id}/thumbnail", get(serve_file_thumbnail)) } +/// Signed-URL upload — standalone, no auth middleware: the HMAC token in the +/// path is the credential (minted by the MCP `create_upload_url` tool). +/// Registered at the literal path the tool advertises. +pub fn signed_upload_routes(max_upload_bytes: usize) -> Router { + Router::new() + .route("/api/v1/files/upload/{token}", put(upload_via_signed_url)) + .layer(DefaultBodyLimit::disable()) + .layer(RequestBodyLimitLayer::new(max_upload_bytes)) +} + /// Dashboard routes (mounted under /api/dashboard/sites/{site_id}) pub fn dashboard_routes(max_upload_bytes: usize) -> Router { Router::new() diff --git a/apps/backend/src/router/mod.rs b/apps/backend/src/router/mod.rs index 9bc212fd..ce2c4a53 100644 --- a/apps/backend/src/router/mod.rs +++ b/apps/backend/src/router/mod.rs @@ -142,6 +142,8 @@ pub fn create_router( ) // ── File serving (no auth — file IDs are effectively opaque) ── .merge(files::file_serve_routes()) + // ── Signed-URL upload (no auth — the HMAC token is the credential) ── + .merge(files::signed_upload_routes(max_upload_bytes)) // ── Dashboard API (/api/dashboard/*) ── .nest( "/api/dashboard", diff --git a/apps/backend/src/router/openapi.rs b/apps/backend/src/router/openapi.rs index df9fda4f..5affdfcc 100644 --- a/apps/backend/src/router/openapi.rs +++ b/apps/backend/src/router/openapi.rs @@ -41,6 +41,7 @@ use crate::models::site::Site; // Public API: Files crate::handlers::file_handler::list_files, crate::handlers::file_handler::upload_file, + crate::handlers::file_handler::upload_via_signed_url, crate::handlers::file_handler::get_file, crate::handlers::file_handler::delete_file_handler, crate::handlers::file_handler::get_file_references, diff --git a/apps/backend/src/secrets.rs b/apps/backend/src/secrets.rs index e3800a3b..18e6e16d 100644 --- a/apps/backend/src/secrets.rs +++ b/apps/backend/src/secrets.rs @@ -1,4 +1,4 @@ -//! Persisted instance secrets (`~/.vcms/secrets.toml`). +//! Persisted instance secrets (`secrets.toml` in the config dir). //! //! On first `serve`/`admin`, a random `HMAC_SECRET` value is //! generated and written to `secrets.toml` (perms `0600` on unix). Every later diff --git a/apps/backend/src/server.rs b/apps/backend/src/server.rs new file mode 100644 index 00000000..b47aea02 --- /dev/null +++ b/apps/backend/src/server.rs @@ -0,0 +1,331 @@ +//! Server bootstrap shared by `vcms serve` and the platform service runners. +//! +//! The full startup sequence (home dir, secrets, config, migrations, REST + gRPC) +//! lives here rather than in `main.rs` so that the Windows Service Control Manager +//! entry point — which lives in the library — can host the exact same server. The +//! caller injects a *shutdown future*; whatever resolves it (Ctrl+C / SIGTERM on a +//! normal run, an SCM stop control on Windows) triggers one graceful drain of both +//! the REST and gRPC servers. + +use std::error::Error; +use std::future::Future; +use std::net::SocketAddr; +use std::sync::Arc; + +use tracing::{debug, info, warn}; + +use crate::cli::Cli; +use crate::config::Config; +use crate::database::init_db_with_config; +use crate::grpc::server::spawn_grpc_server; +use crate::repository::Repository; +use crate::router::create_router; +use crate::services::Services; +use crate::storage::{self, STORAGE_KIND_FILESYSTEM, STORAGE_KIND_S3, StorageRegistry}; + +/// Email identity of the default admin account. Login is by email, so the seed +/// gate keys on this (non-unique display name "admin" would be the wrong column). +const ADMIN_EMAIL: &str = "admin@cms.local"; + +/// Boot the full server and run until `shutdown` resolves. +/// +/// Shared by the foreground `serve` command and the OS service runners. The +/// `shutdown` future is the single trigger that gracefully drains both the REST +/// and gRPC listeners. `on_ready` fires once startup has actually succeeded — +/// database open + migrated and both listeners bound — so a service host (the +/// Windows SCM runner) can report `Running` truthfully instead of optimistically. +pub async fn run( + cli: &Cli, + shutdown: impl Future + Send + 'static, + on_ready: impl FnOnce(), +) -> Result<(), Box> { + // Each early step gets a context prefix: a bare io error ("Access is denied. + // (os error 5)") from a service host is undebuggable without knowing which + // file/step produced it. + crate::paths::ensure().map_err(|e| format!("preparing data directories: {e}"))?; + crate::secrets::ensure().map_err(|e| format!("initializing secrets.toml: {e}"))?; + let config = Config::load(cli).map_err(|e| format!("loading configuration: {e}"))?; + + let _guard = crate::tracing::init_tracing(&config); + + if let Err(e) = config.validate_security() { + return Err(format!("Invalid production security configuration: {e}").into()); + } + + let pool = init_db_with_config(&config) + .await + .map_err(|e| format!("opening database: {e}"))?; + + let repository = Repository::new(&pool); + + seed_admin(&repository).await; + + let storage_registry = initialize_storage(&config); + // Build these once and share them: the REST router and the gRPC server use the + // same `Services` so the single-writer search index is opened only once. + let repository_arc = Arc::new(repository.clone()); + let config_arc = Arc::new(config.clone()); + let services = Services::new(repository_arc.clone(), &pool, &config); + + let backup_destination = crate::services::backup::build_backup_destination(&config) + .map_err(|e| format!("Failed to initialize backup destination: {e}"))?; + let backup_service = Arc::new(crate::services::backup::BackupService::new( + pool.clone(), + storage_registry.clone(), + backup_destination, + &config, + )); + + let app = create_router( + repository.clone(), + config.clone(), + storage_registry.clone(), + services.clone(), + backup_service.clone(), + ); + + // Reconcile backups/restore jobs left mid-flight by a previous process: any + // running/pending row at startup is orphaned (backups only run in-process). + match crate::services::backup::meta::fail_orphaned(&pool, &crate::services::backup::now_iso()).await { + Ok(n) if n > 0 => info!("Reconciled {n} interrupted backup job(s) to failed"), + Ok(_) => {} + Err(e) => tracing::error!("Failed to reconcile interrupted backups: {e}"), + } + + if config.backup_enabled { + let scheduler_service = backup_service.clone(); + tokio::spawn(async move { + crate::services::backup::scheduler::run(scheduler_service).await; + }); + info!("Backup scheduler started"); + } + + // The search indexer is the single writer/consumer: it rebuilds the index when + // empty (first run / wiped), then drains the cross-process queue forever. + if let (Some(search), Some(queue)) = (services.search.clone(), services.search_queue.clone()) { + let repo = repository_arc.clone(); + tokio::spawn(async move { + if search.is_empty() { + info!("Search index is empty; building from database..."); + match search.rebuild_all(&repo).await { + Ok(n) => info!("Search index built: {} entries", n), + Err(e) => tracing::error!("Search index build failed: {}", e), + } + } + crate::services::search::indexer::run(search, queue, repo).await; + }); + } + + let addr: SocketAddr = config + .bind_address + .parse() + .map_err(|e| format!("Invalid BIND_ADDRESS '{}': {e}", config.bind_address))?; + info!("Dashboard UI available at http://{}/dashboard", addr); + info!("REST API server running on http://{}", addr); + info!("GraphQL endpoint at http://{}/api/graphql", addr); + if config.mcp_enabled { + info!("MCP HTTP endpoint at http://{}/mcp", addr); + } + + let grpc_addr: SocketAddr = config + .grpc_bind_address + .parse() + .map_err(|e| format!("Invalid GRPC_BIND_ADDRESS '{}': {e}", config.grpc_bind_address))?; + info!("gRPC server running on {}", grpc_addr); + + // Bind both listeners *before* declaring readiness (and before the serve loops + // spawn): a bind failure — the classic "port already taken" — must surface as a + // startup error, not as a background task that dies after we claimed to be up. + let rest_listener = tokio::net::TcpListener::bind(addr) + .await + .map_err(|e| format!("Failed to bind REST address {addr}: {e}"))?; + let grpc_listener = tokio::net::TcpListener::bind(grpc_addr) + .await + .map_err(|e| format!("Failed to bind gRPC address {grpc_addr}: {e}"))?; + + on_ready(); + + // One shutdown signal fans out to both listeners via a watch channel: the + // injected `shutdown` future flips it, and both servers drain on the change. + let (shutdown_tx, shutdown_rx) = tokio::sync::watch::channel(false); + let shutdown_tx2 = shutdown_tx.clone(); + tokio::spawn(async move { + shutdown.await; + let _ = shutdown_tx.send(true); + }); + + let rest_rx = shutdown_rx.clone(); + let mut rest_handle = tokio::spawn(async move { + axum::serve( + rest_listener, + app.into_make_service_with_connect_info::(), + ) + .with_graceful_shutdown(wait_for_shutdown(rest_rx)) + .await + .map_err(|e| { + tracing::error!("REST server error: {e}"); + e + }) + }); + + let mut grpc_handle = tokio::spawn(spawn_grpc_server( + services.clone(), + repository_arc.clone(), + config_arc.clone(), + storage_registry.clone(), + grpc_listener, + Box::pin(wait_for_shutdown(shutdown_rx)), + )); + + // Whichever server finishes first flips the shutdown signal; the sibling is + // then awaited so both are fully drained before `run()` returns. + let (rest_result, grpc_result) = tokio::select! { + result = &mut rest_handle => { + let _ = shutdown_tx2.send(true); + let grpc_result = grpc_handle.await; + (result, grpc_result) + } + result = &mut grpc_handle => { + let _ = shutdown_tx2.send(true); + let rest_result = rest_handle.await; + (rest_result, result) + } + }; + + match rest_result { + Ok(Ok(())) => {} + Ok(Err(e)) => return Err(e.into()), + Err(e) => return Err(e.into()), + } + match grpc_result { + Ok(Ok(())) => {} + // gRPC's error is `Box`; format it so the + // conversion to this fn's `Box` is unambiguous. + Ok(Err(e)) => return Err(format!("gRPC server error: {e}").into()), + Err(e) => return Err(format!("gRPC server task panicked: {e}").into()), + } + + Ok(()) +} + +/// Resolve once the watch channel reports a shutdown was requested. +async fn wait_for_shutdown(mut rx: tokio::sync::watch::Receiver) { + if *rx.borrow_and_update() { + return; + } + let _ = rx.changed().await; +} + +/// Wait for a Ctrl+C or (on unix) SIGTERM — the foreground `serve` shutdown trigger. +pub async fn shutdown_signal() { + let ctrl_c = async { + tokio::signal::ctrl_c().await.expect("failed to install Ctrl+C handler"); + }; + + #[cfg(unix)] + let terminate = async { + tokio::signal::unix::signal(tokio::signal::unix::SignalKind::terminate()) + .expect("failed to install signal handler") + .recv() + .await; + }; + + #[cfg(not(unix))] + let terminate = std::future::pending::<()>(); + + tokio::select! { + _ = ctrl_c => info!("Received Ctrl+C, shutting down..."), + _ = terminate => info!("Received SIGTERM, shutting down..."), + } +} + +/// Register the configured storage backends (filesystem and/or S3). +pub fn initialize_storage(config: &Config) -> Arc { + let mut storage_registry = StorageRegistry::new(); + + // Use an explicit filesystem path if set; otherwise default to the data dir's + // storage/ so uploads work out of the box — unless S3 is configured and takes over. + let fs_path = match (&config.storage_fs_path, config.has_s3()) { + (Some(path), _) => Some(path.clone()), + (None, false) => Some(crate::paths::storage_dir().to_string_lossy().into_owned()), + (None, true) => None, + }; + + if let Some(fs_path) = fs_path { + match storage::FileSystemStorage::new(&fs_path) { + Ok(fs) => { + storage_registry.register(STORAGE_KIND_FILESYSTEM, Arc::new(fs)); + info!("Filesystem storage initialized at {}", fs_path); + } + Err(error) => warn!("Failed to initialize filesystem storage: {}", error), + } + } + + if config.has_s3() { + match storage::S3Storage::new( + config.s3_access_key_id.as_deref().unwrap(), + config.s3_secret_access_key.as_deref().unwrap(), + config.s3_bucket.as_deref().unwrap(), + config.s3_region.as_deref().unwrap_or("us-east-1"), + config.s3_endpoint.as_deref(), + config.s3_public_url.as_deref(), + ) { + Ok(s3) => { + storage_registry.register(STORAGE_KIND_S3, Arc::new(s3)); + info!("S3 storage initialized"); + } + Err(error) => warn!("Failed to initialize S3 storage: {}", error), + } + } + + if storage_registry.get(STORAGE_KIND_FILESYSTEM).is_none() && storage_registry.get(STORAGE_KIND_S3).is_none() { + warn!("No storage providers configured. Set STORAGE_FS_PATH or S3_* env vars."); + } + + Arc::new(storage_registry) +} + +async fn seed_admin(repository: &Repository) { + debug!("Checking if admin user needs to be seeded"); + let exists = match repository.user.exists(ADMIN_EMAIL).await { + Ok(exists) => exists, + Err(e) => { + tracing::error!("Failed to check for existing admin user; skipping seed: {e}"); + return; + } + }; + if !exists { + info!("Seeding default admin user"); + let id = uuid::Uuid::now_v7().to_string(); + let password_hash = bcrypt::hash("admin", bcrypt::DEFAULT_COST).expect("Failed to hash password"); + repository + .user + .create(&id, "admin", ADMIN_EMAIL, &password_hash) + .await + .expect("Failed to seed admin user"); + repository + .user + .set_instance_role(&id, Some("instance_owner")) + .await + .expect("Failed to assign instance owner"); + repository + .user + .update_password(&id, &password_hash, true) + .await + .expect("Failed to require password change"); + + warn!("Seeded default admin user (admin@cms.local / admin) — CHANGE THE PASSWORD IMMEDIATELY!"); + eprintln!( + "\n\ + ============================ SECURITY WARNING ============================\n\ + A default admin account was created: email 'admin@cms.local' password 'admin'\n\ + Sign in with the email and password. Anyone who can reach this server can log\n\ + in until you change it. Run:\n\ + \n vcms admin reset-password --email admin@cms.local --password \n\n\ + or change it from the dashboard now. Do NOT expose this server until done.\n\ + =========================================================================\n" + ); + } else { + debug!("Admin user already exists, skipping seeding"); + } +} diff --git a/apps/backend/src/service/linux.rs b/apps/backend/src/service/linux.rs new file mode 100644 index 00000000..bc63608b --- /dev/null +++ b/apps/backend/src/service/linux.rs @@ -0,0 +1,137 @@ +//! systemd integration for `vcms service` on Linux. + +use std::path::{Path, PathBuf}; +use std::process::Command; + +use super::{InstallOptions, SERVICE_NAME, require_root, resolve_run_user, systemd_unit}; +use crate::cli::{Cli, ServiceAction}; + +/// Where the generated unit file is written. +const UNIT_PATH: &str = "/etc/systemd/system/vcms.service"; + +/// System data directory the daemon owns (FHS convention for service state). Defined +/// once in `paths::system_home()` so a plain CLI invocation resolves to the same store; +/// always `Some` on Linux. +fn service_home() -> PathBuf { + crate::paths::system_home().expect("system_home is always set on Linux") +} + +pub fn dispatch(action: &ServiceAction, _cli: &Cli) -> Result<(), Box> { + match action { + ServiceAction::Install { user } => install(user.as_deref()), + ServiceAction::Uninstall => uninstall(), + ServiceAction::Status => status(), + ServiceAction::Start => start(), + ServiceAction::Stop => stop(), + } +} + +fn install(user: Option<&str>) -> Result<(), Box> { + require_root("install")?; + if !Path::new("/run/systemd/system").exists() { + return Err("systemd was not detected (no /run/systemd/system). Only systemd-based \ + Linux distributions are supported by `vcms service`." + .into()); + } + + let user = resolve_run_user(user)?; + // The daemon stores everything under one system dir (VCMS_HOME single-layout), + // owned by the run-as account — not that account's per-user `~/.config` etc. + let vcms_home = service_home(); + let exe_path = std::env::current_exe()?; + let opts = InstallOptions { + user: user.clone(), + exe_path, + vcms_home, + }; + + prepare_home(&opts)?; + + std::fs::write(UNIT_PATH, systemd_unit(&opts))?; + println!("Wrote {UNIT_PATH}"); + + run(Command::new("systemctl").arg("daemon-reload"))?; + run(Command::new("systemctl").args(["enable", "--now", SERVICE_NAME]))?; + + println!("Service '{SERVICE_NAME}' installed, enabled at boot, and started (running as {user})."); + println!("Secrets (Postgres/S3 only) go in {}", opts.env_file().display()); + println!("Check it with: vcms service status"); + Ok(()) +} + +fn uninstall() -> Result<(), Box> { + require_root("uninstall")?; + // Best-effort: the service may already be stopped/disabled. + let _ = run(Command::new("systemctl").args(["disable", "--now", SERVICE_NAME])); + if Path::new(UNIT_PATH).exists() { + std::fs::remove_file(UNIT_PATH)?; + println!("Removed {UNIT_PATH}"); + } + run(Command::new("systemctl").arg("daemon-reload"))?; + println!( + "Service '{SERVICE_NAME}' uninstalled. Your data under {} was left intact.", + service_home().display() + ); + Ok(()) +} + +fn start() -> Result<(), Box> { + require_root("start")?; + run(Command::new("systemctl").args(["start", SERVICE_NAME]))?; + println!("Service '{SERVICE_NAME}' started."); + Ok(()) +} + +fn stop() -> Result<(), Box> { + require_root("stop")?; + run(Command::new("systemctl").args(["stop", SERVICE_NAME]))?; + println!("Service '{SERVICE_NAME}' stopped."); + Ok(()) +} + +fn status() -> Result<(), Box> { + let enabled = output_trim(Command::new("systemctl").args(["is-enabled", SERVICE_NAME])); + let active = output_trim(Command::new("systemctl").args(["is-active", SERVICE_NAME])); + println!("Service: {SERVICE_NAME}"); + println!(" enabled at boot: {}", enabled.as_deref().unwrap_or("unknown")); + println!(" running: {}", active.as_deref().unwrap_or("unknown")); + // Detailed view (best-effort; ignored if the unit is absent). + let _ = Command::new("systemctl") + .args(["status", SERVICE_NAME, "--no-pager"]) + .status(); + Ok(()) +} + +/// Create the service home (`/var/lib/vcms`) + optional `.env` (0600) and hand +/// ownership to the run-as account so the daemon can read and write it. +fn prepare_home(opts: &InstallOptions) -> Result<(), Box> { + super::reject_symlink(&opts.vcms_home)?; + std::fs::create_dir_all(&opts.vcms_home)?; + let env_file = opts.env_file(); + super::create_secure_env_file(&env_file)?; + set_mode_600(&env_file)?; + run(Command::new("chown") + .arg("-R") + .arg(format!("{}:", opts.user)) + .arg(&opts.vcms_home))?; + Ok(()) +} + +fn set_mode_600(path: &Path) -> std::io::Result<()> { + use std::os::unix::fs::PermissionsExt; + std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o600)) +} + +fn run(cmd: &mut Command) -> Result<(), Box> { + let status = cmd.status()?; + if !status.success() { + return Err(format!("command {cmd:?} failed with {status}").into()); + } + Ok(()) +} + +fn output_trim(cmd: &mut Command) -> Option { + let out = cmd.output().ok()?; + let s = String::from_utf8_lossy(&out.stdout).trim().to_string(); + if s.is_empty() { None } else { Some(s) } +} diff --git a/apps/backend/src/service/macos.rs b/apps/backend/src/service/macos.rs new file mode 100644 index 00000000..cb07a1a7 --- /dev/null +++ b/apps/backend/src/service/macos.rs @@ -0,0 +1,155 @@ +//! launchd integration for `vcms service` on macOS (system LaunchDaemon). + +use std::path::{Path, PathBuf}; +use std::process::Command; + +use super::{InstallOptions, LAUNCHD_LABEL, launchd_plist, require_root, resolve_run_user}; +use crate::cli::{Cli, ServiceAction}; + +/// Where the generated LaunchDaemon plist is written. +const PLIST_PATH: &str = "/Library/LaunchDaemons/local.vcms.plist"; + +/// System data directory the daemon owns (macOS convention for shared app data). +/// Defined once in `paths::system_home()` so a plain CLI invocation resolves to the +/// same store; always `Some` on macOS. +fn service_home() -> PathBuf { + crate::paths::system_home().expect("system_home is always set on macOS") +} + +/// Modern launchctl service target (`system/