Skip to content

Commit 0eca28d

Browse files
committed
Add sync-ready gate, default protocol/domain, update docs with benchmark findings
- Sync-ready gate: containers wait for Mutagen sync via marker file, no more while-loop workarounds needed in commands - Default routing.protocol to http when port is set - Default domain to {name}.{scdev_domain} when not set - Update all docs: .pnpm-store must be in mutagen ignore (prevents platform-specific native binaries syncing between glibc/musl) - Update examples to use pnpm and node:22-alpine throughout - Update benchmark numbers with true cold start measurements - Remove node_modules named volume pattern from recommendations (mutagen ignore is the correct approach)
1 parent 5f67046 commit 0eca28d

6 files changed

Lines changed: 158 additions & 48 deletions

File tree

CLAUDE.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -53,9 +53,11 @@ When adding a new shared service (easy to miss steps):
5353

5454
## Gotchas
5555

56-
- Mutagen ignored paths are NOT synced in either direction. This breaks IDE autocomplete if `vendor/` or `node_modules/` are ignored.
56+
- Mutagen ignored paths are NOT synced in either direction. `node_modules` and `.pnpm-store` should always be ignored for Node.js projects - they stay inside the container for native speed. IDE autocomplete still works because `pnpm install` also runs on the host (or the IDE uses the host's own node_modules).
57+
- For pnpm projects, `.pnpm-store` MUST be in the mutagen ignore list. pnpm creates a ~500MB content-addressable store inside the project dir when running in a container. Without ignoring it, platform-specific native binaries (glibc vs musl) sync to the host and break when the container image changes.
5758
- Only directory bind mounts are synced via Mutagen. Single-file mounts stay as regular bind mounts.
5859
- The docs page (`docs.shared.<domain>`) doubles as a 404 catch-all via Traefik - unmatched URLs redirect there.
60+
- The sync-ready gate (`/.scdev-sync-ready` marker) automatically holds the container's command until Mutagen sync completes. No need for `while [ ! -f ... ]` workarounds in commands.
5961

6062
## README
6163

README.md

Lines changed: 52 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -75,13 +75,19 @@ name: my-app
7575

7676
services:
7777
app:
78-
image: node:20-alpine
79-
command: npm run dev
78+
image: node:22-alpine
79+
command: corepack enable && pnpm install && pnpm dev --host 0.0.0.0
8080
working_dir: /app
8181
volumes:
82-
- ${PROJECTPATH}:/app # mounts your project directory into the container
82+
- ${PROJECTPATH}:/app
8383
routing:
8484
port: 3000
85+
86+
mutagen:
87+
ignore:
88+
- node_modules
89+
- .pnpm-store
90+
- .nuxt
8591
```
8692
8793
`${PROJECTPATH}` is resolved automatically to your project's absolute path. Other available variables: `${PROJECTNAME}`, `${PROJECTDIR}`, `${SCDEV_DOMAIN}`.
@@ -124,25 +130,30 @@ Every project and shared service gets locally-trusted HTTPS certificates. Your b
124130

125131
File sharing between your host and containers is notoriously slow on macOS. scdev automatically syncs files at native speed - no configuration needed. On Linux this isn't needed (already fast).
126132

127-
How much difference does it make? We benchmarked `pnpm install` on a Nuxt app:
133+
How much difference does it make? We benchmarked a Nuxt 4 app with ~1000 dependencies:
128134

129-
| Approach | pnpm install | Dev server ready |
130-
|----------|-------------|-----------------|
131-
| Docker bind mount (default macOS) | **34.6s** | 7s |
132-
| scdev with file sync | **4.9s** | 4s |
135+
| Approach | pnpm install | Cold start to app ready |
136+
|----------|-------------|------------------------|
137+
| Docker bind mount (default macOS) | **34.6s** | ~42s |
138+
| scdev with file sync | **6.7s** | ~17s |
139+
| scdev warm restart (stop + start) | **2.4s** | ~2s |
133140

134-
That's a **7x speedup** on dependency installation. The trick: scdev syncs your source code via fast file sync, while keeping `node_modules` inside the container where filesystem operations are native speed.
141+
That's a **5x speedup** on cold start and **instant** warm restarts. The trick: scdev syncs your source code via fast file sync, while keeping `node_modules` and other generated files inside the container where filesystem operations are native speed.
135142

136-
Exclude paths you don't need synced:
143+
Exclude paths you don't need synced back to the host:
137144

138145
```yaml
139146
mutagen:
140147
ignore:
141148
- node_modules
149+
- .pnpm-store # pnpm's content-addressable store - platform-specific, don't sync
142150
- .nuxt
151+
- .output
143152
- var/cache
144153
```
145154

155+
**Important:** Always add `.pnpm-store` to the ignore list for pnpm projects. pnpm creates its package store inside the project directory when running in a container. Without ignoring it, ~500MB of platform-specific binaries sync to the host, causing slow syncs and broken native modules when switching images.
156+
146157
### TCP/UDP Routing
147158

148159
Beyond HTTPS, scdev can expose raw TCP and UDP ports. This lets you connect to a database inside a project from your host using tools like DBeaver, pgAdmin, or `psql`:
@@ -167,18 +178,18 @@ Multiple projects can expose different ports without conflicts. Works for MySQL,
167178

168179
### Volumes
169180

170-
**Bind mounts** (`${PROJECTPATH}:/app`) sync your source code into the container. Edits on the host are reflected immediately. On macOS, scdev handles fast sync automatically.
181+
**Bind mounts** (`${PROJECTPATH}:/app`) sync your source code into the container. Edits on the host are reflected immediately. On macOS, scdev handles fast sync automatically via Mutagen. Add `node_modules`, `.pnpm-store`, and build caches to `mutagen.ignore` so they stay inside the container (fast) and don't sync back to the host.
171182

172-
**Named volumes** (`node_modules:/app/node_modules`) are persistent storage managed by scdev. Use these for dependencies, database files, and caches - things that are huge, change constantly, and would kill sync performance:
183+
**Named volumes** (`db_data:/var/lib/postgresql/data`) are persistent storage managed by scdev. Use these for data that must survive `scdev down` - database files, uploaded assets, SQLite databases:
173184

174185
```yaml
175186
volumes:
176187
- ${PROJECTPATH}:/app # your source code (synced to host)
177-
- node_modules:/app/node_modules # dependencies (stays in container, fast)
178-
- db_data:/var/lib/postgresql/data # database files (persists across restarts)
188+
- db_data:/var/lib/postgresql/data # database files (persists across down)
189+
- data:/app/data # SQLite, uploads, etc.
179190
```
180191

181-
Named volumes persist across `scdev stop`/`scdev start` and are removed with `scdev down -v`. No separate declaration needed - scdev discovers them automatically.
192+
Named volumes persist across `scdev stop`/`scdev start` AND `scdev down`. Only removed with `scdev down -v`. No separate declaration needed - scdev discovers them automatically.
182193

183194
### Custom Commands
184195

@@ -194,15 +205,15 @@ Every project has recurring tasks: install deps, run migrations, seed data, run
194205
```bash
195206
# .scdev/commands/setup.just
196207
default:
197-
scdev exec app npm ci
208+
scdev exec app pnpm ci
198209
scdev exec app npx prisma db push
199210
200211
# .scdev/commands/test.just
201212
default:
202-
scdev exec app npm test
213+
scdev exec app pnpm test
203214
204215
watch:
205-
scdev exec app npm test -- --watch
216+
scdev exec app pnpm test -- --watch
206217
```
207218

208219
```bash
@@ -233,7 +244,7 @@ scdev down -v # Remove everything including volumes
233244

234245
```bash
235246
scdev exec app bash # Shell into a container
236-
scdev exec app npm test # Run a command
247+
scdev exec app pnpm test # Run a command
237248
scdev logs # View logs
238249
scdev logs -f app # Follow logs for a service
239250
```
@@ -299,12 +310,11 @@ name: my-api
299310
300311
services:
301312
app:
302-
image: node:20-alpine
303-
command: npm run dev
313+
image: node:22-alpine
314+
command: corepack enable && pnpm install && pnpm dev --host 0.0.0.0
304315
working_dir: /app
305316
volumes:
306317
- ${PROJECTPATH}:/app
307-
- node_modules:/app/node_modules
308318
environment:
309319
DATABASE_URL: postgres://postgres:postgres@db:5432/app
310320
routing:
@@ -317,6 +327,12 @@ services:
317327
environment:
318328
POSTGRES_PASSWORD: postgres
319329
POSTGRES_DB: app
330+
331+
mutagen:
332+
ignore:
333+
- node_modules
334+
- .pnpm-store
335+
- .nuxt
320336
```
321337

322338
### Static Site / Frontend
@@ -326,13 +342,18 @@ name: my-docs
326342
327343
services:
328344
app:
329-
image: node:20-alpine
330-
command: npm run dev
345+
image: node:22-alpine
346+
command: corepack enable && pnpm install && pnpm dev --host 0.0.0.0
331347
working_dir: /app
332348
volumes:
333349
- ${PROJECTPATH}:/app
334350
routing:
335351
port: 5173
352+
353+
mutagen:
354+
ignore:
355+
- node_modules
356+
- .pnpm-store
336357
```
337358

338359
## Configuration Reference
@@ -344,13 +365,18 @@ name: my-app
344365
345366
services:
346367
app:
347-
image: node:20-alpine
348-
command: npm run dev
368+
image: node:22-alpine
369+
command: corepack enable && pnpm install && pnpm dev --host 0.0.0.0
349370
working_dir: /app
350371
volumes:
351372
- ${PROJECTPATH}:/app
352373
routing:
353374
port: 3000
375+
376+
mutagen:
377+
ignore:
378+
- node_modules
379+
- .pnpm-store
354380
```
355381

356382
### Full config with all options

internal/config/loader.go

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,19 @@ func LoadProject(projectDir string) (*ProjectConfig, error) {
7777
cfg.Name = projectName
7878
}
7979

80+
// Default domain to {name}.{scdev_domain}
81+
if cfg.Domain == "" {
82+
cfg.Domain = cfg.Name + "." + DefaultDomain
83+
}
84+
85+
// Default routing protocol to "http" when port is set
86+
for name, svc := range cfg.Services {
87+
if svc.Routing != nil && svc.Routing.Port != 0 && svc.Routing.Protocol == "" {
88+
svc.Routing.Protocol = "http"
89+
cfg.Services[name] = svc
90+
}
91+
}
92+
8093
return &cfg, nil
8194
}
8295

internal/project/mutagen_sync.go

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -294,3 +294,16 @@ func (p *Project) transformVolumesForMutagen(serviceName string, volumes []strin
294294

295295
return result
296296
}
297+
298+
// signalSyncReady writes a marker file into each container that has a Mutagen sync mount,
299+
// unblocking the sync-ready gate in the container's entrypoint wrapper.
300+
func (p *Project) signalSyncReady(ctx context.Context, mounts []MutagenSyncMount) {
301+
for _, mount := range mounts {
302+
containerName := p.ContainerName(mount.ServiceName)
303+
err := p.Runtime.Exec(ctx, containerName,
304+
[]string{"sh", "-c", "touch /.scdev-sync-ready"}, false, runtime.ExecOptions{})
305+
if err != nil {
306+
fmt.Printf("Warning: could not signal sync-ready for %s: %v\n", mount.ServiceName, err)
307+
}
308+
}
309+
}

internal/project/project.go

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,11 @@ func (p *Project) NetworkName() string {
6464
return fmt.Sprintf("%s.scdev", p.Config.Name)
6565
}
6666

67+
// shellQuote wraps a string in single quotes, escaping any embedded single quotes.
68+
func shellQuote(s string) string {
69+
return "'" + strings.ReplaceAll(s, "'", "'\\''") + "'"
70+
}
71+
6772
// VolumeName returns the full volume name for a project volume
6873
// Format: <volume>.<project>.scdev (e.g., db_data.myproject.scdev)
6974
func (p *Project) VolumeName(volume string) string {
@@ -282,6 +287,9 @@ func (p *Project) Start(ctx context.Context) error {
282287

283288
// Wait for initial sync (60 second timeout)
284289
p.waitForInitialSync(ctx, m, mutagenMounts, 60*time.Second)
290+
291+
// Signal containers that sync is ready (unblocks the sync-ready gate)
292+
p.signalSyncReady(ctx, mutagenMounts)
285293
}
286294

287295
// Register project with routing info in state
@@ -404,7 +412,17 @@ func (p *Project) startServiceWithMutagen(ctx context.Context, name string, svc
404412

405413
// Parse command if specified
406414
if svc.Command != "" {
407-
cfg.Command = []string{"sh", "-c", svc.Command}
415+
// When Mutagen is enabled for this service, wrap the command with a sync-ready gate.
416+
// Files arrive via sync AFTER the container starts, so the command would fail immediately
417+
// without this gate. The marker file is written by signalSyncReady() after initial sync.
418+
_, hasMutagenMount := mutagenMounts[name]
419+
if mutagenEnabled && hasMutagenMount {
420+
cfg.Command = []string{"sh", "-c",
421+
"while [ ! -f /.scdev-sync-ready ]; do sleep 0.2; done; exec sh -c " + shellQuote(svc.Command),
422+
}
423+
} else {
424+
cfg.Command = []string{"sh", "-c", svc.Command}
425+
}
408426
}
409427

410428
// Create and start

0 commit comments

Comments
 (0)