Use dynamic software factory harness ports

[habdelra]

Summary

• lay the groundwork for CS-10419 by making the software-factory Playwright harness safe to run alongside other harness instances without port collisions
• switch the harness stack to dynamic port assignment throughout the isolated realm flow instead of assuming canonical local dev ports
• align harness port lifetime with the Playwright testWorker, so each worker gets one stable set of harness ports for the duration of the worker even though realm-server and worker-manager still restart between tests
• align prerender lifetime with the Playwright testWorker, so restarted realm stacks can reuse the same prerender process and avoid repeated standby warmup
• run the software-factory Playwright suite with workers=2 as proof that multiple harnesses can execute concurrently without interfering with each other
• reduce harness log noise by treating expected standby startup misses during realm-server turnover as normal readiness behavior instead of noisy 502/503 errors
• trim one redundant isolated boot in factory-bootstrap.spec.ts by combining the bootstrap creation and idempotency assertions into a single realm run

Why

This PR is part of the groundwork for CS-10419. To support multiple test harnesses running side by side, the harness cannot depend on fixed ports or on per-test port churn that makes URLs unstable inside a worker. The harness now allocates ports dynamically across the stack, keeps those URLs stable for the lifetime of each Playwright testWorker, reuses prerender across tests within that worker, and validates the design by running the browser suite with workers=2.

Testing

• pnpm --dir packages/software-factory exec playwright test tests/factory-bootstrap.spec.ts tests/factory-target-realm.spec.ts --workers=2 --reporter=line
• pnpm --dir packages/software-factory exec playwright test --workers=2 --reporter=line

[github-actions]

Preview deployments

• Host staging preview
• Host production preview

[github-actions]

Host Test Results

    1 files  ±    0      1 suites  ±0   4h 4m 56s ⏱️ + 1h 57m 7s
2 032 tests +    2  2 016 ✅ +    1  15 💤 ± 0  0 ❌ ±0  1 🔥 +1 
4 010 runs  +1 965  3 978 ✅ +1 948  30 💤 +15  1 ❌ +1  1 🔥 +1 

For more details on these errors, see this check.

Results for commit 8f45d6f. ± Comparison against base commit b77095c.

♻️ This comment has been updated with latest results.

[Copilot]

Pull request overview

This PR refactors the software-factory Playwright harness to use dynamically allocated ports and worker-scoped (stable) URLs so multiple harness workers/instances can run concurrently with fewer collisions, while also reducing prerender standby noise and improving error formatting.

Changes:

• Introduces a new modular harness implementation (src/harness/*) that supports dynamic port assignment, compat proxying, and template DB caching metadata.
• Updates Playwright fixtures/tests to use worker-scoped port sets + prerender reuse, and aligns timeouts for slower cold-start scenarios.
• Improves robustness/diagnostics: readiness checks, [object Object]-safe error formatting, and reduced prerender standby startup noise.

Reviewed changes

Copilot reviewed 53 out of 54 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
packages/software-factory/tests/helpers/browser-auth.ts	Requires explicit realm-server URL for auth token bootstrap; makes init-script resilient when storage is unavailable.
packages/software-factory/tests/fixtures.ts	Adds worker-scoped dynamic port allocation and worker-scoped prerender reuse; threads realmServerURL through auth.
packages/software-factory/tests/factory-target-realm.test.ts	Extends unit tests to assert returned realm authorization and readiness/auth flows.
packages/software-factory/tests/factory-target-realm.spec.ts	Uses dynamic realmServerURL when computing target realm URLs in E2E.
packages/software-factory/tests/factory-entrypoint.test.ts	Tests darkfactory module URL resolution from the resolved realm server URL.
packages/software-factory/tests/factory-entrypoint.integration.test.ts	Adds _realm-auth + readiness-check handling and tracks created cards for idempotency.
packages/software-factory/tests/factory-bootstrap.test.ts	Adds regression test for non-serialized response bodies and improves mock fetch to simulate created cards.
packages/software-factory/tests/factory-bootstrap.spec.ts	Consolidates bootstrap/idempotency assertions and adjusts navigation/waits for reliability.
packages/software-factory/tests/darkfactory.spec.ts	Normalizes navigation/wait strategy and increases visibility timeouts to reduce flake.
packages/software-factory/test-fixtures/public-software-factory-source/darkfactory.gts	Switches fixture to reference shared realm module source rather than duplicating the full card module.
packages/software-factory/test-fixtures/darkfactory-adopter/ticket-demo.json	Replaces hard-coded localhost adoptsFrom module URL with placeholder origin for rewrite.
packages/software-factory/test-fixtures/darkfactory-adopter/project-demo.json	Same placeholder adoptsFrom module URL update.
packages/software-factory/test-fixtures/darkfactory-adopter/knowledge-article-demo.json	Same placeholder adoptsFrom module URL update.
packages/software-factory/test-fixtures/darkfactory-adopter/factory-demo.json	Same placeholder adoptsFrom module URL update.
packages/software-factory/test-fixtures/darkfactory-adopter/agent-demo.json	Same placeholder adoptsFrom module URL update.
packages/software-factory/test-fixtures/darkfactory-adopter/Ticket/ticket-001.json	Same placeholder adoptsFrom module URL update.
packages/software-factory/test-fixtures/darkfactory-adopter/Project/demo-project.json	Same placeholder adoptsFrom module URL update.
packages/software-factory/test-fixtures/darkfactory-adopter/KnowledgeArticle/agent-onboarding.json	Same placeholder adoptsFrom module URL update.
packages/software-factory/test-fixtures/darkfactory-adopter/DarkFactory/demo-factory.json	Same placeholder adoptsFrom module URL update.
packages/software-factory/test-fixtures/darkfactory-adopter/AgentProfile/demo-agent.json	Same placeholder adoptsFrom module URL update.
packages/software-factory/src/runtime-metadata.ts	Adds template metadata cache IO utilities and ensures metadata directories exist.
packages/software-factory/src/realm-auth.ts	Adds optional “prime realm” step to pre-fetch/reauth before first request.
packages/software-factory/src/harness/support-services.ts	New: starts support services (synapse/pg/icons/host) and adds prerender server launcher.
packages/software-factory/src/harness/shared.ts	New: shared harness types/constants/helpers (ports, URLs, logging, fixture hashing, etc.).
packages/software-factory/src/harness/isolated-realm-stack.ts	New: isolated stack runner with compat proxy, dynamic ports, fixture URL rewriting, and managed process lifecycle.
packages/software-factory/src/harness/database.ts	New: PG template cloning/snapshotting + URL rewrite + queue/index reset utilities.
packages/software-factory/src/harness/api.ts	New: public harness API (global context, template cache, runtime realm server).
packages/software-factory/src/harness.ts	Replaces monolithic harness implementation with re-exports from the new modular harness.
packages/software-factory/src/factory-target-realm.ts	Returns realm authorization, adds readiness wait, and uses consistent error formatting.
packages/software-factory/src/factory-entrypoint.ts	Uses returned realm authorization + primes realm auth; resolves darkfactory module URL from serverUrl.
packages/software-factory/src/factory-bootstrap.ts	Adds post-write read-back readiness for cards and uses robust error-body formatting.
packages/software-factory/src/error-format.ts	New: shared error formatting to avoid [object Object] leaks and improve diagnostics.
packages/software-factory/src/cli/serve-support.ts	Keeps support wrapper alive for teardown signaling; ensures shared services stay attached.
packages/software-factory/src/cli/serve-realm.ts	Emits runtime ports/realmServerURL metadata; keeps wrapper alive for explicit shutdown.
packages/software-factory/src/cli/cache-realm.ts	Emits more cache metadata and exits explicitly.
packages/software-factory/scripts/lib/boxel.ts	Uses shared error formatting for clearer API errors.
packages/software-factory/playwright.global-setup.ts	Prepares template caches for multiple realm dirs and reduces log noise via filtered progress logging.
packages/software-factory/playwright.config.ts	Increases timeouts and sets workers=2 to exercise parallelism.
packages/software-factory/README.md	Updates documentation to reflect harness-managed ports and placeholder fixture URL rewriting.
packages/runtime-common/virtual-network.ts	Adds mapURL() helper for resolving URL mappings in either direction.
packages/runtime-common/router.ts	Improves 500 error responses to avoid [object Object] and provide better error detail.
packages/realm-server/worker-manager.ts	Writes runtime metadata (pid/port/url) for harness coordination.
packages/realm-server/tests/virtual-network-test.ts	Adds logger setup and tests new VirtualNetwork.mapURL() behavior.
packages/realm-server/server.ts	Adds / _standby host-app serving, injects more runtime config into index HTML, and improves URL mapping behavior.
packages/realm-server/routes.ts	Adds / _standby route wiring for standby warmup via realm-server.
packages/realm-server/prerender/prerenderer.ts	Treats expected standby warmup misses as debug noise rather than warnings.
packages/realm-server/prerender/page-pool.ts	Adds StandbyTargetNotReadyError and makes standby warmup more resilient with clearer logging.
packages/realm-server/middleware/index.ts	Adds proxy request header injection and uses x-boxel-forwarded-url for full request URL construction.
packages/realm-server/main.ts	Writes realm-server runtime metadata (pid/port) for harness coordination.
packages/postgres/pg-adapter.ts	Adds PG_POOL_MAX/PGPOOLMAX support to cap pool size for parallel workers.
packages/matrix/helpers/environment-config.ts	Allows getSynapseURL() to derive from a provided synapse instance.
packages/matrix/docker/synapse/index.ts	Supports dynamic host port selection and returns resolved baseUrl/port in synapse instance.
packages/host/app/router.ts	Moves host standby route to /_standby to match prerender expectations.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[backspace]

Is there a reason to have two names for this?

[habdelra]

Removed the duplicate name in 8f45d6fafa. I checked the actual pg source and neither PG_POOL_MAX nor PGPOOLMAX is a built-in pg env var, so this was our own adapter aliasing rather than compatibility with libpq/node-postgres. I standardized on PG_POOL_MAX and dropped PGPOOLMAX.