Skip to content

feat(agentic): add Agentic runtime app#870

Open
lumberman wants to merge 9 commits into
mainfrom
feat/115-mastra-agentic-runtime-app
Open

feat(agentic): add Agentic runtime app#870
lumberman wants to merge 9 commits into
mainfrom
feat/115-mastra-agentic-runtime-app

Conversation

@lumberman
Copy link
Copy Markdown
Collaborator

@lumberman lumberman commented May 1, 2026

Summary

  • Add apps/agentic as @forge/agentic, with Mastra runtime, Studio, auth-gated service/operator boundaries, LibSQL runtime storage config, package-local docs, and Railway config for the future agentic service.
  • Add Manager-to-Agentic and Agentic-to-Manager HTTP contracts for the first automation dry-run flow; V1 rejects live mode and keeps Manager-owned dry-run reports/job boundaries authoritative.
  • Resolve review blockers: refreshed lockfile against origin/main, hardened production env validation, enforced distinct service/operator/Manager tokens, made dry-run idempotency durable, preserved Manager semantic errors, and added Agentic callback timeouts.

Red/Green TDD

  • Red: focused Agentic env/workflow tests failed before fixes for weak production secrets, token reuse, callback error semantics, and timeout behavior.
  • Red: focused Manager route/store tests failed before fixes for unused/missing idempotency handling and duplicate run creation.
  • Green: Agentic env/auth/workflow tests pass with production hardening, typed Manager failure propagation, and request timeout coverage.
  • Green: Manager route/store tests pass with required idempotency keys, existing-run reuse, in-progress rejection, and completion-failure cleanup coverage.

Validation

  • pnpm --filter @forge/agentic test
  • pnpm --filter @forge/agentic typecheck
  • pnpm --filter @forge/agentic lint
  • pnpm --filter @forge/agentic build
  • pnpm --filter @forge/manager test -- src/app/api/automations src/features/agents src/lib/agentic-automation-dry-run.test.ts src/config/env.test.ts
  • pnpm --filter @forge/manager typecheck
  • pnpm --filter @forge/manager lint
  • pnpm --filter @forge/graphql typecheck
  • pnpm --filter @forge/cms typecheck
  • pnpm turbo run generate --filter=@forge/graphql
  • pnpm run format:check
  • git diff --check

User Smoke Test

  • Built Agentic server locally at http://127.0.0.1:4114 with a fake Manager service at http://127.0.0.1:3999.
  • GET /health returned HTTP 200.
  • Anonymous Studio GET / returned HTTP 401.
  • Service bearer Studio GET / returned HTTP 401.
  • Operator bearer Studio GET / returned HTTP 200 and rendered the Studio shell.
  • Service bearer POST /forge/manager-automation-dry-run returned success with a Manager run id.
  • Repeating the same idempotency key returned the same Manager run id.
  • Manager not_found and invalid_automation failures were preserved in the Agentic response; invalid Manager JSON returned manager_unavailable.
  • Screenshot evidence captured at /tmp/forge-agentic-smoke/operator-studio.png.

Post-Deploy Monitoring & Validation

  • Railway service should be named agentic, and the service Config-as-code Path must resolve to apps/agentic/railway.toml; treat a missing/null deployment configFile as a failed live verification.
  • Confirm deployed GET /health returns HTTP 200 without auth.
  • Confirm deployed Studio rejects anonymous and service-token access, while the operator token can reach Studio.
  • Confirm /forge/manager-automation-dry-run rejects anonymous/operator tokens and accepts only the service token.
  • Trigger one deployed Manager dry-run smoke and verify Manager report data keeps dry-run semantics with no live enrichment jobs created.
  • Watch Railway/Manager logs for agentic, manager-automation-dry-run, manager_unavailable, run_in_progress, dry_run_failed, token validation errors, storage errors, callback timeouts, and duplicate idempotency records during the first 30 minutes after deploy.
  • Healthy signals: CI green, Agentic /health 200, semantic Manager 4xx responses preserved, repeated idempotency key returns the same run, and no Studio access with the service token.
  • Failure signals/rollback: 5xx spike, timeout spike, storage boot errors, duplicate Manager run records, configFile unset for the Railway service, or service-token Studio access. Roll back the Agentic deployment and disable Manager Agentic integration env until corrected.

@railway-app
Copy link
Copy Markdown

railway-app Bot commented May 1, 2026

🚅 Deployed to the forge-pr-870 environment in forge

Service Status Web Updated (UTC)
@forge/web ✅ Success (View Logs) May 2, 2026 at 3:15 pm
@forge/manager ✅ Success (View Logs) May 2, 2026 at 3:11 pm
@forge/cms ✅ Success (View Logs) May 2, 2026 at 3:05 pm

@lumberman
Copy link
Copy Markdown
Collaborator Author

Follow-up validation for the Mastra docs/setup audit:

Summary

  • Scoped MASTRA_SERVICE_API_KEY to only POST /forge/manager-automation-dry-run; Studio/root and built-in Mastra API routes now require MASTRA_OPERATOR_API_KEY.
  • Removed the custom /health route and rely on Mastra's built-in GET /health endpoint.
  • Added Railway PORT fallback while keeping MASTRA_PORT as the explicit override.
  • Updated apps/mastra docs and the implementation plan with the follow-up Red/Green and smoke evidence.

Red/Green TDD

  • Red: pnpm --filter @forge/mastra test failed when PORT was ignored and a service bearer token could pass global middleware for Studio/root access.
  • Green: added tests for PORT fallback/precedence, operator vs service middleware behavior, direct server.auth.authorize, invalid dry-run bearer rejection, and built-in public health behavior.

Validation

  • pnpm --filter @forge/mastra test
  • pnpm --filter @forge/mastra typecheck
  • pnpm --filter @forge/mastra lint
  • pnpm --filter @forge/mastra build
  • pnpm --filter @forge/manager test -- src/app/api/automations src/features/agents src/lib/mastra-automation-dry-run.test.ts
  • pnpm --filter @forge/manager typecheck
  • pnpm run format:check
  • git diff --check

User Smoke

Built Mastra ran on http://127.0.0.1:4112 with PORT=4112 and MASTRA_PORT unset.

  • GET /health -> 200, {"success":true}
  • anonymous GET / -> 401
  • operator bearer GET / -> 200
  • service bearer GET / -> 401
  • service bearer GET /api/agents -> 401
  • service bearer POST /forge/manager-automation-dry-run -> 200, managerAutomationRunDocumentId: "manager-run-smoke-2"

Screenshots captured locally:

  • output/mastra-studio-operator-auth-followup.png
  • output/mastra-studio-service-rejected-followup.png

@railway-app railway-app Bot temporarily deployed to forge / forge-pr-870 May 1, 2026 04:54 Destroyed
@lumberman lumberman changed the title feat(mastra): add Mastra agentic runtime app feat(agentic): add Agentic runtime app May 1, 2026
@lumberman
Copy link
Copy Markdown
Collaborator Author

Renamed the app boundary from Mastra to Agentic per product/deploy naming preference.

What changed

  • App folder is now apps/agentic.
  • Package is now @forge/agentic.
  • Railway intended service/config path is now agentic / apps/agentic/railway.toml.
  • CI/PR scope allowlist now uses agentic instead of mastra.
  • Manager service envs are now AGENTIC_BASE_URL, AGENTIC_SERVICE_API_KEY, MANAGER_AGENTIC_API_KEY, and AGENTIC_REQUEST_TIMEOUT_MS.
  • Manager dry-run callback route is now /api/automations/{id}/agentic-dry-run.
  • Public contract response field is now agenticRunId instead of mastraRunId.

Mastra remains only as the underlying framework/Studio/runtime implementation name, including the required src/mastra project structure.

Validation

  • pnpm --filter @forge/agentic test
  • pnpm --filter @forge/agentic typecheck
  • pnpm --filter @forge/agentic lint
  • pnpm --filter @forge/agentic build
  • pnpm --filter @forge/manager test -- src/app/api/automations src/features/agents src/lib/agentic-automation-dry-run.test.ts
  • pnpm --filter @forge/manager typecheck
  • pnpm run format:check
  • git diff --check

Runtime smoke

Built app started from apps/agentic/.mastra/output/index.mjs with AGENTIC_* vars and PORT=4113:

  • GET /health -> 200 {"success":true}
  • service bearer GET / -> 401
  • operator bearer GET / -> 200
  • service bearer POST /forge/manager-automation-dry-run -> 200, agenticRunId and managerAutomationRunDocumentId: "manager-run-agentic-smoke"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant