feat(portal): integrations feature (HeroUI migration) - #125
Merged
07prajwal2000 merged 27 commits intoJul 26, 2026
Conversation
Enhancements: Initializing UI migration
…em_users linking; pin bson to 7.2.0 CI: portal's `lint` (tsgo --noEmit) never ran the TanStack Router codegen, so the gitignored routeTree.gen.ts was missing in CI and every typed route call failed to typecheck. Add a portal build step before lint in the workflow so the route tree exists first. Auth: link Better Auth users to the canonical system_users row by shared id via a databaseHooks.user.create.before hook (pre-provisioned admins reuse their existing id; SSO JIT users get a system_users row created first as the FK target). create-user/service.ts now creates system_users directly instead of going through the auth-schema user table, and guards against duplicate emails. isSystemAdmin now lives on system_users and is attached to the session via customSession. Deps: mongodb's own bson subdependency was resolving to bson@7.3.1 despite the existing "bson": "7.2.0" resolutions pin, because bun only applies `resolutions` to the top-level package, not nested/transitive ones. bson@7.3.1 calls a node:v8 API Bun doesn't implement, which crashed every mongo-adapter test on import. Added a matching entry under `overrides` (which bun applies tree-wide) to force the nested mongodb->bson resolution down to 7.2.0 as well. Also bumps the migration journal for 0041_far_dreaming_celestial, adds apps/portal and packages/components package.json to the docker kit build context, and updates bun.lock. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…atabase-blocks fix(ci): generate portal route tree before lint; canonical system_users auth linking
Migrate hot-reload pub/sub from Redis to NATS and split the monolith into independently deployable images (admin control plane + scalable workers), keeping the all-in-one kit for quick runs. Worker & pub/sub - Add dedicated NATS abstraction (db/nats.ts) with token auth + reconnect, and a pub/sub layer (db/pubsub.ts); redis.ts is now cache-only and re-exports the pub/sub API so existing call sites are unchanged. - Gate the built-in worker in the admin server behind ENABLE_BUILTIN_WORKER; when false the admin process is control-plane only. - Add readiness/startup health endpoints and a waitForSchema() gate so a freshly started worker waits for the admin server's migrations instead of crashing on "relation app_config does not exist". - Graceful shutdown: SIGTERM/SIGINT handlers in standalone + worker, and a trap in the kit/admin entrypoints (fixes PID-1 signals hanging on stop). Docker - New thin worker image (single bundled JS + sourcemap, no node_modules). - New admin image (server + web + ai-gateway) and a Traefik-fronted production compose with two load-balanced worker replicas. - Kit image now also runs the worker; user API traffic routes to it. - Fix Bun bundle OTLP crash via --conditions=module across all builds. Docs & tooling - Rewrite the self-hosting guide as a decision hub; add Kit and Production pages; KeyGenerator now emits NATS_TOKEN. - Add k6 load-testing harness under testing/load-testing. - CI: kit publishes a rolling nightly on main; kit/admin/worker publish versioned + latest images on v* tags. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…-extraction feat(worker): extract request worker, migrate hot-reload to NATS, split Docker images
… env consolidation
…ured-logging feat: implement structured logging engine, browser bundle compatibility, and env consolidation
FIX: web module resolutions
feat: migrated from process.env to getEnv() function with validation …
Move harness progress off BullMQ job.updateProgress onto NATS pub/sub. BullMQ still triggers/retries runs; only the progress stream moved. - notifications.ts: conversations.<userId> subject, zod discriminated-union message contract, publish/subscribe/bridge helpers - callbacks/index: publish per-event and terminal states; resolve owner userId once per run; skip publish when conversation has no owner - redisService: whole-run-state snapshot persists during run, TTL drops to 60s on any terminal state via finalizeSnapshot - worker: await initializePubSub on init; drop dead QueueEvents bridge - main: start conversations.* bridge into in-process bus Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…rename Client-facing edge for harness progress. socket.io runs on the Bun engine and shares one Bun.serve router with Hono (no Hono adapter for socket.io), mounted under the proxy prefix /_/admin/ai/socket.io/. - socketGateway.ts: authenticates each connection against the better-auth session (handshake refused if unauthenticated), joins the per-user room conversations:<userId>, sends a full_state catch-up from Redis on connect, and fans NATS conversations.* events to rooms as typed update messages (discriminated on type; conversationId + stepId identify each update) - main.ts: route /_/admin/ai/socket.io/* to the engine, else to Hono - notifications.ts / queue.ts: drop the now-dead in-process EventEmitter bridge (socket.io subscribes to conversations.* directly) - rename harness sub-agent node builder -> blockBuilder (AgentNode.BLOCK_BUILDER) to disambiguate from the router "builder" intent; enum-driven so the LLM task-generator contract and graph dispatch stay consistent - docs: harness-client-implementation-guide.md (web client contract + React state edge cases) and notification-implementation.md updates Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…s-nats-progress feat(harness): NATS progress stream + socket.io live updates
added hero ui skills
Vertical-slice endpoints for the new agent harness conversation model (agent_harness_conversations/runs, replacing the legacy location/routeId coupling on ai_chat_conversations): - GET /harness-conversations - paginated list, needUserQuery=true adds a 30-char truncated latest query per conversation - PATCH /harness-conversations/:conversationId - rename - DELETE /harness-conversations/:conversationId - delete - POST /harness-conversations/message - send a message; omitting conversationId creates a new conversation + run and enqueues HARNESS_START_JOB (reuses enqueueHarnessStart); rejects with 409 if the target conversation already has a run in progress Status shown to the user is resolved live from the Redis run snapshot (harness:run:<runId>:snapshot) while a run is in flight, falling back to the conversation's DB status once idle/terminal - the DB status only flips on terminal events so it lags mid-run. Perf hardening for high concurrency on constrained (1 vCPU/1GB) pods: - list's status resolution is now batched (RedisService.getActiveRuns/ getSnapshots via MGET) instead of up to 2 Redis round trips per row - list cache invalidation on mutations uses an O(1) per-user version counter (cacheVersion.ts) instead of deleteCacheKeysByPattern, which runs Redis KEYS - an O(N) full-keyspace scan that blocks Redis's single thread - the post-mutation status refresh query only runs on a cache hit, not after a fresh DB build that already has current statuses Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…s-conversation-api feat: harness conversation api and basic agent UI
Faithfully reproduces the app/web integrations flow in apps/portal with the
new HeroUI/dark-lime look. Nothing about the API contract or the flow changes;
only the UI framework and styling.
Functional changes:
- 3-panel integrations page: left connector rail (Databases/KV/AI/BaaS/
Observability) driving ?group=, center configured-integrations accordion,
right filter panel (search + by-type + collapse).
- Connect flow: group -> variant cascade backed by the server helpers
(getIntegrationsGroups/Variants/getDefaultVariantValue/getSchema/
humanReadableConnectorNames), dispatching to the correct connector form.
- Connector forms: CredentialsUrlForm (Postgres/MySQL/Mongo/Redis/Memcached
with Credentials|URL toggle + optional SSL/database), AiForm (OpenAI/
Anthropic/Gemini/Mistral + OpenAI-Compatible baseUrl), ObservabilityForm
(Loki/OTel Logs with Base64|Credentials toggle).
- AppConfigSelector: every config field accepts a literal value or a
cfg:<key> reference, with a searchable app-config key picker.
- Create / edit (locked group+variant) / delete / test-connection wired to the
same endpoints as web (POST, PUT, DELETE, list/{group}, test-connection),
with getSchema validation before save.
- Deep-linkable via ?group= and ?open=.
Fixes a server-into-browser bundle bleed: the app-config DTO pulled
appConfigDataTypeEnum (a value) from db/schema, dragging @fluxify/lib -> node:vm
into the client and crashing the route. appConfig service DTO imports are now
type-only so the chain is elided. Removed verbatimModuleSyntax from the portal
tsconfig so server integration schemas can be imported for reuse.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Rework the project shell/sidebar to the mockup layout: gradient logo mark + FLUXIFY wordmark, project name, plain styled nav with lime active state, "Back to projects". - Design tokens: layered dark surfaces, lime accent, 12px radius, JetBrains Mono, softer focus halo, themed scrollbars and card refinements. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The integrations feature existed only in
app/web(Next.js + Mantine). This ports the entire integrations flow intoapps/portal(TanStack Router + HeroUI) with the new dark/lime look, so portal reaches feature parity with web for integrations. The API contract and user flow are unchanged — only the UI framework and styling differ.What
Integrations page (3-panel)
?group=.Connect / edit flow
getIntegrationsGroups,getIntegrationsVariants,getDefaultVariantValue,getSchema,humanReadableConnectorNames) — single source of truth, no duplicated connector metadata.cfg:<key>reference, backed by a searchable app-config key picker.POST /v1/{projectId}/integrationsPUT /v1/{projectId}/integrations/{id}DELETE /v1/{projectId}/integrations/{id}GET /v1/{projectId}/integrations/list/{group}GET /v1/{projectId}/integrations/{id}POST /v1/{projectId}/integrations/test-connectiongetSchema(group, variant)before save.?group=and?open=.Bug fix — server-into-browser bundle bleed
The integrations route crashed at load with:
Root cause: the
app-configDTO importedappConfigDataTypeEnum(a runtime value) fromdb/schema, which pulled the@fluxify/libbarrel →node:vminto the client bundle. Fixed by making the portalappConfigservice's DTO imports type-only, so the whole chain is elided at build time. Also removedverbatimModuleSyntaxfrom the portaltsconfigso the server integration schemas can be imported for reuse (adapters stay out of the bundle — verified via build).Supporting UI
Testing
Verified end-to-end in the browser against a running portal (:3001) + server (:5500), full CRUD round-trip with the network trace matching web:
POST 201GET list/{group} 200GET {id} 200DELETE 204→ list refetch200, no stray requestsTypecheck (
tsgo --noEmit) andvite buildare clean.