Skip to content

feat(portal): integrations feature (HeroUI migration) - #125

Merged
07prajwal2000 merged 27 commits into
Fluxify-rest:ui-migrationfrom
yashashkumar:ui-migration-to-hero
Jul 26, 2026
Merged

feat(portal): integrations feature (HeroUI migration)#125
07prajwal2000 merged 27 commits into
Fluxify-rest:ui-migrationfrom
yashashkumar:ui-migration-to-hero

Conversation

@yashashkumar

Copy link
Copy Markdown
Collaborator

Why

The integrations feature existed only in app/web (Next.js + Mantine). This ports the entire integrations flow into apps/portal (TanStack Router + HeroUI) with the new dark/lime look, so portal reaches feature parity with web for integrations. The API contract and user flow are unchanged — only the UI framework and styling differ.

What

Integrations page (3-panel)

  • Left rail — connector categories (Databases / KV / AI / BaaS / Observability), drives ?group=.
  • Center — configured integrations for the selected group as an accordion; empty state prompts "Connect App/Service".
  • Right — filter panel: search by name, filter by type, collapsible.

Connect / edit flow

  • Group → variant cascade sourced from the server helpers (getIntegrationsGroups, getIntegrationsVariants, getDefaultVariantValue, getSchema, humanReadableConnectorNames) — single source of truth, no duplicated connector metadata.
  • Dispatches to the right connector form:
    • CredentialsUrlForm — Postgres / MySQL / MongoDB / Redis / Memcached, with a Credentials|URL toggle and optional SSL / database fields.
    • AiForm — OpenAI / Anthropic / Gemini / Mistral, plus OpenAI-Compatible (base URL).
    • ObservabilityForm — Loki / OpenTelemetry Logs, with a Base64|Credentials toggle.
  • AppConfigSelector — every config field accepts either a literal value or a cfg:<key> reference, backed by a searchable app-config key picker.
  • Create / edit / delete / test-connection hit the same endpoints as web:
    • POST /v1/{projectId}/integrations
    • PUT /v1/{projectId}/integrations/{id}
    • DELETE /v1/{projectId}/integrations/{id}
    • GET /v1/{projectId}/integrations/list/{group}
    • GET /v1/{projectId}/integrations/{id}
    • POST /v1/{projectId}/integrations/test-connection
  • Config is validated against getSchema(group, variant) before save.
  • On edit, group + variant are locked (matching web).
  • Deep-linkable via ?group= and ?open=.

Bug fix — server-into-browser bundle bleed

The integrations route crashed at load with:

The requested module '…__vite-browser-external:vm' does not provide an export named 'Script'

Root cause: the app-config DTO imported appConfigDataTypeEnum (a runtime value) from db/schema, which pulled the @fluxify/lib barrel → node:vm into the client bundle. Fixed by making the portal appConfig service's DTO imports type-only, so the whole chain is elided at build time. Also removed verbatimModuleSyntax from the portal tsconfig so the server integration schemas can be imported for reuse (adapters stay out of the bundle — verified via build).

Supporting UI

  • Project shell / sidebar restyled to the mockup (gradient logo mark, FLUXIFY wordmark, lime active nav, "Back to projects").
  • Design-token refinements: layered dark surfaces, lime accent, 12px radius, JetBrains Mono, softer focus halo, themed scrollbars, card polish.

Testing

Verified end-to-end in the browser against a running portal (:3001) + server (:5500), full CRUD round-trip with the network trace matching web:

  • create → POST 201
  • list refetch → GET list/{group} 200
  • expand/edit → GET {id} 200
  • delete → DELETE 204 → list refetch 200, no stray requests
  • Test Connection, AppConfigSelector key lookup, group/variant cascade, and filters all exercised.

Typecheck (tsgo --noEmit) and vite build are clean.

07prajwal2000 and others added 27 commits July 23, 2026 22:26
Enhancements: Initializing UI migration
…em_users linking; pin bson to 7.2.0

CI: portal's `lint` (tsgo --noEmit) never ran the TanStack Router
codegen, so the gitignored routeTree.gen.ts was missing in CI and every
typed route call failed to typecheck. Add a portal build step before
lint in the workflow so the route tree exists first.

Auth: link Better Auth users to the canonical system_users row by
shared id via a databaseHooks.user.create.before hook (pre-provisioned
admins reuse their existing id; SSO JIT users get a system_users row
created first as the FK target). create-user/service.ts now creates
system_users directly instead of going through the auth-schema user
table, and guards against duplicate emails. isSystemAdmin now lives on
system_users and is attached to the session via customSession.

Deps: mongodb's own bson subdependency was resolving to bson@7.3.1
despite the existing "bson": "7.2.0" resolutions pin, because bun only
applies `resolutions` to the top-level package, not nested/transitive
ones. bson@7.3.1 calls a node:v8 API Bun doesn't implement, which
crashed every mongo-adapter test on import. Added a matching entry
under `overrides` (which bun applies tree-wide) to force the nested
mongodb->bson resolution down to 7.2.0 as well.

Also bumps the migration journal for 0041_far_dreaming_celestial,
adds apps/portal and packages/components package.json to the docker
kit build context, and updates bun.lock.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…atabase-blocks

fix(ci): generate portal route tree before lint; canonical system_users auth linking
Migrate hot-reload pub/sub from Redis to NATS and split the monolith into
independently deployable images (admin control plane + scalable workers),
keeping the all-in-one kit for quick runs.

Worker & pub/sub
- Add dedicated NATS abstraction (db/nats.ts) with token auth + reconnect,
  and a pub/sub layer (db/pubsub.ts); redis.ts is now cache-only and
  re-exports the pub/sub API so existing call sites are unchanged.
- Gate the built-in worker in the admin server behind ENABLE_BUILTIN_WORKER;
  when false the admin process is control-plane only.
- Add readiness/startup health endpoints and a waitForSchema() gate so a
  freshly started worker waits for the admin server's migrations instead of
  crashing on "relation app_config does not exist".
- Graceful shutdown: SIGTERM/SIGINT handlers in standalone + worker, and a
  trap in the kit/admin entrypoints (fixes PID-1 signals hanging on stop).

Docker
- New thin worker image (single bundled JS + sourcemap, no node_modules).
- New admin image (server + web + ai-gateway) and a Traefik-fronted
  production compose with two load-balanced worker replicas.
- Kit image now also runs the worker; user API traffic routes to it.
- Fix Bun bundle OTLP crash via --conditions=module across all builds.

Docs & tooling
- Rewrite the self-hosting guide as a decision hub; add Kit and
  Production pages; KeyGenerator now emits NATS_TOKEN.
- Add k6 load-testing harness under testing/load-testing.
- CI: kit publishes a rolling nightly on main; kit/admin/worker publish
  versioned + latest images on v* tags.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…-extraction

feat(worker): extract request worker, migrate hot-reload to NATS, split Docker images
…ured-logging

feat: implement structured logging engine, browser bundle compatibility, and env consolidation
feat: migrated from process.env to getEnv() function with validation …
Move harness progress off BullMQ job.updateProgress onto NATS pub/sub.
BullMQ still triggers/retries runs; only the progress stream moved.

- notifications.ts: conversations.<userId> subject, zod discriminated-union
  message contract, publish/subscribe/bridge helpers
- callbacks/index: publish per-event and terminal states; resolve owner
  userId once per run; skip publish when conversation has no owner
- redisService: whole-run-state snapshot persists during run, TTL drops to
  60s on any terminal state via finalizeSnapshot
- worker: await initializePubSub on init; drop dead QueueEvents bridge
- main: start conversations.* bridge into in-process bus

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…rename

Client-facing edge for harness progress. socket.io runs on the Bun engine and
shares one Bun.serve router with Hono (no Hono adapter for socket.io), mounted
under the proxy prefix /_/admin/ai/socket.io/.

- socketGateway.ts: authenticates each connection against the better-auth
  session (handshake refused if unauthenticated), joins the per-user room
  conversations:<userId>, sends a full_state catch-up from Redis on connect,
  and fans NATS conversations.* events to rooms as typed update messages
  (discriminated on type; conversationId + stepId identify each update)
- main.ts: route /_/admin/ai/socket.io/* to the engine, else to Hono
- notifications.ts / queue.ts: drop the now-dead in-process EventEmitter bridge
  (socket.io subscribes to conversations.* directly)
- rename harness sub-agent node builder -> blockBuilder (AgentNode.BLOCK_BUILDER)
  to disambiguate from the router "builder" intent; enum-driven so the LLM
  task-generator contract and graph dispatch stay consistent
- docs: harness-client-implementation-guide.md (web client contract + React
  state edge cases) and notification-implementation.md updates

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…s-nats-progress

feat(harness): NATS progress stream + socket.io live updates
Vertical-slice endpoints for the new agent harness conversation model
(agent_harness_conversations/runs, replacing the legacy location/routeId
coupling on ai_chat_conversations):

- GET /harness-conversations - paginated list, needUserQuery=true adds a
  30-char truncated latest query per conversation
- PATCH /harness-conversations/:conversationId - rename
- DELETE /harness-conversations/:conversationId - delete
- POST /harness-conversations/message - send a message; omitting
  conversationId creates a new conversation + run and enqueues
  HARNESS_START_JOB (reuses enqueueHarnessStart); rejects with 409 if the
  target conversation already has a run in progress

Status shown to the user is resolved live from the Redis run snapshot
(harness:run:<runId>:snapshot) while a run is in flight, falling back to
the conversation's DB status once idle/terminal - the DB status only
flips on terminal events so it lags mid-run.

Perf hardening for high concurrency on constrained (1 vCPU/1GB) pods:
- list's status resolution is now batched (RedisService.getActiveRuns/
  getSnapshots via MGET) instead of up to 2 Redis round trips per row
- list cache invalidation on mutations uses an O(1) per-user version
  counter (cacheVersion.ts) instead of deleteCacheKeysByPattern, which
  runs Redis KEYS - an O(N) full-keyspace scan that blocks Redis's
  single thread
- the post-mutation status refresh query only runs on a cache hit, not
  after a fresh DB build that already has current statuses

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…s-conversation-api

feat: harness conversation api and basic agent UI
Faithfully reproduces the app/web integrations flow in apps/portal with the
new HeroUI/dark-lime look. Nothing about the API contract or the flow changes;
only the UI framework and styling.

Functional changes:
- 3-panel integrations page: left connector rail (Databases/KV/AI/BaaS/
  Observability) driving ?group=, center configured-integrations accordion,
  right filter panel (search + by-type + collapse).
- Connect flow: group -> variant cascade backed by the server helpers
  (getIntegrationsGroups/Variants/getDefaultVariantValue/getSchema/
  humanReadableConnectorNames), dispatching to the correct connector form.
- Connector forms: CredentialsUrlForm (Postgres/MySQL/Mongo/Redis/Memcached
  with Credentials|URL toggle + optional SSL/database), AiForm (OpenAI/
  Anthropic/Gemini/Mistral + OpenAI-Compatible baseUrl), ObservabilityForm
  (Loki/OTel Logs with Base64|Credentials toggle).
- AppConfigSelector: every config field accepts a literal value or a
  cfg:<key> reference, with a searchable app-config key picker.
- Create / edit (locked group+variant) / delete / test-connection wired to the
  same endpoints as web (POST, PUT, DELETE, list/{group}, test-connection),
  with getSchema validation before save.
- Deep-linkable via ?group= and ?open=.

Fixes a server-into-browser bundle bleed: the app-config DTO pulled
appConfigDataTypeEnum (a value) from db/schema, dragging @fluxify/lib -> node:vm
into the client and crashing the route. appConfig service DTO imports are now
type-only so the chain is elided. Removed verbatimModuleSyntax from the portal
tsconfig so server integration schemas can be imported for reuse.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Rework the project shell/sidebar to the mockup layout: gradient logo mark +
  FLUXIFY wordmark, project name, plain styled nav with lime active state,
  "Back to projects".
- Design tokens: layered dark surfaces, lime accent, 12px radius, JetBrains
  Mono, softer focus halo, themed scrollbars and card refinements.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@yashashkumar
yashashkumar changed the base branch from main to ui-migration July 26, 2026 19:02
@07prajwal2000
07prajwal2000 merged commit 69dd60c into Fluxify-rest:ui-migration Jul 26, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants