[Feature] Worker-Initiated Remote Connection (Pull Model / Multi-Host Support)

[ElioNeto]

Context

Inspired by Cranker's foundational architectural pattern — where services connect to the router, not the other way around — vyx could support a pull model for worker registration that enables multi-host deployments without Docker, Kubernetes, or a service mesh.

This is a longer-horizon feature that would fundamentally expand vyx's deployment model from "single-machine process orchestrator" to "lightweight distributed API gateway".

Current Model (Push / Local only)

[vyx Core] ── os/exec ──► [Worker process]  (Core spawns worker on same machine)

Limitations:

• All workers must run on the same machine as the Core
• Scaling requires external tools (Docker Compose, K8s)
• Remote workers (e.g., a GPU Python worker on a different host) cannot participate

Proposed Model (Pull / Remote-capable)

[vyx Core] ◄── WebSocket/gRPC ── [Worker SDK]  (Worker dials Core, anywhere on the network)

The worker SDK calls vyx.connect("wss://core-host:9000") and registers itself. The Core learns the worker exists purely from the incoming connection — zero central configuration needed for remote workers.

How it would work

1. Core opens a registration endpoint (e.g., wss://core:9000/_vyx/register).
2. Worker SDK (Node, Python, Go) initiates a persistent WebSocket connection to that endpoint.
3. During the WebSocket handshake, the worker sends its worker_id, routes, and protocol version.
4. Core registers the worker in the MemoryWorkerRepository and starts dispatching requests to it over the same WebSocket connection (bidirectional framing).
5. If the connection drops, the Core marks the worker as unavailable and stops routing to it. The worker SDK auto-reconnects.

Example (Node.js SDK)

import { VyxWorker } from '@vyx/worker';

const worker = new VyxWorker({ connect: 'wss://api-gateway.internal:9000' });

worker.get('/api/products', async (req, res) => {
    res.json({ products: [] });
});

worker.start();

vyx.yaml (optional static declaration)

# Workers can still be declared statically OR register dynamically.
workers:
  - id: node:api
    mode: remote        # 'local' (default, os/exec) | 'remote' (pull model)
    allow_remote: true  # Core accepts self-registration from this worker_id

Relationship to Current Architecture

This feature does not replace the current local os/exec model. Both modes would coexist:

Mode	Use Case
local (current)	Single-machine dev and prod, tight lifecycle control
remote (this issue)	Distributed deployments, GPU workers, edge functions, polyglot services on separate hosts

Acceptance Criteria (MVP)

• Core exposes a WebSocket registration endpoint (/_vyx/register)
• Worker SDK (at least Node.js) can connect to Core via WebSocket instead of UDS
• Core dispatches requests to remote workers over the WebSocket connection
• Worker disconnection is detected and the worker is removed from the route table
• Worker SDK auto-reconnects with exponential backoff
• vyx.yaml supports mode: remote per worker entry
• Security: registration endpoint supports mutual TLS or a shared secret token
• Documentation updated with multi-host deployment guide

Open Questions

• Should the registration endpoint be on the same port as the HTTP gateway or a separate port?
• Should remote workers support the same route annotation system as local workers?
• How does the route map interact with dynamically registered workers (rebuild on connect/disconnect)?

References

• Inspired by: hsbc/cranker-connector — the entire Cranker architecture is built on this pattern
• hsbc/mu-cranker-router — the router side implementation
• core/infrastructure/ipc/uds/ — current local-only transport
• core/domain/ipc/transport.go — Transport interface that would need a WebSocket implementation
• Related: [Feature] UDS Connection Warm Pool (Pre-established Connection Sliding Window) #58 (Connection Pool — warm pool applies to remote connections too)