Include the ability to run specific modules independently

[jeroenrinzema]

Problem Statement

Lunogram currently runs as a single monolithic process where all components start together: the HTTP controllers (management + client API), all NATS JetStream consumers, WASM-based action/provider execution, the cluster leader election, and the scheduler. There is no way to selectively enable or disable individual components.

This is a problem for production deployments where different components have different operational requirements. For example:

• WASM execution (action execution via actions.execute.> / actions.validate.>, campaign sending via campaigns-send) runs third-party logic through Extism — webhooks, provider integrations (Twilio, Resend). These should be isolatable into a restricted namespace with stricter network access policies
• NATS consumers (user events, org events, journey advancement, list recomputation, schema handlers) process internal platform logic and may need to scale independently from the API
• HTTP controllers should be able to run without consuming from NATS streams, serving only API traffic behind a load balancer
• Scheduler (leader election + delayed journey step reconciliation) is leader-only work that doesn't need HTTP or consumer overhead

Running everything in a single process means a noisy neighbour problem — a spike in WASM execution or journey processing can degrade API response times, and there's no way to apply different security policies to the WASM execution layer.

Proposed Solution

Add a configuration option that controls which components a node starts. By default, all components run (preserving current behaviour), but operators can selectively enable only the components they need.

The components to control are:

Module	Description	Key resources
http	HTTP API server	Management + client controllers, console UI, API docs
consumers	NATS JetStream consumers for internal platform logic	users-process, users-schema, users-events-process, users-events-schema, lists-recompute, journeys-advance, organizations-process, organizations-schema, organizations-users-process, organizations-users-schema, organizations-events-process, organizations-events-schema, actions-schema
wasm	WASM-based execution for actions and campaigns	actions.execute.> and actions.validate.> NATS core subscriptions, campaigns-send JetStream consumer, provider + action WASM registries
scheduler	Cluster leader election + delayed journey step scheduler	Redis-based consensus, periodic journey state reconciliation

When a component is disabled, its resources (listeners, goroutines, connections) are not initialised at all — not just idle.

Technical Implementation

• Add a MODULES environment variable (comma-separated list, e.g. MODULES=http,consumers). When empty or unset, all components run (default behaviour)
• Add a Modules []string field to config.Node with parsing logic and validation against known module names
• Refactor cmd/lunogram/main.go run() to conditionally initialise components based on enabled modules:
  • Guard HTTP server startup with http module check
  • Guard NATS JetStream consumer bootstrap + serve (excluding campaigns-send and action subscriptions) with consumers module check
  • Guard WASM registry initialisation, campaigns-send consumer, and actions.execute.> / actions.validate.> subscriptions with wasm module check
  • Guard cluster consensus + scheduler with scheduler module check
• Split consumer.Serve() so that WASM-dependent consumers (campaigns-send, action execute/validate) can be started independently from the internal platform consumers
• WASM registries (provider + action) should only be initialised when the wasm module is enabled
• Shared dependencies (database connections, NATS connection) should only be initialised when at least one component that needs them is enabled
• Ensure graceful shutdown works correctly when only a subset of components is running

Additional Context

The existing NATS namespace (NATS_NAMESPACE) and Redis key prefix (REDIS_KEY_PREFIX) configuration already provide stream/subject isolation. This proposal complements that by controlling which components actually start within a process, enabling deployment topologies like:

┌─────────────────────┐   ┌──────────────────────┐   ┌─────────────────────┐
│  Namespace: default  │   │ Namespace: restricted │   │  Namespace: default  │
│  MODULES=http        │   │ MODULES=wasm          │   │  MODULES=consumers,  │
│                      │   │                       │   │          scheduler   │
│  - Management API    │   │  - Action execution   │   │                      │
│  - Client API        │   │  - Action validation  │   │  - Journey consumers │
│  - Console UI        │   │  - Campaign sending   │   │  - User/org events   │
│                      │   │  - Provider WASM      │   │  - List recomputation│
│                      │   │                       │   │  - Schema handlers   │
│  (no consumers,      │   │  (stricter network    │   │  - Leader election   │
│   no WASM, no sched) │   │   policies applied)   │   │  - Journey scheduler │
└─────────────────────┘   └──────────────────────┘   └─────────────────────┘

Acceptance Criteria

• By default (no MODULES set), all components start — no change in behaviour for existing deployments
• Setting MODULES=http starts only the HTTP server, no NATS consumers, WASM execution, or scheduler
• Setting MODULES=consumers starts only the internal NATS JetStream consumers (excluding WASM-dependent ones), no HTTP server, WASM execution, or scheduler
• Setting MODULES=wasm starts only the WASM registries, campaigns-send consumer, and action execute/validate subscriptions — no HTTP server, internal consumers, or scheduler
• Setting MODULES=scheduler starts only the cluster leader election and delayed journey step scheduler
• Multiple modules can be combined: MODULES=http,consumers starts both HTTP and internal consumers
• Components that are not enabled do not allocate resources (no listeners, no goroutines, no unnecessary connections)
• WASM registries are only loaded when the wasm module is enabled
• Graceful shutdown works correctly for any combination of enabled modules
• Invalid module names in MODULES cause a startup error with a clear message listing valid options