Skip to content

feat: add hub-managed port pool for agent containers#298

Open
zeroasterisk wants to merge 1 commit into
GoogleCloudPlatform:mainfrom
zeroasterisk:feature/port-assignment
Open

feat: add hub-managed port pool for agent containers#298
zeroasterisk wants to merge 1 commit into
GoogleCloudPlatform:mainfrom
zeroasterisk:feature/port-assignment

Conversation

@zeroasterisk
Copy link
Copy Markdown
Contributor

@zeroasterisk zeroasterisk commented Jun 3, 2026

Why

Agents frequently need to run local web servers — dev servers, preview apps, browser-based verification — but have no guaranteed way to get unique, non-colliding host ports. With multiple agents running concurrently on the same broker, even if you exported ports, they would collide. Agents also have no way to tell users a URL where they can access a locally running service.

What

A broker-managed port pool that allocates unique host ports to each agent at container creation time, exposed as environment variables:

  • AVAILABLE_LOCALHOST_PORT_A, AVAILABLE_LOCALHOST_PORT_B — raw port numbers
  • AVAILABLE_LOCALHOST_URL_A, AVAILABLE_LOCALHOST_URL_B — full URLs (when host_url is configured)

Ports are published via docker -p so they're reachable from the host. Allocation happens on scion start, release on scion stop / scion delete / start failure.

Configuration

server:
  broker:
    port_pool:
      enabled: true            # default: true when section is present
      range: "8000-9000"       # default
      ports_per_agent: 2       # default
      host_url: "http://35.232.118.211"  # optional, enables URL env vars

Agent usage

# Start a dev server on the assigned port
npm run dev -- --port $AVAILABLE_LOCALHOST_PORT_A

# Tell the user where to find it
echo "Visit $AVAILABLE_LOCALHOST_URL_A"

How

New files

  • pkg/runtime/portpool.go — Thread-safe port allocator (sync.Mutex, lowest-available-port selection, allocate/release/query)
  • pkg/runtime/portpool_test.go — Table-driven tests covering allocation, multi-agent, exhaustion, release/reuse, concurrent access (50 goroutines)

Modified files

  • pkg/runtime/interface.go — AllocatedPorts and PortHostURL fields on RunConfig
  • pkg/runtime/common.go — Env var injection + -p port publishing in buildCommonRunArgs() (covers Docker, Podman, Apple Container)
  • pkg/runtime/k8s_runtime.go — Env var injection in buildPod() for Kubernetes
  • pkg/api/types.go — PortPoolConfig struct + ParsePortRange() with validation (1-65535)
  • pkg/config/settings_v1.go — V1PortPoolConfig on V1BrokerConfig for settings.yaml
  • pkg/agent/run.go — Port allocation before RunConfig construction, release on start failure
  • pkg/agent/manager.go — Port release in Stop() and Delete() paths, PortPool field
  • cmd/server_foreground.go — Pool initialization from broker settings at startup

Design decisions

  • Opt-in: Pool is nil unless port_pool section is present in settings
  • Broker-scoped: Each broker manages its own pool independently
  • In-memory: No persistence — ports reclaimed on broker restart
  • All runtimes: Docker/Podman/Apple via buildCommonRunArgs, K8s via buildPod

Test plan

  • go build ./... — clean
  • go vet ./pkg/... ./cmd/... — clean
  • go test ./pkg/runtime/... — all pass including new portpool tests
  • go test ./pkg/agent/... ./pkg/api/... — all pass
  • Concurrency test: 50 goroutines allocating simultaneously, no duplicates
  • No double env injection
  • Port release on all exit paths (stop, delete, start failure)

Agents that run local web servers (dev servers, preview apps, test
harnesses) need guaranteed unique host ports. This adds a port pool
managed by the runtime broker that allocates unique ports from a
configurable range at agent startup and releases them on stop/delete.

Each agent receives environment variables:
- AVAILABLE_LOCALHOST_PORT_A, _B (port numbers)
- AVAILABLE_LOCALHOST_URL_A, _B (full URLs, when host_url is configured)

Ports are published via docker -p so they're reachable from the host.

Configuration in settings.yaml:
  server:
    broker:
      port_pool:
        range: "8000-9000"
        ports_per_agent: 2
        host_url: "http://broker-host-ip"

Defaults: range 8000-9000, 2 ports per agent, disabled unless configured.
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a hub-managed port pool mechanism to allocate unique host ports for agent containers. It adds configuration options, implements the PortPool logic with concurrency support and tests, and integrates port allocation and release into the agent manager's lifecycle (start, stop, delete). It also injects these allocated ports as environment variables and publishes them in both Docker and Kubernetes runtimes. Feedback on the PR highlights two issues: first, recreating or restarting an agent can bypass the manager's Delete method, leading to leaked ports and eventual pool exhaustion, so releasing existing ports before allocation is suggested; second, if the host URL is configured with a trailing slash, the constructed agent port URLs will be invalid, so trimming trailing slashes during initialization is recommended.

Comment thread pkg/agent/run.go
Comment on lines +821 to +827
if m.PortPool != nil {
ports, err := m.PortPool.Allocate(opts.Name, m.PortPool.PerAgent())
if err != nil {
return nil, fmt.Errorf("port allocation failed: %w", err)
}
allocatedPorts = ports
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

When an agent container is recreated or restarted, the existing container is deleted via m.Runtime.Delete directly, which bypasses the manager's Delete method and does not release the allocated ports back to the pool. Consequently, calling Allocate again for the same agent name will leak the previously allocated ports, eventually leading to port pool exhaustion.

To prevent this resource leak, release any existing ports for the agent name from the pool before allocating new ones.

Suggested change
if m.PortPool != nil {
ports, err := m.PortPool.Allocate(opts.Name, m.PortPool.PerAgent())
if err != nil {
return nil, fmt.Errorf("port allocation failed: %w", err)
}
allocatedPorts = ports
}
if m.PortPool != nil {
m.PortPool.Release(opts.Name)
ports, err := m.PortPool.Allocate(opts.Name, m.PortPool.PerAgent())
if err != nil {
return nil, fmt.Errorf("port allocation failed: %w", err)
}
allocatedPorts = ports
}

Comment thread cmd/server_foreground.go
Comment on lines +1091 to +1094
} else {
mgr.PortPool = runtime.NewPortPool(pMin, pMax, perAgent, pp.HostURL)
log.Printf("Port pool initialized: range %d-%d, %d ports per agent, host_url=%q", pMin, pMax, perAgent, pp.HostURL)
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If pp.HostURL is configured with a trailing slash (e.g., http://35.232.118.211/), the constructed agent port URLs will contain an invalid double-slash/colon sequence (e.g., http://35.232.118.211/:8000).

Trimming any trailing slashes from the host URL during initialization ensures that the constructed URLs are always valid.

			} else {
				hostURL := strings.TrimRight(pp.HostURL, "/")
				mgr.PortPool = runtime.NewPortPool(pMin, pMax, perAgent, hostURL)
				log.Printf("Port pool initialized: range %d-%d, %d ports per agent, host_url=%q", pMin, pMax, perAgent, hostURL)
			}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant