Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .devcontainer/galactic/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,7 @@ The devcontainer includes the following extensions:

### Forwarded Ports
- **8080** - Metrics endpoint
- **8081** - Health check endpoint
- **9443** - Webhook server
- **5000** - gRPC health endpoint (liveness/readiness probes)

## Capabilities

Expand Down
10 changes: 3 additions & 7 deletions .devcontainer/galactic/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -107,18 +107,14 @@
}
}
},
"forwardPorts": [8080, 8081, 9443],
"forwardPorts": [8080, 5000],
"portsAttributes": {
"8080": {
"label": "Metrics",
"onAutoForward": "silent"
},
"8081": {
"label": "Health",
"onAutoForward": "silent"
},
"9443": {
"label": "Webhook",
"5000": {
"label": "gRPC Health",
"onAutoForward": "silent"
}
},
Expand Down
49 changes: 26 additions & 23 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,58 +2,61 @@

## Architecture Reference

See [ARCHITECTURE.md](ARCHITECTURE.md) for a full architecture reference including module layout, entry points, data flow, configuration, and known constraints.
See [ARCHITECTURE.md](ARCHITECTURE.md) for a full architecture reference including module layout, entry points, data flow, configuration, external dependencies, and known constraints.

## Purpose & Architecture
## Purpose

Galactic is the SRv6 data plane for multi-cloud VPC networking. It consists of a DaemonSet agent (`internal/agent/`) that manages kernel SRv6 routes and VRFs per node, and a CNI plugin (`internal/cni/`) that wires containers into VPC networks. VPC and VPCAttachment CRD management lives in a separate operator project; Galactic receives pre-populated identifiers through the CNI config and acts on them. BGP is used as the control plane for distributing SRv6 routes between agents.
Galactic is the SRv6 data plane for multi-cloud VPC networking. It consists of two binaries deployed on each Kubernetes node:

**Data flow:** CNI invoked with pre-populated VPC/VPCAttachment identifiers → gRPC registers endpoint with agent → agent manages SRv6 ingress routes locally → BGP distributes SRv6 routes between agents.
- **`galactic-cni`** — CNI plugin that wires containers into VPC networks (VRF, veth, SRv6 ingress route) and writes `BGPAdvertisement` CRDs.
- **`galactic-router`** — controller-runtime reconciler that watches Cosmos BGP CRDs and drives an embedded GoBGP server per node to distribute EVPN paths.

**Non-obvious decisions:**
- VPC identifiers are 48-bit hex; VPCAttachment identifiers are 16-bit hex. These are embedded into IPv6 SRv6 endpoint addresses for deterministic route lookups. Both are supplied by an external operator via the CNI config.
- Identifiers are also Base62-encoded for interface naming (VRF: `vrfX-Y`, veth host side: `galX-Y`) to keep kernel interface name length within limits.
- `galactic-cni` is a pure CNI plugin; `main()` calls `cni.RunPlugin()` directly with no CLI layer. `galactic-agent` uses flag parsing for its configuration flags.
- The Kubernetes operator, VPC/VPCAttachment CRDs, and webhook code have been removed from this repository. They live in a separate companion operator project.
VPC and VPCAttachment CRD management lives in a separate companion operator; Galactic receives pre-populated identifiers through the CNI config and acts on them.

**Data flow:** CNI invoked with pre-populated VPC/VPCAttachment identifiers → CNI creates kernel SRv6 state and writes a `BGPAdvertisement` CRD → `galactic-router` reconciles the CRD → GoBGP advertises the EVPN path → BGP distributes routes between nodes.

## Tech Stack

- **Go 1.26** — agent and CNI plugin
- **Multus CNI** — multi-network for pods; NAD generation is handled by the external operator
- **gRPC + protobuf** — CNI-to-agent local communication
- **Go 1.26** — router and CNI plugin
- **controller-runtime** — BGPRouter/BGPPeer/BGPAdvertisement/BGPPolicy reconcilers
- **Cosmos BGP API** (`go.miloapis.com/cosmos`) — BGPRouter, BGPPeer, BGPAdvertisement, BGPPolicy CRDs
- **GoBGP v4** — embedded BGP server (tenant role)
- **SRv6 + netlink** — kernel-level routing; `github.com/vishvananda/netlink`
- **BGP** — control plane for SRv6 route distribution between agents (in progress)
- **Multus CNI** — multi-network for pods; NAD generation handled by the external operator

## Development Workflow

```
task build # produces bin/galactic
task build # produces bin/galactic-cni and bin/galactic-router
task ci # full pipeline: lint → build → test:unit → test:e2e
task test # runs test:unit then test:e2e
task test:unit # unit tests with race detection
task test:e2e # Kind cluster lifecycle test
task lint # golangci-lint; lint-fix applies safe auto-fixes
task run-agent # run agent (requires root / CAP_NET_ADMIN)
task docker-build # build container image (IMG= to override tag)
```

**Before every PR:** `task lint test`.
**Before every PR:** `task ci` (lint → build → test:unit → test:e2e).

## Code Standards

See [CONVENTIONS.md](CONVENTIONS.md) for the full, prescriptive coding standards covering Go naming, error handling, testing patterns, linting, and commit messages.
See [CONVENTIONS.md](CONVENTIONS.md) for the full, prescriptive coding standards covering Go naming, error handling, testing patterns, linting, commit messages, and markdown table alignment.

Summary:
- Go: `gofmt`/`goimports` enforced; golangci-lint with `errcheck`, `staticcheck`, `govet`, `revive`, `gocyclo`, `dupl`, `unused` (see `.golangci.yml`). `lll` excluded from `internal/`.
- Generated protobuf files (`*.pb.go`, `*_grpc.pb.go`) are committed; never hand-edit them.
- Always use `.yaml`, never `.yml`, for YAML files.

## Deployments

- **`deploy/galactic-agent/`** — Kustomize manifests for the agent DaemonSet, RBAC, and ServiceAccount. Apply with `kubectl apply -k deploy/galactic-agent/`.
- **`deploy/containerlab/`** — ContainerLab topology (`gvpc.clab.yaml`) for three Kind clusters (iad, sjc, infra) wired over an IPv6 SRv6 transit mesh. FRR runs as a hostNetwork DaemonSet on each worker for eBGP underlay; GoBGP handles L3VPN type-5 routes over iBGP to the infra route reflector. See `deploy/containerlab/README.md` and `deploy/containerlab/Taskfile.yaml` for bring-up commands.
- **`deploy/galactic-router/`** — Production manifests for the router DaemonSet, RBAC, and ServiceAccount. Apply with `kubectl apply -f deploy/galactic-router/`.
- **`deploy/containerlab/`** — ContainerLab topology (`gvpc.clab.yaml`) for three Kind clusters (dfw, iad, sjc) wired over an IPv6 SRv6 transit mesh. FRR runs as a hostNetwork DaemonSet on each worker for eBGP underlay; `galactic-router` (tenant role) handles EVPN path distribution over iBGP. See `deploy/containerlab/README.md` and `deploy/containerlab/Taskfile.yaml` for bring-up commands.

## New Developer Entry Points

1. Run `task build` to verify toolchain; run `task test` to confirm unit tests pass.
2. Read `internal/cni/cni.go` (cmdAdd/cmdDel) to understand the container attach path.
3. Read `internal/plumbing/intf/intf.go` to understand SRv6 endpoint encoding, interface naming, and base62↔hex conversion.
4. Read `internal/plumbing/srv6/srv6.go` to understand kernel SRv6 ingress route management.
5. Explore `internal/plumbing/` for shared kernel and network primitives (VRF, sysctl, interface naming, SRv6).
2. Read `internal/cni/cni.go` (cmdAdd/cmdDel) to understand the container attach path and how `BGPAdvertisement` CRDs are created.
3. Read `internal/controller/` for the controller-runtime reconcilers (BGPRouter, BGPPeer, BGPAdvertisement, BGPPolicy, Node, Secret). Read `internal/reconcile/reconcile.go` to understand how the BGPRouter CRD is translated into a `DesiredRouter` applied to the runtime.
4. Read `internal/runtime/gobgp/runtime.go` to understand how `DesiredRouter` is applied to GoBGP.
5. Read `internal/plumbing/intf/intf.go` to understand SRv6 endpoint encoding, interface naming, and base62↔hex conversion.
6. Explore `internal/plumbing/` for shared kernel and network primitives (VRF, sysctl, interface naming, SRv6).
Loading
Loading