diff --git a/AGENTS.md b/AGENTS.md index 494bad0..b85e45f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -6,7 +6,7 @@ See [ARCHITECTURE.md](ARCHITECTURE.md) for a full architecture reference includi ## Purpose & Architecture -Galactic is the SRv6 data plane for multi-cloud VPC networking. It consists of a DaemonSet agent (`internal/agent/srv6/`) that manages kernel SRv6 routes and VRFs per node, and a CNI plugin (`internal/cni/`) that registers container endpoints with the agent via gRPC. VPC and VPCAttachment CRD management lives in a separate operator project; Galactic receives pre-populated identifiers through the CNI config and acts on them. BGP is used as the control plane for distributing SRv6 routes between agents. +Galactic is the SRv6 data plane for multi-cloud VPC networking. It consists of a DaemonSet agent (`internal/agent/`) that manages kernel SRv6 routes and VRFs per node, and a CNI plugin (`internal/cni/`) that wires containers into VPC networks. VPC and VPCAttachment CRD management lives in a separate operator project; Galactic receives pre-populated identifiers through the CNI config and acts on them. BGP is used as the control plane for distributing SRv6 routes between agents. **Data flow:** CNI invoked with pre-populated VPC/VPCAttachment identifiers → gRPC registers endpoint with agent → agent manages SRv6 ingress routes locally → BGP distributes SRv6 routes between agents. @@ -20,7 +20,7 @@ Galactic is the SRv6 data plane for multi-cloud VPC networking. It consists of a - **Go 1.26** — agent and CNI plugin - **Multus CNI** — multi-network for pods; NAD generation is handled by the external operator -- **gRPC + protobuf** — CNI-to-agent local communication (`pkg/proto/local/`) +- **gRPC + protobuf** — CNI-to-agent local communication - **SRv6 + netlink** — kernel-level routing; `github.com/vishvananda/netlink` - **BGP** — control plane for SRv6 route distribution between agents (in progress) @@ -54,6 +54,6 @@ Summary: 1. Run `task build` to verify toolchain; run `task test` to confirm unit tests pass. 2. Read `internal/cni/cni.go` (cmdAdd/cmdDel) to understand the container attach path. -3. Read `internal/agent/srv6/srv6.go` to understand the agent entry point and how it manages SRv6 routes and VRFs. -4. Read `pkg/proto/local/local.go` to understand the gRPC interface between the CNI and the agent. -5. Explore `pkg/common/` for shared utilities (VRF management, sysctl helpers, CNI types). +3. Read `internal/plumbing/intf/intf.go` to understand SRv6 endpoint encoding, interface naming, and base62↔hex conversion. +4. Read `internal/plumbing/srv6/srv6.go` to understand kernel SRv6 ingress route management. +5. Explore `internal/plumbing/` for shared kernel and network primitives (VRF, sysctl, interface naming, SRv6). diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 898c84e..8ec1e88 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -44,19 +44,17 @@ galactic/ │ └── galactic-agent/ # Agent binary ├── internal/ │ ├── agent/ # Agent run loop; wires GoBGP, health, metrics, bootstrap -│ │ └── srv6/ # Kernel SRv6 ingress route add/del (END.DT46) │ ├── bootstrap/ # BGPProvider CR lifecycle (create on start, delete on stop) │ ├── cni/ # CNI cmdAdd / cmdDel -│ │ ├── bgp/ # L3VPN path injection into local GoBGP │ │ ├── route/ # Host-side static routes via netlink │ │ └── veth/ # veth pair management │ ├── gobgp/ # Embedded GoBGP server lifecycle -│ └── metrics/ # Prometheus metrics (galactic_agent_*) -├── pkg/common/ -│ ├── cni/ # Shared CNI config types -│ ├── sysctl/ # Interface sysctl helpers -│ ├── util/ # SRv6 encoding, interface naming, base62↔hex -│ └── vrf/ # Linux VRF create/delete/lookup +│ ├── metrics/ # Prometheus metrics (galactic_agent_*) +│ └── plumbing/ # Low-level kernel and network primitives +│ ├── intf/ # Interface naming, base62↔hex encoding, SRv6 endpoint encode/decode +│ ├── srv6/ # SRv6 ingress route add/del (END.DT46) +│ ├── sysctl/ # Interface sysctl helpers +│ └── vrf/ # Linux VRF create/delete/lookup ├── deploy/ │ ├── galactic-agent/ # Kustomize: DaemonSet, RBAC, ServiceAccount │ └── containerlab/ # ContainerLab lab topology and scripts @@ -78,15 +76,15 @@ See [docs/agent-startup.md](docs/agent-startup.md) for the agent startup sequenc | Component | Binary | Role | |-----------|--------|------| -| `internal/agent` | `galactic-agent` | Run loop; wires all agent subsystems | -| `internal/agent/srv6` | `galactic-agent` | Kernel SRv6 route add/del | +| `internal/agent` | `galactic-agent` | Run loop; wires GoBGP, health, metrics, bootstrap | | `internal/bootstrap` | `galactic-agent` | BGPProvider CR lifecycle | | `internal/gobgp` | `galactic-agent` | Embedded GoBGP server | | `internal/metrics` | `galactic-agent` | Prometheus metrics | | `internal/cni` | `galactic-cni` | CNI cmdAdd / cmdDel | -| `internal/cni/bgp` | `galactic-cni` | L3VPN path injection into GoBGP | -| `pkg/common/vrf` | both | Linux VRF management | -| `pkg/common/util` | both | SRv6 encoding, interface naming | +| `internal/plumbing/intf` | both | Interface naming, base62↔hex encoding, SRv6 endpoint encode/decode | +| `internal/plumbing/srv6` | both | SRv6 ingress route add/del (END.DT46) | +| `internal/plumbing/vrf` | both | Linux VRF create/delete/lookup | +| `internal/plumbing/sysctl` | both | Interface sysctl helpers | --- @@ -102,4 +100,4 @@ See [docs/agent-startup.md](docs/agent-startup.md) for the agent startup sequenc ## Known Constraints - **GoBGP RIB is ephemeral.** All BGP state is in-process memory. On restart, sessions and paths must be re-established. The cosmos operator is responsible for re-applying config. -- **No kernel-path unit tests.** `internal/cni`, `internal/agent/srv6`, and `pkg/common/vrf` require `CAP_NET_ADMIN` and a real kernel. Coverage comes from the e2e suite (`task ci:e2etest`), which only runs on `main` and release tags. +- **No kernel-path unit tests.** `internal/cni`, `internal/plumbing/srv6`, and `internal/plumbing/vrf` require `CAP_NET_ADMIN` and a real kernel. `internal/plumbing/intf` is fully unit-testable (pure functions only). Coverage comes from the e2e suite (`task ci:e2etest`), which only runs on `main` and release tags. diff --git a/CONVENTIONS.md b/CONVENTIONS.md index 4d54b10..6079367 100644 --- a/CONVENTIONS.md +++ b/CONVENTIONS.md @@ -11,9 +11,8 @@ This document defines the coding standards, naming rules, error handling pattern - Module: `go.datum.net/galactic` - `cmd/galactic-cni/main.go` — CNI plugin entry point; calls `cni.RunPlugin()` directly - `cmd/galactic-agent/main.go` — agent entry point; parses flags and calls `agent.Run()` -- `pkg/common/` — utilities shared between agent and CNI -- `pkg/proto/local/` — gRPC / protobuf generated files plus hand-written convenience wrapper for CNI-to-agent communication -- `internal/agent/` — agent entry point and gRPC server; `srv6/` subdirectory owns kernel SRv6 route and VRF management +- `internal/plumbing/` — low-level kernel and network primitives shared between agent and CNI (`intf`, `srv6`, `sysctl`, `vrf`) +- `internal/agent/` — agent entry point and gRPC server - `internal/cni/` — CNI plugin (cmdAdd / cmdDel implementation) - `internal/cmd/version/` — ldflags variables (Version, GitCommit, etc.) set at build time - `internal/gobgp/` — embedded GoBGP server lifecycle @@ -39,7 +38,7 @@ import ( "google.golang.org/grpc" - "go.datum.net/galactic/pkg/proto/local" + "go.datum.net/galactic/internal/plumbing/intf" ) ``` @@ -87,7 +86,7 @@ const MaxVPCAttachment uint64 = 0xFFFF ### Code generation -Generated protobuf files (`*.pb.go`, `*_grpc.pb.go` in `pkg/proto/local/`) must never be hand-edited. Regenerate them using the `protoc` toolchain when `.proto` files change. Generated files are committed to version control. +Generated protobuf files (`*.pb.go`, `*_grpc.pb.go`) must never be hand-edited. Regenerate them using the `protoc` toolchain when `.proto` files change. Generated files are committed to version control. ### Linting @@ -135,15 +134,14 @@ for _, tt := range tests { ### What not to test - Do not write tests for generated code (`*.pb.go`, `*_grpc.pb.go`). -- Agent and CNI kernel-path code (`internal/agent/srv6/`, `internal/cni/`) currently has no unit coverage; new code in those paths should prefer integration/e2e over fragile mock-heavy unit tests. +- Agent and CNI kernel-path code (`internal/plumbing/srv6/`, `internal/cni/`) currently has no unit coverage; new code in those paths should prefer integration/e2e over fragile mock-heavy unit tests. --- ## Protobuf / gRPC -- `.proto` files live in `pkg/proto/local/` (CNI-to-agent local gRPC). - Generated `*.pb.go` / `*_grpc.pb.go` files must never be hand-edited. -- Each proto package has a hand-written convenience wrapper (`local.go`) that exposes a cleaner Go API over the generated types. Add helpers there rather than importing generated types directly in application code. +- Each proto package has a hand-written convenience wrapper that exposes a cleaner Go API over the generated types. Add helpers there rather than importing generated types directly in application code. --- diff --git a/Taskfile.yaml b/Taskfile.yaml index 20c1ba2..ce1d476 100644 --- a/Taskfile.yaml +++ b/Taskfile.yaml @@ -77,11 +77,26 @@ tasks: ## build: - desc: Build binary + desc: Build binaries deps: [fmt, vet] + vars: + VERSION: + sh: git describe --tags --always --dirty 2>/dev/null || echo "dev" + GIT_COMMIT: + sh: git rev-parse --short HEAD + GIT_TREE_STATE: + sh: git diff --quiet && echo "clean" || echo "dirty" + BUILD_DATE: + sh: date -u +%Y-%m-%dT%H:%M:%SZ + LDFLAGS: >- + -s -w + -X go.datum.net/galactic/internal/metadata.Version={{.VERSION}} + -X go.datum.net/galactic/internal/metadata.GitCommit={{.GIT_COMMIT}} + -X go.datum.net/galactic/internal/metadata.GitTreeState={{.GIT_TREE_STATE}} + -X go.datum.net/galactic/internal/metadata.BuildDate={{.BUILD_DATE}} cmds: - - go build -o bin/galactic-cni cmd/galactic-cni/main.go - - file bin/galactic-cni + - go build -ldflags "{{.LDFLAGS}}" -o bin/galactic-cni cmd/galactic-cni/main.go + - go build -ldflags "{{.LDFLAGS}}" -o bin/galactic-agent cmd/galactic-agent/main.go docker-build: desc: Build container image @@ -156,8 +171,8 @@ tasks: run: once cmds: - | - if find . -name "*.yml" -not -path "./.git/*" | grep -q .; then - find . -name "*.yml" -not -path "./.git/*" + if find . -name "*.yml" -not -path "./.git/*" -not -path "./deploy/containerlab/clab-gvpc/*" | grep -q .; then + find . -name "*.yml" -not -path "./.git/*" -not -path "./deploy/containerlab/clab-gvpc/*" echo "ERROR: .yml files found; rename to .yaml" exit 1 fi diff --git a/cmd/galactic-agent/main.go b/cmd/galactic-agent/main.go index e0c4d16..bbb479a 100644 --- a/cmd/galactic-agent/main.go +++ b/cmd/galactic-agent/main.go @@ -1,4 +1,7 @@ -// Command galactic-agent is the node-local execution agent for Galactic. +// Copyright 2025 Datum Cloud, Inc. +// +// SPDX-License-Identifier: AGPL-3.0-or-later + package main import ( @@ -23,7 +26,7 @@ func newRootCommand() *cobra.Command { cmd := &cobra.Command{ Use: "galactic-agent", - Short: "Node-local execution agent for Galactic VPC networking", + Short: "BGP Provider implementation for Cosmos", SilenceUsage: true, RunE: func(cmd *cobra.Command, args []string) error { return agent.Run(cmd.Context(), *opts) diff --git a/containers/galactic/Dockerfile b/containers/galactic/Dockerfile index 2a75a5b..80e5a4c 100644 --- a/containers/galactic/Dockerfile +++ b/containers/galactic/Dockerfile @@ -17,7 +17,6 @@ RUN go mod download # Copy the go source COPY cmd/ cmd/ -COPY pkg/ pkg/ COPY internal/ internal/ # Build @@ -27,10 +26,10 @@ COPY internal/ internal/ # by leaving it empty we can ensure that the container and binary shipped on it will have the same platform. RUN CGO_ENABLED=0 GOOS=${TARGETOS:-linux} GOARCH=${TARGETARCH} go build \ -ldflags "-s -w \ - -X go.datum.net/galactic/internal/cmd/version.Version=${VERSION} \ - -X go.datum.net/galactic/internal/cmd/version.GitCommit=${GIT_COMMIT} \ - -X go.datum.net/galactic/internal/cmd/version.GitTreeState=${GIT_TREE_STATE} \ - -X go.datum.net/galactic/internal/cmd/version.BuildDate=${BUILD_DATE}" \ + -X go.datum.net/galactic/internal/metadata.Version=${VERSION} \ + -X go.datum.net/galactic/internal/metadata.GitCommit=${GIT_COMMIT} \ + -X go.datum.net/galactic/internal/metadata.GitTreeState=${GIT_TREE_STATE} \ + -X go.datum.net/galactic/internal/metadata.BuildDate=${BUILD_DATE}" \ -o galactic-cni cmd/galactic-cni/main.go # Use distroless as minimal base image to package the binary diff --git a/internal/agent/srv6/routeingress/routeingress.go b/internal/agent/srv6/routeingress/routeingress.go deleted file mode 100644 index f10869f..0000000 --- a/internal/agent/srv6/routeingress/routeingress.go +++ /dev/null @@ -1,58 +0,0 @@ -// Copyright 2025 Datum Cloud, Inc. -// -// SPDX-License-Identifier: AGPL-3.0-or-later - -package routeingress - -import ( - "net" - - "github.com/vishvananda/netlink" - "github.com/vishvananda/netlink/nl" - - "go.datum.net/galactic/pkg/common/util" - "go.datum.net/galactic/pkg/common/vrf" -) - -func Add(ip *net.IPNet, vpc, vpcAttachment string) error { - dev := util.GenerateInterfaceNameHost(vpc, vpcAttachment) - link, err := netlink.LinkByName(dev) - if err != nil { - return err - } - - vrfId, err := vrf.GetVRFIdForVPC(vpc, vpcAttachment) - if err != nil { - return err - } - - var flags [nl.SEG6_LOCAL_MAX]bool - flags[nl.SEG6_LOCAL_ACTION] = true - flags[nl.SEG6_LOCAL_VRFTABLE] = true - encap := &netlink.SEG6LocalEncap{ - Action: nl.SEG6_LOCAL_ACTION_END_DT46, - Flags: flags, - VrfTable: int(vrfId), - } - route := &netlink.Route{ - Dst: ip, - LinkIndex: link.Attrs().Index, - Encap: encap, - } - return netlink.RouteReplace(route) -} - -func Delete(ip *net.IPNet, vpc, vpcAttachment string) error { - dev := util.GenerateInterfaceNameHost(vpc, vpcAttachment) - link, err := netlink.LinkByName(dev) - if err != nil { - return err - } - - route := &netlink.Route{ - Dst: ip, - LinkIndex: link.Attrs().Index, - Encap: &netlink.SEG6LocalEncap{}, - } - return netlink.RouteDel(route) -} diff --git a/internal/agent/srv6/srv6.go b/internal/agent/srv6/srv6.go deleted file mode 100644 index bac4892..0000000 --- a/internal/agent/srv6/srv6.go +++ /dev/null @@ -1,62 +0,0 @@ -// Copyright 2025 Datum Cloud, Inc. -// -// SPDX-License-Identifier: AGPL-3.0-or-later - -package srv6 - -import ( - "fmt" - - "github.com/vishvananda/netlink" - - "go.datum.net/galactic/internal/agent/srv6/routeingress" - "go.datum.net/galactic/pkg/common/util" -) - -func RouteIngressAdd(ipStr string) error { - ip, err := util.ParseIP(ipStr) - if err != nil { - return fmt.Errorf("invalid ip: %w", err) - } - vpc, vpcAttachment, err := util.DecodeSRv6Endpoint(ip) - if err != nil { - return fmt.Errorf("could not extract SRv6 endpoint: %w", err) - } - vpc, err = util.HexToBase62(vpc) - if err != nil { - return fmt.Errorf("invalid vpc: %w", err) - } - vpcAttachment, err = util.HexToBase62(vpcAttachment) - if err != nil { - return fmt.Errorf("invalid vpcattachment: %w", err) - } - - if err := routeingress.Add(netlink.NewIPNet(ip), vpc, vpcAttachment); err != nil { - return fmt.Errorf("routeingress add failed: %w", err) - } - return nil -} - -func RouteIngressDel(ipStr string) error { - ip, err := util.ParseIP(ipStr) - if err != nil { - return fmt.Errorf("invalid ip: %w", err) - } - vpc, vpcAttachment, err := util.DecodeSRv6Endpoint(ip) - if err != nil { - return fmt.Errorf("could not extract SRv6 endpoint: %w", err) - } - vpc, err = util.HexToBase62(vpc) - if err != nil { - return fmt.Errorf("invalid vpc: %w", err) - } - vpcAttachment, err = util.HexToBase62(vpcAttachment) - if err != nil { - return fmt.Errorf("invalid vpcattachment: %w", err) - } - - if err := routeingress.Delete(netlink.NewIPNet(ip), vpc, vpcAttachment); err != nil { - return fmt.Errorf("routeingress delete failed: %w", err) - } - return nil -} diff --git a/internal/cmd/version/version.go b/internal/cmd/version/version.go deleted file mode 100644 index 2793ed9..0000000 --- a/internal/cmd/version/version.go +++ /dev/null @@ -1,20 +0,0 @@ -// Copyright 2025 Datum Cloud, Inc. -// -// SPDX-License-Identifier: AGPL-3.0-or-later - -package version - -import ( - "fmt" - "runtime" -) - -var ( - // Version information - set via ldflags during build - Version = "dev" - GitCommit = "unknown" - GitTreeState = "unknown" - BuildDate = "unknown" - GoVersion = runtime.Version() - Platform = fmt.Sprintf("%s/%s", runtime.GOOS, runtime.GOARCH) -) diff --git a/internal/cni/cni.go b/internal/cni/cni.go index 0513e74..0f19acb 100644 --- a/internal/cni/cni.go +++ b/internal/cni/cni.go @@ -31,13 +31,12 @@ import ( "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" - "go.datum.net/galactic/internal/agent/srv6" - galversion "go.datum.net/galactic/internal/cmd/version" "go.datum.net/galactic/internal/cni/route" "go.datum.net/galactic/internal/cni/veth" - "go.datum.net/galactic/pkg/common/cni" - "go.datum.net/galactic/pkg/common/util" - "go.datum.net/galactic/pkg/common/vrf" + "go.datum.net/galactic/internal/metadata" + "go.datum.net/galactic/internal/plumbing/intf" + "go.datum.net/galactic/internal/plumbing/srv6" + "go.datum.net/galactic/internal/plumbing/vrf" ) const cniTimeout = 10 * time.Second @@ -55,12 +54,12 @@ func init() { // PluginConf is the CNI plugin configuration passed via stdin on each invocation. type PluginConf struct { types.PluginConf - VPC string `json:"vpc"` - VPCAttachment string `json:"vpcattachment"` - MTU int `json:"mtu,omitempty"` - Terminations []cni.Termination `json:"terminations,omitempty"` - IPAM cni.IPAM `json:"ipam,omitempty"` - SRv6Locator string `json:"srv6_locator"` + VPC string `json:"vpc"` + VPCAttachment string `json:"vpcattachment"` + MTU int `json:"mtu,omitempty"` + Terminations []Termination `json:"terminations,omitempty"` + IPAM IPAM `json:"ipam,omitempty"` + SRv6Locator string `json:"srv6_locator"` } func RunPlugin() { @@ -70,7 +69,7 @@ func RunPlugin() { Del: cmdDel, }, version.All, - fmt.Sprintf("CNI galactic plugin %s", galversion.Version), + fmt.Sprintf("CNI galactic plugin %s", metadata.Version), ) } @@ -217,7 +216,7 @@ func cmdAdd(args *skel.CmdArgs) error { if err := veth.Add(pluginConf.VPC, pluginConf.VPCAttachment, pluginConf.MTU); err != nil { return fmt.Errorf("add veth: %w", err) } - dev := util.GenerateInterfaceNameHost(pluginConf.VPC, pluginConf.VPCAttachment) + dev := intf.GenerateInterfaceNameHost(pluginConf.VPC, pluginConf.VPCAttachment) for _, termination := range pluginConf.Terminations { if err := route.Add(pluginConf.VPC, pluginConf.VPCAttachment, termination.Network, termination.Via, dev); err != nil { return fmt.Errorf("add route %s: %w", termination.Network, err) @@ -227,11 +226,11 @@ func cmdAdd(args *skel.CmdArgs) error { return fmt.Errorf("host-device ADD: %w", err) } - vpcHex, err := util.Base62ToHex(pluginConf.VPC) + vpcHex, err := intf.Base62ToHex(pluginConf.VPC) if err != nil { return fmt.Errorf("decode VPC: %w", err) } - vpcAttachmentHex, err := util.Base62ToHex(pluginConf.VPCAttachment) + vpcAttachmentHex, err := intf.Base62ToHex(pluginConf.VPCAttachment) if err != nil { return fmt.Errorf("decode VPCAttachment: %w", err) } @@ -277,7 +276,7 @@ func cmdAdd(args *skel.CmdArgs) error { return fmt.Errorf("apply BGPVRFInstance: %w", err) } - srv6Endpoint, err := util.EncodeSRv6Endpoint(pluginConf.SRv6Locator, vpcHex, vpcAttachmentHex) + srv6Endpoint, err := intf.EncodeSRv6Endpoint(pluginConf.SRv6Locator, vpcHex, vpcAttachmentHex) if err != nil { return fmt.Errorf("encode SRv6 endpoint: %w", err) } @@ -295,15 +294,15 @@ func cmdDel(args *skel.CmdArgs) error { return err } - vpcHex, err := util.Base62ToHex(pluginConf.VPC) + vpcHex, err := intf.Base62ToHex(pluginConf.VPC) if err != nil { return fmt.Errorf("decode VPC: %w", err) } - vpcAttachmentHex, err := util.Base62ToHex(pluginConf.VPCAttachment) + vpcAttachmentHex, err := intf.Base62ToHex(pluginConf.VPCAttachment) if err != nil { return fmt.Errorf("decode VPCAttachment: %w", err) } - srv6Endpoint, err := util.EncodeSRv6Endpoint(pluginConf.SRv6Locator, vpcHex, vpcAttachmentHex) + srv6Endpoint, err := intf.EncodeSRv6Endpoint(pluginConf.SRv6Locator, vpcHex, vpcAttachmentHex) if err != nil { return fmt.Errorf("encode SRv6 endpoint: %w", err) } @@ -330,7 +329,7 @@ func cmdDel(args *skel.CmdArgs) error { return fmt.Errorf("delete BGPVRFInstance: %w", err) } - dev := util.GenerateInterfaceNameHost(pluginConf.VPC, pluginConf.VPCAttachment) + dev := intf.GenerateInterfaceNameHost(pluginConf.VPC, pluginConf.VPCAttachment) if err := hostDevice("DEL", args, pluginConf); err != nil { return fmt.Errorf("host-device DEL: %w", err) } @@ -352,8 +351,8 @@ func cmdDel(args *skel.CmdArgs) error { type HostDevicePluginConf struct { types.PluginConf - Device string `json:"device"` - IPAM cni.IPAM `json:"ipam,omitempty"` + Device string `json:"device"` + IPAM IPAM `json:"ipam,omitempty"` } func hostDeviceExecutable() string { @@ -369,7 +368,7 @@ func hostDevice(command string, skelArgs *skel.CmdArgs, pluginConf *PluginConf) Name: pluginConf.Name, Type: "host-device", }, - Device: util.GenerateInterfaceNameGuest(pluginConf.VPC, pluginConf.VPCAttachment), + Device: intf.GenerateInterfaceNameGuest(pluginConf.VPC, pluginConf.VPCAttachment), IPAM: pluginConf.IPAM, }) if err != nil { diff --git a/internal/cni/route/route.go b/internal/cni/route/route.go index b63627f..d3ee20e 100644 --- a/internal/cni/route/route.go +++ b/internal/cni/route/route.go @@ -5,14 +5,14 @@ package route import ( + "fmt" "net" "golang.org/x/sys/unix" "github.com/vishvananda/netlink" - gutil "go.datum.net/galactic/pkg/common/util" - "go.datum.net/galactic/pkg/common/vrf" + "go.datum.net/galactic/internal/plumbing/vrf" ) func assembleRoute(vrfId uint32, prefix, nextHop, dev string) (*netlink.Route, error) { @@ -22,9 +22,9 @@ func assembleRoute(vrfId uint32, prefix, nextHop, dev string) (*netlink.Route, e } if nextHop != "" { - routeGw, err := gutil.ParseIP(nextHop) - if err != nil { - return nil, err + routeGw := net.ParseIP(nextHop) + if routeGw == nil { + return nil, fmt.Errorf("cannot parse gateway IP: %s", nextHop) } return &netlink.Route{ Dst: routeDst, @@ -46,7 +46,7 @@ func assembleRoute(vrfId uint32, prefix, nextHop, dev string) (*netlink.Route, e } func Add(vpc, vpcAttachment string, prefix, nextHop, dev string) error { - vrfId, err := vrf.GetVRFIdForVPC(vpc, vpcAttachment) + vrfId, err := vrf.TableID(vpc, vpcAttachment) if err != nil { return err } @@ -58,7 +58,7 @@ func Add(vpc, vpcAttachment string, prefix, nextHop, dev string) error { } func Delete(vpc, vpcAttachment string, prefix, nextHop, dev string) error { - vrfId, err := vrf.GetVRFIdForVPC(vpc, vpcAttachment) + vrfId, err := vrf.TableID(vpc, vpcAttachment) if err != nil { return err } diff --git a/pkg/common/cni/types.go b/internal/cni/types.go similarity index 100% rename from pkg/common/cni/types.go rename to internal/cni/types.go diff --git a/internal/cni/veth/veth.go b/internal/cni/veth/veth.go index 0395de1..35fe774 100644 --- a/internal/cni/veth/veth.go +++ b/internal/cni/veth/veth.go @@ -9,8 +9,8 @@ import ( "github.com/coreos/go-iptables/iptables" "github.com/vishvananda/netlink" - "go.datum.net/galactic/pkg/common/sysctl" - "go.datum.net/galactic/pkg/common/util" + "go.datum.net/galactic/internal/plumbing/intf" + "go.datum.net/galactic/internal/plumbing/sysctl" ) func updateForwardRule(interfaceName string, action string) error { @@ -41,9 +41,9 @@ func updateForwardRule(interfaceName string, action string) error { } func Add(vpc, vpcAttachment string, mtu int) error { - vrfName := util.GenerateInterfaceNameVRF(vpc, vpcAttachment) - hostName := util.GenerateInterfaceNameHost(vpc, vpcAttachment) - guestName := util.GenerateInterfaceNameGuest(vpc, vpcAttachment) + vrfName := intf.GenerateInterfaceNameVRF(vpc, vpcAttachment) + hostName := intf.GenerateInterfaceNameHost(vpc, vpcAttachment) + guestName := intf.GenerateInterfaceNameGuest(vpc, vpcAttachment) veth := &netlink.Veth{ LinkAttrs: netlink.LinkAttrs{ @@ -89,7 +89,7 @@ func Add(vpc, vpcAttachment string, mtu int) error { } func Delete(vpc, vpcAttachment string) error { - hostName := util.GenerateInterfaceNameHost(vpc, vpcAttachment) + hostName := intf.GenerateInterfaceNameHost(vpc, vpcAttachment) if err := updateForwardRule(hostName, "delete"); err != nil { return err diff --git a/internal/metadata/metadata.go b/internal/metadata/metadata.go new file mode 100644 index 0000000..a961dd5 --- /dev/null +++ b/internal/metadata/metadata.go @@ -0,0 +1,35 @@ +// Copyright 2025 Datum Cloud, Inc. +// +// SPDX-License-Identifier: AGPL-3.0-or-later + +// Package metadata exposes build-time variables stamped in via -ldflags. +package metadata + +import ( + "fmt" + "os" + "runtime" +) + +var ( + // Version is the release tag (e.g. v0.1.0). Set at build time. + Version = "dev" + // GitCommit is the short SHA of the build commit. Set at build time. + GitCommit = "unknown" + // GitTreeState is "clean" or "dirty". Set at build time. + GitTreeState = "unknown" + // BuildDate is the RFC3339 UTC timestamp of the build. Set at build time. + BuildDate = "unknown" + // GoVersion is the Go toolchain version used for the build. + GoVersion = runtime.Version() + // Platform is the OS/arch pair of the build host. + Platform = fmt.Sprintf("%s/%s", runtime.GOOS, runtime.GOARCH) + // Executable is the full path to the running binary as reported by the OS. + Executable = func() string { + exe, err := os.Executable() + if err != nil { + return "unknown" + } + return exe + }() +) diff --git a/internal/plumbing/doc.go b/internal/plumbing/doc.go new file mode 100644 index 0000000..d0ae056 --- /dev/null +++ b/internal/plumbing/doc.go @@ -0,0 +1,10 @@ +// Copyright 2025 Datum Cloud, Inc. +// +// SPDX-License-Identifier: AGPL-3.0-or-later + +// Package plumbing provides low-level kernel and network primitives shared +// between the Galactic CNI plugin and agent. Sub-packages cover SRv6 endpoint +// encoding and ingress routing (intf, srv6), Linux VRF lifecycle (vrf), and +// interface sysctl configuration (sysctl). Functions that require CAP_NET_ADMIN +// or a real kernel are not unit-tested; use the e2e suite for coverage. +package plumbing diff --git a/internal/plumbing/intf/intf.go b/internal/plumbing/intf/intf.go new file mode 100644 index 0000000..094ecaf --- /dev/null +++ b/internal/plumbing/intf/intf.go @@ -0,0 +1,94 @@ +// Copyright 2025 Datum Cloud, Inc. +// +// SPDX-License-Identifier: AGPL-3.0-or-later + +// Package intf provides deterministic interface naming, base62↔hex ID +// encoding, and SRv6 endpoint encode/decode for Galactic VPC identifiers. +package intf + +import ( + "encoding/binary" + "fmt" + "net" + "strconv" + "strings" + + "github.com/kenshaw/baseconv" +) + +const interfaceNameTemplate = "G%09s%03s%s" + +// GenerateInterfaceNameVRF returns the kernel interface name for the VRF +// associated with the given base62-encoded VPC and VPCAttachment. +func GenerateInterfaceNameVRF(vpc, vpcAttachment string) string { + return fmt.Sprintf(interfaceNameTemplate, vpc, vpcAttachment, "V") +} + +// GenerateInterfaceNameHost returns the kernel interface name for the host-side +// veth endpoint for the given base62-encoded VPC and VPCAttachment. +func GenerateInterfaceNameHost(vpc, vpcAttachment string) string { + return fmt.Sprintf(interfaceNameTemplate, vpc, vpcAttachment, "H") +} + +// GenerateInterfaceNameGuest returns the kernel interface name for the +// guest-side veth endpoint (moved into the container netns) for the given +// base62-encoded VPC and VPCAttachment. +func GenerateInterfaceNameGuest(vpc, vpcAttachment string) string { + return fmt.Sprintf(interfaceNameTemplate, vpc, vpcAttachment, "G") +} + +// HexToBase62 converts a hex string to base62. VPC and VPCAttachment +// identifiers are hex in SRv6 SIDs but base62 in kernel interface names to +// stay within the 15-character limit. +func HexToBase62(value string) (string, error) { + return baseconv.Convert(strings.ToLower(value), baseconv.DigitsHex, baseconv.Digits62) +} + +// Base62ToHex converts a base62 string to lowercase hex. +func Base62ToHex(value string) (string, error) { + return baseconv.Convert(value, baseconv.Digits62, baseconv.DigitsHex) +} + +// EncodeSRv6Endpoint packs hex VPC (48-bit) and VPCAttachment (16-bit) +// identifiers into the low 64 bits of an IPv6 address within srv6Net. +// srv6Net must be an IPv6 prefix of /64 or shorter. +func EncodeSRv6Endpoint(srv6Net, vpc, vpcAttachment string) (string, error) { + ip, ipnet, err := net.ParseCIDR(srv6Net) + if err != nil { + return "", err + } + if ip.To4() != nil { + return "", fmt.Errorf("provided srv6Net is not IPv6: %s", srv6Net) + } + maskLen, _ := ipnet.Mask.Size() + if maskLen > 64 { + return "", fmt.Errorf("srv6Net must be at least 64 bits long") + } + + vpcInt, err := strconv.ParseUint(vpc, 16, 64) + if err != nil { + return "", fmt.Errorf("invalid vpc %q: %w", vpc, err) + } + vpcAttachmentInt, err := strconv.ParseUint(vpcAttachment, 16, 16) + if err != nil { + return "", fmt.Errorf("invalid vpcAttachment %q: %w", vpcAttachment, err) + } + + binary.BigEndian.PutUint64(ip[8:16], (vpcInt<<16)|vpcAttachmentInt) + return ip.String(), nil +} + +// DecodeSRv6Endpoint extracts the 12-digit hex VPC and 4-digit hex +// VPCAttachment identifiers from the low 64 bits of an SRv6 SID produced by +// EncodeSRv6Endpoint. endpoint must be an IPv6 address. +func DecodeSRv6Endpoint(endpoint net.IP) (string, string, error) { + ep := endpoint.To16() + if ep == nil || endpoint.To4() != nil { + return "", "", fmt.Errorf("provided endpoint is not an IPv6 address: %s", endpoint) + } + + id := binary.BigEndian.Uint64(ep[8:16]) + vpc := (id >> 16) & 0xFFFFFFFFFFFF + vpcAttachment := id & 0xFFFF + return fmt.Sprintf("%012x", vpc), fmt.Sprintf("%04x", vpcAttachment), nil +} diff --git a/internal/plumbing/intf/intf_test.go b/internal/plumbing/intf/intf_test.go new file mode 100644 index 0000000..f94c635 --- /dev/null +++ b/internal/plumbing/intf/intf_test.go @@ -0,0 +1,248 @@ +// Copyright 2025 Datum Cloud, Inc. +// +// SPDX-License-Identifier: AGPL-3.0-or-later + +package intf_test + +import ( + "net" + "testing" + + "go.datum.net/galactic/internal/plumbing/intf" +) + +const ( + testVPCHex = "0000000004d2" // 1234 decimal + testVPCBase62 = "jU" // 1234 decimal in base62 + testVPCAttachmentHex = "002a" // 42 decimal + testVPCAttachmentB62 = "G" // 42 decimal in base62 + + // Formatted for interface name generation (padded by the template). + testVPC = "0000000jU" // base62 of 1234, right-padded to 9 chars + testVPCAttachment = "00G" // base62 of 42, right-padded to 3 chars + + testLocator = "2607:ed40:ff00::/64" + testSIDEncoded = "2607:ed40:ff00::4d2:2a" + testMaxVPCAtt = "ffff" +) + +func TestGenerateInterfaceNameVRF(t *testing.T) { + expected := "G0000000jU00GV" + got := intf.GenerateInterfaceNameVRF(testVPC, testVPCAttachment) + if got != expected { + t.Errorf("GenerateInterfaceNameVRF(%s, %s) = %s, want %s", testVPC, testVPCAttachment, got, expected) + } +} + +func TestGenerateInterfaceNameHost(t *testing.T) { + expected := "G0000000jU00GH" + got := intf.GenerateInterfaceNameHost(testVPC, testVPCAttachment) + if got != expected { + t.Errorf("GenerateInterfaceNameHost(%s, %s) = %s, want %s", testVPC, testVPCAttachment, got, expected) + } +} + +func TestGenerateInterfaceNameGuest(t *testing.T) { + expected := "G0000000jU00GG" + got := intf.GenerateInterfaceNameGuest(testVPC, testVPCAttachment) + if got != expected { + t.Errorf("GenerateInterfaceNameGuest(%s, %s) = %s, want %s", testVPC, testVPCAttachment, got, expected) + } +} + +func TestHexToBase62(t *testing.T) { + tests := []struct { + name string + input string + want string + wantError bool + }{ + {"VPCValue", testVPCHex, testVPCBase62, false}, + {"VPCAttachmentValue", testVPCAttachmentHex, testVPCAttachmentB62, false}, + {"Zero", "0", "0", false}, + {"UppercaseInput", "4D2", testVPCBase62, false}, // input normalised to lowercase + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got, err := intf.HexToBase62(tt.input) + if (err != nil) != tt.wantError { + t.Errorf("HexToBase62(%s) error = %v, wantError = %v", tt.input, err, tt.wantError) + } + if !tt.wantError && got != tt.want { + t.Errorf("HexToBase62(%s) = %s, want %s", tt.input, got, tt.want) + } + }) + } +} + +func TestBase62ToHex(t *testing.T) { + tests := []struct { + name string + input string + want string + wantError bool + }{ + {"VPCValue", testVPCBase62, "4d2", false}, + {"VPCAttachmentValue", testVPCAttachmentB62, "2a", false}, + {"Zero", "0", "0", false}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got, err := intf.Base62ToHex(tt.input) + if (err != nil) != tt.wantError { + t.Errorf("Base62ToHex(%s) error = %v, wantError = %v", tt.input, err, tt.wantError) + } + if !tt.wantError && got != tt.want { + t.Errorf("Base62ToHex(%s) = %s, want %s", tt.input, got, tt.want) + } + }) + } +} + +func TestHexBase62RoundTrip(t *testing.T) { + hexInputs := []string{"4d2", "2a", "0", "ffffffffffff", testMaxVPCAtt} + for _, hex := range hexInputs { + b62, err := intf.HexToBase62(hex) + if err != nil { + t.Errorf("HexToBase62(%s) unexpected error: %v", hex, err) + continue + } + got, err := intf.Base62ToHex(b62) + if err != nil { + t.Errorf("Base62ToHex(%s) unexpected error: %v", b62, err) + continue + } + if got != hex { + t.Errorf("round-trip %s → %s → %s", hex, b62, got) + } + } +} + +func TestEncodeSRv6Endpoint(t *testing.T) { + tests := []struct { + name string + srv6Net string + vpc string + vpcAttachment string + want string + wantError bool + }{ + { + name: "Valid64BitMask", + srv6Net: testLocator, + vpc: testVPCHex, + vpcAttachment: testVPCAttachmentHex, + want: testSIDEncoded, + }, + { + name: "Valid48BitMask", + srv6Net: "2607:ed40:ff00::/48", + vpc: testVPCHex, + vpcAttachment: testVPCAttachmentHex, + want: testSIDEncoded, + }, + { + name: "ZeroIDs", + srv6Net: "2607:ed40:ff00::/64", + vpc: "000000000000", + vpcAttachment: "0000", + want: "2607:ed40:ff00::", + }, + { + name: "IPv4NetworkRejected", + srv6Net: "192.168.0.0/24", + wantError: true, + }, + { + name: "MaskTooNarrow", + srv6Net: "2607:ed40:ff00::/96", + wantError: true, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got, err := intf.EncodeSRv6Endpoint(tt.srv6Net, tt.vpc, tt.vpcAttachment) + if (err != nil) != tt.wantError { + t.Errorf("EncodeSRv6Endpoint() error = %v, wantError = %v", err, tt.wantError) + } + if !tt.wantError && got != tt.want { + t.Errorf("EncodeSRv6Endpoint() = %s, want %s", got, tt.want) + } + }) + } +} + +func TestDecodeSRv6Endpoint(t *testing.T) { + tests := []struct { + name string + endpoint string + wantVPC string + wantVPCAttachment string + wantError bool + }{ + { + name: "Valid", + endpoint: testSIDEncoded, + wantVPC: testVPCHex, + wantVPCAttachment: testVPCAttachmentHex, + }, + { + name: "ZeroIDs", + endpoint: "2607:ed40:ff00::", + wantVPC: "000000000000", + wantVPCAttachment: "0000", + }, + { + name: "MaxVPCAttachment", + endpoint: "2607:ed40:ff00::" + testMaxVPCAtt, + wantVPC: "000000000000", + wantVPCAttachment: testMaxVPCAtt, + }, + { + name: "IPv4Rejected", + endpoint: "192.168.0.1", + wantError: true, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + gotVPC, gotAtt, err := intf.DecodeSRv6Endpoint(net.ParseIP(tt.endpoint)) + if (err != nil) != tt.wantError { + t.Errorf("DecodeSRv6Endpoint() error = %v, wantError = %v", err, tt.wantError) + } + if !tt.wantError && (gotVPC != tt.wantVPC || gotAtt != tt.wantVPCAttachment) { + t.Errorf("DecodeSRv6Endpoint(%s) = %s, %s, want %s, %s", + tt.endpoint, gotVPC, gotAtt, tt.wantVPC, tt.wantVPCAttachment) + } + }) + } +} + +func TestEncodeDecodeSRv6RoundTrip(t *testing.T) { + cases := []struct{ vpc, att string }{ + {testVPCHex, testVPCAttachmentHex}, + {"000000000000", "0000"}, + {"ffffffffffff", testMaxVPCAtt}, + {"000000000001", "0001"}, + } + + for _, c := range cases { + encoded, err := intf.EncodeSRv6Endpoint(testLocator, c.vpc, c.att) + if err != nil { + t.Errorf("EncodeSRv6Endpoint(%s, %s) unexpected error: %v", c.vpc, c.att, err) + continue + } + gotVPC, gotAtt, err := intf.DecodeSRv6Endpoint(net.ParseIP(encoded)) + if err != nil { + t.Errorf("DecodeSRv6Endpoint(%s) unexpected error: %v", encoded, err) + continue + } + if gotVPC != c.vpc || gotAtt != c.att { + t.Errorf("round-trip vpc=%s att=%s → %s → vpc=%s att=%s", c.vpc, c.att, encoded, gotVPC, gotAtt) + } + } +} diff --git a/internal/plumbing/srv6/srv6.go b/internal/plumbing/srv6/srv6.go new file mode 100644 index 0000000..11c5b63 --- /dev/null +++ b/internal/plumbing/srv6/srv6.go @@ -0,0 +1,111 @@ +// Copyright 2025 Datum Cloud, Inc. +// +// SPDX-License-Identifier: AGPL-3.0-or-later + +// Package srv6 manages kernel SRv6 END.DT46 ingress routes for Galactic VPC +// endpoints. It decodes SRv6 SIDs to extract VPC identity and delegates route +// installation to the Linux kernel via netlink. Requires CAP_NET_ADMIN. +package srv6 + +import ( + "fmt" + "net" + + "github.com/vishvananda/netlink" + "github.com/vishvananda/netlink/nl" + + "go.datum.net/galactic/internal/plumbing/intf" + "go.datum.net/galactic/internal/plumbing/vrf" +) + +// RouteIngressAdd installs an SRv6 END.DT46 ingress route for the given SRv6 +// SID, decoding the embedded VPC and VPCAttachment to locate the correct host +// interface and VRF routing table. +func RouteIngressAdd(ipStr string) error { + ip := net.ParseIP(ipStr) + if ip == nil { + return fmt.Errorf("invalid ip: %s", ipStr) + } + vpc, vpcAttachment, err := intf.DecodeSRv6Endpoint(ip) + if err != nil { + return fmt.Errorf("could not extract SRv6 endpoint: %w", err) + } + vpc, err = intf.HexToBase62(vpc) + if err != nil { + return fmt.Errorf("invalid vpc: %w", err) + } + vpcAttachment, err = intf.HexToBase62(vpcAttachment) + if err != nil { + return fmt.Errorf("invalid vpcattachment: %w", err) + } + if err := addIngressRoute(netlink.NewIPNet(ip), vpc, vpcAttachment); err != nil { + return fmt.Errorf("add ingress route failed: %w", err) + } + return nil +} + +// RouteIngressDel removes the SRv6 END.DT46 ingress route previously installed +// by RouteIngressAdd for the given SRv6 SID. +func RouteIngressDel(ipStr string) error { + ip := net.ParseIP(ipStr) + if ip == nil { + return fmt.Errorf("invalid ip: %s", ipStr) + } + vpc, vpcAttachment, err := intf.DecodeSRv6Endpoint(ip) + if err != nil { + return fmt.Errorf("could not extract SRv6 endpoint: %w", err) + } + vpc, err = intf.HexToBase62(vpc) + if err != nil { + return fmt.Errorf("invalid vpc: %w", err) + } + vpcAttachment, err = intf.HexToBase62(vpcAttachment) + if err != nil { + return fmt.Errorf("invalid vpcattachment: %w", err) + } + if err := deleteIngressRoute(netlink.NewIPNet(ip), vpc, vpcAttachment); err != nil { + return fmt.Errorf("delete ingress route failed: %w", err) + } + return nil +} + +func addIngressRoute(ip *net.IPNet, vpc, vpcAttachment string) error { + dev := intf.GenerateInterfaceNameHost(vpc, vpcAttachment) + link, err := netlink.LinkByName(dev) + if err != nil { + return err + } + + vrfID, err := vrf.TableID(vpc, vpcAttachment) + if err != nil { + return err + } + + var flags [nl.SEG6_LOCAL_MAX]bool + flags[nl.SEG6_LOCAL_ACTION] = true + flags[nl.SEG6_LOCAL_VRFTABLE] = true + encap := &netlink.SEG6LocalEncap{ + Action: nl.SEG6_LOCAL_ACTION_END_DT46, + Flags: flags, + VrfTable: int(vrfID), + } + return netlink.RouteReplace(&netlink.Route{ + Dst: ip, + LinkIndex: link.Attrs().Index, + Encap: encap, + }) +} + +func deleteIngressRoute(ip *net.IPNet, vpc, vpcAttachment string) error { + dev := intf.GenerateInterfaceNameHost(vpc, vpcAttachment) + link, err := netlink.LinkByName(dev) + if err != nil { + return err + } + + return netlink.RouteDel(&netlink.Route{ + Dst: ip, + LinkIndex: link.Attrs().Index, + Encap: &netlink.SEG6LocalEncap{}, + }) +} diff --git a/pkg/common/sysctl/sysctl.go b/internal/plumbing/sysctl/sysctl.go similarity index 61% rename from pkg/common/sysctl/sysctl.go rename to internal/plumbing/sysctl/sysctl.go index 79cf711..d32caaf 100644 --- a/pkg/common/sysctl/sysctl.go +++ b/internal/plumbing/sysctl/sysctl.go @@ -2,6 +2,8 @@ // // SPDX-License-Identifier: AGPL-3.0-or-later +// Package sysctl applies kernel sysctl settings required for VRF-based +// container networking. Requires CAP_NET_ADMIN. package sysctl import ( @@ -10,7 +12,7 @@ import ( gosysctl "github.com/lorenzosaino/go-sysctl" ) -var INTERFACE_SETTINGS = []struct { +var interfaceSettings = []struct { format string value string }{ @@ -21,8 +23,10 @@ var INTERFACE_SETTINGS = []struct { {"net.ipv6.conf.%s.proxy_ndp", "1"}, } +// ConfigureInterfaceSysctls applies forwarding, rp_filter, and proxy ARP/NDP +// sysctl settings to iface, which are required for correct VRF packet handling. func ConfigureInterfaceSysctls(iface string) error { - for _, entry := range INTERFACE_SETTINGS { + for _, entry := range interfaceSettings { key := fmt.Sprintf(entry.format, iface) if err := gosysctl.Set(key, entry.value); err != nil { return err diff --git a/pkg/common/vrf/vrf.go b/internal/plumbing/vrf/vrf.go similarity index 72% rename from pkg/common/vrf/vrf.go rename to internal/plumbing/vrf/vrf.go index b8a6c4f..153088a 100644 --- a/pkg/common/vrf/vrf.go +++ b/internal/plumbing/vrf/vrf.go @@ -2,6 +2,9 @@ // // SPDX-License-Identifier: AGPL-3.0-or-later +// Package vrf manages Linux VRF interfaces for Galactic VPC network isolation. +// Each VPC attachment gets its own VRF with a unique routing table ID. +// Requires CAP_NET_ADMIN. package vrf import ( @@ -13,8 +16,8 @@ import ( "golang.org/x/sys/unix" "github.com/vishvananda/netlink" - "go.datum.net/galactic/pkg/common/sysctl" - "go.datum.net/galactic/pkg/common/util" + "go.datum.net/galactic/internal/plumbing/intf" + "go.datum.net/galactic/internal/plumbing/sysctl" ) const minVRFId = uint32(1) @@ -24,11 +27,14 @@ const maxVRFId = uint32(math.MaxUint32 - 1) // scanning the same free table ID and both attempting to create a VRF with it. var vrfMu sync.Mutex +// Add creates a Linux VRF interface for the given base62-encoded VPC and +// VPCAttachment, allocating the next available routing table ID and applying +// the required sysctl settings. Concurrent calls are serialized internally. func Add(vpc, vpcAttachment string) error { vrfMu.Lock() defer vrfMu.Unlock() - name := util.GenerateInterfaceNameVRF(vpc, vpcAttachment) + name := intf.GenerateInterfaceNameVRF(vpc, vpcAttachment) vrfId, err := findNextAvailableVRFId() if err != nil { @@ -57,8 +63,10 @@ func Add(vpc, vpcAttachment string) error { return netlink.LinkSetUp(vrf) } +// Delete flushes all routes from the VRF routing table and removes the VRF +// interface for the given base62-encoded VPC and VPCAttachment. func Delete(vpc, vpcAttachment string) error { - name := util.GenerateInterfaceNameVRF(vpc, vpcAttachment) + name := intf.GenerateInterfaceNameVRF(vpc, vpcAttachment) vrfId, err := getVRFIdForInterface(name) if err != nil { @@ -77,8 +85,10 @@ func Delete(vpc, vpcAttachment string) error { return netlink.LinkDel(link) } -func GetVRFIdForVPC(vpc, vpcAttachment string) (uint32, error) { - return getVRFIdForInterface(util.GenerateInterfaceNameVRF(vpc, vpcAttachment)) +// TableID returns the Linux routing table ID for the VRF associated with the +// given base62-encoded VPC and VPCAttachment. +func TableID(vpc, vpcAttachment string) (uint32, error) { + return getVRFIdForInterface(intf.GenerateInterfaceNameVRF(vpc, vpcAttachment)) } func flush(vrfId uint32) error { diff --git a/pkg/common/util/util.go b/pkg/common/util/util.go deleted file mode 100644 index 02f9a72..0000000 --- a/pkg/common/util/util.go +++ /dev/null @@ -1,90 +0,0 @@ -// Copyright 2025 Datum Cloud, Inc. -// -// SPDX-License-Identifier: AGPL-3.0-or-later - -package util - -import ( - "encoding/binary" - "fmt" - "math/big" - "net" - "strconv" - "strings" - - "github.com/kenshaw/baseconv" -) - -const interfaceNameTemplate = "G%09s%03s%s" - -func GenerateInterfaceNameVRF(vpc, vpcAttachment string) string { - return fmt.Sprintf(interfaceNameTemplate, vpc, vpcAttachment, "V") -} - -func GenerateInterfaceNameHost(vpc, vpcAttachment string) string { - return fmt.Sprintf(interfaceNameTemplate, vpc, vpcAttachment, "H") -} - -func GenerateInterfaceNameGuest(vpc, vpcAttachment string) string { - return fmt.Sprintf(interfaceNameTemplate, vpc, vpcAttachment, "G") -} - -func ParseIP(ip string) (net.IP, error) { - parsed := net.ParseIP(ip) - if parsed == nil { - return nil, fmt.Errorf("cannot parse IP: %v", ip) - } - return parsed, nil -} - -func DecodeSRv6Endpoint(endpoint net.IP) (string, string, error) { - if endpoint.To4() != nil { - return "", "", fmt.Errorf("provided endpoint is not an IPv6 address: %s", endpoint) - } - - endpointNum := new(big.Int).SetBytes(endpoint) - vpcNum := new(big.Int).And( - new(big.Int).Rsh(endpointNum, 16), // drop the vpcattachment bits - big.NewInt(0xFFFFFFFFFFFF), // mask the vpc bits - ) - vpcAttachmentNum := new(big.Int).And( - endpointNum, - big.NewInt(0xFFFF), // mask the vpcattachment bits - ) - - return fmt.Sprintf("%012x", vpcNum), fmt.Sprintf("%04x", vpcAttachmentNum), nil -} - -func EncodeSRv6Endpoint(srv6_net, vpc, vpcAttachment string) (string, error) { - ip, ipnet, err := net.ParseCIDR(srv6_net) - if err != nil { - return "", err - } - if ip.To4() != nil { - return "", fmt.Errorf("provided srv6_net is not IPv6: %s", srv6_net) - } - mask_len, _ := ipnet.Mask.Size() - if mask_len > 64 { - return "", fmt.Errorf("srv6_net must be at least 64 bits long") - } - - vpcInt, err := strconv.ParseUint(vpc, 16, 64) - if err != nil { - return "", fmt.Errorf("invalid vpc %q: %w", vpc, err) - } - vpcAttachmentInt, err := strconv.ParseUint(vpcAttachment, 16, 16) - if err != nil { - return "", fmt.Errorf("invalid vpcAttachment %q: %w", vpcAttachment, err) - } - - binary.BigEndian.PutUint64(ip[8:16], (vpcInt<<16)|vpcAttachmentInt) - return ip.String(), nil -} - -func HexToBase62(value string) (string, error) { - return baseconv.Convert(strings.ToLower(value), baseconv.DigitsHex, baseconv.Digits62) -} - -func Base62ToHex(value string) (string, error) { - return baseconv.Convert(value, baseconv.Digits62, baseconv.DigitsHex) -} diff --git a/pkg/common/util/util_test.go b/pkg/common/util/util_test.go deleted file mode 100644 index 4108978..0000000 --- a/pkg/common/util/util_test.go +++ /dev/null @@ -1,84 +0,0 @@ -// Copyright 2025 Datum Cloud, Inc. -// -// SPDX-License-Identifier: AGPL-3.0-or-later - -package util_test - -import ( - "net" - "reflect" - "testing" - - "go.datum.net/galactic/pkg/common/util" -) - -const ( - testVPC = "0000000jU" // 1234 dec - testVPCAttachment = "00G" // 42 dec - testIPv6Segment = "2607:ed40:ff00::1" -) - -func TestGenerateInterfaceNameVRF(t *testing.T) { - vpc := testVPC - vpcattachment := testVPCAttachment - expected := "G0000000jU00GV" - got := util.GenerateInterfaceNameVRF(vpc, vpcattachment) - if got != expected { - t.Errorf("GenerateInterfaceNameVRF(%s, %s) = %s, want %s", vpc, vpcattachment, got, expected) - } -} - -func TestGenerateInterfaceNameHost(t *testing.T) { - vpc := testVPC - vpcattachment := testVPCAttachment - expected := "G0000000jU00GH" - got := util.GenerateInterfaceNameHost(vpc, vpcattachment) - if got != expected { - t.Errorf("GenerateInterfaceNameHost(%s, %s) = %s, want %s", vpc, vpcattachment, got, expected) - } -} - -func TestGenerateInterfaceNameGuest(t *testing.T) { - vpc := testVPC - vpcattachment := testVPCAttachment - expected := "G0000000jU00GG" - got := util.GenerateInterfaceNameGuest(vpc, vpcattachment) - if got != expected { - t.Errorf("GenerateInterfaceNameGuest(%s, %s) = %s, want %s", vpc, vpcattachment, got, expected) - } -} - -func TestParseIP(t *testing.T) { - tests := []struct { - name string - input string - wantIP net.IP - wantError bool - }{ - {"ValidIPv4", "192.168.0.1", net.ParseIP("192.168.0.1"), false}, - {"ValidIPv6", testIPv6Segment, net.ParseIP(testIPv6Segment), false}, - {"InvalidIP", "not_an_ip", nil, true}, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - got, err := util.ParseIP(tt.input) - if (err != nil) != tt.wantError { - t.Errorf("ParseIP() error = %v, wantError = %v", err, tt.wantError) - } - if !reflect.DeepEqual(got, tt.wantIP) { - t.Errorf("ParseIP() got = %v, want = %v", got, tt.wantIP) - } - }) - } -} - -func TestDecodeSRv6Endpoint(t *testing.T) { - srv6Endpoint := "2607:ed40:ff00::0000:0000:04d2:002a" - vpc := "0000000004d2" - vpcAttachment := "002a" - gotVpc, gotVpcAttachment, _ := util.DecodeSRv6Endpoint(net.ParseIP(srv6Endpoint)) - if gotVpc != vpc || gotVpcAttachment != vpcAttachment { - t.Errorf("DecodeSRv6Endpoint(%s) = %s, %s, want %s, %s", srv6Endpoint, gotVpc, gotVpcAttachment, vpc, vpcAttachment) - } -} diff --git a/scripts/ci.sh b/scripts/ci.sh index 7653ffb..5eb14c3 100755 --- a/scripts/ci.sh +++ b/scripts/ci.sh @@ -6,7 +6,7 @@ COMMAND="${1:-}" case "$COMMAND" in unittest) echo "--- Running Go unit tests" - go test -v -race -coverprofile=coverage.out ./pkg/common/util/... + go test -v -race -coverprofile=coverage.out ./internal/... ;; e2etest) @@ -15,10 +15,12 @@ case "$COMMAND" in trap 'kind delete cluster --name "$CLUSTER_NAME"' EXIT - echo "--- Loading kernel modules required by galactic" - sudo apt-get update -qq - sudo apt-get install -y --no-install-recommends linux-modules-extra-azure - sudo modprobe vrf + if [[ "${GITHUB_ACTIONS:-}" == "true" ]]; then + echo "--- Loading kernel modules required by galactic" + sudo apt-get update -qq + sudo apt-get install -y --no-install-recommends linux-modules-extra-azure + sudo modprobe vrf + fi echo "--- Installing kind" go install sigs.k8s.io/kind@latest