Skip to content

feat(loadbalancers): new selector style annotation to assoc LB with VPC/PN#218

Open
andreaswachs wants to merge 4 commits intoscaleway:masterfrom
kommodity-io:feature/selector-style-annotaion-for-lb-pn-binding
Open

feat(loadbalancers): new selector style annotation to assoc LB with VPC/PN#218
andreaswachs wants to merge 4 commits intoscaleway:masterfrom
kommodity-io:feature/selector-style-annotaion-for-lb-pn-binding

Conversation

@andreaswachs
Copy link
Contributor

Changes

  • Adds support for a new annotation on Kubernetes services of type LoadBalancer named service.beta.kubernetes.io/scw-loadbalancer-pn-names in order to make GitOps easier by using pairs of <VPC-name>/<PN-name>[,<...>/<...>] as opposed to using service.beta.kubernetes.io/scw-loadbalancer-pn-ids that might change over time or adds a second step of configuration when deploying clusters

Backwards compatiblity

Since there are already settings for getting the LB associated with a specific PN, we put the priority as such for acting on specific configuration if multiple are present:
pn-ids (annotation) > pn-names (annotation) > PN_ID env var.

Key details

  • Name format: vpc-name/pn-name required to avoid ambiguity across VPCs. Comma-separated for multiple networks.
  • Project scoping: VPC/PN lookups scoped to the project where cluster nodes run (derived via Instance API), preventing cross-project name collisions.
  • Backend IP resolution: Uses precise IPAM-based lookup via getNodeIPsForPrivateNetworks — queries each node's private NICs and resolves IPs through IPAM rather than relying on extractNodesInternalIps which may return IPs from unrelated private networks.
  • Private LB guard: Updated to accept pn-names (and pn-ids) so private LBs no longer require the PN_ID env var when annotations are set.
  • Error handling: Clear errors for missing/duplicate VPCs and private networks.

This has been tested to work in a self-managed Kubernetes cluster running on Scaleway VMs.

steffen-karlsson and others added 4 commits January 27, 2026 22:29
… without pnNIC (#1)

Signed-off-by: Steffen Karlsson <steffen.karlsson@gmail.com>
* feat(instance): avoid missing Node Refs for Private Network Instances without pnNIC

Signed-off-by: Steffen Karlsson <steffen.karlsson@gmail.com>

* fix(instances,lb): propagate IPAM errors and harden backend pool updates (scaleway#213)

Prevent nodes from being initialized without InternalIPs when IPAM
queries fail transiently. By returning errors instead of only logging
warnings, the node keeps its uninitialized taint and Kubernetes retries
until IPAM is available.

Additionally, use internal IPs when the service annotation
scw-loadbalancer-pn-ids is set (not just the global PN_ID env var),
and refuse to clear existing LB backends when no replacement IPs are
found.

Co-authored-by: Nicklas Frahm <nfm@corti.ai>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* feat: env to disable tags sync from ccm

* fix(examples): update container image for CCM

Signed-off-by: Nicklas Frahm <nfm@corti.ai>

* Update Scaleway CCM image version

Signed-off-by: Andreas Wachs <awa@corti.ai>

* Update Scaleway CCM image version to v0.36.0-kommodity.3

Signed-off-by: Andreas Wachs <awa@corti.ai>

* WIP

Signed-off-by: Andreas Wachs <awa@corti.ai>

* Update k8s-scaleway-ccm-latest.yml

Signed-off-by: Andreas Wachs <awa@corti.ai>

---------

Signed-off-by: Steffen Karlsson <steffen.karlsson@gmail.com>
Signed-off-by: Nicklas Frahm <nfm@corti.ai>
Signed-off-by: Andreas Wachs <awa@corti.ai>
Co-authored-by: Steffen Karlsson <steffen.karlsson@gmail.com>
Co-authored-by: Jérémy THERIN <jtherin@users.noreply.github.com>
Co-authored-by: Nicklas Frahm <nfm@corti.ai>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Jeremy THERIN <jtherin@scaleway.com>
feat(sync): align with upstream
…ssoc

Signed-off-by: Andreas Wachs <awa@corti.ai>
@jtherin
Copy link
Member

jtherin commented Feb 23, 2026

I don't think we will accept this pr, vpc and pn names are not unique, duplicate names are allowed. this pr is too error prone

@nfm-corti
Copy link
Contributor

There is a problem though @jtherin. When provisioning a cluster using cluster API, we can't predict the UUID, which means that we would need to manually look up the ID and add it to GitOps. Currently, there is no way to get a deterministic identifier / selector for a VPC. Do you have any alternative suggestion?

As far as I know the VPC / PN name combination is unique within a project and the CCM is always linked to a single project AFAIK. Are we missing something?

@jtherin
Copy link
Member

jtherin commented Feb 23, 2026

There is a problem though @jtherin. When provisioning a cluster using cluster API, we can't predict the UUID, which means that we would need to manually look up the ID and add it to GitOps.

Kapsule does not create any PN, you provide a PN to kapsule api, so the issue is more likely on cluster api side

As far as I know the VPC / PN name combination is unique within a project

No, you can have many pns with the same name inside the same vpc inside the same project

@nfm-corti
Copy link
Contributor

But this means that we need to pre-create the network with an external orchestration layer, which makes the process rather involved. Would it be an option to hide this behind a feature flag?

@nfm-corti
Copy link
Contributor

nfm-corti commented Feb 23, 2026

Other CCMs, such as Azure allow you to do a similar thing. I feel like having multiple non-unique VPC name / PN names should be a bug not a feature.

While not the same, Azure does allow you to associate a pre-created public IP by name using service.beta.kubernetes.io/azure-pip-name.

If you can't create a deterministic unique identifier, infrastructure is really hard to manage.

@andreaswachs
Copy link
Contributor Author

Hey @jtherin.

I hope that Nicklas comments could be taken up for debate in the team. Especially the comment on perhaps pairs of PVC name and PN names should perhaps be unique.

IF that discussion is not able to materialize or you/the team are in disagreement with us on this, can I propose that we spin this feature into selecting VPC/PN's based on tags on the PNs instead? I see that tags on Scaleway resources are supported and we could add the ability to configure this on ScalewayCluster resources.

Our goal is to simplify our cluster operations and make cluster provisioning with Cluster API more automated and I see this as another solution to the same problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants