Conversation
added 3 commits
April 13, 2026 12:45
…er initial deploy When additional_ports (e.g., gRPC) are added to a scope after the initial deployment, blue-green switch-traffic fails because the ingress template references K8s Services for the blue deployment's additional ports that were never created. This causes FailedBuildModel in the ALB Ingress Controller and a 120s timeout. The fix detects whether the blue deployment's additional_port K8s Services exist during build_context and passes that info to the template. When a blue service is missing, the ingress renders a single-target forward action (100% to the green deployment) instead of a dual-target action that would reference the non-existent service.
…s without blue service When additional_ports (e.g., gRPC) are added to a scope after the blue deployment was created, the ALB has listeners with single-target weights (100% green) instead of the standard blue-green split. The verify script was checking the first matching listener regardless of port, and if it hit the gRPC listener first, the weight comparison (100 vs 10/90) would fail and the script would never check the primary HTTP listener. The fix reads blue_additional_port_services from the deployment context and skips weight verification on listeners for ports where the blue deployment has no K8s service, falling through to verify the primary HTTP listener instead. Also fixes a jq bug: `false // true` returns `true` in jq because the alternative operator treats `false` as falsy. Changed to explicit `if has($k) then .[$k] else true end`.
Weights are stored with newlines for comparison but this caused the mismatch log to split across multiple lines, making deployment logs hard to read. Now formats as "expected=20/80 actual=10/90".
fedemaleh
previously approved these changes
Apr 15, 2026
fedemaleh
approved these changes
Apr 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
switch-trafficfailure whenadditional_ports(e.g., gRPC on port 9014) are added to a scope after the initial deployment was createdbuild_contextand passes a map (blue_additional_port_services) to the template contextblue-green-ingress.yaml.tplrenders a single-target ALB forward action (100% to green deployment) instead of referencing the non-existent blue serviceRoot cause
blue-green-ingress.yaml.tplunconditionally referencesd-{scope_id}-{blue_deployment_id}-{type}-{port}for the old deployment. During active blue-green workflows (blue_green.yaml,switch_traffic.yaml), service templates are only rendered for the green (new) deployment — the blue deployment's additional_port K8s Services are never re-created if they didn't exist at the time of the original deploy. This causesFailedBuildModelin the ALB Ingress Controller, blocking ingress reconciliation and timing out after 120s.Files changed
k8s/deployment/build_context— Checks if blue deployment's additional_port K8s Services exist viakubectl get serviceand injectsblue_additional_port_servicesmap into CONTEXTk8s/deployment/templates/blue-green-ingress.yaml.tpl— Conditionally renders single-target (100% green) or dual-target forward actions based on the mapk8s/deployment/tests/build_context.bats— 6 new unit tests for the service detection logicBackward compatibility
If
blue_additional_port_servicesis absent from the context (old scope agents), the template defaults to the existing dual-target behavior — no breaking change.Test plan
build_context.batstests pass (including 6 new)