Problem
The Helm chart's deployments/kubernetes/chart/reloader/templates/deployment.yaml does not render a strategy: block at all, so the Reloader Deployment always runs with Kubernetes' default RollingUpdate strategy. There is no value in values.yaml to override it.
This is normally fine, but creates a real-world problem during migrations from older Reloader versions where a known bug exists in the old version's startup path. With RollingUpdate, Kubernetes can briefly surge the old ReplicaSet up by one to maintain availability while pulling the new image — booting a fresh old-version pod that then re-triggers the very bug being fixed by the upgrade.
Concrete use case (just hit this)
We were upgrading a fleet from chart v0.0.103 (Reloader appVersion v0.0.103) to chart 2.2.11 (appVersion v1.4.16). v0.0.103 has a well-known cross-controller initialization race where a single package-level controllerInitialized bool is shared between the ConfigMap and Secret controllers — whichever finishes WaitForCacheSync first flips the flag, and remaining Add events from the other controller are then treated as real creates and trigger doRollingUpgrade unconditionally. Cascade-restarts every annotated workload in the namespace.
The race is fixed cleanly in v1.4.16 (separate per-resource flags, Add reloads gated behind ReloadOnCreate). But during the rolling chart upgrade, Kubernetes scaled the old ReplicaSet up by one to maintain availability, booting a new v0.0.103 pod which immediately hit the race and cascaded restarts of 38+ workloads — exactly the bug the upgrade was meant to fix.
Events confirming the timeline:
11:34:23Z ScalingReplicaSet Scaled up replica set <old-rs> to 2 from 1
11:34:23Z ScalingReplicaSet Scaled up replica set <new-rs> to 1
11:34:25Z Started container reloader (old-image v0.0.103 pod)
11:34:25Z <first cascade annotation observed on a watched workload>
11:34:33Z Started container reloader (new-image v1.4.16 pod) — 8s late
For the chart upgrade specifically, we want strategy.type: Recreate so Kubernetes kills all existing pods before creating any new ones — no surge means no new old-version pod ever runs during the transition.
Workaround we shipped
We vendored the chart and patched deployment.yaml:
spec:
{{- with .Values.reloader.deployment.strategy }}
strategy:
{{- toYaml . | nindent 4 }}
{{- end }}
with reloader.deployment.strategy: { type: Recreate } in values. Worked end-to-end across our test cluster.
Proposed change
Add a strategy: value to values.yaml under reloader.deployment and render it in the deployment template only when set. Backwards-compatible — default behavior (RollingUpdate) is unchanged when the value is unset.
templates/deployment.yaml — add right after the spec: line:
spec:
{{- with .Values.reloader.deployment.strategy }}
strategy:
{{- toYaml . | nindent 4 }}
{{- end }}
values.yaml — add under reloader.deployment:
# Deployment update strategy. If unset, Kubernetes default (RollingUpdate) is used.
# Example:
# strategy:
# type: Recreate
#
# strategy:
# type: RollingUpdate
# rollingUpdate:
# maxSurge: 0
# maxUnavailable: 1
strategy: {}
Mirrors how the chart already exposes dnsConfig, topologySpreadConstraints, and similar — same pattern, same idiom.
Why this matters beyond the v0.0.103 migration
- Operators recovering from any future startup-time regression have a knob to disable surge during the upgrade.
- Air-gapped or resource-constrained clusters that don't want headroom for a surge pod (
maxSurge: 0) can express that without a downstream fork.
- Common Helm-chart hygiene — almost every major operator chart exposes update strategy as a value.
Happy to send a PR for this change if it'd help. Thanks!
Problem
The Helm chart's
deployments/kubernetes/chart/reloader/templates/deployment.yamldoes not render astrategy:block at all, so the Reloader Deployment always runs with Kubernetes' defaultRollingUpdatestrategy. There is no value invalues.yamlto override it.This is normally fine, but creates a real-world problem during migrations from older Reloader versions where a known bug exists in the old version's startup path. With
RollingUpdate, Kubernetes can briefly surge the old ReplicaSet up by one to maintain availability while pulling the new image — booting a fresh old-version pod that then re-triggers the very bug being fixed by the upgrade.Concrete use case (just hit this)
We were upgrading a fleet from chart
v0.0.103(Reloader appVersionv0.0.103) to chart2.2.11(appVersionv1.4.16). v0.0.103 has a well-known cross-controller initialization race where a single package-levelcontrollerInitializedbool is shared between the ConfigMap and Secret controllers — whichever finishesWaitForCacheSyncfirst flips the flag, and remaining Add events from the other controller are then treated as real creates and triggerdoRollingUpgradeunconditionally. Cascade-restarts every annotated workload in the namespace.The race is fixed cleanly in v1.4.16 (separate per-resource flags,
Addreloads gated behindReloadOnCreate). But during the rolling chart upgrade, Kubernetes scaled the old ReplicaSet up by one to maintain availability, booting a new v0.0.103 pod which immediately hit the race and cascaded restarts of 38+ workloads — exactly the bug the upgrade was meant to fix.Events confirming the timeline:
For the chart upgrade specifically, we want
strategy.type: Recreateso Kubernetes kills all existing pods before creating any new ones — no surge means no new old-version pod ever runs during the transition.Workaround we shipped
We vendored the chart and patched
deployment.yaml:with
reloader.deployment.strategy: { type: Recreate }in values. Worked end-to-end across our test cluster.Proposed change
Add a
strategy:value tovalues.yamlunderreloader.deploymentand render it in the deployment template only when set. Backwards-compatible — default behavior (RollingUpdate) is unchanged when the value is unset.templates/deployment.yaml— add right after thespec:line:values.yaml— add underreloader.deployment:Mirrors how the chart already exposes
dnsConfig,topologySpreadConstraints, and similar — same pattern, same idiom.Why this matters beyond the v0.0.103 migration
maxSurge: 0) can express that without a downstream fork.Happy to send a PR for this change if it'd help. Thanks!