Skip to content

fix: reset TargetForConfig condition on no-op reconcile#464

Open
steiler wants to merge 2 commits into
sensitivefrom
fix/targetconfig-missing-condition-reset-on-noop
Open

fix: reset TargetForConfig condition on no-op reconcile#464
steiler wants to merge 2 commits into
sensitivefrom
fix/targetconfig-missing-condition-reset-on-noop

Conversation

@steiler

@steiler steiler commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Problem

When a target briefly drops (e.g. srl3 loses its dsctx) the TargetConfigController calls SetConfigsTargetConditionForTarget(TargetForConfigFailed), setting all Configs for that target to Ready=False.

When the target recovers, the reconciler runs buildIntentInputs. If no intent hash has changed (no new configs, no secret rotation) hasChanged=false and the reconciler returned early:

if !hasChanged {
    log.Info("Transact skip, nothing to update")
    return ctrl.Result{}, nil   // ← never resets the stale Failed condition
}

SetConfigsTargetConditionForTarget(TargetForConfigReady) is only called after a successful transaction, so the stale Failed condition was never cleared. Configs stayed at Ready=False permanently until the next unrelated change triggered a new transaction.

This caused the 22-Srl-Update-Replace suite in CI run #460 to fail: srl3 dropped at 07:14:41, no new intents existed to force a transaction, and the setup Config Check Ready timed out 2 minutes later still seeing ['False'].

Fix

Mirror the main-branch behaviour: call SetConfigsTargetConditionForTarget(TargetForConfigReady) in the no-change early-return path so any stale Failed condition is always cleared when the target is confirmed healthy, even when nothing needs to be transacted.

if !hasChanged {
    log.Info("Transact skip, nothing to update")
    if err := r.cfgMgr.SetConfigsTargetConditionForTarget(
        ctx, targetOrig,
        configv1alpha1.TargetForConfigReady("target ready"),
    ); err != nil {
        return ctrl.Result{},
            errors.Wrap(r.handleError(ctx, targetOrig, "cannot update config status", err), errUpdateStatus)
    }
    return ctrl.Result{}, nil
}

Test plan

  • Re-run the sensitive CI suite — 22-Srl-Update-Replace should pass
  • Confirm no regression in 11-Sros-Create-Delete, 12-Srl-Create-Delete, 21-Sros-Update-Replace
  • Verify that deliberately taking a target not-ready and back-ready (no config change) transitions Configs back to Ready=True

Pairs-with: sdcio/integration-tests#108
Pairs-with: sdcio/data-server#460

@steiler steiler requested a review from a team as a code owner July 6, 2026 10:14
When a target briefly drops and recovers but no intent hash changes,
buildIntentInputs returns hasChanged=false and the reconciler returned
early without calling SetConfigsTargetConditionForTarget. Any stale
TargetForConfig=Failed condition written during the outage was never
cleared, leaving all Config CRs permanently at Ready=False.

Mirror the main-branch behaviour: always call
SetConfigsTargetConditionForTarget(TargetForConfigReady) at the end of
every successful reconcile, even when no transaction was needed.

Fixes 22-Srl-Update-Replace CI failures where srl3 briefly lost its
dsctx at 07:14:41, setting configs to Ready=False, which were never
restored because subsequent reconciles detected no hash change and
skipped the condition-reset call.

Co-authored-by: Cursor <cursoragent@cursor.com>
@steiler steiler force-pushed the fix/targetconfig-missing-condition-reset-on-noop branch from c299691 to 1e9e23c Compare July 6, 2026 11:07
…cret re-creation

When a referenced secret is deleted, the configresolver sets Resolver=False
and preserves the last-good SensitiveConfig. If the secret is re-created with
identical content, detectChange returns noChange=true because all hashes still
match — causing the reconciler to exit early without ever clearing the stale
failure condition. The Config would remain non-Ready permanently.

Fix: before early-exiting on noChange, check whether the current Resolver
condition is False. If it is, call handleSuccess to clear it — the secret is
clearly available now (hashes matched), so the stored failure is no longer valid.

Fixes TC5 in the 04-sensitive integration test suite.

Co-authored-by: Cursor <cursoragent@cursor.com>
steiler added a commit to sdcio/integration-tests that referenced this pull request Jul 6, 2026
…ow deviation detection

The intent was sent to the data-server as revertive (default). The data-server's
active sync cycle continuously restores the intended value to the device every
~5 seconds, overwriting any manually injected deviation before the 10-second
deviation-manager poll can observe it. As a result, TC4 never saw devs>0.

Setting revertive: false disables auto-correction so a gnmic-injected deviation
persists long enough to be detected and reported back to Kubernetes.

Pairs-with: sdcio/config-server#464
Co-authored-by: Cursor <cursoragent@cursor.com>
@henderiw

henderiw commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

If the status is failed we need to retry iso setting condition to true in my view otherwise we might miss out some configs

@steiler

steiler commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

what does that mean? do you want to adjust this PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants