Skip to content

OCPBUGS-74223: Re-order logger setting to eliminate race condition#5691

Open
isabella-janssen wants to merge 1 commit intoopenshift:mainfrom
isabella-janssen:ocpbugs-74223
Open

OCPBUGS-74223: Re-order logger setting to eliminate race condition#5691
isabella-janssen wants to merge 1 commit intoopenshift:mainfrom
isabella-janssen:ocpbugs-74223

Conversation

@isabella-janssen
Copy link
Member

@isabella-janssen isabella-janssen commented Feb 24, 2026

Closes: OCPBUGS-74223

- What I did
The moves the logger initialization before any controller-runtime code runs to eliminate the race condition intermittently occurring as described in the Jira ticket.

- How to verify it
We should no longer see the [controller-runtime] log.SetLogger(...) was never called; logs will not be displayed. error when observing machine-config-server logs.

- Description for the changelog
OCPBUGS-74223: Re-order logger setting to eliminate race condition

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 24, 2026
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Feb 24, 2026

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 24, 2026
@isabella-janssen isabella-janssen changed the title (WIP) OCPBUGS-74223: Re-order logger setting to eliminate race condition Feb 27, 2026
@openshift-ci-robot openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Feb 27, 2026
@openshift-ci-robot
Copy link
Contributor

@isabella-janssen: This pull request references Jira Issue OCPBUGS-74223, which is invalid:

  • expected the bug to target the "4.22.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

- What I did

- How to verify it

- Description for the changelog

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@isabella-janssen
Copy link
Member Author

/jira refresh

@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Feb 27, 2026
@openshift-ci-robot
Copy link
Contributor

@isabella-janssen: This pull request references Jira Issue OCPBUGS-74223, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.22.0) matches configured target version for branch (4.22.0)
  • bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @sergiordlr

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot requested a review from sergiordlr February 27, 2026 20:06
@isabella-janssen isabella-janssen marked this pull request as ready for review February 27, 2026 21:24
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 27, 2026
@pablintino
Copy link
Contributor

/lgtm
/retest-required

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Mar 2, 2026
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Mar 2, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: isabella-janssen, pablintino

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [isabella-janssen,pablintino]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ptalgulk01
Copy link

ptalgulk01 commented Mar 13, 2026

Pre-merge testing:

Environment Setup:
OCP Version: 4.22.0-0-2026-03-13-090934-test-ci-ln-c6iti2b-latest
Platform: AWS

Steps:

  • Checked the MCS Logs after installation
oc logs machine-config-server-wg5tv | grep -i logger && oc logs machine-config-server-lj7vh | grep -i logger && oc logs machine-config-server-6m8rc | grep -i logger
I0313 09:32:21.838159       1 certwatcher.go:211] "Updated current TLS certificate" logger="controller-runtime.certwatcher"
I0313 09:32:21.838307       1 certwatcher.go:133] "Starting certificate poll+watcher" logger="controller-runtime.certwatcher" interval="10s"
I0313 09:32:23.954383       1 certwatcher.go:211] "Updated current TLS certificate" logger="controller-runtime.certwatcher"
I0313 09:32:23.954603       1 certwatcher.go:133] "Starting certificate poll+watcher" logger="controller-runtime.certwatcher" interval="10s"
I0313 09:32:25.743421       1 certwatcher.go:211] "Updated current TLS certificate" logger="controller-runtime.certwatcher"
I0313 09:32:25.743607       1 certwatcher.go:133] "Starting certificate poll+watcher" logger="controller-runtime.certwatcher" interval="10s"
  • Tried to delete the MCS pod and monitored the logs able to see the same result like above
  • Even after applying the MC showing the same result
  • Above logs dont show the unexpected logs.

@ptalgulk01
Copy link

/label qe-approved
/verified by @ptalgulk01

@openshift-ci openshift-ci bot added the qe-approved Signifies that QE has signed off on this PR label Mar 13, 2026
@openshift-ci-robot
Copy link
Contributor

@isabella-janssen: This pull request references Jira Issue OCPBUGS-74223, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.22.0) matches configured target version for branch (4.22.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @sergiordlr

Details

In response to this:

Closes: OCPBUGS-74223

- What I did
The moves the logger initialization before any controller-runtime code runs to eliminate the race condition intermittently occurring as described in the Jira ticket.

- How to verify it
We should no longer see the [controller-runtime] log.SetLogger(...) was never called; logs will not be displayed. error when observing machine-config-server logs.

- Description for the changelog
OCPBUGS-74223: Re-order logger setting to eliminate race condition

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Mar 13, 2026
@openshift-ci-robot
Copy link
Contributor

@ptalgulk01: This PR has been marked as verified by @ptalgulk01.

Details

In response to this:

/label qe-approved
/verified by @ptalgulk01

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@isabella-janssen
Copy link
Member Author

/cherrypick release-4.21 release-4.20 release-4.19

@openshift-cherrypick-robot

@isabella-janssen: once the present PR merges, I will cherry-pick it on top of release-4.21 in a new PR and assign it to you.

Details

In response to this:

/cherrypick release-4.21 release-4.20 release-4.19

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci-robot
Copy link
Contributor

/retest-required

Remaining retests: 0 against base HEAD 5f0d9d7 and 2 for PR HEAD f013cc9 in total

@isabella-janssen
Copy link
Member Author

/test unit

@isabella-janssen
Copy link
Member Author

/test e2e-aws-ovn-upgrade

@openshift-ci-robot
Copy link
Contributor

/retest-required

Remaining retests: 0 against base HEAD 3474828 and 1 for PR HEAD f013cc9 in total

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Mar 14, 2026

@isabella-janssen: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. qe-approved Signifies that QE has signed off on this PR verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants