[WIP] AGENT-1449: Add IRI registry credential rotation support#1
Closed
rwsu wants to merge 2 commits intoAGENT-1449-authfrom
Closed
[WIP] AGENT-1449: Add IRI registry credential rotation support#1rwsu wants to merge 2 commits intoAGENT-1449-authfrom
rwsu wants to merge 2 commits intoAGENT-1449-authfrom
Conversation
Implement safe credential rotation for the IRI registry using a desired-vs-current pattern with generation-numbered usernames. The auth secret holds the desired password; the pull secret (read from rendered MachineConfig) holds the deployed password. When they differ, a three-phase rotation is performed: 1. Deploy dual htpasswd (old + new credentials with different usernames) 2. Update pull secret after all MCPs finish rolling out 3. Clean up dual htpasswd to single entry after new pull secret is deployed This avoids authentication deadlocks during rolling MachineConfig updates because the pull secret always contains the old credentials, which are present in every version of the htpasswd. Mid-rotation password changes are handled by verifying htpasswd hashes with bcrypt.CompareHashAndPassword and regenerating if they don't match. Key changes: - Add MachineConfigPool lister/informer to IRI controller - Add reconcileAuthCredentials with three-case rotation logic - Add getDeployedIRICredentials (reads from rendered MC, not API) - Add areAllPoolsUpdated (checks all pools including workers) - Add HtpasswdHasValidEntry, GenerateHtpasswdEntry, GenerateDualHtpasswd, NextIRIUsername, ExtractIRICredentialsFromPullSecret helpers - Vendor golang.org/x/crypto/bcrypt for htpasswd hash generation - Add credential rotation design doc Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
…ial rotation Add three new e2e tests: - TestIRIAuth_UnauthenticatedRequestReturns401: verifies registry rejects unauthenticated requests with 401 when auth is enabled - TestIRIAuth_AuthenticatedRequestSucceeds: verifies registry accepts requests with valid Basic Auth credentials - TestIRIAuth_CredentialRotation: end-to-end test of the three-phase credential rotation (dual htpasswd, pull secret update, cleanup) Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
- What I did
Implement safe credential rotation for the IRI registry using a desired-vs-current pattern with generation-numbered usernames. The auth secret holds the desired password; the pull secret (read from rendered MachineConfig) holds the deployed password. When they differ, a three-phase rotation is performed:
This avoids authentication deadlocks during rolling MachineConfig updates because the pull secret always contains the old credentials, which are present in every version of the htpasswd. Mid-rotation password changes are handled by verifying htpasswd hashes with bcrypt.CompareHashAndPassword and regenerating if they don't match.
Key changes:
- How to verify it
Update the password to trigger the rotation to start:
Verify the /etc/iri-registry/auth/htpasswd has been updated.
Verify iri-registry works for both new and old credentials during rollout.
Verify global pull-secret contains the new credentials after rollout is complete.
- Description for the changelog
Add credential rotation support for the IRI registry. When the auth secret's password field is updated, the controller performs a three-phase rotation: (1) deploys a dual htpasswd with both old and new credentials so all nodes accept both passwords during rollout, (2) updates the global pull secret with the new credentials after all MachineConfigPools are fully updated, and (3) cleans up the dual htpasswd to a single entry once the new credentials are deployed everywhere. This avoids authentication deadlocks caused by api-int load-balancing requests across master nodes that may be at different stages of the rollout.
Also adds e2e tests for registry authentication (401 on unauthenticated requests, 200 with valid credentials) and an end-to-end credential rotation test that exercises all three phases.
Depends on openshift#5765.