Skip to content

OCPBUGS-93863: E2E: Fix tuned active profile check for wrapped performance profiles#1549

Open
mrniranjan wants to merge 2 commits into
openshift:mainfrom
mrniranjan:fix_tuned_profile
Open

OCPBUGS-93863: E2E: Fix tuned active profile check for wrapped performance profiles#1549
mrniranjan wants to merge 2 commits into
openshift:mainfrom
mrniranjan:fix_tuned_profile

Conversation

@mrniranjan

@mrniranjan mrniranjan commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Test 37127 read /etc/tuned/active_profile on the node and expected the performance profile name. That fails when a higher-priority Tuned CR wraps the PP via include= (e.g. ran-du-performance), because the active profile name is the wrapper, not the PP name.

Detect wrapper profiles by scanning Tuned CRs for include= and assert Profile.status.tunedProfile via the API instead of pod exec.

Summary by CodeRabbit

  • Tests
    • Strengthened end-to-end performance profile checks by validating each node’s active Tuned profile via Tuned status rather than reading files from Tuned pods.
    • Improved verification of profile relationships by resolving and walking Tuned include chains to confirm the active profile reaches the expected performance profile, including cycle detection and bounded traversal.
    • Enhanced discovery of include targets by parsing profile content as INI and ignoring non-effective entries.

@coderabbitai

coderabbitai Bot commented Jun 18, 2026

Copy link
Copy Markdown

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

The performanceprofile e2e test now reads the active Tuned profile from node status, parses Tuned include= data, and validates that the active profile reaches the expected performance profile through include chains.

Changes

Tuned Profile Verification Updates

Layer / File(s) Summary
Status-based profile check
test/e2e/performanceprofile/functests/1_performance/performance.go
The test reads tunedv1.Profile.Status.TunedProfile and validates the expected performance profile through include-chain reachability.
Include parsing and traversal helpers
test/e2e/performanceprofile/functests/1_performance/performance.go
New helpers parse Tuned profile INI data, collect [main].include entries, build an include map, and walk include chains with cycle detection and bounded hops.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes


Important

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

❌ Failed checks (1 error)

Check name Status Explanation Resolution
No-Sensitive-Data-In-Logs ❌ Error The new node/profile checks log node.Name in By(...) and testlog.Infof(...), which can expose internal hostnames in test output. Remove or mask node names from logs/messages, or gate them behind debug-only output; keep only non-sensitive identifiers.
✅ Passed checks (14 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed PASS: The relevant Ginkgo title, [test_id:37127] Node should point to right tuned profile, is static and contains no dynamic run-specific data.
Test Structure And Quality ✅ Passed PASS: The new It block has one behavior, no resource creation/cleanup needed, uses bounded Eventually timeout/polling, includes failure messages, and matches repo client/wait patterns.
Microshift Test Compatibility ✅ Passed New test uses only Tuned/PerformanceProfile CRDs and no MicroShift-blocked APIs, namespaces, tags, or HA assumptions were found.
Single Node Openshift (Sno) Test Compatibility ✅ Passed The new test only reads Tuned status via API and walks include chains; it doesn’t require multiple nodes, HA failover, or node-to-node scheduling.
Topology-Aware Scheduling Compatibility ✅ Passed Change is confined to an e2e performance test; no deployment manifests, controllers, or scheduling constraints were modified.
Ote Binary Stdout Contract ✅ Passed New code only logs inside It blocks; no new process-level stdout writes were added, and suite setup already routes controller-runtime logs to stderr.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed The new Ginkgo test uses cluster APIs and node exec only; no IPv4 literals, IP parsing, or external/public network calls were found.
No-Weak-Crypto ✅ Passed PASS: The modified test only adds Tuned profile parsing/chain traversal; no weak crypto, custom crypto, or secret/token comparisons are present.
Container-Privileges ✅ Passed PASS: The PR only changes test logic in performance.go; no container manifests or privileged/securityContext fields were added.
Title check ✅ Passed The title clearly matches the main change: fixing the E2E tuned active profile check for wrapped performance profiles.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@openshift-ci openshift-ci Bot requested review from ffromani and jmencak June 18, 2026 12:51

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/e2e/performanceprofile/functests/1_performance/performance.go`:
- Around line 1430-1439: The loop that returns the first profile containing
include=<performanceProfileName> does not account for Tuned recommendation
semantics and can return an incorrect profile when multiple Tuned CRs include
the performance profile. Instead of returning on the first include= match found
via strings.Contains, resolve the wrapper profile by consulting Tuned's
recommendation selection logic (priority and match criteria) to identify the
active/effective recommended profile. Alternatively, detect when multiple
candidate profiles exist that contain the include match and fail fast with a
clear error rather than returning an ambiguous result.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: ee8fdbda-b6f7-416d-a49b-3d1cfb261e56

📥 Commits

Reviewing files that changed from the base of the PR and between 8978bc5 and 6cd6d9d.

📒 Files selected for processing (1)
  • test/e2e/performanceprofile/functests/1_performance/performance.go

Comment thread test/e2e/performanceprofile/functests/1_performance/performance.go
@mrniranjan

Copy link
Copy Markdown
Contributor Author

/retest-required

1 similar comment
@mrniranjan

Copy link
Copy Markdown
Contributor Author

/retest-required

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
test/e2e/performanceprofile/functests/1_performance/performance.go (1)

1488-1504: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Include-chain walk only follows the first branch; misses targets reachable via other includes.

The direct-include check at Line 1480-1486 inspects all includedProfiles, but the next-hop selection at Line 1488-1495 picks only the first include that exists in includesMap and commits to that single path. If a profile includes multiple parents (e.g. wrapper includes [other, mid] and only mid transitively includes the performance profile), the walk descends into other, hits a dead end, and returns false — a false negative that fails the test even though the active profile does trace back to the performance profile. This should explore all branches (BFS/DFS) rather than a single linear chain.

♻️ Proposed traversal over all branches
-	// Walk the include chain, starting from startProfile
-	current := startProfile
-	visited := make(map[string]bool) // Prevent infinite loops
-	maxHops := len(includesMap)
-	chain := []string{startProfile} // Track the chain for logging
-
-	for hop := 0; hop < maxHops; hop++ {
-		// Found it?
-		if current == targetProfile {
-			testlog.Infof("Profile chain validated: %s", strings.Join(chain, " → "))
-			return true
-		}
-
-		// Already visited? (cycle detection)
-		if visited[current] {
-			testlog.Warningf("Cycle detected in profile chain at %q", current)
-			return false
-		}
-		visited[current] = true
-
-		// What does the current profile include?
-		includedProfiles, exists := includesMap[current]
-		if !exists {
-			testlog.Warningf("Profile %q not found in cluster", current)
-			return false // Profile not found
-		}
-
-		// Check if it directly includes the target
-		for _, included := range includedProfiles {
-			if included == targetProfile {
-				chain = append(chain, targetProfile)
-				testlog.Infof("Profile chain validated: %s", strings.Join(chain, " → "))
-				return true
-			}
-		}
-
-		// Find the next profile in the chain (first include that exists in our map)
-		nextProfile := ""
-		for _, included := range includedProfiles {
-			if _, exists := includesMap[included]; exists {
-				nextProfile = included
-				break
-			}
-		}
-
-		if nextProfile == "" {
-			testlog.Warningf("Dead end in profile chain at %q (no valid includes found)", current)
-			return false // Dead end - no more includes
-		}
-
-		// Move to the next profile in the chain
-		chain = append(chain, nextProfile)
-		current = nextProfile
-	}
-
-	// Shouldn't reach here unless chain is unreasonably deep
-	testlog.Warningf("Profile chain exceeded maximum depth of %d hops", maxHops)
-	return false
+	// DFS over the whole include graph (all branches), with cycle detection.
+	visited := make(map[string]bool)
+	var reachable func(current string) bool
+	reachable = func(current string) bool {
+		if current == targetProfile {
+			return true
+		}
+		if visited[current] {
+			return false
+		}
+		visited[current] = true
+		for _, included := range includesMap[current] {
+			if reachable(included) {
+				return true
+			}
+		}
+		return false
+	}
+	return reachable(startProfile)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/e2e/performanceprofile/functests/1_performance/performance.go` around
lines 1488 - 1504, The include-chain traversal in the performance profile check
is too greedy: the next-hop logic in the chain walk only follows the first
matching entry from includedProfiles, which can miss valid ancestors reachable
through other includes. Update the walk in the profile-chain helper to explore
all branches from each profile (for example with DFS or BFS and a visited set)
instead of committing to a single nextProfile, so profiles like wrapper can
still resolve through mid even if another include is a dead end.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Duplicate comments:
In `@test/e2e/performanceprofile/functests/1_performance/performance.go`:
- Around line 1488-1504: The include-chain traversal in the performance profile
check is too greedy: the next-hop logic in the chain walk only follows the first
matching entry from includedProfiles, which can miss valid ancestors reachable
through other includes. Update the walk in the profile-chain helper to explore
all branches from each profile (for example with DFS or BFS and a visited set)
instead of committing to a single nextProfile, so profiles like wrapper can
still resolve through mid even if another include is a dead end.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: b3645597-1666-42ee-85ed-6805f360cd7f

📥 Commits

Reviewing files that changed from the base of the PR and between 6cd6d9d and 1d3bed3.

📒 Files selected for processing (1)
  • test/e2e/performanceprofile/functests/1_performance/performance.go

Test 37127 read /etc/tuned/active_profile on the node and expected the
performance profile name. That fails when a higher-priority Tuned CR wraps
the PP via include=<pp-profile> (e.g. ran-du-performance), because the
active profile name is the wrapper, not the PP name.

Detect wrapper profiles by scanning Tuned CRs for include=<pp-profile>
and assert Profile.status.tunedProfile via the API instead of pod exec.

Traverse the tuned profile chain

In this commit we are traversing the Tuned profile that contains
includes= and verifying if we can traverse back to Performance Profile

AI Attribution: AIA Human-AI blend, New content, Human-initiated, Reviewed, Cursor 3.0.16 v1.0

Signed-off-by: Niranjan M.R <mniranja@redhat.com>
@mrniranjan

Copy link
Copy Markdown
Contributor Author

/retest-required

@mrniranjan

Copy link
Copy Markdown
Contributor Author

/test e2e-aws-ovn-techpreview

1 similar comment
@mrniranjan

Copy link
Copy Markdown
Contributor Author

/test e2e-aws-ovn-techpreview

func extractIncludedProfiles(profileData string) []string {
cfg, err := ini.Load([]byte(profileData))
if err != nil || cfg == nil {
// Invalid INI data - return empty list

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add log here to make it more clear at what part in the flow the code failed

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in latest commit


// Check if the [main] section has an "include" key
if !cfg.Section("main").HasKey("include") {
return []string{}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in latest commit

// Get the include value
includeValue := cfg.Section("main").Key("include").String()
if includeValue == "" {
return []string{}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto, add log line

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in latest commit

// Check if this node's active profile traces back to the performance profile
tracesToPerformanceProfile := profileIncludesTarget(activeProfile, performanceProfileName, tunedList)

g.Expect(tracesToPerformanceProfile).To(BeTrue(),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

true or false gives us minimum information about the progress of profileIncludesTarget function.
IOW, in case that it return false we have no idea if the node's active profile is not tracing back to a performance profile or something failed down the stack during the execution.

I would say it's is better to also return an error that elaborate in case of failure.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in latest commit


// Get all Tuned profiles once (for lookup)
tunedList := &tunedv1.TunedList{}
Expect(testclient.ControlPlaneClient.List(ctx, tunedList)).To(Succeed(),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tuned objects are on the DataPlaneClient on Hypershift.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in latest commit

- Fix Hypershift compatibility: Use DataPlaneClient for listing Tuned CRs
  On Hypershift clusters, tuned objects are on the data plane, not control plane.

- Add error handling: Change function signatures to return (bool, error)
  Return errors for unexpected conditions (cycles, missing profiles, parsing failures)
  while returning (false, nil) for valid "not found" cases.

  Error cases:
  * Cycle detected in profile chain
  * Profile referenced but not found in cluster
  * Chain exceeded maximum depth
  * INI parsing failed

  Valid "not found" case:
  * Profile chain ends without reaching target (false, nil)

- Add logging for better observability:
  * Log when INI parsing fails with the error details
  * Log when profiles have no 'include' key
  * Log when 'include' value is empty
  * Changed dead-end from Warning to Info (it's a normal case)

@Tal-or Tal-or left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
Thanks!

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jun 29, 2026
@openshift-ci

openshift-ci Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: mrniranjan, Tal-or
Once this PR has been reviewed and has the lgtm label, please assign yanirq for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@mrniranjan mrniranjan changed the title E2E: Fix tuned active profile check for wrapped performance profiles OCPBUGS-93863: E2E: Fix tuned active profile check for wrapped performance profiles Jun 29, 2026
@openshift-ci-robot openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. labels Jun 29, 2026
@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@mrniranjan: This pull request references Jira Issue OCPBUGS-93863, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (5.0.0) matches configured target version for branch (5.0.0)
  • bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Test 37127 read /etc/tuned/active_profile on the node and expected the performance profile name. That fails when a higher-priority Tuned CR wraps the PP via include= (e.g. ran-du-performance), because the active profile name is the wrapper, not the PP name.

Detect wrapper profiles by scanning Tuned CRs for include= and assert Profile.status.tunedProfile via the API instead of pod exec.

Summary by CodeRabbit

  • Tests
  • Strengthened end-to-end performance profile checks by validating each node’s active Tuned profile via Tuned status rather than reading files from Tuned pods.
  • Improved verification of profile relationships by resolving and walking Tuned include chains to confirm the active profile reaches the expected performance profile, including cycle detection and bounded traversal.
  • Enhanced discovery of include targets by parsing profile content as INI and ignoring non-effective entries.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@mrniranjan

Copy link
Copy Markdown
Contributor Author

/verified by @mrniranjan

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Jun 29, 2026
@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@mrniranjan: This PR has been marked as verified by @mrniranjan.

Details

In response to this:

/verified by @mrniranjan

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci

openshift-ci Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

@mrniranjan: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-ovn d8e9852 link true /test e2e-aws-ovn

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants