Skip to content

OAPE-695:codecov integration changes using dockerfile and .sh file#425

Open
siddhibhor-56 wants to merge 1 commit into
openshift:masterfrom
siddhibhor-56:codecov
Open

OAPE-695:codecov integration changes using dockerfile and .sh file#425
siddhibhor-56 wants to merge 1 commit into
openshift:masterfrom
siddhibhor-56:codecov

Conversation

@siddhibhor-56
Copy link
Copy Markdown
Contributor

@siddhibhor-56 siddhibhor-56 commented May 14, 2026

Summary by CodeRabbit

  • Chores
    • Added end-to-end coverage collection workflow with automated setup and artifact collection.
    • Added a coverage-instrumented operator image and Makefile targets to build, push, and collect coverage artifacts for CI and local runs.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label May 14, 2026
@openshift-ci-robot
Copy link
Copy Markdown

openshift-ci-robot commented May 14, 2026

@siddhibhor-56: This pull request references OAPE-695 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the sub-task to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 14, 2026

Walkthrough

This PR introduces E2E coverage infrastructure for cert-manager-operator, comprising a coverage-instrumented Docker image, a bash lifecycle script for setup and collection, and Makefile targets to orchestrate the workflow. The setup phase patches the operator deployment to use the instrumented image and inject coverage environment variables; the collect phase extracts coverage data and optionally uploads it to Codecov.

Changes

E2E Coverage Infrastructure

Layer / File(s) Summary
Coverage-instrumented image and build targets
images/ci/Dockerfile.coverage, Makefile (lines 420–452)
Multi-stage Dockerfile compiles the operator binary with Go coverage flags (-cover, -covermode=count, -coverpkg=./...), prepares /tmp/e2e-cover for coverage data, and sets GOCOVERDIR. Makefile adds COVERAGE_IMG variable and image-build-coverage, image-push-coverage targets to build and push the instrumented image.
E2E coverage lifecycle script and integration
hack/e2e-coverage.sh, Makefile (e2e-coverage-collect target)
Bash script implements setup command to detect and patch the operator deployment (CSV-aware or direct), injecting coverage environment and volume mounts, and collect command to extract coverage data from the operator pod, optionally convert to text format, and upload to Codecov. Makefile e2e-coverage-collect target invokes the collect command with configurable ARTIFACT_DIR.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 9 | ❌ 3

❌ Failed checks (2 warnings, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Single Node Openshift (Sno) Test Compatibility ⚠️ Warning New Ginkgo e2e tests added without SNO protection. test/e2e/overrides_test.go has scheduling tests requiring control-plane nodes and replica tests needing multiple nodes, lacking skip mechanisms. Add [Skipped:SingleReplicaTopology] labels to multi-node tests in overrides_test.go, or add exutil.IsSingleNode() checks, or verify SNO compatibility per custom check guidance.
Title check ❓ Inconclusive The title mentions codecov integration and references the specific Jira ticket (OAPE-695), but uses vague phrasing ('changes using dockerfile and .sh file') that doesn't clearly summarize the main change. Consider revising to be more specific about the main change, e.g., 'Add E2E coverage instrumentation with codecov integration' or similar, to better communicate the primary purpose.
✅ Passed checks (9 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed PR adds three non-test files (Makefile, bash script, Dockerfile). No Ginkgo test code introduced. Check not applicable.
Test Structure And Quality ✅ Passed This PR adds codecov integration infrastructure (Makefile, shell script, Dockerfile) but no Ginkgo test files. The custom check for Ginkgo test code quality is not applicable to this PR.
Microshift Test Compatibility ✅ Passed No new Ginkgo e2e tests are added in this PR. It only adds Makefile targets, a bash script for coverage collection, and a Dockerfile. The MicroShift compatibility check does not apply.
Topology-Aware Scheduling Compatibility ✅ Passed PR introduces E2E coverage infrastructure with no topology-incompatible scheduling constraints. Changes only modify container images, environment variables, and volumes—not scheduling directives.
Ote Binary Stdout Contract ✅ Passed This PR modifies cert-manager-operator, a standard Kubernetes operator, not an OTE (OpenShift Tests Extension) binary. The check is not applicable to regular operators.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed PR does not introduce new Ginkgo e2e tests. Changes are infrastructure only: Makefile targets, bash coverage script, and Dockerfile. No test code to assess for IPv4 or connectivity issues.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci openshift-ci Bot requested review from TrilokGeer and bharath-b-rh May 14, 2026 12:39
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 14, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: siddhibhor-56
Once this PR has been reviewed and has the lgtm label, please assign mytreya-rh for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
images/ci/Dockerfile.coverage (1)

24-24: ⚡ Quick win

Document the rationale for UID/GID 65534.

The numeric UID/GID 65534 corresponds to the nobody user/group, which is a common choice for running unprivileged containers. Consider adding a brief inline comment to clarify this choice for future maintainers.

📝 Suggested comment addition
-RUN mkdir -p /tmp/e2e-cover && chown 65534:65534 /tmp/e2e-cover && chmod 700 /tmp/e2e-cover
+# 65534 is the 'nobody' user/group - a standard unprivileged user for containers
+RUN mkdir -p /tmp/e2e-cover && chown 65534:65534 /tmp/e2e-cover && chmod 700 /tmp/e2e-cover
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@images/ci/Dockerfile.coverage` at line 24, Add a brief inline comment to the
RUN line that changes ownership to UID/GID 65534 so future maintainers know
65534 maps to the unprivileged "nobody" user/group; update the RUN command in
Dockerfile.coverage (the line that creates /tmp/e2e-cover and runs chown
65534:65534 and chmod 700) to include a short comment explaining that 65534 is
used to run as an unprivileged user (nobody) to minimize permissions and improve
security.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@hack/e2e-coverage.sh`:
- Around line 94-99: The fixed "sleep 10" after sending SIGTERM is flaky;
replace the static delay with an adaptive wait that polls pod readiness. Remove
the sleep and instead loop (or call oc wait directly) to check the pod
referenced by "${pod}" in "${NAMESPACE}" for phase=Running and container ready
(or use oc wait pod/"${pod}" --for=condition=Ready --timeout=120s without the
prior sleep), retrying until success or timeout; ensure the logic surrounds the
oc exec kill -TERM 1 call and uses the same pod variable so it handles variable
restart times instead of a fixed 10s pause.
- Around line 122-128: The checksum verification currently does a plain cd and
then runs sha256sum, which hides failures and still attempts cd -; change this
to run verification in a safe context and abort on failure: use a subshell or
pushd/popd so directory changes cannot leak (e.g., (cd "$(dirname
"${codecov_bin}")" && sha256sum -c "$(basename "${codecov_bin}").SHA256SUM"))
and check the exit status of sha256sum, exiting the script on non-zero; only run
chmod +x "${codecov_bin}" after the verification succeeds. Reference:
codecov_bin, sha256sum -c, and chmod +x "${codecov_bin}".

---

Nitpick comments:
In `@images/ci/Dockerfile.coverage`:
- Line 24: Add a brief inline comment to the RUN line that changes ownership to
UID/GID 65534 so future maintainers know 65534 maps to the unprivileged "nobody"
user/group; update the RUN command in Dockerfile.coverage (the line that creates
/tmp/e2e-cover and runs chown 65534:65534 and chmod 700) to include a short
comment explaining that 65534 is used to run as an unprivileged user (nobody) to
minimize permissions and improve security.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d26f2852-30f2-42db-96f6-5f664ed4041a

📥 Commits

Reviewing files that changed from the base of the PR and between 5608078 and a4efab4.

📒 Files selected for processing (3)
  • Makefile
  • hack/e2e-coverage.sh
  • images/ci/Dockerfile.coverage

Comment thread hack/e2e-coverage.sh
Comment thread hack/e2e-coverage.sh
Comment on lines +122 to +128
curl -sS -o "${codecov_bin}" \
https://uploader.codecov.io/latest/linux/codecov
curl -sS -o "${codecov_bin}.SHA256SUM" \
https://uploader.codecov.io/latest/linux/codecov.SHA256SUM

cd "$(dirname "${codecov_bin}")" && sha256sum -c "$(basename "${codecov_bin}").SHA256SUM" && cd - >/dev/null
chmod +x "${codecov_bin}"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Improve error handling in checksum verification.

The directory change for sha256sum verification (line 127) does not properly handle failures. If cd fails or the checksum verification fails, the script continues to cd - which may fail or return to the wrong directory, and the error is suppressed by redirecting to /dev/null.

🔒 Proposed fix for safer checksum verification
-            cd "$(dirname "${codecov_bin}")" && sha256sum -c "$(basename "${codecov_bin}").SHA256SUM" && cd - >/dev/null
+            if ! (cd "$(dirname "${codecov_bin}")" && sha256sum -c "$(basename "${codecov_bin}").SHA256SUM"); then
+                echo "Error: Codecov binary checksum verification failed"
+                exit 1
+            fi
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
curl -sS -o "${codecov_bin}" \
https://uploader.codecov.io/latest/linux/codecov
curl -sS -o "${codecov_bin}.SHA256SUM" \
https://uploader.codecov.io/latest/linux/codecov.SHA256SUM
cd "$(dirname "${codecov_bin}")" && sha256sum -c "$(basename "${codecov_bin}").SHA256SUM" && cd - >/dev/null
chmod +x "${codecov_bin}"
curl -sS -o "${codecov_bin}" \
https://uploader.codecov.io/latest/linux/codecov
curl -sS -o "${codecov_bin}.SHA256SUM" \
https://uploader.codecov.io/latest/linux/codecov.SHA256SUM
if ! (cd "$(dirname "${codecov_bin}")" && sha256sum -c "$(basename "${codecov_bin}").SHA256SUM"); then
echo "Error: Codecov binary checksum verification failed"
exit 1
fi
chmod +x "${codecov_bin}"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@hack/e2e-coverage.sh` around lines 122 - 128, The checksum verification
currently does a plain cd and then runs sha256sum, which hides failures and
still attempts cd -; change this to run verification in a safe context and abort
on failure: use a subshell or pushd/popd so directory changes cannot leak (e.g.,
(cd "$(dirname "${codecov_bin}")" && sha256sum -c "$(basename
"${codecov_bin}").SHA256SUM")) and check the exit status of sha256sum, exiting
the script on non-zero; only run chmod +x "${codecov_bin}" after the
verification succeeds. Reference: codecov_bin, sha256sum -c, and chmod +x
"${codecov_bin}".

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
hack/e2e-coverage.sh (2)

84-91: 💤 Low value

Pod selection during rollout edge case.

Using {.items[0].metadata.name} selects the first pod matching the label. During transient states (e.g., partial rollout), this could pick a terminating pod. Consider filtering by status.phase=Running or adding --field-selector=status.phase=Running to ensure the active pod is selected.

This is likely fine in practice since setup waits for rollout completion before tests run.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@hack/e2e-coverage.sh` around lines 84 - 91, The pod selection currently uses
oc get pod ... -o jsonpath='{.items[0].metadata.name}' which can return a
terminating pod; update the oc invocation to restrict to running pods by adding
a field selector or filter (e.g., include --field-selector=status.phase=Running
or adjust jsonpath to pick an item whose status.phase == "Running") so that the
variable pod (and the POD_LABEL/NAMESPACE lookup) always points to an active
pod; keep the rest of the error handling and echo "Operator pod: ${pod}"
unchanged.

36-41: 💤 Low value

Hardcoded JSON patch indices assume specific CSV structure.

The paths deployments/0 and containers/0 assume the operator deployment is always the first deployment in the CSV and uses the first container. If the CSV structure ever changes, this could silently patch the wrong resource.

Consider using jq or a more targeted approach to find the deployment/container by name rather than index, or add a comment documenting this assumption.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@hack/e2e-coverage.sh` around lines 36 - 41, The JSON patch currently
hardcodes array indices in the oc patch call (paths containing deployments/0 and
containers/0), which can target the wrong deployment/container if the CSV order
changes; update the script to locate the correct deployment and container by
name before patching (e.g., use jq to read .spec.install.spec.deployments[] to
find the index for the deployment with the expected name and its containers[] to
find the container index, then construct the JSON patch with those computed
indices or patch by replacing the whole deployment object), referencing the oc
patch invocation and variables COVERAGE_IMAGE and GOCOVERDIR_PATH to build the
final patch; if you intentionally keep the index approach, add a clear comment
documenting the assumption and fail the script if the expected named
deployment/container is not found.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@hack/e2e-coverage.sh`:
- Around line 84-91: The pod selection currently uses oc get pod ... -o
jsonpath='{.items[0].metadata.name}' which can return a terminating pod; update
the oc invocation to restrict to running pods by adding a field selector or
filter (e.g., include --field-selector=status.phase=Running or adjust jsonpath
to pick an item whose status.phase == "Running") so that the variable pod (and
the POD_LABEL/NAMESPACE lookup) always points to an active pod; keep the rest of
the error handling and echo "Operator pod: ${pod}" unchanged.
- Around line 36-41: The JSON patch currently hardcodes array indices in the oc
patch call (paths containing deployments/0 and containers/0), which can target
the wrong deployment/container if the CSV order changes; update the script to
locate the correct deployment and container by name before patching (e.g., use
jq to read .spec.install.spec.deployments[] to find the index for the deployment
with the expected name and its containers[] to find the container index, then
construct the JSON patch with those computed indices or patch by replacing the
whole deployment object), referencing the oc patch invocation and variables
COVERAGE_IMAGE and GOCOVERDIR_PATH to build the final patch; if you
intentionally keep the index approach, add a clear comment documenting the
assumption and fail the script if the expected named deployment/container is not
found.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: a09920f2-42fc-4c6e-9994-8a55586d507b

📥 Commits

Reviewing files that changed from the base of the PR and between a4efab4 and e3fd3fd.

📒 Files selected for processing (3)
  • Makefile
  • hack/e2e-coverage.sh
  • images/ci/Dockerfile.coverage
🚧 Files skipped from review as they are similar to previous changes (2)
  • images/ci/Dockerfile.coverage
  • Makefile

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 18, 2026

@siddhibhor-56: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants