[rhoai-2.25] RHAIENG-6036: fix trustyai k8s papermill probe timing

[jiridanek]

Summary

Align TrustyAI StatefulSet health probes with pytorch (15/10) so Elyra/Jupyter can finish starting before liveness kills the pod.

Problem: On Build Notebooks k8s papermill (make_test.py), the trustyai pod CrashLoopBackOffs: liveness initialDelaySeconds: 5 probes port 8888 before Jupyter listens (~12s+ with Elyra). Papermill then fails with container not found ("notebook").

Evidence: #2499 CI run 29087867712 — testcontainers 16 passed; k8s phase failed on probe kills (events captured in make_test.py finally dumps).

Fix: jupyter/trustyai/ubi9-python-3.12/kustomize/base/statefulset.yaml only:

• liveness: initialDelaySeconds 5→15, periodSeconds 5→10
• readiness: initialDelaySeconds 10→15, periodSeconds 5→10

Matches jupyter/pytorch/ubi9-python-3.12/kustomize/base/statefulset.yaml on main. Not a cherry-pick -x from main trustyai (trustyai statefulset on main still has 5/10).

Out of scope (already fixed elsewhere)

Symptom	Fix	Status
/opt/app-root/share 755:0 + spinner	#2451	RHAIENG-6041 — testcontainers green on tip
Papermill CSV 404	#2499	Merged
Codeserver mysql test	#2446	RHAIENG-6039 — testcontainers green on tip
Codeserver check-payload FIPS	#2500	Separate PR

Test plan

• jupyter-trustyai-ubi9-python-3.12 amd64 Build Notebooks: k8s papermill (make_test.py) passes
• Testcontainers phase remains green (unchanged by this PR)

Local repro

k3s on 100.105.65.25: Quay trustyai image + #2499 branch — papermill passes with probe 15s patch (/tmp/test-trustyai-quay-k8s.log).

Summary by CodeRabbit

• Bug Fixes
  • Improved service health-check timing to allow more time for startup and stabilize readiness and liveness detection.
  • Preserved existing health-check endpoints and validation thresholds.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign atheo89 for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 88fbc81d-63bf-4e84-8764-95591e47ac0a

📥 Commits

Reviewing files that changed from the base of the PR and between d8115e6 and 293a861.

📒 Files selected for processing (1)

• jupyter/trustyai/ubi9-python-3.12/kustomize/base/statefulset.yaml

🚧 Files skipped from review as they are similar to previous changes (1)

• jupyter/trustyai/ubi9-python-3.12/kustomize/base/statefulset.yaml

---

📝 Walkthrough

Walkthrough

The StatefulSet liveness and readiness probes now wait longer before starting and run at longer intervals. Their endpoints, ports, schemes, and success/failure thresholds remain unchanged.

Changes

StatefulSet probe timing

Layer / File(s)	Summary
Health probe timing jupyter/trustyai/ubi9-python-3.12/kustomize/base/statefulset.yaml	Liveness and readiness probe delays increase to 15 seconds, and polling periods increase to 10 seconds while probe behavior remains otherwise unchanged.

Estimated code review effort: 1 (Trivial) | ~5 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the probe timing fix for the TrustyAI Kubernetes papermill setup.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/rhoai-2.25-ws2-codeserver-trustyai

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

🧹 Nitpick comments (1)

tests/containers/workbenches/jupyterlab/jupyterlab_datascience_test.py (1)

88-93: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Skip guard for code-server images looks correct.

The dual-variant label check (-code-server- for ODH, -codeserver- for RHOAI/RHDS Konflux) is well-documented and consistent with the pattern in conftest.py. The removal of the conditional mysql-connector-python installation block is safe since code-server images now never reach that code path.

As per coding guidelines, consider adding @allure.issue("RHOAIENG-140") to link this test to the Jira issue referenced in the skip message.

♻️ Optional: add allure issue decorator

+    `@allure.issue`("RHOAIENG-140")
     `@allure.description`("Check that mysql client functionality is working with SASL plain auth.")
     def test_mysql_connection(self, tf: TestFrame, datascience_image: Image, subtests):

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/containers/workbenches/jupyterlab/jupyterlab_datascience_test.py`
around lines 88 - 93, Add the `@allure.issue`("RHOAIENG-140") decorator to the
affected JupyterLab datasource test function, alongside its existing test
decorators, so the test is linked to the Jira issue referenced by the
code-server skip guard.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/containers/workbenches/jupyterlab/jupyterlab_datascience_test.py`:
- Around line 88-93: Add the `@allure.issue`("RHOAIENG-140") decorator to the
affected JupyterLab datasource test function, alongside its existing test
decorators, so the test is linked to the Jira issue referenced by the
code-server skip guard.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: a3c9f052-48ee-4f80-9bb9-264502e392f2

📥 Commits

Reviewing files that changed from the base of the PR and between ab66e21 and d8115e6.

📒 Files selected for processing (5)

• jupyter/trustyai/ubi9-python-3.12/Dockerfile.cpu
• jupyter/trustyai/ubi9-python-3.12/Dockerfile.konflux.cpu
• jupyter/trustyai/ubi9-python-3.12/kustomize/base/statefulset.yaml
• scripts/check-payload/config.toml
• tests/containers/workbenches/jupyterlab/jupyterlab_datascience_test.py

[jiridanek]

this may not be actually necessary, maybe the timeouts were a rare flake?