Skip to content

[rhoai-2.25] restore codeserver to last ppc64le-known-good build#2522

Open
ysok wants to merge 2 commits into
red-hat-data-services:rhoai-2.25from
ysok-red-hat-data-services:revert-codeserver-to-c985e11
Open

[rhoai-2.25] restore codeserver to last ppc64le-known-good build#2522
ysok wants to merge 2 commits into
red-hat-data-services:rhoai-2.25from
ysok-red-hat-data-services:revert-codeserver-to-c985e11

Conversation

@ysok

@ysok ysok commented Jul 13, 2026

Copy link
Copy Markdown

Summary

  • Restore codeserver/ubi9-python-3.12 on rhoai-2.25 to commit c985e1117cea (2026-05-29), the last Konflux build that successfully published a ppc64le image for odh-workbench-codeserver-datascience-cpu-py312-rhel9.
  • Evidence: Quay tag rhoai-2.25-linux-ppc64le still points at that commit (Created: 2026-05-29T19:27:43Z, vcs-ref: c985e1117cea...). No newer commit tag contains an updated ppc64le layer; Konflux codeserver on-push has been failing since then.
  • Scope is codeserver only — Jupyter/runtime images are unchanged.

What this rolls back (intentionally)

Relative to current rhoai-2.25, this undoes codeserver-only changes landed after the last good build, including:

  • Gateway API nginx / proxy_pass updates
  • Code Server / VS Code extension bumps (VSIX + Dockerfiles)
  • codeserver pylock.toml / pyproject.toml refreshes
  • related run-nginx.sh / run-code-server.sh changes

Test plan

  • Konflux odh-workbench-codeserver-datascience-cpu-py312-v2-25-on-push succeeds on this PR/merge
  • Confirm Quay publishes refreshed arch tags, especially rhoai-2.25-linux-ppc64le, with vcs-ref matching this change
  • Spot-check amd64/arm64 codeserver still starts (regression expectation: pre–Gateway API nginx behavior)

Made with Cursor

Summary by CodeRabbit

  • New Features

    • Improved code-server routing, redirects, and health-check handling.
    • Added enhanced web server logging and WebSocket support.
    • Updated bundled Python and Jupyter extensions.
  • Bug Fixes

    • Improved access to code-server with and without URL prefixes.
    • Fixed kernel API and health-check proxy behavior.
    • Preserved more reliable startup validation and logging.
  • Updates

    • Updated the bundled code-server and Node.js runtime versions.
    • Broadened supported Linux CPython environments.

Restore codeserver/ubi9-python-3.12 to commit c985e11 (2026-05-29),
the last Konflux push that successfully published a ppc64le image for
odh-workbench-codeserver-datascience-cpu-py312-rhel9.

This intentionally rolls back later codeserver-only changes (Gateway API
nginx proxy updates, Code Server / extension bumps, pylock refreshes)
until a new multi-arch build succeeds on ppc64le.

Co-authored-by: Cursor <cursoragent@cursor.com>
@openshift-ci
openshift-ci Bot requested review from daniellutz and dibryant July 13, 2026 18:56
@openshift-ci

openshift-ci Bot commented Jul 13, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign jiridanek for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai

coderabbitai Bot commented Jul 13, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@ysok, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 49 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 0f50de54-f56f-4032-9c47-8fcdf1c9a6bc

📥 Commits

Reviewing files that changed from the base of the PR and between 0ce0119 and d616456.

📒 Files selected for processing (1)
  • .tekton/odh-workbench-codeserver-datascience-cpu-py312-pull-request.yaml
📝 Walkthrough

Walkthrough

Updates the UBI9 Python 3.12 code-server image builds, dependency and extension versions, nginx routing and logging configuration, startup scripts, and Python environment selection.

Changes

Code-server image build and runtime routing

Layer / File(s) Summary
Image build, dependencies, and packaged extensions
codeserver/ubi9-python-3.12/Dockerfile.*, codeserver/ubi9-python-3.12/get_code_server_rpm.sh, codeserver/ubi9-python-3.12/pyproject.toml, codeserver/ubi9-python-3.12/utils/*.vsix
UBI repository injection, Node.js and code-server defaults, wheel-cache commands, extension pins, test execution, Python environment selection, and LFS extension pointers are updated.
Nginx runtime routing and startup
codeserver/ubi9-python-3.12/nginx/*, codeserver/ubi9-python-3.12/run-nginx.sh, codeserver/ubi9-python-3.12/run-code-server.sh
Nginx gains HTTP maps, JSON logging, an upstream definition, revised health and kernels redirects, simplified code-server proxying, updated prefix handling, and reduced runtime-generated configuration. The generated code-server settings no longer disable AI features.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related PRs

Suggested labels: review-requested

Suggested reviewers: jiridanek, dibryant, daniellutz, atheo89

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly states the main change: restoring codeserver to a last-known-good ppc64le build.
Description check ✅ Passed The PR description covers the rollback scope and test plan, though the template checklist items are still unfilled.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ysok

ysok commented Jul 13, 2026

Copy link
Copy Markdown
Author

/build-codeserver

@github-actions

Copy link
Copy Markdown

@ysok — This PR is from a fork.
The build-rhoai CI job was skipped because subscription
builds (RHEL, AIPCC) need secrets unavailable to forks.
ODH builds and code quality checks still ran.

Recommended: Push your branch to the main repo for full CI:

git remote add upstream https://github.com/red-hat-data-services/notebooks.git
git push upstream HEAD:ysok/your-branch-name

Then open a new PR from that branch.

No push access? A maintainer will cherry-pick and test your changes.

See CONTRIBUTING.md for details.

…/label

Update PaC annotations on the codeserver pull-request PipelineRun so
/build-konflux, /build-codeserver, and kfbuild-* labels can trigger the
build for validating the known-good restore on PR red-hat-data-services#2522.

Co-authored-by: Cursor <cursoragent@cursor.com>
@ysok

ysok commented Jul 13, 2026

Copy link
Copy Markdown
Author

/build-codeserver

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
codeserver/ubi9-python-3.12/run-nginx.sh (1)

21-21: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Unquoted $NB_PREFIX and $NOTEBOOK_ARGS risk word splitting; export masks pipeline failures.

With set -e active, if grep -Po 'hub_host":\K.*?(?=")' fails to match in $NOTEBOOK_ARGS, the failure is masked by the export BASE_URL=$(...) assignment, leaving BASE_URL empty or partially set without aborting. Quoting the variables and separating the assignment from export would make failures visible.

♻️ Proposed refactor
-export BASE_URL=$(echo $NB_PREFIX | awk -F/ '{ print $4"-"$3 }')$(echo $NOTEBOOK_ARGS | grep -Po 'hub_host":"\K.*?(?=")' | awk -F/ '{ print $3 }' | awk -F. '{for (i=2; i<=NF; i++) printf ".%s", $i}')
+BASE_URL="$(echo "$NB_PREFIX" | awk -F/ '{ print $4"-"$3 }')"
+BASE_URL="${BASE_URL}$(echo "$NOTEBOOK_ARGS" | grep -Po 'hub_host":"\K.*?(?=")' | awk -F/ '{ print $3 }' | awk -F. '{for (i=2; i<=NF; i++) printf ".%s", $i}')"
+export BASE_URL
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@codeserver/ubi9-python-3.12/run-nginx.sh` at line 21, Update the BASE_URL
construction in run-nginx.sh to quote NB_PREFIX and NOTEBOOK_ARGS wherever they
are expanded, and split command substitution assignment from export so pipeline
or grep failures are not masked by export. Preserve the existing BASE_URL
value-building behavior while ensuring unmatched hub_host extraction causes the
script to fail under set -e.
codeserver/ubi9-python-3.12/pylock.toml (1)

181-184: 🔒 Security & Privacy | 🔵 Trivial

Restored lockfile reintroduces packages with known HIGH-severity CVEs — track separately rather than fixing in this rollback.

OSV Scanner flags click==8.3.1, protobuf==6.33.3 (JSON recursion depth bypass), pyjwt==2.10.1 (several GHSAs including JWKS SSRF/DoS and algorithm allow-list bypass issues), starlette==1.2.0 (form-limit DoS, host-header poisoning), and urllib3==2.6.3 (decompression-bomb bypass, cross-origin header leak) in this restored lockfile.

Since this PR intentionally restores the exact dependency state of a previously-working Konflux commit, patching these here would work against that goal and risks reintroducing the ppc64le build failures the rollback is meant to fix. Recommend tracking these in a dedicated CVE-remediation follow-up once the rollback is verified stable.

Based on learnings, "do not regenerate or update per-image pylock.toml files as part of CVE-fix PRs ... lock regeneration should be handled by CI/maintainers instead."

Also applies to: 1612-1625, 1854-1857, 2410-2413, 2498-2501

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@codeserver/ubi9-python-3.12/pylock.toml` around lines 181 - 184, Keep the
restored dependency versions in the per-image pylock.toml files unchanged; do
not regenerate or update these lockfiles to remediate the listed CVEs in this
rollback. Track click, protobuf, pyjwt, starlette, and urllib3 vulnerabilities
separately for a dedicated follow-up, with lock regeneration deferred to CI or
maintainers.

Sources: Learnings, Linters/SAST tools

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@codeserver/ubi9-python-3.12/pylock.toml`:
- Around line 181-184: Keep the restored dependency versions in the per-image
pylock.toml files unchanged; do not regenerate or update these lockfiles to
remediate the listed CVEs in this rollback. Track click, protobuf, pyjwt,
starlette, and urllib3 vulnerabilities separately for a dedicated follow-up,
with lock regeneration deferred to CI or maintainers.

In `@codeserver/ubi9-python-3.12/run-nginx.sh`:
- Line 21: Update the BASE_URL construction in run-nginx.sh to quote NB_PREFIX
and NOTEBOOK_ARGS wherever they are expanded, and split command substitution
assignment from export so pipeline or grep failures are not masked by export.
Preserve the existing BASE_URL value-building behavior while ensuring unmatched
hub_host extraction causes the script to fail under set -e.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: ff55e18d-e7ac-419a-9c5a-e6481a4165bb

📥 Commits

Reviewing files that changed from the base of the PR and between 08f4fe3 and 0ce0119.

📒 Files selected for processing (14)
  • codeserver/ubi9-python-3.12/Dockerfile.cpu
  • codeserver/ubi9-python-3.12/Dockerfile.konflux.cpu
  • codeserver/ubi9-python-3.12/get_code_server_rpm.sh
  • codeserver/ubi9-python-3.12/nginx/httpconf/http.conf
  • codeserver/ubi9-python-3.12/nginx/serverconf/proxy.conf.template
  • codeserver/ubi9-python-3.12/nginx/serverconf/proxy.conf.template_nbprefix
  • codeserver/ubi9-python-3.12/pylock.toml
  • codeserver/ubi9-python-3.12/pyproject.toml
  • codeserver/ubi9-python-3.12/run-code-server.sh
  • codeserver/ubi9-python-3.12/run-nginx.sh
  • codeserver/ubi9-python-3.12/utils/ms-python.python-2025.14.0.vsix
  • codeserver/ubi9-python-3.12/utils/ms-python.python-2026.4.0.vsix
  • codeserver/ubi9-python-3.12/utils/ms-toolsai.jupyter-2025.8.0.vsix
  • codeserver/ubi9-python-3.12/utils/ms-toolsai.jupyter-2025.9.1.vsix
💤 Files with no reviewable changes (3)
  • codeserver/ubi9-python-3.12/utils/ms-toolsai.jupyter-2025.9.1.vsix
  • codeserver/ubi9-python-3.12/utils/ms-python.python-2026.4.0.vsix
  • codeserver/ubi9-python-3.12/run-code-server.sh

@openshift-ci

openshift-ci Bot commented Jul 16, 2026

Copy link
Copy Markdown

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant