feat(backend): support KALE_PYPI_PROD_URL for configurable production PyPI index#866
Merged
google-oss-prow[bot] merged 3 commits intoJul 22, 2026
Merged
Conversation
… PyPI index Air-gapped or mirror-only deployments can't reach pypi.org, but Kale always appended it as the hardcoded production fallback in generated pipeline components regardless of other pip index overrides. Add KALE_PYPI_PROD_URL so this fallback is configurable, defaulting to https://pypi.org/simple for backward compatibility. Fixes kubeflow#797 Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: [Bhavdeep Singh] <bhavdeep3singh@gmail.com>
Bhavd33p
force-pushed
the
feat/kale-pypi-prod-url
branch
from
July 1, 2026 18:03
29ab69c to
b35cadb
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR makes the “production” pip simple index URL used in generated KFP components configurable via KALE_PYPI_PROD_URL, while keeping the existing default (https://pypi.org/simple) for backward compatibility. This supports air-gapped and mirror-only deployments that cannot reach public PyPI.
Changes:
- Updated
compute_pip_index_urls()to read the production fallback index fromKALE_PYPI_PROD_URL. - Added unit tests covering default and override behaviors for the new environment variable.
- Documented
KALE_PYPI_PROD_URLalongside other runtime environment variables.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| kale/common/utils.py | Adds KALE_PYPI_PROD_URL support in pip index URL computation and updates related docstring text. |
| kale/tests/unit_tests/test_utils.py | Adds tests validating default and overridden production index behavior. |
| docs/source/user-guide/running-pipelines.md | Documents KALE_PYPI_PROD_URL in the runtime environment variables table. |
| docs/source/api/cli.md | Documents KALE_PYPI_PROD_URL in the CLI environment variables table. |
Comments suppressed due to low confidence (1)
kale/common/utils.py:238
compute_pip_index_urls()can return duplicate index URLs. IfKALE_PIP_INDEX_URLSalready includes the production URL (or it matches the devpi URL), the function appendspypi_prod_urlagain unconditionally; the later “remove duplicates” block is ineffective because it checks membership after appending. Also,KALE_PYPI_PROD_URLisn’t stripped, so whitespace-only values become invalid URLs. Consider normalizingKALE_PYPI_PROD_URLand ensuring it only appears once (as the final entry).
pypi_prod_url = os.getenv("KALE_PYPI_PROD_URL", "https://pypi.org/simple")
urls: list[str]
# explicit override wins
env_override = os.getenv("KALE_PIP_INDEX_URLS")
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+212
to
+215
| return a list with the devpi simple URL (`KALE_DEVPI_SIMPLE_URL`) or | ||
| its default value. | ||
| 3. Otherwise, return the production default: | ||
| ["https://pypi.org/simple"]. | ||
| [`KALE_PYPI_PROD_URL`] (defaults to ["https://pypi.org/simple"]). |
Comment on lines
+149
to
+158
| def test_compute_pip_index_urls_prod_url_override_with_explicit_indexes(monkeypatch): | ||
| """KALE_PYPI_PROD_URL still lands last when KALE_PIP_INDEX_URLS is set.""" | ||
| _clear_env(monkeypatch) | ||
| monkeypatch.setenv("KALE_PYPI_PROD_URL", "https://pypi.internal.example.com/simple") | ||
| monkeypatch.setenv("KALE_PIP_INDEX_URLS", "https://mirror.one/simple") | ||
|
|
||
| assert utils.compute_pip_index_urls() == [ | ||
| "https://mirror.one/simple", | ||
| "https://pypi.internal.example.com/simple", | ||
| ] |
Contributor
Author
|
@jesuino Yes I will do the required changes Edit :Kindly review the changes |
Address Copilot review feedback on kubeflow#866: - compute_pip_index_urls() could return the production URL twice when KALE_PIP_INDEX_URLS (or the devpi URL) already contained it, since the old dedup check ran after the unconditional append and could never fire. Now the production URL is filtered out of the base list and appended once at the end. - KALE_PYPI_PROD_URL is stripped, and a whitespace-only value falls back to the default instead of producing an invalid index URL. - Updated the docstring precedence description and added tests for the duplicate and whitespace-only cases. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: [Bhavdeep Singh] <bhavdeep3singh@gmail.com>
Contributor
|
@Bhavd33p it looks good to me! please rebase your branch so we can see the CI results and merge it! |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: ederign The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
compute_pip_index_urls()hardcoded the production pip index fallback tohttps://pypi.org/simple, appended unconditionally regardless ofKALE_PIP_INDEX_URLS/KALE_DEV_MODEoverridesKALE_PYPI_PROD_URLenv var to make this fallback configurable, defaulting tohttps://pypi.org/simplefor backward compatibilityKALE_PIP_*/KALE_DEV_MODEvars indocs/source/user-guide/running-pipelines.mdanddocs/source/api/cli.mdFixes #797
Test plan
pytest kale/tests/unit_tests/test_utils.py -k compute_pip_index_urls -v— 8 passed (5 existing + 3 new)pytest kale/tests/unit_tests/— 247 passedKALE_PYPI_PROD_URLset → used in place ofpypi.org; unset → falls back tohttps://pypi.org/simple🤖 Generated with Claude Code