feat(backend): Support KALE_KFP_NAMESPACE for configurable KFP server namespace

[harshhh817]

What this does

Adds support for a KALE_KFP_NAMESPACE environment variable so the Kubernetes namespace used for KFP API operations is configurable without hardcoding it in Kale's server config or asking every user to hand-edit kfp_server_config.json.

Closes #798

Why

KFP is not installed in the same namespace across environments (kubeflow, kubeflow-pipelines, or custom). With the namespace hardcoded, pipeline upload/list/run operations target the wrong namespace and users hit 404s, permission errors, or empty experiment lists even when host and auth are correct. An env var is much easier to standardize fleet-wide (via PodDefault, deployment env, or CI secrets) than baking a per-cluster config file.

Changes

• kale/config/kfp_server_config.py
  • Added a single canonical default DEFAULT_NAMESPACE and a get_default_namespace() helper that reads KALE_KFP_NAMESPACE.
  • Changed the namespace Field default to None so load_config() can distinguish "user didn't set a namespace" from "user explicitly chose the default", then resolves it via _apply_default_namespace().
• kale/common/kfp_client_factory.py
  • Removed the duplicate hardcoded "kubeflow" fallback. load_config() now returns an already-resolved namespace; an explicit namespace= argument still wins.
• kale/tests/unit_tests/test_kfp_server_config.py
  • Added tests for: unset env (default), env override, config-file override, config file without namespace falling back to the env var, and explicit-parameter override.

Resolution order (highest priority first)

1. explicit namespace= argument to get_kfp_client()
2. "namespace" in the JSON config file (KALE_CONFIG_PATH)
3. KALE_KFP_NAMESPACE environment variable
4. canonical in-code default (DEFAULT_NAMESPACE)

This mirrors how Kale already configures paths (KALE_CONFIG_PATH).

Testing

uv run pytest kale/tests/unit_tests/test_kfp_server_config.py -q
# 32 passed (24 existing + 8 new)
uv run ruff check   # clean

Note

The issue mentions some branches default to kubeflow-pipelines, but main currently defaults to kubeflow, so I kept kubeflow as the canonical default to avoid changing existing behavior. Happy to switch the default if maintainers prefer.

[ada333]

hi @harshhh817 from quick look the code looks good - please also add the env var to documentation (https://github.com/kubeflow/kale/blob/main/docs/source/user-guide/running-pipelines.md?plain=1#L104)

[harshhh817]

Thanks for the review @ada333! Added KALE_KFP_NAMESPACE to the environment variables table in running-pipelines.md.

[ada333]

LGTM - I tried setting up the env var and running Kale to see if the target namespace changes, thanks for working on this @harshhh817

[jesuino]

Hello,

The changes looks good and it seems to work well. I have a possible suggestion to avoid code duplication, would you please check it?

Thanks!

[jesuino]

I see multiple calls to this method, could this be done in the _postprocess method? learn more about it in kale/config/config.py:

    Validators are useful for single-field checks. To performs post
    initialization/validation checks, over multiple fields, or postprocess the
    validated fields, override the `_validate` and `_postprocess` functions,
    respectively.

Here's a suggested solution (not tested):

    def _postprocess(self):
        """Fill in the default namespace if not explicitly set."""
        if self.namespace is None:
            self.namespace = get_default_namespace()

[harshhh817]

Thanks @jesuino! Moved the default-namespace resolution into KFPServerConfig._postprocess, which removes the _apply_default_namespace helper and its three call sites in load_config. All config unit tests pass.

[google-oss-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from jesuino. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment