chore(docs): katib (optimizer) client support for kubeflow-mcp-server (kep#34)

[Krishna-kg732]

Summary

This KEP proposes implementing the Optimizer client module for
kubeflow-mcp-server, exposing Katib's Experiment, Trial, and Suggestion
lifecycle as 17 MCP tools across 5 categories (Planning, Optimization,
Discovery, Monitoring, Lifecycle).

This implements the "Optimizer (Planned: Phase 2)" node already identified
in the architecture and stub module (kubeflow_mcp.optimizer), making Katib
the natural second client after TrainerClient — completing the inner loop of
train -> evaluate -> tune -> retrain without leaving the MCP interface.

Motivation

AI IDEs and orchestrator agents currently have no structured way to:

• Launch Katib HPO experiments from natural language descriptions
• Inspect experiment progress, individual trial results, or suggestion
  algorithm status
• Retrieve the best hyperparameter configuration from a completed experiment
• Integrate HPO into automated ML pipelines managed by agents

The existing stub declares 8 planned tools with status: "stub" and no
implementations.

Goals

• Implement 17 MCP tools across Planning, Optimization, Discovery,
  Monitoring, and Lifecycle categories
• Decompose experiment creation into agent-friendly tools
  (create_hpo_experiment / create_experiment_from_spec), following the
  trainer's fine_tune/run_custom_training pattern
• Use kubeflow.katib.KatibClient as the primary interface, with
  CustomObjectsApi fallback where the SDK lacks coverage
• Integrate with existing server infrastructure (personas, audit, rate
  limiting, circuit breaker, namespace enforcement)
• Two-phase confirmation for all mutating operations
• Unit tests at ≥80% coverage, plus integration tests against a live Katib
  install

Non-Goals

• NAS (Neural Architecture Search) — follow-up after HPO
• Custom suggestion algorithm deployment
• Katib UI replacement
• Direct Katib DB Manager access (get_trial_metrics(), requires gRPC)
• Wrapping the tune() API (requires Python callables, not MCP-serializable)
• edit_experiment_budget() — deferred

Details

Full design : module structure, tool tables, persona coverage, SDK
compatibility, cross-client integration with TrainerClient, risks and
mitigations, and the testing plan , is in the KEP doc itself.

Status

open for review and discussion before implementation begins.

Context

#34

cc: @jaiakash , @abhijeet-dhumal , @andreyvelich

[google-oss-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign abhijeet-dhumal for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[Copilot]

Pull request overview

Adds a new KEP document proposing Katib (Optimizer) client support in kubeflow-mcp-server, describing the planned MCP tool surface, module structure, personas, and testing approach for integrating hyperparameter optimization workflows alongside the existing Trainer client.

Changes:

• Introduces KEP#0001 describing the Optimizer client scope (17 tools across Planning/Optimization/Discovery/Monitoring/Lifecycle).
• Documents proposed module layout, tool phase grouping, persona access, and cross-client workflow (train -> tune -> retrain).
• Outlines compatibility assumptions, risks/mitigations, and a unit/integration testing plan.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[abhijeet-dhumal]

Thanks @Krishna-kg732 for working on this, the KEP is very well-structured !!
Great work scoping this out 🚀

[abhijeet-dhumal]

@Krishna-kg732 can you resolve/address copilot's reviews too ?

[abhijeet-dhumal]

@andreyvelich @aniket2405 @szaher
can we get more eyes for review here ?

[abhijeet-dhumal]

Should this be pointing to Kubeflow-SDK's OptimizerClient instead of legacy one ?

[Krishna-kg732]

yup , I will change that

[abhijeet-dhumal]

Suggested change

	| SDK Method (`KatibClient`) | MCP Tool | Notes |
	| SDK Method (`OptimizerClient`) | MCP Tool | Notes |

??

[Krishna-kg732]

will take a look

[abhijeet-dhumal]

same here ..
the methods list also needs updating to reflect OptimizerClient's API, can you check ?