[Initiative]: Cloud Native and OCI Compliant Inner-Loop Tooling & Packaging for AI Engineers

[caldeirav]

Name

Cloud Native and OCI Compliant Inner-Loop Tooling & Packaging for AI Engineers

Short description

Integrating the AI developer inner loop into an end-to-end CI/CD process leveraging cloud-native technologies and tooling

Responsible group

TOC

Does the initiative belong to a subproject?

Yes

Subproject name

TOC Artificial Intelligence Initiatives

Primary contact

Vincent Caldeira (vincent.caldeira@gmail.com)

Additional contacts

Ricardo Aravena (raravena80@gmail.com)

Initiative description

Scope definition

Focus on the developer inner loop, everything an AI engineer does on a laptop/desktop before code or models ever reach CI/CD in a cloud-native environment:

• Local container workspaces: Reference inner loop workflow using desktop tooling such as Podman Desktop / Podman AI Lab for root-less, GPU-aware experimentation, including template images for PyTorch/LLM stacks and volume-mounted datasets. ​
• Unified model build & run CLI: Hardening inference on developer machine and agentic frameworks to leverage container-based tooling so engineers can easily spin-up inference, RAG and multi-agent services locally with one command.
• Standard packaging of artefacts: Drive convergence between various implementations such as ModelKit, ModelCar towards the emerging ModelPack spec to create a single OCI-manifest that can hold model weights, metadata and SBOM.
• Inner-loop supply-chain security: Integrate Notary v2 / model authenticity and transparency via Sigstore, LF AI & Data Model Openness Framework-generated model & data cards, plus SBOM annotations directly into the OCI artefact so that security & openness are “baked in” before CI. ​
• Fast hand-off to outer loop: Provide reference GitOps flows (Flux/Argo) that pull the signed artefact into KServe with ModelPack image-mount optimisation, and register versions in Kubeflow Model Registry.

Why it matters for the CNCF

• Closes the skills gap: Today AI engineers live in Python notebooks while cloud-native tools live in YAML. A container-native inner loop brings AI creators into the CNCF ecosystem early, making Kubernetes the default target platform.
• Eliminates fragmentation: Multiple, incompatible model-packaging attempts (Docker model-CLI, KitOps, ONNX zip files, etc.) slow adoption. A CNCF-backed, OCI-compatible spec creates a neutral home and clear interoperability story. ​
• Raises baseline security & transparency: By embedding MOF openness requirements and Sigstore signing before code hits CI, the sub-stream aligns with industry compliance trends and improves trust across end-to-end supply chains. ​
• Accelerates project reuse: The work provides reusable libraries, CRDs and GitOps templates that every other CNCF AI project (KServe, Kubeflow, TrustyAI, etc.) can import rather than reinventing developer tooling.

Key technologies & projects involved

• Container tooling: Podman Desktop, Podman AI Lab, Docker model-runner (observer role)
• Packaging & spec: ModelKit (KitOps), ModelPack, ModelCar, OCI image/artefact spec, Notary v2, Sigstore
• Model runtime & APIs: Ramalama (potential contribution), Agentic Orchestration Frameworks, MCP Servers for tool orchestration
• Kubernetes services: KServe + ModelCars, Kubeflow Model Registry
• Governance & openness: LF AI & DATA Model Openness Framework (MOF) generators, SBOM annotations
• GitOps & automation: Flux, Argo Workflows/Pipelines

Deliverable(s) or exit criteria

1. An technical POC showing <10 min “idea-to-inference” path for cloud-native agent development on a developer laptop.
2. Clearly documented standards for OCI artefact standardization across runtimes and registries.
3. Specification / procedure to achieve MOF Class III compliant model distributions via any OCI registry.
4. Standardised process for leveraging model signing with artefacts-level provenance to support a verified end-to-end CI/CD reference pipeline including outer loop for AI engineering.

[leonardpahlke]

Interesting proposal. I like the 'bottom-up' focus.. starting with the ai engineer.

[angellk]

@caldeirav agree w/ bottom up approach.

Please outline the project timeline and break up the deliverables if necessary.

Okay to move to vote @riaankleinhans

[riaankleinhans]

This initiative has been approved by the TOC and is ready to be worked on with the appropriate TAG and TOC liaison.