Skip to content

feat: support oxia metadata store#1276

Open
mattisonchao wants to merge 7 commits into
masterfrom
feat/oxia-metadata
Open

feat: support oxia metadata store#1276
mattisonchao wants to merge 7 commits into
masterfrom
feat/oxia-metadata

Conversation

@mattisonchao
Copy link
Copy Markdown
Member

@mattisonchao mattisonchao commented May 25, 2026

Motivation

  • Support Oxia as an in-cluster metadata provider for sn-platform and sn-platform-slim without requiring users to configure service addresses manually.
  • Keep ZooKeeper as the default metadata provider and preserve existing ZooKeeper compatibility.
  • Ensure bundled Prometheus can scrape both Oxia dataserver and coordinator metrics without requiring Prometheus Operator.

Modifications

  • Added pulsar_metadata.provider support for zookeeper and oxia, with validation for ZooKeeper-only fields.
  • Added OxiaCluster and OxiaNamespace templates using Oxia 0.16.4 and split namespaces for broker, bookkeeper, pulsar-schema, and function.
  • Added oxia chart values for custom labels, annotations, server replica count, server resources, and coordinator resources.
  • Wired broker, BookKeeper, schema storage, and function-worker config to use Oxia metadata URLs when provider is Oxia.
  • Gated ZooKeeper resources so they are not rendered for Oxia metadata deployments.
  • Added bundled Prometheus scrape config job_name: oxia that scrapes both metrics and coord-metrics ports.
  • Gated Oxia monitoringEnabled on the availability of the PodMonitor API so sn-operator only creates PodMonitor when Prometheus Operator CRDs exist.

Documentation

  • No standalone docs page is required for this PR. The user-facing chart knobs are documented in the chart values.yaml, and examples/sn-platform/values-oxia.yaml shows the minimal Oxia enablement path.

Testing

  • helm lint charts/sn-platform-slim -f examples/sn-platform/values-oxia.yaml --set istio.enabled=true --set monitoring.prometheus=true --set monitoring.grafana=false
  • helm lint charts/sn-platform -f examples/sn-platform/values-oxia.yaml --set istio.enabled=true --set monitoring.prometheus=true --set monitoring.grafana=false
  • helm lint for the sn-platform and sn-platform-slim CI values.
  • helm template for both chart variants with Oxia values.
  • Rendered OxiaCluster overrides for labels, annotations, replicas, server resources, and coordinator resources.
  • Verified oxia.replicaCount < 3 fails template validation for Oxia metadata deployments.
  • Local kind deployment with sn-operator v0.18.10, Istio ambient, no proxy, Oxia 0.16.4, and bundled Prometheus.
  • Verified Pulsar producer/consumer smoke test through a perf pod.
  • Verified bundled Prometheus ingests oxia_dataserver_* and oxia_coordinator_* series under job=oxia.

@mattisonchao mattisonchao requested review from a team as code owners May 25, 2026 16:57
@mattisonchao mattisonchao requested a review from tuteng May 25, 2026 16:57
@github-actions
Copy link
Copy Markdown

@mattisonchao:Thanks for your contribution. For this PR, do we need to update docs?
(The PR template contains info about doc, which helps others know more about the changes. Can you provide doc-related info in this and future PR descriptions? Thanks)

@github-actions github-actions Bot added the doc-info-missing This pr needs to mark a document option in description label May 25, 2026
@tuteng tuteng requested a review from Copilot May 26, 2026 00:11
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

imagePullPolicy: {{ .Values.images.oxia.pullPolicy }}
monitoringEnabled: {{ include "pulsar.oxia.podMonitor.enabled" . }}
server:
replicas: 3
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you make this replicas configurable along with the cpu and memory? Perhaps the minimum number of replicas should be limited to 3

@@ -1189,6 +1193,8 @@ autorecovery:
## metadata deployed
pulsar_metadata:
component: pulsar-init
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just like with other components, how about adding an oxia section to support custom labels, annotations, replicas, CPU, and memory?

mtls:
mode: permissive
{{- end }}
{{- end }}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

trustCertsEnabled: {{ .Values.tls.broker.trustCertsEnabled }}
{{- end }}
{{- if and .Values.istio.enabled .Values.ingress.broker.enabled }}
{{- if .Values.istio.enabled }}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change hasn’t been implemented in sn-platform-slim yet. Please help check it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-info-missing This pr needs to mark a document option in description

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants