Skip to content

use Helm to init rustfs in sandbox #7233

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
BarryWu0812 wants to merge 4 commits into
base: v2
Choose a base branch
from
Open

use Helm to init rustfs in sandbox #7233

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
2ad3940
use Helm to init rustfs in sandbox
BarryWu0812 Apr 18, 2026
69b36d8
Fix naming typo
BarryWu0812 Apr 19, 2026
3dafa36
Fix port num
BarryWu0812 Apr 19, 2026
2ec03f1
Update complete/dev yaml
BarryWu0812 Apr 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions charts/flyte-demo/Chart.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ dependencies:
- name: flyte-binary
repository: file://../flyte-binary
version: v0.2.0

digest: sha256:bc1e2e37ac2f11f9bf547262643c9076d6c86af661b6ffdee77cc6756d687d6e
generated: "2026-04-14T10:42:50.772509+08:00"
- name: rustfs
repository: https://rustfs.github.io/helm
version: 0.0.94
digest: sha256:9bdbd39953aa90c13d03384b1273125aa8e944fc1f1fccb7070f581d078a7db1
generated: "2026-04-18T19:21:38.284835-05:00"
5 changes: 5 additions & 0 deletions charts/flyte-demo/Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,8 @@ dependencies:
version: v0.2.0
repository: file://../flyte-binary
condition: flyte-binary.enabled
- name: rustfs
alias: rustfsupstream
version: 0.0.94
repository: https://rustfs.github.io/helm
condition: rustfs.enabled
8 changes: 0 additions & 8 deletions charts/flyte-demo/templates/_helpers.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,14 +82,6 @@ Name of PersistentVolume and PersistentVolumeClaim for PostgreSQL database
{{- printf "%s-db-storage" .Release.Name -}}
{{- end }}

{{/*
Name of PersistentVolume and PersistentVolumeClaim for RustFS
*/}}
{{- define "flyte-demo.persistence.rustfsVolumeName" -}}
{{- printf "%s-rustfs-storage" .Release.Name -}}
{{- end }}


{{/*
Selector labels for console
*/}}
Expand Down
39 changes: 39 additions & 0 deletions charts/flyte-demo/templates/storage/rustfs-bucket-init-job.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
{{- if and .Values.rustfs.enabled .Values.rustfs.bucketInit.enabled }}
{{- $accessKey := .Values.rustfsupstream.secret.rustfs.access_key }}
{{- $secretKey := .Values.rustfsupstream.secret.rustfs.secret_key }}
apiVersion: batch/v1
kind: Job
metadata:
name: {{ printf "%s-rustfs-bucket-init" .Release.Name | trunc 63 | trimSuffix "-" }}
namespace: {{ .Release.Namespace | quote }}
labels:
{{- include "flyte-demo.labels" . | nindent 4 }}
annotations:
"helm.sh/hook": post-install,post-upgrade
"helm.sh/hook-weight": "10"
"helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
spec:
Copy link
Copy Markdown
Contributor

@popojk popojk Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that the job pod remain in cluster after job finished. could we add a ttlSecondsAfterFinished under spec section to delete it?

backoffLimit: 6
template:
metadata:
labels:
{{- include "flyte-demo.selectorLabels" . | nindent 8 }}
app.kubernetes.io/component: rustfs-bucket-init
Copy link
Copy Markdown
Contributor

@popojk popojk Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious why do we need this label here? I thought this is just an one-off k8s job and maybe we don't need label?

spec:
restartPolicy: OnFailure
containers:
- name: bucket-init
image: "{{ .Values.rustfs.bucketInit.image.repository }}:{{ .Values.rustfs.bucketInit.image.tag }}"
imagePullPolicy: {{ .Values.rustfs.bucketInit.image.pullPolicy }}
command:
- /bin/sh
- -ec
- |
until mc alias set local '{{ .Values.rustfs.bucketInit.endpoint }}' '{{ $accessKey }}' '{{ $secretKey }}'; do
echo "waiting for rustfs endpoint..."
sleep 2
done
{{- range $bucket := .Values.rustfs.defaultBuckets }}
mc mb --ignore-existing local/{{ $bucket }}
{{- end }}
{{- end }}
85 changes: 0 additions & 85 deletions charts/flyte-demo/templates/storage/rustfs/deployment.yaml
View file
Open in desktop

This file was deleted.

17 changes: 0 additions & 17 deletions charts/flyte-demo/templates/storage/rustfs/pv.yaml
View file
Open in desktop

This file was deleted.

17 changes: 0 additions & 17 deletions charts/flyte-demo/templates/storage/rustfs/pvc.yaml
View file
Open in desktop

This file was deleted.

13 changes: 0 additions & 13 deletions charts/flyte-demo/templates/storage/rustfs/secret.yaml
View file
Open in desktop

This file was deleted.

19 changes: 0 additions & 19 deletions charts/flyte-demo/templates/storage/rustfs/service.yaml
View file
Open in desktop

This file was deleted.

35 changes: 31 additions & 4 deletions charts/flyte-demo/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ flyte-binary:
s3:
disableSSL: true
v2Signing: true
endpoint: http://rustfs.{{ .Release.Namespace }}:9000
Copy link
Copy Markdown
Contributor

@popojk popojk Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we keep using rustfs.{{ .Release.Namespace }} as service name instead of hard coding?

endpoint: http://rustfs-svc:9000
authType: accesskey
accessKey: rustfs
secretKey: rustfsstorage
Expand All @@ -52,7 +52,7 @@ flyte-binary:
plugins:
k8s:
default-env-vars:
- FLYTE_AWS_ENDPOINT: http://rustfs.{{ .Release.Namespace }}:9000
- FLYTE_AWS_ENDPOINT: http://rustfs-svc:9000
- FLYTE_AWS_ACCESS_KEY_ID: rustfs
- FLYTE_AWS_SECRET_ACCESS_KEY: rustfsstorage
- _U_EP_OVERRIDE: 'flyte-binary-http.{{ .Release.Namespace }}:8090'
Expand Down Expand Up @@ -111,8 +111,35 @@ flyte-binary:

rustfs:
enabled: true
accessKey: rustfs
secretKey: rustfsstorage
bucketInit:
enabled: true
endpoint: http://rustfs-svc:9000
image:
repository: minio/mc
tag: latest
pullPolicy: IfNotPresent
defaultBuckets:
- flyte-data

# Upstream RustFS chart configuration.
rustfsupstream:
fullnameOverride: rustfs
mode:
standalone:
enabled: true
distributed:
enabled: false
image:
repository: rustfs/rustfs
tag: sandbox
service:
type: NodePort
endpoint:
nodePort: 30002
secret:
rustfs:
access_key: rustfs
secret_key: rustfsstorage
Copy link
Copy Markdown
Contributor

@popojk popojk Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that there is a rustfs test connection pod remain in the cluster, which is not required because the sandbox should failed to startup if connection failed. We can do so by:

test:
      enabled: false


postgresql:
fullnameOverride: postgresql
Expand Down
3 changes: 2 additions & 1 deletion docker/demo-bundled/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ flyte: create_builder
helm-repos:
helm repo add docker-registry https://twuni.github.io/docker-registry.helm
helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo add rustfs https://rustfs.github.io/helm
helm repo update

.PHONY: dep_build
Expand All @@ -51,7 +52,7 @@ manifests: dep_update
--load-restrictor=LoadRestrictionsNone \
kustomize/dev > manifests/dev.yaml

CONSOLE_IMAGE := ghcr.io/flyteorg/flyte-client-v2
CONSOLE_IMAGE := ghcr.io/unionai-oss/flyteconsole-v2
Copy link
Copy Markdown
Contributor

@popojk popojk Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pingsutw I'm not sure if we will pull console image from ghcr.io/unionai-oss in the future? Should we do this fix here?

Copy link
Copy Markdown
Member

@pingsutw pingsutw Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I already changed the image name to ghcr.io/unionai-oss/flyteconsole-v2 in another PR

CONSOLE_TAG := latest

.PHONY: console
Expand Down
1 change: 1 addition & 0 deletions docker/demo-bundled/images/manifest.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,5 @@ docker.io/rancher/local-path-provisioner:v0.0.21
docker.io/rancher/mirrored-coredns-coredns:1.9.1
docker.io/rancher/mirrored-library-busybox:1.34.1
docker.io/rancher/mirrored-pause:3.6
docker.io/minio/mc:latest
docker.io/rustfs/rustfs:sandbox=rustfs/rustfs:latest
Loading
Loading