[Flyte-7005] Supporting Ray autoscalerOptions by 0yukali0 · Pull Request #7111 · flyteorg/flyte

0yukali0 · 2026-03-30T03:09:30Z

Tracking issue

Related #7005

Why are the changes needed?

Ray autoScalerOptions is not supportted in flyte v1

What changes were proposed in this pull request?

Add autoscalerOptions to ray.proto
Generate go files of grpc proto
Parsing the options from grpc and translate it to crd of ray operator.

How was this patch tested?

Labels

Please add one or more of the following labels to categorize your PR:

added: For new features.
changed: For changes in existing functionality.
deprecated: For soon-to-be-removed features.
removed: For features being removed.
fixed: For any bug fixed.
security: In case of vulnerabilities

This is important to improve the readability of release notes.

Setup process

clone flyte and flytekit (branch flyte-7005 and flytekit-7005)
run the flyte backend in sandbox
2.1 flytectl demo start --dev
2.2 make compile && POD_NAMESPACE=flyte ./flyte start --config flyte-single-binary-local.yaml
Update ray example like Ray example
3.1 uv venv -p 3.11 && source .venv/bin/activate && uv pip install -e ~/flytekit ~/flytekit/plugins/flytekit-ray && uv pip install -e ~/flyte/flyteidl
3.2 "pyflyte run --remote ray_example.py ray_workflow --n 10"

Screenshots

I updated the documentation accordingly.
All new and existing tests passed.
All commits are signed-off.

Related PRs

Docs link

codecov · 2026-03-30T03:13:10Z

Codecov Report

❌ Patch coverage is 89.18919% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 56.97%. Comparing base (19daf37) to head (f4429c9).
⚠️ Report is 11 commits behind head on master.

Files with missing lines	Patch %	Lines
flyteplugins/go/tasks/plugins/k8s/ray/ray.go	89.18%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7111      +/-   ##
==========================================
+ Coverage   56.95%   56.97%   +0.01%     
==========================================
  Files         931      931              
  Lines       58188    58283      +95     
==========================================
+ Hits        33143    33208      +65     
- Misses      22004    22021      +17     
- Partials     3041     3054      +13

Flag	Coverage Δ
unittests-datacatalog	`53.51% <ø> (ø)`
unittests-flyteadmin	`53.14% <ø> (ø)`
unittests-flytecopilot	`43.06% <ø> (ø)`
unittests-flytectl	`64.09% <ø> (+0.07%)`	⬆️
unittests-flyteidl	`75.71% <ø> (ø)`
unittests-flyteplugins	`60.27% <89.18%> (+0.10%)`	⬆️
unittests-flytepropeller	`53.71% <ø> (+0.06%)`	⬆️
unittests-flytestdlib	`62.61% <ø> (-0.42%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

machichima · 2026-04-01T09:01:26Z

  string runtime_env_yaml = 5;
 }

+message Resources {


Can we use core.Resources?

flyte/flyteidl/protos/flyteidl/core/tasks.proto

Line 17 in e5905e7

message Resources {

machichima · 2026-04-01T09:14:25Z

+message EnvVar {
+  string name = 1;
+  string value = 2;
+  EnvValueFrom valueFrom = 3;


Why do we need EnvValueFrom here? Can we use core.KeyValuePair as the type for env?

flyte/flyteidl/protos/flyteidl/core/literals.proto

Lines 192 to 198 in e5905e7

message KeyValuePair {

//required.

string key = 1;

//+optional.

string value = 2;

}

machichima · 2026-04-01T09:46:26Z

+
+message AutoscalerOptions {
+  // "Default", "Aggressive", "Conservative"
+  string upscaling_mode = 1;


Could we make this as enum?

enum UpscalingMode { UPSCALING_MODE_UNSPECIFIED = 0; UPSCALING_MODE_DEFAULT = 1; UPSCALING_MODE_AGGRESSIVE = 2; UPSCALING_MODE_CONSERVATIVE = 3; }

machichima · 2026-04-01T09:48:39Z

 	return rayjob, err
 }

+func NewAutoscalerOptions(options *plugins.AutoscalerOptions) *rayv1.AutoscalerOptions {


Let's name it as buildAutoscalerOptions to follow the naming convention (e.g. like buildHeadPodTemplate), and make it private

machichima · 2026-04-01T09:54:19Z

 		},
 		WorkerGroupSpecs:        []rayv1.WorkerGroupSpec{},
 		EnableInTreeAutoscaling: &rayJob.RayCluster.EnableAutoscaling,
+		AutoscalerOptions:       autoScalarOptions,


While protobuf generated GetXXX() has nil check, we can directly pass:

AutoscalerOptions: NewAutoscalerOptions(rayJob.GetRayCluster().GetAutoscalerOptions())

machichima · 2026-04-01T09:56:04Z

Hi @0yukali0,
Could you briefly describe how you test and provide the test script?
Thank you!

0yukali0 · 2026-04-02T13:47:50Z

Hi @machichima , i updated the setup steps and screenshot in the PR descriptions.
i am going to add the resources field in headspec and workgroupspec to replace requests and limits in next commit.

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

0yukali0 · 2026-04-11T01:43:57Z

with kuberay 1.1.0 and 1.5.1，run ray task with 2.54.1 is worked.

machichima · 2026-04-21T01:54:25Z

+			resourceRequirements := v1.ResourceRequirements{}
+			if requests := res.GetRequests(); len(requests) > 0 {
+				resourceRequirements.Requests = convertResourceEntriesToResourceList(requests)
+			}
+			if limits := res.GetLimits(); len(limits) > 0 {
+				resourceRequirements.Limits = convertResourceEntriesToResourceList(limits)
+			}


Could we use flytek8s.ToK8sResourceRequirements here instead?

flyte/flyteplugins/go/tasks/pluginmachinery/flytek8s/utils.go

Lines 52 to 68 in 6e84ac2

func ToK8sResourceRequirements(resources *core.Resources) (*v1.ResourceRequirements, error) {

res := &v1.ResourceRequirements{}

if resources == nil {

return res, nil

}

req, err := ToK8sResourceList(resources.GetRequests())

if err != nil {

return res, err

}

lim, err := ToK8sResourceList(resources.GetLimits())

if err != nil {

return res, err

}

res.Limits = lim

res.Requests = req

return res, nil

}

machichima · 2026-04-21T01:56:01Z

+			autoScalarOptions.Env = []v1.EnvVar{}
+			for _, env := range envs {
+				name := env.GetKey()
+				if val := env.GetValue(); val != "" {
+					autoScalarOptions.Env = append(autoScalarOptions.Env, v1.EnvVar{
+						Name:  name,
+						Value: val,
+					})
+				}
+			}


Could we use flytek8s.ToK8sEnvVar instead?

flyte/flyteplugins/go/tasks/pluginmachinery/flytek8s/utils.go

Line 12 in 6e84ac2

func ToK8sEnvVar(env []*core.KeyValuePair) []v1.EnvVar {

machichima · 2026-04-21T01:59:27Z

+		idleTimeoutTime := options.GetIdleTimeoutSeconds()
+		autoScalarOptions.IdleTimeoutSeconds = &idleTimeoutTime


If the idleTimeoutTime is not set (= 0), this part will overwrite the default value (60 sec) set in kuberay. we should only set autoScalarOptions.IdleTimeoutSeconds if it's not 0 (not empty)

https://github.com/ray-project/kuberay/blob/0706043ae00a0dedfadead423926136e01daaef9/proto/cluster.proto#L145-L147

https://github.com/ray-project/ray/blob/db711158e25231b77ef2fb968320b2319270732d/python/ray/autoscaler/_private/kuberay/autoscaling_config.py#L108-L111

Hi @machichima
I will update it and the following test case to check the setting.

machichima · 2026-04-21T02:01:18Z

 }

+func buildAutoscalerOptions(options *plugins.AutoscalerOptions) *rayv1.AutoscalerOptions {
+	var autoScalarOptions *rayv1.AutoscalerOptions


Suggested change

var autoScalarOptions *rayv1.AutoscalerOptions

var autoScalerOptions *rayv1.AutoscalerOptions

nit

machichima · 2026-04-21T02:03:13Z

+	}
+}
+
+func TestNewAutoscalerOptions(t *testing.T) {


Suggested change

func TestNewAutoscalerOptions(t *testing.T) {

func TestBuildAutoscalerOptions(t *testing.T) {

nit

machichima · 2026-04-21T02:05:35Z

+		assert.Equal(t, resource.MustParse("1Gi"), result.Resources.Limits[corev1.ResourceMemory])
+	})
+
+	t.Run("env literal value", func(t *testing.T) {


Could we add a test for env with empty value (should be skipped)?

flytek8s.ToK8sEnvVar allows empty value, so i will skip the test with empty value.

machichima · 2026-04-21T02:14:54Z

+		assert.Nil(t, buildAutoscalerOptions(nil))
+	})
+
+	t.Run("idle timeout propagated", func(t *testing.T) {


Could we also add a test like this:

t.Run("idle timeout zero should not be set", func(t *testing.T) { result := buildAutoscalerOptions(&plugins.AutoscalerOptions{}) require.NotNil(t, result) assert.Nil(t, result.IdleTimeoutSeconds) })

Which is related to my comment above: https://github.com/flyteorg/flyte/pull/7111/changes#r3114611720
If the IdleTimeoutSeconds is not set, it should remain empty so that Ray can apply its default value

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

0yukali0 force-pushed the flyte-7005 branch 2 times, most recently from 951fbeb to 8a99445 Compare March 31, 2026 14:07

0yukali0 marked this pull request as ready for review March 31, 2026 14:45

0yukali0 force-pushed the flyte-7005 branch from 97559c4 to e5905e7 Compare April 1, 2026 07:32

machichima reviewed Apr 1, 2026

View reviewed changes

0yukali0 force-pushed the flyte-7005 branch from 57f9ebf to c9c99d1 Compare April 2, 2026 03:32

0yukali0 added 13 commits April 6, 2026 14:19

Add autoscaleroptions of proto files

3f3acef

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

fix EnvVar

9eaead8

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

fix resource

57cfe74

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

generate go from proto

841ae94

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

Add options to ray cluster request

7ed7786

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

update proto

4faca42

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

envvar added

15163b5

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

add resource

5679b2b

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

meeting lint

3267152

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

add unit test of new functions

92526f3

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

remvoe envfrom

ce0fceb

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

remote envfrom condition and test

e21d69c

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

regen and fix bugs

6e84ac2

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

0yukali0 force-pushed the flyte-7005 branch from c9c99d1 to 6e84ac2 Compare April 6, 2026 06:19

machichima reviewed Apr 21, 2026

View reviewed changes

0yukali0 added 2 commits April 21, 2026 21:36

set idletime if it is required

4bb66fb

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

adopting k8s utils function

f4429c9

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

	message KeyValuePair {
	//required.
	string key = 1;

	//+optional.
	string value = 2;
	}

	func ToK8sResourceRequirements(resources core.Resources) (v1.ResourceRequirements, error) {
	res := &v1.ResourceRequirements{}
	if resources == nil {
	return res, nil
	}
	req, err := ToK8sResourceList(resources.GetRequests())
	if err != nil {
	return res, err
	}
	lim, err := ToK8sResourceList(resources.GetLimits())
	if err != nil {
	return res, err
	}
	res.Limits = lim
	res.Requests = req
	return res, nil
	}

		idleTimeoutTime := options.GetIdleTimeoutSeconds()
		autoScalarOptions.IdleTimeoutSeconds = &idleTimeoutTime

	var autoScalarOptions *rayv1.AutoscalerOptions
	var autoScalerOptions *rayv1.AutoscalerOptions

	func TestNewAutoscalerOptions(t *testing.T) {
	func TestBuildAutoscalerOptions(t *testing.T) {

Conversation

0yukali0 commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Tracking issue

Related #7005

Why are the changes needed?

What changes were proposed in this pull request?

How was this patch tested?

Labels

Setup process

Screenshots

Related PRs

Docs link

Uh oh!

codecov bot commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

machichima commented Apr 1, 2026

Uh oh!

0yukali0 commented Apr 2, 2026

Uh oh!

0yukali0 commented Apr 11, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

0yukali0 commented Mar 30, 2026 •

edited

Loading

codecov bot commented Mar 30, 2026 •

edited

Loading