Merged
Conversation
When set, passes `--gpus all` to the underlying `docker run` so the demo
cluster can use host NVIDIA GPUs. Requires an NVIDIA-enabled host and a
GPU-capable demo image (e.g. flyte-demo:gpu-latest, being added in
flyteorg/flyte). Default off; existing non-GPU users are unaffected.
Usage:
flyte start demo --gpu --image ghcr.io/flyteorg/flyte-demo:gpu-latest
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
kumare3
reviewed
Apr 21, 2026
| "--volume", | ||
| f"{volume_name}:/var/lib/flyte/storage", | ||
| ] | ||
| if gpu: |
Contributor
There was a problem hiding this comment.
i think we should add some unit tests for these? Also how about an alias devbox
Member
Author
There was a problem hiding this comment.
alias devbox
will change it in a separate PR
On Linux bind mounts, the in-container kubeconfig lands root-owned on the host and kubectl exits non-zero, which surfaces as CalledProcessError rather than PermissionError, so the existing chown-retry branch never fired. macOS avoided this because Docker Desktop remaps ownership.
- _run_container: asserts --gpus all is appended only when gpu=True - _merge_kubeconfig: asserts chown retry fires on both PermissionError and CalledProcessError (the Linux bind-mount case this PR fixes), and that the second failure propagates - demo CLI: asserts --gpu is plumbed through to launch_demo
If the user passes --gpu without --image, use ghcr.io/flyteorg/flyte-demo:gpu-latest rather than the CPU default. An explicit --image is still respected.
# Conflicts: # src/flyte/cli/_start.py
kumare3
approved these changes
Apr 21, 2026
cosmicBboy
approved these changes
Apr 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
--gputoflyte start demo. When set, the underlyingdocker runis invoked with--gpus all, giving the demo container access to host NVIDIA GPUs.Default off — existing non-GPU users are unaffected.
Why
The GPU-capable demo image being added in flyteorg/flyte#7243 configures k3s, containerd, and the NVIDIA device plugin inside the image, but none of that matters if the host GPUs aren't passed through to the container.
--gpus allon thedocker runis the piece the CLI owns.Change
_start.py— new--gpuclick option on thedemocommand._demo.py— threadsgpu: boolthroughlaunch_demo → _launch_demo_rich/plain → _run_step → _run_container, which appends--gpus allwhen true.Test plan
uv tool install --from . flyte;flyte start demo --helplists the new flag.flyte start demo --gpu --image flyte-demo:gpu-localon an A10G host launched the container with--gpus all, a Flyte task withResources(gpu=1)rantorch.cuda.is_available() == True.