What problem are you facing?
#306 fixed GPU pools cold-starting from zero on GKE. That fix works because GKE's autoscaler has custom support for DRA: it recognises the cloud.google.com/gke-nvidia-gpu-dra-driver=true node label and models the ResourceSlices a not-yet-existing node would publish, so it can tell a scaled-to-zero pool would satisfy a GPU ResourceClaim. Nebius's autoscaler appears to have similar custom support.
The upstream Kubernetes cluster-autoscaler doesn't have this in general. Scale-from-zero relies on each cloud provider's TemplateNodeInfo() synthesising the node a pool would create, and only the cluster-api provider models DRA ResourceSlices there (via the capacity.cluster-autoscaler.kubernetes.io/dra-driver annotation added in kubernetes/autoscaler#7804). The AWS provider passes nil slices, so an EKS GPU pool at zero presents no DRA capacity to the autoscaler and never wakes up. Scaling 1→N works because the autoscaler templates off a real node's live slices; 0→1 doesn't. This is tracked upstream, broadly, in kubernetes/autoscaler#7799.
We can't sidestep this on EKS with Karpenter or EKS Auto Mode, because neither supports DRA yet, which is why we run the vanilla cluster-autoscaler. So any Modelplane cluster on a provider without custom autoscaler DRA support — EKS today, and any self-managed cluster running the stock cluster-autoscaler — can't scale GPU pools to zero.
The immediate user-facing problem: the InferenceCluster API accepts minNodeCount: 0 on GPU pools regardless of provider, but on these providers a pool set to zero silently stays pending with cannot allocate all claims instead of cold-starting. Nothing tells the user it won't work.
How could Modelplane help solve your problem?
Mostly this issue is to track the limitation.
In the meantime, we could stop the API silently accepting a configuration that won't work, e.g. reject or warn on minNodeCount: 0 for GPU pools on providers we know don't support it, and document per-provider support.
What problem are you facing?
#306 fixed GPU pools cold-starting from zero on GKE. That fix works because GKE's autoscaler has custom support for DRA: it recognises the
cloud.google.com/gke-nvidia-gpu-dra-driver=truenode label and models theResourceSlicesa not-yet-existing node would publish, so it can tell a scaled-to-zero pool would satisfy a GPUResourceClaim. Nebius's autoscaler appears to have similar custom support.The upstream Kubernetes cluster-autoscaler doesn't have this in general. Scale-from-zero relies on each cloud provider's
TemplateNodeInfo()synthesising the node a pool would create, and only the cluster-api provider models DRAResourceSlicesthere (via thecapacity.cluster-autoscaler.kubernetes.io/dra-driverannotation added in kubernetes/autoscaler#7804). The AWS provider passesnilslices, so an EKS GPU pool at zero presents no DRA capacity to the autoscaler and never wakes up. Scaling1→Nworks because the autoscaler templates off a real node's live slices;0→1doesn't. This is tracked upstream, broadly, in kubernetes/autoscaler#7799.We can't sidestep this on EKS with Karpenter or EKS Auto Mode, because neither supports DRA yet, which is why we run the vanilla cluster-autoscaler. So any Modelplane cluster on a provider without custom autoscaler DRA support — EKS today, and any self-managed cluster running the stock cluster-autoscaler — can't scale GPU pools to zero.
The immediate user-facing problem: the
InferenceClusterAPI acceptsminNodeCount: 0on GPU pools regardless of provider, but on these providers a pool set to zero silently stays pending withcannot allocate all claimsinstead of cold-starting. Nothing tells the user it won't work.How could Modelplane help solve your problem?
Mostly this issue is to track the limitation.
In the meantime, we could stop the API silently accepting a configuration that won't work, e.g. reject or warn on
minNodeCount: 0for GPU pools on providers we know don't support it, and document per-provider support.