Skip to content

[Bug]: GFD generates [nvidia.com/gpu.product](https://nvidia.com/gpu.product) label exceeding Kubernetes 63-character limit with long MIG profile names #2558

@linlingo1222-del

Description

@linlingo1222-del

Environment:
GPU Operator Version: v25.10.1

NFD Version: v0.18.2 (or as packaged in Operator v25.10.1)

GPU Hardware: NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition

MIG Strategy: Single / Mixed (with MIG profile 1g.24gb)

Describe the bug:
When using a GPU with an exceptionally long product name combined with a MIG profile, gpu-feature-discovery (GFD) generates a label value that exceeds the Kubernetes maximum length limit of 63 characters for label values.

As a result, Node Feature Discovery (NFD) ignores the label entirely, preventing the nvidia.com/gpu.product label from being applied to the node. This breaks node selectors that rely on specific GPU product types for pod scheduling.

Steps to reproduce:
Provision a node with a GPU that has a long product name (e.g., NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition).

Enable MIG and configure a profile (e.g., 1g.24gb).

Deploy GPU Operator v25.10.1.

Inspect the NFD worker logs.

Expected behavior:
GFD should sanitize, truncate, or alias the generated string for nvidia.com/gpu.product before writing it to the features file, ensuring the final value length is <= 63 characters to comply with Kubernetes API constraints.

Actual behavior / Logs:
GFD writes the full 67-character string to /etc/kubernetes/node-feature-discovery/features.d/gfd. NFD then rejects it with the following error in the nfd-worker pod logs:

I0618 08:41:52.386491       1 nfd-worker.go:595] "ignoring label with invalid value" labelKey="nvidia.com/gpu.product" labelValue="NVIDIA-RTX-PRO-6000-Blackwell-Max-Q-Workstation-Edition-MIG-1g.24gb" errors=["must be no more than 63 characters"]

Suggested Fix:
Implement a truncation logic or hashing suffix in GFD when the concatenated string of [GPU_NAME]-[MIG_PROFILE] exceeds 63 characters.

Metadata

Metadata

Labels

bugIssue/PR to expose/discuss/fix a bugneeds-triageissue or PR has not been assigned a priority-px label

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions