Skip to content

fix(provider/dra): match nodes by annotated instance ID#405

Open
dmitsh wants to merge 1 commit into
mainfrom
ds-dra
Open

fix(provider/dra): match nodes by annotated instance ID#405
dmitsh wants to merge 1 commit into
mainfrom
ds-dra

Conversation

@dmitsh

@dmitsh dmitsh commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

No description provided.

@dmitsh
dmitsh requested a review from ravisoundar as a code owner July 16, 2026 12:29
@dmitsh
dmitsh requested a review from bugra-gedik-nv July 16, 2026 12:29
@greptile-apps

greptile-apps Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes the DRA provider's node-matching logic to use the topograph.nvidia.com/instance annotation rather than the raw Kubernetes node name when looking up nodes in the compute instances map. It also upgrades the client field from *kubernetes.Clientset to kubernetes.Interface, enabling the new integration test to exercise the full GenerateTopologyConfig path with a fake client.

  • provider.go: GenerateTopologyConfig now reads topology.KeyNodeInstance from node annotations and skips any node where that annotation is absent or empty, then uses the annotation value as the lookup key in the instances map.
  • provider_test.go: New test TestGenerateTopologyConfigUsesAnnotatedInstanceID verifies that a node whose K8s name (k8s-node-1) differs from its annotated instance ID (instance-123) is correctly resolved — a case the old node.Name-based lookup would have silently dropped.
  • provider.goGetNodeAnnotations: This helper still maps topology.KeyNodeInstance to the K8s node name for DRA deployments running node-data-broker. That is backward-compatible for clusters where node names equal instance IDs, but deployments where they differ will need GetNodeAnnotations updated (or the annotation set by another mechanism) to fully benefit from this fix.

Confidence Score: 5/5

Safe to merge — the change is logically correct, backward-compatible for clusters where node names equal instance IDs, and accompanied by a meaningful integration test.

The annotation-based lookup is implemented cleanly with an explicit guard for missing/empty annotations, and the interface upgrade to kubernetes.Interface is a well-established testability improvement. The new test exercises exactly the divergence case that motivated the fix. The only nuance is that GetNodeAnnotations still writes the K8s node name as the instance annotation, so clusters where node names differ from cloud instance IDs must set the annotation through another mechanism — but that is an upstream concern outside this PR's scope.

No files require special attention.

Important Files Changed

Filename Overview
pkg/providers/dra/provider.go Core fix: replaces node.Name lookup with annotation-based instance ID matching; also upgrades client field from *kubernetes.Clientset to kubernetes.Interface for testability
pkg/providers/dra/provider_test.go Adds TestGenerateTopologyConfigUsesAnnotatedInstanceID using fake.NewSimpleClientset; correctly validates that annotation-keyed instance lookup succeeds where node.Name lookup would have failed
CHANGELOG.md Adds changelog entry for the DRA annotation-based matching fix

Reviews (4): Last reviewed commit: "fix(provider/dra): match nodes by annota..." | Re-trigger Greptile

@codecov

codecov Bot commented Jul 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 60.00000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.59%. Comparing base (8468abe) to head (ed84e66).
⚠️ Report is 23 commits behind head on main.

Files with missing lines Patch % Lines
pkg/providers/dra/provider.go 60.00% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #405      +/-   ##
==========================================
+ Coverage   72.15%   73.59%   +1.44%     
==========================================
  Files          89       94       +5     
  Lines        5689     6272     +583     
==========================================
+ Hits         4105     4616     +511     
- Misses       1382     1410      +28     
- Partials      202      246      +44     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@dmitsh
dmitsh force-pushed the ds-dra branch 2 times, most recently from 36ef80a to 23561d5 Compare July 19, 2026 08:13
@github-actions

Copy link
Copy Markdown
Contributor

Signed-off-by: Dmitry Shmulevich <dshmulevich@nvidia.com>

@ArangoGutierrez ArangoGutierrez left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct fix — DRA was the last provider still joining on node.Name, and the new test genuinely discriminates it (node name differs from the instance ID). One optional diagnostic suggestion below.

  1. A clique-labeled node missing this annotation is now skipped silently. The sibling join sites warn on the same skip — internal/k8s/utils.go and engines/slinky/engine.go both do klog.Warningf("missing %q annotation in node %s", ...), and #379 added exactly this diagnostic to the structurally identical clique+annotation loop in Slinky. A klog.Warningf before the continue would restore that signal; otherwise a not-yet-annotated node disappears with no trace, since the BadGateway error only fires when every node is skipped. (pkg/providers/dra/provider.go:113)

if host, ok := i2n[node.Name]; ok {
domainMap.AddHost(clusterID, node.Name, host)
instanceID, ok := node.Annotations[topology.KeyNodeInstance]
if !ok || instanceID == "" {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A clique-labeled node missing this annotation is now skipped silently. The sibling join sites warn on the same skip — internal/k8s/utils.go and engines/slinky/engine.go both do klog.Warningf("missing %q annotation in node %s", ...), and #379 added exactly this diagnostic to the structurally identical clique+annotation loop in Slinky. A klog.Warningf before the continue would restore that signal; otherwise a not-yet-annotated node disappears with no trace, since the BadGateway error only fires when every node is skipped.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants