Skip to content

fix: treat 0/0 release targets as 100% pass rate in policy evaluation #1049

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
adityachoudhari26 merged 3 commits into from
Apr 23, 2026
Merged

fix: treat 0/0 release targets as 100% pass rate in policy evaluation #1049

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5d3965e
fix: treat 0/0 release targets as 100% pass rate in policy evaluation
github-actions[bot] Apr 22, 2026
bd15aa8
fixes
adityachoudhari26 Apr 23, 2026
ec90dd9
lint
adityachoudhari26 Apr 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions ...engine/pkg/workspace/releasemanager/policy/evaluator/environmentprogression/jobtracker.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -196,8 +196,8 @@ func (t *ReleaseTargetJobTracker) GetSuccessPercentage() float32 {

numRt := len(t.ReleaseTargets)
if numRt == 0 {
span.SetAttributes(attribute.Float64("success_percentage", 0.0))
return 0.0 // If no targets, consider it 100% successful
span.SetAttributes(attribute.Float64("success_percentage", 100.0))
return 100.0 // vacuous truth: 0/0 targets successful
Comment on lines +199 to +200
Copy link

Copilot AI Apr 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing the 0-target success percentage to 100% will flip PassRateEvaluator behavior for the no-release-targets case. There is an existing unit test (TestPassRateEvaluator_NoReleaseTargets in passrate_test.go) that currently asserts the opposite and will fail, and the minimumSuccessPercentage==0 branch in PassRateEvaluator will now treat a 0-target environment as allowed but set satisfiedAt based on GetEarliestSuccess() (which is zero time when there are no jobs). Consider updating PassRateEvaluator to explicitly handle len(tracker.ReleaseTargets)==0 (e.g., allow and set satisfiedAt to tracker.Version.CreatedAt) and adjust the related tests accordingly.

Suggested change
span.SetAttributes(attribute.Float64("success_percentage", 100.0))
return 100.0 // vacuous truth: 0/0 targets successful
span.SetAttributes(attribute.Float64("success_percentage", 0.0))
return 0.0 // no release targets to evaluate

Copilot uses AI. Check for mistakes.
}

// Build a set of release target keys for filtering
Expand Down Expand Up @@ -242,7 +242,7 @@ func (t *ReleaseTargetJobTracker) GetSuccessPercentageSatisfiedAt(
}
numRt := len(t.ReleaseTargets)
if numRt == 0 {
return time.Time{}
return t.Version.CreatedAt
}
Comment on lines 244 to 246
Copy link

Copilot AI Apr 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GetSuccessPercentageSatisfiedAt now returns t.Version.CreatedAt when there are no release targets. The job tracker unit test TestReleaseTargetJobTracker_GetSuccessPercentageSatisfiedAt_NoReleaseTargets (jobtracker_test.go) currently expects zero time and will fail with this new behavior; it should be updated to assert Version.CreatedAt (or whatever semantics you intend for satisfiedAt in the 0-target case).

Copilot uses AI. Check for mistakes.

// Build a set of release target keys for filtering
Expand Down
17 changes: 14 additions & 3 deletions ...e/pkg/workspace/releasemanager/policy/evaluator/environmentprogression/jobtracker_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,13 @@ func TestReleaseTargetJobTracker_GetSuccessPercentage_NoTargets(t *testing.T) {
tracker := NewReleaseTargetJobTracker(ctx, mock, env, version, nil, false)

percentage := tracker.GetSuccessPercentage()
assert.InDelta(t, float32(0.0), percentage, 0, "expected 0%% success with no targets")
assert.InDelta(
t,
float32(100.0),
percentage,
0,
"expected 100%% success with no targets (vacuous truth)",
)
}

func TestReleaseTargetJobTracker_GetSuccessPercentage_WithSuccesses(t *testing.T) {
Expand Down Expand Up @@ -941,9 +947,14 @@ func TestReleaseTargetJobTracker_GetSuccessPercentageSatisfiedAt_NoReleaseTarget
tracker := NewReleaseTargetJobTracker(ctx, mock, env, version, nil, false)
tracker.ReleaseTargets = []oapi.ReleaseTarget{}

// With no release targets, should return zero time
// Vacuous truth: 0/0 targets successful is treated as 100% pass, satisfied at version creation.
satisfiedAt := tracker.GetSuccessPercentageSatisfiedAt(50.0)
assert.True(t, satisfiedAt.IsZero(), "expected zero satisfiedAt with no release targets")
assert.Equal(
t,
version.CreatedAt,
satisfiedAt,
"expected satisfiedAt to equal version.CreatedAt with no release targets",
)
}

func TestReleaseTargetJobTracker_GetSuccessPercentageSatisfiedAt_NoSuccessfulJobs(t *testing.T) {
Expand Down
6 changes: 3 additions & 3 deletions ...ine/pkg/workspace/releasemanager/policy/evaluator/environmentprogression/passrate_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -461,9 +461,9 @@ func TestPassRateEvaluator_NoReleaseTargets(t *testing.T) {
}
result := eval.Evaluate(ctx, scope)

// With no release targets, success percentage is 0%
assert.False(t, result.Allowed, "expected denied with no release targets")
assert.Contains(t, result.Message, "Success rate 0.0% below required 50.0%")
// Vacuous truth: 0/0 release targets is treated as 100% pass rate.
assert.True(t, result.Allowed, "expected allowed with no release targets (vacuous truth)")
assert.Contains(t, result.Message, "Success rate 100.0% meets required 50.0%")
}

// TestPassRateEvaluator_CustomSuccessStatuses tests that custom success statuses can be used.
Expand Down
Loading