Skip to content

ci: guard empty matrix categories in integration test workflow#2165

Open
musa-asad wants to merge 2 commits into
mainfrom
fix/integ-matrix-empty-guard
Open

ci: guard empty matrix categories in integration test workflow#2165
musa-asad wants to merge 2 commits into
mainfrom
fix/integ-matrix-empty-guard

Conversation

@musa-asad

@musa-asad musa-asad commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Background

The CloudWatch Agent integration tests run in GitHub Actions. A single workflow (test-artifacts.yml) generates test matrices: one per platform category (EC2 Linux, Windows, Darwin, ECS, EKS, etc.): and feeds each matrix to a downstream job that spins up the test infrastructure.

The workflow supports filtered runs: a human can dispatch it with test_dir_filter or test_os_filter to run only a subset of tests. When a filter is applied, categories that have no matching tests are reduced to an empty array [].

Problem

GitHub Actions requires every matrix to contain at least one entry. When a filtered dispatch empties a category, the downstream job tries to expand fromJson('[]') as a matrix and fails immediately:

Error when evaluating 'strategy' for job 'EC2WinIntegrationTest':
Matrix vector 'arrays' does not contain any values

The job is never created, the workflow run is marked failure, and the error message gives no indication that the failure is expected (the filter simply did not match any tests in that category).

This affects every matrix-consumer job that lacks an emptiness guard: 15 jobs in total across all platform categories. Unfiltered runs (cron, bot) are not affected because all categories stay populated.

Changes

test-artifacts.yml: one line added per job:

if: ${{ needs.GenerateTestMatrix.outputs.<category>_matrix != '[]' }}

When the matrix is empty, the job is cleanly skipped (green checkmark, no error). When the matrix is populated, the guard evaluates to true and has no effect.

Jobs guarded:

Job Matrix variable
EC2LinuxIntegrationTest-0 ec2_linux_matrix_0
EC2NvidiaGPUIntegrationTest ec2_gpu_matrix
LinuxOnPremIntegrationTest ec2_linux_onprem_matrix
EC2SELinuxIntegrationTest ec2_selinux_matrix
EC2LinuxIntegrationTestITAR ec2_linux_itar_matrix
EC2LinuxIntegrationTestCN ec2_linux_china_matrix
EC2WinIntegrationTest ec2_windows_matrix
EC2DarwinIntegrationTest ec2_mac_matrix
ECSEC2IntegrationTest ecs_ec2_launch_daemon_matrix
ECSFargateIntegrationTest ecs_fargate_matrix
EKSIntegrationTest eks_daemon_matrix
EKSPrometheusIntegrationTest eks_deployment_matrix
PerformanceTrackingTest ec2_performance_matrix
EC2WinPerformanceTest ec2_windows_performance_matrix
StressTrackingTest ec2_stress_matrix
EC2WinStressTrackingTest ec2_windows_stress_matrix
GPUEndToEndTest eks_addon_matrix

Also fixed: LinuxOnPremIntegrationTest was missing StartLocalStack in its needs list, causing it to reference a LocalStack output that was not yet available.

Testing

$ grep -c "!= '[]'" .github/workflows/test-artifacts.yml
20
$ python3 -c "import yaml; yaml.safe_load(open('.github/workflows/test-artifacts.yml')); print('YAML OK')"
YAML OK

Validated on a full PR Test run: 318 jobs pass, no matrix expansion failures.

@musa-asad musa-asad requested a review from a team as a code owner June 18, 2026 22:03
@musa-asad musa-asad added the ready for testing Indicates this PR is ready for integration tests to run label Jun 19, 2026
@musa-asad musa-asad force-pushed the fix/integ-matrix-empty-guard branch from 443bd54 to 66b2693 Compare June 22, 2026 18:33
@musa-asad musa-asad marked this pull request as draft June 22, 2026 22:59
@musa-asad musa-asad force-pushed the fix/integ-matrix-empty-guard branch 5 times, most recently from 53dfd05 to 9105d88 Compare June 23, 2026 04:37
@musa-asad musa-asad force-pushed the fix/integ-matrix-empty-guard branch from 9105d88 to 6a58a2f Compare June 23, 2026 04:44
@musa-asad musa-asad self-assigned this Jun 23, 2026
@musa-asad musa-asad marked this pull request as ready for review June 23, 2026 04:57
@musa-asad musa-asad requested review from Paamicky and jefchien June 23, 2026 04:57
@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

This PR was marked stale due to lack of activity.

@github-actions github-actions Bot added the Stale label Jul 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready for testing Indicates this PR is ready for integration tests to run Stale

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants