feat(test-optimization): create final_status tag on test event for cypress#8025
feat(test-optimization): create final_status tag on test event for cypress#8025
Conversation
Overall package sizeSelf size: 5.5 MB Dependency sizes| name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.0.1 | 82.56 kB | 817.39 kB | | dc-polyfill | 0.1.10 | 26.73 kB | 26.73 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #8025 +/- ##
=======================================
Coverage 73.59% 73.60%
=======================================
Files 773 773
Lines 36088 36088
=======================================
+ Hits 26560 26562 +2
+ Misses 9528 9526 -2 Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
✨ Fix all issues with BitsAI or with Cursor
|
BenchmarksBenchmark execution time: 2026-04-17 21:43:30 Comparing candidate commit b200156 in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 1384 metrics, 100 unstable metrics. |
1 participant
What does this PR do?
Create a @test.final_status tag that is added to every cypress test event that represents the final execution of that test (there can be multiple executions of a single test due to retry mechanisms being active)
Motivation
Tracers send each test execution as a span to the backend. When retry mechanisms are enabled a single test can run several times with different outcomes. These mechanisms differ in behavior. Some stop once the test reaches the desired state such as ATR ending when a test passes. Others always run a fixed number of attempts such as EFD or Attempt to fix.
To ensure the correct final result for the customer some intermediate outcomes are suppressed and the testing framework receives an adjusted final status. For example if an early EFD attempt passes but later attempts fail the test is still reported as passed and the failing attempts are hidden to avoid failing CI pipelines.
Both external and internal users have requested a way to query tests in Datadog by this final status. Their main goal is to build monitors and alerts for hard failures on default branches.
To support this a new tag will be added to the last retry of a test named: @test.final_status:pass|fail|skip