feat(test): Add timing measurement for integration tests (#985)#987
Closed
xingzihai wants to merge 1 commit into
Closed
feat(test): Add timing measurement for integration tests (#985)#987xingzihai wants to merge 1 commit into
xingzihai wants to merge 1 commit into
Conversation
This commit implements timing measurement for integration tests as requested in Issue ForNeVeR#985. Changes: - Add TimingHelper.cs with TestTimingResult, TestTimingCollector, and TestTimer classes - Modify IntegrationTestContext.cs to initialize timing collector and save results to JSON - Modify IntegrationTestRunner.cs to measure compile and execution times for both native and Cesium compilers Features: - Measures NativeCompileTime, CesiumCompileTime, NativeExecutionTime, CesiumExecutionTime, TotalTime - Records test name, target framework, target arch, source files, OS, success/failure status - Outputs results to JSON file (artifacts/timing/integration_test_timing_TIMESTAMP.json) - Includes summary statistics (average times, total tests, pass/fail counts) - Works on all operating systems (Windows, Linux, macOS) Co-authored-by: PR Machine Agent
Owner
|
You seems to have opened two PRs addressing the same topic (the other one being #986); let's please focus on one of them? I'll leave the first one open, and will close this one. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR implements timing measurement for integration tests as requested in Issue #985.
Summary
Currently, the integration test suite on Windows periodically takes longer than 10 minutes, which is quite long. This PR adds timing measurement infrastructure to help identify which stages are taking the most time.
Changes
TimingHelper.cs (new file)
TestTimingResult: Record to store timing data for each test executionTestTimingCollector: Static class to collect and aggregate timing results, save to JSONTestTimer: Helper class for measuring execution timeIntegrationTestContext.cs (modified)
TimingOutputDirectorypropertyInitializeOnce()DisposeAsync()IntegrationTestRunner.cs (modified)
TestExecutionResultrecord to capture timing dataCompileAndRunWithNative()andCompileAndRunWithCesium()to return timing dataDoTest()to:TestTimingResultwith full timing breakdownTestTimingCollectorFeatures
✅ Measures timing for:
✅ Records metadata:
✅ Outputs to JSON:
artifacts/timing/integration_test_timing_TIMESTAMP.json✅ Cross-platform:
Testing
The code has been implemented following the existing patterns in the codebase. Tests will run automatically via CI.
Related Issue
Closes #985