reengineering: benchmarking system#73
Draft
iraedeus wants to merge 15 commits intoiraedeus/refactor-metricsfrom
Draft
reengineering: benchmarking system#73iraedeus wants to merge 15 commits intoiraedeus/refactor-metricsfrom
iraedeus wants to merge 15 commits intoiraedeus/refactor-metricsfrom
Conversation
Not up to standards ⛔
|
bb999cc to
85e1cdd
Compare
32f2c21 to
ab266d8
Compare
662f1c1 to
87ac28d
Compare
55d632f to
2979e34
Compare
2979e34 to
b9ce3ab
Compare
b9ce3ab to
39c6903
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces a new
benchmark_executor.pymodule to thepysatl_cpd/benchmarkpackage, providing a robust framework for executing and caching benchmarks for online change-point detection (CPD) algorithms. The module enables running multiple algorithms across datasets and thresholds, with optional disk-based caching to avoid redundant computations.Key additions and features:
Benchmarking orchestration and caching:
BenchmarkExecutorgeneric class, which coordinates the execution of benchmarks for multiple algorithms, datasets, and thresholds, supporting optional result caching via disk dumping.Benchmark metadata management:
BenchmarkRecorddataclass to uniquely identify each benchmark run and store metadata, including algorithm, configuration, dataset, threshold, and cache file path.Integration with existing components:
OnlineAlgorithm,DataProvider,OnlineCpdSolver, andOnlineDetectionTracefrom the core modules, ensuring compatibility and reusability within the CPD framework.