Description
In the OpenVINO backend, statistics collection inside quantize_with_accuracy_control_impl can be redundant when hyperparameter tuning is enabled.
Currently, the same statistics may be recalculated during both the initial quantization pass and the subsequent tuning/restoration steps, which can double the runtime of an already expensive process.
Location
Problem Summary
- Statistics are recalculated unnecessarily during hyperparameter tuning
- Leads to slower quantization
- Wastes compute and reduces user experience for large models
- The operation is expensive and redundant
Suggested Fix
- Implement caching of statistics collected during the initial quantization pass
- Reuse cached statistics for subsequent tuning or restoration steps
- This avoids redundant computation and improves runtime
Example idea:
# Pseudocode
if statistics_cache_exists:
use_cached_statistics()
else:
collect_statistics()
save_to_cache()
Expected Benefits
- Faster hyperparameter tuning and accuracy restoration
- Reduced computation overhead
- Improved user experience and model quantization workflow
- Maintains correctness of accuracy-aware quantization
Description
In the OpenVINO backend, statistics collection inside
quantize_with_accuracy_control_implcan be redundant when hyperparameter tuning is enabled.Currently, the same statistics may be recalculated during both the initial quantization pass and the subsequent tuning/restoration steps, which can double the runtime of an already expensive process.
Location
Problem Summary
Suggested Fix
Example idea:
Expected Benefits