fix(utils): normalize timm confidences and verify assets#2414
Merged
Conversation
- Convert timm classification logits with softmax so confidence values match the normalized scale used by other classification adapters. - Verify asset MD5 hashes after fresh downloads and retry once when a payload is corrupted. - Add focused regressions for timm confidence scaling and asset download integrity paths. --- Co-authored-by: Codex <codex@openai.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #2414 +/- ##
=======================================
Coverage 87% 87%
=======================================
Files 70 70
Lines 10227 10240 +13
=======================================
+ Hits 8853 8868 +15
+ Misses 1374 1372 -2 🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR improves correctness and reliability in two core utilities: asset downloading (by verifying MD5 after fresh downloads and retrying once on corruption) and classification adapters (by normalizing timm logits to probabilities via softmax to match CLIP behavior).
Changes:
- Refactor
sv.download_assetsdownload flow to MD5-verify newly downloaded payloads and retry once on mismatch. - Update
sv.Classifications.from_timmto softmax logits before exposing confidences. - Expand tests to cover MD5 verification/retry behavior and timm softmax normalization; document both changes in the changelog.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| src/supervision/assets/downloader.py | Adds verified download helper(s) and introduces “download → MD5 verify → retry once” behavior for new downloads. |
| tests/assets/test_downloader.py | Adds/updates unit tests validating MD5 verification on fresh downloads and single-retry behavior on corrupted payloads. |
| src/supervision/classification/core.py | Normalizes timm confidences via softmax to ensure probability-scale output. |
| tests/classification/test_core.py | Adds a minimal tensor double softmax + tests for empty-output dtypes and timm softmax normalization. |
| docs/changelog.md | Documents the MD5 verification/retry change and timm confidence normalization in the Unreleased “Fixed” section. |
- Convert from_timm outputs to probabilities before applying thresholds and document that existing thresholds may need retuning. - Add downloader regression coverage for repeated MD5 mismatches so exhausted retries now raise ValueError. --- Co-authored-by: Codex <codex@openai.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request improves the reliability and correctness of asset downloading and classification confidence handling in the codebase. The main highlights are stricter MD5 verification and retry logic for asset downloads, normalization of classification confidences from timm models, and expanded test coverage for these behaviors.
Asset Download Robustness:
src/supervision/assets/downloader.py,tests/assets/test_downloader.py) [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]tests/assets/test_downloader.py) [1] [2] [3] [4] [5] [6] [7] [8]Classification Confidence Normalization:
sv.Classifications.from_timmto apply softmax to model logits before exposing confidence scores, ensuring output is on a normalized probability scale and matches the behavior offrom_clip. (src/supervision/classification/core.py,docs/changelog.md) [1] [2]tests/classification/test_core.py) [1] [2] [3] [4]Documentation:
docs/changelog.md)part of #2408