Skip to content

fix(utils): normalize timm confidences and verify assets#2414

Merged
Borda merged 3 commits into
developfrom
medium/Dataset-4
Jul 7, 2026
Merged

fix(utils): normalize timm confidences and verify assets#2414
Borda merged 3 commits into
developfrom
medium/Dataset-4

Conversation

@Borda

@Borda Borda commented Jul 7, 2026

Copy link
Copy Markdown
Member

This pull request improves the reliability and correctness of asset downloading and classification confidence handling in the codebase. The main highlights are stricter MD5 verification and retry logic for asset downloads, normalization of classification confidences from timm models, and expanded test coverage for these behaviors.

Asset Download Robustness:

  • Refactored the asset download logic to verify MD5 hashes after every fresh download. If a downloaded file fails MD5 verification, it is deleted and the download is retried once before raising an error. This prevents corrupted files from being accepted. (src/supervision/assets/downloader.py, tests/assets/test_downloader.py) [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]
  • Added and updated tests to confirm that corrupted files are detected, retried, and only valid assets are accepted. Tests also ensure MD5 checks are performed on all asset downloads. (tests/assets/test_downloader.py) [1] [2] [3] [4] [5] [6] [7] [8]

Classification Confidence Normalization:

  • Changed sv.Classifications.from_timm to apply softmax to model logits before exposing confidence scores, ensuring output is on a normalized probability scale and matches the behavior of from_clip. (src/supervision/classification/core.py, docs/changelog.md) [1] [2]
  • Added and improved tests for classification adapters, including checks that softmax is applied and output confidence is normalized. (tests/classification/test_core.py) [1] [2] [3] [4]

Documentation:

  • Updated the changelog to document the new asset download verification and classification confidence normalization behaviors. (docs/changelog.md)

part of #2408

- Convert timm classification logits with softmax so confidence values match the normalized scale used by other classification adapters.

- Verify asset MD5 hashes after fresh downloads and retry once when a payload is corrupted.

- Add focused regressions for timm confidence scaling and asset download integrity paths.

---

Co-authored-by: Codex <codex@openai.com>
@Borda Borda requested a review from Copilot July 7, 2026 12:10
@Borda Borda requested a review from SkalskiP as a code owner July 7, 2026 12:10
@codecov

codecov Bot commented Jul 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87%. Comparing base (dde4227) to head (7d386c3).

Additional details and impacted files
@@           Coverage Diff           @@
##           develop   #2414   +/-   ##
=======================================
  Coverage       87%     87%           
=======================================
  Files           70      70           
  Lines        10227   10240   +13     
=======================================
+ Hits          8853    8868   +15     
+ Misses        1374    1372    -2     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves correctness and reliability in two core utilities: asset downloading (by verifying MD5 after fresh downloads and retrying once on corruption) and classification adapters (by normalizing timm logits to probabilities via softmax to match CLIP behavior).

Changes:

  • Refactor sv.download_assets download flow to MD5-verify newly downloaded payloads and retry once on mismatch.
  • Update sv.Classifications.from_timm to softmax logits before exposing confidences.
  • Expand tests to cover MD5 verification/retry behavior and timm softmax normalization; document both changes in the changelog.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/supervision/assets/downloader.py Adds verified download helper(s) and introduces “download → MD5 verify → retry once” behavior for new downloads.
tests/assets/test_downloader.py Adds/updates unit tests validating MD5 verification on fresh downloads and single-retry behavior on corrupted payloads.
src/supervision/classification/core.py Normalizes timm confidences via softmax to ensure probability-scale output.
tests/classification/test_core.py Adds a minimal tensor double softmax + tests for empty-output dtypes and timm softmax normalization.
docs/changelog.md Documents the MD5 verification/retry change and timm confidence normalization in the Unreleased “Fixed” section.

- Convert from_timm outputs to probabilities before applying thresholds and document that existing thresholds may need retuning.
- Add downloader regression coverage for repeated MD5 mismatches so exhausted retries now raise ValueError.

---

Co-authored-by: Codex <codex@openai.com>
@Borda Borda merged commit 74db9e2 into develop Jul 7, 2026
26 checks passed
@Borda Borda deleted the medium/Dataset-4 branch July 7, 2026 19:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants