Skip to content

fix: resolve medium dataset findings#2408

Merged
Borda merged 2 commits into
developfrom
medium/Dataset
Jul 8, 2026
Merged

fix: resolve medium dataset findings#2408
Borda merged 2 commits into
developfrom
medium/Dataset

Conversation

@Borda

@Borda Borda commented Jul 6, 2026

Copy link
Copy Markdown
Member

This pull request introduces several important improvements and bug fixes across dataset handling, metrics computation, and asset management. Key changes include safer handling of user-supplied data to prevent mutation, enhanced error reporting, improved dataset import/export compatibility, and more robust metric calculations. Additionally, asset downloads now include MD5 verification and retry logic, and several utilities have been made more tolerant to data variations.

Dataset handling improvements:

  • DetectionDataset no longer mutates caller-supplied Detections when adding data["class_name"], and now raises a clear ValueError with the image path if annotation class_id values are out of range. [1]], [2]])
  • COCO dataset loading tolerates missing iscrowd and area fields, exposes a use_iscrowd parameter, and mask export writes mask pixel area instead of bbox area. [1]], [2]], [3]], [4]], [5]], [6]])
  • ClassificationDataset.from_folder_structure now skips root-level files and dot entries, and only includes valid class directories and image files. [1]], [2]], [3]])
  • YOLO dataset loading now accepts non-RGB image modes (e.g., RGBA, palette PNGs) when only dimensions are needed. [1]], [2]])

Metrics computation fixes:

  • F1Score, Precision, Recall, and MeanAverageRecall now compute size-bucket metrics by matching on the full detection set before target-bucket scoring, preventing boundary-size matches from being counted as both false positives and false negatives. MASKS metrics now raise a descriptive error if detections lack masks. [1]], [2]], [3]], [4]], [5]], [6]])

Asset download reliability:

  • Fresh asset downloads are now MD5-verified and retried once if the checksum does not match, raising a clear error on repeated failure. [1]], [2]], [3]], [4]])

Classification utilities:

  • Classifications equality now compares NumPy-backed fields by value, and Classifications.from_timm returns softmax probabilities instead of raw logits. [1]], [2]], [3]])

Testing configuration:

  • Pytest configuration now explicitly sets test paths for more reliable test discovery. ([pyproject.tomlR197])

These changes collectively enhance data integrity, error handling, and compatibility with common dataset formats, while improving reliability and transparency in both metrics and asset management.

@Borda Borda requested a review from Copilot July 6, 2026 22:48
@Borda Borda requested a review from SkalskiP as a code owner July 6, 2026 22:48
@codecov

codecov Bot commented Jul 6, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 58.04598% with 73 lines in your changes missing coverage. Please review.
✅ Project coverage is 86%. Comparing base (74db9e2) to head (a48188c).

Additional details and impacted files
@@           Coverage Diff           @@
##           develop   #2408   +/-   ##
=======================================
- Coverage       87%     86%   -1%     
=======================================
  Files           70      70           
  Lines        10240   10330   +90     
=======================================
+ Hits          8868    8872    +4     
- Misses        1372    1458   +86     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses a set of medium-severity findings across Supervision’s dataset utilities, metrics, trackers, and asset management. The changes focus on preventing unintended caller-data mutation, improving dataset format robustness (COCO/YOLO/folder-structure), and correcting size-bucket metric computation.

Changes:

  • Prevents mutation of caller-owned objects in DetectionDataset and ByteTrack, and adds clearer validation/error behavior around invalid dataset annotations.
  • Improves dataset format compatibility (COCO tolerates missing iscrowd/area, YOLO accepts non-RGB modes for size reads) and updates COCO export area semantics for masks.
  • Fixes size-bucket metric computation by matching across the full detection set first, then scoring per bucket; adds explicit errors for MASKS metrics when masks are missing.

Review scores (n/5):

  • Code quality: 4/5
  • Testing: 4/5
  • Documentation/changelog: 4/5

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
tests/tracker/test_byte_tracker.py Adds coverage for ByteTrack activation threshold clamping and input non-mutation.
tests/test_validators.py Adds tests for private validator helpers to ensure consistent ValueError messaging.
tests/metrics/test_size_buckets.py Adds regression tests for size-bucket matching behavior and MASKS target validation.
tests/dataset/test_core.py Adds tests for folder-structure loader robustness and DetectionDataset annotation copying/range checks.
tests/dataset/formats/test_yolo.py Adds test ensuring YOLO loader can handle RGBA PNGs when reading dimensions.
tests/dataset/formats/test_coco.py Adds coverage for missing COCO fields, use_iscrowd, and mask-area export behavior.
tests/classification/test_core.py Extends mocks/tests for timm softmax output and numpy-safe equality.
tests/assets/test_downloader.py Updates/adds tests for MD5 verification and retry logic in asset downloads.
src/supervision/tracker/byte_tracker/core.py Clamps det_thresh below 1.0 and avoids mutating input Detections in tracking updates.
src/supervision/metrics/utils/matching.py Adds helper returning both match correctness and matched target indices to support bucket scoring.
src/supervision/metrics/recall.py Reworks size-bucket recall computation to match globally before bucket scoring; adds MASKS validation.
src/supervision/metrics/precision.py Reworks size-bucket precision computation similarly; adds MASKS validation.
src/supervision/metrics/f1_score.py Reworks size-bucket F1 computation similarly; adds MASKS validation.
src/supervision/metrics/mean_average_recall.py Reworks size-bucket mAR computation similarly; adds MASKS validation.
src/supervision/dataset/formats/yolo.py Removes strict image-mode check to allow non-RGB modes when only dimensions are required.
src/supervision/dataset/formats/coco.py Defaults missing iscrowd/area during load; exports mask area as pixel count when mask present.
src/supervision/dataset/core.py Copies annotations defensively, validates class_id dtype/range, adds use_iscrowd, and hardens folder-structure import.
src/supervision/classification/core.py Adds numpy-safe Classifications.__eq__ and softmaxes timm outputs.
src/supervision/assets/downloader.py Adds MD5 verification, retry-on-mismatch behavior, and clearer failure on repeated MD5 mismatch.
pyproject.toml Sets pytest testpaths for more reliable discovery.
docs/changelog.md Documents the user-visible fixes and behavior changes in the unreleased changelog.

Comment thread src/supervision/assets/downloader.py Outdated
Comment thread src/supervision/dataset/formats/yolo.py Outdated
Comment thread src/supervision/dataset/core.py Outdated
Comment thread src/supervision/assets/downloader.py Outdated
- Reinstated NumPy-safe `Classifications` equality and ordered class-list comparisons in dataset equality.
- Restored greedy matching plus size-bucket scoring for Precision, Recall, F1, and MeanAverageRecall, with regression coverage for the medium-object boundary case.

---

Co-authored-by: Codex <codex@openai.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

- Filter size-bucket precision, recall, and F1 against target boxes so predictions no longer claim the bucket.

- Preserve confidence order for bucketed mAR@K scoring and return zero when a bucket has no support.

- Add regression coverage for bucket matching, empty-support mAR, top-K limits, and missing-mask errors.

---

Co-authored-by: Codex <codex@openai.com>
@Borda Borda merged commit 072f784 into develop Jul 8, 2026
26 checks passed
@Borda Borda deleted the medium/Dataset branch July 8, 2026 06:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants