Skip to content

feat: Detections.from_inference supports compressed RLE masks#2178

Open
leeclemnet wants to merge 4 commits intodevelopfrom
from-inference-rle-mask
Open

feat: Detections.from_inference supports compressed RLE masks#2178
leeclemnet wants to merge 4 commits intodevelopfrom
from-inference-rle-mask

Conversation

@leeclemnet
Copy link
Copy Markdown

@leeclemnet leeclemnet commented Mar 20, 2026

Before submitting
  • Self-reviewed the code
  • Updated documentation, follow Google-style
  • Added docs entry for autogeneration (if new functions/classes)
  • Added/updated tests
  • All tests pass locally

Description

Add RLE mask support to sv.Detections.from_inference for models that return compressed COCO RLE (e.g. SAM3, semantic segmentation), and extend rle_to_mask/mask_to_rle with compressed string encoding/decoding — implemented as a pure Python COCO RLE codec with no new dependencies.

cc @PawelPeczek-Roboflow

Type of Change

  • ✨ New feature (non-breaking change which adds functionality)

Motivation and Context

Workflows currently supports COCO RLE compressed string serialization for segmentation masks, originally for SAM3 instance segmentation and now also for semantic segmentation models where masks may not be contiguous and therefore can't be represented as polygons. We would like sv.Detections.from_inference() to handle such data and automatically decode it into binary masks as it currently does for polygon mask data.

The existing rle_to_mask util supports this conversion but not from compressed strings (see #1952).

Closes #1952

Changes Made

  • Add RLE mask support to sv.Detections.from_inference for models like SAM3 and semantic segmentation that return compressed COCO RLE instead of polygons
  • Extend rle_to_mask to accept compressed COCO RLE strings (no new dependencies — pure Python decoder)
  • Add compressed flag to mask_to_rle for producing COCO RLE strings
  • Move rle_to_mask and mask_to_rle from dataset/utils to detection/utils/converters with the other converters - necessary to avoid circular dependency, and dataset.utils already imports from detection.utils.converters - backwards compatibility maintained through re-export
  • Bugfix: rle_to_mask return type was uint8, now bool as per function signature

Testing

  • I have tested this code locally
  • I have added unit tests that prove my fix is effective or that my feature works
  • All new and existing tests pass

Google Colab (optional)

Colab link:

Screenshots/Videos (optional)

Additional Notes

@leeclemnet leeclemnet changed the title feat: detections.from_inference supports compressed RLE masks feat: Detections.from_inference supports compressed RLE masks Mar 20, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Mar 20, 2026

Codecov Report

❌ Patch coverage is 97.89474% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 76%. Comparing base (d6a24d0) to head (44b817a).

Additional details and impacted files
@@           Coverage Diff           @@
##           develop   #2178   +/-   ##
=======================================
  Coverage       76%     76%           
=======================================
  Files           62      62           
  Lines         7547    7615   +68     
=======================================
+ Hits          5714    5780   +66     
- Misses        1833    1835    +2     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@leeclemnet leeclemnet marked this pull request as ready for review March 20, 2026 21:04
@leeclemnet leeclemnet requested a review from SkalskiP as a code owner March 20, 2026 21:04
Borda
Borda previously approved these changes Mar 21, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds compressed COCO RLE mask support across Supervision’s detection inference pipeline and converters, enabling sv.Detections.from_inference() to decode masks provided as compressed RLE strings/bytes (e.g., SAM3 / semantic segmentation outputs) without adding dependencies.

Changes:

  • Implement pure-Python COCO compressed RLE string encode/decode and extend rle_to_mask / mask_to_rle to support compressed formats.
  • Update inference result processing to accept RLE mask payloads (rle / rle_mask) and build Detections.mask accordingly.
  • Move RLE utilities into detection.utils.converters, update exports/docs, and relocate tests accordingly.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
tests/detection/utils/test_internal.py Adds inference parsing test cases for compressed RLE masks (incl. non-contiguous + tracker_id).
tests/detection/utils/test_converters.py Adds unit tests for compressed RLE codec and mask↔RLE conversions.
tests/dataset/test_utils.py Removes RLE tests from dataset utils (now covered under converters).
src/supervision/detection/utils/internal.py Extends Roboflow/inference result parsing to decode RLE masks.
src/supervision/detection/utils/converters.py Adds COCO compressed RLE codec + compressed support in converters.
src/supervision/dataset/utils.py Removes old implementations and re-exports converters versions for compatibility.
src/supervision/dataset/formats/coco.py Updates imports to use the new converter locations.
src/supervision/init.py Exposes mask_to_rle / rle_to_mask from converters at top-level API.
mkdocs.yml Removes datasets utils docs page from nav.
docs/detection/utils/converters.md Documents rle_to_mask / mask_to_rle under converters.
docs/datasets/utils.md Removes the old dataset utils docs page.

Comment on lines +100 to +102
confidence.append(prediction["confidence"])
masks.append(mask)
if "tracker_id" in prediction:
Copy link

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mask from rle_to_mask is a boolean array, but masks is annotated as list[npt.NDArray[np.uint8]] in this function. With strict mypy enabled, appending a bool mask here will raise a type error. Consider making the list type boolean (and casting the polygon mask to bool before appending) so both RLE and polygon paths are consistent.

Copilot uses AI. Check for mistakes.
Borda and others added 2 commits March 21, 2026 22:00
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
- Add bounds check in _decode_coco_rle_string() so malformed/truncated
  compressed RLE strings raise ValueError instead of IndexError
- Guard process_roboflow_result against non-dict, missing-key, and
  malformed-counts RLE payloads — all fall through to box-only detection
- Fix stale masks type annotation (uint8 → bool_) in internal.py
- Add warn_deprecated() wrappers on rle_to_mask/mask_to_rle re-exports
  in dataset/utils for consistency with project deprecation pattern
- Add negative-path tests: malformed counts, missing size/counts keys,
  non-dict rle, rle_mask fallback key, truncated compressed strings
- Add CHANGELOG entries under develop

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@Borda Borda added the enhancement New feature or request label Mar 21, 2026
@leeclemnet leeclemnet enabled auto-merge (squash) March 23, 2026 15:11
@Borda Borda self-assigned this Mar 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

does rle_to_mask to mask support UTF-8 encoded string?

3 participants