Letterbox resolve to required resolution

[hardikdava]

Description

Currently, supervision has sv.letterbox_image() which resize images to get padded and with same aspect ratio image. But once this image used for detection task, then boxes has to be resized to original image in order to get correct scale and location of the boxes.

List any dependencies that are required for this change.

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

How has this change been tested, please provide a testcase or example of how you tested the change?

YOUR_ANSWER

Any specific deployment considerations

For example, documentation changes, usability, usage/costs, secrets, etc.

Docs

• Docs updated? What were the changes:

Script to reproduce results:

import numpy as np
from ultralytics import YOLO
from PIL import Image
import cv2
import supervision as sv


model = YOLO("yolo11n.pt")

image = Image.open("bus.jpg")

def detect_objects(image):
    results = model([image])[0]
    detections = sv.Detections.from_ultralytics(results)
    return detections


def annotate_img(image, dets):
    box_annotator = sv.BoxAnnotator()
    ann_img = box_annotator.annotate(image.copy(), dets)

    return ann_img


wo_letterbox_detections = detect_objects(image)
box_annotator = sv.BoxAnnotator()
wo_ann_img = box_annotator.annotate(image.copy(), wo_letterbox_detections)

## Process with letterbox
letterbox_resolution = (640, 640)
resolution_wh = (image.width, image.height)
letterboxed_image = sv.letterbox_image(image, letterbox_resolution)
letterbox_detections = detect_objects(letterboxed_image)
letterbox_detections.xyxy = sv.resolve_letterbox(letterbox_detections.xyxy, letterbox_resolution, resolution_wh)

box_annotator = sv.BoxAnnotator()
with_ann_img = box_annotator.annotate(image.copy(), letterbox_detections)
with_ann_img = sv.pillow_to_cv2(with_ann_img)


wo_ann_img = sv.pillow_to_cv2(wo_ann_img)
viz_img = np.hstack([wo_ann_img, with_ann_img])
cv2.imshow("Detection", viz_img)
cv2.waitKey(0)

[LinasKo]

Thank you for a quick submission! I'll take a deeper look whenever I can.

Some preliminary notes:

1. 'Resolve' is very general - undo_letterbox or revert_letterbox seems better.
2. This seems separable into scale_to_fit or center_crop and sv.resize. This would introduce a general function we can use outside of unletterboxing use case. We'd then show in examples how letterboxing can be undone.

[hardikdava]

@LinasKo Thanks for a quick look. I have talked with @onuralpszr and he is suggesting different idea of integration. I will wait for him first.

[onuralpszr]

@hardikdava 👋 I talked with Linas and this is the new approach and he also asked me to add OBB and here is the PR #1711

[hardikdava]

@onuralpszr I saw that you already incorporated requirements for masks and oriented bounding box. Let's g with your PR. I am closing this one.

c.c. @LinasKo