Add real-time barcode scanning API and iOS backend implementation by phildini · Pull Request #4458 · beeware/toga

phildini · 2026-06-13T05:07:13Z

Add Camera.start_scanning(), stop_scanning(), is_scanning() and on_detection callback to toga-core, with BarcodeFormat enum supporting 7 barcode types. Full implementation in the Dummy backend for testing. iOS backend uses AVCaptureSession with AVCaptureMetadataOutput for native barcode detection. Cocoa and Android backends raise NotImplementedError stubs.

Includes 14 new core tests (100% coverage), updated documentation, and towncrier change fragment.

Add the API surface area and the iOS backend for allowing real-time barcode processing off a camera stream. The iOS libraries support this natively, and toga already has a camera widget, this is exposing more features in toga. I believe Android and macOS also expose this, but need to do more research.

So, there's a lot of functionality that gets unlocked by exposing these APIs, but my immediate need is being able to do token auth / token exchange for a BeeWare app by having the hosting site generate a QR code that contains a token the app can use as part of an auth flow.

PR Checklist:

I will abide by the BeeWare Code of Conduct
I have read and have followed the CONTRIBUTING.md file
This PR was generated or assisted using an AI tool

Assisted-by: OpenCode with DeepSeek 4 Flash

Add Camera.start_scanning(), stop_scanning(), is_scanning() and on_detection callback to toga-core, with BarcodeFormat enum supporting 7 barcode types. Full implementation in the Dummy backend for testing. iOS backend uses AVCaptureSession with AVCaptureMetadataOutput for native barcode detection. Cocoa and Android backends raise NotImplementedError stubs. Includes 14 new core tests (100% coverage), updated documentation, and towncrier change fragment.

phildini · 2026-06-13T05:34:39Z

There's some indication that the API wanted for this on Android is https://developers.google.com/ml-kit/vision/barcode-scanning/code-scanner

Add test_barcode_format_str and test_barcode_format_values to exercise all BarcodeFormat members and their __str__ methods.

Replace static code type tests with parametrized test exercising each BarcodeFormat individually through the full scan API, plus a combined test for all types. The str() and member tests are now integrated into a single enumeration check.

phildini · 2026-06-14T02:45:08Z

I really think the test errors here are transient and not related to this PR, but please correct me if I'm wrong

freakboy3742 · 2026-06-14T06:13:30Z

Argh... I think the iOS failure is due to beeware/briefcase-iOS-Xcode-template#75 being merged (part of the whole macOS-26 logging issue).

freakboy3742 · 2026-06-14T08:48:14Z

Ok - I've resolved the issue that was causing problems; the test suite is now passing, but has some coverage gaps.

freakboy3742

This is exciting stuff! Thanks for the PR.

A few comments inline; most of which are fairly minor. The biggest source of concern is the removal of explanatory comments; it's not clear why these have been stripped. There's also the test failure due to some coverage misses.

I haven't run this actual code yet - for the purposes of testing, it would be helpful if there the example/hardware app had an example of barcode scanning - maybe a new tab that scans barcodes on demand and presents a multiline text widget that is populated with found barcodes, cleared each time a session is started?

I have two other architectural questions that don't need to be implemented as part of this PR, but I'd like to have at least an indicative answer to make sure we're not boxing ourselves into a corner:

Is this API compatible with Android's scanner? I can't think of any reason it wouldn't be, but I'd prefer to have a light confirmation before we lock this API in.
This API displays a view controller to display a camera view, and that definitely makes sense as a simple API for getting a barcode. However, I can also imagine that a user might want to have a custom view where they are in control of the visualisation of the camera.

I'm thinking in particular of the "multi-scan" case - I might want a GUI where the top half of the screen is a camera preview, and the bottom half is the list of most recently scanned codes. In that context, I can imagine we might need something like a "CameraPreview" widget, with the ability to connect that widget to scanning processes.

I imagine the setup for that would be to pass a CameraPreview widget to start_scanning(); the UI constructed by the hardware service is then "the UI that is displayed if you don't give me an explcit camera preview". That would presumably also have analog with take_photo() so that users could have their own custom inline camera preview for taking photos. If you've got any other suggestions on how this might be structured, or why this might not be possible, I'd be interested in hearing your thoughts.

freakboy3742 · 2026-06-15T06:34:58Z

            raise PermissionError("App does not have permission to take photos")
+
+    def is_scanning(self):
+        raise NotImplementedError("Barcode scanning is not yet implemented on macOS")


FWIW, the implementation on macOS should be almost identical to iOS - the underlying AVKit calls are the same.

freakboy3742 · 2026-06-15T06:40:07Z

+    def start_scanning(
+        self,
+        device: CameraDevice | None = None,
+        code_types: list[BarcodeFormat] | None = None,


As a convenience, should we also accept BarcodeFormat here? I would have expected the most common use case is "scan for QR Code", rather than "Scan for QR code or aztec code or Code128 barcode or ...".

Would you prefer that over code_types?

I wasn't suggesting changing the argument name - I was only suggesting adding the typing and argument handling shim so that start_scanning(QR) is a legal usage, equivalent to start_scanning([QR]).

start_scanning(code_types=QR) is a little weird because of inconsistent pluralization... but if code_types is the first argument, then you won't need to ever write code_types, and maybe that isn't as obvious as a weirdness?

freakboy3742 · 2026-06-15T07:30:22Z

        else:  # pragma: no cover
-            # The app doesn't have the NSCameraUsageDescription key (e.g., via
-            # `permission.camera` in Briefcase). No-cover because we can't manufacture
-            # this condition in testing.


Any reason all the explanatory comments have been removed?

freakboy3742 · 2026-06-15T07:35:59Z

+        device_input = symbols["capture_device_input"].deviceInputWithDevice_error_(
+            capture_device, None
+        )


Once the symbol is exported, this should be:

Suggested change

device_input = symbols["capture_device_input"].deviceInputWithDevice_error_(

capture_device, None

)

device_input = AVCaptureDeviceInput.deviceInputWithDevice(capture_device, error=None)

freakboy3742 · 2026-06-15T07:36:38Z

+        capture_metadata_output = _scan_symbols()["capture_metadata_output"]
+        for output in session.outputs():
+            if output.isKindOfClass_(capture_metadata_output):
+                output.setMetadataObjectsDelegate_queue_(self._scan_delegate, None)


Rubicon provides a cleaner syntax for multi-argument ObjC methods:

Suggested change

output.setMetadataObjectsDelegate_queue_(self._scan_delegate, None)

output.setMetadataObjectsDelegate(self._scan_delegate, queue=None)

Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>

As a convenience for the common use case of scanning for a single barcode type (e.g. QR), start_scanning() now accepts a bare BarcodeFormat value in addition to a list. A single value is normalized to a one-element list before being passed to the backend.

Replace NotImplementedError stubs on macOS with real AVCaptureMetadataOutput implementation via TogaCameraScannerDelegate and TogaCameraScannerWindow. Add AVCaptureMetadataOutput, AVMetadataMachineReadableCodeObject, and 7 AVMetadataObjectType* constants to cocoa AVFoundation bindings. Restore # for classes that need to be monkeypatched for testing comment and commented-out native_video_quality() function in iOS backend. The code_types parameter now accepts a single BarcodeFormat value as a convenience shorthand. Update docs and changes fragment to reflect macOS support and single-value acceptance.

Follow the standard codebase pattern by declaring ObjCClass and objc_const symbols directly in libs/av_foundation.py instead of using a lazy dict lookup via _scan_symbols(). The BARCODE_FORMAT_MAP is now a static module-level dict using the directly imported AVMetadataObjectType constants.

Restore NSCameraUsageDescription, request_permission thread, take_photo configuration, delegate, and presentation comments that were lost during the rewrite of the iOS camera backend.

freakboy3742 · 2026-06-16T07:04:05Z

@@ -1 +1 @@
-Barcodes (in QR, Code 128, EAN-13, EAN-8, PDF417, Aztec, and Data Matrix formats) can now be scanned with the Camera API, with an `on_detection()` callback being invoked when the camera is in scanning mode and a barcode of the requested type is seen.
+The Camera API gained `start_scanning()`, `stop_scanning()`, and `is_scanning()` methods for real-time barcode and QR code scanning, along with an `on_detection` callback and a `BarcodeFormat` enum. The `code_types` parameter accepts a single `BarcodeFormat` value or a list. The iOS and macOS backends implement scanning using `AVCaptureSession` with `AVCaptureMetadataOutput`, supporting QR codes, Code 128, EAN-13, EAN-8, PDF417, Aztec, and Data Matrix formats. The Android backend raises `NotImplementedError` for scanning operations.


This seems to have reverted the changenote suggestion from before.

(also - the changenote should be 4458.feture.md)

phildini added 2 commits June 13, 2026 16:03

Add tests for BarcodeFormat enum to reach 100% constants coverage

67134a5

Add test_barcode_format_str and test_barcode_format_values to exercise all BarcodeFormat members and their __str__ methods.

Copilot started work on behalf of phildini June 14, 2026 00:43 View session

Defer iOS scan bindings until scanner startup

d555088

Copilot finished work on behalf of phildini June 14, 2026 00:55

freakboy3742 requested changes Jun 15, 2026

View reviewed changes

phildini and others added 6 commits June 15, 2026 12:47

Update changes/camera-scanning.feature.md

fcc8343

Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>

Restore explanatory comments in iOS test probe

ea52118

Restore all explanatory comments removed from iOS backend

8ff7b03

Restore NSCameraUsageDescription, request_permission thread, take_photo configuration, delegate, and presentation comments that were lost during the rewrite of the iOS camera backend.

freakboy3742 reviewed Jun 16, 2026

View reviewed changes

	output.setMetadataObjectsDelegate_queue_(self._scan_delegate, None)
	output.setMetadataObjectsDelegate(self._scan_delegate, queue=None)

		@@ -1 +1 @@
		Barcodes (in QR, Code 128, EAN-13, EAN-8, PDF417, Aztec, and Data Matrix formats) can now be scanned with the Camera API, with an `on_detection()` callback being invoked when the camera is in scanning mode and a barcode of the requested type is seen.
		The Camera API gained `start_scanning()`, `stop_scanning()`, and `is_scanning()` methods for real-time barcode and QR code scanning, along with an `on_detection` callback and a `BarcodeFormat` enum. The `code_types` parameter accepts a single `BarcodeFormat` value or a list. The iOS and macOS backends implement scanning using `AVCaptureSession` with `AVCaptureMetadataOutput`, supporting QR codes, Code 128, EAN-13, EAN-8, PDF417, Aztec, and Data Matrix formats. The Android backend raises `NotImplementedError` for scanning operations.

Uh oh!

Conversation

phildini commented Jun 13, 2026

PR Checklist:

Uh oh!

phildini commented Jun 13, 2026

Uh oh!

phildini commented Jun 14, 2026

Uh oh!

freakboy3742 commented Jun 14, 2026

Uh oh!

freakboy3742 commented Jun 14, 2026

Uh oh!

freakboy3742 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants