diff --git a/docs/content/docs/a-depth-frame.mdx b/docs/content/docs/a-depth-frame.mdx index e4ea2b295e..81a87380de 100644 --- a/docs/content/docs/a-depth-frame.mdx +++ b/docs/content/docs/a-depth-frame.mdx @@ -59,12 +59,13 @@ const buffer = depth.getDepthData() #### Interpreting Depth Data -The Depth Data's layout depends on the [`Depth`](/api/react-native-vision-camera/hybrid-objects/Depth)'s [`pixelFormat`](/api/react-native-vision-camera/hybrid-objects/Depth#pixelformat). For example, in [`'depth-16-bit'`](/api/react-native-vision-camera/type-aliases/DepthPixelFormat), pixels are laid out in 16-bit floats - one float per "pixel". +The Depth Data's layout depends on the [`Depth`](/api/react-native-vision-camera/hybrid-objects/Depth)'s [`pixelFormat`](/api/react-native-vision-camera/hybrid-objects/Depth#pixelformat). On iOS, [`'depth-16-bit'`](/api/react-native-vision-camera/type-aliases/DepthPixelFormat) stores one 16-bit float depth sample per pixel. On Android, [`'depth-16-bit'`](/api/react-native-vision-camera/type-aliases/DepthPixelFormat) maps to `ImageFormat.DEPTH16`, where each 16-bit sample stores range and confidence bits. ```ts const depth = ... const buffer = depth.getDepthData() -if (depth.pixelFormat === 'depth-16') { +if (depth.pixelFormat === 'depth-16-bit') { + // iOS depth-16-bit data is Float16. Android DEPTH16 uses packed Uint16 samples. const pixels = new Float16Array(buffer) // [!code ++] const distanceToFirstPixel = pixels[0] @@ -73,7 +74,8 @@ if (depth.pixelFormat === 'depth-16') { ``` > [!TIP] -> Typically, a single 16-bit float in [`'depth-16-bit'`](/api/react-native-vision-camera/type-aliases/DepthPixelFormat) data represents a distance in meters. +> On iOS, a single 16-bit float in [`'depth-16-bit'`](/api/react-native-vision-camera/type-aliases/DepthPixelFormat) data represents a distance in meters. +> On Android, read `ImageFormat.DEPTH16` as unsigned 16-bit samples: the low 13 bits are range in millimeters and the high 3 bits are confidence. #### Converting between Depth Formats diff --git a/docs/content/docs/a-frame.mdx b/docs/content/docs/a-frame.mdx index 1fc1897cf4..6bd2d8cfca 100644 --- a/docs/content/docs/a-frame.mdx +++ b/docs/content/docs/a-frame.mdx @@ -75,7 +75,7 @@ console.log(frame.pixelFormat) // 'yuv-420-8-bit-full' ``` While the most commonly known [`PixelFormat`](/api/react-native-vision-camera/type-aliases/PixelFormat) is RGB (often [`'rgb-bgra-8-bit'`](/api/react-native-vision-camera/type-aliases/VideoPixelFormat)), it is not natively produced by the Camera, which means it requires expensive conversion causing higher latency, more bandwidth, and overall slower performance. -The Camera's native pixel format is typically YUV (often [`'yuv-420-8-bit-full'`](/api/react-native-vision-camera/type-aliases/VideoPixelFormat)) or a vendor-specific variation of it ([`'private'`](/api/react-native-vision-camera/type-aliases/PixelFormat)), which uses ~50% less memory than RGB and requires little to no conversion overhead. +The Camera's native pixel format is typically YUV (for example [`'yuv-420-8-bit-video'`](/api/react-native-vision-camera/type-aliases/VideoPixelFormat) on iOS native capture or [`'yuv-420-8-bit-full'`](/api/react-native-vision-camera/type-aliases/VideoPixelFormat) for common CPU-readable YUV paths), or a vendor-specific variation of it ([`'private'`](/api/react-native-vision-camera/type-aliases/PixelFormat)), which uses ~50% less memory than RGB and requires little to no conversion overhead. > [!NOTE] > See ["Frame Output: Choosing a Pixel Format"](frame-output#choosing-a-pixel-format) for more information about configuring the Pixel Format a [`CameraFrameOutput`](/api/react-native-vision-camera/hybrid-objects/CameraFrameOutput) streams in. @@ -132,7 +132,7 @@ if (frame.isPlanar) { const uvBuffer = planes[1].getPixelBuffer() const yPixels = new Uint8Array(yBuffer) const uvPixels = new Uint8Array(uvBuffer) - const firstPixel = { y: yPixels[0], u: uvPixels[0], v: uvPixels[0] } + const firstPixel = { y: yPixels[0], u: uvPixels[0], v: uvPixels[1] } } } else { // regular pixel buffer access diff --git a/docs/content/docs/pixel-formats-map.mdx b/docs/content/docs/pixel-formats-map.mdx index 5491e106f5..67a880ca44 100644 --- a/docs/content/docs/pixel-formats-map.mdx +++ b/docs/content/docs/pixel-formats-map.mdx @@ -28,10 +28,13 @@ In a [`CameraFrameOutput`](/api/react-native-vision-camera/hybrid-objects/Camera When you receive a [`Frame`](/api/react-native-vision-camera/hybrid-objects/Frame), you can inspect its [`pixelFormat`](/api/react-native-vision-camera/hybrid-objects/Frame#pixelformat) to find out the precise Pixel Format it uses - here are some examples that map to native pixel formats: -| VisionCamera [`VideoPixelFormat`](/api/react-native-vision-camera/type-aliases/VideoPixelFormat) | iOS [`CVPixelFormatType`](https://developer.apple.com/documentation/corevideo/cvpixelformattype) | Android [`ImageFormat`](https://developer.android.com/reference/android/graphics/ImageFormat) | +| VisionCamera [`PixelFormat`](/api/react-native-vision-camera/type-aliases/PixelFormat) | iOS [`CVPixelFormatType`](https://developer.apple.com/documentation/corevideo/cvpixelformattype) | Android format | |---------------|--------|---------------------| +| [`yuv-420-8-bit-video`](/api/react-native-vision-camera/type-aliases/VideoPixelFormat) | [`420YpCbCr8BiPlanarVideoRange`](https://developer.apple.com/documentation/corevideo/kcvpixelformattype_420ypcbcr8biplanarvideorange) | [`YUV_420_888`](https://developer.android.com/reference/android/graphics/ImageFormat#YUV_420_888) + limited-range `DataSpace` | | [`yuv-420-8-bit-full`](/api/react-native-vision-camera/type-aliases/VideoPixelFormat) | [`420YpCbCr8BiPlanarFullRange`](https://developer.apple.com/documentation/CoreVideo/kCVPixelFormatType_420YpCbCr8BiPlanarFullRange) | [`YUV_420_888`](https://developer.android.com/reference/android/graphics/ImageFormat#YUV_420_888) | | [`yuv-420-10-bit-full`](/api/react-native-vision-camera/type-aliases/VideoPixelFormat) | [`420YpCbCr10BiPlanarFullRange`](https://developer.apple.com/documentation/corevideo/kcvpixelformattype_420ypcbcr10biplanarfullrange) | [`YCBCR_P010`](https://developer.android.com/reference/android/graphics/ImageFormat#YCBCR_P010) | +| [`yuv-444-10-bit-video`](/api/react-native-vision-camera/type-aliases/VideoPixelFormat) | [`444YpCbCr10BiPlanarVideoRange`](https://developer.apple.com/documentation/corevideo/kcvpixelformattype_444ypcbcr10biplanarvideorange) | / | +| [`yuv-444-10-bit-full`](/api/react-native-vision-camera/type-aliases/VideoPixelFormat) | [`444YpCbCr10BiPlanarFullRange`](https://developer.apple.com/documentation/corevideo/kcvpixelformattype_444ypcbcr10biplanarfullrange) | / | | [`rgb-bgra-8-bit`](/api/react-native-vision-camera/type-aliases/VideoPixelFormat) | [`32BGRA`](https://developer.apple.com/documentation/CoreVideo/kCVPixelFormatType_32BGRA) | / | -| [`rgb-rgba-8-bit`](/api/react-native-vision-camera/type-aliases/VideoPixelFormat) | / | [`FLEX_RGBA_888`](https://developer.android.com/reference/android/graphics/ImageFormat#FLEX_RGBA_888) | -| [`private`](/api/react-native-vision-camera/type-aliases/VideoPixelFormat) | / | [`PRIVATE`](https://developer.android.com/reference/android/graphics/ImageFormat#PRIVATE) | +| [`rgb-rgba-8-bit`](/api/react-native-vision-camera/type-aliases/VideoPixelFormat) | [`32RGBA`](https://developer.apple.com/documentation/corevideo/kcvpixelformattype_32rgba) | [`PixelFormat.RGBA_8888`](https://developer.android.com/reference/android/graphics/PixelFormat#RGBA_8888) or [`FLEX_RGBA_8888`](https://developer.android.com/reference/android/graphics/ImageFormat#FLEX_RGBA_8888) | +| [`private`](/api/react-native-vision-camera/type-aliases/PixelFormat) | / | [`PRIVATE`](https://developer.android.com/reference/android/graphics/ImageFormat#PRIVATE) | diff --git a/packages/react-native-vision-camera/android/src/main/java/com/margelo/nitro/camera/extensions/converters/PixelFormat+fromImageFormat.kt b/packages/react-native-vision-camera/android/src/main/java/com/margelo/nitro/camera/extensions/converters/PixelFormat+fromImageFormat.kt index 34118efcd6..718e039100 100644 --- a/packages/react-native-vision-camera/android/src/main/java/com/margelo/nitro/camera/extensions/converters/PixelFormat+fromImageFormat.kt +++ b/packages/react-native-vision-camera/android/src/main/java/com/margelo/nitro/camera/extensions/converters/PixelFormat+fromImageFormat.kt @@ -3,6 +3,7 @@ package com.margelo.nitro.camera.extensions.converters import android.graphics.ImageFormat import com.margelo.nitro.camera.PixelFormat import com.margelo.nitro.camera.utils.PixelRange +import android.graphics.PixelFormat as AndroidPixelFormat fun PixelFormat.Companion.fromImageFormat( imageFormat: Int, @@ -11,6 +12,8 @@ fun PixelFormat.Companion.fromImageFormat( return when (imageFormat) { ImageFormat.FLEX_RGB_888 -> PixelFormat.RGB_RGB_8_BIT ImageFormat.FLEX_RGBA_8888 -> PixelFormat.RGB_RGBA_8_BIT + AndroidPixelFormat.RGB_888 -> PixelFormat.RGB_RGB_8_BIT + AndroidPixelFormat.RGBA_8888 -> PixelFormat.RGB_RGBA_8_BIT ImageFormat.DEPTH16 -> PixelFormat.DEPTH_16_BIT ImageFormat.DEPTH_POINT_CLOUD -> PixelFormat.DEPTH_POINT_CLOUD_32_BIT ImageFormat.YUV_420_888 -> { diff --git a/packages/react-native-vision-camera/android/src/main/java/com/margelo/nitro/camera/utils/ImageFormatUtils.kt b/packages/react-native-vision-camera/android/src/main/java/com/margelo/nitro/camera/utils/ImageFormatUtils.kt index 11997f8e27..6ffd567f44 100644 --- a/packages/react-native-vision-camera/android/src/main/java/com/margelo/nitro/camera/utils/ImageFormatUtils.kt +++ b/packages/react-native-vision-camera/android/src/main/java/com/margelo/nitro/camera/utils/ImageFormatUtils.kt @@ -1,6 +1,7 @@ package com.margelo.nitro.camera.utils import android.graphics.ImageFormat +import android.graphics.PixelFormat as AndroidPixelFormat object ImageFormatUtils { val allVideoFormats = @@ -15,6 +16,8 @@ object ImageFormatUtils { // RGB ImageFormat.FLEX_RGB_888, ImageFormat.FLEX_RGBA_8888, + AndroidPixelFormat.RGB_888, + AndroidPixelFormat.RGBA_8888, ) val allDepthFormats = arrayOf( diff --git a/packages/react-native-vision-camera/ios/Extensions/AVFoundation/AVCaptureDevice.Format+bitDepth.swift b/packages/react-native-vision-camera/ios/Extensions/AVFoundation/AVCaptureDevice.Format+bitDepth.swift index 67fdb58f58..7373013cfd 100644 --- a/packages/react-native-vision-camera/ios/Extensions/AVFoundation/AVCaptureDevice.Format+bitDepth.swift +++ b/packages/react-native-vision-camera/ios/Extensions/AVFoundation/AVCaptureDevice.Format+bitDepth.swift @@ -12,9 +12,11 @@ extension AVCaptureDevice.Format { var bitDepth: DynamicRangeBitDepth { let pixelFormat = self.formatDescription.mediaSubType switch pixelFormat { - case .yuv4208BitVideo, .yuv4208BitFull, .yuv4228BitVideo, .yuv4228BitFull: + case .yuv4208BitVideo, .yuv4208BitFull, .yuv4228BitVideo, .yuv4228BitFull, + .yuv4448BitVideo, .yuv4448BitFull: return .sdr8Bit - case .yuv42010BitVideo, .yuv42010BitFull, .yuv42210BitVideo, .yuv42210BitFull: + case .yuv42010BitVideo, .yuv42010BitFull, .yuv42210BitVideo, .yuv42210BitFull, + .yuv44410BitVideo, .yuv44410BitFull: return .hdr10Bit default: return .unknown diff --git a/packages/react-native-vision-camera/ios/Extensions/AVFoundation/AVCaptureDevice.Format+colorRange.swift b/packages/react-native-vision-camera/ios/Extensions/AVFoundation/AVCaptureDevice.Format+colorRange.swift index 5ad0ee05d8..72686b0534 100644 --- a/packages/react-native-vision-camera/ios/Extensions/AVFoundation/AVCaptureDevice.Format+colorRange.swift +++ b/packages/react-native-vision-camera/ios/Extensions/AVFoundation/AVCaptureDevice.Format+colorRange.swift @@ -12,9 +12,11 @@ extension AVCaptureDevice.Format { var colorRange: ColorRange { let pixelFormat = self.formatDescription.mediaSubType switch pixelFormat { - case .yuv4208BitVideo, .yuv42010BitVideo, .yuv4228BitVideo, .yuv42210BitVideo: + case .yuv4208BitVideo, .yuv42010BitVideo, .yuv4228BitVideo, .yuv42210BitVideo, + .yuv4448BitVideo, .yuv44410BitVideo: return .video - case .yuv4208BitFull, .yuv42010BitFull, .yuv4228BitFull, .yuv42210BitFull: + case .yuv4208BitFull, .yuv42010BitFull, .yuv4228BitFull, .yuv42210BitFull, + .yuv4448BitFull, .yuv44410BitFull: return .full default: return .unknown diff --git a/packages/react-native-vision-camera/ios/Extensions/Converters/AV+PixelFormat.swift b/packages/react-native-vision-camera/ios/Extensions/Converters/AV+PixelFormat.swift index 140bf37eb7..95fcf43eaa 100644 --- a/packages/react-native-vision-camera/ios/Extensions/Converters/AV+PixelFormat.swift +++ b/packages/react-native-vision-camera/ios/Extensions/Converters/AV+PixelFormat.swift @@ -44,6 +44,10 @@ extension PixelFormat { self = .yuv4448BitVideo case .yuv4448BitFull: self = .yuv4448BitFull + case .yuv44410BitVideo: + self = .yuv44410BitVideo + case .yuv44410BitFull: + self = .yuv44410BitFull case .rgbBgra8Bit: self = .rgbBgra8Bit case .rgbRgba8Bit: diff --git a/packages/react-native-vision-camera/ios/Extensions/CoreMedia/CMFormatDescription.MediaSubType+hidden.swift b/packages/react-native-vision-camera/ios/Extensions/CoreMedia/CMFormatDescription.MediaSubType+hidden.swift index 4219fccf82..4593dc5251 100644 --- a/packages/react-native-vision-camera/ios/Extensions/CoreMedia/CMFormatDescription.MediaSubType+hidden.swift +++ b/packages/react-native-vision-camera/ios/Extensions/CoreMedia/CMFormatDescription.MediaSubType+hidden.swift @@ -45,6 +45,11 @@ extension CMFormatDescription.MediaSubType { rawValue: kCVPixelFormatType_444YpCbCr8BiPlanarVideoRange) static let yuv4448BitFull = CMFormatDescription.MediaSubType( rawValue: kCVPixelFormatType_444YpCbCr8BiPlanarFullRange) + // YUV 4:4:4 10-Bit + static let yuv44410BitVideo = CMFormatDescription.MediaSubType( + rawValue: kCVPixelFormatType_444YpCbCr10BiPlanarVideoRange) + static let yuv44410BitFull = CMFormatDescription.MediaSubType( + rawValue: kCVPixelFormatType_444YpCbCr10BiPlanarFullRange) // BGRA static let rgbBgra8Bit = CMFormatDescription.MediaSubType(rawValue: kCVPixelFormatType_32BGRA) diff --git a/packages/react-native-vision-camera/ios/Hybrid Objects/Constraints/ResolvableConstraint/ResolvableConstraint+VideoDynamicRangeConstraint.swift b/packages/react-native-vision-camera/ios/Hybrid Objects/Constraints/ResolvableConstraint/ResolvableConstraint+VideoDynamicRangeConstraint.swift index f250bab54d..5740658633 100644 --- a/packages/react-native-vision-camera/ios/Hybrid Objects/Constraints/ResolvableConstraint/ResolvableConstraint+VideoDynamicRangeConstraint.swift +++ b/packages/react-native-vision-camera/ios/Hybrid Objects/Constraints/ResolvableConstraint/ResolvableConstraint+VideoDynamicRangeConstraint.swift @@ -86,9 +86,11 @@ extension TargetColorSpace { extension CMFormatDescription.MediaSubType { fileprivate var colorRange: ColorRange { switch self { - case .yuv4208BitVideo, .yuv42010BitVideo, .yuv4228BitVideo, .yuv42210BitVideo: + case .yuv4208BitVideo, .yuv42010BitVideo, .yuv4228BitVideo, .yuv42210BitVideo, + .yuv4448BitVideo, .yuv44410BitVideo: return .video - case .yuv4208BitFull, .yuv42010BitFull, .yuv4228BitFull, .yuv42210BitFull: + case .yuv4208BitFull, .yuv42010BitFull, .yuv4228BitFull, .yuv42210BitFull, + .yuv4448BitFull, .yuv44410BitFull: return .full default: return .unknown @@ -134,9 +136,11 @@ extension ColorRange { extension CMFormatDescription.MediaSubType { var bitDepth: DynamicRangeBitDepth { switch self { - case .yuv4208BitVideo, .yuv4208BitFull, .yuv4228BitVideo, .yuv4228BitFull: + case .yuv4208BitVideo, .yuv4208BitFull, .yuv4228BitVideo, .yuv4228BitFull, + .yuv4448BitVideo, .yuv4448BitFull: return .sdr8Bit - case .yuv42010BitVideo, .yuv42010BitFull, .yuv42210BitVideo, .yuv42210BitFull: + case .yuv42010BitVideo, .yuv42010BitFull, .yuv42210BitVideo, .yuv42210BitFull, + .yuv44410BitVideo, .yuv44410BitFull: return .hdr10Bit default: return .unknown diff --git a/packages/react-native-vision-camera/nitrogen/generated/android/c++/JPixelFormat.hpp b/packages/react-native-vision-camera/nitrogen/generated/android/c++/JPixelFormat.hpp index fd8f9aef30..54c385dbb5 100644 --- a/packages/react-native-vision-camera/nitrogen/generated/android/c++/JPixelFormat.hpp +++ b/packages/react-native-vision-camera/nitrogen/generated/android/c++/JPixelFormat.hpp @@ -7,112 +7,117 @@ #pragma once -#include #include "PixelFormat.hpp" +#include namespace margelo::nitro::camera { - using namespace facebook; +using namespace facebook; +/** + * The C++ JNI bridge between the C++ enum "PixelFormat" and the the Kotlin enum "PixelFormat". + */ +struct JPixelFormat final : public jni::JavaClass { +public: + static constexpr auto kJavaDescriptor = "Lcom/margelo/nitro/camera/PixelFormat;"; + +public: /** - * The C++ JNI bridge between the C++ enum "PixelFormat" and the the Kotlin enum "PixelFormat". + * Convert this Java/Kotlin-based enum to the C++ enum PixelFormat. */ - struct JPixelFormat final: public jni::JavaClass { - public: - static constexpr auto kJavaDescriptor = "Lcom/margelo/nitro/camera/PixelFormat;"; + [[maybe_unused]] [[nodiscard]] + PixelFormat toCpp() const { + static const auto clazz = javaClassStatic(); + static const auto fieldOrdinal = clazz->getField("value"); + int ordinal = this->getFieldValue(fieldOrdinal); + return static_cast(ordinal); + } - public: - /** - * Convert this Java/Kotlin-based enum to the C++ enum PixelFormat. - */ - [[maybe_unused]] - [[nodiscard]] - PixelFormat toCpp() const { - static const auto clazz = javaClassStatic(); - static const auto fieldOrdinal = clazz->getField("value"); - int ordinal = this->getFieldValue(fieldOrdinal); - return static_cast(ordinal); - } - - public: - /** - * Create a Java/Kotlin-based enum with the given C++ enum's value. - */ - [[maybe_unused]] - static jni::alias_ref fromCpp(PixelFormat value) { - static const auto clazz = javaClassStatic(); - switch (value) { - case PixelFormat::UNKNOWN: - static const auto fieldUNKNOWN = clazz->getStaticField("UNKNOWN"); - return clazz->getStaticFieldValue(fieldUNKNOWN); - case PixelFormat::YUV_420_8_BIT_VIDEO: - static const auto fieldYUV_420_8_BIT_VIDEO = clazz->getStaticField("YUV_420_8_BIT_VIDEO"); - return clazz->getStaticFieldValue(fieldYUV_420_8_BIT_VIDEO); - case PixelFormat::YUV_420_8_BIT_FULL: - static const auto fieldYUV_420_8_BIT_FULL = clazz->getStaticField("YUV_420_8_BIT_FULL"); - return clazz->getStaticFieldValue(fieldYUV_420_8_BIT_FULL); - case PixelFormat::YUV_420_10_BIT_VIDEO: - static const auto fieldYUV_420_10_BIT_VIDEO = clazz->getStaticField("YUV_420_10_BIT_VIDEO"); - return clazz->getStaticFieldValue(fieldYUV_420_10_BIT_VIDEO); - case PixelFormat::YUV_420_10_BIT_FULL: - static const auto fieldYUV_420_10_BIT_FULL = clazz->getStaticField("YUV_420_10_BIT_FULL"); - return clazz->getStaticFieldValue(fieldYUV_420_10_BIT_FULL); - case PixelFormat::YUV_422_8_BIT_VIDEO: - static const auto fieldYUV_422_8_BIT_VIDEO = clazz->getStaticField("YUV_422_8_BIT_VIDEO"); - return clazz->getStaticFieldValue(fieldYUV_422_8_BIT_VIDEO); - case PixelFormat::YUV_422_8_BIT_FULL: - static const auto fieldYUV_422_8_BIT_FULL = clazz->getStaticField("YUV_422_8_BIT_FULL"); - return clazz->getStaticFieldValue(fieldYUV_422_8_BIT_FULL); - case PixelFormat::YUV_422_10_BIT_VIDEO: - static const auto fieldYUV_422_10_BIT_VIDEO = clazz->getStaticField("YUV_422_10_BIT_VIDEO"); - return clazz->getStaticFieldValue(fieldYUV_422_10_BIT_VIDEO); - case PixelFormat::YUV_422_10_BIT_FULL: - static const auto fieldYUV_422_10_BIT_FULL = clazz->getStaticField("YUV_422_10_BIT_FULL"); - return clazz->getStaticFieldValue(fieldYUV_422_10_BIT_FULL); - case PixelFormat::YUV_444_8_BIT_VIDEO: - static const auto fieldYUV_444_8_BIT_VIDEO = clazz->getStaticField("YUV_444_8_BIT_VIDEO"); - return clazz->getStaticFieldValue(fieldYUV_444_8_BIT_VIDEO); - case PixelFormat::YUV_444_8_BIT_FULL: - static const auto fieldYUV_444_8_BIT_FULL = clazz->getStaticField("YUV_444_8_BIT_FULL"); - return clazz->getStaticFieldValue(fieldYUV_444_8_BIT_FULL); - case PixelFormat::RGB_BGRA_8_BIT: - static const auto fieldRGB_BGRA_8_BIT = clazz->getStaticField("RGB_BGRA_8_BIT"); - return clazz->getStaticFieldValue(fieldRGB_BGRA_8_BIT); - case PixelFormat::RGB_RGBA_8_BIT: - static const auto fieldRGB_RGBA_8_BIT = clazz->getStaticField("RGB_RGBA_8_BIT"); - return clazz->getStaticFieldValue(fieldRGB_RGBA_8_BIT); - case PixelFormat::RGB_RGB_8_BIT: - static const auto fieldRGB_RGB_8_BIT = clazz->getStaticField("RGB_RGB_8_BIT"); - return clazz->getStaticFieldValue(fieldRGB_RGB_8_BIT); - case PixelFormat::RAW_BAYER_PACKED96_12_BIT: - static const auto fieldRAW_BAYER_PACKED96_12_BIT = clazz->getStaticField("RAW_BAYER_PACKED96_12_BIT"); - return clazz->getStaticFieldValue(fieldRAW_BAYER_PACKED96_12_BIT); - case PixelFormat::RAW_BAYER_UNPACKED_16_BIT: - static const auto fieldRAW_BAYER_UNPACKED_16_BIT = clazz->getStaticField("RAW_BAYER_UNPACKED_16_BIT"); - return clazz->getStaticFieldValue(fieldRAW_BAYER_UNPACKED_16_BIT); - case PixelFormat::DEPTH_16_BIT: - static const auto fieldDEPTH_16_BIT = clazz->getStaticField("DEPTH_16_BIT"); - return clazz->getStaticFieldValue(fieldDEPTH_16_BIT); - case PixelFormat::DEPTH_32_BIT: - static const auto fieldDEPTH_32_BIT = clazz->getStaticField("DEPTH_32_BIT"); - return clazz->getStaticFieldValue(fieldDEPTH_32_BIT); - case PixelFormat::DEPTH_POINT_CLOUD_32_BIT: - static const auto fieldDEPTH_POINT_CLOUD_32_BIT = clazz->getStaticField("DEPTH_POINT_CLOUD_32_BIT"); - return clazz->getStaticFieldValue(fieldDEPTH_POINT_CLOUD_32_BIT); - case PixelFormat::DISPARITY_16_BIT: - static const auto fieldDISPARITY_16_BIT = clazz->getStaticField("DISPARITY_16_BIT"); - return clazz->getStaticFieldValue(fieldDISPARITY_16_BIT); - case PixelFormat::DISPARITY_32_BIT: - static const auto fieldDISPARITY_32_BIT = clazz->getStaticField("DISPARITY_32_BIT"); - return clazz->getStaticFieldValue(fieldDISPARITY_32_BIT); - case PixelFormat::PRIVATE: - static const auto fieldPRIVATE = clazz->getStaticField("PRIVATE"); - return clazz->getStaticFieldValue(fieldPRIVATE); - default: - std::string stringValue = std::to_string(static_cast(value)); - throw std::invalid_argument("Invalid enum value (" + stringValue + "!"); - } +public: + /** + * Create a Java/Kotlin-based enum with the given C++ enum's value. + */ + [[maybe_unused]] + static jni::alias_ref fromCpp(PixelFormat value) { + static const auto clazz = javaClassStatic(); + switch (value) { + case PixelFormat::UNKNOWN: + static const auto fieldUNKNOWN = clazz->getStaticField("UNKNOWN"); + return clazz->getStaticFieldValue(fieldUNKNOWN); + case PixelFormat::YUV_420_8_BIT_VIDEO: + static const auto fieldYUV_420_8_BIT_VIDEO = clazz->getStaticField("YUV_420_8_BIT_VIDEO"); + return clazz->getStaticFieldValue(fieldYUV_420_8_BIT_VIDEO); + case PixelFormat::YUV_420_8_BIT_FULL: + static const auto fieldYUV_420_8_BIT_FULL = clazz->getStaticField("YUV_420_8_BIT_FULL"); + return clazz->getStaticFieldValue(fieldYUV_420_8_BIT_FULL); + case PixelFormat::YUV_420_10_BIT_VIDEO: + static const auto fieldYUV_420_10_BIT_VIDEO = clazz->getStaticField("YUV_420_10_BIT_VIDEO"); + return clazz->getStaticFieldValue(fieldYUV_420_10_BIT_VIDEO); + case PixelFormat::YUV_420_10_BIT_FULL: + static const auto fieldYUV_420_10_BIT_FULL = clazz->getStaticField("YUV_420_10_BIT_FULL"); + return clazz->getStaticFieldValue(fieldYUV_420_10_BIT_FULL); + case PixelFormat::YUV_422_8_BIT_VIDEO: + static const auto fieldYUV_422_8_BIT_VIDEO = clazz->getStaticField("YUV_422_8_BIT_VIDEO"); + return clazz->getStaticFieldValue(fieldYUV_422_8_BIT_VIDEO); + case PixelFormat::YUV_422_8_BIT_FULL: + static const auto fieldYUV_422_8_BIT_FULL = clazz->getStaticField("YUV_422_8_BIT_FULL"); + return clazz->getStaticFieldValue(fieldYUV_422_8_BIT_FULL); + case PixelFormat::YUV_422_10_BIT_VIDEO: + static const auto fieldYUV_422_10_BIT_VIDEO = clazz->getStaticField("YUV_422_10_BIT_VIDEO"); + return clazz->getStaticFieldValue(fieldYUV_422_10_BIT_VIDEO); + case PixelFormat::YUV_422_10_BIT_FULL: + static const auto fieldYUV_422_10_BIT_FULL = clazz->getStaticField("YUV_422_10_BIT_FULL"); + return clazz->getStaticFieldValue(fieldYUV_422_10_BIT_FULL); + case PixelFormat::YUV_444_8_BIT_VIDEO: + static const auto fieldYUV_444_8_BIT_VIDEO = clazz->getStaticField("YUV_444_8_BIT_VIDEO"); + return clazz->getStaticFieldValue(fieldYUV_444_8_BIT_VIDEO); + case PixelFormat::YUV_444_8_BIT_FULL: + static const auto fieldYUV_444_8_BIT_FULL = clazz->getStaticField("YUV_444_8_BIT_FULL"); + return clazz->getStaticFieldValue(fieldYUV_444_8_BIT_FULL); + case PixelFormat::YUV_444_10_BIT_VIDEO: + static const auto fieldYUV_444_10_BIT_VIDEO = clazz->getStaticField("YUV_444_10_BIT_VIDEO"); + return clazz->getStaticFieldValue(fieldYUV_444_10_BIT_VIDEO); + case PixelFormat::YUV_444_10_BIT_FULL: + static const auto fieldYUV_444_10_BIT_FULL = clazz->getStaticField("YUV_444_10_BIT_FULL"); + return clazz->getStaticFieldValue(fieldYUV_444_10_BIT_FULL); + case PixelFormat::RGB_BGRA_8_BIT: + static const auto fieldRGB_BGRA_8_BIT = clazz->getStaticField("RGB_BGRA_8_BIT"); + return clazz->getStaticFieldValue(fieldRGB_BGRA_8_BIT); + case PixelFormat::RGB_RGBA_8_BIT: + static const auto fieldRGB_RGBA_8_BIT = clazz->getStaticField("RGB_RGBA_8_BIT"); + return clazz->getStaticFieldValue(fieldRGB_RGBA_8_BIT); + case PixelFormat::RGB_RGB_8_BIT: + static const auto fieldRGB_RGB_8_BIT = clazz->getStaticField("RGB_RGB_8_BIT"); + return clazz->getStaticFieldValue(fieldRGB_RGB_8_BIT); + case PixelFormat::RAW_BAYER_PACKED96_12_BIT: + static const auto fieldRAW_BAYER_PACKED96_12_BIT = clazz->getStaticField("RAW_BAYER_PACKED96_12_BIT"); + return clazz->getStaticFieldValue(fieldRAW_BAYER_PACKED96_12_BIT); + case PixelFormat::RAW_BAYER_UNPACKED_16_BIT: + static const auto fieldRAW_BAYER_UNPACKED_16_BIT = clazz->getStaticField("RAW_BAYER_UNPACKED_16_BIT"); + return clazz->getStaticFieldValue(fieldRAW_BAYER_UNPACKED_16_BIT); + case PixelFormat::DEPTH_16_BIT: + static const auto fieldDEPTH_16_BIT = clazz->getStaticField("DEPTH_16_BIT"); + return clazz->getStaticFieldValue(fieldDEPTH_16_BIT); + case PixelFormat::DEPTH_32_BIT: + static const auto fieldDEPTH_32_BIT = clazz->getStaticField("DEPTH_32_BIT"); + return clazz->getStaticFieldValue(fieldDEPTH_32_BIT); + case PixelFormat::DEPTH_POINT_CLOUD_32_BIT: + static const auto fieldDEPTH_POINT_CLOUD_32_BIT = clazz->getStaticField("DEPTH_POINT_CLOUD_32_BIT"); + return clazz->getStaticFieldValue(fieldDEPTH_POINT_CLOUD_32_BIT); + case PixelFormat::DISPARITY_16_BIT: + static const auto fieldDISPARITY_16_BIT = clazz->getStaticField("DISPARITY_16_BIT"); + return clazz->getStaticFieldValue(fieldDISPARITY_16_BIT); + case PixelFormat::DISPARITY_32_BIT: + static const auto fieldDISPARITY_32_BIT = clazz->getStaticField("DISPARITY_32_BIT"); + return clazz->getStaticFieldValue(fieldDISPARITY_32_BIT); + case PixelFormat::PRIVATE: + static const auto fieldPRIVATE = clazz->getStaticField("PRIVATE"); + return clazz->getStaticFieldValue(fieldPRIVATE); + default: + std::string stringValue = std::to_string(static_cast(value)); + throw std::invalid_argument("Invalid enum value (" + stringValue + "!"); } - }; + } +}; } // namespace margelo::nitro::camera diff --git a/packages/react-native-vision-camera/nitrogen/generated/android/kotlin/com/margelo/nitro/camera/PixelFormat.kt b/packages/react-native-vision-camera/nitrogen/generated/android/kotlin/com/margelo/nitro/camera/PixelFormat.kt index 89419a9722..9ff42977ae 100644 --- a/packages/react-native-vision-camera/nitrogen/generated/android/kotlin/com/margelo/nitro/camera/PixelFormat.kt +++ b/packages/react-native-vision-camera/nitrogen/generated/android/kotlin/com/margelo/nitro/camera/PixelFormat.kt @@ -27,17 +27,19 @@ enum class PixelFormat(@DoNotStrip @Keep val value: Int) { YUV_422_10_BIT_FULL(8), YUV_444_8_BIT_VIDEO(9), YUV_444_8_BIT_FULL(10), - RGB_BGRA_8_BIT(11), - RGB_RGBA_8_BIT(12), - RGB_RGB_8_BIT(13), - RAW_BAYER_PACKED96_12_BIT(14), - RAW_BAYER_UNPACKED_16_BIT(15), - DEPTH_16_BIT(16), - DEPTH_32_BIT(17), - DEPTH_POINT_CLOUD_32_BIT(18), - DISPARITY_16_BIT(19), - DISPARITY_32_BIT(20), - PRIVATE(21); + YUV_444_10_BIT_VIDEO(11), + YUV_444_10_BIT_FULL(12), + RGB_BGRA_8_BIT(13), + RGB_RGBA_8_BIT(14), + RGB_RGB_8_BIT(15), + RAW_BAYER_PACKED96_12_BIT(16), + RAW_BAYER_UNPACKED_16_BIT(17), + DEPTH_16_BIT(18), + DEPTH_32_BIT(19), + DEPTH_POINT_CLOUD_32_BIT(20), + DISPARITY_16_BIT(21), + DISPARITY_32_BIT(22), + PRIVATE(23); companion object } diff --git a/packages/react-native-vision-camera/nitrogen/generated/ios/swift/PixelFormat.swift b/packages/react-native-vision-camera/nitrogen/generated/ios/swift/PixelFormat.swift index c5dbf2268a..872984e11c 100644 --- a/packages/react-native-vision-camera/nitrogen/generated/ios/swift/PixelFormat.swift +++ b/packages/react-native-vision-camera/nitrogen/generated/ios/swift/PixelFormat.swift @@ -5,116 +5,122 @@ /// Copyright © Marc Rousavy @ Margelo /// -/** - * Represents the JS union `PixelFormat`, backed by a C++ enum. - */ +/// Represents the JS union `PixelFormat`, backed by a C++ enum. public typealias PixelFormat = margelo.nitro.camera.PixelFormat -public extension PixelFormat { +extension PixelFormat { /** * Get a PixelFormat for the given String value, or * return `nil` if the given value was invalid/unknown. */ - init?(fromString string: String) { + public init?(fromString string: String) { switch string { - case "unknown": - self = .unknown - case "yuv-420-8-bit-video": - self = .yuv4208BitVideo - case "yuv-420-8-bit-full": - self = .yuv4208BitFull - case "yuv-420-10-bit-video": - self = .yuv42010BitVideo - case "yuv-420-10-bit-full": - self = .yuv42010BitFull - case "yuv-422-8-bit-video": - self = .yuv4228BitVideo - case "yuv-422-8-bit-full": - self = .yuv4228BitFull - case "yuv-422-10-bit-video": - self = .yuv42210BitVideo - case "yuv-422-10-bit-full": - self = .yuv42210BitFull - case "yuv-444-8-bit-video": - self = .yuv4448BitVideo - case "yuv-444-8-bit-full": - self = .yuv4448BitFull - case "rgb-bgra-8-bit": - self = .rgbBgra8Bit - case "rgb-rgba-8-bit": - self = .rgbRgba8Bit - case "rgb-rgb-8-bit": - self = .rgbRgb8Bit - case "raw-bayer-packed96-12-bit": - self = .rawBayerPacked9612Bit - case "raw-bayer-unpacked-16-bit": - self = .rawBayerUnpacked16Bit - case "depth-16-bit": - self = .depth16Bit - case "depth-32-bit": - self = .depth32Bit - case "depth-point-cloud-32-bit": - self = .depthPointCloud32Bit - case "disparity-16-bit": - self = .disparity16Bit - case "disparity-32-bit": - self = .disparity32Bit - case "private": - self = .private - default: - return nil + case "unknown": + self = .unknown + case "yuv-420-8-bit-video": + self = .yuv4208BitVideo + case "yuv-420-8-bit-full": + self = .yuv4208BitFull + case "yuv-420-10-bit-video": + self = .yuv42010BitVideo + case "yuv-420-10-bit-full": + self = .yuv42010BitFull + case "yuv-422-8-bit-video": + self = .yuv4228BitVideo + case "yuv-422-8-bit-full": + self = .yuv4228BitFull + case "yuv-422-10-bit-video": + self = .yuv42210BitVideo + case "yuv-422-10-bit-full": + self = .yuv42210BitFull + case "yuv-444-8-bit-video": + self = .yuv4448BitVideo + case "yuv-444-8-bit-full": + self = .yuv4448BitFull + case "yuv-444-10-bit-video": + self = .yuv44410BitVideo + case "yuv-444-10-bit-full": + self = .yuv44410BitFull + case "rgb-bgra-8-bit": + self = .rgbBgra8Bit + case "rgb-rgba-8-bit": + self = .rgbRgba8Bit + case "rgb-rgb-8-bit": + self = .rgbRgb8Bit + case "raw-bayer-packed96-12-bit": + self = .rawBayerPacked9612Bit + case "raw-bayer-unpacked-16-bit": + self = .rawBayerUnpacked16Bit + case "depth-16-bit": + self = .depth16Bit + case "depth-32-bit": + self = .depth32Bit + case "depth-point-cloud-32-bit": + self = .depthPointCloud32Bit + case "disparity-16-bit": + self = .disparity16Bit + case "disparity-32-bit": + self = .disparity32Bit + case "private": + self = .private + default: + return nil } } /** * Get the String value this PixelFormat represents. */ - var stringValue: String { + public var stringValue: String { switch self { - case .unknown: - return "unknown" - case .yuv4208BitVideo: - return "yuv-420-8-bit-video" - case .yuv4208BitFull: - return "yuv-420-8-bit-full" - case .yuv42010BitVideo: - return "yuv-420-10-bit-video" - case .yuv42010BitFull: - return "yuv-420-10-bit-full" - case .yuv4228BitVideo: - return "yuv-422-8-bit-video" - case .yuv4228BitFull: - return "yuv-422-8-bit-full" - case .yuv42210BitVideo: - return "yuv-422-10-bit-video" - case .yuv42210BitFull: - return "yuv-422-10-bit-full" - case .yuv4448BitVideo: - return "yuv-444-8-bit-video" - case .yuv4448BitFull: - return "yuv-444-8-bit-full" - case .rgbBgra8Bit: - return "rgb-bgra-8-bit" - case .rgbRgba8Bit: - return "rgb-rgba-8-bit" - case .rgbRgb8Bit: - return "rgb-rgb-8-bit" - case .rawBayerPacked9612Bit: - return "raw-bayer-packed96-12-bit" - case .rawBayerUnpacked16Bit: - return "raw-bayer-unpacked-16-bit" - case .depth16Bit: - return "depth-16-bit" - case .depth32Bit: - return "depth-32-bit" - case .depthPointCloud32Bit: - return "depth-point-cloud-32-bit" - case .disparity16Bit: - return "disparity-16-bit" - case .disparity32Bit: - return "disparity-32-bit" - case .private: - return "private" + case .unknown: + return "unknown" + case .yuv4208BitVideo: + return "yuv-420-8-bit-video" + case .yuv4208BitFull: + return "yuv-420-8-bit-full" + case .yuv42010BitVideo: + return "yuv-420-10-bit-video" + case .yuv42010BitFull: + return "yuv-420-10-bit-full" + case .yuv4228BitVideo: + return "yuv-422-8-bit-video" + case .yuv4228BitFull: + return "yuv-422-8-bit-full" + case .yuv42210BitVideo: + return "yuv-422-10-bit-video" + case .yuv42210BitFull: + return "yuv-422-10-bit-full" + case .yuv4448BitVideo: + return "yuv-444-8-bit-video" + case .yuv4448BitFull: + return "yuv-444-8-bit-full" + case .yuv44410BitVideo: + return "yuv-444-10-bit-video" + case .yuv44410BitFull: + return "yuv-444-10-bit-full" + case .rgbBgra8Bit: + return "rgb-bgra-8-bit" + case .rgbRgba8Bit: + return "rgb-rgba-8-bit" + case .rgbRgb8Bit: + return "rgb-rgb-8-bit" + case .rawBayerPacked9612Bit: + return "raw-bayer-packed96-12-bit" + case .rawBayerUnpacked16Bit: + return "raw-bayer-unpacked-16-bit" + case .depth16Bit: + return "depth-16-bit" + case .depth32Bit: + return "depth-32-bit" + case .depthPointCloud32Bit: + return "depth-point-cloud-32-bit" + case .disparity16Bit: + return "disparity-16-bit" + case .disparity32Bit: + return "disparity-32-bit" + case .private: + return "private" } } } diff --git a/packages/react-native-vision-camera/nitrogen/generated/shared/c++/PixelFormat.hpp b/packages/react-native-vision-camera/nitrogen/generated/shared/c++/PixelFormat.hpp index 4302debec1..aae2bc3756 100644 --- a/packages/react-native-vision-camera/nitrogen/generated/shared/c++/PixelFormat.hpp +++ b/packages/react-native-vision-camera/nitrogen/generated/shared/c++/PixelFormat.hpp @@ -25,132 +25,187 @@ namespace margelo::nitro::camera { - /** - * An enum which can be represented as a JavaScript union (PixelFormat). - */ - enum class PixelFormat { - UNKNOWN SWIFT_NAME(unknown) = 0, - YUV_420_8_BIT_VIDEO SWIFT_NAME(yuv4208BitVideo) = 1, - YUV_420_8_BIT_FULL SWIFT_NAME(yuv4208BitFull) = 2, - YUV_420_10_BIT_VIDEO SWIFT_NAME(yuv42010BitVideo) = 3, - YUV_420_10_BIT_FULL SWIFT_NAME(yuv42010BitFull) = 4, - YUV_422_8_BIT_VIDEO SWIFT_NAME(yuv4228BitVideo) = 5, - YUV_422_8_BIT_FULL SWIFT_NAME(yuv4228BitFull) = 6, - YUV_422_10_BIT_VIDEO SWIFT_NAME(yuv42210BitVideo) = 7, - YUV_422_10_BIT_FULL SWIFT_NAME(yuv42210BitFull) = 8, - YUV_444_8_BIT_VIDEO SWIFT_NAME(yuv4448BitVideo) = 9, - YUV_444_8_BIT_FULL SWIFT_NAME(yuv4448BitFull) = 10, - RGB_BGRA_8_BIT SWIFT_NAME(rgbBgra8Bit) = 11, - RGB_RGBA_8_BIT SWIFT_NAME(rgbRgba8Bit) = 12, - RGB_RGB_8_BIT SWIFT_NAME(rgbRgb8Bit) = 13, - RAW_BAYER_PACKED96_12_BIT SWIFT_NAME(rawBayerPacked9612Bit) = 14, - RAW_BAYER_UNPACKED_16_BIT SWIFT_NAME(rawBayerUnpacked16Bit) = 15, - DEPTH_16_BIT SWIFT_NAME(depth16Bit) = 16, - DEPTH_32_BIT SWIFT_NAME(depth32Bit) = 17, - DEPTH_POINT_CLOUD_32_BIT SWIFT_NAME(depthPointCloud32Bit) = 18, - DISPARITY_16_BIT SWIFT_NAME(disparity16Bit) = 19, - DISPARITY_32_BIT SWIFT_NAME(disparity32Bit) = 20, - PRIVATE SWIFT_NAME(private) = 21, - } CLOSED_ENUM; +/** + * An enum which can be represented as a JavaScript union (PixelFormat). + */ +enum class PixelFormat { + UNKNOWN SWIFT_NAME(unknown) = 0, + YUV_420_8_BIT_VIDEO SWIFT_NAME(yuv4208BitVideo) = 1, + YUV_420_8_BIT_FULL SWIFT_NAME(yuv4208BitFull) = 2, + YUV_420_10_BIT_VIDEO SWIFT_NAME(yuv42010BitVideo) = 3, + YUV_420_10_BIT_FULL SWIFT_NAME(yuv42010BitFull) = 4, + YUV_422_8_BIT_VIDEO SWIFT_NAME(yuv4228BitVideo) = 5, + YUV_422_8_BIT_FULL SWIFT_NAME(yuv4228BitFull) = 6, + YUV_422_10_BIT_VIDEO SWIFT_NAME(yuv42210BitVideo) = 7, + YUV_422_10_BIT_FULL SWIFT_NAME(yuv42210BitFull) = 8, + YUV_444_8_BIT_VIDEO SWIFT_NAME(yuv4448BitVideo) = 9, + YUV_444_8_BIT_FULL SWIFT_NAME(yuv4448BitFull) = 10, + YUV_444_10_BIT_VIDEO SWIFT_NAME(yuv44410BitVideo) = 11, + YUV_444_10_BIT_FULL SWIFT_NAME(yuv44410BitFull) = 12, + RGB_BGRA_8_BIT SWIFT_NAME(rgbBgra8Bit) = 13, + RGB_RGBA_8_BIT SWIFT_NAME(rgbRgba8Bit) = 14, + RGB_RGB_8_BIT SWIFT_NAME(rgbRgb8Bit) = 15, + RAW_BAYER_PACKED96_12_BIT SWIFT_NAME(rawBayerPacked9612Bit) = 16, + RAW_BAYER_UNPACKED_16_BIT SWIFT_NAME(rawBayerUnpacked16Bit) = 17, + DEPTH_16_BIT SWIFT_NAME(depth16Bit) = 18, + DEPTH_32_BIT SWIFT_NAME(depth32Bit) = 19, + DEPTH_POINT_CLOUD_32_BIT SWIFT_NAME(depthPointCloud32Bit) = 20, + DISPARITY_16_BIT SWIFT_NAME(disparity16Bit) = 21, + DISPARITY_32_BIT SWIFT_NAME(disparity32Bit) = 22, + PRIVATE SWIFT_NAME(private) = 23, +} CLOSED_ENUM; } // namespace margelo::nitro::camera namespace margelo::nitro { - // C++ PixelFormat <> JS PixelFormat (union) - template <> - struct JSIConverter final { - static inline margelo::nitro::camera::PixelFormat fromJSI(jsi::Runtime& runtime, const jsi::Value& arg) { - std::string unionValue = JSIConverter::fromJSI(runtime, arg); - switch (hashString(unionValue.c_str(), unionValue.size())) { - case hashString("unknown"): return margelo::nitro::camera::PixelFormat::UNKNOWN; - case hashString("yuv-420-8-bit-video"): return margelo::nitro::camera::PixelFormat::YUV_420_8_BIT_VIDEO; - case hashString("yuv-420-8-bit-full"): return margelo::nitro::camera::PixelFormat::YUV_420_8_BIT_FULL; - case hashString("yuv-420-10-bit-video"): return margelo::nitro::camera::PixelFormat::YUV_420_10_BIT_VIDEO; - case hashString("yuv-420-10-bit-full"): return margelo::nitro::camera::PixelFormat::YUV_420_10_BIT_FULL; - case hashString("yuv-422-8-bit-video"): return margelo::nitro::camera::PixelFormat::YUV_422_8_BIT_VIDEO; - case hashString("yuv-422-8-bit-full"): return margelo::nitro::camera::PixelFormat::YUV_422_8_BIT_FULL; - case hashString("yuv-422-10-bit-video"): return margelo::nitro::camera::PixelFormat::YUV_422_10_BIT_VIDEO; - case hashString("yuv-422-10-bit-full"): return margelo::nitro::camera::PixelFormat::YUV_422_10_BIT_FULL; - case hashString("yuv-444-8-bit-video"): return margelo::nitro::camera::PixelFormat::YUV_444_8_BIT_VIDEO; - case hashString("yuv-444-8-bit-full"): return margelo::nitro::camera::PixelFormat::YUV_444_8_BIT_FULL; - case hashString("rgb-bgra-8-bit"): return margelo::nitro::camera::PixelFormat::RGB_BGRA_8_BIT; - case hashString("rgb-rgba-8-bit"): return margelo::nitro::camera::PixelFormat::RGB_RGBA_8_BIT; - case hashString("rgb-rgb-8-bit"): return margelo::nitro::camera::PixelFormat::RGB_RGB_8_BIT; - case hashString("raw-bayer-packed96-12-bit"): return margelo::nitro::camera::PixelFormat::RAW_BAYER_PACKED96_12_BIT; - case hashString("raw-bayer-unpacked-16-bit"): return margelo::nitro::camera::PixelFormat::RAW_BAYER_UNPACKED_16_BIT; - case hashString("depth-16-bit"): return margelo::nitro::camera::PixelFormat::DEPTH_16_BIT; - case hashString("depth-32-bit"): return margelo::nitro::camera::PixelFormat::DEPTH_32_BIT; - case hashString("depth-point-cloud-32-bit"): return margelo::nitro::camera::PixelFormat::DEPTH_POINT_CLOUD_32_BIT; - case hashString("disparity-16-bit"): return margelo::nitro::camera::PixelFormat::DISPARITY_16_BIT; - case hashString("disparity-32-bit"): return margelo::nitro::camera::PixelFormat::DISPARITY_32_BIT; - case hashString("private"): return margelo::nitro::camera::PixelFormat::PRIVATE; - default: [[unlikely]] - throw std::invalid_argument("Cannot convert \"" + unionValue + "\" to enum PixelFormat - invalid value!"); - } +// C++ PixelFormat <> JS PixelFormat (union) +template <> +struct JSIConverter final { + static inline margelo::nitro::camera::PixelFormat fromJSI(jsi::Runtime& runtime, const jsi::Value& arg) { + std::string unionValue = JSIConverter::fromJSI(runtime, arg); + switch (hashString(unionValue.c_str(), unionValue.size())) { + case hashString("unknown"): + return margelo::nitro::camera::PixelFormat::UNKNOWN; + case hashString("yuv-420-8-bit-video"): + return margelo::nitro::camera::PixelFormat::YUV_420_8_BIT_VIDEO; + case hashString("yuv-420-8-bit-full"): + return margelo::nitro::camera::PixelFormat::YUV_420_8_BIT_FULL; + case hashString("yuv-420-10-bit-video"): + return margelo::nitro::camera::PixelFormat::YUV_420_10_BIT_VIDEO; + case hashString("yuv-420-10-bit-full"): + return margelo::nitro::camera::PixelFormat::YUV_420_10_BIT_FULL; + case hashString("yuv-422-8-bit-video"): + return margelo::nitro::camera::PixelFormat::YUV_422_8_BIT_VIDEO; + case hashString("yuv-422-8-bit-full"): + return margelo::nitro::camera::PixelFormat::YUV_422_8_BIT_FULL; + case hashString("yuv-422-10-bit-video"): + return margelo::nitro::camera::PixelFormat::YUV_422_10_BIT_VIDEO; + case hashString("yuv-422-10-bit-full"): + return margelo::nitro::camera::PixelFormat::YUV_422_10_BIT_FULL; + case hashString("yuv-444-8-bit-video"): + return margelo::nitro::camera::PixelFormat::YUV_444_8_BIT_VIDEO; + case hashString("yuv-444-8-bit-full"): + return margelo::nitro::camera::PixelFormat::YUV_444_8_BIT_FULL; + case hashString("yuv-444-10-bit-video"): + return margelo::nitro::camera::PixelFormat::YUV_444_10_BIT_VIDEO; + case hashString("yuv-444-10-bit-full"): + return margelo::nitro::camera::PixelFormat::YUV_444_10_BIT_FULL; + case hashString("rgb-bgra-8-bit"): + return margelo::nitro::camera::PixelFormat::RGB_BGRA_8_BIT; + case hashString("rgb-rgba-8-bit"): + return margelo::nitro::camera::PixelFormat::RGB_RGBA_8_BIT; + case hashString("rgb-rgb-8-bit"): + return margelo::nitro::camera::PixelFormat::RGB_RGB_8_BIT; + case hashString("raw-bayer-packed96-12-bit"): + return margelo::nitro::camera::PixelFormat::RAW_BAYER_PACKED96_12_BIT; + case hashString("raw-bayer-unpacked-16-bit"): + return margelo::nitro::camera::PixelFormat::RAW_BAYER_UNPACKED_16_BIT; + case hashString("depth-16-bit"): + return margelo::nitro::camera::PixelFormat::DEPTH_16_BIT; + case hashString("depth-32-bit"): + return margelo::nitro::camera::PixelFormat::DEPTH_32_BIT; + case hashString("depth-point-cloud-32-bit"): + return margelo::nitro::camera::PixelFormat::DEPTH_POINT_CLOUD_32_BIT; + case hashString("disparity-16-bit"): + return margelo::nitro::camera::PixelFormat::DISPARITY_16_BIT; + case hashString("disparity-32-bit"): + return margelo::nitro::camera::PixelFormat::DISPARITY_32_BIT; + case hashString("private"): + return margelo::nitro::camera::PixelFormat::PRIVATE; + default: + [[unlikely]] throw std::invalid_argument("Cannot convert \"" + unionValue + "\" to enum PixelFormat - invalid value!"); } - static inline jsi::Value toJSI(jsi::Runtime& runtime, margelo::nitro::camera::PixelFormat arg) { - switch (arg) { - case margelo::nitro::camera::PixelFormat::UNKNOWN: return JSIConverter::toJSI(runtime, "unknown"); - case margelo::nitro::camera::PixelFormat::YUV_420_8_BIT_VIDEO: return JSIConverter::toJSI(runtime, "yuv-420-8-bit-video"); - case margelo::nitro::camera::PixelFormat::YUV_420_8_BIT_FULL: return JSIConverter::toJSI(runtime, "yuv-420-8-bit-full"); - case margelo::nitro::camera::PixelFormat::YUV_420_10_BIT_VIDEO: return JSIConverter::toJSI(runtime, "yuv-420-10-bit-video"); - case margelo::nitro::camera::PixelFormat::YUV_420_10_BIT_FULL: return JSIConverter::toJSI(runtime, "yuv-420-10-bit-full"); - case margelo::nitro::camera::PixelFormat::YUV_422_8_BIT_VIDEO: return JSIConverter::toJSI(runtime, "yuv-422-8-bit-video"); - case margelo::nitro::camera::PixelFormat::YUV_422_8_BIT_FULL: return JSIConverter::toJSI(runtime, "yuv-422-8-bit-full"); - case margelo::nitro::camera::PixelFormat::YUV_422_10_BIT_VIDEO: return JSIConverter::toJSI(runtime, "yuv-422-10-bit-video"); - case margelo::nitro::camera::PixelFormat::YUV_422_10_BIT_FULL: return JSIConverter::toJSI(runtime, "yuv-422-10-bit-full"); - case margelo::nitro::camera::PixelFormat::YUV_444_8_BIT_VIDEO: return JSIConverter::toJSI(runtime, "yuv-444-8-bit-video"); - case margelo::nitro::camera::PixelFormat::YUV_444_8_BIT_FULL: return JSIConverter::toJSI(runtime, "yuv-444-8-bit-full"); - case margelo::nitro::camera::PixelFormat::RGB_BGRA_8_BIT: return JSIConverter::toJSI(runtime, "rgb-bgra-8-bit"); - case margelo::nitro::camera::PixelFormat::RGB_RGBA_8_BIT: return JSIConverter::toJSI(runtime, "rgb-rgba-8-bit"); - case margelo::nitro::camera::PixelFormat::RGB_RGB_8_BIT: return JSIConverter::toJSI(runtime, "rgb-rgb-8-bit"); - case margelo::nitro::camera::PixelFormat::RAW_BAYER_PACKED96_12_BIT: return JSIConverter::toJSI(runtime, "raw-bayer-packed96-12-bit"); - case margelo::nitro::camera::PixelFormat::RAW_BAYER_UNPACKED_16_BIT: return JSIConverter::toJSI(runtime, "raw-bayer-unpacked-16-bit"); - case margelo::nitro::camera::PixelFormat::DEPTH_16_BIT: return JSIConverter::toJSI(runtime, "depth-16-bit"); - case margelo::nitro::camera::PixelFormat::DEPTH_32_BIT: return JSIConverter::toJSI(runtime, "depth-32-bit"); - case margelo::nitro::camera::PixelFormat::DEPTH_POINT_CLOUD_32_BIT: return JSIConverter::toJSI(runtime, "depth-point-cloud-32-bit"); - case margelo::nitro::camera::PixelFormat::DISPARITY_16_BIT: return JSIConverter::toJSI(runtime, "disparity-16-bit"); - case margelo::nitro::camera::PixelFormat::DISPARITY_32_BIT: return JSIConverter::toJSI(runtime, "disparity-32-bit"); - case margelo::nitro::camera::PixelFormat::PRIVATE: return JSIConverter::toJSI(runtime, "private"); - default: [[unlikely]] - throw std::invalid_argument("Cannot convert PixelFormat to JS - invalid value: " - + std::to_string(static_cast(arg)) + "!"); - } + } + static inline jsi::Value toJSI(jsi::Runtime& runtime, margelo::nitro::camera::PixelFormat arg) { + switch (arg) { + case margelo::nitro::camera::PixelFormat::UNKNOWN: + return JSIConverter::toJSI(runtime, "unknown"); + case margelo::nitro::camera::PixelFormat::YUV_420_8_BIT_VIDEO: + return JSIConverter::toJSI(runtime, "yuv-420-8-bit-video"); + case margelo::nitro::camera::PixelFormat::YUV_420_8_BIT_FULL: + return JSIConverter::toJSI(runtime, "yuv-420-8-bit-full"); + case margelo::nitro::camera::PixelFormat::YUV_420_10_BIT_VIDEO: + return JSIConverter::toJSI(runtime, "yuv-420-10-bit-video"); + case margelo::nitro::camera::PixelFormat::YUV_420_10_BIT_FULL: + return JSIConverter::toJSI(runtime, "yuv-420-10-bit-full"); + case margelo::nitro::camera::PixelFormat::YUV_422_8_BIT_VIDEO: + return JSIConverter::toJSI(runtime, "yuv-422-8-bit-video"); + case margelo::nitro::camera::PixelFormat::YUV_422_8_BIT_FULL: + return JSIConverter::toJSI(runtime, "yuv-422-8-bit-full"); + case margelo::nitro::camera::PixelFormat::YUV_422_10_BIT_VIDEO: + return JSIConverter::toJSI(runtime, "yuv-422-10-bit-video"); + case margelo::nitro::camera::PixelFormat::YUV_422_10_BIT_FULL: + return JSIConverter::toJSI(runtime, "yuv-422-10-bit-full"); + case margelo::nitro::camera::PixelFormat::YUV_444_8_BIT_VIDEO: + return JSIConverter::toJSI(runtime, "yuv-444-8-bit-video"); + case margelo::nitro::camera::PixelFormat::YUV_444_8_BIT_FULL: + return JSIConverter::toJSI(runtime, "yuv-444-8-bit-full"); + case margelo::nitro::camera::PixelFormat::YUV_444_10_BIT_VIDEO: + return JSIConverter::toJSI(runtime, "yuv-444-10-bit-video"); + case margelo::nitro::camera::PixelFormat::YUV_444_10_BIT_FULL: + return JSIConverter::toJSI(runtime, "yuv-444-10-bit-full"); + case margelo::nitro::camera::PixelFormat::RGB_BGRA_8_BIT: + return JSIConverter::toJSI(runtime, "rgb-bgra-8-bit"); + case margelo::nitro::camera::PixelFormat::RGB_RGBA_8_BIT: + return JSIConverter::toJSI(runtime, "rgb-rgba-8-bit"); + case margelo::nitro::camera::PixelFormat::RGB_RGB_8_BIT: + return JSIConverter::toJSI(runtime, "rgb-rgb-8-bit"); + case margelo::nitro::camera::PixelFormat::RAW_BAYER_PACKED96_12_BIT: + return JSIConverter::toJSI(runtime, "raw-bayer-packed96-12-bit"); + case margelo::nitro::camera::PixelFormat::RAW_BAYER_UNPACKED_16_BIT: + return JSIConverter::toJSI(runtime, "raw-bayer-unpacked-16-bit"); + case margelo::nitro::camera::PixelFormat::DEPTH_16_BIT: + return JSIConverter::toJSI(runtime, "depth-16-bit"); + case margelo::nitro::camera::PixelFormat::DEPTH_32_BIT: + return JSIConverter::toJSI(runtime, "depth-32-bit"); + case margelo::nitro::camera::PixelFormat::DEPTH_POINT_CLOUD_32_BIT: + return JSIConverter::toJSI(runtime, "depth-point-cloud-32-bit"); + case margelo::nitro::camera::PixelFormat::DISPARITY_16_BIT: + return JSIConverter::toJSI(runtime, "disparity-16-bit"); + case margelo::nitro::camera::PixelFormat::DISPARITY_32_BIT: + return JSIConverter::toJSI(runtime, "disparity-32-bit"); + case margelo::nitro::camera::PixelFormat::PRIVATE: + return JSIConverter::toJSI(runtime, "private"); + default: + [[unlikely]] throw std::invalid_argument("Cannot convert PixelFormat to JS - invalid value: " + std::to_string(static_cast(arg)) + "!"); } - static inline bool canConvert(jsi::Runtime& runtime, const jsi::Value& value) { - if (!value.isString()) { + } + static inline bool canConvert(jsi::Runtime& runtime, const jsi::Value& value) { + if (!value.isString()) { + return false; + } + std::string unionValue = JSIConverter::fromJSI(runtime, value); + switch (hashString(unionValue.c_str(), unionValue.size())) { + case hashString("unknown"): + case hashString("yuv-420-8-bit-video"): + case hashString("yuv-420-8-bit-full"): + case hashString("yuv-420-10-bit-video"): + case hashString("yuv-420-10-bit-full"): + case hashString("yuv-422-8-bit-video"): + case hashString("yuv-422-8-bit-full"): + case hashString("yuv-422-10-bit-video"): + case hashString("yuv-422-10-bit-full"): + case hashString("yuv-444-8-bit-video"): + case hashString("yuv-444-8-bit-full"): + case hashString("yuv-444-10-bit-video"): + case hashString("yuv-444-10-bit-full"): + case hashString("rgb-bgra-8-bit"): + case hashString("rgb-rgba-8-bit"): + case hashString("rgb-rgb-8-bit"): + case hashString("raw-bayer-packed96-12-bit"): + case hashString("raw-bayer-unpacked-16-bit"): + case hashString("depth-16-bit"): + case hashString("depth-32-bit"): + case hashString("depth-point-cloud-32-bit"): + case hashString("disparity-16-bit"): + case hashString("disparity-32-bit"): + case hashString("private"): + return true; + default: return false; - } - std::string unionValue = JSIConverter::fromJSI(runtime, value); - switch (hashString(unionValue.c_str(), unionValue.size())) { - case hashString("unknown"): - case hashString("yuv-420-8-bit-video"): - case hashString("yuv-420-8-bit-full"): - case hashString("yuv-420-10-bit-video"): - case hashString("yuv-420-10-bit-full"): - case hashString("yuv-422-8-bit-video"): - case hashString("yuv-422-8-bit-full"): - case hashString("yuv-422-10-bit-video"): - case hashString("yuv-422-10-bit-full"): - case hashString("yuv-444-8-bit-video"): - case hashString("yuv-444-8-bit-full"): - case hashString("rgb-bgra-8-bit"): - case hashString("rgb-rgba-8-bit"): - case hashString("rgb-rgb-8-bit"): - case hashString("raw-bayer-packed96-12-bit"): - case hashString("raw-bayer-unpacked-16-bit"): - case hashString("depth-16-bit"): - case hashString("depth-32-bit"): - case hashString("depth-point-cloud-32-bit"): - case hashString("disparity-16-bit"): - case hashString("disparity-32-bit"): - case hashString("private"): - return true; - default: - return false; - } } - }; + } +}; } // namespace margelo::nitro diff --git a/packages/react-native-vision-camera/src/hooks/useFrameOutput.ts b/packages/react-native-vision-camera/src/hooks/useFrameOutput.ts index 18e7f6be93..c694e4325c 100644 --- a/packages/react-native-vision-camera/src/hooks/useFrameOutput.ts +++ b/packages/react-native-vision-camera/src/hooks/useFrameOutput.ts @@ -77,10 +77,10 @@ export interface UseFrameOutputProps extends Partial { * @discussion * Choosing an appropriate {@linkcode FrameOutputOptions.pixelFormat | pixelFormat} * depends on your Frame Processor's usage. While the most commonly used - * format in visual recognition models is {@linkcode VideoPixelFormat | 'rgb'}, + * target in visual recognition models is {@linkcode TargetVideoPixelFormat | 'rgb'}, * it is by far not the most efficient format for a Camera pipeline as it * requires an additional conversion and uses ~2.6x more bandwidth than - * {@linkcode VideoPixelFormat | 'yuv'}. + * {@linkcode TargetVideoPixelFormat | 'yuv'}. * * If you render to native Surfaces (e.g. via GPU pipelines or Media Encoders), * you may also be able to use {@linkcode TargetVideoPixelFormat | 'native'}, diff --git a/packages/react-native-vision-camera/src/specs/common-types/DepthPixelFormat.ts b/packages/react-native-vision-camera/src/specs/common-types/DepthPixelFormat.ts index 6b57a9c322..6b96884d1a 100644 --- a/packages/react-native-vision-camera/src/specs/common-types/DepthPixelFormat.ts +++ b/packages/react-native-vision-camera/src/specs/common-types/DepthPixelFormat.ts @@ -6,14 +6,19 @@ * formats come from multiple constituent physical cameras * and represents pixel shift between two or more cameras. * - * | Format | Domain | Sample | Bit depth | Layout | - * |----------------------------|------------|----------------------------------|---------------|--------------| - * | `depth-16-bit` | Depth | `[depth]` | 16-bit Float | Planar | - * | `depth-32-bit` | Depth | `[depth]` | 32-bit Float | Planar | - * | `depth-point-cloud-32-bit` | PointCloud | `[x, y, z, confidence]` | 32-bit Float | Interleaved | - * | `disparity-16-bit` | Disparity | `[disparity]` | 16-bit Float | Planar | - * | `disparity-32-bit` | Disparity | `[disparity]` | 32-bit Float | Planar | - * | `unknown` | — | — | — | — | + * | Format | Domain | Sample | Storage | Layout | + * |----------------------------|------------|----------------------------------|---------------------|--------------| + * | `depth-16-bit` | Depth | `[depth]` | Platform 16-bit | Single-plane | + * | `depth-32-bit` | Depth | `[depth]` | 32-bit Float | Single-plane | + * | `depth-point-cloud-32-bit` | PointCloud | `[x, y, z, confidence]` | 32-bit Float | Interleaved | + * | `disparity-16-bit` | Disparity | `[disparity]` | 16-bit Float | Single-plane | + * | `disparity-32-bit` | Disparity | `[disparity]` | 32-bit Float | Single-plane | + * | `unknown` | — | — | — | — | + * + * @note On iOS, `depth-16-bit` maps to `kCVPixelFormatType_DepthFloat16` + * and stores one 16-bit float depth sample in meters. On Android, + * `depth-16-bit` maps to `ImageFormat.DEPTH16`, where each 16-bit sample + * stores 13 bits of range in millimeters plus 3 bits of confidence. */ export type DepthPixelFormat = | 'depth-16-bit' diff --git a/packages/react-native-vision-camera/src/specs/common-types/VideoPixelFormat.ts b/packages/react-native-vision-camera/src/specs/common-types/VideoPixelFormat.ts index e358a61d78..1e3516a28c 100644 --- a/packages/react-native-vision-camera/src/specs/common-types/VideoPixelFormat.ts +++ b/packages/react-native-vision-camera/src/specs/common-types/VideoPixelFormat.ts @@ -4,21 +4,23 @@ import type { PixelFormat } from './PixelFormat' /** * Represents the pixel format of a video image buffer. - * Video Formats are either **YUV** or **RGB**. + * Video Formats are **YUV**, **RGB**, or RAW/Bayer sensor data. * The most commonly used format is `yuv-420-8-bit-video`. * * | Format | Colorspace | Sampling | Bit depth | Color Range | Layout | * |-----------------------------|------------|----------|-----------|--------------|---------------------| - * | `yuv-420-8-bit-video` | YUV | 4:2:0 | 8-bit | Video | Planar | - * | `yuv-420-8-bit-full` | YUV | 4:2:0 | 8-bit | Full | Planar | - * | `yuv-420-10-bit-video` | YUV | 4:2:0 | 10-bit | Video | Planar | - * | `yuv-420-10-bit-full` | YUV | 4:2:0 | 10-bit | Full | Planar | - * | `yuv-422-8-bit-video` | YUV | 4:2:2 | 8-bit | Video | Planar | - * | `yuv-422-8-bit-full` | YUV | 4:2:2 | 8-bit | Full | Planar | - * | `yuv-422-10-bit-video` | YUV | 4:2:2 | 10-bit | Video | Planar | - * | `yuv-422-10-bit-full` | YUV | 4:2:2 | 10-bit | Full | Planar | - * | `yuv-444-8-bit-video` | YUV | 4:4:4 | 8-bit | Video | Planar | - * | `yuv-444-8-bit-full` | YUV | 4:4:4 | 8-bit | Full | Planar | + * | `yuv-420-8-bit-video` | YUV | 4:2:0 | 8-bit | Video | Multi-planar | + * | `yuv-420-8-bit-full` | YUV | 4:2:0 | 8-bit | Full | Multi-planar | + * | `yuv-420-10-bit-video` | YUV | 4:2:0 | 10-bit | Video | Multi-planar | + * | `yuv-420-10-bit-full` | YUV | 4:2:0 | 10-bit | Full | Multi-planar | + * | `yuv-422-8-bit-video` | YUV | 4:2:2 | 8-bit | Video | Multi-planar | + * | `yuv-422-8-bit-full` | YUV | 4:2:2 | 8-bit | Full | Multi-planar | + * | `yuv-422-10-bit-video` | YUV | 4:2:2 | 10-bit | Video | Multi-planar | + * | `yuv-422-10-bit-full` | YUV | 4:2:2 | 10-bit | Full | Multi-planar | + * | `yuv-444-8-bit-video` | YUV | 4:4:4 | 8-bit | Video | Multi-planar | + * | `yuv-444-8-bit-full` | YUV | 4:4:4 | 8-bit | Full | Multi-planar | + * | `yuv-444-10-bit-video` | YUV | 4:4:4 | 10-bit | Video | Multi-planar | + * | `yuv-444-10-bit-full` | YUV | 4:4:4 | 10-bit | Full | Multi-planar | * | `rgb-bgra-8-bit` | RGB | 4:4:4 | 8-bit | Full | Interleaved BGRA | * | `rgb-rgba-8-bit` | RGB | 4:4:4 | 8-bit | Full | Interleaved RGBA | * | `rgb-rgb-8-bit` | RGB | 4:4:4 | 8-bit | Full | Interleaved RGB | @@ -37,6 +39,8 @@ export type VideoPixelFormat = | 'yuv-422-10-bit-full' | 'yuv-444-8-bit-video' | 'yuv-444-8-bit-full' + | 'yuv-444-10-bit-video' + | 'yuv-444-10-bit-full' | 'rgb-bgra-8-bit' | 'rgb-rgba-8-bit' | 'rgb-rgb-8-bit' @@ -56,8 +60,10 @@ export type VideoPixelFormat = * an RGB format like {@linkcode VideoPixelFormat | 'rgb-bgra-8-bit'}, a RAW format like * {@linkcode VideoPixelFormat | 'raw-bayer-packed96-12-bit'}, or a private * format ({@linkcode PixelFormat | 'private'}) and requires zero conversion. - * - `'yuv'`: Choose the YUV format closest to the Camera's native format. Often YUV 4:2:0 - * 8-bit full-range like {@linkcode VideoPixelFormat | 'yuv-420-8-bit-full'}. + * - `'yuv'`: Choose a CPU-readable YUV format. This is currently YUV 4:2:0 + * 8-bit, usually reported as full-range + * ({@linkcode VideoPixelFormat | 'yuv-420-8-bit-full'}) unless the platform + * explicitly reports video-range metadata. * - `'rgb'`: Choose an RGB format. Often 8-bit BGRA like * {@linkcode VideoPixelFormat | 'rgb-bgra-8-bit'}. */ diff --git a/packages/react-native-vision-camera/src/specs/instances/Frame.nitro.ts b/packages/react-native-vision-camera/src/specs/instances/Frame.nitro.ts index dae69e0c08..d8b75fc630 100644 --- a/packages/react-native-vision-camera/src/specs/instances/Frame.nitro.ts +++ b/packages/react-native-vision-camera/src/specs/instances/Frame.nitro.ts @@ -151,7 +151,8 @@ export interface Frame /** * Gets the {@linkcode PixelFormat} of this Frame's pixel data. * Common formats are {@linkcode PixelFormat | 'yuv-420-8-bit-video'} - * for native YUV Frames, {@linkcode PixelFormat | 'rgb-bgra-32-bit'} + * or {@linkcode PixelFormat | 'yuv-420-8-bit-full'} + * for YUV Frames, {@linkcode PixelFormat | 'rgb-bgra-8-bit'} * for processed RGB Frames, or {@linkcode PixelFormat | 'private'} for * zero-copy GPU-only Frames. * @@ -163,7 +164,7 @@ export interface Frame * * @discussion * If the {@linkcode Frame} is a GPU-only buffer, its {@linkcode pixelFormat} - * is {@linkcode PixelFormat | 'private'}, wich allows zero-copy importing it + * is {@linkcode PixelFormat | 'private'}, which allows zero-copy importing it * into GPU pipelines directly, however its pixel data is likely not accessible * on the CPU. * You can use {@linkcode getNativeBuffer | getNativeBuffer()} to get a handle diff --git a/packages/react-native-vision-camera/src/specs/outputs/CameraFrameOutput.nitro.ts b/packages/react-native-vision-camera/src/specs/outputs/CameraFrameOutput.nitro.ts index 35cce51878..73268203a8 100644 --- a/packages/react-native-vision-camera/src/specs/outputs/CameraFrameOutput.nitro.ts +++ b/packages/react-native-vision-camera/src/specs/outputs/CameraFrameOutput.nitro.ts @@ -2,8 +2,12 @@ import type { Sync } from 'react-native-nitro-modules' import type { useFrameOutput } from '../../hooks/useFrameOutput' import type { FrameDroppedReason } from '../common-types/FrameDroppedReason' import type { NativeBuffer } from '../common-types/NativeBuffer' +import type { PixelFormat } from '../common-types/PixelFormat' import type { Size } from '../common-types/Size' -import type { TargetVideoPixelFormat } from '../common-types/VideoPixelFormat' +import type { + TargetVideoPixelFormat, + VideoPixelFormat, +} from '../common-types/VideoPixelFormat' import type { NativeThread } from '../frame-processors/NativeThread.nitro' import type { Frame } from '../instances/Frame.nitro' import type { CameraSession } from '../session/CameraSession.nitro' @@ -73,17 +77,19 @@ export interface FrameOutputOptions { * {@linkcode CameraSessionConfig.nativePixelFormat | nativePixelFormat}. * - Some configurations may natively stream in a * YUV format (e.g. if {@linkcode CameraSessionConfig.nativePixelFormat | nativePixelFormat} == - * {@linkcode TargetVideoPixelFormat | 'yuv-420-8-bit-video'}), + * {@linkcode VideoPixelFormat | 'yuv-420-8-bit-video'}), * in which case {@linkcode TargetVideoPixelFormat | 'yuv'} can also be zero overhead. * - If your Frame Processor absolutely requires to run in RGB, you may * set {@linkcode pixelFormat} to {@linkcode TargetVideoPixelFormat | 'rgb'}, * which comes with additional processing overhead as the Camera pipeline * will convert native frames to RGB (e.g. to - * {@linkcode TargetVideoPixelFormat | 'rgb-bgra-8-bit'}). + * {@linkcode VideoPixelFormat | 'rgb-bgra-8-bit'}). * * @discussion * It is recommended to use {@linkcode TargetVideoPixelFormat | 'native'} - * if possible, as this will use a zero-copy GPU-only path. + * if possible, as this avoids explicit pixel-format conversion. On Android, + * this uses a GPU-only {@linkcode PixelFormat | 'private'} path; on iOS, it + * asks AVFoundation for the device-native CoreVideo format. * Other formats almost always require conversion at * some point, especially on Android. *