Skip to content

feat: Implement Camera Undistortion and Perspective Warping #76

Description

@kalwalt

Description:

To move purecv toward a production-ready AR library, we need to implement the geometric pipeline that accounts for lens characteristics and image perspective.

Scope of Work:

  • remap: Implement the core function for generic image remapping.

  • initUndistortRectifyMap: Create a function to generate the map matrices required for remap. This is crucial for correcting radial and tangential lens distortion.

  • warpPerspective: Implement the transformation function to apply the homography matrix (obtained from findHomography) to an image, enabling planar surface rendering.

  • findFundamentalMat: Implement the algorithm to estimate the fundamental matrix, adding robust outlier rejection (RANSAC) to improve feature matching accuracy in non-planar scenes.

  • examples: Provide a sample demonstration (examples/rectification.rs) showcasing camera undistortion and perspective warping.

Performance & WASM SIMD Requirements

Functions like remap and warpPerspective are pixel-heavy operations. To ensure real-time execution (60fps) on mobile browsers, the implementation must adhere to the following architecture:

  • SIMD Acceleration: The core interpolation loops (e.g., bilinear interpolation) must be optimized using pulp SIMD to process multiple pixels simultaneously per CPU cycle. (Implemented generic SIMD dispatch for f32, f64, and u8 elements).

  • Scalar Fallback: A standard scalar implementation (one pixel at a time) must be provided as a fallback to prevent crashes on older devices or environments that lack native WebAssembly SIMD support.

Context:

Currently, solvePnP assumes a pinhole camera model without distortion. Adding remap and initUndistortRectifyMap will allow us to calibrate against real-world mobile device sensors, significantly reducing tracking "jitter" at the frame edges. warpPerspective provides the final step for mapping content onto detected planes.

Technical Note for Implementation

Since these functions involve significant pixel-looping, they are prime candidates for SIMD-accelerated interpolation (bilinear/nearest-neighbor).

For remap, consider defining a generic interpolation trait:

pub trait Interpolator {
    fn interpolate(&self, src: &Mat, x: f32, y: f32) -> Vec<u8>;
}

This allows you to implement BilinearInterpolator (which is standard for high-quality AR) efficiently using WASM SIMD to process multiple pixels per instruction.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestrustPull requests that update rust coderust-code

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions