diff --git a/Assets/Images/Pipeline.jpeg b/Assets/Images/Pipeline.jpeg new file mode 100644 index 0000000..b345c80 Binary files /dev/null and b/Assets/Images/Pipeline.jpeg differ diff --git a/Assets/Images/Pipeline_1.jpeg b/Assets/Images/Pipeline_1.jpeg new file mode 100644 index 0000000..27bc890 Binary files /dev/null and b/Assets/Images/Pipeline_1.jpeg differ diff --git a/README.md b/README.md index 6f5e4b4..1de449c 100644 --- a/README.md +++ b/README.md @@ -2,8 +2,10 @@ ## Table of contents +- [Introduction](#Introduction) - [Aim](#Aim) - [About the Project](#About-The-Project) +- [File Structure](#File-Structure) - [Getting Started](#Getting-Started) - [Prerequisites and Installations](#Prerequisites-And-Installations) - [Methodologies Proposed](#Methodologies-Proposed) @@ -13,6 +15,16 @@ - [Results](#Results) - [License](#License) +## Introduction + + +OpenCV AI Competition is an international open source competition on computer vision by OpenCV Foundation. +2022 edition of the competition has two exciting tracks - Core OpenCV & Spatial AI. + +Spatial AI track aims to develop a high-quality depth estimation project using OpenCV AI Kit with Depth Pro (OAK-D-Pro) Spatial AI Camera. This camera was designed and developed by Luxonis. + +[OpenCV AI Competition 2022](https://opencv.org/opencv-ai-competition-2022/) + ## Aim @@ -26,12 +38,50 @@ Depth Estimation algorithms using Neural Networks have made enormous strides rec |-----|-----| |![RGB](Assets/Results/Result_Image_2/FrameRGB_3.jpg)|![Depth](Assets/Results/Result_Image_2/FusedRGB_3.jpg)| + +## File Structure +``` +📦Spatial-AI-main +├── 📂Assets +│   ├── 📂Images +│   ├── 📜README.md +│   ├── 📂Results +│   │   ├── 📂Result_Image_1 +│   │   ├── 📂Result_Image_2 +│   │   ├── 📂Result_Image_3 +│   │   ├── 📂Result_Image_4 +│   │   ├── 📂Result_Video_1 +│   │   └── 📂Result_Video_2 +│   └── 📂Videos +├── 📂Fusion +│   ├── 📜Fusion_Canny.py +│   ├── 📜Fusion_Laptop.py +│   ├── 📜Fusion_OakD.py +│   ├── 📂MDEMap +│   ├── 📜README.md +│   ├── 📂RGBMap +│   └── 📂StereoMap +├── 📜main.py +├── 📜Midas.py +├── 📜Processing.py +├── 📜README.md +├── 📜requirements.txt +└── 📂scripts + ├── 📂Blob + ├── 📜blob_generator.py + ├── 📜MDE.py + ├── 📂Onnx + ├── 📜onnx_generator.py + ├── 📜Pre_Processing.py + ├── 📜README.md + └── 📂Weights +``` ## Getting Started ## Prerequisites And Installations -1. Install Depthai and many other important librariesutilized in this project by running following command : +1. Install Depthai and many other important libraries utilized in this project by running following command : ``` pip install -r requirements.txt ``` @@ -44,6 +94,13 @@ Depth Estimation algorithms using Neural Networks have made enormous strides rec ### Part 1 : Implementing Pre + Post Processing While going through various research paper we found that performing pre processing on stereo left and stereo right image and then implementing stereo rectification as well as triangulation method increases depth perception of camera manifolds. As well as reduce its noise.Preprocessing on images with certain touch on it with inbuilt OAK D post processing filters has improved its depth a lot. + + +|Implemented Pipeline:| +|-----------| +|![OAK-D Pipeline_1](Assets/Images/Pipeline_1.jpeg)| + + For Example , |Stereo Map Generated By OAK-D|Stereo Map After Pre+Post Processing| |--------|---------| @@ -79,6 +136,12 @@ MiDaS was trained on 10 datasets (ReDWeb, DIML, Movies, MegaDepth, WSVD, TartanA In this approach we fuse disparity map generated by OAK-D using stereo cameras and disparity map generated using MiDAS model (MDE) on rgb video. This method aims to combine excellent features of Stereo as well as Monocular Depth Estimations and reduce noise generated by one disparity map by superimposing quality of other disparity map. For further information kindly check folder [Fusion](Fusion) + +|Implemented Pipeline:| +|-----------| +|![OAK-D Pipeline](Assets/Images/Pipeline.jpeg)| + + | Original Scene | Steeo Disparity By OAK-D | MiDAS Disparity | Fused Disparity Map | |--------|--------|--------|--------| |![Original-Scene](Assets/Images/OrgRGB.png)|![Stereo-Disparity](Assets/Images/StereoImg.png)|![MiDAS-Disparity](Assets/Images/Midasimage.png)|![Fused-Disparity](Assets/Images/FusedImg.png)|