Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added Assets/Images/Pipeline.jpeg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Assets/Images/Pipeline_1.jpeg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
65 changes: 64 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,10 @@


## Table of contents
- [Introduction](#Introduction)
- [Aim](#Aim)
- [About the Project](#About-The-Project)
- [File Structure](#File-Structure)
- [Getting Started](#Getting-Started)
- [Prerequisites and Installations](#Prerequisites-And-Installations)
- [Methodologies Proposed](#Methodologies-Proposed)
Expand All @@ -13,6 +15,16 @@
- [Results](#Results)
- [License](#License)

## Introduction


OpenCV AI Competition is an international open source competition on computer vision by OpenCV Foundation.
2022 edition of the competition has two exciting tracks - Core OpenCV & Spatial AI.

Spatial AI track aims to develop a high-quality depth estimation project using OpenCV AI Kit with Depth Pro (OAK-D-Pro) Spatial AI Camera. This camera was designed and developed by Luxonis.

[OpenCV AI Competition 2022](https://opencv.org/opencv-ai-competition-2022/)

## Aim


Expand All @@ -26,12 +38,50 @@ Depth Estimation algorithms using Neural Networks have made enormous strides rec
|-----|-----|
|![RGB](Assets/Results/Result_Image_2/FrameRGB_3.jpg)|![Depth](Assets/Results/Result_Image_2/FusedRGB_3.jpg)|


## File Structure
```
📦Spatial-AI-main
├── 📂Assets
│   ├── 📂Images
│   ├── 📜README.md
│   ├── 📂Results
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

│   │   ├── 📂Result_Image_1
│   │   ├── 📂Result_Image_2
│   │   ├── 📂Result_Image_3
│   │   ├── 📂Result_Image_4
│   │   ├── 📂Result_Video_1
│   │   └── 📂Result_Video_2
│   └── 📂Videos
├── 📂Fusion
│   ├── 📜Fusion_Canny.py
│   ├── 📜Fusion_Laptop.py
│   ├── 📜Fusion_OakD.py
│   ├── 📂MDEMap
│   ├── 📜README.md
│   ├── 📂RGBMap
│   └── 📂StereoMap
├── 📜main.py
├── 📜Midas.py
├── 📜Processing.py
├── 📜README.md
├── 📜requirements.txt
└── 📂scripts
├── 📂Blob
├── 📜blob_generator.py
├── 📜MDE.py
├── 📂Onnx
├── 📜onnx_generator.py
├── 📜Pre_Processing.py
├── 📜README.md
└── 📂Weights
```
## Getting Started


## Prerequisites And Installations

1. Install Depthai and many other important librariesutilized in this project by running following command :
1. Install Depthai and many other important libraries utilized in this project by running following command :
```
pip install -r requirements.txt
```
Expand All @@ -44,6 +94,13 @@ Depth Estimation algorithms using Neural Networks have made enormous strides rec
### Part 1 : Implementing Pre + Post Processing

While going through various research paper we found that performing pre processing on stereo left and stereo right image and then implementing stereo rectification as well as triangulation method increases depth perception of camera manifolds. As well as reduce its noise.Preprocessing on images with certain touch on it with inbuilt OAK D post processing filters has improved its depth a lot.


|Implemented Pipeline:|
|-----------|
|![OAK-D Pipeline_1](Assets/Images/Pipeline_1.jpeg)|


For Example ,
|Stereo Map Generated By OAK-D|Stereo Map After Pre+Post Processing|
|--------|---------|
Expand Down Expand Up @@ -79,6 +136,12 @@ MiDaS was trained on 10 datasets (ReDWeb, DIML, Movies, MegaDepth, WSVD, TartanA
In this approach we fuse disparity map generated by OAK-D using stereo cameras and disparity map generated using MiDAS model (MDE) on rgb video. This method aims to combine excellent features of Stereo as well as Monocular Depth Estimations and reduce noise generated by one disparity map by superimposing quality of other disparity map.
For further information kindly check folder [Fusion](Fusion)


|Implemented Pipeline:|
|-----------|
|![OAK-D Pipeline](Assets/Images/Pipeline.jpeg)|


| Original Scene | Steeo Disparity By OAK-D | MiDAS Disparity | Fused Disparity Map |
|--------|--------|--------|--------|
|![Original-Scene](Assets/Images/OrgRGB.png)|![Stereo-Disparity](Assets/Images/StereoImg.png)|![MiDAS-Disparity](Assets/Images/Midasimage.png)|![Fused-Disparity](Assets/Images/FusedImg.png)|
Expand Down