Skip to content

Add MediaPipe gesture-based multirotor control example#9772

Closed
sushantkumarkhobian-lab wants to merge 1 commit into
microsoft:mainfrom
sushantkumarkhobian-lab:sushantkumarkhobian-lab-patch-1
Closed

Add MediaPipe gesture-based multirotor control example#9772
sushantkumarkhobian-lab wants to merge 1 commit into
microsoft:mainfrom
sushantkumarkhobian-lab:sushantkumarkhobian-lab-patch-1

Conversation

@sushantkumarkhobian-lab

Copy link
Copy Markdown

Description

This PR adds a new Python example demonstrating gesture-based multirotor control using AirSim, MediaPipe, and OpenCV.

The example uses webcam-based hand tracking to recognize hand gestures and map them to drone commands in AirSim.

Features

  • Real-time hand tracking using MediaPipe
  • Webcam input with OpenCV
  • Gesture-based drone control
  • Hover command using a closed fist
  • Ascend and descend controls
  • Forward movement control
  • Landing command through gesture recognition

Purpose

This example demonstrates how computer vision can be integrated with AirSim to create intuitive human-drone interaction workflows. It serves as an educational example for users interested in combining AirSim with hand-tracking and gesture-recognition systems.

Dependencies

  • AirSim Python API
  • OpenCV (opencv-python)
  • MediaPipe (mediapipe)

This script implements gesture-based control for a drone using AirSim, MediaPipe, and OpenCV. It recognizes various hand gestures to control the drone's movements such as ascending, descending, and landing.
@sushantkumarkhobian-lab

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

@jonyMarino

Copy link
Copy Markdown
Collaborator

@sushantkumarkhobian-lab Super interesting, thanks for your contribution! Why didn't you use the latest version of AirSim: https://github.com/iamaisim/ProjectAirsim?

@sushantkumarkhobian-lab

Copy link
Copy Markdown
Author

Hi @jonyMarino,
Thank you for the positive feedback!
You are completely right. I realize now that I submitted this Pull Request against the archived repository by mistake. I would love to bring this feature over to the latest version of the platform.
I will close this PR and open a new one over at https://github.com/iamaisim/ProjectAirsim shortly. Thanks for pointing me in the right direction!
Best regards,
Sushant

@sushantkumarkhobian-lab

Copy link
Copy Markdown
Author

Hi @jonyMarino,
Thanks again for the guidance! I've closed this PR since it was targeting the archived repository.
I have successfully opened the new Pull Request with the updated code directly on the active repository here: iamaisim/ProjectAirSim#161
Looking forward to your feedback over there!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants