✋ Gesture-Controlled Virtual Drawing Board (OpenCV + MediaPipe)

A real-time gesture-controlled virtual drawing board built using OpenCV and MediaPipe Hands, where users can draw on the screen using hand gestures — no mouse, no touchscreen, just a webcam.

This project is developed incrementally, focusing on understanding the math, logic, and concepts behind hand tracking and gesture-based interaction.

🚀 Project Overview

The system uses:

Index fingertip as a virtual pen

Pinch gesture (index + thumb) as pen down / pen up

A persistent canvas to store drawings

Real-time geometric distance calculations to control drawing behavior

The result is an interactive virtual whiteboard that responds naturally to hand movements.

🎯 Features Implemented

Real-time webcam feed using OpenCV

Accurate hand detection using MediaPipe Hands

Extraction of individual hand landmarks

Conversion of normalized landmark coordinates to pixel space

Pinch gesture detection using Euclidean distance

Persistent virtual drawing canvas

Continuous line drawing using frame-to-frame tracking

Live debugging visuals (finger point & pinch distance)

🧠 Core Concepts Learned

Computer Vision fundamentals

Coordinate systems (normalized vs pixel coordinates)

Euclidean distance geometry

Gesture state detection (binary logic from continuous data)

Temporal tracking (previous vs current frame)

Canvas-based rendering

Human–Computer Interaction (HCI) basics

🗺️ Project Progress Roadmap
✅ Stage 1: OpenCV Basics

Webcam access and frame capture

Live video display

Safe program exit

✅ Stage 2: MediaPipe Hand Detection

Hand landmark detection

Real-time visualization

Stable tracking

✅ Stage 3: Landmark Extraction & Distance Math

Access landmark coordinates

Understand normalized coordinates

Convert to pixel coordinates

Apply Euclidean distance formula

✅ Stage 4: Hand Movement Detection

Track movement across frames

Detect directional motion

Apply thresholds to reduce noise

✅ Stage 5: Pinch Gesture Detection

Detect index–thumb pinch

Convert distance into gesture state (ON/OFF)

✅ Stage 6: Virtual Drawing Canvas

Create persistent canvas using NumPy

Draw continuous strokes using fingertip motion

Control drawing using pinch gesture

Blend drawing canvas with live video

🛠️ Tech Stack

Python

OpenCV

MediaPipe

NumPy

Math (Geometry)

📐 How It Works (High Level)

Capture webcam frames

Detect hand landmarks using MediaPipe

Convert fingertip coordinates to pixel space

Measure distance between index finger and thumb

If distance < threshold → pen down

Draw a line between previous and current fingertip positions

Store drawing on a persistent canvas

Overlay canvas on live video feed

▶️ How to Run
Prerequisites
pip install opencv-python mediapipe numpy

Run the program
python main.py

Controls

✍️ Pinch index + thumb → draw

✋ Release pinch → stop drawing

❌ Press Q → quit

📂 Project Structure (Current)
.
├── main.py        # Camera loop, hand tracking, drawing logic
├── README.md      # Project documentation

🔜 Future Improvements

Stroke smoothing (reduce shaky lines)

Eraser gesture

Clear canvas gesture

Brush size control

Color selection

Save drawing as image

Modular code structure

💡 Why This Project Matters

This is not just an OpenCV demo.

It demonstrates:

Real-time computer vision

Gesture-based interaction design

Mathematical reasoning applied to visuals

Foundations of AR and HCI systems

📌 Author

Agnik Mondal
Aspiring Python Developer | Computer Vision Enthusiast