Hi @jmplaza — following the Block Library Extension (#69), I'd like to contribute a new
Computer Vision block and wanted to confirm the scope before packaging it.
Block: Hand & Gesture Detector (MediaPipe Hand Landmarker)
VisualCircuit has FaceDetector (Haar) and ObjectDetector (YOLO) but no hand / landmark /
gesture block. This fills that gap and pairs naturally with the Controller blocks
(MotorDriver, Teleoperator) for gesture-driven robotics.
What it does: runs MediaPipe's Hand Landmarker on the input image to detect up to two
hands (21 landmarks each) and estimate a simple gesture (extended-finger count).
Inputs: BGR image (Img)
Outputs (via OutputType parameter):
image — frame with hand landmarks, connections and a gesture label drawn on itlandmarks — flat array of 21 (x, y) normalised landmarks + finger count, so downstream
blocks can react to gestures
Parameter: OutputType ('image' / 'landmarks')
Implementation notes:
- Uses MediaPipe's current Tasks API (
mediapipe.tasks.vision.HandLandmarker) — the
legacy mp.solutions.hands API was removed in MediaPipe 0.10.30+, so the Tasks API is the
durable choice.
- Ships a bundled
hand_landmarker.task model under utils/models/, mirroring how
FaceDetector/ObjectDetector bundle their Haar/YOLO models. One new dependency: mediapipe.
- Mirrors the existing block API (
inputs.read_image / outputs.share_image /
outputs.share_array / synchronise).
I've prototyped and tested it (detects both hands and draws the 21-point skeleton on a real
frame). Once you're happy with the scope, I'll build it in the editor, export the .vc3, and
open a PR on VisualCircuit-resources with a short demo video per the custom-block guide.
Does this fit the library? Happy to adjust the I/O or naming.
Hi @jmplaza — following the Block Library Extension (#69), I'd like to contribute a new
Computer Vision block and wanted to confirm the scope before packaging it.
Block: Hand & Gesture Detector (MediaPipe Hand Landmarker)
VisualCircuit has FaceDetector (Haar) and ObjectDetector (YOLO) but no hand / landmark /
gesture block. This fills that gap and pairs naturally with the Controller blocks
(MotorDriver, Teleoperator) for gesture-driven robotics.
What it does: runs MediaPipe's Hand Landmarker on the input image to detect up to two
hands (21 landmarks each) and estimate a simple gesture (extended-finger count).
Inputs: BGR image (
Img)Outputs (via
OutputTypeparameter):image— frame with hand landmarks, connections and a gesture label drawn on itlandmarks— flat array of 21 (x, y) normalised landmarks + finger count, so downstreamblocks can react to gestures
Parameter:
OutputType('image' / 'landmarks')Implementation notes:
mediapipe.tasks.vision.HandLandmarker) — thelegacy
mp.solutions.handsAPI was removed in MediaPipe 0.10.30+, so the Tasks API is thedurable choice.
hand_landmarker.taskmodel underutils/models/, mirroring howFaceDetector/ObjectDetector bundle their Haar/YOLO models. One new dependency:
mediapipe.inputs.read_image/outputs.share_image/outputs.share_array/synchronise).I've prototyped and tested it (detects both hands and draws the 21-point skeleton on a real
frame). Once you're happy with the scope, I'll build it in the editor, export the
.vc3, andopen a PR on VisualCircuit-resources with a short demo video per the custom-block guide.
Does this fit the library? Happy to adjust the I/O or naming.