← All Projects
Virtual Piano — Hand-Tracking Instrument
Camera-based virtual piano that maps hand keypoints detected by Mediapipe to on-screen piano keys, letting users play music without touching a keyboard.
PythonOpenCV (video capture and rendering)Mediapipe Hand module (21-keypoint hand detection)Audio output for key playback
⚡ Debouncing key triggers so a sustained hover doesn't fire re…⚡ Handling hand occlusion and multi-finger tracking at the sam…⚡ Calibrating key zone hit detection to account for camera ang…
Portfolio Highlights
- →Built a real-time virtual piano using OpenCV and Mediapipe hand-keypoint detection, mapping 21-point hand landmarks to piano keys without physical hardware.
Snapshot
- Period: September 2024
- Source: `github.com/josegibson/Virtual-piano`
- Domain: Computer vision, human-computer interaction
- Status: Prototype with demo video
Stack
- Python
- OpenCV (video capture and rendering)
- Mediapipe Hand module (21-keypoint hand detection)
- Audio output for key playback
What I Built
- Real-time hand keypoint detection pipeline using Mediapipe's Hand module.
- Mapping from detected finger tip positions to on-screen piano key zones.
- Visual overlay rendering piano keys on the camera feed with highlight on "press."
- Audio playback triggered by finger contact with key zones.
- Demo video recorded on physical hardware.