Gesture Synth

Gesture Synth is a free browser instrument that turns hand movements seen by a webcam into music. Players use both hands to choose chords, shape harmony, set volume and tone, change octave or play continuous theremin-style melodies. It needs no account, download, microphone or MIDI controller, and suits people who want to make music through movement, from beginners practising simple songs to players exploring chord progressions.

After the player presses Start and allows camera access, a hand-tracking model running locally in the browser locates hand landmarks in each video frame. Those landmarks become control values for a Web Audio synthesizer that generates sound in real time, with smoothing to reduce jumps caused by tracking noise. Camera frames stay on the device and are never recorded or uploaded, and only preferences such as the selected key are stored locally.

In Gesture mode the left hand selects scale degrees I to VII by finger count, and wrist tilt can switch between major and minor. The right hand sets volume by height, picks triads, inversions or seventh chords by finger count, opens a low-pass filter by wrist tilt and drops an octave with the thumb. Theremin mode maps right-hand height to pitch and left-hand height to volume.

A four-track loop sequencer supports tempos from 30 to 300 BPM, tap tempo and time signatures such as 3/4, 5/4, 6/8 and 7/8, with local WAV export or a WebM fallback. Learn mode guides players through public-domain songs including Twinkle Twinkle Little Star, Ode to Joy and Amazing Grace. Recent desktop Chrome and Edge work best, Chrome on Android is supported, and Safari may use fallbacks.