This project is a web-based synthesizer that allows users to record and play back their own music in real time. The interface combines sound generation with visual interaction by displaying tones at the mouse position and animating a sine wave that reacts dynamically to frequency and amplitude.
The concept was inspired by mobile rhythm games and the theremin—one of the earliest electronic instruments, known for being played without physical contact. Our initial goal was to recreate a theremin-like experience using real-time hand tracking through the webcam, utilizing ml5.js and later Mediapipe. However, due to compatibility issues and tracking limitations, we pivoted to a simpler, more stable version controlled by mouse and keyboard.
The project demonstrates a creative approach to sound interaction and showcases the problem-solving process when working with experimental web technologies and real-time input.