Microphone Rhythm Game

Above is the gameplay of the song Sound of Silence. The layout is similar to tab notation, where each note has a number indicating the tab to hold down, and the column indicates the string.

At the time, I thought the notes rising, rather than falling, would make it easier to read, because in English (and most written languages) we read from top to bottom. But, in retrospect, the notes moving sideways would match the usual direction a guitar is held, and would make it easier to see which note corresponds to which string.

How Does it Work?

In a separate thread, the microphone takes in data at a rate of (usually greater than or equal to) 44100 samples per second. Every 2048 samples, a fast Fourier transform is taken over the most recent 8192 samples (saved in a ring buffer), and the resulting spectrogram is used to calculate if there were any note hits.

Above is the spectrogram in red, and the note "score" function graphed in real-time, which takes into account the second and third harmonic.

This score function, and all of these variables, are in a sense arbitrary. But, they were tuned for the best experience I could get.

There's a good amount of error in my system, and perhaps better spectral analysis tools (and maybe some machine learning techniques) could get better accuracy for measuring note-hits. But, for what it was, it got the job done.