The audio_to_midi_melodia python script allows you to extract the melody of a song and save it to a MIDI file. The script uses the Melodia algorithm to perform melody extraction, taking advantage of the new vamp module that allows running vamp plugins (like Melodia) directly in python.
Once the pitch contour of the melody is extracted, the next (non-trivial!) step is to segment it into notes and quantize the pitch of each note, producing a discrete series of notes that can then be exported into a any symbolic format such as MIDI or JAMS.
Quantizing a continuous pitch sequence into a series of notes is an active area of research and remains and open problem. Still, we can obtain fairly decent results using a series of heuristics:
1. Convert the pitch sequence from Hertz to (fractional) MIDI note numbers
2. Round each value to the nearest integer MIDI note number
3. Optionally apply a median filter to smooth out short jumps in pitch (e.g. due to vibrato)
4. Iterate over the sequence and whenever the pitch changes start a new note
Here's an example:
Once the pitch contour of the melody is extracted, the next (non-trivial!) step is to segment it into notes and quantize the pitch of each note, producing a discrete series of notes that can then be exported into a any symbolic format such as MIDI or JAMS.
Quantizing a continuous pitch sequence into a series of notes is an active area of research and remains and open problem. Still, we can obtain fairly decent results using a series of heuristics:
1. Convert the pitch sequence from Hertz to (fractional) MIDI note numbers
2. Round each value to the nearest integer MIDI note number
3. Optionally apply a median filter to smooth out short jumps in pitch (e.g. due to vibrato)
4. Iterate over the sequence and whenever the pitch changes start a new note
Here's an example:
Original audio
Continuous melody contour extracted by Melodia
Quantized MIDI melody obtained with audio_to_midi_melodia
Continuous melody contour extracted by Melodia
Quantized MIDI melody obtained with audio_to_midi_melodia
Note that exporting to MIDI requires providing a BPM value. You could select a value arbitrarily, estimate it manually, or estimate it automatically using one of the tempo estimation algorithms included in Essentia, Librosa, or if you'd like to stick to vamp plugins QMVP. No BPM is required if you export to JAMS, which directly uses the note onset times estimated from the audio track.
The script is open source and available on GitHub: https://github.com/justinsalamon/audio_to_midi_melodia
The script is open source and available on GitHub: https://github.com/justinsalamon/audio_to_midi_melodia
Once the melody is saved as MIDI it can be imported into any DAW that supports MIDI editing, like GarageBand.
RSS Feed
