​Open-Source Practices for Music Signal Processing Research: Recommendations for Transparent, Sustainable, and Reproducible Audio Research

Writing quality open-source scientific software is crucial for comparable, reproducible research. Many of us learned this the hard way over many years of work (and lines of code). So we decided to write the article we wish we could have read at the beginning of our research careers. We hope you find it useful!

Open-source Practices for Music Signal Processing Research: Recommendations for Transparent, Sustainable, and Reproducible Audio Research
B. McFee, J. W. Kim, M. Cartwright, J. Salamon, R. M. Bittner, and J. P. Bello. 
IEEE Signal Processing Magazine, 36(1):128-137, Jan. 2019.
[IEEE][PDF][BibTeX][Copyright]

But... is this really such a big deal? Do you really need an open source implementation of your research? Can't other researchers just work off the description in your paper? In the plot below (reproduced from the article) we show the performance of 8 different onset detection systems that were all implemented based on the same description (in the caption) with "minor" changes to the implementation:

Performance of 8 systems implemented based on the following description with "minor" implementation differences: "Audio was converted to 22050 Hz (mono), and a 2048-point short-time Fourier transform (STFT) was computed with a 64-sample hop. The STFT was reduced to 128 Mel-frequency bands, and magnitudes were compressed by log scaling. An onset envelope was computed using thresholded spectral differencing, and peaks were selected using the method of Bock et al. [32]."

Ouch!

In the full article we share experiences and advice gained from developing open source software for MIR research, with the hope that practitioners in other related disciplines may benefit from our findings and become effective developers of open source scientific software.

Many of the issues we encounter in MIR applications are likely to recur in more general signal processing areas, as data sets increase in complexity, evaluation becomes more integrated and realistic, and traditionally small research components become integrated with larger systems.