Central to the field of MIR research is the evaluation of algorithms used to extract information from music data. We present mir_eval, an open source software library which provides a transparent and easy-to-use implementation of the most common metrics used to measure the performance of MIR algorithms. In this paper, we enumerate the metrics implemented by mir_eval and quantitatively compare each to existing implementations. When the scores reported by mir_eval differ substantially from the reference, we detail the differences in implementation. We also provide a brief overview of mir_eval’s architecture, design, and intended use.
A massive congratulations to comrades Colin, Brian, Eric, Oriol Dawen and Dan for creating this awesome project, and in particular to Colin for leading this initiative and doing a fantastic job at presenting it at ISMIR today!
You can check out mir_eval here: https://github.com/craffel/mir_eval