Music Identification

Our group developed algorithms for identifying excerpts of music recordings by matching them against a possibly huge database of reference recordings, even if altered in pitch or tempo. Besides, we are able to identify piano pieces instantly by matching a few recorded keystrokes against a database of musical scores, and even find the exact location in the score (and follow along, see Music Alignment).

What does it do?

The algorithm reads an audio file, either pre-recorded or live, and searches for the recording or music piece in a given database of reference recordings or musical scores. If found, it can return any associated information from the database (e.g., the name of the music piece) and the position within the reference file at which the query was matched. It is robust against noise or minor mistakes in the query, as well as tempo changes and transposition. It is not, however, capable of matching a cover version of a song against its original, or a live recording against a studio recording. As the identification is continuous, the algorithm can also generate a cuesheet for a long recording encompassing multiple music pieces.

How does it work?

Our algorithm computes a small number of fingerprints per second both for the reference files, stored in a database, and for the query file, matched against the database. For piano matching, the fingerprints are computed from the reference scores and an automatic transcription of the piano recording. This way, the algorithm is independent of the piano the query is played on. For matching arbitrary music excerpts, fingerprints are computed directly from the audio signal and can only be used to identify a rendition of exactly the same recording. In both cases, the fingerprints are constructed to be robust to time stretching (tempo changes) and pitch shifting (transposition), as occurring in piano interpretation or vinyl playback. Please see Arzt et al. for a detailed description of the piano matching algorithm, and Sonnleitner et al. for the generic fingerprinter.

What is it good for?

The generic fingerprinter can be used to identify music pieces played on radio or used in video productions, given a suitable reference database. The piano identification system can be useful for piano students and scholars.

How can i get it?

Please contact Gerhard Widmer if you would like to license one of our music identification methods.