Abstract. This paper deals with the deep learning revolution in Music Information Research (MIR), i.e. the switch from knowledge-driven hand-crafted systems to data-driven deep-learning systems. To discuss the pro and cons of this revolution, we first review the basic elements of deep learning and explain how those can be used for audio feature learning or for solving difficult MIR tasks. We then discuss the case of hand-crafted features and demonstrate that, while those where indeed shallow and explainable at the start, they tended to be deep, data-driven and unexplainable over time, already before the reign of deep-learning. The development of these data-driven approaches was allowed by the increasing access to large annotated datasets. We therefore argue that these annotated datasets are today the central and most sustainable element of any MIR research. We propose new ways to obtain those at scale. Finally we highlight a set of challenges to be faced by the deep learning revolution in MIR, especially concerning the consideration of music specificities, the explainability of the models (X-AI) and their environmental cost (Green-AI).
Preface
The 14th edition of the International Symposium on Computer Music Multidisciplinary Research CMMR2019 “Perception, Representations, Image, Sound, Music” (https:// cmmr2019.prism.cnrs.fr/) took place in Marseilles on 14th-18th October 2019. This was the second CMMR event to be organized in Marseilles, but in a slightly different context than in 2013, since the present edition marked the creation of the new inter- disciplinary art-science laboratory, PRISM (Perception, Representations, Image, Sound, Music), which we co-founded in 2017 and which very much reflects the spirit of the CMMR conference cycle. PRISM hosts researchers within a large variety of fields, spanning from physics and signal processing, arts and aesthetic sciences to medicine and neuroscience, that all have a common interest in the perception and representation of image, sound and music. The scientific challenge of PRISM is to reveal how the audible, the visible and their interactions generate new forms of sensitive and/or formal representations of the contemporary world.
CMMR2019 was the occasion to inaugurate the PRISM laboratory in the light of the theme of the conference and at the same time honor one of its co-founders, researcher, composer and computer music pioneer Jean-Claude Risset who sadly passed away in November 2016, only two months before the laboratory was officially inaugurated. A scientific session followed by an evening concert was dedicated to him on the first day of the conference. Jean-Claude Risset was also one of the keynote speakers during CMMR2013 in Marseilles and a video of his presentation is available here: https:// cmmr2013.prism.cnrs.fr/K1.html
From the first announcement of CMMR2019 we received a large response from both scientists and artists who wanted to participate in the conference, either by organizing special sessions, presenting demos or installations or proposing workshops and concerts. Among the 15 scientific sessions that took place during the conference, eight special sessions that dealt with various subjects from sound design, immersive media and mobile devices, to music and deafness, embodying musical interaction and phenomenology of the conscious experience were scheduled. We were also lucky to have three internationally renowned keynote speakers with us during this edition: John Chowning, Professor Emeritus at Stanford University, Geoffroy Peeters, Professor at Télécom Paris and Josh McDermott, Associate Professor in the Department of Brain and Cognitive Sciences at MIT. Professor John Chowning, who also wrote a foreword that preceded this preface, and strongly encouraged the creation of multidisciplinary laboratories such as PRISM. His piece Turenas was played at the end of his keynote, followed by the concert in tribute to Jean-Claude Risset. Professor Geoffroy Peeters gave a talk on the evolution of Music Information Retrieval, and in particular the deep learning revolution in MIR, while Dr. Josh McDermott presented classic and recent approaches to auditory scene analysis. The videos of the 3 keynotes are available here:https://cmmr2019.prism.cnrs.fr/keynotes.html
As in previous CMMR editions, an artistic program accompanied the scientific sessions. The program was elaborated with “n+n corsino” and “Groupe de Musique Expérimentale de Marseille” (GMEM). The concerts took place at “La Friche la Belle de Mai” (Friche la Belle de Mai - Une fabrique d’art et de culture et un espace de vie) and consisted of a virtual/augmented concert, a contemporary music concert and an interactive music image concert. Sound installations and videomusic were also presented to the participants.
In addition to the oral paper sessions, poster and demo sessions were organized, as well as five “satellite” workshops on Ubiquitous Music, Musicntwrk, Sound Morphing and the Sonic Continuum, and Introduction and Advanced Programming with the bach Library (for more details see https://cmmr2019.prism.cnrs.fr/workshops.html).
The post-proceedings edition of CMMR2019 is the 14th CMMR publication by Springer Verlag in the Lecture Notes in Computer Science Series (LNCS 2771, LNCS 3310, LNCS 3902, LNCS 4969, LNCS 5493, LNCS 5954, LNCS 6684, LNCS 7172, LNCS 7900, LNCS 8905, LNCS 9617, LNCS 10525, LNCS 11265). This year’s edition contains 46 peer-reviewed chapters. It is divided into 9 sections. The first three sections are related to music information retrieval, computational musicology and composition tools, followed by a section on notations and instruments distributed on mobile devices. The fifth section concerns auditory perception and cognition, while the three following sections are related to sound design and sonic and musical interactions. The last section contains contributions that relate to Jean-Claude Risset’s research.
We would like to thank all the participants of CMMR2019 who made this event possible. A special thanks to our colleagues and students at the PRISM lab as well as n+n corsino and GMEM who took care of the practical organization and insured a smooth and efficient coordination between attendees, speakers, audiences and musicians in both the scientific and artistic programs. We would also like to thank the Program and Music Committee members for their indispensable selection tasks.
Finally, we would like to thank Springer for agreeing to publish the CMMR 2019 post-proceedings edition in their LNCS series.
December 2020
Richard Kronland-Martinet Mitsuko Aramaki Sølvi Ystad
Foreword
“Jean-Claude Risset: Humanist, Composer, Musician, and Scientist”
“The impact of Jean-Claude Risset’s death in 2016 was immediate and stunning - the news traveled fast in the worlds of music and science that had been so enriched by his life’s work. But it was not always so. The path that Jean-Claude had chosen in his first years at Bell Labs, research in acoustics and psychoacoustics, was not guaranteed to be productive in music composition. However, his remarkable ability to hear inside sound pushed him toward discoveries that still resonate these many years later.
I will recount my understanding of his gradual acceptance by the colleagues he cared about, whom he admired, as they learned that there was genius behind his modest presence.”
When I wrote these words of introduction to my Keynote address for the International Symposium on Computer Music Multidisciplinary Research (CMMR) on October 14, 2019, I believed that these five days would be a tribute to Jean-Claude’s musical and scientific contributions and his person. It was, indeed, with excellent papers exposing the most recent research ranging from computational musicology, tools for music composition, to music and the brain.
At the outset of the symposium, there was another event that was especially important to those of us who know the history of musical and scientific research in France and Marseille in particular - the presentation of the PRISM laboratory as a permanent, formal, funded part of the Aix-Marseille University under the leadership of Richard Kronland-Martinet and Sølvi Ystad. Until then, its future was unsure. Jean-Claude’s own research and teaching career was disrupted several times since his first position at Luminy in 1972, because of changes in administrative views regarding computer music. Finally, in his last years, he found a home with PRISM.
Also related to this symposium was another issue, concern for the archiving of Jean-Claude’s extensive scientific writings, musical works and scores, and personal notes that he carefully kept of every presentation at every conference that he attended through his career. There seemed to be little interest in France or digital archiving technology, except at Ina-GRM and Daniel Teruggi and Yann Geslin. Beginning in 2010, Jean-Claude, Max Mathews, and I discussed where these materials should go. Finally, following his death, his family decided that the archive should remain in France, and everyone involved agreed that it should remain intact in its entirety.
Those of us involved also agreed that Stanford University Libraries, one of the most advanced digital archiving facilities globally, would be a mirror archive for Jean-Claude’s work with on-demand access to his works as they become available from PRISM. But the question remained, could it be done, where are the resources, where should the archived original documents be housed?
Among the papers given at the symposium, there was “Jean-Claude Risset and his Interdisciplinary Practice: What do the Archives Tell Us?” Now, as I write these words (27-08-20), I have received an email from Vincent Tiffon, author of the above paper, that he and colleagues are well on their way in the beginning stages of digital documentation.
So, I conclude with the comforting knowledge that the right team is responsible for the ongoing project to digitize and archive Jean-Claude’s work; his chronicle of, and contributions to, the beginning of the digital age of music will be available to all.
Link to the keynote video: https://cmmr2019.prism.cnrs.fr/keynoteChowning.html
John M. Chowning