SMUG-Explain: A Framework for Symbolic Music Graph Explanations

Emmanouil Karystinaios,Francesco Foscarin,Gerhard Widmer

2024-05-15

Abstract:In this work, we present Score MUsic Graph (SMUG)-Explain, a framework for generating and visualizing explanations of graph neural networks applied to arbitrary prediction tasks on musical scores. Our system allows the user to visualize the contribution of input notes (and note features) to the network output, directly in the context of the musical score. We provide an interactive interface based on the music notation engraving library Verovio. We showcase the usage of SMUG-Explain on the task of cadence detection in classical music. All code is available on

Sound,Audio and Speech Processing

What problem does this paper attempt to address?

### What problem does this paper attempt to solve?

This paper aims to solve the interpretability problem when graph neural networks (GNNs) are applied to music scores in the field of music information retrieval (MIR). Specifically, the author proposes a framework named **Score MUsic Graph (SMUG)-Explain** for generating and visualizing explanations in the prediction tasks of graph neural networks on music scores. The main objectives include:

1. **Improving the interpretability of GNN models**: Traditional deep - learning models, including GNNs, usually do not have inherent interpretability when processing music scores. This makes it difficult for researchers to examine the internal working mechanism of the system and gain in - depth understanding of the model's understanding of specific tasks.

2. **Explanations for the music field**: Most of the existing explanation methods are applicable to matrix - form inputs (such as spectrograms or piano rolls), while this paper focuses on MIR systems that use graph structures as inputs. The author believes that graph - based explanations can more intuitively point to individual notes in the score and their neighborhoods, thus providing more musically meaningful explanations.

3. **Interactive visualization interface**: In order to make the explanation results more understandable, the author has developed a web - based interactive interface where users can directly view and analyze the explanation results on the music score. This helps users better understand the prediction logic of the model and combine it with musicological knowledge.

4. **Evaluating the effectiveness of explanations**: The author uses multiple post - hoc explanation methods and verifies the quality of these explanations through quantitative evaluation metrics (such as sufficiency and necessity conditions). They also introduce feature importance analysis to show the impact of each note's features on the model's prediction.

In summary, the main contribution of this paper is to provide a new framework that can not only generate explanations for GNN models on music scores but also help users understand and verify these explanations through an interactive visualization tool. This is of great significance for promoting research in the field of music information retrieval.