Abstract:In this article, we present our work on the sonification of notated complex spectral structures. It is part of a larger research project about the design of a new notation system for representing sound-based musical structures. Complex spectral structures are notated with special symbols in the scores, which can be digitally rendered so that the user can hear key aspects of what has been notated. This hearing of the notated data is significantly different from reading the same data, and reveals the complexity hidden in its simplified notation. The digitally played score is not the music itself but can provide essential information about the music in ways that can only be obtained in sounding form. The playback needs to be designed so that the user can make relevant sonic readings of the sonified data. The sound notation system used here is an adaptation of Thoresen and Hedman's spectromorphological analysis notation. Symbols originally developed by Lasse Thoresen from Pierre Schaeffer's typo-morphology have in this system been adapted to display measurable spectral features of timbrel structure for the composition and transcription of sound-based musical structures. Spectrum category symbols are placed over a spectral grand-staff that combines indications of pitch and frequency values for the combined display of music related to pitch-based and spectral values. Spectral features of a musical structure such as spectral width and density are represented as graphical symbols and sonically rendered. In perceptual experiments we have verified that users can identify spectral notation parameters based on their sonification. This confirms the main principle of sonification that is that the data/dimensions relations in one domain, in our case notated representation of spectral features, are transformed in perceived relations in the audio domain, and back.

The Music Note Ontology

The Music Annotation Pattern

The Music Meta Ontology: a flexible semantic model for the interoperability of music metadata

Structure Analysis Approach of Music Notes for Optical Music Recognition

The HaMSE Ontology: Using Semantic Technologies to support Music Representation Interoperability and Musicological Analysis

A Unified Representation Framework for the Evaluation of Optical Music Recognition Systems

The Internet of Musical Things Ontology

Notation as visual representation of sound-based music

Critical Semantic Properties of Music Notation Datasets

Analyzing Visual Mappings of Traditional and Alternative Music Notation

Enriching Music Mood Annotation By Semantic Association Reasoning

Predicting High-level Music Semantics Using Social Tags via Ontology-based Reasoning.

Coordinate Embedding Transformer Model for Optical Music Recognition on Monophonic Scores

Sonification of Complex Spectral Structures

Unleashing the Power of Sound: Revisiting the Physics of Notations for Modelling with auditory symbols

Understanding Optical Music Recognition

From ontology design to user-centred interfaces for music heritage

Towards a topological fingerprint of music

Sheet Music Transformer: End-To-End Optical Music Recognition Beyond Monophonic Transcription

Do we need more complex representations for structure? A comparison of note duration representation for Music Transformers

Generative Adversarial Network for Musical Notation Recognition during Music Teaching