Abstract:The use of machine learning in artistic music generation leads to controversial discussions of the quality of art, for which objective quantification is nonsensical. We therefore consider a music-generating algorithm as a counterpart to a human musician, in a setting where reciprocal interplay is to lead to new experiences, both for the musician and the audience. To obtain this behaviour, we resort to the framework of recurrent Variational Auto-Encoders (VAE) and learn to generate music, seeded by a human musician. In the learned model, we generate novel musical sequences by interpolation in latent space. Standard VAEs however do not guarantee any form of smoothness in their latent representation. This translates into abrupt changes in the generated music sequences. To overcome these limitations, we regularise the decoder and endow the latent space with a flat Riemannian manifold, i.e., a manifold that is isometric to the Euclidean space. As a result, linearly interpolating in the latent space yields realistic and smooth musical changes that fit the type of machine--musician interactions we aim for. We provide empirical evidence for our method via a set of experiments on music datasets and we deploy our model for an interactive jam session with a professional drummer. The live performance provides qualitative evidence that the latent representation can be intuitively interpreted and exploited by the drummer to drive the interplay. Beyond the musical application, our approach showcases an instance of human-centred design of machine-learning models, driven by interpretability and the interaction with the end user.

Live Coding Machine Learning: Finding the Moments of Intervention in Autonomous Processes

The Machine Learning Algorithm as Creative Musical Tool

Machine Learning practices and infrastructures

Exploring Machine Teaching with Children

Live Coding: A Review of the Literature

Not Just Training, Also Testing: High School Youths' Perspective-Taking through Peer Testing Machine Learning-Powered Applications

A First Step in Using Machine Learning Methods to Enhance Interaction Analysis for Embodied Learning Environments

Coding in the Liberal Arts through Natural Language Processing and Machine Learning

Coding with the machines: machine-assisted coding of rare event data

Coding moves: Design and research of teaching computational thinking through dance choreography and virtual interactions

Performing Structured Improvisations with pre-trained Deep Learning Models

Lost in Translation: Reimagining the Machine Learning Life Cycle in Education

Validating AI-Generated Code with Live Programming

The promise and peril of interactive embodied agents for studying non-verbal communication: a machine learning perspective

Virtual Agents in Live Coding: A Short Review

Improving Students Performance in Small-Scale Online Courses -- A Machine Learning-Based Intervention

Choreographing the Digital Canvas: A Machine Learning Approach to Artistic Performance

The black box aesthetics: performative dynamics between artificial intelligence and human interactive staging

Flat Latent Manifolds for Human-machine Co-creation of Music

Dancing With Algorithms: Interaction Creates Greater Preference and Trust in Machine-Learned Behavior

Addressing UX Practitioners' Challenges in Designing ML Applications: an Interactive Machine Learning Approach