Editorial: Live Coding Sonic Creativities

Anna Xambó Sedó,Gerard Roma,Thor Magnusson

DOI: https://doi.org/10.1017/s135577182300047x

2023-08-02

Organised Sound

Abstract://static.cambridge.org/content/id/urn%3Acambridge.org%3Aid%3Aarticle%3AS135577182300047X/resource/name/firstPage-S135577182300047Xa.jpg

What problem does this paper attempt to address?

Vivace: a collaborative live coding language and platform

Vilson Vieira,Guilherme Lunhani,Geraldo Magela de Castro Rocha Junior,Caleb Mascarenhas Luporini,Daniel Penalva,Ricardo Fabbri,Renato Fabbri

DOI: https://doi.org/10.48550/arXiv.1502.01312

2017-10-31

Abstract:Live coding is a performance and creative technique based on improvised and interactive coding. Many recent endeavors have focused in live coding both because of aesthetics and as a way to alleviate performance drawbacks when the musical instrument is a computer. This paper describes the principles and the design of Vivace, a live coding language and environment built with Web technologies to be executed on web browsers. The approach is compelling by 1) allowing many performers to code simultaneously, 2) the synthesis of audio and video, 3) a very simple syntax, 4) being a multiplatform software. We also strive to contextualize Vivace by means of historical and usage summaries including a live coding sub-genre.

Computers and Society,Human-Computer Interaction
A Study of Editor Features in a Creative Coding Classroom

Andrew McNutt,Anton Outkine,Ravi Chugh

DOI: https://doi.org/10.48550/arXiv.2301.13302

IF: 6.4588

2023-01-30

Human-Computer Interaction

Abstract:Creative coding is a rapidly expanding domain for both artistic expression and computational education. Numerous libraries and IDEs support creative coding, however there has been little consideration of how the environments themselves might be designed to serve these twin goals. To investigate this gap, we implemented and used an experimental editor to teach a sequence of college and high-school creative coding courses. In the first year, we conducted a log analysis of student work (n=39) and surveys regarding prospective features (n=25). These guided our implementation of common enhancements (e.g. color pickers) as well as uncommon ones (e.g. bidirectional shape editing). In the second year, we studied the effects of these features through logging (n=39+) and survey (n=23) studies. Reflecting on the results, we identify opportunities to improve creativity- and novice-focused IDEs and highlight tensions in their design -- as in tools that augment artistry or efficiency but may be perceived as hindering learning.
LyricJam Sonic: A Generative System for Real-Time Composition and Musical Improvisation

Olga Vechtomova,Gaurav Sahu

DOI: https://doi.org/10.48550/arXiv.2210.15638

2022-10-27

Sound

Abstract:Electronic music artists and sound designers have unique workflow practices that necessitate specialized approaches for developing music information retrieval and creativity support tools. Furthermore, electronic music instruments, such as modular synthesizers, have near-infinite possibilities for sound creation and can be combined to create unique and complex audio paths. The process of discovering interesting sounds is often serendipitous and impossible to replicate. For this reason, many musicians in electronic genres record audio output at all times while they work in the studio. Subsequently, it is difficult for artists to rediscover audio segments that might be suitable for use in their compositions from thousands of hours of recordings. In this paper, we describe LyricJam Sonic -- a novel creative tool for musicians to rediscover their previous recordings, re-contextualize them with other recordings, and create original live music compositions in real-time. A bi-modal AI-driven approach uses generated lyric lines to find matching audio clips from the artist's past studio recordings, and uses them to generate new lyric lines, which in turn are used to find other clips, thus creating a continuous and evolving stream of music and lyrics. The intent is to keep the artists in a state of creative flow conducive to music creation rather than taking them into an analytical/critical state of deliberately searching for past audio segments. The system can run in either a fully autonomous mode without user input, or in a live performance mode, where the artist plays live music, while the system "listens" and creates a continuous stream of music and lyrics in response.
ApProgXimate audio: a distributed interactive experiment in sound art and live coding

Chris Kiefer

DOI: https://doi.org/10.1080/14794713.2016.1227599

2016-07-02

International Journal of Performance Arts and Digital Media

Abstract:ApProgXimate Audio is a browser-based sound art experiment, a distributed and participatory piece where code and livecoding are fundamental to the participants’ experience. The piece is based on a coding technique called Approximate Programming. It allows participants to explore complex sound synthesis algorithms through relatively simple techniques, by writing and editing code and through screen-based mouse interaction. It foregrounds code as the primary medium of engagement. This article explains the technical background behind the piece, and then explores the design philosophy through three themes that emerged during its creation: accessibility and paths to engagement in work based on live coding, the blurred boundaries between tools, instruments and artworks, and finally issues surround the web-browser as a venue for creative work.
Live Coding Machine Learning: Finding the Moments of Intervention in Autonomous Processes

Iván Paz,Shelly Knotts

DOI: https://doi.org/10.1162/comj_a_00663

IF: 0.688

2024-02-06

Computer Music Journal

Abstract:Abstract Machine learning (ML) deals with algorithms able to learn from data, with the primary aim of finding optimum solutions to perform tasks autonomously. In recent years there has been development in integrating ML algorithms with live coding practices, raising questions about what to optimize or automate, the agency of the algorithms, and in which parts of the ML processes one might intervene midperformance. Live coding performance practices typically involve conversational interaction with algorithmic processes in real time. In analyzing systems integrating live coding and ML, we consider the musical and performative implications of the “moment of intervention” in the ML model and workflow, and the channels for real-time intervention. We propose a framework for analysis, through which we reflect on the domain-specific algorithms and practices being developed that combine these two practices.

computer science, interdisciplinary applications
Virtual Agents in Live Coding: A Short Review

Anna Xambó

DOI: https://doi.org/10.48550/arXiv.2106.14835

IF: 6.4588

2021-06-28

Human-Computer Interaction

Abstract:AI and live coding has been little explored. This article contributes with a short review of different perspectives of using virtual agents in the practice of live coding looking at past and present as well as pointing to future directions.
Live Coding: A Review of the Literature

Ana Selvaraj,Eda Zhang,Leo Porter,Adalbert Gerald Soosai Raj

DOI: https://doi.org/10.1145/3430665.3456382

2021-06-26

Abstract:One of the goals of computing education research is to document the potential strengths and weaknesses of contemporary teaching methods in computing. Live coding has recently gained attention as one of the best practices for teaching programming. To offer a more comprehensive understanding of the existing body of research about live coding, we reviewed papers in computing education research that investigated the value of live coding in an educational setting. We categorized each paper based on (1) how it defines live coding, (2) whether its version of live coding could be considered active learning, (3) the type of study conducted, (4) types of data collected and the data analysis methods used, (5) evidence provided for the effectiveness of live coding, (6) reported benefits and drawbacks of live coding, and (7) reported theoretical frameworks used to explain the basis, effects or goals of live coding. We found that although live coding has been recommended as one of the best practices for teaching programming, there is a lack of empirical evidence to support claims about the effectiveness of live coding on student learning. Finally, we discuss the implications of our findings and suggest future research directions that could develop a more holistic understanding of this pedagogical technique.
Sonic Writing

Thor Magnusson

DOI: https://doi.org/10.5040/9781501313899

2019-01-01

Abstract:Sonic Writing explores how contemporary music technologies trace their ancestry to previous forms of instruments and media. Studying the domains of instrument design, musical notation, and sound recording under the rubrics of material, symbolic, and signal inscriptions of sound, the book describes how these historical techniques of sonic writing are implemented in new digital music technologies. With a scope ranging from ancient Greek music theory, medieval notation, early modern scientific instrumentation to contemporary multimedia and artificial intelligence, it provides a theoretical grounding for further study and development of technologies of musical expression. The book draws a bespoke affinity and similarity between current musical practices and those from before the advent of notation and recording, stressing the importance of instrument design in the study of new music and projecting how new computational technologies, including machine learning, will transform our musical practices. Sonic Writing offers a richly illustrated study of contemporary musical media, where interactivity, artificial intelligence, and networked devices disclose new possibilities for musical expression. Thor Magnusson provides a conceptual framework for the creation and analysis of this new musical work, arguing that contemporary sonic writing becomes a new form of material and symbolic design–one that is bound to be ephemeral, a system of fluid objects where technologies are continually redesigned in a fast cycle of innovation.
JAMMIN-GPT: Text-based Improvisation using LLMs in Ableton Live

Sven Hollowell,Tashi Namgyal,Paul Marshall

2023-12-06

Abstract:We introduce a system that allows users of Ableton Live to create MIDI-clips by naming them with musical descriptions. Users can compose by typing the desired musical content directly in Ableton's clip view, which is then inserted by our integrated system. This allows users to stay in the flow of their creative process while quickly generating musical ideas. The system works by prompting ChatGPT to reply using one of several text-based musical formats, such as ABC notation, chord symbols, or drum tablature. This is an important step in integrating generative AI tools into pre-existing musical workflows, and could be valuable for content makers who prefer to express their creative vision through descriptive language. Code is available at <a class="link-external link-https" href="https://github.com/supersational/JAMMIN-GPT" rel="external noopener nofollow">this https URL</a>.

Sound,Artificial Intelligence,Human-Computer Interaction,Audio and Speech Processing
Cocreative Interaction: Somax2 and the REACH Project

Gérard Assayag,Laurent Bonnasse-Gahot,Joakim Borg

DOI: https://doi.org/10.1162/comj_a_00662

IF: 0.688

2024-02-06

Computer Music Journal

Abstract:Abstract Somax2 is an artificial intelligence (AI)-based multiagent system for human–machine coimprovisation that generates stylistically coherent streams while continuously listening and adapting to musicians or other agents. The model on which it is based can be used with little configuration to interact with humans in full autonomy, but it also allows fine real-time control of its generative processes and interaction strategies, closer in this case to a “smart” digital instrument. An offspring of the Omax system, conceived at the Institut de Recherche et Coordination Acoustique/Musique, the Somax2 environment is part of the European Research Council Raising Cocreativity in Cyber–Human Musicianship (REACH) project, which studies distributed creativity as a general template for symbiotic interaction between humans and digital systems. It fosters mixed musical reality involving cocreative AI agents. The REACH project puts forward the idea that cocreativity in cyber–human systems results from the emergence of complex joint behavior, produced by interaction and featuring cross-learning mechanisms. Somax2 is a first step toward this ideal, and already shows life-size achievements. This article describes Somax2 extensively, from its theoretical model to its system architecture, through its listening and learning strategies, representation spaces, and interaction policies.

computer science, interdisciplinary applications
A Participatory Live Music Performance with the Open Symphony System

Kate Hayes,Mathieu Barthet,Yongmeng Wu,Leshao Zhang,Nick Bryan-Kinns

DOI: https://doi.org/10.1145/2851581.2889471

2016-05-07

Abstract:Our Open Symphony system reimagines the music experience for a digital age, fostering alliances between performer and audience and our digital selves. Open Symphony enables live participatory music performance where the audience actively engages in the music creation process. This is made possible by using state-of-the-art web technologies and data visualisation techniques. Through collaborations with local performers we will conduct a series of interactive music performance revolutionizing the performance experience both for performers and audiences. The system throws open music-creating possibilities to every participant and is a genuine novel way to demonstrate the field of Human Computer Interaction through computer-supported cooperative creation and multimodal music and visual perception.
Cracking God’s roof: Manifestation and adaptation on the intuitive nature of creating electronic music with tablet computers

Willard G. Van De Bogart

DOI: https://doi.org/10.1386/tear_00027_1

2020-03-01

Technoetic Arts

Abstract:Electronic music is advancing not only in the way it is being used in performance but also in the technological sense, due to software developers advancing the ability of the synthesizer to enable the composer to create newer sounds. The introduction of the amino acid and protein synthesizers from MIT is one such example, along with sampling sounds from interstellar bodies through the process of sonification in order to create presets as additional source material for the composer’s palette. The creative process used in creating electronic music on a tablet computer introduces a new musical instrument to be used in live music performances. The fluidity and immediacy of how electronic sounds can be created with tablet computer synthesizers affords the composer to have a new behavioural sense of using them as a musical instrument that can be played intuitively. Exploring this new interface of musical composition is a subject this article will address as well as the psychological aspects pertaining to how an audience can relate to electronic music as an emerging art form removed from the classical music tradition. It will also discuss how the composer of electronic music can affect the listener’s ability to envision new conceptual landscapes, leading to experiencing new ideas and subjective fields of visionary understandings. The composer’s ability to use conceptual models, which influence the way sounds are made and how those sounds influence the listener’s experience, is an important focus of this article.

English Else
Discovering novel computer music techniques by exploring the space of short computer programs

Ville-Matias Heikkilä

DOI: https://doi.org/10.48550/arXiv.1112.1368

2011-12-07

Abstract:Very short computer programs, sometimes consisting of as few as three arithmetic operations in an infinite loop, can generate data that sounds like music when output as raw PCM audio. The space of such programs was recently explored by dozens of individuals within various on-line communities. This paper discusses the programs resulting from this exploratory work and highlights some rather unusual methods they use for synthesizing sound and generating musical structure.

Sound
Creative Autonomy Through Salience and Multidominance in Interactive Music Systems: Evaluating an Implementation

Fabio Paolizzo,Colin G. Johnson

DOI: https://doi.org/10.48550/arXiv.1711.11319

IF: 6.4588

2017-11-30

Human-Computer Interaction

Abstract:Interactive music systems always exhibit some autonomy in the creative process. The capacity to generate novel material while retaining mutuality to the interaction is proposed here as the bare minimum for creative autonomy in such systems. Video Interactive VST Orchestra is a system incorporating an adaptive technique based both on the concept of salience as a means for retaining mutuality to the interplay and on multidominance in the adaptive generation process as a means for introducing novelty. We call this property reflexive multidominance. A case study providing evidence of such creative autonomy in VIVO is presented.
LiveScaler: Live control of the harmony of an electronic music track

Alice Rixte

2024-01-16

Abstract:In Electronic Dance Music (EDM), many artists use DJing techniques in order to perform their own productions live. As a consequence, they do not have access during the performance to the internal structure of their tracks, and specifically to their equivalent of a partition: MIDI files. On the other hand, if an artist attempts to remix or interpret their own production live, the number of tracks that they can simultaneously control is limited without suitable software. This article introduces LiveScaler, a software that allows live control of the harmony and pitch of electronic music. A set of pitch transformations, termed affine transformations, is presented. These transformations are applied to all MIDI streams of a prepared track. A MaxMSP implementation, in conjunction with Ableton Live, is proposed. Special attention is given to control issues, mapping, and practical live experimentation in the context of EDM.

Sound,Audio and Speech Processing
Sound feedback control model for live electronic performance in the Ecos Study

Ricardo Thomasi,Regis R. A. Faria

DOI: https://doi.org/10.33871/23179937.2021.9.3.3

2021-12-15

Revista Vórtex

Abstract:Through the Ecos Study we investigate possibilities for musical structuring based on theories of emergence, finding a fertile territory for experimentations within Agostino Di Scipio's paradigm of Audible Ecosystems. The inclusion of the acoustic environment as an active component of the musical system led us to rethink classical techniques of sound synthesis and modelling, some traditional models of electroacoustic performance, and to criticize the very role of interactions in a qualitative perspective. A positive feedback control model has been developed aiming at understanding the behaviour and the structure of the ecosystem supporting the implementation of the Spatial Instrument, a core element implemented through MAX/MSP patch and that offers a logical rationale for organizing speakers and microphones in the acoustic space. We introduce the underlying theoretical debate and present the feedback control mechanism as the basis from which the Spatial Instrument is developed.
The Paradox of Creating a Cocreator

Notto J. W. Thelle,Bernt Isak Wærstad

DOI: https://doi.org/10.1162/comj_a_00666

IF: 0.688

2024-02-29

Computer Music Journal

Abstract:Abstract With the latest developments in AI it is becoming increasingly common to view machines as cocreators. In this article, we follow four musicians in the project Co-Creative Spaces through a six-month long collaborative process, in which they created new music through improvising with each other and—subsequently—with computer-based imitations of themselves. These musical agents were trained through machine learning to generate output in the style of the musicians and were capable of both following what they “heard” and initiating new directions in the interaction, leading to the question “What happens to musical cocreation when AI is thus included in the creative cycle?” The musicians involved in Co-Creative Spaces are from Norway and Kenya—two countries with fundamentally different musical traditions. This leads to a second question: “How is the collaboration affected by possible cultural biases inherent in the technology and in the musicians themselves?” These questions were examined as part of two five-day workshops—one at the beginning and one at the end of the project period—before two final concerts. The musicians engaged in improvisation sessions and recorded ensuing discussions. For each workshop day, the musicians also had conversations in focus groups moderated by a fifth project member, who, together with one of the musicians, was also responsible for the development of the software powering the musical agents. The analysis of the data from the workshops paints a complex picture of what it is like being at the intersection between different technological, musical, and cultural paradigms. The machine becomes a cocreator only when humans permit themselves to attribute creative agency to it.

computer science, interdisciplinary applications
Live music programming in Haskell

Henning Thielemann

DOI: https://doi.org/10.48550/arXiv.1303.5768

2013-03-22

Programming Languages

Abstract:We aim for composing algorithmic music in an interactive way with multiple participants. To this end we have developed an interpreter for a sub-language of the non-strict functional programming language Haskell that allows the modification of a program during its execution. Our system can be used both for musical live-coding and for demonstration and education of functional programming.
Live, Rich, and Composable: Qualities for Programming Beyond Static Text

Joshua Horowitz,Jeffrey Heer

DOI: https://doi.org/10.48550/arXiv.2303.06777

2023-03-13

Abstract:Efforts to push programming beyond static textual code have sought to imbue programming with multiple distinct qualities. One long-acknowledged quality is liveness: providing programmers with in-depth feedback about a program's dynamic behavior as the program is edited. A second quality, long-explored but lacking a shared term of art, is richness: allowing programmers to edit programs though domain-specific representations and interactions rather than solely through text. In this paper, we map the relationship between these two qualities and survey past work that exemplifies them. We observe that systems combining liveness and richness often do so at the cost of an essential quality of traditional programming: composability. We argue that, by combining liveness, richness, and composability, programming systems can better capture the full potential of interactive computation without leaving behind the expressivity of traditional code.

Programming Languages,Human-Computer Interaction,Software Engineering
From Peering in the Window to Opening the Door: a constructivist approach to making electroacousticmusic accessible to young listeners

ANNA-MARIE HIGGINS,KEVIN JENNINGS

DOI: https://doi.org/10.1017/s1355771806001464

2006-08-01

Organised Sound

Abstract:A casual listener may not have the relevant cognitive tools to relate to electroacoustic music as an independent art form. Even music students who have had several years' formal training may not know how to listen purposefully to it. The authors investigate the hypothesis that by engaging in the composition of musique concrète , students will become more informed listeners to electroacoustic music in general. This study describes a teacher and ten 16-year-old high school students who use a digital audio editor ( Cool Edit Pro ) to manipulate sound and to structure sound events. A variety of teaching approaches are tested. Having experienced the creative process, the participants display a growing ability to engage in higher-order critical thinking skills in relation to electroacoustic music.

Editorial: Live Coding Sonic Creativities

Vivace: a collaborative live coding language and platform

A Study of Editor Features in a Creative Coding Classroom

LyricJam Sonic: A Generative System for Real-Time Composition and Musical Improvisation

ApProgXimate audio: a distributed interactive experiment in sound art and live coding

Live Coding Machine Learning: Finding the Moments of Intervention in Autonomous Processes

Virtual Agents in Live Coding: A Short Review

Live Coding: A Review of the Literature

Sonic Writing

JAMMIN-GPT: Text-based Improvisation using LLMs in Ableton Live

Cocreative Interaction: Somax2 and the REACH Project

A Participatory Live Music Performance with the Open Symphony System

Cracking God’s roof: Manifestation and adaptation on the intuitive nature of creating electronic music with tablet computers

Discovering novel computer music techniques by exploring the space of short computer programs

Creative Autonomy Through Salience and Multidominance in Interactive Music Systems: Evaluating an Implementation

LiveScaler: Live control of the harmony of an electronic music track

Sound feedback control model for live electronic performance in the Ecos Study

The Paradox of Creating a Cocreator

Live music programming in Haskell

Live, Rich, and Composable: Qualities for Programming Beyond Static Text

From Peering in the Window to Opening the Door: a constructivist approach to making electroacousticmusic accessible to young listeners