2.1. The turntable as an instrument

Musique concrète, pioneered by Pierre Schaeffer and others, established the use of sound samples as raw materials. This was realised by manipulating turntables in the 1950s and continuing later with tape recorders and digital techniques (Schaeffer [1966] Reference Schaeffer2017: 38). A follow-up relates to the creative use of turntables and other devices in Jamaican dub in the late 1960s–early 1970s (Toop Reference Toop1995: 115–18) and early hip hop in the 1970s–1980s (White Reference White1996) to produce popular music. However, it was not until the dawn of affordable digital samplers in the 1980s that we find a popularisation of the use of sound samples as a common musical practice (Rodgers Reference Rodgers2003; Harkins Reference Harkins2019). This was generally linked to the production of pop music and later electronic music.

Harkins provides a useful working definition of digital sampling as ‘the use of digital technologies to record, store, and reproduce any sound’ (Harkins Reference Harkins2019: 4). Tara Rodgers describes the processes involved in electronic music production as ‘selecting, recording, editing and processing sound pieces to be incorporated into a larger musical work’ (Rodgers Reference Rodgers2003: 313). Rodgers also points to the characteristic of electronic musicians of taking the dual role of ‘producers–consumers of pre-recorded sounds and patterns that are transformed by a digital instrument that itself is an object of consumption and transformation’ (ibid.: 315).

2.2. The world as an instrument

Sound maps connect sound samples with their geolocation and have been widely explored since the advent of online databases and geolocation technologies. A prominent precursor is Murray Schafer’s acoustic ecology (Schafer Reference Schafer1977) and the related World Soundscape Project (Schafer Reference Schafer1977; Truax Reference Truax2002) in the 1970s at Simon Fraser University in Vancouver, Canada. This collective brought international attention to the sonic environment and raised awareness about noise pollution through soundscapes and environmental sounds.

Portable digital audio recorders and the internet brought about the possibility of creating crowdsourced sound maps of specific locations. In 2006, the composer Francisco López presented ‘The World as an Instrument’ at MACBA in Barcelona, a workshop where different artists’ work was introduced to reflect the new popular practices of soundscape composition using the ‘real world’ as a sonic palette.Footnote ³

Embedded devices have allowed the creation of worldwide open live streamings, such as Locustream Open Microphone Project, which developed streamboxes and mobile apps for streaming remote soundscapes (Sinclair Reference Sinclair2018). Started in 2005, the project offers a live sound map, the Locustream Soundmap,Footnote ⁴ showcasing a collection of live open microphones across the globe. Liveshout (Chaves and Rebelo Reference Chaves and Rebelo2011) is a mobile app that turns the phone into an open, on-the-move mic that can broadcast and contribute to the Locus Sonus soundmap with live collaborative performances (Sinclair Reference Sinclair2018).

2.3. Audio Commons for music composition and performance

The so-called Web 2.0, also known as the social web, represented a tipping point in the history of the internet, when, in the early 2000s, websites began to emphasise user-generated dynamic online content instead of traditional top-down static web pages (Bleicher Reference Bleicher2006). This included a range of online services that provide access to databases with hundreds of thousands of digital media files. The legal framework of CC licences (Lessig Reference Lessig2004) was devised to promote creativity by legally allowing for new ways of managing this digital content (e.g., sharing, reusing, remixing). This has led to a new community of prosumers, still relevant today, who both produce and consume online digital content (Ritzer and Jurgenson Reference Ritzer and Jurgenson2010), which aligns with the acknowledged prosumption existent in the electronic music culture (Rodgers Reference Rodgers2003). Navigating through these large databases is not an easy task and requires suitable tools.

The Audio Commons Initiative was conceived to promote the use of open audio content as well as to develop relevant technologies to support an ecosystem of databases, media production tools and users (Font et al. Reference Font, Brookes, Fazekas, Guerber, La Burthe and Plans2016). The use of CC sounds in linear media production has been discussed with use cases in composition and performance (Xambó, Font, Fazekas and Barthet Reference Xambó, Font, Fazekas, Barthet and Filimowicz2019).

Online databases such as Freesound offer an application programming interface (API) or software interface to allow communication between software applications. The Freesound APIFootnote ⁵ allows developers to communicate with the Freesound database using a provided set of tools and services to browse, search and retrieve information from Freesound, which works in conjunction with the audio analysis of the sounds using Essentia (Bogdanov et al. Reference Bogdanov, Wack, Gómez, Gulati, Herrera and Mayor2013). Freesound LabsFootnote ⁶ features several projects that foster creative ways of interacting with the CC sound database.

We can also find musical instruments that leverage cloud computing and CC sounds. For example, the smart mandolin generates a soundscape-based accompaniment using Freesound (Turchet and Barthet Reference Turchet and Barthet2018). Smart musical instruments (SMIs), such as the smart mandolin, were defined by Luca Turchet as ‘devices dedicated to playing music, which are equipped with embedded intelligence and are able to communicate with external devices’ (Turchet Reference Turchet2018: 8948). SMIs are an instance of using AI principles in new interfaces for musical expression (NIMEs).

The use of CC sounds in hardware digital samplers has previously been explored. For example, SAMPLER allows users to load and play sounds from Freesound (Font Reference Font2021). As with any traditional music sampler, users need to take several design decisions (e.g., query the sound, shape the sound with an envelope or trim the start/end) before playing samples. Although it is possible to work with multiple sounds at once, there is a limit to the sounds that can be loaded due to the hardware capacity.

These hardware limitations are overcome with the system presented in this article by using software. The system takes a reactable-like approach (Jordà, Geiger, Alonso and Kaltenbrunner Reference Jordà, Geiger, Alonso and Kaltenbrunner2007), whereby there is no distinction between designing a sound and performing with it. The sound is shaped while performed, aligning with process music proposed by Steve Reich, in which the process becomes the piece of music (Reich Reference Reich and Reich1965). Another key difference is that, compared with hardware samplers, which tend to be connected to musical instrument digital interface (MIDI) notes mapping, such as in SAMPLER, our approach explores other avenues. This way, the creativity is not confined to music-note input but, instead, the samples can lead to other musical spaces with their own rhythms, an approach inspired by Gerard Roma and colleagues’ seminal work on the Freesound Radio (Roma, Herrera and Serra Reference Roma, Herrera and Serra2009). Our focus is on providing easy-to-use software with a live-coding interface that fosters sound-based algorithmic music fed by CC sounds.

2.4. CC sounds in live coding

There exist multiple live-coding environments, which typically support different ways of performing audio synthesis, including sample operations. However, access to online crowdsourced databases of sounds is less common. Gibber is one browser-based live-coding environment that allows the live coder to perform audio synthesis with oscillators, synthesisers, audio effects and samples, among others (Roberts, Wright and Kuchera-Morin Reference Roberts, Wright and Kuchera-Morin2015). Amid the different options for manipulating samples, it is possible to retrieve, load, play back and manipulate sounds by using the object Freesound (ibid.). EarSketch is a web-based learning environment that takes a sample-based approach to algorithmic music composition using code and a digital audio workstation (DAW) interface (Freeman, Magerko, Edwards, Mcklin, Lee and Moore Reference Freeman, Magerko, Edwards, Mcklin, Lee and Moore2019). With an emphasis on musical genres, students can work with curated samples, personal samples or sounds from Freesound (ibid.). The audio repurposing of CC sounds from Freesound using music information retrieval (MIR) has been explored by the author and colleagues (Xambó et al. Reference Xambó, Lerch and Freeman2018a) and forms the foundation for this work.

While the manipulation of sounds from crowdsourced libraries in live coding has potential, it also has its limitations (see section 4). The main challenge is navigating unknown sound databases and finding appropriate sounds in live performance. One approach to overcoming the limitations is by combining personal and crowdsourced databases, which was explored by the author and colleagues obtaining promising results (Xambó, Roma, Lerch, Barthet and Fazekas Reference Xambó, Roma, Lerch, Barthet and Fazekas2018b). Ordiales and Bruno investigated the use of CC sounds from RedPanal.org and Freesound combined with sounds from local databases using a hardware interface for live coding (Ordiales and Bruno Reference Ordiales and Bruno2017). Another approach uses AI to enhance the retrieval of CC sounds (see section 3). It is out of the scope of this article to review the use of AI in live coding. An overview of several approaches to live coding using AI was presented by the author in a previous publication (Xambó Reference Xambó2022).

3.1. The MIRLCAuto project

In a nutshell, the system MIRLCAuto (MIRLCa)Footnote ⁷ was built on top of MIRLCRep (Xambó et al. Reference Xambó, Lerch and Freeman2018a), a user-friendly live-coding environment designed within SuperCollider and the Freesound quarkFootnote ⁸ to query CC sounds from the Freesound online database applying MIR techniques. A detailed technical account of MIRLCa and some early findings were published in 2021 (Xambó et al. Reference Xambó, Roma, Roig and Solaz2021).

MIRLCa uses supervised ML algorithms provided by the Fluid Corpus Manipulation (FluCoMa) toolkit (Tremblay, Roma and Green Reference Tremblay, Roma and Green2022). Based on the live coder’s musical preferences, the system learns and predicts the type of sounds the live coder prefers to be retrieved from Freesound. The aim is to offer a flexible and tailorable live-coding CC sound-based music environment. This approach allows ‘taming’ the heterogeneous nature of sounds from crowdsourced libraries towards the live coder’s needs, enhanced with the algorithmic music possibilities brought about by live coding.

Started in 2020, MIRLCa was built on SuperCollider and is in ongoing development. The author is the main developer, informed by conversations with the live-coding community, typically in the form of workshops, following a participatory design process. Its development has been also informed by observing and discussing its use by 16 live coders, including the author, as early adopter users in work-in-progress sessions, impromptu performances and concerts.

3.2. Research question and research methods

This article aims to identify the new sonic challenges and opportunities brought to live coders, computer musicians and sonic artists by the use of MIRLCa, a live-coding environment powered by AI working as a customisable sampler of sounds from around the globe. Here, a reflective retrospection is undertaken to look at the challenges and opportunities of manipulating CC sounds in live coding focusing on the (1) advantages vs disadvantages, and (2) live-coding compositional strategies.

We analysed text (e.g., interview blog posts, workshop attendees’ feedback) and video (e.g., work-in-progress sessions, concerts, impromptu performances), with most of the information publicly available in the form of blog posts and videos (see References section). A total of four workshops, three concerts with eight performances, four work-in-progress video sessions and four impromptu performances with four groups of one solo, two duos and one trio of live coders were analysed (see sections 3.3 and 3.4). These online and onsite activities involved more than 60 workshop participants and 16 live coders, including the author. We sought permission from the individuals named in the article.

To identify patterns of behaviour, the research methods are inspired by qualitative ethnographic analysis (Rosaldo Reference Rosaldo1993) and thematic analysis (Clarke and Braun Reference Clarke and Braun2006). Given the full-length video material of the concerts, work-in-progress sessions and impromptu performances, we also used video analysis techniques (Xambó, Laney, Dobbyn and Jordà Reference Xambó, Laney, Dobbyn, Jordà, Holland, Wilkie, Mulholland and Seago2013).

This research is a follow-up of our previous findings from two of the workshops and one concert with two performances (Xambó et al. Reference Xambó, Roma, Roig and Solaz2021). While in our previous publication, we presented a behind-the-scenes look at the system and explored the concept of personal musical preferences referred to as ‘situated musical actions’ (Xambó et al. Reference Xambó, Roma, Roig and Solaz2021), here we focus on the sonic potential to live coding that this novel approach entails.

3.3. The workshops and the work-in-progress sessions

Overall, we organised four workshops, inviting both beginners and experts in programming. Three workshops were carried out online while the fourth workshop was carried out onsite. Altogether, more than 60 participants attended the workshops.

The workshop ‘Performing with a Virtual Agent: Machine Learning for Live Coding’ was delivered three times in an online format. The three workshops had originally been planned as onsite with local participants but ended up becoming online due to the COVID-19 pandemic, which also allowed the inclusion of participants from around the world. The workshop was co-organised and delivered by the author together with Sam Roig in collaboration with three different organisations and communities: IKLECTIK (London), l’ull cec (Barcelona, Spain) and Leicester Hackspace (Leicester, UK).

The first workshop was held in December 2020 and it was organised in collaboration with IKLECTIK,Footnote ⁹ a platform dedicated to supporting experimental contemporary art and music. The other two workshops were organised in January 2021. One was organised in collaboration with l’ull cec,Footnote ¹⁰ an independent organisation that coordinates activities in the fields of sound art and experimental music and TOPLAP Barcelona,Footnote ¹¹ a Barcelona-based live-coding collective. The last one was organised in collaboration with Leicester Hackspace,Footnote ¹² a venue for makers of digital, electronic, mechanical and creative projects.

The purpose of these hands-on online workshops was to allow the participants (1) to explore the MIRLCRep2 tool (a follow-up version of MIRLCRep), and (2) to expose them to how ML could help improve the live-coding experience when using the MIRLCa tool. By the end of the workshops, the participants were able to train their ML models using our system’s live-coding training methods.

We offered tutorial sessions to help the workshop participants adapt the tool to their own practice. Thanks to the support of l’ull cec, TOPLAP Barcelona and Phonos,Footnote ¹³ a pioneering centre in the fields of electronic and electroacoustic music in Spain, we documented a series of four interviews and work-in-progress videos related to our workshop in Barcelona, featuring Hernani Villaseñor, Ramon Casamajó, Iris Saladino and Iván Paz.

In January 2022, the author together with Iván Paz co-organised the onsite workshop ‘Live Taming Free Sounds’. The workshop was part of the weekend event on-the-fly: Live Coding Hacklab at the Center for Art and Media Karlsruhe (ZKM) in Germany.Footnote ¹⁴ In the Hacklab, we acted as mentors for the topic of ML in live coding (Figure 1).

Figure 1. Video screenshot of the workshop ‘Live Taming Free Sounds’ at on-the-fly: Live Coding Hacklab on 29–30 January 2022, ZKM, Karlsruhe, Germany. Video by Mate Bredan.

The purpose of this hands-on workshop was to allow the participants (1) to get a quick overview of some different approaches to applying ML in live coding, (2) to do a hands-on inspection of how to classify CC sounds to use as a live digital worldwide sampler, and (3) to carry out an aesthetic incursion into sound-based music in live coding. By the end of the workshop, the participants were able to perform in a group with their ML models using our system’s live-coding training methods, as explained in the next section.

3.4. The concerts and impromptu performances

As a follow-up to the online workshops, we adapted our original idea of hosting three public onsite concerts to do what was possible under the pandemic circumstances. Consequently, the first concert, ‘Similar Sounds: A Virtual Agent in Live Coding’, hosted by IKLECTIK in December 2020 in London, was delivered online. The concert consisted of two solo performances by Gerard Roma and the author, followed by a Q&A panel with the two live coders together with the live coder expert in live coding and AI Iván Paz, and moderated by Sam Roig.

The second concert, ‘Different Similar Sounds: A Live Coding Evening “From Scratch”’, hosted by Phonos in April 2021 in Barcelona and organised in collaboration with TOPLAP Barcelona and l’ull cec, was delivered with an audience limitation of 15 people due to the pandemic restrictions. The concert comprised four live coders associated with TOPLAP Barcelona (Ramon Casamajó, Roger Pibernat, Iván Paz and Chigüire), who used MIRLCa ‘from scratch’, adapting the library to their particular approaches and aesthetics. The concert ended with a group improvisation ‘from scratch’ by the four performers.

The third concert, ‘Dirty Dialogues’, was organised by the Music, Technology and Innovation – Institute for Sonic Creativity (MTI²) in collaboration with l’ull cec. This concert was pre-recorded in May 2021 due to the COVID-19 restrictions and premiered online. The concert was an encounter of 11 musicians from the Dirty Electronics Ensemble led by John Richards together with Jon.Ogara and the author in a free music improvisation session (Figure 2). Apart from an online release of the performance and interview with the musicians, a live album was also released in October 2021 on the Chicago netlabel pan y rosas discos (Dirty Electronics Ensemble, Jon.Ogara and Xambó 2021).

Figure 2. A moment of the performance ‘Dirty Dialogues’ with the Dirty Electronics Ensemble, Jon.Ogara and Anna Xambó on 17 May 2021 at PACE, De Montfort University, Leicester, UK. Photo by Sam Roig.

In January 2022, and as part of the workshop at ZKM during the on-the-fly: Live Coding Hacklab weekend, the workshop attendees spent the last two hours of the workshop creating teams and preparing a showcase event, the impromptu performances (Figure 3). In total, there were four groups (one individual, two duos and one trio) together with the presentation of Naoto Hieda’s Hydra Freesound Auto,Footnote ¹⁵ a self-live-coding system. Impromptu ‘from scratch’ sessions were performed by beginners in live coding together with the expert live coders Lina Bautista, Luka Frelih, Olivia Jack, Shelly Knotts and Iván Paz. The ‘from scratch’ sessions typically consisted of playing live for 9 minutes starting from an empty screen and ending with the audience applause (Villaseñor-Ramírez and Paz Reference Villaseñor-Ramírez and Paz2020).

Figure 3. A ‘from scratch’ session with Olivia Jack (left) and the author (right) live coding with MIRLCa at on-the-fly: Live Coding Hacklab on 30 January 2022, ZKM, Karlsruhe, Germany. Photo by Antonio Roberts.

5.1. From scratch

The ‘from scratch’ live-coding technique commonly used in the live coding scenes in Mexico City and Barcelona has been defined as ‘to write code in a blank document against time’ (Villaseñor-Ramírez and Paz Reference Villaseñor-Ramírez and Paz2020: 60), and more particularly as ‘a creative technique that emphasises (visualises) real-time code writing’ (ibid.: 67).

Many of the live coders started ‘from scratch’ with empty canvases in their live-coding sessions with MIRLCa. For example, Hernani Villaseñor and Iván Paz preferred to keep simple code programs, aligned with the intention of a ‘from scratch’ performance in order ‘to find more efficient code structures and syntax (e.g., concise, compact, succinct), to, with the least amount of characters, achieve developing a complete piece’ (ibid.: 66). Needless to say, one of the key principles of the TOPLAP manifestoFootnote ¹⁶ is ‘Obscurantism is dangerous. Show us your screens.’

In both work-in-progress sessions, Hernani Villaseñor and Iván Paz started from a blank canvas and generally worked with two or three groups of two or three sounds each. The sounds were a reminder of everyday sounds, which were processed and assembled in creative ways ‘à la tape music’. At first, the sounds were typically retrieved randomly with the function random(). Then, similar sounds were searched with the function similar(). After that, the sounds’ sample rates were modified with the functions play() or autochopped() to speed the sounds up or down for a period. The following example illustrates a code excerpt of this approach using two groups of sounds:

a = MIRLCa.new

a.random

a.similar

a.play(0.5)

b = MIRLCa.new

b.random

b.similar

b.play(0.2)

b.autochopped(32, 2)

b.delay

Iris Saladino took a different approach to ‘from scratch’ in her work-in-progress session. Her approach was to combine two software systems. First, she used MIRLCRep2 to search and download sounds from Freesound using tags (e.g., ‘sunrise’, ‘traffic’, ‘sunset’) that she saved in a local folder. Second, she processed the sounds ‘from scratch’ using TidalCycles,Footnote ¹⁷ a live-coding environment designed for making patterns with code. The musical result resembled generative ambient music.

5.2. Algorithmic soundscape composition

Many of the live coders took advantage of the bespoke functions available in MIRLCa as well as the built-in functions from their live-coding environments to generate algorithmic music processes. As discussed in the previous section, the tandem random() and similar() functions are often used to retrieve sounds. Here, an initial random sound is retrieved, which is followed by a similar sound to maintain a consistent sonic space. Starting a session with a random sound expresses openness and uncertainty, as well as uniqueness, because the likeliness of two performances starting with the same sound is small. Arguably the combination of random sounds with other ways of retrieving sounds, such as similar sounds or sounds by tag, shows that building a narrative from random sounds is possible, despite this being questioned by Barry Truax when discussing using a random collage of environmental sounds for soundscape composition (Truax Reference Truax2002: 6).

SuperCollider supports algorithmic composition with a wide variety of built-in functions. Gerard Roma and Roger Pibernat started their sessions ‘from scratch’, and combined the provided algorithmic functions of MIRLCa with algorithmic functions or instructions from SuperCollider. Roger Pibernat used TDef in his performance hosted by Phonos to create time patterns that changed parts of the code at a constant rate. For example, the sample rate of a group of sounds was instructed to change every four seconds by randomly selecting an option from a list of three values: a.play([0.25, 0.5, 1]).choose. In his performance at IKLECTIK, Gerard Roma accessed the buffers of the sound samples to apply SuperCollider functions to them. Using JITLib,Footnote ¹⁸ a customised unit generator PlayRnd randomly played five buffers of sounds previously downloaded using the tag and similar functionality in MIRLCa. The following example shows the code:

p = ProxySpace.push

p.fadeTime = 10

∼x.play

∼x.source = {

0.2*mix.ar(PlayRnd.ar((1.5), 0.5, 1))!2;

}

a = MIRLCa.new

a.tag(“snow”, 5)

a.delay(10)

a.similar

a.printbuffers

The MIRLCa functions autochopped or playauto for playing randomly assigned sample rates or similarauto for automatically obtaining similar sounds from a target sound are also used frequently to give some autonomous and algorithmic behaviour to groups of sounds.

5.3. Embracing the error

Learning how to embrace errors is part of the live-coding practice. As Ramon Casamajó mentioned about his performance organised by Phonos: ‘the error is part of the game’. These include minor errors, such as the system not finding any candidate sound from a query, and major errors, such as getting an unwanted sound. As Iván Paz reflected from his concert hosted by Phonos: ‘methods such as similar can produce unpleasant surprises that you have to embrace and control the performance on-the-fly’. In turn, this can prompt free improvisation, as Chigüire commented after their performance hosted by Phonos: ‘There was some degree of unpredictability that made me feel comfortable with less preparation. I didn’t have any concrete idea in mind, I felt much freer to improvise.’

The aesthetics of imperfection in music and the arts has been discussed (Hamilton and Pearson Reference Hamilton and Pearson2020). The values of spontaneity, flaws and unfinished are highlighted as relevant to the improvisatory nature of the creative work. Shelly Knotts remarks that live coding is an error-driven coding activity (Knotts Reference Knotts, Hamilton and Pearson2020: 198) and points out how the unknown, errors and mistakes can become a sociopolitical statement in live coding: ‘By resisting strongly defined boundaries and idealized forms of musical practice based on technical accuracy, live coding remains an inclusive, open-ended and evolving practice grounded in social and creative freedom’ (ibid.: 189–90). The MIRLCa system opens up new dimensions in engaging with the unknown because the live coder cannot have entire control of the incoming sounds that emerge from the real-time queries. Instead, the live coder is prompted to sculpt the incoming new sounds.

5.4. Tailoring and DIY

In the live-coding community, each live coder has their set-up and environment. Some instances illustrated how the live coders combined MIRLCa with their usual software for live coding. Hernani Villaseñor superimposed ‘two editors: Emacs with a transparent background running MIRLCa and, underneath, Atom running HydraFootnote ¹⁹ for capturing the typing with a webcam’. Ramon Casamajó and Iris Saladino used MIRLCa/MIRLCRep2 in combination with TidalCycles. Ramon Casamajó combined downloaded sounds from the internet using MIRLCa with sounds from TidalCycles stored in a local drive: ‘I’ve approached the Tidal side more rhythmically, recovering some patterns I had written for past performances. On the MIRLCa side, I’ve looked for vocals for the first part of the piece and noisy textures for the end.’ As a result, we could experience two sonic spaces or two parallel digital musical instruments. Beyond software, Jon.Ogara explored MIRLCa combined with hardware using the Kinect sensor and Max.

5.5. The role of training the models

The role of training the models for the performance connects with the notion of specific musical contexts, what we termed ‘situated musical actions’ (Xambó et al. Reference Xambó, Roma, Roig and Solaz2021), which can help understand the training of a new model. For example, Gerard Roma trained a model ‘to favour environmental sound and reject human voice and synthetic music loops, which are common in Freesound’ (ibid.: 11). This was done in order to then use it in performance to control the resulting types of sounds: ‘In each section, the results from the tag searches were extended by similarity search and the model was used to ensure that the results remained within the desired aesthetic. This allowed a more open attitude for launching similarity queries with unknown results, encouraging improvisation’ (ibid.: 11).

Likewise, Iván Paz also agreed about the improvisational nature of using a trained model, and the curational role of the live coder, commenting that the result is ‘like knowing what types of sounds you will receive and trying to organise/colour them on-the-fly’. Iván Paz commented about the trade-off relationship between unexpected results and training accuracy: ‘There’s probably a sweet balance between surprises and consistency within the training accuracy.’ Indeed, we are only at the beginning of the possibilities that this musical perspective can bring.

Some of the live coders trained multiple models for different groups of sounds. For example, Jon.Ogara envisioned a long-term project of a diary of neural nets (snapshots of a musical biography or life journey) based on how he reacts to events using singular words. In the collaborations with more than one live coder, each live coder used their trained model/s, which is discussed in the next section.

The IML approach for the training process with MIRLCa as a live-coding session blurs the division between offline and on-the-fly training. In the on-the-fly event at ZKM, Luka Frelih performed a ‘from scratch’ training session for algorave sounds, including canvassing the audience’s opinion to help him label the sounds as ‘good’ or ‘bad’. The brief for the training was to create music that you can dance to. After obtaining a decent training accuracy, the model was tested live. The proof of the success of the model was that some people indeed danced.

5.6. Collaborative constellations

We observed five collaborative performances that were all improvisational by nature. In the five collaborations, each live coder had trained their own model previously and then performed live with their trained models.

In the concert hosted by Phonos, the four live coders ended by performing a ‘from scratch’ session altogether. The layout was configured in such a way that the live coders created an outer ring with the audience inside. Each live coder had two pairs of speakers configuring an 8-multichannel system, a laptop running MIRLCa and a projector. Ramon Casamajó mentioned here the importance of listening to each other: ‘I tried to listen to quite a bit of the rest and look for reactive or complementary sounds to theirs, trying to leave empty spaces as well.’ Although the conversation could be difficult sometimes. Chigüire thought about it ‘as a conversation at a loud bar’, and Iván Paz found that ‘it was very challenging to synchronise all the sounds.’

In the concert at MTI², we explored collaboration with a combination of analogue and digital instruments, acoustic and electronic materials, as well as live coding and DIY sound-making techniques. Both Jon.Ogara and the author performed with MIRLCa, the former as part of an acoustic and electronic ensemble while the latter performed in a live-coding style. The improvisation was an exercise of listening to each other and making a suitable call–response. For example, the improvisation started with three Dirty Electronics Ensemble members performing with different DIY circuits and found objects producing incremental tides of noise, while Jon.Ogara slowly faded in a female voice sound sample that whispered ‘come back alive’ and the author retrieved a repetitive sound sample of a printer printing.

In the showcase at ZKM, there were two duos and one trio who performed ‘from scratch’. Each live coder had a projected screen and could connect to a mixing desk with stereo output. The music ranged from algorave beats to soundscape, to glitch music, with some contrasting sounds that were handled as they appeared in the scene. In the ensembles, there were also a combination of expert live coders and beginners. For beginners, working with sound samples seemed like a suitable low-entry access to start with live coding, because they could refer to familiar sounds, apart from sharing the performance space with experts seemed to be an optimal learning scenario.

Offering a performer-only audio output (via headphones) would be an option to allow the live coder to test the new incoming sound before launching it. Although this feature has been explored in collaborative musical interfaces (Fencott and Bryan-Kinns Reference Fencott and Bryan-Kinns2012), it would not favour the flow of process music and the surprise factor brought by MIRLCa, where the live coder shapes the new incoming sounds that emerge unexpectedly. This connects with the remix culture already anticipated with dub music and what it meant to dub a track, ‘as if music was modelling clay rather than copyright property’ (Toop Reference Toop1995: 118). The tension between control and surprise seems to work well with the MIRLCa system, which promotes freedom and openness commonly found in music improvisation.