Music Theory

Participative Contributions to Networked Music Performance


14 min read

This initial survey forms the core of a master's research project in the Graduate Program in Music at the Federal University of Minas Gerais (UFMG). The study investigates how computer music and advanced network telecommunications can serve as tools for developing interactive composition systems that operate across geographical distances. Another focus involves understanding the interaction strategies among artistic and technological teams during the production and performance of distributed interactive music. This work highlights significant contributions observed in key networked art events, including the opening of the 33rd APAN (Asia-Pacific Advanced Network), the telematic opera Climate Change Opera, and a study of articles published about the Net-Concerts.

Introduction and Theoretical Framework

Networked Music Performance (NMP) enables real-time interaction through computer networks. Participants located in different sites can perform music together as though they were in the same space (Lazzaro & Wawrzynek, 2001). This practice is also called telematic music (Oliveros et al., 2009). These interactions cover performances, rehearsals, jam-sessions, and learning situations such as master classes. Participants connect through high-quality multichannel audio and video links as well as specialized software (Alexandraki et al., 2008). NMP is not meant to replace traditional live stage performance; rather, it supports musical interaction when co-presence is impossible. It allows new forms of musical expression and interaction among audience members located in different places (Sawchuk et al., 2003). The growth of high-speed networks provides a platform that is becoming increasingly practical for real-time media applications. Building NMP and network composition systems brings about challenges that carry both technical and cultural implications (Renaud & Rebelo, 2006). Performing NMP requires knowledge from many domains: composition techniques, sound synthesis, digital signal analysis and processing, interface design, data-transmission protocols, and the topology of the network environment itself.

The term "interactive music systems" was introduced by Rowe in his 1992 book Interactive Music Systems, where he defined them as computational music systems whose behavior changes in response to musical stimuli. Interaction more broadly encompasses both the performer's actions that affect the computer's output and the computer's actions that affect the performer's results (Garnett, 2001). This parallels the communication among musicians in a traditional chamber music setting, where two or more players perform written, improvised, or mixed music (Winkler, 1999). In more complex interactive relationships, a composer can assign multiple roles to a computer: as an instrument, performer, conductor, or even composer. These roles may coexist or shift over time (Lippe, 2002, p. 24).

One of the earliest groups to experiment with musical performance over Internet networks was the League of Automatic Music Composers, formed in the late 1970s. The original members included Jim Horton, Tim Perkis, and John Bischoff. They used networked computers to exchange data and messages, influencing one another while playing. This group later became known as the Hub, which then experimented with remote collaborations between the west and east coasts of the United States. Given the limited bandwidth available at the time, the group exchanged messages and low-quality audio signals.

The Internet as a medium for producing and disseminating NMP challenges conventional notions about making and listening to music. The issues of synchronization and delay in data and audio transmission during live performances can be approached from different angles. For researcher Juan-Pablo Cáceres, the delay of audio transmission over high-speed networks can be used to create musical devices for playing with the network and within it (Cáceres & Renaud, 2008, p. 2), reframing network delay as a favorable characteristic of the medium where the musical work takes place. On the other hand, researcher Julián Jaramillo from the University of São Paulo reports that one of the main problems in NMP lies in managing the network environment:

We have dedicated great attention to solving traditional network problems (delay, connectivity, jitter), because we believe that stability in networked music environments is a crucial point to be addressed. Integrating and synchronizing audio, video, and metadata is very important, as it improves communication among musicians during performance and enables the development of complex strategies. (Arango et al., 2013, p. 2)

Researchers Alain Renaud and Pedro Rebelo argue that the relationship between latency and distance is not fundamentally different from the relationship between an instrumental sound source and the acoustic reflections in a hall. Latency is treated here as an important factor for compositional strategies and sound orchestration. This implies trying to maintain a sense of presence within the network (the virtual space) as well as in the physical performance space, guided by dialogue among the participating teams. There exists a tight relationship between remote musical practice and the technology that makes it possible, which stands as a defining feature of so-called network music (Hickmann, 2012, p. 295). This calls on researchers in the field to reconsider traditional relationships among musicians, audiences, and spaces, and to redefine conventional concepts of composition, interpretation, and the transfer of musical information among everyone involved.

Motivation

Since 2010, I have been a member of the Research Group Poéticas Tecnológicas – Corpo Audiovisual (GP Poética) at IHAC/UFBA. In this group, I have taken part in several interdisciplinary research projects in networked art. These projects have fostered reflection, investigation, and the creation of artistic works that interact with new technologies and advanced telecommunications networks. The work has also contributed to theoretical, conceptual, technical, and artistic development across disciplines. Four of those projects are listed here:

  • LABORATORIUM MAPA D2: This project brought together three artistic research groups and four technology groups. It explored possible connections among different artistic languages and aesthetic propositions using advanced telecommunications networks. The outcome was the telematic artwork FRÁGIL, presented at the DESAFIOS DE ARTE EM REDE event at the Museum of Modern Art of Rio de Janeiro in 2011, involving the cities of Salvador, Rio de Janeiro, and Fortaleza (Silva, 2012, p. 305).
  • Opening of the 33rd APAN: In 2012, I took part in the debut of the live music and telematic dance performance DQ12_DANCING_ACROSS_OCEANS in Thailand. The show involved four cities in its conception and execution: Barcelona (Spain), Salvador (Brazil), Chiang Mai (Thailand), and Daejeon (South Korea). During this presentation, I interacted over the network with Professor BonCheol Goo from the Graduate School of Culture Technology in Daejeon, Korea, and also studied a software application created by Professor Goo for synchronizing audio across the four network environments.
  • DRAMATURGIA DE UM CORPO TELE-SONORO: This was a postdoctoral research project led by Dr. Ivani Santana, coordinator of GP Poética, conducted in 2012 at the Sonic Arts Research Centre (SARC) at Queen's University Belfast, Northern Ireland. I was able to send and receive audio over the network and interact with SARC researchers Pedro Rebelo and Felipe Hickmann.
  • EMBODIED IN VARIOS DARMSTADT '58: Carried out across Mexico, Spain, and Brazil in 2013, this project addresses embodied sonority and reexamines the relationship between music and dance, sound and body. Participants included Spanish musician and programmer Alain Bauman from Konic Theatre, as well as musician and researcher Luiz Naveda from UEMG.

In these projects, I served as a scientific initiation fellow, with responsibilities that included: (1) designing the physical infrastructure, audio capture, and audio spatialization; (2) setting up and managing the virtual audio server for network routing; and (3) managing overall musical composition and global musical gesture according to the spatialization possibilities of the sound material in each involved city.

Through this work, I observed a range of strategies for constructing a networked artwork and distributed music. Among these, I turned my attention to collaborative musical processes in which the sonic result does not spring from a single creative mind but from a collaborative behavior that emerges through interaction among many participants, including artists, computer technicians, and programmers. In this context, the computer is not solely a control instrument; it also serves as a platform for social interaction. Exploring modes of working in composition and performance within advanced Internet networks is the primary motivation for this research, alongside the adaptations of traditional musical activities for the network environment, which becomes the medium for the work itself.

The focus of my master's project settled on strategies for coordinating artistic and professional relationships among teams involved in an NMP project. Different computational interfaces and different creative ideas require reflection on the reach of various tools and programming languages, taking into account how such knowledge can be adapted by a partner team contributing to a networked performance.

Research Objectives

In addressing NMP, this work aims to clarify the different interaction strategies used among artistic and technology teams during the production and performance of distributed music. Specific goals are:

  • To discuss technical, artistic, and cultural challenges in NMP production.
  • To compare compositional and technological approaches across significant NMP works by examining scores, software, and network-environment settings.
  • To identify collaborative strategies in the literature and in studied works for interaction between conventional musical instruments, real-time computer processing, and data transmission over the Internet.
  • To discover important aspects that help minimize geographical barriers to interaction among people involved in performances with remote partners over advanced Internet connections.
  • To clarify which musical situations benefit most from remote interaction offered by advanced Internet networks.

Preliminary Methodology

Following the theoretical review, the next step is to examine case studies in order to deepen understanding of the strategies composers use to handle interactivity among teams in different locations and with audiences. The chosen compositions will illustrate different approaches to creating and disseminating NMP. Potential works for this analysis include NETGRAPH by Pedro Rebelo, SUMMER SNAIL by Felipe Hickmann, PAULISTA by Rui Chaves, DISPARITY by Julián Jaramillo, and SCRATCH-SHOT by André Damião Bandeira, among others. All of these were presented in the Net-Concerts series (Arango et al., 2013) organized by the MOBILE group at ECA/USP in São Paulo and SARC in Belfast.

After this analytical phase, the research will proceed to laboratory jam-session conducted over the network, involving partners from countries such as Colombia, Chile, Argentina, Mexico, and Ireland. I will participate from Belo Horizonte, where I will coordinate the demands related to sending audio over the network. This hands-on phase will test in practice the interaction strategies between teams identified during the literature review, analysis of scores, study of software, and network configuration setups.

Relevant Contributions to NMP

The Internet evolves technically at a rapid pace, and NMP prompts a rethinking of ties between creator, performer, and audience. Remote musical activity demands a renewed study of instrumental tradition, which points to a restructuring of current ideas about liveness and online communication.

The first challenge of the previously mentioned DQ12_DANCING_ACROSS_OCEANS was the use of tools that allowed the entire work to be completed remotely. No team members had to travel to the countries involved. In less than two months, we organized the show—from network engineering to the creation and production of the artistic performance, in addition to studying and using the software Pure Data, Jacktrip, and MAX6.

Professor BonCheol Goo from South Korea created a soundscape to be synchronized with all the video streams from each city. Synchronization was achieved with a patch in MAX6 software; each city used this synchronizer patch. The process worked as follows: a "ping" signal was sent from Chiang Mai to Salvador and Barcelona; once the signal arrived, it was immediately sent back to Chiang Mai, so that the delay (response time in milliseconds) for each route could be calculated. Then, using the "metro" object in MAX6, the soundscape was triggered at the same instant in each country, taking into account each city’s individual round-trip delay. This synchronizer patch offers an efficient solution for shared remote dance and music (Howard et al., 2012).

Researchers Pedro Rebelo and Alain Renaud contributed by developing dynamic musical notation resources (Renaud & Rebelo, 2006; Renaud, 2010), applied in the Net-Concerts #2 staged between USP and SARC in 2012. These resources are called "live-score"—a continuously transforming graphical environment, displayed simultaneously at every connection node and controlled remotely by the composer or conductor. This resource provides an effective strategy for clarifying the relationships among composer/conductor and performers in a Networked Music piece. Researcher Julián Jaramillo notes:

Live-scores constitute a strategy for dealing with the simultaneity of remote events and the synchronization of performers. These graphical environments create a real-time communication instance that accommodates the composer and audience, alongside the participants. As a form of musical notation, live-scores depart from the traditional symbols used to construct scores, yet they maintain a close relationship with music. (Arango, 2014, p. 153)

In April 2014, an NMP workshop took place between the Federal University of Bahia and King's College London to run network tests in preparation for the telematic opera Climate Change Opera, scheduled for 2015. The opera will be performed simultaneously in three countries: the United Kingdom, Brazil, and South Sudan. In each country, a live audience will watch singers accompanied by a live orchestra. Singers and instrumentalists will interact musically and dramatically with artists from the other two countries, seen and heard through projection screens and sound-spatialization systems. The opera is being produced by Ben Cooper-Melchiors, a producer with Young Vic and World Stages, in partnership with various teams and professionals around the world: King's Cultural Institute, King's College in London, the Cinema Arts Network, the Sonic Arts Research Centre (SARC) at Queen's University in Belfast, the theater group Nós do Morro based in Morro do Vidigal in Rio de Janeiro, and the South African choir Heavenly Voices. For this workshop, researcher Alain Renaud, a member of the Mint Lab, was invited to coordinate the network interactions. I participated in the workshop, interacting via audio and video over the network with researchers in London.

As part of the workshop report, researcher Alain Renaud and producer Ben Cooper-Melchiors produced a document titled Golden Rules for Network Music Performance and Test Videos. This report clearly advanced the production and dissemination of NMP. Three of these golden rules stand out:

  1. #2: Keep it simple
    Start with basic audio communication using software such as JackTrip. Add layers as you progress. This is comparable to building a simple melody and adding complexity in future iterations. Think of the construction of a network music performance as a versioning system.

For software development, version control is essential so you can revert to earlier states. Document every step of the process as thoroughly as possible.

Once chosen, a configuration cannot be changed

Two design approaches must be decided at the very start, because they cannot be altered later:

  • Approach 1: All nodes feel as though they share the same room. Two or three nodes distribute their content to construct "one big space," with an audience located at one of the nodes. www.youngvic.org, http://worldstageslondon.com
  • Approach 2: Performers at each node collaborate interactively, but every node retains its own identity. Each node has its own audience, and the perception differs for each group — each space feels distinct yet interacts with others. This is how the Climate Change Opera would function with three audiences. www.mintlab.ch, www.mintlab.ch/youngvictests/docu_4wayclapping_test_vision_ukbr.mov, www.mintlab.ch/youngvictests/docu_1234_test_vision_ukbr.mov, www.mintlab.ch/youngvictests/docu_clapping_test_novision_ukbr.mov

Consider cultural differences

A network performance instantly brings together different cultures, personalities, time zones, and other factors. Participants and coordinators may quickly discover differences that lead to poor communication and frustration. These include aesthetics, language, viewpoints on coordination, lifestyles, schedules, and the speed or ability to troubleshoot. The best way to prevent a gap caused by cultural differences is to appoint a coordinator or director at each location, someone who holds the artistic mission and possesses strong people skills.

The goal of this workshop and the others that will take place in 2015 leading up to the telematic opera's actual production is to clarify the network's technical possibilities, so that the work's creative structure can be considered collaboratively, improving logistics, interaction, and consequently the opera's dramaturgy and performance for the distributed audience.

Conclusões Preliminares

The Internet is a medium undergoing rapid technical evolution, and Networked Music therefore invites reflection on rethinking the ties between creator, performer, and audience. At a distance, musical activity demands a renewed study of instrumental tradition, pointing toward a restructuring of prevailing concepts of liveness and online communication.

In the NMP works examined in the academic papers studied — (ARANGO et al., 2013), (HICKMANN, 2012), (RENAUD; REBELO, 2006), (SILVA, 2012) — we observe a transgression of the notion of audience as well as the creation of new forms of participation and collaboration among geographically distributed musicians. These works tend to emphasize collaborative aspects. This tendency arises from a shift from a centralized communication structure to a decentralized one, where collaboration between creators, performers, network technicians, programmers, and the audience is essential. “A mediação

imposta pela rede delimita um espaço onde as noções de presença, interação e colaboração adquirem nuances particulares” (HICKMANN, 2012, p. 289).