The DaCaRyH Project: Bringing Calypso into the Data Age
The DaCaRyH project—short for “Data science for the study of calypso-rhythm through history”—brought together researchers from very different worlds: ethnomusicologists, music signal processing engineers, music archivists, software developers, and a composer. Funded by agencies in France and the UK, the collaboration ran from February 2016 to November 2017. Rather than simply imposing off-the-shelf computational tools on music scholars, the team set out to let ethnomusicological questions guide the design of computational methods. The resulting insights, the researchers hoped, would enrich both the study of music and the development of intelligent analysis systems.
DaCaRyH had three main goals. The ethnomusicological aim was to bring data science and music information retrieval methods into archival and research practices. The computational goal was to ensure that real-world use cases from ethnomusicology shaped the design of those methods. Finally, a creative strand sought to explore musical style by analysing a dataset across time, and then turn the extracted features into an invented new style.
The musical tradition at the project’s heart was steelband calypso, particularly the Panorama competition held annually in Trinidad and Tobago. Created shortly after the nation gained independence in 1962, Panorama quickly became the most prestigious stage performance for steelbands, exerting a powerful influence on the development of the music.
Background: The big-data debate in music studies
The DaCaRyH project emerged against a backdrop of lively debate about data science, “big data,” and the computational study of culture. Several studies had claimed to discover sweeping historical patterns in music by crunching large datasets. Yet musicologists often questioned the methods and assumptions behind these studies. Some argued that computational analyses treated music superficially, and that their claims to objectivity could mislead both scholars and the public.
Too often, musicologists were not invited to the table when big-data approaches were applied to their own field. The national project “Transforming Musicology” began to change that, and the musicologist David Huron even called it a “moral imperative” for new researchers to become fluent in data science. DaCaRyH took a different starting point: rather than asking which ethnomusicological questions existing tools might answer, the team started with the questions themselves and sought to create bespoke tools to address them.
Groundwork: Building the foundations
Steelband's calypso and the Panorama recordings
Calypso describes a family of related styles originating in Trinidad and Tobago. In its most familiar form, it is a song of social commentary with a distinct rhythmic accompaniment in the Western tonal system. The calypso beat can also exist as a polyrhythmic accompaniment independent of any song. Steelband music evolved from this tradition through a series of dramatic innovations—from drums to bamboo stamping tubes, and eventually to instruments made from metal materials.
The adoption of metal in the 1940s changed everything. Players began to differentiate several pitches on a single metal surface, launching the era of steelpans—melodic idiophones fashioned from 55-gallon oil drums. A music style had inspired an entirely new family of instruments, known in the US as “steeldrums.” Steelpan making later became a highly specialised craft, with makers sinking, shaping, cutting, and tun- ing the drums. Modern tuners often perform “harmonic tuning,” shaping overtones to follow the harmonic series, which significantly affects the instrument’s tone.
The Panorama competition emerged in 1963, just before carnival. Bands must arrange a calypso, producing a distinctive style within the calypso family. Arrangers select a song and cast it into a structure resembling sonata form: an introduction, the calypso’s verse and chorus, variations with modulations, a minor-mode section, a climax called the “jam,” another exposition of the theme, and a coda.
Steelband music offers a rare chance to observe a style’s evolution almost from its beginning. The steelpan became Trinidad and Tobago’s national instrument in 1992, and the Panorama competition remains a rich, consistent source of recordings for study.
Session music transcriptions
The project’s creative strand envisioned turning features extracted from recorded steelband music into an entirely new style. But extracting useful features from noisy field recordings proved extremely challenging. As a proof of concept, the team turned to a different kind of data: transcriptions of traditional “session” music from the UK, Ireland, France, and Cajun traditions. The website thesession.org offers thousands of such transcriptions in ABC notation.
One example transcription from that collection might begin like this:
T: Off To California
R: hornpipe
M: 4/4
K: Gmaj
|:GFGB AGED|GBdg...etc.
The fields indicate title, dance type, meter, key, and then the notes. These transcriptions mainly serve as memory aids for musicians who learn by ear. The crowd-sourced dataset contains more than 23,000 transcriptions of melodies, many of them different versions of the same tune. Its size makes it ideal for modelling with neural networks.
CREM and Telemeta
The audio archives of the CNRS Musée de l’Homme gather field recordings of music from around the world, ranging from 1900 to the present. They are among the most important collections in Europe by quality, quantity, and diversity. The Centre de Recherche en Ethnomusicologie (CREM) manages this intangible heritage, but maintaining such archives, indexing content, and providing access is a constant challenge.
Since 2007, CREM has collaborated with the Laboratory of Musical Acoustics to develop Telemeta, an innovative open-source web platform now online since 2011. Telemeta provides streaming access to more than 43,000 recordings and their metadata, available online for research and experimentation. The project also explored integrating MIR tools to power a semantic search engine. Analysis tools now help CREM staff index and segment audio, using features like speech and singing detection, monophonic or polyphonic identification, and temporal annotation markers.
Ethnomusicological application: A case study
One central thread of DaCaRyH was an interdisciplinary case study combining traditional and computational methods to study Trinidad steelband music in a collection of Panorama recordings spanning over 50 years. This work identified several facts and trends concerning the music of Trinidadian steelbands.
Various hypotheses that were formulated remained unaddressed using traditional methods. Therefore, we approached these ethnomusicological questions through the computational lens of MIR methods (Schedl et al., 2014), enabling large-scale quantitative studies. Specifically, we examined trends in tempo, tuning, and dynamic range over fifty years of the Panorama competition. Our investigation focused on three research questions:
1. Does the tempo of winning Panorama performances tend to increase over time? 2. Do arrangers employ progressively wider dynamic ranges? 3. Has there been a shift in steelbands’ sonic characteristics?
We assembled a dataset of recordings from steelband calypsos and socas performed by the winners (first, second, and third places) of each yearly Panorama competition between 1963 and 2015 (the year 1979 is excepted due to a boycott). This collection comprises 93 recordings, typically each 8–10 minutes in duration, amounting to about 14 hours of audio. All digital recordings were sourced from the “CNRS - Musée de l’Homme” sound archives, accessible via the Telemeta platform (Fillon et al., 2014), as described in Section 3.3. The musical data includes both field and published recordings — a mix of digitized analog captures and born-digital recordings.
Details of this work are presented in Quinton et al. (2017). To summarize, we automatically extracted features such as tempo, loudness, and tuning frequency from each audio recording, offering quantitative measurements to address our research questions. Not all extracted MIR descriptors proved equally reliable, and some did not provide sufficiently robust evidence for drawing conclusions. Where quantitative measurements were conclusive, our findings align with what appears in ethnomusicological literature.
For example, Figure 1 displays the estimated tempo of each recording in our collection across competition years. We observed that certain estimates fall below 110 bpm or above 140 bpm — tempi unusual for this music. Manual inspection revealed that most of these result from errors in the automatic tempo estimation procedure. Nonetheless, in the great majority of cases, the automatically extracted tempo matches human perception well and can therefore be considered reliable.
Tempo (bpm)
From Quinton et al. (2017), the estimated tempo of each recording in our collection.
On the other hand, for dynamic range and tuning, the results obtained from MIR tools on this noisy dataset are far less reliable and thus do not provide enough evidence to support conclusions for our related research questions. This unreliability stems partly from the recording conditions: these are live events, often recorded from positions closer to the audience than to the musicians. In many recordings, audience noise carries greater acoustic power than the music itself. While the human auditory system is remarkably effective at separating such sources, current MIR tools are not. Consequently, the numbers produced by automatic estimation are not musically relevant.
Music recording corpora used in ethnomusicological research are typically either ethnographic recordings or field recordings (both apply in our case), produced under highly heterogeneous, mostly uncontrolled conditions. It is often impossible to assess how these conditions affect the musical recordings’ properties. Our study clearly demonstrates that some MIR tools are more impacted than others by audio quality. Some estimates from MIR tools applied to such corpora may have only limited reliability, while others may be trustworthy. As we have emphasized, results of computational analyses on such datasets must be interpreted cautiously when no means of evaluating feature estimation reliability exist. Additionally, our study highlights that such datasets pose considerable challenges for designing MIR tools robust against uncontrolled recording conditions.
5 Creative application
In Sturm et al. (2016) and Sturm and Ben-Tal (2017), we described applying machine learning to the large folk melody collection discussed in Section 3.2, naming the software “folk-rnn.” The method used — recurrent neural networks (RNNs, Hochreiter and Schmidhuber, 1997) — uses existing context to predict the next step in a sequence. Our initial version applied this directly to the characters of ABC notation, while the second version operates on tokens. To illustrate the difference, consider the common-time meter representation M:4/4 — in the first version, it appears as five characters: M, :, 4, /, 4. In the second version, it is treated as a single symbol or token. Similarly, a bar repeat sign :| becomes two characters but only one token. These examples show that the : character means different things in ABC notation. While an RNN can learn to account for such contextual differences, a token-based representation better captures the musical structure encoded in transcriptions.
After training, the resulting model generates new transcriptions that share many characteristics of the training set. As our evaluation shows (Sturm and Ben-Tal, 2017), the model encodes compositionally useful aspects of style, including correctly counting bars, basic phrase structure, repetition and variation of patterns, and cadence points (though the cadences themselves are not consistently convincing). Moreover, experienced performers working within this tradition had no difficulty identifying good tunes among the large collection of produced transcriptions.
Musicians performed some of these tunes alongside traditional pieces in several concerts and workshops we organized, as well as at a pub session in London. One performer commented that the model produced interesting, stylistically appropriate patterns he had not encountered before. This can be interpreted as machine learning augmenting this musical form — enabling musicians to explore new corners of musical space where plausible session tunes reside. At the same time, we must acknowledge that some might see this as a distortion of the tradition.
We also explored the creative potential of this approach beyond its original domain. We asked two musicians unaffiliated with folk music if they could find transcriptions to play in a concert. Both John Hughes (double bass) and Torbjörn Hultmark (trumpet/soprano trombone) are experienced improvisers who regularly perform in various contexts — from orchestral works to jazz and free improvisation. Neither plays instruments typical in session music. Yet both curated and adapted transcriptions from our published collection and made them work.
This achievement primarily reflects the musicality and creativity of performing musicians. But it also points to the cocreative potential of such AI methods in music. To explore this further, we developed a web-based interface to our model at folkrnn.org, allowing users to generate transcriptions, curate from outputs, or modify initial parameters (such as meter, mode, or opening notes) to explore the range of tunes the model can produce. Users will eventually be able to archive tunes they create using an online repository. There, they can participate in forum discussions, browse and “like” tunes contributed by others, and hopefully exchange feedback on the tool’s potential for music creation. Our goal is to learn from how others use the model creatively, so we can develop it further as a composition assistance tool. We plan to hold a composition competition — open to all but aimed at students — with the winning piece performed at a London concert in October 2018.
Both Sturm and Ben-Tal used the folk-rnn model to compose new works (Sturm et al., 2018). Sturm created several compositions based on selected system outputs, which he arranged electronically or acoustically, and also worked interactively — seeding the model with an initial sequence and curating the resulting outputs. Ben-Tal used the system to produce precompositional material that was substantially edited and adapted during the composition process. Generation was also interactive, fine-tuning parameters and seed sequences to steer outputs away from the model’s core style to better fit Ben-Tal’s compositional idiom. This revealed that our assumptions about what musical learning occurred during training were mistaken. When starting generation from a sequence that deviated from the training style — for example, non-modal patterns — the continuation often lacked the correct number of beats per bar. The model could not apply repetition and variation to unfamiliar patterns, resulting in noodling that was often musically unconnected to the initial material. In other words, the model’s ability to encode relevant features is highly circumscribed and not really musical — at least not in human terms. Therefore, our creative research strand brought to light aspects hidden until we pushed the model beyond simple statistical validation.
6 Lessons learned
6.1 Differences in research practices between engineering and ethnomusicology
Collaboration between MIR — rooted in engineering and computer science — and ethnomusicology, related to anthropology and musicology, reveals significant differences in scientific cultures and practices. These distinctions appear not just in research methods but also in terminology and practices. MIR researchers follow engineering standards centered on benchmarks and statistical significance, while ethnomusicologists gather information through participant observation, producing rich, multifaceted data that is often not reducible to numbers. Thus, cross-disciplinary communication is crucial to bridge this gap, understand each approach’s potential, and become familiar with each other’s terminology, concepts, and aims. This communication is made more complex by differing disciplinary training, yet it is possible due to a shared interest — the music itself.
Some scope is shared, but each discipline is driven by concerns outside this common ground. For instance, in the case study discussed, both ethnomusicology and MIR focus on timbre, loudness, and tempo. Yet in ethnomusicological literature, these are discussed in the context of rivalry (Dudley, 2008; Helmlinger, 2011). Rivalry, a relational characteristic observed in Trinidad and Tobago steelbands, is what ethnomusicologists have emphasized: beyond musical sound, this feature helps understand extramusical behaviors as well as musical evolution. Tempo, timbre, and loudness matter to ethnomusicologists for what they express about musical culture — they require additional interpretation. As Rouget puts it, “la musique, c’est toujours plus que la musique” (Rouget, 1995) — “music is always more than just the music.” Thus, ethnolinguistic focus extends beyond that of MIR. At the same time, MIR’s scope is wider in aiming to answer questions about music in general, not just one specific culture. Reconciling these different academic cultures requires learning to formulation (or reformulate) research questions in ways that incorporate the other field’s perspectives.
6.2 Difficulty in forming questions compatible with tools
Once the scope of both disciplines is narrowed to common ground, we must identify research questions that fit this space. Multidisciplinary collaboration is concretized by different aspects: research questions, methods, and bibliography. Methods and bibliographic work can simply be added together and naturally mix by addition. Research questions, however, are less easily pluridisciplinary — their orientation depends on one’s scientific background. Goody (1977) demonstrates that characteristics of the medium deeply affect its content. In our case, the tools are not simply consequences of research questions; they also shape those questions. Computational ethnomusicology promises that new tools open new research avenues. For the ethnomusicology partner, a key challenge without a background in engineering has been gaining sufficient understanding of the possibilities and limitations offered by MIR tools — enough to suggest ethnomusicology questions that are well adapted to the tools. Cross-readings and extensive exchanges are, of course, essential to overcome these obstacles: ethnomusicologists must internalize this methodological arsenal to effectively formulate anthropological questions. Similarly, MIR researchers would benefit from a deeper understanding of ethnomusicological knowledge context to move beyond extracted features, datasets, and ground truths.
6.3 Validation of computational tools
Computational tools designed for and applied to music research must be presented with a fair evaluation of their reliability or accuracy for the task in a specific context. In our case, manual validation shows that the tempo estimation tool is quite reliable. Most calculated values match what we identify by listening — the MIR tool echoes human interpretation and can therefore be considered reliable for this task. However, double-checking each MIR-produced result manually barely advances effort over manual tempo estimation itself. For these tools to be genuinely useful to non-MIR researchers, they need a “health warning.” We require a way to estimate a tool’s fitness for the items it is unleashed upon. Our tuning frequency estimation work illustrates this problem. Initially, calculations from MIR tools supported our hypothesis; further investigation, informed by understanding the inner workings of these methods, revealed these numbers to be artifacts and therefore unreliable.
Either all computational ethnomusicology will depend on a MIR “technician,” or the tools themselves must provide the user with more contextual information.
7 Conclusion
Our project was funded by a Franco-British initiative recognizing the importance of collaboration across disciplines and cultures to foster research. Our two-year experience shows that challenges with such work are often subtler and less visible than commonly recognized. Disciplines shape not only terminology and methodology but also career trajectories, pressures, academic cultures, and frames of reference. The two strands of work summarized here illustrate two approaches to collaborative effort.
The work described in Section 4 fits an interdisciplinary model where researchers from two domains bring their individual expertise to bear on a problem. It relies on a limited dataset with empirical observations from field researchers. Their knowledge of this music enables hypotheses addressable by MIR tools. Meanwhile, engineering expertise is needed not just to apply tools to the dataset but also to evaluate the results — understanding feature extraction mechanisms is essential for seeing how they interact with collection items.
Our machine learning work with folk music (Section 5) exemplifies disciplines that are much more deeply interwoven. This outcome stems from a collaboration that effectively began about two years before the DaCaRyH project’s start. Extensive discussions — most not tied to an immediate goal like writing a paper or applying for a grant — allowed us to discover shared interests and areas where expertise complements fruitfully. The longer duration also let us understand enough of each other’s methods and concerns to better translate our ideas across different academic cultures.
This suggests a more patient approach is necessary for interdisciplinary collaboration to thrive. That applies to the researchers undertaking such work, the bodies funding it, and the academic institutions setting expectations for research productivity. Simultaneously, we should expand training in research methods within our PhD programs. Computational methods — including both MIR and machine learning — have a role to play in future music research. Unless we who research music and know it intimately engage with these tools, others will. We must embed MIR tutorials in ethnomusicology training. We need to understand the potential and limitations of these tools as research instruments. We should showcase instances where computation serves musical purposes and point out cases where research gets music wrong.
At the same time, the MIR research community should more carefully consider the implicit assumptions underlying the mechanisms they develop and include those when releasing tools. MIR researchers should also engage more with musicologists, ethnomusicologists, and music theorists. Though MIR research will continue to be shaped primarily by commercial music demands, music departments hold valuable knowledge relevant to MIR concerns that remains largely unexplored.
Acknowledgements
Florabelle Spielmann, Ghislaine Glasson Deschaumes, Andrew Thompson
References
Aho WR (1987) Steel band music in Trinidad and Tobago: the creation of a people’s music. Latin American Music Review/Revista de Música Latinoamericana 8(1):26–58
Birth KK (2008) Bacchanalian sentiments; musical experiences and political counterpoints in Trinidad. Duke University Press
Cowley J (1998) Carnival, canboulay and calypso: traditions in the making. Cambridge University Press
Dudley S (2002) The Steelband “Own Tune”: Nationalism, Festivity, and Musical Strategies in Trinidad’s Panorama Competition. Black Music Research Journal pp 13–36
Dudley S (2008) Music from Behind the Bridge: Steelband Spirit and Politics in Trinidad and Tobago (illustrated edition). Oxford University Press Inc.
Fillon T, Pellerin G, Brossier P, Simonnot J, La Défense N (2014) An open web audio platform for ethnomusicological sound archives management and automatic analysis. In: Workshop on Folk Music Analysis (FMA2014), p 36
Fink R (2013) Big (bad) data (editorial). Musicology Now (online)
Goody J (1977) The Domestication of the Savage Mind. Cambridge University Press
Guilbault J (2007) Governing sound: The cultural politics of Trinidad’s carnival musics. University of Chicago Press
Helmlinger A (2011) La virtuosité comme arme de guerre psychologique. Ateliers d’anthropologie 35
Helmlinger A (2012) Pan Jumbie. Mémoire sociale et musicale dans les steelbands (Trinidad et Tobago). Société d’ethnologie
Hochreiter S, Schmidhuber J (1997) Long Short-Term Memory. Neural Computation 9(8):1735–1780
Huron D (2013) On the virtuous and the vexatious in an age of big data. Music Perception 31(1):4–9
van Koningsbruggen PH (1997) Trinidad Carnival: A Quest of National Identity. Caribbean
Marsden A (2015) Music similarity. In: Presentation at Music Similarity: Concepts, Cognition and Computation. URL: http://www.lorentzcenter.nl/lc/web/2015/669/presentations/Marsden.pptx
Mauch M, MacCallum RM, Levy M, Leroi AM (2015) The evolution of popular music: USA 1960–2010. Royal Society Open Science 2(5). DOI: 10.1098/rsos.150081
Quinton E, Spielmann F, Sturm BL (2017) Computational ethnomusicology for exploring trends in Trinidad steelband music through history. In: Proc. CMMR
Regis L (1999) The political calypso: true opposition in Trinidad and Tobago, 1962–1987. University Press of Florida
Rouget G (1995) Ethnomusicologie d’un rituel: la représentation, ou de Vélasquez à Francis Bacon. L’Homme 35(133):77–97
Schedl M, Gomez E, Urbano J (2014) Music information retrieval: Recent developments and applications. Foundations and Trends in Information Retrieval 8(2‑3):127–261
Schellenberg EG, von Scheve C (2012) Emotional cues in American popular music: Five decades of the top 40. Psychology of Aesthetics, Creativity, and the Arts 6(3):196–203
Serra J, Corral A, Boguna M, Haro M, Arcos JL (2012) Measuring the evolution of contemporary western popular music. Sci Rep 2. URL: http://dx.doi.org/10.1038/srep00521
Spielmann F, Helmlinger A, Simonnot J, Fillon T, Pellerin G, Sturm BL, Ben-Tal O, Quinton E (2017) Zoom arrière: L’ethnomusicologie à l’ère du Big Data. Cahiers d’ethnomusicologie 30:9–28
Stuempfle S (1995) The steelband movement: The forging of a national art in Trinidad and Tobago. University of Pennsylvania Press
Sturm BL (2014a) The state of the art ten years after a state of the art: Future research in music information retrieval. J New Music Research 43(2):147–172
Sturm BL (2014b) A survey of evaluation in music genre recognition. In: Nürnberger A, Stober S, Larsen B, Detyniecki M (eds) Adaptive Multimedia Retrieval: Semantics, Context, and Adaptation, vol LNCS 8382, pp 29–66
Sturm BL, Ben-Tal O (2017) Taking the models back to music practice: Evaluating generative transcription models built using deep learning. J Creative Music Systems 2(1)
Sturm BL, Santos JF, Ben-Tal O, Korshunova I (2016) Music transcription modelling and composition using deep learning. In: Proc. 1st Conference on Computer Simulation of Musical Creativity, Huddersfield, UK. URL: https://drive.google.com/file/d/0B1OooSxEtl0FcTBiOGdvSTBmWnc/view
Sturm BL, Ben-Tal O, Monaghan U, Collins N, Herremans D, Chew E, Hadjeres G, Deruty E, Pachet F (2018) Machine learning research that matters for music creation: A case study. J New Music Research (submitted)
Wallmark Z (2013) Big data and musicology: New methods, new questions. Tech. rep., American Musicological Society National Meeting, Pittsburgh, PA
Weiß C, Mauch M, Dixon S, Müller M (2018) Investigating style evolution of western classical music: A computational approach. Musicae Scientiae
Weyde T, Cottrell S, Dykes J, Benetos E, Wolff D, Tidhar D, Kachkaev A, Plumbley M, Dixon S, Barthet M, Gold N, Abdallah S, Alancar-Brayner A, Mahey M, Tovell A (2014) Big data for musicology. In: Proceedings of the 1st International Workshop on Digital Libraries for Musicology, ACM, New York, NY, USA, DLfM ’14, pp 1–3. DOI: 10.1145/2660168.2660187. URL: http://doi.acm.org/10.1145/2660168.2660187