Improvisation and Play in New Media, Games and Experimental Sound Practices
DOI: https://doi.org/10.59350/657xk-rqj17
Editors’ Note: Welcome to Sounding Out!‘s December forum entitled “Sound, Improvisation and New Media Art.” This series explores the nature of improvisation and its relationship to appropriative play cultures within new media art and contemporary sound practice. This series will engage directly with practitioners, who either deploy or facilitate play and improvisation through their work in sonic new media cultures.
The first essay in this series draws from a constellation of disciplinary perspectives that investigate these critical valences, and posits both play and improvisation as critical interventions which can expose, critique and interrupt the proprietary techniques and strategies of contemporary consumer media technologies.
— Guest Editors Skot Deeming and Martin Zeilinger
—
As media art scholars and curators of the annual Vector Game Art & New Media Festival, we are particularly interested in the conceptual convergence between critical approaches to play, on the one hand, and to improvisation, on the other. In this short essay, we therefore ask how improvisational sound-based media practices that use technologies, aesthetics, artifacts, and expressive modes of play may challenge the assimilative advances of corporate capital, carving out sites at which its logic may be opposed and confounded. We contend that there is a critical edge to the cultural milieu of playful computation through which practitioners (whether as artists or simply as players) can recuperate play from the mainstreamed sound cultures of digital capital through improvisation-based approaches. It is in this regard that play, as a participative act, rather than an interactive mode, can become a critical site by which to understand artistic interventions through contemporary technologies. This essay provokes a dialogue between theoretical avenues in popular music studies and game studies in order to show how the often-conflated practices of improvisation and play have the potential to challenge the homogenous and repetitive logic of the technological sector and the music industry.

We must turn first to popular music studies where the critical valence of improvisation is a relatively well-established concept. Jazz improvisation in particular has been widely recognized as a powerful mode of political expression. Charles Hersch has argued that techniques of polyphony allow for jazz to reconcile the seemingly contradictory dynamics of individual expression and collectivity, thus allowing for the articulation of black solidarity through musical interventions (97-8). Because jazz is importantly built on appropriation-based creative models, it yields fluid, fleeting, spontaneous results that are difficult to assimilate into the entertainment industry’s property-based circuits. Digital play exhibits a similar potential of challenging the logic of capital; it harbors a powerful creative potential even in heavily commodified contexts. Play allows us (indeed, requires us) to become active in shaping the improvisational, the field of simultaneously critical, creative and performative contexts of expression.

Like jazz musicians, game players embody and enact a powerful sense of potentially uncommodifiable agency. Both improvisation and play function similarly, and highlight how contexts of music-making and game-playing activate individuals as critical, expressive agents. Players guide themselves through games based on subjective, rather than imposed, motivations. We can see this emerging in gaming cultures through the rise of player-based phenomena such ‘speedrunning,’ a practice that exploits glitches and faults in a game’s architecture in order to reach the end of a game in the shortest possible time. These niche and ephemeral gaming practices highlight the degree to which practices of improvisation in play may constitute a powerful way of pushing ‘through’ capital–both literally, by accessing games through glitches in their proprietary fabric, and figuratively, if we understand such forms of play as ways to resist the capitalist ideologies embedded within the structures of games (such as the accumulation of wealth, power, or points). Having established here how game-play can act as a form of critical improvisation, this essay now intends to show how these practices of playful improvisation are manifest in the technologies, aesthetics, and the cultural contexts of sound-based art.

Where can we locate practices of improvisation in the context of sound art? For us, sound art is improvisational both in form and practice. First, the emerging aesthetic dimensions of the medium are fundamentally related to the participatory practices of making that have been used to develop the instruments of sound art, such as circuit bending, hacking, tinkering. Here, the playful and open-ended practices of making are a form of improvisation, a modulation within the typical technological frames in which musical composition is typically embedded. Second, when gaming devices are turned into musical instruments (or vice versa), users are called upon to rethink improvisational practice as game-play, and experiment with deploying play as composition. In this sense, game players become sound artists, with sound technologies extrapolating and improvising new sounds from the habit of their play. Improvisation and play thus allow emerging communities of practice to counteract the corporate ‘black boxing’ of entertainment technologies and the creation of closed systems of interaction and participation (such as many mainstream video games) which assimilate user activities and foreclose improvisational play.
Such approaches to sound art are represented by many creative practitioners and take a wide variety of forms, including the appropriation and creative redeployment of participative game environments and game engines. Examples include the creation of interactive sound environments and playable instruments within mainstream participative game ecologies such as the Little Big Planet franchise, the hacking and ‘bending’ of proprietary technologies that invoke the vast universe of game culture nostalgically or in a parodic mode (including handheld gaming systems such as the Nintendo Gameboy, which spawned an entire electronic music genre known as chipmusic), or the making of non-proprietary infrastructures for creative expression that encourage open, participative play, such as open source live coding environments including Sonic Pi, originally developed for the Raspberry Pi platform. Following such approaches, game controllers, digital objects, avatars, virtual architectures, and digital artifacts can thus become tools of composition and/or improvisation. Similarly, we might also think of the improvisational frameworks constructed by sound practitioners as ‘playgrounds,’ which allow players to become performers who explore the technological, political, and aesthetic limits of play. But emergent communities of creative practitioners frequently adopt such close systems and technologies subversively, for example by reverse-engineering them and developing alternative uses. In doing so, they alter the contexts in which these technologies are employed, and open them up for use as musical instruments and infrastructure for creative and critical expression. Technological artifacts with unique user communities can thus be conceptualized as frameworks for musical expression. Vintage gaming consoles, games engines, and computational algorithms have all been reconstructed as new forms of musical creation and interaction through participatory creative communities.
In this short essay, we have approached ‘play’ and ‘improvisation’ as related concepts that can serve to recuperate and empower critical perspectives within the game and sound cultures of capital. As we have suggested, playful approaches to improvisational sound-work and perspectives on play that are rooted in musical traditions can be productively discussed both in sound studies contexts and within game studies paradigms. How, then, do sound practitioners appropriate game technology? How do the socio-political dimensions of digital game culture inform musical practice? With the help of our contributors to this series, we explore how critical perspectives on musical improvisation aid our understanding of the cultural and socio-political significance of play and, likewise, how critical theories of play may broaden our understanding of improvisational sound practices. Through building, tinkering, problem solving, and improvising on the technological back end, creative practitioners playfully create and reshape tools, devices, and techniques of improvisational practice that often embody important ethics of openness, dynamism, fluidity, and sharing, posited against the closed culture of proprietary consumer technology by which we are commonly surrounded. Between live coding, modular synthesis, creative computing, hardware hacking, and appropriation-based interactive performances, the contributors to this series operate in many creative modes that oscillate between improvisation and play. What all of them share is a sense that both play and improvisation enable creative expression outside of prescribed, commodified circuits of media consumption. Improvisation and play can yield wonderful aesthetic experiments and experiences, they can provide critical, social commentary, and they can sharpen our sense of the impact that proprietary technology has on our digital cultural landscape. What we are most interested in are moments when these ambitions converge, when improvisation – drawing on techniques or technologies of play and operating in modes that are associated with play – becomes critical.
—
All images used with permission from Vector Game Art & New Media Festival.
Featured image: ”Blip Fest 2011 @ Eyebeam, Day 2” by Wikimedia Commons user Lucius Kwok CC BY-SA.
—
Skot Deeming is an interdisciplinary artist, curator and scholar, whose work spans the spectrum of new media art practice from broadcast media to computational art, experimental videogames, and game art. Drawing on a wealth of practical experience and theoretical knowledge, skot’s practice focuses on new media history and historiography, and DIY technology cultures. This work articulates itself through critical research and writing, exhibitions, installations, and performances. With Martin, Skot is the co-curator of Vector Game Art & New Media Festival., in Toronto, Canada, and the artistic director of the Drone Island Sound Art Festival in Yellowknife, NT. He currently resides in Montreal where he is a doctoral student in the Individualized Program at Concordia University, and a member of the TAG Research Centre, where he investigates critical histories of computational and new media art practices. http://www.mrghosty.com/
Martin Zeilinger (PhD) is a new media researcher, practitioner, and curator whose work focuses on the intersection of digital media art, activism, and intellectual property issues. He is a Lecturer in Media at Anglia Ruskin University, Cambridge/UK, as well as an Associate Researcher at the OCADU Data Materialization Studio. He is co-editor of ‘Dynamic Fair Dealing’ (Univ. of Toronto Press 2014), which develops interdisciplinary perspectives on cultural ownership issues in new media contexts; recent peer-reviewed essays have appeared in the Computer Music Journal, the International Assoc. for the Study of Popular Music Journal, and in Sampling Media (Oxford UP 2014). With Skot, Martin is co-curator of the Toronto-based Vector Game Art & New Media Festival. http://marjz.net/
—

REWIND! . . .If you liked this post, you may also dig:
Playing with Bits, Pieces, and Lightning Bolts: An Interview with Sound Artist Andrea Perkins — Maile Colbert
Digital Analogies: Techniques of Sonic Play — Roger Moseley
Experiments in Aural Resistance: Nordic Role-Playing, Community, and Sound — Aaron Trammell
—
Machinic Ballads: Alan Lomax’s Global Jukebox and the Categorization of Sound Culture
DOI: https://doi.org/10.59350/frax9-v2w90
Today, SO! continues its series reconsidering the life and work of Alan Lomax in his centenary year, edited by Tanya Clement of The University of Texas at Austin. We started out with Mark Davidson‘s reflections on what it means to raise questions about the politics behind Lomax’s efforts to record and collect folk music, and continued a few weeks later with Parker Fishel‘s consideration of Lomax’s famous “Southern Journey” and how it has been appropriated by musicians more recently.
With Clement’s own article below, the series begins to rethink Lomax as a touchstone in current and continuing drives to collect, measure and compute sonic cultures, something that seems hot all of a sudden (see, for instance, coverage of recent digital analysis of trends in pop music at Queen Mary University of London). In her thoughtful, illuminating and inspiring article below, Clement challenges us to consider the politics behind these efforts to search, retrieve and analyze audio, something that the case of Lomax throws into stark relief.
— Special Editor Neil Verma
—
When the Association for Cultural Equity, an organization that Alan Lomax founded in 1983, announced the release of 17,000 music tracks from Lomax’s fieldwork collections, the New York Times heralded the release as a manifestation of Lomax’s Global Jukebox project, a computational experiment for accessing and studying his vast multimedia collection of the world’s culture. The Times piece likens Lomax’s project to Pandora, which allows the listener to search for music “like” music she has already found. Lomax’s biographer, John Szwed, also makes this comparison but modifies his description by proclaiming that unlike Pandora’s recommendations which are “based on personal taste” and “tend to lead sideways . . . to production style,” Lomax’s Global Jukebox idea held the potential to point a listener to “deeper principles of cultural and musical organization” (The Man Who Recorded the World 391).
Gobsmacked by whizbang possibilities, neither the Times nor Szwed discuss the deeper principles behind Lomax’s attempt to represent culture as a global search engine. In the context of the powerful work being accomplished in the Music Information Retrieval (MIR) community and my own project (HiPSTAS) to develop software for making sound collections searchable and accessible, In this article I will argue that how we build systems for searching and retrieving and browsing cultural artifacts as data is a profoundly political act. Recognizing such politics suggests that Lomax’s Global Jukebox project serves as a cautionary tale for how social and cultural contexts — or what Donna Haraway calls our “ways of being” — are reflected in the systems we develop.

The Singer with the Song
The year that Alan Lomax was born (1915), his father John Alan Lomax published a landmark piece heralding seven new types of American ballads for study. American ballads, he argues “reveal the mode of thinking, the character of life, and the point of view, of the vigorous, red-blooded, restless Americans, who could no more live life contented shut in by four walls than could Beowulf and his clan, who sailed the seas around the coasts of Norway and Sweden” (“Some Types of American Folk-Song”, 3). Unlike any other collection of ballads, John’s “American ballad” included the ballads of the miner, the lumbermen, the inland sailor, the soldier, the railroader, “the ballads of the negro; and the ballads of the cowboy . . . [and] the songs of the down-and-out classes, — the outcast girl, the dope fiend, the convict, the jail-bird, and the tramp” (3). Governed by a laudable goal to record the songs of folk cultures at the fringes of mainstream society, the senior Lomax’s view of the communities where he would collect his songs (including jails and state farms), was complex, and can fairly be called both progressive as well as racist (Porterfield 170).
John and Alan went on seven collecting trips together between 1934 and 1936 and co-authored five books on their return. On these trips, they collected songs from people on the street in cities like New Orleans and people in the country, from both church-goers and prisoners. While John held romanticized views of the “noble” southern black man, Alan, on the other hand, indicated a more nuanced understanding of the complexities inherent to his father’s attempt to generalize patterns of “folk” for study. Alan linked “the singer with the song” and was interested in the politics behind prisoners made to sing with guns at their backs and in the cultural lives of people that were so poor in means but so rich in “beautiful harmony, with enormous volume, with total affection” (Szwed 49). While Alan maintained that he was interested in the individual’s story, John believed that “a genuine ballad has no one author. It is therefore the expression of no one mind: it is the product of the folk . . . It might have been written by any one” (“Some Types of American Folk-Song”, 1).

Taxonomies
The Global Jukebox project demonstrates an almost complete reversal in Alan’s concerns. The studies behind the Global Jukebox include Alan’s Cantometrics and Choreometrics, in which he produces taxonomies for studying song and dance and his Parlametrics project, an “experiment in metalinguistics,” which Alan and his collaborators describe as a taxonomy of “patterns of style” in speech based on dynamic changes in pitch, loudness, speed, spacing, rhythm, and timbre (“A stylistic analysis of speaking”). These taxonomies show that Alan’s early consideration for the individual performer gave way to a desire to make folk study more scientific as a cultural mapping like what his father espoused rather than what Szwed and others have seen as Alan’s concerns with the situated politics of individuals.
Alan’s Parlametric study serves as good example. Approaching delegates from the United Nations and soliciting mail-in samples from regions not covered by the U.N. volunteers, Alan and his team collected representative recordings of 114 languages. Then, in order to study the “generally neglected meta-communicational level” in these recordings, the team designed a rating system including 50 codes that (1) “described the distinctive features of each recording,” and (2) “tended to cluster the recordings into sets of similars” that Alan maintains anyone could “readily use” to record “salient differences in conversation style” (19). These clusters pointed to 14 factors that Alan and his team would use to categorize the cultures from which they received samples:
- Repetitiveness
- Timing
- Speech length
- Upglides
- Descending cadence
- Syllabification
- Drawl
- Empathy
- Space
- Dominance/Sharing
- Relaxed/Tense
- Noise
- Breathy
- Forceful
Using these factors, Alan makes some broad assertions. The association of clear syllabification” (the degree to which syllables run together) “is most strongly predicted among gardeners with domesticated animals” and “[t]he association of clear syllabification to feminine autonomy is suggested by the discovery that this mode of speaking predicts and is predicted by permissive rather than restrictive premarital sexual mores” (27). Further, “Dominance vs. Sharing of conversation space” is strongly correlated with settlement size and severity of sexual sanctions,” a statement that Alan immediately rationalizes by noting that “this relation between a more crowded social space, high sexual tension and increased rate of interaction seems to make good sense, even if it does not account for every possibility” (31).
These spurious and broad generalizations were what Lomax hoped to facilitate for all with his Global Jukebox as the access point for “the first numerical models of the full range of global cultural variation in holistic form” for “the scientist, the layman, and the student to explore, experience, and manipulate the broad universe of culture and creativity in a systematic fashion, with audio-visual illustrations at every turn of the road” (“The Global Jukebox,” 318). By leveraging his taxonomies of song, dance, and speech in the computer age, Alan could suddenly associate and differentiate cultures holistically and en masse.
Machinic Methods / Humanistic Questions
As someone who works in the liminal spaces between the humanities and technology, between cultural studies and critique and the machines that increasingly function both as access points and barriers to our cultural artifacts, I see Alan’s switch to generalizable taxonomies as par for the course in the digital age. My own >HiPSTAS project’s primary objective is to develop a virtual research environment in which users can better access and analyze spoken word collections of interest to humanists. We understand that in order for us to search digital sound artifacts, we have to create taxonomies, metadata, keywords and other generalizable frameworks that facilitate discovery.
At the same time that we are using machinic methods, however, we can still ask humanistic questions that open up rather than close down debates and dialogues. In a recent test for the HiPSTAS project, for example, we used machine learning to analyze the recordings in the UT Folklore Center Archives, which comprises 219 hours of field recordings collected by John and Alan Lomax, Américo Paredes, and Owen Wilson, among others (UT Folklore Center Archives, ca. 1928-1981, Dolph Briscoe Center for American History, University of Texas at Austin, Box 2.325/R). In our attempt to predict the presence of different sonic patterns including instrumental music, singing, and speech, the results of our analysis are noteworthy as the visualization shown in this brief movie demonstrates.
from Tanya Clement on Vimeo
Within the results, we see a visualization of how many seconds comprise each file (in blue) and how many of those seconds for each file our software has predicted the presence of instruments (green), speech (red), and song (purple). A subtle yet striking difference emerges in the comparison between the Lomax recordings (created 1926-1941), which are the oldest in the collection, and the others, which were created up until 1968. The Lomax recordings (primarily created by John Lomax) consistently contain the least amount of speech in comparison to what the other files contain.
Of course, there are a number of ways you can read these results. Given the conversation above, one could hypothesize that perhaps the Lomaxes were primarily interested in their participants’ songs rather than their stories. One could also think about it in terms of recording capabilities across time. When the Lomaxes were first recording, John Lomax writes, “The amplifier weighed more than one hundred pounds; the turntable case weighed another one hundred; two Edison batteries weighed seventy-five pounds each. The microphone, cable, the tools, etc., accounted for sufficient weight to make the total five hundred pounds. . . . In order to carry them in the car I tore out the back seat . . .” Even in 1967, forty years later, good recorders still weighed 70 pounds and required a car battery, but tapes were longer and costs were less. More tape and more time at less cost both financially and physically had a big impact on what researchers recorded. At the same time, the data shows that the later recordings are not much longer, but do seem to have more seconds of speech.
There is a danger in these kinds of machine-generated generalities. We employed taxonomies (instrumental, sung, speech) to teach the machine to categorize these patterns, but why these patterns? Are there others? Or did I choose these based on what I already wanted to say about the Lomaxes’ practices? And, I haven’t even mentioned here the subjective practices inherent to choosing algorithms for such work.
These kinds of questions require more research, and more contextualization than this aggregated data set can show. Just as the ballads that John and Alan Lomax once collected were written and sung by someone, so were the communities that Alan interpreted through his Parlametrics made up of individuals, not types. Perhaps Alan’s desire “to record the world” was just and Google, the collector, categorizer, and interface for all things on the Internet, isn’t evil. But the Global Jukebox Project serves as a cautionary tale about the politics behind the speed and efficiency that machinic methods seem to promise, a politics that needs to be far less opaque about its deeper principles and problems.
—
Tanya Clement is an Assistant Professor in the School of Information at the University of Texas at Austin. She has a PhD in English Literature and Language and an MFA in fiction. Her primary area of research is scholarly information infrastructure. She has published widely on digital humanities and digital literacies as well as scholarly editing, modernist literature, and sound studies. Her current research projects include High Performance Sound Technologies in Access and Scholarship (HiPSTAS).
—
Featured image: “Day 21 – Waveform” by Flickr user evil_mel, CC BY-NC 2.0
—

REWIND! . . .If you liked this post, you may also dig:
Revising the Future of Music Technology — Aaron Trammell
SO! Amplifies: Carleton Gholz and the Detroit Sound Conservancy — Carleton Gholz
Sounds of Science: The Mystique of Sonification — Margaret Anne Schedel
—























Recent Comments