Improvisation and Play in New Media, Games and Experimental Sound Practices
Editors’ Note: Welcome to Sounding Out!‘s December forum entitled “Sound, Improvisation and New Media Art.” This series explores the nature of improvisation and its relationship to appropriative play cultures within new media art and contemporary sound practice. This series will engage directly with practitioners, who either deploy or facilitate play and improvisation through their work in sonic new media cultures.
The first essay in this series draws from a constellation of disciplinary perspectives that investigate these critical valences, and posits both play and improvisation as critical interventions which can expose, critique and interrupt the proprietary techniques and strategies of contemporary consumer media technologies.
— Guest Editors Skot Deeming and Martin Zeilinger
As media art scholars and curators of the annual Vector Game Art & New Media Festival, we are particularly interested in the conceptual convergence between critical approaches to play, on the one hand, and to improvisation, on the other. In this short essay, we therefore ask how improvisational sound-based media practices that use technologies, aesthetics, artifacts, and expressive modes of play may challenge the assimilative advances of corporate capital, carving out sites at which its logic may be opposed and confounded. We contend that there is a critical edge to the cultural milieu of playful computation through which practitioners (whether as artists or simply as players) can recuperate play from the mainstreamed sound cultures of digital capital through improvisation-based approaches. It is in this regard that play, as a participative act, rather than an interactive mode, can become a critical site by which to understand artistic interventions through contemporary technologies. This essay provokes a dialogue between theoretical avenues in popular music studies and game studies in order to show how the often-conflated practices of improvisation and play have the potential to challenge the homogenous and repetitive logic of the technological sector and the music industry.
We must turn first to popular music studies where the critical valence of improvisation is a relatively well-established concept. Jazz improvisation in particular has been widely recognized as a powerful mode of political expression. Charles Hersch has argued that techniques of polyphony allow for jazz to reconcile the seemingly contradictory dynamics of individual expression and collectivity, thus allowing for the articulation of black solidarity through musical interventions (97-8). Because jazz is importantly built on appropriation-based creative models, it yields fluid, fleeting, spontaneous results that are difficult to assimilate into the entertainment industry’s property-based circuits. Digital play exhibits a similar potential of challenging the logic of capital; it harbors a powerful creative potential even in heavily commodified contexts. Play allows us (indeed, requires us) to become active in shaping the improvisational, the field of simultaneously critical, creative and performative contexts of expression.
Like jazz musicians, game players embody and enact a powerful sense of potentially uncommodifiable agency. Both improvisation and play function similarly, and highlight how contexts of music-making and game-playing activate individuals as critical, expressive agents. Players guide themselves through games based on subjective, rather than imposed, motivations. We can see this emerging in gaming cultures through the rise of player-based phenomena such ‘speedrunning,’ a practice that exploits glitches and faults in a game’s architecture in order to reach the end of a game in the shortest possible time. These niche and ephemeral gaming practices highlight the degree to which practices of improvisation in play may constitute a powerful way of pushing ‘through’ capital–both literally, by accessing games through glitches in their proprietary fabric, and figuratively, if we understand such forms of play as ways to resist the capitalist ideologies embedded within the structures of games (such as the accumulation of wealth, power, or points). Having established here how game-play can act as a form of critical improvisation, this essay now intends to show how these practices of playful improvisation are manifest in the technologies, aesthetics, and the cultural contexts of sound-based art.
Where can we locate practices of improvisation in the context of sound art? For us, sound art is improvisational both in form and practice. First, the emerging aesthetic dimensions of the medium are fundamentally related to the participatory practices of making that have been used to develop the instruments of sound art, such as circuit bending, hacking, tinkering. Here, the playful and open-ended practices of making are a form of improvisation, a modulation within the typical technological frames in which musical composition is typically embedded. Second, when gaming devices are turned into musical instruments (or vice versa), users are called upon to rethink improvisational practice as game-play, and experiment with deploying play as composition. In this sense, game players become sound artists, with sound technologies extrapolating and improvising new sounds from the habit of their play. Improvisation and play thus allow emerging communities of practice to counteract the corporate ‘black boxing’ of entertainment technologies and the creation of closed systems of interaction and participation (such as many mainstream video games) which assimilate user activities and foreclose improvisational play.
Such approaches to sound art are represented by many creative practitioners and take a wide variety of forms, including the appropriation and creative redeployment of participative game environments and game engines. Examples include the creation of interactive sound environments and playable instruments within mainstream participative game ecologies such as the Little Big Planet franchise, the hacking and ‘bending’ of proprietary technologies that invoke the vast universe of game culture nostalgically or in a parodic mode (including handheld gaming systems such as the Nintendo Gameboy, which spawned an entire electronic music genre known as chipmusic), or the making of non-proprietary infrastructures for creative expression that encourage open, participative play, such as open source live coding environments including Sonic Pi, originally developed for the Raspberry Pi platform. Following such approaches, game controllers, digital objects, avatars, virtual architectures, and digital artifacts can thus become tools of composition and/or improvisation. Similarly, we might also think of the improvisational frameworks constructed by sound practitioners as ‘playgrounds,’ which allow players to become performers who explore the technological, political, and aesthetic limits of play. But emergent communities of creative practitioners frequently adopt such close systems and technologies subversively, for example by reverse-engineering them and developing alternative uses. In doing so, they alter the contexts in which these technologies are employed, and open them up for use as musical instruments and infrastructure for creative and critical expression. Technological artifacts with unique user communities can thus be conceptualized as frameworks for musical expression. Vintage gaming consoles, games engines, and computational algorithms have all been reconstructed as new forms of musical creation and interaction through participatory creative communities.
In this short essay, we have approached ‘play’ and ‘improvisation’ as related concepts that can serve to recuperate and empower critical perspectives within the game and sound cultures of capital. As we have suggested, playful approaches to improvisational sound-work and perspectives on play that are rooted in musical traditions can be productively discussed both in sound studies contexts and within game studies paradigms. How, then, do sound practitioners appropriate game technology? How do the socio-political dimensions of digital game culture inform musical practice? With the help of our contributors to this series, we explore how critical perspectives on musical improvisation aid our understanding of the cultural and socio-political significance of play and, likewise, how critical theories of play may broaden our understanding of improvisational sound practices. Through building, tinkering, problem solving, and improvising on the technological back end, creative practitioners playfully create and reshape tools, devices, and techniques of improvisational practice that often embody important ethics of openness, dynamism, fluidity, and sharing, posited against the closed culture of proprietary consumer technology by which we are commonly surrounded. Between live coding, modular synthesis, creative computing, hardware hacking, and appropriation-based interactive performances, the contributors to this series operate in many creative modes that oscillate between improvisation and play. What all of them share is a sense that both play and improvisation enable creative expression outside of prescribed, commodified circuits of media consumption. Improvisation and play can yield wonderful aesthetic experiments and experiences, they can provide critical, social commentary, and they can sharpen our sense of the impact that proprietary technology has on our digital cultural landscape. What we are most interested in are moments when these ambitions converge, when improvisation – drawing on techniques or technologies of play and operating in modes that are associated with play – becomes critical.
All images used with permission from Vector Game Art & New Media Festival.
Featured image: ”Blip Fest 2011 @ Eyebeam, Day 2” by Wikimedia Commons user Lucius Kwok CC BY-SA.
Skot Deeming is an interdisciplinary artist, curator and scholar, whose work spans the spectrum of new media art practice from broadcast media to computational art, experimental videogames, and game art. Drawing on a wealth of practical experience and theoretical knowledge, skot’s practice focuses on new media history and historiography, and DIY technology cultures. This work articulates itself through critical research and writing, exhibitions, installations, and performances. With Martin, Skot is the co-curator of Vector Game Art & New Media Festival., in Toronto, Canada, and the artistic director of the Drone Island Sound Art Festival in Yellowknife, NT. He currently resides in Montreal where he is a doctoral student in the Individualized Program at Concordia University, and a member of the TAG Research Centre, where he investigates critical histories of computational and new media art practices. http://www.mrghosty.com/
Martin Zeilinger (PhD) is a new media researcher, practitioner, and curator whose work focuses on the intersection of digital media art, activism, and intellectual property issues. He is a Lecturer in Media at Anglia Ruskin University, Cambridge/UK, as well as an Associate Researcher at the OCADU Data Materialization Studio. He is co-editor of ‘Dynamic Fair Dealing’ (Univ. of Toronto Press 2014), which develops interdisciplinary perspectives on cultural ownership issues in new media contexts; recent peer-reviewed essays have appeared in the Computer Music Journal, the International Assoc. for the Study of Popular Music Journal, and in Sampling Media (Oxford UP 2014). With Skot, Martin is co-curator of the Toronto-based Vector Game Art & New Media Festival. http://marjz.net/
REWIND! . . .If you liked this post, you may also dig:
Playing with Bits, Pieces, and Lightning Bolts: An Interview with Sound Artist Andrea Perkins — Maile Colbert
Digital Analogies: Techniques of Sonic Play — Roger Mosley
Experiments in Aural Resistance: Nordic Role-Playing, Community, and Sound — Aaron Trammell
SO! Reads: Jonathan Sterne’s MP3: The Meaning of a Format
The point that had lingered with me after first reading Jonathan Sterne’s essay “The mp3 as Cultural Artifact,” was the idea that the mp3 was a promiscuous technology. “In a media-saturated environment,” Sterne writes, “portability and ease of acquisition trumps monomaniacle attention . . . at the psychoacoustic level as well as the industrial level, the mp3 is designed for promiscuity. This has been a long-term goal in the design of sound reproduction technologies” (836). A technology, promiscuous? I did not have to look far to find support. Like germs, I could find copies of mp3s that I had downloaded from Napster in 2000 scattered across generations of my old hard drives. Often they were redundant, too – iTunes having archived a copy separate from my original download.
But, for Sterne, mp3s are also socially promiscuous. They accumulate in the hard drives of the working class and are shared, almost anywhere, through the branching left/right wires of iPod earbuds. Since the popularization of the mp3, there have been new opportunities to share how we listen with others. This is promise of the mp3, and the reason it forms such a key point of scholarly meditation.
MP3: The Meaning of a Format (Duke University Press, 2012) finds Sterne revisiting many of these key themes, with a larger focus on the genealogical beginnings of the mp3 technology. While many of the book’s chapters are extrapolations of prior work Sterne has done regarding the genealogy of listening practices, this work concerns itself less with the 19th century, and more with the 20th century. Perhaps this is related to some of the methodological decisions Sterne has made in planning the book – in seeking out the genealogical origins of the mp3, Sterne worked from archives and manuals described in interviews by engineers who were fundamental to the technology’s production. As such it finds much in common with Trevor Pinch and Frank Trocco’s Analog Days and Dave Tompkins’ How to Wreck a Nice Beach but incorporates the genealogical methods regarding sonic technology present in Sterne’s earlier work The Audible Past and Emily Thompson’s The Soundscape of Modernity. In MP3 Sterne positions himself as a critical cultural studies scholar working between the humanities and sciences, focusing specifically on the mp3 due to its social and technological relevance today. The critical is key here as MP3 is truly a work devised to underscore the economic connections between the construction of our selves as “hearing subjects” and the media industries.
Certainly, the mp3 can still be considered a promiscuous technology, but it is corporate capitalism that had failed to recognize the extent to which it relies on technological promiscuity to support its infrastructure. This focus, ironically, displaces the mp3 as the main object of Sterne’s analysis. It highlights instead the pathological logic of corporate capitalism, and the ways that this rationality has mutated, now, in the wake of mass replicable, malleable, and iterative digital culture. In other words, the mp3 is endemic to a much larger plot, wherein the culture industries adapt to their own deus ex-machina. The naive development of the mp3 by the motion picture industry is a large part of the story here, but it is only a small bit of a much larger whole. The real story involves understanding how a handful of vested corporate interests have shaped the ways that we interpret and understand what listening is. In MP3 Sterne addresses one of the great questions of sound studies: What are the politics of listening? Or, which individuals and institutions have a vested economic interest in questions of how we hear?
Sterne recalls this drama in three parts, each unfolding in a somewhat autonomous fashion, but unified in so far as they explore the economic interests behind the scientific construction of “hearing subjects.” In the first part, Sterne is at his best exploring AT&T’s (and the affiliated Bell Laboratories’) role in funding psychological, physiological, and cybernetic research on hearing. In the second, Sterne explains how this early research has been applied to the visual and technical abstraction of sound in the 1970s. And, in the third part of this genealogy, he explains how these analogs were made digital, specifically the corporate politics which went into the construction of the mp3 standard. Throughout this surprising and detailed trajectory, Sterne makes the invisibility of corporate interests apparent and explicit.
Sterne also hints toward several powerful economic rationalities that have guided the construction of the mp3. Key among these insights is the monetization of cybernetic discourse, or the incorporation of the human body within a scientific understanding of technical systems. In order to engineer an efficient technical system, the capacities and limits of how we interact with (or serve as parts of) these systems must be taken into account. Sterne refers to this mode of engineering as “perceptual technics,” and he goes to great lengths to explain it.
Basically, at the turn of the 20th century, AT&T had taken a keen interest in the science of how people listen because they wanted to maximize the amount of simultaneous conversations broadcast through a single telephone wire. More conversations meant the purchase of fewer wires, and therefore greater profits. Eventually, drawing on the research of the oft-cited Claude Shannon and Warren Weaver (within SO!: What Mixtapes Can Teach Us About Nois and Pushing Record; and soundBox: Mapping Noise), AT&T recognized an economic problem of technical efficiency within their wires – there was too much ambient noise. Because of this, AT&T sought to limit the audible signal transmitted from one phone to another. This would allow for more signals (and therefore conversations) to be transmitted through the same wire. Physiological research provided clues that some frequencies were more audible than others, so engineers worked to compress audio signals to reflect this scientific abstraction of hearing.
The reduction of listening–as an embodied practice–to the quantification and control of the audible spectrum, is, in other words, the history of compression. Which, according to Sterne, should be understood as the true meaning of the mp3. While the mp3 format, like the CD or cassette, may become obsolete, technologies of compression will not. Sterne argues convincingly that most advances in compression technologies have been guided by the invisible logic of corporate capitalism. It is this exact tendency of compression–to make things smaller and more efficient–that threatened to undo the entire project of corporate and branded music distribution in the year 2000, via platforms like Napster. Sterne is well aware of this irony throughout MP3, and uses the final chapter to discuss, briefly, the moment of cultural transformation that is defined by file-sharing and mass distribution.
Bringing things full circle with a somewhat stoic conclusion about the democratic potentials of this moment, he remarks: “The end of the artificial scarcity of recording is a moment of great potential. Its political outcome is still very much in question, but its political meaning should not be” (224). Sterne points to the globalization and ubiquity of mediated listening as a sign that things may not have changed much even though mass networked society at one point promised freedom from a commodity form which privileged things like “liberal notions of property, alienated labor, and ownership” (224). He argues that even the music industries shall persevere, mostly because people have a sublime attraction to listening and music. In other words: Meet the new boss, same as the old boss. There are few moments of liberation to be found within MP3; it is instead a drama of the status quo where the conspirators of corporate capitalism succeed in spite of themselves.
The disparity in Sterne’s tone, when juxtaposing the nefarious and efficient dispositifs of capitalism with an untroubled and authentic construction of music is striking to say the least. And although Sterne is clear to explain that he locates his scholarship as work on a container technology (the mp3) and not its content (the music), this is a somewhat unsatisfying distinction as an embodied practice, such as listening, must take both into account. And while I agree that the mp3 reflects the promiscuity of corporate capitalism, is this challenged by the plethora of ideological nuance coded into song lyrics and arrangements? Do the corporate ideologies of the music industries flow beyond the container of the mp3 into the music itself? Is there any crosstalk, or overlap between these historical constructions? In other words, what are the limits to theorizing a container technology, and how much does the discursive path of the mp3 sculpt the content of what we listen to?
Despite, or perhaps, because of the rather dystopic scene that Sterne alludes to at the end of MP3, it falls nicely in the space between Sound Studies and Critical Information Studies. It bridges humanistic scholarship on embodied listening practices with a critique of the economic interests that have funded much of the scientific research relating to the phenomenology of sound. To that end, MP3 reveals much about the social construction of hearing and the ways that the familiar mythology of audio fidelity has been produced, discussed and exploited by several communication industries. Even though the mp3 may have been eclipsed by industry as the main object of inquiry in the eponomously titled MP3, Sterne succeeds admirably in detailing the promiscuity of corporate capitalism in the listening practices of our everyday lives.
Aaron Trammell is co-founder and Multimedia Editor of Sounding Out! He is also a Media Studies PhD candidate at Rutgers University.