The Dark Side of Game Audio: The Sounds of Mimetic Control and Affective Conditioning
DOI: https://doi.org/10.59350/t9m7w-rx719
It’s an all too familiar movie trope. A bug hidden in a flower jar. A figure in shadows crouched listening at a door. The tape recording that no one knew existed, revealed at the most decisive of moments. Even the abrupt disconnection of a phone call manages to arouse the suspicion that we are never as alone as we may think. And although surveillance derives its meaning the latin “vigilare” (to watch) and French “sur-“ (over), its deep connotations of listening have all but obliterated that distinction.
This month, SO! Multimedia Editor Aaron Trammell curates a forum on Sound and Surveillance, featuring the work of Robin James and Kathleen Battles. And so it begins, with Aaron asking. . .”Want to Play a Game?” –JS
—
It’s eleven o’clock on a Sunday night and I’m in the back room of a comic book store in Scotch Plains, NJ. Game night is wrapping up. Just as I’m about to leave, someone suggests that we play Pit, a classic game about trading stocks in the early 20th century. Because the game is short, I decide to give it a go and pull a chair up to the table. In Pit, players are given a hand of nine cards of various farm-related suits and frantically trade cards with other players until their entire hand matches the same suit. As play proceeds, players hold up a set of similar cards they are willing to trade and shout, “one, one, one!,” “two, two, two!,” “three, three, three!,” until another player is willing to trade them an equivalent amount of cards in a different suit. The game only gets louder as the shouting escalates and builds to a cacophony.
As I drove home that night, I came to the uncomfortable realization that maybe the game was playing me. I and the rest of the players had adopted similar dispositions over the course of the play. As we fervently shouted to one another trying to trade between sets of indistinguishable commodities, we took on similar, intense, and excited mannerisms. Players who would not scream, who would not participate in the reproduction of the game’s sonic environment, simply lost the game, faded out. As for the rest of us, we became like one another, cookie-cutter reproductions of enthusiastic, stressed, and aggravated stock traders, getting louder as we cornered the market on various goods.
We were caught in a cybernetic-loop, one that encouraged us to take on the characteristics of stock traders. And, for that brief period of time, we succumbed to systems of control with far reaching implications. As I’ve argued before, games are cybernetic mechanisms that facilitate particular modes of feedback between players and the game state. Sound is one of the channels through which this feedback is processed. In a game like Pit, players both listen to other players for cues regarding their best move and shout numbers to the table representing potential trades. In other games, such as Monopoly, players must announce when they wish to buy properties. Although it is no secret that understanding sound is essential to good game design, it is less clear how sound defines the contours of power relationships in these games. This essay offers two games, Mafia, and Escape: The Curse of the Temple as case studies for the ways in which sound is used in the most basic of games, board games. By fostering environments that encourage both mimetic control and affective conditioning game sound draws players into the devious logic of cybernetic systems.
Understanding the various ways that sound is implemented in games is essential to understanding the ways that game sound operates as both a form of mimetic control and affective conditioning. Mimetic control is, at its most simple, the power of imitation. It is the degree to which we become alike when we play games. Mostly, it happens because the rules invoke a variety of protocols which encourage players to interact according to a particular standard of communication. The mood set by game sound is the power of affective conditioning. Because we decide what we interact with on account of our moods, moments of affective conditioning prime players to feel things (such as pleasure), which can encourage players to interact in compulsive, excited, subdued, or frenetic ways with game systems.
A game where sound plays a central and important role is Mafia (which has a number of other variants like Werewolf and The Resistance). In Mafia, some players take the secret role of mafia members who choose players to “kill” at night, while the eyes of the others are closed. Because mafia-team players shuffle around during the game and point to others in order to indicate which players to eliminate while the eyes of the other players are closed, the rules of the game suggest that players tap on things, whistle, chirp, and make other ambient noises while everyone’s eyes are closed. This allows for the mafia-team players to conduct their business secretly, as their motions are well below the din created by the other players. Once players open their eyes, they must work together to deduce which players are part of the mafia, and then vote on who to eliminate from the game. Here players are, in a sense, controlled by the game to provide a soundtrack. What’s more, the eeriness of the sounds produced by the players only accentuate the paranoia players feel when taking part in what’s essentially a lynch-mob.
The ambient sounds produced by players of Mafia have overtones of mimetic control. Protocols governing the use of game audio as a form of communication between bodies and other bodies, or bodies and machines, require that we communicate in particular ways at set intervals. Different than the brutal and martial forms of discipline that drove disciplinary apparatuses like Bentham’s panopticon, the form of control exerted through interactive game audio relies on precisely the opposite premise. What is often termed “The Magic Circle of Play” is suspect here as it promises players a space that is safe and fundamentally separate from events in the outside world. Within this space somewhat hypnotic behavior-patterns take place under the auspices of being just fun, or mere play. Players who refuse to play by the rules are often exiled from this space, as they refuse to enter into this contract of soft social norms with others.
Escape: The Curse of the Temple relies on sound to set a game mood that governs the ways that players interact with each other. In Escape, players have ten minutes (of real time) where they must work together to navigate a maze of cardboard tiles. Over the course of the game there are two moments when players must return to the tile that they started the game on, and these are announced by a CD playing in the background of the room. When this occurs, a gong rings on the CD and rhythms of percussion mount in intensity until players hear a door slam. At this point, if players haven’t returned to their starting tile, they are limited in the actions they can take for the rest of the game. In the moments of calm before players make a mad dash for the entrance, the soundtrack waxes ambient. It offers the sounds of howling-winds, rattling chimes, and yawning corridors.
The game is spooky, overall. The combination of haunting ambient sounds and moments where gameplay is rushed and timed, makes for an adrenaline-fueled experience contained and produced by the game’s ambient soundtrack. The game’s most interesting moments come from points where one player is trapped and players must decide whether they should help their friend or help themselves. The tense, haunting, soundtrack evokes feelings of high-stakes immersion. The game is fun because it produces a tight, stressful, and highly interactive experience. It conditions its players through the clever use of its soundtrack to feel the game in an embodied and visceral way. Like the ways that horror movies have used ambient sounds to a great effect in producing tension in audiences (pp.26-27), Escape: The Curse of the Temple encourages players to immerse themselves in the game world by playing upon the tried and true affective techniques that films have used for years. Immersed players feel an increased sense of engagement with the game and because of this they are willingly primed to engage in the mimetic interactive behaviors that engage them within the game’s cybernetic logic.
These two forms of power, mimetic control and affective conditioning, often overlap and coalesce in games. Sometimes, they meet in the middle during games that offer a more or less adaptive form of sound, like Mafia. Players work together and mimic each other when reproducing the ambient forms of quiet that constitute the atmosphere of terror that permeates the game space. Even the roar of bids which occurs in Pit constitutes a form of affective conditioning that encourages players to buy, buy, buy as fast as possible. Effectively simulating the pressure of The Stock Exchange.
Although there is now a growing discipline around the production of game audio, there is relatively little discourse that attempts to understand how the implementation of sound in games functions as a mode of social control. By looking at the ways that sound is implemented in board and card games, we can gain insight of the ways in which it is implemented in larger technical systems (such as computer games), larger aesthetic systems (such as performance art), economic systems (like casinos and the stock market), and even social systems (like parties). Furthermore, it is easy to describe more clearly the ways in which game audio functions as a form of soft power through techniques of mimetic control and affective conditioning. It is only by understanding how these techniques affect our bodies that we can begin to recognize our interactions with large-scale cybernetic systems that have effects reaching beyond the game itself.
–
Aaron Trammell is co-founder and Multimedia Editor of Sounding Out! He is also a Media Studies PhD candidate at Rutgers University. His dissertation explores the fanzines and politics of underground wargame communities in Cold War America. You can learn more about his work at aarontrammell.com.
—
Featured image “Psychedelic Icon,” by Gwendal Uguen CC BY-NC-SA.
—
REWIND!…If you liked this post, you may also dig:
Papa Sangre and the Construction of Immersion in Audio Games- Enongo Lumumba-Kasongo
Sounding Out! Podcast #31: Game Audio Notes III: The Nature of Sound in Vessel- Leonard J. Paul
Experiments in Aural Resistance: Nordic Role-Playing, Community, and Sound- Aaron Trammell
—
Share this:
Digital Analogies: Techniques of Sonic Play
DOI: https://doi.org/10.59350/9xkfz-dep85
Editor’s Note: Welcome to Sounding Out!‘s fall forum titled “Sound and Play,” where we ask how sound studies, as a discipline, can help us to think through several canonical perspectives on play. While Johan Huizinga had once argued that play is the primeval foundation from which all culture has sprung, it is important to ask where sound fits into this construction of culture; does it too have the potential to liberate or re-entrench our social worlds? Here, Roger Moseley challenges us to rethink the philosophical discourses of both sound and play and locates the moments in which they intersect and interface. From games of Telephone to Guitar Hero, Moseley considers the ways in which sonic play can help us understand the phantasmic binaries of the analog and digital.–AT
—
Throughout the distinguished intellectual lineage of play (where it is touched on by notable philosophers such as Plato, Montaigne, Kant, Schiller, Gadamer, Derrida, and Baudrillard), little attention has been paid to the parallels that can be drawn between sound and play as both media and phenomena. The very name of today’s most prominent cultural and technological locus of play, the video game, overtly privileges the eye at the expense of the ear. As recent research and creative work by such figures as Aaron Oldenburg, Aaron Trammell, George Karalis, and Enongo Lumumba-Kasongo indicates, a surge of interest in audio games, as well as video games that emphasize the importance of sound while eschewing or minimizing visual stimuli, is acting as a salutary corrective to this oculocentrism. In what follows, I suggest that bringing sonic and musical techniques to bear on this history might afford new insights into play and its myriad configurations. Conceiving of play sonically entails thinking of sound playfully. This intersectional logic can, I argue, unpick binarisms that enforce problematic distinctions and constrict thought. To demonstrate this, I conclude by deploying the concept of play to redefine the relationship between the digital and the analog—and vice versa.
How can play be defined in a manner that encompasses its farrago of meanings and associations? For video game designers and theorists Katie Salen and Eric Zimmerman, the answer is deceptively simple: play is “free motion within a more rigid structure” (Rules of Play, 304). To illustrate the flexibility of this definition, Salen and Zimmerman allude to the phenomenon of light playing upon the ocean waves. They leave unexamined, however, the intimacy and richness of the relationship between play and sound. From a scientific perspective, the patterned oscillation of which a sequence of sound waves is constituted consists of free motion within the limits set forth by the laws of physics. When disciplined and deployed as a cultural technique–take the play of musical instruments for example–sonic play is humanized and rendered transitive. But, we might also suggest that instruments play people, citing the sensation of automation with which fingers flash over fretboard or keyboard. Moving further away from anthropocentrism, we can observe how sonic technologies render play intransitive once more. From the barrel organ to the iPod, sound plays without human aid when mechanically reproduced. This way of framing reproduction invokes and extends Roger Caillois’s playful category of mimicry, which can be construed as faithful imitation, deceptive fakery, or even a Baudrillardian attempt to simulate a phenomenon that never existed.
In order to pay due attention to both the technologies through which sonic play is mediated as well as the cultural techniques imbue it with significance, I suggest that we supplement Salen and Zimmerman’s definition by thinking of freedom, motion, and structure in both digital and analogical terms. To an extent, the adoption of this modish epistemological framework acknowledges that conceptions of play are always constrained by their prevailing intellectual context. More importantly, however, I contend that technologies of sonic generation and representation from the seventeenth to the nineteenth centuries can be understood to play with the categories of the digital and the analog avant la lettre (ou le chiffre). The two categories are not mutually exclusive and to treat them as such would be to subjugate the granularity of the analog to the binary logic of the digital. Rather, they co-exist as modes between which sounds and players freely oscillate.
The origins of digital sonic play lie within the human body. As Johan Huizinga put it, “the link between play and instrumental skill is to be sought in the nimble and orderly movements of the fingers.” In the course of musical performance, human digits perform innumerable calculations. At its crudest level, musical performance from a score can be construed as a sort of algorithmic play through which mimetic fidelity is evaluated (and wrong notes relentlessly tallied). This ludic logic is at its most visible in rhythm-action video games such as Guitar Hero in which the score is no longer a text but rather a quantitative analysis. The iconography of these games usually indexes a set of digital technologies used primarily for the recording, editing, and playback of music. On the one hand, this relationship can be traced back to Leibniz’s exposition of ars combinatoria and his “invention” of binary; on the other, it is realized by the hydraulic organ and composing machine devised and programmed by the Jesuit polymath Athanasius Kircher, both of which are depicted in his Musurgia universalis (1650). In media-archaeological terms, the combination of Leibniz’s concepts and Kircher’s mechanisms gave rise to the hardware and software of Joseph Marie Jacquard’s revolutionary loom, Charles Babbage’s prototypical Analytical Engine, the player piano, the IBM punch card, and the MIDI sequencer before resurfacing in Guitar Hero, a piece of software that, in purely algorithmic terms, enlists the player’s digits to verify checksums.
Such digital grids may constitute the field and the rules of sonic play, but they must be supplemented by analog elements if play is to flourish. As detailed in C. P. E. Bach’s Versuch über die wahre Art das Clavier zu spielen (1753/62), the clavichord and its descendants distinguished themselves from the harpsichord and the organ by endowing the keyboard with an infinite sensitivity to touch, thereby enabling a mimetic spectrum of emotional flow with unprecedented verisimilitude. Analogicity also provides another perspective on Caillois’s concept of mimicry, according to which one object or activity playfully stands in for another via imitation, deception, or make-believe. Additionally, the curves of Ernst Chladni’s figures, which materialized sound as sand, exemplify this sonic and mimetic trajectory as they rely on both Hermann von Helmholtz’s pioneering work on acoustics and the complex history of phonography to the development of analog synthesis.
In terms of sonic play, digital and analog elements can be chiastically recombined and reconfigured. A sonic communication game such as Telephone relies on the human propensity for analogy and its corrupting influence on the integrity of information transfer, playfully inverting the conditions and functions of the “real” telephone (which was engineered to compress informational content digitally without jeopardizing meaning). In much electronic dance music, the digital latticework, simultaneously visualized and rendered audible by the sequencer’s grid, constitutes a field of play overlaid with vocals, sweeps, and other analog elements that, in turn, have been captured via digital sampling. As a kind of meta-game, a mash-up plays with sonic elements whose relations can be parsed in the digital terms of Leibnizian recombinatorial play, but equally important are the unintended associations and analogies which inevitably emerge. And while games such as Guitar Hero foreground digital techniques of sonic reproduction, they simultaneously foster diverse forms of analogical play involving the player’s manipulation of the sonic (and social) behavior of her on-screen avatar—and vice versa.
There is no doubt that the status of sound and its mediation through and as play have too often gone unacknowledged. As well as seeking to rectify this state of affairs by stressing the importance of sound in relation to the playful operations of other media, we might also dwell on the distinguishing features that set it apart: sound and the techniques that shape it are unique in the ways they simultaneously trace and are traced by the materials, technologies, and metaphors of play.
–
Roger Moseley is an Assistant Professor in the Department of Music at Cornell University. His most recent research brings a media-archaeological perspective to bear on musical performance and improvisation. He is particularly interested in how the concept of play informs sonic practices and cultural techniques. Active as a collaborative pianist on both modern and historical instruments, he has recently published essays on digital games in the contexts of musical and visual culture. His current book project is entitled Digital Analogies: Interfaces and Techniques of Musical Play.
—
REWIND! . . .If you liked this post, you may also dig:
Papa Sangre and the Construction of Immersion in Audio Games– Enongo Lumumba-Kasongo
Playing with Bits, Pieces, and Lightning Bolts: An Interview With Sound Artist Andrea Parkins— Maile Colbert
Experiments in Aural Resistance: Nordic Role-Playing, Community, and Sound— Aaron Trammell
—
Share this:
Current Top Posts
April 2021
January 2020
November 2019
October 2019
August 2019
April 2014
November 2017
March 2015
December 2017
October 2015
Recent Comments