Editor’s Note: Welcome to Sounding Out!‘s fall forum titled “Sound and Play,” where we ask how sound studies, as a discipline, can help us to think through several canonical perspectives on play. While Johan Huizinga had once argued that play is the primeval foundation from which all culture has sprung, it is important to ask where sound fits into this construction of culture; does it too have the potential to liberate or re-entrench our social worlds? Here, Roger Moseley challenges us to rethink the philosophical discourses of both sound and play and locates the moments in which they intersect and interface. From games of Telephone to Guitar Hero, Moseley considers the ways in which sonic play can help us understand the phantasmic binaries of the analog and digital.–AT
Throughout the distinguished intellectual lineage of play (where it is touched on by notable philosophers such as Plato, Montaigne, Kant, Schiller, Gadamer, Derrida, and Baudrillard), little attention has been paid to the parallels that can be drawn between sound and play as both media and phenomena. The very name of today’s most prominent cultural and technological locus of play, the video game, overtly privileges the eye at the expense of the ear. As recent research and creative work by such figures as Aaron Oldenburg, Aaron Trammell, George Karalis, and Enongo Lumumba-Kasongo indicates, a surge of interest in audio games, as well as video games that emphasize the importance of sound while eschewing or minimizing visual stimuli, is acting as a salutary corrective to this oculocentrism. In what follows, I suggest that bringing sonic and musical techniques to bear on this history might afford new insights into play and its myriad configurations. Conceiving of play sonically entails thinking of sound playfully. This intersectional logic can, I argue, unpick binarisms that enforce problematic distinctions and constrict thought. To demonstrate this, I conclude by deploying the concept of play to redefine the relationship between the digital and the analog—and vice versa.
How can play be defined in a manner that encompasses its farrago of meanings and associations? For video game designers and theorists Katie Salen and Eric Zimmerman, the answer is deceptively simple: play is “free motion within a more rigid structure” (Rules of Play, 304). To illustrate the flexibility of this definition, Salen and Zimmerman allude to the phenomenon of light playing upon the ocean waves. They leave unexamined, however, the intimacy and richness of the relationship between play and sound. From a scientific perspective, the patterned oscillation of which a sequence of sound waves is constituted consists of free motion within the limits set forth by the laws of physics. When disciplined and deployed as a cultural technique–take the play of musical instruments for example–sonic play is humanized and rendered transitive. But, we might also suggest that instruments play people, citing the sensation of automation with which fingers flash over fretboard or keyboard. Moving further away from anthropocentrism, we can observe how sonic technologies render play intransitive once more. From the barrel organ to the iPod, sound plays without human aid when mechanically reproduced. This way of framing reproduction invokes and extends Roger Caillois’s playful category of mimicry, which can be construed as faithful imitation, deceptive fakery, or even a Baudrillardian attempt to simulate a phenomenon that never existed.
In order to pay due attention to both the technologies through which sonic play is mediated as well as the cultural techniques imbue it with significance, I suggest that we supplement Salen and Zimmerman’s definition by thinking of freedom, motion, and structure in both digital and analogical terms. To an extent, the adoption of this modish epistemological framework acknowledges that conceptions of play are always constrained by their prevailing intellectual context. More importantly, however, I contend that technologies of sonic generation and representation from the seventeenth to the nineteenth centuries can be understood to play with the categories of the digital and the analog avant la lettre (ou le chiffre). The two categories are not mutually exclusive and to treat them as such would be to subjugate the granularity of the analog to the binary logic of the digital. Rather, they co-exist as modes between which sounds and players freely oscillate.
The origins of digital sonic play lie within the human body. As Johan Huizinga put it, “the link between play and instrumental skill is to be sought in the nimble and orderly movements of the fingers.” In the course of musical performance, human digits perform innumerable calculations. At its crudest level, musical performance from a score can be construed as a sort of algorithmic play through which mimetic fidelity is evaluated (and wrong notes relentlessly tallied). This ludic logic is at its most visible in rhythm-action video games such as Guitar Hero in which the score is no longer a text but rather a quantitative analysis. The iconography of these games usually indexes a set of digital technologies used primarily for the recording, editing, and playback of music. On the one hand, this relationship can be traced back to Leibniz’s exposition of ars combinatoria and his “invention” of binary; on the other, it is realized by the hydraulic organ and composing machine devised and programmed by the Jesuit polymath Athanasius Kircher, both of which are depicted in his Musurgia universalis (1650). In media-archaeological terms, the combination of Leibniz’s concepts and Kircher’s mechanisms gave rise to the hardware and software of Joseph Marie Jacquard’s revolutionary loom, Charles Babbage’s prototypical Analytical Engine, the player piano, the IBM punch card, and the MIDI sequencer before resurfacing in Guitar Hero, a piece of software that, in purely algorithmic terms, enlists the player’s digits to verify checksums.
Such digital grids may constitute the field and the rules of sonic play, but they must be supplemented by analog elements if play is to flourish. As detailed in C. P. E. Bach’s Versuch über die wahre Art das Clavier zu spielen (1753/62), the clavichord and its descendants distinguished themselves from the harpsichord and the organ by endowing the keyboard with an infinite sensitivity to touch, thereby enabling a mimetic spectrum of emotional flow with unprecedented verisimilitude. Analogicity also provides another perspective on Caillois’s concept of mimicry, according to which one object or activity playfully stands in for another via imitation, deception, or make-believe. Additionally, the curves of Ernst Chladni’s figures, which materialized sound as sand, exemplify this sonic and mimetic trajectory as they rely on both Hermann von Helmholtz’s pioneering work on acoustics and the complex history of phonography to the development of analog synthesis.
In terms of sonic play, digital and analog elements can be chiastically recombined and reconfigured. A sonic communication game such as Telephone relies on the human propensity for analogy and its corrupting influence on the integrity of information transfer, playfully inverting the conditions and functions of the “real” telephone (which was engineered to compress informational content digitally without jeopardizing meaning). In much electronic dance music, the digital latticework, simultaneously visualized and rendered audible by the sequencer’s grid, constitutes a field of play overlaid with vocals, sweeps, and other analog elements that, in turn, have been captured via digital sampling. As a kind of meta-game, a mash-up plays with sonic elements whose relations can be parsed in the digital terms of Leibnizian recombinatorial play, but equally important are the unintended associations and analogies which inevitably emerge. And while games such as Guitar Hero foreground digital techniques of sonic reproduction, they simultaneously foster diverse forms of analogical play involving the player’s manipulation of the sonic (and social) behavior of her on-screen avatar—and vice versa.
There is no doubt that the status of sound and its mediation through and as play have too often gone unacknowledged. As well as seeking to rectify this state of affairs by stressing the importance of sound in relation to the playful operations of other media, we might also dwell on the distinguishing features that set it apart: sound and the techniques that shape it are unique in the ways they simultaneously trace and are traced by the materials, technologies, and metaphors of play.
Roger Moseley is an Assistant Professor in the Department of Music at Cornell University. His most recent research brings a media-archaeological perspective to bear on musical performance and improvisation. He is particularly interested in how the concept of play informs sonic practices and cultural techniques. Active as a collaborative pianist on both modern and historical instruments, he has recently published essays on digital games in the contexts of musical and visual culture. His current book project is entitled Digital Analogies: Interfaces and Techniques of Musical Play.
REWIND! . . .If you liked this post, you may also dig:
Papa Sangre and the Construction of Immersion in Audio Games– Enongo Lumumba-Kasongo
Editor’s Note: Welcome to Sounding Out!‘s fall forum titled “Sound and Play,” where we ask how sound studies, as a discipline, can help us to think through several canonical perspectives on play. While Johan Huizinga had once argued that play is the primeval foundation from which all culture has sprung, it is important to ask where sound fits into this construction of culture; does it too have the potential to liberate or re-entrench our social worlds? SO!’s new regular contributor Enongo Lumumba-Kasongo notes how audio games, like Papa Sangre, often use sound as a gimmick to engage players, and considers the politics of this feint. For whom are audio games immersive, and how does the experience serve to further marginalize certain people or disadvantaged groups?–AT
Immersion is a problem at the heart of sound studies. As Frances Dyson (2009) suggests in Sounding New Media, “Sound is the immersive medium par excellence. Three dimensional, interactive and synesthetic, perceived in the here and now of an embodied space, sound returns to the listener the very same qualities that media mediates…Sound surrounds” (4). Alternately, in the context of games studies (a field that is increasingly engaged with sound studies), issues of sound and immersion have most recently been addressed in terms of instrumental potentialities, historical developments, and technical constraints. Some notable examples include Sander Huiberts’ (2010) M.A. thesis entitled “Captivating Sound: The Role of Audio Immersion for Computer Games,” in which he details technical and philosophical frames of immersion as they relate to the audio of a variety of computer games, and an article by Aaron Oldenburg (2013) entitled “Sonic Mechanics: Audio as Gameplay,” in which he situates the immersive aspects of audio-gameplay within contemporaneous experimental art movements. This research provokes the question: How do those who develop these games construct the idea of immersion through game design and what does this mean for users who challenge this construct? Specifically I would like to challenge Dyson’s claim that sound really is “the immersive medium par excellence” by considering how the concept of immersion in audio-based gameplay can be tied to privileged notions of character and game development.
In order to investigate this problem, I decided to play an audio game and document my daily experiences on a WordPress blog. Based on its simulation of 3D audio Papa Sangre was the first game that came to mind. I also selected the game because of its accessibility; unlike the audio game Deep Sea, which is celebrated for its immersive capacities but is only playable by request at The Museum of Art and Digital Entertainment, Papa Sangre is purchasable as an app for $2.99 and can be played on an iPhone, iPad or iPod. Papa Sangre helps us to consider new possibilities for what is meant by virtual space and it serves as a useful tool for pushing back against essentialisms of “immersion” when talking sound and virtual space.
Papa Sangre is comprised of 25 levels, the completion of which leads player incrementally closer towards the palace of Papa Sangre, a man who has kidnapped a close friend of the protagonist. The game boasts real time binaural audio, meaning that the game’s diegetic sounds (sounds that the character in the game world can “hear”) pan across the player’s headphones in relation to the movement of the game’s protagonist. The objective of each level is to locate and collect musical notes that are scattered through the game’s many topographies while avoiding any number of enemies and obstacles, of course.
A commercial success, Papa Sangre has been named “Game of the Week” by Apple, received a 9/10 rating from IGN, a top review from 148apps, and many positive reviews from fans. Gamezebo concludes an extremely positive review of Papa Sangre by calling it “a completely unique experience. It’s tense and horrifying and never lets you relax. By focusing on one aspect of the game so thoroughly, the developers have managed to create something that does one thing really, really well…Just make sure to play with the lights on.” This commercial attention has yielded academic feedback as well. In a paper entitled “Towards an analysis of Papa Sangre, an audio-only game for the iPhone/iPad,” Andrew Hugill (2012) celebrates games like Papa Sangre for providing “an excellent opportunity for the development of a new framework for electroacoustic music analysis.” Despite such attention–and perhaps because of it–I argue that Papa Sangre deserves a critical second listen.
Between February and April of 2012, I played Papa Sangre several times a day and detailed the auditory environments of the game in my blog posts. However, by the time I reached the final level, I still wasn’t sure how to answer my initial question. Had Papa Sangre really engendered a novel experience or it could simply be thought of as a video game with no video? I noted in my final post:
I am realizing that what makes the audio gaming experience seem so different from the experience of playing video games is the perception that the virtual space, the game itself, only exists through me. The “space” filled by the levels and characters within the game only exists between my ears after it is projected through the headphones and then I extend this world through my limbs to my extremities, which feeds back into the game through the touch screen interface, moving in a loop like an electric current…Headphones are truly a necessity in order to beat the game, and in putting them on, the user becomes the engine through which the game comes to life…When I play video games, even the ones that utilize a first-person perspective, I feel like the game space exists outside of me, or rather ahead of me, and it is through the controller that I am able to project my limbs forward into the game world, which in turn structures how I orient my body. Video game spaces of course, do not exist outside of me, as I need my eyes and ears to interpret the light waves and sound waves that travel back from the screen, but I suppose what matters here is not what is actually happening, but how what is happening is perceived by the user. Audio games have the potential to engender completely different gaming experiences because they make the user feel like he or she is the platform through which the game-space is actualized.
Upon further reflection, however, I recognize that Papa Sangre creates an environment designed to be immersive only to certain kinds of users. A close reading of Papa Sangre reveals bias against both female and disabled players.
Take Papa Sangre’s problematic relationship with blindness. The protagonist is not a visually impaired individual operating in a horrifying new world, but rather a sighted individual who is thrust into a world that is horrifying by virtue of its darkness. The first level of the game is simply entitled “In the Dark.” When the female guide first appears to the protagonist in that same level, she states:
Here you are in the land of the dead, the realm ruled by Papa Sangre…In this underworld it is pitch dark. You cannot see a thing; you can’t even see me, a fluttery watery thing here to help you. But you can listen and move…You must learn how to see with your ears. You will need these powers to save the soul in peril and make your way to the light.
Note the conversation between 3:19 and 3:56.
The game envisions an audience who find blindness to be necessarily terrifying. By equating an inability to see with death and fear, developers are intensifying popular horror genre tropes that diminish the lived experiences of those with visual impairments and unquestioningly present blindness as a problem to overcome. Rather than challenging the relationship between blindness and vulnerability that horror-game developers fetishize, Papa Sangre misses the opportunity to present a visually impaired protagonist who is not crippled by his or her disability.
Disconcertingly, audio games have been tied to game accessibility efforts by developers and players alike for many years. In a 2008 interview Kenji Eno, founder of WARP (a company that specialized in audio games in the late 90s), claimed his interactions with visually impaired gamers yielded a desire to produce audio games. Similarly forums like audiogames.net showcase users and developers interested in games that cater to gamers with impaired vision.
In terms of its actual game-play, PapaSangre is navigable without visual cues. After playing the game for just two weeks I was able to explore each level with my eyes closed. Still, the ease with which gamers can play the game without looking at the screen does not negate the tension caused by recycled depictions of disability that are in many ways built into storyline’s foundation.
The game also fails to engage gender in any complexity. Although the main character’s appearance is never shown, the protagonist is aurally gendered male. Most notable are the deep grunting noises made when he falls to the ground. For me, this acted as a barrier to imagining a fully embodied virtual experience. Those deep grunts revealed many assumptions the designers must have considered about the imagined and perhaps intended audience of the game. While lack of diversity is certainly an issue at the heart of all entertainment media, Papa Sangre‘s oversight directly contradicts the message of the game, wherein the putative goal is to experience an environment that enhances one’s sense of self within the virtual space.
On October 31st, 2013, Somethin’ Else will release Papa Sangre II. A quick look at the trailer suggests that the developers’ have not changed the formula. The 46-second clip warns that the game is “powered by your fear” after noting, “This Halloween, you are dead.”
It appears that an inability to see is still deeply connected with notions of fear and death in the game’s sequel. This does not have to be the case. Why not design a game where impairment is not framed as a hindrance or source of fear? Why not build a game with the option to choose between different sounding voice actors and actresses? Despite its popularity, however, Papa Sangre is by no means representative of general trends across the spectrum of audio-based game design. Oldenburg (2013) points out that over the past decade many independent game developers have been designing experimental “blind games” that eschew themes and representations found in popular video games in favor of the abstract relationships between diegetic sound and in-game movement.
Whether or not they eventually consider the social politics of gaming, Papa Sangre’s developers already send a clear message to all gamers by hardwiring disability and gender into both versions of the game while promoting a limited image of “immersion.” Hopefully as game designers Somethin’ Else grow in popularity and prestige, future developers that use the “Papa Engine” will be more cognizant of the privilege and discrimination embedded in the sonic cues of its framework. Until then, if you are not a sighted male gamer, you must prepare yourself to be immersed in constant aural cues that this experience, like so many others, was not designed with you in mind.
Enongo Lumumba-Kasongo is a PhD student in the Department of Science and Technology Studies at Cornell University. Since completing a senior thesis on digital music software, tacit knowledge, and gender under the guidance of Trevor Pinch, she has become interested in pursuing research in the emergent field of sound studies. She hopes to combine her passion for music with her academic interests in technological systems, bodies, politics and practices that construct and are constructed by sound. More specifically she would like to examine the politics surrounding low-income community studios, as well as the uses of sound in (or as) electronic games. In her free time she produces hip hop beats and raps under the moniker Sammus (based on the video game character, Samus Aran, from the popular Metroid franchise).
REWIND! . . .If you liked this post, you may also dig:
Goalball: Sport, Silence, and Spectatorship– Melissa Helquist
As the practice of sound design becomes ever more refined as a key factor in the immersive aspects of gameplay, it is essential to develop a conceptual vocabulary of the ways that sound is implemented as a cultural facet. In particular, it is important to recognize the power relations at stake within the implementation of the human voice as an interactive narrative trope. And, while I’ve already discussed the ways in which the voice of GLaDOS in Portal invites players to reflect on how they internalize a set of mediated perspectives about how their body ought to be, it is equally important to consider the other ways that a narrator’s voice invites players to reconsider the intersection of agency and surveillance.
This post compares the use of narration in Bastion and The Stanley Parable in an effort to understand how the voice is used as what Karen Collins would refer to as an “interactive non-diagetic sound,” or, in other words, a sound that is triggered by player actions, but not experienced by the character in the game. Specifically, I argue that the voice in these examples is an essential point in the feedback loop between player and game. And, as part of the cybernetics of gameplay, it produces a dispositif of surveillance, akin to Bentham’s panopticon, which lets the player know their actions are constantly being monitored, calculated, and considered by the game’s algorithms. But, while the original panopticon produced the effect of surveillance through the clever use of light, these games use sound to effect surveillance.
Bastion, was developed by the small indie game company Supergiant Games, but was distributed and released by Warner Bros. Interactive Entertainment, first through Microsoft’s distribution service, XBOX Live Arcade, but has since been released more broadly after receiving much critical acclaim. In Bastion, players take the role of a young boy, referred to only as “The Kid,” and adventure around gradually rebuilding a world that has fallen apart since a cataclysmic sundering referred to only as “the Calamity.” Although the world of Bastion is beautiful and visually stimulating, it is the game’s sound design that has earned it much critical acclaim. The game is narrated by a character named “Rucks,” who speaks in a deep weathered voice, with somewhat of a western twang. And, even though The Kid eventually encounters and is able to interact with Rucks within the game, Rucks still relays dialogue in the third-person.
When The Kid and Rucks meet, Rucks evinces with trademark grit, “Sure enough, he finds another. He finds me.” And, while that is a scripted plot point within the game, at other points Rucks’ narration modulates to best reflect the player’s actions. A player who begins the game slowly, exploring nooks and crannies, might hear “The kid walks slowly down the path, checking everything,” while a player who runs straight ahead could hear, “The kid barrels forward, not looking once behind him.” These quotes force the player to recognize that the game is watching, and actively staging narrative commentary about their in-game decisions. This commentary unfolds in aural space, through narration, discrete from the old text (and controller) triggers of “look” and “examine” which used to prompt text-box commentary about the environment of the game. In short, Bastion’s sound design succeeds because it is balanced in such a precise way: players are being constantly engaged with a narration which confirms that they are, in fact, properly interacting within the game world and story.
The Stanley Parable, on the other hand, works deliberately to turn the paradigm of narration on its head. Where Rucks, in Bastion, frequently alluded to how many secrets he was yet to reveal about himself and the game-world, his character ultimately plays a supportive role, helping The Kid to understand the chaotic environment of the game. The narrator in The Stanley Parable, however, plays the antagonist in many ways, attempting to foreshadow and predetermine the actions of the player, or “Stanley.” On the game’s website, a short sentence contextualizes the endeavor, “The Stanley Parable is a Half Life 2 mod about video games.” The game itself is a mod of the “Source engine,” which runs both Half-Life 2, and Portal, was developed by the very small development team of Davey Wreden and William Pugh, and released for free. It is meta-fiction that stages a critique of the context of narrative within interactive games and fiction. Specifically, the game questions the idea of narrative itself by showcasing the ways that players are able to undermine the scripted plots and spaces of a videogame by exploring and experimenting with exploits and bugs in the game’s code and narrative.
Although the narrator in The Stanley Parable will prescribe several decisions to the player over the course of the game, the player is given the agency to contest the story as told by the narrator, and, therefore, to experiment with the plot. As the player reaches a set of two open doors on the way to the employee lounge the narrator reads, “When Stanley came to a set of two open doors, he entered the door on his left.” If the player chooses to travel through the door on the right instead, the narrator will attempt to steer the player back to the main plot tree by saying, “This was not the correct way to the employee lounge, and Stanley knew it perfectly well.” And, then as an open door is revealed, “So he turned left at the first open door, and walked back in the right direction.” If the player continues to ignore the narrator’s advice, he comes to somewhat of a dead end and the narrator reads, “Stanley was so bad at following directions, it’s incredible that he wasn’t fired years ago. Maybe this was why everyone left. No one wanted to be around someone as bad at listening as him.” The player is given several other opportunities to make decisions and lead the story to completion in achieving one of seven endings; each based the decisions the player has made when interacting with the narrator’s dialogue.
The Stanley Parable allows players the agency to see the limitations of linear storytelling where Bastion does not. Where some paths in The Stanley Parable will lead the player into direct conflict with the narrator, other paths do not. There is never a point where the narrator ceases to comment on the player’s actions and activities. Because the sonic feedback of surveillance remains a constant in both games the player remains engaged, in both cases, with the logic of the game-system. In other words, no matter how many times the player defies the narrator in The Stanley Parable, it never seems like the game is breaking. The game world remains constant because the motif of surveillance holds; players know the game still works because the narrator continues to stage commentary – even if it is commentary about the player’s failure to keep to the plot.
For Marc Andrejevic, author of iSpy (2007)–who has written extensively about the abundance of surveillance techniques implemented in digital spaces–the danger of surveillance lies in the production of an asymmetrical power relationship between media producer and media consumer. And, while this is certainly best argued about instances of dataveillance–how companies like Amazon, for example, track customer clicks on and off their website via web cookies in order to better produce exploitable (and in some cases saleable) consumer profiles–it is important to also consider the ways that the implementation of sound also functions as a technique of control.
At its most positive, the sonic panopticism of Bastion and The Stanley Parable offer players a sense of comfort in knowing that the game is operating properly, and not glitching out. Further, players are invited into a more immersive game, which leverages both visual and audio interactivity to lull players into an environment of almost trancelike feedback and play. Clearly, this is the promise of good sound design; it gently alerts players to the presence of a tightly designed and well-implemented game, and produces affects of brand loyalty and trust within a game’s player contingent.
But, while there are clearly aesthetic and market benefits to the implementation of narration in both games, one cannot help but wonder, in the context of post-feminism and self-surveillance, what implications there are in the implementation of the male voice as surveil-er in both games. Just as it was curious in Portal 2 how GLaDOS acted as a critical female voice constantly judging the player’s body image and intelligence, it is curious how much authority is given to the voice of Rucks in Bastion. And while several good critiques have already been written about how the game features only one (somewhat silent, and certainly helpless) female character, and how the game’s villain is portrayed, concretely, as the racially exotic other, it is sadly fitting that the most comforting and well-acclaimed aspects of the game come from the interactivity produced by the voice of its distinctively white male narrator.
The sound design in The Stanley Parable, of course, is more cutting in the ways it stages a commentary about how the voice of the narrator (this time distinctively British), exacts a form of social coercion through techniques of surveillance, and how these techniques serve, namely, to hamper player agency. But, even its own narrative of resistance fails to compel; in fact, it is the uneasy ending of compliance and conformity that is, perhaps, the happiest. This, ironically, reveals one of the key cultural problems of our era: the reciprocal aspects of surveillance and interactivity. If affective resonances of trust, knowledge, and comfort come bundled with the male voice, is it in the vested economic interests of sound design communities to leverage these to make profit? Even though both games have earned critical praise, it is only Bastion that has won awards for sound design. In other words, are we caught in our own feedback loop of comfort, industry, and design?
Aaron Trammell is co-founder and Multimedia Editor of Sounding Out! He is also a Media Studies PhD candidate at Rutgers University. His dissertation explores the fanzines and politics of underground wargame communities in Cold War America. You can learn more about his work at aarontrammell.com.
REWIND! . . .If you liked this post, you may also dig:
DIANE… The Personal Voice Recorder in Twin Peaks– Tom McEnaney
The Role of Sound in Video Games: Pong, Limbo, and Interactivity– Aaron Trammell
Orality and Cybernetics in Battleship– Aaron Trammell