As the practice of sound design becomes ever more refined as a key factor in the immersive aspects of gameplay, it is essential to develop a conceptual vocabulary of the ways that sound is implemented as a cultural facet. In particular, it is important to recognize the power relations at stake within the implementation of the human voice as an interactive narrative trope. And, while I’ve already discussed the ways in which the voice of GLaDOS in Portal invites players to reflect on how they internalize a set of mediated perspectives about how their body ought to be, it is equally important to consider the other ways that a narrator’s voice invites players to reconsider the intersection of agency and surveillance.
This post compares the use of narration in Bastion and The Stanley Parable in an effort to understand how the voice is used as what Karen Collins would refer to as an “interactive non-diagetic sound,” or, in other words, a sound that is triggered by player actions, but not experienced by the character in the game. Specifically, I argue that the voice in these examples is an essential point in the feedback loop between player and game. And, as part of the cybernetics of gameplay, it produces a dispositif of surveillance, akin to Bentham’s panopticon, which lets the player know their actions are constantly being monitored, calculated, and considered by the game’s algorithms. But, while the original panopticon produced the effect of surveillance through the clever use of light, these games use sound to effect surveillance.
Bastion, was developed by the small indie game company Supergiant Games, but was distributed and released by Warner Bros. Interactive Entertainment, first through Microsoft’s distribution service, XBOX Live Arcade, but has since been released more broadly after receiving much critical acclaim. In Bastion, players take the role of a young boy, referred to only as “The Kid,” and adventure around gradually rebuilding a world that has fallen apart since a cataclysmic sundering referred to only as “the Calamity.” Although the world of Bastion is beautiful and visually stimulating, it is the game’s sound design that has earned it much critical acclaim. The game is narrated by a character named “Rucks,” who speaks in a deep weathered voice, with somewhat of a western twang. And, even though The Kid eventually encounters and is able to interact with Rucks within the game, Rucks still relays dialogue in the third-person.
When The Kid and Rucks meet, Rucks evinces with trademark grit, “Sure enough, he finds another. He finds me.” And, while that is a scripted plot point within the game, at other points Rucks’ narration modulates to best reflect the player’s actions. A player who begins the game slowly, exploring nooks and crannies, might hear “The kid walks slowly down the path, checking everything,” while a player who runs straight ahead could hear, “The kid barrels forward, not looking once behind him.” These quotes force the player to recognize that the game is watching, and actively staging narrative commentary about their in-game decisions. This commentary unfolds in aural space, through narration, discrete from the old text (and controller) triggers of “look” and “examine” which used to prompt text-box commentary about the environment of the game. In short, Bastion’s sound design succeeds because it is balanced in such a precise way: players are being constantly engaged with a narration which confirms that they are, in fact, properly interacting within the game world and story.
The Stanley Parable, on the other hand, works deliberately to turn the paradigm of narration on its head. Where Rucks, in Bastion, frequently alluded to how many secrets he was yet to reveal about himself and the game-world, his character ultimately plays a supportive role, helping The Kid to understand the chaotic environment of the game. The narrator in The Stanley Parable, however, plays the antagonist in many ways, attempting to foreshadow and predetermine the actions of the player, or “Stanley.” On the game’s website, a short sentence contextualizes the endeavor, “The Stanley Parable is a Half Life 2 mod about video games.” The game itself is a mod of the “Source engine,” which runs both Half-Life 2, and Portal, was developed by the very small development team of Davey Wreden and William Pugh, and released for free. It is meta-fiction that stages a critique of the context of narrative within interactive games and fiction. Specifically, the game questions the idea of narrative itself by showcasing the ways that players are able to undermine the scripted plots and spaces of a videogame by exploring and experimenting with exploits and bugs in the game’s code and narrative.
Although the narrator in The Stanley Parable will prescribe several decisions to the player over the course of the game, the player is given the agency to contest the story as told by the narrator, and, therefore, to experiment with the plot. As the player reaches a set of two open doors on the way to the employee lounge the narrator reads, “When Stanley came to a set of two open doors, he entered the door on his left.” If the player chooses to travel through the door on the right instead, the narrator will attempt to steer the player back to the main plot tree by saying, “This was not the correct way to the employee lounge, and Stanley knew it perfectly well.” And, then as an open door is revealed, “So he turned left at the first open door, and walked back in the right direction.” If the player continues to ignore the narrator’s advice, he comes to somewhat of a dead end and the narrator reads, “Stanley was so bad at following directions, it’s incredible that he wasn’t fired years ago. Maybe this was why everyone left. No one wanted to be around someone as bad at listening as him.” The player is given several other opportunities to make decisions and lead the story to completion in achieving one of seven endings; each based the decisions the player has made when interacting with the narrator’s dialogue.
The Stanley Parable allows players the agency to see the limitations of linear storytelling where Bastion does not. Where some paths in The Stanley Parable will lead the player into direct conflict with the narrator, other paths do not. There is never a point where the narrator ceases to comment on the player’s actions and activities. Because the sonic feedback of surveillance remains a constant in both games the player remains engaged, in both cases, with the logic of the game-system. In other words, no matter how many times the player defies the narrator in The Stanley Parable, it never seems like the game is breaking. The game world remains constant because the motif of surveillance holds; players know the game still works because the narrator continues to stage commentary – even if it is commentary about the player’s failure to keep to the plot.
For Marc Andrejevic, author of iSpy (2007)–who has written extensively about the abundance of surveillance techniques implemented in digital spaces–the danger of surveillance lies in the production of an asymmetrical power relationship between media producer and media consumer. And, while this is certainly best argued about instances of dataveillance–how companies like Amazon, for example, track customer clicks on and off their website via web cookies in order to better produce exploitable (and in some cases saleable) consumer profiles–it is important to also consider the ways that the implementation of sound also functions as a technique of control.
At its most positive, the sonic panopticism of Bastion and The Stanley Parable offer players a sense of comfort in knowing that the game is operating properly, and not glitching out. Further, players are invited into a more immersive game, which leverages both visual and audio interactivity to lull players into an environment of almost trancelike feedback and play. Clearly, this is the promise of good sound design; it gently alerts players to the presence of a tightly designed and well-implemented game, and produces affects of brand loyalty and trust within a game’s player contingent.
But, while there are clearly aesthetic and market benefits to the implementation of narration in both games, one cannot help but wonder, in the context of post-feminism and self-surveillance, what implications there are in the implementation of the male voice as surveil-er in both games. Just as it was curious in Portal 2 how GLaDOS acted as a critical female voice constantly judging the player’s body image and intelligence, it is curious how much authority is given to the voice of Rucks in Bastion. And while several good critiques have already been written about how the game features only one (somewhat silent, and certainly helpless) female character, and how the game’s villain is portrayed, concretely, as the racially exotic other, it is sadly fitting that the most comforting and well-acclaimed aspects of the game come from the interactivity produced by the voice of its distinctively white male narrator.
The sound design in The Stanley Parable, of course, is more cutting in the ways it stages a commentary about how the voice of the narrator (this time distinctively British), exacts a form of social coercion through techniques of surveillance, and how these techniques serve, namely, to hamper player agency. But, even its own narrative of resistance fails to compel; in fact, it is the uneasy ending of compliance and conformity that is, perhaps, the happiest. This, ironically, reveals one of the key cultural problems of our era: the reciprocal aspects of surveillance and interactivity. If affective resonances of trust, knowledge, and comfort come bundled with the male voice, is it in the vested economic interests of sound design communities to leverage these to make profit? Even though both games have earned critical praise, it is only Bastion that has won awards for sound design. In other words, are we caught in our own feedback loop of comfort, industry, and design?
Aaron Trammell is co-founder and Multimedia Editor of Sounding Out! He is also a Media Studies PhD candidate at Rutgers University. His dissertation explores the fanzines and politics of underground wargame communities in Cold War America. You can learn more about his work at aarontrammell.com.
REWIND! . . .If you liked this post, you may also dig:
DIANE… The Personal Voice Recorder in Twin Peaks– Tom McEnaney
The Role of Sound in Video Games: Pong, Limbo, and Interactivity– Aaron Trammell
Orality and Cybernetics in Battleship– Aaron Trammell
On Sunday evening, Susan discovered the tofu had gone bad. Unfortunately, the entrée for the evening was to be tofu with sweet chili sauce. We connect on Skype at 3:30pm, as Susan is cutting up vegetables. Usually, she has classical music on while she cooks; it helps her concentrate. She’s cut up so many vegetables in her life, that she finds music sweetens the repetitive activity. However, today I hear only the rewarding sound of her knife bisecting baby bok choy.
Susan and I don’t talk about this sound, but it is certainly familiar. She says she cuts up the pieces small, as Mimi likes the chunks of bell pepper to be as little as possible.
My ethnographic work on cooking is birthed from a very personal place: Susan is my Aunt and Mimi my mother. They live together in Kingston, Massachusetts, where Susan’s cooking nourishes Mimi through her ongoing chemo and radiation treatments. Using Skype, I watched and asked questions remotely from Raleigh, North Carolina on three consecutive evenings during their dinner preparation in order to more deeply understand cooking as an art. From the first moment of preparation each night, Susan and I talked about the meal, the cooking techniques, and her feelings about cooking and eating–and I noted that sound emerged as central to her culinary process.
Opening my ethnographic practice up to sonic analysis enables new definitions of both chef and kitchen as lively, complex sites, constantly negotiating with each other. Taking the role of sound into account in the practice of cooking allows me to construct new interpretations of cooking artistry that considers everyday negotiations and embodied limitations not as “threats” to the cooking art, but, instead, as elements that enrich its artistry.
My sonic analysis specifically chafes against dominant formations of ”cooking as art” in the contemporary moment, exemplified by reality television programs such as The Food Network’s Chopped, which constructs a static configuration of space with the cook as subject and the meal as art object. On Chopped, four chefs are given thirty minutes and four ingredients. Using these items, they must make a dish to be judged by a panel of food experts. These items are often strange or incongruous: on one episode, they had to make an appetizer out of frosted wheat cereal, baby red romaine lettuce, black garlic, and quahog clams. The success of a dish is measured by the chef’s ability to balance the necessary experimentation with an implied universal of good taste, texture balance, and pleasuring preparation. In other words, Chopped collapses art-making and capital into the “art object-meal,” reproducing a tired definition of “high art” that necessitates access to wealth and privilege, because the creation of “art” requires expensive foodstuffs, sophisticated kitchen technologies, and a highly controlled visual and sonic environment.
In my Aunt Susan’s kitchen, there are a number of sonic and spatial negotiations that preclude her cooking from the singular criteria of artistry perpetuated by Chopped. Specifically, my Aunt Susan’s cooking does not meet Chopped’s standards because it requires negotiations related to her personal mobility and my mother’s health. My Aunt uses a wheelchair to get around her house, and, as a result, some kitchen appliances are harder to reach. My mother’s chemo and radiation treatments mean that she has both complicated limitations to her diet and fluid culinary desires.
In seeking to understand the fluid challenges of Susan’s cooking, I designed my virtual sensory ethnography by combining two methods, defined respectively by Sarah Pink and Jenna Burrell. In Sensory Ethnography, Pink proposes that the ethnographer is immersed in smells, tastes, sights, and sound during the ethnographic process. Things that might seem mundane such as the sound of onions being chopped, for example, can actually reveal a complex set of relations about the cook and their process. The cook might be listening to the chopping as a rhythm in her process, like background music: the pleasing sound as it hits the cutting board. But if the onion isn’t fresh, the sound is less crisp, less crunch, the sounds changes to speak of a different type of knowledge, and she must act differently in response. In “The Field Site as a Network,” Burrell proposes an understanding of the ethnographic field site as a network rather than a singular object. The field site, in other words, is a heterogeneous set of connections, always expanding. Using a technology like Skype to do ethnography is not “ethnography at a distance,” she implies, rather it is the field site manifested through a multiplicity of connections. It simply reflects the ever-changing set of relations that comprise our world.
After Susan has cut up all the longer-cooking vegetables and set the chili sauce to simmer, we disconnect. At 5:15pm, we connect again. Susan unwraps the tofu, and something isn’t right. She calls Mimi into the kitchen, and, after some deliberation, they decide that steak will need to replace the tofu. It’s a disappointment as the tofu would have tasted best with the sweet chili sauce.
The sonic landscape of Susan’s kitchen has been, up to this, point, fairly solitary and controlled. When Susan welcomes Mimi in, the kitchen becomes a lively space of conversation, interaction, and negotiation. The production of the sonic space in the kitchen, from solitary preparation to lively interaction, is a crucial part of Susan’s art. The kitchen has undergone what Brian Massumi, in his essay “Floating the Social,” calls a “modulation of the dimension of perception [rather] than an encoding of separate pieces of data or a sequencing of units of meaning” (41). Such a sonic modulation challenges the narrative of lone artist-chef creating object-meal. Rather than segmenting the meal into a set of data blocks (chef, food, preparation time, and eater), Susan orchestrates the art of cooking as participatory with Mimi.
In the kitchen, Mimi also examines the tofu. She offers some information about it, and then joins Susan to figure out what other protein might work.
Once the steak is decided upon, Mimi exits and Susan works again at preparing dinner.
Susan’s sonic modulations, in this case conversation, allow for immersion and engagement in the lively sonic space of her kitchen. Mimi and Susan create a co-constitutive relationship between chef and eater. Unlike on Chopped, the eater is a participant rather than a judge.
Susan and I disconnect from Skype to give the steak time to thaw. We connect once again, at 6:20PM, once the steak has been properly thawed. As we discuss how Susan learned to cook, the smoke alarm suddenly comes to life.
An unplanned sonic intervention has occurred. The smoke alarm has its own desires; it insists on total control of the sonic space. Susan’s response is a necessary modulation. She counters the smoke alarm’s desire for sonic control with words, saying that it triggered accidentally because of the steak, sizzling in the pan. The interruption of the smoke alarm exemplifies how Susan’s cooking technique is not one of dominance. Rather than producing clear boundaries between chef and eater, the food and the preparation, the kitchen and its outside, Susan allows for fluid boundaries, welcoming chance and the unknown into her art.
In the context of Susan’s kitchen, Massumi’s definition of modulation applies, however subtle. These domestic modulations are not a movement toward total control, but, instead, a lively negotiation with a set of partly unpredictable relations – an orchestration of the sonic space. The idea of sonic orchestration allows us to consider the complex set of possibilities existing between the choices made by the subject, here, the chef, and the presence of a set of potentialities, such as the smoke alarm. To Susan, the art of cooking is not the reduction or elimination of “threats”; her art is the negotiation of modulations. In contrast to Chopped, where careful boundaries are constructed in order to protect the privilege inherent in its definition of art, Susan’s art lies in her engagement with the lively potentialities of the sonic art of cooking.
Seth Mulliken is a Ph.D. candidate in the Communication, Rhetoric, and Digital Media program at NC State. He does ethnographic research about the co-constitutive relationship between sound and race in public space. Concerned with ubiquitous forms of sonic control, he seeks to locate the variety of interactions, negotiations, and resistances through individual behavior, community, and technology that allow for a wide swath of racial identity productions. He is convinced ginger is an audible spice, but only above 15khz.
REWIND! . . .If you liked this post, you may also dig:
Sound-politics in São Paulo, Brazil–Leonardo Cardoso
When I performed at the 2012 Computers and Writing Conference in Raleigh, North Carolina, I looked around during my fairly abstract 10-minute long improvisation featuring feedback loops, glitches, silences, and circuit-bent instruments, and I noticed the audience’s sometimes visible restlessness, discomfort, and even anxiety. This is a fairly common occurrence when I perform experimental sound art, particularly in contexts in which audiences expect “music” (you can hear my work at 38:30 in the video below). However, for an experimental sound artist to take offense to such reactions is, in my estimation, missing the point of the exercise. That sound art disrupts, agitates, and even offends is a powerfully reaffirming reminder that sound art transcends music and sound; it is a method of revelation, an act that surpasses logical communication, instead challenging the very nature of sound and perception.
As an artist, scholar, and fan, I am drawn toward sound and music that lures me into a new world, an unfamiliar way of being and knowing. Like Lewis Carroll’s Alice, I learn that the rules of my world no longer apply. This happened when I heard J Dilla’s Donuts album, and when I heard Madlib’s Medicine Show #3: Beat Konducta in Africa, when I heard Miles Davis’ Bitches Brew. An artist that continually draws me down the rabbit hole is Walter Gross, an experimental sound/beat artist out of Los Angeles. His work changes the way I usually interact with sonic art, both in terms of his sound and in his approach to physical collage and handcrafted cassette packaging, Gross departs from the comfortable and familiar listening imparted by polished hi-fi 3-minute tracks with definitive beginnings and ends and discernible melodies. Gross instead propels listeners into very unusual (and pleasantly discomforting) soundscapes that demand attention. Almost counter-intuitively, Gross’s visual representations of his work intensify that experience. Consider his 2010 work, Dopamine:
Dopamine is likely a challenging piece for audiences, at least in terms of violating the dominant structures of music. The piece opens with disorienting use of panning, deliberately obscuring degraded audio, largely indiscernible movements and patterns, and so on. His video work likewise presents a fitting yet relatively unusual juxtaposition of youth and destruction, celebration and danger. In terms of both sound and sight, Gross’ work disrupts dominant musical sensibilities, challenging the very patterns and structures within which we can express ideas. He violates tradition, shakes off the canonical baggage carried by prevailing paradigms of Art and Music, and plunges audiences into unfamiliar sensory experiences that require metacognition, reflection, and examination of what sonic art is, and more importantly, what sonic art can be. Gross, in other words, seems to transcend the musician moniker and reach something else entirely. In what follows, I’d like to explore a (very brief) history of such artists, and begin to think about how to frame sonic art as immersion in what Marshall McLuhan called anti-environments: the unconscious environment as raised to conscious attention.
Sound as Art
There exists a strong tradition of experimental noise and sound art, particularly in 20th-century Western avant-garde movements. Futurists were arguably the first to consider noise as music in the European tradition, and were certainly influential in asking artists and audiences to become more aware of the changing social and sonic surroundings . In his 1913 manifesto-of-sorts titled “The Art of Noises,” Italian Futurist Luigi Russolo proposed an orchestral configuration that more aptly represented the range of sounds available to contemporary listeners, namely those sounds that accompanied industrialization and urbanization. The sounds of the Futurist orchestra would include “rumbles, roars, explosions, and crashes.” Russolo built devices called intonarumori to mechanically achieve and manipulate these sounds. His brother, Antonio Russolo, also enacted this new philosophy of modern found sound and composed Corale and Serenata.
Any inquiry of art as anti-environment would be incomplete without a discussion of the great anti-art movement, Dada. Like the Futurists before them, Dadaists used found sound and technology-as-art to violently disrupt conventions of art, beauty, and authorship within the white avant-garde community. Marcel Duchamp’s famous work, “Fountain,” is likely the most familiar Dadaist artifact to contemporary readers, yet the sound poetry of Kurt Schwitters and other Dadaist and Dada-inspired sound pieces such as Erwin Schulhoff’s 1922 work In Futurum (the middle movement of which contains only a rest and the notation “with feeling,” an undoubtable precursor to John Cage’s 4’33”, written 30 years later) created sonic spaces of innovation and strangeness that changed the way audiences listened to both voices and silences. The Russian Cubo-Futurists, especially zaumniks such as Alexei Kruchenykh, made similar ventures into anti-environments. Kruchenykh developed the sound art zaum, which he understood as a transrational language that undercut existing language systems in which the “word [had] been shackled…by its subordination to rational thought” (70). Zaum was a sort of linguistic anti-environment, one rooted in the notion that meaning resided first and foremost in the sound of a word rather than the denotative symbol system that emerged alongside the proliferation of print/visual culture. One could also not underemphasize the work of John Cage, from his prepared piano to his work with organic instruments.
The list of artists, genres, and movements engaged to some extent in the enterprise of anti-environment architecture could go on and be debated indefinitely: Free Jazz, Turntablism/Nu Jazz, Experimental Hip-Hop,Fluxus, Circuit Bending, Prepared Guitar, ProtoPunk, Punk, Post-Punk, New Wave, No Wave. . . in all of these diverse movements, the sonic artists share the tendency to create strange new worlds via sound; worlds that reveal social and technological environments that most people seem unaware of in the moment. This is why media theorist Marshall McLuhan called the artist “indispensible,” because the artist can tell us something about ourselves that we cannot know via ordinary means of perception. Sonic artists expose audiences to auditory phenomena, structures, juxtapositions, etc. that are to various extents hidden, obscured, or ignored as “noise.” The sonic artist is more than just a clever selector and (re)arranger of sound; s/he is a revelatory agent, exposing what is inaudible.
Art as Anti-environment
Anti-environments, however we might define and classify them, are vital not only to artistic communities themselves, but they are also vital to a society of fish in water. In his 1968 text, War and Peace in the Global Village, McLuhan asserts (among other things) that humans remain largely unaware of their new environments, likening them to fish in water: “one thing about which fish know exactly nothing is water, since they have no anti-environment which would enable them to perceive the element they live in” (175). In other words, humans seldom possess or practice a sense of awareness regarding their surroundings because there’s nothing against which surroundings may be contrasted. The “water” to McLuhan represented the various environments (physical, psychological, cultural) shaped by technological innovation, but we can—and should—extend the water metaphor to a range of hegemonic frameworks: constructions of gender, race, ability, and so on.
This essay is certainly not an attempt to generate some sort of evaluative rubric by which to judge artistic or sonic expression objectively. Rather, we might use the concept of anti-environments as a way to frame our subjective experiences and encounters with all sound, and begin listening to unfamiliar sounds as psychedelic (from Greek psyche- “mind” + deloun “reveal”) keys to illuminate the patterns and structures in which listeners exist. We must work to understand our environments and our place in them; if we are to engage critically with our culture, we must first understand existing (yet invisible) patterns and structures that surround us. And we are aided in this effort, in great part, by humanity’s great seekers of pattern recognition, the sonic-psychonautical messengers: the sonic artists.
To return to the sound that inspired this meditation, Walter Gross (among others) is in many ways participating in and propelling the discourse of Leary and McLuhan, Schwitters and Schulhoff, Kruchenykh and Cage,Davis and Sun Ra, Madlib and J Dilla. Gross performs the sonic anti-environment, enacts the revelation of obscured sonic paradigms. For me, Gross can act as a sort of lens through which ordinary sonic patterns and structures become visible. I hear Flying Lotus, Bob Dylan, and The Minutemen differently after Gross. I hear my office, my home, my family’s voices differently after Gross. I hear patterns that weren’t audible before. After Gross, I become aware of how I am continuously trained to expect certain things from the sonic world: compartmentalized units of meaning, clearly stated origins of utterances, linear narratives, repeated/repeatable melodies, and so on.
Likewise, my own sonic art/scholarship approaches the use of sound to reveal the inaudible assumptions present in Western frameworks surrounding sonic production. I will conclude with an illustration of my own work and why sonic anti-environments are so central to my philosophy and method. One of my sonic works, “Toward an Object-Oriented Sonic Phenomenology,” was recently part of an exhibition titled Not For Human Consumption, curated by Julian Weaver of CRISAP in London. I recorded the sounds of a high mast lighting pole using contact microphones. Contact microphones do not “hear” like humans typically hear. Typical (dominant) notions of human hearing (and therefore of sound itself) involve the reception and interpretation of vibrations present in air. Contact microphones instead only interpret the vibrations in solid objects.
By listening through an object–through alien “ears,” so to speak– we can begin to critique the ways that we privilege listening via air, a listening that places humans at the center of the universe. We can consider the ways that sound has very real effects on humans with atypical hearing abilities and nonhuman objects. It is difficult to have such conversations if we never explore sonic anti-environments, if we never break through dominant epistemological models, if we never expose the limits of our own environments.
Featured Image: Beatrix*JAR in Dayton, Ohio, September 9, 2009, by Flickr User Vista Vision
Steven Hammer is a Ph.D. candidate in Rhetoric, Writing, and Culture at North Dakota State University in Fargo, ND, USA. His research deals with various aspects of sonic art, from exploring glitch and proto-glitch practices and theories (e.g., circuit bending), to understanding and producing sound from an object-oriented ontology (e.g., contact microphones). He also researches and facilitates trans-Atlantic translation collaborations between American, European, and African universities. He has multimedia publications with Enculturation, Sensory Studies, as well as forthcoming book chapters with Wiley/IEEE press, and IGI Global Publishing, and has performed creative and academic work at several conferences across North America, including the national Computers and Writing Conference and the Council for Programs in Technical and Scientific Communication. He performs experimental circuit-bent and sampler-based music under the moniker “patchbaydoor,” and has constructed and documented a number of hardware modification projects for his own artistic projects and for other artists in the upper Midwest United States. You can read/hear more atstevenrhammer.com