This post continues our summer Sound and Pleasure series, as the third and final podcast in a three part series by Leonard J. Paul. What is the connection between sound and enjoyment, and how are pleasing sounds designed? Pleasure is, after all, what brings y’all back to Sounding Out! weekly, is it not?
Part of the goal of this series of podcasts has been to reveal the interesting and invisible labor practices which are involved in sound design. In this final entry Leonard J. Paul breaks down his process in designing living sounds for the game Vessel. How does one design empathetic or aggressive sounds? If you need to catch up read Leonard’s last entry where he breaks down the vintage sounds of Retro City Rampage. Also, be sure to be sure to check out last week’s edition where Leonard breaks down his process in designing sound for Sim Cell. But first, listen to this! -AT, Multimedia Editor
CLICK HERE TO DOWNLOAD: Game Audio Notes III: The Nature of Sound in Vessel
SUBSCRIBE TO THE SERIES VIA ITUNES
ADD OUR PODCASTS TO YOUR STITCHER FAVORITES PLAYLIST
Game Audio Notes III: The Nature of Sound in Vessel
Strange Loop Game’s Vessel is set in an alternate world history where a servant class of liquid automatons (called fluros) has gone out of control. The player explores the world and solves puzzles in an effort to restore order. While working on Vessel, I personally recorded all of the sounds so that I could have full control over the soundscape. I recorded all of the game’s samples with a Zoom H4n portable recorder. My emphasis on real sounds was intended to focus the player’s experience of immersion in the game.
This realistic soundscape was supplemented with a variety of techniques that produced sounds that dynamically responded to the changes in the physics engine. Water and other fluids in the game were difficult to model with both the physics engine and the audio engine (FMOD Designer). Because fluids are fundamentally connected to the game’s physics engine, they takes on a variety of different dynamic forms as players interact with the fluid in different ways. In order to address this Kieran Lord, the audio coder, and I considered factors like the amount of liquid in a collision with anything, the hardness of the surface that it was colliding with, the type of liquid in motion, whether the player is experiencing an extreme form of that sound because it is colliding with their head, and, of course, how fast the liquid is travelling.
Although there was a musical score, I designed the effects to be played without music. Each element of the game, for instance a lava fluro’s (one of the game’s rebellious automatons) footsteps, entailed required layers of sound. The footsteps were composed of water sizzling on a hot pan, a gloopy slap of oatmeal and a wet rag hitting the ground. Finding the correct emotional balance to support the game’s story was fundamental to my work as a sound designer. The game’s sound effects were constantly competing with the adaptive music (which is also contingent on player action) that plays throughout the game, so it was important to provide an informative quality to them. The sound effects inform you about the environment while the music sets the emotional underscore of the gameplay and helps guide you in the puzzles.
Defining the character of the fluros was difficult because I wanted players to have empathy for them. This was important to me because there is often no way to avoid destroying them when solving the game’s puzzles. While recording sounds in the back of an antique shop, I came across a vintage Dick Tracey gun that made a fantastic clanking sound when making a siren sound. Since the gun allowed me to control how quickly the siren rose and fell, it was a great way to produce vocalizations for the fluros. I simply recorded the gun’s siren sound, chopped the recording into smaller pieces, and then played back different segments randomly. The metal clanking gave a mechanical feel and the siren’s tone gave a vocal quality to the resulting sound that was perfect for the fluros. I could make the fluros sound excited by choosing a higher pitch range from the sample grains and inform the player when they approached their goal.
I wanted a fluid-based scream to announce a fluro’s death. I tried screaming underwater, screaming into a glass of water, and a few other things, but nothing worked. Eventually, when recording a rubber ear syringe, I found squeezing the water out quickly lent a real shriek while it spit out the last of the water. Not only did this sound really cut through the din of the gears clanking in the mix, but it also bonded a watery yell with the sense of being crushed and running out of breath.
For the final boss, I tried many combinations of glurpy sounds to signify its lava form. Eventually I recorded a nail in a board being dragged across a large rusty metal sheet. Though it was quite excruciating to listen to, I pitched down the recording and combined it with a pitched down and granulated recording of myself growling into a cup of water. This sound perfectly captured the emotion I wanted to feel when encountering a final boss. Although it can take a long time to arrive at the “obvious” sound, simplicity is often the key.
Anticipation is fundamental to a player’s sense of immersion. It carves a larger space for tension to build, for instance a small crescendo of a creaking sound can develop a tension that builds to a sudden and large impact. A whoosh before a punch lands adds extra weight to the force of the punch. These cues are often naturally present in real-world sounds, such as a rush of air sweeping in before a door slams. A small pause might be included just for added suspense and helps to intensify the effect of the door slamming. Dreading the impact is half of the emotion of a large hit .
Recording all of the sounds for Vessel was a large undertaking but since I viewed each recording as a performance, I was able to make the feeling of the world very cohesive. Each sound was designed to immerse the player in the soundscape, but also to allow players enough time to solve puzzles without becoming annoyed with the audio. All sounds have a life of their own and a resonance of memory and time that stays with the them during each playthrough of a game. In Retro City Rampage I left a sonic space for the player to wax nostalgic. In Sim Cell, I worked to breathe life into a set of sterile and synthesized sounds. Each recorded sound in Vessel is alive in comparison, telling stories of time, place and recording with them, that are all their own.
The common theme of my audio work on Retro City Rampage, Sim Cell and Vessel, is that I enjoy putting constraints on myself to inspire my creativity. I focus on what works and removing non-essential elements. Exploring the limits of constraints often provokes interesting and unpredictable results. I like “sculpting” sounds and will often proceed from a rough sketch, polishing and reducing elements until I like what I hear. Typically I remove layers that don’t add an emotive aspect to the sound design. In games there are often many sounds that can play at once, so clarity and focus are necessary when preventing sounds from getting lost in a sonic goo.
In this post I have shown how play and experimentation are fundamental to my creative process. For an aspiring sound artist, spending time with Pure Data, FMOD Studio or Wwise and a personal recorder is a great way to improve their skill with game audio. This series of articles has aimed to reveal the tacit decisions behind the production of game audio that get obscured by the fun of the creative process. Plus, I hope they offer a bit of inspiration to those creating their own sounds in the future.
- My GDC 2012 talk – My site includes an MP3 recording of the talk and the full powerpoint presentation. A full video of the talk is in the GDC Vault but requires membership
- See the Jon Hopkins Soundtrack in the Video Game Vessel video game soundtrack video by me on YouTube – shows use of Lua with music in the custom level editor : https://www.youtube.com/watch?v=KOyjMPPvaY4
Leonard J. Paul attained his Honours degree in Computer Science at Simon Fraser University in BC, Canada with an Extended Minor in Music concentrating in Electroacoustics. He began his work in video games on the Sega Genesis and Super Nintendo Entertainment System and has a twenty year history in composing, sound design and coding for games. He has worked on over twenty major game titles totalling over 6.4 million units sold since 1994, including award-winning AAA titles such as EA’s NBA Jam 2010, NHL11, Need for Speed: Hot Pursuit 2, NBA Live ’95 as well as the indie award-winning title Retro City Rampage.
He is the co-founder of the School of Video Game Audio and has taught game audio students from over thirty different countries online since 2012. His new media works has been exhibited in cities including Surrey, Banff, Victoria, São Paulo, Zürich and San Jose. As a documentary film composer, he had the good fortune of scoring the original music for multi-awarding winning documentary The Corporation which remains the highest-grossing Canadian documentary in history to date. He has performed live electronic music in cities such as Osaka, Berlin, San Francisco, Brooklyn and Amsterdam under the name Freaky DNA.
He is an internationally renowned speaker on the topic of video game audio and has been invited to speak in Vancouver, Lyon, Berlin, Bogotá, London, Banff, San Francisco, San Jose, Porto, Angoulême and other locations around the world.
His writings and presentations are available at http://VideoGameAudio.com
Featured image: Courtesy of Vblank Entertainment (c)2014 – Artwork by Maxime Trépanier.
REWIND! . . .If you liked this post, you may also dig:
Sounding Out! Podcast #31: Hand Made Music in Retro City Rampage– Leonard J. Paul
Papa Sangre and the Construction of Immersion in Audio Games- Enongo Lumumba-Kasongo
As the practice of sound design becomes ever more refined as a key factor in the immersive aspects of gameplay, it is essential to develop a conceptual vocabulary of the ways that sound is implemented as a cultural facet. In particular, it is important to recognize the power relations at stake within the implementation of the human voice as an interactive narrative trope. And, while I’ve already discussed the ways in which the voice of GLaDOS in Portal invites players to reflect on how they internalize a set of mediated perspectives about how their body ought to be, it is equally important to consider the other ways that a narrator’s voice invites players to reconsider the intersection of agency and surveillance.
This post compares the use of narration in Bastion and The Stanley Parable in an effort to understand how the voice is used as what Karen Collins would refer to as an “interactive non-diagetic sound,” or, in other words, a sound that is triggered by player actions, but not experienced by the character in the game. Specifically, I argue that the voice in these examples is an essential point in the feedback loop between player and game. And, as part of the cybernetics of gameplay, it produces a dispositif of surveillance, akin to Bentham’s panopticon, which lets the player know their actions are constantly being monitored, calculated, and considered by the game’s algorithms. But, while the original panopticon produced the effect of surveillance through the clever use of light, these games use sound to effect surveillance.
Bastion, was developed by the small indie game company Supergiant Games, but was distributed and released by Warner Bros. Interactive Entertainment, first through Microsoft’s distribution service, XBOX Live Arcade, but has since been released more broadly after receiving much critical acclaim. In Bastion, players take the role of a young boy, referred to only as “The Kid,” and adventure around gradually rebuilding a world that has fallen apart since a cataclysmic sundering referred to only as “the Calamity.” Although the world of Bastion is beautiful and visually stimulating, it is the game’s sound design that has earned it much critical acclaim. The game is narrated by a character named “Rucks,” who speaks in a deep weathered voice, with somewhat of a western twang. And, even though The Kid eventually encounters and is able to interact with Rucks within the game, Rucks still relays dialogue in the third-person.
When The Kid and Rucks meet, Rucks evinces with trademark grit, “Sure enough, he finds another. He finds me.” And, while that is a scripted plot point within the game, at other points Rucks’ narration modulates to best reflect the player’s actions. A player who begins the game slowly, exploring nooks and crannies, might hear “The kid walks slowly down the path, checking everything,” while a player who runs straight ahead could hear, “The kid barrels forward, not looking once behind him.” These quotes force the player to recognize that the game is watching, and actively staging narrative commentary about their in-game decisions. This commentary unfolds in aural space, through narration, discrete from the old text (and controller) triggers of “look” and “examine” which used to prompt text-box commentary about the environment of the game. In short, Bastion’s sound design succeeds because it is balanced in such a precise way: players are being constantly engaged with a narration which confirms that they are, in fact, properly interacting within the game world and story.
The Stanley Parable, on the other hand, works deliberately to turn the paradigm of narration on its head. Where Rucks, in Bastion, frequently alluded to how many secrets he was yet to reveal about himself and the game-world, his character ultimately plays a supportive role, helping The Kid to understand the chaotic environment of the game. The narrator in The Stanley Parable, however, plays the antagonist in many ways, attempting to foreshadow and predetermine the actions of the player, or “Stanley.” On the game’s website, a short sentence contextualizes the endeavor, “The Stanley Parable is a Half Life 2 mod about video games.” The game itself is a mod of the “Source engine,” which runs both Half-Life 2, and Portal, was developed by the very small development team of Davey Wreden and William Pugh, and released for free. It is meta-fiction that stages a critique of the context of narrative within interactive games and fiction. Specifically, the game questions the idea of narrative itself by showcasing the ways that players are able to undermine the scripted plots and spaces of a videogame by exploring and experimenting with exploits and bugs in the game’s code and narrative.
Although the narrator in The Stanley Parable will prescribe several decisions to the player over the course of the game, the player is given the agency to contest the story as told by the narrator, and, therefore, to experiment with the plot. As the player reaches a set of two open doors on the way to the employee lounge the narrator reads, “When Stanley came to a set of two open doors, he entered the door on his left.” If the player chooses to travel through the door on the right instead, the narrator will attempt to steer the player back to the main plot tree by saying, “This was not the correct way to the employee lounge, and Stanley knew it perfectly well.” And, then as an open door is revealed, “So he turned left at the first open door, and walked back in the right direction.” If the player continues to ignore the narrator’s advice, he comes to somewhat of a dead end and the narrator reads, “Stanley was so bad at following directions, it’s incredible that he wasn’t fired years ago. Maybe this was why everyone left. No one wanted to be around someone as bad at listening as him.” The player is given several other opportunities to make decisions and lead the story to completion in achieving one of seven endings; each based the decisions the player has made when interacting with the narrator’s dialogue.
The Stanley Parable allows players the agency to see the limitations of linear storytelling where Bastion does not. Where some paths in The Stanley Parable will lead the player into direct conflict with the narrator, other paths do not. There is never a point where the narrator ceases to comment on the player’s actions and activities. Because the sonic feedback of surveillance remains a constant in both games the player remains engaged, in both cases, with the logic of the game-system. In other words, no matter how many times the player defies the narrator in The Stanley Parable, it never seems like the game is breaking. The game world remains constant because the motif of surveillance holds; players know the game still works because the narrator continues to stage commentary – even if it is commentary about the player’s failure to keep to the plot.
For Marc Andrejevic, author of iSpy (2007)—who has written extensively about the abundance of surveillance techniques implemented in digital spaces–the danger of surveillance lies in the production of an asymmetrical power relationship between media producer and media consumer. And, while this is certainly best argued about instances of dataveillance–how companies like Amazon, for example, track customer clicks on and off their website via web cookies in order to better produce exploitable (and in some cases saleable) consumer profiles–it is important to also consider the ways that the implementation of sound also functions as a technique of control.
At its most positive, the sonic panopticism of Bastion and The Stanley Parable offer players a sense of comfort in knowing that the game is operating properly, and not glitching out. Further, players are invited into a more immersive game, which leverages both visual and audio interactivity to lull players into an environment of almost trancelike feedback and play. Clearly, this is the promise of good sound design; it gently alerts players to the presence of a tightly designed and well-implemented game, and produces affects of brand loyalty and trust within a game’s player contingent.
But, while there are clearly aesthetic and market benefits to the implementation of narration in both games, one cannot help but wonder, in the context of post-feminism and self-surveillance, what implications there are in the implementation of the male voice as surveil-er in both games. Just as it was curious in Portal 2 how GLaDOS acted as a critical female voice constantly judging the player’s body image and intelligence, it is curious how much authority is given to the voice of Rucks in Bastion. And while several good critiques have already been written about how the game features only one (somewhat silent, and certainly helpless) female character, and how the game’s villain is portrayed, concretely, as the racially exotic other, it is sadly fitting that the most comforting and well-acclaimed aspects of the game come from the interactivity produced by the voice of its distinctively white male narrator.
The sound design in The Stanley Parable, of course, is more cutting in the ways it stages a commentary about how the voice of the narrator (this time distinctively British), exacts a form of social coercion through techniques of surveillance, and how these techniques serve, namely, to hamper player agency. But, even its own narrative of resistance fails to compel; in fact, it is the uneasy ending of compliance and conformity that is, perhaps, the happiest. This, ironically, reveals one of the key cultural problems of our era: the reciprocal aspects of surveillance and interactivity. If affective resonances of trust, knowledge, and comfort come bundled with the male voice, is it in the vested economic interests of sound design communities to leverage these to make profit? Even though both games have earned critical praise, it is only Bastion that has won awards for sound design. In other words, are we caught in our own feedback loop of comfort, industry, and design?
Aaron Trammell is co-founder and Multimedia Editor of Sounding Out! He is also a Media Studies PhD candidate at Rutgers University. His dissertation explores the fanzines and politics of underground wargame communities in Cold War America. You can learn more about his work at aarontrammell.com.
REWIND! . . .If you liked this post, you may also dig:
DIANE… The Personal Voice Recorder in Twin Peaks— Tom McEnaney
The Role of Sound in Video Games: Pong, Limbo, and Interactivity— Aaron Trammell
Orality and Cybernetics in Battleship— Aaron Trammell