Tag Archive | sound design

Sounding Out! Podcast #31: Game Audio Notes III: The Nature of Sound in Vessel

Sound and Pleasure2This post continues our summer Sound and Pleasure series, as the third and final podcast in a three part series by Leonard J. Paul. What is the connection between sound and enjoyment, and how are pleasing sounds designed? Pleasure is, after all, what brings y’all back to Sounding Out! weekly, is it not?

Part of the goal of this series of podcasts has been to reveal the interesting and invisible labor practices which are involved in sound design. In this final entry Leonard J. Paul breaks down his process in designing living sounds for the game Vessel. How does one design empathetic or aggressive sounds? If you need to catch up read Leonard’s last entry where he breaks down the vintage sounds of Retro City Rampage. Also, be sure to be sure to check out last week’s edition where Leonard breaks down his process in designing sound for Sim Cell. But first, listen to this! -AT, Multimedia Editor

CLICK HERE TO DOWNLOADGame Audio Notes III: The Nature of Sound in Vessel

SUBSCRIBE TO THE SERIES VIA ITUNES

ADD OUR PODCASTS TO YOUR STITCHER FAVORITES PLAYLIST


Game Audio Notes III: The Nature of Sound in Vessel

Strange Loop Game’s Vessel is set in an alternate world history where a servant class of liquid automatons (called fluros) has gone out of control. The player explores the world and solves puzzles in an effort to restore order. While working on Vessel, I personally recorded all of the sounds so that I could have full control over the soundscape. I recorded all of the game’s samples with a Zoom H4n portable recorder. My emphasis on real sounds was intended to focus the player’s experience of immersion in the game.

This realistic soundscape was supplemented with a variety of techniques that produced sounds that dynamically responded to the changes in the physics engine. Water and other fluids in the game were  difficult to model with both the physics engine and the audio engine (FMOD Designer). Because fluids are fundamentally connected to the game’s physics engine, they takes on a variety of different dynamic forms as players interact with the fluid in different ways. In order to address this Kieran Lord, the audio coder, and I considered factors like the amount of liquid in a collision with anything, the hardness of the surface that it was colliding with, the type of liquid in motion, whether the player is experiencing an extreme form of that sound because it is colliding with their head, and, of course, how fast the liquid is travelling.

Although there was a musical score, I designed the effects to be played without music. Each element of the game, for instance a lava fluro’s (one of the game’s rebellious automatons) footsteps, entailed required layers of sound. The footsteps were composed of water sizzling on a hot pan, a gloopy slap of oatmeal and a wet rag hitting the ground. Finding the correct emotional balance to support the game’s story was fundamental to my work as a sound designer. The game’s sound effects were constantly competing with the adaptive music (which is also contingent on player action) that plays throughout the game, so it was important to provide an informative quality to them. The sound effects inform you about the environment while the music sets the emotional underscore of the gameplay and helps guide you in the puzzles.

The lava fluro foosteps in FMOD Designer.

The lava fluro foosteps in FMOD Designer. Used with permission (c) 2014 Strange Loop Games

Defining the character of the fluros was difficult because I wanted players to have empathy for them. This was important to me because there is often no way to avoid destroying them when solving the game’s puzzles. While recording sounds in the back of an antique shop, I came across a vintage Dick Tracey gun that made a fantastic clanking sound when making a siren sound. Since the gun allowed me to control how quickly the siren rose and fell, it was a great way to produce vocalizations for the fluros. I simply recorded the gun’s siren sound, chopped the recording into smaller pieces, and then played back different segments randomly. The metal clanking gave a mechanical feel and the siren’s tone gave a vocal quality to the resulting sound that was perfect for the fluros. I could make the fluros sound excited by choosing a higher pitch range from the sample grains and inform the player when they approached their goal.

I wanted a fluid-based scream to announce a fluro’s death. I tried screaming underwater, screaming into a glass of water, and a few other things, but nothing worked. Eventually, when recording a rubber ear syringe, I found squeezing the water out quickly lent a real shriek while it spit out the last of the water. Not only did this sound really cut through the din of the gears clanking in the mix, but it also bonded a watery yell with the sense of being crushed and running out of breath.

Vessel-LavaBoss

Vessel’s Lava boss with audio debug output. Used with permission (c) 2014 Strange Loop Games

For the final boss, I tried many combinations of glurpy sounds to signify its lava form. Eventually I recorded a nail in a board being dragged across a large rusty metal sheet. Though it was quite excruciating to listen to, I pitched down the recording and combined it with a pitched down and granulated recording of myself growling into a cup of water. This sound perfectly captured the emotion I wanted to feel when encountering a final boss.  Although it can take a long time to arrive at the “obvious” sound, simplicity is often the key.

Anticipation is fundamental to a player’s sense of immersion. It carves a larger space for tension to build, for instance a small crescendo of a creaking sound can develop a tension that builds to a sudden and large impact. A whoosh before a punch lands adds extra weight to the force of the punch. These cues are often naturally present in real-world sounds, such as a rush of air sweeping in before a door slams. A small pause might be included just for added suspense and helps to intensify the effect of the door slamming. Dreading the impact is half of the emotion of a large hit .

Vessel-ClockRecording

Recording inside of a clock tower with my H4n recorder for Vessel. Used with permission by the author.

Recording all of the sounds for Vessel was a large undertaking but since I viewed each recording as a performance, I was able to make the feeling of the world very cohesive. Each sound was designed to immerse the player in the soundscape, but also to allow players enough time to solve puzzles without becoming annoyed with the audio. All sounds have a life of their own and a resonance of memory and time that stays with the them during each playthrough of a game. In Retro City Rampage I left a sonic space for the player to wax nostalgic. In Sim Cell, I worked to breathe life into a set of sterile and synthesized sounds. Each recorded sound in Vessel is alive in comparison, telling stories of time, place and recording with them, that are all their own.

The common theme of my audio work on Retro City Rampage, Sim Cell and Vessel, is that I enjoy putting constraints on myself to inspire my creativity. I focus on what works and removing non-essential elements. Exploring the limits of constraints often provokes interesting and unpredictable results. I like “sculpting” sounds and will often proceed from a rough sketch, polishing and reducing elements until I like what I hear. Typically I remove layers that don’t add an emotive aspect to the sound design. In games there are often many sounds that can play at once, so clarity and focus are necessary when preventing sounds from getting lost in a sonic goo.

CherryBlossoms

Cherry blossoms for new beginnings. Used with permission by the author.

In this post I have shown how play and experimentation are fundamental to my creative process. For an aspiring sound artist, spending time with Pure Data, FMOD Studio or Wwise and a personal recorder is a great way to improve their skill with game audio. This series of articles has aimed to reveal the tacit decisions behind the production of game audio that get obscured by the fun of the creative process. Plus, I hope they offer a bit of inspiration to those creating their own sounds in the future.

Additional Resources:

Leonard J. Paul attained his Honours degree in Computer Science at Simon Fraser University in BC, Canada with an Extended Minor in Music concentrating in Electroacoustics. He began his work in video games on the Sega Genesis and Super Nintendo Entertainment System and has a twenty year history in composing, sound design and coding for games. He has worked on over twenty major game titles totalling over 6.4 million units sold since 1994, including award-winning AAA titles such as EA’s NBA Jam 2010NHL11Need for Speed: Hot Pursuit 2NBA Live ’95 as well as the indie award-winning title Retro City Rampage.

He is the co-founder of the School of Video Game Audio and has taught game audio students from over thirty different countries online since 2012. His new media works has been exhibited in cities including Surrey, Banff, Victoria, São Paulo, Zürich and San Jose. As a documentary film composer, he had the good fortune of scoring the original music for multi-awarding winning documentary The Corporation which remains the highest-grossing Canadian documentary in history to date. He has performed live electronic music in cities such as Osaka, Berlin, San Francisco, Brooklyn and Amsterdam under the name Freaky DNA.

He is an internationally renowned speaker on the topic of video game audio and has been invited to speak in Vancouver, Lyon, Berlin, Bogotá, London, Banff, San Francisco, San Jose, Porto, Angoulême and other locations around the world.

His writings and presentations are available at http://VideoGameAudio.com

Featured image: Courtesy of Vblank Entertainment (c)2014 – Artwork by Maxime Trépanier.

tape reelREWIND! . . .If you liked this post, you may also dig:

Sounding Out! Podcast #30: Game Audio Notes I: Growing Sounds for Sim Cell- Leonard J. Paul

Sounding Out! Podcast #31: Hand Made Music in Retro City Rampage– Leonard J. Paul

Papa Sangre and the Construction of Immersion in Audio Games- Enongo Lumumba-Kasongo 

Sound at IASPM-US 2014

For the second weekend in March, the U.S. chapter of the International Association for the Study of Popular Music (IASPM-US) will be holding its annual meeting at the University of North Carolina, Chapel Hill. Like sound studies, popular music studies is fueled by an interdisciplinary spirit, and many of the questions that currently occupy the popular culture corner of sound studies have much in common with those of us who take the study of music seriously. This year’s conference offers a unique theme, “Music Flows,” that centers around questions of water, flows, and liquidity. The conference theme also offers more expansive ideas to flows, including mobility, embodiment, sonic materialities, and ecology. While the theme may strike some as unconventional, it ends up being an excellent metaphor for those of us who study musical flows in fields that prefer static works and communities over transient ones.

The James Taylor Bridge in his native city of Chapel Hill, North Carolina

The James Taylor Bridge over Morgan Creek in his native city of Chapel Hill, North Carolina

Since this is a popular music conference, many of the papers at this meeting take musical texts as their focus (including mine); however, there are still many panels and individual papers that might interest scholars from a sound studies perspective. After all, sound travels better in water than in air, and following that logic, water and sound both feature waves. Indeed, there are papers that take the water waves and sound waves as their inspiration. Compare, for example, SO! guest writer Mack Hagood’s discussion of an early popular recording of water waves against Robin James’s philosophical theorizing about sound waves and Neo-Liberalism in the music of Ludacris. Similarly, many papers take their inspiration from the sounds that come from water or are performed in it: Peter Schultz specifically tackles the sound-design of watery environments in video games, while SO! guest writer Josh Ottum investigates the sounds from the floating garbage island in the middle of the ocean. These papers offer attendees the opportunity to consider the large theoretical consequences of changes to the water and waves in recordings.

Some papers approach water from a perspective focused on materiality and mediation. Craig Eley’s paper offers a historically grounded study of the hydrophone and underwater recording. Peter McMurray’s paper analyzes the problem of making music for watery environments and the challenges of water’s sonic conduciveness. For an athletic perspective on hearing music in the water, Niko Higgins talks about the music that swimmers use in their athletic training. These perspectives on liquid mediation offer a tremendous opportunity to expand sound studies beyond its general dependence on sounds that happen in the open air.

Beyond the more literal takes on the water in music flows, a large portion of the papers have taken their inspiration from the metaphor of social mobility, liquidity, and trade. There are panels and papers that emphasize transnational sonic flows, such as the panel “In and Out of Africa,” and Jason Robinson’s work on recording challenges in a transatlantic jazz collaboration. Two papers in particular deal with the role of African Americans in U.S. diplomatic relations: Darren Mueller’s paper on Dizzy Gillespie as a jazz ambassador, and Kendra Salois’s work on hip-hop diplomacy. Along a similar vein, Yvonne Liao specifically considers ports and their relationship to musical trade in Shanghai’s jazz scene. There is also a paper on the role of music as a social lubricant by Luis-Manuel Garcia that promises to be a real treat.

Megafaun serenades a Chapel Hill, North Carolina crowd, Image by Flickr user  abbyladybug

Megafaun serenades a Chapel Hill, North Carolina crowd, Image by Flickr user abbyladybug

There are also numerous papers that tackle flow and water as a metaphor in music-making and mediation. They include SO! guest writer Mike D’Errico’s study of embodiment and interactivity in digital media, Rebecca Farrugia and Kelly Hay’s study of women’s flow in a Detroit hip-hop scene, and Jonathan Piper’s paper on “sludge” metal. “Anointing Sounds” is a roundtable on music’s materiality and the sounds of religious experience through the Christian metaphors of “anointing” and “healing waters.”  Finally, for those scholars seeking the rare paper on record eaters and collecting, check out SO! guest writer Shawn VanCour and Kyle Barnett’s paper.
.
Other highlights include a keynote by Louise Meintjes, whose book Sounds of Africa! took the musical recording process in studios as a serious object of study, and one of the last papers of the conference, Matthew Somoroff’s study of James Baldwin as a listener and ethnographer..

Finally, it is worth mentioning how many papers address sound studies’ long-standing relationship with soundscapes, ecomusicology, and the environment. There is a panel called “Ecologies of Place” with papers on ecologically-minded music from places as far flung as India, Iceland, Appalachian Ohio, and Canadian parks. There is also a panel on “Urban Soundscapes,” including Robert Fry’s paper on sound, music, and branding at a hot spring resort and Mathew Robert Swiatlowski’s paper on the boom box and the Walkman in urban space.

Many in sound studies cite Jonathan Sterne’s critiques of ocularcentrism in cultural criticism. This conference encourages us to think beyond the air and stasis and shift our focus to the possibilities of liquid metaphors in cultural change.

Scroll down for Kariann’s handpicked panels and papers of interest for sound studies folks perusing IASPM-US.  

Featured Image: One of Chapel Hill’s many ponds, at the Outdoor Education Center, Image by Flickr User Kat St Kat

Kariann Goldschmitt is an Adjunct Assistant Professor at New College of Florida and Ringling College of Art and Design. She holds a Ph.D. in Musicology from UCLA (2009) and was the 2009-2011 Mellon Fellow of Non-Western Music at Colby College in Maine. Her scholarly work focuses on Brazilian music, modes of listening, and sonic branding in the global cultural industries. She has published in The Journal of Popular Music Studies, American Music, Yearbook for Traditional Music, and Luso-Brazilian Review and contributes to the South American cultural magazine, Sounds and Colours.

"Franklin Street, Chapel Hill" by Wikimedia user Caroline Culler, CC BY 3.0

“Franklin Street, Chapel Hill” by Wikimedia user Caroline Culler, CC BY 3.0

Friday, March 14

9:30
“The Fluid “Field”: Recording and Performance in Transatlantic Collaboration”–Jason Robinson, Amherst College
 .
“Swimming What You Hear: The Music of Distance Swimmers”– Niko Higgins, Columbia University
.
10:15-11:45
“In and Out of Africa: From Biodiversity to Cultural Diversity: Negotiating Cultural Sustainability, Difference, and Nationhood through World Music in France,” Aleysia Whitmore, Brown University
 .
“American Afrobeat: Perception and Reception of Antibalas in Nigeria,” Stephanie Shonekan, University of Missouri
 .
“African Sounds in the American South: Community Radio, Pan-Africanism, and Historically Black Colleges, 1950-1986,” Joshua Clark Davis, Duke University
 .
“Thinking the Anthropocene Through Sound ‘Apeman’: The Kinks’ Romantic Expression of Environmental Politics and the Paradox of Human Evolution,”
Sara Gulgas, University of Pittsburgh
 .
“Coming of Age in the Post-3.11 Waterscape: Music and Silence in Japanese Animated Cinema and Children’s Art,” Kyle Harp, University of California, Riverside
.
“Sounds Like Garbage: Paddling Through an Island of Trash Toward a New Sonic Ecology,” Josh Ottum, Ohio University
.
“Watery Textualities: The Perceptual Flow of Metric (Re)evaluation in Radiohead’s ‘Bloom,'” Michael Lupo, CUNY Graduate Center
 .
“Splash, Bubble, and Clink: Topic and Timbre in Aquatic Video Game Environments,” Peter Shultz, University of Chicago
 .
“Just Ludacris Enough: Wave-Forms & Neoliberal Sophrosyne,” Robin James, University of North Carolina-Charlotte
 .
12:00-1:30
Keynote Lecture: Louise Meintjes, Duke University
 .
1:45-3:45
.
“Embodiment and Mediation: Riding the ‘Sound of Here-and-Now’: Locating Groove in Japanese Garage Punk,” Jose Neglia, University of California, Berkeley
.
“Air Flows: Breath, Voice, and Authenticity in Three Recordings,” Greg Weinstein, Columbia College Chicago
.
“‘Them boys kin shore tromp on the strings’: Down-Home Virtuosity in Rural Variety Radio,” David VanderHamm, University of North Carolina at Chapel Hill
.
“‘Less Work, More Flow’: Embodied Interactivity and the Ecology of Digital Media,” Mike D’Errico, University of California Los Angeles
..
1:45
“Secret Sonic Weapon on Record: Dizzy Gillespie and the Ambassadorial Politics of Jazz,” Darren Mueller, Duke University
.
“The Costs of Being Fluid: Popular Music and the Lubrication of Social Frictions,” Luis-Manuel Garcia, Max Planck Institute for Human Development
..
2:15
“Soft Power in Hard Times: Affect, Labor, and Ethics in US ‘Hip Hop Diplomacy,'” Kendra Salois, University of Maryland, College Park
.
2:45
“Listening with Your Face: The Neo-colonial Politics of Underwater Music,” Peter McMurray, Harvard University
.
James Taylor Bridge, Public Domain

So nice we put it twice, The James Taylor Bridge, Public Domain

.

Saturday, March 15

8:30
“Voices of Americas – The Sound of the Radio Programs About Folk Music in Brazil and the USA under the Pan American policy (1936-1945)”–
Rafael Velloso, UFRGS/Brazil & University of Maryland
.
“The Sound of Sludge: Groove, Materiality and Bodily Experience in Sludge Metal”–Jonathan Piper, Independent Scholar
.
8:30-10:00
Urban Soundscapes
“I Can’t Live Without My Radio”: The Sony Walkman & the Stereo Boombox in the Urban Soundscape of the 1980s”–Mathew Robert Swiatlowski, University of North Carolina at Chapel Hill
..
“Sounding Hot Springs: Music and Branding in America’s Spa City”–Robert Fry, Vanderbilt University
.
“Hip Hop Flows (through Detroit): Women’s “Legendary” Work Mapping Marginalization and Sustainability in Urban Sonic Spaces”–Rebekah Farrugia, Oakland University, Kellie Hay, Oakland University
..
10:15-11:45
“Mediating ‘Natural’ Sounds Going Deep: The Hydrophone and the History of Underwater Recording”–Craig Eley, Penn State University
.
“Early Digital Waves: Irv Teibel’s Environments and the Psychologically Ultimate Seashore”–Mack Hagood, Miami University
.
“Sigur Rós and the Soundtrack to Selling Planet Earth”–Matt DelCiampo, Florida State University
.
10:45
“Port sounds: Jazz(-scapes) in 1930s and 1940s Shanghai,” Yvonne Liao, King’s College London
.
1:45-3:45
Ecologies of Place
“Music, Dance, Theater, Water:  Environmental Justice and Ananya Dance Theatre,” Allison Adrian, St. Catherine University
.
“Stone and Ice: Resonant metaphors of Jón Leifs ecological music in Iceland’s soundscape,” Leslie C. Gay Jr., University of Tennessee
.
“Sounds of Recovery and Protest in Appalachian Ohio,” Brian Harnetty, Ohio University
.
“Mediated Ecomusicological Flows: The Nexus of Sonic Materiality and Ecotourism in the National Parks Project,” Kate Galloway, Memorial University of Newfoundland
.
2:45
“Music, Mobility, and Streaming: A Multimedia Lecture by the Killer Apps, Iowa City’s Best All-Mobile-Phone Cover Band,”Kembrew McLeod, University of Iowa and Loren Glass, University of Iowa
.
"Cheerleaders, UNC, 1989" by Flickr user North Carolina Digital Heritage Center, CC BY-NC-ND 2.0

“Cheerleaders, UNC, 1989” by Flickr user North Carolina Digital Heritage Center, CC BY-NC-ND 2.0

.

Sunday, March 16

8:30
“Tracking Edible Phonography: Record Eating, Collecting, and Musical Taste,” Shawn VanCour, NYU and Kyle Barnett, Bellarmine University
8:30-10
“Anointing Sounds: Holy Ghost Reservoirs in an Age of Mass Media (Roundtable),”  James Bielo, Miami University, Anderson Blanton, University of North Carolina at Chapel Hill, and Rory Johnson, Miami University
.
11:45
“Voices Above His Head: James Baldwin as Listener and Ethnographer,” Matthew Somoroff, Duke University
.
Chapel Hill's finest, WUNC, image by Flickr user Keith Weston

Chapel Hill’s finest, WUNC, image by Flickr user Keith Weston

<span>%d</span> bloggers like this: