Archive | Sampling and Remix Culture RSS for this section

SO! Amplifies: Shizu Saldamando’s OUROBOROS

tumblr_inline_neh6uxnXox1s7hwiq

Document3SO! Amplifies. . .a highly-curated, rolling mini-post series by which we editors hip you to cultural makers and organizations doing work we really really dig.  You’re welcome!

Shizu Saldamando’s solo multimedia exhibition OUROBOROS explores the social dynamics and physical codes integrated within contemporary group dancing and civic participation.  OUROBOROS represents the ancient symbol of revolutionary cycles, rebirth, and circle dancing.  The show, opening at South of Sunset–an exhibition and performance space in Echo Park, Los Angeles, run by Elizabeth DiGiovanni and Megan Dudley–will include a selection of large-scale, photorealistic works on paper documenting the intimate social interactions observed within LA’s dance club scene, as well as recent video work. South of Sunset is pleased to premiere her most recent video, a juxtaposition of footage of traditional Japanese dancers at an Obon festival and punk rockers in a mosh-pit at a show in East LA.  

10420431_10152498003563182_4882224398129219579_n

“Ouroboros” 2 channel video, 4:58 looped 2014  //  “La Sandra” colored pencil, glitter, spray paint on paper 25” x 32″ 2014

Opening of Ourboros, South of Sunset, 11/12/14, Image by Devon Tsuno

Opening of Ourboros, South of Sunset, 11/12/14, Image by Devon Tsuno

Even as her recent interview with NPR Latino amplifies the “quiet radical politics” of her work, the sonics of Shizu’s work are loudly resonant. Her pencil and ink drawings, glittery gilt paintings, and video pieces reverberate with the sights and sounds of the two California cities she has called home–San Francisco and Los Angeles–the three cultures that have profoundly shaped her–Mexican, Japanese, and American–and the myriad voices, favorite bands, and energy of the friends she photographs out at dance clubs and concerts while “documenting the vibe” of LA music subcultures.

The exhibit runs from November 12th to December 3rd 2014; the gallery is open on Sundays 1 – 4 pm through November 23 (and by appointment).

Featured image: Ozzie and Grace, 2014, Shizu Saldamando, colored pencil and spray paint on paper, 25 x 32 inches.

Shizu Saldamando (b. 1978, San Francisco) is a multidisciplinary artist based in Los Angeles. A graduate of UCLA (BA, 2000) and CalArts (MFA, 2005), she has been the subject of recent solo exhibitions at the Vincent Price Art Museum (Los Angeles, 2013), Moore College of Art and Design (Philadelphia, 2012), and Steve Turner Contemporary Art (Los Angeles, 2010). Saldamando’s work has also been included in influential group exhibitions including Portraits of the Encounter at the Smithsonian National Portrait Gallery (Washington DC, 2011), Audience as Subject at Yerba Buena Center for the Arts (San Francisco, 2010), and Phantom Sightings: Art After the Chicano Movement at Los Angeles County Museum of Art (Los Angeles, 2008).

tape reelREWIND! . . .If you liked this post, you may also dig:

SO! Amplifies: Mendi+Keith Obadike and Sounding Race in America

Maile Colbert, Rui Costa, and Jeff Cain’s “Radio Terramoto” 

Yvon Bonenfant’s Voice Bubbles App 

“Cremation of senses in friendly fire”: on sound and biopolitics (via KMFDM & World War Z)

drum crisper fayltrash

There’s a 20-year gap in chronology between KMFDM’s 1993 song “A Drug Against War” and Marc Forster’s 2013 film World War Z, but sonically and ideologically they’re very, very similar. They contain the same kinds of sounds–machine guns, military orders barked over radios, buzzing crowds–and they use these sounds in the same way: to build sonic intensity past its breaking point (a “sonic bombardment brighter than sunlight,” as the KFMDM lyrics say). Their sonic similarity is evidence of neoliberalism’s intensification in the 20 years between them: what was once avant-garde opposition is later mainstream norm.

The songs’ sonic similarity reveals the central role of sound in contemporary biopolitics. By listening closely to “A Drug Against War” and the soundscape of World War Z—a film in which Brad Pitt saves humanity from a zombie apocalypse by giving all survivors a terminal disease—I show sound as more than a privileged aesthetic domain; sound actually provides the epistemic background and the concrete mechanisms for organizing society. Just as vision and “the gaze” are the ideological and technological foundation of panopticism, sound is the ideological and technological foundation of contemporary biopolitics. Much more is at stake in this post than just a song and a film: it takes on how—and why—society is organized as it is. It’s also about a particular understanding of “the sonic”: sound as dynamic patterning.

Because “A Drug Against War” lays out, in fairly elementary form, this “biopolitical” sonic vocabulary, it makes sense to start there. But before I do that, I will briefly define what I understand as ‘biopolitics.’

Life

Like “neoliberalism,” “biopolitics” is a trendy concept whose precise meaning can get lost in loose usage. By “biopolitics,” I mean both an ideology of health and vitality and a political strategy whose medium is “life.” “Life,” here, isn’t individual health, wellness, or existence; it’s the ongoing vitality of the segment of society that counts as “society” tout court (e.g., in white supremacy, that segment would be whites). Biopolitics manages society like a living thing; for example, we often talk about the “health” of the economy, or use metrics such as obesity rates to compare different countries.

"Overweight or obese population OECD 2010" by ZH8000 - Own work. Licensed under Creative Commons Attribution-Share Alike 3.0 via Wikimedia Commons

“Overweight or obese population OECD 2010″ by ZH8000 – Own work. Licensed under Creative Commons Attribution-Share Alike 3.0 via Wikimedia Commons

As Foucault explains in Society Must Be Defended, biopolitics’ “basic function is to improve life, to prolong its duration, to improve its chances, to avoid accidents, and to compensate for failings” (254). But to do that, power sometimes has to kill. Pruning my raspberry bush causes more berries to sprout, for example, just as weightlifting tears all my muscle fibers so they’ll rebuild in bigger, stronger shape. Killing off the weak is a positive investment in society’s overall strength. Again, Foucault:

The fact that the other dies does not mean simply that I live in the sense that his death guarantees my safety; the death of the other, the death of the bad race, of the inferior race (or the degen­erate, or the abnormal) is something that will make life in general healthier: healthier and purer. (Society, 255).

Hitler’s “final solution” is an obvious example of this biopolitical approach to killing, but this practice also informs many contemporary US policies and practices. In the US, black people as a population have significantly higher mortality rates than any other race, for example. Following Foucault, we could say it’s in the interest of white supremacist society to maintain a high mortality rate among black populations because this makes white supremacist society “healthier.” The key point here is this: biopolitics promotes and administers life by generalizing and naturalizing what Foucault calls “the relationship of war: ‘In order to live, you must destroy your enemies’” (Society 256). Biopolitical warfare is precisely what is waged in both “A Drug Against War” and World War Z, and sound emerges as a weapon of choice.

“Kill Everything”

DrugagainstwarIn its original context, KMFDM’s “A Drug Against War” used sound to counter US Presidents Reagan and Bush 1’s War on Drugs/”New World Order” thinking. Indeed, in 1993, it was hard not to hear it as a response to 1991’s Operation Desert Storm, the US’s first military action as the ‘winner’ of the Cold War. It performs, in music, the “cremation of senses in friendly fire,” that its lyrics describe. It burns out our hearing, realizing through sound the sort of “creative destruction” or “shock doctrine” that characterizes neoliberalism more generally. In this rather Nietzschean model, the only way to make something “stronger than ever, ever before” is to first kill it. Death is the means to the most vibrant life.

The lyric–“stronger than ever, ever before”–is the first line of the chorus. At the end of every verse, there’s a short drumroll that leads into it. As S. Alexander Reed notes in Assimilate: A Critical History of Industrial Music, “clocking in at “322 bpm, the eighth-note snare fills at the end of the verses fire at about eleven rounds every second–the same rate as an AK-47” (29). This flourish foreshadows the gesture that, in the song’s bridge [after the second chorus, around 2:17 in the video above], musically “cremates” our senses in friendly fire–in this case, in the rapid fire of percussion. This rapid-fire percussion is one of the sonic elements that “Drug” shares with WWZ; in fact, the chorus uses what is likely (according to Reed) a machine gun sample. The machine gun effect mimics blast drumming. As Ronald Bogue explains in Deleuze’s Wake, blast drumming is a “tactic of accelerating meters to the point of collapse,” produced through the “cut-time alteration of downbeat kick drum and offbeat snare, the accent being heard on the offbeat but felt on the downbeat” (99). “A Drug Against War”’s AK-47 rolls actually accelerate to the point of auditory collapse, i.e. to the point at which humans generally can’t distinguish individual sonic events—the aural equivalent of seeing 24 frames per second as one continuous image. The AK-47’s rolls of ‘friendly’ fire cremate our sense of hearing.

The song’s chorus includes many other sonic elements shared by World War Z: doppler effects (such as the sounds of dropping bombs or planes buzzing the ground), rubble being moved around, military orders barked over radio. In the bridge several kinds of crowd noises are introduced: first, guitars buzz like a swarm of insects; then, a call-and-response in which singer Sascha Koneitzko echoes the chorus (which reverses the usual order in which the chorus echoes the individual leader); finally, a chaotic rabble of voices builds in intensity and leads into the sense-cremating climax.

KMFDM, 1 October 2009, Image by Flickr User Axel Taferner

KMFDM, 1 October 2009, Image by Flickr User Axel Taferner

An extended and intensified version of the “friendly fire” at the end of each verse, “A Drug Against War”’s climax builds to a peak by layering two full measures of AK-47-style drumroll on top of sounds of rabble, evoking the image of the military firing on an unruly crowd. This roll barrels towards the point of auditory collapse–if it got much faster, we’d be unable to distinguish individual rhythmic events, and hear a constant buzz (like in the beginning of the bridge), not a series of eighth notes. The roll’s forward momentum intensifies musical energy to an apex, culminating on the downbeat of the next measure in a florid lead guitar solo.

Describing the song as “sonic bombardment brighter than sunlight,” the lyrics confirm the music (and vice versa). The song overdrives sound until it sublimates into something else–if sunlight is more intense radiation than even soundwaves, here soundwaves amplify to a state more powerful than that. Cremating our senses in friendly fire, KMFDM channels soundwaves into a revolutionary drug, a drug against war. The band presents cleansing fire meant to purify us of disease: just as a fever kills pathogens in our bodies, the song burns our senses to kill a pathogenic ideology. Overdriving mainstream musical taste, offering something so brutal, so damaging to one’s ears, that only the avant-garde can survive, KMFDM inoculates the population against its most reactionary, war-mongering elements. “A Drug Against War” uses sound to perform a biopolitical operation, one that emerges as the basis of WWZ’s plot: the only way to save the human race from the zombies is to kill everything.

WWZ

WWZ Stencil Duncan CWorld War Z intensifies the horrors of contemporary biopolitics to the point that the only way to recuperate from them is to intensify them even further: in order for humanity to survive, everyone must be dead on their feet. In the sci-fi universe of World War Z, zombies aren’t eating for their survival, but for the survival of the virus they carry; they only attack and eat prey that are also (and primarily) attractive hosts for the virus. Pitt’s character, protagonist Gerry Lane, discovers that terminally ill humans aren’t legible to the zombies as human—that is, as attractive hosts. They won’t live long enough and/or are too weak to aggressively spread the virus. So, he decides the best way to protect humans from zombies and the virus they carry is to infect the remaining people with a deadly but ultimately curable illness. The World Health Organization develops a vaccine that allows healthy people to ‘pass’ as terminal cases. The only difference remaining in the post apocalyptic world of WWZ is between the quasi-dead and the walking dead. Death is the drug against WWZ.

The film doesn’t represent or express the biopolitical recuperation of death visually, but sonically: to make audiences feel what the narrative depicts, WWZ cremates their sense of hearing–often with more amplified and complex versions of the same sonic elements mobilized in “Drug.” Doppler effects, crowd noises, machine guns, military orders barked over radio bombards the film’s audience as sonic “friendly fire.” Though the film’s soundtrack doesn’t actually blow out its audience’s ears (what lawsuits!), it repeatedly simulates sonic cremation; the tinitus-y buzzing one hears after auditory trauma–what one hears in lieu of hearing—functions as a constant refrain. Narratively climactic moments are composed, cinematically, as sonic overdrive. The massive car crash as everyone tries to evacuate NYC in the beginning of the film, the moment when Pitt’s character thinks he may have been infected atop the NJ apartment building, the plane crash outside the Cardiff WHO office–each of these events culminates in tinitus-y ringing. As physical and psychological trauma overwhelms the characters, the film pretends to inflict overwhelming—cremating—auditory trauma on its audience.

World-War-Z-Review-01

WWZ Screen Capture

In the WWZ universe, sound is destructive; it unleashes the zombie horde. At 49:00, a soldier says: “remember these things are drawn to sound…there’s only one way we’re getting you on that plane, and that’s quiet.” In a scene set in Jerusalem, excessive sound turns something miraculously positive—a Muslim girl and a Jewish girl leading a mixed crowd in song, a mini Arab-Israeli peace accord—Into a massacre. The sound attracts the zombie horde, leading them to swarm and overrun Jerusalem’s walls. Similarly, at the film’s end, Pitt’s character empties a soda machine so the cascade of cans will attract zombies away from the doors he needs to enter. By this point, Pitt’s character has injected himself with a deadly disease, effectively killing himself in order to preserve himself from zombification. The cascade of cans aesthetically represents this narrative point and hearkens back to KMFDM. The cans drop out of the machine at an increasingly rapid rate, mimicking “Drug”’s intensification of percussion events to and/or past the limit of human hearing. Just as Pitt’s character has crashed his body, the cascade of cans crashes our hearing.

The climax presents a narrative and the auditory convergence on the same biopolitical idea: kill everything, because then the best will bounce back, phoenix-like, from that sensory cremation, stronger than ever. Zombies can’t rebound from death, but still-living humans sure can (via immunization). Like a sonic bombardment brighter and more radiant than sunlight, this anti-zombie camouflage tactic phase-shifts death into exceptionally lively life. Just as the muted, tinitus-y moments in the film make the subsequent scenes feel comparatively more sonically rich and dynamic, intentional and carefully managed mass extinction ultimately makes the living more vibrant.

Sound & Biopolitics

Such vibrancy–that is, what Julian Henriques dubs “the dynamics of [the] periodic motion of vibrations” in Sounding BodiesReggae Sound Systems, Performance Techniques, and Ways of Knowing (265)–is what “life” and “sound,” as they are conceived by and function in contemporary biopolitics, have in common. “Sound,” according to Henriques is “a particular kind of periodic motion, variation and change” (247). Sound waves are dynamic patterns of intensities (pressure); they move through matter and respond in turn, both to that movement itself and the secondary sound waves (harmonics) that movement produces. WWZ treats this notion of periodic motion, variation, and change as the conceptual basis for the ideally biopolitical “life.” At around 20:00, when Pitt’s character attempts to convince the Latino family sheltering him to leave their apartment with him, he says “movement is life…Moviemento es vida.” Sedentary fortresses protect no one from zombies–we see this repeatedly in the film. The only way to survive is by rapidly adjusting to new conditions. The dynamism of adaptive flows—the ability to bounce back and recuperate (like an echoing pressure wave), to dynamically recombine (like both harmonizing frequencies and like a virus), to find signal in noise–this dynamism is life. Because it adapts to new challenges, because it moves, varies, and changes, life can bounce back from total annihilation, stronger than ever before. Only life lived like sound can be properly and sufficiently resilient. In WWZ, the zombie virus is a eugenic tool that weeds out insufficiently “sonic” life, life that is too static to respond to capitalist and biopolitical mandates for calculable motion, variation, and change.

WWZ Shooting in Glasgow, Scotland, Image by Flickr User Gerry McKay

WWZ Shooting in Glasgow, Scotland, Image by Flickr User Gerry McKay

When read through “Drug,” WWZ illustrates the epistemic and ontological importance of sound to contemporary biopolitics. We think “life” works like we think sound works. Because “life” is the object and the mechanism of biopolitical government, power works on and through us sonically. If we want to analyze, critique, and fight the institutions, structures, and practices that put power to work for white supremacy, cis/het patriarchy, and all other forms of domination, then we need to start thinking and working sonically, too.

Some theorists, such as Elizabeth Grosz and Adriana Cavarero incorrectly think this move to sound and voice is itself revolutionary and counter-hegemonic. Just as the critique posed in 1993’s “Drug” has been co-opted by 2013’s WWZ, white feminist theory’s sonic counter-modernities are the medium of biopolitical white supremacist patriarchy. When we think and work sonically, we’re working with the master’s tools; to bring down the master’s house, we have to use them critically and strategically.

Featured Image by Flickr User crisper fayltrash

__

Robin James is Associate Professor of Philosophy at UNC Charlotte. She is author of two books: Resilience & Melancholy: pop music, feminism, and neoliberalism will be published by Zer0 books this fall, and The Conjectural Body: gender, race and the philosophy of music was published by Lexington Books in 2010. Her work on feminism, race, contemporary continental philosophy, pop music, and sound studies has appeared in The New Inquiry, Hypatia, differences, Contemporary Aesthetics, and the Journal of Popular Music Studies. She is also a digital sound artist and musician. She blogs at its-her-factory.com and is a regular contributor to Cyborgology.

tape reelREWIND!…If you liked this post, check out:

Queer Timbres, Queered Elegy: Diamanda Galás’s The Plague Mass and the First Wave of the AIDS Crisis–Airek Beauchamp

On Sound and Pleasure: Meditations on the Human VoiceYvon Bonenfant

The Noises of Finance– Nick Knouf

 

Sounding Out! Podcast #31: Game Audio Notes III: The Nature of Sound in Vessel

Sound and Pleasure2This post continues our summer Sound and Pleasure series, as the third and final podcast in a three part series by Leonard J. Paul. What is the connection between sound and enjoyment, and how are pleasing sounds designed? Pleasure is, after all, what brings y’all back to Sounding Out! weekly, is it not?

Part of the goal of this series of podcasts has been to reveal the interesting and invisible labor practices which are involved in sound design. In this final entry Leonard J. Paul breaks down his process in designing living sounds for the game Vessel. How does one design empathetic or aggressive sounds? If you need to catch up read Leonard’s last entry where he breaks down the vintage sounds of Retro City Rampage. Also, be sure to be sure to check out last week’s edition where Leonard breaks down his process in designing sound for Sim Cell. But first, listen to this! -AT, Multimedia Editor

-

CLICK HERE TO DOWNLOADGame Audio Notes III: The Nature of Sound in Vessel

SUBSCRIBE TO THE SERIES VIA ITUNES

ADD OUR PODCASTS TO YOUR STITCHER FAVORITES PLAYLIST

-
Game Audio Notes III: The Nature of Sound in Vessel

Strange Loop Game’s Vessel is set in an alternate world history where a servant class of liquid automatons (called fluros) has gone out of control. The player explores the world and solves puzzles in an effort to restore order. While working on Vessel, I personally recorded all of the sounds so that I could have full control over the soundscape. I recorded all of the game’s samples with a Zoom H4n portable recorder. My emphasis on real sounds was intended to focus the player’s experience of immersion in the game.

This realistic soundscape was supplemented with a variety of techniques that produced sounds that dynamically responded to the changes in the physics engine. Water and other fluids in the game were  difficult to model with both the physics engine and the audio engine (FMOD Designer). Because fluids are fundamentally connected to the game’s physics engine, they takes on a variety of different dynamic forms as players interact with the fluid in different ways. In order to address this Kieran Lord, the audio coder, and I considered factors like the amount of liquid in a collision with anything, the hardness of the surface that it was colliding with, the type of liquid in motion, whether the player is experiencing an extreme form of that sound because it is colliding with their head, and, of course, how fast the liquid is travelling.

Although there was a musical score, I designed the effects to be played without music. Each element of the game, for instance a lava fluro’s (one of the game’s rebellious automatons) footsteps, entailed required layers of sound. The footsteps were composed of water sizzling on a hot pan, a gloopy slap of oatmeal and a wet rag hitting the ground. Finding the correct emotional balance to support the game’s story was fundamental to my work as a sound designer. The game’s sound effects were constantly competing with the adaptive music (which is also contingent on player action) that plays throughout the game, so it was important to provide an informative quality to them. The sound effects inform you about the environment while the music sets the emotional underscore of the gameplay and helps guide you in the puzzles.

The lava fluro foosteps in FMOD Designer.

The lava fluro foosteps in FMOD Designer. Used with permission (c) 2014 Strange Loop Games

Defining the character of the fluros was difficult because I wanted players to have empathy for them. This was important to me because there is often no way to avoid destroying them when solving the game’s puzzles. While recording sounds in the back of an antique shop, I came across a vintage Dick Tracey gun that made a fantastic clanking sound when making a siren sound. Since the gun allowed me to control how quickly the siren rose and fell, it was a great way to produce vocalizations for the fluros. I simply recorded the gun’s siren sound, chopped the recording into smaller pieces, and then played back different segments randomly. The metal clanking gave a mechanical feel and the siren’s tone gave a vocal quality to the resulting sound that was perfect for the fluros. I could make the fluros sound excited by choosing a higher pitch range from the sample grains and inform the player when they approached their goal.

I wanted a fluid-based scream to announce a fluro’s death. I tried screaming underwater, screaming into a glass of water, and a few other things, but nothing worked. Eventually, when recording a rubber ear syringe, I found squeezing the water out quickly lent a real shriek while it spit out the last of the water. Not only did this sound really cut through the din of the gears clanking in the mix, but it also bonded a watery yell with the sense of being crushed and running out of breath.

Vessel-LavaBoss

Vessel’s Lava boss with audio debug output. Used with permission (c) 2014 Strange Loop Games

For the final boss, I tried many combinations of glurpy sounds to signify its lava form. Eventually I recorded a nail in a board being dragged across a large rusty metal sheet. Though it was quite excruciating to listen to, I pitched down the recording and combined it with a pitched down and granulated recording of myself growling into a cup of water. This sound perfectly captured the emotion I wanted to feel when encountering a final boss.  Although it can take a long time to arrive at the “obvious” sound, simplicity is often the key.

Anticipation is fundamental to a player’s sense of immersion. It carves a larger space for tension to build, for instance a small crescendo of a creaking sound can develop a tension that builds to a sudden and large impact. A whoosh before a punch lands adds extra weight to the force of the punch. These cues are often naturally present in real-world sounds, such as a rush of air sweeping in before a door slams. A small pause might be included just for added suspense and helps to intensify the effect of the door slamming. Dreading the impact is half of the emotion of a large hit .

Vessel-ClockRecording

Recording inside of a clock tower with my H4n recorder for Vessel. Used with permission by the author.

Recording all of the sounds for Vessel was a large undertaking but since I viewed each recording as a performance, I was able to make the feeling of the world very cohesive. Each sound was designed to immerse the player in the soundscape, but also to allow players enough time to solve puzzles without becoming annoyed with the audio. All sounds have a life of their own and a resonance of memory and time that stays with the them during each playthrough of a game. In Retro City Rampage I left a sonic space for the player to wax nostalgic. In Sim Cell, I worked to breathe life into a set of sterile and synthesized sounds. Each recorded sound in Vessel is alive in comparison, telling stories of time, place and recording with them, that are all their own.

The common theme of my audio work on Retro City Rampage, Sim Cell and Vessel, is that I enjoy putting constraints on myself to inspire my creativity. I focus on what works and removing non-essential elements. Exploring the limits of constraints often provokes interesting and unpredictable results. I like “sculpting” sounds and will often proceed from a rough sketch, polishing and reducing elements until I like what I hear. Typically I remove layers that don’t add an emotive aspect to the sound design. In games there are often many sounds that can play at once, so clarity and focus are necessary when preventing sounds from getting lost in a sonic goo.

CherryBlossoms

Cherry blossoms for new beginnings. Used with permission by the author.

In this post I have shown how play and experimentation are fundamental to my creative process. For an aspiring sound artist, spending time with Pure Data, FMOD Studio or Wwise and a personal recorder is a great way to improve their skill with game audio. This series of articles has aimed to reveal the tacit decisions behind the production of game audio that get obscured by the fun of the creative process. Plus, I hope they offer a bit of inspiration to those creating their own sounds in the future.

Additional Resources:

-

Leonard J. Paul attained his Honours degree in Computer Science at Simon Fraser University in BC, Canada with an Extended Minor in Music concentrating in Electroacoustics. He began his work in video games on the Sega Genesis and Super Nintendo Entertainment System and has a twenty year history in composing, sound design and coding for games. He has worked on over twenty major game titles totalling over 6.4 million units sold since 1994, including award-winning AAA titles such as EA’s NBA Jam 2010NHL11Need for Speed: Hot Pursuit 2NBA Live ’95 as well as the indie award-winning title Retro City Rampage.

He is the co-founder of the School of Video Game Audio and has taught game audio students from over thirty different countries online since 2012. His new media works has been exhibited in cities including Surrey, Banff, Victoria, São Paulo, Zürich and San Jose. As a documentary film composer, he had the good fortune of scoring the original music for multi-awarding winning documentary The Corporation which remains the highest-grossing Canadian documentary in history to date. He has performed live electronic music in cities such as Osaka, Berlin, San Francisco, Brooklyn and Amsterdam under the name Freaky DNA.

He is an internationally renowned speaker on the topic of video game audio and has been invited to speak in Vancouver, Lyon, Berlin, Bogotá, London, Banff, San Francisco, San Jose, Porto, Angoulême and other locations around the world.

His writings and presentations are available at http://VideoGameAudio.com

-

Featured image: Courtesy of Vblank Entertainment (c)2014 – Artwork by Maxime Trépanier.

tape reelREWIND! . . .If you liked this post, you may also dig:

Sounding Out! Podcast #30: Game Audio Notes I: Growing Sounds for Sim Cell- Leonard J. Paul

Sounding Out! Podcast #31: Hand Made Music in Retro City Rampage- Leonard J. Paul

Papa Sangre and the Construction of Immersion in Audio Games- Enongo Lumumba-Kasongo 

Sounding Out! Podcast #29: Game Audio Notes I: Growing Sounds for Sim Cell

cell Visual Slice_Base_02-1360x768

Sound and Pleasure2A pair of firsts! This is both the lead post in our summer Sound and Pleasure series, and the first podcast in a three part series by Leonard J. Paul. What is the connection between sound and enjoyment, and how are pleasing sounds designed? Pleasure is, after all, what brings y’all back to Sounding Out! weekly, is it not?

In today’s installment Leonard peels back the curtain of game audio design and reveals his creative process. For anyone curious as to what creative decisions lead to the bloops, bleeps, and ambient soundscapes of video games, this is essential listening. Stay tuned for next Monday’s installment on the process of designing sound for Retro City Rampage, and next month’s episode which focuses on the game Vessel. Today, Leonard begins by picking apart his design process at a cellular level. Literally! -AT, Multimedia Editor

-

CLICK HERE TO DOWNLOADGame Audio Notes I: Growing Sounds for Sim Cell

SUBSCRIBE TO THE SERIES VIA ITUNES

ADD OUR PODCASTS TO YOUR STITCHER FAVORITES PLAYLIST

-

Game Audio Notes I: Growing Sounds for Sim Cell

Sim Cell is an educational game released in Spring 2014 by Strange Loop Games. Published by Amplify, a branch of News Corp, it teaches students how the human cell works. Players take control of a small vessel that is shrunk down to the size of a cell and solve the tasks set by the game while also learning about the human cell. This essay unpacks the design decisions behind the simulation of a variety of natural phenomena (motion, impact, and voice) in Sim Cell.

For the design of this game I decided to focus on the elements of life itself and attempted to “grow” the music and sound design from synthetic sounds. I used the visual scripting language Pure Data (PD) to program both the sounds and the music. The music is generated from a set of rules and patterns that cause each playback of a song to be slightly different each time. Each of the sound effects are crafted from small programs that are based on the design of analogue modular synthesizers. Basic synthetic audio elements such as filtered noise, sawtooth waves and sine waves were all used in the game’s sound design.

An image of the game's space-like world.

A screenshot of Sim Cell’s microscopic space-like world. Used with permission (c) 2014 Amplify.

The visuals of the game give a feeling of being in an “inner space” that mirrors outer space. I took my inspiration for Sim Cell‘s sound effects from Louis and Bebe Baron’s score to Forbidden Planet. I used simple patches in PD to assemble all of the sounds for the game from synthesis. There aren’t any recorded samples in the sound design – all of the sound effects are generated from mathematics.

The digital effects used in the sound design were built to emulate the effects available in vintage studios such as plate reverb and analog delay. I found a simulation of a plate reverb that from a modified open source patch from the RjDj project, and constructed an analogue delay by using a low-pass filter on a delayed signal.

https://www.youtube.com/watch?v=3m86ftny1uY&feature=youtu.be

In keeping with the vintage theme, I used elements of sound design from the early days of video games as well. Early arcade games such as Space Invaders used custom audio synthesis microchips for each of their sounds. In order to emulate this, I gave every sound its own patch when doing the synthesis for Sim Cell. I learned to appreciate this ethic of design when playing Combat on the Atari 2600 while growing up. The Atari 2600 could only output two sounds at once thus had a very limited palette of tones. All of the source code for the sounds and music for Sim Cell are less than 1 megabyte, which shows how powerful the efficient coding of mathematics can be.

Vector image on the left, raster graphics on the right.

Vector image on the left, raster graphics on the right. Photo used with permission by the author.

Another cool thing about generating the sounds from mathematics is that users can “zoom in” on sounds in the same way that one can zoom in to a vector drawing. In vector drawings the lines are smooth when you zoom in, as opposed to rasterized pictures (such as a JPEG) which reveal a blurry set of pixels upon zooming. When code can change sounds in real-time, it makes them come alive and lends a sense of flexibility to the composition.

For a human feeling I often filter the sounds using formant frequencies which simulate the resonant qualities of vowel sounds, thus offering a vocal quality to the sample. For alarm sounds in Sim Cell I used minor second intervals. This lent a sense of dissonance which informed players that they needed to adjust their gameplay in order to navigate treacherous areas. Motion was captured through a whoosh sound that used filtered and modulated noise. Some sounds were pitched up to make things seem like they were travelling towards the player and likewise they exploited a sense of doppler shift when pitched down, giving the feel that a sound was traveling away from the player. Together, these techniques produced a sense of immersion for the player while simultaneously building a realistic soundscape.

Our decision to use libPD for this synthesis turned problematic and its processing requirements were too high. In order to remedy this, we decided to convert our audio into samples. Consider a photograph of a sculpture. Our sounds, like sculpture in the photograph, could now only be viewed from one direction. This meant that the music now only had a single version and that sound effects would repeat as well. A small fix was exporting the file with a set of five different intensities from 0 to 1.0. Like taking a photograph from several angles, this meant that the game could play a sample at intensity levels of 20%, 40%, 60%, 80% and 100%. Although a truly random sense of variation was lost, this method still conveyed the intensity of impacts (and other similar events) generated by the physics engine of the game.

An image of the inexpensive open-source computer: The Rasberry PI

An image of the inexpensive open-source computer: The Rasberry PI. Photo used with permission by the author.

PD is a great way to learn and play with digital audio since you can change the patch while it is running, just like you might do with a real analogue synthesizer. There’s plenty of other neat stuff that PD can do, like being able to be run on the Raspberry Pi, so you could code your own effects pedals and make your own synths using PD for around $50 or so. For video games, you can use libPD to integrate PD patches into your Android or iOS apps as well. I hope this essay has offered some insight as to my process when using PD. I’ve included some links below for those interested in learning more.

Additional resources:

-

Leonard J. Paul attained his Honours degree in Computer Science at Simon Fraser University in BC, Canada with an Extended Minor in Music concentrating in Electroacoustics. He began his work in video games on the Sega Genesis and Super Nintendo Entertainment System and has a twenty year history in composing, sound design and coding for games. He has worked on over twenty major game titles totalling over 6.4 million units sold since 1994, including award-winning AAA titles such as EA’s NBA Jam 2010NHL11Need for Speed: Hot Pursuit 2NBA Live ’95 as well as the indie award-winning title Retro City Rampage.

He is the co-founder of the School of Video Game Audio and has taught game audio students from over thirty different countries online since 2012. His new media works has been exhibited in cities including Surrey, Banff, Victoria, São Paulo, Zürich and San Jose. As a documentary film composer, he had the good fortune of scoring the original music for multi-awarding winning documentary The Corporation which remains the highest-grossing Canadian documentary in history to date. He has performed live electronic music in cities such as Osaka, Berlin, San Francisco, Brooklyn and Amsterdam under the name Freaky DNA.

He is an internationally renowned speaker on the topic of video game audio and has been invited to speak in Vancouver, Lyon, Berlin, Bogotá, London, Banff, San Francisco, San Jose, Porto, Angoulême and other locations around the world.

His writings and presentations are available at http://VideoGameAudio.com

-

Featured image: Concept art for Sim Cell. Used with permission (c) 2014 Amplify.

tape reelREWIND! . . .If you liked this post, you may also dig:

Sounding Out! Podcast #10: Interview with Theremin Master Eric Ross- Aaron Trammell

Papa Sangre and the Construction of Immersion in Audio Games- Enongo Lumumba-Kasongo

Playing with Bits, Pieces, and Lightning Bolts: An Interview with Sound Artist Andrea Parkins

%d bloggers like this: