Archive | methodology RSS for this section

SO! Amplifies: Maile Colbert, Rui Costa, and Jeff Cain’s “Radio Terramoto”

BEDFM-1974.27.1570

Document3SO! Amplifies. . .a highly-curated, rolling mini-post series by which we editors hip you to cultural makers and organizations doing work we really really dig.  You’re welcome!

This November 1st will mark the 259th anniversary of the Great Lisbon Earthquake on All Saints Day, 1755, which destroyed a quarter of the city and beget consequential tsunamis and fires. “Radio Terramoto” is a soundwalk research and art project designed to bring this seemingly distant devastation into contemporary consciousness. Based on the idea of listening to sound from a past historical event, “Radio Terramoto” is a traveling audience immersive event. It’s inaugural procession, made up of the creators and audience members, followed a path from the Convento do Carmo down to the River Targus in Lisbon, Portugal. It was also performed this summer in the town of Viseu, Portugal, as part of the Invisible Places, Sounding Cities Symposium in July 2014.

.

“Radio Terramoto” is a radio transmission from All Saints Day, 1755. We are not sure how or why the forty minutes were recorded, but having been discovered aaccidentallyit has proven to be an important record of the experience of the people caught in the earthquake. We follow our mysterious ghost recorder from the Convento, where people were gathered for mass. The first wave hits and the convent crumbles. As people run to the river, we follow their path as the buildings around us burst into flames and collapse. Upon reaching the river in a panic, we are only to be greeted by the water pulling out, revealing flopping fish and shipwrecks, pulling towards the ocean to fuel the giant wave that would finally overcome our poor recorder. From here, the transmission stops. (To read this summary in Portuguese click here).

Maile and Rui lisbon end

The end of the inaugural “Radio Terramoto” performance in Lisbon, 11/1/13

The project and research for “Radio Terramoto” asks the question, what can listening to the past reveal about the now, both in artistic practice and scientific research? Its site-based (yet mobile) sound design weaves between the present and the past and is based on research on the earthquake, using documents of first hand experiences and the first seismic and “earthquake”-proof architecture that came after what may be the largest earthquake recorded in history.

processionFor the original “Radio Terramoto” soundwalk in Lisbon, first performed November 1, 2013, we walked with the audience bearing a transmitter; the audience carried radios and cell phones tuned into the specific frequency of the transmission. The soundwalk also included hand-held sculptural octahedra created using a geometric framing system designed by Jake Dotson, assembled as a singular form approximating a Pombaline cage, the first modern earthquake resistant architecture. The radio transmitter, and other key electrical devices were suspended in these 1 foot 3 inch octahetra made of brightly colored sticks of wood held together with friction and tension. The large cage broke apart into the individual octahedra to aid in the transportation of equipment and in providing a visual wayfinding aide for the participants.

mrcarrying

“Radio Terramoto” procession, Maile Costa and Rui Colbert in the foreground, Lisbon, Portugal, November 1, 2013

Like Maile’s “Passageira em Casa,” a traveling intermedia work that explores the concept of “home,” “Radio Terramoto” changes to be site and context specific with each presentation.  When we led a performance in Viseu, Portugal, for example, we began at the Sé de Viseu, moved through the old city center, and ended at a small body of water off the Avenida Emídio Navarro.

.

.

Planned future performances of “Radio Terramoto” include a version in Los Angeles that will unite the original team in collaboration with Jesse Gilbert. Gilbert created the program SpectralGL, a cell phone app  that enables sound to visually affect the landscape from the video camera as the audience member walks. As Los Angeles has its own fraught relationship with earthquakes, we expect this performance to be particularly resonant and thought provoking.

Images courtesy of the artists and Jennifer Stoever (Viseu shots)

Rui Costa is a sound artist from Lisbon, Portugal. He is a founding member and artistic director of Binaural/Nodar, an arts organization founded in 2004 and dedicated to the promotion of context-specific and participatory art projects in rural communities of the Gralheira mountain range, northern Portugal. Rui has been performing and exhibiting his work since 1998 in festivals, galleries and museums across Portugal, Spain, Italy and the United States and has been collaborating regularly with the Italian vocal performer Manuela Barile and the American intermedia artist Maile Colbert. Rui Costa is also a regular speaker in conferences and gives workshops dedicated to sound art. For more from Binaural/Nodar, please check out the organization’s soundcloudvimeo, and flickr.

Maile Colbert is a multi-media artist with a concentration on sound and video who relocated from Los Angeles, US to Lisbon, Portugal. She is a regular writer for Sounding Out!

Jeff Cain is an artist, designer, curator and director of the Shed Research Institute a multidisciplinary art, research, curatorial, and design studio in Lincoln Heights, Los Angeles.

tape reelREWIND!…If you liked this post, you may also dig:

Wayback Sound Machine: Sound Through Time, Space, and Place-Maile Colbert

Six Years in Nodar: Sound Art in a Rural Context–Rui Costa

SO! Amplifies: Eric Leonardson and World Listening Day 18 July 2014

Sounding Out! Podcast #31: Game Audio Notes III: The Nature of Sound in Vessel

Sound and Pleasure2This post continues our summer Sound and Pleasure series, as the third and final podcast in a three part series by Leonard J. Paul. What is the connection between sound and enjoyment, and how are pleasing sounds designed? Pleasure is, after all, what brings y’all back to Sounding Out! weekly, is it not?

Part of the goal of this series of podcasts has been to reveal the interesting and invisible labor practices which are involved in sound design. In this final entry Leonard J. Paul breaks down his process in designing living sounds for the game Vessel. How does one design empathetic or aggressive sounds? If you need to catch up read Leonard’s last entry where he breaks down the vintage sounds of Retro City Rampage. Also, be sure to be sure to check out last week’s edition where Leonard breaks down his process in designing sound for Sim Cell. But first, listen to this! -AT, Multimedia Editor

-

CLICK HERE TO DOWNLOADGame Audio Notes III: The Nature of Sound in Vessel

SUBSCRIBE TO THE SERIES VIA ITUNES

ADD OUR PODCASTS TO YOUR STITCHER FAVORITES PLAYLIST

-
Game Audio Notes III: The Nature of Sound in Vessel

Strange Loop Game’s Vessel is set in an alternate world history where a servant class of liquid automatons (called fluros) has gone out of control. The player explores the world and solves puzzles in an effort to restore order. While working on Vessel, I personally recorded all of the sounds so that I could have full control over the soundscape. I recorded all of the game’s samples with a Zoom H4n portable recorder. My emphasis on real sounds was intended to focus the player’s experience of immersion in the game.

This realistic soundscape was supplemented with a variety of techniques that produced sounds that dynamically responded to the changes in the physics engine. Water and other fluids in the game were  difficult to model with both the physics engine and the audio engine (FMOD Designer). Because fluids are fundamentally connected to the game’s physics engine, they takes on a variety of different dynamic forms as players interact with the fluid in different ways. In order to address this Kieran Lord, the audio coder, and I considered factors like the amount of liquid in a collision with anything, the hardness of the surface that it was colliding with, the type of liquid in motion, whether the player is experiencing an extreme form of that sound because it is colliding with their head, and, of course, how fast the liquid is travelling.

Although there was a musical score, I designed the effects to be played without music. Each element of the game, for instance a lava fluro’s (one of the game’s rebellious automatons) footsteps, entailed required layers of sound. The footsteps were composed of water sizzling on a hot pan, a gloopy slap of oatmeal and a wet rag hitting the ground. Finding the correct emotional balance to support the game’s story was fundamental to my work as a sound designer. The game’s sound effects were constantly competing with the adaptive music (which is also contingent on player action) that plays throughout the game, so it was important to provide an informative quality to them. The sound effects inform you about the environment while the music sets the emotional underscore of the gameplay and helps guide you in the puzzles.

The lava fluro foosteps in FMOD Designer.

The lava fluro foosteps in FMOD Designer. Used with permission (c) 2014 Strange Loop Games

Defining the character of the fluros was difficult because I wanted players to have empathy for them. This was important to me because there is often no way to avoid destroying them when solving the game’s puzzles. While recording sounds in the back of an antique shop, I came across a vintage Dick Tracey gun that made a fantastic clanking sound when making a siren sound. Since the gun allowed me to control how quickly the siren rose and fell, it was a great way to produce vocalizations for the fluros. I simply recorded the gun’s siren sound, chopped the recording into smaller pieces, and then played back different segments randomly. The metal clanking gave a mechanical feel and the siren’s tone gave a vocal quality to the resulting sound that was perfect for the fluros. I could make the fluros sound excited by choosing a higher pitch range from the sample grains and inform the player when they approached their goal.

I wanted a fluid-based scream to announce a fluro’s death. I tried screaming underwater, screaming into a glass of water, and a few other things, but nothing worked. Eventually, when recording a rubber ear syringe, I found squeezing the water out quickly lent a real shriek while it spit out the last of the water. Not only did this sound really cut through the din of the gears clanking in the mix, but it also bonded a watery yell with the sense of being crushed and running out of breath.

Vessel-LavaBoss

Vessel’s Lava boss with audio debug output. Used with permission (c) 2014 Strange Loop Games

For the final boss, I tried many combinations of glurpy sounds to signify its lava form. Eventually I recorded a nail in a board being dragged across a large rusty metal sheet. Though it was quite excruciating to listen to, I pitched down the recording and combined it with a pitched down and granulated recording of myself growling into a cup of water. This sound perfectly captured the emotion I wanted to feel when encountering a final boss.  Although it can take a long time to arrive at the “obvious” sound, simplicity is often the key.

Anticipation is fundamental to a player’s sense of immersion. It carves a larger space for tension to build, for instance a small crescendo of a creaking sound can develop a tension that builds to a sudden and large impact. A whoosh before a punch lands adds extra weight to the force of the punch. These cues are often naturally present in real-world sounds, such as a rush of air sweeping in before a door slams. A small pause might be included just for added suspense and helps to intensify the effect of the door slamming. Dreading the impact is half of the emotion of a large hit .

Vessel-ClockRecording

Recording inside of a clock tower with my H4n recorder for Vessel. Used with permission by the author.

Recording all of the sounds for Vessel was a large undertaking but since I viewed each recording as a performance, I was able to make the feeling of the world very cohesive. Each sound was designed to immerse the player in the soundscape, but also to allow players enough time to solve puzzles without becoming annoyed with the audio. All sounds have a life of their own and a resonance of memory and time that stays with the them during each playthrough of a game. In Retro City Rampage I left a sonic space for the player to wax nostalgic. In Sim Cell, I worked to breathe life into a set of sterile and synthesized sounds. Each recorded sound in Vessel is alive in comparison, telling stories of time, place and recording with them, that are all their own.

The common theme of my audio work on Retro City Rampage, Sim Cell and Vessel, is that I enjoy putting constraints on myself to inspire my creativity. I focus on what works and removing non-essential elements. Exploring the limits of constraints often provokes interesting and unpredictable results. I like “sculpting” sounds and will often proceed from a rough sketch, polishing and reducing elements until I like what I hear. Typically I remove layers that don’t add an emotive aspect to the sound design. In games there are often many sounds that can play at once, so clarity and focus are necessary when preventing sounds from getting lost in a sonic goo.

CherryBlossoms

Cherry blossoms for new beginnings. Used with permission by the author.

In this post I have shown how play and experimentation are fundamental to my creative process. For an aspiring sound artist, spending time with Pure Data, FMOD Studio or Wwise and a personal recorder is a great way to improve their skill with game audio. This series of articles has aimed to reveal the tacit decisions behind the production of game audio that get obscured by the fun of the creative process. Plus, I hope they offer a bit of inspiration to those creating their own sounds in the future.

Additional Resources:

-

Leonard J. Paul attained his Honours degree in Computer Science at Simon Fraser University in BC, Canada with an Extended Minor in Music concentrating in Electroacoustics. He began his work in video games on the Sega Genesis and Super Nintendo Entertainment System and has a twenty year history in composing, sound design and coding for games. He has worked on over twenty major game titles totalling over 6.4 million units sold since 1994, including award-winning AAA titles such as EA’s NBA Jam 2010NHL11Need for Speed: Hot Pursuit 2NBA Live ’95 as well as the indie award-winning title Retro City Rampage.

He is the co-founder of the School of Video Game Audio and has taught game audio students from over thirty different countries online since 2012. His new media works has been exhibited in cities including Surrey, Banff, Victoria, São Paulo, Zürich and San Jose. As a documentary film composer, he had the good fortune of scoring the original music for multi-awarding winning documentary The Corporation which remains the highest-grossing Canadian documentary in history to date. He has performed live electronic music in cities such as Osaka, Berlin, San Francisco, Brooklyn and Amsterdam under the name Freaky DNA.

He is an internationally renowned speaker on the topic of video game audio and has been invited to speak in Vancouver, Lyon, Berlin, Bogotá, London, Banff, San Francisco, San Jose, Porto, Angoulême and other locations around the world.

His writings and presentations are available at http://VideoGameAudio.com

-

Featured image: Courtesy of Vblank Entertainment (c)2014 – Artwork by Maxime Trépanier.

tape reelREWIND! . . .If you liked this post, you may also dig:

Sounding Out! Podcast #30: Game Audio Notes I: Growing Sounds for Sim Cell- Leonard J. Paul

Sounding Out! Podcast #31: Hand Made Music in Retro City Rampage- Leonard J. Paul

Papa Sangre and the Construction of Immersion in Audio Games- Enongo Lumumba-Kasongo 

Sounding Out! Podcast #30: Game Audio Notes II: Hand Made Music in Retro City Rampage

01-RCR-VinylAlbumCover

Sound and Pleasure2This post continues our summer Sound and Pleasure series, as the second **bonus Monday** podcast in a three part series by Leonard J. Paul. What is the connection between sound and enjoyment, and how are pleasing sounds designed? Pleasure is, after all, what brings y’all back to Sounding Out! weekly, is it not?

Who doesn’t like retrogames? As a kid I kept to a straight diet of NES pixels and sounds. This installment reveals the technical and creative proficiencies involved with the composition of retro sound, and it. is. amazing! Our final installment on game audio design will run Thursday, 6/26/2014, and feature some notes on the process of designing sound for the game Vessel. Also, be sure to be sure to check out last week’s edition where Leonard breaks down his process in designing sound for Sim CellBut first, Retro City Rampage! -AT, Multimedia Editor

 

P.S. The first 25 folks to follow @soundingoutblog, @VideoGameAudio or @RetroCR on Twitter following the publication of this podcast will win a free download code for Retro City Rampage sent to them via direct message courtesy of Leonard Paul!

-

CLICK HERE TO DOWNLOADGame Audio Notes II: Hand Made Music in Retro City Rampage

SUBSCRIBE TO THE SERIES VIA ITUNES

ADD OUR PODCASTS TO YOUR STITCHER FAVORITES PLAYLIST

-

Game Audio Notes II: Hand Made Music in Retro City Rampage

Retro City Rampage (RCR) is a retro vibe two-dimensional open world game with plenty of parodies from the 80s and 90s. It’s basically what Grand Theft Auto would be if it was on the Nintendo Entertainment System (NES). Because the world of computer game design has recently embraced retro-aesthetics, the game was released for almost every platform. Its Nintendo 3DS version, was even a critical success, scoring a high 83% on Metacritic. My goal with the sound design of RCR was to produce a sound that was both an homage to the original sounds of the NES, but with a heart-felt intensity such that it preserved the feeling of my own nostalgia as well. The sound design of RCR follows in the tradition of independent games that are working hard to recover an aesthetic from the halcyon days of gaming.

I wanted the sound design of RCR to be as nostalgic as possible. To do this, I started by researching the work of others with this sepia-drenched 8-bit aesthetic in their own work. The open source scores that Jake Kaufman (aka “virt”) used for his albums FX1 and FX2, were particularly valuable here. Later on I came across a chiptune tutorial that Matt Creamer (aka “Norrin Radd”) had made and was able to bring him onboard to complete our team of composers for the game. I was able to borrow the same setup that he used for his music and adapt it to my process for creating sound effects. I used the open source music software OpenMPT for creating the sound effects as well as use the C++ sound code for playing back the music in the game as well. Getting the code from OpenMPT meant that new code didn’t need to be created and it ensured that the songs and sound effects would play back perfectly without any issues in the game.

02-RCR-OpenMPT

A Screenshot of OpenMPT software. Image courtesy of the author.

OpenMPT is a music tracker (or mod tracker) program for Windows. Mod trackers began on the Commodore Amiga in 1987 with the release of the Ultimate Soundtracker. The Amiga supported 4 channels of 8-bit sampled sound and usually had very low sampling rates to conserve memory as the original Amiga 1000 usually had only 256 KB of RAM. In other words, the samples were short and rough, but creative engineers found ways to work within these limitations. For RCR we used the Impulse Tracker format that was first released in 1995 for DOS.

Early sound designers needed to code the sounds in an arcane language called assembly code that was quite difficult to understand unless you were a computer programmer. Trackers were a first step toward making audio for games easier to make. They allowed sound designers to work with musical notes and effects abstractly, using a notation language far easier than hard-core assembly code. Later consoles, such as the Super Nintendo Entertainment System (SNES), allowed sound designers to simply input sampled sounds to be played back during the game. We wanted the sound design of RCR to be from the “classic era” of video games (before sampled sounds became the norm) and to feed off of the nostalgia surrounding this era for the player.

I typed in thousands of notes and effect commands in by hand, when creating the score for RCR. This level of detail and control has a direct aesthetic effect on the audio. Game sound programmers working in the 1980s lacked the sophisticated tools of automation that are standard in the industry today. This attention to detail and nuance was essential for the nostalgic sound associated with classic video games that I wanted to produce in my work. Just as using paper can be contrasted with modern music composition software, the mode in which one creates has a direct effect on the results of the composition.

03-rcr_cassette_action

Courtesy of Vblank Entertainment (c)2014 – Artwork by Maxime Trépanier.

While creating the sound effects for RCR, I learned how to do tracking and decided I wanted to add some music to RCR as well.  The catch was that the synthesis capabilities of the NES were extremely limited including only two pulse waves (often used for lead instruments), a triangle wave (usually used for a bass instrument), a noise channel (frequently used for drums), and a crude sampled sound channel (commonly used for muffled sound effects).

Even though we decided early on that we wanted the game to have a nostalgic feel, we made a set of careful decisions in order to avoid being locked into the tricky technical details that sound artists who worked on games for the original NES had originally faced. One key difference was that we didn’t limit the overall polyphony of sounds playing at the same time to the original NES specification. We limited each individual sound effect and song within the NES specification (for example, a single sound effect couldn’t use three pulse waves), but we decided not to drop a channel out when a sound effect would have preempted the score from one of the music channel. Typically in original NES games the music moved aside in order to accommodate the sound effects and so notes that used the pulse wave track were frequently dropped. Because this sort of interruption is unpleasant, we made choices that allowed us to work around it – inspiring our creative choices rather than limiting them. Although we took a few other technical liberties, nearly all of the sounds and songs of RCR could play on an actual NES with minimal modification.

04-RCR_GhettoBlaster

Courtesy of Vblank Entertainment (c)2014 – Artwork by Maxime Trépanier.

Using an open source mod tracker format allowed us a lot of flexibility when creating the audio for RCR. Although using a mod tracker to type in sound effects by hand was a laborious process it added an authenticity to the result that would have been difficult to achieve otherwise. Working with these strict limitations forced me to make different choices in my creative process that helped me invest a sense of ownership in the results. The hand crafted NES synth sounds I described above are ultimately just symbols pointing towards their real-world counterparts, and tellingly they rely on the imagination of the listener to bridge the gap between the real and the symbolic. Nostalgia allows us to fill these gaps and allow listeners the space to hear their own memories within the game.

Additional resources:

-

Leonard J. Paul attained his Honours degree in Computer Science at Simon Fraser University in BC, Canada with an Extended Minor in Music concentrating in Electroacoustics. He began his work in video games on the Sega Genesis and Super Nintendo Entertainment System and has a twenty year history in composing, sound design and coding for games. He has worked on over twenty major game titles totalling over 6.4 million units sold since 1994, including award-winning AAA titles such as EA’s NBA Jam 2010NHL11Need for Speed: Hot Pursuit 2NBA Live ’95 as well as the indie award-winning title Retro City Rampage.

He is the co-founder of the School of Video Game Audio and has taught game audio students from over thirty different countries online since 2012. His new media works has been exhibited in cities including Surrey, Banff, Victoria, São Paulo, Zürich and San Jose. As a documentary film composer, he had the good fortune of scoring the original music for multi-awarding winning documentary The Corporation which remains the highest-grossing Canadian documentary in history to date. He has performed live electronic music in cities such as Osaka, Berlin, San Francisco, Brooklyn and Amsterdam under the name Freaky DNA.

He is an internationally renowned speaker on the topic of video game audio and has been invited to speak in Vancouver, Lyon, Berlin, Bogotá, London, Banff, San Francisco, San Jose, Porto, Angoulême and other locations around the world.

His writings and presentations are available at http://VideoGameAudio.com

-

Featured image: Courtesy of Vblank Entertainment (c)2014 – Artwork by Maxime Trépanier.

tape reelREWIND! . . .If you liked this post, you may also dig:

Sounding Out! Podcast #30: Game Audio Notes I: Growing Sounds for Sim Cell- Leonard J. Paul

A Series of Mistakes: Nullsleep and the Art of 8-bit Composition- Aaron Trammell

Digital Analogies: Techniques of Sonic Play- Roger Moseley

 

Sounding Out! Podcast #29: Game Audio Notes I: Growing Sounds for Sim Cell

cell Visual Slice_Base_02-1360x768

Sound and Pleasure2A pair of firsts! This is both the lead post in our summer Sound and Pleasure series, and the first podcast in a three part series by Leonard J. Paul. What is the connection between sound and enjoyment, and how are pleasing sounds designed? Pleasure is, after all, what brings y’all back to Sounding Out! weekly, is it not?

In today’s installment Leonard peels back the curtain of game audio design and reveals his creative process. For anyone curious as to what creative decisions lead to the bloops, bleeps, and ambient soundscapes of video games, this is essential listening. Stay tuned for next Monday’s installment on the process of designing sound for Retro City Rampage, and next month’s episode which focuses on the game Vessel. Today, Leonard begins by picking apart his design process at a cellular level. Literally! -AT, Multimedia Editor

-

CLICK HERE TO DOWNLOADGame Audio Notes I: Growing Sounds for Sim Cell

SUBSCRIBE TO THE SERIES VIA ITUNES

ADD OUR PODCASTS TO YOUR STITCHER FAVORITES PLAYLIST

-

Game Audio Notes I: Growing Sounds for Sim Cell

Sim Cell is an educational game released in Spring 2014 by Strange Loop Games. Published by Amplify, a branch of News Corp, it teaches students how the human cell works. Players take control of a small vessel that is shrunk down to the size of a cell and solve the tasks set by the game while also learning about the human cell. This essay unpacks the design decisions behind the simulation of a variety of natural phenomena (motion, impact, and voice) in Sim Cell.

For the design of this game I decided to focus on the elements of life itself and attempted to “grow” the music and sound design from synthetic sounds. I used the visual scripting language Pure Data (PD) to program both the sounds and the music. The music is generated from a set of rules and patterns that cause each playback of a song to be slightly different each time. Each of the sound effects are crafted from small programs that are based on the design of analogue modular synthesizers. Basic synthetic audio elements such as filtered noise, sawtooth waves and sine waves were all used in the game’s sound design.

An image of the game's space-like world.

A screenshot of Sim Cell’s microscopic space-like world. Used with permission (c) 2014 Amplify.

The visuals of the game give a feeling of being in an “inner space” that mirrors outer space. I took my inspiration for Sim Cell‘s sound effects from Louis and Bebe Baron’s score to Forbidden Planet. I used simple patches in PD to assemble all of the sounds for the game from synthesis. There aren’t any recorded samples in the sound design – all of the sound effects are generated from mathematics.

The digital effects used in the sound design were built to emulate the effects available in vintage studios such as plate reverb and analog delay. I found a simulation of a plate reverb that from a modified open source patch from the RjDj project, and constructed an analogue delay by using a low-pass filter on a delayed signal.

https://www.youtube.com/watch?v=3m86ftny1uY&feature=youtu.be

In keeping with the vintage theme, I used elements of sound design from the early days of video games as well. Early arcade games such as Space Invaders used custom audio synthesis microchips for each of their sounds. In order to emulate this, I gave every sound its own patch when doing the synthesis for Sim Cell. I learned to appreciate this ethic of design when playing Combat on the Atari 2600 while growing up. The Atari 2600 could only output two sounds at once thus had a very limited palette of tones. All of the source code for the sounds and music for Sim Cell are less than 1 megabyte, which shows how powerful the efficient coding of mathematics can be.

Vector image on the left, raster graphics on the right.

Vector image on the left, raster graphics on the right. Photo used with permission by the author.

Another cool thing about generating the sounds from mathematics is that users can “zoom in” on sounds in the same way that one can zoom in to a vector drawing. In vector drawings the lines are smooth when you zoom in, as opposed to rasterized pictures (such as a JPEG) which reveal a blurry set of pixels upon zooming. When code can change sounds in real-time, it makes them come alive and lends a sense of flexibility to the composition.

For a human feeling I often filter the sounds using formant frequencies which simulate the resonant qualities of vowel sounds, thus offering a vocal quality to the sample. For alarm sounds in Sim Cell I used minor second intervals. This lent a sense of dissonance which informed players that they needed to adjust their gameplay in order to navigate treacherous areas. Motion was captured through a whoosh sound that used filtered and modulated noise. Some sounds were pitched up to make things seem like they were travelling towards the player and likewise they exploited a sense of doppler shift when pitched down, giving the feel that a sound was traveling away from the player. Together, these techniques produced a sense of immersion for the player while simultaneously building a realistic soundscape.

Our decision to use libPD for this synthesis turned problematic and its processing requirements were too high. In order to remedy this, we decided to convert our audio into samples. Consider a photograph of a sculpture. Our sounds, like sculpture in the photograph, could now only be viewed from one direction. This meant that the music now only had a single version and that sound effects would repeat as well. A small fix was exporting the file with a set of five different intensities from 0 to 1.0. Like taking a photograph from several angles, this meant that the game could play a sample at intensity levels of 20%, 40%, 60%, 80% and 100%. Although a truly random sense of variation was lost, this method still conveyed the intensity of impacts (and other similar events) generated by the physics engine of the game.

An image of the inexpensive open-source computer: The Rasberry PI

An image of the inexpensive open-source computer: The Rasberry PI. Photo used with permission by the author.

PD is a great way to learn and play with digital audio since you can change the patch while it is running, just like you might do with a real analogue synthesizer. There’s plenty of other neat stuff that PD can do, like being able to be run on the Raspberry Pi, so you could code your own effects pedals and make your own synths using PD for around $50 or so. For video games, you can use libPD to integrate PD patches into your Android or iOS apps as well. I hope this essay has offered some insight as to my process when using PD. I’ve included some links below for those interested in learning more.

Additional resources:

-

Leonard J. Paul attained his Honours degree in Computer Science at Simon Fraser University in BC, Canada with an Extended Minor in Music concentrating in Electroacoustics. He began his work in video games on the Sega Genesis and Super Nintendo Entertainment System and has a twenty year history in composing, sound design and coding for games. He has worked on over twenty major game titles totalling over 6.4 million units sold since 1994, including award-winning AAA titles such as EA’s NBA Jam 2010NHL11Need for Speed: Hot Pursuit 2NBA Live ’95 as well as the indie award-winning title Retro City Rampage.

He is the co-founder of the School of Video Game Audio and has taught game audio students from over thirty different countries online since 2012. His new media works has been exhibited in cities including Surrey, Banff, Victoria, São Paulo, Zürich and San Jose. As a documentary film composer, he had the good fortune of scoring the original music for multi-awarding winning documentary The Corporation which remains the highest-grossing Canadian documentary in history to date. He has performed live electronic music in cities such as Osaka, Berlin, San Francisco, Brooklyn and Amsterdam under the name Freaky DNA.

He is an internationally renowned speaker on the topic of video game audio and has been invited to speak in Vancouver, Lyon, Berlin, Bogotá, London, Banff, San Francisco, San Jose, Porto, Angoulême and other locations around the world.

His writings and presentations are available at http://VideoGameAudio.com

-

Featured image: Concept art for Sim Cell. Used with permission (c) 2014 Amplify.

tape reelREWIND! . . .If you liked this post, you may also dig:

Sounding Out! Podcast #10: Interview with Theremin Master Eric Ross- Aaron Trammell

Papa Sangre and the Construction of Immersion in Audio Games- Enongo Lumumba-Kasongo

Playing with Bits, Pieces, and Lightning Bolts: An Interview with Sound Artist Andrea Parkins

%d bloggers like this: