Welcome back to Sculpting the Film Soundtrack, SO!‘s new series on changing notions about how sound works in recent film and in recent film theory, edited by Katherine Spring.
Two weeks ago, Benjamin Wright started things off with a fascinating study of Hans Zimmer, a highly influential composer whose film scoring borders on engineering — or whose engineering borders on music — in many major Hollywood releases. This week we turn to the opposite end of the spectrum to a seemingly smaller film, Shane Carruth’s Upstream Color (2013), which has made quite a few waves among sound studies scholars and fans of sound design, even earning a Special Jury Prize for sound at Sundance.
To unpack the many mysteries of the film and explore its place in the field of contemporary filmmaking, we are happy to welcome musicologist and film scholar Danijela Kulezic-Wilson of University College Cork. Listen to Upstream Color through her ears (it’s currently available to stream on Netflix) and perhaps you’ll get a sense of why you’ll have to listen to it two or three more times. At least.
When Shane Carruth’s film Upstream Color was released in 2013, critics described it in various ways—as a body horror film, a sci-fi thriller, a love story, and an art-house head-scratcher—but they all agreed that it was a film “not quite like any other”. And while the film’s cryptic imagery and non-linear editing account for most of the “what the hell?” reactions (see here for example), I argue that the reason for its distinctively hypnotic effect is Carruth’s musical approach to the film’s form: he organizes the images and sounds according to principles of music, including the use of repetition, rhythmic structuring, and antiphony.
The resulting musicality of Upstream Color may not be surprising given that Carruth composed most of the score, and also, as Jonathan Romney has noted in Sight & Sound, Carruth has said on many occasions that he was hoping “people would watch this film repeatedly, as they might listen to a favourite album” (52). In this sense, Carruth (whose DIY toolkit also includes writing, directing, acting, producing, cinematography, and editing) joins the ranks of filmmakers such as Darren Aronofsky and Joe Wright who recognize that, despite our culture’s obsession with the cinematic and narrative aspects of “visual” media, music governs film’s deepest foundations.
Upstream Color is a story about a woman, Kris, who is kidnapped by a drug manufacturer (referred to in the credits as Thief) and contaminated with a worm that keeps her in a trance-like state during which the Thief strips her of all her savings. Kris is subsequently dewormed by a character known as the Sampler, who transfers the parasite into a pig that maintains a physical and/or metaphorical connection to Kris. Kris later meets and falls in love with Jeff who, we eventually discover, has been a victim of the same ordeal. Although the bizarreness of the plot has encouraged numerous interpretations, the film’s unconventional audio-visual language suggests that its story of two people who share supressed memories of the same traumatic experience shouldn’t be taken at face value, but rather serves as a metaphor for existential anxiety resulting from being influenced by unknown forces.
Such an interpretation owes as much to the film’s disregard for the rules of classical storytelling as it does to a formally innovative soundtrack, one that uses musicality as an overarching organizing principle. The fact that Carruth wrote the score and script simultaneously (discussed in the video below) indicates the extent to which music was from the beginning considered an integral part of the film’s expressive language. More importantly, as the scenes discussed in this post suggest, the musical logic of the film is even more pervasive than the style, role, and placement of the actual score.
Whereas feature films traditionally assign a central role to speech, allowing music and sound effects supporting roles only, Upstream Color breaks down the conventional soundtrack hierarchy, often reversing the roles of each constitutive element. For example, hardly any information in the film that could be considered vital to understanding the story is communicated through speech. Instead, images, sound, music, and editing–for which Carruth shares the credit with fellow indie director David Lowery–are the principal elements that create the atmosphere, convey the sense of the protagonists’ brokenness, and reveal the connection between the characters. At the same time, characters’ conversations are either muted or their speech is blended with music in such a way that we’re encouraged to focus on body language or mise-en-scène rather than trying to discern every spoken word. For example, Jeff and Kris’s flirting with one another during their initial meetings (at roughly 0.44.20-0.47.00 of the film) is conveyed primarily through gestures, glances, and fragmentary editing rather than speech, which would be more typical for this sort of narrative situation.
Further undercutting the significance of speech across the film is how the film has been edited to resemble the flow of music. For example, non-linear jumps in the narrative are often arranged in such a way as to create syncopated audio-visual rhymes. This technique is particularly obvious in the montage sequence in which Kris and Jeff argue over the ownership of their memories, whose similarities suggest that they were implanted during the characters’ kidnappings. In this sequence, both the passing of time and the recurrence of the characters’ argument is conveyed through the repetition of images that become visual refrains: Kris and Jeff lying on a bed, watching birds flying above trees, touching each other. Some of these can be seen in the film’s official trailer:
The scene’s images and sounds are fragmented into a non-linear assembly of pieces of the same conversation the characters had at different times and places, like the verses and the choruses of a song. Importantly, the assemblage is also patterned, with phrases like “we should go on a trip” and “where should we go?” heard in refrain. The first time we hear Jeff say “we should go on a trip” and suggesting that they go “somewhere bright”, his words are played in sync with the image of him and Kris lying on the bed. The following few shots, accompanied only by music, symbolize the “honeymoon” phase of their relationship: the couple kiss, hold hands, and walk with their hands around each other’s waists. A shift in mood is marked by the repetition of the dialogue, with Jeff again saying “we should go on a trip” – only this time, the phrase plays asynchronously over a shot of Jeff and Kris pushing a table into the house that they have moved into together. Finally, the frustration that starts infiltrating the characters’ increasingly heated arguments is alleviated by the repetition of the sentence “They could be starlings.” As it is spoken three times by both characters in an antiphonic exchange, the phrase emphasizes the underlying strength of their connection and gives the scene a rhythmic balance. Across this sequence, the musical organization of audio-visual refrains prompts us to recognize the psychic connection between Kris and Jeff, and even to begin to guess the sinister reason for it.
While speech in Upstream Color is often stripped of its traditional role as a principal source of information, sound and music are given important narrative functions, illuminating hidden connections between the characters. In one of the most memorable scenes, the Sampler is revealed to be not only a pig farmer but also a field recordist and sound artist who symbolizes the hidden source of everything that affects Kris and Jeff from afar. As we hear the sounds of the Sampler’s outdoor recordings merge with and emulate the sounds made and heard by Kris and Jeff at home and at work, the soundtrack eloquently establishes the connection between all three characters while also giving us a look “behind the scenes” of Kris’s and Jeff’s lives and suggesting how they are influenced from a distance.
In one sense, by calling attention to the very act of recording sound, the scene exposes how films are constructed, offering a reflexive glimpse into usually hidden processes of production. The implied idea here–that the visible and audible are products of not-so-obvious processes of formation–refers not only to the medium of film but also to the complexity of the inner workings of someone’s mind. Thus the Sampler’s role, his actions, and his relationship to Kris, Jeff, and other infected victims can be interpreted as a metaphor for the subconscious programming – all the familial, social and cultural influences – that all of us are exposed to from an early age. The Sampler is portrayed symbolically as the Creator, a force whose actions affect the protagonists’ lives without them knowing it. The fact that he is simultaneously represented as a sound artist establishes sound-making and musicality as the film’s primary creative principles.
Considering Carruth’s very deliberate departure from the conventions of even what David Bordwell calls “intensified” storytelling, it is fair to say that Upstream Color is a film that weakens the strong narrative role traditionally given to oral language. What is intensified here are the musical and sensuous qualities of the audio-visual material and a mode of perception that encourages absorption of the subtext (in other words, the metaphorical meaning of the film) as well as the text.
The musical organization of film form and soundtrack is no longer limited to independent projects such as Carruth’s Upstream Color. As I have shown elsewhere, musicality has become an extremely influential principle in contemporary cinema, acting as an inspiration and model for editing, camera movement, movement within a scene and sound design. Some of the most interesting results of a musical approach to film include Aronofsky’s “hip hop montage” in Pi (1998) and Requiem for a Dream (2000), Jim Jarmusch’s rhythmically structured film poems (The Limits of Control, 2009; Only Lovers Left Alive, 2013), the interchangeable use of musique concrète and environmental sound in Gus Van Sant’s Death Trilogy and films by Peter Strickland (Katalin Varga, 2009; Berberian Sound Studio, 2012); the choreographed mise-en-scène in Joe Wright’s Anna Karenina (2012); the musicalization of language in Harmony Korine’s Spring Breakers (2012); and the foregrounding of musical material over intelligible speech in Drake Doremus’s Breathe In (2013). Given the breadth of these examples, it’s no exaggeration to say that filmmakers’ growing affinity for a musical approach to film is changing the landscape of contemporary cinema.
Danijela Kulezic-Wilson teaches film music, film sound, and comparative arts at University College Cork. Her research interests include approaches to film that emphasize its inherent musical properties, the use of musique concrète and silence in film, the musicality of sound design, and musical aspects of the plays of Samuel Beckett. Danijela’s publications include essays on film rhythm, musical and film time, the musical use of silence in film, Darren Aronofsky’s Pi, P.T. Anderson’s Magnolia, Peter Strickland’s Katalin Varga, Gus Van Sant’s Death Trilogy, Prokofiev’s music for Eisenstein’s films, and Jim Jarmusch’s Dead Man. She has also worked as a music editor on documentaries, short films, and television.
All images taken from the film.
REWIND! . . .If you liked this post, you may also dig:
Animal Renderings: The Library of Natural Sounds– Jonathan Skinner
Welcome to World Listening Month 2014, our annual forum on listening in observation of World Listening Day on July 18th, 2014. World Listening Day is a time to think about the impacts we have on our auditory environments and, in turn, its affects on us [for the full deets, peep our recent SO! Amplifies post by Eric Leonardson, Executive Director of the World Listening Project]. We kick off our month of thinking critically about listening with a post by media historian Brian Hanrahan, who listens deeply to sonic traces of the past to prompt us to question our desires for contemporary media representations of “reality.” It also marks the global 100 year anniversary of World War I this August 2014: a moment of silence. –J. Stoever, Editor-in-Chief
For some reason that I don’t fully understand, I am very emotionally moved by the space around a sound. I almost think that sometimes I am recording space with a sound in it, rather than sound in a space. -Walter Murch
If you want to listen to the past, there’s never been a time like the present. Every year, it seems, new old recordings are identified, new techniques developed to recover sounds thought irrecoverable. Here is Bismarck’s voice, preserved on a cylinder in 1889. Here, older still, is Edison’s. There is the astonishing recuperation of phonautograms – reverberation traced onto soot-blackened paper in the mid-nineteenth century, digitally processed and played back in our own. But as that processing underlines, no sound recording straightforwardly reproduces the real. An acoustic artifact is a compound of materiality, form and meaning, but also a place where technology meets desire. Old recordings meet the listener’s longing halfway; they invoke a reality always out of reach. And not simply a longing to hear, but also to touch, and be moved by, the fact of an absent existence.
Take, for instance, HMV 09308. In October 1918, just before the end of the Great War, William Gaisberg, a sound recordist of the pre-electric era, took recording equipment to the Western Front in order to capture the sound of British artillery shelling German lines with poison gas. Gaisberg died not long after, probably from Spanish flu, although some say he was weakened by gas exposure during the recording. Nonetheless the “Gas Shell Bombardment” record – a 12-inch HMV shellac disc, just over 2 minutes at 78 rpm – was released a few weeks later, just as the war came to an end. Initially intended to promote War Bonds, ultimately the record was used to raise money for disabled veterans.
For decades, the HMV recording had a reputation as one of the very earliest “actuality” recordings – one documenting a real location and event beyond the performative space of the studio, imprinted with the audible material trace of an actual moment in space and time. Documents like this – no matter what the technology – usually come with additional symbolic authentication. Here, the record’s label does some of that work. This “historic recording,” says the subtitle, is an “actual record taken on the front line.” Publicity pieces drove home the message. In the popular HMV magazine The Voice, Gaisberg – or probably his posthumous ghost-writer – described the expedition in detail, claiming the track to be a “true representation of the bombardment.”
In the same issue, a Major C.J.C. Street compared the recording to his own experience on the Front. “Its realism,” he wrote, ”took my breath away… I played the record many times… finding at each attempt some well-remembered detail.” He didn’t say so in his article, but Street – an artillery officer, a novelist and a propaganda man for the intelligence agency MI7 – was in fact the impresario of the record. This was not the first time he had found astute uses for sound media. The previous year he had put together a record that set artillery drill commands to popular tunes – the recording was both a propaganda release and an army training tool for new recruits. With the Gas Shell record, Street knew he wasn’t just selling recorded sound, but also an auratic sense of closeness to an overwhelming reality, the palpable proximity of war and death. Authenticating detail helped to underpin this sense of an absent real made present. Street cued the listener for those “well-remembered details.” In particular, he singled out one indistinct rattly flap-whizz noise, hearing in it, he claimed, the sound of a round with a “loose driving-band.”
The record stayed in the HMV catalog until 1945, but only in the early 1990s were its production history and authenticity claims seriously examined. In specialist journals, archivists, collectors and amateur historians undertook a collective forensic and critical analysis. A promising auditory witness was located: 95-year-old Lt.-Col. Montagu Cleeve another former artillery officer, in his time a developer of “Boche Buster” railway gun, later a music professor – was invited to critically assess the recording. Cleeve vouched unreservedly for its authenticity. He heard in it, he said, an unmistakable succession of sounds – the clang of the breech, the gigantic report of the firing explosion, the distinctive whiny whistle of a gas shell on its way across no-man’s-land. Others looked to data rather than the memories of old soldiers. One expert on pre-electric recording noted the angles commanded in firing instructions, correlated them with known muzzle velocities for 4.5 and 6-inch howitzers, then used this and other information to “definitively” explain the counter-intuitive anti-Doppler sound of the shells’ whistling. He also identified the audible echo effect – the curious “double report” of the guns heard here – as the sound of a brass recording horn violently resonating at a distance of exactly 26.5 meters from the guns.
Eventually, skepticism won out. Close listening at slow speeds – just careful attention and notation, nothing more elaborate – revealed inconsistencies and oddities in the firing noises. The bongs, plops and whistles seemed internally inconsistent. Some of the artillery sounds – ostensibly a battery of four, firing in quick succession – varied implausibly with each successive firing. Physical evidence from the record’s groove, as well as extraneous noises – surface crackle and fizz, and, audible within the recording, the swish of a turntable – seemed to indicate at least two rudimentary overdubs, in which the output of one acoustic horn was relayed into a second, possibly using an auxetophone, an early compressed-air amplifier. All this resulted in a double- or triple-layered sonic artifact. Finally – the crucial evidence, although oddly it was hardly noticed at the time – an alternative take was located. In this take, according to its discoverer, the entire theatrics of gunnery command is simply absent, and there is no sound at all of whistling shells in motion. What was left was a skeleton sequence of clicks, thuds and cracks, supplemented with only a single closing insert, the portentous injunction “Feed the Guns with War Bonds!”
In short, it seems highly likely that any original field recording was, at the very least, post-dramatized with performed voices and percussive and whistling sound effects. So, it is tempting to say, that clears that up. The recording’s inauthenticity is proven. File under Fake. But in fact, if we don’t stop there, if we set aside narrow and absolutist ideas of authenticity, and instead explore the recording’s ambiguity and hybridity, then Gas Shell Bombardment becomes all the more interesting as an historical artifact.
Let’s assume, for the sake of argument, that some form of basic recording was done in France, very possibly a staged barrage specifically performed for Gaisberg’s visit, and that this recording then had effects added back at HMV in London. The record might then be seen less as a straightforward documentary, and instead as an unusual version of the “descriptive speciality,” a genre of miniature phonographic vignette dating back to the 1890s, far predating longer-form radio drama. Very little is known about these early media artworks, but it is a fair generalization to say that in America the genre was more slanted towards vaudeville comedy, whereas in Europe, imperial and military scenes predominated. As early as 1890, for example, there had been German phonographic representations of battles from the Franco-Prussian war. The Great War saw a flourishing of the genre. Scholars are just beginning to take an interest these old phonographs; here’s one recent essay on the “Angel of Mons,” for example, a British acoustic vignette of a famous incident on the Western Front.
Listen to a 1915 German descriptive speciality, depicting the attack on the fortress of Liège the previous year:
As a descriptive speciality, Gas Shell Bombardment is unusual because it incorporates an actual indexical trace. But such traces – as emphasized by Charles Sanders Pierce and many later media-theoreticians– do not resemble their referent, they are caused by it. The bullet hole does not look much like a bullet; thunder is lightning’s trace, not its likeness. But for Street and Gaisberg, the trace’s lack of resemblance caused problems: the original recording’s lack of detail, cues and clues, but above all its lack of internal dimensionality, created a perceptual shortfall and a lack of credibility. Maybe they hoped that the guns, by sheer force of amplitude, would overcome the spatially impoverished, reverbless reproduction of pre-electric recording. If so, it didn’t work. Without added effects, the guns’ trace was as flat and “body-less” as a sequence of Morse. It was a sound without a scene. The producers’ interventions aimed to thicken the primary artifact with referential-sounding detail, but also to heighten the sense of materiality and spatiality, and to strengthen the sense of diegetic presence, of worlded thereness. The soldiers’ voices – louder and quieter, close-up and farther-out – and the fake-Doppler of the “shell whistling” lent the recording narrative direction (literally, some trajectory) and “authenticating” points of detail. But above all they gave a sense of internal space to the recording, a space into which the listener could direct her attention.
In this context, we can only admire the creativity and performative élan of the unknown production crew. We know little about effects production in early phonography. It is a safe bet that some techniques were adopted from theatre, and that there was overlap with silent film accompaniment. But whatever the method used, it would have called for the awkward orchestration of a limited number of iconic sounds to create an impression of a spatially coherent and materially detailed sonic environment. The recordist and his team would first have had to imagine how relative loudness – of voices, of material objects struck and sounded – might create a sense of spatial depth when transduced through the horn’s crude interface. Then they would have had to perform this as a live overdub, keeping time with the base track of the gun recording played through another horn. And all this done with participants and equipment crowded tightly around the mouth of the huge horn, crammed into the tiny pick-up arc, a scene looking something like this image of Leopold Stokowski’s pre-electric recording sessions or this photograph of the recording of a cello concerto.
As well as this hybrid of trace and live performance, there is another performance here – Gaisberg’s journey itself. With twenty years of recording experience, Gaisberg was probably very well aware that the expedition would not yield a “realistic” recording of the guns. But the expedition had to be made, so that it could be said to have taken place. Expectations had to be primed and colored, so that, to use André Bazin’s famous phrase about photographs, the recording could partake in an “irrational power to… bear the belief” of the listener. The journey, and the accounts of Gaisberg and Street are not a supplement to the “true representation” of the gas bombardment. They are part of that representation. Moreover, in subsequent writing it is noticeable that the manner of Gaisberg’s death becomes a rhetorical amplification for the authenticity of the recording’s trace, as if his fatal inhalation (of gas molecules or flu bacilli) were itself a deadly indexation, paralleling the recording’s claim to capture the breath of the War, and even of History itself.
In media-historical terms, the Gas Shell Bombardment recording can be understood as a late, transitional artifact from phonography’s pre-microphonic era. The desire for the sonic trace, for an ever more immersive proximity to events was there, but electro-acoustic technology was not yet in place. Two years later, in 1920, Horace Merriman and Lionel Guest made the first experimental electrical recording, arguably also the first true field recording. The event, appropriately enough, was an official war memorial service in London, where Merriman and Guest – working for Columbia Records – put microphones in Westminster Abbey, running cables to a remote recording van parked in the street outside, where they sat amidst heating ovens and cutting lathes. By the end of the 1920s, remote recording and broadcasting, while never straightforward, were well on the way to ubiquity.
Claims made on behalf of technologies of reproduction may seem simplistic, but there’s a grain of truth to their simplicity. If there were nothing special – even magical – in the referentiality of the camera that captures the moment, the recording that’s like being there, the liveness of the live broadcast, these things would not play the role they do in everyday life and in the ideological fabric of society. But there is falsehood too, in over-simplifying the nature and affective charge of old photographs, old footage, old recordings. These are made things, composed of different materials, media, signs and conventions; they are inseparable from the desires and expectations they induce and direct. They function in part by mimesis and verisimilitude, but also through the gaps, blank spots and false illusions of their trace. They can – rightly – intensify our feeling towards the past, but should also prompt us to think about our own desires and investments.
Image by Flickr User DrakeGoodman, “Horchposten im Spengtrichter vor Neuve-Chapelle 6km nördlich von La Bassée Nordfrankreich 1916,” A trio of lightly equipped soldiers from an unidentified formation oblige the photographer by looking serious and pretending they’re just metres from the enemy, listening for activity in his lines. The improvised “listening device” is actually a large funnel, probably liberated from a nearby farm.
Brían Hanrahan is a film, media and cultural historian, whose work focuses on the history of acoustic media, German and European cinema and the culture of the Weimar Republic.
Edited post-publication at 8:00 pm EST on July 7, 2014
REWIND!…If you liked this post, you may also dig:
A Brief History of Auto-Tune–Owen Marshall
DIY Histories: Podcasting the Past–Andrew Salvati
SO! Amplifies. . .a highly-curated, rolling mini-post series by which we editors hip you to cultural makers and organizations doing work we really really dig. You’re welcome!
During a preview of the UK tour of my performance Beacons, I noticed something intriguing. There were a couple of children in the audience, and they were singing along with me. I mean, not just singing – doing all kinds of vocal things along with me.
You have to understand that I’m an extended – or I now prefer the term extra-normal – vocalist. I make artwork from the ‘weird’ qualities of the voice. Now, the term weird, when it comes to the voice, is always culturally relative. But let’s face it – in almost any of the industrialized cultures, we don’t exactly let the voice go wild, and so the category of “weird” is pretty large and full. Our cultures are a Freudian fiesta of vocal repression and sublimation. When’s the last time you heard a heartfelt, loose, easy series of adult sobs at a business meeting, or a wail of deep mourning at a funeral? When it comes to the voice, we’re pretty uptight.
However, here were some children absolutely letting loose and going along for the ride. This has happened before. During my piece Soie Soyeuse, during which I wrap the audience in habotai silk, some children got right into it. I was five feet away making really incredibly ‘ugly’ chewing noises and groans of gnashing teeth and they delightedly voiced along.
Right now, everything’s becoming user-driven. The audience loves to co-make art. And why not? So I thought: let’s go for broke, here. Let’s try to create some environments and experiences in which kids (and the kid in all of us) can really explore what their voices can do. Let’s give them freedom to experiment. Let’s reward them for wildness. Let’s try to make it child and family friendly, but beautiful and rich and somehow sensual, despite the limitations of the tablet format. I worked like a dog to get financing. I collaborated with scientists. And so, Voice Bubbles for iPad was born.
It’s free, thanks to our funders. It’s got no silly pop up ads and in-app purchases. Kids love it and will use it for hours, happily. It lets them record, improvise, structure and add effects to their voices. They can play back little sequences or structured phrases. They can fool with what else their voices can do and be.
It makes them feel, and makes we adults question, just why we need to keep the voice so under wraps when it’s a source of so much playful joy and intense engagement. Give it to a child to enjoy. Enjoy it yourself.
*You can explore other related artworks at www.yourvivaciousvoice.com
Yvon Bonenfant is Reader in Performing Arts at the University of Winchester. He likes voices that do what voices don’t usually do, and he likes bodies that don’t do what bodies usually do. He makes art starting from these sounds and movements. These unusual, intermedia works have been produced in 10 countries in the last 10 years, and his writing published in journals such as Performance Research, Choreographic Practices, and Studies in Theatre and Performance. He currently holds a Large Arts Award from the Wellcome Trust and funding from Arts Council England to collaborate with speech scientists on the development of a series of participatory, extra-normal voice artworks for children and families; see www.yourvivaciousvoice.com. Despite his air of Lenin, he does frighteningly accurate vocal imitations of both Axl Rose and Jon Bon Jovi. www.yvonbonenfant.com.
REWIND!…If you liked this post, you may also dig:
SO! Amplifies: Regina Bradley’s Outkasted Conversations– Regina Bradley
This is Your Body on the Velvet Underground–Jacob Smith
Digital Analogies: Techniques of Sonic Play–Roger Moseley
This post continues our summer Sound and Pleasure series, as the third and final podcast in a three part series by Leonard J. Paul. What is the connection between sound and enjoyment, and how are pleasing sounds designed? Pleasure is, after all, what brings y’all back to Sounding Out! weekly, is it not?
Part of the goal of this series of podcasts has been to reveal the interesting and invisible labor practices which are involved in sound design. In this final entry Leonard J. Paul breaks down his process in designing living sounds for the game Vessel. How does one design empathetic or aggressive sounds? If you need to catch up read Leonard’s last entry where he breaks down the vintage sounds of Retro City Rampage. Also, be sure to be sure to check out last week’s edition where Leonard breaks down his process in designing sound for Sim Cell. But first, listen to this! -AT, Multimedia Editor
CLICK HERE TO DOWNLOAD: Game Audio Notes III: The Nature of Sound in Vessel
SUBSCRIBE TO THE SERIES VIA ITUNES
ADD OUR PODCASTS TO YOUR STITCHER FAVORITES PLAYLIST
Game Audio Notes III: The Nature of Sound in Vessel
Strange Loop Game’s Vessel is set in an alternate world history where a servant class of liquid automatons (called fluros) has gone out of control. The player explores the world and solves puzzles in an effort to restore order. While working on Vessel, I personally recorded all of the sounds so that I could have full control over the soundscape. I recorded all of the game’s samples with a Zoom H4n portable recorder. My emphasis on real sounds was intended to focus the player’s experience of immersion in the game.
This realistic soundscape was supplemented with a variety of techniques that produced sounds that dynamically responded to the changes in the physics engine. Water and other fluids in the game were difficult to model with both the physics engine and the audio engine (FMOD Designer). Because fluids are fundamentally connected to the game’s physics engine, they takes on a variety of different dynamic forms as players interact with the fluid in different ways. In order to address this Kieran Lord, the audio coder, and I considered factors like the amount of liquid in a collision with anything, the hardness of the surface that it was colliding with, the type of liquid in motion, whether the player is experiencing an extreme form of that sound because it is colliding with their head, and, of course, how fast the liquid is travelling.
Although there was a musical score, I designed the effects to be played without music. Each element of the game, for instance a lava fluro’s (one of the game’s rebellious automatons) footsteps, entailed required layers of sound. The footsteps were composed of water sizzling on a hot pan, a gloopy slap of oatmeal and a wet rag hitting the ground. Finding the correct emotional balance to support the game’s story was fundamental to my work as a sound designer. The game’s sound effects were constantly competing with the adaptive music (which is also contingent on player action) that plays throughout the game, so it was important to provide an informative quality to them. The sound effects inform you about the environment while the music sets the emotional underscore of the gameplay and helps guide you in the puzzles.
Defining the character of the fluros was difficult because I wanted players to have empathy for them. This was important to me because there is often no way to avoid destroying them when solving the game’s puzzles. While recording sounds in the back of an antique shop, I came across a vintage Dick Tracey gun that made a fantastic clanking sound when making a siren sound. Since the gun allowed me to control how quickly the siren rose and fell, it was a great way to produce vocalizations for the fluros. I simply recorded the gun’s siren sound, chopped the recording into smaller pieces, and then played back different segments randomly. The metal clanking gave a mechanical feel and the siren’s tone gave a vocal quality to the resulting sound that was perfect for the fluros. I could make the fluros sound excited by choosing a higher pitch range from the sample grains and inform the player when they approached their goal.
I wanted a fluid-based scream to announce a fluro’s death. I tried screaming underwater, screaming into a glass of water, and a few other things, but nothing worked. Eventually, when recording a rubber ear syringe, I found squeezing the water out quickly lent a real shriek while it spit out the last of the water. Not only did this sound really cut through the din of the gears clanking in the mix, but it also bonded a watery yell with the sense of being crushed and running out of breath.
For the final boss, I tried many combinations of glurpy sounds to signify its lava form. Eventually I recorded a nail in a board being dragged across a large rusty metal sheet. Though it was quite excruciating to listen to, I pitched down the recording and combined it with a pitched down and granulated recording of myself growling into a cup of water. This sound perfectly captured the emotion I wanted to feel when encountering a final boss. Although it can take a long time to arrive at the “obvious” sound, simplicity is often the key.
Anticipation is fundamental to a player’s sense of immersion. It carves a larger space for tension to build, for instance a small crescendo of a creaking sound can develop a tension that builds to a sudden and large impact. A whoosh before a punch lands adds extra weight to the force of the punch. These cues are often naturally present in real-world sounds, such as a rush of air sweeping in before a door slams. A small pause might be included just for added suspense and helps to intensify the effect of the door slamming. Dreading the impact is half of the emotion of a large hit .
Recording all of the sounds for Vessel was a large undertaking but since I viewed each recording as a performance, I was able to make the feeling of the world very cohesive. Each sound was designed to immerse the player in the soundscape, but also to allow players enough time to solve puzzles without becoming annoyed with the audio. All sounds have a life of their own and a resonance of memory and time that stays with the them during each playthrough of a game. In Retro City Rampage I left a sonic space for the player to wax nostalgic. In Sim Cell, I worked to breathe life into a set of sterile and synthesized sounds. Each recorded sound in Vessel is alive in comparison, telling stories of time, place and recording with them, that are all their own.
The common theme of my audio work on Retro City Rampage, Sim Cell and Vessel, is that I enjoy putting constraints on myself to inspire my creativity. I focus on what works and removing non-essential elements. Exploring the limits of constraints often provokes interesting and unpredictable results. I like “sculpting” sounds and will often proceed from a rough sketch, polishing and reducing elements until I like what I hear. Typically I remove layers that don’t add an emotive aspect to the sound design. In games there are often many sounds that can play at once, so clarity and focus are necessary when preventing sounds from getting lost in a sonic goo.
In this post I have shown how play and experimentation are fundamental to my creative process. For an aspiring sound artist, spending time with Pure Data, FMOD Studio or Wwise and a personal recorder is a great way to improve their skill with game audio. This series of articles has aimed to reveal the tacit decisions behind the production of game audio that get obscured by the fun of the creative process. Plus, I hope they offer a bit of inspiration to those creating their own sounds in the future.
- My GDC 2012 talk – My site includes an MP3 recording of the talk and the full powerpoint presentation. A full video of the talk is in the GDC Vault but requires membership
- See the Jon Hopkins Soundtrack in the Video Game Vessel video game soundtrack video by me on YouTube – shows use of Lua with music in the custom level editor : https://www.youtube.com/watch?v=KOyjMPPvaY4
Leonard J. Paul attained his Honours degree in Computer Science at Simon Fraser University in BC, Canada with an Extended Minor in Music concentrating in Electroacoustics. He began his work in video games on the Sega Genesis and Super Nintendo Entertainment System and has a twenty year history in composing, sound design and coding for games. He has worked on over twenty major game titles totalling over 6.4 million units sold since 1994, including award-winning AAA titles such as EA’s NBA Jam 2010, NHL11, Need for Speed: Hot Pursuit 2, NBA Live ’95 as well as the indie award-winning title Retro City Rampage.
He is the co-founder of the School of Video Game Audio and has taught game audio students from over thirty different countries online since 2012. His new media works has been exhibited in cities including Surrey, Banff, Victoria, São Paulo, Zürich and San Jose. As a documentary film composer, he had the good fortune of scoring the original music for multi-awarding winning documentary The Corporation which remains the highest-grossing Canadian documentary in history to date. He has performed live electronic music in cities such as Osaka, Berlin, San Francisco, Brooklyn and Amsterdam under the name Freaky DNA.
He is an internationally renowned speaker on the topic of video game audio and has been invited to speak in Vancouver, Lyon, Berlin, Bogotá, London, Banff, San Francisco, San Jose, Porto, Angoulême and other locations around the world.
His writings and presentations are available at http://VideoGameAudio.com
Featured image: Courtesy of Vblank Entertainment (c)2014 – Artwork by Maxime Trépanier.
REWIND! . . .If you liked this post, you may also dig:
Sounding Out! Podcast #31: Hand Made Music in Retro City Rampage- Leonard J. Paul
Papa Sangre and the Construction of Immersion in Audio Games- Enongo Lumumba-Kasongo