Welcome to our new series Sculpting the Film Soundtrack, which brings you new perspectives on sound and filmmaking. As Guest Editor, we’re honored and delighted to have Katherine Spring, Associate Professor of Film Studies at Wilfrid Laurier University. Spring is the author of an exciting and important new book Saying it With Songs: Popular Music and the Coming of Sound to Hollywood Cinema. Read it! You’ll find an impeccably researched work that’s the definition of how the history of film sound and media convergence ought to be written.
But before rushing back to the early days, stick around here on SO! for the first of our three installments in Sculpting the Film Soundtrack.
It’s been 35 years since film editor and sound designer Walter Murch used the sounds of whirring helicopter blades in place of an orchestral string section in Apocalypse Now, in essence blurring the boundary between two core components of the movie soundtrack: music and sound effects. This blog series explores other ways in which filmmakers have treated the soundtrack as a holistic entity, one in which the traditional divisions between music, effects, and speech have been disrupted in the name of sculpting innovative sonic textures.
In three entries, Benjamin Wright, Danijela Kulezic-Wilson, and Randolph Jordan will examine the integrated soundtrack from a variety of perspectives, including technology, labor, aesthetic practice, theoretical frameworks, and suggest that the dissolution of the boundaries between soundtrack categories can prompt us to apprehend film sound in new ways. If, as Murch himself once said, “Listening to interestingly arranged sounds makes you hear differently,” then the time is ripe for considering how and what we might hear across the softening edges of the film soundtrack.
– Guest Editor Katherine Spring
Composing a sound world for Man of Steel (2013), Zack Snyder’s recent Superman reboot, had Hans Zimmer thinking about telephone wires stretching across the plains of Clark Kent’s boyhood home in Smallville. “What would that sound like,” he said in an interview last year. “That wind making those telephone wires buzz – how could I write a piece of music out of that?” The answer, as it turned out, was not blowing in the wind, but sliding up and down the scale of a pedal steel guitar, the twangy lap instruments of country music. In recording sessions, Zimmer instructed a group of pedal steel players to experiment with sustains, reverb, and pitches that, when mixed into the final track, accompany Superman leaping over tall buildings at a single bound.
His work on Man of Steel, just one of his most recent films in a long and celebrated career, exemplifies his unique take on composing for cinema. “I would have been just as happy being a recording engineer as a composer,” remarked Zimmer last year in an interview to commemorate the release of a percussion library he created in collaboration with Spitfire Audio, a British sample library developer. “Sometimes it’s very difficult to stop me from mangling sounds, engineering, and doing any of those things, and actually getting me to sit down and write the notes.” Dubbed the “HZ01 London Ensembles,” the library consists of a collection of percussion recordings featuring many of the same musicians who have performed for Zimmer’s film scores, playing everything from tamtams to taikos, buckets to bombos, timpani to anvils. According to Spitfire’s founders, the library recreates Zimmer’s approach to percussion recording by offering a “distillation of a decade’s worth of musical experimentation and innovation.”
In many ways, the collection is a reminder not just of the influence of Zimmer’s work on contemporary film, television, and video game composers but also of his distinctive approach to film scoring, one that emphasizes sonic experimentation and innovation. Having spent the early part of his career as a synth programmer and keyboardist for new wave bands such as The Buggles and Ultravox, then as a protégé of English film composer Stanley Myers, Zimmer has cultivated a hybrid electronic-orchestral aesthetic that uses a range of analog and digital oscillators, filters, and amplifiers to twist and augment solo instrument samples into a synthesized whole.
Zimmer played backup keyboards on “Video Killed the Radio Star.”
In a very short time, Zimmer has become a dominant voice in contemporary film music with a sound that blends melody with dissonance and electronic minimalism with rock and roll percussion. His early Hollywood successes, Driving Miss Daisy (1989) and Days of Thunder (1990), combined catchy themes and electronic passages with propulsive rhythms, while his score for Black Rain (1989), which featured taiko drums, electronic percussion, and driving ostinatos, laid the groundwork for an altogether new kind of action film score, one that Zimmer refined over the next two decades on projects such as The Rock (1996), Gladiator (2000), and The Pirates of the Caribbean series.
What is especially intriguing about Zimmer’s sound is the way in which he combines the traditional role of the composer, who fashions scores around distinct melodies (or “leitmotifs”), with that of the recording engineer, who focuses on sculpting sounds. Zimmer may not be the first person in the film business to experiment with synthesized tones and electronic arrangements – you’d have to credit Bebe and Louis Barron (Forbidden Planet, 1956), Vangelis (Chariots of Fire, 1981), Jerry Goldsmith (Logan’s Run, 1976), and Giorgio Moroder (Midnight Express, 1981) for pushing that envelope – but he has turned modern film composing into an engineering art, something that few other film composers can claim.
One thing that separates Zimmer’s working method from that of other composers is that he does not confine himself to pen and paper, or even keyboard and computer monitor. Instead, he invites musicians to his studio or a sound stage for an impromptu jam session to find and hone the musical syntax of a project. Afterwards, he returns to his studio and uses the raw samples from the sessions to compose the rest of the score, in much the same way that a recording engineer creates the architecture of a sound mix.
“There is something about that collaborative process that happens in music all the time,” Zimmer told an interviewer in 2010. “That thing that can only happen with eye contact and when people are in the same room and they start making music and they are fiercely dependent on each other. They cannot sound good without the other person’s part.”
Zimmer facilitates the social and aesthetic contours of these off-the-cuff performances and later sculpts the samples into the larger fabric of a score. In most cases, these partnerships have provided the equivalent of a pop hook to much of Zimmer’s output: Lebo M’s opening vocal in The Lion King (1994), Johnny Marr’s reverb-heavy guitar licks in Inception, Lisa Gerrard’s ethereal vocals in Gladiator and Black Hawk Down (2002), and the recent contributions of the so-called “Magnificent Six” musicians to The Amazing Spider Man 2 (2014).
The melodic hooks are simple but infectious – even Zimmer admits he writes “stupidly simple music” that can often be played with one finger on the piano. But what matters most are the colors that frame those notes and the performances that imbue those simple melodies with a personality. Zimmer’s work on Christopher Nolan’s Dark Knight trilogy revolves around a deceptively simple rising two-note motif that often signifies the presence of the caped crusader, but the pounding taiko hits and bleeding brass figures that surround it do as much to conjure up images of Gotham City as cinematographer Wally Pfister’s neo-noir photography. The heroic aspects of the Batman character are muted in Zimmer’s score except for the presence of the expansive brass figures and taiko hits, which reach an operatic crescendo in the finale, where the image of Batman escaping into the blinding light of the city is accompanied by a grand statement of the two-note figure backed by a driving string ostinato. Throughout the series, a string ostinato and taikos set the pace for action sequences and hint at the presence of Batman who lies somewhere in the shadows of Gotham.
Zimmer’s expressive treatment of musical colors also characterizes his engineering practices, which are more commonly used in the recording industry. Music scholar Paul Théberge has noted that the recording engineer’s interest in an aesthetic of recorded musical “sound” led to an increased demand for control over the recording process, especially in the early days of multitrack rock recording where overdubbing created a separate, hierarchical space for solo instruments. Likewise for Zimmer, it’s not just about capturing individual sounds from an orchestra but also layering them into a synthesized product. Zimmer is also interested in experimenting with acoustic performances, pushing musicians to play their instruments in unconventional ways or playing his notes “the wrong way,” as he demonstrates here in the making of the Joker’s theme from The Dark Knight:
The significance of the cooperative aspects of these musical performances and their treatment as musical “colors” to be modulated, tweaked, and polished rests on a paradoxical treatment of sound. While he often finds his sound world among the wrong notes, mistakes, and impromptu performances of world musicians, Zimmer is also often criticized for removing traces of an original performance by obscuring it with synth drones and distortion. In some cases, like in The Peacemaker (1997), the orchestration is mushy and sounds overly processed. But in other cases, the trace of a solo performance can constitute a thematic motif in the same way that a melody serves to identify place, space, or character in classical film music. Compare, for instance, Danny Elfman’s opening title theme for Tim Burton’s Batman (1989) and Zimmer’s opening title music for The Dark Knight. While Elfman creates a suite of themes around a central Batman motif, Zimmer builds a sparse sound world that introduces a sustained note on the electric cello that will eventually be identified with the Joker. It’s the timbre of the cello, not its melody, that carries its identifying features.
To texture the sounds in Man of Steel, Zimmer also commissioned Chas Smith, a Los Angeles-based composer, performer, and exotic instrument designer to construct instruments from “junk” objects Smith found around the city that could be played with a bow or by hand while also functioning as metal art works. The highly abstract designs carry names that give some hint to their origins – “Bertoia 718” named after modern sculptor and furniture designer Harry Bertoia; “Copper Box” named for the copper rods that comprise its design; and “Tin Sheet” that, when prodded, sounds like futuristic thunderclaps.
Smith’s performances of his exotic instruments are woven into the fabric of the score, providing it with a sort of musical sound design. Consider General Zod’s suite of themes and motifs, titled “Arcade” on the 2-disc version of the soundtrack. The motif is built around a call-and-answer ostinato for strings and brass that is interrupted by Smith’s sculptural dissonance. It’s the sound of an otherworldly menace, organic but processed, sculpted into a conventional motif-driven sound world.
Zimmer remains a fixture in contemporary film music partly because, as music critic Jon Burlingame has pointed out, he has a relentless desire to search for fresh approaches to a film’s musical landscape. This pursuit begins with his extracting of sounds and colors from live performances and electronically engineering them during the scoring process. Such heightened attention to sound texture and color motivated the creation of the Spitfire percussion library, but can only hint at the experimentation and improvisational nature that goes into Zimmer’s work. In each of his film scores, the music tells a story that is tailored to the demands of the narrative, but the sounds reveal Zimmer’s urge to manipulate sound samples until they are, in his own words, “polished like a diamond.”
Ben Wright holds a Provost Postdoctoral Fellowship from the University of Southern California in the School of Cinematic Arts. In 2011, he received his Ph.D. in Cultural Studies from the Institute for Comparative Studies in Literature, Art and Culture at Carleton University. His research focuses on the study of production cultures, especially exploring the industrial, social, and technological effects of labor structures within the American film industry. His work on production culture, film sound and music, and screen comedy has appeared in numerous journals and anthologies. He is currently completing a manuscript on the history of contemporary sound production, titled Hearing Hollywood: Art, Industry, and Labor in Hollywood Film Sound.
All images creative commons.
REWIND! . . .If you liked this post, you may also dig:
A pair of firsts! This is both the lead post in our summer Sound and Pleasure series, and the first podcast in a three part series by Leonard J. Paul. What is the connection between sound and enjoyment, and how are pleasing sounds designed? Pleasure is, after all, what brings y’all back to Sounding Out! weekly, is it not?
In today’s installment Leonard peels back the curtain of game audio design and reveals his creative process. For anyone curious as to what creative decisions lead to the bloops, bleeps, and ambient soundscapes of video games, this is essential listening. Stay tuned for next Monday’s installment on the process of designing sound for Retro City Rampage, and next month’s episode which focuses on the game Vessel. Today, Leonard begins by picking apart his design process at a cellular level. Literally! -AT, Multimedia Editor
CLICK HERE TO DOWNLOAD: Game Audio Notes I: Growing Sounds for Sim Cell
SUBSCRIBE TO THE SERIES VIA ITUNES
ADD OUR PODCASTS TO YOUR STITCHER FAVORITES PLAYLIST
Game Audio Notes I: Growing Sounds for Sim Cell
Sim Cell is an educational game released in Spring 2014 by Strange Loop Games. Published by Amplify, a branch of News Corp, it teaches students how the human cell works. Players take control of a small vessel that is shrunk down to the size of a cell and solve the tasks set by the game while also learning about the human cell. This essay unpacks the design decisions behind the simulation of a variety of natural phenomena (motion, impact, and voice) in Sim Cell.
For the design of this game I decided to focus on the elements of life itself and attempted to “grow” the music and sound design from synthetic sounds. I used the visual scripting language Pure Data (PD) to program both the sounds and the music. The music is generated from a set of rules and patterns that cause each playback of a song to be slightly different each time. Each of the sound effects are crafted from small programs that are based on the design of analogue modular synthesizers. Basic synthetic audio elements such as filtered noise, sawtooth waves and sine waves were all used in the game’s sound design.
The visuals of the game give a feeling of being in an “inner space” that mirrors outer space. I took my inspiration for Sim Cell‘s sound effects from Louis and Bebe Baron’s score to Forbidden Planet. I used simple patches in PD to assemble all of the sounds for the game from synthesis. There aren’t any recorded samples in the sound design – all of the sound effects are generated from mathematics.
The digital effects used in the sound design were built to emulate the effects available in vintage studios such as plate reverb and analog delay. I found a simulation of a plate reverb that from a modified open source patch from the RjDj project, and constructed an analogue delay by using a low-pass filter on a delayed signal.
In keeping with the vintage theme, I used elements of sound design from the early days of video games as well. Early arcade games such as Space Invaders used custom audio synthesis microchips for each of their sounds. In order to emulate this, I gave every sound its own patch when doing the synthesis for Sim Cell. I learned to appreciate this ethic of design when playing Combat on the Atari 2600 while growing up. The Atari 2600 could only output two sounds at once thus had a very limited palette of tones. All of the source code for the sounds and music for Sim Cell are less than 1 megabyte, which shows how powerful the efficient coding of mathematics can be.
Another cool thing about generating the sounds from mathematics is that users can “zoom in” on sounds in the same way that one can zoom in to a vector drawing. In vector drawings the lines are smooth when you zoom in, as opposed to rasterized pictures (such as a JPEG) which reveal a blurry set of pixels upon zooming. When code can change sounds in real-time, it makes them come alive and lends a sense of flexibility to the composition.
For a human feeling I often filter the sounds using formant frequencies which simulate the resonant qualities of vowel sounds, thus offering a vocal quality to the sample. For alarm sounds in Sim Cell I used minor second intervals. This lent a sense of dissonance which informed players that they needed to adjust their gameplay in order to navigate treacherous areas. Motion was captured through a whoosh sound that used filtered and modulated noise. Some sounds were pitched up to make things seem like they were travelling towards the player and likewise they exploited a sense of doppler shift when pitched down, giving the feel that a sound was traveling away from the player. Together, these techniques produced a sense of immersion for the player while simultaneously building a realistic soundscape.
Our decision to use libPD for this synthesis turned problematic and its processing requirements were too high. In order to remedy this, we decided to convert our audio into samples. Consider a photograph of a sculpture. Our sounds, like sculpture in the photograph, could now only be viewed from one direction. This meant that the music now only had a single version and that sound effects would repeat as well. A small fix was exporting the file with a set of five different intensities from 0 to 1.0. Like taking a photograph from several angles, this meant that the game could play a sample at intensity levels of 20%, 40%, 60%, 80% and 100%. Although a truly random sense of variation was lost, this method still conveyed the intensity of impacts (and other similar events) generated by the physics engine of the game.
PD is a great way to learn and play with digital audio since you can change the patch while it is running, just like you might do with a real analogue synthesizer. There’s plenty of other neat stuff that PD can do, like being able to be run on the Raspberry Pi, so you could code your own effects pedals and make your own synths using PD for around $50 or so. For video games, you can use libPD to integrate PD patches into your Android or iOS apps as well. I hope this essay has offered some insight as to my process when using PD. I’ve included some links below for those interested in learning more.
- “The Generative Music and Procedural Sound Design of Sim Cell” – School of Video Game Audio
- Pure Data – download extended from Puredata.info for all platforms
- Download 10 free patches demonstrating the sound effects in Sim Cell from my site at VideoGameAudio.com – search for “Patches” section for “10 Procedural Sound Design Patches” – Power down, fat synth, spaceship, whoosh, robot blabber, missile, alising scream, alien blast, computery, explosion and bonus synth cat meow sound
- Miller Puckette’s book “The Theory and Technique of Electronic Music” – free 300+ page book using Pure Data
- “Designing Sound” by Andy Farnell has great patches in it from fires to guns to birds – great for game audio
- Download student patches from previous times I’ve taught at Emily Carr University
Leonard J. Paul attained his Honours degree in Computer Science at Simon Fraser University in BC, Canada with an Extended Minor in Music concentrating in Electroacoustics. He began his work in video games on the Sega Genesis and Super Nintendo Entertainment System and has a twenty year history in composing, sound design and coding for games. He has worked on over twenty major game titles totalling over 6.4 million units sold since 1994, including award-winning AAA titles such as EA’s NBA Jam 2010, NHL11, Need for Speed: Hot Pursuit 2, NBA Live ’95 as well as the indie award-winning title Retro City Rampage.
He is the co-founder of the School of Video Game Audio and has taught game audio students from over thirty different countries online since 2012. His new media works has been exhibited in cities including Surrey, Banff, Victoria, São Paulo, Zürich and San Jose. As a documentary film composer, he had the good fortune of scoring the original music for multi-awarding winning documentary The Corporation which remains the highest-grossing Canadian documentary in history to date. He has performed live electronic music in cities such as Osaka, Berlin, San Francisco, Brooklyn and Amsterdam under the name Freaky DNA.
He is an internationally renowned speaker on the topic of video game audio and has been invited to speak in Vancouver, Lyon, Berlin, Bogotá, London, Banff, San Francisco, San Jose, Porto, Angoulême and other locations around the world.
His writings and presentations are available at http://VideoGameAudio.com
Featured image: Concept art for Sim Cell. Used with permission (c) 2014 Amplify.
REWIND! . . .If you liked this post, you may also dig: