Multidisciplinary composer and media alchemist Navid Navab and his team at the Topological Media Lab based at Concordia University (Montreal) presented Aquaphoneia, a sound installation which transmutes voice into water and water into air at Biennale Nemo in Paris in December 2017 (and will run until March 2018). I conducted this interview in the context of the first presentation of Aquaphoneia originally conceptualized for Ars Electronica 2016: RADICAL ATOMS and the alchemists of our time. This version of the piece looked at technology through the lens of the living materiality. As Prof. Hiroshi Ishii, of the MIT Media Lab’s Tangible Media Group, stated, artists “suggest completely new ways of looking at the role of science in our society and the interplay of technology and nature.”
EB [Esther Bourdages]: The theme of the 2016 Ars Electronica Festival, RADICAL ATOMS – and the alchemists of our time, is very close to the Topological Media Lab’s mission: transmutation and alchemy on the philosophical and phenomenological level. For Aquaphoneia, can you expand on alchemy and specifically on how this art piece stands out from your past work? How did alchemical thought process and production techniques come up in the process of the piece?
NN [Navid Navad]: When the 2016 theme for Ars Electronica Festival was announced I was happily surprised and thought: finally, things are coming to light at a much larger scale. Yes, please can we reverse the still prominent European Modernism’s separations—between the conceptual and the material, the precise and the messy, the sciences and the arts—and go back to the holistic richness of alchemical matter? This transition that we are currently experiencing calls for a shift away from representational technologies: from interfaces to stuff, from objects to fields of matter-in-process, from fixed concepts to processes that enact concepts. For over a decade, we as alchemists have been engaging with “bodies and materials that are always suffused with ethical, vital and material power.”
The Topological Media Lab [TML] is occupied by people who are living to fuse and confuse, ready to unlearn the apparent practicality of isolated disciplines, while playfully improvising new pathways to understanding potential futures. The TML hosts an array of projects for thinking-feeling through poetry-infused-matter and breathing life into static forms—which to me is an effortlessly artistic process, and all the while inseparable from a rigorously philosophical or scientific one. Even though it might take decades for the kinds of computational-materials that we are envisioning today to be engineered from ground up at an atomic level, with what is possible today, we explore how the messy stuff of the world could become computationally charged with the potential for play: sounding, dancing, and co-performing new ways of living with or without us.
Aquaphoneia comes out of this rich ecology of experiments. In Aquaphoneia, voice and water become irreversibly fused. The installation listens to the visitors, and transmutes their utterances into aqueous voice, which then is further enriched and purified through alchemical processes.
To fully realize this liquid dream, we went to great lengths in order to fuse the messy behaviour of matter flowing throughout the installation with meticulously correlated and localized sonic behaviour. For example, the temporal texture of boiling liquid in one chamber is perceptually inseparable from the spectral entropy of simmering voices which then evaporate into a cloud of spectral mist. All of this dynamic activity is finely localized: the sounds acoustically emit exactly from where the action occurs, rather than spatially schizophying loudspeakers elsewhere.
On another hand, our material-computational-centric approach lead to a tough yet rewarding meditation on control and process. As a composer, I had to let go of all desires for immediate control over sounds and surrender important rhythmical and compositional decisions to messy material processes. In Alchemical Mercury (2009), Karen Pinkus quotes Marcel Duchamp: “alchemy is a kind of philosophy: a kind of thinking that leads to a way of understanding” (159). For us, in the process of creating Aquaphoneia, essentially what had to be understood and then given up was our attachment to our far-too-human notions of time and tempo. Instead we embraced and worked within the infinitely rich and pluri-textural tempi of matter. Technically and compositionally this meant that most of our focus had to be placed on merging the continuous richness of material processes with our computational processes through an array of techniques: temporal pattern following, audio-mosaicing, continuous tracking of fields of activity using computer vision and acoustic sensing techniques in order to synthesize highly correlated sonic morphologies, careful integration of structure-born-sound, etc. We were able to co-articulate compositions by constraining material processes sculpturally, and then letting the liquid voice and the laws of thermodynamics do their thing.
[EB]: One of the first elements that we notice in the installation is the brass horn connected to an old Edison sound recording machine, that now turns into liquid instead of wax cylinders. In fact, it came from an Edison talking machine. You repurpose an authentic artifact, but you do not fall into the trap of nostalgia, and neither into the role of collector, but you embrace innovation with a dynamic approach which excavates past media technologies in order to understand or surpass contemporary audio technologies. Where does the use of the Edison horn come from and how does it speak to your relationship with the superposition of history?
[NN]: The history of sound reproduction involves transforming audible pressure patterns or sound energy into solid matter and vice versa. The historic Edison recording machines gathered sound energy to etch pressure patterns onto tinfoil wrapped around a cylindrical drum. Sound waves, focussed at the narrow end of the horn, caused a small diaphragm to vibrate, which in turn caused a miniature steel-blade stylus to move and emboss grooves in the cylinder. The tin foil would later on be replaced by wax cylinders, vinyl disks and eventually digital encoding.
Aquaphoneia engages the intimately recursive relationship between sounding technologies and material transmutations. Our digital audio workstations are an in fact an inclusive part of this history, this endless chain of analog transmutation between energy and matter. Under the fiction of the digital there is always the murmur of electrons and of matter-energy fields in physical transmutation. As J. Fargier writes on an early book on Nam June Paik (1989) “The digital is the analog correspondence of the alchemists’ formula for gold” (translation by NN). Well, yes. The digital revolution has allowed us to shape, compute, purify, and sculpt sounds like never before… but then often at the hefty cost of a disembodying process, with interfaces that are linked to sounds only through layers upon layers of representation, far detached from resonating bodies and the sexy flux of sounding matter.
Aquaphoneia playfully juxtaposes material-computational histories of talking machines within an imaginary assemblage: sounds are fully materialized and messed with tangibly within an immediate medium very much like clay or water or perhaps more like a yet to be realized alchemico-sonic-matter. This odd assemblage orchestrates liquid sounds leveraging intuitive worldly notions—such as freezing, melting, dripping, swishing, boiling, splashing, whirling, vaporizing—and in the process borrows alchemical tactics expanding across material sciences, applied phenomenology, metaphysics, expanded materiology, and the arts. Aquaphoneia’s alchemical chambers set these materials, metaphors, and forces into play against one another. After the initial ritual of offering one’s voice to the assemblage, the aqueous voice starts performing for and with itself, and human visitors have the opportunity to watch and participate as they would when encountering the unpredictable order of an enchanted forest river.
It is also noteworthy that the horn resembles a black hole. The edge of the horn acts like an event horizon, separating sounds from their source-context. Sounds, once having passed the acousmatic event horizon, cannot return to the world that they once knew. Voices leaving the body of their human or non-human speaker, fall into the narrow depths of the horn, and are squeezed into spatio-temporal infinity. Disembodied voices, are immediately reborn again with a new liquid body that flows though alchemical chambers for sonic and metaphysical purification.
Much of my work deals with the poetics of schizophonia (separation of sound from their sources). Sound reproduction (technologies), from Edison’s talking machines to our current systems, transcode back and forth between the concrete and acousmatic, situated and abstract, materialized and dematerialized, analogue and digital. Often sounds are encoded into a stiff medium which then may be processed with an interface, eventually decoded, and re-manifested again as sound. Aquaphoneia ends this nervous cycle of separation anxiety and re-attachment by synthesizing a sounding medium capable of contemporary computational powers such as memory, and adaptive spectro-temporal modulation and morphing. To adapt Marshall McLuhan, instead of encoding and decoding a presumed message with representational technologies, it enchants the medium.
[EB]: There is the tendency to think that artwork from Media Labs are stable and high tech. Aquaphoneia uses analog and digital technologies with a Do-It-Yourself (DIY) touch in the aesthetic. Since your lab is multidisciplinary oriented and influenced by diverse fields of knowledge, can you develop on the DIY dimension in Aquaphoneia under the gaze of Clint Enns—cinematographer in the experimental field of cinema—: “Adopting a DIY methodology means choosing freedom over convenience”?
[NN]: Aquaphoneia is a truly eclectic assemblage lost in time. Aquaphoneia’s mixed form reflects its extremely fluid, collaborative and playful creative process. Instead of coming up with a definitive design and executing it industrially, Aquaphoneia’s realization involved a much more playful process, where every little aspect of the installation—materials, sounds, software, electronics, etc.—was playfully investigated and messed with. Every little detail matters and every process, undulating back and forth between conception to execution, is an artistic process. The research-creation process leading to the works that come out of our lab are as critical to us as the final and fully produced art works. This was also true for the alchemists who, through their process, were seeking to develop new approaches for understanding the world, relating to matter, and surpassing nature.
Our research-creation activities concern experimenting with ethico-aesthetics of collective thinking-making: humans, non-humans, machines, and materials enacting and co-articulating the ever-changing material-social networks of relations which shape them. This DIY art-all-the-way approach, while providing a healthy dose of aesthetic freedom, is also an ethical one: we live with and within our designs and grow with them. That being said, we are not attached to a DIY process in the same way that some maker cultures might be. Sometimes we blindly find and repurpose something that does something cool, complicated, and mysterious and that is fantastic, sort of like philosophy of media meets cyber dumpster diving meets DIY hacker space meets cutting edge tech research meets miniMax (minimum engineering with maximum impact) meets speculative whatever…
For example, at some point we decided to gather sonic vapour in a glass dome and condense it back into drops, which were then guided to fall into the bottom of the installation. The purified drop of voice—sonic “lapis philosophorum”—was to fall into the depths of the earth beneath and shine upward like sonic gold, connecting heaven and earth. We had to execute this opus magnum inside a very small hole in the base of the installation. The water drop needed to be immediately sensed and sonified, leading to sounds coming out of the same hole, along with synchronized light. You can imagine that if we were relying on “black-boxed” technologies and ready-made techniques then this task would have seemed like a nightmare to design and fabricate. The water drop was to fall all the way to the bottom of the hole where it would be acoustically sensed by a small apparatus that had to be acoustically isolated from everything else. Then the result of the sonification had to be pushed through the very same hole with a high degree of intelligibility and in a way that it would be seamlessly localized. Meanwhile, light had to shine through this hole in sync with the sounds but the source of light had to remain hidden.
The solution to this technical puzzle came to us effortlessly when playing around with random stuff. We found a hipster product—a little plastic horn—that was made for turning your iPod into a gramophone. Then a speaker was mounted inside of this plastic horn in order to focus sounds towards the end tip of the horn. The back of the speaker was fully covered with foam and duct tape to stop any sound from escaping anywhere except for where we wanted it to appear. A small hole was drilled into the brass pipe in the base of the installation. Our advanced hipster horn-tip-sound-laser-thing was then inserted, allowing crisp sounds to enter the brass hole and emit from it without any visible clues for the perceiver as to where the speaker was hidden. Meanwhile, a similar lighting solution was created so that in a very small footprint we can focus, direct, and bounce enough directional light in the brass pipe without ever getting in the way of the water drops.
We had to engage with this sort of detailed fabrication/composition process throughout the whole installation in order to come up with solutions to sense the behaviour of the materials and liquids locally and to manifest them sonically and visually so that there would be no separation from local material behaviours and their computational enchantment. In trying to do so we discovered that more often than not, there was no ready-made solution or technique to rely on, and at the same time we didn’t have months ahead of us to engage in an abstract design and fabrication process. We had limited hours of collective play time to leverage and to come up with innovative techniques that we didn’t even know could exist and that was really fun.
Aquaphoneia is a rich sound art piece – a manifesto by itself about innovation and inventiveness. The sound installation demonstrates that the main crafters Navid Navad and his partner Michael Montanaro, in collaboration with other members of the Topological Media Lab, swim easily into the multidisciplinary art. They are are not afraid to experiment and engage with the material, which results in an interlacing of forms, a mixture of historic references, and an interesting fusion of “low” and “high” technology. I was able to catch some of the build up of the art piece, and it was fantastic to witness the lab as a playful messy artistic field with a little team of scholars fusing their different backgrounds in convergence on the marriage of art and science.
Aquaphoneia, a sound installation which transmutes voice into water and water into air at Biennale Nemo in Paris runs until March 2018.
- NAVID NAVAB art direction, sound/installation concept and design, audiovisual composition, programming, behaviour design
- MICHAEL MONTANARO art direction, visual/installation concept, design and fabrication
- PETER VAN HAAFTEN electronics, sound, programming
- consulting assistants: Nima Navab (embedded lighting design) Joseph Thibodeau (electronics)
- research collaboration: Topological Media Lab
Featured Image: Aquaphoneia, Paris, Biennale Némo, 17 October 2017 – 18 March 2018, Credit: Navid Navad, 2017
Esther Bourdages works in the visual arts and technology art field as a writer, independent curator and scholar. Her curatorial research explores art forms such as site-specific art, installation and sculpture, often in conjunction with sound. She has authored many articles and critical commentaries on contemporary art. As a musician, she performs under the name of Esther B – she plays turntables, handles vinyl records, and records soundscapes. She works and lives in Montreal.
Navid Navab is a Montreal based media alchemist, multidisciplinary composer, audiovisual sculptor, phono-menologist, and gestureBender. Interested in the poetics of gesture, materiality, and embodiment, his work investigates the transmutation of matter and the enrichment of its inherent performative quali- ties. Navid uses gestures, rhythms and vibration from everyday life as basis for realtime compositions, resulting in augmented acoustical poetry and painterly light that enchants improvisational and pedestrian movements.
Navad currently co-directs the Topological Media Lab, where he leverages phenomenological studies to inform the the creation of computationally-augmented performance environments. His works, which which take on the form of gestural sound compositions, responsive architecture, site specific interven- tions, theatrical interactive installations, kinetic sound sculptures and multimodal comprovisational per- formances, have been presented internationally at diverse venues such as Canadian Center for Architec- ture, Festival du Nouveau Cinema, Ars Electronica Festival Linz, HKW Berlin, WesternFront Vancouver, McCord Museum, Musée d’art Contemporain de Montréal, Contemporary Arts Museum Houston, Inter- national Digital Arts Biennial, Musiikin Aika Finland, and Festival International Montréal/Nouvelles Mu- siques, among others. www.navidnavab.net
REWIND! . . .If you liked this post, you may also dig:
Sounding Our Utopia: An Interview With Mileece— Maile Colbert
Optophones and Musical Print–Mara Mills
**This post was co-authored by Marit J. MacArthur and Lee M. Miller
Like it or not, we are now accustomed to contemporary pop vocalists manipulating their voices using Autotune and other tools or effects for pitch correction. We may exult in it, and congratulate ourselves on our sophisticated appreciation of the options available to the contemporary vocalist. In another mood, we may scream for low-fi and acoustic music, feel cynical about the possibility that we might ever hear an unmediated voice, live or recorded (if we ever did), and/or laugh off the notion of authenticity in performance entirely. Of course, rather than tricking the audience or trying to sound somehow “better” than they are, many performers manipulate their voices to pose questions about the nature of performance—Reggie Watts and Anna Deavere Smith come readily to mind—and to test essentialist assumptions about and perceptions of voice and sound.
Watts, in an exemplary 2012 TED Talk, plays with the different sorts of authority and affect conveyed by, among other voices: upper-class-British-absurd-explanatory, affectively-meaningful-nonsense-foreign-language, and caz-hip-hop-introducing-a-song-chat. Inhabiting and playing with different voices, he amuses listeners into recognizing how much intonation—the rise and fall of pitch—and other acoustic features affect our perception of a speaker’s voice, and how much we expect people to speak in ways that match our assumptions about their identities.
We cannot all be so talented at vocal imitation, however. And in sound, voice and performance studies concerned with speech, machine-assisted manipulation of vocal recordings—which we term “vocal deformance”—is much less common than in the creative industries. A playful approach to vocal deformance, as a critical and creative practice, has much to teach us about our perceptions of speech in general, and performative speech in particular. Too often, when we use archival poetry recordings in our teaching, they may reify an idea that students are often loathe to relinquish: a poem is a finished art object, weighted with authorial intention and biographical significance, with one possible interpretation (the instructor’s). When we play a single canonical recording of T.S. Eliot reading The Waste Land in 1946, for instance, his particular intonation, together with assumptions that he was a stuffy, overeducated, repressed snob, can foreclose the possibility of a fresh encounter with the many voices of the poem and a multitude of interpretations.
Using vocal deformance in the classroom and in our own research and scholarship, we can unsettle overdetermined readings of poems, essentialist assumptions about the poets who speak them and questions of poetic authority, and recover the crucial oral components of poetry. Below we offer some examples of vocal deformance of poetry readings, and consider the potential and limits of this technique for teaching and research with recordings of performative speech. As John Hyland wrote in Sounding Out! in 2014, “The act of listening to recorded poetry … poses particular analytic challenges, which become more complex when the politics of identity are brought to bear on … questions of voice and poetry.” Among these challenges are essentialist assumptions, both about identity and recording medium, which are difficult to avoid when we listen. Hyland concludes that, when we listen to recordings, “the poet’s voice falsely takes on an authoritative ‘aura,’ as Walter Benjamin used that word”; one way to counter to this is to listen to the same poem read by the poet at different points in their career, in different contexts, as Hyland does with three recordings of Amiri Baraka’s “Black Dada Nihilismus.” Another approach is to play with recordings.
The concept of deformance dates to a 1999 essay by Jerome McGann and Lisa Samuels. They take inspiration from Emily Dickinson, who sometimes liked to read poems backward, for the potential insights of reading against the form, scrambling the original sequence, and so on. According to McGann and Samuels, Dickinson’s
critical model is performative, not intellectual [. . . ]. it is anti-theoretical: not because it is opposed to theory (i.e., speculative thought), but because it places theory in a subordinated relation to practice. Deformative moves reinvestigate the terms … [of] critical commentary [, with] dramatic exposure of subjectivity as a live and highly informative option of interpretive commentary, if not indeed one of its essential features. [our italics]
Too often in the literature class room, the subjectivity of interpretation is something of a problem. While we might initially encourage a somewhat fluffy reader-response discussion of a poem, eventually we might also worry that students are simply wandering too far from it, following their own random associations with a phrase or metaphor, without learning to parse the rich intricacy of the whole poem. One effect of vocal deformance is that it makes space for the playful response, and also keeps bringing students back to the telling phrase, to the words of the poem, imagining what difference it makes if they are said in different ways, trying on different interpretations, as it were.
While vocal deformance can be applied to any performative speech, it particularly lends itself to poetry recordings. Poetry is, of course, an oral form with a fraught relationship between text and performance, and poetry reading styles are often perceived to be highly conventional, so that we feel we are listening to a Poem rather than a particular poem. From a literary and performance studies perspective, what could be more tiredly familiar than a canonical recording of a canonical poem by a canonical poet in a conventional style of poetry reading that deadens the audience to the charms and nuances of that poem? And how can we do something productive and interesting with the (sometimes extremely) idiosyncratic subjectivity of student responses to canonical texts?
As an interpretive practice, vocal deformance opens up new possibilities for testing assumptions about performance, poetic authority and gender, and, potentially, about race, class, education, region, and canonicity. Is The Waste Land (1922) the deadly serious poem that many readers often take it to be, partly because it is presented to them as an immensely influential Modernist monolith? How does T.S. Eliot’s seemingly grim reading of it, and our perception of his style, contribute to such an interpretation of the poem? After all, the working title of the poem was “He Do the Police in Different Voices,” from Charles Dickens’s Our Mutual Friend (1864-65), and it includes many different voices or speakers, from the clairvoyant Madame Sosistris to Tiresias. What better way to defamiliarize and exploit the authority of the poem than to deform Eliot’s authoritative reading voice?
How do we respond to the now-canonical voice of Eliot reading the opening lines of The Waste Land, “April is the cruellest month, breeding / lilacs out of the dead land”?
Okay. Now what if we raise his pitch? Is he suddenly his own great-aunt? What does the same lament mean, spoken by a voice that sounds like an elderly woman?
And if we leave his pitch alone, but speed up his speaking rate, does he suddenly sound like an old-school radio announcer, the poem a deranged weather forecast?
In terms of digital humanities research, a refreshing aspect of vocal deformance is that it avoids some of the easy and misleading reassurances of the empirical move. It’s not that it only clarifies what we thought we were hearing (as visualizing intonation through pitch contours can), but that it encourages multiplicity in listening.
Vocal deformance is essentially a playful strategy for defamiliarization that reminds us, in many ways, of the subjective, creative, even arbitrary nature of interpretation. In this, it has clear affinities with the OULIPO movement (which Dickinson’s practice of reading backwards presages). It may help us imagine, create and respond to alternative sequences and versions of recorded canonical texts—and to any apparently stable, singular performance of a text. The art of the glitch is one deformative practice, with the goal countering screen essentialism, the unreflective assumption that a digital artefact is immutable, stable and coherent. For an example of glitching photographs, see Trevor Owens’s “Glitching Files for Understanding: Avoiding Screen Essentialism in Three Easy Steps,” and Michael Kramer’s blog post about using audio deformance in a digital folk music history seminar at Northwestern University, “Distorting History to Make It More Accurate,” which demonstrates some potential insights gained by glitching newspaper images, photographs and music (Bob Dylan’s “Tangled Up in Blue”). John Melillo and Johanna Skibsrud’s “Two Sides for Wallace Stevens,” on Harvard’s Woodberry Poetry Room site, also offers a beguiling example of audio deformance.
Most deformative practices work with text and image, however, and the few that manipulate recordings introduce noise, skipped phrases, repetition, etc., usually without changing the acoustic features of the voice. It is well worth applying deformance more often to speech, not only in linguistics and the neurobiology of speech perception, but in humanistic study of performative speech because our perception of speech is nothing if not subjective, not to say mysterious, for two reasons.
First, our expectations of what we will hear influence what we do hear, from simple sounds to complex language comprehension. Often these expectations, which can be visual, auditory, cultural, etc., have been naturalized by the listener over time as unconscious reactions. Though many have anecdotal experience of this phenomenon (see an example about a black student, a white teacher, and a black student-teacher disagreeing on what the student said in a 2012 Sounding Out! piece by Christina Sharpe), it is has been demonstrated in many experiments as well. For instance, our perception of foreign-accented speech changes rapidly as we hear a few sentences and calibrate our internal expectations, as shown by Clarke and Garret’s 2004 study “Rapid Adaptation to Foreign-accented English.” And, according to Richard Warren’s “Perceptural Restoration of Missing Speech Sounds” (1970), “when natural speech is interrupted by noisy gaps like a cough or a slammed door, we unknowingly “fill in” the noise, vividly hearing speech sounds that do not exist acoustically. This phenomenon arises both from linguistic expectations as well as our deep familiarity with basic speech acoustics, as shown in Shahin, Bishop, and Miller’s “Neural mechanisms for illusory filling-in of degraded speech.” Similarly, in an illusion called the McGurk effect—noted by Harry McGurk and John MacDonald in 1976—just seeing a talker’s lip movements changes the perception of speech sounds categorically, say from “buck” to “duck.”
Though much of this reshaping of our acoustic perception happens unconsciously, we can also profoundly alter what we hear through selective attention. Particularly in everyday acoustic environments, we hear speech better when we expect it, and when it matches our specific expectations: from a given location, from a certain talker or type of talker, at a certain pitch, and so on (See “Speech Recognition in Adverse Conditions: A Review” by Mattys, Davis, et al. 2012). Perceptual filters fundamentally constrain our experience: if we attend to a talker in one ear, we may not even realize when a second talker in the other ear switches from English to German, as Cherry concluded in “Some Experiments on the Recognition of Speech” in 1953. Social and cultural knowledge also changes what we hear. Listening to someone whom a listener visually perceives as a “non-native speaker” can make speech sound not only more “accented” (see Donald Rubin’s “Nonlanguage Factors Affecting S Judgments of Nonnative English-Speaking Teaching Assistants” from 1992)—what we might call a subjective quality—but, as Molly Babel and Jamie Russell found in 2015’s “Expectations and Speech Intelligibility,” it can also trigger speech processing reactions that make the speech less intelligible to the listener making visual judgments regarding accented speech.
Given what we know about the brain, the fact that expectations affect perception—of recorded voices reading poems, in this case—should not come as a surprise. A growing consensus holds that the brain’s job is not merely to represent the world; rather it strives to predict the world, make inferences about it, and correct those expectations whenever a mismatch is detected (see Knill and Pouget’s “The Bayesian brain: the role of uncertainty in neural coding and computation”  and Karl Friston’s 2010 “The free-energy principle: a unified brain theory?”) In somewhat familiar environments and situations (pretty much everything after infancy), predictive inference is far more efficient than continually rendering the perceptual world de novo. This means that vocal deformance—particularly when it manipulates a known voice, as with canonical poets, or a familiar way of speaking, as with conventional poetry reading styles—waves a red flag at the brain. Change wakes up the quiescent, habitual brain to something new and potentially informative, because the voice does not fit our expectations for what the person would or should sound like. Listen to Reggie Watts!
This effect can also operate inversely; that is, if we do not expect someone to have a particular voice, we may adjust the stories we tell ourselves about our perceptions, to better match our expectations. In musicology, we might think of Nina Eidsheim’s article on the racialized reception of opera singer Marian Anderson, the first African American to sing at New York’s Metropolitan Opera:
the timbre of her voice has routinely (if often admiringly) been characterized as ‘black,’ … [despite] classical music’s minimal indulgence of individual style … this distinction [has] to be based on an assumption that the black body is intrinsically different from the white body and that even when emitting a timbre recognized as classical, the resonance of a singer’s black body is evident (3, 4).
Certainly, as Jennifer Stoever writes, “listening [is] an interpretive site where racial difference is coded, produced, and policed” (62). The same is true of gender difference and many other identity markers and cultural factors related to authority and authenticity. As Shai Burstyn notes in the article “In Quest of the Period Ear,” about attempts to imagine how contemporary audiences experienced medieval music, “culture plays a highly significant—though not exclusive—role in shaping the cognitive skills of its members” (695). If it is remarkably difficult to escape our stereotypical expectations and perceptions of what a person’s voice “should” sound like, that is partly because our brain uses such expectations to make predictions about our sonic experience. We cannot overcome our expectations through good will alone, and engaging with these issues in the classroom, which can be challenging, also provides an opportunity to help students think critically about essentialism and voice, for those moments when a student in the back of the room mutters in surprise that Langston Hughes “doesn’t sound black,” or exclaims that Walt Whitman “doesn’t sound gay.” Though it is not designed to assess stereotyping in speech perception, the Harvard implicit bias test is a good way to engage students in questions of cultural bias and perception [also, see “So You Flunked a Racism Test. Now What?”].
Furthermore, our affective responses to acoustic, non-verbal qualities of speech matter tremendously to our interpretation of verbal semantics, of the meaning of the words spoken. According to voice perception research in Foundations of Voice Studies: An Interdisciplinary Approach to Voice Production and Perception by Jody Kreiman and Diana Sidtis, when we listen to speech, “[s]ome authors … have claimed that normal adults usually believe the tone of voice rather than the words…. For example, the contrast in ‘I feel just fine’ spoken in a tense, tentative tone might be politely ignored, while, ‘I’m not angry’ spoken in hot anger would not” (304). The teacher’s boring tone of voice on the Peanuts cartoon makes the point.
In other words, we pick up on the affective meaning of a speaker’s tone of voice, and weigh it against the semantic meaning of the words spoken. While Kreiman and Sidtis argue that tone cannot be reduced to intonation patterns, “the fundamental frequency of the human voice [pitch] … heads the list of important cues for emotional meanings” (311). Pitch manipulation, then, changes the affective meaning of speech. Tone of voice is also influenced by other acoustic features, including speaking rate or tempo, and rhythm. In poetry recordings, the poet’s tone of voice influences the listener’s interpretation of a poem.
Two fundamental intonation patterns are rising or falling pitch. In American English, relatively high or relatively low pitch at the end of an utterance, compared to the beginning and middle, seems to carry distinct meanings, as demonstrated by Janet Pierrehumbert and Julia Hirschberg. They developed the ToBI (Tones and Break Indices) system for marking the prosody or intonation of speech. Rising intonation can make any utterance sound like a question, whether it is one or not. A relatively high pitch at the end of an utterance—called a high boundary tone—can make the speaker sound less confident or assertive, and more open to other’s opinions. Rising intonation implies that more is to come, that the utterance is not conclusive or concluded, that it should be understood in connection to the next utterance, and sometimes, that the speaker seeks the listener’s agreement before proceeding.
Uptalk, notably practiced among Generation Xers and now millennials, sounds conciliatory, agreeable and open, on the one hand, and lacking in confidence and authority, on the other—depending on the listener and the context. Marybeth Seitz-Brown argues that criticizing uptalk “implies that if women just spoke like men, our ideas would be valuable … [and] sexist listeners would magically understand us, and we would be taken seriously. But the problem is not with feminized qualities, of speech or otherwise, the problem is that our culture pathologizes feminine traits as something to be ashamed of or apologize for.”
Conversely, women can be criticized when they sound too much like men; see “Why Do So Many People Hate the Sound of Hillary Clinton’s Voice?” Falling intonation—and ending an utterance on a relatively low pitch, or a low boundary tone, implies conclusion, closure and confidence. The utterance, such intonation implies, finishes the argument (if there is one), does not seek the listener’s agreement or opinion, and suggests that this utterance can be understood on its own, without connection to subsequent utterances. Donald Trump, for example, is fond of falling intonation and low boundary tones (for a parody of masculine confident declarative intonation, have a listen at Troy McClure from The Simpsons.)
Of course, not all women use uptalk, and not all men use falling intonation with low boundary tones. In American culture, for better or worse, low boundary tones do seem to carry a tone of authority. And in poetry reading as well. Eliot’s original, and now canonical, reading of the opening line of The Waste Land, “April is the cruellest month,” uses falling intonation, so that it sounds like a confident assertion, with a low boundary tone on “month.”
“[B]reeding / lilacs out of the dead land” sounds like a steady, inevitable process, ending on a slightly higher relative pitch, implying that there is more to come, and that the phrase should be understood in connection to the next line, “mixing / Memory and desire, stirring / Dull roots with spring rain.”
What is so compelling and seemingly authoritative about Eliot’s reading style? In some basic sense, the falling intonation of the first phrase does it. Why does it strike many contemporary listeners as pompous? How might we undercut the seeming authority of the Eliotic voice? Make him do uptalk. Here we have simply inverted his intonation.
Suddenly he sounds doubtful. The opening line becomes a question—“April is the cruellest month[?]”—instead of a confident statement. Suddenly, Eliot himself expresses the skepticism or confusion many an undergraduate has felt—before we encountered this poem, did we not assume that spring, the return of life and fertility, is a cheerful escape from winter? And his deformed recital of “breeding / Lilacs out the dead land” suddenly sounds more like an agonized complaint, expressing the painful, reluctant awakening of desire in one who had found the dull sleep of winter comforting. Inverting the typical poetic authority of falling intonation into uptalk may embolden readers to entertain very different readings of the poem’s opening.
The editors of Poetry Archive had hopes of stimulating listeners of The Waste Land when they made available a 1935 recording of the poem, claiming: “Whilst the sound quality is understandably not so good, the recording is fascinating for Eliot’s faster, more energetic rendition. Listening to this urgent interpretation blows the dust of this iconic poem and helps us encounter it afresh.” However, if the fundamental falling intonation pattern of Eliot’s reading style doesn’t change—and overall, it doesn’t, between the 1935 and 1946 recordings—his voice may remain, for listeners, an aloof poetic authority.
Falling intonation with low boundary tones, then, is a fundental tone of poetic authority. Listen to Adrienne Rich reading from her poem, “What Kind of Times Are These” (1995), which leads the reader to a place “between two stands of trees … near … [where] the persecuted / … disappeared into the shadows.” She insists, “this isn’t a Russian poem, this is not somewhere else but here,” and concludes, “to have you listen at all it’s necessary / to talk about trees.”
She sounds like she means it. Rich has to write poems about nature, her tone implies, to wake people up to the political horrors of the American past and present. Poetry as a form, in pastoral guise, allows her to sneak in political content, potentially grabbing the attention of people who might only listen to poetry if they think it is safely, simply about nature. (Click here to hear the entire poem, starting at 4:01.)
When we invert her intonation, turning it into uptalk, she sounds as if she is questioning the wisdom of this approach, and/or chiding her listeners for making her take it. In this case, uptalk exerts a different kind of authority, the challenging question.
Is it ethical to manipulate the intonation and other vocal qualities on poetry recordings, for the purposes of teaching and research? Obviously it would not be, if we were to present the manipulated recordings as the authentic voice of a poet. And all peoples have the right to protect culturally sensitive recordings, such as sacred songs, music, dances and prayers; see “Native American Intellectual Property Issues.” Otherwise, potential conflicts are similar to those with sampling in the music industry (See Kembrew McLeod and Peter DiCola’s Creative License: The Law and Culture of Digital Sampling ). Vocal deformance, however, can help remind us that no single reading of a poem, by the poet or someone else, is the ultimately authoritative one.
In teaching writing, we (the authors) sometimes ask our students to explore alternative methods of presenting the same material. This can be as simple as writing the same sentence, the thesis for instance, in three different ways, or it can involve a different format. Write a poem, record oneself reading it, then try to represent it with a collage of images. Turn a 2,000-word essay into a 250-word presentation with verbal and sonic components. An instructive trick with the opening line of W.H. Auden’s “Musée des Beaux Arts” (1940): “About suffering they were never wrong, the Old Masters.” They were never wrong, the Old Masters, about suffering. The Old Masters were never wrong about suffering. Each version of the opening creates a subtly different emphasis, on suffering versus the wisdom of the Old Masters.
Too often, we lock ourselves into one approach, and cannot imagine an alternative. Locked into one approach, too often we cannot imagine an alternative. Alternatives we cannot imagine, if we lock into one approach too quickly. Writing three different opening paragraphs to the same essay, or rearranging the lines of the poem, stimulates our imagination and our critical faculties because it dramatizes different possibilities, possibilities that offer a different emphasis. And when we play with the pitch, intonation and speaking rate of a poem, this can change the tone as dramatically, from a challenge to confession, or an assertion into doubt.
In the classroom, poet Harryette Mullen is often popular with students, both for her poems on the page and for her expressive reading style, while students can sometimes resist recordings by Adrienne Rich (saying that she sounds lecture-y) and Louise Glück (saying that she sounds bored by her own poems), even as they are engaged by the poems on the page.
When Mullen reads “Present Tense” (2002)—a beguiling comical poem, loosely about the grammatical present and the speaker’s and the world’s present circumstances —what is it about her contrastive intonation that sounds expressive? She ends her opening phrases, “Now that my ears are connected to a random answer machine” with rising intonation and high boundary tones. This draws the reader on: keep listening, the statement’s not finished.
When we flip the intonation pattern, so that each utterance ends on a relatively low pitch, she sounds more conventional, a poetic authority declaring observations, confident and closed off.
Another tone of poetic authority approaches pure monotony. It was practiced by Alfred Lord Tennyson, Irish modernist poet W.B. Yeats and, perhaps through Yeats’s influence, by American poets such as Yvor Winters. Note how similar they sound here. Winters reads the opening of his poem “The Journey” (1931), moving into a Yeatsian monotone after the title and location of Snake River Country, “I now remembered slowly how I came[.]”
Here is Yeats reading the opening of “The Lake Isle of Innisfree”: “I will arise and go now, and go to Innisfree[.]”
All we have to do to turn Winters into Yeats is raise his pitch a bit:
Monotone performance is—at least acoustically—quite uninformative for the brain. Early parts of the auditory brain rapidly adapt or habituate to a wide array of regularities such as pitch and temporal pattern, and they only signal when the pattern changes, as noted in “Early selective-attention effect on evoked potential reinterpreted” (Näätänen, Gaillard et al., 1978). But expectations can work differently across speech’s descriptive dimensions. When speech is usually vivid, as in a direct quotation (“He said ‘I’m leaving now”), higher-level voice-processing areas in the right temporal lobe actually work harder to process (unexpectedly) monotone quotes, according to Yao, Belin, et al.’s “Brain ‘talks over’ boring quotes: top-down activation of voice-selective areas while listening to monotonous direct speech quotations” (2012). In other words, sameness of pitch often means the brain must work harder to grasp meaning.
Interestingly, David Hadbawnik relates in Sounding Out! his disappointment with the productions of three audio recordings of three poetic specimens from Middle English created with SPARSAR, because “they produced monotone outputs that fail to account for prosody.” Vocal deformance might allow him to try to approximate Middle English prosody with the specimens.
MacArthur has written elsewhere about “poet voice,” which she also calls “monotonous incantation.” But how close are contemporary canonical poets to actual monotone, compared to Tennyson and Yeats? Here is Glück, whose reading style is often mentioned as an example of Poet Voice, reading the third stanza of “The Wild Iris” (1992): “It is terrible to survive / as consciousness / buried in the dark earth”:
Not much manipulation is required to make it purely monotone, which may account for some students saying she sounds bored by her own poems—though they do not say that about Yeats. They say Yeats’s voice makes them feel like they are in church.
Ideally, Glück’s manner of reading her poem should not prevent students from appreciating it. While in other contexts we may defend women’s use of uptalk, it also seems fair to raise the point that academic poetry reading can seem to discourage the expression of affect. (See Donald Hall’s well-known polemic, “The Poetry Reading: Public Performance / Private Art” (1985) and David Groff’s “The Peril of the Poetry Reading: Page Versus Performance” .) Vocal deformance, among other strategies, might help students perceive as much drama in Glück’s poems as they do in Mullen’s—and find as much as poetic authority in both poets’ voices as they do in Yeats’s churchy one. Here, we’ve manipulated Glück’s voice to sound more like Mullen’s style of reading, with a wider pitch range and rising intonation and high boundary tones.
If we want to explore alternatives to conventional modes of reading poetry, as many do, directly deforming the acoustic qualities of canonical recordings is an excellent way to defamiliarize performance conventions. Ideally, it can help us listen to alternate versions of the history of poetic performance and to different, unimagined possibilities in the present. Given the extraordinary vitality of spoken word and slam poetry outside the academy, it would be a missed opportunity to suppress varied reading styles in the classroom. At the same time, it would be a great shame to leave behind canonical American poetry when the poets’ reading styles fail to appeal to students.
Finally, if we want to liberate students from the anti-performative tendencies of academic culture, resist essentialist readings of poems according to our assumptions about the identities of the poets who wrote them, and dramatize the idea that there are many ways to read a poem, vocal deformance can help, alongside other strategies. As Yvon Bonenfant wrote in a 2014 Sounding Out! piece, “we are mostly neurotic, or otherwise hung up on, what kinds of sounds we make, where and when.”
Instead, let’s play in different voices.
NOTE: To illustrate vocal deformance, we used Straight, a state-of-the-art open-source voice synthesis program developed by Hideki Kawahara at Wakayama University in Japan, with the Advanced Telecommunications Research Institute and the Auditory Brain Project. We also used Drift, an open-source pitch-tracking tool that uses an algorithm developed by Byung Suk Lee and Dan Ellis, implemented by Robert Ochshorn and Max Hawkins with support from MacArthur’s ACLS Digital Innovations Fellowship in 2015-16, to visualize intonation with pitch contours.
Marit J. MacArthur is associate professor of English at California State University, Bakersfield, and a research associate in Cinema and Digital Media at the University of California, Davis.
Lee M. Miller is associate professor of Neurobiology, Physiology, & Behavior at the University of California, Davis, and technical director of the Center for Mind and Brain.
Featured Image: Cropped and Enlarged version of Bill Smith’s “Voice Glitch,” Attribution 2.0 Generic (CC BY 2.0)
A Listening Mind: Sound Learning in a Literature Classroom–Nicole Furlonge
Audio Culture Studies: Scaffolding a Sequence of Assignments–Jentery Sayers
SO! Amplifies. . .a highly-curated, rolling mini-post series by which we editors hip you to cultural makers and organizations doing work we really really dig. You’re welcome!
The first annual Sounding Board sound exhibit was held at The Companion Gallery in Austin, Texas on December 3 – 6, 2015, as part of the 60th anniversary meeting of the Society of Ethnomusicology (SEM). In the promotional literature for the show, the curator, Leonardo Cardoso (Texas A&M), described its objective: to give students, ethnographers, ethnomusicologists, and any “sound-minded” people an opportunity to share research and contemplate fieldwork from different perspectives. Cardoso hoped that SEM Sounding Board would “stimulate dialogue between ethnomusicology and other fields, especially sound studies, sound art, ecomusicology, anthropology, and media studies.” He also sought to facilitate interaction between the local community in Austin and SEM scholars who traveled to attend the conference.
I spoke with Cardoso about this exhibit on several occasions. When I asked him why name the exhibit “Sounding Board”?, he told me that Veit Erlmann (University of Texas Austin), described once described his role as a mentor as someone to bounce ideas off of, like a sounding board. In a similar way, Cardoso’s vision for the first annual SEM sound art exhibit was to create opportunities for scholars and local people to meet and discuss sound, ethnography, art, and fieldwork in an open context, and learn from each other while interacting in that space. He designed Sounding Board as a place where “ideas are amplified” and scholars and community members can make fruitful connections because they have an opportunity to reflect and discuss research with people from different backgrounds.
In his invitation to SEM attendees, Cardoso described Sounding Board as
[eight] sound works that probe into sonic in-placements (water and wind), sonic displacements (the telephone, the radio, and the microphone), sonic emplacements (the acoustic territories of urban Taiwan, the Brazilian hinterlands, and West Texas), and sonic mix-placements (in Mexico City).
This collective sound exhibit showcases the creative work of scholars attentive to the spatial, acoustemological, and ethnographic potential of sound. SEM SOUNDING BOARD challenges distinctions between sound-as-episteme and sound-as-performance, sound-as-ethnography and sound-as-art.
Interactive, Immersive, Ethnographic Sound Art
The playfully engaging work, Pool of Sound, welcomed me to the interactive SEM Sounding Board exhibit. As soon as I walked into The Companion Gallery, I noticed the eye catching 1st Annual SEM Sounding Board poster near a studio monitor on a stand, facing another monitor, placed directly across from it, about 20 feet away. A large illuminated circular area gleamed in between the silent speakers. When I moved into the light, I suddenly heard the clear sounds of gently rushing water, but only for an instant, then there was silence again, as soon as I stood still. As I turned and stepped towards one of the speakers I heard the rushing water return. The gurgling sound mirrored my movement and when I stopped, the sound of the water stopped.
Lina Dib (Rice University) created the piece, with an
enchanted zone [that] literally becomes a pool of sound where sound becomes substance, something to be physically and playfully encountered. In other words, sound with this installation becomes palpable, sound is made (in)to matter. The larger the visitors’ gestures, the louder and stronger the sound of water becomes.
Dib cites Jean-Luc Nancy in her work’s description, understanding her piece as an embodiment of Nancy’s observation in Listening that sound envelops the listener: “Sound has no hidden face; it is all in front, in back, and outside inside, inside-out.”
While experimenting with the intersections of sound and gesture in Dib’s Pool of Sound, I noticed someone sit down at an antique-looking wooden desk across the gallery, pick up an old school, land line telephone, dial a number, and start writing on a notecard. The person at the desk was experiencing Schizophone, Calling Son Jarocho, a installation by Craig Campbell (University of Texas Austin) and collaborators, Julian Etienne, Juan-Pablo Gonzalez, and Cameron Quevedo. When the person hung up and left, I sat down, braced the phone between my ear and shoulder and listened to a dial tone.
I dialed a few numbers and started to hear a conversation through the receiver: musicians were speaking in Spanish, discussing certain subtleties of a Son Jarocho performance. I felt like I was eavesdropping. I dialed another number and the sounds of Son Jarocho music flooded my ear. This installation provides numerous sound bytes of field recordings related to Son Jarocho music of Mexico. Each recording is described on a notecard that gives ethnographic descriptions of the situation. Campbell also asks the listener to participate in the piece by filling out a card to leave a record of their experience. The artist says that his “work builds on R. Murray Schafer’s ‘schizophonia’ to signal the profound but also banal experience of listening to recorded sound. The schizophone recruits the telephone–a mundane, though now largely residual technology–to frame and structure an encounter with archival recordings.”
A few feet away from the Schizophone desk, a poster stand held a flyer for the piece Wind Noise by Marina Peterson (Ohio University). A pair of headphones clung to the stand.
When I put the headphones on I expected to hear some cinematic blowing, or the soft sound of a summer breeze. Instead, I heard a familiar, dreaded, thumping noise. Peterson’s work indulges in a recording taboo: the clipping, dull thud of wind hitting an unprotected microphone.
As I listened, I thought about noise and how to define it. Usually, this thudding sound would bother me and I would cut out chunks of recordings to get rid of it. But in the context of a sound art exhibit, I found myself examining this noise, and listening to it as art. This reinterpretation of sound in relation to space reminded me of David Novak’s discussion of “Noise” as a genre in the context of Japanese music coffeehouses in his article, “2.5 meters of space: Japanese music coffeehouses and experimental practices of listening.” Peterson discusses her work as an exploration of technology, mediation, and the microphone. She describes these recordings as
an effort to reveal the microphone as technology by disrupting it. Wind noise is sound as touch – this is the sound produced by touching the microphone, whether by finger, breath, or air. These recordings do not capture the sound of wind, but the sound wind makes on the microphone. The sound the microphone makes when touched by wind.
In a recessed corner of the gallery I saw a music stand with a piece of paper on it. I didn’t know if it was part of the Sounding Board installation, or just a piece of equipment, set aside. As I stepped up to the stand to read the paper, I unexpectedly stepped into a chamber of sound. A Holosonics AudioSpotlight AS-24i directional speaker, mounted on the ceiling, beamed a column of music into that area, which a listener can hear only when directly below the speaker.
The piece is called Resting Place, by Michael Austin (Howard University). In the description of this work Austin states:
Resting Place is based on the old cowboy song ‘Bury Me Not on the Lone Prairie.’ Not only does this work confront listeners with thoughts of mortality and final resting places, it embodies the wide open spaces of my childhood home and serves as a place of peace and relief for the here and now.
Austin grew up in the countryside of the Texas Panhandle, and his work intends to bring a piece of that Texan soundscape to a corner of the gallery. I could hear the sounds of birds, wind, and water combined with chant and meditative, drone music; they were all sounds that would usually communicate rest and peace. Unfortunately, I had a difficult time entering that relaxed frame of mind because the recording of the Texan soundscape often clipped, which disrupted my concentration on the calming aspects of the field recording. Composers such as Annea Lockwood and Janet Cardiff use binaural microphones to capture nature sounds up close and create an intimate surround sound experience for the listener; although I am fascinated by the concept of creating a “soundscape chamber” by using a hyper directional speaker, I would love to hear the details of Austin’s field recordings through a nice pair of headphones.
Resting Place and Wind Noise invite contemplation as the listener receives sound. In contrast, the broadcasting sound piece by Tom Miller (Berkeley College) is intensely interactive. In Radio Texas International, a Micro Radio Station in the Austin Wavescape, Miller creates an experience where it is possible to broadcast sound and listen to recordings.
Resting Place and Wind Noise invite contemplation as the listener passively receives sound. In contrast, the broadcasting sound piece by Tom Miller (Berkeley College) is intensely interactive. In Radio Texas International, a Micro Radio Station in the Austin Wavescape, Miller creates an experience where it is possible to broadcast sound and listen to recordings. Miller explains that for this piece he
operate[s] a low power Mini FM Micro Radio station in the gallery… Tuning to open frequencies, a legal micro power transmitter broadcast[s] to receivers distributed within a 200-foot radius as a hyperlocal, pop-up intervention into the FM band. Using headsets, listeners will tune the radio dials seeking to locate the signal interspersed with the music, religious broadcasts, news, foreign language programming and static of the local radio wavescape.
In the video of his work you can hear several different ethnographic recordings that are broadcasted by Miller in the gallery, and at the same time intertwine with the sounds of local radio stations in Austin. Besides broadcasting field recordings, Miller also aired live interviews and music throughout the three-day exhibit. I was delighted to have the chance to play some traditional Irish music on the air for Radio Texas International.
雜 (dza) is a piece by Yun Emily Wang (University of Toronto) and Wendy Hsu (Dept. of Cultural Affairs, City of Los Angeles), who created a work that inhabits a cardboard box. The artist Zimoun often uses percussive elements to explore acoustics and cardboard, but in 雜 (dza), Wang and Hsu employ the box as a resonator to amplify and combine sounds emitted from headphones playing loops. The listener is asked to put their head in the box to hear a cacophony of intermingled field recordings that create a decontextualized soundscape of Taiwan.
The artists explain that “These composed loops recontextualize the sonic materiality of the informal economy and quotidian life exemplified at a Taiwanese night market, and interact with the spatial and sonic elements of the venue and its role within the emerging art-as-enterprise share economy.
There were two pieces of interactive, ethnographic sound art that integrated both audio and visual elements of fieldwork in Mexico City, and Brazil. Dry Signals by Michael Silvers (University of Illinois Urbana – Champaign) invites the auditor to “touch the screen” and listen to field recordings. The touchscreen of the laptop displays an image of a painting of a small town surrounded by mountains, near water.
I put the headphones on and touched a part of the image of the town that caught my attention: a traditional forró trio standing on the porch of small pink house (no.8). In the headphones I immediately heard the rhythmic music produced by the musicians playing the drum, the accordion, and a triangle.
Silvers describes the inspiration for Dry Signals as an exploration of
the sounds of drought in northeastern Brazil. From trickling reservoir spillways… to the music and shuffling feet of dance parties in dusty fields, these sounds tell stories of labor, birds, politics, agriculture, plants, mass media, corruption, water, and the quotidian experience of life in the semi-arid Brazilian hinterlands.
The artists take advantage of touchscreen technology to give the viewer a chance to curate their own soundtrack of their experience of the painting. There is no lag in the experience of touching, listening and viewing the village and surrounding landscape. Even though the field recordings are not uniform in sound quality, I enjoyed the experience of hearing an ethnographic audio record of a small town in northeastern Brazil, by touching an image of it.
Anthony Rasmussen (UC Riverside) provides an opportunity to peek in on urban street scenes filmed throughout Mexico City in his work, El Caracol: A Stroll through Space and Time in Mexico City.
Some of the most compelling scenes in the 20 minute loop of video and audio depict street protests in Mexico City which are accompanied by ambient sounds from the field recording, combined with subtle music, and seemingly unconnected background conversation.
The artist explains that
the video element consists of footage captured while walking through various sites in Mexico City and represents the phenomenological ‘present’. The audio element provides a counterpoint to the visual; as the loop begins the audio corresponds to the action on screen, but with increasing frequency (based on the ‘Fibonacci Spiral’) the contemporary sounds will be ‘ruptured’ by historical recordings of Mexico City that drift further back in time.
I particularly enjoyed the sections where the connection between the audio and the video was unclear. Toby Butler’s article “A walk of art: the potential of the sound walk,” traces the efforts of different artists and their uses of the sound walk in their work, but he does not describe any endeavors like Rasmussen’s, where ethnographic footage is the prime source of the walk. I wondered about the position of peering through the hole to watch Rasmussen’s field recording of Mexico City, and I realized that at times, gazing through the hole gave me the sense that I was the ethnographer gathering footage.
“always more sound to experience”
I visited the Sounding Board exhibit several times while attending the SEM conference. Every time I left I felt like there was always more sound to experience. I wanted to hear all of the numerous field recording of Son Jarocho material presented by Campbell’s Schizophone; Miller’s Radio Texas International changed every time I listened and I wondered what ethnographic material I might encounter the next time I tuned in. I never tired of Lina Dib’s Pool of Sound because it gave me the chance to perform the gurgling of water, using gesture. Apart from the evocative expressions of ethnology as art, Sounding Board converted The Companion Gallery into an interactive playground of sound.
The live performances in the gallery on Friday night brought the ethnographic sound art to life. When I listened to at least twenty members of the Comunidad Fandango of Austin perform and dance Son Jarocho music in the gallery on Friday evening, I began to make connections to the field recordings that I heard in Schizophone. When Bruno Vinezof and Forró de Quintal took the stage to play forró music from northeastern Brazil, I could feel the groove of the drum that was merely suggested in the field recording that I had listened to in Dry Signals. It was a unique pleasure to observe and participate in these musical traditions with my body, after having encountered them earlier through headphones as sound art.
When I spoke with Cardoso he was especially grateful to the Son Jarocho community of Austin, who volunteered to participate in the show by gathering in The Companion Gallery for a Fandango. He emphasized the grassroots aspect of this community music making event which came about because Cardoso knows the group and their passion for Son Jarocho music.
Cardoso plans to expand the variety of works and disciplines involved in next year’s Sounding Board to include media studies, literature, film, and the visual arts. As SEM 2016 will be meeting in Washington DC and co-hosted by Smithsonian Folkways Recordings (and George Washington University), this should not only be possible, but especially exciting.
Featured image: Lina Dib’s “Pool of Sound” by Matt Morris
Jay Loomis is a composer, a performer, and a graduate student in ethnomusicology at Stony Brook University with a particular interest in transnationalism, soundscapes, improvisation, wind instruments, and electronic music. He hosts a radio show called “Face the Music,” and recently curated a sound installation called “SOUNDREAMS” at Stony Brook University, which used geo-located sounds and music strategically placed around the university campus which people heard by using a smart phone app called Recho. Jay hand crafts Native American and other kinds of flutes, and leads flute making workshops in local libraries and schools. He plays a variety of wind instruments from around the world. He recently led workshops in a contemporary music festival in Cuenca, Ecuador (FIMAC: Festival Internacional de Musica Academica Contemporanea). Participants in Jay’s workshops arranged music and created flutes as a practical way to examine how indigenous music making practices and pre colonial instruments can contribute to the world of contemporary academic music.
REWIND!…If you liked this post, you may also dig:
Sound and Curation; or, Cruisin’ through the galleries, posing as an audiophiliac–reina alejandra prado
SO! Amplifies: Shizu Saldamando’s OUROBOROS–J.L. Stoever
SO! Amplifies: Mendi+Keith Obadike and Sounding Race in America–Mendi + Keith Obadike
Editors’ Note: Welcome to Sounding Out!‘s December forum entitled “Sound, Improvisation and New Media Art.” This series explores the nature of improvisation and its relationship to appropriative play cultures within new media art and contemporary sound practice. This series will engage directly with practitioners, who either deploy or facilitate play and improvisation through their work in sonic new media cultures.
The first essay in this series draws from a constellation of disciplinary perspectives that investigate these critical valences, and posits both play and improvisation as critical interventions which can expose, critique and interrupt the proprietary techniques and strategies of contemporary consumer media technologies.
— Guest Editors Skot Deeming and Martin Zeilinger
As media art scholars and curators of the annual Vector Game Art & New Media Festival, we are particularly interested in the conceptual convergence between critical approaches to play, on the one hand, and to improvisation, on the other. In this short essay, we therefore ask how improvisational sound-based media practices that use technologies, aesthetics, artifacts, and expressive modes of play may challenge the assimilative advances of corporate capital, carving out sites at which its logic may be opposed and confounded. We contend that there is a critical edge to the cultural milieu of playful computation through which practitioners (whether as artists or simply as players) can recuperate play from the mainstreamed sound cultures of digital capital through improvisation-based approaches. It is in this regard that play, as a participative act, rather than an interactive mode, can become a critical site by which to understand artistic interventions through contemporary technologies. This essay provokes a dialogue between theoretical avenues in popular music studies and game studies in order to show how the often-conflated practices of improvisation and play have the potential to challenge the homogenous and repetitive logic of the technological sector and the music industry.
We must turn first to popular music studies where the critical valence of improvisation is a relatively well-established concept. Jazz improvisation in particular has been widely recognized as a powerful mode of political expression. Charles Hersch has argued that techniques of polyphony allow for jazz to reconcile the seemingly contradictory dynamics of individual expression and collectivity, thus allowing for the articulation of black solidarity through musical interventions (97-8). Because jazz is importantly built on appropriation-based creative models, it yields fluid, fleeting, spontaneous results that are difficult to assimilate into the entertainment industry’s property-based circuits. Digital play exhibits a similar potential of challenging the logic of capital; it harbors a powerful creative potential even in heavily commodified contexts. Play allows us (indeed, requires us) to become active in shaping the improvisational, the field of simultaneously critical, creative and performative contexts of expression.
Like jazz musicians, game players embody and enact a powerful sense of potentially uncommodifiable agency. Both improvisation and play function similarly, and highlight how contexts of music-making and game-playing activate individuals as critical, expressive agents. Players guide themselves through games based on subjective, rather than imposed, motivations. We can see this emerging in gaming cultures through the rise of player-based phenomena such ‘speedrunning,’ a practice that exploits glitches and faults in a game’s architecture in order to reach the end of a game in the shortest possible time. These niche and ephemeral gaming practices highlight the degree to which practices of improvisation in play may constitute a powerful way of pushing ‘through’ capital–both literally, by accessing games through glitches in their proprietary fabric, and figuratively, if we understand such forms of play as ways to resist the capitalist ideologies embedded within the structures of games (such as the accumulation of wealth, power, or points). Having established here how game-play can act as a form of critical improvisation, this essay now intends to show how these practices of playful improvisation are manifest in the technologies, aesthetics, and the cultural contexts of sound-based art.
Where can we locate practices of improvisation in the context of sound art? For us, sound art is improvisational both in form and practice. First, the emerging aesthetic dimensions of the medium are fundamentally related to the participatory practices of making that have been used to develop the instruments of sound art, such as circuit bending, hacking, tinkering. Here, the playful and open-ended practices of making are a form of improvisation, a modulation within the typical technological frames in which musical composition is typically embedded. Second, when gaming devices are turned into musical instruments (or vice versa), users are called upon to rethink improvisational practice as game-play, and experiment with deploying play as composition. In this sense, game players become sound artists, with sound technologies extrapolating and improvising new sounds from the habit of their play. Improvisation and play thus allow emerging communities of practice to counteract the corporate ‘black boxing’ of entertainment technologies and the creation of closed systems of interaction and participation (such as many mainstream video games) which assimilate user activities and foreclose improvisational play.
Such approaches to sound art are represented by many creative practitioners and take a wide variety of forms, including the appropriation and creative redeployment of participative game environments and game engines. Examples include the creation of interactive sound environments and playable instruments within mainstream participative game ecologies such as the Little Big Planet franchise, the hacking and ‘bending’ of proprietary technologies that invoke the vast universe of game culture nostalgically or in a parodic mode (including handheld gaming systems such as the Nintendo Gameboy, which spawned an entire electronic music genre known as chipmusic), or the making of non-proprietary infrastructures for creative expression that encourage open, participative play, such as open source live coding environments including Sonic Pi, originally developed for the Raspberry Pi platform. Following such approaches, game controllers, digital objects, avatars, virtual architectures, and digital artifacts can thus become tools of composition and/or improvisation. Similarly, we might also think of the improvisational frameworks constructed by sound practitioners as ‘playgrounds,’ which allow players to become performers who explore the technological, political, and aesthetic limits of play. But emergent communities of creative practitioners frequently adopt such close systems and technologies subversively, for example by reverse-engineering them and developing alternative uses. In doing so, they alter the contexts in which these technologies are employed, and open them up for use as musical instruments and infrastructure for creative and critical expression. Technological artifacts with unique user communities can thus be conceptualized as frameworks for musical expression. Vintage gaming consoles, games engines, and computational algorithms have all been reconstructed as new forms of musical creation and interaction through participatory creative communities.
In this short essay, we have approached ‘play’ and ‘improvisation’ as related concepts that can serve to recuperate and empower critical perspectives within the game and sound cultures of capital. As we have suggested, playful approaches to improvisational sound-work and perspectives on play that are rooted in musical traditions can be productively discussed both in sound studies contexts and within game studies paradigms. How, then, do sound practitioners appropriate game technology? How do the socio-political dimensions of digital game culture inform musical practice? With the help of our contributors to this series, we explore how critical perspectives on musical improvisation aid our understanding of the cultural and socio-political significance of play and, likewise, how critical theories of play may broaden our understanding of improvisational sound practices. Through building, tinkering, problem solving, and improvising on the technological back end, creative practitioners playfully create and reshape tools, devices, and techniques of improvisational practice that often embody important ethics of openness, dynamism, fluidity, and sharing, posited against the closed culture of proprietary consumer technology by which we are commonly surrounded. Between live coding, modular synthesis, creative computing, hardware hacking, and appropriation-based interactive performances, the contributors to this series operate in many creative modes that oscillate between improvisation and play. What all of them share is a sense that both play and improvisation enable creative expression outside of prescribed, commodified circuits of media consumption. Improvisation and play can yield wonderful aesthetic experiments and experiences, they can provide critical, social commentary, and they can sharpen our sense of the impact that proprietary technology has on our digital cultural landscape. What we are most interested in are moments when these ambitions converge, when improvisation – drawing on techniques or technologies of play and operating in modes that are associated with play – becomes critical.
All images used with permission from Vector Game Art & New Media Festival.
Featured image: ”Blip Fest 2011 @ Eyebeam, Day 2” by Wikimedia Commons user Lucius Kwok CC BY-SA.
Skot Deeming is an interdisciplinary artist, curator and scholar, whose work spans the spectrum of new media art practice from broadcast media to computational art, experimental videogames, and game art. Drawing on a wealth of practical experience and theoretical knowledge, skot’s practice focuses on new media history and historiography, and DIY technology cultures. This work articulates itself through critical research and writing, exhibitions, installations, and performances. With Martin, Skot is the co-curator of Vector Game Art & New Media Festival., in Toronto, Canada, and the artistic director of the Drone Island Sound Art Festival in Yellowknife, NT. He currently resides in Montreal where he is a doctoral student in the Individualized Program at Concordia University, and a member of the TAG Research Centre, where he investigates critical histories of computational and new media art practices. http://www.mrghosty.com/
Martin Zeilinger (PhD) is a new media researcher, practitioner, and curator whose work focuses on the intersection of digital media art, activism, and intellectual property issues. He is a Lecturer in Media at Anglia Ruskin University, Cambridge/UK, as well as an Associate Researcher at the OCADU Data Materialization Studio. He is co-editor of ‘Dynamic Fair Dealing’ (Univ. of Toronto Press 2014), which develops interdisciplinary perspectives on cultural ownership issues in new media contexts; recent peer-reviewed essays have appeared in the Computer Music Journal, the International Assoc. for the Study of Popular Music Journal, and in Sampling Media (Oxford UP 2014). With Skot, Martin is co-curator of the Toronto-based Vector Game Art & New Media Festival. http://marjz.net/
REWIND! . . .If you liked this post, you may also dig:
Digital Analogies: Techniques of Sonic Play — Roger Mosley