Archive | Sound and Science RSS for this section

Critical bandwidths: hearing #metoo and the construction of a listening public on the web

“A focus on listening [with technology] shifts the idea of freedom of speech from having a platform of expression to having the possibility of communication” (K. Lacey)

One of the biggest social media event of the past decade, #metoo stands out as a pivotal shift in the future of gender relations. Despite its persistence since October 2017, #metoo is still under-theorized, and since its permutations generate countless hashtag sub-categories each passing week, making sense of it presents a conceptual quagmire. Tracing its history, identifying key moments, mapping its pro- and counter-currents present equally tough challenges to both data science and feminist scholars.

Meta-communication about #metoo abounds. Infographics and visualizations attempt to contain its organic growth into perceivable maps and charts; pop news media constantly report on its evolution in likes, counts, and retweets, as well as—and increasingly—in number of convictions, lawsuits, and reports. At the same time, #metoo has arguably created a discernible listening public in the way that Kate Lacey (2013) argues emerged with national radio: women’s stories have never been listened to with such wide reach and rapt attention.

How did #metoo create new listening publics? “#metoo” by Flickr User Prachatai, (CC BY-NC-ND 2.0)

The project I discuss here takes ‘hearing’ #metoo a step further into the auditory realm in the form of data sonification so as to to re-imagine an audience compelled to earwitness not just the scope but the emotional impact of women’s stories. Data sonification is a growing field, which from its inception has crossed between art and science. It involves a conceptual or semantic translation of data into relevant sonic parameters in a way that utilizes perceptual gestalts to convey information through sound.

Brady Marks and I created the #metoo sonification you’ll hear below by drawing from a public dataset spanning October 2017 to the early Spring of 2018 obtained from data.world. Individual tweets using the hashtag are sonified using female battle cries from video games; the number of retweets and followers forms a sort of swelling and contracting background vocal texture to represent the reach of each message. The dataset is then sped up anywhere between 10x to 1000x in order to represent perceivable ebbs and flows of the hashtag’s life over time. The deliberate aim in this design was to convey a different sensibility of social media content, one that demands emotional and intellectual attention over a duration of time. Given Twitter’s visual zeitgeist whereby individual tweets are perceived at a glance and quickly become lost in the noise of the platform the affective attitude towards “contagious” events becomes arguably impersonal. A sonification such as this asks the listener to spend 30 minutes listening to 1 month of #metoo: something impossible to achieve on the actual platform, or in a single visualization. The aim, then, is to interrupt social media’s habitual and disposable engagements with pressing civic debates. 

MeToo: It's About Power | Morningside Center for Teaching Social  Responsibility

A critique of big data visualization

To date, there have been more than 19 million #MeToo tweets from over 85 countries; on Facebook more than 24 million people participated in the conversation by posting, reacting, and commenting over 77 million times since October 15, 2017. In a global information society ‘big data’ is translated into creative infographics in order to simultaneously educate an overwhelmed public and elicit urgency and accord for political action. Yet ideological and political considerations around the design of visual information have lagged behind enthusiasm for making data ‘easy to understand’. At the other end of the spectrum, social media delivers personalized micro-trends directly and in real time to always-mobile users, reinforcing their information silos (Rambukkana 2015). Between these extremes, the mechanisms by which relevant local, marginalized or emergent issues come to be communicated to the wider public are constrained.

“A wordcloud featuring #metoo” by www.scootergenius.com (CC BY 2.0)

With this big idea in mind, the question we ask here is what would it mean to hear data?  Emergent work in sonification suggests that sound may afford a unique way to experience large-scale data suitable for raising public awareness of important current issues (Winters & Weinberg 2015). The uptake of sonification by the artistic community (see Rory Viner, Robert Alexander, among many others) signals its strengths in producing affective associations to data for non-specialized audiences, despite its shortcomings as a scientific analysis tool (Supper, 2018). Some of the more esoteric uses of sonification have been in the service of capturing what Supper calls ‘the sublime’ – as in Margaret Schedel’s “Sounds of Science: The Mystique of Sonification.”

Who’s listening on social media?

Within the Western canon of sound studies “constitutive technicities” (Gallope 2011) or what Sterne calls “perceptual technics” embody historically situated ways of listening that center technology as a co-defining factor in our relationship with sound. Within this frame, media sociologist Kate Lacey traces the emergence of the modern listening public through the history of radio. Using the metaphor of ‘listening in” and “listening out,” Lacey reframes media citizenship by pointing out that listening is a cultural as well as a perceptual act with defined political dimensions:

Listening out is the practice of being open to the multiplicity of texts and voices and thinking of texts in the context of and in relation to a difference and how they resonate across time and in different spaces. But at the same time, it is the practice and experience of living in a media age that produces and heightens the requirement, the context, the responsibilities and the possibilities of listening out (198)

According to Lacey, a focus on listening instead of spectatorship challenges the implicit active/passive dualism of civic participation in Western contexts. More importantly, she argues, we need to move away from the notion of “giving voice” and instead create meaningful possibilities to listen, in a political sense. Data sonification doesn’t so much ‘give voice to the voiceless’ but creates a novel relationship to perceiving larger patterns and movements.

Our interactions with media, therefore, are always already presumptive of particular dialogical relations. Every speech act, every message implies a listening audience that will resonate understanding. In other words, how are we already listening in to #metoo?  How and why might data sonification enable us to “listen out” for it instead? In order to get a different hearing, what should #metoo sound like?

What should #metoo sound like? “de #metoo à #wetogether” by Flickr user Jeanne Menjoulet, Paris 8 mars 2018 (CC BY 2.0)

Sonifying #metoo: the battle cries of gender-based violence

It is unrealistic to expect that your everyday person will read large archives of testimony on sexual harassment and gender-based violence. Because of their massive scale, archives of #metoo testimony pose a significant challenge to the possibility for meaningful communication around this issue.  Essentially drowning each other out, individual voices remain unheard in the zeitgeist of media platforms that automates quantification while speeding up engagement with individual contributions. To reaffirm the importance of voice would mean to reaffirm inter-subjectivity and to recognize polyphony as an “existential position of humanity” (Ihde 2007, 178). This was the problem to sonify here: how to retain individual voices while creating the possibility for listening to the whole issue at hand. Inspired by the idea of listening out, myself and artist collaborator Brady Marks set out to sonify #metoo as a way of eliciting the possibility for a new listening public.

Sophitia Alexandra of the Soul Calibur franchise, close up from Flickr User Ngo Quang Minh‘s image from Soul Calibur III (CC BY-SA 2.0)

The #metoo sonification project intersected deeply with my work on the female voice in videogames. My choice to use a mixed selection of battle cry samples from Soul Calibur, an arcade fighting game, was intuitive. Battle cries are pre-recorded banks of combat sounds that video game characters perform in the course of the story. Instances of #metoo on Twitter presumably represent the experiences of individual women, pumping a virtual fist in the air, no longer silent about the realities of gender-based violence. So hearing #metoo posts as battle cries of powerful game heroines made sense to me. But it’s the meta layers of meaning that are even more intuitive: as I’ve discussed elsewhere, female battle cries are notoriously gendered and sexualized. Listening to a reel of sampled battle cries is almost indistinguishable from listening to a pornographic soundscape. Abstracted in this sonification, away from the cartoonish hyper-reality of a game world, these voices are even more eerie, giving almost physical substance to the subject matter of #metoo. Just as the female voice in media secretly fulfils the furtive desires of the “neglected erogenous zone” of the ear (Pettman 2017, 17), #metoo is an embodiment of the conflation of sex with consent: the basis of what we now call ‘rape culture.’

Sonifying real-time data such as Twitter presents not only semantic (how should it sound like) but also time-scale challenges. If we are to sonify a month – e.g. the month of November 2017 (just weeks after the explosion of #metoo) – but we don’t want to spend a month listening, then that involves some conceptual time-scaling. Time-scaling means speeding up instances that already happen multiple times a second on a platform as instantaneous and global as Twitter. Below are samples of three different sonifications of #metoo data, following different moments in the initial explosion of the hashtag and rendered at different time compressions. Listen to them one at a time and note your sensual and emotive experience of tweets closer to real-time playback, compared to the audible patterns that emerge from compressing longer periods of time inside the same length audio file. You might find that the density is different. Closer to real-time the battle cries are more distinctive, while at higher time compressions what emerges instead is an expanding and contracting polyphonic texture.

Vocalizations of female pleasure/affect, video game battle cries already have a special relationship to technologies of audio sampling and digital reproduction as Corbett & Kapsalis describe in Aural sex: the female orgasm in popular sound.”  This means that the perceptual technics involved in listening to recorded female voices are already coded with sexual connotations. Battle cries in games are purposely exaggerated so as to carry the bulk of emotional content in the game’s experiential matrix. Roland Barthes’ notion of the “grain of the voice”—the presence of the body in (singing) voice—is frequently evoked in describing the substantive role that game voices play in the construction of game world immersion and realism. In the #metoo sonification, I decontextualize the grain of the voice—there are no visual images, narrative, or gameplay; the battle cries are also acousmatic, in that there are no bodies visually represented from which these sounds emanate.

The battle cry in this #metoo sonification is the ultimate disembodied voice, resisting what Kaja Silverman (1988) calls the “norm of synchronization” with a female body in The Acoustic Mirror (83). As acousmatic voices, these battle cries could be said to exist on a different conceptual and perceptual plane, “disturbing the taxonomies upon which patriarchy depends,” to quote Dominic Pettman in Sonic Intimacy. (22). In other words, the sounds exist in a boundary space between combat sounds and orgasmic sounds highlighting for the listener the dissonance between the supposed empowerment of ‘speaking out’ within a culture that remains staunchly set up to sexualize women; something one can hardly ignore given the media’s reserved treatment of #metoo.

“Princess Zelda’s New Mouth” by Flickr UserMouthGuy2013, Public Domain

Liberated from the game world these voices now speak for themselves in the #metoo sonification, their sensuality all the more hyper-real. The player has no control here, as the battle cries are not linked to specific game actions, rather they are synchronized autonomously to instances of #metoo confessionals.  In fact, the density of the sonification as time speeds up will overwhelm listeners with its boundlessness; echoing how contemporary media treats the sounds of the female orgasm as a renewable and inexhaustible resource, even as reports of sexual harassment and gender-based violence continue to pile on in 2021. Yet we intend that the subject matter resists pleasure, rendering the sonic experience traumatic as the chilling realization sets in that listeners are hailed to accountability by #metoo. The experience should instead be unsettling, impactful, grotesque, and deeply embodied. 

Concluding remarks

Listening both metaphorically and literally goes to the very heart of questions to do with the politics and experience of living and communicating in the media age. In her paper on the sonic geographies of the voice, AM Kanngieser notes in “A Sonic Geography of Voice“: “The voice, in its expression of affective and ethico-political forces, creates worlds” (337). It is not just in the grain but in the enunciation that battle cries find their political significance in this sonification. As the hyper-real gasps and moans of game heroines animate individual moments of #metoo the codification of cartoonish voices resists being subconsciously “absorbed into the dialogic exchange” (342) of habitual media consumption. Listening to the sonification is instead an experience of re-coding the voice, reconfiguring the embedded meanings of game sound to a new and contradictory context: a space that challenges neoliberal appropriations of radical communication and discourse (348). This is not data sonification that delights the listener or simply grants them access to ‘information’ in a different format; rather it calls on the listener to de-normalize their received technicity and perceptions and to connect to the emotional inter-subjectivity of this call to action.

Most importantly, the #metoo sonification invites the auditeur to listen in, to take an active role in the reconfiguration of meanings and absorb their political dimensions. These are the stories of #metoo; these are the voices of women, of men, of marginalized peoples, emerging from the zeitgeist of Twitter to ask us to earwitness gender-based violence. We are a new listening public, wanting and needing to create new worlds. A critical bandwidth is the smallest perceivable unit of auditory change, in psychology terms. This sonification begs the question, how many battle cries will it take for us to end gender-based violence by fostering equitable worlds?

Featured Image: “Listen to What You See” by Flickr User Hernán Piñera (CC BY-SA 2.0)

Milena Droumeva is an Assistant Professor and the Glenfraser Endowed Professor in Sound Studies at Simon Fraser University specializing in mobile media, sound studies, gender, and sensory ethnography. Milena has worked extensively in educational research on game-based learning and computational literacy, formerly as a post-doctoral fellow at the Institute for Research on Digital Learning at York University. Milena has a background in acoustic ecology and works across the fields of urban soundscape research, sonification for public engagement, as well as gender and sound in video games. Current research projects include sound ethnographies of the city (livable soundscapes), mobile curation, critical soundmapping, and sensory ethnography. Check out Milena’s Story Map, “Soundscapes of Productivity” about coffee shop soundscapes as the office ambience of the creative economy freelance workers. 

Milena is a former board member of the International Community on Auditory Displays, an alumni of the Institute for Research on Digital Learning at York University, and former Research Think-Tank and Academic Advisor in learning innovation for the social enterprise InWithForward.  More recently, Milena serves on the board for the Hush City Mobile Project founded by Dr. Antonella Radicchi, as well as WISWOS, founded by Dr. Linda O Keeffe.

tape reel

REWIND! . . .If you liked this post, you may also dig:

One Scream is All it Takes: Voice Activated Personal Safety, Audio Surveillance, and Gender ViolenceMaría Edurne Zuazu 

Gendered Sonic Violence, from the Waiting Room to the Locker Room–Rebecca Lentjes

SO! Amplifies: Mendi+Keith Obadike and Sounding Race in America

detritus 1 & 2 and V.F(i)n_1&2 : The Sounds and Images of Postnational Violence in Mexico–Luz María Sánchez

Hearing EugenicsVibrant Lives

SO! Amplifies: The Electric Golem (Trevor Pinch and James Spitznagel)

SO! Amplifies. . .a highly-curated, rolling mini-post series by which we editors hip you to cultural makers and organizations doing work we really really dig.  You’re welcome!

On March 24th, 2019 the record release party for The Electric Golem’s 6th CD Golemology was held at the Loft in Ithaca, New York. The Electric Golem is an avant-garde synthesizer duo featuring Trevor Pinch and James Spitznagel, that has been in existence for about ten years.

Trevor Pinch is a local sound artist and professor at Cornell University. He is an STS (Science and Technology Studies) and Sound Studies scholar. As a key thinker of STS, Trevor is the coproducer of theories about Sociology of Scientific Knowledge, Social Construction of Technology (SCOT), and the role of users in technological history and innovation. However, Trevor’s interest in dates back much farther; he built his first modular synthesizer when he was a physics student in London in the 1970s.

The other half of The Electric Golem, James Spitznagel, is a multi-media artist who uses the iPad as a musical instrument and to create digital paintings. While he has played many roles in the music and culture industries—guitarist in a rock band, record store owner, art gallery and guitar shop investor, and even business manager for the Andy Warhol Museum—he moved to Ithaca to focus on producing abstract art: digital paintings and experimental, improvisational music. Being an energetic and enthusiastic person who has unrestrained fantasies, James finds that everything around him can be his inspiration.

Pinch and Spitznagel formed the group after Spitznagel read Analog Days: The Invention and Impact of the Moog Synthesizer (by Trevor Pinch and Frank Trocco) and realized Pinch also lived in Ithaca. Spitznagel simply looked his name up in the phone book and called him up: “I go, ‘is this Trevor Pinch?’ He said, ‘yes.’ I said, ‘well, you don’t know me, but I just read your book and I love it.’”  And then they got together for a beer and have been best friends and collaborators ever since.  Once Spitznagel heard about Pinch’s homemade synthesizer, he asked Trevor to try to make something together and it turned out to be a fascinating mixture of analog–Trevor’s synth, Moog Prodigy, and a Minimoog–and James’s digital instruments.

Building from this first moment of discovery, The Electric Golem’s music is electronic, experimental, and totally improvised. Typically, the pieces of music last twenty minutes to half an hour and expresses their interaction with the machines and with each other in the studio. James is much more controlling of the tone and rhythm, and patches the sound as he goes along, whereas Trevor is much more about making spontaneous weird sounds. They complement each other and the creation process is usually by random and spontaneous, as Spitznagel describes: “I didn’t tell Trevor what to do or what to play, but I said, here’s the piece of music I’ve written. He just instinctively knew what add to it.” Reciprocally, “he might just play something that I go, oh, I can weave in and out of the ambient sound he’s putting there.”

Trevor Pinch, Electric Golem at Elmira College, 2012

For the duo, the process of producing music becomes a shared experience with their listeners. The music is ever changing and evolving. In addition, unexpected drama adds vitality to the palette. “The iPad might freeze up or synthesizer might break somehow,” Spitznagel notes, “that’s happened to us, but we carry on. Like Trevor looks at me and says, it’s not working there. Or, I look at him and go, I have to reboot my computer, it’s not working. But, those times actually inspire us to try new things and go beyond what we are doing.” James explained. Their inspiration comes from the unknown, which just emerges from their practice. “Generally, this sort of music is completely unique to Electric Golem.” Trevor concluded.

The name “Electric Golem” comes from a series of books with Golem in the titles that Trevor collaborated on with his mentor Harry Collins. “The golem is a creature of Jewish mythology,” Pinch and Collins wrote in The Golem, What You Should Know about Science, “it is a humanoid made by man with clay and water, with incantations and spells. It is powerful, it grows a little more powerful every day.  It will follow orders, do your work, and protect you from the ever threatening enemy.  But it is clumsy and dangerous.  Without control, a golem may destroy its masters with its flailing vigour” (1).  Noting Trevor’s association with the concept of the Golem, Spitznagel added the “Electric” twist not just as a metaphor for their sound but also because “it’s kind of like a retro name.” The Electric Golem mushroomed from there, and in the past decade they have had many invitations and bookings to play out, receiving the first recording contract from the Ricochet Dream label, and have played with a bunch of notable musicians, such as Malcolm Cecil of Tonto’s Expanding Head Band, Simeon of Silver Apples, and “Future Man” (aka Roy Wooten), and they haven’t stopped there.

According to Pinch, the key feature of The Electric Golem’s music is its ability to encompass different moods. “I think Electric Golem has become good at one thing: its changing and transitioning from one sort of mood of music to another. And we have become quite good at those transitions. I think people would say that’s what they kind of like about us.” These sorts of slow transitions construct a unique texture of sound that can be quite cinematic, so much so that in 2012, the Electric Golem performed the accompaniment to the silent movie A Trip to the Moon, a special Cornell cinema event. Overall, as improvised experimental music, it is sometimes challenging to listen to, with no regular rhythm or reliable melody. Trevor produces warm, rich drones from the analog side that contrast with the sharper digital rhythms that James programs. In short, the Electric Golem varies between these two affects but the music goes far beyond the representation of emotional states; sometimes it conjures up the feeling of the vastness of space and time.

Experimental music, is a collaboration and negotiation process between instruments and their users.  No matter if analog or digital, instruments have autonomy; they are non-human actors with their own agency to some extent. As Trevor Pinch intimates, “I understand the general sort of sound that can be produced, but the particular details of how it will work out, you don’t really know, that’s much more spontaneous, you have to react to that.”  Instruments can often be uncontrollable–making their own sounds—so that Electric Golem must respond in kind. “So, it’s sort of like higher level meta-control versus actually doing what you’re doing in response to the instrument that combines together,” Trevor describes, “which I think is the secret to controlling these sorts of instruments.” It is incredible that Pinch and Spitznagel know each other so well—and each know their instruments so well–that they can improvise for long periods with no trouble. Trevor says: “Follow the use of these instruments! Follow the instruments! They are not essentialized. They are just stabilized temporarily.”

On the whole, The Electric Golem shows an artistic form which breaks the traditional paradigm, deconstructs and then reconstructs it, seeking to free sound from the instruments. Their music is beyond pure melody and rhythm, beyond the expression of existence, expressing more of an aesthetic state of transcendence. They challenge what music is, and what musical instruments are; they challenge divisions between the identities of engineer and musician. Electric Golem’s music co-constructs art and technology and binds them together; art, for them, is a mode of presenting technology, and vice versa, technology is a pathway through which art can flourish.

My favorite Electric Golem piece is called “Heart of the Golem.” What is the heart of the Golem? According to Pinch, “It is a mystery, a process of unfolding and discovery. It is somewhere where analog and digital sound meet, and an improvisation.” What the magic is remains unknown and unlimited, just like the future of the Electric Golem.

Featured Image: Courtesy of The Electric Golem

Qiushi Xu is a PhD candidate in the subject of Philosophy of Science and Technology in Tsinghua University, Beijing and in a joint PhD program in the Department of Science and Technology Studies in Cornell University, working with Prof. Trevor Pinch. Her research areas are Sound Studies, STS, Cultural Studies and Gender Studies. Her current research focuses on the sociology of piano sound and the negotiation and construction of piano sound in the recording studio (PhD dissertation), gender issues in recording industry, experimental music, auscultation and sound therapy. She holds an MA in Cultural and Creative Industries from King’s College London; a BA in Recording Arts and a BA in Journalism and Communication from the University of China, Beijing. She is also an amateur pianist, writer, and traditional Chinese painter. As a multiculturalist, she is am fascinated by different forms of art and culture in different cultural contexts.

%d bloggers like this: