“A focus on listening [with technology] shifts the idea of freedom of speech from having a platform of expression to having the possibility of communication” (K. Lacey)
One of the biggest social media event of the past decade, #metoo stands out as a pivotal shift in the future of gender relations. Despite its persistence since October 2017, #metoo is still under-theorized, and since its permutations generate countless hashtag sub-categories each passing week, making sense of it presents a conceptual quagmire. Tracing its history, identifying key moments, mapping its pro- and counter-currents present equally tough challenges to both data science and feminist scholars.
Meta-communication about #metoo abounds. Infographics and visualizations attempt to contain its organic growth into perceivable maps and charts; pop news media constantly report on its evolution in likes, counts, and retweets, as well as—and increasingly—in number of convictions, lawsuits, and reports. At the same time, #metoo has arguably created a discernible listening public in the way that Kate Lacey (2013) argues emerged with national radio: women’s stories have never been listened to with such wide reach and rapt attention.
The project I discuss here takes ‘hearing’ #metoo a step further into the auditory realm in the form of data sonification so as to to re-imagine an audience compelled to earwitness not just the scope but the emotional impact of women’s stories. Data sonification is a growing field, which from its inception has crossed between art and science. It involves a conceptual or semantic translation of data into relevant sonic parameters in a way that utilizes perceptual gestalts to convey information through sound.
Brady Marks and I created the #metoo sonification you’ll hear below by drawing from a public dataset spanning October 2017 to the early Spring of 2018 obtained from data.world. Individual tweets using the hashtag are sonified using female battle cries from video games; the number of retweets and followers forms a sort of swelling and contracting background vocal texture to represent the reach of each message. The dataset is then sped up anywhere between 10x to 1000x in order to represent perceivable ebbs and flows of the hashtag’s life over time. The deliberate aim in this design was to convey a different sensibility of social media content, one that demands emotional and intellectual attention over a duration of time. Given Twitter’s visual zeitgeist whereby individual tweets are perceived at a glance and quickly become lost in the noise of the platform the affective attitude towards “contagious” events becomes arguably impersonal. A sonification such as this asks the listener to spend 30 minutes listening to 1 month of #metoo: something impossible to achieve on the actual platform, or in a single visualization. The aim, then, is to interrupt social media’s habitual and disposable engagements with pressing civic debates.
A critique of big data visualization
To date, there have been more than 19 million #MeToo tweets from over 85 countries; on Facebook more than 24 million people participated in the conversation by posting, reacting, and commenting over 77 million times since October 15, 2017. In a global information society ‘big data’ is translated into creative infographics in order to simultaneously educate an overwhelmed public and elicit urgency and accord for political action. Yet ideological and political considerations around the design of visual information have lagged behind enthusiasm for making data ‘easy to understand’. At the other end of the spectrum, social media delivers personalized micro-trends directly and in real time to always-mobile users, reinforcing their information silos (Rambukkana 2015). Between these extremes, the mechanisms by which relevant local, marginalized or emergent issues come to be communicated to the wider public are constrained.
With this big idea in mind, the question we ask here is what would it mean to hear data? Emergent work in sonification suggests that sound may afford a unique way to experience large-scale data suitable for raising public awareness of important current issues (Winters & Weinberg 2015). The uptake of sonification by the artistic community (see Rory Viner, Robert Alexander, among many others) signals its strengths in producing affective associations to data for non-specialized audiences, despite its shortcomings as a scientific analysis tool (Supper, 2018). Some of the more esoteric uses of sonification have been in the service of capturing what Supper calls ‘the sublime’ – as in Margaret Schedel’s “Sounds of Science: The Mystique of Sonification.”
Who’s listening on social media?
Within the Western canon of sound studies “constitutive technicities” (Gallope 2011) or what Sterne calls “perceptual technics” embody historically situated ways of listening that center technology as a co-defining factor in our relationship with sound. Within this frame, media sociologist Kate Lacey traces the emergence of the modern listening public through the history of radio. Using the metaphor of ‘listening in” and “listening out,” Lacey reframes media citizenship by pointing out that listening is a cultural as well as a perceptual act with defined political dimensions:
Listening out is the practice of being open to the multiplicity of texts and voices and thinking of texts in the context of and in relation to a difference and how they resonate across time and in different spaces. But at the same time, it is the practice and experience of living in a media age that produces and heightens the requirement, the context, the responsibilities and the possibilities of listening out (198)
According to Lacey, a focus on listening instead of spectatorship challenges the implicit active/passive dualism of civic participation in Western contexts. More importantly, she argues, we need to move away from the notion of “giving voice” and instead create meaningful possibilities to listen, in a political sense. Data sonification doesn’t so much ‘give voice to the voiceless’ but creates a novel relationship to perceiving larger patterns and movements.
Our interactions with media, therefore, are always already presumptive of particular dialogical relations. Every speech act, every message implies a listening audience that will resonate understanding. In other words, how are we already listening in to #metoo? How and why might data sonification enable us to “listen out” for it instead? In order to get a different hearing, what should #metoo sound like?
Sonifying #metoo: the battle cries of gender-based violence
It is unrealistic to expect that your everyday person will read large archives of testimony on sexual harassment and gender-based violence. Because of their massive scale, archives of #metoo testimony pose a significant challenge to the possibility for meaningful communication around this issue. Essentially drowning each other out, individual voices remain unheard in the zeitgeist of media platforms that automates quantification while speeding up engagement with individual contributions. To reaffirm the importance of voice would mean to reaffirm inter-subjectivity and to recognize polyphony as an “existential position of humanity” (Ihde 2007, 178). This was the problem to sonify here: how to retain individual voices while creating the possibility for listening to the whole issue at hand. Inspired by the idea of listening out, myself and artist collaborator Brady Marks set out to sonify #metoo as a way of eliciting the possibility for a new listening public.
The #metoo sonification project intersected deeply with my work on the female voice in videogames. My choice to use a mixed selection of battle cry samples from Soul Calibur, an arcade fighting game, was intuitive. Battle cries are pre-recorded banks of combat sounds that video game characters perform in the course of the story. Instances of #metoo on Twitter presumably represent the experiences of individual women, pumping a virtual fist in the air, no longer silent about the realities of gender-based violence. So hearing #metoo posts as battle cries of powerful game heroines made sense to me. But it’s the meta layers of meaning that are even more intuitive: as I’ve discussed elsewhere, female battle cries are notoriously gendered and sexualized. Listening to a reel of sampled battle cries is almost indistinguishable from listening to a pornographic soundscape. Abstracted in this sonification, away from the cartoonish hyper-reality of a game world, these voices are even more eerie, giving almost physical substance to the subject matter of #metoo. Just as the female voice in media secretly fulfils the furtive desires of the “neglected erogenous zone” of the ear (Pettman 2017, 17), #metoo is an embodiment of the conflation of sex with consent: the basis of what we now call ‘rape culture.’
Sonifying real-time data such as Twitter presents not only semantic (how should it sound like) but also time-scale challenges. If we are to sonify a month – e.g. the month of November 2017 (just weeks after the explosion of #metoo) – but we don’t want to spend a month listening, then that involves some conceptual time-scaling. Time-scaling means speeding up instances that already happen multiple times a second on a platform as instantaneous and global as Twitter. Below are samples of three different sonifications of #metoo data, following different moments in the initial explosion of the hashtag and rendered at different time compressions. Listen to them one at a time and note your sensual and emotive experience of tweets closer to real-time playback, compared to the audible patterns that emerge from compressing longer periods of time inside the same length audio file. You might find that the density is different. Closer to real-time the battle cries are more distinctive, while at higher time compressions what emerges instead is an expanding and contracting polyphonic texture.
Vocalizations of female pleasure/affect, video game battle cries already have a special relationship to technologies of audio sampling and digital reproduction as Corbett & Kapsalis describe in “Aural sex: the female orgasm in popular sound.” This means that the perceptual technics involved in listening to recorded female voices are already coded with sexual connotations. Battle cries in games are purposely exaggerated so as to carry the bulk of emotional content in the game’s experiential matrix. Roland Barthes’ notion of the “grain of the voice”—the presence of the body in (singing) voice—is frequently evoked in describing the substantive role that game voices play in the construction of game world immersion and realism. In the #metoo sonification, I decontextualize the grain of the voice—there are no visual images, narrative, or gameplay; the battle cries are also acousmatic, in that there are no bodies visually represented from which these sounds emanate.
The battle cry in this #metoo sonification is the ultimate disembodied voice, resisting what Kaja Silverman (1988) calls the “norm of synchronization” with a female body in The Acoustic Mirror (83). As acousmatic voices, these battle cries could be said to exist on a different conceptual and perceptual plane, “disturbing the taxonomies upon which patriarchy depends,” to quote Dominic Pettman in Sonic Intimacy. (22). In other words, the sounds exist in a boundary space between combat sounds and orgasmic sounds highlighting for the listener the dissonance between the supposed empowerment of ‘speaking out’ within a culture that remains staunchly set up to sexualize women; something one can hardly ignore given the media’s reserved treatment of #metoo.
Liberated from the game world these voices now speak for themselves in the #metoo sonification, their sensuality all the more hyper-real. The player has no control here, as the battle cries are not linked to specific game actions, rather they are synchronized autonomously to instances of #metoo confessionals. In fact, the density of the sonification as time speeds up will overwhelm listeners with its boundlessness; echoing how contemporary media treats the sounds of the female orgasm as a renewable and inexhaustible resource, even as reports of sexual harassment and gender-based violence continue to pile on in 2021. Yet we intend that the subject matter resists pleasure, rendering the sonic experience traumatic as the chilling realization sets in that listeners are hailed to accountability by #metoo. The experience should instead be unsettling, impactful, grotesque, and deeply embodied.
Listening both metaphorically and literally goes to the very heart of questions to do with the politics and experience of living and communicating in the media age. In her paper on the sonic geographies of the voice, AM Kanngieser notes in “A Sonic Geography of Voice“: “The voice, in its expression of affective and ethico-political forces, creates worlds” (337). It is not just in the grain but in the enunciation that battle cries find their political significance in this sonification. As the hyper-real gasps and moans of game heroines animate individual moments of #metoo the codification of cartoonish voices resists being subconsciously “absorbed into the dialogic exchange” (342) of habitual media consumption. Listening to the sonification is instead an experience of re-coding the voice, reconfiguring the embedded meanings of game sound to a new and contradictory context: a space that challenges neoliberal appropriations of radical communication and discourse (348). This is not data sonification that delights the listener or simply grants them access to ‘information’ in a different format; rather it calls on the listener to de-normalize their received technicity and perceptions and to connect to the emotional inter-subjectivity of this call to action.
Most importantly, the #metoo sonification invites the auditeur to listen in, to take an active role in the reconfiguration of meanings and absorb their political dimensions. These are the stories of #metoo; these are the voices of women, of men, of marginalized peoples, emerging from the zeitgeist of Twitter to ask us to earwitness gender-based violence. We are a new listening public, wanting and needing to create new worlds. A critical bandwidth is the smallest perceivable unit of auditory change, in psychology terms. This sonification begs the question, how many battle cries will it take for us to end gender-based violence by fostering equitable worlds?
Milena Droumeva is an Assistant Professor and the Glenfraser Endowed Professor in Sound Studies at Simon Fraser University specializing in mobile media, sound studies, gender, and sensory ethnography. Milena has worked extensively in educational research on game-based learning and computational literacy, formerly as a post-doctoral fellow at the Institute for Research on Digital Learning at York University. Milena has a background in acoustic ecology and works across the fields of urban soundscape research, sonification for public engagement, as well as gender and sound in video games. Current research projects include sound ethnographies of the city (livable soundscapes), mobile curation, critical soundmapping, and sensory ethnography. Check out Milena’s Story Map, “Soundscapes of Productivity” about coffee shop soundscapes as the office ambience of the creative economy freelance workers.
Milena is a former board member of the International Community on Auditory Displays, an alumni of the Institute for Research on Digital Learning at York University, and former Research Think-Tank and Academic Advisor in learning innovation for the social enterprise InWithForward. More recently, Milena serves on the board for the Hush City Mobile Project founded by Dr. Antonella Radicchi, as well as WISWOS, founded by Dr. Linda O Keeffe.
REWIND! . . .If you liked this post, you may also dig:
Hearing Eugenics—Vibrant Lives
One Scream is All it Takes: Voice Activated Personal Safety, Audio Surveillance, and Gender Violence
Just a few days ago, London Metro Police Officer Wayne Couzens pled guilty to the rape and murder of Sarah Everard by, a 33-year-old woman he abducted while she walked home from a friend’s house. Since the news broke of her disappearance in March 2021, the UK has been going through a moment of national “soul-searching.” The national reckoning has included a range of discussions–about casual and spectacular misogynistic violence, about a victim-blaming criminal justice system that fails to address said violence–and responses, including a vigil in south London that was met with aggressive policing, that has itself entered into and furthered the UK’s soul-searching. There has also been a surge in the installation of personal safety apps on mobile phones; One Scream (OS), “voice activated personal safety,” is one of them.
Available for Android and iOS devices, OS claims to detect and be triggered by a woman’s (true) “panic scream,” and, after 20 seconds and unless the alarm is cancelled, it will send both a text message to the user’s chosen contacts and an automated call with the location to a nominated contact. The app is meant to help women in situations where dialing 999, (assumed to be the natural and preferred response to danger), is not viable for the user and, in the ideal embodiment, this nominated contact, “the helper,” is the police. OS did automatically contact police (and required a paid subscription) in 2016, but it did not work out well and by 2018, was declared a work in progress: “What we really want is for the app to dial 999 when it detects a panic scream, but first, we need to prove how accurate it is. That’s where you come in. . .” OS is currently in beta and free (while in beta). It is unclear whether the developers have given up with that utmost expression of OS.
OS is based on the premise that men fight and women scream —“It is an innate response for females in danger to scream for help”—and its correct functioning requires its users to be ready to do so, even if such an innate and instinctive response doesn’t come naturally to them: “If you do not scream, the app will not be able to detect you.” However, there are two discriminations in terms of scream analysis, in how the app discriminates while listening for and to screams, and in failing to detect or respond to them. The first has to do with who can use the app (i.e., whose panicked screams are able to trigger it) in the first place. This is presented in terms of gender and age—for the moment, OS can listen to “girls aged 14+ and women under 60,” where cisgender, as in anything OS, is taken for granted. It is, however, a matter of acoustic parameters set by the developers (notably, of reaching a certain high pitch and loudness threshold). Which is why the app was implemented to include a “screamometer” for potential users to scream, hard, figure out, and see whether they can reach “the intensity that is needed to set it off” (confetti means they do). The second one discriminates true panicked screams from other types of screams (e.g., happiness, untrue panic). As presented by the developers, both discriminations are problematic and misleading, and so is “the science behind screaming” One Scream‘s website boasts of.
The app does not quite distinguish true from fake screams, nor joy from panic for that matter. Instead, One Scream listens for “roughness,” which a team of scream researchers—it truly is a “tiny science lesson” —has identified as scream’s “privileged acoustic niche” for communicating alarm. According to this 2015 study in Current Biology, “roughness” is the distinctive quality of effective, compelling human screams (and of artificial alarms) in terms of their ability to trigger listeners and in terms of perceived urgency. Abrupt increases in loudness and pitch are not unique to screams. The rougher the scream, then, the greater its perceived “alarmess” and its alarming effect. That’s why developers say OS “hears real distress,” essentially “just as your own ear.” However, other studies suggest your own ears might not be so great at distinguishing happiness from fear and scream research, and particularly the specific “bit” OS builds on, by and large assumes, relies on, and furthers the irrelevance of “real” on the scream vocalizer end.
In OS’s pledge to its users, the app’s fine-tuning to its scream niche—i.e., to rough temporal modulations between 30 and 150Hz—is as important, as is the developers (flawed) insistence on the irredeemably uniqueness of true panic’s scream vocalizations, which they posit are instinctive and can’t be plotted or counterfeit: “Experience has shown that it is difficult for women to fake their scream.” Yet, current scream analysis and research primarily and largely relies on screams delivered by human research subjects (often university students, ideally drama students) in response to prompts for the purposes of studying them as well as, especially, on screams extracted from commercial movies and sound effect libraries. The same applies to the other types of vocalizations (e.g., neutral and valenced speech, screamed sentences, laughter, etc.) produced or retrieved for the purposes of figuring out what it is that makes a scream a scream, and how to translate that into a set of quantifiable parameters to capitalize on that knowledge, regardless of the agenda.
Because of their interest for audio surveillance applications, screams are currently a contested object and a hot commodity. Much as is the case with other scream distinction/detection enterprises, the initial training of OS most likely involved that vast and available bank of crafted scream renditions—by professional actors, machines, combinations of those, by and for an industry otherwise partial to female non-speech sounds—conveniently the exact type of “thick with body” female voicings OS is also invested in. For some readers, myself included, this might come across as creepy and, science-wise, flimsy.
Scream research often relies on how human listeners recruited for the cause respond to audio samples. Apparently, whether the scream is “real,” acted, or post-produced is neither something study subjects necessarily distinguish nor a determining factor in how they rate and react. In terms of machines learning to scream-mine audio data, it is what it is: “natural corpora with extreme emotional manifestation and atypical sounds events for surveillance applications” are scarce, unreliable, and largely unavailable because of their private character. That is no longer the case for OS, which has been accruing, and machine-learning from, its beta-user screams as well as how users themselves monitor/rate their screams and the app’s sensibility. OS users’ screams might not be exactly ad lib, as users/vocalizers first practice with the “screamometer” to learn to scream for and as a means to interface with OS, but it’s as natural a corpora as it gets, and it’s free for the users of the screams. OS not only echoes “voice stress analysis” technologies invested in distinguishing true from fake or in ranking urgency, but, as part and parcel of a larger scream surveillance enterprise, also public surveillance technologies such as ShotSpotter, all of which Lawrence Abu Hamdan has brilliantly dissected in his essay on the recording of the police gunshots that killed Michael Brown in Ferguson, Missouri in 2014.
Chilla is a strikingly similar app developed and available in India—although there’s a nuanced difference in the developer’s rationale for Chilla, which in its pursuance of scream-activated personal safety also aims to compensate for the fact that many girls and women don’t call “parents or police” for help when harassed or in danger. As presented, Chilla responds both to assaults and to women’s ambivalence towards their guardians. The latter is, too, a manifestation of the breadth of gender-based violence as a socio-cultural problem, one that Chilla is trained to fail to listen to and one that, because of OS’s particular niche user market, is simply out of the purview of its UK counterpart.
That problem–and that failure–is neither exclusive to India nor to scream-activated personal safety apps. Calling 999 in the UK, 911 in the US, or 091 in Spain, where I am writing, doesn’t come naturally to many targets of sexual and gender-based violence because they don’t conceive police as a help or because, directly, they see it as a risk—to themselves and/or to others. As Angela Ritchie has copiously documented in Invisible No More: Police Violence Against Black Women and Women of Color, women of color and Black women in particular are at extremely high risk for rape and sexual abuse by police officers, as high as 1 in 5 women in New York City alone.
OS, then, is framed as a pragmatic, partial answer to a problem it doesn’t solve: “We should never have to dress in a certain way…but we do.” The specifics of how OS would actually “save” or even has saved its users in particular scenarios go unexplained, because OS is meant to help with feeling safe; getting into the details, and the what ifs, compromises that service. This sense of safety has two components and is based on two promises: one, that OS will listen to your (panic) scream, and, two, as of now via the intermediacy of your contacts, the police will go save you. The second component and its assumed self-evidence speaks to the app’s whiteness and of its target market of white, securitized, cisgender female subjects.
Over and above its acoustic profiling, the app is simply not designed with every woman in mind. OS’s branding is about a certain lifestyle—of going for early runs and dates with cis-men, of taking time for yourself because you’re super busy at your white-collar job and going for night runs, of taking inspiration from “world” women and skipping if running isn’t for you. This lifestyle is also sold: sold as always under the threat of rape–despite its “rightfulness”–sold in a way that animates the feelings of insecurity and disempowerment that One Scream advertizes itself as capable of reversing. Safety, then, is sold as retrievable with OS.
Wearable or otherwise portable technologies to keep women “safe,” specifically from sexual assaults, are not new and are varied. These have been vigorously protested, particularly from feminist standpoints other than the white, securitized, capitalist brand OS professes—because, in (partly) delegating safety on technologies women then become personally responsible for, these technologies further “blame” women. For authorities and the patriarchy, this shift in blame is a relief. In discussing the racialized securitization of US university campuses, Kwame Holmes notes how despite “reactionary attacks” on campus feminism (e.g., so-called “snowflakes” complaining about bad sex) and authorities’ effective reluctance to acknowledge and challenge rape culture, anti-sexual assault technologies tend to be welcomed and accepted. As Holmes also notes, there’s no paradox in that. Those technologies flatten the discussion, deactivate more radical feminist critiques and potential strategies, and protect the status quo—not so much women and not those who, whenever an alarm sounds and especially when security forces respond, readily become insecure.
It is not a stretch to think that OS could potentially amplify the insecurities of Black and brown people subject to white panic (screams) and to its violence, something other audio surveillance technologies are already contributing to, at least it’s not a greater stretch than to entertain situations in which police would show up and save an OS user before it’s too late. Even if it’s never triggered, as developers seem to assume will be the case for the majority of installed units—”Many people have never faced a situation where they have had to panic scream”—it’s trapped in a securitization logic that ultimately relies on masculine authority, one that calls for the expansion of CCTV cameras, wherein women are never quite secure (see Sarah Everard’s vigil).
One Scream’s FAQs cover selected worries that users have or OS anticipates they might have. Among these, there are privacy concerns (i.e., does it listen to your conversations?) and the fear the alarm will activate “when it shouldn’t.” In the Apple Store user reviews, there’s a more popular type of concern: OS not responding to users’ screams. In other words, there’s simultaneously a worry about OS listening and detecting too much and about OS failing to listen “when it matters.” These anxieties around OS’s listening excesses and insufficiencies touch on (audio) surveillance paradoxical workings: does OS encroach on the everyday life of those within users’ cell phones’ earshot while not necessarily delivering on an otherwise modest promise of safety in highly specific scenarios? There’s a unified developer response to these concerns: OS “is trained to detect panic screams only.”
Featured Image: By Flicker User Dirk Haun. Image appears to be a woman screaming on a street corner, but is actually an advertisement on the window of a T-Mobile cell phone shop (CC BY 2.0)
María Edurne Zuazu works in music, sound, and media studies, and researches the intersections of material culture and sonic practices in relation to questions of cultural memory, social and environmental justice, and the production of knowledge (and of ignorance) in the West during the 20th and 21st centuries. María has presented on topics ranging from sound and multimedia art and obsolete musical instruments, to aircraft sound and popular music, and published articles on telenovela, weaponized uses of sound, music and historical memory, and music videos. She received her PhD in Music from The CUNY Graduate Center, and has been the recipient of Fulbright and Fundación La Caixa fellowships. She is a 2021-2022 Fellow at Cornell’s Society for the Humanities.
REWIND! . . .If you liked this post, you may also dig:
Flâneuse>La caminanta–Amanda Gutierrez
Echo and the Chorus of Female Machines—AO Roberts