Part of this surge in scholarly attention and artistic projects is premised on the idea that sound and sound recordings are important additions to cultural heritage in documenting histories and personal evidence, and yet, they are often viewed as supplementary adjuncts to more physical or visual archival artefacts. This subjugation to the primacy of the visual extends to the arts, humanities, and social sciences, but this is beginning to change as scholars across these fields increasingly argue that sound (with its attendant listening) is an especially critical medium for cultivating different modes of attention, forging affective relations, producing alternative knowledges, revealing hidden narratives, and attuning to neglected pasts.
Two digital sound archival projects created in response to the pandemic, The Pandemic SensoryArchive and#StayHomeSounds, are especially instructive in thinking about sound as a key medium for engaging with the monumental a/effects of the present and as important contributions to cultural history. Like other pandemic digital sound archival projects, these two projects sought to document the present for the future – creating a “past” in real time, based on the underlying assumption that sound – as a material-discursive apparatus – can offer particularly generative possibilities in this context. The methods, scope, and presentation design employed by these two web-based archival platforms generates a sense of intimacy, proximity, andcollectivity in otherwise surreal, secluded, uncertain, detached, and disconnected situations, much like in radio and podcasting though in this case with different infrastructure and interactivity.
In these online, mediated spaces where worlds intensely collide and conflate, and users become flattened out and disembodied, new configurations of intimacy, subjectivities, and world-building emerge through alternative forms of affective archival engagement. This was (and still is) particularly important and complex during COVID-19, which is marked not only by a series of indefinite lockdowns and uneven distribution of intervention measures, but an affective logic whereby life is completely reconfigured and capacities within the world are diminished and redistributed. Part of making mass sound archives usable relies on the medium for circulation, the presentation for users, and what user participation empowers for these living histories, as Fabiola Hanna makes clear. These two projects generate what Hanna identifies as a particular orientation in digital humanities projects through a politics of listening that necessitates an active mode of participation that is not simply one-directional but a two-way engagement.
The Archive of Intimacy, later renamed The Pandemic Sensory Archive (PSA), wascreated by professors William Tullett (Associate Professor in Sensory History, Anglia Ruskin University, U.K.) and Hannah McCann (Senior Lecturer in Cultural Studies, University of Melbourne, Australia) with the goal of exploring the senses through a digital platform and to act as an open data bank of contributions from the public. Their open call for contributions asks for a response that considers two questions: What smells, sights, sounds, touch, and/or tastes do you associate with the pandemic? Has your experiences of the senses (smell/sight/hearing/touching/taste) changed at all as a result of the pandemic? Contributors are then asked to drop a pin on the map (though this is not a typical Google Maps rendition of a specific locale, but a sort of simplified graphic representation of sensory input/output waves) and follow the prompts to anonymously submit. The digital map is divided among the five senses and features the entries that contain a title and brief response, such as:
Quiet where there should not be quiet: “Being in my flat in the centre of town on my own with nobody else around on a Saturday. Everything weirdly, eerily, quiet.”
Birdsong: “Hearing birdsong in the garden having not noticed it before, no longer drowned out.”
You’re on mute! “Did the conversation in meetings become less robust as we all sit there on mute, politely waiting for our turn to speak?”
Less sound, more sound: “Blissfully quiet at night as curfew curtails the normal constant traffic roars, far more voices in the early morning and through the day as people‘exercise together’ to socialize in the park”
Complimenting this map, which may seem limited in scope but allows users to engage without having to sift through an overwhelming amount of content, is the Sound category page, where four interviews are embedded with “sensory experts on sound during the Covid-19 pandemic,” including Shoshana Rosenberg, Andrew Mitchell, Martin Stewart, and Stephen Sullivan. One interview considers how the pandemic clarified the immensely relational dimension of artistic sound practice and that the lack of access to intimacy during lockdown instigated a radical reformatting and questioning of what it means, more broadly and now, to be intimate and close in creating sound art. For them, what the pandemic spelled out is that intimacy is fragile and valuable, and that this delicate balance and fluctuating ratio has come to the fore during this time.
The initial designation for the platform, “the archive of intimacy,” is worth meditating on to consider the particular forms of intimacy in this context, perhaps through Lauren Berlant’s “intimate publics” – a concept that captures the affective and collective dimensions of intimacy among strangers. The notion lends itself to understanding the mediated social intimacy in these spaces and the different affective experiences they invite in varying capacities through sound.The connection between imagined publics and community through sound has, of course, been conceptualized by scholars who do historical work on radio and podcast studies, but it can also be extended to these digital, affective, pandemic sound archives. Evidenced in the submission prompts and interview data, the emphasis on the distinct shifts and palpable changes resulting from this new situation, and its accompanying affective logic, can be read as a strategy for cultivating intimacy and connection because attending to these changes may render their intensities as less alarming.
“Listening” by Flickr User Silvia Siri, April 4, 2020CC BY-NC 2.0 DEED
Through the descriptions and dialogues of these new affective environments grounded in sound, some users might feel a sense of camaraderie, connection, and affinity to these novel experiences and how it relates or compares to one’s own (or the sense of not being so alone), especially in the personalized, diaristic, idiosyncratic tone in the short text extracts and longer interview forms. According to Tizian Zumthurm and Stefan Krebs (2022), digital spaces enable this type of “self-affirmation: by contributing and following the contributions of others, users are assured that they are not alone in whatever they experienced” (492). They also point out that as a result crowdsourced archives, particularly related to traumatic events, have a curative function.
Although the PSA may not be what likely comes to mind when thinking about a digital sound archive, presumably composed exclusively of musical or field recordings, it provides an entry point into a confluence of concerns to grapple with some of the key questions and issues related to sound, intimacy, and affect during the pandemic. In particular, diversity in form – between short excerpts and lengthier conversations, creates different engagement options for users based on preference and capacity (quick snapshots of the sonic changes in daily life or deeper explorations concerning sonic worlds), and across sensory inputs. Moreover, interviews add an oral history aspect to the project, which some scholars argue is more empowering and intimate than other modes of telling or sharing history. As a historically feminist practice, oral history has the potential to expose ignored topics and present diversified perspectives on traumatic pasts (like the 1918 pandemic) which is also especially important considering the research areas and professional backgrounds among the interview experts.
#StayHomeSounds is part of a larger project led by UK-based sound artist Stuart Fowkes, who created Cities and Memories in 2014, which Milena Droumeva describes as a “one-of-a-kind sonic portal dedicated to the exploration of place, sound and memory” (147). The website boasts being the largest sound project in the world with over 5000 sound recordings from over 1000 different contributors across 100 territories worldwide. It encompasses field recordings, sound art, and sound mapping, and each location features two sounds: the original field recording of that place and a reimagined sound that presents that place and time as somewhere, something else. The listener can explore sites through their actual sounds or the reimagined versions, flipping between the two different sound worlds. #StayHomeSounds is one of the latest ongoing sub-projects on the site and it is a collection of recordings during the pandemic from all over the world mostly done using cell phone recordings.
Bari lockdown sound recorded by Roberto Lippolis.
Although there is a wide range of quality and content, #StayHomeSounds offers a glimpse into the everyday sonic realities of quarantine life that cut across geography, life, and circumstances. The immensely mundane soundscapes and sheer multitude of recordings across cities and regions allows us to listen comparatively and try to notice the striking sonic cultures of different places even in lockdown. Those submitting sounds are required to provide a reflective text, and an elective representative image, to accompany their recording which details the changes in the soundscape as well as any a/effects that change has produced on other aspects of life.
An entry from Vancouver reads “Their chorus runs day and night and is a most pleasant soundtrack to both fall asleep with and wake up to. In this clip the background birds have joined in to add their avian melody to the amphibian bass line.”
Another from the Greek island of Crete, “after a heavy rain last night, the chirping of the birds woke me up this morning. It was such a powerful sound, like waking up from a sweet dream or a bad nightmare. I think that due to quarantine measures the nature’s sounds are more clear than even before.”
Athens, Greece lockdown sound recorded by Stamatis Mitrou.
In New Orleans, “I’m thankful for my quiet spot out here on the edges of town, but I worry about how the city can recover and for all those sick, out of work, or unable to stay home.”
New Orleans lockdown sounds recorded by Elizabeth Joan Kelly.
The objectives of the project, as described by Fowkes in the online text, are largely affective or affectively oriented, that is, to establish a sense of connection in the present, “how it feels at this unique moment,” by being able to discover new relations to place, to others, and to our sentient selves, through these sonic recordings and texts. In an interview with the Thomson Reuters Foundation, Fowkes said, “(You can) see what other people are hearing around the world and also read their stories and see that actually people are feeling similarly… hopefully that helps to make us feel a little bit more connected.” The breadth of contributions in terms of different locations and number of entries helps build this sense of connection, increasing the possibility of similar experiences to be seen and heard.
Lockdown sound from Lagos, Nigeria recorded by Ibukun Sunday.
By attending to personal struggles, observations, and speculations in relation to sound, these two digital sound archival projects gesture towards the intersections of intimacy, memory, and world-building, and alleviate and mediate some of the dominant and pervading affects that marked lockdown and remote life. In undertaking this project, I found pleasure in the informality of the responses and both the fresh insights and shared resonances, creating an experience that was jointly intimate (feeling seen and validated) and expansive (an opening to alternative experiences). In cultivating openness and a space for difference, and making the reflections and recordings publicly available, so that we can listen together but apart, the projects cultivate new forms of intimacy, empathy, collectivity, and nostalgia.
Dhaka, Bangladesh lockdown sound recorded by youKnowWho.
But, of course, the potential affective experience with the entries and recordings is not a given, much like with any critical scholarly intervention or artwork that attempts to raise awareness (in this case, to both the grave and minute effects of the pandemic) and resist dominant narratives (that the pandemic is under-control, over, or effects only one’s respiratory system), there is no guarantee that the intended experience will transpire in every engagement, but the possibility to do so – to cultivate intimacy and world-building at a time of profound uncertainty and physical distance – is nonetheless still valuable. Much like the diversity in responses, undoubtedly there are varying degrees and types of resonances, perceptions, and impacts within each visit.
Using an open access, crowdsourced approach, the PSA and #soundsathome construct participatory, community archives, creating and remediating documents and recordings for collective access and engagement on behalf of a global community that underwent monumental change, disruption, and loss. Calling explicit attention to palpable sensory shifts and disruptions is a central way to track, record, and make sense of the immense changes in this historical moment, and to illuminate the inequalities in environments and experiences that have been exacerbated by the (lack of) responses by governments and policy. The very existence of these projects and their participation through listening marks a resistance to the discourses of a “return to normalcy” or that we are on the other end of the pandemic. Because affects live in the body and are not often considered as objects of knowledge, the ongoing presence, use, and discussion of these two projects amplifies the a/effects, and a resistance to the affective logic of the pandemic, that they seek to produce. By considering COVID as an unprecedented, deeply affective, traumatic event, these online spaces operate to archive this moment in time and its myriad sonic dimensions, bringing these affective worlds into dialogue through an intimate exchange and assemblage between different bodies, experiences, and locations.
—
Emily Collins is an interdisciplinary researcher, writer, educator, and PhD Candidate in Cinema and Media Studies at York University in Tkaronto (Toronto) whose work draws on sound studies, feminist theory, critical disability studies, and cultural theory to examine sonic social relations and materiality through entanglements of resistance and care within contemporary artworks and creative practices. As a cultural worker and active member in the arts community, Emily has worked at diverse film, visual arts, and digital media organizations, institutions, and research networks within Canada and abroad, including Archive/Counter-Archive, PUBLIC Journal, VUCAVU, Festival Scope (Paris), the Toronto International Film Festival, and the Walter Phillips Gallery at Banff Centre for Arts and Creativity.
—
REWIND! . . .If you liked this post, you may also dig:
“A focus on listening [with technology] shifts the idea of freedom of speech from having a platform of expression to having the possibility of communication” (K. Lacey)
One of the biggest social media event of the past decade, #metoo stands out as a pivotal shift in the future of gender relations. Despite its persistence since October 2017, #metoo is still under-theorized, and since its permutations generate countless hashtag sub-categories each passing week, making sense of it presents a conceptual quagmire. Tracing its history, identifying key moments, mapping its pro- and counter-currents present equally tough challenges to both data science and feminist scholars.
Meta-communication about #metoo abounds. Infographics and visualizations attempt to contain its organic growth into perceivable maps and charts; pop news media constantly report on its evolution in likes, counts, and retweets, as well as—and increasingly—in number of convictions, lawsuits, and reports. At the same time, #metoo has arguably created a discernible listening public in the way that Kate Lacey (2013) argues emerged with national radio: women’s stories have never been listened to with such wide reach and rapt attention.
The project I discuss here takes ‘hearing’ #metoo a step further into the auditory realm in the form of data sonification so as to to re-imagine an audience compelled to earwitness not just the scope but the emotional impact of women’s stories. Data sonification is a growing field, which from its inception has crossed between art and science. It involves a conceptual or semantic translation of data into relevant sonic parameters in a way that utilizes perceptual gestalts to convey information through sound.
Brady Marks and I created the #metoo sonification you’ll hear below by drawing from a public dataset spanning October 2017 to the early Spring of 2018 obtained from data.world. Individual tweets using the hashtag are sonified using female battle cries from video games; the number of retweets and followers forms a sort of swelling and contracting background vocal texture to represent the reach of each message. The dataset is then sped up anywhere between 10x to 1000x in order to represent perceivable ebbs and flows of the hashtag’s life over time. The deliberate aim in this design was to convey a different sensibility of social media content, one that demands emotional and intellectual attention over a duration of time. Given Twitter’s visual zeitgeist whereby individual tweets are perceived at a glance and quickly become lost in the noise of the platform the affective attitude towards “contagious” events becomes arguably impersonal. A sonification such as this asks the listener to spend 30 minutes listening to 1 month of #metoo: something impossible to achieve on the actual platform, or in a single visualization. The aim, then, is to interrupt social media’s habitual and disposable engagements with pressing civic debates.
A critique of big data visualization
To date, there have been more than 19 million #MeToo tweets from over 85 countries; on Facebook more than 24 million people participated in the conversation by posting, reacting, and commenting over 77 million times since October 15, 2017. In a global information society ‘big data’ is translated into creative infographics in order to simultaneously educate an overwhelmed public and elicit urgency and accord for political action. Yet ideological and political considerations around the design of visual information have lagged behind enthusiasm for making data ‘easy to understand’. At the other end of the spectrum, social media delivers personalized micro-trends directly and in real time to always-mobile users, reinforcing their information silos (Rambukkana 2015). Between these extremes, the mechanisms by which relevant local, marginalized or emergent issues come to be communicated to the wider public are constrained.
Within the Western canon of sound studies “constitutive technicities” (Gallope 2011) or what Sterne calls “perceptual technics” embody historically situated ways of listening that center technology as a co-defining factor in our relationship with sound. Within this frame, media sociologist Kate Lacey traces the emergence of the modern listening public through the history of radio. Using the metaphor of ‘listening in” and “listening out,” Lacey reframes media citizenship by pointing out that listening is a cultural as well as a perceptual act with defined political dimensions:
Listening out is the practice of being open to the multiplicity of texts and voices and thinking of texts in the context of and in relation to a difference and how they resonate across time and in different spaces. But at the same time, it is the practice and experience of living in a media age that produces and heightens the requirement, the context, the responsibilities and the possibilities of listening out (198)
According to Lacey, a focus on listening instead of spectatorship challenges the implicit active/passive dualism of civic participation in Western contexts. More importantly, she argues, we need to move away from the notion of “giving voice” and instead create meaningful possibilities to listen, in a political sense. Data sonification doesn’t so much ‘give voice to the voiceless’ but creates a novel relationship to perceiving larger patterns and movements.
Our interactions with media, therefore, are always already presumptive of particular dialogical relations. Every speech act, every message implies a listening audience that will resonate understanding. In other words, how are we already listening in to #metoo? How and why might data sonification enable us to “listen out” for it instead? In order to get a different hearing, what should #metoo sound like?
Sonifying #metoo: the battle cries of gender-based violence
It is unrealistic to expect that your everyday person will read large archives of testimony on sexual harassment and gender-based violence. Because of their massive scale, archives of #metoo testimony pose a significant challenge to the possibility for meaningful communication around this issue. Essentially drowning each other out, individual voices remain unheard in the zeitgeist of media platforms that automates quantification while speeding up engagement with individual contributions. To reaffirm the importance of voice would mean to reaffirm inter-subjectivity and to recognize polyphony as an “existential position of humanity” (Ihde 2007, 178). This was the problem to sonify here: how to retain individual voices while creating the possibility for listening to the whole issue at hand. Inspired by the idea of listening out, myself and artist collaborator Brady Marks set out to sonify #metoo as a way of eliciting the possibility for a new listening public.
Sophitia Alexandra of the Soul Calibur franchise, close up from Flickr User Ngo Quang Minh‘s image from Soul Calibur III (CC BY-SA 2.0)
The #metoo sonification project intersected deeply with my work on the female voice in videogames. My choice to use a mixed selection of battle cry samples from Soul Calibur, an arcade fighting game, was intuitive. Battle cries are pre-recorded banks of combat sounds that video game characters perform in the course of the story. Instances of #metoo on Twitter presumably represent the experiences of individual women, pumping a virtual fist in the air, no longer silent about the realities of gender-based violence. So hearing #metoo posts as battle cries of powerful game heroines made sense to me. But it’s the meta layers of meaning that are even more intuitive: as I’ve discussed elsewhere, female battle cries are notoriously gendered and sexualized. Listening to a reel of sampled battle cries is almost indistinguishable from listening to a pornographic soundscape. Abstracted in this sonification, away from the cartoonish hyper-reality of a game world, these voices are even more eerie, giving almost physical substance to the subject matter of #metoo. Just as the female voice in media secretly fulfils the furtive desires of the “neglected erogenous zone” of the ear (Pettman 2017, 17), #metoo is an embodiment of the conflation of sex with consent: the basis of what we now call ‘rape culture.’
Sonifying real-time data such as Twitter presents not only semantic (how should it sound like) but also time-scale challenges. If we are to sonify a month – e.g. the month of November 2017 (just weeks after the explosion of #metoo) – but we don’t want to spend a month listening, then that involves some conceptual time-scaling. Time-scaling means speeding up instances that already happen multiple times a second on a platform as instantaneous and global as Twitter. Below are samples of three different sonifications of #metoo data, following different moments in the initial explosion of the hashtag and rendered at different time compressions. Listen to them one at a time and note your sensual and emotive experience of tweets closer to real-time playback, compared to the audible patterns that emerge from compressing longer periods of time inside the same length audio file. You might find that the density is different. Closer to real-time the battle cries are more distinctive, while at higher time compressions what emerges instead is an expanding and contracting polyphonic texture.
Vocalizations of female pleasure/affect, video game battle cries already have a special relationship to technologies of audio sampling and digital reproduction as Corbett & Kapsalis describe in “Aural sex: the female orgasm in popular sound.” This means that the perceptual technics involved in listening to recorded female voices are already coded with sexual connotations. Battle cries in games are purposely exaggerated so as to carry the bulk of emotional content in the game’s experiential matrix. Roland Barthes’ notion of the “grain of the voice”—the presence of the body in (singing) voice—is frequently evoked in describing the substantive role that game voices play in the construction of game world immersion and realism. In the #metoo sonification, I decontextualize the grain of the voice—there are no visual images, narrative, or gameplay; the battle cries are also acousmatic, in that there are no bodies visually represented from which these sounds emanate.
The battle cry in this #metoo sonification is the ultimate disembodied voice, resisting what Kaja Silverman (1988) calls the “norm of synchronization” with a female body in The Acoustic Mirror (83). As acousmatic voices, these battle cries could be said to exist on a different conceptual and perceptual plane, “disturbing the taxonomies upon which patriarchy depends,” to quote Dominic Pettman in Sonic Intimacy. (22). In other words, the sounds exist in a boundary space between combat sounds and orgasmic sounds highlighting for the listener the dissonance between the supposed empowerment of ‘speaking out’ within a culture that remains staunchly set up to sexualize women; something one can hardly ignore given the media’s reserved treatment of #metoo.
“Princess Zelda’s New Mouth” by Flickr UserMouthGuy2013, Public Domain
Liberated from the game world these voices now speak for themselves in the #metoo sonification, their sensuality all the more hyper-real. The player has no control here, as the battle cries are not linked to specific game actions, rather they are synchronized autonomously to instances of #metoo confessionals. In fact, the density of the sonification as time speeds up will overwhelm listeners with its boundlessness; echoing how contemporary media treats the sounds of the female orgasm as a renewable and inexhaustible resource, even as reports of sexual harassment and gender-based violence continue to pile on in 2021. Yet we intend that the subject matter resists pleasure, rendering the sonic experience traumatic as the chilling realization sets in that listeners are hailed to accountability by #metoo. The experience should instead be unsettling, impactful, grotesque, and deeply embodied.
Concluding remarks
Listening both metaphorically and literally goes to the very heart of questions to do with the politics and experience of living and communicating in the media age. In her paper on the sonic geographies of the voice, AM Kanngieser notes in “A Sonic Geography of Voice“: “The voice, in its expression of affective and ethico-political forces, creates worlds” (337). It is not just in the grain but in the enunciation that battle cries find their political significance in this sonification. As the hyper-real gasps and moans of game heroines animate individual moments of #metoo the codification of cartoonish voices resists being subconsciously “absorbed into the dialogic exchange” (342) of habitual media consumption. Listening to the sonification is instead an experience of re-coding the voice, reconfiguring the embedded meanings of game sound to a new and contradictory context: a space that challenges neoliberal appropriations of radical communication and discourse (348). This is not data sonification that delights the listener or simply grants them access to ‘information’ in a different format; rather it calls on the listener to de-normalize their received technicity and perceptions and to connect to the emotional inter-subjectivity of this call to action.
Most importantly, the #metoo sonification invites the auditeur to listen in, to take an active role in the reconfiguration of meanings and absorb their political dimensions. These are the stories of #metoo; these are the voices of women, of men, of marginalized peoples, emerging from the zeitgeist of Twitter to ask us to earwitness gender-based violence. We are a new listening public, wanting and needing to create new worlds. A critical bandwidth is the smallest perceivable unit of auditory change, in psychology terms. This sonification begs the question, how many battle cries will it take for us to end gender-based violence by fostering equitable worlds?
Milena Droumeva is an Assistant Professor and the Glenfraser Endowed Professor in Sound Studies at Simon Fraser University specializing in mobile media, sound studies, gender, and sensory ethnography. Milena has worked extensively in educational research on game-based learning and computational literacy, formerly as a post-doctoral fellow at the Institute for Research on Digital Learning at York University. Milena has a background in acoustic ecology and works across the fields of urban soundscape research, sonification for public engagement, as well as gender and sound in video games. Current research projects include sound ethnographies of the city (livable soundscapes), mobile curation, critical soundmapping, and sensory ethnography. Check out Milena’s Story Map, “Soundscapes of Productivity” about coffee shop soundscapes as the office ambience of the creative economy freelance workers.
Sounding Out! 
Recent Comments