Part of this surge in scholarly attention and artistic projects is premised on the idea that sound and sound recordings are important additions to cultural heritage in documenting histories and personal evidence, and yet, they are often viewed as supplementary adjuncts to more physical or visual archival artefacts. This subjugation to the primacy of the visual extends to the arts, humanities, and social sciences, but this is beginning to change as scholars across these fields increasingly argue that sound (with its attendant listening) is an especially critical medium for cultivating different modes of attention, forging affective relations, producing alternative knowledges, revealing hidden narratives, and attuning to neglected pasts.
Two digital sound archival projects created in response to the pandemic, The Pandemic SensoryArchive and#StayHomeSounds, are especially instructive in thinking about sound as a key medium for engaging with the monumental a/effects of the present and as important contributions to cultural history. Like other pandemic digital sound archival projects, these two projects sought to document the present for the future – creating a “past” in real time, based on the underlying assumption that sound – as a material-discursive apparatus – can offer particularly generative possibilities in this context. The methods, scope, and presentation design employed by these two web-based archival platforms generates a sense of intimacy, proximity, andcollectivity in otherwise surreal, secluded, uncertain, detached, and disconnected situations, much like in radio and podcasting though in this case with different infrastructure and interactivity.
In these online, mediated spaces where worlds intensely collide and conflate, and users become flattened out and disembodied, new configurations of intimacy, subjectivities, and world-building emerge through alternative forms of affective archival engagement. This was (and still is) particularly important and complex during COVID-19, which is marked not only by a series of indefinite lockdowns and uneven distribution of intervention measures, but an affective logic whereby life is completely reconfigured and capacities within the world are diminished and redistributed. Part of making mass sound archives usable relies on the medium for circulation, the presentation for users, and what user participation empowers for these living histories, as Fabiola Hanna makes clear. These two projects generate what Hanna identifies as a particular orientation in digital humanities projects through a politics of listening that necessitates an active mode of participation that is not simply one-directional but a two-way engagement.
The Archive of Intimacy, later renamed The Pandemic Sensory Archive (PSA), wascreated by professors William Tullett (Associate Professor in Sensory History, Anglia Ruskin University, U.K.) and Hannah McCann (Senior Lecturer in Cultural Studies, University of Melbourne, Australia) with the goal of exploring the senses through a digital platform and to act as an open data bank of contributions from the public. Their open call for contributions asks for a response that considers two questions: What smells, sights, sounds, touch, and/or tastes do you associate with the pandemic? Has your experiences of the senses (smell/sight/hearing/touching/taste) changed at all as a result of the pandemic? Contributors are then asked to drop a pin on the map (though this is not a typical Google Maps rendition of a specific locale, but a sort of simplified graphic representation of sensory input/output waves) and follow the prompts to anonymously submit. The digital map is divided among the five senses and features the entries that contain a title and brief response, such as:
Quiet where there should not be quiet: “Being in my flat in the centre of town on my own with nobody else around on a Saturday. Everything weirdly, eerily, quiet.”
Birdsong: “Hearing birdsong in the garden having not noticed it before, no longer drowned out.”
You’re on mute! “Did the conversation in meetings become less robust as we all sit there on mute, politely waiting for our turn to speak?”
Less sound, more sound: “Blissfully quiet at night as curfew curtails the normal constant traffic roars, far more voices in the early morning and through the day as people‘exercise together’ to socialize in the park”
Complimenting this map, which may seem limited in scope but allows users to engage without having to sift through an overwhelming amount of content, is the Sound category page, where four interviews are embedded with “sensory experts on sound during the Covid-19 pandemic,” including Shoshana Rosenberg, Andrew Mitchell, Martin Stewart, and Stephen Sullivan. One interview considers how the pandemic clarified the immensely relational dimension of artistic sound practice and that the lack of access to intimacy during lockdown instigated a radical reformatting and questioning of what it means, more broadly and now, to be intimate and close in creating sound art. For them, what the pandemic spelled out is that intimacy is fragile and valuable, and that this delicate balance and fluctuating ratio has come to the fore during this time.
The initial designation for the platform, “the archive of intimacy,” is worth meditating on to consider the particular forms of intimacy in this context, perhaps through Lauren Berlant’s “intimate publics” – a concept that captures the affective and collective dimensions of intimacy among strangers. The notion lends itself to understanding the mediated social intimacy in these spaces and the different affective experiences they invite in varying capacities through sound.The connection between imagined publics and community through sound has, of course, been conceptualized by scholars who do historical work on radio and podcast studies, but it can also be extended to these digital, affective, pandemic sound archives. Evidenced in the submission prompts and interview data, the emphasis on the distinct shifts and palpable changes resulting from this new situation, and its accompanying affective logic, can be read as a strategy for cultivating intimacy and connection because attending to these changes may render their intensities as less alarming.
“Listening” by Flickr User Silvia Siri, April 4, 2020CC BY-NC 2.0 DEED
Through the descriptions and dialogues of these new affective environments grounded in sound, some users might feel a sense of camaraderie, connection, and affinity to these novel experiences and how it relates or compares to one’s own (or the sense of not being so alone), especially in the personalized, diaristic, idiosyncratic tone in the short text extracts and longer interview forms. According to Tizian Zumthurm and Stefan Krebs (2022), digital spaces enable this type of “self-affirmation: by contributing and following the contributions of others, users are assured that they are not alone in whatever they experienced” (492). They also point out that as a result crowdsourced archives, particularly related to traumatic events, have a curative function.
Although the PSA may not be what likely comes to mind when thinking about a digital sound archive, presumably composed exclusively of musical or field recordings, it provides an entry point into a confluence of concerns to grapple with some of the key questions and issues related to sound, intimacy, and affect during the pandemic. In particular, diversity in form – between short excerpts and lengthier conversations, creates different engagement options for users based on preference and capacity (quick snapshots of the sonic changes in daily life or deeper explorations concerning sonic worlds), and across sensory inputs. Moreover, interviews add an oral history aspect to the project, which some scholars argue is more empowering and intimate than other modes of telling or sharing history. As a historically feminist practice, oral history has the potential to expose ignored topics and present diversified perspectives on traumatic pasts (like the 1918 pandemic) which is also especially important considering the research areas and professional backgrounds among the interview experts.
#StayHomeSounds is part of a larger project led by UK-based sound artist Stuart Fowkes, who created Cities and Memories in 2014, which Milena Droumeva describes as a “one-of-a-kind sonic portal dedicated to the exploration of place, sound and memory” (147). The website boasts being the largest sound project in the world with over 5000 sound recordings from over 1000 different contributors across 100 territories worldwide. It encompasses field recordings, sound art, and sound mapping, and each location features two sounds: the original field recording of that place and a reimagined sound that presents that place and time as somewhere, something else. The listener can explore sites through their actual sounds or the reimagined versions, flipping between the two different sound worlds. #StayHomeSounds is one of the latest ongoing sub-projects on the site and it is a collection of recordings during the pandemic from all over the world mostly done using cell phone recordings.
Bari lockdown sound recorded by Roberto Lippolis.
Although there is a wide range of quality and content, #StayHomeSounds offers a glimpse into the everyday sonic realities of quarantine life that cut across geography, life, and circumstances. The immensely mundane soundscapes and sheer multitude of recordings across cities and regions allows us to listen comparatively and try to notice the striking sonic cultures of different places even in lockdown. Those submitting sounds are required to provide a reflective text, and an elective representative image, to accompany their recording which details the changes in the soundscape as well as any a/effects that change has produced on other aspects of life.
An entry from Vancouver reads “Their chorus runs day and night and is a most pleasant soundtrack to both fall asleep with and wake up to. In this clip the background birds have joined in to add their avian melody to the amphibian bass line.”
Another from the Greek island of Crete, “after a heavy rain last night, the chirping of the birds woke me up this morning. It was such a powerful sound, like waking up from a sweet dream or a bad nightmare. I think that due to quarantine measures the nature’s sounds are more clear than even before.”
Athens, Greece lockdown sound recorded by Stamatis Mitrou.
In New Orleans, “I’m thankful for my quiet spot out here on the edges of town, but I worry about how the city can recover and for all those sick, out of work, or unable to stay home.”
New Orleans lockdown sounds recorded by Elizabeth Joan Kelly.
The objectives of the project, as described by Fowkes in the online text, are largely affective or affectively oriented, that is, to establish a sense of connection in the present, “how it feels at this unique moment,” by being able to discover new relations to place, to others, and to our sentient selves, through these sonic recordings and texts. In an interview with the Thomson Reuters Foundation, Fowkes said, “(You can) see what other people are hearing around the world and also read their stories and see that actually people are feeling similarly… hopefully that helps to make us feel a little bit more connected.” The breadth of contributions in terms of different locations and number of entries helps build this sense of connection, increasing the possibility of similar experiences to be seen and heard.
Lockdown sound from Lagos, Nigeria recorded by Ibukun Sunday.
By attending to personal struggles, observations, and speculations in relation to sound, these two digital sound archival projects gesture towards the intersections of intimacy, memory, and world-building, and alleviate and mediate some of the dominant and pervading affects that marked lockdown and remote life. In undertaking this project, I found pleasure in the informality of the responses and both the fresh insights and shared resonances, creating an experience that was jointly intimate (feeling seen and validated) and expansive (an opening to alternative experiences). In cultivating openness and a space for difference, and making the reflections and recordings publicly available, so that we can listen together but apart, the projects cultivate new forms of intimacy, empathy, collectivity, and nostalgia.
Dhaka, Bangladesh lockdown sound recorded by youKnowWho.
But, of course, the potential affective experience with the entries and recordings is not a given, much like with any critical scholarly intervention or artwork that attempts to raise awareness (in this case, to both the grave and minute effects of the pandemic) and resist dominant narratives (that the pandemic is under-control, over, or effects only one’s respiratory system), there is no guarantee that the intended experience will transpire in every engagement, but the possibility to do so – to cultivate intimacy and world-building at a time of profound uncertainty and physical distance – is nonetheless still valuable. Much like the diversity in responses, undoubtedly there are varying degrees and types of resonances, perceptions, and impacts within each visit.
Using an open access, crowdsourced approach, the PSA and #soundsathome construct participatory, community archives, creating and remediating documents and recordings for collective access and engagement on behalf of a global community that underwent monumental change, disruption, and loss. Calling explicit attention to palpable sensory shifts and disruptions is a central way to track, record, and make sense of the immense changes in this historical moment, and to illuminate the inequalities in environments and experiences that have been exacerbated by the (lack of) responses by governments and policy. The very existence of these projects and their participation through listening marks a resistance to the discourses of a “return to normalcy” or that we are on the other end of the pandemic. Because affects live in the body and are not often considered as objects of knowledge, the ongoing presence, use, and discussion of these two projects amplifies the a/effects, and a resistance to the affective logic of the pandemic, that they seek to produce. By considering COVID as an unprecedented, deeply affective, traumatic event, these online spaces operate to archive this moment in time and its myriad sonic dimensions, bringing these affective worlds into dialogue through an intimate exchange and assemblage between different bodies, experiences, and locations.
—
Emily Collins is an interdisciplinary researcher, writer, educator, and PhD Candidate in Cinema and Media Studies at York University in Tkaronto (Toronto) whose work draws on sound studies, feminist theory, critical disability studies, and cultural theory to examine sonic social relations and materiality through entanglements of resistance and care within contemporary artworks and creative practices. As a cultural worker and active member in the arts community, Emily has worked at diverse film, visual arts, and digital media organizations, institutions, and research networks within Canada and abroad, including Archive/Counter-Archive, PUBLIC Journal, VUCAVU, Festival Scope (Paris), the Toronto International Film Festival, and the Walter Phillips Gallery at Banff Centre for Arts and Creativity.
—
REWIND! . . .If you liked this post, you may also dig:
Still from artist’s mock-up of The Cyborg’s Prosody(2022-present), copyright Dorothy R. Santos
In summer 2021, sound artist, engineer, musician, and educatorJohann Diedrick convened a panel at the intersection of racial bias, listening, and AI technology at Pioneerworks in Brooklyn, NY. Diedrick, 2021 Mozilla Creative Media award recipient and creator of such works as Dark Matters, is currently working on identifying the origins of racial bias in voice interface systems. Dark Matters, according to Squeaky Wheel, “exposes the absence of Black speech in the datasets used to train voice interface systems in consumer artificial intelligence products such as Alexa and Siri. Utilizing 3D modeling, sound, and storytelling, the project challenges our communities to grapple with racism and inequity through speech and the spoken word, and how AI systems underserve Black communities.” And now, he’s working with SO! as guest editor for this series (along with ed-in-chief JS!). It kicked off with AminaAbbas-Nazari’s post, helping us to understand how Speech AI systems operate from a very limiting set of assumptions about the human voice. Then, Golden Owenstook a deep historical dive into the racialized sound of servitude in America and how this impacts Intelligent Virtual Assistants. Last week, Michelle Pfeifer explored how some nations are attempting to draw sonic borders, despite the fact that voices are not passports. Today, Dorothy R. Santos wraps up the series with a meditation on what we lose due to the intensified surveilling, tracking, and modulation of our voices. [To read the full series, click here] –JS
—
Still from artist’s mock-up of The Cyborg’s Prosody(2022-present), copyright Dorothy R. Santos
In 2010, science fiction writer Charles Yu wrote a story titled “Standard Loneliness Package,” where emotions are outsourced to another human being. While Yu’s story is a literal depiction, albeit fictitious, of what might be entailed and the considerations that need to be made of emotional labor, it was published a year prior to Apple introducing Siri as its official voice assistant for the iPhone. Humans are not meant to be viewed as a type of technology, yet capitalist and neoliberal logics continue to turn to technology as a solution to erase or filter what is least desirable even if that means the literal modification of voice, accent, and language. What do these actions do to the body at risk of severe fragmentation and compartmentalization?
I weep.
I wail.
I gnash my teeth.
Underneath it all, I am smiling. I am giggling.
I am at a funeral. My client’s heart aches, and inside of it is my heart, not aching, the opposite of aching—doing that, whatever it is.
Yu sets the scene by providing specific examples of feelings of pain and loss that might be handed off to an agent who absorbs the feelings. He shows us, in one way, what a world might look and feel like if we were to go to the extreme of eradicating and off loading our most vulnerable moments to an agent or technician meant to take on this labor. Although written well over a decade ago, its prescient take on the future of feelings wasn’t too far off from where we find ourselves in 2023. How does the voice play into these connections between Yu’s story and what we’re facing in the technological age of voice recognition, speech synthesis, and assistive technologies? How might we re-imagine having the choice to displace our burdens onto another being or entity? Taking a cue from Yu’s story, technologies are being created that pull at the heartstrings of our memories and nostalgia. Yet what happens when we are thrust into a perpetual state of grieving and loss?
Humans are made to forget. Unlike a computer, we are fed information required for our survival. When it comes to language and expression, it is often a stochastic process of figuring out for whom we speak and who is on the receiving end of our communication and speech. Artist and scholar Fabiola Hanna believes polyvocality necessitates an active and engaged listener, which then produces our memories. Machines have become the listeners to our sonic landscapes as well as capturers, surveyors, and documents of our utterances.
The past few years may have been a remarkable advancement in voice tech with companies such as Amazon and Sanas AI, a voice recognition platform that allows a user to apply a vocal filter onto any human voice, with a discernible accent, that transforms the speech into Standard American English. Yet their hopes for accent elimination and voice mimicry foreshadow a future of design without justice and software development sans cultural and societal considerations, something I work through in my artwork in progress, The Cyborg’s Prosody (2022-present).
The Cyborg’s Prosody is an interactive web-based artwork (optimized for mobile) that requires participants to read five vignettes that increasingly incorporate Tagalog words and phrases that must be repeated by the player. The work serves as a type of parody, as an “accent induction school” — providing a decolonial method of exploring how language and accents are learned and preserved. The work is a response to the creation of accent reduction schools and coaches in the Philippines. Originally, the work was meant to be a satire and parody of these types of services, but shifted into a docu-poetic work of my mother’s immigration story and learning and becoming fluent in American English.
Still from artist’s mock-up of The Cyborg’s Prosody(2022-present), copyright Dorothy R. Santos
Even though English is a compulsory language in the Philippines, it is a language learned within the parameters of an educational institution and not common speech outside of schools and businesses. From the call center agents hired at Vox Elite, a BPO company based in the Philippines, to a Filipino immigrant navigating her way through a new environment, the embodiment of language became apparent throughout the stages of research and the creative interventions of the past few years.
In Fall 2022, I gave an artist talk about The Cyborg’s Prosody to a room of predominantly older, white, cisgender male engineers and computer scientists. Apparently, my work caused a stir in one of the conversations between a small group of attendees. A couple of the engineers chose to not address me directly, but I overheard a debate between guests with one of the engineers asking, “What is her project supposed to teach me about prosody? What does mimicking her mom teach me?” He became offended by the prospect of a work that de-centered his language, accent, and what was most familiar to him.The Cyborg’s Prosody is a reversal of what is perceived as a foreign accented voice in the United States into a performance for both the cyborg and the player. I introduce the term western vocal drag to convey the caricature of gender through drag performance, which is apropos and akin to the vocal affect many non-western speakers effectuate in their speech.
The concept of western vocal drag became a way for me to understand and contemplate the ways that language becomes performative through its embodiment. Whether it is learning American vernacular to the complex tenses that give meaning to speech acts, there is always a failure or queering of language when a particular affect and accent is emphasized in one’s speech. The delivery of speech acts is contingent upon setting, cultural context, and whether or not there is a type of transaction occurring between the speaker and listener. In terms of enhancement of speech and accent to conform to a dominant language in the workplace and in relation to global linguistic capitalism, scholar Vijay A. Ramjattan states in that there is no such thing as accent elimination or even reduction. Rather, an accent is modified. The stakes are high when taking into consideration the marketing and branding of software such as Sanas AI that proposes an erasure of non-dominant foreign accented voices.
The biggest fear related to the use of artificial intelligence within voice recognition and speech technologies is the return to a Standard American English (and accent) preferred by a general public that ceases to address, acknowledge, and care about linguistic diversity and inclusion. The technology itself has been marketed as a way for corporations and the BPO companies they hire to mind the mental health of the call center agents subjected to racism and xenophobia just by the mere sound of their voice and accent. The challenge, moving forward, is reversing the need to serve the western world.
A transorality or vocality presents itself when thinking about scholar April Baker-Bell’s work Black Linguistic Consciousness. When Black youth are taught and required to speak with what is considered Standard American English, this presents a type of disciplining that perpetuates raciolinguistic ideologies of what is acceptable speech. Baker-Bell focuses on an antiracist linguistic pedagogy where Black youth are encouraged to express themselves as a shift towards understanding linguistic bias. Deeply inspired by her scholarship, I started to wonder about the process for working on how to begin framing language learning in terms of a multi-consciousness that includes cultural context and affect as a way to bridge gaps in understanding.
Still from artist’s mock-up of The Cyborg’s Prosody(2022-present), copyright Dorothy R. Santos
Or, let’s re-think this concept or idea that a bad version of English exists. As Cathy Park Hong brilliantly states, “Bad English is my heritage…To other English is to make audible the imperial power sewn into the language, to slit English open so its dark histories slide out.” It is necessary for us all to reconfigure our perceptions of how we listen and communicate that perpetuates seeking familiarity and agreement, but encourages respecting and honoring our differences.
—
Featured Image: Still from artist’s mock-up of The Cyborg’s Prosody(2022-present), copyright Dorothy R. Santos
—
Dorothy R. Santos, Ph.D. (she/they) is a Filipino American storyteller, poet, artist, and scholar whose academic and research interests include feminist media histories, critical medical anthropology, computational media, technology, race, and ethics. She has her Ph.D. in Film and Digital Media with a designated emphasis in Computational Media from the University of California, Santa Cruz and was a Eugene V. Cota-Robles fellow. She received her Master’s degree in Visual and Critical Studies at the California College of the Arts and holds Bachelor’s degrees in Philosophy and Psychology from the University of San Francisco. Her work has been exhibited at Ars Electronica, Rewire Festival, Fort Mason Center for Arts & Culture, Yerba Buena Center for the Arts, and the GLBT Historical Society.
Her writing appears in art21, Art in America, Ars Technica, Hyperallergic, Rhizome, Slate, and Vice Motherboard. Her essay “Materiality to Machines: Manufacturing the Organic and Hypotheses for Future Imaginings,” was published in The Routledge Companion to Biology in Art and Architecture. She is a co-founder of REFRESH, a politically-engaged art and curatorial collective and serves as a member of the Board of Directors for the Processing Foundation. In 2022, she received the Mozilla Creative Media Award for her interactive, docu-poetics work The Cyborg’s Prosody (2022). She serves as an advisory board member for POWRPLNT, slash arts, and House of Alegria.
—
REWIND! . . .If you liked this post, you may also dig:
Sounding Out! 
Recent Comments