Part of this surge in scholarly attention and artistic projects is premised on the idea that sound and sound recordings are important additions to cultural heritage in documenting histories and personal evidence, and yet, they are often viewed as supplementary adjuncts to more physical or visual archival artefacts. This subjugation to the primacy of the visual extends to the arts, humanities, and social sciences, but this is beginning to change as scholars across these fields increasingly argue that sound (with its attendant listening) is an especially critical medium for cultivating different modes of attention, forging affective relations, producing alternative knowledges, revealing hidden narratives, and attuning to neglected pasts.
Two digital sound archival projects created in response to the pandemic, The Pandemic SensoryArchive and#StayHomeSounds, are especially instructive in thinking about sound as a key medium for engaging with the monumental a/effects of the present and as important contributions to cultural history. Like other pandemic digital sound archival projects, these two projects sought to document the present for the future – creating a “past” in real time, based on the underlying assumption that sound – as a material-discursive apparatus – can offer particularly generative possibilities in this context. The methods, scope, and presentation design employed by these two web-based archival platforms generates a sense of intimacy, proximity, andcollectivity in otherwise surreal, secluded, uncertain, detached, and disconnected situations, much like in radio and podcasting though in this case with different infrastructure and interactivity.
In these online, mediated spaces where worlds intensely collide and conflate, and users become flattened out and disembodied, new configurations of intimacy, subjectivities, and world-building emerge through alternative forms of affective archival engagement. This was (and still is) particularly important and complex during COVID-19, which is marked not only by a series of indefinite lockdowns and uneven distribution of intervention measures, but an affective logic whereby life is completely reconfigured and capacities within the world are diminished and redistributed. Part of making mass sound archives usable relies on the medium for circulation, the presentation for users, and what user participation empowers for these living histories, as Fabiola Hanna makes clear. These two projects generate what Hanna identifies as a particular orientation in digital humanities projects through a politics of listening that necessitates an active mode of participation that is not simply one-directional but a two-way engagement.
The Archive of Intimacy, later renamed The Pandemic Sensory Archive (PSA), wascreated by professors William Tullett (Associate Professor in Sensory History, Anglia Ruskin University, U.K.) and Hannah McCann (Senior Lecturer in Cultural Studies, University of Melbourne, Australia) with the goal of exploring the senses through a digital platform and to act as an open data bank of contributions from the public. Their open call for contributions asks for a response that considers two questions: What smells, sights, sounds, touch, and/or tastes do you associate with the pandemic? Has your experiences of the senses (smell/sight/hearing/touching/taste) changed at all as a result of the pandemic? Contributors are then asked to drop a pin on the map (though this is not a typical Google Maps rendition of a specific locale, but a sort of simplified graphic representation of sensory input/output waves) and follow the prompts to anonymously submit. The digital map is divided among the five senses and features the entries that contain a title and brief response, such as:
Quiet where there should not be quiet: “Being in my flat in the centre of town on my own with nobody else around on a Saturday. Everything weirdly, eerily, quiet.”
Birdsong: “Hearing birdsong in the garden having not noticed it before, no longer drowned out.”
You’re on mute! “Did the conversation in meetings become less robust as we all sit there on mute, politely waiting for our turn to speak?”
Less sound, more sound: “Blissfully quiet at night as curfew curtails the normal constant traffic roars, far more voices in the early morning and through the day as people‘exercise together’ to socialize in the park”
Complimenting this map, which may seem limited in scope but allows users to engage without having to sift through an overwhelming amount of content, is the Sound category page, where four interviews are embedded with “sensory experts on sound during the Covid-19 pandemic,” including Shoshana Rosenberg, Andrew Mitchell, Martin Stewart, and Stephen Sullivan. One interview considers how the pandemic clarified the immensely relational dimension of artistic sound practice and that the lack of access to intimacy during lockdown instigated a radical reformatting and questioning of what it means, more broadly and now, to be intimate and close in creating sound art. For them, what the pandemic spelled out is that intimacy is fragile and valuable, and that this delicate balance and fluctuating ratio has come to the fore during this time.
The initial designation for the platform, “the archive of intimacy,” is worth meditating on to consider the particular forms of intimacy in this context, perhaps through Lauren Berlant’s “intimate publics” – a concept that captures the affective and collective dimensions of intimacy among strangers. The notion lends itself to understanding the mediated social intimacy in these spaces and the different affective experiences they invite in varying capacities through sound.The connection between imagined publics and community through sound has, of course, been conceptualized by scholars who do historical work on radio and podcast studies, but it can also be extended to these digital, affective, pandemic sound archives. Evidenced in the submission prompts and interview data, the emphasis on the distinct shifts and palpable changes resulting from this new situation, and its accompanying affective logic, can be read as a strategy for cultivating intimacy and connection because attending to these changes may render their intensities as less alarming.
“Listening” by Flickr User Silvia Siri, April 4, 2020CC BY-NC 2.0 DEED
Through the descriptions and dialogues of these new affective environments grounded in sound, some users might feel a sense of camaraderie, connection, and affinity to these novel experiences and how it relates or compares to one’s own (or the sense of not being so alone), especially in the personalized, diaristic, idiosyncratic tone in the short text extracts and longer interview forms. According to Tizian Zumthurm and Stefan Krebs (2022), digital spaces enable this type of “self-affirmation: by contributing and following the contributions of others, users are assured that they are not alone in whatever they experienced” (492). They also point out that as a result crowdsourced archives, particularly related to traumatic events, have a curative function.
Although the PSA may not be what likely comes to mind when thinking about a digital sound archive, presumably composed exclusively of musical or field recordings, it provides an entry point into a confluence of concerns to grapple with some of the key questions and issues related to sound, intimacy, and affect during the pandemic. In particular, diversity in form – between short excerpts and lengthier conversations, creates different engagement options for users based on preference and capacity (quick snapshots of the sonic changes in daily life or deeper explorations concerning sonic worlds), and across sensory inputs. Moreover, interviews add an oral history aspect to the project, which some scholars argue is more empowering and intimate than other modes of telling or sharing history. As a historically feminist practice, oral history has the potential to expose ignored topics and present diversified perspectives on traumatic pasts (like the 1918 pandemic) which is also especially important considering the research areas and professional backgrounds among the interview experts.
#StayHomeSounds is part of a larger project led by UK-based sound artist Stuart Fowkes, who created Cities and Memories in 2014, which Milena Droumeva describes as a “one-of-a-kind sonic portal dedicated to the exploration of place, sound and memory” (147). The website boasts being the largest sound project in the world with over 5000 sound recordings from over 1000 different contributors across 100 territories worldwide. It encompasses field recordings, sound art, and sound mapping, and each location features two sounds: the original field recording of that place and a reimagined sound that presents that place and time as somewhere, something else. The listener can explore sites through their actual sounds or the reimagined versions, flipping between the two different sound worlds. #StayHomeSounds is one of the latest ongoing sub-projects on the site and it is a collection of recordings during the pandemic from all over the world mostly done using cell phone recordings.
Bari lockdown sound recorded by Roberto Lippolis.
Although there is a wide range of quality and content, #StayHomeSounds offers a glimpse into the everyday sonic realities of quarantine life that cut across geography, life, and circumstances. The immensely mundane soundscapes and sheer multitude of recordings across cities and regions allows us to listen comparatively and try to notice the striking sonic cultures of different places even in lockdown. Those submitting sounds are required to provide a reflective text, and an elective representative image, to accompany their recording which details the changes in the soundscape as well as any a/effects that change has produced on other aspects of life.
An entry from Vancouver reads “Their chorus runs day and night and is a most pleasant soundtrack to both fall asleep with and wake up to. In this clip the background birds have joined in to add their avian melody to the amphibian bass line.”
Another from the Greek island of Crete, “after a heavy rain last night, the chirping of the birds woke me up this morning. It was such a powerful sound, like waking up from a sweet dream or a bad nightmare. I think that due to quarantine measures the nature’s sounds are more clear than even before.”
Athens, Greece lockdown sound recorded by Stamatis Mitrou.
In New Orleans, “I’m thankful for my quiet spot out here on the edges of town, but I worry about how the city can recover and for all those sick, out of work, or unable to stay home.”
New Orleans lockdown sounds recorded by Elizabeth Joan Kelly.
The objectives of the project, as described by Fowkes in the online text, are largely affective or affectively oriented, that is, to establish a sense of connection in the present, “how it feels at this unique moment,” by being able to discover new relations to place, to others, and to our sentient selves, through these sonic recordings and texts. In an interview with the Thomson Reuters Foundation, Fowkes said, “(You can) see what other people are hearing around the world and also read their stories and see that actually people are feeling similarly… hopefully that helps to make us feel a little bit more connected.” The breadth of contributions in terms of different locations and number of entries helps build this sense of connection, increasing the possibility of similar experiences to be seen and heard.
Lockdown sound from Lagos, Nigeria recorded by Ibukun Sunday.
By attending to personal struggles, observations, and speculations in relation to sound, these two digital sound archival projects gesture towards the intersections of intimacy, memory, and world-building, and alleviate and mediate some of the dominant and pervading affects that marked lockdown and remote life. In undertaking this project, I found pleasure in the informality of the responses and both the fresh insights and shared resonances, creating an experience that was jointly intimate (feeling seen and validated) and expansive (an opening to alternative experiences). In cultivating openness and a space for difference, and making the reflections and recordings publicly available, so that we can listen together but apart, the projects cultivate new forms of intimacy, empathy, collectivity, and nostalgia.
Dhaka, Bangladesh lockdown sound recorded by youKnowWho.
But, of course, the potential affective experience with the entries and recordings is not a given, much like with any critical scholarly intervention or artwork that attempts to raise awareness (in this case, to both the grave and minute effects of the pandemic) and resist dominant narratives (that the pandemic is under-control, over, or effects only one’s respiratory system), there is no guarantee that the intended experience will transpire in every engagement, but the possibility to do so – to cultivate intimacy and world-building at a time of profound uncertainty and physical distance – is nonetheless still valuable. Much like the diversity in responses, undoubtedly there are varying degrees and types of resonances, perceptions, and impacts within each visit.
Using an open access, crowdsourced approach, the PSA and #soundsathome construct participatory, community archives, creating and remediating documents and recordings for collective access and engagement on behalf of a global community that underwent monumental change, disruption, and loss. Calling explicit attention to palpable sensory shifts and disruptions is a central way to track, record, and make sense of the immense changes in this historical moment, and to illuminate the inequalities in environments and experiences that have been exacerbated by the (lack of) responses by governments and policy. The very existence of these projects and their participation through listening marks a resistance to the discourses of a “return to normalcy” or that we are on the other end of the pandemic. Because affects live in the body and are not often considered as objects of knowledge, the ongoing presence, use, and discussion of these two projects amplifies the a/effects, and a resistance to the affective logic of the pandemic, that they seek to produce. By considering COVID as an unprecedented, deeply affective, traumatic event, these online spaces operate to archive this moment in time and its myriad sonic dimensions, bringing these affective worlds into dialogue through an intimate exchange and assemblage between different bodies, experiences, and locations.
—
Emily Collins is an interdisciplinary researcher, writer, educator, and PhD Candidate in Cinema and Media Studies at York University in Tkaronto (Toronto) whose work draws on sound studies, feminist theory, critical disability studies, and cultural theory to examine sonic social relations and materiality through entanglements of resistance and care within contemporary artworks and creative practices. As a cultural worker and active member in the arts community, Emily has worked at diverse film, visual arts, and digital media organizations, institutions, and research networks within Canada and abroad, including Archive/Counter-Archive, PUBLIC Journal, VUCAVU, Festival Scope (Paris), the Toronto International Film Festival, and the Walter Phillips Gallery at Banff Centre for Arts and Creativity.
—
REWIND! . . .If you liked this post, you may also dig:
In summer 2021, sound artist, engineer, musician, and educator Johann Diedrick convened a panel at the intersection of racial bias, listening, and AI technology at Pioneerworks in Brooklyn, NY. Diedrick, 2021 Mozilla Creative Media award recipient and creator of such works as Dark Matters, is currently working on identifying the origins of racial bias in voice interface systems. Dark Matters, according to Squeaky Wheel, “exposes the absence of Black speech in the datasets used to train voice interface systems in consumer artificial intelligence products such as Alexa and Siri. Utilizing 3D modeling, sound, and storytelling, the project challenges our communities to grapple with racism and inequity through speech and the spoken word, and how AI systems underserve Black communities.” And now, he’s working with SO! as guest editor for this series for Sounding Out! (along with ed-in-chief JS!). It starts today, with AminaAbbas-Nazari, helping us to understand how Speech AI systems operate from a very limiting set of assumptions about the human voice– are we training it, or is it actually training us?
—
Hi, good morning. I’m calling in from Bangalore, India.” I’m talking on speakerphone to a man with an obvious Indian accent. He pauses. “Now I have enabled the accent translation,” he says. It’s the same person, but he sounds completely different: loud and slightly nasal, impossible to distinguish from the accents of my friends in Brooklyn.
The AI startup erasing call center worker accents: is it fighting bias – or perpetuating it? (Wilfred Chan, 24 August 2022)
This telephone interaction was recounted in The Guardianreporting on a Silicon Valley tech start-up called Sanas. The company provides AI enabled technology for real-time voice modification for call centre workers voices to sound more “Western”. The company describes this venture as a solution to improve communication between typically American callers and call centre workers, who might be based in countries such as Philippines and India. Meanwhile, research has found that major companies’ AI interactive speech systems exhibit considerable racial imbalance when trying to recognise Black voices compared to white speakers. As a result, in the hopes of being better heard and understood, Google smart speaker users with regional or ethnic American accents relay that they find themselves contorting their mouths to imitate Midwestern American accents.
These instances describe racial biases present in voice interactions with AI enabled and mediated communication systems, whereby sounding ‘Western’ entitles one to more efficient communication, better usability, or increased access to services. This is not a problem specific to AI though. Linguistics researcher John Baugh, writing in 2002, describes how linguistic profiling is known to have resulted in housing being denied to people of colour in the US via telephone interactions. Jennifer Stoever‘s The Sonic Color Line (2016) presents a cultural and political history of the racialized body and how it both informed and was informed by emergent sound technologies. AI mediated communication repeats and reinforces biases that pre-exist the technology itself, but also helping it become even more widely pervasive.
Mozilla’s commendable Common Voice project aims to ‘teach machines how real people speak’ by building an open source, multi-language dataset of voices to improve usability for non-Western speaking or sounding voices. But singer and musicologist, Nina Sun Eidsheim describes how ’a specific voice’s sonic potentiality [in] its execution can exceed imagination’ (7), and voices as having ‘an infinity of unrealised manifestations’ (8) in The Race of Sound (2019). Eidsheim’s sentiments describe a vocal potential, through musicality, that exists beyond ideas of accents and dialects, and vocal markers of categorised identity. As a practicing vocal performer, I recognise and resonate with Eidsheim’s ideas I have a particular interest in extended and experimental vocality, especially gained through my time singing with Musarc Choir and working with artist Fani Parali. In these instances, I have experienced the pleasurable challenge of being asked to vocalise the mythical, animal, imagined, alien and otherworldly edges of the sonic sphere, to explore complex relations between bodies, ecologies, space and time, illuminated through vocal expression.
Joy by Flickr user François Karm, cropped by SO! (CC BY-NC 2.0)
Following from Eidsheim, and through my own vocal practice, I believe AI’s prerequisite of voices as “fixed, extractable, and measurable ‘sound object[s]’ located within the body” is over-simplistic and reductive. Voices, within systems of AI, are made to seem only as computable delineations of person, personality and identity, constrained to standardised stereotypes. By highlighting vocal potential, I offer a unique critique of the way voices are currently comprehended in AI recognition systems. When we appreciate the voice beyond the homogenous, we give it authority and autonomy, ultimately leading to a fuller understanding of the voice and its sounding capabilities.
My current PhD research, Speculative Voicing, applies thinking about the voice from a musical perspective to the sound and sounding of voices in artificially intelligent conversational systems. Herby the voice becomes an instrument of the body to explore its sonic materiality, vocal potential and extremities of expression, rather than being comprehended in conjunction to vocal markers of identity aligning to categories of race, gender, age, etc. In turn, this opens space for the voice to be understood as a shapeshifting, morphing and malleable entity, with immense sounding potential beyond what might be considered ordinary or everyday speech. Over the long term this provides discussion of how experimenting with vocal potential may illuminate more diverse perspectives about our sense of self and being in relation to vocal sounding.
Vocal and movement artist Elaine Mitchener exhibits the disillusion of the voice as ‘fixed’ perfectly in her performance of Christian Marclay’s No!, which I attended one hot summer’s evening at the London Contemporary Music Festival in 2022. Marclay’s graphic score uses cut outs from comic book strips to direct the performer to vocalise a myriad of ‘No”s.
In connection with Fraenkel Gallery’s 2021 exhibition, experimental vocalist Elaine Mitchener performs Christian Marclay’s graphic score, “No!” Image by author.
Mitchener’s rendering of the piece involved the cooperation and coordination of her entire body, carefully crafting lips, teeth, tongue, muscles and ligaments to construct each iteration of ‘No.’ Each transmutation of Mitchener’s ‘No’s’ came with a distinct meaning, context, and significance, contained within the vocalisation of this one simple syllable. Every utterance explored a new vocal potential, enabled by her body alone. In the context of AI mediated communication, we can see this way of working with the voice renders the idea of the voice as ‘fixed’ as redundant. Mitchener’s vocal potential demonstrates that voices can and do exist beyond AI’s prescribed comprehension of vocal sounding.
In order to further understand how AI transcribes understandings of voice onto notions of identity, and vocal potential, I produced the practice project Polyphonic Embodiment(s) as part of my PhD research, in collaboration with Nestor Pestana, with AI development by Sitraka Rakotoniaina. The AI we created for this project is based upon a speech-to-face recognition AI that aims to be able to tell what your face looks like from the sound of your voice. The prospective impact of this AI is deeply unsettling, as its intended applications are wide-ranging – from entertainment to security, and as previously described AI recognition systems are inherently biased.
Still from project video for Polyphonic Embodiment(s). Image by author.
This multi-modal form of comprehending voice is also a hot topic of research being conducted by major research institutions including Oxford University and Massachusetts Institute of Technology. We wanted to explore this AI recognition programme in conjunction with an understanding of vocal potential and the voice as a sonic material shaped by the body. As the project title suggests, the work invites people to consider the multi-dimensional nature of voice and vocal identity from an embodied standpoint. Additionally, it calls for contemplation of the relationships between voice and identity, and individuals having multiple or evolving versions of identity. The collaboration with the custom-made AI software creates a feedback loop to reflect on how peoples’ vocal sounding is “seen” by AI, to contest the way voices are currently heard, comprehended and utilised by AI, and indeed the AI industry.
The video documentation for this project shows ‘facial’ images produced by the voice-to-face recognition AI, when activated by my voice, modified with simple DIY voice devices. Each new voice variation, created by each device, produces a different outputted face image. Some images perhaps resemble my face? (e.g. Device #8) some might be considered more masculine? (e.g. Device #10) and some are just disconcerting (e.g. Device #4). The speculative nature of Polyphonic Embodiment(s) is not to suggest that people should modify their voices in interaction with AI communication systems. Rather the simple devices work with bodily architecture and exaggerate its materiality, considering it as a flexible instrument to explore vocal potential. In turn this sheds light on the normative assumptions contained within AI’s readings of voice and its relationships to facial image and identity construction.
Through this artistic, practice-led research I hope to evolve and augment discussion around how the sounding of voices is comprehended by different disciplines of research. Taking a standpoint from music and design practice, I believe this can contest ways of working in the realms of AI mediated communication and shape the ways we understand notions of (vocal) identity: as complex, fluid, malleable, and ultimately not reducible to Western logics of sounding.
—
Featured Image: Still image from Polyphonic Embodiments, courtesy of author.
—
Amina Abbas-Nazari is a practicing speculative designer, researcher, and vocal performer. Amina has researched the voice in conjunction with emerging technology, through practice, since 2008 and is now completing a PhD in the School of Communication at the Royal College of Art, focusing on the sound and sounding of voices in artificially intelligent conversational systems. She has presented her work at the London Design Festival, Design Museum, Barbican Centre, V&A, Milan Furniture Fair, Venice Architecture Biennial, Critical Media Lab, Switzerland, Litost Gallery, Prague and Harvard University, America. She has performed internationally with choirs and regularly collaborates with artists as an experimental vocalist
—
REWIND! . . .If you liked this post, you may also dig:
Recent Comments