Archive | Digital Humanities RSS for this section

“Share your story” – but who will listen?


DH ListeningEditor’s Note: Welcome to the second installment in the “DH and Listening” blog series for World Listening Month, our annual forum that prompts readers to reflect on what it means to listen. This year’s forum considers the role of “listening” in the digital humanities (DH, for short). We at Sounding Out! are stoked to hear about (and listen to) all the new projects out there that archive sound, but we wonder whether the digital humanities engage enough with the the notion of listening. After all, what’s a sound without someone to listen to it?

On our opening week, Jacqueline Wernimont from the Vibrant Lives team shared with us about the ethics of listening to 20th century sterilization victims’ records. Then, Emmanuelle Sonntag introduces us to a figure from a long time ago, “la soeur écoute,” a nun whose was responsible for sitting in and listening when another nun had a visitor. This week, Fabiola Hanna reflects upon what DH means when it talks about participatory practices. –Liana Silva, Managing Editor

Germanic and Holocaust historian and Digital Humanist Todd Presner has worked through more than 52,000 testimonies of the Shoah Foundation’s Visual History Archive, perhaps the largest archive of testimony, to investigate “the ethics of the algorithm.” The scale of this participatory archive has prompted Presner to ask: is it possible for a computer to listen to testimony?

Digital Humanities (DH) projects are, in varying degrees, led by the desire to engage with a wider public. Some often include actions such as inviting participants to share their stories, images, audio clips, drawings, and videos. In order to avoid lumping all DH works under a broad category that includes, among others, text analysis, mapping, visualization, 3D, and archiving. Below I examine DH projects that collect oral histories: memories, stories, and testimonies.

I argue that it is not enough to share stories; it is also important to recognize how these stories are shared. A majority of these living histories end up in an audio or video database displayed in full on a webpage, not unlike oral history transcripts ending up in a dusty closet. Listening as an active mode of participation can provide us with a framework that reveals the relationships, aesthetics and politics engendered by participation. Although seemingly about access (and some have started questioning whether users of such an archive should have access by default, or whether you need to be part of a community in order to access certain stories, see Mukurtu CMS), my focus is rather on the medium that these stories are circulated in, how they are displayed to the user and what the user’s participation does to these stories.

I take the space here, on the one hand, to think through participation from two fields, archives and interactive documentary (i-docs), and on the other hand to write about how sound studies, particularly an attention to listening and its aesthetics, its affordances, and its politics, can offer another approach to participation.

SO! Screen Capture from "The WorryBox Project"

SO! Screen Capture from Irene Lustzig’s “The WorryBox Project”

DH + participation: steps towards making place for popular

The work of requesting participation is in itself an accomplishment. It’s only been since the 1950s/60s, and perhaps more broadly since the 1990s, that ordinary peoples’ stories have been valued in the tradition that oral historians have established. It is possible to trace the field’s origins to Augusto Boal’s Theatre of the Oppressed (itself rooted in Paolo Friere’s work), to Howard Zinn’s A People’s History of the United States, or to postmodernism, the death of the author, and the end of grand narratives. With the arrival of Web 2.0, oral history seemed like the perfect medium for the ideals of participation on the Internet, precisely because it was about the everyday and involved “anyone” (with an internet connection). Video, audio and live streaming technologies made it possible for oral history to be recorded and displayed, and social media enabled unprecedented circulation through sharing. So the recent push to preserve everyday histories online, in DH projects, follows these ideals of stories from below.

Listening to stories as a mode of engagement is fitting with regards to oral history and testimony. Making space with openness and silence for someone’s testimony, especially when difficult, is a mode of participation that we aspire to. Media scholar Wendy Chun has argued that “a politics and practice of listening [is] a necessary complement to a politics of testifying.” In the case of Holocaust testimony, for instance, many scholars have argued that there is a responsibility to listen (Laub & Felman and Henry Greenspan).

I differentiate between two types of listening:

  1. Listening on the part of the users of these DH projects. Is it realistic to expect that users will listen to large archives of testimony? In Listening Publics, Kate Lacey writes that listening has been absent from theories of the public sphere, even where “the objective of political agency is often characterized as being to find a voice – which surely implies finding a public that will listen, and that has a will to listen” (viii). Because of their scale, these large archives of testimony pose a significant design challenge to the user, which could be resolved through curation. To build on one of the most successful digital storytelling projects, the Storycorps team knows very well that if they didn’t curate and edit together shorter versions, then no one would listen to the longer interviews in full.


  1. Listening on the part of the interface and its design. This involves questioning the medium and its effects on what it houses. Here I find Susan Bickford’s work very useful in her definition of listening and what it accomplishes: ‘the riskiness of listening comes partly from the possibility that what we hear will require change from us’ (1996, 149). If interfaces for stories pre-close any openness to what contributed stories could change, then listening is not being considered. This is more directly seen in the tagging of videos and their categorization without additional interpretational work. This will to not “add” to these stories seems to come from the premise that these testimonies should “speak for themselves” (I get to this point in a little bit); that no added interpretation is needed, even that any added interpretation distracts from the directness of the stories. But this often also means the medium and its effects on these stories are not carefully examined.

Contrary to Western cultural beliefs, listening can be seen as an active mode of participation in conversation. As Jodi Dean rightly argues, it is not enough to express one’s opinion, or for a message to be circulated: it is crucial to get a response to the message. Lisbeth Lipari writes that even in Dean’s formulation, however, the response as speech is still emphasized, thereby ignoring the work that happens beforehand: the listening. Apart from the major contribution of Susan Bickford, mentioned above, with the concept of political listening, little attention has been paid to listening in the field of political theory.

DissonanceAndDemocracyBySusanBickford and TheOtherSideOfLanguageByGemmaCorradiFiumara

As arguably the most important philosopher on listening, Gemma Corradi Fiumara, writes: “listening involves the renunciation of a predominantly moulding and ordering activity; a giving up sustained by the expectation of a new and different quality of relationship” (as quoted by Lipari, 8). For Fiumara, listening is a mode of participation that is open and that does not pre-close potential possible contributions. Listening, therefore, as a mode of engagement can provide a useful method for thinking about participation in various contexts.

DH + Archives + i-docs

If a mission of a certain DH project is to engage with a wider public, then preservation of stories is not enough. More often than not, DH projects are already under difficult strain of resources, due to costs for developing technological projects and time for interviewing, collecting, indexing, tagging, uploading and making sense of these larger collections. DH have taken on a lot: preserving and archiving everyday stories with the mission to share with a wider public. It is useful to draw from experts who have been working through ideas of preservation and access: archivists. Archives generally have a mission of preservation, but when demands of sharing with a public arise, there are not many models from which to draw. Rick Prelinger writes that although the archive as a theoretical concept is overly theorized, its practice (in the plural form of the word) is under theorized.


Image of StoryCorps collection area by Aaron “tango” Tang (CC BY 2.0).

Francis X Blouin and William G. Rosenberg trace the intersection of history and archives and found that Ranke, during the enlightenment, conceptualized history as a scientific endeavor in that truth could be extracted from archives through rigorous methodologies. This led to the idea that documents could “speak for themselves” (Blouin, 24). As if simply making documents available, without providing context of any sort, would reveal their inner truth. This is one of many cases where the reading of documents is taken for granted. It also ignores the effect that archivists have on the collecting, saving, and indexing of documents. Influential archivists such as Terry Cooke, Richard Brown and Brian Brothman have brought about new attitudes to repositories with an acknowledgment of the effect that archivists have on documents (as quoted by Richard J Cox, 33). This relatively recent push in archival theory, therefore, points to the flaws in the claim that documents on their own can represent themselves: that would be ignoring all the various power relationships at play, as well as the medium itself in which the stories are communicated.

In thinking about collaboration, with the unstated but implied goal of providing knowledge of a given subject, non-fiction film has experimented by inviting self-interested participants online and offline to contribute to projects. Sandra Gaudenzi makes clear that interactive documentaries, also called i-docs (documentaries designed for online or mobile viewing), are very distinct and that it is important to recognize the varying strategies employed by these digital projects: “uploading content is the most common way to collaborate in the case of online documentary, but it is definitely not the only one. […] contributions of content lead to co-creation but not to co-authorship, since the latter require a degree of intervention in the overall concept (i.e., form) of the product.” In her differentiation of co-creation and co-authorship, Gaudenzi points to the medium-limiting effect that users have in their participation. The “degree of intervention,” as a coming in between that leads to change, which Gaudenzi requires for co-authored documentaries, might be attained through listening, a listening through the work itself, such as The WorryBox Project.

Listening in The WorryBox Project

Irene Lusztig’s The WorryBox Project (2011) web-based documentary invites mothers to write their worries in an online form, which Lusztig then individually writes down on a piece of paper, rolls-up and stores in a physical box. The act of writing down in analog form what was input into a digital box goes an extra step beyond collection. It is emblematic of an extended care for these worries. The additional layer of documenting these actions, as part of the artwork itself, not only points to the embodiment of these worries but also to their representation.


Because listening has often been thought of as passive and feminine, it is also associated with a specific kind of caring, also in its passive and feminine forms. But I don’t equate that definition of care with listening, which surely the readers of this blog will question. In this case, care extends as a mode of listening because it pushes against the active/passive binary for speaking/listening. I use care in the sense it gave to curating (care is its early root), where curating points to the work of engagement and conversation.  Curating as a form of listening.

ListeningPublicsByKateLacey and ListeningThinkingBeingByLisbethLipari

This extended care would be an example of the listening I am thinking of. The work of translating the form’s digital input into the handwritten words and the documentation of that act stretch the layering of interpretation and with it the care given to these submissions. The change that occurs in this writing is definitely not an agreement or an endorsement of what is submitted but an open attitude of listening, of reception. This reinforces the notion that listening is not passive, but an active mode of participation. 

This mode of listening does not imply an understanding, as the receipt of information in cybernetics theory suggested by Shannon and Weaver. Rather this mode of communication embraces the notion that listening does not necessarily mean an understanding. The audio action collaborative Ultra-Red conveys this idea perfectly: “Sound possesses a palpable promiscuity in relation to the body. One may say what one means, but somewhere between the mouth that speaks and the ear that hears, sound always exceeds its master. Politically, it may be useful to say that we hear the truth in the voice or that we listen with obedience, but sound always transgresses such duties.” So for DH storytelling projects to be participatory, it is beneficial to allow stories the space for listening, in a similar sense to Hannah Arendt’s “distancing” in order to understand. This brings me to media and communication’s scholar Lisbeth Lipari’s term for this: “listening otherwise.”

SO! Screen Capture of Irene Lustzig's "The Worry  Project"

SO! Screen Capture of Irene Lustzig’s “The Worry Project”

For Lipari, in engaging with difficult stories or perhaps stories that offer different perspectives: “It is […] a listening otherwise that suspends the willfulness of self- and foreknowledge in order to receive the singularities of the alterity of the other” (185, emphasis in original). For this “listening otherwise” to work online, I argue that it is important for the employed structures, systems and processes to reflect this specific kind of participation. In other words, in addition to placing “different voices” side-by-side, it is equally critical for these different perspectives to exist in a space where the medium, whether the recording, the software, the interface and its design, listens to what these differences are. An interface designed in order not to predetermine, not to predict or preclose what others might contribute would then be an example of this listening. Anthropomorphizing aside, for a computer to listen to testimony, as Presner asks, it must be designed to be open to changing itself.

Fabiola Hanna is a new media artist & software designer currently working towards a PhD in Film and Digital Media at UC Santa Cruz where she also holds an MFA in Digital Arts & New Media. Her doctoral work is on building an automatic editing machine that weaves together competing narratives about the history of Lebanon, which has led her to engage with software studies & digital humanities, archives & memory and new media art activism.  

Her work has been exhibited widely in California at the Museum of Art and History in Santa Cruz, the New Children’s Museum in San Diego, the SubZero Festival in San Jose, the Digital Arts Research Center in Santa Cruz, and the Maker Faire in San Mateo. She is also a 2015 fellow of the Institute for Critical Social Inquiry at the New School, New York and has previously taught at University of California, San Diego and at various maker spaces including FabLab San Diego and MakerPlace.

Her website is and she can be reached by email fhanna {@} ucsc {dot} edu.

Featured Image: “Story Corps” by Steve Rhodes, (CC BY-NC-ND 2.0)

tape reelREWIND!…If you liked this post, you may also dig:

What Mixtapes Can Teach Us About Noise: Reading Shannon and Weaver in 2010–Aaron Trammell

Machinic Ballads: Alan Lomax’s Global Jukebox and the Categorization of Sound Culture–Tanya Clement

Digital Analogies: Techniques of Sonic Play“-Roger Moseley

Hearing Eugenics

"Water under 12.5 Hz vibration" by Jordi Torrents, CC BY-SA 4.0, via Wikimedia Commons

DH ListeningEditor’s Note: Today we start off a series, a propos for World Listening Day 2016 on digital humanities and listening. As I mentioned in my Call for Abstracts in March, this forum considers the role of “listening” in the digital humanities (DH, for short). We at Sounding Out! are stoked to hear about (and listen to) all the new projects out there that archive sound, but we wonder whether the digital humanities engage enough with the the notion of listening. After all, what’s a sound without someone to listen to it? The posts this month consider: how have particular digital studies, projects, apps, and online archives addressed, challenged, expanded, played with, sharpened, questioned, and/or shifted “listening”? What happens to digital humanities when we use “listening” as a keyword rather than (or alongside) “sound”? 

We will be hosting the work of DH scholars who are doing exactly that: prompting readers to consider what it means to listen in the context of DH projects. Fabiola Hanna will be reflecting upon what DH means when it talks about participatory practices. Emmanuelle Sonntag, who has written for SO! before, will be addressing listening from the starting point of the documentary Chosen (Custody of the Eyes). Today, however, we start things off with a collaborative piece from the Vibrant Lives team on the ethics of listening to 20th century sterilization victims’ records.

Don’t just stand there. Take a seat and listen.-Liana M. Silva, Managing Editor

In the 1920s a young woman was admitted by her mother to a mental institution in California. The local doctor recommended her for sterilization with the following notes:

has been reported to have interest in sexual encounters

Mother is pregnant and cannot care for her (thinks she may be able to post-sterilization).

This brief note is representative of the stories of the roughly 20,000 people who were sterilized in California institutions of mental health. The soundscape of these institutions is largely lost to the past. We cannot recover the sounds of treatment spaces, family visits, recreation, and everyday life of those in the care of the state of California who were considered feeble, insane, or otherwise out of control.

Like the conversations about illness and reproduction presumably had in those halls, the sounds of salpingectomies (removal of fallopian tubes), vasectomies (severing the vas deferens), and, later, tubal ligations are lost to us. In the absence of human rights violations, this is perhaps as it should be; we cannot collect the minutiae of everyday life. But in situations where reproductive and disability rights have been limited, where we can see race and gendered bias, we may well have need of telling such stories.

Reparative justice best practices dictate that survivors should be able to tell their own stories on their own terms. How can we listen to such stories when the majority of our survivors have died and we have little to nothing in their own words?


A redacted sterilization form from Stockton State Hospital. Image courtesy of Alexandra Minna Stern

While conversations between patients, parents, and doctors might be lost to us in terms of playback, they have embodied traces in the nearly 20,000 people sterilized in California between 1919 and the 1950s under eugenic sterilization laws. The 19,995 sterilization recommendations and notes, brought together under the project Eugenic Rubicon: California’s Sterilization Stories, cannot currently be made publicly available due to U.S. patient privacy laws. Important documentary films like No Más Bebés, which tells the story of Mexican-American women sterilized without consent at Los Angeles County – USC Medical Center in the 1960s and 1970s, have made it possible for us to hear accounts of such reproductive injustice first hand. But for the thousands of people sterilized between 1909 and the repeal of eugenics laws in 1979, we must find other ways to listen and to hear.

Given the privacy restrictions on working with this dataset and our concerns to care for the people who are represented therein, we (the Vibrant Lives team) felt it was important to find alternative methods that did more than de-identified and quantified graphs could do. We know all too well that we can’t recover the past “as it was.” Nevertheless, we are working to bring the emotional and intellectual power of sound and critical listening to a largely unheard history of sterilization of Latinx people. Specifically, our project prompts listeners to consider how listening fits into reparative justice for the victims of sterilization.

"Listen" by Flickr user Fe Ilya, CC BY-SA 2.0

“Listen” by Flickr user Fe Ilya, CC BY-SA 2.0


Listening Toward under the Law

That eugenics laws and their surgical enactments played out in racialized and gendered ways is not surprising but bears repeating. For example, according to work by Alexandra Minna Stern, Nicole Novak, Natalie Lira, and Kate O’Connor, patients with Spanish or Hispanic surnames were three times as likely to be sterilized as their non-Hispanic counterparts. Those lost sounds have traces in California’s Latinx communities, both in terms of the community structures themselves, but also in terms of soundscapes that never were because of sterilization. This acoustic ecosystem in which the politics of race, gender, nation, and mental health converged in dramatic fashion is recorded only in the bodies and medical records of the patients and the 21st century communities shaped by the children, born and unborn, of these patients.

Not only are we limited to working with the textual, institutionally generated remnants of the past, we are also constrained by 21st century health and personal data privacy laws. Our archive is a set of medical records and as such this collection contains sensitive patient data that must be de-identified and used in accordance with contemporary HIPAA (Health Information Portability and Accountability Act) regulations and IRB protocols.

This means that we cannot reveal names, dates, and other identifying information regarding those who were sterilized in the first half of the 20th century. We are unable to tell individual stories of sterilization lest the individual be identified. Traditionally, historians have used fictional composites to tell such stories and our collaborator Alexandra Minna Stern used this method in her 2015 second edition of Eugenic Nation.

The HIPAA guidelines and their impact on how we tell the history of medicine raises important legal questions about how we might balance a public right to know about practices (we’d call them abuses) within state-run facilities with the need to protect patients’ rights to privacy regarding their own reproductive and mental health. In some cases, it seems as though the privacy guidelines protect the state more than they protect any individual patient. In fact, we have seen a remarkable lack of concern for these records in their discovery and transmission. The records themselves were largely abandoned when Stern discovered the microfilm reels in the 2000s. They were lost again after she returned them to the state after having made a copy. The originals are lost as far as we know.


“Archive” by Flickr user Carolina Prysyazhnyuk, CC BY-SA 2.0

Listening Toward the Past

Vibrant Lives is working not with sounds found, but with archival records found and then sonified (transformed into sound) as a way of listening toward those rooms, conversations, and procedures. In brief, this sonification entails the following steps

  • Selecting a subset of the large data set (we can’t currently process the whole)
  • Selecting between two and four axes of information, such as gender, race, age at sterilization recommendation, consent, or nationality
  • Mapping the informational values into numerical space – sonification requires the creation of a dataset whose limits are 1 and -1 (based on how the speakers work)

This work has been done to date using two tools: Sonification Sandbox, an open source tool developed at the University of Georgia, and GarageBand, a proprietary music making tool that comes with Macintosh computers. We use Sonification Sandbox to create the score first and then turn to GarageBand because it has a greater range of instrumentation available. The sonification process is still very experimental and exploratory. Team member Jacqueline Wernimont does all of our sonifications for us and she is trained as a historian of literature and technology. While she has extensive experience within digital humanities methodologies, sonification is a new effort for us.

We have begun producing short sample tracks that allow us to enact the kind of listening toward that we’re advocating for. In the track below, we have data from the age, gender, and consent axes for the period 1940-1949. Additionally, this sample draws only from what we’ve described as “Spanish surname” patients, the vast majority of whom were American-born of Mexican descent, although they also include some other Latinx national communities.

Latinx Eugenics Sample Track


As you listen, each note represents one Spanish-surnamed person recommended for sterilization. The children, both boys and girls under 18, who were sterilized without consent are the highest notes, and the adult men who were sterilized with consent are the lowest.

Listening Toward as Ethical and Communal

Listening is always about an ethical relationship and it is particularly fraught when the effort to listen and to encourage others to listen entails hearing about a person’s most intimate health information and experiences. This is particularly true when those experiences may include trauma from unwanted surgery or other experiences.

While we might think of patient privacy as a form of care, in this instance we find ourselves wondering who these regulations actually serve. According to the updated 2013 HIPAA guidelines, personal health records are no longer considered sensitive information 50 years after death (it was previously 100 years).  Preliminary estimates by our team indicate that as many as 1,000 survivors might be alive in 2016. However, while the vast majority of the people discussed in the records are no longer alive, family and friends may well be.

We respect the need for family members and friends to privacy when it comes to the health records of their loved ones. At the same time, an essential component of most restorative justice programs, like those undertaken for North Carolina eugenic sterilizations, is an articulation of the violations, which HIPAA blocks in many ways (North Carolina’s cases were revealed by investigative journalists who are not subject to HIPAA and the IRB regulations that we must adhere to as academics). As a consequence, those who might most benefit from reparations – sterilized individuals and their immediate families, including children – are likely to die before the privacy laws enable us to draw attention to the individual impacted by the racialized and gendered discrimination evident in the records.

The sonification of these records and the companion participatory performances that we facilitate allow us to intervene and share these important stories before all of the survivors and family members have passed away. We have the opportunity to drive justice-oriented processes forward while there is still time.

Consent/Non-consent Sample Track (entire population)

Vibrant Lives focuses not just on the stories but also on the people who listen to the audio. We spend time watching how our audiences participate in listening toward the history of eugenic sterilization in California. Below are images of recent presentations of this work in which we’ve incorporated both haptic (touch-based) and sonic performance.


Participants listening at a presentation at the University of Kansas. Photo by Elika Ortega.


Part of what we see here is the attentive posture of our participants – leaning in to feel a history of sterilization. The haptics are being shared with a thin, red metal wire that the participants have to touch lightly in order to not dampen the signal for others. For us, this is an effort to bring care for the experiences of others into the performance. The history of eugenics has impacted communities and we are creating communal aural and tactile experiences as a way to disrupt the notion that academic work and knowledge is a solitary endeavor.

The performance captured above is also an exercise in patience and as such expresses a willingness on the part of the participants to sit with a disturbing history. The sample people are listening to and feeling here is 100 seconds long with each note/vibration corresponding to one person who was sterilized. In most performances the participants stay for the duration of the piece, but there have been instances where people have touched a haptic piece and then walked quickly away. We can’t know why some have chosen to walk away.

KUEugenicsImage3PamellaLachSome of those who have stayed have shared with us that they felt responsible to feel and hear each person. It’s an abstraction, to be sure, but we are intrigued by the power of listening and feeling to encourage people to not simply look and walk away. As one participant at a Michigan performance noted, the “tingling (from the haptics) lingers, it’s spooky.” Another participant at the same performance indicated that she felt “more implicated” having engaged with a multi-media experience than with a visual like a graph or chart. When asked why, she responded “I’ve felt it and will continue to remember that, but still will likely do nothing in response.”

In creating performances where participants have to care for one another and care enough about the people represented in the data to stay through a durational piece, we are working to redress the extraordinary lack of care that the records represent, both in terms of testifying to the violence done to men’s and women’s bodies and in terms of the State of California’s lack of regard for this history.


Sounds Felt, Sounds Touched


Our work is an ongoing experiment. We’ve moved from haptics along a wire, to haptic spheres that vibrate with the sonification. The image above is from one of these events this spring. We’ve retained the communal effect while transforming the embodied structure of the event. Participants now gather around, encircling the object as they listen toward a history of reproductive injustices. People still tend to lean in – to have heads lowered in a posture of intense focus. The sphere itself demands that someone cradle it and it also requires that people touch lightly once again so as to not dampen the experience for others.

We plan to expand our durational events in our next iteration known as “Safe Harbor” in which we hope to explore how to best care for those people sterilized by the state by caring for their data. In this instance we are thinking of sounds (and more) that we’ll make together with impacted communities. For this work we are particularly interested in engaging audience members in the hosting and care of the eugenics data and, by extension, the survivors.

As a way of enacting a site-specific response to both historical and contemporary human and reproductive rights violations that have occurred in the state, we plan to stage this durational event in California. We’ll begin by inviting audiences to help build and shape an empty warehouse space with us, transforming the empty space into a place of care where we can listen toward these histories. The audience will be invited to converse about the research and reflect upon conversations through making, creating, and ultimately building up our safe harbor.

We plan to listen to and co-create with impacted communities through collective making of the space. As a result, Safe Harbor will enact a cooperative improvisational process shaping socially responsive dialogue – performing, hearing, listening, documenting, and rebuilding notions of care in real time. What we hope to discover here are shared sounds of resistance, repair, and healing. Sounds that might let us listen toward the past, while also creating more just futures.

Featured image: “Water under 12.5 Hz vibration” by Jordi Torrents, CC BY-SA 4.0, via Wikimedia Commons

Vibrant Lives is a collaborative team that makes, stages, and performs as part of interactive multimedia installations. Jessica Rajko and Eileen Standley are both professors in the Dance area of the School of Film, Theater, and Dance at Arizona State University (ASU). Jacqueline Wernimont’s home department at ASU is English and she’s a digital humanities and digital archives specialist. Wernimont and Rajko are also multimedia artists/faculty working in Arts, Media, and Engineering.

The data derives from a larger project, known as Eugenic Rubicon: California’s Sterilization Stories, a multidisciplinary collaboration among Arizona State University, University of Illinois Urbana-Champaign, and University of Michigan. This larger collaboration includes historical demography and epidemiology, public health, history of medicine, digital storytelling, data visualization, and the construction of interactive digital platforms. This team is quite large, with our center of gravity residing at the University of Michigan where historian of science Alexandra Minna Stern directs the Eugenic Rubicon lab. Stern discovered the microfilms of more than 20,000 eugenic sterilization patient records in 2013. Stern and her team have created a dataset with this unique set of patient records that includes 212 discrete variables culled from over 30,000 individual documents. This resource is the first of its kind, encompassing almost one-third of the total sterilizations performed in 32 states in the U.S. in the 20th century.

tape reelREWIND!…If you liked this post, you may also dig:

Sound Off! // Comment Klatsch #4: Sound and The Digital Humanities, or #dhsound

EPISODE LI: Creating New Words from Old Sounds–Marcella Ernest, Candace Gala, Leslie Harper, and Daryn McKenny

Radio Preservation Task Force Conference: Sound History and the Logistics of Social Recognition-Josh Sheppard

SO! Amplifies: Think About Sound App and Map


soampSO! Amplifies. . .a highly-curated, rolling mini-post series by which we editors hip you to cultural makers and organizations doing work we really really dig.  You’re welcome!

What is it about the environmental soundscape which makes us ‘tune-in’ or ‘tune-out’ to particular sounds? Do we as humans tend to seek out quiet zones for our acoustic pleasure or are there those among us who find urban soundscapes a more comforting prospect?  Researchers at Glasgow Caledonian University in the UK have developed a mobile phone application to allow the personalized assessment of such questions regarding environmental soundscapes. I developed the free Think About Sound interactive map–downloadable as a mobile app and viewable online–to allow users to experience various locations in Glasgow by using 3D audio recordings and panoramic visuals.


Think About Sound is a novel tool which shifts the traditional paradigm of environmental soundscape assessment using an experience sampling methodology. Over the last decade, smart phone ownership has increased immeasurably; I applied the technology here in order to allow local, in-situ soundscape assessment as participants go about their daily routine. Based on the Adobe PhoneGap Build platform, the application has been developed using HTML, CSS, and JavaScript, employing jQuery Mobile to provide a simple and clean navigational structure to the user.  Crowdsourcing data in this way has enormous potential to create rich and diverse data sets, where both qualitative and quantitative descriptions of environmental surroundings can be gathered in a flexible and non-invasive way.

By using a self-reporting methodology, Think About Sound removes the listener from traditional laboratory-based soundscape evaluation and locates them in real world experiences as they go about their day-to-day activities.   The application aims to find out the various types of sound encountered as users understand them, asking how users feel before and after a recorded sound event and enabling them to describe the circumstances in which they heard the sound event. Think About Sound also asks listeners to provide semantic descriptors for the sound, toward the ultimate aim of creating more sophisticated environmental sound maps which communicate both location-specific sound information and the subjective effect of sound upon the listener.

To further enrich the experience, data sent from the application can be viewed online at with an accompanying map where the public can view and audition submissions using the familiar Google map format.  You will also find links to download the app in multiple formats.


I hoped that by collecting data in this way and to this scale, that I can obtain and share a greater understanding of how we perceive soundscapes. The next steps for the project includes the development of audio technology to analyze sound recordings, automatically predicting annoyance, valence (the emotional value associated with a stimulus), and the arousal features of environmental sounds for particular users.

While locale remains important, this research has far more reaching implications beyond my local region. Submissions on an international level can help us to understand how we perceive our environmental soundscapes, help shape local noise policy, and provide others with an understanding of sounds in their local area.  What I want from you, the reader, is your help via contributions to this worldwide soundscape project. Stop for a minute and take in your sonic surroundings. What can you hear? How does it make you feel? Comfort? Anxiety?. . .Stop for a minute, listen and think about sound.


Adam Craig is a Ph.D researcher studying at Glasgow Caledonian University in the UK within the School of Engineering and Built Environment. After obtaining a first class honours in his undergraduate Audio Technology degree in 2011, Adam went on to embark on a Ph.D concentrating his research on using advanced audio technology for the creation of environmental sound maps. He Is currently a member of the AudioLab Research team at GCU and is a member of the Institute of Acoustics and the Audio Engineering Society. Out with his academic research, Adam teaches sound engineering to high-school students at a community based service within his local education authority, and at West College Scotland.

REWIND!…If you liked this post, you may also dig:tape reel

SO! Amplifies: Ian Rawes and the London Sound Survey–Ian Rawes

SO! Amplifies: Cities and Memory–Stuart Fowkes

SO! Amplifies: Feminatronic

SO! Amplifies: #hearmyhome and the Soundscapes of the Everyday–Cassie J. Brownell and Jon M. Wargo

“All their ioynts & properties”: Orthography and Sound in Early English Poetry

Mixed Media Painting (Detail) by Choichun Leung / Dumbo Arts Cen

Medieval SoundEach of the essays in this month’s “Medieval Sound” forum focuses on sound as it, according to Steve Goodman’s essay “The Ontology of Vibrational Force,” in The Sound Studies Reader“comes to the rescue of thought rather than the inverse, forcing it to vibrate, loosening up its organized or petrified body (70).  These investigations into medieval sound lend themselves to a variety of presentation methods loosening up the “petrified body” of academic presentation. Each essay challenges concepts of how to hear the Middle Ages and how the sounds of the Middle Ages continue to echo in our own soundscapes.

The posts in this series begins an ongoing conversation about medieval sound in Sounding Out!. Our opening gambit in April 2016, “Multimodality and Lyric Sound,” reframes how we consider the lyric from England to Spain, from the twelfth through the sixteenth centuries, pushing ideas of openness, flexibility, and productive creativity. We will post several follow-ups throughout the rest of 2016 focusing on “Remediating Medieval Sound.”  And, HEAR YE!, in April 2017, look for a second series on Aural Ecologies of noise! –Guest Editors Dorothy Kim and Christopher Roman

During the “grammar wars” of the sixteenth century, when some scholars sought to restrict English letters to an “isomorphic” (or phonemic) relationship between sound and spelling, Richard Mulcaster emerged as a champion of a more broad and complex vernacular orthography in his 1582 Elementarie, with profound implications for the growing English language. In particular, Mulcaster’s insight that language is shaped by “custom” bolsters the argument for variety over phonemic standardization; as he writes (in the 1925 Oxford University Press reprint):

letters ca[n] expresse sou[n]ds withall their ioynts & properties, no fuller than the pencill can the form and lineaments of the face, whose praise is not life but likeness: as the letters yeld not alwaie the same, which sound exactlie requireth, but allwaie the nearest, wherwith custom is content (99).

Mulcaster seems to strongly indicate here that we should not expect vernacular orthography to capture sound in any reliable way. Custom – meaning actual usage, etymological roots, and so on, muddies the waters of spelling-based sound. Anyone familiar with Modern English, given its complex conventions and silent, or variably pronounced letters, might agree. Yet as John Wesley notes in “Mulcaster’s Tyrant Sound,” “Mulcaster’s orthography continually oscillates … between a conception of letters as completely divorced from sound, and one that finds sound and sight interacting (not always in conflict)” (349).

Indeed, looking more closely at Mulcaster’s statement, it is possible to discern the specter of a scribe, pencil in hand, attempting to capture the “joints and properties” of a poet’s sound in letters – using different letters depending on the dialectal orthography of the compositional time and space: a listening body making a visual representation of sound.

Image of a scribe by Nathan Adams @Flickr CC BY-NC-ND

Image of a scribe by Nathan Adams @Flickr CC BY-NC-ND

Certainly, we encounter instances of orthographical representations of distinctive sounds in more recent and deliberate dialect poetry. John Hyland, writing on sound and African diasporic poetry in his unpublished dissertation Atlantic Reverberations, notes of late nineteenth- / early twentieth-century black dialect poems that “in a certain way, they are meant to be read and heard as talking machines. The sound technology, in this case, is the poem; it is a construction and a recording of a ‘black’ voice that socially and culturally functions in a manner analogous to the gramophone” (31). [Ed. Note: you can also read his work on Sounding Out! here]. Following this, it is interesting to try to figure out how the text of a poem can gesture toward and suggest a “sound” that belongs to a localized (even stereotyped) body, despite the text’s being mute on the page. New computational linguistic programs can help play back these ancient sound files, but based on my experience in using them with Middle English texts, there is a ways to go in matching their analytic capabilities with idiosyncratic early English spelling to produce satisfying results.

While late medieval English poems cannot be thought of as deliberate “constructions” of a voice in quite the same way as some contemporary forms of black poetry—although portions of Chaucer’s The Reeve’s Tale, in which “northern speech” is represented through alternate spellings and diction, might be an exception—I propose that we imagine orthography (and by extension, the scribes who implemented it) as a kind of “sound technology” analogous to early sound reproduction devices such as the phonograph. The analogy will break down at certain points, but in suggesting it I hope to answer two related questions: First, to what extent can orthography reliably encode sound (or, to put it another way, offer a score for “decoding” sound)? Second, can we extend back in time the critique of sound technology made by recent “sound theorists,” who tend to focus on texts and technologies from the modern period – and, if so, what can be gained thereby?

A medieval alphabet. Image by Cesar Ojeda @Flickr CC BY-NC-ND

A medieval alphabet. Image by Cesar Ojeda @Flickr CC BY-NC-ND

The answer to the first question is, I believe, a very qualified “yes.” Yes, English orthography can and does encode sound, but like so many things about the English language and its highly idiosyncratic spelling conventions, it’s complicated. The second question leads to an elusive, but promising, framework in which to consider the role of orthography in shaping English poetic sound. In short, I will argue that orthographic profiles act as a sort of “performance” in which spelling collaborates with the sound of language to offer a socio-linguistic context within which to experience a given poem.s

I used SPARSAR, “an expressive poetry reader” developed by computational linguists Rodolfo Delmonte and Anton Maria Prati, to produce audio recordings of three poetic specimens from Middle English: The specimens are: Osbern Bokenham’s “Life of St. Anne,” (ll. 41-64); Geoffrey Chaucer’s The Franklin’s Prologue, (V.709-28); and John Audelay’s “Conclusion,” (ll.1-13). The specimens were chosen with the idea of dialectal variety in mind: Bokenham is associated with Suffolk, Chaucer with London, and Audelay with the West Midlands. According to its creators, SPARSAR uses “prosodic durational parameters” for English syllables developed by the authors with the aim of “evaluat[ing] objective presumed syllable and feet prosodic distribution at line level,” producing a version of a poem that can be read by Text to Speech (TTS) software with “an appropriate expressivity” (73).

The problem with TTS software is that it produces monotone outputs that fail to account for prosody, let alone expressivity. SPARSAR, I hoped, would “level the playing field” between differing orthographies in the poems, from an analytic standpoint: “the poem is translated into a phonetic form preserving its visual structure and its subdivision into lines and stanzas. Phonetically translated words are associated to mean duration values taking into account position in the word and stress” (Delmonte and Prati 74). Yet textual analysis of poems in Middle English is challenging precisely because of orthographical variation, and this holds true for SPARSAR as well. Before SPARSAR could recognize the speech of the poems, I had to run them through a “normalizing” program—VARD, although Morphadorner offers a similar service—thus in some ways defeating the purpose of testing orthography’s ability to encode sound, as well as muddling SPARSAR’s fine-grained prosodic analysis.

Audelay first with and then without SPARSER.


Audelay for VARD. Image used with permission by the author.

Thus, for example, in the second line of my selection from John Audelay, “Here may ye cnow hwat ys this worlde,” I replaced “cnow” with the modern “know” (thus losing the /k/ sound), “hwat” with “what,” “ys” with “is,” and “worlde” with “world.” One easily discerns the loss of information with respect to not only sound, but also scansion (the inflectional “e” ending). SPARSAR also does not account for the Great Vowel Shift in reproducing Middle English pronunciations. For how this would change the pronunciation of some words, see Simon Horobin, Does Spelling Matter? (2013).

The recordings produced by the resulting SPARSAR files are, then, a record of failure with respect to my project. But they are instructive failures, nonetheless, and I include them here alongside the “raw” TTS recordings of the poems to illustrate my point: orthography is a key way of encoding sound, yielding both geographic and temporal sonic data. Moreover, such failures – and they are inevitable to a certain extent in any digital analysis of pre-modern English – point out the urgent need to work ever backwards and include ever more variant spellings in software databases. I am reminded of the comical difficulties that iPhone’s Siri software has in deciphering Scots English, though in that scenario the trouble is not necessarily with orthography, but accent.

Imagine, however, Siri attempting to decipher an accurately voiced reading of a Robert Burns poem based on its textual appearance.

Echoing Mulcaster’s statement on orthography, Ralph H. Emerson writes in “English Spelling and Its Relation to Sound” that “alphabetic spelling … [can] be a kind of backbone that supports the flesh and muscle of all the phonetic and phonemic variants in different dialects and idiolects” (260). The problem, as Mulcaster would point out, is the matter of those phonemic variants, a point Emerson concedes: “Western orthography … is largely a tale of how people have squeezed as many values as possible out of the very short Roman alphabet” (262). As Simon Horobin writes, the phoneme /r/ is pronounced differently in England even among different speakers of Northern dialect; the various realizations of /r/ that do not alter meaning require the designation of an “allophone”: [r] (21). In order to encode all the various pronunciations, we would need further phonemic symbols. At the risk of oversimplifying several hundred years of the development of spelling conventions in English, the desire for a less arbitrary connection between letter and sound is at the heart of the debates and attempted reforms in which grammarians like Mulcaster were involved.

Bokenham first with and then without SPARSER.

Bokenham text for SPARSER. Image used with permission by the author.

Bokenham for VARD. Image used with permission by the author.

Yet as much as Mulcaster argues for an orthography freed from “tyrant sound,” as Wesley notes, his arguments keep circling back to sound’s importance. Wesley writes, “Despite his claims regarding the ‘heard’ Z and its subjugation to the ‘sene’ S, the sound of Z creates a variety of problems for Mulcaster; in fact, its sound means Mulcaster must adjust the appearance and frequency of various other letters” (348). The reason for this is the complex set of rules in English regarding how letters interact with and influence each other in shaping the sound of a word. A more systematic analysis of these rules is precisely what allows Emerson to argue that orthography can, in fact, encode sound in English. He writes, “almost any dialect can be described as a plausible and usually predictable realization of the spelling, one word at a time” (265). Emerson describes a “four-step process” for this spelling-based description, which begins with “segment[ing] the spelling into elemental graphemes”; then “assign[ing] the segments their proper graphophonemes, that is, their abstract but systematically universal protovalues.” The next step “shows how the graphophonemes are phonemically realized in particular circumstances within individual dialects.” The example Emerson gives is “hair,” which segments into “H + AI + R, or //her//” (265). From this, he argues, we can reliably derive all the variant pronunciations of “hair.” To complete the process, a litereme is needed; e.g., the litereme <<s>> matches the phoneme //s// but also provides the “natural characteristic spelling” that expresses the “s” and “soft c” in English (266). Emerson concludes,

To describe how spelling encodes sound in a particular language is simply to chart the relationships between segments on these different levels … the litereme is the key: THAT is what everything else is really standing for, spellings and sounds and graphophonemes alike. (The letters themselves can be thought of as archiliteremes, with <<C>>having the reflexes <<k>> and <<s>>, <<A>> having <<ā>> and <<ă>>, etc.) … The simple universal phonology of written English gives birth to the infinite particularities of spoken English. (267)

Fortunately – for the purposes of looking at orthographically expressed Middle English variants, “present orthography still represents the pronunciation of Middle English” (Emerson 267). Unfortunately, though the letter-to-sound relationship in Middle English is much closer, we cannot be sure what those pronunciations were. But we can guess, and orthography is our best (perhaps only) clue.

From letters to sounds. Image by Michael Summers @Flickr CC BY-NC-ND.

From letters to sounds. Image by Michael Summers @Flickr CC BY-NC-ND.

The notion of “suggested” pronunciation is indeed where sound studies offers a compelling model for considering the relationship between poem and scribe, and how the former is recorded orthographically by the latter. Jonathan Sterne’s The Audible Past proposes to construct a “history of sound” that troubles the notion of “face to face” communication as being more “authentic” than sound reproduction technologies, adding, “This history of sound begins by positing sound, hearing, and listening as historical problems rather than as constants on which to build a history” (22). As medievalists know all too well, scribes and scribal variation in the copying of texts constitute a rich field of study when it comes to the question of textual “authenticity.”

Intriguing in light of such variation is Sterne’s idea of “transducers, which turn sound into something else and that something else back into sound” (22) – for which we might read orthography as a kind of technology with “moving parts” that work in concert to reproduce sound, as outlined above. More intriguing, perhaps, is Sterne’s description of the development of “audile technique,” a “practice of listening” that he bases on a study of “virtuosic and highly technical listening skills” during the nineteenth century and the advent of the telegraph, phonograph, and telephone. Sterne writes that with audile technique:

listening became more directional and directed, more oriented toward constructs of private space and private property. The construct of acoustic space as private space in turn made it possible for sound to become a commodity. Audile technique did not occur in the collective, communal space of oral discourse and tradition (if such a space ever existed); it happened in a highly segmented, isolated, individuated acoustic space (24).

In this context, Chaucer’s famous admonition to his scribe “Adam” becomes all the more charged and suggestive (650 in The Riverside Chaucer). We must certainly think of medieval scribes as early practitioners of “audile technique,” taking advantage of orthography as a tool by which to turn sound into a commodity in the form of manuscripts for various occasions and audiences.

Chaucer first with and then without SPARSER.

Chaucer for SPARSER. Image used with permission by the author.

Chaucer for VARD. Image used with permission by the author.

Rethinking medieval texts in this way leaves us with a collaborative sonic performance in which the particular orthographies of the scribes help to pull an author’s text into a certain sound-space, even if it is inexact and in some sense inauthentic. Our ability to “hear” that space, to share it, is limited by our limited mapping of the incredibly multiform ways that English was uttered, and how those utterances were scored on the page by poets and scribes. Wesley notes the importance of discipline to Mulcaster’s educational-grammatical program as set forth in the Elementarie; discipline also hovers over the listening bodies performing audile technique for the sake of increasingly commodified sound spaces described by Sterne. English letters resist such discipline, sliding around various orthographies depending on time, place, poet, and scribe. In order to begin to use programs like SPARSAR to recreate, however tentatively, the sound they encode, we must loosen the standardizing discipline of our technology in parsing letters of the past.

Featured image “Mixed Media Painting” by See-ming Lee @Flickr CC BY-SA

David Hadbawnik is a poet, translator, and medieval scholar. His Aeneid Books 1-6 were published by Shearsman Books in 2015. He is the editor and publisher of Habenicht Press and the journal kadar koli, a co-editor of eth press, which focuses on creative interactions with medieval texts, and associate director of punctum books. Currently, he is an Assistant Professor of English at the American University of Kuwait.

tape reelREWIND! . . .If you liked this post, you may also dig:
The Hysterical Alphabet–John Corbett, Terri Kapsalis, and Danny Thompson

The Eldrich Voice: H.P. Lovecraft’s Weird Phonography–James A. Steintrager

Mouthing the Passion: Richard Rolle’s Soundscapes–Christopher Roman

%d bloggers like this: