Tag Archive | Sound

Sounding Out! Podcast #60: Standing Rock, Protest, Sound and Power (Part 1)

CLICK HERE TO DOWNLOADStanding Rock, Protest, Sound and Power

SUBSCRIBE TO THE SERIES VIA ITUNES

ADD OUR PODCASTS TO YOUR STITCHER FAVORITES PLAYLIST

On March 10th 2017, The Standing Rock Sioux Tribe and Indigenous grassroots leaders called upon allies across the United States and around the world to peacefully March on Washington DC. The March on Washington was to exist, resist, and rise in solidarity with the Indigenous peoples of the world whose rights protect mother earth for the future generations of all. The March on Washington was a reaction to the United States government’s unwillingness to be accountable for the construction recent Dakota Access Pipeline (DAPL) on the Standing Rock Sioux Tribe’s land. This and other subsequent events such as the election of a new administration, increasing threats to native land, and violence of the police have galvanized indigenous communities in the last four months. Thousands have taken to the streets and to rural sites of political occupation.

Join Marcella Ernest as she discusses the sounds of these protests with Nancy Mithlo. They discuss the noises made by the minds, bodies, and songs of those who have taken to public spaces to confront and object to the current political moment. Understanding the sonic elements of protest helps us to better understand how protest is heard and felt.

Marcella Ernest is a Native American (Ojibwe) interdisciplinary video artist and scholar. Her work combines electronic media with sound design with film and photography in a variety of formats; using multi-media installations incorporating large-scale projections and experimental film aesthetics. Currently living in California, Marcella is completing an interdisciplinary Ph.D. in American Studies at the University of New Mexico. Drawing upon a Critical Indigenous Studies framework to explore how “Indianness” and Indigenity are represented in studies of American and Indigenous visual and popular culture, her primary research is an engagement with contemporary Native art to understand how members of colonized groups use a re-mix of experimental video and sound design as a means for cultural and political expressions of resistance.

Nancy Mithlo teaches in the Art History and Visual Arts department at Occidental as an Associate Professor while also working at the Autry in program development, exhibition planning and community outreach. She comes to Occidental from the University of Wisconsin at Madison, where she was an Associate Professor of Art History and American Indian studies. Prior to joining the Wisconsin faculty in 2001, Mithlo taught at Smith College, Santa Fe Community College, the University of New Mexico and the Institute of American Arts.

Featured image “Hey Wells Fargo – No DAPL! Rally” by Joe Piette @Flickr CC BY-NC.

tape reelREWIND! . . .If you liked this post, you may also dig:

Sounding Out! Podcast #47: Finding the Lost Sounds of Kaibah — Marcella Ernest

Sounding Out! Podcast #51: Creating New Worlds From Old Sounds – Marcella Ernest

Sounding Out! Podcast #58: The Meaning of Silence – Marcella Ernest

Advertisements

Playing with the Past in the Imagined Middle Ages: Music and Soundscape in Video Game

series co-editors Dorothy Kim and Christopher Roman

Each of the essays in our “Medieval Sound” forum focuses on sound as it, according to Steve Goodman’s essay “The Ontology of Vibrational Force,” in The Sound Studies Reader“comes to the rescue of thought rather than the inverse, forcing it to vibrate, loosening up its organized or petrified body (70).  These investigations into medieval sound lend themselves to a variety of presentation methods loosening up the “petrified body” of academic presentation. Each essay challenges concepts of how to hear the Middle Ages and how the sounds of the Middle Ages continue to echo in our own soundscapes.

Read all the previous posts here, and, HEAR YE!, in April 2017, look for a second series on Aural Ecologies of noise! –Guest Editors Dorothy Kim and Christopher Roman

Freed though they have been from the historiographical pit of the Dark Ages, the Middle Ages inevitably slip ever further into the past. Nonetheless, they have never been easier to visit. We have but to open our computers or turn on our televisions to be transported into the past. As any good Sci-fi show will tell you, we must be careful when we travel into the past; we can change things.

The medium on which I wish to focus – videogame – relies precisely upon on this ability to affect change. It takes aspects of filmic medievalism but must also confront an intrinsic interactivity. This interactive capacity may seem to authenticate further the experience of the past by creating a rich and responsive world but it also frees aspects of narrative agency from the control of game designers, composers, and sound engineers. In this article, I will demonstrate some of the ways in which issues of space/place, identity, orientalism/otherness, and the norms of the medium itself can play out. In recomposing the past – be that with a nod to authenticity, within the realms of historical metafiction, or even the imagined (neo-)medievalism of the fantasy genre – videogames create something that sits between the past and present that nonetheless has a profound effect on the public conception of the medieval soundscape. My focus here is on CD Projekt Red’s high-fantasy game The Witcher III: Wild Hunt, addressing not only the musical score but also wider aspects of soundscape such as vocal accent, foley, and manipulation of the aural field.

Still from The Witcher III: Wild Hunt.

Still from The Witcher III: Wild Hunt.

.

The genre of fantasy could be described as medievalist in origin and aesthetic, taking clear inspiration from the medieval world and often using it, or a close approximation, as a geographical, historical, and cultural setting. Many of the more fantastical elements of fantasy too are drawn from medieval bestiaries and the genre of the medieval romance. The late Umberto Eco popularised the term neo-medievalism, albeit in a rather pejorative sense and in opposition to ‘responsible philological study’, to describe this interaction between medieval history and the fantastical. Perhaps due to the rather negative associations of neo-medievalism, both terms tend to be used somewhat interchangeably today.

A divide could perhaps be suggested as to whether medieval aspects are presented as unproblematic and nostalgic, or treated in a more critical and distanced manner. For instance, David Marshall defines neo-medievalism as ‘a self-conscious, ahistorical, non-nostalgic imagining or reuse of the historical Middle Ages that selectively appropriates iconic images…to construct a presentist space that disrupts traditional depictions of the medieval.’ In contrast, Kim Selling notes of medievalism that, after the breakdown of modernist historical metanarratives, the pre-modern world of the medieval offers a ‘rich, satisfying, and authentic’ counterpoint to the ‘profound social, spiritual, and political dislocation’ of postmodernism. From this viewpoint, the world of medievalist fantasy offers pure escapism back to a world in which old certainties can be re-asserted; perhaps, as Elkins notes, a reaction against the rationalistic, anti-heroic, materialist, and empiricist bent of modern society.

Regardless of the author’s framing of ‘the medieval’, it offers many narrative advantages. As Kim Selling has noted, in placing an imagined world in a simplified version of the Western European Middle Ages, the fantasy author can make use of narrative shorthands. The world of kings, queens, knights, peasants, dragons, magic, witches, and elves is already known from myth and fairy tale. These aspects require no explanation but rather build upon a tradition of understood western folkloric conventions – conventions that can also imply certain types of social structure.

The Witcher is certainly situated in a simplified medieval Europe (see the similarity between the coasts of Poland and Nilfgaard) and it clearly occupies a world with presentist concerns. More than this, its extreme use of narrative indeterminacy betrays a staunch adherence to postmodern concepts of storytelling and a distinct anti-heroic edge. The player navigates a world of many shades of grey, often choosing the lesser of many evils. They make decisions – which often result in unintended consequences – based on their own morality and desired outcomes. This said, perhaps the question of whether it could be defined as medievalist or neo-medievalist depends on the actions of the player and precisely the kind of narrative they construct.

An early example of unintended consequences and choosing the lesser of many evils

The visual world inhabited by The Witcher certainly relies on appropriating iconic images; the analogue aural world on appropriating iconic sounds, accents, and musical ideas as the player continuously explores regions with distinct identities, social structures, urban/pastoral settings, and religions. The rational, urban, and merchantile world of Oxenfurt (a city famous for its world-class university with a crest clearly based on that of the University of Cambridge and a name that hints at the university town of Oxford’s medieval etymology) and Novigrad are clearly at odds with the rural peasant life of the Velen wastelands or the Celtic imagery of the Skellige isles.

screen-shot-2016-10-02-at-8-00-15-pm

The landscape and soundscape of Oxenfurt

The landscape and soundscape of Novigrad

The landscape and soundscape of Velen (with multiple examples of the ‘combat music’)

The landscape and soundscape of Skellige (with the ‘combat music’ adapted for this area)

Like much medievalist fantasy the tension between rationalism and the fantastical is one of the central elements of dramatic tension. As the great cities of this world grow and the expansionist Nilfgaardian Empire press forward, the pre-modern world of The Witcher shrinks. Allied to the spread of rationalism, we also see the ‘Church of the Eternal Fire’ (an identifiable stand-in for Christianity) seek to eradicate magic, non-human populations, and more identifiably ‘pagan’ religions. The architecture of its buildings invite comparison with European sacred architecture and its witch hunters and inquisitions with particularly regrettable episodes of European history.

The Temple of the Eternal Fire burning people at the stake

Just as the visuals employ iconic imagery to imply certain ideas, the soundscape’s most important function is to provide the aural analogue of this. One aspect is the score. Within this, individual musical stems are layered in response to player actions. As the composer Marcin Przybylowicz noted in a recent interview with the Tech Times ‘[t]he cues (that were interactive) are divided into smaller layers, which come together, in the case of a combat cue, only when we are dealing with a very powerful enemy. If the enemy is small … only the first layer of the piece will play’. These layers are re-orchestrated in each area so as to preserve its aural identity. In a separate interview with IC-Radio.de, Przybylowicz notes:

No Mans (sic.) Land [Velen] is a war ravaged land… It’s also full of slavic references, pagan beliefs etc. …. Then, there’s Novigrad – the biggest city in northern kingdoms. … I decided music in Novigrad should be more civilized – that’s why there are lots of string instruments playing there (dulcimer, bouzouki, guitars, lutes, cimbalom etc.), and overall tone of the music is lighter, [and] reminds [me] a bit music of [the] Renaissance. Finally, [the] Skellige Isles – [a] region with Celtic, Scottish and Norse references, that had to be reflected in music as well. Use of bagpipes, flutes and Scandinavian folk instruments corresponds with that setting. On top of that, I had to think how it would all work together. That’s where our themes come in … We use those themes in every major location …. [and] we reorchestrate them with instruments corresponding to a particular region.

Combat music in Velen and Skellige respectively. Note how both utilise the orchestration of their own respective areas to alter the main theme and how additional layers are added to the music depending on the intensity of the fighting.

In viewing the different medievalisms on offer in this game, a comparison of the three areas mentioned above is instructive. The Skellige isles, an archipelago which looks curiously like Scottish islands, are occupied by inhabitants with Irish accents. As can be heard in the videos above, the clearly Celtic-influenced music of this area supports this association and the relative lack of diegetic music in this area, combined with natural sounds (the sea, wind, storms) combine to give a sense that the music is a part of the geography. Celtic folk music as a shorthand for the Middle Ages is nothing new. Simon Nugent has noted the tendency of many historically-situated films to draw on Celtic influence. His work has shown that the creation of ‘Celtic’ folk has little to do with a discrete geographical area or with historical accuracy but rather is a modern marketing creation that plays on associations with nature and an escape from modernity. This is precisely the case in The Witcher where, rather than utilising a real historical Celtic medieval repertory, it instead draws on aural cues from the popular medievalism of the filmic soundscape tradition, filtered through the need for an indeterminate score. This ‘packaging’ brings with it associations of an ‘authentic’ Celtic folk tradition as a remnant of the ‘true folk tradition’ that once existed for everyday people elsewhere.

We can perhaps see a link to the works of fantasy writers such as Gael Baudino and Patricia Kennealy-Morrison who turn to pagan Celtic sources as an alternative to what they perceive as the medieval Christian degradation of women, as Jane Tolmey has noted. This association between pagan prehistory, matriarchy, and freedom for women seems a common theme in the popular conception of the past, echoed by theorists such as Albert Classen. The fact that Skellige evokes a recognisably ‘Celtic’ soundscape (relying heavily on the Polish Folk band Percival who collaborated on both new works and who took several from a previous album which were then adapted for indeterminate playback) therefore comes with many associations, drawn almost entirely from the filmic soundscape tradition. This is a pre-modern, pagan land; a land with an authentic peasant class: roughhewn but honest. This Celtic imagery and soundscape also offers a counterpoint to the sexual politics of other areas. Women can more easily participate in areas which might otherwise be seen as male-dominated: depending on the actions of the player, a woman may rule Skellige. A matriarchal class of priestesses govern the region’s predominant religion in stark contrast to the male Priests in other areas. The soundscape here is therefore absolutely crucial to the identity of this area. In creating a Celtic sonic identity – equal part music and accent – the game designers have created a rich culture that need only be hinted at to be understood.

By contrast, the urban world of Novigrad and Oxenfurt is far less folk-influenced. Unlike the other areas of the game, it does not draw so heavily on either the pre-existent or newly composed music of Percival, make such explicit use of folk instrumentation, and seems far more closely related to a Renaissance dance music tradition. The urban/pastrol divide is enhanced by the sounds of a busy city compared to the sounds of nature and the frequent cries of the townsfolk give a sense of bustling urban life (this time with accents from the North of England – compare HBO’s Game of Thrones).

Compared to the wind instruments and female vocals of Skellige, percussion and plucked/struck strings predominate. There is indeed more of a Renaissance feel in an area already touched by modernity. Most notably, there is more diegetic music in these two cities. We frequently see and hear small Renaissance dance bands, using period instruments, entertaining crowds (watch from around 3:45 of the above video of Novigrad for an example; note how diegetic music slowly enters the soundtrack as they are approached). Perhaps the most significant diegetic moment, however, is the song by the Trobaritz Priscilla.  The audio and visuals are surprisingly well matched, and the tuning of the lute adds emphasis to the fact of live performance. As a video cut-scene, this is one of few moments in the game where the player has no ability to affect their environs and must simply watch and listen.

Priscilla’s song

Far more so than in other areas, the music in Novigrad and Oxenfurt is for and by people. This is in marked contrast to the soundscapes found on Skellige and in Velen where the music is almost purely non-diegetic. In contrast to the pre-Christian matriarchal associations of the priestesses of Skellige is the aural handling of the Temple of the Eternal Fire in Novigrad and its male priests. As Adam Whittaker has recently identified, there is a clear link between musicological discourse on purely vocal performance in Christian sacred space in Early Music, and its representation on screen. Male a capella voices, and a ‘chanting’ vocal style (again hinting at plainchant, rather than using pre-existent chant music), are often used to denote the aural identity of a church. Precisely this kind of vocal delivery is added to the soundtrack as the player moves closer to the Temple of the Eternal Fire, explicitly linking this Christian association with the aural presence of the Temple. The contrast with the female-dominated vocals elsewhere enhances the distinction and re-enforces the links between the ostensibly pre-Christian worlds of Skellige and Velen and the Christian associations made in Novigrad.

Approaching the Temple of the Eternal Fire

The music of the Velen wasteland and the neighbouring White Orchard, like Skellige, is more folk influenced, yet clearly distinct. Gone are the Celtic folk influences; instead, this area fuses a cinematic pastoral idyll (again laden with nature sounds and with a peasantry speaking with West Country accents – compare the idyllic ‘Shire’ in Peter Jackson’s The Lord of the Rings) with a dark and sinister undertone which draws on the ‘otherness’ of many non-western and folk instruments, particularly the kemenche, electric cello, hurdy gurdy, bowed gusli, gheychak, and the bowed yaylı – the vast majority of which only enter the soundscape in this area. The use of the medieval as a source of dangerous and primitive ‘otherness’ is common at the moment (evidinced, for instance, by the many recent descriptions in the West of the so-called Islamic State as ‘medieval’) and draws on modernist thought which characterized the middle ages as a period of dark and dangerous alterity between the glories of antiquity and the Renaissance.

That the soundscape of The Witcher draws on competing categorisations of the medieval as ‘dangerous’ and ‘pastoral’ says much about the mutability of medievalisms. Musically, this ‘otherness’ can be expressed as a kind of orientalism, both exotic and dangerous, and the microtonal inflections and use of glissandi here give a sinister undertone to what is otherwise a quintessentially pastoral film score. This area has one of the most memorable parts of the entire soundtrack ‘Ladies of the Wood’, underscoring a genuinely horrific narrative and visuals (some of the consequences of which are shown in the first video of this post with a lengthier section of the music given below). The exotic instrumentation combined with the driving repetition, at odds with an audio usually so responsive to the player’s actions, makes the experience unsettling and claustrophobic.

Ladies of the Wood

Taken together, these very distinctive sound worlds serve to demonstrate some of the many medieval soundscapes which permeate our collective consciousness. In utilising iconic aural cues, composers and sound designers in a neo-medievalist tradition can conjure up particular cultural and social structures with ease, taking many of the shorthands which have emerged from the TV and film traditions in recent years. The indeterminacy inherent to videogame, and the common response of using audio stems, means that the soundscape moves far beyond what is possible in TV and film. However, it also makes problematic the concept of using real, historically-informed music from the period being invoked which would not be able to respond to the interactivity of the world. In The Witcher, the decisions we make effect the world around us, including its soundscape. The effect is crucial both to helping to conjure the world of The Witcher and to helping us feel immersed in it – perhaps paradoxically this may make the soundworld invoked seem more authentic even if it simultaneously reduces the possibility of using ‘authentic’ historical repertoire.

Featured image by Carlos Santos @YouTube CC BY.

James Cook is a University Teacher in Music at the University of Sheffield. His work focuses on the musical period that falls neatly between the Middle Ages and the Renaissance. In particular, the ways that musical cultures in this period interact and how expatriate groups (merchants, clergy, and nobility) imported and used music. Some of his work (like this essay) concerns the representation of early music on stage and screen, be that the use of ‘real’ early music in multimedia productions, the imaginative re-scoring of historical dramas, or even the popular medievalism of the fantasy genre.

tape reelREWIND! . . .If you liked this post, you may also dig:
Sounding Out! Podcast #49: Sound and Sexuality in Video Games – Milena Droumeva and Aaron Trammell

Sounding Out! Podcast #29: Game Audio Notes I: Growing Sounds for Sim Cell–Leonard J. Paul

Papa Sangre and the Construction of Immersion in Audio Games— Enongo Lumumba-Kasongo

The Dark Side of Game Audio: The Sounds of Mimetic Control and Affective Conditioning – Aaron Trammell

Sounding Out! Podcast #54: The Sound of Magic

Medieval SoundEach of the essays in this month’s “Medieval Sound” forum focuses on sound as it, according to Steve Goodman’s essay “The Ontology of Vibrational Force,” in The Sound Studies Reader“comes to the rescue of thought rather than the inverse, forcing it to vibrate, loosening up its organized or petrified body (70).  These investigations into medieval sound lend themselves to a variety of presentation methods loosening up the “petrified body” of academic presentation. Each essay challenges concepts of how to hear the Middle Ages and how the sounds of the Middle Ages continue to echo in our own soundscapes.

The posts and podcast in this series begins an ongoing conversation about medieval sound in Sounding Out!. Our opening gambit in April 2016, “Multimodality and Lyric Sound,” reframes how we consider the lyric from England to Spain, from the twelfth through the sixteenth centuries, pushing ideas of openness, flexibility, and productive creativity. We will post several follow-ups throughout the rest of 2016 focusing on “Remediating Medieval Sound.”  And, HEAR YE!, in April 2017, look for a second series on Aural Ecologies of noise! –Guest Editors Dorothy Kim and Christopher Roman

CLICK HERE TO DOWNLOADThe Sound of Magic

SUBSCRIBE TO THE SERIES VIA ITUNES

ADD OUR PODCASTS TO YOUR STITCHER FAVORITES PLAYLIST

Medieval charms run the gamut from offering protection for journeys (travel was often perilous) to warding your cattle from thieves (the runic letter for ‘cattle’ also means ‘wealth’) to various kinds of healing for people, animals and even the earth. Many of them include verses that are meant to be sung.

What is the sound of magic? How do you sing it properly without notation? Does it affect the efficacy of the charm if you sing it wrong?

‘Sing ðis gealdor’ Sing this charm the Anglo-Saxon texts command. The words are even linked as ‘galdorsangas’ incantations, but the doom-and-gloom 11th century preacher Archbishop Wulfstan uses that term in the pejorative sense of things to avoid, lumping it together with ‘sorceries’ as things to avoid. In its time the right way of singing was understood but, as is the case about much of the social context, we have lost the specifics.

How to recreate an Anglo-Saxon charm in a modern sound file then? If you’re going to do it right, how do you capture the magic in a way that’s true to the source material and yet accessible to a modern audience (even if it’s just my students)? I was determined to do it and do it right.

K. A. Laity is the author of the novels White RabbitKnight of the White HartA Cut-Throat BusinessLush SituationOwl StretchingPelzmantelThe Mangrove LegacyChastity Flame and the collections Unquiet Dreams and Unikirja, as well as editor of Weird NoirNoir Carnival and Drag Noir, writer of other stories, plays and essays. Her stories tend to slip across genres and categories, but all display intelligence and humour. Myths and fairy tales influence much of her writing. The short stories in Dreambook [originally Unikirja] found their inspiration from The Kalevala, Kanteletar, and other Finnish myths and legends: the stories won the 2005 Eureka Short Story Fellowship and a 2006 Finlandia Foundation grant.

Dr. Laity teaches medieval literature, film, digital humanities and popular culture at the College of Saint Rose, though she was at NUI Galway as a Fulbright scholar for the 2011-2 academic year.

REWIND!…If you liked this post, you may also dig:tape reel

‘A Clateryng of Knokkes’: Multimodality and Performativity in “The Blacksmith’s Lament”–Katherine Jager

Mouthing the Passion: Richard Rolle’s Soundscapes–Christopher Roman

EPISODE LI: Creating New Words from Old Sounds–Marcella Ernest, Candace Gala, Leslie Harper, and Daryn McKenny

Optophones and Musical Print

The word type, as scanned by the optophone.

From E.E. Fournier d’Albe, The Moon Element (New York: D. Appleton & Company, 1924), 141.

 

In 1912, British physicist Edmund Fournier d’Albe built a device that he called the optophone, which converted light into tones. The first model—“the exploring optophone”—was meant to be a travel aid; it converted light into a sound of analogous intensity. A subsequent model, “the reading optophone,” scanned print using lamp-light separated into beams by a perforated disk. The pattern of light reflected back from a given character triggered a corresponding set of tones in a telephone receiver. d’Albe initially worked with 8 beams, producing 8 tones based on a diatonic scale. He settled on 5 notes: lower G, and then middle C, D, E and G. (Sol, do, re, mi, sol.) The optophone became known as a “musical print” machine. It was popularized by Mary Jameson, a blind student who achieved reading speeds of 60 words per minute.

Photograph of the optophone, an early scanner with a rounded glass bookrest.

Reading Optophone, held at Blind Veterans UK (formerly St. Dunstan’s). Photographed by the author. With thanks to Robert Baker for helping me search through the storeroom to locate this item, which was previously uncatalogued and “lost” for many years.

 

Scientific illustration of the optophone, showing a book on the bookrest and a pair of headphones for listening to the tonal output.

Schematic of optophone from Vetenskapen och livet (1922)

In the field of media studies, the optophone has become renowned through its imaginary repurposings by a number of modernist artists. For one thing, the optophone finds brief mention in Finnegan’s Wake. In turn, Marshall McLuhan credited James Joyce’s novel for being a new medium, turning text into sound. In “New Media as Political Forms,” McLuhan says that Joyce’s own “optophone principle” releases us from “the metallic and rectilinear embrace of the printed page.” More familiar within media studies today, Dada artist Raoul Hausmann patented (London 1935), but did not successfully build, an optophone presumably inspired by d’Albe’s model, which he hoped would be employed in audiovisual performances. This optophone was meant to convert sound into light as well as the reverse. It was part of a broader contemporary impulse to produce color music and synaesthetic art. Hausmann also wrote optophonetic poetry, based on the sounds and rhythms of “pure phonemes” and non-linguistic noises. In response, Francis Picabia painted two optophone portraits in 1921 and 22. Optophone I, below, is composed of lines that might be sound waves, with a pattern that disorders vision.

Francis Picabia's Optophone I, a series of concentric black circles with a female figure at the center.

Francis Picabia, Optophone I (1922)

Theorists have repeatedly located Hausmann’s device at the origin of new media. Authors in the Audiovisuology, Media Archaeology, and Beyond Art: A Third Culture anthologies credit Hausmann’s optophone with bringing-into-being cybernetics, digitization, the CD-ROM, audiovisual experiments in video art, and “primitive computers.” It seems to have escaped notice that d’Albe also used the optophone to create electrical music. In his book, The Moon Element, he writes:

Needless to say, any succession or combination of musical notes can be picked out by properly arranged transparencies, and I have succeeded in transcribing a number of musical compositions in this manner, which are, of course, only audible in the telephone. These notes, in the absence of all other sounding mechanism, are particularly pure and free from overtones. Indeed, a musical optophone worked by this intermittent light, has been arranged by means of a simple keyboard, and some very pleasant effects may thus be obtained, more especially as the loudness and duration of the different notes is under very complete and separate control.

E.E. Fournier d’Albe, The Moon Element (New York: D. Appleton & Company, 1924), 107.

d’Albe’s device is typically portrayed as a historical cul-de-sac, with few users and no real technical influence. Yet optophones continued to be designed for blind people throughout the twentieth century; at least one model has users even today. Musical print machines, or “direct translators,” co-existed with more complex OCR-devices—optical character recognizers that converted printed words into synthetic speech. Both types of reading machine contributed to today’s procedures for scanning and document digitization. Arguably, reading optophones intervened more profoundly into the order of print than did Hausmann’s synaesthetic machine: they not only translated between the senses, they introduced a new symbolic system by which to read. Like braille, later vibrating models proposed that the skin could also read.

In December 1922, the Optophone was brought to the United States from the United Kingdom for a demonstration before a number of educators who worked with blind children; only two schools ordered the device. Reading machine development accelerated in the U.S. around World War II. In his position as chair of the National Defense Research Committee, Vannevar Bush established a Committee on Sensory Devices in 1944, largely for the purpose of rehabilitating blind soldiers. The other options for reading—braille and Talking Books—were relatively scarce and had a high cost of production. Reading machines promised to give blind readers access to magazines and ephemeral print (recipes, signs, mail), which was arguably more important than access to books.

Piechowski, wearing a suit, scans the pen of the A-2 reader over a document.

Joe Piechowski with the A-2 reader. Courtesy of Rob Flory.

At RCA (Radio Corporation of America), the television innovator Vladimir Zworykin became involved with this project. Zworykin had visited Fournier d’Albe in London in the 19-teens and seen a demonstration of the optophone. Working with Les Flory and Winthrop Pike, Zworykin built an initial machine known as the A-2 that operated on the same principles, but used a different mechanism for scanning—an electric stylus, which was publicized as “the first pen that reads.” Following the trail of citations for RCA’s “Reading Aid for the Blind” patent (US 2420716A, filed 1944), it is clear that the “pen” became an aid in domains far afield from blindness. It was repurposed as an optical probe for measuring the oxygen content of blood (1958); an “optical system for facsimile scanners” (1972); and, in a patent awarded to Burroughs Corporation in 1964, a light gun. This gun, in turn, found its way into the handheld controls for the first home video game system, produced by Sanders Associates.

The A-2 optophone was tested on three blind research subjects, including ham radio enthusiast Joe Piechowski, who was more of a technical collaborator. According to the reports RCA submitted to the CSD, these readers were able to correlate the “chirping” or “tweeting” sounds of the machine with letters “at random with about eighty percent accuracy” after 60 hours of practice. Close spacing on a printed page made it difficult to differentiate between letters; readers also had difficulty moving the stylus at a steady pace and in a straight line. Piechowski achieved reading speeds of 20 words per minute, which RCA deemed too slow.

Attempts were made to incorporate “human factors” and create a more efficient tonal code, to reduce reading time as well as learning time and confusion between letters. One alternate auditory display was known as the compressed optophone. Rather than generate multiple tones or chords for a single printed letter, which was highly redundant and confusing to the ear, the compressed version identified only certain features of a printed letter: such as the presence of an ascender or descender. Below is a comparison between the tones of the original optophone and the compressed version, recorded by physicist Patrick Nye in 1965. The following eight lower case letters make up the source material: f, i, k, j, p, q, r, z.

Original record in the author’s possession. With thanks to Elaine Nye, who generously tracked down two of her personal copies at the author’s request. The second copy is now held at Haskins Laboratories.

An image of the letter r as scanned by the optophone and compressed optophone.

From Patrick Nye, “An Investigation of Audio Outputs for a Reading Machine,” AFB Research Bulletin (July 1965): 30.

 

Because of the seeming limitations of tonal reading, RCA engineers re-directed their research to add character recognition to the scanning process. This was controversial, direct translators like the optophone being perceived as too difficult because they required blind people to do something akin to learning to read print—learning a symbolic tonal or tactile code. At an earlier moment, braille had been critiqued on similar grounds; many in the blind community have argued that mainstream anxieties about braille sprang from its symbolic difference. Speed, moreover, is relative. Reading machine users protested that direct translators like the optophone were inexpensive to build and already available—why wait for the refinement of OCR and synthetic speech? Nevertheless, between November 1946 and May 1947, Zworykin, Flory, and Pike worked on a prototype “letter reading machine,” today widely considered to be the first successful example of optical character recognition (OCR). Before reliable synthetic speech, this device spelled out words letter by letter using tape recordings. The Letter-Reader was too massive and expensive for personal use, however. It also had an operating speed of 20 words per minute—thus it was hardly an improvement over the A-2 translator.

Haskins Laboratories, another affiliate of the Committee on Sensory Devices, began working on the reading machine problem around the same time, ultimately completing an enormous amount of research into synthetic speech and—as argued by Donald Shankweiler and Carol Fowler—the “speech code” itself. In the 1940s, before workable text-to-speech, researchers at Haskins wanted to determine whether tones or artificial phonemes (“speech-like speech”) were easier to read by ear. They developed a “machine dialect of English,” named wuhzi: “a transliteration of written English which preserved the phonetic patterns of the words.” An example can be played below. The eight source words are: With, Will, Were, From, Been, Have, This, That.

Original record in the author’s possession. From Patrick Nye, “An Investigation of Audio Outputs for a Reading Machine” (1965). With thanks to Elaine Nye.

Based on the results of tests with several human subjects, the Haskins researchers concluded that aural reading via speech-like sounds was necessarily faster than reading musical tones. Like the RCA engineers, they felt that a requirement of these machines should be a fast rate of reading. Minimally, they felt that reading speed should keep pace with rapid speech, at about 200 words per minute.

Funded by the Veterans Administration, members of Mauch Laboratories in Ohio worked on both musical optophones and spelled-speech recognition machines from the 1950s into the 1970s. One of their many devices, the Visotactor, was a direct-translator with vibro-tactile output for four fingers. Another, the Visotoner, was a portable nine-channel optophone. All of the Mauch machines were tested by Harvey Lauer, a technology transfer specialist for the Veterans Administration for over thirty years, himself blind. Below is an excerpt from a Visotoner demonstration, recorded by Lauer in 1971.

Visotoner demonstration. Original 7” open reel tape in author’s possession. With thanks to Harvey Lauer for sharing items from his impressive collection and for collaborating with the author over many years.

Lauer's fingers are pictured in the finger-rests of the Visotactor, scanning a document.

Harvey Lauer reading with the Visotactor, a text-to-tactile translator, 1977.

Later on the same tape, Lauer discusses using the Visotoner to read mail, identify currency, check over his own typing, and read printed charts or graphics. He achieved reading speeds of 40 words per minute with the device. Lauer has also told me that he prefers the sound of the Visotoner to that of other optophone models—he compares its sound to Debussy, or the music for dream sequences in films.

Mauch also developed a spelled speech OCR machine called the Cognodictor, which was similar to the RCA model but made use of synthetic speech. In the recording below, Lauer demonstrates this device by reading a print-out about IBM fonts. He simultaneously reads the document with the Visotoner, which reveals glitches in the Cognodictor’s spelling.

Original 7” open reel tape in the author’s possession. With thanks to Harvey Lauer.

A hand uses the metal probe of the Cognodictor to scan a typed document.

The Cognodictor. Glendon Smith and Hans Mauch, “Research and Development in the Field of Reading Machines for the Blind,” Bulletin of Prosthetics Research (Spring 1977): 65.

In 1972, at the request of Lauer and other blind reading machine users, Mauch assembled a stereo-optophone with ten channels, called the Stereotoner. This device was distributed through the VA but never marketed, and most of the documentation exists in audio format, specifically in sets of training tapes that were made for blinded veterans who were the test subjects. Some promotional materials, such as the short video below, were recorded for sighted audiences—presumably teachers, rehabilitation specialists, or funding agencies.

Mauch Stereo Toner from Sounding Out! on Vimeo.

Video courtesy of Harvey Lauer.

Mary Jameson corresponded with Lauer about the stereotoner, via tape and braille, in the 1970s. In the braille letter pictured below she comments, “I think that stereotoner signals are the clearest I have heard.”

Scan of a braille letter from Jameson to Lauer.

Letter courtesy of Harvey Lauer. Transcribed by Shafeka Hashash.

In 1973, with the marketing of the Kurzweil Reader, funding for direct translation optophones ceased. The Kurzweil Reader was advertised as the first machine capable of multi-font OCR; it was made up of a digital computer and flatbed scanner and it could recognize a relatively large number of typefaces. Kurzweil recalls in his book The Age of Spiritual Machines that this technology quickly transferred to Lexis-Nexis as a way to retrieve information from scanned documents. As Lauer explained to me, the abandonment of optophones was a serious problem for people with print disabilities: the Kurzweil Readers were expensive ($10,000-$50,000 each); early models were not portable and were mostly purchased by libraries. Despite being advertised as omnifont readers, they could not in fact recognize most printed material. The very fact of captchas speaks to the continued failures of perfect character recognition by machines. And, as the “familiarization tapes” distributed to blind readers indicate, the early synthetic speech interface was not transparent—training was required to use the Kurzweil machines.

Original cassette in the author’s possession. 

A young Kurzweil stands by his reading machine, demonstrated by Jernigan, who is seated.

Raymond Kurzweil and Kenneth Jernigan with the Kurzweil Reading Machine (NFB, 1977). Courtesy National Federation of the Blind.

Lauer always felt that the ideal reading machine should have both talking OCR and direct-translation capabilities, the latter being used to get a sense of the non-text items on a printed page, or to “preview material and read unusual and degraded print.” Yet the long history of the optophone demonstrates that certain styles of decoding have been more easily naturalized than others—and symbols have increasingly been favored if they bear a close relation to conventional print or speech. Finally, as computers became widely available, the focus for blind readers shifted, as Lauer puts it, “from reading print to gaining access to computers.” Today, many electronic documents continue to be produced without OCR, and thus cannot be translated by screen readers; graphical displays and videos are largely inaccessible; and portable scanners are far from universal, leaving most “ephemeral” print still unreadable.

Mara Mills is an Assistant Professor of Media, Culture, and Communication at New York University, working at the intersection of disability studies and media studies. She is currently completing a book titled On the Phone: Deafness and Communication Engineering. Articles from this project can be found in Social Text, differences, the IEEE Annals of the History of Computing, and The Oxford Handbook of Sound Studies. Her second book project, Print Disability and New Reading Formats, examines the reformatting of print over the course of the past century by blind and other print disabled readers, with a focus on Talking Books and electronic reading machines. This research is supported by NSF Award #1354297.

%d bloggers like this: