Archive | Digital Media RSS for this section

Further Experiments in Agent-based Musical Composition

Editor’s Note:  WARNING: THE FOLLOWING POST IS INTERACTIVE!!!  This week’s post is especially designed by one of our regulars, Andreas Duus Pape, to spark conversation and provoke debate on its comment page.  I  have directly solicited feedback and commentary from several top sound studies scholars, thinkers, artists, and musicians, who will be posting at various times throughout the day and the week–responding to Andreas, responding to each other, and responding to your comments.  Look for commentary by Bill Bahng Boyer (NYU), Maile Colbert(Binaural/Nodar, Faculdade de Belas Artes da Universidade do Porto), Adriana Knouf(Cornell University), Primus Luta (AvantUrb, Concrète Sound System), Alejandro L. Madrid (University of Illinois at Chicago), Tara Rodgers (University of Maryland), Jonathan Skinner (ecopoetics),  Jonathan Sterne (McGill University), Aaron Trammell (Rutgers University, Sounding Out!) and yours truly (Binghamton University, Sounding Out!).  Full bios of our special respondents follow the post. We wholeheartedly wish to entice you this Monday to play. . .and listen. . .and then share your thoughts via the comment page. . .and play again. . .listen again. . .read the comments. . .and share more thoughts. . .yeah, just go ahead and loop that.  –JS, Editor-in-Chief

DOI: https://doi.org/10.59350/ybq4g-k1x93

I’m a musician and an economist. Sometimes you will find me playing acoustic folk rock and blues on guitar, harmonica and voice. And at other times I will be at work, where I apply my expertise in game theory to the computer modeling of social phenomena. I create simulations of people interacting – such as how people decide which way to vote on an issue such as a tax levy, or how people learn to sort objects given to them in an experiment. In these simulations, the user can set up characteristics of the environment, such as the number of people and their individual goals. After things are set up, users watch these interactions unfold. The simulation is a little story, and one need only tweak the inputs to see how the story changes.

As a musician, I was curious if a program that generates social stories could be refashioned to generate musical pieces. I wanted to build a music-generation engine that the listener could tweak in order to get a different piece each time. But not just any tune – a piece with some flow, some story. I like that tension between randomness and structure. On one hand, I want every song to vary in unpredictable ways; on the other hand, I want to create music and not structureless noise.

I created a basic story of predators and prey, whimsically naming the prey “Peters,” represented by rabbits, and the predators “Wolves.” My simulation depicts a plain in the savannah with a green oasis. The prey seek the oasis and the predators seek the prey. Each character has its own goals and the closer they are to achieving them, the happier they are. Both predators and prey want to have stomachs full of food, so naturally they want to be close to their target (be it prey or oasis). As they travel through the savannah, they learn what choices (directions of movement) make them happier, and use this experience to guide them.

Photo by bantam10 @Flickr

So how does this story become music? To this question there are two answers: a technical one and an intuitive one. The intuitive answer is that in real life the story of predators and prey plays out geographically on the savannah, but musically this is a story that plays out over a sonic landscape. To elaborate, I abstracted the movement of the prey and predator on the geography of the plain into the musical geometry of a sonic landscape. The farther north an agent travels, the higher the pitch. And, the farther east an agent travels the longer the duration. In other words, as an agent travels to the northwest, she makes longer-lasting tones that are higher pitched. I also mapped happiness to volume, so that happy agents make louder tones. Finally, so that each agent would have a distinct voice as they traveled through this space, I chose different instruments for each agent.

In the video below I assigned the “church organ” sound to prey, and the “brass section” sound to predators.

Ultimately, there are some things that I like about this piece and others that I do not.

As a harmonica player, I improvise by creating and resolving tension. I think this piece does that well. The predator will pursue the prey into a quiet, low-pitch corner, creating a distant, rumbling sound – only to watch prey escape to the densely polyphonic northwest corner. There is an ebb and flow to this chase that I recognize from blues harmonica solos. In contrast to my experience as a harmonica player, however, I have found that some of the most compelling parts of the dynamics come from the layering of notes. The addition of notes yields a rich sonic texture, much like adding notes to a chord on an organ.

Unfortunately, for largely technical reasons, there is a lack of coherent rhythm and pacing. The programming platform (agent-based modeling software called NetLogo) is not designed to have the interface proceed in real-time. Basically, the overall speed of the piece can change as the processing load increases or decreases. I found that as agents learnt more about their surroundings (and more system resources are allocated to this “memory”), they became slower and slower. To fix this, I capped the size of their memory banks so that they would forget their oldest memories. The closest I have come to a rhythmic structure is by ordering the way that the agents play. This technique makes the piece have a call-and-response feel. If only the piece to had a coherent rhythm,  then I could imagine playing harmonica along with it.

One last comment on pitch: while an earlier version of this piece mapped each step in space to a semitone, things sounded too mechanical. Even though this was the easiest and most intuitive decision from a technical standpoint, it was aesthetically lacking, so I have now integrated traditional musical scales. The minor scale, in my opinion, is the most interesting as it makes the predator/prey dynamic sound appropriately foreboding.

Photo by deivorytower @Flickr.

You can play this piece yourself. Simply go to this link with Java enabled in your browser (recommended: Google Chrome). Pressing “Setup” then “Go” will create your own run of the piece. As it is running, you can adjust the slider above the graphic window to change the speed. Press “Go” again to stop the model, adjust any parameters you wish and press “Setup” and “Go” again to see how the piece changes. Here are some parameters to try: instA and instB to change the instruments associated with prey and predators; PlayEveryXSteps to change the pace of the piece (higher results in a slower paced piece); Num-PackAs and Num-PackBs changes the number of prey and predators; the vertical PeterVol and WolfVol adjust the overall volume of prey and predators.

In regards to my version of “Peter and the Wolf,” I have a number of things that I’m curious about.

First, how does this relate to what you think of as music? Do you like listening to it? Which elements do you like and which do you dislike? For example, what do you think about about the tension and rhythm – do you agree the first works and that the second could be improved? Would you listen to this for enjoyments’ sake, and what would it take for this to be more than a novelty? What do you think about the narrative that drives the piece? I chose the predator and prey narrative, admittedly, on a whim. Do you think there might be some other narrative or agent specific goals that might better drive this piece? Is there any metaphor that might better describe this piece? As a listener do you enjoy the experience of being able to customize and configure the piece? What would you like to have control over that is missing here? Would you like more interaction with the piece or less interaction?

Finally, and perhaps most importantly, what do you think of the premise? Can simple electronic agents (albeit ones which interact socially) aspire to create music? Is there something promising in this act of simulation? Is music-making necessarily a human activity and is this kind of work destined to be artificial and uncanny?

Thanks for listening. I look forward to your thoughts.

“The Birth of Electronic Man.” Photo by xdxd_vs_xdxd @Flickr.

– – –

Andreas Duus Pape is an economist and a musician. As an economist, he studies microeconomic theory and game theory–that is, the analysis of strategy and the construction of models to understand social phenomena–and the theory of individual choice, including how to forecast the behavior of agents who construct models of social phenomena. As a musician, he plays folk in the tradition of Dylan and Guthrie, blues in the tradition of Williamson and McTell, and country in the tradition of Nelson and Cash. Pape is an assistant Professor in the department of Economics at Binghamton University and is a faculty member of the Collective Dynamics of Complex Systems (CoCo) Research Group.

– – –

Guest Respondents on the Comment Page (in alphabetical order)

Bill Bahng Boyer is a doctoral candidate in music at New York University who is completing a dissertation on public listening in the New York City subway system.

Maile Colbert  is an intermedia artist with a concentration in sound and video, living and working between New York and Portugal. She is an associated artist at Binaural/Nodar.

N. Adriana Knouf is a Ph.D. candidate in information science at Cornell University.

Primus Luta is a writer and an artist exploring the intersection of technology and art; he maintains his own AvantUrb site and is a founding member of the live electronic music collective Concrète Sound System.

Alejandro L. Madrid is Associate Professor of Latin American and Latino Studies at the University of Illinois at Chicago and a cultural theorist and music scholar whose research focuses on the intersection of modernity, tradition, globalization, and ethnic identity in popular and art music, dance, and expressive culture from Mexico, the U.S.-Mexico border, and the circum-Caribbean.

Tara Rodgers is an Assistant Professor of Women’s Studies and a faculty fellow in the Digital Cultures & Creativity program at the University of Maryland. As Analog Tara, she has released electronic music on compilations such as the Le Tigre 12″ and Source Records/Germany, and exhibited sound art at venues including Eyebeam (NYC) and the Museum of Contemporary Canadian Art (Toronto).

Jonathan Skinner founded and edits the journal ecopoetics, which features creative-critical intersections between writing and ecology. Skinner also writes ecocriticism on contemporary poetry and poetics.

Jonathan Sterne teaches in the Department of Art History and Communication Studies and the History and Philosophy of Science Program at McGill University. His latest book, Mp3 The Meaning of a Format comes out this fall from Duke University Press.

Jennifer Stoeveris co-founder, Editor-in-Chief and Guest Posts Editor for Sounding Out! She is also Assistant Professor of English at Binghamton University and a former Fellow at the Society for the Humanities at Cornell University (2011-2012).

Aaron Trammell is Multimedia Editor of Sounding Out! and a Ph.D. Candidate in Media and Communications at Rutgers University.

Audio Culture Studies: Scaffolding a Sequence of Assignments

Developed to explore the relationship between sound and learning, our fall forum on “Sound and Pedagogy” blends the thinking of our editors (Liana Silva), recruited guests (D. Travers Scott), and one of the winners of our recent Call For Posts (Jentery Sayers) to explore how listening impacts the writing process, the teachable moment, and the syllabus (and vice versa). If you need a make-up assignment for the first post by D. Travers Scott, click here, or for last week’s post by Liana Silva, click here. Otherwise, get ready to take some furious notes, because today’s post from Jentery Sayers offers a generously revealing—and unprecedented—glimpse inside the development of his audio culture course over several years. Chock full of suggestions, prompts, unique assignments, and troubleshooting tips, Sayers’s offering is refreshingly frank about the unique challenges facing sound studies scholars in the classroom. As in most classes, information this deep may need some time to settle in and effect some change, so we’ll check back with you in spring with a Sound and Pedagogy “refresher course.” It’s already on our syllabus.–JS, Editor in Chief 

DOI: https://doi.org/10.59350/kyyzr-5hq42

This semester, I am teaching my fourth course in audio culture studies. Since students and faculty in the humanities rarely have the opportunity to intensively examine sound in the context of higher education, thus far I have learned that—even for teachers who are well-versed in sound studies—planning and structuring work in an audio culture course is more difficult than it may at first appear. For instance, to what degree should an audio culture course intersect with visual culture studies? What cultural assumptions about sound inform how students learn about it, not to mention how instructors teach it? When engaging histories of sound reproduction, how might students combine knowing and doing, or composing about and with sound? And how should work in an audio culture course be assessed? Through what sorts of learning outcomes? With what forms of student preparation and self-reflexivity in mind?

In the following paragraphs, I engage these questions by unpacking a portfolio-based assignment sequence from an undergraduate course I taught at the University of Washington-Bothell in 2010: “Technologies of Expression: Sound Reproduction Studies.”

Screen Grab of Technologies of Expression

As I prepare for my “Audio Culture Studies” undergraduate course at the University of Victoria this semester, the above questions are guiding my lesson plans as well as my revisions of previous teaching materials. Yet more importantly, I imagine they—or variants of them—resonate with other instructors invested in sound studies, and I hope this post will spark a conversation about how to continue developing sound studies curricula in higher education. In one sense, such curricula should be and are distinct from work anchored in visual culture, print, film, and e-text studies. Nevertheless, there are a variety of suggestive intersections across these fields, and my approach to audio culture courses often builds upon them, especially since my training is in literary and cultural studies.

To begin “Sound Reproduction Studies” (heretoafter “205”) at the University of Washington-Bothell, I told students to let me know (in writing) how they were going to record sound during the course. After all, I did not know what technologies they could access, and I wanted to make sure that—with help from the University’s library and digital media services—recording equipment was available to them. I also informed them that, throughout the course, we would be focusing on basic recording and editing but not high fidelity techniques or post-production. In other words, I was not expecting them to gain competencies common in sound art and audio engineering, and my humanities-oriented assessment of their audio work would privilege content and composition over the aesthetics of recording and playback. For a course anchored primarily in cultural studies, they were welcome to use everyday audio equipment intended for consumers and amateurs, and fortunately that equipment was available on loan through the University.

After that request, I asked them to “crowdsource” a brief history of sound reproduction using SIMILE’s Exhibit 2 framework. While I have my reservations about crowdsourcing, for the purposes of the class—especially at the beginning of the quarter—it was appropriate. As the prompt suggests, I wanted to survey what students already knew about audio, together with where their interests in sound tended to constellate.

Screen Grab of the Crowdsourcing Prompt, graphic of a cylinder phonograph

From the start of the course, I also wanted to situate visual and audio cultures in the same space, encouraging students to think about the tensions and intersections between them as well as the epistemologies and affects they enable. Here’s a screen grab of what students produced in a week (between two class meetings):

I used this crowdsourced exhibit as the basis for an introductory lecture on the vexed legacies of sound reproduction. It helped me identify gaps in student knowledge while also underscoring what was familiar to them (e.g., Edison’s phonograph, the turntable, and auto-tune). The exhibit became a handy vehicle for posing questions to explore for the balance of the course: How is noise defined differently over time, by whom, and for whom? How is sound naturalized, and to what effects on critical inquiry? How is sound embodied, and why is it so often associated with emotion, affect, and immersion? When studying the history of sound reproduction, what roles do old technologies and media play in our research? How has the composition and playback of audio changed from decade to decade?

In other words, the exhibit was a low-stakes launching pad into the course material. I found it instilled a bit of confidence in students by giving them some sense of authority over the material. And it gave them a concrete sense of what I meant by “audio culture” (as opposed to, say, “music history”). Early in the course, I also appreciated the opportunity to highlight the deep and rich history of sound reproduction, a history which no doubt informs contemporary obsessions with all things digital. Where I failed as an instructor was by not returning to the exhibit later in the quarter. In future courses, I’ll give them an opportunity to add, revise, and even remove content where appropriate. Such a gesture would add a layer of self-reflexivity and iterative knowledge-making to the learning process, premised on questions like: “Now that you’ve studied audio for seven weeks, how would you revise your contributions to the crowdsourced exhibit you helped compose earlier in the course? How has your perception of audio’s history changed and why?”

After the crowdsourced exhibit of sound reproduction, the students in 205 shifted toward composing with audio, exploring ways to combine knowing and doing sound studies. For this shift, I relied on another low-stakes assignment: the “audiography.”

Screen Grab of the Prompt, graphic of a diagram of the interior of the ear

Here, students had the opportunity to share “playlists” of sounds typical to their everyday lives. However, the ultimate motivation of the exercise was twofold: (1) for them to closely attend to those common soundscapes, organize them (around a theme of some sort), and record them; and (2) for them to develop some basic competencies in the free audio editor and recorder, Audacity (based on a workshop conducted during class). Although the very notion of a “soundscape” was likely new to them, and even though most of them had never recorded audio, the concept of a playlist was personal and familiar enough to make the assignment approachable. It also stressed the fact that, like images and texts, people design and structure arguments through sound. A playlist of everyday sounds involves rhetorical strategies not entirely distinct from, say, an academic essay or television broadcast.

For the audiography (as well as every other audio composition in 205), the students were given constraints (e.g., “no narration,” “no effects,” and “eight to fifteen recordings only”) similar to guidelines for academic essays. And once the audio files were composed, the students uploaded them to our class audio blog (using a WordPress theme that unfortunately is no longer supported or maintained). The benefit of the blog was that everyone in the course could hear everyone else’s audio work. Additionally, I could play student audio during class meetings, treating it as an object of inquiry for discussion. In so doing, I made sure that—during the quarter—we discussed every student’s audio work at least once.

Audiographies are great for getting at cultural assumptions about sound, especially when students are able to respond to them through repurposing, voice-over, or a similar strategy. For the “re: audiography” prompt, I asked students to do just this.

Screen Grab of Prompt, close up of a microphone

They downloaded a peer’s audiography (from the class audio blog), edited it, produced a story through it (including voice-over tracks), and circulated the final product. While, quite obviously, this exercise became a gateway into studying montage, cut-up, turntable, DJ, and laptop practices, it also gave students an opportunity to assess how an audience member might interpret their audio work, the soundscapes with which they are familiar, and the rhetorical strategies motivating their compositions. “Re: audiography” relies on re-situating audio in a new context, as it demands thinking about how setting, history, and ideologies all affect interpretation. In so doing, it also requires significant consideration of the risks, play, warrants, and claims involved in an audio composition relying heavily on existing work. As an instructor, I was sure to remind students that such play is not innocent; it is quite serious, and—in the context of 205—it was not intended to offend or mock another student’s work, interests, or background. Consequently, adding this line to the prompt was key: “Don’t be rude to them and recognize that they worked hard to make their audiographies. What’s more, the sounds involved mean something to them.”

As I plan for my audio culture studies course at the University of Victoria, I am revising the “re: audiography” prompt to make it more concrete, with clear parameters. In particular, Step 5 in the instructions is too vague; it lacks sufficient focus, without being anchored in a particular genre or example to which students can refer. As such, during 205 I ultimately worried that the flexibility in the prompt induced student anxiety. Without a genre in mind, they had too many avenues to explore, even if all of those avenues afforded ways of engaging cultural assumptions about sound.

After the audiography prompts in 205, I assigned students two “found sound” exercises, which furthered the combination of knowing and doing through a line of research. Both exercises were scaffolded toward the production of an audio documentary that was academic in character.

The first “found sound” prompt asked them to gather a series of historical audio clips related to a particular medium and begin formulating a research question about them.

Screen Grab of Prompt

The audio piece that resulted included not only the audio clips but also the student’s initial thoughts, questions, and concerns about the research. One significant benefit of this exercise was that it acted in conversation with other strategies for academic argumentation, namely how arguments involving actual audio differ from and intersect with arguments devoid of audio. In other words, it introduced students to the very notion of multimodal argumentation, including the ways in which audio functions in scholarly communications. Students also had to consider what it means for an argument to be heard, not read.

As an instructor, I find that I still tend to map norms for text- and image-based argumentation onto audio work. For instance, the language of the “found sound” prompt resonates with a prompt I would write for a text-based “thought piece.” It is a translation of modes, without much consideration for how audio-based arguments might actually serve distinct and perhaps divergent aims—aims premised on, say, affect or non-sequential arrangement. For future iterations of this prompt, I plan to highlight such possibilities, brushing against my own proclivities for validating audio-based arguments through text- and image-based conventions.

Comparable to the “re: audiography” prompt, the students in 205 responded to each other’s “found sound” exercises, this time through “re: found sound.”

Screen Grab of Prompt

Rather than repurposing or sampling a student’s work, in this case students recorded verbal feedback, which essentially acted as a form of peer review. Perhaps above all else, this exercise allowed students to articulate an understanding of how to structure and evaluate audio compositions, preparing them for the audio documentary. Yet one of the most effective components of the prompt is this request: “Identify at least one thing (but preferably more) that you think your peer is overlooking or not considering. This thing can be a concept (e.g., ‘speaking with’), an example work (e.g., film, audio recording, or text), or a perspective (e.g., conflicting opinions on the topic). When you make this gesture, explain why it’s relevant to not only his or her audio documentary, but also the history of sound reproduction.” From my perspective as an instructor, this intervention in a peer’s work is difficult yet rewarding. It is a response that demands its own research, as it asks the reviewer to more or less deconstruct what is being heard.

“Re: found sound” also requires students to meticulously consider how they are saying what they are saying when critiquing someone else’s work. For instance, tone, inflection, and dialogic gestures are foregrounded. To be sure, students make similar decisions when responding through text. Nonetheless, the shift in modality affords a learning experience arguably rare in higher education humanities.

For future iterations of “re: found sound,” I will ask students to submit written feedback in tandem with the audio, not only for accessibility purposes but also to stress correspondences between the media. Again, one of my ongoing concerns in audio culture courses is that I will—in a reactionary fashion—privilege speaking and listening over seeing, reading, and watching. Such reactionary approaches risk dividing critical approaches when they can instead be suggestively blended through interdisciplinary research.

The two “found sound” prompts guided students in 205 toward an audio documentary, which was steeped in a specific historical issue and informed largely by cultural studies approaches to audio technologies and media.

Screen Grab of Prompt

To prepare students for this genre, I provided example documentaries, and we also conducted several workshops on how to make and critique those documentaries. In the end, these documentaries were a real pleasure to hear, and one of the biggest difficulties is listening and re-listening to them closely. After all, the evaluation of audio operates through a different temporality than text, and often times it lends itself to focusing too much on production and post-production (rather than, in this case, the actual content of the audio documentary). Requesting transcripts, and even the original AUP files (or a similar format), can facilitate fair and thorough evaluation. I also try to listen to each documentary at least twice, with and without headphones. Finally, as an instructor I have learned that audio documentaries will always take students twice as long as I assume they will. Despite popular assumptions about the speediness and ease of digital production, the genre is labor-intensive and layered with samples, effects, voice-over, and transitions. It is also often new to students in higher education.

Even though the documentary was the most involved piece of the students’ audio portfolios in 205, it was not the final submission. In a self-reflexive fashion, the students were asked to submit a final remix of their work (PDF).

Screen Grab of Prompt

At its core, the final remix intersected audio theory, practice, and history. Intended for educated, non-expert audiences beyond the course, it asked students to provide specific examples of how they fulfilled particular outcomes in the course. Among other things, the samples in the remix were intended to demonstrate an awareness of: (1) audience, (2) the technocultural history of sound, (3) a theory of listening, (4) audio composition and post-production, (5) constructive, audio-based peer review, and (6) persuasive media strategies for audio-based argumentation.

Perhaps quite obviously, this reflective remix borrows from a long history in writing studies. And I have found it quite effective for identifying what and how students learned from the course. That said, what I enjoy most about it is how it requires thinking about sound as matter: its arrangement and manipulation, its existence in various iterations and formats, its function as tangible, audible evidence. For, when returning briefly to those questions in the first paragraph of this piece, I tend to think the greatest challenge facing sound studies curricula and pedagogy is the challenge of sound’s materiality. As sound studies scholars, how do we foster spaces and opportunities where people can learn about the history of sound as an object with its own fluctuations, politics, aesthetics, and material particulars? How do we teach audio culture as that which can be simultaneously inhabited and examined from a distance? As a thing, a mode of composition, and a way of thinking, feeling, and acting?

Featured Image “Burn Station @ inSIRACJE festival” by Flickr User Paula Rey

Jentery Sayers is an Assistant Professor of English at the University of Victoria, where he teaches cultural studies, digital humanities, and 20th-century U.S. fiction. His writing has appeared in Kairos: Rhetoric, Technology, and Pedagogy; The New Work of Composing; Computational Culture; The New Everyday; Writing and the Digital Generation; Off Paper; Collaborative Approaches to the Digital in English Studies; and ProfHacker. Among his other work involving audio, his current book project is a cultural history of magnetic recording. He’s grateful to Jennifer Stoever at Sounding Out! for feedback on drafts of this piece.