Scott Barton at the musical helm.

Mixing AI with Music & Literature

Interdisciplinary class combines computer science, music and literature to teach students the relationship between music composed by humans and music composed by artificial intelligence

Share
Share

If you ever heard the echoey bass from PVC pipe aerophones blended with the metallic timbre of cymbals streaming from Alden Memorial, that’s Scott Barton making music … with artificial intelligence.

In B-Term, Barton, associate professor of Music, and Jim Cocola, associate professor of Literature, Film, and Media and associate head of Humanities & Arts, taught a class called “Artificial Composition,” which combined computer science, literary studies, and music composition to explore how humans and machines interact—particularly while performing music.

We’re interested in looking at what humans do well, what robots do well, and how each intersects when it comes to musical composition and performances.
–Scott Barton

During class, Barton and Cocola led eight undergrads and one graduate student in discussion of the history of AI's role in human expression, and how humans create musical pieces where people and robots are the performers.

“We’re interested in looking at what humans do well, what robots do well, and how each intersects when it comes to musical composition and performances,” says Barton.

Creating AI-human blended music is nothing new to Barton, who often collaborates with colleagues in both the Music, Perception, and Robotics Lab and with Expressive Machines Musical Instruments (EMMI), a robotic musical instrument collective. Together they helped build musical robots like Poly-tangent Automatic Multi-monochord (PAM) and Configurable Automatic Drumming Instrument (CADI).

A performance with PAM and CADI goes like this: Barton operates a computer that is electronically tethered to both machines. As he types in inputs, algorithms kick in and prompt PAM and CADI to play, listening to the commands and responding autonomously in real time. The result is an alloy of sounds created by man and machine simultaneously.

Musical Robots in Action

“It makes you think about the process of composition,” Barton says, “such as repetition, for example, which provides a trajectory of contours that enables and inspires improvisation.”

The class also explored the technical side of this type of music, from algorithm creation to coding, and the more literary side, such as writing music, and how each influences a robot-human performance.

Similar to the literary nuances of poetry, Cocola says that music, algorithms, coding, and the resulting sound inform each other. “They further our understanding of human expression,” he says. “It’s about our connections, and everyone’s interpretation of it is different.”

The “Artificial Composition” class will be offered again in 2020. Jesse Abeyta ’21, who is majoring in computer science, recommends it to other students.

“‘Artificial Composition’ was a great opportunity to explore the intersection of computing and creativity, which seems mostly absent from the normal computer science curriculum,” Abeyta says. “We were given a great amount of freedom in our essays and projects, so it always felt like we were writing or doing something based on what we were interested in rather than what we had to do.”

Since the class itself was small, Abeyta said his favorite part was engaging in lively discussions and debates on AI, music, and more. “I ended up learning a grab bag of information about music, literature, AI, and humanity in general,” he says. “This was hands down my favorite course at WPI.”

Barton and Cocola hope the class sparked students’ curiosity about how artificial intelligence both reflects and influences our identities and relationships with others.

“We approached AI and creative practice in ways that were both abstract and concrete,” Barton says. “By engaging with those outside of our traditional fields, we were able to identify aspects of the topic that otherwise would have been missed. It was a lesson in the power of collaboration.”

--By Jessica Messier

Scott Barton wrote an entire album featuring electro-acoustic music last summer, called “Stylistic Alchemies." Learn more

Topics:

Profile(s):
Jim Cocola|Scott D Barton

Related Stories