Researchers at the University of Rochester Hajim School of Engineering & Applied Sciences have developed a web-based system called BachDuet that allows users to improvise duets with an artificial intelligence (AI) counterpart in real time. By visiting the BachDuet website, users can play duets with the AI agent using a computer keyboard, mouse, touchscreen, or MIDI keyboard.
A new program developed by Rochester researchers uses machine learning to allow users to improvise duets with an artificial intelligence partner.
To play a duet with German composer Johann Sebastian Bach, you don’t have to travel back to the 18th century; thanks to a new program developed by researchers at the University of Rochester, you only need a computer.
The web-based program, called BachDuetwas developed by Zhiyao Duan, an associate professor of electrical and computer engineering and of computer science, and members of his lab, including Yongi Zang ’23 and PhD student Christodoulos Benetatos. BachDuet allows a person to improvise duets in the style of Bach with an artificial intelligence (AI) counterpoint in real time. By visiting the BachDuet website, a user can play duets with the AI agent using a computer keyboard, mouse, touchscreen, or MIDI keyboard.
Duan foresees the program could have applications in music education, entertainment, and music therapy.
In BachDuet, the AI agent will either follow the human musician, lead the improvisation, or “just goof around,” Duan says.
But the researchers foresee the program could have applications in music education, entertainment, and music therapy. For instance, says Duan, “most of the existing musical therapies are listening-based, but in recent years we have seen some playing-based music therapies that have been shown to help improve cognitive functions.”
BachDuet is part of Duan’s research projectfunded by a National Science Foundation Faculty Early Career Development (CAREER) award, to develop algorithms and systems that allow humans to work with machines to harmonize pitch, coordinate musical timing, and convey expression in music.
His lab uses signal processing, artificial intelligence, and machine learning algorithms, which can empower machines with music perception skills, more expressive performance, and deeper “understanding” of music theory and composition. For this project, the team input hundreds of Bach’s compositions and used machine learning techniques to allow the program to “study” the music. During improvisation, the program assumes a 4/4 time signature and makes a decision about which note to trigger or sustain.
“The goal of our project is to develop tools that allow humans to play music with computer systems in a way similar to human musicians collaborating with each other,” Duan says. “With BachDuet, we wanted to develop a tool that allows musicians and novices alike to practice counterpoint improvisation.”
“And,” he adds, “to have fun.”
Read more
The mysteries of music—and the key of dataResearchers at the University are at the cutting edge of the intersection of data science and music: developing databases to study music history and creating algorithms to automatically identify a genre or singer.
Category: Featured