A Composer’s Helper: Using AI to Create New Harmonies

While generative AI can create some beautiful images and impressive essays, it doesn’t always give the user much control. You add in a prompt, push a button, and hope for the best.

But a new tool aims to give creators more ownership in a unique format: Symbolic music.

The Anticipatory Music Transformer allows users to have fine-grained control over the creative composition process. A composer can write part of a song and then ask the model to fill in the rest, provide suggestions for accompaniments, or offer alternative variations.

“It’s really more a composer's helper,” says John Thickstun, one of the scholars behind Anticipatory Music Transformer, developed out of the Center for Research on Foundation Models within Stanford HAI. Thickstun himself learned to play the cello as a child and performed in youth orchestras. “I know a little bit of music theory, but I never took advanced composition classes. I'm not a composer, but I have some amount of musical taste and I know what I like to listen to. I can play with the system and it understands the rules of composition. I can say, oh, I like that, do more of that, or I don't like that, let's go in a different direction.”

The music transformer, built using the generative pretrained Transformer architecture (GPT) that powers language models like ChatGPT, facilitates a co-creation process where composers iteratively collaborate with the tool, choosing what to write themselves and what to delegate to AI. This approach allows composers to keep fragments of the generated music that they like while discarding the rest.

The Anticipatory Music Transformer focuses on symbolic music rather than musical audio. The model's training is based on a principle called anticipation, allowing it to predict upcoming musical elements and provide more controllable and interactive outputs.

The model is publicly available. While there's still work to be done to make this a seamless interactive tool for musicians and producers, the creators are actively working to integrate the model with music sequencing software.

“Basically I wanted to build tools for composers and musicians to make their lives easier and more interesting,” Thickstun says. “I'm intrigued by the possibilities that tools like this could open up for more people to get involved in music composition.”

See how the Anticipatory Music Transformer works: 

The Anticipatory Music Transformer was developed by Stanford postdoctoral scholar John Thickstun, Stanford HAI Research Engineering Lead David Hall, Carnegie Mellon Assistant Professor of CS Chris Donahue, and Center for Research on Foundation Models Director Percy Liang.