What's new in v1.1:
➤ We’re introducing a brand new prompt set to our Music Arenas, consisting of 1,000 prompts (500 per benchmark) curated and balanced across 17 genres, ranging from Classical and Jazz & Blues to EDM, Metal and Reggae.
➤ Each prompt tests song structure, mood, instrumentation or vocal style, with longer, more complex prompts that reflect today's prompting behavior and the emerging capabilities of frontier music models.
➤ Panel votes only: ratings come entirely from blind votes by our recruited evaluator panel. Public Arena votes do not count toward the rating.
Our updated leaderboards are launching with more than 37,000 blind human preference votes: more than 16,000 on AA-Music-Vocal v1.1 and more than 21,000 on AA-Music-Instrumental v1.1, with over 2,000 votes for every ranked model. On AA-Music-Vocal v1.1 we do not supply lyrics, so the benchmark measures each model's end-to-end songwriting.