AI model STREAM generates editable dance motions from text and music

Researchers have developed STREAM, a new AI model designed for generating editable dance motions from text and music. Unlike previous models that struggle with modality collapse, STREAM uses a decoupled diffusion transformer to maintain user control over choreography while aligning motion with musical rhythms. The model incorporates Adaptive Layer Normalization for text-based structure and a novel Bimodal Energy-Based Attention Module for musical integration. To evaluate its performance, the team introduced the Exchange Evaluation Protocol and Editable Dance Score, demonstrating STREAM's ability to act as a collaborative partner for choreographers. AI

IMPACT Enhances AI's role in creative fields by enabling more controllable and collaborative motion generation for dance.

RANK_REASON Publication of a research paper detailing a new AI model and evaluation protocol. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• Adaptive Layer Normalization
• Bimodal Energy-Based Attention Module
• Editable Dance Score
• Exchange Evaluation Protocol
• Motorica++
• Seong Jong Yoo
• STREAM

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →