ernie-research/NAVA
Baidu's ERNIE Team has released NAVA, a 6.3 billion parameter model capable of generating synchronized audio and video from a single text prompt. NAVA utilizes an Align-then-Fuse MMDiT architecture to achieve state-of-the-art performance on benchmarks like Verse-Bench for audio-visual synchronization and video quality. The model can generate one minute of 720p video with synchronized audio in approximately one minute and offers features like precise multi-timbre control and language-described camera control. AI
IMPACT Sets new SOTA on audio-visual synchronization benchmarks with a smaller parameter count, potentially lowering the barrier for high-quality AV generation.