Can Language Models Learn to Listen?
Researchers have developed a framework to generate appropriate facial responses for a listener in social interactions based on the speaker's words. This approach treats quantized facial gesture elements as additional language tokens for a transformer-based large language model. Initializing the transformer with pre-trained language model weights yielded higher quality responses than training from scratch, demonstrating fluent and semantically relevant generated motion. AI
IMPACT Demonstrates LLMs' potential for multimodal understanding and generation, extending their capabilities beyond text.