CRAM: Centroid-Routing and Adaptive MoE for Multimodal Continual Instruction Tuning
Researchers have developed new methods to improve multimodal continual instruction tuning for large language models. CRAM focuses on isolating task-specific patterns into independent modules and using adaptive-rank instantiation to efficiently allocate parameters. ProtoAda introduces format-aware task prototypes to align task assignment with both semantics and output structure, consolidating updates geometrically. PROXYMIX learns a dynamic replay controller on a small proxy model and transfers it to a larger target model to mitigate forgetting and preserve alignment behavior. AI
IMPACT These methods aim to improve the adaptability and robustness of multimodal LLMs in real-world, evolving deployment scenarios.