ECA: Efficient Continual Alignment for Open-Ended Image-to-Text Generation
Researchers have developed Efficient Continual Alignment (ECA), a novel approach for open-ended image-to-text generation that addresses the challenge of adapting models to evolving data distributions without access to past data. ECA utilizes a Mixture of Query module, Fisher Dynamic Expansion based on the Fisher Information Matrix, and Dictionary Replay to retain knowledge and adapt to new visual data. This method aims to prevent catastrophic forgetting and improve performance in scenarios where the predominant image categories shift over time, as demonstrated on newly constructed benchmarks. AI