Ambroise Odonnat

Vision Transformer finetuning benefits from non-smooth components

A new research paper published on arXiv explores the concept of "plasticity" in Vision Transformers, defining it as the average rate of change within model components. The study suggests that prioritizing components wit…