Contrastive Spectral Rectification: Test-Time Defense towards Zero-shot Adversarial Robustness of CLIP
Researchers have developed a new test-time defense method called Contrastive Spectral Rectification (CSR) to improve the adversarial robustness of vision-language models like CLIP. This method addresses the vulnerability of these models to adversarial examples by exploiting their spectral bias, which causes feature inconsistency under frequency attenuation. CSR optimizes a rectification perturbation to realign inputs with the natural manifold, demonstrating significant performance gains over existing methods on multiple benchmarks with only a modest increase in inference time. AI
IMPACT Enhances the security of vision-language models against adversarial attacks, potentially enabling wider deployment in sensitive applications.