Adversarial Attacks Leverage Interference Between Features in Superposition
A new research paper proposes that adversarial attacks on AI models can be explained by the phenomenon of feature superposition. This occurs when neural networks represent more concepts than they have dimensions, forcing interference between representations. This interference makes models vulnerable, as perturbations targeting one concept can affect others, leading to predictable and transferable attacks. The findings suggest that adversarial vulnerability can be a byproduct of representational compression in neural networks. AI
IMPACT Explains adversarial vulnerability as a byproduct of representational compression, potentially guiding the development of more robust AI models.