Certification from Examples is Hard for Circuits and Transformers under Minimal Overparametrization
A new research paper explores the difficulty of certifying the exact behavior of neural networks, particularly Transformers and circuits, even with minimal overparametrization. The study demonstrates that adding even a single extra gate to threshold circuits can exponentially increase the size of certification certificates required. Similar hardness results are shown for log-precision Transformers, indicating that ensuring exactness guarantees for these models is a computationally challenging problem. AI
IMPACT Demonstrates theoretical limitations in certifying neural network behavior, potentially impacting the development of reliable AI systems.