A mathematical theory of balancing relational generalization and memorization
Researchers have developed a new mathematical theory to explain how learning systems balance generalization with memorization of exceptions. They introduced a novel task, transitive inference with exceptions, to study this ability. Their analysis of kernel ridge regression and pretrained language models revealed that successful generalization is sensitive to representational geometry and that these models can make systematic mistakes predicted by the theory. AI
IMPACT Provides a theoretical framework for understanding and improving how AI models handle exceptions to general rules.