Transformers learn transitive inference, mirroring animal behavior

Researchers have demonstrated that Transformer models can learn transitive inference, the ability to deduce relationships like A < C from A < B and B < C. When trained solely on adjacent comparisons from a hidden order, these models develop a one-dimensional representation where entity embeddings align with their rank. This geometric structure mirrors the symbolic distance effect observed in animal behavior, where more distant comparisons are easier to process. The findings offer a mechanistic explanation for transitive inference by connecting cognitive science with neural network representations. AI

IMPACT Demonstrates how neural networks can replicate complex cognitive abilities like transitive inference, potentially informing future AI development.

RANK_REASON Academic paper detailing a new finding in model capabilities. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• arXiv
• Transformer

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →