On the Theoretical Limitations of Embedding-based Link Prediction
A new research paper explores the theoretical limitations of embedding-based link prediction models, particularly those using linear output layers. The authors demonstrate how these linear layers can create a "rank bottleneck," restricting the model's ability to represent complex functions and fit training data, especially for large and dense graphs. The paper proposes and empirically validates the use of non-linear output layers, such as mixtures, to overcome this bottleneck with minimal parameter overhead, leading to improved performance. AI
IMPACT Identifies a fundamental limitation in common AI model architectures, suggesting a path for improved performance on large-scale graph data.