ERAlign: Energy-based Representation Alignment of GNNs and LLMs on Text-attributed Graphs
Researchers have developed ERAlign, a novel framework for aligning representations from Graph Neural Networks (GNNs) and Large Language Models (LLMs) on text-attributed graphs. This approach utilizes Energy-based Models (EBMs) to project GNN-encoded graph structures and LLM-derived text embeddings into a shared latent space, ensuring distributional consistency. The framework introduces Energy Discrepancy (ED) to improve training efficiency and reduce energy landscape distortion. Empirical results across eight datasets show ERAlign achieving state-of-the-art performance in various supervision and cross-task transfer scenarios. AI
IMPACT Enhances representation learning for graph-structured data with textual attributes, potentially improving performance in areas like knowledge graph completion and recommendation systems.