Bilevel Graph Structure Learning, Revisited: Inner-Channel Origins of the Reported Gain
Researchers have re-examined bilevel graph structure learning, a technique that jointly optimizes model parameters and graph structures to enhance graph neural networks. Their findings suggest that the performance gains are significantly driven by training dynamics within the inner loop, rather than solely by the rewiring of the graph as previously assumed. To isolate these effects, they introduced a control method called frozen-$\phi$, which freezes the graph structure while maintaining the inner-loop training schedule. This diagnostic revealed that the inner training dynamics account for a substantial portion of the performance gains, sometimes matching or exceeding the full bilevel approach. AI
IMPACT Challenges the established understanding of performance gains in graph structure learning, suggesting a shift in focus towards optimizing training dynamics.