Researchers have conducted a pre-registered study to test the K-way energy probe's reduction on predictive coding networks, specifically examining the impact of removing cross-entropy (CE) loss. The findings indicate that CE is a significant component, as its removal in standard predictive coding networks reduced the probe-softmax gap by half. When applied to bidirectional predictive coding, the probe exceeded softmax across all tested conditions, even though the bidirectional model did not show substantially greater latent movement. Further analysis revealed that approximately two-thirds of the probe-softmax gap is due to logit-scale effects that can be adjusted with temperature scaling, while the remaining third is a scale-invariant ranking advantage of CE-trained representations. AI
Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →
RANK_REASON The submission is an academic paper on arXiv detailing experimental findings related to AI model training and evaluation.
