Neural network symmetry readout depends on positional encoding

A new paper explores how the symmetries within neural network weights are influenced by the positional encoding (PE) and the specific readout methods used for analysis. Researchers found that the PE can obscure or reveal certain symmetries, even if the underlying function possesses them. This dependence was demonstrated using MLPs trained on 2D signed distance functions, where different PEs like DyadicAxisPE and TriAxisPE showed distinct patterns in their ability to detect symmetries such as D4 and D3 rotations. AI

IMPACT This research provides a deeper understanding of how neural network architectures and analysis methods interact, potentially guiding future model design and interpretability techniques.

RANK_REASON The cluster contains a single academic paper detailing a new theoretical finding in neural network analysis. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• DyadicAxisPE
• TriAxisPE

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →