Brief

Researchers have explored the functional redundancy within the weight space of recurrent neural networks, specifically using ordered real Schur coordinates in one-layer tanh RNNs. This method separates spectral blocks from nonnormal couplings, allowing for structured ablations while keeping input and readout maps constant. In a fixed-length copy task, certain nonnormal Schur couplings could be removed with minimal impact on performance, while others were crucial for accurate autonomous replay. The study found that the profile of loss-preserving ablations varies across different tasks and trained solutions, indicating approximate functional invariances rather than universal symmetries in recurrent weight space. AI