A Random-Matrix Criterion for Initializing Gated Recurrent Neural Networks
Researchers have developed a new criterion for initializing weights in gated recurrent neural networks, crucial for the performance of reservoir computing models. This criterion, derived from random-matrix theory, helps identify an effective critical point that separates ordered and chaotic phases in randomly initialized models. The method closely tracks the optimal gain for gated RNNs on forecasting tasks and could inform future initialization strategies. AI
IMPACT Provides a new theoretical framework for improving the training and performance of recurrent neural networks.