Recurrent neural networks approximate continuous functions
Researchers have demonstrated that Recurrent Neural Networks (RNNs) can uniformly approximate any continuous function on a closed interval. This is achieved by allowing the network to run for a longer duration rather than requiring a new network for improved accuracy. The study introduces a new model, the Turing machine with neural units (TMNU), which bridges algorithmic freedom with RNN simulation capabilities, leading to convergence rates tied to polynomial approximation. The findings are supported by minimax lower bounds indicating that runtime is a necessary resource in this fixed-network approximation approach. AI
IMPACT Establishes a theoretical foundation for RNNs to approximate complex functions, potentially influencing future model design and understanding.