Recurrent Neural Networks
PulseAugur coverage of Recurrent Neural Networks — every cluster mentioning Recurrent Neural Networks across labs, papers, and developer communities, ranked by signal.
11 day(s) with sentiment data
-
Local cycles identified as key design principle for neural network computation
Researchers have identified key structural design principles that enhance the computational abilities of recurrent neural networks. By training numerous networks to compute Boolean functions, they discovered that networ…
-
Time series modeling needs dynamical systems perspective, paper argues
A new position paper proposes that time series modeling should adopt a dynamical systems perspective to advance the field. The authors argue that most time series originate from underlying dynamical systems, and acknowl…
-
New research shows RNNs can approximate continuous functions with fixed networks
A new paper explores the theoretical capabilities of recurrent neural networks (RNNs) in approximating continuous functions. The research demonstrates that a single RNN with fixed weights and hidden dimensions can unifo…
-
Deep learning models leverage energy features for improved surface classification in robotics
Researchers have explored the use of energy-derived features for surface classification in mobile robotics, comparing their effectiveness against inertial data. Utilizing deep learning models such as CNNs, RNNs, transfo…
-
Batch Layers Crucial for Real-Time Fraud Detection Integrity
This article discusses the critical role of batch layers in maintaining the integrity of real-time fraud detection systems. It emphasizes that while real-time scoring is important, robust batch processes are essential f…
-
Transformers Explained: Self-Attention, Parallel Processing, and LLM Architecture
Transformers, a neural network architecture, revolutionized AI by processing tokens in parallel rather than sequentially like Recurrent Neural Networks (RNNs). This parallel processing, enabled by the self-attention mec…
-
New Random Attention module enhances mobile sleep staging efficiency
Researchers have developed a new temporal modeling module called Random Attention (RA) designed for efficient sleep staging on mobile devices. RA utilizes fixed random projections for similarity-based aggregation, reduc…
-
Biologically grounded neural networks leverage mouse brain data
Researchers have developed biologically grounded recurrent neural networks by leveraging data from the MICrONS program, which combines electron microscopy and calcium imaging of mouse visual cortex. These networks utili…
-
LSTM networks show near-critical dynamics at optimal training
Researchers have explored the concept of criticality in artificial neural networks, specifically within Long Short-Term Memory (LSTM) models. They observed that smaller LSTMs, when optimally trained, exhibit scale-free …
-
New LSTM stability method outperforms existing models
Researchers have developed a new method to ensure the stability of Long Short-Term Memory (LSTM) networks used in system identification, particularly for nonlinear dynamical systems like thermal processes. Their approac…
-
Survey details ML methods for neural activity dynamics
This paper surveys machine learning methods for analyzing neural activity dynamics, focusing on Latent Variable Models (LVMs). It categorizes LVMs into single-region dynamics, multi-region communication, and behavior-al…
-
Neural networks develop world models from predictive statistics
Researchers have explored how neural networks, specifically transformers and recurrent networks, develop internal representations of world dynamics. Using a simplified model of constrained random walks on a lattice, the…
-
RNN stability improved with new backward coherence theory
Researchers have developed a new theoretical framework called backward coherence to analyze hidden-state stability in recurrent neural networks (RNNs). This approach treats the hidden-state sequence as a quasi-reverse-m…
-
New method trains recurrent networks without recurrence
Researchers have developed a new method called Supervised Memory Training (SMT) to pretrain recurrent neural networks (RNNs) without relying on traditional recurrence. SMT trains RNNs by reducing the process to supervis…
-
RNNs show paradoxical preference for training noise
Researchers have discovered that recurrent neural networks (RNNs) can develop a paradoxical preference for noise during training. Contrary to the expectation that noise should be removed for optimal performance, these n…
-
Transformer models struggle with state tracking and data efficiency compared to RNNs
A new research paper published on arXiv explores the limitations of transformer-based language models in state tracking, a critical aspect for understanding sequential data. The study reveals that transformers require s…
-
New Residual Reservoir Memory Networks Enhance RNNs
Researchers have developed a new type of Recurrent Neural Network called Residual Reservoir Memory Networks (ResRMNs). This model combines a linear memory reservoir with a non-linear reservoir that uses residual orthogo…
-
AI training framed as Hamilton-Jacobi PDE problem
Researchers have formulated neural network training as a Hamilton-Jacobi initial-value problem. This framework connects gradient steps to solving viscous Hamilton-Jacobi equations, revealing shared mathematical structur…
-
Researchers analyze critical organization in deep neural networks
Researchers have rigorously studied the thermodynamic limit of deep neural networks (DNNs) and recurrent neural networks (RNNs), focusing on sigmoid activation functions. They demonstrated that in a specific parameter r…
-
New RNN module boosts BCI accuracy and explainability
Researchers have developed a new Post-Recurrent Module (PRM) to enhance the explainability and performance of Recurrent Neural Networks (RNNs) used in P300-based Brain-Computer Interfaces (BCIs). This module improves cl…