Two-Layer Linear Auto-Regressive Models Estimate Latent States
Researchers have demonstrated that two-layer linear auto-regressive models can learn to approximate Kalman filtering when trained on data from partially observed linear dynamical systems. The study shows that the models' learned hidden representations align with the state estimates produced by the optimal Kalman filter, even without explicit knowledge of the underlying dynamics. This finding is supported by theoretical insights into Kalman filter approximation by auto-regressive models, the benign optimization landscape of two-layer models, and finite-sample guarantees on prediction and state recovery errors. AI
IMPACT This research provides theoretical grounding for how auto-regressive models learn latent states, potentially informing the design of more effective sequential data models.