Temporal difference learning
PulseAugur coverage of Temporal difference learning — every cluster mentioning Temporal difference learning across labs, papers, and developer communities, ranked by signal.
5 day(s) with sentiment data
-
New Horizon-Scalable Q-Learning method improves long-horizon off-policy value learning
Researchers have introduced Horizon-Scalable Q-Learning (HQL), a novel method designed to improve the accuracy of off-policy value learning in tasks with long horizons. HQL dynamically balances one-step and multi-step t…
-
New TD learning theory adapts to incomplete data coverage
Researchers have developed a new theoretical framework for temporal difference (TD) learning in reinforcement learning that adapts to incomplete data coverage. The approach identifies the "subgraph Bellman variance" as …
-
New TD framework calibrates black-box simulators using Bellman orthogonality
Researchers have introduced a new temporal-difference (TD) framework designed to calibrate black-box simulators. This framework utilizes one-step transitions from a target system and noisy simulator-generated continuati…
-
New TD framework calibrates black-box simulators using Bellman orthogonality
Researchers have developed a new derivative-free temporal-difference (TD) framework for calibrating black-box simulators. This framework uses one-step transitions from a target system and noisy simulator-generated conti…
-
New recipe for data-adaptive stopping boundaries in iterative algorithms
Researchers have developed a general method for creating data-adaptive stopping boundaries for iterative algorithms. This approach simplifies error analysis by focusing on a single requirement: time-uniform concentratio…
-
New paper details stationary bias in nonlinear stochastic approximation
A new paper published on arXiv explores stationary bias and extrapolation in nonlinear two-timescale stochastic approximation. The research details how constant-step stochastic approximation can lead to a persistent non…
-
New method tackles common-mode errors in low-timestep spiking neural networks
Researchers have identified common-mode errors as a key limitation in deep spiking Q-networks (DSQNs) when operating at low simulation timesteps. These errors, shared across action values, significantly hinder temporal …
-
New paper details statistical rates for asynchronous TD learning
Researchers have detailed statistical rates for asynchronous temporal-difference (TD) learning with Markovian data. The study focuses on the last iterate of standard tabular TD learning, proving that it can achieve a sp…
-
New Central Limit Theorem for TD Learning Published on arXiv
This paper, submitted to arXiv, presents a non-asymptotic central limit theorem for vector-valued martingale differences using Stein's method. The authors extend this to functions of Markov Chains and demonstrate its ap…
-
New inference method improves temporal-difference learning accuracy
Researchers have developed a new method called Self-Normalized Inference for Constant-Stepsize Temporal-Difference Learning. This technique allows for more accurate inference from single Markov trajectories by accountin…
-
New TRACE method enhances AI agent tool-use on long-horizon tasks · 2 sources tracked
Researchers have developed TRACE, a novel method for improving the performance of multi-turn AI agents in complex, long-horizon tasks. This technique addresses the challenge of credit assignment by deriving per-action r…
-
New Framework Generates Complex Physics Word Problems Using LLMs and RL
Researchers have developed ARVRE, a novel framework for generating complex and solvable physics word problems. This two-stage system uses temporal-difference learning to create valid physics equation chains and an agent…
-
TD learning fails to improve LLM few-shot retrieval on GSM8K
A researcher explored TD learning for improving retrieval of few-shot examples in LLM reasoning, aiming to assign learned values to traces based on their utility. The experiment involved storing reasoning traces, retrie…
-
Lyapunov framework analyzes stochastic algorithm convergence
Researchers have published a paper detailing a Lyapunov-based framework for analyzing the finite-time convergence of stochastic iterative algorithms. This approach uses generalized Moreau envelopes as universal Lyapunov…