Chunking the Critic: A Transformer-based Soft Actor-Critic with N-Step Returns
Researchers have developed a new sequence-conditioned critic for Soft Actor-Critic (SAC) that uses a lightweight Transformer to model trajectory context. This approach integrates N-step returns without importance sampling, allowing it to capture temporal structure for long-horizon and sparse-reward problems. The method demonstrates consistent performance improvements over standard SAC and other baselines on local-motion benchmarks, particularly for long-trajectory control tasks. AI
IMPACT Enhances reinforcement learning capabilities for complex, long-horizon tasks by improving value estimation.