Brief

Richard Sutton, a pioneer in reinforcement learning, has partnered with Chinese haptic technology company HeShan Technology to launch a "Robot Kindergarten" project. This initiative aims to train embodied AI agents through real-world trial and error, emphasizing the importance of first-person experience over imitation learning. The project will leverage HeShan's advanced haptic sensors, which provide detailed tactile feedback, to enable robots to learn from physical interactions and develop a deeper understanding of the world. AI

Researchers have developed a novel reinforcement learning (RL) approach to optimize ion shuttling on trapped-ion quantum computers. This method addresses the high-dimensional optimization challenge that arises with increasing numbers of ions, outperforming current heuristic techniques. The RL approach achieved up to a 36.3% reduction in shuttling operations and is adaptable to various chip architectures, offering a valuable tool for designing future quantum computing hardware. AI

Researchers have developed a new framework called Pontryagin-Guided Direct Policy Optimization (PG-DPO) to address limitations in reinforcement learning methods. Traditional approaches using Bellman-style recursions struggle with non-exponential discounting, which is common in modeling human preferences and survival scenarios. PG-DPO abandons recursion, instead integrating the Pontryagin Maximum Principle with Monte Carlo rollouts to achieve better accuracy and stability on specialized benchmarks. AI

Researchers have developed a novel offline reinforcement learning algorithm to create personalized physical activity recommendations. This algorithm analyzes step count data and health biomarkers from the All of Us Research Program to optimize daily step distributions for improved cardiometabolic risk. Simulation studies indicate the approach outperforms existing continuous-action RL methods, suggesting increased and more consistent physical activity for better health outcomes. AI

Researchers have developed Mem-π, a novel framework designed to enhance the adaptive memory capabilities of large language model (LLM) agents. Unlike traditional methods that rely on static retrieval from memory banks, Mem-π employs a separate, dedicated model to generate context-specific guidance dynamically. This approach allows the agent to decide when and what guidance to produce, leading to more efficient and relevant task execution. In evaluations across various agentic benchmarks, Mem-π demonstrated significant improvements, particularly in web navigation tasks where it achieved over 30% relative gains compared to existing memory baselines. AI

Researchers have developed DeCoR, a novel reinforcement learning framework designed to optimize urban street design and traffic signal control. The system first learns to generate optimal crosswalk layouts by encoding pedestrian networks as graphs. Subsequently, it develops adaptive signal timings to minimize delays for both pedestrians and vehicles. In simulations on a real-world urban corridor, DeCoR significantly reduced pedestrian wait times and improved traffic flow, demonstrating robustness to varying demand and layout changes. AI

Researchers have developed PREFINE, a novel method for adapting pre-trained reinforcement learning policies to incorporate safety constraints without full retraining. This technique leverages trajectory-level preferences, similar to how Direct Preference Optimization (DPO) is used for LLMs, to fine-tune policies for safer behavior. PREFINE has demonstrated a significant reduction in constraint violations and failures, exceeding 60%, while preserving original reward performance. The method offers improved data and computational efficiency compared to traditional offline RL or imitation learning approaches. AI

Researchers have developed a new reinforcement learning framework, called FPRO, to optimize the design and manufacturing of free-form pipes in aeroengines. This approach integrates domain-specific manufacturing knowledge as constraints within the reinforcement learning process. FPRO generates collision-free, manufacturable pipe paths that are then directly translated into fabrication instructions for a six-axis bending machine, demonstrating practical feasibility through real-world validation. AI

Researchers have developed a new reinforcement learning (RL) approach called Y-wise Affine Neural Network (YANN-RL) designed for control in chemical process systems. This method aims to overcome the typical challenges of trust and lengthy training times associated with RL in this domain. By providing confident and interpretable starting points for control schemes, YANN-RL demonstrated reduced training time and data requirements in case studies involving a CSTR, a four-tank system, and an extraction column. AI

Researchers have developed a novel reinforcement learning policy called pcsp, designed to enable scalable and controllable non-player characters (NPCs) in life-simulation games. This single policy is conditioned on LLM embeddings of persona descriptions, allowing for distinct and consistent NPC behaviors. The method significantly outperforms chance in zero-shot persona identification and achieves faster inference times compared to LLM-based policies, demonstrating its viability in commercial game engines. AI

Researchers have developed a reinforcement learning (RL) framework to automate and shorten the process of analyzing security protocols using the Tamarin tool. This new method, inspired by AlphaZero, employs a neural heuristic to guide a Monte Carlo Tree Search, learning from completed subproofs. Evaluations on 16 case studies show that the RL approach finds more proofs automatically and generates shorter proofs compared to existing methods, significantly reducing the human effort required for protocol verification. AI

Researchers have developed new theoretical frameworks for optimizing decision-making processes in machine learning. One paper introduces regret-based stopping criteria for Bayesian optimization, ensuring solutions are within a specified epsilon-optimality with high probability. Another study focuses on reinforcement learning for multinomial logistic MDPs, proposing an algorithm with improved regret bounds that are proven to be minimax optimal. A third paper addresses risk-sensitive reinforcement learning in discounted MDPs, providing sample complexity bounds for learning optimal policies under recursive entropic risk measures. AI

Researchers have developed a new reinforcement learning framework called the Deep Microcanonical Graph Generator (DMGG) to create graphs with precisely controlled structural properties. This method allows for exact enforcement of constraints, unlike previous models that only enforced them in expectation. DMGG utilizes a policy-guided search to efficiently generate graphs with specific assortativity, a measure of degree-degree correlation, significantly accelerating the process and enabling more accurate analysis of structure-function relationships. AI

Researchers have developed Mahjax, a new GPU-accelerated simulator for the complex game of Riichi Mahjong, implemented in JAX. This tool is designed to facilitate reinforcement learning research, particularly for agents learning from scratch rather than relying on human play data. Mahjax achieves high throughput, processing up to 2 million steps per second on multiple GPUs, and has been validated for training agents to improve their performance. AI

A new study on arXiv explores how different training data curricula impact the performance of reinforcement learning (RL) agents designed to work with large language models (LLMs) and external memory banks. The research found that the composition of training data significantly influences an agent's specialization rather than uniformly boosting performance. A mixed curriculum combining different benchmarks yielded the best overall results, while training on a narrow out-of-domain set specifically improved temporal reasoning skills. AI

Researchers have developed new methods for reinforcement learning policies that aim to improve efficiency and expressiveness. One approach, Score-Based One-step MeanFlow Policy Optimization (SOM), constructs a target velocity field using Q-function scores and a probability flow ODE, enabling state-of-the-art performance in online RL with reduced training and inference times. Another development, Stochastic MeanFlow Policies (SMFP), offers a one-step generative policy class that maps noise to actions through a MeanFlow transformation, providing a unified objective for stable and exploratory policy improvement in off-policy settings. AI

Two new research papers explore advanced reinforcement learning techniques for safer autonomous driving. The first paper introduces a multi-agent reinforcement learning (MARL) approach where self-driving cars and pedestrians are co-trained, leading to a 30% reduction in collisions compared to baseline methods by better anticipating unpredictable pedestrian behavior. The second paper proposes a Cognitive-Physical Reinforcement Learning (CoPhy) framework that integrates knowledge from vision-language models and uses a predictive world model to ensure safety and compliance with driving intent, achieving state-of-the-art results on benchmarks. AI

A new chapter explores the integration of artificial intelligence into serious games, aiming to overcome limitations like static scenarios and authoring bottlenecks. It discusses how AI, including LLMs and reinforcement learning, can enable dynamic scenario variation, adaptive pacing, and better learner modeling. The chapter also addresses the challenges of implementing AI in these systems, such as ensuring validity, transparency, and learner trust, while acknowledging the limited empirical evidence on long-term learning outcomes. AI

Researchers have developed a new framework called CRiSP that uses reinforcement learning and Transformer-based policies to improve the initial state preparation for Variational Quantum Algorithms (VQAs). This method aims to overcome limitations like barren plateaus and local minima, outperforming existing Clifford initialization techniques on QAOA benchmarks. Separately, another study explores quantum reinforcement learning for process synthesis, proposing state encoding algorithms to enhance scalability and demonstrating competitive performance against classical RL methods on flowsheet synthesis problems. AI

Researchers have developed new methods to enhance flow matching models, a type of generative AI. One approach, "Precise," improves reinforcement learning post-training by using SDE-consistent stochastic sampling for better alignment and faster optimization. Another paper explores "Sparse Compositional Flow Matching" for embodied AI trajectories, composing motion primitives directly in physical space for improved accuracy. A survey also reviews diffusion and flow matching models for tabular data, highlighting challenges and future directions, while other work investigates "Transition Matching" as a potentially superior alternative to flow matching for certain distributions and introduces "Flow Mismatching" for unsupervised anomaly detection. AI

Researchers have developed CoTrace, a framework to measure and expose goal-level contributions in human-AI collaboration, revealing that while AI accounts for a smaller percentage of overall goal-shaping, it significantly contributes to concrete requirements and indirect influences. Separately, a new method called DGPO aims to improve reinforcement learning for LLMs by addressing coarse-grained credit assignment issues in complex reasoning tasks. Additionally, a study on the entropy of the Ukrainian language provides an upper bound and compares it to LLM performance, while another paper explores using Sparse Autoencoders for out-of-distribution detection in vision transformers. AI