Brief

Researchers have developed a new model-based bootstrap method for controlled Markov chains, particularly useful in offline reinforcement learning scenarios where the data-generating policy is unknown. This technique establishes distributional consistency for transition estimators and extends to policy evaluation and recovery, providing asymptotically valid confidence intervals for value and Q-functions. Experimental results on the RiverSwim problem demonstrate that the proposed confidence intervals offer improved calibration and coverage compared to existing methods, especially with limited data. AI

Researchers have developed Q-Ising, a novel three-stage pipeline for dynamic treatment allocation in networks. This method integrates network structure with dynamic treatment strategies, addressing limitations of existing approaches. Q-Ising estimates network adoption dynamics using a Bayesian dynamic Ising model, augments treatment histories with latent states, and learns a dynamic policy through offline reinforcement learning. The approach quantifies uncertainty in dynamic decisions and provides interpretable spillover estimates, demonstrating superior performance over static benchmarks in microfinance network data. AI