Researchers have developed a novel method for optimizing online A/B testing, a standard practice for evaluating changes in user-facing systems like recommendation policies. The new approach jointly optimizes both the sampling of actions and the use of observations to estimate treatment effects, aiming to reduce the high variance often associated with traditional A/B tests. This method derives a variance-optimal action-level logging policy and uses historical logs to further reduce variance, leading to more accurate estimations and improved statistical power with less online traffic. AI
IMPACT Enables more efficient and reliable policy evaluation in online systems, potentially accelerating product development cycles.
RANK_REASON The cluster contains a research paper detailing a new methodology for A/B testing.
- A/B testing
- alphaXiv
- arXiv
- average treatment effect
- CatalyzeX Code Finder for Papers
- Connected Papers
- Control variates
- CORE Recommender
- DagsHub
- Gotit.pub
- Hugging Face
- IArxiv
- Litmaps
- Logging Policy
- pairwise off-policy evaluation
- ScienceCast
- scite Smart Citations
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →