English(EN) Variance Reduction for Heavy-Tailed Monetization Metrics in Ranking Experiments via Post-Stratification

ShareChat 通过事后分层增强 A/B 测试的货币化指标

作者 PulseAugur 编辑部 · [2 个来源] · 2026-06-02 18:14

研究人员开发了一个新框架，以提高在线实验的统计功效，特别是针对重尾货币化指标。该方法结合了事后分层和 CUPED，利用实验前数据来提高灵敏度，而无需增加流量。在 ShareChat 部署后，该技术显著降低了方差并提高了决策稳定性，在流量减少约 45% 的情况下达到了相似的置信度水平。 AI

影响提高了 AI 驱动的排名和推荐系统的 A/B 测试的可靠性，从而实现了更高效的决策。

排序理由该集群包含一篇详细介绍在线实验新统计方法的学术论文。

AI 生成摘要 · Google Gemini · 来自 2 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 0 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Research; 该集群包含一篇详细介绍在线实验新统计方法的学术论文。
Source corroboration: 2 independent sources; Multiple independent publishers reporting the same story raises confidence that it's real and newsworthy.
Topics: paper, product, other; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: 90 days old; Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

完整方法见我们的编辑标准。

arXiv stat.ML TIER_1 English(EN) · Neeti Pokharna, Olivier Jeunen, Yatharth Saraf, Aleksei Ustimenko · 2026-06-04 04:00

通过事后分层法减少排序实验中重尾变现指标的方差

arXiv:2606.04110v1 Announce Type: cross Abstract: Online evaluation of ranking and retrieval systems often relies on downstream monetization metrics such as app revenue or creator earnings. These metrics are typically heavy-tailed, with a small fraction of users dominating both m…
arXiv stat.ML TIER_1 English(EN) · Aleksei Ustimenko · 2026-06-02 18:14

通过事后分层减少排序实验中重尾货币化指标的方差

Online evaluation of ranking and retrieval systems often relies on downstream monetization metrics such as app revenue or creator earnings. These metrics are typically heavy-tailed, with a small fraction of users dominating both mean and variance, leading to low statistical power…