PulseAugur
EN
LIVE 06:15:25
ENTITY A/B testing

A/B testing

PulseAugur coverage of A/B testing — every cluster mentioning A/B testing across labs, papers, and developer communities, ranked by signal.

Show in brief
Total · 30d
5
23 over 90d
Releases · 30d
0
0 over 90d
Papers · 30d
3
15 over 90d
TIER MIX · 90D
TOPICS
RELATIONSHIPS
SENTIMENT · 30D

2 day(s) with sentiment data

RECENT · PAGE 1/2 · 24 TOTAL
  1. COMMENTARY · CL_353108 ·

    LLM prompt A/B testing risks false positives; mSPRT offers a solution

    Testing LLM prompts using standard A/B testing methods can lead to false positives due to continuous monitoring. The article explains that repeatedly checking results on a live dashboard increases the chance of observin…

  2. TOOL · CL_326823 ·

    AI system optimizes instant bookings in hospitality industry

    Researchers have developed a novel smart instant book filter system designed to enhance the hospitality industry's online marketplaces. This system intelligently promotes instant book listings while ensuring the overall…

  3. RESEARCH · CL_298469 ·

    LLMs enhance e-commerce ad relevance evaluation

    Researchers have developed a new metric for evaluating Dynamic Product Advertising (DPA) retrieval systems in e-commerce. This metric leverages large language models (LLMs) to assess user-product relevance, overcoming l…

  4. RESEARCH · CL_289495 ·

    Language models predict e-commerce search layout engagement · 2 sources tracked

    Researchers have developed a method using language models to evaluate page-level layout decisions in e-commerce search without requiring costly A/B testing. The study, published on arXiv, found that representation-based…

  5. TOOL · CL_268870 ·

    New research uses A/B test data to guide adaptive experiment design

    A new research paper explores using historical data from A/B tests to inform the design of adaptive experiments, particularly those employing contextual bandits. The study introduces a method that combines off-policy ev…

  6. TOOL · CL_245424 ·

    New BAFF filters mitigate training data interference in RTB A/B tests

    Researchers have developed a Bid-Aware Filter Family (BAFF) to address training data interference in real-time bidding (RTB) A/B tests. This interference occurs when control and treatment models are trained on shared lo…

  7. RESEARCH · CL_222846 ·

    GNNs scaled for friend recommendations with multi-hash embeddings and temporal sampling

    Researchers have developed a scalable Graph Neural Network (GNN) system for friend recommendation on large social graphs, addressing challenges in production environments. The system utilizes multi-hash ID embeddings to…

  8. TOOL · CL_205616 ·

    New offline evaluation task predicts ranking system impression share shifts

    Researchers have introduced "impression share prediction" as a novel offline evaluation task for ranking systems, aiming to forecast how a new model would distribute impressions across different optimization goals befor…

  9. TOOL · CL_206747 ·

    Together AI enables A/B testing for LLM endpoints in production

    Together AI has introduced a new feature allowing users to conduct A/B tests directly on their endpoints for LLMs. This functionality enables developers to split live traffic between a control model and up to 20 variant…

  10. TOOL · CL_203512 ·

    Developer uses A/B testing to optimize LLM-generated YouTube titles

    A developer details their experience using Large Language Models (LLMs) to automatically generate YouTube video titles and descriptions, facing challenges in objectively evaluating the quality of the AI-generated conten…

  11. COMMENTARY · CL_199210 ·

    Spotify explores LLMs for A/B testing automation

    Spotify's engineering blog explores the potential for Large Language Models (LLMs) to automate and enhance A/B testing processes. The article discusses how LLMs could analyze user behavior, predict outcomes, and even ge…

  12. TOOL · CL_193738 ·

    LLM A/B test prediction struggles with reliability, study finds

    A new research paper explores the effectiveness of large language models in predicting the outcomes of A/B tests for web page designs. The study found that while a Gemini 3 Flash model could achieve a moderate agreement…

  13. TOOL · CL_167935 ·

    New engine refuses to answer to ensure accurate causal inference

    A new causal inference engine has been developed to address the challenges of determining true cause-and-effect relationships in business analytics, particularly when A/B testing is not feasible. The engine builds a cau…

  14. TOOL · CL_167247 ·

    Generative models and adaptive testing boost ad creative performance

    Researchers have developed a novel workflow for optimizing ad creatives by integrating generative models with adaptive testing. This method uses a predictive model trained on historical A/B tests to refine and rank vari…

  15. RESEARCH · CL_147466 ·

    New A/B testing method reduces variance using policy overlap · 2 sources tracked

    Researchers have developed a novel experimental protocol to accelerate A/B testing by reducing variance through policy overlap. This method leverages $\Delta$-Off-Policy Estimation to obtain unbiased estimates for avera…

  16. COMMENTARY · CL_122072 ·

    A/B tests can mislead about feature impact, warns new guide

    A recent article highlights that A/B tests, often considered the gold standard for causal inference in feature rollouts, can be misleading if their underlying assumptions are not carefully examined. The piece uses a fic…

  17. TOOL · CL_123199 ·

    New A/B testing method improves algorithm comparison accuracy

    A new research paper proposes an improved method for comparing algorithms, particularly in the context of A/B testing for online services. The study reveals that traditional A/B testing can sometimes be less accurate th…

  18. TOOL · CL_121229 ·

    LLM search evaluation improved with historical user data · arXiv

    Researchers have developed a new method for evaluating search engine results using Large Language Models (LLMs) that incorporates historical user interaction data. This "behavior-grounded" approach uses Query-Relevance-…

  19. TOOL · CL_96126 ·

    New framework validates LLM surrogacy for A/B testing

    A new statistical framework has been developed to address the use of large language models (LLMs) in place of human participants for A/B testing. The framework adapts surrogate endpoint theory to assess when LLM outcome…

  20. TOOL · CL_62086 ·

    Causal ML offers solution for B2B revenue optimization

    Traditional A/B testing is often ineffective for B2B revenue optimization due to small sample sizes and long sales cycles. This article proposes using Causal Machine Learning, specifically Propensity Score Matching, to …