Quantifying Uncertainty in AI Visibility: A Statistical Framework for Generative Search Measurement
A new research paper introduces a statistical framework to address the inherent variability in AI-powered search engines. The study highlights that identical queries can yield different results and cite different sources, making single-run citation share metrics misleading. Researchers analyzed Perplexity, OpenAI's SearchGPT, and Google Gemini, finding citation distributions follow a power-law and rankings are unstable across samples. The paper advocates for reporting citation visibility with uncertainty estimates and provides guidance on sample sizes for reliable confidence intervals. AI
IMPACT Highlights the need for uncertainty quantification in AI search metrics, potentially influencing how performance is measured and compared.