A new research paper introduces a statistical framework to address the inherent variability in AI-powered search engines. The study highlights that identical queries can yield different results and cite different sources, making single-run citation share metrics misleading. Researchers analyzed Perplexity, OpenAI's SearchGPT, and Google Gemini, finding citation distributions follow a power-law and rankings are unstable across samples. The paper advocates for reporting citation visibility with uncertainty estimates and provides guidance on sample sizes for reliable confidence intervals. AI
IMPACT Highlights the need for uncertainty quantification in AI search metrics, potentially influencing how performance is measured and compared.
RANK_REASON Academic paper introducing a new statistical framework for evaluating AI search engine performance. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →