Researchers have developed new methods to estimate the total variation (TV) distance between distributions generated by autoregressive models. These methods address the challenge that different inference engines, even when serving the same model weights, can produce distinct output distributions due to implementation choices and optimizations. The proposed algorithms offer improved query complexity under various access models, including sample access, logit access, and noisy logit access, with empirical evaluations demonstrating their practicality and robustness. AI
IMPACT Provides a method to quantify differences in outputs from different LLM inference engines, crucial for model evaluation and deployment.
RANK_REASON The cluster contains an academic paper detailing new theoretical and empirical results on a specific machine learning problem. [lever_c_demoted from research: ic=1 ai=1.0]
- arXiv
- autoregressive model
- Kullback–Leibler divergence
- Meel et al.
- SGLang
- Total variation distance of probability measures
- vLLM
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
