Prediction-powered inference
PulseAugur coverage of Prediction-powered inference — every cluster mentioning Prediction-powered inference across labs, papers, and developer communities, ranked by signal.
3 day(s) with sentiment data
-
New AI evaluation method boosts accuracy with limited data
Researchers have developed a new methodology called Prediction-Powered Smoothing (PP-S) to improve the accuracy of AI system evaluations, particularly in scenarios with limited labeled data. This Bayesian approach integ…
-
LLM Digital Twins: Can AI Reduce Human Measurement in Research?
A new research paper explores the potential of LLM-based digital twins to reduce the need for human data collection in scientific inference. The study introduces 'statistical substitutability' as a criterion to evaluate…
-
Small language models boosted for claim checking with new calibration technique
Researchers have developed NN-PPI, a novel method to improve the accuracy of small language models (SLMs) for claim check-worthiness detection. This technique calibrates model predictions post-inference without requirin…
-
New statistical framework leverages generative models for improved inference
Researchers have introduced Generation-Powered Inference (GPI), a novel statistical framework designed to enhance inference on distribution-valued parameters by leveraging auxiliary generative models. This method is par…
-
New papers demystify prediction-powered inference for statistical analysis
Two new arXiv papers explore the concept of Prediction-Powered Inference (PPI), a framework that uses machine learning predictions to improve statistical inference when outcomes are difficult to measure. The first paper…
-
New research quantifies cost of per-class coverage under distribution shift
Researchers have characterized the cost of achieving valid per-class coverage in recognition systems when distribution shift occurs between training and testing data. They found that while split conformal prediction mai…
-
New framework optimizes LLM-augmented surveys using human respondent allocation
A new research paper published on arXiv introduces a framework for optimizing survey design when using large language models (LLMs) for response generation. The framework addresses the challenge of LLM accuracy variabil…
-
New paper analyzes Prediction-Powered Inference, finding no universal 'free lunch'
A new paper titled "No Free Lunch: Non-Asymptotic Analysis of Prediction-Powered Inference" analyzes the effectiveness of Prediction-Powered Inference (PPI) strategies. The research provides a finite-sample analysis of …
-
LLMs improve ranking evaluation with new reliability methods
Two new research papers introduce methods to improve the reliability of Large Language Models (LLMs) in ranking tasks. One paper, PRECISE, uses Prediction-Powered Inference to combine human and LLM judgments, reducing e…
-
New methods improve LLM evaluation accuracy with AI and human insights
Researchers have developed new methods to improve the accuracy and calibration of Large Language Model (LLM) evaluations. One approach, Conformal Elo Estimation, uses LLM judgments to estimate Elo ratings, achieving res…
-
New MEC method enhances semi-supervised inference with better uncertainty quantification
Researchers have developed a new method called Machine-Learning-Assisted Generalized Entropy Calibration (MEC) to improve semi-supervised inference and uncertainty quantification. MEC is a cross-fitted, calibration-weig…