Researchers have introduced Aggregate-then-Calibrate (AtC), a novel two-stage framework designed to improve human-centered assessment tasks. This approach addresses the limitations of relying solely on human judgments or model-generated scores by combining both. The first stage aggregates comparative judgments, considering annotator reliability, to form a consensus ranking. The second stage calibrates predictive model scores to ensure ordinal consistency with this ranking, while retaining quantitative information. Theoretical analysis and empirical results demonstrate that AtC enhances accuracy and robustness compared to methods using only human or model assessments. AI
IMPACT This framework could improve the reliability and accuracy of AI-assisted decision-making processes by better integrating human expertise with model outputs.
RANK_REASON The cluster describes a novel framework presented in a research paper. [lever_c_demoted from research: ic=1 ai=1.0]
Read on Hugging Face Daily Papers →
- Aggregate-then-Calibrate
- Anatomical Therapeutic Chemical Classification System
- Hugging Face Daily Papers
- isotonic calibration
- rank-aggregation model
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
