Support Vector Rubrics: Closing the Gap Between Self-Generated and Human Rubrics
Researchers have developed a new framework called Support Vector Rubrics (SVR) to improve the evaluation of large language model outputs. SVR addresses the limitation of self-generated rubrics by focusing on discriminating between closely ranked responses, rather than just describing good ones. This approach uses preference data to learn a rubric bank and a prompt-conditioned selector, significantly narrowing the gap between AI-generated and human-defined evaluation criteria. AI
IMPACT This new framework could lead to more reliable and nuanced LLM evaluations, improving model development and deployment.