LexRubric: A Rubric-Guided Diagnostic Benchmark for Open-Ended Legal Tasks
Researchers have developed LexRubric, a new benchmark designed to evaluate the performance of large language models on open-ended legal tasks in Chinese. The benchmark includes 649 instances covering legal consultation and judicial examination, with over 12,000 expert-written scoring criteria across six dimensions. Initial tests on 18 LLMs revealed varying capability profiles, indicating that current models still struggle with complex legal reasoning. AI
IMPACT This benchmark will help identify weaknesses in LLMs for legal applications, guiding future development for more reliable AI in law.