Crystal: Characterizing Relative Impact of Scholarly Publications
Researchers have developed Crystal, a new method that uses large language models to assess the impact of scholarly publications by analyzing their citation context. Unlike previous methods that evaluate citations in isolation, Crystal jointly ranks all cited papers within a single document, allowing for relative comparisons. This approach mitigates LLM positional bias through randomized ranking and aggregation, leading to improved accuracy and efficiency. Crystal has demonstrated superior performance compared to existing state-of-the-art classifiers and aligns with long-term scientific recognition, with a new dataset and code released for public use. AI
IMPACT Introduces a novel LLM-based methodology for evaluating academic research impact, potentially improving scholarly discovery and assessment.