LLM novelty evaluations unstable due to prompt design, study finds

A new study published on arXiv reveals significant instability in how large language models (LLMs) evaluate the novelty of ideas. Researchers found that minor changes in prompt design for LLM judges can drastically alter their assessments, sometimes leading to accuracy below chance. Even when presented with identical idea pairs, simply informing the LLM about prior reviewer opinions could flip its verdict on over half the cases. The study also noted that retrieval augmentation and larger reasoning budgets offered little improvement, and purpose-built novelty evaluators performed worse than a basic prompted LLM baseline. These findings cast doubt on reported novelty gains from automated ideation systems and highlight the need for more robust evaluation methods. AI

IMPACT Raises concerns about the reliability of AI-generated ideas and the methods used to evaluate them.

RANK_REASON Academic paper detailing a controlled study on LLM evaluation methods. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• large language models
• Noy Sternlicht
• OpenReview

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →