LLMs' literary quality judgments reveal focus on structure and voice

Researchers have developed a method to extract and test implicit theories of literary quality from Large Language Models (LLMs). In a study using DeepSeek, the model achieved 79.3% accuracy in classifying texts into six quality tiers, revealing a preference for intentionality, craft, depth, and distinctive voice over mere correctness. Further experiments showed that structural degradation and voice simplification had a more significant negative impact on perceived quality than lexical simplification. Exploratory comparisons with Qwen QwQ indicated similar patterns in quality judgments. AI

IMPACT Provides insights into how LLMs perceive and evaluate creative text, potentially improving automated writing feedback and computational aesthetics.

RANK_REASON The cluster describes a research paper published on arXiv detailing a new method for evaluating LLM reasoning and its application to literary quality. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• alphaXiv
• arXiv
• CatalyzeX
• Connected Papers
• DagsHub
• DeepSeek
• Gotit.pub
• Hugging Face
• Litmaps
• ScienceCast
• scite Smart Citations

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →