A new paper explores the use of Large Language Models (LLMs) as evaluators for literary fictions, comparing their assessments to those of human judges. The study found that while LLMs cannot replace human experts, they can serve as a useful tool for providing an initial perspective on the editorial quality of texts. The research utilized the GrAImes framework to analyze microfictions and short stories in both Spanish and English, yielding a Krippendorff's alpha reliability coefficient below 0.66 when compared to human evaluations. AI
IMPACT LLMs show potential as auxiliary tools for literary evaluation, offering initial insights into text quality.
RANK_REASON The item is a research paper detailing a new methodology for evaluating literary fictions using LLMs. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- arXiv
- CatalyzeX
- Connected Papers
- DagsHub
- Gotit.pub
- GrAImes
- Hugging Face
- Influence Flower
- Krippendorff's alpha
- Large Language Models
- Litmaps
- ScienceCast
- scite Smart Citations
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →