N-gram models better predict reading time than transformers, study finds

A new paper proposes that traditional n-gram language models may be better predictors of naturalistic reading time than complex transformer models. The research suggests that while transformers excel at next-word prediction, their probabilities do not correlate as strongly with reading time metrics as simpler n-gram statistics do. The study found that neural language models whose predictions align most closely with n-gram probabilities also best predict eye-tracking data on naturalistic text. AI

IMPACT Suggests a potential limitation in current transformer models for tasks involving natural language understanding beyond simple prediction.

RANK_REASON The cluster contains an academic paper detailing research findings. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• alphaXiv
• arXiv
• CatalyzeX
• DagsHub
• Gotit.pub
• Hugging Face
• Influence Flower
• James Michaelov
• n-gram
• ScienceCast
• transformers

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →