Researchers have explored how Large Language Models (LLMs) like GPT-2 can model human sentence comprehension by examining the impact of context window size and coreference. Their findings indicate a U-shaped relationship, where both restricted contexts simulating working memory and expanded contexts up to 1,000 tokens show high psycholinguistic fit. Disrupting long-range coreference relations by pronominalizing entities significantly degraded the predictive power of larger context windows, suggesting that tracking these relations is crucial for aligning LM surprisal with human reading behavior. AI
IMPACT Provides insights into how LLMs process language, potentially informing future model development for better human-like comprehension.
RANK_REASON Academic paper published on arXiv detailing research into LLM behavior. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
