New research reveals language models struggle with consistent temporal encoding

A new paper from arXiv investigates how language models encode the current year, finding that models struggle with consistent temporal reasoning. Researchers developed two tasks to probe this: an associative task using verb tense and a declarative task directly asking for the year. While instruction-tuned models can estimate the current year within a year of their training data cutoff, their internal mechanisms for associative and declarative tasks differ significantly. Attempts to update the current year through prompting, supervised fine-tuning (SFT), or weight editing proved largely unsuccessful in shifting both notions simultaneously, highlighting a challenge in keeping language models temporally up-to-date. AI

IMPACT Highlights a limitation in current LLMs regarding temporal reasoning, potentially impacting applications requiring up-to-date information.

RANK_REASON The cluster contains a research paper detailing findings about language model capabilities. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• alphaXiv
• arXiv
• CatalyzeX Code Finder for Papers
• CORE Recommender
• DagsHub
• Gotit.pub
• Hugging Face
• Influence Flower
• ScienceCast
• Suze Van Adrichem

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →