Researchers have developed a new family of looped transformers called Baton, designed to improve latent fact composition and two-hop reasoning in language models. Unlike standard transformers, Baton variants pass intermediate entities as token embeddings, enabling them to answer complex questions where intermediate entities are not explicitly present in training examples. While Baton models achieve high accuracy when trained to predict only the answer, their performance declines when trained on every token, a phenomenon linked to the next-token prediction loss. AI
IMPACT Introduces a novel transformer architecture that could improve reasoning capabilities in LLMs.
RANK_REASON This is a research paper detailing a new transformer architecture. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- arXiv
- CatalyzeX Code Finder for Papers
- Connected Papers
- CORE Recommender
- DagsHub
- DiscoLoop
- Gotit.pub
- Hugging Face
- Influence Flower
- Litmaps
- ScienceCast
- scite Smart Citations
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →