When 2D Tasks Meet 1D Serialization: On Serialization Friction in Structured Tasks
A new paper explores the challenges large language models face when processing 2D structured data by converting it into 1D sequences. Researchers found that this "serialization friction" hinders performance on tasks like matrix transpose and Conway's Game of Life. A vision-augmented pathway that preserves the 2D layout significantly outperformed a text-only pathway, suggesting that maintaining spatial structure is crucial for these types of tasks. AI
IMPACT Highlights potential limitations of current LLM input processing for structured 2D data and suggests a path for improvement.