Gradient Adaptation

Language models can learn during inference, research finds

A new research paper explores the concept of language models learning during inference, a process termed dynamic evaluation. The study found that standard softmax-attention models, when allowed to perform gradient steps…