Encode Errors: Representational Retrieval of In-Context Demonstrations for Multilingual Grammatical Error Correction
Researchers have developed a new method for improving Grammatical Error Correction (GEC) in large language models (LLMs) by focusing on retrieving relevant in-context demonstrations. Their approach, termed Grammatical Error Representation (GER), extracts internal states from LLMs that encode grammatical errors, rather than relying on semantic similarity. This GER-based retrieval significantly enhances few-shot performance on multilingual GEC tasks, achieving results comparable to closed-source models like Deepseek2.5 and GPT-4o-mini for high-resource languages and surpassing baselines for low-resource languages. AI
IMPACT Enhances LLM capabilities in grammatical error correction, particularly for low-resource languages, offering a more interpretable approach.