Categorical Prior Lock-in: Why In-Context Learning Fails for Structured Data
A new research paper identifies a phenomenon called "categorical prior lock-in" that limits the effectiveness of in-context learning (ICL) for large language models when generating structured data. The study found that while ICL can improve numerical accuracy, it struggles to reproduce rare categories in tabular data. Parameter-efficient fine-tuning methods like LoRA can overcome this but introduce risks of memorization and output instability. AI
IMPACT Identifies a key limitation in LLM adaptability for structured data generation, potentially impacting applications relying on ICL.