KITE: Kernelized and Information Theoretic Exemplars for In-Context Learning
Researchers have developed KITE, a new method for selecting examples in large language models' in-context learning. KITE uses an information theory-driven approach to optimize example selection for specific user queries, aiming to minimize prediction error. The method incorporates the kernel trick for high-dimensional spaces and a regularizer for example diversity, showing empirical improvements over existing retrieval techniques. AI
IMPACT Improves LLM adaptability to new tasks with limited data by optimizing example selection.