Researchers have developed CRUMB, a novel inference wrapper designed to improve the efficiency of prior-fitted networks (PFNs). PFNs are powerful tabular foundation models that can perform in-context learning, but their self-attention mechanisms lead to computationally expensive inference with large datasets. CRUMB addresses this by clustering test queries, selecting distributionally matched training subsets using MMD minimization, and then performing inference on these reduced batches. This method is architecture-agnostic and has demonstrated superior performance on the TabArena benchmark compared to existing context selection strategies, while also showing resilience to covariate drift. AI
IMPACT Enhances efficiency for tabular foundation models, potentially enabling broader application of in-context learning.
RANK_REASON The cluster contains an academic paper detailing a new method for improving model inference.
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →