New models and methods boost tabular foundation model efficiency
ByPulseAugur Editorial·[11 sources]·
Researchers are developing new tabular foundation models (TFMs) to improve efficiency and performance. TabSwift enhances the TabPFN architecture with row-wise attention and learnable tokens for competitive accuracy and faster inference. LimiX-2M, a smaller model, also outperforms larger baselines by addressing attention bottlenecks and using a novel tokenization framework. Additionally, efforts are underway to speed up TFM pretraining through community-driven 'speedruns' and to compress datasets for faster inference and reduced memory usage.
AI
IMPACT
These advancements aim to make tabular foundation models more efficient and accessible, potentially accelerating their adoption in real-world applications.
RANK_REASON
Multiple research papers introducing new models and methods for tabular foundation models.
arXiv:2605.19662v2 Announce Type: replace Abstract: Tabular foundation models based on pretrained prior-data fitted networks~(PFNs) have shown strong generalization on diverse tabular tasks, but they are typically designed for \emph{non-strategic} settings where data distribution…
arXiv cs.AI
TIER_1English(EN)·Ankai Hao, Ke Chen, Huan Li, Lidan Shou·
arXiv:2606.09004v1 Announce Type: new Abstract: Feature engineering remains essential for tabular data analysis, and Large Language Models (LLMs) have emerged as a promising paradigm for automating this process, giving rise to LLM-powered AuTomated Tabular feature Engineering (LA…
arXiv:2606.07345v1 Announce Type: new Abstract: Tabular foundation models, exemplified by TabPFN, perform prediction via in-context learning, inferring test labels directly from labeled training examples. They have demonstrated competitive performance, particularly on small-to-me…
Tabular foundation models, exemplified by TabPFN, perform prediction via in-context learning, inferring test labels directly from labeled training examples. They have demonstrated competitive performance, particularly on small-to-medium datasets. However, recent tabular foundatio…
arXiv cs.LG
TIER_1English(EN)·Yuanrui Wang, Xingxuan Zhang, Han Yu, Mingchao Ming, Gang Ren, Hao Yuan, Li Mao, Yunjia Zhang, Chun Yuan, Peng Cui·
arXiv:2606.04485v1 Announce Type: new Abstract: Tabular foundation models (TFMs) increasingly rival tree ensembles, but their performance is often compute-inefficient: with standard affine scalar tokenization, each feature injects value variation through an essentially one-dimens…
arXiv cs.LG
TIER_1English(EN)·Salih Bora Ozturk, Alexander Pfefferle, Frank Hutter·
arXiv:2606.03681v1 Announce Type: new Abstract: Pretraining cost is a major bottleneck for research on tabular foundation models, slowing the iteration cycle for new architectures, priors, and optimization ideas. Yet the community lacks a simple way to compare and accumulate pret…
Pretraining cost is a major bottleneck for research on tabular foundation models, slowing the iteration cycle for new architectures, priors, and optimization ideas. Yet the community lacks a simple way to compare and accumulate pretraining speedups. We introduce a community speed…
arXiv cs.LG
TIER_1English(EN)·Guri Zab\"ergja, Rafiq Kamel, Arlind Kadra, Christian M. M. Frey, Josif Grabocka·
arXiv:2602.05649v2 Announce Type: replace Abstract: The long-standing dominance of gradient-boosted decision trees for tabular data has recently been challenged by in-context learning tabular foundation models. In-context learning methods fit and predict in one forward pass witho…
arXiv stat.ML
TIER_1English(EN)·Al Zadid Sultan Bin Habib, Md Younus Ahamed, Prashnna Kumar Gyawali, Gianfranco Doretto, Donald A. Adjeroh·
arXiv:2606.05441v1 Announce Type: cross Abstract: We investigate how to make small tabular foundation models effective for High-Dimensional, Low-Sample Size (HDLSS) tabular prediction without retraining large backbones. We introduce Graph-guided Ordering with Local Refinement (GO…
arXiv stat.ML
TIER_1English(EN)·Donald A. Adjeroh·
We investigate how to make small tabular foundation models effective for High-Dimensional, Low-Sample Size (HDLSS) tabular prediction without retraining large backbones. We introduce Graph-guided Ordering with Local Refinement (GO-LR), show its equivalence to weighted Minimum Lin…
arXiv stat.ML
TIER_1English(EN)·Donald A. Adjeroh·
We investigate how to make small tabular foundation models effective for High-Dimensional, Low-Sample Size (HDLSS) tabular prediction without retraining large backbones. We introduce Graph-guided Ordering with Local Refinement (GO-LR), show its equivalence to weighted Minimum Lin…
