New research advances tabular foundation models for efficiency and robustness
ByPulseAugur Editorial·[18 sources]·
Multiple research papers explore advancements in tabular foundation models (TFMs), focusing on improving their efficiency, robustness, and adaptability. Techniques such as knowledge distillation are being used to create smaller, faster student models from larger TFMs, while new methods like RelICL and VIP-COP address challenges in relational learning and context optimization. Additionally, research is investigating architectural choices and parameter-efficient adaptation strategies to enhance TFM performance under various data conditions, including subpopulation shifts and large-scale datasets.
AI
IMPACT
These advancements aim to make tabular foundation models more efficient, robust, and adaptable for a wider range of real-world applications.
RANK_REASON
Multiple research papers published on arXiv detailing new methods and models for tabular foundation models.
arXiv:2610.12005v1 Announce Type: new Abstract: Which forms of test-time compute improve the predictions of strong pretrained tabular foundation models (TFMs)? We systematically study this along three axes: adaptation, aggregation, and context construction. Our evaluation spans m…
arXiv:2610.10317v1 Announce Type: new Abstract: Tabular foundation models (TFMs) are pretrained across diverse tabular tasks and make predictions on a new table at inference time using its labeled examples as context. Most recent TFMs perform such in-context prediction with stack…
arXiv:2610.02347v1 Announce Type: new Abstract: Training-data attribution aims to identify which training examples shape model behavior, yet validating such claims is difficult because causal training influence is rarely observable. We argue that controlled synthetic pretraining …
arXiv:2610.01435v1 Announce Type: new Abstract: Tabular foundation models (TFMs) achieve strong predictive performance through in-context learning, yet repeatedly conditioning on labeled data makes inference expensive. Knowledge distillation can reduce this cost by transferring t…
arXiv:2610.01725v1 Announce Type: new Abstract: Tabular foundation models achieve state-of-the-art performance on single-table tasks without any training. Recent work suggests that they are also well-suited for relational learning via deep feature synthesis (DFS), which flattens …
arXiv cs.AI
TIER_1English(EN)·Seonghwi Kim, Sung Ho Jo, Minwoo Chae·
arXiv:2610.01143v1 Announce Type: cross Abstract: Despite strong mean accuracy, tabular foundation models (TFMs) can perform poorly on underrepresented groups under subpopulation shift, where group proportions change between training and deployment. We propose DR-TFM, a parameter…
arXiv cs.LG
TIER_1English(EN)·Pengfei Li, Mohammad Khalil·
arXiv:2609.39628v1 Announce Type: new Abstract: This paper proposes MIND, a marginal-invariant neural dependency diffusion model for mixed-type tabular data. MIND does not directly learn the joint distribution in the original heterogeneous feature space. Instead, it first maps di…
arXiv:2605.12904v2 Announce Type: replace Abstract: Tabular foundation models (TFMs) have emerged as a powerful paradigm for in-context learning on structured data, enabling direct prediction on new tabular tasks without task-specific training. However, their effectiveness is con…
arXiv cs.LG
TIER_1English(EN)·Weihao Kong, Erez Louidor Ilan, Shuxin Nie, Taman Narayan, Rajat Sen, Yichen Zhou, Deqing Fu, Samet Oymak, Abhimanyu Das·
arXiv:2609.37959v1 Announce Type: new Abstract: Tabular machine learning typically relies on per-dataset workflows, fitting tree ensembles or running AutoML searches from scratch for every task. We present TabFM, a 400M-parameter tabular foundation model that formulates supervise…
arXiv:2609.36883v1 Announce Type: new Abstract: Tabular foundation models (TFMs) are increasingly popular because they deliver strong predictions on new datasets through in-context learning, without task-specific training or extensive tuning. Yet released TFMs differ simultaneous…
arXiv cs.LG
TIER_1English(EN)·David Schnurr, Felix Sarnthein, Thomas Hofmann, Imanol Schlag·
arXiv:2609.36337v1 Announce Type: new Abstract: Tabular foundation models achieve strong performance by conditioning on labelled examples in context, but softmax attention limits their use on large datasets. Existing linear-time alternatives, however, are mostly causal, and their…
arXiv cs.LG
TIER_1English(EN)·Deqing Fu, Huangyuan Su, Rajat Sen, Taman Narayan, Sujay Sanghavi, Abhimanyu Das, Weihao Kong·
arXiv:2609.37989v1 Announce Type: new Abstract: Tabular foundation models achieve strong zero-shot accuracy on structured data by pretraining on synthetic tables, but they ignore the column names, task descriptions, and auxiliary files that carry dataset semantics. Meanwhile, sel…
arXiv:2609.27679v2 Announce Type: replace Abstract: A tabular foundation model must discover which distinctions matter for each new table without updating its parameters. We develop in-situ representation refinement: support labels guide changes to the episode's representations, …
arXiv cs.LG
TIER_1English(EN)·Maximilian Schambach, Clemens Biehl, Sam Thelin·
arXiv:2609.31306v2 Announce Type: new Abstract: Tabular in-context learners such as TabPFN, Mitra, or ConTextTab rely on alternating row and column attention over 2D sequences of latent embeddings. These attention patterns differ markedly from the one-dimensional case in language…
arXiv:2609.28208v2 Announce Type: replace Abstract: Wide tables offer tabular foundation models more evidence, but accessing it can exhaust their memory: full-width pairwise mixing grows quadratically with the number of columns, while feature selection makes inputs affordable by …
Tabular foundation models achieve strong zero-shot accuracy on structured data by pretraining on synthetic tables, but they ignore the column names, task descriptions, and auxiliary files that carry dataset semantics. Meanwhile, self-evolving machine learning engineering (MLE) ag…
Tabular machine learning typically relies on per-dataset workflows, fitting tree ensembles or running AutoML searches from scratch for every task. We present TabFM, a 400M-parameter tabular foundation model that formulates supervised tabular prediction as in-context learning. Tab…
arXiv cs.CV
TIER_1English(EN)·Zhenghao Zhao, Chi Zhang, Qingshuang Chen, Yelin Kim·
arXiv:2610.02597v1 Announce Type: new Abstract: Vision foundation models such as DINOv2, SigLIP2, and MASt3R develop complementary capabilities from different pretraining objectives, yet their knowledge remains distributed across separate, specialized models. Multi-teacher knowle…
