{"id":108714,"url":"https://github.com/jxucoder/awesome-tabular-foundation-models","name":"awesome-tabular-foundation-models","description":"A curated list of resources, papers, and code for Tabular Foundation Models (TFMs), also known as Large Tabular Models (LTMs).","projects_count":130,"last_synced_at":"2026-07-22T19:00:18.381Z","repository":{"id":330255828,"uuid":"1122111499","full_name":"jxucoder/Awesome-Tabular-Foundation-Models","owner":"jxucoder","description":"A curated list of resources, papers, and code for Tabular Foundation Models (TFMs), also known as Large Tabular Models (LTMs).","archived":false,"fork":false,"pushed_at":"2026-06-27T17:01:35.000Z","size":457,"stargazers_count":8,"open_issues_count":0,"forks_count":2,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-07-04T04:03:38.058Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jxucoder.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-24T05:55:17.000Z","updated_at":"2026-06-27T17:01:39.000Z","dependencies_parsed_at":"2026-06-25T06:02:49.361Z","dependency_job_id":null,"html_url":"https://github.com/jxucoder/Awesome-Tabular-Foundation-Models","commit_stats":null,"previous_names":["jxucoder/awesome-tabular-foundation-models"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jxucoder/Awesome-Tabular-Foundation-Models","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jxucoder%2FAwesome-Tabular-Foundation-Models","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jxucoder%2FAwesome-Tabular-Foundation-Models/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jxucoder%2FAwesome-Tabular-Foundation-Models/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jxucoder%2FAwesome-Tabular-Foundation-Models/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jxucoder","download_url":"https://codeload.github.com/jxucoder/Awesome-Tabular-Foundation-Models/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jxucoder%2FAwesome-Tabular-Foundation-Models/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35773465,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-07-20T02:08:10.276Z","status":"online","status_checked_at":"2026-07-22T02:00:06.236Z","response_time":124,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"created_at":"2026-01-17T18:48:31.365Z","updated_at":"2026-07-22T19:00:18.382Z","primary_language":null,"list_of_lists":false,"displayable":true,"categories":["Papers","Key References from Position Paper","Software \u0026 Code","Introduction","Benchmarks \u0026 Evaluation","Tutorials \u0026 Talks"],"sub_categories":["Tabular Classification \u0026 Regression","Related Benchmarks","Potential Applications","Foundations \u0026 Position Papers","Synthetic Data \u0026 Generation","Graph \u0026 Relational Data","Causal Inference","Physical Systems \u0026 ODEs","Optimization","Architectures \u0026 Training","Theory \u0026 Analysis"],"readme":"# Awesome Tabular Foundation Models [![Awesome](https://awesome.re/badge.svg)](https://awesome.re)\n\nA curated list of resources, papers, and code for **Tabular Foundation Models (TFMs)**, also known as **Large Tabular Models (LTMs)**.\n\n## Introduction\n\n**Tabular Foundation Models (TFMs)** are a class of machine learning models pre-trained on diverse tabular datasets to perform tasks on new, unseen tables—often in a single forward pass or with minimal fine-tuning.\n\nDespite tabular data being the dominant modality in many fields—from electronic healthcare records to census data, from cybersecurity to credit scoring, and from finance to natural sciences—it has received surprisingly little attention in the foundation model era. As highlighted by [van Breugel \u0026 van der Schaar (2024)](https://arxiv.org/abs/2405.01147), tabular foundation models are heavily underrepresented in ML research compared to language and vision models.\n\n## Why Tabular Foundation Models?\n\n### The Case for LTMs\n\n1. **Ubiquity of Tabular Data**: Tabular data is everywhere in the real world. Quantitative research across sciences relies on these datasets, which in turn progresses scientific knowledge and influences public policy. This is reflected in the prominence of tabular data in competitions like Kaggle and KDD Cup.\n\n2. **Unsolved Challenges**: Despite two decades of ML research, tree-based models like XGBoost remain among top performers for supervised tabular learning. This presents unique opportunities for foundation model approaches.\n\n3. **Potential Impact**: LTMs could revolutionize how science and ML use tabular data—not as single datasets analyzed in isolation, but contextualized with respect to related datasets.\n\n### Why Tabular FMs Have Been Overlooked\n\n- **Data scarcity**: Until recently, there was a lack of large tabular metadatasets\n- **Difficulty**: Tabular ML baselines are strong; new methods may not consistently outperform them\n- **Human perception**: Text and images are more \"naturally\" interpretable; tabular data is hard to visually inspect\n- **Evaluation challenges**: Unlike images/text where human judgment can assess quality, tabular data evaluation is less intuitive\n\n### Potential Applications\n\nLTMs could serve as invaluable tools for:\n\n- **Data preprocessing \u0026 cleaning**: Automated outlier detection, data validation\n- **Dataset discovery**: Finding relevant datasets across domains or knowledge bases (e.g., Wikipedia tables)\n- **Data augmentation**: Few-shot generation of additional columns, performing semantic joins\n- **Synthetic data generation**: Privacy-preserving data sharing, bias reduction, domain simulation\n- **Automated meta-analyses**: Cross-dataset analysis and insights\n- **Embeddings**: Row or dataset embeddings for downstream prediction tasks\n\n## Contents\n\n- [Introduction](#introduction)\n- [Why Tabular Foundation Models?](#why-tabular-foundation-models)\n- [Software \u0026 Code](#software--code)\n- [Papers](#papers)\n  - [Foundations \u0026 Position Papers](#foundations--position-papers)\n  - [Tabular Classification \u0026 Regression](#tabular-classification--regression)\n  - [Synthetic Data \u0026 Generation](#synthetic-data--generation)\n  - [Graph \u0026 Relational Data](#graph--relational-data)\n  - [Causal Inference](#causal-inference)\n  - [Physical Systems \u0026 ODEs](#physical-systems--odes)\n  - [Optimization](#optimization)\n  - [Architectures \u0026 Training](#architectures--training)\n  - [Theory \u0026 Analysis](#theory--analysis)\n- [Benchmarks \u0026 Evaluation](#benchmarks--evaluation)\n- [Tutorials \u0026 Talks](#tutorials--talks)\n- [Contributing](#contributing)\n\n## Software \u0026 Code\n\n- **[PFNs](https://github.com/automl/PFNs)**: The official general-purpose library for creating and training Prior-Data Fitted Networks.\n- **[TabPFN](https://github.com/PriorLabs/TabPFN)**: The official repository for the TabPFN model.\n- **[TabICL](https://github.com/soda-inria/tabicl)**: Official implementation of TabICL and TabICLv2, open-source tabular foundation models for classification and regression.\n- **[TabSwift](https://github.com/LAMDA-Tabular/TabSwift)**: Official code for TabSwift, a lightweight row-wise attention TFM with adaptive early-exit.\n- **[Nori](https://github.com/Synthefy/synthefy-nori)**: Synthefy's fully open-source (weights, inference, and training code) tabular foundation model for regression (`pip install synthefy-nori`).\n- **[TabTune](https://arxiv.org/abs/2511.02802)**: A unified library for inference and fine-tuning across multiple tabular foundation models.\n- **[TabArena](https://github.com/autogluon/tabarena)**: A living benchmarking system for tabular ML with a public leaderboard at [tabarena.ai](https://tabarena.ai).\n\n## Papers\n\n### Foundations \u0026 Position Papers\n\n- **Why Tabular Foundation Models Should Be a Research Priority** (ICML 2024)\n  *Boris van Breugel, Mihaela van der Schaar*\n  [Paper](https://arxiv.org/abs/2405.01147)\n  \u003e A seminal position paper arguing for developing Large Tabular Models (LTMs). Discusses why tabular data is underrepresented in FM research, proposes desiderata for LTMs, and outlines potential applications: from few-shot tabular models to automating data science, and from synthetic data to empowering multidisciplinary scientific discovery.\n\n- **Unlocking the Full Potential of Data Science Requires Tabular Foundation Models, Agents, and Humans** (NeurIPS 2025 Position Paper Track)\n  *Tianji Cong, Julian Martin Eisenschlos, Daniel Gomm, Leo Grinsztajn, Andreas C Mueller, Anupam Sanghi, Jan-Micha Bodensohn, Vadim Borisov, et al.*\n  [Paper](https://openreview.net/forum?id=aXMPvmBAm5)\n  \u003e Argues that the future of data science lies in collaborative systems that tightly integrate agents, tabular foundation models (TFMs), and human experts. Presents a research agenda for more accessible, robust, and human-centered data science.\n\n- **Transformers Can Do Bayesian Inference** (ICLR 2022)\n  *Samuel Müller, Noah Hollmann, Sebastian Pineda Arango, Josif Grabocka, Frank Hutter*\n  [Paper](https://openreview.net/forum?id=KSugKcbNf9) | [Code](https://github.com/automl/PFNs)\n  \u003e The seminal paper introducing Prior-Data Fitted Networks (PFNs). It demonstrates how Transformers can be trained to approximate posterior predictive distributions for diverse priors, including Gaussian Processes and simple neural networks.\n\n- **Towards Foundation Models for Learning on Tabular Data** (arXiv 2023)\n  *Hongyu Zhang, Xingyu Wen, Shuai Zheng, Wei Xu, Jiang Bian*\n  [Paper](https://arxiv.org/abs/2310.07338)\n  \u003e Explores approaches toward building foundation models specifically designed for tabular data learning.\n\n### Tabular Classification \u0026 Regression\n\n- **TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second** (ICLR 2023)\n  *Noah Hollmann, Samuel Müller, Katharina Eggensperger, Frank Hutter*\n  [Paper](https://openreview.net/forum?id=cp5PboIqf7r) | [Code](https://github.com/automl/TabPFN)\n  \u003e Applies PFNs to tabular classification. TabPFN is a single model pre-trained on synthetic data that achieves state-of-the-art performance on small tabular datasets, often beating Gradient Boosted Decision Trees without tuning.\n\n- **TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Models** (arXiv 2025)\n  *Léo Grinsztajn, Klemens Flöge, Oscar Key, Felix Birkel, Philipp Jund, Brendan Roof, Benjamin Jäger, Dominik Safaric, Simone Alessi, Adrian Hayler, Mihir Manium, Rosen Yu, Felix Jablonski, Shi Bin Hoo, Anurag Garg, Jake Robertson, Magnus Bühler, Vladyslav Moroshan, Lennart Purucker, Clara Cornu, Lilly Charlotte Wehrhahn, Alessandro Bonetto, Bernhard Schölkopf, Sauraj Gambhir, Noah Hollmann, Frank Hutter*\n  [Paper](https://arxiv.org/abs/2511.08667) | [Code](https://github.com/PriorLabs/TabPFN)\n  \u003e TabPFN-2.5 is built for datasets with up to 50,000 data points and 2,000 features, a 20x increase in data cells compared to TabPFNv2. Default TabPFN-2.5 has a 100% win rate against default XGBoost on small to medium-sized classification datasets and an 87% win rate on larger datasets up to 100K samples. Includes a distillation engine to convert TabPFN-2.5 into compact MLPs or tree ensembles for low-latency production deployment.\n\n- **TabPFN-3 Technical Report** (Technical Report, 2026)\n  *Prior Labs Team*\n  [Paper](https://arxiv.org/abs/2605.13986)\n  \u003e The next generation of TabPFN, scaling state-of-the-art in-context learning to datasets with up to 1M training rows on a single GPU. Features a redesigned architecture with an attention-based many-class decoder, an improved preprocessing pipeline, inference-time optimizations, and an enhanced synthetic SCM prior. Brings substantial gains on time series, relational, and tabular-text data.\n\n- **TabSwift: An Efficient Tabular Foundation Model with Row-Wise Attention** (ICML 2026, Spotlight)\n  *Si-Yang Liu, Han-Jia Ye*\n  [Paper](https://arxiv.org/abs/2606.07345) | [Code](https://github.com/LAMDA-Tabular/TabSwift)\n  \u003e Revisits the original TabPFN design, showing a lightweight row-wise attention-only backbone stays competitive with two enhancements: gated attention stabilization and learnable register tokens for global context. Supports both classification and regression, and adds an adaptive layer-wise early-exit mechanism that dynamically adjusts inference depth per sample for anytime, latency-sensitive serving.\n\n- **Nori: A Tabular Foundation Model for Regression Trained on Synthetic Data** (Open-source release, Synthefy, 2026)\n  *Synthefy (Po-han Li, Aditya Narayanan, Sai Shankar Narasimhan, et al.)*\n  [Model](https://huggingface.co/Synthefy/Nori) | [Code](https://github.com/Synthefy/synthefy-nori) | [Blog](https://www.synthefy.com/blog/synthefy-tabular-release)\n  \u003e A compact (~6M parameter) FeaturesTransformer for tabular regression via in-context learning, trained entirely on synthetic data. Alternates feature attention (across columns) and sample attention (across rows), uses RBF feature embeddings with native missing-value handling, and predicts a full quantile distribution via pinball loss. Reported #1 on aggregate across 96 real-world datasets, beating tuned XGBoost and LightGBM on ~80% of them at ~1/10 the size of peers.\n\n- **TabDPT: Scaling Tabular Foundation Models on Real Data** (NeurIPS 2025)\n  *University of Toronto / Layer 6 AI*\n  [Paper](https://www.cs.toronto.edu/~mvolkovs/NeurIPS2025_TabDPT.pdf)\n  \u003e A foundation model pre-trained on real-world tabular datasets using self-supervised learning (masking) and retrieval. Shows robust generalization to unseen tables without task-specific fine-tuning.\n\n- **TabICL: A Tabular Foundation Model for In-Context Learning** (ICLR 2025)\n  [Paper](https://openreview.net/forum?id=0VvD1PmNzM)\n  \u003e A model trained on millions of synthetic tables for direct in-context task learning. Emphasizes scalability and efficiency.\n\n- **XTab: Cross-table Pretraining for Tabular Transformers** (ICML 2023)\n  *Bingzhao Zhu, Xingjian Shi, Nick Erickson, Mu Li, George Karypis, Mahsa Shoaran*\n  [Paper](https://arxiv.org/abs/2305.06090)\n  \u003e Cross-table pretraining approach enabling knowledge transfer across diverse tabular datasets with different schemas.\n\n- **On Finetuning Tabular Foundation Models** (arXiv 2025)\n  *Ivan Rubachev, Akim Kotelnikov, Nikolay Kartashev*\n  [Paper](https://arxiv.org/abs/2506.08982) | [Code](https://github.com/yandex-research/tabpfn-finetuning)\n  \u003e Systematically evaluates finetuning strategies for TabPFN (specifically TabPFNv2), finding that full finetuning is efficient and effective for adapting to new tasks.\n\n- **TuneTables: Context Optimization for Scalable Prior-Data Fitted Networks** (ICML 2024)\n  *Benjamin Feuer, Kelly Zhang, Jennifer J. Sun, Asma Ghandeharioun, Frank Hutter, et al.*\n  [Paper](https://arxiv.org/abs/2402.11137) | [Code](https://github.com/automl/TuneTables)\n  \u003e Proposes \"context optimization\" (prompt tuning) to scale PFNs to larger datasets and improve performance by optimizing the input context.\n\n- **Scaling TabPFN: Sketching and Feature Selection for Tabular Prior-Data Fitted Networks** (NeurIPS 2023 Workshop)\n  *Benjamin Feuer, Chinmay Hegde, Niv Cohen*\n  [Paper](https://openreview.net/forum?id=b0OhN0ii36)\n  \u003e Investigates how to summarize large training datasets before feeding them to TabPFN. Studies sketching and feature selection methods, finding that feature dimensionality reduction is more impactful than sample sketching for scaling TabPFN to datasets exceeding its 100-feature/1000-sample limits.\n\n- **Drift-Resilient TabPFN: In-Context Learning Temporal Distribution Shifts on Tabular Data** (NeurIPS 2024)\n  *Kai Helli, David Schnurr, Noah Hollmann, Samuel Müller, Frank Hutter*\n  [Paper](https://neurips.cc/virtual/2024/poster/93581) | [OpenReview](https://openreview.net/forum?id=93581)\n  \u003e Addresses temporal distribution shifts in tabular data using a TabPFN-based approach. Uses structural causal models (SCMs) that gradually shift over time, with a secondary SCM to model parameter changes. Outperforms XGBoost, CatBoost, and standard TabPFN on 18 datasets while maintaining strong calibration.\n\n- **TabICLv2: A Better, Faster, Scalable, and Open Tabular Foundation Model** (arXiv 2026)\n  *Jingang Qu, David Holzmüller, Gaël Varoquaux, Marine Le Morvan*\n  [Paper](https://arxiv.org/abs/2602.11139) | [Code](https://github.com/soda-inria/tabicl)\n  \u003e A new state-of-the-art tabular foundation model for regression and classification, built on a novel synthetic data generation engine, scalable softmax attention, and the Muon optimizer. Without any tuning, TabICLv2 surpasses RealTabPFN-2.5 (hyperparameter-tuned, ensembled, and fine-tuned) on TabArena and TALENT benchmarks while being 10x faster.\n\n- **Real-TabPFN: Improving Tabular Foundation Models via Continued Pre-training With Real-World Data** (arXiv 2025)\n  *Anurag Garg, Muhammad Ali, Noah Hollmann, Lennart Purucker, Samuel Müller, Frank Hutter*\n  [Paper](https://arxiv.org/abs/2507.03971)\n  \u003e Shows that TabPFN performance can be significantly boosted by continued pre-training on a small, curated collection of large real-world datasets from OpenML and Kaggle, bridging the synthetic-to-real gap. Real-TabPFN outperforms default TabPFNv2 and every other baseline on 29 AutoML Benchmark datasets.\n\n- **EquiTabPFN: A Target-Permutation Equivariant Prior Fitted Network** (arXiv 2025)\n  *Michael Arbel, David Salinas, Frank Hutter*\n  [Paper](https://arxiv.org/abs/2502.06684)\n  \u003e Addresses the limitation that TabPFN-like models are constrained to a fixed number of target dimensions due to a lack of target equivariance. Introduces a fully target-equivariant architecture that eliminates the \"equivariance gap,\" matching or surpassing existing methods on benchmarks with more classes than seen during pre-training.\n\n- **TabSTAR: A Foundation Tabular Model With Semantically Target-Aware Representations** (NeurIPS 2025)\n  *Alan Arazi, Eilam Shapira, Roi Reichart*\n  [Paper](https://arxiv.org/abs/2505.18125) | [Code](https://github.com/alanarazi7/TabSTAR)\n  \u003e Introduces target-aware tokens that integrate the target variable's identity as model input, combined with semantic encoding via a pretrained language model. Achieves state-of-the-art on classification benchmarks with text features through multi-task pretraining and LoRA finetuning.\n\n- **TARTE: Table Foundation Models — On Knowledge Pre-training for Tabular Learning** (TMLR 2025)\n  *Myung Jun Kim, Félix Lefebvre, Gaëtan Brison, Alexandre Perez-Lebel, Gaël Varoquaux*\n  [Paper](https://arxiv.org/abs/2505.14415) | [Code](https://github.com/soda-inria/tarte-ai)\n  \u003e A pre-trained tabular model that captures associations between strings and numbers via knowledge pre-training. Enables simple downstream learners like Ridge regression to become strong baselines, and its transformer backbone can be reused and specialized for complex tables.\n\n- **TabPFN-TS: From Tables to Time — Extending TabPFN-v2 to Time Series Forecasting** (arXiv 2025)\n  *Shi Bin Hoo, Samuel Müller, David Salinas, Frank Hutter*\n  [Paper](https://arxiv.org/abs/2501.02945)\n  \u003e Treats forecasting as a tabular regression problem by combining lightweight temporal featurization with TabPFN-v2. Requires no time-series-specific pretraining and, despite its compact 11M parameters, achieves state-of-the-art performance on covariate-informed forecasting.\n\n- **TabClustPFN: A Prior-Fitted Network for Tabular Data Clustering** (arXiv 2026)\n  *Tianqi Zhao, Guanyang Wang, Yan Shuo Tan, Qiong Zhang*\n  [Paper](https://arxiv.org/abs/2601.21656)\n  \u003e A PFN for tabular data clustering that performs amortized Bayesian inference over both cluster assignments and cluster cardinality. Pretrained on synthetic datasets, it clusters unseen datasets in a single forward pass without retraining or hyperparameter tuning.\n\n- **TabImpute: Universal Zero-Shot Imputation for Tabular Data** (arXiv 2025)\n  *Jacob Feitelberg, Dwaipayan Saha, Kyuseong Choi, Zaid Ahmad, Anish Agarwal, Raaz Dwivedi*\n  [Paper](https://arxiv.org/abs/2510.02625) | [Code](https://github.com/jacobf18/tabular)\n  \u003e Building on TabPFN, a pre-trained transformer delivering accurate and fast zero-shot imputations. Introduces entry-wise featurization for 100x speedup and MissBench, a comprehensive benchmark with 42 OpenML tables and 13 missingness patterns.\n\n- **TabDPT-Turbo: Efficient In-Context Learning for Tabular Prediction** (ICML 2026 FMSD Workshop)\n  *Rasa Hosseinzadeh, Alex Labach, Zexin Xue, Shuyi Han, Valentin Thomas, Anthony L. Caterini*\n  [OpenReview](https://openreview.net/forum?id=Y00pwFyrHR)\n  \u003e An efficiency-focused variant of TabDPT that speeds up in-context learning for tabular prediction.\n\n- **Localized TabICLv2: Scaling Tabular In-Context Learning through k-NN** (ICML 2026 FMSD Workshop)\n  *Beimnet Bekele Guta*\n  [OpenReview](https://openreview.net/forum?id=ddITrUyMTB)\n  \u003e Scales TabICLv2 to larger datasets by restricting the in-context examples to a k-nearest-neighbor retrieval around each query.\n\n- **FlexTab: Towards a Flexible Encoder-Decoder Architecture for Tabular In-Context Learning** (ICML 2026 FMSD Workshop)\n  *Marek Polewczyk, Maximilian Schambach, Marco Spinaci, Sam Thelin, Johannes Höhne*\n  [OpenReview](https://openreview.net/forum?id=fOph6xxdyP)\n  \u003e Proposes a flexible encoder-decoder backbone for tabular in-context learning, aiming to handle heterogeneous schemas and tasks.\n\n- **Memory Efficient Tabular Foundation Models** (ICML 2026 FMSD Workshop)\n  *Shuting Luo, Monika Mikhail Kanaan, Cameron Gordon, Anna Leontjeva, Simon Lucey*\n  [OpenReview](https://openreview.net/forum?id=1Ov4RAWuW4)\n  \u003e Studies techniques to reduce the memory footprint of tabular foundation models at inference and/or training time.\n\n- **TFM-Retouche: A Lightweight Input-Space Adapter for Tabular Foundation Models** (ICML 2026 FMSD Workshop)\n  *Duong Nguyen, Mohammed Jawhar, Nicolas Chesneau*\n  [OpenReview](https://openreview.net/forum?id=P1bvn0jvGX)\n  \u003e Adapts frozen tabular foundation models to new tasks via a lightweight input-space transformation rather than fine-tuning model weights.\n\n- **Pocket Foundation Models: Distilling TFMs into CPU-Ready GBDTs** (ICML 2026 FMSD Workshop)\n  *Aditya Tanna, Nassim Bouarour, Mohamed Bouadi, Vinay Sankarapu, Pratinav Seth*\n  [OpenReview](https://openreview.net/forum?id=n1TUx8fHpv)\n  \u003e Distills tabular foundation models into compact gradient-boosted decision trees for fast, CPU-friendly deployment.\n\n- **Bounded Context Management for Tabular Foundation Models on Stream Learning** (ICML 2026 FMSD Workshop, Spotlight)\n  *Jinmo Lee, Doyun Choi, Moongi Choi, Jaemin Yoo*\n  [OpenReview](https://openreview.net/forum?id=L94GfndIir)\n  \u003e Manages a bounded in-context set so that tabular foundation models can operate under streaming data with limited memory.\n\n- **Online Test-Time Adaptation in Tabular Data with Minimal High-Certainty Samples** (ICML 2026 FMSD Workshop)\n  *Mingming Zhang, Zhiqing Xiao, Junbo Zhao*\n  [OpenReview](https://openreview.net/forum?id=rmpLcZtJ4l)\n  \u003e Performs online test-time adaptation for tabular models using a small set of high-confidence samples.\n\n- **Agentic Data Intelligence for General Tabular Modeling** (ICML 2026 FMSD Workshop)\n  *Jun-Peng Jiang, An-Yang Ji, Jia-Yi Zhu, Han-Jia Ye*\n  [OpenReview](https://openreview.net/forum?id=pj1XShgzSv)\n  \u003e Explores an agentic pipeline that automates general tabular modeling tasks.\n\n- **Correcting Class Imbalance in Prior-Data Fitted Networks for Tabular Classification** (ICML 2026 FMSD Workshop)\n  *Samuel McDowell, Nathan Stromberg, Lalitha Sankar*\n  [OpenReview](https://openreview.net/forum?id=96HA4mxjkH)\n  \u003e Addresses degraded PFN performance under class imbalance in tabular classification.\n\n- **SurvivalPFN: Amortizing Survival Prediction via In-Context Bayesian Inference** (ICML 2026 FMSD Workshop, Spotlight)\n  *Shi-ang Qi, Vahid Balazadeh, Michael Cooper, Russell Greiner, Rahul G. Krishnan*\n  [OpenReview](https://openreview.net/forum?id=PDik7bpFhE)\n  \u003e A PFN-style model that performs amortized in-context Bayesian inference for survival (time-to-event) prediction.\n\n- **SurvPFN: Towards Foundation Models for Survival Predictions** (ICML 2026 FMSD Workshop)\n  *Samuel Böhm, Lennart Purucker, Frank Hutter, Pascal Schlosser*\n  [OpenReview](https://openreview.net/forum?id=kDEHp7ytr2)\n  \u003e Works toward prior-data fitted foundation models tailored to survival analysis tasks.\n\n- **Staying Alive: Uncensored Survival Analysis with Tabular Foundation Models** (ICML 2026 FMSD Workshop)\n  *Mariana Vargas Vieyra*\n  [OpenReview](https://openreview.net/forum?id=2EFykQheZD)\n  \u003e Investigates applying tabular foundation models to survival analysis, handling censoring in time-to-event data.\n\n### Synthetic Data \u0026 Generation\n\n- **TabuLa: Harnessing Language Models for Tabular Data Synthesis** (arXiv 2023)\n  *Zilong Zhao, Robert Birke, Lydia Y. Chen*\n  [Paper](https://arxiv.org/abs/2310.12746)\n  \u003e Leverages language models for generating synthetic tabular data.\n\n- **CTSyn: A Foundation Model for Cross Tabular Data Generation** (arXiv 2024)\n  *Xiaofeng Lin, Chenheng Xu, Matthew Yang, Guang Cheng*\n  [Paper](https://arxiv.org/abs/2406.04619)\n  \u003e A diffusion-based generative foundation model for tabular data. Uses an autoencoder to consolidate diverse tables into a unified latent space and a conditional latent diffusion model for generation, conditioned on table schema. Outperforms existing synthesizers on standard benchmarks in both utility and diversity.\n\n- **A Generative Foundation Model for Heterogeneous Tabular Data** (ICML 2026 FMSD Workshop)\n  *Xiangjian Jiang, Mingxuan Liu, Nikola Simidjievski, Tassilo Klein, Mateja Jamnik*\n  [OpenReview](https://openreview.net/forum?id=RcsaxrdpfE)\n  \u003e A generative foundation model designed to synthesize heterogeneous tabular data across mixed column types.\n\n- **TableFactory: Generating Semantically Linked Tabular Data via Multi-Agent Behavioral Simulation** (ICML 2026 FMSD Workshop)\n  *Mingxuan Liu, Xiangjian Jiang, Johannes Hoffart, Tassilo Klein*\n  [OpenReview](https://openreview.net/forum?id=3bzbWeaL5j)\n  \u003e Generates semantically linked tabular data by simulating the behavior of multiple interacting agents.\n\n- **Hierarchical Synthetic Tabular Data Generation: A Hybrid Top-Down and Bottom-Up Framework** (ICML 2026 FMSD Workshop)\n  *Junfeng Nie, Alvin Jin, Xiaohui Chen*\n  [OpenReview](https://openreview.net/forum?id=RiaXCBoWje)\n  \u003e A hybrid framework combining top-down and bottom-up strategies for hierarchical synthetic tabular data generation.\n\n- **Implicit Reward Alignment For Training Causally-Coherent Tabular Data Generators** (ICML 2026 FMSD Workshop)\n  *Matea Gjika, Giuseppe Iannone, Luca Sfragara, Pavithra Harsha, Georgia Perakis*\n  [OpenReview](https://openreview.net/forum?id=Bei8F38H9r)\n  \u003e Uses implicit reward alignment to train tabular data generators that preserve causal coherence.\n\n- **From Noisy Oracles to Useful Constraints: LLM-Guided Constraint Selection for Synthetic Tabular Data** (ICML 2026 FMSD Workshop)\n  *Tejumade Afonja, Joscha Cüppers, Mario Fritz*\n  [OpenReview](https://openreview.net/forum?id=1k9oK22A3R)\n  \u003e Leverages LLMs to select useful constraints from noisy oracle signals to improve synthetic tabular data generation.\n\n### Graph \u0026 Relational Data\n\n- **GraphPFN: A Prior-Data Fitted Graph Foundation Model** (NeurIPS 2025)\n  [Paper](https://arxiv.org/abs/2509.21489)\n  \u003e Introduces GraphPFN, a PFN designed for node-level prediction tasks. It utilizes a novel prior distribution of synthetic attributed graphs and incorporates graph-aware structured causal models to generate node attributes and targets.\n\n- **Introducing KumoRFM: A Foundation Model for In-Context Learning on Relational Data** (Whitepaper, 2025)\n  *Matthias Fey, Vid Kocijan, Federico Lopez, Jure Leskovec*\n  [Blog \u0026 Whitepaper](https://kumo.ai/company/news/kumo-relational-foundation-model/)\n  \u003e A Relational Foundation Model (RFM) extending in-context learning to multi-table relational graphs. It uses a Relational Graph Transformer to reason across arbitrary schemas and supports zero-shot prediction.\n\n- **Foundation Models for Tabular Data within Systemic Context (FMSLT)** (arXiv 2025)\n  [Paper](https://arxiv.org/abs/2505.19825)\n  \u003e Proposes modeling tabular data not in isolation but with explicit semantic and operational context (e.g., linked tables, foreign keys), aiming for a richer foundation than flat tabular learning.\n\n- **G2T-FM: Turning Tabular Foundation Models into Graph Foundation Models** (arXiv 2025)\n  *Dmitry Eremeev, Gleb Bazhenov, Oleg Platonov, Artem Babenko, Liudmila Prokhorenkova*\n  [Paper](https://arxiv.org/abs/2508.20906) | [Code](https://github.com/yandex-research/G2T-FM)\n  \u003e Transforms graph tasks into tabular ones by augmenting features with neighborhood aggregations and structural features (degree, PageRank, Laplacian eigenvectors). Achieves strong results in a fully in-context regime, outperforming existing GFMs and performing on par with well-tuned GNNs.\n\n- **TFM4GAD: Tabular Foundation Models are Strong Graph Anomaly Detectors** (WebConf 2026)\n  *Yunhui Liu, et al.*\n  [Paper](https://arxiv.org/abs/2601.17301) | [Code](https://github.com/Cloudy1225/TFM4GAD)\n  \u003e Adapts tabular foundation models for graph anomaly detection by flattening the graph into an augmented feature table with Laplacian embeddings, structural characteristics, and anomaly-sensitive neighborhood aggregations. The best variant achieves 89.92% AUROC, surpassing the strongest trained baseline.\n\n- **Large-Scale Pretraining unlocks Few-Shot Prediction for Relational Data** (ICML 2026 FMSD Workshop)\n  *Rishabh Ranjan, Vignesh Kothapalli, Harshvardhan Agarwal, Charilaos I. Kanatsoulis, Roshan Reddy Upendra, Tom Palczewski, Carlos Guestrin, Jure Leskovec*\n  [OpenReview](https://openreview.net/forum?id=oQINTd9din)\n  \u003e Shows that large-scale pretraining enables few-shot prediction across multi-table relational databases.\n\n- **Parameter-Free Encoders Remain Viable for RDB Foundation Models** (ICML 2026 FMSD Workshop)\n  *Linjie Xu, David Wipf*\n  [OpenReview](https://openreview.net/forum?id=wRWaegFYMx)\n  \u003e Argues that parameter-free encoders remain a competitive design choice for relational database (RDB) foundation models.\n\n- **PluRel-to-RDB-PFN: Schema-Guided Synthetic Relational Pretraining** (ICML 2026 FMSD Workshop)\n  *Mohammad Sadeq Abolhasani, Viswanath Ganapathy*\n  [OpenReview](https://openreview.net/forum?id=RpNvhdvd2v)\n  \u003e A PFN for relational databases pretrained on schema-guided synthetic relational data.\n\n- **Context Window Failures in Relational Foundation Models** (ICML 2026 FMSD Workshop)\n  *Denis Oliveira Correa, Francisco Galuppo Azevedo*\n  [OpenReview](https://openreview.net/forum?id=lkuOIfXLwJ)\n  \u003e Analyzes how relational foundation models degrade when relevant context exceeds their effective context window.\n\n- **Beyond Average Leaderboards: When Explicit Graph Priors Help Tabular Foundation Models** (ICML 2026 FMSD Workshop)\n  *Franck Le, Keith Grueneberg, Erich M. Nahum, Vadim Sheinin*\n  [OpenReview](https://openreview.net/forum?id=4gmLDG0aGC)\n  \u003e Examines the conditions under which adding explicit graph priors benefits tabular foundation models, beyond aggregate leaderboard scores.\n\n- **Can LLMs Use Relational Transformer Embeddings?** (ICML 2026 FMSD Workshop)\n  *Francisco Galuppo Azevedo, Clarissa Lima Loures*\n  [OpenReview](https://openreview.net/forum?id=Z2n7WcIy6j)\n  \u003e Investigates whether large language models can effectively consume embeddings produced by relational transformers.\n\n### Causal Inference\n\n- **Foundation Models for Causal Inference via Prior-Data Fitted Networks (CausalFM)** (NeurIPS 2025)\n  *Yuchen Ma, et al.*\n  [Paper](https://arxiv.org/abs/2506.10914)\n  \u003e A framework for training PFN-based foundation models for various causal inference settings (back-door, front-door adjustments).\n\n- **Do-PFN: In-Context Learning for Causal Effect Estimation** (NeurIPS 2025)\n  *Jake Robertson, et al.*\n  [Paper](https://arxiv.org/abs/2506.06039)\n  \u003e Applies PFNs to estimate causal effects without knowledge of the underlying causal graph.\n\n- **Causal Foundation Models with Continuous Treatments** (ICML 2026 FMSD Workshop)\n  *Christopher Stith, Medha Barath, Vahid Balazadeh, Jesse C. Cresswell, Rahul G. Krishnan*\n  [OpenReview](https://openreview.net/forum?id=DzcWAYcR2n)\n  \u003e Extends PFN-based causal foundation models to settings with continuous (rather than binary) treatments.\n\n- **Foundation Models for Partial Causal Identification** (ICML 2026 FMSD Workshop)\n  *Alexis Bellot, Anish Dhir*\n  [OpenReview](https://openreview.net/forum?id=jCbehzZBsk)\n  \u003e Studies foundation models for cases where causal effects are only partially identifiable.\n\n- **Inducing Causal Order through Tabular In-Context Learning** (ICML 2026 FMSD Workshop)\n  *Sascha Xu, Sarah Mameche, Jilles Vreeken*\n  [OpenReview](https://openreview.net/forum?id=U4KiOBxY1X)\n  \u003e Uses tabular in-context learning to infer a causal ordering among variables.\n\n- **CausalTab: Pretraining Across Causal Environments for Tabular Causal Discovery** (ICML 2026 FMSD Workshop)\n  *Zi-Rong Li, Si-Yang Liu, Tian-Zuo Wang, Han-Jia Ye*\n  [OpenReview](https://openreview.net/forum?id=og3UVhP7M1)\n  \u003e Pretrains across diverse causal environments to enable tabular causal discovery.\n\n- **Causal Foundation Models Perform Better without Post-treatment Variables** (ICML 2026 FMSD Workshop)\n  *Junha Ham, Deokgyu Kim, Doeun Kim, Serjin Kim, Sanghack Lee*\n  [OpenReview](https://openreview.net/forum?id=ULoLF1aOo1)\n  \u003e Shows that excluding post-treatment variables improves the accuracy of causal foundation models.\n\n- **A Causal Foundation Model for Structure and Outcome Prediction** (ICML 2026 FMSD Workshop)\n  *Max Zhu, Martino Mansoldo, Ching-Hao Wang, Stefan Groha*\n  [OpenReview](https://openreview.net/forum?id=GOf9c4lOCf)\n  \u003e A causal foundation model that jointly targets causal structure and outcome prediction.\n\n- **Bayesian Tabular Few-shot Learning with Causal Information** (ICML 2026 FMSD Workshop)\n  *Ole Ossen, Jake Robertson, Arik Reuter, Magnus Bühler, Lennart Purucker, Frank Hutter*\n  [OpenReview](https://openreview.net/forum?id=2yvEiFhNCT)\n  \u003e Incorporates causal information into Bayesian few-shot learning for tabular tasks.\n\n### Physical Systems \u0026 ODEs\n\n- **Decoupled-Value Attention for Prior-Data Fitted Networks: GP Inference for Physical Equations** (NeurIPS 2025)\n  *Kaustubh Sharma, et al.*\n  [Paper](https://arxiv.org/abs/2509.20950)\n  \u003e Introduces Decoupled-Value Attention (DVA) to improve performance in physical systems and high-dimensional regression. Specifically targets inference for physical equations (ODEs/PDEs) using PFNs.\n\n### Optimization\n\n- **Efficient Bayesian Learning Curve Extrapolation using Prior-Data Fitted Networks** (NeurIPS 2023)\n  *Steven Adriaensen, et al.*\n  [Paper](https://proceedings.neurips.cc/paper_files/paper/2023/hash/3f1a5e8bfcc3005724d246abe454c1e5-Abstract-Conference.html)\n  \u003e LC-PFN is trained to extrapolate learning curves, enabling accurate posterior predictive distributions and efficient model selection / early stopping.\n\n- **PFNs4BO: In-Context Learning for Bayesian Optimization** (ICML 2023)\n  *Samuel Müller, et al.*\n  [Paper](https://arxiv.org/abs/2305.17535)\n  \u003e Explores the application of PFNs as surrogate models for Bayesian Optimization.\n\n- **Context-Aware Learning Curve Extrapolation with Prior-Data Fitted Networks** (ICML 2026 FMSD Workshop)\n  *Cheng Yan, Steven Adriaensen, Tom Julian Viering*\n  [OpenReview](https://openreview.net/forum?id=oN4FIXBVeS)\n  \u003e Extends PFN-based learning-curve extrapolation to incorporate additional context about the training run.\n\n- **Can Tabular Foundation Models Predict Algorithm Runtime Distributions?** (ICML 2026 FMSD Workshop)\n  *Hagverdi Ibrahimli, Steven Adriaensen*\n  [OpenReview](https://openreview.net/forum?id=H6t3IZfnqt)\n  \u003e Investigates whether tabular foundation models can predict the runtime distributions of algorithms.\n\n- **Covariance-Aware Transformers for Quadratic Programming and Decision Making** (ICML 2026 FMSD Workshop)\n  *Kutay Tire, Yufan Zhang, Ege Onur Taga, Samet Oymak*\n  [OpenReview](https://openreview.net/forum?id=XLFOyHaZmq)\n  \u003e Introduces covariance-aware transformers for quadratic programming and downstream decision-making problems.\n\n- **Lookahead Automated Feature Engineering for Tabular Prediction via Kaggle-Guided Knowledge Transfer** (ICML 2026 FMSD Workshop)\n  *Si-Yang Liu, Zong-Da Li, Chenming Xu, Han Li, Rui-Qiao Chen, Han-Jia Ye*\n  [OpenReview](https://openreview.net/forum?id=FNtlPbGvwc)\n  \u003e Automates feature engineering for tabular prediction using lookahead search and knowledge transferred from Kaggle solutions.\n\n- **Evolutionary Feature Engineering for Structured Data** (ICML 2026 FMSD Workshop)\n  *Ege Onur Taga, Yilin Zhuang, Muhammed Emrullah Ildiz, Petros Mol, Abhimanyu Das, Karthik Duraisamy, Samet Oymak*\n  [OpenReview](https://openreview.net/forum?id=EruNY8fps7)\n  \u003e Applies evolutionary search to automatically discover useful features for structured data.\n\n### Architectures \u0026 Training\n\n- **MotherNet: A Foundational Hypernetwork for Tabular Classification** (arXiv 2023)\n  *Samuel Müller, Frank Hutter*\n  [Paper](https://arxiv.org/abs/2312.08598) | [Code](https://github.com/automl/MotherNet)\n  \u003e A hypernetwork trained to generate the weights of a child network for a new tabular task in a single forward pass.\n\n- **TabularFM: An Open Framework For Tabular Foundational Models** (arXiv 2024)\n  [Paper](https://arxiv.org/abs/2406.09837)\n  \u003e A framework and dataset corpus designed to facilitate the training of various generative and tabular foundation models.\n\n- **State-Space Models for Tabular Prior-Data Fitted Networks** (arXiv 2025)\n  *Felix Koch, Marcel Wever, Fabian Raisch, Benjamin Tischler*\n  [Paper](https://arxiv.org/abs/2510.14573)\n  \u003e Investigates using Hydra, a bidirectional linear-time structured state space model (SSM), as an alternative to Transformers in TabPFN. Proposes repeated context permutations (RCP) to reduce order-sensitivity. Achieves competitive predictive performance with reduced computational and memory complexity.\n\n- **MultiModalPFN: Extending Prior-Data Fitted Networks for Multimodal Tabular Learning** (CVPR 2026)\n  *Wall Kim, Chaeyoung Song, Hanul Kim*\n  [Paper](https://arxiv.org/abs/2602.20223) | [Code](https://github.com/too-z/MultiModalPFN)\n  \u003e Extends TabPFN to handle tabular and non-tabular modalities (images, text) in a unified manner. Uses modality projectors with multi-head gated MLP and cross-attention pooler to transform non-tabular embeddings into tabular-compatible tokens.\n\n- **Robust Tabular Foundation Models (RTFM)** (arXiv 2025)\n  *Matthew Peroni, Franck Le, Vadim Sheinin*\n  [Paper](https://arxiv.org/abs/2512.03307)\n  \u003e A model-agnostic adversarial training framework that adapts the synthetic data generator to emphasize challenging datasets during training. Applied to TabPFN V2, RTFM improves benchmark performance by up to 6% in mean normalized AUC using fewer than 100K additional synthetic datasets.\n\n- **Speedrunning Tabular Foundation Model Pretraining** (ICML 2026 FMSD Workshop)\n  *Salih Bora Öztürk, Alexander Pfefferle, Frank Hutter*\n  [OpenReview](https://openreview.net/forum?id=QT1ySCPeW3)\n  \u003e Investigates how to dramatically accelerate the pretraining of tabular foundation models.\n\n- **Optimizing Pre-Training of Tabular Foundation Models by Shaping Geometry** (ICML 2026 FMSD Workshop)\n  *Humzah Merchant, Sriniketh Vangaru, Randall Balestriero*\n  [OpenReview](https://openreview.net/forum?id=IYnHchzvYB)\n  \u003e Improves tabular foundation model pretraining by shaping the geometry of the learned representation space.\n\n- **RAD-TFM: Robust and Domain-Adapted Tabular Foundation Models** (ICML 2026 FMSD Workshop)\n  *Matthew Peroni, Franck Le, Vadim Sheinin*\n  [OpenReview](https://openreview.net/forum?id=5BkHclEOW0)\n  \u003e Proposes robustness and domain-adaptation techniques for tabular foundation models.\n\n- **Mutual Information-Guided Corruption for Improved Self-Supervised Representation Learning in Tabular Data** (ICML 2026 FMSD Workshop)\n  *Michael Lawson, Emerald Sy, Kehui Zhang, Raymond H. Chan, Kannie W. Y. Chan, Rosa H. M. Chan*\n  [OpenReview](https://openreview.net/forum?id=T8qbmiE0yZ)\n  \u003e Uses mutual information to guide the corruption process in self-supervised tabular representation learning.\n\n- **Enhancing Tabular Learners with Context-Aware Semantic Embeddings** (ICML 2026 FMSD Workshop)\n  *Günther Schindler, Maximilian Schambach, Johannes Höhne*\n  [OpenReview](https://openreview.net/forum?id=QArxQg4U71)\n  \u003e Augments tabular learners with context-aware semantic embeddings of columns and values.\n\n- **Towards Pretraining Text Encoders for TabPFN** (ICML 2026 FMSD Workshop)\n  *Mustafa Tajjar, Alexander Pfefferle, Lennart Purucker, Frank Hutter*\n  [OpenReview](https://openreview.net/forum?id=dA8IZj8R46)\n  \u003e Explores pretraining dedicated text encoders to handle textual columns within TabPFN.\n\n- **HGR-TabE: Universal Tabular Embeddings via Maximal Correlation Alignment** (ICML 2026 FMSD Workshop)\n  *Niharika S. D'Souza, Liane Vogel, Kavitha Srinivas, Sola Shirai, Oktie Hassanzadeh, Horst Samulowitz*\n  [OpenReview](https://openreview.net/forum?id=FRj6pclhXE)\n  \u003e Learns universal tabular embeddings using maximal correlation (HGR) alignment.\n\n- **Learned Sequence Representations over Raw Credit Events for Credit-Abuse Scoring** (ICML 2026 FMSD Workshop)\n  *Tianming Zhou, Jiarui Xu, Nitesh Kumar, Alexander Statnikov*\n  [OpenReview](https://openreview.net/forum?id=8Zxm19jTVM)\n  \u003e Learns representations directly from raw credit-event sequences to improve credit-abuse scoring, instead of relying on hand-engineered tabular features.\n\n### Theory \u0026 Analysis\n\n- **Statistical Foundations of Prior-Data Fitted Networks** (ICML 2023)\n  *Thomas Nagler*\n  [Paper](https://arxiv.org/abs/2305.11097)\n  \u003e Provides a theoretical framework for PFNs, offering a frequentist interpretation and analyzing their convergence properties.\n\n- **What exactly has TabPFN learned to do?** (ICLR 2024 Blogposts Track / arXiv 2025)\n  *Calvin McCarter*\n  [Paper](https://arxiv.org/abs/2502.08978) | [Code](https://github.com/calvinmccarter/tabpfn-eval)\n  \u003e An empirical analysis treating TabPFN as a black-box function approximator to understand its learned inductive biases. Explores behavior on simple 1D/2D settings and out-of-distribution tasks (gene expression, MNIST). Includes 2025 re-analysis on TabPFN-v2, showing it can approximately learn the parity function with impressive sample efficiency.\n\n- **Towards Fair In-Context Learning with Tabular Foundation Models** (arXiv 2025)\n  *Patrik Kenfack, Samira Ebrahimi Kahou, Ulrich Aïvodji*\n  [Paper](https://arxiv.org/abs/2505.09503) | [Code](https://github.com/patrikken/Fair-TabICL)\n  \u003e The first investigation of fairness in tabular in-context learning, evaluating TabPFNv2, TabICL, and TabDPT. Finds that an uncertainty-based sample selection strategy consistently improves group fairness metrics (demographic parity, equalized odds) with minimal impact on accuracy.\n\n- **Light-Weight Benchmarks Reveal the Hidden Hardware Cost of Zero-Shot Tabular Foundation Models** (arXiv 2025)\n  *Ishaan Gangwani, Aayam Bansal*\n  [Paper](https://arxiv.org/abs/2512.00888)\n  \u003e A reproducible benchmark pairing test accuracy with wall-clock latency, peak CPU RAM, and peak GPU VRAM. Shows that zero-shot TFMs incur up to 10,000x latency penalties vs. tree ensembles, suggesting their main value lies in rapid prototyping on small tables rather than production inference at scale.\n\n- **Do Tabular Foundation Models Learn Rules or Memorize Exemplars?** (ICML 2026 FMSD Workshop)\n  *Amir Rezaei Balef, Mykhailo Koshil, Behzad Nourani-Koliji, Katharina Eggensperger*\n  [OpenReview](https://openreview.net/forum?id=9nCMtYGxQt)\n  \u003e Probes whether tabular foundation models generalize via rule learning or rely on memorizing training exemplars.\n\n- **Probing Memorization of Tabular In-Context Learning** (ICML 2026 FMSD Workshop)\n  *Francesco Capano, Jonas Böhler*\n  [OpenReview](https://openreview.net/forum?id=7DZ3u0SD4b)\n  \u003e Investigates the extent to which tabular in-context learners memorize their context examples.\n\n- **Tabular Foundation Models Are Effectively Shallow** (ICML 2026 FMSD Workshop)\n  *Irene Cannistraci, Julia E. Vogt*\n  [OpenReview](https://openreview.net/forum?id=kCnZUf1VYC)\n  \u003e Argues that tabular foundation models behave as effectively shallow function approximators.\n\n- **Where Computation Lives Inside TabPFN: Causal Localisation of Attention Head Function** (ICML 2026 FMSD Workshop)\n  *Atharva Gupta, Dhruv Kumar, Murari Mandal, Saurabh Deshpande*\n  [OpenReview](https://openreview.net/forum?id=LXSawSSeA9)\n  \u003e Uses causal localization to identify which attention heads carry out specific computations inside TabPFN.\n\n- **Statistically Indistinguishable, Operationally Distinct: A Formal Barrier for Tabular Foundation Models** (ICML 2026 FMSD Workshop)\n  *Tassilo Klein, Johannes Hoffart*\n  [OpenReview](https://openreview.net/forum?id=TUYc2XUdwz)\n  \u003e Establishes a formal barrier showing tabular foundation models can be statistically indistinguishable yet operationally distinct.\n\n- **On the Uncertainty in Prior-Data Fitted Network Pretraining** (ICML 2026 FMSD Workshop)\n  *Manuel Hülskamp, Julius Kobialka, Emanuel Sommer, David Rügamer*\n  [OpenReview](https://openreview.net/forum?id=5Shv4Ar4N9)\n  \u003e Analyzes sources of uncertainty arising during the pretraining of prior-data fitted networks.\n\n- **What You Pretrain On Matters: Synthetic Task Distributions Determine Tabular Foundation Model Quality** (ICML 2026 FMSD Workshop)\n  *Mohamed Bouadi, Nassim Bouarour, Shivam Dubey, Varun Kulkarni, Aditya Tanna, Vinay Sankarapu*\n  [OpenReview](https://openreview.net/forum?id=QfXHxB9VSS)\n  \u003e Shows that the choice of synthetic task distribution used in pretraining strongly determines tabular foundation model quality.\n\n- **When Data Is Scarce: The Strength of the Prior in Tabular Foundation Models** (ICML 2026 FMSD Workshop)\n  *Florian D. Leeuwen, Sara van Erp*\n  [OpenReview](https://openreview.net/forum?id=dznQA3JHfI)\n  \u003e Examines how strongly the learned prior drives tabular foundation model predictions in low-data regimes.\n\n- **Towards Evaluating Data Priors for Tabular Foundation Models** (ICML 2026 FMSD Workshop)\n  *Zeynep Türkmen, Kürşat Kaya, Alexander Pfefferle, Frank Hutter*\n  [OpenReview](https://openreview.net/forum?id=GUDjbVGFc1)\n  \u003e Works toward principled evaluation of the data priors used to train tabular foundation models.\n\n- **Inspectable Tabular Foundation Models via In-Context Kernel Learning** (ICML 2026 FMSD Workshop)\n  *Ratmir Miftachov, Bruno Charron, Simon Valentin*\n  [OpenReview](https://openreview.net/forum?id=Q1P83jtxXY)\n  \u003e Makes tabular foundation models more inspectable by framing in-context learning as kernel learning.\n\n- **Objective and data-driven Bayesian inference using TabPFN models** (ICML 2026 FMSD Workshop)\n  *Elias Chaibub Neto*\n  [OpenReview](https://openreview.net/forum?id=YRcXyqoemK)\n  \u003e Uses TabPFN models to perform objective, data-driven Bayesian inference.\n\n- **Lost in Aggregation: How Benchmarks Overlook Irreplaceable Model Strengths** (ICML 2026 FMSD Workshop)\n  *Andrej Tschalzev, Stefan Lüdtke, Heiner Stuckenschmidt, Christian Bartelt*\n  [OpenReview](https://openreview.net/forum?id=5B1lb8jrgo)\n  \u003e Argues that aggregate benchmark scores can obscure model-specific strengths that matter in practice.\n\n- **Revisiting Metafeatures to Explain Model Differences on Tabular Data** (ICML 2026 FMSD Workshop)\n  *Markus Herre, Andrej Tschalzev, Sascha Marton, Christian Bartelt*\n  [OpenReview](https://openreview.net/forum?id=FJSkVoD4k3)\n  \u003e Revisits dataset metafeatures to explain when and why models differ on tabular data.\n\n- **Training Fair Tabular Foundation Models** (ICML 2026 FMSD Workshop, Spotlight)\n  *Patrik Kenfack, Jesse C. Cresswell, Anthony L. Caterini, Samira Ebrahimi Kahou, Ulrich Aïvodji*\n  [OpenReview](https://openreview.net/forum?id=ajIvCEbadL)\n  \u003e Proposes methods for training tabular foundation models that satisfy group fairness criteria.\n\n- **FairOpt-PFN: Amortized Counterfactual Fairness with Optimal Fair Targets** (ICML 2026 FMSD Workshop)\n  *Enes Hasani, Jake Robertson, Frank Hutter*\n  [OpenReview](https://openreview.net/forum?id=M8o7jbKX9P)\n  \u003e A PFN approach to amortized counterfactual fairness that learns optimal fair targets.\n\n- **Auditing and Fixing Economic Validity in Tabular Foundation Models for Discrete Choice** (ICML 2026 FMSD Workshop)\n  *Yingshuo Wang, Xian Sun, Yanhang Li, Zhichao Fan, Zexin Zhuang*\n  [OpenReview](https://openreview.net/forum?id=Tda0qprAXQ)\n  \u003e Audits and corrects economic validity issues when tabular foundation models are used for discrete-choice modeling.\n\n- **Dataset Inference for Data Provenance and Privacy Auditing in Tabular Foundation Models** (ICML 2026 FMSD Workshop, Spotlight)\n  *Dariush Wahdany, Jesse C. Cresswell, Naiqing Guan, Atiyeh Ashari Ghomi, Franziska Boenisch, Adam Dziedzic*\n  [OpenReview](https://openreview.net/forum?id=u2uOPq1u6I)\n  \u003e Uses dataset inference to audit data provenance and privacy in tabular foundation models.\n\n- **TabPATE: Differentially Private Tabular In-Context Learning Without Public Data** (ICML 2026 FMSD Workshop)\n  *Dariush Wahdany, Matthew Jagielski, Jesse C. Cresswell, Adam Dziedzic, Franziska Boenisch*\n  [OpenReview](https://openreview.net/forum?id=Q5iGt2Z0Ql)\n  \u003e Brings differentially private in-context learning to tabular data without requiring public data.\n\n## Benchmarks \u0026 Evaluation\n\nBased on [van Breugel \u0026 van der Schaar (2024)](https://arxiv.org/abs/2405.01147), LTM benchmarks should evaluate models across multiple dimensions:\n\n### Tasks\n\n| Task | Description | Metrics |\n|------|-------------|---------|\n| **Supervised Learning** | Predictive performance using LTM embeddings or direct generation | Accuracy, AUC, RMSE |\n| **Synthetic Data Generation** | Conditional and unconditional generation quality | Train-on-synthetic-test-on-real, fidelity, diversity, ε-identifiability |\n| **Imputation** | Single imputation (E[X_unobs\\|X_obs]) or multiple imputation (sampling from p(X_unobs\\|X_obs)) | MAE, coverage, calibration |\n| **LTMs for Science** | Dimensionality reduction, data cleaning, clustering, cross-dataset retrieval | Task-specific metrics |\n\n### Experimental Settings\n\n| Setting | Description | Key Considerations |\n|---------|-------------|-------------------|\n| **Few-shot** | Performance on new datasets with few samples | With/without fine-tuning; robustness |\n| **Zero-shot** | No samples from target dataset | Requires true generalization |\n| **In-distribution** | Hold-out test from training data | Quantify generalization gaps |\n\n### Related Benchmarks\n\n- **[TabArena](https://tabarena.ai)** - A continuously maintained \"living\" benchmark for tabular ML (NeurIPS 2025, [paper](https://arxiv.org/abs/2506.16791)). Curates 51 real-world datasets and evaluates tree-based models, neural networks, and tabular foundation models with a public Elo leaderboard. The de facto standard for ranking modern TFMs.\n- **TabZilla** - Comprehensive tabular data benchmark\n- **OpenML** - Large collection of tabular datasets\n- **Kaggle Competitions** - Real-world tabular challenges\n\n#### From the ICML 2026 FMSD Workshop\n\n- **[MulTaBench: Benchmarking Multimodal Tabular Learning with Text and Image](https://openreview.net/forum?id=r19rlhngOD)** - A benchmark for multimodal tabular learning that combines text and image features. *Alan Arazi, Eilam Shapira, Shoham Grunblat, Mor Ventura, Elad Hoffer, Gioia Blayer, David Holzmüller, Lennart Purucker, Gaël Varoquaux, Frank Hutter, Roi Reichart.*\n- **[Are Tabular Foundation Model Rankings Reliable? A Generalizability Theory Analysis of RelBench and DBInfer](https://openreview.net/forum?id=7jbzkGYag6)** - Uses generalizability theory to assess the reliability of TFM rankings on RelBench and DBInfer. *Dinesh Katupputhur Ramprasath, Tom Palczewski, Joe Meyer, Roshan Reddy Upendra, Minghua Li.*\n- **[Realistic Evaluation of TabPFN v2.5 in Open Environments](https://openreview.net/forum?id=qway3qFkUL)** - Evaluates TabPFN v2.5 under realistic open-environment conditions. *Zi-Jian Cheng, Ziyi Jia, Lan-Zhe Guo.*\n- **[Ensembling Tabular Foundation Models: A Diversity Ceiling and a Calibration Trap](https://openreview.net/forum?id=FZaZoe67ne)** - Analyzes the limits of ensembling TFMs, highlighting a diversity ceiling and a calibration trap. *Aditya Tanna, Yash Jignesh Desai, Pratinav Seth, Mohamed Bouadi, Nassim Bouarour, Vinay Sankarapu.*\n- **[Exploring Differences Between Tabular Enterprise Data and Public Benchmarks](https://openreview.net/forum?id=PXSBtjo3Gd)** - Contrasts the characteristics of enterprise tabular data with public benchmarks. *Myung Jun Kim, Maximilian Schambach, Frank Essenberger, Andre Sres, Johannes Höhne.*\n- **[Benchmarking Attention for Tabular Foundation Models](https://openreview.net/forum?id=rwtcugrpDq)** - Benchmarks attention mechanisms used in tabular foundation models. *Maximilian Schambach, Clemens Biehl, Sam Thelin.*\n- **[Beyond Accuracy: Toward Trustworthy Tabular Foundation Models in Industrial Applications](https://openreview.net/forum?id=r3RAi8Kqzl)** - Looks beyond accuracy toward trustworthiness of TFMs in industrial settings. *Johannes Keler, Matthias Woehrle, Jan Achterhold, Mark Schillinger, Maria Lyssenko, Luiz Ricardo Douat.*\n- **[Benchmarking Tabular Foundation Models for Churn Prediction](https://openreview.net/forum?id=LtXucHLtiN)** - Benchmarks tabular foundation models on customer churn prediction. *Sobhan Seyedzadeh, Mostafa Karimi.*\n\n## Tutorials \u0026 Talks\n\n- **[Getting Started with PFNs](https://github.com/automl/PFNs/blob/main/Tutorial_1_Basics.ipynb)**: Official tutorial notebook from the PFNs repository.\n\n## Key References from Position Paper\n\nThe following works are referenced in [van Breugel \u0026 van der Schaar (2024)](https://arxiv.org/abs/2405.01147) as relevant to LTM development:\n\n- Tabular data surveys: [Borisov et al. (2022)](https://arxiv.org/abs/2110.01889), [Shwartz-Ziv \u0026 Armon (2022)](https://arxiv.org/abs/2106.03253)\n- Benchmarking: [Grinsztajn et al. (2022)](https://arxiv.org/abs/2207.08815), [Gorishniy et al. (2021)](https://arxiv.org/abs/2106.11959)\n- Synthetic data: [van Breugel \u0026 van der Schaar (2023)](https://arxiv.org/abs/2302.04311)\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request to add new papers, resources, or fix links.\n\n1. Fork the repository.\n2. Create a new branch.\n3. Add your resource.\n4. Submit a PR.\n\n---\n*Last updated: June 2026*\n","projects_url":"https://awesome.ecosyste.ms/api/v1/lists/jxucoder%2Fawesome-tabular-foundation-models/projects"}