Workbench / Research

How tabular foundation models predict from a table

Updated

How it works

From pre-training to a prediction.

This is how the tabular foundation models the Workbench runs today work, and the design LTFM keeps.

  1. 01 · Pre-training

    Learn shared patterns

    Train a transformer across synthetic datasets to learn patterns that transfer between tasks.

    What changesModel weights are learned.

  2. 02 · Inference

    Use history as context

    Supply observed states, transitions and related records. The pre-trained model uses that context to answer the question.

    What changesThe input changes; weights stay fixed.

  3. 03 · Output

    Report uncertainty

    Return a prediction with an uncertainty band and reasons. When evidence is too weak, pass the case to the existing process.

    What you receiveA score, its uncertainty and an explanation.

What LTFM reads

Entities, their state, their history and every transition.

A longitudinal table connects records through time. This synthetic supply chain shows how a change in one entity can affect related entities.

Example · a supply-to-customer chainNormalWatchStressedFlagged▸ Transition└ Relation
Proven on tables

A PFN, out of the box, beat four hours of expert tuning in 2.8 seconds.

Hollmann et al., Nature 637 (2025)

Published PFN result. This does not measure LTFM performance.

LTFM extends this to time and relations
Time-seriesTimesFM, Chronos-2
TabularTabPFN → SAP
RelationalKumoRFM → NVIDIA
LTFMentities + state + history + transitions
Building the foundation model · internal stages

Two stages to a new foundation model.

One model, built from the inside out. Each stage ships something real before the next begins.

LTFMone model, built from the inside out
Stage 1 · LTFM coreBuilding now

Longitudinal state

A model of each entity's state and how it moves between states. The internal core of LTFM, tested first on financial-services tables through the Workbench.

Stage 2 · Full LTFM · the goal

LTFM

The same core, generalised to every industry.

What we're testing

Three open questions, each with a preregistered design.

Written plans, with the controls and outcome rules fixed before any model runs. None has been run yet; results will be published here either way.

  1. 01 · Time

    Does a frozen temporal encoder beat an analyst's features?

    When an entity's history is too wide to fit a tabular model's input, something has to compress it. A frozen time-series encoder is compared with a bank of handcrafted summaries at the same width, across three frozen tabular foundation models.

    StatusPlan and code scaffolded. Not yet run.

  2. 02 · Explanation

    Can a foundation model teach a simple, auditable equation?

    A tabular foundation model acts as a teacher; a short logistic equation and a symbolic-regression student learn from its probabilities. The test is whether the student beats the same equation fit directly on labels, at equal complexity, on credit datasets.

    StatusPlan written with non-foundation-model controls. Not yet run.

  3. 03 · Relations

    How much of a database can automatic features carry into a flat table?

    A pipeline that connects to a relational database, discovers keys and timestamps, generates point-in-time features across tables and selects a budget of them for a tabular foundation model, measured against the published benchmarks for the same tasks.

    StatusPlan written. Starting with public relational benchmarks.

Why now

Tabular models are three years behind the LLM curve, and accelerating faster.

Model releases over time. The tabular timeline is shifted three years to sit on the same axis.

NOW FOR TABULAR0510152025Jan 2022Jan 2025Jan 2023Jan 2026ChatGPTLLaMAGPT-4Llama 2GeminiMixtralTabPFNv2KumoRFMTabPFN-2.5NEXUSKumoRFM-2TabFMTabPFN-3LimiX-2
Solid: LLMs 2021–24 (bold dates). Dashed: tabular foundation models 2024–26 (lighter dates). Each dot is a model release. Shaded: where tabular models are now.

Tabular foundation models: same curve, three years later.

The market is moving · ex: Revolut

PRAGMA FM

A BERT-style foundation model on longitudinal banking transactions.

FraudCredit riskLifetime valueUnderwriting
We're next on the curve

LTFM: longitudinal, calibrated, opinionated.

Training → inference

Every version makes the next one smarter.

Deployments show which patterns matter in real tasks. Those become better priors for the next round of pre-training.

synthetic worlds
pretraining
public longitudinal data
general representations
better LTFM
better priors
real-world task patterns
customer deployments
↑ repeat↓ deploy
Reading list

25

papers, read & connected before we wrote a line of LTFM.

CausalFMForecastFMTimeFNBayesNNLimiXPRAGMA

…and more. We open research intake every 8 weeks. Researchers working on tabular or longitudinal models: write to hello@knoww.ai.

Request your free month

What do you want to predict?

Five runs, 50,000 rows each, free for a month. Tell us the question and what you use today. We review every request by hand and email you when your workspace is ready. If you already know you want a pilot next to your current model, say so below.

Request your free month
We send a confirmation link here.
The question you'd like answered ahead of time.
It goes on the leaderboard next to the foundation models.

We review every request by hand and email you when your workspace is ready. Please don't include sensitive data.