Large language models (LLMs) dominate many AI tasks yet remain weak on one of the most common workloads: predictive analytics over tabular data. The emerging field of tabular foundation models seeks to understand this gap.

Researchers tested a frontier LLM in a pure inference regime—one generation pass over a prompt containing all training and test data, without tools, scaffolding, or fine‑tuning. Five failure hypotheses were examined: (a) handling noisy or non‑linearly separable data, (b) the linearised CSV format obscuring column structure, (c) tokenisation of numeric values, (d) the number of test points per query, and (e) input dimensionality.

Controlled experiments ruled out (a)–(d). Dimensionality proved decisive: sweeping random linear projections of 31 benchmark datasets showed LLM accuracy declines as dimensionality grows, whereas nine classical baselines stay flat or improve. A comparison with 252 configured classical models revealed that in two dimensions the LLM behaves like a local, distance‑based method—up to 91.6% grid agreement—yet in higher dimensions no classical model reproduces its predictions.

The study does not pinpoint an internal mechanism but demonstrates that LLM capability erodes with dimensionality in a way no noise‑corrupted classical learner mimics, explaining why LLMs still lose to long‑standing baselines on activity‑table tasks.