Machine learning competitions increasingly come down to ensemble engineering. A slight improvement in performance can translate to millions of dollars in value. The best model isn't a single architecture—it's a carefully constructed combination of approaches that learn in different ways from different data.

Gradient boosted models like CatBoost have dominated tabular and time series prediction, but pre-trained models such as TabPFN and Chronos are now matching or exceeding them on certain benchmarks. Rather than choosing between these fundamentally different approaches, combining them eliminates most weaknesses while retaining most strengths. The intuition is straightforward: just as combining base estimators in gradient boosting outperforms any single prediction, combining whole models outperforms any single model.

The author outlines a three-layer stacking approach. Layer 1 trains base models using bootstrap aggregation for tabular data or rolling window cross-validation for time series. Layer 2 adds Layer 1 predictions as features, testing strategies like weighted averaging or training a meta-model on base outputs. Layer 3 creates the final model for evaluation. The approach emphasizes trying all strategies since they're computationally efficient.