Researchers at THUML lab of Tsinghua University released Timer-XL, a decoder‑only Transformer built specifically for time‑series forecasting. Unlike earlier variants that required separate models for different horizon lengths, Timer‑XL accepts arbitrary look‑back and prediction windows while handling exogenous covariates. It also captures multivariate dynamics and seasonality.

Timer‑XL’s standout feature is TimeAttention, an adaptation of self‑attention that respects temporal order and mitigates overfitting common in generic Transformers. Benchmarks on daily traffic data show stable performance up to 8,760 tokens—roughly a year of hourly observations—far surpassing models such as MOIRAI (4K tokens) and Tiny‑Time‑Mixers, which degrade after 1K tokens and maintains low computational overhead.

By consolidating variable input‑output lengths into a single model, Timer‑XL simplifies deployment pipelines and reduces the need for task‑specific fine‑tuning. Its decoder‑only design aligns with recent findings that such architectures outperform encoder‑only counterparts on pure forecasting tasks. The paper’s extensive experiments confirm that specializing a foundation model for prediction yields measurable gains over more generalist approaches and scalability across industries today.