Benchmarks

TSFM benchmark archive

Published benchmark results and upstream leaderboards for time series foundation models. Each benchmark answers a different forecasting question — pick the one that matches your workload.

Which benchmark should you use?

FEV BenchMASE Skill Score

How well does a model generalize to unseen real-world forecasting tasks?

Pretrained models run zero-shot under the benchmark's leakage policy. Task-specific models (classical baselines, fine-tuned models) may use each task's training split. All models are ranked by MASE-based Skill Score relative to a seasonal naive baseline; Win Rate measures pairwise wins against all included models, with half credit for ties.

GIFT-EvalAverage MASE rank

Which models stay strong across heterogeneous datasets and probabilistic settings?

Models are scored on grouped benchmark slices and ranked by average rank, with Weighted Quantile Loss providing a secondary read on probabilistic accuracy.

BOOMCRPS

How do models behave on observability telemetry instead of academic datasets?

Models are ranked on real observability time series using CRPS for probabilistic quality and MASE for point accuracy, aggregated across production telemetry collected from real infrastructure workloads.

ImpermanentMean MASE / CRPS rank

Does model performance hold up as real time passes and the data distribution shifts?

The benchmark uses a prequential protocol: models forecast before outcomes exist, scores accumulate over time, and rankings reflect sustained performance under live temporal change rather than one-off wins on a frozen split.

ARFBenchAccuracy

Which multimodal models can reason about anomalies, timing, magnitude, and cross-series structure in production telemetry?

750 multiple-choice QA pairs built from 142 real Datadog observability time series and 63 production incidents, spanning eight question categories grouped into three difficulty tiers. Models receive a templated question, a metric description, and the plotted time series, and are scored on Accuracy and macro-F1 against expert-reviewed answers.

TIMESee official metric selection

How do models behave across realistic tasks and different temporal patterns?

The benchmark evaluates 12 foundation models across 98 tasks, combining dataset quality checks with forecasting-relevant pattern analysis. Consult the official leaderboard for current submissions and selected metrics.

QuitoBenchOverall mean rank

Does a model remain strong across easy and difficult forecasting regimes?

The study compares statistical, deep learning and foundation-model approaches across eight regimes and multiple context lengths. Results are published study comparisons, rather than a continuously updated competition ranking.

Frequently asked questions

What are TSFM benchmarks?
TSFM benchmarks are benchmark-specific pages on TSFM.ai that track how time series foundation models perform across zero-shot, probabilistic, observability, and temporal generalization workloads.
Which TSFM benchmark should I use first?
Start with FEV Bench for general zero-shot model selection, then cross-check GIFT-Eval for probabilistic robustness, BOOM for observability-style data, and Impermanent for temporal drift and live evaluation.
Do the TSFM benchmark pages include directory model coverage?
Yes. Each benchmark page highlights which leaderboard entries map to models in the TSFM.ai directory and links out to the matching model surface when available.
How often are benchmark pages refreshed?
Mirrored benchmark results are cached for up to 12 hours and revalidated on visits. TIME links to its official interactive leaderboard; QuitoBench links to published study results.

Related benchmark reading