FEV Bench is a forecasting benchmark of 100 tasks from 96 real-world time-series datasets across 7 domains, including covariate-rich and multivariate settings. This leaderboard view shows MASE-based Skill Score for point-forecasting accuracy. Pretrained models are evaluated under the benchmark's zero-shot/leakage policy; task-specific models may train on each task's training split.

How often is the FEV Bench leaderboard updated?

TSFM caches upstream results for up to 12 hours and revalidates them when the page is visited.

What does MASE Skill Score measure?

Skill Score quantifies how much better a model performs compared to a seasonal naive baseline, using MASE (Mean Absolute Scaled Error) as the base metric. Higher values indicate a larger improvement over the baseline across all 100 benchmark tasks.

Should I use FEV Bench to pick a model for production?

FEV Bench is a strong starting point for general-purpose point-forecasting accuracy, but you should cross-check with domain-specific benchmarks like BOOM (for observability data) or GIFT-Eval (for probabilistic robustness) before committing to a production choice.

Upstream results·96 datasets·Upstream results cached for up to 12 hours; refreshed on visits

FEV Bench

FEV Bench evaluates time series foundation models across 100 forecasting tasks drawn from 96 real-world datasets spanning 7 domains, including 46 tasks with covariates. Pretrained models are evaluated under a zero-shot / leakage policy; task-specific models may train on each task's training split. This view orders all published results by MASE Skill Score; the official interactive leaderboard allows different model selections and sorts by pairwise Win Rate.

Page data loaded 10/28/2026 · Source revision 193e4fcd

What this benchmark answers

How well does a model generalize to unseen real-world forecasting tasks?

Methodology

Pretrained models run zero-shot under the benchmark's leakage policy. Task-specific models (classical baselines, fine-tuned models) may use each task's training split. All models are ranked by MASE-based Skill Score relative to a seasonal naive baseline; Win Rate measures pairwise wins against all included models, with half credit for ties.

TimeRSI-1.5 leads with MASE Skill Score 41.44, followed by TimeRSI (40.58) and timeh2o-v1 (39.95).

17 of 44 ranked entries in the model directory · Higher is better

Top 12 of 44 results. The full table is below.

Rankings

MASE Skill Score vs Win Rate

Directory model (filled)Not in directory (outline)

Full results

	Model			Inference (s/100)	Train overlap	Failures	Directory
1	TimeRSI-1.5	41.44	87.19	523.95	0%	0	—
2	TimeRSI	40.58	81.33	1007.64	0%	0	—
3	timeh2o-v1	39.95	85.86	2.82	0%	0	—
4	Timestral-209M	39.49	83.67	0.51	0%	0	—
5	Timestral-123M	39.19	82.21	0.32	0%	0	—
6	t0-gamma	37.99	78.84	4.38	0%	0	—
7	timesfm-3→	37.82	74.09	3.41	0%	0	View model
8	Timestral-28M	37.27	74.00	0.11	0%	0	—
9	ChakraTS	36.93	76.23	14.22	0%	0	—
10	Chronos-2→	35.50	65.35	0.84	0%	0	View model
11	Lingjiang2.0	35.38	68.14	8.89	0%	0	—
12	iAmTime	35.36	68.49	0.56	0%	0	—
13	t0-beta→	34.86	61.51	2.29	0%	0	View model
14	Timestral-5M	34.54	64.53	0.04	0%	0	—
15	TiRex-2→	33.74	60.42	0.27	0%	0	View model
16	timeh2o-v1-tiny	33.26	63.79	0.19	0%	0	—
17	Toto-2.0-2.5B→	32.54	63.19	5.58	0%	0	View model
18	Toto-2.0-1B→	32.39	63.02	2.48	0%	0	View model
19	Toto-2.0-313m→	32.10	60.98	1.06	0%	0	View model
20	TS-ICL	31.11	47.23	2.49	0%	0	—
21	Toto-2.0-22m→	30.61	53.30	0.25	0%	0	View model
22	TabPFN-TS-3	30.56	48.65	234.62	0%	0	—
23	TimesFM-2.5→	30.20	50.58	1.87	10%	0	View model
24	TiRex→	30.01	51.53	0.24	1%	0	View model
25	TabPFN-TS	29.83	43.22	109.39	0%	1	—
26	citras-fm	29.33	44.35	—	0%	0	—
27	Toto-1.0→	28.23	43.53	22.35	8%	0	View model
28	Toto-2.0-4m→	28.04	41.49	0.20	0%	0	View model
29	FlowState→	27.83	47.47	2.29	8%	0	View model
30	Moirai-2.0→	27.22	38.30	0.35	28%	0	View family
31	Chronos-Bolt→	26.52	36.98	0.26	0%	0	View family
32	Sundial-Base→	24.75	31.70	8.01	1%	0	View model
33	CatBoost	23.69	33.33	0.31	0%	0	—
34	LightGBM	21.69	31.60	0.27	0%	0	—
35	TFT	20.48	27.50	0.99	0%	0	—
36	PatchTST→	18.37	23.65	0.71	0%	0	View model
37	DeepAR	17.52	22.24	1.26	0%	3	—
38	Stat. Ensemble	16.44	29.09	146.94	0%	4	—
39	AutoARIMA	11.63	20.33	20.14	0%	4	—
40	AutoTheta	10.99	18.67	3.28	0%	0	—
41	AutoETS	2.26	21.41	3.48	0%	3	—
42	Seasonal Naive	0.000	11.12	0.48	0%	0	—
43	Naive	-16.67	10.30	0.47	0%	0	—
44	Drift	-18.14	9.58	0.45	0%	0	—

What FEV Bench measures

FEV Bench runs 100 forecasting tasks from 96 datasets across 7 domains, including 46 tasks with covariates. The benchmark separates pretrained foundation models — evaluated zero-shot under a leakage policy — from task-specific models that may train on each task's training split. Both tracks appear in the leaderboard; the training runtime column distinguishes them. Several pretrained models have declared partial training-corpus overlap with benchmark datasets; the benchmark imputes affected tasks rather than excluding models entirely.

MASE Skill Score vs Win Rate

TSFM orders all published model results by MASE Skill Score: how far a model beats the seasonal naive baseline, averaged across all 100 tasks. Win Rate captures pairwise wins against other included models, with half credit for ties. FEV Bench also publishes SQL (scaled quantile loss) for probabilistic accuracy and WQL/WAPE as secondary metrics. A model with a high MASE Skill Score but a modest Win Rate is strong on average but inconsistent; the reverse means it wins often but by narrow margins.

How to interpret it

—This view shows the full MASE leaderboard. FEV Bench also reports SQL (probabilistic) and WQL/WAPE metrics.
—High MASE Skill Score means the model beats a seasonal naive baseline by a wider margin across all 100 tasks.
—Several pretrained models have declared training-corpus overlap with benchmark datasets; affected tasks are imputed per the benchmark's leakage policy.
—Cross-check with domain-specific benchmarks before making a production choice.

Frequently asked questions

What is FEV Bench?: FEV Bench is a forecasting benchmark of 100 tasks from 96 real-world time-series datasets across 7 domains, including covariate-rich and multivariate settings. This leaderboard view shows MASE-based Skill Score for point-forecasting accuracy. Pretrained models are evaluated under the benchmark's zero-shot/leakage policy; task-specific models may train on each task's training split.
How often is the FEV Bench leaderboard updated?: TSFM caches upstream results for up to 12 hours and revalidates them when the page is visited.
What does MASE Skill Score measure?: Skill Score quantifies how much better a model performs compared to a seasonal naive baseline, using MASE (Mean Absolute Scaled Error) as the base metric. Higher values indicate a larger improvement over the baseline across all 100 benchmark tasks.
Should I use FEV Bench to pick a model for production?: FEV Bench is a strong starting point for general-purpose point-forecasting accuracy, but you should cross-check with domain-specific benchmarks like BOOM (for observability data) or GIFT-Eval (for probabilistic robustness) before committing to a production choice.

Title	Date
FEV Bench: The Zero-Shot Forecasting Benchmark Explained	2026-02-10	Read →
Zero-Shot Forecasting: Why It Matters	2024-04-12	Read →
The Challenges of Benchmarking TSFMs	2024-11-05	Read →