Benchmarks

FEV Bench evaluates how well foundation models forecast unseen time series without any fine-tuning. It tests models on a diverse collection of real-world datasets spanning retail, energy, finance, traffic, and nature domains at multiple frequencies (minutely through yearly). Models are ranked by Skill Score, a composite metric that measures accuracy relative to a naive seasonal baseline across all datasets. Win Rate tracks how often a model beats the majority of competitors on individual datasets, capturing consistency alongside average performance.

GIFT-Eval is a comprehensive benchmark designed to test foundation models on both univariate and multivariate forecasting tasks. It spans 23 datasets across 7 domains including energy, transport, nature, economics, web traffic, healthcare, and sales. Models are evaluated on probabilistic accuracy using Weighted Quantile Loss (WQL) and ranked across all dataset-frequency combinations. The Average Rank metric provides a robust, outlier-resistant measure of overall model quality — a model that consistently places in the top 3 across all datasets will have a lower average rank than one that dominates a few datasets but performs poorly on others.