MOMENT: CMU's Model for Time Series Understanding

Most time series foundation models specialize in a single task, typically forecasting. MOMENT, developed by researchers at Carnegie Mellon University, takes a fundamentally different approach. Released in early 2024 alongside the paper "MOMENT: A Family of Open Time-Series Foundation Models", it demonstrates that a single pretrained backbone can handle forecasting, classification, anomaly detection, and imputation through lightweight task-specific heads.

Architecture: Masked Transformers for Time Series

MOMENT's architecture draws inspiration from BERT rather than GPT. Where autoregressive models like Chronos generate future values token by token, MOMENT uses a masked transformer encoder that processes the entire input sequence bidirectionally. This design choice is deliberate: many time series tasks (classification, anomaly detection, imputation) benefit from seeing context in both directions, not just left-to-right.

The input pipeline begins with a patching strategy borrowed from Vision Transformers (ViT). Raw time series values are segmented into fixed-size non-overlapping patches, each containing a contiguous window of observations. For a series of length 512 with a patch size of 8, the model processes 64 patch tokens rather than 512 individual timesteps. This dramatically reduces the quadratic attention cost and allows the model to capture longer-range dependencies within a manageable compute budget.

Each patch is projected into the model's embedding space through a lightweight linear layer, with positional encodings added to preserve temporal ordering. The embedded patches pass through a stack of standard transformer encoder blocks with multi-head self-attention and feed-forward layers.

The MOMENT family includes multiple sizes. MOMENT-Small uses a transformer with roughly 40 million parameters, while MOMENT-Large scales to around 350 million. All variants share the same patching and embedding strategy, differing primarily in depth and hidden dimension.

The Time Series Pile

A foundation model is only as good as its pretraining data. The CMU team assembled the Time Series Pile, a curated collection of publicly available time series datasets spanning diverse domains: electricity consumption, weather readings, traffic flows, economic indicators, and medical signals. The collection was designed for diversity rather than sheer volume, ensuring the model encounters varied temporal patterns, frequencies, and statistical properties during pretraining.

Pretraining uses a masked reconstruction objective. Random patches are masked out, and the model learns to reconstruct them from surrounding context. This self-supervised approach requires no labels, allowing the model to learn general-purpose temporal representations from raw data.

Four Tasks, One Backbone

MOMENT's defining feature is multi-task capability through task-specific heads attached to the shared pretrained encoder.

Forecasting works through a masked autoregressive approach. The model masks future patches and reconstructs them conditioned on observed history. A linear projection head maps the encoder's output representations back to raw time series values for the forecast horizon. Because the encoder is bidirectional, forecasting requires masking future positions during inference to prevent information leakage.

Classification follows the CLS token paradigm from NLP. A special classification token is prepended to the patch sequence, and its final-layer representation is passed through a classification head (typically a small MLP) to produce class probabilities. The CLS token attends to all patches, aggregating the full series into a single representation suitable for downstream classification tasks like activity recognition or fault diagnosis.

Anomaly detection leverages reconstruction error. The model reconstructs input patches, and anomalies are identified where reconstruction error is unusually high. The intuition is straightforward: if the model has learned normal temporal patterns during pretraining, abnormal segments will be poorly reconstructed. A per-patch error score is computed, and a threshold (tunable per application) flags anomalous regions.

Imputation uses masked reconstruction directly. Missing segments are treated as masked patches, and the model fills them in using bidirectional context from observed values on both sides. This is the task most naturally aligned with MOMENT's pretraining objective, and it tends to be the strongest out-of-the-box capability.

Benchmark Performance

Across the four tasks, MOMENT demonstrates competitive or superior performance compared to specialized baselines. On standard forecasting benchmarks (ETTh1, ETTh2, Weather, Electricity), MOMENT matches or comes close to models like PatchTST that were explicitly designed for forecasting. On the UCR time series classification archive, MOMENT's representations outperform many supervised baselines when used with simple linear probes. Anomaly detection results on datasets like SMD and MSL show reconstruction-based scoring is effective without any task-specific fine-tuning.

The key result is not dominance on any single task but consistent competence across all four. A single MOMENT checkpoint can be deployed as a general-purpose time series engine, whereas using specialized models requires maintaining four separate systems.

Multi-Task Advantages

Training a single model across multiple objectives creates beneficial inductive biases. The masked reconstruction pretraining teaches the model both local pattern recognition (useful for anomaly detection and imputation) and global sequence understanding (useful for classification and forecasting). Features learned for one task transfer to others: the ability to detect anomalous patterns, for example, implies an understanding of what normal patterns look like, which directly benefits forecasting.

This stands in contrast to single-task TSFMs like Chronos, which are optimized entirely for next-token prediction. Chronos excels at forecasting but offers no native pathway to classification or anomaly detection without significant additional work. For a deeper look at how different models compare across tasks, see our benchmarking challenges analysis.

Open Source and Practical Use

The MOMENT weights and code are publicly available, released under a permissive license. The model can be loaded through standard PyTorch or Hugging Face interfaces. Task-specific heads can be swapped or fine-tuned with minimal labeled data, making MOMENT a practical starting point for teams that need multiple time series capabilities without the overhead of managing separate models.

On TSFM.ai, MOMENT is available through the standard forecasting and anomaly detection endpoints, with routing logic that directs multi-task requests to MOMENT when its unified architecture offers the best accuracy-to-latency tradeoff. Explore the full list of supported models on our model catalog, or compare MOMENT against alternatives in our multivariate forecasting overview. For teams evaluating which TSFM to adopt, MOMENT is a particularly strong choice when the use case spans beyond pure forecasting.