About CHRONOS

CHRONOS is a language modeling framework for time series which is introduced by Amazon in early 2024. The probabilistic model treats the time series as a sequence and uses Google AI's T5 model (text to text transfer transformer) for training. The zero-shot forecasting model boasts superior performance over classical statistical forecasting models and specialized deep learning models after a comprehensive evaluation on more that 40 datasets. Zero-shot forecasting refers to the ability of models to generate to forecasts from unseen datasets. This is obtained by using synthetic data along with real data to train CHRONOS models as per the original paper [1].

How do CHRONOS models work ?

First of all, the time stamps (hourly, daily, monthly etc..) are not taken into account and the entire values are considered as tokens. The time series values are tokenized by scaling (mean scaling) and quantizing (binning/grouping values) into a fixed vocabulary, followed by training a pretrained language model (T5).

For deeper understanding, refer to the image below and go through the paper [1].

Forecasting using CHRONOS

To work with CHRONOS, California daily female birth dataset(1959) with 365 samples is used. To access the data, visit below given link.(https://raw.githubusercontent.com/jbrownlee/Datasets/master/daily-total-female-births.csv )

1. To begin with, install prerequisites from github.

!pip install git+https://github.com/amazon-science/chronos-forecasting.git

2. Import necessary libraries including Chronos Pipeline.

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import torch  # PyTorch to convert values ino torch tensors.
from chronos import ChronosPipeline

3. Choose Model and initialize Pipeline.

There are 5 models available with CHRONOS namely tiny, mini, small, base and large with increasing size and number of parameters. CHRONOS 'tiny' model is chosen here since dataset is small and for faster response. device_map can be chosen appropriately as 'cpu' , 'auto', 'cuda' etc.. The pipeline is initialized as follows.

'''initialize CHRONOS pipeline'''
pipeline = ChronosPipeline.from_pretrained("amazon/chronos-t5-tiny",
                                            device_map = "auto",
                                            torch_dtype = torch.bfloat16)

4. Import and split the data

series = pd.read_csv("/content/femalebirthCali.csv") # import data
train = series.head(320) # train data
test = series.tail(45) # test data

5. Define variable and forecasting steps/horizon

Forecasting 'Births' in the train dataframe for 45 steps (horizon).

to_predict = torch.tensor(train['Births'])# variable for training and predict
horizon = 45 # forecasting steps

6. Forecast using CHRONOS

Forecast for a step of 45 using CHRONOS pipeline's predict method.

forecast = pipeline.predict(to_predict, horizon)

7. Visualize forecast

Since CHRONOS does probabilistic forecast, the predicted values will be in the form of a dense tensor. For better understanding and visualization, median (50th percentile), upper bound (90th percentile), lower bound (10th percentile) are extracted from the forecasted vaues as below and plotted.

up, median, low = np.quantile(forecast[0].numpy(),[0.1,0.5,0.9], axis = 0)

'''plot the forecat'''
plt.subplots(figsize=(10,3))
plt.plot(train.Births, label = 'train data')
plt.plot(test.Births, label = 'test data')
plt.plot(np.arange(320,365),median, label = 'median forecast')
plt.legend()

8. Plot forecast with bounds

plt.subplots(figsize=(7,3))
plt.plot(median, label = 'forecast')
plt.plot(list(test['Births']), label = 'train data')
plt.fill_between(range(0,45), low, up, alpha=0.3, label="Bounds")
plt.legend()

Conclusion

The forecast looks promising with the CHRONOS 'tiny' model. This page just aims to explain fundamental workings and demonstrate how the model can be accessed with ease. Performance evaluation can be done by comparing with other models by choosing metrics such as MAE , MSE etc.. It is strongly recommended to go through the base paper [1] for better understanding of the model architecture.

References

1. https://arxiv.org/pdf/2403.07815
2. https://machinelearningmastery.com/time-series-datasets-for-machine-learning/
3. https://www.youtube.com/watch?v=WxazoCVkBhg

✉️ mkk.rakesh@gmail.com