Navigate
Courses Tutorials Practice Pricing Tools Upgrade

One global model across many series

Today let's see how one model can forecast many series at once, and when that works better than fitting a separate model to each series.

Say you own the monthly turnover forecast for Australian retail. Turnover is the money shops take in during a month, and here it is in A$ million. The data has 8 states and 20 industries, and 148 of the state and industry combinations run through December 2018. Each combination is one series, so you have 148 series to forecast.

That leaves you a choice: fit one model to each series, or fit one model to the rows of all the series together.

The chart below shows one industry, Clothing retailing, in each of the 8 states from January 2016 to December 2018, which is 36 months each.

The states differ a lot in size. On the one chart, Northern Territory sits almost flat along the bottom next to New South Wales. Switch it to facet_wrap(~State) and look at months 12, 24 and 36 in each panel, which are the Decembers. Every state peaks in December, in all 3 years.

The 148 series and how the forecasts are scored

The data comes from the Australian Bureau of Statistics Retail Trade survey, and the tsibbledata package ships it as aus_retail. It holds 152 series, each one an industry in a state, with monthly turnover in A$ million. We keep the 148 series that run through December 2018.

Every model in this lesson is fitted to data up to December 2017 and then scored on the 12 months of 2018. Those 12 months are the holdout: months that no model was fitted to. Each 2018 month is forecast one step ahead, which means from the observed months before it. That gives 148 series times 12 months, or 1,776 test rows.

Most of the lesson keeps only the last 24 months before the cutoff, January 2016 to December 2017, which is a short history. Later we give the series more history and repeat the comparison.

The score is MAPE, the mean absolute percentage error. For each month we take the gap between the actual and the forecast, divide it by the actual and write it as a percent. Averaging over the 12 months gives one MAPE per series, and averaging over the 148 series gives the score. Because every error is a share of its own actual, a small series and a large series count the same.

The baseline is seasonal naive, which forecasts each month with the same month a year earlier. It fits nothing, so a fitted model is worth using only if it scores better.

The code below builds the table of series, sets the cutoff and the MAPE function, and scores seasonal naive.

RInteractive R
# Build the 148 retail series, define the split and MAPE, and score seasonal naive on 2018 library(tsibble) library(tsibbledata) library(dplyr) retail <- tsibbledata::aus_retail %>% as_tibble() %>% transmute(series = `Series ID`, state = State, industry = Industry, month = Month, turnover = Turnover) %>% group_by(series) %>% filter(max(month) == yearmonth("2018 Dec")) %>% ungroup() %>% arrange(series, month) cutoff <- yearmonth("2017 Dec") mape <- function(actual, forecast) { mean(abs(actual - forecast) / actual) * 100 } last_year <- retail %>% filter(month > cutoff - 12, month <= cutoff) %>% mutate(month = month + 12) %>% select(series, month, forecast = turnover) test_rows <- retail %>% filter(month > cutoff) %>% left_join(last_year, by = c("series", "month")) snaive_mape <- test_rows %>% group_by(series) %>% summarise(mape = mape(turnover, forecast)) cat("series: ", n_distinct(retail$series), "\n", sep = "") #> series: 148 cat("states: ", n_distinct(retail$state), "\n", sep = "") #> states: 8 cat("industries: ", n_distinct(retail$industry), "\n", sep = "") #> industries: 20 cat("test rows: ", nrow(test_rows), "\n", sep = "") #> test rows: 1776 cat("seasonal naive MAPE: ", round(mean(snaive_mape$mape), 2), "\n", sep = "") #> seasonal naive MAPE: 5.92

  

  





yearmonth() turns text into a month we can compare and add to. So cutoff - 12 is December 2016, and month + 12 moves each 2017 month forward a year, which lines it up with the 2018 month it forecasts.



The output confirms 148 series, 8 states, 20 industries and 1,776 test rows. Seasonal naive scores a MAPE of 5.92, so its forecasts are off by 5.92% of the actual turnover on average. That is the number every fitted model has to beat.