Navigate
Courses Tutorials Practice Pricing Tools Upgrade

How DeepAR, N-BEATS and N-HiTS forecast

Today let's see what DeepAR, N-BEATS and N-HiTS each change in a neural network to forecast a time series, and how to check whether that change actually helps.

Suppose you plan stock for a takeaway food chain in Australia. You need to forecast the monthly takeaway food turnover, which is the total sales of takeaway food outlets, for each of 8 states and territories. The forecast has to reach 12 months ahead, and the unit is million Australian dollars.

The numbers come from the Australian Bureau of Statistics retail trade survey. There is one monthly series per state, from April 1982 to December 2018. We will train on the data up to December 2017 and keep the 12 months of 2018 aside. That way, every forecast can be checked against what really happened.

The chart below shows one of the eight series, New South Wales (NSW), for the three years 2015 to 2017. Turnover climbs from one year to the next, and inside each year the same shape repeats: February is the lowest month and December the highest.

All 8 series have this kind of shape, at very different levels. We will forecast them with one neural network, and each of the three models changes one part of that network:

  • DeepAR forecasts a probability distribution for each month, instead of a single number.
  • N-BEATS builds the forecast from a chain of blocks, where each block explains part of the series and passes the rest on.
  • N-HiTS is N-BEATS with a smaller input and a smaller output in each block, so that it stays cheap on long horizons.

Switch between the line and the bars above and look at how the yearly pattern sits on top of the upward climb.

Turning eight series into training windows

A neural network learns from examples. Here, one example is a pair: 24 past months as the input, and the 12 months right after them as the target. We cut these pairs out of the series and call each one a training window. The 24 months the network looks back on are the lookback, and the 12 months it forecasts are the horizon.

The first code block loads the takeaway food series, splits off the 12 months of 2018, and prints each state's average monthly turnover in the training period.

RInteractive R
# Load the takeaway food series for the 8 states and hold out the 12 months of 2018 library(tsibbledata) retail <- as.data.frame(aus_retail) takeaway <- retail[retail$Industry == "Takeaway food services", ] state_code <- c( "Australian Capital Territory" = "ACT", "New South Wales" = "NSW", "Northern Territory" = "NT", "Queensland" = "QLD", "South Australia" = "SA", "Tasmania" = "TAS", "Victoria" = "VIC", "Western Australia" = "WA" ) takeaway$state <- unname(state_code[takeaway$State]) states <- unname(state_code) series <- list() train <- list() test <- list() for (s in states) { rows <- takeaway[takeaway$state == s, ] rows <- rows[order(rows$Month), ] series[[s]] <- rows$Turnover train[[s]] <- head(rows$Turnover, -12) # everything up to December 2017 test[[s]] <- tail(rows$Turnover, 12) # the 12 months of 2018 } # Number of training months, then average monthly turnover in million Australian dollars sapply(train, length) #> ACT NSW NT QLD SA TAS VIC WA #> 429 429 357 429 429 429 429 429 round(sapply(train, mean), 1) #> ACT NSW NT QLD SA TAS VIC WA #> 11.6 239.2 10.6 159.7 48.5 14.6 164.3 73.1 round(mean(train$NSW) / mean(train$NT), 1) #> [1] 22.6

  

  





NSW averages 239.2 million a month and the Northern Territory (NT) 10.6 million, so NSW is 22.6 times the size of NT. NT also has fewer months, because its series starts in April 1988.



One network has to learn from all eight series. But the squared error would be dominated by the big states, because a 5% miss on NSW is a much larger number than a 5% miss on NT. So we divide each window by the mean of its 24 input months, and subtract 1. A scaled value of 0 then means the average of that window, and -0.10 means 10% below it.



The network now sees only the shape of a window, whichever state it comes from. To get a forecast back in million dollars, we add 1 and multiply by the same window mean.



We slide the window one month at a time over the last 180 training months, January 2003 to December 2017. Every state has at least that much history, and 15 years keeps the training fast. Each state gives 145 windows: 180 months, minus the 24 of the input and the 12 of the target, plus 1.



The next block builds the 1,160 windows (8 states times 145) and prints the size of the input matrix X and the target matrix Y.




  

    
RInteractive R

    
# Build 1,160 training windows: 24 scaled months in, the next 12 scaled months out
n_train_months <- 180
lookback <- 24
horizon <- 12
n_windows <- n_train_months - lookback - horizon + 1

X_rows <- list()
Y_rows <- list()
window_state <- character(0)
for (s in states) {
  last_180 <- tail(train[[s]], n_train_months)
  for (i in seq_len(n_windows)) {
    input <- last_180[i:(i + lookback - 1)]
    target <- last_180[(i + lookback):(i + lookback + horizon - 1)]
    window_mean <- mean(input)
    X_rows[[length(X_rows) + 1]] <- input / window_mean - 1
    Y_rows[[length(Y_rows) + 1]] <- target / window_mean - 1
    window_state <- c(window_state, s)
  }
}
X <- do.call(rbind, X_rows)
Y <- do.call(rbind, Y_rows)

# Size of X and Y, then the first three scaled values of the first NSW window
dim(X)
#> [1] 1160   24
dim(Y)
#> [1] 1160   12
round(X[window_state == "NSW", ][1, 1:3], 3)
#> [1] -0.080 -0.260 -0.178

    

      
      
    

    

  

  





Each row of X is one window of 24 scaled months, and the same row of Y holds the 12 scaled months that followed it. The first NSW window starts in January 2003. That month is 8.0% below the average of its 24 months, and February 2003 is 26.0% below it.