Navigate
Courses Tutorials Practice Pricing Tools Upgrade

Bagging and combining forecasts

Today let's look at the cheapest reliable trick in forecasting: instead of trusting one model's forecast, build several and combine them.

Suppose you are the demand planner at an airline. Every December you forecast passenger traffic for the coming calendar year, so the airline knows how many seats, crews and gates to plan for.

The series behind this lesson is AirPassengers, a dataset built into R: the monthly total of international airline passengers, in thousands, from January 1949 to December 1960, 144 months in all. We will train on the first 132 of those months, through December 1959, and hold back the 12 months of 1960 to check every forecast in this lesson against what actually happened.

The chart below plots the full series, the training months in one colour and the twelve held-out months of 1960 in the other.

Notice how the held-out stretch, the second colour, keeps climbing above anything the training months ever showed. That is exactly the stretch every forecast in this lesson has to reach.

Why one model's forecast depends on the data you happened to get

Let's start with the obvious thing to do: fit one model on the training months, and forecast with it.

ets() fits an exponential smoothing model to a time series automatically. It tries several combinations of a level, a trend and a season, and keeps the combination that fits best. Run it on the 132 training months, then forecast the 12 months of 1960.

RInteractive R
# Fit ETS on the training months and forecast the 12 held-out months of 1960 library(forecast) train <- window(AirPassengers, end = c(1959, 12)) test <- window(AirPassengers, start = c(1960, 1), end = c(1960, 12)) fit_ets <- ets(train) fit_ets$method #> [1] "ETS(M,Ad,M)" fc_ets <- forecast(fit_ets, h = 12) round(as.numeric(fc_ets$mean), 1) #> [1] 411.9 407.0 467.3 450.7 451.5 513.2 569.9 567.6 496.1 432.2 376.6 424.8 as.numeric(test) #> [1] 417 391 419 461 472 535 622 606 508 461 390 432

  

  





ets() settled on ETS(M,Ad,M): multiplicative errors, a damped additive trend, and a multiplicative season. Additive means the trend adds a fixed amount to the series each month rather than scaling with its level; damped means that added amount shrinks the further out the forecast runs, instead of carrying the same slope forever. Multiplicative means the swings scale up with the level of the series, which fits AirPassengers well since its summer peaks grow taller as the years go by.



Now score the forecast against what actually happened.




  

    
RInteractive R

    
# Score the forecast against the real 1960 totals
mae_ets  <- mean(abs(test - fc_ets$mean))
rmse_ets <- sqrt(mean((test - fc_ets$mean)^2))
sprintf("MAE = %.2f, RMSE = %.2f", mae_ets, rmse_ets)
#> [1] "MAE = 22.80, RMSE = 27.40"

    

      
      
    

    

  

  





MAE, the mean absolute error, averages how far off each month's forecast was, ignoring whether it ran too high or too low. RMSE, the root mean squared error, does something similar but squares each gap before averaging, so a few big misses weigh more heavily than many small ones.



fit_ets was fitted on exactly the 132 months that happened to occur between January 1949 and December 1959. Had the airline's bookings landed a little differently in any of those months, ets() would have produced a slightly different model, and a different forecast. The MAE of 22.80 above is not some fixed property of ETS on this series. It is what you get from the one history that actually happened, and a model's forecast is only ever as good as the one past it was shown.