Navigate
Courses Tutorials Practice Pricing Tools Upgrade

Monitoring forecast accuracy over time

Today let's understand what to watch on a forecast once it is already running, not just how accurate it looked on the day you built it.

Fernbank Bakery is a small wholesale bakery, and its best-selling product is a sourdough loaf. Below are the last 130 days of case orders for that loaf.

For the first two months, orders climb gently, the ordinary drift of a bakery that is slowly growing. Then two things happen that are not ordinary drift: a short, sharp jump partway through the series, and later a jump that never settles back down. Neither one explains itself from the chart alone. This lesson works out how to watch a forecast's own errors closely enough to catch exactly that kind of change, and to tell the two apart.

Forecasting two horizons from the same average

A forecasting model can look fine on the day you launch it and still go wrong later, without anyone changing a line of code. That is exactly why its accuracy needs watching over time, not just measuring once before it ships.

Fernbank forecasts its sourdough orders with a trailing 7-day average: every day, it averages the last 7 days of actual orders and calls that the forecast. The bakery uses this one number for two different jobs. Tomorrow's bake decision needs a forecast 1 day ahead.

Next week's flour order needs a forecast 7 days ahead. How far ahead a forecast is aimed at is called its horizon, and Fernbank is really running the same average at two different horizons: 1 day and 7 days.

Press Run to build Fernbank's 130 days of orders and see both horizons side by side.

RInteractive R
# Build Fernbank's 130 days of orders and the trailing 7-day average that forecasts them n_days <- 130 seed <- 11 noise <- numeric(n_days) for (t in 1:n_days) { seed <- (seed * 48271) %% 2147483647 noise[t] <- (seed %% 13) - 6 } orders <- 200 + 0.4 * (1:n_days) + noise orders[61:62] <- orders[61:62] + 35 orders[91:n_days] <- orders[91:n_days] + 15 trail7 <- rep(NA, n_days) for (s in 7:n_days) { trail7[s] <- mean(orders[(s - 6):s]) } days <- 55:62 data.frame( day = days, horizon_1_forecast = round(trail7[days - 1], 2), horizon_7_forecast = round(trail7[days - 7], 2), actual_orders = round(orders[days], 2) ) #> day horizon_1_forecast horizon_7_forecast actual_orders #> 1 55 221.97 214.86 224.0 #> 2 56 223.23 215.54 220.4 #> 3 57 223.20 216.51 227.8 #> 4 58 224.03 218.49 217.2 #> 5 59 222.86 219.17 218.6 #> 6 60 222.83 220.00 227.0 #> 7 61 223.23 221.97 265.4 #> 8 62 228.63 223.23 255.8

  

  





The orders for days 1 to 130 are not pulled from R's random number generator. They come from a repeatable formula, a base level that climbs 0.4 cases a day plus a repeatable wobble, so every reader who runs this sees the exact same 130 days. Day 61 and 62 then get a one-off festival bump of 35 cases, and from day 91 onward a competitor's closing adds a permanent 15 cases.



horizon_1_forecast is yesterday's trailing average, used to forecast today's bake. horizon_7_forecast is the trailing average from a week ago, used seven days back to forecast today's flour order. Look at days 55 to 60: both forecasts sit close to each other and close to the actual orders, because nothing unusual is happening yet.



Then day 61 arrives. Actual orders jump to 265.4, but both forecasts stay far below it, at 223.23 and 221.97, because both are built purely from past orders, and a one-off festival leaves no trace in the data that came before it.



Note
horizon_1_forecast and horizon_7_forecast are both trailing 7-day averages. The only difference is which day each one was computed on: horizon 1 uses yesterday's average, horizon 7 uses the average from a week ago. Same formula, same kind of input, different point in time.