Choosing how often to refit a forecast model

Today let's understand how often a forecast model should be refit, and what goes wrong on either side of the right answer.

Highgate Paper Co. is a wholesale paper products supplier. Its retail stationery customers place orders every week, and Highgate forecasts next week's order volume to plan production and shipping. Here are its last 100 weeks of orders, in cases a week. For the first 60 weeks, demand holds steady around 300 cases. Then, at week 61, a regional retail chain starts placing standing weekly orders, and the level steps up to 390 cases and stays there.

Highgate's forecast comes from a model, and like any forecasting model, it has to be refit now and then: re-estimated on the latest orders. Refit it too often, and you risk chasing noise instead of a real pattern. Refit it too rarely, and you risk missing that the level actually moved, the way it does at week 61 above. This lesson works out, from Highgate's own 100 weeks, how often is actually often enough.

The two ways a refit schedule goes wrong

Refit means re-estimating a forecasting model's parameters on new data. For Highgate's order forecasts, that means recomputing whatever the model uses to predict next week's orders, using the most recent weeks of actual orders.

A refit schedule can fail two different ways. Refit too often, and ordinary week-to-week noise pushes the forecast around, even on weeks when nothing about the real level of demand has changed. Refit too rarely, and the forecast keeps repeating an old level long after the real level has moved, the way Highgate's does at week 61.

Both failures cost real forecast accuracy, and the cost can be measured directly in Highgate's own numbers.