Exponential smoothing

The recursions#

Simple exponential smoothing (SES). Component form:

$$ \hat{y}_{t+h|t} = \ell_t, \qquad \ell_t = \alpha y_t + (1-\alpha)\ell_{t-1} $$

• $\ell_t$ — the level at time $t$.
• $\alpha \in [0,1]$ — the smoothing parameter for the level.
• $h$ — the forecast horizon. The forecast function is flat in $h$.

Expanding gives the weighted-average form $\hat{y}{t+1|t} = \sum{j=0}^{t-1} \alpha(1-\alpha)^j y_{t-j} + (1-\alpha)^t \ell_0$, showing geometrically decaying weights and the persistent influence of the initial state $\ell_0$.

Holt's linear trend.

$$ \ell_t = \alpha y_t + (1-\alpha)(\ell_{t-1} + b_{t-1}), \qquad b_t = \beta^(\ell_t - \ell_{t-1}) + (1-\beta^)b_{t-1} $$ $$ \hat{y}_{t+h|t} = \ell_t + h\,b_t $$

• $b_t$ — the slope estimate at time $t$.
• $\beta^* \in [0,1]$ — the smoothing parameter for the slope.

Damped trend (Gardner and McKenzie, 1985) replaces $h$ by $\sum_{i=1}^{h}\phi^i$ and multiplies $b_{t-1}$ by $\phi$ in both recursions, with $\phi \in (0,1)$. As $h \to \infty$ the forecast converges to $\ell_t + \phi b_t/(1-\phi)$ rather than diverging.

Holt-Winters additive, with period $m$:

$$ \ell_t = \alpha(y_t - s_{t-m}) + (1-\alpha)(\ell_{t-1}+b_{t-1}) $$ $$ b_t = \beta^(\ell_t - \ell_{t-1}) + (1-\beta^)b_{t-1} $$ $$ s_t = \gamma(y_t - \ell_{t-1} - b_{t-1}) + (1-\gamma)s_{t-m} $$ $$ \hat{y}_{t+h|t} = \ell_t + h b_t + s_{t-m+h_m^+}, \qquad h_m^+ = \lfloor (h-1) \bmod m \rfloor + 1 $$

• $s_t$ — the seasonal index for the seasonal position of time $t$.
• $\gamma \in [0, 1-\alpha]$ — the seasonal smoothing parameter. The upper bound comes from the state-space stability region, not from the range of a probability.

The multiplicative form replaces the subtractions and additions involving $s$ with divisions and multiplications.

The ETS state-space framework#

Hyndman, Koehler, Snyder and Grose (2002) put exponential smoothing on a statistical footing by writing each method as an innovations state-space model

$$ y_t = w(\mathbf{x}{t-1}) + r(\mathbf{x}{t-1})\,\varepsilon_t, \qquad \mathbf{x}t = f(\mathbf{x}{t-1}) + g(\mathbf{x}_{t-1})\,\varepsilon_t $$

with a single source of error $\varepsilon_t \sim \mathrm{N}(0,\sigma^2)$. The taxonomy is ETS(Error, Trend, Seasonal), each slot taking values from {A, M, N} plus damped variants — thirty combinations, of which fifteen are numerically stable in practice.

This buys three things the ad hoc recursions never had:

1. A likelihood. Parameters and initial states are estimated jointly by maximum likelihood, and models are compared by AICc.
2. Prediction intervals. Analytic variance expressions exist for most members; for the rest, simulation from the innovations model works.
3. Automatic model selection. AutoETS in statsforecast, or ets() in R's forecast package, searches the space by AICc.

Additive-error ETS(A,N,N) coincides with ARIMA(0,1,1); ETS(A,A,N) with ARIMA(0,2,2); ETS(A,A,A) with a restricted seasonal ARIMA. The multiplicative-error and multiplicative-seasonal members have no ARIMA equivalent, and non-linear members have no finite-order linear representation at all. The classes overlap; neither contains the other.

Parameter space and initialisation#

The naive constraint $\alpha, \beta^, \gamma \in [0,1]$ is not the correct one. The admissible region is defined by requiring the roots of the model's characteristic equation to lie inside the unit circle, so that the weight on past observations decays. For ETS(A,A,N) the admissible region is $0 < \alpha < 2$, $0 < \beta^ < 4/\alpha - 2$, which extends well beyond the unit square. Hyndman et al. (2008), chapter 10, derives these.

Initialisation matters more than most practitioners expect on short series. initialization_method="estimated" treats $\ell_0, b_0, s_{-m+1:0}$ as free parameters in the likelihood; "heuristic" uses a decomposition-based starting guess. On series shorter than three or four seasonal cycles the two can differ materially.

Empirical standing#

Exponential smoothing is not a historical curiosity. In M3 (Makridakis and Hibon, 2000) damped-trend exponential smoothing was among the top performers across 3003 series. In M4 (2018) the winning entry, Smyl's ES-RNN, was a hybrid that used Holt-Winters-style level and seasonality updates with a shared recurrent network learning the residual dynamics across series. In M5 (2020), on hierarchical retail data, simple exponentially-smoothed baselines were beaten mainly by gradient-boosted trees over engineered features — an approach covered in feature engineering for time series.

The recurring empirical finding across all of them: method complexity does not monotonically improve accuracy, and combinations of simple methods are unusually strong.

Reading#

• Brown (1959), Statistical Forecasting for Inventory Control — the origin, motivated by naval inventory.
• Holt (1957/2004), Forecasting seasonals and trends by exponentially weighted moving averages, reprinted in IJF 20(1).
• Winters (1960), Forecasting sales by exponentially weighted moving averages, Management Science 6(3).
• Gardner and McKenzie (1985), Forecasting trends in time series, Management Science 31(10).
• Hyndman, Koehler, Ord and Snyder (2008), Forecasting with Exponential Smoothing: The State Space Approach, Springer — the definitive treatment.
• Hyndman and Athanasopoulos, Forecasting: Principles and Practice, 3rd ed., chapter 8 — otexts.com/fpp3/expsmooth.html.

What to learn next#

• ARIMA — the other classical family, built on autocorrelation rather than smoothing.
• Evaluating a forecast — how to compare these fairly across many origins.
• Prophet — a modern tool that handles holidays and moving festivals directly.