Multivariate forecasting

The VAR model#

A VAR($p$) for a $K$-dimensional vector $\mathbf{y}_t$ is

$$ \mathbf{y}_t = \mathbf{c} + A_1 \mathbf{y}_{t-1} + \dots + A_p \mathbf{y}_{t-p} + \mathbf{u}_t, \qquad \mathbf{u}_t \sim \mathrm{WN}(\mathbf{0}, \Sigma_u) $$

• $\mathbf{y}_t \in \mathbb{R}^K$ — all $K$ series stacked at time $t$.
• $A_i \in \mathbb{R}^{K \times K}$ — the coefficient matrix at lag $i$.
• $\Sigma_u$ — the contemporaneous covariance of the innovations, generally not diagonal.

Stability requires all roots of $\det(I_K - A_1 z - \dots - A_p z^p) = 0$ to lie outside the unit circle. Under stability the process has a Wold representation $\mathbf{y}t = \boldsymbol{\mu} + \sum{i=0}^{\infty}\Phi_i \mathbf{u}{t-i}$ and the $h$-step forecast satisfies $\hat{\mathbf{y}}{T+h|T} \to \boldsymbol{\mu}$ as $h\to\infty$. That limit is exactly what the developer output demonstrates numerically.

Because every equation has identical regressors, equation-by-equation OLS is efficient — seemingly-unrelated-regressions machinery buys nothing here. Parameter count is $K(Kp+1)$, growing quadratically in $K$, which is the binding constraint in practice.

Order selection and the curse of dimensionality#

Lag order is chosen by AIC, HQ, SC/BIC or FPE. AIC over-selects asymptotically but often forecasts better; BIC is consistent. With $K=5$ and monthly data, moving from $p=2$ to $p=4$ adds 50 coefficients per equation.

Standard remedies:

• Bayesian VAR with a Minnesota prior (Litterman, 1986; Doan, Litterman and Sims, 1984), which shrinks each equation toward a random walk with prior variance decaying in lag length. Bańbura, Giannone and Reichlin (2010) show large BVARs with appropriately tightened shrinkage forecast well with over 100 variables.
• Factor models. Extract a few common factors by principal components, then run a VAR on the factors. Stock and Watson (2002) established the diffusion-index approach.
• Reduced-rank and sparse VARs, imposing structure on ${A_i}$ directly.

Cointegration and the VECM#

If each component is $I(1)$ but some linear combination $\boldsymbol{\beta}'\mathbf{y}_t$ is $I(0)$, the series are cointegrated. Differencing then discards the long-run relationship. The correct form is the vector error correction model

$$ \Delta \mathbf{y}t = \boldsymbol{\alpha}\boldsymbol{\beta}'\mathbf{y}{t-1} + \sum_{i=1}^{p-1}\Gamma_i \Delta\mathbf{y}_{t-i} + \mathbf{u}_t $$

• $\boldsymbol{\beta} \in \mathbb{R}^{K \times r}$ — the cointegrating vectors; $r$ is the cointegration rank.
• $\boldsymbol{\alpha} \in \mathbb{R}^{K \times r}$ — the adjustment speeds toward equilibrium.

Johansen's trace and maximum-eigenvalue tests determine $r$ from the eigenvalues of a reduced-rank regression. statsmodels.tsa.vector_ar.vecm implements both the test and the estimator. Granger's representation theorem establishes the equivalence between cointegration and the existence of an error-correction form.

Granger causality, stated correctly#

$x$ Granger-causes $y$ if

$$ \mathrm{MSE}!\left(\mathbb{E}[y_{t+1} \mid \mathcal{F}t]\right) < \mathrm{MSE}!\left(\mathbb{E}[y{t+1} \mid \mathcal{F}t \setminus {x{1:t}}]\right) $$

The test is an $F$ or Wald test of the joint restriction that the lag coefficients on $x$ are zero in the $y$ equation. Three conditions are required and routinely violated:

1. Both series stationary. On $I(1)$ data the test over-rejects severely. Toda and Yamamoto (1995) give a lag-augmentation procedure that is valid regardless of the integration order.
2. No omitted common driver. If $z$ drives both with different lags, $x$ will Granger-cause $y$ with no causal link between them.
3. Sampling frequency at least as fine as the causal mechanism. Aggregating a daily mechanism to monthly data can reverse the apparent direction. Marcellino (1999) analyses temporal aggregation effects.

Granger himself framed the concept as predictive precedence, not intervention. Causal claims need the interventional machinery of Pearl (2009) or a design that supports it.

Structural identification#

The reduced-form innovations $\mathbf{u}_t$ are correlated across equations, so an impulse response to "a shock in series 1" is not defined without further assumptions. A structural VAR writes $\mathbf{u}_t = B\boldsymbol{\varepsilon}_t$ with $\boldsymbol{\varepsilon}_t$ orthogonal, and $B$ must be identified. Common schemes:

• Cholesky (recursive) ordering. Identification comes entirely from the assumed ordering, and results can change materially when it is permuted. Report robustness across orderings or the analysis is uninformative.
• Long-run restrictions (Blanchard and Quah, 1989).
• Sign restrictions (Uhlig, 2005), which give set rather than point identification.
• External instruments / proxy SVAR (Stock and Watson, 2012; Mertens and Ravn, 2013).

Sims (1980), Macroeconomics and Reality, is the paper that introduced VARs as an alternative to incredible identifying restrictions in large structural models, and the identification debate has run ever since.

Modern alternatives#

Global deep models handle the many-series case where classical VARs cannot. DeepAR (Salinas et al., 2020) conditions on covariates and pools across series. Temporal Fusion Transformers (Lim et al., 2021) separate static covariates, known-future inputs and observed-past inputs explicitly — a taxonomy worth adopting regardless of model, because it forces the availability question this lesson opens with. Gradient-boosted trees over engineered cross-series features remain the pragmatic default, as covered in feature engineering for time series.

The availability constraint is model-independent. No architecture removes the need to know a covariate's future value, and the developer block's 265.5 against 119.5 is the general shape of that cost.

Reading#

• Lütkepohl (2005), New Introduction to Multiple Time Series Analysis, Springer — the standard reference for VAR and VECM.
• Sims (1980), Macroeconomics and Reality, Econometrica 48(1).
• Granger (1969), Investigating causal relations by econometric models and cross-spectral methods, Econometrica 37(3).
• Johansen (1991), Estimation and hypothesis testing of cointegration vectors in Gaussian vector autoregressive models, Econometrica 59(6).
• Toda and Yamamoto (1995), Statistical inference in vector autoregressions with possibly integrated processes, Journal of Econometrics 66.
• Bańbura, Giannone and Reichlin (2010), Large Bayesian vector auto regressions, Journal of Applied Econometrics 25(1).
• Lim, Arık, Loeff and Pfister (2021), Temporal Fusion Transformers, IJF 37(4) — arxiv.org/abs/1912.09363.

What to learn next#

• Evaluating a forecast — the only way to know whether the extra series earned its place.
• Feature engineering for time series — the practical route to using many helper series at once.
• Monitoring and model drift — what happens when a helper series changes behaviour after you deploy.