ANOVA

The decomposition#

For $k$ groups with sizes $n_j$, means $\bar{y}_j$, grand mean $\bar{y}$, and $N = \sum_j n_j$ observations total:

$$ \underbrace{\sum_{j=1}^{k}\sum_{i=1}^{n_j} (y_{ij} - \bar{y})^2}{SS{total}} = \underbrace{\sum_{j=1}^{k} n_j (\bar{y}j - \bar{y})^2}{SS_{between}} + \underbrace{\sum_{j=1}^{k}\sum_{i=1}^{n_j} (y_{ij} - \bar{y}j)^2}{SS_{within}} $$

Where:

• $y_{ij}$ — observation $i$ in group $j$.
• $SS_{between}$ — variation of group means around the grand mean.
• $SS_{within}$ — variation of observations around their own group mean.

The test statistic divides each sum by its degrees of freedom:

$$ F = \frac{SS_{between} / (k-1)}{SS_{within} / (N-k)} $$

Under $H_0$ (all group means equal) with normal errors of common variance $\sigma^2$, $F \sim F_{k-1,\,N-k}$. Both numerator and denominator estimate $\sigma^2$ under $H_0$, so $F$ concentrates near 1; under the alternative the numerator inflates by $\frac{1}{k-1}\sum_j n_j(\mu_j - \bar\mu)^2$.

ANOVA as a linear model#

One-way ANOVA is the linear model $y_{ij} = \mu + \tau_j + \varepsilon_{ij}$ with $\varepsilon_{ij} \sim \mathcal{N}(0, \sigma^2)$, tested against the intercept-only model. The F-test is the general nested-model comparison; regression F-tests, two-way ANOVA with interactions, and ANCOVA are the same machinery with different design matrices. This is the cleanest way to remember the assumptions: independent errors, homoscedasticity, normality — in decreasing order of importance.

Assumption failure modes#

• Heteroscedasticity with unequal $n_j$ biases the level in either direction (liberal when big groups are quiet, conservative when big groups are noisy). Welch's ANOVA (Welch, 1951) and the Alexander–Govern test replace the pooled denominator; simulation evidence favours them as defaults.
• Non-normality: the F level is fairly robust for moderate $N$ by the CLT, but power suffers under heavy tails; the rank-based Kruskal–Wallis test (next lesson) is the standard fallback.
• Dependence (clustered or repeated measurements) is the assumption that actually destroys the test; mixed models or repeated-measures ANOVA are required.

Post-hoc theory#

Tukey's HSD (Tukey, 1949) controls the family-wise error rate over all $\binom{k}{2}$ pairwise comparisons using the studentised range distribution $q_{k, N-k}$; each interval is $\bar{y}_i - \bar{y}j \pm \frac{q{1-\alpha}}{\sqrt{2}} \, s_p \sqrt{\frac{1}{n_i} + \frac{1}{n_j}}$. Scheffé's method covers all contrasts (any weighted combination of means) at extra width; Dunnett's compares every group to one control at less width.

Effect size#

$\eta^2 = SS_{between}/SS_{total}$ is the variance fraction explained by group membership; $\omega^2$ corrects its upward bias:

$$ \omega^2 = \frac{SS_{between} - (k-1)\,MS_{within}}{SS_{total} + MS_{within}} $$

Fisher introduced the framework in Statistical Methods for Research Workers (1925) and the agricultural design context in The Design of Experiments (1935).

What to learn next#

• Non-parametric tests — the rank-based fallback when spreads and outliers misbehave.
• Multiple testing correction — the false-alarm budget problem ANOVA partially solves.
• Confidence intervals — reporting gaps with uncertainty instead of verdicts.