Anomaly detection

Framing#

Given ${y_t}$, an anomaly detector is a map to a score $a_t \in \mathbb{R}$ and a threshold rule $\mathbf{1}{a_t > \tau}$. The residual-based family constructs

$$ r_t = y_t - \hat{y}{t \mid \mathcal{F}{t-1}}, \qquad a_t = \frac{|r_t - \mathrm{med}(r)|}{c \cdot \mathrm{MAD}(r)} $$

• $\hat{y}{t\mid\mathcal{F}{t-1}}$ — the expectation from a model using only prior information.
• $\mathrm{MAD}(r) = \mathrm{med}(|r - \mathrm{med}(r)|)$.
• $c = 1/\Phi^{-1}(0.75) \approx 1.4826$, making $c \cdot \mathrm{MAD}$ a consistent estimator of $\sigma$ under normality. The reciprocal $0.6745 = \Phi^{-1}(0.75)$ appears when the scaling is written as a multiplier on the numerator instead.

The causality requirement matters for deployment. STL and centred rolling windows use future observations and are valid only for retrospective analysis. For real-time detection the expectation must come from a one-step-ahead forecast, or from an online decomposition.

Chandola, Banerjee and Kumar (2009), Anomaly detection: A survey, ACM Computing Surveys 41(3), gives the standard taxonomy: point, contextual and collective anomalies. Seasonal series are the canonical contextual case, and the contextual variable is the phase within the cycle.

Robustness, quantified#

The breakdown point of an estimator is the smallest fraction of arbitrarily corrupted observations that can drive it to an arbitrary value.

MAD's low Gaussian efficiency is the price of its breakdown point. Rousseeuw and Croux (1993), Alternatives to the median absolute deviation, JASA 88(424), propose $S_n$ and $Q_n$ with the same 50 percent breakdown and materially better efficiency; $Q_n$ is based on the 0.25 quantile of pairwise distances $|x_i - x_j|$ and computable in $O(n \log n)$.

MAD also degenerates when more than half the data are identical — a real failure on sparse count series where the median and MAD are both zero, giving division by zero. Guard with a floor, or use a quantile-range scale instead.

Extreme value theory, instead of a hand-picked threshold#

Choosing $\tau$ by eye is the weakest link. The peaks-over-threshold approach places it on a footing. The Pickands-Balkema-de Haan theorem states that for a broad class of distributions the conditional excess distribution converges to a generalised Pareto:

$$ P(X - u \le x \mid X > u) \;\to\; G_{\xi,\beta}(x) = 1 - \left(1 + \frac{\xi x}{\beta}\right)^{-1/\xi} $$

• $u$ — a high initial threshold.
• $\xi$ — the shape (tail index); $\xi>0$ is heavy-tailed.
• $\beta$ — the scale.

Fit $(\xi, \beta)$ by maximum likelihood on the exceedances, then set the alarm threshold at a chosen exceedance probability $q$. Siffer, Fouque, Termier and Largouët (2017), Anomaly Detection in Streams with Extreme Value Theory, KDD, give SPOT and DSPOT, which do this in streaming form with a drifting threshold and no distributional assumption on the bulk.

Evaluation is the hardest part of this subfield#

Class imbalance of 1:1000 or worse makes accuracy meaningless. Use precision, recall and PR-AUC, not ROC-AUC — under extreme imbalance ROC-AUC stays high while precision is unusable (Davis and Goadrich, 2006).

Time complicates the definition of a true positive. An event lasting six hours detected at hour three: is that one hit, one hit and five misses, or a hit with a three-hour delay? Two conventions dominate:

• Point-adjust (used by many deep-learning papers): if any point inside a true anomalous segment is flagged, all points in that segment count as detected. Kim et al. (2022), Towards a Rigorous Evaluation of Time-series Anomaly Detection, AAAI, show this metric is so permissive that a random scorer achieves high F1 on standard benchmarks. Results reported under point-adjust are not comparable to anything.
• Range-based precision and recall (Tatbul et al., 2018, NeurIPS) parameterise credit for existence, size, position and cardinality of overlaps, and behave sensibly.

The benchmark datasets themselves are also compromised. Wu and Keogh (2023), Current Time Series Anomaly Detection Benchmarks are Flawed, TKDE, document triviality, mislabelling and run-to-failure bias in Yahoo S5, Numenta NAB and SMAP/MSL, and show that a one-line baseline is competitive on much of them. Treat published leaderboards on those datasets with scepticism.

Where the field is#

Reconstruction-based deep methods — autoencoders, variational autoencoders, GAN discriminators — score by reconstruction error. See autoencoders for the mechanism. They handle multivariate, high-dimensional streams that classical methods cannot.

The honest summary from the benchmark critiques above: on univariate seasonal series, a well-specified decomposition with a robust residual score is hard to beat, and much of the reported deep-learning advantage in this area dissolves under a correct evaluation protocol. The genuine gains are in high-dimensional multivariate settings with cross-signal dependence, which is where classical residual methods have nothing to offer.

Reading#

• Chandola, Banerjee and Kumar (2009), Anomaly detection: A survey, ACM Computing Surveys 41(3).
• Rousseeuw and Croux (1993), Alternatives to the median absolute deviation, JASA 88(424).
• Siffer et al. (2017), Anomaly Detection in Streams with Extreme Value Theory, KDD.
• Tatbul, Lee, Zdonik, Alam and Gottschlich (2018), Precision and Recall for Time Series, NeurIPS.
• Kim, Choi, Choi, Lee and Yoon (2022), Towards a Rigorous Evaluation of Time-series Anomaly Detection, AAAI — arxiv.org/abs/2109.05257.
• Wu and Keogh (2023), Current Time Series Anomaly Detection Benchmarks are Flawed, IEEE TKDE 35(3).

What to learn next#

• Monitoring and model drift — the same idea applied to a model's own behaviour in production.
• Multivariate forecasting — detecting anomalies across several signals at once.
• Trend, seasonality and noise — the decomposition every good detector is built on.