Irregular clinical time series

Formalising the problem#

Regular time series methods assume a fixed sampling interval Δt. Clinical time series are better modelled as a marked point process: a sequence of (time, value) pairs

{ (t_1, x_1), (t_2, x_2), ..., (t_n, x_n) },   t_1 < t_2 < ... < t_n

where the arrival times t_i are themselves a random process, not a fixed grid. Critically, the arrival process is not independent of the underlying patient state — this is sometimes called informative observation time or informatively missing data, distinct from the MCAR/MAR/MNAR taxonomy of missing values covered in what hospital data actually looks like, because here the timing itself is the missing-data mechanism.

Encoding irregularity for a model#

Three standard representations, in increasing order of how much timing information they preserve:

1. Last-observation-carried-forward + resample to fixed Δt
   -- loses all timing information, the naive approach shown above

2. (value, time-since-last-observation, mask) triplets per feature per step
   -- keeps timing as an explicit input; used by GRU-D

3. Continuous-time models
   -- treat t as a continuous input, no discretisation at all

GRU-D (Che et al., 2018) augments a GRU with a learned decay term. The hidden state decays toward the population mean as the gap since the last real observation grows:

gamma_t = exp( -max(0, W_gamma * delta_t + b_gamma) )
x_hat_t = m_t * x_t + (1 - m_t) * (gamma_t * x_last + (1 - gamma_t) * x_mean)

• delta_t — time since the feature was last actually observed
• m_t — a binary mask, 1 if observed at step t, 0 if missing
• gamma_t — a learned decay factor in (0, 1], computed per feature
• x_last, x_mean — the last observed value and the population mean for that feature

As delta_t grows, gamma_t shrinks toward 0, and the imputed value drifts toward the population mean rather than staying frozen at the last reading — a principled generalisation of the forward-fill shown in the developer block.

Neural ODEs and latent ODE models (Rubanova, Chen & Duvenaud, 2019) go further, treating the hidden state as the solution to a differential equation that can be evaluated at any real-valued time, removing discretisation entirely.

Cost#

For a patient with n irregular observations across d features:

Fixed-grid resampling at fine Δt can make sequence length explode independent of how much data actually exists — a patient measured twice in a week, resampled to five-minute bins, produces thousands of mostly-fabricated steps.

Key references#

• Che, Z. et al. (2018). Recurrent Neural Networks for Multivariate Time Series with Missing Values. Scientific Reports 8. Introduces GRU-D.
• Rubanova, Y., Chen, R. & Duvenaud, D. (2019). Latent ODEs for Irregularly-Sampled Time Series. NeurIPS.
• Little, R. & Rubin, D. (2019). Statistical Analysis with Missing Data (3rd ed.). Wiley. The standard reference on missingness mechanisms, extended here to missingness in time.
• Shukla, S. & Marlin, B. (2021). Multi-Time Attention Networks for Irregularly Sampled Time Series. ICLR.

Current state#

GRU-D and its descendants remain common baselines in clinical ML papers. Transformer-based approaches with continuous-time positional encodings (extending the ideas in rotary position embeddings to real-valued, non-integer positions) are an active area, but a well-engineered "time since last observation" feature fed to a gradient-boosted tree, as in the developer block, remains a strong and far cheaper baseline in most published comparisons.

What to learn next#

• Anomaly detection — the general technique, once timing is handled correctly.
• Predicting patient deterioration — putting these features to work in a real prediction task.
• Label leakage in clinical models — how an informative observation-time signal can itself leak the label.