Cross-validation

Cross-validation evaluates a model on several different splits of the data and averages the results, instead of trusting one split.

One train-test-split is one practice exam — and a single exam can flatter or punish you by luck of the questions. Cross-validation sets five different papers from the same syllabus and averages your marks. The average is a steadier estimate, and the spread across papers tells you how much luck is in play.

The standard recipe is k-fold. Cut the data into k equal parts (folds), commonly five. Train on four folds, test on the held-out fifth. Rotate until every fold has been the test set once, then average the five scores.

fold:     1    2    3    4    5
round 1: TEST train train train train
round 2: train TEST train train train
   ...                                → mean score ± spread

The price is training k models instead of one. That is trivial for scikit-learn models on small tables — where cross-validation is standard — and prohibitive for large neural networks, where a single held-out validation-set is the norm.

Two cautions. Do all preprocessing inside each fold, or statistics leak from test folds into training — a classic form of data-leakage. And for time-ordered data, random folds let the model peek at the future; use time-based splits instead.

Where to go next#

• Full lesson: Train/test split
• Related terms: train-test-split, validation-set, overfitting, data-leakage