Bias-variance tradeoff

The bias-variance tradeoff is the tension between a model that is too rigid to capture the pattern and one so flexible it captures the noise too.

Think of fitting a suit. A one-size kurta fits nobody well — that is high bias: the same wrong answer for everyone, consistently. A suit stitched to every wrinkle of one photograph fits that pose only — that is high variance: it changes wildly with each new photo. A good tailor cuts to the body, not to the wrinkle.

In model terms, bias is error from assumptions that are too strong, like forcing a straight line through curved data. Variance is error from sensitivity to the particular training sample: retrain on slightly different data and the model swings. High bias shows up as underfitting; high variance shows up as overfitting.

too rigid   ←———————— sweet spot ————————→   too flexible
high bias                                    high variance
bad on train AND test                        great on train, bad on test

You steer along this axis with model size, regularization, and more training data. More data reduces variance without adding bias, which is why "get more data" is such common advice.

Where to go next#

• Full lesson: Overfitting and underfitting
• Related terms: overfitting, underfitting, regularization, cross-validation