Regression

Regression is the task of predicting a continuous number rather than a category.

An experienced fruit vendor glances at a watermelon and says "four and a half kilos". Not "heavy" or "light" — a number on a scale, judged from size, shape and sound when tapped. Regression models make that kind of estimate from features: tomorrow's electricity demand, a flat's price, minutes until your food arrives.

The line between regression and classification is what the answer looks like. Categories from a fixed menu: classification. A number where 4.5 is meaningfully between 4 and 5: regression. The distinction decides your loss-function, your metrics, and often your model.

Being wrong also works differently here. A classification is right or wrong; a regression is wrong by an amount. Metrics capture that amount in different moods:

MAE   average absolute error         honest, in the units you care about
RMSE  root mean squared error        punishes large misses extra hard
R²    variance explained             1.0 is perfect, 0 means "predict the average"

Choose by asking what an error costs. If one huge miss is catastrophic — under-ordering stock for a festival — RMSE's bias toward big errors is a feature. The simplest starting model is linear-regression, which is also the best first baseline for most tabular problems.

Where to go next#

• Full lesson: Linear regression
• Related terms: classification, linear-regression, loss-function, supervised-learning