Logistic regression

Logistic regression is a classifier that outputs a probability: it computes a weighted sum of the inputs, then squashes it into the range 0 to 1.

Despite the name, it does classification, not regression — the historical name stuck. Think of a loan officer's mental arithmetic: salary adds points, an existing default subtracts many, years at the same job adds a few. The running total is not a decision yet. It becomes one by conversion into a confidence — "82% likely to repay" — and a cutoff turns confidence into approve or reject.

The squashing step is the sigmoid function, an S-shaped curve that maps any number into (0, 1). Large positive totals approach 1, large negative ones approach 0, and zero lands at exactly 0.5.

weighted sum:   −3      −1       0      +1      +3
sigmoid     :  0.05    0.27    0.50    0.73    0.95

Three reasons this remains everywhere. It outputs genuine probabilities, so the decision threshold can be tuned to the business — flag fraud at 0.3, approve loans at 0.9. Its weights are readable, which regulators in banking and medicine often require. And it is the standard baseline every fancier classifier must beat. Its multi-class extension replaces the sigmoid with softmax — which is, incidentally, the same final layer every neural classifier ends with.

Where to go next#

• Full lesson: Logistic regression
• Related terms: linear-regression, classification, softmax, baseline