Learn AI Create the Future — with Pranay
Main site

scikit-learn Properly

The fit, predict, transform contract

Every scikit-learn tool follows one contract — fit learns from data, transform reshapes data, predict gives answers — so learning it once unlocks the whole library.

Read these first

On this page 5
  1. Why it exists
  2. How it works
  3. A real example you have seen
  4. Remember this
  5. What to learn next

One lesson, three depths. Pick the one that fits you today — you can switch any time.

Beginner — No maths. Plain English.

Every tool in scikit-learn has the same three buttons: fit to learn, transform to reshape data, and predict to give answers.

Think of a tailor stitching you a kurta. First he takes your measurements once, with a tape. After that, he can stitch you ten kurtas without measuring you again. Measuring is fit. Stitching from the saved measurements is predict.

scikit-learn calls each of its tools an estimator — any object that can learn something from data. A spam filter is an estimator. So is the little tool that rescales numbers before training.

Why it exists

Machine learning libraries used to be a junk drawer. Every algorithm had its own function names, its own input format, its own quirks. Learning a new algorithm meant learning a new interface from scratch.

scikit-learn made one promise instead. Every estimator, no matter how fancy, obeys the same contract. Once you know the contract, you know the whole library. Swapping a decision tree for a neural network changes one line.

How it works

training data --fit-->      estimator   (it measures, and remembers)

new data      --predict-->  answers     (using only what it remembered)
new data      --transform-> reshaped data

fit is the only step that learns. Everything the estimator learned gets stored inside it, like the tailor's notebook of measurements.

predict is for models: give me data, I give you answers. transform is for data-preparation tools: give me data, I give it back cleaned or rescaled. Neither one learns anything new. They only apply the notebook.

A real example you have seen

Your email's spam filter was fitted once, on millions of old emails that people had marked. Now, every time a new mail arrives, the filter runs predict on it. It does not re-learn from scratch per email. It applies what it remembered.

Remember this

  • An estimator is any scikit-learn tool that learns from data.
  • fit learns and remembers. It is the only step that looks at answers.
  • predict and transform reuse what was remembered — on data the estimator has never seen.

What to learn next

Developer — Code and libraries.

Setup

bash
pip install scikit-learn

Written and tested against scikit-learn 1.7. The contract shown here has been stable for over a decade, so nearby versions behave the same.

One contract, two kinds of estimator

contract.py
import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler

# Six students: [hours studied, hours slept]. 1 means they passed.
X = np.array([[1., 8.], [2., 7.], [3., 6.],
              [6., 6.], [7., 5.], [8., 4.]])
y = np.array([0, 0, 0, 1, 1, 1])

scaler = StandardScaler()
scaler.fit(X)                        # learn: the mean and spread of each column
print("learned means:", scaler.mean_)
print("learned spreads:", scaler.scale_.round(3))

X_scaled = scaler.transform(X)       # apply: shift and squash using what fit stored
print("first row, scaled:", X_scaled[0].round(3))

model = LogisticRegression()
model.fit(X_scaled, y)               # learn: one weight per column, plus a bias
new_student = scaler.transform([[5., 6.]])
print("prediction:", model.predict(new_student))
print("confidence:", model.predict_proba(new_student).round(3))
Output
learned means: [4.5 6. ]
learned spreads: [2.63  1.291]
first row, scaled: [-1.331  1.549]
prediction: [1]
confidence: [[0.451 0.549]]

The walkthrough

The trailing underscore is a promise. scaler.mean_ ends with _, and that is a library-wide convention: any attribute ending in an underscore was learned during fit. scaler.mean_ did not exist before fit ran. Hyperparameters you chose yourself, like LogisticRegression(C=1.0), never carry the underscore.

A transformer and a model are the same species. StandardScaler and LogisticRegression both have fit. They differ only in what comes after: transformers answer transform, models answer predict. Some, like KMeans, answer both.

The new student goes through the same scaler. We scale [[5., 6.]] with the scaler fitted on training data — we do not fit a fresh one. The tailor uses your old measurements; he does not measure the new kurta.

That confidence is honest. 0.549 means the model finds this student a near coin-flip. Five study hours sits right between the pass and fail groups, so a weak answer is the correct answer.

Calling transform before fit fails loudly:

python
StandardScaler().transform([[1., 2.]])
Output
sklearn.exceptions.NotFittedError: This StandardScaler instance is not fitted yet. Call 'fit' with appropriate arguments before using this estimator.

Common mistakes

Fitting the scaler again on new data. Calling scaler.fit(new_student) before transforming recentres the data around the new point. Predictions become garbage with no error raised. Fit on training data once; afterwards, only transform.

Expecting fit to accumulate. Calling fit a second time wipes everything learned the first time and starts fresh. Estimators that can genuinely learn in increments expose a separate method, partial_fit.

Passing a 1D array. X must be 2D: rows are examples, columns are features. Passing np.array([1., 2., 3.]) raises a ValueError telling you to reshape with .reshape(-1, 1). One column means one feature; the error message spells out both fixes.

Typing fit_transform on test data out of habit. fit_transform(X_train) is fine — it is fit then transform in one call. On test data it silently re-learns, which is the first mistake again in disguise. The rule: fit_transform for training data, plain transform for everything after.

Try it yourself

Predict a student with 2 study hours and 8 sleep hours, and read the confidence. Then print model.coef_ and check which of the two features got the larger weight. Does the sign of each weight match your intuition?

What to learn next

Researcher — Mathematics and papers.

The API as a formal design

The contract is documented in Buitinck et al. (2013), API design for machine learning software: experiences from the scikit-learn project. Three interfaces compose:

  • Estimator: anything with fit(X, y=None). Fitting maps a hyperparameter setting plus data to a fitted state. Hyperparameters are constructor arguments; learned state is the set of trailing-underscore attributes.
  • Predictor: adds predict(X), optionally predict_proba and decision_function.
  • Transformer: adds transform(X) and, by mixin, fit_transform.

Two invariants carry the entire ecosystem:

  1. __init__ performs no work and no validation — it stores arguments verbatim. This makes get_params() / set_params() a complete serialisation of the unfitted estimator, which is what GridSearchCV mutates.
  2. clone(est) builds a fresh unfitted copy from get_params() alone. Every meta-estimator (Pipeline, GridSearchCV, cross_val_score) relies on clone semantics to guarantee no state leaks between folds.

Statefulness, formally

For a transformer, fit estimates parameters t(D_train) from training data D_train, and transform(X) applies a fixed function g(X; t). The design forces the estimation step and the application step apart. That separation is exactly what makes leakage detectable in code review: any call of fit on non-training data is a bug you can grep for.

Complexity notes

The contract fixes interfaces, not costs. For the two estimators above:

  • StandardScaler.fit is one pass: O(nd) time, O(d) memory, for n samples and d features.
  • LogisticRegression with the default lbfgs solver costs O(nd) per iteration, with iteration count depending on conditioning — which is one reason scaling first matters.

Modern extensions

  • set_output API (1.2+): scaler.set_output(transform="pandas") makes transformers emit DataFrames with column names preserved.
  • Estimator tags (rewritten in 1.6 as __sklearn_tags__): machine-readable claims about what an estimator supports — sparse input, multilabel output — consumed by check_estimator and meta-estimators.
  • Metadata routing (1.3+, SLEP006): a uniform mechanism for passing sample_weight and groups through nested meta-estimators.
  • check_is_fitted(est) implements the fitted test used everywhere: does any trailing-underscore attribute exist?

The API paper's central claim has aged well: composability comes from a small number of enforced conventions, not from inheritance hierarchies. Pipeline is possible because every step is a value of the same type.

What to learn next