Versioning a model API

Setup#

pip install scikit-learn pandas numpy pytest

Two contracts, one model#

"""Two versions of the same underlying model, served under two frozen,
independent contracts. v1 never changes shape once real callers depend on it.
"""
import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

COLUMNS = ["income", "years", "age"]
MODEL_VERSION = "2026.03"


def make_training_data(n=400, seed=0):
    rng = np.random.RandomState(seed)
    X = pd.DataFrame({
        "income": rng.uniform(5, 80, n).round(1),
        "years": rng.uniform(0, 10, n).round(1),
        "age": rng.randint(21, 65, n).astype(float),
    })
    score = 0.05 * X["income"] + 0.35 * X["years"] + 0.01 * X["age"] - 3.5
    y = (score + rng.normal(0, 0.8, n) > 0).astype(int)
    return X, y


X, y = make_training_data()
MODEL = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000)).fit(X, y)


def _score(applicant: dict) -> float:
    row = pd.DataFrame([applicant], columns=COLUMNS)
    return float(MODEL.predict_proba(row)[0, 1])


def predict_v1(applicant: dict) -> dict:
    """FROZEN since launch. Exactly these two keys, these two types. Never add,
    remove, or rename a field here.
    """
    probability = _score(applicant)
    return {
        "repaid": int(probability >= 0.5),
        "confidence": round(probability, 2),
    }


def predict_v2(applicant: dict) -> dict:
    """The current version. Free to evolve -- this is where new fields go."""
    probability = _score(applicant)
    return {
        "decision": "approve" if probability >= 0.5 else "decline",
        "probability": round(probability, 4),
        "model_version": MODEL_VERSION,
    }


applicant = {"income": 60.0, "years": 7.0, "age": 34.0}
print("v1:", predict_v1(applicant))
print("v2:", predict_v2(applicant))

v1: {'repaid': 1, 'confidence': 0.99}
v2: {'decision': 'approve', 'probability': 0.9941, 'model_version': '2026.03'}

v2 renamed repaid to decision and changed it from an integer to a string, added a full-precision probability instead of a rounded confidence, and added a version tag v1 never had. Every one of those is a breaking change for anything still reading v1's shape — which is exactly why they live in v2 instead of being edited into v1.

A contract test that actually enforces the freeze#

A promise that is only written in a design document gets broken eventually, by someone who never read the document. A promise enforced by a test gets caught the moment it is broken, by name.

from api import predict_v1, predict_v2

V1_APPLICANT = {"income": 60.0, "years": 7.0, "age": 34.0}


def test_v1_response_has_exactly_the_frozen_keys_and_types():
    response = predict_v1(V1_APPLICANT)
    assert set(response.keys()) == {"repaid", "confidence"}
    assert isinstance(response["repaid"], int)
    assert response["repaid"] in (0, 1)
    assert isinstance(response["confidence"], float)
    assert 0.0 <= response["confidence"] <= 1.0


def test_v1_repaid_field_is_never_renamed_to_decision():
    # a real, common "helpful improvement" that would break every existing v1 caller
    response = predict_v1(V1_APPLICANT)
    assert "decision" not in response
    assert "repaid" in response


def test_v2_is_free_to_carry_more_fields_than_v1():
    response = predict_v2(V1_APPLICANT)
    assert set(response.keys()) == {"decision", "probability", "model_version"}

pytest test_contract.py -q

...                                                                      [100%]
3 passed in 0.86s

Now break something on purpose#

A well-meaning "improvement": add a more precise probability field to v1, alongside the existing confidence — surely harmless, it only adds information.

# in api.py, predict_v1 becomes:
return {
    "repaid": int(probability >= 0.5),
    "confidence": round(probability, 2),
    "probability": round(probability, 4),  # "only adding more detail" -- feels harmless
}

F..                                                                      [100%]
================================== FAILURES ===================================
___________ test_v1_response_has_exactly_the_frozen_keys_and_types ____________

    def test_v1_response_has_exactly_the_frozen_keys_and_types():
        response = predict_v1(V1_APPLICANT)
>       assert set(response.keys()) == {"repaid", "confidence"}
E       AssertionError: assert {'confidence'...ty', 'repaid'} == {'confidence', 'repaid'}
E         
E         Extra items in the left set:
E         'probability'
E         Use -v to get more diff

test_contract.py:8: AssertionError
=========================== short test summary info ===========================
FAILED test_contract.py::test_v1_response_has_exactly_the_frozen_keys_and_types
1 failed, 2 passed in 1.28s

This is a genuinely debatable case, which is exactly why it is worth including: adding a field rarely breaks a caller reading specific known fields by name — but it is not free of risk either. Some client libraries validate a response against a strict schema and reject anything with an unexpected extra field; some callers iterate over every key in the response and would now see one they do not recognise. The contract test does not decide whether this change is acceptable — it makes sure the decision is made on purpose, by a human looking at a failing, precisely-named test, rather than by accident.

Common mistakes#

Treating "nobody complained yet" as proof nothing broke. A caller that silently ignores an unexpected field, or crashes in a way nobody is monitoring, does not complain — it quietly fails instead. A contract test catches the change at the moment it is made, which is far cheaper than waiting for a bug report.

No version at all, ever. A service with one unversioned endpoint has no way to improve the contract without breaking every existing caller on that exact day. Even a service with exactly one real client benefits from v1 existing as a name, because it makes the next change a deliberate decision instead of a silent edit.

Versioning the model but not the API. Retraining a model and improving its accuracy is not, by itself, a contract-breaking change — the shape of predict_v1's response is unaffected by which weights are behind _score. Confusing "the model changed" with "the API changed" leads teams to either version too often or not carefully enough.

Retiring a version with no notice. Turning off v1 the moment v2 ships breaks every caller who has not migrated yet, which defeats the entire purpose of having versioned it in the first place. Publish a deprecation date, communicate it, and only remove v1 after that date has actually passed.

Putting the version somewhere inconsistent. Whether it lives in the URL path (/v1/predict), a header (API-Version: 1), or the request body, pick one convention and use it everywhere in your API — a service that versions some endpoints one way and others a different way is a constant source of integration mistakes for callers.

Try it yourself#

Add a third contract test asserting that predict_v1's confidence field is always rounded to exactly two decimal places — not one, not three. Then change the rounding to round(probability, 3) and watch the test name tell you precisely what changed, without needing to read a diff to find out.

What to learn next#

• KServe and Seldon — production platforms that handle versioned model serving, including much of this lesson's contract and routing logic, for you.
• Switching between model providers — a related but distinct kind of contract change, where the thing behind your API is not your own model at all.
• Model registries — tracking the model-version axis this lesson's researcher section separates from the contract-version axis.