MLflow

Setup#

pip install mlflow scikit-learn pandas

This is a large install — MLflow pulls in a web server, a database layer and plotting libraries. Expect a few hundred megabytes and a few minutes on a normal connection. Everything after that is local and offline.

A complete tracked sweep#

The example below writes to a single SQLite file in your working directory. It never contacts a server.

import mlflow
import mlflow.sklearn
import numpy as np
import pandas as pd
from mlflow.models import infer_signature
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score

# Where the records go: one SQLite file in this folder. No server, no account.
mlflow.set_tracking_uri("sqlite:///mlflow.db")
mlflow.set_experiment("loan-default")

rng = np.random.RandomState(0)
cols = ["income", "years", "age", "cards", "enquiries", "utilisation"]
X = pd.DataFrame(rng.normal(size=(300, 6)), columns=cols)
y = (X["income"] + 0.8 * X["years"] - 0.5 * X["age"]
     + rng.normal(0, 0.7, 300) > 0).astype(int)

for C in [0.01, 0.1, 1.0, 10.0]:
    with mlflow.start_run(run_name=f"logreg-C{C}"):
        model = LogisticRegression(C=C, max_iter=1000)
        accuracy = float(cross_val_score(model, X, y, cv=5).mean())
        model.fit(X, y)

        mlflow.log_param("C", C)
        mlflow.log_param("model", "logistic_regression")
        mlflow.log_metric("cv_accuracy", accuracy)
        # The signature records the column names and types. Without it the
        # model will happily accept columns in the wrong order.
        mlflow.sklearn.log_model(
            model, name="model",
            signature=infer_signature(X, model.predict(X)))
        print(f"C={C:<6} cv_accuracy={accuracy:.4f}")

runs = mlflow.search_runs(order_by=["metrics.cv_accuracy DESC"])
best = runs.iloc[0]
print("\nbest run")
print("  C          ", best["params.C"])
print("  cv_accuracy", round(best["metrics.cv_accuracy"], 4))
print("  runs logged", len(runs))

loaded = mlflow.pyfunc.load_model(f"runs:/{best['run_id']}/model")
sample = X.iloc[6:12]
print("\nreloaded model")
print("  normal order  ->", loaded.predict(sample))

# Columns arriving in a scrambled order: the signature puts them back by name.
print("  reversed cols ->", loaded.predict(sample[cols[::-1]]))

# A column missing entirely: the signature refuses to run at all.
try:
    loaded.predict(sample.drop(columns=["utilisation"]))
except Exception as err:
    print("  missing col   ->", type(err).__name__)

2026/08/20 14:08:08 INFO mlflow.store.db.utils: Creating initial MLflow database tables...
2026/08/20 14:08:08 INFO mlflow.store.db.utils: Updating database tables
2026/08/20 14:08:09 INFO mlflow.tracking.fluent: Experiment with name 'loan-default' does not exist. Creating a new experiment.
C=0.01   cv_accuracy=0.8767
C=0.1    cv_accuracy=0.8833
C=1.0    cv_accuracy=0.8867
C=10.0   cv_accuracy=0.8900

best run
  C           10.0
  cv_accuracy 0.89
  runs logged 4

reloaded model
  normal order  -> [1 1 0 1 0 0]
  reversed cols -> [1 1 0 1 0 0]
  missing col   -> MlflowException

The three timestamped INFO lines appear once, on the first run, while the database is created. Your timestamps will differ, and the four accuracy figures should match. Run it twice and runs logged becomes 8 — MLflow appends, it does not replace.

Read the last three lines again#

normal order and reversed cols produce identical predictions. The second call passed all six columns backwards. The stored signature matched them by name and put them back in place.

Compare that with the first lesson in this section, where the same mistake silently reversed a loan decision. The signature is a dozen extra characters and it removes an entire class of production bug.

missing col raises MlflowException. The full message is long: it prints the data it received and the schema it expected, side by side. It is loud, ugly and exactly right — the model refused rather than guessing.

See it in a browser#

mlflow ui --backend-store-uri sqlite:///mlflow.db

Open http://127.0.0.1:5000. You get a sortable table of runs, checkboxes to compare any two, and the stored files for each run. Stop it with Ctrl+C.

No output block for this one. The terminal prints a server banner whose contents vary by version, and the interesting result is a web page.

The bits of the API worth knowing#

mlflow.set_tracking_uri("sqlite:///mlflow.db") — the storage backend. Recent MLflow versions have retired the old ./mlruns folder store and will raise an error if you point at one, so start with SQLite. For a team, swap in a PostgreSQL URI and nothing else in your script changes.

with mlflow.start_run(): — the context manager marks the run finished even if your training crashes. A crashed run recorded as FAILED is far more useful than a missing row.

mlflow.autolog() — one line before training, and MLflow logs the parameters, metrics and model for supported libraries without any log_param calls. Convenient for exploring, and worth turning off for production pipelines where you want to control precisely what is recorded.

infer_signature(X, y_pred) — reads the column names and types out of your training frame. Pass a DataFrame, not a NumPy array, or you get an unnamed tensor schema and lose the name checking entirely.

runs:/<run_id>/model — a URI that resolves through the tracking store. It survives files being moved, which raw paths do not.

A note for older installs. The name= argument to log_model arrived in MLflow 3. On MLflow 2.x the same call is mlflow.sklearn.log_model(model, artifact_path="model", signature=...). Check with python -c "import mlflow; print(mlflow.__version__)" if the call raises a TypeError.

Common mistakes#

Logging the model without a signature. MLflow warns and carries on. You lose schema enforcement, which is most of the reason to use log_model instead of joblib.dump.

Committing mlflow.db and mlruns/ to git. They are outputs, and they grow fast. Add both to .gitignore and keep the tracking store somewhere shared instead.

Assuming a logged model is reproducible. MLflow records your Python package versions. It does not record your system libraries, your CUDA driver or your operating system. That gap is what Docker exists to close.

Using mlflow ui as a production server. It is a development server bound to localhost. A shared deployment needs mlflow server with a real database, an artifact store, and authentication in front of it.

One experiment for everything. Runs from three unrelated projects in one list is the same problem as one folder of files. Call set_experiment with a meaningful name per project.

Try it yourself#

Add mlflow.log_param("data_hash", ...) using the fingerprint function from the experiment tracking lesson. Then change one value in X, re-run, and sort by accuracy in the UI.

You will see two runs with the same settings and different scores. The fingerprint column is the only thing on the page that explains why.

What to learn next#

• Docker for ML — closing the reproducibility gap MLflow leaves open.
• Model serving — turning a logged model into something an app can call.
• CI/CD for machine learning — gating a promotion on tests that actually ran.