Take-home assignments that pass

What reviewers score#

Typical rubric weights, from companies that publish them and reviewers who talk:

Weights vary; the ordering rarely does. Note where "model choice" sits: inside that 20% framing row, at best.

The structure that passes#

churn-takehome/
├── README.md            ← findings FIRST, run instructions second
├── requirements.txt     ← pinned versions, e.g. pandas==2.2.3
├── data/                ← the given file, or a download script — never 500 MB in git
├── notebooks/
│   └── exploration.ipynb   ← the messy truth, labelled as exploration
├── src/
│   ├── prepare.py       ← raw data in, clean table out
│   ├── train.py         ← fits and saves the model, seed fixed
│   └── evaluate.py      ← THE file the reviewer runs
└── report.md            ← one page: decisions, results, what you would do next

The README opens with three sentences of findings — "churn concentrates in month-one users; a two-feature model reaches 0.81 AUC against a 0.74 baseline; the strongest lever is activation, not retention offers" — then one command:

pip install -r requirements.txt
python src/evaluate.py

The one file the reviewer runs#

Shrunk to twelve hand-typed rows so the mechanics are visible, this is the shape of evaluate.py — baseline printed first, always:

# The reviewer runs ONE file. Make it this one, and print the baseline first.

data = [  # (hours_active, orders_last_month, churned) — 12 rows, enough to show the shape
    (1, 0, 1), (2, 0, 1), (1, 1, 1), (3, 0, 1), (2, 1, 1),
    (9, 4, 0), (12, 5, 0), (8, 3, 0), (15, 6, 0), (11, 2, 0),
    (4, 1, 1), (7, 3, 0),
]

train, test = data[:9], data[9:]     # real submissions split by time or id, never after peeking

churn_rate = sum(r[2] for r in train) / len(train)
majority = round(churn_rate)          # baseline: predict the commonest training label

def rule_model(hours, orders):
    return 1 if hours < 5 else 0      # one explainable rule; the bar any model must beat

def accuracy(rows, predict):
    hits = sum(predict(h, o) == y for h, o, y in rows)
    return hits / len(rows)

print(f"majority-class baseline : {accuracy(test, lambda h, o: majority):.2f}")
print(f"one-rule model          : {accuracy(test, rule_model):.2f}")

majority-class baseline : 0.33
one-rule model          : 1.00

And here is the honesty the round rewards: that 1.00 means nothing — the test set has three rows. A real submission says so in exactly those words, next to the number, and reports a metric fit for the class balance, per model evaluation. Writing "perfect score, but n=3, so this is noise" is worth more than the score itself.

The submission checklist#

Run this before sending, every time:

• Fresh clone, fresh environment, one command — does it run end to end?
• Is every dependency pinned to a version in requirements.txt?
• Does a baseline appear before any model, and does the report compare against it?
• Was every preprocessing step (scaling, encoding, imputation) fitted on training data only? See train-test split.
• If rows have timestamps, is the split by time, with the test set entirely in the future?
• Does the metric fit the class balance — no accuracy bragging on imbalanced data?
• Does the README lead with findings, not with "clone this repo"?
• Did you state your time spent, and what you would do with one more day?
• Is there any file the reviewer does not need? Delete it.

Common mistakes#

Leakage. The classic: fitting the scaler on all rows before splitting, so test statistics leak into training. The temporal version: random-splitting time-stamped data, letting the model peek at the future. Reviewers hunt for these first, and either one usually ends the review.

No baseline. An accuracy in a vacuum is uninterpretable. The majority class, or last month's value, takes five lines and makes every other number mean something.

The 40-cell notebook as the deliverable. Exploration is honest work — keep it, labelled. The deliverable is scripts plus a one-page report. A notebook the reviewer must run top-to-bottom, hoping cell 23 was not executed out of order, is a coin flip.

Over-engineering. A deep network on 800 rows, a config framework, six abstractions — these read as poor judgement, not skill. Match the tool to the data size; XGBoost or simpler wins most take-home-sized tables.

Blowing the time box. Asked for four hours, spending twenty is a negative signal with its own name in review discussions. State your hours in the README and cut scope, not honesty, to fit them.

Silent failure handling. Dropping 12% of rows without a sentence explaining why leaves the reviewer assuming the worst. Every discarded row and filled gap gets one line — see handling missing data.

Try it yourself#

Take any old project and give it the twenty-five-minute treatment: hand the repo to a friend, start a timer, and watch silently. Note where they get stuck in the first five minutes — that friction, not your model, is what a reviewer would have scored.

What to learn next#

• ML case-study interviews — the live version of the thinking a take-home tests.
• Experiment tracking — the professional habit behind reproducible submissions.
• Spam detector project — a small end-to-end build to practise the whole checklist on.