Epoch

An epoch is one full pass of your whole training dataset through the model.

Revising for an exam is the same idea. Reading the textbook once, cover to cover, is one epoch. Reading it three times is three epochs. The first pass gives you the shape of the subject; the second fixes the parts you skimmed; somewhere after that you stop learning and start reciting the book back word for word.

Inside one epoch, the model does not update once. It works through the data in batches, and each batch causes one weight update.

50,000 examples ÷ batch size 100 = 500 updates in one epoch
10 epochs                        = 5,000 updates in total

How many epochs is right#

There is no fixed answer, and the honest method is to watch rather than guess. Track the loss on data the model does not train on. While validation loss falls, keep going. When training loss keeps falling but validation loss turns upward, the model has moved from learning to memorising — that is overfitting, and the epoch immediately before the turn is the model you want.

epoch:      1     3     5     7     9    11
train  :  0.92  0.61  0.44  0.31  0.22  0.15
valid  :  0.95  0.66  0.52  0.49  0.53  0.61
                                ▲
                        best model is here

Early stopping automates that: keep a copy of the best weights so far, and end training when validation loss has not improved for a set number of epochs. Fine-tuning a large pretrained model usually needs very few epochs — often one to three — because the model already knows most of what it needs, and more passes mostly erase that general knowledge.

Where to go next#

• Full lesson: How neural networks learn
• Related terms: batch-size, overfitting, gradient-descent, hyperparameter