Deep Learning Section 031
Optimisers, Schedulers and the Training Loop
Everything that surrounds the model: how weights are updated, how the learning rate moves, and how you know it is working.
9 of 9 lessons published Three reading levels on every lesson
Lessons in order
Work top to bottom. Each lesson assumes the one above it.
- Choosing between SGD, Adam and AdamW
- Parameter groups and per-layer learning rates
- Learning rate schedulers
- Finding a learning rate that works
- Logits, softmax and picking the right loss
- Writing a validation loop that reports the truth
- TorchMetrics
- Early stopping and keeping the best model
- PyTorch Lightning