Unit 6.1 – Model Checkpointing and Early Stopping

Code

• Part 2 & 3: 6.1-checkpointing

What we covered in this video lecture

In this lecture, we will explore the concept of early stopping. Previously, we trained a neural network for a given number of epochs and then kept only the last model, assuming the last model corresponds to the “best trained” one. However, if our model starts overfitting, it’s typically not the last model that is the best model.

In this lecture, we will monitor the validation set accuracy during training and create model checkpoints of the best model during training. The best model checkpoint — the model with the highest validation set accuracy — is then selected for test set evaluation. Note that even though we call this concept “early stopping,” we are not literally stopping the model training early. Instead, we are training the model for the same number of epochs as we usually would, but instead of sticking to the last model, we select the “best” model.

Additional resources if you want to learn more

If you are interested in literally stopping the training early, check out the [EarlyStopping documentation](https://pytorch-lightning.readthedocs.io/en/stable/common/early_stopping.html. The EarlyStopping callback stops the training early given a certain patience parameter — i.e., a patience of 5 could be set such that the training stops if the predictive performance doesn’t improve within a span of 5 epochs.

Watch Video 1 Mark complete and go to Unit 6.2 →