EarlyStopping can't access learning rate logs from LearningRateMonitor

callbacks
1

Hi everybody,

new to Lightning so this question might sound a little naive (sorry about that).
I have a training script performing a straight image classification task.
I’m using a ReduceLROnPlateau scheduler and I would like to stop the training if the learning rate reaches a minimum value.
To do that, I tried inserting an EarlyStopping(monitor='lr', mode='min', stopping_threshold=min_lr_value) callback, logging the learning rate with a LearningRateMonitor.
However, the training fails after the first epoch with the following error:

RuntimeError: Early stopping conditioned on metric `lr` which is not available. Pass in or modify your `EarlyStopping` callback to use any of the following: `train_loss`, `val_loss`

The learning rate gets correctly logged on Wandb though with the lr nametag.
Am I missing something? Or do I have to explicitly log each metric of interest for the EarlyStopping callback to be able to monitor it?
Thanks for any help you can give me!

2

The learning rate monitor callback logs the values directly to the logger. It doesn’t register it as a “monitored” value in Lightning. So the EarlyStopping can’t check it.

To make this work, you could self.log("lr", learning_rate) the learning rate yourself and then reference it in the EarlyStopping callback. This is probably the easiest way.

Your use case could also be a good feature request for lightning.

3

Hi @awaelchli, thanks for the reply!
I guess logging the learning rate myself can do the trick for now, but I agree that this feature might come in handy for someone else.
Does Lightning have a specific guideline for feature requests? Like opening an issue/pull request on GitHub or here on the forum?
Thank you again for your help!

4

Yes, a feature request on GitHub would be the best :smiley:, using the FeatureRequest button here.

It contains a simple form with some basic questions.

5

@awaelchli I made a feature request following your suggestion.
Thanks again for the help!

1 Like