I don’t understand how to resume the training (from the last checkpoint). The following: trainer = pl.Trainer(gpus=1, default_root_dir=save_dir) saves but does not resume from the last checkpoint. The following code starts the training from scratch (but I read that it should resume): logger = T…

How to resume training

goku November 30, 2020, 7:39pm 4

I don’t think that’s possible since a new Trainer instance won’t have any info regarding the checkpoint state saved in the previous training.

Fast Forward or Rewind Training

Topic		Replies	Views
Resume from checkpoint with elastic training	5	3338	September 16, 2020
Resume_from_checkpoint not work Trainer	4	7072	December 7, 2022
Training stuck on resume Trainer	1	961	May 31, 2023
How to use `LightningCLI` to start training from a checkpoint at epoch 0? Trainer	0	936	February 19, 2022
Error resuming from checkpoint with multiple GPUs	3	3140	January 30, 2022