Resume from checkpoint with elastic training

4

I see. You could do something like this:

CHECKPOINT = "hardcode/path/to/checkpoint.ckpt"

def train(model):
    if Path(CHECKPOINT).exists():
        trainer = pl.Trainer.resume_from_checkpoint(CHECKPOINT)
    else:
        checkpoint_callback = ModelCheckpoint(filepath=CHECKPOINT)
        trainer = pl.Trainer(callbacks=[checkpoint_callback])
    trainer.fit(model)

I agree that this does seem a little clunky; I will think if there is a better approach.