I am using huggingface with Pytorch lightning and and I am saving the model with Model_checkpoint method. It saves the file as .ckpt. I want to load the model using huggingface method .from_pretrained(), but I would get the warning the all of the layers are reinitialized ( I renamed my file to pytorch_model.bin) . Any idea how to correctly save the model in order to be re-used using the .from_pretrained() methods.
Hey, I am no specialist how huggingface saves their model, but our checkpointing mechanism does not just store the model’s weights but also some hyperparameters, optimizer and trainer state.
If you want to load with HF, you probably need to save only the weights (e.g. using the weights_only flag of the ModelCheckpoint callback.
Note however, that this won’t allow you to resume training properly as the trainer and optimizer state are not stored and also that this does not give you any guarantee to be compatible with HF.