Supercharge training (intermediate)¶
Enable training features¶
Enable advanced training features using Trainer arguments. These are SOTA techniques that are automatically integrated into your training loop without changes to your code.
# train 1T+ parameter models with DeepSpeed/FSDP trainer = Trainer( devices=4, accelerator="gpu", strategy="deepspeed_stage_2", precision="16-mixed", ) # 20+ helpful arguments for rapid idea iteration trainer = Trainer( max_epochs=10, min_epochs=5, overfit_batches=1 ) # access the latest state of the art techniques trainer = Trainer(callbacks=[StochasticWeightAveraging(...)])
Extend the Trainer¶
If you have multiple lines of code with similar functionalities, you can use callbacks to easily group them together and toggle all of those lines on or off at the same time.
trainer = Trainer(callbacks=[AWSCheckpoints()])