How-to Guides

Convert to Fabric in 5 minutes

Learn how to add Fabric to your PyTorch code

Scale your model with Accelerators

Take advantage of your hardware with a switch of a flag

Structure your Fabric code

Best practices for setting up your training script with Fabric

Launch distributed training

Launch a Python script on multiple devices and machines

Launch Fabric in a notebook

Launch on multiple devices from within a Jupyter notebook

Improve performance with Mixed-Precision training

Save memory and speed up training using mixed precision

Organize your model code with with LightningModule

Organize your code in a LightningModule and use it with Fabric

Encapsulate code into Callbacks

Make use of the Callback system in Fabric

Track and visualize experiments

Learn how Fabric helps you remove boilerplate code for tracking metrics with a logger

Save and load model progress

Efficient saving and loading of model weights, training state, hyperparameters and more.

Build your own Trainer

Take our Fabric Trainer template and customize it for your needs

Use efficient gradient accumulation

Learn how to perform efficient gradient accumulation in distributed settings

Distribute communication

Learn all about communication primitives for distributed operation. Gather, reduce, broadcast, etc.

Use multiple models and optimizers

See how flexible Fabric is to work with multiple models and optimizers!

Speed up models by compiling them

Use torch.compile to speed up models on modern hardware

Train models with billions of parameters

Train the largest models with FSDP/TP across multiple GPUs and machines

Initialize models efficiently

Reduce the time and peak memory usage for model initialization

Save and load very large models

Save and load very large models efficiently with distributed checkpoints