How-to Guides

Basics

Convert to Fabric in 5 minutes

Learn how to add Fabric to your PyTorch code

basic

Scale your model with Accelerators

Take advantage of your hardware with a switch of a flag

basic

Structure your Fabric code

Best practices for setting up your training script with Fabric

basic

Launch distributed training

Launch a Python script on multiple devices and machines

basic

Launch Fabric in a notebook

Launch on multiple devices from within a Jupyter notebook

basic

Improve performance with Mixed-Precision training

Save memory and speed up training using mixed precision

basic

Build your own Trainer

Organize your model code with with LightningModule

Organize your code in a LightningModule and use it with Fabric

intermediate

Encapsulate code into Callbacks

Make use of the Callback system in Fabric

intermediate

Track and visualize experiments

Learn how Fabric helps you remove boilerplate code for tracking metrics with a logger

intermediate

Save and load model progress

Efficient saving and loading of model weights, training state, hyperparameters and more.

intermediate

Build your own Trainer

Take our Fabric Trainer template and customize it for your needs

intermediate

Advanced Topics

Use efficient gradient accumulation

Learn how to perform efficient gradient accumulation in distributed settings

advanced

Distribute communication

Learn all about communication primitives for distributed operation. Gather, reduce, broadcast, etc.

advanced

Use multiple models and optimizers

See how flexible Fabric is to work with multiple models and optimizers!

advanced

Speed up models by compiling them

Use torch.compile to speed up models on modern hardware

advanced

Train models with billions of parameters

Train the largest models with FSDP/TP across multiple GPUs and machines

advanced

Initialize models efficiently

Reduce the time and peak memory usage for model initialization

advanced

Save and load very large models

Save and load very large models efficiently with distributed checkpoints

advanced