---
title: Advanced skills
description: >-
  - Use efficient gradient accumulation — Learn how to perform efficient
  gradient accumulation in distributed settings - Distribute communication —
  Learn...
canonical: 'https://lightning.ai/docs/fabric/stable/get-started-in-steps/advanced'
---
# Advanced skills {#advanced-skills}

- [Use efficient gradient accumulation](/docs/fabric/2.6.6/reference/advanced/gradient_accumulation.md) — Learn how to perform efficient gradient accumulation in distributed settings

- [Distribute communication](/docs/fabric/2.6.6/reference/advanced/distributed_communication.md) — Learn all about communication primitives for distributed operation. Gather, reduce, broadcast, etc.

- [Use multiple models and optimizers](/docs/fabric/2.6.6/reference/advanced/multiple_setup.md) — See how flexible Fabric is to work with multiple models and optimizers!

- [Speed up models by compiling them](/docs/fabric/2.6.6/reference/advanced/compile.md) — Use torch.compile to speed up models on modern hardware

- [Train models with billions of parameters](/docs/fabric/2.6.6/reference/advanced/model_parallel.md) — Train the largest models with FSDP/TP across multiple GPUs and machines

- [Save and load very large models](/docs/fabric/2.6.6/reference/guide/checkpoint/distributed_checkpoint.md) — Save and load very large models efficiently with distributed checkpoints

---

**Navigation**

- Parent: [Get started in steps](/docs/fabric/2.6.6/get-started-in-steps.md)