Models Agents Infrastructure Pricing Docs Blog Security

Models Agents Infrastructure Pricing Docs Blog Security

Lightning Fabric

Platform

Lightning AIDevelop, train and deploy on Lightning AI Python SDKProgrammatic access to the Lightning AI platform CLIAccess the Lightning AI platform from the command line

Open Source Libraries

PyTorch LightningFinetune and pretrain AI models. Focus on science, not engineering.Lightning FabricScale foundation models to 1000s of GPUs with expert-level control TorchMetrics90+ PyTorch metrics, optimized for distributed training ThunderCompiler for PyTorch: Make PyTorch models faster!LitLoggerExperiment tracking built into Lightning AI LitDataStream and process datasets at scale LitServeEasily serve AI models — Lightning fast LitAILLM router and minimal agent framework in one LitGPT20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

Stable (2.6.6)

Home

Convert PyTorch code to Fabric

Accelerate your code with Fabric

Get started in steps

Basic skills Intermediate skills Advanced skills

Full API Reference

Fabric Arguments Fabric Methods Accelerators Collectives Environments Fabric IO Loggers Precision Strategies Utilities

Accelerator CPUAccelerator CUDAAccelerator MPSAccelerator XLAAccelerator Fabric CSVLogger Logger TensorBoardLogger Collective SingleDeviceCollective TorchCollective lightning.fabric.plugins.environments.cluster_environment.ClusterEnvironment lightning.fabric.plugins.environments.kubeflow.KubeflowEnvironment lightning.fabric.plugins.environments.lightning.LightningEnvironment lightning.fabric.plugins.environments.lsf.LSFEnvironment lightning.fabric.plugins.environments.mpi.MPIEnvironment lightning.fabric.plugins.environments.slurm.SLURMEnvironment lightning.fabric.plugins.environments.torchelastic.TorchElasticEnvironment lightning.fabric.plugins.environments.xla.XLAEnvironment CheckpointIO TorchCheckpointIO XLACheckpointIO BitsandbytesPrecision DeepSpeedPrecision DoublePrecision FSDPPrecision HalfPrecision MixedPrecision Precision TransformerEnginePrecision XLAPrecision DataParallelStrategy DDPStrategy DeepSpeedStrategy FSDPStrategy ModelParallelStrategy ParallelStrategy SingleDeviceStrategy SingleDeviceXLAStrategy Strategy XLAFSDPStrategy XLAStrategy

More

Style Guide Examples Glossary How-tos

Reference

Speed up models by compiling them Communication between distributed processes Efficient Gradient Accumulation Efficient initialization

Training models with billions of parameters with FSDP Tensor Parallel 2D Parallelism

Multiple Models and Optimizers

Launch distributed training Fabric in Notebooks Save memory with mixed precision

Saving and Loading Checkpoints Saving and Loading Distributed Checkpoints

Organize Your Code

LitLogger Weights and Biases

Track and Visualize Experiments

Bare Bones Cluster Run single or multi-node on Lightning Studios Other Cluster Environments Run on a SLURM Managed Cluster

Docs/Stable (2.6.6)/Full API Reference/API/Collectives

Collectives

Warning

This is an experimental feature.

Collectives

Collective — Interface for collective operations.
TorchCollective — Collective operations using torch.distributed.
SingleDeviceCollective — Support for collective operations on a single device (no-op).

Was this page helpful?

Thanks for the feedback.

What went wrong?

Inaccurate or out of dateHard to understandMissing informationHard to find things / poorly organizedCode sample didn't workSomething is broken (link, image, layout)

On this page

Collectives

Share with AI

Open in ChatGPT Open in Claude

Was this helpful?

Thanks for the feedback.

What went wrong?

Inaccurate or out of dateHard to understandMissing informationHard to find things / poorly organizedCode sample didn't workSomething is broken (link, image, layout)

↑↓ Navigate↵ Open⌘↵ New tabesc Close