---
title: Strategy
description: >-
  Strategy controls the model distribution across training, evaluation, and
  prediction to be used by the Trainer. It can be controlled by passing
  different
canonical: 'https://lightning.ai/docs/pytorch/stable/reference/extensions/strategy'
---
# Strategy {#strategy}

Strategy controls the model distribution across training, evaluation, and prediction to be used by the [Trainer](/docs/pytorch/2.6.6/core-api/trainer.md). It can be controlled by passing different
strategy with aliases (`"ddp"`, `"ddp_spawn"`, `"deepspeed"` and so on) as well as a custom strategy to the `strategy` parameter for Trainer.

The Strategy in PyTorch Lightning handles the following responsibilities:

* Launch and teardown of training processes (if applicable).
* Setup communication between processes (NCCL, GLOO, MPI, and so on).
* Provide a unified communication interface for reduction, broadcast, and so on.
* Owns the `LightningModule`
* Handles/owns optimizers and schedulers.

Strategy is a composition of one [Accelerator](/docs/pytorch/2.6.6/reference/extensions/accelerator.md), one [Precision Plugin](/docs/pytorch/2.6.6/reference/extensions/plugins.md#precision-plugins), a [CheckpointIO](/docs/pytorch/2.6.6/reference/extensions/plugins.md#checkpointio-plugins)
plugin and other optional plugins such as the [ClusterEnvironment](/docs/pytorch/2.6.6/reference/extensions/plugins.md#cluster-environments).

![Illustration of the Strategy as a composition of the Accelerator and several plugins](https://pl-public-data.s3.amazonaws.com/docs/static/images/strategies/overview.jpeg)

We expose Strategies mainly for expert users that want to extend Lightning for new hardware support or new distributed backends (e.g. a backend not yet supported by [PyTorch](https://pytorch.org/docs/stable/distributed.html#backends) itself).

## Selecting a Built-in Strategy {#selecting-a-built-in-strategy}

Built-in strategies can be selected in two ways.

1. Pass the shorthand name to the `strategy` Trainer argument
2. Import a Strategy from `lightning.pytorch.strategies`, instantiate it and pass it to the `strategy` Trainer argument

The latter allows you to configure further options on the specific strategy.
Here are some examples:

```python
# Training with the DistributedDataParallel strategy on 4 GPUs
trainer = Trainer(strategy="ddp", accelerator="gpu", devices=4)

# Training with the DistributedDataParallel strategy on 4 GPUs, with options configured
trainer = Trainer(strategy=DDPStrategy(static_graph=True), accelerator="gpu", devices=4)

# Training with the DDP Spawn strategy using auto accelerator selection
trainer = Trainer(strategy="ddp_spawn", accelerator="auto", devices=4)

# Training with the DeepSpeed strategy on available GPUs
trainer = Trainer(strategy="deepspeed", accelerator="gpu", devices="auto")

# Training with the DDP strategy using 3 CPU processes
trainer = Trainer(strategy="ddp", accelerator="cpu", devices=3)

# Training with the DDP Spawn strategy on 8 TPU cores
trainer = Trainer(strategy="ddp_spawn", accelerator="tpu", devices=8)
```

The below table lists all relevant strategies available in Lightning with their corresponding short-hand name:

| Name | Class | Description |
| --- | --- | --- |
| fsdp | `FSDPStrategy` | Strategy for Fully Sharded Data Parallel training. [Learn more.](/docs/pytorch/2.6.6/reference/advanced/model_parallel/fsdp.md) |
| ddp | `DDPStrategy` | Strategy for multi-process single-device training on one or multiple nodes. [Learn more.](/docs/pytorch/2.6.6/reference/accelerators/gpu_intermediate.md#distributed-data-parallel) |
| ddp_spawn | `DDPStrategy` | Same as "ddp" but launches processes using `torch.multiprocessing.spawn` method and joins processes after training finishes. [Learn more.](/docs/pytorch/2.6.6/reference/accelerators/gpu_intermediate.md#distributed-data-parallel-spawn) |
| deepspeed | `DeepSpeedStrategy` | Provides capabilities to run training using the DeepSpeed library, with training optimizations for large billion parameter models. [Learn more.](/docs/pytorch/2.6.6/reference/advanced/model_parallel/deepspeed.md) |
| xla | `XLAStrategy` | Strategy for training on multiple TPU devices using the `torch_xla.distributed.xla_multiprocessing.spawn` method. [Learn more.](/docs/pytorch/2.6.6/reference/accelerators/tpu.md) |
| single_xla | `SingleXLAStrategy` | Strategy for training on a single XLA device, like TPUs. [Learn more.](/docs/pytorch/2.6.6/reference/accelerators/tpu.md) |

## Third-party Strategies {#third-party-strategies}

There are powerful third-party strategies that integrate well with Lightning but aren't maintained as part of the `lightning` package.
Checkout the gallery over [here](/docs/pytorch/2.6.6/reference/integrations/strategies.md).

## Create a Custom Strategy {#create-a-custom-strategy}

Every strategy in Lightning is a subclass of one of the main base classes: `Strategy`, `SingleDeviceStrategy` or `ParallelStrategy`.

![Strategy base classes](https://pl-public-data.s3.amazonaws.com/docs/static/images/strategies/hierarchy.jpeg)

As an expert user, you may choose to extend either an existing built-in Strategy or create a completely new one by
subclassing the base classes.

```python
from lightning.pytorch.strategies import DDPStrategy

class CustomDDPStrategy(DDPStrategy):
    def configure_ddp(self):
        self.model = MyCustomDistributedDataParallel(
            self.model,
            device_ids=...,
        )

    def setup(self, trainer):
        # you can access the accelerator and plugins directly
        self.accelerator.setup()
        self.precision_plugin.connect(...)
```

The custom strategy can then be passed into the `Trainer` directly via the `strategy` parameter.

```python
# custom strategy
trainer = Trainer(strategy=CustomDDPStrategy())
```

Since the strategy also hosts the Accelerator and various plugins, you can customize all of them to work together as you like:

```python
# custom strategy, with new accelerator and plugins
accelerator = MyAccelerator()
precision_plugin = MyPrecisionPlugin()
strategy = CustomDDPStrategy(accelerator=accelerator, precision_plugin=precision_plugin)
trainer = Trainer(strategy=strategy)
```

---

**Navigation**

- Parent: [Extensions](/docs/pytorch/2.6.6/reference/extensions.md)