---
title: DDPStrategy
description: Strategy for multi-process single-device training on one or multiple nodes.
canonical: >-
  https://lightning.ai/docs/pytorch/stable/optional-api/api/lightning.pytorch.strategies.DDPStrategy
---
# DDPStrategy {#ddpstrategy}

### `class lightning.pytorch.strategies.DDPStrategy(accelerator=None, parallel_devices=None, cluster_environment=None, checkpoint_io=None, precision_plugin=None, ddp_comm_state=None, ddp_comm_hook=None, ddp_comm_wrapper=None, model_averaging_period=None, process_group_backend=None, timeout=datetime.timedelta(seconds=1800), start_method='popen', **kwargs)` {#lightning.pytorch.strategies.DDPStrategy}

Bases: [`ParallelStrategy`](/docs/pytorch/2.5.5/optional-api/api/lightning.pytorch.strategies.ParallelStrategy.md#lightning.pytorch.strategies.ParallelStrategy "lightning.pytorch.strategies.parallel.ParallelStrategy")

Strategy for multi-process single-device training on one or multiple nodes.

#### `barrier(*args, **kwargs)` {#lightning.pytorch.strategies.DDPStrategy.barrier}

Synchronizes all processes which blocks processes until the whole group enters this function.

Parameters:

**name** – an optional name to pass into barrier.

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

#### `broadcast(obj, src=0)` {#lightning.pytorch.strategies.DDPStrategy.broadcast}

Broadcasts an object to all processes.

Parameters:

*   **obj** ([`TypeVar`](https://docs.python.org/3/library/typing.html#typing.TypeVar "(in Python v3.15)")(`TBroadcast`)) – the object to broadcast
    
*   **src** ([`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")) – source rank
    

Return type:

[`TypeVar`](https://docs.python.org/3/library/typing.html#typing.TypeVar "(in Python v3.15)")(`TBroadcast`)

#### `model_to_device()` {#lightning.pytorch.strategies.DDPStrategy.model_to_device}

Moves the model to the correct device.

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

#### `on_exception(exception)` {#lightning.pytorch.strategies.DDPStrategy.on_exception}

Called when the trainer execution is interrupted by an exception.

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

#### `optimizer_step(optimizer, closure, model=None, **kwargs)` {#lightning.pytorch.strategies.DDPStrategy.optimizer_step}

Performs the actual optimizer step.

Parameters:

*   **optimizer** ([`Optimizer`](https://docs.pytorch.org/docs/stable/optim.html#torch.optim.Optimizer "(in PyTorch v2.15)")) – the optimizer performing the step
    
*   **closure** ([`Callable`](https://docs.python.org/3/library/typing.html#typing.Callable "(in Python v3.15)")\[\[\], [`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")\]) – closure calculating the loss value
    
*   **model** (`Union`\[[`LightningModule`](/docs/pytorch/2.5.5/optional-api/api/lightning.pytorch.core.LightningModule.md#lightning.pytorch.core.LightningModule "lightning.pytorch.core.module.LightningModule"), [`Module`](https://docs.pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module "(in PyTorch v2.15)"), [`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")\]) – reference to the model, optionally defining optimizer step related hooks
    
*   **\*\*kwargs** ([`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")) – Any extra arguments to `optimizer.step`
    

Return type:

[`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")

#### `pre_backward(closure_loss)` {#lightning.pytorch.strategies.DDPStrategy.pre_backward}

Run before precision plugin executes backward.

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

#### `reduce(tensor, group=None, reduce_op='mean')` {#lightning.pytorch.strategies.DDPStrategy.reduce}

Reduces a tensor from several distributed processes to one aggregated tensor.

Parameters:

*   **tensor** ([`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")) – the tensor to sync and reduce
    
*   **group** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")\]) – the process group to gather results from. Defaults to all processes (world)
    
*   **reduce\_op** (`Union`\[[`ReduceOp`](https://docs.pytorch.org/docs/stable/distributed.html#torch.distributed.ReduceOp "(in PyTorch v2.15)"), [`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")\]) – the reduction operation. Defaults to ‘mean’/’avg’. Can also be a string ‘sum’ to calculate the sum during reduction.
    

Return type:

[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")

Returns:

reduced value, except when the input was not a tensor the output remains is unchanged

#### `setup(trainer)` {#lightning.pytorch.strategies.DDPStrategy.setup}

Sets up the accelerator, plugins and initializes the optimizers (if needed).

Parameters:

**trainer** ([`Trainer`](/docs/pytorch/2.5.5/optional-api/api/lightning.pytorch.trainer.trainer.Trainer.md#lightning.pytorch.trainer.trainer.Trainer "lightning.pytorch.trainer.trainer.Trainer")) – the trainer instance

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

#### `setup_environment()` {#lightning.pytorch.strategies.DDPStrategy.setup_environment}

Setup any processes or distributed connections.

This is called before the LightningModule/DataModule setup hook which allows the user to access the accelerator environment before setup is complete.

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

#### `teardown()` {#lightning.pytorch.strategies.DDPStrategy.teardown}

This method is called to teardown the training process.

It is the right place to release memory and free other resources.

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

#### `property is_distributed: bool` {#lightning.pytorch.strategies.DDPStrategy.is_distributed}

Legacy property kept for backwards compatibility.

#### `property root_device: device` {#lightning.pytorch.strategies.DDPStrategy.root_device}

Return the root device.

---

**Navigation**

- Parent: [Api](/docs/pytorch/2.5.5/optional-api/api.md)