---
title: thunder.plugins.DDP
description: Plugin for enabling Distributed Data Parallel (DDP) training in Thunder.
canonical: >-
  https://lightning.ai/docs/thunder/stable/api-reference/generated/thunder.plugins.DDP
---
# thunder.plugins.DDP {#thunder-plugins-ddp}

### `class thunder.plugins.DDP(bucket_size_in_mb=25.0, broadcast_from=None, process_group=None)` {#thunder.plugins.DDP}

[\[source\]](https://github.com/Lightning-AI/lightning-thunder/blob/0.2.6/thunder/plugins/distributed.py#L13-L55)

Bases: `Plugin`

Plugin for enabling Distributed Data Parallel (DDP) training in Thunder.

This plugin applies the necessary transforms to bucket and synchronize gradients across multiple processes, using a specified process group for communication.

See [https://github.com/pytorch/pytorch/blob/v2.7.0/torch/nn/parallel/distributed.py#L326](https://github.com/pytorch/pytorch/blob/v2.7.0/torch/nn/parallel/distributed.py#L326) for more details.

Parameters:

*   **bucket\_size\_in\_mb** ([`float`](https://docs.python.org/3/builtins/functions.html#float "(in Python v3.15)")) – float, default 25.0 Size in megabytes of the gradient bucket in DDP.
    
*   **broadcast\_from** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")\]) – int | None, default None Global rank ID to broadcast model parameters from at initialization. If None, no explicit broadcast is performed.
    
*   **process\_group** – Optional\[ProcessGroup\], default is the current default process group
    
*   **bucket\_size\_in\_mb** ([_float_](https://docs.python.org/3/builtins/functions.html#float "(in Python v3.15)")) –
    
*   **broadcast\_from** ([_int_](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)") _|_ _None_) –
    

#### `__init__(bucket_size_in_mb=25.0, broadcast_from=None, process_group=None)` {#thunder.plugins.DDP.__init__}

[\[source\]](https://github.com/Lightning-AI/lightning-thunder/blob/0.2.6/thunder/plugins/distributed.py#L30-L41)

Parameters:

*   **bucket\_size\_in\_mb** ([_float_](https://docs.python.org/3/builtins/functions.html#float "(in Python v3.15)")) –
    
*   **broadcast\_from** ([_Union_](https://docs.python.org/3/library/typing.html#typing.Union "(in Python v3.15)")_\[__None__,_ [_int_](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")_\]_) –
    

Methods

- [`__init__([bucket_size_in_mb, ...])`](#thunder.plugins.DDP.__init__)
- `setup_executors()` — rtype:Optional[list[Executor]]
- `setup_lookasides()` — rtype:Optional[list[Lookaside]]
- [`setup_transforms()`](#thunder.plugins.DDP.setup_transforms) — Constructs the list of graph-level transforms.

Attributes

- `policy`

#### `setup_transforms()` {#thunder.plugins.DDP.setup_transforms}

[\[source\]](https://github.com/Lightning-AI/lightning-thunder/blob/0.2.6/thunder/plugins/distributed.py#L43-L55)

Constructs the list of graph-level transforms.

Returns:

list\[Transform\] contains the DDP transform over the process group.

---

**Navigation**

- Parent: [Generated](/docs/thunder/0.2.6/api-reference/generated.md)