DDP2Strategy

class pytorch_lightning.strategies.DDP2Strategy(accelerator=None, parallel_devices=None, cluster_environment=None, checkpoint_io=None, precision_plugin=None, ddp_comm_state=None, ddp_comm_hook=None, ddp_comm_wrapper=None, model_averaging_period=None, process_group_backend=None, **kwargs)

Bases: pytorch_lightning.strategies.ddp.DDPStrategy

DDP2 behaves like DP in one node, but synchronization across nodes behaves like in DDP.

model_to_device()

Moves the model to the correct device.

reduce(collection, *args, **kwargs)

Reduces a collection of tensors from all processes. It can be applied to just a single tensor. In DDP2, the reduction here is only across local devices within the node.

Parameters

• collection (Union[Metric, Tensor, int, float, Mapping[str, Union[Metric, Tensor, int, float]]]) – The collection of tensors to sync and reduce.

• *args – ignored for DDP2

• **kwargs – ignored for DDP2

Return type

Union[Metric, Tensor, int, float, Mapping[str, Union[Metric, Tensor, int, float]]]

Returns

Reduced tensor values or the same value if it was not or did not contain a tensor.

property root_device

Return the root device.