DataParallelStrategy¶
- class pytorch_lightning.strategies.DataParallelStrategy(accelerator=None, parallel_devices=None, checkpoint_io=None, precision_plugin=None)[source]¶
Bases:
pytorch_lightning.strategies.parallel.ParallelStrategy
Implements data-parallel training in a single process, i.e., the model gets replicated to each device and each gets a split of the data.
- barrier(*args, **kwargs)[source]¶
Synchronizes all processes which blocks processes until the whole group enters this function.
- batch_to_device(batch, device=None, dataloader_idx=0)[source]¶
Moves the batch to the correct device.
The input and the output is the same type.
- reduce(collection, group=None, reduce_op='mean')[source]¶
Reduces a collection of tensors from all processes. It can be applied to just a single tensor.
- Parameters:
- Return type:
TypeVar
(TReduce
)- Returns:
Reduced tensor values or the same value if it was not or did not contain a tensor.
- reduce_boolean_decision(decision)[source]¶
Reduce a boolean decision across all processes.
- Return type:
- training_step(*args, **kwargs)[source]¶
The actual training step.
See
training_step()
for more details
- validation_step(*args, **kwargs)[source]¶
The actual validation step.
See
validation_step()
for more details
- property root_device: torch.device¶
Return the root device.