DDPSpawnPlugin¶
- class pytorch_lightning.plugins.training_type.DDPSpawnPlugin(parallel_devices=None, num_nodes=None, cluster_environment=None, sync_batchnorm=None, ddp_comm_state=None, ddp_comm_hook=None, ddp_comm_wrapper=None, **kwargs)[source]¶
Bases:
pytorch_lightning.plugins.training_type.parallel.ParallelPlugin
Spawns processes using the
torch.multiprocessing.spawn()
method and joins processes after training finishes.- barrier(*args, **kwargs)[source]¶
Forces all possibly joined processes to wait for each other
- Return type
- reduce(tensor, group=None, reduce_op='mean')[source]¶
Reduces a tensor from several distributed processes to one aggregated tensor.
- Parameters
- Return type
- Returns
reduced value, except when the input was not a tensor the output remains is unchanged
- property root_device¶
Returns the root device