SingleDeviceStrategy¶
- class lightning.pytorch.strategies.SingleDeviceStrategy(device='cpu', accelerator=None, checkpoint_io=None, precision_plugin=None)[source]¶
Bases:
Strategy
Strategy that handles communication on a single device.
- all_gather(tensor, group=None, sync_grads=False)[source]¶
Perform a all_gather on all processes.
- Return type:
Tensor
- barrier(*args, **kwargs)[source]¶
Synchronizes all processes which blocks processes until the whole group enters this function.
- reduce(tensor, *args, **kwargs)[source]¶
Reduces a tensor from several distributed processes to one aggregated tensor. Since this strategy only operates with a single device, the reduction is simply the identity.