SingleDeviceStrategy
- class pytorch_lightning.strategies.SingleDeviceStrategy(device='cpu', accelerator=None, checkpoint_io=None, precision_plugin=None)[source]
Bases:
pytorch_lightning.strategies.strategy.Strategy
Strategy that handles communication on a single device.
- all_gather(tensor, group=None, sync_grads=False)[source]
Perform a all_gather on all processes.
- Return type
- barrier(*args, **kwargs)[source]
Synchronizes all processes which blocks processes until the whole group enters this function.
- Parameters
- Return type
- broadcast(obj, src=0)[source]
Broadcasts an object to all processes.
- reduce(tensor, *args, **kwargs)[source]
Reduces a tensor from several distributed processes to one aggregated tensor. As this plugin only operates with a single device, the reduction is simply the identity.
- Parameters
tensor (Any | torch.Tensor) – the tensor to sync and reduce
- Return type
Any | torch.Tensor
- Returns
the unmodified input as reduction is not needed for single process operation
- setup(trainer)[source]
Setup plugins for the trainer fit and creates optimizers.
- teardown()[source]
This method is called to teardown the training process.
It is the right place to release memory and free other resources.
- Return type
- property is_global_zero: bool
Whether the current process is the rank zero process not only on the local node, but for all nodes.
- Return type
- property root_device: torch.device
Returns the root device.
- Return type