SingleDeviceStrategy¶

class pytorch_lightning.strategies.SingleDeviceStrategy(device='cpu', accelerator=None, checkpoint_io=None, precision_plugin=None)[source]¶

Bases: pytorch_lightning.strategies.strategy.Strategy

Strategy that handles communication on a single device.

all_gather(tensor, group=None, sync_grads=False)[source]¶

Perform a all_gather on all processes.

Return type: torch.Tensor

barrier(*args, **kwargs)[source]¶

Synchronizes all processes which blocks processes until the whole group enters this function.

Parameters: name¶ – an optional name to pass into barrier.
Return type: None

broadcast(obj, src=0)[source]¶

Broadcasts an object to all processes.

Parameters

obj¶ (object) – the object to broadcast
src¶ (int) – source rank

Return type

object

model_to_device()[source]¶

Moves the model to the correct device.

Return type: None

reduce(tensor, *args, **kwargs)[source]¶

Reduces a tensor from several distributed processes to one aggregated tensor. As this plugin only operates with a single device, the reduction is simply the identity.

Parameters

tensor¶ (Any | torch.Tensor) – the tensor to sync and reduce
*args¶ – ignored
**kwargs¶ – ignored

Return type

Any | torch.Tensor

Returns

the unmodified input as reduction is not needed for single process operation

setup(trainer)[source]¶

Setup plugins for the trainer fit and creates optimizers.

Parameters: trainer¶ (Trainer) – the trainer instance
Return type: None

teardown()[source]¶

This method is called to teardown the training process.

It is the right place to release memory and free other resources.

Return type: None

property is_global_zero: bool¶

Whether the current process is the rank zero process not only on the local node, but for all nodes.

Return type: bool

property root_device: torch.device¶

Returns the root device.

Return type: device