Shortcuts

SingleDeviceStrategy

class lightning_fabric.strategies.SingleDeviceStrategy(device='cpu', accelerator=None, checkpoint_io=None, precision=None)[source]

Bases: lightning_fabric.strategies.strategy.Strategy

Strategy that handles communication on a single device.

all_gather(tensor, group=None, sync_grads=False)[source]

Perform a all_gather on all processes.

Return type

Tensor

all_reduce(tensor, *args, **kwargs)[source]

Reduces a tensor from several distributed processes to one aggregated tensor. As this plugin only operates with a single device, the reduction is simply the identity.

Parameters
  • tensor (Union[Any, Tensor]) – the tensor to sync and reduce

  • *args – ignored

  • **kwargs – ignored

Return type

Union[Any, Tensor]

Returns

the unmodified input as reduction is not needed for single process operation

barrier(*args, **kwargs)[source]

Synchronizes all processes which blocks processes until the whole group enters this function.

Parameters

name – an optional name to pass into barrier.

Return type

None

broadcast(obj, src=0)[source]

Broadcasts an object to all processes.

Parameters
  • obj (TypeVar(TBroadcast)) – the object to broadcast

  • src (int) – source rank

Return type

TypeVar(TBroadcast)

module_to_device(module)[source]

Moves the model to the correct device.

Return type

None

property is_global_zero: bool

Whether the current process is the rank zero process not only on the local node, but for all nodes.

Return type

bool

property root_device: torch.device

Returns the root device.

Return type

device


© Copyright Copyright (c) 2018-2023, Lightning AI et al...

Built with Sphinx using a theme provided by Read the Docs.