---
title: Strategy
description: >-
  Base class for all strategies that change the behaviour of the training,
  validation and test- loop.
canonical: >-
  https://lightning.ai/docs/fabric/stable/full-api-reference/api/generated/lightning.fabric.strategies.Strategy
---
# Strategy {#strategy}

### `class lightning.fabric.strategies.Strategy(accelerator=None, checkpoint_io=None, precision=None)` {#lightning.fabric.strategies.Strategy}

Bases: [`ABC`](https://docs.python.org/3/library/abc.html#abc.ABC "(in Python v3.15)")

Base class for all strategies that change the behaviour of the training, validation and test- loop.

#### `_configure_launcher()` {#lightning.fabric.strategies.Strategy._configure_launcher}

Attach the launcher based on Strategy.

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

#### `abstract all_gather(tensor, group=None, sync_grads=False)` {#lightning.fabric.strategies.Strategy.all_gather}

Perform an all\_gather on all processes.

Parameters:

*   **tensor** ([`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")) – the tensor to all\_gather
    
*   **group** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")\]) – the process group to gather results from
    
*   **sync\_grads** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")) – flag that allows users to synchronize gradients for all\_gather op
    

Return type:

[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")

#### `abstract all_reduce(tensor, group=None, reduce_op='mean')` {#lightning.fabric.strategies.Strategy.all_reduce}

Reduces the given tensor (e.g. across GPUs/processes).

Parameters:

*   **tensor** (`Union`\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)"), [`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")\]) – the tensor to sync and reduce
    
*   **group** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")\]) – the process group to reduce
    
*   **reduce\_op** (`Union`\[[`ReduceOp`](https://docs.pytorch.org/docs/stable/distributed.html#torch.distributed.ReduceOp "(in PyTorch v2.15)"), [`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")\]) – the reduction operation. Defaults to ‘mean’. Can also be a string ‘sum’ or ReduceOp.
    

Return type:

`Union`\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)"), [`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")\]

#### `backward(tensor, module, *args, **kwargs)` {#lightning.fabric.strategies.Strategy.backward}

Forwards backward-calls to the precision plugin.

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

#### `abstract barrier(name=None)` {#lightning.fabric.strategies.Strategy.barrier}

Synchronizes all processes which blocks processes until the whole group enters this function.

Parameters:

**name** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\]) – an optional name to pass into barrier.

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

#### `batch_to_device(batch, device=None)` {#lightning.fabric.strategies.Strategy.batch_to_device}

Moves the batch to the correct device.

The returned batch is of the same type as the input batch, just having all tensors on the correct device.

Parameters:

*   **batch** ([`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")) – The batch of samples to move to the correct device
    
*   **device** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`device`](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.device "(in PyTorch v2.15)")\]) – The target device
    

Return type:

[`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")

#### `abstract broadcast(obj, src=0)` {#lightning.fabric.strategies.Strategy.broadcast}

Broadcasts an object to all processes.

Parameters:

*   **obj** ([`TypeVar`](https://docs.python.org/3/library/typing.html#typing.TypeVar "(in Python v3.15)")(`TBroadcast`)) – the object to broadcast
    
*   **src** ([`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")) – source rank
    

Return type:

[`TypeVar`](https://docs.python.org/3/library/typing.html#typing.TypeVar "(in Python v3.15)")(`TBroadcast`)

#### `clip_gradients_norm(module, optimizer, max_norm, norm_type=2.0, error_if_nonfinite=True)` {#lightning.fabric.strategies.Strategy.clip_gradients_norm}

Clip gradients by norm.

Return type:

[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")

#### `clip_gradients_value(module, optimizer, clip_val)` {#lightning.fabric.strategies.Strategy.clip_gradients_value}

Clip gradients by value.

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

#### `get_module_state_dict(module)` {#lightning.fabric.strategies.Strategy.get_module_state_dict}

Returns model state.

Return type:

[`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), `Union`\[[`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)"), [`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")\]\]

#### `get_optimizer_state(optimizer)` {#lightning.fabric.strategies.Strategy.get_optimizer_state}

Returns state of an optimizer.

Allows for syncing/collating optimizer state from processes in custom plugins.

Return type:

[`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")\]

#### `load_checkpoint(path, state=None, strict=True, weights_only=None)` {#lightning.fabric.strategies.Strategy.load_checkpoint}

Load the contents from a checkpoint and restore the state of the given objects.

Parameters:

*   **path** (`Union`\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`Path`](https://docs.python.org/3/library/pathlib.html#pathlib.Path "(in Python v3.15)")\]) – A path to where the file is located
    
*   **state** (`Union`\[[`Module`](https://docs.pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module "(in PyTorch v2.15)"), [`Optimizer`](https://docs.pytorch.org/docs/stable/optim.html#torch.optim.Optimizer "(in PyTorch v2.15)"), [`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), `Union`\[[`Module`](https://docs.pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module "(in PyTorch v2.15)"), [`Optimizer`](https://docs.pytorch.org/docs/stable/optim.html#torch.optim.Optimizer "(in PyTorch v2.15)"), [`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")\]\], [`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")\]) –
    
    Can be one of:
    
    *   A dictionary of objects whose state will be restored in-place from the checkpoint path.
        
    *   `None` or the empty dict: The loaded checkpoint will be returned in full.
        
    *   A [`Module`](https://docs.pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module "(in PyTorch v2.15)") instance, if the checkpoint file contains a raw module state dict.
        
    *   A [`Optimizer`](https://docs.pytorch.org/docs/stable/optim.html#torch.optim.Optimizer "(in PyTorch v2.15)") instance, if the checkpoint file contains a raw optimizer state.
        
    
*   **strict** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")) – Whether to enforce that the keys in state match the keys in the checkpoint.
    
*   **weights\_only** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")\]) – Defaults to `None`. If `True`, restricts loading to `state_dicts` of plain `torch.Tensor` and other primitive types. If loading a checkpoint from a trusted source that contains an `nn.Module`, use `weights_only=False`. If loading checkpoint from an untrusted source, we recommend using `weights_only=True`. For more information, please refer to the [PyTorch Developer Notes on Serialization Semantics](https://docs.pytorch.org/docs/main/notes/serialization.html#id3).
    

Return type:

[`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")\]

Returns:

The remaining items that were not restored into the given state dictionary. If no state dictionary is given, the full checkpoint will be returned.

#### `load_module_state_dict(module, state_dict, strict=True)` {#lightning.fabric.strategies.Strategy.load_module_state_dict}

Loads the given state into the model.

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

#### `module_init_context(empty_init=None)` {#lightning.fabric.strategies.Strategy.module_init_context}

A context manager wrapping the model instantiation.

Here, the strategy can control how the parameters of the model get created (device, dtype) and or apply other patches to the model.

Parameters:

**empty\_init** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")\]) – Whether to initialize the model with empty weights (uninitialized memory). If `None`, the strategy will decide. Some strategies may not support all options.

Return type:

[`AbstractContextManager`](https://docs.python.org/3/library/contextlib.html#contextlib.AbstractContextManager "(in Python v3.15)")

#### `abstract module_to_device(module)` {#lightning.fabric.strategies.Strategy.module_to_device}

Moves the model to the correct device.

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

#### `optimizer_step(optimizer, **kwargs)` {#lightning.fabric.strategies.Strategy.optimizer_step}

Performs the actual optimizer step.

Parameters:

*   **optimizer** (`Optimizable`) – the optimizer performing the step
    
*   **\*\*kwargs** ([`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")) – Any extra arguments to `optimizer.step`
    

Return type:

[`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")

#### `process_dataloader(dataloader)` {#lightning.fabric.strategies.Strategy.process_dataloader}

Wraps the dataloader if necessary.

Parameters:

**dataloader** ([`DataLoader`](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader "(in PyTorch v2.15)")) – iterable. Ideally of type: [`torch.utils.data.DataLoader`](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader "(in PyTorch v2.15)")

Return type:

[`DataLoader`](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader "(in PyTorch v2.15)")

#### `reduce_boolean_decision(decision, all=True)` {#lightning.fabric.strategies.Strategy.reduce_boolean_decision}

Reduce a boolean decision across all processes.

Return type:

[`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")

#### `save_checkpoint(path, state, storage_options=None, filter=None)` {#lightning.fabric.strategies.Strategy.save_checkpoint}

Save model, optimizer, and other state as a checkpoint file.

Parameters:

*   **path** (`Union`\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`Path`](https://docs.python.org/3/library/pathlib.html#pathlib.Path "(in Python v3.15)")\]) – A path to where the file(s) should be saved
    
*   **state** ([`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), `Union`\[[`Module`](https://docs.pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module "(in PyTorch v2.15)"), [`Optimizer`](https://docs.pytorch.org/docs/stable/optim.html#torch.optim.Optimizer "(in PyTorch v2.15)"), [`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")\]\]) – A dictionary with contents to be saved. If the dict contains modules or optimizers, their state-dict will be retrieved and converted automatically.
    
*   **storage\_options** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")\]) – Additional options for the `CheckpointIO` plugin
    
*   **filter** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`Callable`](https://docs.python.org/3/library/typing.html#typing.Callable "(in Python v3.15)")\[\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")\], [`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")\]\]\]) – An optional dictionary containing filter callables that return a boolean indicating whether the given item should be saved (`True`) or filtered out (`False`). Each filter key should match a state key, where its filter will be applied to the `state_dict` generated.
    

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

#### `setup_environment()` {#lightning.fabric.strategies.Strategy.setup_environment}

Setup any processes or distributed connections.

This must be called by the framework at the beginning of every process, before any distributed communication takes place.

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

#### `setup_module(module)` {#lightning.fabric.strategies.Strategy.setup_module}

Performs setup for the model, e.g., by wrapping it by another class.

Return type:

[`Module`](https://docs.pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module "(in PyTorch v2.15)")

#### `setup_module_and_optimizers(module, optimizers, scheduler=None)` {#lightning.fabric.strategies.Strategy.setup_module_and_optimizers}

Set up a model and multiple optimizers together.

The returned objects are expected to be in the same order they were passed in. The default implementation will call [`setup_module()`](#lightning.fabric.strategies.Strategy.setup_module "lightning.fabric.strategies.Strategy.setup_module") and [`setup_optimizer()`](#lightning.fabric.strategies.Strategy.setup_optimizer "lightning.fabric.strategies.Strategy.setup_optimizer") on the inputs.

Return type:

[`tuple`](https://docs.python.org/3/builtins/stdtypes.html#tuple "(in Python v3.15)")\[[`Module`](https://docs.pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module "(in PyTorch v2.15)"), [`list`](https://docs.python.org/3/builtins/stdtypes.html#list "(in Python v3.15)")\[[`Optimizer`](https://docs.pytorch.org/docs/stable/optim.html#torch.optim.Optimizer "(in PyTorch v2.15)")\], [`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[`_LRScheduler`\]\]

#### `setup_optimizer(optimizer)` {#lightning.fabric.strategies.Strategy.setup_optimizer}

Performs setup for the optimizer, e.g., by wrapping it by another class.

Return type:

[`Optimizer`](https://docs.pytorch.org/docs/stable/optim.html#torch.optim.Optimizer "(in PyTorch v2.15)")

#### `teardown()` {#lightning.fabric.strategies.Strategy.teardown}

This method is called to teardown the training process.

It is the right place to release memory and free other resources.

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

#### `tensor_init_context()` {#lightning.fabric.strategies.Strategy.tensor_init_context}

Controls how tensors get created (device, dtype).

Return type:

[`AbstractContextManager`](https://docs.python.org/3/library/contextlib.html#contextlib.AbstractContextManager "(in Python v3.15)")

#### `abstract property is_global_zero: bool` {#lightning.fabric.strategies.Strategy.is_global_zero}

Whether the current process is the rank zero process not only on the local node, but for all nodes.

#### `abstract property root_device: device` {#lightning.fabric.strategies.Strategy.root_device}

Returns the root device.

---

**Navigation**

- Parent: [Generated](/docs/fabric/2.6.6/full-api-reference/api/generated.md)