---
title: FSDPPrecision
description: Precision plugin for training with Fully Sharded Data Parallel (FSDP).
canonical: >-
  https://lightning.ai/docs/pytorch/stable/optional-api/api/lightning.pytorch.plugins.precision.FSDPPrecision
---
# FSDPPrecision {#fsdpprecision}

### `class lightning.pytorch.plugins.precision.FSDPPrecision(precision, scaler=None)` {#lightning.pytorch.plugins.precision.FSDPPrecision}

Bases: [`Precision`](/docs/pytorch/2.6.6/optional-api/api/lightning.pytorch.plugins.precision.Precision.md#lightning.pytorch.plugins.precision.Precision "lightning.pytorch.plugins.precision.precision.Precision")

Precision plugin for training with Fully Sharded Data Parallel (FSDP).

Warning

This is an [experimental](/docs/pytorch/2.6.6/reference/versioning.md#experimental-api) feature.

Parameters:

*   **precision** ([`Literal`](https://docs.python.org/3/library/typing.html#typing.Literal "(in Python v3.15)")\[`'32-true'`, `'16-true'`, `'bf16-true'`, `'16-mixed'`, `'bf16-mixed'`\]) – Full precision (32-true), half precision (16-true, bf16-true) or mixed precision (16-mixed, bf16-mixed).
    
*   **scaler** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[`ShardedGradScaler`\]) – An optional `torch.distributed.fsdp.sharded_grad_scaler.ShardedGradScaler` to use.
    

Raises:

[**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError "(in Python v3.15)") – If unsupported `precision` is provided.

#### `clip_grad_by_norm(*_, **__)` {#lightning.pytorch.plugins.precision.FSDPPrecision.clip_grad_by_norm}

Clip gradients by norm.

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

#### `convert_input(data)` {#lightning.pytorch.plugins.precision.FSDPPrecision.convert_input}

Convert model inputs (forward) to the floating point precision type of this plugin.

This is a no-op in the base precision plugin, since we assume the data already has the desired type (default is torch.float32).

Return type:

[`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")

#### `convert_module(module)` {#lightning.pytorch.plugins.precision.FSDPPrecision.convert_module}

Convert the module parameters to the precision type this plugin handles.

This is optional and depends on the precision limitations during optimization.

Return type:

[`Module`](https://docs.pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module "(in PyTorch v2.15)")

#### `convert_output(data)` {#lightning.pytorch.plugins.precision.FSDPPrecision.convert_output}

Convert outputs to the floating point precision type expected after model’s forward.

This is a no-op in the base precision plugin, since we assume the data already has the desired type (default is torch.float32).

Return type:

[`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")

#### `forward_context()` {#lightning.pytorch.plugins.precision.FSDPPrecision.forward_context}

A contextmanager for managing model forward/training\_step/evaluation\_step/predict\_step.

Return type:

[`AbstractContextManager`](https://docs.python.org/3/library/contextlib.html#contextlib.AbstractContextManager "(in Python v3.15)")

#### `load_state_dict(state_dict)` {#lightning.pytorch.plugins.precision.FSDPPrecision.load_state_dict}

Called when loading a checkpoint, implement to reload precision plugin state given precision plugin state\_dict.

Parameters:

**state\_dict** ([`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")\]) – the precision plugin state returned by `state_dict`.

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

#### `module_init_context()` {#lightning.pytorch.plugins.precision.FSDPPrecision.module_init_context}

Instantiate module parameters or tensors in the precision type this plugin handles.

This is optional and depends on the precision limitations during optimization.

Return type:

[`AbstractContextManager`](https://docs.python.org/3/library/contextlib.html#contextlib.AbstractContextManager "(in Python v3.15)")

#### `optimizer_step(optimizer, model, closure, **kwargs)` {#lightning.pytorch.plugins.precision.FSDPPrecision.optimizer_step}

Hook to run the optimizer step.

Return type:

[`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")

#### `pre_backward(tensor, module)` {#lightning.pytorch.plugins.precision.FSDPPrecision.pre_backward}

Runs before precision plugin executes backward.

Parameters:

*   **tensor** ([`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")) – The tensor that will be used for backpropagation
    
*   **module** ([`LightningModule`](/docs/pytorch/2.6.6/optional-api/api/lightning.pytorch.core.LightningModule.md#lightning.pytorch.core.LightningModule "lightning.pytorch.core.module.LightningModule")) – The module that was involved in producing the tensor and whose parameters need the gradients
    

Return type:

[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")

#### `state_dict()` {#lightning.pytorch.plugins.precision.FSDPPrecision.state_dict}

Called when saving a checkpoint, implement to generate precision plugin state\_dict.

Return type:

[`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")\]

Returns:

A dictionary containing precision plugin state.

#### `tensor_init_context()` {#lightning.pytorch.plugins.precision.FSDPPrecision.tensor_init_context}

Controls how tensors get created (device, dtype).

Return type:

[`AbstractContextManager`](https://docs.python.org/3/library/contextlib.html#contextlib.AbstractContextManager "(in Python v3.15)")

---

**Navigation**

- Parent: [Api](/docs/pytorch/2.6.6/optional-api/api.md)