---
title: deepspeed
description: >-
  Convert ZeRO 2 or 3 checkpoint into a single fp32 consolidated state_dict file
  that can be loaded with

  torch.load(file) + load_state_dict() and used for training without DeepSpeed.
  It gets copied into
canonical: >-
  https://lightning.ai/docs/pytorch/stable/optional-api/api/lightning.pytorch.utilities.deepspeed
---
# deepspeed {#deepspeed}

Functions

- [`convert_zero_checkpoint_to_fp32_state_dict`](#lightning.pytorch.utilities.deepspeed.convert_zero_checkpoint_to_fp32_state_dict) — Convert ZeRO 2 or 3 checkpoint into a single fp32 consolidated state_dict file that can be loaded with torch.load(file) + load_state_dict() and used for training without DeepSpeed.
- `ds_checkpoint_dir` — rtype:str

Utilities that can be used with Deepspeed.

### `lightning.pytorch.utilities.deepspeed.convert_zero_checkpoint_to_fp32_state_dict(checkpoint_dir, output_file, tag=None)` {#lightning.pytorch.utilities.deepspeed.convert_zero_checkpoint_to_fp32_state_dict}

Convert ZeRO 2 or 3 checkpoint into a single fp32 consolidated `state_dict` file that can be loaded with `torch.load(file)` + `load_state_dict()` and used for training without DeepSpeed. It gets copied into the top level checkpoint dir, so the user can easily do the conversion at any point in the future. Once extracted, the weights don’t require DeepSpeed and can be used in any application. Additionally the script has been modified to ensure we keep the lightning state inside the state dict for being able to run `` LightningModule.load_from_checkpoint('...')` ``.

Parameters:

*   **checkpoint\_dir** (`Union`\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`Path`](https://docs.python.org/3/library/pathlib.html#pathlib.Path "(in Python v3.15)")\]) – path to the desired checkpoint folder. (one that contains the tag-folder, like `global_step14`)
    
*   **output\_file** (`Union`\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`Path`](https://docs.python.org/3/library/pathlib.html#pathlib.Path "(in Python v3.15)")\]) – path to the pytorch fp32 state\_dict output file (e.g. path/pytorch\_model.bin)
    
*   **tag** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\]) – checkpoint tag used as a unique identifier for checkpoint. If not provided will attempt to load tag in the file named `latest` in the checkpoint folder, e.g., `global_step14`
    

Return type:

[`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")\]

Examples:

```python
# Lightning deepspeed has saved a directory instead of a file
convert_zero_checkpoint_to_fp32_state_dict(
    "lightning_logs/version_0/checkpoints/epoch=0-step=0.ckpt/",
    "lightning_model.pt"
)
```

---

**Navigation**

- Parent: [Api](/docs/pytorch/2.6.6/optional-api/api.md)