---
title: Utilities
description: >-
  Function that sets the seed for pseudo-random number generators in: torch,
  numpy, and Python’s random module.

  In addition, sets the following environment variables:
canonical: 'https://lightning.ai/docs/fabric/stable/full-api-reference/api/utilities'
---
# Utilities {#utilities}

### `lightning.fabric.utilities.seed.seed_everything(seed=None, workers=False, verbose=True)` {#lightning.fabric.utilities.seed.seed_everything}

Function that sets the seed for pseudo-random number generators in: torch, numpy, and Python’s random module. In addition, sets the following environment variables:

*   `PL_GLOBAL_SEED`: will be passed to spawned subprocesses (e.g. ddp\_spawn backend).
    
*   `PL_SEED_WORKERS`: (optional) is set to 1 if `workers=True`.
    

Parameters:

*   **seed** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")\]) – the integer value seed for global random state in Lightning. If `None`, it will read the seed from `PL_GLOBAL_SEED` env variable. If `None` and the `PL_GLOBAL_SEED` env variable is not set, then the seed defaults to 0. If seed is not in bounds or cannot be cast to int, a ValueError is raised.
    
*   **workers** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")) – if set to `True`, will properly configure all dataloaders passed to the Trainer with a `worker_init_fn`. If the user already provides such a function for their dataloaders, setting this argument will have no influence. See also: [`pl_worker_init_function()`](#lightning.fabric.utilities.seed.pl_worker_init_function "lightning.fabric.utilities.seed.pl_worker_init_function").
    
*   **verbose** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")) – Whether to print a message on each rank with the seed being set.
    

Return type:

[`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")

### `lightning.fabric.utilities.seed.pl_worker_init_function(worker_id, rank=None)` {#lightning.fabric.utilities.seed.pl_worker_init_function}

The worker\_init\_fn that Lightning automatically adds to your dataloader if you previously set the seed with `seed_everything(seed, workers=True)`.

See also the PyTorch documentation on [randomness in DataLoaders](https://pytorch.org/docs/stable/notes/randomness.html#dataloader).

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

### `lightning.fabric.utilities.data.suggested_max_num_workers(local_world_size)` {#lightning.fabric.utilities.data.suggested_max_num_workers}

Suggests an upper bound of `num_workers` to use in a PyTorch [`DataLoader`](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader "(in PyTorch v2.15)") based on the number of CPU cores available on the system and the number of distributed processes in the current machine.

Parameters:

**local\_world\_size** ([`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")) – The number of distributed processes running on the current machine. Set this to the number of devices configured in Fabric/Trainer.

Return type:

[`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")

### `lightning.fabric.utilities.distributed.is_shared_filesystem(strategy, path=None, timeout=3)` {#lightning.fabric.utilities.distributed.is_shared_filesystem}

Checks whether the filesystem under the given path is shared across all processes.

This function should only be used in a context where distributed is initialized.

Parameters:

*   **strategy** ([`Strategy`](/docs/fabric/2.6.6/full-api-reference/generated/lightning.fabric.strategies.Strategy.md#lightning.fabric.strategies.Strategy "lightning.fabric.strategies.strategy.Strategy")) – The strategy being used, either from Fabric (`fabric.strategy`) or from Trainer (`trainer.strategy`).
    
*   **path** (`Union`\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`Path`](https://docs.python.org/3/library/pathlib.html#pathlib.Path "(in Python v3.15)"), [`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")\]) – The path to check. Defaults to the current working directory. The user must have permissions to write to this path or the parent folder, and the filesystem must be writable.
    
*   **timeout** ([`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")) – If any of the processes can’t list the file created by rank 0 within this many seconds, the filesystem is determined to be not shared.
    

Return type:

[`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")

### `lightning.fabric.utilities.warnings.disable_possible_user_warnings(module='')` {#lightning.fabric.utilities.warnings.disable_possible_user_warnings}

Ignore warnings of the category `PossibleUserWarning` from Lightning.

For more granular control over which warnings to ignore, use [`warnings.filterwarnings()`](https://docs.python.org/3/library/warnings.html#warnings.filterwarnings "(in Python v3.15)") directly.

Parameters:

**module** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")) – Name of the module for which the warnings should be ignored (e.g., `'lightning.pytorch.strategies'`). Default: Disables warnings from all modules.

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

### `lightning.fabric.utilities.throughput.measure_flops(model, forward_fn, loss_fn=None)` {#lightning.fabric.utilities.throughput.measure_flops}

Utility to compute the total number of FLOPs used by a module during training or during inference.

It’s recommended to create a meta-device model for this:

Example:

```python
with torch.device("meta"):
    model = MyModel()
    x = torch.randn(2, 32)

model_fwd = lambda: model(x)
fwd_flops = measure_flops(model, model_fwd)

model_loss = lambda y: y.sum()
fwd_and_bwd_flops = measure_flops(model, model_fwd, model_loss)
```

Parameters:

*   **model** ([`Module`](https://docs.pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module "(in PyTorch v2.15)")) – The model whose FLOPs should be measured.
    
*   **forward\_fn** ([`Callable`](https://docs.python.org/3/library/typing.html#typing.Callable "(in Python v3.15)")\[\[\], [`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")\]) – A function that runs `forward` on the model and returns the result.
    
*   **loss\_fn** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`Callable`](https://docs.python.org/3/library/typing.html#typing.Callable "(in Python v3.15)")\[\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")\], [`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")\]\]) – A function that computes the loss given the `forward_fn` output. If provided, the loss and backward FLOPs will be included in the result.
    

Return type:

[`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")

### `class lightning.fabric.utilities.data.AttributeDict` {#lightning.fabric.utilities.data.AttributeDict}

Bases: [`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict "(in Python v3.15)")

A container to store state variables of your program.

This is a drop-in replacement for a Python dictionary, with the additional functionality to access and modify keys through attribute lookup for convenience.

Use this to define the state of your program, then pass it to [`save()`](/docs/fabric/2.6.6/full-api-reference/generated/lightning.fabric.fabric.Fabric.md#lightning.fabric.fabric.Fabric.save "lightning.fabric.fabric.Fabric.save") and [`load()`](/docs/fabric/2.6.6/full-api-reference/generated/lightning.fabric.fabric.Fabric.md#lightning.fabric.fabric.Fabric.load "lightning.fabric.fabric.Fabric.load").

Example

```python
>>> import torch
>>> model = torch.nn.Linear(2, 2)
>>> state = AttributeDict(model=model, iter_num=0)
>>> state.model
Linear(in_features=2, out_features=2, bias=True)
>>> state.iter_num += 1
>>> state.iter_num
1
>>> state
"iter_num": 1
"model":    Linear(in_features=2, out_features=2, bias=True)
```

### `class lightning.fabric.utilities.throughput.ThroughputMonitor(fabric, **kwargs)` {#lightning.fabric.utilities.throughput.ThroughputMonitor}

Bases: [`Throughput`](#lightning.fabric.utilities.throughput.Throughput "lightning.fabric.utilities.throughput.Throughput")

Computes throughput.

This class will automatically keep a count of the number of log calls (`step`). But that can be modified as desired. For manual logging, using [`Throughput`](#lightning.fabric.utilities.throughput.Throughput "lightning.fabric.utilities.throughput.Throughput") directly might be desired.

Example:

```python
logger = ...
fabric = Fabric(logger=logger)
throughput = ThroughputMonitor(fabric)
t0 = time()
for i in range(1, 100):
    do_work()
    if torch.cuda.is_available(): torch.cuda.synchronize()  # required or else time() won't be correct
    throughput.update(time=time() - t0, batches=i, samples=i)
    if i % 10 == 0:
        throughput.compute_and_log(step=i)
```

Parameters:

*   **fabric** ([`Fabric`](/docs/fabric/2.6.6/full-api-reference/generated/lightning.fabric.fabric.Fabric.md#lightning.fabric.fabric.Fabric "lightning.fabric.fabric.Fabric")) – The Fabric object.
    
*   **\*\*kwargs** ([`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")) – See available parameters in [`Throughput`](#lightning.fabric.utilities.throughput.Throughput "lightning.fabric.utilities.throughput.Throughput")
    

#### `compute_and_log(step=None, **kwargs)` {#lightning.fabric.utilities.throughput.ThroughputMonitor.compute_and_log}

See [`Throughput.compute()`](#lightning.fabric.utilities.throughput.Throughput.compute "lightning.fabric.utilities.throughput.Throughput.compute")

Parameters:

*   **step** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")\]) – Can be used to override the logging step.
    
*   **\*\*kwargs** ([`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")) – See available parameters in [`Throughput.compute()`](#lightning.fabric.utilities.throughput.Throughput.compute "lightning.fabric.utilities.throughput.Throughput.compute")
    

Return type:

[`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), `Union`\[[`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)"), [`float`](https://docs.python.org/3/builtins/functions.html#float "(in Python v3.15)")\]\]

### `class lightning.fabric.utilities.throughput.Throughput(available_flops=None, world_size=1, window_size=100, separator='/')` {#lightning.fabric.utilities.throughput.Throughput}

Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object "(in Python v3.15)")

Computes throughput.

| Key | Value |
| --- | --- |
| 
batches\_per\_sec | Rolling average (over `window_size` most recent updates) of the number of batches

processed per second







 |
| 

samples\_per\_sec | Rolling average (over `window_size` most recent updates) of the number of samples

processed per second







 |
| 

items\_per\_sec | Rolling average (over `window_size` most recent updates) of the number of items

processed per second







 |
| 

flpps\_per\_sec | Rolling average (over `window_size` most recent updates) of the number of flops

processed per second







 |
| device/batches\_per\_sec | batches\_per\_sec divided by world size |
| device/samples\_per\_sec | samples\_per\_sec divided by world size |
| device/items\_per\_sec | items\_per\_sec divided by world size. This may include padding depending on the data |
| device/flops\_per\_sec | flops\_per\_sec divided by world size. |
| device/mfu | device/flops\_per\_sec divided by world size. |
| time | Total elapsed time |
| batches | Total batches seen |
| samples | Total samples seen |
| lengths | Total items seen |

Example:

```python
throughput = Throughput()
t0 = time()
for i in range(1000):
    do_work()
    if torch.cuda.is_available(): torch.cuda.synchronize()  # required or else time() won't be correct
    throughput.update(time=time() - t0, samples=i)
    if i % 10 == 0:
        print(throughput.compute())
```

Notes

*   The implementation assumes that devices FLOPs are all the same as it normalizes by the world size and only takes a single `available_flops` value.
    
*   items\_per\_sec, flops\_per\_sec and MFU do not account for padding if present. We suggest using samples\_per\_sec or batches\_per\_sec to measure throughput under this circumstance.
    

Parameters:

*   **available\_flops** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`float`](https://docs.python.org/3/builtins/functions.html#float "(in Python v3.15)")\]) – Number of theoretical flops available for a single device.
    
*   **world\_size** ([`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")) – Number of devices available across hosts. Global metrics are not included if the world size is 1.
    
*   **window\_size** ([`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")) – Number of batches to use for a rolling average.
    
*   **separator** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")) – Key separator to use when creating per-device and global metrics.
    

#### `compute()` {#lightning.fabric.utilities.throughput.Throughput.compute}

Compute throughput metrics.

Return type:

[`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), `Union`\[[`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)"), [`float`](https://docs.python.org/3/builtins/functions.html#float "(in Python v3.15)")\]\]

#### `update(*, time, batches, samples, lengths=None, flops=None)` {#lightning.fabric.utilities.throughput.Throughput.update}

Update throughput metrics.

Parameters:

*   **time** ([`float`](https://docs.python.org/3/builtins/functions.html#float "(in Python v3.15)")) – Total elapsed time in seconds. It should monotonically increase by the iteration time with each call.
    
*   **batches** ([`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")) – Total batches seen per device. It should monotonically increase with each call.
    
*   **samples** ([`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")) – Total samples seen per device. It should monotonically increase by the batch size with each call.
    
*   **lengths** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")\]) – Total length of the samples seen. It should monotonically increase by the lengths of a batch with each call.
    
*   **flops** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")\]) – Flops elapased per device since last `update()` call. You can easily compute this by using [`measure_flops()`](#lightning.fabric.utilities.throughput.measure_flops "lightning.fabric.utilities.throughput.measure_flops") and multiplying it by the number of batches that have been processed. The value might be different in each device if the batch size is not the same.
    

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

---

**Navigation**

- Parent: [Api](/docs/fabric/2.6.6/full-api-reference/api.md)