---
title: ParallelStrategy
description: Strategy for training with multiple processes in parallel.
canonical: >-
  https://lightning.ai/docs/fabric/stable/full-api-reference/api/generated/lightning.fabric.strategies.ParallelStrategy
---
# ParallelStrategy {#parallelstrategy}

### `class lightning.fabric.strategies.ParallelStrategy(accelerator=None, parallel_devices=None, cluster_environment=None, checkpoint_io=None, precision=None)` {#lightning.fabric.strategies.ParallelStrategy}

Bases: [`Strategy`](/docs/fabric/2.6.6/full-api-reference/api/generated/lightning.fabric.strategies.Strategy.md#lightning.fabric.strategies.Strategy "lightning.fabric.strategies.strategy.Strategy"), [`ABC`](https://docs.python.org/3/library/abc.html#abc.ABC "(in Python v3.15)")

Strategy for training with multiple processes in parallel.

#### `all_gather(tensor, group=None, sync_grads=False)` {#lightning.fabric.strategies.ParallelStrategy.all_gather}

Perform a all\_gather on all processes.

Return type:

[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")

#### `reduce_boolean_decision(decision, all=True)` {#lightning.fabric.strategies.ParallelStrategy.reduce_boolean_decision}

Reduces a boolean decision over distributed processes. By default is analogous to `all` from the standard library, returning `True` only if all input decisions evaluate to `True`. If `all` is set to `False`, it behaves like `any` instead.

Parameters:

*   **decision** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")) – A single input decision.
    
*   **all** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")) – Whether to logically emulate `all` or `any`. Defaults to True.
    

Returns:

The reduced boolean decision.

Return type:

[bool](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")

#### `teardown()` {#lightning.fabric.strategies.ParallelStrategy.teardown}

This method is called to teardown the training process.

It is the right place to release memory and free other resources.

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

#### `property distributed_sampler_kwargs: Optional[dict[str, Any]]` {#lightning.fabric.strategies.ParallelStrategy.distributed_sampler_kwargs}

Arguments for the `DistributedSampler`.

If this method is not defined, or it returns `None`, then the `DistributedSampler` will not be used.

#### `property is_global_zero: bool` {#lightning.fabric.strategies.ParallelStrategy.is_global_zero}

Whether the current process is the rank zero process not only on the local node, but for all nodes.

---

**Navigation**

- Parent: [Generated](/docs/fabric/2.6.6/full-api-reference/api/generated.md)