---
title: TorchSyncBatchNorm
description: >-
  A plugin that wraps all batch normalization layers of a model with
  synchronization logic for multiprocessing.
canonical: >-
  https://lightning.ai/docs/pytorch/stable/optional-api/api/lightning.pytorch.plugins.TorchSyncBatchNorm
---
# TorchSyncBatchNorm {#torchsyncbatchnorm}

### `class lightning.pytorch.plugins.TorchSyncBatchNorm` {#lightning.pytorch.plugins.TorchSyncBatchNorm}

Bases: [`LayerSync`](/docs/pytorch/2.6.6/optional-api/api/lightning.pytorch.plugins.LayerSync.md#lightning.pytorch.plugins.LayerSync "lightning.pytorch.plugins.layer_sync.LayerSync")

A plugin that wraps all batch normalization layers of a model with synchronization logic for multiprocessing.

This plugin has no effect in single-device operation.

#### `apply(model)` {#lightning.pytorch.plugins.TorchSyncBatchNorm.apply}

Add global batchnorm for a model spread across multiple GPUs and nodes.

Override this method to synchronize batchnorm layers between specific process groups instead of the whole world.

Parameters:

**model** ([`Module`](https://docs.pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module "(in PyTorch v2.15)")) – Reference to the current LightningModule

Return type:

[`Module`](https://docs.pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module "(in PyTorch v2.15)")

Returns:

LightningModule with batchnorm layers synchronized within the process groups.

#### `revert(model)` {#lightning.pytorch.plugins.TorchSyncBatchNorm.revert}

Convert the wrapped batchnorm layers back to regular batchnorm layers.

Parameters:

**model** ([`Module`](https://docs.pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module "(in PyTorch v2.15)")) – Reference to the current LightningModule

Return type:

[`Module`](https://docs.pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module "(in PyTorch v2.15)")

Returns:

LightningModule with regular batchnorm layers that will no longer sync across processes.

---

**Navigation**

- Parent: [Api](/docs/pytorch/2.6.6/optional-api/api.md)