---
title: GradientAccumulationScheduler
description: Change gradient accumulation factor according to scheduling.
canonical: >-
  https://lightning.ai/docs/pytorch/stable/optional-api/api/lightning.pytorch.callbacks.GradientAccumulationScheduler
---
# GradientAccumulationScheduler {#gradientaccumulationscheduler}

### `class lightning.pytorch.callbacks.GradientAccumulationScheduler(scheduling)` {#lightning.pytorch.callbacks.GradientAccumulationScheduler}

Bases: [`Callback`](/docs/pytorch/2.6.6/optional-api/api/lightning.pytorch.callbacks.Callback.md#lightning.pytorch.callbacks.Callback "lightning.pytorch.callbacks.callback.Callback")

Change gradient accumulation factor according to scheduling.

Parameters:

**scheduling** ([`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict "(in Python v3.15)")\[[`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)"), [`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")\]) – scheduling in format {epoch: accumulation\_factor}

Note

The argument scheduling is a dictionary. Each key represent an epoch and its associated accumulation factor value. Warning: Epoch are zero-indexed c.f it means if you want to change the accumulation factor after 4 epochs, set `Trainer(accumulate_grad_batches={4: factor})` or `GradientAccumulationScheduler(scheduling={4: factor})`. For more info check the example below.

Raises:

*   [**TypeError**](https://docs.python.org/3/builtins/exceptions.html#TypeError "(in Python v3.15)") – If `scheduling` is an empty `dict`, or not all keys and values of `scheduling` are integers.
    
*   [**IndexError**](https://docs.python.org/3/builtins/exceptions.html#IndexError "(in Python v3.15)") – If `minimal_epoch` is less than 0.
    

Example:

```python
>>> from lightning.pytorch import Trainer
>>> from lightning.pytorch.callbacks import GradientAccumulationScheduler

# from epoch 5, it starts accumulating every 2 batches. Here we have 4 instead of 5
# because epoch (key) should be zero-indexed.
>>> accumulator = GradientAccumulationScheduler(scheduling={4: 2})
>>> trainer = Trainer(callbacks=[accumulator])
```

#### `on_train_epoch_start(trainer, *_)` {#lightning.pytorch.callbacks.GradientAccumulationScheduler.on_train_epoch_start}

Called when the train epoch begins.

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

#### `on_train_start(trainer, pl_module)` {#lightning.pytorch.callbacks.GradientAccumulationScheduler.on_train_start}

Performns a configuration validation before training starts and raises errors for incompatible settings.

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

---

**Navigation**

- Parent: [Api](/docs/pytorch/2.6.6/optional-api/api.md)