---
title: BackboneFinetuning
description: Finetune a backbone model based on a learning rate user-defined scheduling.
canonical: >-
  https://lightning.ai/docs/pytorch/stable/optional-api/api/lightning.pytorch.callbacks.BackboneFinetuning
---
# BackboneFinetuning {#backbonefinetuning}

### `class lightning.pytorch.callbacks.BackboneFinetuning(unfreeze_backbone_at_epoch=10, lambda_func=<function multiplicative>, backbone_initial_ratio_lr=0.1, backbone_initial_lr=None, should_align=True, initial_denom_lr=10.0, train_bn=True, verbose=False, rounding=12)` {#lightning.pytorch.callbacks.BackboneFinetuning}

Bases: [`BaseFinetuning`](/docs/pytorch/2.6.6/optional-api/api/lightning.pytorch.callbacks.BaseFinetuning.md#lightning.pytorch.callbacks.BaseFinetuning "lightning.pytorch.callbacks.finetuning.BaseFinetuning")

Finetune a backbone model based on a learning rate user-defined scheduling.

When the backbone learning rate reaches the current model learning rate and `should_align` is set to True, it will align with it for the rest of the training.

Parameters:

*   **unfreeze\_backbone\_at\_epoch** ([`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")) – Epoch at which the backbone will be unfreezed.
    
*   **lambda\_func** ([`Callable`](https://docs.python.org/3/library/typing.html#typing.Callable "(in Python v3.15)")) – Scheduling function for increasing backbone learning rate.
    
*   **backbone\_initial\_ratio\_lr** ([`float`](https://docs.python.org/3/builtins/functions.html#float "(in Python v3.15)")) – Used to scale down the backbone learning rate compared to rest of model
    
*   **backbone\_initial\_lr** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`float`](https://docs.python.org/3/builtins/functions.html#float "(in Python v3.15)")\]) – Optional, Initial learning rate for the backbone. By default, we will use `current_learning /  backbone_initial_ratio_lr`
    
*   **should\_align** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")) – Whether to align with current learning rate when backbone learning reaches it.
    
*   **initial\_denom\_lr** ([`float`](https://docs.python.org/3/builtins/functions.html#float "(in Python v3.15)")) – When unfreezing the backbone, the initial learning rate will `current_learning_rate /  initial_denom_lr`.
    
*   **train\_bn** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")) – Whether to make Batch Normalization trainable.
    
*   **verbose** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")) – Display current learning rate for model and backbone
    
*   **rounding** ([`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")) – Precision for displaying learning rate
    

Example:

```python
>>> import torch
>>> import torch.nn as nn
>>> from lightning.pytorch import LightningModule, Trainer
>>> from lightning.pytorch.callbacks import BackboneFinetuning
>>> import torchvision.models as models
>>>
>>> class TransferLearningModel(LightningModule):
...     def __init__(self, num_classes=10):
...         super().__init__()
...         # REQUIRED: Your model must have a 'backbone' attribute
...         self.backbone = models.resnet50(weights=None)
...         # Remove the final classification layer from backbone
...         self.backbone = nn.Sequential(*list(self.backbone.children())[:-1])
...
...         # Add your task-specific head
...         self.head = nn.Sequential(
...             nn.Flatten(),
...             nn.Linear(2048, 512),
...             nn.ReLU(),
...             nn.Linear(512, num_classes)
...         )
...
...     def forward(self, x):
...         # Extract features with backbone
...         features = self.backbone(x)
...         # Classify with head
...         return self.head(features)
...
...     def configure_optimizers(self):
...         # Initially only optimize the head - backbone will be added by callback
...         return torch.optim.Adam(self.head.parameters(), lr=1e-3)
...
>>> # Setup the callback
>>> multiplicative = lambda epoch: 1.5
>>> backbone_finetuning = BackboneFinetuning(
...     unfreeze_backbone_at_epoch=10,  # Start unfreezing at epoch 10
...     lambda_func=multiplicative,     # Gradually increase backbone LR
...     backbone_initial_ratio_lr=0.1,  # Start backbone at 10% of head LR
... )
>>> model = TransferLearningModel()
>>> trainer = Trainer(callbacks=[backbone_finetuning])
```

#### `finetune_function(pl_module, epoch, optimizer)` {#lightning.pytorch.callbacks.BackboneFinetuning.finetune_function}

Called when the epoch begins.

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

#### `freeze_before_training(pl_module)` {#lightning.pytorch.callbacks.BackboneFinetuning.freeze_before_training}

Override to add your freeze logic.

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

#### `load_state_dict(state_dict)` {#lightning.pytorch.callbacks.BackboneFinetuning.load_state_dict}

Called when loading a checkpoint, implement to reload callback state given callback’s `state_dict`.

Parameters:

**state\_dict** ([`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")\]) – the callback state returned by `state_dict`.

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

#### `on_fit_start(trainer, pl_module)` {#lightning.pytorch.callbacks.BackboneFinetuning.on_fit_start}

Raises:

**MisconfigurationException** – If LightningModule has no nn.Module backbone attribute.

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

#### `state_dict()` {#lightning.pytorch.callbacks.BackboneFinetuning.state_dict}

Called when saving a checkpoint, implement to generate callback’s `state_dict`.

Return type:

[`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")\]

Returns:

A dictionary containing callback state.

---

**Navigation**

- Parent: [Api](/docs/pytorch/2.6.6/optional-api/api.md)