---
title: TransformerEnginePrecision
description: |-
  Plugin for training with fp8 precision via nvidia’s
  Transformer Engine.
canonical: >-
  https://lightning.ai/docs/pytorch/stable/optional-api/api/lightning.pytorch.plugins.precision.TransformerEnginePrecision
---
# TransformerEnginePrecision {#transformerengineprecision}

### `class lightning.pytorch.plugins.precision.TransformerEnginePrecision(*, weights_dtype, recipe=None, replace_layers=None, fallback_compute_dtype=None)` {#lightning.pytorch.plugins.precision.TransformerEnginePrecision}

Bases: [`Precision`](/docs/pytorch/2.6.6/optional-api/api/lightning.pytorch.plugins.precision.Precision.md#lightning.pytorch.plugins.precision.Precision "lightning.pytorch.plugins.precision.precision.Precision"), [`TransformerEnginePrecision`](/docs/fabric/stable/api/generated/lightning.fabric.plugins.precision.TransformerEnginePrecision#lightning.fabric.plugins.precision.TransformerEnginePrecision "(in lightning v2.7.0.dev)")

Plugin for training with fp8 precision via nvidia’s [Transformer Engine](https://docs.nvidia.com/deeplearning/transformer-engine).

Warning

This is an [experimental](/docs/pytorch/2.6.6/reference/versioning.md#experimental-api) feature.

Parameters:

*   **dtype** – The weights dtype to use.
    
*   **recipe** (`Union`\[[`Mapping`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Mapping "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")\], `DelayedScaling`, [`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")\]) – Recipe for the DelayedScaling [configuration](https://docs.nvidia.com/deeplearning/transformer-engine/user-guide/api/common.html#transformer_engine.common.recipe.DelayedScaling). In dict format or the dataclass format.
    
*   **replace\_layers** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")\]) – Whether to replace `Linear` and `LayerNorm` layers automatically with their Transformer Engine alternatives. Note that they don’t subclass the torch equivalents so checks like `isinstance(l, torch.nn.Linear)` will not pass.
    

Note

Support for FP8 in the linear layers with this plugin is currently limited to tensors with shapes where the dimensions are divisible by 8 and 16 respectively. You might want to add padding to your inputs to conform to this restriction.

---

**Navigation**

- Parent: [Api](/docs/pytorch/2.6.6/optional-api/api.md)