---
title: BitsandbytesPrecision
description: Plugin for quantizing weights with bitsandbytes.
canonical: >-
  https://lightning.ai/docs/fabric/stable/full-api-reference/api/generated/lightning.fabric.plugins.precision.BitsandbytesPrecision
---
# BitsandbytesPrecision {#bitsandbytesprecision}

### `class lightning.fabric.plugins.precision.BitsandbytesPrecision(mode, dtype=None, ignore_modules=None)` {#lightning.fabric.plugins.precision.BitsandbytesPrecision}

Bases: [`Precision`](/docs/fabric/2.6.6/full-api-reference/api/generated/lightning.fabric.plugins.precision.Precision.md#lightning.fabric.plugins.precision.Precision "lightning.fabric.plugins.precision.precision.Precision")

Plugin for quantizing weights with [bitsandbytes](https://github.com/bitsandbytes-foundation/bitsandbytes).

Warning

This is an [experimental](/docs/pytorch/stable/versioning#experimental-api "(in PyTorch Lightning v2.7.0.dev)") feature.

Note

The optimizer is not automatically replaced with `bitsandbytes.optim.Adam8bit` or equivalent 8-bit optimizers.

Parameters:

*   **mode** ([`Literal`](https://docs.python.org/3/library/typing.html#typing.Literal "(in Python v3.15)")\[`'nf4'`, `'nf4-dq'`, `'fp4'`, `'fp4-dq'`, `'int8'`, `'int8-training'`\]) – The quantization mode to use.
    
*   **dtype** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`dtype`](https://docs.pytorch.org/docs/stable/tensor_attributes.html#torch.dtype "(in PyTorch v2.15)")\]) – The compute dtype to use.
    
*   **ignore\_modules** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`set`](https://docs.python.org/3/builtins/stdtypes.html#set "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\]\]) – The submodules whose Linear layers should not be replaced, for example. `{"lm_head"}`. This might be desirable for numerical stability. The string will be checked in as a prefix, so a value like “transformer.blocks” will ignore all linear layers in all of the transformer blocks.
    

#### `convert_input(data)` {#lightning.fabric.plugins.precision.BitsandbytesPrecision.convert_input}

Convert model inputs (forward) to the floating point precision type of this plugin.

This is a no-op in the base precision plugin, since we assume the data already has the desired type (default is torch.float32).

Return type:

[`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")

#### `convert_module(module)` {#lightning.fabric.plugins.precision.BitsandbytesPrecision.convert_module}

Convert the module parameters to the precision type this plugin handles.

This is optional and depends on the precision limitations during optimization.

Return type:

[`Module`](https://docs.pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module "(in PyTorch v2.15)")

#### `convert_output(data)` {#lightning.fabric.plugins.precision.BitsandbytesPrecision.convert_output}

Convert outputs to the floating point precision type expected after model’s forward.

This is a no-op in the base precision plugin, since we assume the data already has the desired type (default is torch.float32).

Return type:

[`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")

#### `forward_context()` {#lightning.fabric.plugins.precision.BitsandbytesPrecision.forward_context}

A contextmanager for managing model forward/training\_step/evaluation\_step/predict\_step.

Return type:

[`AbstractContextManager`](https://docs.python.org/3/library/contextlib.html#contextlib.AbstractContextManager "(in Python v3.15)")

#### `module_init_context()` {#lightning.fabric.plugins.precision.BitsandbytesPrecision.module_init_context}

Instantiate module parameters or tensors in the precision type this plugin handles.

This is optional and depends on the precision limitations during optimization.

Return type:

[`AbstractContextManager`](https://docs.python.org/3/library/contextlib.html#contextlib.AbstractContextManager "(in Python v3.15)")

#### `tensor_init_context()` {#lightning.fabric.plugins.precision.BitsandbytesPrecision.tensor_init_context}

Controls how tensors get created (device, dtype).

Return type:

[`AbstractContextManager`](https://docs.python.org/3/library/contextlib.html#contextlib.AbstractContextManager "(in Python v3.15)")

---

**Navigation**

- Parent: [Generated](/docs/fabric/2.6.6/full-api-reference/api/generated.md)