---
title: thunder.plugins.QuantizeInt4
description: Plugin for 4-bit integer quantization using BitsAndBytes.
canonical: >-
  https://lightning.ai/docs/thunder/stable/api-reference/generated/thunder.plugins.QuantizeInt4
---
# thunder.plugins.QuantizeInt4 {#thunder-plugins-quantizeint4}

### `class thunder.plugins.QuantizeInt4` {#thunder.plugins.QuantizeInt4}

[\[source\]](https://github.com/Lightning-AI/lightning-thunder/blob/0.2.6/thunder/plugins/quantization.py#L4-L37)

Bases: `Plugin`

Plugin for 4-bit integer quantization using BitsAndBytes.

This plugin applies a 4-bit linear quantization transform to model weights, reducing memory footprint and improving throughput for both training and inference.

See [https://github.com/bitsandbytes-foundation/bitsandbytes/blob/main/bitsandbytes/functional.py#L889](https://github.com/bitsandbytes-foundation/bitsandbytes/blob/main/bitsandbytes/functional.py#L889) for more details.

#### `__init__()` {#thunder.plugins.QuantizeInt4.__init__}

[\[source\]](https://github.com/Lightning-AI/lightning-thunder/blob/thunder/plugins.py)

Methods

- [`__init__()`](#thunder.plugins.QuantizeInt4.__init__)
- [`setup_executors()`](#thunder.plugins.QuantizeInt4.setup_executors) — Fetches the BitsAndBytes quantization executor.
- `setup_lookasides()` — rtype:Optional[list[Lookaside]]
- [`setup_transforms()`](#thunder.plugins.QuantizeInt4.setup_transforms) — Fetches the BitsAndBytes quantization transform.

Attributes

- `policy`

#### `setup_executors()` {#thunder.plugins.QuantizeInt4.setup_executors}

[\[source\]](https://github.com/Lightning-AI/lightning-thunder/blob/0.2.6/thunder/plugins/quantization.py#L27-L37)

Fetches the BitsAndBytes quantization executor.

Returns:

A list containing the Transformer Engine executor.

Return type:

[list](https://docs.python.org/3/builtins/stdtypes.html#list "(in Python v3.15)")\[[Executor](/docs/thunder/extend/thunder.extend.Executor#thunder.extend.Executor "thunder.extend.Executor")\]

#### `setup_transforms()` {#thunder.plugins.QuantizeInt4.setup_transforms}

[\[source\]](https://github.com/Lightning-AI/lightning-thunder/blob/0.2.6/thunder/plugins/quantization.py#L15-L25)

Fetches the BitsAndBytes quantization transform.

Returns:

A list containing the Transformer Engine executor.

Return type:

[list](https://docs.python.org/3/builtins/stdtypes.html#list "(in Python v3.15)")\[Transform\]

---

**Navigation**

- Parent: [Generated](/docs/thunder/0.2.6/api-reference/generated.md)