---
title: Perplexity
description: Perplexity measures how well a language model predicts a text sample.
canonical: 'https://lightning.ai/docs/torchmetrics/stable/text/perplexity'
---
# Perplexity {#perplexity}

## Module Interface {#module-interface}

### `class torchmetrics.text.perplexity.Perplexity(ignore_index=None, **kwargs)` {#torchmetrics.text.perplexity.Perplexity}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/text/perplexity.py#L29-L131)

Perplexity measures how well a language model predicts a text sample.

It’s calculated as the average number of bits per word a model needs to represent the sample.

As input to `forward` and `update` the metric accepts the following input:

*   `preds` ([`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")): Logits or a unnormalized score assigned to each token in a sequence with shape \[batch\_size, seq\_len, vocab\_size\], which is the output of a language model. Scores will be normalized internally using softmax.
    
*   `target` ([`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")): Ground truth values with a shape \[batch\_size, seq\_len\]
    

As output of `forward` and `compute` the metric returns the following output:

*   `perp` ([`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")): A tensor with the perplexity score
    

Parameters:

*   **ignore\_index** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")\]) – Integer specifying a target class to ignore. If given, this class index does not contribute to the returned score.
    
*   **kwargs** ([`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")\]) – Additional keyword arguments, see [Advanced metric settings](/docs/torchmetrics/1.9.0/user-guide/overview.md#metric-kwargs) for more info.
    

Examples

```python
>>> from torch import rand, randint
>>> from torchmetrics.text import Perplexity
>>> preds = rand(2, 8, 5)
>>> target = randint(5, (2, 8))
>>> target[0, 6:] = -100
>>> perp = Perplexity(ignore_index=-100)
>>> perp(preds, target)
tensor(5.8540)
```

#### `plot(val=None, ax=None)` {#torchmetrics.text.perplexity.Perplexity.plot}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/text/perplexity.py#L91-L131)

Plot a single or multiple values from the metric.

Parameters:

*   **val** (`Union`\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)"), [`Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "(in Python v3.15)")\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")\], [`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")\]) – Either a single result from calling metric.forward or metric.compute or a list of these results. If no value is provided, will automatically call metric.compute and plot that result.
    
*   **ax** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`Axes`](https://matplotlib.org/stable/api/_as_gen/matplotlib.axes.Axes.html#matplotlib.axes.Axes "(in Matplotlib v3.11.2)")\]) – An matplotlib axis object. If provided will add plot to that axis
    

Return type:

[`tuple`](https://docs.python.org/3/builtins/stdtypes.html#tuple "(in Python v3.15)")\[[`Figure`](https://matplotlib.org/stable/api/_as_gen/matplotlib.figure.Figure.html#matplotlib.figure.Figure "(in Matplotlib v3.11.2)"), `Union`\[[`Axes`](https://matplotlib.org/stable/api/_as_gen/matplotlib.axes.Axes.html#matplotlib.axes.Axes "(in Matplotlib v3.11.2)"), [`ndarray`](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray "(in NumPy v2.5)")\]\]

Returns:

Figure and Axes object

Raises:

[**ModuleNotFoundError**](https://docs.python.org/3/builtins/exceptions.html#ModuleNotFoundError "(in Python v3.15)") – If matplotlib is not installed

```python
>>> # Example plotting a single value
>>> import torch
>>> from torchmetrics.text import Perplexity
>>> metric = Perplexity()
>>> metric.update(torch.rand(2, 8, 5), torch.randint(5, (2, 8)))
>>> fig_, ax_ = metric.plot()
```

[![../../\_images/perplexity-1.png](/docs/torchmetrics/1.9.0/static/_images/perplexity-1.png)](/docs/torchmetrics/1.9.0/static/_images/perplexity-1.png)

```python
>>> # Example plotting multiple values
>>> import torch
>>> from torchmetrics.text import Perplexity
>>> metric = Perplexity()
>>> values = [ ]
>>> for _ in range(10):
...     values.append(metric(torch.rand(2, 8, 5), torch.randint(5, (2, 8))))
>>> fig_, ax_ = metric.plot(values)
```

[![../../\_images/perplexity-2.png](/docs/torchmetrics/1.9.0/static/_images/perplexity-2.png)](/docs/torchmetrics/1.9.0/static/_images/perplexity-2.png)

## Functional Interface {#functional-interface}

### `torchmetrics.functional.text.perplexity.perplexity(preds, target, ignore_index=None)` {#torchmetrics.functional.text.perplexity.perplexity}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/functional/text/perplexity.py#L114-L142)

Perplexity measures how well a language model predicts a text sample.

This metric is calculated as the average number of bits per word a model needs to represent the sample.

Parameters:

*   **preds** ([`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")) – Logits or a unnormalized score assigned to each token in a sequence with shape \[batch\_size, seq\_len, vocab\_size\], which is the output of a language model. Scores will be normalized internally using softmax.
    
*   **target** ([`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")) – Ground truth values with a shape \[batch\_size, seq\_len\].
    
*   **ignore\_index** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")\]) – Integer specifying a target class to ignore. If given, this class index does not contribute to the returned score.
    

Return type:

[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")

Returns:

Perplexity value

Examples

```python
>>> from torch import rand, randint
>>> preds = rand(2, 8, 5)
>>> target = randint(5, (2, 8))
>>> target[0, 6:] = -100
>>> perplexity(preds, target, ignore_index=-100)
tensor(5.8540)
```

---

**Navigation**

- Parent: [Text](/docs/torchmetrics/1.9.0/text.md)