---
title: SQuAD
description: >-
  Calculate SQuAD Metric which is a metric for evaluating question answering
  models.
canonical: 'https://lightning.ai/docs/torchmetrics/stable/text/squad'
---
# SQuAD {#squad}

## Module Interface {#module-interface}

### `class torchmetrics.text.SQuAD(**kwargs)` {#torchmetrics.text.SQuAD}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/text/squad.py#L35-L168)

Calculate [SQuAD Metric](https://arxiv.org/abs/1606.05250) which is a metric for evaluating question answering models.

This metric corresponds to the scoring script for version 1 of the Stanford Question Answering Dataset (SQuAD).

As input to `forward` and `update` the metric accepts the following input:

*   `preds` (`Dict`): A Dictionary or List of Dictionary-s that map `id` and `prediction_text` to the respective values
    
    Example `prediction`:
    
    > ```python
    > {"prediction_text": "TorchMetrics is awesome", "id": "123"}
    > ```
    
*   `target` (`Dict`): A Dictionary or List of Dictionary-s that contain the `answers` and `id` in the SQuAD Format.
    
    > Example `target`:
    > 
    > ```python
    > {
    >     'answers': [{'answer_start': [1], 'text': ['This is a test answer']}],
    >     'id': '1',
    > }
    > ```
    > 
    > Reference SQuAD Format:
    > 
    > ```python
    > {
    >     'answers': {'answer_start': [1], 'text': ['This is a test text']},
    >     'context': 'This is a test context.',
    >     'id': '1',
    >     'question': 'Is this a test?',
    >     'title': 'train test'
    > }
    > ```
    

As output of `forward` and `compute` the metric returns the following output:

*   `squad` (`Dict`): A dictionary containing the F1 score (key: “f1”),
    
    and Exact match score (key: “exact\_match”) for the batch.
    

Parameters:

**kwargs** ([`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")) – Additional keyword arguments, see [Advanced metric settings](/docs/torchmetrics/1.9.0/user-guide/overview.md#metric-kwargs) for more info.

Example

```python
>>> from torchmetrics.text import SQuAD
>>> preds = [{"prediction_text": "1976", "id": "56e10a3be3433e1400422b22"}]
>>> target = [{"answers": {"answer_start": [97], "text": ["1976"]}, "id": "56e10a3be3433e1400422b22"}]
>>> squad = SQuAD()
>>> squad(preds, target)
{'exact_match': tensor(100.), 'f1': tensor(100.)}
```

#### `plot(val=None, ax=None)` {#torchmetrics.text.SQuAD.plot}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/text/squad.py#L126-L168)

Plot a single or multiple values from the metric.

Parameters:

*   **val** (`Union`\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)"), [`Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "(in Python v3.15)")\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")\], [`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")\]) – Either a single result from calling metric.forward or metric.compute or a list of these results. If no value is provided, will automatically call metric.compute and plot that result.
    
*   **ax** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`Axes`](https://matplotlib.org/stable/api/_as_gen/matplotlib.axes.Axes.html#matplotlib.axes.Axes "(in Matplotlib v3.11.2)")\]) – An matplotlib axis object. If provided will add plot to that axis
    

Return type:

[`tuple`](https://docs.python.org/3/builtins/stdtypes.html#tuple "(in Python v3.15)")\[[`Figure`](https://matplotlib.org/stable/api/_as_gen/matplotlib.figure.Figure.html#matplotlib.figure.Figure "(in Matplotlib v3.11.2)"), `Union`\[[`Axes`](https://matplotlib.org/stable/api/_as_gen/matplotlib.axes.Axes.html#matplotlib.axes.Axes "(in Matplotlib v3.11.2)"), [`ndarray`](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray "(in NumPy v2.5)")\]\]

Returns:

Figure and Axes object

Raises:

[**ModuleNotFoundError**](https://docs.python.org/3/builtins/exceptions.html#ModuleNotFoundError "(in Python v3.15)") – If matplotlib is not installed

```python
>>> # Example plotting a single value
>>> from torchmetrics.text import SQuAD
>>> metric = SQuAD()
>>> preds = [{"prediction_text": "1976", "id": "56e10a3be3433e1400422b22"}]
>>> target = [{"answers": {"answer_start": [97], "text": ["1976"]}, "id": "56e10a3be3433e1400422b22"}]
>>> metric.update(preds, target)
>>> fig_, ax_ = metric.plot()
```

[![../../\_images/squad-1.png](/docs/torchmetrics/1.9.0/static/_images/squad-1.png)](/docs/torchmetrics/1.9.0/static/_images/squad-1.png)

```python
>>> # Example plotting multiple values
>>> from torchmetrics.text import SQuAD
>>> metric = SQuAD()
>>> preds = [{"prediction_text": "1976", "id": "56e10a3be3433e1400422b22"}]
>>> target = [{"answers": {"answer_start": [97], "text": ["1976"]}, "id": "56e10a3be3433e1400422b22"}]
>>> values = [ ]
>>> for _ in range(10):
...     values.append(metric(preds, target))
>>> fig_, ax_ = metric.plot(values)
```

[![../../\_images/squad-2.png](/docs/torchmetrics/1.9.0/static/_images/squad-2.png)](/docs/torchmetrics/1.9.0/static/_images/squad-2.png)

## Functional Interface {#functional-interface}

### `torchmetrics.functional.text.squad(preds, target)` {#torchmetrics.functional.text.squad}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/functional/text/squad.py#L195-L252)

Calculate [SQuAD Metric](https://arxiv.org/abs/1606.05250) .

Parameters:

*   **preds** (`Union`\[[`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\], [`list`](https://docs.python.org/3/builtins/stdtypes.html#list "(in Python v3.15)")\[[`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\]\]\]) –
    
    A Dictionary or List of Dictionary-s that map id and prediction\_text to the respective values.
    
    Example prediction:
    
    ```python
    {"prediction_text": "TorchMetrics is awesome", "id": "123"}
    ```
    
*   **target** (`Union`\[[`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), `Union`\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), `Union`\[[`list`](https://docs.python.org/3/builtins/stdtypes.html#list "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\], [`list`](https://docs.python.org/3/builtins/stdtypes.html#list "(in Python v3.15)")\[[`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")\]\]\]\]\], [`list`](https://docs.python.org/3/builtins/stdtypes.html#list "(in Python v3.15)")\[[`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), `Union`\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), `Union`\[[`list`](https://docs.python.org/3/builtins/stdtypes.html#list "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\], [`list`](https://docs.python.org/3/builtins/stdtypes.html#list "(in Python v3.15)")\[[`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")\]\]\]\]\]\]\]) –
    
    A Dictionary or List of Dictionary-s that contain the answers and id in the SQuAD Format.
    
    Example target:
    
    ```python
    {
        'answers': [{'answer_start': [1], 'text': ['This is a test answer']}],
        'id': '1',
    }
    ```
    
    Reference SQuAD Format:
    
    ```python
    {
        'answers': {'answer_start': [1], 'text': ['This is a test text']},
        'context': 'This is a test context.',
        'id': '1',
        'question': 'Is this a test?',
        'title': 'train test'
    }
    ```
    

Return type:

[`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")\]

Returns:

Dictionary containing the F1 score, Exact match score for the batch.

Example

```python
>>> from torchmetrics.functional.text.squad import squad
>>> preds = [{"prediction_text": "1976", "id": "56e10a3be3433e1400422b22"}]
>>> target = [{"answers": {"answer_start": [97], "text": ["1976"]},"id": "56e10a3be3433e1400422b22"}]
>>> squad(preds, target)
{'exact_match': tensor(100.), 'f1': tensor(100.)}
```

Raises:

[**KeyError**](https://docs.python.org/3/builtins/exceptions.html#KeyError "(in Python v3.15)") – If the required keys are missing in either predictions or targets.

References

\[1\] SQuAD: 100,000+ Questions for Machine Comprehension of Text by Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, Percy Liang [SQuAD Metric](https://arxiv.org/abs/1606.05250) .

---

**Navigation**

- Parent: [Text](/docs/torchmetrics/1.9.0/text.md)