## Module Interface¶

Calculate SQuAD Metric which is a metric for evaluating question answering models.

This metric corresponds to the scoring script for version 1 of the Stanford Question Answering Dataset (SQuAD).

As input to forward and update the metric accepts the following input:

• preds (Dict): A Dictionary or List of Dictionary-s that map id and prediction_text to the respective values

Example prediction:

{"prediction_text": "TorchMetrics is awesome", "id": "123"}

• target (Dict): A Dictionary or List of Dictionary-s that contain the answers and id in the SQuAD Format.

Example target:

{
'id': '1',
}


{
'context': 'This is a test context.',
'id': '1',
'question': 'Is this a test?',
'title': 'train test'
}


As output of forward and compute the metric returns the following output:

• squad (Dict): A dictionary containing the F1 score (key: “f1”),

and Exact match score (key: “exact_match”) for the batch.

Parameters:

kwargs (Any) – Additional keyword arguments, see Advanced metric settings for more info.

Example

>>> from torchmetrics.text import SQuAD
>>> preds = [{"prediction_text": "1976", "id": "56e10a3be3433e1400422b22"}]
{'exact_match': tensor(100.), 'f1': tensor(100.)}

plot(val=None, ax=None)[source]

Plot a single or multiple values from the metric.

Parameters:
Return type:
Returns:

Figure and Axes object

Raises:

ModuleNotFoundError – If matplotlib is not installed

>>> # Example plotting a single value
>>> preds = [{"prediction_text": "1976", "id": "56e10a3be3433e1400422b22"}]
>>> metric.update(preds, target)
>>> fig_, ax_ = metric.plot()

>>> # Example plotting multiple values
>>> preds = [{"prediction_text": "1976", "id": "56e10a3be3433e1400422b22"}]
>>> values = [ ]
>>> for _ in range(10):
...     values.append(metric(preds, target))
>>> fig_, ax_ = metric.plot(values)


## Functional Interface¶

Parameters:
• A Dictionary or List of Dictionary-s that map id and prediction_text to the respective values.

Example prediction:

{"prediction_text": "TorchMetrics is awesome", "id": "123"}


• A Dictionary or List of Dictionary-s that contain the answers and id in the SQuAD Format.

Example target:

{
'id': '1',
}


{
'context': 'This is a test context.',
'id': '1',
'question': 'Is this a test?',
'title': 'train test'
}


Return type:
Returns:

Dictionary containing the F1 score, Exact match score for the batch.

Example

>>> from torchmetrics.functional.text.squad import squad
>>> preds = [{"prediction_text": "1976", "id": "56e10a3be3433e1400422b22"}]