---
title: Retrieval AUROC
description: >-
  Compute area under the receiver operating characteristic curve (AUROC) for
  information retrieval.
canonical: 'https://lightning.ai/docs/torchmetrics/stable/retrieval/auroc'
---
# Retrieval AUROC {#retrieval-auroc}

## Module Interface {#module-interface}

### `class torchmetrics.retrieval.RetrievalAUROC(empty_target_action='neg', ignore_index=None, top_k=None, max_fpr=None, aggregation='mean', **kwargs)` {#torchmetrics.retrieval.RetrievalAUROC}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/retrieval/auroc.py#L29-L164)

Compute area under the receiver operating characteristic curve (AUROC) for information retrieval.

Works with binary target data. Accepts float predictions from a model output.

As input to `forward` and `update` the metric accepts the following input:

*   `preds` ([`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")): A float tensor of shape `(N, ...)`
    
*   `target` ([`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")): A long or bool tensor of shape `(N, ...)`
    
*   `indexes` ([`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")): A long tensor of shape `(N, ...)` which indicate to which query a prediction belongs
    

As output to `forward` and `compute` the metric returns the following output:

*   `auroc@k` ([`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")): A single-value tensor with the auroc value of the predictions `preds` w.r.t. the labels `target`.
    

All `indexes`, `preds` and `target` must have the same dimension and will be flatten at the beginning, so that for example, a tensor of shape `(N, M)` is treated as `(N * M, )`. Predictions will be first grouped by `indexes` and then will be computed as the mean of the metric over each query.

Parameters:

*   **empty\_target\_action** ([`Literal`](https://docs.python.org/3/library/typing.html#typing.Literal "(in Python v3.15)")\[`'error'`, `'skip'`, `'neg'`, `'pos'`\]) –
    
    Specify what to do with queries that do not have at least a positive `target`. Choose from:
    
    *   `'neg'`: those queries count as `0.0` (default)
        
    *   `'pos'`: those queries count as `1.0`
        
    *   `'skip'`: skip those queries; if all queries are skipped, `0.0` is returned
        
    *   `'error'`: raise a `ValueError`
        
    
*   **ignore\_index** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")\]) – Ignore predictions where the target is equal to this number.
    
*   **top\_k** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")\]) – Consider only the top k elements for each query (default: `None`, which considers them all)
    
*   **max\_fpr** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`float`](https://docs.python.org/3/builtins/functions.html#float "(in Python v3.15)")\]) – If not `None`, calculates standardized partial AUC over the range `[0, max_fpr]`.
    
*   **aggregation** (`Union`\[[`Literal`](https://docs.python.org/3/library/typing.html#typing.Literal "(in Python v3.15)")\[`'mean'`, `'median'`, `'min'`, `'max'`\], [`Callable`](https://docs.python.org/3/library/typing.html#typing.Callable "(in Python v3.15)")\]) –
    
    Specify how to aggregate over indexes. Can either a custom callable function that takes in a single tensor and returns a scalar value or one of the following strings:
    
    *   `'mean'`: average value is returned
        
    *   `'median'`: median value is returned
        
    *   `'max'`: max value is returned
        
    *   `'min'`: min value is returned
        
    
*   **kwargs** ([`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")) – Additional keyword arguments, see [Advanced metric settings](/docs/torchmetrics/1.9.0/user-guide/overview.md#metric-kwargs) for more info.
    

Raises:

*   [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError "(in Python v3.15)") – If `empty_target_action` is not one of `error`, `skip`, `neg` or `pos`.
    
*   [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError "(in Python v3.15)") – If `ignore_index` is not None or an integer.
    
*   [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError "(in Python v3.15)") – If `top_k` is not `None` or not an integer greater than 0.
    

Example

```python
>>> from torch import tensor
>>> from torchmetrics.retrieval import RetrievalAUROC
>>> indexes = tensor([0, 0, 0, 1, 1, 1, 1])
>>> preds = tensor([0.2, 0.3, 0.5, 0.1, 0.3, 0.5, 0.2])
>>> target = tensor([False, False, True, False, True, False, True])
>>> rmap = RetrievalAUROC()
>>> rmap(preds, target, indexes=indexes)
tensor(0.7500)
```

#### `plot(val=None, ax=None)` {#torchmetrics.retrieval.RetrievalAUROC.plot}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/retrieval/auroc.py#L124-L164)

Plot a single or multiple values from the metric.

Parameters:

*   **val** (`Union`\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)"), [`Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "(in Python v3.15)")\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")\], [`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")\]) – Either a single result from calling metric.forward or metric.compute or a list of these results. If no value is provided, will automatically call metric.compute and plot that result.
    
*   **ax** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`Axes`](https://matplotlib.org/stable/api/_as_gen/matplotlib.axes.Axes.html#matplotlib.axes.Axes "(in Matplotlib v3.11.2)")\]) – An matplotlib axis object. If provided will add plot to that axis
    

Return type:

[`tuple`](https://docs.python.org/3/builtins/stdtypes.html#tuple "(in Python v3.15)")\[[`Figure`](https://matplotlib.org/stable/api/_as_gen/matplotlib.figure.Figure.html#matplotlib.figure.Figure "(in Matplotlib v3.11.2)"), `Union`\[[`Axes`](https://matplotlib.org/stable/api/_as_gen/matplotlib.axes.Axes.html#matplotlib.axes.Axes "(in Matplotlib v3.11.2)"), [`ndarray`](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray "(in NumPy v2.5)")\]\]

Returns:

Figure and Axes object

Raises:

[**ModuleNotFoundError**](https://docs.python.org/3/builtins/exceptions.html#ModuleNotFoundError "(in Python v3.15)") – If matplotlib is not installed

```python
>>> import torch
>>> from torchmetrics.retrieval import RetrievalAUROC
>>> # Example plotting a single value
>>> metric = RetrievalAUROC()
>>> metric.update(torch.rand(10,), torch.randint(2, (10,)), indexes=torch.randint(2,(10,)))
>>> fig_, ax_ = metric.plot()
```

[![../../\_images/auroc-11.png](/docs/torchmetrics/1.9.0/static/_images/auroc-11.png)](/docs/torchmetrics/1.9.0/static/_images/auroc-11.png)

```python
>>> import torch
>>> from torchmetrics.retrieval import RetrievalAUROC
>>> # Example plotting multiple values
>>> metric = RetrievalAUROC()
>>> values = []
>>> for _ in range(10):
...     values.append(metric(torch.rand(10,), torch.randint(2, (10,)), indexes=torch.randint(2,(10,))))
>>> fig, ax = metric.plot(values)
```

[![../../\_images/auroc-21.png](/docs/torchmetrics/1.9.0/static/_images/auroc-21.png)](/docs/torchmetrics/1.9.0/static/_images/auroc-21.png)

## Functional Interface {#functional-interface}

### `torchmetrics.functional.retrieval.retrieval_auroc(preds, target, top_k=None, max_fpr=None)` {#torchmetrics.functional.retrieval.retrieval_auroc}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/functional/retrieval/auroc.py#L22-L64)

Compute area under the receiver operating characteristic curve (AUROC) for information retrieval.

`preds` and `target` should be of the same shape and live on the same device. If no `target` is `True`, `0` is returned. `target` must be either bool or integers and `preds` must be `float`, otherwise an error is raised.

Parameters:

*   **preds** ([`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")) – estimated probabilities of each document to be relevant.
    
*   **target** ([`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")) – ground truth about each document being relevant or not.
    
*   **top\_k** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")\]) – consider only the top k elements (default: `None`, which considers them all)
    
*   **max\_fpr** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`float`](https://docs.python.org/3/builtins/functions.html#float "(in Python v3.15)")\]) – If not `None`, calculates standardized partial AUC over the range `[0, max_fpr]`.
    

Return type:

[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")

Returns:

a single-value tensor with the auroc value of the predictions `preds` w.r.t. the labels `target`.

Raises:

[**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError "(in Python v3.15)") – If `top_k` is not `None` or an integer larger than 0.

Example

```python
>>> from torchmetrics.functional.retrieval import retrieval_auroc
>>> preds = tensor([0.2, 0.3, 0.5])
>>> target = tensor([True, False, True])
>>> retrieval_auroc(preds, target)
tensor(0.5000)
```

---

**Navigation**

- Parent: [Retrieval](/docs/torchmetrics/1.9.0/retrieval.md)