---
title: Calibration Error
description: Top-label Calibration Error.
canonical: 'https://lightning.ai/docs/torchmetrics/stable/classification/calibration_error'
---
# Calibration Error {#calibration-error}

## Module Interface {#module-interface}

### `class torchmetrics.CalibrationError(**kwargs)` {#class-torchmetrics-calibrationerror-kwargs}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/classification/calibration_error.py#L345-L392)

[Top-label Calibration Error](https://arxiv.org/abs/1909.10155).

The expected calibration error can be used to quantify how well a given model is calibrated e.g. how well the predicted output probabilities of the model matches the actual probabilities of the ground truth distribution. Three different norms are implemented, each corresponding to variations on the calibration error metric.

```math
\text{ECE} = \sum_i^N b_i \|(p_i - c_i)\|, \text{L1 norm (Expected Calibration Error)}
```

```math
\text{MCE} = \max_{i} (p_i - c_i), \text{Infinity norm (Maximum Calibration Error)}
```

```math
\text{RMSCE} = \sqrt{\sum_i^N b_i(p_i - c_i)^2}, \text{L2 norm (Root Mean Square Calibration Error)}
```

Where $`p_i`$ is the top-1 prediction accuracy in bin $`i`$, $`c_i`$ is the average confidence of predictions in bin $`i`$, and $`b_i`$ is the fraction of data points in bin $`i`$. Bins are constructed in an uniform way in the \[0,1\] range.

This function is a simple wrapper to get the task specific versions of this metric, which is done by setting the `task` argument to either `'binary'` or `'multiclass'`. See the documentation of [`BinaryCalibrationError`](#torchmetrics.classification.BinaryCalibrationError "torchmetrics.classification.BinaryCalibrationError") and [`MulticlassCalibrationError`](#torchmetrics.classification.MulticlassCalibrationError "torchmetrics.classification.MulticlassCalibrationError") for the specific details of each argument influence and examples.

#### `static __new__(cls, task, n_bins=15, norm='l1', num_classes=None, ignore_index=None, validate_args=True, **kwargs)` {#static-new-cls-task-n-bins-15-norm-l1-num-classes-none-ignore-index-none-validate-args-true-kwargs}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/classification/calibration_error.py#L373-L392)

Initialize task metric.

Return type:

[`Metric`](/docs/torchmetrics/1.9.0/api-reference/metric.md#torchmetrics.Metric "torchmetrics.metric.Metric")

### BinaryCalibrationError {#binarycalibrationerror}

### `class torchmetrics.classification.BinaryCalibrationError(n_bins=15, norm='l1', ignore_index=None, validate_args=True, **kwargs)` {#torchmetrics.classification.BinaryCalibrationError}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/classification/calibration_error.py#L42-L187)

[Top-label Calibration Error](https://arxiv.org/abs/1909.10155) for binary tasks.

The expected calibration error can be used to quantify how well a given model is calibrated e.g. how well the predicted output probabilities of the model matches the actual probabilities of the ground truth distribution. Three different norms are implemented, each corresponding to variations on the calibration error metric.

```math
\text{ECE} = \sum_i^N b_i \|(p_i - c_i)\|, \text{L1 norm (Expected Calibration Error)}
```

```math
\text{MCE} = \max_{i} (p_i - c_i), \text{Infinity norm (Maximum Calibration Error)}
```

```math
\text{RMSCE} = \sqrt{\sum_i^N b_i(p_i - c_i)^2}, \text{L2 norm (Root Mean Square Calibration Error)}
```

Where $`p_i`$ is the top-1 prediction accuracy in bin $`i`$, $`c_i`$ is the average confidence of predictions in bin $`i`$, and $`b_i`$ is the fraction of data points in bin $`i`$. Bins are constructed in an uniform way in the \[0,1\] range.

As input to `forward` and `update` the metric accepts the following input:

*   `preds` ([`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")): A float tensor of shape `(N, ...)` containing probabilities or logits for each observation. If preds has values outside \[0,1\] range we consider the input to be logits and will auto apply sigmoid per element.
    
*   `target` ([`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")): An int tensor of shape `(N, ...)` containing ground truth labels, and therefore only contain {0,1} values (except if ignore\_index is specified). The value 1 always encodes the positive class.
    

As output to `forward` and `compute` the metric returns the following output:

*   `bce` ([`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")): A scalar tensor containing the calibration error
    

Additional dimension `...` will be flattened into the batch dimension.

Parameters:

*   **n\_bins** ([`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")) – Number of bins to use when computing the metric.
    
*   **norm** ([`Literal`](https://docs.python.org/3/library/typing.html#typing.Literal "(in Python v3.15)")\[`'l1'`, `'l2'`, `'max'`\]) – Norm used to compare empirical and expected probability bins.
    
*   **ignore\_index** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")\]) – Specifies a target value that is ignored and does not contribute to the metric calculation
    
*   **validate\_args** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")) – bool indicating if input arguments and tensors should be validated for correctness. Set to `False` for faster computations.
    
*   **kwargs** ([`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")) – Additional keyword arguments, see [Advanced metric settings](/docs/torchmetrics/1.9.0/user-guide/overview.md#metric-kwargs) for more info.
    

Example

```python
>>> from torch import tensor
>>> from torchmetrics.classification import BinaryCalibrationError
>>> preds = tensor([0.25, 0.25, 0.55, 0.75, 0.75])
>>> target = tensor([0, 0, 1, 1, 1])
>>> metric = BinaryCalibrationError(n_bins=2, norm='l1')
>>> metric(preds, target)
tensor(0.2900)
>>> bce = BinaryCalibrationError(n_bins=2, norm='l2')
>>> bce(preds, target)
tensor(0.2918)
>>> bce = BinaryCalibrationError(n_bins=2, norm='max')
>>> bce(preds, target)
tensor(0.3167)
```

#### `plot(val=None, ax=None)` {#torchmetrics.classification.BinaryCalibrationError.plot}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/classification/calibration_error.py#L147-L187)

Plot a single or multiple values from the metric.

Parameters:

*   **val** (`Union`\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)"), [`Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "(in Python v3.15)")\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")\], [`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")\]) – Either a single result from calling metric.forward or metric.compute or a list of these results. If no value is provided, will automatically call metric.compute and plot that result.
    
*   **ax** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`Axes`](https://matplotlib.org/stable/api/_as_gen/matplotlib.axes.Axes.html#matplotlib.axes.Axes "(in Matplotlib v3.11.2)")\]) – An matplotlib axis object. If provided will add plot to that axis
    

Return type:

[`tuple`](https://docs.python.org/3/builtins/stdtypes.html#tuple "(in Python v3.15)")\[[`Figure`](https://matplotlib.org/stable/api/_as_gen/matplotlib.figure.Figure.html#matplotlib.figure.Figure "(in Matplotlib v3.11.2)"), `Union`\[[`Axes`](https://matplotlib.org/stable/api/_as_gen/matplotlib.axes.Axes.html#matplotlib.axes.Axes "(in Matplotlib v3.11.2)"), [`ndarray`](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray "(in NumPy v2.5)")\]\]

Returns:

Figure object and Axes object

Raises:

[**ModuleNotFoundError**](https://docs.python.org/3/builtins/exceptions.html#ModuleNotFoundError "(in Python v3.15)") – If matplotlib is not installed

```python
>>> from torch import rand, randint
>>> # Example plotting a single value
>>> from torchmetrics.classification import BinaryCalibrationError
>>> metric = BinaryCalibrationError(n_bins=2, norm='l1')
>>> metric.update(rand(10), randint(2,(10,)))
>>> fig_, ax_ = metric.plot()
```

[![../../\_images/calibration\_error-1.png](/docs/torchmetrics/1.9.0/static/_images/calibration_error-1.png)](/docs/torchmetrics/1.9.0/static/_images/calibration_error-1.png)

```python
>>> from torch import rand, randint
>>> # Example plotting multiple values
>>> from torchmetrics.classification import BinaryCalibrationError
>>> metric = BinaryCalibrationError(n_bins=2, norm='l1')
>>> values = [ ]
>>> for _ in range(10):
...     values.append(metric(rand(10), randint(2,(10,))))
>>> fig_, ax_ = metric.plot(values)
```

[![../../\_images/calibration\_error-2.png](/docs/torchmetrics/1.9.0/static/_images/calibration_error-2.png)](/docs/torchmetrics/1.9.0/static/_images/calibration_error-2.png)

### MulticlassCalibrationError {#multiclasscalibrationerror}

### `class torchmetrics.classification.MulticlassCalibrationError(num_classes, n_bins=15, norm='l1', ignore_index=None, validate_args=True, **kwargs)` {#torchmetrics.classification.MulticlassCalibrationError}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/classification/calibration_error.py#L190-L342)

[Top-label Calibration Error](https://arxiv.org/abs/1909.10155) for multiclass tasks.

The expected calibration error can be used to quantify how well a given model is calibrated e.g. how well the predicted output probabilities of the model matches the actual probabilities of the ground truth distribution. Three different norms are implemented, each corresponding to variations on the calibration error metric.

```math
\text{ECE} = \sum_i^N b_i \|(p_i - c_i)\|, \text{L1 norm (Expected Calibration Error)}
```

```math
\text{MCE} = \max_{i} (p_i - c_i), \text{Infinity norm (Maximum Calibration Error)}
```

```math
\text{RMSCE} = \sqrt{\sum_i^N b_i(p_i - c_i)^2}, \text{L2 norm (Root Mean Square Calibration Error)}
```

Where $`p_i`$ is the top-1 prediction accuracy in bin $`i`$, $`c_i`$ is the average confidence of predictions in bin $`i`$, and $`b_i`$ is the fraction of data points in bin $`i`$. Bins are constructed in an uniform way in the \[0,1\] range.

As input to `forward` and `update` the metric accepts the following input:

*   `preds` ([`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")): A float tensor of shape `(N, C, ...)` containing probabilities or logits for each observation. If preds has values outside \[0,1\] range we consider the input to be logits and will auto apply softmax per sample.
    
*   `target` ([`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")): An int tensor of shape `(N, ...)` containing ground truth labels, and therefore only contain values in the \[0, n\_classes-1\] range (except if ignore\_index is specified).
    

Tip

Additional dimension `...` will be flattened into the batch dimension.

As output to `forward` and `compute` the metric returns the following output:

*   `mcce` ([`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")): A scalar tensor containing the calibration error
    

Parameters:

*   **num\_classes** ([`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")) – Integer specifying the number of classes
    
*   **n\_bins** ([`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")) – Number of bins to use when computing the metric.
    
*   **norm** ([`Literal`](https://docs.python.org/3/library/typing.html#typing.Literal "(in Python v3.15)")\[`'l1'`, `'l2'`, `'max'`\]) – Norm used to compare empirical and expected probability bins.
    
*   **ignore\_index** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")\]) – Specifies a target value that is ignored and does not contribute to the metric calculation
    
*   **validate\_args** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")) – bool indicating if input arguments and tensors should be validated for correctness. Set to `False` for faster computations.
    
*   **kwargs** ([`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")) – Additional keyword arguments, see [Advanced metric settings](/docs/torchmetrics/1.9.0/user-guide/overview.md#metric-kwargs) for more info.
    

Example

```python
>>> from torch import tensor
>>> from torchmetrics.classification import MulticlassCalibrationError
>>> preds = tensor([[0.25, 0.20, 0.55],
...                 [0.55, 0.05, 0.40],
...                 [0.10, 0.30, 0.60],
...                 [0.90, 0.05, 0.05]])
>>> target = tensor([0, 1, 2, 0])
>>> metric = MulticlassCalibrationError(num_classes=3, n_bins=3, norm='l1')
>>> metric(preds, target)
tensor(0.2000)
>>> mcce = MulticlassCalibrationError(num_classes=3, n_bins=3, norm='l2')
>>> mcce(preds, target)
tensor(0.2082)
>>> mcce = MulticlassCalibrationError(num_classes=3, n_bins=3, norm='max')
>>> mcce(preds, target)
tensor(0.2333)
```

#### `plot(val=None, ax=None)` {#torchmetrics.classification.MulticlassCalibrationError.plot}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/classification/calibration_error.py#L302-L342)

Plot a single or multiple values from the metric.

Parameters:

*   **val** (`Union`\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)"), [`Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "(in Python v3.15)")\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")\], [`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")\]) – Either a single result from calling metric.forward or metric.compute or a list of these results. If no value is provided, will automatically call metric.compute and plot that result.
    
*   **ax** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`Axes`](https://matplotlib.org/stable/api/_as_gen/matplotlib.axes.Axes.html#matplotlib.axes.Axes "(in Matplotlib v3.11.2)")\]) – An matplotlib axis object. If provided will add plot to that axis
    

Return type:

[`tuple`](https://docs.python.org/3/builtins/stdtypes.html#tuple "(in Python v3.15)")\[[`Figure`](https://matplotlib.org/stable/api/_as_gen/matplotlib.figure.Figure.html#matplotlib.figure.Figure "(in Matplotlib v3.11.2)"), `Union`\[[`Axes`](https://matplotlib.org/stable/api/_as_gen/matplotlib.axes.Axes.html#matplotlib.axes.Axes "(in Matplotlib v3.11.2)"), [`ndarray`](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray "(in NumPy v2.5)")\]\]

Returns:

Figure object and Axes object

Raises:

[**ModuleNotFoundError**](https://docs.python.org/3/builtins/exceptions.html#ModuleNotFoundError "(in Python v3.15)") – If matplotlib is not installed

```python
>>> from torch import randn, randint
>>> # Example plotting a single value
>>> from torchmetrics.classification import MulticlassCalibrationError
>>> metric = MulticlassCalibrationError(num_classes=3, n_bins=3, norm='l1')
>>> metric.update(randn(20,3).softmax(dim=-1), randint(3, (20,)))
>>> fig_, ax_ = metric.plot()
```

[![../../\_images/calibration\_error-3.png](/docs/torchmetrics/1.9.0/static/_images/calibration_error-3.png)](/docs/torchmetrics/1.9.0/static/_images/calibration_error-3.png)

```python
>>> from torch import randn, randint
>>> # Example plotting a multiple values
>>> from torchmetrics.classification import MulticlassCalibrationError
>>> metric = MulticlassCalibrationError(num_classes=3, n_bins=3, norm='l1')
>>> values = []
>>> for _ in range(20):
...     values.append(metric(randn(20,3).softmax(dim=-1), randint(3, (20,))))
>>> fig_, ax_ = metric.plot(values)
```

[![../../\_images/calibration\_error-4.png](/docs/torchmetrics/1.9.0/static/_images/calibration_error-4.png)](/docs/torchmetrics/1.9.0/static/_images/calibration_error-4.png)

## Functional Interface {#functional-interface}

### `torchmetrics.functional.calibration_error(preds, target, task, n_bins=15, norm='l1', num_classes=None, ignore_index=None, validate_args=True)` {#torchmetrics.functional.calibration_error}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/functional/classification/calibration_error.py#L321-L365)

[Top-label Calibration Error](https://arxiv.org/abs/1909.10155).

The expected calibration error can be used to quantify how well a given model is calibrated e.g. how well the predicted output probabilities of the model matches the actual probabilities of the ground truth distribution. Three different norms are implemented, each corresponding to variations on the calibration error metric. :rtype: [`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")

```math
\text{ECE} = \sum_i^N b_i \|(p_i - c_i)\|, \text{L1 norm (Expected Calibration Error)}
```

```math
\text{MCE} = \max_{i} (p_i - c_i), \text{Infinity norm (Maximum Calibration Error)}
```

```math
\text{RMSCE} = \sqrt{\sum_i^N b_i(p_i - c_i)^2}, \text{L2 norm (Root Mean Square Calibration Error)}
```

Where $`p_i`$ is the top-1 prediction accuracy in bin $`i`$, $`c_i`$ is the average confidence of predictions in bin $`i`$, and $`b_i`$ is the fraction of data points in bin $`i`$. Bins are constructed in an uniform way in the \[0,1\] range.

This function is a simple wrapper to get the task specific versions of this metric, which is done by setting the `task` argument to either `'binary'` or `'multiclass'`. See the documentation of [`binary_calibration_error()`](#torchmetrics.functional.classification.binary_calibration_error "torchmetrics.functional.classification.binary_calibration_error") and [`multiclass_calibration_error()`](#torchmetrics.functional.classification.multiclass_calibration_error "torchmetrics.functional.classification.multiclass_calibration_error") for the specific details of each argument influence and examples.

### binary_calibration_error {#binary-calibration-error}

### `torchmetrics.functional.classification.binary_calibration_error(preds, target, n_bins=15, norm='l1', ignore_index=None, validate_args=True)` {#torchmetrics.functional.classification.binary_calibration_error}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/functional/classification/calibration_error.py#L142-L208)

[Top-label Calibration Error](https://arxiv.org/abs/1909.10155) for binary tasks.

The expected calibration error can be used to quantify how well a given model is calibrated e.g. how well the predicted output probabilities of the model matches the actual probabilities of the ground truth distribution. Three different norms are implemented, each corresponding to variations on the calibration error metric.

```math
\text{ECE} = \sum_i^N b_i \|(p_i - c_i)\|, \text{L1 norm (Expected Calibration Error)}
```

```math
\text{MCE} = \max_{i} (p_i - c_i), \text{Infinity norm (Maximum Calibration Error)}
```

```math
\text{RMSCE} = \sqrt{\sum_i^N b_i(p_i - c_i)^2}, \text{L2 norm (Root Mean Square Calibration Error)}
```

Where $`p_i`$ is the top-1 prediction accuracy in bin $`i`$, $`c_i`$ is the average confidence of predictions in bin $`i`$, and $`b_i`$ is the fraction of data points in bin $`i`$. Bins are constructed in an uniform way in the \[0,1\] range.

Accepts the following input tensors:

*   `preds` (float tensor): `(N, ...)`. Preds should be a tensor containing probabilities or logits for each observation. If preds has values outside \[0,1\] range we consider the input to be logits and will auto apply sigmoid per element.
    
*   `target` (int tensor): `(N, ...)`. Target should be a tensor containing ground truth labels, and therefore only contain {0,1} values (except if ignore\_index is specified). The value 1 always encodes the positive class.
    

Additional dimension `...` will be flattened into the batch dimension.

Parameters:

*   **preds** ([`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")) – Tensor with predictions
    
*   **target** ([`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")) – Tensor with true labels
    
*   **n\_bins** ([`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")) – Number of bins to use when computing the metric.
    
*   **norm** ([`Literal`](https://docs.python.org/3/library/typing.html#typing.Literal "(in Python v3.15)")\[`'l1'`, `'l2'`, `'max'`\]) – Norm used to compare empirical and expected probability bins.
    
*   **ignore\_index** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")\]) – Specifies a target value that is ignored and does not contribute to the metric calculation
    
*   **validate\_args** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")) – bool indicating if input arguments and tensors should be validated for correctness. Set to `False` for faster computations.
    

Return type:

[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")

Example

```python
>>> from torchmetrics.functional.classification import binary_calibration_error
>>> preds = torch.tensor([0.25, 0.25, 0.55, 0.75, 0.75])
>>> target = torch.tensor([0, 0, 1, 1, 1])
>>> binary_calibration_error(preds, target, n_bins=2, norm='l1')
tensor(0.2900)
>>> binary_calibration_error(preds, target, n_bins=2, norm='l2')
tensor(0.2918)
>>> binary_calibration_error(preds, target, n_bins=2, norm='max')
tensor(0.3167)
```

### multiclass_calibration_error {#multiclass-calibration-error}

### `torchmetrics.functional.classification.multiclass_calibration_error(preds, target, num_classes, n_bins=15, norm='l1', ignore_index=None, validate_args=True)` {#torchmetrics.functional.classification.multiclass_calibration_error}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/functional/classification/calibration_error.py#L249-L318)

[Top-label Calibration Error](https://arxiv.org/abs/1909.10155) for multiclass tasks.

The expected calibration error can be used to quantify how well a given model is calibrated e.g. how well the predicted output probabilities of the model matches the actual probabilities of the ground truth distribution. Three different norms are implemented, each corresponding to variations on the calibration error metric.

```math
\text{ECE} = \sum_i^N b_i \|(p_i - c_i)\|, \text{L1 norm (Expected Calibration Error)}
```

```math
\text{MCE} = \max_{i} (p_i - c_i), \text{Infinity norm (Maximum Calibration Error)}
```

```math
\text{RMSCE} = \sqrt{\sum_i^N b_i(p_i - c_i)^2}, \text{L2 norm (Root Mean Square Calibration Error)}
```

Where $`p_i`$ is the top-1 prediction accuracy in bin $`i`$, $`c_i`$ is the average confidence of predictions in bin $`i`$, and $`b_i`$ is the fraction of data points in bin $`i`$. Bins are constructed in an uniform way in the \[0,1\] range.

Accepts the following input tensors:

*   `preds` (float tensor): `(N, C, ...)`. Preds should be a tensor containing probabilities or logits for each observation. If preds has values outside \[0,1\] range we consider the input to be logits and will auto apply softmax per sample.
    
*   `target` (int tensor): `(N, ...)`. Target should be a tensor containing ground truth labels, and therefore only contain values in the \[0, n\_classes-1\] range (except if ignore\_index is specified).
    

Additional dimension `...` will be flattened into the batch dimension.

Parameters:

*   **preds** ([`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")) – Tensor with predictions
    
*   **target** ([`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")) – Tensor with true labels
    
*   **num\_classes** ([`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")) – Integer specifying the number of classes
    
*   **n\_bins** ([`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")) – Number of bins to use when computing the metric.
    
*   **norm** ([`Literal`](https://docs.python.org/3/library/typing.html#typing.Literal "(in Python v3.15)")\[`'l1'`, `'l2'`, `'max'`\]) – Norm used to compare empirical and expected probability bins.
    
*   **ignore\_index** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")\]) – Specifies a target value that is ignored and does not contribute to the metric calculation
    
*   **validate\_args** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")) – bool indicating if input arguments and tensors should be validated for correctness. Set to `False` for faster computations.
    

Return type:

[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")

Example

```python
>>> from torchmetrics.functional.classification import multiclass_calibration_error
>>> preds = torch.tensor([[0.25, 0.20, 0.55],
...                       [0.55, 0.05, 0.40],
...                       [0.10, 0.30, 0.60],
...                       [0.90, 0.05, 0.05]])
>>> target = torch.tensor([0, 1, 2, 0])
>>> multiclass_calibration_error(preds, target, num_classes=3, n_bins=3, norm='l1')
tensor(0.2000)
>>> multiclass_calibration_error(preds, target, num_classes=3, n_bins=3, norm='l2')
tensor(0.2082)
>>> multiclass_calibration_error(preds, target, num_classes=3, n_bins=3, norm='max')
tensor(0.2333)
```

---

**Navigation**

- Parent: [Classification](/docs/torchmetrics/1.9.0/classification.md)