---
title: ChrF Score
description: Calculate chrf score of machine translated text with one or more references.
canonical: 'https://lightning.ai/docs/torchmetrics/stable/text/chrf_score'
---
# ChrF Score {#chrf-score}

## Module Interface {#module-interface}

### `class torchmetrics.text.CHRFScore(n_char_order=6, n_word_order=2, beta=2.0, lowercase=False, whitespace=False, return_sentence_level_score=False, **kwargs)` {#torchmetrics.text.CHRFScore}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/text/chrf.py#L53-L250)

Calculate [chrf score](https://aclanthology.org/W15-3049) of machine translated text with one or more references.

This implementation supports both ChrF score computation introduced in [chrF score](https://aclanthology.org/W15-3049) and [chrF++ score](https://aclanthology.org/W17-4770) introduced in [chrF++ score](https://aclanthology.org/W17-4770). This implementation follows the implementations from [https://github.com/m-popovic/chrF](https://github.com/m-popovic/chrF) and [https://github.com/mjpost/sacrebleu/blob/master/sacrebleu/metrics/chrf.py](https://github.com/mjpost/sacrebleu/blob/master/sacrebleu/metrics/chrf.py).

As input to `forward` and `update` the metric accepts the following input:

*   `preds` (`Sequence`): An iterable of hypothesis corpus
    
*   `target` (`Sequence`): An iterable of iterables of reference corpus
    

As output of `forward` and `compute` the metric returns the following output:

*   `chrf` ([`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")): If return\_sentence\_level\_score=True return a list of sentence-level chrF/chrF++ scores, else return a corpus-level chrF/chrF++ score
    

Parameters:

*   **n\_char\_order** ([`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")) – A character n-gram order. If `n_char_order=6`, the metrics refers to the official chrF/chrF++.
    
*   **n\_word\_order** ([`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")) – A word n-gram order. If `n_word_order=2`, the metric refers to the official chrF++. If `n_word_order=0`, the metric is equivalent to the original ChrF.
    
*   **beta** ([`float`](https://docs.python.org/3/builtins/functions.html#float "(in Python v3.15)")) – parameter determining an importance of recall w.r.t. precision. If `beta=1`, their importance is equal.
    
*   **lowercase** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")) – An indication whether to enable case-insensitivity.
    
*   **whitespace** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")) – An indication whether keep whitespaces during n-gram extraction.
    
*   **return\_sentence\_level\_score** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")) – An indication whether a sentence-level chrF/chrF++ score to be returned.
    
*   **kwargs** ([`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")) – Additional keyword arguments, see [Advanced metric settings](/docs/torchmetrics/1.9.0/user-guide/overview.md#metric-kwargs) for more info.
    

Raises:

*   [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError "(in Python v3.15)") – If `n_char_order` is not an integer greater than or equal to 1.
    
*   [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError "(in Python v3.15)") – If `n_word_order` is not an integer greater than or equal to 0.
    
*   [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError "(in Python v3.15)") – If `beta` is smaller than 0.
    

Example

```python
>>> from torchmetrics.text import CHRFScore
>>> preds = ['the cat is on the mat']
>>> target = [['there is a cat on the mat', 'a cat is on the mat']]
>>> chrf = CHRFScore()
>>> chrf(preds, target)
tensor(0.8640)
```

#### `plot(val=None, ax=None)` {#torchmetrics.text.CHRFScore.plot}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/text/chrf.py#L208-L250)

Plot a single or multiple values from the metric.

Parameters:

*   **val** (`Union`\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)"), [`Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "(in Python v3.15)")\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")\], [`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")\]) – Either a single result from calling metric.forward or metric.compute or a list of these results. If no value is provided, will automatically call metric.compute and plot that result.
    
*   **ax** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`Axes`](https://matplotlib.org/stable/api/_as_gen/matplotlib.axes.Axes.html#matplotlib.axes.Axes "(in Matplotlib v3.11.2)")\]) – An matplotlib axis object. If provided will add plot to that axis
    

Return type:

[`tuple`](https://docs.python.org/3/builtins/stdtypes.html#tuple "(in Python v3.15)")\[[`Figure`](https://matplotlib.org/stable/api/_as_gen/matplotlib.figure.Figure.html#matplotlib.figure.Figure "(in Matplotlib v3.11.2)"), `Union`\[[`Axes`](https://matplotlib.org/stable/api/_as_gen/matplotlib.axes.Axes.html#matplotlib.axes.Axes "(in Matplotlib v3.11.2)"), [`ndarray`](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray "(in NumPy v2.5)")\]\]

Returns:

Figure and Axes object

Raises:

[**ModuleNotFoundError**](https://docs.python.org/3/builtins/exceptions.html#ModuleNotFoundError "(in Python v3.15)") – If matplotlib is not installed

```python
>>> # Example plotting a single value
>>> from torchmetrics.text import CHRFScore
>>> metric = CHRFScore()
>>> preds = ['the cat is on the mat']
>>> target = [['there is a cat on the mat', 'a cat is on the mat']]
>>> metric.update(preds, target)
>>> fig_, ax_ = metric.plot()
```

[![../../\_images/chrf\_score-1.png](/docs/torchmetrics/1.9.0/static/_images/chrf_score-1.png)](/docs/torchmetrics/1.9.0/static/_images/chrf_score-1.png)

```python
>>> # Example plotting multiple values
>>> from torchmetrics.text import CHRFScore
>>> metric = CHRFScore()
>>> preds = ['the cat is on the mat']
>>> target = [['there is a cat on the mat', 'a cat is on the mat']]
>>> values = [ ]
>>> for _ in range(10):
...     values.append(metric(preds, target))
>>> fig_, ax_ = metric.plot(values)
```

[![../../\_images/chrf\_score-2.png](/docs/torchmetrics/1.9.0/static/_images/chrf_score-2.png)](/docs/torchmetrics/1.9.0/static/_images/chrf_score-2.png)

## Functional Interface {#functional-interface}

### `torchmetrics.functional.text.chrf_score(preds, target, n_char_order=6, n_word_order=2, beta=2.0, lowercase=False, whitespace=False, return_sentence_level_score=False)` {#torchmetrics.functional.text.chrf_score}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/functional/text/chrf.py#L524-L638)

Calculate [chrF score](https://aclanthology.org/W15-3049) of machine translated text with one or more references.

This implementation supports both chrF score computation introduced in \[1\] and chrF++ score introduced in [chrF++ score](https://aclanthology.org/W17-4770). This implementation follows the implementations from [https://github.com/m-popovic/chrF](https://github.com/m-popovic/chrF) and [https://github.com/mjpost/sacrebleu/blob/master/sacrebleu/metrics/chrf.py](https://github.com/mjpost/sacrebleu/blob/master/sacrebleu/metrics/chrf.py).

Parameters:

*   **preds** (`Union`\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\]\]) – An iterable of hypothesis corpus.
    
*   **target** ([`Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "(in Python v3.15)")\[`Union`\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\]\]\]) – An iterable of iterables of reference corpus.
    
*   **n\_char\_order** ([`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")) – A character n-gram order. If n\_char\_order=6, the metrics refers to the official chrF/chrF++.
    
*   **n\_word\_order** ([`int`](https://docs.python.org/3/builtins/functions.html#int "(in Python v3.15)")) – A word n-gram order. If n\_word\_order=2, the metric refers to the official chrF++. If n\_word\_order=0, the metric is equivalent to the original chrF.
    
*   **beta** ([`float`](https://docs.python.org/3/builtins/functions.html#float "(in Python v3.15)")) – A parameter determining an importance of recall w.r.t. precision. If beta=1, their importance is equal.
    
*   **lowercase** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")) – An indication whether to enable case-insensitivity.
    
*   **whitespace** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")) – An indication whether to keep whitespaces during character n-gram extraction.
    
*   **return\_sentence\_level\_score** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")) – An indication whether a sentence-level chrF/chrF++ score to be returned.
    

Return type:

`Union`\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)"), [`tuple`](https://docs.python.org/3/builtins/stdtypes.html#tuple "(in Python v3.15)")\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)"), [`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")\]\]

Returns:

A corpus-level chrF/chrF++ score. (Optionally) A list of sentence-level chrF/chrF++ scores if return\_sentence\_level\_score=True.

Raises:

*   [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError "(in Python v3.15)") – If `n_char_order` is not an integer greater than or equal to 1.
    
*   [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError "(in Python v3.15)") – If `n_word_order` is not an integer greater than or equal to 0.
    
*   [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError "(in Python v3.15)") – If `beta` is smaller than 0.
    

Example

```python
>>> from torchmetrics.functional.text import chrf_score
>>> preds = ['the cat is on the mat']
>>> target = [['there is a cat on the mat', 'a cat is on the mat']]
>>> chrf_score(preds, target)
tensor(0.8640)
```

References

\[1\] chrF: character n-gram F-score for automatic MT evaluation by Maja Popović [chrF score](https://aclanthology.org/W15-3049)

\[2\] chrF++: words helping character n-grams by Maja Popović [chrF++ score](https://aclanthology.org/W17-4770)

---

**Navigation**

- Parent: [Text](/docs/torchmetrics/1.9.0/text.md)