---
title: ROUGE Score
description: 'Calculate Rouge Score, used for automatic summarization.'
canonical: 'https://lightning.ai/docs/torchmetrics/stable/text/rouge_score'
---
# ROUGE Score {#rouge-score}

## Module Interface {#module-interface}

### `class torchmetrics.text.rouge.ROUGEScore(use_stemmer=False, normalizer=None, tokenizer=None, accumulate='best', rouge_keys=('rouge1', 'rouge2', 'rougeL', 'rougeLsum'), **kwargs)` {#torchmetrics.text.rouge.ROUGEScore}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/text/rouge.py#L37-L237)

[Calculate Rouge Score](https://en.wikipedia.org/wiki/ROUGE_\(metric\)), used for automatic summarization.

This implementation should imitate the behaviour of the `rouge-score` package Python ROUGE Implementation

As input to `forward` and `update` the metric accepts the following input:

*   `preds` (`Sequence`): An iterable of predicted sentences or a single predicted sentence
    
*   `target` (`Sequence`): An iterable of target sentences or an iterable of interables of target sentences or a single target sentence
    

As output of `forward` and `compute` the metric returns the following output:

*   `rouge` (`Dict`): A dictionary of tensor rouge scores for each input str rouge key
    

Parameters:

*   **use\_stemmer** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")) – Use Porter stemmer to strip word suffixes to improve matching.
    
*   **normalizer** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`Callable`](https://docs.python.org/3/library/typing.html#typing.Callable "(in Python v3.15)")\[\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\], [`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\]\]) – A user’s own normalizer function. If this is `None`, replacing any non-alpha-numeric characters with spaces is default. This function must take a `str` and return a `str`.
    
*   **tokenizer** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`Callable`](https://docs.python.org/3/library/typing.html#typing.Callable "(in Python v3.15)")\[\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\], [`Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\]\]\]) – A user’s own tokenizer function. If this is `None`, splitting by spaces is default This function must take a `str` and return `Sequence[str]`
    
*   **accumulate** ([`Literal`](https://docs.python.org/3/library/typing.html#typing.Literal "(in Python v3.15)")\[`'avg'`, `'best'`\]) –
    
    Useful in case of multi-reference rouge score.
    
    *   `avg` takes the avg of all references with respect to predictions
        
    *   `best` takes the best fmeasure score obtained between prediction and multiple corresponding references.
        
    
*   **rouge\_keys** (`Union`\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`tuple`](https://docs.python.org/3/builtins/stdtypes.html#tuple "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`...`](https://docs.python.org/3/builtins/constants.html#Ellipsis "(in Python v3.15)")\]\]) – A list of rouge types to calculate. Keys that are allowed are `rougeL`, `rougeLsum`, and `rouge1` through `rouge9`.
    
*   **kwargs** ([`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")) – Additional keyword arguments, see [Advanced metric settings](/docs/torchmetrics/1.9.0/user-guide/overview.md#metric-kwargs) for more info.
    

Example

```python
>>> from torchmetrics.text.rouge import ROUGEScore
>>> preds = "My name is John"
>>> target = "Is your name John"
>>> rouge = ROUGEScore()
>>> from pprint import pprint
>>> pprint(rouge(preds, target))
{'rouge1_fmeasure': tensor(0.7500),
 'rouge1_precision': tensor(0.7500),
 'rouge1_recall': tensor(0.7500),
 'rouge2_fmeasure': tensor(0.),
 'rouge2_precision': tensor(0.),
 'rouge2_recall': tensor(0.),
 'rougeL_fmeasure': tensor(0.5000),
 'rougeL_precision': tensor(0.5000),
 'rougeL_recall': tensor(0.5000),
 'rougeLsum_fmeasure': tensor(0.5000),
 'rougeLsum_precision': tensor(0.5000),
 'rougeLsum_recall': tensor(0.5000)}
```

Raises:

*   [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError "(in Python v3.15)") – If the python packages `nltk` is not installed.
    
*   [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError "(in Python v3.15)") – If any of the `rouge_keys` does not belong to the allowed set of keys.
    

#### `plot(val=None, ax=None)` {#torchmetrics.text.rouge.ROUGEScore.plot}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/text/rouge.py#L195-L237)

Plot a single or multiple values from the metric.

Parameters:

*   **val** (`Union`\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)"), [`Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "(in Python v3.15)")\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")\], [`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")\]) – Either a single result from calling metric.forward or metric.compute or a list of these results. If no value is provided, will automatically call metric.compute and plot that result.
    
*   **ax** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`Axes`](https://matplotlib.org/stable/api/_as_gen/matplotlib.axes.Axes.html#matplotlib.axes.Axes "(in Matplotlib v3.11.2)")\]) – An matplotlib axis object. If provided will add plot to that axis
    

Return type:

[`tuple`](https://docs.python.org/3/builtins/stdtypes.html#tuple "(in Python v3.15)")\[[`Figure`](https://matplotlib.org/stable/api/_as_gen/matplotlib.figure.Figure.html#matplotlib.figure.Figure "(in Matplotlib v3.11.2)"), `Union`\[[`Axes`](https://matplotlib.org/stable/api/_as_gen/matplotlib.axes.Axes.html#matplotlib.axes.Axes "(in Matplotlib v3.11.2)"), [`ndarray`](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray "(in NumPy v2.5)")\]\]

Returns:

Figure and Axes object

Raises:

[**ModuleNotFoundError**](https://docs.python.org/3/builtins/exceptions.html#ModuleNotFoundError "(in Python v3.15)") – If matplotlib is not installed

```python
>>> # Example plotting a single value
>>> from torchmetrics.text.rouge import ROUGEScore
>>> metric = ROUGEScore()
>>> preds = "My name is John"
>>> target = "Is your name John"
>>> metric.update(preds, target)
>>> fig_, ax_ = metric.plot()
```

```python
>>> # Example plotting multiple values
>>> from torchmetrics.text.rouge import ROUGEScore
>>> metric = ROUGEScore()
>>> preds = "My name is John"
>>> target = "Is your name John"
>>> values = [ ]
>>> for _ in range(10):
...     values.append(metric(preds, target))
>>> fig_, ax_ = metric.plot(values)
```

## Functional Interface {#functional-interface}

### `torchmetrics.functional.text.rouge.rouge_score(preds, target, accumulate='best', use_stemmer=False, normalizer=None, tokenizer=None, rouge_keys=('rouge1', 'rouge2', 'rougeL', 'rougeLsum'))` {#torchmetrics.functional.text.rouge.rouge_score}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/functional/text/rouge.py#L409-L513)

Calculate [Calculate Rouge Score](https://en.wikipedia.org/wiki/ROUGE_\(metric\)) , used for automatic summarization.

Parameters:

*   **preds** (`Union`\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\]\]) – An iterable of predicted sentences or a single predicted sentence.
    
*   **target** (`Union`\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\], [`Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "(in Python v3.15)")\[[`Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\]\]\]) – An iterable of iterables of target sentences or an iterable of target sentences or a single target sentence.
    
*   **accumulate** ([`Literal`](https://docs.python.org/3/library/typing.html#typing.Literal "(in Python v3.15)")\[`'avg'`, `'best'`\]) –
    
    Useful in case of multi-reference rouge score.
    
    *   `avg` takes the avg of all references with respect to predictions
        
    *   `best` takes the best fmeasure score obtained between prediction and multiple corresponding references.
        
    
*   **use\_stemmer** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool "(in Python v3.15)")) – Use Porter stemmer to strip word suffixes to improve matching.
    
*   **normalizer** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`Callable`](https://docs.python.org/3/library/typing.html#typing.Callable "(in Python v3.15)")\[\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\], [`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\]\]) – A user’s own normalizer function. If this is `None`, replacing any non-alpha-numeric characters with spaces is default. This function must take a `str` and return a `str`.
    
*   **tokenizer** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`Callable`](https://docs.python.org/3/library/typing.html#typing.Callable "(in Python v3.15)")\[\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\], [`Sequence`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\]\]\]) – A user’s own tokenizer function. If this is `None`, splitting by spaces is default This function must take a `str` and return `Sequence[str]`
    
*   **rouge\_keys** (`Union`\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`tuple`](https://docs.python.org/3/builtins/stdtypes.html#tuple "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`...`](https://docs.python.org/3/builtins/constants.html#Ellipsis "(in Python v3.15)")\]\]) – A list of rouge types to calculate. Keys that are allowed are `rougeL`, `rougeLsum`, and `rouge1` through `rouge9`.
    

Return type:

[`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)"), [`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")\]

Returns:

Python dictionary of rouge scores for each input rouge key.

Example

```python
>>> from torchmetrics.functional.text.rouge import rouge_score
>>> preds = "My name is John"
>>> target = "Is your name John"
>>> from pprint import pprint
>>> pprint(rouge_score(preds, target))
{'rouge1_fmeasure': tensor(0.7500),
 'rouge1_precision': tensor(0.7500),
 'rouge1_recall': tensor(0.7500),
 'rouge2_fmeasure': tensor(0.),
 'rouge2_precision': tensor(0.),
 'rouge2_recall': tensor(0.),
 'rougeL_fmeasure': tensor(0.5000),
 'rougeL_precision': tensor(0.5000),
 'rougeL_recall': tensor(0.5000),
 'rougeLsum_fmeasure': tensor(0.5000),
 'rougeLsum_precision': tensor(0.5000),
 'rougeLsum_recall': tensor(0.5000)}
```

Raises:

*   [**ModuleNotFoundError**](https://docs.python.org/3/builtins/exceptions.html#ModuleNotFoundError "(in Python v3.15)") – If the python package `nltk` is not installed.
    
*   [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError "(in Python v3.15)") – If any of the `rouge_keys` does not belong to the allowed set of keys.
    

References

\[1\] ROUGE: A Package for Automatic Evaluation of Summaries by Chin-Yew Lin. [https://aclanthology.org/W04-1013/](https://aclanthology.org/W04-1013/)

---

**Navigation**

- Parent: [Text](/docs/torchmetrics/1.9.0/text.md)