---
title: CLIP Score
description: Calculates CLIP Score which is a text-to-image similarity metric.
canonical: 'https://lightning.ai/docs/torchmetrics/stable/multimodal/clip_score'
---
# CLIP Score {#clip-score}

## Module Interface {#module-interface}

### `class torchmetrics.multimodal.clip_score.CLIPScore(model_name_or_path='openai/clip-vit-large-patch14', **kwargs)` {#torchmetrics.multimodal.clip_score.CLIPScore}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/multimodal/clip_score.py#L49-L263)

Calculates [CLIP Score](https://arxiv.org/abs/2104.08718) which is a text-to-image similarity metric.

CLIP Score is a reference free metric that can be used to evaluate the correlation between a generated caption for an image and the actual content of the image, as well as the similarity between texts or images. It has been found to be highly correlated with human judgement. The metric is defined as:

```math
\text{CLIPScore(I, C)} = max(100 * cos(E_I, E_C), 0)
```

which corresponds to the cosine similarity between visual [CLIP](https://arxiv.org/abs/2103.00020) embedding $`E_i`$ for an image $`i`$ and textual CLIP embedding $`E_C`$ for an caption $`C`$. The score is bound between 0 and 100 and the closer to 100 the better.

Additionally, the CLIP Score can be calculated for the same modalities:

```math
\text{CLIPScore(I_1, I_2)} = max(100 * cos(E_{I_1}, E_{I_2}), 0)
```

where $`E_{I_1}`$ and $`E_{I_2}`$ are the visual embeddings for images $`I_1`$ and $`I_2`$.

```math
\text{CLIPScore(T_1, T_2)} = max(100 * cos(E_{T_1}, E_{T_2}), 0)
```

where $`E_{T_1}`$ and $`E_{T_2}`$ are the textual embeddings for texts $`T_1`$ and $`T_2`$.

Caution

Metric is not scriptable

Note

The default CLIP and processor used in this implementation has a maximum sequence length of 77 for text inputs. If you need to process longer captions, you can use the zer0int/LongCLIP-L-Diffusers model which has a maximum sequence length of 248.

As input to `forward` and `update` the metric accepts the following input

*   source: Source input.
    
    > This can be:
    > 
    > *   Images: `Tensor` or list of `Tensor`
    >     
    >     > If a single tensor, it should have shape `(N, C, H, W)`. If a list of tensors, each tensor should have shape `(C, H, W)`. `C` is the number of channels, `H` and `W` are the height and width of the image.
    >     
    > *   Text: `str` or list of `str`
    >     
    >     > Either a single caption or a list of captions.
    >     
    
*   target: Target input.
    
    > This can be:
    > 
    > *   Images: `Tensor` or list of `Tensor`
    >     
    >     > If a single tensor, it should have shape `(N, C, H, W)`. If a list of tensors, each tensor should have shape `(C, H, W)`. `C` is the number of channels, `H` and `W` are the height and width of the image.
    >     
    > *   Text: `str` or list of `str`
    >     
    >     > Either a single caption or a list of captions.
    >     
    

As output of forward and compute the metric returns the following output

*   `clip_score` ([`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")): float scalar tensor with mean CLIP score over samples
    

Parameters:

*   **model\_name\_or\_path** (`Union`\[[`Literal`](https://docs.python.org/3/library/typing.html#typing.Literal "(in Python v3.15)")\[`'openai/clip-vit-base-patch16'`, `'openai/clip-vit-base-patch32'`, `'openai/clip-vit-large-patch14-336'`, `'openai/clip-vit-large-patch14'`, `'jinaai/jina-clip-v2'`, `'zer0int/LongCLIP-L-Diffusers'`, `'zer0int/LongCLIP-GmP-ViT-L-14'`\], [`Callable`](https://docs.python.org/3/library/typing.html#typing.Callable "(in Python v3.15)")\[\[\], [`tuple`](https://docs.python.org/3/builtins/stdtypes.html#tuple "(in Python v3.15)")\[[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)"), [`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")\]\]\]) –
    
    string indicating the version of the CLIP model to use. Available models are:
    
    *   ”openai/clip-vit-base-patch16”
        
    *   ”openai/clip-vit-base-patch32”
        
    *   ”openai/clip-vit-large-patch14-336”
        
    *   ”openai/clip-vit-large-patch14”
        
    *   ”jinaai/jina-clip-v2”
        
    *   ”zer0int/LongCLIP-L-Diffusers”
        
    *   ”zer0int/LongCLIP-GmP-ViT-L-14”
        
    
    Alternatively, a callable function that returns a tuple of CLIP compatible model and processor instances can be passed in. By compatible, we mean that the processors \_\_call\_\_ method should accept a list of strings and list of images and that the model should have a get\_image\_features and get\_text\_features methods.
    
*   **kwargs** ([`Any`](https://docs.python.org/3/library/typing.html#typing.Any "(in Python v3.15)")) – Additional keyword arguments, see [Advanced metric settings](/docs/torchmetrics/1.9.0/user-guide/overview.md#metric-kwargs) for more info.
    

Raises:

[**ModuleNotFoundError**](https://docs.python.org/3/builtins/exceptions.html#ModuleNotFoundError "(in Python v3.15)") – If transformers package is not installed or version is lower than 4.10.0

Example

```python
>>> from torchmetrics.multimodal.clip_score import CLIPScore
>>> metric = CLIPScore(model_name_or_path="openai/clip-vit-base-patch16")
>>> image = torch.randint(255, (3, 224, 224), generator=torch.Generator().manual_seed(42))
>>> score = metric(image, "a photo of a cat")
>>> score.detach().round()
tensor(24.)
```

Example

```python
>>> from torchmetrics.multimodal.clip_score import CLIPScore
>>> metric = CLIPScore(model_name_or_path="openai/clip-vit-base-patch16")
>>> image1 = torch.randint(255, (3, 224, 224), generator=torch.Generator().manual_seed(42))
>>> image2 = torch.randint(255, (3, 224, 224), generator=torch.Generator().manual_seed(43))
>>> score = metric(image1, image2)
>>> score.detach().round()
tensor(99.)
```

Example

```python
>>> from torchmetrics.multimodal.clip_score import CLIPScore
>>> metric = CLIPScore(model_name_or_path="openai/clip-vit-base-patch16")
>>> score = metric("28-year-old chef found dead in San Francisco mall",
...               "A 28-year-old chef who recently moved to San Francisco was found dead.")
>>> score.detach().round()
tensor(91.)
```

#### `compute()` {#torchmetrics.multimodal.clip_score.CLIPScore.compute}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/multimodal/clip_score.py#L221-L223)

Compute accumulated clip score.

Return type:

[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")

#### `plot(val=None, ax=None)` {#torchmetrics.multimodal.clip_score.CLIPScore.plot}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/multimodal/clip_score.py#L225-L263)

Plot a single or multiple values from the metric.

Parameters:

*   **val** (`Union`\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)"), [`Sequence`](https://docs.python.org/3/library/typing.html#typing.Sequence "(in Python v3.15)")\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")\], [`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")\]) – Either a single result from calling metric.forward or metric.compute or a list of these results. If no value is provided, will automatically call metric.compute and plot that result.
    
*   **ax** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional "(in Python v3.15)")\[[`Axes`](https://matplotlib.org/stable/api/_as_gen/matplotlib.axes.Axes.html#matplotlib.axes.Axes "(in Matplotlib v3.11.2)")\]) – An matplotlib axis object. If provided will add plot to that axis
    

Return type:

[`tuple`](https://docs.python.org/3/builtins/stdtypes.html#tuple "(in Python v3.15)")\[[`Figure`](https://matplotlib.org/stable/api/_as_gen/matplotlib.figure.Figure.html#matplotlib.figure.Figure "(in Matplotlib v3.11.2)"), `Union`\[[`Axes`](https://matplotlib.org/stable/api/_as_gen/matplotlib.axes.Axes.html#matplotlib.axes.Axes "(in Matplotlib v3.11.2)"), [`ndarray`](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.html#numpy.ndarray "(in NumPy v2.5)")\]\]

Returns:

Figure and Axes object

Raises:

[**ModuleNotFoundError**](https://docs.python.org/3/builtins/exceptions.html#ModuleNotFoundError "(in Python v3.15)") – If matplotlib is not installed

```python
>>> # Example plotting a single value
>>> import torch
>>> from torchmetrics.multimodal.clip_score import CLIPScore
>>> metric = CLIPScore(model_name_or_path="openai/clip-vit-base-patch16")
>>> metric.update(torch.randint(255, (3, 224, 224)), "a photo of a cat")
>>> fig_, ax_ = metric.plot()
```

```python
>>> # Example plotting multiple values
>>> import torch
>>> from torchmetrics.multimodal.clip_score import CLIPScore
>>> metric = CLIPScore(model_name_or_path="openai/clip-vit-base-patch16")
>>> values = [ ]
>>> for _ in range(10):
...     values.append(metric(torch.randint(255, (3, 224, 224)), "a photo of a cat"))
>>> fig_, ax_ = metric.plot(values)
```

#### `update(source, target)` {#torchmetrics.multimodal.clip_score.CLIPScore.update}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/multimodal/clip_score.py#L197-L219)

Update CLIP score on a batch of images and text.

Parameters:

*   **source** (`Union`\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)"), [`List`](https://docs.python.org/3/library/typing.html#typing.List "(in Python v3.15)")\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")\], [`List`](https://docs.python.org/3/library/typing.html#typing.List "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\], [`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\]) – Source input. This can be: - Images: Either a single \[N, C, H, W\] tensor or a list of \[C, H, W\] tensors. - Text: Either a single caption or a list of captions.
    
*   **target** (`Union`\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)"), [`List`](https://docs.python.org/3/library/typing.html#typing.List "(in Python v3.15)")\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")\], [`List`](https://docs.python.org/3/library/typing.html#typing.List "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\], [`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\]) – Target input. This can be: - Images: Either a single \[N, C, H, W\] tensor or a list of \[C, H, W\] tensors. - Text: Either a single caption or a list of captions.
    

Raises:

*   [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError "(in Python v3.15)") – If not all images have format \[C, H, W\]
    
*   [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError "(in Python v3.15)") – If the number of images and captions do not match
    

Return type:

[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")

## Functional Interface {#functional-interface}

### `torchmetrics.functional.multimodal.clip_score.clip_score(source, target, model_name_or_path='openai/clip-vit-large-patch14')` {#torchmetrics.functional.multimodal.clip_score.clip_score}

[\[source\]](https://github.com/Lightning-AI/torchmetrics/blob/v1.9.0/src/torchmetrics/functional/multimodal/clip_score.py#L248-L354)

Calculates [CLIP Score](https://arxiv.org/abs/2104.08718) which is a text-to-image similarity metric.

CLIP Score is a reference free metric that can be used to evaluate the correlation between a generated caption for an image and the actual content of the image, as well as the similarity between texts or images. It has been found to be highly correlated with human judgement. The metric is defined as:

```math
\text{CLIPScore(I, C)} = max(100 * cos(E_I, E_C), 0)
```

which corresponds to the cosine similarity between visual [CLIP](https://arxiv.org/abs/2103.00020) embedding $`E_i`$ for an image $`i`$ and textual CLIP embedding $`E_C`$ for an caption $`C`$. The score is bound between 0 and 100 and the closer to 100 the better.

Additionally, the CLIP Score can be calculated for the same modalities:

```math
\text{CLIPScore(I_1, I_2)} = max(100 * cos(E_{I_1}, E_{I_2}), 0)
```

where $`E_{I_1}`$ and $`E_{I_2}`$ are the visual embeddings for images $`I_1`$ and $`I_2`$.

```math
\text{CLIPScore(T_1, T_2)} = max(100 * cos(E_{T_1}, E_{T_2}), 0)
```

where $`E_{T_1}`$ and $`E_{T_2}`$ are the textual embeddings for texts $`T_1`$ and $`T_2`$.

Note

Metric is not scriptable

Note

The default CLIP and processor used in this implementation has a maximum sequence length of 77 for text inputs. If you need to process longer captions, you can use the zer0int/LongCLIP-L-Diffusers model which has a maximum sequence length of 248.

Parameters:

*   **source** (`Union`\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)"), [`List`](https://docs.python.org/3/library/typing.html#typing.List "(in Python v3.15)")\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")\], [`List`](https://docs.python.org/3/library/typing.html#typing.List "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\], [`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\]) – Source input. This can be: - Images: Either a single \[N, C, H, W\] tensor or a list of \[C, H, W\] tensors. - Text: Either a single caption or a list of captions.
    
*   **target** (`Union`\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)"), [`List`](https://docs.python.org/3/library/typing.html#typing.List "(in Python v3.15)")\[[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")\], [`List`](https://docs.python.org/3/library/typing.html#typing.List "(in Python v3.15)")\[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\], [`str`](https://docs.python.org/3/builtins/stdtypes.html#str "(in Python v3.15)")\]) – Target input. This can be: - Images: Either a single \[N, C, H, W\] tensor or a list of \[C, H, W\] tensors. - Text: Either a single caption or a list of captions.
    
*   **model\_name\_or\_path** (`Union`\[[`Literal`](https://docs.python.org/3/library/typing.html#typing.Literal "(in Python v3.15)")\[`'openai/clip-vit-base-patch16'`, `'openai/clip-vit-base-patch32'`, `'openai/clip-vit-large-patch14-336'`, `'openai/clip-vit-large-patch14'`, `'jinaai/jina-clip-v2'`, `'zer0int/LongCLIP-L-Diffusers'`, `'zer0int/LongCLIP-GmP-ViT-L-14'`\], [`Callable`](https://docs.python.org/3/library/typing.html#typing.Callable "(in Python v3.15)")\[\[\], [`tuple`](https://docs.python.org/3/builtins/stdtypes.html#tuple "(in Python v3.15)")\[[`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)"), [`None`](https://docs.python.org/3/builtins/constants.html#None "(in Python v3.15)")\]\]\]) –
    
    String indicating the version of the CLIP model to use. Available models are: - “openai/clip-vit-base-patch16” - “openai/clip-vit-base-patch32” - “openai/clip-vit-large-patch14-336” - “openai/clip-vit-large-patch14” - “jinaai/jina-clip-v2” - “zer0int/LongCLIP-L-Diffusers” - “zer0int/LongCLIP-GmP-ViT-L-14”
    
    Alternatively, a callable function that returns a tuple of CLIP compatible model and processor instances can be passed in. By compatible, we mean that the processors \_\_call\_\_ method should accept a list of strings and list of images and that the model should have a get\_image\_features and get\_text\_features methods.
    

Raises:

*   [**ModuleNotFoundError**](https://docs.python.org/3/builtins/exceptions.html#ModuleNotFoundError "(in Python v3.15)") – If transformers package is not installed or version is lower than 4.10.0
    
*   [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError "(in Python v3.15)") – If not all images have format \[C, H, W\]
    
*   [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError "(in Python v3.15)") – If the number of images and captions do not match
    

Return type:

[`Tensor`](https://docs.pytorch.org/docs/stable/tensors.html#torch.Tensor "(in PyTorch v2.15)")

Example

```python
>>> from torchmetrics.functional.multimodal import clip_score
>>> image = torch.randint(255, (3, 224, 224), generator=torch.Generator().manual_seed(42))
>>> score = clip_score(image, "a photo of a cat", "openai/clip-vit-base-patch16")
>>> score.detach().round(decimals=3)
tensor(24.4260)
```

Example

```python
>>> from torchmetrics.functional.multimodal import clip_score
>>> image1 = torch.randint(255, (3, 224, 224), generator=torch.Generator().manual_seed(42))
>>> image2 = torch.randint(255, (3, 224, 224), generator=torch.Generator().manual_seed(43))
>>> score = clip_score(image1, image2, "openai/clip-vit-base-patch16")
>>> score.detach().round(decimals=3)
tensor(99.4860)
```

Example

```python
>>> from torchmetrics.functional.multimodal import clip_score
>>> score = clip_score(
...     "28-year-old chef found dead in San Francisco mall",
...     "A 28-year-old chef who recently moved to San Francisco was found dead.",
...     "openai/clip-vit-base-patch16"
... )
>>> score.detach().round(decimals=3)
tensor(91.3950)
```

---

**Navigation**

- Parent: [Multimodal](/docs/torchmetrics/1.9.0/multimodal.md)