Welcome to TorchMetrics

TorchMetrics is a collection of 100+ PyTorch metrics implementations and an easy-to-use API to create custom metrics. It offers:

• A standardized interface to increase reproducibility

• Reduces Boilerplate

• Distributed-training compatible

• Rigorously tested

• Automatic accumulation over batches

• Automatic synchronization between multiple devices

You can use TorchMetrics in any PyTorch model, or within PyTorch Lightning to enjoy the following additional benefits:

• Your data will always be placed on the same device as your metrics

• You can log Metric objects directly in Lightning to reduce even more boilerplate

---

Install TorchMetrics

For pip users

pip install torchmetrics

Or directly from conda

conda install -c conda-forge torchmetrics

---

New to TorchMetrics?

Use this quickstart guide to learn key concepts.

TorchMetrics with PyTorch Lightning

Easily use TorchMetrics in your PyTorch Lightning code.

Metrics

View the full list of metrics and filter by task and data type.

Overview

A detailed overview of the TorchMetrics API and concepts.

Custom Metrics

Learn how to implement a custom metric with TorchMetrics.

API Reference

Detailed descriptions of each API package.

User Guide

• Quick Start
  • Install
  • Using TorchMetrics
  • Implementing your own metric
• All TorchMetrics
• TorchMetrics’ gallery
• Audio domain
• Image domain
• Text domain
• Structure Overview
  • Metrics and devices
  • Metrics and memory management
  • Saving and loading metrics
  • Metrics in Distributed Data Parallel (DDP) mode
  • Metrics and 16-bit precision
  • Metric Arithmetic
  • MetricCollection
  • Metric wrappers
  • Module vs Functional Metrics
  • Metrics and differentiability
  • Metrics and hyperparameter optimization
  • Advanced metric settings
• Plotting
  • Plotting a single step
  • Plotting multi steps
  • Plotting a collection of metrics
  • Advance example
• Implementing a Metric
  • Base interface
  • Working with list states
  • Metric attributes
  • Plot interface
  • Internal implementation details
  • Contributing your metric to TorchMetrics
• TorchMetrics in PyTorch Lightning
  • Logging TorchMetrics
  • Common Pitfalls

Aggregation

• Concatenation
  • Module Interface
• Maximum
  • Module Interface
• Mean
  • Module Interface
• Minimum
  • Module Interface
• Running Mean
  • Module Interface
• Running Sum
  • Module Interface
• Sum
  • Module Interface

Audio

• Complex Scale-Invariant Signal-to-Noise Ratio (C-SI-SNR)
  • Module Interface
  • Functional Interface
• Deep Noise Suppression Mean Opinion Score (DNSMOS)
  • Module Interface
  • Functional Interface
• Perceptual Evaluation of Speech Quality (PESQ)
  • Module Interface
  • Functional Interface
• Permutation Invariant Training (PIT)
  • Module Interface
  • Functional Interface
• Scale-Invariant Signal-to-Distortion Ratio (SI-SDR)
  • Module Interface
  • Functional Interface
• Scale-Invariant Signal-to-Noise Ratio (SI-SNR)
  • Module Interface
  • Functional Interface
• Short-Time Objective Intelligibility (STOI)
  • Module Interface
  • Functional Interface
• Signal to Distortion Ratio (SDR)
  • Module Interface
  • Functional Interface
• Signal-to-Noise Ratio (SNR)
  • Module Interface
  • Functional Interface
• Source Aggregated Signal-to-Distortion Ratio (SA-SDR)
  • Module Interface
  • Functional Interface
• Speech-to-Reverberation Modulation Energy Ratio (SRMR)
  • Module Interface
  • Functional Interface

Classification

• Accuracy
  • Module Interface
  • Functional Interface
• AUROC
  • Module Interface
  • Functional Interface
• Average Precision
  • Module Interface
  • Functional Interface
• Calibration Error
  • Module Interface
  • Functional Interface
• Cohen Kappa
  • Module Interface
  • Functional Interface
• Confusion Matrix
  • Module Interface
  • Functional Interface
• Coverage Error
  • Module Interface
  • Functional Interface
• Dice
  • Module Interface
  • Functional Interface
• Exact Match
  • Module Interface
  • Functional Interface
• F-1 Score
  • Module Interface
  • Functional Interface
• F-Beta Score
  • Module Interface
  • Functional Interface
• Group Fairness
  • Module Interface
  • Functional Interface
• Hamming Distance
  • Module Interface
  • Functional Interface
• Hinge Loss
  • Module Interface
  • Functional Interface
• Jaccard Index
  • Module Interface
  • Functional Interface
• Label Ranking Average Precision
  • Module Interface
  • Functional Interface
• Label Ranking Loss
  • Module Interface
  • Functional Interface
• Matthews Correlation Coefficient
  • Module Interface
  • Functional Interface
• Precision
  • Module Interface
  • Functional Interface
• Precision At Fixed Recall
  • Module Interface
  • Functional Interface
• Precision Recall Curve
  • Module Interface
  • Functional Interface
• Recall
  • Module Interface
  • Functional Interface
• Recall At Fixed Precision
  • Module Interface
  • Functional Interface
• ROC
  • Module Interface
  • Functional Interface
• Sensitivity At Specificity
  • Module Interface
  • Functional Interface
• Specificity
  • Module Interface
  • Functional Interface
• Specificity At Sensitivity
  • Module Interface
  • Functional Interface
• Stat Scores
  • Module Interface
  • Functional Interface

Clustering

• Adjusted Mutual Information Score
  • Module Interface
  • Functional Interface
• Adjusted Rand Score
  • Module Interface
  • Functional Interface
• Calinski Harabasz Score
  • Module Interface
  • Functional Interface
• Completeness Score
  • Module Interface
  • Functional Interface
• Davies Bouldin Score
  • Module Interface
  • Functional Interface
• Dunn Index
  • Module Interface
  • Functional Interface
• Fowlkes-Mallows Index
  • Module Interface
  • Functional Interface
• Homogeneity Score
  • Module Interface
  • Functional Interface
• Mutual Information Score
  • Module Interface
  • Functional Interface
• Normalized Mutual Information Score
  • Module Interface
  • Functional Interface
• Rand Score
  • Module Interface
  • Functional Interface
• V-Measure Score
  • Module Interface
  • Functional Interface

Detection

• Complete Intersection Over Union (cIoU)
  • Module Interface
  • Functional Interface
• Distance Intersection Over Union (dIoU)
  • Module Interface
  • Functional Interface
• Generalized Intersection Over Union (gIoU)
  • Module Interface
  • Functional Interface
• Intersection Over Union (IoU)
  • Module Interface
  • Functional Interface
• Mean-Average-Precision (mAP)
  • Module Interface
• Modified Panoptic Quality
  • Module Interface
  • Functional Interface
• Panoptic Quality
  • Module Interface
  • Functional Interface

Image

• Error Relative Global Dim. Synthesis (ERGAS)
  • Module Interface
  • Functional Interface
• Frechet Inception Distance (FID)
  • Module Interface
• Image Gradients
  • Functional Interface
• Inception Score
  • Module Interface
• Kernel Inception Distance
  • Module Interface
• Learned Perceptual Image Patch Similarity (LPIPS)
  • Module Interface
  • Functional Interface
• Memorization-Informed Frechet Inception Distance (MiFID)
  • Module Interface
• Multi-Scale SSIM
  • Module Interface
  • Functional Interface
• Peak Signal-to-Noise Ratio (PSNR)
  • Module Interface
  • Functional Interface
• Peak Signal To Noise Ratio With Blocked Effect
  • Module Interface
  • Functional Interface
• Perceptual Path Length (PPL)
  • Module Interface
  • Functional Interface
• Quality with No Reference
  • Module Interface
  • Functional Interface
• Relative Average Spectral Error (RASE)
  • Module Interface
  • Functional Interface
• Root Mean Squared Error Using Sliding Window
  • Module Interface
  • Functional Interface
• Spatial Correlation Coefficient (SCC)
  • Module Interface
  • Functional Interface
• Spatial Distortion Index
  • Module Interface
  • Functional Interface
• Spectral Angle Mapper
  • Module Interface
  • Functional Interface
• Spectral Distortion Index
  • Module Interface
  • Functional Interface
• Structural Similarity Index Measure (SSIM)
  • Module Interface
  • Functional Interface
• Total Variation (TV)
  • Module Interface
  • Functional Interface
• Universal Image Quality Index
  • Module Interface
  • Functional Interface
• Visual Information Fidelity (VIF)
  • Module Interface
  • Functional Interface

Multimodal

• CLIP Image Quality Assessment (CLIP-IQA)
  • Module Interface
  • Functional Interface
• CLIP Score
  • Module Interface
  • Functional Interface

Nominal

• Cramer’s V
  • Module Interface
  • Functional Interface
• Fleiss Kappa
  • Module Interface
  • Functional Interface
• Pearson’s Contingency Coefficient
  • Module Interface
  • Functional Interface
• Theil’s U
  • Module Interface
  • Functional Interface
• Tschuprow’s T
  • Module Interface
  • Functional Interface

Pairwise

• Cosine Similarity
  • Functional Interface
• Euclidean Distance
  • Functional Interface
• Linear Similarity
  • Functional Interface
• Manhattan Distance
  • Functional Interface
• Minkowski Distance
  • Functional Interface

Regression

• Concordance Corr. Coef.
  • Module Interface
  • Functional Interface
• Cosine Similarity
  • Module Interface
  • Functional Interface
• Critical Success Index (CSI)
  • Module Interface
  • Functional Interface
• Explained Variance
  • Module Interface
  • Functional Interface
• Kendall Rank Corr. Coef.
  • Module Interface
  • Functional Interface
• KL Divergence
  • Module Interface
  • Functional Interface
• Log Cosh Error
  • Module Interface
  • Functional Interface
• Mean Absolute Error (MAE)
  • Module Interface
  • Functional Interface
• Mean Absolute Percentage Error (MAPE)
  • Module Interface
  • Functional Interface
• Mean Squared Error (MSE)
  • Module Interface
  • Functional Interface
• Mean Squared Log Error (MSLE)
  • Module Interface
  • Functional Interface
• Minkowski Distance
  • Module Interface
  • Functional Interface
• Pearson Corr. Coef.
  • Module Interface
  • Functional Interface
• R2 Score
  • Module Interface
  • Functional Interface
• Relative Squared Error (RSE)
  • Module Interface
  • Functional Interface
• Spearman Corr. Coef.
  • Module Interface
  • Functional Interface
• Symmetric Mean Absolute Percentage Error (SMAPE)
  • Module Interface
  • Functional Interface
• Tweedie Deviance Score
  • Module Interface
  • Functional Interface
• Weighted MAPE
  • Module Interface
  • Functional Interface

Retrieval

• Retrieval AUROC
  • Module Interface
  • Functional Interface
• Retrieval Fall-Out
  • Module Interface
  • Functional Interface
• Retrieval Hit Rate
  • Module Interface
  • Functional Interface
• Retrieval Mean Average Precision (MAP)
  • Module Interface
  • Functional Interface
• Retrieval Mean Reciprocal Rank (MRR)
  • Module Interface
  • Functional Interface
• Retrieval Normalized DCG
  • Module Interface
  • Functional Interface
• Retrieval Precision
  • Module Interface
  • Functional Interface
• Retrieval Precision Recall Curve
  • Module Interface
  • Functional Interface
• Retrieval R-Precision
  • Module Interface
  • Functional Interface
• Retrieval Recall
  • Module Interface
  • Functional Interface

Segmentation

• Generalized Dice Score
  • Module Interface
  • Functional Interface
• Hausdorff Distance
  • Module Interface
  • Functional Interface
• Mean Intersection over Union (mIoU)
  • Module Interface
  • Functional Interface

Shape

• Procrustes Disparity
  • Module Interface
  • Functional Interface

Text

• BERT Score
  • Module Interface
  • Functional Interface
• BLEU Score
  • Module Interface
  • Functional Interface
• Char Error Rate
  • Module Interface
  • Functional Interface
• ChrF Score
  • Module Interface
  • Functional Interface
• Edit Distance
  • Module Interface
  • Functional Interface
• Extended Edit Distance
  • Module Interface
  • Functional Interface
• InfoLM
  • Module Interface
  • Functional Interface
• Match Error Rate
  • Module Interface
  • Functional Interface
• Perplexity
  • Module Interface
  • Functional Interface
• ROUGE Score
  • Module Interface
  • Functional Interface
• Sacre BLEU Score
  • Module Interface
  • Functional Interface
• SQuAD
  • Module Interface
  • Functional Interface
• Translation Edit Rate (TER)
  • Module Interface
  • Functional Interface
• Word Error Rate
  • Module Interface
  • Functional Interface
• Word Info. Lost
  • Module Interface
  • Functional Interface
• Word Info. Preserved
  • Module Interface
  • Functional Interface

Wrappers

• Bootstrapper
  • Module Interface
• Classwise Wrapper
  • Module Interface
• Feature Sharing
  • Module Interface
• Metric Tracker
  • Module Interface
• Min / Max
  • Module Interface
• Multi-output Wrapper
  • Module Interface
• Multi-task Wrapper
  • Module Interface
• Running
  • Module Interface
• Transformations
  • Module Interface

API Reference

• torchmetrics.Metric
  • Metric
    • Metric.add_state()
    • Metric.clone()
    • Metric.compute()
    • Metric.double()
    • Metric.float()
    • Metric.forward()
    • Metric.half()
    • Metric.persistent()
    • Metric.plot()
    • Metric.reset()
    • Metric.set_dtype()
    • Metric.state_dict()
    • Metric.sync()
    • Metric.sync_context()
    • Metric.type()
    • Metric.unsync()
    • Metric.update()
    • Metric.device
    • Metric.dtype
    • Metric.metric_state
    • Metric.update_called
    • Metric.update_count
• torchmetrics.utilities
  • torchmetrics.utilities.data
    • select_topk
    • to_categorical
    • to_onehot
    • dim_zero_cat
    • dim_zero_max
    • dim_zero_mean
    • dim_zero_min
    • dim_zero_sum
  • torchmetrics.utilities.distributed
    • gather_all_tensors
  • torchmetrics.utilities.exceptions
    • TorchMetricsUserError
    • TorchMetricsUserWarning

Community

• TorchMetrics Governance
• Contributor Covenant Code of Conduct
• Contributing
• Changelog

Indices and tables

• Index

• Module Index

• Search Page