{"id":5648365,"date":"2023-07-10T08:36:34","date_gmt":"2023-07-10T12:36:34","guid":{"rendered":"https:\/\/lightning.ai\/pages\/?p=5648365"},"modified":"2023-07-13T12:34:17","modified_gmt":"2023-07-13T16:34:17","slug":"torchmetrics-a-short-history-to-100-metrics","status":"publish","type":"post","link":"https:\/\/lightning.ai\/pages\/blog\/torchmetrics-a-short-history-to-100-metrics\/","title":{"rendered":"TorchMetrics \u2014 a short history to 100+ metrics"},"content":{"rendered":"

As a celebration of v1.0.0 of Torchmetrics being released<\/a>, in this blogpost we will discuss the history of Torchmetrics and how the interface have evolved over time.<\/p>\n

It all started in the pretty early days of Pytorch Lightning. More specifically on April 1st, 2020. On this date, the first proposal<\/a> for a metric interface was submitted to PyTorch Lightning<\/a>, after an extensive discussion between William Falcon<\/a> (the founder of PyTorch Lightning<\/a>), Jiri Borovec<\/a> (the first fulltime developer of PyTorch Lightning), Tullie Murell<\/a>, Davide Testuggine<\/a> (both working at FAIR at this time) and Justus Schock<\/a> (interested in contributing such a package; later became the first Tech Lead for metrics in PyTorch Lightning<\/a>). This discussion was predated by a more general discussion in the PyTorch issues<\/a>, showing the general need for a unified metrics package.<\/p>\n

The initial draft was then quickly refined by a few other PRs and finally landed on April 3rd (2020) with the following main interface introduced by this<\/a> PR:<\/p>\n

class Metric(torch.nn.Module, ABC):\r\n    \"\"\"\r\n    Abstract Base Class for metric implementation.\r\n    Should be used to implement metrics that\r\n    1. Return multiple Outputs\r\n    2. Handle their own DDP sync\r\n    \"\"\"\r\n    def __init__(self, name: str):\r\n        ...\r\n\r\n    def forward(self, *args, **kwargs) -> torch.Tensor:\r\n        \"\"\"\r\n        Implements the actual metric computation.\r\n        Returns:\r\n            metric value\r\n        \"\"\"\r\n        raise NotImplementedError\r\n     ...\r\n<\/code><\/pre>\n
It also included a lot of utilities to make dtype and devices available from the Module itself \u2014 a lot of which can still be found today in the DeviceDtypeModuleMixin<\/a><\/code>\u00a0that currently lives in lightning.fabric<\/code>.<\/p>\n
The core design elements of the Metric<\/code> class at this time were:<\/p>\n