Running

Module Interface

class torchmetrics.wrappers.Running(base_metric, window=5)

Running wrapper for metrics.

Using this wrapper allows for calculating metrics over a running window of values, instead of the whole history of values. This is beneficial when you want to get a better estimate of the metric during training and don’t want to wait for the whole training to finish to get epoch level estimates.

The running window is defined by the window argument. The window is a fixed size and this wrapper will store a duplicate of the underlying metric state for each value in the window. Thus memory usage will increase linearly with window size. Use accordingly. Also note that the running only works with metrics that have the full_state_update set to False.

Importantly, the wrapper does not alter the value of the forward method of the underlying metric. Thus, forward will still return the value on the current batch. To get the running value call compute instead.

Parameters:

• base_metric (Metric) – The metric to wrap.

• window (int) – The size of the running window.

Example (single metric):

>>> from torch import tensor
>>> from torchmetrics.wrappers import Running
>>> from torchmetrics.aggregation import SumMetric
>>> metric = Running(SumMetric(), window=3)
>>> for i in range(6):
...     current_val = metric(tensor([i]))
...     running_val = metric.compute()
...     total_val = tensor(sum(list(range(i+1))))  # value we would get from `compute` without running
...     print(f"{current_val=}, {running_val=}, {total_val=}")
current_val=tensor(0.), running_val=tensor(0.), total_val=tensor(0)
current_val=tensor(1.), running_val=tensor(1.), total_val=tensor(1)
current_val=tensor(2.), running_val=tensor(3.), total_val=tensor(3)
current_val=tensor(3.), running_val=tensor(6.), total_val=tensor(6)
current_val=tensor(4.), running_val=tensor(9.), total_val=tensor(10)
current_val=tensor(5.), running_val=tensor(12.), total_val=tensor(15)

Example (metric collection):

>>> from torch import tensor
>>> from torchmetrics.wrappers import Running
>>> from torchmetrics import MetricCollection
>>> from torchmetrics.aggregation import SumMetric, MeanMetric
>>> # note that running is input to collection, not the other way
>>> metric = MetricCollection({"sum": Running(SumMetric(), 3), "mean": Running(MeanMetric(), 3)})
>>> for i in range(6):
...     current_val = metric(tensor([i]))
...     running_val = metric.compute()
...     print(f"{current_val=}, {running_val=}")
current_val={'mean': tensor(0.), 'sum': tensor(0.)}, running_val={'mean': tensor(0.), 'sum': tensor(0.)}
current_val={'mean': tensor(1.), 'sum': tensor(1.)}, running_val={'mean': tensor(0.5000), 'sum': tensor(1.)}
current_val={'mean': tensor(2.), 'sum': tensor(2.)}, running_val={'mean': tensor(1.), 'sum': tensor(3.)}
current_val={'mean': tensor(3.), 'sum': tensor(3.)}, running_val={'mean': tensor(2.), 'sum': tensor(6.)}
current_val={'mean': tensor(4.), 'sum': tensor(4.)}, running_val={'mean': tensor(3.), 'sum': tensor(9.)}
current_val={'mean': tensor(5.), 'sum': tensor(5.)}, running_val={'mean': tensor(4.), 'sum': tensor(12.)}

forward(*args, **kwargs)

Forward input to the underlying metric and save state afterwards.

Return type:

Any

plot(val=None, ax=None)

Plot a single or multiple values from the metric.

Parameters:

• val (Union[Tensor, Sequence[Tensor], None]) – Either a single result from calling metric.forward or metric.compute or a list of these results. If no value is provided, will automatically call metric.compute and plot that result.

• ax (Optional[Axes]) – An matplotlib axis object. If provided will add plot to that axis

Return type:

tuple[Figure, Union[Axes, ndarray]]

Returns:

Figure and Axes object

Raises:

ModuleNotFoundError – If matplotlib is not installed

>>> # Example plotting a single value
>>> import torch
>>> from torchmetrics.wrappers import Running
>>> from torchmetrics.aggregation import SumMetric
>>> metric = Running(SumMetric(), 2)
>>> metric.update(torch.randn(20, 2))
>>> fig_, ax_ = metric.plot()

>>> # Example plotting multiple values
>>> import torch
>>> from torchmetrics.wrappers import Running
>>> from torchmetrics.aggregation import SumMetric
>>> metric = Running(SumMetric(), 2)
>>> values = [ ]
>>> for _ in range(3):
...     values.append(metric(torch.randn(20, 2)))
>>> fig_, ax_ = metric.plot(values)

reset()

Reset metric.

Return type:

None