DistributionNLL#

class torch_uncertainty.metrics.regression.DistributionNLL(reduction='mean', **kwargs)[source]#

Computes the Negative Log-Likelihood (NLL) metric for classification tasks.

This metric evaluates the performance of a probabilistic classification model by calculating the negative log likelihood of the predicted probabilities. For a batch of size \(B\) with \(C\) classes, the negative log likelihood is defined as:

\[\ell(p, y) = -\frac{1}{B} \sum_{i=1}^B \log(p_{i, y_i})\]

where \(p_{i, y_i}\) is the predicted probability for the true class \(y_i\) of sample \(i\).

Parameters:

  • reduction (str, optional) –

    Determines how to reduce the computed loss over the batch dimension:

    • 'mean' [default]: Averages the loss across samples in the batch.

    • 'sum': Sums the loss across samples in the batch.

    • 'none' or None: Returns the loss for each sample without reducing.

  • kwargs – Additional keyword arguments as described in Advanced Metric Settings.

Inputs:
  • probs: \((B, C)\)

    A Tensor containing the predicted probabilities for C classes, where each row corresponds to a sample in the batch.

  • target: \((B,)\)

    A Tensor containing the ground truth labels as integers in the range \([0, C-1]\).

Note

Ensure that the probabilities in probs are normalized to sum to one:

\[\sum_{c=1}^C p_{i, c} = 1 \quad \forall i \in [1, B].\]

Warning

If reduction is not one of 'mean', 'sum', 'none', or None, a ValueError will be raised.

Example:

from torch_uncertainty.metrics.classification.categorical_nll import (
    CategoricalNLL,
)

metric = CategoricalNLL(reduction="mean")
probs = torch.tensor([[0.7, 0.3], [0.4, 0.6]])
target = torch.tensor([0, 1])
metric.update(probs, target)
print(metric.compute())
# Output: tensor(0.4338)
compute()[source]#

Computes NLL based on inputs passed in to update previously.

update(dist, target, padding_mask=None)[source]#

Update state with the predicted distributions and the targets.

Parameters:

  • dist (torch.distributions.Distribution) – Predicted distributions.

  • target (Tensor) – Ground truth labels.

  • padding_mask (Tensor, optional) – The padding mask. Defaults to None. Sets the loss to 0 for padded values.