CategoricalNLL#

class torch_uncertainty.metrics.classification.CategoricalNLL(reduction='mean', **kwargs)[source]#

Computes the Negative Log-Likelihood (NLL) metric for classification tasks.

This metric evaluates the performance of a probabilistic classification model by calculating the negative log likelihood of the predicted probabilities. For a batch of size \(B\) with \(C\) classes, the negative log likelihood is defined as:

\[\ell(p, y) = -\frac{1}{B} \sum_{i=1}^B \log(p_{i, y_i})\]

where \(p_{i, y_i}\) is the predicted probability for the true class \(y_i\) of sample \(i\).

Parameters:

  • reduction (str, optional) –

    Determines how to reduce the computed loss over the batch dimension:

    • 'mean' [default]: Averages the loss across samples in the batch.

    • 'sum': Sums the loss across samples in the batch.

    • 'none' or None: Returns the loss for each sample without reducing.

  • kwargs – Additional keyword arguments as described in Advanced Metric Settings.

Inputs:
  • probs: \((B, C)\)

    A Tensor containing the predicted probabilities for C classes, where each row corresponds to a sample in the batch.

  • target: \((B,)\)

    A Tensor containing the ground truth labels as integers in the range \([0, C-1]\).

Note

Ensure that the probabilities in probs are normalized to sum to one:

\[\sum_{c=1}^C p_{i, c} = 1 \quad \forall i \in [1, B].\]

Warning

If reduction is not one of 'mean', 'sum', 'none', or None, a ValueError will be raised.

Example:

from torch_uncertainty.metrics.classification.categorical_nll import (
    CategoricalNLL,
)

metric = CategoricalNLL(reduction="mean")
probs = torch.tensor([[0.7, 0.3], [0.4, 0.6]])
target = torch.tensor([0, 1])
metric.update(probs, target)
print(metric.compute())
# Output: tensor(0.4338)
compute()[source]#

Computes the final NLL score based on the accumulated state.

Returns:

A scalar if reduction is ‘mean’ or ‘sum’; otherwise, a tensor of shape \((B,)\) if reduction is ‘none’.

Return type:

Tensor

update(probs, target)[source]#

Update state with prediction probabilities and targets.

Parameters:

  • probs (Tensor) – Probabilities from the model.

  • target (Tensor) – Ground truth labels.

For each sample \(i\), the negative log likelihood is computed as:

\[\ell_i = -\log(p_{i, y_i}),\]

where \(p_{i, y_i}\) is the predicted probability for the true class \(y_i\).