GroupingLoss

class torch_uncertainty.metrics.classification.GroupingLoss(**kwargs)

Metric to estimate the Top-label Grouping Loss.

Parameters:

kwargs – Additional keyword arguments, see Advanced metric settings.

Inputs:

• probs: \((B, C)\) or \((B, N, C)\)

• target: \((B)\) or \((B, C)\)

• features: \((B, F)\) or \((B, N, F)\)

where \(B\) is the batch size, \(C\) is the number of classes and \(N\) is the number of estimators.

Warning

Make sure that the probabilities in probs are normalized to sum to one.

Raises:

ValueError – If reduction is not one of 'mean', 'sum', 'none' or None.

Reference:

Perez-Lebel, Alexandre, Le Morvan, Marine and Varoquaux, Gaël. Beyond calibration: estimating the grouping loss of modern neural networks. In ICLR 2023.

compute()

Compute the final Brier score based on inputs passed to update.

Returns:

The final value(s) for the Brier score

Return type:

torch.Tensor

update(probs, target, features)

Accumulate the tensors for the estimation of the Grouping Loss.

Parameters:

• probs (Tensor) – A probability tensor of shape (batch, num_classes), (batch, num_estimators, num_classes), or (batch) if binary classification

• target (Tensor) – A tensor of ground truth labels of shape (batch, num_classes) or (batch)

• features (Tensor) – A tensor of features of shape (batch, num_estimators, num_features) or (batch, num_features)