CoverageRate#

class torch_uncertainty.metrics.classification.CoverageRate(num_classes=None, average='micro', validate_args=True, **kwargs)[source]#

Empirical coverage rate metric.

For a prediction set \(\mathcal{C}(X)\) returned by a conformal predictor (or any set-valued predictor), the coverage rate is the fraction of test points whose ground-truth label is contained in the predicted set:

\[\text{Coverage} = \frac{1}{N} \sum_{i=1}^{N} \mathbf{1}\!\left[ y_i \in \mathcal{C}(x_i) \right].\]

With average="macro", the per-class coverage rates are averaged uniformly:

\[\text{Coverage}_{\text{macro}} = \frac{1}{C} \sum_{c=1}^{C} \frac{\sum_{i:\, y_i = c} \mathbf{1}\!\left[ y_i \in \mathcal{C}(x_i) \right]} {\sum_{i:\, y_i = c} 1}.\]
Parameters:

  • num_classes (int | None) – Number of classes. Defaults to None.

  • average (str) –

    Defines the reduction that is applied over labels. Defaults to "macro". Should be one of the following:

    • 'macro': Compute the metric for each class separately and find their unweighted mean. This does not take label imbalance into account.

    • 'micro': Sum statistics across over all labels.

  • validate_args (bool) – Whether to validate the arguments. Defaults to True.

  • kwargs – Additional keyword arguments, see Advanced metric settings.

Raises:

  • ValueError – If num_classes is None and average is not micro.

  • ValueError – If num_classes is not an integer larger than 1.

  • ValueError – If average is not one of macro or micro.

compute()[source]#

Compute the coverage rate.

Returns:

The coverage rate.

Return type:

Tensor

update(preds, target)[source]#

Update the metric state with predictions and targets.

Parameters:

  • preds (Tensor) – Predicted sets tensor of shape (B, C), where B is the batch size and C is the number of classes.

  • target (Tensor) – Target labels tensor of shape (B,).

Return type:

None