SCODAUGRC#

class torch_uncertainty.metrics.classification.SCODAUGRC(**kwargs)[source]#

Calculate The Area Under the Generalized Risk-Coverage curve (AUGRC).

The Area Under the Generalized Risk-Coverage curve (AUGRC) for selective classification performance assessment. Unlike AURC, it weights the error rate at each coverage level by the coverage itself, reducing the influence of the most confident samples and making the metric more sensitive to performance at intermediate coverage levels.

Using the same notation as AURC, the AUGRC is defined as

\[\text{AUGRC} = \int_0^1 \kappa \cdot r(\kappa)\,\mathrm{d}\kappa \approx \frac{1}{N} \sum_{k=1}^{N} \frac{k}{N} \cdot r\!\left(\tfrac{k}{N}\right)\]

As input to forward and update the metric accepts the following input:

preds (Tensor): A float tensor of shape (N, ...) containing probabilities for each observation.
target (Tensor): An int tensor of shape (N, ...) containing ground-truth labels.

As output to forward and compute the metric returns the following output:

Augrc (Tensor): A scalar tensor containing the area under the risk-coverage curve

Parameters:: kwargs – Additional keyword arguments.

References

[1] Traub et al. Overcoming Common Flaws in the Evaluation of Selective Classification Systems.