SCODCovAtxRisk#

class torch_uncertainty.metrics.classification.SCODCovAtxRisk(risk_threshold, ood_cost=0.5, **kwargs)[source]#

Calculate the maximum coverage at a specified SCOD risk.

This metric applies CovAtxRisk to the joint SCOD loss defined by SCODAURC.

Let \(r(\kappa_k)\) denote the empirical SCOD selective risk at coverage \(\kappa_k=k/N\). For a risk threshold \(\tau\in[0,1]\), coverage at SCOD risk \(\tau\) is

\[\operatorname{SCOD\text{-}Cov@Risk}(\tau) = \max\left\{ \kappa_k : r(\kappa_k)\leq\tau,\quad k\in\{1,\ldots,N\} \right\}.\]

If no positive coverage satisfies the risk constraint, the metric returns nan. Because empirical selective risk need not be monotonic in coverage, the metric considers every available coverage and returns the largest admissible one.

As input to forward and update, the metric accepts:

ood_scores (Tensor): Float tensor containing OOD scores, where larger values indicate more OOD-like samples.
classification_errors (Tensor): Boolean or binary tensor indicating misclassified ID samples.
is_ood (Tensor): Boolean or binary tensor indicating OOD samples.

As output to forward and compute, the metric returns:

scod_coverage (Tensor): Scalar tensor containing the maximum coverage satisfying the SCOD risk constraint. Higher values are better.

Parameters:

risk_threshold (float) – Maximum admissible SCOD risk \(\tau\).
ood_cost (float) – Relative cost \(c_{\mathrm{OOD}}\) of accepting an OOD sample. The cost of accepting a misclassified ID sample is 1 - ood_cost. Defaults to 0.5.
kwargs – Additional keyword arguments passed to torchmetrics.Metric.

References

[1] Xia & Bouganis. Augmenting Softmax Information for Selective Classification with Out-of-Distribution Data. ACCV, 2022..

[2] Narasimhan et al. Plugin Estimators for Selective Classification with Out-of-Distribution Detection..