AUSE

class torch_uncertainty.metrics.AUSE(**kwargs)

The Area Under the Sparsification Error curve (AUSE) metric to estimate the quality of the uncertainty estimates, i.e., how much they coincide with the true errors.

Parameters:

kwargs – Additional keyword arguments, see Advanced metric settings.

Reference:

From the paper Uncertainty estimates and multi-hypotheses for optical flow. In ECCV, 2018.

Inputs:

• scores: Uncertainty scores of shape \((B,)\). A higher score means a higher uncertainty.

• errors: Binary errors of shape \((B,)\),

where \(B\) is the batch size.

Note

A higher AUSE means a lower quality of the uncertainty estimates.

compute()

Compute the Area Under the Sparsification Error curve (AUSE) based on inputs passed to update.

Returns:

The AUSE.

Return type:

Tensor

plot(ax=None, plot_oracle=True, plot_value=True)

Plot the sparsification curve corresponding to the inputs passed to update, and the oracle sparsification curve.

Parameters:

• ax (Axes | None, optional) – An matplotlib axis object. If provided will add plot to this axis. Defaults to None.

• plot_oracle (bool, optional) – Whether to plot the oracle sparsification curve. Defaults to True.

• plot_value (bool, optional) – Whether to plot the AUSE value. Defaults to True.

Returns:

Figure object and Axes object

Return type:

tuple[[Figure | None], Axes]

update(scores, errors)

Store the scores and their associated errors for later computation.

Parameters:

• scores (Tensor) – uncertainty scores of shape \((B,)\)

• errors (Tensor) – binary errors of shape \((B,)\)