API Reference

Routines

The routine are the main building blocks of the library. They define the framework in which the models are trained and evaluated. They allow for easy computation of different metrics crucial for uncertainty estimation in different contexts, namely classification, regression and segmentation.

Classification

ClassificationRoutine

Routine for training & testing on classification tasks.

Segmentation

SegmentationRoutine

Routine for training & testing on segmentation tasks.

Regression

RegressionRoutine

Routine for training & testing on regression tasks.

Pixelwise Regression

PixelRegressionRoutine

Routine for training & testing on pixel regression tasks.

Baselines

TorchUncertainty provide lightning-based models that can be easily trained and evaluated. These models inherit from the routines and are specifically designed to benchmark different methods in similar settings, here with constant architectures.

Classification

ResNetBaseline	ResNet backbone baseline for classification providing support for various versions and architectures.
VGGBaseline	VGG backbone baseline for classification providing support for various versions and architectures.
WideResNetBaseline	Wide-ResNet28x10 backbone baseline for classification providing support for various versions.

Regression

MLPBaseline

MLP baseline for regression providing support for various versions.

Segmentation

DeepLabBaseline
SegFormerBaseline	SegFormer backbone baseline for segmentation providing support for various versions and architectures.

Monocular Depth Estimation

BTSBaseline

Layers

Ensemble layers

PackedLinear	Packed-Ensembles-style Linear layer.
PackedConv2d	Packed-Ensembles-style Conv2d layer.
BatchLinear	BatchEnsemble-style Linear layer.
BatchConv2d	BatchEnsemble-style Conv2d layer.
MaskedLinear	Masksembles-style Linear layer.
MaskedConv2d	Masksembles-style Conv2d layer.

Bayesian layers

BayesLinear	Bayesian Linear Layer with Mixture of Normals prior and Normal posterior.
BayesConv1d	Bayesian Conv1d Layer with Mixture of Normals prior and Normal posterior.
BayesConv2d	Bayesian Conv2d Layer with Gaussian Mixture prior and Normal posterior.
BayesConv3d	Bayesian Conv3d Layer with Gaussian mixture prior and Normal posterior.
LPBNNLinear	LPBNN-style linear layer.
LPBNNConv2d	LPBNN-style 2D convolutional layer.

Models

Wrappers

Functions

deep_ensembles	Build a Deep Ensembles out of the original models.
mc_dropout	MC Dropout wrapper for a model.

Classes

CheckpointEnsemble	Ensemble of models at different points in the training trajectory.
EMA	Exponential Moving Average.
MCDropout	MC Dropout wrapper for a model containing nn.Dropout modules.
StochasticModel
SWA	Stochastic Weight Averaging.
SWAG	Stochastic Weight Averaging Gaussian (SWAG).

Metrics

Classification

Proper Scores

BrierScore	The Brier Score Metric.
CategoricalNLL	The Negative Log Likelihood Metric.

Out-of-Distribution Detection

AURC	Area Under the Risk-Coverage curve.
FPRx	The False Positive Rate at x% Recall metric.
FPR95	The False Positive Rate at 95% Recall metric.

Selective Classification

AUGRC	Area Under the Generalized Risk-Coverage curve.
RiskAtxCov	Risk at given Coverage.
RiskAt80Cov	Risk at 80% Coverage.
CovAtxRisk	Coverage at x Risk.
CovAt5Risk	Coverage at 5% Risk.

Calibration

AdaptiveCalibrationError	Adaptive Top-label Calibration Error.
CalibrationError	Top-label Calibration Error.

Diversity

Disagreement	The Disagreement Metric to estimate the confidence of an ensemble of estimators.
Entropy	The Shannon Entropy Metric to estimate the confidence of a single model or the mean confidence across estimators.
MutualInformation	The Mutual Information Metric to estimate the epistemic uncertainty of an ensemble of estimators.
VariationRatio

Others

GroupingLoss

Metric to estimate the Top-label Grouping Loss.

Regression

DistributionNLL	The Negative Log Likelihood Metric.
Log10	The Log10 metric.
MeanAbsoluteErrorInverse	Mean Absolute Error of the inverse predictions (iMAE).
MeanGTRelativeAbsoluteError	Compute Mean Absolute Error relative to the Ground Truth (MAErel
MeanGTRelativeSquaredError	Compute mean squared error relative to the Ground Truth (MSErel or SRE).
MeanSquaredErrorInverse	Mean Squared Error of the inverse predictions (iMSE).
MeanSquaredLogError	MeanSquaredLogError (MSELog) regression metric.
SILog	The Scale-Invariant Logarithmic Loss metric.
ThresholdAccuracy	The Threshold Accuracy metric, a.k.a.

Segmentation

MeanIntersectionOverUnion

Compute the MeanIntersection over Union (IoU) score.

Others

AUSE	The Area Under the Sparsification Error curve (AUSE) metric to estimate the quality of the uncertainty estimates, i.e., how much they coincide with the true errors.

Losses

DistributionNLLLoss	Negative Log-Likelihood loss using given distributions as inputs.
KLDiv	KL divergence loss for Bayesian Neural Networks.
ELBOLoss	The Evidence Lower Bound (ELBO) loss for Bayesian Neural Networks.
BetaNLL	The Beta Negative Log-likelihood loss.
DECLoss	The deep evidential classification loss.
DERLoss	The Deep Evidential Regression loss.
FocalLoss	Focal-Loss for classification tasks.
ConflictualLoss	The Conflictual Loss.
ConfidencePenaltyLoss	The Confidence Penalty Loss.
KLDiv	KL divergence loss for Bayesian Neural Networks.
ELBOLoss	The Evidence Lower Bound (ELBO) loss for Bayesian Neural Networks.
BCEWithLogitsLSLoss	Binary Cross Entropy with Logits Loss with label smoothing.

Post-Processing Methods

MCBatchNorm	Monte Carlo Batch Normalization wrapper.
LaplaceApprox	Laplace approximation for uncertainty estimation.

Scaling Methods

TemperatureScaler	Temperature scaling post-processing for calibrated probabilities.
VectorScaler	Vector scaling post-processing for calibrated probabilities.
MatrixScaler	Matrix scaling post-processing for calibrated probabilities.

Datamodules

Classification

CIFAR10DataModule	DataModule for CIFAR10.
CIFAR100DataModule	DataModule for CIFAR100.
MNISTDataModule	DataModule for MNIST.
TinyImageNetDataModule
ImageNetDataModule	DataModule for ImageNet.

UCI Tabular Classification

BankMarketingDataModule	The Bank Marketing UCI classification datamodule.
DOTA2GamesDataModule	The Dota2 Games UCI classification datamodule.
HTRU2DataModule	The HTRU2 UCI classification datamodule.
OnlineShoppersDataModule	The online shoppers intention UCI classification datamodule.
SpamBaseDataModule	The Bank Marketing UCI classification datamodule.

Regression

UCIRegressionDataModule

The UCI regression datasets.

Segmentation

CamVidDataModule	DataModule for the CamVid dataset.
CityscapesDataModule	DataModule for the Cityscapes dataset.
MUADDataModule	Segmentation DataModule for the MUAD dataset.

Datasets

Classification

MNISTC	The corrupted MNIST-C Dataset.
NotMNIST	The notMNIST dataset.
CIFAR10C	The corrupted CIFAR-10-C Dataset.
CIFAR100C
CIFAR10H	CIFAR-10H Dataset.
CIFAR10N	CIFAR-10N Dataset.
CIFAR100N
ImageNetA
ImageNetC	The corrupted ImageNet-C dataset.
ImageNetO
ImageNetR
TinyImageNet	Inspired by https://gist.github.com/z-a-f/b862013c0dc2b540cf96a123a6766e54.
TinyImageNetC	The corrupted TinyImageNet-C Dataset.
OpenImageO	OpenImage-O dataset.

UCI Tabular Classification

BankMarketing	The bank Marketing UCI classification dataset.
DOTA2Games	The DOTA 2 Games UCI classification dataset.
HTRU2	The HTRU2 UCI classification dataset.
OnlineShoppers	The Online Shoppers Intention UCI classification dataset.
SpamBase	The SpamBase UCI classification dataset.

Regression

UCIRegression

The UCI regression datasets.

Segmentation

CamVid	CamVid Dataset.
Cityscapes

Others & Cross-Categories

Fractals	Dataset used for PixMix augmentations.
FrostImages
KITTIDepth
MUAD	The MUAD Dataset.
NYUv2	NYUv2 depth dataset.