.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_tutorials/Post_Hoc_Methods/tutorial_temperature.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_tutorials_Post_Hoc_Methods_tutorial_temperature.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_tutorials_Post_Hoc_Methods_tutorial_temperature.py:


Improve Top-label Calibration with Temperature Scaling
======================================================

In this tutorial, we use *TorchUncertainty* to improve the calibration
of the top-label predictions and the reliability of the underlying neural network.

This tutorial provides extensive details on how to use the TemperatureScaler
class and its derivatives, VectorScaler, MatrixScaler, and Dirichlet calibration, as well
as IsotonicRegressionScaler. However, please note that this is usually done
automatically in the datamodule when setting the `postprocess_set` to val or test.

Through this tutorial, we also see how to use the datamodules outside any Lightning trainers,
and how to use TorchUncertainty's models.

Note:
~~~~

The Expected Calibration Error (ECE) is not sufficient to properly assess the calibration properties of a model.

1. Loading the Utilities
~~~~~~~~~~~~~~~~~~~~~~~~

In this tutorial, we will need:

- TorchUncertainty's Calibration Error metric to compute to evaluate the top-label calibration with ECE and plot the reliability diagrams
- the CIFAR-100 datamodule to handle the data
- a ResNet 18 as starting model
- the temperature scaler to improve the top-label calibration
- a utility function to download HF models easily

If you use the classification routine, the plots will be automatically available in the tensorboard logs if you use the `log_plots` flag.

.. GENERATED FROM PYTHON SOURCE LINES 37-43

.. code-block:: Python

    from torch_uncertainty.datamodules import CIFAR100DataModule
    from torch_uncertainty.metrics import CalibrationError
    from torch_uncertainty.models.classification import resnet
    from torch_uncertainty.post_processing import TemperatureScaler
    from torch_uncertainty.utils import load_hf








.. GENERATED FROM PYTHON SOURCE LINES 44-49

2. Loading a model from TorchUncertainty's HF
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To avoid training a model on CIFAR-100 from scratch, we load a model from Hugging Face.
This can be done in a one liner:

.. GENERATED FROM PYTHON SOURCE LINES 49-58

.. code-block:: Python


    # Build the model
    model = resnet(in_channels=3, num_classes=100, arch=18, style="cifar", conv_bias=False)

    # Download the weights (the config is not used here)
    weights, config = load_hf("resnet18_c100")

    # Load the weights in the pre-built model
    model.load_state_dict(weights)




.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    <All keys matched successfully>



.. GENERATED FROM PYTHON SOURCE LINES 59-66

3. Setting up the Datamodule and Dataloaders
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To get the dataloader from the datamodule, just call prepare_data, setup, and
extract the first element of the test dataloader list. There are more than one
element if eval_ood is True: the dataloader of in-distribution data and the dataloader
of out-of-distribution data. Otherwise, it is a list of 1 element.

.. GENERATED FROM PYTHON SOURCE LINES 66-71

.. code-block:: Python


    dm = CIFAR100DataModule(root="./data", eval_ood=False, batch_size=32)
    dm.prepare_data()
    dm.setup("test")








.. GENERATED FROM PYTHON SOURCE LINES 72-82

4. Iterating on the Dataloader and Computing the ECE
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

We first split the original test set into a calibration set and a test set for proper evaluation.

When computing the ECE, you need to provide the likelihoods associated with the inputs.
To do this, just call PyTorch's softmax.

To avoid lengthy computations, we restrict the calibration computation to a subset
of the test set.

.. GENERATED FROM PYTHON SOURCE LINES 82-103

.. code-block:: Python


    from torch.utils.data import DataLoader, random_split

    # Split datasets
    dataset = dm.test
    cal_dataset, test_dataset = random_split(dataset, [4000, len(dataset) - 4000])
    test_dataloader = DataLoader(test_dataset, batch_size=128)
    calibration_dataloader = DataLoader(cal_dataset, batch_size=128)

    # Initialize the ECE
    ece = CalibrationError(task="multiclass", num_classes=100)

    # Iterate on the calibration dataloader
    for sample, target in test_dataloader:
        logits = model(sample)
        probs = logits.softmax(-1)
        ece.update(probs, target)

    # Compute & print the calibration error
    print(f"ECE before scaling - {ece.compute():.3%}.")





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    ECE before scaling - 10.773%.




.. GENERATED FROM PYTHON SOURCE LINES 104-106

We also compute and plot the top-label calibration figure. We see that the
model is not well calibrated.

.. GENERATED FROM PYTHON SOURCE LINES 106-110

.. code-block:: Python

    fig, ax = ece.plot()
    fig.tight_layout()
    fig.show()




.. image-sg:: /auto_tutorials/Post_Hoc_Methods/images/sphx_glr_tutorial_temperature_001.png
   :alt: Reliability Diagram
   :srcset: /auto_tutorials/Post_Hoc_Methods/images/sphx_glr_tutorial_temperature_001.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 111-118

5. Fitting the Scaler to Improve the Calibration
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The TemperatureScaler has one parameter that can be used to temper the softmax.
We minimize the tempered cross-entropy on a calibration set that we define here as
a subset of the test set and containing 1000 data. Look at the code run by TemperatureScaler
`fit` method for more details.

.. GENERATED FROM PYTHON SOURCE LINES 118-123

.. code-block:: Python


    # Fit the scaler on the calibration dataset
    scaled_model = TemperatureScaler(model=model)
    scaled_model.fit(dataloader=calibration_dataloader)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


      0%|          | 0/32 [00:00<?, ?it/s]
      3%|▎         | 1/32 [00:00<00:11,  2.73it/s]
      6%|▋         | 2/32 [00:00<00:11,  2.64it/s]
      9%|▉         | 3/32 [00:01<00:11,  2.60it/s]
     12%|█▎        | 4/32 [00:01<00:10,  2.60it/s]
     16%|█▌        | 5/32 [00:01<00:10,  2.60it/s]
     19%|█▉        | 6/32 [00:02<00:09,  2.60it/s]
     22%|██▏       | 7/32 [00:02<00:09,  2.60it/s]
     25%|██▌       | 8/32 [00:03<00:09,  2.60it/s]
     28%|██▊       | 9/32 [00:03<00:08,  2.60it/s]
     31%|███▏      | 10/32 [00:03<00:08,  2.60it/s]
     34%|███▍      | 11/32 [00:04<00:08,  2.60it/s]
     38%|███▊      | 12/32 [00:04<00:07,  2.60it/s]
     41%|████      | 13/32 [00:04<00:07,  2.61it/s]
     44%|████▍     | 14/32 [00:05<00:06,  2.61it/s]
     47%|████▋     | 15/32 [00:05<00:06,  2.61it/s]
     50%|█████     | 16/32 [00:06<00:06,  2.60it/s]
     53%|█████▎    | 17/32 [00:06<00:05,  2.61it/s]
     56%|█████▋    | 18/32 [00:06<00:05,  2.61it/s]
     59%|█████▉    | 19/32 [00:07<00:04,  2.61it/s]
     62%|██████▎   | 20/32 [00:07<00:04,  2.61it/s]
     66%|██████▌   | 21/32 [00:08<00:04,  2.61it/s]
     69%|██████▉   | 22/32 [00:08<00:03,  2.61it/s]
     72%|███████▏  | 23/32 [00:08<00:03,  2.61it/s]
     75%|███████▌  | 24/32 [00:09<00:03,  2.60it/s]
     78%|███████▊  | 25/32 [00:09<00:02,  2.60it/s]
     81%|████████▏ | 26/32 [00:09<00:02,  2.60it/s]
     84%|████████▍ | 27/32 [00:10<00:01,  2.60it/s]
     88%|████████▊ | 28/32 [00:10<00:01,  2.60it/s]
     91%|█████████ | 29/32 [00:11<00:01,  2.60it/s]
     94%|█████████▍| 30/32 [00:11<00:00,  2.60it/s]
     97%|█████████▋| 31/32 [00:11<00:00,  2.60it/s]
    100%|██████████| 32/32 [00:11<00:00,  2.67it/s]




.. GENERATED FROM PYTHON SOURCE LINES 124-131

6. Iterating Again to Compute the Improved ECE
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

We can directly use the scaler as a calibrated model.

Note that you will need to first reset the ECE metric to avoid mixing the scores of
the previous and current iterations.

.. GENERATED FROM PYTHON SOURCE LINES 131-145

.. code-block:: Python


    # Reset the ECE
    ece.reset()

    # Iterate on the test dataloader
    for sample, target in test_dataloader:
        logits = scaled_model(sample)
        probs = logits.softmax(-1)
        ece.update(probs, target)

    print(
        f"ECE after scaling - {ece.compute():.3%} with temperature {scaled_model.temperature[0].item():.3}."
    )





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    ECE after scaling - 2.882% with temperature 1.38.




.. GENERATED FROM PYTHON SOURCE LINES 146-149

We finally compute and plot the scaled top-label calibration figure. We see
that the model is now better calibrated. If the temperature is greater than 1,
the final model is less confident than before.

.. GENERATED FROM PYTHON SOURCE LINES 149-153

.. code-block:: Python

    fig, ax = ece.plot()
    fig.tight_layout()
    fig.show()




.. image-sg:: /auto_tutorials/Post_Hoc_Methods/images/sphx_glr_tutorial_temperature_002.png
   :alt: Reliability Diagram
   :srcset: /auto_tutorials/Post_Hoc_Methods/images/sphx_glr_tutorial_temperature_002.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 154-164

The top-label calibration should be improved.

7. What about Vector Scaling?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The VectorScaler has as many parameters as the number of classes to temper the softmax.
Instead of a single parameter for all classes, vector scaling fits one temperature for each
output class.
It can be used just like the TemperatureScaler but we need to specify the number of classes.
Can it continue improving the calibration of our model?

.. GENERATED FROM PYTHON SOURCE LINES 164-188

.. code-block:: Python


    from torch_uncertainty.post_processing import VectorScaler

    # Fit the scaler on the calibration dataset
    scaled_model = VectorScaler(num_classes=100, model=model)
    scaled_model.fit(dataloader=calibration_dataloader)

    # Reset the ECE
    ece.reset()

    # Iterate on the test dataloader
    for sample, target in test_dataloader:
        logits = scaled_model(sample)
        probs = logits.softmax(-1)
        ece.update(probs, target)

    print(
        f"ECE after vector scaling - {ece.compute():.3%} with average temperature {scaled_model.temperature[0].mean():.3}."
    )

    fig, ax = ece.plot()
    fig.tight_layout()
    fig.show()




.. image-sg:: /auto_tutorials/Post_Hoc_Methods/images/sphx_glr_tutorial_temperature_003.png
   :alt: Reliability Diagram
   :srcset: /auto_tutorials/Post_Hoc_Methods/images/sphx_glr_tutorial_temperature_003.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none


      0%|          | 0/32 [00:00<?, ?it/s]
      3%|▎         | 1/32 [00:00<00:11,  2.65it/s]
      6%|▋         | 2/32 [00:00<00:11,  2.59it/s]
      9%|▉         | 3/32 [00:01<00:11,  2.60it/s]
     12%|█▎        | 4/32 [00:01<00:10,  2.59it/s]
     16%|█▌        | 5/32 [00:01<00:10,  2.59it/s]
     19%|█▉        | 6/32 [00:02<00:10,  2.59it/s]
     22%|██▏       | 7/32 [00:02<00:09,  2.59it/s]
     25%|██▌       | 8/32 [00:03<00:09,  2.59it/s]
     28%|██▊       | 9/32 [00:03<00:08,  2.59it/s]
     31%|███▏      | 10/32 [00:03<00:08,  2.59it/s]
     34%|███▍      | 11/32 [00:04<00:08,  2.59it/s]
     38%|███▊      | 12/32 [00:04<00:07,  2.59it/s]
     41%|████      | 13/32 [00:05<00:07,  2.58it/s]
     44%|████▍     | 14/32 [00:05<00:06,  2.58it/s]
     47%|████▋     | 15/32 [00:05<00:06,  2.58it/s]
     50%|█████     | 16/32 [00:06<00:06,  2.58it/s]
     53%|█████▎    | 17/32 [00:06<00:05,  2.58it/s]
     56%|█████▋    | 18/32 [00:06<00:05,  2.58it/s]
     59%|█████▉    | 19/32 [00:07<00:05,  2.58it/s]
     62%|██████▎   | 20/32 [00:07<00:04,  2.58it/s]
     66%|██████▌   | 21/32 [00:08<00:04,  2.59it/s]
     69%|██████▉   | 22/32 [00:08<00:03,  2.59it/s]
     72%|███████▏  | 23/32 [00:08<00:03,  2.58it/s]
     75%|███████▌  | 24/32 [00:09<00:03,  2.58it/s]
     78%|███████▊  | 25/32 [00:09<00:02,  2.58it/s]
     81%|████████▏ | 26/32 [00:10<00:02,  2.59it/s]
     84%|████████▍ | 27/32 [00:10<00:01,  2.58it/s]
     88%|████████▊ | 28/32 [00:10<00:01,  2.59it/s]
     91%|█████████ | 29/32 [00:11<00:01,  2.58it/s]
     94%|█████████▍| 30/32 [00:11<00:00,  2.59it/s]
     97%|█████████▋| 31/32 [00:11<00:00,  2.58it/s]
    100%|██████████| 32/32 [00:12<00:00,  2.65it/s]
    ECE after vector scaling - 3.002% with average temperature 1.36.




.. GENERATED FROM PYTHON SOURCE LINES 189-200

It is most likely not the case: we don't have much data to fit more parameters, and might
therefore overfit the calibration set to some extent.
Note that the VectorScaler can also change the predictions of the model.

8. What about Matrix Scaling?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The MatrixScaler has as a number of parameters equal to the square of the number of classes
to temper the softmax.
It can be used just like the TemperatureScaler but we need to specify the number of classes.
Can it continue improving the calibration of our model?

.. GENERATED FROM PYTHON SOURCE LINES 200-224

.. code-block:: Python


    from torch_uncertainty.post_processing import MatrixScaler

    # Fit the scaler on the calibration dataset
    scaled_model = MatrixScaler(num_classes=100, model=model)
    scaled_model.fit(dataloader=calibration_dataloader)

    # Reset the ECE
    ece.reset()

    # Iterate on the test dataloader
    for sample, target in test_dataloader:
        logits = scaled_model(sample)
        probs = logits.softmax(-1)
        ece.update(probs, target)

    print(
        f"ECE after matrix scaling - {ece.compute():.3%} with average diagonal temperature {scaled_model.temperature[0].diagonal().mean():.3}."
    )

    fig, ax = ece.plot()
    fig.tight_layout()
    fig.show()




.. image-sg:: /auto_tutorials/Post_Hoc_Methods/images/sphx_glr_tutorial_temperature_004.png
   :alt: Reliability Diagram
   :srcset: /auto_tutorials/Post_Hoc_Methods/images/sphx_glr_tutorial_temperature_004.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none


      0%|          | 0/32 [00:00<?, ?it/s]
      3%|▎         | 1/32 [00:00<00:11,  2.68it/s]
      6%|▋         | 2/32 [00:00<00:11,  2.61it/s]
      9%|▉         | 3/32 [00:01<00:11,  2.59it/s]
     12%|█▎        | 4/32 [00:01<00:10,  2.60it/s]
     16%|█▌        | 5/32 [00:01<00:10,  2.60it/s]
     19%|█▉        | 6/32 [00:02<00:10,  2.60it/s]
     22%|██▏       | 7/32 [00:02<00:09,  2.59it/s]
     25%|██▌       | 8/32 [00:03<00:09,  2.59it/s]
     28%|██▊       | 9/32 [00:03<00:08,  2.59it/s]
     31%|███▏      | 10/32 [00:03<00:08,  2.59it/s]
     34%|███▍      | 11/32 [00:04<00:08,  2.58it/s]
     38%|███▊      | 12/32 [00:04<00:07,  2.58it/s]
     41%|████      | 13/32 [00:05<00:07,  2.59it/s]
     44%|████▍     | 14/32 [00:05<00:06,  2.59it/s]
     47%|████▋     | 15/32 [00:05<00:06,  2.58it/s]
     50%|█████     | 16/32 [00:06<00:06,  2.58it/s]
     53%|█████▎    | 17/32 [00:06<00:05,  2.57it/s]
     56%|█████▋    | 18/32 [00:06<00:05,  2.57it/s]
     59%|█████▉    | 19/32 [00:07<00:05,  2.58it/s]
     62%|██████▎   | 20/32 [00:07<00:04,  2.57it/s]
     66%|██████▌   | 21/32 [00:08<00:04,  2.56it/s]
     69%|██████▉   | 22/32 [00:08<00:03,  2.57it/s]
     72%|███████▏  | 23/32 [00:08<00:03,  2.57it/s]
     75%|███████▌  | 24/32 [00:09<00:03,  2.58it/s]
     78%|███████▊  | 25/32 [00:09<00:02,  2.58it/s]
     81%|████████▏ | 26/32 [00:10<00:02,  2.58it/s]
     84%|████████▍ | 27/32 [00:10<00:01,  2.58it/s]
     88%|████████▊ | 28/32 [00:10<00:01,  2.58it/s]
     91%|█████████ | 29/32 [00:11<00:01,  2.58it/s]
     94%|█████████▍| 30/32 [00:11<00:00,  2.58it/s]
     97%|█████████▋| 31/32 [00:12<00:00,  2.58it/s]
    100%|██████████| 32/32 [00:12<00:00,  2.65it/s]
    ECE after matrix scaling - 44.206% with average diagonal temperature 0.0175.




.. GENERATED FROM PYTHON SOURCE LINES 225-234

Here it is a definitive no, we don't have enough data to fit a MatrixScaler in this case.
Note that the MatrixScaler can also change the predictions of the model.

9. Can Dirichlet Calibration help?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Dirichlet calibration is somewhat similar to matrix scaling, but performs the matrix multiplication
directly on the softmax values. Moreover, it includes a regularization mecanism to minimize the L2 norm of the
off-diagonal matrix coefficients (lambda_reg) and bias vector (mu_reg).

.. GENERATED FROM PYTHON SOURCE LINES 234-256

.. code-block:: Python


    from torch_uncertainty.post_processing import DirichletScaler

    # Fit the scaler on the calibration dataset
    scaled_model = DirichletScaler(num_classes=100, model=model, lambda_reg=1, mu_reg=1)
    scaled_model.fit(dataloader=calibration_dataloader)

    # Reset the ECE
    ece.reset()

    # Iterate on the test dataloader
    for sample, target in test_dataloader:
        logits = scaled_model(sample)
        probs = logits.softmax(-1)
        ece.update(probs, target)

    print(f"ECE after Dirichlet calibration - {ece.compute():.3%}.")

    fig, ax = ece.plot()
    fig.tight_layout()
    fig.show()




.. image-sg:: /auto_tutorials/Post_Hoc_Methods/images/sphx_glr_tutorial_temperature_005.png
   :alt: Reliability Diagram
   :srcset: /auto_tutorials/Post_Hoc_Methods/images/sphx_glr_tutorial_temperature_005.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none


      0%|          | 0/32 [00:00<?, ?it/s]
      3%|▎         | 1/32 [00:00<00:11,  2.73it/s]
      6%|▋         | 2/32 [00:00<00:11,  2.66it/s]
      9%|▉         | 3/32 [00:01<00:10,  2.64it/s]
     12%|█▎        | 4/32 [00:01<00:10,  2.63it/s]
     16%|█▌        | 5/32 [00:01<00:10,  2.62it/s]
     19%|█▉        | 6/32 [00:02<00:09,  2.62it/s]
     22%|██▏       | 7/32 [00:02<00:09,  2.62it/s]
     25%|██▌       | 8/32 [00:03<00:09,  2.62it/s]
     28%|██▊       | 9/32 [00:03<00:08,  2.62it/s]
     31%|███▏      | 10/32 [00:03<00:08,  2.62it/s]
     34%|███▍      | 11/32 [00:04<00:08,  2.62it/s]
     38%|███▊      | 12/32 [00:04<00:07,  2.61it/s]
     41%|████      | 13/32 [00:04<00:07,  2.61it/s]
     44%|████▍     | 14/32 [00:05<00:06,  2.61it/s]
     47%|████▋     | 15/32 [00:05<00:06,  2.61it/s]
     50%|█████     | 16/32 [00:06<00:06,  2.61it/s]
     53%|█████▎    | 17/32 [00:06<00:05,  2.61it/s]
     56%|█████▋    | 18/32 [00:06<00:05,  2.61it/s]
     59%|█████▉    | 19/32 [00:07<00:04,  2.61it/s]
     62%|██████▎   | 20/32 [00:07<00:04,  2.61it/s]
     66%|██████▌   | 21/32 [00:08<00:04,  2.61it/s]
     69%|██████▉   | 22/32 [00:08<00:03,  2.61it/s]
     72%|███████▏  | 23/32 [00:08<00:03,  2.61it/s]
     75%|███████▌  | 24/32 [00:09<00:03,  2.61it/s]
     78%|███████▊  | 25/32 [00:09<00:02,  2.61it/s]
     81%|████████▏ | 26/32 [00:09<00:02,  2.61it/s]
     84%|████████▍ | 27/32 [00:10<00:01,  2.61it/s]
     88%|████████▊ | 28/32 [00:10<00:01,  2.61it/s]
     91%|█████████ | 29/32 [00:11<00:01,  2.61it/s]
     94%|█████████▍| 30/32 [00:11<00:00,  2.61it/s]
     97%|█████████▋| 31/32 [00:11<00:00,  2.61it/s]
    100%|██████████| 32/32 [00:11<00:00,  2.68it/s]
    ECE after Dirichlet calibration - 16.906%.




.. GENERATED FROM PYTHON SOURCE LINES 257-272

The results are somewhat better than MatrixScaling, but we again likely do not have enough data to
correclty tune the parameters of Dirichlet scaling.

Remark
~~~~~~

Temperature scaling is very efficient when the calibration set is representative of the test set.
In this case, we say that the calibration and test set are drawn from the same distribution.
However, this may not hold true in real-world cases where dataset shift could happen.

References
----------

- **Expected Calibration Error:** Naeini, M. P., Cooper, G. F., & Hauskrecht, M. (2015). Obtaining Well Calibrated Probabilities Using Bayesian Binning. In `AAAI 2015 <https://arxiv.org/pdf/1411.0160.pdf>`_.
- **Temperature Scaling:** Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On calibration of modern neural networks. In `ICML 2017 <https://arxiv.org/pdf/1706.04599.pdf>`_.


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (2 minutes 24.360 seconds)


.. _sphx_glr_download_auto_tutorials_Post_Hoc_Methods_tutorial_temperature.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: tutorial_temperature.ipynb <tutorial_temperature.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: tutorial_temperature.py <tutorial_temperature.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: tutorial_temperature.zip <tutorial_temperature.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_