Note
Go to the end to download the full example code.
Improved Ensemble parameter-efficiency with Packed-Ensembles#
This tutorial is adapted from a notebook part of a lecture given at the `Helmholtz AI Conference <https://haicon24.de/>`_ by Sebastian Starke, Peter Steinbach, Gianni Franchi, and Olivier Laurent.
In this notebook will work on the MNIST dataset that was introduced by Corinna Cortes, Christopher J.C. Burges, and later modified by Yann LeCun in the foundational paper:
The MNIST dataset consists of 70 000 images of handwritten digits from 0 to 9. The images are grayscale and 28x28-pixel sized. The task is to classify the images into their respective digits. The dataset can be automatically downloaded using the torchvision library.
In this notebook, we will train a model and an ensemble on this task and evaluate their performance. The performance will consist in the following metrics: - Accuracy: the proportion of correctly classified images, - Brier score: a measure of the quality of the predicted probabilities, - Calibration error: a measure of the calibration of the predicted probabilities, - Negative Log-Likelihood: the value of the loss on the test set.
Throughout this notebook, we abstract the training and evaluation process using PyTorch Lightning and TorchUncertainty.
Similarly to keras for tensorflow, PyTorch Lightning is a high-level interface for PyTorch that simplifies the training and evaluation process using a Trainer. TorchUncertainty is partly built on top of PyTorch Lightning and provides tools to train and evaluate models with uncertainty quantification.
TorchUncertainty includes datamodules that handle the data loading and preprocessing. We don’t use them here for tutorial purposes.
1. Download, instantiate and visualize the datasets#
The dataset is automatically downloaded using torchvision. We then visualize a few images to see a bit what we are working with.
import torch
import torchvision.transforms as T
# We set the number of epochs to some very low value for the sake of time
MAX_EPOCHS = 3
# Create the transforms for the images
train_transform = T.Compose(
[
T.ToTensor(),
# We perform random cropping as data augmentation
T.RandomCrop(28, padding=4),
# As for the MNIST1d dataset, we normalize the data
T.Normalize((0.1307,), (0.3081,)),
]
)
test_transform = T.Compose(
[
T.Grayscale(num_output_channels=1),
T.ToTensor(),
T.CenterCrop(28),
T.Normalize((0.1307,), (0.3081,)),
]
)
# Download and instantiate the dataset
from torch.utils.data import Subset
from torchvision.datasets import MNIST, FashionMNIST
train_data = MNIST(root="./data/", download=True, train=True, transform=train_transform)
test_data = MNIST(root="./data/", train=False, transform=test_transform)
# We only take the first 10k images to have the same number of samples as the test set using torch Subsets
ood_data = Subset(
FashionMNIST(root="./data/", download=True, transform=test_transform),
indices=range(10000),
)
# Create the corresponding dataloaders
from torch.utils.data import DataLoader
train_dl = DataLoader(train_data, batch_size=512, shuffle=True, num_workers=8)
test_dl = DataLoader(test_data, batch_size=2048, shuffle=False, num_workers=4)
ood_dl = DataLoader(ood_data, batch_size=2048, shuffle=False, num_workers=4)
0%| | 0.00/9.91M [00:00<?, ?B/s]
1%| | 98.3k/9.91M [00:00<00:15, 624kB/s]
4%|▎ | 360k/9.91M [00:00<00:06, 1.46MB/s]
7%|▋ | 721k/9.91M [00:00<00:04, 2.12MB/s]
13%|█▎ | 1.31M/9.91M [00:00<00:02, 3.22MB/s]
24%|██▍ | 2.42M/9.91M [00:00<00:01, 5.35MB/s]
45%|████▌ | 4.49M/9.91M [00:00<00:00, 9.42MB/s]
55%|█████▌ | 5.47M/9.91M [00:00<00:00, 8.97MB/s]
97%|█████████▋| 9.57M/9.91M [00:00<00:00, 17.9MB/s]
100%|██████████| 9.91M/9.91M [00:01<00:00, 9.63MB/s]
0%| | 0.00/28.9k [00:00<?, ?B/s]
100%|██████████| 28.9k/28.9k [00:00<00:00, 367kB/s]
0%| | 0.00/1.65M [00:00<?, ?B/s]
6%|▌ | 98.3k/1.65M [00:00<00:02, 622kB/s]
22%|██▏ | 360k/1.65M [00:00<00:01, 1.23MB/s]
48%|████▊ | 786k/1.65M [00:00<00:00, 2.18MB/s]
81%|████████▏ | 1.34M/1.65M [00:00<00:00, 3.17MB/s]
100%|██████████| 1.65M/1.65M [00:00<00:00, 2.96MB/s]
0%| | 0.00/4.54k [00:00<?, ?B/s]
100%|██████████| 4.54k/4.54k [00:00<00:00, 9.88MB/s]
0%| | 0.00/26.4M [00:00<?, ?B/s]
15%|█▍ | 3.83M/26.4M [00:00<00:00, 38.2MB/s]
58%|█████▊ | 15.4M/26.4M [00:00<00:00, 83.8MB/s]
100%|██████████| 26.4M/26.4M [00:00<00:00, 89.2MB/s]
0%| | 0.00/29.5k [00:00<?, ?B/s]
100%|██████████| 29.5k/29.5k [00:00<00:00, 3.72MB/s]
0%| | 0.00/4.42M [00:00<?, ?B/s]
84%|████████▍ | 3.74M/4.42M [00:00<00:00, 37.2MB/s]
100%|██████████| 4.42M/4.42M [00:00<00:00, 41.2MB/s]
0%| | 0.00/5.15k [00:00<?, ?B/s]
100%|██████████| 5.15k/5.15k [00:00<00:00, 21.1MB/s]
You could replace all this cell by simply loading the MNIST datamodule from TorchUncertainty. Now, let’s visualize a few images from the dataset. For this task, we use the viz_data dataset that applies no transformation to the images.
# Datasets without transformation to visualize the unchanged data
viz_data = MNIST(root="./data/", train=False)
ood_viz_data = FashionMNIST(root="./data/", download=True)
print("In distribution data:")
viz_data[0][0]
In distribution data:
<PIL.Image.Image image mode=L size=28x28 at 0x77AF223C7290>
print("Out of distribution data:")
ood_viz_data[0][0]
Out of distribution data:
<PIL.Image.Image image mode=L size=28x28 at 0x77AF223C7B90>
2. Create & train the model#
We will create a simple convolutional neural network (CNN): the LeNet model (also introduced by LeCun).
import torch.nn as nn
import torch.nn.functional as F
class LeNet(nn.Module):
def __init__(
self,
in_channels: int,
num_classes: int,
) -> None:
super().__init__()
self.conv1 = nn.Conv2d(in_channels, 6, (5, 5))
self.conv2 = nn.Conv2d(6, 16, (5, 5))
self.pooling = nn.AdaptiveAvgPool2d((4, 4))
self.fc1 = nn.Linear(256, 120)
self.fc2 = nn.Linear(120, 84)
self.fc3 = nn.Linear(84, num_classes)
def forward(self, x: torch.Tensor) -> torch.Tensor:
out = F.relu(self.conv1(x))
out = F.max_pool2d(out, 2)
out = F.relu(self.conv2(out))
out = F.max_pool2d(out, 2)
out = torch.flatten(out, 1)
out = F.relu(self.fc1(out))
out = F.relu(self.fc2(out))
return self.fc3(out) # No softmax in the model!
# Instantiate the model, the images are in grayscale so the number of channels is 1
model = LeNet(in_channels=1, num_classes=10)
We now need to define the optimization recipe: - the optimizer, here the standard stochastic gradient descent (SGD) with a learning rate of 0.05 - the scheduler, here cosine annealing.
def optim_recipe(model, lr_mult: float = 1.0):
optimizer = torch.optim.SGD(model.parameters(), lr=0.05 * lr_mult)
scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=10)
return {"optimizer": optimizer, "scheduler": scheduler}
To train the model, we use TorchUncertainty, a library that we have developed to ease the training and evaluation of models with uncertainty.
Note: To train supervised classification models we most often use the cross-entropy loss. With weight-decay, minimizing this loss amounts to finding a Maximum a posteriori (MAP) estimate of the model parameters. This means that the model is trained to predict the most likely class for each input given a diagonal Gaussian prior on the weights.
from torch_uncertainty import TUTrainer
from torch_uncertainty.routines import ClassificationRoutine
# Create the trainer that will handle the training
trainer = TUTrainer(accelerator="gpu", devices=1, max_epochs=MAX_EPOCHS, enable_progress_bar=False)
# The routine is a wrapper of the model that contains the training logic with the metrics, etc
routine = ClassificationRoutine(
num_classes=10,
model=model,
loss=nn.CrossEntropyLoss(),
optim_recipe=optim_recipe(model),
eval_ood=True,
)
# In practice, avoid performing the validation on the test set (if you do model selection)
trainer.fit(routine, train_dataloaders=train_dl, val_dataloaders=test_dl)
Evaluate the trained model on the test set - pay attention to the cls/Acc metric
perf = trainer.test(routine, dataloaders=[test_dl, ood_dl])
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Test metric ┃ Classification ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Acc │ 85.730% │
│ Brier │ 0.22017 │
│ Entropy │ 0.65978 │
│ NLL │ 0.47970 │
└──────────────┴───────────────────────────┘
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Test metric ┃ Calibration ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ ECE │ 7.709% │
│ aECE │ 7.682% │
└──────────────┴───────────────────────────┘
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Test metric ┃ OOD Detection ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ AUPR │ 77.939% │
│ AUROC │ 79.149% │
│ Entropy │ 0.65978 │
│ FPR95 │ 64.490% │
└──────────────┴───────────────────────────┘
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Test metric ┃ Selective Classification ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ AUGRC │ 2.649% │
│ AURC │ 3.340% │
│ Cov@5Risk │ 73.740% │
│ Risk@80Cov │ 6.387% │
└──────────────┴───────────────────────────┘
This table provides a lot of information:
OOD Detection: Binary Classification MNIST vs. FashionMNIST - AUPR/AUROC/FPR95: Measures the quality of the OOD detection. The higher the better for AUPR and AUROC, the lower the better for FPR95.
Calibration: Reliability of the Predictions - ECE: Expected Calibration Error. The lower the better. - aECE: Adaptive Expected Calibration Error. The lower the better. (~More precise version of the ECE)
Classification Performance - Accuracy: The ratio of correctly classified images. The higher the better. - Brier: The quality of the predicted probabilities (Mean Squared Error of the predictions vs. ground-truth). The lower the better. - Negative Log-Likelihood: The value of the loss on the test set. The lower the better.
Selective Classification & Grouping Loss - We talk about these points later in the “To go further” section.
By setting eval_shift to True, we could also evaluate the performance of the models on MNIST-C, a dataset close to MNIST but with perturbations.
3. Training an ensemble of models with TorchUncertainty#
You have two options here, you can either train the ensemble directly if you have enough memory, otherwise, you can train independent models and do the ensembling during the evaluation (sometimes called inference).
In this case, we will do it sequentially. In this tutorial, you have the choice between training multiple models, which will take time if you have no GPU, or downloading the pre-trained models that we have prepared for you.
Training the ensemble
To train the ensemble, you will have to use the “deep_ensembles” function from TorchUncertainty, which will replicate and change the initialization of your networks to ensure diversity.
from torch_uncertainty.models import deep_ensembles
from torch_uncertainty.transforms import RepeatTarget
# Create the ensemble model
ensemble = deep_ensembles(
LeNet(in_channels=1, num_classes=10),
num_estimators=2,
task="classification",
reset_model_parameters=True,
)
trainer = TUTrainer(accelerator="gpu", devices=1, max_epochs=MAX_EPOCHS)
ens_routine = ClassificationRoutine(
is_ensemble=True,
num_classes=10,
model=ensemble,
loss=nn.CrossEntropyLoss(), # The loss for the training
format_batch_fn=RepeatTarget(2), # How to handle the targets when comparing the predictions
optim_recipe=optim_recipe(
ensemble, 2.0
), # The optimization scheme with the optimizer and the scheduler as a dictionnary
eval_ood=True, # We want to evaluate the OOD-related metrics
)
trainer.fit(ens_routine, train_dataloaders=train_dl, val_dataloaders=test_dl)
ens_perf = trainer.test(ens_routine, dataloaders=[test_dl, ood_dl])
Sanity Checking: | | 0/? [00:00<?, ?it/s]
Sanity Checking: 0%| | 0/2 [00:00<?, ?it/s]
Sanity Checking DataLoader 0: 0%| | 0/2 [00:00<?, ?it/s]
Sanity Checking DataLoader 0: 50%|█████ | 1/2 [00:00<00:00, 373.56it/s]
Sanity Checking DataLoader 0: 100%|██████████| 2/2 [00:00<00:00, 324.00it/s]
Training: | | 0/? [00:00<?, ?it/s]
Training: 0%| | 0/118 [00:00<?, ?it/s]
Epoch 0: 0%| | 0/118 [00:00<?, ?it/s]
Epoch 0: 1%| | 1/118 [00:00<00:10, 11.25it/s]
Epoch 0: 1%| | 1/118 [00:00<00:10, 11.15it/s, v_num=1, train_loss=2.310]
Epoch 0: 2%|▏ | 2/118 [00:00<00:05, 21.61it/s, v_num=1, train_loss=2.310]
Epoch 0: 2%|▏ | 2/118 [00:00<00:05, 21.37it/s, v_num=1, train_loss=2.310]
Epoch 0: 3%|▎ | 3/118 [00:00<00:04, 24.97it/s, v_num=1, train_loss=2.310]
Epoch 0: 3%|▎ | 3/118 [00:00<00:04, 24.82it/s, v_num=1, train_loss=2.300]
Epoch 0: 3%|▎ | 4/118 [00:00<00:03, 32.46it/s, v_num=1, train_loss=2.300]
Epoch 0: 3%|▎ | 4/118 [00:00<00:03, 32.17it/s, v_num=1, train_loss=2.310]
Epoch 0: 4%|▍ | 5/118 [00:00<00:02, 39.51it/s, v_num=1, train_loss=2.310]
Epoch 0: 4%|▍ | 5/118 [00:00<00:02, 39.17it/s, v_num=1, train_loss=2.310]
Epoch 0: 5%|▌ | 6/118 [00:00<00:02, 46.14it/s, v_num=1, train_loss=2.310]
Epoch 0: 5%|▌ | 6/118 [00:00<00:02, 45.78it/s, v_num=1, train_loss=2.310]
Epoch 0: 6%|▌ | 7/118 [00:00<00:02, 52.48it/s, v_num=1, train_loss=2.310]
Epoch 0: 6%|▌ | 7/118 [00:00<00:02, 52.08it/s, v_num=1, train_loss=2.300]
Epoch 0: 7%|▋ | 8/118 [00:00<00:01, 57.89it/s, v_num=1, train_loss=2.300]
Epoch 0: 7%|▋ | 8/118 [00:00<00:01, 57.76it/s, v_num=1, train_loss=2.300]
Epoch 0: 8%|▊ | 9/118 [00:00<00:01, 61.71it/s, v_num=1, train_loss=2.300]
Epoch 0: 8%|▊ | 9/118 [00:00<00:01, 61.61it/s, v_num=1, train_loss=2.300]
Epoch 0: 8%|▊ | 10/118 [00:00<00:01, 66.03it/s, v_num=1, train_loss=2.300]
Epoch 0: 8%|▊ | 10/118 [00:00<00:01, 65.91it/s, v_num=1, train_loss=2.300]
Epoch 0: 9%|▉ | 11/118 [00:00<00:01, 68.58it/s, v_num=1, train_loss=2.300]
Epoch 0: 9%|▉ | 11/118 [00:00<00:01, 68.46it/s, v_num=1, train_loss=2.300]
Epoch 0: 10%|█ | 12/118 [00:00<00:01, 72.18it/s, v_num=1, train_loss=2.300]
Epoch 0: 10%|█ | 12/118 [00:00<00:01, 72.06it/s, v_num=1, train_loss=2.310]
Epoch 0: 11%|█ | 13/118 [00:00<00:01, 75.73it/s, v_num=1, train_loss=2.310]
Epoch 0: 11%|█ | 13/118 [00:00<00:01, 75.60it/s, v_num=1, train_loss=2.300]
Epoch 0: 12%|█▏ | 14/118 [00:00<00:01, 79.40it/s, v_num=1, train_loss=2.300]
Epoch 0: 12%|█▏ | 14/118 [00:00<00:01, 79.28it/s, v_num=1, train_loss=2.310]
Epoch 0: 13%|█▎ | 15/118 [00:00<00:01, 83.16it/s, v_num=1, train_loss=2.310]
Epoch 0: 13%|█▎ | 15/118 [00:00<00:01, 83.03it/s, v_num=1, train_loss=2.300]
Epoch 0: 14%|█▎ | 16/118 [00:00<00:01, 86.77it/s, v_num=1, train_loss=2.300]
Epoch 0: 14%|█▎ | 16/118 [00:00<00:01, 86.66it/s, v_num=1, train_loss=2.310]
Epoch 0: 14%|█▍ | 17/118 [00:00<00:01, 88.31it/s, v_num=1, train_loss=2.310]
Epoch 0: 14%|█▍ | 17/118 [00:00<00:01, 88.17it/s, v_num=1, train_loss=2.300]
Epoch 0: 15%|█▌ | 18/118 [00:00<00:01, 90.70it/s, v_num=1, train_loss=2.300]
Epoch 0: 15%|█▌ | 18/118 [00:00<00:01, 90.57it/s, v_num=1, train_loss=2.310]
Epoch 0: 16%|█▌ | 19/118 [00:00<00:01, 92.23it/s, v_num=1, train_loss=2.310]
Epoch 0: 16%|█▌ | 19/118 [00:00<00:01, 92.04it/s, v_num=1, train_loss=2.300]
Epoch 0: 17%|█▋ | 20/118 [00:00<00:01, 95.29it/s, v_num=1, train_loss=2.300]
Epoch 0: 17%|█▋ | 20/118 [00:00<00:01, 95.12it/s, v_num=1, train_loss=2.300]
Epoch 0: 18%|█▊ | 21/118 [00:00<00:00, 98.22it/s, v_num=1, train_loss=2.300]
Epoch 0: 18%|█▊ | 21/118 [00:00<00:00, 98.11it/s, v_num=1, train_loss=2.300]
Epoch 0: 19%|█▊ | 22/118 [00:00<00:01, 84.62it/s, v_num=1, train_loss=2.300]
Epoch 0: 19%|█▊ | 22/118 [00:00<00:01, 84.51it/s, v_num=1, train_loss=2.300]
Epoch 0: 19%|█▉ | 23/118 [00:00<00:01, 86.86it/s, v_num=1, train_loss=2.300]
Epoch 0: 19%|█▉ | 23/118 [00:00<00:01, 86.74it/s, v_num=1, train_loss=2.300]
Epoch 0: 20%|██ | 24/118 [00:00<00:01, 88.90it/s, v_num=1, train_loss=2.300]
Epoch 0: 20%|██ | 24/118 [00:00<00:01, 88.79it/s, v_num=1, train_loss=2.300]
Epoch 0: 21%|██ | 25/118 [00:00<00:01, 91.17it/s, v_num=1, train_loss=2.300]
Epoch 0: 21%|██ | 25/118 [00:00<00:01, 91.00it/s, v_num=1, train_loss=2.300]
Epoch 0: 22%|██▏ | 26/118 [00:00<00:00, 93.90it/s, v_num=1, train_loss=2.300]
Epoch 0: 22%|██▏ | 26/118 [00:00<00:00, 93.52it/s, v_num=1, train_loss=2.300]
Epoch 0: 23%|██▎ | 27/118 [00:00<00:00, 96.36it/s, v_num=1, train_loss=2.300]
Epoch 0: 23%|██▎ | 27/118 [00:00<00:00, 95.98it/s, v_num=1, train_loss=2.300]
Epoch 0: 24%|██▎ | 28/118 [00:00<00:00, 98.75it/s, v_num=1, train_loss=2.300]
Epoch 0: 24%|██▎ | 28/118 [00:00<00:00, 98.36it/s, v_num=1, train_loss=2.300]
Epoch 0: 25%|██▍ | 29/118 [00:00<00:00, 101.07it/s, v_num=1, train_loss=2.300]
Epoch 0: 25%|██▍ | 29/118 [00:00<00:00, 100.66it/s, v_num=1, train_loss=2.300]
Epoch 0: 25%|██▌ | 30/118 [00:00<00:00, 101.04it/s, v_num=1, train_loss=2.300]
Epoch 0: 25%|██▌ | 30/118 [00:00<00:00, 100.95it/s, v_num=1, train_loss=2.300]
Epoch 0: 26%|██▋ | 31/118 [00:00<00:00, 102.51it/s, v_num=1, train_loss=2.300]
Epoch 0: 26%|██▋ | 31/118 [00:00<00:00, 102.42it/s, v_num=1, train_loss=2.300]
Epoch 0: 27%|██▋ | 32/118 [00:00<00:00, 103.94it/s, v_num=1, train_loss=2.300]
Epoch 0: 27%|██▋ | 32/118 [00:00<00:00, 103.86it/s, v_num=1, train_loss=2.300]
Epoch 0: 28%|██▊ | 33/118 [00:00<00:00, 105.26it/s, v_num=1, train_loss=2.300]
Epoch 0: 28%|██▊ | 33/118 [00:00<00:00, 105.16it/s, v_num=1, train_loss=2.300]
Epoch 0: 29%|██▉ | 34/118 [00:00<00:00, 106.64it/s, v_num=1, train_loss=2.300]
Epoch 0: 29%|██▉ | 34/118 [00:00<00:00, 106.54it/s, v_num=1, train_loss=2.300]
Epoch 0: 30%|██▉ | 35/118 [00:00<00:00, 107.90it/s, v_num=1, train_loss=2.300]
Epoch 0: 30%|██▉ | 35/118 [00:00<00:00, 107.81it/s, v_num=1, train_loss=2.300]
Epoch 0: 31%|███ | 36/118 [00:00<00:00, 109.04it/s, v_num=1, train_loss=2.300]
Epoch 0: 31%|███ | 36/118 [00:00<00:00, 108.95it/s, v_num=1, train_loss=2.300]
Epoch 0: 31%|███▏ | 37/118 [00:00<00:00, 110.20it/s, v_num=1, train_loss=2.300]
Epoch 0: 31%|███▏ | 37/118 [00:00<00:00, 110.11it/s, v_num=1, train_loss=2.300]
Epoch 0: 32%|███▏ | 38/118 [00:00<00:00, 97.58it/s, v_num=1, train_loss=2.300]
Epoch 0: 32%|███▏ | 38/118 [00:00<00:00, 97.46it/s, v_num=1, train_loss=2.300]
Epoch 0: 33%|███▎ | 39/118 [00:00<00:00, 99.23it/s, v_num=1, train_loss=2.300]
Epoch 0: 33%|███▎ | 39/118 [00:00<00:00, 99.08it/s, v_num=1, train_loss=2.300]
Epoch 0: 34%|███▍ | 40/118 [00:00<00:00, 100.83it/s, v_num=1, train_loss=2.300]
Epoch 0: 34%|███▍ | 40/118 [00:00<00:00, 100.71it/s, v_num=1, train_loss=2.290]
Epoch 0: 35%|███▍ | 41/118 [00:00<00:00, 102.44it/s, v_num=1, train_loss=2.290]
Epoch 0: 35%|███▍ | 41/118 [00:00<00:00, 102.32it/s, v_num=1, train_loss=2.300]
Epoch 0: 36%|███▌ | 42/118 [00:00<00:00, 104.03it/s, v_num=1, train_loss=2.300]
Epoch 0: 36%|███▌ | 42/118 [00:00<00:00, 103.90it/s, v_num=1, train_loss=2.290]
Epoch 0: 36%|███▋ | 43/118 [00:00<00:00, 105.50it/s, v_num=1, train_loss=2.290]
Epoch 0: 36%|███▋ | 43/118 [00:00<00:00, 105.42it/s, v_num=1, train_loss=2.300]
Epoch 0: 37%|███▋ | 44/118 [00:00<00:00, 106.54it/s, v_num=1, train_loss=2.300]
Epoch 0: 37%|███▋ | 44/118 [00:00<00:00, 106.48it/s, v_num=1, train_loss=2.300]
Epoch 0: 38%|███▊ | 45/118 [00:00<00:00, 107.56it/s, v_num=1, train_loss=2.300]
Epoch 0: 38%|███▊ | 45/118 [00:00<00:00, 107.49it/s, v_num=1, train_loss=2.300]
Epoch 0: 39%|███▉ | 46/118 [00:00<00:00, 107.68it/s, v_num=1, train_loss=2.300]
Epoch 0: 39%|███▉ | 46/118 [00:00<00:00, 107.64it/s, v_num=1, train_loss=2.300]
Epoch 0: 40%|███▉ | 47/118 [00:00<00:00, 109.04it/s, v_num=1, train_loss=2.300]
Epoch 0: 40%|███▉ | 47/118 [00:00<00:00, 108.97it/s, v_num=1, train_loss=2.300]
Epoch 0: 41%|████ | 48/118 [00:00<00:00, 110.39it/s, v_num=1, train_loss=2.300]
Epoch 0: 41%|████ | 48/118 [00:00<00:00, 110.32it/s, v_num=1, train_loss=2.290]
Epoch 0: 42%|████▏ | 49/118 [00:00<00:00, 111.73it/s, v_num=1, train_loss=2.290]
Epoch 0: 42%|████▏ | 49/118 [00:00<00:00, 111.65it/s, v_num=1, train_loss=2.300]
Epoch 0: 42%|████▏ | 50/118 [00:00<00:00, 113.04it/s, v_num=1, train_loss=2.300]
Epoch 0: 42%|████▏ | 50/118 [00:00<00:00, 112.97it/s, v_num=1, train_loss=2.300]
Epoch 0: 43%|████▎ | 51/118 [00:00<00:00, 114.17it/s, v_num=1, train_loss=2.300]
Epoch 0: 43%|████▎ | 51/118 [00:00<00:00, 114.10it/s, v_num=1, train_loss=2.300]
Epoch 0: 44%|████▍ | 52/118 [00:00<00:00, 114.92it/s, v_num=1, train_loss=2.300]
Epoch 0: 44%|████▍ | 52/118 [00:00<00:00, 114.86it/s, v_num=1, train_loss=2.290]
Epoch 0: 45%|████▍ | 53/118 [00:00<00:00, 115.71it/s, v_num=1, train_loss=2.290]
Epoch 0: 45%|████▍ | 53/118 [00:00<00:00, 115.65it/s, v_num=1, train_loss=2.290]
Epoch 0: 46%|████▌ | 54/118 [00:00<00:00, 114.77it/s, v_num=1, train_loss=2.290]
Epoch 0: 46%|████▌ | 54/118 [00:00<00:00, 114.70it/s, v_num=1, train_loss=2.290]
Epoch 0: 47%|████▋ | 55/118 [00:00<00:00, 115.59it/s, v_num=1, train_loss=2.290]
Epoch 0: 47%|████▋ | 55/118 [00:00<00:00, 115.53it/s, v_num=1, train_loss=2.290]
Epoch 0: 47%|████▋ | 56/118 [00:00<00:00, 110.28it/s, v_num=1, train_loss=2.290]
Epoch 0: 47%|████▋ | 56/118 [00:00<00:00, 110.21it/s, v_num=1, train_loss=2.290]
Epoch 0: 48%|████▊ | 57/118 [00:00<00:00, 111.17it/s, v_num=1, train_loss=2.290]
Epoch 0: 48%|████▊ | 57/118 [00:00<00:00, 111.08it/s, v_num=1, train_loss=2.290]
Epoch 0: 49%|████▉ | 58/118 [00:00<00:00, 112.07it/s, v_num=1, train_loss=2.290]
Epoch 0: 49%|████▉ | 58/118 [00:00<00:00, 111.90it/s, v_num=1, train_loss=2.290]
Epoch 0: 50%|█████ | 59/118 [00:00<00:00, 112.68it/s, v_num=1, train_loss=2.290]
Epoch 0: 50%|█████ | 59/118 [00:00<00:00, 112.48it/s, v_num=1, train_loss=2.290]
Epoch 0: 51%|█████ | 60/118 [00:00<00:00, 113.48it/s, v_num=1, train_loss=2.290]
Epoch 0: 51%|█████ | 60/118 [00:00<00:00, 113.34it/s, v_num=1, train_loss=2.290]
Epoch 0: 52%|█████▏ | 61/118 [00:00<00:00, 114.33it/s, v_num=1, train_loss=2.290]
Epoch 0: 52%|█████▏ | 61/118 [00:00<00:00, 114.19it/s, v_num=1, train_loss=2.290]
Epoch 0: 53%|█████▎ | 62/118 [00:00<00:00, 115.16it/s, v_num=1, train_loss=2.290]
Epoch 0: 53%|█████▎ | 62/118 [00:00<00:00, 115.02it/s, v_num=1, train_loss=2.290]
Epoch 0: 53%|█████▎ | 63/118 [00:00<00:00, 115.83it/s, v_num=1, train_loss=2.290]
Epoch 0: 53%|█████▎ | 63/118 [00:00<00:00, 115.71it/s, v_num=1, train_loss=2.290]
Epoch 0: 54%|█████▍ | 64/118 [00:00<00:00, 116.07it/s, v_num=1, train_loss=2.290]
Epoch 0: 54%|█████▍ | 64/118 [00:00<00:00, 115.94it/s, v_num=1, train_loss=2.290]
Epoch 0: 55%|█████▌ | 65/118 [00:00<00:00, 116.61it/s, v_num=1, train_loss=2.290]
Epoch 0: 55%|█████▌ | 65/118 [00:00<00:00, 116.55it/s, v_num=1, train_loss=2.290]
Epoch 0: 56%|█████▌ | 66/118 [00:00<00:00, 117.28it/s, v_num=1, train_loss=2.290]
Epoch 0: 56%|█████▌ | 66/118 [00:00<00:00, 117.22it/s, v_num=1, train_loss=2.290]
Epoch 0: 57%|█████▋ | 67/118 [00:00<00:00, 117.43it/s, v_num=1, train_loss=2.290]
Epoch 0: 57%|█████▋ | 67/118 [00:00<00:00, 117.37it/s, v_num=1, train_loss=2.290]
Epoch 0: 58%|█████▊ | 68/118 [00:00<00:00, 118.06it/s, v_num=1, train_loss=2.290]
Epoch 0: 58%|█████▊ | 68/118 [00:00<00:00, 117.99it/s, v_num=1, train_loss=2.290]
Epoch 0: 58%|█████▊ | 69/118 [00:00<00:00, 118.66it/s, v_num=1, train_loss=2.290]
Epoch 0: 58%|█████▊ | 69/118 [00:00<00:00, 118.61it/s, v_num=1, train_loss=2.290]
Epoch 0: 59%|█████▉ | 70/118 [00:00<00:00, 119.26it/s, v_num=1, train_loss=2.290]
Epoch 0: 59%|█████▉ | 70/118 [00:00<00:00, 119.20it/s, v_num=1, train_loss=2.290]
Epoch 0: 60%|██████ | 71/118 [00:00<00:00, 119.81it/s, v_num=1, train_loss=2.290]
Epoch 0: 60%|██████ | 71/118 [00:00<00:00, 119.75it/s, v_num=1, train_loss=2.290]
Epoch 0: 61%|██████ | 72/118 [00:00<00:00, 119.91it/s, v_num=1, train_loss=2.290]
Epoch 0: 61%|██████ | 72/118 [00:00<00:00, 119.85it/s, v_num=1, train_loss=2.290]
Epoch 0: 62%|██████▏ | 73/118 [00:00<00:00, 120.51it/s, v_num=1, train_loss=2.290]
Epoch 0: 62%|██████▏ | 73/118 [00:00<00:00, 120.47it/s, v_num=1, train_loss=2.290]
Epoch 0: 63%|██████▎ | 74/118 [00:00<00:00, 121.11it/s, v_num=1, train_loss=2.290]
Epoch 0: 63%|██████▎ | 74/118 [00:00<00:00, 121.06it/s, v_num=1, train_loss=2.290]
Epoch 0: 64%|██████▎ | 75/118 [00:00<00:00, 119.43it/s, v_num=1, train_loss=2.290]
Epoch 0: 64%|██████▎ | 75/118 [00:00<00:00, 119.38it/s, v_num=1, train_loss=2.290]
Epoch 0: 64%|██████▍ | 76/118 [00:00<00:00, 120.05it/s, v_num=1, train_loss=2.290]
Epoch 0: 64%|██████▍ | 76/118 [00:00<00:00, 120.00it/s, v_num=1, train_loss=2.280]
Epoch 0: 65%|██████▌ | 77/118 [00:00<00:00, 120.63it/s, v_num=1, train_loss=2.280]
Epoch 0: 65%|██████▌ | 77/118 [00:00<00:00, 120.58it/s, v_num=1, train_loss=2.290]
Epoch 0: 66%|██████▌ | 78/118 [00:00<00:00, 121.23it/s, v_num=1, train_loss=2.290]
Epoch 0: 66%|██████▌ | 78/118 [00:00<00:00, 121.19it/s, v_num=1, train_loss=2.290]
Epoch 0: 67%|██████▋ | 79/118 [00:00<00:00, 121.77it/s, v_num=1, train_loss=2.290]
Epoch 0: 67%|██████▋ | 79/118 [00:00<00:00, 121.72it/s, v_num=1, train_loss=2.280]
Epoch 0: 68%|██████▊ | 80/118 [00:00<00:00, 122.27it/s, v_num=1, train_loss=2.280]
Epoch 0: 68%|██████▊ | 80/118 [00:00<00:00, 122.21it/s, v_num=1, train_loss=2.290]
Epoch 0: 69%|██████▊ | 81/118 [00:00<00:00, 123.08it/s, v_num=1, train_loss=2.290]
Epoch 0: 69%|██████▊ | 81/118 [00:00<00:00, 123.02it/s, v_num=1, train_loss=2.280]
Epoch 0: 69%|██████▉ | 82/118 [00:00<00:00, 123.88it/s, v_num=1, train_loss=2.280]
Epoch 0: 69%|██████▉ | 82/118 [00:00<00:00, 123.83it/s, v_num=1, train_loss=2.280]
Epoch 0: 70%|███████ | 83/118 [00:00<00:00, 123.26it/s, v_num=1, train_loss=2.280]
Epoch 0: 70%|███████ | 83/118 [00:00<00:00, 123.21it/s, v_num=1, train_loss=2.280]
Epoch 0: 71%|███████ | 84/118 [00:00<00:00, 123.76it/s, v_num=1, train_loss=2.280]
Epoch 0: 71%|███████ | 84/118 [00:00<00:00, 123.71it/s, v_num=1, train_loss=2.280]
Epoch 0: 72%|███████▏ | 85/118 [00:00<00:00, 124.22it/s, v_num=1, train_loss=2.280]
Epoch 0: 72%|███████▏ | 85/118 [00:00<00:00, 124.18it/s, v_num=1, train_loss=2.280]
Epoch 0: 73%|███████▎ | 86/118 [00:00<00:00, 124.67it/s, v_num=1, train_loss=2.280]
Epoch 0: 73%|███████▎ | 86/118 [00:00<00:00, 124.62it/s, v_num=1, train_loss=2.280]
Epoch 0: 74%|███████▎ | 87/118 [00:00<00:00, 125.14it/s, v_num=1, train_loss=2.280]
Epoch 0: 74%|███████▎ | 87/118 [00:00<00:00, 125.10it/s, v_num=1, train_loss=2.280]
Epoch 0: 75%|███████▍ | 88/118 [00:00<00:00, 125.61it/s, v_num=1, train_loss=2.280]
Epoch 0: 75%|███████▍ | 88/118 [00:00<00:00, 125.56it/s, v_num=1, train_loss=2.280]
Epoch 0: 75%|███████▌ | 89/118 [00:00<00:00, 126.03it/s, v_num=1, train_loss=2.280]
Epoch 0: 75%|███████▌ | 89/118 [00:00<00:00, 125.98it/s, v_num=1, train_loss=2.280]
Epoch 0: 76%|███████▋ | 90/118 [00:00<00:00, 126.44it/s, v_num=1, train_loss=2.280]
Epoch 0: 76%|███████▋ | 90/118 [00:00<00:00, 126.39it/s, v_num=1, train_loss=2.280]
Epoch 0: 77%|███████▋ | 91/118 [00:00<00:00, 126.16it/s, v_num=1, train_loss=2.280]
Epoch 0: 77%|███████▋ | 91/118 [00:00<00:00, 126.11it/s, v_num=1, train_loss=2.280]
Epoch 0: 78%|███████▊ | 92/118 [00:00<00:00, 126.61it/s, v_num=1, train_loss=2.280]
Epoch 0: 78%|███████▊ | 92/118 [00:00<00:00, 126.56it/s, v_num=1, train_loss=2.280]
Epoch 0: 79%|███████▉ | 93/118 [00:00<00:00, 127.02it/s, v_num=1, train_loss=2.280]
Epoch 0: 79%|███████▉ | 93/118 [00:00<00:00, 126.98it/s, v_num=1, train_loss=2.280]
Epoch 0: 80%|███████▉ | 94/118 [00:00<00:00, 127.46it/s, v_num=1, train_loss=2.280]
Epoch 0: 80%|███████▉ | 94/118 [00:00<00:00, 127.41it/s, v_num=1, train_loss=2.270]
Epoch 0: 81%|████████ | 95/118 [00:00<00:00, 122.00it/s, v_num=1, train_loss=2.270]
Epoch 0: 81%|████████ | 95/118 [00:00<00:00, 121.95it/s, v_num=1, train_loss=2.270]
Epoch 0: 81%|████████▏ | 96/118 [00:00<00:00, 122.48it/s, v_num=1, train_loss=2.270]
Epoch 0: 81%|████████▏ | 96/118 [00:00<00:00, 122.43it/s, v_num=1, train_loss=2.280]
Epoch 0: 82%|████████▏ | 97/118 [00:00<00:00, 123.20it/s, v_num=1, train_loss=2.280]
Epoch 0: 82%|████████▏ | 97/118 [00:00<00:00, 123.14it/s, v_num=1, train_loss=2.270]
Epoch 0: 83%|████████▎ | 98/118 [00:00<00:00, 123.92it/s, v_num=1, train_loss=2.270]
Epoch 0: 83%|████████▎ | 98/118 [00:00<00:00, 123.86it/s, v_num=1, train_loss=2.270]
Epoch 0: 84%|████████▍ | 99/118 [00:00<00:00, 124.62it/s, v_num=1, train_loss=2.270]
Epoch 0: 84%|████████▍ | 99/118 [00:00<00:00, 124.58it/s, v_num=1, train_loss=2.270]
Epoch 0: 85%|████████▍ | 100/118 [00:00<00:00, 125.33it/s, v_num=1, train_loss=2.270]
Epoch 0: 85%|████████▍ | 100/118 [00:00<00:00, 125.28it/s, v_num=1, train_loss=2.270]
Epoch 0: 86%|████████▌ | 101/118 [00:00<00:00, 125.97it/s, v_num=1, train_loss=2.270]
Epoch 0: 86%|████████▌ | 101/118 [00:00<00:00, 125.92it/s, v_num=1, train_loss=2.270]
Epoch 0: 86%|████████▋ | 102/118 [00:00<00:00, 125.91it/s, v_num=1, train_loss=2.270]
Epoch 0: 86%|████████▋ | 102/118 [00:00<00:00, 125.86it/s, v_num=1, train_loss=2.270]
Epoch 0: 87%|████████▋ | 103/118 [00:00<00:00, 125.02it/s, v_num=1, train_loss=2.270]
Epoch 0: 87%|████████▋ | 103/118 [00:00<00:00, 124.88it/s, v_num=1, train_loss=2.260]
Epoch 0: 88%|████████▊ | 104/118 [00:00<00:00, 125.74it/s, v_num=1, train_loss=2.260]
Epoch 0: 88%|████████▊ | 104/118 [00:00<00:00, 125.59it/s, v_num=1, train_loss=2.260]
Epoch 0: 89%|████████▉ | 105/118 [00:00<00:00, 126.46it/s, v_num=1, train_loss=2.260]
Epoch 0: 89%|████████▉ | 105/118 [00:00<00:00, 126.30it/s, v_num=1, train_loss=2.260]
Epoch 0: 90%|████████▉ | 106/118 [00:00<00:00, 127.19it/s, v_num=1, train_loss=2.260]
Epoch 0: 90%|████████▉ | 106/118 [00:00<00:00, 127.03it/s, v_num=1, train_loss=2.260]
Epoch 0: 91%|█████████ | 107/118 [00:00<00:00, 127.92it/s, v_num=1, train_loss=2.260]
Epoch 0: 91%|█████████ | 107/118 [00:00<00:00, 127.76it/s, v_num=1, train_loss=2.270]
Epoch 0: 92%|█████████▏| 108/118 [00:00<00:00, 128.65it/s, v_num=1, train_loss=2.270]
Epoch 0: 92%|█████████▏| 108/118 [00:00<00:00, 128.49it/s, v_num=1, train_loss=2.260]
Epoch 0: 92%|█████████▏| 109/118 [00:00<00:00, 129.38it/s, v_num=1, train_loss=2.260]
Epoch 0: 92%|█████████▏| 109/118 [00:00<00:00, 129.22it/s, v_num=1, train_loss=2.260]
Epoch 0: 93%|█████████▎| 110/118 [00:00<00:00, 130.08it/s, v_num=1, train_loss=2.260]
Epoch 0: 93%|█████████▎| 110/118 [00:00<00:00, 129.92it/s, v_num=1, train_loss=2.250]
Epoch 0: 94%|█████████▍| 111/118 [00:00<00:00, 125.59it/s, v_num=1, train_loss=2.250]
Epoch 0: 94%|█████████▍| 111/118 [00:00<00:00, 125.48it/s, v_num=1, train_loss=2.260]
Epoch 0: 95%|█████████▍| 112/118 [00:00<00:00, 126.09it/s, v_num=1, train_loss=2.260]
Epoch 0: 95%|█████████▍| 112/118 [00:00<00:00, 125.97it/s, v_num=1, train_loss=2.260]
Epoch 0: 96%|█████████▌| 113/118 [00:00<00:00, 126.76it/s, v_num=1, train_loss=2.260]
Epoch 0: 96%|█████████▌| 113/118 [00:00<00:00, 126.63it/s, v_num=1, train_loss=2.260]
Epoch 0: 97%|█████████▋| 114/118 [00:00<00:00, 127.41it/s, v_num=1, train_loss=2.260]
Epoch 0: 97%|█████████▋| 114/118 [00:00<00:00, 127.28it/s, v_num=1, train_loss=2.250]
Epoch 0: 97%|█████████▋| 115/118 [00:00<00:00, 128.00it/s, v_num=1, train_loss=2.250]
Epoch 0: 97%|█████████▋| 115/118 [00:00<00:00, 127.88it/s, v_num=1, train_loss=2.250]
Epoch 0: 98%|█████████▊| 116/118 [00:00<00:00, 128.60it/s, v_num=1, train_loss=2.250]
Epoch 0: 98%|█████████▊| 116/118 [00:00<00:00, 128.46it/s, v_num=1, train_loss=2.250]
Epoch 0: 99%|█████████▉| 117/118 [00:00<00:00, 129.24it/s, v_num=1, train_loss=2.250]
Epoch 0: 99%|█████████▉| 117/118 [00:00<00:00, 129.09it/s, v_num=1, train_loss=2.250]
Epoch 0: 100%|██████████| 118/118 [00:00<00:00, 129.92it/s, v_num=1, train_loss=2.250]
Epoch 0: 100%|██████████| 118/118 [00:00<00:00, 129.91it/s, v_num=1, train_loss=2.240]
Validation: | | 0/? [00:00<?, ?it/s]
Validation: 0%| | 0/5 [00:00<?, ?it/s]
Validation DataLoader 0: 0%| | 0/5 [00:00<?, ?it/s]
Validation DataLoader 0: 20%|██ | 1/5 [00:00<00:00, 256.25it/s]
Validation DataLoader 0: 40%|████ | 2/5 [00:00<00:00, 251.68it/s]
Validation DataLoader 0: 60%|██████ | 3/5 [00:00<00:00, 239.57it/s]
Validation DataLoader 0: 80%|████████ | 4/5 [00:00<00:00, 235.43it/s]
Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 53.85it/s]
Epoch 0: 100%|██████████| 118/118 [00:01<00:00, 92.94it/s, v_num=1, train_loss=2.240, Acc=44.80]
Epoch 0: 100%|██████████| 118/118 [00:01<00:00, 92.82it/s, v_num=1, train_loss=2.240, Acc=44.80]
Epoch 0: 0%| | 0/118 [00:00<?, ?it/s, v_num=1, train_loss=2.240, Acc=44.80]
Epoch 1: 0%| | 0/118 [00:00<?, ?it/s, v_num=1, train_loss=2.240, Acc=44.80]
Epoch 1: 1%| | 1/118 [00:00<00:31, 3.77it/s, v_num=1, train_loss=2.240, Acc=44.80]
Epoch 1: 1%| | 1/118 [00:00<00:31, 3.76it/s, v_num=1, train_loss=2.240, Acc=44.80]
Epoch 1: 2%|▏ | 2/118 [00:00<00:15, 7.36it/s, v_num=1, train_loss=2.240, Acc=44.80]
Epoch 1: 2%|▏ | 2/118 [00:00<00:15, 7.35it/s, v_num=1, train_loss=2.240, Acc=44.80]
Epoch 1: 3%|▎ | 3/118 [00:00<00:10, 10.75it/s, v_num=1, train_loss=2.240, Acc=44.80]
Epoch 1: 3%|▎ | 3/118 [00:00<00:10, 10.74it/s, v_num=1, train_loss=2.230, Acc=44.80]
Epoch 1: 3%|▎ | 4/118 [00:00<00:08, 13.93it/s, v_num=1, train_loss=2.230, Acc=44.80]
Epoch 1: 3%|▎ | 4/118 [00:00<00:08, 13.92it/s, v_num=1, train_loss=2.240, Acc=44.80]
Epoch 1: 4%|▍ | 5/118 [00:00<00:06, 16.90it/s, v_num=1, train_loss=2.240, Acc=44.80]
Epoch 1: 4%|▍ | 5/118 [00:00<00:06, 16.89it/s, v_num=1, train_loss=2.240, Acc=44.80]
Epoch 1: 5%|▌ | 6/118 [00:00<00:05, 20.02it/s, v_num=1, train_loss=2.240, Acc=44.80]
Epoch 1: 5%|▌ | 6/118 [00:00<00:05, 20.01it/s, v_num=1, train_loss=2.240, Acc=44.80]
Epoch 1: 6%|▌ | 7/118 [00:00<00:04, 23.05it/s, v_num=1, train_loss=2.240, Acc=44.80]
Epoch 1: 6%|▌ | 7/118 [00:00<00:04, 23.03it/s, v_num=1, train_loss=2.220, Acc=44.80]
Epoch 1: 7%|▋ | 8/118 [00:00<00:04, 24.10it/s, v_num=1, train_loss=2.220, Acc=44.80]
Epoch 1: 7%|▋ | 8/118 [00:00<00:04, 24.07it/s, v_num=1, train_loss=2.220, Acc=44.80]
Epoch 1: 8%|▊ | 9/118 [00:00<00:04, 26.75it/s, v_num=1, train_loss=2.220, Acc=44.80]
Epoch 1: 8%|▊ | 9/118 [00:00<00:04, 26.69it/s, v_num=1, train_loss=2.230, Acc=44.80]
Epoch 1: 8%|▊ | 10/118 [00:00<00:03, 29.28it/s, v_num=1, train_loss=2.230, Acc=44.80]
Epoch 1: 8%|▊ | 10/118 [00:00<00:03, 29.21it/s, v_num=1, train_loss=2.220, Acc=44.80]
Epoch 1: 9%|▉ | 11/118 [00:00<00:03, 28.89it/s, v_num=1, train_loss=2.220, Acc=44.80]
Epoch 1: 9%|▉ | 11/118 [00:00<00:03, 28.83it/s, v_num=1, train_loss=2.210, Acc=44.80]
Epoch 1: 10%|█ | 12/118 [00:00<00:03, 30.27it/s, v_num=1, train_loss=2.210, Acc=44.80]
Epoch 1: 10%|█ | 12/118 [00:00<00:03, 30.21it/s, v_num=1, train_loss=2.200, Acc=44.80]
Epoch 1: 11%|█ | 13/118 [00:00<00:03, 32.42it/s, v_num=1, train_loss=2.200, Acc=44.80]
Epoch 1: 11%|█ | 13/118 [00:00<00:03, 32.34it/s, v_num=1, train_loss=2.200, Acc=44.80]
Epoch 1: 12%|█▏ | 14/118 [00:00<00:03, 34.63it/s, v_num=1, train_loss=2.200, Acc=44.80]
Epoch 1: 12%|█▏ | 14/118 [00:00<00:03, 34.54it/s, v_num=1, train_loss=2.200, Acc=44.80]
Epoch 1: 13%|█▎ | 15/118 [00:00<00:02, 36.81it/s, v_num=1, train_loss=2.200, Acc=44.80]
Epoch 1: 13%|█▎ | 15/118 [00:00<00:02, 36.72it/s, v_num=1, train_loss=2.200, Acc=44.80]
Epoch 1: 14%|█▎ | 16/118 [00:00<00:02, 38.95it/s, v_num=1, train_loss=2.200, Acc=44.80]
Epoch 1: 14%|█▎ | 16/118 [00:00<00:02, 38.85it/s, v_num=1, train_loss=2.180, Acc=44.80]
Epoch 1: 14%|█▍ | 17/118 [00:00<00:02, 41.06it/s, v_num=1, train_loss=2.180, Acc=44.80]
Epoch 1: 14%|█▍ | 17/118 [00:00<00:02, 40.96it/s, v_num=1, train_loss=2.200, Acc=44.80]
Epoch 1: 15%|█▌ | 18/118 [00:00<00:02, 43.15it/s, v_num=1, train_loss=2.200, Acc=44.80]
Epoch 1: 15%|█▌ | 18/118 [00:00<00:02, 43.04it/s, v_num=1, train_loss=2.180, Acc=44.80]
Epoch 1: 16%|█▌ | 19/118 [00:00<00:02, 42.62it/s, v_num=1, train_loss=2.180, Acc=44.80]
Epoch 1: 16%|█▌ | 19/118 [00:00<00:02, 42.59it/s, v_num=1, train_loss=2.190, Acc=44.80]
Epoch 1: 17%|█▋ | 20/118 [00:00<00:02, 44.38it/s, v_num=1, train_loss=2.190, Acc=44.80]
Epoch 1: 17%|█▋ | 20/118 [00:00<00:02, 44.35it/s, v_num=1, train_loss=2.170, Acc=44.80]
Epoch 1: 18%|█▊ | 21/118 [00:00<00:02, 46.23it/s, v_num=1, train_loss=2.170, Acc=44.80]
Epoch 1: 18%|█▊ | 21/118 [00:00<00:02, 46.20it/s, v_num=1, train_loss=2.170, Acc=44.80]
Epoch 1: 19%|█▊ | 22/118 [00:00<00:01, 48.04it/s, v_num=1, train_loss=2.170, Acc=44.80]
Epoch 1: 19%|█▊ | 22/118 [00:00<00:01, 48.01it/s, v_num=1, train_loss=2.180, Acc=44.80]
Epoch 1: 19%|█▉ | 23/118 [00:00<00:01, 49.82it/s, v_num=1, train_loss=2.180, Acc=44.80]
Epoch 1: 19%|█▉ | 23/118 [00:00<00:01, 49.79it/s, v_num=1, train_loss=2.160, Acc=44.80]
Epoch 1: 20%|██ | 24/118 [00:00<00:01, 51.59it/s, v_num=1, train_loss=2.160, Acc=44.80]
Epoch 1: 20%|██ | 24/118 [00:00<00:01, 51.56it/s, v_num=1, train_loss=2.140, Acc=44.80]
Epoch 1: 21%|██ | 25/118 [00:00<00:01, 53.32it/s, v_num=1, train_loss=2.140, Acc=44.80]
Epoch 1: 21%|██ | 25/118 [00:00<00:01, 53.30it/s, v_num=1, train_loss=2.130, Acc=44.80]
Epoch 1: 22%|██▏ | 26/118 [00:00<00:01, 54.77it/s, v_num=1, train_loss=2.130, Acc=44.80]
Epoch 1: 22%|██▏ | 26/118 [00:00<00:01, 54.74it/s, v_num=1, train_loss=2.110, Acc=44.80]
Epoch 1: 23%|██▎ | 27/118 [00:00<00:01, 55.58it/s, v_num=1, train_loss=2.110, Acc=44.80]
Epoch 1: 23%|██▎ | 27/118 [00:00<00:01, 55.55it/s, v_num=1, train_loss=2.140, Acc=44.80]
Epoch 1: 24%|██▎ | 28/118 [00:00<00:01, 56.98it/s, v_num=1, train_loss=2.140, Acc=44.80]
Epoch 1: 24%|██▎ | 28/118 [00:00<00:01, 56.95it/s, v_num=1, train_loss=2.120, Acc=44.80]
Epoch 1: 25%|██▍ | 29/118 [00:00<00:01, 55.01it/s, v_num=1, train_loss=2.120, Acc=44.80]
Epoch 1: 25%|██▍ | 29/118 [00:00<00:01, 54.94it/s, v_num=1, train_loss=2.110, Acc=44.80]
Epoch 1: 25%|██▌ | 30/118 [00:00<00:01, 56.42it/s, v_num=1, train_loss=2.110, Acc=44.80]
Epoch 1: 25%|██▌ | 30/118 [00:00<00:01, 56.33it/s, v_num=1, train_loss=2.080, Acc=44.80]
Epoch 1: 26%|██▋ | 31/118 [00:00<00:01, 57.79it/s, v_num=1, train_loss=2.080, Acc=44.80]
Epoch 1: 26%|██▋ | 31/118 [00:00<00:01, 57.70it/s, v_num=1, train_loss=2.090, Acc=44.80]
Epoch 1: 27%|██▋ | 32/118 [00:00<00:01, 59.12it/s, v_num=1, train_loss=2.090, Acc=44.80]
Epoch 1: 27%|██▋ | 32/118 [00:00<00:01, 59.02it/s, v_num=1, train_loss=2.080, Acc=44.80]
Epoch 1: 28%|██▊ | 33/118 [00:00<00:01, 60.32it/s, v_num=1, train_loss=2.080, Acc=44.80]
Epoch 1: 28%|██▊ | 33/118 [00:00<00:01, 60.23it/s, v_num=1, train_loss=2.080, Acc=44.80]
Epoch 1: 29%|██▉ | 34/118 [00:00<00:01, 61.58it/s, v_num=1, train_loss=2.080, Acc=44.80]
Epoch 1: 29%|██▉ | 34/118 [00:00<00:01, 61.49it/s, v_num=1, train_loss=2.030, Acc=44.80]
Epoch 1: 30%|██▉ | 35/118 [00:00<00:01, 62.81it/s, v_num=1, train_loss=2.030, Acc=44.80]
Epoch 1: 30%|██▉ | 35/118 [00:00<00:01, 62.72it/s, v_num=1, train_loss=2.070, Acc=44.80]
Epoch 1: 31%|███ | 36/118 [00:00<00:01, 64.01it/s, v_num=1, train_loss=2.070, Acc=44.80]
Epoch 1: 31%|███ | 36/118 [00:00<00:01, 63.91it/s, v_num=1, train_loss=2.050, Acc=44.80]
Epoch 1: 31%|███▏ | 37/118 [00:00<00:01, 64.85it/s, v_num=1, train_loss=2.050, Acc=44.80]
Epoch 1: 31%|███▏ | 37/118 [00:00<00:01, 64.76it/s, v_num=1, train_loss=2.050, Acc=44.80]
Epoch 1: 32%|███▏ | 38/118 [00:00<00:01, 66.02it/s, v_num=1, train_loss=2.050, Acc=44.80]
Epoch 1: 32%|███▏ | 38/118 [00:00<00:01, 65.92it/s, v_num=1, train_loss=2.030, Acc=44.80]
Epoch 1: 33%|███▎ | 39/118 [00:00<00:01, 67.15it/s, v_num=1, train_loss=2.030, Acc=44.80]
Epoch 1: 33%|███▎ | 39/118 [00:00<00:01, 67.05it/s, v_num=1, train_loss=2.100, Acc=44.80]
Epoch 1: 34%|███▍ | 40/118 [00:00<00:01, 68.28it/s, v_num=1, train_loss=2.100, Acc=44.80]
Epoch 1: 34%|███▍ | 40/118 [00:00<00:01, 68.18it/s, v_num=1, train_loss=2.100, Acc=44.80]
Epoch 1: 35%|███▍ | 41/118 [00:00<00:01, 69.38it/s, v_num=1, train_loss=2.100, Acc=44.80]
Epoch 1: 35%|███▍ | 41/118 [00:00<00:01, 69.29it/s, v_num=1, train_loss=2.020, Acc=44.80]
Epoch 1: 36%|███▌ | 42/118 [00:00<00:01, 70.46it/s, v_num=1, train_loss=2.020, Acc=44.80]
Epoch 1: 36%|███▌ | 42/118 [00:00<00:01, 70.37it/s, v_num=1, train_loss=2.000, Acc=44.80]
Epoch 1: 36%|███▋ | 43/118 [00:00<00:01, 71.39it/s, v_num=1, train_loss=2.000, Acc=44.80]
Epoch 1: 36%|███▋ | 43/118 [00:00<00:01, 71.36it/s, v_num=1, train_loss=1.950, Acc=44.80]
Epoch 1: 37%|███▋ | 44/118 [00:00<00:01, 72.38it/s, v_num=1, train_loss=1.950, Acc=44.80]
Epoch 1: 37%|███▋ | 44/118 [00:00<00:01, 72.35it/s, v_num=1, train_loss=1.950, Acc=44.80]
Epoch 1: 38%|███▊ | 45/118 [00:00<00:01, 72.97it/s, v_num=1, train_loss=1.950, Acc=44.80]
Epoch 1: 38%|███▊ | 45/118 [00:00<00:01, 72.94it/s, v_num=1, train_loss=1.930, Acc=44.80]
Epoch 1: 39%|███▉ | 46/118 [00:00<00:01, 69.30it/s, v_num=1, train_loss=1.930, Acc=44.80]
Epoch 1: 39%|███▉ | 46/118 [00:00<00:01, 69.27it/s, v_num=1, train_loss=1.910, Acc=44.80]
Epoch 1: 40%|███▉ | 47/118 [00:00<00:01, 70.27it/s, v_num=1, train_loss=1.910, Acc=44.80]
Epoch 1: 40%|███▉ | 47/118 [00:00<00:01, 70.23it/s, v_num=1, train_loss=1.990, Acc=44.80]
Epoch 1: 41%|████ | 48/118 [00:00<00:00, 71.08it/s, v_num=1, train_loss=1.990, Acc=44.80]
Epoch 1: 41%|████ | 48/118 [00:00<00:00, 71.03it/s, v_num=1, train_loss=2.030, Acc=44.80]
Epoch 1: 42%|████▏ | 49/118 [00:00<00:00, 72.16it/s, v_num=1, train_loss=2.030, Acc=44.80]
Epoch 1: 42%|████▏ | 49/118 [00:00<00:00, 72.13it/s, v_num=1, train_loss=1.990, Acc=44.80]
Epoch 1: 42%|████▏ | 50/118 [00:00<00:00, 73.24it/s, v_num=1, train_loss=1.990, Acc=44.80]
Epoch 1: 42%|████▏ | 50/118 [00:00<00:00, 73.21it/s, v_num=1, train_loss=1.850, Acc=44.80]
Epoch 1: 43%|████▎ | 51/118 [00:00<00:00, 74.32it/s, v_num=1, train_loss=1.850, Acc=44.80]
Epoch 1: 43%|████▎ | 51/118 [00:00<00:00, 74.29it/s, v_num=1, train_loss=1.910, Acc=44.80]
Epoch 1: 44%|████▍ | 52/118 [00:00<00:00, 75.38it/s, v_num=1, train_loss=1.910, Acc=44.80]
Epoch 1: 44%|████▍ | 52/118 [00:00<00:00, 75.35it/s, v_num=1, train_loss=1.940, Acc=44.80]
Epoch 1: 45%|████▍ | 53/118 [00:00<00:00, 76.39it/s, v_num=1, train_loss=1.940, Acc=44.80]
Epoch 1: 45%|████▍ | 53/118 [00:00<00:00, 76.37it/s, v_num=1, train_loss=1.900, Acc=44.80]
Epoch 1: 46%|████▌ | 54/118 [00:00<00:00, 76.96it/s, v_num=1, train_loss=1.900, Acc=44.80]
Epoch 1: 46%|████▌ | 54/118 [00:00<00:00, 76.93it/s, v_num=1, train_loss=2.050, Acc=44.80]
Epoch 1: 47%|████▋ | 55/118 [00:00<00:00, 77.94it/s, v_num=1, train_loss=2.050, Acc=44.80]
Epoch 1: 47%|████▋ | 55/118 [00:00<00:00, 77.91it/s, v_num=1, train_loss=2.070, Acc=44.80]
Epoch 1: 47%|████▋ | 56/118 [00:00<00:00, 78.63it/s, v_num=1, train_loss=2.070, Acc=44.80]
Epoch 1: 47%|████▋ | 56/118 [00:00<00:00, 78.59it/s, v_num=1, train_loss=1.880, Acc=44.80]
Epoch 1: 48%|████▊ | 57/118 [00:00<00:00, 79.42it/s, v_num=1, train_loss=1.880, Acc=44.80]
Epoch 1: 48%|████▊ | 57/118 [00:00<00:00, 79.39it/s, v_num=1, train_loss=1.840, Acc=44.80]
Epoch 1: 49%|████▉ | 58/118 [00:00<00:00, 80.23it/s, v_num=1, train_loss=1.840, Acc=44.80]
Epoch 1: 49%|████▉ | 58/118 [00:00<00:00, 80.20it/s, v_num=1, train_loss=1.810, Acc=44.80]
Epoch 1: 50%|█████ | 59/118 [00:00<00:00, 81.14it/s, v_num=1, train_loss=1.810, Acc=44.80]
Epoch 1: 50%|█████ | 59/118 [00:00<00:00, 81.05it/s, v_num=1, train_loss=1.890, Acc=44.80]
Epoch 1: 51%|█████ | 60/118 [00:00<00:00, 81.96it/s, v_num=1, train_loss=1.890, Acc=44.80]
Epoch 1: 51%|█████ | 60/118 [00:00<00:00, 81.88it/s, v_num=1, train_loss=1.910, Acc=44.80]
Epoch 1: 52%|█████▏ | 61/118 [00:00<00:00, 82.82it/s, v_num=1, train_loss=1.910, Acc=44.80]
Epoch 1: 52%|█████▏ | 61/118 [00:00<00:00, 82.78it/s, v_num=1, train_loss=1.950, Acc=44.80]
Epoch 1: 53%|█████▎ | 62/118 [00:00<00:00, 83.58it/s, v_num=1, train_loss=1.950, Acc=44.80]
Epoch 1: 53%|█████▎ | 62/118 [00:00<00:00, 83.51it/s, v_num=1, train_loss=1.850, Acc=44.80]
Epoch 1: 53%|█████▎ | 63/118 [00:00<00:00, 84.58it/s, v_num=1, train_loss=1.850, Acc=44.80]
Epoch 1: 53%|█████▎ | 63/118 [00:00<00:00, 84.47it/s, v_num=1, train_loss=1.780, Acc=44.80]
Epoch 1: 54%|█████▍ | 64/118 [00:00<00:00, 84.24it/s, v_num=1, train_loss=1.780, Acc=44.80]
Epoch 1: 54%|█████▍ | 64/118 [00:00<00:00, 84.15it/s, v_num=1, train_loss=1.800, Acc=44.80]
Epoch 1: 55%|█████▌ | 65/118 [00:00<00:00, 84.96it/s, v_num=1, train_loss=1.800, Acc=44.80]
Epoch 1: 55%|█████▌ | 65/118 [00:00<00:00, 84.90it/s, v_num=1, train_loss=1.760, Acc=44.80]
Epoch 1: 56%|█████▌ | 66/118 [00:00<00:00, 84.05it/s, v_num=1, train_loss=1.760, Acc=44.80]
Epoch 1: 56%|█████▌ | 66/118 [00:00<00:00, 84.00it/s, v_num=1, train_loss=1.760, Acc=44.80]
Epoch 1: 57%|█████▋ | 67/118 [00:00<00:00, 84.76it/s, v_num=1, train_loss=1.760, Acc=44.80]
Epoch 1: 57%|█████▋ | 67/118 [00:00<00:00, 84.70it/s, v_num=1, train_loss=1.760, Acc=44.80]
Epoch 1: 58%|█████▊ | 68/118 [00:00<00:00, 85.20it/s, v_num=1, train_loss=1.760, Acc=44.80]
Epoch 1: 58%|█████▊ | 68/118 [00:00<00:00, 85.17it/s, v_num=1, train_loss=1.790, Acc=44.80]
Epoch 1: 58%|█████▊ | 69/118 [00:00<00:00, 83.61it/s, v_num=1, train_loss=1.790, Acc=44.80]
Epoch 1: 58%|█████▊ | 69/118 [00:00<00:00, 83.54it/s, v_num=1, train_loss=1.860, Acc=44.80]
Epoch 1: 59%|█████▉ | 70/118 [00:00<00:00, 84.31it/s, v_num=1, train_loss=1.860, Acc=44.80]
Epoch 1: 59%|█████▉ | 70/118 [00:00<00:00, 84.23it/s, v_num=1, train_loss=1.860, Acc=44.80]
Epoch 1: 60%|██████ | 71/118 [00:00<00:00, 84.97it/s, v_num=1, train_loss=1.860, Acc=44.80]
Epoch 1: 60%|██████ | 71/118 [00:00<00:00, 84.89it/s, v_num=1, train_loss=1.810, Acc=44.80]
Epoch 1: 61%|██████ | 72/118 [00:00<00:00, 85.73it/s, v_num=1, train_loss=1.810, Acc=44.80]
Epoch 1: 61%|██████ | 72/118 [00:00<00:00, 85.63it/s, v_num=1, train_loss=1.750, Acc=44.80]
Epoch 1: 62%|██████▏ | 73/118 [00:00<00:00, 86.58it/s, v_num=1, train_loss=1.750, Acc=44.80]
Epoch 1: 62%|██████▏ | 73/118 [00:00<00:00, 86.48it/s, v_num=1, train_loss=1.710, Acc=44.80]
Epoch 1: 63%|██████▎ | 74/118 [00:00<00:00, 87.15it/s, v_num=1, train_loss=1.710, Acc=44.80]
Epoch 1: 63%|██████▎ | 74/118 [00:00<00:00, 87.06it/s, v_num=1, train_loss=1.690, Acc=44.80]
Epoch 1: 64%|██████▎ | 75/118 [00:00<00:00, 87.86it/s, v_num=1, train_loss=1.690, Acc=44.80]
Epoch 1: 64%|██████▎ | 75/118 [00:00<00:00, 87.83it/s, v_num=1, train_loss=1.690, Acc=44.80]
Epoch 1: 64%|██████▍ | 76/118 [00:00<00:00, 88.35it/s, v_num=1, train_loss=1.690, Acc=44.80]
Epoch 1: 64%|██████▍ | 76/118 [00:00<00:00, 88.32it/s, v_num=1, train_loss=1.690, Acc=44.80]
Epoch 1: 65%|██████▌ | 77/118 [00:00<00:00, 88.64it/s, v_num=1, train_loss=1.690, Acc=44.80]
Epoch 1: 65%|██████▌ | 77/118 [00:00<00:00, 88.61it/s, v_num=1, train_loss=1.840, Acc=44.80]
Epoch 1: 66%|██████▌ | 78/118 [00:00<00:00, 89.31it/s, v_num=1, train_loss=1.840, Acc=44.80]
Epoch 1: 66%|██████▌ | 78/118 [00:00<00:00, 89.29it/s, v_num=1, train_loss=1.940, Acc=44.80]
Epoch 1: 67%|██████▋ | 79/118 [00:00<00:00, 89.87it/s, v_num=1, train_loss=1.940, Acc=44.80]
Epoch 1: 67%|██████▋ | 79/118 [00:00<00:00, 89.84it/s, v_num=1, train_loss=1.800, Acc=44.80]
Epoch 1: 68%|██████▊ | 80/118 [00:00<00:00, 90.43it/s, v_num=1, train_loss=1.800, Acc=44.80]
Epoch 1: 68%|██████▊ | 80/118 [00:00<00:00, 90.40it/s, v_num=1, train_loss=1.640, Acc=44.80]
Epoch 1: 69%|██████▊ | 81/118 [00:00<00:00, 90.99it/s, v_num=1, train_loss=1.640, Acc=44.80]
Epoch 1: 69%|██████▊ | 81/118 [00:00<00:00, 90.96it/s, v_num=1, train_loss=1.640, Acc=44.80]
Epoch 1: 69%|██████▉ | 82/118 [00:00<00:00, 91.44it/s, v_num=1, train_loss=1.640, Acc=44.80]
Epoch 1: 69%|██████▉ | 82/118 [00:00<00:00, 91.41it/s, v_num=1, train_loss=1.640, Acc=44.80]
Epoch 1: 70%|███████ | 83/118 [00:00<00:00, 91.92it/s, v_num=1, train_loss=1.640, Acc=44.80]
Epoch 1: 70%|███████ | 83/118 [00:00<00:00, 91.90it/s, v_num=1, train_loss=1.610, Acc=44.80]
Epoch 1: 71%|███████ | 84/118 [00:00<00:00, 92.43it/s, v_num=1, train_loss=1.610, Acc=44.80]
Epoch 1: 71%|███████ | 84/118 [00:00<00:00, 92.41it/s, v_num=1, train_loss=1.780, Acc=44.80]
Epoch 1: 72%|███████▏ | 85/118 [00:00<00:00, 92.93it/s, v_num=1, train_loss=1.780, Acc=44.80]
Epoch 1: 72%|███████▏ | 85/118 [00:00<00:00, 92.90it/s, v_num=1, train_loss=1.850, Acc=44.80]
Epoch 1: 73%|███████▎ | 86/118 [00:00<00:00, 90.44it/s, v_num=1, train_loss=1.850, Acc=44.80]
Epoch 1: 73%|███████▎ | 86/118 [00:00<00:00, 90.39it/s, v_num=1, train_loss=1.870, Acc=44.80]
Epoch 1: 74%|███████▎ | 87/118 [00:00<00:00, 91.20it/s, v_num=1, train_loss=1.870, Acc=44.80]
Epoch 1: 74%|███████▎ | 87/118 [00:00<00:00, 91.11it/s, v_num=1, train_loss=1.890, Acc=44.80]
Epoch 1: 75%|███████▍ | 88/118 [00:00<00:00, 91.85it/s, v_num=1, train_loss=1.890, Acc=44.80]
Epoch 1: 75%|███████▍ | 88/118 [00:00<00:00, 91.75it/s, v_num=1, train_loss=1.650, Acc=44.80]
Epoch 1: 75%|███████▌ | 89/118 [00:00<00:00, 92.57it/s, v_num=1, train_loss=1.650, Acc=44.80]
Epoch 1: 75%|███████▌ | 89/118 [00:00<00:00, 92.48it/s, v_num=1, train_loss=1.610, Acc=44.80]
Epoch 1: 76%|███████▋ | 90/118 [00:00<00:00, 93.30it/s, v_num=1, train_loss=1.610, Acc=44.80]
Epoch 1: 76%|███████▋ | 90/118 [00:00<00:00, 93.20it/s, v_num=1, train_loss=1.590, Acc=44.80]
Epoch 1: 77%|███████▋ | 91/118 [00:00<00:00, 94.01it/s, v_num=1, train_loss=1.590, Acc=44.80]
Epoch 1: 77%|███████▋ | 91/118 [00:00<00:00, 93.91it/s, v_num=1, train_loss=1.440, Acc=44.80]
Epoch 1: 78%|███████▊ | 92/118 [00:00<00:00, 94.71it/s, v_num=1, train_loss=1.440, Acc=44.80]
Epoch 1: 78%|███████▊ | 92/118 [00:00<00:00, 94.61it/s, v_num=1, train_loss=1.450, Acc=44.80]
Epoch 1: 79%|███████▉ | 93/118 [00:00<00:00, 95.42it/s, v_num=1, train_loss=1.450, Acc=44.80]
Epoch 1: 79%|███████▉ | 93/118 [00:00<00:00, 95.32it/s, v_num=1, train_loss=1.530, Acc=44.80]
Epoch 1: 80%|███████▉ | 94/118 [00:00<00:00, 94.82it/s, v_num=1, train_loss=1.530, Acc=44.80]
Epoch 1: 80%|███████▉ | 94/118 [00:00<00:00, 94.79it/s, v_num=1, train_loss=1.720, Acc=44.80]
Epoch 1: 81%|████████ | 95/118 [00:00<00:00, 95.26it/s, v_num=1, train_loss=1.720, Acc=44.80]
Epoch 1: 81%|████████ | 95/118 [00:00<00:00, 95.24it/s, v_num=1, train_loss=1.880, Acc=44.80]
Epoch 1: 81%|████████▏ | 96/118 [00:01<00:00, 95.53it/s, v_num=1, train_loss=1.880, Acc=44.80]
Epoch 1: 81%|████████▏ | 96/118 [00:01<00:00, 95.50it/s, v_num=1, train_loss=1.770, Acc=44.80]
Epoch 1: 82%|████████▏ | 97/118 [00:01<00:00, 96.12it/s, v_num=1, train_loss=1.770, Acc=44.80]
Epoch 1: 82%|████████▏ | 97/118 [00:01<00:00, 96.09it/s, v_num=1, train_loss=1.630, Acc=44.80]
Epoch 1: 83%|████████▎ | 98/118 [00:01<00:00, 96.77it/s, v_num=1, train_loss=1.630, Acc=44.80]
Epoch 1: 83%|████████▎ | 98/118 [00:01<00:00, 96.74it/s, v_num=1, train_loss=1.630, Acc=44.80]
Epoch 1: 84%|████████▍ | 99/118 [00:01<00:00, 97.40it/s, v_num=1, train_loss=1.630, Acc=44.80]
Epoch 1: 84%|████████▍ | 99/118 [00:01<00:00, 97.37it/s, v_num=1, train_loss=1.580, Acc=44.80]
Epoch 1: 85%|████████▍ | 100/118 [00:01<00:00, 98.03it/s, v_num=1, train_loss=1.580, Acc=44.80]
Epoch 1: 85%|████████▍ | 100/118 [00:01<00:00, 98.00it/s, v_num=1, train_loss=1.580, Acc=44.80]
Epoch 1: 86%|████████▌ | 101/118 [00:01<00:00, 98.66it/s, v_num=1, train_loss=1.580, Acc=44.80]
Epoch 1: 86%|████████▌ | 101/118 [00:01<00:00, 98.64it/s, v_num=1, train_loss=1.590, Acc=44.80]
Epoch 1: 86%|████████▋ | 102/118 [00:01<00:00, 97.83it/s, v_num=1, train_loss=1.590, Acc=44.80]
Epoch 1: 86%|████████▋ | 102/118 [00:01<00:00, 97.80it/s, v_num=1, train_loss=1.560, Acc=44.80]
Epoch 1: 87%|████████▋ | 103/118 [00:01<00:00, 98.36it/s, v_num=1, train_loss=1.560, Acc=44.80]
Epoch 1: 87%|████████▋ | 103/118 [00:01<00:00, 98.28it/s, v_num=1, train_loss=1.510, Acc=44.80]
Epoch 1: 88%|████████▊ | 104/118 [00:01<00:00, 97.33it/s, v_num=1, train_loss=1.510, Acc=44.80]
Epoch 1: 88%|████████▊ | 104/118 [00:01<00:00, 97.25it/s, v_num=1, train_loss=1.470, Acc=44.80]
Epoch 1: 89%|████████▉ | 105/118 [00:01<00:00, 97.95it/s, v_num=1, train_loss=1.470, Acc=44.80]
Epoch 1: 89%|████████▉ | 105/118 [00:01<00:00, 97.86it/s, v_num=1, train_loss=1.530, Acc=44.80]
Epoch 1: 90%|████████▉ | 106/118 [00:01<00:00, 97.18it/s, v_num=1, train_loss=1.530, Acc=44.80]
Epoch 1: 90%|████████▉ | 106/118 [00:01<00:00, 97.11it/s, v_num=1, train_loss=1.510, Acc=44.80]
Epoch 1: 91%|█████████ | 107/118 [00:01<00:00, 97.71it/s, v_num=1, train_loss=1.510, Acc=44.80]
Epoch 1: 91%|█████████ | 107/118 [00:01<00:00, 97.62it/s, v_num=1, train_loss=1.510, Acc=44.80]
Epoch 1: 92%|█████████▏| 108/118 [00:01<00:00, 98.25it/s, v_num=1, train_loss=1.510, Acc=44.80]
Epoch 1: 92%|█████████▏| 108/118 [00:01<00:00, 98.16it/s, v_num=1, train_loss=1.500, Acc=44.80]
Epoch 1: 92%|█████████▏| 109/118 [00:01<00:00, 98.89it/s, v_num=1, train_loss=1.500, Acc=44.80]
Epoch 1: 92%|█████████▏| 109/118 [00:01<00:00, 98.79it/s, v_num=1, train_loss=1.470, Acc=44.80]
Epoch 1: 93%|█████████▎| 110/118 [00:01<00:00, 99.50it/s, v_num=1, train_loss=1.470, Acc=44.80]
Epoch 1: 93%|█████████▎| 110/118 [00:01<00:00, 99.41it/s, v_num=1, train_loss=1.330, Acc=44.80]
Epoch 1: 94%|█████████▍| 111/118 [00:01<00:00, 100.05it/s, v_num=1, train_loss=1.330, Acc=44.80]
Epoch 1: 94%|█████████▍| 111/118 [00:01<00:00, 99.95it/s, v_num=1, train_loss=1.550, Acc=44.80]
Epoch 1: 95%|█████████▍| 112/118 [00:01<00:00, 100.67it/s, v_num=1, train_loss=1.550, Acc=44.80]
Epoch 1: 95%|█████████▍| 112/118 [00:01<00:00, 100.58it/s, v_num=1, train_loss=1.780, Acc=44.80]
Epoch 1: 96%|█████████▌| 113/118 [00:01<00:00, 101.29it/s, v_num=1, train_loss=1.780, Acc=44.80]
Epoch 1: 96%|█████████▌| 113/118 [00:01<00:00, 101.19it/s, v_num=1, train_loss=1.640, Acc=44.80]
Epoch 1: 97%|█████████▋| 114/118 [00:01<00:00, 101.17it/s, v_num=1, train_loss=1.640, Acc=44.80]
Epoch 1: 97%|█████████▋| 114/118 [00:01<00:00, 101.07it/s, v_num=1, train_loss=1.580, Acc=44.80]
Epoch 1: 97%|█████████▋| 115/118 [00:01<00:00, 101.77it/s, v_num=1, train_loss=1.580, Acc=44.80]
Epoch 1: 97%|█████████▋| 115/118 [00:01<00:00, 101.66it/s, v_num=1, train_loss=1.420, Acc=44.80]
Epoch 1: 98%|█████████▊| 116/118 [00:01<00:00, 102.34it/s, v_num=1, train_loss=1.420, Acc=44.80]
Epoch 1: 98%|█████████▊| 116/118 [00:01<00:00, 102.23it/s, v_num=1, train_loss=1.420, Acc=44.80]
Epoch 1: 99%|█████████▉| 117/118 [00:01<00:00, 102.94it/s, v_num=1, train_loss=1.420, Acc=44.80]
Epoch 1: 99%|█████████▉| 117/118 [00:01<00:00, 102.82it/s, v_num=1, train_loss=1.460, Acc=44.80]
Epoch 1: 100%|██████████| 118/118 [00:01<00:00, 103.54it/s, v_num=1, train_loss=1.460, Acc=44.80]
Epoch 1: 100%|██████████| 118/118 [00:01<00:00, 103.53it/s, v_num=1, train_loss=1.390, Acc=44.80]
Validation: | | 0/? [00:00<?, ?it/s]
Validation: 0%| | 0/5 [00:00<?, ?it/s]
Validation DataLoader 0: 0%| | 0/5 [00:00<?, ?it/s]
Validation DataLoader 0: 20%|██ | 1/5 [00:00<00:00, 282.98it/s]
Validation DataLoader 0: 40%|████ | 2/5 [00:00<00:00, 260.37it/s]
Validation DataLoader 0: 60%|██████ | 3/5 [00:00<00:00, 269.35it/s]
Validation DataLoader 0: 80%|████████ | 4/5 [00:00<00:00, 258.16it/s]
Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 52.96it/s]
Epoch 1: 100%|██████████| 118/118 [00:01<00:00, 79.53it/s, v_num=1, train_loss=1.390, Acc=72.20]
Epoch 1: 100%|██████████| 118/118 [00:01<00:00, 79.48it/s, v_num=1, train_loss=1.390, Acc=72.20]
Epoch 1: 0%| | 0/118 [00:00<?, ?it/s, v_num=1, train_loss=1.390, Acc=72.20]
Epoch 2: 0%| | 0/118 [00:00<?, ?it/s, v_num=1, train_loss=1.390, Acc=72.20]
Epoch 2: 1%| | 1/118 [00:00<00:37, 3.15it/s, v_num=1, train_loss=1.390, Acc=72.20]
Epoch 2: 1%| | 1/118 [00:00<00:37, 3.14it/s, v_num=1, train_loss=1.430, Acc=72.20]
Epoch 2: 2%|▏ | 2/118 [00:00<00:18, 6.19it/s, v_num=1, train_loss=1.430, Acc=72.20]
Epoch 2: 2%|▏ | 2/118 [00:00<00:18, 6.18it/s, v_num=1, train_loss=1.590, Acc=72.20]
Epoch 2: 3%|▎ | 3/118 [00:00<00:12, 9.09it/s, v_num=1, train_loss=1.590, Acc=72.20]
Epoch 2: 3%|▎ | 3/118 [00:00<00:12, 9.06it/s, v_num=1, train_loss=1.580, Acc=72.20]
Epoch 2: 3%|▎ | 4/118 [00:00<00:09, 12.00it/s, v_num=1, train_loss=1.580, Acc=72.20]
Epoch 2: 3%|▎ | 4/118 [00:00<00:09, 11.96it/s, v_num=1, train_loss=1.400, Acc=72.20]
Epoch 2: 4%|▍ | 5/118 [00:00<00:07, 14.85it/s, v_num=1, train_loss=1.400, Acc=72.20]
Epoch 2: 4%|▍ | 5/118 [00:00<00:07, 14.80it/s, v_num=1, train_loss=1.320, Acc=72.20]
Epoch 2: 5%|▌ | 6/118 [00:00<00:06, 17.64it/s, v_num=1, train_loss=1.320, Acc=72.20]
Epoch 2: 5%|▌ | 6/118 [00:00<00:06, 17.58it/s, v_num=1, train_loss=1.290, Acc=72.20]
Epoch 2: 6%|▌ | 7/118 [00:00<00:05, 20.36it/s, v_num=1, train_loss=1.290, Acc=72.20]
Epoch 2: 6%|▌ | 7/118 [00:00<00:05, 20.31it/s, v_num=1, train_loss=1.300, Acc=72.20]
Epoch 2: 7%|▋ | 8/118 [00:00<00:04, 22.91it/s, v_num=1, train_loss=1.300, Acc=72.20]
Epoch 2: 7%|▋ | 8/118 [00:00<00:04, 22.86it/s, v_num=1, train_loss=1.340, Acc=72.20]
Epoch 2: 8%|▊ | 9/118 [00:00<00:04, 25.25it/s, v_num=1, train_loss=1.340, Acc=72.20]
Epoch 2: 8%|▊ | 9/118 [00:00<00:04, 25.19it/s, v_num=1, train_loss=1.310, Acc=72.20]
Epoch 2: 8%|▊ | 10/118 [00:00<00:03, 27.79it/s, v_num=1, train_loss=1.310, Acc=72.20]
Epoch 2: 8%|▊ | 10/118 [00:00<00:03, 27.70it/s, v_num=1, train_loss=1.460, Acc=72.20]
Epoch 2: 9%|▉ | 11/118 [00:00<00:03, 30.23it/s, v_num=1, train_loss=1.460, Acc=72.20]
Epoch 2: 9%|▉ | 11/118 [00:00<00:03, 30.14it/s, v_num=1, train_loss=1.660, Acc=72.20]
Epoch 2: 10%|█ | 12/118 [00:00<00:03, 32.68it/s, v_num=1, train_loss=1.660, Acc=72.20]
Epoch 2: 10%|█ | 12/118 [00:00<00:03, 32.58it/s, v_num=1, train_loss=1.560, Acc=72.20]
Epoch 2: 11%|█ | 13/118 [00:00<00:02, 35.08it/s, v_num=1, train_loss=1.560, Acc=72.20]
Epoch 2: 11%|█ | 13/118 [00:00<00:03, 34.97it/s, v_num=1, train_loss=1.310, Acc=72.20]
Epoch 2: 12%|█▏ | 14/118 [00:00<00:02, 37.42it/s, v_num=1, train_loss=1.310, Acc=72.20]
Epoch 2: 12%|█▏ | 14/118 [00:00<00:02, 37.31it/s, v_num=1, train_loss=1.300, Acc=72.20]
Epoch 2: 13%|█▎ | 15/118 [00:00<00:02, 39.54it/s, v_num=1, train_loss=1.300, Acc=72.20]
Epoch 2: 13%|█▎ | 15/118 [00:00<00:02, 39.47it/s, v_num=1, train_loss=1.280, Acc=72.20]
Epoch 2: 14%|█▎ | 16/118 [00:00<00:02, 41.59it/s, v_num=1, train_loss=1.280, Acc=72.20]
Epoch 2: 14%|█▎ | 16/118 [00:00<00:02, 41.49it/s, v_num=1, train_loss=1.240, Acc=72.20]
Epoch 2: 14%|█▍ | 17/118 [00:00<00:02, 43.22it/s, v_num=1, train_loss=1.240, Acc=72.20]
Epoch 2: 14%|█▍ | 17/118 [00:00<00:02, 43.13it/s, v_num=1, train_loss=1.430, Acc=72.20]
Epoch 2: 15%|█▌ | 18/118 [00:00<00:02, 39.54it/s, v_num=1, train_loss=1.430, Acc=72.20]
Epoch 2: 15%|█▌ | 18/118 [00:00<00:02, 39.50it/s, v_num=1, train_loss=1.470, Acc=72.20]
Epoch 2: 16%|█▌ | 19/118 [00:00<00:02, 41.41it/s, v_num=1, train_loss=1.470, Acc=72.20]
Epoch 2: 16%|█▌ | 19/118 [00:00<00:02, 41.36it/s, v_num=1, train_loss=1.500, Acc=72.20]
Epoch 2: 17%|█▋ | 20/118 [00:00<00:02, 43.23it/s, v_num=1, train_loss=1.500, Acc=72.20]
Epoch 2: 17%|█▋ | 20/118 [00:00<00:02, 43.18it/s, v_num=1, train_loss=1.310, Acc=72.20]
Epoch 2: 18%|█▊ | 21/118 [00:00<00:02, 45.06it/s, v_num=1, train_loss=1.310, Acc=72.20]
Epoch 2: 18%|█▊ | 21/118 [00:00<00:02, 44.97it/s, v_num=1, train_loss=1.200, Acc=72.20]
Epoch 2: 19%|█▊ | 22/118 [00:00<00:02, 46.86it/s, v_num=1, train_loss=1.200, Acc=72.20]
Epoch 2: 19%|█▊ | 22/118 [00:00<00:02, 46.75it/s, v_num=1, train_loss=1.200, Acc=72.20]
Epoch 2: 19%|█▉ | 23/118 [00:00<00:01, 48.52it/s, v_num=1, train_loss=1.200, Acc=72.20]
Epoch 2: 19%|█▉ | 23/118 [00:00<00:01, 48.43it/s, v_num=1, train_loss=1.180, Acc=72.20]
Epoch 2: 20%|██ | 24/118 [00:00<00:01, 50.10it/s, v_num=1, train_loss=1.180, Acc=72.20]
Epoch 2: 20%|██ | 24/118 [00:00<00:01, 49.99it/s, v_num=1, train_loss=1.170, Acc=72.20]
Epoch 2: 21%|██ | 25/118 [00:00<00:01, 51.64it/s, v_num=1, train_loss=1.170, Acc=72.20]
Epoch 2: 21%|██ | 25/118 [00:00<00:01, 51.53it/s, v_num=1, train_loss=1.330, Acc=72.20]
Epoch 2: 22%|██▏ | 26/118 [00:00<00:01, 52.49it/s, v_num=1, train_loss=1.330, Acc=72.20]
Epoch 2: 22%|██▏ | 26/118 [00:00<00:01, 52.46it/s, v_num=1, train_loss=1.500, Acc=72.20]
Epoch 2: 23%|██▎ | 27/118 [00:00<00:01, 53.93it/s, v_num=1, train_loss=1.500, Acc=72.20]
Epoch 2: 23%|██▎ | 27/118 [00:00<00:01, 53.90it/s, v_num=1, train_loss=1.280, Acc=72.20]
Epoch 2: 24%|██▎ | 28/118 [00:00<00:01, 55.29it/s, v_num=1, train_loss=1.280, Acc=72.20]
Epoch 2: 24%|██▎ | 28/118 [00:00<00:01, 55.26it/s, v_num=1, train_loss=1.190, Acc=72.20]
Epoch 2: 25%|██▍ | 29/118 [00:00<00:01, 56.60it/s, v_num=1, train_loss=1.190, Acc=72.20]
Epoch 2: 25%|██▍ | 29/118 [00:00<00:01, 56.57it/s, v_num=1, train_loss=1.200, Acc=72.20]
Epoch 2: 25%|██▌ | 30/118 [00:00<00:01, 57.90it/s, v_num=1, train_loss=1.200, Acc=72.20]
Epoch 2: 25%|██▌ | 30/118 [00:00<00:01, 57.87it/s, v_num=1, train_loss=1.120, Acc=72.20]
Epoch 2: 26%|██▋ | 31/118 [00:00<00:01, 59.16it/s, v_num=1, train_loss=1.120, Acc=72.20]
Epoch 2: 26%|██▋ | 31/118 [00:00<00:01, 59.13it/s, v_num=1, train_loss=1.110, Acc=72.20]
Epoch 2: 27%|██▋ | 32/118 [00:00<00:01, 60.39it/s, v_num=1, train_loss=1.110, Acc=72.20]
Epoch 2: 27%|██▋ | 32/118 [00:00<00:01, 60.35it/s, v_num=1, train_loss=1.120, Acc=72.20]
Epoch 2: 28%|██▊ | 33/118 [00:00<00:01, 61.62it/s, v_num=1, train_loss=1.120, Acc=72.20]
Epoch 2: 28%|██▊ | 33/118 [00:00<00:01, 61.58it/s, v_num=1, train_loss=1.250, Acc=72.20]
Epoch 2: 29%|██▉ | 34/118 [00:00<00:01, 62.23it/s, v_num=1, train_loss=1.250, Acc=72.20]
Epoch 2: 29%|██▉ | 34/118 [00:00<00:01, 62.19it/s, v_num=1, train_loss=1.520, Acc=72.20]
Epoch 2: 30%|██▉ | 35/118 [00:00<00:01, 63.38it/s, v_num=1, train_loss=1.520, Acc=72.20]
Epoch 2: 30%|██▉ | 35/118 [00:00<00:01, 63.35it/s, v_num=1, train_loss=1.310, Acc=72.20]
Epoch 2: 31%|███ | 36/118 [00:00<00:01, 64.53it/s, v_num=1, train_loss=1.310, Acc=72.20]
Epoch 2: 31%|███ | 36/118 [00:00<00:01, 64.50it/s, v_num=1, train_loss=1.150, Acc=72.20]
Epoch 2: 31%|███▏ | 37/118 [00:00<00:01, 65.65it/s, v_num=1, train_loss=1.150, Acc=72.20]
Epoch 2: 31%|███▏ | 37/118 [00:00<00:01, 65.62it/s, v_num=1, train_loss=1.120, Acc=72.20]
Epoch 2: 32%|███▏ | 38/118 [00:00<00:01, 64.43it/s, v_num=1, train_loss=1.120, Acc=72.20]
Epoch 2: 32%|███▏ | 38/118 [00:00<00:01, 64.40it/s, v_num=1, train_loss=1.230, Acc=72.20]
Epoch 2: 33%|███▎ | 39/118 [00:00<00:01, 65.53it/s, v_num=1, train_loss=1.230, Acc=72.20]
Epoch 2: 33%|███▎ | 39/118 [00:00<00:01, 65.50it/s, v_num=1, train_loss=1.120, Acc=72.20]
Epoch 2: 34%|███▍ | 40/118 [00:00<00:01, 66.60it/s, v_num=1, train_loss=1.120, Acc=72.20]
Epoch 2: 34%|███▍ | 40/118 [00:00<00:01, 66.57it/s, v_num=1, train_loss=1.060, Acc=72.20]
Epoch 2: 35%|███▍ | 41/118 [00:00<00:01, 67.80it/s, v_num=1, train_loss=1.060, Acc=72.20]
Epoch 2: 35%|███▍ | 41/118 [00:00<00:01, 67.74it/s, v_num=1, train_loss=1.110, Acc=72.20]
Epoch 2: 36%|███▌ | 42/118 [00:00<00:01, 68.65it/s, v_num=1, train_loss=1.110, Acc=72.20]
Epoch 2: 36%|███▌ | 42/118 [00:00<00:01, 68.61it/s, v_num=1, train_loss=1.410, Acc=72.20]
Epoch 2: 36%|███▋ | 43/118 [00:00<00:01, 69.66it/s, v_num=1, train_loss=1.410, Acc=72.20]
Epoch 2: 36%|███▋ | 43/118 [00:00<00:01, 69.62it/s, v_num=1, train_loss=1.370, Acc=72.20]
Epoch 2: 37%|███▋ | 44/118 [00:00<00:01, 70.65it/s, v_num=1, train_loss=1.370, Acc=72.20]
Epoch 2: 37%|███▋ | 44/118 [00:00<00:01, 70.61it/s, v_num=1, train_loss=1.420, Acc=72.20]
Epoch 2: 38%|███▊ | 45/118 [00:00<00:01, 71.60it/s, v_num=1, train_loss=1.420, Acc=72.20]
Epoch 2: 38%|███▊ | 45/118 [00:00<00:01, 71.57it/s, v_num=1, train_loss=1.210, Acc=72.20]
Epoch 2: 39%|███▉ | 46/118 [00:00<00:00, 72.27it/s, v_num=1, train_loss=1.210, Acc=72.20]
Epoch 2: 39%|███▉ | 46/118 [00:00<00:00, 72.24it/s, v_num=1, train_loss=1.130, Acc=72.20]
Epoch 2: 40%|███▉ | 47/118 [00:00<00:00, 73.18it/s, v_num=1, train_loss=1.130, Acc=72.20]
Epoch 2: 40%|███▉ | 47/118 [00:00<00:00, 73.15it/s, v_num=1, train_loss=1.040, Acc=72.20]
Epoch 2: 41%|████ | 48/118 [00:00<00:00, 74.10it/s, v_num=1, train_loss=1.040, Acc=72.20]
Epoch 2: 41%|████ | 48/118 [00:00<00:00, 74.07it/s, v_num=1, train_loss=1.010, Acc=72.20]
Epoch 2: 42%|████▏ | 49/118 [00:00<00:00, 75.02it/s, v_num=1, train_loss=1.010, Acc=72.20]
Epoch 2: 42%|████▏ | 49/118 [00:00<00:00, 74.99it/s, v_num=1, train_loss=1.060, Acc=72.20]
Epoch 2: 42%|████▏ | 50/118 [00:00<00:00, 75.60it/s, v_num=1, train_loss=1.060, Acc=72.20]
Epoch 2: 42%|████▏ | 50/118 [00:00<00:00, 75.57it/s, v_num=1, train_loss=1.090, Acc=72.20]
Epoch 2: 43%|████▎ | 51/118 [00:00<00:00, 76.45it/s, v_num=1, train_loss=1.090, Acc=72.20]
Epoch 2: 43%|████▎ | 51/118 [00:00<00:00, 76.42it/s, v_num=1, train_loss=1.070, Acc=72.20]
Epoch 2: 44%|████▍ | 52/118 [00:00<00:00, 77.30it/s, v_num=1, train_loss=1.070, Acc=72.20]
Epoch 2: 44%|████▍ | 52/118 [00:00<00:00, 77.26it/s, v_num=1, train_loss=1.100, Acc=72.20]
Epoch 2: 45%|████▍ | 53/118 [00:00<00:00, 78.14it/s, v_num=1, train_loss=1.100, Acc=72.20]
Epoch 2: 45%|████▍ | 53/118 [00:00<00:00, 78.10it/s, v_num=1, train_loss=1.200, Acc=72.20]
Epoch 2: 46%|████▌ | 54/118 [00:00<00:00, 78.77it/s, v_num=1, train_loss=1.200, Acc=72.20]
Epoch 2: 46%|████▌ | 54/118 [00:00<00:00, 78.74it/s, v_num=1, train_loss=1.210, Acc=72.20]
Epoch 2: 47%|████▋ | 55/118 [00:00<00:00, 79.58it/s, v_num=1, train_loss=1.210, Acc=72.20]
Epoch 2: 47%|████▋ | 55/118 [00:00<00:00, 79.55it/s, v_num=1, train_loss=1.250, Acc=72.20]
Epoch 2: 47%|████▋ | 56/118 [00:00<00:00, 80.42it/s, v_num=1, train_loss=1.250, Acc=72.20]
Epoch 2: 47%|████▋ | 56/118 [00:00<00:00, 80.33it/s, v_num=1, train_loss=1.440, Acc=72.20]
Epoch 2: 48%|████▊ | 57/118 [00:00<00:00, 81.15it/s, v_num=1, train_loss=1.440, Acc=72.20]
Epoch 2: 48%|████▊ | 57/118 [00:00<00:00, 81.10it/s, v_num=1, train_loss=1.150, Acc=72.20]
Epoch 2: 49%|████▉ | 58/118 [00:00<00:00, 81.66it/s, v_num=1, train_loss=1.150, Acc=72.20]
Epoch 2: 49%|████▉ | 58/118 [00:00<00:00, 81.59it/s, v_num=1, train_loss=0.968, Acc=72.20]
Epoch 2: 50%|█████ | 59/118 [00:00<00:00, 82.43it/s, v_num=1, train_loss=0.968, Acc=72.20]
Epoch 2: 50%|█████ | 59/118 [00:00<00:00, 82.36it/s, v_num=1, train_loss=0.884, Acc=72.20]
Epoch 2: 51%|█████ | 60/118 [00:00<00:00, 82.75it/s, v_num=1, train_loss=0.884, Acc=72.20]
Epoch 2: 51%|█████ | 60/118 [00:00<00:00, 82.70it/s, v_num=1, train_loss=0.799, Acc=72.20]
Epoch 2: 52%|█████▏ | 61/118 [00:00<00:00, 83.44it/s, v_num=1, train_loss=0.799, Acc=72.20]
Epoch 2: 52%|█████▏ | 61/118 [00:00<00:00, 83.41it/s, v_num=1, train_loss=0.917, Acc=72.20]
Epoch 2: 53%|█████▎ | 62/118 [00:00<00:00, 84.16it/s, v_num=1, train_loss=0.917, Acc=72.20]
Epoch 2: 53%|█████▎ | 62/118 [00:00<00:00, 84.13it/s, v_num=1, train_loss=0.897, Acc=72.20]
Epoch 2: 53%|█████▎ | 63/118 [00:00<00:00, 84.84it/s, v_num=1, train_loss=0.897, Acc=72.20]
Epoch 2: 53%|█████▎ | 63/118 [00:00<00:00, 84.81it/s, v_num=1, train_loss=0.994, Acc=72.20]
Epoch 2: 54%|█████▍ | 64/118 [00:00<00:00, 85.56it/s, v_num=1, train_loss=0.994, Acc=72.20]
Epoch 2: 54%|█████▍ | 64/118 [00:00<00:00, 85.52it/s, v_num=1, train_loss=1.210, Acc=72.20]
Epoch 2: 55%|█████▌ | 65/118 [00:00<00:00, 86.21it/s, v_num=1, train_loss=1.210, Acc=72.20]
Epoch 2: 55%|█████▌ | 65/118 [00:00<00:00, 86.17it/s, v_num=1, train_loss=1.460, Acc=72.20]
Epoch 2: 56%|█████▌ | 66/118 [00:00<00:00, 83.41it/s, v_num=1, train_loss=1.460, Acc=72.20]
Epoch 2: 56%|█████▌ | 66/118 [00:00<00:00, 83.34it/s, v_num=1, train_loss=0.970, Acc=72.20]
Epoch 2: 57%|█████▋ | 67/118 [00:00<00:00, 84.16it/s, v_num=1, train_loss=0.970, Acc=72.20]
Epoch 2: 57%|█████▋ | 67/118 [00:00<00:00, 84.06it/s, v_num=1, train_loss=0.939, Acc=72.20]
Epoch 2: 58%|█████▊ | 68/118 [00:00<00:00, 84.87it/s, v_num=1, train_loss=0.939, Acc=72.20]
Epoch 2: 58%|█████▊ | 68/118 [00:00<00:00, 84.77it/s, v_num=1, train_loss=0.962, Acc=72.20]
Epoch 2: 58%|█████▊ | 69/118 [00:00<00:00, 85.57it/s, v_num=1, train_loss=0.962, Acc=72.20]
Epoch 2: 58%|█████▊ | 69/118 [00:00<00:00, 85.47it/s, v_num=1, train_loss=0.862, Acc=72.20]
Epoch 2: 59%|█████▉ | 70/118 [00:00<00:00, 86.27it/s, v_num=1, train_loss=0.862, Acc=72.20]
Epoch 2: 59%|█████▉ | 70/118 [00:00<00:00, 86.16it/s, v_num=1, train_loss=0.838, Acc=72.20]
Epoch 2: 60%|██████ | 71/118 [00:00<00:00, 87.13it/s, v_num=1, train_loss=0.838, Acc=72.20]
Epoch 2: 60%|██████ | 71/118 [00:00<00:00, 87.01it/s, v_num=1, train_loss=0.908, Acc=72.20]
Epoch 2: 61%|██████ | 72/118 [00:00<00:00, 87.97it/s, v_num=1, train_loss=0.908, Acc=72.20]
Epoch 2: 61%|██████ | 72/118 [00:00<00:00, 87.85it/s, v_num=1, train_loss=1.090, Acc=72.20]
Epoch 2: 62%|██████▏ | 73/118 [00:00<00:00, 88.69it/s, v_num=1, train_loss=1.090, Acc=72.20]
Epoch 2: 62%|██████▏ | 73/118 [00:00<00:00, 88.65it/s, v_num=1, train_loss=1.380, Acc=72.20]
Epoch 2: 63%|██████▎ | 74/118 [00:00<00:00, 88.98it/s, v_num=1, train_loss=1.380, Acc=72.20]
Epoch 2: 63%|██████▎ | 74/118 [00:00<00:00, 88.95it/s, v_num=1, train_loss=1.020, Acc=72.20]
Epoch 2: 64%|██████▎ | 75/118 [00:00<00:00, 89.60it/s, v_num=1, train_loss=1.020, Acc=72.20]
Epoch 2: 64%|██████▎ | 75/118 [00:00<00:00, 89.57it/s, v_num=1, train_loss=0.904, Acc=72.20]
Epoch 2: 64%|██████▍ | 76/118 [00:00<00:00, 90.22it/s, v_num=1, train_loss=0.904, Acc=72.20]
Epoch 2: 64%|██████▍ | 76/118 [00:00<00:00, 90.19it/s, v_num=1, train_loss=0.819, Acc=72.20]
Epoch 2: 65%|██████▌ | 77/118 [00:00<00:00, 90.82it/s, v_num=1, train_loss=0.819, Acc=72.20]
Epoch 2: 65%|██████▌ | 77/118 [00:00<00:00, 90.80it/s, v_num=1, train_loss=0.865, Acc=72.20]
Epoch 2: 66%|██████▌ | 78/118 [00:00<00:00, 91.42it/s, v_num=1, train_loss=0.865, Acc=72.20]
Epoch 2: 66%|██████▌ | 78/118 [00:00<00:00, 91.39it/s, v_num=1, train_loss=0.881, Acc=72.20]
Epoch 2: 67%|██████▋ | 79/118 [00:00<00:00, 91.98it/s, v_num=1, train_loss=0.881, Acc=72.20]
Epoch 2: 67%|██████▋ | 79/118 [00:00<00:00, 91.95it/s, v_num=1, train_loss=1.030, Acc=72.20]
Epoch 2: 68%|██████▊ | 80/118 [00:00<00:00, 92.57it/s, v_num=1, train_loss=1.030, Acc=72.20]
Epoch 2: 68%|██████▊ | 80/118 [00:00<00:00, 92.55it/s, v_num=1, train_loss=1.160, Acc=72.20]
Epoch 2: 69%|██████▊ | 81/118 [00:00<00:00, 93.28it/s, v_num=1, train_loss=1.160, Acc=72.20]
Epoch 2: 69%|██████▊ | 81/118 [00:00<00:00, 93.25it/s, v_num=1, train_loss=1.680, Acc=72.20]
Epoch 2: 69%|██████▉ | 82/118 [00:00<00:00, 93.34it/s, v_num=1, train_loss=1.680, Acc=72.20]
Epoch 2: 69%|██████▉ | 82/118 [00:00<00:00, 93.31it/s, v_num=1, train_loss=1.270, Acc=72.20]
Epoch 2: 70%|███████ | 83/118 [00:00<00:00, 93.95it/s, v_num=1, train_loss=1.270, Acc=72.20]
Epoch 2: 70%|███████ | 83/118 [00:00<00:00, 93.86it/s, v_num=1, train_loss=1.090, Acc=72.20]
Epoch 2: 71%|███████ | 84/118 [00:00<00:00, 94.52it/s, v_num=1, train_loss=1.090, Acc=72.20]
Epoch 2: 71%|███████ | 84/118 [00:00<00:00, 94.42it/s, v_num=1, train_loss=0.853, Acc=72.20]
Epoch 2: 72%|███████▏ | 85/118 [00:00<00:00, 91.47it/s, v_num=1, train_loss=0.853, Acc=72.20]
Epoch 2: 72%|███████▏ | 85/118 [00:00<00:00, 91.42it/s, v_num=1, train_loss=0.878, Acc=72.20]
Epoch 2: 73%|███████▎ | 86/118 [00:00<00:00, 92.16it/s, v_num=1, train_loss=0.878, Acc=72.20]
Epoch 2: 73%|███████▎ | 86/118 [00:00<00:00, 92.11it/s, v_num=1, train_loss=0.776, Acc=72.20]
Epoch 2: 74%|███████▎ | 87/118 [00:00<00:00, 92.85it/s, v_num=1, train_loss=0.776, Acc=72.20]
Epoch 2: 74%|███████▎ | 87/118 [00:00<00:00, 92.80it/s, v_num=1, train_loss=0.679, Acc=72.20]
Epoch 2: 75%|███████▍ | 88/118 [00:00<00:00, 93.54it/s, v_num=1, train_loss=0.679, Acc=72.20]
Epoch 2: 75%|███████▍ | 88/118 [00:00<00:00, 93.49it/s, v_num=1, train_loss=0.711, Acc=72.20]
Epoch 2: 75%|███████▌ | 89/118 [00:00<00:00, 94.21it/s, v_num=1, train_loss=0.711, Acc=72.20]
Epoch 2: 75%|███████▌ | 89/118 [00:00<00:00, 94.17it/s, v_num=1, train_loss=0.791, Acc=72.20]
Epoch 2: 76%|███████▋ | 90/118 [00:00<00:00, 94.87it/s, v_num=1, train_loss=0.791, Acc=72.20]
Epoch 2: 76%|███████▋ | 90/118 [00:00<00:00, 94.84it/s, v_num=1, train_loss=0.956, Acc=72.20]
Epoch 2: 77%|███████▋ | 91/118 [00:00<00:00, 95.54it/s, v_num=1, train_loss=0.956, Acc=72.20]
Epoch 2: 77%|███████▋ | 91/118 [00:00<00:00, 95.50it/s, v_num=1, train_loss=0.956, Acc=72.20]
Epoch 2: 78%|███████▊ | 92/118 [00:00<00:00, 96.19it/s, v_num=1, train_loss=0.956, Acc=72.20]
Epoch 2: 78%|███████▊ | 92/118 [00:00<00:00, 96.15it/s, v_num=1, train_loss=0.986, Acc=72.20]
Epoch 2: 79%|███████▉ | 93/118 [00:00<00:00, 95.95it/s, v_num=1, train_loss=0.986, Acc=72.20]
Epoch 2: 79%|███████▉ | 93/118 [00:00<00:00, 95.92it/s, v_num=1, train_loss=0.779, Acc=72.20]
Epoch 2: 80%|███████▉ | 94/118 [00:00<00:00, 96.43it/s, v_num=1, train_loss=0.779, Acc=72.20]
Epoch 2: 80%|███████▉ | 94/118 [00:00<00:00, 96.40it/s, v_num=1, train_loss=0.808, Acc=72.20]
Epoch 2: 81%|████████ | 95/118 [00:00<00:00, 96.91it/s, v_num=1, train_loss=0.808, Acc=72.20]
Epoch 2: 81%|████████ | 95/118 [00:00<00:00, 96.88it/s, v_num=1, train_loss=0.698, Acc=72.20]
Epoch 2: 81%|████████▏ | 96/118 [00:00<00:00, 97.39it/s, v_num=1, train_loss=0.698, Acc=72.20]
Epoch 2: 81%|████████▏ | 96/118 [00:00<00:00, 97.36it/s, v_num=1, train_loss=0.644, Acc=72.20]
Epoch 2: 82%|████████▏ | 97/118 [00:00<00:00, 97.85it/s, v_num=1, train_loss=0.644, Acc=72.20]
Epoch 2: 82%|████████▏ | 97/118 [00:00<00:00, 97.82it/s, v_num=1, train_loss=0.670, Acc=72.20]
Epoch 2: 83%|████████▎ | 98/118 [00:00<00:00, 98.36it/s, v_num=1, train_loss=0.670, Acc=72.20]
Epoch 2: 83%|████████▎ | 98/118 [00:00<00:00, 98.33it/s, v_num=1, train_loss=0.677, Acc=72.20]
Epoch 2: 84%|████████▍ | 99/118 [00:01<00:00, 98.97it/s, v_num=1, train_loss=0.677, Acc=72.20]
Epoch 2: 84%|████████▍ | 99/118 [00:01<00:00, 98.94it/s, v_num=1, train_loss=0.771, Acc=72.20]
Epoch 2: 85%|████████▍ | 100/118 [00:01<00:00, 99.57it/s, v_num=1, train_loss=0.771, Acc=72.20]
Epoch 2: 85%|████████▍ | 100/118 [00:01<00:00, 99.54it/s, v_num=1, train_loss=1.060, Acc=72.20]
Epoch 2: 86%|████████▌ | 101/118 [00:01<00:00, 99.57it/s, v_num=1, train_loss=1.060, Acc=72.20]
Epoch 2: 86%|████████▌ | 101/118 [00:01<00:00, 99.54it/s, v_num=1, train_loss=1.370, Acc=72.20]
Epoch 2: 86%|████████▋ | 102/118 [00:01<00:00, 99.45it/s, v_num=1, train_loss=1.370, Acc=72.20]
Epoch 2: 86%|████████▋ | 102/118 [00:01<00:00, 99.42it/s, v_num=1, train_loss=1.080, Acc=72.20]
Epoch 2: 87%|████████▋ | 103/118 [00:01<00:00, 95.04it/s, v_num=1, train_loss=1.080, Acc=72.20]
Epoch 2: 87%|████████▋ | 103/118 [00:01<00:00, 95.01it/s, v_num=1, train_loss=0.921, Acc=72.20]
Epoch 2: 88%|████████▊ | 104/118 [00:01<00:00, 95.52it/s, v_num=1, train_loss=0.921, Acc=72.20]
Epoch 2: 88%|████████▊ | 104/118 [00:01<00:00, 95.48it/s, v_num=1, train_loss=0.872, Acc=72.20]
Epoch 2: 89%|████████▉ | 105/118 [00:01<00:00, 96.04it/s, v_num=1, train_loss=0.872, Acc=72.20]
Epoch 2: 89%|████████▉ | 105/118 [00:01<00:00, 95.96it/s, v_num=1, train_loss=0.743, Acc=72.20]
Epoch 2: 90%|████████▉ | 106/118 [00:01<00:00, 96.57it/s, v_num=1, train_loss=0.743, Acc=72.20]
Epoch 2: 90%|████████▉ | 106/118 [00:01<00:00, 96.47it/s, v_num=1, train_loss=0.689, Acc=72.20]
Epoch 2: 91%|█████████ | 107/118 [00:01<00:00, 97.18it/s, v_num=1, train_loss=0.689, Acc=72.20]
Epoch 2: 91%|█████████ | 107/118 [00:01<00:00, 97.08it/s, v_num=1, train_loss=0.666, Acc=72.20]
Epoch 2: 92%|█████████▏| 108/118 [00:01<00:00, 97.80it/s, v_num=1, train_loss=0.666, Acc=72.20]
Epoch 2: 92%|█████████▏| 108/118 [00:01<00:00, 97.70it/s, v_num=1, train_loss=0.649, Acc=72.20]
Epoch 2: 92%|█████████▏| 109/118 [00:01<00:00, 98.41it/s, v_num=1, train_loss=0.649, Acc=72.20]
Epoch 2: 92%|█████████▏| 109/118 [00:01<00:00, 98.31it/s, v_num=1, train_loss=0.705, Acc=72.20]
Epoch 2: 93%|█████████▎| 110/118 [00:01<00:00, 99.02it/s, v_num=1, train_loss=0.705, Acc=72.20]
Epoch 2: 93%|█████████▎| 110/118 [00:01<00:00, 98.93it/s, v_num=1, train_loss=0.748, Acc=72.20]
Epoch 2: 94%|█████████▍| 111/118 [00:01<00:00, 99.37it/s, v_num=1, train_loss=0.748, Acc=72.20]
Epoch 2: 94%|█████████▍| 111/118 [00:01<00:00, 99.27it/s, v_num=1, train_loss=0.914, Acc=72.20]
Epoch 2: 95%|█████████▍| 112/118 [00:01<00:00, 99.99it/s, v_num=1, train_loss=0.914, Acc=72.20]
Epoch 2: 95%|█████████▍| 112/118 [00:01<00:00, 99.88it/s, v_num=1, train_loss=0.895, Acc=72.20]
Epoch 2: 96%|█████████▌| 113/118 [00:01<00:00, 100.60it/s, v_num=1, train_loss=0.895, Acc=72.20]
Epoch 2: 96%|█████████▌| 113/118 [00:01<00:00, 100.49it/s, v_num=1, train_loss=0.824, Acc=72.20]
Epoch 2: 97%|█████████▋| 114/118 [00:01<00:00, 101.20it/s, v_num=1, train_loss=0.824, Acc=72.20]
Epoch 2: 97%|█████████▋| 114/118 [00:01<00:00, 101.09it/s, v_num=1, train_loss=0.764, Acc=72.20]
Epoch 2: 97%|█████████▋| 115/118 [00:01<00:00, 101.78it/s, v_num=1, train_loss=0.764, Acc=72.20]
Epoch 2: 97%|█████████▋| 115/118 [00:01<00:00, 101.66it/s, v_num=1, train_loss=0.599, Acc=72.20]
Epoch 2: 98%|█████████▊| 116/118 [00:01<00:00, 102.37it/s, v_num=1, train_loss=0.599, Acc=72.20]
Epoch 2: 98%|█████████▊| 116/118 [00:01<00:00, 102.26it/s, v_num=1, train_loss=0.660, Acc=72.20]
Epoch 2: 99%|█████████▉| 117/118 [00:01<00:00, 102.97it/s, v_num=1, train_loss=0.660, Acc=72.20]
Epoch 2: 99%|█████████▉| 117/118 [00:01<00:00, 102.86it/s, v_num=1, train_loss=0.702, Acc=72.20]
Epoch 2: 100%|██████████| 118/118 [00:01<00:00, 103.57it/s, v_num=1, train_loss=0.702, Acc=72.20]
Epoch 2: 100%|██████████| 118/118 [00:01<00:00, 103.56it/s, v_num=1, train_loss=0.609, Acc=72.20]
Validation: | | 0/? [00:00<?, ?it/s]
Validation: 0%| | 0/5 [00:00<?, ?it/s]
Validation DataLoader 0: 0%| | 0/5 [00:00<?, ?it/s]
Validation DataLoader 0: 20%|██ | 1/5 [00:00<00:00, 495.96it/s]
Validation DataLoader 0: 40%|████ | 2/5 [00:00<00:00, 344.78it/s]
Validation DataLoader 0: 60%|██████ | 3/5 [00:00<00:00, 295.89it/s]
Validation DataLoader 0: 80%|████████ | 4/5 [00:00<00:00, 274.14it/s]
Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 53.24it/s]
Epoch 2: 100%|██████████| 118/118 [00:01<00:00, 79.59it/s, v_num=1, train_loss=0.609, Acc=80.40]
Epoch 2: 100%|██████████| 118/118 [00:01<00:00, 79.54it/s, v_num=1, train_loss=0.609, Acc=80.40]
Epoch 2: 100%|██████████| 118/118 [00:01<00:00, 79.38it/s, v_num=1, train_loss=0.609, Acc=80.40]
Testing: | | 0/? [00:00<?, ?it/s]
Testing: 0%| | 0/5 [00:00<?, ?it/s]
Testing DataLoader 0: 0%| | 0/5 [00:00<?, ?it/s]
Testing DataLoader 0: 20%|██ | 1/5 [00:00<00:00, 191.36it/s]
Testing DataLoader 0: 40%|████ | 2/5 [00:00<00:00, 139.10it/s]
Testing DataLoader 0: 60%|██████ | 3/5 [00:00<00:00, 149.87it/s]
Testing DataLoader 0: 80%|████████ | 4/5 [00:00<00:00, 153.67it/s]
Testing DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 55.96it/s]
Testing DataLoader 0: 0%| | 0/5 [00:00<?, ?it/s]
Testing DataLoader 1: 0%| | 0/5 [00:00<?, ?it/s]
Testing DataLoader 1: 20%|██ | 1/5 [00:00<00:00, 182.28it/s]
Testing DataLoader 1: 40%|████ | 2/5 [00:00<00:00, 183.25it/s]
Testing DataLoader 1: 60%|██████ | 3/5 [00:00<00:00, 165.41it/s]
Testing DataLoader 1: 80%|████████ | 4/5 [00:00<00:00, 165.69it/s]
Testing DataLoader 1: 100%|██████████| 5/5 [00:00<00:00, 56.23it/s]
Testing DataLoader 1: 100%|██████████| 5/5 [00:00<00:00, 49.53it/s]
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Test metric ┃ Classification ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Acc │ 80.450% │
│ Brier │ 0.29080 │
│ Entropy │ 0.68304 │
│ NLL │ 0.57907 │
└──────────────┴───────────────────────────┘
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Test metric ┃ Calibration ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ ECE │ 4.714% │
│ aECE │ 4.691% │
└──────────────┴───────────────────────────┘
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Test metric ┃ OOD Detection ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ AUPR │ 85.023% │
│ AUROC │ 87.277% │
│ Entropy │ 0.68304 │
│ FPR95 │ 37.560% │
│ ens_Disagre… │ 0.69800 │
│ ens_Entropy │ 1.25226 │
│ ens_MI │ 0.17854 │
└──────────────┴───────────────────────────┘
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Test metric ┃ Selective Classification ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ AUGRC │ 4.949% │
│ AURC │ 6.690% │
│ Cov@5Risk │ nan% │
│ Risk@80Cov │ 12.500% │
└──────────────┴───────────────────────────┘
Feel free to run the notebook on your machine for a longer duration.
We need to multiply the learning rate by 2 to account for the fact that we have 2 models in the ensemble and that we average the loss over all the predictions.
#### Downloading the pre-trained models
We have put the pre-trained models on Hugging Face that you can download with the utility function “hf_hub_download” imported just below. These models are trained for 75 epochs and are therefore not comparable to the all the others trained in this notebook. The pretrained models can be seen on HuggingFace and TorchUncertainty’s are there.
from torch_uncertainty.utils.hub import hf_hub_download
all_models = []
for i in range(8):
hf_hub_download(
repo_id="ENSTA-U2IS/tutorial-models",
filename=f"version_{i}.ckpt",
local_dir="./models/",
)
model = LeNet(in_channels=1, num_classes=10)
state_dict = torch.load(f"./models/version_{i}.ckpt", map_location="cpu", weights_only=True)[
"state_dict"
]
state_dict = {k.replace("model.", ""): v for k, v in state_dict.items()}
model.load_state_dict(state_dict)
all_models.append(model)
from torch_uncertainty.models import deep_ensembles
from torch_uncertainty.transforms import RepeatTarget
ensemble = deep_ensembles(
all_models,
num_estimators=None,
task="classification",
reset_model_parameters=True,
)
ens_routine = ClassificationRoutine(
is_ensemble=True,
num_classes=10,
model=ensemble,
loss=nn.CrossEntropyLoss(), # The loss for the training
format_batch_fn=RepeatTarget(8), # How to handle the targets when comparing the predictions
optim_recipe=None, # No optim recipe as the model is already trained
eval_ood=True, # We want to evaluate the OOD-related metrics
)
trainer = TUTrainer(accelerator="gpu", devices=1, max_epochs=MAX_EPOCHS)
ens_perf = trainer.test(ens_routine, dataloaders=[test_dl, ood_dl])
Testing: | | 0/? [00:00<?, ?it/s]
Testing: 0%| | 0/5 [00:00<?, ?it/s]
Testing DataLoader 0: 0%| | 0/5 [00:00<?, ?it/s]
Testing DataLoader 0: 20%|██ | 1/5 [00:00<00:00, 63.11it/s]
Testing DataLoader 0: 40%|████ | 2/5 [00:00<00:00, 59.99it/s]
Testing DataLoader 0: 60%|██████ | 3/5 [00:00<00:00, 60.83it/s]
Testing DataLoader 0: 80%|████████ | 4/5 [00:00<00:00, 61.31it/s]
Testing DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 46.55it/s]
Testing DataLoader 0: 0%| | 0/5 [00:00<?, ?it/s]
Testing DataLoader 1: 0%| | 0/5 [00:00<?, ?it/s]
Testing DataLoader 1: 20%|██ | 1/5 [00:00<00:00, 64.07it/s]
Testing DataLoader 1: 40%|████ | 2/5 [00:00<00:00, 61.58it/s]
Testing DataLoader 1: 60%|██████ | 3/5 [00:00<00:00, 62.16it/s]
Testing DataLoader 1: 80%|████████ | 4/5 [00:00<00:00, 62.46it/s]
Testing DataLoader 1: 100%|██████████| 5/5 [00:00<00:00, 50.06it/s]
Testing DataLoader 1: 100%|██████████| 5/5 [00:00<00:00, 45.06it/s]
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Test metric ┃ Classification ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Acc │ 99.610% │
│ Brier │ 0.00677 │
│ Entropy │ 0.02816 │
│ NLL │ 0.01454 │
└──────────────┴───────────────────────────┘
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Test metric ┃ Calibration ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ ECE │ 0.459% │
│ aECE │ 0.451% │
└──────────────┴───────────────────────────┘
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Test metric ┃ OOD Detection ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ AUPR │ 98.980% │
│ AUROC │ 99.205% │
│ Entropy │ 0.02816 │
│ FPR95 │ 2.630% │
│ ens_Disagre… │ 0.38779 │
│ ens_Entropy │ 1.01787 │
│ ens_MI │ 0.23446 │
└──────────────┴───────────────────────────┘
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Test metric ┃ Selective Classification ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ AUGRC │ 0.004% │
│ AURC │ 0.004% │
│ Cov@5Risk │ 100.000% │
│ Risk@80Cov │ 0.000% │
└──────────────┴───────────────────────────┘
4. From Deep Ensembles to Packed-Ensembles#
In the paper Packed-Ensembles for Efficient Uncertainty Quantification published at the International Conference on Learning Representations (ICLR) in 2023, we introduced a modification of Deep Ensembles to make it more computationally-efficient. The idea is to pack the ensemble members into a single model, which allows us to train the ensemble in a single forward pass. This modification is particularly useful when the ensemble size is large, as it is often the case in practice.
We will need to update the model and replace the layers with their Packed equivalents. You can find the documentation of the Packed-Linear layer using this link, and the Packed-Conv2D, here.
import torch
import torch.nn as nn
from torch_uncertainty.layers import PackedConv2d, PackedLinear
class PackedLeNet(nn.Module):
def __init__(
self,
in_channels: int,
num_classes: int,
alpha: int,
num_estimators: int,
) -> None:
super().__init__()
self.num_estimators = num_estimators
self.conv1 = PackedConv2d(
in_channels,
6,
(5, 5),
alpha=alpha,
num_estimators=num_estimators,
first=True,
)
self.conv2 = PackedConv2d(
6,
16,
(5, 5),
alpha=alpha,
num_estimators=num_estimators,
)
self.pooling = nn.AdaptiveAvgPool2d((4, 4))
self.fc1 = PackedLinear(256, 120, alpha=alpha, num_estimators=num_estimators)
self.fc2 = PackedLinear(120, 84, alpha=alpha, num_estimators=num_estimators)
self.fc3 = PackedLinear(
84,
num_classes,
alpha=alpha,
num_estimators=num_estimators,
last=True,
)
def forward(self, x: torch.Tensor) -> torch.Tensor:
out = F.relu(self.conv1(x))
out = F.max_pool2d(out, 2)
out = F.relu(self.conv2(out))
out = F.max_pool2d(out, 2)
out = torch.flatten(out, 1)
out = F.relu(self.fc1(out))
out = F.relu(self.fc2(out))
return self.fc3(out) # Again, no softmax in the model
# Instantiate the model, the images are in grayscale so the number of channels is 1
packed_model = PackedLeNet(in_channels=1, num_classes=10, alpha=2, num_estimators=4)
# Create the trainer that will handle the training
trainer = TUTrainer(accelerator="gpu", devices=1, max_epochs=MAX_EPOCHS)
# The routine is a wrapper of the model that contains the training logic with the metrics, etc
packed_routine = ClassificationRoutine(
is_ensemble=True,
num_classes=10,
model=packed_model,
loss=nn.CrossEntropyLoss(),
format_batch_fn=RepeatTarget(4),
optim_recipe=optim_recipe(packed_model, 4.0),
eval_ood=True,
)
# In practice, avoid performing the validation on the test set
trainer.fit(packed_routine, train_dataloaders=train_dl, val_dataloaders=test_dl)
packed_perf = trainer.test(packed_routine, dataloaders=[test_dl, ood_dl])
Sanity Checking: | | 0/? [00:00<?, ?it/s]
Sanity Checking: 0%| | 0/2 [00:00<?, ?it/s]
Sanity Checking DataLoader 0: 0%| | 0/2 [00:00<?, ?it/s]
Sanity Checking DataLoader 0: 50%|█████ | 1/2 [00:00<00:00, 21.50it/s]
Sanity Checking DataLoader 0: 100%|██████████| 2/2 [00:00<00:00, 40.00it/s]
Training: | | 0/? [00:00<?, ?it/s]
Training: 0%| | 0/118 [00:00<?, ?it/s]
Epoch 0: 0%| | 0/118 [00:00<?, ?it/s]
Epoch 0: 1%| | 1/118 [00:00<00:08, 13.76it/s]
Epoch 0: 1%| | 1/118 [00:00<00:08, 13.68it/s, v_num=3, train_loss=2.310]
Epoch 0: 2%|▏ | 2/118 [00:00<00:07, 14.97it/s, v_num=3, train_loss=2.310]
Epoch 0: 2%|▏ | 2/118 [00:00<00:07, 14.74it/s, v_num=3, train_loss=2.300]
Epoch 0: 3%|▎ | 3/118 [00:00<00:05, 21.62it/s, v_num=3, train_loss=2.300]
Epoch 0: 3%|▎ | 3/118 [00:00<00:05, 21.29it/s, v_num=3, train_loss=2.310]
Epoch 0: 3%|▎ | 4/118 [00:00<00:04, 27.82it/s, v_num=3, train_loss=2.310]
Epoch 0: 3%|▎ | 4/118 [00:00<00:04, 27.41it/s, v_num=3, train_loss=2.310]
Epoch 0: 4%|▍ | 5/118 [00:00<00:03, 33.71it/s, v_num=3, train_loss=2.310]
Epoch 0: 4%|▍ | 5/118 [00:00<00:03, 33.11it/s, v_num=3, train_loss=2.310]
Epoch 0: 5%|▌ | 6/118 [00:00<00:02, 39.13it/s, v_num=3, train_loss=2.310]
Epoch 0: 5%|▌ | 6/118 [00:00<00:02, 38.46it/s, v_num=3, train_loss=2.310]
Epoch 0: 6%|▌ | 7/118 [00:00<00:02, 44.21it/s, v_num=3, train_loss=2.310]
Epoch 0: 6%|▌ | 7/118 [00:00<00:02, 43.48it/s, v_num=3, train_loss=2.310]
Epoch 0: 7%|▋ | 8/118 [00:00<00:02, 48.42it/s, v_num=3, train_loss=2.310]
Epoch 0: 7%|▋ | 8/118 [00:00<00:02, 47.68it/s, v_num=3, train_loss=2.310]
Epoch 0: 8%|▊ | 9/118 [00:00<00:02, 52.29it/s, v_num=3, train_loss=2.310]
Epoch 0: 8%|▊ | 9/118 [00:00<00:02, 51.55it/s, v_num=3, train_loss=2.310]
Epoch 0: 8%|▊ | 10/118 [00:00<00:01, 55.52it/s, v_num=3, train_loss=2.310]
Epoch 0: 8%|▊ | 10/118 [00:00<00:01, 54.92it/s, v_num=3, train_loss=2.310]
Epoch 0: 9%|▉ | 11/118 [00:00<00:01, 59.41it/s, v_num=3, train_loss=2.310]
Epoch 0: 9%|▉ | 11/118 [00:00<00:01, 58.77it/s, v_num=3, train_loss=2.300]
Epoch 0: 10%|█ | 12/118 [00:00<00:01, 63.07it/s, v_num=3, train_loss=2.300]
Epoch 0: 10%|█ | 12/118 [00:00<00:01, 62.42it/s, v_num=3, train_loss=2.300]
Epoch 0: 11%|█ | 13/118 [00:00<00:01, 66.55it/s, v_num=3, train_loss=2.300]
Epoch 0: 11%|█ | 13/118 [00:00<00:01, 65.86it/s, v_num=3, train_loss=2.310]
Epoch 0: 12%|█▏ | 14/118 [00:00<00:01, 69.93it/s, v_num=3, train_loss=2.310]
Epoch 0: 12%|█▏ | 14/118 [00:00<00:01, 69.13it/s, v_num=3, train_loss=2.300]
Epoch 0: 13%|█▎ | 15/118 [00:00<00:01, 72.90it/s, v_num=3, train_loss=2.300]
Epoch 0: 13%|█▎ | 15/118 [00:00<00:01, 71.99it/s, v_num=3, train_loss=2.310]
Epoch 0: 14%|█▎ | 16/118 [00:00<00:01, 75.86it/s, v_num=3, train_loss=2.310]
Epoch 0: 14%|█▎ | 16/118 [00:00<00:01, 74.92it/s, v_num=3, train_loss=2.300]
Epoch 0: 14%|█▍ | 17/118 [00:00<00:01, 78.73it/s, v_num=3, train_loss=2.300]
Epoch 0: 14%|█▍ | 17/118 [00:00<00:01, 77.76it/s, v_num=3, train_loss=2.300]
Epoch 0: 15%|█▌ | 18/118 [00:00<00:01, 79.62it/s, v_num=3, train_loss=2.300]
Epoch 0: 15%|█▌ | 18/118 [00:00<00:01, 78.94it/s, v_num=3, train_loss=2.310]
Epoch 0: 16%|█▌ | 19/118 [00:00<00:01, 82.13it/s, v_num=3, train_loss=2.310]
Epoch 0: 16%|█▌ | 19/118 [00:00<00:01, 81.44it/s, v_num=3, train_loss=2.300]
Epoch 0: 17%|█▋ | 20/118 [00:00<00:01, 84.59it/s, v_num=3, train_loss=2.300]
Epoch 0: 17%|█▋ | 20/118 [00:00<00:01, 83.87it/s, v_num=3, train_loss=2.300]
Epoch 0: 18%|█▊ | 21/118 [00:00<00:01, 77.73it/s, v_num=3, train_loss=2.300]
Epoch 0: 18%|█▊ | 21/118 [00:00<00:01, 77.15it/s, v_num=3, train_loss=2.300]
Epoch 0: 19%|█▊ | 22/118 [00:00<00:01, 79.86it/s, v_num=3, train_loss=2.300]
Epoch 0: 19%|█▊ | 22/118 [00:00<00:01, 79.24it/s, v_num=3, train_loss=2.300]
Epoch 0: 19%|█▉ | 23/118 [00:00<00:01, 81.98it/s, v_num=3, train_loss=2.300]
Epoch 0: 19%|█▉ | 23/118 [00:00<00:01, 81.35it/s, v_num=3, train_loss=2.300]
Epoch 0: 20%|██ | 24/118 [00:00<00:01, 84.01it/s, v_num=3, train_loss=2.300]
Epoch 0: 20%|██ | 24/118 [00:00<00:01, 83.40it/s, v_num=3, train_loss=2.300]
Epoch 0: 21%|██ | 25/118 [00:00<00:01, 85.96it/s, v_num=3, train_loss=2.300]
Epoch 0: 21%|██ | 25/118 [00:00<00:01, 85.38it/s, v_num=3, train_loss=2.300]
Epoch 0: 22%|██▏ | 26/118 [00:00<00:01, 87.84it/s, v_num=3, train_loss=2.300]
Epoch 0: 22%|██▏ | 26/118 [00:00<00:01, 87.25it/s, v_num=3, train_loss=2.300]
Epoch 0: 23%|██▎ | 27/118 [00:00<00:01, 89.12it/s, v_num=3, train_loss=2.300]
Epoch 0: 23%|██▎ | 27/118 [00:00<00:01, 88.56it/s, v_num=3, train_loss=2.300]
Epoch 0: 24%|██▎ | 28/118 [00:00<00:01, 89.82it/s, v_num=3, train_loss=2.300]
Epoch 0: 24%|██▎ | 28/118 [00:00<00:01, 89.27it/s, v_num=3, train_loss=2.310]
Epoch 0: 25%|██▍ | 29/118 [00:00<00:00, 90.30it/s, v_num=3, train_loss=2.310]
Epoch 0: 25%|██▍ | 29/118 [00:00<00:00, 89.77it/s, v_num=3, train_loss=2.310]
Epoch 0: 25%|██▌ | 30/118 [00:00<00:00, 91.54it/s, v_num=3, train_loss=2.310]
Epoch 0: 25%|██▌ | 30/118 [00:00<00:00, 90.98it/s, v_num=3, train_loss=2.300]
Epoch 0: 26%|██▋ | 31/118 [00:00<00:00, 92.69it/s, v_num=3, train_loss=2.300]
Epoch 0: 26%|██▋ | 31/118 [00:00<00:00, 92.14it/s, v_num=3, train_loss=2.300]
Epoch 0: 27%|██▋ | 32/118 [00:00<00:00, 93.77it/s, v_num=3, train_loss=2.300]
Epoch 0: 27%|██▋ | 32/118 [00:00<00:00, 93.23it/s, v_num=3, train_loss=2.300]
Epoch 0: 28%|██▊ | 33/118 [00:00<00:00, 95.27it/s, v_num=3, train_loss=2.300]
Epoch 0: 28%|██▊ | 33/118 [00:00<00:00, 94.68it/s, v_num=3, train_loss=2.310]
Epoch 0: 29%|██▉ | 34/118 [00:00<00:00, 96.67it/s, v_num=3, train_loss=2.310]
Epoch 0: 29%|██▉ | 34/118 [00:00<00:00, 96.11it/s, v_num=3, train_loss=2.300]
Epoch 0: 30%|██▉ | 35/118 [00:00<00:00, 98.09it/s, v_num=3, train_loss=2.300]
Epoch 0: 30%|██▉ | 35/118 [00:00<00:00, 97.53it/s, v_num=3, train_loss=2.300]
Epoch 0: 31%|███ | 36/118 [00:00<00:00, 98.67it/s, v_num=3, train_loss=2.300]
Epoch 0: 31%|███ | 36/118 [00:00<00:00, 98.18it/s, v_num=3, train_loss=2.300]
Epoch 0: 31%|███▏ | 37/118 [00:00<00:00, 98.83it/s, v_num=3, train_loss=2.300]
Epoch 0: 31%|███▏ | 37/118 [00:00<00:00, 98.39it/s, v_num=3, train_loss=2.300]
Epoch 0: 32%|███▏ | 38/118 [00:00<00:00, 99.75it/s, v_num=3, train_loss=2.300]
Epoch 0: 32%|███▏ | 38/118 [00:00<00:00, 99.32it/s, v_num=3, train_loss=2.300]
Epoch 0: 33%|███▎ | 39/118 [00:00<00:00, 100.68it/s, v_num=3, train_loss=2.300]
Epoch 0: 33%|███▎ | 39/118 [00:00<00:00, 100.26it/s, v_num=3, train_loss=2.300]
Epoch 0: 34%|███▍ | 40/118 [00:00<00:00, 93.37it/s, v_num=3, train_loss=2.300]
Epoch 0: 34%|███▍ | 40/118 [00:00<00:00, 92.97it/s, v_num=3, train_loss=2.300]
Epoch 0: 35%|███▍ | 41/118 [00:00<00:00, 94.65it/s, v_num=3, train_loss=2.300]
Epoch 0: 35%|███▍ | 41/118 [00:00<00:00, 94.25it/s, v_num=3, train_loss=2.300]
Epoch 0: 36%|███▌ | 42/118 [00:00<00:00, 95.94it/s, v_num=3, train_loss=2.300]
Epoch 0: 36%|███▌ | 42/118 [00:00<00:00, 95.53it/s, v_num=3, train_loss=2.300]
Epoch 0: 36%|███▋ | 43/118 [00:00<00:00, 97.14it/s, v_num=3, train_loss=2.300]
Epoch 0: 36%|███▋ | 43/118 [00:00<00:00, 96.75it/s, v_num=3, train_loss=2.300]
Epoch 0: 37%|███▋ | 44/118 [00:00<00:00, 98.31it/s, v_num=3, train_loss=2.300]
Epoch 0: 37%|███▋ | 44/118 [00:00<00:00, 97.94it/s, v_num=3, train_loss=2.300]
Epoch 0: 38%|███▊ | 45/118 [00:00<00:00, 99.10it/s, v_num=3, train_loss=2.300]
Epoch 0: 38%|███▊ | 45/118 [00:00<00:00, 98.77it/s, v_num=3, train_loss=2.300]
Epoch 0: 39%|███▉ | 46/118 [00:00<00:00, 100.07it/s, v_num=3, train_loss=2.300]
Epoch 0: 39%|███▉ | 46/118 [00:00<00:00, 99.57it/s, v_num=3, train_loss=2.300]
Epoch 0: 40%|███▉ | 47/118 [00:00<00:00, 100.67it/s, v_num=3, train_loss=2.300]
Epoch 0: 40%|███▉ | 47/118 [00:00<00:00, 100.33it/s, v_num=3, train_loss=2.300]
Epoch 0: 41%|████ | 48/118 [00:00<00:00, 100.88it/s, v_num=3, train_loss=2.300]
Epoch 0: 41%|████ | 48/118 [00:00<00:00, 100.55it/s, v_num=3, train_loss=2.300]
Epoch 0: 42%|████▏ | 49/118 [00:00<00:00, 101.63it/s, v_num=3, train_loss=2.300]
Epoch 0: 42%|████▏ | 49/118 [00:00<00:00, 101.28it/s, v_num=3, train_loss=2.300]
Epoch 0: 42%|████▏ | 50/118 [00:00<00:00, 102.37it/s, v_num=3, train_loss=2.300]
Epoch 0: 42%|████▏ | 50/118 [00:00<00:00, 102.01it/s, v_num=3, train_loss=2.300]
Epoch 0: 43%|████▎ | 51/118 [00:00<00:00, 102.91it/s, v_num=3, train_loss=2.300]
Epoch 0: 43%|████▎ | 51/118 [00:00<00:00, 102.59it/s, v_num=3, train_loss=2.300]
Epoch 0: 44%|████▍ | 52/118 [00:00<00:00, 103.50it/s, v_num=3, train_loss=2.300]
Epoch 0: 44%|████▍ | 52/118 [00:00<00:00, 103.17it/s, v_num=3, train_loss=2.300]
Epoch 0: 45%|████▍ | 53/118 [00:00<00:00, 104.17it/s, v_num=3, train_loss=2.300]
Epoch 0: 45%|████▍ | 53/118 [00:00<00:00, 103.83it/s, v_num=3, train_loss=2.300]
Epoch 0: 46%|████▌ | 54/118 [00:00<00:00, 104.79it/s, v_num=3, train_loss=2.300]
Epoch 0: 46%|████▌ | 54/118 [00:00<00:00, 104.47it/s, v_num=3, train_loss=2.300]
Epoch 0: 47%|████▋ | 55/118 [00:00<00:00, 105.41it/s, v_num=3, train_loss=2.300]
Epoch 0: 47%|████▋ | 55/118 [00:00<00:00, 105.09it/s, v_num=3, train_loss=2.300]
Epoch 0: 47%|████▋ | 56/118 [00:00<00:00, 106.01it/s, v_num=3, train_loss=2.300]
Epoch 0: 47%|████▋ | 56/118 [00:00<00:00, 105.64it/s, v_num=3, train_loss=2.300]
Epoch 0: 48%|████▊ | 57/118 [00:00<00:00, 106.91it/s, v_num=3, train_loss=2.300]
Epoch 0: 48%|████▊ | 57/118 [00:00<00:00, 106.56it/s, v_num=3, train_loss=2.300]
Epoch 0: 49%|████▉ | 58/118 [00:00<00:00, 107.82it/s, v_num=3, train_loss=2.300]
Epoch 0: 49%|████▉ | 58/118 [00:00<00:00, 107.46it/s, v_num=3, train_loss=2.300]
Epoch 0: 50%|█████ | 59/118 [00:00<00:00, 108.79it/s, v_num=3, train_loss=2.300]
Epoch 0: 50%|█████ | 59/118 [00:00<00:00, 108.34it/s, v_num=3, train_loss=2.300]
Epoch 0: 51%|█████ | 60/118 [00:00<00:00, 102.50it/s, v_num=3, train_loss=2.300]
Epoch 0: 51%|█████ | 60/118 [00:00<00:00, 102.19it/s, v_num=3, train_loss=2.290]
Epoch 0: 52%|█████▏ | 61/118 [00:00<00:00, 103.37it/s, v_num=3, train_loss=2.290]
Epoch 0: 52%|█████▏ | 61/118 [00:00<00:00, 103.03it/s, v_num=3, train_loss=2.300]
Epoch 0: 53%|█████▎ | 62/118 [00:00<00:00, 104.23it/s, v_num=3, train_loss=2.300]
Epoch 0: 53%|█████▎ | 62/118 [00:00<00:00, 103.88it/s, v_num=3, train_loss=2.300]
Epoch 0: 53%|█████▎ | 63/118 [00:00<00:00, 105.17it/s, v_num=3, train_loss=2.300]
Epoch 0: 53%|█████▎ | 63/118 [00:00<00:00, 104.74it/s, v_num=3, train_loss=2.300]
Epoch 0: 54%|█████▍ | 64/118 [00:00<00:00, 106.01it/s, v_num=3, train_loss=2.300]
Epoch 0: 54%|█████▍ | 64/118 [00:00<00:00, 105.58it/s, v_num=3, train_loss=2.290]
Epoch 0: 55%|█████▌ | 65/118 [00:00<00:00, 106.80it/s, v_num=3, train_loss=2.290]
Epoch 0: 55%|█████▌ | 65/118 [00:00<00:00, 106.40it/s, v_num=3, train_loss=2.290]
Epoch 0: 56%|█████▌ | 66/118 [00:00<00:00, 107.62it/s, v_num=3, train_loss=2.290]
Epoch 0: 56%|█████▌ | 66/118 [00:00<00:00, 107.21it/s, v_num=3, train_loss=2.300]
Epoch 0: 57%|█████▋ | 67/118 [00:00<00:00, 108.37it/s, v_num=3, train_loss=2.300]
Epoch 0: 57%|█████▋ | 67/118 [00:00<00:00, 107.98it/s, v_num=3, train_loss=2.300]
Epoch 0: 58%|█████▊ | 68/118 [00:00<00:00, 108.55it/s, v_num=3, train_loss=2.300]
Epoch 0: 58%|█████▊ | 68/118 [00:00<00:00, 108.17it/s, v_num=3, train_loss=2.300]
Epoch 0: 58%|█████▊ | 69/118 [00:00<00:00, 109.19it/s, v_num=3, train_loss=2.300]
Epoch 0: 58%|█████▊ | 69/118 [00:00<00:00, 108.90it/s, v_num=3, train_loss=2.300]
Epoch 0: 59%|█████▉ | 70/118 [00:00<00:00, 109.75it/s, v_num=3, train_loss=2.300]
Epoch 0: 59%|█████▉ | 70/118 [00:00<00:00, 109.49it/s, v_num=3, train_loss=2.300]
Epoch 0: 60%|██████ | 71/118 [00:00<00:00, 110.25it/s, v_num=3, train_loss=2.300]
Epoch 0: 60%|██████ | 71/118 [00:00<00:00, 109.98it/s, v_num=3, train_loss=2.290]
Epoch 0: 61%|██████ | 72/118 [00:00<00:00, 110.85it/s, v_num=3, train_loss=2.290]
Epoch 0: 61%|██████ | 72/118 [00:00<00:00, 110.49it/s, v_num=3, train_loss=2.300]
Epoch 0: 62%|██████▏ | 73/118 [00:00<00:00, 111.47it/s, v_num=3, train_loss=2.300]
Epoch 0: 62%|██████▏ | 73/118 [00:00<00:00, 111.19it/s, v_num=3, train_loss=2.300]
Epoch 0: 63%|██████▎ | 74/118 [00:00<00:00, 112.15it/s, v_num=3, train_loss=2.300]
Epoch 0: 63%|██████▎ | 74/118 [00:00<00:00, 111.86it/s, v_num=3, train_loss=2.300]
Epoch 0: 64%|██████▎ | 75/118 [00:00<00:00, 112.84it/s, v_num=3, train_loss=2.300]
Epoch 0: 64%|██████▎ | 75/118 [00:00<00:00, 112.56it/s, v_num=3, train_loss=2.290]
Epoch 0: 64%|██████▍ | 76/118 [00:00<00:00, 113.01it/s, v_num=3, train_loss=2.290]
Epoch 0: 64%|██████▍ | 76/118 [00:00<00:00, 112.72it/s, v_num=3, train_loss=2.300]
Epoch 0: 65%|██████▌ | 77/118 [00:00<00:00, 113.68it/s, v_num=3, train_loss=2.300]
Epoch 0: 65%|██████▌ | 77/118 [00:00<00:00, 113.38it/s, v_num=3, train_loss=2.300]
Epoch 0: 66%|██████▌ | 78/118 [00:00<00:00, 107.89it/s, v_num=3, train_loss=2.300]
Epoch 0: 66%|██████▌ | 78/118 [00:00<00:00, 107.62it/s, v_num=3, train_loss=2.300]
Epoch 0: 67%|██████▋ | 79/118 [00:00<00:00, 108.55it/s, v_num=3, train_loss=2.300]
Epoch 0: 67%|██████▋ | 79/118 [00:00<00:00, 108.28it/s, v_num=3, train_loss=2.290]
Epoch 0: 68%|██████▊ | 80/118 [00:00<00:00, 109.09it/s, v_num=3, train_loss=2.290]
Epoch 0: 68%|██████▊ | 80/118 [00:00<00:00, 108.80it/s, v_num=3, train_loss=2.290]
Epoch 0: 69%|██████▊ | 81/118 [00:00<00:00, 109.76it/s, v_num=3, train_loss=2.290]
Epoch 0: 69%|██████▊ | 81/118 [00:00<00:00, 109.46it/s, v_num=3, train_loss=2.290]
Epoch 0: 69%|██████▉ | 82/118 [00:00<00:00, 110.48it/s, v_num=3, train_loss=2.290]
Epoch 0: 69%|██████▉ | 82/118 [00:00<00:00, 110.12it/s, v_num=3, train_loss=2.290]
Epoch 0: 70%|███████ | 83/118 [00:00<00:00, 111.13it/s, v_num=3, train_loss=2.290]
Epoch 0: 70%|███████ | 83/118 [00:00<00:00, 110.77it/s, v_num=3, train_loss=2.290]
Epoch 0: 71%|███████ | 84/118 [00:00<00:00, 111.79it/s, v_num=3, train_loss=2.290]
Epoch 0: 71%|███████ | 84/118 [00:00<00:00, 111.42it/s, v_num=3, train_loss=2.290]
Epoch 0: 72%|███████▏ | 85/118 [00:00<00:00, 112.41it/s, v_num=3, train_loss=2.290]
Epoch 0: 72%|███████▏ | 85/118 [00:00<00:00, 112.04it/s, v_num=3, train_loss=2.290]
Epoch 0: 73%|███████▎ | 86/118 [00:00<00:00, 112.42it/s, v_num=3, train_loss=2.290]
Epoch 0: 73%|███████▎ | 86/118 [00:00<00:00, 112.17it/s, v_num=3, train_loss=2.290]
Epoch 0: 74%|███████▎ | 87/118 [00:00<00:00, 113.02it/s, v_num=3, train_loss=2.290]
Epoch 0: 74%|███████▎ | 87/118 [00:00<00:00, 112.74it/s, v_num=3, train_loss=2.290]
Epoch 0: 75%|███████▍ | 88/118 [00:00<00:00, 113.56it/s, v_num=3, train_loss=2.290]
Epoch 0: 75%|███████▍ | 88/118 [00:00<00:00, 113.27it/s, v_num=3, train_loss=2.290]
Epoch 0: 75%|███████▌ | 89/118 [00:00<00:00, 114.14it/s, v_num=3, train_loss=2.290]
Epoch 0: 75%|███████▌ | 89/118 [00:00<00:00, 113.86it/s, v_num=3, train_loss=2.290]
Epoch 0: 76%|███████▋ | 90/118 [00:00<00:00, 114.70it/s, v_num=3, train_loss=2.290]
Epoch 0: 76%|███████▋ | 90/118 [00:00<00:00, 114.42it/s, v_num=3, train_loss=2.290]
Epoch 0: 77%|███████▋ | 91/118 [00:00<00:00, 115.32it/s, v_num=3, train_loss=2.290]
Epoch 0: 77%|███████▋ | 91/118 [00:00<00:00, 114.99it/s, v_num=3, train_loss=2.290]
Epoch 0: 78%|███████▊ | 92/118 [00:00<00:00, 115.89it/s, v_num=3, train_loss=2.290]
Epoch 0: 78%|███████▊ | 92/118 [00:00<00:00, 115.55it/s, v_num=3, train_loss=2.290]
Epoch 0: 79%|███████▉ | 93/118 [00:00<00:00, 116.44it/s, v_num=3, train_loss=2.290]
Epoch 0: 79%|███████▉ | 93/118 [00:00<00:00, 116.10it/s, v_num=3, train_loss=2.290]
Epoch 0: 80%|███████▉ | 94/118 [00:00<00:00, 116.19it/s, v_num=3, train_loss=2.290]
Epoch 0: 80%|███████▉ | 94/118 [00:00<00:00, 115.96it/s, v_num=3, train_loss=2.290]
Epoch 0: 81%|████████ | 95/118 [00:00<00:00, 116.70it/s, v_num=3, train_loss=2.290]
Epoch 0: 81%|████████ | 95/118 [00:00<00:00, 116.43it/s, v_num=3, train_loss=2.280]
Epoch 0: 81%|████████▏ | 96/118 [00:00<00:00, 116.89it/s, v_num=3, train_loss=2.280]
Epoch 0: 81%|████████▏ | 96/118 [00:00<00:00, 116.64it/s, v_num=3, train_loss=2.290]
Epoch 0: 82%|████████▏ | 97/118 [00:00<00:00, 111.91it/s, v_num=3, train_loss=2.290]
Epoch 0: 82%|████████▏ | 97/118 [00:00<00:00, 111.68it/s, v_num=3, train_loss=2.290]
Epoch 0: 83%|████████▎ | 98/118 [00:00<00:00, 112.42it/s, v_num=3, train_loss=2.290]
Epoch 0: 83%|████████▎ | 98/118 [00:00<00:00, 112.20it/s, v_num=3, train_loss=2.290]
Epoch 0: 84%|████████▍ | 99/118 [00:00<00:00, 112.95it/s, v_num=3, train_loss=2.290]
Epoch 0: 84%|████████▍ | 99/118 [00:00<00:00, 112.72it/s, v_num=3, train_loss=2.290]
Epoch 0: 85%|████████▍ | 100/118 [00:00<00:00, 113.14it/s, v_num=3, train_loss=2.290]
Epoch 0: 85%|████████▍ | 100/118 [00:00<00:00, 112.90it/s, v_num=3, train_loss=2.290]
Epoch 0: 86%|████████▌ | 101/118 [00:00<00:00, 113.41it/s, v_num=3, train_loss=2.290]
Epoch 0: 86%|████████▌ | 101/118 [00:00<00:00, 113.18it/s, v_num=3, train_loss=2.290]
Epoch 0: 86%|████████▋ | 102/118 [00:00<00:00, 113.31it/s, v_num=3, train_loss=2.290]
Epoch 0: 86%|████████▋ | 102/118 [00:00<00:00, 113.09it/s, v_num=3, train_loss=2.280]
Epoch 0: 87%|████████▋ | 103/118 [00:00<00:00, 113.62it/s, v_num=3, train_loss=2.280]
Epoch 0: 87%|████████▋ | 103/118 [00:00<00:00, 113.40it/s, v_num=3, train_loss=2.290]
Epoch 0: 88%|████████▊ | 104/118 [00:00<00:00, 113.94it/s, v_num=3, train_loss=2.290]
Epoch 0: 88%|████████▊ | 104/118 [00:00<00:00, 113.73it/s, v_num=3, train_loss=2.280]
Epoch 0: 89%|████████▉ | 105/118 [00:00<00:00, 114.04it/s, v_num=3, train_loss=2.280]
Epoch 0: 89%|████████▉ | 105/118 [00:00<00:00, 113.82it/s, v_num=3, train_loss=2.290]
Epoch 0: 90%|████████▉ | 106/118 [00:00<00:00, 114.38it/s, v_num=3, train_loss=2.290]
Epoch 0: 90%|████████▉ | 106/118 [00:00<00:00, 114.16it/s, v_num=3, train_loss=2.280]
Epoch 0: 91%|█████████ | 107/118 [00:00<00:00, 114.91it/s, v_num=3, train_loss=2.280]
Epoch 0: 91%|█████████ | 107/118 [00:00<00:00, 114.67it/s, v_num=3, train_loss=2.290]
Epoch 0: 92%|█████████▏| 108/118 [00:00<00:00, 115.30it/s, v_num=3, train_loss=2.290]
Epoch 0: 92%|█████████▏| 108/118 [00:00<00:00, 115.06it/s, v_num=3, train_loss=2.280]
Epoch 0: 92%|█████████▏| 109/118 [00:00<00:00, 115.80it/s, v_num=3, train_loss=2.280]
Epoch 0: 92%|█████████▏| 109/118 [00:00<00:00, 115.56it/s, v_num=3, train_loss=2.280]
Epoch 0: 93%|█████████▎| 110/118 [00:00<00:00, 116.31it/s, v_num=3, train_loss=2.280]
Epoch 0: 93%|█████████▎| 110/118 [00:00<00:00, 116.06it/s, v_num=3, train_loss=2.280]
Epoch 0: 94%|█████████▍| 111/118 [00:00<00:00, 116.83it/s, v_num=3, train_loss=2.280]
Epoch 0: 94%|█████████▍| 111/118 [00:00<00:00, 116.55it/s, v_num=3, train_loss=2.280]
Epoch 0: 95%|█████████▍| 112/118 [00:00<00:00, 117.37it/s, v_num=3, train_loss=2.280]
Epoch 0: 95%|█████████▍| 112/118 [00:00<00:00, 117.06it/s, v_num=3, train_loss=2.280]
Epoch 0: 96%|█████████▌| 113/118 [00:00<00:00, 117.74it/s, v_num=3, train_loss=2.280]
Epoch 0: 96%|█████████▌| 113/118 [00:00<00:00, 117.44it/s, v_num=3, train_loss=2.280]
Epoch 0: 97%|█████████▋| 114/118 [00:00<00:00, 118.23it/s, v_num=3, train_loss=2.280]
Epoch 0: 97%|█████████▋| 114/118 [00:00<00:00, 117.93it/s, v_num=3, train_loss=2.280]
Epoch 0: 97%|█████████▋| 115/118 [00:00<00:00, 118.72it/s, v_num=3, train_loss=2.280]
Epoch 0: 97%|█████████▋| 115/118 [00:00<00:00, 118.42it/s, v_num=3, train_loss=2.280]
Epoch 0: 98%|█████████▊| 116/118 [00:00<00:00, 119.15it/s, v_num=3, train_loss=2.280]
Epoch 0: 98%|█████████▊| 116/118 [00:00<00:00, 118.85it/s, v_num=3, train_loss=2.280]
Epoch 0: 99%|█████████▉| 117/118 [00:00<00:00, 119.58it/s, v_num=3, train_loss=2.280]
Epoch 0: 99%|█████████▉| 117/118 [00:00<00:00, 119.27it/s, v_num=3, train_loss=2.280]
Epoch 0: 100%|██████████| 118/118 [00:00<00:00, 119.48it/s, v_num=3, train_loss=2.280]
Epoch 0: 100%|██████████| 118/118 [00:00<00:00, 119.47it/s, v_num=3, train_loss=2.280]
Validation: | | 0/? [00:00<?, ?it/s]
Validation: 0%| | 0/5 [00:00<?, ?it/s]
Validation DataLoader 0: 0%| | 0/5 [00:00<?, ?it/s]
Validation DataLoader 0: 20%|██ | 1/5 [00:00<00:00, 457.89it/s]
Validation DataLoader 0: 40%|████ | 2/5 [00:00<00:00, 65.58it/s]
Validation DataLoader 0: 60%|██████ | 3/5 [00:00<00:00, 85.69it/s]
Validation DataLoader 0: 80%|████████ | 4/5 [00:00<00:00, 100.66it/s]
Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 52.43it/s]
Epoch 0: 100%|██████████| 118/118 [00:01<00:00, 90.40it/s, v_num=3, train_loss=2.280, Acc=40.20]
Epoch 0: 100%|██████████| 118/118 [00:01<00:00, 90.35it/s, v_num=3, train_loss=2.280, Acc=40.20]
Epoch 0: 0%| | 0/118 [00:00<?, ?it/s, v_num=3, train_loss=2.280, Acc=40.20]
Epoch 1: 0%| | 0/118 [00:00<?, ?it/s, v_num=3, train_loss=2.280, Acc=40.20]
Epoch 1: 1%| | 1/118 [00:00<00:33, 3.52it/s, v_num=3, train_loss=2.280, Acc=40.20]
Epoch 1: 1%| | 1/118 [00:00<00:33, 3.50it/s, v_num=3, train_loss=2.280, Acc=40.20]
Epoch 1: 2%|▏ | 2/118 [00:00<00:18, 6.13it/s, v_num=3, train_loss=2.280, Acc=40.20]
Epoch 1: 2%|▏ | 2/118 [00:00<00:19, 6.08it/s, v_num=3, train_loss=2.270, Acc=40.20]
Epoch 1: 3%|▎ | 3/118 [00:00<00:12, 9.06it/s, v_num=3, train_loss=2.270, Acc=40.20]
Epoch 1: 3%|▎ | 3/118 [00:00<00:12, 9.00it/s, v_num=3, train_loss=2.280, Acc=40.20]
Epoch 1: 3%|▎ | 4/118 [00:00<00:09, 11.91it/s, v_num=3, train_loss=2.280, Acc=40.20]
Epoch 1: 3%|▎ | 4/118 [00:00<00:09, 11.83it/s, v_num=3, train_loss=2.270, Acc=40.20]
Epoch 1: 4%|▍ | 5/118 [00:00<00:07, 14.69it/s, v_num=3, train_loss=2.270, Acc=40.20]
Epoch 1: 4%|▍ | 5/118 [00:00<00:07, 14.58it/s, v_num=3, train_loss=2.270, Acc=40.20]
Epoch 1: 5%|▌ | 6/118 [00:00<00:06, 17.38it/s, v_num=3, train_loss=2.270, Acc=40.20]
Epoch 1: 5%|▌ | 6/118 [00:00<00:06, 17.26it/s, v_num=3, train_loss=2.270, Acc=40.20]
Epoch 1: 6%|▌ | 7/118 [00:00<00:05, 20.01it/s, v_num=3, train_loss=2.270, Acc=40.20]
Epoch 1: 6%|▌ | 7/118 [00:00<00:05, 19.87it/s, v_num=3, train_loss=2.270, Acc=40.20]
Epoch 1: 7%|▋ | 8/118 [00:00<00:04, 22.51it/s, v_num=3, train_loss=2.270, Acc=40.20]
Epoch 1: 7%|▋ | 8/118 [00:00<00:04, 22.36it/s, v_num=3, train_loss=2.270, Acc=40.20]
Epoch 1: 8%|▊ | 9/118 [00:00<00:04, 24.81it/s, v_num=3, train_loss=2.270, Acc=40.20]
Epoch 1: 8%|▊ | 9/118 [00:00<00:04, 24.71it/s, v_num=3, train_loss=2.270, Acc=40.20]
Epoch 1: 8%|▊ | 10/118 [00:00<00:04, 26.80it/s, v_num=3, train_loss=2.270, Acc=40.20]
Epoch 1: 8%|▊ | 10/118 [00:00<00:04, 26.69it/s, v_num=3, train_loss=2.270, Acc=40.20]
Epoch 1: 9%|▉ | 11/118 [00:00<00:03, 28.99it/s, v_num=3, train_loss=2.270, Acc=40.20]
Epoch 1: 9%|▉ | 11/118 [00:00<00:03, 28.86it/s, v_num=3, train_loss=2.270, Acc=40.20]
Epoch 1: 10%|█ | 12/118 [00:00<00:03, 31.09it/s, v_num=3, train_loss=2.270, Acc=40.20]
Epoch 1: 10%|█ | 12/118 [00:00<00:03, 30.96it/s, v_num=3, train_loss=2.270, Acc=40.20]
Epoch 1: 11%|█ | 13/118 [00:00<00:03, 33.13it/s, v_num=3, train_loss=2.270, Acc=40.20]
Epoch 1: 11%|█ | 13/118 [00:00<00:03, 32.99it/s, v_num=3, train_loss=2.260, Acc=40.20]
Epoch 1: 12%|█▏ | 14/118 [00:00<00:02, 35.08it/s, v_num=3, train_loss=2.260, Acc=40.20]
Epoch 1: 12%|█▏ | 14/118 [00:00<00:02, 34.94it/s, v_num=3, train_loss=2.250, Acc=40.20]
Epoch 1: 13%|█▎ | 15/118 [00:00<00:02, 36.99it/s, v_num=3, train_loss=2.250, Acc=40.20]
Epoch 1: 13%|█▎ | 15/118 [00:00<00:02, 36.85it/s, v_num=3, train_loss=2.260, Acc=40.20]
Epoch 1: 14%|█▎ | 16/118 [00:00<00:02, 38.84it/s, v_num=3, train_loss=2.260, Acc=40.20]
Epoch 1: 14%|█▎ | 16/118 [00:00<00:02, 38.68it/s, v_num=3, train_loss=2.250, Acc=40.20]
Epoch 1: 14%|█▍ | 17/118 [00:00<00:02, 40.63it/s, v_num=3, train_loss=2.250, Acc=40.20]
Epoch 1: 14%|█▍ | 17/118 [00:00<00:02, 40.47it/s, v_num=3, train_loss=2.250, Acc=40.20]
Epoch 1: 15%|█▌ | 18/118 [00:00<00:02, 40.11it/s, v_num=3, train_loss=2.250, Acc=40.20]
Epoch 1: 15%|█▌ | 18/118 [00:00<00:02, 39.95it/s, v_num=3, train_loss=2.260, Acc=40.20]
Epoch 1: 16%|█▌ | 19/118 [00:00<00:02, 41.79it/s, v_num=3, train_loss=2.260, Acc=40.20]
Epoch 1: 16%|█▌ | 19/118 [00:00<00:02, 41.62it/s, v_num=3, train_loss=2.250, Acc=40.20]
Epoch 1: 17%|█▋ | 20/118 [00:00<00:02, 43.42it/s, v_num=3, train_loss=2.250, Acc=40.20]
Epoch 1: 17%|█▋ | 20/118 [00:00<00:02, 43.24it/s, v_num=3, train_loss=2.250, Acc=40.20]
Epoch 1: 18%|█▊ | 21/118 [00:00<00:02, 44.97it/s, v_num=3, train_loss=2.250, Acc=40.20]
Epoch 1: 18%|█▊ | 21/118 [00:00<00:02, 44.81it/s, v_num=3, train_loss=2.250, Acc=40.20]
Epoch 1: 19%|█▊ | 22/118 [00:00<00:02, 46.51it/s, v_num=3, train_loss=2.250, Acc=40.20]
Epoch 1: 19%|█▊ | 22/118 [00:00<00:02, 46.34it/s, v_num=3, train_loss=2.250, Acc=40.20]
Epoch 1: 19%|█▉ | 23/118 [00:00<00:01, 47.83it/s, v_num=3, train_loss=2.250, Acc=40.20]
Epoch 1: 19%|█▉ | 23/118 [00:00<00:01, 47.66it/s, v_num=3, train_loss=2.240, Acc=40.20]
Epoch 1: 20%|██ | 24/118 [00:00<00:01, 49.36it/s, v_num=3, train_loss=2.240, Acc=40.20]
Epoch 1: 20%|██ | 24/118 [00:00<00:01, 49.19it/s, v_num=3, train_loss=2.240, Acc=40.20]
Epoch 1: 21%|██ | 25/118 [00:00<00:01, 50.91it/s, v_num=3, train_loss=2.240, Acc=40.20]
Epoch 1: 21%|██ | 25/118 [00:00<00:01, 50.74it/s, v_num=3, train_loss=2.240, Acc=40.20]
Epoch 1: 22%|██▏ | 26/118 [00:00<00:01, 51.97it/s, v_num=3, train_loss=2.240, Acc=40.20]
Epoch 1: 22%|██▏ | 26/118 [00:00<00:01, 51.81it/s, v_num=3, train_loss=2.240, Acc=40.20]
Epoch 1: 23%|██▎ | 27/118 [00:00<00:01, 53.38it/s, v_num=3, train_loss=2.240, Acc=40.20]
Epoch 1: 23%|██▎ | 27/118 [00:00<00:01, 53.20it/s, v_num=3, train_loss=2.230, Acc=40.20]
Epoch 1: 24%|██▎ | 28/118 [00:00<00:01, 54.84it/s, v_num=3, train_loss=2.230, Acc=40.20]
Epoch 1: 24%|██▎ | 28/118 [00:00<00:01, 54.66it/s, v_num=3, train_loss=2.240, Acc=40.20]
Epoch 1: 25%|██▍ | 29/118 [00:00<00:01, 56.27it/s, v_num=3, train_loss=2.240, Acc=40.20]
Epoch 1: 25%|██▍ | 29/118 [00:00<00:01, 56.09it/s, v_num=3, train_loss=2.230, Acc=40.20]
Epoch 1: 25%|██▌ | 30/118 [00:00<00:01, 57.65it/s, v_num=3, train_loss=2.230, Acc=40.20]
Epoch 1: 25%|██▌ | 30/118 [00:00<00:01, 57.47it/s, v_num=3, train_loss=2.220, Acc=40.20]
Epoch 1: 26%|██▋ | 31/118 [00:00<00:01, 58.88it/s, v_num=3, train_loss=2.220, Acc=40.20]
Epoch 1: 26%|██▋ | 31/118 [00:00<00:01, 58.70it/s, v_num=3, train_loss=2.220, Acc=40.20]
Epoch 1: 27%|██▋ | 32/118 [00:00<00:01, 60.08it/s, v_num=3, train_loss=2.220, Acc=40.20]
Epoch 1: 27%|██▋ | 32/118 [00:00<00:01, 59.90it/s, v_num=3, train_loss=2.220, Acc=40.20]
Epoch 1: 28%|██▊ | 33/118 [00:00<00:01, 61.16it/s, v_num=3, train_loss=2.220, Acc=40.20]
Epoch 1: 28%|██▊ | 33/118 [00:00<00:01, 60.98it/s, v_num=3, train_loss=2.210, Acc=40.20]
Epoch 1: 29%|██▉ | 34/118 [00:00<00:01, 61.81it/s, v_num=3, train_loss=2.210, Acc=40.20]
Epoch 1: 29%|██▉ | 34/118 [00:00<00:01, 61.62it/s, v_num=3, train_loss=2.210, Acc=40.20]
Epoch 1: 30%|██▉ | 35/118 [00:00<00:01, 62.91it/s, v_num=3, train_loss=2.210, Acc=40.20]
Epoch 1: 30%|██▉ | 35/118 [00:00<00:01, 62.72it/s, v_num=3, train_loss=2.210, Acc=40.20]
Epoch 1: 31%|███ | 36/118 [00:00<00:01, 63.95it/s, v_num=3, train_loss=2.210, Acc=40.20]
Epoch 1: 31%|███ | 36/118 [00:00<00:01, 63.77it/s, v_num=3, train_loss=2.210, Acc=40.20]
Epoch 1: 31%|███▏ | 37/118 [00:00<00:01, 64.81it/s, v_num=3, train_loss=2.210, Acc=40.20]
Epoch 1: 31%|███▏ | 37/118 [00:00<00:01, 64.55it/s, v_num=3, train_loss=2.200, Acc=40.20]
Epoch 1: 32%|███▏ | 38/118 [00:00<00:01, 66.02it/s, v_num=3, train_loss=2.200, Acc=40.20]
Epoch 1: 32%|███▏ | 38/118 [00:00<00:01, 65.74it/s, v_num=3, train_loss=2.190, Acc=40.20]
Epoch 1: 33%|███▎ | 39/118 [00:00<00:01, 67.21it/s, v_num=3, train_loss=2.190, Acc=40.20]
Epoch 1: 33%|███▎ | 39/118 [00:00<00:01, 66.93it/s, v_num=3, train_loss=2.180, Acc=40.20]
Epoch 1: 34%|███▍ | 40/118 [00:00<00:01, 63.63it/s, v_num=3, train_loss=2.180, Acc=40.20]
Epoch 1: 34%|███▍ | 40/118 [00:00<00:01, 63.45it/s, v_num=3, train_loss=2.170, Acc=40.20]
Epoch 1: 35%|███▍ | 41/118 [00:00<00:01, 64.61it/s, v_num=3, train_loss=2.170, Acc=40.20]
Epoch 1: 35%|███▍ | 41/118 [00:00<00:01, 64.43it/s, v_num=3, train_loss=2.170, Acc=40.20]
Epoch 1: 36%|███▌ | 42/118 [00:00<00:01, 65.55it/s, v_num=3, train_loss=2.170, Acc=40.20]
Epoch 1: 36%|███▌ | 42/118 [00:00<00:01, 65.37it/s, v_num=3, train_loss=2.180, Acc=40.20]
Epoch 1: 36%|███▋ | 43/118 [00:00<00:01, 66.62it/s, v_num=3, train_loss=2.180, Acc=40.20]
Epoch 1: 36%|███▋ | 43/118 [00:00<00:01, 66.43it/s, v_num=3, train_loss=2.180, Acc=40.20]
Epoch 1: 37%|███▋ | 44/118 [00:00<00:01, 67.67it/s, v_num=3, train_loss=2.180, Acc=40.20]
Epoch 1: 37%|███▋ | 44/118 [00:00<00:01, 67.48it/s, v_num=3, train_loss=2.150, Acc=40.20]
Epoch 1: 38%|███▊ | 45/118 [00:00<00:01, 68.62it/s, v_num=3, train_loss=2.150, Acc=40.20]
Epoch 1: 38%|███▊ | 45/118 [00:00<00:01, 68.43it/s, v_num=3, train_loss=2.160, Acc=40.20]
Epoch 1: 39%|███▉ | 46/118 [00:00<00:01, 69.70it/s, v_num=3, train_loss=2.160, Acc=40.20]
Epoch 1: 39%|███▉ | 46/118 [00:00<00:01, 69.44it/s, v_num=3, train_loss=2.150, Acc=40.20]
Epoch 1: 40%|███▉ | 47/118 [00:00<00:01, 70.68it/s, v_num=3, train_loss=2.150, Acc=40.20]
Epoch 1: 40%|███▉ | 47/118 [00:00<00:01, 70.45it/s, v_num=3, train_loss=2.130, Acc=40.20]
Epoch 1: 41%|████ | 48/118 [00:00<00:00, 71.35it/s, v_num=3, train_loss=2.130, Acc=40.20]
Epoch 1: 41%|████ | 48/118 [00:00<00:00, 71.10it/s, v_num=3, train_loss=2.140, Acc=40.20]
Epoch 1: 42%|████▏ | 49/118 [00:00<00:00, 72.16it/s, v_num=3, train_loss=2.140, Acc=40.20]
Epoch 1: 42%|████▏ | 49/118 [00:00<00:00, 71.91it/s, v_num=3, train_loss=2.140, Acc=40.20]
Epoch 1: 42%|████▏ | 50/118 [00:00<00:00, 72.87it/s, v_num=3, train_loss=2.140, Acc=40.20]
Epoch 1: 42%|████▏ | 50/118 [00:00<00:00, 72.70it/s, v_num=3, train_loss=2.130, Acc=40.20]
Epoch 1: 43%|████▎ | 51/118 [00:00<00:00, 73.63it/s, v_num=3, train_loss=2.130, Acc=40.20]
Epoch 1: 43%|████▎ | 51/118 [00:00<00:00, 73.46it/s, v_num=3, train_loss=2.130, Acc=40.20]
Epoch 1: 44%|████▍ | 52/118 [00:00<00:00, 74.35it/s, v_num=3, train_loss=2.130, Acc=40.20]
Epoch 1: 44%|████▍ | 52/118 [00:00<00:00, 74.17it/s, v_num=3, train_loss=2.140, Acc=40.20]
Epoch 1: 45%|████▍ | 53/118 [00:00<00:00, 74.93it/s, v_num=3, train_loss=2.140, Acc=40.20]
Epoch 1: 45%|████▍ | 53/118 [00:00<00:00, 74.74it/s, v_num=3, train_loss=2.100, Acc=40.20]
Epoch 1: 46%|████▌ | 54/118 [00:00<00:00, 75.69it/s, v_num=3, train_loss=2.100, Acc=40.20]
Epoch 1: 46%|████▌ | 54/118 [00:00<00:00, 75.50it/s, v_num=3, train_loss=2.080, Acc=40.20]
Epoch 1: 47%|████▋ | 55/118 [00:00<00:00, 76.56it/s, v_num=3, train_loss=2.080, Acc=40.20]
Epoch 1: 47%|████▋ | 55/118 [00:00<00:00, 76.37it/s, v_num=3, train_loss=2.130, Acc=40.20]
Epoch 1: 47%|████▋ | 56/118 [00:00<00:00, 77.24it/s, v_num=3, train_loss=2.130, Acc=40.20]
Epoch 1: 47%|████▋ | 56/118 [00:00<00:00, 77.05it/s, v_num=3, train_loss=2.220, Acc=40.20]
Epoch 1: 48%|████▊ | 57/118 [00:00<00:00, 78.11it/s, v_num=3, train_loss=2.220, Acc=40.20]
Epoch 1: 48%|████▊ | 57/118 [00:00<00:00, 77.91it/s, v_num=3, train_loss=2.170, Acc=40.20]
Epoch 1: 49%|████▉ | 58/118 [00:00<00:00, 78.79it/s, v_num=3, train_loss=2.170, Acc=40.20]
Epoch 1: 49%|████▉ | 58/118 [00:00<00:00, 78.59it/s, v_num=3, train_loss=2.150, Acc=40.20]
Epoch 1: 50%|█████ | 59/118 [00:00<00:00, 79.62it/s, v_num=3, train_loss=2.150, Acc=40.20]
Epoch 1: 50%|█████ | 59/118 [00:00<00:00, 79.42it/s, v_num=3, train_loss=2.150, Acc=40.20]
Epoch 1: 51%|█████ | 60/118 [00:00<00:00, 77.15it/s, v_num=3, train_loss=2.150, Acc=40.20]
Epoch 1: 51%|█████ | 60/118 [00:00<00:00, 76.98it/s, v_num=3, train_loss=2.170, Acc=40.20]
Epoch 1: 52%|█████▏ | 61/118 [00:00<00:00, 77.83it/s, v_num=3, train_loss=2.170, Acc=40.20]
Epoch 1: 52%|█████▏ | 61/118 [00:00<00:00, 77.66it/s, v_num=3, train_loss=2.110, Acc=40.20]
Epoch 1: 53%|█████▎ | 62/118 [00:00<00:00, 78.44it/s, v_num=3, train_loss=2.110, Acc=40.20]
Epoch 1: 53%|█████▎ | 62/118 [00:00<00:00, 78.28it/s, v_num=3, train_loss=2.090, Acc=40.20]
Epoch 1: 53%|█████▎ | 63/118 [00:00<00:00, 79.10it/s, v_num=3, train_loss=2.090, Acc=40.20]
Epoch 1: 53%|█████▎ | 63/118 [00:00<00:00, 78.94it/s, v_num=3, train_loss=2.070, Acc=40.20]
Epoch 1: 54%|█████▍ | 64/118 [00:00<00:00, 79.88it/s, v_num=3, train_loss=2.070, Acc=40.20]
Epoch 1: 54%|█████▍ | 64/118 [00:00<00:00, 79.71it/s, v_num=3, train_loss=2.070, Acc=40.20]
Epoch 1: 55%|█████▌ | 65/118 [00:00<00:00, 80.65it/s, v_num=3, train_loss=2.070, Acc=40.20]
Epoch 1: 55%|█████▌ | 65/118 [00:00<00:00, 80.48it/s, v_num=3, train_loss=2.070, Acc=40.20]
Epoch 1: 56%|█████▌ | 66/118 [00:00<00:00, 81.38it/s, v_num=3, train_loss=2.070, Acc=40.20]
Epoch 1: 56%|█████▌ | 66/118 [00:00<00:00, 81.22it/s, v_num=3, train_loss=2.070, Acc=40.20]
Epoch 1: 57%|█████▋ | 67/118 [00:00<00:00, 81.81it/s, v_num=3, train_loss=2.070, Acc=40.20]
Epoch 1: 57%|█████▋ | 67/118 [00:00<00:00, 81.65it/s, v_num=3, train_loss=2.140, Acc=40.20]
Epoch 1: 58%|█████▊ | 68/118 [00:00<00:00, 82.01it/s, v_num=3, train_loss=2.140, Acc=40.20]
Epoch 1: 58%|█████▊ | 68/118 [00:00<00:00, 81.84it/s, v_num=3, train_loss=2.140, Acc=40.20]
Epoch 1: 58%|█████▊ | 69/118 [00:00<00:00, 82.72it/s, v_num=3, train_loss=2.140, Acc=40.20]
Epoch 1: 58%|█████▊ | 69/118 [00:00<00:00, 82.54it/s, v_num=3, train_loss=2.100, Acc=40.20]
Epoch 1: 59%|█████▉ | 70/118 [00:00<00:00, 83.42it/s, v_num=3, train_loss=2.100, Acc=40.20]
Epoch 1: 59%|█████▉ | 70/118 [00:00<00:00, 83.25it/s, v_num=3, train_loss=2.140, Acc=40.20]
Epoch 1: 60%|██████ | 71/118 [00:00<00:00, 84.12it/s, v_num=3, train_loss=2.140, Acc=40.20]
Epoch 1: 60%|██████ | 71/118 [00:00<00:00, 83.95it/s, v_num=3, train_loss=2.100, Acc=40.20]
Epoch 1: 61%|██████ | 72/118 [00:00<00:00, 84.82it/s, v_num=3, train_loss=2.100, Acc=40.20]
Epoch 1: 61%|██████ | 72/118 [00:00<00:00, 84.65it/s, v_num=3, train_loss=2.080, Acc=40.20]
Epoch 1: 62%|██████▏ | 73/118 [00:00<00:00, 85.50it/s, v_num=3, train_loss=2.080, Acc=40.20]
Epoch 1: 62%|██████▏ | 73/118 [00:00<00:00, 85.32it/s, v_num=3, train_loss=2.060, Acc=40.20]
Epoch 1: 63%|██████▎ | 74/118 [00:00<00:00, 86.14it/s, v_num=3, train_loss=2.060, Acc=40.20]
Epoch 1: 63%|██████▎ | 74/118 [00:00<00:00, 85.98it/s, v_num=3, train_loss=2.060, Acc=40.20]
Epoch 1: 64%|██████▎ | 75/118 [00:00<00:00, 86.45it/s, v_num=3, train_loss=2.060, Acc=40.20]
Epoch 1: 64%|██████▎ | 75/118 [00:00<00:00, 86.30it/s, v_num=3, train_loss=2.120, Acc=40.20]
Epoch 1: 64%|██████▍ | 76/118 [00:00<00:00, 86.79it/s, v_num=3, train_loss=2.120, Acc=40.20]
Epoch 1: 64%|██████▍ | 76/118 [00:00<00:00, 86.63it/s, v_num=3, train_loss=2.080, Acc=40.20]
Epoch 1: 65%|██████▌ | 77/118 [00:00<00:00, 87.27it/s, v_num=3, train_loss=2.080, Acc=40.20]
Epoch 1: 65%|██████▌ | 77/118 [00:00<00:00, 87.12it/s, v_num=3, train_loss=2.040, Acc=40.20]
Epoch 1: 66%|██████▌ | 78/118 [00:00<00:00, 87.82it/s, v_num=3, train_loss=2.040, Acc=40.20]
Epoch 1: 66%|██████▌ | 78/118 [00:00<00:00, 87.60it/s, v_num=3, train_loss=2.040, Acc=40.20]
Epoch 1: 67%|██████▋ | 79/118 [00:00<00:00, 87.97it/s, v_num=3, train_loss=2.040, Acc=40.20]
Epoch 1: 67%|██████▋ | 79/118 [00:00<00:00, 87.81it/s, v_num=3, train_loss=2.060, Acc=40.20]
Epoch 1: 68%|██████▊ | 80/118 [00:00<00:00, 88.45it/s, v_num=3, train_loss=2.060, Acc=40.20]
Epoch 1: 68%|██████▊ | 80/118 [00:00<00:00, 88.29it/s, v_num=3, train_loss=2.090, Acc=40.20]
Epoch 1: 69%|██████▊ | 81/118 [00:00<00:00, 88.97it/s, v_num=3, train_loss=2.090, Acc=40.20]
Epoch 1: 69%|██████▊ | 81/118 [00:00<00:00, 88.78it/s, v_num=3, train_loss=2.100, Acc=40.20]
Epoch 1: 69%|██████▉ | 82/118 [00:00<00:00, 89.58it/s, v_num=3, train_loss=2.100, Acc=40.20]
Epoch 1: 69%|██████▉ | 82/118 [00:00<00:00, 89.40it/s, v_num=3, train_loss=2.050, Acc=40.20]
Epoch 1: 70%|███████ | 83/118 [00:00<00:00, 90.15it/s, v_num=3, train_loss=2.050, Acc=40.20]
Epoch 1: 70%|███████ | 83/118 [00:00<00:00, 89.98it/s, v_num=3, train_loss=2.020, Acc=40.20]
Epoch 1: 71%|███████ | 84/118 [00:00<00:00, 90.41it/s, v_num=3, train_loss=2.020, Acc=40.20]
Epoch 1: 71%|███████ | 84/118 [00:00<00:00, 90.26it/s, v_num=3, train_loss=2.010, Acc=40.20]
Epoch 1: 72%|███████▏ | 85/118 [00:00<00:00, 87.82it/s, v_num=3, train_loss=2.010, Acc=40.20]
Epoch 1: 72%|███████▏ | 85/118 [00:00<00:00, 87.65it/s, v_num=3, train_loss=2.020, Acc=40.20]
Epoch 1: 73%|███████▎ | 86/118 [00:00<00:00, 88.42it/s, v_num=3, train_loss=2.020, Acc=40.20]
Epoch 1: 73%|███████▎ | 86/118 [00:00<00:00, 88.24it/s, v_num=3, train_loss=2.010, Acc=40.20]
Epoch 1: 74%|███████▎ | 87/118 [00:00<00:00, 89.01it/s, v_num=3, train_loss=2.010, Acc=40.20]
Epoch 1: 74%|███████▎ | 87/118 [00:00<00:00, 88.83it/s, v_num=3, train_loss=2.030, Acc=40.20]
Epoch 1: 75%|███████▍ | 88/118 [00:00<00:00, 89.60it/s, v_num=3, train_loss=2.030, Acc=40.20]
Epoch 1: 75%|███████▍ | 88/118 [00:00<00:00, 89.42it/s, v_num=3, train_loss=2.030, Acc=40.20]
Epoch 1: 75%|███████▌ | 89/118 [00:00<00:00, 90.18it/s, v_num=3, train_loss=2.030, Acc=40.20]
Epoch 1: 75%|███████▌ | 89/118 [00:00<00:00, 89.99it/s, v_num=3, train_loss=2.070, Acc=40.20]
Epoch 1: 76%|███████▋ | 90/118 [00:00<00:00, 90.59it/s, v_num=3, train_loss=2.070, Acc=40.20]
Epoch 1: 76%|███████▋ | 90/118 [00:00<00:00, 90.44it/s, v_num=3, train_loss=1.970, Acc=40.20]
Epoch 1: 77%|███████▋ | 91/118 [00:00<00:00, 91.03it/s, v_num=3, train_loss=1.970, Acc=40.20]
Epoch 1: 77%|███████▋ | 91/118 [00:01<00:00, 90.87it/s, v_num=3, train_loss=2.030, Acc=40.20]
Epoch 1: 78%|███████▊ | 92/118 [00:01<00:00, 91.42it/s, v_num=3, train_loss=2.030, Acc=40.20]
Epoch 1: 78%|███████▊ | 92/118 [00:01<00:00, 91.28it/s, v_num=3, train_loss=1.990, Acc=40.20]
Epoch 1: 79%|███████▉ | 93/118 [00:01<00:00, 91.58it/s, v_num=3, train_loss=1.990, Acc=40.20]
Epoch 1: 79%|███████▉ | 93/118 [00:01<00:00, 91.44it/s, v_num=3, train_loss=2.050, Acc=40.20]
Epoch 1: 80%|███████▉ | 94/118 [00:01<00:00, 92.00it/s, v_num=3, train_loss=2.050, Acc=40.20]
Epoch 1: 80%|███████▉ | 94/118 [00:01<00:00, 91.84it/s, v_num=3, train_loss=2.120, Acc=40.20]
Epoch 1: 81%|████████ | 95/118 [00:01<00:00, 92.52it/s, v_num=3, train_loss=2.120, Acc=40.20]
Epoch 1: 81%|████████ | 95/118 [00:01<00:00, 92.37it/s, v_num=3, train_loss=2.000, Acc=40.20]
Epoch 1: 81%|████████▏ | 96/118 [00:01<00:00, 93.06it/s, v_num=3, train_loss=2.000, Acc=40.20]
Epoch 1: 81%|████████▏ | 96/118 [00:01<00:00, 92.90it/s, v_num=3, train_loss=1.980, Acc=40.20]
Epoch 1: 82%|████████▏ | 97/118 [00:01<00:00, 93.58it/s, v_num=3, train_loss=1.980, Acc=40.20]
Epoch 1: 82%|████████▏ | 97/118 [00:01<00:00, 93.43it/s, v_num=3, train_loss=1.960, Acc=40.20]
Epoch 1: 83%|████████▎ | 98/118 [00:01<00:00, 94.10it/s, v_num=3, train_loss=1.960, Acc=40.20]
Epoch 1: 83%|████████▎ | 98/118 [00:01<00:00, 93.94it/s, v_num=3, train_loss=1.960, Acc=40.20]
Epoch 1: 84%|████████▍ | 99/118 [00:01<00:00, 94.59it/s, v_num=3, train_loss=1.960, Acc=40.20]
Epoch 1: 84%|████████▍ | 99/118 [00:01<00:00, 94.45it/s, v_num=3, train_loss=1.960, Acc=40.20]
Epoch 1: 85%|████████▍ | 100/118 [00:01<00:00, 94.98it/s, v_num=3, train_loss=1.960, Acc=40.20]
Epoch 1: 85%|████████▍ | 100/118 [00:01<00:00, 94.84it/s, v_num=3, train_loss=2.020, Acc=40.20]
Epoch 1: 86%|████████▌ | 101/118 [00:01<00:00, 94.94it/s, v_num=3, train_loss=2.020, Acc=40.20]
Epoch 1: 86%|████████▌ | 101/118 [00:01<00:00, 94.80it/s, v_num=3, train_loss=2.060, Acc=40.20]
Epoch 1: 86%|████████▋ | 102/118 [00:01<00:00, 94.75it/s, v_num=3, train_loss=2.060, Acc=40.20]
Epoch 1: 86%|████████▋ | 102/118 [00:01<00:00, 94.61it/s, v_num=3, train_loss=2.020, Acc=40.20]
Epoch 1: 87%|████████▋ | 103/118 [00:01<00:00, 95.14it/s, v_num=3, train_loss=2.020, Acc=40.20]
Epoch 1: 87%|████████▋ | 103/118 [00:01<00:00, 94.96it/s, v_num=3, train_loss=1.920, Acc=40.20]
Epoch 1: 88%|████████▊ | 104/118 [00:01<00:00, 95.46it/s, v_num=3, train_loss=1.920, Acc=40.20]
Epoch 1: 88%|████████▊ | 104/118 [00:01<00:00, 95.31it/s, v_num=3, train_loss=1.920, Acc=40.20]
Epoch 1: 89%|████████▉ | 105/118 [00:01<00:00, 95.80it/s, v_num=3, train_loss=1.920, Acc=40.20]
Epoch 1: 89%|████████▉ | 105/118 [00:01<00:00, 95.65it/s, v_num=3, train_loss=1.960, Acc=40.20]
Epoch 1: 90%|████████▉ | 106/118 [00:01<00:00, 96.15it/s, v_num=3, train_loss=1.960, Acc=40.20]
Epoch 1: 90%|████████▉ | 106/118 [00:01<00:00, 96.00it/s, v_num=3, train_loss=1.970, Acc=40.20]
Epoch 1: 91%|█████████ | 107/118 [00:01<00:00, 96.44it/s, v_num=3, train_loss=1.970, Acc=40.20]
Epoch 1: 91%|█████████ | 107/118 [00:01<00:00, 96.28it/s, v_num=3, train_loss=1.970, Acc=40.20]
Epoch 1: 92%|█████████▏| 108/118 [00:01<00:00, 95.85it/s, v_num=3, train_loss=1.970, Acc=40.20]
Epoch 1: 92%|█████████▏| 108/118 [00:01<00:00, 95.69it/s, v_num=3, train_loss=1.970, Acc=40.20]
Epoch 1: 92%|█████████▏| 109/118 [00:01<00:00, 96.22it/s, v_num=3, train_loss=1.970, Acc=40.20]
Epoch 1: 92%|█████████▏| 109/118 [00:01<00:00, 96.07it/s, v_num=3, train_loss=1.930, Acc=40.20]
Epoch 1: 93%|█████████▎| 110/118 [00:01<00:00, 96.73it/s, v_num=3, train_loss=1.930, Acc=40.20]
Epoch 1: 93%|█████████▎| 110/118 [00:01<00:00, 96.53it/s, v_num=3, train_loss=1.980, Acc=40.20]
Epoch 1: 94%|█████████▍| 111/118 [00:01<00:00, 97.20it/s, v_num=3, train_loss=1.980, Acc=40.20]
Epoch 1: 94%|█████████▍| 111/118 [00:01<00:00, 96.99it/s, v_num=3, train_loss=1.970, Acc=40.20]
Epoch 1: 95%|█████████▍| 112/118 [00:01<00:00, 97.70it/s, v_num=3, train_loss=1.970, Acc=40.20]
Epoch 1: 95%|█████████▍| 112/118 [00:01<00:00, 97.48it/s, v_num=3, train_loss=1.920, Acc=40.20]
Epoch 1: 96%|█████████▌| 113/118 [00:01<00:00, 98.19it/s, v_num=3, train_loss=1.920, Acc=40.20]
Epoch 1: 96%|█████████▌| 113/118 [00:01<00:00, 97.97it/s, v_num=3, train_loss=1.930, Acc=40.20]
Epoch 1: 97%|█████████▋| 114/118 [00:01<00:00, 98.68it/s, v_num=3, train_loss=1.930, Acc=40.20]
Epoch 1: 97%|█████████▋| 114/118 [00:01<00:00, 98.46it/s, v_num=3, train_loss=1.970, Acc=40.20]
Epoch 1: 97%|█████████▋| 115/118 [00:01<00:00, 99.16it/s, v_num=3, train_loss=1.970, Acc=40.20]
Epoch 1: 97%|█████████▋| 115/118 [00:01<00:00, 98.94it/s, v_num=3, train_loss=1.960, Acc=40.20]
Epoch 1: 98%|█████████▊| 116/118 [00:01<00:00, 99.59it/s, v_num=3, train_loss=1.960, Acc=40.20]
Epoch 1: 98%|█████████▊| 116/118 [00:01<00:00, 99.37it/s, v_num=3, train_loss=1.910, Acc=40.20]
Epoch 1: 99%|█████████▉| 117/118 [00:01<00:00, 100.07it/s, v_num=3, train_loss=1.910, Acc=40.20]
Epoch 1: 99%|█████████▉| 117/118 [00:01<00:00, 99.85it/s, v_num=3, train_loss=1.870, Acc=40.20]
Epoch 1: 100%|██████████| 118/118 [00:01<00:00, 100.56it/s, v_num=3, train_loss=1.870, Acc=40.20]
Epoch 1: 100%|██████████| 118/118 [00:01<00:00, 100.53it/s, v_num=3, train_loss=1.800, Acc=40.20]
Validation: | | 0/? [00:00<?, ?it/s]
Validation: 0%| | 0/5 [00:00<?, ?it/s]
Validation DataLoader 0: 0%| | 0/5 [00:00<?, ?it/s]
Validation DataLoader 0: 20%|██ | 1/5 [00:00<00:00, 453.10it/s]
Validation DataLoader 0: 40%|████ | 2/5 [00:00<00:00, 143.87it/s]
Validation DataLoader 0: 60%|██████ | 3/5 [00:00<00:00, 135.54it/s]
Validation DataLoader 0: 80%|████████ | 4/5 [00:00<00:00, 150.53it/s]
Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 53.16it/s]
Epoch 1: 100%|██████████| 118/118 [00:01<00:00, 79.08it/s, v_num=3, train_loss=1.800, Acc=48.80]
Epoch 1: 100%|██████████| 118/118 [00:01<00:00, 79.03it/s, v_num=3, train_loss=1.800, Acc=48.80]
Epoch 1: 0%| | 0/118 [00:00<?, ?it/s, v_num=3, train_loss=1.800, Acc=48.80]
Epoch 2: 0%| | 0/118 [00:00<?, ?it/s, v_num=3, train_loss=1.800, Acc=48.80]
Epoch 2: 1%| | 1/118 [00:00<00:29, 3.92it/s, v_num=3, train_loss=1.800, Acc=48.80]
Epoch 2: 1%| | 1/118 [00:00<00:30, 3.90it/s, v_num=3, train_loss=2.040, Acc=48.80]
Epoch 2: 2%|▏ | 2/118 [00:00<00:17, 6.76it/s, v_num=3, train_loss=2.040, Acc=48.80]
Epoch 2: 2%|▏ | 2/118 [00:00<00:17, 6.70it/s, v_num=3, train_loss=2.050, Acc=48.80]
Epoch 2: 3%|▎ | 3/118 [00:00<00:11, 9.97it/s, v_num=3, train_loss=2.050, Acc=48.80]
Epoch 2: 3%|▎ | 3/118 [00:00<00:11, 9.89it/s, v_num=3, train_loss=1.980, Acc=48.80]
Epoch 2: 3%|▎ | 4/118 [00:00<00:08, 13.09it/s, v_num=3, train_loss=1.980, Acc=48.80]
Epoch 2: 3%|▎ | 4/118 [00:00<00:08, 12.99it/s, v_num=3, train_loss=1.950, Acc=48.80]
Epoch 2: 4%|▍ | 5/118 [00:00<00:07, 16.06it/s, v_num=3, train_loss=1.950, Acc=48.80]
Epoch 2: 4%|▍ | 5/118 [00:00<00:07, 15.94it/s, v_num=3, train_loss=1.890, Acc=48.80]
Epoch 2: 5%|▌ | 6/118 [00:00<00:05, 18.99it/s, v_num=3, train_loss=1.890, Acc=48.80]
Epoch 2: 5%|▌ | 6/118 [00:00<00:05, 18.84it/s, v_num=3, train_loss=1.890, Acc=48.80]
Epoch 2: 6%|▌ | 7/118 [00:00<00:05, 21.76it/s, v_num=3, train_loss=1.890, Acc=48.80]
Epoch 2: 6%|▌ | 7/118 [00:00<00:05, 21.65it/s, v_num=3, train_loss=1.910, Acc=48.80]
Epoch 2: 7%|▋ | 8/118 [00:00<00:04, 22.75it/s, v_num=3, train_loss=1.910, Acc=48.80]
Epoch 2: 7%|▋ | 8/118 [00:00<00:04, 22.60it/s, v_num=3, train_loss=1.880, Acc=48.80]
Epoch 2: 8%|▊ | 9/118 [00:00<00:04, 25.12it/s, v_num=3, train_loss=1.880, Acc=48.80]
Epoch 2: 8%|▊ | 9/118 [00:00<00:04, 25.00it/s, v_num=3, train_loss=1.880, Acc=48.80]
Epoch 2: 8%|▊ | 10/118 [00:00<00:03, 27.42it/s, v_num=3, train_loss=1.880, Acc=48.80]
Epoch 2: 8%|▊ | 10/118 [00:00<00:03, 27.27it/s, v_num=3, train_loss=1.850, Acc=48.80]
Epoch 2: 9%|▉ | 11/118 [00:00<00:03, 29.76it/s, v_num=3, train_loss=1.850, Acc=48.80]
Epoch 2: 9%|▉ | 11/118 [00:00<00:03, 29.61it/s, v_num=3, train_loss=1.850, Acc=48.80]
Epoch 2: 10%|█ | 12/118 [00:00<00:03, 32.04it/s, v_num=3, train_loss=1.850, Acc=48.80]
Epoch 2: 10%|█ | 12/118 [00:00<00:03, 31.88it/s, v_num=3, train_loss=1.880, Acc=48.80]
Epoch 2: 11%|█ | 13/118 [00:00<00:03, 33.91it/s, v_num=3, train_loss=1.880, Acc=48.80]
Epoch 2: 11%|█ | 13/118 [00:00<00:03, 33.77it/s, v_num=3, train_loss=1.880, Acc=48.80]
Epoch 2: 12%|█▏ | 14/118 [00:00<00:02, 35.98it/s, v_num=3, train_loss=1.880, Acc=48.80]
Epoch 2: 12%|█▏ | 14/118 [00:00<00:02, 35.83it/s, v_num=3, train_loss=1.890, Acc=48.80]
Epoch 2: 13%|█▎ | 15/118 [00:00<00:02, 37.99it/s, v_num=3, train_loss=1.890, Acc=48.80]
Epoch 2: 13%|█▎ | 15/118 [00:00<00:02, 37.82it/s, v_num=3, train_loss=1.940, Acc=48.80]
Epoch 2: 14%|█▎ | 16/118 [00:00<00:02, 35.45it/s, v_num=3, train_loss=1.940, Acc=48.80]
Epoch 2: 14%|█▎ | 16/118 [00:00<00:02, 35.31it/s, v_num=3, train_loss=1.920, Acc=48.80]
Epoch 2: 14%|█▍ | 17/118 [00:00<00:02, 37.28it/s, v_num=3, train_loss=1.920, Acc=48.80]
Epoch 2: 14%|█▍ | 17/118 [00:00<00:02, 37.12it/s, v_num=3, train_loss=1.870, Acc=48.80]
Epoch 2: 15%|█▌ | 18/118 [00:00<00:02, 39.07it/s, v_num=3, train_loss=1.870, Acc=48.80]
Epoch 2: 15%|█▌ | 18/118 [00:00<00:02, 38.91it/s, v_num=3, train_loss=1.780, Acc=48.80]
Epoch 2: 16%|█▌ | 19/118 [00:00<00:02, 40.87it/s, v_num=3, train_loss=1.780, Acc=48.80]
Epoch 2: 16%|█▌ | 19/118 [00:00<00:02, 40.66it/s, v_num=3, train_loss=1.790, Acc=48.80]
Epoch 2: 17%|█▋ | 20/118 [00:00<00:02, 42.57it/s, v_num=3, train_loss=1.790, Acc=48.80]
Epoch 2: 17%|█▋ | 20/118 [00:00<00:02, 42.38it/s, v_num=3, train_loss=1.840, Acc=48.80]
Epoch 2: 18%|█▊ | 21/118 [00:00<00:02, 44.26it/s, v_num=3, train_loss=1.840, Acc=48.80]
Epoch 2: 18%|█▊ | 21/118 [00:00<00:02, 44.07it/s, v_num=3, train_loss=1.840, Acc=48.80]
Epoch 2: 19%|█▊ | 22/118 [00:00<00:02, 45.91it/s, v_num=3, train_loss=1.840, Acc=48.80]
Epoch 2: 19%|█▊ | 22/118 [00:00<00:02, 45.71it/s, v_num=3, train_loss=1.840, Acc=48.80]
Epoch 2: 19%|█▉ | 23/118 [00:00<00:02, 47.47it/s, v_num=3, train_loss=1.840, Acc=48.80]
Epoch 2: 19%|█▉ | 23/118 [00:00<00:02, 47.32it/s, v_num=3, train_loss=1.730, Acc=48.80]
Epoch 2: 20%|██ | 24/118 [00:00<00:01, 48.77it/s, v_num=3, train_loss=1.730, Acc=48.80]
Epoch 2: 20%|██ | 24/118 [00:00<00:01, 48.61it/s, v_num=3, train_loss=1.800, Acc=48.80]
Epoch 2: 21%|██ | 25/118 [00:00<00:01, 50.14it/s, v_num=3, train_loss=1.800, Acc=48.80]
Epoch 2: 21%|██ | 25/118 [00:00<00:01, 49.97it/s, v_num=3, train_loss=1.770, Acc=48.80]
Epoch 2: 22%|██▏ | 26/118 [00:00<00:01, 51.45it/s, v_num=3, train_loss=1.770, Acc=48.80]
Epoch 2: 22%|██▏ | 26/118 [00:00<00:01, 51.29it/s, v_num=3, train_loss=1.780, Acc=48.80]
Epoch 2: 23%|██▎ | 27/118 [00:00<00:01, 52.75it/s, v_num=3, train_loss=1.780, Acc=48.80]
Epoch 2: 23%|██▎ | 27/118 [00:00<00:01, 52.58it/s, v_num=3, train_loss=1.880, Acc=48.80]
Epoch 2: 24%|██▎ | 28/118 [00:00<00:01, 54.01it/s, v_num=3, train_loss=1.880, Acc=48.80]
Epoch 2: 24%|██▎ | 28/118 [00:00<00:01, 53.84it/s, v_num=3, train_loss=1.910, Acc=48.80]
Epoch 2: 25%|██▍ | 29/118 [00:00<00:01, 55.39it/s, v_num=3, train_loss=1.910, Acc=48.80]
Epoch 2: 25%|██▍ | 29/118 [00:00<00:01, 55.20it/s, v_num=3, train_loss=1.850, Acc=48.80]
Epoch 2: 25%|██▌ | 30/118 [00:00<00:01, 56.79it/s, v_num=3, train_loss=1.850, Acc=48.80]
Epoch 2: 25%|██▌ | 30/118 [00:00<00:01, 56.58it/s, v_num=3, train_loss=1.790, Acc=48.80]
Epoch 2: 26%|██▋ | 31/118 [00:00<00:01, 58.12it/s, v_num=3, train_loss=1.790, Acc=48.80]
Epoch 2: 26%|██▋ | 31/118 [00:00<00:01, 57.94it/s, v_num=3, train_loss=1.840, Acc=48.80]
Epoch 2: 27%|██▋ | 32/118 [00:00<00:01, 58.84it/s, v_num=3, train_loss=1.840, Acc=48.80]
Epoch 2: 27%|██▋ | 32/118 [00:00<00:01, 58.66it/s, v_num=3, train_loss=1.820, Acc=48.80]
Epoch 2: 28%|██▊ | 33/118 [00:00<00:01, 60.15it/s, v_num=3, train_loss=1.820, Acc=48.80]
Epoch 2: 28%|██▊ | 33/118 [00:00<00:01, 59.95it/s, v_num=3, train_loss=1.800, Acc=48.80]
Epoch 2: 29%|██▉ | 34/118 [00:00<00:01, 61.43it/s, v_num=3, train_loss=1.800, Acc=48.80]
Epoch 2: 29%|██▉ | 34/118 [00:00<00:01, 61.23it/s, v_num=3, train_loss=1.800, Acc=48.80]
Epoch 2: 30%|██▉ | 35/118 [00:00<00:01, 62.66it/s, v_num=3, train_loss=1.800, Acc=48.80]
Epoch 2: 30%|██▉ | 35/118 [00:00<00:01, 62.47it/s, v_num=3, train_loss=1.780, Acc=48.80]
Epoch 2: 31%|███ | 36/118 [00:00<00:01, 63.85it/s, v_num=3, train_loss=1.780, Acc=48.80]
Epoch 2: 31%|███ | 36/118 [00:00<00:01, 63.68it/s, v_num=3, train_loss=1.770, Acc=48.80]
Epoch 2: 31%|███▏ | 37/118 [00:00<00:01, 64.50it/s, v_num=3, train_loss=1.770, Acc=48.80]
Epoch 2: 31%|███▏ | 37/118 [00:00<00:01, 64.32it/s, v_num=3, train_loss=1.750, Acc=48.80]
Epoch 2: 32%|███▏ | 38/118 [00:00<00:01, 61.80it/s, v_num=3, train_loss=1.750, Acc=48.80]
Epoch 2: 32%|███▏ | 38/118 [00:00<00:01, 61.62it/s, v_num=3, train_loss=1.760, Acc=48.80]
Epoch 2: 33%|███▎ | 39/118 [00:00<00:01, 62.94it/s, v_num=3, train_loss=1.760, Acc=48.80]
Epoch 2: 33%|███▎ | 39/118 [00:00<00:01, 62.74it/s, v_num=3, train_loss=1.740, Acc=48.80]
Epoch 2: 34%|███▍ | 40/118 [00:00<00:01, 64.11it/s, v_num=3, train_loss=1.740, Acc=48.80]
Epoch 2: 34%|███▍ | 40/118 [00:00<00:01, 63.85it/s, v_num=3, train_loss=1.750, Acc=48.80]
Epoch 2: 35%|███▍ | 41/118 [00:00<00:01, 65.22it/s, v_num=3, train_loss=1.750, Acc=48.80]
Epoch 2: 35%|███▍ | 41/118 [00:00<00:01, 64.97it/s, v_num=3, train_loss=1.770, Acc=48.80]
Epoch 2: 36%|███▌ | 42/118 [00:00<00:01, 66.30it/s, v_num=3, train_loss=1.770, Acc=48.80]
Epoch 2: 36%|███▌ | 42/118 [00:00<00:01, 66.05it/s, v_num=3, train_loss=1.700, Acc=48.80]
Epoch 2: 36%|███▋ | 43/118 [00:00<00:01, 67.37it/s, v_num=3, train_loss=1.700, Acc=48.80]
Epoch 2: 36%|███▋ | 43/118 [00:00<00:01, 67.12it/s, v_num=3, train_loss=1.710, Acc=48.80]
Epoch 2: 37%|███▋ | 44/118 [00:00<00:01, 68.37it/s, v_num=3, train_loss=1.710, Acc=48.80]
Epoch 2: 37%|███▋ | 44/118 [00:00<00:01, 68.15it/s, v_num=3, train_loss=1.710, Acc=48.80]
Epoch 2: 38%|███▊ | 45/118 [00:00<00:01, 69.24it/s, v_num=3, train_loss=1.710, Acc=48.80]
Epoch 2: 38%|███▊ | 45/118 [00:00<00:01, 69.00it/s, v_num=3, train_loss=1.770, Acc=48.80]
Epoch 2: 39%|███▉ | 46/118 [00:00<00:01, 69.75it/s, v_num=3, train_loss=1.770, Acc=48.80]
Epoch 2: 39%|███▉ | 46/118 [00:00<00:01, 69.52it/s, v_num=3, train_loss=1.770, Acc=48.80]
Epoch 2: 40%|███▉ | 47/118 [00:00<00:01, 70.59it/s, v_num=3, train_loss=1.770, Acc=48.80]
Epoch 2: 40%|███▉ | 47/118 [00:00<00:01, 70.35it/s, v_num=3, train_loss=1.800, Acc=48.80]
Epoch 2: 41%|████ | 48/118 [00:00<00:00, 71.39it/s, v_num=3, train_loss=1.800, Acc=48.80]
Epoch 2: 41%|████ | 48/118 [00:00<00:00, 71.15it/s, v_num=3, train_loss=1.780, Acc=48.80]
Epoch 2: 42%|████▏ | 49/118 [00:00<00:00, 72.16it/s, v_num=3, train_loss=1.780, Acc=48.80]
Epoch 2: 42%|████▏ | 49/118 [00:00<00:00, 71.92it/s, v_num=3, train_loss=1.770, Acc=48.80]
Epoch 2: 42%|████▏ | 50/118 [00:00<00:00, 72.93it/s, v_num=3, train_loss=1.770, Acc=48.80]
Epoch 2: 42%|████▏ | 50/118 [00:00<00:00, 72.70it/s, v_num=3, train_loss=1.640, Acc=48.80]
Epoch 2: 43%|████▎ | 51/118 [00:00<00:00, 73.69it/s, v_num=3, train_loss=1.640, Acc=48.80]
Epoch 2: 43%|████▎ | 51/118 [00:00<00:00, 73.45it/s, v_num=3, train_loss=1.670, Acc=48.80]
Epoch 2: 44%|████▍ | 52/118 [00:00<00:00, 74.43it/s, v_num=3, train_loss=1.670, Acc=48.80]
Epoch 2: 44%|████▍ | 52/118 [00:00<00:00, 74.19it/s, v_num=3, train_loss=1.650, Acc=48.80]
Epoch 2: 45%|████▍ | 53/118 [00:00<00:00, 75.08it/s, v_num=3, train_loss=1.650, Acc=48.80]
Epoch 2: 45%|████▍ | 53/118 [00:00<00:00, 74.92it/s, v_num=3, train_loss=1.680, Acc=48.80]
Epoch 2: 46%|████▌ | 54/118 [00:00<00:00, 75.52it/s, v_num=3, train_loss=1.680, Acc=48.80]
Epoch 2: 46%|████▌ | 54/118 [00:00<00:00, 75.34it/s, v_num=3, train_loss=1.750, Acc=48.80]
Epoch 2: 47%|████▋ | 55/118 [00:00<00:00, 76.41it/s, v_num=3, train_loss=1.750, Acc=48.80]
Epoch 2: 47%|████▋ | 55/118 [00:00<00:00, 76.21it/s, v_num=3, train_loss=1.820, Acc=48.80]
Epoch 2: 47%|████▋ | 56/118 [00:00<00:00, 77.27it/s, v_num=3, train_loss=1.820, Acc=48.80]
Epoch 2: 47%|████▋ | 56/118 [00:00<00:00, 77.08it/s, v_num=3, train_loss=1.850, Acc=48.80]
Epoch 2: 48%|████▊ | 57/118 [00:00<00:00, 78.11it/s, v_num=3, train_loss=1.850, Acc=48.80]
Epoch 2: 48%|████▊ | 57/118 [00:00<00:00, 77.93it/s, v_num=3, train_loss=1.760, Acc=48.80]
Epoch 2: 49%|████▉ | 58/118 [00:00<00:00, 78.92it/s, v_num=3, train_loss=1.760, Acc=48.80]
Epoch 2: 49%|████▉ | 58/118 [00:00<00:00, 78.76it/s, v_num=3, train_loss=1.670, Acc=48.80]
Epoch 2: 50%|█████ | 59/118 [00:00<00:00, 79.01it/s, v_num=3, train_loss=1.670, Acc=48.80]
Epoch 2: 50%|█████ | 59/118 [00:00<00:00, 78.85it/s, v_num=3, train_loss=1.590, Acc=48.80]
Epoch 2: 51%|█████ | 60/118 [00:00<00:00, 76.70it/s, v_num=3, train_loss=1.590, Acc=48.80]
Epoch 2: 51%|█████ | 60/118 [00:00<00:00, 76.54it/s, v_num=3, train_loss=1.590, Acc=48.80]
Epoch 2: 52%|█████▏ | 61/118 [00:00<00:00, 77.36it/s, v_num=3, train_loss=1.590, Acc=48.80]
Epoch 2: 52%|█████▏ | 61/118 [00:00<00:00, 77.19it/s, v_num=3, train_loss=1.580, Acc=48.80]
Epoch 2: 53%|█████▎ | 62/118 [00:00<00:00, 78.16it/s, v_num=3, train_loss=1.580, Acc=48.80]
Epoch 2: 53%|█████▎ | 62/118 [00:00<00:00, 77.97it/s, v_num=3, train_loss=1.560, Acc=48.80]
Epoch 2: 53%|█████▎ | 63/118 [00:00<00:00, 78.93it/s, v_num=3, train_loss=1.560, Acc=48.80]
Epoch 2: 53%|█████▎ | 63/118 [00:00<00:00, 78.75it/s, v_num=3, train_loss=1.640, Acc=48.80]
Epoch 2: 54%|█████▍ | 64/118 [00:00<00:00, 79.70it/s, v_num=3, train_loss=1.640, Acc=48.80]
Epoch 2: 54%|█████▍ | 64/118 [00:00<00:00, 79.51it/s, v_num=3, train_loss=1.780, Acc=48.80]
Epoch 2: 55%|█████▌ | 65/118 [00:00<00:00, 80.42it/s, v_num=3, train_loss=1.780, Acc=48.80]
Epoch 2: 55%|█████▌ | 65/118 [00:00<00:00, 80.24it/s, v_num=3, train_loss=1.720, Acc=48.80]
Epoch 2: 56%|█████▌ | 66/118 [00:00<00:00, 81.15it/s, v_num=3, train_loss=1.720, Acc=48.80]
Epoch 2: 56%|█████▌ | 66/118 [00:00<00:00, 80.99it/s, v_num=3, train_loss=1.760, Acc=48.80]
Epoch 2: 57%|█████▋ | 67/118 [00:00<00:00, 81.61it/s, v_num=3, train_loss=1.760, Acc=48.80]
Epoch 2: 57%|█████▋ | 67/118 [00:00<00:00, 81.46it/s, v_num=3, train_loss=1.610, Acc=48.80]
Epoch 2: 58%|█████▊ | 68/118 [00:00<00:00, 81.93it/s, v_num=3, train_loss=1.610, Acc=48.80]
Epoch 2: 58%|█████▊ | 68/118 [00:00<00:00, 81.77it/s, v_num=3, train_loss=1.610, Acc=48.80]
Epoch 2: 58%|█████▊ | 69/118 [00:00<00:00, 82.49it/s, v_num=3, train_loss=1.610, Acc=48.80]
Epoch 2: 58%|█████▊ | 69/118 [00:00<00:00, 82.32it/s, v_num=3, train_loss=1.730, Acc=48.80]
Epoch 2: 59%|█████▉ | 70/118 [00:00<00:00, 83.03it/s, v_num=3, train_loss=1.730, Acc=48.80]
Epoch 2: 59%|█████▉ | 70/118 [00:00<00:00, 82.88it/s, v_num=3, train_loss=1.750, Acc=48.80]
Epoch 2: 60%|██████ | 71/118 [00:00<00:00, 83.57it/s, v_num=3, train_loss=1.750, Acc=48.80]
Epoch 2: 60%|██████ | 71/118 [00:00<00:00, 83.41it/s, v_num=3, train_loss=1.670, Acc=48.80]
Epoch 2: 61%|██████ | 72/118 [00:00<00:00, 84.27it/s, v_num=3, train_loss=1.670, Acc=48.80]
Epoch 2: 61%|██████ | 72/118 [00:00<00:00, 84.10it/s, v_num=3, train_loss=1.630, Acc=48.80]
Epoch 2: 62%|██████▏ | 73/118 [00:00<00:00, 84.95it/s, v_num=3, train_loss=1.630, Acc=48.80]
Epoch 2: 62%|██████▏ | 73/118 [00:00<00:00, 84.78it/s, v_num=3, train_loss=1.670, Acc=48.80]
Epoch 2: 63%|██████▎ | 74/118 [00:00<00:00, 85.62it/s, v_num=3, train_loss=1.670, Acc=48.80]
Epoch 2: 63%|██████▎ | 74/118 [00:00<00:00, 85.45it/s, v_num=3, train_loss=1.650, Acc=48.80]
Epoch 2: 64%|██████▎ | 75/118 [00:00<00:00, 86.28it/s, v_num=3, train_loss=1.650, Acc=48.80]
Epoch 2: 64%|██████▎ | 75/118 [00:00<00:00, 86.12it/s, v_num=3, train_loss=1.620, Acc=48.80]
Epoch 2: 64%|██████▍ | 76/118 [00:00<00:00, 86.76it/s, v_num=3, train_loss=1.620, Acc=48.80]
Epoch 2: 64%|██████▍ | 76/118 [00:00<00:00, 86.59it/s, v_num=3, train_loss=1.560, Acc=48.80]
Epoch 2: 65%|██████▌ | 77/118 [00:00<00:00, 87.43it/s, v_num=3, train_loss=1.560, Acc=48.80]
Epoch 2: 65%|██████▌ | 77/118 [00:00<00:00, 87.25it/s, v_num=3, train_loss=1.620, Acc=48.80]
Epoch 2: 66%|██████▌ | 78/118 [00:00<00:00, 88.07it/s, v_num=3, train_loss=1.620, Acc=48.80]
Epoch 2: 66%|██████▌ | 78/118 [00:00<00:00, 87.88it/s, v_num=3, train_loss=1.580, Acc=48.80]
Epoch 2: 67%|██████▋ | 79/118 [00:00<00:00, 86.72it/s, v_num=3, train_loss=1.580, Acc=48.80]
Epoch 2: 67%|██████▋ | 79/118 [00:00<00:00, 86.57it/s, v_num=3, train_loss=1.530, Acc=48.80]
Epoch 2: 68%|██████▊ | 80/118 [00:00<00:00, 87.35it/s, v_num=3, train_loss=1.530, Acc=48.80]
Epoch 2: 68%|██████▊ | 80/118 [00:00<00:00, 87.19it/s, v_num=3, train_loss=1.540, Acc=48.80]
Epoch 2: 69%|██████▊ | 81/118 [00:00<00:00, 87.12it/s, v_num=3, train_loss=1.540, Acc=48.80]
Epoch 2: 69%|██████▊ | 81/118 [00:00<00:00, 86.96it/s, v_num=3, train_loss=1.550, Acc=48.80]
Epoch 2: 69%|██████▉ | 82/118 [00:00<00:00, 87.65it/s, v_num=3, train_loss=1.550, Acc=48.80]
Epoch 2: 69%|██████▉ | 82/118 [00:00<00:00, 87.44it/s, v_num=3, train_loss=1.570, Acc=48.80]
Epoch 2: 70%|███████ | 83/118 [00:00<00:00, 88.28it/s, v_num=3, train_loss=1.570, Acc=48.80]
Epoch 2: 70%|███████ | 83/118 [00:00<00:00, 88.05it/s, v_num=3, train_loss=1.600, Acc=48.80]
Epoch 2: 71%|███████ | 84/118 [00:00<00:00, 88.89it/s, v_num=3, train_loss=1.600, Acc=48.80]
Epoch 2: 71%|███████ | 84/118 [00:00<00:00, 88.66it/s, v_num=3, train_loss=1.660, Acc=48.80]
Epoch 2: 72%|███████▏ | 85/118 [00:00<00:00, 89.50it/s, v_num=3, train_loss=1.660, Acc=48.80]
Epoch 2: 72%|███████▏ | 85/118 [00:00<00:00, 89.28it/s, v_num=3, train_loss=1.580, Acc=48.80]
Epoch 2: 73%|███████▎ | 86/118 [00:00<00:00, 90.10it/s, v_num=3, train_loss=1.580, Acc=48.80]
Epoch 2: 73%|███████▎ | 86/118 [00:00<00:00, 89.87it/s, v_num=3, train_loss=1.590, Acc=48.80]
Epoch 2: 74%|███████▎ | 87/118 [00:00<00:00, 90.36it/s, v_num=3, train_loss=1.590, Acc=48.80]
Epoch 2: 74%|███████▎ | 87/118 [00:00<00:00, 90.15it/s, v_num=3, train_loss=1.580, Acc=48.80]
Epoch 2: 75%|███████▍ | 88/118 [00:00<00:00, 90.77it/s, v_num=3, train_loss=1.580, Acc=48.80]
Epoch 2: 75%|███████▍ | 88/118 [00:00<00:00, 90.56it/s, v_num=3, train_loss=1.610, Acc=48.80]
Epoch 2: 75%|███████▌ | 89/118 [00:00<00:00, 90.98it/s, v_num=3, train_loss=1.610, Acc=48.80]
Epoch 2: 75%|███████▌ | 89/118 [00:00<00:00, 90.77it/s, v_num=3, train_loss=1.550, Acc=48.80]
Epoch 2: 76%|███████▋ | 90/118 [00:00<00:00, 91.33it/s, v_num=3, train_loss=1.550, Acc=48.80]
Epoch 2: 76%|███████▋ | 90/118 [00:00<00:00, 91.18it/s, v_num=3, train_loss=1.550, Acc=48.80]
Epoch 2: 77%|███████▋ | 91/118 [00:00<00:00, 91.73it/s, v_num=3, train_loss=1.550, Acc=48.80]
Epoch 2: 77%|███████▋ | 91/118 [00:00<00:00, 91.59it/s, v_num=3, train_loss=1.470, Acc=48.80]
Epoch 2: 78%|███████▊ | 92/118 [00:00<00:00, 92.15it/s, v_num=3, train_loss=1.470, Acc=48.80]
Epoch 2: 78%|███████▊ | 92/118 [00:01<00:00, 91.99it/s, v_num=3, train_loss=1.420, Acc=48.80]
Epoch 2: 79%|███████▉ | 93/118 [00:01<00:00, 92.56it/s, v_num=3, train_loss=1.420, Acc=48.80]
Epoch 2: 79%|███████▉ | 93/118 [00:01<00:00, 92.41it/s, v_num=3, train_loss=1.420, Acc=48.80]
Epoch 2: 80%|███████▉ | 94/118 [00:01<00:00, 92.93it/s, v_num=3, train_loss=1.420, Acc=48.80]
Epoch 2: 80%|███████▉ | 94/118 [00:01<00:00, 92.79it/s, v_num=3, train_loss=1.560, Acc=48.80]
Epoch 2: 81%|████████ | 95/118 [00:01<00:00, 93.17it/s, v_num=3, train_loss=1.560, Acc=48.80]
Epoch 2: 81%|████████ | 95/118 [00:01<00:00, 93.02it/s, v_num=3, train_loss=1.540, Acc=48.80]
Epoch 2: 81%|████████▏ | 96/118 [00:01<00:00, 93.53it/s, v_num=3, train_loss=1.540, Acc=48.80]
Epoch 2: 81%|████████▏ | 96/118 [00:01<00:00, 93.38it/s, v_num=3, train_loss=1.670, Acc=48.80]
Epoch 2: 82%|████████▏ | 97/118 [00:01<00:00, 93.69it/s, v_num=3, train_loss=1.670, Acc=48.80]
Epoch 2: 82%|████████▏ | 97/118 [00:01<00:00, 93.54it/s, v_num=3, train_loss=1.640, Acc=48.80]
Epoch 2: 83%|████████▎ | 98/118 [00:01<00:00, 94.07it/s, v_num=3, train_loss=1.640, Acc=48.80]
Epoch 2: 83%|████████▎ | 98/118 [00:01<00:00, 93.92it/s, v_num=3, train_loss=1.540, Acc=48.80]
Epoch 2: 84%|████████▍ | 99/118 [00:01<00:00, 94.37it/s, v_num=3, train_loss=1.540, Acc=48.80]
Epoch 2: 84%|████████▍ | 99/118 [00:01<00:00, 94.22it/s, v_num=3, train_loss=1.490, Acc=48.80]
Epoch 2: 85%|████████▍ | 100/118 [00:01<00:00, 94.58it/s, v_num=3, train_loss=1.490, Acc=48.80]
Epoch 2: 85%|████████▍ | 100/118 [00:01<00:00, 94.43it/s, v_num=3, train_loss=1.450, Acc=48.80]
Epoch 2: 86%|████████▌ | 101/118 [00:01<00:00, 94.94it/s, v_num=3, train_loss=1.450, Acc=48.80]
Epoch 2: 86%|████████▌ | 101/118 [00:01<00:00, 94.79it/s, v_num=3, train_loss=1.470, Acc=48.80]
Epoch 2: 86%|████████▋ | 102/118 [00:01<00:00, 94.98it/s, v_num=3, train_loss=1.470, Acc=48.80]
Epoch 2: 86%|████████▋ | 102/118 [00:01<00:00, 94.83it/s, v_num=3, train_loss=1.390, Acc=48.80]
Epoch 2: 87%|████████▋ | 103/118 [00:01<00:00, 95.24it/s, v_num=3, train_loss=1.390, Acc=48.80]
Epoch 2: 87%|████████▋ | 103/118 [00:01<00:00, 95.10it/s, v_num=3, train_loss=1.440, Acc=48.80]
Epoch 2: 88%|████████▊ | 104/118 [00:01<00:00, 95.73it/s, v_num=3, train_loss=1.440, Acc=48.80]
Epoch 2: 88%|████████▊ | 104/118 [00:01<00:00, 95.59it/s, v_num=3, train_loss=1.380, Acc=48.80]
Epoch 2: 89%|████████▉ | 105/118 [00:01<00:00, 96.03it/s, v_num=3, train_loss=1.380, Acc=48.80]
Epoch 2: 89%|████████▉ | 105/118 [00:01<00:00, 95.83it/s, v_num=3, train_loss=1.470, Acc=48.80]
Epoch 2: 90%|████████▉ | 106/118 [00:01<00:00, 96.41it/s, v_num=3, train_loss=1.470, Acc=48.80]
Epoch 2: 90%|████████▉ | 106/118 [00:01<00:00, 96.20it/s, v_num=3, train_loss=1.450, Acc=48.80]
Epoch 2: 91%|█████████ | 107/118 [00:01<00:00, 96.74it/s, v_num=3, train_loss=1.450, Acc=48.80]
Epoch 2: 91%|█████████ | 107/118 [00:01<00:00, 96.53it/s, v_num=3, train_loss=1.420, Acc=48.80]
Epoch 2: 92%|█████████▏| 108/118 [00:01<00:00, 96.88it/s, v_num=3, train_loss=1.420, Acc=48.80]
Epoch 2: 92%|█████████▏| 108/118 [00:01<00:00, 96.67it/s, v_num=3, train_loss=1.430, Acc=48.80]
Epoch 2: 92%|█████████▏| 109/118 [00:01<00:00, 97.39it/s, v_num=3, train_loss=1.430, Acc=48.80]
Epoch 2: 92%|█████████▏| 109/118 [00:01<00:00, 97.17it/s, v_num=3, train_loss=1.530, Acc=48.80]
Epoch 2: 93%|█████████▎| 110/118 [00:01<00:00, 97.87it/s, v_num=3, train_loss=1.530, Acc=48.80]
Epoch 2: 93%|█████████▎| 110/118 [00:01<00:00, 97.66it/s, v_num=3, train_loss=1.630, Acc=48.80]
Epoch 2: 94%|█████████▍| 111/118 [00:01<00:00, 98.36it/s, v_num=3, train_loss=1.630, Acc=48.80]
Epoch 2: 94%|█████████▍| 111/118 [00:01<00:00, 98.15it/s, v_num=3, train_loss=1.440, Acc=48.80]
Epoch 2: 95%|█████████▍| 112/118 [00:01<00:00, 98.85it/s, v_num=3, train_loss=1.440, Acc=48.80]
Epoch 2: 95%|█████████▍| 112/118 [00:01<00:00, 98.63it/s, v_num=3, train_loss=1.390, Acc=48.80]
Epoch 2: 96%|█████████▌| 113/118 [00:01<00:00, 99.32it/s, v_num=3, train_loss=1.390, Acc=48.80]
Epoch 2: 96%|█████████▌| 113/118 [00:01<00:00, 99.12it/s, v_num=3, train_loss=1.480, Acc=48.80]
Epoch 2: 97%|█████████▋| 114/118 [00:01<00:00, 99.81it/s, v_num=3, train_loss=1.480, Acc=48.80]
Epoch 2: 97%|█████████▋| 114/118 [00:01<00:00, 99.60it/s, v_num=3, train_loss=1.460, Acc=48.80]
Epoch 2: 97%|█████████▋| 115/118 [00:01<00:00, 100.20it/s, v_num=3, train_loss=1.460, Acc=48.80]
Epoch 2: 97%|█████████▋| 115/118 [00:01<00:00, 99.99it/s, v_num=3, train_loss=1.430, Acc=48.80]
Epoch 2: 98%|█████████▊| 116/118 [00:01<00:00, 100.65it/s, v_num=3, train_loss=1.430, Acc=48.80]
Epoch 2: 98%|█████████▊| 116/118 [00:01<00:00, 100.44it/s, v_num=3, train_loss=1.420, Acc=48.80]
Epoch 2: 99%|█████████▉| 117/118 [00:01<00:00, 101.13it/s, v_num=3, train_loss=1.420, Acc=48.80]
Epoch 2: 99%|█████████▉| 117/118 [00:01<00:00, 100.91it/s, v_num=3, train_loss=1.400, Acc=48.80]
Epoch 2: 100%|██████████| 118/118 [00:01<00:00, 101.62it/s, v_num=3, train_loss=1.400, Acc=48.80]
Epoch 2: 100%|██████████| 118/118 [00:01<00:00, 101.60it/s, v_num=3, train_loss=1.340, Acc=48.80]
Validation: | | 0/? [00:00<?, ?it/s]
Validation: 0%| | 0/5 [00:00<?, ?it/s]
Validation DataLoader 0: 0%| | 0/5 [00:00<?, ?it/s]
Validation DataLoader 0: 20%|██ | 1/5 [00:00<00:00, 404.66it/s]
Validation DataLoader 0: 40%|████ | 2/5 [00:00<00:00, 61.17it/s]
Validation DataLoader 0: 60%|██████ | 3/5 [00:00<00:00, 81.03it/s]
Validation DataLoader 0: 80%|████████ | 4/5 [00:00<00:00, 96.49it/s]
Validation DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 52.79it/s]
Epoch 2: 100%|██████████| 118/118 [00:01<00:00, 79.44it/s, v_num=3, train_loss=1.340, Acc=82.20]
Epoch 2: 100%|██████████| 118/118 [00:01<00:00, 79.39it/s, v_num=3, train_loss=1.340, Acc=82.20]
Epoch 2: 100%|██████████| 118/118 [00:01<00:00, 79.27it/s, v_num=3, train_loss=1.340, Acc=82.20]
Testing: | | 0/? [00:00<?, ?it/s]
Testing: 0%| | 0/5 [00:00<?, ?it/s]
Testing DataLoader 0: 0%| | 0/5 [00:00<?, ?it/s]
Testing DataLoader 0: 20%|██ | 1/5 [00:00<00:00, 192.39it/s]
Testing DataLoader 0: 40%|████ | 2/5 [00:00<00:00, 161.30it/s]
Testing DataLoader 0: 60%|██████ | 3/5 [00:00<00:00, 165.72it/s]
Testing DataLoader 0: 80%|████████ | 4/5 [00:00<00:00, 168.80it/s]
Testing DataLoader 0: 100%|██████████| 5/5 [00:00<00:00, 55.03it/s]
Testing DataLoader 0: 0%| | 0/5 [00:00<?, ?it/s]
Testing DataLoader 1: 0%| | 0/5 [00:00<?, ?it/s]
Testing DataLoader 1: 20%|██ | 1/5 [00:00<00:00, 151.07it/s]
Testing DataLoader 1: 40%|████ | 2/5 [00:00<00:00, 78.03it/s]
Testing DataLoader 1: 60%|██████ | 3/5 [00:00<00:00, 96.39it/s]
Testing DataLoader 1: 80%|████████ | 4/5 [00:00<00:00, 109.34it/s]
Testing DataLoader 1: 100%|██████████| 5/5 [00:00<00:00, 55.56it/s]
Testing DataLoader 1: 100%|██████████| 5/5 [00:00<00:00, 49.07it/s]
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Test metric ┃ Classification ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Acc │ 82.150% │
│ Brier │ 0.38291 │
│ Entropy │ 1.57200 │
│ NLL │ 0.87024 │
└──────────────┴───────────────────────────┘
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Test metric ┃ Calibration ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ ECE │ 31.711% │
│ aECE │ 31.711% │
└──────────────┴───────────────────────────┘
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Test metric ┃ OOD Detection ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ AUPR │ 78.680% │
│ AUROC │ 83.286% │
│ Entropy │ 1.57200 │
│ FPR95 │ 45.300% │
│ ens_Disagre… │ 0.72700 │
│ ens_Entropy │ 1.66604 │
│ ens_MI │ 0.26949 │
└──────────────┴───────────────────────────┘
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Test metric ┃ Selective Classification ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ AUGRC │ 3.991% │
│ AURC │ 5.157% │
│ Cov@5Risk │ 60.680% │
│ Risk@80Cov │ 10.475% │
└──────────────┴───────────────────────────┘
The training time should be approximately similar to the one of the single model that you trained before. However, please note that we are working with very small models, hence completely underusing your GPU. As such, the training time is not representative of what you would observe with larger models.
You can read more on Packed-Ensembles in the paper or the Medium post.
To Go Further & More Concepts of Uncertainty in ML#
Question 1: Have a look at the models in the “lightning_logs”. If you are on your own machine, try to visualize the learning curves with tensorboard –logdir lightning_logs.
Question 2: Add a cell below and try to find the errors made by packed-ensembles on the test set. Visualize the errors and their labels and look at the predictions of the different sub-models. Are they similar? Can you think of uncertainty scores that could help you identify these errors?
Selective Classification#
Selective classification or “prediction with rejection” is a paradigm in uncertainty-aware machine learning where the model can decide not to make a prediction if the confidence score given by the model is below some pre-computed threshold. This can be useful in real-world applications where the cost of making a wrong prediction is high.
In constrast to calibration, the values of the confidence scores are not important, only the order of the scores. Ideally, the best model will order all the correct predictions first, and all the incorrect predictions last. In this case, there will be a threshold so that all the predictions above the threshold are correct, and all the predictions below the threshold are incorrect.
In TorchUncertainty, we look at 3 different metrics for selective classification: - AURC: The area under the Risk (% of errors) vs. Coverage (% of classified samples) curve. This curve expresses how the risk of the model evolves as we increase the coverage (the proportion of predictions that are above the selection threshold). This metric will be minimized by a model able to perfectly separate the correct and incorrect predictions.
The following metrics are computed at a fixed risk and coverage level and that have practical interests. The idea of these metrics is that you can set the selection threshold to achieve a certain level of risk and coverage, as required by the technical constraints of your application: - Coverage at 5% Risk: The proportion of predictions that are above the selection threshold when it is set for the risk to egal 5%. Set the risk threshold to your application constraints. The higher the better. - Risk at 80% Coverage: The proportion of errors when the coverage is set to 80%. Set the coverage threshold to your application constraints. The lower the better.
Grouping Loss#
The grouping loss is a measure of uncertainty orthogonal to calibration. Have a look at this paper to learn about it. Check out their small library GLest. TorchUncertainty includes a wrapper of the library to compute the grouping loss with eval_grouping_loss parameter.
Total running time of the script: (0 minutes 26.587 seconds)