{"cells": [{"cell_type": "markdown", "id": "14e28b67", "metadata": {"papermill": {"duration": 0.004712, "end_time": "2023-10-11T19:17:43.890508", "exception": false, "start_time": "2023-10-11T19:17:43.885796", "status": "completed"}, "tags": []}, "source": ["\n", "# Introduction to PyTorch Lightning\n", "\n", "* **Author:** Lightning.ai\n", "* **License:** CC BY-SA\n", "* **Generated:** 2023-10-11T19:15:44.096249\n", "\n", "In this notebook, we'll go over the basics of lightning by preparing models to train on the [MNIST Handwritten Digits dataset](https://en.wikipedia.org/wiki/MNIST_database).\n", "\n", "---\n", "Open in [![Open In Colab](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAHUAAAAUCAYAAACzrHJDAAAIuUlEQVRoQ+1ZaVRURxb+qhdolmbTUVSURpZgmLhHbQVFZIlGQBEXcMvJhKiTEzfigjQg7oNEJ9GMGidnjnNMBs2czIzajksEFRE1xklCTKJiQLRFsUGkoUWw+82pamn79etGYoKek1B/4NW99/tu3e/dquJBAGD27NkHALxKf39WY39gyrOi+i3xqGtUoePJrFmznrmgtModorbTu8YRNZk5cybXTvCtwh7o6NR2KzuZMWNGh6jtVt7nA0ymT5/eJlF9POrh7PAQl6s8bGYa3PUum//htmebVtLRqW0q01M5keTk5FZFzU0oRle3+zxwg5Hgtb+PZiL/ZVohxCI+hL5JgjmfjPxZ26+33BG3dA+ealHPM4gQAo5rU59gsI8bRvl54t3Ca62mvHyUAhtOlLd5WSQpKcluBjumnoCLs1EARkVd9E8l3p9y2i7RbQ1B6pFwu/YDgW8KbHJHMTQrwnjz2oZm9M4pavOCfo5jWrgCaaMVcMs6/pNhDr0+AMN93XlxV7R6DNpyzi7W/OE+yIrsjU6rTrbKV5cd/pNyItOmTbMp6sbBB+EqaYJY4cWE3VUciNt1TpgfcRFv71Fi54xT5kSoyLvOBEJMOMxWXkFlBeBSX4u6Zkcs+3KszYRtiapbNRqF31UgetVuc8z9vBXIv1qD+F1f83B6uDlCUyfsZGepGPpmg01OB7EITQbhS9ribKy+DmP1DUiClLz4bnIHVOqa7BY+Z1wg5g3zgUvyehiNpnJKxSLc/ts76LKm0BzX3c0RNy1yXjDcB5lWoro4iNHQxM+f1kWeWQARAWQS++trISJTp061Kep25X/MycwtjuctSC5rxo7ppi7VNUox5+PhPHtrsS2O1qJ6yx1QujQUzm9sh6hbkBlvvGcN8hYnwjUjH6kjfZEd5c/jitz5Jc5U3ENnFynKl4eB7nyEgP2UZ+Yz3/rVEbyYr27qELrtC4FIC0J7sc7xWnmccdHfRRTs0VB+cA4lt+oFcRR/wUeH8FG5w2Mbx8FQ8TXEvv1xYf4wBP3O2WyL3/UVjpXWgIqaFeUPr+wTmDvUB7njH6/bOv+HRg4SqioAg5GDe1aB3ZeMTJkyRSBqkLsWqSEm0fZVBEN94zEZnYvrdx1JL5cxe+a+AbhSJecRRHW/ikTFRTa38dtQlNZ5CRKwFvUtZU/kvBoEF9Uxni/XqIM+dwKbTw3rhcxIf7gmr2M+H6SMwx8iBzJbw5oxeG3Lv5FX9B3AGaHPS8e8z77H7v9VMpvPG5ug1enh7eGK8h0LBTwUb+GInqzInlRUK65DmTPQu4c3+uQKjwKK77zwUxBX4Tq7yR1RuiwUsqlrABCM6esHdXoy47fk4+prYKy8ZF574x4V5BnHQBuf4g9Z9ld8U36L2aktZNNplNfw7zotwWTy5MkCUft4aLEopJj5/OPHl1BQqeAVOnHgNSQOqmBzq9V9cfEm/yx5ubMGKS9cYPZ3vx2OS/c6PVHUuUO7Y1Pci3BO/1zgq18byebfGemLtNF+6JRtOvMk926ibussZqM+1mNz4TWkH7rCbM5phwGRGDAaoF8fY5OHFnlldAA8sgoEXKnDukA1NgSeNjqkJT9brbN4pC9WRweYXyLugR73c+MYvyWfu0yC6+mjzN1Isfw3FKJS98CU/zI1IHFkFPR52cHL2FJk0sB6kMTERIGo9GzcPkLNfA0cwdwi/hfEYO86ZMd9w+y1egfM2T2Eh/vesMNwljSzuZRT420SW3eqy8N6aHMmwmnFUZ7/PGVPbIoNZvNU1BURdHs0bT2+HjL8sDSM2e6vi4Lj5NW8WOLVA6RTT2azxLV+bglaFNqLieqemS/gWkw7NyoAHo+2dEsiivengjKsPFoqWOvbSh/kxPaxyW/JRzH2Fl3EzD9/xjAefJqB3usKUFn/0Gb+S/d/jy3FN2yLOmnSJJtn6oehByEiHPSeXnDxFGPRnoFoaBJjcdQlbDwcjL1zTNuQpoxD7R0OG0uUTMi0fkVwdzBdYIwcwZunxrVJVLplNm54BZp7jfDfYLoNyqQi1K6KxIdHzmN+QQ2WjFIwUT2zTGdlRXo4NFXVUO4sgX5dFC7f0aP/ZlNeUjFBuL8Xjl6uRuP6aMjSjpjzsH62FDU7JhBuGccEXIvDfJFFBc/gHw80dklfCVYnRaDfpiJcutPA4F7qJsfJeUPQI+1fqMlNhFx1FM0GDqkjFVg7NojlQ0Vt4aM5ReSqcbpaCg8nCW5lRsBvbT4T1TLfFptsfh7gItzuKTdJSEiwKSrt1vcmnEXXrsLbYnWDA1bu+z2WKy9Arq+1KRqdfKsoBo0GcdtEpS/B1bO4v0cFiUhkjskvKcMrWwtAPHuwQq8Z+4LZ1vTQANfXt4J0DwZX9gWa9qh4XDM/voC9JXfwYEMMHJcfNtusn82ihvliVUwg5KrPGVf6GH94ZJpEZBen6EC4qYTHA1dXhW0JIex8txzv//c8lhzXIi/BFxOH9jGbQhZsRalTIBZZ8KkGyZAxeRQvXkFF1TWz/Hm46jNYUnjPbt3JxIkT7f6dSj8qfJJyVvBxgaIlblOyjtysNHWN9fjjqWi7glJfW3/S0Hlj2XnA8PhKT9w6g3Qx3XiXhvuxQsuT1proxBKI/AaZqY1Xz5muvY8G8XkRRCaHsfQsRAFDH/tZPbcYuHotOG0FRIqB4HR3wNVoIPLtz8ycTguu+jpEigE218vd1YCr5m+HpHMvEI9u4LTXwNWaLjl0iPwGAmIpeHx1VeCqTJdPs1/vweweQPO3HC24NhOhnTphwoQnfv6QSY2ICbkNmdSA4h87oaLaiYfn5diIEd4att2erOwJXbPUHp953p6orQVSUVWRAXBT8c/dJ5L9xhzaJGp71GR/wFP8P5V2z10NSC9T93QM2xUg8fHxT+zU9ijeU4naHon8CjFJXFzc8/kn+dN06q9QgF98SYSo2Xen2NjYZy5sR6f+4nLSK5Iam2PH/x87a1YN/t5sBgAAAABJRU5ErkJggg==){height=\"20px\" width=\"117px\"}](https://colab.research.google.com/github/PytorchLightning/lightning-tutorials/blob/publication/.notebooks/lightning_examples/mnist-hello-world.ipynb)\n", "\n", "Give us a \u2b50 [on Github](https://www.github.com/Lightning-AI/lightning/)\n", "| Check out [the documentation](https://pytorch-lightning.readthedocs.io/en/stable/)\n", "| Join us [on Slack](https://www.pytorchlightning.ai/community)"]}, {"cell_type": "markdown", "id": "76eb4d71", "metadata": {"papermill": {"duration": 0.003791, "end_time": "2023-10-11T19:17:43.898299", "exception": false, "start_time": "2023-10-11T19:17:43.894508", "status": "completed"}, "tags": []}, "source": ["## Setup\n", "This notebook requires some packages besides pytorch-lightning."]}, {"cell_type": "code", "execution_count": 1, "id": "9f4f2ba7", "metadata": {"colab": {}, "colab_type": "code", "execution": {"iopub.execute_input": "2023-10-11T19:17:43.906826Z", "iopub.status.busy": "2023-10-11T19:17:43.906598Z", "iopub.status.idle": "2023-10-11T19:19:16.223109Z", "shell.execute_reply": "2023-10-11T19:19:16.221646Z"}, "id": "LfrJLKPFyhsK", "lines_to_next_cell": 0, "papermill": {"duration": 92.323974, "end_time": "2023-10-11T19:19:16.225919", "exception": false, "start_time": "2023-10-11T19:17:43.901945", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\r\n", "\u001b[0m"]}], "source": ["! pip install --quiet \"pandas\" \"torchmetrics>=0.7, <1.3\" \"urllib3\" \"lightning>=2.0.0\" \"setuptools>=68.0.0, <68.3.0\" \"torchvision\" \"pytorch-lightning>=1.4, <2.1.0\" \"ipython[notebook]>=8.0.0, <8.17.0\" \"torchmetrics >=0.11.0\" \"seaborn\" \"torch>=1.8.1, <2.1.0\" \"matplotlib>=3.0.0, <3.9.0\""]}, {"cell_type": "code", "execution_count": 2, "id": "fa6ca114", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T19:19:16.241909Z", "iopub.status.busy": "2023-10-11T19:19:16.241535Z", "iopub.status.idle": "2023-10-11T19:19:20.298920Z", "shell.execute_reply": "2023-10-11T19:19:20.297076Z"}, "papermill": {"duration": 4.069363, "end_time": "2023-10-11T19:19:20.301611", "exception": false, "start_time": "2023-10-11T19:19:16.232248", "status": "completed"}, "tags": []}, "outputs": [], "source": ["\n", "# ------------------- Preliminaries ------------------- #\n", "import os\n", "from dataclasses import dataclass\n", "from typing import Tuple\n", "\n", "import lightning as L\n", "import pandas as pd\n", "import seaborn as sn\n", "import torch\n", "from IPython.display import display\n", "from lightning.pytorch.loggers import CSVLogger\n", "from torch import nn\n", "from torch.nn import functional as F\n", "from torch.utils.data import DataLoader, random_split\n", "from torchmetrics import Accuracy\n", "from torchvision import transforms\n", "from torchvision.datasets import MNIST\n", "\n", "# ------------------- Configuration ------------------- #\n", "\n", "\n", "@dataclass\n", "class Config:\n", "    \"\"\"Configuration options for the Lightning MNIST example.\n", "\n", "    Args:\n", "        data_dir : The path to the directory where the MNIST dataset is stored. Defaults to the value of\n", "            the 'PATH_DATASETS' environment variable or '.' if not set.\n", "\n", "        save_dir : The path to the directory where the training logs will be saved. Defaults to 'logs/'.\n", "\n", "        batch_size : The batch size to use during training. Defaults to 256 if a GPU is available,\n", "            or 64 otherwise.\n", "\n", "        max_epochs : The maximum number of epochs to train the model for. Defaults to 3.\n", "\n", "        accelerator : The accelerator to use for training. Can be one of \"cpu\", \"gpu\", \"tpu\", \"ipu\", \"auto\".\n", "\n", "        devices : The number of devices to use for training. Defaults to 1.\n", "\n", "    Examples:\n", "        This dataclass can be used to specify the configuration options for training a PyTorch Lightning model on the\n", "        MNIST dataset. A new instance of this dataclass can be created as follows:\n", "\n", "        >>> config = Config()\n", "\n", "        The default values for each argument are shown in the documentation above. If desired, any of these values can be\n", "        overridden when creating a new instance of the dataclass:\n", "\n", "        >>> config = Config(batch_size=128, max_epochs=5)\n", "    \"\"\"\n", "\n", "    data_dir: str = os.environ.get(\"PATH_DATASETS\", \".\")\n", "    save_dir: str = \"logs/\"\n", "    batch_size: int = 256 if torch.cuda.is_available() else 64\n", "    max_epochs: int = 3\n", "    accelerator: str = \"auto\"\n", "    devices: int = 1\n", "\n", "\n", "config = Config()"]}, {"cell_type": "markdown", "id": "507ccd62", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.006133, "end_time": "2023-10-11T19:19:20.314347", "exception": false, "start_time": "2023-10-11T19:19:20.308214", "status": "completed"}, "tags": []}, "source": ["## Simplest example\n", "\n", "Here's the simplest most minimal example with just a training loop (no validation, no testing).\n", "\n", "**Keep in Mind** - A `LightningModule` *is* a PyTorch `nn.Module` - it just has a few more helpful features."]}, {"cell_type": "code", "execution_count": 3, "id": "4a2a4091", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T19:19:20.329307Z", "iopub.status.busy": "2023-10-11T19:19:20.328270Z", "iopub.status.idle": "2023-10-11T19:19:20.341979Z", "shell.execute_reply": "2023-10-11T19:19:20.341006Z"}, "papermill": {"duration": 0.023944, "end_time": "2023-10-11T19:19:20.344384", "exception": false, "start_time": "2023-10-11T19:19:20.320440", "status": "completed"}, "tags": []}, "outputs": [], "source": ["\n", "\n", "class MNISTModel(L.LightningModule):\n", "    \"\"\"A PyTorch Lightning module for classifying images in the MNIST dataset.\n", "\n", "    Attributes:\n", "        l1 : A linear layer that maps input features to output features.\n", "\n", "    Methods:\n", "        forward(x):\n", "            Performs a forward pass through the model.\n", "\n", "        training_step(batch, batch_nb):\n", "            Defines a single training step for the model.\n", "\n", "        configure_optimizers():\n", "            Configures the optimizer to use during training.\n", "\n", "    Examples:\n", "        The MNISTModel class can be used to create and train a PyTorch Lightning model for classifying images in the MNIST\n", "        dataset. To create a new instance of the model, simply instantiate the class:\n", "\n", "        >>> model = MNISTModel()\n", "\n", "        The model can then be trained using a PyTorch Lightning trainer object:\n", "\n", "        >>> trainer = pl.Trainer()\n", "        >>> trainer.fit(model)\n", "    \"\"\"\n", "\n", "    def __init__(self):\n", "        \"\"\"Initializes a new instance of the MNISTModel class.\"\"\"\n", "        super().__init__()\n", "        self.l1 = torch.nn.Linear(28 * 28, 10)\n", "\n", "    def forward(self, x: torch.Tensor) -> torch.Tensor:\n", "        \"\"\"Performs a forward pass through the model.\n", "\n", "        Args:\n", "            x : The input tensor to pass through the model.\n", "\n", "        Returns:\n", "            activated : The output tensor produced by the model.\n", "\n", "        Examples:\n", "            >>> model = MNISTModel()\n", "            >>> x = torch.randn(1, 1, 28, 28)\n", "            >>> output = model(x)\n", "        \"\"\"\n", "        flattened = x.view(x.size(0), -1)\n", "        hidden = self.l1(flattened)\n", "        activated = torch.relu(hidden)\n", "\n", "        return activated\n", "\n", "    def training_step(self, batch: Tuple[torch.Tensor, torch.Tensor], batch_nb: int) -> torch.Tensor:\n", "        \"\"\"Defines a single training step for the model.\n", "\n", "        Args:\n", "            batch: A tuple containing the input and target tensors for the batch.\n", "            batch_nb: The batch number.\n", "\n", "        Returns:\n", "            torch.Tensor: The loss value for the current batch.\n", "\n", "        Examples:\n", "            >>> model = MNISTModel()\n", "            >>> x = torch.randn(1, 1, 28, 28)\n", "            >>> y = torch.tensor([1])\n", "            >>> loss = model.training_step((x, y), 0)\n", "        \"\"\"\n", "        x, y = batch\n", "        loss = F.cross_entropy(self(x), y)\n", "        return loss\n", "\n", "    def configure_optimizers(self) -> torch.optim.Optimizer:\n", "        \"\"\"Configures the optimizer to use during training.\n", "\n", "        Returns:\n", "            torch.optim.Optimizer: The optimizer to use during training.\n", "\n", "        Examples:\n", "            >>> model = MNISTModel()\n", "            >>> optimizer = model.configure_optimizers()\n", "        \"\"\"\n", "        return torch.optim.Adam(self.parameters(), lr=0.02)"]}, {"cell_type": "markdown", "id": "e189cea8", "metadata": {"papermill": {"duration": 0.006164, "end_time": "2023-10-11T19:19:20.356982", "exception": false, "start_time": "2023-10-11T19:19:20.350818", "status": "completed"}, "tags": []}, "source": ["By using the `Trainer` you automatically get:\n", "1. Tensorboard logging\n", "2. Model checkpointing\n", "3. Training and validation loop\n", "4. early-stopping"]}, {"cell_type": "code", "execution_count": 4, "id": "8ea1b3bb", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T19:19:20.371087Z", "iopub.status.busy": "2023-10-11T19:19:20.370720Z", "iopub.status.idle": "2023-10-11T19:19:38.567377Z", "shell.execute_reply": "2023-10-11T19:19:38.566580Z"}, "papermill": {"duration": 18.206536, "end_time": "2023-10-11T19:19:38.569794", "exception": false, "start_time": "2023-10-11T19:19:20.363258", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\n", "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to /__w/14/s/.datasets/MNIST/raw/train-images-idx3-ubyte.gz\n"]}, {"name": "stderr", "output_type": "stream", "text": ["\r", "  0%|          | 0/9912422 [00:00<?, ?it/s]"]}, {"name": "stderr", "output_type": "stream", "text": ["\r", " 45%|\u2588\u2588\u2588\u2588\u258d     | 4423680/9912422 [00:00<00:00, 44140128.13it/s]"]}, {"name": "stderr", "output_type": "stream", "text": ["\r", " 95%|\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u258c| 9437184/9912422 [00:00<00:00, 47607761.85it/s]"]}, {"name": "stderr", "output_type": "stream", "text": ["\r", "100%|\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 9912422/9912422 [00:00<00:00, 47157242.46it/s]"]}, {"name": "stderr", "output_type": "stream", "text": ["\n"]}, {"name": "stdout", "output_type": "stream", "text": ["Extracting /__w/14/s/.datasets/MNIST/raw/train-images-idx3-ubyte.gz to /__w/14/s/.datasets/MNIST/raw\n"]}, {"name": "stdout", "output_type": "stream", "text": ["\n", "Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz\n", "Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz to /__w/14/s/.datasets/MNIST/raw/train-labels-idx1-ubyte.gz\n"]}, {"name": "stderr", "output_type": "stream", "text": ["\r", "  0%|          | 0/28881 [00:00<?, ?it/s]"]}, {"name": "stderr", "output_type": "stream", "text": ["\r", "100%|\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 28881/28881 [00:00<00:00, 47152858.63it/s]"]}, {"name": "stderr", "output_type": "stream", "text": ["\n"]}, {"name": "stdout", "output_type": "stream", "text": ["Extracting /__w/14/s/.datasets/MNIST/raw/train-labels-idx1-ubyte.gz to /__w/14/s/.datasets/MNIST/raw\n", "\n", "Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz\n", "Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz to /__w/14/s/.datasets/MNIST/raw/t10k-images-idx3-ubyte.gz\n"]}, {"name": "stderr", "output_type": "stream", "text": ["\r", "  0%|          | 0/1648877 [00:00<?, ?it/s]"]}, {"name": "stderr", "output_type": "stream", "text": ["\r", "100%|\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 1648877/1648877 [00:00<00:00, 37055704.43it/s]"]}, {"name": "stderr", "output_type": "stream", "text": ["\n"]}, {"name": "stdout", "output_type": "stream", "text": ["Extracting /__w/14/s/.datasets/MNIST/raw/t10k-images-idx3-ubyte.gz to /__w/14/s/.datasets/MNIST/raw\n", "\n", "Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz\n"]}, {"name": "stdout", "output_type": "stream", "text": ["Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz to /__w/14/s/.datasets/MNIST/raw/t10k-labels-idx1-ubyte.gz\n"]}, {"name": "stderr", "output_type": "stream", "text": ["\r", "  0%|          | 0/4542 [00:00<?, ?it/s]"]}, {"name": "stderr", "output_type": "stream", "text": ["\r", "100%|\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 4542/4542 [00:00<00:00, 9379876.30it/s]"]}, {"name": "stderr", "output_type": "stream", "text": ["\n"]}, {"name": "stdout", "output_type": "stream", "text": ["Extracting /__w/14/s/.datasets/MNIST/raw/t10k-labels-idx1-ubyte.gz to /__w/14/s/.datasets/MNIST/raw\n", "\n"]}, {"name": "stderr", "output_type": "stream", "text": ["GPU available: True (cuda), used: True\n"]}, {"name": "stderr", "output_type": "stream", "text": ["TPU available: False, using: 0 TPU cores\n"]}, {"name": "stderr", "output_type": "stream", "text": ["IPU available: False, using: 0 IPUs\n"]}, {"name": "stderr", "output_type": "stream", "text": ["HPU available: False, using: 0 HPUs\n"]}, {"name": "stderr", "output_type": "stream", "text": ["/usr/local/lib/python3.10/dist-packages/lightning/pytorch/trainer/connectors/logger_connector/logger_connector.py:67: UserWarning: Starting from v1.9.0, `tensorboardX` has been removed as a dependency of the `lightning.pytorch` package, due to potential conflicts with other packages in the ML ecosystem. For this reason, `logger=True` will use `CSVLogger` as the default logger, unless the `tensorboard` or `tensorboardX` packages are found. Please `pip install lightning[extra]` or one of them to enable TensorBoard support by default\n", "  warning_cache.warn(\n"]}, {"name": "stderr", "output_type": "stream", "text": ["You are using a CUDA device ('NVIDIA GeForce RTX 3090') that has Tensor Cores. To properly utilize them, you should set `torch.set_float32_matmul_precision('medium' | 'high')` which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision\n"]}, {"name": "stderr", "output_type": "stream", "text": ["LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [2,3]\n"]}, {"name": "stderr", "output_type": "stream", "text": ["\n", "  | Name | Type   | Params\n", "--------------------------------\n", "0 | l1   | Linear | 7.9 K \n", "--------------------------------\n", "7.9 K     Trainable params\n", "0         Non-trainable params\n", "7.9 K     Total params\n", "0.031     Total estimated model params size (MB)\n"]}, {"name": "stderr", "output_type": "stream", "text": ["/usr/local/lib/python3.10/dist-packages/lightning/pytorch/trainer/connectors/data_connector.py:442: PossibleUserWarning: The dataloader, train_dataloader, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 64 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.\n", "  rank_zero_warn(\n"]}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "37b0c1b7cbe74ab383f5bdefe7aed12b", "version_major": 2, "version_minor": 0}, "text/plain": ["Training: 0it [00:00, ?it/s]"]}, "metadata": {}, "output_type": "display_data"}, {"name": "stderr", "output_type": "stream", "text": ["`Trainer.fit` stopped: `max_epochs=3` reached.\n"]}], "source": ["# Init our model\n", "mnist_model = MNISTModel()\n", "\n", "# Init DataLoader from MNIST Dataset\n", "train_ds = MNIST(config.data_dir, train=True, download=True, transform=transforms.ToTensor())\n", "\n", "# Create a dataloader\n", "train_loader = DataLoader(train_ds, batch_size=config.batch_size)\n", "\n", "# Initialize a trainer\n", "trainer = L.Trainer(\n", "    accelerator=config.accelerator,\n", "    devices=config.devices,\n", "    max_epochs=config.max_epochs,\n", ")\n", "\n", "# Train the model \u26a1\n", "trainer.fit(mnist_model, train_loader)"]}, {"cell_type": "markdown", "id": "c642887e", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.01001, "end_time": "2023-10-11T19:19:38.591383", "exception": false, "start_time": "2023-10-11T19:19:38.581373", "status": "completed"}, "tags": []}, "source": ["## A more complete MNIST Lightning Module Example\n", "\n", "That wasn't so hard was it?\n", "\n", "Now that we've got our feet wet, let's dive in a bit deeper and write a more complete `LightningModule` for MNIST...\n", "\n", "This time, we'll bake in all the dataset specific pieces directly in the `LightningModule`.\n", "This way, we can avoid writing extra code at the beginning of our script every time we want to run it.\n", "\n", "---\n", "\n", "### Note what the following built-in functions are doing:\n", "\n", "1. [prepare_data()](https://lightning.ai/docs/pytorch/stable/common/lightning_module.html#prepare-data) \ud83d\udcbe\n", "    - This is where we can download the dataset. We point to our desired dataset and ask torchvision's `MNIST` dataset class to download if the dataset isn't found there.\n", "    - **Note we do not make any state assignments in this function** (i.e. `self.something = ...`)\n", "\n", "2. [setup(stage)](https://lightning.ai/docs/pytorch/stable/common/lightning_module.html#setup) \u2699\ufe0f\n", "    - Loads in data from file and prepares PyTorch tensor datasets for each split (train, val, test).\n", "    - Setup expects a 'stage' arg which is used to separate logic for 'fit' and 'test'.\n", "    - If you don't mind loading all your datasets at once, you can set up a condition to allow for both 'fit' related setup and 'test' related setup to run whenever `None` is passed to `stage` (or ignore it altogether and exclude any conditionals).\n", "    - **Note this runs across all GPUs and it *is* safe to make state assignments here**\n", "\n", "3. [x_dataloader()](https://lightning.ai/docs/pytorch/stable/api/pytorch_lightning.core.hooks.DataHooks.html#pytorch_lightning.core.hooks.DataHooks.train_dataloader) \u267b\ufe0f\n", "    - `train_dataloader()`, `val_dataloader()`, and `test_dataloader()` all return PyTorch `DataLoader` instances that are created by wrapping their respective datasets that we prepared in `setup()`"]}, {"cell_type": "code", "execution_count": 5, "id": "b590d798", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T19:19:38.612552Z", "iopub.status.busy": "2023-10-11T19:19:38.612287Z", "iopub.status.idle": "2023-10-11T19:19:38.632716Z", "shell.execute_reply": "2023-10-11T19:19:38.632022Z"}, "papermill": {"duration": 0.032842, "end_time": "2023-10-11T19:19:38.634253", "exception": false, "start_time": "2023-10-11T19:19:38.601411", "status": "completed"}, "tags": []}, "outputs": [], "source": ["\n", "\n", "class LitMNIST(L.LightningModule):\n", "    \"\"\"PyTorch Lightning module for training a multi-layer perceptron (MLP) on the MNIST dataset.\n", "\n", "    Attributes:\n", "        data_dir : The path to the directory where the MNIST data will be downloaded.\n", "\n", "        hidden_size : The number of units in the hidden layer of the MLP.\n", "\n", "        learning_rate : The learning rate to use for training the MLP.\n", "\n", "    Methods:\n", "        forward(x):\n", "            Performs a forward pass through the MLP.\n", "\n", "        training_step(batch, batch_idx):\n", "            Defines a single training step for the MLP.\n", "\n", "        validation_step(batch, batch_idx):\n", "            Defines a single validation step for the MLP.\n", "\n", "        test_step(batch, batch_idx):\n", "            Defines a single testing step for the MLP.\n", "\n", "        configure_optimizers():\n", "            Configures the optimizer to use for training the MLP.\n", "\n", "        prepare_data():\n", "            Downloads the MNIST dataset.\n", "\n", "        setup(stage=None):\n", "            Splits the MNIST dataset into train, validation, and test sets.\n", "\n", "        train_dataloader():\n", "            Returns a DataLoader for the training set.\n", "\n", "        val_dataloader():\n", "            Returns a DataLoader for the validation set.\n", "\n", "        test_dataloader():\n", "            Returns a DataLoader for the test set.\n", "    \"\"\"\n", "\n", "    def __init__(self, data_dir: str = config.data_dir, hidden_size: int = 64, learning_rate: float = 2e-4):\n", "        \"\"\"Initializes a new instance of the LitMNIST class.\n", "\n", "        Args:\n", "            data_dir : The path to the directory where the MNIST data will be downloaded. Defaults to config.data_dir.\n", "\n", "            hidden_size : The number of units in the hidden layer of the MLP (default is 64).\n", "\n", "            learning_rate : The learning rate to use for training the MLP (default is 2e-4).\n", "        \"\"\"\n", "        super().__init__()\n", "\n", "        # Set our init args as class attributes\n", "        self.data_dir = data_dir\n", "        self.hidden_size = hidden_size\n", "        self.learning_rate = learning_rate\n", "\n", "        # Hardcode some dataset specific attributes\n", "        self.num_classes = 10\n", "        self.dims = (1, 28, 28)\n", "        channels, width, height = self.dims\n", "\n", "        self.transform = transforms.Compose(\n", "            [\n", "                transforms.ToTensor(),\n", "                transforms.Normalize((0.1307,), (0.3081,)),\n", "            ]\n", "        )\n", "\n", "        # Define PyTorch model\n", "        self.model = nn.Sequential(\n", "            nn.Flatten(),\n", "            nn.Linear(channels * width * height, hidden_size),\n", "            nn.ReLU(),\n", "            nn.Dropout(0.1),\n", "            nn.Linear(hidden_size, hidden_size),\n", "            nn.ReLU(),\n", "            nn.Dropout(0.1),\n", "            nn.Linear(hidden_size, self.num_classes),\n", "        )\n", "\n", "        self.val_accuracy = Accuracy(task=\"multiclass\", num_classes=10)\n", "        self.test_accuracy = Accuracy(task=\"multiclass\", num_classes=10)\n", "\n", "    def forward(self, x: torch.Tensor) -> torch.Tensor:\n", "        \"\"\"Performs a forward pass through the MLP.\n", "\n", "        Args:\n", "            x : The input data.\n", "\n", "        Returns:\n", "            torch.Tensor: The output of the MLP.\n", "        \"\"\"\n", "        x = self.model(x)\n", "        return F.log_softmax(x, dim=1)\n", "\n", "    def training_step(self, batch: Tuple[torch.Tensor, torch.Tensor], batch_nb: int) -> torch.Tensor:\n", "        \"\"\"Defines a single training step for the MLP.\n", "\n", "        Args:\n", "            batch: A tuple containing the input data and target labels.\n", "\n", "            batch_idx: The index of the current batch.\n", "\n", "        Returns:\n", "            (torch.Tensor): The training loss.\n", "        \"\"\"\n", "        x, y = batch\n", "        logits = self(x)\n", "        loss = F.nll_loss(logits, y)\n", "        return loss\n", "\n", "    def validation_step(self, batch: Tuple[torch.Tensor, torch.Tensor], batch_nb: int) -> None:\n", "        \"\"\"Defines a single validation step for the MLP.\n", "\n", "        Args:\n", "            batch : A tuple containing the input data and target labels.\n", "            batch_idx : The index of the current batch.\n", "        \"\"\"\n", "        x, y = batch\n", "        logits = self(x)\n", "        loss = F.nll_loss(logits, y)\n", "        preds = torch.argmax(logits, dim=1)\n", "        self.val_accuracy.update(preds, y)\n", "\n", "        # Calling self.log will surface up scalars for you in TensorBoard\n", "        self.log(\"val_loss\", loss, prog_bar=True)\n", "        self.log(\"val_acc\", self.val_accuracy, prog_bar=True)\n", "\n", "    def test_step(self, batch: Tuple[torch.Tensor, torch.Tensor], batch_nb: int) -> None:\n", "        \"\"\"Defines a single testing step for the MLP.\n", "\n", "        Args:\n", "            batch : A tuple containing the input data and target labels.\n", "            batch_idx : The index of the current batch.\n", "        \"\"\"\n", "        x, y = batch\n", "        logits = self(x)\n", "        loss = F.nll_loss(logits, y)\n", "        preds = torch.argmax(logits, dim=1)\n", "        self.test_accuracy.update(preds, y)\n", "\n", "        # Calling self.log will surface up scalars for you in TensorBoard\n", "        self.log(\"test_loss\", loss, prog_bar=True)\n", "        self.log(\"test_acc\", self.test_accuracy, prog_bar=True)\n", "\n", "    def configure_optimizers(self) -> torch.optim.Optimizer:\n", "        \"\"\"Configures the optimizer to use for training the MLP.\n", "\n", "        Returns:\n", "            torch.optim.Optimizer: The optimizer.\n", "        \"\"\"\n", "        optimizer = torch.optim.Adam(self.parameters(), lr=self.learning_rate)\n", "\n", "        return optimizer\n", "\n", "    # ------------------------------------- #\n", "    # DATA RELATED HOOKS\n", "    # ------------------------------------- #\n", "\n", "    def prepare_data(self) -> None:\n", "        \"\"\"Downloads the MNIST dataset.\"\"\"\n", "        MNIST(self.data_dir, train=True, download=True)\n", "\n", "        MNIST(self.data_dir, train=False, download=True)\n", "\n", "    def setup(self, stage: str = None) -> None:\n", "        \"\"\"Splits the MNIST dataset into train, validation, and test sets.\n", "\n", "        Args:\n", "            stage : The current stage (either \"fit\" or \"test\"). Defaults to None.\n", "        \"\"\"\n", "        # Assign train/val datasets for use in dataloaders\n", "        if stage == \"fit\" or stage is None:\n", "            mnist_full = MNIST(self.data_dir, train=True, transform=self.transform)\n", "\n", "            self.mnist_train, self.mnist_val = random_split(mnist_full, [55000, 5000])\n", "\n", "        # Assign test dataset for use in dataloader(s)\n", "        if stage == \"test\" or stage is None:\n", "            self.mnist_test = MNIST(self.data_dir, train=False, transform=self.transform)\n", "\n", "    def train_dataloader(self) -> DataLoader:\n", "        \"\"\"Returns a DataLoader for the training set.\n", "\n", "        Returns:\n", "            DataLoader: The training DataLoader.\n", "        \"\"\"\n", "        return DataLoader(self.mnist_train, batch_size=config.batch_size)\n", "\n", "    def val_dataloader(self) -> DataLoader:\n", "        \"\"\"Returns a DataLoader for the validation set.\n", "\n", "        Returns:\n", "            DataLoader: The validation DataLoader.\n", "        \"\"\"\n", "        return DataLoader(self.mnist_val, batch_size=config.batch_size)\n", "\n", "    def test_dataloader(self) -> DataLoader:\n", "        \"\"\"Returns a DataLoader for the test set.\n", "\n", "        Returns:\n", "            DataLoader: The test DataLoader.\n", "        \"\"\"\n", "        return DataLoader(self.mnist_test, batch_size=config.batch_size)"]}, {"cell_type": "code", "execution_count": 6, "id": "3e4b6829", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T19:19:38.647554Z", "iopub.status.busy": "2023-10-11T19:19:38.647330Z", "iopub.status.idle": "2023-10-11T19:20:06.384372Z", "shell.execute_reply": "2023-10-11T19:20:06.383605Z"}, "papermill": {"duration": 27.746092, "end_time": "2023-10-11T19:20:06.386494", "exception": false, "start_time": "2023-10-11T19:19:38.640402", "status": "completed"}, "tags": []}, "outputs": [{"name": "stderr", "output_type": "stream", "text": ["GPU available: True (cuda), used: True\n"]}, {"name": "stderr", "output_type": "stream", "text": ["TPU available: False, using: 0 TPU cores\n"]}, {"name": "stderr", "output_type": "stream", "text": ["IPU available: False, using: 0 IPUs\n"]}, {"name": "stderr", "output_type": "stream", "text": ["HPU available: False, using: 0 HPUs\n"]}, {"name": "stderr", "output_type": "stream", "text": ["LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [2,3]\n"]}, {"name": "stderr", "output_type": "stream", "text": ["\n", "  | Name          | Type               | Params\n", "-----------------------------------------------------\n", "0 | model         | Sequential         | 55.1 K\n", "1 | val_accuracy  | MulticlassAccuracy | 0     \n", "2 | test_accuracy | MulticlassAccuracy | 0     \n", "-----------------------------------------------------\n", "55.1 K    Trainable params\n", "0         Non-trainable params\n", "55.1 K    Total params\n", "0.220     Total estimated model params size (MB)\n"]}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "2c07ea9039cb4476a1a8dc08bc9808c7", "version_major": 2, "version_minor": 0}, "text/plain": ["Sanity Checking: 0it [00:00, ?it/s]"]}, "metadata": {}, "output_type": "display_data"}, {"name": "stderr", "output_type": "stream", "text": ["/usr/local/lib/python3.10/dist-packages/lightning/pytorch/trainer/connectors/data_connector.py:442: PossibleUserWarning: The dataloader, val_dataloader, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 64 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.\n", "  rank_zero_warn(\n"]}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "a496173fbae644c6913bc1165e6e15a3", "version_major": 2, "version_minor": 0}, "text/plain": ["Training: 0it [00:00, ?it/s]"]}, "metadata": {}, "output_type": "display_data"}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "935105a8a90b4507a57417d2dd17b32a", "version_major": 2, "version_minor": 0}, "text/plain": ["Validation: 0it [00:00, ?it/s]"]}, "metadata": {}, "output_type": "display_data"}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "c05d49bc81f64628ba3f1c0773394344", "version_major": 2, "version_minor": 0}, "text/plain": ["Validation: 0it [00:00, ?it/s]"]}, "metadata": {}, "output_type": "display_data"}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "5ef6a7e7e41a44cd9b227c830b1474c5", "version_major": 2, "version_minor": 0}, "text/plain": ["Validation: 0it [00:00, ?it/s]"]}, "metadata": {}, "output_type": "display_data"}, {"name": "stderr", "output_type": "stream", "text": ["`Trainer.fit` stopped: `max_epochs=3` reached.\n"]}], "source": ["# Instantiate the LitMNIST model\n", "model = LitMNIST()\n", "\n", "# Instantiate a PyTorch Lightning trainer with the specified configuration\n", "trainer = L.Trainer(\n", "    accelerator=config.accelerator,\n", "    devices=config.devices,\n", "    max_epochs=config.max_epochs,\n", "    logger=CSVLogger(save_dir=config.save_dir),\n", ")\n", "\n", "# Train the model using the trainer\n", "trainer.fit(model)"]}, {"cell_type": "markdown", "id": "37774f48", "metadata": {"papermill": {"duration": 0.011502, "end_time": "2023-10-11T19:20:06.404153", "exception": false, "start_time": "2023-10-11T19:20:06.392651", "status": "completed"}, "tags": []}, "source": ["### Testing\n", "\n", "To test a model, call `trainer.test(model)`.\n", "\n", "Or, if you've just trained a model, you can just call `trainer.test()` and Lightning will automatically\n", "test using the best saved checkpoint (conditioned on val_loss)."]}, {"cell_type": "code", "execution_count": 7, "id": "da572e76", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T19:20:06.427157Z", "iopub.status.busy": "2023-10-11T19:20:06.426978Z", "iopub.status.idle": "2023-10-11T19:20:07.850798Z", "shell.execute_reply": "2023-10-11T19:20:07.850259Z"}, "papermill": {"duration": 1.435929, "end_time": "2023-10-11T19:20:07.852072", "exception": false, "start_time": "2023-10-11T19:20:06.416143", "status": "completed"}, "tags": []}, "outputs": [{"name": "stderr", "output_type": "stream", "text": ["Restoring states from the checkpoint path at logs/lightning_logs/version_0/checkpoints/epoch=2-step=645.ckpt\n"]}, {"name": "stderr", "output_type": "stream", "text": ["LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [2,3]\n"]}, {"name": "stderr", "output_type": "stream", "text": ["Loaded model weights from the checkpoint at logs/lightning_logs/version_0/checkpoints/epoch=2-step=645.ckpt\n"]}, {"name": "stderr", "output_type": "stream", "text": ["/usr/local/lib/python3.10/dist-packages/lightning/pytorch/trainer/connectors/data_connector.py:442: PossibleUserWarning: The dataloader, test_dataloader, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 64 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.\n", "  rank_zero_warn(\n"]}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "84a8558070594e57aa7d892c85785833", "version_major": 2, "version_minor": 0}, "text/plain": ["Testing: 0it [00:00, ?it/s]"]}, "metadata": {}, "output_type": "display_data"}, {"data": {"text/html": ["<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">\u250f\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2513\n", "\u2503<span style=\"font-weight: bold\">        Test metric        </span>\u2503<span style=\"font-weight: bold\">       DataLoader 0        </span>\u2503\n", "\u2521\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2529\n", "\u2502<span style=\"color: #008080; text-decoration-color: #008080\">         test_acc          </span>\u2502<span style=\"color: #800080; text-decoration-color: #800080\">    0.9228000044822693     </span>\u2502\n", "\u2502<span style=\"color: #008080; text-decoration-color: #008080\">         test_loss         </span>\u2502<span style=\"color: #800080; text-decoration-color: #800080\">    0.25202980637550354    </span>\u2502\n", "\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n", "</pre>\n"], "text/plain": ["\u250f\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2513\n", "\u2503\u001b[1m \u001b[0m\u001b[1m       Test metric       \u001b[0m\u001b[1m \u001b[0m\u2503\u001b[1m \u001b[0m\u001b[1m      DataLoader 0       \u001b[0m\u001b[1m \u001b[0m\u2503\n", "\u2521\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2529\n", "\u2502\u001b[36m \u001b[0m\u001b[36m        test_acc         \u001b[0m\u001b[36m \u001b[0m\u2502\u001b[35m \u001b[0m\u001b[35m   0.9228000044822693    \u001b[0m\u001b[35m \u001b[0m\u2502\n", "\u2502\u001b[36m \u001b[0m\u001b[36m        test_loss        \u001b[0m\u001b[36m \u001b[0m\u2502\u001b[35m \u001b[0m\u001b[35m   0.25202980637550354   \u001b[0m\u001b[35m \u001b[0m\u2502\n", "\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n"]}, "metadata": {}, "output_type": "display_data"}, {"data": {"text/plain": ["[{'test_loss': 0.25202980637550354, 'test_acc': 0.9228000044822693}]"]}, "execution_count": 7, "metadata": {}, "output_type": "execute_result"}], "source": ["trainer.test(ckpt_path=\"best\")"]}, {"cell_type": "markdown", "id": "1e64ccd6", "metadata": {"papermill": {"duration": 0.012018, "end_time": "2023-10-11T19:20:07.870339", "exception": false, "start_time": "2023-10-11T19:20:07.858321", "status": "completed"}, "tags": []}, "source": ["### Bonus Tip\n", "\n", "You can keep calling `trainer.fit(model)` as many times as you'd like to continue training"]}, {"cell_type": "code", "execution_count": 8, "id": "fe9227f0", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T19:20:07.893915Z", "iopub.status.busy": "2023-10-11T19:20:07.893744Z", "iopub.status.idle": "2023-10-11T19:20:08.114959Z", "shell.execute_reply": "2023-10-11T19:20:08.114466Z"}, "papermill": {"duration": 0.233489, "end_time": "2023-10-11T19:20:08.117072", "exception": false, "start_time": "2023-10-11T19:20:07.883583", "status": "completed"}, "tags": []}, "outputs": [{"name": "stderr", "output_type": "stream", "text": ["/usr/local/lib/python3.10/dist-packages/lightning/pytorch/callbacks/model_checkpoint.py:617: UserWarning: Checkpoint directory logs/lightning_logs/version_0/checkpoints exists and is not empty.\n", "  rank_zero_warn(f\"Checkpoint directory {dirpath} exists and is not empty.\")\n", "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [2,3]\n"]}, {"name": "stderr", "output_type": "stream", "text": ["\n", "  | Name          | Type               | Params\n", "-----------------------------------------------------\n", "0 | model         | Sequential         | 55.1 K\n", "1 | val_accuracy  | MulticlassAccuracy | 0     \n", "2 | test_accuracy | MulticlassAccuracy | 0     \n", "-----------------------------------------------------\n", "55.1 K    Trainable params\n", "0         Non-trainable params\n", "55.1 K    Total params\n", "0.220     Total estimated model params size (MB)\n"]}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "f41a57de3dc441cf83c3917a96d37f22", "version_major": 2, "version_minor": 0}, "text/plain": ["Sanity Checking: 0it [00:00, ?it/s]"]}, "metadata": {}, "output_type": "display_data"}, {"name": "stderr", "output_type": "stream", "text": ["`Trainer.fit` stopped: `max_epochs=3` reached.\n"]}], "source": ["trainer.fit(model)"]}, {"cell_type": "markdown", "id": "0878bd72", "metadata": {"papermill": {"duration": 0.01378, "end_time": "2023-10-11T19:20:08.145011", "exception": false, "start_time": "2023-10-11T19:20:08.131231", "status": "completed"}, "tags": []}, "source": ["In Colab, you can use the TensorBoard magic function to view the logs that Lightning has created for you!"]}, {"cell_type": "code", "execution_count": 9, "id": "c54f8924", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T19:20:08.168663Z", "iopub.status.busy": "2023-10-11T19:20:08.168418Z", "iopub.status.idle": "2023-10-11T19:20:08.482524Z", "shell.execute_reply": "2023-10-11T19:20:08.481877Z"}, "papermill": {"duration": 0.326153, "end_time": "2023-10-11T19:20:08.484824", "exception": false, "start_time": "2023-10-11T19:20:08.158671", "status": "completed"}, "tags": []}, "outputs": [{"data": {"text/html": ["<div>\n", "<style scoped>\n", "    .dataframe tbody tr th:only-of-type {\n", "        vertical-align: middle;\n", "    }\n", "\n", "    .dataframe tbody tr th {\n", "        vertical-align: top;\n", "    }\n", "\n", "    .dataframe thead th {\n", "        text-align: right;\n", "    }\n", "</style>\n", "<table border=\"1\" class=\"dataframe\">\n", "  <thead>\n", "    <tr style=\"text-align: right;\">\n", "      <th></th>\n", "      <th>val_loss</th>\n", "      <th>val_acc</th>\n", "      <th>test_loss</th>\n", "      <th>test_acc</th>\n", "    </tr>\n", "    <tr>\n", "      <th>epoch</th>\n", "      <th></th>\n", "      <th></th>\n", "      <th></th>\n", "      <th></th>\n", "    </tr>\n", "  </thead>\n", "  <tbody>\n", "    <tr>\n", "      <th>0</th>\n", "      <td>0.427113</td>\n", "      <td>0.8896</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>1</th>\n", "      <td>0.317400</td>\n", "      <td>0.9096</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>2</th>\n", "      <td>0.272898</td>\n", "      <td>0.9174</td>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "    </tr>\n", "    <tr>\n", "      <th>3</th>\n", "      <td>NaN</td>\n", "      <td>NaN</td>\n", "      <td>0.25203</td>\n", "      <td>0.9228</td>\n", "    </tr>\n", "  </tbody>\n", "</table>\n", "</div>"], "text/plain": ["       val_loss  val_acc  test_loss  test_acc\n", "epoch                                        \n", "0      0.427113   0.8896        NaN       NaN\n", "1      0.317400   0.9096        NaN       NaN\n", "2      0.272898   0.9174        NaN       NaN\n", "3           NaN      NaN    0.25203    0.9228"]}, "metadata": {}, "output_type": "display_data"}, {"data": {"text/plain": ["<seaborn.axisgrid.FacetGrid at 0x7f2588433cd0>"]}, "execution_count": 9, "metadata": {}, "output_type": "execute_result"}, {"data": {"image/png": "iVBORw0KGgoAAAANSUhEUgAAAj4AAAHpCAYAAACC4TamAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAABK+UlEQVR4nO3deVyVZf7/8fdhOQeQTURRESU11AxxBZemLDGmxUmb0spcy6YpnZSa0hatLJmmby6pLdMvs82yTKsZHUtJnVLS0lxT3HcBcTkIIss59++PowdJMA6iKPfr+XjcDw/3ue77/lycjr677uu+b4thGIYAAABMwKu6CwAAALhUCD4AAMA0CD4AAMA0CD4AAMA0CD4AAMA0CD4AAMA0CD4AAMA0rojgYxiGcnJyxC2HAADAhbgigs+JEycUEhKiEydOVHcpAADgCnZFBB8AAICqQPABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACm4VPdBQBAuZwO159e3q4/cw9LJw5KRaekopNS8SmpKL/kT1uQ1KZv9dUL4LJH8AFQcU6nK2T8NnDUqisFRbjaHNkh7f+5jGBy0hVYvH2lP6aU7PP9P0mn7OfusyhfchZJvd+U2t7navvTO9KyV8qvL7wFwQfAeRF8gJoiL1sqyHGFi+L806Mi+SWvw5pKjTq42mZtkdbN+k3bM0Hl9LoHFpWMtLz1B+lwuuQoKPvYPcZKf3jc9XrX/6T/jCy/Tltw6eBzcK1UYC+/fVF+yWv/MCmogeTjJ/n6n/tnSNTv/ZYAmBzBB6hqjqLSIxeBEZKvn+u9Q+sl+75zRzbOvK7bQoq7x9U2e5v0zdNnvX9WSDnz5/BVUkgjV/vZ90t708qvq9OwkuBzbLe0fMr5+1GUL9kCT/ep8NzQ4+Uj+Qa4Qoe3tWR9aJTU9MazAkmAq/9nwok1sPR+7nxbsnj9pq2/60/fgNLtOz/sWgCgkgg+qPkMQyrM+03QODtA5EsN20lB9V3tdy6V9v90+v0yTr/UaSYlvexqm3NIevfm0vtzFpc+/gOLpahOrtc/TJI2zS2/1pa3lwSfopPStm/P37eiUyWvrYGSb62SwODjd1aI8HfVfUZYU6nzo6fb+pcEjTNtff0lH1tJ+/s+kyyW0u28y/nro3mia6moFrdUvC0AXCCCD65MTqeUf0zKzXQteYdLXucedv3D3Wvy6bbFUkrk+ffX90Ppmj+5Xm9bJKVNK79tw/Ylr728Jfve8tv6+LvmqZwRfrUU2fGscOF31mt/qf61JW1DG0t/mnZWu7MDzel1Z0Z7JOn+Oefv49nqxkh/nFDx9rWbVLwtAFzGCD64fBiGa5JrbtbpMJNV8jo3S2rdR7q6p6vtitelxePK31eteiXBx9tXsnhLhkPytpUxsuEnWWuVbNuoo9RuQOlAcvbpl8D6JW39w6QHU8uYbxLgGjGxWErXdePTrqUi/GtL7QdUrC0AoEIIPrj4CnJ/MyqTVfLzbZMkr9O3k5raQTq6o/z9hDYpCT6B9Vx/+oe55tAE1j39Z8TpK4walN52zD5XcPGqwK2rWvdxLRXh7eMKSgCAK0Klgs/06dP16quvKiMjQ3FxcZo6dari4+PLbFtUVKSUlBS9//77OnDggFq0aKFXXnlFf/zjHy+ocFSzolO/GZE5fYop/Grp2jtdbXYukz65VyrKK38/Nz0n1Qp3vfYPdf1pCzkryNRzjd4E1pOi/1Cy3bV/lmLvdo3mVMTZIzoAANPyOPjMnj1bycnJeuutt5SQkKDJkycrKSlJ6enpqlev3jntn332WX300Ud655131LJlS33zzTfq06ePVqxYoXbt2lVJJ1BFHEW/GZU5HWra3V8y8ffrv0mbviz/8uNrepcEH7/gktDjG1ASZALrlYzOnLlcWnJNoLXWcp0u+j1nT7wFAKCCLIZhGJ5skJCQoE6dOmnaNNfkT6fTqaioKI0YMUKjR48+p33Dhg31zDPP6NFHH3Wv+/Of/yx/f3999NFHFTpmTk6OQkJCZLfbFRwc7Em5cDqkk0dKhxlvXyn2Ltf7ednSzNtd7+cfLXsfg/4jXXV6tOWrR6VfTn9u3rbfBJl6rom7Z+alFJ2Scg643rMFlr1vAAAuIY9GfAoLC7V69WqNGTPGvc7Ly0uJiYlKSyv7/iEFBQXy8/Mrtc7f318//PBDuccpKChQQUHJPUNycnI8KbPmM4zTVzSdNfG3YTspvLnr/TUfSCv/5ToVlXdYMpyltw9vURJ8rIHS4c0l71m8S4eZWvVck2zPuP7vUtfHXO/7hZw7efdsvn6lL6EGAKCaeRR8srOz5XA4FBERUWp9RESEtmzZUuY2SUlJmjhxoq6//no1a9ZMqampmjt3rhwOR7nHSUlJ0QsvvOBJaVc+w3DddTf39Kmm/GNSq9tL3v98iHRsV8mozdmXSEvSLa+WBJ+CE1LmhrPetLjm0ZyZ+Ht2GPH1kwZ+7VofGOEKOeebAFw7+kJ7CgBAtbnoV3VNmTJFw4YNU8uWLWWxWNSsWTMNGTJEM2bMKHebMWPGKDk52f1zTk6OoqKu0FvRF+aVhBX/2q77p0iuZxl9P7FkxCYvy3WTPDeL9Fx2yU3iDq2Vju4svW//2iUTf89MEJakFre6RnXOjNwEhJd/szlJanpDVfQUAIDLnkfBJzw8XN7e3srMzCy1PjMzU/Xr1y9zm7p16+rLL7/UqVOndOTIETVs2FCjR49W06ZNyz2OzWaTzXYZT14tLigJK/ValzyOYMVUad+qkvdys6TC3JLtOg6Vbp/ken3KLqXPP3ff1qCS00yFuSVXOt38suu2/meudqpVt/wJvmFXuRYAAFCKR8HHarWqQ4cOSk1NVe/evSW5JjenpqZq+PDh593Wz89PkZGRKioq0hdffKG+fS+zJyg7iqWT2a5Qc+YutblZJaMyZ98Z+NRZVzT9dYUU0dr1ek9a2WHGx+/0BN+zJmZHtJZum1h6YnCtepI1oOz6Wt5aNf0EAMDEPD7VlZycrEGDBqljx46Kj4/X5MmTlZeXpyFDhkiSBg4cqMjISKWkuJ6+vHLlSh04cEBt27bVgQMH9Pzzz8vpdOrJJ5+s2p5U1PbF0o4l5162ffKIJENq3EUautDV1umQVr5Z9n68fF1h5ewnR7fr7zptdPbl2rXqSragcycBB9WXOj1wUboIAADK5nHw6devnw4fPqyxY8cqIyNDbdu21cKFC90Tnvfu3SuvsybHnjp1Ss8++6x27typwMBA3Xrrrfrwww8VGhpaZZ3wyJ608p/DZPFyhZ0zaoVL3Uaee/+ZWnVd82t+G2Za3nbRygYAABfO4/v4VIcqvY/P9lRpx3fnnmIKjJACwkrfUA8AANQo5gs+AADAtCrwxEYAAICageADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMg+ADAABMo1LBZ/r06YqOjpafn58SEhK0atWq87afPHmyWrRoIX9/f0VFRWnUqFE6depUpQoGAACoLI+Dz+zZs5WcnKxx48ZpzZo1iouLU1JSkrKysspsP2vWLI0ePVrjxo3T5s2b9e6772r27Nl6+umnL7h4AAAAT1gMwzA82SAhIUGdOnXStGnTJElOp1NRUVEaMWKERo8efU774cOHa/PmzUpNTXWve/zxx7Vy5Ur98MMPZR6joKBABQUF7p9zcnIUFRUlu92u4OBgT8oFAABw82jEp7CwUKtXr1ZiYmLJDry8lJiYqLS0tDK36dq1q1avXu0+HbZz504tWLBAt956a7nHSUlJUUhIiHuJiorypEwAAIAy+XjSODs7Ww6HQxEREaXWR0REaMuWLWVuc9999yk7O1vXXXedDMNQcXGxHn744fOe6hozZoySk5PdP58Z8QEAALgQF/2qrqVLl2rChAl64403tGbNGs2dO1fz58/X+PHjy93GZrMpODi41AIAAHChPBrxCQ8Pl7e3tzIzM0utz8zMVP369cvc5rnnntOAAQP04IMPSpJiY2OVl5enhx56SM8884y8vLiiHgAAXBoepQ6r1aoOHTqUmqjsdDqVmpqqLl26lLnNyZMnzwk33t7ekiQP51UDAABcEI9GfCQpOTlZgwYNUseOHRUfH6/JkycrLy9PQ4YMkSQNHDhQkZGRSklJkST16tVLEydOVLt27ZSQkKDt27frueeeU69evdwBCAAA4FLwOPj069dPhw8f1tixY5WRkaG2bdtq4cKF7gnPe/fuLTXC8+yzz8pisejZZ5/VgQMHVLduXfXq1Usvv/xy1fUCAACgAjy+j091yMnJUUhICPfxAQAAF4SZxQAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQIPgAAwDQqFXymT5+u6Oho+fn5KSEhQatWrSq3bffu3WWxWM5ZbrvttkoXDQAAUBkeB5/Zs2crOTlZ48aN05o1axQXF6ekpCRlZWWV2X7u3Lk6dOiQe9m4caO8vb119913X3DxAAAAnrAYhmF4skFCQoI6deqkadOmSZKcTqeioqI0YsQIjR49+ne3nzx5ssaOHatDhw6pVq1aZbYpKChQQUGB++ecnBxFRUXJbrcrODjYk3IBAADcPBrxKSws1OrVq5WYmFiyAy8vJSYmKi0trUL7ePfdd3XPPfeUG3okKSUlRSEhIe4lKirKkzIBAADK5FHwyc7OlsPhUERERKn1ERERysjI+N3tV61apY0bN+rBBx88b7sxY8bIbre7l3379nlSJgAAQJl8LuXB3n33XcXGxio+Pv687Ww2m2w22yWqCgAAmIVHIz7h4eHy9vZWZmZmqfWZmZmqX7/+ebfNy8vTp59+qgceeMDzKgEAAKqAR8HHarWqQ4cOSk1Nda9zOp1KTU1Vly5dzrvt559/roKCAt1///2VqxQAAOACeXyqKzk5WYMGDVLHjh0VHx+vyZMnKy8vT0OGDJEkDRw4UJGRkUpJSSm13bvvvqvevXurTp06VVM5AACAhzwOPv369dPhw4c1duxYZWRkqG3btlq4cKF7wvPevXvl5VV6ICk9PV0//PCDvv3226qpGgAAoBI8vo9PdcjJyVFISAj38QEAABeEZ3UBAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAADTIPgAAIBSoqOjNXny5Aq1tVgs+vLLLy9qPVWJ4AMAAEyD4AMAAEyD4AMAQAUYhqGThcXVshiGUeE6//Wvf6lhw4ZyOp2l1t9xxx0aOnSoduzYoTvuuEMREREKDAxUp06dtHjx4ir7PW3YsEE33XST/P39VadOHT300EPKzc11v7906VLFx8erVq1aCg0NVbdu3bRnzx5J0rp163TjjTcqKChIwcHB6tChg37++ecqq02SfKp0bwAA1FD5RQ5dM/abajn2ry8mKcBasX+y7777bo0YMUJLlixRjx49JElHjx7VwoULtWDBAuXm5urWW2/Vyy+/LJvNpg8++EC9evVSenq6GjdufEF15uXlKSkpSV26dNFPP/2krKwsPfjggxo+fLhmzpyp4uJi9e7dW8OGDdMnn3yiwsJCrVq1ShaLRZLUv39/tWvXTm+++aa8vb21du1a+fr6XlBNv0XwAQCgBqldu7ZuueUWzZo1yx185syZo/DwcN14443y8vJSXFycu/348eM1b948ff311xo+fPgFHXvWrFk6deqUPvjgA9WqVUuSNG3aNPXq1UuvvPKKfH19Zbfbdfvtt6tZs2aSpFatWrm337t3r/7+97+rZcuWkqSrr776guopS6WCz/Tp0/Xqq68qIyNDcXFxmjp1quLj48ttf/z4cT3zzDOaO3eujh49qiZNmmjy5Mm69dZbK104AACXkr+vt359Manaju2J/v37a9iwYXrjjTdks9n08ccf65577pGXl5dyc3P1/PPPa/78+Tp06JCKi4uVn5+vvXv3XnCdmzdvVlxcnDv0SFK3bt3kdDqVnp6u66+/XoMHD1ZSUpJ69uypxMRE9e3bVw0aNJAkJScn68EHH9SHH36oxMRE3X333e6AVFU8nuMze/ZsJScna9y4cVqzZo3i4uKUlJSkrKysMtsXFhaqZ8+e2r17t+bMmaP09HS98847ioyMvODiAQC4VCwWiwKsPtWynDkVVFG9evWSYRiaP3++9u3bp++//179+/eXJD3xxBOaN2+eJkyYoO+//15r165VbGysCgsLL8av7Rzvvfee0tLS1LVrV82ePVsxMTH68ccfJUnPP/+8Nm3apNtuu03fffedrrnmGs2bN69Kj+/xiM/EiRM1bNgwDRkyRJL01ltvaf78+ZoxY4ZGjx59TvsZM2bo6NGjWrFihfs8XXR09HmPUVBQoIKCAvfPOTk5npYJAIBp+fn56c4779THH3+s7du3q0WLFmrfvr0kafny5Ro8eLD69OkjScrNzdXu3bur5LitWrXSzJkzlZeX5x71Wb58uby8vNSiRQt3u3bt2qldu3YaM2aMunTpolmzZqlz586SpJiYGMXExGjUqFG699579d5777lrrQoejfgUFhZq9erVSkxMLNmBl5cSExOVlpZW5jZff/21unTpokcffVQRERG69tprNWHCBDkcjnKPk5KSopCQEPcSFRXlSZkAAJhe//793QMTZ0Z7JNe8mblz52rt2rVat26d7rvvvnOuALuQY/r5+WnQoEHauHGjlixZohEjRmjAgAGKiIjQrl27NGbMGKWlpWnPnj369ttvtW3bNrVq1Ur5+fkaPny4li5dqj179mj58uX66aefSs0BqgoeBZ/s7Gw5HA5FRESUWh8REaGMjIwyt9m5c6fmzJkjh8OhBQsW6LnnntNrr72ml156qdzjjBkzRna73b3s27fPkzIBADC9m266SWFhYUpPT9d9993nXj9x4kTVrl1bXbt2Va9evZSUlOQeDbpQAQEB+uabb3T06FF16tRJd911l3r06KFp06a539+yZYv+/Oc/KyYmRg899JAeffRR/eUvf5G3t7eOHDmigQMHKiYmRn379tUtt9yiF154oUpqO8NieHBzgIMHDyoyMlIrVqxQly5d3OuffPJJLVu2TCtXrjxnm5iYGJ06dUq7du2St7drctbEiRP16quv6tChQxU6bk5OjkJCQmS32xUcHFzRcgEAAErxaI5PeHi4vL29lZmZWWp9Zmam6tevX+Y2DRo0kK+vrzv0SK5zgBkZGSosLJTVaq1E2QAAAJ7z6FSX1WpVhw4dlJqa6l7ndDqVmppaagTobN26ddP27dtLnT/cunWrGjRoQOgBAOAy9vHHHyswMLDMpXXr1tVdXqV4fFVXcnKyBg0apI4dOyo+Pl6TJ09WXl6e+yqvgQMHKjIyUikpKZKkv/71r5o2bZoee+wxjRgxQtu2bdOECRP0t7/9rWp7AgAAqtSf/vQnJSQklPleVd9R+VLxOPj069dPhw8f1tixY5WRkaG2bdtq4cKF7gnPe/fulZdXyUBSVFSUvvnmG40aNUpt2rRRZGSkHnvsMT311FNV1wsAAFDlgoKCFBQUVN1lVCmPJjdXFyY3AwCAqsDT2QEAgGkQfAAAgGkQfAAAgGkQfAAAgGkQfAAAgGkQfAAAQCnR0dGaPHlydZdxURB8AACAaRB8AACAaRB8AADwRGHe+RdHcUnb4sLzty3KL2lrGGW38dC//vUvNWzYsNQzMiXpjjvu0NChQ7Vjxw7dcccdioiIUGBgoDp16qTFixdX9rehiRMnKjY2VrVq1VJUVJQeeeQR5ebmlmqzfPlyde/eXQEBAapdu7aSkpJ07NgxSa5nfv7zn/9U8+bNZbPZ1LhxY7388suVruf3ePzICgAATG1Cw/O/f/dMqXUf1+vvXpRWTC2/bcN20kNLXa9PHpFebXZum+ftHpV39913a8SIEVqyZIl69OghSTp69KgWLlyoBQsWKDc3V7feeqtefvll2Ww2ffDBB+rVq5fS09PVuHFjj44lSV5eXnr99dd11VVXaefOnXrkkUf05JNP6o033pAkrV27Vj169NDQoUM1ZcoU+fj4aMmSJXI4HJKkMWPG6J133tGkSZN03XXX6dChQ9qyZYvHdVQUwQcAgBqkdu3auuWWWzRr1ix38JkzZ47Cw8N14403ysvLS3Fxce7248eP17x58/T1119r+PDhHh9v5MiR7tfR0dF66aWX9PDDD7uDzz//+U917NjR/bMk95PdT5w4oSlTpmjatGkaNGiQJKlZs2a67rrrPK6jogg+AAB44umD53/f21by+qaxUvcx5be1nDXjJKDO7++7gvr3769hw4bpjTfekM1m08cff6x77rlHXl5eys3N1fPPP6/58+fr0KFDKi4uVn5+vvbu3VupYy1evFgpKSnasmWLcnJyVFxcrFOnTunkyZMKCAjQ2rVrdffdd5e57ebNm1VQUOAOaJcCc3wAAPCEtdb5F++zxhR8rOdv6+tf0tZiKbtNJfTq1UuGYWj+/Pnat2+fvv/+e/Xv31+S9MQTT2jevHmaMGGCvv/+e61du1axsbEqLCz0+Di7d+/W7bffrjZt2uiLL77Q6tWrNX36dEly78/f37/c7c/33sVC8AEAoIbx8/PTnXfeqY8//liffPKJWrRoofbt20tyTTQePHiw+vTpo9jYWNWvX1+7d++u1HFWr14tp9Op1157TZ07d1ZMTIwOHiw9atWmTRulpqaWuf3VV18tf3//ct+/GDjVBQBADdS/f3/dfvvt2rRpk+6//373+quvvlpz585Vr169ZLFY9Nxzz51zBVhFNW/eXEVFRZo6dap69eql5cuX66233irVZsyYMYqNjdUjjzyihx9+WFarVUuWLNHdd9+t8PBwPfXUU3ryySdltVrVrVs3HT58WJs2bdIDDzxwQf0vDyM+AADUQDfddJPCwsKUnp6u++67z71+4sSJql27trp27apevXopKSnJPRrkqbi4OE2cOFGvvPKKrr32Wn388cdKSUkp1SYmJkbffvut1q1bp/j4eHXp0kVfffWVfHxcYy/PPfecHn/8cY0dO1atWrVSv379lJWVVfmO/w6LYRjGRdt7FcnJyVFISIjsdruCg4OruxwAAHCFYsQHAACYBsEHAACU6eOPP1ZgYGCZy5l78VxpONUFAADKdOLECWVmZpb5nq+vr5o0aXKJK7pwXNUFAADKFBQUpKCgoOouo0pxqgsAAJgGwQcAAJgGwQcAAJgGwQcAAJgGwQcAAJgGwQcAAFSp3bt3y2KxaO3atdVdyjkIPgAA1DDdu3fXyJEjq2x/gwcPVu/evatsf9WJ4AMAAEyD4AMAgAdOFp3UyaKTOvPgg/zifJ0sOimH0yFJKnAU6GTRSRU5iyRJRY4inSw6qUJHoSSp2Fmsk0Undar4lCTJaTjd+yzvGJ4YPHiwli1bpilTpshischisWj37t3auHGjbrnlFgUGBioiIkIDBgxQdna2e7s5c+YoNjZW/v7+qlOnjhITE5WXl6fnn39e77//vr766iv3/pYuXepxXcuWLVN8fLxsNpsaNGig0aNHq7i4+HePL0lLly5VfHy8atWqpdDQUHXr1k179uzxuAaJ4AMAgEcSZiUoYVaCjhUckyTd+597lTArQWuy1kiSxnw/RgmzEjRn6xxJ0jsb3lHCrAT986d/SpJS96YqYVaC/rr4r5Kkncd3KmFWgv74xR/LPYYnpkyZoi5dumjYsGE6dOiQDh06pKCgIN10001q166dfv75Zy1cuFCZmZnq27evJOnQoUO69957NXToUG3evFlLly7VnXfeKcMw9MQTT6hv37764x//6N5f165dParpwIEDuvXWW9WpUyetW7dOb775pt5991299NJLv3v84uJi9e7dWzfccIPWr1+vtLQ0PfTQQ7JYLB7/biQeWQEAQI0SEhIiq9WqgIAA1a9fX5L00ksvqV27dpowYYK73YwZMxQVFaWtW7cqNzdXxcXFuvPOO93P34qNjXW39ff3V0FBgXt/nnrjjTcUFRWladOmyWKxqGXLljp48KCeeuopjR07VocOHSr3+EePHpXdbtftt9+uZs2aSZJatWpVqTokgg8AAB5Zed9KSZK/j78k6ZPbP5FhGLJ52yRJKX9I0UvdXpKvt68kaVjsMA1uPVg+Xq5/cns07qGV962Ul8V10qVpaFP3Pss7xoVat26dlixZosDAwHPe27Fjh26++Wb16NFDsbGxSkpK0s0336y77rpLtWvXrpLjb968WV26dCk1StOtWzfl5uZq//79iouLK/f4YWFhGjx4sJKSktSzZ08lJiaqb9++atCgQaVq4VQXAAAeCPANUIBvgPsfcX8ffwX4Bsjby1uSZPO2KcA3QL5eruDj6+2rAN8AWb2tkiQfLx8F+AbIz8dPkuRl8XLvs7xjXKjc3Fz16tVLa9euLbVs27ZN119/vby9vbVo0SL997//1TXXXKOpU6eqRYsW2rVrV5Uc//f83vHfe+89paWlqWvXrpo9e7ZiYmL0448/VupYBB8AAGoYq9Uqh8Ph/rl9+/batGmToqOj1bx581JLrVq1JEkWi0XdunXTCy+8oF9++UVWq1Xz5s0rc3+eatWqldLS0kpN1l6+fLmCgoLUqFGj3z2+JLVr105jxozRihUrdO2112rWrFmVqoXgAwBADRMdHa2VK1dq9+7dys7O1qOPPqqjR4/q3nvv1U8//aQdO3bom2++0ZAhQ+RwOLRy5UpNmDBBP//8s/bu3au5c+fq8OHD7rk00dHRWr9+vdLT05Wdna2ioiKP6nnkkUe0b98+jRgxQlu2bNFXX32lcePGKTk5WV5eXuc9/q5duzRmzBilpaVpz549+vbbb7Vt27bKz/MxrgB2u92QZNjt9uouBQCAy156errRuXNnw9/f35Bk7Nq1y9i6davRp08fIzQ01PD39zdatmxpjBw50nA6ncavv/5qJCUlGXXr1jVsNpsRExNjTJ061b2/rKwso2fPnkZgYKAhyViyZMl5j79r1y5DkvHLL7+41y1dutTo1KmTYbVajfr16xtPPfWUUVRUZBiGcd7jZ2RkGL179zYaNGhgWK1Wo0mTJsbYsWMNh8NRqd+NxTAqcZOASywnJ0chISGy2+0KDg6u7nIAAMAVilNdAADANAg+AADAIxMmTFBgYGCZyy233FLd5Z0Xp7oAAIBHjh49qqNHj5b5nr+/vyIjIy9xRRXHDQwBAIBHwsLCFBYWVt1lVAqnugAAgGkQfAAAgGkQfAAAgGkQfAAAgGlUKvhMnz5d0dHR8vPzU0JCglatWlVu25kzZ8pisZRa/Pz8Kl0wAABAZXkcfGbPnq3k5GSNGzdOa9asUVxcnJKSkpSVlVXuNsHBwTp06JB72bNnzwUVDQAAUBkeB5+JEydq2LBhGjJkiK655hq99dZbCggI0IwZM8rdxmKxqH79+u4lIiLigooGAADl6969u0aOHFll+xs8eLB69+5dZfurTh4Fn8LCQq1evVqJiYklO/DyUmJiotLS0srdLjc3V02aNFFUVJTuuOMObdq06bzHKSgoUE5OTqkFAADgQnkUfLKzs+VwOM4ZsYmIiFBGRkaZ27Ro0UIzZszQV199pY8++khOp1Ndu3bV/v37yz1OSkqKQkJC3EtUVJQnZQIAcNE4T54sczEcDkmS4XSWWn+GUVRU9rb5+SX7LiwsWV9YWKn6Bg8erGXLlmnKlCnuubW7d+/Wxo0bdcsttygwMFAREREaMGCAsrOz3dvNmTNHsbGx8vf3V506dZSYmKi8vDw9//zzev/99/XVV1+597d06dLfreOpp55STEyMAgIC1LRpUz333HMqKioq1ebf//63OnXqJD8/P4WHh6tPnz7u9woKCvTUU08pKipKNptNzZs317vvvlup38nZLvqdm7t06aIuXbq4f+7atatatWqlt99+W+PHjy9zmzFjxig5Odn9c05ODuEHAHBZSG/focz1jd9/X7US4lW0d692/PH086p8fNRq4wZJ0rHPPlPm+JfO2c7avJma/ec/kqSs//s/HfvgQ0lS7YEDVP/ppz2ub8qUKdq6dauuvfZavfjii5IkX19fxcfH68EHH9SkSZOUn5+vp556Sn379tV3332nQ4cO6d5779U///lP9enTRydOnND3338vwzD0xBNPaPPmzcrJydF7770nSRW6a3NQUJBmzpyphg0basOGDRo2bJiCgoL05JNPSpLmz5+vPn366JlnntEHH3ygwsJCLViwwL39wIEDlZaWptdff11xcXHatWtXqaBWWR4Fn/DwcHl7eyszM7PU+szMTNWvX79C+/D19VW7du20ffv2ctvYbDbZbDZPSgMAAJJCQkJktVoVEBDg/rf5pZdeUrt27TRhwgR3uxkzZigqKkpbt25Vbm6uiouLdeedd6pJkyaSpNjYWHdbf39/FRQUVPjfekl69tln3a+jo6P1xBNP6NNPP3UHn5dffln33HOPXnjhBXe7uLg4SdLWrVv12WefadGiRe7pNU2bNvX0V1Emj4KP1WpVhw4dlJqa6p7k5HQ6lZqaquHDh1doHw6HQxs2bNCtt97qcbEAAFS3FmtWl7necvp/2H0bNy6zTe2+fRV61qmckg0t7pf1nnhC9c5MSvapupMy69at05IlSxQYGHjOezt27NDNN9+sHj16KDY2VklJSbr55pt11113qXbt2pU+5uzZs/X6669rx44d7mB19oPG165dq2HDhpW57dq1a+Xt7a0bbrih0scvj8e/1eTkZA0aNEgdO3ZUfHy8Jk+erLy8PA0ZMkSSa2gqMjJSKSkpkqQXX3xRnTt3VvPmzXX8+HG9+uqr2rNnjx588MGq7QkAAJeAV0DAed+3eHnJUkYbi6+vLL6+59+31SpZrRdUX1lyc3PVq1cvvfLKK+e816BBA3l7e2vRokVasWKFvv32W02dOlXPPPOMVq5cqauuusrj46Wlpal///564YUXlJSUpJCQEH366ad67bXX3G38/f3L3f58710oj4NPv379dPjwYY0dO1YZGRlq27atFi5c6J7wvHfvXnl5lcyZPnbsmIYNG6aMjAzVrl1bHTp00IoVK3TNNddUXS8AAICb1WqV4/Rka0lq3769vvjiC0VHR8unnJEki8Wibt26qVu3bho7dqyaNGmiefPmKTk5+Zz9/Z4VK1aoSZMmeuaZZ9zrfnsPvzZt2ig1NdU9cHK22NhYOZ1OLVu2rNSV5FXCuALY7XZDkmG326u7FAAALnvDhg0zOnXqZOzatcs4fPiwceDAAaNu3brGXXfdZaxatcrYvn27sXDhQmPw4MFGcXGx8eOPPxovv/yy8dNPPxl79uwxPvvsM8NqtRoLFiwwDMMwXn75ZaNx48bGli1bjMOHDxuFhYXnPf5XX31l+Pj4GJ988omxfft2Y8qUKUZYWJgREhLibrNkyRLDy8vLGDt2rPHrr78a69evN/7xj3+43x88eLARFRVlzJs3z9i5c6exZMkSY/bs2Rf8uyH4AABQw6SnpxudO3c2/P39DUnGrl27jK1btxp9+vQxQkNDDX9/f6Nly5bGyJEjDafTafz6669GUlKSUbduXcNmsxkxMTHG1KlT3fvLysoyevbsaQQGBhqSjCVLlvxuDX//+9+NOnXqGIGBgUa/fv2MSZMmlQo+hmEYX3zxhdG2bVvDarUa4eHhxp133ul+Lz8/3xg1apTRoEEDw2q1Gs2bNzdmzJhxwb8bi2EYRtWOIVW9nJwchYSEyG63l5oYBQAA4Amezg4AAEyD4AMAADwyYcIEBQYGlrnccsst1V3eeXGqCwAAeOTo0aM6evRome/5+/srMjLyEldUcRf9kRUAAKBmCQsLq9BjKy5HnOoCAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmYargU+Rwav+xk9VdBgAAqCamCj5z1+zXjf+3VE/P26CDx/OruxwAAHCJmSr4/Lz7mIochmat3Kvury7V2K82KsN+qrrLAgAAl4jFMAyjuov4PTk5OQoJCZHdbldwcPAF7WvlziOauGirVu46Kkmy+njpvvjGeqR7M9UL9quKcgEAwGXKdMHnjBU7sjVp0Vb9tPuYJMnm46UBnZvoLzc0U90gW5UcAwAAXF5MG3wkyTAM/bA9WxMXbdUve49Lkvx9vTWwaxP95fpmCqtlrbJjAQCA6mfq4HOGYRhatvWwJi3aqnX77ZKkAKu3BneN1rA/NFVtAhAAADUCwecshmHouy1ZmrR4qzYeyJEkBdp8NLRbtB64rqlCAnwv2rEBAMDFR/Apg2EYWvRrpiYt3qbNh1wBKMjPRw9cd5WGXneVgv0IQAAAXIkIPufhdBr6ZlOGJi3eqq2ZuZKkYD8fPXR9Uw3udpUCbT6XrBYAAHDhCD4V4HQaWrDxkCYv3qbtWa4AFBrgq4eub6pBXaJViwAEAMAVgeDjAYfT0H/WH9SUxdu0MztPkhRWy6qHb2iqAZ2j5W/1rrbaAADA7yP4VEKxw6mv1x3UlNRt2nPE9eyv8ECb/tq9mfonNJafLwEIAIDLEcHnAhQ7nJr7ywG9nrpN+4+5nv1VL8imR7o30z3xBCAAAC43lXpW1/Tp0xUdHS0/Pz8lJCRo1apVFdru008/lcViUe/evStz2MuOj7eX+naM0pInuivlzlhFhvor60SBnv/3r+r+6lJ9+OMeFRQ7qrtMAABwmscjPrNnz9bAgQP11ltvKSEhQZMnT9bnn3+u9PR01atXr9ztdu/ereuuu05NmzZVWFiYvvzyywof83Id8fmtgmKHPvt5v6Z/t10ZOa6Hn0aG+mv4Tc11V4dG8vU21TNhAQC47HgcfBISEtSpUydNmzZNkuR0OhUVFaURI0Zo9OjRZW7jcDh0/fXXa+jQofr+++91/PjxGhl8zjhV5NDsn/Zp+pLtyjpRIElqVNtff7vpavVpH0kAAgCgmnj0L3BhYaFWr16txMTEkh14eSkxMVFpaWnlbvfiiy+qXr16euCBByp0nIKCAuXk5JRariR+vt4a1DVa/3vyRj13+zUKD7Rp/7F8PfnFeiVOXKYvVu9XscNZ3WUCAGA6HgWf7OxsORwORURElFofERGhjIyMMrf54Ycf9O677+qdd96p8HFSUlIUEhLiXqKiojwp87Lh5+utB667St8/eaOeubWV6tSyas+Rk3r883XqOel/+vKXA3I4L/u55QAA1BgX9ZzLiRMnNGDAAL3zzjsKDw+v8HZjxoyR3W53L/v27buIVV58/lZvDbu+qf735I166o8tFRrgq13ZeRo5e61unrRM/153UE4CEAAAF51HtxwODw+Xt7e3MjMzS63PzMxU/fr1z2m/Y8cO7d69W7169XKvczpdp3h8fHyUnp6uZs2anbOdzWaTzWbzpLQrQi2bj/7avZkGdGmi91fs1r/+t1M7DudpxCe/aOp32zQyMUZ/bF1fXl6W6i4VAIAaqVKTm+Pj4zV16lRJriDTuHFjDR8+/JzJzadOndL27dtLrXv22Wd14sQJTZkyRTExMbJarb97zCttcnNF5Zwq0ns/7Nb/+2GnTpwqliS1rB+kUT1jdPM1EbJYCEAAAFSlSl3OPmjQIL399tuKj4/X5MmT9dlnn2nLli2KiIjQwIEDFRkZqZSUlDK3Hzx4cI2/qstT9vwivfvDLs34YZdyC1wBqHXDYI1KjFGPVvUIQAAAVBGPn67Zr18/HT58WGPHjlVGRobatm2rhQsXuic87927V15eXK7tiRB/XyX3jNHQbtF65/udem/5bm06mKMHP/hZbRqFaFTPGHWPqUsAAgDgAvHIisvQ0bxC/et/O/X+it3KL3Ld+bld41CNSozRH64OJwABAFBJBJ/LWHZugd5etkMfpO1RQbFrUnjHJrWV3DNGXZrVIQABAOAhgs8VICvnlN5ctkMfr9yrwtMBKOGqMI3qGaPOTetUc3UAAFw5CD5XkAz7Kb25dLs+WbVPhafv/NyteR2NSoxRx+iwaq4OAIDLH8HnCnTweL6mL9muz37epyKH6+P7w9XhGtUzRu0b167m6gAAuHwRfK5g+4+d1PQl2/X5z/tVfPrOzze2qKtRPWPUplFo9RYHAMBliOBTA+w9clJTv9umuWc9+yuxVT2NTIzRtZEh1VwdAACXD4JPDbI7O0+vf7dNX/5yQGce/ZXUOkIjE2PUqgG/NwAACD410I7DuXo9dZu+XndQZz7d22Ib6LHEqxUTEVS9xQEAUI0IPjXYtswTmpy6TfPXH5IkWSzS7W0a6rEeV6t5vcBqrg4AgEuP4GMCWzJyNGXxNv13Y4Ykycsi3dE2Un/rcbWuCq9VzdUBAHDpEHxMZNNBuyYv3qZFv2ZKcgWgPu0a6W89mqtJHQIQAKDmI/iY0Ib9dk1avFXfbcmSJHl7WXRX+0YaflNzRYUFVHN1AABcPAQfE1u777gmLdqqZVsPS5J8vCy6u2OUht/UXJGh/tVcHQAAVY/gA63ec1STFm3TD9uzJUm+3hbd06mxHrmxmRqEEIAAADUHwQduq3Yd1aRFW5W284gkyerjpfviG+uR7s1UL9ivmqsDAODCEXxwjhU7sjV50Tat2n1UkmTz8dL9nZvo4RuaqW6QrZqrAwCg8gg+KJNhGFq+/YgmLkrXmr3HJUl+vl4a1CVaD13fVHUCCUAAgCsPwQfnZRiG/rctWxMXbdW6fcclSQFWbw3qGq2H/tBUtWtZq7dAAAA8QPBBhRiGoSXpWZq0aJs2HLBLkmpZvTX0uqv04HVNFRLgW80VAgDw+wg+8IhhGFq8OUsTF23V5kM5kqQgm48e+MNVGnrdVQr2IwABAC5fBB9UitNp6NtfMzRp0TalZ56QJAX7+WjYH5pqcLdoBRGAAACXIYIPLojTaei/GzM0efFWbcvKlSSFBvjqoeubalCXaNWy+VRzhQAAlCD4oEo4nIb+s/6gpqRu087DeZKksFpW/eX6phrQpYkCrAQgAED1I/igSjmchr5ed0BTFm/T7iMnJUnhgVY9fEMz3d+5ifx8vau5QgCAmRF8cFEUO5ya98sBvf7dNu07mi9Jqhtk0yPdm+ne+MYEIABAtSD44KIqcjg1d81+vZ66XQeOuwJQ/WA/PXpjM/XtFCWbDwEIAHDpEHxwSRQWO/X56n2a9t12HbKfkiQ1DPHTozc1190domT18armCgEAZkDwwSVVUOzQ7J/2afqS7crMKZAkRYb66289muvO9o3k600AAgBcPAQfVItTRQ7NWrlXbyzdoexcVwBqHBagv/W4Wr3bNpQPAQgAcBEQfFCt8gsd+njlHr25dIeO5BVKkq4Kr6W/9WiuP8VFytvLUs0VAgBqEoIPLgsnC4v1Qdoevb1sh46dLJIkNatbS48lxui22AYEIABAlSD44LKSW1Cs91fs1r/+t1P2fFcAurpeoEYmxuiWa+vLiwAEALgABB9clk6cKtJ7y3frne936sSpYklSy/pBGpkYo6TWEbJYCEAAAM8RfHBZs+cXacYPuzTjh106UeAKQK0bBmtkYowSW9UjAAEAPELwwRXh+MlC/b/vd+m95buUV+iQJLVpFKJRiTHq3qIuAQgAUCEEH1xRjuYV6p3vd2rm8t3KL3IFoLZRoUruGaM/XB1OAAIAnBfBB1ek7NwC/et/O/VB2m6dKnJKkjo2qa1RPWPUtVkdAhAAoEwEH1zRsk6c0ltLd+rjlXtUUOwKQPFXhSm5Z4w6N61TzdUBAC43BB/UCJk5p/Tm0h2atXKvCh2uANS1WR2N6hmjTtFh1VwdAOByQfBBjXLInq/pS7Zr9k/7VORw/af9h6vDdUfbSLVpFKJmdQO5GSIAmBjBBzXS/mMnNX3JDn3+8z4VO0v+E69l9VbryBC1iQxRm6hQxTUKUeOwAOYEAYBJEHxQo+07elIf/bhHv+w9ro0H7Tp5+lL4s4X4+6pNoxC1aRSi2MhQxUWFqH6wH2EIAGoggg9Mw+E0tONwrtbtO64NB+xat9+uzQdz3HOCzlY3yOYaFWoU6g5FdQJt1VA1AKAqEXxgaoXFTm3NPKF1+49r/T671h+wa2vmCTmc534tIkP9T4cg1ymyaxuFKNjPtxqqBgBUFsEH+I38Qod+PWTX+v2uZd3+49p5OK/Mtk3Da7lOkZ0OQ60bhsjf6n2JKwYAVBTBB6iAE6eKtOGAKwhtOB2G9h/LP6edl0WKiQhyjwy1aRSilvWDZfXxqoaqAQC/RfABKulIboE7DLmW48o6UXBOO6u3l1o2KB2Grq4XxGX1AFANKhV8pk+frldffVUZGRmKi4vT1KlTFR8fX2bbuXPnasKECdq+fbuKiop09dVX6/HHH9eAAQMqfDyCD64UGfZTWr//uPsU2YYDdh0/WXROO39fb10bGey+iqxNo1A1CQuQF2EIAC4qj4PP7NmzNXDgQL311ltKSEjQ5MmT9fnnnys9PV316tU7p/3SpUt17NgxtWzZUlarVf/5z3/0+OOPa/78+UpKSqrQMQk+uFIZhqF9R/PdIWjdvuPaeMDufsL82YL8fEouqW/kus9QwxAuqweAquRx8ElISFCnTp00bdo0SZLT6VRUVJRGjBih0aNHV2gf7du312233abx48dXqD3BBzWJw2loV3au1u1znR5bf8CuTQdzVFh87mX14YFWxZa6rD5UdYO4rB4AKsuj4FNYWKiAgADNmTNHvXv3dq8fNGiQjh8/rq+++uq82xuGoe+++05/+tOf9OWXX6pnz55ltisoKFBBQclciZycHEVFRRF8UGMVOZxKzzhxes6Q61RZesaJUnedPqNhiJ9i3ZfVhyo2MkQhAVxWDwAV4eNJ4+zsbDkcDkVERJRaHxERoS1btpS7nd1uV2RkpAoKCuTt7a033nij3NAjSSkpKXrhhRc8KQ24ovl6e+nayBBdGxmie+MbS5JOFTn066Ec91Vk6/fbteNwrg7aT+mg/ZS+2ZTp3j66ToD7kvo2jULVumGwatk8+noDgClckr8Zg4KCtHbtWuXm5io1NVXJyclq2rSpunfvXmb7MWPGKDk52f3zmREfwEz8fL3VvnFttW9c270ut6BYG88aFVq/3669R09q9xHX8u91ByW5LqtvXi+w1CmyVg2CZPPhHkMAzM2j4BMeHi5vb29lZmaWWp+Zman69euXu52Xl5eaN28uSWrbtq02b96slJSUcoOPzWaTzcY8BuC3Am0+6ty0jjo3reNedyyvsNQpsvX77crIOaWtmbnampmrOav3S5J8vS1qUT/Ifefp2MhQxUQEysebewwBMA+Pgo/ValWHDh2UmprqnuPjdDqVmpqq4cOHV3g/Tqez1BweAJVXu5ZV18fU1fUxdd3rsnJOue8ttO70n8dOFmnjgRxtPJCjWStd7fx8vdS6YYhiI0Pcl9VfVacWl9UDqLE8PtWVnJysQYMGqWPHjoqPj9fkyZOVl5enIUOGSJIGDhyoyMhIpaSkSHLN1+nYsaOaNWumgoICLViwQB9++KHefPPNqu0JALd6wX5KvMZPide45uMZhqH9x/JdYeiA67lkGw/YdaKgWKv3HNPqPcfc2wbZfHRtZEipGy42qu3PZfUAagSPg0+/fv10+PBhjR07VhkZGWrbtq0WLlzonvC8d+9eeXmVDJ3n5eXpkUce0f79++Xv76+WLVvqo48+Ur9+/aquFwDOy2KxKCosQFFhAbqtTQNJktNpaNeRPNeo0D67Nhywa9NBVxhK23lEaTuPuLcPq+W6rD7urOeS1Qv2q67uAECl8cgKAG7FDqe2ZeW6T5Ft2G/XlowcFTnO/WuifrDrsvozYahNZIhq17JWQ9UAUHEEHwDndarIoS0ZJ7ThrPlC27NyVcYthtQ4LMAdhto0CtW1kSEK5LJ6AJcRgg8Aj+UVFGvTwZyzriQ7rt1HTp7TzmKRmtUNdM0XinQ9huOaBsHy8+WyegDVg+ADoErYTxa5nke2/7jW7z+uDfvtOmg/dU47Hy+LYiKCFBfluqS+TaMQtagfJF8uqwdwCRB8AFw0h08UaMOB4yXPJdtv15G8wnPaWX28dE2DYPcpsjaNQtS0bqC8uaweQBUj+AC4ZAzD0EH7Ka3f53o465kwdOJU8Tlta1m9S11WH9coVFFhXFYP4MIQfABUK6fT0J6jJ8+6rP64Nh7IUX6R45y2oQG+py+rDz09iTpU9UO4rB5AxRF8AFx2ih1ObT+c6544vX6/XZsPlX1Zfb0gm3tU6EwYCuOyegDlIPgAuCIUFDuUnnGiVBjamnmizMvqG9X2V/vGtTXlnracGgNQCjfYAHBFsPl4n574HCqpiSQpv9ChTQftp2+26ApDO7PztP9YvoL8fAk9AM5B8AFwxfK3eqtjdJg6Roe519nzi7TpgF0FDmc1VgbgckXwAVCjhPj7qmvz8OouA8BlijuGAQAA0yD4AAAA0yD4AAAA0yD4AAAA0yD4AAAA0yD4AAAA0yD4AAAA0yD4AAAA0yD4AAAA0yD4AAAA0yD4AAAA0yD4AAAA0yD4AAAA0yD4AAAA0yD4AAAA0yD4AAAA0yD4AAAA0yD4AAAA0yD4AAAA0yD4AAAA0yD4AAAA0yD4AAAA0yD4AAAA0yD4AAAA0yD4AAAA0yD4AAAA0yD4AAAA0yD4AAAA0/Cp7gIqwjAMSVJOTk41VwIAMIugoCBZLJbqLgNV7IoIPidOnJAkRUVFVXMlAACzsNvtCg4Oru4yUMUsxpnhlMuY0+nUwYMHLzh95+TkKCoqSvv27TPdf8z03Xx9N2u/JfP23az9li5O3xnxqZmuiBEfLy8vNWrUqMr2FxwcbLq/FM6g7+bru1n7LZm372btt2TuvqNimNwMAABMg+ADAABMw1TBx2azady4cbLZbNVdyiVH383Xd7P2WzJv383ab8ncfYdnrojJzQAAAFXBVCM+AADA3Ag+AADANAg+AADANAg+AADANGpc8Jk+fbqio6Pl5+enhIQErVq16rztP//8c7Vs2VJ+fn6KjY3VggULLlGlVc+Tvs+cOVMWi6XU4ufndwmrrRr/+9//1KtXLzVs2FAWi0Vffvnl726zdOlStW/fXjabTc2bN9fMmTMvep0Xg6d9X7p06TmfucViUUZGxqUpuIqkpKSoU6dOCgoKUr169dS7d2+lp6f/7nZX+ne9Mv2uKd/zN998U23atHHfnLBLly7673//e95trvTPGxdPjQo+s2fPVnJyssaNG6c1a9YoLi5OSUlJysrKKrP9ihUrdO+99+qBBx7QL7/8ot69e6t3797auHHjJa78wnnad8l1h9NDhw65lz179lzCiqtGXl6e4uLiNH369Aq137Vrl2677TbdeOONWrt2rUaOHKkHH3xQ33zzzUWutOp52vcz0tPTS33u9erVu0gVXhzLli3To48+qh9//FGLFi1SUVGRbr75ZuXl5ZW7TU34rlem31LN+J43atRI//jHP7R69Wr9/PPPuummm3THHXdo06ZNZbavCZ83LiKjBomPjzceffRR988Oh8No2LChkZKSUmb7vn37GrfddlupdQkJCcZf/vKXi1rnxeBp39977z0jJCTkElV3aUgy5s2bd942Tz75pNG6detS6/r162ckJSVdxMouvor0fcmSJYYk49ixY5ekpkslKyvLkGQsW7as3DY16bt+RkX6XRO/52fUrl3b+H//7/+V+V5N/LxRdWrMiE9hYaFWr16txMRE9zovLy8lJiYqLS2tzG3S0tJKtZekpKSkcttfrirTd0nKzc1VkyZNFBUVdd7/e6pJaspnfiHatm2rBg0aqGfPnlq+fHl1l3PB7Ha7JCksLKzcNjXxc69Iv6Wa9z13OBz69NNPlZeXpy5dupTZpiZ+3qg6NSb4ZGdny+FwKCIiotT6iIiIcucwZGRkeNT+clWZvrdo0UIzZszQV199pY8++khOp1Ndu3bV/v37L0XJ1aa8zzwnJ0f5+fnVVNWl0aBBA7311lv64osv9MUXXygqKkrdu3fXmjVrqru0SnM6nRo5cqS6deuma6+9ttx2NeW7fkZF+12TvucbNmxQYGCgbDabHn74Yc2bN0/XXHNNmW1r2ueNqnVFPJ0dVa9Lly6l/m+pa9euatWqld5++22NHz++GivDxdKiRQu1aNHC/XPXrl21Y8cOTZo0SR9++GE1VlZ5jz76qDZu3Kgffvihuku5pCra75r0PW/RooXWrl0ru92uOXPmaNCgQVq2bFm54QcoT40Z8QkPD5e3t7cyMzNLrc/MzFT9+vXL3KZ+/foetb9cVabvv+Xr66t27dpp+/btF6PEy0Z5n3lwcLD8/f2rqarqEx8ff8V+5sOHD9d//vMfLVmyRI0aNTpv25ryXZc86/dvXcnfc6vVqubNm6tDhw5KSUlRXFycpkyZUmbbmvR5o+rVmOBjtVrVoUMHpaamutc5nU6lpqaWex64S5cupdpL0qJFi8ptf7mqTN9/y+FwaMOGDWrQoMHFKvOyUFM+86qydu3aK+4zNwxDw4cP17x58/Tdd9/pqquu+t1tasLnXpl+/1ZN+p47nU4VFBSU+V5N+LxxEVX37Oqq9Omnnxo2m82YOXOm8euvvxoPPfSQERoaamRkZBiGYRgDBgwwRo8e7W6/fPlyw8fHx/i///s/Y/Pmzca4ceMMX19fY8OGDdXVhUrztO8vvPCC8c033xg7duwwVq9ebdxzzz2Gn5+fsWnTpurqQqWcOHHC+OWXX4xffvnFkGRMnDjR+OWXX4w9e/YYhmEYo0ePNgYMGOBuv3PnTiMgIMD4+9//bmzevNmYPn264e3tbSxcuLC6ulBpnvZ90qRJxpdffmls27bN2LBhg/HYY48ZXl5exuLFi6urC5Xy17/+1QgJCTGWLl1qHDp0yL2cPHnS3aYmftcr0++a8j0fPXq0sWzZMmPXrl3G+vXrjdGjRxsWi8X49ttvDcOomZ83Lp4aFXwMwzCmTp1qNG7c2LBarUZ8fLzx448/ut+74YYbjEGDBpVq/9lnnxkxMTGG1Wo1WrdubcyfP/8SV1x1POn7yJEj3W0jIiKMW2+91VizZk01VH1hzlyi/dvlTF8HDRpk3HDDDeds07ZtW8NqtRpNmzY13nvvvUted1XwtO+vvPKK0axZM8PPz88ICwszunfvbnz33XfVU/wFKKvPkkp9jjXxu16ZfteU7/nQoUONJk2aGFar1ahbt67Ro0cPd+gxjJr5eePisRiGYVy68SUAAIDqU2Pm+AAAAPwegg8AADANgg8AADANgg8AADANgg8AADANgg8AADANgg8AADANgg8AADANgg+AUpYuXSqLxaLjx49XdykAUOUIPgAAwDQIPgAAwDQIPsBlxul0KiUlRVdddZX8/f0VFxenOXPmSCo5DTV//ny1adNGfn5+6ty5szZu3FhqH1988YVat24tm82m6Ohovfbaa6XeLygo0FNPPaWoqCjZbDY1b95c7777bqk2q1evVseOHRUQEKCuXbsqPT394nYcAC4Bgg9wmUlJSdEHH3ygt956S5s2bdKoUaN0//33a9myZe42f//73/Xaa6/pp59+Ut26ddWrVy8VFRVJcgWWvn376p577tGGDRv0/PPP67nnntPMmTPd2w8cOFCffPKJXn/9dW3evFlvv/22AgMDS9XxzDPP6LXXXtPPP/8sHx8fDR069JL0HwAuJp7ODlxGCgoKFBYWpsWLF6tLly7u9Q8++KBOnjyphx56SDfeeKM+/fRT9evXT5J09OhRNWrUSDNnzlTfvn3Vv39/HT58WN9++617+yeffFLz58/Xpk2btHXrVrVo0UKLFi1SYmLiOTUsXbpUN954oxYvXqwePXpIkhYsWKDbbrtN+fn58vPzu8i/BQC4eBjxAS4j27dv18mTJ9WzZ08FBga6lw8++EA7duxwtzs7FIWFhalFixbavHmzJGnz5s3q1q1bqf1269ZN27Ztk8Ph0Nq1a+Xt7a0bbrjhvLW0adPG/bpBgwaSpKysrAvuIwBUJ5/qLgBAidzcXEnS/PnzFRkZWeo9m81WKvxUlr+/f4Xa+fr6ul9bLBZJrvlHAHAlY8QHuIxcc801stls2rt3r5o3b15qiYqKcrf78ccf3a+PHTumrVu3qlWrVpKkVq1aafny5aX2u3z5csXExMjb21uxsbFyOp2l5gwBgFkw4gNcRoKCgvTEE09o1KhRcjqduu6662S327V8+XIFBwerSZMmkqQXX3xRderUUUREhJ555hmFh4erd+/ekqTHH39cnTp10vjx49WvXz+lpaVp2rRpeuONNyRJ0dHRGjRokIYOHarXX39dcXFx2rNnj7KystS3b9/q6joAXBIEH+AyM378eNWtW1cpKSnauXOnQkND1b59ez399NPuU03/+Mc/9Nhjj2nbtm1q27at/v3vf8tqtUqS2rdvr88++0xjx47V+PHj1aBBA7344osaPHiw+xhvvvmmnn76aT3yyCM6cuSIGjdurKeffro6ugsAlxRXdQFXkDNXXB07dkyhoaHVXQ4AXHGY4wMAAEyD4AMAAEyDU10AAMA0GPEBAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACmQfABAACm8f8BBMG7r29c8mQAAAAASUVORK5CYII=", "text/plain": ["<Figure size 610.25x500 with 1 Axes>"]}, "metadata": {}, "output_type": "display_data"}], "source": ["\n", "# Read in the training metrics from the CSV file generated by the logger\n", "metrics = pd.read_csv(f\"{trainer.logger.log_dir}/metrics.csv\")\n", "\n", "# Remove the \"step\" column, which is not needed for our analysis\n", "del metrics[\"step\"]\n", "\n", "# Set the epoch column as the index, for easier plotting\n", "metrics.set_index(\"epoch\", inplace=True)\n", "\n", "# Display the first few rows of the metrics table, excluding any columns with all NaN values\n", "display(metrics.dropna(axis=1, how=\"all\").head())\n", "\n", "# Create a line plot of the training metrics using Seaborn\n", "sn.relplot(data=metrics, kind=\"line\")"]}, {"cell_type": "markdown", "id": "ab41247a", "metadata": {"papermill": {"duration": 0.015653, "end_time": "2023-10-11T19:20:08.516469", "exception": false, "start_time": "2023-10-11T19:20:08.500816", "status": "completed"}, "tags": []}, "source": ["## Congratulations - Time to Join the Community!\n", "\n", "Congratulations on completing this notebook tutorial! If you enjoyed this and would like to join the Lightning\n", "movement, you can do so in the following ways!\n", "\n", "### Star [Lightning](https://github.com/Lightning-AI/lightning) on GitHub\n", "The easiest way to help our community is just by starring the GitHub repos! This helps raise awareness of the cool\n", "tools we're building.\n", "\n", "### Join our [Slack](https://www.pytorchlightning.ai/community)!\n", "The best way to keep up to date on the latest advancements is to join our community! Make sure to introduce yourself\n", "and share your interests in `#general` channel\n", "\n", "\n", "### Contributions !\n", "The best way to contribute to our community is to become a code contributor! At any time you can go to\n", "[Lightning](https://github.com/Lightning-AI/lightning) or [Bolt](https://github.com/Lightning-AI/lightning-bolts)\n", "GitHub Issues page and filter for \"good first issue\".\n", "\n", "* [Lightning good first issue](https://github.com/Lightning-AI/lightning/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22)\n", "* [Bolt good first issue](https://github.com/Lightning-AI/lightning-bolts/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22)\n", "* You can also contribute your own notebooks with useful examples !\n", "\n", "### Great thanks from the entire Pytorch Lightning Team for your interest !\n", "\n", "[![Pytorch Lightning](data:image/png;base64,NDA0OiBOb3QgRm91bmQ=){height=\"60px\" width=\"240px\"}](https://pytorchlightning.ai)"]}, {"cell_type": "raw", "metadata": {"raw_mimetype": "text/restructuredtext"}, "source": [".. customcarditem::\n", "   :header: Introduction to PyTorch Lightning\n", "   :card_description: In this notebook, we'll go over the basics of lightning by preparing models to train on the [MNIST Handwritten Digits dataset](https://en.wikipedia.org/wiki/MNIST_database).\n", "   :tags: Image,GPU/TPU,Lightning-Examples"]}], "metadata": {"jupytext": {"cell_metadata_filter": "id,colab_type,colab,-all", "formats": "ipynb,py:percent", "main_language": "python"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.12"}, "papermill": {"default_parameters": {}, "duration": 147.033922, "end_time": "2023-10-11T19:20:10.054216", "environment_variables": {}, "exception": null, "input_path": "lightning_examples/mnist-hello-world/hello-world.ipynb", "output_path": ".notebooks/lightning_examples/mnist-hello-world.ipynb", "parameters": {}, "start_time": "2023-10-11T19:17:43.020294", "version": "2.4.0"}, "widgets": {"application/vnd.jupyter.widget-state+json": {"state": {"006a9301d3a84392a71d878bbeed3511": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_97ab3c0c78804bc0bbde53b8b2e55e6c", "placeholder": "\u200b", "style": "IPY_MODEL_8965ef59bf34431e93c7da89333a7310", "tabbable": null, "tooltip": null, "value": " 40/40 [00:01&lt;00:00, 30.89it/s]"}}, "04dd9f29703349f08aff7c75c727f1e2": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_44c49640b6f24840af1b8ef008d8dffa", "max": 2.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_f2838f431f7e4e798001a84ff74e7aa4", "tabbable": null, "tooltip": null, "value": 2.0}}, "0d81bddb74574dcab9225353f6e78fbe": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "0f695cf7d9204648ba9fa0d032898a58": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_744975617f7e41f4b17bbf90c40fb545", "placeholder": "\u200b", "style": "IPY_MODEL_79411b05122443b5aedbec33c52b2f91", "tabbable": null, "tooltip": null, "value": " 2/2 [00:00&lt;00:00,  6.65it/s]"}}, "0fa5fc43c2f34bef8f2170183de93a5e": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_5af4c511b76a42d98af28fcd4a6c9304", "max": 235.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_e1926a40bdc94ff4befdda49a8bdd647", "tabbable": null, "tooltip": null, "value": 235.0}}, "1288bc679f454015ae1580f8fed38965": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": "100%"}}, "1467088761cb4cb496b6c623c0d4d9ac": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "16d9e02ea0564b4a8d6865e1365d4467": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "1c6aafeaf4264955bfebff0f5b63b1a4": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_642edd7af84a42eab36758a321a54761", "placeholder": "\u200b", "style": "IPY_MODEL_fe62aeaaf915406892bfbd8f32471f78", "tabbable": null, "tooltip": null, "value": "Epoch 2: 100%"}}, "1c9cca9c42fb40d29e0401630c765de4": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "211f5a9d5cbe4163834914ca62fdcce7": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": "100%"}}, "29d2db67ee7745859b73dff08018df0c": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "2c07ea9039cb4476a1a8dc08bc9808c7": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_34e39e83b4854e3b9e00b480f7838b6a", "IPY_MODEL_04dd9f29703349f08aff7c75c727f1e2", "IPY_MODEL_0f695cf7d9204648ba9fa0d032898a58"], "layout": "IPY_MODEL_a29f800788584e15987e448c87122a7f", "tabbable": null, "tooltip": null}}, "34e39e83b4854e3b9e00b480f7838b6a": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_0d81bddb74574dcab9225353f6e78fbe", "placeholder": "\u200b", "style": "IPY_MODEL_4df953ac685345df9eae188f6c5f8d9d", "tabbable": null, "tooltip": null, "value": "Sanity Checking DataLoader 0: 100%"}}, "3785bcfe256a412b831bfbfeab3ea420": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "37b0c1b7cbe74ab383f5bdefe7aed12b": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_589ef1a62ffe43fc8e06b5fdab05820e", "IPY_MODEL_0fa5fc43c2f34bef8f2170183de93a5e", "IPY_MODEL_f10ed9f6ff2c4e91bb8d2c75a93309e8"], "layout": "IPY_MODEL_c2d4c1727053442c8f0363ea74c4fdfb", "tabbable": null, "tooltip": null}}, "38430cf275f440f8b225b3e85a2975e5": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "3972c740653447bca468dc6e0bbe97a3": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_29d2db67ee7745859b73dff08018df0c", "placeholder": "\u200b", "style": "IPY_MODEL_d7f260af39564aaa8ece6622b6108c6b", "tabbable": null, "tooltip": null, "value": "Validation DataLoader 0: 100%"}}, "39954bd9699c4370b5fdb7f541619b64": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%"}}, "3a579f813fcd46e9b741624fbf11314c": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "3a8921308dcf4957b61ef7f2ff4d5745": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "4115586d5d194449b7590dfa52b0da0b": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%"}}, "44c49640b6f24840af1b8ef008d8dffa": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "4906905814b74af09a0df6b53a0d41b5": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_3a8921308dcf4957b61ef7f2ff4d5745", "max": 20.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_3785bcfe256a412b831bfbfeab3ea420", "tabbable": null, "tooltip": null, "value": 20.0}}, "4df953ac685345df9eae188f6c5f8d9d": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "55eefe955b39469e87fae3046b8bd0c4": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_ad05f075da4f41bb8d757cbcd71f803b", "max": 215.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_f374b680a9cb4244b694872eee00da58", "tabbable": null, "tooltip": null, "value": 215.0}}, "589ef1a62ffe43fc8e06b5fdab05820e": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_d6e3b1677d3c47639cc62487b274d1cf", "placeholder": "\u200b", "style": "IPY_MODEL_d925348847a64825bfc647ad69ee2d92", "tabbable": null, "tooltip": null, "value": "Epoch 2: 100%"}}, "5af4c511b76a42d98af28fcd4a6c9304": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "5ef6a7e7e41a44cd9b227c830b1474c5": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_d4b31d6ba20d4cc9b134e26111a0c65f", "IPY_MODEL_4906905814b74af09a0df6b53a0d41b5", "IPY_MODEL_8109752a8654409e9b626dde19351903"], "layout": "IPY_MODEL_4115586d5d194449b7590dfa52b0da0b", "tabbable": null, "tooltip": null}}, "642edd7af84a42eab36758a321a54761": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "6ce2f4b6816e4127b1e49762dddd22d7": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_1c9cca9c42fb40d29e0401630c765de4", "placeholder": "\u200b", "style": "IPY_MODEL_e76a2c2e41104fd6a23da52ef3f4e7e7", "tabbable": null, "tooltip": null, "value": "Sanity Checking DataLoader 0: 100%"}}, "6d2c6c16fbdc411fadee43fbdca547be": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_38430cf275f440f8b225b3e85a2975e5", "placeholder": "\u200b", "style": "IPY_MODEL_c0538a7fe2da4dd995bd2fa7a3baee90", "tabbable": null, "tooltip": null, "value": " 215/215 [00:09&lt;00:00, 23.72it/s, v_num=0, val_loss=0.273, val_acc=0.917]"}}, "6d332a77979e40f7b8714ff0971042d0": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_7d44b1060f314c3790664cc41fb00b5e", "placeholder": "\u200b", "style": "IPY_MODEL_a8cdad3a77c6469f9458aa06b32fd56e", "tabbable": null, "tooltip": null, "value": "Validation DataLoader 0: 100%"}}, "6e67ef500fe84b7db3795b7d5fae8896": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%"}}, "72fa92b5c66b4f398c56c71454398842": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "744975617f7e41f4b17bbf90c40fb545": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "766192e8fbf04bf39e1af871ae5a1ec6": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "780365f03a9e430088fd3b36b1e09045": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "79411b05122443b5aedbec33c52b2f91": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "7ad8a36d9cae4e9b9c77a33564209b1c": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "7d44b1060f314c3790664cc41fb00b5e": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "8109752a8654409e9b626dde19351903": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_bd44d0874b5845a680f5a1c53f3d50c9", "placeholder": "\u200b", "style": "IPY_MODEL_3a579f813fcd46e9b741624fbf11314c", "tabbable": null, "tooltip": null, "value": " 20/20 [00:00&lt;00:00, 28.69it/s]"}}, "84a8558070594e57aa7d892c85785833": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_e38c9b3c58d94f35a35d09be8369df3e", "IPY_MODEL_c8699dc72563413ab00545cfb09c2e63", "IPY_MODEL_006a9301d3a84392a71d878bbeed3511"], "layout": "IPY_MODEL_211f5a9d5cbe4163834914ca62fdcce7", "tabbable": null, "tooltip": null}}, "8965ef59bf34431e93c7da89333a7310": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "8d483e6d2cca4dc1b533d19248a812ee": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_8e63f081a8b04e1f9cf453dde8bfd01d", "placeholder": "\u200b", "style": "IPY_MODEL_e2d1960833d44e8c9fda539bdc9ce0cc", "tabbable": null, "tooltip": null, "value": " 20/20 [00:00&lt;00:00, 28.21it/s]"}}, "8e63f081a8b04e1f9cf453dde8bfd01d": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "8f06397a2c874ed8b0b0542219047b95": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "903b5312a903478da3b8de5211832553": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "935105a8a90b4507a57417d2dd17b32a": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_3972c740653447bca468dc6e0bbe97a3", "IPY_MODEL_f6331dd7346b4f94ab7b862a52a7c3f4", "IPY_MODEL_8d483e6d2cca4dc1b533d19248a812ee"], "layout": "IPY_MODEL_6e67ef500fe84b7db3795b7d5fae8896", "tabbable": null, "tooltip": null}}, "936dfad7c7464591994820a8d88c1c01": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_903b5312a903478da3b8de5211832553", "placeholder": "\u200b", "style": "IPY_MODEL_780365f03a9e430088fd3b36b1e09045", "tabbable": null, "tooltip": null, "value": " 20/20 [00:00&lt;00:00, 28.32it/s]"}}, "95fbd11bc98f4395b8a7154b78694974": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "97ab3c0c78804bc0bbde53b8b2e55e6c": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "997d11c32f3343979b3dcc0d024bb8aa": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "9dc4db89cc5e402ebc3e885d7f2cb5fa": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "9f2cece3b6b64c1a98fb95a98d055c93": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "a29f800788584e15987e448c87122a7f": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%"}}, "a46568b830a74b8a857e1ea8e8c42785": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "a496173fbae644c6913bc1165e6e15a3": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_1c6aafeaf4264955bfebff0f5b63b1a4", "IPY_MODEL_55eefe955b39469e87fae3046b8bd0c4", "IPY_MODEL_6d2c6c16fbdc411fadee43fbdca547be"], "layout": "IPY_MODEL_1288bc679f454015ae1580f8fed38965", "tabbable": null, "tooltip": null}}, "a8cdad3a77c6469f9458aa06b32fd56e": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "ad04c806cd2245bd98a355a12714e4fc": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "ad05f075da4f41bb8d757cbcd71f803b": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "b240671cc50a44959dced7a0b11e47c5": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%"}}, "b4830252aed340f19e2cede8c6365ba1": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "bd44d0874b5845a680f5a1c53f3d50c9": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "c0538a7fe2da4dd995bd2fa7a3baee90": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "c05d49bc81f64628ba3f1c0773394344": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_6d332a77979e40f7b8714ff0971042d0", "IPY_MODEL_e9575c081e024ddc84642f4492a8b05e", "IPY_MODEL_936dfad7c7464591994820a8d88c1c01"], "layout": "IPY_MODEL_39954bd9699c4370b5fdb7f541619b64", "tabbable": null, "tooltip": null}}, "c2d4c1727053442c8f0363ea74c4fdfb": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": "100%"}}, "c8699dc72563413ab00545cfb09c2e63": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_766192e8fbf04bf39e1af871ae5a1ec6", "max": 40.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_1467088761cb4cb496b6c623c0d4d9ac", "tabbable": null, "tooltip": null, "value": 40.0}}, "d006768d30ca4aa4b5196fb8f458312e": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "d4b31d6ba20d4cc9b134e26111a0c65f": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_fc8fa1e8b48e474b92878704355cfeb6", "placeholder": "\u200b", "style": "IPY_MODEL_16d9e02ea0564b4a8d6865e1365d4467", "tabbable": null, "tooltip": null, "value": "Validation DataLoader 0: 100%"}}, "d6e3b1677d3c47639cc62487b274d1cf": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "d7f260af39564aaa8ece6622b6108c6b": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "d925348847a64825bfc647ad69ee2d92": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "df861e8defee4be6aaf746ed6b84002c": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_9f2cece3b6b64c1a98fb95a98d055c93", "max": 2.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_a46568b830a74b8a857e1ea8e8c42785", "tabbable": null, "tooltip": null, "value": 2.0}}, "e1926a40bdc94ff4befdda49a8bdd647": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "e2d1960833d44e8c9fda539bdc9ce0cc": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "e38c9b3c58d94f35a35d09be8369df3e": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_9dc4db89cc5e402ebc3e885d7f2cb5fa", "placeholder": "\u200b", "style": "IPY_MODEL_997d11c32f3343979b3dcc0d024bb8aa", "tabbable": null, "tooltip": null, "value": "Testing DataLoader 0: 100%"}}, "e76a2c2e41104fd6a23da52ef3f4e7e7": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "e8d91b2a186d4f378c72ae2e359892c6": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_ad04c806cd2245bd98a355a12714e4fc", "placeholder": "\u200b", "style": "IPY_MODEL_d006768d30ca4aa4b5196fb8f458312e", "tabbable": null, "tooltip": null, "value": " 2/2 [00:00&lt;00:00, 50.31it/s]"}}, "e9575c081e024ddc84642f4492a8b05e": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_95fbd11bc98f4395b8a7154b78694974", "max": 20.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_b4830252aed340f19e2cede8c6365ba1", "tabbable": null, "tooltip": null, "value": 20.0}}, "ead8c87b141c430483826a1888a8ffdc": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "f10ed9f6ff2c4e91bb8d2c75a93309e8": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_ead8c87b141c430483826a1888a8ffdc", "placeholder": "\u200b", "style": "IPY_MODEL_7ad8a36d9cae4e9b9c77a33564209b1c", "tabbable": null, "tooltip": null, "value": " 235/235 [00:05&lt;00:00, 44.77it/s, v_num=0]"}}, "f2838f431f7e4e798001a84ff74e7aa4": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "f374b680a9cb4244b694872eee00da58": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "f41a57de3dc441cf83c3917a96d37f22": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_6ce2f4b6816e4127b1e49762dddd22d7", "IPY_MODEL_df861e8defee4be6aaf746ed6b84002c", "IPY_MODEL_e8d91b2a186d4f378c72ae2e359892c6"], "layout": "IPY_MODEL_b240671cc50a44959dced7a0b11e47c5", "tabbable": null, "tooltip": null}}, "f6331dd7346b4f94ab7b862a52a7c3f4": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_8f06397a2c874ed8b0b0542219047b95", "max": 20.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_72fa92b5c66b4f398c56c71454398842", "tabbable": null, "tooltip": null, "value": 20.0}}, "fc8fa1e8b48e474b92878704355cfeb6": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "fe62aeaaf915406892bfbd8f32471f78": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}}, "version_major": 2, "version_minor": 0}}}, "nbformat": 4, "nbformat_minor": 5}