{"cells": [{"cell_type": "markdown", "id": "4a709358", "metadata": {"papermill": {"duration": 0.012367, "end_time": "2023-10-11T16:28:35.825360", "exception": false, "start_time": "2023-10-11T16:28:35.812993", "status": "completed"}, "tags": []}, "source": ["\n", "# Tutorial 10: Autoregressive Image Modeling\n", "\n", "* **Author:** Phillip Lippe\n", "* **License:** CC BY-SA\n", "* **Generated:** 2023-10-11T16:26:07.497672\n", "\n", "In this tutorial, we implement an autoregressive likelihood model for the task of image modeling.\n", "Autoregressive models are naturally strong generative models that constitute one of the current\n", "state-of-the-art architectures on likelihood-based image modeling,\n", "and are also the basis for large language generation models such as GPT3.\n", "We will focus on the PixelCNN architecture in this tutorial, and apply it to MNIST modeling.\n", "This notebook is part of a lecture series on Deep Learning at the University of Amsterdam.\n", "The full list of tutorials can be found at https://uvadlc-notebooks.rtfd.io.\n", "\n", "\n", "---\n", "Open in [![Open In Colab](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAHUAAAAUCAYAAACzrHJDAAAIuUlEQVRoQ+1ZaVRURxb+qhdolmbTUVSURpZgmLhHbQVFZIlGQBEXcMvJhKiTEzfigjQg7oNEJ9GMGidnjnNMBs2czIzajksEFRE1xklCTKJiQLRFsUGkoUWw+82pamn79etGYoKek1B/4NW99/tu3e/dquJBAGD27NkHALxKf39WY39gyrOi+i3xqGtUoePJrFmznrmgtModorbTu8YRNZk5cybXTvCtwh7o6NR2KzuZMWNGh6jtVt7nA0ymT5/eJlF9POrh7PAQl6s8bGYa3PUum//htmebVtLRqW0q01M5keTk5FZFzU0oRle3+zxwg5Hgtb+PZiL/ZVohxCI+hL5JgjmfjPxZ26+33BG3dA+ealHPM4gQAo5rU59gsI8bRvl54t3Ca62mvHyUAhtOlLd5WSQpKcluBjumnoCLs1EARkVd9E8l3p9y2i7RbQ1B6pFwu/YDgW8KbHJHMTQrwnjz2oZm9M4pavOCfo5jWrgCaaMVcMs6/pNhDr0+AMN93XlxV7R6DNpyzi7W/OE+yIrsjU6rTrbKV5cd/pNyItOmTbMp6sbBB+EqaYJY4cWE3VUciNt1TpgfcRFv71Fi54xT5kSoyLvOBEJMOMxWXkFlBeBSX4u6Zkcs+3KszYRtiapbNRqF31UgetVuc8z9vBXIv1qD+F1f83B6uDlCUyfsZGepGPpmg01OB7EITQbhS9ribKy+DmP1DUiClLz4bnIHVOqa7BY+Z1wg5g3zgUvyehiNpnJKxSLc/ts76LKm0BzX3c0RNy1yXjDcB5lWoro4iNHQxM+f1kWeWQARAWQS++trISJTp061Kep25X/MycwtjuctSC5rxo7ppi7VNUox5+PhPHtrsS2O1qJ6yx1QujQUzm9sh6hbkBlvvGcN8hYnwjUjH6kjfZEd5c/jitz5Jc5U3ENnFynKl4eB7nyEgP2UZ+Yz3/rVEbyYr27qELrtC4FIC0J7sc7xWnmccdHfRRTs0VB+cA4lt+oFcRR/wUeH8FG5w2Mbx8FQ8TXEvv1xYf4wBP3O2WyL3/UVjpXWgIqaFeUPr+wTmDvUB7njH6/bOv+HRg4SqioAg5GDe1aB3ZeMTJkyRSBqkLsWqSEm0fZVBEN94zEZnYvrdx1JL5cxe+a+AbhSJecRRHW/ikTFRTa38dtQlNZ5CRKwFvUtZU/kvBoEF9Uxni/XqIM+dwKbTw3rhcxIf7gmr2M+H6SMwx8iBzJbw5oxeG3Lv5FX9B3AGaHPS8e8z77H7v9VMpvPG5ug1enh7eGK8h0LBTwUb+GInqzInlRUK65DmTPQu4c3+uQKjwKK77zwUxBX4Tq7yR1RuiwUsqlrABCM6esHdXoy47fk4+prYKy8ZF574x4V5BnHQBuf4g9Z9ld8U36L2aktZNNplNfw7zotwWTy5MkCUft4aLEopJj5/OPHl1BQqeAVOnHgNSQOqmBzq9V9cfEm/yx5ubMGKS9cYPZ3vx2OS/c6PVHUuUO7Y1Pci3BO/1zgq18byebfGemLtNF+6JRtOvMk926ibussZqM+1mNz4TWkH7rCbM5phwGRGDAaoF8fY5OHFnlldAA8sgoEXKnDukA1NgSeNjqkJT9brbN4pC9WRweYXyLugR73c+MYvyWfu0yC6+mjzN1Isfw3FKJS98CU/zI1IHFkFPR52cHL2FJk0sB6kMTERIGo9GzcPkLNfA0cwdwi/hfEYO86ZMd9w+y1egfM2T2Eh/vesMNwljSzuZRT420SW3eqy8N6aHMmwmnFUZ7/PGVPbIoNZvNU1BURdHs0bT2+HjL8sDSM2e6vi4Lj5NW8WOLVA6RTT2azxLV+bglaFNqLieqemS/gWkw7NyoAHo+2dEsiivengjKsPFoqWOvbSh/kxPaxyW/JRzH2Fl3EzD9/xjAefJqB3usKUFn/0Gb+S/d/jy3FN2yLOmnSJJtn6oehByEiHPSeXnDxFGPRnoFoaBJjcdQlbDwcjL1zTNuQpoxD7R0OG0uUTMi0fkVwdzBdYIwcwZunxrVJVLplNm54BZp7jfDfYLoNyqQi1K6KxIdHzmN+QQ2WjFIwUT2zTGdlRXo4NFXVUO4sgX5dFC7f0aP/ZlNeUjFBuL8Xjl6uRuP6aMjSjpjzsH62FDU7JhBuGccEXIvDfJFFBc/gHw80dklfCVYnRaDfpiJcutPA4F7qJsfJeUPQI+1fqMlNhFx1FM0GDqkjFVg7NojlQ0Vt4aM5ReSqcbpaCg8nCW5lRsBvbT4T1TLfFptsfh7gItzuKTdJSEiwKSrt1vcmnEXXrsLbYnWDA1bu+z2WKy9Arq+1KRqdfKsoBo0GcdtEpS/B1bO4v0cFiUhkjskvKcMrWwtAPHuwQq8Z+4LZ1vTQANfXt4J0DwZX9gWa9qh4XDM/voC9JXfwYEMMHJcfNtusn82ihvliVUwg5KrPGVf6GH94ZJpEZBen6EC4qYTHA1dXhW0JIex8txzv//c8lhzXIi/BFxOH9jGbQhZsRalTIBZZ8KkGyZAxeRQvXkFF1TWz/Hm46jNYUnjPbt3JxIkT7f6dSj8qfJJyVvBxgaIlblOyjtysNHWN9fjjqWi7glJfW3/S0Hlj2XnA8PhKT9w6g3Qx3XiXhvuxQsuT1proxBKI/AaZqY1Xz5muvY8G8XkRRCaHsfQsRAFDH/tZPbcYuHotOG0FRIqB4HR3wNVoIPLtz8ycTguu+jpEigE218vd1YCr5m+HpHMvEI9u4LTXwNWaLjl0iPwGAmIpeHx1VeCqTJdPs1/vweweQPO3HC24NhOhnTphwoQnfv6QSY2ICbkNmdSA4h87oaLaiYfn5diIEd4att2erOwJXbPUHp953p6orQVSUVWRAXBT8c/dJ5L9xhzaJGp71GR/wFP8P5V2z10NSC9T93QM2xUg8fHxT+zU9ijeU4naHon8CjFJXFzc8/kn+dN06q9QgF98SYSo2Xen2NjYZy5sR6f+4nLSK5Iam2PH/x87a1YN/t5sBgAAAABJRU5ErkJggg==){height=\"20px\" width=\"117px\"}](https://colab.research.google.com/github/PytorchLightning/lightning-tutorials/blob/publication/.notebooks/course_UvA-DL/10-autoregressive-image-modeling.ipynb)\n", "\n", "Give us a \u2b50 [on Github](https://www.github.com/Lightning-AI/lightning/)\n", "| Check out [the documentation](https://pytorch-lightning.readthedocs.io/en/stable/)\n", "| Join us [on Slack](https://www.pytorchlightning.ai/community)"]}, {"cell_type": "markdown", "id": "158b4700", "metadata": {"papermill": {"duration": 0.015621, "end_time": "2023-10-11T16:28:35.853167", "exception": false, "start_time": "2023-10-11T16:28:35.837546", "status": "completed"}, "tags": []}, "source": ["## Setup\n", "This notebook requires some packages besides pytorch-lightning."]}, {"cell_type": "code", "execution_count": 1, "id": "36e0ec76", "metadata": {"colab": {}, "colab_type": "code", "execution": {"iopub.execute_input": "2023-10-11T16:28:35.877471Z", "iopub.status.busy": "2023-10-11T16:28:35.876916Z", "iopub.status.idle": "2023-10-11T16:32:50.702473Z", "shell.execute_reply": "2023-10-11T16:32:50.701484Z"}, "id": "LfrJLKPFyhsK", "lines_to_next_cell": 0, "papermill": {"duration": 254.842354, "end_time": "2023-10-11T16:32:50.706380", "exception": false, "start_time": "2023-10-11T16:28:35.864026", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\r\n", "\u001b[0m"]}], "source": ["! pip install --quiet \"lightning>=2.0.0\" \"torch>=1.8.1, <2.1.0\" \"setuptools>=68.0.0, <68.3.0\" \"matplotlib\" \"torchmetrics>=0.7, <1.3\" \"ipython[notebook]>=8.0.0, <8.17.0\" \"matplotlib>=3.0.0, <3.9.0\" \"pytorch-lightning>=1.4, <2.1.0\" \"torchvision\" \"urllib3\" \"seaborn\""]}, {"cell_type": "markdown", "id": "2b2df267", "metadata": {"papermill": {"duration": 0.017062, "end_time": "2023-10-11T16:32:50.742553", "exception": false, "start_time": "2023-10-11T16:32:50.725491", "status": "completed"}, "tags": []}, "source": ["
\n", "\n", "Similar to the language generation you have seen in assignment 2, autoregressive models work on images by modeling the likelihood of a pixel given all previous ones.\n", "For instance, in the picture below, we model the pixel $x_i$ as a conditional probability distribution\n", "based on all previous (here blue) pixels (figure credit - [Aaron van den Oord et al. ](https://arxiv.org/abs/1601.06759)):\n", "\n", "
\n", "\n", "Generally, autoregressive model over high-dimensional data $\\mathbf{x}$ factor the joint distribution as the following product of conditionals:\n", "\n", "$$p(\\mathbf{x})=p(x_1, ..., x_n)=\\prod_{i=1}^{n} p(x_i|x_1,...,x_{i-1})$$\n", "\n", "Learning these conditionals is often much simpler than learning the joint distribution $p(\\mathbf{x})$ all together.\n", "However, disadvantages of autoregressive models include slow sampling, especially for large images,\n", "as we need height-times-width forward passes through the model.\n", "In addition, for some applications, we require a latent space as modeled in VAEs and Normalizing Flows.\n", "For instance, in autoregressive models, we cannot interpolate between two images because of the lack of a latent representation.\n", "We will explore and discuss these benefits and drawbacks alongside with our implementation.\n", "\n", "Our implementation will focus on the [PixelCNN](https://arxiv.org/pdf/1606.05328.pdf) [2] model which has been discussed in detail in the lecture.\n", "Most current SOTA models use PixelCNN as their fundamental architecture,\n", "and various additions have been proposed to improve the performance\n", "(e.g. [PixelCNN++](https://arxiv.org/pdf/1701.05517.pdf) and [PixelSNAIL](http://proceedings.mlr.press/v80/chen18h/chen18h.pdf)).\n", "Hence, implementing PixelCNN is a good starting point for our short tutorial.\n", "\n", "First of all, we need to import our standard libraries. Similarly as in\n", "the last couple of tutorials, we will use [PyTorch\n", "Lightning](https://lightning.ai/docs/pytorch/stable/) here as\n", "well."]}, {"cell_type": "code", "execution_count": 2, "id": "7fd6bdb9", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:32:50.768763Z", "iopub.status.busy": "2023-10-11T16:32:50.768292Z", "iopub.status.idle": "2023-10-11T16:32:56.292995Z", "shell.execute_reply": "2023-10-11T16:32:56.292200Z"}, "papermill": {"duration": 5.539522, "end_time": "2023-10-11T16:32:56.294556", "exception": false, "start_time": "2023-10-11T16:32:50.755034", "status": "completed"}, "tags": []}, "outputs": [{"name": "stderr", "output_type": "stream", "text": ["Global seed set to 42\n"]}, {"name": "stdout", "output_type": "stream", "text": ["Using device cuda:0\n"]}, {"data": {"text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["\n", "import math\n", "import os\n", "import urllib.request\n", "from urllib.error import HTTPError\n", "\n", "import lightning as L\n", "\n", "# Imports for plotting\n", "import matplotlib.pyplot as plt\n", "import matplotlib_inline.backend_inline\n", "import numpy as np\n", "import seaborn as sns\n", "import torch\n", "import torch.nn as nn\n", "import torch.nn.functional as F\n", "import torch.optim as optim\n", "import torch.utils.data as data\n", "import torchvision\n", "from lightning.pytorch.callbacks import LearningRateMonitor, ModelCheckpoint\n", "from matplotlib.colors import to_rgb\n", "from torch import Tensor\n", "from torchvision import transforms\n", "from torchvision.datasets import MNIST\n", "from tqdm.notebook import tqdm\n", "\n", "plt.set_cmap(\"cividis\")\n", "%matplotlib inline\n", "matplotlib_inline.backend_inline.set_matplotlib_formats(\"svg\", \"pdf\") # For export\n", "\n", "# Path to the folder where the datasets are/should be downloaded (e.g. MNIST)\n", "DATASET_PATH = os.environ.get(\"PATH_DATASETS\", \"data\")\n", "# Path to the folder where the pretrained models are saved\n", "CHECKPOINT_PATH = os.environ.get(\"PATH_CHECKPOINT\", \"saved_models/tutorial12\")\n", "\n", "# Setting the seed\n", "L.seed_everything(42)\n", "\n", "# Ensure that all operations are deterministic on GPU (if used) for reproducibility\n", "torch.backends.cudnn.deterministic = True\n", "torch.backends.cudnn.benchmark = False\n", "\n", "# Fetching the device that will be used throughout this notebook\n", "device = torch.device(\"cpu\") if not torch.cuda.is_available() else torch.device(\"cuda:0\")\n", "print(\"Using device\", device)"]}, {"cell_type": "markdown", "id": "815134e9", "metadata": {"papermill": {"duration": 0.012459, "end_time": "2023-10-11T16:32:56.317820", "exception": false, "start_time": "2023-10-11T16:32:56.305361", "status": "completed"}, "tags": []}, "source": ["We again provide a pretrained model, which is downloaded below:"]}, {"cell_type": "code", "execution_count": 3, "id": "251ddfda", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:32:56.401586Z", "iopub.status.busy": "2023-10-11T16:32:56.400791Z", "iopub.status.idle": "2023-10-11T16:32:56.828302Z", "shell.execute_reply": "2023-10-11T16:32:56.827305Z"}, "papermill": {"duration": 0.443193, "end_time": "2023-10-11T16:32:56.830959", "exception": false, "start_time": "2023-10-11T16:32:56.387766", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Downloading https://raw.githubusercontent.com/phlippe/saved_models/main/tutorial12/PixelCNN.ckpt...\n"]}], "source": ["# Github URL where saved models are stored for this tutorial\n", "base_url = \"https://raw.githubusercontent.com/phlippe/saved_models/main/tutorial12/\"\n", "# Files to download\n", "pretrained_files = [\"PixelCNN.ckpt\"]\n", "# Create checkpoint path if it doesn't exist yet\n", "os.makedirs(CHECKPOINT_PATH, exist_ok=True)\n", "\n", "# For each file, check whether it already exists. If not, try downloading it.\n", "for file_name in pretrained_files:\n", " file_path = os.path.join(CHECKPOINT_PATH, file_name)\n", " if not os.path.isfile(file_path):\n", " file_url = base_url + file_name\n", " print(\"Downloading %s...\" % file_url)\n", " try:\n", " urllib.request.urlretrieve(file_url, file_path)\n", " except HTTPError as e:\n", " print(\n", " \"Something went wrong. Please try to download the file from the GDrive folder, or contact the author with the full output including the following error:\\n\",\n", " e,\n", " )"]}, {"cell_type": "markdown", "id": "5a5a3eee", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.011057, "end_time": "2023-10-11T16:32:56.854980", "exception": false, "start_time": "2023-10-11T16:32:56.843923", "status": "completed"}, "tags": []}, "source": ["Similar to the Normalizing Flows in Tutorial 11, we will work on the\n", "MNIST dataset and use 8-bits per pixel (values between 0 and 255). The\n", "dataset is loaded below:"]}, {"cell_type": "code", "execution_count": 4, "id": "38deeb8b", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:32:56.881735Z", "iopub.status.busy": "2023-10-11T16:32:56.881173Z", "iopub.status.idle": "2023-10-11T16:32:58.173909Z", "shell.execute_reply": "2023-10-11T16:32:58.173260Z"}, "papermill": {"duration": 1.315515, "end_time": "2023-10-11T16:32:58.182836", "exception": false, "start_time": "2023-10-11T16:32:56.867321", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\n", "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to /__w/13/s/.datasets/MNIST/raw/train-images-idx3-ubyte.gz\n"]}, {"name": "stderr", "output_type": "stream", "text": ["\r", " 0%| | 0/9912422 [00:00 only make them a tensor\n", "transform = transforms.Compose([transforms.ToTensor(), discretize])\n", "\n", "# Loading the training dataset. We need to split it into a training and validation part\n", "train_dataset = MNIST(root=DATASET_PATH, train=True, transform=transform, download=True)\n", "L.seed_everything(42)\n", "train_set, val_set = torch.utils.data.random_split(train_dataset, [50000, 10000])\n", "\n", "# Loading the test set\n", "test_set = MNIST(root=DATASET_PATH, train=False, transform=transform, download=True)\n", "\n", "# We define a set of data loaders that we can use for various purposes later.\n", "train_loader = data.DataLoader(train_set, batch_size=128, shuffle=True, drop_last=True, pin_memory=True, num_workers=4)\n", "val_loader = data.DataLoader(val_set, batch_size=128, shuffle=False, drop_last=False, num_workers=4)\n", "test_loader = data.DataLoader(test_set, batch_size=128, shuffle=False, drop_last=False, num_workers=4)"]}, {"cell_type": "markdown", "id": "60fbf039", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.015206, "end_time": "2023-10-11T16:32:58.213957", "exception": false, "start_time": "2023-10-11T16:32:58.198751", "status": "completed"}, "tags": []}, "source": ["A good practice is to always visualize some data examples to get an intuition of the data:"]}, {"cell_type": "code", "execution_count": 5, "id": "bbf284d2", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:32:58.254674Z", "iopub.status.busy": "2023-10-11T16:32:58.254203Z", "iopub.status.idle": "2023-10-11T16:32:58.430171Z", "shell.execute_reply": "2023-10-11T16:32:58.429503Z"}, "papermill": {"duration": 0.210926, "end_time": "2023-10-11T16:32:58.438607", "exception": false, "start_time": "2023-10-11T16:32:58.227681", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMzQxLjY3NDgzODcwOTcgMTgwLjcyIF0gL0NvbnRlbnRzIDkgMCBSIC9Bbm5vdHMgMTAgMCBSID4+CmVuZG9iago5IDAgb2JqCjw8IC9MZW5ndGggMTIgMCBSIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nFWOSw7CMAxE9z7FnCDfKkmXQKWIZWHBAaJQiCioVKLXx61AhcWzPJbHHtnk1zXlQ9xidyS5qjSSRmE6KBRmgkZkOlKserKVFs5XwdYsb79SByW84Zla2wvRmQZ4YRas4TpvB69qD+2csAbPjBPukBv+MvKrwkx8PeI/2LD4HeYgH+v3cOoh9xrNAy219AYPKzF0CmVuZHN0cmVhbQplbmRvYmoKMTIgMCBvYmoKMTQ4CmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjMgMCBvYmoKPDwgPj4KZW5kb2JqCjQgMCBvYmoKPDwgL0ExIDw8IC9UeXBlIC9FeHRHU3RhdGUgL0NBIDEgL2NhIDEgPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgL0kxIDEzIDAgUiA+PgplbmRvYmoKMTMgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL1N1YnR5cGUgL0ltYWdlIC9XaWR0aCA0NTUgL0hlaWdodCAyMzEKL0NvbG9yU3BhY2UgWyAvSW5kZXhlZCAvRGV2aWNlUkdCIDIyMgoo/////f39+/v7+fn59/f39fX18/Pz8fHx7+/v7e3t6+vr6enp5+fn5eXl4+Pj4eHh39/f3d3d29vb2dnZ19fX1dXV09PT0dHRz8/Pzc3Ny8vLycnJx8fHxcXFw8PDwcHBv7+/vb29u7u7ubm5t7e3tbW1s7Ozr6+vra2tqampp6enpaWloaGhmZmZl5eXlZWVk5OTkZGRj4+PjY2Ni4uLiYmJh4eHhYWFg4ODfX19e3t7eXl5d3d3dXV1c3NzcXFxb29vZ2dnY2NjX19fXV1dW1tbWVlZV1dXVVVVU1NTUVFRT09PS0tLSUlJR0dHRUVFQ0NDQUFBPz8/Ozs7OTk5Nzc3NTU1MzMzMTExLy8vKysrJycnJSUlIyMjHx8fHR0dGxsbGRkZFxcXFRUVExMTERERDw8PXHJcclxyCwsLCQkJBwcHBQUFAwMDAQEB/v7+/Pz8+vr6+Pj49PT08vLy8PDw7u7u7Ozs6urq6Ojo5ubm5OTk4uLi4ODg3t7e3Nzc2tra2NjY1tbW1NTU0NDQzs7OzMzMxsbGxMTEwsLCwMDAvr6+vLy8urq6uLi4tra2tLS0srKysLCwrq6urKysqqqqqKiopKSkoqKioKCgnp6enJycmpqamJiYlpaWlJSUioqKiIiIhoaGhISEgoKCgICAfn5+fHx8eHh4dnZ2dHR0cnJycHBwbm5uampqZmZmZGRkYmJiYGBgXl5eXFxcXFxcWlpaWFhYVlZWVFRUUFBQTExMSkpKSEhIRkZGREREQkJCQEBAPj4+PDw8Ojo6ODg4NDQ0MjIyMDAwLi4uLCwsKioqXChcKFwoJiYmJCQkIiIiICAgHh4eHBwcGhoaGBgYFhYWFBQUEhISEBAQDg4ODAwMXG5cblxuCAgIBgYGBAQEAgICAAAAKQpdCi9CaXRzUGVyQ29tcG9uZW50IDggL0ZpbHRlciAvRmxhdGVEZWNvZGUKL0RlY29kZVBhcm1zIDw8IC9QcmVkaWN0b3IgMTAgL0NvbG9ycyAxIC9Db2x1bW5zIDQ1NSAvQml0c1BlckNvbXBvbmVudCA4ID4+Ci9MZW5ndGggMTQgMCBSID4+CnN0cmVhbQp4nO2d/58WRQHHOUIoSkwNAoqyiAL6AgRFREREJwlpGET0BSlM0JCzODwBM0PlLoOKhKDSS6wsoKgEosvEwzs7EdPokoMj4W/p8+F2X+ztM7M7++U5juHz/gXcZ3c/O/t+8Jmd2ZkZ8FvhAwMu9gWIUpBHP5BHP5BHP5BHP5BHPwg9nusrlFeVPHn0I08e/ciTRz/y5NGPPHn0I08e/ciTRz/y5NFCZ2fnO8GzoE/y8iKPycijU26/z5NHp9x+n3eZejwBFi5cWA/qwAAwZswYe27RPGfy5HWDrVu3siDbQdXziiCPduRRHi9hj58A40BNb5qamuy5RfIykTXvVrAF1PUwB1Q1ryjyaEYe5TFPXlHk0czl5PF/gHWBlpaWRjAI1FRy9913/wWYc8srSApZ87aBwOE9gEWtal5R5NGMPMpjnryiyKOZy8Xjf4ChUmOgQD3nKcCmhMmTJ9t34i1+DSSfKmv53gICj4ZvYTpZ8z4D7ge4Ycx8BLD0p06dypQnjzHkUR7z5MmjPEbysnmcNWvWe0G6Q1JbW5unnCjBqQcBT3H48GHDHv8Eu3fvfgMYCVDbakktp1v5zl3wWF9fvwA4H5c5b+XKlTsBK4l39cASB3+9a8SIEQuBc548xpBHecyU1+ce16xZ800wA6xduzZB3Lp16xaBFwLOnj2bp5zfAOEZ586dy01Uy0tYsmTJh8F6EIl9HKSWM/2GBIQet2zZ4nxMrjz8CFLYG8GfwZ49e6IeAW722qPAKU8eY8ijPGbKk0d5rMxz8Njd3Z0gD/wU/Ay0t7eXUE7UbQ6HZx46dOgk8GZgjn4HYEtvgbwos2fPHgjoccqUKW7H5Mn7OsAzI221B3dt2bJlfwvYCgKX94Gurq70PHmMIo/ymCVPHuXRnFfIY1NTE99fuR2UV05qYSsjX4WN5V1xxRUs42QQbEmu4zjlhfDFnM2bNwdtq3WdnZ3OZcqctx/g6j8CDC3E3IRvFL7EQ1nG5ubm9Dx5DJFHecya16cezwB+1NOpEqUBjAYrVqyIHcP/8Xcl/g/d+b7yUXTfvn3fD+Dj65EjR/jJz0HN+cfVdexFSz6Nc948UHeBwON8cEMPnwPpp0nNewnwBm7YsIF3y7IX/mlcA3glqHmk58ljiDxGkEd5jCGPJXlk/1dl3eZq0ApYuZlbASsnPwb8+9NPP52nnMncBNjxiCtx6x50zuMbnBGP48aNGw/4EBdsYU8hN01JfLRMzWMLcdDDmHA1uMNhl+T111+fsKM8xpBHecySJ49EHs152TwOGTKEPWJ8MW8HqLQcZf369aweZC1nMm8HQVVrOUg/IK9HO1eCjo6OvHkTABXt37/fssdZMHHiRO70EFi9enV6+eQxRB7lMUuePBJ5NOcleAzbaSMMHDjwXmBowjbDR/U2kKWcyYQep02b5nZAXo/4Fm6oBF/kIRywgMpep7klPTFvJbgKUJH9SjhOP+h/TH43N5InjyHyKI+ueRfFI0vjZCuZm4FrOVOYP3/+68EPwOnTp92OyePxOrBq1SrDTnxy5puK2OkVkDXvvyB8rbHysfAkmApeB4Kd0i9bHmPIozy65snjeeTRmnfJeMQTWwcc8oyZ5mDM4/GTwLwTRydsAlBp6TxMzePzI0sR6Vbkk/C8efNYgYrcuMmJo7FjefIYIo/ymCVPHuXRnpfbI3vjRo0axbdaONKNtlA7iO3ER+clIEs5LXwaBGf9O3A+Lo/Hd4PKeg62rOJMkKznLFq0KG/edMAqzODBg38YwIGA2MI6HMdXvA3gP+XRUM70HeVRHrPkyaM82vMSPCb3MPIlmX09PAHMO/0GZC2nAT4gc4YVnJFroyQPlMub9wEQaSfHd2VF8H4u+wPb2tp+BPjJcFAgj9NwPQYaGhrCJgH+m8BXJxxEHmg9AJzLJ48h8iiPWfL63KP5vUd3HnjgAb4Dn7WcBvL+NmbK46QgF+bQJe8CHA7Nux7ZPAuUkxcOQP4CCLayBZb/RGD5FuBcPnkMkUd5zJknj+7I4/OAb+Nk9TcM8MrMM7Tn8fgTwDNv3LhxGch0bKa8M2fOXAvqjLA2wgHhd4KS8ir5LGDVx3m6EHk0II/ymCevEnlMLqfbzl56JJ8HWVRyHpR/gNRc54K9Ct4KePaxY8c6H5c3jxUpfm8iAvm+zKFDhzjEvPy8GNcFjdSo/bgdII9m5FEe8+TFqJJH0tbWlizvu+DmHlYnDw46l72cnFSeIfzVtY9yKi+vKAXy+CYlnx35+yiPBfOKIo9m5NEpTx5Lpv96LJesee8H9MgKQF/kFaVA3ldA2CUpjwXziiKPZuTRKU8eS0YeDbz44ovfBvTI+d8nTZqUMEViCXklII8G5NE1Tx5LRh4NyKNrXv/2eOzYsbCxgWuxvPzyy9XNK4ECed8CvwRcoGzp0qWZ8uSxZOTRgDy65vVvj8pzzZNHP/Lk0Y88efQjTx79yJNHP/Lk0Y88efQjL/QoLm3k0Q/k0Q/k0Q/k0Q/k0Q/k0Q/0/OhHnjz6kSePfuTJox958uhHnjz6kSePfuTJox958uhHnjz6kSePfuTJox958uhHnjz6kSePfuT1X4//Ak/1cA/g2LnI+oi1tbUl51XCdVgeBJ8CzgdV5r0ARoG6urqrwfvA78HevXtZNs7My5Wstm3bFk4zyXUgx48f/yGwEjjlyaMdeUzPTd9RHuUxT14l8piea9/hCOCiiFyGeGBvgmk7SENDA3f6GDgDCuTZ+RPgfeUcjM4HxfJuvPHGEeAP4NChQ8mXugZwkUnegZkzZ3IZyMFgwoQJXJY5OU8e7chjAvJoRR7lMbPHxSC26TBwPj45r7W1FYYaAmk1do8XttRMBXnzEuEyyfT4HeB8UCwP4m4HeeI5tzSXSa07v6iobXU2eUxFHhOQR1eq6LGxsZEPN7GtXASLgU6nSM7r6uq6ClARV1/c0ptgy/dAxCNvs/0J7yJ7LAAXUv4gkMf0vETkMQF5dEUe3fMS8dfj8uXLDR7p0LA5Mde+ww3gALDv0QGmT58erQw9+eSTefMsdHd3NwF6vBI4H1eSx+PHj48BjN+5cye+3l3JefJoQR6TkUcn5DFrnoXL0CPbAcrz6EZra2vQI3m+S7J8j88880y4jucfgfNxBcrH+ZBfAh8HqFpx+VC2H992223pefJoQR6TkUcrfecRP4R8WDR8chE8Vvf3sS893gQeffTRNwHm8c/m5uavAuc8ebQgj065bnl25LF3njxa8NpjY2Ojof+R0CNqQc65bnl2qu5x8+bN1fUId5DVHK7fhVvL9x65yPSdIGuePFqQR6dc5+u0II+98+TRgtced+zYYfmEbaxOr+lcKh5HjhxZXY+jR48OAyaAtra2zNcYyZNHC/LolOt8nRbksXeePFrw1iNfE7W/hMNP7ZYrczNdq4Fdu3Zd4h7vuOOOL4GPgg0AhbgPcBjdSZA1Tx4tyKNTrvN1WpDH3nnuHu0tqOx/vLcHqNzBHfn3AReIGC7o8TR4BATDknnyIWDBggWWA/LkLQT19fV0yKHQ/wbOx+bJuxVA6ROAA6yGAXx1OC7AOU8eDcijPMqjE/JoznPzSFONILYJdZsBvWF1h58sB4d7UaScEZ4HkXFzdPgwSC1nppBagMLQ426Q6diC5WOn4+ieR0u3aULk0Y48yqM8OiGP5jw3j6y2UM3ixYunRuSh+sJ21eCTxdaKkCHXrVQxOjo6OCNUxOMvQLl5rwG2d+JGsr7h9pJMgTwDnJ0DKlHTqufUHU558hhDHuUxT54BeSwhz1+PEEePjT2DyvmHZaKV8jweB78Dc+bMaQW7AP/EM3nEIeeJSpjoKUteBNYugvbxh4DzcXnzLJw8eZKt5xw755QnjzHkUR7z5Fko2ePyYKRc0GRqfucxxDxiwJxr+bSlpYU/edeC5Pms4HAZKJpXybNgLYBHlvhrwPlYe14nyHQa8GvAm/Dcc8+l58ljDHmUR3uePMpjGR4DgW4zOuJx0jJqoDLX/mlNpGuxpjeRLcOHD+cs7ekXlafesRHA40TA/s5Mx1bmTQPO031F4LQkrG4dPXo0PU8eDcijPMpj8Kk8BpTpMdNUx0FTa/JO9jzWBSKtp8n1nAAuaJI3z8xfAWc8wQ1Mb9x0ynsP2AqWLl2a6UxfBPQ4Y8aM9Dx5jCGP8mjO6w8ey8ee9wroLS3dY/n9VlzQKmhcfRVkK5wxj2//XwcGDRrE5lIuZjULHDx4kO87Wk6DZ+P7AcvY3t6eniePMeTRDXmUR3m05vUjj3yjYurUqTGP14BfgVtAbW0t57YNdqhO/2NQx6kbO3ZstoIl5vExFM+BbDLl6DgGbNq0iYMOHg5ARWhOAGfYHzZsGMs4EzjlyWMMeXRDHuVRHq15/cgj6ejoWA8eA/SIYn8ZRHbgDTnQQ3X6Hzkor3yPETgKj+uqnDhx4hh4HNRVsn379oTFVyrz5DGGPLohj/J4Th5tef3Mo/Jy5smjH3ny6EeePPqRJ49+5MmjH3ny6EeePPqRJ49+5MmjH3ny6EeePPqRJ49+5MmjH3ny6Ede6FFc2sijH8ijH8ijH8ijH8ijH8ijH/wfjEm7MQplbmRzdHJlYW0KZW5kb2JqCjE0IDAgb2JqCjM0NDMKZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFsgMTEgMCBSIF0gL0NvdW50IDEgPj4KZW5kb2JqCjE1IDAgb2JqCjw8IC9DcmVhdG9yIChNYXRwbG90bGliIHYzLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjguMCkgL0NyZWF0aW9uRGF0ZSAoRDoyMDIzMTAxMTE2MzI1OFopCj4+CmVuZG9iagp4cmVmCjAgMTYKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDUxNzYgMDAwMDAgbiAKMDAwMDAwMDYwNyAwMDAwMCBuIAowMDAwMDAwNjI4IDAwMDAwIG4gCjAwMDAwMDA2ODggMDAwMDAgbiAKMDAwMDAwMDcwOSAwMDAwMCBuIAowMDAwMDAwNzMwIDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM0NCAwMDAwMCBuIAowMDAwMDAwNTg3IDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDU2NyAwMDAwMCBuIAowMDAwMDAwNzYyIDAwMDAwIG4gCjAwMDAwMDUxNTUgMDAwMDAgbiAKMDAwMDAwNTIzNiAwMDAwMCBuIAp0cmFpbGVyCjw8IC9TaXplIDE2IC9Sb290IDEgMCBSIC9JbmZvIDE1IDAgUiA+PgpzdGFydHhyZWYKNTM4NwolJUVPRgo=", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T16:32:58.298110\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["def show_imgs(imgs):\n", " num_imgs = imgs.shape[0] if isinstance(imgs, Tensor) else len(imgs)\n", " nrow = min(num_imgs, 4)\n", " ncol = int(math.ceil(num_imgs / nrow))\n", " imgs = torchvision.utils.make_grid(imgs, nrow=nrow, pad_value=128)\n", " imgs = imgs.clamp(min=0, max=255)\n", " np_imgs = imgs.cpu().numpy()\n", " plt.figure(figsize=(1.5 * nrow, 1.5 * ncol))\n", " plt.imshow(np.transpose(np_imgs, (1, 2, 0)), interpolation=\"nearest\")\n", " plt.axis(\"off\")\n", " plt.show()\n", " plt.close()\n", "\n", "\n", "show_imgs([train_set[i][0] for i in range(8)])"]}, {"cell_type": "markdown", "id": "02183103", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.012911, "end_time": "2023-10-11T16:32:58.470168", "exception": false, "start_time": "2023-10-11T16:32:58.457257", "status": "completed"}, "tags": []}, "source": ["## Masked autoregressive convolutions\n", "\n", "The core module of PixelCNN is its masked convolutions.\n", "In contrast to language models, we don't apply an LSTM on each pixel one-by-one.\n", "This would be inefficient because images are grids instead of sequences.\n", "Thus, it is better to rely on convolutions that have shown great success in deep CNN classification models.\n", "\n", "Nevertheless, we cannot just apply standard convolutions without any changes.\n", "Remember that during training of autoregressive models, we want to use teacher forcing which both helps the model training, and significantly reduces the time needed for training.\n", "For image modeling, teacher forcing is implemented by using a training image as input to the model, and we want to obtain as output the prediction for each pixel based on *only* its predecessors.\n", "Thus, we need to ensure that the prediction for a specific pixel can only be influenced by its predecessors and not by its own value or any \"future\" pixels.\n", "For this, we apply convolutions with a mask.\n", "\n", "Which mask we use depends on the ordering of pixels we decide on, i.e. which is the first pixel we predict,\n", "which is the second one, etc.\n", "The most commonly used ordering is to denote the upper left pixel as the start pixel,\n", "and sort the pixels row by row, as shown in the visualization at the top of the tutorial.\n", "Thus, the second pixel is on the right of the first one (first row, second column),\n", "and once we reach the end of the row, we start in the second row, first column.\n", "If we now want to apply this to our convolutions, we need to ensure that the prediction of pixel 1\n", "is not influenced by its own \"true\" input, and all pixels on its right and in any lower row.\n", "In convolutions, this means that we want to set those entries of the weight matrix to zero that take pixels on the right and below into account.\n", "As an example for a 5x5 kernel, see a mask below (figure credit - [Aaron van den Oord](https://arxiv.org/pdf/1606.05328.pdf)):\n", "\n", "
\n", "\n", "Before looking into the application of masked convolutions in PixelCNN\n", "in detail, let's first implement a module that allows us to apply an\n", "arbitrary mask to a convolution:"]}, {"cell_type": "code", "execution_count": 6, "id": "b20c6c1e", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:32:58.510742Z", "iopub.status.busy": "2023-10-11T16:32:58.510255Z", "iopub.status.idle": "2023-10-11T16:32:58.516902Z", "shell.execute_reply": "2023-10-11T16:32:58.516237Z"}, "lines_to_next_cell": 2, "papermill": {"duration": 0.039385, "end_time": "2023-10-11T16:32:58.522867", "exception": false, "start_time": "2023-10-11T16:32:58.483482", "status": "completed"}, "tags": []}, "outputs": [], "source": ["class MaskedConvolution(nn.Module):\n", " def __init__(self, c_in, c_out, mask, **kwargs):\n", " \"\"\"Implements a convolution with mask applied on its weights.\n", "\n", " Args:\n", " c_in: Number of input channels\n", " c_out: Number of output channels\n", " mask: Tensor of shape [kernel_size_H, kernel_size_W] with 0s where\n", " the convolution should be masked, and 1s otherwise.\n", " kwargs: Additional arguments for the convolution\n", " \"\"\"\n", " super().__init__()\n", " # For simplicity: calculate padding automatically\n", " kernel_size = (mask.shape[0], mask.shape[1])\n", " dilation = 1 if \"dilation\" not in kwargs else kwargs[\"dilation\"]\n", " padding = tuple(dilation * (kernel_size[i] - 1) // 2 for i in range(2))\n", " # Actual convolution\n", " self.conv = nn.Conv2d(c_in, c_out, kernel_size, padding=padding, **kwargs)\n", "\n", " # Mask as buffer => it is no parameter but still a tensor of the module\n", " # (must be moved with the devices)\n", " self.register_buffer(\"mask\", mask[None, None])\n", "\n", " def forward(self, x):\n", " self.conv.weight.data *= self.mask # Ensures zero's at masked positions\n", " return self.conv(x)"]}, {"cell_type": "markdown", "id": "e61f2bd5", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.013881, "end_time": "2023-10-11T16:32:58.550654", "exception": false, "start_time": "2023-10-11T16:32:58.536773", "status": "completed"}, "tags": []}, "source": ["### Vertical and horizontal convolution stacks\n", "\n", "To build our own autoregressive image model, we could simply stack a few masked convolutions on top of each other.\n", "This was actually the case for the original PixelCNN model, discussed in the paper\n", "[Pixel Recurrent Neural Networks](https://arxiv.org/pdf/1601.06759.pdf), but this leads to a considerable issue.\n", "When sequentially applying a couple of masked convolutions, the receptive field of a pixel\n", "show to have a \"blind spot\" on the right upper side, as shown in the figure below\n", "(figure credit - [Aaron van den Oord et al. ](https://arxiv.org/pdf/1606.05328.pdf)):\n", "\n", "
\n", "\n", "Although a pixel should be able to take into account all other pixels above and left of it,\n", "a stack of masked convolutions does not allow us to look to the upper pixels on the right.\n", "This is because the features of the pixels above, which we use for convolution,\n", "do not contain any information of the pixels on the right of the same row.\n", "If they would, we would be \"cheating\" and actually looking into the future.\n", "To overcome this issue, van den Oord et.\n", "al [2] proposed to split the convolutions into a vertical and a horizontal stack.\n", "The vertical stack looks at all pixels above the current one, while the horizontal takes into account all on the left.\n", "While keeping both of them separate, we can actually look at the pixels on the right with the vertical stack without breaking any of our assumptions.\n", "The two convolutions are also shown in the figure above.\n", "\n", "Let us implement them here as follows:"]}, {"cell_type": "code", "execution_count": 7, "id": "bc16dcf7", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:32:58.579953Z", "iopub.status.busy": "2023-10-11T16:32:58.579133Z", "iopub.status.idle": "2023-10-11T16:32:58.586061Z", "shell.execute_reply": "2023-10-11T16:32:58.585382Z"}, "papermill": {"duration": 0.024054, "end_time": "2023-10-11T16:32:58.587949", "exception": false, "start_time": "2023-10-11T16:32:58.563895", "status": "completed"}, "tags": []}, "outputs": [], "source": ["class VerticalStackConvolution(MaskedConvolution):\n", " def __init__(self, c_in, c_out, kernel_size=3, mask_center=False, **kwargs):\n", " # Mask out all pixels below. For efficiency, we could also reduce the kernel\n", " # size in height, but for simplicity, we stick with masking here.\n", " mask = torch.ones(kernel_size, kernel_size)\n", " mask[kernel_size // 2 + 1 :, :] = 0\n", "\n", " # For the very first convolution, we will also mask the center row\n", " if mask_center:\n", " mask[kernel_size // 2, :] = 0\n", "\n", " super().__init__(c_in, c_out, mask, **kwargs)\n", "\n", "\n", "class HorizontalStackConvolution(MaskedConvolution):\n", " def __init__(self, c_in, c_out, kernel_size=3, mask_center=False, **kwargs):\n", " # Mask out all pixels on the left. Note that our kernel has a size of 1\n", " # in height because we only look at the pixel in the same row.\n", " mask = torch.ones(1, kernel_size)\n", " mask[0, kernel_size // 2 + 1 :] = 0\n", "\n", " # For the very first convolution, we will also mask the center pixel\n", " if mask_center:\n", " mask[0, kernel_size // 2] = 0\n", "\n", " super().__init__(c_in, c_out, mask, **kwargs)"]}, {"cell_type": "markdown", "id": "d6ad6a83", "metadata": {"papermill": {"duration": 0.013791, "end_time": "2023-10-11T16:32:58.615712", "exception": false, "start_time": "2023-10-11T16:32:58.601921", "status": "completed"}, "tags": []}, "source": ["Note that we have an input argument called `mask_center`. Remember that\n", "the input to the model is the actual input image. Hence, the very first\n", "convolution we apply cannot use the center pixel as input, but must be\n", "masked. All consecutive convolutions, however, should use the center\n", "pixel as we otherwise lose the features of the previous layer. Hence,\n", "the input argument `mask_center` is True for the very first\n", "convolutions, and False for all others."]}, {"cell_type": "markdown", "id": "e2cb5d6f", "metadata": {"papermill": {"duration": 0.013643, "end_time": "2023-10-11T16:32:58.642976", "exception": false, "start_time": "2023-10-11T16:32:58.629333", "status": "completed"}, "tags": []}, "source": ["### Visualizing the receptive field\n", "\n", "To validate our implementation of masked convolutions, we can visualize the receptive field we obtain with such convolutions.\n", "We should see that with increasing number of convolutional layers, the receptive field grows in both vertical and horizontal direction, without the issue of a blind spot.\n", "The receptive field can be empirically measured by backpropagating an arbitrary loss for the output features of a speicifc pixel with respect to the input.\n", "We implement this idea below, and visualize the receptive field below."]}, {"cell_type": "code", "execution_count": 8, "id": "ed9b059e", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:32:58.671501Z", "iopub.status.busy": "2023-10-11T16:32:58.670577Z", "iopub.status.idle": "2023-10-11T16:32:59.242548Z", "shell.execute_reply": "2023-10-11T16:32:59.241837Z"}, "papermill": {"duration": 0.592577, "end_time": "2023-10-11T16:32:59.248506", "exception": false, "start_time": "2023-10-11T16:32:58.655929", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMzcxLjUyIDE5MS44NTIyNzI3MjczIF0gL0NvbnRlbnRzIDkgMCBSIC9Bbm5vdHMgMTAgMCBSID4+CmVuZG9iago5IDAgb2JqCjw8IC9MZW5ndGggMTIgMCBSIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nJ1Qy27CMBC871fMEQ51vHYdJ8dSWtTeaCNxQD1UwYREhDaAQP37LkHpA1Uqra2Rduy1Z3aiYdiVeXgYDXD9SNEnyzfEqAQFNCrBHoyRoCAtrCbrWTkj5bIrOWWVOGO8lTP9nS6I5tTAK9OCY6Os8cdtT+k6YIIVoitR24hkJdiL0ujEYNM+9AcTuis7hbxGdMcYvmBMYzTSUHz96cCpkfE0LuQt2CnWLjE2iW0M9k4G+vCT1zTIKLqVNoNs3gaQzWiK3qQPlyiXWu/QC2Wx2IYZ1n0YrRLbLbnJw+u23AXMy7Cc9fGE7J5uMjpaM5pVmnZy/82HDlP9ns8PYm1U5uyoDKdKa81sJarL87MalKvn9dvf0qF3+fSWJgplbmRzdHJlYW0KZW5kb2JqCjEyIDAgb2JqCjI4NAplbmRvYmoKMTAgMCBvYmoKWyBdCmVuZG9iagoxOSAwIG9iago8PCAvTGVuZ3RoIDI2NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9UrmRAzEMy7cKlsBfUj2+uXFg958ewD07MTFLEQBB925RORs/bSXLj/zYZWdJ5Jb3oG3yuqLqBqmbIHPJcckVYpbyuBIkFi1lJtZnqoPycQ1qFb7wEzMT0yFJxBJyUo8irI+vg9f1HNxfN+n8GhkfdGxQekuSq6BUw75ytBI7lupdg+yDppvS6jPTruyApfGGrNSkTn8d9b8jLMKk3khFByEWv9PLHbIspBzU27l+A+Fd7YJYT6087BBp3lZ6SxXM5swETBltO6yAtVljwlQJ8BbNIdRaiMwXOq2I+eTc0cE0VXkaIsNShYPtPaM1XOgaEkvD+UnGBOa/8PqsyG1//wBwaGe6CmVuZHN0cmVhbQplbmRvYmoKMjAgMCBvYmoKPDwgL0xlbmd0aCA5MiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9jcENwDAIA/9MwQgQAsT7VFUf6f7fJhHqBx8G2RhgYbM14MHZwJfS2je9pEWT2ghWtUXdUJ67FKVYXUelTMJPmTt/UnQc7XAO29/W5ThN4+hf99D9AQ9KHgsKZW5kc3RyZWFtCmVuZG9iagoyMSAwIG9iago8PCAvTGVuZ3RoIDMwNyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9kktuAzEMQ/c+hS4QwPrZnvOkKLqY3n/bJyXpihzZFkVqlrpMWVMekDSThH/p8HCxnfI7bM9mZuBaopeJ5ZTn0BVi7qJ82cxGXVknxeqEZjq36FE5Fwc2Taqfqyyl3S54Dtcmnlv2ET+80KAe1DUuCTd0V6NlKTRjqvt/0nv8jDLgakxdbFKrex88XkRV6OgHR4kiY5cX5+NBCelKwmhaiJV3RQNB7vK0ynsJ7tveasiyB6mYzjspZrDrdFIubheHIR7I8qjw5aPYa0LP+LArJfRI2IYzcifuaMbm1MjikP7ejQRLj65oIfPgr27WLmC8UzpFYmROcqxpi1VO91AU07nDvQwQ9WxFQylzkdXqX8POC2uWbBZ4SvoFHqPdJksOVtnbqE7vrTzZ0PcfWtd0HwplbmRzdHJlYW0KZW5kb2JqCjIyIDAgb2JqCjw8IC9MZW5ndGggMjMyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVRSW7EMAy7+xX8wADW7rwnxaCH9v/XUsoUCEAltrglYmMjAi8x+DmI3PiSNaMmfmdyV/wsT4VHwq3gSRSBl+FedoLLG8ZlPw4zH7yXVs6kxpMMyEU2PTwRMtglEDowuwZ12Gbaib4h4bMjUs1GltPXEvTSKgTKU7bf6YISbav6c/usC2372hNOdnvqSeUTiOeWrMBl4xWTxVgGPVG5SzF9kOpsoSehvCifg2w+aohElyhn4InBwSjQDuy57WfiVSFoXd2nbWOoRkrH078NTU2SCPlECWe2NO4W/n/Pvb7X+w9OIVQRCmVuZHN0cmVhbQplbmRvYmoKMjMgMCBvYmoKPDwgL0xlbmd0aCAyMzEgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicNU85kgQhDMt5hT4wVRjbQL+np7Y22Pl/upKZTpDwIcnTEx2ZeJkjI7Bmx9taZCBm4FNMxb/2tA8TqvfgHiKUiwthhpFw1qzjbp6OF/92lc9YB+82+IpZXhDYwkzWVxZnLtsFY2mcxDnJboxdE7GNda2nU1hHMKEMhHS2w5Qgc1Sk9MmOMuboOJEnnovv9tssdjl+DusLNo0hFef4KnqCNoOi7HnvAhpyQf9d3fgeRbvoJSAbCRbWUWLunOWEX712dB61KBJzQppBLhMhzekqphCaUKyzo6BSUXCpPqforJ9/5V9cLQplbmRzdHJlYW0KZW5kb2JqCjI0IDAgb2JqCjw8IC9MZW5ndGggMjQ5IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD1QO45EIQzrOYUv8CTyI3AeRqstZu/frgOaKVBMfrYzJNARgUcMMZSv4yWtoK6Bv4tC8W7i64PCIKtDUiDOeg+IdOymNpETOh2cMz9hN2OOwEUxBpzpdKY9ByY5+8IKhHMbZexWSCeJqiKO6jOOKZ4qe594FiztyDZbJ5I95CDhUlKJyaWflMo/bcqUCjpm0QQsErngZBNNOMu7SVKMGZQy6h6mdiJ9rDzIozroZE3OrCOZ2dNP25n4HHC3X9pkTpXHdB7M+Jy0zoM5Fbr344k2B02N2ujs9xNpKi9Sux1anX51EpXdGOcYEpdnfxnfZP/5B/6HWiIKZW5kc3RyZWFtCmVuZG9iagoyNSAwIG9iago8PCAvTGVuZ3RoIDEzNiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxNj0EOAzEIA+95hZ9AIEB4z1ZVD9v/X0vYdtMLHsmAbFEGgSWHeIcb4dHbD99FNhVn45xfUiliIZhPcJ8wUxyNKXfyY4+AcZRqLKdoeF5Lzk3DFy13Ey2lrZeTGW+47pf3R5VtkQ1Fzy0LQtdskvkygQd8GJhHdeNppcfd9myv9vwAzmw0SQplbmRzdHJlYW0KZW5kb2JqCjI2IDAgb2JqCjw8IC9MZW5ndGggMzQxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEVSS25EMQjbv1NwgUjhl5DztKq6mN5/W5tM1c3gCWBseMtTpmTKsLklIyTXlE99IkOspvw0ciQipvhJCQV2lY/Ha0usjeyRqBSf2vHjsfRGptkVWvXu0aXNolHNysg5yBChnhW6snvUDtnwelxIuu+UzSEcy/9QgSxl3XIKJUFb0HfsEd8PHa6CK4JhsGsug+1lMtT/+ocWXO9992LHLoAWrOe+wQ4AqKcTtAXIGdruNiloAFW6i0nCo/J6bnaibKNV6fkcADMOMHLAiCVbHb7R3gCWfV3oRY2K/StAUVlA/MjVdsHeMclIcBbmBo69cDzFmXBLOMYCQIq94hh68CXY5i9Xroia8Al1umQvvMKe2ubnQpMId60ADl5kw62ro6iW7ek8gvZnRXJGjNSLODohklrSOYLi0qAeWuNcN7HibSOxuVff7h/hnC9c9usXS+yExAplbmRzdHJlYW0KZW5kb2JqCjI3IDAgb2JqCjw8IC9MZW5ndGggMTY0IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWQx3EFMQxD76oCJTCACvWsx/MP6/6vhvTTQXoYQgxiT8KwXFdxYXTDj7ctMw1/RxnuxvoyY7zVWCAn6AMMkYmr0aT6dsUZqvTk1WKuo6JcLzoiEsyS46tAI3w6sseTtrYz/XReH+wh7xP/KirnbmEBLqruQPlSH/HUj9lR6pqhjyorax5q2leEXRFK2z4upzJO3b0DWuG9las92u8/HnY68gplbmRzdHJlYW0KZW5kb2JqCjI4IDAgb2JqCjw8IC9MZW5ndGggNzIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicMzK3UDBQsDQBEoYWJgrmZgYKKYZcQL6piblCLhdIDMTKAbMMgLQlnIKIZ4CYIG0QxSAWRLGZiRlEHZwBkcvgSgMAJdsWyQplbmRzdHJlYW0KZW5kb2JqCjI5IDAgb2JqCjw8IC9MZW5ndGggNDcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicMzK3UDBQsDQBEoYWJgrmZgYKKYZclhBWLhdMLAfMAtGWcAoinsGVBgC5Zw0nCmVuZHN0cmVhbQplbmRvYmoKMzAgMCBvYmoKPDwgL0xlbmd0aCAxNjMgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicRZA7EgMhDEN7TqEj+CMDPs9mMik2929j2GxSwNNYIIO7E4LU2oKJ6IKHtiXdBe+tBGdj/Ok2bjUS5AR1gFak42iUUn25xWmVdPFoNnMrC60THWYOepSjGaAQOhXe7aLkcqbuzvlDcPVf9b9i3TmbiYHJyh0IzepT3Pk2O6K6usn+pMfcrNd+K+xVYWlZS8sJt527ZkAJ3FM52qs9Px8KOvYKZW5kc3RyZWFtCmVuZG9iagozMSAwIG9iago8PCAvTGVuZ3RoIDIzOSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxNUMltBDEM+7sKNTDA6By7HgeLPLL9f0PKCZKXaEviofKUW5bKZfcjOW/JuuVDh06VafJu0M2vsf6jDAJ2/1BUEK0lsUrMXNJusTRJL9nDOI2Xa7WO56l7hFmjePDj2NMpgek9MsFms705MKs9zg6QTrjGr+rTO5UkA4m6kPNCpQrrHtQloo8r25hSnU4t5RiXn+h7fI4APcXejdzRx8sXjEa1LajRapU4DzATU9GVcauRgZQTBkNnR1c0C6XIynpCNcKNOaGZvcNwYAPLs4Skpa1SvA9lAegCXdo64zRKgo4Awt8ojPX6Bqr8XjcKZW5kc3RyZWFtCmVuZG9iagozMiAwIG9iago8PCAvTGVuZ3RoIDE2MCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFkDkSAzEIBHO9gidIXIL3rMu1wfr/qQfWR6LpAjQcuhZNynoUaD7psUahutBr6CxKkkTBFpIdUKdjiDsoSExIY5JIth6DI5pYs12YmVQqs1LhtGnFwr/ZWtXIRI1wjfyJ6QZU/E/qXJTwTYOvkjH6GFS8O4OMSfheRdxaMe3+RDCxGfYJb0UmBYSJsanZvs9ghsz3Ctc4x/MNTII36wplbmRzdHJlYW0KZW5kb2JqCjMzIDAgb2JqCjw8IC9MZW5ndGggMTggL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicMza0UDCAwxRDrjQAHeYDUgplbmRzdHJlYW0KZW5kb2JqCjM0IDAgb2JqCjw8IC9MZW5ndGggMTMzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWPSw4EIQhE95yijsDHH+dxMumFc//tgJ1uE2M9hVSBuYKhPS5rA50VHyEZtvG3qZaORVk+VHpSVg/J4Iesxssh3KAs8IJJKoYhUIuYGpEtZW63gNs2DbKylVOljrCLozCP9rRsFR5folsidZI/g8QqL9zjuh3Ipda73qKLvn+kATEJCmVuZHN0cmVhbQplbmRvYmoKMzUgMCBvYmoKPDwgL0xlbmd0aCA3NSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJwztTRSMFAwNgASpmZGCqYm5gophlxAPoiVy2VoZApm5XAZWZopWFgAGSZm5lAhmIYcLmNTc6ABQEXGpmAaqj+HK4MrDQCVkBLvCmVuZHN0cmVhbQplbmRvYmoKMzYgMCBvYmoKPDwgL0xlbmd0aCAxNDEgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPY/BDsMwCEPv+Qr/QKTYKaF8T6dqh+7/ryNLuwt6AmOMhdDQG6qaw4Zgm+PF0iVUa/gUxUAlN8iZYA6lpNIdR5F6YjgYXB60G47isej6EbuSZn3QxkK6JWiAe6xTadymcRPEHTUF6inqnKO8ELmfqWfYNJLdNLOSc7gNv3vPU9f/p6u8y/kFvXcu/gplbmRzdHJlYW0KZW5kb2JqCjE3IDAgb2JqCjw8IC9UeXBlIC9Gb250IC9CYXNlRm9udCAvQk1RUURWK0RlamFWdVNhbnMgL0ZpcnN0Q2hhciAwIC9MYXN0Q2hhciAyNTUKL0ZvbnREZXNjcmlwdG9yIDE2IDAgUiAvU3VidHlwZSAvVHlwZTMgL05hbWUgL0JNUVFEVitEZWphVnVTYW5zCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnRNYXRyaXggWyAwLjAwMSAwIDAgMC4wMDEgMCAwIF0KL0NoYXJQcm9jcyAxOCAwIFIKL0VuY29kaW5nIDw8IC9UeXBlIC9FbmNvZGluZwovRGlmZmVyZW5jZXMgWyAzMiAvc3BhY2UgNjYgL0IgODcgL1cgOTcgL2EgOTkgL2MgL2QgL2UgL2YgL2cgL2ggL2kgMTA4IC9sIDExMCAvbiAxMTIgL3AKMTE0IC9yIDExNiAvdCAxMTggL3YgMTIxIC95IF0KPj4KL1dpZHRocyAxNSAwIFIgPj4KZW5kb2JqCjE2IDAgb2JqCjw8IC9UeXBlIC9Gb250RGVzY3JpcHRvciAvRm9udE5hbWUgL0JNUVFEVitEZWphVnVTYW5zIC9GbGFncyAzMgovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Bc2NlbnQgOTI5IC9EZXNjZW50IC0yMzYgL0NhcEhlaWdodCAwCi9YSGVpZ2h0IDAgL0l0YWxpY0FuZ2xlIDAgL1N0ZW1WIDAgL01heFdpZHRoIDEzNDIgPj4KZW5kb2JqCjE1IDAgb2JqClsgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAKNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCAzMTggNDAxIDQ2MCA4MzggNjM2Cjk1MCA3ODAgMjc1IDM5MCAzOTAgNTAwIDgzOCAzMTggMzYxIDMxOCAzMzcgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNgo2MzYgNjM2IDMzNyAzMzcgODM4IDgzOCA4MzggNTMxIDEwMDAgNjg0IDY4NiA2OTggNzcwIDYzMiA1NzUgNzc1IDc1MiAyOTUKMjk1IDY1NiA1NTcgODYzIDc0OCA3ODcgNjAzIDc4NyA2OTUgNjM1IDYxMSA3MzIgNjg0IDk4OSA2ODUgNjExIDY4NSAzOTAgMzM3CjM5MCA4MzggNTAwIDUwMCA2MTMgNjM1IDU1MCA2MzUgNjE1IDM1MiA2MzUgNjM0IDI3OCAyNzggNTc5IDI3OCA5NzQgNjM0IDYxMgo2MzUgNjM1IDQxMSA1MjEgMzkyIDYzNCA1OTIgODE4IDU5MiA1OTIgNTI1IDYzNiAzMzcgNjM2IDgzOCA2MDAgNjM2IDYwMCAzMTgKMzUyIDUxOCAxMDAwIDUwMCA1MDAgNTAwIDEzNDIgNjM1IDQwMCAxMDcwIDYwMCA2ODUgNjAwIDYwMCAzMTggMzE4IDUxOCA1MTgKNTkwIDUwMCAxMDAwIDUwMCAxMDAwIDUyMSA0MDAgMTAyMyA2MDAgNTI1IDYxMSAzMTggNDAxIDYzNiA2MzYgNjM2IDYzNiAzMzcKNTAwIDUwMCAxMDAwIDQ3MSA2MTIgODM4IDM2MSAxMDAwIDUwMCA1MDAgODM4IDQwMSA0MDEgNTAwIDYzNiA2MzYgMzE4IDUwMAo0MDEgNDcxIDYxMiA5NjkgOTY5IDk2OSA1MzEgNjg0IDY4NCA2ODQgNjg0IDY4NCA2ODQgOTc0IDY5OCA2MzIgNjMyIDYzMiA2MzIKMjk1IDI5NSAyOTUgMjk1IDc3NSA3NDggNzg3IDc4NyA3ODcgNzg3IDc4NyA4MzggNzg3IDczMiA3MzIgNzMyIDczMiA2MTEgNjA1CjYzMCA2MTMgNjEzIDYxMyA2MTMgNjEzIDYxMyA5ODIgNTUwIDYxNSA2MTUgNjE1IDYxNSAyNzggMjc4IDI3OCAyNzggNjEyIDYzNAo2MTIgNjEyIDYxMiA2MTIgNjEyIDgzOCA2MTIgNjM0IDYzNCA2MzQgNjM0IDU5MiA2MzUgNTkyIF0KZW5kb2JqCjE4IDAgb2JqCjw8IC9CIDE5IDAgUiAvVyAyMCAwIFIgL2EgMjEgMCBSIC9jIDIyIDAgUiAvZCAyMyAwIFIgL2UgMjQgMCBSIC9mIDI1IDAgUgovZyAyNiAwIFIgL2ggMjcgMCBSIC9pIDI4IDAgUiAvbCAyOSAwIFIgL24gMzAgMCBSIC9wIDMxIDAgUiAvciAzMiAwIFIKL3NwYWNlIDMzIDAgUiAvdCAzNCAwIFIgL3YgMzUgMCBSIC95IDM2IDAgUiA+PgplbmRvYmoKMyAwIG9iago8PCAvRjEgMTcgMCBSID4+CmVuZG9iago0IDAgb2JqCjw8IC9BMSA8PCAvVHlwZSAvRXh0R1N0YXRlIC9DQSAxIC9jYSAxID4+ID4+CmVuZG9iago1IDAgb2JqCjw8ID4+CmVuZG9iago2IDAgb2JqCjw8ID4+CmVuZG9iago3IDAgb2JqCjw8IC9JMSAxMyAwIFIgL0kyIDE0IDAgUiA+PgplbmRvYmoKMTMgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL1N1YnR5cGUgL0ltYWdlIC9XaWR0aCAyMjYgL0hlaWdodCAyMjYKL0NvbG9yU3BhY2UgWyAvSW5kZXhlZCAvRGV2aWNlUkdCIDEgKP3nNwAiTSkgXSAvQml0c1BlckNvbXBvbmVudCAxCi9GaWx0ZXIgL0ZsYXRlRGVjb2RlCi9EZWNvZGVQYXJtcyA8PCAvUHJlZGljdG9yIDEwIC9Db2xvcnMgMSAvQ29sdW1ucyAyMjYgL0JpdHNQZXJDb21wb25lbnQgMSA+PgovTGVuZ3RoIDM3IDAgUiA+PgpzdHJlYW0KeJztySEBACAQBLBrQv8WJHvwCBTq2eyyLmastdZaa621/2wlw9rXe7LWWmuttdY23w351RDUCmVuZHN0cmVhbQplbmRvYmoKMzcgMCBvYmoKNTQKZW5kb2JqCjE0IDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiAxICj95zcAIk0pIF0gL0JpdHNQZXJDb21wb25lbnQgMQovRmlsdGVyIC9GbGF0ZURlY29kZQovRGVjb2RlUGFybXMgPDwgL1ByZWRpY3RvciAxMCAvQ29sb3JzIDEgL0NvbHVtbnMgMjI2IC9CaXRzUGVyQ29tcG9uZW50IDEgPj4KL0xlbmd0aCAzOCAwIFIgPj4Kc3RyZWFtCnic7ckhAQAgEASwa0L/FiR78AgU6tnssi5mrLXWWmuttf9sJcPa13uy1lprrbXWNt8N+dUQ1AplbmRzdHJlYW0KZW5kb2JqCjM4IDAgb2JqCjU0CmVuZG9iagoyIDAgb2JqCjw8IC9UeXBlIC9QYWdlcyAvS2lkcyBbIDExIDAgUiBdIC9Db3VudCAxID4+CmVuZG9iagozOSAwIG9iago8PCAvQ3JlYXRvciAoTWF0cGxvdGxpYiB2My44LjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcpCi9Qcm9kdWNlciAoTWF0cGxvdGxpYiBwZGYgYmFja2VuZCB2My44LjApIC9DcmVhdGlvbkRhdGUgKEQ6MjAyMzEwMTExNjMyNTlaKQo+PgplbmRvYmoKeHJlZgowIDQwCjAwMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAwMDAxNiAwMDAwMCBuIAowMDAwMDA3ODU5IDAwMDAwIG4gCjAwMDAwMDY5OTAgMDAwMDAgbiAKMDAwMDAwNzAyMiAwMDAwMCBuIAowMDAwMDA3MDgyIDAwMDAwIG4gCjAwMDAwMDcxMDMgMDAwMDAgbiAKMDAwMDAwNzEyNCAwMDAwMCBuIAowMDAwMDAwMDY1IDAwMDAwIG4gCjAwMDAwMDAzNDQgMDAwMDAgbiAKMDAwMDAwMDcyMyAwMDAwMCBuIAowMDAwMDAwMjA4IDAwMDAwIG4gCjAwMDAwMDA3MDMgMDAwMDAgbiAKMDAwMDAwNzE2NyAwMDAwMCBuIAowMDAwMDA3NTEzIDAwMDAwIG4gCjAwMDAwMDU3MzEgMDAwMDAgbiAKMDAwMDAwNTUyNCAwMDAwMCBuIAowMDAwMDA1MTE2IDAwMDAwIG4gCjAwMDAwMDY3ODQgMDAwMDAgbiAKMDAwMDAwMDc0MyAwMDAwMCBuIAowMDAwMDAxMDgwIDAwMDAwIG4gCjAwMDAwMDEyNDQgMDAwMDAgbiAKMDAwMDAwMTYyNCAwMDAwMCBuIAowMDAwMDAxOTI5IDAwMDAwIG4gCjAwMDAwMDIyMzMgMDAwMDAgbiAKMDAwMDAwMjU1NSAwMDAwMCBuIAowMDAwMDAyNzY0IDAwMDAwIG4gCjAwMDAwMDMxNzggMDAwMDAgbiAKMDAwMDAwMzQxNSAwMDAwMCBuIAowMDAwMDAzNTU5IDAwMDAwIG4gCjAwMDAwMDM2NzggMDAwMDAgbiAKMDAwMDAwMzkxNCAwMDAwMCBuIAowMDAwMDA0MjI2IDAwMDAwIG4gCjAwMDAwMDQ0NTkgMDAwMDAgbiAKMDAwMDAwNDU0OSAwMDAwMCBuIAowMDAwMDA0NzU1IDAwMDAwIG4gCjAwMDAwMDQ5MDIgMDAwMDAgbiAKMDAwMDAwNzQ5NCAwMDAwMCBuIAowMDAwMDA3ODQwIDAwMDAwIG4gCjAwMDAwMDc5MTkgMDAwMDAgbiAKdHJhaWxlcgo8PCAvU2l6ZSA0MCAvUm9vdCAxIDAgUiAvSW5mbyAzOSAwIFIgPj4Kc3RhcnR4cmVmCjgwNzAKJSVFT0YK", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T16:32:59.086723\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["inp_img = torch.zeros(1, 1, 11, 11)\n", "inp_img.requires_grad_()\n", "\n", "\n", "def show_center_recep_field(img, out):\n", " \"\"\"Calculates the gradients of the input with respect to the output center pixel, and visualizes the overall\n", " receptive field.\n", "\n", " Args:\n", " img: Input image for which we want to calculate the receptive field on.\n", " out: Output features/loss which is used for backpropagation, and should be\n", " the output of the network/computation graph.\n", " \"\"\"\n", " # Determine gradients\n", " loss = out[0, :, img.shape[2] // 2, img.shape[3] // 2].sum() # L1 loss for simplicity\n", " # Retain graph as we want to stack multiple layers and show the receptive field of all of them\n", " loss.backward(retain_graph=True)\n", " img_grads = img.grad.abs()\n", " img.grad.fill_(0) # Reset grads\n", "\n", " # Plot receptive field\n", " img = img_grads.squeeze().cpu().numpy()\n", " fig, ax = plt.subplots(1, 2)\n", " _ = ax[0].imshow(img)\n", " ax[1].imshow(img > 0)\n", " # Mark the center pixel in red if it doesn't have any gradients (should be\n", " # the case for standard autoregressive models)\n", " show_center = img[img.shape[0] // 2, img.shape[1] // 2] == 0\n", " if show_center:\n", " center_pixel = np.zeros(img.shape + (4,))\n", " center_pixel[center_pixel.shape[0] // 2, center_pixel.shape[1] // 2, :] = np.array([1.0, 0.0, 0.0, 1.0])\n", " for i in range(2):\n", " ax[i].axis(\"off\")\n", " if show_center:\n", " ax[i].imshow(center_pixel)\n", " ax[0].set_title(\"Weighted receptive field\")\n", " ax[1].set_title(\"Binary receptive field\")\n", " plt.show()\n", " plt.close()\n", "\n", "\n", "show_center_recep_field(inp_img, inp_img)"]}, {"cell_type": "markdown", "id": "4b2cc8e1", "metadata": {"papermill": {"duration": 0.023645, "end_time": "2023-10-11T16:32:59.294318", "exception": false, "start_time": "2023-10-11T16:32:59.270673", "status": "completed"}, "tags": []}, "source": ["Let's first visualize the receptive field of a horizontal convolution\n", "without the center pixel. We use a small, arbitrary input image\n", "($11\\times 11$ pixels), and calculate the loss for the center pixel. For\n", "simplicity, we initialize all weights with 1 and the bias with 0, and\n", "use a single channel. This is sufficient for our visualization purposes."]}, {"cell_type": "code", "execution_count": 9, "id": "693bf136", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:32:59.325572Z", "iopub.status.busy": "2023-10-11T16:32:59.324600Z", "iopub.status.idle": "2023-10-11T16:32:59.664773Z", "shell.execute_reply": "2023-10-11T16:32:59.664021Z"}, "papermill": {"duration": 0.358925, "end_time": "2023-10-11T16:32:59.667560", "exception": false, "start_time": "2023-10-11T16:32:59.308635", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMzcxLjUyIDE5MS44NTIyNzI3MjczIF0gL0NvbnRlbnRzIDkgMCBSIC9Bbm5vdHMgMTAgMCBSID4+CmVuZG9iago5IDAgb2JqCjw8IC9MZW5ndGggMTIgMCBSIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nJ1QTU8CMRC9z694RzjY7UztdvcookRv6CYcjAezlGUJoAsE4r93gKwfXEQ7ecm8Tqfz5iX9uK3L+DDo4fqRki9WrokxU1SwmCl2YAwUFVllC3KBjRdN523KOZvMiwSnd/YnnRJNqEEwcgCnYpyEY7hTuooYYYnkSqetdeRMsdNJgxOBzaEx7EXYNg0a5QLJHaP/iiEN0Wix+v7LnlOjq1lcaB/YG7Y+E5elLgUHr8t8aikX1CsoudVngmJyWL4Y0xM6oy58Znzugkcn1tV0E8dYdSHWZK49Winj26beRkzqOB938Yzinm4KOkoTyybP23H/9Yb2W/3ujVhp3ZGz3RHOjbWW2ak7l+fb06uXL6v3vxlCH5GVksMKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyODIKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0xlbmd0aCAyNjQgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9MZW5ndGggOTIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0xlbmd0aCAzMDcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvTGVuZ3RoIDIzMiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9MZW5ndGggMjMxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDI0OSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0xlbmd0aCAxMzYgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvTGVuZ3RoIDM0MSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvTGVuZ3RoIDE2NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvTGVuZ3RoIDcyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvTGVuZ3RoIDQ3IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9MZW5ndGggMTYzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0xlbmd0aCAyMzkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0xlbmd0aCAxNjAgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvTGVuZ3RoIDE4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvTGVuZ3RoIDEzMyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9MZW5ndGggNzUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9MZW5ndGggMTQxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvVHlwZSAvRm9udCAvQmFzZUZvbnQgL0JNUVFEVitEZWphVnVTYW5zIC9GaXJzdENoYXIgMCAvTGFzdENoYXIgMjU1Ci9Gb250RGVzY3JpcHRvciAxNiAwIFIgL1N1YnR5cGUgL1R5cGUzIC9OYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdCi9DaGFyUHJvY3MgMTggMCBSCi9FbmNvZGluZyA8PCAvVHlwZSAvRW5jb2RpbmcKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCj4+Ci9XaWR0aHMgMTUgMCBSID4+CmVuZG9iagoxNiAwIG9iago8PCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucyAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvQXNjZW50IDkyOSAvRGVzY2VudCAtMjM2IC9DYXBIZWlnaHQgMAovWEhlaWdodCAwIC9JdGFsaWNBbmdsZSAwIC9TdGVtViAwIC9NYXhXaWR0aCAxMzQyID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiAyICj95zf/AAAAIk0pIF0gL0JpdHNQZXJDb21wb25lbnQgMgovRmlsdGVyIC9GbGF0ZURlY29kZQovRGVjb2RlUGFybXMgPDwgL1ByZWRpY3RvciAxMCAvQ29sb3JzIDEgL0NvbHVtbnMgMjI2IC9CaXRzUGVyQ29tcG9uZW50IDIgPj4KL0xlbmd0aCAzNyAwIFIgPj4Kc3RyZWFtCnic7cuxDQAgDMCwciTcx8yVtB8gFgacMZJjXTaDJEmSJEmSJEmSJEmSJEmSJMm6WevZIEmSfCJPIkmSJEmSJEmSJEmSJEmSJMnv5AaKEP9FCmVuZHN0cmVhbQplbmRvYmoKMzcgMCBvYmoKODAKZW5kb2JqCjE0IDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiAyICj95zf/AAAAIk0pIF0gL0JpdHNQZXJDb21wb25lbnQgMgovRmlsdGVyIC9GbGF0ZURlY29kZQovRGVjb2RlUGFybXMgPDwgL1ByZWRpY3RvciAxMCAvQ29sb3JzIDEgL0NvbHVtbnMgMjI2IC9CaXRzUGVyQ29tcG9uZW50IDIgPj4KL0xlbmd0aCAzOCAwIFIgPj4Kc3RyZWFtCnic7cuxDQAgDMCwciTcx8yVtB8gFgacMZJjXTaDJEmSJEmSJEmSJEmSJEmSJMm6WevZIEmSfCJPIkmSJEmSJEmSJEmSJEmSJMnv5AaKEP9FCmVuZHN0cmVhbQplbmRvYmoKMzggMCBvYmoKODAKZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFsgMTEgMCBSIF0gL0NvdW50IDEgPj4KZW5kb2JqCjM5IDAgb2JqCjw8IC9DcmVhdG9yIChNYXRwbG90bGliIHYzLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjguMCkgL0NyZWF0aW9uRGF0ZSAoRDoyMDIzMTAxMTE2MzI1OVopCj4+CmVuZG9iagp4cmVmCjAgNDAKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDc5MTUgMDAwMDAgbiAKMDAwMDAwNjk4OCAwMDAwMCBuIAowMDAwMDA3MDIwIDAwMDAwIG4gCjAwMDAwMDcwODAgMDAwMDAgbiAKMDAwMDAwNzEwMSAwMDAwMCBuIAowMDAwMDA3MTIyIDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM0NCAwMDAwMCBuIAowMDAwMDAwNzIxIDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDcwMSAwMDAwMCBuIAowMDAwMDA3MTY1IDAwMDAwIG4gCjAwMDAwMDc1NDAgMDAwMDAgbiAKMDAwMDAwNTcyOSAwMDAwMCBuIAowMDAwMDA1NTIyIDAwMDAwIG4gCjAwMDAwMDUxMTQgMDAwMDAgbiAKMDAwMDAwNjc4MiAwMDAwMCBuIAowMDAwMDAwNzQxIDAwMDAwIG4gCjAwMDAwMDEwNzggMDAwMDAgbiAKMDAwMDAwMTI0MiAwMDAwMCBuIAowMDAwMDAxNjIyIDAwMDAwIG4gCjAwMDAwMDE5MjcgMDAwMDAgbiAKMDAwMDAwMjIzMSAwMDAwMCBuIAowMDAwMDAyNTUzIDAwMDAwIG4gCjAwMDAwMDI3NjIgMDAwMDAgbiAKMDAwMDAwMzE3NiAwMDAwMCBuIAowMDAwMDAzNDEzIDAwMDAwIG4gCjAwMDAwMDM1NTcgMDAwMDAgbiAKMDAwMDAwMzY3NiAwMDAwMCBuIAowMDAwMDAzOTEyIDAwMDAwIG4gCjAwMDAwMDQyMjQgMDAwMDAgbiAKMDAwMDAwNDQ1NyAwMDAwMCBuIAowMDAwMDA0NTQ3IDAwMDAwIG4gCjAwMDAwMDQ3NTMgMDAwMDAgbiAKMDAwMDAwNDkwMCAwMDAwMCBuIAowMDAwMDA3NTIxIDAwMDAwIG4gCjAwMDAwMDc4OTYgMDAwMDAgbiAKMDAwMDAwNzk3NSAwMDAwMCBuIAp0cmFpbGVyCjw8IC9TaXplIDQwIC9Sb290IDEgMCBSIC9JbmZvIDM5IDAgUiA+PgpzdGFydHhyZWYKODEyNgolJUVPRgo=", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T16:32:59.497963\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["horiz_conv = HorizontalStackConvolution(c_in=1, c_out=1, kernel_size=3, mask_center=True)\n", "horiz_conv.conv.weight.data.fill_(1)\n", "horiz_conv.conv.bias.data.fill_(0)\n", "horiz_img = horiz_conv(inp_img)\n", "show_center_recep_field(inp_img, horiz_img)"]}, {"cell_type": "markdown", "id": "42097881", "metadata": {"papermill": {"duration": 0.016191, "end_time": "2023-10-11T16:32:59.700384", "exception": false, "start_time": "2023-10-11T16:32:59.684193", "status": "completed"}, "tags": []}, "source": ["The receptive field is shown in yellow, the center pixel in red, and all other pixels outside of the receptive field are dark blue.\n", "As expected, the receptive field of a single horizontal convolution with the center pixel masked and a $3\\times3$ kernel is only the pixel on the left.\n", "If we use a larger kernel size, more pixels would be taken into account on the left.\n", "\n", "Next, let's take a look at the vertical convolution:"]}, {"cell_type": "code", "execution_count": 10, "id": "5bdf4270", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:32:59.735907Z", "iopub.status.busy": "2023-10-11T16:32:59.735254Z", "iopub.status.idle": "2023-10-11T16:33:00.098283Z", "shell.execute_reply": "2023-10-11T16:33:00.097301Z"}, "papermill": {"duration": 0.386416, "end_time": "2023-10-11T16:33:00.103087", "exception": false, "start_time": "2023-10-11T16:32:59.716671", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMzcxLjUyIDE5MS44NTIyNzI3MjczIF0gL0NvbnRlbnRzIDkgMCBSIC9Bbm5vdHMgMTAgMCBSID4+CmVuZG9iago5IDAgb2JqCjw8IC9MZW5ndGggMTIgMCBSIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nJ1QTU8CMRC9z694RzjY7UztdvcookRv6CYcjAezlGUJoAsE4r93gKwfXEQ7ecm8Tqfz5iX9uK3L+DDo4fqRki9WrokxU1SwmCl2YAwUFVllC3KBjRdN523KOZvMiwSnd/YnnRJNqEEwcgCnYpyEY7hTuooYYYnkSqetdeRMsdNJgxOBzaEx7EXYNg0a5QLJHaP/iiEN0Wix+v7LnlOjq1lcaB/YG7Y+E5elLgUHr8t8aikX1CsoudVngmJyWL4Y0xM6oy58Znzugkcn1tV0E8dYdSHWZK49Winj26beRkzqOB938Yzinm4KOkoTyybP23H/9Yb2W/3ujVhp3ZGz3RHOjbWW2ak7l+fb06uXL6v3vxlCH5GVksMKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyODIKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0xlbmd0aCAyNjQgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9MZW5ndGggOTIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0xlbmd0aCAzMDcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvTGVuZ3RoIDIzMiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9MZW5ndGggMjMxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDI0OSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0xlbmd0aCAxMzYgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvTGVuZ3RoIDM0MSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvTGVuZ3RoIDE2NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvTGVuZ3RoIDcyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvTGVuZ3RoIDQ3IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9MZW5ndGggMTYzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0xlbmd0aCAyMzkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0xlbmd0aCAxNjAgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvTGVuZ3RoIDE4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvTGVuZ3RoIDEzMyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9MZW5ndGggNzUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9MZW5ndGggMTQxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvVHlwZSAvRm9udCAvQmFzZUZvbnQgL0JNUVFEVitEZWphVnVTYW5zIC9GaXJzdENoYXIgMCAvTGFzdENoYXIgMjU1Ci9Gb250RGVzY3JpcHRvciAxNiAwIFIgL1N1YnR5cGUgL1R5cGUzIC9OYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdCi9DaGFyUHJvY3MgMTggMCBSCi9FbmNvZGluZyA8PCAvVHlwZSAvRW5jb2RpbmcKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCj4+Ci9XaWR0aHMgMTUgMCBSID4+CmVuZG9iagoxNiAwIG9iago8PCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucyAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvQXNjZW50IDkyOSAvRGVzY2VudCAtMjM2IC9DYXBIZWlnaHQgMAovWEhlaWdodCAwIC9JdGFsaWNBbmdsZSAwIC9TdGVtViAwIC9NYXhXaWR0aCAxMzQyID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiAyICj95zf/AAAAIk0pIF0gL0JpdHNQZXJDb21wb25lbnQgMgovRmlsdGVyIC9GbGF0ZURlY29kZQovRGVjb2RlUGFybXMgPDwgL1ByZWRpY3RvciAxMCAvQ29sb3JzIDEgL0NvbHVtbnMgMjI2IC9CaXRzUGVyQ29tcG9uZW50IDIgPj4KL0xlbmd0aCAzNyAwIFIgPj4Kc3RyZWFtCnic7csxCsAgEADB+6R5n615ZcTOQoSghTjbLhPvz3KQJEmSJEmSJEmSJEmS5DqZo48kSfIc2Sqp9gw3SZLkdjmJJEmSJEmSJEmSJEmSJEmSJK+THzKLcE0KZW5kc3RyZWFtCmVuZG9iagozNyAwIG9iago4OAplbmRvYmoKMTQgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL1N1YnR5cGUgL0ltYWdlIC9XaWR0aCAyMjYgL0hlaWdodCAyMjYKL0NvbG9yU3BhY2UgWyAvSW5kZXhlZCAvRGV2aWNlUkdCIDIgKP3nN/8AAAAiTSkgXSAvQml0c1BlckNvbXBvbmVudCAyCi9GaWx0ZXIgL0ZsYXRlRGVjb2RlCi9EZWNvZGVQYXJtcyA8PCAvUHJlZGljdG9yIDEwIC9Db2xvcnMgMSAvQ29sdW1ucyAyMjYgL0JpdHNQZXJDb21wb25lbnQgMiA+PgovTGVuZ3RoIDM4IDAgUiA+PgpzdHJlYW0KeJztyzEKwCAQAMH7pHmfrXllxM5ChKCFONsuE+/PcpAkSZIkSZIkSZIkSZLkOpmjjyRJ8hzZKqn2DDdJkuR2OYkkSZIkSZIkSZIkSZIkSZIkr5MfMotwTQplbmRzdHJlYW0KZW5kb2JqCjM4IDAgb2JqCjg4CmVuZG9iagoyIDAgb2JqCjw8IC9UeXBlIC9QYWdlcyAvS2lkcyBbIDExIDAgUiBdIC9Db3VudCAxID4+CmVuZG9iagozOSAwIG9iago8PCAvQ3JlYXRvciAoTWF0cGxvdGxpYiB2My44LjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcpCi9Qcm9kdWNlciAoTWF0cGxvdGxpYiBwZGYgYmFja2VuZCB2My44LjApIC9DcmVhdGlvbkRhdGUgKEQ6MjAyMzEwMTExNjMzMDBaKQo+PgplbmRvYmoKeHJlZgowIDQwCjAwMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAwMDAxNiAwMDAwMCBuIAowMDAwMDA3OTMxIDAwMDAwIG4gCjAwMDAwMDY5ODggMDAwMDAgbiAKMDAwMDAwNzAyMCAwMDAwMCBuIAowMDAwMDA3MDgwIDAwMDAwIG4gCjAwMDAwMDcxMDEgMDAwMDAgbiAKMDAwMDAwNzEyMiAwMDAwMCBuIAowMDAwMDAwMDY1IDAwMDAwIG4gCjAwMDAwMDAzNDQgMDAwMDAgbiAKMDAwMDAwMDcyMSAwMDAwMCBuIAowMDAwMDAwMjA4IDAwMDAwIG4gCjAwMDAwMDA3MDEgMDAwMDAgbiAKMDAwMDAwNzE2NSAwMDAwMCBuIAowMDAwMDA3NTQ4IDAwMDAwIG4gCjAwMDAwMDU3MjkgMDAwMDAgbiAKMDAwMDAwNTUyMiAwMDAwMCBuIAowMDAwMDA1MTE0IDAwMDAwIG4gCjAwMDAwMDY3ODIgMDAwMDAgbiAKMDAwMDAwMDc0MSAwMDAwMCBuIAowMDAwMDAxMDc4IDAwMDAwIG4gCjAwMDAwMDEyNDIgMDAwMDAgbiAKMDAwMDAwMTYyMiAwMDAwMCBuIAowMDAwMDAxOTI3IDAwMDAwIG4gCjAwMDAwMDIyMzEgMDAwMDAgbiAKMDAwMDAwMjU1MyAwMDAwMCBuIAowMDAwMDAyNzYyIDAwMDAwIG4gCjAwMDAwMDMxNzYgMDAwMDAgbiAKMDAwMDAwMzQxMyAwMDAwMCBuIAowMDAwMDAzNTU3IDAwMDAwIG4gCjAwMDAwMDM2NzYgMDAwMDAgbiAKMDAwMDAwMzkxMiAwMDAwMCBuIAowMDAwMDA0MjI0IDAwMDAwIG4gCjAwMDAwMDQ0NTcgMDAwMDAgbiAKMDAwMDAwNDU0NyAwMDAwMCBuIAowMDAwMDA0NzUzIDAwMDAwIG4gCjAwMDAwMDQ5MDAgMDAwMDAgbiAKMDAwMDAwNzUyOSAwMDAwMCBuIAowMDAwMDA3OTEyIDAwMDAwIG4gCjAwMDAwMDc5OTEgMDAwMDAgbiAKdHJhaWxlcgo8PCAvU2l6ZSA0MCAvUm9vdCAxIDAgUiAvSW5mbyAzOSAwIFIgPj4Kc3RhcnR4cmVmCjgxNDIKJSVFT0YK", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T16:32:59.935650\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["vert_conv = VerticalStackConvolution(c_in=1, c_out=1, kernel_size=3, mask_center=True)\n", "vert_conv.conv.weight.data.fill_(1)\n", "vert_conv.conv.bias.data.fill_(0)\n", "vert_img = vert_conv(inp_img)\n", "show_center_recep_field(inp_img, vert_img)"]}, {"cell_type": "markdown", "id": "777aa66b", "metadata": {"papermill": {"duration": 0.017694, "end_time": "2023-10-11T16:33:00.148304", "exception": false, "start_time": "2023-10-11T16:33:00.130610", "status": "completed"}, "tags": []}, "source": ["The vertical convolution takes all pixels above into account. Combining\n", "these two, we get the L-shaped receptive field of the original masked\n", "convolution:"]}, {"cell_type": "code", "execution_count": 11, "id": "9c0e0e05", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:33:00.187044Z", "iopub.status.busy": "2023-10-11T16:33:00.186447Z", "iopub.status.idle": "2023-10-11T16:33:00.518898Z", "shell.execute_reply": "2023-10-11T16:33:00.518237Z"}, "papermill": {"duration": 0.358833, "end_time": "2023-10-11T16:33:00.524310", "exception": false, "start_time": "2023-10-11T16:33:00.165477", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMzcxLjUyIDE5MS44NTIyNzI3MjczIF0gL0NvbnRlbnRzIDkgMCBSIC9Bbm5vdHMgMTAgMCBSID4+CmVuZG9iago5IDAgb2JqCjw8IC9MZW5ndGggMTIgMCBSIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nJ1QTU8CMRC9z694RzjY7UztdvcookRv6CYcjAezlGUJoAsE4r93gKwfXEQ7ecm8Tqfz5iX9uK3L+DDo4fqRki9WrokxU1SwmCl2YAwUFVllC3KBjRdN523KOZvMiwSnd/YnnRJNqEEwcgCnYpyEY7hTuooYYYnkSqetdeRMsdNJgxOBzaEx7EXYNg0a5QLJHaP/iiEN0Wix+v7LnlOjq1lcaB/YG7Y+E5elLgUHr8t8aikX1CsoudVngmJyWL4Y0xM6oy58Znzugkcn1tV0E8dYdSHWZK49Winj26beRkzqOB938Yzinm4KOkoTyybP23H/9Yb2W/3ujVhp3ZGz3RHOjbWW2ak7l+fb06uXL6v3vxlCH5GVksMKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyODIKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0xlbmd0aCAyNjQgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9MZW5ndGggOTIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0xlbmd0aCAzMDcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvTGVuZ3RoIDIzMiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9MZW5ndGggMjMxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDI0OSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0xlbmd0aCAxMzYgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvTGVuZ3RoIDM0MSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvTGVuZ3RoIDE2NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvTGVuZ3RoIDcyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvTGVuZ3RoIDQ3IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9MZW5ndGggMTYzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0xlbmd0aCAyMzkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0xlbmd0aCAxNjAgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvTGVuZ3RoIDE4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvTGVuZ3RoIDEzMyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9MZW5ndGggNzUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9MZW5ndGggMTQxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvVHlwZSAvRm9udCAvQmFzZUZvbnQgL0JNUVFEVitEZWphVnVTYW5zIC9GaXJzdENoYXIgMCAvTGFzdENoYXIgMjU1Ci9Gb250RGVzY3JpcHRvciAxNiAwIFIgL1N1YnR5cGUgL1R5cGUzIC9OYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdCi9DaGFyUHJvY3MgMTggMCBSCi9FbmNvZGluZyA8PCAvVHlwZSAvRW5jb2RpbmcKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCj4+Ci9XaWR0aHMgMTUgMCBSID4+CmVuZG9iagoxNiAwIG9iago8PCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucyAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvQXNjZW50IDkyOSAvRGVzY2VudCAtMjM2IC9DYXBIZWlnaHQgMAovWEhlaWdodCAwIC9JdGFsaWNBbmdsZSAwIC9TdGVtViAwIC9NYXhXaWR0aCAxMzQyID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiAyICj95zf/AAAAIk0pIF0gL0JpdHNQZXJDb21wb25lbnQgMgovRmlsdGVyIC9GbGF0ZURlY29kZQovRGVjb2RlUGFybXMgPDwgL1ByZWRpY3RvciAxMCAvQ29sb3JzIDEgL0NvbHVtbnMgMjI2IC9CaXRzUGVyQ29tcG9uZW50IDIgPj4KL0xlbmd0aCAzNyAwIFIgPj4Kc3RyZWFtCnic7csxDgARFEDB75CcT+2UtBKFbFYh5rUvE+1jNUiSJEmSJEmSJEmSJEnyP1ljjiRJ8jKZ8qisNkmS5HG5E0mSJEmSJEmSJEmSJEmSJEk+JzthQS1FCmVuZHN0cmVhbQplbmRvYmoKMzcgMCBvYmoKODYKZW5kb2JqCjE0IDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiAyICj95zf/AAAAIk0pIF0gL0JpdHNQZXJDb21wb25lbnQgMgovRmlsdGVyIC9GbGF0ZURlY29kZQovRGVjb2RlUGFybXMgPDwgL1ByZWRpY3RvciAxMCAvQ29sb3JzIDEgL0NvbHVtbnMgMjI2IC9CaXRzUGVyQ29tcG9uZW50IDIgPj4KL0xlbmd0aCAzOCAwIFIgPj4Kc3RyZWFtCnic7csxDgARFEDB75CcT+2UtBKFbFYh5rUvE+1jNUiSJEmSJEmSJEmSJEnyP1ljjiRJ8jKZ8qisNkmS5HG5E0mSJEmSJEmSJEmSJEmSJEk+JzthQS1FCmVuZHN0cmVhbQplbmRvYmoKMzggMCBvYmoKODYKZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFsgMTEgMCBSIF0gL0NvdW50IDEgPj4KZW5kb2JqCjM5IDAgb2JqCjw8IC9DcmVhdG9yIChNYXRwbG90bGliIHYzLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjguMCkgL0NyZWF0aW9uRGF0ZSAoRDoyMDIzMTAxMTE2MzMwMFopCj4+CmVuZG9iagp4cmVmCjAgNDAKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDc5MjcgMDAwMDAgbiAKMDAwMDAwNjk4OCAwMDAwMCBuIAowMDAwMDA3MDIwIDAwMDAwIG4gCjAwMDAwMDcwODAgMDAwMDAgbiAKMDAwMDAwNzEwMSAwMDAwMCBuIAowMDAwMDA3MTIyIDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM0NCAwMDAwMCBuIAowMDAwMDAwNzIxIDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDcwMSAwMDAwMCBuIAowMDAwMDA3MTY1IDAwMDAwIG4gCjAwMDAwMDc1NDYgMDAwMDAgbiAKMDAwMDAwNTcyOSAwMDAwMCBuIAowMDAwMDA1NTIyIDAwMDAwIG4gCjAwMDAwMDUxMTQgMDAwMDAgbiAKMDAwMDAwNjc4MiAwMDAwMCBuIAowMDAwMDAwNzQxIDAwMDAwIG4gCjAwMDAwMDEwNzggMDAwMDAgbiAKMDAwMDAwMTI0MiAwMDAwMCBuIAowMDAwMDAxNjIyIDAwMDAwIG4gCjAwMDAwMDE5MjcgMDAwMDAgbiAKMDAwMDAwMjIzMSAwMDAwMCBuIAowMDAwMDAyNTUzIDAwMDAwIG4gCjAwMDAwMDI3NjIgMDAwMDAgbiAKMDAwMDAwMzE3NiAwMDAwMCBuIAowMDAwMDAzNDEzIDAwMDAwIG4gCjAwMDAwMDM1NTcgMDAwMDAgbiAKMDAwMDAwMzY3NiAwMDAwMCBuIAowMDAwMDAzOTEyIDAwMDAwIG4gCjAwMDAwMDQyMjQgMDAwMDAgbiAKMDAwMDAwNDQ1NyAwMDAwMCBuIAowMDAwMDA0NTQ3IDAwMDAwIG4gCjAwMDAwMDQ3NTMgMDAwMDAgbiAKMDAwMDAwNDkwMCAwMDAwMCBuIAowMDAwMDA3NTI3IDAwMDAwIG4gCjAwMDAwMDc5MDggMDAwMDAgbiAKMDAwMDAwNzk4NyAwMDAwMCBuIAp0cmFpbGVyCjw8IC9TaXplIDQwIC9Sb290IDEgMCBSIC9JbmZvIDM5IDAgUiA+PgpzdGFydHhyZWYKODEzOAolJUVPRgo=", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T16:33:00.357656\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["horiz_img = vert_img + horiz_img\n", "show_center_recep_field(inp_img, horiz_img)"]}, {"cell_type": "markdown", "id": "b1ffbf74", "metadata": {"papermill": {"duration": 0.016053, "end_time": "2023-10-11T16:33:00.555854", "exception": false, "start_time": "2023-10-11T16:33:00.539801", "status": "completed"}, "tags": []}, "source": ["If we stack multiple horizontal and vertical convolutions, we need to take two aspects into account:\n", "\n", "1.\n", "The center should not be masked anymore for the following convolutions as the features at the pixel's position are already independent of its actual value.\n", "If it is hard to imagine why we can do this, just change the value below to `mask_center=True` and see what happens.\n", "2.\n", "The vertical convolution is not allowed to work on features from the horizontal convolution.\n", "In the feature map of the horizontal convolutions, a pixel contains information about all of the \"true\" pixels on the left.\n", "If we apply a vertical convolution which also uses features from the right, we effectively expand our receptive field to the true input which we want to prevent.\n", "Thus, the feature maps can only be merged for the horizontal convolution.\n", "\n", "Using this, we can stack the convolutions in the following way. We have\n", "two feature streams: one for the vertical stack, and one for the\n", "horizontal stack. The horizontal convolutions can operate on the joint\n", "features of the previous horizontals and vertical convolutions, while\n", "the vertical stack only takes its own previous features as input. For a\n", "quick implementation, we can therefore sum the horizontal and vertical\n", "output features at each layer, and use those as final output features to\n", "calculate the loss on. An implementation of 4 consecutive layers is\n", "shown below. Note that we reuse the features from the other convolutions\n", "with `mask_center=True` from above."]}, {"cell_type": "code", "execution_count": 12, "id": "b8d27ef2", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:33:00.588553Z", "iopub.status.busy": "2023-10-11T16:33:00.587962Z", "iopub.status.idle": "2023-10-11T16:33:02.580244Z", "shell.execute_reply": "2023-10-11T16:33:02.579479Z"}, "papermill": {"duration": 2.014108, "end_time": "2023-10-11T16:33:02.586272", "exception": false, "start_time": "2023-10-11T16:33:00.572164", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Layer 2\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMzcxLjUyIDE5MS44NTIyNzI3MjczIF0gL0NvbnRlbnRzIDkgMCBSIC9Bbm5vdHMgMTAgMCBSID4+CmVuZG9iago5IDAgb2JqCjw8IC9MZW5ndGggMTIgMCBSIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nJ1QTU8CMRC9z694RzjY7UztdvcookRv6CYcjAezlGUJoAsE4r93gKwfXEQ7ecm8Tqfz5iX9uK3L+DDo4fqRki9WrokxU1SwmCl2YAwUFVllC3KBjRdN523KOZvMiwSnd/YnnRJNqEEwcgCnYpyEY7hTuooYYYnkSqetdeRMsdNJgxOBzaEx7EXYNg0a5QLJHaP/iiEN0Wix+v7LnlOjq1lcaB/YG7Y+E5elLgUHr8t8aikX1CsoudVngmJyWL4Y0xM6oy58Znzugkcn1tV0E8dYdSHWZK49Winj26beRkzqOB938Yzinm4KOkoTyybP23H/9Yb2W/3ujVhp3ZGz3RHOjbWW2ak7l+fb06uXL6v3vxlCH5GVksMKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyODIKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0xlbmd0aCAyNjQgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9MZW5ndGggOTIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0xlbmd0aCAzMDcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvTGVuZ3RoIDIzMiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9MZW5ndGggMjMxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDI0OSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0xlbmd0aCAxMzYgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvTGVuZ3RoIDM0MSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvTGVuZ3RoIDE2NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvTGVuZ3RoIDcyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvTGVuZ3RoIDQ3IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9MZW5ndGggMTYzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0xlbmd0aCAyMzkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0xlbmd0aCAxNjAgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvTGVuZ3RoIDE4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvTGVuZ3RoIDEzMyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9MZW5ndGggNzUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9MZW5ndGggMTQxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvVHlwZSAvRm9udCAvQmFzZUZvbnQgL0JNUVFEVitEZWphVnVTYW5zIC9GaXJzdENoYXIgMCAvTGFzdENoYXIgMjU1Ci9Gb250RGVzY3JpcHRvciAxNiAwIFIgL1N1YnR5cGUgL1R5cGUzIC9OYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdCi9DaGFyUHJvY3MgMTggMCBSCi9FbmNvZGluZyA8PCAvVHlwZSAvRW5jb2RpbmcKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCj4+Ci9XaWR0aHMgMTUgMCBSID4+CmVuZG9iagoxNiAwIG9iago8PCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucyAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvQXNjZW50IDkyOSAvRGVzY2VudCAtMjM2IC9DYXBIZWlnaHQgMAovWEhlaWdodCAwIC9JdGFsaWNBbmdsZSAwIC9TdGVtViAwIC9NYXhXaWR0aCAxMzQyID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiA2ICj95zfIt2VmaXA1RWz/AACUjncAIk0pIF0KL0JpdHNQZXJDb21wb25lbnQgNCAvRmlsdGVyIC9GbGF0ZURlY29kZQovRGVjb2RlUGFybXMgPDwgL1ByZWRpY3RvciAxMCAvQ29sb3JzIDEgL0NvbHVtbnMgMjI2IC9CaXRzUGVyQ29tcG9uZW50IDQgPj4KL0xlbmd0aCAzNyAwIFIgPj4Kc3RyZWFtCnic7c2xCYAwEEDRNPa6guAEcQRxACEruP8IVkkKA2lEFN4vL3d54Xy7QCQSiUQikUgkEolEIpFIJBKJRCKRSCQSG8XSXFpS7qjDukgkEolEIpH4C7F+OZXGkBuKnYhEIpFIJBKJnffYaN1ye+eaSCQSiUQi8bvi8xGJRCKRSCQSiUQikUgkEolEIpFIJBKJRCKRSCQSiUQikUgkEolEIvHeBejld1UKZW5kc3RyZWFtCmVuZG9iagozNyAwIG9iagoxNjYKZW5kb2JqCjE0IDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiAyICj95zf/AAAAIk0pIF0gL0JpdHNQZXJDb21wb25lbnQgMgovRmlsdGVyIC9GbGF0ZURlY29kZQovRGVjb2RlUGFybXMgPDwgL1ByZWRpY3RvciAxMCAvQ29sb3JzIDEgL0NvbHVtbnMgMjI2IC9CaXRzUGVyQ29tcG9uZW50IDIgPj4KL0xlbmd0aCAzOCAwIFIgPj4Kc3RyZWFtCnic7cuxDQAQEADANyTzqU3JAp+IRMNdfzEO9TBN0zRN0zRN0zQfmT0ypmmapmma5u4sdWkjY5qmeWluM03TNE3TNE3TNE3TNE3TNM3v5gSmHBJ7CmVuZHN0cmVhbQplbmRvYmoKMzggMCBvYmoKODMKZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFsgMTEgMCBSIF0gL0NvdW50IDEgPj4KZW5kb2JqCjM5IDAgb2JqCjw8IC9DcmVhdG9yIChNYXRwbG90bGliIHYzLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjguMCkgL0NyZWF0aW9uRGF0ZSAoRDoyMDIzMTAxMTE2MzMwMFopCj4+CmVuZG9iagp4cmVmCjAgNDAKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDgwMTcgMDAwMDAgbiAKMDAwMDAwNjk4OCAwMDAwMCBuIAowMDAwMDA3MDIwIDAwMDAwIG4gCjAwMDAwMDcwODAgMDAwMDAgbiAKMDAwMDAwNzEwMSAwMDAwMCBuIAowMDAwMDA3MTIyIDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM0NCAwMDAwMCBuIAowMDAwMDAwNzIxIDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDcwMSAwMDAwMCBuIAowMDAwMDA3MTY1IDAwMDAwIG4gCjAwMDAwMDc2MzkgMDAwMDAgbiAKMDAwMDAwNTcyOSAwMDAwMCBuIAowMDAwMDA1NTIyIDAwMDAwIG4gCjAwMDAwMDUxMTQgMDAwMDAgbiAKMDAwMDAwNjc4MiAwMDAwMCBuIAowMDAwMDAwNzQxIDAwMDAwIG4gCjAwMDAwMDEwNzggMDAwMDAgbiAKMDAwMDAwMTI0MiAwMDAwMCBuIAowMDAwMDAxNjIyIDAwMDAwIG4gCjAwMDAwMDE5MjcgMDAwMDAgbiAKMDAwMDAwMjIzMSAwMDAwMCBuIAowMDAwMDAyNTUzIDAwMDAwIG4gCjAwMDAwMDI3NjIgMDAwMDAgbiAKMDAwMDAwMzE3NiAwMDAwMCBuIAowMDAwMDAzNDEzIDAwMDAwIG4gCjAwMDAwMDM1NTcgMDAwMDAgbiAKMDAwMDAwMzY3NiAwMDAwMCBuIAowMDAwMDAzOTEyIDAwMDAwIG4gCjAwMDAwMDQyMjQgMDAwMDAgbiAKMDAwMDAwNDQ1NyAwMDAwMCBuIAowMDAwMDA0NTQ3IDAwMDAwIG4gCjAwMDAwMDQ3NTMgMDAwMDAgbiAKMDAwMDAwNDkwMCAwMDAwMCBuIAowMDAwMDA3NjE5IDAwMDAwIG4gCjAwMDAwMDc5OTggMDAwMDAgbiAKMDAwMDAwODA3NyAwMDAwMCBuIAp0cmFpbGVyCjw8IC9TaXplIDQwIC9Sb290IDEgMCBSIC9JbmZvIDM5IDAgUiA+PgpzdGFydHhyZWYKODIyOAolJUVPRgo=", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T16:33:00.852363\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}, {"name": "stdout", "output_type": "stream", "text": ["Layer 3\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMzcxLjUyIDE5MS44NTIyNzI3MjczIF0gL0NvbnRlbnRzIDkgMCBSIC9Bbm5vdHMgMTAgMCBSID4+CmVuZG9iago5IDAgb2JqCjw8IC9MZW5ndGggMTIgMCBSIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nJ1QTU8CMRC9z694RzjY7UztdvcookRv6CYcjAezlGUJoAsE4r93gKwfXEQ7ecm8Tqfz5iX9uK3L+DDo4fqRki9WrokxU1SwmCl2YAwUFVllC3KBjRdN523KOZvMiwSnd/YnnRJNqEEwcgCnYpyEY7hTuooYYYnkSqetdeRMsdNJgxOBzaEx7EXYNg0a5QLJHaP/iiEN0Wix+v7LnlOjq1lcaB/YG7Y+E5elLgUHr8t8aikX1CsoudVngmJyWL4Y0xM6oy58Znzugkcn1tV0E8dYdSHWZK49Winj26beRkzqOB938Yzinm4KOkoTyybP23H/9Yb2W/3ujVhp3ZGz3RHOjbWW2ak7l+fb06uXL6v3vxlCH5GVksMKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyODIKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0xlbmd0aCAyNjQgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9MZW5ndGggOTIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0xlbmd0aCAzMDcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvTGVuZ3RoIDIzMiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9MZW5ndGggMjMxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDI0OSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0xlbmd0aCAxMzYgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvTGVuZ3RoIDM0MSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvTGVuZ3RoIDE2NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvTGVuZ3RoIDcyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvTGVuZ3RoIDQ3IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9MZW5ndGggMTYzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0xlbmd0aCAyMzkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0xlbmd0aCAxNjAgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvTGVuZ3RoIDE4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvTGVuZ3RoIDEzMyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9MZW5ndGggNzUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9MZW5ndGggMTQxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvVHlwZSAvRm9udCAvQmFzZUZvbnQgL0JNUVFEVitEZWphVnVTYW5zIC9GaXJzdENoYXIgMCAvTGFzdENoYXIgMjU1Ci9Gb250RGVzY3JpcHRvciAxNiAwIFIgL1N1YnR5cGUgL1R5cGUzIC9OYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdCi9DaGFyUHJvY3MgMTggMCBSCi9FbmNvZGluZyA8PCAvVHlwZSAvRW5jb2RpbmcKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCj4+Ci9XaWR0aHMgMTUgMCBSID4+CmVuZG9iagoxNiAwIG9iago8PCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucyAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvQXNjZW50IDkyOSAvRGVzY2VudCAtMjM2IC9DYXBIZWlnaHQgMAovWEhlaWdodCAwIC9JdGFsaWNBbmdsZSAwIC9TdGVtViAwIC9NYXhXaWR0aCAxMzQyID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiAxMiAo/ec3481SxrVnX2NuUllsJz1tCzNwACtk/wAAgoB4dnZ2N0ZsACJNKSBdCi9CaXRzUGVyQ29tcG9uZW50IDQgL0ZpbHRlciAvRmxhdGVEZWNvZGUKL0RlY29kZVBhcm1zIDw8IC9QcmVkaWN0b3IgMTAgL0NvbG9ycyAxIC9Db2x1bW5zIDIyNiAvQml0c1BlckNvbXBvbmVudCA0ID4+Ci9MZW5ndGggMzcgMCBSID4+CnN0cmVhbQp4nO3WrQ0CQRCA0Q0VQAvn0EcH0AItoPAgrgAEDdACLaDJGYpC7YzgEgw/OXifnOzkrZxy+3SFSCQSiUQikUgkEolEIpE4frHfRetoFS3b2iKH+TCXOyKRSCQSicS/F6+5fo5m0bTUJk0Uv2g30ZZIJBKJRCJxDGIz0PwUXaKXXVZEIpFIJBKJPyD2uZSHWc72h9pxaPtJRCKRSCQSiV8U3xqRSCQSiUQikUgkEolEIpFIJBKJRCKRSCQSiUQikUgkEolEIpFIJBIfuwNrVoLyCmVuZHN0cmVhbQplbmRvYmoKMzcgMCBvYmoKMjA4CmVuZG9iagoxNCAwIG9iago8PCAvVHlwZSAvWE9iamVjdCAvU3VidHlwZSAvSW1hZ2UgL1dpZHRoIDIyNiAvSGVpZ2h0IDIyNgovQ29sb3JTcGFjZSBbIC9JbmRleGVkIC9EZXZpY2VSR0IgMiAo/ec3/wAAACJNKSBdIC9CaXRzUGVyQ29tcG9uZW50IDIKL0ZpbHRlciAvRmxhdGVEZWNvZGUKL0RlY29kZVBhcm1zIDw8IC9QcmVkaWN0b3IgMTAgL0NvbG9ycyAxIC9Db2x1bW5zIDIyNiAvQml0c1BlckNvbXBvbmVudCAyID4+Ci9MZW5ndGggMzggMCBSID4+CnN0cmVhbQp4nO3LsRGAIBAAsGcd98F5HMX6p4TGkvPkrCDpEznpDtM0TdM0TfNxxbvDNE3TNE3TNE3TXHmW2p05Ypqm+f/8xjRN0zRN0zRN0zRN0zRN0zS3mw0E/K1rCmVuZHN0cmVhbQplbmRvYmoKMzggMCBvYmoKODgKZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFsgMTEgMCBSIF0gL0NvdW50IDEgPj4KZW5kb2JqCjM5IDAgb2JqCjw8IC9DcmVhdG9yIChNYXRwbG90bGliIHYzLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjguMCkgL0NyZWF0aW9uRGF0ZSAoRDoyMDIzMTAxMTE2MzMwMVopCj4+CmVuZG9iagp4cmVmCjAgNDAKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDgwODMgMDAwMDAgbiAKMDAwMDAwNjk4OCAwMDAwMCBuIAowMDAwMDA3MDIwIDAwMDAwIG4gCjAwMDAwMDcwODAgMDAwMDAgbiAKMDAwMDAwNzEwMSAwMDAwMCBuIAowMDAwMDA3MTIyIDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM0NCAwMDAwMCBuIAowMDAwMDAwNzIxIDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDcwMSAwMDAwMCBuIAowMDAwMDA3MTY1IDAwMDAwIG4gCjAwMDAwMDc3MDAgMDAwMDAgbiAKMDAwMDAwNTcyOSAwMDAwMCBuIAowMDAwMDA1NTIyIDAwMDAwIG4gCjAwMDAwMDUxMTQgMDAwMDAgbiAKMDAwMDAwNjc4MiAwMDAwMCBuIAowMDAwMDAwNzQxIDAwMDAwIG4gCjAwMDAwMDEwNzggMDAwMDAgbiAKMDAwMDAwMTI0MiAwMDAwMCBuIAowMDAwMDAxNjIyIDAwMDAwIG4gCjAwMDAwMDE5MjcgMDAwMDAgbiAKMDAwMDAwMjIzMSAwMDAwMCBuIAowMDAwMDAyNTUzIDAwMDAwIG4gCjAwMDAwMDI3NjIgMDAwMDAgbiAKMDAwMDAwMzE3NiAwMDAwMCBuIAowMDAwMDAzNDEzIDAwMDAwIG4gCjAwMDAwMDM1NTcgMDAwMDAgbiAKMDAwMDAwMzY3NiAwMDAwMCBuIAowMDAwMDAzOTEyIDAwMDAwIG4gCjAwMDAwMDQyMjQgMDAwMDAgbiAKMDAwMDAwNDQ1NyAwMDAwMCBuIAowMDAwMDA0NTQ3IDAwMDAwIG4gCjAwMDAwMDQ3NTMgMDAwMDAgbiAKMDAwMDAwNDkwMCAwMDAwMCBuIAowMDAwMDA3NjgwIDAwMDAwIG4gCjAwMDAwMDgwNjQgMDAwMDAgbiAKMDAwMDAwODE0MyAwMDAwMCBuIAp0cmFpbGVyCjw8IC9TaXplIDQwIC9Sb290IDEgMCBSIC9JbmZvIDM5IDAgUiA+PgpzdGFydHhyZWYKODI5NAolJUVPRgo=", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T16:33:01.342255\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}, {"name": "stdout", "output_type": "stream", "text": ["Layer 4\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMzcxLjUyIDE5MS44NTIyNzI3MjczIF0gL0NvbnRlbnRzIDkgMCBSIC9Bbm5vdHMgMTAgMCBSID4+CmVuZG9iago5IDAgb2JqCjw8IC9MZW5ndGggMTIgMCBSIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nJ1QTU8CMRC9z694RzjY7UztdvcookRv6CYcjAezlGUJoAsE4r93gKwfXEQ7ecm8Tqfz5iX9uK3L+DDo4fqRki9WrokxU1SwmCl2YAwUFVllC3KBjRdN523KOZvMiwSnd/YnnRJNqEEwcgCnYpyEY7hTuooYYYnkSqetdeRMsdNJgxOBzaEx7EXYNg0a5QLJHaP/iiEN0Wix+v7LnlOjq1lcaB/YG7Y+E5elLgUHr8t8aikX1CsoudVngmJyWL4Y0xM6oy58Znzugkcn1tV0E8dYdSHWZK49Winj26beRkzqOB938Yzinm4KOkoTyybP23H/9Yb2W/3ujVhp3ZGz3RHOjbWW2ak7l+fb06uXL6v3vxlCH5GVksMKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyODIKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0xlbmd0aCAyNjQgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9MZW5ndGggOTIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0xlbmd0aCAzMDcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvTGVuZ3RoIDIzMiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9MZW5ndGggMjMxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDI0OSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0xlbmd0aCAxMzYgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvTGVuZ3RoIDM0MSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvTGVuZ3RoIDE2NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvTGVuZ3RoIDcyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvTGVuZ3RoIDQ3IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9MZW5ndGggMTYzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0xlbmd0aCAyMzkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0xlbmd0aCAxNjAgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvTGVuZ3RoIDE4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvTGVuZ3RoIDEzMyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9MZW5ndGggNzUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9MZW5ndGggMTQxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvVHlwZSAvRm9udCAvQmFzZUZvbnQgL0JNUVFEVitEZWphVnVTYW5zIC9GaXJzdENoYXIgMCAvTGFzdENoYXIgMjU1Ci9Gb250RGVzY3JpcHRvciAxNiAwIFIgL1N1YnR5cGUgL1R5cGUzIC9OYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdCi9DaGFyUHJvY3MgMTggMCBSCi9FbmNvZGluZyA8PCAvVHlwZSAvRW5jb2RpbmcKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCj4+Ci9XaWR0aHMgMTUgMCBSID4+CmVuZG9iagoxNiAwIG9iago8PCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucyAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvQXNjZW50IDkyOSAvRGVzY2VudCAtMjM2IC9DYXBIZWlnaHQgMAovWEhlaWdodCAwIC9JdGFsaWNBbmdsZSAwIC9TdGVtViAwIC9NYXhXaWR0aCAxMzQyID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiAyNAoo/ec3m5N2mJF2bG1yLkFsKj9tJT1tITtuAC1pETVvACNQ/wAAAFwoW+vUSwAqYL+waqaccwAsZn18eHh4dmBkblZcXG1GUGs3RmwAIk0pCl0KL0JpdHNQZXJDb21wb25lbnQgOCAvRmlsdGVyIC9GbGF0ZURlY29kZQovRGVjb2RlUGFybXMgPDwgL1ByZWRpY3RvciAxMCAvQ29sb3JzIDEgL0NvbHVtbnMgMjI2IC9CaXRzUGVyQ29tcG9uZW50IDggPj4KL0xlbmd0aCAzNyAwIFIgPj4Kc3RyZWFtCnic7dpJUiQxFAXBooFmbObp/iflAm/BogGzKI91SvqupSwPH/kOvz3A94dYCLEQYiHEQoiFEAshFkIshFjoaInXo7vV1ehs9T6aH64d59FrRkRERERERERERERERERERMRjIK4t50Cno4fV/ehk9TK6WN2OEBERERERERERERERERERERETxMvR2+rP6N/qMFrs+3Vpf1eIiIiIiIiIiIiIiIiIiIiIx0r88vPUekx6Wj2PXlfno8fVT/09hYiIiIiIiIiIiIiIiIiIiIj448S1fL14zdbi65vRPPr/h4iIiIiIiIiIiIiIiIiIiBgjpkIshFgIsRBiIcRCiIUQCyEWQiyEWAixEGIhxEKIhRALIRZCLIRYCLEQYiHEQoiFEAshFkIshFgIsRBiIcRCiIUQCyEWQiyEWAixEGIhxEKIhRALIRZCLIRYCLEQYiHEQoiFEAshFkIshFgIsdAn62RUogplbmRzdHJlYW0KZW5kb2JqCjM3IDAgb2JqCjM1NAplbmRvYmoKMTQgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL1N1YnR5cGUgL0ltYWdlIC9XaWR0aCAyMjYgL0hlaWdodCAyMjYKL0NvbG9yU3BhY2UgWyAvSW5kZXhlZCAvRGV2aWNlUkdCIDIgKP3nN/8AAAAiTSkgXSAvQml0c1BlckNvbXBvbmVudCAyCi9GaWx0ZXIgL0ZsYXRlRGVjb2RlCi9EZWNvZGVQYXJtcyA8PCAvUHJlZGljdG9yIDEwIC9Db2xvcnMgMSAvQ29sdW1ucyAyMjYgL0JpdHNQZXJDb21wb25lbnQgMiA+PgovTGVuZ3RoIDM4IDAgUiA+PgpzdHJlYW0KeJzty7ENgDAMALD0Hf4p93AKc66kDzA0aiUk7N2RRXeYpml+Yl4x4zBN0zRN0zRN0zRN0zTNLbP14cw3pmmaS2eBaZqmaZqmaZqmaZqmaZqmaf5uPqnfbB8KZW5kc3RyZWFtCmVuZG9iagozOCAwIG9iago4OQplbmRvYmoKMiAwIG9iago8PCAvVHlwZSAvUGFnZXMgL0tpZHMgWyAxMSAwIFIgXSAvQ291bnQgMSA+PgplbmRvYmoKMzkgMCBvYmoKPDwgL0NyZWF0b3IgKE1hdHBsb3RsaWIgdjMuOC4wLCBodHRwczovL21hdHBsb3RsaWIub3JnKQovUHJvZHVjZXIgKE1hdHBsb3RsaWIgcGRmIGJhY2tlbmQgdjMuOC4wKSAvQ3JlYXRpb25EYXRlIChEOjIwMjMxMDExMTYzMzAxWikKPj4KZW5kb2JqCnhyZWYKMCA0MAowMDAwMDAwMDAwIDY1NTM1IGYgCjAwMDAwMDAwMTYgMDAwMDAgbiAKMDAwMDAwODI2OCAwMDAwMCBuIAowMDAwMDA2OTg4IDAwMDAwIG4gCjAwMDAwMDcwMjAgMDAwMDAgbiAKMDAwMDAwNzA4MCAwMDAwMCBuIAowMDAwMDA3MTAxIDAwMDAwIG4gCjAwMDAwMDcxMjIgMDAwMDAgbiAKMDAwMDAwMDA2NSAwMDAwMCBuIAowMDAwMDAwMzQ0IDAwMDAwIG4gCjAwMDAwMDA3MjEgMDAwMDAgbiAKMDAwMDAwMDIwOCAwMDAwMCBuIAowMDAwMDAwNzAxIDAwMDAwIG4gCjAwMDAwMDcxNjUgMDAwMDAgbiAKMDAwMDAwNzg4NCAwMDAwMCBuIAowMDAwMDA1NzI5IDAwMDAwIG4gCjAwMDAwMDU1MjIgMDAwMDAgbiAKMDAwMDAwNTExNCAwMDAwMCBuIAowMDAwMDA2NzgyIDAwMDAwIG4gCjAwMDAwMDA3NDEgMDAwMDAgbiAKMDAwMDAwMTA3OCAwMDAwMCBuIAowMDAwMDAxMjQyIDAwMDAwIG4gCjAwMDAwMDE2MjIgMDAwMDAgbiAKMDAwMDAwMTkyNyAwMDAwMCBuIAowMDAwMDAyMjMxIDAwMDAwIG4gCjAwMDAwMDI1NTMgMDAwMDAgbiAKMDAwMDAwMjc2MiAwMDAwMCBuIAowMDAwMDAzMTc2IDAwMDAwIG4gCjAwMDAwMDM0MTMgMDAwMDAgbiAKMDAwMDAwMzU1NyAwMDAwMCBuIAowMDAwMDAzNjc2IDAwMDAwIG4gCjAwMDAwMDM5MTIgMDAwMDAgbiAKMDAwMDAwNDIyNCAwMDAwMCBuIAowMDAwMDA0NDU3IDAwMDAwIG4gCjAwMDAwMDQ1NDcgMDAwMDAgbiAKMDAwMDAwNDc1MyAwMDAwMCBuIAowMDAwMDA0OTAwIDAwMDAwIG4gCjAwMDAwMDc4NjQgMDAwMDAgbiAKMDAwMDAwODI0OSAwMDAwMCBuIAowMDAwMDA4MzI4IDAwMDAwIG4gCnRyYWlsZXIKPDwgL1NpemUgNDAgL1Jvb3QgMSAwIFIgL0luZm8gMzkgMCBSID4+CnN0YXJ0eHJlZgo4NDc5CiUlRU9GCg==", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T16:33:01.860988\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}, {"name": "stdout", "output_type": "stream", "text": ["Layer 5\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMzcxLjUyIDE5MS44NTIyNzI3MjczIF0gL0NvbnRlbnRzIDkgMCBSIC9Bbm5vdHMgMTAgMCBSID4+CmVuZG9iago5IDAgb2JqCjw8IC9MZW5ndGggMTIgMCBSIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nJ1QTU8CMRC9z694RzjY7UztdvcookRv6CYcjAezlGUJoAsE4r93gKwfXEQ7ecm8Tqfz5iX9uK3L+DDo4fqRki9WrokxU1SwmCl2YAwUFVllC3KBjRdN523KOZvMiwSnd/YnnRJNqEEwcgCnYpyEY7hTuooYYYnkSqetdeRMsdNJgxOBzaEx7EXYNg0a5QLJHaP/iiEN0Wix+v7LnlOjq1lcaB/YG7Y+E5elLgUHr8t8aikX1CsoudVngmJyWL4Y0xM6oy58Znzugkcn1tV0E8dYdSHWZK49Winj26beRkzqOB938Yzinm4KOkoTyybP23H/9Yb2W/3ujVhp3ZGz3RHOjbWW2ak7l+fb06uXL6v3vxlCH5GVksMKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyODIKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0xlbmd0aCAyNjQgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9MZW5ndGggOTIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0xlbmd0aCAzMDcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvTGVuZ3RoIDIzMiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9MZW5ndGggMjMxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDI0OSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0xlbmd0aCAxMzYgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvTGVuZ3RoIDM0MSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvTGVuZ3RoIDE2NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvTGVuZ3RoIDcyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvTGVuZ3RoIDQ3IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9MZW5ndGggMTYzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0xlbmd0aCAyMzkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0xlbmd0aCAxNjAgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvTGVuZ3RoIDE4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvTGVuZ3RoIDEzMyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9MZW5ndGggNzUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9MZW5ndGggMTQxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvVHlwZSAvRm9udCAvQmFzZUZvbnQgL0JNUVFEVitEZWphVnVTYW5zIC9GaXJzdENoYXIgMCAvTGFzdENoYXIgMjU1Ci9Gb250RGVzY3JpcHRvciAxNiAwIFIgL1N1YnR5cGUgL1R5cGUzIC9OYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdCi9DaGFyUHJvY3MgMTggMCBSCi9FbmNvZGluZyA8PCAvVHlwZSAvRW5jb2RpbmcKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCj4+Ci9XaWR0aHMgMTUgMCBSID4+CmVuZG9iagoxNiAwIG9iago8PCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucyAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvQXNjZW50IDkyOSAvRGVzY2VudCAtMjM2IC9DYXBIZWlnaHQgMAovWEhlaWdodCAwIC9JdGFsaWNBbmdsZSAwIC9TdGVtViAwIC9NYXhXaWR0aCAxMzQyID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiAzNwoo/ec37NVK1MFe0r9gpZtzoJd1iIV4ACRSe3t3ZmlwWF1tV11tRE9rQU1rACZVLkFsKz9tCzNwACNQ/wAAAFwoW828YgAqYgArZJSOd4KAeAAsZnZ2dlRabFFYbExUbAAuazdGbCQ8bgAvbxY2bxQ2bwAiTSkKXQovQml0c1BlckNvbXBvbmVudCA4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlCi9EZWNvZGVQYXJtcyA8PCAvUHJlZGljdG9yIDEwIC9Db2xvcnMgMSAvQ29sdW1ucyAyMjYgL0JpdHNQZXJDb21wb25lbnQgOCA+PgovTGVuZ3RoIDM3IDAgUiA+PgpzdHJlYW0KeJzt27lOw0AYhVGzhn0JENYQ1vd/RVoXtwtG5up85Wg09nE5+j18p5ah29QmdJ36DMWN6cT46PSOETMgIiIiIiIiIiIiIiIiIiIiIs6XmHYun0JXqZPQUeow9Jxah95Tq1DEICIiIiIiIiIiIiIiIiIiIv5f4kXqK3SaOgjtpIbQXuoxdJ76CC1SiIiIiIiIiIiIiIiIiIiIiIgNxJfUQ+gutRvaTx2HLlP3oS1v4BARERERERERERERERERERHnRoyraXoqjjCdhdL80/o19JZKXzcNSq3STdSWf74hIiIiIiIiIiIiIiIiIiIiIv45Ma4mdjpykTbGE29CcePvh4iIOApxyhAREUchThkiIuIoxClDREQcNTtiVYgNITaE2BBiQ4gNITaE2BBiQ4gNITaE2BBiQ4gNITaE2BBiQ4gNITaE2BBiQ4gNITaE2BBiQ4gNITaE2BBiQ4gNITaE2BBiQ4gNITaE2BBiQ4gNITaE2BBiQ4gNITaE2BBiQ4gNITaE2BBiQ4gNITaE2NAPuU5lsQplbmRzdHJlYW0KZW5kb2JqCjM3IDAgb2JqCjQwMwplbmRvYmoKMTQgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL1N1YnR5cGUgL0ltYWdlIC9XaWR0aCAyMjYgL0hlaWdodCAyMjYKL0NvbG9yU3BhY2UgWyAvSW5kZXhlZCAvRGV2aWNlUkdCIDIgKP3nN/8AAAAiTSkgXSAvQml0c1BlckNvbXBvbmVudCAyCi9GaWx0ZXIgL0ZsYXRlRGVjb2RlCi9EZWNvZGVQYXJtcyA8PCAvUHJlZGljdG9yIDEwIC9Db2xvcnMgMSAvQ29sdW1ucyAyMjYgL0JpdHNQZXJDb21wb25lbnQgMiA+PgovTGVuZ3RoIDM4IDAgUiA+PgpzdHJlYW0KeJzty7ERABAQAMFXJPWJVekb+BGQ2UtvNkKSJEmSJEmSJN3WejZW1SRJknwky3WIJEmSJEmSJEmSJEmSJEmSJL+TGx1KsYUKZW5kc3RyZWFtCmVuZG9iagozOCAwIG9iago3MQplbmRvYmoKMiAwIG9iago8PCAvVHlwZSAvUGFnZXMgL0tpZHMgWyAxMSAwIFIgXSAvQ291bnQgMSA+PgplbmRvYmoKMzkgMCBvYmoKPDwgL0NyZWF0b3IgKE1hdHBsb3RsaWIgdjMuOC4wLCBodHRwczovL21hdHBsb3RsaWIub3JnKQovUHJvZHVjZXIgKE1hdHBsb3RsaWIgcGRmIGJhY2tlbmQgdjMuOC4wKSAvQ3JlYXRpb25EYXRlIChEOjIwMjMxMDExMTYzMzAyWikKPj4KZW5kb2JqCnhyZWYKMCA0MAowMDAwMDAwMDAwIDY1NTM1IGYgCjAwMDAwMDAwMTYgMDAwMDAgbiAKMDAwMDAwODMzNyAwMDAwMCBuIAowMDAwMDA2OTg4IDAwMDAwIG4gCjAwMDAwMDcwMjAgMDAwMDAgbiAKMDAwMDAwNzA4MCAwMDAwMCBuIAowMDAwMDA3MTAxIDAwMDAwIG4gCjAwMDAwMDcxMjIgMDAwMDAgbiAKMDAwMDAwMDA2NSAwMDAwMCBuIAowMDAwMDAwMzQ0IDAwMDAwIG4gCjAwMDAwMDA3MjEgMDAwMDAgbiAKMDAwMDAwMDIwOCAwMDAwMCBuIAowMDAwMDAwNzAxIDAwMDAwIG4gCjAwMDAwMDcxNjUgMDAwMDAgbiAKMDAwMDAwNzk3MSAwMDAwMCBuIAowMDAwMDA1NzI5IDAwMDAwIG4gCjAwMDAwMDU1MjIgMDAwMDAgbiAKMDAwMDAwNTExNCAwMDAwMCBuIAowMDAwMDA2NzgyIDAwMDAwIG4gCjAwMDAwMDA3NDEgMDAwMDAgbiAKMDAwMDAwMTA3OCAwMDAwMCBuIAowMDAwMDAxMjQyIDAwMDAwIG4gCjAwMDAwMDE2MjIgMDAwMDAgbiAKMDAwMDAwMTkyNyAwMDAwMCBuIAowMDAwMDAyMjMxIDAwMDAwIG4gCjAwMDAwMDI1NTMgMDAwMDAgbiAKMDAwMDAwMjc2MiAwMDAwMCBuIAowMDAwMDAzMTc2IDAwMDAwIG4gCjAwMDAwMDM0MTMgMDAwMDAgbiAKMDAwMDAwMzU1NyAwMDAwMCBuIAowMDAwMDAzNjc2IDAwMDAwIG4gCjAwMDAwMDM5MTIgMDAwMDAgbiAKMDAwMDAwNDIyNCAwMDAwMCBuIAowMDAwMDA0NDU3IDAwMDAwIG4gCjAwMDAwMDQ1NDcgMDAwMDAgbiAKMDAwMDAwNDc1MyAwMDAwMCBuIAowMDAwMDA0OTAwIDAwMDAwIG4gCjAwMDAwMDc5NTEgMDAwMDAgbiAKMDAwMDAwODMxOCAwMDAwMCBuIAowMDAwMDA4Mzk3IDAwMDAwIG4gCnRyYWlsZXIKPDwgL1NpemUgNDAgL1Jvb3QgMSAwIFIgL0luZm8gMzkgMCBSID4+CnN0YXJ0eHJlZgo4NTQ4CiUlRU9GCg==", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T16:33:02.421336\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["# Initialize convolutions with equal weight to all input pixels\n", "horiz_conv = HorizontalStackConvolution(c_in=1, c_out=1, kernel_size=3, mask_center=False)\n", "horiz_conv.conv.weight.data.fill_(1)\n", "horiz_conv.conv.bias.data.fill_(0)\n", "vert_conv = VerticalStackConvolution(c_in=1, c_out=1, kernel_size=3, mask_center=False)\n", "vert_conv.conv.weight.data.fill_(1)\n", "vert_conv.conv.bias.data.fill_(0)\n", "\n", "# We reuse our convolutions for the 4 layers here. Note that in a standard network,\n", "# we don't do that, and instead learn 4 separate convolution. As this cell is only for\n", "# visualization purposes, we reuse the convolutions for all layers.\n", "for l_idx in range(4):\n", " vert_img = vert_conv(vert_img)\n", " horiz_img = horiz_conv(horiz_img) + vert_img\n", " print(\"Layer %i\" % (l_idx + 2))\n", " show_center_recep_field(inp_img, horiz_img)"]}, {"cell_type": "markdown", "id": "0c34bd39", "metadata": {"papermill": {"duration": 0.018461, "end_time": "2023-10-11T16:33:02.628825", "exception": false, "start_time": "2023-10-11T16:33:02.610364", "status": "completed"}, "tags": []}, "source": ["The receptive field above it visualized for the horizontal stack, which includes the features of the vertical convolutions.\n", "It grows over layers without any blind spot as we had before.\n", "The difference between \"weighted\" and \"binary\" receptive field is that for the latter, we check whether there are any gradients flowing back to this pixel.\n", "This indicates that the center pixel indeed can use information from this pixel.\n", "Nevertheless, due to the convolution weights, some pixels have a stronger effect on the prediction than others.\n", "This is visualized in the weighted receptive field by plotting the gradient magnitude for each pixel instead of a binary yes/no.\n", "\n", "\n", "Another receptive field we can check is the one for the vertical stack\n", "as the one above is for the horizontal stack. Let's visualize it below:"]}, {"cell_type": "code", "execution_count": 13, "id": "6cb0b47d", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:33:02.679685Z", "iopub.status.busy": "2023-10-11T16:33:02.678503Z", "iopub.status.idle": "2023-10-11T16:33:03.078740Z", "shell.execute_reply": "2023-10-11T16:33:03.078030Z"}, "papermill": {"duration": 0.430988, "end_time": "2023-10-11T16:33:03.080865", "exception": false, "start_time": "2023-10-11T16:33:02.649877", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMzcxLjUyIDE5MS44NTIyNzI3MjczIF0gL0NvbnRlbnRzIDkgMCBSIC9Bbm5vdHMgMTAgMCBSID4+CmVuZG9iago5IDAgb2JqCjw8IC9MZW5ndGggMTIgMCBSIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nJ1QTU8CMRC9z694RzjY7UztdvcookRv6CYcjAezlGUJoAsE4r93gKwfXEQ7ecm8Tqfz5iX9uK3L+DDo4fqRki9WrokxU1SwmCl2YAwUFVllC3KBjRdN523KOZvMiwSnd/YnnRJNqEEwcgCnYpyEY7hTuooYYYnkSqetdeRMsdNJgxOBzaEx7EXYNg0a5QLJHaP/iiEN0Wix+v7LnlOjq1lcaB/YG7Y+E5elLgUHr8t8aikX1CsoudVngmJyWL4Y0xM6oy58Znzugkcn1tV0E8dYdSHWZK49Winj26beRkzqOB938Yzinm4KOkoTyybP23H/9Yb2W/3ujVhp3ZGz3RHOjbWW2ak7l+fb06uXL6v3vxlCH5GVksMKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyODIKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0xlbmd0aCAyNjQgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9MZW5ndGggOTIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0xlbmd0aCAzMDcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvTGVuZ3RoIDIzMiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9MZW5ndGggMjMxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDI0OSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0xlbmd0aCAxMzYgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvTGVuZ3RoIDM0MSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvTGVuZ3RoIDE2NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvTGVuZ3RoIDcyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvTGVuZ3RoIDQ3IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9MZW5ndGggMTYzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0xlbmd0aCAyMzkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0xlbmd0aCAxNjAgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvTGVuZ3RoIDE4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvTGVuZ3RoIDEzMyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9MZW5ndGggNzUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9MZW5ndGggMTQxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvVHlwZSAvRm9udCAvQmFzZUZvbnQgL0JNUVFEVitEZWphVnVTYW5zIC9GaXJzdENoYXIgMCAvTGFzdENoYXIgMjU1Ci9Gb250RGVzY3JpcHRvciAxNiAwIFIgL1N1YnR5cGUgL1R5cGUzIC9OYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdCi9DaGFyUHJvY3MgMTggMCBSCi9FbmNvZGluZyA8PCAvVHlwZSAvRW5jb2RpbmcKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCj4+Ci9XaWR0aHMgMTUgMCBSID4+CmVuZG9iagoxNiAwIG9iago8PCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucyAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvQXNjZW50IDkyOSAvRGVzY2VudCAtMjM2IC9DYXBIZWlnaHQgMAovWEhlaWdodCAwIC9JdGFsaWNBbmdsZSAwIC9TdGVtViAwIC9NYXhXaWR0aCAxMzQyID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiAxNgoo/ec3pZtzACRSZGdvACZVKj9tIjtuCDNwACVU/wAA38pWACpikYx3AC1pTlZsNERsACJNKSBdCi9CaXRzUGVyQ29tcG9uZW50IDggL0ZpbHRlciAvRmxhdGVEZWNvZGUKL0RlY29kZVBhcm1zIDw8IC9QcmVkaWN0b3IgMTAgL0NvbG9ycyAxIC9Db2x1bW5zIDIyNiAvQml0c1BlckNvbXBvbmVudCA4ID4+Ci9MZW5ndGggMzcgMCBSID4+CnN0cmVhbQp4nO3buY3gABDEQN7/P/lHewmMeRR3sV22oGG7AsTfy9fDj8uXw+fLp8P54PXG8/TVeI5hEzdxEzdxEzdxEzdxEzdxEzdxEzdxEzdxE1/uxPeXX4c/lw+Hn5d3h/PB643n6avxHLOJm7iJm7iJm7iJm7iJm7iJm7iJm7iJm/h6J368XN+Ifl+uyO8XDueD1xvP01fjOWYTN3ETN3ETN3ETN3ETN3ETN3ETN3ETN3ETX/DEN/AFbhM3cRM3cRM3cRM3cRM3cRM3cRM3cRM38dVOfAN/vm3iJm7iJm7iJm7iJm7iJm7iJm7iJm7iJm7iK5v4/307PHSah+5soomH7myiiYfubKKJh+5soomH7myiiYfubKKJh+5soomH7myiiYfubKKJh+68gYkh6gAfdYCPOsBHHeCjDvBRB/ioA3zUAT7qAB91gI86wEcd4KMO8FEH+KgDfNQBPuoAH3WAjzrARx3gow7wUQf4qAN81AE+6gAfdYCPOsBHHeCjDvBRB/ioA3zUAT7qAB91gI86wEcd4KMO8FEH+KgDfNQBPuoAH3WAjzrARx3gow7wUQf4qAN81AE+6gAfdYCPOsBHHeCjDvBRB/ioA3zUAT7qAB91gI86wEcd4KMO8FEH+KgDfNQBPuoAH3WAjzrARx3gow7wUQf4qAN81AE+6gAfdYCPOsBHHeCjDvBRB/ioA3zUAT7qAB91gI86wEcd4KMO8FEH+KgDfNQBPuoAH3WAjzrARx3gow7wUQf4qAN81AE+6gAfdYCPOsBHHeCjDvBRB/j+AY1R2XQKZW5kc3RyZWFtCmVuZG9iagozNyAwIG9iago1OTcKZW5kb2JqCjE0IDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiAyICj95zf/AAAAIk0pIF0gL0JpdHNQZXJDb21wb25lbnQgMgovRmlsdGVyIC9GbGF0ZURlY29kZQovRGVjb2RlUGFybXMgPDwgL1ByZWRpY3RvciAxMCAvQ29sb3JzIDEgL0NvbHVtbnMgMjI2IC9CaXRzUGVyQ29tcG9uZW50IDIgPj4KL0xlbmd0aCAzOCAwIFIgPj4Kc3RyZWFtCnic7cuxEQAQEADBb5L6pFTpG/ghEdlLbzZCkiRJkiRJkiRdtqpmy3q5B0mS5Gt5iCRJkiRJkiRJkiRJkiRJkiS/kxuWTwrLCmVuZHN0cmVhbQplbmRvYmoKMzggMCBvYmoKNzEKZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFsgMTEgMCBSIF0gL0NvdW50IDEgPj4KZW5kb2JqCjM5IDAgb2JqCjw8IC9DcmVhdG9yIChNYXRwbG90bGliIHYzLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjguMCkgL0NyZWF0aW9uRGF0ZSAoRDoyMDIzMTAxMTE2MzMwM1opCj4+CmVuZG9iagp4cmVmCjAgNDAKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDg0NjcgMDAwMDAgbiAKMDAwMDAwNjk4OCAwMDAwMCBuIAowMDAwMDA3MDIwIDAwMDAwIG4gCjAwMDAwMDcwODAgMDAwMDAgbiAKMDAwMDAwNzEwMSAwMDAwMCBuIAowMDAwMDA3MTIyIDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM0NCAwMDAwMCBuIAowMDAwMDAwNzIxIDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDcwMSAwMDAwMCBuIAowMDAwMDA3MTY1IDAwMDAwIG4gCjAwMDAwMDgxMDEgMDAwMDAgbiAKMDAwMDAwNTcyOSAwMDAwMCBuIAowMDAwMDA1NTIyIDAwMDAwIG4gCjAwMDAwMDUxMTQgMDAwMDAgbiAKMDAwMDAwNjc4MiAwMDAwMCBuIAowMDAwMDAwNzQxIDAwMDAwIG4gCjAwMDAwMDEwNzggMDAwMDAgbiAKMDAwMDAwMTI0MiAwMDAwMCBuIAowMDAwMDAxNjIyIDAwMDAwIG4gCjAwMDAwMDE5MjcgMDAwMDAgbiAKMDAwMDAwMjIzMSAwMDAwMCBuIAowMDAwMDAyNTUzIDAwMDAwIG4gCjAwMDAwMDI3NjIgMDAwMDAgbiAKMDAwMDAwMzE3NiAwMDAwMCBuIAowMDAwMDAzNDEzIDAwMDAwIG4gCjAwMDAwMDM1NTcgMDAwMDAgbiAKMDAwMDAwMzY3NiAwMDAwMCBuIAowMDAwMDAzOTEyIDAwMDAwIG4gCjAwMDAwMDQyMjQgMDAwMDAgbiAKMDAwMDAwNDQ1NyAwMDAwMCBuIAowMDAwMDA0NTQ3IDAwMDAwIG4gCjAwMDAwMDQ3NTMgMDAwMDAgbiAKMDAwMDAwNDkwMCAwMDAwMCBuIAowMDAwMDA4MDgxIDAwMDAwIG4gCjAwMDAwMDg0NDggMDAwMDAgbiAKMDAwMDAwODUyNyAwMDAwMCBuIAp0cmFpbGVyCjw8IC9TaXplIDQwIC9Sb290IDEgMCBSIC9JbmZvIDM5IDAgUiA+PgpzdGFydHhyZWYKODY3OAolJUVPRgo=", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T16:33:02.906042\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["show_center_recep_field(inp_img, vert_img)"]}, {"cell_type": "markdown", "id": "4f11b075", "metadata": {"papermill": {"duration": 0.025983, "end_time": "2023-10-11T16:33:03.129794", "exception": false, "start_time": "2023-10-11T16:33:03.103811", "status": "completed"}, "tags": []}, "source": ["As we have discussed before, the vertical stack only looks at pixels above the one we want to predict.\n", "Hence, we can validate that our implementation works as we initially expected it to.\n", "As a final step, let's clean up the computation graph we still had kept\n", "in memory for the visualization of the receptive field:"]}, {"cell_type": "code", "execution_count": 14, "id": "6a3e2981", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:33:03.177517Z", "iopub.status.busy": "2023-10-11T16:33:03.173856Z", "iopub.status.idle": "2023-10-11T16:33:03.187847Z", "shell.execute_reply": "2023-10-11T16:33:03.186786Z"}, "papermill": {"duration": 0.035686, "end_time": "2023-10-11T16:33:03.189444", "exception": false, "start_time": "2023-10-11T16:33:03.153758", "status": "completed"}, "tags": []}, "outputs": [], "source": ["del inp_img, horiz_conv, vert_conv"]}, {"cell_type": "markdown", "id": "ce9f1ee6", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.019031, "end_time": "2023-10-11T16:33:03.227796", "exception": false, "start_time": "2023-10-11T16:33:03.208765", "status": "completed"}, "tags": []}, "source": ["## Gated PixelCNN\n", "\n", "
\n", "\n", "In the next step, we will use the masked convolutions to build a full autoregressive model, called Gated PixelCNN.\n", "The difference between the original PixelCNN and Gated PixelCNN is the use of separate horizontal and vertical stacks.\n", "However, in literature, you often see that people refer to the Gated PixelCNN simply as \"PixelCNN\".\n", "Hence, in the following, if we say \"PixelCNN\", we usually mean the gated version.\n", "What \"Gated\" refers to in the model name is explained next.\n", "\n", "### Gated Convolutions\n", "\n", "For visualizing the receptive field, we assumed a very simplified stack of vertical and horizontal convolutions.\n", "Obviously, there are more sophisticated ways of doing it, and PixelCNN uses gated convolutions for this.\n", "Specifically, the Gated Convolution block in PixelCNN looks as follows\n", "(figure credit - [Aaron van den Oord et al. ](https://arxiv.org/pdf/1606.05328.pdf)):\n", "\n", "
\n", "\n", "The left path is the vertical stack (the $N\\times N$ convolution is masked correspondingly),\n", "and the right path is the horizontal stack.\n", "Gated convolutions are implemented by having a twice as large output channel size,\n", "and combine them by a element-wise multiplication of $\\tanh$ and a sigmoid.\n", "For a linear layer, we can express a gated activation unit as follows:\n", "\n", "$$\\mathbf{y} = \\tanh\\left(\\mathbf{W}_{f}\\mathbf{x}\\right)\\odot\\sigma\\left(\\mathbf{W}_{g}\\mathbf{x}\\right)$$\n", "\n", "For simplicity, biases have been neglected and the linear layer split into two part, $\\mathbf{W}_{f}$ and $\\mathbf{W}_{g}$.\n", "This concept resembles the input and modulation gate in an LSTM, and has been used in many other architectures as well.\n", "The main motivation behind this gated activation is that it might allow to model more complex interactions and simplifies learning.\n", "But as in any other architecture, this is mostly a design choice and can be considered a hyperparameters.\n", "\n", "Besides the gated convolutions, we also see that the horizontal stack uses a residual connection while the vertical stack does not.\n", "This is because we use the output of the horizontal stack for prediction.\n", "Each convolution in the vertical stack also receives a strong gradient signal\n", "as it is only two $1\\times 1$ convolutions away from the residual connection,\n", "and does not require another residual connection to all its earleri layers.\n", "\n", "The implementation in PyTorch is fairly straight forward for this block,\n", "because the visualization above gives us a computation graph to follow:"]}, {"cell_type": "code", "execution_count": 15, "id": "47bf5a5e", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:33:03.290181Z", "iopub.status.busy": "2023-10-11T16:33:03.289672Z", "iopub.status.idle": "2023-10-11T16:33:03.298299Z", "shell.execute_reply": "2023-10-11T16:33:03.297335Z"}, "lines_to_next_cell": 2, "papermill": {"duration": 0.041182, "end_time": "2023-10-11T16:33:03.300732", "exception": false, "start_time": "2023-10-11T16:33:03.259550", "status": "completed"}, "tags": []}, "outputs": [], "source": ["class GatedMaskedConv(nn.Module):\n", " def __init__(self, c_in, **kwargs):\n", " \"\"\"Gated Convolution block implemented the computation graph shown above.\"\"\"\n", " super().__init__()\n", " self.conv_vert = VerticalStackConvolution(c_in, c_out=2 * c_in, **kwargs)\n", " self.conv_horiz = HorizontalStackConvolution(c_in, c_out=2 * c_in, **kwargs)\n", " self.conv_vert_to_horiz = nn.Conv2d(2 * c_in, 2 * c_in, kernel_size=1, padding=0)\n", " self.conv_horiz_1x1 = nn.Conv2d(c_in, c_in, kernel_size=1, padding=0)\n", "\n", " def forward(self, v_stack, h_stack):\n", " # Vertical stack (left)\n", " v_stack_feat = self.conv_vert(v_stack)\n", " v_val, v_gate = v_stack_feat.chunk(2, dim=1)\n", " v_stack_out = torch.tanh(v_val) * torch.sigmoid(v_gate)\n", "\n", " # Horizontal stack (right)\n", " h_stack_feat = self.conv_horiz(h_stack)\n", " h_stack_feat = h_stack_feat + self.conv_vert_to_horiz(v_stack_feat)\n", " h_val, h_gate = h_stack_feat.chunk(2, dim=1)\n", " h_stack_feat = torch.tanh(h_val) * torch.sigmoid(h_gate)\n", " h_stack_out = self.conv_horiz_1x1(h_stack_feat)\n", " h_stack_out = h_stack_out + h_stack\n", "\n", " return v_stack_out, h_stack_out"]}, {"cell_type": "markdown", "id": "70fb33a4", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.023851, "end_time": "2023-10-11T16:33:03.349989", "exception": false, "start_time": "2023-10-11T16:33:03.326138", "status": "completed"}, "tags": []}, "source": ["### Building the model\n", "\n", "Using the gated convolutions, we can now build our PixelCNN model.\n", "The architecture consists of multiple stacked GatedMaskedConv blocks, where we add an additional dilation factor to a few convolutions.\n", "This is used to increase the receptive field of the model and allows to take a larger context into account during generation.\n", "As a reminder, dilation on a convolution works looks as follows\n", "(figure credit - [Vincent Dumoulin and Francesco Visin](https://arxiv.org/pdf/1603.07285.pdf)):\n", "\n", "
\n", "\n", "Note that the smaller output size is only because the animation assumes no padding.\n", "In our implementation, we will pad the input image correspondingly.\n", "Alternatively to dilated convolutions, we could downsample the input and use a encoder-decoder architecture as in PixelCNN++ [3].\n", "This is especially beneficial if we want to build a very deep autoregressive model.\n", "Nonetheless, as we seek to train a reasonably small model, dilated convolutions are the more efficient option to use here.\n", "\n", "Below, we implement the PixelCNN model as a PyTorch Lightning module.\n", "Besides the stack of gated convolutions, we also have the initial\n", "horizontal and vertical convolutions which mask the center pixel, and a\n", "final $1\\times 1$ convolution which maps the output features to class\n", "predictions. To determine the likelihood of a batch of images, we first\n", "create our initial features using the masked horizontal and vertical\n", "input convolution. Next, we forward the features through the stack of\n", "gated convolutions. Finally, we take the output features of the\n", "horizontal stack, and apply the $1\\times 1$ convolution for\n", "classification. We use the bits per dimension metric for the likelihood,\n", "similarly to Tutorial 11 and assignment 3."]}, {"cell_type": "code", "execution_count": 16, "id": "227ece04", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:33:03.399604Z", "iopub.status.busy": "2023-10-11T16:33:03.399067Z", "iopub.status.idle": "2023-10-11T16:33:03.417286Z", "shell.execute_reply": "2023-10-11T16:33:03.416282Z"}, "papermill": {"duration": 0.044676, "end_time": "2023-10-11T16:33:03.419123", "exception": false, "start_time": "2023-10-11T16:33:03.374447", "status": "completed"}, "tags": []}, "outputs": [], "source": ["class PixelCNN(L.LightningModule):\n", " def __init__(self, c_in, c_hidden):\n", " super().__init__()\n", " self.save_hyperparameters()\n", "\n", " # Initial convolutions skipping the center pixel\n", " self.conv_vstack = VerticalStackConvolution(c_in, c_hidden, mask_center=True)\n", " self.conv_hstack = HorizontalStackConvolution(c_in, c_hidden, mask_center=True)\n", " # Convolution block of PixelCNN. We use dilation instead of downscaling\n", " self.conv_layers = nn.ModuleList(\n", " [\n", " GatedMaskedConv(c_hidden),\n", " GatedMaskedConv(c_hidden, dilation=2),\n", " GatedMaskedConv(c_hidden),\n", " GatedMaskedConv(c_hidden, dilation=4),\n", " GatedMaskedConv(c_hidden),\n", " GatedMaskedConv(c_hidden, dilation=2),\n", " GatedMaskedConv(c_hidden),\n", " ]\n", " )\n", " # Output classification convolution (1x1)\n", " self.conv_out = nn.Conv2d(c_hidden, c_in * 256, kernel_size=1, padding=0)\n", "\n", " self.example_input_array = train_set[0][0][None]\n", "\n", " def forward(self, x):\n", " \"\"\"Forward image through model and return logits for each pixel.\n", "\n", " Args:\n", " x: Image tensor with integer values between 0 and 255.\n", " \"\"\"\n", " # Scale input from 0 to 255 back to -1 to 1\n", " x = (x.float() / 255.0) * 2 - 1\n", "\n", " # Initial convolutions\n", " v_stack = self.conv_vstack(x)\n", " h_stack = self.conv_hstack(x)\n", " # Gated Convolutions\n", " for layer in self.conv_layers:\n", " v_stack, h_stack = layer(v_stack, h_stack)\n", " # 1x1 classification convolution\n", " # Apply ELU before 1x1 convolution for non-linearity on residual connection\n", " out = self.conv_out(F.elu(h_stack))\n", "\n", " # Output dimensions: [Batch, Classes, Channels, Height, Width]\n", " out = out.reshape(out.shape[0], 256, out.shape[1] // 256, out.shape[2], out.shape[3])\n", " return out\n", "\n", " def calc_likelihood(self, x):\n", " # Forward pass with bpd likelihood calculation\n", " pred = self.forward(x)\n", " nll = F.cross_entropy(pred, x, reduction=\"none\")\n", " bpd = nll.mean(dim=[1, 2, 3]) * np.log2(np.exp(1))\n", " return bpd.mean()\n", "\n", " @torch.no_grad()\n", " def sample(self, img_shape, img=None):\n", " \"\"\"Sampling function for the autoregressive model.\n", "\n", " Args:\n", " img_shape: Shape of the image to generate (B,C,H,W)\n", " img (optional): If given, this tensor will be used as\n", " a starting image. The pixels to fill\n", " should be -1 in the input tensor.\n", " \"\"\"\n", " # Create empty image\n", " if img is None:\n", " img = torch.zeros(img_shape, dtype=torch.long).to(device) - 1\n", " # Generation loop\n", " for h in tqdm(range(img_shape[2]), leave=False):\n", " for w in range(img_shape[3]):\n", " for c in range(img_shape[1]):\n", " # Skip if not to be filled (-1)\n", " if (img[:, c, h, w] != -1).all().item():\n", " continue\n", " # For efficiency, we only have to input the upper part of the image\n", " # as all other parts will be skipped by the masked convolutions anyways\n", " pred = self.forward(img[:, :, : h + 1, :])\n", " probs = F.softmax(pred[:, :, c, h, w], dim=-1)\n", " img[:, c, h, w] = torch.multinomial(probs, num_samples=1).squeeze(dim=-1)\n", " return img\n", "\n", " def configure_optimizers(self):\n", " optimizer = optim.Adam(self.parameters(), lr=1e-3)\n", " scheduler = optim.lr_scheduler.StepLR(optimizer, 1, gamma=0.99)\n", " return [optimizer], [scheduler]\n", "\n", " def training_step(self, batch, batch_idx):\n", " loss = self.calc_likelihood(batch[0])\n", " self.log(\"train_bpd\", loss)\n", " return loss\n", "\n", " def validation_step(self, batch, batch_idx):\n", " loss = self.calc_likelihood(batch[0])\n", " self.log(\"val_bpd\", loss)\n", "\n", " def test_step(self, batch, batch_idx):\n", " loss = self.calc_likelihood(batch[0])\n", " self.log(\"test_bpd\", loss)"]}, {"cell_type": "markdown", "id": "ed569c97", "metadata": {"papermill": {"duration": 0.023337, "end_time": "2023-10-11T16:33:03.476301", "exception": false, "start_time": "2023-10-11T16:33:03.452964", "status": "completed"}, "tags": []}, "source": ["To sample from the autoregressive model, we need to iterate over all dimensions of the input.\n", "We start with an empty image, and fill the pixels one by one, starting from the upper left corner.\n", "Note that as for predicting $x_i$, all pixels below it have no influence on the prediction.\n", "Hence, we can cut the image in height without changing the prediction while increasing efficiency.\n", "Nevertheless, all the loops in the sampling function already show that it will take us quite some time to sample.\n", "A lot of computation could be reused across loop iterations as those the features on the already predicted pixels will not change over iterations.\n", "Nevertheless, this takes quite some effort to implement, and is often not done in implementations because in the end,\n", "autoregressive sampling remains sequential and slow.\n", "Hence, we settle with the default implementation here.\n", "\n", "Before training the model, we can check the full receptive field of the model on an MNIST image of size $28\\times 28$:"]}, {"cell_type": "code", "execution_count": 17, "id": "37ef0017", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:33:03.526873Z", "iopub.status.busy": "2023-10-11T16:33:03.526391Z", "iopub.status.idle": "2023-10-11T16:33:13.921338Z", "shell.execute_reply": "2023-10-11T16:33:13.911749Z"}, "papermill": {"duration": 10.42969, "end_time": "2023-10-11T16:33:13.930120", "exception": false, "start_time": "2023-10-11T16:33:03.500430", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMzcxLjUyIDE5MS44NTIyNzI3MjczIF0gL0NvbnRlbnRzIDkgMCBSIC9Bbm5vdHMgMTAgMCBSID4+CmVuZG9iago5IDAgb2JqCjw8IC9MZW5ndGggMTIgMCBSIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nJ1QTU8CMRC9z694RzjY7UztdvcookRv6CYcjAezlGUJoAsE4r93gKwfXEQ7ecm8Tqfz5iX9uK3L+DDo4fqRki9WrokxU1SwmCl2YAwUFVllC3KBjRdN523KOZvMiwSnd/YnnRJNqEEwcgCnYpyEY7hTuooYYYnkSqetdeRMsdNJgxOBzaEx7EXYNg0a5QLJHaP/iiEN0Wix+v7LnlOjq1lcaB/YG7Y+E5elLgUHr8t8aikX1CsoudVngmJyWL4Y0xM6oy58Znzugkcn1tV0E8dYdSHWZK49Winj26beRkzqOB938Yzinm4KOkoTyybP23H/9Yb2W/3ujVhp3ZGz3RHOjbWW2ak7l+fb06uXL6v3vxlCH5GVksMKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyODIKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0xlbmd0aCAyNjQgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9MZW5ndGggOTIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0xlbmd0aCAzMDcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvTGVuZ3RoIDIzMiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9MZW5ndGggMjMxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDI0OSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0xlbmd0aCAxMzYgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvTGVuZ3RoIDM0MSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvTGVuZ3RoIDE2NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvTGVuZ3RoIDcyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvTGVuZ3RoIDQ3IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9MZW5ndGggMTYzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0xlbmd0aCAyMzkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0xlbmd0aCAxNjAgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvTGVuZ3RoIDE4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvTGVuZ3RoIDEzMyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9MZW5ndGggNzUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9MZW5ndGggMTQxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvVHlwZSAvRm9udCAvQmFzZUZvbnQgL0JNUVFEVitEZWphVnVTYW5zIC9GaXJzdENoYXIgMCAvTGFzdENoYXIgMjU1Ci9Gb250RGVzY3JpcHRvciAxNiAwIFIgL1N1YnR5cGUgL1R5cGUzIC9OYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdCi9DaGFyUHJvY3MgMTggMCBSCi9FbmNvZGluZyA8PCAvVHlwZSAvRW5jb2RpbmcKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCj4+Ci9XaWR0aHMgMTUgMCBSID4+CmVuZG9iagoxNiAwIG9iago8PCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucyAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvQXNjZW50IDkyOSAvRGVzY2VudCAtMjM2IC9DYXBIZWlnaHQgMAovWEhlaWdodCAwIC9JdGFsaWNBbmdsZSAwIC9TdGVtViAwIC9NYXhXaWR0aCAxMzQyID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN0IC9TdWJ0eXBlIC9JbWFnZSAvV2lkdGggMjI2IC9IZWlnaHQgMjI2Ci9Db2xvclNwYWNlIFsgL0luZGV4ZWQgL0RldmljZVJHQiAxNQoo/ec3ADFwACZVACNPACRSACZXITtuHTluACNQ/wAAAFwoWwBcKFxcACtkACxnBDJwACJNKSBdCi9CaXRzUGVyQ29tcG9uZW50IDQgL0ZpbHRlciAvRmxhdGVEZWNvZGUKL0RlY29kZVBhcm1zIDw8IC9QcmVkaWN0b3IgMTAgL0NvbG9ycyAxIC9Db2x1bW5zIDIyNiAvQml0c1BlckNvbXBvbmVudCA0ID4+Ci9MZW5ndGggMzcgMCBSID4+CnN0cmVhbQp4nO3VoU7EQBAG4LXIOjQGTw3uTAUOQSpJUH2FKhQhIUGgq/ANb1CBRdTwAD15suYgIZhjG3pJJVzSQ/D9ZiYj5luzmbDZdwKRSCQSiUQikUgkEolEIpFIJBKJRCKRSCQSiUQikUj8e/Hj7jsPsV+naXo69m9Zlp3F/ibOFkQicS/iZiIN29NR2M62ryESifOJ63Hz8NPPY/2s6/qpbduXKNzmeX4xecWCSCTOKg6ZXuP3qFx2XfeaJMlhnN03TfP8kyVEInEncdh+FDNc3ZNYj8uyvI7Ksu/7VVEUVyGEg6qqHn8jEYnEncRZQyQSiUQikUgkEolEIpFIJBKJRCKRSCQSiUQikUgkEolEIpFIJBKJxP8pfgEnzHpfCmVuZHN0cmVhbQplbmRvYmoKMzcgMCBvYmoKMjgzCmVuZG9iagoxNCAwIG9iago8PCAvVHlwZSAvWE9iamVjdCAvU3VidHlwZSAvSW1hZ2UgL1dpZHRoIDIyNiAvSGVpZ2h0IDIyNgovQ29sb3JTcGFjZSBbIC9JbmRleGVkIC9EZXZpY2VSR0IgMiAo/ec3/wAAACJNKSBdIC9CaXRzUGVyQ29tcG9uZW50IDIKL0ZpbHRlciAvRmxhdGVEZWNvZGUKL0RlY29kZVBhcm1zIDw8IC9QcmVkaWN0b3IgMTAgL0NvbG9ycyAxIC9Db2x1bW5zIDIyNiAvQml0c1BlckNvbXBvbmVudCAyID4+Ci9MZW5ndGggMzggMCBSID4+CnN0cmVhbQp4nO3ZMQEAIAzEwJpAGiZQxYxKPPxAGS77KUidsF3kn7LCSJIkSZIkSZIkSZIkSZIkSZLslWOu8AGQfTKLJEmSJEmSJEmSJEmSJEmSJMkH8gLidNhUCmVuZHN0cmVhbQplbmRvYmoKMzggMCBvYmoKODUKZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFsgMTEgMCBSIF0gL0NvdW50IDEgPj4KZW5kb2JqCjM5IDAgb2JqCjw8IC9DcmVhdG9yIChNYXRwbG90bGliIHYzLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjguMCkgL0NyZWF0aW9uRGF0ZSAoRDoyMDIzMTAxMTE2MzMxM1opCj4+CmVuZG9iagp4cmVmCjAgNDAKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDgxNjcgMDAwMDAgbiAKMDAwMDAwNjk4OCAwMDAwMCBuIAowMDAwMDA3MDIwIDAwMDAwIG4gCjAwMDAwMDcwODAgMDAwMDAgbiAKMDAwMDAwNzEwMSAwMDAwMCBuIAowMDAwMDA3MTIyIDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM0NCAwMDAwMCBuIAowMDAwMDAwNzIxIDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDcwMSAwMDAwMCBuIAowMDAwMDA3MTY1IDAwMDAwIG4gCjAwMDAwMDc3ODcgMDAwMDAgbiAKMDAwMDAwNTcyOSAwMDAwMCBuIAowMDAwMDA1NTIyIDAwMDAwIG4gCjAwMDAwMDUxMTQgMDAwMDAgbiAKMDAwMDAwNjc4MiAwMDAwMCBuIAowMDAwMDAwNzQxIDAwMDAwIG4gCjAwMDAwMDEwNzggMDAwMDAgbiAKMDAwMDAwMTI0MiAwMDAwMCBuIAowMDAwMDAxNjIyIDAwMDAwIG4gCjAwMDAwMDE5MjcgMDAwMDAgbiAKMDAwMDAwMjIzMSAwMDAwMCBuIAowMDAwMDAyNTUzIDAwMDAwIG4gCjAwMDAwMDI3NjIgMDAwMDAgbiAKMDAwMDAwMzE3NiAwMDAwMCBuIAowMDAwMDAzNDEzIDAwMDAwIG4gCjAwMDAwMDM1NTcgMDAwMDAgbiAKMDAwMDAwMzY3NiAwMDAwMCBuIAowMDAwMDAzOTEyIDAwMDAwIG4gCjAwMDAwMDQyMjQgMDAwMDAgbiAKMDAwMDAwNDQ1NyAwMDAwMCBuIAowMDAwMDA0NTQ3IDAwMDAwIG4gCjAwMDAwMDQ3NTMgMDAwMDAgbiAKMDAwMDAwNDkwMCAwMDAwMCBuIAowMDAwMDA3NzY3IDAwMDAwIG4gCjAwMDAwMDgxNDggMDAwMDAgbiAKMDAwMDAwODIyNyAwMDAwMCBuIAp0cmFpbGVyCjw8IC9TaXplIDQwIC9Sb290IDEgMCBSIC9JbmZvIDM5IDAgUiA+PgpzdGFydHhyZWYKODM3OAolJUVPRgo=", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T16:33:13.736191\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["test_model = PixelCNN(c_in=1, c_hidden=64)\n", "inp = torch.zeros(1, 1, 28, 28)\n", "inp.requires_grad_()\n", "out = test_model(inp)\n", "show_center_recep_field(inp, out.squeeze(dim=2))\n", "del inp, out, test_model"]}, {"cell_type": "markdown", "id": "1f00c4b2", "metadata": {"papermill": {"duration": 0.023927, "end_time": "2023-10-11T16:33:13.977670", "exception": false, "start_time": "2023-10-11T16:33:13.953743", "status": "completed"}, "tags": []}, "source": ["The visualization shows that for predicting any pixel, we can take almost half of the image into account.\n", "However, keep in mind that this is the \"theoretical\" receptive field and not necessarily\n", "the [effective receptive field](https://arxiv.org/pdf/1701.04128.pdf), which is usually much smaller.\n", "For a stronger model, we should therefore try to increase the receptive\n", "field even further. Especially, for the pixel on the bottom right, the\n", "very last pixel, we would be allowed to take into account the whole\n", "image. However, our current receptive field only spans across 1/4 of the\n", "image. An encoder-decoder architecture can help with this, but it also\n", "shows that we require a much deeper, more complex network in\n", "autoregressive models than in VAEs or energy-based models."]}, {"cell_type": "markdown", "id": "d26b4d9e", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.02414, "end_time": "2023-10-11T16:33:14.026163", "exception": false, "start_time": "2023-10-11T16:33:14.002023", "status": "completed"}, "tags": []}, "source": ["### Training loop\n", "\n", "To train the model, we again can rely on PyTorch Lightning and write a\n", "function below for loading the pretrained model if it exists. To reduce\n", "the computational cost, we have saved the validation and test score in\n", "the checkpoint already:"]}, {"cell_type": "code", "execution_count": 18, "id": "5efc54ef", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:33:14.076835Z", "iopub.status.busy": "2023-10-11T16:33:14.076302Z", "iopub.status.idle": "2023-10-11T16:33:14.085316Z", "shell.execute_reply": "2023-10-11T16:33:14.084465Z"}, "papermill": {"duration": 0.03622, "end_time": "2023-10-11T16:33:14.086824", "exception": false, "start_time": "2023-10-11T16:33:14.050604", "status": "completed"}, "tags": []}, "outputs": [], "source": ["def train_model(**kwargs):\n", " # Create a PyTorch Lightning trainer with the generation callback\n", " trainer = L.Trainer(\n", " default_root_dir=os.path.join(CHECKPOINT_PATH, \"PixelCNN\"),\n", " accelerator=\"auto\",\n", " devices=1,\n", " max_epochs=150,\n", " callbacks=[\n", " ModelCheckpoint(save_weights_only=True, mode=\"min\", monitor=\"val_bpd\"),\n", " LearningRateMonitor(\"epoch\"),\n", " ],\n", " )\n", " result = None\n", " # Check whether pretrained model exists. If yes, load it and skip training\n", " pretrained_filename = os.path.join(CHECKPOINT_PATH, \"PixelCNN.ckpt\")\n", " if os.path.isfile(pretrained_filename):\n", " print(\"Found pretrained model, loading...\")\n", " model = PixelCNN.load_from_checkpoint(pretrained_filename)\n", " ckpt = torch.load(pretrained_filename, map_location=device)\n", " result = ckpt.get(\"result\", None)\n", " else:\n", " model = PixelCNN(**kwargs)\n", " trainer.fit(model, train_loader, val_loader)\n", " model = model.to(device)\n", "\n", " if result is None:\n", " # Test best model on validation and test set\n", " val_result = trainer.test(model, dataloaders=val_loader, verbose=False)\n", " test_result = trainer.test(model, dataloaders=test_loader, verbose=False)\n", " result = {\"test\": test_result, \"val\": val_result}\n", " return model, result"]}, {"cell_type": "markdown", "id": "458169ad", "metadata": {"papermill": {"duration": 0.024165, "end_time": "2023-10-11T16:33:14.135326", "exception": false, "start_time": "2023-10-11T16:33:14.111161", "status": "completed"}, "tags": []}, "source": ["Training the model is time consuming and we recommend using the provided pre-trained model for going through this notebook.\n", "However, feel free to play around with the hyperparameter like number of layers etc.\n", "if you want to get a feeling for those.\n", "\n", "When calling the training function with a pre-trained model, we automatically load it and print its test performance:"]}, {"cell_type": "code", "execution_count": 19, "id": "e4c53b11", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:33:14.186116Z", "iopub.status.busy": "2023-10-11T16:33:14.185398Z", "iopub.status.idle": "2023-10-11T16:33:15.110991Z", "shell.execute_reply": "2023-10-11T16:33:15.110044Z"}, "papermill": {"duration": 0.95267, "end_time": "2023-10-11T16:33:15.112725", "exception": false, "start_time": "2023-10-11T16:33:14.160055", "status": "completed"}, "tags": []}, "outputs": [{"name": "stderr", "output_type": "stream", "text": ["GPU available: True (cuda), used: True\n"]}, {"name": "stderr", "output_type": "stream", "text": ["TPU available: False, using: 0 TPU cores\n"]}, {"name": "stderr", "output_type": "stream", "text": ["IPU available: False, using: 0 IPUs\n"]}, {"name": "stderr", "output_type": "stream", "text": ["HPU available: False, using: 0 HPUs\n"]}, {"name": "stderr", "output_type": "stream", "text": ["/usr/local/lib/python3.10/dist-packages/lightning/pytorch/trainer/connectors/logger_connector/logger_connector.py:67: UserWarning: Starting from v1.9.0, `tensorboardX` has been removed as a dependency of the `lightning.pytorch` package, due to potential conflicts with other packages in the ML ecosystem. For this reason, `logger=True` will use `CSVLogger` as the default logger, unless the `tensorboard` or `tensorboardX` packages are found. Please `pip install lightning[extra]` or one of them to enable TensorBoard support by default\n", " warning_cache.warn(\n"]}, {"name": "stdout", "output_type": "stream", "text": ["Found pretrained model, loading...\n"]}, {"name": "stderr", "output_type": "stream", "text": ["Lightning automatically upgraded your loaded checkpoint from v0.9.0 to v2.0.9.post0. To apply the upgrade to your files permanently, run `python -m lightning.pytorch.utilities.upgrade_checkpoint --file saved_models/tutorial12/PixelCNN.ckpt`\n"]}, {"name": "stdout", "output_type": "stream", "text": ["Test bits per dimension: 0.808bpd\n"]}], "source": ["model, result = train_model(c_in=1, c_hidden=64)\n", "test_res = result[\"test\"][0]\n", "print(\n", " \"Test bits per dimension: %4.3fbpd\" % (test_res[\"test_loss\"] if \"test_loss\" in test_res else test_res[\"test_bpd\"])\n", ")"]}, {"cell_type": "markdown", "id": "b190872e", "metadata": {"papermill": {"duration": 0.025107, "end_time": "2023-10-11T16:33:15.163281", "exception": false, "start_time": "2023-10-11T16:33:15.138174", "status": "completed"}, "tags": []}, "source": ["With a test performance of 0.809bpd, the PixelCNN significantly outperforms the normalizing flows we have seen in Tutorial 11.\n", "Considering image modeling as an autoregressive problem simplifies the learning process as predicting\n", "one pixel given the ground truth of all others is much easier than predicting all pixels at once.\n", "In addition, PixelCNN can explicitly predict the pixel values by a discrete softmax while\n", "Normalizing Flows have to learn transformations in continuous latent space.\n", "These two aspects allow the PixelCNN to achieve a notably better performance.\n", "\n", "To fully compare the models, let's also measure the number of parameters of the PixelCNN:"]}, {"cell_type": "code", "execution_count": 20, "id": "06692cba", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:33:15.217137Z", "iopub.status.busy": "2023-10-11T16:33:15.216804Z", "iopub.status.idle": "2023-10-11T16:33:15.223414Z", "shell.execute_reply": "2023-10-11T16:33:15.222521Z"}, "papermill": {"duration": 0.036785, "end_time": "2023-10-11T16:33:15.224982", "exception": false, "start_time": "2023-10-11T16:33:15.188197", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Number of parameters: 852,160\n"]}], "source": ["num_params = sum(np.prod(param.shape) for param in model.parameters())\n", "print(f\"Number of parameters: {num_params:,}\")"]}, {"cell_type": "markdown", "id": "6c29b2dc", "metadata": {"papermill": {"duration": 0.025038, "end_time": "2023-10-11T16:33:15.275381", "exception": false, "start_time": "2023-10-11T16:33:15.250343", "status": "completed"}, "tags": []}, "source": ["Compared to the multi-scale normalizing flows, the PixelCNN has considerably less parameters.\n", "Of course, the number of parameters depend on our hyperparameter choices.\n", "Nevertheless, in general, it can be said that autoregressive models\n", "require considerably less parameters than normalizing flows to reach\n", "good performance, based on the reasons stated above. Still,\n", "autoregressive models are much slower in sampling than normalizing\n", "flows, which limits their possible applications."]}, {"cell_type": "markdown", "id": "650e8e62", "metadata": {"papermill": {"duration": 0.024652, "end_time": "2023-10-11T16:33:15.324881", "exception": false, "start_time": "2023-10-11T16:33:15.300229", "status": "completed"}, "tags": []}, "source": ["## Sampling\n", "\n", "One way of qualitatively analysing generative models is by looking at the actual samples.\n", "Let's therefore use our sampling function to generate a few digits:"]}, {"cell_type": "code", "execution_count": 21, "id": "18b6732a", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:33:15.376913Z", "iopub.status.busy": "2023-10-11T16:33:15.376547Z", "iopub.status.idle": "2023-10-11T16:36:13.221467Z", "shell.execute_reply": "2023-10-11T16:36:13.220786Z"}, "papermill": {"duration": 177.87393, "end_time": "2023-10-11T16:36:13.223466", "exception": false, "start_time": "2023-10-11T16:33:15.349536", "status": "completed"}, "tags": []}, "outputs": [{"name": "stderr", "output_type": "stream", "text": ["Global seed set to 1\n"]}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "41ccb9d6e78546d0ac13c9a76568eb34", "version_major": 2, "version_minor": 0}, "text/plain": [" 0%| | 0/28 [00:00\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T16:36:13.161773\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["L.seed_everything(1)\n", "samples = model.sample(img_shape=(16, 1, 28, 28))\n", "show_imgs(samples.cpu())"]}, {"cell_type": "markdown", "id": "f4b907f6", "metadata": {"papermill": {"duration": 0.020867, "end_time": "2023-10-11T16:36:13.267347", "exception": false, "start_time": "2023-10-11T16:36:13.246480", "status": "completed"}, "tags": []}, "source": ["Most of the samples can be identified as digits, and overall we achieve a better quality than we had in normalizing flows.\n", "This goes along with the lower likelihood we achieved with autoregressive models.\n", "Nevertheless, we also see that there is still place for improvement\n", "as a considerable amount of samples cannot be identified (for example the first row).\n", "Deeper autoregressive models are expected to achieve better quality,\n", "as they can take more context into account for generating the pixels.\n", "\n", "Note that on Google Colab, you might see different results, specifically with a white line at the top.\n", "After some debugging, it seemed that the difference occurs inside the dilated convolution,\n", "as it gives different results for different batch sizes.\n", "However, it is hard to debug this further as it might be a bug of the installed PyTorch version on Google Colab.\n", "\n", "The trained model itself is not restricted to any specific image size.\n", "However, what happens if we actually sample a larger image than we had\n", "seen in our training dataset? Let's try below to sample images of size\n", "$64\\times64$ instead of $28\\times28$:"]}, {"cell_type": "code", "execution_count": 22, "id": "4b949b08", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:36:13.313820Z", "iopub.status.busy": "2023-10-11T16:36:13.313477Z", "iopub.status.idle": "2023-10-11T16:43:24.891241Z", "shell.execute_reply": "2023-10-11T16:43:24.890046Z"}, "papermill": {"duration": 431.60664, "end_time": "2023-10-11T16:43:24.894244", "exception": false, "start_time": "2023-10-11T16:36:13.287604", "status": "completed"}, "tags": []}, "outputs": [{"name": "stderr", "output_type": "stream", "text": ["Global seed set to 1\n"]}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "3c1504bd07f7429992e82357ed0c93c3", "version_major": 2, "version_minor": 0}, "text/plain": [" 0%| | 0/64 [00:00\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T16:43:24.829956\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["L.seed_everything(1)\n", "samples = model.sample(img_shape=(8, 1, 64, 64))\n", "show_imgs(samples.cpu())"]}, {"cell_type": "markdown", "id": "15942ca5", "metadata": {"papermill": {"duration": 0.021495, "end_time": "2023-10-11T16:43:24.942585", "exception": false, "start_time": "2023-10-11T16:43:24.921090", "status": "completed"}, "tags": []}, "source": ["The larger images show that changing the size of the image during testing confuses the model\n", "and generates abstract figures (you can sometimes spot a digit in the upper left corner).\n", "In addition, sampling for images of 64x64 pixels take more than a minute on a GPU.\n", "Clearly, autoregressive models cannot be scaled to large images without changing the sampling procedure such as with [forecasting](https://arxiv.org/abs/2002.09928).\n", "Our implementation is also not the most efficient as many computations can be stored and reused throughout the sampling process.\n", "Nevertheless, the sampling procedure stays sequential which is\n", "inherently slower than parallel generation like done in normalizing\n", "flows."]}, {"cell_type": "markdown", "id": "f75b50b6", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.021261, "end_time": "2023-10-11T16:43:24.985654", "exception": false, "start_time": "2023-10-11T16:43:24.964393", "status": "completed"}, "tags": []}, "source": ["### Autocompletion\n", "\n", "One common application done with autoregressive models is\n", "auto-completing an image. As autoregressive models predict pixels one by\n", "one, we can set the first $N$ pixels to predefined values and check how\n", "the model completes the image. For implementing this, we just need to\n", "skip the iterations in the sampling loop that already have a value\n", "unequals -1. See above in our PyTorch Lightning module for the specific\n", "implementation. In the cell below, we randomly take three images from\n", "the training set, mask about the lower half of the image, and let the\n", "model autocomplete it. To see the diversity of samples, we do this 12\n", "times for each image:"]}, {"cell_type": "code", "execution_count": 23, "id": "6b4a0719", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:43:25.029639Z", "iopub.status.busy": "2023-10-11T16:43:25.029208Z", "iopub.status.idle": "2023-10-11T16:43:30.928842Z", "shell.execute_reply": "2023-10-11T16:43:30.923862Z"}, "papermill": {"duration": 5.923941, "end_time": "2023-10-11T16:43:30.930528", "exception": false, "start_time": "2023-10-11T16:43:25.006587", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Original image and input image to sampling:\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMTc1LjUyMjUgOTcuNTYgXSAvQ29udGVudHMgOSAwIFIgL0Fubm90cyAxMCAwIFIgPj4KZW5kb2JqCjkgMCBvYmoKPDwgL0xlbmd0aCAxMiAwIFIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicVY4xDsIwDEV3n+KfIImDkpQRqBQxFgYOEIVCRItKJXp93A6tGJ7kZ9nf1nX+PlO+xCNOV9KbpZEYRWhhUIQJjCi0ZMQ64uCUs9aJvDbZB+W8NMxaPYjuNCAou8CeFc+T1U6xxyfjhh76IMGjpBdhksiI/1+GZdFWmI/LqrNrYuqgz4z6jYYa+gGxYy7BCmVuZHN0cmVhbQplbmRvYmoKMTIgMCBvYmoKMTQzCmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjMgMCBvYmoKPDwgPj4KZW5kb2JqCjQgMCBvYmoKPDwgL0ExIDw8IC9UeXBlIC9FeHRHU3RhdGUgL0NBIDEgL2NhIDEgPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgL0kxIDEzIDAgUiA+PgplbmRvYmoKMTMgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL1N1YnR5cGUgL0ltYWdlIC9XaWR0aCAyMjQgL0hlaWdodCAxMTYKL0NvbG9yU3BhY2UgWyAvSW5kZXhlZCAvRGV2aWNlUkdCIDcwCij////9/f37+/v19fXv7+/t7e3r6+vj4+Pf39/R0dHPz8/Hx8fDw8PBwcG/v7+5ubmlpaWVlZWTk5OPj4+Li4t9fX1jY2NfX19RUVFPT09HR0c7Ozs1NTUjIyMdHR0bGxsZGRkTExP6+vry8vLw8PDs7Ozo6Ojk5OTY2NjQ0NDCwsK6urq0tLSurq6srKygoKCKioqIiIiAgIBycnJgYGBWVlZQUFBMTExEREQyMjIwMDAsLCwqKioiIiIaGhoUFBQSEhIODg4MDAxcblxuXG4GBgYEBAQAAAApCl0KL0JpdHNQZXJDb21wb25lbnQgOCAvRmlsdGVyIC9GbGF0ZURlY29kZQovRGVjb2RlUGFybXMgPDwgL1ByZWRpY3RvciAxMCAvQ29sb3JzIDEgL0NvbHVtbnMgMjI0IC9CaXRzUGVyQ29tcG9uZW50IDggPj4KL0xlbmd0aCAxNCAwIFIgPj4Kc3RyZWFtCnic7dxrU9pAGMXxRFoLFEUUWyxUUYpoixe8lWJBQOH7f6SenY0dlSRuMs/qZDn/l8Bw+L0Id/AuHM977xtgOwKzHoFZb2mA87fo4h3GCLSwSaDkGIEWNgmUHCPQwiaBkmMEWtgkUHKMQAubBEqOEWhhk0DJMQItbBIoOWYEvEKe522iyMs8oNlsZrL56pinx6Ivo7eMxggMNgnMLnAymRwj3/ev0eL5f1Cv1/uIisXiDXptM3psosd8PRa51dNbxbgtApcH2Gw2fd1XFNyMfdRoNLbQIfL/d47SA5vPxh7JequxsOXHbRFIoDNAHO3B9a2iCsrn835opVLpHqUHXj8bC7bCx0p6jEAC57iWE9RqtV5M5HK5X0g9gQtOiT0mTICSEfgYgVkHBtXr9Ut0pNtG/X5fnXGGgPuBRqORySaBIhH4IueBoX1D6oUggDvIcJNAkQg06TNSD4DtdnsXGW4SKBKBJi0NsFqtJtgkUCQCTXIbWKvVPqCfaDqdJtgkUCQCYxsg4NTxd4qSbRIoEoGxOQ/8goL3Cn+jZJsEikRgdOPxWH0FALg1FPt5UugmgSIRGJ3zwFR3ME82CRSJwOicB3a7XaXrdDrfUfJNAkUiMLw7VCgUFLBcLqfbJFAkAsNzHqi+NQ3cJzQYDNJtEigSgeE5D9xAAK6j1JsEikTgYn/RAQJwBVUqlT2UfJNAkQhczHngLQreq1A/ZxgOh+k2CRSJQHubBEqOEWhhk0DJMQItbBIoObY0f0PtbARmPQKznvPAfz0XsDAKZW5kc3RyZWFtCmVuZG9iagoxNCAwIG9iago1ODYKZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFsgMTEgMCBSIF0gL0NvdW50IDEgPj4KZW5kb2JqCjE1IDAgb2JqCjw8IC9DcmVhdG9yIChNYXRwbG90bGliIHYzLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjguMCkgL0NyZWF0aW9uRGF0ZSAoRDoyMDIzMTAxMTE2NDMyNVopCj4+CmVuZG9iagp4cmVmCjAgMTYKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDE4NDAgMDAwMDAgbiAKMDAwMDAwMDU5NSAwMDAwMCBuIAowMDAwMDAwNjE2IDAwMDAwIG4gCjAwMDAwMDA2NzYgMDAwMDAgbiAKMDAwMDAwMDY5NyAwMDAwMCBuIAowMDAwMDAwNzE4IDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDMzNyAwMDAwMCBuIAowMDAwMDAwNTc1IDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDU1NSAwMDAwMCBuIAowMDAwMDAwNzUwIDAwMDAwIG4gCjAwMDAwMDE4MjAgMDAwMDAgbiAKMDAwMDAwMTkwMCAwMDAwMCBuIAp0cmFpbGVyCjw8IC9TaXplIDE2IC9Sb290IDEgMCBSIC9JbmZvIDE1IDAgUiA+PgpzdGFydHhyZWYKMjA1MQolJUVPRgo=", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T16:43:25.050269\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}, {"name": "stderr", "output_type": "stream", "text": ["Global seed set to 1\n"]}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "2edb8f542ed54952acbb2523f3684bfc", "version_major": 2, "version_minor": 0}, "text/plain": [" 0%| | 0/28 [00:00\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T16:43:27.120153\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}, {"name": "stdout", "output_type": "stream", "text": ["Original image and input image to sampling:\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMTc1LjUyMjUgOTcuNTYgXSAvQ29udGVudHMgOSAwIFIgL0Fubm90cyAxMCAwIFIgPj4KZW5kb2JqCjkgMCBvYmoKPDwgL0xlbmd0aCAxMiAwIFIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicVY4xDsIwDEV3n+KfIImDkpQRqBQxFgYOEIVCRItKJXp93A6tGJ7kZ9nf1nX+PlO+xCNOV9KbpZEYRWhhUIQJjCi0ZMQ64uCUs9aJvDbZB+W8NMxaPYjuNCAou8CeFc+T1U6xxyfjhh76IMGjpBdhksiI/1+GZdFWmI/LqrNrYuqgz4z6jYYa+gGxYy7BCmVuZHN0cmVhbQplbmRvYmoKMTIgMCBvYmoKMTQzCmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjMgMCBvYmoKPDwgPj4KZW5kb2JqCjQgMCBvYmoKPDwgL0ExIDw8IC9UeXBlIC9FeHRHU3RhdGUgL0NBIDEgL2NhIDEgPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgL0kxIDEzIDAgUiA+PgplbmRvYmoKMTMgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL1N1YnR5cGUgL0ltYWdlIC9XaWR0aCAyMjQgL0hlaWdodCAxMTYKL0NvbG9yU3BhY2UgWyAvSW5kZXhlZCAvRGV2aWNlUkdCIDY4Cij////7+/v5+fn39/fx8fHp6enl5eXd3d3Z2dnX19fV1dXBwcG/v7+9vb27u7u1tbWtra2hoaGVlZWRkZF1dXVzc3NxcXFnZ2dXV1dTU1NRUVFPT09LS0s7OzsbGxsXFxcVFRUTExNcclxyXHIDAwP+/v78/Pzy8vLq6uro6Ojm5ubW1tbMzMzGxsa8vLy6urq4uLi0tLSysrKsrKyYmJiAgIBqampiYmJaWlpMTExCQkJAQEA6OjowMDAgICAYGBgODg5cblxuXG4ICAgGBgYCAgIAAAApCl0KL0JpdHNQZXJDb21wb25lbnQgOCAvRmlsdGVyIC9GbGF0ZURlY29kZQovRGVjb2RlUGFybXMgPDwgL1ByZWRpY3RvciAxMCAvQ29sb3JzIDEgL0NvbHVtbnMgMjI0IC9CaXRzUGVyQ29tcG9uZW50IDggPj4KL0xlbmd0aCAxNCAwIFIgPj4Kc3RyZWFtCnic7ZrZktJAGEZR0VFZBBVFUZYxigtR1AEURBjl/Z/JryuxBkJIL3Q3lfCdy06nTs5N8leS0lXBKZ36AlzDwLzDwLxzNoEbH1ydQMZAB06vMgY6cHqVMdCB06vMMPA3qFQqP4G+00BWiWQ6ZzFQ5mTgQaeBzG/ger3+DIIg+Ab0nTqydSQLIpnOVTIwU8rALKeOzHtgH4RhGEQ8B/pOVVns2pJpuBiYLS1u4FcQC4fD4R+g71SV7bqGei4GKkkZmOpUlZ0k8AGIpdPpVEe40Q7cdRnKGJgpNXOqyhioIjMMfAc6nY6ZU1X2PzByGcoYmC4tfKAYETWFG+NAIxcDs6UMzHKqyrwHPgO3AJxPgLFTVRa7AiMXA9OdBQ7s9/ufQDw6iTfbxk65zAYMTMBAdadcZgPdwHa7HdwQB74ELyJeAUWnXGYDBiZgYN4DR6PRVuAj0Gg03oJ45Q7ASuYEwECbMDDBbuBB7oFfIMspl9mAgQkYWKzA9+DDHuVyuQQuQOqsykCbMDDBOQVWq9VLsL9nPp/fBWLPcrk86JTLbMDABOcU2Gq10vcMBoOPQER2u92DTrnMBgxMwMBNcQLr9fr+TUashGEo4l6DLKdcZgMGJih8YLPZ3BrVfoBerycO/AWLxeILwIGHQOaUy2zAwAQMzHvgarXa+n9TUKvVHoPv4GbxKZA55TIbMDABA/MeCK7BfRCkIV7dTyaTN0DmVJIdDQP3KXygQDwAx+PxVtltMJvNxLdQRaey7CgYmA4DFZzKsqMwDLTh9CpjoAOnVxkDHTi9yhjowOlVxkAHTq8yBjpwepUx0IHTq4yBDpxeZQx04PQqY6ADp1cZAx04vcrOJrCwMDDvMDDvMDDv/ANQi27OCmVuZHN0cmVhbQplbmRvYmoKMTQgMCBvYmoKNjU5CmVuZG9iagoyIDAgb2JqCjw8IC9UeXBlIC9QYWdlcyAvS2lkcyBbIDExIDAgUiBdIC9Db3VudCAxID4+CmVuZG9iagoxNSAwIG9iago8PCAvQ3JlYXRvciAoTWF0cGxvdGxpYiB2My44LjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcpCi9Qcm9kdWNlciAoTWF0cGxvdGxpYiBwZGYgYmFja2VuZCB2My44LjApIC9DcmVhdGlvbkRhdGUgKEQ6MjAyMzEwMTExNjQzMjdaKQo+PgplbmRvYmoKeHJlZgowIDE2CjAwMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAwMDAxNiAwMDAwMCBuIAowMDAwMDAxOTEwIDAwMDAwIG4gCjAwMDAwMDA1OTUgMDAwMDAgbiAKMDAwMDAwMDYxNiAwMDAwMCBuIAowMDAwMDAwNjc2IDAwMDAwIG4gCjAwMDAwMDA2OTcgMDAwMDAgbiAKMDAwMDAwMDcxOCAwMDAwMCBuIAowMDAwMDAwMDY1IDAwMDAwIG4gCjAwMDAwMDAzMzcgMDAwMDAgbiAKMDAwMDAwMDU3NSAwMDAwMCBuIAowMDAwMDAwMjA4IDAwMDAwIG4gCjAwMDAwMDA1NTUgMDAwMDAgbiAKMDAwMDAwMDc1MCAwMDAwMCBuIAowMDAwMDAxODkwIDAwMDAwIG4gCjAwMDAwMDE5NzAgMDAwMDAgbiAKdHJhaWxlcgo8PCAvU2l6ZSAxNiAvUm9vdCAxIDAgUiAvSW5mbyAxNSAwIFIgPj4Kc3RhcnR4cmVmCjIxMjEKJSVFT0YK", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T16:43:27.187600\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}, {"name": "stderr", "output_type": "stream", "text": ["Global seed set to 1\n"]}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "d1e2d868d5cb4fc3acdfc43ba118d35b", "version_major": 2, "version_minor": 0}, "text/plain": [" 0%| | 0/28 [00:00\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T16:43:29.009993\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}, {"name": "stdout", "output_type": "stream", "text": ["Original image and input image to sampling:\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgMTc1LjUyMjUgOTcuNTYgXSAvQ29udGVudHMgOSAwIFIgL0Fubm90cyAxMCAwIFIgPj4KZW5kb2JqCjkgMCBvYmoKPDwgL0xlbmd0aCAxMiAwIFIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicVY4xDsIwDEV3n+KfIImDkpQRqBQxFgYOEIVCRItKJXp93A6tGJ7kZ9nf1nX+PlO+xCNOV9KbpZEYRWhhUIQJjCi0ZMQ64uCUs9aJvDbZB+W8NMxaPYjuNCAou8CeFc+T1U6xxyfjhh76IMGjpBdhksiI/1+GZdFWmI/LqrNrYuqgz4z6jYYa+gGxYy7BCmVuZHN0cmVhbQplbmRvYmoKMTIgMCBvYmoKMTQzCmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjMgMCBvYmoKPDwgPj4KZW5kb2JqCjQgMCBvYmoKPDwgL0ExIDw8IC9UeXBlIC9FeHRHU3RhdGUgL0NBIDEgL2NhIDEgPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgL0kxIDEzIDAgUiA+PgplbmRvYmoKMTMgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL1N1YnR5cGUgL0ltYWdlIC9XaWR0aCAyMjQgL0hlaWdodCAxMTYKL0NvbG9yU3BhY2UgWyAvSW5kZXhlZCAvRGV2aWNlUkdCIDc4Cij////9/f35+fn39/fz8/Pv7+/t7e3r6+vp6enh4eHb29vT09PJycm/v7+7u7upqamnp6eTk5ORkZGPj4+NjY2JiYmFhYV3d3dxcXFdXV1ZWVlXV1dVVVVDQ0M/Pz8xMTErKysjIyMfHx8dHR0bGxsPDw8LCwsHBwcFBQX+/v78/Pz09PTs7Ozo6Ojm5ube3t7a2trOzs6+vr66urqurq6oqKikpKScnJyYmJiWlpaAgIB2dnZqampkZGRgYGBeXl5UVFRQUFBAQEA+Pj44ODgmJiYgICAWFhYSEhIQEBAODg4GBgYEBAQCAgIAAAApCl0KL0JpdHNQZXJDb21wb25lbnQgOCAvRmlsdGVyIC9GbGF0ZURlY29kZQovRGVjb2RlUGFybXMgPDwgL1ByZWRpY3RvciAxMCAvQ29sb3JzIDEgL0NvbHVtbnMgMjI0IC9CaXRzUGVyQ29tcG9uZW50IDggPj4KL0xlbmd0aCAxNCAwIFIgPj4Kc3RyZWFtCnic7Zxrc9JAGEZBQEVUEBTxVlAEBLl6t0otIIhB+f8/x2cnmwEqNJvhXSvLcz61YScnZ6bsdpOB2KnjxK76AmzDwEOHgYfO0QQu/wWnVyBjoAUnAyVlDLTgZKCkjIEWnAyUlDHQgpOBkjIGWnAyUFLGQAtOBkrKGGjByUBJWdTAR6Df78dBs9l8B86A53kRnKYy7er7rubKFUXGwO1SBl7m/D8Da7XaB5AALR949U+tTCbzDBg6Q2U1X7bpiq9cGQMXA48uEH/96vQ3wXA4/ALWAsFrMJlMRALPfJl2DX3XBdnEUMZABroQ+BJgPVKn/g5w5Dk493kLtLfX6/0E+wX6rv7Ktdzl6l3mYuBRBX4FeBM8AL/A5ovqSKlUugEw5jPYL1AKBq7BwIMOnE6nbdDtdtVcs31MvV6/BbCneQ/CnAwUgYEBzgcWCgW9Dds9ZjweqzGYZB6DMCcDRWBgQDabVRePpXD7679BLpdTYwaDwSsQ5mSgCAwMcDpQ3b9Lp9Pq4nee6CHQe7TRaGTiZKAIDFQ4H/gD6Iv/ewGfz+f3wTWgxxg6GSgCAwOwDqr7ymvboGq1+gTEYrH4invA0MlAERgY4HxgsVhUE0gymXyj6XQ66kgqlVI3KG4D/MbAJQNlMQ5cLBafQLvd1qtdK5FI3AX6CWTH7/0GDJ0MFIGBAc4HajzP049Zz09OTvRB9b/qdYD6F8DQyUARGHgB5wO38hSoaSefz0dwMlAEBppwB6i9Et6YEZwMFIGBJrgdOJvN1BqoJhkGbjgZKAIDQ6lUKsEOkYEbTgaKwMBQGLjLyUAR9g9sNBofgfpMT7lcjuBkoAgMtOdkoKSMgRacDJSUMdCCk4GSsqP5GmpnYeChw8BDx/nAPyUA2gAKZW5kc3RyZWFtCmVuZG9iagoxNCAwIG9iago2NjcKZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFsgMTEgMCBSIF0gL0NvdW50IDEgPj4KZW5kb2JqCjE1IDAgb2JqCjw8IC9DcmVhdG9yIChNYXRwbG90bGliIHYzLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjguMCkgL0NyZWF0aW9uRGF0ZSAoRDoyMDIzMTAxMTE2NDMyOVopCj4+CmVuZG9iagp4cmVmCjAgMTYKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDE5NDIgMDAwMDAgbiAKMDAwMDAwMDU5NSAwMDAwMCBuIAowMDAwMDAwNjE2IDAwMDAwIG4gCjAwMDAwMDA2NzYgMDAwMDAgbiAKMDAwMDAwMDY5NyAwMDAwMCBuIAowMDAwMDAwNzE4IDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDMzNyAwMDAwMCBuIAowMDAwMDAwNTc1IDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDU1NSAwMDAwMCBuIAowMDAwMDAwNzUwIDAwMDAwIG4gCjAwMDAwMDE5MjIgMDAwMDAgbiAKMDAwMDAwMjAwMiAwMDAwMCBuIAp0cmFpbGVyCjw8IC9TaXplIDE2IC9Sb290IDEgMCBSIC9JbmZvIDE1IDAgUiA+PgpzdGFydHhyZWYKMjE1MwolJUVPRgo=", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T16:43:29.073236\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}, {"name": "stderr", "output_type": "stream", "text": ["Global seed set to 1\n"]}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "1efb8e0f64c14d1db0009992b98008c5", "version_major": 2, "version_minor": 0}, "text/plain": [" 0%| | 0/28 [00:00\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T16:43:30.870659\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["def autocomplete_image(img):\n", " # Remove lower half of the image\n", " img_init = img.clone()\n", " img_init[:, 10:, :] = -1\n", " print(\"Original image and input image to sampling:\")\n", " show_imgs([img, img_init])\n", " # Generate 12 example completions\n", " img_init = img_init.unsqueeze(dim=0).expand(12, -1, -1, -1).to(device)\n", " L.seed_everything(1)\n", " img_generated = model.sample(img_init.shape, img_init)\n", " print(\"Autocompletion samples:\")\n", " show_imgs(img_generated)\n", "\n", "\n", "for i in range(1, 4):\n", " img = train_set[i][0]\n", " autocomplete_image(img)"]}, {"cell_type": "markdown", "id": "ff51c9a7", "metadata": {"papermill": {"duration": 0.023729, "end_time": "2023-10-11T16:43:30.979962", "exception": false, "start_time": "2023-10-11T16:43:30.956233", "status": "completed"}, "tags": []}, "source": ["For the first two digits (7 and 6), we see that the 12 samples all\n", "result in a shape which resemble the original digit. Nevertheless, there\n", "are some style difference in writing the 7, and some deformed sixes in\n", "the samples. When autocompleting the 9 below, we see that the model can\n", "fit multiple digits to it. We obtain diverse samples from 0, 3, 8 and 9.\n", "This shows that despite having no latent space, we can still obtain\n", "diverse samples from an autoregressive model."]}, {"cell_type": "markdown", "id": "030bf4ea", "metadata": {"papermill": {"duration": 0.025649, "end_time": "2023-10-11T16:43:31.030490", "exception": false, "start_time": "2023-10-11T16:43:31.004841", "status": "completed"}, "tags": []}, "source": ["### Visualization of the predictive distribution (softmax)\n", "\n", "Autoregressive models use a softmax over 256 values to predict the next pixel.\n", "This gives the model a large flexibility as the probabilities for each pixel value can be learned independently if necessary.\n", "However, the values are actually not independent because the values 32 and 33 are much closer than 32 and 255.\n", "In the following, we visualize the softmax distribution that the model predicts to gain insights how it has learned the relationships of close-by pixels.\n", "\n", "To do this, we first run the model on a batch of images and store the output softmax distributions:"]}, {"cell_type": "code", "execution_count": 24, "id": "55d2964b", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:43:31.080165Z", "iopub.status.busy": "2023-10-11T16:43:31.079886Z", "iopub.status.idle": "2023-10-11T16:43:31.175788Z", "shell.execute_reply": "2023-10-11T16:43:31.174763Z"}, "papermill": {"duration": 0.122471, "end_time": "2023-10-11T16:43:31.177292", "exception": false, "start_time": "2023-10-11T16:43:31.054821", "status": "completed"}, "tags": []}, "outputs": [], "source": ["det_loader = data.DataLoader(train_set, batch_size=128, shuffle=False, drop_last=False)\n", "imgs, _ = next(iter(det_loader))\n", "imgs = imgs.to(device)\n", "with torch.no_grad():\n", " out = model(imgs)\n", " out = F.softmax(out, dim=1)\n", " mean_out = out.mean(dim=[0, 2, 3, 4]).cpu().numpy()\n", " out = out.cpu().numpy()"]}, {"cell_type": "markdown", "id": "94e3f1cb", "metadata": {"papermill": {"duration": 0.02434, "end_time": "2023-10-11T16:43:31.226172", "exception": false, "start_time": "2023-10-11T16:43:31.201832", "status": "completed"}, "tags": []}, "source": ["Before diving into the model, let's visualize the distribution of the pixel values in the whole dataset:"]}, {"cell_type": "code", "execution_count": 25, "id": "7f6ac797", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:43:31.275410Z", "iopub.status.busy": "2023-10-11T16:43:31.275133Z", "iopub.status.idle": "2023-10-11T16:43:32.705125Z", "shell.execute_reply": "2023-10-11T16:43:32.704222Z"}, "papermill": {"duration": 1.456515, "end_time": "2023-10-11T16:43:32.706522", "exception": false, "start_time": "2023-10-11T16:43:31.250007", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgNDA3LjAyIDMwMC42NTI2MjUgXSAvQ29udGVudHMgOSAwIFIgL0Fubm90cyAxMCAwIFIgPj4KZW5kb2JqCjkgMCBvYmoKPDwgL0xlbmd0aCAxMiAwIFIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnic3V1Nj1xHcrz3r+jj7kHF+v44SpZNQDdJhH0wfFjQ3LUEUoZW2PXfd2bV9MssdpRBL5c2qgQQmsmYjunI6fe6oiLf61ffvvvrT2/f/fD6m/s//Hh7Jd+9/e3m7j/Tvz/d7f1n+vdfd3d/Tf/+dLP03YdbtMVYT1++f3wZrDU5+ewT1ez87X/cbn+8vfqaHv7b3ZrmSo7FplqfvonNupZtqfc/829+Pf3A7X/66dstelPuvpgQLf/WD7fQmqleld5fpRZMfDy58Thd6U/31/tHhCEV4+ixORtH///zu/u/3H+5v/rasyp3/47+/Uz/RqeklzfqZaqmBV/8/PxUVf/224+37++/PoitcYn+Cg/u/u3rl+rtV/oL2ftXlqCUTOxknbCZWB2xvf1w++bN7dU/ubtz9zd/7H+8N/9++9f77+zv7/92f/Pd7R/f3L7vv+/vp9ZFa6KL9Nef5Ory5+t1IbywEX/8BME5fjnF3jtTW070StSKdfnzFXvnTGC2kqJ1n6DY+frlJAcbTCg12TAfc6r8d5DcvKnM5nMJ9VMkN/8FJddoarbRz39lXf58yaFEE5itVFvSJ0j2KX85yZ2pVlNsq02dU6X0t+oshlhIarLJtRJSfpFIwozNlaSTPNH7O/f7+5ufb9m0VlNILmU/OsAnMgJcMIn+y5kl36PxzrYgJIVkMEvoP0yN+pV/jS3lzo18+dJF46wPkR9HDKX1p/H2w50Yvvr23c9/+Oe//PiHX3776sNPv/zlt/u3/3n//sv02yV6QdXqUpCGq9rndNwla5KvpZQavf8/abnfoeV8prakhJTLukFqn9Ny7zIdK8WG1CItW6TlrvQj+0v03H16z0NtvYP/Dz2v1J4Yc26q51L72xcG3PRKbxw1+RYttevzXuek0oZqa6JOr1tuHy3/X/UqDGW+VRuad4n+UCaOBa4PiRa8+bHEjX1FbFKXvvj5+/TzP7y+feJP8upZFqNf5VQbvfQdvUQq/WkysfSHfYS8V4ivtDahl2V/41PrWl2mVfU3e/am67TNtufeOBNLX4A+9eZC6CvCSnaPjnUmXd24M6SSSs09d4ak0FtMzM+duZBSqUfWJ/fSr8Gkqxt3hp5woGMgPHcm0td9ufjUmQspfJ7PpbSXfg0mXd24M6TSpkin5afOJFoC1Pzxi+m9AHRazznxq+S94tHVjfvCIkMOHx8x1JdsUszl6Sh7rxCyDclFcpuPdnUmXd24M6zSVffxEUOdKaa62J6OsvcKKc3kkoP0qzPp6sadYZXPXWnG1uCejrD3CimZVYb+zjxYdGXjjpBCWlg/rVY+3IqltYgL4eMVznuF0HuPpTOsLS+9Gky6um9nWGWkk8LHaxXqDK1FXCMf/tyZC6nBhJwdmcb3iklXN+4Mq4z0VvLcGbYRpdjnM68gvItRqQnp0a/OpKsbd4ZVQldA7yx0GkGuQBB9jhWmM868XSX0BCWZ6KEnEESv5ITpjPUdq3TQE9A7TG7QEwiiz7XCdMgZmFRW6AlKMS1DTyAIdyN699KtwSO1jbtCChP0A3Sm8B75gQugs0rKKbvy0qnBo6sb94VFQj9Q6YEN+gFBqAd0xDSXH+0afkBV9+1MVwn9QHWmZOgHBCHfGPmr+OhXZ9LVjTvDKp+7EowN0A8IQu9CrdKrpDx6NVU27ggp9NAP8ArNQj8gCHmA5nOr7aVXg0lXN+5MpJ+FfqAmOn9CPyCIPpMI0yHnF1YJ/UDNpgboBwTR61thOmPV21VCP1CrcTglECQ18tmV5D361Zl0dePOsEroByrpwxmBINkaso0l5Ee/Rkagqht3hlTijKDRz+KMQJDMj661lZd+DSZd3bczrBJnBPR1wxmBIMVywhR7TiJMurpxZ0glzgha4OAceIILyJ53XmJf/QuPrm7cFxYJPUGj9SvOCATJyZTisvWPdnUmXd24M6wSeoJG+nBGIEimtV3LdFw9+jU8gapu3BlW+dyVTKt76AkEofOttSV0TzRYdGXjjpBCnBG0ajzOCATRu3TCdMbeHavEGUFrJuGMQBCdIAnTGblSVwk9AS3WTMEhgYKm3lxcZ/RmyIS2wFl2hNAXKChRuVmrejacgSrv3B3WCa2B42lpnBUoKCU6yQRb3dW0TqbLO3eHdOK4gN54TcJ5gYL0xp0iO2M/r+vEkQF56sfY+XN3LkgbJUV2hn/qOnF04Cw/eeQTBCn06BZLD68VlS7v3Bt+htAqOFv5QhLkFRRUoqHfVMPVssdA0aO6c29YJTQLzjaTcYKgIFJYXIo1Xi3rZLq8c3dY53NnnDUNpwgK0mbyhegMf9k14iCBnrHxOElQEEnjlV6Lj4a9ZAmqvHN3vHE4THAu8GQvcg4Kml43QnbKa4d1YvPgIk/3QvMgUC4mezr1xKtpnUyXd+4O68TuwWWe8YXuQaAUjKOib1fThntQ5Z27wzqxe3DFeJwsKGjuzkV2SndIJw4XnKsm4XRBQYm/osWgezRtkOnyzt0hGThgcK6ZihMGBU3eSsgO8VasE4cMji/9gymDIPTW5HwqLjxa9pIzqPLGvekysXvwnqd/oXsQaHrHErJD3rG6TuwffOAJYOgfBNIXfymyM64JGzpBZ/gKZuwfBNKjNS9EZ0zbdI04dHCenjVOHRSkhwEU2RkzAl0nDh4c6Yg4eVBQ5nvQOL4vxntNpss7d4d1Yv/gK88CQ/8gUOJLv1OI7WpaJ9PlnbvDOrF/8I1ngqF/EGjuzkV2SndYJ/YPwRm/SB8EmnZJheyQXVLWuUgfgjdxkT4INHVHyE7pDulcpA8hmLJIHwRK3riQfEmPpg0yXd65O6RzkT6ExLPCyD9cyLQGFKpD1oBdJvYPIfO0MPQPAqVsaiqxpqtnnUyXd+4O68T+IRSeGIb+QSA9zKbIzphxGzpBZ6qpi/xBID1M/EJ0xnxx17jIHyJ5gEX+INDkrITsEHdFOu0if4iOp4ahfxCoJfoqlOhemvZCpss7d4d1Yv9AYvIifxAo8503ve+DkIpMl3fuDuvE/iEGnh+G/kGgaZdUyA7ZJe06sX+IybhF/iBQrrwoHhvswqWqO/eGVC7Sh5jpodg9CJRoYZNt6rdnUWS6vHN3SOcifYg8KYHdg0DTlIWQHTJlwToX6UOsPFWM3MOF6NF0RXXGxPqQid1DsjxXDN2DQNP6WMgOWR93ndg9JMeTxdA9CDRlnkJ2SObZdYLOkJBF+iDQlOkNokPyPNa4SB9SNHaRPgg07eQI2SE7OaxzkT7Qe3FYpA8Cza+bi+yU1w7rxO4hZZ4ohu5BoGmPVMgO2SPtOrF7oCdfF+mDQNMaUMgOWQN2ndg9pEZPHrsHgabUU8gOST1Z5yJ9IIVhkT4INO2SCtkhu6Ssc5E+8C75In0QaLpuRsgOuW6mBwXYP9A6t+H04UKmPQuhOmTPosvE/oEUukX6IBDZqFJrrtKz4R9UeefusE7sH2jFEhfpg0AxGW+jLf5qWifT5Z27wzpBZ7Ipi/RBoMrrZVr8uathU2nnrpDGRfqQq7GL9EGgaR5QyA6ZByQZbZE+5MYzxdA/CFSK8c4F3/crhEyXd+4O68T+oVieKYb+QaBpgl3IDplg7zqxfyiOp4qhfxCob/7RWi9cTetkurxzd1gn9g8lGLtIHwSapiyE7JApC9a5yB/4Y6QW+YNAMZia6WVSHk0bZLq8c3dI5yJ/KMnkRf4g0OSuhOwQd8U6F/lDyTxdjPzDhUzZjFAdks10mdg/lMqzxdA/CDQ5TyE7xHl2ndg/lMazxdA/CDR5TyE7xHt2nc+dqfSTi/xBoGm3axAdstPFGhf5Q3WmLfIHgaZET8gOSfRY5yJ/qIEHJaB/EGhy5EJ2iCPvOrF/IHcdF/mDQFPeKWSH5J1dJ/YPNfFUMfQPAk3+QcgO8Q9dJ/YPla8Nwv5BoOmdXMgOeSdnnYv8oVbiwf5BoCkVFrJDUmHWucgfajNpkT8INF2TJmSHXJPGOhf5Q7M8XYz8w4VMqadQHZJ6dpnYPzTPE1rQPwgUaOkXXj42U5Hp8s7dYZ3YPzT+PCnsHwSaVjtCdshqp+sEnYkmL/IHgabcahAdklmxxkX+0Oi9eJE/CJQiSaPHt0fDBpku79wdHqXB/qEVniqG/kGgyV0J2SHuquvE/qFVniqG/kGgadZWyA6Zte06sX9ozZRF/iDQfGRdZKccWawT+gdvLXkA6B8UFGu/UYPzV9M6mS7v252uE+cP3nrjcf6gIG2jFNkZ7qrrxPmDt8EknD8oSN/8TpGdcU+8rhPnD7SK4/li4B8EKYWvTXN9KayodHnn3rBM6B+8zTxdjPyDgnSAp8jOyPWGTugfvC08XYz8g4L0lroiO2OnfegEnakm4fxBQXrj5oXojL2crhHnD942U3H+oCC9nFFkZ6xyuk6cP3jneKoY+QcFaZOpyM7wnkMn9A/esUOC/kFBuZrSWulb6YpMl3fuDuuE/sG7wHPFyD8oaDqyhOyQI6vrxP7BRdNw/qAgbRQU2SH+gXXi/ME7etY4f1CQjvAU2RnJXteJ8wfviok4f1CQHpBUZGfMTXadOH/wrvJ8MfIPFzI5T6E6xHl2mdg/uMbTxdA/CDQ5TyE7xHl2ndg/eMfTxdA/CDQfVxfZIcdV1wk6403C+YOC+geF0HqvfyL9C9NU27kvpBInEN6Tr8YJhIKm9yohO+S9inQWnEB4z5kudhACTc5TyA5xn10ndhA+81wxdBAC6ctkFNkZV88MndhBkI6MEwgF6aEBRXbGLMHQiR2EZxeAHYRA85F1kZ1yZJHORQIReD8POwiB9C0aFNkZd27oOhcJRHAmLhIIgfStExXZGXdU7DoXCUTwPGGMHMSFTLvIQnXILnKXiR1ECDxfDB2EQPMr5yI75ZXDOrGDCInni6GDEEiPDiiyMyYKhk7QmWziIoEQaH7ddKJTXjOkcZFAhGLKIoEQaHofF7JD3sdZ5yKBCI3niqF/EEhfKq3IzriCeujE/iFaniuG/kGg+XxzkR1yvuk6sX+IjieLoX8QSF/qoMjOuAJi6MT+gcTURQIh0JTPCNkh+QzrXCQQMdLRgf2DQPqDlBXZGZ+v3HUuEojId8bG/kGg+ci6yE45skjnIoGImSeMkX+4EH3zbEV1xj21h0zsH+ivXxcJhED6QnJFdsb15UMn9g+x8Xwx9A8CJSo3a1XTRgKhyjt3h3U+dybRwxYJhEDTfs4gOmQvhzUu8gf6u5dF/iDQlOkJ2SGZHunMi/yBHFJb5A8CTROTQnbIxGTXif0DreP8In8QaHLlQnaIM+86sX9IiSeLoX8QaDobC9khZ+OuE/uHlOncgf2DQJN/ELJD/APrXOQPqRq7yB8EmuZQhOyQORTWucgf+NyxyB8E4g99qC4P/yBkurxzd0jnIn8gF5Bw/nAh036gUB2yH9hlYv+QHc8XQ/8g0JTOCNkh6UzXif1DDjxfDP2DQPqjTRXZGZ94OnSCzkQTFvmDQNNRNYhOOaJI4yJ/IA+QF/mDQJOzErJD3BXrXOQP9KzbIn8QSN9QUpGdcZ/JoRP7h1x5rhj6B4Hm7lxkp3SHdWL/kBtPFkP/INDkH4TsEP/QdWL/UKwpi/xBoGhNazblcDVtXAGhyht3h3Uu8ofiSCH2DwJNu8hCdsguMutc5A8lGL/IHwSix6fQbLyaNsh0eefukM5F/lAizxcj/3Ah9OiUcint0bJBpcs794ZlYv9QEk8XQ/8gULX8icI1pqtnnUyXd+4O68T+oRSeLob+QaDJPwjZIf6h6wSdqSYs8geBpt2uQXTIThdrXOQPpZm8yB8Emq4RFrJDrhEmnWmRP/A5Y5E/CDR35yI7pDtdJ/YPld019g8CTWmwkB2SBned2D/UwHPF0D8IpG9Ir8jOuE/90In9Q40mL/IHgaYddiE7ZIeddS7yh0oKF/mDQNN5R8hOOe+QzkX+UHkyFPsHgchQxRJjco+mDTJd3rk7pHORP9TK88XIP1xI8Ybsd3P50bKXOzCp8s69YZnYP9TG08XQPwg0+XIhO8SXd53YP/RvsH8QiKRFG/y4nlHIdHnj7nSdoDPe+EX+INDkHwbRIf6BNS7yh8Z7Dtg/CDR5TiE7xHOyzkX+0CJPFUP/INA0oSNkh0zodJ3YP7TMU8XQPwg0pXpCdkiy13Vi/9AKzxVD/yDQdI8hITvkHkNdJ/YPrZq8yB8Emo+si+yUI4t0LvKH1kxd5A8CTamwkB2SCrNOnD8E64zD+YOC9EdaKbIzPumq68T5Q7Ce54uBfxCETjTF2tIPIEWlyzv3hmVC/xBs4Oli5B8UpA8gRXbGcTV0Qv8QbOTpYuQfFJQrf/JVv2eO4lLVnXvDKkFfsvE4fVCQvqXkC9EZd5nsGnH6EGwxCacPCtLWUpGd4ThZZ8TpQ7CVZ4qRe1CQ3tBSZGfscw2d0D0EZ3mmGLkHBelIXJGdkZQPndA9BOd4qhi5BwVVenwuoeSraZ1Ml3fuDuuE7iE4bxJOHxSkP5RIkZ3xWUVdJ04fggum4vRBQfoGBIrsjPsSdJ04faA3a3rW2D0IpG8oqcjOuM9k14nTh+AyTxcj93Ah00pHqA5Z7XSZ2D24wrPF0D0IpD9iUJGd8cmDQyd2D67ybDF0DwLp0VFFdsZE6dD53BlvjcPpg4IKl10eK51BpEsbd4U14vQheGciTh8UpMcFFNkZUwRdJ04fgvc8Uwz9g0D6sipFdsbVVkMn9g8+8Ewx9A8C6dBXkZ2RBQ+d2D/4xFPF0D8IpDfSFdkZ++tDJ/YPPpuE0wcF6UtkFNkZV850nTh9IBNJKznsHwTSt1lSZGfcfanrXKQPvvG9jaF/EEgPcSmyM2a7us5F+hAsTxcj/3Ah81nnojrkrNNlYv8QHM8WQ/8gUGErnnL/4FdFpss7d4d1Yv8QPM8WQ/8gUImGflMNV886l6ru3BtWCfoSjVukDwJN65xBdMgahzUu0oeQTFykDwKVQOfkFMujX4NLVXfuTTJhkT2EzPPE0DsINJ2JheyUMzHrxN4hFJ4nht5BIP0hBorsjM82GDqxdwiNJ4qhdxBI58DCdUYO3FVi5xB7Lgedg0DTDqmQHbJDyjoXyUN0piySB4H0jaEV2Rn3i+46F8lDpDedRfIg0JSWC9khaTnrXCQPMfJcMXIOF6I/dEdRnfFZPEMmdg4x8VQxdA4CTZMEQnbIJEHXiZ1DzDxVDJ2DQJWMpqWDKV5N62S6vHN3WCfoTDV2kTwINB9VneiUI4o0LpKHyGtc7B0EmjIZITskl2Gdi+QhWZ4mhu5BIH0JsCI748rgoRO7h+R4mhi6B4FK5O10H8rVtDG5pMo7d4d1YvfAMxKL5EEgfRsYRXbG3WGGTuwfUjRxkTwINPkHITvEP7DORfJA3jovkgeBCl/Saf1Y5wiZLu/cHdK5SB5SNm2RPAik97uE64z9Lla5yB1S5bli5B4uZErKheqQpLzLxO4hNZ4qhu5BoOqMt+3xwrnIdHnn7rBO7B7o718WuYNAhRbEoTifrqaN4EGVN+5O1wk6441dJA8CTVfLDKJDrpRhjYvkIQcTFsmDQNM+l5Adss9FOv0ie8iRp4mhexBIJ3nCdUaS11Vi70DHRl0kDwJV4k/B5XS1rJPp8s7dYZ3YO+TC08TQOwhEb0mp2tg/NkWR6fLO3WGd2DvQ+j8ssgeBSjOeXIItV9M6mS7v3B3SucgecjN5kT0IxLeTdLGOTE/IdHnn7jRejkDvUHgaAHsHgegQajGnEB5NG2S6vHF3WOcieyiep4qRe7iQad5NqA6Zd+sysXsg6xgX2YNAfK/JkoPq2cgeVHnn7rBO7B7ozFEW2YNA016gkB2yF9h1gs4kUofdg0CVH1vrWAYOIl3auSt8eQJ2D+wAFtmDQJX3u4KP/tGwQabLO3eHdC6yh1J5lhi6B4Gcd4boQvCPrg22qb5zf1gpdhB0Ti2L9EEgR2ub1ug35KtvnW2q79wfVoo9BK1W7CJ/EMjVwPc6qa1dfetsU33j/nSl2EVUb8IigRDI0Xt4LbQm9FffOttU37k/pHSRQdArIC0yCIFc4j6k4dCFTJev7rz62rNwf//u7ujsTprc/fX91bfv/vrT23c/vP7m/va320cd/PBSaLRWSD73/cXbj7fA9zHy0889Sh/95BPf00PVj8ljl3zf3/4brljuhQplbmRzdHJlYW0KZW5kb2JqCjEyIDAgb2JqCjYxMjkKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTcgMCBvYmoKPDwgL0xlbmd0aCAzOTUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPVJLbsVACNvnFFyg0vCbz3lSVd28+29rQ1KpKryJMcYwfcqQueVLXRJxhcm3Xq5bPKZ8LltamXmIu4uNJT623JfuIbZddC6xOB1H8gsynSpEqM2q0aH4QpaFB5BO8KELwn05/uMvgMHXsA244T0yQbAk5ilCxm5RGZoSQRFh55EVqKRQn1nC31Hu6/cyBWpvjKULYxz0CbQFQm1IxALqQABE7JRUrZCOZyQTvxXdZ2IcYOfRsgGuGVRElnvsx4ipzqiMvETEPk9N+iiWTC1Wxm5TGV/8lIzUfHQFKqk08pTy0FWz0AtYiXkS9jn8SPjn1mwhhjpu1vKJ5R8zxTISzmBLOWChl+NH4NtZdRGuHbm4znSBH5XWcEy0637I9U/+dNtazXW8cgiiQOVNQfC7Dq5GscTEMj6djSl6oiywGpq8RjPBYRAR1vfDyAMa/XK8EDSnayK0WCKbtWJEjYpscz29BNZM78U51sMTwmzvndahsjMzKiGC2rqGautAdrO+83C2nz8z6KJtCmVuZHN0cmVhbQplbmRvYmoKMTggMCBvYmoKPDwgL0xlbmd0aCAyNDkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTVFJigMwDLvnFfpAIV6TvKdDmUPn/9fKDoU5BAmvkpOWmFgLDzGEHyw9+JEhczf9G36i2btZepLJ2f+Y5yJTUfhSqC5iQl2IG8+hEfA9oWsSWbG98Tkso5lzvgcfhbgEM6EBY31JMrmo5pUhE04MdRwOWqTCuGtiw+Ja0TyN3G77RmZlJoQNj2RC3BiAiCDrArIYLJQ2NhMyWc4D7Q3JDVpg16kbUYuCK5TWCXSiVsSqzOCz5tZ2N0Mt8uCoffH6aFaXYIXRS/VYeF+FPpipmXbukkJ64U07IsweCqQyOy0rtXvE6m6B+j/LUvD9yff4Ha8PzfxcnAplbmRzdHJlYW0KZW5kb2JqCjE5IDAgb2JqCjw8IC9MZW5ndGggOTQgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicRY3BEcAgCAT/VEEJCgraTyaTh/b/jRAyfGDnDu6EBQu2eUYfBZUmXhVYB0pj3FCPQL3hci3J3AUPcCd/2tBUnJbTd2mRSVUp3KQSef8OZyaQqHnRY533C2P7IzwKZW5kc3RyZWFtCmVuZG9iagoyMCAwIG9iago8PCAvVHlwZSAvWE9iamVjdCAvU3VidHlwZSAvRm9ybSAvQkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0xlbmd0aCAzOQovRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJzjMjQwUzA2NVXI5TI3NgKzcsAsI3MjIAski2BBZDO40gAV8wp8CmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0xlbmd0aCAzMjIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicNVG7bcUwDOw1BRcwIH4lzeMgSJG3f5s72qlI07wfVV4ypVwudckqWWHypUN1iqZ8nmam/A71kOOYHtkhulPWlnsYFpaJeUodsZos93ALNr4AmhJzC/H3CPArgFHARKBu8fcPulkSQBoU/BTomquWWGICDYuFrdkV4lbdKVi4q/h2JLkHCXIxWehTDkWKKbfAfBks2ZFanOtyWQr/bn0CGmGFOOyzi0TgecADTCT+ZIBszz5b7OrqRTZ2hjjp0ICLgJvNJAFBUzirPrhh+2q75ueZKCc4OdavojG+DU7mS1LeV7nHz6BB3vgzPGd3jlAOmlAI9N0CIIfdwEaEPrXPwC4Dtkm7d2NK+ZxkKb4ENgr2qFMdyvBi7MxWb9j8x+jKZlFskJX10ekOytygE2Ieb2ShW7K2+zcPs33/AV8Ze2QKZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvTGVuZ3RoIDgzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWMuw3AMAhEe6ZgBH4m9j5RlMLevw0QJW64J909XB0JmSluM8NDBp4MLIZdcYH0ljALXEdQjp3so2HVvuoEjfWmUvPvD5Se7KzihusBAkIaZgplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9MZW5ndGggMzIwIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVSS24FMQjbzym4QKXwT87zqqqLvvtvaxO9FUwwYOMpL1nSS77UJdulw+RbH/clsULej+2azFLF9xazFM8tr0fPEbctCgRREz1YmS8VItTP9Og6qHBKn4FXCLcUG7yDSQCDavgHHqUzIFDnQMa7YjJSA4Ik2HNpcQiJciaJf6S8nt8nraSh9D1Zmcvfk0ul0B1NTugBxcrFSaBdSfmgmZhKRJKX632xQvSGwJI8PkcxyYDsNoltogUm5x6lJczEFDqwxwK8ZprVVehgwh6HKYxXC7OoHmzyWxOVpB2t4xnZMN7LMFNioeGwBdTmYmWC7uXjNa/CiO1Rk13DcO6WzXcI0Wj+GxbK4GMVkoBHp7ESDWk4wIjAnl44xV7zEzkOwIhjnZosDGNoJqd6jonA0J6zpWHGxx5a9fMPVOl8hwplbmRzdHJlYW0KZW5kb2JqCjI0IDAgb2JqCjw8IC9MZW5ndGggMzQwIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVSOW4EMQzr/Qp9IIBu2+/ZIEiR/L8NqdkUA3F0UpQ7WlR2y4eFVLXsdPm0ldoSN+R3ZYXECcmrEu1ShkiovFYh1e+ZMq+3NWcEyFKlwuSk5HHJgj/DpacLx/m2sa/lyB2PHlgVI6FEwDLFxOgals7usGZbfpZpwI94hJwr1i3HWAVSG9047Yr3oXktsgaIvZmWigodVokWfkHxoEeNffYYVFgg0e0cSXCMiVCRgHaB2kgMOXssdlEf9DMoMRPo2htF3EGBJZKYOcW6dPTf+NCxoP7YjDe/OirpW1pZY9I+G+2Uxiwy6XpY9HTz1seDCzTvovzn1QwSNGWNksYHrdo5hqKZUVZ4t0OTDc0xxyHzDp7DGQlK+jwUv48lEx2UyN8ODaF/Xx6jjJw23gLmoj9tFQcO4rPDXrmBFUoXa5L3AalM6IHp/6/xtb7X1x8d7YDGCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0xlbmd0aCAyNTEgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicLVFJcgNBCLvPK/SEZqffY5crh+T/1wjKBwYNi0B0WuKgjJ8gLFe85ZGraMPfMzGC3wWHfivXbVjkQFQgSWNQNaF28Xr0HthxmAnMk9awDGasD/yMKdzoxeExGWe312XUEOxdrz2ZQcmsXMQlExdM1WEjZw4/mTIutHM9NyDnRliXYZBuVhozEo40hUghhaqbpM4EQRKMrkaNNnIU+6Uvj3SGVY2oMexzLW1fz004a9DsWKzy5JQeXXEuJxcvrBz09TYDF1FprPJASMD9bg/1c7KT33hL584W0+N7zcnywlRgxZvXbkA21eLfvIjj+4yv5+f5/ANfYFuICmVuZHN0cmVhbQplbmRvYmoKMjYgMCBvYmoKPDwgL0xlbmd0aCAyMTUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicNVE5DgMhDOz3Ff5AJIwveE+iKM3+v82M0VYewVyGtJQhmfJSk6gh5VM+epkunLrc18xqNOeWtC1zgLi2vC+tksCJZoiDwWmYuAGaPAFD19GoUUMXHtDUpVMosNwEPoq3bg/dY7WBl7Yh54kgYigZLEHNqUUTFm3PJ6Q1v16LG96X7d3IU6XGlhiBBgFWOBzX6NfwlT1PJtF0FTLUqzXLGAkTRSI8+Y6m1RPrWjTSMhLUxhGsagO8O/0wTgAAE3HLAmSfSpSz5MRvsfSzBlf6/gGfR1SWCmVuZHN0cmVhbQplbmRvYmoKMTUgMCBvYmoKPDwgL1R5cGUgL0ZvbnQgL0Jhc2VGb250IC9CTVFRRFYrRGVqYVZ1U2FucyAvRmlyc3RDaGFyIDAgL0xhc3RDaGFyIDI1NQovRm9udERlc2NyaXB0b3IgMTQgMCBSIC9TdWJ0eXBlIC9UeXBlMyAvTmFtZSAvQk1RUURWK0RlamFWdVNhbnMKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvRm9udE1hdHJpeCBbIDAuMDAxIDAgMCAwLjAwMSAwIDAgXQovQ2hhclByb2NzIDE2IDAgUgovRW5jb2RpbmcgPDwgL1R5cGUgL0VuY29kaW5nCi9EaWZmZXJlbmNlcyBbIDQ4IC96ZXJvIC9vbmUgL3R3byAvdGhyZWUgL2ZvdXIgL2ZpdmUgL3NpeCA1NiAvZWlnaHQgL25pbmUgXQo+PgovV2lkdGhzIDEzIDAgUiA+PgplbmRvYmoKMTQgMCBvYmoKPDwgL1R5cGUgL0ZvbnREZXNjcmlwdG9yIC9Gb250TmFtZSAvQk1RUURWK0RlamFWdVNhbnMgL0ZsYWdzIDMyCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0FzY2VudCA5MjkgL0Rlc2NlbnQgLTIzNiAvQ2FwSGVpZ2h0IDAKL1hIZWlnaHQgMCAvSXRhbGljQW5nbGUgMCAvU3RlbVYgMCAvTWF4V2lkdGggMTM0MiA+PgplbmRvYmoKMTMgMCBvYmoKWyA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMAo2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDMxOCA0MDEgNDYwIDgzOCA2MzYKOTUwIDc4MCAyNzUgMzkwIDM5MCA1MDAgODM4IDMxOCAzNjEgMzE4IDMzNyA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2CjYzNiA2MzYgMzM3IDMzNyA4MzggODM4IDgzOCA1MzEgMTAwMCA2ODQgNjg2IDY5OCA3NzAgNjMyIDU3NSA3NzUgNzUyIDI5NQoyOTUgNjU2IDU1NyA4NjMgNzQ4IDc4NyA2MDMgNzg3IDY5NSA2MzUgNjExIDczMiA2ODQgOTg5IDY4NSA2MTEgNjg1IDM5MCAzMzcKMzkwIDgzOCA1MDAgNTAwIDYxMyA2MzUgNTUwIDYzNSA2MTUgMzUyIDYzNSA2MzQgMjc4IDI3OCA1NzkgMjc4IDk3NCA2MzQgNjEyCjYzNSA2MzUgNDExIDUyMSAzOTIgNjM0IDU5MiA4MTggNTkyIDU5MiA1MjUgNjM2IDMzNyA2MzYgODM4IDYwMCA2MzYgNjAwIDMxOAozNTIgNTE4IDEwMDAgNTAwIDUwMCA1MDAgMTM0MiA2MzUgNDAwIDEwNzAgNjAwIDY4NSA2MDAgNjAwIDMxOCAzMTggNTE4IDUxOAo1OTAgNTAwIDEwMDAgNTAwIDEwMDAgNTIxIDQwMCAxMDIzIDYwMCA1MjUgNjExIDMxOCA0MDEgNjM2IDYzNiA2MzYgNjM2IDMzNwo1MDAgNTAwIDEwMDAgNDcxIDYxMiA4MzggMzYxIDEwMDAgNTAwIDUwMCA4MzggNDAxIDQwMSA1MDAgNjM2IDYzNiAzMTggNTAwCjQwMSA0NzEgNjEyIDk2OSA5NjkgOTY5IDUzMSA2ODQgNjg0IDY4NCA2ODQgNjg0IDY4NCA5NzQgNjk4IDYzMiA2MzIgNjMyIDYzMgoyOTUgMjk1IDI5NSAyOTUgNzc1IDc0OCA3ODcgNzg3IDc4NyA3ODcgNzg3IDgzOCA3ODcgNzMyIDczMiA3MzIgNzMyIDYxMSA2MDUKNjMwIDYxMyA2MTMgNjEzIDYxMyA2MTMgNjEzIDk4MiA1NTAgNjE1IDYxNSA2MTUgNjE1IDI3OCAyNzggMjc4IDI3OCA2MTIgNjM0CjYxMiA2MTIgNjEyIDYxMiA2MTIgODM4IDYxMiA2MzQgNjM0IDYzNCA2MzQgNTkyIDYzNSA1OTIgXQplbmRvYmoKMTYgMCBvYmoKPDwgL2VpZ2h0IDE3IDAgUiAvZml2ZSAxOCAwIFIgL2ZvdXIgMTkgMCBSIC9uaW5lIDIxIDAgUiAvb25lIDIyIDAgUgovc2l4IDIzIDAgUiAvdGhyZWUgMjQgMCBSIC90d28gMjUgMCBSIC96ZXJvIDI2IDAgUiA+PgplbmRvYmoKMyAwIG9iago8PCAvRjEgMTUgMCBSID4+CmVuZG9iago0IDAgb2JqCjw8IC9BMSA8PCAvVHlwZSAvRXh0R1N0YXRlIC9DQSAwIC9jYSAxID4+Ci9BMiA8PCAvVHlwZSAvRXh0R1N0YXRlIC9DQSAxIC9jYSAxID4+Ci9BMyA8PCAvVHlwZSAvRXh0R1N0YXRlIC9DQSAxIC9jYSAwLjUgPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgL0YxLURlamFWdVNhbnMtbWludXMgMjAgMCBSID4+CmVuZG9iagoyIDAgb2JqCjw8IC9UeXBlIC9QYWdlcyAvS2lkcyBbIDExIDAgUiBdIC9Db3VudCAxID4+CmVuZG9iagoyNyAwIG9iago8PCAvQ3JlYXRvciAoTWF0cGxvdGxpYiB2My44LjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcpCi9Qcm9kdWNlciAoTWF0cGxvdGxpYiBwZGYgYmFja2VuZCB2My44LjApIC9DcmVhdGlvbkRhdGUgKEQ6MjAyMzEwMTExNjQzMzJaKQo+PgplbmRvYmoKeHJlZgowIDI4CjAwMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAwMDAxNiAwMDAwMCBuIAowMDAwMDExNzA4IDAwMDAwIG4gCjAwMDAwMTE0NDUgMDAwMDAgbiAKMDAwMDAxMTQ3NyAwMDAwMCBuIAowMDAwMDExNjE3IDAwMDAwIG4gCjAwMDAwMTE2MzggMDAwMDAgbiAKMDAwMDAxMTY1OSAwMDAwMCBuIAowMDAwMDAwMDY1IDAwMDAwIG4gCjAwMDAwMDAzNDAgMDAwMDAgbiAKMDAwMDAwNjU2NSAwMDAwMCBuIAowMDAwMDAwMjA4IDAwMDAwIG4gCjAwMDAwMDY1NDQgMDAwMDAgbiAKMDAwMDAxMDI1NCAwMDAwMCBuIAowMDAwMDEwMDQ3IDAwMDAwIG4gCjAwMDAwMDk2ODEgMDAwMDAgbiAKMDAwMDAxMTMwNyAwMDAwMCBuIAowMDAwMDA2NTg1IDAwMDAwIG4gCjAwMDAwMDcwNTMgMDAwMDAgbiAKMDAwMDAwNzM3NSAwMDAwMCBuIAowMDAwMDA3NTQxIDAwMDAwIG4gCjAwMDAwMDc3MTMgMDAwMDAgbiAKMDAwMDAwODEwOCAwMDAwMCBuIAowMDAwMDA4MjYzIDAwMDAwIG4gCjAwMDAwMDg2NTYgMDAwMDAgbiAKMDAwMDAwOTA2OSAwMDAwMCBuIAowMDAwMDA5MzkzIDAwMDAwIG4gCjAwMDAwMTE3NjggMDAwMDAgbiAKdHJhaWxlcgo8PCAvU2l6ZSAyOCAvUm9vdCAxIDAgUiAvSW5mbyAyNyAwIFIgPj4Kc3RhcnR4cmVmCjExOTE5CiUlRU9GCg==", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T16:43:31.994806\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["sns.set()\n", "plot_args = {\"color\": to_rgb(\"C0\") + (0.5,), \"edgecolor\": \"C0\", \"linewidth\": 0.5, \"width\": 1.0}\n", "plt.hist(imgs.view(-1).cpu().numpy(), bins=256, density=True, **plot_args)\n", "plt.yscale(\"log\")\n", "plt.xticks([0, 64, 128, 192, 256])\n", "plt.show()\n", "plt.close()"]}, {"cell_type": "markdown", "id": "d72383c6", "metadata": {"papermill": {"duration": 0.026872, "end_time": "2023-10-11T16:43:32.762902", "exception": false, "start_time": "2023-10-11T16:43:32.736030", "status": "completed"}, "tags": []}, "source": ["As we would expect from the seen images, the pixel value 0 (black) is the dominant value, followed by a batch of values between 250 and 255.\n", "Note that we use a log scale on the y-axis due to the big imbalance in the dataset.\n", "Interestingly, the pixel values 64, 128 and 191 also stand out which is likely due to the quantization used during the creation of the dataset.\n", "For RGB images, we would also see two peaks around 0 and 255,\n", "but the values in between would be much more frequent than in MNIST\n", "(see Figure 1 in the [PixelCNN++](https://arxiv.org/pdf/1701.05517.pdf) for a visualization on CIFAR10).\n", "\n", "Next, we can visualize the distribution our model predicts (in average):"]}, {"cell_type": "code", "execution_count": 26, "id": "af7ec5d7", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:43:32.817636Z", "iopub.status.busy": "2023-10-11T16:43:32.817237Z", "iopub.status.idle": "2023-10-11T16:43:34.550640Z", "shell.execute_reply": "2023-10-11T16:43:34.549871Z"}, "papermill": {"duration": 1.762708, "end_time": "2023-10-11T16:43:34.552080", "exception": false, "start_time": "2023-10-11T16:43:32.789372", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgNDA3LjAyIDMwMC42NTI2MjUgXSAvQ29udGVudHMgOSAwIFIgL0Fubm90cyAxMCAwIFIgPj4KZW5kb2JqCjkgMCBvYmoKPDwgL0xlbmd0aCAxMiAwIFIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnic1d1NjyTHcQbge/+KPkoHxub3x5E07QV44wfsg+GDsF7JJLg0KELy33e8VT0dkTUZxlqrtREgVtp9Yzpn8pnuqsqOrqo3X7//64/v3n/39qv7P3x/eyP/evfbLd5/4j9/uof7T/znv+7x/pb//OkW+F8fbiV0Con/+vPLX3MI1GpqqXIW1n/+x+32x9ubL/nhv90Dzdhb6aGO8eofZYY4W+jj/md857fLF9z+p6++3Uqifk+dcgn4rh9ueU4aSUU/P6OZqbz8cOfjdHL8uL/eLwPm2inyY1ujyP//5/f3f7n/cn/zZcKs4v0b/vMT/zmlxPLGlnVSbW3Esfx8KtXf/fb97dv7ry8DB4qVfwsvYx//fPtIb7/ybyjcvwhcqo3CMRj+u08qI/Jo7z7cvvrh9uaf4j3G+w9/PH55P/z77V/vvwu/v//b/Ydvbv/4w+3b4/v9/WYbS6CeQ6l1ma6OP32+MedzNPxXP2LCrXy+GacUacwSZlxmrONPn3GKkTJGw3/xI2Yc0/h8U84hU6i9p8trTsV/hynPRPUYDf99zJRn+oxTHoX4V5pbWaes4k+fcu78vMZo+K98xJRTbZ9vysdIg/9njB67bFMl+lvn2Snd+6DCZCXFxr/dc4o8Md6QjV4xPZnv7+Lv7z/8dGs056i5xtrSKYANGRdi5idK5c0ppnznX0gMM8sgnaeBUfLxxQz1K75N6P0OyMdfY6EYUi54HI/Q5/FjvPtw5xG++Pr9T3/45798/4dffvviw4+//OW3+9f/ef/283jHGinkwT+/gKvsU8RjadT6rC3OWMf/CXnyQJ7ipDTYRJGr7FPIE89xptlH5N3zVOSxH6/sz2EeP96cn1OH4P+D+ehUM//4TZlL9rcfGAB9ZIojhBB54zI/7XnOs+RXXhgVLz6TPLyQ/6+s8jmzNEfIM+GpFqicB7gpVz7gbS+HuOU4IqZ6TN34+vvy9d+9vX3kV+LouQ6a+djLfsEbiJao8QM7Xg6NRwk45rhUflaVNPjYhJ+Wx45PhlpiPqr+yqeNJcA2kUo7Dsde2TwrtVILqZa0iOnUsYwxf5bhqWR+DuTXMs9Km9T4ACC1xUunjmWM+bNM4e3wsWh5JfOsdNbghVyOi5dOHcsY82eZyocAOO59LfOs8OHxCLWFtnqp1LGMMX+WaVTLsbJ/JfOssEHltUh55fVMHcsY82eZTmOz9T3Sznvr2dqMq5NKHYts5s0ak/grIh9JvRJ5Vnj+AWuAxemZOBYx5v7h1gPP4lhOX1WkwhJ8iBd6Xax06lfGmj/LRGoR76C9lnlW8OwYZY68eOnUsYwxf5ZJxI95fTz8s6qwQR35OIRbuF5Cxy7G7NmFV491uxqQCggiz7S90npJHcsY82eZykdn27WAVLAXCrXXsnjp1LGMMX+Wafz37VpAKtCYKaexeOnUsYwxf5bpNOt2LSCVlo8tbUyLl04dyxjzZ5lJKW3XAlLho9zGe6NRVy+VOpYx5v/hNviBc7sWkEo73kZLNa9eKvUrY82fZSL118+XM+W1Yhp4O3B1Uqljkc28WYO3EGm7FpAKp7GHY5f8YJLAsYcxczY51n67lYBU2qA+Z6ljlVKpYxlj/ixTqbbtSkAqvP3I8fFegvJSqWMZY/4s02js3hn/WVX4KLc0fnrkxUunjmWM+bPMoLjvDEiFj1MmH9rm1UunjmWM+bPMpLzvC0iFpzfbGGXhUqFjF2P2H26Tv3bfFZAKpyPU0OeipVO/Mtb8WSbS3HcFpNJ58IgPPC1eOnUsY8yfZTIfkmxXAlKpvHeOdYa0eOnUsYwxf5Yp/MDtSkAqLfA6sYU4Fi+dOpYx5s8ylX/zr1WOlOfOA45Zrk7P1LHIZt6s0WjuuwJSGYkSSwiTChx7GDNnk8HT264EpILp1cLfZ5HSqWMZY/4sM6nuewJS4XVzrYG/zeKlU8cyxvw/3PgYjfq+KaBKDZ9q7D3nVUylfm1MAeAkCvvOgCqVTiG0HlcylXq2MQBgk3mNvF0PqFKtVHqv7UKmY886BgF00DLbrgpUCTvo2M+PBanBdOxZxyCATqWx7xGoUp38Coo11BVNx551DALo4Iffrg5UiacWcpkzrWg69qxjEEBnUN73ClSp8ePbaKVd0FTsWccggM6ktnveHHEZNGfObbVSqWeTzcTZIwaa+46BKtVGLdYaFZWOHKtY04dMorjvG6hS4x1143mmFUzHnnUMAuhknuF2xaBKPMOSa0plRdOxZx2DADqF+r5/oEq8ox4tplEuaCr2rGMQQIf3wfsegirxIXDmzUu6oqnYs45BAJ1Oad9HUCXe/Eb+Tild0FTsWccggM6guu8mqFLNlEKOx8dBNJqKPesYBNCZ6Jhs1w1S4jjVHM4FuQymY886BgHrpEhh31VQJd788iFwj21F07FjHYsAOonnsV83SIlfQiHk1vMFTcWedQwC6GQ+3t3IHHHFWaP1/AiNxlKxZ5XN1CFSaOw7DKrUjpN0whiCpSPPKsb0IcM/9b7PoEqlEn+Xx8pBBtOxZx2DADp89L/vNagSTpWsNdfVTKWebQwA2Az+x37dIKUaacbSW7uQqdizjkEAnUnT6DZIqQYKc5aQLmgq9qxjELBOjpSMfoOUjpbUfKyqZDAdO9axCKCTqBj9BinhA+WjjLSaqdSzjQEAm0zd6DZIqRReeKdxHhfLYDr2rGMQQKdSMLoNUuI9U62zHh8lUYPp2LOOQQCdxuPsVw1S4uVBCaWMeEFTsWcdgwA6nerueXPEvEtqvYY6L1gq9qyymTpEBg2j3yCl2iiHNlpSWCryrGJM/7xUXjD6DVJqkWJ/2cw8x1KpYxsLADaRstFtkNIY/BSJIZaVTMeedQwC6PA/jG6DlHAAzMfDY65oOvasYxBAJ9M0ug1SqoV64FdUvqCp2LOOQQCdStHoNkiJd9SRv81DR9BU7FnHIIBOo2J0G6RUePHUa055RdOxZx2DADqdXx37dYOUaqIR82wXNB171jEIoDNoGt0GKVUsvDlPFzQVe9YxCFiH9zvJ6DZIiTcwvaT22GcJmood61gE0IlUN1vkM+aXTsql5nHBUrFnlc3UIcITMboNUsKHA2JovSgsFXlWMaYPmULB6DZIiXfauedxrjVlMB171jEIoIPf/37lICV+EcVZSxgrmo496xgE0GlUjX6DlPBhdDQWLmaSerYxAGDTaRjdBinhYDin8NjiPAfTsWcdgwA6k3/q/bpBSvwCGrGnMlc0HXvWMQhYpwXKRrdBSrwZbo0Xl/OCpmLHOhYBdCLauNt1g5T4cC8MXmfOFU3HnnUMAugkmka/QUq8VMgh11kvaCr2rGMQQAdXB9uvG6SEj0FG/l5xRdOxZx2DADr43PlG5ojL5OVli2GuWDr2rLKZOkQadaPfIKU2+SnSUkyCpSPPKsb0IdNpGv0GKfGmZeJy+uMCpmLPOgYBdCYlo+Mgpc6HeaPHWFc0HXvWMQhYp/P3NToOUsqTSn7cCEANpmPHOhYBdCINo+MgJT6QaYF3221F07FnHYMAOpmC0XGQUuVDmlLCuUWWwXTsWccggE6hbHQcpFQCGlTt7MfIYDr2rGMQQAd30NivHKRUM082P5rkMpiOPesYBNBpNIyOg5Sw+O6FF5gXNBV71jEIoDMoGh0HKaGlOXOJ84KmYs86BgF0eK+82yIfMc+sBj7eKxcsFXtW2UydRQY+h7VfOUiJd9dp1PPS74+BdORYxZo+ZHD0v185SKniYqGpnmcoymA69qxjEEAnUzI6DlLiGeMNrfOMIRlMx551DALoFCpGx0FKhZdSvLE9bsGh0VTsWccggE6lbvQcpITG5mzxfO9PBtOxZx2DADqdgtFzkBJumtlTOM+OWdCesWcdgwA62O/sVw5SKvx0iWmcnU4ZTMeedQwC6EzcRXO7cpASWr+plhhXNB171jEIWGfy0b/Rc5BSOT64349bCajBdOxYxyKATqJg9BykxDun1sN5xqIyk9SzjQEAG15V7541R1wyDX5ciRcqFXtW2UwdIoWa0XGQEjYpvGd6yBwD6cizijF9yFT+ve/XDVIqPLXOk2srmI496xgE0MG11/brBinhIs6plNlWNB171jEIoDOoGB0HKVVc3C/lx0b4OZiOPesYBNBBD3e/bpBSmTw+b1/yiqZjzzoGwYdbCoHmvuOgSmXwDGM+15wKTcV+dUwC6OBzw9t1gyrVTLXzAmGuaDr2rGMQQCfjPn27dYMq8Qx7y/G4aIdGU7FnHYMAOoX6vuOgSm3iQ8YtpRVNx551DALo8H5n33FQJV4+ldHqsflVg+nYs45BAB3cQ2ojc8S4J87M8bisgBpEx55VNlOHyOCt6XbloErH6XfpvNTWYyAdeVYxpg+ZSWPfcVAlnCDUck1XMBV71jEIWCdG/qm3KwdVqvzDcZrniqZjxzoWAXTQU9muHFSJD4JDmTWuZir1bGMAwCZT2/cbVAkLhDbLcSLrSvaMPesYBNApNPf9BlVihlxHnPkV2jP2rGMQQId/6n2/QZV488LLzFrGiqZjzzoGAXQ6lX2/QZV4x51mbA+d52A69qxjEEBnUN/3G1QJ7d45ZqsXNBV71jEIoDN5hvt1g5Rq5X1TDiOuaDr2rGMQsE6KlDbPmzPG+YlpnK2FFesZO1bZTR0iieq+46BKuLtQxCUXsmgtmWcXAwA2mfq+56BKvLPmnVE/bjSqBtOxZx2DADq8tdj3HFQJFxbtuR8X7FWD6dizjkEAnUZ533NQJWxwB89zXNBU7FnHIIBOp7bvOagSTmbl2Z4LK4WmYs86BgF0Bg2j5yAlnK5ZZzv34TKYjj3rGASskwNFo+cgJV4k5FDOG9aqwXTsWMcigE6kYvQcpFQYhLc1czVTqWcbAwA2/A+j4yAl3nHzkDXkC5mKPesYBNDJ+ITsduUgJZzqEUoaq5lKPdsYALCplDZb4zM+Ph7K04wXKhV7VtlMHSKNitFvkBIuFjTTeULiC5aKPKsY04dMp270G6TEDCG2MfIrsGfsWccggM7kGe5XDVLqPOORH3tvGUzHnnUMAtYpPI7Rb5BSxWUi23k7AjWYjh3rWATQibhf+HbVIKWCi+HwoGVF07FnHYMAOrgq5n7VICV+kvB+e+ayounYs45BAJ3Ca+r9qkFKvPmtkccvFzQVe9YxCKCDOyPuVw1SwsdoYy3n+6MymI496xgE0GnUjI6DlPhJEkPnfEXTsWcdgwA6/fhE/m7dICXcPrFxsV3QVOxZxyCADu79t5E5Yl5x40P7Ma5YOvasspk6i6B/a3QcpIRNLu4omQVLR45VrOlDJvJX71cOUmq4t1BK4wKmY886BgF0Ek2j3yClihsEPi4NrgbTsWcdgwA6BUdw25WDlPCBdD4ITnFF07FnHYMAOpW3qPuVg5SOy8ofT5cFTceedQwC6DTqRr9BSnjjJsbzRNYV7Rl71jEIoDMoGP0GKfEGprHH+Tk3haZizzoGAXQmZaPfICU+HB5jnBdSV4Pp2LOOQcA6jb+v0XGQEl5CuI/2WNF07FjHIoBOpGF0HKR0fFRgzPPddYWmYs86BgF0cIecjcwR4zYWEx+huGCp2LPKZuoQKZSNnoOUcEg8cC60wlKRZxVj+pCp1Iyeg5SwM8qzlHIBU7FnHYMAOg13N96uHKSEexKE0I/ToDWaij3rGATQ4aN/o+cgJV6CjxHPy+6rwXTsWccggM6kYvQcpFRwM2RmSK/QnrFnHYOAdXrgo//9ykFKpfNOu/TRLmgqdqxjEUAn4lrO25WDlJihHGfTv0J7xp51DALoZEpGz0FKPLXUewl1RdOxZx2DADqFqtFzkFLDyZu95HpBU7FnHYMAOpWPc/crByn1gQ/g9BxXNB171jEIoNMp7J43R3x8VmuM8zM5MoiOPatspg6RgXM4tisHKeEN0dnjcUOzFywVeVYxpg+ZiSs/blcOUsLbfbPV8/QPBaZizzoGAesMflUYPQcp4TToPvN5BohCU7FjHYsAOomi0XOQEj5y3edM8xXaM/asYxBAJ2PluF05SKnicji89k6v0J6xZx2DADqFmtFzkBJavrPP0C5oKvasYxBAp9I0eg5S4s1vy3mex8YKTcWedQwC6HRKRs9BSrUQf5fy2O4Imoo96xgE0MG9w/crBymhvdnHaPkV2jP2rGMQQGdSN3oOUqr4ED+PPS5oKvasYxCwzowUNlvkM8ZFO3iBOVcrlTo22U0cHgmvh+26QUq8we1t1OMyf4+BdORZxZg+ZDJVo+MgJXwAJ8ZwvqeuwFTsWccggE6hYXQcpIQWJh/inVfwWNCesWcdgwA6jYLRcZASbqDdy3GvVm0mqWcbAwA2OH93v2qQEiv00MIDR5M9Y886BgF0Bm859qsGKeH0+cqvpHZBU7FnHYMAOujG7VcNUqqN+Ahm5HRBU7FnHYPgwy2HyEf+21WDKlV+fKjtXDUoNBX71TEJoJOo7PsNqsQHNzXX48oLaiyVerYxAGCT+au3awZV4hdQw6X/2kqmY886BgF0Cs3ds+aIeUGZU6/H7Ww1loo9q2ymDhHelu67DapUcevNko7b5rxgqcizijF9yHQq+26DKmGGMxznImovST3bGACwGbxm3K4ZVKkOyr2EFC9kKvasYxCwTgy4D8NuzaBKDSclxhrHiqZjxzoWAXQi5X2vQZXapMbb4HhFU7FnHYMAOonqvtegSnhTYqTR5oqmY886BgF0Mo19r0GVaqKEk3zLBU3FnnUMAuhUivtegyph4V1niemCpmLPOgYBdNpxH4bdqkFKOATmsY+mgkZTsWcdgwA6ndq+16BK+Ig6v5p6XtF07FnHIIDOoO0W+YixQGgtz7Fi6dizymbqLJLw5uZ+3SAlJhil5DEXrGfkWMWaPmQiP3S/bpASz7C0ftylVXtJ6tnGAIANzsDcrxukxEvLUmo7351QZCr2rGMQQCfT3PcaVAmnwIwxj4tmrmjP2LOOQQCdSmnfbVAlnAKTx3khO42mYs86BgF0GtV9t0GVeIajh1QvZpJ6tjEAYNOp73sNqgSF2ctxgu+F7CX2rGMQQGdSMHoNUsKpvhweF/vTaCr2rGMQsE4OlI1eg5Qaxuc9U13RdOxYxyKATqRqdBukdJzy2x4rToWmYs86BgF0Eo3d8+aIW+XD4ZTPtZTCUrFnlc3UIVL4B9+vGqTEG5XOu6aaBEtHnlWM6UOmUja6DVLiuMx57qJkLJV6tjEAYNOoGd0GKVV8eOJxgocaTMeedQwC6HSaRrdBSrzsnqXggGZFU7FnHYMAOpOS0W2QEg7yeBk1r2gq9qxjELBOCbi33XbVIKVWcLHeONOKpmPHOhYBdCJ1o9sgJWboiQdPr9CesWcdgwA6mYLRbZASH850fs7Mi5mknm0MANjg3IT9qkFKHIfU6nFqkCZTsWcdgwA6larRa5ASP0dqzP1871ihqdizjkEAncZ7nY3MEY9AIcYRLlg69qyymTpEcBb8ftUgpZZxcb9xrhrOgXTkWcWYPmQmz2O/apBSG3inr539cAWmYs86BgHrVP5qo9sgpePjso8L36jBdOxYxyKATqRhdBukxDHvt7m6ounYs45BAB30tPfrBinx1GoYAFnRVOxZxyCATqFidBukdGxgYm7jgqZizzoGAXQq/2O/bpBS50OaUsfjufMcTMeedQwC6DSaRr9BSm1SHLOf7xnLYDr2rGMQQIdfHUa/QUpYP8Vw3phKo6nYs45BAJ1Jxeg3SImfJJMfP65oKvasYxCwTgvUN8+bM+Yp4WZmj62xYKnYscpu6hDBWnq/cpAS/62HOI6LbL1gqcizijF9yGQeZ79ykBI6C3w8fL5rvIA9Y886BgF0ClWj4yClHimFls92jAymY886BgF0Kg2j4yAlZhjtcXmOFe0Ze9YxCKDDP7zRcZASP770dn7+T8ZSqWcbAwA2g7LRb5ASP0VaHfHcU8lgOvasYxBAB2e37NcNUuq4wVsO51mKCk3FnnUMAtbpgabRb5AS/630fN5aSKOp2LGORQCdRNHoOEiJHx/KKOd5nAvaM/asYxBAB58d3q8bpMSPL/waOo92FJqKPesYBNApvG7cyBwxv4hGnTWNFUvHnlU2U4dIxR07tusGKeEpMtKMU7B05FnFmD5kOiWj4yAl/tvAMXC6gKnYs45BAJ3Be+H9ukFKMQ6eYuzn6Zsy2pJ79jEQ4DOpGz0HKUVeLfBLqJwfqVBuOvfsYyCwz4gUjK6DlOIIFFuOtV7cdO7Yx0KAT6Js9B2kFHnflOMox1XJtJvOPfsYCPDJVI3Og5Ri4VVDiO1cmis3nT993nyZMPV0/+YeKWFW8f72/ubr93/98d37795+dX/32+1i+OERTD5aqKkdZ5bcvr9lXKMoLV/3El2+8tV4rx6qvkwea4737e2/Ad4W5eMKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iago2MzEwCmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjE3IDAgb2JqCjw8IC9MZW5ndGggMzk1IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD1SS27FQAjb5xRcoNLwm895UlXdvPtva0NSqSq8iTHGMH3KkLnlS10ScYXJt16uWzymfC5bWpl5iLuLjSU+ttyX7iG2XXQusTgdR/ILMp0qRKjNqtGh+EKWhQeQTvChC8J9Of7jL4DB17ANuOE9MkGwJOYpQsZuURmaEkERYeeRFaikUJ9Zwt9R7uv3MgVqb4ylC2Mc9Am0BUJtSMQC6kAAROyUVK2QjmckE78V3WdiHGDn0bIBrhlURJZ77MeIqc6ojLxExD5PTfoolkwtVsZuUxlf/JSM1Hx0BSqpNPKU8tBVs9ALWIl5EvY5/Ej459ZsIYY6btbyieUfM8UyEs5gSzlgoZfjR+DbWXURrh25uM50gR+V1nBMtOt+yPVP/nTbWs11vHIIokDlTUHwuw6uRrHExDI+nY0peqIssBqavEYzwWEQEdb3w8gDGv1yvBA0p2sitFgim7ViRI2KbHM9vQTWTO/FOdbDE8Js753WobIzMyohgtq6hmrrQHazvvNwtp8/M+iibQplbmRzdHJlYW0KZW5kb2JqCjE4IDAgb2JqCjw8IC9MZW5ndGggMjQ5IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nE1RSYoDMAy75xX6QCFek7ynQ5lD5//Xyg6FOQQJr5KTlphYCw8xhB8sPfiRIXM3/Rt+otm7WXqSydn/mOciU1H4UqguYkJdiBvPoRHwPaFrElmxvfE5LKOZc74HH4W4BDOhAWN9STK5qOaVIRNODHUcDlqkwrhrYsPiWtE8jdxu+0ZmZSaEDY9kQtwYgIgg6wKyGCyUNjYTMlnOA+0NyQ1aYNepG1GLgiuU1gl0olbEqszgs+bWdjdDLfLgqH3x+mhWl2CF0Uv1WHhfhT6YqZl27pJCeuFNOyLMHgqkMjstK7V7xOpugfo/y1Lw/cn3+B2vD838XJwKZW5kc3RyZWFtCmVuZG9iagoxOSAwIG9iago8PCAvTGVuZ3RoIDk0IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWNwRHAIAgE/1RBCQoK2k8mk4f2/40QMnxg5w7uhAULtnlGHwWVJl4VWAdKY9xQj0C94XItydwFD3Anf9rQVJyW03dpkUlVKdykEnn/DmcmkKh50WOd9wtj+yM8CmVuZHN0cmVhbQplbmRvYmoKMjAgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL1N1YnR5cGUgL0Zvcm0gL0JCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9MZW5ndGggMzkKL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnic4zI0MFMwNjVVyOUyNzYCs3LALCNzIyALJItgQWQzuNIAFfMKfAplbmRzdHJlYW0KZW5kb2JqCjIxIDAgb2JqCjw8IC9MZW5ndGggMzIyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVRu23FMAzsNQUXMCB+Jc3jIEiRt3+bO9qpSNO8H1VeMqVcLnXJKllh8qVDdYqmfJ5mpvwO9ZDjmB7ZIbpT1pZ7GBaWiXlKHbGaLPdwCza+AJoScwvx9wjwK4BRwESgbvH3D7pZEkAaFPwU6JqrllhiAg2Lha3ZFeJW3SlYuKv4diS5BwlyMVnoUw5Fiim3wHwZLNmRWpzrclkK/259AhphhTjss4tE4HnAA0wk/mSAbM8+W+zq6kU2doY46dCAi4CbzSQBQVM4qz64Yftqu+bnmSgnODnWr6Ixvg1O5ktS3le5x8+gQd74Mzxnd45QDppQCPTdAiCH3cBGhD61z8AuA7ZJu3djSvmcZCm+BDYK9qhTHcrwYuzMVm/Y/MfoymZRbJCV9dHpDsrcoBNiHm9koVuytvs3D7N9/wFfGXtkCmVuZHN0cmVhbQplbmRvYmoKMjIgMCBvYmoKPDwgL0xlbmd0aCA4MyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFjLsNwDAIRHumYAR+JvY+UZTC3r8NECVuuCfdPVwdCZkpbjPDQwaeDCyGXXGB9JYwC1xHUI6d7KNh1b7qBI31plLz7w+Unuys4obrAQJCGmYKZW5kc3RyZWFtCmVuZG9iagoyMyAwIG9iago8PCAvTGVuZ3RoIDMyMCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UktuBTEI288puECl8E/O86qqi777b2sTvRVMMGDjKS9Z0ku+1CXbpcPkWx/3JbFC3o/tmsxSxfcWsxTPLa9HzxG3LQoEURM9WJkvFSLUz/ToOqhwSp+BVwi3FBu8g0kAg2r4Bx6lMyBQ50DGu2IyUgOCJNhzaXEIiXImiX+kvJ7fJ62kofQ9WZnL35NLpdAdTU7oAcXKxUmgXUn5oJmYSkSSl+t9sUL0hsCSPD5HMcmA7DaJbaIFJucepSXMxBQ6sMcCvGaa1VXoYMIehymMVwuzqB5s8lsTlaQdreMZ2TDeyzBTYqHhsAXU5mJlgu7l4zWvwojtUZNdw3Duls13CNFo/hsWyuBjFZKAR6exEg1pOMCIwJ5eOMVe8xM5DsCIY52aLAxjaCaneo6JwNCes6VhxsceWvXzD1TpfIcKZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDM0MCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UjluBDEM6/0KfSCAbtvv2SBIkfy/DanZFANxdFKUO1pUdsuHhVS17HT5tJXaEjfkd2WFxAnJqxLtUoZIqLxWIdXvmTKvtzVnBMhSpcLkpORxyYI/w6WnC8f5trGv5cgdjx5YFSOhRMAyxcToGpbO7rBmW36WacCPeIScK9Ytx1gFUhvdOO2K96F5LbIGiL2ZlooKHVaJFn5B8aBHjX32GFRYINHtHElwjIlQkYB2gdpIDDl7LHZRH/QzKDET6NobRdxBgSWSmDnFunT03/jQsaD+2Iw3vzoq6VtaWWPSPhvtlMYsMul6WPR089bHgws076L859UMEjRljZLGB63aOYaimVFWeLdDkw3NMcch8w6ewxkJSvo8FL+PJRMdlMjfDg2hf18eo4ycNt4C5qI/bRUHDuKzw165gRVKF2uS9wGpTOiB6f+v8bW+19cfHe2AxgplbmRzdHJlYW0KZW5kb2JqCjI1IDAgb2JqCjw8IC9MZW5ndGggMjUxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nC1RSXIDQQi7zyv0hGan32OXK4fk/9cIygcGDYtAdFrioIyfICxXvOWRq2jD3zMxgt8Fh34r121Y5EBUIEljUDWhdvF69B7YcZgJzJPWsAxmrA/8jCnc6MXhMRlnt9dl1BDsXa89mUHJrFzEJRMXTNVhI2cOP5kyLrRzPTcg50ZYl2GQblYaMxKONIVIIYWqm6TOBEESjK5GjTZyFPulL490hlWNqDHscy1tX89NOGvQ7Fis8uSUHl1xLicXL6wc9PU2AxdRaazyQEjA/W4P9XOyk994S+fOFtPje83J8sJUYMWb125ANtXi37yI4/uMr+fn+fwDX2BbiAplbmRzdHJlYW0KZW5kb2JqCjI2IDAgb2JqCjw8IC9MZW5ndGggMjE1IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVROQ4DIQzs9xX+QCSML3hPoijN/r/NjNFWHsFchrSUIZnyUpOoIeVTPnqZLpy63NfMajTnlrQtc4C4trwvrZLAiWaIg8FpmLgBmjwBQ9fRqFFDFx7Q1KVTKLDcBD6Kt24P3WO1gZe2IeeJIGIoGSxBzalFExZtzyekNb9eixvel+3dyFOlxpYYgQYBVjgc1+jX8JU9TybRdBUy1Ks1yxgJE0UiPPmOptUT61o00jIS1MYRrGoDvDv9ME4AABNxywJkn0qUs+TEb7H0swZX+v4Bn0dUlgplbmRzdHJlYW0KZW5kb2JqCjE1IDAgb2JqCjw8IC9UeXBlIC9Gb250IC9CYXNlRm9udCAvQk1RUURWK0RlamFWdVNhbnMgL0ZpcnN0Q2hhciAwIC9MYXN0Q2hhciAyNTUKL0ZvbnREZXNjcmlwdG9yIDE0IDAgUiAvU3VidHlwZSAvVHlwZTMgL05hbWUgL0JNUVFEVitEZWphVnVTYW5zCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnRNYXRyaXggWyAwLjAwMSAwIDAgMC4wMDEgMCAwIF0KL0NoYXJQcm9jcyAxNiAwIFIKL0VuY29kaW5nIDw8IC9UeXBlIC9FbmNvZGluZwovRGlmZmVyZW5jZXMgWyA0OCAvemVybyAvb25lIC90d28gL3RocmVlIC9mb3VyIC9maXZlIC9zaXggNTYgL2VpZ2h0IC9uaW5lIF0KPj4KL1dpZHRocyAxMyAwIFIgPj4KZW5kb2JqCjE0IDAgb2JqCjw8IC9UeXBlIC9Gb250RGVzY3JpcHRvciAvRm9udE5hbWUgL0JNUVFEVitEZWphVnVTYW5zIC9GbGFncyAzMgovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Bc2NlbnQgOTI5IC9EZXNjZW50IC0yMzYgL0NhcEhlaWdodCAwCi9YSGVpZ2h0IDAgL0l0YWxpY0FuZ2xlIDAgL1N0ZW1WIDAgL01heFdpZHRoIDEzNDIgPj4KZW5kb2JqCjEzIDAgb2JqClsgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAKNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCAzMTggNDAxIDQ2MCA4MzggNjM2Cjk1MCA3ODAgMjc1IDM5MCAzOTAgNTAwIDgzOCAzMTggMzYxIDMxOCAzMzcgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNgo2MzYgNjM2IDMzNyAzMzcgODM4IDgzOCA4MzggNTMxIDEwMDAgNjg0IDY4NiA2OTggNzcwIDYzMiA1NzUgNzc1IDc1MiAyOTUKMjk1IDY1NiA1NTcgODYzIDc0OCA3ODcgNjAzIDc4NyA2OTUgNjM1IDYxMSA3MzIgNjg0IDk4OSA2ODUgNjExIDY4NSAzOTAgMzM3CjM5MCA4MzggNTAwIDUwMCA2MTMgNjM1IDU1MCA2MzUgNjE1IDM1MiA2MzUgNjM0IDI3OCAyNzggNTc5IDI3OCA5NzQgNjM0IDYxMgo2MzUgNjM1IDQxMSA1MjEgMzkyIDYzNCA1OTIgODE4IDU5MiA1OTIgNTI1IDYzNiAzMzcgNjM2IDgzOCA2MDAgNjM2IDYwMCAzMTgKMzUyIDUxOCAxMDAwIDUwMCA1MDAgNTAwIDEzNDIgNjM1IDQwMCAxMDcwIDYwMCA2ODUgNjAwIDYwMCAzMTggMzE4IDUxOCA1MTgKNTkwIDUwMCAxMDAwIDUwMCAxMDAwIDUyMSA0MDAgMTAyMyA2MDAgNTI1IDYxMSAzMTggNDAxIDYzNiA2MzYgNjM2IDYzNiAzMzcKNTAwIDUwMCAxMDAwIDQ3MSA2MTIgODM4IDM2MSAxMDAwIDUwMCA1MDAgODM4IDQwMSA0MDEgNTAwIDYzNiA2MzYgMzE4IDUwMAo0MDEgNDcxIDYxMiA5NjkgOTY5IDk2OSA1MzEgNjg0IDY4NCA2ODQgNjg0IDY4NCA2ODQgOTc0IDY5OCA2MzIgNjMyIDYzMiA2MzIKMjk1IDI5NSAyOTUgMjk1IDc3NSA3NDggNzg3IDc4NyA3ODcgNzg3IDc4NyA4MzggNzg3IDczMiA3MzIgNzMyIDczMiA2MTEgNjA1CjYzMCA2MTMgNjEzIDYxMyA2MTMgNjEzIDYxMyA5ODIgNTUwIDYxNSA2MTUgNjE1IDYxNSAyNzggMjc4IDI3OCAyNzggNjEyIDYzNAo2MTIgNjEyIDYxMiA2MTIgNjEyIDgzOCA2MTIgNjM0IDYzNCA2MzQgNjM0IDU5MiA2MzUgNTkyIF0KZW5kb2JqCjE2IDAgb2JqCjw8IC9laWdodCAxNyAwIFIgL2ZpdmUgMTggMCBSIC9mb3VyIDE5IDAgUiAvbmluZSAyMSAwIFIgL29uZSAyMiAwIFIKL3NpeCAyMyAwIFIgL3RocmVlIDI0IDAgUiAvdHdvIDI1IDAgUiAvemVybyAyNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE1IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMCAvY2EgMSA+PgovQTIgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMSA+PgovQTMgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMC41ID4+ID4+CmVuZG9iago1IDAgb2JqCjw8ID4+CmVuZG9iago2IDAgb2JqCjw8ID4+CmVuZG9iago3IDAgb2JqCjw8IC9GMS1EZWphVnVTYW5zLW1pbnVzIDIwIDAgUiA+PgplbmRvYmoKMiAwIG9iago8PCAvVHlwZSAvUGFnZXMgL0tpZHMgWyAxMSAwIFIgXSAvQ291bnQgMSA+PgplbmRvYmoKMjcgMCBvYmoKPDwgL0NyZWF0b3IgKE1hdHBsb3RsaWIgdjMuOC4wLCBodHRwczovL21hdHBsb3RsaWIub3JnKQovUHJvZHVjZXIgKE1hdHBsb3RsaWIgcGRmIGJhY2tlbmQgdjMuOC4wKSAvQ3JlYXRpb25EYXRlIChEOjIwMjMxMDExMTY0MzM0WikKPj4KZW5kb2JqCnhyZWYKMCAyOAowMDAwMDAwMDAwIDY1NTM1IGYgCjAwMDAwMDAwMTYgMDAwMDAgbiAKMDAwMDAxMTg4OSAwMDAwMCBuIAowMDAwMDExNjI2IDAwMDAwIG4gCjAwMDAwMTE2NTggMDAwMDAgbiAKMDAwMDAxMTc5OCAwMDAwMCBuIAowMDAwMDExODE5IDAwMDAwIG4gCjAwMDAwMTE4NDAgMDAwMDAgbiAKMDAwMDAwMDA2NSAwMDAwMCBuIAowMDAwMDAwMzQwIDAwMDAwIG4gCjAwMDAwMDY3NDYgMDAwMDAgbiAKMDAwMDAwMDIwOCAwMDAwMCBuIAowMDAwMDA2NzI1IDAwMDAwIG4gCjAwMDAwMTA0MzUgMDAwMDAgbiAKMDAwMDAxMDIyOCAwMDAwMCBuIAowMDAwMDA5ODYyIDAwMDAwIG4gCjAwMDAwMTE0ODggMDAwMDAgbiAKMDAwMDAwNjc2NiAwMDAwMCBuIAowMDAwMDA3MjM0IDAwMDAwIG4gCjAwMDAwMDc1NTYgMDAwMDAgbiAKMDAwMDAwNzcyMiAwMDAwMCBuIAowMDAwMDA3ODk0IDAwMDAwIG4gCjAwMDAwMDgyODkgMDAwMDAgbiAKMDAwMDAwODQ0NCAwMDAwMCBuIAowMDAwMDA4ODM3IDAwMDAwIG4gCjAwMDAwMDkyNTAgMDAwMDAgbiAKMDAwMDAwOTU3NCAwMDAwMCBuIAowMDAwMDExOTQ5IDAwMDAwIG4gCnRyYWlsZXIKPDwgL1NpemUgMjggL1Jvb3QgMSAwIFIgL0luZm8gMjcgMCBSID4+CnN0YXJ0eHJlZgoxMjEwMAolJUVPRgo=", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T16:43:33.841615\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["plt.bar(np.arange(mean_out.shape[0]), mean_out, **plot_args)\n", "plt.yscale(\"log\")\n", "plt.xticks([0, 64, 128, 192, 256])\n", "plt.show()\n", "plt.close()"]}, {"cell_type": "markdown", "id": "09600e38", "metadata": {"papermill": {"duration": 0.029302, "end_time": "2023-10-11T16:43:34.612326", "exception": false, "start_time": "2023-10-11T16:43:34.583024", "status": "completed"}, "tags": []}, "source": ["This distribution is very close to the actual dataset distribution.\n", "This is in general a good sign, but we can see a slightly smoother histogram than above.\n", "\n", "Finally, to take a closer look at learned value relations, we can\n", "visualize the distribution for individual pixel predictions to get a\n", "better intuition. For this, we pick 4 random images and pixels, and\n", "visualize their distribution below:"]}, {"cell_type": "code", "execution_count": 27, "id": "4f580dd0", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:43:34.676189Z", "iopub.status.busy": "2023-10-11T16:43:34.675468Z", "iopub.status.idle": "2023-10-11T16:43:38.754010Z", "shell.execute_reply": "2023-10-11T16:43:38.753235Z"}, "papermill": {"duration": 4.11602, "end_time": "2023-10-11T16:43:38.757667", "exception": false, "start_time": "2023-10-11T16:43:34.641647", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgNjEyLjkgMzY3LjE4MDYyNSBdIC9Db250ZW50cyA5IDAgUiAvQW5ub3RzIDEwIDAgUiA+PgplbmRvYmoKOSAwIG9iago8PCAvTGVuZ3RoIDEyIDAgUiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJzc3U2zLLdxLup5/4oeSgOCSHxjKF3ajNBM0g7fgcMDBU3piLG2HDwM+f79my+qVyEBZEk0ubnPKVghea/MbqxVT1ejABQK+PKrb//7r998+4evf/v8f/74+LL/9M0PD3p+x//9y9M+v+P//n9Pen7N//3Lw/JPHx+JnKn8r7fXv3zKhopNLnLIjj/+r8fjz48vf8Nv/uFpTaWcQraxlOWHUC3VZHN5/m/83q+HFzz+0asfj5BNfjpbTD5+7ceHt2S8Tz4FGX+TcR+rqe/xVsIQaX/498+5aBe9Sf71n/CkSMY9//e3z//3+bfnl79xOEx6/o7/+x3/t8FNtPw7Uq4uh/EP7uHhr3j88fH75/fvJVtDkT+V98Lbj1+/oo/v+ROzzy8sp2Iy1ApzOdUnVQZzxOV98/Hx2w+PL/+VnkTPD39uH+eH/3z8+/NX9tfP/3h++N3jXz48ft9+4yc/bqJsvPPVuvHARfwTHDmRfZXmc/pRR57CL3/oOZiaSvTTkZ/hT3HgKZiAwmzI0f2oIydXfvFDd96ZRJ5iHY9dxD/BwTvnDKG0EnMIP+7gq/vlD57/DhdzLlPlJOKf4uBzNRmlpZT9jzvnXUy/+MEfBbpskuXjnevkHv+ph53573HOGxd8dLUGPvTjgC1enAsE+Gj74f+Kfv388N0jmVr5axIpJneAoPLjBHkT+f9SCgzwDMaRrT6fheBweinBlFpqDsnW8ColtgSDfs9/gLU5PwH++icFQ1wnBRQYjC+1IX/z8clFf/HVt9/96d/+/sc//e2HLz7+9W9//+H51X89f//Lfi7RmUyhZDd9Lj3+sz6XUIyPMdQUyQbxudhU8mf+WNyP/1j4m9T+vv9zH0vOprhUfJk+lh7/ydcHh88le75IMI/ng3ef5XOpN+L3lluzvpS5thLxn8dfi4m5BldT+DzfinQnfVeMjfyH0aTf4z9Ln5uFJlXyNQHrs/D7O/FzKZS4hpjqHhH/yfwF/HxNKJYL5cqMj/5n8XP/0vpiS2T0a337rv9T2PxxgPyVtZ6bUzFzFzQc/U/nI2fSew80tA6riU2gv/55+frHH77+ka9snVtuITpXqkvPL4rlzlLgjh2nuBvO0lyVOD8m3kSifXhcUsWH1wsawtzR/e2tgS4cME5hrLfeLT7vcVe4o8ytFEuD2hC+P4+q0HSqJ0tF4XkluMnB7+UqkwY1Gd2BR2NgH2eKp8yV2exzJtCR43qNQhzZZPj+QLoDA3mTvQu0fr/OhKvEb44YkRjdRPj+QLoDAwX++z1f0maf97irwcRkU8iTmgjfn0dVYJ1oouereF54zoS3XBpfpqqTakP0/jw6A/skE3yI3H+efc6EK96UEPiSJdVk9P48ugLzZG6hRmfXb9eZcJX/WSkeX6+uJsP3B9IdGIhb3T7y12QBOhOu8JU8OxvT6CbD9wfSHRioGvIp1rQAnQluLnNDuWQfJzcRvj+Q7vDxkblz5rOrSwejJ1wmE7jSCWV0k+HbA104NCDueNa61EE94Uo2jrv5rfss3GR4ByDNgYHQziupLH2MnkBny8VMJU1uInx/IN2BgRw3+qovSyejJ1zhjkWtJfnJTYTvD6Q7MBD//cHastZBZ4I7pcalUKkObkP4/kC6AwNxXyFgIsTs8x73Fs2deNw8kGoifH8eVYF1Ir+NQl46GT3B1bHhxnJNTqoN0fvz6Azsk4wPjvJa/5wJDvClyh7dLcEmw/cH0h0YiK/TweGG6Ax0JtDfCj7VNLnJ8P2BdAcGKoaCj2npZvQEGjt8uH5xE+H7A+kODFSN5d54WqugM+EqhuZ9pJFNRO/Poys0nhoCviUrzyvhajRcVFrURHgHIM3h46NYU0JMaelk9AQ3c7gtGGLrsAs3Gb490IUDA5HBlIa4NIF6wvPhBpuic4PbEL4/kO7AQM60qbPLQEdPOAzO84W9zUPrBQ3h+wPpDgzkudWXc1w6GT3hKu5Ghxrc6CbD9wfSHRgo8Pv4S7P4vMcxCTFiCuakJsP351EVWCdym69SWCugM+H4n1RKdGVQk9H78+gM7JO4yVe10+dMeI9zplY7sg3h+wPpDgyUDUUbw9LJ6AncUy455HbTvRc0hO8PpDswEKaAkVvaiGfcZWeiz8nXUU2G78+jKjSdGglzoVaeV8ITH2Is3C6UakN0Bx6NgX2q4To2+fX0ORPcnzCxxEB+YBvC9wfSHT4+uBLhq7VfZ0r1BDpbLmab4+gmw7cHunBgIOLvSbBuGeToCYyHBYzHu9FNhu8PpDswENez3OabJym+iQSGM7LnanlkE9H78+gKzOMNd8PDegHrCdwyJb6Qu0lNhu8PpDswUDA+JnLLIEdPYE5yTLY4GtyG8P2BdAcGQj2byjpbsydQF4dSii2D2xC+P5DuwECJG305rtM1e6LNa6m2FBrdZPj+QLoDA2Vu9fHXZBnk6Amuh011Lo1sMnp/Hl2h8dRY1qkcZ9zlYIo9JkX1UmR0BxzFgG0Kt/fwugXnTLQ5qxQoD2QieH8cHYF1uAWTLH9LFp0zgbMklpxTGdFk+P5AusPHB2HCbuL/XfoXIuNyZIvUbidLuB69PdCVA4TQiKG0TtYUGTy9iSrZ0ywn4hsg6RRAciYk5+tSDYkMbukkG62b8Ib4Bkg6BZC84T8ezz8vSGfGU+L+BRfnJzwZ3wBJpwBSMC75vF7tRQaPOUWbbUoj3hDfAEmnAFI0lEIoSp10ZlyK6MnTq04SeCK+AZJOAaRkbIqYprognRncJIxU64In4xsg6RQHUk38vqXrITJoK9bCjaEFT8S3QNIogJRNSZxS6qQz4wpWuHC5rc8h8WR8AySdAkh8MCm7dbqryLjijPcUSpnxRHwDJJ0CSJWbhrmukzpF5hh3jfwVm/BkfAMknQLLjvGvTSXltU7qGdyf51+TMk14Mn5/pAsKIBE3DZUbiz3hfUD4uP73gmR4AyCVAT5co2Rr17mdIoMLGCVbj+roLGoIb0CkS8DIG5dtXid4ikyrmqmEo0Mi7GR8AySdAkh8MmTibtiKdGZc4qrZ+hxpwpPxDZB0CiBFPJaKNQMXpDOD58YKN6nrZCfCGxDpEAdRzXx5WrsjPYMxNS6Lcp7oZHwLJI0CSHwuoI2j1EhnBg8nYF4n0YQn4xsg6RTHsqzc53Jx7Y70jOd/lszd/DTiDfENkHQKIBWTMp8Na3ekZ7B4X8ghWDfiDfENkHQKIFUTc0xBqZPOjCvW2EAl5glPxjdA0ikYyfE7c/Jh7Y70DGZTp+JcTROejN8f6YICSMQNw7yuiNIT3qItxLVQHemG+AZEKgSEnMH6en6tkHrGRY6T97kOdEN4AyJdAkYe6wMHvw6P9AweHs+Je/iT3RDfAEmnAFLgtmCldUqoyGBdVBe8TwueiG+ApFMcSDVXZVj7TOAhYFdz8WmkG+JbECkQEIoG6+a6dYCkZzALwsYQ2xRZQSfDGxDpEjBKBsuVr5NDRYZ/DwZCuN8x2cn4Bkg6BZAyN3GorhNERcZh0DFTzWnCk/ENkHQKIBUTi0u0DpD0DNauCony0dIWeDK+AZJOASQ+zuI9rd2RnsEClY64eDfhyfgGSDoFI3lrfAl2nSwqMg5L6lEJMU54Mn5/pAsKIPGfXLhiWeuknsFig9ladwy2CTwZ3wBJpwCSM9x9D3btkfSMp8hHjK1rRrwhvgGSTgEkb2xJtE4hFRnc47fBUXvUU5Q1xDdA0ikOpFoSl6MhvTKYy09U/OtMEngivgWSRgGkwE1D5c7/exw3QrAFxXE3UsKJ+AZAGgN4UKFwg2ftj/QMWkK+ejqG2TqciG4ApEOAKHGTsJSqnENnBlcvPKBWy0Qn4xsg6RRAwnPBNdZ1eKRnuDFkrA3pVRedZQ3xDZB0CiDh0VfrlAmkPXN8rfj9ZcQb4hsg6RRAwnHaWpQa6cy4bE1ui+1NeDK+AZJOwUgBxRBW+pqResYlzIhI/rgZKfBk/P5IFxRAIsM9ea9MIO0ZzMv2Nfi2oMGAJ+IbIOkUQHLGVm/zWif1DHc4TAze0oQ3xDdA0ikOpFp9zusQSc9g/lpCL82NeEN8CySNAkjelBqCMoG0ZzBRtFQKIU14Mr4Bkk4BpMC9ikjrxe1MgCIny0WvdGd8AyIVAkKRXxu5E7YSnRk0hiIaQzTRifAGRLoEjBIfZ4pp7ZL0DNazTplSyqPdEN8ASacAEn7ITplC2jNYEdTb8l4fnWUN8Q2QdAogFW7m5HXLj57A431UfXwRCToR34BIhYBQRSkpKlX2mcGM45JYo050IrwBkS7BRtHyqYBzYTHqGVTOGFCzebKT8fsjXVAAidvN1uJW2YJ0ZoBR8TCtgnfGN0DSKYDE7WZrc1i7Iz3DEZNjojjZifAGRDrEQcSvDcr00Z5p50uKRxtSyInwFkQaBIi41Wwdp1aiM4Nb1zak2jbUkXQyvgGSTgEkbjZbV8I6PNIzWE6VYsquzngivgGSTgEkbjdbH5X5oz3jqsPzRj7Pdj28AZEOASJuNdvgvFIfnRmHzWADpeAmOhnfAEmnABL/YEP16+BIz2Buv3Oe8mQnwhsQ6RAg4mazjUkRek+0SaLk6vFUpIQT8Q2IVAgIcaPZJq9MHu0Z1Dq+4EnIkU6GNyDSJdgIu5ZzO9mtQyM9g+dDLJbrpclOxu+PdEEBJNxwzdmtXZGeabvthETHiSTwZHwDJJ3iQKqWL+RrV6RncJjceeXm4oQn41sgaRRActwkrKQMaPeMd8m4UsIxw7aXNcQ3QNIpgMQng62F1u5Iz2B/vey8pTriDfENkHQKIAWDmWnK5NGewT2QBIw84cn4Bkg6BZAito9xVqmTzoyLFtWQP56vFXgyvgGSTgEkTLematcOSc9gIY1E+bUmksCT8Q2QdAogZePJJWXyaM8Ao9R21V/wzvgGSDoFkAr3K7yf95J/kxmMqRUu75hBIvFEfAMknQJI1RD5Wtchkp5p042xRmSc8UR8AySdgpGw0zlxc3DtlvQMHjjGs7THApsCT8bvj3RBcSBVin7eV/5NZrC9QSZvbRzxhvgWSBoFkIirlaS0AM5EW//Ip+QmuiG+AZEKASHHZ0LKygzSnkHFU1A8jXQyvAGRLgEjfzRzlProzOARiOBCOGbZSDsR3wBJpwASVusvVNZhkp5pc474SxX8jCfiGyDpFECKeLixKBNIewb7qsVY7GQnwxsQ6RAgStworFGZPtoz2FnN1ZiOkaRe1hDfAEmnABIfp7Mur4MkPYOV/X22+RgkEXgyvgGSTgEkbi5zNzWvHZKewf5G0VvfNlGVeDK+AZJOASSM4nNzee2Q9IzDns6W4rE0gsCT8Q2QdIoDqTrHFygN6ZXhfqzh/2/DjCfjWyBpFIyEndGdt8o9yZ5xuJsdrT1WaxF4Mn5/pAsKIBEGqJWL25nAfbVSXiMkQk6ENwBSGeDjTEJrea2Oegaj/ClSndxEdAMg3QFC3Ax0keLaGekZF6sJtq02PsrJ+AZIOgWQAne6YlGmj/YMpkByN9Yel36BJ+MbIOkUQIrGu6SdSO8JjBOFEGr0I90Q34BIhYBQ4vZgdmEdHOkZPEYb7Hsz+yxqCG9ApEvAKHNzELfvV6Mz413GumzhmGLbyxriGyDpFEAq3Bwsad6b/k1m2o2izN38OuIN8Q2QdIoDqfJV3K+DIz3TNuyNsR4rIvWyhvgWSBoFkLgZ6K1VJpD2DNZiRw//WKO1lzXEN0DSKRgJO6h7m5UJpD2DtdiZhY47/wJPxu+PdEEBJOIjoODX4ZGewfJQ5B21LTQHPBHfAEmnAJIzkY/Trf2RnsF9kGxDqQueiG+ApFMAyZvg+TK1tiR7BqOOmftpx1ItAk/GN0DSKYAUjPc+OqVOOjN4UqTUEo/7tQJPxjdA0imAFI3zQZlqeyaw1kjNqS50Mr4BkQoBocQVb6jKHNKeQe3s228Z5ER0AyDdAULZWN8ebVyEzgxqneL4UuYmORnfAEmnOJAq//WkfNPODDCCLfZ18R/wzvgWSBoFkAo3CrNVpo/0DB448snyfyY8Gd8ASacAEubJ5DzvbP8mM9irtmYuu0x4Mr4Bkk7x8eGw3zrW7l0GSUQGHQ9fuN/hJzwZvz3SFQWQuAvvK60TSEUGw2rFOZvKhCfjGyDpFEBy3Cised7g/k1mcCvEVmsXPBnfAEmnAJI3Ho87LoMkIoPVRvGgfy4znohvgKRTACkYhznqS4dEZNpjEMnGtm6txJPxDZB0CiBFQ4HWRUh74thkjK9ndaKT8Q2IVAgIJSwgzu9bic4M/xr+TtFCJ8MbEOkSh1FFV37pkoiMx8Uscv91shviWyBpFEDKpnAdvE4gFRl0PWri/r4f8Yb4Bkg6BZD4YEJYZ2v1hIvORBdcSROdjG9ApEJAqJoUIjedV6Iz43IyhbtmhUY6Gd6ASJdgI2y1HviVa3ekZzBvjf9lo5/sZPz+SBcUQMIch5TzWh/1DM6YGNyxiKbEk/ENkHQKIDluEmY8ur8gnZnWNws+HDWSxBPxDZB0CiB5bhQWWqePiox3Hk+MlvaYqChriG+ApFMAKXCrsJR5e/s3mcEdNb6OxRRHvCG+AZJOAaTIzcIa0zpE0jMu83GFeGzoK/FkfAMkneJAqtG6dQ6pyGBSDffSiNKEJ+NbIGkUQOKmTrR13uD+TWba8/2Om9VuxTvjGyDpFEDiQ4qU4jpE0jNYLCpy1JYJT8Y3QNIpgFQMel5r3/ZM4ELvS06hTnQyvgGRCgGhamL0dt7d/k1m8Lwxm/iaRzoZ3oBIl2Aj7LbO/YuwnkY9g+nruSbv3WQn4/dHuqAAEhkfQ1inkYoMzhisgOjLhCfjGyDpFEBy3BaMtE4jFRl0YmuoeIhtwhPxDZB0CiB5QzEWv46S9AwG+Yuvx0qREk/GN0DSKYCEPcS59l27JD2DtpD1x1wIaSfCGxDpEAdRjdmtk0hFpk07TnzANNHJ+BZIGgWQIjcKc3VKjXRmMDybU4nVT3gyvgGSTgGkxM3Cwt2KFenMtO0PCMc44cn4Bkg6BZAy7gX5dRKpyGAHbW4N5WO4TeDJ+AZIOgWQionJ2nmL+zeZwb5+VPm4/Ywn4hsg6RRAqhjFz7T2SXrGBdyfjZXchCfjGyDpFIyEHdcThXUqqchgzRHvSn61ADqejN8f6YICSPwnJ0fzFvdvMoOn1wP/ovbkkcST8Q2QdAogOUPJrbtr9oS37fl162mkG+IbEKkQEPLGJh/XeaQi43LgX8X/yQPdEN6ASJc4jGoKbt7e/k1msNUYDtHPdjK+BZJGAaRgSsIexyvSmUEXNuWAW7UjnoxvgKRTAAn7Y8aoTCPtmTbPv4YUaMKT8Q2QdAogJZO4f6pMI+0ZbMpiCzcZ3YQn4xsg6RRAygYLrtV1kKRnXMbzfpGO4TaBJ+MbIOkUQCrcLMypKtf/M+MJD2rHfNy37WUN8Q2QdAogVW4WFq/MJO0ZFxPGHgvVEW+Ib4CkUzASdl3nV5e1TuoZtIzKa6dxaSfC9ye6gAARcbuw5rJ2SHoG1zDu0fpjBqCgk/ENkHQKIDmDHTLX79qZwKCac8kedyQFnYxvQKRCHEI1E2WV6JVBzyNT9G3hWkEnw1sQaRIw8qZkKnntkPQMNhyLNYTjrq2wk/ENkHQKIPHJkPkCvnZIesbxcdWMgf4JT8Y3QNIpgBS546Xs1tYTmL8Won0JdTkR3gBIZYBPMjH7mhSgM4O1xnPkEyeOcDK8AZEuASP+IYekTCHtGSylwV8qe3RFhJ2Mb4CkUwCpGJ+jn7e3f5MZdNGs5UZQnPBkfAMknQJI3MTJeL5xRTozmL3GPf183EISeDK+AZJOwUjYcT2nrEwh7RmcPORSbCuySzwZvz/SBQWQiNuEOcxb3L/JDNbRDtXzr5vwZHwDJJ3iQOLvC//lGtIrg+tZjjYd90cEnoxvgaRRAAnLQJaidNl6pt1P47Mn1glPxjdA0imA5LldWKMyjbRnMMmPW47HUtEST8Y3QNIpgBRMKpZ7GSvSmXEJW9i4dDQBJJ6Ib4CkUwApmlisMqx9JvCQX+Wa+hhDEnQyvgGRCgGhZPD4lTKHtGdQ8TgsyxYnOhHegEiXgBHW7nF+fijgTWawwA8frz+6JMJOxjdA0imAVIwr2GxlRToz6PFzHy3GMuHJ+AZIOgWQqqHiszKJtGfQ00/cm3V1wpPxDZB0CkbiK7gtISjDSD2D0X7Ll/zjFpLAk/H7I11QHEhc8ZJbL209gydq8XTN0SUReDK+BZJGASQypcSiTCLtGaw5mgJOnwlPxjdA0imAxG3Bgk1XVqQzg+GQRPy1oglPxjdA0imAxEdQuFuxDpP0jKdkMiaLlhFviG+ApFMAKXDLMFdlEmnPoD/L/7DH4j+9rCG+AZJOAaRo8Cj/vNX9m8y0/mzk4uOMJ+IbIOkUQErcNKzersMkPYO5kIF8PJ7ZFngyvgGSTgEk7HFgrTKRtGfwz0K5JprxRHwDJJ0CSMVQtVmZSNozmMFGMSRHE56Mb4CkUwCpGltJaUq+4mgWEYXkJzcR3oBHQzhwasVemRrPK8MRw/3XMLHJ6BZAGgQTYQ0oPP+xnkE9g5tF3MHHqhojnYzfH+mCAkho4Hj+eUU6M20DBOvy8ZStwJPxDZB0CiA5frG6VmvPYGJW9DW2baIknoxvgKRTAKnNuSrK9NGeaRhYcywpeO/xDZB0CiAF/oErFqVOOjPoyWIg5LjwCzwZ3wBJpwBSNL6qj7T1TJt5xMUteDK+AZJOAaTExWDa7Ip0ZtAzo0T2WEBa4Mn4Bkg6BZAytwpzWjdFFhmM05aSS0oznohvgKRTAKlww7D4vA6Q9Ex7sgbz2cqEJ+MbIOkUB1KtVXmE9Exw+W09regmOhnfgkiBgBB/XSqfFUqFdGZwlNHZ1wpSnU6GNyDSJdgIm65bG9I6ONIz6Opzz8Mfo9rCTsbvj3RBASRuO1siZRppz2B6baqpHPdsBZ6Mb4CkUwCJ286WlMb2mcCzxxS5f5YmOhnfgEiFgBDXKdbFuFbZPYNnj0KO9lgaqdPJ8AZEugSMuN1svVOmkPYMpo1kG3Kpk52Mb4CkUwCJ281ciDKFtGfwvGhMJbky4cn4Bkg6BZC43cw1cFiHSHoGZ0wl60qe8GR8AySdAkjcbrbRB6XWPjOYyR64VXQsRC7wZHwDJJ0CSFgBKlllCmnP4L6sLxg7mvBkfAMkneJA4lMhB6VOOjMYhEz8u45FNgSejG+BpFEAidvONge/9kh6pi2BYEs9dv3rZQ3xE+nL3zgcuHv+7skcOCZ6fv388qtv//uv33z7h69/+/zmh8cM+fGI+FhNPSJvj8cfH97yqXrYDi8W8fkta8l6If3FooR/WPKXv6H2gfJxWa6KczrAlx/Cu337UPGRPr8eXvT4R+94PLgTjEWD+D/jcSQbx0N7e4WGP1S8e4jzx/Pn9vH0F8y/4h+fze1DJXyoz++e9NA+VX4Xts2quJU/fAI9PqOKE8Zyi/v5l6P49kXgH79+RR/fP4gdv7Cc8nhOAKXZFGx8UmVBR1zgNx8fv/3w+PJf6cm/6MOfH5bf8eE/H//+/JX99fM/nh9+9/iXD4/f/9IIwRFmxkSs1Duc4D3+CRAChaO0krE4+Y9BSOFzKnCf3HtHafman/FPoZCwly2Xdqyw8WMUyJXPyBD98SxCjSODiH8ChugSd8W4tFD9j1So7nMqVD5ZHQUs8Dko9PinUOAeBbXSrM/uRzFwo/ozMsiK3eNZp0Iu94q9h35yzUjEHan+t/Np4bCMAV9cuNtwMPDB8x+BtQ2YoJv8in79/PDdg8/KWrAkVEzuUELdyQniLi3/X+IqlzOBy+WOfy8E19FeSjCllppDwrMmRymuJVj5e/x+m7nF8rTv/8QmOdb5gAK56Fzb3/fNxycX/cVX3373p3/7+x//9Lcfvvj417/9/YfnV//1/P3n/bASv5lbWSn0D6uHfvqHlYYPCyso8ZeBG+Y51s/yYdU7fyaVmCta6/pn0kOf6jPhTlO0rMs9Ae8+y2eSbvyZeOeYC/tdnp+JCH2izwR9DuIGcwr8P/GzfCb+zp9JxEA2XwT6hUaEfvJngo9VfCaB//bqIvdicFP4Z30mGKbwxWLF/Xz9kdj3j+TnWaKH/vjR/eh/1kN/DK//n/XQ+c8zkQlxF6MWvrp4l111r84TViHAChAy8yYz7TPlwtotZlHWED976Dtw6SjgwgwOzCVbuc5MG5aP3FicGIf4Vlw6CrgwlyMkDOnPXGcGLMnltuHMpPge3gpLJwEW5nSg1lqxzkxbntdZ13ZVk4gyvhWXjgIuzO9INig115lpi62lkFOZGGV8Ky4d5eCqNWUM969crwzGpfH8J82MMr4Zl4YCLsz6yAED/zPXmWnTQWs+JhZLRhnfiktHYa5suY1WaK3nzwRqqIT2sJ8QZXwnLJ0EVu3mT8FNkRnrzOD+SOLf1aYaCUQZ3gpLN4GWw4JrEStLzFpnBhOzQm1P+Y2KMr4Vl44CLswO4S6jW7nOTFvDpjpnaWKU8a24dBRwYZ4IZhktWu8JT9ngLuTr3DpLGuJbYakksMJ0EUpu7fv0jLdY9D/bGkdEGd4KSzeBFuaN8ImiXBLPTHtIKaTcZt2Ksob4Vlw6Crja0Ix1Sr11ZnD7PAZ/PKMkyhriW3HpKODCXBK+0q2dn57BkHXwIbiJcYhvxaWjHFzVhkBr56dncPXjq2BpWyhIRhnfjEtDARcm50TSmvNnBvPiqwshhZlRxLfi0lGYq3DznN9H68BNz+BuD7esYkgTo4zvxHWBAi5un9uEdTgWrjPTNg+qNpWZUca34tJRwIWbXNnZte7qGbSvcgq1LdwpyhriW3HpKODy3ObM1a5N+p7xLprqOVsmRhnfiktHAVfgRmdRzq1XHJPEuXOYjiZqL2eIb0WlgQAqcnOzcjkr1Znx5AwXxlc/SThEt6LSSYCVuLFZa10HbXoGrfac+DJYJkQZ34pLRwFXNpZsqmvnp2fwbBSlcAxD9KJkeCssneTAqkS+rF2fnvGYi2BdSjQiDvHNuDQUcBVTiA9pHbXpGVe5i1NS24RQFCXDW2HpJMCqWP8cL1ywzgxa7IHbV3kyFOGtsHQSxqr8YvKhrN2enkFlXkNMmUbEIb4T1wUKuMhECpTXeqtn8HBnoZzaMsWirCG+FZeOAi5nAoWS16Zpz7ga+TvnvJsZZXwrLh0FXN54ijGv3Z6ecbWikeVrnRlFfCsuHQVcmPyXXFbqrjPTBv5ics5PjDK+FZeOAq5oiLCqw8p1ZrAmTXY5HLMhJKOIb8Wlo4ArcYMzp7R2f3oGXzruE9ajrygYZXwrLh3l4KpU+O/XuF4Z76i1HWhiHOKbcWko4MISvVUZDzwTnqwpWOk4TYgyvhWWSgIrPhbCVKwV68zgGxfIuVfF9V7UEN4KSzeBFqoeG6JSb50ZtK2If82r3hKKIr4Vl47y8REs/1pHFJdhG5HBYq0hcufQT4wyvhHXFQq4yARHZZ2wKzK4N+1ytS5MjDK+FZeOAi7HP7i4TtgVGW8x/kA2T4xDfCsuHQVc3jjMBlmGbkTGcau0lsgN+JFxiG/FpaOAC48zcyt96QCJTHvm0RdqddTAKOJbceko4IqG6560ztkVGYxpka3HbuWirCG+FZeOcnBVPgyv1F1nBnOZK/mS4sg4xDfj0lDAlUxxCXvcLFxnpo0vu5DtwijiW3HpKODCgEJah7p6ol3/nMWDtwPiEN8KSyWBVcFTdMEpFdeZQVvU53ysI9iLGsJbYekm0Krc4CxY1mPROjOY+cCNUpvCpCjjW3HpKMxF/E7Hb1s7QD2DW64l5Nq60aKsIb4T1wUKuPBDVbqLZwIN0RS4UZpGxCG+FZZKAitnHDZaWAZuRKYNlUaXjnpLIIrwVli6CbS8IX/sUjRrnRmMw4fkQsyTooxvxaWjgCsY6ymR8k08M/jKWazwlCdGGd+KS0c5uKrns2UduOkZV60hrq3irNjDm2FpJMDiy5r3dh2eFxlX215yPuUZUcS34tJRwJVM9j5bpeY6MziJsP5Le0R4YBTxrbh0FHBlPoIQ7Dps0zNt0je3sVwcGYf4Vlw6CrgKHt2x66RdkcEdxMI2lSZGGd+KS0cBVzXBx7xO2xUZ/ubxYZYQ4sQo41tx6ShYc9ViNd+wPj0sMmChakuoK+MZ34nrAuVYotZpjyz2BF8D21KjbX+1AVHEt8JSSWDluL2ZyzprV2RwN98Fh3vVA6IMb4Wlm0DLc3uzxLLWWz2DJ8ewI0R7FEqUNcS34tJRDq7KZZS1+9MzbQeN6uloRAhGGd+MS0MBV+B2QK3rxF2RwWMYxcZAC6OIb8Wlo4ArmhxsykrVdWZwB5G/dMHOjDK+FZeOAq5kUuC6aO0A9UzbNyBn59PIOMS34tJRwJWxPL9dJ+6KTFvHJmO27sg4xLfi0lHAVfidfNxK3XVmjhEtW16trs4o41tx6SjgqsYHH9I6eNMzGF6uziZaGEV8Ky4dhbk8WpqB1om7IuPRdqBSXq3Us6whvhPXBQq4yFAIHFu5zkzbkjNR9DQyDvGtuHQUcPGxhRjjOnjTM605GgPWOhgZZXwrLh3l4KqBj2XtBPVMa1+5TGVmlPHNuDQUcHlTQtLuxvYMdqMi7uwc4/SCUca34tJRwBW40ZnXJ1p6wlvst5Jje1xYlDTEt8JSSWAVucVZfFh7QD3TnrkrFAsNiEN4KyzdBFqJG5xYPGrVOjN4Eh3L+R+DzlJRxLfi0lHAlfmdNQel3jozbXt4H2yeFEV4KyydBFiFTw+LDUIWrDODTrRL2NJgRBziW3HpKODiyicSKdN2e6Y1REs+Nl0RZQ3xrbh0FOYKtm114teaq2cwK9CWSH5mlPGduC5QwMVtzOjQg1m4zoyr2F0ktnUOpKIIb4WlkxxYNXqnzNvtmXb141aDjTOiiG/GpaGAy5kSuZC15uoZbOUcyBafJ0YZ34pLRwGXNzlihYyV68xwMxQLRKRcZ0YR34pLRwFXwBYF65rqPeGt529fouO5xV7SEN8KSyWBVcTiGJaUiuvMoK2QUtutViIO4a2wdBNocdWDVZDWfnXPtEXLckzH+Hwva4hvxaWjgCtzczMHZeJuz7SvnG9bi0+MIr4Vl44CrsLNTWVd4p7g38JNhZhf9dZZ0hDfCkslgVXlxmYpdh206RmM+2FL7Ffz9ESU4a2wdBPWipYvazUq03Z7Bo32UiodM26EoozvxHWBcnDxC50ybbdnPO681pB9GBmH+GZcGgq4sNeuLXUdtumZtvCpy/V4OL2XNcS34tJRwIVFvgkD7AvXmcF4A3ZWPO4rSkYR34pLRwEXH0FyTpm22zMYh/dYQCNPjDK+FZeOAq5guI1Zq1J3nZn2aEEu7nhCo5c1xLfi0lHAhaUnfSpr/6dn8Mx+ickfcyJ6WUN8Ky4dBVzJ8J/ulam7PcMexnt63eTvZQ3xrbh0FHBl3JOwRam7zgwWi022plfddZY1xLfi0lHAVQylmJXv4nsCMx+STcetMmEowltRqSCQqnxoKSizdnvGU+HOdLuVKAmH8FZYusmhxd8jykqtdWYwBclh9bI8Kg7xzbg0FOZKlpubueR12KZnWmVOqdY8Mg7xnbguUMDVNsCIyqTdnnHFcry8mhC9rCG+FZeOAi7Hzc3q0lp19QyufS7Ra/UkySjiW3HpKODy3NysNa1DNz2DkfiQgj3qLsEo41tx6SjgCobfmJRJuz2Ds8hzpzpNiiK8FZZOAqxofCavTNntGdRQJZMNs2EPb4WlkwArGf63jeuwTc9gRQhyLhU/Icr4Vlw6Crj4iLPLce369AzmbAX+PRRGxiG+FZeOAq5ibObaZ9V6T2AcnpLzx4BzL2mIb4WlkhxWNWMzDA3rlXF8NmVHPpURUYY3w9JMoFW5og5FO7XOTNtjuNTXSrtCUca34tJRmCtb3KWPyoTdnsGMrUBcYJoZRXwnrgsUcJFJOa3bpvdEu39ILh3T3iSiiG+FpZLAypmYU/Vrz6dnXpujY0UbiTiEt8LSTaDlua2ZkzJdt2dau4qK93FUHOJbceko4Arc2OQjVuqtM4PFMwofcg4To4xvxaWjgCseGxSsgzY9gylInqw/bsL2sob4Vlw6CrgSNzdrVibs9gyufqWkY+coUdYQ34pLRwFXNrbYoF0WzwyWlApUwzE7qZc1xLfi0lEOrlqI3Dpo0zOY9l1SCa9mxFnWEN+MS0MBV+HvExVlzm7PtDti3ONxcWKU8a24dBRwVT6YtmLUwnVmMFGkuPLqLkpGEd+KS0dhrsIvLp77yAtXzzhuOxRfXgv2C0YZ34nrAgVc3DAvXplucyZQQ2H376OiF4gyvhWWSgIrZ7ifnJRJuz3T+tCWezplRJThrbB0E2hxK7NEb9d6q2fQifYp5eOqKBVFfCsuHQVcwfA3ytq1A9QzLuOflmKZGGV8Ky4dBVzRUElJmbTbM20rpJyOYa5elAxvhaWTAItrajwyoFRcZ+a15E/yfkQc4ltx6SgHVy3F1rX70zPYrgaPnx83FiWjiG/GpaGAK3Nzs2StDXFmHPenS+LKqsyMIr4Vl44CLhxMDUWpuc4MHiuw0ZaFUca34tJRwFXbi8s6dNMzWDswR0fHYkCCUca34tJRmKvyr622KFN2e8bF3G65HmvQC0YZ34nrAgVcuFPfFkBauM5Ma7sjnidGGd+KS0cBlzNchFOmOPcMGgwx+HKsniQYZXwrLh0FXJ6L4fNj7QL1DLrRkev0Y5ReMMr4Vlw6CrgC1rnTtlrsGfQM+VeVMDPK+FZcOgq4ouEmlHI79kygucDd6HS0ugSijG+FpZIcVrVGq8za7RmXMLZMx94Z3VBEN6PSRGCVTKkxK1N2ewaLu8Uccw2ToYxvxaWjgCubXFNI68BNz+BRu1iCPRbVEIwyvhWXjgKuwi/OFNfuT8+AJdt07Es5Kr6Ht8LSSYBVubGZizJpt2egYpP1x9qeI+IZ34pLR/n4iLj1XEuMS80lMm39rcy/oMyMIr4R1xUKuPiHWl1cOj8iwwGu2ytfCidGGd+KS0cBl+Ni+IiXzo/IuJTbSGB7IEoyyvhWXDoKuLhtbi1X3CvXmXGZ0CbNrSEqGWV8Ky4dBVzcOrfkw9L5ERmPrU757a1PKMoa4ltx6Sjg4va5deuKQD1xrMAVXUgTooxvhaWSHFbVuuyViuvMoOGeUwxhRBzCm2FpJtDi1rnlE2UZtBEZlyv/LkrtIfRBUcS34tJRwIU2eaB14q7I4KnqyvVTpZFxiG/FpaOAi9vnNqzD8z2BNZKIEn/pRsQhvhWWSgKrihHQ6JZBG5HB3O/gkm2zkASiDG+FpZuwFvE7+VjWSbsi04ZnAvE3blYU8Z24LlDAxT/w8a6TdkUGc5kzX/zag5yirCG+FZeOAi5um9uc1v10RcYlb7B9VJuGJMoa4ltx6Sjg8gZrfNPa+ekZlyomUHJmZhTxrbh0FHCh/8KvXi+LPYOVbCjxEaeJUca34tJRDq5qa16X2hWZNrOGa/eQJ0YZ34xLQwEXHzFZzL1duM4MroDVu5hmxR7eCksnAVYyfOC0TtsVGbSuYrH5GIeQiCK+FZeOAq5sEhGXs3KdGdybTiV58hOjjG/FpaOAq5iI3bwXrFcctVOM9XVidUER3gpK4wBT5QaTp3XCrshw74bfTDT6ieBWUDoIUzlr+AtU6npO9QyWTK825WO4RhDK+E5cFyjg4j+ZQqzrcE3PtDuHLhS3MIr4Vlw6CricIYpunawrMm0705hKrBOjjG/FpaOAC7ssx1rWeqtnMPko2+KOW2SCUca34tJRDq5KKZV1yKZn8Ewi5pu22/iSUcY349JQwBW4oZn9OllXZPAoFPlQ25NjA6OIb8Wlo4ArckOTW+NK3XVmMCmX+P2UZkYR34pLRwFX4tZmyXkdsukZ3N8hrDAfZkYR34pLRwFX5oYmNgpZuc6Mq9aUiPUpJ0YZ34pLRwFXMcFZUi6M7wm+Bppa6nvN1RFlfCsslQRWlX+wJa3jNT2DvrOtLBRHRBneCks3YS1vscB3XDdLFxmHPUWq83lWlPGduC5QwEVYytSt03VFBg9xlmqzmxllfCsuHQVcfGzOrZv99ATGZhxxg8HPiCK+FZZKcljxdynFtZLvGQzQ5FSzTyOiDG+GpZlAy5viglcm6/aMi9yuIsI6saOijG/FpaOAK5jsoo1KvXVm0IVOOXAnemKU8a24dBRwRcNHksPa+ekZl5OpNdhME6OMb8Wlo4ArGW6fB2Wybs9gPIuSy5QnRhnfiktHAVfGk78UlLrrzLQpSAmbR02MMr4Vl44CroIfSlgHbnoGY/HccAjVT4wyvhWXjgKuykdcojJft2fQhfYeMyAmRhnfiktHYa5gucmJhsHC1TOooyKXX2fFHt4J64IEWMQtzlr9OmzTMxjNKrktIDgiyvhWXDrKwVU9H52mdSRcTFg+kI7pugJRxjfDUkhg5QxXO16ZrtszmE+TcgxHZ7EjyvBWWLoJtLzJ3lllLkTPOK7Xs8eqBpOijG/FpaOAK/AROIy8LFxnBr1C7uTEttyPZJTxrbh0FHBFw/2/oEzX7RkM0FC1qcyMMr4Vl44CrmQCJ5Th+Z7BYlIlUvaToghvhaWTACtzCzMUWjs/PYM5zJG/fTVMiDK+FZeOAi7uv/gYlcm6PeMKYX6uL5OiCG+FpZMAqxryic+PFevMuFQwGdCGGVHGt+LSUZgrWmM9v3zt+vSMw6535EKgmVHEd+K6QDm4qs9JmazbMy5GbAZrj3peMMr4ZlwaCrgIq6F7u9ZcPdPOopDdMYFEMMr4Vlw6CrgcGuW1rsM2PdM6OlhjcVIU4a2wdBJg4QhqUqbt9gxGZyp3pl2dEGV8Ky4dBVzBxMClKDXXmeGq3NTsg50ZZXwrLh0FXJHfSUor4ky0J/UT+WNKuECU8a2wVBJYJcNt86zM2e0ZLIIXPdGr3joRZXgrLN0EWtm44EJR6q0zg6WkUuD/nRBFeCssnQRYxVDwVNZBm55xDoMztpQZUca34tJRwFWNDb4oE3Z7Bvemybl8PKgoGGV8Ky4d5eCqIURlwm7PYD6gQ80+M8r4ZlwaCnMla0qILq/DNj2DAYdK0R03YQWjjO/EdYECLjI5xJrXzk/PtOX4Q/YxzowivhWXjgIux9e2lJQ5uz2DhmiqXF2FiVHGt+LSUcDlucGZfVrrrp7BxO9cU2mb5kpGGd+KS0cBFwYTinYftmcw/Jco51ImRhnfiktHAVfkNmdRLoxnwvG/2Id8nBFFfCsslQRWiRsCNUSl4jozeHgsJewwPCLK8FZYugm0sqFoKa7dn55pvcIayjHxTSjK+FZcOgq4StuuR5m02zO4JcZA6PNMjCK+FZeOcnDVqJ5c7wk89WpLyHlGlPHNsBQSWFXDB+7COmzTM6jNCWuA5xFRhrfC0k1YC7uycfdFmbDbM6jOi88ph0lRxnfiukABF5kU+Y9f662e4e4z5uiG486PYJTxrbh0FHA5E2Pwfh246Zn2GCeXfsyeF4wyvhWXjgIub0KMVpmw2zN4MrF6bmWViVHGt+LSUcAVjI+cXzs/PYPbFvxrwrGwp2CU8a24dBRwReO4R+PXgZuewZqBlSwdbS7BKONbceko4Erc4Mzk1v5Pz7QudOXCysQo41tx6SjgytzkxHbxK9eZcaFw8SnRzCjjW3HpKAdXjdyOUuquM4NnX6u33HyYGGV8My4NBVyFG51VmeR8Jo7tmFM41hIUiDK+FZZKAqvKLc5alTm7PYOZ3zmWmuKIKMNbYekmrFX4xckmWuutnsEAjQ3RHTMFhaKM78R1gQIuMjGRp/Wb2DNoinr/vlC/YJTxrbh0FHDhkVZnlQk3PYNuNCaLpDwxyvhWXDoKuLzxyWW7Vl09g0XTHZe8MMr4Vlw6CriC4aMOymY/PeMKFv7Jx+LgQlGEt8LSSYAVDXEzQJm02zPOB2zYedTzwlCEt8LSSYCV+NhCrkq9dWZc4C40eX/MuBGIMr4Vl45ycHGbKSirAfUM10yGi4vH7DfBKOObcWko4MqmcG9PmbTbMw73x/CEQZ4YZXwrLh0FXHwwKSlL7Z4JBjGFS8qToQhvRaWCQKrivjPnV6ozw78AQ8w+ToQyvBWWbsJalX9tKk6ZstszuGnhc0nHcoJCUcZ34rpAARdxU7NUZdJuz7TbrZn/GSZGGd+KS0cBF3f2UlXmvZ2J1mL3lY4hG4Eo41thqSSw4gPO1ue139MzHDI1OJoRZXgrLN0EWsFQJqtM2O0ZVzFJxMWYJ0UZP7m+/I3jA3+65++ezIFjoufXzy+/+va///rNt3/4+rfPb354dFLJ+VHGPSYGH/G3x+OPj2SjyeOrj9D8wquylwLmV4uCrsr+8jeEY7N8bJbb5Tkd6MsP4d2/fbD4WJ9fDy96/KN3PB4h40/Nxod3GOzyfJxlPfwmw5QLHi864u39Q4Q/nj+3j2cq+h+fwu2TJHySz+/4v+2jfIwfJQulXNuEZPH39ujwVzChOEWsofj8y1l2+/HrV/Tx/YNY7QvLqZj4TERhLqf65HOuYBeSbz4+fvvh8eW/0pPo+eHPD8sv//Cfj39//sr++vkfzw+/e/zLh8fvf5FjJuKyuFa1bjhoEf4ER01kX6X5nH7EUafwix925q57KtGPR31GP8VBY5YoCrMBt2T++VGTK7/0YTvvDEYRYx2OW4Q/wYG7tkcFl1Yidor5EQde3S9+4NzmcjG3x8rkgffwpzjwdguAS+PGsP8x57mL6Zc+8FZeKKZUwpSZofrt4Z96xJn/moDhaEeBL9e+vg6WD5EvRSVHHGg/8l/Rr58fvntgtVD+XkSKyR0WqOc4Qd5E/r+UcPDPYBzZ6nshOJReSuA/vtQcEhZHkaWw5ff4/Zav67B+/ZOwo7nzAQVy0XgiIOPDeXLRX3z17Xd/+re///FPf/vhi49//dvff3h+9V/P3/+SHwk3/Pn40aoZPpIe/snVjuM/J1WOs2K11X2ez6Tch77iIUBrI430Pfyz6LlNiUYZ94a9+yzy6T7yxFeGjJkwZaQX8Z9lT1RMsIH7PTGTwOe6HRXxL6Effry+L7VZ/h/Tx0JJpcS5zhHxn6cfPBZzdOyE8f7Pce67G5373CG33MSbqx0R/8n6BfopY6Ay5MRfgJ9Z82SGw3KJ3EHO1/jn1fYnqPnj+Hq3//FzBhSG1z/+pwMKWNy2uvT8grgVg8Vb+RpQWr8ZC7likZkx8yYyJWIdmvwaqnovSUbPwYSb+lwpsA8Z6y1f5Rae94THufMaO+/lyOjtdXSDhtOqwaLpvDJcX6YQQtvV6SxIBjfQURGYp63NmjGTaOY5M9w9IR9D65AJNBG9vc+FAvu0xVgDKV+uM8MHGUK0IQ1qMnp7nwsF9sHqqx4LIs4874lYsFhYKXU0E9Hb6+gGjNMWW8VjtovOmcFpkrCR6GAmgrfXuUBgnra6asRcoJnnzGChmMIt2QlNRG/vc6HAPlhPNTqrfLnOTOR346G1OqqJ6O19LhTYp62gWjDfZ/Y5M2gLVl/aXCehJqK397lQYJ+2ZCpWD198zgxaOc5RWzWulySjt/e5UPj4yFgjNWNtoNmnZ7i9w1dzakvz9pJk9O4+VwrNh7ubta71T89EdHZDaA8k9JJkdAMfVYF92jKoqaxdi57hFk/hd7fb2b0kGb29z4UC+2DhU1Szq8+ZwXeKSpu5K9B68PY6Fwasg/6ltUWpfc4Mf4+sL1wnD2YyenufCwX2aUubZuXL9Z6I0WAxCJtGMxG9vY5uwDhtJdOQ155Fz+A0KXzCuMFMBG+vc4HAPFi71FFWap4zk/g3ZeesH9Bk9PY+Fwrs01YrLXkd1egZtAZDsBNaD95e58KAddripDGt/Yqe4R5ozsG26bW9JBm9vc+FAvtgNdLgklL5nJm2z6217YlxoSait/e5UGg+NYSa1nGNnuEWYCLKblST0Q18VIWPj9LWG01p7Vf0TMJDEtb6OKqJ6N19rhTYBwuMJrTvFp8zk1xbHqVJCDURvb3PhQL7YEVRfuM6rtEzGCukUloPS6iJ6O19LhTYpy0hij3GF58zwxfy7Kg4GtVE9PY+Fwrs01YNDWvj8EzgaZCExTwGMxm9vY5uwDhYJLRSUCqfM4MlfTGQmqWZDN5e5wKBedqqoOrJc2baKlal7S3RCxLB2+tcGLBOWwQ0hrVf0TMpGf5Hbvs895Jk9PY+Fwrsg3U/ya0NwzORrfHcU29b3vRyZPT2OrpBw6mRMNNJ0Xll+DTJiWLN0kwGN9BREZinreuZvHLynBnuP9SMja5HNBG9vc+FwsdHxUKe3ivToHqG+w/8daK2/5ZQE9G7+1wpsA9W7gzWraMaPZMJ42DH8lK9JBm9vc+FAvu0pTrzMv/wTWQSFtStbYHYXpAI3l7nwoB12sqcQbl09QzOk0Tc2hnMZPT2PhcK7IOlOBMHV58zk/k6xb2Htl6UUBPR2/tcKLBPW3uzKNMwe6bNX8nHGqa9JBm9vc+FAvu0xTajMg+zZ7gVGC0mpw5qMnp7nwsF9sHqmsXROqbRM9kZn2toa2YJNRG9vc+FQvOpsShTNc5EKsY77mqFwUxGN9DRDBinrZ+ZlHmYPVM4WkpyTprJ4O11LhCYpxpsfWyVuufMoJETSjpahmdJMnp7nwuFjw+y/OJE6H7PQCKFRVpxnfKDm4zeXejSAURtWcykzMUUqWMd29yGMSa59/D9jS4gYNTWwvR1rYZEipvL1fHVnCY6Eb6/0QUEjLACpseT5KvRmcoe88Nc26VQ0onw/Y0uIGDUFr7MysVepPiEcSnhpulIJ8L3N7qAgFFb7jIUrT46UxhWJVvcJNej9xe6YIAQVrjkf6/jHSJVHKa2xOBGOBm+v9EFxGGElT+VOZoilauJlIutI50M72CkQsCoLWcZi1YbnakcsSALf8kmOhG+v9EFBIywimV2ymRWkcIUVmwrO9OJ8P2NLiBg1NaurMqcTZHCds78bjfRyfD9jS4gsBhYW7AyZaU+6ilskleoVjfSyfDtja4gYITxZu0uYs/wV6oWrE8ywsnw/YV0BgBhQMxaZfqmSOVk2uJZcYQT0fsLXTiAqK1HmZUZnCJVvMFUsjZwLwqT4fsbXUDAqK3lyV0vxehMcdfe4TlmN9LJ8P2NLiBgFI3NDsv5rUZnCjeAHH+90kQnwvc3uoA4jGp2RZnNKVJoKGY+4plOhHcwUiFghHXifYxafXSm2savFF9XtbMwGb6/0QXEsVBqztwpVfohPZWDyRhznOhk+P5GFxAwKiblUJXJnSLVnvapJeeJToTvb3QBASPut+eYglYfnSl0OKL3IY10Mnx/owsINnL81px8UPohPYULmKfXaXSWJaK3F7pigBBxezArS5v0TOE/rXjXFicTRcnw/YV0BgA5bgzm7JWqqKfQI0tU2r37XpaM3l/owgFEHgtm4mmelehM8eliC5dQRzkZvr/RBQSMArcFKymzPkWqYLmObGOa6ET4/kYXEIdRzVUbxD4z3ABy3P6JfoST4R2ENAYARe6y2+iUEZGeQp/V5XoM83c4Eb2/0IUDiPiyXbh+0S5oZypXTLU6triXciJ8f6MLCBhlkwpVZQ6oSGE6WhsFmehE+P5GFxAwKty6cYmUEZGeyrV9qcIk16P3F7pggBC2K/CelP5HT2U+ZFvL64rW4UT4/kYXEGzkrfElWGU2qEgxRqyB0kp3hm9vdAUBI/6bS8jK0pwi1aZehXTMyBJ0Inx/owsIGDlDfKBW6YT0FPaZjS6mNNLJ8P2NLiBg5LknkUiZHSpSGROK7eteSC9Mhu9vdAFxGNWSuCDV6JXKyYTKX6qZToR3MFIhYBS4Pajd3X9PcHfMupBKHtlk+P4+KgJ0IrcEuZ2jdEF6Ct2xEilkySaD9/e5YIBQ4oZgKVU7g85UwQLTwVYa4WT4/kYXEDDitg0nqjIa0lNtcyyue9xIJ8P3N7qAgFHh+tY6bWZoT+VosBD3MR4i6ET4/kYXEDDiHnvljoRWG50pjOTXiEvYSCfC9ze6gGCjgHK4G6GMiPRUe4Ao52MaTS9Mhm9vdAUBIzLcdffa3NCewkTZkEJbvkEUJsP3N7qAgJEztnqblfqop7CYQ3CpLWoqCpPh+xtdQBxGtXr+wqhGrxR3yPCITHtYT9KJ8A5GKgSMvCmV36z0QXqqJL6AWTruWws6Eb6/0QUEjEK7NimXtTPTFovL6ejJCjgRvr+QzgCgyC/mVqAyINJTWLUpco8jjnAien+hCwcQJRNriknph/QUny6+BFfiJCfC9ze6gIBR5tZgdtrc0J4qeLChJDfJ9ej9hS4YIFS4LZiVrTl6pjoTuH/Wlm0SRcnw/YV0BgBhi5aSolZXn6mSjQ3lWBetlyWj9xe6cGCiyD/wBVybFdpTBYtWcp81THIifHujKwgYcVvZWhuVmqin+ITh0yV7N9GJ8P2NLiBgxG1lPs6g9EB6qjhTHH+t0kQnwvc3uoA4jPjFQZsV2lNoAtVsj0uaoBPhHYxUCBhxa9k67Bq1Gp2pgkFZG47n0gWdCN/f6AICRtxctg5726xGZ6py2GZ7jKz1wmT4/kYXEDDi9rL1UZsa2lMFW3bUcIzQ9sJk+P5GFxAwwnq5wXmtPjpTWH0YV3m/0J3h+xtdQMAot03ovTIi0lMFA0U4YyY6Eb6/0QUEjLjRbGPSiN4zJRjubqRYJzgRvr+QzgAgbjLzl0WbG9pT/I3Cww1lghPR+wtdODBROq5LThkR6amCNdXs+/esy4nw7Y2uIGBE3BrM2Sn9kJ4quIzFTHmiE+H7G11AHEbV8kVc6Yf0FDAcNk9c6d7DOxipEDDi/patpA1h9xQ3FF2J8fiu9cJk+P5GFxAw8vx9qYWUfkhPccUTyMe2bLMoTIbvb3QBAaNgMPtMmx3aU9xxTaG8+vyCToTvb3QBASPsFsQ/avXRmSrF4C71cddR0Inw/Y0uIGCU+PtC1Sr9kJ4qFRuW0jELQtCJ8P2NLiBgxEdDLmmzQ3uKK+cY+belkU6G7290AQEjrAiCe9CK0ZniEyZlX1/10VmYDN/f6AICRtg90deqjIv0FF/AoqV8PF8k6ET4/kYXEGyEPcgpJG2WaE9h2ozN4Xg0XdCJ8O2NriAOo0rRK/veixQu8j6AZaQT4R2MVAgYkeErk3bpPzOVjOeOawgjnAzfX0hnAJAzmetgbYpoT1WP9YupxhFORO8vdOEAIuzmmsOy9/ubTGEYLZScaJST4fsbXUDAKGBHTirKuEhPYSCNyy4znQjf3+gCAkaRW4OlaDNEe4q/VNVlR3Wkk+H7G11AwChxa7BGbYZoT6FThhtoE50M39/oAgJG2ThnXVbGRXqKTxjueFhLI50M39/oAgJGBT/UrPRDegqDjfWYQ9uL6sH7+1wgwIf77I77p0ofpKcKDrnao58m2ET4/kYXEIdRdc4nrS46U1zppGDf6+tOJ8I7GKkQbIQ9yx3uQq9GPVWTKeU1z7iXJaK3F7pigBCZ7Lx2QTszXCnnkNwx30iwifD9hXQGADmTXAjLBvFvMsUWzhXyYYY7o/cXunAAEfe0XKSo9EB6Cl16+97T74XJ8P2NLiBgFExwsWgzRHuqVOP5dx13HQWdCN/f6AICRtF4l9TT6D1Tnal8BbMTnAzfX0hnAFDC2kQuKKMhPVWx945/bVvU4UT0/kIXDiDK/AO3kJXeR0/VaEou6di7QMiJ8P2NLiBgVLgdWNKyc/ybTBU8qM9tITfSyfD9jS4gDqPqqvfKaEhPoavhkz3W7BF0IryDkQoBo2qKt1abHdpT/K+a+euVJjoRvr/RBQQbYX9zb7M2O7SncPuV/9U21xWFyfDtja4gYER8CBS8MhrSU/x+G2Oqi9x79P5CFwwQcnwmOHJKJ6SnKpYw8uVYmV/AifD9jS4gYORN8K5o80N7inv11Yec40Qnwvc3uoCAUeArk49Oq43OVM38/pSOq5qgE+H7G11AwCga54M2hfbMYKfLRO5YMLwXJcP3F9IZAJQM+VC1yaE9xSqYEnI8VnyWJaP3F7pwAFE21se07DD/JlMNo9rj2aJJ7hW+v9EFxGFU+c8n7Yt2ptCxT+E1p0/QifAORioEjNq+3labINJTjOH5X3mlO8P3N7qAgBH3SLFjg1Ydnam2N3F26dgatJc2xO+vdEHx8eGwDzqW3V3HRUQKGiG716PFo94Zv7vSJQWUiFuElZT5oSKFPeWrw55Os56I31/pggJKjluCNS+bz7/JFM4ePuTWeRWFyfD9jS4gYORxNEFZekWkGMSU6I49C0RpQ/z+ShcUUArGYQr62h8RKXyvYvTR51lPxO+vdEEBpWgokLKUaM/gjIk1ujrT9fD9hXQGACVjcSdoHR4RKZwuLnNVnSY5Eb6/0YXEgVTRlV/7JCJFfO7YaEOOM56I76CkUkApm8K1sDJTVKRwzpAtxw4Yg56I31/pggJKha/iQZme1TOw8DYVX1e7M35/Ix0CRNWkwNemtVsiUq3J6EPMfrE7w/c3upBgJOyKHvilSq+kp3ARIwouxRlPxG+vdEUBJeK3ppyVOqmnuE8bsRVvHPFk+P5GFxAwcnw0OSSlkdRTjOFSTu3Rq5HuDN/f6AICRp7bhYWUGaMi1foe/rilJgqT4fsbXUDAKHC7sJRlK/o3mWrXsOTcaifi91e6oIBS5IZhjUkZJ+mpipUyYz26bb0wGb6/0QXEYVSjdcrEUZE6zhgfjgZAL22I76CkUkApmRJtXbajf5OpWtF6xMJrA54M39/oAgJGfEiRUlRGSXqKrDNcWK002g3x+ytdUECpmBTxrhXpPdNG+9niGCaRdiJ+fyMdAkQVA2Z22Y/+TabwrSophzzQyej9hS4cmAg7pHPvIiinUU+hMZRCydGNdEP89kpXFFAi42MIygxSkWoaySbvFb33+P2VLiig5NpeBcocUpFC/ZxsPfZ2HvRE/P5KFxRQ8oZiLF4ZKOkpssmkXF2ss56I31/pggJKGH/la5XSL+kpspl/W4rtbvagJ+L3V7qgOJRqzE6ZSSpSfCljlUBxxJPhHYxUCBhFbhpmtHVWozPV7q1hUdE82g3x+ytdUEAptX0KndIv6Sm0G2sI5WgrST0Rv7/SBQWUMjcPq1fmk4oUNMiW+GoHDHpn/P5KFxRQKiYma5eN6t9kql3JomvTawe8Hr6/0QUEjKrBgDUpXZOeQpuIr2Vt89CBrofvb3QBwUbYOT1RUGaVitQxo82H14kk7ET89kpXFFDCk46Ols3q32QKg5A5HstnicJk+P5GFxAw4i9McsqGmT2DO/6O3DHfX5Q1xO9vpEOAqG1dFJVppSLVMLJ93XUb7d7D9ze6kDiQagpu2az+TaagUXOlttrIhHfGd1BSKaAUTEnYt1hROlPcEjLBcZGLnojfX+mCAkrR5BSjNqu0p4gC/5P8q+LuejJ+f6ULCihxczAlp80r7anW3ydbI016Mn5/pQsKKGVuG6ZaldGSnmqzbEopPs16In5/pQsKKGH18Jyq1gw4U2SjCTGSi7OeiN9f6YICShU70HhtcmlPoZsWXfuSTXoifn+lCwpW4h9caquuLUo9hSs+nzL2dS4JPRG/vdIVBZSIW4g1F6Vv0lOopblZdKyCOOiJ+P2VLiig5IzN6l3KMwML3I70K917+P5COsMBVDNR1oVeKTQcKfjSNvSRciK8g5EqASRvSqaSlb5JT0GjUKKjlzvinfH7K11QQInbzdlFbW5pT7Wxo2r90YOTeiJ+f6ULCihF7P2g7M7WM+1KH4s9JpdKOxG/v5EOASI+zOxr0ozOFEZquW8bwmwnwvc3upAAUjaYTqvNLu0pIo+1AHKZ8WT8/koXFFBqo7B+2dD+Taa4G2tQdkmTnozfX+mCAkrVuJxsVMZLeooomlJriX7Sk/H7K11QsBL2Wc8pa/NLewrfrBRSTHnWE/HbK11RQIm4dZjDsrX9m0xhjM2SP04lidfD9ze6gDiMuI/Bf7pq9Eqh9qk+uGPehLCT8R2UVAooOayhWbTOW08RWWOD9TOeCN/f6AICRp4bhzVqc0x7ioh7tZHPHjfbifj9lS4ooBRMKpZ7G4rSmSLCoglcYJz1RPz+ShcUUMLuNFYb6D4zqKCTp1zDbCfi9zfSIUCUDB7P0iaY9hQRNovOtS1NJu1E+P5GFxJA4qPhdy1PDbzJFFEy/PU69kAc8ET8/koXFFAqxhVvvdIz6SloEFc+bb32Se+M31/pggJK1VDxWZth2lMcMz46dzweKPRk/P5KFxSslLixU0LQBpV6CtexGko55r0LPRm/vdIVxaHEvVRyygWup4iyKdhpw896Ir6DkkoBJTJ8LhRthmlPUXvgzYXgV70zfn+lCwoo4UnIFJcN799kCn3+Uui1upvUE/H7K11QQIkPoWRHyohJT+GcsSmntiLgoCfi91e6oIBS4BZirtos054iKsZW7tLGWU/E7690QQGlaPDQ/7Lt/ZtMYSySuNd/XOOknojfX+mCAkqJm4jVW2XMpKc4Y3BD8pjTJfRk/P5KFxRQwjNs1mpzTXuq9de4jraznozfX+mCAkoFO2hmba5pT+GcSZaPOU96Mn5/pQsKKFWuf0lrU74SuI5R5R/rJCfj9xdSGQ6fWrGdpir0SuE7hQbkMfbW5UR0ByEVgo2yNaViU6TVqKfwjcLqkq+eSbeT8dsrXVFAiUyuPlSlPuopnDM+YubNpCfj91e6oICS41frS7z2FNpDudSQadYT8fsrXVBAyZuIQQ+lZ9JTqKFdxOYAs56I31/pggJKwYQaY9HqpTOFbxbuJx31ktQT8fsrXVBAKRpf9UfgegpjkfzLjodNBJ4I39/oAgJGicvBxFrF6EwBgynssX7JaHfG7690QQEltAhzUrZQFilo5BLD8aDgqHfG7690QQEljHgUn5Xxkp6CBoV4TJ4Y8d7D9ze6gDiMaq3aE6dnBvUzOe6uzXIyvoORBgGiimlrWVt5uqfQuPY2Yt7NYCfD9ze6kGAk7NFubUjKSElP4YyxntN1wpPx2ytdUUCJG9CWSJtl2lP8nTJUQjpmvgs9Gb+/0gUFlLAKGWlN7jPTrmGu+qPFLe1E/P5GOgSIuPVsXYxKvd1T5LBQGblj+FbYifD9jS4kgMStZ65WtBmmPcUOBntLH6NuEk/E7690QQElbj5zKdoc055C3eOp2FedJPRE/P5KFxRQ4uYz18JBGS3pKeJjzj7T0QiQeiJ+f6ULCihx+9lGH7Sq+0yh/vEu+GPFQKkn4vdXuqCAEjehbbLaLNOeIu9M5h6In/Vk/P5KFxSHUrUpB61eOlOUsa5rLnXWk/EdlFQKKHEj2ubglc5JT1FMmEeS8qwn46fSl79xOHT3/N2TQXBU9Pz6+eVX3/73X7/59g9f//b5zQ+PSfLjESDu98Qj8vZ4/PHhLfd+Dlz5WhGe37GUqxbRXyre/w/L/fI31D5OPiZrKuV0cC8/hHf59pHiA31+Pbzo8Y/e8Xj46LDUEP9nOIpk43Bcb6/I8GeK9w5x/mD+3D6Y/oLpF/zjE7l9moRP8/ndkx7ax8nvwq5blWjE7+EZVJwofM7F51+O0ttXgH/8+hV9fP8gNvzCcsonrOPJpdmEWSgVM3m5uG8+Pn774fHlv2JSxvPDnx+WX//hPx///vyV/fXzP54ffvf4lw+P3//CAAF3oUuOvo5ndQ9/AoCAye8orWRf848ASOEzCnB33HMfPM3f6zP8KQQSdsHl0o41OP65ALny+QiiTwajfzUOBCL8CQgit3ICSgvV/yiB6j6jQOUzlC8jzo8CPfwpBLj7QK0067ET0j8lcDF9PgJRCwe+OlrHNfVZg/fIT64EKZnY//Dg+MrvuZfJ1/rwIuAD578Aax3w4XePX9Gvnx++e/C5iEdoI1+73SGEepITxF1X/r+UAPEMbdEt3wvB1RKlpPZihvwevwZLBAD69U/smGOdD3gflhOs7c/45uOTS/jiq2+/+9O//f2Pf/rbD198/Ovf/v7D86v/ev7+c34euNETqHDz7v3z6JFP9Hlk3GVzXE15bBzcPw8+7XGO/hIfSPzxH4gvtaH+3/KBYC0Obj/a4s5PRIQ+0UeCNdIdZWzfGMvn+UjCjT8S7h4Gh4dazk/kjHyiD4SvyfzbbOU+aKL4WSotf99Ki/DQQ6wx1v6J9NCn+ki4I5esJbSqsv8sH4n7v+kjQV/98aN71P+sr/4YXv8/66vzn2didUD9wqGpa5y1ob46U9zmw7YhQ+ZNZo7dfK2NR/wsa4ifPfUNtHQTaGFWh68Y15+1zozH2el92/pIFCXDO1npIrDC3I6QMLo/W50ZLJ5h67FNpChKhney0kVghSkeqPdWqzMTMUJmQ/EToQjvZKWLwApzPRJ+WKzODF/LHL/9dV6dRcnwTla6yGFVa8oY71+tXpnkTHEptEfWJaEI72WlicAKMz8yHv5crM5M5ko8ZFfdSCjDO1npImzFPVIsBbFW7WeCO8FEoaYZUIQ3ktI9AEX82lJwJ2SWOjP85sR90zZjTQCK6E5SOgioHLcrUU+vVGcm8wFzt9tNgD26k5TuASnMB+GejFulzgxf9Cr50B5gE0XJ8E5WugisMCuE+4or1XuCqyTruDHlR0AZ3klK9QAUJoYQnnhcpM4MZpCmis2UB0AR3UlKBwEVZofwOaJc/s4M5kZUSrWMgjK8k5UuAivMEfH8TVqtzgxmb9uCtSUGQhneyUoXgRUOmK9ra8+mZwo3yyn7tlSZKEqGd7LSRQ6ragMeYlOsXpmSsIpizjQRivBeVpoIrDDnJpLWXD8zjJJz8gvgK7iTk67BToXb3vw+WkdhegazTKlEnyc+Ed7I6kIEVlggIWEZicXqzPDpw7+pljARivBOVroIrLj5bbOza13VM9ViJ+/cFgsQRcnwTla6CKw8typztWuDvWcqnou3jvxCeIZ3stJFYBW4WVmUs+oVr9hMIOW24YLkE+GdnDQNKEX+d+VyVqczU6NJIZajt3zyieBOTroHpBJuu3BPbpU6MxUTvUvNfgIU4Z2sdBFYZWPJprr2a3oG005czdT2qRJlDfGdtHSTQ6sS+bL2bHoGe59Urpi8nxVFfC8tzQRa2HLJ2bIOxfQMKqdo/attdRYlwztZ6SKwqlzxOLxwsToztRjyHs+pjIQivJOVLsJWFbcUfChr/6Zn8G3jhmcJYTQc4htpXZhAi0ykQHmts3oGKjnk3HZInRTP+E5augm0HHeAQ8lra7RnKhYh9jGlEVGGd7LSRWDljSfu1a19nJ5pW4CUY4l0UZQM72Sli8AK06aSy0qddWZaCyHkY3VraSjjO2npJtDC7fWELdIXrTODrUFrdoHcrCjiO2npJtBK3MbMKa39nZ7BF47PpqMJLxBFeCcrXeSwqlT479esXhnsXuyxSLibDGV8Ly3NBFp8SaOqDPidCeyr6jOeuZoNRXwnK1UEVIWbmDUrU2p7Bhv0xByCGwlFdCcpHQRU1SRnsUTaQnVm2t4qNcZUZkIR30lLN/n4CNbiER2KywiNyGD3kEIhvb6FQlHE99G6MoEWmeCorLNqRQbrq9aEmQyzoojvpKWbQMsZ71xc59WKDPY2yD5n62dFEd9JSzeBljcOMzqWMRqRafuw5ljaLL5BUcR30tJNoIUHkbldvvR4RAZPu3GzvS23IhFFeCcrXQRWmKQX0jq9VmTQTPDRprZznTSU8Z20dJNDq/JheKXWOjOonfjrh8UlJkUR30tLM4FWwgMQ1i89HpFBqypX7t+UWVHEd9LSTaCVTXZpHdLqibaiduWWe5gNRXwnK1UEVIWbmTk4pdI6M1h2rHDxPo2GMryTlU4CLDxo356pX7DODNb7S7FtlTQhivhOWroJaxG/05WyzrYVGawhiTZCSrOiiG+kdWECLeJWZlU6iGcCZ1DBuIyfDUV8JytVBFSYM2QdLaM0IoPWZ6qxzW2XhD26k5QOAipvyB+7Cc1UZwb1OJZxPWosSSjiO2npJtAKxnpKpHwJzwxOIe/fWw5SUcR30tJNDq3q+VxZR2l6pq2Ekqm0xW4mxTO+l5ZmAq1oivd2HYYXGXzjcuVGQ5wVRXwnLd0EWnzA3mer1FtnBte9zADOz4oivpOWbgKtzEcQgl1HaXqGqynjc3oN0whFGd9JSzeBVjHRR7vOwRUZ1E+FgncTogjvZKWLwKqa4GNeZ+GKDNrqofi4Gor4Tlq6CdZCtVhdN6wP+IoMvm+RfPEToghvZHUhcqwb67THC3sCX7YaiqtpEpTxnaxUEVA5bmTmsk7BFRlywWAoOabJUIR3stJJgOUNdqcua43VM2hPhRBdW9V7QBTxnbR0k0Orchll7fH0DKqmmqsvaVKU8b20NBNoBW5j1rrOwhUZclynR6KcJkUZ30lLN4FWNJkPJivV1pnBOVRKjTZPijK+k5ZuAq1kUuB6aO3x9Axh3T5fKIVZUcR30tJNoJWxUL5dZ+KKDHnCDMlENCuK+E5augm0Cr/T5aTUW2cGi+AW53KbFzkoivhOWroJtCofjA9pHanpGahQ2+ViVTzjO2npJqzFFbXD+gxrn6dnmgp2xXaK4nt8I60LE2hx1RNCSWu91TNtl6fivUuToozvpKWbQIsPLcQY15GansHOYT7XtuHAgNjDO1npIodVDXwsa7+nZ3DlwyrxLkyGMr6XlmYCLW9KSNrt1p4hnwzlSMe9C6ko4jtp6SbQCniAcH00pSeI38wt99dYvDQU8Z2sVBFQYZ2G4sPa6ekZdG6w+jbFyVCEd7LSSYCVuJWJ5Z1WrDPT2p/tds6MKOI7aekm0Mr8zpqDUmedGe7g8NtTjH5WFPGdtHQTaBVMqA1+7fP0DGonSta1JysGRRHfSUs3gVY1LhIps3F7Bte9EPl//awo4jtp6SashU0k+frv13qrZzAmU4q1Ly2hKOIbaV2YQIsbl5GPY+3z9Azx6cTJEvOsKOI7aekmh1aN3ikTcnsG3zgOJ5oRe3gvK00EVtwEiFzIWmv1DPFXz2eylmZDEd9JSzeBljeZ2wBuHanpGXzfIh9yXBRFfCct3QRawaC5tDa2zgTOoBAKNxlmQxHfyUoVAVU0MSZLSpV1ZnACod0eJ8Ie3UlKBwEVmkop09qV7hm0ECJ/09paUAOhiO+kpZtAiw8m5qDMx+0ZQpfQuaOZJRF7eCcrXQRW3FDSlgvuCfRqsq+UZ8Ae3klK9QBU5cYlt8DXEZqeQUuqYPfAWVCEd7LSSRgrWm5c1qjMw+0Z4uZ6Ksm96vaOKOMbaV2YHFr8QqfMw+0Z4grdBe9XRRHfS0szgRaZkmyp6whNz+Ac8sG5GmdFEd9JSzeBljM5UaxrX6dnMMpXXD7WNpKKMr6Tlm4CLT4CvvwrM3F7BucQtxBsDpOijO+kpZtAixuWydWq1FtnhrCze4p8eLOiiO+kpZtAK5rAf3xZuzs9Q9x1ttG5YzquVBTxnbR0E2glw3+6V+bj9gxhWCa7Y0v0QVHEd9LSTaCVjcNeC0q9dWb+/+7uJld23IYC8Pyt4q1AEfXPYYIADWSWZAlBDwKkRwGy/vBY9SRaYm2Agwa6D+/1bX+oUlEuWcakmVOPuRyKOvekZZtAawRqtRtvxF8FjE5FWvZnL3htqHNPVqYIqFjOrBVjMe6uPEM51z5vxn8ZrtiTlU0ysbh16saYtSro12PutZQTUeW+tCwT0WpRGoCOTbFOrV15VFLLsRmKv3JHWl9MoEXSZ8rIc896doXKCA3PHM+noso9adkm0ErSZzK+iL+0VoVKkS5B+is6FVXuScs2gVaWPpO53VdrdoVKD7HX2TxoxB17srJFYFWC/GIzVuLuyvN+q4kuwh17srJFYFXxVWk21uHuCkZyjmlchDv2ZGWLwKqFJFOWel+n2RWg1NziXAD4Nly5Jy3bBFo9UE8y7txaq0KFcRNdS3QqqtyTlm0CLWyaksvdlq4ChiZ5BeW5/k8bqtyTlSkyqbjjORWW1acClNx5PijzbbhiX1YWCbA4jF6G9cJaFcwCZQ5Y5t40GlHlnrRsE9HqMfReq7EOd1dwXpVaGuVQ1LkjrS8m0KLQpAm/W9JVIDnJmEqc99JpQ5V7sjJFQJXwTuJ8T3Z2heT0ucuvH4Q79SRlg4AqS4PZm7ECd1dIJs5ptNTqQahzT1q2CbSKfKDhzsFba1XkD4TIkZ+ntmtFnXvSsk2gVaXDlDnLfYVmV6gWOWItc28arahyT1q2CbRwMxx3Yw3uruC8eqrza2mNuGNPVrYIrOStNGKxPgxXBe83wpqQehqq3JOWbTK1eBCl+wrNrkBlSKtey624cl9algm0RhiDhrEId1egwj3Hz2vrpbhyT1q2CbQ49JGqsQ53V6jJWI4zLoeizj1p2SaiNbCQL+Nrh1NrV6hyoExlrqZRijp3pPXFBFoyxRvZWEyzCni/PYu682mock9WpgioUpBpcTPW4u6KzJaxSWLq/W2oY09WNgmwMh5cKKP0jbUqmNvgPsO555FGVLknLdsEWgWTvBjvOc+u4DWUeqtz8xCtqHJPWrYJtGqg0ZqxFndXCJfeYyv5RNyxJytbBFZNusye2Ri0VgXvt9hinneWa0OVe9KyTaYWjxH5nvPsCnqEWHuedzwpRZ370rJMoCXTljG61TysCvrPlLDP0aGoc09atgm0hnSZXIYxaq0KYR1brp0ORBV7srJFYMXPD4/7Os2u4BWE9bbzqpYy1LknLdtEtFj+LMdhrMPdFWo5xNJ/vbSUosodaX0xgRYF+aeOe9TaFXzyNfx6ORVV7knLNoFWCnKIZCxb3hUMT718HkCqEFXsycoWgVWWwyQ2VuLuytOtl9hzPQx17knLNoFWCcTZevDhrlCrgXP8XAPUiir3pGWbQKsGaZuMb1tXgbAmhIhiOg1V7snKFJlUzDUay3B3ZU6a6fPwImWoYl9WFgmwPrdL3DOeXUFH1eUY40JUuSct2wRaPXRupd3XaXaFGoWaWp4LtbSiyj1p2SbQwvMLu8yLb61VoS5/K6dW6FDUuSct2wRaLF1mH8Za3F3B5auaCs3nYilFnXvSsk3++FGj/CaPWq9xS1WeTz6WV9E4FVXuR+ubCbQIG9umes15VIVqC1glmfqhqHNPWrYJtKQJYOZ7Oa6qUK3y6/gq7FRUuSct2wRaGf8ug/attSrP91+l1me3P62oc09atgm0pC+PlMs161EVdAo95Rb7qahyT1q2CbSkMY/p3tBnF6hx6D2neBmq3JOVKTKpOKaejUFrVXCxL8cynocyaEMV+7KySIAlXXmUl8l1pUZVqPUg8+e5D80LUeWetGwTaElXHqW5vGY9qvLMBZMIpVNR5Z60bBNo4bbncl+M3wXCUvjWx/Nkd22oc09WpgioGHtE1nRdqVEVfOg1+VMHoUo9SdkgQkXym3Iu92pcVcEskOq8ffUluGNHVl9EYCXdeGx8r8VVleczr9C8X+dlqHJPWrYJtKQbj73dD7dVFYzitfT5LOmXoso9adkm0MrSYY5M92xnV/DtROQhs8BTUeWetGwTaBVpMTnea3FVBXNAlgM/19xfiir3pGWbTC2O3O9tcVUF77iBTcbSqahyX1qWCbRqGDJOx3vCsyv46Bsl8XMV+aWock9atgm0Wuj4EvCe8ewKmqrY4nPz3Atxx56sbBFYyQSPSI5zW60KxnI5Q+Z+Gqrck5ZtAq0R5DPOeF198kckcnp2rzsFf+WepCwPOHGQWR7dK3FVBaM4lnW39BZUqScpW0SsUgyZ8uD7VbUrz10CbXArp6HKHWl9MYGW/DuVyvf1mV3B6uRCg2M5FHXuScs2gRYeevxMhy+tVXnuA+syszEUV+5JyzaBVg5Rznrco9au4B2XiwT5UNS5Jy3bZGqx9JbjvkqzK/jca3i6YTkVVe5LyzKBlkxZqOd7Ja6qoPmU48ZnX8SXoso9adkm0KrSY47YjXFrVXD1aqSRPq8tpahyT1q2CbSa9Jmj9/s6za7gCkPHI+jaqahyT1q2CbS6dJpc7rW4qoLdQqTDyvVU1LknLdsEWiOUFMn4SPxVIGze3Vt81mO9DFXuycoUARWHnOJo90WaXcECoxbHXKCsDVXsycomEawccUtcvR9driroQPFMyEgnosodaX0xgRYFSinda3FVBa+h2lOa3/JoRZV70rJNoCWnltL9QJ5dIDywaESenZY2VLknK1NkUnHKrd7j+65gIJeGnWkchir2ZWWRAAuNZcnGOtxdwZrI2vO8M/qFqHJPWrYJtLAYpsZqjFmrgpGcP+O7NlypJynbA1J4Imbt5Z7t7ArOq0jD0McpqHJPWrYJtLD8vxVjDe6uPOeFe+/HrbhyT1q2CbS6tJidijFmrQpmNam0/uxu9FJUuSct2wRaQ3pM+bn7Ks2uYBfqlOkz31GKOvekZZtAi6XHHNVYhrsrz77KnQ7CFXpysjXEqUTpL9EHXE67glfPiI3p9NO5I60vJtAiaTCZ8319ZleePr08N8+9FXXuScs2mVqc5eQsrFmASY/y+Zdvw5X7sjJEQJXCyJSNFbi7grfbc2WU3oY69mRlkwALzx5P0VjrsCt4HmvMeeR2Iqrck5ZtAq0iZ5B6uuc6u/I8rS9m4nQqqtyTlm0CLTxMJxdjFe6ukABVKnXeY6EVVe5JyzaBVgtFCsYl+F3BWI4tN+dddEpR5560bBNoycnkMuie7+wKnvWbe418Kurck5ZtAi1MWWo1VuHuyjOaV+75UlS5Jy3bBFrYMLnhNstLa1XwbMPeapt3/2pFlXvSsk1Eq6IJkB+/Zz27gtdQwwmnU1HljrS+mEwtzvi21NL6VCjXkGjwyIeizn1pWSbQIuk1R473uLUreEZ5HbjX91DUuSct2wRaSXrNwXxfp9kVyhxGbTXlQ1HnnrRsE2jhDLgZ63F3Bc+DzK3PvfO1os49adkm0MJNcXIUY9xaFRnUZWDPWMr2VtS5Jy3bBFpVfpOMBmIVCF8Zxt76ZahyT1amCKga7k3txmLcXUHD3mvC96svQx17srJJgNVDKgmPerywVgUjeaOeZ6+lEHXuScs2gdYIVDKN+2rNrqCnkvNt7VJUuSct2wRaHGLJw1iLuysyij/7brZ2Kqrck5ZtMrW4lGqsxd0VyiWUluv89kIrqtyXlmUiWi2GUWrq99WaXUGXMKS7ep4m/VJUuSOtLybQotBL5X7PenYFKkkG85ZvxZV70rJNoJVwi0kzluPuyjM+pVTnclytqHJPWrYJtDI2t8jtHrd2BZ99uCWllFNR5Z60bBNoyUBdhvW1665gNE+xzi0mNeKOPVnZIrCSKXEZxgfiKmBu02tPo52CKvdkZYqASsbowqUaQ9aqEFZ2x/Jp45Whij1Z2STA6oFqpHrPeXaFcNNAkxlOOhB17knLNoHWCLHGYazG3RVK6BEIL6dDUeWetGyTqcXVfGn9KsAkForPk+gOw5X7sjJEQIXrw8/dJZfVqqCdKiR/ZbwNdezJyiYRrB5xYYqN1bi7QnLGLdNnjyiFqHNHWl9MoEWhVfmfv8esXcH7beAJyZeiyj1p2SbQSqHWkvN9nWZXKI1QR6tzHxGtqHJPWrYJtGToqTUaa3J3hSTGhdH5Xb5WVLknLdsEWiXkituXbq1VIWzvUOjzXb5WVLknLdsEWjUkeSvl+zrNruA1FHvNXE9FlXvSsk2ghR1bu4xAt9aqYCbYWiqlHoo696Rlm0Cr48Grw1iXuytEXUZziU9FnXvSsk2mFkvDlIxxa1WkAQ2dS5vNqVZUuS8tywRaQ3pNNhYxrwJFlpahtLnnijLUuScrUwRULI0ms7Eod1eIYqAsx09vQx17srJJBGvID7fY6B6zdoWw0XKnNvqJqHJHWl9MoEVBOstM99twV4hSoBLnugeNuGNPVrYIrFIoTf7rnvPsCt5vPBJnOg1V7knLNoFWDrk9NwNcWqtCsYSRS56fhkpR5560bBNolZCanM59rWZXKKbAnMvzbNaXoso9adkm0KqBWiFjRe6ucA0yge6zediH0rEnK1sEVk1OrXQ2Rq1Vkd8LPcnwdBi+ck9atsnUYmksje18dgUvoZblgG9EHfuyskRg1cNojYzVuLvCHCI3mttEKUIVe7KyRWA1Qm/N2Bp3FSjKX2olz/tb95FeuScrUwRUjGc1Sf22WhV5r+XBNPfFXUfSqScpG0So+NmQOxkrcXdF5s0yFZxPxNpHUqkjqS8ekCLpLQcbq3B3RUyKnPCc57wAV+zJyhaBVZLOko2lbKsg7VQb5fP22wfSsScp0wNQOUgDnvs9xdkVfNLVkub+PetIOvUkZYOAqgTqFI21t7siwzfxc0/FW1DFy+pPf05y2j/Tz7/9lGPhjOjnbz//9Nff//fvf/3+j9/+8vNf//2xPZXlHzqmPnDLC/L//Pjxzx8t1tBfPzyT8+e+HPn89fNn1WG+HfnvP/4PJwPoxgplbmRzdHJlYW0KZW5kb2JqCjEyIDAgb2JqCjI3MzIyCmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjE3IDAgb2JqCjw8IC9MZW5ndGggMzk1IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD1SS27FQAjb5xRcoNLwm895UlXdvPtva0NSqSq8iTHGMH3KkLnlS10ScYXJt16uWzymfC5bWpl5iLuLjSU+ttyX7iG2XXQusTgdR/ILMp0qRKjNqtGh+EKWhQeQTvChC8J9Of7jL4DB17ANuOE9MkGwJOYpQsZuURmaEkERYeeRFaikUJ9Zwt9R7uv3MgVqb4ylC2Mc9Am0BUJtSMQC6kAAROyUVK2QjmckE78V3WdiHGDn0bIBrhlURJZ77MeIqc6ojLxExD5PTfoolkwtVsZuUxlf/JSM1Hx0BSqpNPKU8tBVs9ALWIl5EvY5/Ej459ZsIYY6btbyieUfM8UyEs5gSzlgoZfjR+DbWXURrh25uM50gR+V1nBMtOt+yPVP/nTbWs11vHIIokDlTUHwuw6uRrHExDI+nY0peqIssBqavEYzwWEQEdb3w8gDGv1yvBA0p2sitFgim7ViRI2KbHM9vQTWTO/FOdbDE8Js753WobIzMyohgtq6hmrrQHazvvNwtp8/M+iibQplbmRzdHJlYW0KZW5kb2JqCjE4IDAgb2JqCjw8IC9MZW5ndGggMjQ5IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nE1RSYoDMAy75xX6QCFek7ynQ5lD5//Xyg6FOQQJr5KTlphYCw8xhB8sPfiRIXM3/Rt+otm7WXqSydn/mOciU1H4UqguYkJdiBvPoRHwPaFrElmxvfE5LKOZc74HH4W4BDOhAWN9STK5qOaVIRNODHUcDlqkwrhrYsPiWtE8jdxu+0ZmZSaEDY9kQtwYgIgg6wKyGCyUNjYTMlnOA+0NyQ1aYNepG1GLgiuU1gl0olbEqszgs+bWdjdDLfLgqH3x+mhWl2CF0Uv1WHhfhT6YqZl27pJCeuFNOyLMHgqkMjstK7V7xOpugfo/y1Lw/cn3+B2vD838XJwKZW5kc3RyZWFtCmVuZG9iagoxOSAwIG9iago8PCAvTGVuZ3RoIDk0IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWNwRHAIAgE/1RBCQoK2k8mk4f2/40QMnxg5w7uhAULtnlGHwWVJl4VWAdKY9xQj0C94XItydwFD3Anf9rQVJyW03dpkUlVKdykEnn/DmcmkKh50WOd9wtj+yM8CmVuZHN0cmVhbQplbmRvYmoKMjAgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL1N1YnR5cGUgL0Zvcm0gL0JCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9MZW5ndGggMzkKL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnic4zI0MFMwNjVVyOUyNzYCs3LALCNzIyALJItgQWQzuNIAFfMKfAplbmRzdHJlYW0KZW5kb2JqCjIxIDAgb2JqCjw8IC9MZW5ndGggMzIyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVRu23FMAzsNQUXMCB+Jc3jIEiRt3+bO9qpSNO8H1VeMqVcLnXJKllh8qVDdYqmfJ5mpvwO9ZDjmB7ZIbpT1pZ7GBaWiXlKHbGaLPdwCza+AJoScwvx9wjwK4BRwESgbvH3D7pZEkAaFPwU6JqrllhiAg2Lha3ZFeJW3SlYuKv4diS5BwlyMVnoUw5Fiim3wHwZLNmRWpzrclkK/259AhphhTjss4tE4HnAA0wk/mSAbM8+W+zq6kU2doY46dCAi4CbzSQBQVM4qz64Yftqu+bnmSgnODnWr6Ixvg1O5ktS3le5x8+gQd74Mzxnd45QDppQCPTdAiCH3cBGhD61z8AuA7ZJu3djSvmcZCm+BDYK9qhTHcrwYuzMVm/Y/MfoymZRbJCV9dHpDsrcoBNiHm9koVuytvs3D7N9/wFfGXtkCmVuZHN0cmVhbQplbmRvYmoKMjIgMCBvYmoKPDwgL0xlbmd0aCA4MyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFjLsNwDAIRHumYAR+JvY+UZTC3r8NECVuuCfdPVwdCZkpbjPDQwaeDCyGXXGB9JYwC1xHUI6d7KNh1b7qBI31plLz7w+Unuys4obrAQJCGmYKZW5kc3RyZWFtCmVuZG9iagoyMyAwIG9iago8PCAvTGVuZ3RoIDMyMCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UktuBTEI288puECl8E/O86qqi777b2sTvRVMMGDjKS9Z0ku+1CXbpcPkWx/3JbFC3o/tmsxSxfcWsxTPLa9HzxG3LQoEURM9WJkvFSLUz/ToOqhwSp+BVwi3FBu8g0kAg2r4Bx6lMyBQ50DGu2IyUgOCJNhzaXEIiXImiX+kvJ7fJ62kofQ9WZnL35NLpdAdTU7oAcXKxUmgXUn5oJmYSkSSl+t9sUL0hsCSPD5HMcmA7DaJbaIFJucepSXMxBQ6sMcCvGaa1VXoYMIehymMVwuzqB5s8lsTlaQdreMZ2TDeyzBTYqHhsAXU5mJlgu7l4zWvwojtUZNdw3Duls13CNFo/hsWyuBjFZKAR6exEg1pOMCIwJ5eOMVe8xM5DsCIY52aLAxjaCaneo6JwNCes6VhxsceWvXzD1TpfIcKZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDM0MCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UjluBDEM6/0KfSCAbtvv2SBIkfy/DanZFANxdFKUO1pUdsuHhVS17HT5tJXaEjfkd2WFxAnJqxLtUoZIqLxWIdXvmTKvtzVnBMhSpcLkpORxyYI/w6WnC8f5trGv5cgdjx5YFSOhRMAyxcToGpbO7rBmW36WacCPeIScK9Ytx1gFUhvdOO2K96F5LbIGiL2ZlooKHVaJFn5B8aBHjX32GFRYINHtHElwjIlQkYB2gdpIDDl7LHZRH/QzKDET6NobRdxBgSWSmDnFunT03/jQsaD+2Iw3vzoq6VtaWWPSPhvtlMYsMul6WPR089bHgws076L859UMEjRljZLGB63aOYaimVFWeLdDkw3NMcch8w6ewxkJSvo8FL+PJRMdlMjfDg2hf18eo4ycNt4C5qI/bRUHDuKzw165gRVKF2uS9wGpTOiB6f+v8bW+19cfHe2AxgplbmRzdHJlYW0KZW5kb2JqCjI1IDAgb2JqCjw8IC9MZW5ndGggMjUxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nC1RSXIDQQi7zyv0hGan32OXK4fk/9cIygcGDYtAdFrioIyfICxXvOWRq2jD3zMxgt8Fh34r121Y5EBUIEljUDWhdvF69B7YcZgJzJPWsAxmrA/8jCnc6MXhMRlnt9dl1BDsXa89mUHJrFzEJRMXTNVhI2cOP5kyLrRzPTcg50ZYl2GQblYaMxKONIVIIYWqm6TOBEESjK5GjTZyFPulL490hlWNqDHscy1tX89NOGvQ7Fis8uSUHl1xLicXL6wc9PU2AxdRaazyQEjA/W4P9XOyk994S+fOFtPje83J8sJUYMWb125ANtXi37yI4/uMr+fn+fwDX2BbiAplbmRzdHJlYW0KZW5kb2JqCjI2IDAgb2JqCjw8IC9MZW5ndGggMjE1IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVROQ4DIQzs9xX+QCSML3hPoijN/r/NjNFWHsFchrSUIZnyUpOoIeVTPnqZLpy63NfMajTnlrQtc4C4trwvrZLAiWaIg8FpmLgBmjwBQ9fRqFFDFx7Q1KVTKLDcBD6Kt24P3WO1gZe2IeeJIGIoGSxBzalFExZtzyekNb9eixvel+3dyFOlxpYYgQYBVjgc1+jX8JU9TybRdBUy1Ks1yxgJE0UiPPmOptUT61o00jIS1MYRrGoDvDv9ME4AABNxywJkn0qUs+TEb7H0swZX+v4Bn0dUlgplbmRzdHJlYW0KZW5kb2JqCjE1IDAgb2JqCjw8IC9UeXBlIC9Gb250IC9CYXNlRm9udCAvQk1RUURWK0RlamFWdVNhbnMgL0ZpcnN0Q2hhciAwIC9MYXN0Q2hhciAyNTUKL0ZvbnREZXNjcmlwdG9yIDE0IDAgUiAvU3VidHlwZSAvVHlwZTMgL05hbWUgL0JNUVFEVitEZWphVnVTYW5zCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnRNYXRyaXggWyAwLjAwMSAwIDAgMC4wMDEgMCAwIF0KL0NoYXJQcm9jcyAxNiAwIFIKL0VuY29kaW5nIDw8IC9UeXBlIC9FbmNvZGluZwovRGlmZmVyZW5jZXMgWyA0OCAvemVybyAvb25lIC90d28gL3RocmVlIC9mb3VyIC9maXZlIC9zaXggNTYgL2VpZ2h0IC9uaW5lIF0KPj4KL1dpZHRocyAxMyAwIFIgPj4KZW5kb2JqCjE0IDAgb2JqCjw8IC9UeXBlIC9Gb250RGVzY3JpcHRvciAvRm9udE5hbWUgL0JNUVFEVitEZWphVnVTYW5zIC9GbGFncyAzMgovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Bc2NlbnQgOTI5IC9EZXNjZW50IC0yMzYgL0NhcEhlaWdodCAwCi9YSGVpZ2h0IDAgL0l0YWxpY0FuZ2xlIDAgL1N0ZW1WIDAgL01heFdpZHRoIDEzNDIgPj4KZW5kb2JqCjEzIDAgb2JqClsgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAKNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCAzMTggNDAxIDQ2MCA4MzggNjM2Cjk1MCA3ODAgMjc1IDM5MCAzOTAgNTAwIDgzOCAzMTggMzYxIDMxOCAzMzcgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNgo2MzYgNjM2IDMzNyAzMzcgODM4IDgzOCA4MzggNTMxIDEwMDAgNjg0IDY4NiA2OTggNzcwIDYzMiA1NzUgNzc1IDc1MiAyOTUKMjk1IDY1NiA1NTcgODYzIDc0OCA3ODcgNjAzIDc4NyA2OTUgNjM1IDYxMSA3MzIgNjg0IDk4OSA2ODUgNjExIDY4NSAzOTAgMzM3CjM5MCA4MzggNTAwIDUwMCA2MTMgNjM1IDU1MCA2MzUgNjE1IDM1MiA2MzUgNjM0IDI3OCAyNzggNTc5IDI3OCA5NzQgNjM0IDYxMgo2MzUgNjM1IDQxMSA1MjEgMzkyIDYzNCA1OTIgODE4IDU5MiA1OTIgNTI1IDYzNiAzMzcgNjM2IDgzOCA2MDAgNjM2IDYwMCAzMTgKMzUyIDUxOCAxMDAwIDUwMCA1MDAgNTAwIDEzNDIgNjM1IDQwMCAxMDcwIDYwMCA2ODUgNjAwIDYwMCAzMTggMzE4IDUxOCA1MTgKNTkwIDUwMCAxMDAwIDUwMCAxMDAwIDUyMSA0MDAgMTAyMyA2MDAgNTI1IDYxMSAzMTggNDAxIDYzNiA2MzYgNjM2IDYzNiAzMzcKNTAwIDUwMCAxMDAwIDQ3MSA2MTIgODM4IDM2MSAxMDAwIDUwMCA1MDAgODM4IDQwMSA0MDEgNTAwIDYzNiA2MzYgMzE4IDUwMAo0MDEgNDcxIDYxMiA5NjkgOTY5IDk2OSA1MzEgNjg0IDY4NCA2ODQgNjg0IDY4NCA2ODQgOTc0IDY5OCA2MzIgNjMyIDYzMiA2MzIKMjk1IDI5NSAyOTUgMjk1IDc3NSA3NDggNzg3IDc4NyA3ODcgNzg3IDc4NyA4MzggNzg3IDczMiA3MzIgNzMyIDczMiA2MTEgNjA1CjYzMCA2MTMgNjEzIDYxMyA2MTMgNjEzIDYxMyA5ODIgNTUwIDYxNSA2MTUgNjE1IDYxNSAyNzggMjc4IDI3OCAyNzggNjEyIDYzNAo2MTIgNjEyIDYxMiA2MTIgNjEyIDgzOCA2MTIgNjM0IDYzNCA2MzQgNjM0IDU5MiA2MzUgNTkyIF0KZW5kb2JqCjE2IDAgb2JqCjw8IC9laWdodCAxNyAwIFIgL2ZpdmUgMTggMCBSIC9mb3VyIDE5IDAgUiAvbmluZSAyMSAwIFIgL29uZSAyMiAwIFIKL3NpeCAyMyAwIFIgL3RocmVlIDI0IDAgUiAvdHdvIDI1IDAgUiAvemVybyAyNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE1IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMCAvY2EgMSA+PgovQTIgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMSA+PgovQTMgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMSAvY2EgMC41ID4+ID4+CmVuZG9iago1IDAgb2JqCjw8ID4+CmVuZG9iago2IDAgb2JqCjw8ID4+CmVuZG9iago3IDAgb2JqCjw8IC9GMS1EZWphVnVTYW5zLW1pbnVzIDIwIDAgUiA+PgplbmRvYmoKMiAwIG9iago8PCAvVHlwZSAvUGFnZXMgL0tpZHMgWyAxMSAwIFIgXSAvQ291bnQgMSA+PgplbmRvYmoKMjcgMCBvYmoKPDwgL0NyZWF0b3IgKE1hdHBsb3RsaWIgdjMuOC4wLCBodHRwczovL21hdHBsb3RsaWIub3JnKQovUHJvZHVjZXIgKE1hdHBsb3RsaWIgcGRmIGJhY2tlbmQgdjMuOC4wKSAvQ3JlYXRpb25EYXRlIChEOjIwMjMxMDExMTY0MzM4WikKPj4KZW5kb2JqCnhyZWYKMCAyOAowMDAwMDAwMDAwIDY1NTM1IGYgCjAwMDAwMDAwMTYgMDAwMDAgbiAKMDAwMDAzMjkwMSAwMDAwMCBuIAowMDAwMDMyNjM4IDAwMDAwIG4gCjAwMDAwMzI2NzAgMDAwMDAgbiAKMDAwMDAzMjgxMCAwMDAwMCBuIAowMDAwMDMyODMxIDAwMDAwIG4gCjAwMDAwMzI4NTIgMDAwMDAgbiAKMDAwMDAwMDA2NSAwMDAwMCBuIAowMDAwMDAwMzM5IDAwMDAwIG4gCjAwMDAwMjc3NTggMDAwMDAgbiAKMDAwMDAwMDIwOCAwMDAwMCBuIAowMDAwMDI3NzM2IDAwMDAwIG4gCjAwMDAwMzE0NDcgMDAwMDAgbiAKMDAwMDAzMTI0MCAwMDAwMCBuIAowMDAwMDMwODc0IDAwMDAwIG4gCjAwMDAwMzI1MDAgMDAwMDAgbiAKMDAwMDAyNzc3OCAwMDAwMCBuIAowMDAwMDI4MjQ2IDAwMDAwIG4gCjAwMDAwMjg1NjggMDAwMDAgbiAKMDAwMDAyODczNCAwMDAwMCBuIAowMDAwMDI4OTA2IDAwMDAwIG4gCjAwMDAwMjkzMDEgMDAwMDAgbiAKMDAwMDAyOTQ1NiAwMDAwMCBuIAowMDAwMDI5ODQ5IDAwMDAwIG4gCjAwMDAwMzAyNjIgMDAwMDAgbiAKMDAwMDAzMDU4NiAwMDAwMCBuIAowMDAwMDMyOTYxIDAwMDAwIG4gCnRyYWlsZXIKPDwgL1NpemUgMjggL1Jvb3QgMSAwIFIgL0luZm8gMjcgMCBSID4+CnN0YXJ0eHJlZgozMzExMgolJUVPRgo=", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T16:43:36.491534\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["fig, ax = plt.subplots(2, 2, figsize=(10, 6))\n", "for i in range(4):\n", " ax_sub = ax[i // 2][i % 2]\n", " ax_sub.bar(np.arange(out.shape[1], dtype=np.int32), out[i + 4, :, 0, 14, 14], **plot_args)\n", " ax_sub.set_yscale(\"log\")\n", " ax_sub.set_xticks([0, 64, 128, 192, 256])\n", "plt.show()\n", "plt.close()"]}, {"cell_type": "markdown", "id": "49dde0fb", "metadata": {"papermill": {"duration": 0.044463, "end_time": "2023-10-11T16:43:38.843419", "exception": false, "start_time": "2023-10-11T16:43:38.798956", "status": "completed"}, "tags": []}, "source": ["Overall we see a very diverse set of distributions, with a usual peak\n", "for 0 and close to 1. However, the distributions in the first row show a\n", "potentially undesirable behavior. For instance, the value 242 has a\n", "1000x lower likelihood than 243 although they are extremely close and\n", "can often not be distinguished. This shows that the model might have not\n", "generalized well over pixel values. The better solution to this problem\n", "is to use discrete logitics mixtures instead of a softmax distribution.\n", "A discrete logistic distribution can be imagined as discretized, binned\n", "Gaussians. Using a mixture of discrete logistics instead of a softmax\n", "introduces an inductive bias to the model to assign close-by values\n", "similar likelihoods. We can visualize a discrete logistic below:"]}, {"cell_type": "code", "execution_count": 28, "id": "a45a1bc4", "metadata": {"execution": {"iopub.execute_input": "2023-10-11T16:43:38.921790Z", "iopub.status.busy": "2023-10-11T16:43:38.921403Z", "iopub.status.idle": "2023-10-11T16:43:39.681965Z", "shell.execute_reply": "2023-10-11T16:43:39.681237Z"}, "papermill": {"duration": 0.801647, "end_time": "2023-10-11T16:43:39.683536", "exception": false, "start_time": "2023-10-11T16:43:38.881889", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgNDA5LjMyNSAyMzEuNjEwNjI1IF0gL0NvbnRlbnRzIDkgMCBSIC9Bbm5vdHMgMTAgMCBSID4+CmVuZG9iago5IDAgb2JqCjw8IC9MZW5ndGggMTIgMCBSIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nNXdTXMcVbIG4H39ilqaBenz/bHElxlHsLqAIu6CmMVgDCPCPQQ2AzP//mZWd0vVlW96HNhaJDMOrBep1O8jqfucVHfV8y9f/37/6vU3L1+s//Pt8vzxrVfvlrj+zH9+WsP6M//5Y43rS/7z0xL4rdNSwqScKv/9zcPfU47UYmj81zf8bjdv/mNZflyef8EHeLcGmrG30kMdQ71RZoizhT7Wt/K5X968w/K+916W2mjwZyuJZqvyeU9LnnwjbrI3D1ni253GJbx87E223ehfV3XYnAuNNbbGtde3r9f/W/+5Pv8iSbe4fsV/fuY/Z7FH04VNe6d+e/vOyc1nXb5dvl5/vR4wUKz8Vbgec3vz5SVdfuWvUFg/D/yfWie5+ZXO4q9Oy4u75flf4xrjevfj9lW7+2H5bn0WQ/hs/dt699Xyl7vl6+1TfbqCMQcK7fA1eAg/Qc0YJ20H+JCm8SmbjkT58MW8Zp+iZ480+ofVTE9YM/EHtnHo+RB+gqIpFYof2jQ/ZdPRb0vy25+iX2/0YeXKE5bLme+i4/G+8Rp+gpo5DRrxA38u61M2VXf4J3yH/yd7jkgfWrPd1Nz/cM9EQe4oJvX9AdLtAf73M77FNPL1n/XZ/b85mvyw1yfTtNbXZ6/frL///c2/Xj+d6Ac+qv7ZH//t4HlQH3WKiQ3KjyJP+I1zOVIbVEoso+xqPmYfWbMVSi322eJo/6VpevKmM8tiKpe0a/qYfWTTwd/aubdaRs/vL1qevKisGfLMKcRd0134kVVjrJRirvz/Ov/LV7U9fdnCH99maPuuD9nHVi2BH/j5x5Qftdv7m46nb9oD5VJmHPuqj+HHduUbFltNOcz83vt5PsLT3y3FWfnfofe+L/sYfmxZOWKJYfBd03x/13T7oCYH+VwOF9u2CZyF+K5yWx2+95GN332mHsblke2teqz75fu/f3//5v63//xJ23xmSHOEPFOsnd8o551jypV3ku26dyzbVpNXaX/Y77/evP83L5cPfE/Zln7O/5HmVPsizhPv/ubtNuLN+3JwnG2b+sKtDGgqMrlTTSOmo4CVYzHPMqjpJpP4KIct2Zv35VDMtQxoKjKp8wNsPf502DkW8yyDmm4yke9ai74/MXMo5loGNBUZfhhqIev7EyvHYp5lUNNNJlLuUd+fmDkUcy0DmopM4IVhUY/Cdo7FPMugpptMoBnAusXMoZhrGdCUZTKv1FvX6xYrN8Qcy8Cmm4x8oF632DkUcy0DmorMkLmCXreYORbzLIOaikyfNIdet5g5FvMsg5puMoX3QXrdYudQzLUMaCoybfI+SK9bzByLeZZBTTeZwkfR6xY7h2KuZUBTkamTApjDmDkW8yyDmm4ymauA9YyZQzHXMqCpyJRBFcxhzByLeZZBTTeZTBnMYewcirmWAU1FJg+KYA5j5ljMswxqusnwW2AOY+dQzLUMaCoyqfM+CKxbrByLeZZBTTeZxB8I1i1mDsVcy4CmIhM7JTCHMXMs5lkGNd1kIk00nzFzKOZaBjQVmdCoozmMlWMxzzKo6SYTqaI5jJlDMdcyoCnLpNkogTmMlRtijmVg000m8j5Ir1vsHIq5lgFNRWbIc1n0usXMsZhnGdR0k+HPCuYwdg7FXMuApiLTK2UwhzFzLOZZBjXdZOSZWnrdYudQzLUMaCoyjW84mMOYORbzLIOaikyd1MAcxsyxmGcZ1HSTKVTAHMbOoZhrGdBUZMrkfRBYt1g5FvMsg5puMpn3QWDdYuZQzLUMaCoy8mIPMIcxcyzmWQY13WQyVTCHsXMo5loGNBWZNCiBOYyZYzHPMqjpJpMpgDmMnUMx1zKgqcjETgPMYcwci3mWQU03mUQVzWfMHIq5lgFNRSZ03geBdYuVYzHPMqjpJpN4HwTWLWYOxVzLgKYsw+/E+yC9brFyQ8yxDGy6yURqYA5j51DMtQxoKjKjUQFzGDPHYp5lUNNNJlIEcxg7h2KuZUBTken1+tatgJVjMc8yqOkmE6iDOYydQzHXMqCpyLTK+yC9bjFzLOZZBjXdZPgoYA5j51DMtQxoKjK18j4IrGesHIt5lkFNRaZMGmAOY+ZYzLMMarrJFKpgDmPnUMy1DGgqMnKSJTCHMXMs5lkGNd1kCkUwh7FzKOZaBjQVGTmZFpjDmDkW8yyDmm4ymfdBYN1i5lDMtQxoKjJx8D4IrFusHIt5lkFNN5nM+yCwbjFzKOZaBjQVmdBpojmMlWMxzzKo6SaTqKM5jJlDMdcyoCnLzE4FjGGM2PBy7IKKCkuiBGYwVgy1PKuAnqwyOgUwfzFijOVYBfUUlch7H71WsWKI5VkF9GQVOU8umLsYMcZyrIJ6ikrkPY9eo1gxxPKsAnqyipyaEcxbjBhjOVZBPUUl0ACzFiuGWJ5VQE9WqZUamLMYMcZyrIJ6igp/GJixWDHE8qwCesoJrCpFMF8xYozlWAX13E7RxHsb8FprHGMsxyqo53Y6A97ZgNdZGzHE8qwCem4vcOOdDXjOC44xlmMV1HN7Mjgl9HokI4ZYnlVAz+3pQRTgc11gjLEcq6Ce269FqKNJihFDLM8qoKfsmamCOQpOMZVjE1BTHoAoo3PDwBRCORYBLU9LRC9HgyFW8usBSp6Wjl63CENo5FcDdORvjoRe4KrThpCaaw9UnUE6ei20TpvJ5FgEdD8t27Uw1GpUpx06ddciqLtcAw2dY0GnEzpN3yKgu1wzDZ2PQ6U5Iqct9SuCurPIQOduUWnGTr4faVD301IyOs+PSktETsX39wjqziIDnRNKpTUhpy11LAK6n5Za0PnDVNomcmq+71lRdxaZBM41p9LZkdP0/eiLup+WVtB5CY9pLwU4nVO/Iqg7i0x0DstjOlICTufUsQjozpu8is53ekwnYpq+H3tR89MyAjoz7iFtsWmkc+jXAzVnj4rOoXxIWw0dKJ1TxyKg+2mZAZ1v+5D2MDTTOfTrgZqzR0NnZj+kI6SslS6pYxHQ/bTEEAmcxv8Yz9qnlrqkfk1ge0Fp6KoPh7inuC08Dge5xp5VQH1WiRFdJeQQ95Z7BliX2LEKqi8qDV1V5hAPbgewrrFnFVBfJtAJXYXoEM8y8tRY19ixCqovKp3AVav2caYQcx/zgLWLPauA+qySE7rK2T6WkxXVgLCusWMVVF9UOroq3j7m+pzHpLGusWcVUJ9VuB24iuI+5r+NmFtWWA+xYxVUX1QGuurmPs7cKMY6FNZD7FkF1GeVmtFVWvdxplr5m6IprIfYsQqqLyoDXdV3H2eaOcSeFNZD7FkF1GeVVtBVoPdx4W+KxAc+YO1ixyqovqhMAlcN38eVas/tvA/aHWQXe1YB9eXJBAVdZX4fd0olpdAOWLvYsQqqLyqTdzZ6vbKLt6cF85bngLWLPauA+qwyCu9n9HrlMa6FcpzjvOJ/PMg+dqyC6rPK5GOA+cpj3CL/qOSZxy3WPnasguqLSqUC5iuPcY/8o9JGzbdY+9izCqh/WlLg99TzlV08Oo2R48Fql/o1geXFpNLU05VdHGPgXWBsudxa3eSeXQAAu8TIOxu1XtnFMcvzlPqc5eC1zx27IABx4RusJyy7ODY5p0NstRy89rlnFwAgT4KMvLtRa5ZdHGeiEFJSXvvcsQsCEBcuoqcsuziFeP396Y3XTe7ZBQCwS+avup6z7OLL9wX//dbrJnfsggDEpVPWk5ZdfL0f2X4HtPfa555dAAC78BtVz1p28eVxZ8x58Nrnjl0QgLh06nrasouv65SRDl773LMLAGCXKr/90uuXx/iyqm23WrvUsQkqLyaD9zl67fIYX7Y/8UC1jz2rgPqs0jJvdPTK5THetsolxHGLtY8dq6D6ojLkPdX9ymN8HqvMUG6x9rFnFVCfVXqmCV7rs4u3Edz5abT7g+xixyqovqhMiuD1Prt4G9fm0g9Yu9izCqgvL4LinwQwbdnF22i/biO4/UF2sWMVVF9UJjUwb9nF26+BWisHrF3sWQXUZ5XJNxtMW3bx9ivDtD2ZZ3+QXexYBdU/LTnwClXPWvbx+dfLfSish9ivCqwvKpW3Nmq9so/PT0XoXWE9xJ5VQH1W4d1M1XOWfbw9bSWVqbAeYscqqL6oVOp6yrKPz09x2oa1t1gPsWcVUJ9VUqSgZyz7+Px0uPNLLW+xrrFjFVRfVBolPWHZx+enTraosa6xZxVQX164Hano+cohnmV7rDkc45I6NkHlxaTxe6rVyiEeOc+oqa6xZxVQn1VK5H2NXq3cxr2V7Ukrh4NcY8cqqL6odN7X6NXKbdz5ExSAdYk9q4D6rFITZT1dOcazbsP840EusWMVVF9UOjU9XTnGI1SEdYk9q4D6rNISDT1dOcZtzgSwLrFjFVRfVAYFcEaVQ9xqbgjrHHtWAfVZpfMeD5xV5RC3lBPCOseOVVB9URm8r9HrlWM8t1MPqYNM52ckgvXlBDyZ9zV6vXKM+3mOcDxI9z5HQPVFZfK+Rq9XjvFFBWN5VgH1WWUWSmC6YsTY0LEK6ikqkwqYrhzjy/fK8SDuv1dQ/dNSQqGupytGbBj6VYE9RWXS1NMVM4ZYnlVAT1aJlaKerlgxxnKsgnqySgq8s9FngjNijOVYBfUUlco7G302OCuGWJ5VQE85aWDgnY16rLVijOVYBfUUlUZBz1fMGGJ5VgE9WaVE+Q2gugMxYozlWAX1FJVGVc9XzBhieVYBPVmlRup6vmLFGMuxCuopKp2Cnq+YMcTyrAJ6skpLvLPRCxMjxliOVVBPUem8s9ELEyuGWJ5VQE9W6Yl3NnphYsQYy7EK6ikqnaYepJgxxPKsAnrK6Z8zRT0bsGKM5VgF9RSVQQXMV6wYYnlWAT1ZZWZqYJBixBjLsQrqKSqDBhikWDHE8qwCep6WGoo8u+34M2HEBpZfFdhTVCbfYPVYa8YQy7MK6MkqUZ53rs+4b8QYy7EK6ikqk7qeDZgxxPKsAnqySqoU9GzAijGWYxXUk1UyH0PPBqwYYzlWQT1FpVLRswEzhlieVUBPubxJoK5nA1aMsRyroJ6iUnlno9crVgyxPKuAnqxSI+9s9MLEiDGWYxXUU1Qa72z0wsSKIZZnFdCTVVqkBq4SZMQYy7EK6ikqXETPBswYYnlWAT1ZpSeKejZgxRjLsQrqKSodXfvWjCGWZxXQk1VGQldItmKM5VgF9RSVjq6kbcYQy7MK6CkXqsvomutWjLEcq6CeojJ4Z6MXJlYMsTyrgJ6npYVMRc8GjNjA8qsCe4rKoK5nA2YMsTyrgJ6sErOcW0Zd3dCIMZZjFdRTVCZFPRswY4jlWQX0ZJVUqOjZgBVjLMcqqKeoTN7ZqMdaM4ZYnlVAT1aRm61nA1aMsRyroJ5ycd3AOxv1WGvFGMuxCuopKpWyng2YMcTyrAJ6skrlzwmuxGzEGMuxCuopKpUGuBqzFUMszyqgJ6u0SEHPBqwYYzlWQT1FpVHSswEzhlieVUBPVumRdzZ6YWLEGMuxCuopKo13NnphYsUQy7MK6MkqQ87dqxcmRoyxHKugnqLSKYL5ihVDLM8qoCerzEQFDFKMGGM5VkE9RaVTA4MUK4ZYnlVAz9PSQ6KhZwNGbGD5VYE9RWVQ1LMBM4ZYnlVAT1aJmXc26rHWijGWYxXUU1QG72zUY60ZQyzPKqAnq6TMOxv1WGvFGMuxCuopKpOCng2YMcTyrAJ6sgrf7KRnA1aMsRyroJ6iMqnq2YAZQyzPKqAnq5RCXc8GrBhjOVZBPUVlXt+4VTFiiOVZBfRklVp5Z6MXJkaMsRyroJ6s0vjD9GzAijGWYxXUU1Qq72z0wsSKIZZnFdCTVXqgoWcDVoyxHKugnqLSKOrZgBlDLM8qoCerjEhZzwasGGM5VkE9RaVRBfMVK4ZYnlVAT1aZkQYYpBgxxnKsgnqKSuedjV6YWDHE8qwCep6WERLvbNRjrREbWH5VYE9R6byzUY+1ZgyxPKuAnqwSE3U9G7BijOVYBfUUlU5TzwbMGGJ5VgE9WSVlSno2YMUYy7EK6ikqg4qeDZgxxPKsAnqySs7U9GzAijGWYxXUU1QG72zUY60ZQyzPKqAnq5TCOxu9XjFijOVYBfUUlck7G70wsWKI5VkF9GSVWqjq2YAVYyzHKqinqEwuohcmVgyxPKuAnqzSKgU9G7BijOVYBfVklc7H0LMBK8ZYjlVQT1GpVPVswIwhlmcV0JNVRuCdjV6YGDHGcqyCeopK5Z2NXphYMcTyrAJ6ssqMvLPRCxMjxlgPKs+/SFI4rV+t/PHSJa4v1+dfvv79/tXrb16+WF+9W5Tc6RrxJ+Sfz8sxl2+XzDfi+L19zY7vC46qP3z/nrsD2EcNXCRQrOtPXORn/vPH+c2Xl3T5dYlrWD8P/J+ivErobMNfjuthXp2WF3fL87/yf0/r3Y98xLDe/bB8tz778v7dq7ef8ack3jFc/lmfvf7t9frml5/u3/12/2r9gf/19v77f/12/8s/P1v/tt59tfzlbvma//f/oufBugplbmRzdHJlYW0KZW5kb2JqCjEyIDAgb2JqCjQ4NjgKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTcgMCBvYmoKPDwgL0xlbmd0aCAxNjQgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPZDBEUMhCETvVrElgIBAPclkcvi//2tAk1xkHWD3qTuBkFGHM8Nn4smD07E0cG8VjGsIryP0CE0Ck8DEwZp4DAsBp2GRYy7fVZZVp5Wumo2e171jQdVplzUNbdqB8q2PP8I13qPwGuweQgexKHRuZVoLmVg8a5w7zKPM535O23c9GK2m1Kw3ctnXPTrL1FBeWvuEzmi0/SfXL7sxXh+FFDkICmVuZHN0cmVhbQplbmRvYmoKMTggMCBvYmoKPDwgL0xlbmd0aCAxNzAgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPZBLEsMgDEP3nEJHAP+A87TT6YLcf1vLmXSDFGPLL0RXdOyVh8fGlI33aGNPhC1c5XQaTlMZj4u7Zl2gy2Ey02+8mrnAVGGR1eyi+hi8ofOsZoevVTMxhDeZEhpgKndyD/X1pzjt25KQbFdh0J0apLMwzJH8PRBTc9BziJH8I19ya2HQmeYXFy2rGa1lTNHsYapsLQzqjUF3yvXUeq7zMBHv8wPfQT5kCmVuZHN0cmVhbQplbmRvYmoKMTkgMCBvYmoKPDwgL0xlbmd0aCAzMDcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMCAwIG9iago8PCAvTGVuZ3RoIDI0NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFkU1yBSEIhPeeoi/wquRXPc+kUllM7r8NzbwkK1qF5gPTAhNH8BJD7ImVEx8yfC/oMny3MjvwOtmZcE+4blzDZcMzYVvgOyrLO15Dd7ZSP52hqu8aOd4uUjV0ZWSfeqGaC8yQiK4RWXQrl3VA05TuUuEabFuCFPVKrCedoDToEcrwd5RrfHUTT6+x5FTNIVrNrRMairBseEHUySQRtQ2LJ5ZzIVH5qhurOi5gkyXi9IDcoJVmfHpSSREwg3ysyWjMAjbQk7tnF8aaSx5Fjlc0mLA7STXwgPfitr73NnGP8xf4hXff/ysOfdcCPn8AS/5dBgplbmRzdHJlYW0KZW5kb2JqCjIxIDAgb2JqCjw8IC9MZW5ndGggMjMyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVRSW7EMAy7+xX8wADW7rwnxaCH9v/XUsoUCEAltrglYmMjAi8x+DmI3PiSNaMmfmdyV/wsT4VHwq3gSRSBl+FedoLLG8ZlPw4zH7yXVs6kxpMMyEU2PTwRMtglEDowuwZ12Gbaib4h4bMjUs1GltPXEvTSKgTKU7bf6YISbav6c/usC2372hNOdnvqSeUTiOeWrMBl4xWTxVgGPVG5SzF9kOpsoSehvCifg2w+aohElyhn4InBwSjQDuy57WfiVSFoXd2nbWOoRkrH078NTU2SCPlECWe2NO4W/n/Pvb7X+w9OIVQRCmVuZHN0cmVhbQplbmRvYmoKMjIgMCBvYmoKPDwgL0xlbmd0aCAyMzEgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicNU85kgQhDMt5hT4wVRjbQL+np7Y22Pl/upKZTpDwIcnTEx2ZeJkjI7Bmx9taZCBm4FNMxb/2tA8TqvfgHiKUiwthhpFw1qzjbp6OF/92lc9YB+82+IpZXhDYwkzWVxZnLtsFY2mcxDnJboxdE7GNda2nU1hHMKEMhHS2w5Qgc1Sk9MmOMuboOJEnnovv9tssdjl+DusLNo0hFef4KnqCNoOi7HnvAhpyQf9d3fgeRbvoJSAbCRbWUWLunOWEX712dB61KBJzQppBLhMhzekqphCaUKyzo6BSUXCpPqforJ9/5V9cLQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9MZW5ndGggMjQ5IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD1QO45EIQzrOYUv8CTyI3AeRqstZu/frgOaKVBMfrYzJNARgUcMMZSv4yWtoK6Bv4tC8W7i64PCIKtDUiDOeg+IdOymNpETOh2cMz9hN2OOwEUxBpzpdKY9ByY5+8IKhHMbZexWSCeJqiKO6jOOKZ4qe594FiztyDZbJ5I95CDhUlKJyaWflMo/bcqUCjpm0QQsErngZBNNOMu7SVKMGZQy6h6mdiJ9rDzIozroZE3OrCOZ2dNP25n4HHC3X9pkTpXHdB7M+Jy0zoM5Fbr344k2B02N2ujs9xNpKi9Sux1anX51EpXdGOcYEpdnfxnfZP/5B/6HWiIKZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDM5NSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9UktuxUAI2+cUXKDS8JvPeVJV3bz7b2tDUqkqvIkxxjB9ypC55UtdEnGFybderls8pnwuW1qZeYi7i40lPrbcl+4htl10LrE4HUfyCzKdKkSozarRofhCloUHkE7woQvCfTn+4y+AwdewDbjhPTJBsCTmKULGblEZmhJBEWHnkRWopFCfWcLfUe7r9zIFam+MpQtjHPQJtAVCbUjEAupAAETslFStkI5nJBO/Fd1nYhxg59GyAa4ZVESWe+zHiKnOqIy8RMQ+T036KJZMLVbGblMZX/yUjNR8dAUqqTTylPLQVbPQC1iJeRL2OfxI+OfWbCGGOm7W8onlHzPFMhLOYEs5YKGX40fg21l1Ea4dubjOdIEfldZwTLTrfsj1T/5021rNdbxyCKJA5U1B8LsOrkaxxMQyPp2NKXqiLLAamrxGM8FhEBHW98PIAxr9crwQNKdrIrRYIpu1YkSNimxzPb0E1kzvxTnWwxPCbO+d1qGyMzMqIYLauoZq60B2s77zcLafPzPoom0KZW5kc3RyZWFtCmVuZG9iagoyNSAwIG9iago8PCAvTGVuZ3RoIDI0OSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxNUUmKAzAMu+cV+kAhXpO8p0OZQ+f/18oOhTkECa+Sk5aYWAsPMYQfLD34kSFzN/0bfqLZu1l6ksnZ/5jnIlNR+FKoLmJCXYgbz6ER8D2haxJZsb3xOSyjmXO+Bx+FuAQzoQFjfUkyuajmlSETTgx1HA5apMK4a2LD4lrRPI3cbvtGZmUmhA2PZELcGICIIOsCshgslDY2EzJZzgPtDckNWmDXqRtRi4IrlNYJdKJWxKrM4LPm1nY3Qy3y4Kh98fpoVpdghdFL9Vh4X4U+mKmZdu6SQnrhTTsizB4KpDI7LSu1e8TqboH6P8tS8P3J9/gdrw/N/FycCmVuZHN0cmVhbQplbmRvYmoKMjYgMCBvYmoKPDwgL0xlbmd0aCA5NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFjcERwCAIBP9UQQkKCtpPJpOH9v+NEDJ8YOcO7oQFC7Z5Rh8FlSZeFVgHSmPcUI9AveFyLcncBQ9wJ3/a0FScltN3aZFJVSncpBJ5/w5nJpCoedFjnfcLY/sjPAplbmRzdHJlYW0KZW5kb2JqCjI3IDAgb2JqCjw8IC9MZW5ndGggMzQxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEVSS25EMQjbv1NwgUjhl5DztKq6mN5/W5tM1c3gCWBseMtTpmTKsLklIyTXlE99IkOspvw0ciQipvhJCQV2lY/Ha0usjeyRqBSf2vHjsfRGptkVWvXu0aXNolHNysg5yBChnhW6snvUDtnwelxIuu+UzSEcy/9QgSxl3XIKJUFb0HfsEd8PHa6CK4JhsGsug+1lMtT/+ocWXO9992LHLoAWrOe+wQ4AqKcTtAXIGdruNiloAFW6i0nCo/J6bnaibKNV6fkcADMOMHLAiCVbHb7R3gCWfV3oRY2K/StAUVlA/MjVdsHeMclIcBbmBo69cDzFmXBLOMYCQIq94hh68CXY5i9Xroia8Al1umQvvMKe2ubnQpMId60ADl5kw62ro6iW7ek8gvZnRXJGjNSLODohklrSOYLi0qAeWuNcN7HibSOxuVff7h/hnC9c9usXS+yExAplbmRzdHJlYW0KZW5kb2JqCjI4IDAgb2JqCjw8IC9MZW5ndGggNzIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicMzK3UDBQsDQBEoYWJgrmZgYKKYZcQL6piblCLhdIDMTKAbMMgLQlnIKIZ4CYIG0QxSAWRLGZiRlEHZwBkcvgSgMAJdsWyQplbmRzdHJlYW0KZW5kb2JqCjI5IDAgb2JqCjw8IC9MZW5ndGggNDcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicMzK3UDBQsDQBEoYWJgrmZgYKKYZclhBWLhdMLAfMAtGWcAoinsGVBgC5Zw0nCmVuZHN0cmVhbQplbmRvYmoKMzAgMCBvYmoKPDwgL0xlbmd0aCAxNjMgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicRZA7EgMhDEN7TqEj+CMDPs9mMik2929j2GxSwNNYIIO7E4LU2oKJ6IKHtiXdBe+tBGdj/Ok2bjUS5AR1gFak42iUUn25xWmVdPFoNnMrC60THWYOepSjGaAQOhXe7aLkcqbuzvlDcPVf9b9i3TmbiYHJyh0IzepT3Pk2O6K6usn+pMfcrNd+K+xVYWlZS8sJt527ZkAJ3FM52qs9Px8KOvYKZW5kc3RyZWFtCmVuZG9iagozMSAwIG9iago8PCAvTGVuZ3RoIDIxOCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9ULmNBDEMy12FGljAeu2pZxaLS6b/9Ej59iLRFkVSKjWZkikvdZQlWVPeOnyWxA55huVuZDYlKkUvk7Al99AK8X2J5hT33dWWs0M0l2g5fgszKqobHdNLNppwKhO6oNzDM/oNbXQDVocesVsg0KRg17YgcscPGAzBmROLIgxKTQb/rnKPn16LGz7D8UMUkZIO5jX/WP3ycw2vU48nkW5vvuJenKkOAxEckpq8I11YsS4SEWk1QU3PwFotgLu3Xv4btCO6DED2icRxmlKOob9rcKXPL+UnU9gKZW5kc3RyZWFtCmVuZG9iagozMiAwIG9iago8PCAvTGVuZ3RoIDgzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWMuw3AMAhEe6ZgBH4m9j5RlMLevw0QJW64J909XB0JmSluM8NDBp4MLIZdcYH0ljALXEdQjp3so2HVvuoEjfWmUvPvD5Se7KzihusBAkIaZgplbmRzdHJlYW0KZW5kb2JqCjMzIDAgb2JqCjw8IC9MZW5ndGggNTEgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicMza0UDBQMDQwB5JGhkCWkYlCiiEXSADEzOWCCeaAWQZAGqI4B64mhyuDKw0A4bQNmAplbmRzdHJlYW0KZW5kb2JqCjM0IDAgb2JqCjw8IC9MZW5ndGggMTYwIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWQORIDMQgEc72CJ0hcgvesy7XB+v+pB9ZHoukCNBy6Fk3KehRoPumxRqG60GvoLEqSRMEWkh1Qp2OIOyhITEhjkki2HoMjmlizXZiZVCqzUuG0acXCv9la1chEjXCN/InpBlT8T+pclPBNg6+SMfoYVLw7g4xJ+F5F3Fox7f5EMLEZ9glvRSYFhImxqdm+z2CGzPcK1zjH8w1MgjfrCmVuZHN0cmVhbQplbmRvYmoKMzUgMCBvYmoKPDwgL0xlbmd0aCAzMzQgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicLVJLcsUgDNtzCl2gM/gH5DzpdLp4vf+2kpNFRg5g9DHlholKfFkgt6PWxLeNzECF4a+rzIXPSNvIOojLkIu4ki2Fe0Qs5DHEPMSC76vxHh75rMzJswfGL9l3Dyv21IRlIePFGdphFcdhFeRYsHUhqnt4U6TDqSTY44v/PsVzLQQtfEbQgF/kn6+O4PmSFmn3mG3TrnqwTDuqpLAcbE9zXiZfWme5Oh7PB8n2rtgRUrsCFIW5M85z4SjTVka0FnY2SGpcbG+O/VhK0IVuXEaKI5CfqSI8oKTJzCYK4o+cHnIqA2Hqmq50chtVcaeezDWbi7czSWbrvkixmcJ5XTiz/gxTZrV5J89yotSpCO+xZ0vQ0Dmunr2WWWh0mxO8pITPxk5PTr5XM+shORUJqWJaV8FpFJliCdsSX1NRU5p6Gf778u7xO37+ASxzfHMKZW5kc3RyZWFtCmVuZG9iagozNiAwIG9iago8PCAvTGVuZ3RoIDMyMCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UktuBTEI288puECl8E/O86qqi777b2sTvRVMMGDjKS9Z0ku+1CXbpcPkWx/3JbFC3o/tmsxSxfcWsxTPLa9HzxG3LQoEURM9WJkvFSLUz/ToOqhwSp+BVwi3FBu8g0kAg2r4Bx6lMyBQ50DGu2IyUgOCJNhzaXEIiXImiX+kvJ7fJ62kofQ9WZnL35NLpdAdTU7oAcXKxUmgXUn5oJmYSkSSl+t9sUL0hsCSPD5HMcmA7DaJbaIFJucepSXMxBQ6sMcCvGaa1VXoYMIehymMVwuzqB5s8lsTlaQdreMZ2TDeyzBTYqHhsAXU5mJlgu7l4zWvwojtUZNdw3Duls13CNFo/hsWyuBjFZKAR6exEg1pOMCIwJ5eOMVe8xM5DsCIY52aLAxjaCaneo6JwNCes6VhxsceWvXzD1TpfIcKZW5kc3RyZWFtCmVuZG9iagozNyAwIG9iago8PCAvTGVuZ3RoIDE4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozOCAwIG9iago8PCAvTGVuZ3RoIDEzMyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM5IDAgb2JqCjw8IC9MZW5ndGggMzQwIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVSOW4EMQzr/Qp9IIBu2+/ZIEiR/L8NqdkUA3F0UpQ7WlR2y4eFVLXsdPm0ldoSN+R3ZYXECcmrEu1ShkiovFYh1e+ZMq+3NWcEyFKlwuSk5HHJgj/DpacLx/m2sa/lyB2PHlgVI6FEwDLFxOgals7usGZbfpZpwI94hJwr1i3HWAVSG9047Yr3oXktsgaIvZmWigodVokWfkHxoEeNffYYVFgg0e0cSXCMiVCRgHaB2kgMOXssdlEf9DMoMRPo2htF3EGBJZKYOcW6dPTf+NCxoP7YjDe/OirpW1pZY9I+G+2Uxiwy6XpY9HTz1seDCzTvovzn1QwSNGWNksYHrdo5hqKZUVZ4t0OTDc0xxyHzDp7DGQlK+jwUv48lEx2UyN8ODaF/Xx6jjJw23gLmoj9tFQcO4rPDXrmBFUoXa5L3AalM6IHp/6/xtb7X1x8d7YDGCmVuZHN0cmVhbQplbmRvYmoKNDAgMCBvYmoKPDwgL0xlbmd0aCAyNTEgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicLVFJcgNBCLvPK/SEZqffY5crh+T/1wjKBwYNi0B0WuKgjJ8gLFe85ZGraMPfMzGC3wWHfivXbVjkQFQgSWNQNaF28Xr0HthxmAnMk9awDGasD/yMKdzoxeExGWe312XUEOxdrz2ZQcmsXMQlExdM1WEjZw4/mTIutHM9NyDnRliXYZBuVhozEo40hUghhaqbpM4EQRKMrkaNNnIU+6Uvj3SGVY2oMexzLW1fz004a9DsWKzy5JQeXXEuJxcvrBz09TYDF1FprPJASMD9bg/1c7KT33hL584W0+N7zcnywlRgxZvXbkA21eLfvIjj+4yv5+f5/ANfYFuICmVuZHN0cmVhbQplbmRvYmoKNDEgMCBvYmoKPDwgL0xlbmd0aCAxNzQgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTZBJDkMhDEP3nMIXqIQzwOc8v6q6aO+/rUMHdYH85CBwPDzQcSQudGTojI4rmxzjwLMgY+LROP/JuD7EMUHdoi1Yl3bH2cwSc8IyMQK2RsnZPKLAD8dcCBJklx++wCAiXY/5VvNZk/TPtzvdj7q0Zl89osCJ7AjFsAFXgP26x4FLwvle0+SXKiVjE4fygeoiUjY7oRC1VOxyqoqz3ZsrcBX0/NFD7u0FtSM83wplbmRzdHJlYW0KZW5kb2JqCjQyIDAgb2JqCjw8IC9MZW5ndGggNzUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjQzIDAgb2JqCjw8IC9MZW5ndGggODkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicNU25EYAwDOs9hUfAj0i8D8dRhP1b7IQ0lk6fEcoHa+QBguGNLyH4oi8ZhLULDyr7SHTYRA1nFSQTw68s8KqcFW1zJRPZWUyjs0HL9K3tb4Meuj/djhwKCmVuZHN0cmVhbQplbmRvYmoKNDQgMCBvYmoKPDwgL0xlbmd0aCAxNDEgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPY/BDsMwCEPv+Qr/QKTYKaF8T6dqh+7/ryNLuwt6AmOMhdDQG6qaw4Zgm+PF0iVUa/gUxUAlN8iZYA6lpNIdR5F6YjgYXB60G47isej6EbuSZn3QxkK6JWiAe6xTadymcRPEHTUF6inqnKO8ELmfqWfYNJLdNLOSc7gNv3vPU9f/p6u8y/kFvXcu/gplbmRzdHJlYW0KZW5kb2JqCjQ1IDAgb2JqCjw8IC9MZW5ndGggMjE1IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVROQ4DIQzs9xX+QCSML3hPoijN/r/NjNFWHsFchrSUIZnyUpOoIeVTPnqZLpy63NfMajTnlrQtc4C4trwvrZLAiWaIg8FpmLgBmjwBQ9fRqFFDFx7Q1KVTKLDcBD6Kt24P3WO1gZe2IeeJIGIoGSxBzalFExZtzyekNb9eixvel+3dyFOlxpYYgQYBVjgc1+jX8JU9TybRdBUy1Ks1yxgJE0UiPPmOptUT61o00jIS1MYRrGoDvDv9ME4AABNxywJkn0qUs+TEb7H0swZX+v4Bn0dUlgplbmRzdHJlYW0KZW5kb2JqCjE1IDAgb2JqCjw8IC9UeXBlIC9Gb250IC9CYXNlRm9udCAvQk1RUURWK0RlamFWdVNhbnMgL0ZpcnN0Q2hhciAwIC9MYXN0Q2hhciAyNTUKL0ZvbnREZXNjcmlwdG9yIDE0IDAgUiAvU3VidHlwZSAvVHlwZTMgL05hbWUgL0JNUVFEVitEZWphVnVTYW5zCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnRNYXRyaXggWyAwLjAwMSAwIDAgMC4wMDEgMCAwIF0KL0NoYXJQcm9jcyAxNiAwIFIKL0VuY29kaW5nIDw8IC9UeXBlIC9FbmNvZGluZwovRGlmZmVyZW5jZXMgWyAzMiAvc3BhY2UgNDYgL3BlcmlvZCA0OCAvemVybyAvb25lIC90d28gL3RocmVlIC9mb3VyIC9maXZlIC9zaXggNTYKL2VpZ2h0IDY4IC9EIDgwIC9QIDk3IC9hIC9iIC9jIC9kIC9lIDEwMyAvZyAxMDUgL2kgMTA4IC9sIDExMCAvbiAvbyAxMTQgL3IKL3MgL3QgL3UgL3YgMTIwIC94IC95IF0KPj4KL1dpZHRocyAxMyAwIFIgPj4KZW5kb2JqCjE0IDAgb2JqCjw8IC9UeXBlIC9Gb250RGVzY3JpcHRvciAvRm9udE5hbWUgL0JNUVFEVitEZWphVnVTYW5zIC9GbGFncyAzMgovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Bc2NlbnQgOTI5IC9EZXNjZW50IC0yMzYgL0NhcEhlaWdodCAwCi9YSGVpZ2h0IDAgL0l0YWxpY0FuZ2xlIDAgL1N0ZW1WIDAgL01heFdpZHRoIDEzNDIgPj4KZW5kb2JqCjEzIDAgb2JqClsgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAKNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCAzMTggNDAxIDQ2MCA4MzggNjM2Cjk1MCA3ODAgMjc1IDM5MCAzOTAgNTAwIDgzOCAzMTggMzYxIDMxOCAzMzcgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNgo2MzYgNjM2IDMzNyAzMzcgODM4IDgzOCA4MzggNTMxIDEwMDAgNjg0IDY4NiA2OTggNzcwIDYzMiA1NzUgNzc1IDc1MiAyOTUKMjk1IDY1NiA1NTcgODYzIDc0OCA3ODcgNjAzIDc4NyA2OTUgNjM1IDYxMSA3MzIgNjg0IDk4OSA2ODUgNjExIDY4NSAzOTAgMzM3CjM5MCA4MzggNTAwIDUwMCA2MTMgNjM1IDU1MCA2MzUgNjE1IDM1MiA2MzUgNjM0IDI3OCAyNzggNTc5IDI3OCA5NzQgNjM0IDYxMgo2MzUgNjM1IDQxMSA1MjEgMzkyIDYzNCA1OTIgODE4IDU5MiA1OTIgNTI1IDYzNiAzMzcgNjM2IDgzOCA2MDAgNjM2IDYwMCAzMTgKMzUyIDUxOCAxMDAwIDUwMCA1MDAgNTAwIDEzNDIgNjM1IDQwMCAxMDcwIDYwMCA2ODUgNjAwIDYwMCAzMTggMzE4IDUxOCA1MTgKNTkwIDUwMCAxMDAwIDUwMCAxMDAwIDUyMSA0MDAgMTAyMyA2MDAgNTI1IDYxMSAzMTggNDAxIDYzNiA2MzYgNjM2IDYzNiAzMzcKNTAwIDUwMCAxMDAwIDQ3MSA2MTIgODM4IDM2MSAxMDAwIDUwMCA1MDAgODM4IDQwMSA0MDEgNTAwIDYzNiA2MzYgMzE4IDUwMAo0MDEgNDcxIDYxMiA5NjkgOTY5IDk2OSA1MzEgNjg0IDY4NCA2ODQgNjg0IDY4NCA2ODQgOTc0IDY5OCA2MzIgNjMyIDYzMiA2MzIKMjk1IDI5NSAyOTUgMjk1IDc3NSA3NDggNzg3IDc4NyA3ODcgNzg3IDc4NyA4MzggNzg3IDczMiA3MzIgNzMyIDczMiA2MTEgNjA1CjYzMCA2MTMgNjEzIDYxMyA2MTMgNjEzIDYxMyA5ODIgNTUwIDYxNSA2MTUgNjE1IDYxNSAyNzggMjc4IDI3OCAyNzggNjEyIDYzNAo2MTIgNjEyIDYxMiA2MTIgNjEyIDgzOCA2MTIgNjM0IDYzNCA2MzQgNjM0IDU5MiA2MzUgNTkyIF0KZW5kb2JqCjE2IDAgb2JqCjw8IC9EIDE3IDAgUiAvUCAxOCAwIFIgL2EgMTkgMCBSIC9iIDIwIDAgUiAvYyAyMSAwIFIgL2QgMjIgMCBSIC9lIDIzIDAgUgovZWlnaHQgMjQgMCBSIC9maXZlIDI1IDAgUiAvZm91ciAyNiAwIFIgL2cgMjcgMCBSIC9pIDI4IDAgUiAvbCAyOSAwIFIKL24gMzAgMCBSIC9vIDMxIDAgUiAvb25lIDMyIDAgUiAvcGVyaW9kIDMzIDAgUiAvciAzNCAwIFIgL3MgMzUgMCBSCi9zaXggMzYgMCBSIC9zcGFjZSAzNyAwIFIgL3QgMzggMCBSIC90aHJlZSAzOSAwIFIgL3R3byA0MCAwIFIgL3UgNDEgMCBSCi92IDQyIDAgUiAveCA0MyAwIFIgL3kgNDQgMCBSIC96ZXJvIDQ1IDAgUiA+PgplbmRvYmoKMyAwIG9iago8PCAvRjEgMTUgMCBSID4+CmVuZG9iago0IDAgb2JqCjw8IC9BMSA8PCAvVHlwZSAvRXh0R1N0YXRlIC9DQSAwIC9jYSAxID4+Ci9BMiA8PCAvVHlwZSAvRXh0R1N0YXRlIC9DQSAxIC9jYSAxID4+Ci9BMyA8PCAvVHlwZSAvRXh0R1N0YXRlIC9DQSAxIC9jYSAwLjUgPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgPj4KZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFsgMTEgMCBSIF0gL0NvdW50IDEgPj4KZW5kb2JqCjQ2IDAgb2JqCjw8IC9DcmVhdG9yIChNYXRwbG90bGliIHYzLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjguMCkgL0NyZWF0aW9uRGF0ZSAoRDoyMDIzMTAxMTE2NDMzOVopCj4+CmVuZG9iagp4cmVmCjAgNDcKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMTUzMDQgMDAwMDAgbiAKMDAwMDAxNTA2OSAwMDAwMCBuIAowMDAwMDE1MTAxIDAwMDAwIG4gCjAwMDAwMTUyNDEgMDAwMDAgbiAKMDAwMDAxNTI2MiAwMDAwMCBuIAowMDAwMDE1MjgzIDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM0MSAwMDAwMCBuIAowMDAwMDA1MzA1IDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwNTI4NCAwMDAwMCBuIAowMDAwMDEzNjcyIDAwMDAwIG4gCjAwMDAwMTM0NjUgMDAwMDAgbiAKMDAwMDAxMjk5NCAwMDAwMCBuIAowMDAwMDE0NzI1IDAwMDAwIG4gCjAwMDAwMDUzMjUgMDAwMDAgbiAKMDAwMDAwNTU2MiAwMDAwMCBuIAowMDAwMDA1ODA1IDAwMDAwIG4gCjAwMDAwMDYxODUgMDAwMDAgbiAKMDAwMDAwNjUwMiAwMDAwMCBuIAowMDAwMDA2ODA3IDAwMDAwIG4gCjAwMDAwMDcxMTEgMDAwMDAgbiAKMDAwMDAwNzQzMyAwMDAwMCBuIAowMDAwMDA3OTAxIDAwMDAwIG4gCjAwMDAwMDgyMjMgMDAwMDAgbiAKMDAwMDAwODM4OSAwMDAwMCBuIAowMDAwMDA4ODAzIDAwMDAwIG4gCjAwMDAwMDg5NDcgMDAwMDAgbiAKMDAwMDAwOTA2NiAwMDAwMCBuIAowMDAwMDA5MzAyIDAwMDAwIG4gCjAwMDAwMDk1OTMgMDAwMDAgbiAKMDAwMDAwOTc0OCAwMDAwMCBuIAowMDAwMDA5ODcxIDAwMDAwIG4gCjAwMDAwMTAxMDQgMDAwMDAgbiAKMDAwMDAxMDUxMSAwMDAwMCBuIAowMDAwMDEwOTA0IDAwMDAwIG4gCjAwMDAwMTA5OTQgMDAwMDAgbiAKMDAwMDAxMTIwMCAwMDAwMCBuIAowMDAwMDExNjEzIDAwMDAwIG4gCjAwMDAwMTE5MzcgMDAwMDAgbiAKMDAwMDAxMjE4NCAwMDAwMCBuIAowMDAwMDEyMzMxIDAwMDAwIG4gCjAwMDAwMTI0OTIgMDAwMDAgbiAKMDAwMDAxMjcwNiAwMDAwMCBuIAowMDAwMDE1MzY0IDAwMDAwIG4gCnRyYWlsZXIKPDwgL1NpemUgNDcgL1Jvb3QgMSAwIFIgL0luZm8gNDYgMCBSID4+CnN0YXJ0eHJlZgoxNTUxNQolJUVPRgo=", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T16:43:39.263469\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["mu = Tensor([128])\n", "sigma = Tensor([2.0])\n", "\n", "\n", "def discrete_logistic(x, mu, sigma):\n", " return torch.sigmoid((x + 0.5 - mu) / sigma) - torch.sigmoid((x - 0.5 - mu) / sigma)\n", "\n", "\n", "x = torch.arange(256)\n", "p = discrete_logistic(x, mu, sigma)\n", "\n", "# Visualization\n", "plt.figure(figsize=(6, 3))\n", "plt.bar(x.numpy(), p.numpy(), **plot_args)\n", "plt.xlim(96, 160)\n", "plt.title(\"Discrete logistic distribution\")\n", "plt.xlabel(\"Pixel value\")\n", "plt.ylabel(\"Probability\")\n", "plt.show()\n", "plt.close()"]}, {"cell_type": "markdown", "id": "aa95bd8a", "metadata": {"papermill": {"duration": 0.042576, "end_time": "2023-10-11T16:43:39.769774", "exception": false, "start_time": "2023-10-11T16:43:39.727198", "status": "completed"}, "tags": []}, "source": ["Instead of the softmax, the model would output mean and standard\n", "deviations for the $K$ logistics we use in the mixture. This is one of\n", "the improvements in autoregressive models that PixelCNN++ [3] has\n", "introduced compared to the original PixelCNN."]}, {"cell_type": "markdown", "id": "0419b360", "metadata": {"papermill": {"duration": 0.043157, "end_time": "2023-10-11T16:43:39.854743", "exception": false, "start_time": "2023-10-11T16:43:39.811586", "status": "completed"}, "tags": []}, "source": ["## Conclusion\n", "\n", "In this tutorial, we have looked at autoregressive image modeling, and\n", "implemented the PixelCNN architecture. With the usage of masked\n", "convolutions, we are able to apply a convolutional network in which a\n", "pixel is only influenced by all its predecessors. Separating the masked\n", "convolution into a horizontal and vertical stack allowed us to remove\n", "the known blind spot on the right upper row of a pixel. In experiments,\n", "autoregressive models outperformed normalizing flows in terms of bits\n", "per dimension, but are much slower to sample from. Improvements, that we\n", "have not implemented ourselves here, are discrete logistic mixtures, a\n", "downsampling architecture, and changing the pixel order in a diagonal\n", "fashion (see PixelSNAIL). Overall, autoregressive models are another,\n", "strong family of generative models, which however are mostly used in\n", "sequence tasks because of their linear scaling in sampling time than\n", "quadratic as on images."]}, {"cell_type": "markdown", "id": "05634737", "metadata": {"papermill": {"duration": 0.042491, "end_time": "2023-10-11T16:43:39.939987", "exception": false, "start_time": "2023-10-11T16:43:39.897496", "status": "completed"}, "tags": []}, "source": ["## References\n", "[1] van den Oord, A., et al.\n", "\"Pixel Recurrent Neural Networks.\"\n", "arXiv preprint arXiv:1601.06759 (2016).\n", "[Link](https://arxiv.org/abs/1601.06759)\n", "\n", "[2] van den Oord, A., et al.\n", "\"Conditional Image Generation with PixelCNN Decoders.\"\n", "In Advances in Neural Information Processing Systems 29, pp.\n", "4790\u20134798 (2016).\n", "[Link](http://papers.nips.cc/paper/6527-conditional-image-generation-with-pixelcnn-decoders.pdf)\n", "\n", "[3] Salimans, Tim, et al.\n", "\"PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications.\"\n", "arXiv preprint arXiv:1701.05517 (2017).\n", "[Link](https://arxiv.org/abs/1701.05517)"]}, {"cell_type": "markdown", "id": "d3b1e11e", "metadata": {"papermill": {"duration": 0.041858, "end_time": "2023-10-11T16:43:40.023939", "exception": false, "start_time": "2023-10-11T16:43:39.982081", "status": "completed"}, "tags": []}, "source": ["## Congratulations - Time to Join the Community!\n", "\n", "Congratulations on completing this notebook tutorial! If you enjoyed this and would like to join the Lightning\n", "movement, you can do so in the following ways!\n", "\n", "### Star [Lightning](https://github.com/Lightning-AI/lightning) on GitHub\n", "The easiest way to help our community is just by starring the GitHub repos! This helps raise awareness of the cool\n", "tools we're building.\n", "\n", "### Join our [Slack](https://www.pytorchlightning.ai/community)!\n", "The best way to keep up to date on the latest advancements is to join our community! Make sure to introduce yourself\n", "and share your interests in `#general` channel\n", "\n", "\n", "### Contributions !\n", "The best way to contribute to our community is to become a code contributor! At any time you can go to\n", "[Lightning](https://github.com/Lightning-AI/lightning) or [Bolt](https://github.com/Lightning-AI/lightning-bolts)\n", "GitHub Issues page and filter for \"good first issue\".\n", "\n", "* [Lightning good first issue](https://github.com/Lightning-AI/lightning/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22)\n", "* [Bolt good first issue](https://github.com/Lightning-AI/lightning-bolts/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22)\n", "* You can also contribute your own notebooks with useful examples !\n", "\n", "### Great thanks from the entire Pytorch Lightning Team for your interest !\n", "\n", "[![Pytorch Lightning](data:image/png;base64,NDA0OiBOb3QgRm91bmQ=){height=\"60px\" width=\"240px\"}](https://pytorchlightning.ai)"]}, {"cell_type": "raw", "metadata": {"raw_mimetype": "text/restructuredtext"}, "source": [".. customcarditem::\n", " :header: Tutorial 10: Autoregressive Image Modeling\n", " :card_description: In this tutorial, we implement an autoregressive likelihood model for the task of image modeling. Autoregressive models are naturally strong generative models that constitute...\n", " :tags: Image,GPU/TPU,UvA-DL-Course\n", " :image: _static/images/course_UvA-DL/10-autoregressive-image-modeling.jpg"]}], "metadata": {"jupytext": {"cell_metadata_filter": "colab,colab_type,id,-all", "formats": "ipynb,py:percent", "main_language": "python"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.12"}, "papermill": {"default_parameters": {}, "duration": 907.793146, "end_time": "2023-10-11T16:43:42.431236", "environment_variables": {}, "exception": null, "input_path": "course_UvA-DL/10-autoregressive-image-modeling/Autoregressive_Image_Modeling.ipynb", "output_path": ".notebooks/course_UvA-DL/10-autoregressive-image-modeling.ipynb", "parameters": {}, "start_time": "2023-10-11T16:28:34.638090", "version": "2.4.0"}, "widgets": {"application/vnd.jupyter.widget-state+json": {"state": {"0e56a919ff994f698fd233e0cedbaaf8": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "0f631515cd3e46bc90b7534b7288422b": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_b76de0e9228b426695822f45773f07fd", "placeholder": "\u200b", "style": "IPY_MODEL_1ded9662ce6c4e1ea617f11a7e99c44a", "tabbable": null, "tooltip": null, "value": " 93%"}}, "1ded9662ce6c4e1ea617f11a7e99c44a": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "1e120d9cf8004b40aa5e2160aa0a54a8": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": null}}, "1efb8e0f64c14d1db0009992b98008c5": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_0f631515cd3e46bc90b7534b7288422b", "IPY_MODEL_e2299437f50e43a3a63aa7cb2138121d", "IPY_MODEL_70670ef05b0b466fa4b10336919ade05"], "layout": "IPY_MODEL_ef71c3339e4d41caaefad4ae3fa58506", "tabbable": null, "tooltip": null}}, "1f7aab1c20b94af1a21f27e0e17eab70": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "20aaeac66e434350ba7b9a3887c28d3e": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "286aab3bed5640b9b86f53c61774d760": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "2cfff5ba0d0544238647ca6b66571412": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_9ab63dc599f949769c2f3964a10bf1b7", "placeholder": "\u200b", "style": "IPY_MODEL_96e30cb44cf941e18da0a827aa22fa58", "tabbable": null, "tooltip": null, "value": " 28/28 [00:01<00:00, 13.40it/s]"}}, "2edb8f542ed54952acbb2523f3684bfc": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_e479166e719048508d553287dd5d1fa2", "IPY_MODEL_fee1273514f844129fc63c113e48983d", "IPY_MODEL_bd8457fb6ec842adae2927a139b75dd6"], "layout": "IPY_MODEL_1e120d9cf8004b40aa5e2160aa0a54a8", "tabbable": null, "tooltip": null}}, "3c1504bd07f7429992e82357ed0c93c3": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_be291b59904c43568a1c98da286dce34", "IPY_MODEL_4de6e0c4c5a140c7899db1e8d813ba90", "IPY_MODEL_7358c01648b1468eafa9b22ce5bc22b7"], "layout": "IPY_MODEL_4db3183a6fb04702bfadd7f316f4a672", "tabbable": null, "tooltip": null}}, "41ccb9d6e78546d0ac13c9a76568eb34": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_457d3022ee93421ebfa582ed2b36fb95", "IPY_MODEL_9c79aca42dae4225b6671202e0240022", "IPY_MODEL_b1f79a205e8e4cdc94b5b590f8cc35f8"], "layout": "IPY_MODEL_aae0ad24a9544268adc47b7ee1d74ee0", "tabbable": null, "tooltip": null}}, "4287a288447f40d6ab0b5baa22a19c34": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_20aaeac66e434350ba7b9a3887c28d3e", "placeholder": "\u200b", "style": "IPY_MODEL_442571c5f61f4c7d917e4cba64662d96", "tabbable": null, "tooltip": null, "value": "100%"}}, "442571c5f61f4c7d917e4cba64662d96": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "457d3022ee93421ebfa582ed2b36fb95": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_b6a7298a39a54cc18f0ac65f46925b18", "placeholder": "\u200b", "style": "IPY_MODEL_0e56a919ff994f698fd233e0cedbaaf8", "tabbable": null, "tooltip": null, "value": "100%"}}, "4b9ecf4f1b57450ca305f2a942a42708": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": null}}, "4db3183a6fb04702bfadd7f316f4a672": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": null}}, "4de6e0c4c5a140c7899db1e8d813ba90": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_be17ea1ee216442c866045e4ba435bc3", "max": 64.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_86fb5ea0a828471b87200aaf2523170d", "tabbable": null, "tooltip": null, "value": 64.0}}, "576fbf78aee640b8858dd5732441d85d": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "688b1e51f42b4c1d8a8e5225df145632": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "6f2ca2f208994e7ab6ffb973d117e609": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "70670ef05b0b466fa4b10336919ade05": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_286aab3bed5640b9b86f53c61774d760", "placeholder": "\u200b", "style": "IPY_MODEL_ac869c606c6e4610acd99ba41923888f", "tabbable": null, "tooltip": null, "value": " 26/28 [00:01<00:00, 14.35it/s]"}}, "7243cacdc61044ffac82a99d63361d34": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "7358c01648b1468eafa9b22ce5bc22b7": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_7c2b64e05c954190a7b7320b62e05850", "placeholder": "\u200b", "style": "IPY_MODEL_a77a947b74664433ae9085a5418e4a76", "tabbable": null, "tooltip": null, "value": " 64/64 [07:11<00:00, 6.56s/it]"}}, "7395d51d66f74a91b813bd91570e0dce": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "7c2b64e05c954190a7b7320b62e05850": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "7f31ccc1ef3944fe81aacfa0a09a980d": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "86fb5ea0a828471b87200aaf2523170d": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "88989ec6bb41465fac2c1a7eda8be3dc": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_9ad8569c97f8429e981c1065729821c0", "max": 28.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_6f2ca2f208994e7ab6ffb973d117e609", "tabbable": null, "tooltip": null, "value": 28.0}}, "96e30cb44cf941e18da0a827aa22fa58": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "9ab63dc599f949769c2f3964a10bf1b7": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "9ad8569c97f8429e981c1065729821c0": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "9c79aca42dae4225b6671202e0240022": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_7f31ccc1ef3944fe81aacfa0a09a980d", "max": 28.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_576fbf78aee640b8858dd5732441d85d", "tabbable": null, "tooltip": null, "value": 28.0}}, "9d39ae4886cb42588072d053b24224e4": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "a7554b9341ee43e6b29cd9a39a830c18": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "a77a947b74664433ae9085a5418e4a76": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "aae0ad24a9544268adc47b7ee1d74ee0": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": null}}, "ac869c606c6e4610acd99ba41923888f": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "b1f79a205e8e4cdc94b5b590f8cc35f8": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_e872215a19ca467a9542e0d2c4a7a91b", "placeholder": "\u200b", "style": "IPY_MODEL_e9181668620f4323bfead217a729c30a", "tabbable": null, "tooltip": null, "value": " 28/28 [02:57<00:00, 6.34s/it]"}}, "b6a7298a39a54cc18f0ac65f46925b18": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "b76de0e9228b426695822f45773f07fd": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "bd8457fb6ec842adae2927a139b75dd6": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_1f7aab1c20b94af1a21f27e0e17eab70", "placeholder": "\u200b", "style": "IPY_MODEL_d411295bc6f045cd96e4d8d77b9342db", "tabbable": null, "tooltip": null, "value": " 27/28 [00:01<00:00, 11.90it/s]"}}, "be17ea1ee216442c866045e4ba435bc3": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "be291b59904c43568a1c98da286dce34": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_a7554b9341ee43e6b29cd9a39a830c18", "placeholder": "\u200b", "style": "IPY_MODEL_9d39ae4886cb42588072d053b24224e4", "tabbable": null, "tooltip": null, "value": "100%"}}, "c3eba258a0de491fb084306e80ba90e9": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "c58bb9d0ea7b4d489b3c1d6d4303a34c": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "d1e2d868d5cb4fc3acdfc43ba118d35b": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_4287a288447f40d6ab0b5baa22a19c34", "IPY_MODEL_88989ec6bb41465fac2c1a7eda8be3dc", "IPY_MODEL_2cfff5ba0d0544238647ca6b66571412"], "layout": "IPY_MODEL_4b9ecf4f1b57450ca305f2a942a42708", "tabbable": null, "tooltip": null}}, "d411295bc6f045cd96e4d8d77b9342db": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "e2299437f50e43a3a63aa7cb2138121d": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_c58bb9d0ea7b4d489b3c1d6d4303a34c", "max": 28.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_7395d51d66f74a91b813bd91570e0dce", "tabbable": null, "tooltip": null, "value": 28.0}}, "e479166e719048508d553287dd5d1fa2": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "HTMLView", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_688b1e51f42b4c1d8a8e5225df145632", "placeholder": "\u200b", "style": "IPY_MODEL_7243cacdc61044ffac82a99d63361d34", "tabbable": null, "tooltip": null, "value": " 96%"}}, "e58f6fe439334ac5a5b801be93a545a8": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "e872215a19ca467a9542e0d2c4a7a91b": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "e9181668620f4323bfead217a729c30a": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "HTMLStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "HTMLStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "StyleView", "background": null, "description_width": "", "font_size": null, "text_color": null}}, "ef71c3339e4d41caaefad4ae3fa58506": {"model_module": "@jupyter-widgets/base", "model_module_version": "2.0.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "2.0.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "2.0.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border_bottom": null, "border_left": null, "border_right": null, "border_top": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": null}}, "fee1273514f844129fc63c113e48983d": {"model_module": "@jupyter-widgets/controls", "model_module_version": "2.0.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "2.0.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "2.0.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_allow_html": false, "layout": "IPY_MODEL_e58f6fe439334ac5a5b801be93a545a8", "max": 28.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_c3eba258a0de491fb084306e80ba90e9", "tabbable": null, "tooltip": null, "value": 28.0}}}, "version_major": 2, "version_minor": 0}}}, "nbformat": 4, "nbformat_minor": 5}