{"cells": [{"cell_type": "markdown", "id": "ac5012e0", "metadata": {"papermill": {"duration": 0.009832, "end_time": "2022-05-03T01:16:05.501977", "exception": false, "start_time": "2022-05-03T01:16:05.492145", "status": "completed"}, "tags": []}, "source": ["\n", "# Tutorial 10: Autoregressive Image Modeling\n", "\n", "* **Author:** Phillip Lippe\n", "* **License:** CC BY-SA\n", "* **Generated:** 2022-05-03T02:43:16.674251\n", "\n", "In this tutorial, we implement an autoregressive likelihood model for the task of image modeling.\n", "Autoregressive models are naturally strong generative models that constitute one of the current\n", "state-of-the-art architectures on likelihood-based image modeling,\n", "and are also the basis for large language generation models such as GPT3.\n", "We will focus on the PixelCNN architecture in this tutorial, and apply it to MNIST modeling.\n", "This notebook is part of a lecture series on Deep Learning at the University of Amsterdam.\n", "The full list of tutorials can be found at https://uvadlc-notebooks.rtfd.io.\n", "\n", "\n", "---\n", "Open in [![Open In Colab](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAHUAAAAUCAYAAACzrHJDAAAIuUlEQVRoQ+1ZaVRURxb+qhdolmbTUVSURpZgmLhHbQVFZIlGQBEXcMvJhKiTEzfigjQg7oNEJ9GMGidnjnNMBs2czIzajksEFRE1xklCTKJiQLRFsUGkoUWw+82pamn79etGYoKek1B/4NW99/tu3e/dquJBAGD27NkHALxKf39WY39gyrOi+i3xqGtUoePJrFmznrmgtModorbTu8YRNZk5cybXTvCtwh7o6NR2KzuZMWNGh6jtVt7nA0ymT5/eJlF9POrh7PAQl6s8bGYa3PUum//htmebVtLRqW0q01M5keTk5FZFzU0oRle3+zxwg5Hgtb+PZiL/ZVohxCI+hL5JgjmfjPxZ26+33BG3dA+ealHPM4gQAo5rU59gsI8bRvl54t3Ca62mvHyUAhtOlLd5WSQpKcluBjumnoCLs1EARkVd9E8l3p9y2i7RbQ1B6pFwu/YDgW8KbHJHMTQrwnjz2oZm9M4pavOCfo5jWrgCaaMVcMs6/pNhDr0+AMN93XlxV7R6DNpyzi7W/OE+yIrsjU6rTrbKV5cd/pNyItOmTbMp6sbBB+EqaYJY4cWE3VUciNt1TpgfcRFv71Fi54xT5kSoyLvOBEJMOMxWXkFlBeBSX4u6Zkcs+3KszYRtiapbNRqF31UgetVuc8z9vBXIv1qD+F1f83B6uDlCUyfsZGepGPpmg01OB7EITQbhS9ribKy+DmP1DUiClLz4bnIHVOqa7BY+Z1wg5g3zgUvyehiNpnJKxSLc/ts76LKm0BzX3c0RNy1yXjDcB5lWoro4iNHQxM+f1kWeWQARAWQS++trISJTp061Kep25X/MycwtjuctSC5rxo7ppi7VNUox5+PhPHtrsS2O1qJ6yx1QujQUzm9sh6hbkBlvvGcN8hYnwjUjH6kjfZEd5c/jitz5Jc5U3ENnFynKl4eB7nyEgP2UZ+Yz3/rVEbyYr27qELrtC4FIC0J7sc7xWnmccdHfRRTs0VB+cA4lt+oFcRR/wUeH8FG5w2Mbx8FQ8TXEvv1xYf4wBP3O2WyL3/UVjpXWgIqaFeUPr+wTmDvUB7njH6/bOv+HRg4SqioAg5GDe1aB3ZeMTJkyRSBqkLsWqSEm0fZVBEN94zEZnYvrdx1JL5cxe+a+AbhSJecRRHW/ikTFRTa38dtQlNZ5CRKwFvUtZU/kvBoEF9Uxni/XqIM+dwKbTw3rhcxIf7gmr2M+H6SMwx8iBzJbw5oxeG3Lv5FX9B3AGaHPS8e8z77H7v9VMpvPG5ug1enh7eGK8h0LBTwUb+GInqzInlRUK65DmTPQu4c3+uQKjwKK77zwUxBX4Tq7yR1RuiwUsqlrABCM6esHdXoy47fk4+prYKy8ZF574x4V5BnHQBuf4g9Z9ld8U36L2aktZNNplNfw7zotwWTy5MkCUft4aLEopJj5/OPHl1BQqeAVOnHgNSQOqmBzq9V9cfEm/yx5ubMGKS9cYPZ3vx2OS/c6PVHUuUO7Y1Pci3BO/1zgq18byebfGemLtNF+6JRtOvMk926ibussZqM+1mNz4TWkH7rCbM5phwGRGDAaoF8fY5OHFnlldAA8sgoEXKnDukA1NgSeNjqkJT9brbN4pC9WRweYXyLugR73c+MYvyWfu0yC6+mjzN1Isfw3FKJS98CU/zI1IHFkFPR52cHL2FJk0sB6kMTERIGo9GzcPkLNfA0cwdwi/hfEYO86ZMd9w+y1egfM2T2Eh/vesMNwljSzuZRT420SW3eqy8N6aHMmwmnFUZ7/PGVPbIoNZvNU1BURdHs0bT2+HjL8sDSM2e6vi4Lj5NW8WOLVA6RTT2azxLV+bglaFNqLieqemS/gWkw7NyoAHo+2dEsiivengjKsPFoqWOvbSh/kxPaxyW/JRzH2Fl3EzD9/xjAefJqB3usKUFn/0Gb+S/d/jy3FN2yLOmnSJJtn6oehByEiHPSeXnDxFGPRnoFoaBJjcdQlbDwcjL1zTNuQpoxD7R0OG0uUTMi0fkVwdzBdYIwcwZunxrVJVLplNm54BZp7jfDfYLoNyqQi1K6KxIdHzmN+QQ2WjFIwUT2zTGdlRXo4NFXVUO4sgX5dFC7f0aP/ZlNeUjFBuL8Xjl6uRuP6aMjSjpjzsH62FDU7JhBuGccEXIvDfJFFBc/gHw80dklfCVYnRaDfpiJcutPA4F7qJsfJeUPQI+1fqMlNhFx1FM0GDqkjFVg7NojlQ0Vt4aM5ReSqcbpaCg8nCW5lRsBvbT4T1TLfFptsfh7gItzuKTdJSEiwKSrt1vcmnEXXrsLbYnWDA1bu+z2WKy9Arq+1KRqdfKsoBo0GcdtEpS/B1bO4v0cFiUhkjskvKcMrWwtAPHuwQq8Z+4LZ1vTQANfXt4J0DwZX9gWa9qh4XDM/voC9JXfwYEMMHJcfNtusn82ihvliVUwg5KrPGVf6GH94ZJpEZBen6EC4qYTHA1dXhW0JIex8txzv//c8lhzXIi/BFxOH9jGbQhZsRalTIBZZ8KkGyZAxeRQvXkFF1TWz/Hm46jNYUnjPbt3JxIkT7f6dSj8qfJJyVvBxgaIlblOyjtysNHWN9fjjqWi7glJfW3/S0Hlj2XnA8PhKT9w6g3Qx3XiXhvuxQsuT1proxBKI/AaZqY1Xz5muvY8G8XkRRCaHsfQsRAFDH/tZPbcYuHotOG0FRIqB4HR3wNVoIPLtz8ycTguu+jpEigE218vd1YCr5m+HpHMvEI9u4LTXwNWaLjl0iPwGAmIpeHx1VeCqTJdPs1/vweweQPO3HC24NhOhnTphwoQnfv6QSY2ICbkNmdSA4h87oaLaiYfn5diIEd4att2erOwJXbPUHp953p6orQVSUVWRAXBT8c/dJ5L9xhzaJGp71GR/wFP8P5V2z10NSC9T93QM2xUg8fHxT+zU9ijeU4naHon8CjFJXFzc8/kn+dN06q9QgF98SYSo2Xen2NjYZy5sR6f+4nLSK5Iam2PH/x87a1YN/t5sBgAAAABJRU5ErkJggg==){height=\"20px\" width=\"117px\"}](https://colab.research.google.com/github/PytorchLightning/lightning-tutorials/blob/publication/.notebooks/course_UvA-DL/10-autoregressive-image-modeling.ipynb)\n", "\n", "Give us a \u2b50 [on Github](https://www.github.com/PytorchLightning/pytorch-lightning/)\n", "| Check out [the documentation](https://pytorch-lightning.readthedocs.io/en/stable/)\n", "| Join us [on Slack](https://www.pytorchlightning.ai/community)"]}, {"cell_type": "markdown", "id": "f93a850a", "metadata": {"papermill": {"duration": 0.009298, "end_time": "2022-05-03T01:16:05.519499", "exception": false, "start_time": "2022-05-03T01:16:05.510201", "status": "completed"}, "tags": []}, "source": ["## Setup\n", "This notebook requires some packages besides pytorch-lightning."]}, {"cell_type": "code", "execution_count": 1, "id": "390783ee", "metadata": {"colab": {}, "colab_type": "code", "execution": {"iopub.execute_input": "2022-05-03T01:16:05.536508Z", "iopub.status.busy": "2022-05-03T01:16:05.536140Z", "iopub.status.idle": "2022-05-03T01:16:09.150742Z", "shell.execute_reply": "2022-05-03T01:16:09.149839Z"}, "id": "LfrJLKPFyhsK", "lines_to_next_cell": 0, "papermill": {"duration": 3.625893, "end_time": "2022-05-03T01:16:09.153162", "exception": false, "start_time": "2022-05-03T01:16:05.527269", "status": "completed"}, "tags": []}, "outputs": [], "source": ["! pip install --quiet \"ipython[notebook]\" \"torchmetrics>=0.7\" \"setuptools==59.5.0\" \"matplotlib\" \"seaborn\" \"torchvision\" \"pytorch-lightning>=1.4\" \"torch>=1.8\""]}, {"cell_type": "markdown", "id": "a678dd05", "metadata": {"papermill": {"duration": 0.007803, "end_time": "2022-05-03T01:16:09.169962", "exception": false, "start_time": "2022-05-03T01:16:09.162159", "status": "completed"}, "tags": []}, "source": ["
\n", "\n", "Similar to the language generation you have seen in assignment 2, autoregressive models work on images by modeling the likelihood of a pixel given all previous ones.\n", "For instance, in the picture below, we model the pixel $x_i$ as a conditional probability distribution\n", "based on all previous (here blue) pixels (figure credit - [Aaron van den Oord et al. ](https://arxiv.org/abs/1601.06759)):\n", "\n", "
\n", "\n", "Generally, autoregressive model over high-dimensional data $\\mathbf{x}$ factor the joint distribution as the following product of conditionals:\n", "\n", "$$p(\\mathbf{x})=p(x_1, ..., x_n)=\\prod_{i=1}^{n} p(x_i|x_1,...,x_{i-1})$$\n", "\n", "Learning these conditionals is often much simpler than learning the joint distribution $p(\\mathbf{x})$ all together.\n", "However, disadvantages of autoregressive models include slow sampling, especially for large images,\n", "as we need height-times-width forward passes through the model.\n", "In addition, for some applications, we require a latent space as modeled in VAEs and Normalizing Flows.\n", "For instance, in autoregressive models, we cannot interpolate between two images because of the lack of a latent representation.\n", "We will explore and discuss these benefits and drawbacks alongside with our implementation.\n", "\n", "Our implementation will focus on the [PixelCNN](https://arxiv.org/pdf/1606.05328.pdf) [2] model which has been discussed in detail in the lecture.\n", "Most current SOTA models use PixelCNN as their fundamental architecture,\n", "and various additions have been proposed to improve the performance\n", "(e.g. [PixelCNN++](https://arxiv.org/pdf/1701.05517.pdf) and [PixelSNAIL](http://proceedings.mlr.press/v80/chen18h/chen18h.pdf)).\n", "Hence, implementing PixelCNN is a good starting point for our short tutorial.\n", "\n", "First of all, we need to import our standard libraries. Similarly as in\n", "the last couple of tutorials, we will use [PyTorch\n", "Lightning](https://pytorch-lightning.readthedocs.io/en/stable/) here as\n", "well."]}, {"cell_type": "code", "execution_count": 2, "id": "2ccbddb0", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:16:09.187195Z", "iopub.status.busy": "2022-05-03T01:16:09.186887Z", "iopub.status.idle": "2022-05-03T01:16:12.393770Z", "shell.execute_reply": "2022-05-03T01:16:12.393066Z"}, "papermill": {"duration": 3.217496, "end_time": "2022-05-03T01:16:12.395321", "exception": false, "start_time": "2022-05-03T01:16:09.177825", "status": "completed"}, "tags": []}, "outputs": [{"name": "stderr", "output_type": "stream", "text": ["/usr/lib/python3.8/site-packages/apex/pyprof/__init__.py:5: FutureWarning: pyprof will be removed by the end of June, 2022\n", " warnings.warn(\"pyprof will be removed by the end of June, 2022\", FutureWarning)\n"]}, {"name": "stderr", "output_type": "stream", "text": ["WARNING:root:Bagua cannot detect bundled NCCL library, Bagua will try to use system NCCL instead. If you encounter any error, please run `import bagua_core; bagua_core.install_deps()` or the `bagua_install_deps.py` script to install bundled libraries.\n"]}, {"name": "stderr", "output_type": "stream", "text": ["/tmp/ipykernel_4215/1092408338.py:27: DeprecationWarning: `set_matplotlib_formats` is deprecated since IPython 7.23, directly use `matplotlib_inline.backend_inline.set_matplotlib_formats()`\n", " set_matplotlib_formats(\"svg\", \"pdf\") # For export\n", "Global seed set to 42\n"]}, {"name": "stdout", "output_type": "stream", "text": ["Using device cuda:0\n"]}], "source": ["\n", "import math\n", "import os\n", "import urllib.request\n", "from urllib.error import HTTPError\n", "\n", "# Imports for plotting\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "import pytorch_lightning as pl\n", "import seaborn as sns\n", "import torch\n", "import torch.nn as nn\n", "import torch.nn.functional as F\n", "import torch.optim as optim\n", "import torch.utils.data as data\n", "import torchvision\n", "from IPython.display import set_matplotlib_formats\n", "from matplotlib.colors import to_rgb\n", "from pytorch_lightning.callbacks import LearningRateMonitor, ModelCheckpoint\n", "from torch import Tensor\n", "from torchvision import transforms\n", "from torchvision.datasets import MNIST\n", "from tqdm.notebook import tqdm\n", "\n", "plt.set_cmap(\"cividis\")\n", "%matplotlib inline\n", "set_matplotlib_formats(\"svg\", \"pdf\") # For export\n", "\n", "# Path to the folder where the datasets are/should be downloaded (e.g. MNIST)\n", "DATASET_PATH = os.environ.get(\"PATH_DATASETS\", \"data\")\n", "# Path to the folder where the pretrained models are saved\n", "CHECKPOINT_PATH = os.environ.get(\"PATH_CHECKPOINT\", \"saved_models/tutorial12\")\n", "\n", "# Setting the seed\n", "pl.seed_everything(42)\n", "\n", "# Ensure that all operations are deterministic on GPU (if used) for reproducibility\n", "torch.backends.cudnn.determinstic = True\n", "torch.backends.cudnn.benchmark = False\n", "\n", "# Fetching the device that will be used throughout this notebook\n", "device = torch.device(\"cpu\") if not torch.cuda.is_available() else torch.device(\"cuda:0\")\n", "print(\"Using device\", device)"]}, {"cell_type": "markdown", "id": "8dbe6c09", "metadata": {"papermill": {"duration": 0.240774, "end_time": "2022-05-03T01:16:12.658095", "exception": false, "start_time": "2022-05-03T01:16:12.417321", "status": "completed"}, "tags": []}, "source": ["We again provide a pretrained model, which is downloaded below:"]}, {"cell_type": "code", "execution_count": 3, "id": "63c94f5b", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:16:13.202438Z", "iopub.status.busy": "2022-05-03T01:16:13.202144Z", "iopub.status.idle": "2022-05-03T01:16:13.424846Z", "shell.execute_reply": "2022-05-03T01:16:13.424172Z"}, "papermill": {"duration": 0.50745, "end_time": "2022-05-03T01:16:13.426521", "exception": false, "start_time": "2022-05-03T01:16:12.919071", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Downloading https://raw.githubusercontent.com/phlippe/saved_models/main/tutorial12/PixelCNN.ckpt...\n"]}], "source": ["# Github URL where saved models are stored for this tutorial\n", "base_url = \"https://raw.githubusercontent.com/phlippe/saved_models/main/tutorial12/\"\n", "# Files to download\n", "pretrained_files = [\"PixelCNN.ckpt\"]\n", "# Create checkpoint path if it doesn't exist yet\n", "os.makedirs(CHECKPOINT_PATH, exist_ok=True)\n", "\n", "# For each file, check whether it already exists. If not, try downloading it.\n", "for file_name in pretrained_files:\n", " file_path = os.path.join(CHECKPOINT_PATH, file_name)\n", " if not os.path.isfile(file_path):\n", " file_url = base_url + file_name\n", " print(\"Downloading %s...\" % file_url)\n", " try:\n", " urllib.request.urlretrieve(file_url, file_path)\n", " except HTTPError as e:\n", " print(\n", " \"Something went wrong. Please try to download the file from the GDrive folder, or contact the author with the full output including the following error:\\n\",\n", " e,\n", " )"]}, {"cell_type": "markdown", "id": "64fecce1", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.125989, "end_time": "2022-05-03T01:16:13.561816", "exception": false, "start_time": "2022-05-03T01:16:13.435827", "status": "completed"}, "tags": []}, "source": ["Similar to the Normalizing Flows in Tutorial 11, we will work on the\n", "MNIST dataset and use 8-bits per pixel (values between 0 and 255). The\n", "dataset is loaded below:"]}, {"cell_type": "code", "execution_count": 4, "id": "5fa19277", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:16:13.580665Z", "iopub.status.busy": "2022-05-03T01:16:13.580301Z", "iopub.status.idle": "2022-05-03T01:16:13.634661Z", "shell.execute_reply": "2022-05-03T01:16:13.633998Z"}, "papermill": {"duration": 0.065658, "end_time": "2022-05-03T01:16:13.636466", "exception": false, "start_time": "2022-05-03T01:16:13.570808", "status": "completed"}, "tags": []}, "outputs": [{"name": "stderr", "output_type": "stream", "text": ["/usr/local/lib/python3.8/dist-packages/torchvision/datasets/mnist.py:498: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:180.)\n", " return torch.from_numpy(parsed.astype(m[2], copy=False)).view(*s)\n", "Global seed set to 42\n"]}], "source": ["# Convert images from 0-1 to 0-255 (integers). We use the long datatype as we will use the images as labels as well\n", "def discretize(sample):\n", " return (sample * 255).to(torch.long)\n", "\n", "\n", "# Transformations applied on each image => only make them a tensor\n", "transform = transforms.Compose([transforms.ToTensor(), discretize])\n", "\n", "# Loading the training dataset. We need to split it into a training and validation part\n", "train_dataset = MNIST(root=DATASET_PATH, train=True, transform=transform, download=True)\n", "pl.seed_everything(42)\n", "train_set, val_set = torch.utils.data.random_split(train_dataset, [50000, 10000])\n", "\n", "# Loading the test set\n", "test_set = MNIST(root=DATASET_PATH, train=False, transform=transform, download=True)\n", "\n", "# We define a set of data loaders that we can use for various purposes later.\n", "train_loader = data.DataLoader(train_set, batch_size=128, shuffle=True, drop_last=True, pin_memory=True, num_workers=4)\n", "val_loader = data.DataLoader(val_set, batch_size=128, shuffle=False, drop_last=False, num_workers=4)\n", "test_loader = data.DataLoader(test_set, batch_size=128, shuffle=False, drop_last=False, num_workers=4)"]}, {"cell_type": "markdown", "id": "aedb06b9", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.008192, "end_time": "2022-05-03T01:16:13.653804", "exception": false, "start_time": "2022-05-03T01:16:13.645612", "status": "completed"}, "tags": []}, "source": ["A good practice is to always visualize some data examples to get an intuition of the data:"]}, {"cell_type": "code", "execution_count": 5, "id": "4d26abe3", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:16:13.672106Z", "iopub.status.busy": "2022-05-03T01:16:13.671803Z", "iopub.status.idle": "2022-05-03T01:16:13.975557Z", "shell.execute_reply": "2022-05-03T01:16:13.974942Z"}, "papermill": {"duration": 0.31518, "end_time": "2022-05-03T01:16:13.977197", "exception": false, "start_time": "2022-05-03T01:16:13.662017", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUiAvTWVkaWFCb3ggWyAwIDAgMzM1LjI5OTM1NDgzODcgMTc3LjQ4IF0KL1BhcmVudCAyIDAgUiAvUmVzb3VyY2VzIDggMCBSIC9UeXBlIC9QYWdlID4+CmVuZG9iago5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTIgMCBSID4+CnN0cmVhbQp4nFVOQQ6CQBC7zyv6gt2dHWCXI0hCPKIHH0CISgSjJPJ9Zz0oHpq0TdvUNsPr2g+HtsbuSPan+oUYI9mKcV7gMCpWMFr8h5z6E4nkxpel5LnK21ZyCCaL6rktTaEL0UwPBOM/EO9MTJ0sSgzgQoyLeA44YYatfLqhjxSr1lttimekKS6y70o/we4ZzR0ddfQG7/YxsQplbmRzdHJlYW0KZW5kb2JqCjEyIDAgb2JqCjE0OQplbmRvYmoKMTAgMCBvYmoKWyBdCmVuZG9iagozIDAgb2JqCjw8ID4+CmVuZG9iago0IDAgb2JqCjw8IC9BMSA8PCAvQ0EgMCAvVHlwZSAvRXh0R1N0YXRlIC9jYSAxID4+Ci9BMiA8PCAvQ0EgMSAvVHlwZSAvRXh0R1N0YXRlIC9jYSAxID4+ID4+CmVuZG9iago1IDAgb2JqCjw8ID4+CmVuZG9iago2IDAgb2JqCjw8ID4+CmVuZG9iago3IDAgb2JqCjw8IC9JMSAxMyAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9CaXRzUGVyQ29tcG9uZW50IDgKL0NvbG9yU3BhY2UgWy9JbmRleGVkIC9EZXZpY2VSR0IgMjIyICj////+/v79/f38/Pz7+/v6+vr5+fn4+Pj39/f19fX09PTz8/Py8vLx8fHw8PDv7+/u7u7t7e3s7Ozr6+vq6urp6eno6Ojn5+fm5ubl5eXk5OTj4+Pi4uLh4eHg4ODf39/e3t7d3d3c3Nzb29va2trZ2dnY2NjX19fW1tbV1dXU1NTT09PR0dHQ0NDPz8/Ozs7Nzc3MzMzLy8vJycnHx8fGxsbFxcXExMTDw8PCwsLBwcHAwMC/v7++vr69vb28vLy7u7u6urq5ubm4uLi3t7e2tra1tbW0tLSzs7OysrKwsLCvr6+urq6tra2srKyqqqqpqamoqKinp6elpaWkpKSioqKhoaGgoKCenp6cnJyampqZmZmYmJiXl5eWlpaVlZWUlJSTk5ORkZGPj4+NjY2Li4uKioqJiYmIiIiHh4eGhoaFhYWEhISDg4OCgoKAgIB+fn59fX18fHx7e3t5eXl4eHh3d3d2dnZ1dXV0dHRzc3NycnJxcXFwcHBvb29ubm5qampnZ2dmZmZkZGRjY2NiYmJgYGBfX19eXl5dXV1cXFxcXFxbW1taWlpZWVlYWFhXV1dWVlZVVVVUVFRTU1NRUVFQUFBPT09MTExLS0tKSkpJSUlISEhHR0dGRkZFRUVERERDQ0NCQkJBQUFAQEA/Pz8+Pj48PDw7Ozs6Ojo5OTk4ODg3Nzc1NTU0NDQzMzMyMjIxMTEwMDAvLy8uLi4sLCwrKysqKipcKFwoXCgnJycmJiYlJSUkJCQjIyMiIiIgICAfHx8eHh4dHR0cHBwbGxsaGhoZGRkYGBgXFxcWFhYVFRUUFBQTExMSEhIREREQEBAPDw8ODg5cclxyXHIMDAwLCwtcblxuXG4JCQkICAgHBwcGBgYFBQUEBAQDAwMCAgIBAQEAAAApXQovRGVjb2RlUGFybXMgPDwgL0NvbG9ycyAxIC9Db2x1bW5zIDMyMSAvUHJlZGljdG9yIDEwID4+Ci9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9IZWlnaHQgMTY0IC9MZW5ndGggMTQgMCBSIC9TdWJ0eXBlIC9JbWFnZQovVHlwZSAvWE9iamVjdCAvV2lkdGggMzIxID4+CnN0cmVhbQp4nO2d+19M+R/HZ6ZRKWFGJRoUWdEWspVLqnHLJSqXZV1qI7llv+7azTJZwmZlKXeKWLnLZWklconUisxmhPlbvu85M6dpTufymflMzNr384c8zmVe8/48M5/z+ZxzOiP7H0KH7HMX8K8HDdKCBmlBg7SYDRqdxn8vDg3SxqFB2jg0SBuHBmnj0CBt3JdhsLa2dvTo0adPOymOFDRIi6MGHz16VFrapUsXuVwuk0VHR3d4jVI76fVarRaqmTHDKXHkoEFa0CAtDhnctGnTpEmTFK3ExMR0eI2ie9TV1cXHyxny8ujj7AIN0oIGabHDoMGg1+uLi4ujojw9PRVt6d69+86dHVyj6B5Tpkxh9Pn6+hoM9HF2gQZpITX45MmTtp9bW8g/xfn5+TKZLCODZ5PB0NLSIlyjaHWDBg1iDO60+UUKIB23Z8+ekBBoFyQmJUHJr15JxaFBW9CgOJ/FYE7O5MmThfwpFAUFBUQ1vnoVHh4O++/fb7P62rVry5cvV6kiIiKKi4VqFKnOYhDG94cOHRLdTzqusXH27NnQz7uZgFKZf8PCSktLRePQYCvONvjxY319/dq1a728eLx17Xr+/PmbJj58+EBU49On5lfu2mU06YTsy5cvL1ni7+/PZmZl2dtkBsZgfHy86E5Ecfn5YMzPz++XXzZv3swadHPz8vI6d+6ccBwabAUNEvBJDer1vP3e1KlTU1NTL160t8b9+82v79EjPT29Xz9u7MiRI/V6e5tsNG7ZskWpVMrlixcvFt6JKO7x48chIeDroqlpVVVVB0xotVpGYlBQ0MuXQnFo0Awa/FcYhMFzXt6bN28cqlGvT0pKgkGHNa5z585abUZGBrOQxdsLShisqxs+fDgznK6trRWtSjpOp9MpFMuWLbMZ2sPCli09evSAAtPShOLQoBmnGmxuboZVzNyGRa2OjIx89uyZdSf4X/+S73+2aJPhAJ6b+5UJOMiXlJQYjXPnzlWYDu0wc7SzyUBhodyC2WBR0UETR44cEXqFQFxlZaVarQ4IgEZxN715M2DAAHiH1FShODRoRIOuZxAmgjb9X3BwcFkZ9H+72gJd2sSJE3ftOnr0KHmTuZw4oVKp4B2Ep7SicVFRrMFJkyZNnz7dzY1ZgKktLPAenQXilixZwkyBeV5RVmaeJP/2m1AcGjSiQVc36OMDs8L8/OTkZO7ohgEmtgcPkjaZyzffMH2suqZGaA9Cgzz06lVRUUEYl5KSAp50Ou56mPTPnAlbhg4d+u6dUHVo0Oh8g+YhJItSqdFo2o6E29G1a3l5OVGTuTAGly5dKrwHoUH4PQa0xcfHRyaD3oc70uaNa2zs06cPeGr/Bps2bWJmdcW8Z3/RIAsadD2DUJGILz7OnDlD1GQbioqKunUbNWpUU1OT8E5EBmNjX79+bbOlrKzMzw+2VFdXS8c9f24+lWpzuG1oaMjMdHd3Z7aIVocG0eB/1CCMNLp166ZQzJC4a5LI4E8/cbfU19eHhvr5+XEnusKjGWgFO/WtqSksLJTJZJbGZfDeJGCNQ4PGT2QQpkrDhg0rLHz69Cn4io1tsykg4PLly0RNtrJjxw7mtYcPHxbdj8hgQoLNpxgW4uPj/fzOnz9PGLdq1Sr4rHp7j2Xw94cF+P2mpo4YMcLNDQ0SxKFBVzPIOwVWqVS5JlauXMnd9MMPpE1maWiIiIiAV44eree9QkcY9+237IgafhHM+V+YyJaXjx8/HlYNHkweZzAYFi5cqFabj8mengkJCcxFdn+TzN27d4tWhwaNaND1DHLPsIoyZMiQykrSJrNYOkGFRCcoFffqFXsTNRAXF5eeDh7Y5Zwc++OYy+wHzp41Lz9/Dv9t1OoHDx6IVocG28Q5y+CVK1c0Go20vMDAwLNnDZwb6IkMJibCy3v37l1VJbWnRFxzc/PAgQM551Zh2KDT6d6/tz/Olt9/h4+06O0kaFAcNGh/nC2OGwROnTol4TArK+v48eOO1PjixYuvv4aEcePGSbdCOg4Ow4mJrDx3d/e9e4uKihyPawMzbThwQKo6NCgIlUGgvJzXXN++fc+YeMd78YqkxmvXrjE9AM+lNEfi7II8rrq6WqWCTzEadDQODdLGOcGgg0jHzZo1S6GIjY11UpxdkMfdvn2bmeKhQUfj0CBtnOsa/PPPTp06ma7jK9PT09vf8GhvnJOrs4IGaeNc1+CtW8whvV+/fvfv36ePsxPyuLdv3373nZeX1/XrUnFokB80SBvnuga/vDg0SBuHBmnj0CBtHBqkjcOnytOCBmlBg7SgQVrQIC1okBYczdDGoUHaODRIG4cGaePQIG0cGqSNQ4O0cWiQNg4N0sahQdq4T2rw3r17+SZ8fRUKBfvXk9ynCZPH2fDs2bPw8K1bt5JXd/PmzWHD5HJ5cPC0adM2btz4449Qmk6ny8iYMmVKoum2Tihx+vRFixY1NjYKx6FBNGjhizdYUjJ27NjQ0FBlKwrm7hmlUq1Wjx27evXq5mY74tqzfft2uVylUhFWd+zYsbCwsJ9/3rt3L+8bf/z48fXrkpKSDRtGjBjh7e2dklJfX88bhwbRoAVXNnjhwgXrwn7Og/WFa2xDWVmZWt1Gmo1BdiEzkzSOl4SEBLncw8ODsLrm5mapR8paePfuXW5uLvP3o7xxaFAKaoNRUXBEsi5qNBrII6mxDS9fvuzTBzzFxcXFt8Is9O/fnzXo4cFzJO0wg+QYDIYFCxagQXvjrKBBx+KsUBusqbExCPpslklrPHiQ96/tKyoqVq1aZekU16xZQxrHQa/Xx8TEmB7b2Et0P/sN3rkTHR0Nw+7Zs3kefYwGCXC+QTgWO2RQGDhQW6Z4jhs8efIk87ed27ZtE92PvLq3b99WVq5fv97Dw93dHUarf/8tFIcG+UGDtNU50WBuLhx+bdY43yB1P+hUgydOzJ8/v2fPnnI5/EhLu3v3rmgcGmwHGhThsxiMirKZFxsZgw4+gpsXZxhkv+mFyuCJE2lpacxf/UVFRSUmJp458573gSGcODTYSkcZTE7mroFBtfAJms9jMCIigt5gZKT5gSEpKW2ery0dhwZbQYMuaTAzM7P9iRhY2V4rUY28zJkzxzUM/vPP1atXV6xYERCgVCqDgoJ0uoaGBuk4NNgKGnRJg+0HzzAv1phITk6GjRqNzEKyRapdBpuakpKSmMvukODjw/MlzkRxpaVdunQBf76+Dx8+FN2TKK6uDkyuXLnS2zswMHDbNqFnExnRoBDOMwiyYGhpXYBPr6wVWIBVNTX7rZDXyHLlCnutzsfHZ8wYoRqlYgoKoCAwuHy51J52VXf3bqTp8CxyJwkaFAcNOhrHQm8QeruampoLF1hz0NXl5jJrLgifnSGvsaKiIjiYNThv3jwH41paWlJSQB/0WcInUeyvjqG+vj4yEvpY5gsyheLQoAhokKI6BjqDmZlgMMp0vR1+cOfBjhi8c+fOunXr8vJgHjxnDvzw9fVl/YWF8dwYJRHHAt0UM5geOnSo6H6EcVwaGgICAqKjo4Xj0KA4jhqsYa7QmUbMGs7pVTOc8/5SNRYXF8+bN3DgQN57t8LCwoSfSy3d5NOnT3t5gUEo9a+/RPfkiSP58vLs7Gwo9Y8/hOLQoBRoUCyu4wwy8kRutkxO5l48EaqRXcnOfq1PFWYXBg8enJ8v9EZEHVfv3mBw5syZYl98xxO3dOlSsbvQWGDMAPGmryvhj0ODUqBBwbgONSh1tzQzspaukaG2tjbYMnQWvo9648aNhHFc9u3b5+4OTRQa9QrHTZgwQau9LvqIRhOXLl2C+LVrheLQYMcYdJz2cdXV1bzSbBYcn9WFh4czw+kXL17YWd3Vq1djYz09PaOjMzIycnJyfv21rq6O+4qqqpCQECjw4kWhODSIBr98gy0tLZmZmVZpAwYM+P77Bw8eFBQUxMQoLYNqx+fF7qZekOjbYnjimprOnTuXnR0UFARdXWgoTNXHmNBqtXkm4uLiAgOhvg0bNgjHoUHXN2hkhlT+/gsXLgSD2dk3btywNmA3A82sLikpyXGDFh4+fHj48OFHj27dupWVlWXztVkzZoh8OSsaZEGDX77BLzoODdLGoUHaODRIG4cGaePQIG0cGqSNQ4O0cWiQNg4N0sahQdo4fKo8LWiQFjRICxqkBQ3S8n+rQtglCmVuZHN0cmVhbQplbmRvYmoKMTQgMCBvYmoKMzEyNAplbmRvYmoKMiAwIG9iago8PCAvQ291bnQgMSAvS2lkcyBbIDExIDAgUiBdIC9UeXBlIC9QYWdlcyA+PgplbmRvYmoKMTUgMCBvYmoKPDwgL0NyZWF0aW9uRGF0ZSAoRDoyMDIyMDUwMzAzMTYxMyswMicwMCcpCi9DcmVhdG9yIChNYXRwbG90bGliIHYzLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjUuMSkgPj4KZW5kb2JqCnhyZWYKMCAxNgowMDAwMDAwMDAwIDY1NTM1IGYgCjAwMDAwMDAwMTYgMDAwMDAgbiAKMDAwMDAwNDg3NSAwMDAwMCBuIAowMDAwMDAwNjA4IDAwMDAwIG4gCjAwMDAwMDA2MjkgMDAwMDAgbiAKMDAwMDAwMDcyOCAwMDAwMCBuIAowMDAwMDAwNzQ5IDAwMDAwIG4gCjAwMDAwMDA3NzAgMDAwMDAgbiAKMDAwMDAwMDA2NSAwMDAwMCBuIAowMDAwMDAwMzQ0IDAwMDAwIG4gCjAwMDAwMDA1ODggMDAwMDAgbiAKMDAwMDAwMDIwOCAwMDAwMCBuIAowMDAwMDAwNTY4IDAwMDAwIG4gCjAwMDAwMDA4MDIgMDAwMDAgbiAKMDAwMDAwNDg1NCAwMDAwMCBuIAowMDAwMDA0OTM1IDAwMDAwIG4gCnRyYWlsZXIKPDwgL0luZm8gMTUgMCBSIC9Sb290IDEgMCBSIC9TaXplIDE2ID4+CnN0YXJ0eHJlZgo1MDkyCiUlRU9GCg==\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2022-05-03T03:16:13.723073\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.5.1, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}], "source": ["def show_imgs(imgs):\n", " num_imgs = imgs.shape[0] if isinstance(imgs, Tensor) else len(imgs)\n", " nrow = min(num_imgs, 4)\n", " ncol = int(math.ceil(num_imgs / nrow))\n", " imgs = torchvision.utils.make_grid(imgs, nrow=nrow, pad_value=128)\n", " imgs = imgs.clamp(min=0, max=255)\n", " np_imgs = imgs.cpu().numpy()\n", " plt.figure(figsize=(1.5 * nrow, 1.5 * ncol))\n", " plt.imshow(np.transpose(np_imgs, (1, 2, 0)), interpolation=\"nearest\")\n", " plt.axis(\"off\")\n", " plt.show()\n", " plt.close()\n", "\n", "\n", "show_imgs([train_set[i][0] for i in range(8)])"]}, {"cell_type": "markdown", "id": "ddea9aaf", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.008551, "end_time": "2022-05-03T01:16:13.995344", "exception": false, "start_time": "2022-05-03T01:16:13.986793", "status": "completed"}, "tags": []}, "source": ["## Masked autoregressive convolutions\n", "\n", "The core module of PixelCNN is its masked convolutions.\n", "In contrast to language models, we don't apply an LSTM on each pixel one-by-one.\n", "This would be inefficient because images are grids instead of sequences.\n", "Thus, it is better to rely on convolutions that have shown great success in deep CNN classification models.\n", "\n", "Nevertheless, we cannot just apply standard convolutions without any changes.\n", "Remember that during training of autoregressive models, we want to use teacher forcing which both helps the model training, and significantly reduces the time needed for training.\n", "For image modeling, teacher forcing is implemented by using a training image as input to the model, and we want to obtain as output the prediction for each pixel based on *only* its predecessors.\n", "Thus, we need to ensure that the prediction for a specific pixel can only be influenced by its predecessors and not by its own value or any \"future\" pixels.\n", "For this, we apply convolutions with a mask.\n", "\n", "Which mask we use depends on the ordering of pixels we decide on, i.e. which is the first pixel we predict,\n", "which is the second one, etc.\n", "The most commonly used ordering is to denote the upper left pixel as the start pixel,\n", "and sort the pixels row by row, as shown in the visualization at the top of the tutorial.\n", "Thus, the second pixel is on the right of the first one (first row, second column),\n", "and once we reach the end of the row, we start in the second row, first column.\n", "If we now want to apply this to our convolutions, we need to ensure that the prediction of pixel 1\n", "is not influenced by its own \"true\" input, and all pixels on its right and in any lower row.\n", "In convolutions, this means that we want to set those entries of the weight matrix to zero that take pixels on the right and below into account.\n", "As an example for a 5x5 kernel, see a mask below (figure credit - [Aaron van den Oord](https://arxiv.org/pdf/1606.05328.pdf)):\n", "\n", "
\n", "\n", "Before looking into the application of masked convolutions in PixelCNN\n", "in detail, let's first implement a module that allows us to apply an\n", "arbitrary mask to a convolution:"]}, {"cell_type": "code", "execution_count": 6, "id": "2b78e845", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:16:14.013826Z", "iopub.status.busy": "2022-05-03T01:16:14.013542Z", "iopub.status.idle": "2022-05-03T01:16:14.019594Z", "shell.execute_reply": "2022-05-03T01:16:14.019019Z"}, "lines_to_next_cell": 2, "papermill": {"duration": 0.017078, "end_time": "2022-05-03T01:16:14.021072", "exception": false, "start_time": "2022-05-03T01:16:14.003994", "status": "completed"}, "tags": []}, "outputs": [], "source": ["class MaskedConvolution(nn.Module):\n", " def __init__(self, c_in, c_out, mask, **kwargs):\n", " \"\"\"Implements a convolution with mask applied on its weights.\n", "\n", " Args:\n", " c_in: Number of input channels\n", " c_out: Number of output channels\n", " mask: Tensor of shape [kernel_size_H, kernel_size_W] with 0s where\n", " the convolution should be masked, and 1s otherwise.\n", " kwargs: Additional arguments for the convolution\n", " \"\"\"\n", " super().__init__()\n", " # For simplicity: calculate padding automatically\n", " kernel_size = (mask.shape[0], mask.shape[1])\n", " dilation = 1 if \"dilation\" not in kwargs else kwargs[\"dilation\"]\n", " padding = tuple(dilation * (kernel_size[i] - 1) // 2 for i in range(2))\n", " # Actual convolution\n", " self.conv = nn.Conv2d(c_in, c_out, kernel_size, padding=padding, **kwargs)\n", "\n", " # Mask as buffer => it is no parameter but still a tensor of the module\n", " # (must be moved with the devices)\n", " self.register_buffer(\"mask\", mask[None, None])\n", "\n", " def forward(self, x):\n", " self.conv.weight.data *= self.mask # Ensures zero's at masked positions\n", " return self.conv(x)"]}, {"cell_type": "markdown", "id": "4ccdf7b4", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.008485, "end_time": "2022-05-03T01:16:14.038123", "exception": false, "start_time": "2022-05-03T01:16:14.029638", "status": "completed"}, "tags": []}, "source": ["### Vertical and horizontal convolution stacks\n", "\n", "To build our own autoregressive image model, we could simply stack a few masked convolutions on top of each other.\n", "This was actually the case for the original PixelCNN model, discussed in the paper\n", "[Pixel Recurrent Neural Networks](https://arxiv.org/pdf/1601.06759.pdf), but this leads to a considerable issue.\n", "When sequentially applying a couple of masked convolutions, the receptive field of a pixel\n", "show to have a \"blind spot\" on the right upper side, as shown in the figure below\n", "(figure credit - [Aaron van den Oord et al. ](https://arxiv.org/pdf/1606.05328.pdf)):\n", "\n", "
\n", "\n", "Although a pixel should be able to take into account all other pixels above and left of it,\n", "a stack of masked convolutions does not allow us to look to the upper pixels on the right.\n", "This is because the features of the pixels above, which we use for convolution,\n", "do not contain any information of the pixels on the right of the same row.\n", "If they would, we would be \"cheating\" and actually looking into the future.\n", "To overcome this issue, van den Oord et.\n", "al [2] proposed to split the convolutions into a vertical and a horizontal stack.\n", "The vertical stack looks at all pixels above the current one, while the horizontal takes into account all on the left.\n", "While keeping both of them separate, we can actually look at the pixels on the right with the vertical stack without breaking any of our assumptions.\n", "The two convolutions are also shown in the figure above.\n", "\n", "Let us implement them here as follows:"]}, {"cell_type": "code", "execution_count": 7, "id": "40bf986c", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:16:14.056503Z", "iopub.status.busy": "2022-05-03T01:16:14.055969Z", "iopub.status.idle": "2022-05-03T01:16:14.061570Z", "shell.execute_reply": "2022-05-03T01:16:14.060983Z"}, "papermill": {"duration": 0.016361, "end_time": "2022-05-03T01:16:14.063091", "exception": false, "start_time": "2022-05-03T01:16:14.046730", "status": "completed"}, "tags": []}, "outputs": [], "source": ["class VerticalStackConvolution(MaskedConvolution):\n", " def __init__(self, c_in, c_out, kernel_size=3, mask_center=False, **kwargs):\n", " # Mask out all pixels below. For efficiency, we could also reduce the kernel\n", " # size in height, but for simplicity, we stick with masking here.\n", " mask = torch.ones(kernel_size, kernel_size)\n", " mask[kernel_size // 2 + 1 :, :] = 0\n", "\n", " # For the very first convolution, we will also mask the center row\n", " if mask_center:\n", " mask[kernel_size // 2, :] = 0\n", "\n", " super().__init__(c_in, c_out, mask, **kwargs)\n", "\n", "\n", "class HorizontalStackConvolution(MaskedConvolution):\n", " def __init__(self, c_in, c_out, kernel_size=3, mask_center=False, **kwargs):\n", " # Mask out all pixels on the left. Note that our kernel has a size of 1\n", " # in height because we only look at the pixel in the same row.\n", " mask = torch.ones(1, kernel_size)\n", " mask[0, kernel_size // 2 + 1 :] = 0\n", "\n", " # For the very first convolution, we will also mask the center pixel\n", " if mask_center:\n", " mask[0, kernel_size // 2] = 0\n", "\n", " super().__init__(c_in, c_out, mask, **kwargs)"]}, {"cell_type": "markdown", "id": "dabc3cac", "metadata": {"papermill": {"duration": 0.008514, "end_time": "2022-05-03T01:16:14.080096", "exception": false, "start_time": "2022-05-03T01:16:14.071582", "status": "completed"}, "tags": []}, "source": ["Note that we have an input argument called `mask_center`. Remember that\n", "the input to the model is the actual input image. Hence, the very first\n", "convolution we apply cannot use the center pixel as input, but must be\n", "masked. All consecutive convolutions, however, should use the center\n", "pixel as we otherwise lose the features of the previous layer. Hence,\n", "the input argument `mask_center` is True for the very first\n", "convolutions, and False for all others."]}, {"cell_type": "markdown", "id": "a96c9d99", "metadata": {"papermill": {"duration": 0.008513, "end_time": "2022-05-03T01:16:14.097369", "exception": false, "start_time": "2022-05-03T01:16:14.088856", "status": "completed"}, "tags": []}, "source": ["### Visualizing the receptive field\n", "\n", "To validate our implementation of masked convolutions, we can visualize the receptive field we obtain with such convolutions.\n", "We should see that with increasing number of convolutional layers, the receptive field grows in both vertical and horizontal direction, without the issue of a blind spot.\n", "The receptive field can be empirically measured by backpropagating an arbitrary loss for the output features of a speicifc pixel with respect to the input.\n", "We implement this idea below, and visualize the receptive field below."]}, {"cell_type": "code", "execution_count": 8, "id": "fc26555f", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:16:14.115816Z", "iopub.status.busy": "2022-05-03T01:16:14.115233Z", "iopub.status.idle": "2022-05-03T01:16:14.268019Z", "shell.execute_reply": "2022-05-03T01:16:14.267423Z"}, "papermill": {"duration": 0.163539, "end_time": "2022-05-03T01:16:14.269523", "exception": false, "start_time": "2022-05-03T01:16:14.105984", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUiAvTWVkaWFCb3ggWyAwIDAgMzQ5LjIgMTgxLjcwNjgxODE4MTggXQovUGFyZW50IDIgMCBSIC9SZXNvdXJjZXMgOCAwIFIgL1R5cGUgL1BhZ2UgPj4KZW5kb2JqCjkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMiAwIFIgPj4Kc3RyZWFtCnicnVFNUwIxDL3nV7wjHOwmLWW7RxBl9IbuDAfGg7MUWAZQPgbGf28WQRQ9iO1kmqQvr8lr0onbsogP3TauHyk5RcWaBFNKWoLxGoyp2g6CLr6DWPNzco3MWPVmB0+CmJSbQYKm+DyscBOiBS2RKrgy8dYo6rDPw1VEHwskLVs1o32p7ZSmqwziHSrK6jyyFXMkd4LOC3rUw/JYyBh/La5iWuqYjCuth7Bh5zhr+owzSNMbd2qhmFM7R3KrMIt8tJ8iH9IAtX4dPhifudSjFsvxZBOHWNVh2QR3XHpTxNdNuY0YlXE2rOMJ+T3d5PTRoYTMfD72b0nopyS/EO/VsZeqY1n/0NvQYFVH/i5Pu1w8r94uE4TeARASlAYKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyODMKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNjQgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggOTIgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzMDcgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIzMiA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjMxID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDI0OSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMzYgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDM0MSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE2NCA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDcyID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDQ3ID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTYzID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMzkgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNjAgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE4ID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEzMyA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNzUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTQxID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvQmFzZUZvbnQgL0JNUVFEVitEZWphVnVTYW5zIC9DaGFyUHJvY3MgMTggMCBSCi9FbmNvZGluZyA8PAovRGlmZmVyZW5jZXMgWyAzMiAvc3BhY2UgNjYgL0IgODcgL1cgOTcgL2EgOTkgL2MgL2QgL2UgL2YgL2cgL2ggL2kgMTA4IC9sIDExMCAvbiAxMTIgL3AKMTE0IC9yIDExNiAvdCAxMTggL3YgMTIxIC95IF0KL1R5cGUgL0VuY29kaW5nID4+Ci9GaXJzdENoYXIgMCAvRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Gb250RGVzY3JpcHRvciAxNiAwIFIKL0ZvbnRNYXRyaXggWyAwLjAwMSAwIDAgMC4wMDEgMCAwIF0gL0xhc3RDaGFyIDI1NSAvTmFtZSAvQk1RUURWK0RlamFWdVNhbnMKL1N1YnR5cGUgL1R5cGUzIC9UeXBlIC9Gb250IC9XaWR0aHMgMTUgMCBSID4+CmVuZG9iagoxNiAwIG9iago8PCAvQXNjZW50IDkyOSAvQ2FwSGVpZ2h0IDAgL0Rlc2NlbnQgLTIzNiAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvRm9udE5hbWUgL0JNUVFEVitEZWphVnVTYW5zCi9JdGFsaWNBbmdsZSAwIC9NYXhXaWR0aCAxMzQyIC9TdGVtViAwIC9UeXBlIC9Gb250RGVzY3JpcHRvciAvWEhlaWdodCAwID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL0NBIDAgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PgovQTIgPDwgL0NBIDEgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9CaXRzUGVyQ29tcG9uZW50IDggL0NvbG9yU3BhY2UgWy9JbmRleGVkIC9EZXZpY2VSR0IgMSAo/ec3ACJNKV0KL0RlY29kZVBhcm1zIDw8IC9Db2xvcnMgMSAvQ29sdW1ucyAxNTMgL1ByZWRpY3RvciAxMCA+PgovRmlsdGVyIC9GbGF0ZURlY29kZSAvSGVpZ2h0IDE1MyAvTGVuZ3RoIDM3IDAgUiAvU3VidHlwZSAvSW1hZ2UKL1R5cGUgL1hPYmplY3QgL1dpZHRoIDE1MyA+PgpzdHJlYW0KeJztzrEJwEAMwEBn/6XTpkkheHgXpwHEzbO1uQ34jaxH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1jskm2+Hloc2ZH1D1jdkfUPWN2R9Q7Yhsh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zb6/sBQbPWskKZW5kc3RyZWFtCmVuZG9iagozNyAwIG9iagoxODgKZW5kb2JqCjE0IDAgb2JqCjw8IC9CaXRzUGVyQ29tcG9uZW50IDggL0NvbG9yU3BhY2UgWy9JbmRleGVkIC9EZXZpY2VSR0IgMSAo/ec3ACJNKV0KL0RlY29kZVBhcm1zIDw8IC9Db2xvcnMgMSAvQ29sdW1ucyAxNTMgL1ByZWRpY3RvciAxMCA+PgovRmlsdGVyIC9GbGF0ZURlY29kZSAvSGVpZ2h0IDE1MyAvTGVuZ3RoIDM4IDAgUiAvU3VidHlwZSAvSW1hZ2UKL1R5cGUgL1hPYmplY3QgL1dpZHRoIDE1MyA+PgpzdHJlYW0KeJztzrEJwEAMwEBn/6XTpkkheHgXpwHEzbO1uQ34jaxH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1jskm2+Hloc2ZH1D1jdkfUPWN2R9Q7Yhsh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zb6/sBQbPWskKZW5kc3RyZWFtCmVuZG9iagozOCAwIG9iagoxODgKZW5kb2JqCjIgMCBvYmoKPDwgL0NvdW50IDEgL0tpZHMgWyAxMSAwIFIgXSAvVHlwZSAvUGFnZXMgPj4KZW5kb2JqCjM5IDAgb2JqCjw8IC9DcmVhdGlvbkRhdGUgKEQ6MjAyMjA1MDMwMzE2MTQrMDInMDAnKQovQ3JlYXRvciAoTWF0cGxvdGxpYiB2My41LjEsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcpCi9Qcm9kdWNlciAoTWF0cGxvdGxpYiBwZGYgYmFja2VuZCB2My41LjEpID4+CmVuZG9iagp4cmVmCjAgNDAKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDgxMjIgMDAwMDAgbiAKMDAwMDAwNjk4OCAwMDAwMCBuIAowMDAwMDA3MDIwIDAwMDAwIG4gCjAwMDAwMDcxMTkgMDAwMDAgbiAKMDAwMDAwNzE0MCAwMDAwMCBuIAowMDAwMDA3MTYxIDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM0MyAwMDAwMCBuIAowMDAwMDAwNzIxIDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDcwMSAwMDAwMCBuIAowMDAwMDA3MjA0IDAwMDAwIG4gCjAwMDAwMDc2NjMgMDAwMDAgbiAKMDAwMDAwNTcyOSAwMDAwMCBuIAowMDAwMDA1NTIyIDAwMDAwIG4gCjAwMDAwMDUxMTQgMDAwMDAgbiAKMDAwMDAwNjc4MiAwMDAwMCBuIAowMDAwMDAwNzQxIDAwMDAwIG4gCjAwMDAwMDEwNzggMDAwMDAgbiAKMDAwMDAwMTI0MiAwMDAwMCBuIAowMDAwMDAxNjIyIDAwMDAwIG4gCjAwMDAwMDE5MjcgMDAwMDAgbiAKMDAwMDAwMjIzMSAwMDAwMCBuIAowMDAwMDAyNTUzIDAwMDAwIG4gCjAwMDAwMDI3NjIgMDAwMDAgbiAKMDAwMDAwMzE3NiAwMDAwMCBuIAowMDAwMDAzNDEzIDAwMDAwIG4gCjAwMDAwMDM1NTcgMDAwMDAgbiAKMDAwMDAwMzY3NiAwMDAwMCBuIAowMDAwMDAzOTEyIDAwMDAwIG4gCjAwMDAwMDQyMjQgMDAwMDAgbiAKMDAwMDAwNDQ1NyAwMDAwMCBuIAowMDAwMDA0NTQ3IDAwMDAwIG4gCjAwMDAwMDQ3NTMgMDAwMDAgbiAKMDAwMDAwNDkwMCAwMDAwMCBuIAowMDAwMDA3NjQzIDAwMDAwIG4gCjAwMDAwMDgxMDIgMDAwMDAgbiAKMDAwMDAwODE4MiAwMDAwMCBuIAp0cmFpbGVyCjw8IC9JbmZvIDM5IDAgUiAvUm9vdCAxIDAgUiAvU2l6ZSA0MCA+PgpzdGFydHhyZWYKODMzOQolJUVPRgo=\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2022-05-03T03:16:14.190200\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.5.1, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}], "source": ["inp_img = torch.zeros(1, 1, 11, 11)\n", "inp_img.requires_grad_()\n", "\n", "\n", "def show_center_recep_field(img, out):\n", " \"\"\"Calculates the gradients of the input with respect to the output center pixel, and visualizes the overall\n", " receptive field.\n", "\n", " Args:\n", " img: Input image for which we want to calculate the receptive field on.\n", " out: Output features/loss which is used for backpropagation, and should be\n", " the output of the network/computation graph.\n", " \"\"\"\n", " # Determine gradients\n", " loss = out[0, :, img.shape[2] // 2, img.shape[3] // 2].sum() # L1 loss for simplicity\n", " # Retain graph as we want to stack multiple layers and show the receptive field of all of them\n", " loss.backward(retain_graph=True)\n", " img_grads = img.grad.abs()\n", " img.grad.fill_(0) # Reset grads\n", "\n", " # Plot receptive field\n", " img = img_grads.squeeze().cpu().numpy()\n", " fig, ax = plt.subplots(1, 2)\n", " _ = ax[0].imshow(img)\n", " ax[1].imshow(img > 0)\n", " # Mark the center pixel in red if it doesn't have any gradients (should be\n", " # the case for standard autoregressive models)\n", " show_center = img[img.shape[0] // 2, img.shape[1] // 2] == 0\n", " if show_center:\n", " center_pixel = np.zeros(img.shape + (4,))\n", " center_pixel[center_pixel.shape[0] // 2, center_pixel.shape[1] // 2, :] = np.array([1.0, 0.0, 0.0, 1.0])\n", " for i in range(2):\n", " ax[i].axis(\"off\")\n", " if show_center:\n", " ax[i].imshow(center_pixel)\n", " ax[0].set_title(\"Weighted receptive field\")\n", " ax[1].set_title(\"Binary receptive field\")\n", " plt.show()\n", " plt.close()\n", "\n", "\n", "show_center_recep_field(inp_img, inp_img)"]}, {"cell_type": "markdown", "id": "54449b79", "metadata": {"papermill": {"duration": 0.009223, "end_time": "2022-05-03T01:16:14.288391", "exception": false, "start_time": "2022-05-03T01:16:14.279168", "status": "completed"}, "tags": []}, "source": ["Let's first visualize the receptive field of a horizontal convolution\n", "without the center pixel. We use a small, arbitrary input image\n", "($11\\times 11$ pixels), and calculate the loss for the center pixel. For\n", "simplicity, we initialize all weights with 1 and the bias with 0, and\n", "use a single channel. This is sufficient for our visualization purposes."]}, {"cell_type": "code", "execution_count": 9, "id": "36685715", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:16:14.308211Z", "iopub.status.busy": "2022-05-03T01:16:14.307942Z", "iopub.status.idle": "2022-05-03T01:16:14.466871Z", "shell.execute_reply": "2022-05-03T01:16:14.464698Z"}, "papermill": {"duration": 0.171722, "end_time": "2022-05-03T01:16:14.469285", "exception": false, "start_time": "2022-05-03T01:16:14.297563", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUiAvTWVkaWFCb3ggWyAwIDAgMzQ5LjIgMTgxLjcwNjgxODE4MTggXQovUGFyZW50IDIgMCBSIC9SZXNvdXJjZXMgOCAwIFIgL1R5cGUgL1BhZ2UgPj4KZW5kb2JqCjkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMiAwIFIgPj4Kc3RyZWFtCnicnVFLTwIxEL7Pr/iOcLA701K2PYIo0Ru6CQfjwSwFlgDKIxD/vbMKPvAidjLpzPSb19esl3ZVme76XVzeU/bllRsSzCjrCCYbMGaqewj6+AlijS/ItaKxas0PlgQxObeDBA3xqVvjpkRLWiFXcK3irVHUQU7ddcIQS2QdWw+jc6nutUxfK4h3qEvWd65SLpDdCHrPGNAAq2MSY/I9sfZppSsyLjQXwoad49j2kSOk7Y37al8uqFsgu1aYRTF+36AY0QMawyZ8MD663KORqsl0m0ZYN2HZBHc8+lKml221SxhXaT5q4hHFLV0V9DGhhGg+m/2bDvpNh0Q+EmLPJcSyfpm3ocVKiPydkW61fFq/nscBvQHN+pCiCmVuZHN0cmVhbQplbmRvYmoKMTIgMCBvYmoKMjgxCmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjE5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjY0ID4+CnN0cmVhbQp4nD1SuZEDMQzLtwqWwF9SPb65cWD3nx7APTsxMUsRAEH3blE5Gz9tJcuP/NhlZ0nklvegbfK6ouoGqZsgc8lxyRVilvK4EiQWLWUm1meqg/JxDWoVvvATMxPTIUnEEnJSjyKsj6+D1/Uc3F836fwaGR90bFB6S5KroFTDvnK0EjuW6l2D7IOmm9LqM9Ou7ICl8Yas1KROfx31vyMswqTeSEUHIRa/08sdsiykHNTbuX4D4V3tglhPrTzsEGneVnpLFczmzARMGW07rIC1WWPCVAnwFs0h1FqIzBc6rYj55NzRwTRVeRoiw1KFg+09ozVc6BoSS8P5ScYE5r/w+qzIbX//AHBoZ7oKZW5kc3RyZWFtCmVuZG9iagoyMCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDkyID4+CnN0cmVhbQp4nD2NwQ3AMAgD/0zBCBACxPtUVR/p/t8mEeoHHwbZGGBhszXgwdnAl9LaN72kRZPaCFa1Rd1QnrsUpVhdR6VMwk+ZO39SdBztcA7b39blOE3j6F/30P0BD0oeCwplbmRzdHJlYW0KZW5kb2JqCjIxIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzA3ID4+CnN0cmVhbQp4nD2SS24DMQxD9z6FLhDA+tme86Qoupjef9snJemKHNkWRWqWukxZUx6QNJOEf+nwcLGd8jtsz2Zm4Fqil4nllOfQFWLuonzZzEZdWSfF6oRmOrfoUTkXBzZNqp+rLKXdLngO1yaeW/YRP7zQoB7UNS4JN3RXo2UpNGOq+3/Se/yMMuBqTF1sUqt7HzxeRFXo6AdHiSJjlxfn40EJ6UrCaFqIlXdFA0Hu8rTKewnu295qyLIHqZjOOylmsOt0Ui5uF4chHsjyqPDlo9hrQs/4sCsl9EjYhjNyJ+5oxubUyOKQ/t6NBEuPrmgh8+CvbtYuYLxTOkViZE5yrGmLVU73UBTTucO9DBD1bEVDKXOR1epfw84La5ZsFnhK+gUeo90mSw5W2duoTu+tPNnQ9x9a13QfCmVuZHN0cmVhbQplbmRvYmoKMjIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMzIgPj4Kc3RyZWFtCnicNVFJbsQwDLv7FfzAANbuvCfFoIf2/9dSyhQIQCW2uCViYyMCLzH4OYjc+JI1oyZ+Z3JX/CxPhUfCreBJFIGX4V52gssbxmU/DjMfvJdWzqTGkwzIRTY9PBEy2CUQOjC7BnXYZtqJviHhsyNSzUaW09cS9NIqBMpTtt/pghJtq/pz+6wLbfvaE052e+pJ5ROI55aswGXjFZPFWAY9UblLMX2Q6myhJ6G8KJ+DbD5qiESXKGfgicHBKNAO7LntZ+JVIWhd3adtY6hGSsfTvw1NTZII+UQJZ7Y07hb+f8+9vtf7D04hVBEKZW5kc3RyZWFtCmVuZG9iagoyMyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIzMSA+PgpzdHJlYW0KeJw1TzmSBCEMy3mFPjBVGNtAv6entjbY+X+6kplOkPAhydMTHZl4mSMjsGbH21pkIGbgU0zFv/a0DxOq9+AeIpSLC2GGkXDWrONuno4X/3aVz1gH7zb4illeENjCTNZXFmcu2wVjaZzEOclujF0TsY11radTWEcwoQyEdLbDlCBzVKT0yY4y5ug4kSeei+/22yx2OX4O6ws2jSEV5/gqeoI2g6Lsee8CGnJB/13d+B5Fu+glIBsJFtZRYu6c5YRfvXZ0HrUoEnNCmkEuEyHN6SqmEJpQrLOjoFJRcKk+p+isn3/lX1wtCmVuZHN0cmVhbQplbmRvYmoKMjQgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNDkgPj4Kc3RyZWFtCnicPVA7jkQhDOs5hS/wJPIjcB5Gqy1m79+uA5opUEx+tjMk0BGBRwwxlK/jJa2groG/i0LxbuLrg8Igq0NSIM56D4h07KY2kRM6HZwzP2E3Y47ARTEGnOl0pj0HJjn7wgqEcxtl7FZIJ4mqIo7qM44pnip7n3gWLO3INlsnkj3kIOFSUonJpZ+Uyj9typQKOmbRBCwSueBkE004y7tJUowZlDLqHqZ2In2sPMijOuhkTc6sI5nZ00/bmfgccLdf2mROlcd0Hsz4nLTOgzkVuvfjiTYHTY3a6Oz3E2kqL1K7HVqdfnUSld0Y5xgSl2d/Gd9k//kH/odaIgplbmRzdHJlYW0KZW5kb2JqCjI1IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTM2ID4+CnN0cmVhbQp4nE2PQQ4DMQgD73mFn0AgQHjPVlUP2/9fS9h20wseyYBsUQaBJYd4hxvh0dsP30U2FWfjnF9SKWIhmE9wnzBTHI0pd/Jjj4BxlGosp2h4XkvOTcMXLXcTLaWtl5MZb7jul/dHlW2RDUXPLQtC12yS+TKBB3wYmEd142mlx932bK/2/ADObDRJCmVuZHN0cmVhbQplbmRvYmoKMjYgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzNDEgPj4Kc3RyZWFtCnicRVJLbkQxCNu/U3CBSOGXkPO0qrqY3n9bm0zVzeAJYGx4y1OmZMqwuSUjJNeUT30iQ6ym/DRyJCKm+EkJBXaVj8drS6yN7JGoFJ/a8eOx9Eam2RVa9e7Rpc2iUc3KyDnIEKGeFbqye9QO2fB6XEi675TNIRzL/1CBLGXdcgolQVvQd+wR3w8droIrgmGway6D7WUy1P/6hxZc7333YscugBas577BDgCopxO0BcgZ2u42KWgAVbqLScKj8npudqJso1Xp+RwAMw4wcsCIJVsdvtHeAJZ9XehFjYr9K0BRWUD8yNV2wd4xyUhwFuYGjr1wPMWZcEs4xgJAir3iGHrwJdjmL1euiJrwCXW6ZC+8wp7a5udCkwh3rQAOXmTDraujqJbt6TyC9mdFckaM1Is4OiGSWtI5guLSoB5a41w3seJtI7G5V9/uH+GcL1z26xdL7ITECmVuZHN0cmVhbQplbmRvYmoKMjcgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNjQgPj4Kc3RyZWFtCnicRZDHcQUxDEPvqgIlMIAK9azH8w/r/q+G9NNBehhCDGJPwrBcV3FhdMOPty0zDX9HGe7G+jJjvNVYICfoAwyRiavRpPp2xRmq9OTVYq6jolwvOiISzJLjq0AjfDqyx5O2tjP9dF4f7CHvE/8qKuduYQEuqu5A+VIf8dSP2VHqmqGPKitrHmraV4RdEUrbPi6nMk7dvQNa4b2Vqz3a7z8edjryCmVuZHN0cmVhbQplbmRvYmoKMjggMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA3MiA+PgpzdHJlYW0KeJwzMrdQMFCwNAEShhYmCuZmBgophlxAvqmJuUIuF0gMxMoBswyAtCWcgohngJggbRDFIBZEsZmJGUQdnAGRy+BKAwAl2xbJCmVuZHN0cmVhbQplbmRvYmoKMjkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA0NyA+PgpzdHJlYW0KeJwzMrdQMFCwNAEShhYmCuZmBgophlyWEFYuF0wsB8wC0ZZwCiKewZUGALlnDScKZW5kc3RyZWFtCmVuZG9iagozMCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE2MyA+PgpzdHJlYW0KeJxFkDsSAyEMQ3tOoSP4IwM+z2YyKTb3b2PYbFLA01ggg7sTgtTagonogoe2Jd0F760EZ2P86TZuNRLkBHWAVqTjaJRSfbnFaZV08Wg2cysLrRMdZg56lKMZoBA6Fd7touRypu7O+UNw9V/1v2LdOZuJgcnKHQjN6lPc+TY7orq6yf6kx9ys134r7FVhaVlLywm3nbtmQAncUznaqz0/Hwo69gplbmRzdHJlYW0KZW5kb2JqCjMxIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjM5ID4+CnN0cmVhbQp4nE1QyW0EMQz7uwo1MMDoHLseB4s8sv1/Q8oJkpdoS+Kh8pRblspl9yM5b8m65UOHTpVp8m7Qza+x/qMMAnb/UFQQrSWxSsxc0m6xNEkv2cM4jZdrtY7nqXuEWaN48OPY0ymB6T0ywWazvTkwqz3ODpBOuMav6tM7lSQDibqQ80KlCuse1CWijyvbmFKdTi3lGJef6Ht8jgA9xd6N3NHHyxeMRrUtqNFqlTgPMBNT0ZVxq5GBlBMGQ2dHVzQLpcjKekI1wo05oZm9w3BgA8uzhKSlrVK8D2UB6AJd2jrjNEqCjgDC3yiM9foGqvxeNwplbmRzdHJlYW0KZW5kb2JqCjMyIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTYwID4+CnN0cmVhbQp4nEWQORIDMQgEc72CJ0hcgvesy7XB+v+pB9ZHoukCNBy6Fk3KehRoPumxRqG60GvoLEqSRMEWkh1Qp2OIOyhITEhjkki2HoMjmlizXZiZVCqzUuG0acXCv9la1chEjXCN/InpBlT8T+pclPBNg6+SMfoYVLw7g4xJ+F5F3Fox7f5EMLEZ9glvRSYFhImxqdm+z2CGzPcK1zjH8w1MgjfrCmVuZHN0cmVhbQplbmRvYmoKMzMgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxOCA+PgpzdHJlYW0KeJwzNrRQMIDDFEOuNAAd5gNSCmVuZHN0cmVhbQplbmRvYmoKMzQgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMzMgPj4Kc3RyZWFtCnicRY9LDgQhCET3nKKOwMcf53Ey6YVz/+2AnW4TYz2FVIG5gqE9LmsDnRUfIRm28beplo5FWT5UelJWD8ngh6zGyyHcoCzwgkkqhiFQi5gakS1lbreA2zYNsrKVU6WOsIujMI/2tGwVHl+iWyJ1kj+DxCov3OO6Hcil1rveoou+f6QBMQkKZW5kc3RyZWFtCmVuZG9iagozNSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDc1ID4+CnN0cmVhbQp4nDO1NFIwUDA2ABKmZkYKpibmCimGXEA+iJXLZWhkCmblcBlZmilYWAAZJmbmUCGYhhwuY1NzoAFARcamYBqqP4crgysNAJWQEu8KZW5kc3RyZWFtCmVuZG9iagozNiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE0MSA+PgpzdHJlYW0KeJw9j8EOwzAIQ+/5Cv9ApNgpoXxPp2qH7v+vI0u7C3oCY4yF0NAbqprDhmCb48XSJVRr+BTFQCU3yJlgDqWk0h1HkXpiOBhcHrQbjuKx6PoRu5JmfdDGQrolaIB7rFNp3KZxE8QdNQXqKeqco7wQuZ+pZ9g0kt00s5JzuA2/e89T1/+nq7zL+QW9dy7+CmVuZHN0cmVhbQplbmRvYmoKMTcgMCBvYmoKPDwgL0Jhc2VGb250IC9CTVFRRFYrRGVqYVZ1U2FucyAvQ2hhclByb2NzIDE4IDAgUgovRW5jb2RpbmcgPDwKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCi9UeXBlIC9FbmNvZGluZyA+PgovRmlyc3RDaGFyIDAgL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvRm9udERlc2NyaXB0b3IgMTYgMCBSCi9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdIC9MYXN0Q2hhciAyNTUgL05hbWUgL0JNUVFEVitEZWphVnVTYW5zCi9TdWJ0eXBlIC9UeXBlMyAvVHlwZSAvRm9udCAvV2lkdGhzIDE1IDAgUiA+PgplbmRvYmoKMTYgMCBvYmoKPDwgL0FzY2VudCA5MjkgL0NhcEhlaWdodCAwIC9EZXNjZW50IC0yMzYgL0ZsYWdzIDMyCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovSXRhbGljQW5nbGUgMCAvTWF4V2lkdGggMTM0MiAvU3RlbVYgMCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL1hIZWlnaHQgMCA+PgplbmRvYmoKMTUgMCBvYmoKWyA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMAo2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDMxOCA0MDEgNDYwIDgzOCA2MzYKOTUwIDc4MCAyNzUgMzkwIDM5MCA1MDAgODM4IDMxOCAzNjEgMzE4IDMzNyA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2CjYzNiA2MzYgMzM3IDMzNyA4MzggODM4IDgzOCA1MzEgMTAwMCA2ODQgNjg2IDY5OCA3NzAgNjMyIDU3NSA3NzUgNzUyIDI5NQoyOTUgNjU2IDU1NyA4NjMgNzQ4IDc4NyA2MDMgNzg3IDY5NSA2MzUgNjExIDczMiA2ODQgOTg5IDY4NSA2MTEgNjg1IDM5MCAzMzcKMzkwIDgzOCA1MDAgNTAwIDYxMyA2MzUgNTUwIDYzNSA2MTUgMzUyIDYzNSA2MzQgMjc4IDI3OCA1NzkgMjc4IDk3NCA2MzQgNjEyCjYzNSA2MzUgNDExIDUyMSAzOTIgNjM0IDU5MiA4MTggNTkyIDU5MiA1MjUgNjM2IDMzNyA2MzYgODM4IDYwMCA2MzYgNjAwIDMxOAozNTIgNTE4IDEwMDAgNTAwIDUwMCA1MDAgMTM0MiA2MzUgNDAwIDEwNzAgNjAwIDY4NSA2MDAgNjAwIDMxOCAzMTggNTE4IDUxOAo1OTAgNTAwIDEwMDAgNTAwIDEwMDAgNTIxIDQwMCAxMDIzIDYwMCA1MjUgNjExIDMxOCA0MDEgNjM2IDYzNiA2MzYgNjM2IDMzNwo1MDAgNTAwIDEwMDAgNDcxIDYxMiA4MzggMzYxIDEwMDAgNTAwIDUwMCA4MzggNDAxIDQwMSA1MDAgNjM2IDYzNiAzMTggNTAwCjQwMSA0NzEgNjEyIDk2OSA5NjkgOTY5IDUzMSA2ODQgNjg0IDY4NCA2ODQgNjg0IDY4NCA5NzQgNjk4IDYzMiA2MzIgNjMyIDYzMgoyOTUgMjk1IDI5NSAyOTUgNzc1IDc0OCA3ODcgNzg3IDc4NyA3ODcgNzg3IDgzOCA3ODcgNzMyIDczMiA3MzIgNzMyIDYxMSA2MDUKNjMwIDYxMyA2MTMgNjEzIDYxMyA2MTMgNjEzIDk4MiA1NTAgNjE1IDYxNSA2MTUgNjE1IDI3OCAyNzggMjc4IDI3OCA2MTIgNjM0CjYxMiA2MTIgNjEyIDYxMiA2MTIgODM4IDYxMiA2MzQgNjM0IDYzNCA2MzQgNTkyIDYzNSA1OTIgXQplbmRvYmoKMTggMCBvYmoKPDwgL0IgMTkgMCBSIC9XIDIwIDAgUiAvYSAyMSAwIFIgL2MgMjIgMCBSIC9kIDIzIDAgUiAvZSAyNCAwIFIgL2YgMjUgMCBSCi9nIDI2IDAgUiAvaCAyNyAwIFIgL2kgMjggMCBSIC9sIDI5IDAgUiAvbiAzMCAwIFIgL3AgMzEgMCBSIC9yIDMyIDAgUgovc3BhY2UgMzMgMCBSIC90IDM0IDAgUiAvdiAzNSAwIFIgL3kgMzYgMCBSID4+CmVuZG9iagozIDAgb2JqCjw8IC9GMSAxNyAwIFIgPj4KZW5kb2JqCjQgMCBvYmoKPDwgL0ExIDw8IC9DQSAwIC9UeXBlIC9FeHRHU3RhdGUgL2NhIDEgPj4KL0EyIDw8IC9DQSAxIC9UeXBlIC9FeHRHU3RhdGUgL2NhIDEgPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgL0kxIDEzIDAgUiAvSTIgMTQgMCBSID4+CmVuZG9iagoxMyAwIG9iago8PCAvQml0c1BlckNvbXBvbmVudCA4IC9Db2xvclNwYWNlIFsvSW5kZXhlZCAvRGV2aWNlUkdCIDIgKP3nNwAiTf8AACldCi9EZWNvZGVQYXJtcyA8PCAvQ29sb3JzIDEgL0NvbHVtbnMgMTUzIC9QcmVkaWN0b3IgMTAgPj4KL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0hlaWdodCAxNTMgL0xlbmd0aCAzNyAwIFIgL1N1YnR5cGUgL0ltYWdlCi9UeXBlIC9YT2JqZWN0IC9XaWR0aCAxNTMgPj4Kc3RyZWFtCnic7c6xCYBQEECx0/2HttXC4gniF5IJMtuq5uvALbPOrDPrzDqzzqwz68w6s86sM+vMOrPOrDPrzDqzzqwz68w6s86sM+vMOrPOrDPrzDqzzqwz68w6s86sezybi/3MzMzMzMzM7Dez15l1Zp1ZZ9aZdWadWWfWmXVmnVln1pl1Zp1ZZ9aZdWadWWfWmXVmnVln1pl1Zp1ZZ9aZdWadWWfWmXVm3bqzAwnmW2UKZW5kc3RyZWFtCmVuZG9iagozNyAwIG9iagoxNzIKZW5kb2JqCjE0IDAgb2JqCjw8IC9CaXRzUGVyQ29tcG9uZW50IDggL0NvbG9yU3BhY2UgWy9JbmRleGVkIC9EZXZpY2VSR0IgMiAo/ec3ACJN/wAAKV0KL0RlY29kZVBhcm1zIDw8IC9Db2xvcnMgMSAvQ29sdW1ucyAxNTMgL1ByZWRpY3RvciAxMCA+PgovRmlsdGVyIC9GbGF0ZURlY29kZSAvSGVpZ2h0IDE1MyAvTGVuZ3RoIDM4IDAgUiAvU3VidHlwZSAvSW1hZ2UKL1R5cGUgL1hPYmplY3QgL1dpZHRoIDE1MyA+PgpzdHJlYW0KeJztzrEJgFAQQLHT/Ye21cLiCeIXkgky26rm68Ats86sM+vMOrPOrDPrzDqzzqwz68w6s86sM+vMOrPOrDPrzDqzzqwz68w6s86sM+vMOrPOrDPrzDqzzqx7PJuL/czMzMzMzMzsN7PXmXVmnVln1pl1Zp1ZZ9aZdWadWWfWmXVmnVln1pl1Zp1ZZ9aZdWadWWfWmXVmnVln1pl1Zp1ZZ9aZdWbdurMDCeZbZQplbmRzdHJlYW0KZW5kb2JqCjM4IDAgb2JqCjE3MgplbmRvYmoKMiAwIG9iago8PCAvQ291bnQgMSAvS2lkcyBbIDExIDAgUiBdIC9UeXBlIC9QYWdlcyA+PgplbmRvYmoKMzkgMCBvYmoKPDwgL0NyZWF0aW9uRGF0ZSAoRDoyMDIyMDUwMzAzMTYxNCswMicwMCcpCi9DcmVhdG9yIChNYXRwbG90bGliIHYzLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjUuMSkgPj4KZW5kb2JqCnhyZWYKMCA0MAowMDAwMDAwMDAwIDY1NTM1IGYgCjAwMDAwMDAwMTYgMDAwMDAgbiAKMDAwMDAwODA5NCAwMDAwMCBuIAowMDAwMDA2OTg2IDAwMDAwIG4gCjAwMDAwMDcwMTggMDAwMDAgbiAKMDAwMDAwNzExNyAwMDAwMCBuIAowMDAwMDA3MTM4IDAwMDAwIG4gCjAwMDAwMDcxNTkgMDAwMDAgbiAKMDAwMDAwMDA2NSAwMDAwMCBuIAowMDAwMDAwMzQzIDAwMDAwIG4gCjAwMDAwMDA3MTkgMDAwMDAgbiAKMDAwMDAwMDIwOCAwMDAwMCBuIAowMDAwMDAwNjk5IDAwMDAwIG4gCjAwMDAwMDcyMDIgMDAwMDAgbiAKMDAwMDAwNzY0OCAwMDAwMCBuIAowMDAwMDA1NzI3IDAwMDAwIG4gCjAwMDAwMDU1MjAgMDAwMDAgbiAKMDAwMDAwNTExMiAwMDAwMCBuIAowMDAwMDA2NzgwIDAwMDAwIG4gCjAwMDAwMDA3MzkgMDAwMDAgbiAKMDAwMDAwMTA3NiAwMDAwMCBuIAowMDAwMDAxMjQwIDAwMDAwIG4gCjAwMDAwMDE2MjAgMDAwMDAgbiAKMDAwMDAwMTkyNSAwMDAwMCBuIAowMDAwMDAyMjI5IDAwMDAwIG4gCjAwMDAwMDI1NTEgMDAwMDAgbiAKMDAwMDAwMjc2MCAwMDAwMCBuIAowMDAwMDAzMTc0IDAwMDAwIG4gCjAwMDAwMDM0MTEgMDAwMDAgbiAKMDAwMDAwMzU1NSAwMDAwMCBuIAowMDAwMDAzNjc0IDAwMDAwIG4gCjAwMDAwMDM5MTAgMDAwMDAgbiAKMDAwMDAwNDIyMiAwMDAwMCBuIAowMDAwMDA0NDU1IDAwMDAwIG4gCjAwMDAwMDQ1NDUgMDAwMDAgbiAKMDAwMDAwNDc1MSAwMDAwMCBuIAowMDAwMDA0ODk4IDAwMDAwIG4gCjAwMDAwMDc2MjggMDAwMDAgbiAKMDAwMDAwODA3NCAwMDAwMCBuIAowMDAwMDA4MTU0IDAwMDAwIG4gCnRyYWlsZXIKPDwgL0luZm8gMzkgMCBSIC9Sb290IDEgMCBSIC9TaXplIDQwID4+CnN0YXJ0eHJlZgo4MzExCiUlRU9GCg==\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2022-05-03T03:16:14.387027\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.5.1, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}], "source": ["horiz_conv = HorizontalStackConvolution(c_in=1, c_out=1, kernel_size=3, mask_center=True)\n", "horiz_conv.conv.weight.data.fill_(1)\n", "horiz_conv.conv.bias.data.fill_(0)\n", "horiz_img = horiz_conv(inp_img)\n", "show_center_recep_field(inp_img, horiz_img)"]}, {"cell_type": "markdown", "id": "eb69c846", "metadata": {"papermill": {"duration": 0.009616, "end_time": "2022-05-03T01:16:14.489249", "exception": false, "start_time": "2022-05-03T01:16:14.479633", "status": "completed"}, "tags": []}, "source": ["The receptive field is shown in yellow, the center pixel in red, and all other pixels outside of the receptive field are dark blue.\n", "As expected, the receptive field of a single horizontal convolution with the center pixel masked and a $3\\times3$ kernel is only the pixel on the left.\n", "If we use a larger kernel size, more pixels would be taken into account on the left.\n", "\n", "Next, let's take a look at the vertical convolution:"]}, {"cell_type": "code", "execution_count": 10, "id": "9b2bbcc9", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:16:14.509577Z", "iopub.status.busy": "2022-05-03T01:16:14.509301Z", "iopub.status.idle": "2022-05-03T01:16:14.664602Z", "shell.execute_reply": "2022-05-03T01:16:14.663922Z"}, "papermill": {"duration": 0.167378, "end_time": "2022-05-03T01:16:14.666149", "exception": false, "start_time": "2022-05-03T01:16:14.498771", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUiAvTWVkaWFCb3ggWyAwIDAgMzQ5LjIgMTgxLjcwNjgxODE4MTggXQovUGFyZW50IDIgMCBSIC9SZXNvdXJjZXMgOCAwIFIgL1R5cGUgL1BhZ2UgPj4KZW5kb2JqCjkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMiAwIFIgPj4Kc3RyZWFtCnicnVFLTwIxEL7Pr/iOcLA701K2PYIo0Ru6CQfjwSwFlgDKIxD/vbMKPvAidjLpzPSb19esl3ZVme76XVzeU/bllRsSzCjrCCYbMGaqewj6+AlijS/ItaKxas0PlgQxObeDBA3xqVvjpkRLWiFXcK3irVHUQU7ddcIQS2QdWw+jc6nutUxfK4h3qEvWd65SLpDdCHrPGNAAq2MSY/I9sfZppSsyLjQXwoad49j2kSOk7Y37al8uqFsgu1aYRTF+36AY0QMawyZ8MD663KORqsl0m0ZYN2HZBHc8+lKml221SxhXaT5q4hHFLV0V9DGhhGg+m/2bDvpNh0Q+EmLPJcSyfpm3ocVKiPydkW61fFq/nscBvQHN+pCiCmVuZHN0cmVhbQplbmRvYmoKMTIgMCBvYmoKMjgxCmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjE5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjY0ID4+CnN0cmVhbQp4nD1SuZEDMQzLtwqWwF9SPb65cWD3nx7APTsxMUsRAEH3blE5Gz9tJcuP/NhlZ0nklvegbfK6ouoGqZsgc8lxyRVilvK4EiQWLWUm1meqg/JxDWoVvvATMxPTIUnEEnJSjyKsj6+D1/Uc3F836fwaGR90bFB6S5KroFTDvnK0EjuW6l2D7IOmm9LqM9Ou7ICl8Yas1KROfx31vyMswqTeSEUHIRa/08sdsiykHNTbuX4D4V3tglhPrTzsEGneVnpLFczmzARMGW07rIC1WWPCVAnwFs0h1FqIzBc6rYj55NzRwTRVeRoiw1KFg+09ozVc6BoSS8P5ScYE5r/w+qzIbX//AHBoZ7oKZW5kc3RyZWFtCmVuZG9iagoyMCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDkyID4+CnN0cmVhbQp4nD2NwQ3AMAgD/0zBCBACxPtUVR/p/t8mEeoHHwbZGGBhszXgwdnAl9LaN72kRZPaCFa1Rd1QnrsUpVhdR6VMwk+ZO39SdBztcA7b39blOE3j6F/30P0BD0oeCwplbmRzdHJlYW0KZW5kb2JqCjIxIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzA3ID4+CnN0cmVhbQp4nD2SS24DMQxD9z6FLhDA+tme86Qoupjef9snJemKHNkWRWqWukxZUx6QNJOEf+nwcLGd8jtsz2Zm4Fqil4nllOfQFWLuonzZzEZdWSfF6oRmOrfoUTkXBzZNqp+rLKXdLngO1yaeW/YRP7zQoB7UNS4JN3RXo2UpNGOq+3/Se/yMMuBqTF1sUqt7HzxeRFXo6AdHiSJjlxfn40EJ6UrCaFqIlXdFA0Hu8rTKewnu295qyLIHqZjOOylmsOt0Ui5uF4chHsjyqPDlo9hrQs/4sCsl9EjYhjNyJ+5oxubUyOKQ/t6NBEuPrmgh8+CvbtYuYLxTOkViZE5yrGmLVU73UBTTucO9DBD1bEVDKXOR1epfw84La5ZsFnhK+gUeo90mSw5W2duoTu+tPNnQ9x9a13QfCmVuZHN0cmVhbQplbmRvYmoKMjIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMzIgPj4Kc3RyZWFtCnicNVFJbsQwDLv7FfzAANbuvCfFoIf2/9dSyhQIQCW2uCViYyMCLzH4OYjc+JI1oyZ+Z3JX/CxPhUfCreBJFIGX4V52gssbxmU/DjMfvJdWzqTGkwzIRTY9PBEy2CUQOjC7BnXYZtqJviHhsyNSzUaW09cS9NIqBMpTtt/pghJtq/pz+6wLbfvaE052e+pJ5ROI55aswGXjFZPFWAY9UblLMX2Q6myhJ6G8KJ+DbD5qiESXKGfgicHBKNAO7LntZ+JVIWhd3adtY6hGSsfTvw1NTZII+UQJZ7Y07hb+f8+9vtf7D04hVBEKZW5kc3RyZWFtCmVuZG9iagoyMyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIzMSA+PgpzdHJlYW0KeJw1TzmSBCEMy3mFPjBVGNtAv6entjbY+X+6kplOkPAhydMTHZl4mSMjsGbH21pkIGbgU0zFv/a0DxOq9+AeIpSLC2GGkXDWrONuno4X/3aVz1gH7zb4illeENjCTNZXFmcu2wVjaZzEOclujF0TsY11radTWEcwoQyEdLbDlCBzVKT0yY4y5ug4kSeei+/22yx2OX4O6ws2jSEV5/gqeoI2g6Lsee8CGnJB/13d+B5Fu+glIBsJFtZRYu6c5YRfvXZ0HrUoEnNCmkEuEyHN6SqmEJpQrLOjoFJRcKk+p+isn3/lX1wtCmVuZHN0cmVhbQplbmRvYmoKMjQgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNDkgPj4Kc3RyZWFtCnicPVA7jkQhDOs5hS/wJPIjcB5Gqy1m79+uA5opUEx+tjMk0BGBRwwxlK/jJa2groG/i0LxbuLrg8Igq0NSIM56D4h07KY2kRM6HZwzP2E3Y47ARTEGnOl0pj0HJjn7wgqEcxtl7FZIJ4mqIo7qM44pnip7n3gWLO3INlsnkj3kIOFSUonJpZ+Uyj9typQKOmbRBCwSueBkE004y7tJUowZlDLqHqZ2In2sPMijOuhkTc6sI5nZ00/bmfgccLdf2mROlcd0Hsz4nLTOgzkVuvfjiTYHTY3a6Oz3E2kqL1K7HVqdfnUSld0Y5xgSl2d/Gd9k//kH/odaIgplbmRzdHJlYW0KZW5kb2JqCjI1IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTM2ID4+CnN0cmVhbQp4nE2PQQ4DMQgD73mFn0AgQHjPVlUP2/9fS9h20wseyYBsUQaBJYd4hxvh0dsP30U2FWfjnF9SKWIhmE9wnzBTHI0pd/Jjj4BxlGosp2h4XkvOTcMXLXcTLaWtl5MZb7jul/dHlW2RDUXPLQtC12yS+TKBB3wYmEd142mlx932bK/2/ADObDRJCmVuZHN0cmVhbQplbmRvYmoKMjYgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzNDEgPj4Kc3RyZWFtCnicRVJLbkQxCNu/U3CBSOGXkPO0qrqY3n9bm0zVzeAJYGx4y1OmZMqwuSUjJNeUT30iQ6ym/DRyJCKm+EkJBXaVj8drS6yN7JGoFJ/a8eOx9Eam2RVa9e7Rpc2iUc3KyDnIEKGeFbqye9QO2fB6XEi675TNIRzL/1CBLGXdcgolQVvQd+wR3w8droIrgmGway6D7WUy1P/6hxZc7333YscugBas577BDgCopxO0BcgZ2u42KWgAVbqLScKj8npudqJso1Xp+RwAMw4wcsCIJVsdvtHeAJZ9XehFjYr9K0BRWUD8yNV2wd4xyUhwFuYGjr1wPMWZcEs4xgJAir3iGHrwJdjmL1euiJrwCXW6ZC+8wp7a5udCkwh3rQAOXmTDraujqJbt6TyC9mdFckaM1Is4OiGSWtI5guLSoB5a41w3seJtI7G5V9/uH+GcL1z26xdL7ITECmVuZHN0cmVhbQplbmRvYmoKMjcgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNjQgPj4Kc3RyZWFtCnicRZDHcQUxDEPvqgIlMIAK9azH8w/r/q+G9NNBehhCDGJPwrBcV3FhdMOPty0zDX9HGe7G+jJjvNVYICfoAwyRiavRpPp2xRmq9OTVYq6jolwvOiISzJLjq0AjfDqyx5O2tjP9dF4f7CHvE/8qKuduYQEuqu5A+VIf8dSP2VHqmqGPKitrHmraV4RdEUrbPi6nMk7dvQNa4b2Vqz3a7z8edjryCmVuZHN0cmVhbQplbmRvYmoKMjggMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA3MiA+PgpzdHJlYW0KeJwzMrdQMFCwNAEShhYmCuZmBgophlxAvqmJuUIuF0gMxMoBswyAtCWcgohngJggbRDFIBZEsZmJGUQdnAGRy+BKAwAl2xbJCmVuZHN0cmVhbQplbmRvYmoKMjkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA0NyA+PgpzdHJlYW0KeJwzMrdQMFCwNAEShhYmCuZmBgophlyWEFYuF0wsB8wC0ZZwCiKewZUGALlnDScKZW5kc3RyZWFtCmVuZG9iagozMCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE2MyA+PgpzdHJlYW0KeJxFkDsSAyEMQ3tOoSP4IwM+z2YyKTb3b2PYbFLA01ggg7sTgtTagonogoe2Jd0F760EZ2P86TZuNRLkBHWAVqTjaJRSfbnFaZV08Wg2cysLrRMdZg56lKMZoBA6Fd7touRypu7O+UNw9V/1v2LdOZuJgcnKHQjN6lPc+TY7orq6yf6kx9ys134r7FVhaVlLywm3nbtmQAncUznaqz0/Hwo69gplbmRzdHJlYW0KZW5kb2JqCjMxIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjM5ID4+CnN0cmVhbQp4nE1QyW0EMQz7uwo1MMDoHLseB4s8sv1/Q8oJkpdoS+Kh8pRblspl9yM5b8m65UOHTpVp8m7Qza+x/qMMAnb/UFQQrSWxSsxc0m6xNEkv2cM4jZdrtY7nqXuEWaN48OPY0ymB6T0ywWazvTkwqz3ODpBOuMav6tM7lSQDibqQ80KlCuse1CWijyvbmFKdTi3lGJef6Ht8jgA9xd6N3NHHyxeMRrUtqNFqlTgPMBNT0ZVxq5GBlBMGQ2dHVzQLpcjKekI1wo05oZm9w3BgA8uzhKSlrVK8D2UB6AJd2jrjNEqCjgDC3yiM9foGqvxeNwplbmRzdHJlYW0KZW5kb2JqCjMyIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTYwID4+CnN0cmVhbQp4nEWQORIDMQgEc72CJ0hcgvesy7XB+v+pB9ZHoukCNBy6Fk3KehRoPumxRqG60GvoLEqSRMEWkh1Qp2OIOyhITEhjkki2HoMjmlizXZiZVCqzUuG0acXCv9la1chEjXCN/InpBlT8T+pclPBNg6+SMfoYVLw7g4xJ+F5F3Fox7f5EMLEZ9glvRSYFhImxqdm+z2CGzPcK1zjH8w1MgjfrCmVuZHN0cmVhbQplbmRvYmoKMzMgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxOCA+PgpzdHJlYW0KeJwzNrRQMIDDFEOuNAAd5gNSCmVuZHN0cmVhbQplbmRvYmoKMzQgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMzMgPj4Kc3RyZWFtCnicRY9LDgQhCET3nKKOwMcf53Ey6YVz/+2AnW4TYz2FVIG5gqE9LmsDnRUfIRm28beplo5FWT5UelJWD8ngh6zGyyHcoCzwgkkqhiFQi5gakS1lbreA2zYNsrKVU6WOsIujMI/2tGwVHl+iWyJ1kj+DxCov3OO6Hcil1rveoou+f6QBMQkKZW5kc3RyZWFtCmVuZG9iagozNSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDc1ID4+CnN0cmVhbQp4nDO1NFIwUDA2ABKmZkYKpibmCimGXEA+iJXLZWhkCmblcBlZmilYWAAZJmbmUCGYhhwuY1NzoAFARcamYBqqP4crgysNAJWQEu8KZW5kc3RyZWFtCmVuZG9iagozNiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE0MSA+PgpzdHJlYW0KeJw9j8EOwzAIQ+/5Cv9ApNgpoXxPp2qH7v+vI0u7C3oCY4yF0NAbqprDhmCb48XSJVRr+BTFQCU3yJlgDqWk0h1HkXpiOBhcHrQbjuKx6PoRu5JmfdDGQrolaIB7rFNp3KZxE8QdNQXqKeqco7wQuZ+pZ9g0kt00s5JzuA2/e89T1/+nq7zL+QW9dy7+CmVuZHN0cmVhbQplbmRvYmoKMTcgMCBvYmoKPDwgL0Jhc2VGb250IC9CTVFRRFYrRGVqYVZ1U2FucyAvQ2hhclByb2NzIDE4IDAgUgovRW5jb2RpbmcgPDwKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCi9UeXBlIC9FbmNvZGluZyA+PgovRmlyc3RDaGFyIDAgL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvRm9udERlc2NyaXB0b3IgMTYgMCBSCi9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdIC9MYXN0Q2hhciAyNTUgL05hbWUgL0JNUVFEVitEZWphVnVTYW5zCi9TdWJ0eXBlIC9UeXBlMyAvVHlwZSAvRm9udCAvV2lkdGhzIDE1IDAgUiA+PgplbmRvYmoKMTYgMCBvYmoKPDwgL0FzY2VudCA5MjkgL0NhcEhlaWdodCAwIC9EZXNjZW50IC0yMzYgL0ZsYWdzIDMyCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovSXRhbGljQW5nbGUgMCAvTWF4V2lkdGggMTM0MiAvU3RlbVYgMCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL1hIZWlnaHQgMCA+PgplbmRvYmoKMTUgMCBvYmoKWyA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMAo2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDMxOCA0MDEgNDYwIDgzOCA2MzYKOTUwIDc4MCAyNzUgMzkwIDM5MCA1MDAgODM4IDMxOCAzNjEgMzE4IDMzNyA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2CjYzNiA2MzYgMzM3IDMzNyA4MzggODM4IDgzOCA1MzEgMTAwMCA2ODQgNjg2IDY5OCA3NzAgNjMyIDU3NSA3NzUgNzUyIDI5NQoyOTUgNjU2IDU1NyA4NjMgNzQ4IDc4NyA2MDMgNzg3IDY5NSA2MzUgNjExIDczMiA2ODQgOTg5IDY4NSA2MTEgNjg1IDM5MCAzMzcKMzkwIDgzOCA1MDAgNTAwIDYxMyA2MzUgNTUwIDYzNSA2MTUgMzUyIDYzNSA2MzQgMjc4IDI3OCA1NzkgMjc4IDk3NCA2MzQgNjEyCjYzNSA2MzUgNDExIDUyMSAzOTIgNjM0IDU5MiA4MTggNTkyIDU5MiA1MjUgNjM2IDMzNyA2MzYgODM4IDYwMCA2MzYgNjAwIDMxOAozNTIgNTE4IDEwMDAgNTAwIDUwMCA1MDAgMTM0MiA2MzUgNDAwIDEwNzAgNjAwIDY4NSA2MDAgNjAwIDMxOCAzMTggNTE4IDUxOAo1OTAgNTAwIDEwMDAgNTAwIDEwMDAgNTIxIDQwMCAxMDIzIDYwMCA1MjUgNjExIDMxOCA0MDEgNjM2IDYzNiA2MzYgNjM2IDMzNwo1MDAgNTAwIDEwMDAgNDcxIDYxMiA4MzggMzYxIDEwMDAgNTAwIDUwMCA4MzggNDAxIDQwMSA1MDAgNjM2IDYzNiAzMTggNTAwCjQwMSA0NzEgNjEyIDk2OSA5NjkgOTY5IDUzMSA2ODQgNjg0IDY4NCA2ODQgNjg0IDY4NCA5NzQgNjk4IDYzMiA2MzIgNjMyIDYzMgoyOTUgMjk1IDI5NSAyOTUgNzc1IDc0OCA3ODcgNzg3IDc4NyA3ODcgNzg3IDgzOCA3ODcgNzMyIDczMiA3MzIgNzMyIDYxMSA2MDUKNjMwIDYxMyA2MTMgNjEzIDYxMyA2MTMgNjEzIDk4MiA1NTAgNjE1IDYxNSA2MTUgNjE1IDI3OCAyNzggMjc4IDI3OCA2MTIgNjM0CjYxMiA2MTIgNjEyIDYxMiA2MTIgODM4IDYxMiA2MzQgNjM0IDYzNCA2MzQgNTkyIDYzNSA1OTIgXQplbmRvYmoKMTggMCBvYmoKPDwgL0IgMTkgMCBSIC9XIDIwIDAgUiAvYSAyMSAwIFIgL2MgMjIgMCBSIC9kIDIzIDAgUiAvZSAyNCAwIFIgL2YgMjUgMCBSCi9nIDI2IDAgUiAvaCAyNyAwIFIgL2kgMjggMCBSIC9sIDI5IDAgUiAvbiAzMCAwIFIgL3AgMzEgMCBSIC9yIDMyIDAgUgovc3BhY2UgMzMgMCBSIC90IDM0IDAgUiAvdiAzNSAwIFIgL3kgMzYgMCBSID4+CmVuZG9iagozIDAgb2JqCjw8IC9GMSAxNyAwIFIgPj4KZW5kb2JqCjQgMCBvYmoKPDwgL0ExIDw8IC9DQSAwIC9UeXBlIC9FeHRHU3RhdGUgL2NhIDEgPj4KL0EyIDw8IC9DQSAxIC9UeXBlIC9FeHRHU3RhdGUgL2NhIDEgPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgL0kxIDEzIDAgUiAvSTIgMTQgMCBSID4+CmVuZG9iagoxMyAwIG9iago8PCAvQml0c1BlckNvbXBvbmVudCA4IC9Db2xvclNwYWNlIFsvSW5kZXhlZCAvRGV2aWNlUkdCIDIgKP3nNwAiTf8AACldCi9EZWNvZGVQYXJtcyA8PCAvQ29sb3JzIDEgL0NvbHVtbnMgMTUzIC9QcmVkaWN0b3IgMTAgPj4KL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0hlaWdodCAxNTMgL0xlbmd0aCAzNyAwIFIgL1N1YnR5cGUgL0ltYWdlCi9UeXBlIC9YT2JqZWN0IC9XaWR0aCAxNTMgPj4Kc3RyZWFtCnic7dixCYBAAATB1/6LNjUR3FdRZLaAY+Iby1cbbwMOI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch607JxNjIyMjIyMjKyy6377pkk65H1yHpkPbIeWe+Hv8bjkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIehthHlndCmVuZHN0cmVhbQplbmRvYmoKMzcgMCBvYmoKMTg1CmVuZG9iagoxNCAwIG9iago8PCAvQml0c1BlckNvbXBvbmVudCA4IC9Db2xvclNwYWNlIFsvSW5kZXhlZCAvRGV2aWNlUkdCIDIgKP3nNwAiTf8AACldCi9EZWNvZGVQYXJtcyA8PCAvQ29sb3JzIDEgL0NvbHVtbnMgMTUzIC9QcmVkaWN0b3IgMTAgPj4KL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0hlaWdodCAxNTMgL0xlbmd0aCAzOCAwIFIgL1N1YnR5cGUgL0ltYWdlCi9UeXBlIC9YT2JqZWN0IC9XaWR0aCAxNTMgPj4Kc3RyZWFtCnic7dixCYBAAATB1/6LNjUR3FdRZLaAY+Iby1cbbwMOI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch607JxNjIyMjIyMjKyy6377pkk65H1yHpkPbIeWe+Hv8bjkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIehthHlndCmVuZHN0cmVhbQplbmRvYmoKMzggMCBvYmoKMTg1CmVuZG9iagoyIDAgb2JqCjw8IC9Db3VudCAxIC9LaWRzIFsgMTEgMCBSIF0gL1R5cGUgL1BhZ2VzID4+CmVuZG9iagozOSAwIG9iago8PCAvQ3JlYXRpb25EYXRlIChEOjIwMjIwNTAzMDMxNjE0KzAyJzAwJykKL0NyZWF0b3IgKE1hdHBsb3RsaWIgdjMuNS4xLCBodHRwczovL21hdHBsb3RsaWIub3JnKQovUHJvZHVjZXIgKE1hdHBsb3RsaWIgcGRmIGJhY2tlbmQgdjMuNS4xKSA+PgplbmRvYmoKeHJlZgowIDQwCjAwMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAwMDAxNiAwMDAwMCBuIAowMDAwMDA4MTIwIDAwMDAwIG4gCjAwMDAwMDY5ODYgMDAwMDAgbiAKMDAwMDAwNzAxOCAwMDAwMCBuIAowMDAwMDA3MTE3IDAwMDAwIG4gCjAwMDAwMDcxMzggMDAwMDAgbiAKMDAwMDAwNzE1OSAwMDAwMCBuIAowMDAwMDAwMDY1IDAwMDAwIG4gCjAwMDAwMDAzNDMgMDAwMDAgbiAKMDAwMDAwMDcxOSAwMDAwMCBuIAowMDAwMDAwMjA4IDAwMDAwIG4gCjAwMDAwMDA2OTkgMDAwMDAgbiAKMDAwMDAwNzIwMiAwMDAwMCBuIAowMDAwMDA3NjYxIDAwMDAwIG4gCjAwMDAwMDU3MjcgMDAwMDAgbiAKMDAwMDAwNTUyMCAwMDAwMCBuIAowMDAwMDA1MTEyIDAwMDAwIG4gCjAwMDAwMDY3ODAgMDAwMDAgbiAKMDAwMDAwMDczOSAwMDAwMCBuIAowMDAwMDAxMDc2IDAwMDAwIG4gCjAwMDAwMDEyNDAgMDAwMDAgbiAKMDAwMDAwMTYyMCAwMDAwMCBuIAowMDAwMDAxOTI1IDAwMDAwIG4gCjAwMDAwMDIyMjkgMDAwMDAgbiAKMDAwMDAwMjU1MSAwMDAwMCBuIAowMDAwMDAyNzYwIDAwMDAwIG4gCjAwMDAwMDMxNzQgMDAwMDAgbiAKMDAwMDAwMzQxMSAwMDAwMCBuIAowMDAwMDAzNTU1IDAwMDAwIG4gCjAwMDAwMDM2NzQgMDAwMDAgbiAKMDAwMDAwMzkxMCAwMDAwMCBuIAowMDAwMDA0MjIyIDAwMDAwIG4gCjAwMDAwMDQ0NTUgMDAwMDAgbiAKMDAwMDAwNDU0NSAwMDAwMCBuIAowMDAwMDA0NzUxIDAwMDAwIG4gCjAwMDAwMDQ4OTggMDAwMDAgbiAKMDAwMDAwNzY0MSAwMDAwMCBuIAowMDAwMDA4MTAwIDAwMDAwIG4gCjAwMDAwMDgxODAgMDAwMDAgbiAKdHJhaWxlcgo8PCAvSW5mbyAzOSAwIFIgL1Jvb3QgMSAwIFIgL1NpemUgNDAgPj4Kc3RhcnR4cmVmCjgzMzcKJSVFT0YK\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2022-05-03T03:16:14.587678\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.5.1, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}], "source": ["vert_conv = VerticalStackConvolution(c_in=1, c_out=1, kernel_size=3, mask_center=True)\n", "vert_conv.conv.weight.data.fill_(1)\n", "vert_conv.conv.bias.data.fill_(0)\n", "vert_img = vert_conv(inp_img)\n", "show_center_recep_field(inp_img, vert_img)"]}, {"cell_type": "markdown", "id": "76faaa78", "metadata": {"papermill": {"duration": 0.010251, "end_time": "2022-05-03T01:16:14.687305", "exception": false, "start_time": "2022-05-03T01:16:14.677054", "status": "completed"}, "tags": []}, "source": ["The vertical convolution takes all pixels above into account. Combining\n", "these two, we get the L-shaped receptive field of the original masked\n", "convolution:"]}, {"cell_type": "code", "execution_count": 11, "id": "b3b149c7", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:16:14.709021Z", "iopub.status.busy": "2022-05-03T01:16:14.708743Z", "iopub.status.idle": "2022-05-03T01:16:14.861762Z", "shell.execute_reply": "2022-05-03T01:16:14.861182Z"}, "papermill": {"duration": 0.16569, "end_time": "2022-05-03T01:16:14.863236", "exception": false, "start_time": "2022-05-03T01:16:14.697546", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUiAvTWVkaWFCb3ggWyAwIDAgMzQ5LjIgMTgxLjcwNjgxODE4MTggXQovUGFyZW50IDIgMCBSIC9SZXNvdXJjZXMgOCAwIFIgL1R5cGUgL1BhZ2UgPj4KZW5kb2JqCjkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMiAwIFIgPj4Kc3RyZWFtCnicnVFLTwIxEL7Pr/iOcLA701K2PYIo0Ru6CQfjwSwFlgDKIxD/vbMKPvAidjLpzPSb19esl3ZVme76XVzeU/bllRsSzCjrCCYbMGaqewj6+AlijS/ItaKxas0PlgQxObeDBA3xqVvjpkRLWiFXcK3irVHUQU7ddcIQS2QdWw+jc6nutUxfK4h3qEvWd65SLpDdCHrPGNAAq2MSY/I9sfZppSsyLjQXwoad49j2kSOk7Y37al8uqFsgu1aYRTF+36AY0QMawyZ8MD663KORqsl0m0ZYN2HZBHc8+lKml221SxhXaT5q4hHFLV0V9DGhhGg+m/2bDvpNh0Q+EmLPJcSyfpm3ocVKiPydkW61fFq/nscBvQHN+pCiCmVuZHN0cmVhbQplbmRvYmoKMTIgMCBvYmoKMjgxCmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjE5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjY0ID4+CnN0cmVhbQp4nD1SuZEDMQzLtwqWwF9SPb65cWD3nx7APTsxMUsRAEH3blE5Gz9tJcuP/NhlZ0nklvegbfK6ouoGqZsgc8lxyRVilvK4EiQWLWUm1meqg/JxDWoVvvATMxPTIUnEEnJSjyKsj6+D1/Uc3F836fwaGR90bFB6S5KroFTDvnK0EjuW6l2D7IOmm9LqM9Ou7ICl8Yas1KROfx31vyMswqTeSEUHIRa/08sdsiykHNTbuX4D4V3tglhPrTzsEGneVnpLFczmzARMGW07rIC1WWPCVAnwFs0h1FqIzBc6rYj55NzRwTRVeRoiw1KFg+09ozVc6BoSS8P5ScYE5r/w+qzIbX//AHBoZ7oKZW5kc3RyZWFtCmVuZG9iagoyMCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDkyID4+CnN0cmVhbQp4nD2NwQ3AMAgD/0zBCBACxPtUVR/p/t8mEeoHHwbZGGBhszXgwdnAl9LaN72kRZPaCFa1Rd1QnrsUpVhdR6VMwk+ZO39SdBztcA7b39blOE3j6F/30P0BD0oeCwplbmRzdHJlYW0KZW5kb2JqCjIxIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzA3ID4+CnN0cmVhbQp4nD2SS24DMQxD9z6FLhDA+tme86Qoupjef9snJemKHNkWRWqWukxZUx6QNJOEf+nwcLGd8jtsz2Zm4Fqil4nllOfQFWLuonzZzEZdWSfF6oRmOrfoUTkXBzZNqp+rLKXdLngO1yaeW/YRP7zQoB7UNS4JN3RXo2UpNGOq+3/Se/yMMuBqTF1sUqt7HzxeRFXo6AdHiSJjlxfn40EJ6UrCaFqIlXdFA0Hu8rTKewnu295qyLIHqZjOOylmsOt0Ui5uF4chHsjyqPDlo9hrQs/4sCsl9EjYhjNyJ+5oxubUyOKQ/t6NBEuPrmgh8+CvbtYuYLxTOkViZE5yrGmLVU73UBTTucO9DBD1bEVDKXOR1epfw84La5ZsFnhK+gUeo90mSw5W2duoTu+tPNnQ9x9a13QfCmVuZHN0cmVhbQplbmRvYmoKMjIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMzIgPj4Kc3RyZWFtCnicNVFJbsQwDLv7FfzAANbuvCfFoIf2/9dSyhQIQCW2uCViYyMCLzH4OYjc+JI1oyZ+Z3JX/CxPhUfCreBJFIGX4V52gssbxmU/DjMfvJdWzqTGkwzIRTY9PBEy2CUQOjC7BnXYZtqJviHhsyNSzUaW09cS9NIqBMpTtt/pghJtq/pz+6wLbfvaE052e+pJ5ROI55aswGXjFZPFWAY9UblLMX2Q6myhJ6G8KJ+DbD5qiESXKGfgicHBKNAO7LntZ+JVIWhd3adtY6hGSsfTvw1NTZII+UQJZ7Y07hb+f8+9vtf7D04hVBEKZW5kc3RyZWFtCmVuZG9iagoyMyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIzMSA+PgpzdHJlYW0KeJw1TzmSBCEMy3mFPjBVGNtAv6entjbY+X+6kplOkPAhydMTHZl4mSMjsGbH21pkIGbgU0zFv/a0DxOq9+AeIpSLC2GGkXDWrONuno4X/3aVz1gH7zb4illeENjCTNZXFmcu2wVjaZzEOclujF0TsY11radTWEcwoQyEdLbDlCBzVKT0yY4y5ug4kSeei+/22yx2OX4O6ws2jSEV5/gqeoI2g6Lsee8CGnJB/13d+B5Fu+glIBsJFtZRYu6c5YRfvXZ0HrUoEnNCmkEuEyHN6SqmEJpQrLOjoFJRcKk+p+isn3/lX1wtCmVuZHN0cmVhbQplbmRvYmoKMjQgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNDkgPj4Kc3RyZWFtCnicPVA7jkQhDOs5hS/wJPIjcB5Gqy1m79+uA5opUEx+tjMk0BGBRwwxlK/jJa2groG/i0LxbuLrg8Igq0NSIM56D4h07KY2kRM6HZwzP2E3Y47ARTEGnOl0pj0HJjn7wgqEcxtl7FZIJ4mqIo7qM44pnip7n3gWLO3INlsnkj3kIOFSUonJpZ+Uyj9typQKOmbRBCwSueBkE004y7tJUowZlDLqHqZ2In2sPMijOuhkTc6sI5nZ00/bmfgccLdf2mROlcd0Hsz4nLTOgzkVuvfjiTYHTY3a6Oz3E2kqL1K7HVqdfnUSld0Y5xgSl2d/Gd9k//kH/odaIgplbmRzdHJlYW0KZW5kb2JqCjI1IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTM2ID4+CnN0cmVhbQp4nE2PQQ4DMQgD73mFn0AgQHjPVlUP2/9fS9h20wseyYBsUQaBJYd4hxvh0dsP30U2FWfjnF9SKWIhmE9wnzBTHI0pd/Jjj4BxlGosp2h4XkvOTcMXLXcTLaWtl5MZb7jul/dHlW2RDUXPLQtC12yS+TKBB3wYmEd142mlx932bK/2/ADObDRJCmVuZHN0cmVhbQplbmRvYmoKMjYgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzNDEgPj4Kc3RyZWFtCnicRVJLbkQxCNu/U3CBSOGXkPO0qrqY3n9bm0zVzeAJYGx4y1OmZMqwuSUjJNeUT30iQ6ym/DRyJCKm+EkJBXaVj8drS6yN7JGoFJ/a8eOx9Eam2RVa9e7Rpc2iUc3KyDnIEKGeFbqye9QO2fB6XEi675TNIRzL/1CBLGXdcgolQVvQd+wR3w8droIrgmGway6D7WUy1P/6hxZc7333YscugBas577BDgCopxO0BcgZ2u42KWgAVbqLScKj8npudqJso1Xp+RwAMw4wcsCIJVsdvtHeAJZ9XehFjYr9K0BRWUD8yNV2wd4xyUhwFuYGjr1wPMWZcEs4xgJAir3iGHrwJdjmL1euiJrwCXW6ZC+8wp7a5udCkwh3rQAOXmTDraujqJbt6TyC9mdFckaM1Is4OiGSWtI5guLSoB5a41w3seJtI7G5V9/uH+GcL1z26xdL7ITECmVuZHN0cmVhbQplbmRvYmoKMjcgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNjQgPj4Kc3RyZWFtCnicRZDHcQUxDEPvqgIlMIAK9azH8w/r/q+G9NNBehhCDGJPwrBcV3FhdMOPty0zDX9HGe7G+jJjvNVYICfoAwyRiavRpPp2xRmq9OTVYq6jolwvOiISzJLjq0AjfDqyx5O2tjP9dF4f7CHvE/8qKuduYQEuqu5A+VIf8dSP2VHqmqGPKitrHmraV4RdEUrbPi6nMk7dvQNa4b2Vqz3a7z8edjryCmVuZHN0cmVhbQplbmRvYmoKMjggMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA3MiA+PgpzdHJlYW0KeJwzMrdQMFCwNAEShhYmCuZmBgophlxAvqmJuUIuF0gMxMoBswyAtCWcgohngJggbRDFIBZEsZmJGUQdnAGRy+BKAwAl2xbJCmVuZHN0cmVhbQplbmRvYmoKMjkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA0NyA+PgpzdHJlYW0KeJwzMrdQMFCwNAEShhYmCuZmBgophlyWEFYuF0wsB8wC0ZZwCiKewZUGALlnDScKZW5kc3RyZWFtCmVuZG9iagozMCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE2MyA+PgpzdHJlYW0KeJxFkDsSAyEMQ3tOoSP4IwM+z2YyKTb3b2PYbFLA01ggg7sTgtTagonogoe2Jd0F760EZ2P86TZuNRLkBHWAVqTjaJRSfbnFaZV08Wg2cysLrRMdZg56lKMZoBA6Fd7touRypu7O+UNw9V/1v2LdOZuJgcnKHQjN6lPc+TY7orq6yf6kx9ys134r7FVhaVlLywm3nbtmQAncUznaqz0/Hwo69gplbmRzdHJlYW0KZW5kb2JqCjMxIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjM5ID4+CnN0cmVhbQp4nE1QyW0EMQz7uwo1MMDoHLseB4s8sv1/Q8oJkpdoS+Kh8pRblspl9yM5b8m65UOHTpVp8m7Qza+x/qMMAnb/UFQQrSWxSsxc0m6xNEkv2cM4jZdrtY7nqXuEWaN48OPY0ymB6T0ywWazvTkwqz3ODpBOuMav6tM7lSQDibqQ80KlCuse1CWijyvbmFKdTi3lGJef6Ht8jgA9xd6N3NHHyxeMRrUtqNFqlTgPMBNT0ZVxq5GBlBMGQ2dHVzQLpcjKekI1wo05oZm9w3BgA8uzhKSlrVK8D2UB6AJd2jrjNEqCjgDC3yiM9foGqvxeNwplbmRzdHJlYW0KZW5kb2JqCjMyIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTYwID4+CnN0cmVhbQp4nEWQORIDMQgEc72CJ0hcgvesy7XB+v+pB9ZHoukCNBy6Fk3KehRoPumxRqG60GvoLEqSRMEWkh1Qp2OIOyhITEhjkki2HoMjmlizXZiZVCqzUuG0acXCv9la1chEjXCN/InpBlT8T+pclPBNg6+SMfoYVLw7g4xJ+F5F3Fox7f5EMLEZ9glvRSYFhImxqdm+z2CGzPcK1zjH8w1MgjfrCmVuZHN0cmVhbQplbmRvYmoKMzMgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxOCA+PgpzdHJlYW0KeJwzNrRQMIDDFEOuNAAd5gNSCmVuZHN0cmVhbQplbmRvYmoKMzQgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMzMgPj4Kc3RyZWFtCnicRY9LDgQhCET3nKKOwMcf53Ey6YVz/+2AnW4TYz2FVIG5gqE9LmsDnRUfIRm28beplo5FWT5UelJWD8ngh6zGyyHcoCzwgkkqhiFQi5gakS1lbreA2zYNsrKVU6WOsIujMI/2tGwVHl+iWyJ1kj+DxCov3OO6Hcil1rveoou+f6QBMQkKZW5kc3RyZWFtCmVuZG9iagozNSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDc1ID4+CnN0cmVhbQp4nDO1NFIwUDA2ABKmZkYKpibmCimGXEA+iJXLZWhkCmblcBlZmilYWAAZJmbmUCGYhhwuY1NzoAFARcamYBqqP4crgysNAJWQEu8KZW5kc3RyZWFtCmVuZG9iagozNiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE0MSA+PgpzdHJlYW0KeJw9j8EOwzAIQ+/5Cv9ApNgpoXxPp2qH7v+vI0u7C3oCY4yF0NAbqprDhmCb48XSJVRr+BTFQCU3yJlgDqWk0h1HkXpiOBhcHrQbjuKx6PoRu5JmfdDGQrolaIB7rFNp3KZxE8QdNQXqKeqco7wQuZ+pZ9g0kt00s5JzuA2/e89T1/+nq7zL+QW9dy7+CmVuZHN0cmVhbQplbmRvYmoKMTcgMCBvYmoKPDwgL0Jhc2VGb250IC9CTVFRRFYrRGVqYVZ1U2FucyAvQ2hhclByb2NzIDE4IDAgUgovRW5jb2RpbmcgPDwKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCi9UeXBlIC9FbmNvZGluZyA+PgovRmlyc3RDaGFyIDAgL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvRm9udERlc2NyaXB0b3IgMTYgMCBSCi9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdIC9MYXN0Q2hhciAyNTUgL05hbWUgL0JNUVFEVitEZWphVnVTYW5zCi9TdWJ0eXBlIC9UeXBlMyAvVHlwZSAvRm9udCAvV2lkdGhzIDE1IDAgUiA+PgplbmRvYmoKMTYgMCBvYmoKPDwgL0FzY2VudCA5MjkgL0NhcEhlaWdodCAwIC9EZXNjZW50IC0yMzYgL0ZsYWdzIDMyCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovSXRhbGljQW5nbGUgMCAvTWF4V2lkdGggMTM0MiAvU3RlbVYgMCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL1hIZWlnaHQgMCA+PgplbmRvYmoKMTUgMCBvYmoKWyA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMAo2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDMxOCA0MDEgNDYwIDgzOCA2MzYKOTUwIDc4MCAyNzUgMzkwIDM5MCA1MDAgODM4IDMxOCAzNjEgMzE4IDMzNyA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2CjYzNiA2MzYgMzM3IDMzNyA4MzggODM4IDgzOCA1MzEgMTAwMCA2ODQgNjg2IDY5OCA3NzAgNjMyIDU3NSA3NzUgNzUyIDI5NQoyOTUgNjU2IDU1NyA4NjMgNzQ4IDc4NyA2MDMgNzg3IDY5NSA2MzUgNjExIDczMiA2ODQgOTg5IDY4NSA2MTEgNjg1IDM5MCAzMzcKMzkwIDgzOCA1MDAgNTAwIDYxMyA2MzUgNTUwIDYzNSA2MTUgMzUyIDYzNSA2MzQgMjc4IDI3OCA1NzkgMjc4IDk3NCA2MzQgNjEyCjYzNSA2MzUgNDExIDUyMSAzOTIgNjM0IDU5MiA4MTggNTkyIDU5MiA1MjUgNjM2IDMzNyA2MzYgODM4IDYwMCA2MzYgNjAwIDMxOAozNTIgNTE4IDEwMDAgNTAwIDUwMCA1MDAgMTM0MiA2MzUgNDAwIDEwNzAgNjAwIDY4NSA2MDAgNjAwIDMxOCAzMTggNTE4IDUxOAo1OTAgNTAwIDEwMDAgNTAwIDEwMDAgNTIxIDQwMCAxMDIzIDYwMCA1MjUgNjExIDMxOCA0MDEgNjM2IDYzNiA2MzYgNjM2IDMzNwo1MDAgNTAwIDEwMDAgNDcxIDYxMiA4MzggMzYxIDEwMDAgNTAwIDUwMCA4MzggNDAxIDQwMSA1MDAgNjM2IDYzNiAzMTggNTAwCjQwMSA0NzEgNjEyIDk2OSA5NjkgOTY5IDUzMSA2ODQgNjg0IDY4NCA2ODQgNjg0IDY4NCA5NzQgNjk4IDYzMiA2MzIgNjMyIDYzMgoyOTUgMjk1IDI5NSAyOTUgNzc1IDc0OCA3ODcgNzg3IDc4NyA3ODcgNzg3IDgzOCA3ODcgNzMyIDczMiA3MzIgNzMyIDYxMSA2MDUKNjMwIDYxMyA2MTMgNjEzIDYxMyA2MTMgNjEzIDk4MiA1NTAgNjE1IDYxNSA2MTUgNjE1IDI3OCAyNzggMjc4IDI3OCA2MTIgNjM0CjYxMiA2MTIgNjEyIDYxMiA2MTIgODM4IDYxMiA2MzQgNjM0IDYzNCA2MzQgNTkyIDYzNSA1OTIgXQplbmRvYmoKMTggMCBvYmoKPDwgL0IgMTkgMCBSIC9XIDIwIDAgUiAvYSAyMSAwIFIgL2MgMjIgMCBSIC9kIDIzIDAgUiAvZSAyNCAwIFIgL2YgMjUgMCBSCi9nIDI2IDAgUiAvaCAyNyAwIFIgL2kgMjggMCBSIC9sIDI5IDAgUiAvbiAzMCAwIFIgL3AgMzEgMCBSIC9yIDMyIDAgUgovc3BhY2UgMzMgMCBSIC90IDM0IDAgUiAvdiAzNSAwIFIgL3kgMzYgMCBSID4+CmVuZG9iagozIDAgb2JqCjw8IC9GMSAxNyAwIFIgPj4KZW5kb2JqCjQgMCBvYmoKPDwgL0ExIDw8IC9DQSAwIC9UeXBlIC9FeHRHU3RhdGUgL2NhIDEgPj4KL0EyIDw8IC9DQSAxIC9UeXBlIC9FeHRHU3RhdGUgL2NhIDEgPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgL0kxIDEzIDAgUiAvSTIgMTQgMCBSID4+CmVuZG9iagoxMyAwIG9iago8PCAvQml0c1BlckNvbXBvbmVudCA4IC9Db2xvclNwYWNlIFsvSW5kZXhlZCAvRGV2aWNlUkdCIDIgKP3nNwAiTf8AACldCi9EZWNvZGVQYXJtcyA8PCAvQ29sb3JzIDEgL0NvbHVtbnMgMTUzIC9QcmVkaWN0b3IgMTAgPj4KL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0hlaWdodCAxNTMgL0xlbmd0aCAzNyAwIFIgL1N1YnR5cGUgL0ltYWdlCi9UeXBlIC9YT2JqZWN0IC9XaWR0aCAxNTMgPj4Kc3RyZWFtCnic7c6xDYAwAMCwwv9Hs7IgkSKgg3NA5LGt2vgbcBlZj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1puWjbuRkZGRkZGRkT2W7edml2RkZGRkZGSfy16PrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWW1d2AJ4IWScKZW5kc3RyZWFtCmVuZG9iagozNyAwIG9iagoxODEKZW5kb2JqCjE0IDAgb2JqCjw8IC9CaXRzUGVyQ29tcG9uZW50IDggL0NvbG9yU3BhY2UgWy9JbmRleGVkIC9EZXZpY2VSR0IgMiAo/ec3ACJN/wAAKV0KL0RlY29kZVBhcm1zIDw8IC9Db2xvcnMgMSAvQ29sdW1ucyAxNTMgL1ByZWRpY3RvciAxMCA+PgovRmlsdGVyIC9GbGF0ZURlY29kZSAvSGVpZ2h0IDE1MyAvTGVuZ3RoIDM4IDAgUiAvU3VidHlwZSAvSW1hZ2UKL1R5cGUgL1hPYmplY3QgL1dpZHRoIDE1MyA+PgpzdHJlYW0KeJztzrENgDAAwLDC/0ezsiCRIqCDc0Dksa3a+BtwGVmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWm5aNu5GRkZGRkZGRPZbt52aXZGRkZGRkZJ/LXo+sR9Yj65H1yHpkPbIeWY+sR9Yj65H1yHpkPbIeWY+sR9Yj65H1yHpkPbIeWY+sR9Yj65H1yHpkPbIeWY+sR9ZbV3YAnghZJwplbmRzdHJlYW0KZW5kb2JqCjM4IDAgb2JqCjE4MQplbmRvYmoKMiAwIG9iago8PCAvQ291bnQgMSAvS2lkcyBbIDExIDAgUiBdIC9UeXBlIC9QYWdlcyA+PgplbmRvYmoKMzkgMCBvYmoKPDwgL0NyZWF0aW9uRGF0ZSAoRDoyMDIyMDUwMzAzMTYxNCswMicwMCcpCi9DcmVhdG9yIChNYXRwbG90bGliIHYzLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjUuMSkgPj4KZW5kb2JqCnhyZWYKMCA0MAowMDAwMDAwMDAwIDY1NTM1IGYgCjAwMDAwMDAwMTYgMDAwMDAgbiAKMDAwMDAwODExMiAwMDAwMCBuIAowMDAwMDA2OTg2IDAwMDAwIG4gCjAwMDAwMDcwMTggMDAwMDAgbiAKMDAwMDAwNzExNyAwMDAwMCBuIAowMDAwMDA3MTM4IDAwMDAwIG4gCjAwMDAwMDcxNTkgMDAwMDAgbiAKMDAwMDAwMDA2NSAwMDAwMCBuIAowMDAwMDAwMzQzIDAwMDAwIG4gCjAwMDAwMDA3MTkgMDAwMDAgbiAKMDAwMDAwMDIwOCAwMDAwMCBuIAowMDAwMDAwNjk5IDAwMDAwIG4gCjAwMDAwMDcyMDIgMDAwMDAgbiAKMDAwMDAwNzY1NyAwMDAwMCBuIAowMDAwMDA1NzI3IDAwMDAwIG4gCjAwMDAwMDU1MjAgMDAwMDAgbiAKMDAwMDAwNTExMiAwMDAwMCBuIAowMDAwMDA2NzgwIDAwMDAwIG4gCjAwMDAwMDA3MzkgMDAwMDAgbiAKMDAwMDAwMTA3NiAwMDAwMCBuIAowMDAwMDAxMjQwIDAwMDAwIG4gCjAwMDAwMDE2MjAgMDAwMDAgbiAKMDAwMDAwMTkyNSAwMDAwMCBuIAowMDAwMDAyMjI5IDAwMDAwIG4gCjAwMDAwMDI1NTEgMDAwMDAgbiAKMDAwMDAwMjc2MCAwMDAwMCBuIAowMDAwMDAzMTc0IDAwMDAwIG4gCjAwMDAwMDM0MTEgMDAwMDAgbiAKMDAwMDAwMzU1NSAwMDAwMCBuIAowMDAwMDAzNjc0IDAwMDAwIG4gCjAwMDAwMDM5MTAgMDAwMDAgbiAKMDAwMDAwNDIyMiAwMDAwMCBuIAowMDAwMDA0NDU1IDAwMDAwIG4gCjAwMDAwMDQ1NDUgMDAwMDAgbiAKMDAwMDAwNDc1MSAwMDAwMCBuIAowMDAwMDA0ODk4IDAwMDAwIG4gCjAwMDAwMDc2MzcgMDAwMDAgbiAKMDAwMDAwODA5MiAwMDAwMCBuIAowMDAwMDA4MTcyIDAwMDAwIG4gCnRyYWlsZXIKPDwgL0luZm8gMzkgMCBSIC9Sb290IDEgMCBSIC9TaXplIDQwID4+CnN0YXJ0eHJlZgo4MzI5CiUlRU9GCg==\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2022-05-03T03:16:14.784501\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.5.1, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}], "source": ["horiz_img = vert_img + horiz_img\n", "show_center_recep_field(inp_img, horiz_img)"]}, {"cell_type": "markdown", "id": "a5e2e147", "metadata": {"papermill": {"duration": 0.010932, "end_time": "2022-05-03T01:16:14.885665", "exception": false, "start_time": "2022-05-03T01:16:14.874733", "status": "completed"}, "tags": []}, "source": ["If we stack multiple horizontal and vertical convolutions, we need to take two aspects into account:\n", "\n", "1.\n", "The center should not be masked anymore for the following convolutions as the features at the pixel's position are already independent of its actual value.\n", "If it is hard to imagine why we can do this, just change the value below to `mask_center=True` and see what happens.\n", "2.\n", "The vertical convolution is not allowed to work on features from the horizontal convolution.\n", "In the feature map of the horizontal convolutions, a pixel contains information about all of the \"true\" pixels on the left.\n", "If we apply a vertical convolution which also uses features from the right, we effectively expand our receptive field to the true input which we want to prevent.\n", "Thus, the feature maps can only be merged for the horizontal convolution.\n", "\n", "Using this, we can stack the convolutions in the following way. We have\n", "two feature streams: one for the vertical stack, and one for the\n", "horizontal stack. The horizontal convolutions can operate on the joint\n", "features of the previous horizontals and vertical convolutions, while\n", "the vertical stack only takes its own previous features as input. For a\n", "quick implementation, we can therefore sum the horizontal and vertical\n", "output features at each layer, and use those as final output features to\n", "calculate the loss on. An implementation of 4 consecutive layers is\n", "shown below. Note that we reuse the features from the other convolutions\n", "with `mask_center=True` from above."]}, {"cell_type": "code", "execution_count": 12, "id": "9a47802f", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:16:14.909833Z", "iopub.status.busy": "2022-05-03T01:16:14.909225Z", "iopub.status.idle": "2022-05-03T01:16:15.556948Z", "shell.execute_reply": "2022-05-03T01:16:15.556304Z"}, "papermill": {"duration": 0.662127, "end_time": "2022-05-03T01:16:15.558642", "exception": false, "start_time": "2022-05-03T01:16:14.896515", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Layer 2\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUiAvTWVkaWFCb3ggWyAwIDAgMzQ5LjIgMTgxLjcwNjgxODE4MTggXQovUGFyZW50IDIgMCBSIC9SZXNvdXJjZXMgOCAwIFIgL1R5cGUgL1BhZ2UgPj4KZW5kb2JqCjkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMiAwIFIgPj4Kc3RyZWFtCnicnVFLTwIxEL7Pr/iOcLA701K2PYIo0Ru6CQfjwSwFlgDKIxD/vbMKPvAidjLpzPSb19esl3ZVme76XVzeU/bllRsSzCjrCCYbMGaqewj6+AlijS/ItaKxas0PlgQxObeDBA3xqVvjpkRLWiFXcK3irVHUQU7ddcIQS2QdWw+jc6nutUxfK4h3qEvWd65SLpDdCHrPGNAAq2MSY/I9sfZppSsyLjQXwoad49j2kSOk7Y37al8uqFsgu1aYRTF+36AY0QMawyZ8MD663KORqsl0m0ZYN2HZBHc8+lKml221SxhXaT5q4hHFLV0V9DGhhGg+m/2bDvpNh0Q+EmLPJcSyfpm3ocVKiPydkW61fFq/nscBvQHN+pCiCmVuZHN0cmVhbQplbmRvYmoKMTIgMCBvYmoKMjgxCmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjE5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjY0ID4+CnN0cmVhbQp4nD1SuZEDMQzLtwqWwF9SPb65cWD3nx7APTsxMUsRAEH3blE5Gz9tJcuP/NhlZ0nklvegbfK6ouoGqZsgc8lxyRVilvK4EiQWLWUm1meqg/JxDWoVvvATMxPTIUnEEnJSjyKsj6+D1/Uc3F836fwaGR90bFB6S5KroFTDvnK0EjuW6l2D7IOmm9LqM9Ou7ICl8Yas1KROfx31vyMswqTeSEUHIRa/08sdsiykHNTbuX4D4V3tglhPrTzsEGneVnpLFczmzARMGW07rIC1WWPCVAnwFs0h1FqIzBc6rYj55NzRwTRVeRoiw1KFg+09ozVc6BoSS8P5ScYE5r/w+qzIbX//AHBoZ7oKZW5kc3RyZWFtCmVuZG9iagoyMCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDkyID4+CnN0cmVhbQp4nD2NwQ3AMAgD/0zBCBACxPtUVR/p/t8mEeoHHwbZGGBhszXgwdnAl9LaN72kRZPaCFa1Rd1QnrsUpVhdR6VMwk+ZO39SdBztcA7b39blOE3j6F/30P0BD0oeCwplbmRzdHJlYW0KZW5kb2JqCjIxIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzA3ID4+CnN0cmVhbQp4nD2SS24DMQxD9z6FLhDA+tme86Qoupjef9snJemKHNkWRWqWukxZUx6QNJOEf+nwcLGd8jtsz2Zm4Fqil4nllOfQFWLuonzZzEZdWSfF6oRmOrfoUTkXBzZNqp+rLKXdLngO1yaeW/YRP7zQoB7UNS4JN3RXo2UpNGOq+3/Se/yMMuBqTF1sUqt7HzxeRFXo6AdHiSJjlxfn40EJ6UrCaFqIlXdFA0Hu8rTKewnu295qyLIHqZjOOylmsOt0Ui5uF4chHsjyqPDlo9hrQs/4sCsl9EjYhjNyJ+5oxubUyOKQ/t6NBEuPrmgh8+CvbtYuYLxTOkViZE5yrGmLVU73UBTTucO9DBD1bEVDKXOR1epfw84La5ZsFnhK+gUeo90mSw5W2duoTu+tPNnQ9x9a13QfCmVuZHN0cmVhbQplbmRvYmoKMjIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMzIgPj4Kc3RyZWFtCnicNVFJbsQwDLv7FfzAANbuvCfFoIf2/9dSyhQIQCW2uCViYyMCLzH4OYjc+JI1oyZ+Z3JX/CxPhUfCreBJFIGX4V52gssbxmU/DjMfvJdWzqTGkwzIRTY9PBEy2CUQOjC7BnXYZtqJviHhsyNSzUaW09cS9NIqBMpTtt/pghJtq/pz+6wLbfvaE052e+pJ5ROI55aswGXjFZPFWAY9UblLMX2Q6myhJ6G8KJ+DbD5qiESXKGfgicHBKNAO7LntZ+JVIWhd3adtY6hGSsfTvw1NTZII+UQJZ7Y07hb+f8+9vtf7D04hVBEKZW5kc3RyZWFtCmVuZG9iagoyMyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIzMSA+PgpzdHJlYW0KeJw1TzmSBCEMy3mFPjBVGNtAv6entjbY+X+6kplOkPAhydMTHZl4mSMjsGbH21pkIGbgU0zFv/a0DxOq9+AeIpSLC2GGkXDWrONuno4X/3aVz1gH7zb4illeENjCTNZXFmcu2wVjaZzEOclujF0TsY11radTWEcwoQyEdLbDlCBzVKT0yY4y5ug4kSeei+/22yx2OX4O6ws2jSEV5/gqeoI2g6Lsee8CGnJB/13d+B5Fu+glIBsJFtZRYu6c5YRfvXZ0HrUoEnNCmkEuEyHN6SqmEJpQrLOjoFJRcKk+p+isn3/lX1wtCmVuZHN0cmVhbQplbmRvYmoKMjQgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNDkgPj4Kc3RyZWFtCnicPVA7jkQhDOs5hS/wJPIjcB5Gqy1m79+uA5opUEx+tjMk0BGBRwwxlK/jJa2groG/i0LxbuLrg8Igq0NSIM56D4h07KY2kRM6HZwzP2E3Y47ARTEGnOl0pj0HJjn7wgqEcxtl7FZIJ4mqIo7qM44pnip7n3gWLO3INlsnkj3kIOFSUonJpZ+Uyj9typQKOmbRBCwSueBkE004y7tJUowZlDLqHqZ2In2sPMijOuhkTc6sI5nZ00/bmfgccLdf2mROlcd0Hsz4nLTOgzkVuvfjiTYHTY3a6Oz3E2kqL1K7HVqdfnUSld0Y5xgSl2d/Gd9k//kH/odaIgplbmRzdHJlYW0KZW5kb2JqCjI1IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTM2ID4+CnN0cmVhbQp4nE2PQQ4DMQgD73mFn0AgQHjPVlUP2/9fS9h20wseyYBsUQaBJYd4hxvh0dsP30U2FWfjnF9SKWIhmE9wnzBTHI0pd/Jjj4BxlGosp2h4XkvOTcMXLXcTLaWtl5MZb7jul/dHlW2RDUXPLQtC12yS+TKBB3wYmEd142mlx932bK/2/ADObDRJCmVuZHN0cmVhbQplbmRvYmoKMjYgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzNDEgPj4Kc3RyZWFtCnicRVJLbkQxCNu/U3CBSOGXkPO0qrqY3n9bm0zVzeAJYGx4y1OmZMqwuSUjJNeUT30iQ6ym/DRyJCKm+EkJBXaVj8drS6yN7JGoFJ/a8eOx9Eam2RVa9e7Rpc2iUc3KyDnIEKGeFbqye9QO2fB6XEi675TNIRzL/1CBLGXdcgolQVvQd+wR3w8droIrgmGway6D7WUy1P/6hxZc7333YscugBas577BDgCopxO0BcgZ2u42KWgAVbqLScKj8npudqJso1Xp+RwAMw4wcsCIJVsdvtHeAJZ9XehFjYr9K0BRWUD8yNV2wd4xyUhwFuYGjr1wPMWZcEs4xgJAir3iGHrwJdjmL1euiJrwCXW6ZC+8wp7a5udCkwh3rQAOXmTDraujqJbt6TyC9mdFckaM1Is4OiGSWtI5guLSoB5a41w3seJtI7G5V9/uH+GcL1z26xdL7ITECmVuZHN0cmVhbQplbmRvYmoKMjcgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNjQgPj4Kc3RyZWFtCnicRZDHcQUxDEPvqgIlMIAK9azH8w/r/q+G9NNBehhCDGJPwrBcV3FhdMOPty0zDX9HGe7G+jJjvNVYICfoAwyRiavRpPp2xRmq9OTVYq6jolwvOiISzJLjq0AjfDqyx5O2tjP9dF4f7CHvE/8qKuduYQEuqu5A+VIf8dSP2VHqmqGPKitrHmraV4RdEUrbPi6nMk7dvQNa4b2Vqz3a7z8edjryCmVuZHN0cmVhbQplbmRvYmoKMjggMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA3MiA+PgpzdHJlYW0KeJwzMrdQMFCwNAEShhYmCuZmBgophlxAvqmJuUIuF0gMxMoBswyAtCWcgohngJggbRDFIBZEsZmJGUQdnAGRy+BKAwAl2xbJCmVuZHN0cmVhbQplbmRvYmoKMjkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA0NyA+PgpzdHJlYW0KeJwzMrdQMFCwNAEShhYmCuZmBgophlyWEFYuF0wsB8wC0ZZwCiKewZUGALlnDScKZW5kc3RyZWFtCmVuZG9iagozMCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE2MyA+PgpzdHJlYW0KeJxFkDsSAyEMQ3tOoSP4IwM+z2YyKTb3b2PYbFLA01ggg7sTgtTagonogoe2Jd0F760EZ2P86TZuNRLkBHWAVqTjaJRSfbnFaZV08Wg2cysLrRMdZg56lKMZoBA6Fd7touRypu7O+UNw9V/1v2LdOZuJgcnKHQjN6lPc+TY7orq6yf6kx9ys134r7FVhaVlLywm3nbtmQAncUznaqz0/Hwo69gplbmRzdHJlYW0KZW5kb2JqCjMxIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjM5ID4+CnN0cmVhbQp4nE1QyW0EMQz7uwo1MMDoHLseB4s8sv1/Q8oJkpdoS+Kh8pRblspl9yM5b8m65UOHTpVp8m7Qza+x/qMMAnb/UFQQrSWxSsxc0m6xNEkv2cM4jZdrtY7nqXuEWaN48OPY0ymB6T0ywWazvTkwqz3ODpBOuMav6tM7lSQDibqQ80KlCuse1CWijyvbmFKdTi3lGJef6Ht8jgA9xd6N3NHHyxeMRrUtqNFqlTgPMBNT0ZVxq5GBlBMGQ2dHVzQLpcjKekI1wo05oZm9w3BgA8uzhKSlrVK8D2UB6AJd2jrjNEqCjgDC3yiM9foGqvxeNwplbmRzdHJlYW0KZW5kb2JqCjMyIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTYwID4+CnN0cmVhbQp4nEWQORIDMQgEc72CJ0hcgvesy7XB+v+pB9ZHoukCNBy6Fk3KehRoPumxRqG60GvoLEqSRMEWkh1Qp2OIOyhITEhjkki2HoMjmlizXZiZVCqzUuG0acXCv9la1chEjXCN/InpBlT8T+pclPBNg6+SMfoYVLw7g4xJ+F5F3Fox7f5EMLEZ9glvRSYFhImxqdm+z2CGzPcK1zjH8w1MgjfrCmVuZHN0cmVhbQplbmRvYmoKMzMgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxOCA+PgpzdHJlYW0KeJwzNrRQMIDDFEOuNAAd5gNSCmVuZHN0cmVhbQplbmRvYmoKMzQgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMzMgPj4Kc3RyZWFtCnicRY9LDgQhCET3nKKOwMcf53Ey6YVz/+2AnW4TYz2FVIG5gqE9LmsDnRUfIRm28beplo5FWT5UelJWD8ngh6zGyyHcoCzwgkkqhiFQi5gakS1lbreA2zYNsrKVU6WOsIujMI/2tGwVHl+iWyJ1kj+DxCov3OO6Hcil1rveoou+f6QBMQkKZW5kc3RyZWFtCmVuZG9iagozNSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDc1ID4+CnN0cmVhbQp4nDO1NFIwUDA2ABKmZkYKpibmCimGXEA+iJXLZWhkCmblcBlZmilYWAAZJmbmUCGYhhwuY1NzoAFARcamYBqqP4crgysNAJWQEu8KZW5kc3RyZWFtCmVuZG9iagozNiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE0MSA+PgpzdHJlYW0KeJw9j8EOwzAIQ+/5Cv9ApNgpoXxPp2qH7v+vI0u7C3oCY4yF0NAbqprDhmCb48XSJVRr+BTFQCU3yJlgDqWk0h1HkXpiOBhcHrQbjuKx6PoRu5JmfdDGQrolaIB7rFNp3KZxE8QdNQXqKeqco7wQuZ+pZ9g0kt00s5JzuA2/e89T1/+nq7zL+QW9dy7+CmVuZHN0cmVhbQplbmRvYmoKMTcgMCBvYmoKPDwgL0Jhc2VGb250IC9CTVFRRFYrRGVqYVZ1U2FucyAvQ2hhclByb2NzIDE4IDAgUgovRW5jb2RpbmcgPDwKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCi9UeXBlIC9FbmNvZGluZyA+PgovRmlyc3RDaGFyIDAgL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvRm9udERlc2NyaXB0b3IgMTYgMCBSCi9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdIC9MYXN0Q2hhciAyNTUgL05hbWUgL0JNUVFEVitEZWphVnVTYW5zCi9TdWJ0eXBlIC9UeXBlMyAvVHlwZSAvRm9udCAvV2lkdGhzIDE1IDAgUiA+PgplbmRvYmoKMTYgMCBvYmoKPDwgL0FzY2VudCA5MjkgL0NhcEhlaWdodCAwIC9EZXNjZW50IC0yMzYgL0ZsYWdzIDMyCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovSXRhbGljQW5nbGUgMCAvTWF4V2lkdGggMTM0MiAvU3RlbVYgMCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL1hIZWlnaHQgMCA+PgplbmRvYmoKMTUgMCBvYmoKWyA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMAo2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDMxOCA0MDEgNDYwIDgzOCA2MzYKOTUwIDc4MCAyNzUgMzkwIDM5MCA1MDAgODM4IDMxOCAzNjEgMzE4IDMzNyA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2CjYzNiA2MzYgMzM3IDMzNyA4MzggODM4IDgzOCA1MzEgMTAwMCA2ODQgNjg2IDY5OCA3NzAgNjMyIDU3NSA3NzUgNzUyIDI5NQoyOTUgNjU2IDU1NyA4NjMgNzQ4IDc4NyA2MDMgNzg3IDY5NSA2MzUgNjExIDczMiA2ODQgOTg5IDY4NSA2MTEgNjg1IDM5MCAzMzcKMzkwIDgzOCA1MDAgNTAwIDYxMyA2MzUgNTUwIDYzNSA2MTUgMzUyIDYzNSA2MzQgMjc4IDI3OCA1NzkgMjc4IDk3NCA2MzQgNjEyCjYzNSA2MzUgNDExIDUyMSAzOTIgNjM0IDU5MiA4MTggNTkyIDU5MiA1MjUgNjM2IDMzNyA2MzYgODM4IDYwMCA2MzYgNjAwIDMxOAozNTIgNTE4IDEwMDAgNTAwIDUwMCA1MDAgMTM0MiA2MzUgNDAwIDEwNzAgNjAwIDY4NSA2MDAgNjAwIDMxOCAzMTggNTE4IDUxOAo1OTAgNTAwIDEwMDAgNTAwIDEwMDAgNTIxIDQwMCAxMDIzIDYwMCA1MjUgNjExIDMxOCA0MDEgNjM2IDYzNiA2MzYgNjM2IDMzNwo1MDAgNTAwIDEwMDAgNDcxIDYxMiA4MzggMzYxIDEwMDAgNTAwIDUwMCA4MzggNDAxIDQwMSA1MDAgNjM2IDYzNiAzMTggNTAwCjQwMSA0NzEgNjEyIDk2OSA5NjkgOTY5IDUzMSA2ODQgNjg0IDY4NCA2ODQgNjg0IDY4NCA5NzQgNjk4IDYzMiA2MzIgNjMyIDYzMgoyOTUgMjk1IDI5NSAyOTUgNzc1IDc0OCA3ODcgNzg3IDc4NyA3ODcgNzg3IDgzOCA3ODcgNzMyIDczMiA3MzIgNzMyIDYxMSA2MDUKNjMwIDYxMyA2MTMgNjEzIDYxMyA2MTMgNjEzIDk4MiA1NTAgNjE1IDYxNSA2MTUgNjE1IDI3OCAyNzggMjc4IDI3OCA2MTIgNjM0CjYxMiA2MTIgNjEyIDYxMiA2MTIgODM4IDYxMiA2MzQgNjM0IDYzNCA2MzQgNTkyIDYzNSA1OTIgXQplbmRvYmoKMTggMCBvYmoKPDwgL0IgMTkgMCBSIC9XIDIwIDAgUiAvYSAyMSAwIFIgL2MgMjIgMCBSIC9kIDIzIDAgUiAvZSAyNCAwIFIgL2YgMjUgMCBSCi9nIDI2IDAgUiAvaCAyNyAwIFIgL2kgMjggMCBSIC9sIDI5IDAgUiAvbiAzMCAwIFIgL3AgMzEgMCBSIC9yIDMyIDAgUgovc3BhY2UgMzMgMCBSIC90IDM0IDAgUiAvdiAzNSAwIFIgL3kgMzYgMCBSID4+CmVuZG9iagozIDAgb2JqCjw8IC9GMSAxNyAwIFIgPj4KZW5kb2JqCjQgMCBvYmoKPDwgL0ExIDw8IC9DQSAwIC9UeXBlIC9FeHRHU3RhdGUgL2NhIDEgPj4KL0EyIDw8IC9DQSAxIC9UeXBlIC9FeHRHU3RhdGUgL2NhIDEgPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgL0kxIDEzIDAgUiAvSTIgMTQgMCBSID4+CmVuZG9iagoxMyAwIG9iago8PCAvQml0c1BlckNvbXBvbmVudCA4Ci9Db2xvclNwYWNlIFsvSW5kZXhlZCAvRGV2aWNlUkdCIDYgKP3nN8i3ZZSOd2ZpcDVFbAAiTf8AACldCi9EZWNvZGVQYXJtcyA8PCAvQ29sb3JzIDEgL0NvbHVtbnMgMTUzIC9QcmVkaWN0b3IgMTAgPj4KL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0hlaWdodCAxNTMgL0xlbmd0aCAzNyAwIFIgL1N1YnR5cGUgL0ltYWdlCi9UeXBlIC9YT2JqZWN0IC9XaWR0aCAxNTMgPj4Kc3RyZWFtCnic7c4xEoIADABBRPT/T7YlBQ7XAMVem0myy+epLXcDDiPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemS9INtG79G6b47mGhkZGRkZGRnZQfPha7TsG8yVjIyMjIyMjOxU25+++86fJCMjIyMjI7tcdnFkPbIeWY+sR9Yj65H1yHpkPbIeWY+sR9Yj65H1yHpkPbIeWY+sR9Yj65H1yHpkPbIeWY+sR9Yj65H1yHpkPbLec2U/TFS2kgplbmRzdHJlYW0KZW5kb2JqCjM3IDAgb2JqCjE5NQplbmRvYmoKMTQgMCBvYmoKPDwgL0JpdHNQZXJDb21wb25lbnQgOCAvQ29sb3JTcGFjZSBbL0luZGV4ZWQgL0RldmljZVJHQiAyICj95zcAIk3/AAApXQovRGVjb2RlUGFybXMgPDwgL0NvbG9ycyAxIC9Db2x1bW5zIDE1MyAvUHJlZGljdG9yIDEwID4+Ci9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9IZWlnaHQgMTUzIC9MZW5ndGggMzggMCBSIC9TdWJ0eXBlIC9JbWFnZQovVHlwZSAvWE9iamVjdCAvV2lkdGggMTUzID4+CnN0cmVhbQp4nO3OwQmAQBDAwNP+i/brRzAg5z4mBYRZx9TW34DHyHpkPbIeWY+sR9Yj65H1yHpkPbIeWY+sR9Yj65H1yHpkPbIeWS/I1heRkZGRkZGRkZGRkZGRkZGRDZOd994vycjIyMjIyLbLNkfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+vNlV3DzVMjCmVuZHN0cmVhbQplbmRvYmoKMzggMCBvYmoKMTc1CmVuZG9iagoyIDAgb2JqCjw8IC9Db3VudCAxIC9LaWRzIFsgMTEgMCBSIF0gL1R5cGUgL1BhZ2VzID4+CmVuZG9iagozOSAwIG9iago8PCAvQ3JlYXRpb25EYXRlIChEOjIwMjIwNTAzMDMxNjE1KzAyJzAwJykKL0NyZWF0b3IgKE1hdHBsb3RsaWIgdjMuNS4xLCBodHRwczovL21hdHBsb3RsaWIub3JnKQovUHJvZHVjZXIgKE1hdHBsb3RsaWIgcGRmIGJhY2tlbmQgdjMuNS4xKSA+PgplbmRvYmoKeHJlZgowIDQwCjAwMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAwMDAxNiAwMDAwMCBuIAowMDAwMDA4MTMyIDAwMDAwIG4gCjAwMDAwMDY5ODYgMDAwMDAgbiAKMDAwMDAwNzAxOCAwMDAwMCBuIAowMDAwMDA3MTE3IDAwMDAwIG4gCjAwMDAwMDcxMzggMDAwMDAgbiAKMDAwMDAwNzE1OSAwMDAwMCBuIAowMDAwMDAwMDY1IDAwMDAwIG4gCjAwMDAwMDAzNDMgMDAwMDAgbiAKMDAwMDAwMDcxOSAwMDAwMCBuIAowMDAwMDAwMjA4IDAwMDAwIG4gCjAwMDAwMDA2OTkgMDAwMDAgbiAKMDAwMDAwNzIwMiAwMDAwMCBuIAowMDAwMDA3NjgzIDAwMDAwIG4gCjAwMDAwMDU3MjcgMDAwMDAgbiAKMDAwMDAwNTUyMCAwMDAwMCBuIAowMDAwMDA1MTEyIDAwMDAwIG4gCjAwMDAwMDY3ODAgMDAwMDAgbiAKMDAwMDAwMDczOSAwMDAwMCBuIAowMDAwMDAxMDc2IDAwMDAwIG4gCjAwMDAwMDEyNDAgMDAwMDAgbiAKMDAwMDAwMTYyMCAwMDAwMCBuIAowMDAwMDAxOTI1IDAwMDAwIG4gCjAwMDAwMDIyMjkgMDAwMDAgbiAKMDAwMDAwMjU1MSAwMDAwMCBuIAowMDAwMDAyNzYwIDAwMDAwIG4gCjAwMDAwMDMxNzQgMDAwMDAgbiAKMDAwMDAwMzQxMSAwMDAwMCBuIAowMDAwMDAzNTU1IDAwMDAwIG4gCjAwMDAwMDM2NzQgMDAwMDAgbiAKMDAwMDAwMzkxMCAwMDAwMCBuIAowMDAwMDA0MjIyIDAwMDAwIG4gCjAwMDAwMDQ0NTUgMDAwMDAgbiAKMDAwMDAwNDU0NSAwMDAwMCBuIAowMDAwMDA0NzUxIDAwMDAwIG4gCjAwMDAwMDQ4OTggMDAwMDAgbiAKMDAwMDAwNzY2MyAwMDAwMCBuIAowMDAwMDA4MTEyIDAwMDAwIG4gCjAwMDAwMDgxOTIgMDAwMDAgbiAKdHJhaWxlcgo8PCAvSW5mbyAzOSAwIFIgL1Jvb3QgMSAwIFIgL1NpemUgNDAgPj4Kc3RhcnR4cmVmCjgzNDkKJSVFT0YK\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2022-05-03T03:16:14.993975\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.5.1, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}, {"name": "stdout", "output_type": "stream", "text": ["Layer 3\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUiAvTWVkaWFCb3ggWyAwIDAgMzQ5LjIgMTgxLjcwNjgxODE4MTggXQovUGFyZW50IDIgMCBSIC9SZXNvdXJjZXMgOCAwIFIgL1R5cGUgL1BhZ2UgPj4KZW5kb2JqCjkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMiAwIFIgPj4Kc3RyZWFtCnicnVFLTwIxEL7Pr/iOcLA701K2PYIo0Ru6CQfjwSwFlgDKIxD/vbMKPvAidjLpzPSb19esl3ZVme76XVzeU/bllRsSzCjrCCYbMGaqewj6+AlijS/ItaKxas0PlgQxObeDBA3xqVvjpkRLWiFXcK3irVHUQU7ddcIQS2QdWw+jc6nutUxfK4h3qEvWd65SLpDdCHrPGNAAq2MSY/I9sfZppSsyLjQXwoad49j2kSOk7Y37al8uqFsgu1aYRTF+36AY0QMawyZ8MD663KORqsl0m0ZYN2HZBHc8+lKml221SxhXaT5q4hHFLV0V9DGhhGg+m/2bDvpNh0Q+EmLPJcSyfpm3ocVKiPydkW61fFq/nscBvQHN+pCiCmVuZHN0cmVhbQplbmRvYmoKMTIgMCBvYmoKMjgxCmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjE5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjY0ID4+CnN0cmVhbQp4nD1SuZEDMQzLtwqWwF9SPb65cWD3nx7APTsxMUsRAEH3blE5Gz9tJcuP/NhlZ0nklvegbfK6ouoGqZsgc8lxyRVilvK4EiQWLWUm1meqg/JxDWoVvvATMxPTIUnEEnJSjyKsj6+D1/Uc3F836fwaGR90bFB6S5KroFTDvnK0EjuW6l2D7IOmm9LqM9Ou7ICl8Yas1KROfx31vyMswqTeSEUHIRa/08sdsiykHNTbuX4D4V3tglhPrTzsEGneVnpLFczmzARMGW07rIC1WWPCVAnwFs0h1FqIzBc6rYj55NzRwTRVeRoiw1KFg+09ozVc6BoSS8P5ScYE5r/w+qzIbX//AHBoZ7oKZW5kc3RyZWFtCmVuZG9iagoyMCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDkyID4+CnN0cmVhbQp4nD2NwQ3AMAgD/0zBCBACxPtUVR/p/t8mEeoHHwbZGGBhszXgwdnAl9LaN72kRZPaCFa1Rd1QnrsUpVhdR6VMwk+ZO39SdBztcA7b39blOE3j6F/30P0BD0oeCwplbmRzdHJlYW0KZW5kb2JqCjIxIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzA3ID4+CnN0cmVhbQp4nD2SS24DMQxD9z6FLhDA+tme86Qoupjef9snJemKHNkWRWqWukxZUx6QNJOEf+nwcLGd8jtsz2Zm4Fqil4nllOfQFWLuonzZzEZdWSfF6oRmOrfoUTkXBzZNqp+rLKXdLngO1yaeW/YRP7zQoB7UNS4JN3RXo2UpNGOq+3/Se/yMMuBqTF1sUqt7HzxeRFXo6AdHiSJjlxfn40EJ6UrCaFqIlXdFA0Hu8rTKewnu295qyLIHqZjOOylmsOt0Ui5uF4chHsjyqPDlo9hrQs/4sCsl9EjYhjNyJ+5oxubUyOKQ/t6NBEuPrmgh8+CvbtYuYLxTOkViZE5yrGmLVU73UBTTucO9DBD1bEVDKXOR1epfw84La5ZsFnhK+gUeo90mSw5W2duoTu+tPNnQ9x9a13QfCmVuZHN0cmVhbQplbmRvYmoKMjIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMzIgPj4Kc3RyZWFtCnicNVFJbsQwDLv7FfzAANbuvCfFoIf2/9dSyhQIQCW2uCViYyMCLzH4OYjc+JI1oyZ+Z3JX/CxPhUfCreBJFIGX4V52gssbxmU/DjMfvJdWzqTGkwzIRTY9PBEy2CUQOjC7BnXYZtqJviHhsyNSzUaW09cS9NIqBMpTtt/pghJtq/pz+6wLbfvaE052e+pJ5ROI55aswGXjFZPFWAY9UblLMX2Q6myhJ6G8KJ+DbD5qiESXKGfgicHBKNAO7LntZ+JVIWhd3adtY6hGSsfTvw1NTZII+UQJZ7Y07hb+f8+9vtf7D04hVBEKZW5kc3RyZWFtCmVuZG9iagoyMyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIzMSA+PgpzdHJlYW0KeJw1TzmSBCEMy3mFPjBVGNtAv6entjbY+X+6kplOkPAhydMTHZl4mSMjsGbH21pkIGbgU0zFv/a0DxOq9+AeIpSLC2GGkXDWrONuno4X/3aVz1gH7zb4illeENjCTNZXFmcu2wVjaZzEOclujF0TsY11radTWEcwoQyEdLbDlCBzVKT0yY4y5ug4kSeei+/22yx2OX4O6ws2jSEV5/gqeoI2g6Lsee8CGnJB/13d+B5Fu+glIBsJFtZRYu6c5YRfvXZ0HrUoEnNCmkEuEyHN6SqmEJpQrLOjoFJRcKk+p+isn3/lX1wtCmVuZHN0cmVhbQplbmRvYmoKMjQgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNDkgPj4Kc3RyZWFtCnicPVA7jkQhDOs5hS/wJPIjcB5Gqy1m79+uA5opUEx+tjMk0BGBRwwxlK/jJa2groG/i0LxbuLrg8Igq0NSIM56D4h07KY2kRM6HZwzP2E3Y47ARTEGnOl0pj0HJjn7wgqEcxtl7FZIJ4mqIo7qM44pnip7n3gWLO3INlsnkj3kIOFSUonJpZ+Uyj9typQKOmbRBCwSueBkE004y7tJUowZlDLqHqZ2In2sPMijOuhkTc6sI5nZ00/bmfgccLdf2mROlcd0Hsz4nLTOgzkVuvfjiTYHTY3a6Oz3E2kqL1K7HVqdfnUSld0Y5xgSl2d/Gd9k//kH/odaIgplbmRzdHJlYW0KZW5kb2JqCjI1IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTM2ID4+CnN0cmVhbQp4nE2PQQ4DMQgD73mFn0AgQHjPVlUP2/9fS9h20wseyYBsUQaBJYd4hxvh0dsP30U2FWfjnF9SKWIhmE9wnzBTHI0pd/Jjj4BxlGosp2h4XkvOTcMXLXcTLaWtl5MZb7jul/dHlW2RDUXPLQtC12yS+TKBB3wYmEd142mlx932bK/2/ADObDRJCmVuZHN0cmVhbQplbmRvYmoKMjYgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzNDEgPj4Kc3RyZWFtCnicRVJLbkQxCNu/U3CBSOGXkPO0qrqY3n9bm0zVzeAJYGx4y1OmZMqwuSUjJNeUT30iQ6ym/DRyJCKm+EkJBXaVj8drS6yN7JGoFJ/a8eOx9Eam2RVa9e7Rpc2iUc3KyDnIEKGeFbqye9QO2fB6XEi675TNIRzL/1CBLGXdcgolQVvQd+wR3w8droIrgmGway6D7WUy1P/6hxZc7333YscugBas577BDgCopxO0BcgZ2u42KWgAVbqLScKj8npudqJso1Xp+RwAMw4wcsCIJVsdvtHeAJZ9XehFjYr9K0BRWUD8yNV2wd4xyUhwFuYGjr1wPMWZcEs4xgJAir3iGHrwJdjmL1euiJrwCXW6ZC+8wp7a5udCkwh3rQAOXmTDraujqJbt6TyC9mdFckaM1Is4OiGSWtI5guLSoB5a41w3seJtI7G5V9/uH+GcL1z26xdL7ITECmVuZHN0cmVhbQplbmRvYmoKMjcgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNjQgPj4Kc3RyZWFtCnicRZDHcQUxDEPvqgIlMIAK9azH8w/r/q+G9NNBehhCDGJPwrBcV3FhdMOPty0zDX9HGe7G+jJjvNVYICfoAwyRiavRpPp2xRmq9OTVYq6jolwvOiISzJLjq0AjfDqyx5O2tjP9dF4f7CHvE/8qKuduYQEuqu5A+VIf8dSP2VHqmqGPKitrHmraV4RdEUrbPi6nMk7dvQNa4b2Vqz3a7z8edjryCmVuZHN0cmVhbQplbmRvYmoKMjggMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA3MiA+PgpzdHJlYW0KeJwzMrdQMFCwNAEShhYmCuZmBgophlxAvqmJuUIuF0gMxMoBswyAtCWcgohngJggbRDFIBZEsZmJGUQdnAGRy+BKAwAl2xbJCmVuZHN0cmVhbQplbmRvYmoKMjkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA0NyA+PgpzdHJlYW0KeJwzMrdQMFCwNAEShhYmCuZmBgophlyWEFYuF0wsB8wC0ZZwCiKewZUGALlnDScKZW5kc3RyZWFtCmVuZG9iagozMCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE2MyA+PgpzdHJlYW0KeJxFkDsSAyEMQ3tOoSP4IwM+z2YyKTb3b2PYbFLA01ggg7sTgtTagonogoe2Jd0F760EZ2P86TZuNRLkBHWAVqTjaJRSfbnFaZV08Wg2cysLrRMdZg56lKMZoBA6Fd7touRypu7O+UNw9V/1v2LdOZuJgcnKHQjN6lPc+TY7orq6yf6kx9ys134r7FVhaVlLywm3nbtmQAncUznaqz0/Hwo69gplbmRzdHJlYW0KZW5kb2JqCjMxIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjM5ID4+CnN0cmVhbQp4nE1QyW0EMQz7uwo1MMDoHLseB4s8sv1/Q8oJkpdoS+Kh8pRblspl9yM5b8m65UOHTpVp8m7Qza+x/qMMAnb/UFQQrSWxSsxc0m6xNEkv2cM4jZdrtY7nqXuEWaN48OPY0ymB6T0ywWazvTkwqz3ODpBOuMav6tM7lSQDibqQ80KlCuse1CWijyvbmFKdTi3lGJef6Ht8jgA9xd6N3NHHyxeMRrUtqNFqlTgPMBNT0ZVxq5GBlBMGQ2dHVzQLpcjKekI1wo05oZm9w3BgA8uzhKSlrVK8D2UB6AJd2jrjNEqCjgDC3yiM9foGqvxeNwplbmRzdHJlYW0KZW5kb2JqCjMyIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTYwID4+CnN0cmVhbQp4nEWQORIDMQgEc72CJ0hcgvesy7XB+v+pB9ZHoukCNBy6Fk3KehRoPumxRqG60GvoLEqSRMEWkh1Qp2OIOyhITEhjkki2HoMjmlizXZiZVCqzUuG0acXCv9la1chEjXCN/InpBlT8T+pclPBNg6+SMfoYVLw7g4xJ+F5F3Fox7f5EMLEZ9glvRSYFhImxqdm+z2CGzPcK1zjH8w1MgjfrCmVuZHN0cmVhbQplbmRvYmoKMzMgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxOCA+PgpzdHJlYW0KeJwzNrRQMIDDFEOuNAAd5gNSCmVuZHN0cmVhbQplbmRvYmoKMzQgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMzMgPj4Kc3RyZWFtCnicRY9LDgQhCET3nKKOwMcf53Ey6YVz/+2AnW4TYz2FVIG5gqE9LmsDnRUfIRm28beplo5FWT5UelJWD8ngh6zGyyHcoCzwgkkqhiFQi5gakS1lbreA2zYNsrKVU6WOsIujMI/2tGwVHl+iWyJ1kj+DxCov3OO6Hcil1rveoou+f6QBMQkKZW5kc3RyZWFtCmVuZG9iagozNSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDc1ID4+CnN0cmVhbQp4nDO1NFIwUDA2ABKmZkYKpibmCimGXEA+iJXLZWhkCmblcBlZmilYWAAZJmbmUCGYhhwuY1NzoAFARcamYBqqP4crgysNAJWQEu8KZW5kc3RyZWFtCmVuZG9iagozNiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE0MSA+PgpzdHJlYW0KeJw9j8EOwzAIQ+/5Cv9ApNgpoXxPp2qH7v+vI0u7C3oCY4yF0NAbqprDhmCb48XSJVRr+BTFQCU3yJlgDqWk0h1HkXpiOBhcHrQbjuKx6PoRu5JmfdDGQrolaIB7rFNp3KZxE8QdNQXqKeqco7wQuZ+pZ9g0kt00s5JzuA2/e89T1/+nq7zL+QW9dy7+CmVuZHN0cmVhbQplbmRvYmoKMTcgMCBvYmoKPDwgL0Jhc2VGb250IC9CTVFRRFYrRGVqYVZ1U2FucyAvQ2hhclByb2NzIDE4IDAgUgovRW5jb2RpbmcgPDwKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCi9UeXBlIC9FbmNvZGluZyA+PgovRmlyc3RDaGFyIDAgL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvRm9udERlc2NyaXB0b3IgMTYgMCBSCi9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdIC9MYXN0Q2hhciAyNTUgL05hbWUgL0JNUVFEVitEZWphVnVTYW5zCi9TdWJ0eXBlIC9UeXBlMyAvVHlwZSAvRm9udCAvV2lkdGhzIDE1IDAgUiA+PgplbmRvYmoKMTYgMCBvYmoKPDwgL0FzY2VudCA5MjkgL0NhcEhlaWdodCAwIC9EZXNjZW50IC0yMzYgL0ZsYWdzIDMyCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovSXRhbGljQW5nbGUgMCAvTWF4V2lkdGggMTM0MiAvU3RlbVYgMCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL1hIZWlnaHQgMCA+PgplbmRvYmoKMTUgMCBvYmoKWyA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMAo2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDMxOCA0MDEgNDYwIDgzOCA2MzYKOTUwIDc4MCAyNzUgMzkwIDM5MCA1MDAgODM4IDMxOCAzNjEgMzE4IDMzNyA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2CjYzNiA2MzYgMzM3IDMzNyA4MzggODM4IDgzOCA1MzEgMTAwMCA2ODQgNjg2IDY5OCA3NzAgNjMyIDU3NSA3NzUgNzUyIDI5NQoyOTUgNjU2IDU1NyA4NjMgNzQ4IDc4NyA2MDMgNzg3IDY5NSA2MzUgNjExIDczMiA2ODQgOTg5IDY4NSA2MTEgNjg1IDM5MCAzMzcKMzkwIDgzOCA1MDAgNTAwIDYxMyA2MzUgNTUwIDYzNSA2MTUgMzUyIDYzNSA2MzQgMjc4IDI3OCA1NzkgMjc4IDk3NCA2MzQgNjEyCjYzNSA2MzUgNDExIDUyMSAzOTIgNjM0IDU5MiA4MTggNTkyIDU5MiA1MjUgNjM2IDMzNyA2MzYgODM4IDYwMCA2MzYgNjAwIDMxOAozNTIgNTE4IDEwMDAgNTAwIDUwMCA1MDAgMTM0MiA2MzUgNDAwIDEwNzAgNjAwIDY4NSA2MDAgNjAwIDMxOCAzMTggNTE4IDUxOAo1OTAgNTAwIDEwMDAgNTAwIDEwMDAgNTIxIDQwMCAxMDIzIDYwMCA1MjUgNjExIDMxOCA0MDEgNjM2IDYzNiA2MzYgNjM2IDMzNwo1MDAgNTAwIDEwMDAgNDcxIDYxMiA4MzggMzYxIDEwMDAgNTAwIDUwMCA4MzggNDAxIDQwMSA1MDAgNjM2IDYzNiAzMTggNTAwCjQwMSA0NzEgNjEyIDk2OSA5NjkgOTY5IDUzMSA2ODQgNjg0IDY4NCA2ODQgNjg0IDY4NCA5NzQgNjk4IDYzMiA2MzIgNjMyIDYzMgoyOTUgMjk1IDI5NSAyOTUgNzc1IDc0OCA3ODcgNzg3IDc4NyA3ODcgNzg3IDgzOCA3ODcgNzMyIDczMiA3MzIgNzMyIDYxMSA2MDUKNjMwIDYxMyA2MTMgNjEzIDYxMyA2MTMgNjEzIDk4MiA1NTAgNjE1IDYxNSA2MTUgNjE1IDI3OCAyNzggMjc4IDI3OCA2MTIgNjM0CjYxMiA2MTIgNjEyIDYxMiA2MTIgODM4IDYxMiA2MzQgNjM0IDYzNCA2MzQgNTkyIDYzNSA1OTIgXQplbmRvYmoKMTggMCBvYmoKPDwgL0IgMTkgMCBSIC9XIDIwIDAgUiAvYSAyMSAwIFIgL2MgMjIgMCBSIC9kIDIzIDAgUiAvZSAyNCAwIFIgL2YgMjUgMCBSCi9nIDI2IDAgUiAvaCAyNyAwIFIgL2kgMjggMCBSIC9sIDI5IDAgUiAvbiAzMCAwIFIgL3AgMzEgMCBSIC9yIDMyIDAgUgovc3BhY2UgMzMgMCBSIC90IDM0IDAgUiAvdiAzNSAwIFIgL3kgMzYgMCBSID4+CmVuZG9iagozIDAgb2JqCjw8IC9GMSAxNyAwIFIgPj4KZW5kb2JqCjQgMCBvYmoKPDwgL0ExIDw8IC9DQSAwIC9UeXBlIC9FeHRHU3RhdGUgL2NhIDEgPj4KL0EyIDw8IC9DQSAxIC9UeXBlIC9FeHRHU3RhdGUgL2NhIDEgPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgL0kxIDEzIDAgUiAvSTIgMTQgMCBSID4+CmVuZG9iagoxMyAwIG9iago8PCAvQml0c1BlckNvbXBvbmVudCA4Ci9Db2xvclNwYWNlIFsvSW5kZXhlZCAvRGV2aWNlUkdCIDEyICj95zfjzVLGtWeCgHh2dnZfY25SWWw3RmwnPW0LM3AAK2QAIk3/AAApXQovRGVjb2RlUGFybXMgPDwgL0NvbG9ycyAxIC9Db2x1bW5zIDE1MyAvUHJlZGljdG9yIDEwID4+Ci9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9IZWlnaHQgMTUzIC9MZW5ndGggMzcgMCBSIC9TdWJ0eXBlIC9JbWFnZQovVHlwZSAvWE9iamVjdCAvV2lkdGggMTUzID4+CnN0cmVhbQp4nO3WO27DQBAFQepryfb9z6uUE5BwA4Ilw9XpYB8q3OXrXVteDdiMrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6u7LP0W10HV3WzdN8NifJyMjIyMjI/rVsjp5Hh9Gy7jga6Mt9REZGRkZGRka2ITtudxp9jH7hF0RGRkZGRkb2V2RzZn5g5u173d7kzyMjIyMjIyN7kuylkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIeu8rewC3r5bCCmVuZHN0cmVhbQplbmRvYmoKMzcgMCBvYmoKMjIzCmVuZG9iagoxNCAwIG9iago8PCAvQml0c1BlckNvbXBvbmVudCA4IC9Db2xvclNwYWNlIFsvSW5kZXhlZCAvRGV2aWNlUkdCIDIgKP3nNwAiTf8AACldCi9EZWNvZGVQYXJtcyA8PCAvQ29sb3JzIDEgL0NvbHVtbnMgMTUzIC9QcmVkaWN0b3IgMTAgPj4KL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0hlaWdodCAxNTMgL0xlbmd0aCAzOCAwIFIgL1N1YnR5cGUgL0ltYWdlCi9UeXBlIC9YT2JqZWN0IC9XaWR0aCAxNTMgPj4Kc3RyZWFtCnic7c7BCYAwAMDA6v5D+/WjGFBa5DJAuLGt2pgNuIysR9Yj65H1yHpkPbIeWY+sR9Yj65H1yHq3svF1ZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkv5bt5+6WzyMjIyMjIyN7STY1sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zb13ZAYviSg8KZW5kc3RyZWFtCmVuZG9iagozOCAwIG9iagoxNzIKZW5kb2JqCjIgMCBvYmoKPDwgL0NvdW50IDEgL0tpZHMgWyAxMSAwIFIgXSAvVHlwZSAvUGFnZXMgPj4KZW5kb2JqCjM5IDAgb2JqCjw8IC9DcmVhdGlvbkRhdGUgKEQ6MjAyMjA1MDMwMzE2MTUrMDInMDAnKQovQ3JlYXRvciAoTWF0cGxvdGxpYiB2My41LjEsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcpCi9Qcm9kdWNlciAoTWF0cGxvdGxpYiBwZGYgYmFja2VuZCB2My41LjEpID4+CmVuZG9iagp4cmVmCjAgNDAKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDgxNzYgMDAwMDAgbiAKMDAwMDAwNjk4NiAwMDAwMCBuIAowMDAwMDA3MDE4IDAwMDAwIG4gCjAwMDAwMDcxMTcgMDAwMDAgbiAKMDAwMDAwNzEzOCAwMDAwMCBuIAowMDAwMDA3MTU5IDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM0MyAwMDAwMCBuIAowMDAwMDAwNzE5IDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDY5OSAwMDAwMCBuIAowMDAwMDA3MjAyIDAwMDAwIG4gCjAwMDAwMDc3MzAgMDAwMDAgbiAKMDAwMDAwNTcyNyAwMDAwMCBuIAowMDAwMDA1NTIwIDAwMDAwIG4gCjAwMDAwMDUxMTIgMDAwMDAgbiAKMDAwMDAwNjc4MCAwMDAwMCBuIAowMDAwMDAwNzM5IDAwMDAwIG4gCjAwMDAwMDEwNzYgMDAwMDAgbiAKMDAwMDAwMTI0MCAwMDAwMCBuIAowMDAwMDAxNjIwIDAwMDAwIG4gCjAwMDAwMDE5MjUgMDAwMDAgbiAKMDAwMDAwMjIyOSAwMDAwMCBuIAowMDAwMDAyNTUxIDAwMDAwIG4gCjAwMDAwMDI3NjAgMDAwMDAgbiAKMDAwMDAwMzE3NCAwMDAwMCBuIAowMDAwMDAzNDExIDAwMDAwIG4gCjAwMDAwMDM1NTUgMDAwMDAgbiAKMDAwMDAwMzY3NCAwMDAwMCBuIAowMDAwMDAzOTEwIDAwMDAwIG4gCjAwMDAwMDQyMjIgMDAwMDAgbiAKMDAwMDAwNDQ1NSAwMDAwMCBuIAowMDAwMDA0NTQ1IDAwMDAwIG4gCjAwMDAwMDQ3NTEgMDAwMDAgbiAKMDAwMDAwNDg5OCAwMDAwMCBuIAowMDAwMDA3NzEwIDAwMDAwIG4gCjAwMDAwMDgxNTYgMDAwMDAgbiAKMDAwMDAwODIzNiAwMDAwMCBuIAp0cmFpbGVyCjw8IC9JbmZvIDM5IDAgUiAvUm9vdCAxIDAgUiAvU2l6ZSA0MCA+PgpzdGFydHhyZWYKODM5MwolJUVPRgo=\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2022-05-03T03:16:15.156550\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.5.1, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}, {"name": "stdout", "output_type": "stream", "text": ["Layer 4\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUiAvTWVkaWFCb3ggWyAwIDAgMzQ5LjIgMTgxLjcwNjgxODE4MTggXQovUGFyZW50IDIgMCBSIC9SZXNvdXJjZXMgOCAwIFIgL1R5cGUgL1BhZ2UgPj4KZW5kb2JqCjkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMiAwIFIgPj4Kc3RyZWFtCnicnVFLTwIxEL7Pr/iOcLA701K2PYIo0Ru6CQfjwSwFlgDKIxD/vbMKPvAidjLpzPSb19esl3ZVme76XVzeU/bllRsSzCjrCCYbMGaqewj6+AlijS/ItaKxas0PlgQxObeDBA3xqVvjpkRLWiFXcK3irVHUQU7ddcIQS2QdWw+jc6nutUxfK4h3qEvWd65SLpDdCHrPGNAAq2MSY/I9sfZppSsyLjQXwoad49j2kSOk7Y37al8uqFsgu1aYRTF+36AY0QMawyZ8MD663KORqsl0m0ZYN2HZBHc8+lKml221SxhXaT5q4hHFLV0V9DGhhGg+m/2bDvpNh0Q+EmLPJcSyfpm3ocVKiPydkW61fFq/nscBvQHN+pCiCmVuZHN0cmVhbQplbmRvYmoKMTIgMCBvYmoKMjgxCmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjE5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjY0ID4+CnN0cmVhbQp4nD1SuZEDMQzLtwqWwF9SPb65cWD3nx7APTsxMUsRAEH3blE5Gz9tJcuP/NhlZ0nklvegbfK6ouoGqZsgc8lxyRVilvK4EiQWLWUm1meqg/JxDWoVvvATMxPTIUnEEnJSjyKsj6+D1/Uc3F836fwaGR90bFB6S5KroFTDvnK0EjuW6l2D7IOmm9LqM9Ou7ICl8Yas1KROfx31vyMswqTeSEUHIRa/08sdsiykHNTbuX4D4V3tglhPrTzsEGneVnpLFczmzARMGW07rIC1WWPCVAnwFs0h1FqIzBc6rYj55NzRwTRVeRoiw1KFg+09ozVc6BoSS8P5ScYE5r/w+qzIbX//AHBoZ7oKZW5kc3RyZWFtCmVuZG9iagoyMCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDkyID4+CnN0cmVhbQp4nD2NwQ3AMAgD/0zBCBACxPtUVR/p/t8mEeoHHwbZGGBhszXgwdnAl9LaN72kRZPaCFa1Rd1QnrsUpVhdR6VMwk+ZO39SdBztcA7b39blOE3j6F/30P0BD0oeCwplbmRzdHJlYW0KZW5kb2JqCjIxIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzA3ID4+CnN0cmVhbQp4nD2SS24DMQxD9z6FLhDA+tme86Qoupjef9snJemKHNkWRWqWukxZUx6QNJOEf+nwcLGd8jtsz2Zm4Fqil4nllOfQFWLuonzZzEZdWSfF6oRmOrfoUTkXBzZNqp+rLKXdLngO1yaeW/YRP7zQoB7UNS4JN3RXo2UpNGOq+3/Se/yMMuBqTF1sUqt7HzxeRFXo6AdHiSJjlxfn40EJ6UrCaFqIlXdFA0Hu8rTKewnu295qyLIHqZjOOylmsOt0Ui5uF4chHsjyqPDlo9hrQs/4sCsl9EjYhjNyJ+5oxubUyOKQ/t6NBEuPrmgh8+CvbtYuYLxTOkViZE5yrGmLVU73UBTTucO9DBD1bEVDKXOR1epfw84La5ZsFnhK+gUeo90mSw5W2duoTu+tPNnQ9x9a13QfCmVuZHN0cmVhbQplbmRvYmoKMjIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMzIgPj4Kc3RyZWFtCnicNVFJbsQwDLv7FfzAANbuvCfFoIf2/9dSyhQIQCW2uCViYyMCLzH4OYjc+JI1oyZ+Z3JX/CxPhUfCreBJFIGX4V52gssbxmU/DjMfvJdWzqTGkwzIRTY9PBEy2CUQOjC7BnXYZtqJviHhsyNSzUaW09cS9NIqBMpTtt/pghJtq/pz+6wLbfvaE052e+pJ5ROI55aswGXjFZPFWAY9UblLMX2Q6myhJ6G8KJ+DbD5qiESXKGfgicHBKNAO7LntZ+JVIWhd3adtY6hGSsfTvw1NTZII+UQJZ7Y07hb+f8+9vtf7D04hVBEKZW5kc3RyZWFtCmVuZG9iagoyMyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIzMSA+PgpzdHJlYW0KeJw1TzmSBCEMy3mFPjBVGNtAv6entjbY+X+6kplOkPAhydMTHZl4mSMjsGbH21pkIGbgU0zFv/a0DxOq9+AeIpSLC2GGkXDWrONuno4X/3aVz1gH7zb4illeENjCTNZXFmcu2wVjaZzEOclujF0TsY11radTWEcwoQyEdLbDlCBzVKT0yY4y5ug4kSeei+/22yx2OX4O6ws2jSEV5/gqeoI2g6Lsee8CGnJB/13d+B5Fu+glIBsJFtZRYu6c5YRfvXZ0HrUoEnNCmkEuEyHN6SqmEJpQrLOjoFJRcKk+p+isn3/lX1wtCmVuZHN0cmVhbQplbmRvYmoKMjQgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNDkgPj4Kc3RyZWFtCnicPVA7jkQhDOs5hS/wJPIjcB5Gqy1m79+uA5opUEx+tjMk0BGBRwwxlK/jJa2groG/i0LxbuLrg8Igq0NSIM56D4h07KY2kRM6HZwzP2E3Y47ARTEGnOl0pj0HJjn7wgqEcxtl7FZIJ4mqIo7qM44pnip7n3gWLO3INlsnkj3kIOFSUonJpZ+Uyj9typQKOmbRBCwSueBkE004y7tJUowZlDLqHqZ2In2sPMijOuhkTc6sI5nZ00/bmfgccLdf2mROlcd0Hsz4nLTOgzkVuvfjiTYHTY3a6Oz3E2kqL1K7HVqdfnUSld0Y5xgSl2d/Gd9k//kH/odaIgplbmRzdHJlYW0KZW5kb2JqCjI1IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTM2ID4+CnN0cmVhbQp4nE2PQQ4DMQgD73mFn0AgQHjPVlUP2/9fS9h20wseyYBsUQaBJYd4hxvh0dsP30U2FWfjnF9SKWIhmE9wnzBTHI0pd/Jjj4BxlGosp2h4XkvOTcMXLXcTLaWtl5MZb7jul/dHlW2RDUXPLQtC12yS+TKBB3wYmEd142mlx932bK/2/ADObDRJCmVuZHN0cmVhbQplbmRvYmoKMjYgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzNDEgPj4Kc3RyZWFtCnicRVJLbkQxCNu/U3CBSOGXkPO0qrqY3n9bm0zVzeAJYGx4y1OmZMqwuSUjJNeUT30iQ6ym/DRyJCKm+EkJBXaVj8drS6yN7JGoFJ/a8eOx9Eam2RVa9e7Rpc2iUc3KyDnIEKGeFbqye9QO2fB6XEi675TNIRzL/1CBLGXdcgolQVvQd+wR3w8droIrgmGway6D7WUy1P/6hxZc7333YscugBas577BDgCopxO0BcgZ2u42KWgAVbqLScKj8npudqJso1Xp+RwAMw4wcsCIJVsdvtHeAJZ9XehFjYr9K0BRWUD8yNV2wd4xyUhwFuYGjr1wPMWZcEs4xgJAir3iGHrwJdjmL1euiJrwCXW6ZC+8wp7a5udCkwh3rQAOXmTDraujqJbt6TyC9mdFckaM1Is4OiGSWtI5guLSoB5a41w3seJtI7G5V9/uH+GcL1z26xdL7ITECmVuZHN0cmVhbQplbmRvYmoKMjcgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNjQgPj4Kc3RyZWFtCnicRZDHcQUxDEPvqgIlMIAK9azH8w/r/q+G9NNBehhCDGJPwrBcV3FhdMOPty0zDX9HGe7G+jJjvNVYICfoAwyRiavRpPp2xRmq9OTVYq6jolwvOiISzJLjq0AjfDqyx5O2tjP9dF4f7CHvE/8qKuduYQEuqu5A+VIf8dSP2VHqmqGPKitrHmraV4RdEUrbPi6nMk7dvQNa4b2Vqz3a7z8edjryCmVuZHN0cmVhbQplbmRvYmoKMjggMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA3MiA+PgpzdHJlYW0KeJwzMrdQMFCwNAEShhYmCuZmBgophlxAvqmJuUIuF0gMxMoBswyAtCWcgohngJggbRDFIBZEsZmJGUQdnAGRy+BKAwAl2xbJCmVuZHN0cmVhbQplbmRvYmoKMjkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA0NyA+PgpzdHJlYW0KeJwzMrdQMFCwNAEShhYmCuZmBgophlyWEFYuF0wsB8wC0ZZwCiKewZUGALlnDScKZW5kc3RyZWFtCmVuZG9iagozMCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE2MyA+PgpzdHJlYW0KeJxFkDsSAyEMQ3tOoSP4IwM+z2YyKTb3b2PYbFLA01ggg7sTgtTagonogoe2Jd0F760EZ2P86TZuNRLkBHWAVqTjaJRSfbnFaZV08Wg2cysLrRMdZg56lKMZoBA6Fd7touRypu7O+UNw9V/1v2LdOZuJgcnKHQjN6lPc+TY7orq6yf6kx9ys134r7FVhaVlLywm3nbtmQAncUznaqz0/Hwo69gplbmRzdHJlYW0KZW5kb2JqCjMxIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjM5ID4+CnN0cmVhbQp4nE1QyW0EMQz7uwo1MMDoHLseB4s8sv1/Q8oJkpdoS+Kh8pRblspl9yM5b8m65UOHTpVp8m7Qza+x/qMMAnb/UFQQrSWxSsxc0m6xNEkv2cM4jZdrtY7nqXuEWaN48OPY0ymB6T0ywWazvTkwqz3ODpBOuMav6tM7lSQDibqQ80KlCuse1CWijyvbmFKdTi3lGJef6Ht8jgA9xd6N3NHHyxeMRrUtqNFqlTgPMBNT0ZVxq5GBlBMGQ2dHVzQLpcjKekI1wo05oZm9w3BgA8uzhKSlrVK8D2UB6AJd2jrjNEqCjgDC3yiM9foGqvxeNwplbmRzdHJlYW0KZW5kb2JqCjMyIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTYwID4+CnN0cmVhbQp4nEWQORIDMQgEc72CJ0hcgvesy7XB+v+pB9ZHoukCNBy6Fk3KehRoPumxRqG60GvoLEqSRMEWkh1Qp2OIOyhITEhjkki2HoMjmlizXZiZVCqzUuG0acXCv9la1chEjXCN/InpBlT8T+pclPBNg6+SMfoYVLw7g4xJ+F5F3Fox7f5EMLEZ9glvRSYFhImxqdm+z2CGzPcK1zjH8w1MgjfrCmVuZHN0cmVhbQplbmRvYmoKMzMgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxOCA+PgpzdHJlYW0KeJwzNrRQMIDDFEOuNAAd5gNSCmVuZHN0cmVhbQplbmRvYmoKMzQgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMzMgPj4Kc3RyZWFtCnicRY9LDgQhCET3nKKOwMcf53Ey6YVz/+2AnW4TYz2FVIG5gqE9LmsDnRUfIRm28beplo5FWT5UelJWD8ngh6zGyyHcoCzwgkkqhiFQi5gakS1lbreA2zYNsrKVU6WOsIujMI/2tGwVHl+iWyJ1kj+DxCov3OO6Hcil1rveoou+f6QBMQkKZW5kc3RyZWFtCmVuZG9iagozNSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDc1ID4+CnN0cmVhbQp4nDO1NFIwUDA2ABKmZkYKpibmCimGXEA+iJXLZWhkCmblcBlZmilYWAAZJmbmUCGYhhwuY1NzoAFARcamYBqqP4crgysNAJWQEu8KZW5kc3RyZWFtCmVuZG9iagozNiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE0MSA+PgpzdHJlYW0KeJw9j8EOwzAIQ+/5Cv9ApNgpoXxPp2qH7v+vI0u7C3oCY4yF0NAbqprDhmCb48XSJVRr+BTFQCU3yJlgDqWk0h1HkXpiOBhcHrQbjuKx6PoRu5JmfdDGQrolaIB7rFNp3KZxE8QdNQXqKeqco7wQuZ+pZ9g0kt00s5JzuA2/e89T1/+nq7zL+QW9dy7+CmVuZHN0cmVhbQplbmRvYmoKMTcgMCBvYmoKPDwgL0Jhc2VGb250IC9CTVFRRFYrRGVqYVZ1U2FucyAvQ2hhclByb2NzIDE4IDAgUgovRW5jb2RpbmcgPDwKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCi9UeXBlIC9FbmNvZGluZyA+PgovRmlyc3RDaGFyIDAgL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvRm9udERlc2NyaXB0b3IgMTYgMCBSCi9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdIC9MYXN0Q2hhciAyNTUgL05hbWUgL0JNUVFEVitEZWphVnVTYW5zCi9TdWJ0eXBlIC9UeXBlMyAvVHlwZSAvRm9udCAvV2lkdGhzIDE1IDAgUiA+PgplbmRvYmoKMTYgMCBvYmoKPDwgL0FzY2VudCA5MjkgL0NhcEhlaWdodCAwIC9EZXNjZW50IC0yMzYgL0ZsYWdzIDMyCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovSXRhbGljQW5nbGUgMCAvTWF4V2lkdGggMTM0MiAvU3RlbVYgMCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL1hIZWlnaHQgMCA+PgplbmRvYmoKMTUgMCBvYmoKWyA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMAo2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDMxOCA0MDEgNDYwIDgzOCA2MzYKOTUwIDc4MCAyNzUgMzkwIDM5MCA1MDAgODM4IDMxOCAzNjEgMzE4IDMzNyA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2CjYzNiA2MzYgMzM3IDMzNyA4MzggODM4IDgzOCA1MzEgMTAwMCA2ODQgNjg2IDY5OCA3NzAgNjMyIDU3NSA3NzUgNzUyIDI5NQoyOTUgNjU2IDU1NyA4NjMgNzQ4IDc4NyA2MDMgNzg3IDY5NSA2MzUgNjExIDczMiA2ODQgOTg5IDY4NSA2MTEgNjg1IDM5MCAzMzcKMzkwIDgzOCA1MDAgNTAwIDYxMyA2MzUgNTUwIDYzNSA2MTUgMzUyIDYzNSA2MzQgMjc4IDI3OCA1NzkgMjc4IDk3NCA2MzQgNjEyCjYzNSA2MzUgNDExIDUyMSAzOTIgNjM0IDU5MiA4MTggNTkyIDU5MiA1MjUgNjM2IDMzNyA2MzYgODM4IDYwMCA2MzYgNjAwIDMxOAozNTIgNTE4IDEwMDAgNTAwIDUwMCA1MDAgMTM0MiA2MzUgNDAwIDEwNzAgNjAwIDY4NSA2MDAgNjAwIDMxOCAzMTggNTE4IDUxOAo1OTAgNTAwIDEwMDAgNTAwIDEwMDAgNTIxIDQwMCAxMDIzIDYwMCA1MjUgNjExIDMxOCA0MDEgNjM2IDYzNiA2MzYgNjM2IDMzNwo1MDAgNTAwIDEwMDAgNDcxIDYxMiA4MzggMzYxIDEwMDAgNTAwIDUwMCA4MzggNDAxIDQwMSA1MDAgNjM2IDYzNiAzMTggNTAwCjQwMSA0NzEgNjEyIDk2OSA5NjkgOTY5IDUzMSA2ODQgNjg0IDY4NCA2ODQgNjg0IDY4NCA5NzQgNjk4IDYzMiA2MzIgNjMyIDYzMgoyOTUgMjk1IDI5NSAyOTUgNzc1IDc0OCA3ODcgNzg3IDc4NyA3ODcgNzg3IDgzOCA3ODcgNzMyIDczMiA3MzIgNzMyIDYxMSA2MDUKNjMwIDYxMyA2MTMgNjEzIDYxMyA2MTMgNjEzIDk4MiA1NTAgNjE1IDYxNSA2MTUgNjE1IDI3OCAyNzggMjc4IDI3OCA2MTIgNjM0CjYxMiA2MTIgNjEyIDYxMiA2MTIgODM4IDYxMiA2MzQgNjM0IDYzNCA2MzQgNTkyIDYzNSA1OTIgXQplbmRvYmoKMTggMCBvYmoKPDwgL0IgMTkgMCBSIC9XIDIwIDAgUiAvYSAyMSAwIFIgL2MgMjIgMCBSIC9kIDIzIDAgUiAvZSAyNCAwIFIgL2YgMjUgMCBSCi9nIDI2IDAgUiAvaCAyNyAwIFIgL2kgMjggMCBSIC9sIDI5IDAgUiAvbiAzMCAwIFIgL3AgMzEgMCBSIC9yIDMyIDAgUgovc3BhY2UgMzMgMCBSIC90IDM0IDAgUiAvdiAzNSAwIFIgL3kgMzYgMCBSID4+CmVuZG9iagozIDAgb2JqCjw8IC9GMSAxNyAwIFIgPj4KZW5kb2JqCjQgMCBvYmoKPDwgL0ExIDw8IC9DQSAwIC9UeXBlIC9FeHRHU3RhdGUgL2NhIDEgPj4KL0EyIDw8IC9DQSAxIC9UeXBlIC9FeHRHU3RhdGUgL2NhIDEgPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgL0kxIDEzIDAgUiAvSTIgMTQgMCBSID4+CmVuZG9iagoxMyAwIG9iago8PCAvQml0c1BlckNvbXBvbmVudCA4Ci9Db2xvclNwYWNlIFsvSW5kZXhlZCAvRGV2aWNlUkdCIDI0ICj95zfr1Eu/sGqmnHObk3aYkXZ9fHh4eHZsbXJgZG5WXFxtRlBrN0ZsLkFsKj9tJT1tITtuETVvAC1pACxmACpgAFwoWwAjUAAiTf8AACldCi9EZWNvZGVQYXJtcyA8PCAvQ29sb3JzIDEgL0NvbHVtbnMgMTUzIC9QcmVkaWN0b3IgMTAgPj4KL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0hlaWdodCAxNTMgL0xlbmd0aCAzNyAwIFIgL1N1YnR5cGUgL0ltYWdlCi9UeXBlIC9YT2JqZWN0IC9XaWR0aCAxNTMgPj4Kc3RyZWFtCnic7dhJTgMxAETRJkxhhkBo7n9Sti6kRCoUiabz/tbT887yNC+16a8BByPrI+sj6yPrI+sj6/sh+4w+orfoKXocy6FcllvmcWRkZGRkZGSrl+XCPGIbXUabsavoLnqN9hEZGRkZGRnZ6mW76CG6ji6iaSyYm7zQS0RGRkZGRkZ2ZrJjr6B8ztxEt2P30XP0Hp3ox4WMjIyMjIzsv8hy6v5IOfNrbD5JZGRkZGRkZL+WLSiyPrI+sj6yPrI+sj6yPrI+sj6yPrI+sj6yPrI+sj6yPrI+sj6yPrI+sj6yPrI+sj6yPrI+sj6yPrI+sj6yPrI+sj6yvuXKvgHcThi8CmVuZHN0cmVhbQplbmRvYmoKMzcgMCBvYmoKMjU0CmVuZG9iagoxNCAwIG9iago8PCAvQml0c1BlckNvbXBvbmVudCA4IC9Db2xvclNwYWNlIFsvSW5kZXhlZCAvRGV2aWNlUkdCIDIgKP3nNwAiTf8AACldCi9EZWNvZGVQYXJtcyA8PCAvQ29sb3JzIDEgL0NvbHVtbnMgMTUzIC9QcmVkaWN0b3IgMTAgPj4KL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0hlaWdodCAxNTMgL0xlbmd0aCAzOCAwIFIgL1N1YnR5cGUgL0ltYWdlCi9UeXBlIC9YT2JqZWN0IC9XaWR0aCAxNTMgPj4Kc3RyZWFtCnic7c7BCYAwAACx1v2H9iuC4InQPpIJMuauxurAI7POrDPrzDqzzqy7zcZKZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmrx1X8xdmZmZmZmZmn2cbMevMOrPOrDPrzDqzzqwz68w6s86sM+vMOrPOrDPrzDqzzqwz68w6s86sM+vMOrPOrDPrzDqzzqwz68y6fWcnQ9496wplbmRzdHJlYW0KZW5kb2JqCjM4IDAgb2JqCjE1OQplbmRvYmoKMiAwIG9iago8PCAvQ291bnQgMSAvS2lkcyBbIDExIDAgUiBdIC9UeXBlIC9QYWdlcyA+PgplbmRvYmoKMzkgMCBvYmoKPDwgL0NyZWF0aW9uRGF0ZSAoRDoyMDIyMDUwMzAzMTYxNSswMicwMCcpCi9DcmVhdG9yIChNYXRwbG90bGliIHYzLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjUuMSkgPj4KZW5kb2JqCnhyZWYKMCA0MAowMDAwMDAwMDAwIDY1NTM1IGYgCjAwMDAwMDAwMTYgMDAwMDAgbiAKMDAwMDAwODIzMiAwMDAwMCBuIAowMDAwMDA2OTg2IDAwMDAwIG4gCjAwMDAwMDcwMTggMDAwMDAgbiAKMDAwMDAwNzExNyAwMDAwMCBuIAowMDAwMDA3MTM4IDAwMDAwIG4gCjAwMDAwMDcxNTkgMDAwMDAgbiAKMDAwMDAwMDA2NSAwMDAwMCBuIAowMDAwMDAwMzQzIDAwMDAwIG4gCjAwMDAwMDA3MTkgMDAwMDAgbiAKMDAwMDAwMDIwOCAwMDAwMCBuIAowMDAwMDAwNjk5IDAwMDAwIG4gCjAwMDAwMDcyMDIgMDAwMDAgbiAKMDAwMDAwNzc5OSAwMDAwMCBuIAowMDAwMDA1NzI3IDAwMDAwIG4gCjAwMDAwMDU1MjAgMDAwMDAgbiAKMDAwMDAwNTExMiAwMDAwMCBuIAowMDAwMDA2NzgwIDAwMDAwIG4gCjAwMDAwMDA3MzkgMDAwMDAgbiAKMDAwMDAwMTA3NiAwMDAwMCBuIAowMDAwMDAxMjQwIDAwMDAwIG4gCjAwMDAwMDE2MjAgMDAwMDAgbiAKMDAwMDAwMTkyNSAwMDAwMCBuIAowMDAwMDAyMjI5IDAwMDAwIG4gCjAwMDAwMDI1NTEgMDAwMDAgbiAKMDAwMDAwMjc2MCAwMDAwMCBuIAowMDAwMDAzMTc0IDAwMDAwIG4gCjAwMDAwMDM0MTEgMDAwMDAgbiAKMDAwMDAwMzU1NSAwMDAwMCBuIAowMDAwMDAzNjc0IDAwMDAwIG4gCjAwMDAwMDM5MTAgMDAwMDAgbiAKMDAwMDAwNDIyMiAwMDAwMCBuIAowMDAwMDA0NDU1IDAwMDAwIG4gCjAwMDAwMDQ1NDUgMDAwMDAgbiAKMDAwMDAwNDc1MSAwMDAwMCBuIAowMDAwMDA0ODk4IDAwMDAwIG4gCjAwMDAwMDc3NzkgMDAwMDAgbiAKMDAwMDAwODIxMiAwMDAwMCBuIAowMDAwMDA4MjkyIDAwMDAwIG4gCnRyYWlsZXIKPDwgL0luZm8gMzkgMCBSIC9Sb290IDEgMCBSIC9TaXplIDQwID4+CnN0YXJ0eHJlZgo4NDQ5CiUlRU9GCg==\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2022-05-03T03:16:15.312945\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.5.1, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}, {"name": "stdout", "output_type": "stream", "text": ["Layer 5\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUiAvTWVkaWFCb3ggWyAwIDAgMzQ5LjIgMTgxLjcwNjgxODE4MTggXQovUGFyZW50IDIgMCBSIC9SZXNvdXJjZXMgOCAwIFIgL1R5cGUgL1BhZ2UgPj4KZW5kb2JqCjkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMiAwIFIgPj4Kc3RyZWFtCnicnVFLTwIxEL7Pr/iOcLA701K2PYIo0Ru6CQfjwSwFlgDKIxD/vbMKPvAidjLpzPSb19esl3ZVme76XVzeU/bllRsSzCjrCCYbMGaqewj6+AlijS/ItaKxas0PlgQxObeDBA3xqVvjpkRLWiFXcK3irVHUQU7ddcIQS2QdWw+jc6nutUxfK4h3qEvWd65SLpDdCHrPGNAAq2MSY/I9sfZppSsyLjQXwoad49j2kSOk7Y37al8uqFsgu1aYRTF+36AY0QMawyZ8MD663KORqsl0m0ZYN2HZBHc8+lKml221SxhXaT5q4hHFLV0V9DGhhGg+m/2bDvpNh0Q+EmLPJcSyfpm3ocVKiPydkW61fFq/nscBvQHN+pCiCmVuZHN0cmVhbQplbmRvYmoKMTIgMCBvYmoKMjgxCmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjE5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjY0ID4+CnN0cmVhbQp4nD1SuZEDMQzLtwqWwF9SPb65cWD3nx7APTsxMUsRAEH3blE5Gz9tJcuP/NhlZ0nklvegbfK6ouoGqZsgc8lxyRVilvK4EiQWLWUm1meqg/JxDWoVvvATMxPTIUnEEnJSjyKsj6+D1/Uc3F836fwaGR90bFB6S5KroFTDvnK0EjuW6l2D7IOmm9LqM9Ou7ICl8Yas1KROfx31vyMswqTeSEUHIRa/08sdsiykHNTbuX4D4V3tglhPrTzsEGneVnpLFczmzARMGW07rIC1WWPCVAnwFs0h1FqIzBc6rYj55NzRwTRVeRoiw1KFg+09ozVc6BoSS8P5ScYE5r/w+qzIbX//AHBoZ7oKZW5kc3RyZWFtCmVuZG9iagoyMCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDkyID4+CnN0cmVhbQp4nD2NwQ3AMAgD/0zBCBACxPtUVR/p/t8mEeoHHwbZGGBhszXgwdnAl9LaN72kRZPaCFa1Rd1QnrsUpVhdR6VMwk+ZO39SdBztcA7b39blOE3j6F/30P0BD0oeCwplbmRzdHJlYW0KZW5kb2JqCjIxIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzA3ID4+CnN0cmVhbQp4nD2SS24DMQxD9z6FLhDA+tme86Qoupjef9snJemKHNkWRWqWukxZUx6QNJOEf+nwcLGd8jtsz2Zm4Fqil4nllOfQFWLuonzZzEZdWSfF6oRmOrfoUTkXBzZNqp+rLKXdLngO1yaeW/YRP7zQoB7UNS4JN3RXo2UpNGOq+3/Se/yMMuBqTF1sUqt7HzxeRFXo6AdHiSJjlxfn40EJ6UrCaFqIlXdFA0Hu8rTKewnu295qyLIHqZjOOylmsOt0Ui5uF4chHsjyqPDlo9hrQs/4sCsl9EjYhjNyJ+5oxubUyOKQ/t6NBEuPrmgh8+CvbtYuYLxTOkViZE5yrGmLVU73UBTTucO9DBD1bEVDKXOR1epfw84La5ZsFnhK+gUeo90mSw5W2duoTu+tPNnQ9x9a13QfCmVuZHN0cmVhbQplbmRvYmoKMjIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMzIgPj4Kc3RyZWFtCnicNVFJbsQwDLv7FfzAANbuvCfFoIf2/9dSyhQIQCW2uCViYyMCLzH4OYjc+JI1oyZ+Z3JX/CxPhUfCreBJFIGX4V52gssbxmU/DjMfvJdWzqTGkwzIRTY9PBEy2CUQOjC7BnXYZtqJviHhsyNSzUaW09cS9NIqBMpTtt/pghJtq/pz+6wLbfvaE052e+pJ5ROI55aswGXjFZPFWAY9UblLMX2Q6myhJ6G8KJ+DbD5qiESXKGfgicHBKNAO7LntZ+JVIWhd3adtY6hGSsfTvw1NTZII+UQJZ7Y07hb+f8+9vtf7D04hVBEKZW5kc3RyZWFtCmVuZG9iagoyMyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIzMSA+PgpzdHJlYW0KeJw1TzmSBCEMy3mFPjBVGNtAv6entjbY+X+6kplOkPAhydMTHZl4mSMjsGbH21pkIGbgU0zFv/a0DxOq9+AeIpSLC2GGkXDWrONuno4X/3aVz1gH7zb4illeENjCTNZXFmcu2wVjaZzEOclujF0TsY11radTWEcwoQyEdLbDlCBzVKT0yY4y5ug4kSeei+/22yx2OX4O6ws2jSEV5/gqeoI2g6Lsee8CGnJB/13d+B5Fu+glIBsJFtZRYu6c5YRfvXZ0HrUoEnNCmkEuEyHN6SqmEJpQrLOjoFJRcKk+p+isn3/lX1wtCmVuZHN0cmVhbQplbmRvYmoKMjQgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNDkgPj4Kc3RyZWFtCnicPVA7jkQhDOs5hS/wJPIjcB5Gqy1m79+uA5opUEx+tjMk0BGBRwwxlK/jJa2groG/i0LxbuLrg8Igq0NSIM56D4h07KY2kRM6HZwzP2E3Y47ARTEGnOl0pj0HJjn7wgqEcxtl7FZIJ4mqIo7qM44pnip7n3gWLO3INlsnkj3kIOFSUonJpZ+Uyj9typQKOmbRBCwSueBkE004y7tJUowZlDLqHqZ2In2sPMijOuhkTc6sI5nZ00/bmfgccLdf2mROlcd0Hsz4nLTOgzkVuvfjiTYHTY3a6Oz3E2kqL1K7HVqdfnUSld0Y5xgSl2d/Gd9k//kH/odaIgplbmRzdHJlYW0KZW5kb2JqCjI1IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTM2ID4+CnN0cmVhbQp4nE2PQQ4DMQgD73mFn0AgQHjPVlUP2/9fS9h20wseyYBsUQaBJYd4hxvh0dsP30U2FWfjnF9SKWIhmE9wnzBTHI0pd/Jjj4BxlGosp2h4XkvOTcMXLXcTLaWtl5MZb7jul/dHlW2RDUXPLQtC12yS+TKBB3wYmEd142mlx932bK/2/ADObDRJCmVuZHN0cmVhbQplbmRvYmoKMjYgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzNDEgPj4Kc3RyZWFtCnicRVJLbkQxCNu/U3CBSOGXkPO0qrqY3n9bm0zVzeAJYGx4y1OmZMqwuSUjJNeUT30iQ6ym/DRyJCKm+EkJBXaVj8drS6yN7JGoFJ/a8eOx9Eam2RVa9e7Rpc2iUc3KyDnIEKGeFbqye9QO2fB6XEi675TNIRzL/1CBLGXdcgolQVvQd+wR3w8droIrgmGway6D7WUy1P/6hxZc7333YscugBas577BDgCopxO0BcgZ2u42KWgAVbqLScKj8npudqJso1Xp+RwAMw4wcsCIJVsdvtHeAJZ9XehFjYr9K0BRWUD8yNV2wd4xyUhwFuYGjr1wPMWZcEs4xgJAir3iGHrwJdjmL1euiJrwCXW6ZC+8wp7a5udCkwh3rQAOXmTDraujqJbt6TyC9mdFckaM1Is4OiGSWtI5guLSoB5a41w3seJtI7G5V9/uH+GcL1z26xdL7ITECmVuZHN0cmVhbQplbmRvYmoKMjcgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNjQgPj4Kc3RyZWFtCnicRZDHcQUxDEPvqgIlMIAK9azH8w/r/q+G9NNBehhCDGJPwrBcV3FhdMOPty0zDX9HGe7G+jJjvNVYICfoAwyRiavRpPp2xRmq9OTVYq6jolwvOiISzJLjq0AjfDqyx5O2tjP9dF4f7CHvE/8qKuduYQEuqu5A+VIf8dSP2VHqmqGPKitrHmraV4RdEUrbPi6nMk7dvQNa4b2Vqz3a7z8edjryCmVuZHN0cmVhbQplbmRvYmoKMjggMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA3MiA+PgpzdHJlYW0KeJwzMrdQMFCwNAEShhYmCuZmBgophlxAvqmJuUIuF0gMxMoBswyAtCWcgohngJggbRDFIBZEsZmJGUQdnAGRy+BKAwAl2xbJCmVuZHN0cmVhbQplbmRvYmoKMjkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA0NyA+PgpzdHJlYW0KeJwzMrdQMFCwNAEShhYmCuZmBgophlyWEFYuF0wsB8wC0ZZwCiKewZUGALlnDScKZW5kc3RyZWFtCmVuZG9iagozMCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE2MyA+PgpzdHJlYW0KeJxFkDsSAyEMQ3tOoSP4IwM+z2YyKTb3b2PYbFLA01ggg7sTgtTagonogoe2Jd0F760EZ2P86TZuNRLkBHWAVqTjaJRSfbnFaZV08Wg2cysLrRMdZg56lKMZoBA6Fd7touRypu7O+UNw9V/1v2LdOZuJgcnKHQjN6lPc+TY7orq6yf6kx9ys134r7FVhaVlLywm3nbtmQAncUznaqz0/Hwo69gplbmRzdHJlYW0KZW5kb2JqCjMxIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjM5ID4+CnN0cmVhbQp4nE1QyW0EMQz7uwo1MMDoHLseB4s8sv1/Q8oJkpdoS+Kh8pRblspl9yM5b8m65UOHTpVp8m7Qza+x/qMMAnb/UFQQrSWxSsxc0m6xNEkv2cM4jZdrtY7nqXuEWaN48OPY0ymB6T0ywWazvTkwqz3ODpBOuMav6tM7lSQDibqQ80KlCuse1CWijyvbmFKdTi3lGJef6Ht8jgA9xd6N3NHHyxeMRrUtqNFqlTgPMBNT0ZVxq5GBlBMGQ2dHVzQLpcjKekI1wo05oZm9w3BgA8uzhKSlrVK8D2UB6AJd2jrjNEqCjgDC3yiM9foGqvxeNwplbmRzdHJlYW0KZW5kb2JqCjMyIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTYwID4+CnN0cmVhbQp4nEWQORIDMQgEc72CJ0hcgvesy7XB+v+pB9ZHoukCNBy6Fk3KehRoPumxRqG60GvoLEqSRMEWkh1Qp2OIOyhITEhjkki2HoMjmlizXZiZVCqzUuG0acXCv9la1chEjXCN/InpBlT8T+pclPBNg6+SMfoYVLw7g4xJ+F5F3Fox7f5EMLEZ9glvRSYFhImxqdm+z2CGzPcK1zjH8w1MgjfrCmVuZHN0cmVhbQplbmRvYmoKMzMgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxOCA+PgpzdHJlYW0KeJwzNrRQMIDDFEOuNAAd5gNSCmVuZHN0cmVhbQplbmRvYmoKMzQgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMzMgPj4Kc3RyZWFtCnicRY9LDgQhCET3nKKOwMcf53Ey6YVz/+2AnW4TYz2FVIG5gqE9LmsDnRUfIRm28beplo5FWT5UelJWD8ngh6zGyyHcoCzwgkkqhiFQi5gakS1lbreA2zYNsrKVU6WOsIujMI/2tGwVHl+iWyJ1kj+DxCov3OO6Hcil1rveoou+f6QBMQkKZW5kc3RyZWFtCmVuZG9iagozNSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDc1ID4+CnN0cmVhbQp4nDO1NFIwUDA2ABKmZkYKpibmCimGXEA+iJXLZWhkCmblcBlZmilYWAAZJmbmUCGYhhwuY1NzoAFARcamYBqqP4crgysNAJWQEu8KZW5kc3RyZWFtCmVuZG9iagozNiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE0MSA+PgpzdHJlYW0KeJw9j8EOwzAIQ+/5Cv9ApNgpoXxPp2qH7v+vI0u7C3oCY4yF0NAbqprDhmCb48XSJVRr+BTFQCU3yJlgDqWk0h1HkXpiOBhcHrQbjuKx6PoRu5JmfdDGQrolaIB7rFNp3KZxE8QdNQXqKeqco7wQuZ+pZ9g0kt00s5JzuA2/e89T1/+nq7zL+QW9dy7+CmVuZHN0cmVhbQplbmRvYmoKMTcgMCBvYmoKPDwgL0Jhc2VGb250IC9CTVFRRFYrRGVqYVZ1U2FucyAvQ2hhclByb2NzIDE4IDAgUgovRW5jb2RpbmcgPDwKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCi9UeXBlIC9FbmNvZGluZyA+PgovRmlyc3RDaGFyIDAgL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvRm9udERlc2NyaXB0b3IgMTYgMCBSCi9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdIC9MYXN0Q2hhciAyNTUgL05hbWUgL0JNUVFEVitEZWphVnVTYW5zCi9TdWJ0eXBlIC9UeXBlMyAvVHlwZSAvRm9udCAvV2lkdGhzIDE1IDAgUiA+PgplbmRvYmoKMTYgMCBvYmoKPDwgL0FzY2VudCA5MjkgL0NhcEhlaWdodCAwIC9EZXNjZW50IC0yMzYgL0ZsYWdzIDMyCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovSXRhbGljQW5nbGUgMCAvTWF4V2lkdGggMTM0MiAvU3RlbVYgMCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL1hIZWlnaHQgMCA+PgplbmRvYmoKMTUgMCBvYmoKWyA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMAo2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDMxOCA0MDEgNDYwIDgzOCA2MzYKOTUwIDc4MCAyNzUgMzkwIDM5MCA1MDAgODM4IDMxOCAzNjEgMzE4IDMzNyA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2CjYzNiA2MzYgMzM3IDMzNyA4MzggODM4IDgzOCA1MzEgMTAwMCA2ODQgNjg2IDY5OCA3NzAgNjMyIDU3NSA3NzUgNzUyIDI5NQoyOTUgNjU2IDU1NyA4NjMgNzQ4IDc4NyA2MDMgNzg3IDY5NSA2MzUgNjExIDczMiA2ODQgOTg5IDY4NSA2MTEgNjg1IDM5MCAzMzcKMzkwIDgzOCA1MDAgNTAwIDYxMyA2MzUgNTUwIDYzNSA2MTUgMzUyIDYzNSA2MzQgMjc4IDI3OCA1NzkgMjc4IDk3NCA2MzQgNjEyCjYzNSA2MzUgNDExIDUyMSAzOTIgNjM0IDU5MiA4MTggNTkyIDU5MiA1MjUgNjM2IDMzNyA2MzYgODM4IDYwMCA2MzYgNjAwIDMxOAozNTIgNTE4IDEwMDAgNTAwIDUwMCA1MDAgMTM0MiA2MzUgNDAwIDEwNzAgNjAwIDY4NSA2MDAgNjAwIDMxOCAzMTggNTE4IDUxOAo1OTAgNTAwIDEwMDAgNTAwIDEwMDAgNTIxIDQwMCAxMDIzIDYwMCA1MjUgNjExIDMxOCA0MDEgNjM2IDYzNiA2MzYgNjM2IDMzNwo1MDAgNTAwIDEwMDAgNDcxIDYxMiA4MzggMzYxIDEwMDAgNTAwIDUwMCA4MzggNDAxIDQwMSA1MDAgNjM2IDYzNiAzMTggNTAwCjQwMSA0NzEgNjEyIDk2OSA5NjkgOTY5IDUzMSA2ODQgNjg0IDY4NCA2ODQgNjg0IDY4NCA5NzQgNjk4IDYzMiA2MzIgNjMyIDYzMgoyOTUgMjk1IDI5NSAyOTUgNzc1IDc0OCA3ODcgNzg3IDc4NyA3ODcgNzg3IDgzOCA3ODcgNzMyIDczMiA3MzIgNzMyIDYxMSA2MDUKNjMwIDYxMyA2MTMgNjEzIDYxMyA2MTMgNjEzIDk4MiA1NTAgNjE1IDYxNSA2MTUgNjE1IDI3OCAyNzggMjc4IDI3OCA2MTIgNjM0CjYxMiA2MTIgNjEyIDYxMiA2MTIgODM4IDYxMiA2MzQgNjM0IDYzNCA2MzQgNTkyIDYzNSA1OTIgXQplbmRvYmoKMTggMCBvYmoKPDwgL0IgMTkgMCBSIC9XIDIwIDAgUiAvYSAyMSAwIFIgL2MgMjIgMCBSIC9kIDIzIDAgUiAvZSAyNCAwIFIgL2YgMjUgMCBSCi9nIDI2IDAgUiAvaCAyNyAwIFIgL2kgMjggMCBSIC9sIDI5IDAgUiAvbiAzMCAwIFIgL3AgMzEgMCBSIC9yIDMyIDAgUgovc3BhY2UgMzMgMCBSIC90IDM0IDAgUiAvdiAzNSAwIFIgL3kgMzYgMCBSID4+CmVuZG9iagozIDAgb2JqCjw8IC9GMSAxNyAwIFIgPj4KZW5kb2JqCjQgMCBvYmoKPDwgL0ExIDw8IC9DQSAwIC9UeXBlIC9FeHRHU3RhdGUgL2NhIDEgPj4KL0EyIDw8IC9DQSAxIC9UeXBlIC9FeHRHU3RhdGUgL2NhIDEgPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgL0kxIDEzIDAgUiAvSTIgMTQgMCBSID4+CmVuZG9iagoxMyAwIG9iago8PCAvQml0c1BlckNvbXBvbmVudCA4Ci9Db2xvclNwYWNlIFsvSW5kZXhlZCAvRGV2aWNlUkdCIDM3ICj95zfs1UrUwV7Sv2DNvGKlm3Ogl3WUjneIhXiCgHh7e3d2dnZmaXBYXW1XXW1UWmxRWGxMVGxET2tBTWs3RmwuQWwrP20kPG4WNm8UNm8LM3AAL28ALmsALGYAK2QAKmIAXChbACZVACRSACNQACJN/wAAKV0KL0RlY29kZVBhcm1zIDw8IC9Db2xvcnMgMSAvQ29sdW1ucyAxNTMgL1ByZWRpY3RvciAxMCA+PgovRmlsdGVyIC9GbGF0ZURlY29kZSAvSGVpZ2h0IDE1MyAvTGVuZ3RoIDM3IDAgUiAvU3VidHlwZSAvSW1hZ2UKL1R5cGUgL1hPYmplY3QgL1dpZHRoIDE1MyA+PgpzdHJlYW0KeJzt2bduwzAYAGGlO9XpPXGK8/6vmJU3XpDBFu42gqD0aSN+TWv0jT7QK3pGj2Pc4jE+kq8jZUqWLFmyZMmSzUHGzTd0j87RCVqMnaJL9IBWiJRkyZIlS5Ys2Sxkn+gJXaBDtIOmsT10jG7QC/pCyZIlS5YsWbLZy5boCO2j3bEDdIbu0DsSt6BkyZIlS5Ys2bbIuOTEheORa8RBytXYLeLHcsbCe4/4u5MsWbJkyZIl2xYZl3TyIPd47mds/S8lS5YsWbJkyf4s26CS+ZL5kvmS+ZL5kvmS+ZL5kvmS+ZL5kvmS+ZL5kvmS+ZL5kvmS+ZL5kvmS+ZL5kvmS+ZL5kvmS+ZL5kvmS+ZL5kvmS+TZX9guYX5emCmVuZHN0cmVhbQplbmRvYmoKMzcgMCBvYmoKMjc5CmVuZG9iagoxNCAwIG9iago8PCAvQml0c1BlckNvbXBvbmVudCA4IC9Db2xvclNwYWNlIFsvSW5kZXhlZCAvRGV2aWNlUkdCIDIgKP3nNwAiTf8AACldCi9EZWNvZGVQYXJtcyA8PCAvQ29sb3JzIDEgL0NvbHVtbnMgMTUzIC9QcmVkaWN0b3IgMTAgPj4KL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0hlaWdodCAxNTMgL0xlbmd0aCAzOCAwIFIgL1N1YnR5cGUgL0ltYWdlCi9UeXBlIC9YT2JqZWN0IC9XaWR0aCAxNTMgPj4Kc3RyZWFtCnic7c6xCcAwEMBAO/sPndYpBW9IcTeA0FoAAAAAAAAAAAAA8PGc9ghnzpw5c+bMWTibyVzgrHPWOeucdc46Z52zzlnnrHPWOeucdc46Z52zzlnnrHPWOeucdc46Z52zzlnnrHPWOeucdc46Z52zzlnnrHPW/ffsBTlYLrcKZW5kc3RyZWFtCmVuZG9iagozOCAwIG9iagoxMjQKZW5kb2JqCjIgMCBvYmoKPDwgL0NvdW50IDEgL0tpZHMgWyAxMSAwIFIgXSAvVHlwZSAvUGFnZXMgPj4KZW5kb2JqCjM5IDAgb2JqCjw8IC9DcmVhdGlvbkRhdGUgKEQ6MjAyMjA1MDMwMzE2MTUrMDInMDAnKQovQ3JlYXRvciAoTWF0cGxvdGxpYiB2My41LjEsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcpCi9Qcm9kdWNlciAoTWF0cGxvdGxpYiBwZGYgYmFja2VuZCB2My41LjEpID4+CmVuZG9iagp4cmVmCjAgNDAKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDgyNjAgMDAwMDAgbiAKMDAwMDAwNjk4NiAwMDAwMCBuIAowMDAwMDA3MDE4IDAwMDAwIG4gCjAwMDAwMDcxMTcgMDAwMDAgbiAKMDAwMDAwNzEzOCAwMDAwMCBuIAowMDAwMDA3MTU5IDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM0MyAwMDAwMCBuIAowMDAwMDAwNzE5IDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDY5OSAwMDAwMCBuIAowMDAwMDA3MjAyIDAwMDAwIG4gCjAwMDAwMDc4NjIgMDAwMDAgbiAKMDAwMDAwNTcyNyAwMDAwMCBuIAowMDAwMDA1NTIwIDAwMDAwIG4gCjAwMDAwMDUxMTIgMDAwMDAgbiAKMDAwMDAwNjc4MCAwMDAwMCBuIAowMDAwMDAwNzM5IDAwMDAwIG4gCjAwMDAwMDEwNzYgMDAwMDAgbiAKMDAwMDAwMTI0MCAwMDAwMCBuIAowMDAwMDAxNjIwIDAwMDAwIG4gCjAwMDAwMDE5MjUgMDAwMDAgbiAKMDAwMDAwMjIyOSAwMDAwMCBuIAowMDAwMDAyNTUxIDAwMDAwIG4gCjAwMDAwMDI3NjAgMDAwMDAgbiAKMDAwMDAwMzE3NCAwMDAwMCBuIAowMDAwMDAzNDExIDAwMDAwIG4gCjAwMDAwMDM1NTUgMDAwMDAgbiAKMDAwMDAwMzY3NCAwMDAwMCBuIAowMDAwMDAzOTEwIDAwMDAwIG4gCjAwMDAwMDQyMjIgMDAwMDAgbiAKMDAwMDAwNDQ1NSAwMDAwMCBuIAowMDAwMDA0NTQ1IDAwMDAwIG4gCjAwMDAwMDQ3NTEgMDAwMDAgbiAKMDAwMDAwNDg5OCAwMDAwMCBuIAowMDAwMDA3ODQyIDAwMDAwIG4gCjAwMDAwMDgyNDAgMDAwMDAgbiAKMDAwMDAwODMyMCAwMDAwMCBuIAp0cmFpbGVyCjw8IC9JbmZvIDM5IDAgUiAvUm9vdCAxIDAgUiAvU2l6ZSA0MCA+PgpzdGFydHhyZWYKODQ3NwolJUVPRgo=\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2022-05-03T03:16:15.477300\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.5.1, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}], "source": ["# Initialize convolutions with equal weight to all input pixels\n", "horiz_conv = HorizontalStackConvolution(c_in=1, c_out=1, kernel_size=3, mask_center=False)\n", "horiz_conv.conv.weight.data.fill_(1)\n", "horiz_conv.conv.bias.data.fill_(0)\n", "vert_conv = VerticalStackConvolution(c_in=1, c_out=1, kernel_size=3, mask_center=False)\n", "vert_conv.conv.weight.data.fill_(1)\n", "vert_conv.conv.bias.data.fill_(0)\n", "\n", "# We reuse our convolutions for the 4 layers here. Note that in a standard network,\n", "# we don't do that, and instead learn 4 separate convolution. As this cell is only for\n", "# visualization purposes, we reuse the convolutions for all layers.\n", "for l_idx in range(4):\n", " vert_img = vert_conv(vert_img)\n", " horiz_img = horiz_conv(horiz_img) + vert_img\n", " print(\"Layer %i\" % (l_idx + 2))\n", " show_center_recep_field(inp_img, horiz_img)"]}, {"cell_type": "markdown", "id": "ab51be63", "metadata": {"papermill": {"duration": 0.013162, "end_time": "2022-05-03T01:16:15.586957", "exception": false, "start_time": "2022-05-03T01:16:15.573795", "status": "completed"}, "tags": []}, "source": ["The receptive field above it visualized for the horizontal stack, which includes the features of the vertical convolutions.\n", "It grows over layers without any blind spot as we had before.\n", "The difference between \"weighted\" and \"binary\" receptive field is that for the latter, we check whether there are any gradients flowing back to this pixel.\n", "This indicates that the center pixel indeed can use information from this pixel.\n", "Nevertheless, due to the convolution weights, some pixels have a stronger effect on the prediction than others.\n", "This is visualized in the weighted receptive field by plotting the gradient magnitude for each pixel instead of a binary yes/no.\n", "\n", "\n", "Another receptive field we can check is the one for the vertical stack\n", "as the one above is for the horizontal stack. Let's visualize it below:"]}, {"cell_type": "code", "execution_count": 13, "id": "2a3fd12c", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:16:15.614432Z", "iopub.status.busy": "2022-05-03T01:16:15.614118Z", "iopub.status.idle": "2022-05-03T01:16:15.769388Z", "shell.execute_reply": "2022-05-03T01:16:15.768692Z"}, "papermill": {"duration": 0.170923, "end_time": "2022-05-03T01:16:15.770970", "exception": false, "start_time": "2022-05-03T01:16:15.600047", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUiAvTWVkaWFCb3ggWyAwIDAgMzQ5LjIgMTgxLjcwNjgxODE4MTggXQovUGFyZW50IDIgMCBSIC9SZXNvdXJjZXMgOCAwIFIgL1R5cGUgL1BhZ2UgPj4KZW5kb2JqCjkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMiAwIFIgPj4Kc3RyZWFtCnicnVFLTwIxEL7Pr/iOcLA701K2PYIo0Ru6CQfjwSwFlgDKIxD/vbMKPvAidjLpzPSb19esl3ZVme76XVzeU/bllRsSzCjrCCYbMGaqewj6+AlijS/ItaKxas0PlgQxObeDBA3xqVvjpkRLWiFXcK3irVHUQU7ddcIQS2QdWw+jc6nutUxfK4h3qEvWd65SLpDdCHrPGNAAq2MSY/I9sfZppSsyLjQXwoad49j2kSOk7Y37al8uqFsgu1aYRTF+36AY0QMawyZ8MD663KORqsl0m0ZYN2HZBHc8+lKml221SxhXaT5q4hHFLV0V9DGhhGg+m/2bDvpNh0Q+EmLPJcSyfpm3ocVKiPydkW61fFq/nscBvQHN+pCiCmVuZHN0cmVhbQplbmRvYmoKMTIgMCBvYmoKMjgxCmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjE5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjY0ID4+CnN0cmVhbQp4nD1SuZEDMQzLtwqWwF9SPb65cWD3nx7APTsxMUsRAEH3blE5Gz9tJcuP/NhlZ0nklvegbfK6ouoGqZsgc8lxyRVilvK4EiQWLWUm1meqg/JxDWoVvvATMxPTIUnEEnJSjyKsj6+D1/Uc3F836fwaGR90bFB6S5KroFTDvnK0EjuW6l2D7IOmm9LqM9Ou7ICl8Yas1KROfx31vyMswqTeSEUHIRa/08sdsiykHNTbuX4D4V3tglhPrTzsEGneVnpLFczmzARMGW07rIC1WWPCVAnwFs0h1FqIzBc6rYj55NzRwTRVeRoiw1KFg+09ozVc6BoSS8P5ScYE5r/w+qzIbX//AHBoZ7oKZW5kc3RyZWFtCmVuZG9iagoyMCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDkyID4+CnN0cmVhbQp4nD2NwQ3AMAgD/0zBCBACxPtUVR/p/t8mEeoHHwbZGGBhszXgwdnAl9LaN72kRZPaCFa1Rd1QnrsUpVhdR6VMwk+ZO39SdBztcA7b39blOE3j6F/30P0BD0oeCwplbmRzdHJlYW0KZW5kb2JqCjIxIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzA3ID4+CnN0cmVhbQp4nD2SS24DMQxD9z6FLhDA+tme86Qoupjef9snJemKHNkWRWqWukxZUx6QNJOEf+nwcLGd8jtsz2Zm4Fqil4nllOfQFWLuonzZzEZdWSfF6oRmOrfoUTkXBzZNqp+rLKXdLngO1yaeW/YRP7zQoB7UNS4JN3RXo2UpNGOq+3/Se/yMMuBqTF1sUqt7HzxeRFXo6AdHiSJjlxfn40EJ6UrCaFqIlXdFA0Hu8rTKewnu295qyLIHqZjOOylmsOt0Ui5uF4chHsjyqPDlo9hrQs/4sCsl9EjYhjNyJ+5oxubUyOKQ/t6NBEuPrmgh8+CvbtYuYLxTOkViZE5yrGmLVU73UBTTucO9DBD1bEVDKXOR1epfw84La5ZsFnhK+gUeo90mSw5W2duoTu+tPNnQ9x9a13QfCmVuZHN0cmVhbQplbmRvYmoKMjIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMzIgPj4Kc3RyZWFtCnicNVFJbsQwDLv7FfzAANbuvCfFoIf2/9dSyhQIQCW2uCViYyMCLzH4OYjc+JI1oyZ+Z3JX/CxPhUfCreBJFIGX4V52gssbxmU/DjMfvJdWzqTGkwzIRTY9PBEy2CUQOjC7BnXYZtqJviHhsyNSzUaW09cS9NIqBMpTtt/pghJtq/pz+6wLbfvaE052e+pJ5ROI55aswGXjFZPFWAY9UblLMX2Q6myhJ6G8KJ+DbD5qiESXKGfgicHBKNAO7LntZ+JVIWhd3adtY6hGSsfTvw1NTZII+UQJZ7Y07hb+f8+9vtf7D04hVBEKZW5kc3RyZWFtCmVuZG9iagoyMyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIzMSA+PgpzdHJlYW0KeJw1TzmSBCEMy3mFPjBVGNtAv6entjbY+X+6kplOkPAhydMTHZl4mSMjsGbH21pkIGbgU0zFv/a0DxOq9+AeIpSLC2GGkXDWrONuno4X/3aVz1gH7zb4illeENjCTNZXFmcu2wVjaZzEOclujF0TsY11radTWEcwoQyEdLbDlCBzVKT0yY4y5ug4kSeei+/22yx2OX4O6ws2jSEV5/gqeoI2g6Lsee8CGnJB/13d+B5Fu+glIBsJFtZRYu6c5YRfvXZ0HrUoEnNCmkEuEyHN6SqmEJpQrLOjoFJRcKk+p+isn3/lX1wtCmVuZHN0cmVhbQplbmRvYmoKMjQgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNDkgPj4Kc3RyZWFtCnicPVA7jkQhDOs5hS/wJPIjcB5Gqy1m79+uA5opUEx+tjMk0BGBRwwxlK/jJa2groG/i0LxbuLrg8Igq0NSIM56D4h07KY2kRM6HZwzP2E3Y47ARTEGnOl0pj0HJjn7wgqEcxtl7FZIJ4mqIo7qM44pnip7n3gWLO3INlsnkj3kIOFSUonJpZ+Uyj9typQKOmbRBCwSueBkE004y7tJUowZlDLqHqZ2In2sPMijOuhkTc6sI5nZ00/bmfgccLdf2mROlcd0Hsz4nLTOgzkVuvfjiTYHTY3a6Oz3E2kqL1K7HVqdfnUSld0Y5xgSl2d/Gd9k//kH/odaIgplbmRzdHJlYW0KZW5kb2JqCjI1IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTM2ID4+CnN0cmVhbQp4nE2PQQ4DMQgD73mFn0AgQHjPVlUP2/9fS9h20wseyYBsUQaBJYd4hxvh0dsP30U2FWfjnF9SKWIhmE9wnzBTHI0pd/Jjj4BxlGosp2h4XkvOTcMXLXcTLaWtl5MZb7jul/dHlW2RDUXPLQtC12yS+TKBB3wYmEd142mlx932bK/2/ADObDRJCmVuZHN0cmVhbQplbmRvYmoKMjYgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzNDEgPj4Kc3RyZWFtCnicRVJLbkQxCNu/U3CBSOGXkPO0qrqY3n9bm0zVzeAJYGx4y1OmZMqwuSUjJNeUT30iQ6ym/DRyJCKm+EkJBXaVj8drS6yN7JGoFJ/a8eOx9Eam2RVa9e7Rpc2iUc3KyDnIEKGeFbqye9QO2fB6XEi675TNIRzL/1CBLGXdcgolQVvQd+wR3w8droIrgmGway6D7WUy1P/6hxZc7333YscugBas577BDgCopxO0BcgZ2u42KWgAVbqLScKj8npudqJso1Xp+RwAMw4wcsCIJVsdvtHeAJZ9XehFjYr9K0BRWUD8yNV2wd4xyUhwFuYGjr1wPMWZcEs4xgJAir3iGHrwJdjmL1euiJrwCXW6ZC+8wp7a5udCkwh3rQAOXmTDraujqJbt6TyC9mdFckaM1Is4OiGSWtI5guLSoB5a41w3seJtI7G5V9/uH+GcL1z26xdL7ITECmVuZHN0cmVhbQplbmRvYmoKMjcgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNjQgPj4Kc3RyZWFtCnicRZDHcQUxDEPvqgIlMIAK9azH8w/r/q+G9NNBehhCDGJPwrBcV3FhdMOPty0zDX9HGe7G+jJjvNVYICfoAwyRiavRpPp2xRmq9OTVYq6jolwvOiISzJLjq0AjfDqyx5O2tjP9dF4f7CHvE/8qKuduYQEuqu5A+VIf8dSP2VHqmqGPKitrHmraV4RdEUrbPi6nMk7dvQNa4b2Vqz3a7z8edjryCmVuZHN0cmVhbQplbmRvYmoKMjggMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA3MiA+PgpzdHJlYW0KeJwzMrdQMFCwNAEShhYmCuZmBgophlxAvqmJuUIuF0gMxMoBswyAtCWcgohngJggbRDFIBZEsZmJGUQdnAGRy+BKAwAl2xbJCmVuZHN0cmVhbQplbmRvYmoKMjkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA0NyA+PgpzdHJlYW0KeJwzMrdQMFCwNAEShhYmCuZmBgophlyWEFYuF0wsB8wC0ZZwCiKewZUGALlnDScKZW5kc3RyZWFtCmVuZG9iagozMCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE2MyA+PgpzdHJlYW0KeJxFkDsSAyEMQ3tOoSP4IwM+z2YyKTb3b2PYbFLA01ggg7sTgtTagonogoe2Jd0F760EZ2P86TZuNRLkBHWAVqTjaJRSfbnFaZV08Wg2cysLrRMdZg56lKMZoBA6Fd7touRypu7O+UNw9V/1v2LdOZuJgcnKHQjN6lPc+TY7orq6yf6kx9ys134r7FVhaVlLywm3nbtmQAncUznaqz0/Hwo69gplbmRzdHJlYW0KZW5kb2JqCjMxIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjM5ID4+CnN0cmVhbQp4nE1QyW0EMQz7uwo1MMDoHLseB4s8sv1/Q8oJkpdoS+Kh8pRblspl9yM5b8m65UOHTpVp8m7Qza+x/qMMAnb/UFQQrSWxSsxc0m6xNEkv2cM4jZdrtY7nqXuEWaN48OPY0ymB6T0ywWazvTkwqz3ODpBOuMav6tM7lSQDibqQ80KlCuse1CWijyvbmFKdTi3lGJef6Ht8jgA9xd6N3NHHyxeMRrUtqNFqlTgPMBNT0ZVxq5GBlBMGQ2dHVzQLpcjKekI1wo05oZm9w3BgA8uzhKSlrVK8D2UB6AJd2jrjNEqCjgDC3yiM9foGqvxeNwplbmRzdHJlYW0KZW5kb2JqCjMyIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTYwID4+CnN0cmVhbQp4nEWQORIDMQgEc72CJ0hcgvesy7XB+v+pB9ZHoukCNBy6Fk3KehRoPumxRqG60GvoLEqSRMEWkh1Qp2OIOyhITEhjkki2HoMjmlizXZiZVCqzUuG0acXCv9la1chEjXCN/InpBlT8T+pclPBNg6+SMfoYVLw7g4xJ+F5F3Fox7f5EMLEZ9glvRSYFhImxqdm+z2CGzPcK1zjH8w1MgjfrCmVuZHN0cmVhbQplbmRvYmoKMzMgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxOCA+PgpzdHJlYW0KeJwzNrRQMIDDFEOuNAAd5gNSCmVuZHN0cmVhbQplbmRvYmoKMzQgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMzMgPj4Kc3RyZWFtCnicRY9LDgQhCET3nKKOwMcf53Ey6YVz/+2AnW4TYz2FVIG5gqE9LmsDnRUfIRm28beplo5FWT5UelJWD8ngh6zGyyHcoCzwgkkqhiFQi5gakS1lbreA2zYNsrKVU6WOsIujMI/2tGwVHl+iWyJ1kj+DxCov3OO6Hcil1rveoou+f6QBMQkKZW5kc3RyZWFtCmVuZG9iagozNSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDc1ID4+CnN0cmVhbQp4nDO1NFIwUDA2ABKmZkYKpibmCimGXEA+iJXLZWhkCmblcBlZmilYWAAZJmbmUCGYhhwuY1NzoAFARcamYBqqP4crgysNAJWQEu8KZW5kc3RyZWFtCmVuZG9iagozNiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE0MSA+PgpzdHJlYW0KeJw9j8EOwzAIQ+/5Cv9ApNgpoXxPp2qH7v+vI0u7C3oCY4yF0NAbqprDhmCb48XSJVRr+BTFQCU3yJlgDqWk0h1HkXpiOBhcHrQbjuKx6PoRu5JmfdDGQrolaIB7rFNp3KZxE8QdNQXqKeqco7wQuZ+pZ9g0kt00s5JzuA2/e89T1/+nq7zL+QW9dy7+CmVuZHN0cmVhbQplbmRvYmoKMTcgMCBvYmoKPDwgL0Jhc2VGb250IC9CTVFRRFYrRGVqYVZ1U2FucyAvQ2hhclByb2NzIDE4IDAgUgovRW5jb2RpbmcgPDwKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCi9UeXBlIC9FbmNvZGluZyA+PgovRmlyc3RDaGFyIDAgL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvRm9udERlc2NyaXB0b3IgMTYgMCBSCi9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdIC9MYXN0Q2hhciAyNTUgL05hbWUgL0JNUVFEVitEZWphVnVTYW5zCi9TdWJ0eXBlIC9UeXBlMyAvVHlwZSAvRm9udCAvV2lkdGhzIDE1IDAgUiA+PgplbmRvYmoKMTYgMCBvYmoKPDwgL0FzY2VudCA5MjkgL0NhcEhlaWdodCAwIC9EZXNjZW50IC0yMzYgL0ZsYWdzIDMyCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovSXRhbGljQW5nbGUgMCAvTWF4V2lkdGggMTM0MiAvU3RlbVYgMCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL1hIZWlnaHQgMCA+PgplbmRvYmoKMTUgMCBvYmoKWyA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMAo2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDMxOCA0MDEgNDYwIDgzOCA2MzYKOTUwIDc4MCAyNzUgMzkwIDM5MCA1MDAgODM4IDMxOCAzNjEgMzE4IDMzNyA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2CjYzNiA2MzYgMzM3IDMzNyA4MzggODM4IDgzOCA1MzEgMTAwMCA2ODQgNjg2IDY5OCA3NzAgNjMyIDU3NSA3NzUgNzUyIDI5NQoyOTUgNjU2IDU1NyA4NjMgNzQ4IDc4NyA2MDMgNzg3IDY5NSA2MzUgNjExIDczMiA2ODQgOTg5IDY4NSA2MTEgNjg1IDM5MCAzMzcKMzkwIDgzOCA1MDAgNTAwIDYxMyA2MzUgNTUwIDYzNSA2MTUgMzUyIDYzNSA2MzQgMjc4IDI3OCA1NzkgMjc4IDk3NCA2MzQgNjEyCjYzNSA2MzUgNDExIDUyMSAzOTIgNjM0IDU5MiA4MTggNTkyIDU5MiA1MjUgNjM2IDMzNyA2MzYgODM4IDYwMCA2MzYgNjAwIDMxOAozNTIgNTE4IDEwMDAgNTAwIDUwMCA1MDAgMTM0MiA2MzUgNDAwIDEwNzAgNjAwIDY4NSA2MDAgNjAwIDMxOCAzMTggNTE4IDUxOAo1OTAgNTAwIDEwMDAgNTAwIDEwMDAgNTIxIDQwMCAxMDIzIDYwMCA1MjUgNjExIDMxOCA0MDEgNjM2IDYzNiA2MzYgNjM2IDMzNwo1MDAgNTAwIDEwMDAgNDcxIDYxMiA4MzggMzYxIDEwMDAgNTAwIDUwMCA4MzggNDAxIDQwMSA1MDAgNjM2IDYzNiAzMTggNTAwCjQwMSA0NzEgNjEyIDk2OSA5NjkgOTY5IDUzMSA2ODQgNjg0IDY4NCA2ODQgNjg0IDY4NCA5NzQgNjk4IDYzMiA2MzIgNjMyIDYzMgoyOTUgMjk1IDI5NSAyOTUgNzc1IDc0OCA3ODcgNzg3IDc4NyA3ODcgNzg3IDgzOCA3ODcgNzMyIDczMiA3MzIgNzMyIDYxMSA2MDUKNjMwIDYxMyA2MTMgNjEzIDYxMyA2MTMgNjEzIDk4MiA1NTAgNjE1IDYxNSA2MTUgNjE1IDI3OCAyNzggMjc4IDI3OCA2MTIgNjM0CjYxMiA2MTIgNjEyIDYxMiA2MTIgODM4IDYxMiA2MzQgNjM0IDYzNCA2MzQgNTkyIDYzNSA1OTIgXQplbmRvYmoKMTggMCBvYmoKPDwgL0IgMTkgMCBSIC9XIDIwIDAgUiAvYSAyMSAwIFIgL2MgMjIgMCBSIC9kIDIzIDAgUiAvZSAyNCAwIFIgL2YgMjUgMCBSCi9nIDI2IDAgUiAvaCAyNyAwIFIgL2kgMjggMCBSIC9sIDI5IDAgUiAvbiAzMCAwIFIgL3AgMzEgMCBSIC9yIDMyIDAgUgovc3BhY2UgMzMgMCBSIC90IDM0IDAgUiAvdiAzNSAwIFIgL3kgMzYgMCBSID4+CmVuZG9iagozIDAgb2JqCjw8IC9GMSAxNyAwIFIgPj4KZW5kb2JqCjQgMCBvYmoKPDwgL0ExIDw8IC9DQSAwIC9UeXBlIC9FeHRHU3RhdGUgL2NhIDEgPj4KL0EyIDw8IC9DQSAxIC9UeXBlIC9FeHRHU3RhdGUgL2NhIDEgPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgL0kxIDEzIDAgUiAvSTIgMTQgMCBSID4+CmVuZG9iagoxMyAwIG9iago8PCAvQml0c1BlckNvbXBvbmVudCA4Ci9Db2xvclNwYWNlIFsvSW5kZXhlZCAvRGV2aWNlUkdCIDE2ICj95zffylalm3ORjHdkZ29OVmw0RGwqP20iO24IM3AALWkAKmIAJlUAJVQAJFIAIk3/AAApXQovRGVjb2RlUGFybXMgPDwgL0NvbG9ycyAxIC9Db2x1bW5zIDE1MyAvUHJlZGljdG9yIDEwID4+Ci9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9IZWlnaHQgMTUzIC9MZW5ndGggMzcgMCBSIC9TdWJ0eXBlIC9JbWFnZQovVHlwZSAvWE9iamVjdCAvV2lkdGggMTUzID4+CnN0cmVhbQp4nO3ZOWoEQRBFwdG+L/c/rdz+5hMSDE08t6jMMAvq8jV9TG/Ty/Q8PR3bo722I3fdUi5kZGRkZGRkZ5B9Tq/T43Q/3U23x/Zor+3IXbcUMjIyMjIyslPI3qd9sjxMu/Bmuhzbo722I3fdUsjIyMjIyMhOIbveVxAZGRkZGRnZGWTX+7tDRkZGRkZGdj7Zb/s+9jcjyXpkPbIeWY+sR9b7I9k/RNYj65H1yHpkPbIeWY+sR9Yj65H1yHpkPbIeWY+sR9Yj65H1yHpkPbIeWY+sR9Yj65H1yHpkPbIeWY+sR9Yj612v7AfvEECHCmVuZHN0cmVhbQplbmRvYmoKMzcgMCBvYmoKMjM1CmVuZG9iagoxNCAwIG9iago8PCAvQml0c1BlckNvbXBvbmVudCA4IC9Db2xvclNwYWNlIFsvSW5kZXhlZCAvRGV2aWNlUkdCIDIgKP3nNwAiTf8AACldCi9EZWNvZGVQYXJtcyA8PCAvQ29sb3JzIDEgL0NvbHVtbnMgMTUzIC9QcmVkaWN0b3IgMTAgPj4KL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0hlaWdodCAxNTMgL0xlbmd0aCAzOCAwIFIgL1N1YnR5cGUgL0ltYWdlCi9UeXBlIC9YT2JqZWN0IC9XaWR0aCAxNTMgPj4Kc3RyZWFtCnic7c7BCcAwEMCwtPsP3W++hguUIA1gvBYAAAAAAAAAAAAA13hGvLuZpLPOWeesc9Y565x1Q2cHOOucdc46Z52zzlnnrHPWOeucdc46Z52zzlnnrHPWOeucdc46Z52zzlnnrHPWOeucdc46Z52zzlnnrHPWOeucdf89+wBsPzJFCmVuZHN0cmVhbQplbmRvYmoKMzggMCBvYmoKMTI4CmVuZG9iagoyIDAgb2JqCjw8IC9Db3VudCAxIC9LaWRzIFsgMTEgMCBSIF0gL1R5cGUgL1BhZ2VzID4+CmVuZG9iagozOSAwIG9iago8PCAvQ3JlYXRpb25EYXRlIChEOjIwMjIwNTAzMDMxNjE1KzAyJzAwJykKL0NyZWF0b3IgKE1hdHBsb3RsaWIgdjMuNS4xLCBodHRwczovL21hdHBsb3RsaWIub3JnKQovUHJvZHVjZXIgKE1hdHBsb3RsaWIgcGRmIGJhY2tlbmQgdjMuNS4xKSA+PgplbmRvYmoKeHJlZgowIDQwCjAwMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAwMDAxNiAwMDAwMCBuIAowMDAwMDA4MTU2IDAwMDAwIG4gCjAwMDAwMDY5ODYgMDAwMDAgbiAKMDAwMDAwNzAxOCAwMDAwMCBuIAowMDAwMDA3MTE3IDAwMDAwIG4gCjAwMDAwMDcxMzggMDAwMDAgbiAKMDAwMDAwNzE1OSAwMDAwMCBuIAowMDAwMDAwMDY1IDAwMDAwIG4gCjAwMDAwMDAzNDMgMDAwMDAgbiAKMDAwMDAwMDcxOSAwMDAwMCBuIAowMDAwMDAwMjA4IDAwMDAwIG4gCjAwMDAwMDA2OTkgMDAwMDAgbiAKMDAwMDAwNzIwMiAwMDAwMCBuIAowMDAwMDA3NzU0IDAwMDAwIG4gCjAwMDAwMDU3MjcgMDAwMDAgbiAKMDAwMDAwNTUyMCAwMDAwMCBuIAowMDAwMDA1MTEyIDAwMDAwIG4gCjAwMDAwMDY3ODAgMDAwMDAgbiAKMDAwMDAwMDczOSAwMDAwMCBuIAowMDAwMDAxMDc2IDAwMDAwIG4gCjAwMDAwMDEyNDAgMDAwMDAgbiAKMDAwMDAwMTYyMCAwMDAwMCBuIAowMDAwMDAxOTI1IDAwMDAwIG4gCjAwMDAwMDIyMjkgMDAwMDAgbiAKMDAwMDAwMjU1MSAwMDAwMCBuIAowMDAwMDAyNzYwIDAwMDAwIG4gCjAwMDAwMDMxNzQgMDAwMDAgbiAKMDAwMDAwMzQxMSAwMDAwMCBuIAowMDAwMDAzNTU1IDAwMDAwIG4gCjAwMDAwMDM2NzQgMDAwMDAgbiAKMDAwMDAwMzkxMCAwMDAwMCBuIAowMDAwMDA0MjIyIDAwMDAwIG4gCjAwMDAwMDQ0NTUgMDAwMDAgbiAKMDAwMDAwNDU0NSAwMDAwMCBuIAowMDAwMDA0NzUxIDAwMDAwIG4gCjAwMDAwMDQ4OTggMDAwMDAgbiAKMDAwMDAwNzczNCAwMDAwMCBuIAowMDAwMDA4MTM2IDAwMDAwIG4gCjAwMDAwMDgyMTYgMDAwMDAgbiAKdHJhaWxlcgo8PCAvSW5mbyAzOSAwIFIgL1Jvb3QgMSAwIFIgL1NpemUgNDAgPj4Kc3RhcnR4cmVmCjgzNzMKJSVFT0YK\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2022-05-03T03:16:15.691029\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.5.1, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}], "source": ["show_center_recep_field(inp_img, vert_img)"]}, {"cell_type": "markdown", "id": "022312f7", "metadata": {"papermill": {"duration": 0.013688, "end_time": "2022-05-03T01:16:15.799489", "exception": false, "start_time": "2022-05-03T01:16:15.785801", "status": "completed"}, "tags": []}, "source": ["As we have discussed before, the vertical stack only looks at pixels above the one we want to predict.\n", "Hence, we can validate that our implementation works as we initially expected it to.\n", "As a final step, let's clean up the computation graph we still had kept\n", "in memory for the visualization of the receptive field:"]}, {"cell_type": "code", "execution_count": 14, "id": "09b45e18", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:16:15.828100Z", "iopub.status.busy": "2022-05-03T01:16:15.827794Z", "iopub.status.idle": "2022-05-03T01:16:15.831556Z", "shell.execute_reply": "2022-05-03T01:16:15.830958Z"}, "papermill": {"duration": 0.019775, "end_time": "2022-05-03T01:16:15.833022", "exception": false, "start_time": "2022-05-03T01:16:15.813247", "status": "completed"}, "tags": []}, "outputs": [], "source": ["del inp_img, horiz_conv, vert_conv"]}, {"cell_type": "markdown", "id": "85dd0683", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.013595, "end_time": "2022-05-03T01:16:15.860177", "exception": false, "start_time": "2022-05-03T01:16:15.846582", "status": "completed"}, "tags": []}, "source": ["## Gated PixelCNN\n", "\n", "
\n", "\n", "In the next step, we will use the masked convolutions to build a full autoregressive model, called Gated PixelCNN.\n", "The difference between the original PixelCNN and Gated PixelCNN is the use of separate horizontal and vertical stacks.\n", "However, in literature, you often see that people refer to the Gated PixelCNN simply as \"PixelCNN\".\n", "Hence, in the following, if we say \"PixelCNN\", we usually mean the gated version.\n", "What \"Gated\" refers to in the model name is explained next.\n", "\n", "### Gated Convolutions\n", "\n", "For visualizing the receptive field, we assumed a very simplified stack of vertical and horizontal convolutions.\n", "Obviously, there are more sophisticated ways of doing it, and PixelCNN uses gated convolutions for this.\n", "Specifically, the Gated Convolution block in PixelCNN looks as follows\n", "(figure credit - [Aaron van den Oord et al. ](https://arxiv.org/pdf/1606.05328.pdf)):\n", "\n", "
\n", "\n", "The left path is the vertical stack (the $N\\times N$ convolution is masked correspondingly),\n", "and the right path is the horizontal stack.\n", "Gated convolutions are implemented by having a twice as large output channel size,\n", "and combine them by a element-wise multiplication of $\\tanh$ and a sigmoid.\n", "For a linear layer, we can express a gated activation unit as follows:\n", "\n", "$$\\mathbf{y} = \\tanh\\left(\\mathbf{W}_{f}\\mathbf{x}\\right)\\odot\\sigma\\left(\\mathbf{W}_{g}\\mathbf{x}\\right)$$\n", "\n", "For simplicity, biases have been neglected and the linear layer split into two part, $\\mathbf{W}_{f}$ and $\\mathbf{W}_{g}$.\n", "This concept resembles the input and modulation gate in an LSTM, and has been used in many other architectures as well.\n", "The main motivation behind this gated activation is that it might allow to model more complex interactions and simplifies learning.\n", "But as in any other architecture, this is mostly a design choice and can be considered a hyperparameters.\n", "\n", "Besides the gated convolutions, we also see that the horizontal stack uses a residual connection while the vertical stack does not.\n", "This is because we use the output of the horizontal stack for prediction.\n", "Each convolution in the vertical stack also receives a strong gradient signal\n", "as it is only two $1\\times 1$ convolutions away from the residual connection,\n", "and does not require another residual connection to all its earleri layers.\n", "\n", "The implementation in PyTorch is fairly straight forward for this block,\n", "because the visualization above gives us a computation graph to follow:"]}, {"cell_type": "code", "execution_count": 15, "id": "7ce9d372", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:16:15.888835Z", "iopub.status.busy": "2022-05-03T01:16:15.888354Z", "iopub.status.idle": "2022-05-03T01:16:15.895014Z", "shell.execute_reply": "2022-05-03T01:16:15.894448Z"}, "lines_to_next_cell": 2, "papermill": {"duration": 0.022755, "end_time": "2022-05-03T01:16:15.896504", "exception": false, "start_time": "2022-05-03T01:16:15.873749", "status": "completed"}, "tags": []}, "outputs": [], "source": ["class GatedMaskedConv(nn.Module):\n", " def __init__(self, c_in, **kwargs):\n", " \"\"\"Gated Convolution block implemented the computation graph shown above.\"\"\"\n", " super().__init__()\n", " self.conv_vert = VerticalStackConvolution(c_in, c_out=2 * c_in, **kwargs)\n", " self.conv_horiz = HorizontalStackConvolution(c_in, c_out=2 * c_in, **kwargs)\n", " self.conv_vert_to_horiz = nn.Conv2d(2 * c_in, 2 * c_in, kernel_size=1, padding=0)\n", " self.conv_horiz_1x1 = nn.Conv2d(c_in, c_in, kernel_size=1, padding=0)\n", "\n", " def forward(self, v_stack, h_stack):\n", " # Vertical stack (left)\n", " v_stack_feat = self.conv_vert(v_stack)\n", " v_val, v_gate = v_stack_feat.chunk(2, dim=1)\n", " v_stack_out = torch.tanh(v_val) * torch.sigmoid(v_gate)\n", "\n", " # Horizontal stack (right)\n", " h_stack_feat = self.conv_horiz(h_stack)\n", " h_stack_feat = h_stack_feat + self.conv_vert_to_horiz(v_stack_feat)\n", " h_val, h_gate = h_stack_feat.chunk(2, dim=1)\n", " h_stack_feat = torch.tanh(h_val) * torch.sigmoid(h_gate)\n", " h_stack_out = self.conv_horiz_1x1(h_stack_feat)\n", " h_stack_out = h_stack_out + h_stack\n", "\n", " return v_stack_out, h_stack_out"]}, {"cell_type": "markdown", "id": "c6b07015", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.013417, "end_time": "2022-05-03T01:16:15.923800", "exception": false, "start_time": "2022-05-03T01:16:15.910383", "status": "completed"}, "tags": []}, "source": ["### Building the model\n", "\n", "Using the gated convolutions, we can now build our PixelCNN model.\n", "The architecture consists of multiple stacked GatedMaskedConv blocks, where we add an additional dilation factor to a few convolutions.\n", "This is used to increase the receptive field of the model and allows to take a larger context into accout during generation.\n", "As a reminder, dilation on a convolution works looks as follows\n", "(figure credit - [Vincent Dumoulin and Francesco Visin](https://arxiv.org/pdf/1603.07285.pdf)):\n", "\n", "
\n", "\n", "Note that the smaller output size is only because the animation assumes no padding.\n", "In our implementation, we will pad the input image correspondingly.\n", "Alternatively to dilated convolutions, we could downsample the input and use a encoder-decoder architecture as in PixelCNN++ [3].\n", "This is especially beneficial if we want to build a very deep autoregressive model.\n", "Nonetheless, as we seek to train a reasonably small model, dilated convolutions are the more efficient option to use here.\n", "\n", "Below, we implement the PixelCNN model as a PyTorch Lightning module.\n", "Besides the stack of gated convolutions, we also have the initial\n", "horizontal and vertical convolutions which mask the center pixel, and a\n", "final $1\\times 1$ convolution which maps the output features to class\n", "predictions. To determine the likelihood of a batch of images, we first\n", "create our initial features using the masked horizontal and vertical\n", "input convolution. Next, we forward the features through the stack of\n", "gated convolutions. Finally, we take the output features of the\n", "horizontal stack, and apply the $1\\times 1$ convolution for\n", "classification. We use the bits per dimension metric for the likelihood,\n", "similarly to Tutorial 11 and assignment 3."]}, {"cell_type": "code", "execution_count": 16, "id": "a21254d8", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:16:15.954994Z", "iopub.status.busy": "2022-05-03T01:16:15.952612Z", "iopub.status.idle": "2022-05-03T01:16:15.968528Z", "shell.execute_reply": "2022-05-03T01:16:15.967872Z"}, "papermill": {"duration": 0.032274, "end_time": "2022-05-03T01:16:15.970151", "exception": false, "start_time": "2022-05-03T01:16:15.937877", "status": "completed"}, "tags": []}, "outputs": [], "source": ["class PixelCNN(pl.LightningModule):\n", " def __init__(self, c_in, c_hidden):\n", " super().__init__()\n", " self.save_hyperparameters()\n", "\n", " # Initial convolutions skipping the center pixel\n", " self.conv_vstack = VerticalStackConvolution(c_in, c_hidden, mask_center=True)\n", " self.conv_hstack = HorizontalStackConvolution(c_in, c_hidden, mask_center=True)\n", " # Convolution block of PixelCNN. We use dilation instead of downscaling\n", " self.conv_layers = nn.ModuleList(\n", " [\n", " GatedMaskedConv(c_hidden),\n", " GatedMaskedConv(c_hidden, dilation=2),\n", " GatedMaskedConv(c_hidden),\n", " GatedMaskedConv(c_hidden, dilation=4),\n", " GatedMaskedConv(c_hidden),\n", " GatedMaskedConv(c_hidden, dilation=2),\n", " GatedMaskedConv(c_hidden),\n", " ]\n", " )\n", " # Output classification convolution (1x1)\n", " self.conv_out = nn.Conv2d(c_hidden, c_in * 256, kernel_size=1, padding=0)\n", "\n", " self.example_input_array = train_set[0][0][None]\n", "\n", " def forward(self, x):\n", " \"\"\"Forward image through model and return logits for each pixel.\n", "\n", " Args:\n", " x: Image tensor with integer values between 0 and 255.\n", " \"\"\"\n", " # Scale input from 0 to 255 back to -1 to 1\n", " x = (x.float() / 255.0) * 2 - 1\n", "\n", " # Initial convolutions\n", " v_stack = self.conv_vstack(x)\n", " h_stack = self.conv_hstack(x)\n", " # Gated Convolutions\n", " for layer in self.conv_layers:\n", " v_stack, h_stack = layer(v_stack, h_stack)\n", " # 1x1 classification convolution\n", " # Apply ELU before 1x1 convolution for non-linearity on residual connection\n", " out = self.conv_out(F.elu(h_stack))\n", "\n", " # Output dimensions: [Batch, Classes, Channels, Height, Width]\n", " out = out.reshape(out.shape[0], 256, out.shape[1] // 256, out.shape[2], out.shape[3])\n", " return out\n", "\n", " def calc_likelihood(self, x):\n", " # Forward pass with bpd likelihood calculation\n", " pred = self.forward(x)\n", " nll = F.cross_entropy(pred, x, reduction=\"none\")\n", " bpd = nll.mean(dim=[1, 2, 3]) * np.log2(np.exp(1))\n", " return bpd.mean()\n", "\n", " @torch.no_grad()\n", " def sample(self, img_shape, img=None):\n", " \"\"\"Sampling function for the autoregressive model.\n", "\n", " Args:\n", " img_shape: Shape of the image to generate (B,C,H,W)\n", " img (optional): If given, this tensor will be used as\n", " a starting image. The pixels to fill\n", " should be -1 in the input tensor.\n", " \"\"\"\n", " # Create empty image\n", " if img is None:\n", " img = torch.zeros(img_shape, dtype=torch.long).to(device) - 1\n", " # Generation loop\n", " for h in tqdm(range(img_shape[2]), leave=False):\n", " for w in range(img_shape[3]):\n", " for c in range(img_shape[1]):\n", " # Skip if not to be filled (-1)\n", " if (img[:, c, h, w] != -1).all().item():\n", " continue\n", " # For efficiency, we only have to input the upper part of the image\n", " # as all other parts will be skipped by the masked convolutions anyways\n", " pred = self.forward(img[:, :, : h + 1, :])\n", " probs = F.softmax(pred[:, :, c, h, w], dim=-1)\n", " img[:, c, h, w] = torch.multinomial(probs, num_samples=1).squeeze(dim=-1)\n", " return img\n", "\n", " def configure_optimizers(self):\n", " optimizer = optim.Adam(self.parameters(), lr=1e-3)\n", " scheduler = optim.lr_scheduler.StepLR(optimizer, 1, gamma=0.99)\n", " return [optimizer], [scheduler]\n", "\n", " def training_step(self, batch, batch_idx):\n", " loss = self.calc_likelihood(batch[0])\n", " self.log(\"train_bpd\", loss)\n", " return loss\n", "\n", " def validation_step(self, batch, batch_idx):\n", " loss = self.calc_likelihood(batch[0])\n", " self.log(\"val_bpd\", loss)\n", "\n", " def test_step(self, batch, batch_idx):\n", " loss = self.calc_likelihood(batch[0])\n", " self.log(\"test_bpd\", loss)"]}, {"cell_type": "markdown", "id": "b8b2d450", "metadata": {"papermill": {"duration": 0.013553, "end_time": "2022-05-03T01:16:15.998517", "exception": false, "start_time": "2022-05-03T01:16:15.984964", "status": "completed"}, "tags": []}, "source": ["To sample from the autoregressive model, we need to iterate over all dimensions of the input.\n", "We start with an empty image, and fill the pixels one by one, starting from the upper left corner.\n", "Note that as for predicting $x_i$, all pixels below it have no influence on the prediction.\n", "Hence, we can cut the image in height without changing the prediction while increasing efficiency.\n", "Nevertheless, all the loops in the sampling function already show that it will take us quite some time to sample.\n", "A lot of computation could be reused across loop iterations as those the features on the already predicted pixels will not change over iterations.\n", "Nevertheless, this takes quite some effort to implement, and is often not done in implementations because in the end,\n", "autoregressive sampling remains sequential and slow.\n", "Hence, we settle with the default implementation here.\n", "\n", "Before training the model, we can check the full receptive field of the model on an MNIST image of size $28\\times 28$:"]}, {"cell_type": "code", "execution_count": 17, "id": "bc7072bd", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:16:16.027903Z", "iopub.status.busy": "2022-05-03T01:16:16.027590Z", "iopub.status.idle": "2022-05-03T01:16:16.314450Z", "shell.execute_reply": "2022-05-03T01:16:16.313835Z"}, "papermill": {"duration": 0.303095, "end_time": "2022-05-03T01:16:16.316094", "exception": false, "start_time": "2022-05-03T01:16:16.012999", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUiAvTWVkaWFCb3ggWyAwIDAgMzQ5LjIgMTgxLjcwNjgxODE4MTggXQovUGFyZW50IDIgMCBSIC9SZXNvdXJjZXMgOCAwIFIgL1R5cGUgL1BhZ2UgPj4KZW5kb2JqCjkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMiAwIFIgPj4Kc3RyZWFtCnicnVFLTwIxEL7Pr/iOcLA701K2PYIo0Ru6CQfjwSwFlgDKIxD/vbMKPvAidjLpzPSb19esl3ZVme76XVzeU/bllRsSzCjrCCYbMGaqewj6+AlijS/ItaKxas0PlgQxObeDBA3xqVvjpkRLWiFXcK3irVHUQU7ddcIQS2QdWw+jc6nutUxfK4h3qEvWd65SLpDdCHrPGNAAq2MSY/I9sfZppSsyLjQXwoad49j2kSOk7Y37al8uqFsgu1aYRTF+36AY0QMawyZ8MD663KORqsl0m0ZYN2HZBHc8+lKml221SxhXaT5q4hHFLV0V9DGhhGg+m/2bDvpNh0Q+EmLPJcSyfpm3ocVKiPydkW61fFq/nscBvQHN+pCiCmVuZHN0cmVhbQplbmRvYmoKMTIgMCBvYmoKMjgxCmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjE5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjY0ID4+CnN0cmVhbQp4nD1SuZEDMQzLtwqWwF9SPb65cWD3nx7APTsxMUsRAEH3blE5Gz9tJcuP/NhlZ0nklvegbfK6ouoGqZsgc8lxyRVilvK4EiQWLWUm1meqg/JxDWoVvvATMxPTIUnEEnJSjyKsj6+D1/Uc3F836fwaGR90bFB6S5KroFTDvnK0EjuW6l2D7IOmm9LqM9Ou7ICl8Yas1KROfx31vyMswqTeSEUHIRa/08sdsiykHNTbuX4D4V3tglhPrTzsEGneVnpLFczmzARMGW07rIC1WWPCVAnwFs0h1FqIzBc6rYj55NzRwTRVeRoiw1KFg+09ozVc6BoSS8P5ScYE5r/w+qzIbX//AHBoZ7oKZW5kc3RyZWFtCmVuZG9iagoyMCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDkyID4+CnN0cmVhbQp4nD2NwQ3AMAgD/0zBCBACxPtUVR/p/t8mEeoHHwbZGGBhszXgwdnAl9LaN72kRZPaCFa1Rd1QnrsUpVhdR6VMwk+ZO39SdBztcA7b39blOE3j6F/30P0BD0oeCwplbmRzdHJlYW0KZW5kb2JqCjIxIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzA3ID4+CnN0cmVhbQp4nD2SS24DMQxD9z6FLhDA+tme86Qoupjef9snJemKHNkWRWqWukxZUx6QNJOEf+nwcLGd8jtsz2Zm4Fqil4nllOfQFWLuonzZzEZdWSfF6oRmOrfoUTkXBzZNqp+rLKXdLngO1yaeW/YRP7zQoB7UNS4JN3RXo2UpNGOq+3/Se/yMMuBqTF1sUqt7HzxeRFXo6AdHiSJjlxfn40EJ6UrCaFqIlXdFA0Hu8rTKewnu295qyLIHqZjOOylmsOt0Ui5uF4chHsjyqPDlo9hrQs/4sCsl9EjYhjNyJ+5oxubUyOKQ/t6NBEuPrmgh8+CvbtYuYLxTOkViZE5yrGmLVU73UBTTucO9DBD1bEVDKXOR1epfw84La5ZsFnhK+gUeo90mSw5W2duoTu+tPNnQ9x9a13QfCmVuZHN0cmVhbQplbmRvYmoKMjIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMzIgPj4Kc3RyZWFtCnicNVFJbsQwDLv7FfzAANbuvCfFoIf2/9dSyhQIQCW2uCViYyMCLzH4OYjc+JI1oyZ+Z3JX/CxPhUfCreBJFIGX4V52gssbxmU/DjMfvJdWzqTGkwzIRTY9PBEy2CUQOjC7BnXYZtqJviHhsyNSzUaW09cS9NIqBMpTtt/pghJtq/pz+6wLbfvaE052e+pJ5ROI55aswGXjFZPFWAY9UblLMX2Q6myhJ6G8KJ+DbD5qiESXKGfgicHBKNAO7LntZ+JVIWhd3adtY6hGSsfTvw1NTZII+UQJZ7Y07hb+f8+9vtf7D04hVBEKZW5kc3RyZWFtCmVuZG9iagoyMyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIzMSA+PgpzdHJlYW0KeJw1TzmSBCEMy3mFPjBVGNtAv6entjbY+X+6kplOkPAhydMTHZl4mSMjsGbH21pkIGbgU0zFv/a0DxOq9+AeIpSLC2GGkXDWrONuno4X/3aVz1gH7zb4illeENjCTNZXFmcu2wVjaZzEOclujF0TsY11radTWEcwoQyEdLbDlCBzVKT0yY4y5ug4kSeei+/22yx2OX4O6ws2jSEV5/gqeoI2g6Lsee8CGnJB/13d+B5Fu+glIBsJFtZRYu6c5YRfvXZ0HrUoEnNCmkEuEyHN6SqmEJpQrLOjoFJRcKk+p+isn3/lX1wtCmVuZHN0cmVhbQplbmRvYmoKMjQgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNDkgPj4Kc3RyZWFtCnicPVA7jkQhDOs5hS/wJPIjcB5Gqy1m79+uA5opUEx+tjMk0BGBRwwxlK/jJa2groG/i0LxbuLrg8Igq0NSIM56D4h07KY2kRM6HZwzP2E3Y47ARTEGnOl0pj0HJjn7wgqEcxtl7FZIJ4mqIo7qM44pnip7n3gWLO3INlsnkj3kIOFSUonJpZ+Uyj9typQKOmbRBCwSueBkE004y7tJUowZlDLqHqZ2In2sPMijOuhkTc6sI5nZ00/bmfgccLdf2mROlcd0Hsz4nLTOgzkVuvfjiTYHTY3a6Oz3E2kqL1K7HVqdfnUSld0Y5xgSl2d/Gd9k//kH/odaIgplbmRzdHJlYW0KZW5kb2JqCjI1IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTM2ID4+CnN0cmVhbQp4nE2PQQ4DMQgD73mFn0AgQHjPVlUP2/9fS9h20wseyYBsUQaBJYd4hxvh0dsP30U2FWfjnF9SKWIhmE9wnzBTHI0pd/Jjj4BxlGosp2h4XkvOTcMXLXcTLaWtl5MZb7jul/dHlW2RDUXPLQtC12yS+TKBB3wYmEd142mlx932bK/2/ADObDRJCmVuZHN0cmVhbQplbmRvYmoKMjYgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzNDEgPj4Kc3RyZWFtCnicRVJLbkQxCNu/U3CBSOGXkPO0qrqY3n9bm0zVzeAJYGx4y1OmZMqwuSUjJNeUT30iQ6ym/DRyJCKm+EkJBXaVj8drS6yN7JGoFJ/a8eOx9Eam2RVa9e7Rpc2iUc3KyDnIEKGeFbqye9QO2fB6XEi675TNIRzL/1CBLGXdcgolQVvQd+wR3w8droIrgmGway6D7WUy1P/6hxZc7333YscugBas577BDgCopxO0BcgZ2u42KWgAVbqLScKj8npudqJso1Xp+RwAMw4wcsCIJVsdvtHeAJZ9XehFjYr9K0BRWUD8yNV2wd4xyUhwFuYGjr1wPMWZcEs4xgJAir3iGHrwJdjmL1euiJrwCXW6ZC+8wp7a5udCkwh3rQAOXmTDraujqJbt6TyC9mdFckaM1Is4OiGSWtI5guLSoB5a41w3seJtI7G5V9/uH+GcL1z26xdL7ITECmVuZHN0cmVhbQplbmRvYmoKMjcgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNjQgPj4Kc3RyZWFtCnicRZDHcQUxDEPvqgIlMIAK9azH8w/r/q+G9NNBehhCDGJPwrBcV3FhdMOPty0zDX9HGe7G+jJjvNVYICfoAwyRiavRpPp2xRmq9OTVYq6jolwvOiISzJLjq0AjfDqyx5O2tjP9dF4f7CHvE/8qKuduYQEuqu5A+VIf8dSP2VHqmqGPKitrHmraV4RdEUrbPi6nMk7dvQNa4b2Vqz3a7z8edjryCmVuZHN0cmVhbQplbmRvYmoKMjggMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA3MiA+PgpzdHJlYW0KeJwzMrdQMFCwNAEShhYmCuZmBgophlxAvqmJuUIuF0gMxMoBswyAtCWcgohngJggbRDFIBZEsZmJGUQdnAGRy+BKAwAl2xbJCmVuZHN0cmVhbQplbmRvYmoKMjkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA0NyA+PgpzdHJlYW0KeJwzMrdQMFCwNAEShhYmCuZmBgophlyWEFYuF0wsB8wC0ZZwCiKewZUGALlnDScKZW5kc3RyZWFtCmVuZG9iagozMCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE2MyA+PgpzdHJlYW0KeJxFkDsSAyEMQ3tOoSP4IwM+z2YyKTb3b2PYbFLA01ggg7sTgtTagonogoe2Jd0F760EZ2P86TZuNRLkBHWAVqTjaJRSfbnFaZV08Wg2cysLrRMdZg56lKMZoBA6Fd7touRypu7O+UNw9V/1v2LdOZuJgcnKHQjN6lPc+TY7orq6yf6kx9ys134r7FVhaVlLywm3nbtmQAncUznaqz0/Hwo69gplbmRzdHJlYW0KZW5kb2JqCjMxIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjM5ID4+CnN0cmVhbQp4nE1QyW0EMQz7uwo1MMDoHLseB4s8sv1/Q8oJkpdoS+Kh8pRblspl9yM5b8m65UOHTpVp8m7Qza+x/qMMAnb/UFQQrSWxSsxc0m6xNEkv2cM4jZdrtY7nqXuEWaN48OPY0ymB6T0ywWazvTkwqz3ODpBOuMav6tM7lSQDibqQ80KlCuse1CWijyvbmFKdTi3lGJef6Ht8jgA9xd6N3NHHyxeMRrUtqNFqlTgPMBNT0ZVxq5GBlBMGQ2dHVzQLpcjKekI1wo05oZm9w3BgA8uzhKSlrVK8D2UB6AJd2jrjNEqCjgDC3yiM9foGqvxeNwplbmRzdHJlYW0KZW5kb2JqCjMyIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTYwID4+CnN0cmVhbQp4nEWQORIDMQgEc72CJ0hcgvesy7XB+v+pB9ZHoukCNBy6Fk3KehRoPumxRqG60GvoLEqSRMEWkh1Qp2OIOyhITEhjkki2HoMjmlizXZiZVCqzUuG0acXCv9la1chEjXCN/InpBlT8T+pclPBNg6+SMfoYVLw7g4xJ+F5F3Fox7f5EMLEZ9glvRSYFhImxqdm+z2CGzPcK1zjH8w1MgjfrCmVuZHN0cmVhbQplbmRvYmoKMzMgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxOCA+PgpzdHJlYW0KeJwzNrRQMIDDFEOuNAAd5gNSCmVuZHN0cmVhbQplbmRvYmoKMzQgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMzMgPj4Kc3RyZWFtCnicRY9LDgQhCET3nKKOwMcf53Ey6YVz/+2AnW4TYz2FVIG5gqE9LmsDnRUfIRm28beplo5FWT5UelJWD8ngh6zGyyHcoCzwgkkqhiFQi5gakS1lbreA2zYNsrKVU6WOsIujMI/2tGwVHl+iWyJ1kj+DxCov3OO6Hcil1rveoou+f6QBMQkKZW5kc3RyZWFtCmVuZG9iagozNSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDc1ID4+CnN0cmVhbQp4nDO1NFIwUDA2ABKmZkYKpibmCimGXEA+iJXLZWhkCmblcBlZmilYWAAZJmbmUCGYhhwuY1NzoAFARcamYBqqP4crgysNAJWQEu8KZW5kc3RyZWFtCmVuZG9iagozNiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE0MSA+PgpzdHJlYW0KeJw9j8EOwzAIQ+/5Cv9ApNgpoXxPp2qH7v+vI0u7C3oCY4yF0NAbqprDhmCb48XSJVRr+BTFQCU3yJlgDqWk0h1HkXpiOBhcHrQbjuKx6PoRu5JmfdDGQrolaIB7rFNp3KZxE8QdNQXqKeqco7wQuZ+pZ9g0kt00s5JzuA2/e89T1/+nq7zL+QW9dy7+CmVuZHN0cmVhbQplbmRvYmoKMTcgMCBvYmoKPDwgL0Jhc2VGb250IC9CTVFRRFYrRGVqYVZ1U2FucyAvQ2hhclByb2NzIDE4IDAgUgovRW5jb2RpbmcgPDwKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCi9UeXBlIC9FbmNvZGluZyA+PgovRmlyc3RDaGFyIDAgL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvRm9udERlc2NyaXB0b3IgMTYgMCBSCi9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdIC9MYXN0Q2hhciAyNTUgL05hbWUgL0JNUVFEVitEZWphVnVTYW5zCi9TdWJ0eXBlIC9UeXBlMyAvVHlwZSAvRm9udCAvV2lkdGhzIDE1IDAgUiA+PgplbmRvYmoKMTYgMCBvYmoKPDwgL0FzY2VudCA5MjkgL0NhcEhlaWdodCAwIC9EZXNjZW50IC0yMzYgL0ZsYWdzIDMyCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovSXRhbGljQW5nbGUgMCAvTWF4V2lkdGggMTM0MiAvU3RlbVYgMCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL1hIZWlnaHQgMCA+PgplbmRvYmoKMTUgMCBvYmoKWyA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMAo2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDMxOCA0MDEgNDYwIDgzOCA2MzYKOTUwIDc4MCAyNzUgMzkwIDM5MCA1MDAgODM4IDMxOCAzNjEgMzE4IDMzNyA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2CjYzNiA2MzYgMzM3IDMzNyA4MzggODM4IDgzOCA1MzEgMTAwMCA2ODQgNjg2IDY5OCA3NzAgNjMyIDU3NSA3NzUgNzUyIDI5NQoyOTUgNjU2IDU1NyA4NjMgNzQ4IDc4NyA2MDMgNzg3IDY5NSA2MzUgNjExIDczMiA2ODQgOTg5IDY4NSA2MTEgNjg1IDM5MCAzMzcKMzkwIDgzOCA1MDAgNTAwIDYxMyA2MzUgNTUwIDYzNSA2MTUgMzUyIDYzNSA2MzQgMjc4IDI3OCA1NzkgMjc4IDk3NCA2MzQgNjEyCjYzNSA2MzUgNDExIDUyMSAzOTIgNjM0IDU5MiA4MTggNTkyIDU5MiA1MjUgNjM2IDMzNyA2MzYgODM4IDYwMCA2MzYgNjAwIDMxOAozNTIgNTE4IDEwMDAgNTAwIDUwMCA1MDAgMTM0MiA2MzUgNDAwIDEwNzAgNjAwIDY4NSA2MDAgNjAwIDMxOCAzMTggNTE4IDUxOAo1OTAgNTAwIDEwMDAgNTAwIDEwMDAgNTIxIDQwMCAxMDIzIDYwMCA1MjUgNjExIDMxOCA0MDEgNjM2IDYzNiA2MzYgNjM2IDMzNwo1MDAgNTAwIDEwMDAgNDcxIDYxMiA4MzggMzYxIDEwMDAgNTAwIDUwMCA4MzggNDAxIDQwMSA1MDAgNjM2IDYzNiAzMTggNTAwCjQwMSA0NzEgNjEyIDk2OSA5NjkgOTY5IDUzMSA2ODQgNjg0IDY4NCA2ODQgNjg0IDY4NCA5NzQgNjk4IDYzMiA2MzIgNjMyIDYzMgoyOTUgMjk1IDI5NSAyOTUgNzc1IDc0OCA3ODcgNzg3IDc4NyA3ODcgNzg3IDgzOCA3ODcgNzMyIDczMiA3MzIgNzMyIDYxMSA2MDUKNjMwIDYxMyA2MTMgNjEzIDYxMyA2MTMgNjEzIDk4MiA1NTAgNjE1IDYxNSA2MTUgNjE1IDI3OCAyNzggMjc4IDI3OCA2MTIgNjM0CjYxMiA2MTIgNjEyIDYxMiA2MTIgODM4IDYxMiA2MzQgNjM0IDYzNCA2MzQgNTkyIDYzNSA1OTIgXQplbmRvYmoKMTggMCBvYmoKPDwgL0IgMTkgMCBSIC9XIDIwIDAgUiAvYSAyMSAwIFIgL2MgMjIgMCBSIC9kIDIzIDAgUiAvZSAyNCAwIFIgL2YgMjUgMCBSCi9nIDI2IDAgUiAvaCAyNyAwIFIgL2kgMjggMCBSIC9sIDI5IDAgUiAvbiAzMCAwIFIgL3AgMzEgMCBSIC9yIDMyIDAgUgovc3BhY2UgMzMgMCBSIC90IDM0IDAgUiAvdiAzNSAwIFIgL3kgMzYgMCBSID4+CmVuZG9iagozIDAgb2JqCjw8IC9GMSAxNyAwIFIgPj4KZW5kb2JqCjQgMCBvYmoKPDwgL0ExIDw8IC9DQSAwIC9UeXBlIC9FeHRHU3RhdGUgL2NhIDEgPj4KL0EyIDw8IC9DQSAxIC9UeXBlIC9FeHRHU3RhdGUgL2NhIDEgPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgL0kxIDEzIDAgUiAvSTIgMTQgMCBSID4+CmVuZG9iagoxMyAwIG9iago8PCAvQml0c1BlckNvbXBvbmVudCA4Ci9Db2xvclNwYWNlIFsvSW5kZXhlZCAvRGV2aWNlUkdCIDE1ICj95zchO24dOW4EMnAAMXAALGcAK2QAXChcXABcKFsAJlcAJlUAJFIAI1AAI08AIk3/AAApXQovRGVjb2RlUGFybXMgPDwgL0NvbG9ycyAxIC9Db2x1bW5zIDE1MyAvUHJlZGljdG9yIDEwID4+Ci9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9IZWlnaHQgMTUzIC9MZW5ndGggMzcgMCBSIC9TdWJ0eXBlIC9JbWFnZQovVHlwZSAvWE9iamVjdCAvV2lkdGggMTUzID4+CnN0cmVhbQp4nO3Uy0oDQRBA0fh+Rv//b6UriwwEwYtIYjh31U3B9NnU7D4utd25Ad9G1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrXZ/s/djc96vN/W2a+4z2ZGRXJTty9pvnj4bNaOMlIyP7E9lm/eencDg+rx5XB8PL6gRNRvb/ZdPJvo3kYXU/zehpyh8nI7sg2eH519Vhv+Z4O43kbrpZ7Vaf0284ZGRkP5CdN7IeWY+sR9Yj65H1yHpkPbIeWY+sR9Yj65H1yHpkPbIeWY+sR9Yj65H1yHpkPbIeWY+sR9Yj65H1yHpkPbIeWY+s9wUnhO4jCmVuZHN0cmVhbQplbmRvYmoKMzcgMCBvYmoKMjYyCmVuZG9iagoxNCAwIG9iago8PCAvQml0c1BlckNvbXBvbmVudCA4IC9Db2xvclNwYWNlIFsvSW5kZXhlZCAvRGV2aWNlUkdCIDIgKP3nNwAiTf8AACldCi9EZWNvZGVQYXJtcyA8PCAvQ29sb3JzIDEgL0NvbHVtbnMgMTUzIC9QcmVkaWN0b3IgMTAgPj4KL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0hlaWdodCAxNTMgL0xlbmd0aCAzOCAwIFIgL1N1YnR5cGUgL0ltYWdlCi9UeXBlIC9YT2JqZWN0IC9XaWR0aCAxNTMgPj4Kc3RyZWFtCnic7c6xCcAwEAAxO/sPHUif4sDgL6QJtPZU63bgl1k3fbYmMuvMOrPOrDPrzDqzzqwz68w6s86sM+vMOrPOrDPrzDqzzqwz68w6s86sM+vMOrPOrDPrzDqzzqwz68w6s+7k7PnsM8zMzGYy68w6s86sM+vMOrPOrDPrzDqzzqwz68w6s86sM+vMOrPOrDPrzDqzzqwz68w6s86sM+vMOrPOrDPrzDqz7gU7RDDgCmVuZHN0cmVhbQplbmRvYmoKMzggMCBvYmoKMTczCmVuZG9iagoyIDAgb2JqCjw8IC9Db3VudCAxIC9LaWRzIFsgMTEgMCBSIF0gL1R5cGUgL1BhZ2VzID4+CmVuZG9iagozOSAwIG9iago8PCAvQ3JlYXRpb25EYXRlIChEOjIwMjIwNTAzMDMxNjE2KzAyJzAwJykKL0NyZWF0b3IgKE1hdHBsb3RsaWIgdjMuNS4xLCBodHRwczovL21hdHBsb3RsaWIub3JnKQovUHJvZHVjZXIgKE1hdHBsb3RsaWIgcGRmIGJhY2tlbmQgdjMuNS4xKSA+PgplbmRvYmoKeHJlZgowIDQwCjAwMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAwMDAxNiAwMDAwMCBuIAowMDAwMDA4MjI4IDAwMDAwIG4gCjAwMDAwMDY5ODYgMDAwMDAgbiAKMDAwMDAwNzAxOCAwMDAwMCBuIAowMDAwMDA3MTE3IDAwMDAwIG4gCjAwMDAwMDcxMzggMDAwMDAgbiAKMDAwMDAwNzE1OSAwMDAwMCBuIAowMDAwMDAwMDY1IDAwMDAwIG4gCjAwMDAwMDAzNDMgMDAwMDAgbiAKMDAwMDAwMDcxOSAwMDAwMCBuIAowMDAwMDAwMjA4IDAwMDAwIG4gCjAwMDAwMDA2OTkgMDAwMDAgbiAKMDAwMDAwNzIwMiAwMDAwMCBuIAowMDAwMDA3NzgxIDAwMDAwIG4gCjAwMDAwMDU3MjcgMDAwMDAgbiAKMDAwMDAwNTUyMCAwMDAwMCBuIAowMDAwMDA1MTEyIDAwMDAwIG4gCjAwMDAwMDY3ODAgMDAwMDAgbiAKMDAwMDAwMDczOSAwMDAwMCBuIAowMDAwMDAxMDc2IDAwMDAwIG4gCjAwMDAwMDEyNDAgMDAwMDAgbiAKMDAwMDAwMTYyMCAwMDAwMCBuIAowMDAwMDAxOTI1IDAwMDAwIG4gCjAwMDAwMDIyMjkgMDAwMDAgbiAKMDAwMDAwMjU1MSAwMDAwMCBuIAowMDAwMDAyNzYwIDAwMDAwIG4gCjAwMDAwMDMxNzQgMDAwMDAgbiAKMDAwMDAwMzQxMSAwMDAwMCBuIAowMDAwMDAzNTU1IDAwMDAwIG4gCjAwMDAwMDM2NzQgMDAwMDAgbiAKMDAwMDAwMzkxMCAwMDAwMCBuIAowMDAwMDA0MjIyIDAwMDAwIG4gCjAwMDAwMDQ0NTUgMDAwMDAgbiAKMDAwMDAwNDU0NSAwMDAwMCBuIAowMDAwMDA0NzUxIDAwMDAwIG4gCjAwMDAwMDQ4OTggMDAwMDAgbiAKMDAwMDAwNzc2MSAwMDAwMCBuIAowMDAwMDA4MjA4IDAwMDAwIG4gCjAwMDAwMDgyODggMDAwMDAgbiAKdHJhaWxlcgo8PCAvSW5mbyAzOSAwIFIgL1Jvb3QgMSAwIFIgL1NpemUgNDAgPj4Kc3RhcnR4cmVmCjg0NDUKJSVFT0YK\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2022-05-03T03:16:16.236555\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.5.1, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}], "source": ["test_model = PixelCNN(c_in=1, c_hidden=64)\n", "inp = torch.zeros(1, 1, 28, 28)\n", "inp.requires_grad_()\n", "out = test_model(inp)\n", "show_center_recep_field(inp, out.squeeze(dim=2))\n", "del inp, out, test_model"]}, {"cell_type": "markdown", "id": "0520e95d", "metadata": {"papermill": {"duration": 0.014284, "end_time": "2022-05-03T01:16:16.346211", "exception": false, "start_time": "2022-05-03T01:16:16.331927", "status": "completed"}, "tags": []}, "source": ["The visualization shows that for predicting any pixel, we can take almost half of the image into account.\n", "However, keep in mind that this is the \"theoretical\" receptive field and not necessarily\n", "the [effective receptive field](https://arxiv.org/pdf/1701.04128.pdf), which is usually much smaller.\n", "For a stronger model, we should therefore try to increase the receptive\n", "field even further. Especially, for the pixel on the bottom right, the\n", "very last pixel, we would be allowed to take into account the whole\n", "image. However, our current receptive field only spans across 1/4 of the\n", "image. An encoder-decoder architecture can help with this, but it also\n", "shows that we require a much deeper, more complex network in\n", "autoregressive models than in VAEs or energy-based models."]}, {"cell_type": "markdown", "id": "5b76944b", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.014534, "end_time": "2022-05-03T01:16:16.375409", "exception": false, "start_time": "2022-05-03T01:16:16.360875", "status": "completed"}, "tags": []}, "source": ["### Training loop\n", "\n", "To train the model, we again can rely on PyTorch Lightning and write a\n", "function below for loading the pretrained model if it exists. To reduce\n", "the computational cost, we have saved the validation and test score in\n", "the checkpoint already:"]}, {"cell_type": "code", "execution_count": 18, "id": "da25cba4", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:16:16.405985Z", "iopub.status.busy": "2022-05-03T01:16:16.405670Z", "iopub.status.idle": "2022-05-03T01:16:16.412607Z", "shell.execute_reply": "2022-05-03T01:16:16.411988Z"}, "papermill": {"duration": 0.024108, "end_time": "2022-05-03T01:16:16.414064", "exception": false, "start_time": "2022-05-03T01:16:16.389956", "status": "completed"}, "tags": []}, "outputs": [], "source": ["def train_model(**kwargs):\n", " # Create a PyTorch Lightning trainer with the generation callback\n", " trainer = pl.Trainer(\n", " default_root_dir=os.path.join(CHECKPOINT_PATH, \"PixelCNN\"),\n", " gpus=1 if str(device).startswith(\"cuda\") else 0,\n", " max_epochs=150,\n", " callbacks=[\n", " ModelCheckpoint(save_weights_only=True, mode=\"min\", monitor=\"val_bpd\"),\n", " LearningRateMonitor(\"epoch\"),\n", " ],\n", " )\n", " result = None\n", " # Check whether pretrained model exists. If yes, load it and skip training\n", " pretrained_filename = os.path.join(CHECKPOINT_PATH, \"PixelCNN.ckpt\")\n", " if os.path.isfile(pretrained_filename):\n", " print(\"Found pretrained model, loading...\")\n", " model = PixelCNN.load_from_checkpoint(pretrained_filename)\n", " ckpt = torch.load(pretrained_filename, map_location=device)\n", " result = ckpt.get(\"result\", None)\n", " else:\n", " model = PixelCNN(**kwargs)\n", " trainer.fit(model, train_loader, val_loader)\n", " model = model.to(device)\n", "\n", " if result is None:\n", " # Test best model on validation and test set\n", " val_result = trainer.test(model, dataloaders=val_loader, verbose=False)\n", " test_result = trainer.test(model, dataloaders=test_loader, verbose=False)\n", " result = {\"test\": test_result, \"val\": val_result}\n", " return model, result"]}, {"cell_type": "markdown", "id": "6be5aa47", "metadata": {"papermill": {"duration": 0.014373, "end_time": "2022-05-03T01:16:16.442812", "exception": false, "start_time": "2022-05-03T01:16:16.428439", "status": "completed"}, "tags": []}, "source": ["Training the model is time consuming and we recommend using the provided pre-trained model for going through this notebook.\n", "However, feel free to play around with the hyperparameter like number of layers etc.\n", "if you want to get a feeling for those.\n", "\n", "When calling the training function with a pre-trained model, we automatically load it and print its test performance:"]}, {"cell_type": "code", "execution_count": 19, "id": "fc6b5d9f", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:16:16.473077Z", "iopub.status.busy": "2022-05-03T01:16:16.472533Z", "iopub.status.idle": "2022-05-03T01:16:20.152006Z", "shell.execute_reply": "2022-05-03T01:16:20.151347Z"}, "papermill": {"duration": 3.696163, "end_time": "2022-05-03T01:16:20.153614", "exception": false, "start_time": "2022-05-03T01:16:16.457451", "status": "completed"}, "tags": []}, "outputs": [{"name": "stderr", "output_type": "stream", "text": ["GPU available: True, used: True\n"]}, {"name": "stderr", "output_type": "stream", "text": ["TPU available: False, using: 0 TPU cores\n"]}, {"name": "stderr", "output_type": "stream", "text": ["IPU available: False, using: 0 IPUs\n"]}, {"name": "stderr", "output_type": "stream", "text": ["HPU available: False, using: 0 HPUs\n"]}, {"name": "stdout", "output_type": "stream", "text": ["Found pretrained model, loading...\n"]}, {"name": "stdout", "output_type": "stream", "text": ["Test bits per dimension: 0.808bpd\n"]}], "source": ["model, result = train_model(c_in=1, c_hidden=64)\n", "test_res = result[\"test\"][0]\n", "print(\n", " \"Test bits per dimension: %4.3fbpd\" % (test_res[\"test_loss\"] if \"test_loss\" in test_res else test_res[\"test_bpd\"])\n", ")"]}, {"cell_type": "markdown", "id": "1cc2a507", "metadata": {"papermill": {"duration": 0.014606, "end_time": "2022-05-03T01:16:20.184465", "exception": false, "start_time": "2022-05-03T01:16:20.169859", "status": "completed"}, "tags": []}, "source": ["With a test performance of 0.809bpd, the PixelCNN significantly outperforms the normalizing flows we have seen in Tutorial 11.\n", "Considering image modeling as an autoregressive problem simplifies the learning process as predicting\n", "one pixel given the ground truth of all others is much easier than predicting all pixels at once.\n", "In addition, PixelCNN can explicitly predict the pixel values by a discrete softmax while\n", "Normalizing Flows have to learn transformations in continuous latent space.\n", "These two aspects allow the PixelCNN to achieve a notably better performance.\n", "\n", "To fully compare the models, let's also measure the number of parameters of the PixelCNN:"]}, {"cell_type": "code", "execution_count": 20, "id": "22a8c8ef", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:16:20.214659Z", "iopub.status.busy": "2022-05-03T01:16:20.214367Z", "iopub.status.idle": "2022-05-03T01:16:20.219650Z", "shell.execute_reply": "2022-05-03T01:16:20.219053Z"}, "papermill": {"duration": 0.022285, "end_time": "2022-05-03T01:16:20.221148", "exception": false, "start_time": "2022-05-03T01:16:20.198863", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Number of parameters: 852,160\n"]}], "source": ["num_params = sum(np.prod(param.shape) for param in model.parameters())\n", "print(f\"Number of parameters: {num_params:,}\")"]}, {"cell_type": "markdown", "id": "60077a94", "metadata": {"papermill": {"duration": 0.01484, "end_time": "2022-05-03T01:16:20.250791", "exception": false, "start_time": "2022-05-03T01:16:20.235951", "status": "completed"}, "tags": []}, "source": ["Compared to the multi-scale normalizing flows, the PixelCNN has considerably less parameters.\n", "Of course, the number of parameters depend on our hyperparameter choices.\n", "Nevertheless, in general, it can be said that autoregressive models\n", "require considerably less parameters than normalizing flows to reach\n", "good performance, based on the reasons stated above. Still,\n", "autoregressive models are much slower in sampling than normalizing\n", "flows, which limits their possible applications."]}, {"cell_type": "markdown", "id": "6fa15511", "metadata": {"papermill": {"duration": 0.014581, "end_time": "2022-05-03T01:16:20.280488", "exception": false, "start_time": "2022-05-03T01:16:20.265907", "status": "completed"}, "tags": []}, "source": ["## Sampling\n", "\n", "One way of qualitatively analysing generative models is by looking at the actual samples.\n", "Let's therefore use our sampling function to generate a few digits:"]}, {"cell_type": "code", "execution_count": 21, "id": "6933d2aa", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:16:20.311418Z", "iopub.status.busy": "2022-05-03T01:16:20.310823Z", "iopub.status.idle": "2022-05-03T01:16:24.595059Z", "shell.execute_reply": "2022-05-03T01:16:24.594402Z"}, "papermill": {"duration": 4.301175, "end_time": "2022-05-03T01:16:24.596618", "exception": false, "start_time": "2022-05-03T01:16:20.295443", "status": "completed"}, "tags": []}, "outputs": [{"name": "stderr", "output_type": "stream", "text": ["Global seed set to 1\n"]}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "63e9c4cd6e6244ba94c7ca134d3410f4", "version_major": 2, "version_minor": 0}, "text/plain": [" 0%| | 0/28 [00:00\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2022-05-03T03:16:24.551170\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.5.1, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}], "source": ["pl.seed_everything(1)\n", "samples = model.sample(img_shape=(16, 1, 28, 28))\n", "show_imgs(samples.cpu())"]}, {"cell_type": "markdown", "id": "9677f507", "metadata": {"papermill": {"duration": 0.015192, "end_time": "2022-05-03T01:16:24.628513", "exception": false, "start_time": "2022-05-03T01:16:24.613321", "status": "completed"}, "tags": []}, "source": ["Most of the samples can be identified as digits, and overall we achieve a better quality than we had in normalizing flows.\n", "This goes along with the lower likelihood we achieved with autoregressive models.\n", "Nevertheless, we also see that there is still place for improvement\n", "as a considerable amount of samples cannot be identified (for example the first row).\n", "Deeper autoregressive models are expected to achieve better quality,\n", "as they can take more context into account for generating the pixels.\n", "\n", "Note that on Google Colab, you might see different results, specifically with a white line at the top.\n", "After some debugging, it seemed that the difference occurs inside the dilated convolution,\n", "as it gives different results for different batch sizes.\n", "However, it is hard to debug this further as it might be a bug of the installed PyTorch version on Google Colab.\n", "\n", "The trained model itself is not restricted to any specific image size.\n", "However, what happens if we actually sample a larger image than we had\n", "seen in our training dataset? Let's try below to sample images of size\n", "$64\\times64$ instead of $28\\times28$:"]}, {"cell_type": "code", "execution_count": 22, "id": "195d9493", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:16:24.660131Z", "iopub.status.busy": "2022-05-03T01:16:24.659849Z", "iopub.status.idle": "2022-05-03T01:16:51.855806Z", "shell.execute_reply": "2022-05-03T01:16:51.855151Z"}, "papermill": {"duration": 27.213854, "end_time": "2022-05-03T01:16:51.857599", "exception": false, "start_time": "2022-05-03T01:16:24.643745", "status": "completed"}, "tags": []}, "outputs": [{"name": "stderr", "output_type": "stream", "text": ["Global seed set to 1\n"]}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "77d61b0e1c464cde8b971939de0b889c", "version_major": 2, "version_minor": 0}, "text/plain": [" 0%| | 0/64 [00:00\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2022-05-03T03:16:51.817909\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.5.1, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}], "source": ["pl.seed_everything(1)\n", "samples = model.sample(img_shape=(8, 1, 64, 64))\n", "show_imgs(samples.cpu())"]}, {"cell_type": "markdown", "id": "417f8aae", "metadata": {"papermill": {"duration": 0.015969, "end_time": "2022-05-03T01:16:51.893531", "exception": false, "start_time": "2022-05-03T01:16:51.877562", "status": "completed"}, "tags": []}, "source": ["The larger images show that changing the size of the image during testing confuses the model\n", "and generates abstract figures (you can sometimes spot a digit in the upper left corner).\n", "In addition, sampling for images of 64x64 pixels take more than a minute on a GPU.\n", "Clearly, autoregressive models cannot be scaled to large images without changing the sampling procedure such as with [forecasting](https://arxiv.org/abs/2002.09928).\n", "Our implementation is also not the most efficient as many computations can be stored and reused throughout the sampling process.\n", "Nevertheless, the sampling procedure stays sequential which is\n", "inherently slower than parallel generation like done in normalizing\n", "flows."]}, {"cell_type": "markdown", "id": "a0559a7f", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.016267, "end_time": "2022-05-03T01:16:51.925771", "exception": false, "start_time": "2022-05-03T01:16:51.909504", "status": "completed"}, "tags": []}, "source": ["### Autocompletion\n", "\n", "One common application done with autoregressive models is\n", "auto-completing an image. As autoregressive models predict pixels one by\n", "one, we can set the first $N$ pixels to predefined values and check how\n", "the model completes the image. For implementing this, we just need to\n", "skip the iterations in the sampling loop that already have a value\n", "unequals -1. See above in our PyTorch Lightning module for the specific\n", "implementation. In the cell below, we randomly take three images from\n", "the training set, mask about the lower half of the image, and let the\n", "model autocomplete it. To see the diversity of samples, we do this 12\n", "times for each image:"]}, {"cell_type": "code", "execution_count": 23, "id": "c133356d", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:16:51.959591Z", "iopub.status.busy": "2022-05-03T01:16:51.959235Z", "iopub.status.idle": "2022-05-03T01:16:59.392703Z", "shell.execute_reply": "2022-05-03T01:16:59.392052Z"}, "papermill": {"duration": 7.452237, "end_time": "2022-05-03T01:16:59.394222", "exception": false, "start_time": "2022-05-03T01:16:51.941985", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Original image and input image to sampling:\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUiAvTWVkaWFCb3ggWyAwIDAgMTcyLjM4Mzc1IDk1Ljk0IF0KL1BhcmVudCAyIDAgUiAvUmVzb3VyY2VzIDggMCBSIC9UeXBlIC9QYWdlID4+CmVuZG9iago5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTIgMCBSID4+CnN0cmVhbQp4nFWOsQ7CMAxEd3/FfUESp42SjIVKFWNh4AOiCKhoEVSiv4/TAcpwku90z7Zu8/uW8rHbYX8i/XNpJsZAumFcZhgMogWMDv8lI/lI7K2qQuWduPvGRadiLYnZTKVxJZroCa/sKnZexRUIrFyNV8YZE3Rjy3H5Q7QI2AnDLqAsCfaLpxH6wGgf6KmnD97CL4AKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoxNDQKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMyAwIG9iago8PCA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL0NBIDAgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PgovQTIgPDwgL0NBIDEgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSID4+CmVuZG9iagoxMyAwIG9iago8PCAvQml0c1BlckNvbXBvbmVudCA4Ci9Db2xvclNwYWNlIFsvSW5kZXhlZCAvRGV2aWNlUkdCIDcwICj////9/f37+/v6+vr19fXy8vLw8PDv7+/t7e3s7Ozr6+vo6Ojk5OTj4+Pf39/Y2NjR0dHQ0NDPz8/Hx8fDw8PCwsLBwcG/v7+6urq5ubm0tLSurq6srKylpaWgoKCVlZWTk5OPj4+Li4uKioqIiIiAgIB9fX1ycnJjY2NgYGBfX19WVlZRUVFQUFBPT09MTExHR0dEREQ7Ozs1NTUyMjIwMDAsLCwqKiojIyMiIiIdHR0bGxsaGhoZGRkUFBQTExMSEhIODg4MDAxcblxuXG4GBgYEBAQAAAApXQovRGVjb2RlUGFybXMgPDwgL0NvbG9ycyAxIC9Db2x1bW5zIDE1OCAvUHJlZGljdG9yIDEwID4+Ci9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9IZWlnaHQgODIgL0xlbmd0aCAxNCAwIFIgL1N1YnR5cGUgL0ltYWdlCi9UeXBlIC9YT2JqZWN0IC9XaWR0aCAxNTggPj4Kc3RyZWFtCnic7dfXcoJAGIZh1gY2Yu8K9oJdsDdEc/+XFCBMYkECs2qE+b9DGN55DnYUiPpbj/hvgPGAhzNb8D4fs/rDY8DDiQEPJwY8nBjwcGLAw4kBDycGPJzYfV673SYIgmVv7xyPx9PJGk9r3cbU1v0Y8J7BE8VoNIoQ4riLyzxfq9UoiorFer2eWZ6oxNTWRYxXYmorptMC3vN4mw1S1+mo/d1uN5mUy+VgEGmrVqtmeRst1lFiohKbKLHgecxZPI77fjAQYBiGptHVEglJkszyOC0WUGL0TSyhxID3Qp4k5fN5r9f7W/F4MpkMy94/LK/8z31znrrxeNxsfihrNBqjkXylVJJtPp9vvQaezXlXGw6HFCXz+v2+QRF4+ovH4wj5/f7lcgk8R/IqlYpxEXg25HW7JEkmk4fDAXgWeYIgkKT8i1wo/FkEnt14rVZLfQ0dDIBnlbffy1/yCKVSKb1vFuDZmvdz8AxPHvD0l8vJuFBoPp8Dz1m87XYbici8bNZcEXjn43keoXA4LAjAcxyvWCwilE6nzRaBZxvebOZ2uxFyuVwMs1qtgGeJN52qLys0TS8W5orAsw3PehF4OEXg4RSfwHvbAQ9nwMPZF28utk0KZW5kc3RyZWFtCmVuZG9iagoxNCAwIG9iago0NjMKZW5kb2JqCjIgMCBvYmoKPDwgL0NvdW50IDEgL0tpZHMgWyAxMSAwIFIgXSAvVHlwZSAvUGFnZXMgPj4KZW5kb2JqCjE1IDAgb2JqCjw8IC9DcmVhdGlvbkRhdGUgKEQ6MjAyMjA1MDMwMzE2NTIrMDInMDAnKQovQ3JlYXRvciAoTWF0cGxvdGxpYiB2My41LjEsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcpCi9Qcm9kdWNlciAoTWF0cGxvdGxpYiBwZGYgYmFja2VuZCB2My41LjEpID4+CmVuZG9iagp4cmVmCjAgMTYKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDE3MzUgMDAwMDAgbiAKMDAwMDAwMDU5NyAwMDAwMCBuIAowMDAwMDAwNjE4IDAwMDAwIG4gCjAwMDAwMDA3MTcgMDAwMDAgbiAKMDAwMDAwMDczOCAwMDAwMCBuIAowMDAwMDAwNzU5IDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDMzOCAwMDAwMCBuIAowMDAwMDAwNTc3IDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDU1NyAwMDAwMCBuIAowMDAwMDAwNzkxIDAwMDAwIG4gCjAwMDAwMDE3MTUgMDAwMDAgbiAKMDAwMDAwMTc5NSAwMDAwMCBuIAp0cmFpbGVyCjw8IC9JbmZvIDE1IDAgUiAvUm9vdCAxIDAgUiAvU2l6ZSAxNiA+PgpzdGFydHhyZWYKMTk1MgolJUVPRgo=\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2022-05-03T03:16:51.994794\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.5.1, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}, {"name": "stderr", "output_type": "stream", "text": ["Global seed set to 1\n"]}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "2bc166726f334ce9bcba379172cb2d04", "version_major": 2, "version_minor": 0}, "text/plain": [" 0%| | 0/28 [00:00\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2022-05-03T03:16:54.675693\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.5.1, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}, {"name": "stdout", "output_type": "stream", "text": ["Original image and input image to sampling:\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUiAvTWVkaWFCb3ggWyAwIDAgMTcyLjM4Mzc1IDk1Ljk0IF0KL1BhcmVudCAyIDAgUiAvUmVzb3VyY2VzIDggMCBSIC9UeXBlIC9QYWdlID4+CmVuZG9iago5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTIgMCBSID4+CnN0cmVhbQp4nFWOsQ7CMAxEd3/FfUESp42SjIVKFWNh4AOiCKhoEVSiv4/TAcpwku90z7Zu8/uW8rHbYX8i/XNpJsZAumFcZhgMogWMDv8lI/lI7K2qQuWduPvGRadiLYnZTKVxJZroCa/sKnZexRUIrFyNV8YZE3Rjy3H5Q7QI2AnDLqAsCfaLpxH6wGgf6KmnD97CL4AKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoxNDQKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMyAwIG9iago8PCA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL0NBIDAgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PgovQTIgPDwgL0NBIDEgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSID4+CmVuZG9iagoxMyAwIG9iago8PCAvQml0c1BlckNvbXBvbmVudCA4Ci9Db2xvclNwYWNlIFsvSW5kZXhlZCAvRGV2aWNlUkdCIDY4ICj////+/v78/Pz7+/v5+fn39/fy8vLx8fHq6urp6eno6Ojm5ubl5eXd3d3Z2dnX19fW1tbV1dXMzMzGxsbBwcG/v7+9vb28vLy7u7u6urq4uLi1tbW0tLSysrKtra2srKyhoaGYmJiVlZWRkZGAgIB1dXVzc3NxcXFqampnZ2diYmJaWlpXV1dTU1NRUVFPT09MTExLS0tCQkJAQEA7Ozs6OjowMDAgICAbGxsYGBgXFxcVFRUTExMODg5cclxyXHJcblxuXG4ICAgGBgYDAwMCAgIAAAApXQovRGVjb2RlUGFybXMgPDwgL0NvbG9ycyAxIC9Db2x1bW5zIDE1OCAvUHJlZGljdG9yIDEwID4+Ci9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9IZWlnaHQgODIgL0xlbmd0aCAxNCAwIFIgL1N1YnR5cGUgL0ltYWdlCi9UeXBlIC9YT2JqZWN0IC9XaWR0aCAxNTggPj4Kc3RyZWFtCnic7dhXj4JAFIZhxo5rb9h717X33v7/b9oZgu7qEpnNIeq457tDkjfPBRBEKr/0pGcD7g95kAnBO5mzsukx5EFiyIPEOHnz+TwSGQwGHEWeWITFkPcSvO02kUgQksvl4LwtixEW+xe85XKpKISt0WhAeax1jiHv+bxMJqP2XK79fg/lXVouo9ab8Px+v5psNg16PLxLizOGvAfwHA5Ht2sWj7U4Y8h7AI8+TA173Dyu1lvwajWr1UpIqVSC82osRlgMec/n0ZeVUEh9jtKXeTjvbxOe1263CfnmdTqdT7Z+v4884XnhcFjjpVKpbNZisagHdrs9m9W7l5Gnz/s9j8czmSDvHXhut/vj55xOSZK83punNfIE5EWjm83m6sxwKMsyIdPpFHlGvGr19sxqFQgEZHk2myFPcF48fnXt0QNFodfeaKRXRJ62fD6vPfd6vd5iQX85HsfjcTKZJCQYDOoXkScGb71ea19b6WKxYrFYKJwPK5UK8oy+Eux2Pp/v5lWPvszX64fDAXmC804netOm02eZzdZqtej/3TtF5EGKpsaQB4khDxJDHiSGPEgMeZAY8iAx5EFiQvBedsiDDHmQfQGIbT5JCmVuZHN0cmVhbQplbmRvYmoKMTQgMCBvYmoKNDU1CmVuZG9iagoyIDAgb2JqCjw8IC9Db3VudCAxIC9LaWRzIFsgMTEgMCBSIF0gL1R5cGUgL1BhZ2VzID4+CmVuZG9iagoxNSAwIG9iago8PCAvQ3JlYXRpb25EYXRlIChEOjIwMjIwNTAzMDMxNjU0KzAyJzAwJykKL0NyZWF0b3IgKE1hdHBsb3RsaWIgdjMuNS4xLCBodHRwczovL21hdHBsb3RsaWIub3JnKQovUHJvZHVjZXIgKE1hdHBsb3RsaWIgcGRmIGJhY2tlbmQgdjMuNS4xKSA+PgplbmRvYmoKeHJlZgowIDE2CjAwMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAwMDAxNiAwMDAwMCBuIAowMDAwMDAxNzI0IDAwMDAwIG4gCjAwMDAwMDA1OTcgMDAwMDAgbiAKMDAwMDAwMDYxOCAwMDAwMCBuIAowMDAwMDAwNzE3IDAwMDAwIG4gCjAwMDAwMDA3MzggMDAwMDAgbiAKMDAwMDAwMDc1OSAwMDAwMCBuIAowMDAwMDAwMDY1IDAwMDAwIG4gCjAwMDAwMDAzMzggMDAwMDAgbiAKMDAwMDAwMDU3NyAwMDAwMCBuIAowMDAwMDAwMjA4IDAwMDAwIG4gCjAwMDAwMDA1NTcgMDAwMDAgbiAKMDAwMDAwMDc5MSAwMDAwMCBuIAowMDAwMDAxNzA0IDAwMDAwIG4gCjAwMDAwMDE3ODQgMDAwMDAgbiAKdHJhaWxlcgo8PCAvSW5mbyAxNSAwIFIgL1Jvb3QgMSAwIFIgL1NpemUgMTYgPj4Kc3RhcnR4cmVmCjE5NDEKJSVFT0YK\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2022-05-03T03:16:54.738525\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.5.1, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}, {"name": "stderr", "output_type": "stream", "text": ["Global seed set to 1\n"]}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "82dca1386ee94406bbc359ae1c7dafbe", "version_major": 2, "version_minor": 0}, "text/plain": [" 0%| | 0/28 [00:00\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2022-05-03T03:16:57.019824\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.5.1, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}, {"name": "stdout", "output_type": "stream", "text": ["Original image and input image to sampling:\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUiAvTWVkaWFCb3ggWyAwIDAgMTcyLjM4Mzc1IDk1Ljk0IF0KL1BhcmVudCAyIDAgUiAvUmVzb3VyY2VzIDggMCBSIC9UeXBlIC9QYWdlID4+CmVuZG9iago5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTIgMCBSID4+CnN0cmVhbQp4nFWOsQ7CMAxEd3/FfUESp42SjIVKFWNh4AOiCKhoEVSiv4/TAcpwku90z7Zu8/uW8rHbYX8i/XNpJsZAumFcZhgMogWMDv8lI/lI7K2qQuWduPvGRadiLYnZTKVxJZroCa/sKnZexRUIrFyNV8YZE3Rjy3H5Q7QI2AnDLqAsCfaLpxH6wGgf6KmnD97CL4AKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoxNDQKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMyAwIG9iago8PCA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL0NBIDAgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PgovQTIgPDwgL0NBIDEgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSID4+CmVuZG9iagoxMyAwIG9iago8PCAvQml0c1BlckNvbXBvbmVudCA4Ci9Db2xvclNwYWNlIFsvSW5kZXhlZCAvRGV2aWNlUkdCIDc4ICj////+/v79/f38/Pz5+fn39/f09PTz8/Pv7+/t7e3s7Ozr6+vp6eno6Ojm5ubh4eHe3t7b29va2trT09POzs7Jycm/v7++vr67u7u6urqurq6pqamoqKinp6ekpKScnJyYmJiWlpaTk5ORkZGPj4+NjY2JiYmFhYWAgIB3d3d2dnZxcXFqampkZGRgYGBeXl5dXV1ZWVlXV1dVVVVUVFRQUFBDQ0NAQEA/Pz8+Pj44ODgxMTErKysmJiYjIyMgICAfHx8dHR0bGxsWFhYSEhIQEBAPDw8ODg4LCwsHBwcGBgYFBQUEBAQCAgIAAAApXQovRGVjb2RlUGFybXMgPDwgL0NvbG9ycyAxIC9Db2x1bW5zIDE1OCAvUHJlZGljdG9yIDEwID4+Ci9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9IZWlnaHQgODIgL0xlbmd0aCAxNCAwIFIgL1N1YnR5cGUgL0ltYWdlCi9UeXBlIC9YT2JqZWN0IC9XaWR0aCAxNTggPj4Kc3RyZWFtCnic7ddpV4JAFIBhmQkMNXMpU3M3W9yX0tJSwxRL4v//nO6MtllqNHmSuveLAsf3PB8GBh2nGz2O3wYsH+SJjC145s/M6Y/HkCcSQ55IDHkiMeSJxJAnEkOeSAx5IrHVvGq1GggQQiQpGo2Wy7quC/BYK8BaEmuVl7SQt2beaDRKJGRZpmwI4R/BYLDZbFrn8VbipfUcW9hC3pp5sDoo9Xg8pVI+n58VYRRFubqyyuMtylol1iJvW4tjtuVpmgZ3GaXtdts0u93uOZtIZFr1+YbDoQXerEVZy3xpRWYt3yct5K2TVywWYYkcHxuG8XrSMAqFgtvtJiSdTlvgfWPszLu5cblcXq9X0+YujMdjv98vSbFYDHk25R0dTTewDxdarRbflWq1GvIW8Q4PAQE379zpx8d4PA5X9vYmkwny7MiDd7PdXUB8/M3Z2XRTu7hYWETeZvMGgwFHvHt43N3lcrmtrSlvSfHf80x258LryvPGdXtbr9cdDjKbTCaDPDvzTk5ghTmdzgM2OztwsL0N2v19+Ia8VbyHh1QqBa98/D6V5XA4zP+NcmilUkGenXl8dF3n/0gvL/khPK1VFcj9fh95q3lz02g0KM1ms8uLyLMlLxQKEQJLEXnWeb2eqqqUIu8v8q6v+QaHPOQh7/3c3yeTSUXpdDrI+3O8rxaRJ1JEnkhxDbyNHeSJDPJE5gkEVtsWCmVuZHN0cmVhbQplbmRvYmoKMTQgMCBvYmoKNTMwCmVuZG9iagoyIDAgb2JqCjw8IC9Db3VudCAxIC9LaWRzIFsgMTEgMCBSIF0gL1R5cGUgL1BhZ2VzID4+CmVuZG9iagoxNSAwIG9iago8PCAvQ3JlYXRpb25EYXRlIChEOjIwMjIwNTAzMDMxNjU3KzAyJzAwJykKL0NyZWF0b3IgKE1hdHBsb3RsaWIgdjMuNS4xLCBodHRwczovL21hdHBsb3RsaWIub3JnKQovUHJvZHVjZXIgKE1hdHBsb3RsaWIgcGRmIGJhY2tlbmQgdjMuNS4xKSA+PgplbmRvYmoKeHJlZgowIDE2CjAwMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAwMDAxNiAwMDAwMCBuIAowMDAwMDAxODIzIDAwMDAwIG4gCjAwMDAwMDA1OTcgMDAwMDAgbiAKMDAwMDAwMDYxOCAwMDAwMCBuIAowMDAwMDAwNzE3IDAwMDAwIG4gCjAwMDAwMDA3MzggMDAwMDAgbiAKMDAwMDAwMDc1OSAwMDAwMCBuIAowMDAwMDAwMDY1IDAwMDAwIG4gCjAwMDAwMDAzMzggMDAwMDAgbiAKMDAwMDAwMDU3NyAwMDAwMCBuIAowMDAwMDAwMjA4IDAwMDAwIG4gCjAwMDAwMDA1NTcgMDAwMDAgbiAKMDAwMDAwMDc5MSAwMDAwMCBuIAowMDAwMDAxODAzIDAwMDAwIG4gCjAwMDAwMDE4ODMgMDAwMDAgbiAKdHJhaWxlcgo8PCAvSW5mbyAxNSAwIFIgL1Jvb3QgMSAwIFIgL1NpemUgMTYgPj4Kc3RhcnR4cmVmCjIwNDAKJSVFT0YK\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2022-05-03T03:16:57.083516\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.5.1, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}, {"name": "stderr", "output_type": "stream", "text": ["Global seed set to 1\n"]}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "491346b20e55493992ead44415c9f78d", "version_major": 2, "version_minor": 0}, "text/plain": [" 0%| | 0/28 [00:00\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2022-05-03T03:16:59.356620\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.5.1, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}], "source": ["def autocomplete_image(img):\n", " # Remove lower half of the image\n", " img_init = img.clone()\n", " img_init[:, 10:, :] = -1\n", " print(\"Original image and input image to sampling:\")\n", " show_imgs([img, img_init])\n", " # Generate 12 example completions\n", " img_init = img_init.unsqueeze(dim=0).expand(12, -1, -1, -1).to(device)\n", " pl.seed_everything(1)\n", " img_generated = model.sample(img_init.shape, img_init)\n", " print(\"Autocompletion samples:\")\n", " show_imgs(img_generated)\n", "\n", "\n", "for i in range(1, 4):\n", " img = train_set[i][0]\n", " autocomplete_image(img)"]}, {"cell_type": "markdown", "id": "43a50c67", "metadata": {"papermill": {"duration": 0.018273, "end_time": "2022-05-03T01:16:59.433091", "exception": false, "start_time": "2022-05-03T01:16:59.414818", "status": "completed"}, "tags": []}, "source": ["For the first two digits (7 and 6), we see that the 12 samples all\n", "result in a shape which resemble the original digit. Nevertheless, there\n", "are some style difference in writing the 7, and some deformed sixes in\n", "the samples. When autocompleting the 9 below, we see that the model can\n", "fit multiple digits to it. We obtain diverse samples from 0, 3, 8 and 9.\n", "This shows that despite having no latent space, we can still obtain\n", "diverse samples from an autoregressive model."]}, {"cell_type": "markdown", "id": "71e55467", "metadata": {"papermill": {"duration": 0.018019, "end_time": "2022-05-03T01:16:59.469082", "exception": false, "start_time": "2022-05-03T01:16:59.451063", "status": "completed"}, "tags": []}, "source": ["### Visualization of the predictive distribution (softmax)\n", "\n", "Autoregressive models use a softmax over 256 values to predict the next pixel.\n", "This gives the model a large flexibility as the probabilities for each pixel value can be learned independently if necessary.\n", "However, the values are actually not independent because the values 32 and 33 are much closer than 32 and 255.\n", "In the following, we visualize the softmax distribution that the model predicts to gain insights how it has learned the relationships of close-by pixels.\n", "\n", "To do this, we first run the model on a batch of images and store the output softmax distributions:"]}, {"cell_type": "code", "execution_count": 24, "id": "a091c7bc", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:16:59.507240Z", "iopub.status.busy": "2022-05-03T01:16:59.506768Z", "iopub.status.idle": "2022-05-03T01:16:59.667743Z", "shell.execute_reply": "2022-05-03T01:16:59.667046Z"}, "papermill": {"duration": 0.181773, "end_time": "2022-05-03T01:16:59.669511", "exception": false, "start_time": "2022-05-03T01:16:59.487738", "status": "completed"}, "tags": []}, "outputs": [], "source": ["det_loader = data.DataLoader(train_set, batch_size=128, shuffle=False, drop_last=False)\n", "imgs, _ = next(iter(det_loader))\n", "imgs = imgs.to(device)\n", "with torch.no_grad():\n", " out = model(imgs)\n", " out = F.softmax(out, dim=1)\n", " mean_out = out.mean(dim=[0, 2, 3, 4]).cpu().numpy()\n", " out = out.cpu().numpy()"]}, {"cell_type": "markdown", "id": "133011d4", "metadata": {"papermill": {"duration": 0.017858, "end_time": "2022-05-03T01:16:59.706466", "exception": false, "start_time": "2022-05-03T01:16:59.688608", "status": "completed"}, "tags": []}, "source": ["Before diving into the model, let's visualize the distribution of the pixel values in the whole dataset:"]}, {"cell_type": "code", "execution_count": 25, "id": "47cad649", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:16:59.743599Z", "iopub.status.busy": "2022-05-03T01:16:59.743309Z", "iopub.status.idle": "2022-05-03T01:17:01.345582Z", "shell.execute_reply": "2022-05-03T01:17:01.344953Z"}, "papermill": {"duration": 1.623691, "end_time": "2022-05-03T01:17:01.348159", "exception": false, "start_time": "2022-05-03T01:16:59.724468", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUiAvTWVkaWFCb3ggWyAwIDAgMzg0LjcgMjUxLjk4MDYyNSBdCi9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUiAvVHlwZSAvUGFnZSA+PgplbmRvYmoKOSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEyIDAgUiA+PgpzdHJlYW0KeJzdXUuPNMdxvO+vmKN1YLEys55H0bQJ6EaJsA+GDwJN2SKGMihC0t93ZvVOV9Z2CKBFUUCXhE/arZiNnYid7npmx6eff/Pn33/9za+/+Ozxz795+3R+9/UPb/T49u3TX9Ljv3946P884uNb/fcX/foL+/4t6nffvUlLoepXz/evOFPoLRbO2hTXb//n7e13bzF0qiXVmFt7fPwm9Ui9xNoef7Rf+MXlBec3bx9e/ZbYfnsNksZv03dWa8iu5flqSfo+X2/p+CnfMt7k948PdCIptAdTDSk9/vjN498ff3h8+ks+vPmV/vtW/x3erCbmqm+6UePlvblW/7vffvP25eP7F3EMlNX3F/f49ov31rfv9c8TH59EhXIKycgOwh5SI2X7+ru3z756fPqv9CB6fPW78ef66r/e/uPxT/EXj/98fPWrt3/56u3L8fv+Hlrf7AOjPyixprb+IXzzT1dLRm1spQj9CLUl/VxymWIovdROi1zf/NPlctS2wZaS9B+hl7j9bIJbDbGV2Msq2DX/HQTXEvJg41zqjxHc+ecSLEVCqjnF9S/sm3+6YMkc+mCLpecfIVh9+bkEDx69S6aWE/fzJjpb/laRNfCjSiAuMUuVRIc+UxViaTWbtin2n+gXj6++fSuh95YlUy58yLeblwIkIet/SjG9jxT0musySaqqMBYZL1aXvrdfE6t1VPH1JaVAkSXZzylD7eNtfP2def7J5998+9t/+9NvfvuHHz757vd/+NMPj8//9/Hlz2E2CamUXvLsslzTT7HbbpR691BhtaX8D/Gbb+B3y6GItEbT79n0k/xupKpS1tGJpOb8pjou6J/DcPrxhkvrw75/tOEsPdTYKNY5JptNf/MoQNRxlqzvsVWh1vJPvKNU9UpabFlt/ut+x5ff/w+j5NDFOgyWzpT1b6T4GMSqgDcbLLwPY9MY9apBf/nrr38sr//1F28/8pU6Qnajzk+y6Ic0ZKo9iY1HeyDSsXX+gDwdwqaul96sF3ADWN+sg+fPbumMqoxVr5aLMyXqaFM/T1dnJpL1Y9io5/7u18HkW+/qS7HXFr2JXH0hHRIoVK6+nEixz04rTIdb70y+9ba+mMZkei++cCi1ZQa+nEjJ42bD6eXWYPKtt/XFNEor6eqLhFaq/uzVlxOpHHIqzPxyazD51tv6YhrZvrv4kkPMRQrw5UT0q6YWRXq5NZh86219MY2k+NWXEjhlnVpefTmRquOCmIjSy63B5Ftv64tpHO/04ou+ZUkc29WXE9GvpDYePc9k8q239UXfGPecgC8tFBZi4MuJ6Fc6V6/H5+Vk8q239UU1UkvlY6+jvvSgs4+Y+tWXE7GJOfUo+d2tg8m33tYX1RitG7n4UkknSVHHrRdfJqIOdLvR9He3Dibfeldf9B7Ze5H48e6qvnCgZoP5iy0vwN9fJ88Od90hMTMDV8TunY2urrwAP3qbPDuM6YbEROnjnVVd0a9L1en31ZYT8XfXybTDPXdoFJ0AXn3JoeZSMvDlRLRfLikJv8waRK7xtq6YQrreQNSVEnrKuX6cYz8dUpL+JumpvLwaTL71tr6Yxtgj8KUFkpQ68OVEbNSvY7dWX24NJt96W19Uo774OpGuPQhLImDLieSqk8TUKb+bNYh8421dUYlka5gfXWn6S4lFPvr1dIgN+UvT28y7V4PIN97VFZMY62WqrK5Q0HkwZ+DKiRTtlKX2sVF3EvnG27pCtt2c+8feRm3h0PS19eOqw9Mh/h4ymXa4s5jGljMBX1KIVW+gwJcT8ePZybTDKHdoHKOMiy/Z7hGRrmvdE8milw7zcRmdTL71tr6YRpEMfCkh5dIF+HIiOQXSL0t9uTWYfOttfTGNOpm5dtFNe9qUtVO5+nIiRe8pjXpsL7cGk2+9rS+mkagDX1rQ72sFvpyI9jzqD489/snkW2/ri2mMdo/86EuPIbKUfl3TnUjOoddiuyRPx+Rb7+qLabT1NOALBSYuBHw5kawz5qQfjP7u1sHkW2/ri2rUTjdfB7udQ9I+R4AvJ5JLyNmunne3DibfeltfOMhxRubii/a2Per99erLidQaKrfU++HWO5Nvva0vYvdI/TNffUmh1i4V+HIifkVuMu2wTjc0pkrXte6ud8/SuF8XdSfid4cm0w57RkOj6H+vvtSgP84EfDmRxZeTaQtfTKP2HdfpdG+qoVw68OcEUtF+p4zueNK4xtt6YgJJB2hXT3rIksFxlxPIMVQp1Pll1ODxrbd1xSTqS68r3RRVHityXep2kF+Rm1w7rNMNkTpwJ2QNhU6srwfWnJCfCDmyHeZHQ2VU7LrkrX2t9rfU+Lq46yC9yUZKlMvLsoPMN9/XGwmxV8rIG9vtiDqsB96ckA5YUreZ4rtl72S++b7eqMpWLidczBqVV3spyJoTyqwdUrf9kafj8q33dcZEXse9FEsopeXLLsrTQ36u+GLaYf54SJTL/pn5UoOO73NEvpxQ7oFzq5RPu4zLt97XGROp873rEjjFHmIqia+DPAetH5mTbI9PjamkWoA3pDySJQFvJmQHmrP20XRaNsh88229GSpjAYUCRBT0k8GgUsBBqdlXtkP79GS++b7eqEprQN5wKKTDN+TNCa3enGR7eKMqo94krkviRBJaZAITKAdpJ91Uaq4vyw4y33xfbyTUbrtlwJsUeqcIagYc5KdLjmyLWZSpbEVA2QBRCfp1B3UDDtIOiSm1mF6WHWS++b7emErrd4E3qqJYDQDw5oSWfmqS7dFPmcrEoHyAqIWcawX1Aw7yJVqObIfKrUOlECghIOq2UFdBDYGD/NEYR7bDiZlDJUdQRkAcQ5dcQB2Bg/y2viPbYbf/UBk7KCUg1r89pwxqCRykE6dapNbTscHlWu/rDFtBRAPFBMQShCSBagIHpawTqN4LvQw7yHzzfb1RlbE1UFCgHws7G3RZBXw6ZHXmpNrDmRRSrxUUFejtVF9N4Pj8RJY10Em1xRqoiWylgMIC4hL0E8DQmhNavTnJ9vBGVdacQXEBcQtRPxygusBBSacFMQqflh1kvvm+3pjKlECJAfFYtYugxsBBy6hvkm0x6hsq9c0Cb8R+tHRQZ+CgTCEpubTTskHmm2/rzVDJl6PzZg2FIrmDWgMH+TNok2uHk2mHSDhVEA6NUwP1Bg7yp38n1w5ngg+R8XIy73ioXdSZEKg5cNAyiTq5tphDmUirQgLr55IDRy6g7sBBjUPXT0dLL8cOMt98X29y0LF9BKUHJCWI9tCg9sBB2Q5gka1sPT2Zb76vN6qylQ7KD/SaCNpDZ1B/4KBlHXSSbbEOaiqrHQAH3jSdQOvXyJsTKtGWIhLVl2UHmW++rzemMlVQhkDSQ89NQB2Cg5J+1aNttTw9mW++rzemUgooRdCJUKCkU0bgzYSWIxOTbIszE0MlX08zmjccxM7mgTXiCfmj5I5shxPmh0rKoCSBko7eOBOoSXDQMiSeZFuMiYfKmEBZAtkzXylFUJfgoGUvc5JtsZdpKo/nI1690b995A5qExy07NdNsi3262zRsjcG5Ql6sw06xG2gPsFBy8rNJNti5cZUtsqgREH/7IFbrKBGwUHr5+Yk2+NzoyJqIVCmQDpGSaVXUKfgoGU1dJJtsRo6VOaYwTQ866vz9YGDT4csI79JtcXIb4iUDgoWKJP2NBUNik9k2cecVFvsYw6R3FDRQpYQpSRUtDChZSV0km2xEjpU2uN7gTcpMI9H21+9OSFf1OLIdqh1OVTGiqoWdFybdCiHqhYmtKxPTLIt1idMZbQnhABvio5PhFHVwoTUBGKW8ehfR+ab7+uNTgh7u4x7zZoaqlKgqoUJ6cyySzlWQ08q13hfX1QiKmImHbX1RhEVLUyoij00mWN9uTWqmF3jfX1pNkm+XC9qTLHIgBhR1cKElmN9J9cWp/pMpI5EUNVCoSC5dVS1MKHSdGLQe04vxw4y33xfb0ylrUABb+w52fpC5M0JLafPJ9kWp8+HSkHxBlTERiYo38BBSSyqzB7b+fRkvvm+3phKRgEHVFLonFHCgYOWMxOTbIszE0MloZADKiUQJZRy4KAUdXKQ5Fjpm2S++b7eqMrYO6pasGcgREFJBw5aZlGTbItZlKpkO8EJVs+1r0naFaGqhQktuy6TbItdF1PZqn179aZbbTtKPHDQMsOcZFvMME1lLRVVLVS9Z9SIUg8ctMwwJ9kWM0xTWTIKPqDKIeaOkg8ctKxoTbItVrSGyoTCD0hnRGyxgcibE1p26ybZFrt1Q6WgAASqKSSpKAHBQX76Pbl2mH8PjYwiEKjmoBJQBoKDln3MSbbFPuZQSSgGgWoJja5njJ8OWeYLk2qL+cIQGVEUAlWdR0dBi+cnsvTek2qL3rvW0HtHcQj6/0HFoTwEBy37u5Nsi/1dU9kaikSgFm2miDIRHORrxybXDrVjprEWFItgsaFZfxLVLExo2cOcZFvsYZrKklE0AjWVkjvKRnCQ6NSJxzLW05P55vt6YyoTiEegpt1MaigfwUF+WHNS7TCqGRLhAaSW7XA9Skhw0LIRdXJtsQ81RDIISaBWggIoJcFBekdhjvn9Ynpx+db7OmMiCQUlUKv6TUZJCQ5a5k+TbIv501AZUViCdsJBp0goLcFByynZSbbFKVlVWWzVEqydtx5aF5SY4KD1kjrJ9rimVGVrKDSBOoXYCKUmOCgl7Yta6+Vl2UHmm2/rjam0am3QeXcOXCNKTnDQMouaZFvMokxlKSg8QdtCyh2lJzhoeTLdJNviyXRDZUYBCtRTKKmhBAUH2XK51FanZYPMN9/XG1OZUIgC9RyaTotQxcKElr26SbbFXt1QyShIgXoNkQtKUnDQsnY+ybZYOx8qYaIadTs9nlGagoOWVZtJtsWqzVAZUaAC9R5STChRwUHL+GaSbTG+UZXSOwpV4Ki/twtKVXDQMtGcZDvMNIfK1lCwAkcKtTFKVnBQsXCb0srLsYPLtd7XGdVYK4pW4Mg6W7z27U+H+AvHUe1wPQ2RpaCABY7az+jN9Drsm4ifFjiqHWYLQ2TOKGRB/+iqoqGQBQf5DTlHtsM+3aEyoZQFjiXo2KWDegUH+eONjmyHU4+HSkEpCxwt7amglAUH+emkI9thlnmoZJSywLGFrgygXsFBfjrpyHaYZR4qCaQs6PDNQk5RyoKD1kvqxbXFFWUigSs6LumCMhYc1DlkvarG2b2Dx7fc1xEVqP3rdZGPiUNqjBIWHLR0TSfXFj2TibSTiegDY1UqhBIWHOQnkI5sh3nlUFkKSlhg0rdeIkpYcJAvZnFkO9S4DJX2REZ0SZUQU0cJCw7yG/+ObIfzAIfKhBIW7E2zNJSw4KD1mjrJ9rimTKWghAUm63srSlhwkH9UgiPb4QkKh0pGCQtMPRSdQoJaBQf5BxY6sh2eY3ioJJSwwByD/uFRwoKD/CKwI9thbfhQGVHCgg739X4qKGHBQcvnZpJt8bkxlTpYA7UKzBKoMUpYcJA/AODIdjgXMFTWhh6bqjeOIJVQwoKD1s/NSbbH50ZVlooSFlSYzogiSlhw0NKHT7It+nBTmTNKWGC2CN2OEhYc5AuZHdkO9c2HyoQyFlhV6P0WZSw4aL3fnGR73G9MpaCMBeYeiK/PLHk6xJclOKodqhUOkYwyFlhiECrgSP5Elv2WSbXFfssQSShjgYW0n8koY8FBPojYke2QT3yojChjgYX11QllLDhouaIm2RZXlKlsHWUs6IwxtCYoY8FB/lHVjmyHJ1gPlfoNqFfQ14eo+kC9goN8gbcj26Hue6gsFWQssJTAJaKMBQcl67HLOEo9qVzjfX1RibCsmcXec0cJCw5alm1Ori1WbYbIDBIWWFoo0lDCgoOWLbqTa4sduiEyoYQFlh50fo0SFhzkDzU6sh3OOh4qBSUscKIQqaCEBQctc+9JtsXce6gklLDAKkWHtChhwUH+Jjy5drgLD40R5SuwjtqkJ5Sv4KBlpjDJtpgpmMrWUb4CJ/1GeyFQreCg5RzJJNviHImprA3lK3DKOqJllK/gINWrv6tmell2kPnm+3qjKm1fFqye6y2jF0L5Cg5aVvom2RYrfaYyF5SvwKnpaD+ifAUHLbsuk2yLXRdTac+fAUNivZ+KdJSv4CAfDurIdsgMPVTC8mbW+2nmivIVHLReUyfZFtfUUCkoX4EzhUoF5Ss4aJlKTbIt5lJDJaN8Bc7a18SM8hUc5B/n6Mh2eMrjoZJQvgLr/TT2hPIVHLR6c5Lt4Y2pjChfQefQgdv13MDTIX6+MJl2mC+YxNpQugLnElJlNCQ+EbEnPUpvp1sHlW++rzMqslSUrsD2HEIdwaFqhQkt68OTbIv1YVOZC0pXYO1lWo4oXcFB5SjZHQ9HdWS++b7eqMqUUboClxiUBKUrOKjkQFEap5dlB5lvvq03Q2VC6QpsRXHcULqCgyrZ6dB+HMmfZL75vt6YSgHpClw4JKooXcFBy2Th5NpirjBEoo0FvV+UWFC+goOWFa0X1RbrWUMjgXwFLtrJ9IzyFRzkK3Qn1xZ1u0NkRPkKrPdRHZigfAUHrdacZHt4Yw9z6ihfgYvqq4zyFRy07O1Osi32dk2ljUTQJdWCFEL5Cg7yD313ZDs8C36ozBXlK3DpIeeI8hUctKyeT7ItVs9NpVVggNXzavo6yldw0Hq/Ocm2uN8MlamjmgUdt3VuKF/BQRZTol/kfFo2yHzzfb0xlYLyFbhKIKoJ1SxMqIyjNfbE/Kcn88339cZUsn179SYFiQXlKzhomX9Psi3m30MlVVSzUHNIPaN8BQflbgWGlfJp2SDzzff1xlRGeBCpllBaQvkKDlrmC5NsixmDqdTRP6pZqFXvGYLyFRy0TDIn2RazTFOpHwFUs1B7UA0oX8FBy3mbSbbFeRtTmStKWOCmPDmihAUHLbt1k2yL3TpTmQpKWOCm4365PhH96ZDleT+Taovn/QyRGSUscONQuKEF9BNZrqdJtcX1NEQmlLDATcduVFHCgoOWHd5JtsUO71ApKGFB5wMhxoISFhzkQ6Ic2Q7ZUYdKQhkL3EqgnlHGgoNaCpFSLeW0bJD55vt6YyojyljQEW+QllDGgoPWa+ok2+Oa0rdaO8hY0Dtt0L4YZSw4qEQ7a56ovhwbXL71vs40e3gROobULJOFUcqCg/wDHk+qHZ7vaBJzBRkL3GPomVDGgoOWCebJtcX80kTariNYP+8cKEWUseCgZT1rkm2xnmUqJaOMBe4SROWimoUJLXvfk2yLve+hMqGMBe72TUMZCw6qys9i1bxPT+ab7+uNqRSUscA9hxoLylhwkMv7cVwbpAAdGhklLHAvofWMEhYctDwtYJJt8bSAoZJQwoJyhNgSSlhw0PKAx0m2xRMeh8qIEha4ay9cBSUsOMgPbCbXDiMb01g6yleQqD9aGOUrOMhl9TmuDRL8hsZcUbqCRAo6t0bpCg7yxz4d2Q6nQYfKBCcKEsdTPlG6goOKhMj2uLWXZQeZb76vN6pSMkpXkJhUX0fpCg7yRwAc2Q4nAw6VCaUrSMx6M20oXcFBvhDKke1QH3WoFJSuILGEFCtKV3CQ38p1ZDvs8B4qGeUrSKwh92s14tMhfpHcUe2wdn6IJJSvILGF2jI4ajMRX8ziqHaocTlERpSvILGHXgXlKzjIP/LIke3wJKShUqeKoGJBiAIV7qBiwUH+KJYj2+GE1lCZG8pXEOIgmVC+goOWu80k2+JuYyq1xwUVC0KqT3siULHgoJL0l0lP5WXZQeab7+uNqpQC8hWEtAvWGTWoWHBQsbM1tuT5cmxw+db7OmMiwXCYcujUUMKCg5ZhzTvTFmOaIVFAxoLoe6ZYUcaCg+wcfmSp5bTLuHzrfZ0xkYwyFrSTUZ6CMhYctN6BT7I97sCmklDGglAPqWWUseAgHxjgyHbIEThURpSxIBztvAPKWHCQ38J1ZDvs7A6VpaOMBR2uhVYEZSw4yK11Oq4NVkCHRttrvK6bC0uImVHCgoP8Y5gd2Q5PZx4qU0UJC8I6ZkuEEhYc5Le4HdkOO99Dpa2wgJVz1lEbd5Sw4CAfa+PIdki7OVRmlLAgXEKhhhIWHOS3/x3ZDqcCDpUJJSyIqmixooQFB1WLpWup99OyQeab7+uNqRRU3SzcdIRSUMKCg9Zr6iTb45oylYwSFkRi0D4aJSw4aNlzmWRb7LkMlRElLIhQkJpQwoKDfLmuI9uhineoLB0lLIhYVqGghAUHFXtYs36Eysuyg8w339cbVakgqFYQkVDzdRXw6RD/gBZHtcNzW4bIVFHCgr31nggcyZ+Iny1Mph1mCyZRCspXECmBJKJ8BQeVHohaOVbOJ5lvvq83qpIzylcQURXa/4JaBQctS1qTbIs1raEyoXwFkRZybChfwUHLzvck22Lne6gUlK8g0vXVBeUrOKhqp6QflN5OywaZb76vN6aSQb6C2GPdW0b5Cg4qLSTRWcN0zLh8622dGSLhVEGVxJpQwoKDfCXL5NqhvuUQGUHCgiTRQZughAUHLQtaJ9cW61kmMneUsCAp6aiNUcKCg5ZNukm2xS6dqUwNJSxIyqEkQgkLDhoPI7Hbzsuyg8w339cbVanzHlCtoFOk0CSihAUHVTsVql3VadlB5pvv642q5IwyFiTpPYM6ylhwkA59q35SWn9ZdpD55vt600YpKahXEHuCRmwoY8FBemOx9dBIp2WDzDff1xtTKShjQXIM0ivKWHCQPeBHb7+tnZYNMt98W2+GSkYZC5Ip5FZQxoKDlrNrk2yLs2tDJaGMBckqpWaUseAg7ZAoJqvfeHoy33xfb0xlRBkLkiX0klDGgoOWtb5JtsVan6nMHWUsSM6BMqOMBQfVFJhHrMLTk/nm+3qjKlNDGQvaC6kKQhkLDqo5RLYA0ZdlB5lvvq83qlIqyliQXEOWiDIWHKR3lSCjqPnpyXzzfb1RldqzoJoFS+TTrgjVLEyIdA4Vtcui07ODbWm/rzumM6OUBck9aDcMUhYmQnYdRc7H9sLkWtrv643JFJS0IDpKib2igfGJUBqRmK/Pzcm1tN/WmyGTUdaCFA6sHw5UuTAhYjWExhbV07Mt7ac7n/6STTo/fvUgNU9V0eOLx6eff/Pn33/9za+/+Ozx9Q9vHxz87r1BZ/k6Rytj3ePtN29S9QJeXvbe8uF1F7aPP+heNX/0r7F9+fZ/92udUwplbmRzdHJlYW0KZW5kb2JqCjEyIDAgb2JqCjY4ODAKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTcgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzOTUgPj4Kc3RyZWFtCnicPVJLbsVACNvnFFyg0vCbz3lSVd28+29rQ1KpKryJMcYwfcqQueVLXRJxhcm3Xq5bPKZ8LltamXmIu4uNJT623JfuIbZddC6xOB1H8gsynSpEqM2q0aH4QpaFB5BO8KELwn05/uMvgMHXsA244T0yQbAk5ilCxm5RGZoSQRFh55EVqKRQn1nC31Hu6/cyBWpvjKULYxz0CbQFQm1IxALqQABE7JRUrZCOZyQTvxXdZ2IcYOfRsgGuGVRElnvsx4ipzqiMvETEPk9N+iiWTC1Wxm5TGV/8lIzUfHQFKqk08pTy0FWz0AtYiXkS9jn8SPjn1mwhhjpu1vKJ5R8zxTISzmBLOWChl+NH4NtZdRGuHbm4znSBH5XWcEy0637I9U/+dNtazXW8cgiiQOVNQfC7Dq5GscTEMj6djSl6oiywGpq8RjPBYRAR1vfDyAMa/XK8EDSnayK0WCKbtWJEjYpscz29BNZM78U51sMTwmzvndahsjMzKiGC2rqGautAdrO+83C2nz8z6KJtCmVuZHN0cmVhbQplbmRvYmoKMTggMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNDkgPj4Kc3RyZWFtCnicTVFJigMwDLvnFfpAIV6TvKdDmUPn/9fKDoU5BAmvkpOWmFgLDzGEHyw9+JEhczf9G36i2btZepLJ2f+Y5yJTUfhSqC5iQl2IG8+hEfA9oWsSWbG98Tkso5lzvgcfhbgEM6EBY31JMrmo5pUhE04MdRwOWqTCuGtiw+Ja0TyN3G77RmZlJoQNj2RC3BiAiCDrArIYLJQ2NhMyWc4D7Q3JDVpg16kbUYuCK5TWCXSiVsSqzOCz5tZ2N0Mt8uCoffH6aFaXYIXRS/VYeF+FPpipmXbukkJ64U07IsweCqQyOy0rtXvE6m6B+j/LUvD9yff4Ha8PzfxcnAplbmRzdHJlYW0KZW5kb2JqCjE5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggOTQgPj4Kc3RyZWFtCnicRY3BEcAgCAT/VEEJCgraTyaTh/b/jRAyfGDnDu6EBQu2eUYfBZUmXhVYB0pj3FCPQL3hci3J3AUPcCd/2tBUnJbTd2mRSVUp3KQSef8OZyaQqHnRY533C2P7IzwKZW5kc3RyZWFtCmVuZG9iagoyMCAwIG9iago8PCAvQkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzOQovU3VidHlwZSAvRm9ybSAvVHlwZSAvWE9iamVjdCA+PgpzdHJlYW0KeJzjMjQwUzA2NVXI5TI3NgKzcsAsI3MjIAski2BBZDO40gAV8wp8CmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzMjIgPj4Kc3RyZWFtCnicNVG7bcUwDOw1BRcwIH4lzeMgSJG3f5s72qlI07wfVV4ypVwudckqWWHypUN1iqZ8nmam/A71kOOYHtkhulPWlnsYFpaJeUodsZos93ALNr4AmhJzC/H3CPArgFHARKBu8fcPulkSQBoU/BTomquWWGICDYuFrdkV4lbdKVi4q/h2JLkHCXIxWehTDkWKKbfAfBks2ZFanOtyWQr/bn0CGmGFOOyzi0TgecADTCT+ZIBszz5b7OrqRTZ2hjjp0ICLgJvNJAFBUzirPrhh+2q75ueZKCc4OdavojG+DU7mS1LeV7nHz6BB3vgzPGd3jlAOmlAI9N0CIIfdwEaEPrXPwC4Dtkm7d2NK+ZxkKb4ENgr2qFMdyvBi7MxWb9j8x+jKZlFskJX10ekOytygE2Ieb2ShW7K2+zcPs33/AV8Ze2QKZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDgzID4+CnN0cmVhbQp4nEWMuw3AMAhEe6ZgBH4m9j5RlMLevw0QJW64J909XB0JmSluM8NDBp4MLIZdcYH0ljALXEdQjp3so2HVvuoEjfWmUvPvD5Se7KzihusBAkIaZgplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzIwID4+CnN0cmVhbQp4nDVSS24FMQjbzym4QKXwT87zqqqLvvtvaxO9FUwwYOMpL1nSS77UJdulw+RbH/clsULej+2azFLF9xazFM8tr0fPEbctCgRREz1YmS8VItTP9Og6qHBKn4FXCLcUG7yDSQCDavgHHqUzIFDnQMa7YjJSA4Ik2HNpcQiJciaJf6S8nt8nraSh9D1Zmcvfk0ul0B1NTugBxcrFSaBdSfmgmZhKRJKX632xQvSGwJI8PkcxyYDsNoltogUm5x6lJczEFDqwxwK8ZprVVehgwh6HKYxXC7OoHmzyWxOVpB2t4xnZMN7LMFNioeGwBdTmYmWC7uXjNa/CiO1Rk13DcO6WzXcI0Wj+GxbK4GMVkoBHp7ESDWk4wIjAnl44xV7zEzkOwIhjnZosDGNoJqd6jonA0J6zpWHGxx5a9fMPVOl8hwplbmRzdHJlYW0KZW5kb2JqCjI0IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzQwID4+CnN0cmVhbQp4nDVSOW4EMQzr/Qp9IIBu2+/ZIEiR/L8NqdkUA3F0UpQ7WlR2y4eFVLXsdPm0ldoSN+R3ZYXECcmrEu1ShkiovFYh1e+ZMq+3NWcEyFKlwuSk5HHJgj/DpacLx/m2sa/lyB2PHlgVI6FEwDLFxOgals7usGZbfpZpwI94hJwr1i3HWAVSG9047Yr3oXktsgaIvZmWigodVokWfkHxoEeNffYYVFgg0e0cSXCMiVCRgHaB2kgMOXssdlEf9DMoMRPo2htF3EGBJZKYOcW6dPTf+NCxoP7YjDe/OirpW1pZY9I+G+2Uxiwy6XpY9HTz1seDCzTvovzn1QwSNGWNksYHrdo5hqKZUVZ4t0OTDc0xxyHzDp7DGQlK+jwUv48lEx2UyN8ODaF/Xx6jjJw23gLmoj9tFQcO4rPDXrmBFUoXa5L3AalM6IHp/6/xtb7X1x8d7YDGCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNTEgPj4Kc3RyZWFtCnicLVFJcgNBCLvPK/SEZqffY5crh+T/1wjKBwYNi0B0WuKgjJ8gLFe85ZGraMPfMzGC3wWHfivXbVjkQFQgSWNQNaF28Xr0HthxmAnMk9awDGasD/yMKdzoxeExGWe312XUEOxdrz2ZQcmsXMQlExdM1WEjZw4/mTIutHM9NyDnRliXYZBuVhozEo40hUghhaqbpM4EQRKMrkaNNnIU+6Uvj3SGVY2oMexzLW1fz004a9DsWKzy5JQeXXEuJxcvrBz09TYDF1FprPJASMD9bg/1c7KT33hL584W0+N7zcnywlRgxZvXbkA21eLfvIjj+4yv5+f5/ANfYFuICmVuZHN0cmVhbQplbmRvYmoKMjYgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMTUgPj4Kc3RyZWFtCnicNVE5DgMhDOz3Ff5AJIwveE+iKM3+v82M0VYewVyGtJQhmfJSk6gh5VM+epkunLrc18xqNOeWtC1zgLi2vC+tksCJZoiDwWmYuAGaPAFD19GoUUMXHtDUpVMosNwEPoq3bg/dY7WBl7Yh54kgYigZLEHNqUUTFm3PJ6Q1v16LG96X7d3IU6XGlhiBBgFWOBzX6NfwlT1PJtF0FTLUqzXLGAkTRSI8+Y6m1RPrWjTSMhLUxhGsagO8O/0wTgAAE3HLAmSfSpSz5MRvsfSzBlf6/gGfR1SWCmVuZHN0cmVhbQplbmRvYmoKMTUgMCBvYmoKPDwgL0Jhc2VGb250IC9CTVFRRFYrRGVqYVZ1U2FucyAvQ2hhclByb2NzIDE2IDAgUgovRW5jb2RpbmcgPDwKL0RpZmZlcmVuY2VzIFsgNDggL3plcm8gL29uZSAvdHdvIC90aHJlZSAvZm91ciAvZml2ZSAvc2l4IDU2IC9laWdodCAvbmluZSBdCi9UeXBlIC9FbmNvZGluZyA+PgovRmlyc3RDaGFyIDAgL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvRm9udERlc2NyaXB0b3IgMTQgMCBSCi9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdIC9MYXN0Q2hhciAyNTUgL05hbWUgL0JNUVFEVitEZWphVnVTYW5zCi9TdWJ0eXBlIC9UeXBlMyAvVHlwZSAvRm9udCAvV2lkdGhzIDEzIDAgUiA+PgplbmRvYmoKMTQgMCBvYmoKPDwgL0FzY2VudCA5MjkgL0NhcEhlaWdodCAwIC9EZXNjZW50IC0yMzYgL0ZsYWdzIDMyCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovSXRhbGljQW5nbGUgMCAvTWF4V2lkdGggMTM0MiAvU3RlbVYgMCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL1hIZWlnaHQgMCA+PgplbmRvYmoKMTMgMCBvYmoKWyA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMAo2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDMxOCA0MDEgNDYwIDgzOCA2MzYKOTUwIDc4MCAyNzUgMzkwIDM5MCA1MDAgODM4IDMxOCAzNjEgMzE4IDMzNyA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2CjYzNiA2MzYgMzM3IDMzNyA4MzggODM4IDgzOCA1MzEgMTAwMCA2ODQgNjg2IDY5OCA3NzAgNjMyIDU3NSA3NzUgNzUyIDI5NQoyOTUgNjU2IDU1NyA4NjMgNzQ4IDc4NyA2MDMgNzg3IDY5NSA2MzUgNjExIDczMiA2ODQgOTg5IDY4NSA2MTEgNjg1IDM5MCAzMzcKMzkwIDgzOCA1MDAgNTAwIDYxMyA2MzUgNTUwIDYzNSA2MTUgMzUyIDYzNSA2MzQgMjc4IDI3OCA1NzkgMjc4IDk3NCA2MzQgNjEyCjYzNSA2MzUgNDExIDUyMSAzOTIgNjM0IDU5MiA4MTggNTkyIDU5MiA1MjUgNjM2IDMzNyA2MzYgODM4IDYwMCA2MzYgNjAwIDMxOAozNTIgNTE4IDEwMDAgNTAwIDUwMCA1MDAgMTM0MiA2MzUgNDAwIDEwNzAgNjAwIDY4NSA2MDAgNjAwIDMxOCAzMTggNTE4IDUxOAo1OTAgNTAwIDEwMDAgNTAwIDEwMDAgNTIxIDQwMCAxMDIzIDYwMCA1MjUgNjExIDMxOCA0MDEgNjM2IDYzNiA2MzYgNjM2IDMzNwo1MDAgNTAwIDEwMDAgNDcxIDYxMiA4MzggMzYxIDEwMDAgNTAwIDUwMCA4MzggNDAxIDQwMSA1MDAgNjM2IDYzNiAzMTggNTAwCjQwMSA0NzEgNjEyIDk2OSA5NjkgOTY5IDUzMSA2ODQgNjg0IDY4NCA2ODQgNjg0IDY4NCA5NzQgNjk4IDYzMiA2MzIgNjMyIDYzMgoyOTUgMjk1IDI5NSAyOTUgNzc1IDc0OCA3ODcgNzg3IDc4NyA3ODcgNzg3IDgzOCA3ODcgNzMyIDczMiA3MzIgNzMyIDYxMSA2MDUKNjMwIDYxMyA2MTMgNjEzIDYxMyA2MTMgNjEzIDk4MiA1NTAgNjE1IDYxNSA2MTUgNjE1IDI3OCAyNzggMjc4IDI3OCA2MTIgNjM0CjYxMiA2MTIgNjEyIDYxMiA2MTIgODM4IDYxMiA2MzQgNjM0IDYzNCA2MzQgNTkyIDYzNSA1OTIgXQplbmRvYmoKMTYgMCBvYmoKPDwgL2VpZ2h0IDE3IDAgUiAvZml2ZSAxOCAwIFIgL2ZvdXIgMTkgMCBSIC9uaW5lIDIxIDAgUiAvb25lIDIyIDAgUgovc2l4IDIzIDAgUiAvdGhyZWUgMjQgMCBSIC90d28gMjUgMCBSIC96ZXJvIDI2IDAgUiA+PgplbmRvYmoKMyAwIG9iago8PCAvRjEgMTUgMCBSID4+CmVuZG9iago0IDAgb2JqCjw8IC9BMSA8PCAvQ0EgMCAvVHlwZSAvRXh0R1N0YXRlIC9jYSAxID4+Ci9BMiA8PCAvQ0EgMSAvVHlwZSAvRXh0R1N0YXRlIC9jYSAxID4+Ci9BMyA8PCAvQ0EgMSAvVHlwZSAvRXh0R1N0YXRlIC9jYSAwLjUgPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgL0YxLURlamFWdVNhbnMtbWludXMgMjAgMCBSID4+CmVuZG9iagoyIDAgb2JqCjw8IC9Db3VudCAxIC9LaWRzIFsgMTEgMCBSIF0gL1R5cGUgL1BhZ2VzID4+CmVuZG9iagoyNyAwIG9iago8PCAvQ3JlYXRpb25EYXRlIChEOjIwMjIwNTAzMDMxNzAxKzAyJzAwJykKL0NyZWF0b3IgKE1hdHBsb3RsaWIgdjMuNS4xLCBodHRwczovL21hdHBsb3RsaWIub3JnKQovUHJvZHVjZXIgKE1hdHBsb3RsaWIgcGRmIGJhY2tlbmQgdjMuNS4xKSA+PgplbmRvYmoKeHJlZgowIDI4CjAwMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAwMDAxNiAwMDAwMCBuIAowMDAwMDEyNDU4IDAwMDAwIG4gCjAwMDAwMTIxOTUgMDAwMDAgbiAKMDAwMDAxMjIyNyAwMDAwMCBuIAowMDAwMDEyMzY3IDAwMDAwIG4gCjAwMDAwMTIzODggMDAwMDAgbiAKMDAwMDAxMjQwOSAwMDAwMCBuIAowMDAwMDAwMDY1IDAwMDAwIG4gCjAwMDAwMDAzMzkgMDAwMDAgbiAKMDAwMDAwNzMxNSAwMDAwMCBuIAowMDAwMDAwMjA4IDAwMDAwIG4gCjAwMDAwMDcyOTQgMDAwMDAgbiAKMDAwMDAxMTAwNCAwMDAwMCBuIAowMDAwMDEwNzk3IDAwMDAwIG4gCjAwMDAwMTA0MzEgMDAwMDAgbiAKMDAwMDAxMjA1NyAwMDAwMCBuIAowMDAwMDA3MzM1IDAwMDAwIG4gCjAwMDAwMDc4MDMgMDAwMDAgbiAKMDAwMDAwODEyNSAwMDAwMCBuIAowMDAwMDA4MjkxIDAwMDAwIG4gCjAwMDAwMDg0NjMgMDAwMDAgbiAKMDAwMDAwODg1OCAwMDAwMCBuIAowMDAwMDA5MDEzIDAwMDAwIG4gCjAwMDAwMDk0MDYgMDAwMDAgbiAKMDAwMDAwOTgxOSAwMDAwMCBuIAowMDAwMDEwMTQzIDAwMDAwIG4gCjAwMDAwMTI1MTggMDAwMDAgbiAKdHJhaWxlcgo8PCAvSW5mbyAyNyAwIFIgL1Jvb3QgMSAwIFIgL1NpemUgMjggPj4Kc3RhcnR4cmVmCjEyNjc1CiUlRU9GCg==\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2022-05-03T03:17:00.513705\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.5.1, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["sns.set()\n", "plot_args = {\"color\": to_rgb(\"C0\") + (0.5,), \"edgecolor\": \"C0\", \"linewidth\": 0.5, \"width\": 1.0}\n", "plt.hist(imgs.view(-1).cpu().numpy(), bins=256, density=True, **plot_args)\n", "plt.yscale(\"log\")\n", "plt.xticks([0, 64, 128, 192, 256])\n", "plt.show()\n", "plt.close()"]}, {"cell_type": "markdown", "id": "f98767ee", "metadata": {"papermill": {"duration": 0.02111, "end_time": "2022-05-03T01:17:01.392485", "exception": false, "start_time": "2022-05-03T01:17:01.371375", "status": "completed"}, "tags": []}, "source": ["As we would expect from the seen images, the pixel value 0 (black) is the dominant value, followed by a batch of values between 250 and 255.\n", "Note that we use a log scale on the y-axis due to the big imbalance in the dataset.\n", "Interestingly, the pixel values 64, 128 and 191 also stand out which is likely due to the quantization used during the creation of the dataset.\n", "For RGB images, we would also see two peaks around 0 and 255,\n", "but the values in between would be much more frequent than in MNIST\n", "(see Figure 1 in the [PixelCNN++](https://arxiv.org/pdf/1701.05517.pdf) for a visualization on CIFAR10).\n", "\n", "Next, we can visualize the distribution our model predicts (in average):"]}, {"cell_type": "code", "execution_count": 26, "id": "da11313f", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:17:01.436004Z", "iopub.status.busy": "2022-05-03T01:17:01.435489Z", "iopub.status.idle": "2022-05-03T01:17:03.130399Z", "shell.execute_reply": "2022-05-03T01:17:03.129776Z"}, "papermill": {"duration": 1.720005, "end_time": "2022-05-03T01:17:03.133217", "exception": false, "start_time": "2022-05-03T01:17:01.413212", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUiAvTWVkaWFCb3ggWyAwIDAgMzg0LjcgMjUxLjk4MDYyNSBdCi9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUiAvVHlwZSAvUGFnZSA+PgplbmRvYmoKOSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEyIDAgUiA+PgpzdHJlYW0KeJzVnUuPLMdxhff9K3ppLZg334+laNoX0I4iYS8MLwSaskXwyqAISX/fcapmqk5VRwC0KAoICqRmzunJnvimK19VkfHhi2//8odvvv3tx8+f//zV48P53Tc/PtLzu8eHX6fnf//4lP884/M7+fev8vVHfP+I8t2nR5k1DPnq+7evckthzdhzEylev/2fx+P3jxhWGr2O2OZ83r+pK6bV45jPP+ENP7684PjmcXv1o2a8+wilbu8mv9kYoZHy/btS5fd8/5X2n2Jl+yV/eN6aK6WG+cxphFqff/r2+e/PPz4//DrvbH4j/34n/+5srhDbDC3lXvPldyOV3/vx1ePL5w/vDceQmnB/b3v79uOb+vhB/jzx+VkUq7UQt8Zibe25Qp1JWvvm0+Pzr58f/jU9U3p+/fvtz/X1fz3+4/lP8VfP/3x+/ZvHv3z9+HJ7v79HrA98YOQHeyytpUuwLP/8aJM0vbeWSq8/Idxef6l4c4qhr1r7JVxSf360OYqGxmLOo/yEaFOev1i4s0s4rfR1jZfkv0PAo4e8tVbSzD8l4JV/qYBLz2GOnqWDuXQsJP/8gEvLoWyt1bjSTwg4t/5LBby1M3Lo+AscXegh/K0hjpCffYbcV2p1pd736BBTiH2OhsjOUP8p/er59XePHtaSd22p9bwHj55LjFRCk396R7TPGuSCW+VsZEgMaKVsLxZGP+Bt4sAoFd+/TDWkmEvFz0kLY22/xjefQPyzL7797nf/9uevfvfHHz/79Ic//vnH5xf/+/zyl0AtF2uoUX7/fMAm6efgTrmGWPPIvY+V/yG8swPeMkVJo/VxfrhJ+lm8xwoz9bxmrWkR7zS2y/mXAJ5+OvAy14bvHw08l4VpWq40ITulv3kKUIR4Li20ueaIMu/5mT3KEFZlxtkEs807vvP+f4Aqe1zyuYhl5dTkbyT+NoOVAB6YKbzNYes25RVAf7Vf/7y8/rcfHz/xlTI9frQhs+eJMfUz+WyVIn26TM4HJqNLwh/oca7O9+RkRLf6mgvy0dRFlpnz5y7JGPF/enS0IlOv8kLmdFqUq6vLuHbhxapXLlb0wiXJLKWuqnA5HPlqTpmjX2mx6paLEb1wkZnSkL97feVyOF06r1lHTVdapLrlYkQvXEqY2wz3lcvh9CbTgiKr/AusU3RLxYhdqMi6Xb6K7ZXK4eCrsnqbd1aH6paLEb1wkRXgHsMLl8ORn+6jx9jvtA7VLRcjeuEiv3JerSpcDkcItCRLrHandahuuRjRC5cZukzjZVn5wuVwZGSWNTZGIKbFqlsuRvTCRZYkEf3oK5fDkU9GHWWVeaHFqlsuRvSfHiPKOqCXOF64nI4QWKm0su60DtUrFyt64ZJDmi1nhcvhCIFZcx3pTutQ3XIxohcuJRSZmdX5yuVwpCcpudeZL7RYdcvFiF641NDkz98VLoeDkUfaXRdYJLqlYsQuVFoYMtjKiviFyuG0Hlqda+ULK1bdcjGiFy49rBpXVLgcTpMRKDZsg19okeqWixG9cJkhSTOvS+nD6NgBi7GtCytW3VLRYxco0nEmbOO9QHk3msQ0tqUzkWLRLRM99E+PKW8aR38dnQ+jySUjy8JtK/sERaJXJkbowiTJS3uTruEFyuH0FFKfK09GxaJbKkbwgiWHOXGH/BXL4cjcZI2VtjuXZ0usuuViRC9caoij1vK60306vYYszW2jzdkSq265GNELlxZyL6UpXA5H+tZc8orlQotVt1yM6IVLD7VtD168cDkc3L5vtfV0pUWqWy5G9MJlhF5TWgqXw5HZbO5D1odXWqS65WJEL1xmmCWm9LrPfTo9yxCU8mgXWqy65WJE/+mxYpBXxvLK5XTkilktp5wutFj1ysWKXrgkGVXGaq/z3NNp0tf2XsuNFqluuRjRCxfpPfcnRF64HI4QqLPOmO60DtUtFyN64VJCm00CfeVyOCMLizVmv9Bi1S0XI3rhUsMYdaTXne7TaTLHbVXGngstVt1yMaIXLi3IUCv/e+VyODKDkx8es11pkeqWixG9cBkhtdza61r6dHoMc8w06oUWq265GNELlykxJPm7v3I5nFpCmWV/PvFsiVW3XIzohcsKrUgjrzvdp4MnfspqqVxoseqWixH9p0eKMYyEJ7VfwJCFMTnjifgrL1K9kjHjB5oUVtzucryiOaxWQk4z5n4jRrJfNgYAsClBJiOpaWwOq010tavXGzKS/bIxAIBNDXn2qF1R705bQf4/9XwDRrJfMnr4AIOHEl5nvu96rTJGzxrzBRWrfplokYNID71LEwqTd0fmt7ll+XgQKdb8MtFjB5URZsuzvm5+kyW9SF1pXVkdml8qRuzAskKsaXQNy2E1PJ4717bdwrRI9svGACBskrSDW/DKlXRaMijP2eKWB8XISHbLxgIANilUvFZjc1gNWYo9b9NcRkayXzYGALDJQQaZpuQHkCUQJlJz1wuyQ/bLxgAANiWMhZtECpvDqhM3BvZtTGqMZb9sDABgI7OS2YuSJ0BWa/IBmXX1KzKW/bIxAIBNRz6eRKiwOSzpdIuoL8hI9svGAAA2EgXy/JVZ8GnJxZPrKPVKjFS/ZIzwQWYGUZKSM0CWfDywK7Ovts/GWPbLxgAANiuM7XAIhc1htRFSjrPckZHsl40BQNjkGFaJS8kdIKsm5PtvCavUFqluyVjhg0wOEuxUsgfIanieqqTtgTtqjGW/bAwAYCPjTJxTySAgq0qoa4zRr8hY9svGAAA2VVbRYyhZBGRVJFKMNu7ISPbLxgAANi302buSSUBWl8soljLmFRnLftkYAMCmS3fampJNQFafYcReyo3YqfolY4QPMjPEXquSUUBWxaaeQJhXYCz7ZWMAAJsl8UkcGpvDwk240ed2k58aY9kvGwMAjt3Cj+aiLMAPR1YFct3gltMFGMtuyRjhA0yS8NS1wuHUjvvZ/Qrr0Pwy0SMHk4ztqKSM2ofTkO8X92y2syVW/XLRo9/ProvIt1G2zU+rVWGQ12pXXqT6JWPEDzQtyJQ2KlkGZA1ZQBakWlyJseyXjQEAbNBX9KVkGpAl090pg/OWmkONseyXjQEAbEZoo00l24CsugKOi2v9hoxkv2wMAGAjU9leh5JxQJYMz6X3t1GbkJHsl40BAGxWWNvBkAqbw6o19FSlE74hI9kvGwOAsJF5fqq5K5kHZFXkScb8NoCfyEh2y8YCADY5yFy2KdkHZDWJN6a+HZFCjbHsl40BAGxKaPu5wa9sDks+IHPlUvMVGct+2RgAwKbK3347CPCVzWHVEUZKc3uwnpGR7JeNAQBsZI6yZlYyEcjC7f800r6TdTbGsl82BgCwGSHOkZVsBLKqtN9znfWKjGW/bAwAYDNDHj0pGQlkYRsYT+3VGzKS/bIxAIDNkr99i01Zg5+WXDx5jrElllJjLPtlYwAQNhJfb7hv8srmtCoKBMi106/IWHbLxgIANgmnL00lO4Gs2kOLddV5Q0ayXzYGALApIRacVauwOSzpYlZfOa4bMpL9sjEAgE0NOcehZSiclgzbMhGWRdQNGcl+2RgAwKaFGnEkhsLmsGS611tq26EQjIxkv2wMAGAjfcaaTbuk3p3t4cbW9h2KsymW/ZLRwwcYmfRrScuHITOYXhb2ahgWq36pqLEDygxroNLNK5V3p2ccJ5L6ZFYs+qWiBy9YcHyXjMFalsJpyZUjsvxzwcWqWzJW/ECTcHZK1jIVTgtHlOay9h7mbIxlv2wMAGCTQ6vl9RT/79kqQ4ai7ecvyFj2y8YAADYljJK1OgZkYRsiz/22wtkWqX7JGOGDTA0LjrJzfloVydwrznQDRrJfNgYAsOkhxaXVMiCryEJbVgb7Lt/ZGMt+2RgAwGagHa2eAVl1hjZGW+WKjGW/bAwAYDNDnQOvfGVzWFhe15S2o24ZGcl+2RgAwGbhHP6h5SmcFpJ8UJ6o3ZCR7JeNAUDYDFkD9aZVNyALVTNzm+OGjGW3bCwAYCNz/la1CgdkyVhda0/74/hnYyz7ZWMAAJsSci1alQOyKg4cT6nfiJ2qXzJG+CAjf/mStToHZJUV2uxrfxr/bIxlv2wMAGCDtWLSah2QJZeOXDdtv99yNsayXzYGALDpYaao1Tsga5v+lt7uyEj2y8YAADYjrLW0mgdkyTRm4BS1ekNGsl82BgCwWSHNqdU9IGt78H7tybtE7FT9kjHCFzJze6Jeq31AloxGsXccqnwFRrJbNhYAsEmh9abVPyBLPh+jl7xnKhAykv2yMQCAjYTSqlID4XRqCauMuO8REzCS/ZLRwwcYCa+qTyAdTsWJhO3tIZsTF6l+uejRg0sLqWSlFsLpVPQsJeZ6pUWqXy569ODSMdPX6iGQJeH22faqZ8SLVL9kjPiBZoSWolYTgSwcq1zy3I/LOhtj2S8bAwDYTHn10uoikNWiDEJx3oAdol8uRvDgsnDulVYXgazaQlltzBsulv2yMQAIm5VCRGKyMmifliwhi7T41tWcyEh2y8YCADY55N61+ghk1RlGjXkrmsfISPbLxgAANiVUlFRR9s1Pq+UgSwNZMl2RseyXjQEAbGrotWp1EsjqCdvBfb+FeTbGsl82BgCwaWGWrNVKIEviTWX1fQP0bIxlv2wMAGAzQsxqvTSytpI0KY96RcayXzYGALDBPdqo1UwgC8ulFuO6IWPZLxsDANgsmaMsrW4CWTKRiTLXK+2GjGS/bAwAnx45yvvOqdVOIKuinNHa6tMzsVP1SsYMH2RSGGNo1RPIwsdD3qDeeJ2qXzJG+CCTw+pdq59AljCote+p71dgh+yXjQEAbGpIIilZCmRh0VTqzPUF2SH7ZWMAABuJr1atjAJZ0qtIe3NLRWVkJPtlYwAAmx5E0OookCUDdaltT9VgZCT7ZWMAAJsRRs5aHQWykKpRMs4AvSIj2S8bAwDYzLBSUuoonE5d8sXcjylkYCT7JaOHL2BSDK97oLuKLMJYZ0kXTKy65aFEDRYpZBxsr/B4dyaqAsRyQiLBLw09bhDJAXW/lLwEsmQ8zrngxEYmxapfMkb8QCPf9K5VUCCrDqSd1nYnRrJfNgYAsJFfHQcOalfSYUlHEuPMK1+RseyXjQEAbDoWPVoFBbLqxHK653xFxrJfNgYAsEH96KJVUCCrljBi6euOjGS/bAwAYCMDTc5aBQWyMP2XD8iWnsHISPbLxgAANiv0lLQKCmTJFHfWNbYtcUZGsl82BgBhk2OYMWoVFMhC0mmMY9+gIWQku2VjAQCbFJbMVZTMBLLk4qlvNRuZ2Kn6JWOEDzIlpDG1U1HJwqOdqcXtuIgrsEP2y8YAADY1lD60Cgpk4Ri11FNeN2Qk+2VjAACbFkTSKiiQhenvymXvbQgZyX7ZGADARq6LWrUaCmR1PHBfaqlXZCz7ZWMAAJsRVilaDQWysDQouW0JltQYy37ZGADAZoUkvpKZQJYM1SmX7XRhaotUv2SM8IVMiaFId6rkJZCFj4e80T4nPhtj2S0bCwDYpNCwWNTYHJZ0uXNKg+mKjGW/bAwAYIOnW5dWQ4Gs7RHYOPf9CUJGsl82BgCwKaiorNVQIEs+IDB7viEj2S8bAwDYtICTuJUl+OEg0b3O9EbmBEayXzJ6+ADTQ1azmE9nP3M5btlOhItUv1z06MEFvzO+fuXy7uBcuSTvk6+0SPXLRY8eXFBSpGp1FMhq8iHJdax64cWqXzJG/ECzgvyQVkeBLOlOsPnQ15UYy37ZGACETU1BdK2OAlm4lb1m2W9bEjKS3bKxAIBNDjlGrY4CWdh+qPJG6YaMZL9sDABgU0KZS6ujQBZurvT4dm/3guyQ/bIxAIBNDW1MrY4CWTjcPcrX+QXZIftlYwAAmxYGbkoqe+enJfHWPmpdN2Qk+2VjAACbLmNN1+ookIWLp62xT/oIGcl+2RgAwAb9adPqKJCF7c62Z/YwsVP1S8YIH2QWnt3UcpjJwjQmpzrbDRjJftkYAIQN6uTmolVRIAtPk6dRtqQnRkayWzYWALBJYaSsVVEgC8ORvM8+ShEykv2yMQCATQ7yYq2KAllbSYCUt7Ko1BjLftkYAMCmhriiVkWBLAxMpe53eM+2SPVLxggfZFrIY2o1FMhCKZLY2nbUyBXYIftlYwAAm465m1ZDgSyZBs86S74jI9kvGwMA2IzQ8SiRxuaw8DBWlwXmK7JD9svGAAA2MtLUptVQIKutkHpp7YaMZb9sDADCpuMubdVqKJAlONIabz3xSexU3ZKxwgeZhCellQoKp4Oa5lsy4YUWqX656MEDiyyg1QeQ3g2cm5v6dmzE2QyJfpmokQOJfBOTUj3hdFAEavXay4UUiX6p6MEDS5XrIGrVE8iSma6wWLFdcLHql4wRP9C0sMbSqieQhVLmMp3bd4XPxlj2y8YAADYyrGDo1S6nw8Kj0y0tBdkh+2VjAACbGUobWvUEsgRCL73sty4vyA7ZLxsDANis0JByoOyYnxZu4I459zt0hIxkv2wMAMJmRCyatfoJZKHTLbX1fkXGsls2FgCwwXntRaufQFZdAWq9IWPZLxsDANiUkFLGK1/ZHBa2O2uZ+0OOF2SH7JeNAQBstvPRhpalcFo4MLfOnOcLskP2y8YAADYN6bfqk0enhQMj+iz7/IaQkeyXjQEAbGQcxsHbGpvDwjZ5y2kfw8/GWPbLxgAANiPMPrX6CWThWPeSV81XZCz7ZWMAABvpM9rQKiiQhfuUVS6f9oLskP2yMQAImyntyDpay1M4ra00dXrbNSdkJLtlYwEAmyTzt6ZVUCALeXNZ+t9+Q0ayXzYGALDJARvfWqbCaeGokdhqWjdkJPtlYwAAmxJmKloFBbJkQFqx7sXVqDGW/bIxAIANztVLWg0FslpB9ztHuSEj2S8bAwDY4KyrqNVQIEu63zFjfWNzNMayXzYGALCRKMZSaiicTksoe/T2vOPZFMt+yejhA8wMrWtno56O9Lc1jbSd2Hi2xKpfLnr04LJQpkepoXA6dQoKtH6lRapfLnr0wmXJAIObbcoG+mnheZq6VTg6Wzo1t1Ss2IEFZyg3rX4CWU3ijSNvJcOYFsl+2RgAwKaEgqNFlL3z02oDyZb74eWMjGS/bAwAYFNDS0WroUBWT0FmLW93ds/GWPbLxgAANk2ui6zVUCBLFtgZI/W6ImPZLxsDANj0MFfSaiiQJRB6TbjNckd2yH7ZGADAZoY4o1ZDgSwssKWx/VwJQkayXzYGALBZOHxbq6FAlgzVRSY0+9752RjLftkYAD49SpQfRWXG17kwWa2HtlrfCvExMpK9sjEBgE2SPkNdJ5CFJUHOcXushhpj2S8bAwDY5DBL02ookFWReNnz9tD9Fdkh+2VjAACbumUnK3kKZKFwd0XexhUZy37ZGADApskcpWg1FMiS5WSvcW0JUNQYy37ZGADARsbhmLUqCmQhkWXknNMLskP2y8YAADZDxpqk1VEgC4ksK0uwV2Qs+2VjAACbiX1MrY4CWbLgzjXthfkYGcl+2RgAwGaF1ZdWR4EsFDzKuZf2guyQ/bIxAAiblAIy4pRMBbJaCXXOtdoNGclu2VgAwEb+9jLoKJkKZLUaykj74oCRkeyXjQEAbEpopSt1FE6ntdDSGiPfgJHsl4wePsDUMJSnkN51JLOMVLeMd0JFql8mWuQg0vYjjBQm7470JAPnjxCnQ/HLQ48bREZIODz4ddOcLFlZxyI42pUUqX7JGPEDDTJUklZNgSxZIMlSKZZ+I0ayXzYGALBZMt5GrZoCWVhY9zL3ddPZGMt+2RgAhE2WV/elVVMgCxPdmGO9EiPVLRkrfJCRq6JNrZYCWU0ii2vmfgNGsl82BgCwKSHWodVSIKvhocYWt1txV2SH7JeNAQBsUAmsa7UUyJKJyxAEY96QkeyXjQEAbFqouWm1FMiSiW6vaYw7MpL9sjEAgE3HSZ5aLQWyGqplzdnWDRnJftkYAMBG+oxYtGRmskYMZca3HfOzMZb9sjEAgM0Ma2WtmgJZ0unG9JY2R42x7JeNAUDYlBjSjFo1BbJ6lJXTnGVekbHslo0FAGxSKCirrLE5LDwCO2LcPzdnYyz7ZWMAAJscmvyckqVAlgxIK9f1xuZERrJfNgYAsClh1KFVUyCrLel0+/5YOSMj2S8bAwDY1LBK1+opkIWOpaTU5hUZy37ZGADApgcZcrR6CmR1TPJKqumGjGS/bAwAYIOnFqtWT4GsnrYN4K00NSMj2S8bAwDYzNBi0eopkNVwhHfssV2RseyXjQEAbJa8Oiv1FE6nZxRsmWldgbHsl4wevoCpMUy1+trpyFgk108t6YKLVbdcjOjBJYc4YlLG7sNpHVUv01blnWiR6peLHj24lJDb0uopkIWZ3ZDhqF95keqXjBE/0FT50anVUyCrl5BWG/uc5myMZb9sDABg03BSj1ZPgSyBMMrAG92RHbJfNgYAsOlh5qbVUyALG1cDFfquyFj2y8YAADYzYJtXyVQgSz4gTdqfN2Qs+2VjAACbFTLqUGtsDqtXlJfYH7tnZCT7ZWMAEDYthoLj05T989PqWG63GssNGclu2VgAwEbWQTNp9RTIEgg5rbkdkXVFdsh+2RgAwEZCQYlYZT58WtvFk2vLN2Qk+2VjAACbElZbWkUFsuQDEnFHd92QkeyXjQEAbFpIdWoVFciSAWmtkfYVJiEj2S8bAwDY9FDK0CoqkNW30mJtO86HkZHsl40BAGxGaNKxapkKp5ViClUwrHRldtH90jEQgM7c6rs3ZRV+WqmsgGTd/XYmUWPdLx0DAejgtJ6i1VUgK0kfnFAydd2ose6XjoFA6PSEqstaZQWyEiqM1Txqv1Fj3S0dCwHoZFRj0WorkJVyDwkHKbQbNdYPOh9+nRF6fv7mmUJGVOn58fnhi2//8odvvv3tx8+f3/z4uBH89CbIWn/MTfn+8fjqUYZcxJeXvSm31720dv9BetX5o1ZrXz7+D5+/gO0KZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iago2NzEzCmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjE3IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzk1ID4+CnN0cmVhbQp4nD1SS27FQAjb5xRcoNLwm895UlXdvPtva0NSqSq8iTHGMH3KkLnlS10ScYXJt16uWzymfC5bWpl5iLuLjSU+ttyX7iG2XXQusTgdR/ILMp0qRKjNqtGh+EKWhQeQTvChC8J9Of7jL4DB17ANuOE9MkGwJOYpQsZuURmaEkERYeeRFaikUJ9Zwt9R7uv3MgVqb4ylC2Mc9Am0BUJtSMQC6kAAROyUVK2QjmckE78V3WdiHGDn0bIBrhlURJZ77MeIqc6ojLxExD5PTfoolkwtVsZuUxlf/JSM1Hx0BSqpNPKU8tBVs9ALWIl5EvY5/Ej459ZsIYY6btbyieUfM8UyEs5gSzlgoZfjR+DbWXURrh25uM50gR+V1nBMtOt+yPVP/nTbWs11vHIIokDlTUHwuw6uRrHExDI+nY0peqIssBqavEYzwWEQEdb3w8gDGv1yvBA0p2sitFgim7ViRI2KbHM9vQTWTO/FOdbDE8Js753WobIzMyohgtq6hmrrQHazvvNwtp8/M+iibQplbmRzdHJlYW0KZW5kb2JqCjE4IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjQ5ID4+CnN0cmVhbQp4nE1RSYoDMAy75xX6QCFek7ynQ5lD5//Xyg6FOQQJr5KTlphYCw8xhB8sPfiRIXM3/Rt+otm7WXqSydn/mOciU1H4UqguYkJdiBvPoRHwPaFrElmxvfE5LKOZc74HH4W4BDOhAWN9STK5qOaVIRNODHUcDlqkwrhrYsPiWtE8jdxu+0ZmZSaEDY9kQtwYgIgg6wKyGCyUNjYTMlnOA+0NyQ1aYNepG1GLgiuU1gl0olbEqszgs+bWdjdDLfLgqH3x+mhWl2CF0Uv1WHhfhT6YqZl27pJCeuFNOyLMHgqkMjstK7V7xOpugfo/y1Lw/cn3+B2vD838XJwKZW5kc3RyZWFtCmVuZG9iagoxOSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDk0ID4+CnN0cmVhbQp4nEWNwRHAIAgE/1RBCQoK2k8mk4f2/40QMnxg5w7uhAULtnlGHwWVJl4VWAdKY9xQj0C94XItydwFD3Anf9rQVJyW03dpkUlVKdykEnn/DmcmkKh50WOd9wtj+yM8CmVuZHN0cmVhbQplbmRvYmoKMjAgMCBvYmoKPDwgL0JCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzkKL1N1YnR5cGUgL0Zvcm0gL1R5cGUgL1hPYmplY3QgPj4Kc3RyZWFtCnic4zI0MFMwNjVVyOUyNzYCs3LALCNzIyALJItgQWQzuNIAFfMKfAplbmRzdHJlYW0KZW5kb2JqCjIxIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzIyID4+CnN0cmVhbQp4nDVRu23FMAzsNQUXMCB+Jc3jIEiRt3+bO9qpSNO8H1VeMqVcLnXJKllh8qVDdYqmfJ5mpvwO9ZDjmB7ZIbpT1pZ7GBaWiXlKHbGaLPdwCza+AJoScwvx9wjwK4BRwESgbvH3D7pZEkAaFPwU6JqrllhiAg2Lha3ZFeJW3SlYuKv4diS5BwlyMVnoUw5Fiim3wHwZLNmRWpzrclkK/259AhphhTjss4tE4HnAA0wk/mSAbM8+W+zq6kU2doY46dCAi4CbzSQBQVM4qz64Yftqu+bnmSgnODnWr6Ixvg1O5ktS3le5x8+gQd74Mzxnd45QDppQCPTdAiCH3cBGhD61z8AuA7ZJu3djSvmcZCm+BDYK9qhTHcrwYuzMVm/Y/MfoymZRbJCV9dHpDsrcoBNiHm9koVuytvs3D7N9/wFfGXtkCmVuZHN0cmVhbQplbmRvYmoKMjIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA4MyA+PgpzdHJlYW0KeJxFjLsNwDAIRHumYAR+JvY+UZTC3r8NECVuuCfdPVwdCZkpbjPDQwaeDCyGXXGB9JYwC1xHUI6d7KNh1b7qBI31plLz7w+Unuys4obrAQJCGmYKZW5kc3RyZWFtCmVuZG9iagoyMyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDMyMCA+PgpzdHJlYW0KeJw1UktuBTEI288puECl8E/O86qqi777b2sTvRVMMGDjKS9Z0ku+1CXbpcPkWx/3JbFC3o/tmsxSxfcWsxTPLa9HzxG3LQoEURM9WJkvFSLUz/ToOqhwSp+BVwi3FBu8g0kAg2r4Bx6lMyBQ50DGu2IyUgOCJNhzaXEIiXImiX+kvJ7fJ62kofQ9WZnL35NLpdAdTU7oAcXKxUmgXUn5oJmYSkSSl+t9sUL0hsCSPD5HMcmA7DaJbaIFJucepSXMxBQ6sMcCvGaa1VXoYMIehymMVwuzqB5s8lsTlaQdreMZ2TDeyzBTYqHhsAXU5mJlgu7l4zWvwojtUZNdw3Duls13CNFo/hsWyuBjFZKAR6exEg1pOMCIwJ5eOMVe8xM5DsCIY52aLAxjaCaneo6JwNCes6VhxsceWvXzD1TpfIcKZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDM0MCA+PgpzdHJlYW0KeJw1UjluBDEM6/0KfSCAbtvv2SBIkfy/DanZFANxdFKUO1pUdsuHhVS17HT5tJXaEjfkd2WFxAnJqxLtUoZIqLxWIdXvmTKvtzVnBMhSpcLkpORxyYI/w6WnC8f5trGv5cgdjx5YFSOhRMAyxcToGpbO7rBmW36WacCPeIScK9Ytx1gFUhvdOO2K96F5LbIGiL2ZlooKHVaJFn5B8aBHjX32GFRYINHtHElwjIlQkYB2gdpIDDl7LHZRH/QzKDET6NobRdxBgSWSmDnFunT03/jQsaD+2Iw3vzoq6VtaWWPSPhvtlMYsMul6WPR089bHgws076L859UMEjRljZLGB63aOYaimVFWeLdDkw3NMcch8w6ewxkJSvo8FL+PJRMdlMjfDg2hf18eo4ycNt4C5qI/bRUHDuKzw165gRVKF2uS9wGpTOiB6f+v8bW+19cfHe2AxgplbmRzdHJlYW0KZW5kb2JqCjI1IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjUxID4+CnN0cmVhbQp4nC1RSXIDQQi7zyv0hGan32OXK4fk/9cIygcGDYtAdFrioIyfICxXvOWRq2jD3zMxgt8Fh34r121Y5EBUIEljUDWhdvF69B7YcZgJzJPWsAxmrA/8jCnc6MXhMRlnt9dl1BDsXa89mUHJrFzEJRMXTNVhI2cOP5kyLrRzPTcg50ZYl2GQblYaMxKONIVIIYWqm6TOBEESjK5GjTZyFPulL490hlWNqDHscy1tX89NOGvQ7Fis8uSUHl1xLicXL6wc9PU2AxdRaazyQEjA/W4P9XOyk994S+fOFtPje83J8sJUYMWb125ANtXi37yI4/uMr+fn+fwDX2BbiAplbmRzdHJlYW0KZW5kb2JqCjI2IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjE1ID4+CnN0cmVhbQp4nDVROQ4DIQzs9xX+QCSML3hPoijN/r/NjNFWHsFchrSUIZnyUpOoIeVTPnqZLpy63NfMajTnlrQtc4C4trwvrZLAiWaIg8FpmLgBmjwBQ9fRqFFDFx7Q1KVTKLDcBD6Kt24P3WO1gZe2IeeJIGIoGSxBzalFExZtzyekNb9eixvel+3dyFOlxpYYgQYBVjgc1+jX8JU9TybRdBUy1Ks1yxgJE0UiPPmOptUT61o00jIS1MYRrGoDvDv9ME4AABNxywJkn0qUs+TEb7H0swZX+v4Bn0dUlgplbmRzdHJlYW0KZW5kb2JqCjE1IDAgb2JqCjw8IC9CYXNlRm9udCAvQk1RUURWK0RlamFWdVNhbnMgL0NoYXJQcm9jcyAxNiAwIFIKL0VuY29kaW5nIDw8Ci9EaWZmZXJlbmNlcyBbIDQ4IC96ZXJvIC9vbmUgL3R3byAvdGhyZWUgL2ZvdXIgL2ZpdmUgL3NpeCA1NiAvZWlnaHQgL25pbmUgXQovVHlwZSAvRW5jb2RpbmcgPj4KL0ZpcnN0Q2hhciAwIC9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnREZXNjcmlwdG9yIDE0IDAgUgovRm9udE1hdHJpeCBbIDAuMDAxIDAgMCAwLjAwMSAwIDAgXSAvTGFzdENoYXIgMjU1IC9OYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovU3VidHlwZSAvVHlwZTMgL1R5cGUgL0ZvbnQgL1dpZHRocyAxMyAwIFIgPj4KZW5kb2JqCjE0IDAgb2JqCjw8IC9Bc2NlbnQgOTI5IC9DYXBIZWlnaHQgMCAvRGVzY2VudCAtMjM2IC9GbGFncyAzMgovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Gb250TmFtZSAvQk1RUURWK0RlamFWdVNhbnMKL0l0YWxpY0FuZ2xlIDAgL01heFdpZHRoIDEzNDIgL1N0ZW1WIDAgL1R5cGUgL0ZvbnREZXNjcmlwdG9yIC9YSGVpZ2h0IDAgPj4KZW5kb2JqCjEzIDAgb2JqClsgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAKNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCAzMTggNDAxIDQ2MCA4MzggNjM2Cjk1MCA3ODAgMjc1IDM5MCAzOTAgNTAwIDgzOCAzMTggMzYxIDMxOCAzMzcgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNgo2MzYgNjM2IDMzNyAzMzcgODM4IDgzOCA4MzggNTMxIDEwMDAgNjg0IDY4NiA2OTggNzcwIDYzMiA1NzUgNzc1IDc1MiAyOTUKMjk1IDY1NiA1NTcgODYzIDc0OCA3ODcgNjAzIDc4NyA2OTUgNjM1IDYxMSA3MzIgNjg0IDk4OSA2ODUgNjExIDY4NSAzOTAgMzM3CjM5MCA4MzggNTAwIDUwMCA2MTMgNjM1IDU1MCA2MzUgNjE1IDM1MiA2MzUgNjM0IDI3OCAyNzggNTc5IDI3OCA5NzQgNjM0IDYxMgo2MzUgNjM1IDQxMSA1MjEgMzkyIDYzNCA1OTIgODE4IDU5MiA1OTIgNTI1IDYzNiAzMzcgNjM2IDgzOCA2MDAgNjM2IDYwMCAzMTgKMzUyIDUxOCAxMDAwIDUwMCA1MDAgNTAwIDEzNDIgNjM1IDQwMCAxMDcwIDYwMCA2ODUgNjAwIDYwMCAzMTggMzE4IDUxOCA1MTgKNTkwIDUwMCAxMDAwIDUwMCAxMDAwIDUyMSA0MDAgMTAyMyA2MDAgNTI1IDYxMSAzMTggNDAxIDYzNiA2MzYgNjM2IDYzNiAzMzcKNTAwIDUwMCAxMDAwIDQ3MSA2MTIgODM4IDM2MSAxMDAwIDUwMCA1MDAgODM4IDQwMSA0MDEgNTAwIDYzNiA2MzYgMzE4IDUwMAo0MDEgNDcxIDYxMiA5NjkgOTY5IDk2OSA1MzEgNjg0IDY4NCA2ODQgNjg0IDY4NCA2ODQgOTc0IDY5OCA2MzIgNjMyIDYzMiA2MzIKMjk1IDI5NSAyOTUgMjk1IDc3NSA3NDggNzg3IDc4NyA3ODcgNzg3IDc4NyA4MzggNzg3IDczMiA3MzIgNzMyIDczMiA2MTEgNjA1CjYzMCA2MTMgNjEzIDYxMyA2MTMgNjEzIDYxMyA5ODIgNTUwIDYxNSA2MTUgNjE1IDYxNSAyNzggMjc4IDI3OCAyNzggNjEyIDYzNAo2MTIgNjEyIDYxMiA2MTIgNjEyIDgzOCA2MTIgNjM0IDYzNCA2MzQgNjM0IDU5MiA2MzUgNTkyIF0KZW5kb2JqCjE2IDAgb2JqCjw8IC9laWdodCAxNyAwIFIgL2ZpdmUgMTggMCBSIC9mb3VyIDE5IDAgUiAvbmluZSAyMSAwIFIgL29uZSAyMiAwIFIKL3NpeCAyMyAwIFIgL3RocmVlIDI0IDAgUiAvdHdvIDI1IDAgUiAvemVybyAyNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE1IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL0NBIDAgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PgovQTIgPDwgL0NBIDEgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PgovQTMgPDwgL0NBIDEgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMC41ID4+ID4+CmVuZG9iago1IDAgb2JqCjw8ID4+CmVuZG9iago2IDAgb2JqCjw8ID4+CmVuZG9iago3IDAgb2JqCjw8IC9GMS1EZWphVnVTYW5zLW1pbnVzIDIwIDAgUiA+PgplbmRvYmoKMiAwIG9iago8PCAvQ291bnQgMSAvS2lkcyBbIDExIDAgUiBdIC9UeXBlIC9QYWdlcyA+PgplbmRvYmoKMjcgMCBvYmoKPDwgL0NyZWF0aW9uRGF0ZSAoRDoyMDIyMDUwMzAzMTcwMiswMicwMCcpCi9DcmVhdG9yIChNYXRwbG90bGliIHYzLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjUuMSkgPj4KZW5kb2JqCnhyZWYKMCAyOAowMDAwMDAwMDAwIDY1NTM1IGYgCjAwMDAwMDAwMTYgMDAwMDAgbiAKMDAwMDAxMjI5MSAwMDAwMCBuIAowMDAwMDEyMDI4IDAwMDAwIG4gCjAwMDAwMTIwNjAgMDAwMDAgbiAKMDAwMDAxMjIwMCAwMDAwMCBuIAowMDAwMDEyMjIxIDAwMDAwIG4gCjAwMDAwMTIyNDIgMDAwMDAgbiAKMDAwMDAwMDA2NSAwMDAwMCBuIAowMDAwMDAwMzM5IDAwMDAwIG4gCjAwMDAwMDcxNDggMDAwMDAgbiAKMDAwMDAwMDIwOCAwMDAwMCBuIAowMDAwMDA3MTI3IDAwMDAwIG4gCjAwMDAwMTA4MzcgMDAwMDAgbiAKMDAwMDAxMDYzMCAwMDAwMCBuIAowMDAwMDEwMjY0IDAwMDAwIG4gCjAwMDAwMTE4OTAgMDAwMDAgbiAKMDAwMDAwNzE2OCAwMDAwMCBuIAowMDAwMDA3NjM2IDAwMDAwIG4gCjAwMDAwMDc5NTggMDAwMDAgbiAKMDAwMDAwODEyNCAwMDAwMCBuIAowMDAwMDA4Mjk2IDAwMDAwIG4gCjAwMDAwMDg2OTEgMDAwMDAgbiAKMDAwMDAwODg0NiAwMDAwMCBuIAowMDAwMDA5MjM5IDAwMDAwIG4gCjAwMDAwMDk2NTIgMDAwMDAgbiAKMDAwMDAwOTk3NiAwMDAwMCBuIAowMDAwMDEyMzUxIDAwMDAwIG4gCnRyYWlsZXIKPDwgL0luZm8gMjcgMCBSIC9Sb290IDEgMCBSIC9TaXplIDI4ID4+CnN0YXJ0eHJlZgoxMjUwOAolJUVPRgo=\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2022-05-03T03:17:02.301019\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.5.1, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["plt.bar(np.arange(mean_out.shape[0]), mean_out, **plot_args)\n", "plt.yscale(\"log\")\n", "plt.xticks([0, 64, 128, 192, 256])\n", "plt.show()\n", "plt.close()"]}, {"cell_type": "markdown", "id": "cf0a2cb5", "metadata": {"papermill": {"duration": 0.023298, "end_time": "2022-05-03T01:17:03.183116", "exception": false, "start_time": "2022-05-03T01:17:03.159818", "status": "completed"}, "tags": []}, "source": ["This distribution is very close to the actual dataset distribution.\n", "This is in general a good sign, but we can see a slightly smoother histogram than above.\n", "\n", "Finally, to take a closer look at learned value relations, we can\n", "visualize the distribution for individual pixel predictions to get a\n", "better intuition. For this, we pick 4 random images and pixels, and\n", "visualize their distribution below:"]}, {"cell_type": "code", "execution_count": 27, "id": "9e0e2604", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:17:03.230436Z", "iopub.status.busy": "2022-05-03T01:17:03.230097Z", "iopub.status.idle": "2022-05-03T01:17:08.533968Z", "shell.execute_reply": "2022-05-03T01:17:08.533373Z"}, "papermill": {"duration": 5.334101, "end_time": "2022-05-03T01:17:08.539894", "exception": false, "start_time": "2022-05-03T01:17:03.205793", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUiAvTWVkaWFCb3ggWyAwIDAgNjEyLjkgMzYwLjcwMDYyNSBdCi9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUiAvVHlwZSAvUGFnZSA+PgplbmRvYmoKOSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEyIDAgUiA+PgpzdHJlYW0KeJzc3U+T9bZ1Lvp5f4oe2gNBwMLfNbSvElV55liVO0hl4FLkXKv0OuWjcs7Xv+sBdxOLxKKPYunIyXLKztvP6mY3f3tvECBB8PMvvvnPP379zT99+ev3/+d3b5+vr77+/i29f/v2+a/S+79//y7/8x7fv5X//m/595f4+i3KV5/eWqLA8q/vXv/KLYYeY6MqUbx++f+9vf3hLQZOvZUe6xjv9y8Kx8Qt9vH+v/ALv9y+4fzi7fbdb2+lh/5OsQYqLQ7503JMIeeWW1HxdzrONYf68dfNn78k8+/98/tlw51rqfWd5Ptafv1feU9lBJLC8X/v/+ub9//3/U/vn/+KDrzfvMMyvfCuypVD60y9XP72lV7+orffvf32/c8fG44hVXlhZNvv2Pb88stX+vZn+X3x/bMopdpCmhuj3vg9jR7yaD2WuSdff3r79Vfvn/9jek/p/as/zJf1q397+5f3X8Rfvv/r+1e/efuHr95+O3/tTwLxdvyxO0RK8ndR5kgXCRX/BBQpxdfWcm8/nKKVn9mil8Bt1HylONOfQqKVULCxWHqlH06RaPy8FpQptJRT5QuGin8CDSIKCVsbtZfyX9Bg+pk1OMqP9j6ubZ2KfwqNzqFja631/F/4mFBtP6/GsfWcwhiplnujv/K/FaIHaYyoh0qUuuxdT8feY5+lTR1JkGXXl8Uv0i/fv/r2rQVm+WjVVBsdOmhNpZDktZH/tFbkBXovgVLk3M+NYHfWVkoYPLgLHJfXVsosiO6f5Q+Isfd36L/+mUpI0rAVbLDIK8bzTfD1J7xWn33xzbe//+e//O73f/r+s09//NNfvn//4j/ef/szvkjS3PTEQ95O1xdp5T/qRZJ+hvQ15M+vtXT1IsU2+s/8GsUf/hrJB23+ff9NXiOWd3qVN3W5vUYr/5uPNoQXiXOQl7qXUQvln+VFav9TX4uc5Q3NrSS6vhYq/1GvRaYWYu4xt0o8fpbXgv67vRZZ1GQcIfsfM1Oq8geFcgwvKFf5on0MMMocj4Q6WR++/+3y/f/05fN33rf8hu4+0WBq75/JQZfk4NtKSRheyctH3OTTcil8pwq5tNCbHIzwjlgbusQykvm1H60HFAxG5T0dM21YHzmNFJL0/0u9EF5iZ1YmyaTinGIahtWrQC1LJ4LSfActQp26s7JMBIvCyKlLQ3rHOgs0sO89x3Y11LEzLRtFtHLomXCEumudBRrSOFHsqd0QVexMy0YRrSI7kuVAe8f6yKU5l65tya3fCFXszMokEaoaas4j9s3qLBBXURmFx4VQp86sbBPBaqHIAS2WDessUB9hSE+rXglV6szKJhGrLt3rSnH/EJ4FYpKOdKSRr4Q6dqZlo4iW/OW5DuZN6yyIR4ipx5uhSp1Z2SRixdKxbJXbZnUWaJQQicsN8AydSdkgn956xHiUeBvprALJ567nhM/dBVDHvrQeUKaWDLGZtxZrFWjMpinS1VCl7qwsErFK0r8cbWxDnVXAAJBKxKn/K6GKnWnZKKIl/86cxzbWWQXqPcRapKNwRdSxMy0bRbRkR0qMY2+1zgLOno5RE40roo6dadkoolVCLbhef8f6yHGikyvHwRfCS+zMyiQRqio/lkrfxjqrQAM7Fzl1TXhJnVnZJoLVQi6U+t5mnQWSd1Om2mu/GurYmZaNIlpd9otG387PrALGgKW3QjfDlTqzsknEaoRUcm3bWGcV0KPqsaXer4Q6dqZlo4gWh1gKtb3ROgu4BsjUBucroo6dadkoU4tL4badoVkFTOvopVIeN0QVu9OyUD69jRhGqa1tI55VmJMAem5cr4g69qX1gCJaSd4gLdetq7UKOZKMc4joYqhTZ1Y2iVhRmJNKt3M0q0BV9q63Nk/IrA1dYmdaNopoZelo9l63Ec8qEEOo10NrIerYmZaNIlpFfk4+WhvWR07ouLO8hehKqGNnViaJUFXpZ7L8/ZvVWaCB2SycMl8IderMyjYRrCb7xdYb6yxkzFuU7c7TomtDl9iZlo0iWh3nPGvZRjyrgAv1PY/cL4Y6dWZlk4jVCLFiqt+d6iOXDmiQQeDI+QqoY2dWJsmk4po4b8OdVchRfrbxcTX13M4ldWdlmQgWh1Gp5f2NdRaooe85xigXw0vsTMtG+fSGAV7NeZ/KtgoYBMZae70h6tiX1gOKaKXQaom0naVZBZzoky21OW1NIerYmZaNIloUMEn+Ptn0O1Ugls4VSVeBr4g6dqZlo4hWDrJDZT8grgIuQOd5LeeKqGNnWjaKaJWQa0u0naVZBWLpXsWRG18RdexMy0YRLdmv2sY+CXcV5o1dbbTZVdCIKnamZaOIVpO+phzXtjHPKhwzjKQP2q6IOnamZaOIFi7GD0rbeZpVyFG2zam1fEG8xM60bJSpxXXs02nOnBoH7uLTLoSX2J2VQSJUQ7qa+L7N6izMucm4TYgvhDp1ZmWbCBbLoDjmuLdZZwFvoT5oUL8a6tiZlo3y6S1Jy9Nawuy0G5eqUMfFr87zlgrFqGNfXk8sAEtBOgRtn4urKtJPkH5o4nklWkPq3JuY7QIxCqVR5q35UpUcc0hEac50UNu65N7EbBeI5SB7gfubN7GzklMMTIRJk1dJnXsTs10gVqRznvvenVAV3EcnzTvPOW1qW5fcm5jtArEakvx7GO3YWaF5D6L8b7lLqtybmO0CsRZiq2lsZ3FUBVdcqfdCd0mdexOzXQ4xbvJz24hIVdBH7dJZnYunaEmd+xOzXCDWA04xDKMdOytYK0a2zLHfJHXuTcx2gdiQ/minfYqzqmCefJZ/xnqXVLk3MdsFYiz90c773F1VOc47D9nkTVLn3sRsFyyIJr+2jdb3dmxVMCdicCvzaqyW1LkzsQcXiCXpjxrXaVch0wg1lj7PEKotXXJvXqYKuChkLNpoeJ0V3GbAvWAWnHa8xN68bBaAZelOxb7P5FWVo3XPcd5poCF17k3MdoFYCaknGSTuYmcFS4Px6MeqnVpS597EbBeI1RA7YbnDTeys4H7FUajfIVXszctWOby409hn9aoKzhlSG5nGzVHn/sQsF4i1MHqu1WjFzgruahmjtJvjSr1p2SbHsrQYQdd9bLQqGUutJWmqboqX3JuY7QKxEVovvE/wVRXqWNU1Cc5V8pJ7E7NdIMah4iujDTsr1CvWLovz7paLpMq9idkuIkYRdyjmso+NVgVz63uP6dXXV5Iqdyb24AKxJD3Rvi8TtAoZp1qj6PSr4yX35mWqgItw5rTnvRFbFarzHqo8J1CsTV1ib142C8AyFkqWEeEOdlbk94TMfRxn9hWkzr2J2S4QK9IP5bTP/VUVzJaT/sMxt/AiqXJvYrbLIcadjRP7ZwG3ruc4jrWntKPO/XkZKuCq0luPlfbzO6uC6SdUcp+3GCtHHXvzslkA1gLWet/nAKsK7mDHicLjrL6C1Lk3MdsFYj20kXifB6wqlCOeQBHpLqlzb2K2C8RGqINa2s/vrApWhcu1xVJvkjr3Jma7QIwDbjdL++hoVbCibBxFxkR3SZV7E7NdRAxrx48S9znBqoLZFL3GwnST1LkzsQcXiKUgH66+L8+rKlgDtOQ25hqgWlLn3sRsF4hRSKOWuA+QVgWzwlLLY65RpbZ1yb2J2S4Qkw7CaGmfI6wqmE0Rc6NjDufa1iX3Jma7HGI8pIdgtGNnZd7zkWsZ/Sapc39ilgvEivRHjTkWHzkuEfWajt6+RlyxNytLBFJV+qGYFb1bnZXZ30qNjrmby1Cl3rRsFXg16YWOwcZ766zgSCi/63W2Qjnq3JuY7QIxLMDBlffzO6uSaV6MHC+xc1uX3JuY7QKxEQpHMmYHrwo+fcQV+3qRvOTexGwXiHHIHHkYrdhZoYYLuOOYkKIgVezNy1YRr4LNpDb2MzyrgtW+GhYap5ujzp2JPbhALIXElI25wauClfd6j+O4mqskde5NzHaBGIXIWfZsFzsrxFjWXtr6epPUuTcx2+UQY86972d4VgUz80tp0mG9Sercn5jlArEcBsvP7iOjVZlzgGummO+SKvcmZrtArATZq7QfKM8CXLq07vOZCTfHM/fmZaqAq8r3Vhkd7l5nBV2uMmTzfHNUsTcvmwVgDQs51raPj1YFi66mIePufIW85N7EbBeIdemKdjLmBq8KVZYtpuORtGpbl9ybmO0CsTlNYn/I0CrgrtJKtXO+Oercm5epAi7GVlo12vyzglFQymkcnX3lqGJvXjaLgNUo/VDOxszgVUHr3vrr87gYV+pM68EEWtJrjzHWvf1aFbiM3MdcreKmeObexGwXiEmvPcZe9pHRqmClvlHjmM9w1JI69yZmuxxi8r3FmBm8KvO9VCkfbb6S1Lk/McsFYvI2iSSlXeysYLlfjtxzuUuq3JuY7QIx6bhHebPsZ3hWBTKR8jhmC1wlz9ybmO0CMem5x1yNycGrQqOHmuXfdJdUuTcx2wViDbfXUjbasbMi3dOQW6z1Bqlib162Cryk5x4L5/38zqrgzo8csU7RzVHn3sRsF4hhgm9tBthHAXOACeelN0eVe/MyVcAlXffYsjE3eFXQVFGSTbero469edksAtak7x6lDd/P76wK7ixKsr3jUpuC1LkzsQcXiCXpifZO+/hoVY7ngWElgZukzr2J2S6HGEdph/bx0aqQDLpzL9TGTVLn/sQsF4iR9EQ5Gaf1VyXjVraKe+Svkpfcm5jtArEsPVHGrVab2FnB6h6VyvE8FLWtS+5NzHaBWAmYUWjMDV4VXCOqI7/WrNOSKvcmZrtArAaJKRrt2FkhXP5olfNdUufexGwXiLVQpIcQ9/HRqmAFmVa4tRukir152SrwkjFhombMDF4VwMjbqB6zz6+OZ+5NzHaBGAY5WcaFu9hZwRnDPs8dXiFV7M3LVoEXS5c9M+/neFZlzjGXXiptjir3Jma7iFiPIabSjBnCq4L9pS5bLDdJnTsTe3A5xDhVnKA3xF4V3OmdZd/KuEnq3J+Y5QKxFIYME/eOxVnAldsYG+d6c9S5Ny9TBVzyhTTkxvTgVUFj1csYxyoWy1HH3rxsFoDJriTZbaMNOyu4WSa3NpjvkCr3Jma7QKxIR3SksZ/lWZU5IwzNe7lLqtybmO0CsSo90TGM+cGrkqP0UeX3HPMs1rYuuTcx2wVicqhLXI35wauCp0e2NHqvV8lL7k3MdoFYDzTXbNrFzgqeC1IrjWN+sJZUuTcx2wViI2A43ffx0argaWw5cuYbpIq9edkq8GIZ4sioeh8frQrJm6pXtPU3R517E7NdDjEmkh6CJfaqUK1h5PHxFluSOvcnZrmIGG5UoByNi7qrQjXJsFveTHSXVLkzsQcXiKXQKRsHyrOAa5HcSjreYcpR5968TBVwSd+ASql7I7YquOiBZ5dWvjrq2JuXzQKwOY081X18tCqyc2G0kY9+hYLUuTcx2wViGOLUYcwSXpX5dOrR87GyjJLUuTcx2wViNWRq1lvso4BzX5FrHOPmqHNvXqYKuJr0QzuV/RzPquDkV2ulHgPw5ahjb142C8C69EM7GzOEVyUTbjDtRHyFvOTexGwXiA3ph45WjDbsrOCq2mj8WtF7beuSexOzXQ4xJs55P8ezKseT0JmPVfaVpM79iVkuEGMZ4MRozBBeFTyvocpHcD4NXUvq3JuY7SJiHEPPsRtzhFcFT2ygzNKHuEnq3JnYgwvEkuxKKnk/x7MqWGkN8wSO8aSS1Lk3MdsFYoQPVqJ9gLQquFJUmepxAUlJ6tybmO0CsRxKlo/X3n1dFZxhZYrtWMNISercm5jtArEScs6VjHbsrOA+I+RH70JJ6tybmO0CMdmzXIyJ1WcBq7hmOs67KsWVerMyRUDVQsqFjQnCq4KmPSeqx7T9Zahjb142C8A6ls1syWi/zgpWh4+x9uOhgApS597EbJdDjGU3kvFxPCto2VtkOtan1pIq9ydmuUBs4Ob3aEzjWRU6LuFSv0Gq2JuXrQIvll5o79FoxM4KnvXdE+XjvhDlqHNvYrbLpzeK8s1Ynns7x6MqcwzE0ttqN0md+xJ7coFYkl4op31+sKpQw4PSc5rrFGlJnXsTs10ghmXfufPWiqkK7pXkSNz6TVLn3sRsF4jlkHHT7XaOR1Vme0U5zjUYtKTOvYnZLhArsmcp7TOEVQV3zPSM1QVukjr3Jma7QKyGJB0E40P5UZhPTJT/1Bujir1pmSbAknYb18620zuqgsW7O70uQipFHXvzslkOMMaph210pCokI8ican21+QpS5f7ELBeI9TCkCd/nB6sKxo2ppT7nUmhJnXsTs10gNkIvZZ9Ytwoku5cyngl1c9S5Ny9TBVwcWqnSdd+9zgr1hNVvZdNXRx1787JZBCzJr8We7WOjVcF8Q07lo9+6IHXuTOzBBWJJfrL1vrdhq4L3UpN9n8+S15I69yZmu0CMpBfaS9u7YauCUaOMhdpc61xL6tybmO0CsSx7NtI+O1hVcuqyv1S5XiUvuTcx2wViRbqiYzSjHTsruAaZWq/1JnnJvYnZLhCr0hfl2vZzPKuCT18Z/bhKpCV17k3MdjnEuEbaJwirCmY4kYwg51VbLalzf2KWC8QalmkVgF3srMwlK6gdz/+7SZ65NzHbBWIdN8O0up/jWRWcL8RpCh43SZ17E7NdIDZCq/ihDeyjMGe2Fh6l3hx17s3LVAEXh1pzLEYjdlaolcAy5uZyc1SxNy+bRcDmwmC5l/0NtioYCUmbdTx3RkPq3JnYgwvEUsi1lH2GsKrgvVRjrvEuqXNvYrYLxEj+9pr2GcKqQlX6XLhKVG+SOvcmZrtATLrstWLxuU3srMiu40HWba5NqiV17k3MdoFYCbHKYW8fIa0K+lyx9jxXrdCSOvcmZrscYlw77TOEVWXONqdCcw7KRVLl/sQsF4hV6Yt2JqMdOys4L51HinMOipbUuTcx2wViuPl9oCnaxM7KfJ5KKrXxTVLn3sRsF4jJsLBy3mcIqwrW+UjyP3SX1Lk3MdsFYiPUFiMZ7dhZwfNMK8fex01S597EbBeIccCOpX2QtCpUZH9ZfOgmqXNvYraLiOUYsLDtPlNYVebqO9LYv3oXS1LnzsQeXCCWgux4Sns7tipYh6FSTKXfJHXuTcx2gZg03o32hw+vgvwW+eHWYrk56tybl6kCrox5mHWfKKwqmFRRBo1xc9SxNy+b5QDjVmQHLbBXBc9O5MQ81624QKrcn5jlAjEs4oFHyO9iZwWj7ZzS6zipJHXuTcx2gVgNvdVqzBNelXkfyOj9uIakJHXuTcx2gVgL0nKTMU94VXAOrCbZa75J6tybmO0CsS590ca8n+lZFWq4DJLTMQlKSercm5jtArEhfdHe2OhanJWMCQNzCfQLpI69edkq8GLpiY5sTBVeFaoUcu2d+Op4yb2J2S4iVmQz8t1jb8VWBYPIkWIpfJPUuTOxBxeIJemMch/7+GhVcEQsIlXbTVLn3sRsF4hRiN26snsWcMYwEc0HaWlGFXvTMk0OLO4pdVPrVcEwaHAeR0dsKerYn5fFArAcRk+j76OjVcHTEyvn2G6OKvbmZavAqwQ5yFVjnvCqkCDgyabH3CflqHNvYrYLxGpo3XgK5Spg3mFjnF29Oercm5epAi4Z2/SM2feb11nBYwga53wMjJajjr152SwA6wFTpI05wquCNWTSsZL3DVLl3sRsF4iNkGU02Iw27KzgNCvTa5KFglSxNy9bBV4cqLdY97M7q4JZh6OOVOvNUefexGwXEasxpN66MUN4VdCTGCXlXm+SOncm9uACsSTd0C5/+C52VrCmfpVfNFdb05I69yZmuxxiMsxJdT+7syoYDVUZOs4V/bSkzv2JWS4Qkw9WH8MYSq4KrkEWPu4+Uo4r9aZlm0ArSz8Uu7VrnRWciY6UytGGKUWdexOzXSBWQhsRJxw2sbOCmScpcjtmPWlJlXsTs10gVkMd0TilfxaoYzXq2tvdUefevEwVcGGGb2rG9OBVQVOFhddeH8nlqGJvXjYLwHrAre/GbTSrglGjjIXKsViKgtS5NzHbBWLyt+P5FfvYaFVwSjrWxo1ukjr3Jma7QIxDGrkbs4NXZZ6ZyETHvW1KUufexGwXEWvyNhmlGKfEVgWXu7nWYxVELalzZ2IPLocYj4pHIhpirwru9y7SwpcbpIr9eVkq8EphDHmT7GOjVcE/Y6zxuH9SOercm5jtAjH5YrSa9lZsVea5nB6Pc2IacsXevGwVeM3HFlHaz/CsSk7yThoU+cqoY29etgq8inREOxvzgldlnmfNKc+1ldW2Lrk3MdsFYjVgWYpktGFnBSdapbs6eJNUuTcx2wViLeD5kXE/v7MqeFv1FGulm6TOvYnZLhDrspkYjbnBqwIZOS7G16fyInnm3sRsF4iNgCekGHODVwU/nqUzkfguqXJvYrYLxDhETkbP9ZUf5ygS53FT1Lk3LcvksGLGA4QtrVcFZ3Ki7Hq5IKrQn5VlIlo9hsF4BNumtSqYfiL/PZ40phV17kzswQViKXTOhffWa1XQs6dSyjHdSUnq3JuY7QIxkm82F1teFbytpO8w8iapcm9itgvE8Egs9Kp2sbNyyLREZEm+cm9itgvEcHGx1mG0Y2cFJ6blVw2+QarYm5etAq8aMpu3TK4Kzqz2nGtuN0edexOzXSDWZDOYFb2LnRWMsnsfvd8gVezNy1aBV5deaG/7Q+ZVBf3T3NrrDkDlqHNvYrYLxIZ0REfu+/mdVcGduLWncdwJoiR17k3MdjnEmNm4j/ksYMFgFpXjbjblqHN/XoYKuHB3grxfjCbsrGAWXUo5Hst9LEcde/OyWQRsSM89xtL2czurgrPSQ34X3RxV7MzrQQVe0jGIKRmzglcFczNLkr4X3Rx17k3MdoGY9NtjMjr6ZwHLoaRWYu03R5178zJVwCW99jiftbl5nRXcgvuxGOJSXKE3K5sEWNJjj5mM+cCrggmHhGeB3BF17k3MdoGYdDtlI8Z84FXBsgvY++PWPyWpc29itgvEpM8uzXfZz+2sCt5LBU8Y45ukzr2J2S4Qkz57rLkYTf5Zwf1YSbZ8TEBUkjr3Jma7QEx67bFFY07wquCGLObXO0w5nqk3Ldvk0OKIOZeW1quCS5C41+OYVqEUde5PzHKBmHTaYy95HxatylzMI7ZxrLe2tnXJT7HPf0Wy4+/0/pv3JC6yT+n9y/fPv/jmP//49Tf/9OWv37/+/u2iOuTvmEGuOATHNu9XffvdW44p5MNYf6+K7z+xbdfcxPpW9fN/dbuf/yphn6LsUwyc5Dg3sbcvyof7fEHxcr5/efmmt7/2E29vWY4Kcb6tLnvR5N/9sgNHcvkz1c9ecnlh/jBfmPUNl1/wX3tT05s4JLy279/Kf60XV34cz/NjzIrQL8WK77zqbRNDqu//fm59fvnlK33781sS0c+ilHLD46NkaxHDgPc0cGm89Vjmznz96e3XX71//o/pXX7bV394i/JjX/3b27+8/yL+8v1f37/6zds/fPX2259VpVDCfCQsunR546/4J1ApqRxbGz1z/+EqrfzdWDrLh45Su7cHZ/xTsDQ861u2dqwc84NZEo2/l0vNcjTqeCraxUXFP4FLpRYKtlYwS+KHszD93VhY3uCUCtbg1Swr/ilY5sIv2FrMnX64C9X293LRBw55UdsYsaV15FjR39zephSq2hGikKVzPrCC9DhMICF/BFbqEI8F9Iv0y/evvn2bvdGaa6qNDjI0xlKQXkSV/zRpyKVSAqXIeW0Ex+m1lYITi9zFkMtrKzQLQv5n/P7YpTf0Hj/+iadzRRmeYoOy6c7z7/v6E17Az7745tvf//Nffvf7P33/2ac//ukv379/8R/vv/07vnK1hZ66dAjWK7eiv/2Va5dX7lhoTHp5GW3hz/HKsZsXSEajvVRcpTtfoBX9VC/QIDQ3LdLASsY/xwvUvLxAoOxdjmbtfIFU9BO9QDnNk/W1JZKB0M/yAmU3L1BpMqrMg+t6gVb0N79AcjTSLxCuppRScXm4/cgXCKdT8oijysvy/PrEj9fnJ4TNb1Pgbz998Pb8/f+10wfyd4bKBJrPuOHaXhmc02tMJx8twhImuvKdruAF7i31OaFBbeuSn6cP3NnZQrDDHJnMuHhxtzsrOFonyscDMtW2LrlfO1sIdpguUxouY9ztzspcf0PeZCPfTVXu184Wgh0mzqCt2+3OylwqlAslupnq3K+dLQQ7TKJpsRjt3VmZa9TGHOluqnO/drbQYcdYEKFZdq8KHW81muMdbapzz3aWEOwwt6YXXPK4252VOZeXX9O6tKnO/drZQmLXY+g80n6oOAto1XqtnfNNVOdu5WwfwGE5zDFwbegud1YwCSBTK8eBYonq2K+cDQQ6GZAwmv2d7qxgllyRJq23G6nO/drZQrCbJ3Ap0253VnDPRxIhppupzv3a2UKww5QcGaLudB8FnMilnnLlq+gl9ytn+gAOM3NSo30wtirEPUQ5Jox6Eb3EfuVsINBhio68hYwD7FlB/w1L+1G7kercr50tBLt5fimS0dadFeKG1YtiuZvq3K+dLQQ7TN2R4+Y+FlsVnJ5Prbacb6Y692tnCx12HEtJ+1hsVWh2SZjn4t7aVOee7Swh2GFiVE3WgOKs4I5B6f22NO6mKvdrZwuJ3ZABgvxc2s89rQougMVeOLWbqc7d2j0IwU4+drHVuA/HVgVLhLSGm0pupjr3a2cLwU5GCLFT3Nu7VUE/rhyTSK6mOvdrZwvBLktHt+MZnpvdWcmEBQN7mTeZqG1dcr92thDsivR0h/Gue+W4B4wH15qvnpfcr5ulAzXZr8iynd3trOTY5FNZei7a85L6dbN9INewCjhWmt/kzgrGDbKtOpdAVdu65H7tbCHYyTA+RZw62uzOCu7TZ86t30hV7FfO9jnkOKU89pHYquQoR4LUeq5X0Uvu2c4Sgh2eYklyjNztzgpxwu11svNX00vu184Wgh1W+yF842Z3VuZ0Omo4OFxNde7XzhYSO5ZvTrmMfSS2KjgiVOmDjHEz1blbuwch2KVQU0l9b+9WZT7dUfZ9PpBDm+rcr50tBDtMniyj733iVSHhKqnGeDfVuV87Wwh2OeRUa99HYqtCjP1NpW6mKvdrZwvBDpM3G3WjvTsr88xmi/WYbadMde7XzhaCXQ0pNW77eGxVcOW1y+H0OHuiTVXu184Wgp0MqlJvbR+RrQoxVl8tdTPVuV87W+iw4zRkRyy7V0U+ooFbGoOvppfcs50lBLsuPV02TniehYydj9T4Lqpzv3KmD+CGdHO5G9OyVwUHU5bPaG0X0UvsV84GAh2HRhF7tdGdFawr0HvNdBNVsV852+fTW4nyaymlup17UhU8baDLZ5RuoCr2KvfkA7mEizNjn5GtKrjan2mkTVTnfu1sIdjhFIj89dtBQlVyxGMa+nE/tdrWJfdrZwvBLgfCNJzt3JOqzEYtlp6vpDr2K2f7QA5318sIYRuJqQo+mRUr/NFdVOV+7Wwh2NUQZdf2KdmqQoxH3B1PbNOkKvYrZ/scciz7k4227qxg1jpHHvNxilpU557tLCHYtTCoxbyNw1QF59PlqNrn8lkXU5X7tbOFYNdDp7afslsFHEkzce90Fb3kfuVMH8AN6eH2QkZTd1bQ960k+0oX0UvsV84GAh3L7o9E2zhMVTDDJMaIJ1ZeSXXu184WErskP0nyY/tIbFVwzTqXcTxsXW3rkru1exCCXUKDZQxizwK6vjmnMmdKqC1dcr9ypg/gsOBwpLSdc1IVnAwWkDpvbVKiOvYrZwOBToah+XgY2p3urNC8QWzwMRBTpDr3a2cLwa6EmPG4pN3urGC9z1pz57upzv3a2UKHHWfCarGG3auCccPIKad6N1W5ZztLCHZVdl8OlvtYbFUwt4lxQyzdTHXu184Wgl3D86d6NNq7s0JzUmersd5Mde7XzhaCXZddKSXuZ55WBbP9U6uYqnM11blfO1sIdtLc5xr3OdmqMq+5csx0M73kfu1sIdhxKBmTvna7s4LTczJkPa4dqm1dcr92thDWR45YlLvst7SrCvrAY8Sc76Qrdiv34HOsLE3W7bGrgDNMo+cxbp6X3K+c6QM4kk5uH/uUbFXJESsO95r6TVTFfuVsINBl6eOOOvaWblXmE81TH3NVHbWtS+7XzhY67Fi2MfbR2KrgIWuZW+YbqYo9y1k+kMNwinmfj60qeF7k6F2+uIuq3K+dLQS7GnqJrRuN3VnBuKFTpnkXpzbVuV87Wwh2DTfz576PxVYF/bdRYznGscpU537tbCHYdTy/I+7zsVUF8665jXbMAlCmOvdrZwvBbshPUm9Ge3dWcH5O2jVMwb6a6tyvnS0EOzlSllzafu5pVTBwqNx4zvfXpjr3a2cLiV2WzWAZk30stio5YZGdmOYKJ2pbl9yt3YMQ7FJIpYy2t3erQl26JdzLYbe2dcn92tlCsJMdlX2q+7mnVUEXuMjvKfVuqnK/drbQYcel0T4nW1XQj2uUZfdvpjr3bGcJwS7LF826oL0qeG4s7uws5W6qcr92thDsivR0+37b0yoQc5jP5Bo3UZ37lTN9ACefuTJy2QdkqzJvDMuxHBd4lKiK/crZQKBr0svFSms73VnBPcTSgRvzvv8Lqcr92tlCsOvyk9LHMNq6s4L5EpGovY4Ty1Tnfu1sIdiNkDGU38djqzLH+yXm0W+mOvdrZwvBThr7mpIxL3tV0P8dPdd6N9W5XztbSOwKHoWUpPna7FYF/V9uqZZxN1W5W7sHIdilECvh1urN7qzMOU7Srh3nn5Spzv3a2UKHHddMxtTsVcE8nVQjHb07Zapzz3aWEOzkiyob2du7VaGBJ2eXdMwFUKY692tnC8Euh15leLWff1oVLMUmP93ajVTFfuVsH8iV0GrdH6mwCpjfVJv0SMrNU+d+5UwfwNVQa4vJaOrOyuyGlNFyu4rq2K+cDQS6Foq8c9J+CmBVcMdEoThyuZJecr92thDsOh7SW4yZ2auCT2bP5TjxtDalY79ytg/k8LcbC4yvAvE8KmDQf/NUuV850wdwLP2yMeJ+3mlVcFKzy7i11quojv3K2UBCV6P0brkac7JXBWMGzDJ59ekUqcrd2j0IHXbyjWTMyV6VTEn2d/BxXWxt65J7trOEYJfCaHHwft5pVTC/KQ1+zT9Z27rkfu1sIdhhWbpUeR+HrQruha1tHE9N0KY692tnC8FOdqURGXOyV2WuhTW4x3Yz1blfO1sIdkXeOsRstHdnZd5rUl6zPDXpiv3K2T6Qq0F2HpcFN7mzgtUm2siJ+01U537tbCHYtSD7kI152auCMZe0bpzy1fSS+7WzhWDXA+FBMEZrd1bmqs6R67HoztrWJfdrZwvBbuCmuW58ZD8KWNFJ9pqOkydKVOd+5UwfwLHsZyvGtOxVyUmOqtK8NbqIXmK/cjbQQcetp260dWdlzgqLox5XY9e2LrlnO0tI7Jo09a2Pvp93WhUcD2TIxaNcTS+5W7sHIdgl6eOOaszKXhXqsvt9UO1X00vu184Wgp10zxpT25u7VcGxVEZex6RsTbpiv3K2D+Sy9HEZU9I3ubNC8zHPZRzHWC2qcr92thDsSpAfbMac7FXB2yu3yMfditpU5X7tbCHY1ZDxuPC9a7cqaNWYPxo7Rbpiv3K2D+SaNPWyf/uZp1XBtQg8IbvdRXXu184Wgl0Pqcv7Zx+Lrcp8jl3s9Zi1o01V7tfOFoLdCLHnsneLzwLh0WIjpeMMgBLVuV850+eAw803xWjqzgpJHrG67k1Ux57lLCDQcRi9DOtNd1bmSD/F/urYLVKd+7WzhcSuRzwRoRozslcFs+lk33u5karYrdyDD+RSaL3R3iU+C7jqKr3fWPjmqXO/cqYP4CjU3jjvw7BVmYfRRK9F7Zaojv3K2UCgy9K97c2Yi70q+FgO2fxxLVaTqtyvnS0EuyL925GN6YmrQm3uYDkuiilTnfu1s4VghxNHHGk/67QquHA48JTnG6mK/crZPpBr0r/lbszEXhV8Mqts8Liuo0VV7tfOFoJdD3HEYh1izwrubIoybj3WtdemKvdrZwsddhhM0X7WaVUw07+O1/2dilTFnuUsH8iNMEYaxmTsVcGogXrK4w66Yr9ytg/kOODqgjEXe1Vw93BLo7zauiWqc792tpDYjYipI5T2c06rQsew9bjZSZGq2K3cgw/k0FplY6rTWUCDFmvKhW6eOvcrZ/oAjoKM45sxF3tV5hg/Ni5XUJX6dbN5AJdDHjXHvZ1blTnGx42cfAPVuV87Wwh2JchRMsZ9FLYq1PCcjpgT3Ux17tfOFoJdDQnPfdlHYasy7/RPKb7ed8tU537tbCHYNene9sxGY3dW5gpYvcWSb6Y692tnCx12PEbkfRy2KoRLYG3E3m6mOvdsZwnBrksPd3Sre3JWsNJkHLlWvpnq3K+dLQS7IV1cli93u7OCO0xiz68nx2pTlfu1s4Vgx/Obx37eaVXwPKwaUzxWs1OmOvdrZwuJHcuv5TiMudirQrjZhNM4nkSpTHXu1u5BCHYpyH/r2Nu7VcHwIXdp5PhmqnO/drYQ7CjIJsiYyL4q6Is0onIcZ5Wpzv3a2UKwy7IZYmM+9qpgvF9ab73eTHXu184Wgl0JibP1+NhVed1B3I+nFmtTlfu1s4VgV4N01Yxr2WcBPREZtPZjRKZEde5XzvQ54JhrNCZjrwpVIWpcU7uK6tiznAUEuhYG127Mxl4VrJbYi3xQ241U537tbCHYdemetdL280+rgvs4S0/peH6sMtW5XztbCHZDvrmnuo/HVgVGGXeb8G565n7tbCHYyZCK+zDmY6/KNEq1mKYfuV87W+jTW8UyTTxq3do7VUEfLvHc2s1U5V7tnoRgJ0Mq3AaxjcdUBX241AjTJm6mKvdrZwvBjmQzzPuUbFXB3OueY5pPd9amOvdrZwvBTkYIMUrTv9udFZwb7tKlG/luqnK/drYQ7GSMEFMu23hMVXLEk7GpzHZNbeuS+7WzhWAng4RI+yJZq4AJiaWPEctV9JL7lTN9DjiO1LPR2J0VDB6axHwT1bFnOQsIdDJCiPIW2s49qQp1DPy5zqfYXUhV7tfOFoKdjBBiSfusbFWRw2goqdVBV9NL7tfOFoKdjBBwY/VO91HAGmKRZcP5KnrJ/cqZPoDD9S05Sm7nnlQFD4rFIjFMV1Ed+5WzgYQuyU9GXGjY6FZlHg+yxHdSnbu1exCCnYwOYuN9TraqYJ5OlUHXuJGq2K+c7QM5GRvE3vbHjKsK1YGnYsuo/yaqc792thDssvRwR077SGxVqJWQqfa5WoI21blfO1sIdkW6uPLd+yF2VbCaU+ryy8bNVOd+7Wyhw44j932RbFWhjlX/an7ZKVOVe7azhGCHyZqxxH0wtio4muK2k1zvpir3a2cLwa4FYUj77GxVmb24VuucLaFNde7XzhaCnex+SrKd3e6szLPCo+TjrJ0y1blfO1sIdkN6aGS861452jTpwB3P/tOeOvfrZulAjWUIP5fM3dzOCtUm76w4Ur546tSvm+0jchRDTlmOkpvcqtCQQyqnyHwT1blbuwch2KWAhzXzftZpVXDlUIpzeZML6Yr9ytk+kKOQUqV9Traq4InOffR6nHXSoir3a2cLwS6HmCqPva1bFcwKq4VeVyeUqc792tlChx3jcc37eadVwf1hiUfmfDPVuWc7Swh2Rfq3Pe9zslUFRkT5dT3xanrmfu1sIdhJkyW9jG60d2cFc69xDafeTXXu184Wgl3DLf2972eeVgXXv+LA4+zvpir3a2cLwU6a+8Rln5OtKvN4GvOId1Od+7WzhWA38Fi/ZBxmPwro/3JvLW2iKvcrZ/oAjkOmONp+2mlVMMyPcZRXB2WJqtivnA0kdFk2I1to+2hsVUh2MEXqqdxIde7W7kEIdikkItrnZKsK5oSlVvq4karYr5ztAznZUaL98WKrgLNL8rPHM061p879ypk+BxxTbnU/RqzKPMGUG89biJWojj3LWUCgy2FQycZs7FWRncOdEr1upCr3a2cLwa6ETjVWo6U7K3Okn+rr/Loy1blfO1sIdjU0qr3sI7FVwbWvGDuVzVTlfu1sIdi1IOODYszGXhUsXpeYeK42oU117tfOFoJdly6u9M2M9u6s4LrhoI8jxSJVsV852wdyQ3q48n37eadVwVOdC+VxzDxRojr3a2cLwQ6n3UY1pmOvCsb5Ocrbq9xNVe7XzhYSuxKll4sex2a3KnOpzlpau5Ou2K3cgw/kknRymfN+1mlVcGauDjmujpuozv3a2UKHHWfZVYvuKGBtGJLjwTEWU6I69yxn+ABOvsgpG7OxV2Xetl458riK6tivnA0Euhx6pmjMOlkVQjd49BLvpDr3a2cLwa7IrsgbaB+LrQrGq6OVVulmqnO/drYQ7GqQkWkx5mOvCs4w5Z5TvJvq3K+dLQS7FooUjMsTq4IV2ArWiWk3U537tbOFYNeDQIy0j8VWBXPWecjQq99Mde7XzhaCnfztuVZjPvaqEB6gkNKxsrM21blfO1sIdoxTvhSN9u6sUMuBeaSX3TLVuV87W0jsagwxy7fvo7FVoYoFFMfr/hNlqnO3dg9Chx3n3oz52KtCNYWC23X63VTlnu0sIdgl6emOHPf2blXw/qodjz29m6rcr50tBDssXjKk1drtzsq8mzN22fubqc792tlCsMOucDPmZa8KTjPJEaG2cTPVuV87Wwh2JdQSMVFzszsrVORjKps/7qNQpjr3a2cLwa7KTyaji3IW5hm62hq3m6jO/cqZPoBrQUYI3ZiWvSpYcjJ+PGRHierYr5wNBLoue0ZlGG3dWZmPE2u9vPoni1Tnfu1sIdiNkEpOYz//tCpENeRRIt1Nde7XzhaCHYdY8jBmZa8Knp5QqMfjbgBlqnO/drbQYcelVGNW9qrgWexDdn0+aUeb6tyznSUkdi3KF1WOl5vdquBcCadcyt1U527tHoRgl0Ivlfs+HlsVzHFK0s4da3hoU5X7tbOFYCdDqtKaMTF7VdD/zYXyMTFbmercr50tBLss/dyOU+ab3VnBvH9qqR3HCmWqc792thDsivzksC5qrwrObw6Kg+6mOvdrZwvBTrpoZRiH2bNA0sAxp0h0F1W5XznTB3BNdoxLNRq7s4LbEluu8bg7W4mq2K+cDQQ6HCajHCl3urOC+eujjVjajVTnfu1sIdiNEGscxtzsVcE1xF4+bglQpjr3a2cLHXZczbfdRwEPKc6jjmONcS2qcs9yhg/gOIgClf3c06rgcJClZZsPitGiKvYrZwMJXY+hV2JjXvaq4HjAJbe4karcrd2DEOxSaFX2Ym/rVoVyC3IgjaXcTHXu184Wgh2FWkvO+7mnVcG4ocVcjjkoylTnfu1sIdjlILsUjZnZq4K7YUuWDfLNVOd+7Wwh2JWQq9T3sdiq4EoOtzLyuJnq3K+dLQQ72TMZYOX93NOqYL3OEUdM426qcr92thDsmvRye6J9OLYqGOt3OTgc8yiUqc792tlCsOvSz+3DmJ+9KlRKSGmUWG+mOvdrZwsddlylv2a0d2dl3otdOlG+mercs50lBLshPV02prafhbm+Tuot55uozv3KmT6AY9xPwsbk7FWhWuWD+brHeIGq1K+bzSNwQ765xZb2lm5V8KkkPCy23UFV7tbuQQh2mAiRcto/rqsio9bAXbZ4I1WxXznbB3IUSqNoTHpaFYz1S0sp15uozv3a2UKwyyE36nFv6lYFz04YqRTqN1Od+7WzhWCHy4K5GI8ZWxXqeHpi2UhV7FfO9oFcxd4nY1b2qlBOIUasiHUT1blfO1sIdk12tHQ2WruzQqXi7rp8XBhTpjr3a2cLHXbcajGWyloV2ab8OL9WQFGmOvdsZwnBrofRWjJmZa8KtSQtXIvHKFaZ6tyvnS0EuxF6a8Zy2WeBsDxWlv1uN1Gd+5UzfQDH0sftUt/lzgqlFHgUPob/S1THfuVsIKFj+bVtkDEre1VwFSf1dEzKVqIqdiv34AO5JD3cwcac7FXBFetKOR6reChRnfu1s4VgR9LQszFF8Sxg1MBj9ME3UZ37lTN9AJeDjAxy3wdiq4I3V+LUS7+K6tivnA0EuoLWKhrzsVeF5J8UU8k3URWfcp//it5lt+n9N+9JVGSP0vuX759/8c1//vHrb/7py1+/f/3929I9ZYf8LSrOIlxjbHM20Nvv3pp8W79885Hcv+9hy/cfv3+v2szTlj//VcJ+RdmvGOT90g7u7YvyIT9fUryg719evuntr/3E21vp+Et7yOX15+ARvfl4o634Ox2nXkPlOo8488d1IC/MH+YLc9vwD3wPzxcz4cV8/1b++7/f9lcTN5h3PuaYrz97pfqvEUj1Jokh1fd//9j08eWXr/Ttz/Kr4vtnUUq4FDG3Rb3xO4cy8PSirz+9/fqr98//Mb2n9P7VH96ifPtX//b2L++/iL98/9f3r37z9g9fvf32/+6+y2E+ZMoc6bLzKv7xe59SfG0s9/YD9r6Vn2/350XxUfN178/0J9h5zOnFtmLplX7A3icaP9vuY3yBM6GVL/uv4h8PQPMBNrKxUXspPwSA6ecDkL4b1T7vRdQAK/4JAOZVDtmY9K/zD3n/U20/G8DccIs4IjYe19Z6xX/zm58Cvdcqx4skDUDE9Ke529hZOXSNXrHLy+AX6ZfvX3371rBYSs011UaHClpEKSQ5pFXcpgyGdzl4p8h5bQT7gq3w/GbB+zN+TZQjPXBf/0wFC+3ngp+TLXSef8bXn/BSfPbFN9/+/p//8rvf/+n7zz798U9/+f79i/94/+3P8hpwC1UOEVSur8GKf9RrIL1FSjIeHngX/iyvQfsf+BrImzTIGKTfei0r/lGvgTTsMjgcuZca+Wd5DfL/xNdgPqir0fYirPxvfhUGXgU5Hg+sjiP7nuqPexm6COYRhwwp+vOrED9ehR/Dl2VH337MUOzy/W8/aiiGhXKZ2vtnKXGWv77UlmmOO7AObMNclEvlO1UZ6A31cZzoPLek03Mg9j8d6olDoFKIOco/N6ePQi5BxuplrgGztqNTP0w2xlRiLBE5LKZXpWT8ohqbxtOhJyZTQ5zmcq4dk0ruTmcFe9/GuOup1A/UA4dAzbVbsXDBBnVWKuGph73WC59O/UA9cAgUFmrNWAPx7vRRqLJzciie59kUnkr9MNkYojSXZB2YGnJnOit44+RW55oFC0+FfpgeNMRpLr9aMRHk7nRWapcObT7WFlV6KvUD9cAhUFhrtVI0PndnRQbTuF7Sr3or9MP0gCFMc1nVgQked6azIi1Rw8pmNzyV+oF64BCouYZq5XtP/TtVQV9pjDIueir0w/SA8emtY7nUTrwPW1ZF+uBlpDzvFV1b0qkbqCeOCSXjW+a9fVoVLIaH5aLzhU+nnqBMDoGaC6G2sY9cVkV6TTKWO1ZIXVvSqR+oBw6BwqqnnMc+dFmV2SzRqDc+lfqBeuAQKPw7xmG0UWeloj8QaT4HdW1Jp36gHjgEaq5o2o1P3kcBj5+s3OeUf4WnUj9MNoYozcVLcQv6xnRW8MYZtc9bXheeCv0wPWiIE1YqpdSN9umsYJlw5jZPyik9lfqBeuAQqLks6ej7qZVVQT+8pTxHc2tLOvUD9cAhUHMN0tr2wcuqyDAl9thGufDp1A/UA4dAYcHRQs1oos6KkEjrXYnvfGfqB+qBY0JxKdz20yur0qQ7wD0xX/h06gnK5Pj0NuZSoq3tw5dVkdYokvwrX/lU6gbqiUOgsG5oy3XvRK2KvHcwRzzmK59K/UA9cAgUFgmVH9xPsKwKzvqOnObsXMWnUj9QDxwCNVcE7XUfvqwKnpOca5kdAcWnUj9QDxwCNRcALXt/8yzgKeZNRnZXPJ36YbIxRAlrfbLswM50VlrCogx4cJnC06EfpgcNcZoLe5pvp7Mi75ySR87toqdTP1APHAI1V/GsZR++rEqroXXqc+762pJO/UA9cAgUFu1MtPc1z0LjkGWzs1Op8FTqh8nGmEpcEyaFGUyvirxxxqjtaJ9OPBV6YjI1xGkuxtmy8XY6Ky2HNEqba+ooPZX6gXrg+PTGWHkzZ2PG2Kq0hpYIv+XCp1I3UE8cApXmFV7aT6+sSsdVu0xzNLe2pFM/UA8cAjXX1OzbLM7vVKVhGTB0Ay58OvUD9cAhUHMBzWIc8lZF3js1HfPD14ZU6IfpAUOYsFZmS7SfXFmVXkNPjedCkApPpX6gHjgEai6MOYxZrauCTmXkY7rh2pJO/UA9cAjUXAWzGtNaV0W6lXVEeftc+VTqB+qBQ6Cw5OWgtJ9cWRU87yx+fPTOLenUD9QDx4TiOozpK2ehdTwWLl/oVuaJyIIQobmSZTOmtK4Krh/gydxZw+nQD9ODhjhxwDOio9E2nRX0kyLuarno6dQP1APHp7cU5Zub/O8+clGlLvtPjeftP2tbOnVD9QgCq7kuZTMmtqoSVAq3+Um7EX7EjrAeRIA1F6LMvDdVqoSJdbhlsN8MVewI60EEWFh6MkfeBzGq1Cmk2Cn3m6GKHWE9iABrLjbZjX6CKslbKKeUXljLUMWOsB5EgDVXlyzDarPOEs7+1krzJLnamI4dYT2IAAvLSdY09pMuqjRSiCW1Vq6GOnaE9SByYHGTH9yHNarUB1bBOtb4VxvTsSssUwRYc8nIOqw26yz1IpvMeGb91VDFjrAeRICFRSI7GZOEVUkOe5K1yjdDFTvCehAB1lwXko0psKo0YuhJ9q5dDXXsCOtBBKtmzZUgWzfarFVqOKNXOLaroY79YD2JAAuLP1oXSVdFPm3cKc5FIdWmdOyIyvaAFIXcYzTmwqrSXKdKNtivgip1RPUAAqu5umM3psOqEuaXjUYpXQl17AjrQQRYc4VMGegZWGcJj1QaLZaboY4dYT2IAKuGiEvpxlBnleTzJm+hOq89aEMVO8J6EDmwpIkextRYVZKeZ0/SBS03QxW7wjJFgNXC6DIUttqss4SH/lI8Hi+tNqZjR1gPIsfKpL0XqsZQZ5VwY1+V8U26GurYEdaDCLBGaL2wMVNWlfCsRsb04Zuhih1hPYgAi0PttRWrzTpLGNNQf40L18Z07AjrQUSwSH60t1yMoc4q4bAnO5duhjr2g/UkAqwkXcxurICzKh1PIqut3gVV7IjK9oAUSf+y92w0WKuE8Z+06vPKoBJUqSOqBxBYZelejpKNUzOr1KV/kCvVcSNUsSOsBxFgzbUTkzGFVpWkFY8Uj9myamM6doT1IHJgcWfr1PtZ6dApWBH1IqhjV1SWB6QwYz9WMk7NrJIMlRNxn3OLlaBKHVE9gMCqhT4SGVNpValjYZdSN0IVO8J6EAFWD20kNibUqlLHg9X68Wh1bahiR1gPIsAaoQ5qyTg1s0pYIUDeSnNGtjZUsSOsBxFg4QkEOSdjmLNKvQUiij3fDFXsCOtBRLByDHmUaEyuVSVRqVE+bmMzPGM/WE8iwEqBRunGwrGqhJlqkcdcC04bqtgR1oMIsCikUUs0RjqrNHKQfufx+Ae1MR07wnoQAVbGxYZkTLhVpV4DD6zBfzXUsSOsB5EDi0eTDZlYrxKWqqqvuSAXwo/UFZXpAaqCJ3tY76pXobP8/5E2PRU7gjI1wFSlayndJWOUs0odlwab/Pvip0JHUA8eoGrSsRyDrffUWRoUpMsZj5Mya2M6doT1IAKsLh1LrmyclFkl6aBX5jSfHq02pmNHWA8iwBqhcCRrqu0q9RIwla+km6GKHWE9iACLQ2bpT1ot1lk6bj0dI98MVewI60FEsAq2k7C04Ia1SrjFq43jqQXaUMV+sJ5EgJVCYsrWVNtVksapEB3re2lDFTvCehABFgXpeMdutFmrNHpodVApV0MdO8J6EDmwmHPvxomZVZKhX4+x17uhil1hmSLAymGw/LAxyFmlUcUnHc8V04YqdoT1IAKsEvp8hulu9VEZmF1LdAdcqSMoWwNOVb65jmack1mlUWQU2OMxeXT5qdQR1QMIrFrAVKpmDHRWSd4/lEj2+0aoYkdYDyLA6tK57GRNtF2lMRfdO1b/0oYqdoT1IAKsIZ3Lbjy2ZlU4hZioH2fc16Z07IjK9oAUYzOtWg37WZKj3Ug1Z74I6tQR1QOIWNUoHUvO1iTbVRpYILT22G+EKvaD9SQCLOmFRyyibmCdpSFNVIqvaVjKUMWOsB5EgCW9cOl7F2OQs0ryFpKtFRo3QxU7wnoQObDkm4s1yXaV0EGnctxcrwhX6orK9ACVdMMjSc2gOksDj2bJ1PpNUMWOsB5EgCUd8UijGKdlVmng1tRWRrsZqtgR1oMIsLCUXK7WLNtVGniiubyD+GaoYkdYDyLAmtM4KFst1lkSlTgqUdsMz9gR1oMIsKQvHotkBtZZwgMjmMfrY7gMVewI60EEWNIZj/KFYfVRGTmk1l9HwgW4UkdQtgacpCMe5d/GMGeV5KOWc27z0YnKT6WOqB5AxKpFLNcUyTgts0rSRW+F6zG7TxGq2A/WkwiwkvQteydjmLNK8mErA08ruxmq2BHWg8iBxVF6AMYwZ5WgwlyKYfgRu8IyRYBF0rnkZJ1yXyWsZ9+TAN0MVewI60EEWHnO/UzGQGeVpHFKMk4+1sVShip2hPUgAqwSMFXPmmS7SniSC7d6TO9Thip2hPUgAqwaJKdotVlnCY8paaUft/QqQxU7wnoQAVYLRbJoDHRWSVSwpAX3zfCMHWE9iABLdkuGwdYk21Ua6FqlSHdDFTvCehABlvzxKedotVlnSQ57uY68GarYEdaDCLA4pJRlIGxgnSU8gSrjuQk3QxU7wnoQEawe8TSSZk23XSXMJJJOQrsbqtgP1pPIgcWpZjbarFVCH0EOe2XcDFXsCssUAVYKQ4Z7Rs/hrLDo9BrrTVDHjqhsD0gR5st2a67tKjHJMW/047avJahSR1QPILCSL5J8iqz26izh9B5mIvcroY4dYT2IAKtI93KkYZyeWSWc4StJftPNUMWOsB5EgFWlezmGNdV2lfB4qiT9hSuhSh1RPXiAqknnkqs10XaVpHcQmWIsV0EdO8J6EAFWD0SRunFyZpUYDfqQ7sHVUMeOsB5EgDUChsPdGOisEh7bzLHcDXXsCOtBBFgcIsmw2BjorJKMaGIt/birUBmq2BHWg8iBhQXtm9VmnSVpnHIffFyqUIYqdoVliggWriRTjtaV1VViadOpDW5XQx37wXoSAVYKeLyL8cY6K9JByLWPnG6CKnZEZXtAikKjUqrRYK0SjnkN69deBVXqiOoBBFZyNKOaqjHQWSWcVKiFjxsw18Z07AjrQQRYRdqcOqzptqs0nxKee0w3QxU7wnoQAZa8O6iZb6yPinzaapGOAl8FdeyIyvaAFFZr6lSMEzOrNHDky+OY5nduS6eOqB5AYIXV+Dpbk21XifFWkm5CuhLq2BHWgwiwpM3BNXervTpL8q9S4/GYk7UtlTqievA4qJgY99lYVK8SRjOyuZ5ugip2hWWKAIvDyDFaU21XaWDRbR71bqhiR1gPIoLFMfQcuzXVdpXk5+UAmI6nBihDFfvBehIBVpJ9SSUbJ2ZWCfcB5FaP4bMyVLEjrAcRYFGomRIZ45xVkrdQKz0di9oqQxU7wnoQAVYOJdOwJtyuEk4qsHQVxtVQx46wHkSANadjV7LarLMkKsyZjxH0xfCMHWE9iACrYs1ja3LyWcFzemOSrY0r4SV3hGWLwKqFlAtb821XST54vVI8VhI7t6VTR1QPILDqIebaktVinSWoSI/9mGB0JfyIHWE9iBxYLPuRrE/hWcKJhT76ZqhiV1imCLCGdDB7tCbNrBLj6euD6W6oYkdYDyLAYulWdiwqsGOdJTy1vkszftwrt7Z2yR1xPZh8epMGWvZlFOMhOqoElsG9HlNnroxn7obr0QRc0vZkTsaUW1USoDwizROiamM6doT1IAIsLLHDnfd2S5Xkw0eFx7yVUBuq2BHWgwiwMnarGOvzqFKK+G0lz6kfamuX3BHXgwm4irw/EBpcZ0neRNxKIr4q6tgR1oMIsGpIJRlL3K4K3kHSNh13fKltXXJHWLYIrFqIuLC1n6VRJXkH9ZJlkxdDnTqiegA5rBinEfYhjyph0Fx6K3P5MLW1S+6KyzQBVw9DGmxj4q0qyZuottZTvyrq2BHWgwiw8GEqxiy2VcHnrbcR6yZ4xo6obA9I4dxwlR65QXWWpPNZcuZXN+sUVKkjqgcQsUrye4t8qzHcWSVRqaPmMjbCM/aD9SQCrCQ/2nDH/I51lnBepnCds4q0oYodYT2IAItkt3ppRidrlbiFztT63VDFjrAeRICVpX85kjHtVpXmoGaUcoyj19YuuSOuBxNwFelhDvTBd66zhLNYTfpW7aqoY0dYDyLAqrPlacYpmlWSTxyn+dCvq6GKHWE9iBxYXCMZc29VCW+hWusoN0MVu8IyRYDVZIej7LuBdZZEJY9Yjk7pxfCMHWE9iABL9q2mVo0TNKuEE8hEQpSviJfcEdeDCbhGaBU/tWt9VObZ9shc6o545o6wbBFYSUe85lisRusszQ8cp7ma39qWTh1RPYCIFcmPyuilGG+sVZon+bDM9JVQx36wnkSAlXCHRDEm4KoSVFKuPNfzuyGeuSOuBxNwkfzxNRmTcFUJrVPMOJl8Z1S5I64HE3DlkKS7lI3zNKuUIpbjzlz7nVHljrgeTMBVQqxyfDMGPasEFizZOh/jcWM8c0dcDyYHl7w5yJiKq0py+IsDz7C/KurYFZYpAqwqPc3OZLVcZ4lZRjr1mAuiDVXsCOtBBFhNepqjkTHsWSVczZGdPxZBVFu75I64HkzAhQfscTYm46oSjn61DM58Z1S5I64HE3DhwlaMZLVbZ2ke/TIeU3FnVLkjrgcTcHHACfZkjH1WCV3R2l4fRYV4po6oHjyEKseQWyrGhFxVeg2e6dWbV4Qq98P1ZAKuFKhRSka7tUo4QRrz8ehCtTEdO8J6EAEWhdTIeFrtquDIh5+uV0EdO6KyPSCVQ2y5GvNxVQkoPaYxJ8BfCc/YEdYDyaHFrVC02qyzlDCVTToMxxwRrahyV1ymCbhKGA3PFze4zlKK0r3qnbnfGVXuiOvBBFw19FarNSF3lVKigNXM52NENaPOHXE9mICrhdYaWVNyVwldUCww1vKNUeeOuB5MwNUxRY3ZOGGzSphxNIjjuCrq2BHWgwiwhvQ0e2OrB3GW0KAPzHrgK+Ild8T1YAIulq7myNas3FVKsQTpsY9jdrxmVLkjrgcT4SqyHfn2YbRcq4TeFXOspd0ZVe6H68kEXEk6nNyHMfRZJXQX0C09zkFoRpU74nowAReF2M3Lr2cFKDI+LMd80yvimTvCskUOK+4pdRvrVYJK6dTmWsFXxDN2hWWSQEuOaz2Nbgx/VinFHlKlfsx004oqd8T1YAKuIgMYqtYE3VWaZ7Iyccp3RpU74nowAde8Jmg873BVZlehYg3cO6LKHWHZIrBqofbMzcI6Szij3Euqx1BRIarYEdYDCbR6wNxka4LuKqWUAu61OK6+KkWdO+J6MAHXwHmq3Kx26yylOPAUC9nCjVHnjrgeTMDFgXqL1Thts0oplYA1vEe/MercEdeDiXDVGFJv3Zqmu0r40I3BH715xahyP1xPJuBK0t3s8pcbXGcJJ//mgvrpzqhyR1wPJgeXjGRSNU7crNJso+qQX3hj1LkrLtMEXCT9zTGsweIqgSWX0SjtjGfuiOvBBFxZepxcrem6q5QizpamETdGlTviejABVwltzOkfO9dZSiliuXiMea6MOnfE9WACrhrqiNYJ+rOCBn3IP9uGqHJHWLYIrBo+S82arbtKKcm+11ap3hBV7AjrgQRaslvyU9b9K6uUElbKwK+7K6rcEdeDCbjkjx85ZmMAtErouefS+nH/q2ZUuSOuBxNwYV3l3K3puquU0pBfkNsYd0aVO+J6MBGuFoOMj4t1omuVMNAhwrnAO6PK/XA9mRxcPGoi47C4SmijmNPr9mrNqHJXXKYJuDBAljGMMQBapclSa6FiMH7kjrgeTMBFoY9Wk9F2rRJ6V6lUOs51aUaVO+J6MAGXfDE6JePkzSrhXUSl4ib0G6PKHXE9mICrSJezszVpd5VSwjNVjulvGvFMHVE9eICqBizykKx26yyllENujVO7E6rcEdeDCbga7pTO0Thxs0pSCfJlahujyh1xPZiAq8t2YrQm7q4SxoVFDoPHLWWaUeWOuB5MwDVC4titiburhHdRj+W1cqBmVLkjrgcTcHGI0sk0sF4FHPkKxXLc86MJVe6IyvQ4oJjxLFuT6lXCx63VMtJVcIWuoEwPoeoxDMbTw3aqVcJHrdTMx1p4mlDlfrieTMCVQudc2GixVglvokqZ4saockdcDybgIvlue83hVZrXpkvGM81vjCp3xPVgAi48pRydJ4PrLKExr4T77O6MKnfE9WACrhIKz6XHdq6zhA8dZ/l3ujOq3BHXgwm4ashs3664SmjSU4k5lzujyh1xPZiASwbGjPnKBtdZwmnlxMzHbC7NqHJHXA8m4Oqyw70ZjzpXJbBwK687PK+MZ+6I68EEXEM6nCN345TNKoEFv+24Zn1lPHNHXA8mB5d8mqzbh8/KvNya4pgP/LsgqtwVliUCK5YOp7xRrIbrLB1d9zhuhGfoCOqBQ6SGdM1jLM04XbNKeP8kptcCxFpQ5X64nkzAlXAffrIm7K5SohhynM3UlVHnjrgeTMAlXXMsW2BofVTQkI/W2rECsULUuSMsWwRW0i+PVKvRwK9SoiJAuCZ2RdSxI6wHEmhJt1wGetZk3VVKlAOOgMcaxFpR5Y64HkzAJd1y2Yo1WXeV0D4VonZM1tWMKnfE9WACLumWS5NdjJM2q5Sohi6/6LhZUTOq3BHXgwm4pBsQay5WO3+W0EbJXtfBd0aVO+J6MAGXdMxji9Zk3VVKWf66mFu8M+rcEdeDycHFcS7KaXG9SqlhEf6Gu+6ujDp3xWWagEu657HjmuDOdZZSnc/yjtxujDo/uT7/Fcmuv9P7b96TyMhepfcv3z//4pv//OPX3/zTl79+//r7txvppyNIHYu/1LkQ09vv3nJMsvmJrL9Vxbcf2LZqbmF96/rxv7bVz3+VsD9R9icGTr0d1NsX5UN9vpx4Md+/vHzT21/7ibe3XAlLUsn/XfahxXrZq+9eif4r1Y/qWF6SP8yXZNVvm/+B7+X5gqY3eUHfv5X/Wq+o/DgeUcfSA7+8Aiu+saq3Sgypvv/7ufH55Zev9O3Pb0kkP4tSknYsVGwsNjwSYt62Ib/l609vv/7q/fN/xFnE96/+8Bbl+7/6t7d/ef9F/OX7v75/9Zu3f/jq7bc/F0ShhLlCNfP1Db7iHw9RUjk2Nnrm/gMgWvl7SMjYP2dK7f5RP+OfQAItNjZ2LMLyf5ZINP4OFDW3gFORXC8UKv7xFJWw7oVsrHD+QRJMfw8JLljIsVC+Sqz4J5CQ0UiaG4sZzwn7P1JQbX8HCtVql/nSyXDgo70/g7+5rUwt1PX3Fwop9ZRrHCMdEth/+f24kUsUFssv0i/fv/r2Td6ZPKSnLUMTOqDQnEohychY/tMaPN6lO54i57URHFixlTa/WTz/jF8Tu/RS3uPHP/HsqEi54OdkC53nn/H1J7w6n33xzbe//+e//O73f/r+s09//NNfvn//4j/ef/t3eVlwOa/W2NfrspKf6IXpeFhIbXiyMlf1wqQ+37P/N16Z+sNfmTx46v63e2XmY3DRX8vnS6Oin+i1wQL/vVDtHQux/yyvTfHw2sigs0j7y2W9Niv6iV6bJEe7WGkkjjLq+VlatOygRUt1hMyx1zW0UNFP9drIuDDmhCe3Jfny53ht6L/la/Pf5xyA/J3SqSLofkZdBqECmUvNryGadBkJi4JcSt/pkrSFTV6geZpSbUzH5xkAT2wPNmDD5JX5aKGd7Sxl+SRIj/zFdm5Mxy7ZHmzAhkkspeG6w8Z2lgoH6fwe8zXUxnTsku3BBmy4eI4G0mA7S2jJpVrrVVPHLtkebMCGaS0tFqttO0uVg+waNnfVVLFLtgebg425dVyPMNhepYa7LHE0vmrq2CmbaQM2THbpBdclNraz1LPsO7dUrpo6dsn2YCNsMgbuPJJxRDgrMu5OrdG4Ua7UI9mDC8SSfPMYuGyzkZ0lcZKu7vFY7rUtnbpEe6CBGkm3FW27oXaWepN9x6Yvlip1ifYgAzRMhJHR033g8J0u4aCZcpqLhaiN6dgl24MN2DAhRo6ChtpHpWOJyVpyulrq2CWaLQMzzIpJjYwR1Sr1HrjXksbVUqUu0R5ooIbJMfLGsQ6fZ2m0gIeqvI4F58Z07JLtwQZsmCSTI1nt2lmSf+HqNY+rpo5dsj3YgA2TZeTAaAyoVgkTc3sTkKumjl2yPdgcbBxLScaAapVGCVxSpBvmSp2imTJAwwykmsyBwVkaNbQ20nGqSFmq2CXbg42wDenqyw8m41TRKg0ZblI9VsPTmir2yPZkAzbp7MeGu3l2trMkb6s+0pjXf7Wmil2yPdiATXr7sVM0WrZVGhziXOtNU35kLsEeVACWpc/aORpDg1USnJ5i3Bk/UpdoDzJAK9JlHdZ77FXgJMfLGOfUVbUhHbskM10AVqWzyrIhg+wssfx0KyVfJVXokuxBBmhNuqrMbJwgWiV5SxWsxdhulip2yfZgAzYZDqXY2BhKrRIenCT9ijafqKG2dsldwj3oHHCcUh7GYGqVAFR6plZ20DN3CmfqAG6EkSgO40zRKqEhoxLnQ/PUxnTsku3BBmzoPxC+c2c7S9zk80h0jAyUpopdsj3YCBvLd8sXwxhQrRI+jrGMY30ctbVL7hHuSQdwKdRUUjfat1UCUI6xHgfUK+iZu4R70AEchSJ70I0O7yrJJ7JIl+O4A2htTMcu2R5swJZDTrV2Y2i1Sng8juxcms81Vlu75C7hHnQAhy8anke1w52l2eGQPttcJ/oCqnKXcA86gKshpcbNGGat0nwiLb1Oh2vPFbtke7ABW5NebG/NGGitEj6QMqJ6jRg0p8pdwj3oHHCchuyJCfcqzUeR10rH1T4NqnKncKYO4HrAvQLWAfWjgs/jiNxyuXOq3CWbbQO1Id1Y7tYc51WaD7kaqb2OCyenjl2yPeDAjUOjKL0ww+0sYb3Wwrke03W1p8pdwj3ofHorUX6vtE91P5WkSvNhO7Gn1u6gKncI96gDOOmDURrGRGdVwjuLci5tA1W5S7gHHcBRyCR//n5kUCU89YMxfbLfQVXuEu5BB3A5EGbD7CeTVAnvrIQHdm+gKncJ96ADONydnrHU7Q53lnB/o+wijzuozl3CPegAroZI0kjtgy1VQrejxxJHvYHq3CXcg84Bx7JD2WrjzhLastapEd9BVe4UztQBnPT8qcW8D7dUaT6IM45y5zxTl2gPMkDrQd4uxkm4VcF6nRHnPtKdUuUu2WwbqA3pxvZCVuN2lrD8HZcmH8crp45dsj3gwI0DlgGgfailSliUcqTc54zdi6fKXcI96Ahckh8l+TljqLVKWPwUq6C2dAdVuUe4Jx3AJenFsjVEPSuJElanKMexVHOq3CWbbQM1kiFmpLSfSlIl9G7neljpxqlil2wPOHDLIeXjsV2b21nCASDjyT+bp8pdwj3oAG4uQtKS9Tk9S3hnyeezzun1F1CVu4R70DngOMsbyDiVtEoASq0MTjvomTuFM3UAV8PIORoXGVQJH0nRGXEDVblLuAcdwEnznrOMzg24s4S+GmOOPd9BVe4S7kEHcF32pZRonEpapUQDT7NMOd1BVe4S7kEHcCPUXKMxPVqV0JYNbjWlO6jKXcI96ACOAx4xaEyRViWMDlpqsZQ7qMpdwj3oYP3fiPWli3FHuCrhI1lrza93nAJVuUe4J51j4WQybzhdlSS7HqnNm/0umit2iWbLwIykI9uHMT9alZL8M3PH7fNXTBW7ZHvAgVuWfuyow2jdVgk9tdqkQeO7p8pdwj3oHHAsGxnGYGuV0Iox1xjLHVTlTuFMHcBJrz8zG1OkVQkNWU291X4HVblLuAcdwNXQZbe61cSdpfnOEh6qd1CVu4R70AFcC61Ik2UMtlYpZWnWBkvn4waqc5dwDzqA63j6RDQmSatSIg5yIOgx3UB17hLuQQdwMs6UerPauLOEJaLx7B2+g+rcJdyDDuBYdiuXZpxQWiUASUx99/yIXbI92Aib7DlhERBjqLVK00c216rB+ZF7hHvSAVwKqZTRjBZuldDNLUw1tRuozl3CPegATnayyADdOJ20SnjUX+n1WMrnAqpyl3APOgccF9krY8C1SvPYmTqVdgdVuVM4UwdwOYzSzKvQqySRtGWxzSdUa1Cdu4R70AFckb5sN+48WpUkn83Mrb0+qIpT5S7ZbBuoVenIjlyM0dYqYVSVRnzNIFycOnbJ9oADtyb9WCxJZridpdnFbbHGfvPUuUu4Bx3AdflR7sVq385SEisJj0Gq9lyxS7YHG7CNkGss2RhqrRLaMemtURl3TpW7hHvQARzLH5+SNUV6lXDczC3yXOftAqpyl3APOgJX8DigJO3VDrdKOG00ZKOZ7qAq9wj3pAO4ORWwZmOwtUqCJMOERvNJMRdQlbuEe9A54LhmsmZKr9LRxR3xmC2iQVXuFM7UAZx82KpsxWjjVinJ8YAyJtTcQVXuEu5BB3A59CqfNuOE0irhI1lkSNo2UJW7hHvQAVwJrVbj2QGrgvdVJum39Tunyl2y2TZQq6HWFpPVwJ0lvK16zK8JmIpTxS7ZHnDgJh+02noyxvarhC5H6bkcnV/tqXKXcA86gJPdqr1YU6VXKWU8ijGlyndQlbuEe9ABHP54a6XtVcGgqifZdr9zqtwlm20DNZZu7BjROJm0SuioybbomEOoOFXsku0BR9xqlF4sV2ua9ColGZbWONrreKo8Ve4R7knngJPvJGua9Col6bJJJ230cQPVuVM4UwdwCScgBxunk1YJ76zU0oh8A/3/u7ufHNlxGwzg+3eKPoFG1H8tEwQYILskRxjMIousAuT84Wd1SyyZvAAHM8D0x+p6zz+4bMlmWTJ3CWfoAC6F3viUqEy1TgnXKXtK+XlezQeoyF3CGTqA421pKWlt0qeEPSuWVtYzHiSoyF3CGTqAQ/NpmlM7xu0SYZ4QiXq7QGXuEs7QAVwNhTdjKLOtU6I8w8gNd50/QWXuEs7QAVzDQgFZa5Y+JQD1PgbVN+jOXcIZOoDrIWEJFO0Yt0vPRH50WvNUCSpyl3CGDuAwoKhd+6T+VHAko9nz7BenzF2y6TZQm7yNPO5XpluntM4AbbT04tyxSzYDZ7lhodyuHd92CbODzm+4niwiPUXuFE7VYbiGtgU+RimXk07pARq1DVJAf3KPcJYO4IhHsrztynTrlKjwuQAdcP0GFblLOEMHcIlHsjM15RB3Ss+elXpN6QYVuUs4QwdwmUeyczblktIp4STQB2/oC1TkLuEMHcCVwL/ZtFbpU3o+kom3e9ygIncJZ+gArobcKWut0qdEBVscE6UbVOQu4QwdwLWQeopVuaR0SgBKlVKiN+jOXcIZOoDrgXB8V6Zbp0SlBx7wvj1P7JLNsAHbCLHzD4raTwXHMRqlr3tbElPkLtl0m6U2O1aUUdm+S/CJufcyX5w7dsqm4sBthtHLUHe3XcKAI9VGq41QeorcJZyhw3A9ht6x8OQb7pQI9wHLWGsbfYCK3COcpQM4Cq23pIx8d4Uq/90mDzjKxSlzl2y6DdQSnw/bzMpM65Qen8azhPbm/Ildshk4cMs8hu1Na5M+Jf4jsAgIb+XtKXKXcIYO4AqPYUfWeghPiXBHhgcc67lJElTkLuEMHcBVHsTOmJSLSadETIHH0dLteWKXbIYN2Hjo2mfXmqRPCefNTDwvGBenzF3CGTqA6yGOWNQT6i7hA9nS5MHHBSpzl3CGzoKbgygpl5JOiSrP5WtO66tHElTkTuFUHcANLDA2tD7pUwJQybzd8w26c5dwhg7gZugjVa1R+pSoUZgz59guUJm7hDN0GG7wq0fmn99wp0QV30RttG5BC1CZe4SzdABHPA7LWnPSruADGSufDcbNKXKXbLoN1FLgM2LTGqVPiSpP54knpvPiFLFLNgMHbjnkUXNUjm+nhEnVpHFj/oQuyQwXkPEPo8WoTLNOCftU43HHt5qgFLlLOEMHcDXQaE1rkT4lahl7WF2dvgJU5i7hDB3ANR7DPg1Zb7hdwucR1yfXFREBKnOXcIbOgptjxKlMtE7pGWzw4HY9G0OCitwpnKoDuM5j2NHVUcguYXhbRyvrmUkCVOYu4QwdwA0ew/LkXDvG7RLGaVhW6wUqc5dwhg7g5vPqoVxMOiVqCYv0lHXPXoDK3CWcocNwEzdY4tBapE+JGv6AXFbvpQQVuUc4SwdwFPi/OpRj3CkR+nsjE9EFKnOXcIYO4PDU+5S05vJTwtkz4VEO/QYVuUs4Qwdwmd8nTa1N+pSeGULpbZYbVOQu4QwdwOEbCVldKPWUMENIpacXqMxdwhk6gKuBR2TaLehdIbS+8SRrdV5KTpG7ZNNtlhoWWdB6pE8Jn8cec16PvjycMnbKpuLADYs91a61SJ8SxmpEhSuXp8xdwhk6gOv4BkxpyiWlU8KQgz+aay0V4Slil2yGDdgGv7rjcSpvtl2i1kNplNdTugSnzF3CGTqAmzyOxVeHFLhdwnlz5PrdQihBRe4SztD5z68a+VfnqPV9hBOl58zJv76+PSNBRe4QztQBHJ4HjUfeK3C7hNaGxNv6XByXoDJ3CWfoAC7x+8ypdEmLEpqQJmXKL1CRu4QzdADHo/4Y+VCvwO3Sc+uPP5J93KAidwln6AAOj/3kn96TLVECEPEo5DmWXaA7dwln6ACO5wAxKc+cOhUMOvDIwXlpitglmi6zzGZMPWuHt13CVcoeiWK9MEXslE3FgRuP/yPvOu9LSaJEnM/R1rLFH54idwln6ACOJwCxkNIkLUqYi8bOVC9QkbuEM3QAxxOAWJTbDKdCbTwLkD1fZvvgFLlLNt0Gajz6j7Wm96UkUXpOmoxE85NTxi7ZDBx2I/5V3iqlRVqUnp6QlkZql6fMPcJZOoDjwX9sU2mRFqXnzFnLc2n3w/PELtkMG7Dx0D/2pqydLUpPPy+PPNK4OUXuEs7QAVzGIgGZlGnWKeG8Wfif51syH6Aidwln6ACu8DCWX66cUE8Jn8icqD/P8P0AFblLOENnwU0sIKMd4XbpuZvQc3uDitwpnKoDuBoGRbRlveF2CXMD4mPbrDeoyF3CGTqAa4EJSGmSFiXchymzrxsxElTmLuEMHcD10Ij4jRS4XcJHsvP/pctTxC7ZDBuwjcAnSW1v+y7ApqEZur8pd+4STZUB2Qw81SSlRVqUME7j8S2eVvOBKVKXaIYNsyUeRlAeU9nXTglHsBippHlxytwjnKUDOOIfSp3KZaRTQq/bjKms65YSVOQu4QwdwKHLoyalQVqUANTqfJ61cnn+xC7ZDBuw5RCpzqEc304JX/UY/AY93Zwidwln6Cy4SY1PiircdwkXjPjdG6ULVOZO4VQdwBV8KTIr7dGihFNAw3KB7QKVuUs4Qwdw9Rm3du0It0sYdHCe16VLASpzl3CGDuD46E6DQwVul3DlgzLvZvUGFblLOEMHcJ1HsrMo7dGixJN3nhu0cXuK2CWbYQM2npinSNop9aeCzqPOc9A10ZKYInfJpttAbYacIu8xCtsuwYfHupTri3PHLtkMHHbL/D78Fk2Zap3SMzMYc67rvdJT5B7hLB3AYan1lJTmaFF69qwY1xRVep7YJZthAzbeSHzRSlH7qWADW/oegwhLEbtE02WW2Uy5VeWUcErPA7haaVQ+MWXslE3FgVsOI5WstUWfEvEWE+9fs92eIncJZ+gAjocRqcaqHdt2CdN3PmnG1VYjQUXuEs7QARy+KV/x/Oc33C7hGYNx9LpuOwtQmbuEM3QA1wKP/ovWFn1K2MSZymzzBhW5SzhDB3CdB7GdinaM2yXMpngmn9b1EAEqc5dwhg7gBo9i+YXKhaRTolJx7YjWREuAytwlnKEDuMnD2FG13uhTep5Ljvl7v0FF7hLO0GG4EnlAgdHFG+6UsIRA6anNfoOK3COcpQM44sEsnlipwO0SNrHWVmK/QUXuEs7QWXAz82aqbquCDcw/k61Pzex4smXIwCyFkSlrrdGn9Kz/1As9K2NLTBG7ZDNw4JZDzylq/SGnRHwKxXO21/lUeMrcJZyhAzieoLNBUiZbp4QFUirl2i9PEbtkM2zAVgNPOovWGn1KxB9NPg2kMW5OkbuEM3QAxwMJrmg3GE4Jy2ZN3EqgG1TkLuEMHcDxZuUySJlqnRJh5NHiWqb4A1TkLuEMHcDxXz7XqrVGnxJOAWPU73YuASpzl3CGDuB4spRbitoxbpew+mmNs63LSQJU5i7hDB2GqzHEzK9XplqnRDwtLfiuTL9AZe4RztJZcDP3prVGnxLlHIhSX18MlKAidwqn6gCOeCw7clSOcadEPJ/PfdLqhJOgIncJZ+gALvFgdsypXE46JZwEeixldeNLUJG7hDN0AIdt4cO7MuE6JSwbGwcPetsFKnOXcIYO4HjOVPhttGPcLnEUsEzxumQuQGXuEs7QAVzlXyVtNLIrlCOewlXW140kp8hdsuk2UGu8VdS1HulTwvSgpti/d7fNKWOXbAYO3DpWsCtDO77tEk4Apc1R0+Upc5dwhg7gRuDDOw3lgtIpETod1hoWF6jIXcIZOoCbIZY8tBbpU8KRbHbCQe0CFblLOENnwc1SqtYifUrYs/KkWV6gIncKp+owXOOzYqmpK5eUTglDjt5yftak/AAVuUc4SwdwFHqp/HlT4HYJQFR6Wg9u/ATduUs4QwdwKbTSmtYnfUrPsSyPsu7eS1CRu4QzdACXeSzLAzHlGHdKuAIyYszryogEFblLOEMHcIV/dah3oU8JJwGq1F6eJ3bJZtiArfJmDe2Uuiv4OPaYBvUbU+Qu2XQbqDUeyM5StcPbLlHCRXKal6ZIXaIZNFDr/CmLVJWp1ikRNn6kEcetKXKXcIYO4EaINQ6tUfqUKBUGGv1Zb/cDVOQu4QydBTervsP9VMBDudX17edPzp07ZdNsoDYDC6SiXEo6JcqE/vtR6yenjF2yGTjs1vEAwTS1JulTInxrN46Wbk+Ze4SzdABHoVXeDOX4dkr4QPI+1iLdoCJ3CWfoAC6FWkvOyqWkUyJ835knpOsbNBJU5C7hDB3A5VBqjVqT9CkRzYCH5X3DHVCZu4QzdADHg4nKL1CmWadEiQKNMr/hDqjMXcIZOoCr/JdvWDLxDbdLz56VMh5H+wkqc5dwhg7gGo9kO0+bFLhdwly09fl9Q1CAytwlnKEDuM5j2T60VulTIjyrsee2OvMFqMxdwhk6C25WHpZpx7hdIsL8gMZ6yIMEFblTOFUHcHjS/dRazHeF4rN0bHrWz5KcMnfJpttAbfJQdk6tU/qU4FNbm+PNuWOXbAYOuw1+dYuNlOPbKVHkN+BpAtHtKXKPcJYO4Ij3GT5GKZ/TUyLCcs/0fdVXgMrcJZyhA7iElRWi1pp0Stizcut4uOonqMxdwhk6gMu86alH5RB3ShR5osBK69tuElTkLuEMHcDxD403TLmkdEoUeSerk/+9QUXuEs7QAVwN1HivUaZbpzQLbsX8uO03k7FLNsMGbI03svSpHeF2iSL/aYN4Hv/J+ZG7hDN0FtxstWhPnDol7Fidvi/EnTeTsVM21QZsPYzWSGuSPiUsW9zqaq8RmCd1iWbIAG3wljftIdK7MnmCML+f6CsgT+qSTHeB2OTha+cXKGS7xB/DWHofl6RIXaIZNKw2+c9tI2mt0afUWxiJP4jjE1PGHtksG7ARj1zH1BqjT4l9Shl1Lb37obljl2yGDdgSj1un1jG4Kzy0rSn3OT4tZewSTZeBWeaRQ8xdmVedEp8oWx5tTav2e8nUJZpBA7USqFPU2qFPiX9/tBrnjSnizfbbXxJv+Ff6+vsXsQtvE339/vXb3/7837//+POfv//164///jq0gvU/MqZeA8/Wngsrv/71q8Ua+sdrV3K9zHjf+7fv1553Md73H7/+D/sBxGAKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyNzAzNQplbmRvYmoKMTAgMCBvYmoKWyBdCmVuZG9iagoxNyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDM5NSA+PgpzdHJlYW0KeJw9UktuxUAI2+cUXKDS8JvPeVJV3bz7b2tDUqkqvIkxxjB9ypC55UtdEnGFybderls8pnwuW1qZeYi7i40lPrbcl+4htl10LrE4HUfyCzKdKkSozarRofhCloUHkE7woQvCfTn+4y+AwdewDbjhPTJBsCTmKULGblEZmhJBEWHnkRWopFCfWcLfUe7r9zIFam+MpQtjHPQJtAVCbUjEAupAAETslFStkI5nJBO/Fd1nYhxg59GyAa4ZVESWe+zHiKnOqIy8RMQ+T036KJZMLVbGblMZX/yUjNR8dAUqqTTylPLQVbPQC1iJeRL2OfxI+OfWbCGGOm7W8onlHzPFMhLOYEs5YKGX40fg21l1Ea4dubjOdIEfldZwTLTrfsj1T/5021rNdbxyCKJA5U1B8LsOrkaxxMQyPp2NKXqiLLAamrxGM8FhEBHW98PIAxr9crwQNKdrIrRYIpu1YkSNimxzPb0E1kzvxTnWwxPCbO+d1qGyMzMqIYLauoZq60B2s77zcLafPzPoom0KZW5kc3RyZWFtCmVuZG9iagoxOCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDI0OSA+PgpzdHJlYW0KeJxNUUmKAzAMu+cV+kAhXpO8p0OZQ+f/18oOhTkECa+Sk5aYWAsPMYQfLD34kSFzN/0bfqLZu1l6ksnZ/5jnIlNR+FKoLmJCXYgbz6ER8D2haxJZsb3xOSyjmXO+Bx+FuAQzoQFjfUkyuajmlSETTgx1HA5apMK4a2LD4lrRPI3cbvtGZmUmhA2PZELcGICIIOsCshgslDY2EzJZzgPtDckNWmDXqRtRi4IrlNYJdKJWxKrM4LPm1nY3Qy3y4Kh98fpoVpdghdFL9Vh4X4U+mKmZdu6SQnrhTTsizB4KpDI7LSu1e8TqboH6P8tS8P3J9/gdrw/N/FycCmVuZHN0cmVhbQplbmRvYmoKMTkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA5NCA+PgpzdHJlYW0KeJxFjcERwCAIBP9UQQkKCtpPJpOH9v+NEDJ8YOcO7oQFC7Z5Rh8FlSZeFVgHSmPcUI9AveFyLcncBQ9wJ3/a0FScltN3aZFJVSncpBJ5/w5nJpCoedFjnfcLY/sjPAplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9CQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDM5Ci9TdWJ0eXBlIC9Gb3JtIC9UeXBlIC9YT2JqZWN0ID4+CnN0cmVhbQp4nOMyNDBTMDY1VcjlMjc2ArNywCwjcyMgCySLYEFkM7jSABXzCnwKZW5kc3RyZWFtCmVuZG9iagoyMSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDMyMiA+PgpzdHJlYW0KeJw1UbttxTAM7DUFFzAgfiXN4yBIkbd/mzvaqUjTvB9VXjKlXC51ySpZYfKlQ3WKpnyeZqb8DvWQ45ge2SG6U9aWexgWlol5Sh2xmiz3cAs2vgCaEnML8fcI8CuAUcBEoG7x9w+6WRJAGhT8FOiaq5ZYYgINi4Wt2RXiVt0pWLir+HYkuQcJcjFZ6FMORYopt8B8GSzZkVqc63JZCv9ufQIaYYU47LOLROB5wANMJP5kgGzPPlvs6upFNnaGOOnQgIuAm80kAUFTOKs+uGH7arvm55koJzg51q+iMb4NTuZLUt5XucfPoEHe+DM8Z3eOUA6aUAj03QIgh93ARoQ+tc/ALgO2Sbt3Y0r5nGQpvgQ2CvaoUx3K8GLszFZv2PzH6MpmUWyQlfXR6Q7K3KATYh5vZKFbsrb7Nw+zff8BXxl7ZAplbmRzdHJlYW0KZW5kb2JqCjIyIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggODMgPj4Kc3RyZWFtCnicRYy7DcAwCER7pmAEfib2PlGUwt6/DRAlbrgn3T1cHQmZKW4zw0MGngwshl1xgfSWMAtcR1COneyjYdW+6gSN9aZS8+8PlJ7srOKG6wECQhpmCmVuZHN0cmVhbQplbmRvYmoKMjMgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzMjAgPj4Kc3RyZWFtCnicNVJLbgUxCNvPKbhApfBPzvOqqou++29rE70VTDBg4ykvWdJLvtQl26XD5Fsf9yWxQt6P7ZrMUsX3FrMUzy2vR88Rty0KBFETPViZLxUi1M/06DqocEqfgVcItxQbvINJAINq+AcepTMgUOdAxrtiMlIDgiTYc2lxCIlyJol/pLye3yetpKH0PVmZy9+TS6XQHU1O6AHFysVJoF1J+aCZmEpEkpfrfbFC9IbAkjw+RzHJgOw2iW2iBSbnHqUlzMQUOrDHArxmmtVV6GDCHocpjFcLs6gebPJbE5WkHa3jGdkw3sswU2Kh4bAF1OZiZYLu5eM1r8KI7VGTXcNw7pbNdwjRaP4bFsrgYxWSgEensRINaTjAiMCeXjjFXvMTOQ7AiGOdmiwMY2gmp3qOicDQnrOlYcbHHlr18w9U6XyHCmVuZHN0cmVhbQplbmRvYmoKMjQgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzNDAgPj4Kc3RyZWFtCnicNVI5bgQxDOv9Cn0ggG7b79kgSJH8vw2p2RQDcXRSlDtaVHbLh4VUtex0+bSV2hI35HdlhcQJyasS7VKGSKi8ViHV75kyr7c1ZwTIUqXC5KTkccmCP8OlpwvH+baxr+XIHY8eWBUjoUTAMsXE6BqWzu6wZlt+lmnAj3iEnCvWLcdYBVIb3TjtiveheS2yBoi9mZaKCh1WiRZ+QfGgR4199hhUWCDR7RxJcIyJUJGAdoHaSAw5eyx2UR/0MygxE+jaG0XcQYElkpg5xbp09N/40LGg/tiMN786KulbWllj0j4b7ZTGLDLpelj0dPPWx4MLNO+i/OfVDBI0ZY2Sxget2jmGoplRVni3Q5MNzTHHIfMOnsMZCUr6PBS/jyUTHZTI3w4NoX9fHqOMnDbeAuaiP20VBw7is8NeuYEVShdrkvcBqUzogen/r/G1vtfXHx3tgMYKZW5kc3RyZWFtCmVuZG9iagoyNSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDI1MSA+PgpzdHJlYW0KeJwtUUlyA0EIu88r9IRmp99jlyuH5P/XCMoHBg2LQHRa4qCMnyAsV7zlkatow98zMYLfBYd+K9dtWORAVCBJY1A1oXbxevQe2HGYCcyT1rAMZqwP/Iwp3OjF4TEZZ7fXZdQQ7F2vPZlByaxcxCUTF0zVYSNnDj+ZMi60cz03IOdGWJdhkG5WGjMSjjSFSCGFqpukzgRBEoyuRo02chT7pS+PdIZVjagx7HMtbV/PTThr0OxYrPLklB5dcS4nFy+sHPT1NgMXUWms8kBIwP1uD/VzspPfeEvnzhbT43vNyfLCVGDFm9duQDbV4t+8iOP7jK/n5/n8A19gW4gKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIxNSA+PgpzdHJlYW0KeJw1UTkOAyEM7PcV/kAkjC94T6Iozf6/zYzRVh7BXIa0lCGZ8lKTqCHlUz56mS6cutzXzGo055a0LXOAuLa8L62SwIlmiIPBaZi4AZo8AUPX0ahRQxce0NSlUyiw3AQ+irduD91jtYGXtiHniSBiKBksQc2pRRMWbc8npDW/Xosb3pft3chTpcaWGIEGAVY4HNfo1/CVPU8m0XQVMtSrNcsYCRNFIjz5jqbVE+taNNIyEtTGEaxqA7w7/TBOAAATccsCZJ9KlLPkxG+x9LMGV/r+AZ9HVJYKZW5kc3RyZWFtCmVuZG9iagoxNSAwIG9iago8PCAvQmFzZUZvbnQgL0JNUVFEVitEZWphVnVTYW5zIC9DaGFyUHJvY3MgMTYgMCBSCi9FbmNvZGluZyA8PAovRGlmZmVyZW5jZXMgWyA0OCAvemVybyAvb25lIC90d28gL3RocmVlIC9mb3VyIC9maXZlIC9zaXggNTYgL2VpZ2h0IC9uaW5lIF0KL1R5cGUgL0VuY29kaW5nID4+Ci9GaXJzdENoYXIgMCAvRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Gb250RGVzY3JpcHRvciAxNCAwIFIKL0ZvbnRNYXRyaXggWyAwLjAwMSAwIDAgMC4wMDEgMCAwIF0gL0xhc3RDaGFyIDI1NSAvTmFtZSAvQk1RUURWK0RlamFWdVNhbnMKL1N1YnR5cGUgL1R5cGUzIC9UeXBlIC9Gb250IC9XaWR0aHMgMTMgMCBSID4+CmVuZG9iagoxNCAwIG9iago8PCAvQXNjZW50IDkyOSAvQ2FwSGVpZ2h0IDAgL0Rlc2NlbnQgLTIzNiAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvRm9udE5hbWUgL0JNUVFEVitEZWphVnVTYW5zCi9JdGFsaWNBbmdsZSAwIC9NYXhXaWR0aCAxMzQyIC9TdGVtViAwIC9UeXBlIC9Gb250RGVzY3JpcHRvciAvWEhlaWdodCAwID4+CmVuZG9iagoxMyAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxNiAwIG9iago8PCAvZWlnaHQgMTcgMCBSIC9maXZlIDE4IDAgUiAvZm91ciAxOSAwIFIgL25pbmUgMjEgMCBSIC9vbmUgMjIgMCBSCi9zaXggMjMgMCBSIC90aHJlZSAyNCAwIFIgL3R3byAyNSAwIFIgL3plcm8gMjYgMCBSID4+CmVuZG9iagozIDAgb2JqCjw8IC9GMSAxNSAwIFIgPj4KZW5kb2JqCjQgMCBvYmoKPDwgL0ExIDw8IC9DQSAwIC9UeXBlIC9FeHRHU3RhdGUgL2NhIDEgPj4KL0EyIDw8IC9DQSAxIC9UeXBlIC9FeHRHU3RhdGUgL2NhIDEgPj4KL0EzIDw8IC9DQSAxIC9UeXBlIC9FeHRHU3RhdGUgL2NhIDAuNSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvRjEtRGVqYVZ1U2Fucy1taW51cyAyMCAwIFIgPj4KZW5kb2JqCjIgMCBvYmoKPDwgL0NvdW50IDEgL0tpZHMgWyAxMSAwIFIgXSAvVHlwZSAvUGFnZXMgPj4KZW5kb2JqCjI3IDAgb2JqCjw8IC9DcmVhdGlvbkRhdGUgKEQ6MjAyMjA1MDMwMzE3MDcrMDInMDAnKQovQ3JlYXRvciAoTWF0cGxvdGxpYiB2My41LjEsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcpCi9Qcm9kdWNlciAoTWF0cGxvdGxpYiBwZGYgYmFja2VuZCB2My41LjEpID4+CmVuZG9iagp4cmVmCjAgMjgKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMzI2MTQgMDAwMDAgbiAKMDAwMDAzMjM1MSAwMDAwMCBuIAowMDAwMDMyMzgzIDAwMDAwIG4gCjAwMDAwMzI1MjMgMDAwMDAgbiAKMDAwMDAzMjU0NCAwMDAwMCBuIAowMDAwMDMyNTY1IDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDMzOSAwMDAwMCBuIAowMDAwMDI3NDcxIDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAyNzQ0OSAwMDAwMCBuIAowMDAwMDMxMTYwIDAwMDAwIG4gCjAwMDAwMzA5NTMgMDAwMDAgbiAKMDAwMDAzMDU4NyAwMDAwMCBuIAowMDAwMDMyMjEzIDAwMDAwIG4gCjAwMDAwMjc0OTEgMDAwMDAgbiAKMDAwMDAyNzk1OSAwMDAwMCBuIAowMDAwMDI4MjgxIDAwMDAwIG4gCjAwMDAwMjg0NDcgMDAwMDAgbiAKMDAwMDAyODYxOSAwMDAwMCBuIAowMDAwMDI5MDE0IDAwMDAwIG4gCjAwMDAwMjkxNjkgMDAwMDAgbiAKMDAwMDAyOTU2MiAwMDAwMCBuIAowMDAwMDI5OTc1IDAwMDAwIG4gCjAwMDAwMzAyOTkgMDAwMDAgbiAKMDAwMDAzMjY3NCAwMDAwMCBuIAp0cmFpbGVyCjw8IC9JbmZvIDI3IDAgUiAvUm9vdCAxIDAgUiAvU2l6ZSAyOCA+PgpzdGFydHhyZWYKMzI4MzEKJSVFT0YK\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2022-05-03T03:17:05.689750\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.5.1, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["fig, ax = plt.subplots(2, 2, figsize=(10, 6))\n", "for i in range(4):\n", " ax_sub = ax[i // 2][i % 2]\n", " ax_sub.bar(np.arange(out.shape[1], dtype=np.int32), out[i + 4, :, 0, 14, 14], **plot_args)\n", " ax_sub.set_yscale(\"log\")\n", " ax_sub.set_xticks([0, 64, 128, 192, 256])\n", "plt.show()\n", "plt.close()"]}, {"cell_type": "markdown", "id": "d04e2802", "metadata": {"papermill": {"duration": 0.031268, "end_time": "2022-05-03T01:17:08.606848", "exception": false, "start_time": "2022-05-03T01:17:08.575580", "status": "completed"}, "tags": []}, "source": ["Overall we see a very diverse set of distributions, with a usual peak\n", "for 0 and close to 1. However, the distributions in the first row show a\n", "potentially undesirable behavior. For instance, the value 242 has a\n", "1000x lower likelihood than 243 although they are extremely close and\n", "can often not be distinguished. This shows that the model might have not\n", "generlized well over pixel values. The better solution to this problem\n", "is to use discrete logitics mixtures instead of a softmax distribution.\n", "A discrete logistic distribution can be imagined as discretized, binned\n", "Gaussians. Using a mixture of discrete logistics instead of a softmax\n", "introduces an inductive bias to the model to assign close-by values\n", "similar likelihoods. We can visualize a discrete logistic below:"]}, {"cell_type": "code", "execution_count": 28, "id": "57dbd542", "metadata": {"execution": {"iopub.execute_input": "2022-05-03T01:17:08.669742Z", "iopub.status.busy": "2022-05-03T01:17:08.669076Z", "iopub.status.idle": "2022-05-03T01:17:09.750560Z", "shell.execute_reply": "2022-05-03T01:17:09.749931Z"}, "papermill": {"duration": 1.116065, "end_time": "2022-05-03T01:17:09.753242", "exception": false, "start_time": "2022-05-03T01:17:08.637177", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUiAvTWVkaWFCb3ggWyAwIDAgNDA5LjMyNSAyMjguMzcwNjI1IF0KL1BhcmVudCAyIDAgUiAvUmVzb3VyY2VzIDggMCBSIC9UeXBlIC9QYWdlID4+CmVuZG9iago5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTIgMCBSID4+CnN0cmVhbQp4nNXdTXMbR5IG4Hv/ij7KB5fq++NojWcU4dPYZsQeJuYwlmkvHcI4bHns3X+/mQWALHS+qVVY4iG9oVnhFdnE+4hEV6WAxssv739/eHP/zetX+1++3V4+3Xrzbgv7T9vLL8L+47ud/mf3+0/06w/6/Wu+vXm6ddqyHy7FQr9/+/j7GLtLzVf67Vv6sJub/71tP2zejdBqbr70vh9v5OHDqL71/Vf+oq/FBzze2A4fvW2luk5fJkc3auEveNrSCK7eZG8fs+ir8/kSXj73Jpv39pddHDal7PoeanKevvD9/l/7v/eXX8Qz1Vf06yf6daZ6Mt3ItDXXbu/fObn5qtu329f7L9cDehcK8V+POW++vqTbL/S35PfPPf1RbY7vfnFhUr85ba/u9pd/C3sI+90P86/r7vvtH/uL4P1n+z/3u6+2v95tX88v9ekKhuSdr4e/g8fwE9QMYbh5gA9pGp6zaY/0bX0oesk+Rc8WXG8fVjM+Y81In1j7oedj+AmKxphd+NCm6Tmb9nZbkm5/in6tug8rl5+xXEr02ByOj43X8BPUTPQo38MH/lyW52wqHvBP+AH/T/bswX1ozXpTc/3hHtF5fqAYrq0HiLcH+PtndI9dT9f/9hcP/0PRoNNeG6HSf21/cf92//1fb/9z/3yiH3hW/bM//vPgqbvWy2ATHZTOIs/4jXM5Ep1HRy7Rh6XmU/aRNeletVpKSDGW8P6m8dmb0jfhiLm1ujR9yj6yaae/0Fx876P4+P6m+dmbhkB/gT6nEpeqS/iRXUOgv9aYa821lf+nbH3+snSo3tOIfS37FH5sWf5R9Tlmn71P7y/bn78s/Wj2morPa9mn8GPL0h2rnR7zKz3m5/eWfc615rUs/XT2nHwba9mn8GPL0s9srbQoCL349z8Oh3h7ZuOjfM7HC3VuAek+tZHnEvG9pzf68BGb75fT26/ihPfzd//67uHtw2//+ydx09khju7pmz+URjfyefsYU6EdZL1uIPPcb9JS7Q/94/ebj//m9faBH8l708/pD90YYnNEOWGdT33r6fQ9OTjO3Ku+MisDmrJMaq7EHuJRQMuxmGUZ1HTKRDrKYV/29n05FDMtA5qyTGx0ki3Hnw49x2KWZVDTKRPooTXLxxM1h2KmZUBTlqHTUPVJPp5oORazLIOaTpngUgvy8UTNoZhpGdCUZXx1IYuzsJ5jMcsyqOmU8bSvAusWNYdipmVAU5JJo7ja5LpFyxUxwzKw6ZThT5TrFj2HYqZlQFOW6cXR54vHEzXHYpZlUFOWacONLtctao7FLMugplMm0z5Irlv0HIqZlgFNWaYO2gfJdYuaYzHLMqjplMl0FLlu0XMoZloGNGWZMpwHcxg1x2KWZVDTKZOoCljPqDkUMy0DmrJM7q6AOYyaYzHLMqjplEkugTmMnkMx0zKgKcuk7gKYw6g5FrMsg5pOGboF5jB6DsVMy4CmLBMb7YPAukXLsZhlGdR0ykT6RLBuUXMoZloGNGWZ0FwEcxg1x2KWZVDTKRPcQPMZNYdipmVAU5bx1TU0h9FyLGZZBjWdMsEVNIdRcyhmWgY0JZk4qotgDqPliphhGdh0ygTaB8l1i55DMdMyoCnL9EJV5LpFzbGYZRnUdMrQVwVzGD2HYqZlQFOWacUlMIdRcyxmWQY1nTL8TC25btFzKGZaBjRlmUp3HMxh1ByLWZZBTVmmDFfBHEbNsZhlGdR0ymSXwRxGz6GYaRnQlGXyoH0QWLdoORazLIOaTplE+yCwblFzKGZaBjRlGX4aOZjDqDkWsyyDmk6Z5AqYw+g5FDMtA5qyTOwugjmMmmMxyzKo6ZShuwzmMHoOxUzLgKYsE5rrYA6j5ljMsgxqOmWiK2g+o+ZQzLQMaMoyvtE+CKxbtByLWZZBTadMpH0QWLeoORQzLQOakgx9EO2D5LpFyxUxwzKw6ZQJroI5jJ5DMdMyoCnL9OoymMOoORazLIOaTpngApjD6DkUMy0DmrJMK9dbtwJajsUsy6CmU8a7BuYweg7FTMuApixTC+2D5LpFzbGYZRnUdMrQUcAcRs+hmGkZ0JRlSqF9EFjPaDkWsyyDmrJMHq6DOYyaYzHLMqjplMmugDmMnkMx0zKgKcvwlZbAHEbNsZhlGdR0ymQXwBxGz6GYaRnQlGX4ilpgDqPmWMyyDGo6ZRLtg8C6Rc2hmGkZ0JRlQqd9EFi3aDkWsyyDmk6ZRPsgsG5RcyhmWgY0ZRnf3EBzGC3HYpZlUNMpE11Dcxg1h2KmZUBTkhnNZTCGUWLFy7ALKsos0UUwg9FiqGVZBfQkld6cB/MXJcZYhlVQT1YJtPeRaxUthliWVUBPUuGL5YK5ixJjLMMqqCerBNrzyDWKFkMsyyqgJ6nU6gKYtygxxjKsgnqyincdzFq0GGJZVgE9SaUUV8GcRYkxlmEV1JNV6NPAjEWLIZZlFdCTL2BVXADzFSXGWIZVUM95iSba24DXWuMYYxlWQT3n5QxoZwNeZ63EEMuyCug5X+BGOxvwnBccYyzDKqjnfDK4i+j1SEoMsSyrgJ7z6UHOw+e6wBhjGVZBPec/i7iGJilKDLEsq4CevGd2BcxRcIqpDJuAmnwCcgldGwamEMqwCGh52gJ6ORoMsZJdD1DytDX0ukUYQiO7GqAjfXNE9AJXmVaEVE17oOoE0tBroWVaVSbDIqD7aZtvhiFWozJt0KmZFkHd+Y3Q0DUWZDqg07AtArrzG6eh63GINAXkNFO7Iqg7iXR07RaRJuxk+0yDup+2nNB1fkSaA3LKtr9HUHcS6eiaUCItATkV4yKg+2krGV0/TKS1I6eZ2hVB3UlkOHCtOZGOipyG7fUI6n7aakbXJTymLUfgdE7tiqDuJDLQNSyPaQ8dOJ1TwyKgO23yCrre6TEdOQGnc2pXBHU/bd2ja+Me0hoA0zm064Gak0dBV1E+pDUPgJRtr+BR89M2PLre9iGlgwapdEntiqDuJFLRtdkPaRutAadzalgEdD9twQcHLuR/jOcb3AqpS2rXBLZnlIre9+EQt+hDk1bX2LIKqE8qIaD3CTnErYaaAdYlNqyC6rNKRe8rc4h7klLd9poEFufpc0TvQHSIRz7PFm+PcUkNm6DybNIceL+qNaZGIfaj1FNq2QSUJ5MU0bubrTFfpCjl+Z7PB6prbFgF1WeVht4Nb42p0UijJol1jS2rgPqkQu3AuyeuMf2utVCDwHqMDaug+qzS0bttrnFyKeSYosB6jC2rgPqkUhJ6d9Y1Tq6kMYrEeowNq6D6rNLRu/mucXIjhNSjwHqMLauA+qRSM3r35zXOvIhtvR+wltiwCqrPKsOBdwtf40L1Y0/9gLXEllVAfX4SQUbvLr/GzdG3SJ8D+vUgS2xYBdVnlUFbGrleWeLhemh1jAPWEltWAfVJpWfazMj1ylNc6Idm9DbHJ8tB1tiwCqpPKoOOAaYqTzG/JKW3Xtst1hobVkH1WaW4DKYqTzG/qGtE38ct1hpbVgH1T1v09JFyqrLE/GJR30PKt1hrbFcF1meVQqsxsV5Z4uC7az2NftC6yS27AAByCYH2NmLFssQhVUefXPLRa80NuyAAdqm0uxFrliWe13oLtc8d8uq15pZdAAA//THQ/kasWpY49MFn4hHywWvNDbsgAHahInLOssRh8DXVB20JD15rbtkFAJBLis7LScsSX74v+vw3wtVrzQ27IAB2aS7JWcsSXx9H+oFriS2rgPqkQjeKnLUs8eWsk0s8aK25YRcEwC7NNTltWeLzKiX783OKF681t+wCAMil8D99ydXLU3xZ1M4nMS0HWWPDKqg+q9AZRc5blviyATr/ED0dZI0tq4D6pEJ3OMt5yxKfN8sj11usNTasguqzSuePFI8sT/FlsBLqLdYaW1YB9UmlJTfAq3yW+DyEOy/mloMssWEVVJ9VhgvglT5LzAPb4ecIYT3IEltWAfX55U/ZJTBvWeI53G8XlaeDLLFhFVSfVYarYN6yxOd/CMr9gLXEllVAfVIZdLfBtGWJ5z8a5ovK00GW2LAKqn/akve0uxHrlTU+/wPzfEnLLdZjbFcF1meVQpsbsV5Z4/OTEebTjW+xHmPLKqA+qQT6mnLOssbziSt+Pi39FusxNqyC6rNKcU1OWdaYn+SU/dzw3GI9xpZVQH1SiYF2eWK9ssbnJ8SNLLGusWEVVJ9VqotyxrLG5+dOxiSxrrFlFVCfX7IdXJYTlkM8cgzxgLXEhlVQfVap9JFivXKIe/IZYF1jyyqgPqnkQDsbuV65jWkXOJ+4cjjINTasguqzytzliceV27hFP5+ofsS6xJZVQH1SKdElOV85xiOfv1cOB7nEhlVQfVZprsr5yjHu3jeAdYktq4D6pFKj63K+cozpwQRhXWLDKqg+q3S6w3K9cohrmS/dFljF+Cu6YX1SabzLk+uVQ1xjDAjrHBtWQfVZpdPORq5XjvE4r22PBxnW17aoPl98J9HORq5XjnFrGWGdY8MqqD6rDNrZyPXKMb6oYCzLKqA+qYzsIpivKDE2NKyCerLKcBnMV47x5XvleBDz3yuo/mnLPrsm5ytKrBjaVYE9WWW4IecragyxLKuAnqQSigtyvqLFGMuwCupJKtHTzkZeBU6JMZZhFdSTVQrtbOSV4LQYYllWAT35goGedjbiXKvFGMuwCurJKtV5OV9RY4hlWQX0JJUcXJLzFS3GWIZVUE9Wqa7I+YoaQyzLKqAnqZTAzxAVDyBKjLEMq6CerNKcl/MVNYZYllVAT1KpkXY2cmGixBjLsArqySqNdjZyYaLFEMuyCuhJKi3SzkYuTJQYYxlWQT1ZpbkhBylqDLEsq4CefOlnfnaOXJgoMcYyrIJ6skp3GcxXtBhiWVYBPUllJFfBIEWJMZZhFdSTVbrrYJCixRDLsgroedqK51cpiHOtEitYdlVgT1YZtLMR51o1hliWVUBPUgmZdjbyavtKjLEMq6CerDJck7MBNYZYllVAT1KJxXk5G9BijGVYBfUklUTHkLMBLcZYhlVQT1YpLsvZgBpDLMsqoCe/tYl3Tc4GtBhjGVZBPVml0M5Grle0GGJZVgE9SaUE2tnIhYkSYyzDKqgnq1Ta2ciFiRZDLMsqoCep1OAqeIcgJcZYhlVQT1ahInI2oMYQy7IK6EkqLbogZwNajLEMq6CerNLQ+96qMcSyrAJ6kkqP6N2RtRhjGVZBPVmloXfRVmOIZVkF9OQ3qUvo/da1GGMZVkE9WaXTzkYuTLQYYllWAT1PW/XJZTkbUGIFy64K7Mkq3TU5G1BjiGVZBfQklZDckLMBLcZYhlVQT1YZLsjZgBpDLMsqoCepxOyynA1oMcYyrIJ6ssqgnY0416oxxLKsAnqSCt9tORvQYoxlWAX15DfW9bSzEedaLcZYhlVQT1YpLsnZgBpDLMsqoCepFPqa4F2YlRhjGVZBPVmluA7eiVmLIZZlFdCTVGpwXs4GtBhjGVZBPVmluihnA2oMsSyrgJ6k0gLtbOTCRIkxlmEV1JNVKu1s5MJEiyGWZRXQk1R6oJ2NXJgoMcYyrIJ6skpzAcxXtBhiWVYBPUllRJfBIEWJMZZhFdSTVZqrYJCixRDLsgroedqaj67L2YASK1h2VWBPVukuyNmAGkMsyyqgJ6mERDsbca7VYoxlWAX1ZJVOOxtxrlVjiGVZBfQklZhoZyPOtVqMsQyroJ6sMpyXswE1hliWVUBPUqG7HeVsQIsxlmEV1JNVhityNqDGEMuyCuhJKjm7JmcDWoyxDKugnqwyrjduVZQYYllWAT1JpRTa2ciFiRJjLMMqqCepVPo0ORvQYoxlWAX1ZJVCOxu5MNFiiGVZBfQkleZdl7MBLcZYhlVQT1apLsjZgBpDLMsqoCep9OCSnA1oMcYyrIJ6skp1BcxXtBhiWVYBPUllBNfBIEWJMZZhFdSTVRrtbOTCRIshlmUV0PO0dR9pZyPOtUqsYNlVgT1ZpdHORpxr1RhiWVYBPUklRNfkbECLMZZhFdSTVZobcjagxhDLsgroSSoxuShnA1qMsQyroJ6s0l2WswE1hliWVUBPUknJVTkb0GKMZVgF9WSVTjsbca5VY4hlWQX0JJWcaWcj1ytKjLEMq6CerDJoZyMXJloMsSyrgJ6kUrIrcjagxRjLsArqySqDisiFiRZDLMsqoCep1OK8nA1oMcYyrIJ6kkqjY8jZgBZjLMMqqCerFFfkbECNIZZlFdCTVLqnnY1cmCgxxjKsgnqySqGdjVyYaDHEsqwCepLKCLSzkQsTJcZYjyovv4hcOO5f7fT53CXsr/eXX97//vDm/pvXr/Y37zYhd7pG0Vfn8+WY27dbojtx/N6+ZsePBUeVn75+5HIA/aieingXyv4jFfmJfv1xvvn6km6/bGH3++ee/ijwq4TONjHE62HenLZXd/vLv9Gfx/3uBzqi3+++3/6xv/jy4d2bXz+jL+lox3D5b39x/9v9/vbnHx/e/fbwZv+e/t+vD9/957eHn//92f7P/e6r7a9329f0f/8Hb8PDTAplbmRzdHJlYW0KZW5kb2JqCjEyIDAgb2JqCjQ4ODkKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTcgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNjQgPj4Kc3RyZWFtCnicPZDBEUMhCETvVrElgIBAPclkcvi//2tAk1xkHWD3qTuBkFGHM8Nn4smD07E0cG8VjGsIryP0CE0Ck8DEwZp4DAsBp2GRYy7fVZZVp5Wumo2e171jQdVplzUNbdqB8q2PP8I13qPwGuweQgexKHRuZVoLmVg8a5w7zKPM535O23c9GK2m1Kw3ctnXPTrL1FBeWvuEzmi0/SfXL7sxXh+FFDkICmVuZHN0cmVhbQplbmRvYmoKMTggMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNzAgPj4Kc3RyZWFtCnicPZBLEsMgDEP3nEJHAP+A87TT6YLcf1vLmXSDFGPLL0RXdOyVh8fGlI33aGNPhC1c5XQaTlMZj4u7Zl2gy2Ey02+8mrnAVGGR1eyi+hi8ofOsZoevVTMxhDeZEhpgKndyD/X1pzjt25KQbFdh0J0apLMwzJH8PRBTc9BziJH8I19ya2HQmeYXFy2rGa1lTNHsYapsLQzqjUF3yvXUeq7zMBHv8wPfQT5kCmVuZHN0cmVhbQplbmRvYmoKMTkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzMDcgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDI0NCA+PgpzdHJlYW0KeJxFkU1yBSEIhPeeoi/wquRXPc+kUllM7r8NzbwkK1qF5gPTAhNH8BJD7ImVEx8yfC/oMny3MjvwOtmZcE+4blzDZcMzYVvgOyrLO15Dd7ZSP52hqu8aOd4uUjV0ZWSfeqGaC8yQiK4RWXQrl3VA05TuUuEabFuCFPVKrCedoDToEcrwd5RrfHUTT6+x5FTNIVrNrRMairBseEHUySQRtQ2LJ5ZzIVH5qhurOi5gkyXi9IDcoJVmfHpSSREwg3ysyWjMAjbQk7tnF8aaSx5Fjlc0mLA7STXwgPfitr73NnGP8xf4hXff/ysOfdcCPn8AS/5dBgplbmRzdHJlYW0KZW5kb2JqCjIxIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjMyID4+CnN0cmVhbQp4nDVRSW7EMAy7+xX8wADW7rwnxaCH9v/XUsoUCEAltrglYmMjAi8x+DmI3PiSNaMmfmdyV/wsT4VHwq3gSRSBl+FedoLLG8ZlPw4zH7yXVs6kxpMMyEU2PTwRMtglEDowuwZ12Gbaib4h4bMjUs1GltPXEvTSKgTKU7bf6YISbav6c/usC2372hNOdnvqSeUTiOeWrMBl4xWTxVgGPVG5SzF9kOpsoSehvCifg2w+aohElyhn4InBwSjQDuy57WfiVSFoXd2nbWOoRkrH078NTU2SCPlECWe2NO4W/n/Pvb7X+w9OIVQRCmVuZHN0cmVhbQplbmRvYmoKMjIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMzEgPj4Kc3RyZWFtCnicNU85kgQhDMt5hT4wVRjbQL+np7Y22Pl/upKZTpDwIcnTEx2ZeJkjI7Bmx9taZCBm4FNMxb/2tA8TqvfgHiKUiwthhpFw1qzjbp6OF/92lc9YB+82+IpZXhDYwkzWVxZnLtsFY2mcxDnJboxdE7GNda2nU1hHMKEMhHS2w5Qgc1Sk9MmOMuboOJEnnovv9tssdjl+DusLNo0hFef4KnqCNoOi7HnvAhpyQf9d3fgeRbvoJSAbCRbWUWLunOWEX712dB61KBJzQppBLhMhzekqphCaUKyzo6BSUXCpPqforJ9/5V9cLQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjQ5ID4+CnN0cmVhbQp4nD1QO45EIQzrOYUv8CTyI3AeRqstZu/frgOaKVBMfrYzJNARgUcMMZSv4yWtoK6Bv4tC8W7i64PCIKtDUiDOeg+IdOymNpETOh2cMz9hN2OOwEUxBpzpdKY9ByY5+8IKhHMbZexWSCeJqiKO6jOOKZ4qe594FiztyDZbJ5I95CDhUlKJyaWflMo/bcqUCjpm0QQsErngZBNNOMu7SVKMGZQy6h6mdiJ9rDzIozroZE3OrCOZ2dNP25n4HHC3X9pkTpXHdB7M+Jy0zoM5Fbr344k2B02N2ujs9xNpKi9Sux1anX51EpXdGOcYEpdnfxnfZP/5B/6HWiIKZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDM5NSA+PgpzdHJlYW0KeJw9UktuxUAI2+cUXKDS8JvPeVJV3bz7b2tDUqkqvIkxxjB9ypC55UtdEnGFybderls8pnwuW1qZeYi7i40lPrbcl+4htl10LrE4HUfyCzKdKkSozarRofhCloUHkE7woQvCfTn+4y+AwdewDbjhPTJBsCTmKULGblEZmhJBEWHnkRWopFCfWcLfUe7r9zIFam+MpQtjHPQJtAVCbUjEAupAAETslFStkI5nJBO/Fd1nYhxg59GyAa4ZVESWe+zHiKnOqIy8RMQ+T036KJZMLVbGblMZX/yUjNR8dAUqqTTylPLQVbPQC1iJeRL2OfxI+OfWbCGGOm7W8onlHzPFMhLOYEs5YKGX40fg21l1Ea4dubjOdIEfldZwTLTrfsj1T/5021rNdbxyCKJA5U1B8LsOrkaxxMQyPp2NKXqiLLAamrxGM8FhEBHW98PIAxr9crwQNKdrIrRYIpu1YkSNimxzPb0E1kzvxTnWwxPCbO+d1qGyMzMqIYLauoZq60B2s77zcLafPzPoom0KZW5kc3RyZWFtCmVuZG9iagoyNSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDI0OSA+PgpzdHJlYW0KeJxNUUmKAzAMu+cV+kAhXpO8p0OZQ+f/18oOhTkECa+Sk5aYWAsPMYQfLD34kSFzN/0bfqLZu1l6ksnZ/5jnIlNR+FKoLmJCXYgbz6ER8D2haxJZsb3xOSyjmXO+Bx+FuAQzoQFjfUkyuajmlSETTgx1HA5apMK4a2LD4lrRPI3cbvtGZmUmhA2PZELcGICIIOsCshgslDY2EzJZzgPtDckNWmDXqRtRi4IrlNYJdKJWxKrM4LPm1nY3Qy3y4Kh98fpoVpdghdFL9Vh4X4U+mKmZdu6SQnrhTTsizB4KpDI7LSu1e8TqboH6P8tS8P3J9/gdrw/N/FycCmVuZHN0cmVhbQplbmRvYmoKMjYgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA5NCA+PgpzdHJlYW0KeJxFjcERwCAIBP9UQQkKCtpPJpOH9v+NEDJ8YOcO7oQFC7Z5Rh8FlSZeFVgHSmPcUI9AveFyLcncBQ9wJ3/a0FScltN3aZFJVSncpBJ5/w5nJpCoedFjnfcLY/sjPAplbmRzdHJlYW0KZW5kb2JqCjI3IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzQxID4+CnN0cmVhbQp4nEVSS25EMQjbv1NwgUjhl5DztKq6mN5/W5tM1c3gCWBseMtTpmTKsLklIyTXlE99IkOspvw0ciQipvhJCQV2lY/Ha0usjeyRqBSf2vHjsfRGptkVWvXu0aXNolHNysg5yBChnhW6snvUDtnwelxIuu+UzSEcy/9QgSxl3XIKJUFb0HfsEd8PHa6CK4JhsGsug+1lMtT/+ocWXO9992LHLoAWrOe+wQ4AqKcTtAXIGdruNiloAFW6i0nCo/J6bnaibKNV6fkcADMOMHLAiCVbHb7R3gCWfV3oRY2K/StAUVlA/MjVdsHeMclIcBbmBo69cDzFmXBLOMYCQIq94hh68CXY5i9Xroia8Al1umQvvMKe2ubnQpMId60ADl5kw62ro6iW7ek8gvZnRXJGjNSLODohklrSOYLi0qAeWuNcN7HibSOxuVff7h/hnC9c9usXS+yExAplbmRzdHJlYW0KZW5kb2JqCjI4IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNzIgPj4Kc3RyZWFtCnicMzK3UDBQsDQBEoYWJgrmZgYKKYZcQL6piblCLhdIDMTKAbMMgLQlnIKIZ4CYIG0QxSAWRLGZiRlEHZwBkcvgSgMAJdsWyQplbmRzdHJlYW0KZW5kb2JqCjI5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNDcgPj4Kc3RyZWFtCnicMzK3UDBQsDQBEoYWJgrmZgYKKYZclhBWLhdMLAfMAtGWcAoinsGVBgC5Zw0nCmVuZHN0cmVhbQplbmRvYmoKMzAgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNjMgPj4Kc3RyZWFtCnicRZA7EgMhDEN7TqEj+CMDPs9mMik2929j2GxSwNNYIIO7E4LU2oKJ6IKHtiXdBe+tBGdj/Ok2bjUS5AR1gFak42iUUn25xWmVdPFoNnMrC60THWYOepSjGaAQOhXe7aLkcqbuzvlDcPVf9b9i3TmbiYHJyh0IzepT3Pk2O6K6usn+pMfcrNd+K+xVYWlZS8sJt527ZkAJ3FM52qs9Px8KOvYKZW5kc3RyZWFtCmVuZG9iagozMSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIxOCA+PgpzdHJlYW0KeJw9ULmNBDEMy12FGljAeu2pZxaLS6b/9Ej59iLRFkVSKjWZkikvdZQlWVPeOnyWxA55huVuZDYlKkUvk7Al99AK8X2J5hT33dWWs0M0l2g5fgszKqobHdNLNppwKhO6oNzDM/oNbXQDVocesVsg0KRg17YgcscPGAzBmROLIgxKTQb/rnKPn16LGz7D8UMUkZIO5jX/WP3ycw2vU48nkW5vvuJenKkOAxEckpq8I11YsS4SEWk1QU3PwFotgLu3Xv4btCO6DED2icRxmlKOob9rcKXPL+UnU9gKZW5kc3RyZWFtCmVuZG9iagozMiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDgzID4+CnN0cmVhbQp4nEWMuw3AMAhEe6ZgBH4m9j5RlMLevw0QJW64J909XB0JmSluM8NDBp4MLIZdcYH0ljALXEdQjp3so2HVvuoEjfWmUvPvD5Se7KzihusBAkIaZgplbmRzdHJlYW0KZW5kb2JqCjMzIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNTEgPj4Kc3RyZWFtCnicMza0UDBQMDQwB5JGhkCWkYlCiiEXSADEzOWCCeaAWQZAGqI4B64mhyuDKw0A4bQNmAplbmRzdHJlYW0KZW5kb2JqCjM0IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTYwID4+CnN0cmVhbQp4nEWQORIDMQgEc72CJ0hcgvesy7XB+v+pB9ZHoukCNBy6Fk3KehRoPumxRqG60GvoLEqSRMEWkh1Qp2OIOyhITEhjkki2HoMjmlizXZiZVCqzUuG0acXCv9la1chEjXCN/InpBlT8T+pclPBNg6+SMfoYVLw7g4xJ+F5F3Fox7f5EMLEZ9glvRSYFhImxqdm+z2CGzPcK1zjH8w1MgjfrCmVuZHN0cmVhbQplbmRvYmoKMzUgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzMzQgPj4Kc3RyZWFtCnicLVJLcsUgDNtzCl2gM/gH5DzpdLp4vf+2kpNFRg5g9DHlholKfFkgt6PWxLeNzECF4a+rzIXPSNvIOojLkIu4ki2Fe0Qs5DHEPMSC76vxHh75rMzJswfGL9l3Dyv21IRlIePFGdphFcdhFeRYsHUhqnt4U6TDqSTY44v/PsVzLQQtfEbQgF/kn6+O4PmSFmn3mG3TrnqwTDuqpLAcbE9zXiZfWme5Oh7PB8n2rtgRUrsCFIW5M85z4SjTVka0FnY2SGpcbG+O/VhK0IVuXEaKI5CfqSI8oKTJzCYK4o+cHnIqA2Hqmq50chtVcaeezDWbi7czSWbrvkixmcJ5XTiz/gxTZrV5J89yotSpCO+xZ0vQ0Dmunr2WWWh0mxO8pITPxk5PTr5XM+shORUJqWJaV8FpFJliCdsSX1NRU5p6Gf778u7xO37+ASxzfHMKZW5kc3RyZWFtCmVuZG9iagozNiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDMyMCA+PgpzdHJlYW0KeJw1UktuBTEI288puECl8E/O86qqi777b2sTvRVMMGDjKS9Z0ku+1CXbpcPkWx/3JbFC3o/tmsxSxfcWsxTPLa9HzxG3LQoEURM9WJkvFSLUz/ToOqhwSp+BVwi3FBu8g0kAg2r4Bx6lMyBQ50DGu2IyUgOCJNhzaXEIiXImiX+kvJ7fJ62kofQ9WZnL35NLpdAdTU7oAcXKxUmgXUn5oJmYSkSSl+t9sUL0hsCSPD5HMcmA7DaJbaIFJucepSXMxBQ6sMcCvGaa1VXoYMIehymMVwuzqB5s8lsTlaQdreMZ2TDeyzBTYqHhsAXU5mJlgu7l4zWvwojtUZNdw3Duls13CNFo/hsWyuBjFZKAR6exEg1pOMCIwJ5eOMVe8xM5DsCIY52aLAxjaCaneo6JwNCes6VhxsceWvXzD1TpfIcKZW5kc3RyZWFtCmVuZG9iagozNyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE4ID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozOCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEzMyA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzQwID4+CnN0cmVhbQp4nDVSOW4EMQzr/Qp9IIBu2+/ZIEiR/L8NqdkUA3F0UpQ7WlR2y4eFVLXsdPm0ldoSN+R3ZYXECcmrEu1ShkiovFYh1e+ZMq+3NWcEyFKlwuSk5HHJgj/DpacLx/m2sa/lyB2PHlgVI6FEwDLFxOgals7usGZbfpZpwI94hJwr1i3HWAVSG9047Yr3oXktsgaIvZmWigodVokWfkHxoEeNffYYVFgg0e0cSXCMiVCRgHaB2kgMOXssdlEf9DMoMRPo2htF3EGBJZKYOcW6dPTf+NCxoP7YjDe/OirpW1pZY9I+G+2Uxiwy6XpY9HTz1seDCzTvovzn1QwSNGWNksYHrdo5hqKZUVZ4t0OTDc0xxyHzDp7DGQlK+jwUv48lEx2UyN8ODaF/Xx6jjJw23gLmoj9tFQcO4rPDXrmBFUoXa5L3AalM6IHp/6/xtb7X1x8d7YDGCmVuZHN0cmVhbQplbmRvYmoKNDAgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNTEgPj4Kc3RyZWFtCnicLVFJcgNBCLvPK/SEZqffY5crh+T/1wjKBwYNi0B0WuKgjJ8gLFe85ZGraMPfMzGC3wWHfivXbVjkQFQgSWNQNaF28Xr0HthxmAnMk9awDGasD/yMKdzoxeExGWe312XUEOxdrz2ZQcmsXMQlExdM1WEjZw4/mTIutHM9NyDnRliXYZBuVhozEo40hUghhaqbpM4EQRKMrkaNNnIU+6Uvj3SGVY2oMexzLW1fz004a9DsWKzy5JQeXXEuJxcvrBz09TYDF1FprPJASMD9bg/1c7KT33hL584W0+N7zcnywlRgxZvXbkA21eLfvIjj+4yv5+f5/ANfYFuICmVuZHN0cmVhbQplbmRvYmoKNDEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNzQgPj4Kc3RyZWFtCnicTZBJDkMhDEP3nMIXqIQzwOc8v6q6aO+/rUMHdYH85CBwPDzQcSQudGTojI4rmxzjwLMgY+LROP/JuD7EMUHdoi1Yl3bH2cwSc8IyMQK2RsnZPKLAD8dcCBJklx++wCAiXY/5VvNZk/TPtzvdj7q0Zl89osCJ7AjFsAFXgP26x4FLwvle0+SXKiVjE4fygeoiUjY7oRC1VOxyqoqz3ZsrcBX0/NFD7u0FtSM83wplbmRzdHJlYW0KZW5kb2JqCjQyIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNzUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjQzIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggODkgPj4Kc3RyZWFtCnicNU25EYAwDOs9hUfAj0i8D8dRhP1b7IQ0lk6fEcoHa+QBguGNLyH4oi8ZhLULDyr7SHTYRA1nFSQTw68s8KqcFW1zJRPZWUyjs0HL9K3tb4Meuj/djhwKCmVuZHN0cmVhbQplbmRvYmoKNDQgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNDEgPj4Kc3RyZWFtCnicPY/BDsMwCEPv+Qr/QKTYKaF8T6dqh+7/ryNLuwt6AmOMhdDQG6qaw4Zgm+PF0iVUa/gUxUAlN8iZYA6lpNIdR5F6YjgYXB60G47isej6EbuSZn3QxkK6JWiAe6xTadymcRPEHTUF6inqnKO8ELmfqWfYNJLdNLOSc7gNv3vPU9f/p6u8y/kFvXcu/gplbmRzdHJlYW0KZW5kb2JqCjQ1IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjE1ID4+CnN0cmVhbQp4nDVROQ4DIQzs9xX+QCSML3hPoijN/r/NjNFWHsFchrSUIZnyUpOoIeVTPnqZLpy63NfMajTnlrQtc4C4trwvrZLAiWaIg8FpmLgBmjwBQ9fRqFFDFx7Q1KVTKLDcBD6Kt24P3WO1gZe2IeeJIGIoGSxBzalFExZtzyekNb9eixvel+3dyFOlxpYYgQYBVjgc1+jX8JU9TybRdBUy1Ks1yxgJE0UiPPmOptUT61o00jIS1MYRrGoDvDv9ME4AABNxywJkn0qUs+TEb7H0swZX+v4Bn0dUlgplbmRzdHJlYW0KZW5kb2JqCjE1IDAgb2JqCjw8IC9CYXNlRm9udCAvQk1RUURWK0RlamFWdVNhbnMgL0NoYXJQcm9jcyAxNiAwIFIKL0VuY29kaW5nIDw8Ci9EaWZmZXJlbmNlcyBbIDMyIC9zcGFjZSA0NiAvcGVyaW9kIDQ4IC96ZXJvIC9vbmUgL3R3byAvdGhyZWUgL2ZvdXIgL2ZpdmUgL3NpeCA1NgovZWlnaHQgNjggL0QgODAgL1AgOTcgL2EgL2IgL2MgL2QgL2UgMTAzIC9nIDEwNSAvaSAxMDggL2wgMTEwIC9uIC9vIDExNCAvcgovcyAvdCAvdSAvdiAxMjAgL3ggL3kgXQovVHlwZSAvRW5jb2RpbmcgPj4KL0ZpcnN0Q2hhciAwIC9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnREZXNjcmlwdG9yIDE0IDAgUgovRm9udE1hdHJpeCBbIDAuMDAxIDAgMCAwLjAwMSAwIDAgXSAvTGFzdENoYXIgMjU1IC9OYW1lIC9CTVFRRFYrRGVqYVZ1U2FucwovU3VidHlwZSAvVHlwZTMgL1R5cGUgL0ZvbnQgL1dpZHRocyAxMyAwIFIgPj4KZW5kb2JqCjE0IDAgb2JqCjw8IC9Bc2NlbnQgOTI5IC9DYXBIZWlnaHQgMCAvRGVzY2VudCAtMjM2IC9GbGFncyAzMgovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Gb250TmFtZSAvQk1RUURWK0RlamFWdVNhbnMKL0l0YWxpY0FuZ2xlIDAgL01heFdpZHRoIDEzNDIgL1N0ZW1WIDAgL1R5cGUgL0ZvbnREZXNjcmlwdG9yIC9YSGVpZ2h0IDAgPj4KZW5kb2JqCjEzIDAgb2JqClsgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAKNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCAzMTggNDAxIDQ2MCA4MzggNjM2Cjk1MCA3ODAgMjc1IDM5MCAzOTAgNTAwIDgzOCAzMTggMzYxIDMxOCAzMzcgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNgo2MzYgNjM2IDMzNyAzMzcgODM4IDgzOCA4MzggNTMxIDEwMDAgNjg0IDY4NiA2OTggNzcwIDYzMiA1NzUgNzc1IDc1MiAyOTUKMjk1IDY1NiA1NTcgODYzIDc0OCA3ODcgNjAzIDc4NyA2OTUgNjM1IDYxMSA3MzIgNjg0IDk4OSA2ODUgNjExIDY4NSAzOTAgMzM3CjM5MCA4MzggNTAwIDUwMCA2MTMgNjM1IDU1MCA2MzUgNjE1IDM1MiA2MzUgNjM0IDI3OCAyNzggNTc5IDI3OCA5NzQgNjM0IDYxMgo2MzUgNjM1IDQxMSA1MjEgMzkyIDYzNCA1OTIgODE4IDU5MiA1OTIgNTI1IDYzNiAzMzcgNjM2IDgzOCA2MDAgNjM2IDYwMCAzMTgKMzUyIDUxOCAxMDAwIDUwMCA1MDAgNTAwIDEzNDIgNjM1IDQwMCAxMDcwIDYwMCA2ODUgNjAwIDYwMCAzMTggMzE4IDUxOCA1MTgKNTkwIDUwMCAxMDAwIDUwMCAxMDAwIDUyMSA0MDAgMTAyMyA2MDAgNTI1IDYxMSAzMTggNDAxIDYzNiA2MzYgNjM2IDYzNiAzMzcKNTAwIDUwMCAxMDAwIDQ3MSA2MTIgODM4IDM2MSAxMDAwIDUwMCA1MDAgODM4IDQwMSA0MDEgNTAwIDYzNiA2MzYgMzE4IDUwMAo0MDEgNDcxIDYxMiA5NjkgOTY5IDk2OSA1MzEgNjg0IDY4NCA2ODQgNjg0IDY4NCA2ODQgOTc0IDY5OCA2MzIgNjMyIDYzMiA2MzIKMjk1IDI5NSAyOTUgMjk1IDc3NSA3NDggNzg3IDc4NyA3ODcgNzg3IDc4NyA4MzggNzg3IDczMiA3MzIgNzMyIDczMiA2MTEgNjA1CjYzMCA2MTMgNjEzIDYxMyA2MTMgNjEzIDYxMyA5ODIgNTUwIDYxNSA2MTUgNjE1IDYxNSAyNzggMjc4IDI3OCAyNzggNjEyIDYzNAo2MTIgNjEyIDYxMiA2MTIgNjEyIDgzOCA2MTIgNjM0IDYzNCA2MzQgNjM0IDU5MiA2MzUgNTkyIF0KZW5kb2JqCjE2IDAgb2JqCjw8IC9EIDE3IDAgUiAvUCAxOCAwIFIgL2EgMTkgMCBSIC9iIDIwIDAgUiAvYyAyMSAwIFIgL2QgMjIgMCBSIC9lIDIzIDAgUgovZWlnaHQgMjQgMCBSIC9maXZlIDI1IDAgUiAvZm91ciAyNiAwIFIgL2cgMjcgMCBSIC9pIDI4IDAgUiAvbCAyOSAwIFIKL24gMzAgMCBSIC9vIDMxIDAgUiAvb25lIDMyIDAgUiAvcGVyaW9kIDMzIDAgUiAvciAzNCAwIFIgL3MgMzUgMCBSCi9zaXggMzYgMCBSIC9zcGFjZSAzNyAwIFIgL3QgMzggMCBSIC90aHJlZSAzOSAwIFIgL3R3byA0MCAwIFIgL3UgNDEgMCBSCi92IDQyIDAgUiAveCA0MyAwIFIgL3kgNDQgMCBSIC96ZXJvIDQ1IDAgUiA+PgplbmRvYmoKMyAwIG9iago8PCAvRjEgMTUgMCBSID4+CmVuZG9iago0IDAgb2JqCjw8IC9BMSA8PCAvQ0EgMCAvVHlwZSAvRXh0R1N0YXRlIC9jYSAxID4+Ci9BMiA8PCAvQ0EgMSAvVHlwZSAvRXh0R1N0YXRlIC9jYSAxID4+Ci9BMyA8PCAvQ0EgMSAvVHlwZSAvRXh0R1N0YXRlIC9jYSAwLjUgPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgPj4KZW5kb2JqCjIgMCBvYmoKPDwgL0NvdW50IDEgL0tpZHMgWyAxMSAwIFIgXSAvVHlwZSAvUGFnZXMgPj4KZW5kb2JqCjQ2IDAgb2JqCjw8IC9DcmVhdGlvbkRhdGUgKEQ6MjAyMjA1MDMwMzE3MDkrMDInMDAnKQovQ3JlYXRvciAoTWF0cGxvdGxpYiB2My41LjEsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcpCi9Qcm9kdWNlciAoTWF0cGxvdGxpYiBwZGYgYmFja2VuZCB2My41LjEpID4+CmVuZG9iagp4cmVmCjAgNDcKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMTUzMjUgMDAwMDAgbiAKMDAwMDAxNTA5MCAwMDAwMCBuIAowMDAwMDE1MTIyIDAwMDAwIG4gCjAwMDAwMTUyNjIgMDAwMDAgbiAKMDAwMDAxNTI4MyAwMDAwMCBuIAowMDAwMDE1MzA0IDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM0MSAwMDAwMCBuIAowMDAwMDA1MzI2IDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwNTMwNSAwMDAwMCBuIAowMDAwMDEzNjkzIDAwMDAwIG4gCjAwMDAwMTM0ODYgMDAwMDAgbiAKMDAwMDAxMzAxNSAwMDAwMCBuIAowMDAwMDE0NzQ2IDAwMDAwIG4gCjAwMDAwMDUzNDYgMDAwMDAgbiAKMDAwMDAwNTU4MyAwMDAwMCBuIAowMDAwMDA1ODI2IDAwMDAwIG4gCjAwMDAwMDYyMDYgMDAwMDAgbiAKMDAwMDAwNjUyMyAwMDAwMCBuIAowMDAwMDA2ODI4IDAwMDAwIG4gCjAwMDAwMDcxMzIgMDAwMDAgbiAKMDAwMDAwNzQ1NCAwMDAwMCBuIAowMDAwMDA3OTIyIDAwMDAwIG4gCjAwMDAwMDgyNDQgMDAwMDAgbiAKMDAwMDAwODQxMCAwMDAwMCBuIAowMDAwMDA4ODI0IDAwMDAwIG4gCjAwMDAwMDg5NjggMDAwMDAgbiAKMDAwMDAwOTA4NyAwMDAwMCBuIAowMDAwMDA5MzIzIDAwMDAwIG4gCjAwMDAwMDk2MTQgMDAwMDAgbiAKMDAwMDAwOTc2OSAwMDAwMCBuIAowMDAwMDA5ODkyIDAwMDAwIG4gCjAwMDAwMTAxMjUgMDAwMDAgbiAKMDAwMDAxMDUzMiAwMDAwMCBuIAowMDAwMDEwOTI1IDAwMDAwIG4gCjAwMDAwMTEwMTUgMDAwMDAgbiAKMDAwMDAxMTIyMSAwMDAwMCBuIAowMDAwMDExNjM0IDAwMDAwIG4gCjAwMDAwMTE5NTggMDAwMDAgbiAKMDAwMDAxMjIwNSAwMDAwMCBuIAowMDAwMDEyMzUyIDAwMDAwIG4gCjAwMDAwMTI1MTMgMDAwMDAgbiAKMDAwMDAxMjcyNyAwMDAwMCBuIAowMDAwMDE1Mzg1IDAwMDAwIG4gCnRyYWlsZXIKPDwgL0luZm8gNDYgMCBSIC9Sb290IDEgMCBSIC9TaXplIDQ3ID4+CnN0YXJ0eHJlZgoxNTU0MgolJUVPRgo=\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2022-05-03T03:17:09.154707\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.5.1, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["mu = Tensor([128])\n", "sigma = Tensor([2.0])\n", "\n", "\n", "def discrete_logistic(x, mu, sigma):\n", " return torch.sigmoid((x + 0.5 - mu) / sigma) - torch.sigmoid((x - 0.5 - mu) / sigma)\n", "\n", "\n", "x = torch.arange(256)\n", "p = discrete_logistic(x, mu, sigma)\n", "\n", "# Visualization\n", "plt.figure(figsize=(6, 3))\n", "plt.bar(x.numpy(), p.numpy(), **plot_args)\n", "plt.xlim(96, 160)\n", "plt.title(\"Discrete logistic distribution\")\n", "plt.xlabel(\"Pixel value\")\n", "plt.ylabel(\"Probability\")\n", "plt.show()\n", "plt.close()"]}, {"cell_type": "markdown", "id": "f694c17c", "metadata": {"papermill": {"duration": 0.03899, "end_time": "2022-05-03T01:17:09.830177", "exception": false, "start_time": "2022-05-03T01:17:09.791187", "status": "completed"}, "tags": []}, "source": ["Instead of the softmax, the model would output mean and standard\n", "deviations for the $K$ logistics we use in the mixture. This is one of\n", "the improvements in autoregressive models that PixelCNN++ [3] has\n", "introduced compared to the original PixelCNN."]}, {"cell_type": "markdown", "id": "5a5de595", "metadata": {"papermill": {"duration": 0.033282, "end_time": "2022-05-03T01:17:09.897035", "exception": false, "start_time": "2022-05-03T01:17:09.863753", "status": "completed"}, "tags": []}, "source": ["## Conclusion\n", "\n", "In this tutorial, we have looked at autoregressive image modeling, and\n", "implemented the PixelCNN architecture. With the usage of masked\n", "convolutions, we are able to apply a convolutional network in which a\n", "pixel is only influenced by all its predecessors. Separating the masked\n", "convolution into a horizontal and vertical stack allowed us to remove\n", "the known blind spot on the right upper row of a pixel. In experiments,\n", "autoregressive models outperformed normalizing flows in terms of bits\n", "per dimension, but are much slower to sample from. Improvements, that we\n", "have not implemented ourselves here, are discrete logistic mixtures, a\n", "downsampling architecture, and changing the pixel order in a diagonal\n", "fashion (see PixelSNAIL). Overall, autoregressive models are another,\n", "strong family of generative models, which however are mostly used in\n", "sequence tasks because of their linear scaling in sampling time than\n", "quadratic as on images."]}, {"cell_type": "markdown", "id": "bf795f1f", "metadata": {"papermill": {"duration": 0.038641, "end_time": "2022-05-03T01:17:09.969453", "exception": false, "start_time": "2022-05-03T01:17:09.930812", "status": "completed"}, "tags": []}, "source": ["## References\n", "[1] van den Oord, A., et al.\n", "\"Pixel Recurrent Neural Networks.\"\n", "arXiv preprint arXiv:1601.06759 (2016).\n", "[Link](https://arxiv.org/abs/1601.06759)\n", "\n", "[2] van den Oord, A., et al.\n", "\"Conditional Image Generation with PixelCNN Decoders.\"\n", "In Advances in Neural Information Processing Systems 29, pp.\n", "4790\u20134798 (2016).\n", "[Link](http://papers.nips.cc/paper/6527-conditional-image-generation-with-pixelcnn-decoders.pdf)\n", "\n", "[3] Salimans, Tim, et al.\n", "\"PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications.\"\n", "arXiv preprint arXiv:1701.05517 (2017).\n", "[Link](https://arxiv.org/abs/1701.05517)"]}, {"cell_type": "markdown", "id": "caf39f7a", "metadata": {"papermill": {"duration": 0.037628, "end_time": "2022-05-03T01:17:10.042002", "exception": false, "start_time": "2022-05-03T01:17:10.004374", "status": "completed"}, "tags": []}, "source": ["## Congratulations - Time to Join the Community!\n", "\n", "Congratulations on completing this notebook tutorial! If you enjoyed this and would like to join the Lightning\n", "movement, you can do so in the following ways!\n", "\n", "### Star [Lightning](https://github.com/PyTorchLightning/pytorch-lightning) on GitHub\n", "The easiest way to help our community is just by starring the GitHub repos! This helps raise awareness of the cool\n", "tools we're building.\n", "\n", "### Join our [Slack](https://www.pytorchlightning.ai/community)!\n", "The best way to keep up to date on the latest advancements is to join our community! Make sure to introduce yourself\n", "and share your interests in `#general` channel\n", "\n", "\n", "### Contributions !\n", "The best way to contribute to our community is to become a code contributor! At any time you can go to\n", "[Lightning](https://github.com/PyTorchLightning/pytorch-lightning) or [Bolt](https://github.com/PyTorchLightning/lightning-bolts)\n", "GitHub Issues page and filter for \"good first issue\".\n", "\n", "* [Lightning good first issue](https://github.com/PyTorchLightning/pytorch-lightning/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22)\n", "* [Bolt good first issue](https://github.com/PyTorchLightning/lightning-bolts/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22)\n", "* You can also contribute your own notebooks with useful examples !\n", "\n", "### Great thanks from the entire Pytorch Lightning Team for your interest !\n", "\n", "[![Pytorch Lightning](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAByMAAAE8CAYAAABjFR0gAAAACXBIWXMAAAsSAAALEgHS3X78AAAgAElEQVR4nOzdT2ydV3rn+eP3FQLYoVJsZFEgPU4pjYDCAO5QobxLai6dypKKlFQ2Mroj2inImwnEkrwZIl2kqgOtJJWcrGTUlKikoZpFqkr+s0zGZLuzk1xiV23kDiZyKkVNLYKRIkWFALqvB7/X5zBH9+Wfe8lz3r/fD0BI5qXFe+/73vc95zzneZ7nPvvsMwNsZ3F6Y9YYM2sfvmeMWb2wPnlvmx8HAAAAAAAAAAAANhGMRMHi9Ma4MWbBfn2h8APGvGuMWb6wPnmn8AgAAAAAAAAAAABgEYzEMxanN+YVaDTGfGmId+a6ApYX1icfFB4BAAAAAAAAAABA5xGMRG5xeuOIMeaKMaY34jvyUP/fhfXJ5cIjAAAAAAAAAAAA6DSCkR1nS7IqCHlqn+/Ep8aY+Qvrk6uFRwAAAAAAAAAAANBJBCM7bHF6Y3mHvpB7tWaDkve6/v4CAAAAAAAAAAB0HcHIDlqc3pg1xqwM2Rdyr95W70n6SQIAAAAAAAAAAHQXwcgOWZzeOGSDkKP2hdwr9ZNcuLA+udL19x4AAAAAAAAAAKCLCEZ2gO0LqZKsZyp6tes2KEk/SQAAAAAAAAAAgA4hGNlyi9MbCzYQGbIv5F69a4OS9JMEAAAAAAAAAADoAIKRLWX7Ql4xxkzX7BU+tM/rCv0kAQAAAAAAAAAA2o1gZMvYvpAK9h2v+Sv7VBmb9JMEAAAAAAAAAABoL4KRLWH7Qqok61LDXtGaLd16p/AIAAAAAAAAAAAAGo1gZAssTm/M276QX2rwq7lug5KUbgUAAAAAAAAAAGgJgpENZvtCKgjZa8lLemh7SS4XHgEAAAAAAAAAAEDjEIxsIFuSVX0hT7X0Jaqf5PyF9cnVwiMAAAAAAAAAAABoDIKRDbM4vbFse0N+oQMvd80GJe8VHgEAAAAAAAAAAEDtEYxsiMXpjRM2G7LJfSH36m2Vo6WfJAAAAAAAAAAAQLMQjKy5xemNIzYI2Za+kHv10AYkrzTz6QMAAAAAAAAAAHQPwciasn0hVZL1TNffiwHrKlNLP0kAAAAAAAAAAID6IxhZQ4vTGws2ENmFvpB79a4NStJPEgAAAAAAAAAAoKYIRtbI4vTGrDFmpaN9IffqvMrY0k8SAAAAAAAAAACgfghG1sDi9MYh2xfyeNffiz361PaTXGnkswcAAAAAAAAAAGgpgpEVsn0hVZJ1qbNvQlhrNihJP0kAAAAAAAAAAIAaIBhZkcXpjXmbDUlfyPCu236SlG4FAAAAAAAAAACoEMHIktm+kApCTnfqhZfvoe0ludy1Fw4AAAAAAAAAAFAXBCNLYvtCKjB2qhMvuD4+tVmSN7v+RgAAAAAAAAAAAJSNYGRkXl/IBUqyVmrNBiXvdPg9AAAAAAAAAAAAKBXByIhsX0hlQ36ptS+yed7WMaGfJAAAAAAAAAAAQHwEIyNYnN44YvtC9lr34trhoQ1IXun6GwEAAAAAAAAAABATwciAbEnWK/SFbAz1k5y/sD652vU3AgAAAAAAAAAAIAaCkYEsTm8s0xeysd61/STvdf2NAAAAAAAAAAAACIlg5D4tTm+csNmQ9IVsvvM6lvSTBAAAAAAAAAAACINg5B7RF7K1HtosyZWuvxEAAAAAAAAAAAD7RTByRLYvpEqynmnUE8eo1m1Qkn6SAAAAAAAAAAAAe0QwcgSL0xsLNhBJX8juuK5jTj9JAAAAAAAAAACA0RGMHMLi9MasMWaFvpCd9dCW5KWfJAAAAAAAAAAAwAgIRu5gcXrjkA1C0hcS8qnNkqSfJAAAAAAAAAAAwBAIRm6BvpDYxZrtJ3ln5x8DAAAAAAAAAADoNoKRAxanN+ZtSU76QmI3121QktKtAAAAAAAAAAAAWyAYadm+kApCThceBLb30PaSXN72JwAAAAAAAAAAADqq88FI2xdSQcjjhQeB4X1qsyRv8p4BAAAAAAAAAAB8rrPBSNsXcsEYs1R4ENg79ZOcv7A+eY/3EAAAAAAAAAAAdF0ng5H0hUQJ3jbGLNNPEgAAAAAAAAAAdFmngpG2L6R6+/UKDwLhPbQBySu8twAAAAAAAAAAoIs6EYy0fSEVhDxVeBCI71NbunWV9xoAAAAAAAAAAHRJq4ORXl/IBUqyogbe1blIP0kAAAAAAAAAANAVrQ1G2r6Qyob8UuFBoFrn1bOUfpIAAAAAAAAAAKDtWheMpC8kGuKhzZJc4YABAAAAAAAAAIC2ak0wkr6QaKh1G5SknyQAAAAAAAAAAGidVgQjF6c3lukLiYajnyQAAAAAAAAAAGidRgcjF6c3Tqj3Hn0h0RIP7flMP0kAAAAAAAAAANAKjQxGLk5vHLFBG/pCoo0+Vclh+kkCAAAAAAAAAICma1QwcnF6Y9wGIekLiS5Ys0FJ+kkCAAAAAAAAAIBGakwwkr6Q6LDrtp8kpVsBAAAAAAAAAECj1D4YSV9IIPfQ9pJc5u0AAAAAAAAAAABNUdtgJH0hgS19arMkb271IAAAAAAAAAAAQJ3ULhhJX0hgKGs2KHmHtwsAAAAAAAAAANRVrYKRi9Mb6gm5TF9IYGj0kwQAAAAAAAAAALVVi2Dk4vTGrDFmhb6QwJ6on+TyhfXJK7x9AAAAAAAAAACgTioNRi5ObxyyQUj6QgL7p36S8xfWJ1d5LwEAAAAAAAAAQB1UEoy0fSFVjvVM4UEA+7Vmg5L3eCcBAAAAAAAAAECVSg9G0hcSKM3btnwr/SQBAAAAAAAAAEAlSgtG0hcSqAT9JAEAAAAAAAAAQGWiByPpCwnUAv0kAQAAAAAAAABA6aIFI+kLCdTSu8aYBfpJAgAAAAAAAACAMkQJRtIXEqi988aYK/STBAAAAAAAAAAAMQUNRtIXEmiUhzZLcoXDBgAAAAAAAAAAYggSjKQvJNBo6zYoST9JAAAAAAAAAAAQ1L6CkfSFBFqFfpIAAAAAAAAAACCoPQcjF6c35tVzjr6QQOvQTxIAAAAAAAAAAAQxcjDS9oVUEHK68CCAtvhUWc/0kwQAAAAAAAAAAPsxdDDS9oVUEPJ44UEAbbVmg5L0kwQAAAAAAAAAACPbNRhp+0IuGGOWCg8C6IrrNihJP0kAAAAAAAAAADC0HYOR9IUE4Hlorwf0kwQAAAAAAAAAAEPZMhhJX0gAO6CfJAAAAAAAAAAAGMozwUj6QgIYgfpJLlxYn7zDmwYAAAAAAAAAALayGYxcnN44YYxZoSQrgBG9TpYkAAAAAAAAAADYSh6MtGVZP9zicQAYBgFJAAAAAAAAAABQ8Nz/8es/HTfGqMzilwqPAuVYpz9p4z00xhy6sD75oOtvBAAAAAAAAAAA+DeJMeYEgUhU6PqF9ckjxphPOQiN9gV7LQEAAAAAAAAAANiUEEBAhRSInF+c3jhEQLwVZrv+BgAAAAAAAAAAgGcpGDle+C4QXx6ItL/lCO93Kxzq+hsAAAAAAAAAAACepWBkr/BdIC4/EGnIqAMAAAAAAAAAAGinhOOKkg0GIg3BSAAAAAAAAAAAgHYiGIkybRWIlOnCdwAAAAAAAAAAANB4BCNRli0DkYvTG2RFAgAAAAAAAAAAtBTBSJRhy0CkRTASAAAAAAAAAACgpQhGIradApFypPAdAAAAAAAAAAAAtALBSMS0WyDSkBkJAAAAAAAAAADQXgQjEcuugcjF6Q1lRX6h8AAAAAAAAAAAAABagWAkYhgmI9JQohUAAAAAAAAAAKDdCEYitPUhA5GGEq0AAAAAAAAAAADtRjASIa2PGGAkGAkAAAAAAAAAANBiBCMRSh6IvLA++WCYf29xemPcGPOlwgMAAAAAAAAAAABoDYKRCGGkQKRFViQAAAAAAAAAAEDLEYzEfu0lEClHCt8BAAAAAAAAAABAqxCMxH7sNRBpyIwEAAAAAAAAAABoP4KR2Kv9BCKlV/gOAAAAAAAAAAAAWoVgJPZiX4HIxekNsiIBAAAAAAAAAAA6gGAkRrXfjEhDv0gAAAAAAAAAAIBuUDByjWONIYUIRBr6RQIAAAAAAAAAAHTDAY4zhhQqEGkIRgIAAAAAdnHIfsWyygEY2nKkf3fFGHOv8F10gSpmjUd6nXeMMSHWrgA0R8wxA9cUIBCCkRhGsEDk4vSGBpxfKDwAYC/0eTphA/y9hr6DO2XnuwWiB3bwd4/Fik6Yt19ongX7WQXQzWt3rAUgghXdpfNqKeKrf67wHWwn1nFY5fPdWVcizmFfZbMB0DkxxwxcU4BACEZiN58GzIg09IsEgjrhBlv379/Pv5pkbGzMTE1N7TQBLTyWZdnjJElu24HgKgPCVjq01bFHI8Ta3Q6g/uYjXrsJVnTcBx98YDY2NoK9CceOHTMTExOF72Nnn3zyibl06dKOPzPKMZibmyt8H93zzjvvBHvNr7zyipmZmSl8H0B3hBwzcE0BwiMYiZ08VLAjYCDSUKIVCO/NN980t2/fbtU7e/DgQQUq878fPnw4D1xqIDgxMTE2MTHRswuebtfbu8aYm/aL0hkt0cbzuq1Onz6dfwGArt2hEKyA8/777wcdE9gxZeH72Nnjx4+DHYejR48WvoduChmMFAIHQLeFHjNwTQHCIhiJ7Ty0GZGhy60RjASwq0ePHm0OIN2fbqI6OTmZL2C4r4mJiePGGH1dM8Zct+XcyJgEAKBkIRd/CFYAAAAAQHsQjMRWogQiF6c3VHrvS4UHAGAEKrmhL+14E2VQKntCX2NjY6eMMaf6/f6P0zS9ZAOTAAAAAAAAAICKJLzxGBArI9LQLxIILv9MNa1XZGiuf83s7Kw5f/68+fjjj02api8rUzLLsn+wvTUBAAAAAAAAABUgGAlfzECkoUQrENy4sZmC+JyyJdW7Tj2rFJRMkuQlY8wPbNlWNkQAAAAAAAAAQMkIRsKJHYg0BCMBlEU9q/ygpDGmZ4z5oTFmmYMAAAAAAAAAAOUhGAlTRiBycXpDGVzThQcAICIXlLx8+bJ5/PixftFSv9//EVmSAAAAAABUapy2KgDQHQQjUUZGpGHhH0CVbty4YY4dO2bW1tbyfpJZln1kjJnnoAAAAAAAUCrNxW8aY/4/Y8wCbz0AdAPByG4rKxBpKNEKoGqPHj0y586dy7MkkyQZM8ZcM8ascGAAAAAAAIhKGZArWZY9snPx47zdANAtBCO7q8xApCEYCaAulCX52muvubKtp+yOzHEOEAAAAAAAwahK2pV+v/+PxpgfaP795MmTsQ8++CCfkwMAuuUAx7uTyg5ESq/wHQCoyCeffJKXbb169aqZmpo63u/3P0rT9MvGmAccEwAAAAAA9uSQsiCzLDubJMlL+gfSNDUKQK6uruZfAIBuIhjZPaUHIhenN8iKBFA7Ktv65ptvuoDkywQkAQAAAADYk3lbijUvv5okiVlbW9sMQGr+DQDoNoKR3bNQckaksWUZAKB2BgOSxphVrlkAAAAAAOxKWZDzNgvyoH74/v375rvf/W4egNzY2OAdBABsIhjZLa9fWJ9cqeAVkxkJoLYGApLTaqpvd3UCAAAAAIBnjasXpHpAGpsFqTKs77//vrl9+3bhhwEAMAQjO6WqQKQhGAmg7gYCkppQ3TPGLHPgAAAAAADYNG5bnLzssiAVhKQMKwBgN8kuj6MdKgtELk5vqNzhFwoPAEDNaPK0vLxsHj9+rCe2xEYKAAAAAACesaBApPpBHjt2zNy4cYNAJABgKAQj26/KjEjDYj6AJvnkk0/ygKRkWfaeLT8DAAAAAEDnqT+kNvC6eTMAAMMiGNluVQci5UjhOwBQY2q0r1IztgF/1ddQAAAAAADqYFbzZM2ZyYYEAIyKnpHtVYdApCEzEkATXbp0yRw9elT9I48bY04YY25yIFGmqakpc/DgwUa855OTk4XvAQAAAGidPOHg9u3bHFnU0uHDh4M9Lea5QHgEI9upFoHIxemNQ8aYLxUeAFB76v0wMTGx7dNUkGS3QZ7+/+3+jY8//rjwvbt3727urnSTmyonOQpIXr16VWVo/iJJkl8xxjwo/BCiOXfunOvfWXs6d3W+hPTWW2+ZmZkZTjAAAFALGpfcunWLgwF0W97GZGNjo+vvA2rq7NmzHBqgxghGtk9dMiINWZFAcykYGSgQsjb4jX6//8szMzMvD35/u9+nPo4KSupL5WDKot+ncq0nT55UetqCMYamGCVSZiBy5xv0NtwrfAcAALRBYUwfCJv9AAAhxFwsYp4LBEIwsl3qFIg0BCOBVnh1hxexp8FemqaF71numjFuy78ou/rQ1NRUT4GpkydPmvv375t33nnHvP/++4X/OQb9LgVmn3/++bfSNF1hEFqK5YYFfj8rfCcsguAAAKBqzO0BAHW2GjkgCSCAhDexNeoWiDRMWIBWWN3hKzT3u27aAMy8vY78O2PM7xlj3lXZ16WlpTwYOTsb/xKjsrE3btxQAPUXCQoBAAAAAAAAwOgIRrZD7QKR9IsEENADG6A8YYz5VWPMdQUlL168aJaXl/P+lTGpVKt6F/b7/T9wPTIAAAAAAAAAAMMhGNl852uYEWnIigQQyT2bMflqlmU/mZubM1evXo0akBzIjlwo/AAAAAAAAAAAYFsEI5vt+oX1ybqWDSQYCSCm1SRJft0Ys6Z+krEDkh988EH+Z7/f/1rhQQAAAAAAAADAtghGNpcCkfM1fvYEIwHE9sBea667gGQsGxsbZm1tTdmRL9pysQAAAAAAAACAIRCMbKZaByLpFwmgZPMuQ/LcuXPRfvP777/v/kowEgAAAAAAAACGRDCyeeqeEWnIigRQgRPqIXny5Elz9OjRKL99dXXVPH782GRZ9vuFBwEAAAAAAAAAWyIY2SxNCEQagpEAKvAgSZI/1K9dXo7XSlcBySRJDnKdAwAAAAAAAIDhEIxsjqYEIg2L9AAqsqpr5cTEhDl9+nSUZ3D79m33V65zAAAAAAAAADAEgpHN0KRApKFfJIAK5WmRr732WpRnQDASAAAAAAAAAEZDMLL+mhaIBIAq3dN1c2xszBw7diz409jY2DD379/XX3uFBwEAAAAAAAAABQQj641AJACM7or+j9nZOMmLn3zyifvrkcKDAAAAAAAAAIBnEIysryYHIh8WvgMA5bmTZdlPer2eOXjwYPBfevfuXffX8cKDAAAAAIA2GbdtOga/2JwKAMAIDvBm1VLTMyJvGmNOFb4LACVJkuT7xpgzyo58//33g/5SLzNSE9DVwg8AKItLfx5Mg/b/e/AzurrN94FQZu2ipVugHN9msfKOMebBwN85L+M5Yr8OuWtEv9//5TRNXx74jev2WDywx8V93WvRewEgnnHvejPuXW9+LU3TF7f5re66c89+rQ7cI1AuF3A8kmXZTJIkL+3227Mse5wkye2B+8adwg8CALC73eaS/pyxceMFgpH1824LSrMuE4wEUDFtijgzNTUV/Fk8evSo8D0AUbkBeL44tMuC3qDB/q5L7i/9fv+naZresgP41ZYEgma9YOxWE5f9WNjHwpoLArnnc8h+bWdwUvXAXtfrFhDyz8tXRjgvzRbnZs47L1e9BWmMTufXCfu15Xudpqn5+OOPN/9b/aanpqamvR857v6SZdkju9Fp1Z6LBAnQJaHvJ757DQ/2H/LuA7+z3X3g5z//ub+hcdPAdcddq/KxSr/f/3Gapn/jXXdGeU473WP3o+nHaztagzuRZdlXkiQZcz+TJMnmfeLWrVuF//Xw4cN5JZ6JiYmxiYmJnn+/8e4bN0c8fjGEPCdinFtHShiHx+nhEs7KHj5bux3X7TZltnWzQ8xrX9Pes+XCd3Z3ZIfqX1td+1d3eAzDc+MIt5Fpy3nLgKXCd4xZs8fhjre+sVezEa+Z9whG1su6HQQ12oX1yXuL0xtvKxDQ9QMKoDL5jVcTxNAGMiMHaSDx94Xv7oNdmP47+y/cswGBNkwe9F4tZ1n2+0mShK+na8yvMihutG0DCW5BTyWTtTlAf/c3Cbj/1uKQvyFhcnJSi0X5n/qamZnRguGLNuDgBvTveot+TTx/ZreZnISw3eR0K7P22OnP6S0e381Wk7DxPU6sQzthFyyfuXa5wNbGxkb+5Z+Xg+eoHD16NP/Tnaf6U/esgfPSZVt8jyDYUHSOzPf7/T/yMx7v379vbt++vXnd0H/rGO3EXS90nHRcjh49enBsbOyU3XB5zV4rrpDNio7Q4tiHkV7q+Zpc20eRj1EGrzXuPqDrjK43g/eC3eh64+4Jr7zyiu4H+rf1dcYGtr5jrzu7jU8WIq7FNPF4bUfHcSHLsjfc/fxnP/tZfr9wX7vdKwZ594xn7hvevXylovvGfMTxYQhf2Gbs1yUxEjq23ZTpZfOueoGLpo8xY57nrzZszBf6fdjq8zn4O/xgGBsqd6Zx1bydTz6Tfa95ir78DTC6Hzn+GoebP9qNTW5DjH8tWbPzx1GPx4mI44g1gpH1oUDk7IX1yVYsMFxYn1xYnN4YJ0MSQFXUN3JmZmbXsjqj2mVB4Z5doDyuxYfHjx8XfmBUU1NTL46Njbld1r1+v380TdMvN3yyoLJHH2nn8ZMnT7bcKb4XWjjWlzcQRrNsG0hYXV3dDCYMuzCkz6o/cPf/7ijgYBeM8q+JiYnjNgj0LTs2u9LEANDly5fzxdCtXvOoTp8+nX8NobCoZ2wgbqtF2Z2emwvUGbup5OzZs4WfKdkR+9q+6jImdO1yi5WaLI56HfNfv85vnyaYbjFzdnZ2zFvM/POKFzPr6pBfmUUBAR2PDz74IH9vR11MFhdU9o+TjsuxY8d0TDavFRprJEnyDXtMgFZToG2r7LC90P13bm6uaW/XCRvoyxdl3bXGjVF2uq8Nw/3/+vfeeeedfJHR3gf0pcDWGbs4uGavedvdB/IsVv0boTT0eG3nmXuG7ud6z2/cuLHvOYk7D/RviT12z9zLbcbrpSruG7ov7uWeuBWNz0NQ4D0mfQ5mZmai/o4A8s/s2tra5rkzDLcpcyuDmzIHN79tlc3rBS5WmrzWEPI8txtDCt9vAp0bly5dGvqZ7rRZz507jn9+6T3aKhhmA95/42WHd31D5bjdyHTOrXW47HuNrUYZRwzOHR23IUbHxq5tbH7G7Zzl+/bzvVtgMr8mvfnmm4UH9uPq1av5/00wsh5aFYh0VG52cTq/kBGQBFC6JEn+H2NM8GDkEG66YOTyctiNw/r35ubmXrYDiBOFH2iGQy4QqclhyIUS+/4YG0BCcxyxi3vBAgnDcgEH11vWZSPoPLJl0655AaDlpgS5QwUizbOLTdv1yd1yUc8tzu6ygWNLoZ57APP+hNF/bdtNAkNwAVz/vFQQ7NixY35gkiDYwLmnDUBaxAu5COVzizr60uLya6+9ZuymJ10nvpkkyR9u8xkBWkGLZaHGbVoka1Bwa95+xvN5he6L3/3ud6OPUXT/dPcbLfzquqPNQXZx8UMbONi2hHrIMXbDjtd2XIWFPNvDHUfda2O14fDHC/Y+7jJer/X7/T9N0/RPyryP67XWaIyFf5NXHwk5fh/clGm2CGC4DQ9u04MXuNCmzOtN3fwW+jxvajBS4+JQ78NWQW93Pvn3GjeX1p92E4bb6HutyefUPo3bTa1ntWHXVVHQear3MOT9ZzCgqY1E+mzrM97r9V6y978zdlPM/7nbxoNY9wuCkYiKgCSACilo0NONt+RJ10qWZX+mHcwa4IccXGgBVAO7qamp4zsEBupuOUYg0u3YtmVtq+7LguHM2kWhfLeeFoU0KI8VSBiGCwIpoLFVAMhOohoTlAxhh2NRWNTTZzr0pKoimwvP/oTRBQfLtlMQrIrFzBrY8twr8/i4BWbde7QRxh6PD211hPmdJvYAGkMluf9M9wJlL7ispSqCObqvuvuQ5jbKVpiZmdH46YfGmLftNZHrzvZ0LP9CC8FaoNc9Y5QMtBDc8dP4Uvfxubm5F9nMgir5Gx40xnTzac1/JiYm3Nxnt0xsYJObSzsDVUUqzw6vwLILQrpNrbr/7DC/Dkq/R/c6fblNTd6mmG95Gw+u+Bub+v3+r6k1TixJyw5yU023+UOogKQdIANAmSoLFqini0pV6GYfkiYMXqmNJvZrURbLKbdwHJLbrZ2m6cVqXyKGcMhOaLVw39P5cP78+XxgXObgfDcuAKTnpednMwRP2b6wKyP2UGwblVr+HwoGaVHPHb+Y2QUlme33+z/S4qAWnxUY14KvMlGqCkQO0iRWz+d3f/d388B9mqZuMfMfGpwxP4oT9rUWzr0q6Hql46HzxC6+5KVbO3IsgLZy45QfuHuBrrnnzp2rRVaZnoOuO97Y5Iy9LrqJR2+rLJaOGrdVa3QsDyoTUveMsgORPlc9R+eUAtw24/ZD+zy7PLZExTSm0VzMzX20Gc9uGv3QXhOPcIwwCn8+/dZbb+XnlK04c83OucIumNXHrL0vL+ne4z5XuvZXtdbhNjW5eaSek20rdcpubFp1x0Pzy5jjCIKR9XF8cXqjzQFJlQ95vfAAALRTfj0/efJk8Bfn+ubZicGhwg/Umzan5CWRQtMOY4u+XfU1bnfd/f1gELIugZ6tuIH7YFDSBhwWtvhfWsULLrrrjT7HP3QLtHU/fkMat9eODzVB1kRZ1xRNGOtazkwT2S0WM39gJ5JNuzcMY8sF5bqcezpPdM5oYm/7iv6AkuFAIy27cYruBdpoUOXi4U50/dN1R9dD26v5Q3fdCdG33uctSjZp4XjWbpw6rrGbjqUWxeuyaUrnlALcel52bHncPl8CPqicC1p4m61cJnbXN2Rij9yGSp1TXlCybRsx3HrHh5qbaY7mAn912rDrNh4oeULrG95n/MOnT5/+98L/EBjByHo51fKA5AoBSQAdoRIHa65xdGheMK9R2ZH9fv9rxk5uQtKCtDJRbYkJSlTV0+x2mXRN4hb+NHh/8uSJTrpv2V2drV048hYgD9kFiGvuGGqBtgUlWV2mXZ61rV27mqg9p3AAACAASURBVCg3JavEX8x0E8ksy37UskC5O0a1XFD26dqg52cDAWfIdAEa45BdgFvS5/fy5cv5vaDu/fVc1ZSB606U39MwK24xWBunNHar67F0m1n0PO3Goh+6DZzojNpuInPn58CGzK5U40AELrt/YCPGP7QgS/JIv9//yK13aE6pOVodNzP53PqGCxIfOHDgN83OrVr2jWBk/RCQBIB2yK/lCriEpl1lGuBkWfbVBi1ynlC5B020Qy9oeO8xWSj1dMVfEGp6Jp3OXwUcNGj3dnX+sANZktoteUrXnjqVLd2nK36mnY6prq9NtEVm3rdaFAg74x+jugcH9Px0nXNlW+3CBAFJoL60YepHWoBzmfFVlvHci4HrTu0XP0vwJbfBqAkbp/T89DwV8LFB5WsNbcmBvcmDkXUe32yRif0DsiSxH27uMpDd39Tr3nyWZR9pXUDZkLofN21O6QeJNRYiGNk9BCQBYP/ynVUVZres9Pv9n6qXoZrBh+SazdsF56bsSsx3+IYOYCj7dGZmRlmXP/abbqMWDtmswXx3oCt11oJMupzrFafsCbtw1Kbgz5ZcILIFvajG7fVi89ysa6bdqFygfCAQ1ujMXRccaNIx0vN02apamCAgCdTWvN0wNabrp+5xTQ3k6brjMuy6HIz02wA0bTHYlca048ol2k+gTvxMbJcl2YZxJqqzRXb/UgPn0wqgXtM4QusCyoZs8pzSBSVjtFZyCEbWV1cCkg8LDwJAQKEHAl7Z1XuFBwekafptfUcBydC0YGI+L316rgHnyyFXXi/0jk/XKzJN00uFB1GlPMvA3x1Y92ymvVL2hBega0XwZ5BeW4sCkUdsyeBpvZY2npt6XZrU67Onz6B26jbxnNQ9w5VKbOJ5NxiQtIsrAOrjmdLjbmzddNr4pYBkF7WhAofuGV5A8hQlW1E3LqOt6eNM1Ic7pxpYVWTFlXfXmL9pVRV2EjOgSjCy3roQkJwlIAkgkp7dsRfLrsFIVzbUBcxC0o5nr0Rk3Qf/+SQ69O6qgwcP5oHeLMsesXO4Vp7JMmj67sBhuOCPFsHaOCnXtbRFgciP/B5SbT039br02bOlj8aaeE5qMbnpk3o/IGlLHXOvAuphpYWlxzd1NTOyLa97ICB5jf58qBs3zvTaA9DrFPui63fDqoo8M45o68brGAhG1l/bA5J3CEgCiCBfcI2xcH748GH312GCkQ+MMdfHxsai9I70Fk5q3auu3+9/zUQo0ere0yRJLhceRFXm7aJJq7IMhuH6/bhJuQ3+NL0Rf06vq0WByDEFIvWaukClj/RZbGpAsg1cQNKVNGOxDqjcMwuILbi/oYV0XirYI1mW/aXrKwjUieY9Gmda1xjjYD/cmNlt8K1xQDIfR+g67WV0YkgEI5uBgCQAjCYPAMTYnTQxMeH+Okww0rjsyJMnTxYe2C8F97wSPnXdNTafpumLGlCGzkDy3lMyTeph3pU706C8bVkGw3KTcrtL+MM2TMpbkD24GYjUselKINLRZ5GAZLVcBoH5fFH5z1lUBipDIHIPRmlTgXA0l3Wb3J4+ffpfeWtbK1+7aOr1yI0zrWtt2YyJargNvq4McA0DkpuBSAVOu9yjea8IRjYHAUkAGF4+AL5161bwt8zLjLxTeHBr+rm1qakpfyIfjBfwqWvAI39eoQNTs7OzLjD8LgsjtbAZiGRx79lJuQ08EPypzqEsy/6bFvLUf7CrQXK9br1+G5B8ryF9WFpF10WvnBmbaIDyzROI3DfG3CXTfUOtOQ4cOPCbda+Gg/1p8ua/gbnP+8x9sF+uMo9tS3SlJm/oghtHvPXWW61vRRMLwchmISAJALtTtsFxlUOLscigoGK/3/+pLcE6rPzaHaNUq+vDmGXZ2cKD1dOx6Ok4hM5S9fpw1mVg2mUEIrdANloYk5OT7t8Z5ZrrjPf7/feTJDmo7Oym9x/cL71+vQ/qmWmMudnsV9NMWlT2+kdSygwozwnGKmgqV9Ehy7L/wmYi1NXA3IeNb9iXgb7rqgRWdWkbjSO+5cYRZETuHcHI5iEgCQA7yxf3XJAuJAUi1f8xTdNRUy5XFMCcm5vzF9aD0CBIJSzs4nLdSqLku3dDHwu9hzMzM5qQ/8QYs1r4AZTpiOsRyeJekZ+N9tlnn60xKR+dVxp72Gx034p203apR+RuvP6fvRpM6jvJnYv9fv9PuSYApThke+61pQcyOkbzPS+znns3aktzH2/jG1UgsC+uZKttTbRU4XrXETeOUNsFxhGj89ZB7xGMbCYFJO8sTm+0cvJKQBLAPowrQ1CDlRil+F555RX315EDYGmaXtSfCkiGVtNSrToWb+hYrK6GjRe6XpFJknyj8CDKlJe/1O/TLlgG5Vtz2WjPPffcL9meFyiHFuuO67y8dOkSb7lHZYVqMKnvLJ2TuiaonzIl94D4bIb8mII5ocekQFm0udPeu8+wkQV15m36OM44B/ulc8nLDv+LCq5/qrTzl67lR+iKX13hbTAmGNlg01oMJyAJAM9YUDk+Lf7HqN/u9Xzcy0qGsiP/xSsvGowWVlSW1pavqMt94YSOhZ5byGNx8ODBvNxtlmWPKDNYLVf+UosjXe3DN6ya9ryoPX3WvWvmKGVaNYZcop/H1pRhUfGkvvMUFDGfX0e/1vX3Aojsiu696rnnPndAE2ks45WbJ8DTIv1+/9dsoLk1BkoLH+r6Mcb+aE1Jaw4247bs7PBljSNUjazrLT9COdCOl9FZLiA5e2F9ci99dGpNAUm9Nrvo/4WuH2wAuzriFp9jlGg1Nhipcqtpmu6lXOCDNE3/amxs7JQW2EMHb/TvqUymnZxWXr6n3++fS9M0+MLP7OxsXirXGPOdPfaQQxjLbnGvTllnKqWsgLWjIH1d+jmo54U+p2NjY2dsIJ30jG3oWqvPusuCNsa8O0KZVmVl/0WSJPlCSB2O/+B56VOQuopgqSb1mlT3er2X6nLfiEVlgbzduPn7XXUmt85LZUfOzc29aKsaUMoMCE9rCWc0N1BZtaro+q/7gNvUqP8+fPhw/nddj+7evZv/XdclfdGHqho6Rjouul/oz8H79q1bt/LMRB2vqjJjNMfVfE+VgJIkGfm+rTmot7l2aK7qUMzxiu7V3jwvGP/+X1eqlNC2CjN6PVoHOH36tCstTJ9s7IvOJ10jJiYmypxLb44jqmz54cYRujfpGtn0cQTByOYjIAkAtnSCgl8aJMSYKHmTo78uPDg8jWBOaRIZOhipRU39u8qySNO06kXlWReoCj0QsgFXQ3ZZpTYD/1Ut7mkArs+kG5Srh+hO9Fw1MNdCkv6sokybrkt6v65evarP6X+1WZKtDqhrYUmLXu44ub67I3p7xEDZsnbN6ppYxXHWa1RJb/3pvnaj81MLm3q+obPJd6L7pQ2QL9lg2L0dfrwR3GKmO+92WoT0rwt6/8teXNbualu6fYFgJBCevzGl7E0fuv64wM9u94Fer/fMf7sWB7omUXkiLh0fHadhgmCDY03Nc3R8yrxv6/fYjSyKlJ4YtUrMftqFaA4Wqz+8jsHS0lLh+x3R2uoUCh7p2E5MTJyy4xw2YpajleeUrn/aBH3x4kXz9OnTPz1w4MBvFX4osKaPI9xX3RCMbIeuBCRX7GsFgEErWtiPufisSaq1n9KgWuhdm5iY6GlQEXLhcyDLYuTJaWD5zsfQCyh6z+zC8lobFs2bKnbgfyduwWhwwG0zlv/OBvf8DLr8g/vCCy8cnZmZGfMXkpQV5haRyqLPvHa1nzx58kUbYGtdmS1XSlkLXttMnNYK3ylyx/HmCBmRxt+9WmbGrgJgyuK0u3UHH17b4rx0tFhwROdnr9cb03mtxThdy3Vuxg6O6fOrhaKzZ88au8HjROGHGkLnnI7BFuecO9/8D7o2VIy/8MILMzMzMwfddUHnjQKE+oyWcW1zu5enpqambQkz7mtAOPnGFN3ry7zP61qkgM3AvWDN3gPueJ9z96SO2HvBuP277glfmZubG9N9VJuYdD/QdYmMyXA0p1DFCnf9z7JMNTL/xh4Xd5z8a7I7PrpWz2ZZ9tszMzMv6f/XMSrz3qHz2QYVR5nv7TcQNKuNSy64FPJ1agylsY+OQZIk/3mb8VKbHWnzi9M5YwPNy/QpL01rzyld/7QRZGZm5jdLqCxSp3HE+sD9yXjX9EP2a8txhMtsr9M4gmBke3QpQ5KAJACfBiDHtaAXa/FZi+u6kduAx36DfFrw7akPWuiFZg0y7OR0vsJgpAZCp1QeM3Qw0usdR1ZkdTZ7JpQ1KNfnT0EGHX+3az3Lsp8kSfJ9V6JFwdGdaEejHaBrLHGi3+//Tq/Xe1HBH52rmiiXlX1QUYmZ6LSYpInTwK77de813omdCdrv9/+8zED54GKm9a59vavDLqjZ81OTx/l+v/8H2lSi91FBSd3XYr4WLaDq8zUxMXHcfj4adT5uMWFf9xZdd3z/7ft+yL5uLbTmlQt0rdH7UkZ/OR1jGww+wb0NCOZQv99/6+c//3lpG1O2uBat2WvRzV3uff51Kh+722uTrkvzL7zwwldPnjw5puu0FhJ1XaIP8t5pTKkxgrepLT9OSZLstqD9wLs/rvj37eeff/5rp0+f/kWdA/q3Y28k0vhbi8s6N5IkGbb05WBwdeRfq3G0Wg1oDBtyzOxK8idJ8scNqRLwWeE7AWgsqUoNo9AcRl+DXNnGOpRsdK1ktBmbjVdb0rX+w60e2K+9nFMK9G1F55POqyoqiQxSlSF9ZVn2zSGu3Xulth/nnjx50oRxhH99b8Q4gmBku7Q9IPmAgCQAz7i92fY0yNaCcKybqgYG5vN+Dn9VeHB0NxVIUZ8uLd6HnBxoYKgJiV1UrmqwHyUrUu+VFg5sEKrKrM8uq2RQrh3nXums61qwT5JkxyDDNh7Ya8ZNG7zMB+gqHaQduxr8l7GIVEWJmdgUvNH7p+OUZdmjJEkuV1D2c96Vh44dKNdips5LF3i116VvDDFZ3InO6YU0TZUtO68J9tzc3Eta9NN5GfM1NXHXuo6BPkdeIPi6ff6jnnP37Lm6YjOVF1544YVz6nHk3vuYfZx0XG0wcp5gJBCMNk79ojYVxF6E1/hU14kA16JB+aaWJEny65J6BJ48efKgC3jVsexa3SlzXvcNLfTa+/Yf7nMDjrtv63hf0XhSC+S6p8bezGKzI8dK3kSkMc6ZLSpA7Ivre1ZxVZ+RuDLvIWjsbKs6ZMaY/2mM+X+H/We/+MUvHp2YmNiyvrBfRUbP1VVnqSIwOTDOpHfkFrYLLO/FwDn1o2HnJlmW/XtlfBce2KZMta5DsXvJbkXnss2OfCniNVDrDWOMI+IgGNk+LiB54sL6ZOt2nBCQBGDN2p5rL8YORBpvx2aohUK7UP8t/buhgzqaZHgLm6X3jrQ9K/PnEZJb9LfvHapR9aB8IXBmnctec4tIx7WIpKzP2Jl1JZeYicbPMOj3+/9ijLmYJMmVKnph2t2xeZA3JmVD6rptA695EDLCrtwVu+liQWXR9JouX76cZ+vFMLBrvfbZkf6Ccr/f/3Gapv8pUFm3B7Ykk87hlampqeNaQIvVI8vYEuteqdbxtveRBUqQV+hQsCD0WHSQ27Cg+8HTp0//9sCBA/97hBKT/nVp855QRuZ8m+i+oeu53dx23S7OhrrePnBVabIs+0ttZnHj2Fi0GG/nRo2raLCDxtz/dN/W2CAEjSs1/zDGfDTqhjCb/bQVV1JYf57QGEOfAa09aP6h31dmdpvGmdrEZ7N5Q8/nWkHvUahNDN459T9HKdm6w/m0bZlqrTvpfqTnXmag22VH2vWB0NfA0sYR/sbrksYRy8pwd+OImPepnWx7pqHRNJlVWdNW1om2WZ+ztgwTgG45ZBftP1QgUoMDZeTEnIjbpuvGBkNCbfJYUV8M/dtazA/JZSQqKBj0Hx6OMpNe1MAm9DHRcbbBjkYGbVqgtEG5JlAKumiCo0CDMeY37CJPrInrPVsi8VWbtZz//i16zwVlJ1B5EC3qL4pE1y69BhuIVEDoZRvYrWKBYV49PXTtiZnFpmu2XrNdzHw7SZJfj3hNemDfT52XjzXZjzlh9BZAat3H1C0ou/tymqb/IdKkXdeEr+tY6/fFvB54C4L0UgL2L79Q6j4ec36gcakW8+z94LytchCz1527J/yGxioKROl+FHoe0UZ6j7xA5PmIY8qbSZJ8+bPPPvtnHZ9QwaqtcN9opZABFdd3fdkGkf6dMeZ1XTs0v3IZvAqal0VrFNrU2uT+5A30fwV6yq5M9Yqdc/2KMeZXdT1VRRxd79577708qFbWPclVBFO7JhssDSmfC8UeR+geoYxhe2/6eknjiAV/HKHXWMU4gmBke33BZkgSkATQBrN28PP3rh+hsiHLKBXpTSRDLjg/SJLkexp4aFd1SBowaUFeQcEKyqDkvy909o4CAHqvbJlcdlJWo5TFvYFgz/lIgYbtrNrg0nUFOsoIQGh3soJoTStZ5AKR9v1RQOjLVfaAcQHdmGXRdG5qwqjAoDHm9yJk6m5H5+WX9Xs1aXRlw0PTIpE3qT9Uwusa2TYLyjFpB/HrLiAZa7LuLSq3ct4GlEjl5L8ae+OUNoZog4i9H7xaciWSO3assqZ7MAHJ3blqBl7pu5juPPfccz2dG5pDhp7nOcpA0nmeZdnRwoNA0QNbdUNBpFe1iVBBSc3rYo0rB3ktXAhGtsM9m233SzbQ/UiZt2Vs6HW8cyrkfEDjiDfKGEfoHuGNI8ps1VD5OIJgZLsRkATQVEdc6cJ+v/+Ptql3HoQ8f/58Pmguo7SIlxW5FqH8Qz4ZjrFrNtLAbDdHXP/O0JlJocvkYmTjZWRFDgR7Xq+izLBXaivPiNKEKtZCkvGyI5sWjNQEygUiI2etDuOEAroqrxurPJB/biowWEFfozsuIKnnEeuc9D7ftcyOLHlB2dFGpLf1e2NtgPLumWS4APujjI2xmD2slBGpjSHe/aCKEpluHeQ6Acmd6f7tVdooa6yle3Ye4dF4Kdax0b3DZpqFzgpCu63azZ75XEfjyjJKNep8tQH0rxQeRNO5QPfbWjsrK8itDfhWyPGz5pUHY44jtP7X9XEEwcj2cwHJVu4+UUDywvqkFsDfLjwIIITVIb+W9/B1c/Df1o4qY8xnxpgfGmOuKQCiDD8tMr/11lv5oMYLtEWlm7FKTVgxJq/aTbamAZvKUobkla3olZjhki9ehw5W6b2xQY+1EjPk8KzNjNdYg/Itgj1Vl+O9YgOifuAtOH1WbSCi15SsKC3EutKsNQlabZbSiUHXoIFzs6rrkBY31Rsx2uKm7q92oeiNwoMVq2hB2VlwWQQxAsEuiN7v93+58CCAofX7/T8yEcaijj7/LiOy4vuBM+8vJOJZ/lzO9hYu06rbyBKrXOutW7fcX8mqx15orvMbrqxwGQFJ9cy3AXQ2X7WPKwH6uts8GTsgqfGzV9UliH6/n980Yo0j9J64jMi6jSPKqDrnEIzsBgUkf7A4vdGoXfejuLA+uWBTmwGE1Rvya2kPX8cH/+2f/exnYypbqOCjSqKpFOsrr7ySTyQ1eC2TBgmuL1jE8oP5qF+L+6GVnOGymTkXOljsDWLpFVmRLMvOmmd3Hwalwa8mLObzxvl1GJQ7K36Jxlh9VeqejebTwp6ujerfmqbpsRqUTdZmi54mojGy5fV63cQsSZI/rsG5edMtbsZYNNJmA7tQdLBu2bpuMbeCBWX3e3X8/U1KQWnsY/uuAtibI/oM6bMUI0teYwB33bVZb3UZq8xrs4TGUrGuT02lyipeNn0Vx2vZlS6M3JuvlqXV0QgqK/wlXUPKCEhSlr4TVvxqLrEDkt4aYYgA9yGNI7RRONY4wt2n6ziO0KbLmL2OfQcK30GbXVuc3lDgrnULuovTG+OU7wPCGeUmpMXavWQNafE4Vkm9/VImjCaOmkAmSRJzVK5s0J/0er2XNDgJ+X4oKKjj+MILL7yRJEnsIEeUzCSdW5oY9fv9n6ZpSjCyGrMxS2C6HnDW6zXMftV5d2RsbOzMxYsXo2wc0GdVE5MXXnjhq0mS1HrjmJ6nXdi7WGWPSE9e+SPW7lXv9b5dow0RytL7Sq/Xe1nnjj4/ITeB6Dqu6659b2vxmr2S6VUtKBub5fLuxMTEcWVHlb1BCsCu8vtnrAoqWqT3+tXW6gKgvs1Zlv3DyZMnD+raVEYriybwFsGrKPsvD5IkuawNu5pXhs468Y4zwUjsxwNdQz777LNP5+bmfknnVazr6N27d91fZ1m/bTXXXuKjpaWlMR330G18nIFWB/u9N+fzylgbsL1xxNfrOo44ffr0QV0DYo8jCEZ2T+sCkjYQqQ/ydOFBANEpk6JNk14FR7ydz38YO/MnSZJv6NocepLqMlzm5uZchku0636/3/9amqbBB26uV2Sapt8uPIiy5It7sRbevUF5nYI9gxRsPzI1NdVTgN8LngZjP6tjdhJUdj/CobjNAXaTRi0WEFSST9eeGOenNqXY1/uTyJtSRmazA29OTEx8SbuOXWZxYMdt1nvV2a9+3+Cqj4PO++Na4A59zmmhRjuSAy2mAJ2TZdnvJ0kS5X7gl4lO07RW9wPrgZ2z/EDjqjJ6ddWdNo14m1iq3Dyl+8aSjkmZJfCAET147rnnVKnqh9qIp7WdGJtQvcARAfT2u2OrylzTtU8bemO0e/EC3PvOto05r3TjCNt6qI6B+FLHEZRp7aZrbSnZSiASQGi6+XqT1zKCAje1uK8bfugeYF6mYsxr/rz6eioQGXrSQonW6vX7/d8xkTINtFBUs96DO5lXuRlNpGKU2vImPbXt8e0+j3aXf+UBqtildFQm3PzbhpE6vF7fHTvp1s7ad+3ENsZX5T199Hmz1RfWa5CN66oZBL9fx+rHC3TEIVVxUInWGJ+lqstED+mmzd6OUsWhaWo0h9D44bo23oXuP+6d65S8RAgaW349VisAxwYkWb/thhV3X/I2FgblBbjH9/nvRptXujYnVp3XPDbHEQQjEcsVG8hrLAKRAELT5L2C4Ih2IX1HA//QN30NpuwArRdxB2KUslgDZfnqUA6yi9R/6UWVaI3B9Uxwvdhq7l6SJJf0OY3RS0HBSPVcVWZH4cGasKU7TY02B0QrpaOsSJcFU+PNEA/sztoTNmgY46vyLF1tWrBqcRySJPm+secIgNrILxSxshlqUCZ6WPncpayeT3Vm53M/rUmmef4cvPtZEAEX4QHniusdF2uco/kOOkUbeh/pvhSrd6633rUf0cYRA5n6nR9HGIKRnfaFyJkyURGIBBCaFhvOnj2rYMBju/O5zEyYvFRDjB1jXi+1GMFVBTh7GgCGLtVLVmQt5MGeyIt77zaoJOGysqIUlIsxmdJnKEmSg3Xc4V6z7DQnnzTeunWr8MB+uetPmqbUVKuYl0lSl+tE/jxCZ7gM9LwBMJr8c+OVawvGW5CrY3nWQfdcFl6XS7W6IEqapn9deLAa+X3j8OHDNXk6wPbcJlFXISS0gb6RaD/XOzdagCtQgDs/H2O0n2riOCJ2diTByG6rbSmwnRCIBBCaFhVdllaSJMcq2LF0z5VECL1r1su4eiPCztl8QOUFPIPQ8fCykuidVZ0yBuV1L8/6DFuyM1p2pFW7ybm3O7o2/SyzLPuKrm1eECcI1xvTZlSwGaJiXuC/LjuJ8+fxyiuvFB7YD8q0AnuXZdlvmwjjFY1H7captQZV6cjH5rFK4jVBDccs+bljzyWg7vKS9JqLx9h8yXink64oO1LrXKHbHAzYcyWwfr+f3zhCjyN0P/I2YDOOsAhGdlvjassTiAQQmhYa3nnnHaNdxMaY1ysMfuXZkaH7vGjAryCHzbgKuQllPMuyryoYELpEq3sPyEqqVpZlR+/fvx+8b4I3KG/S4p6zEmsyVeedwt4CWl0CQkeSJBmLESh3G0LSNP2rwoMonbcxpS5YVAZqxvWLDK2hVTp0jVrX/CZWSby68zaL1KYcnu5joTPqgVhcJluXNzUgqM22RKE33ptnq+TsORjp+kWGxjhiawQj0RgEIgGEpsHQQCCyykFCtF2IN27cyP/s9/vnCg/u3byCAaEDkQrw6Lgo4ENWUqUO6fhGHpRfKTzYALEmU3qvFdzv9/thU64CqOHCXr6hLkZJvrr1KES+QPBPNXsb1iIGI+n9BYwmWolW735Qm6oAQ8rvXzEWfZvAzutMnTa81fA+Buwkv4bQHxsB1fm+lD+pGOsedawuNKSox4tgZLfVvXHqoBUCkQBCUUDk4sWLdQlE5mKVgNTASl/a8RUqKz7LsrMmQolWHRcdEwV8Cg+iTFGDPerN2sBBuRNtcG4/py/WLSBRw4W9fOdrjMzIXq/nSrQ2bZyMBvMWQBpXuQaoWH4/UCWHkLQx0KviUGYf+RDy8VVXg5E2A3Gt8ACAYT1wmVERy2qy+apb7mizeU0D3Pm5GLoalFfq/d3Cg/UXdRxBMLLbGlMabXF6Qwt/xwsPAMAeqD/k0tKSC4jUIhBp3YxVAtILGobo0XdCJbHW1taCD9q8cjCNzJprkXxBPPQOQQ3KbbD5bwoPNke0yZQX/K1VQMIu7K0XHqhOlB2s7pimafrXhQeBiOihBOxZHowMvXnKu8c3sXf5PW2qUbUVANij/NoXer7jbSRk81XHJEnyf2sdoIYlq/NzMfQm18OHD7u/NnGDa9RxBMHIbmvEwNoGIk8VHgCAEWmXs0qWKuClYEKSJF+uWSk+1dO/rEGaV8oyCJVTVQlI9XkMsBNx3njlX0NRELaBDb7bKl/cC71A7pX7bGpWZC7WZMp7v/fc8yKi2mSG9Pv9XzYRzk9v0tjExefW6Wq/MwAjiZJd45VibuT9IE3TvIkW11Gg3rys7roF5/JrH71OEVAelPPmW63GOGJ7BCO7rfYfCAKRAEJ57bXX8uCZK92TJMmv1HSXUh4cjdEw8VfO0wAAIABJREFUXgFJ9QFUZmPhweEpSHJcE6fQu8ea3kuwZaKUwfQmtE0vgZmPoUJPprz3u47ByNpQyekYfT1adH62QtMn8QBKETujoamb4/L7WMT+tgAC8KoM1a1sab4JkQ0NCCgfz9fwvhSl4o63CZtxxACCkd316YX1yVp/IAhEAghBi8vvvPOOOXv2rOt7dt4OOOra/0XX5uu66Yeu0e5Ktfb7/XOFB4eXl3nVexqSJjrq1ZZl2U9YeG4vb0Lb9GAPi3wVU6Z3aC06PwEA++C1S2jqImL+vLuSgQIguHw+TjASodX1nIrYMoFxxACCkd1V64VeApEA9kuDnOXl5Twb0tY6XzPG/IYxZrkBb26eHalszpC08/Ljjz/Os4r2WApmPMuyNxQEWF0NextxmaBJknyj8CBKl2XZUa9sUDD6LKr/QAuOaD4493Y8BuG953G6xbdDtKxRe37+uPAAUBJXghgA9ikfp9iNmJ3hLXLXddMpAHQVAe5miTaOIBjZXbXd8U0gEsB+KBNSQcj33nvPzM3NucDH63ZxvynZLqtaENfCeOg+DSrVai0UHtzdiSRJDurfCLlzTLvPVaI1y7LHTe8l2BYq5xsjGGk+L7H5d4VvNk+UHY5eqSZsLw9G3rp1a9sf2I80Tf+J9x5V8DYLAaiYxt/qL89xaBavYgUVDgAAlWnRJuzgCEZ2Vy0zIwlEAtgLF8xS6VBlQvpByDRN/xeXadgkaZpeMhGyIxVIVGZjlmVfHbU3RZZl3zReuddQVI5WO66SJPkeO5mBevFK1QEA0BlPnz79R442AAAYldbc+v3+P/PGFRGM7KaHF9Yna7dTjEAkgFG4AOSlS5fMhx9+aJaWlvwSe40NQnpWtCNbgdXQwQAFJJX5pkzHwoPbm02S5KW1tbXg2VunT592f21CCV2gU7zsbHq5AgA64ZNPPjG/8Au/8L9ytAEAwKgYR2zvwLaPoM1qt5hEIBLAbrQgrubJ+vPo0aPPlC9VFmSapn+lAF6apq0py5MkyWVjzJL6KSrrMxRlNurfVKZjkiTDBmzzsq5emdcgdCxtSaW1Bjf3BgAAAOpipOonAAAAnmjjCIKR3VSrYCSBSKC7lPE32BNRwSljG1vrS4Eqr/+Hb91ez1oVgByg6+OSK0EbijIb1ZtqZmbmJdtLc7f7gnq0HVcPwdXVsLcQvTbrSuFBAAAAoDrjDW0hcMTYzAwAAFCZRo8jbt++XXhgvwhGdlNtgpEEIoF6OnfuXJ6FuB319xsMIkbmsubu2WtYV8oF6vVen5iYOKWgXcisRP1bKmtrjJkf4v3MsyJD94pUsNn190zT9GbhB1Apfc5DU++E559//tfSNOXgYq/yydxO9ygAQLdoTBmyjcCtW7fcOPlIQ+cd2khoHj16VHgAAKribZCY5SCgJvK5JeOIgkOF7wRCMLJ7atMvkkAkUF9a5LU3zp2s7fDYKAZvzHe8nUP0KPs8OzJKMFJB57GxsVM22Ljdbq3xLMveePLkSfASrQpESpqmFwsPomprU1NTvdDPQRPQmZmZFwsPNE8+gb57927QJ65JkLXd5xGf3yOC99I19vyMcd4DAKLRXKGnKiohFxFVDcRq9CJijIwGANgrNkighjS3PM44ooDMSARTiw8AgUigMZ7jUFVu9enTp387MzPzm8pGDVlu6caNG+b06dPGZkduVyb1RJIkBxWIDD15eO2119xfh+1biYbTAN9udBimPHCdRck48EpSt7X0dK0pc9c6RA9bAGiO0BtUvM1GszuMkeusR4lWAACGwzjiGSotOx1rHJEUvoO2q3xxi0AkAIzmwIED3zbPBu+C+OCDD/J/Jsuys9v9e1mWfdNEKNGqTE9bBvQ6WWC1tFmuJCRvQHuk4e9PlJ2CMbL92kilnYfI3h+ZyulYTT8/AaAr8o1NodtHaLyiDSpZln2l8GD9nTD0iwQAYBiMI4ryKlCxqisQjOyeSrMQCEQCwJ6saPFdZU1DBoeUpba2tmaSJHlpm74Ns3rs448/DlqyQk6ePOn+2sTd5l2Qb17yMvWC8II9W51vjZFl2e+bCAN0bxJEZuQO0jT9u+0f3bsWBcsBoCui9RFeXV3VGHnMBfca5IR7/gAAYEeMI4ry50swEkFcWJ+sbERKIBIA9i5N0zw70vVZDMXrAzm/xT+pXpJ5OdeQFHCxQZc1gi61lZeoDD0ob/gOQeeQC9KHRs/IoeXXjaNHjwb9R70JV9MmjK3kHV/uEwC2k18fQmc0mGeDeY26J2jDlMZaBCMBANhVlE3YprnjiPEsy74acxxBMLJb1qp6tQQiAWDf8gzC0KVaNcCwzbVP2drwjnqmHddjoQch9IpshDwYGWtxz+4Q3CoA3gTzJlLGgReMZAVxZ1GC5eoBarMjp11fUNQCwXkAO1nTImLoUue6z9tewoNj5DqbV693ApEAAAxtTesejCNyJ7RWE3McQTCyWyoZkRKIBIAgtBh7XX0W1W8xJC87csH7Z/O/v/POO0F/lwZ4yu7MsuwRwchai9I7weyejVt7/X7/awOvIxj1Qez3+z9u4vtSsmiZMK6XLtmRANAYUbLlzbPVQRYKD9aQ6/UeuqoJAAAtxjjCcuOI0OuAPoKR3VJ6MJJAJAAElWdHev0Wg3CL7y7IYkszvBGjNIN77kmSXC48iFpRUCxGsEelMG02bq+B2WfzaZq+qM+MsuhCcpOfNE3jNGdol2jBcnfNy7LsbOFBAEAd5RfuGIuI3/3ud115+XMNyGqYd2XkvR7IAABgZ4wjPpePI7TWsbGxUXgwFIKRHVJ2v0gCkQAQ3B1XQiLkQEkDDQ04FGSx2WoqzXBQmV+hAy5eVidZkTWXpunf6BnOzs4Gf6LeTrvlwoM15nYKxsiK9EqOUlttOFHK6eh6qIVcTcR0+hd+AABQN/l9M8Z4ReNgZTXY8vJXCj9QH+P9fv9P9WwuXrzICYoY8pLpoUvkA0ANRAtGNmkckWXZn5nIWZGGYGSnrJf5YglEAkA0+QAmdKlWN+BQsMUFXLSLKyQtEtnG4NddzzfUWrRBuYJ5Xq/SpgR8FlzGgbI7Q/MWUQlGDifa4vPVq1fdXxsVLAeAjlKQ5F2NMb3ey8FoPNyAMcuyNhXquZIViUjyMoahN4F5jhS+AwDleOA2unZ4HHFFCQl6rjGzIg3ByE4pbWGLQCQARHUzy7KfqO9iyIGSnw2kr7W1teCDkNdee839lazIZogW7JFLly7lf/b7/T8vPFg/4y5Iv7wcPj6lhR31i9Rnm0D90G6aSOengs26HtpSwk0Jls8G/mJREECT5GMWjY9DU1aDu/dnWfZeDcusqarJGS10xs5mAEK7deuW+xfrXr4QQLtFm1s2ZBxxqqxxBMHI7iglGEkgEgDic/0WQ/eO9LKB/EbbQShwqmCL+hCS+dUYUTMN1JtPAZ80TV9uQAbainYKanAeY6egm/QkSfL9woPYzp0syx7FyNw13vUwy7K/KDxYH8t6D4wxnxljPgz8Nc+ZB6BB8o1uoSuHONqkomwBjQX6/f5HNVpIPJJlWb6pSwudodsrACUiGAmgSnkwMsamJrP1OKIuNscR586dK2UccaDwHbRV9IVfApEAUJqVLMv+y7Fjx8YUHAk1YHDZQAo+hS5Defr06fzPNE0vFR5EnWlQflyD8hi75LRwpsD32NjYkh2r1DFQvaD3QGXPYu0U9HZgkjU8AgVvx8bGTmnxOXQfT10DlSHe6/VessHyugXM83H3kydP8pJ8CpKHCJTrvbTltDkXATSJ20B1XJtUYpRTV0UH9cubmZl52bZNqHrThhYQP1IfqvPnz0d5zUBstnShsRUZbvKGA6jIPVuqtadyrTFKnmscoTHK1NTUy3auVatxRFll3glGdsP6hfXJBzFfKYFIACjVgyRJvqdFeAUxQi7Ch17QN7YEpZ6nMniSJGGS2Sw6XtcUoIgRiFPwRAHJixcv6vx4P0mSL7ueNDVxwhjzrcePH0cpz2ps1nCv18tLtCZJUqfX3gT5+DP0ddDRMde/W8Ngef66dV5qo0fIiaPKadtrNecigKbJN1BpzBIrMKesAWXOT01NubWPBRsILdvmAuIHH3wQ5R4IlOHu3bvut1AeHkDVNMfqaT4Ua+7/5ptvDo4jqgpIbo4jlLFZ5jiCMq3dEHUxYXF6Y4FAJACULh8duYzDUDQICT0Q0aLQ2NiYspi+U9GCDfZOx+u6MqVi9Y5UuVYFOjUQrlkPBQ3Q/9LYoFSsnYKuFIwrv4yRrCqIq2BujFLCW/T3OFT4oXId6ff7P4oViPSu1ZQLBtBEK/1+/6eh+6r7dF/QQqK99p6qqGTrCT8QGWvBFCiDPksa02RZ9tu84QAqpgpkjzSO0Ib6GLYYR/yo6nGEMjbLRDCyG6Lt4l6c3lAE/1uFBwAAseVlJBQkitUzLRSvt+UVzopGitqHSRSM1EA4SZKXsiz7HzXYHT3rlyxRwDQW7by0KIu5By6IG6u/h46919/j/QqD5fM6J9VjVeVj9XkMHSCnXDCApkvT9NsmQl91n1tIdH2vsyz7B1tJIbZxO5b+gRufEIhEGyiTWeMsDUU4oACq5OaWZYwjNJfzxhFlXP9qMY4gGNkNUVbQbCDyWuEBAEBZ8pGDF8yoHQVKbf+xd20AFc0TNfvM0UDYC0h+VOGChMY3H7oBesySJS4TTdmnZA3vmTJh/kXXwVg7WLVbVOemJos2C6bMDMlxGxy89uTJk7HLly/nZQJD9Qp2/HLBNe3d2nozMzOm3+//uOvvA7BPV5TVoPtrrHuCsQuJyk53m1W0sGfLxMa6P8zb68MZ9djTIialWVE2ZR6rl1po3rlMdB1A1a7EnlsaO47Q7/DGER+WNY5QNnqV4wgFI3uF76JNPr2wPhl88ZdAJADUQilBov0g66sdkiT5holQFniQApJ2QD5mB+RlLkpsBn00QI8diDTPvp8svuzdgzRN/0pB3Zg7WF2w3O5eLSt794gNfp7S4rPOlxs3bhR+KASvXPB3Snhd2Eaapv+09SMAhqS+6pd1T9DGjdi0WUULerpGq1+lMebv7Vgi1GLivM2YuJam6YsaI2lsHasnJrCTNE3/zm6iC0pVKGy1h15JWcYAsB3NLS/Gnls6ZY8jNJ+N2Vt7GGRGtl/wnc0EIgGgPsoKEu3FQKbNTU6bRoveh8nRgPytt97K+8cYY5ZsH4XYgZ8TdqfgZtAndiBSi4k2a/g6WcP7tpklHnMHqxcs1y/5YeQg8oJfllWvLVbfUvPsxhHKaQNouiuu51MZm/W0oKeFPZWct2OXU3Yx8Y6u5SMuKI7bYEzet0rrLqoYoZKwuk5rjBQ6Mx4YQV7FI8bnyvUss/3aq27XAKDb8nGE1gQaPo74Z38coaCn5rNVjyMOFL6DtrkT8vUQiASA2rmZZdmfzc7OHtQifJ0WKNxOMld3H82WpumfaAygAWzs4Ld2SGvRTb9rZmbmZRv4uW6DPyEDd7P23+ylaZoHmjQJiP050meVrMigdE5cHxsbO6Xrjo5hLFosU1BQGTdjY2NLWZa9YTeFhMr+PmIDgr0nT57kryVWNqRDuWB0WFdKEi+EXheouQf2uvytMsYsjq7XGkeo/67uRVNTU9N6DvrSomaSJB/b47DVdVa9qv+9FgzdN3QP0HhI94CYm1GAEej8Pa5SrRsbG0HfNy3GqyrJ0tLSWL/f/+923sEGqZbxyvyyERN1tjmO0JyvjEoLZvdxxOMkSW63YRxBMLL9gk2wCEQCQC1poPSdsbGxM1pQjr1oPQo9HztookRrO6w8ffr0azMzM7+pAbIGtjFpkUMLiDqP9OfExMQpu0twzQZ+bm4zEN/NuM2EPKesM/2sdgpevXq1tHIlej02+PM2k/FglEn41ddee21M5WdCL5L5lDWrc0UT016vpwnftSzLvmk3Xqzs8byctf1KdY7n5+TFixdLmTgSGEeHdaVlzXjhO+2nnk9/pA1NZYxZHG1m0j1CX8qm0O9W//SjR48eHBsb6+10ziVJkl/77969m99jynrOwAjyTQ0KKMU4P11VknPnzv3i2NjYt/r9/ltpmn7bjvm7tKGitbwKJsx/UHf5OKLX69VlHDHWlnEEwch2e3hhfTLIDZtAJADUmnaNntHuqboEI71Mm++RadMeBw4c0C7lDxWE0QC3jExcNxj3gpJuEH7NBiZX7de9bSa2h+zXrP3KB/DKhNRgXZ+ZMgfqmkzos2qzJAj+hKONGZeUrVjGDlYFO/U7dDxV8mZmZuYlt3PVGLPuLZw9GNjBOu6VH9P5eCTLst+2pV/zfiHaFRu7TLCjz1UDygV3MZCCiOpY2j4Gvc6uvNatpGn6n1RZQdmRutaVXT1E9wmNMdzYXIvwXlbQM7TxhPKraIB8ffGVV16JVoXCbfjStWtubu5FtWywX8aO+80OSRduvLXd4wAwtDRN/1hrH3UZRxi7lrAVzSFjbsYNiWBkuwW5AROIBIDay0sUKnOszF1bO/GafRNsaRedXG9PTEyc0SKB6+9SBheU1EKeJgM6173A5NKwT0EDdX1G9G+VXa5EC5GazJjPdy7+IYH64JZVNlXZimVdC92CmXau6rqnCaItqTNd+OFtqISO/h2dk2Vev+teLtj7fB6h7zCAPVBg4m1VD9G9t6wya9vRImZZFRiASO5lWfYTbcCK2R5EC+r6zCrg6bKCNP63436zU2aQ+bzvpDb8/S5BSQD7tFqncYSxc8+mIxjZbvvOiiQQCQCNodKAp9Rnr+pgpJsw2t2rlGBpn+V+v/8HJ0+efLGK8h8KUCgIqi8FgHSu6evw4cN+6Z9NWihRqRL9f/qqcsegzezUX98luBKHDfKWvoNV55ULzrvsF7dzVRkEvlu3buX/5c7Hqnp4KHha56xIsoQABLDQ7/e/ojJrGiPXqZ0B0ERJknxfFXkUJIxdxWGrrCCNr7Ya7xs7B9XcYG5u7mCWZe8lSfJLhR8CgNEsM44Ii2Bku+1rdZBAJAA0ymq/3/+xeuNoklZlg2oFACx6RbbTgzRN/6ML+GhQXlWAT79XX03oq6TPhYI//X7/p2mazhd+AKFUvoPVZb+4nauxSpnthxbrFBy3fX0XavcEASAQlWvNsuyjs2fPjmkzSJVjZKAFNL87o3FtWSXlfTt9fv2MIQUkbTl8siMB7McDxhFhJW16MXjWhfXJPd90CUQCQPOkaZqn5Sg4VBW7GzUPuBCMbDWNMc6rL+jFixe33aGMz2mDgAuKpWk6R3nW6LSD9ce9Xq/S62GdeeWC/zPnI4CWu5Mkifo+5ZtDtuvbCGAod+wG2HzeV0feJslDHFIAATCOCIhgZHut7fWVEYgEgMZaUY8MBQOrCg7pd5vPAy7fLjyItlE0410/0IYifRY1aVHg1hjzeogy+tiV28H6+OzZs0wYBygjUouIT58+/VtjzJXCDwBA+6zYrPl8MwabqIC9cxtgvb7TteJlSBKMBBAK44hACEa2156yIglEAkCzJUly2dheYFXwSrSywN0N89odrSC0y7TCv9Ek5erVqy4QeZ1s4VKxg3ULeh9cedYDBw78x+JPoApedglZqkA8Kkl9XddB3ZtZSAT2bEVVcDT+Z3wFoEMYRwRAMLK9Rt51TyASAFohDwJWUZpQgciJiQljgy4sqHaDMtC+7DJy67pDugouEGkXafSZoE9k+djB6nFZuubzjSsK1N4r/BCGkc+zjh49GuzNsvdOQ+Y0EN08C4mju3//vvt/jjToaSOiNE3/RP/6W2+9xdsMoEsYR+yBN444RDCyvUbKjCQQCQCtoSDgdS2+e1mKpfB+H9lf3fIgSZL/TZlWCkZSsrUQiHyXQGSl2MFazNJ9m+v0vrDZBmi2ZxYSyezandeDb7zGTxPlWlG5d5V9pz83gI5hHDEiv5cvwch2Wr+wPjn0JJlAJAC0Tp4dWWapVg3ANBlVyc69lgpHo6kkpjIkH+u863LJVpVbJCOydjq/g/XSpUt+cHyh8AMA0C2b9wVljIfMdAa6QuXe6c8NoKMYR+wRwch2GnoRmEAkALSSyrytaWBU1qDI7YhN0/RS4UF0hQtI/kQlWxX86FrQR5+5GzduDAYiyaKqh87uYNXmAG+zCMFxAPjcvCvlrftCW7K7NPby+tACMd3z+3Nz3gHomNaOI2Ku4xCMbKeheo0QiASAVsuzI8so1aqByuzsrFHfQEr/dZ4Ckr+uoEev1+tU0EeTDwUivTKYBH3qp1M7WHVt1jmpzQH6TKq/K8FxAHiGMsVfd9ldTQ+o6L7mrvtASVZci5CLFy/SPw1A1xTGEU2+Dmoc8f7770ddwyEY2U67ZkYSiASA1rvpMtRiL6oo4KkJaJIk3yk8iC56YIMem0GfsvuXlkmTDWWBavKhSYgmI5TBrDWNgc+3bQfrIL9cMIFIANjRiio76FqpLHIF85p2b9BYRD27dd2fmJgoPA5EtrnZy6sQAgBd8cw4QsG8Jo8jNE8+fPhw4WdCIRjZPp9eWJ+8t9OrIhAJAN2QJMllU0LvSO/fv1J4EF31wC5MfP3555//l6WlpVaWbVVGsCYbygLV5EOTELKDG2HZ38HatnNT56VfLphAJADs6k6apv/BbVbRvUHX0SZk0GvDl56rxuO2SgnlMlGFPCCpYLg2ImosAgAdUhhHNKUSj8YRWtPw1w1ttacoCEa2z44lWglEAkCn5EGRmJNB/dt2B/a76htS+AF03ZU0TX/LlW1t4i7BrWiRTwEslaNyZVltwGfHcRhqZXMHqzs3m57B63a0euflefqWAsBItFnlV13vdWUI1HUxUc9Jz00bvuxY/O0kSX7FEIxEdfKNiK5kq3pWU7YVQMdoHPEbGkcoS7Ip4wiv1cyr+guZkRjFtiVaCUQCQOdoAXpNCxSxFiW8xXuyIrGdzV2CypLULkEFfprYr08LKqdPnzbvvffeZjakHbAvEPBppGd2sGoipglZE8uLuWxILzPmVTsZBgCMRpvrtJPvVVdyTYuJusbWYdOKWzzUc9Jze/r06d8yFkGNaE74qmsX0paNiAAwgjt2HPF7uhb6Qck6jiO0ZuiNI/K4UsyNJAQj22fLYCSBSADorPy+EKN/jAKcCshogLXd/QfwLKdp+rKyaHU+1nmX4CAXhNSCiv60wZ6v20AW537z5TtYtaDr+oVpN38TMkvcZFIZCPY6f91mxnBeRkbmEdB6q/Y+/3suU1KbVlZXV/N7RJllKDUOUUBH4xC3eGg3RP3egQMHfotrPmpmNUmSX1eWjStX6MbQ3DsBdMhNOy973c0z3ThC1WzK3AC7wzjiVRs4LW0ccaDwHTTZwwvrk4XyYAQigdbKd77qpvLo0SOOMkrnasq73pTAEJRtcMIOeJdnZmZ6Ggzfv38/D6hoYF6n65kmCBq0a2e3qMegMeZSkiRXyD5onTt2QXc+y7Jvzs3NvaTj/sEHH+STttu3b9fq9SoI+eabb7rdrPlkMk3TP27TgrRXHqhun7Wbxpglt0EBQOvdtF9HlDXwwgsvfHVubm5M94jHjx/nYxfdI/S1sbER7L1Q0EbXegU9tfnPs2Y3eBGARJ09sFk2GjMvT0xMnNJ9U19ra2v550WfnZCfGQCoqZUDBw6s2DWQeY0jTp48Oab1NK2DuDFEV8YRBCPbhUAk0C26cRzXgF69y4AyKQiuEhMKziRJssKbjxGt2sG4vhYmJiaOa5egvhT80eKEvqqgQbsG7JocuIxiZUIq6E4QshNW7DXtmaDkJ598snluVrVwts25+ZMkSb6RpmnrrsNe1lHdFtzv/Ou//uvPJiYmvqj7IAHJ+GL2rWkzMpCC03rLfJIk83Zj1Qk/MClaVNT94u7du/mf2mA1zGYWbX7S2FqLhjrf9d9+VRN7rf+O7QdPj3b4n++6j0vv2V6Sy3Zs9Uav13tJC+PKmPQ/M/qs6DOj/waAFtKcRpnjC24c8cUvfvEroccRLghZ13EEwch2eSYYuTi9cYRAJNBquoksnzx58gu73aB0Q9tu8VQ3qsESnrZ5Mdph1p0DIWmR2J4n3+t6cOb/b+8OYuO6z8SA//2egMCKhAjoIRDdrBWgUC5pydC6tVvS2T0astLsxQK2ktvAuuzWiuUTDyuqB56sgEZPNnYbCwXknrqyrWsRCW1vllaD7iWCgUgbhEYOxcorI7l4xsU3+j9mxCEpinqPfDPv9wPGkvmGFN/jzPCb//f/vi+eP7EpoO6v2RE38u1YJCX7/f6fvfLKKy9U1QbVrum6dwmOqoL26rahXcrNkeTUxIhqU57ZelIybsePH1+IRbO4VYnJTz/9tPEFs3g8njhxYlidu8ljc7Uoimtjn7TP4jn1NG2HNsYvVUVyNQvtwIEDYxsu99s3vvGN/55SejOSkZu9Nm0Xd42q2lRLtm0vnnfsv6qqiaFruf3a2ZHNVYvf/va3Xzp69OihDRUI6+J3RsQ3Kb/WPeE9183878TC5U5fB4+MfaTl4ndpVzR0rq37HbmFWPxeLopiOVcZV8+ZP9nsORPPkyrGGr1u1eJ8kLhsXrxG1TlSYy/inUl8H1Qlj+rQtZgy3idlk9Qt4EFez433m2k0jjh69OhCrAVtFUfcvn17/e8NxRGNi2Tk/ZTSi235hngm65ntnIjUtgOm24O8m+YXFgbYQixILOx0UfRpVC1a8y7XTmsiGdlBEcOcL8tyfZfg888//6cLCwsvVIF4tSgRCxLVY3p0QWInNu4YjP/fJGlyc6Ql26RVH9zL339TX7uLPsi3SJif6vf7//H48ePfr5Ijo4/L+LN6XO5G9biM15R4Y73xDWZuxfo3bX5s1rnQGhXJBw4c+IuxA+2w3O/3/2R+fv77kv+N8n62Hs/6evHA75ZtVRurUl5UPJaTLXM5Fp+rPjk2tlR/7/f7v0kpfZb/90FOKt3Lf+520XD4b0VFRZ0idmpAkwujbXuEY2AnAAAWmUlEQVRcdelcd6J6jK9u8pwZPm8OHjz40vz8/DAIqlrSP0mV7H/C4vwzi++nK0n0iEUnKM5p8n1Qo5uvY9NhVRk37Zp4/uSqv7GPT5AbG2LeuZHXxfU4ot/v/7N471Hdqck4ou7ROSNrLg8O5G9SMnI6DB9oI4nIb3X9gkAHxHP9u7lyYztz2+yUvdP1yrYpFomd2lvIjVSP3dQqajjwm3oNE4FlWabRndPPP//8ifn5+Re2WpAY3SU4KhI7T6g07Y28Abgx4a+HVeKM+sVr3WpZlqtVYjIelwcPHvzh/Pz84c0el1s9JkfFgtkmCfGh/Abz02pHa1mWbX693fiGd6dJpMcW6kdcy5WpbX0+PijL8o9z/LVZfLVd3DXq3obfo5Jvj9u4OMP+uFN12mBHquf1tpXrOc5pRIOLiHU+H8+PfWR6delcd2PsOTOSWBiNEzbGDMfybWg02d+gppJdbaxwvTT2kfo08bt9Et8H3WvwOrcxfmrk+dOmqr+aVMnEx+KIjXFDQ3HE8P1Lg5ua7hzIJ7cXL9g0745EJPvIQsH+uac6jc30+/2fRIAS7QTrFG3pMkkPr31NW985nYPtIxuqDYaLoxt3CY6KuabRCTJ/6N6Gm58fuzFMTG6xo3+4MNbv9/9FJM93+LWrN+bV5qB4XN5refJxoy4utD7IjwOAthjGRVpXMkUebIjXt03yZ9X7hSbiqC5tzLDG1LyureXZ2NR+ww0fdY95GnUgv5C/OXaESfNFXvyQiGS/TNtOFJh0Z8uyfCESkXW2aI0WgtFCJCp2yrKUjGSvPdisUma7XYET3rKFyTC2o3+7xyQANGS4iFj3eIaRGWRd74jCZNiYwARgZxqJI0Zmot4pVnozNxosM2fvPJCIZB/dX+nN7GSHGrBHvvrqq5+kBlq0VrMMyrL867GDAADAvhgMBj+sZhnXaWQGn2QkAEypfr//p3Fmt27dqvUER9q0DmdGptxWRyJrspn7yX4yAwHaZfHAgQP/OuaV1R1EjLRo1ZoOAIA2GauGGgwGLxVFcehJ32O/3//7siz/39iBR1XvbZ5hW5kriuJw3bF/OHHiRPVXyUgAptXcdutcT4gntiv0q9rxtv136Fx0V7t5c7tT2Z2RDgvDmZFppTcTswYXJSSBXXhdVSS0ztnUQFVkJCKPHj0af70yAQsyAAB0yGYLhZ999lmKasEn2Wr2dEppYTAY/OeiKP79DufV7ZfhBuEbN8bysc/s+PHjwxncRVFIRgIwrWL82MJW5/bb3/5201mK0T3g+PHjW35ejiN+XBTFX+bNTW01XEdsMI54WBTFemXkaEIyLsrs2GcBPO5+vFDlVs9Ae8T84DMRJDWRjMzMigQAoFWKooiywIWTJ0/WMu8oYt8LFy7EQmP0F/vb2Ijb0jj4SCx0/u53v6s9/o958blNa/0llwDQIrlLwvdHOgI8s9OnT6e33norfpH+PFdHtnEdPeKI/xBxRN3JyJE44nb8pxg9GAnJld5MlKT+NKX0xdhnAzx6bbi00ps5JhEJrdRIVWTsZJqfnx8GZy0NngAA6LZhjBpxax0ino6E5Icffjj8aoPB4L/kNm5tcz4qQq9evVr7tzVyLcX/AEy1siyHG29eeuml2k4zfjdfunRp+PfBYPBJLiBom4gjDsf3+vDhw1q/tZFrOYwjirF7PEpKruYL8+7YQaDLojVjJCGXPQqgnQaDwVvRiqpaNKlL7OZKj4Kzy370AAC0ULRYq3URMRblLl++nH72s59F5eWhwWDwcVQQjN1x/xzr9/tvNxH/p8ev5Z2xgwAwXW6kmuOIlDc3vf/++8M4ot/vf9K2OCKldHEP4oitk5HpUULywUpvJnrOf/cJQziB6RevAT9Y6c1EW1Zz4qC9zsZupmirUOdupsOHD6fFxcVhj3ctWgEAaKlGFhFTrmy4fv16LCR+p9/v/6+2LCTGomZZlt+MRc66qxlCvAfIVEYCMO2Gv+tGfvfVJn5P3759Ozb4x4zq1bZcx5wcTU3FETtORlZWejP3Vnoz8RN4Oc+IA7ojnvM/iteAaOPs5w7t1u/3L6QcRNQp2lNFj/eiKP6rhwAAAC0VG2d70Vo0ZhTVbXl5eZiQjIXEliQkP4jvJRY3m2jRGtfw6NGjKW9OtikZgGl3L0YTRRwRm/LrFnOo7969G1/1TN7oP9VxRFzHHEd8VH3sicnISsyGixlx5klCJ4zOhbzmRw4TYbEKItbW1mr9fl977bXqr63ZvQUAAJsYdvFooqohbUhIDgaDf9jHGZJxnmc+//zz4eJmE1555ZXqq1oTAKATyrL8n6mhOCIqD8+dO7eekNznjU3DOCK+l6biiChsyNbjiB0nIysj8ySvjB0EpoG5kDCZzqbcQqpO0VJhZCfTPY8NAABabLjgNZJIq10kJGOuUoxHSCn9XXxoDy/HkXyOZ2K+UywgNtFWLW2xiAgAU67RTU1VQrJq2ToYDH6dUjo1dsfmHMktU4eJyPhemoojRq7h7pOR6Q/zJGPR8wfmScLUMBcSJtexamd0zIus0+nTp6uvZlYkAABtt95irYlWrZXLly+nS5cupUgIppQu9vv9/xvrbmN3rNfZXI35aiwgRrIwV1fUzoZEADrqTiQIFxYWGosjIvn3xhtvDEcsFUVxKKX0tzlBeGzszvWq4oiFphORG+KI9TzDrpKRlZghl+dJ/sg8SZhY5kLC5BtWRX7yySe1nkgEXhGA5Z1adkQDANB6ZVn+TWq4OjLl2Ds27lXVDSmlX+TFxLqTkme/+uqr/51S+nlUY0ZVZpMLiOnxqkgbEgHolKIofpb2II6IZORI29aFlNKv8u/dxuOIiF+ajCO2Kmx47uuvvx67824tza5Fa4rzKaVvNXYmQF1iLuSqdqww+QaDwT9FQPHyyy/XGkxE26c8L/Kn5kUCADAhogXZP0bVYlNt1jaKCoCInaMiMz2Kz39dFMX/yItwu9n0G9/4qX6//2dlWb4QH4ik53vvvZdu3bo1duc6xYbEjz/+uDqHP2r0HwOA9tnzOCI2AUW1ZK4mTNHlIW+uulFnHPHOO+801lWhsl0cUWsyMj1KSB7JC5Znxg4CbRFzIc9rxwpTIaoif379+vXh/Jo6RcvXgwcPflkUxXdG2yoAAEDLRRLwTLRSrbt7yHYiKRnVANFdpDIYDB4WRXE7Lyje26Lt6Vxe/FzM1RHrIs6Pc2g6CVmJ9xS5GuR1lZEAdNS+xBGRlIzb/Pz8+scGg0Gsy93KccSDLZKTizmOmGtzHFF7MrKyNLs2l5OSC2MHgf0ScyGXV3oz9Q6VA/ZNzKeJtlCx6FHn7qYIfi5evJjy5oWzY3cAAID2irlLv4qZ6iMtR/fM4cOHh9UUcYsE5aFDh3b8T8f3HAuGcYvNgU22Uduoqmbo9/u/Kcvyn4/dAQC6YV/jiPh9XMUQkxRHRIeIq1evbhlHNJaMrCzNrp3KSckXxw4Ce+V+TkLa1QjTJTb+/F20Woh2DnWK4CG3mfrBFruuAACgzYZVDTGTKW77KRYVo/Xa9773vU0XFGPhcG1tbbi5cC8XDTeK65SrMcaqGQCgY1oXR1R/bjQpcUTjyciKeZKwL77ImwFWtWSFqdRI24hqJ1Oupt6bBvkAAFCvYzFz6fe///03o4tILNKxtajAiFlSeU7Vv9zyjgDQDeKIp1B1WPvqq6/+z4EDB/7NZp9ZjH2kISu9meVc3nqlFVcHpl881+biuScRCVMpesGfiYHadfevjyArG9vFBAAAE+JeWZbvRCXihQsX/My2EW1lq/nzZVn++db3BIDOEEfsUFRsVtfowIEDf7HVZ+1ZMjI9Skg+WOnNnM0t326O3QGoQzy3Xo7n2kpvZrPB+MB0iG4DVQVjbWIhIgZNR393yUgAACbcclQ1LCwsjG64Y4PLly9X7WMvGdEAAOvEETsQG5p2EkfsaTKystKbubPSm4m2bz/Ks+yAZxfPpdfjubXSm7nhesJ06/f7P4kTvH79eq3n+dprrw3/LMvyr8cOAgDAhIlKv8Fg8OVbb71VzURnRFQyxHynWGyN9UTXBgD+oOoY8MYbb4gjNvE0ccS+JCMrK72Zayu9mWM5Y/rF2B2AnfgiP4eiJasqJuiGU2VZvhCJyLp71keP98zrCQAA0+BOURR/Gefx/vvvD1uJ8UjE/rEZcTAYPCzL8o9dFgAYE5V+r0flX8QR0VGMR542jtjXZGTFPEnYNXMhoZuGLVrrnhW5uLiYjh49mvJrizbPAABMi9hodyUWEt955x0LiXkB8eLFi7GA+GVRFP82pWRNAQA2tx5HvPfee+KIXcYRz3399ddjH9xPS7Nri7mcc6FV3xi0S8yFXNaOFTopNu/86vPPPx+tYqxF7PCK1goxdzal5PUFAIBpE4uJZ+7evZvefvvt2ruMTIpqATF7XVcUANiR9Tji3Llz6eHDh528ahviiBjFeG3sTptoXTKysjS7djYnJV8cOwjdFS1Zz2vHCp02DHwuXbpUa2VktKv6+OOPY0fTr4ui+KOxOwAAwHQYxtNffvnlcP5TLCh2iUQkADwTccQu44hWtGndTE62zJknCeviuXBMIhI67chgMPhxBDw3btRbuBg93kNRFH81dhAAAKbH2arV2tWrV9Pp06c786NdXl5eb6mWKxmsLwDA01mPI6LDWN1dy9rsWeOI1lZGjlqaXTuWqyTPjB2E6fdRroY0vw2IgOfnH374Ybp8+XJtFyN63UeV5cGDB6PP+3fMiwEAoAOGsXWc5s2bN4fx9bS2bY0uKDEr8/jx4+nrr7/+p+eeey5GI90ZuyMAsFOPxRGRqJvWtq11xRETkYysmCdJx/RyEtLcNmBoMBj8QyQLT548WetCyUiLhSs5mAIAgC6Y6/f7/60sy+9H95GolIyNf9O0mBiVn9FGLio4Yr00pXTK5kMAqMXcYDD4ONbqIo6ISsmIJaZJnXHERCUjK+ZJMuXMhQQ2ExtyfnH79u1hEFCnCJRid1NK6bspJVXYAAB0zXK/33+7LMtvfv7558OEZHQOmeSk5EsvvZQuXLhQxfnhpyml1bE7AgDP4kis5aeUhrv8I46IpGSMV5qyOOJSzsnt2kQmI9OjhGT1Q47bt8buAJMpntSrK70ZuxSBjYYDst9+++1a50VGcPHee++lvLtpcewOAADQDY+NCIoKh0hIRmJyktq3Rnx/7ty5ND8/X33oZu5+YtMhADRHHPEEE5uMrOR5krGz69WxgzA5zIUEthMbcP4xdlfVPRh7pCry5ZSSttAAAHTdcDFxMBj8uCiKYU+yu3fvpuvXr6dPP/10+Pc2WlxcHLZSqxYPB4PBr4ui+E8ppWtd/4ECwB56LCmZxBHrJj4ZWcnzJCMpOTt2ENrrfuwsMBcSeILhUOzYTXX58uXt7/kUYrj2K6+8klRFAgDAmCN5LtL50bWm2CAYnUriduvWrbFP2kuxqTA2K8YC4tGjR4f/cr/f//uyLC/nzioAwP6YyDgiJyH/qok4YmqSkZU8T3JV61ZaLuZCLq/0ZsxrAHbiTgQuJ0+eHLZ2iJYJOxVBzsZ2EKO7nQaDwcOiKP6Vtk0AALClY3nz3qnRzlzRgi0WEuO2F9UOMzMzw/cC1a1aOMyu5IVDm50BoF22jSMiMfnLX/5y6uOIqUtGpsfnSV4cOwj7792ciDQXEtiJCFh+1cSVyrum/zwnOwEAgCerKh0WB4PBvyuK4nD1GbGoGAuJkZiMv8fC4sOHD596cTEWC2OBsPrzxIkTw+qFQ4cObbzrR7l9WtysMQBA+z1VHBGetoJyszgi/tyQfEx7HUdMZTKyYp4kLXMzt2RVfQQ8jbP5NurOUwQJcznQGXVvJNgAAAB2by5XOyz2+/0TZVm+sN1XikXGWGDcaItk47rBYPBlURS3csXCDRWQADAV1uOIwWDww9Hk5GaiA1rcNpqEOGKqk5EV8yTZZ+ZCAgAAQDccyQuL1abAY/lWWdjsKuTxCbdHPnRj5M97xioAQCfsKo7o9/u/Kcvys5EPtS6O6EQysrI0uxatW5fNk2SPmAsJAAAAAAB0WqeSkekP8yQjIfnm2EGoTwx8PW8uJAAAAAAA0GWdS0ZW8jzJD7Yqa4VdupmTkHdcQAAAAAAAoOs6m4ys5HmSkZR8cewg7Nz9nIS85poBAAAAAAA80vlkZMU8SXYp5kKurvRmll1AAAAAAACAx0lGjsjzJFdTSmfGDsI4cyEBAAAAAAC2IRm5iaXZtbmclDRPks3EXMjlld7MjU2OAQAAAAAAkElGbmNpdu1UTkqaJ0nKcyEjCfmBqwEAAAAAAPBkkpE7sDS7FvMAz5sn2Vlf5KT0qpasAAAAAAAAOycZuUPmSXbWR3ku5L2uXwgAAAAAAICnJRn5lJZm1xajVad5klOvl5OQ5kICAAAAAADskmTkLi3Nrp3NSUnzJKfLFzkJaS4kAAAAAADAM5KMfAa5det58ySnxiVzIQEAAAAAAOojGVmDpdm1Y3me5KsTfzLddDOldNZcSAAAAAAAgHpJRtYoz5OMpOTs1JzUdLufk5DmQgIAAAAAADRAMrIBeZ7kqtatrRVzIZdXejOrXb8QAAAAAAAATSpc3fqt9GY+SCkdyzMIaZd342cjEQkAAAAAANA8lZENy/MkIzm5MNUn2n4xF/L8Sm/mTtcvBAAAAAAAwF6RjNwjeZ5kJCVf7MQJt8f9nIS81vULAQAAAAAAsNckI/fY0uza+ZhXaJ5k42Iu5OpKb2Z5ys8TAAAAAACgtSQj98HS7NqRnJB8s3Mnvzeu5GrIB104WQAAAAAAgLaSjNxHS7Nrc1G9Z55kbWIu5PJKb+bGlJwPAAAAAADARJOMbIGl2bVTOSlpnuTu3M9JyA8m8ZsHAAAAAACYVpKRLbI0uxatW8+bJ/lULuXZkFqyAgAAAAAAtIxkZMvkeZJRJXmm69fiCT7KcyHvbX83AAAAAAAA9otkZEstza4tRutR8yTH9HIS0lxIAAAAAACAlpOMbLml2bWzOSnZ9XmSX+S5kKtjRwAAAAAAAGglycgJkFu3xizJix29BO/mRKS5kAAAAAAAABNEMnKCLM2uHcvzJF/tyCnfTCmdNRcSAAAAAABgMklGTqA8TzKSkrNTeor381zIa2NHAAAAAAAAmBiSkRNsaXbtfJ4n+a0pOaWYC7m60ptZHjsCAAAAAADAxJGMnHB5nmQk796c8FO5kqshzYUEAAAAAACYEpKRUyLPk/wgpbQwYWcUcyGXV3ozN8aOAAAAAAAAMNEkI6dMnicZSckXW35m93MS8oOxIwAAAAAAAEwFycgptTS7Fq1bz7d0nuSlPBtSS1YAAAAAAIApJhk5xfI8ydWU0pmWnOVHeS7kvbEjAAAAAAAATB3JyA5Yml2by0nJ/Zon2ctJSHMhAQAAAAAAOkQyskOWZtfOxpzGPZwn+UWeC7k6dgQAAAAAAICpJxnZMbl16/k9mCf5bk5EmgsJAAAAAADQUZKRHbU0u3Yst259teYrcDOldNZcSAAAAAAAACQjO25pdm0xJyVnn/FK3M9zIa+NHQEAAAAAAKCTJCMZyvMkV3fRujXmQq6u9GaWx44AAAAAAADQaUXXLwCPrPRmPkgpHcuzHnfqSkppTiISAAAAAACAzaiMZEyeJxkJxlNbVEpeydWQd8aOAAAAAAAAQCYZybaWZtfmUkpHqvus9GZubHd/AAAAAAAAGEop/X/v3BJVuTshzwAAAABJRU5ErkJggg==){height=\"60px\" width=\"240px\"}](https://pytorchlightning.ai)"]}, {"cell_type": "raw", "metadata": {"raw_mimetype": "text/restructuredtext"}, "source": [".. customcarditem::\n", " :header: Tutorial 10: Autoregressive Image Modeling\n", " :card_description: In this tutorial, we implement an autoregressive likelihood model for the task of image modeling. Autoregressive models are naturally strong generative models that constitute...\n", " :tags: Image,GPU/TPU,UvA-DL-Course\n", " :image: _static/images/course_UvA-DL/10-autoregressive-image-modeling.jpg"]}], "metadata": {"jupytext": {"cell_metadata_filter": "colab_type,colab,id,-all", "formats": "ipynb,py:percent", "main_language": "python"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.10"}, "papermill": {"default_parameters": {}, "duration": 67.471608, "end_time": "2022-05-03T01:17:11.404766", "environment_variables": {}, "exception": null, "input_path": "course_UvA-DL/10-autoregressive-image-modeling/Autoregressive_Image_Modeling.ipynb", "output_path": ".notebooks/course_UvA-DL/10-autoregressive-image-modeling.ipynb", "parameters": {}, "start_time": "2022-05-03T01:16:03.933158", "version": "2.3.4"}, "widgets": {"application/vnd.jupyter.widget-state+json": {"state": {"0053c768d8694f69ad9f090bc7dd3e53": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_c74e75c66bfa4828a2227cf46d861496", "placeholder": "\u200b", "style": "IPY_MODEL_7a22126bdc614c3e8adbb44f2b5b69cb", "value": " 96%"}}, "03f97d8a4381448ba64353e80c4353ac": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_434fb56dc0e64cf1aa51ef3362d4f55c", "placeholder": "\u200b", "style": "IPY_MODEL_36e6f9a8c30247d8b3774215a520e163", "value": " 28/28 [00:04<00:00, 6.53it/s]"}}, "0955a161c78f4c6294b7463dcde991c4": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "125ec8e43b154050bbfadd3e0de21485": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_89a6a010595c41e496c8c179fc504622", "max": 28.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_12c8cb30c5514029894a2b906aa87b5a", "value": 28.0}}, "12c8cb30c5514029894a2b906aa87b5a": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "1c42120556214fdbbfd902358ae01291": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": ""}}, "28cd28d983eb490dad4c47e90c0a7c16": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "2bc166726f334ce9bcba379172cb2d04": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_68a6fd55da8d4753bb4a8d5d0d7f7c2e", "IPY_MODEL_125ec8e43b154050bbfadd3e0de21485", "IPY_MODEL_ce1ad59ac40645fc95ceed341a73818b"], "layout": "IPY_MODEL_9b433b71d731463e8849250fb35e8f96"}}, "36e6f9a8c30247d8b3774215a520e163": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": ""}}, "3835ae31957543f6bb49eb72f0ab5f0d": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_e72b83c7fe884736b4de3dca82aac052", "placeholder": "\u200b", "style": "IPY_MODEL_3b3fa45fd4fc450cb1f99268aaa8b9c0", "value": " 28/28 [00:02<00:00, 10.29it/s]"}}, "39eff407318e44f4b83b2ed74b19e641": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "3b3fa45fd4fc450cb1f99268aaa8b9c0": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": ""}}, "3bb05083d49d46bc97def13644356207": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": ""}}, "3e0bbd9c186c4fc3926f23ea8748c1b2": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "434fb56dc0e64cf1aa51ef3362d4f55c": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "480e436a71444d538a13d939c22ba256": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": ""}}, "491346b20e55493992ead44415c9f78d": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_0053c768d8694f69ad9f090bc7dd3e53", "IPY_MODEL_81f34696967e4e7898742dee31119448", "IPY_MODEL_ed3634d2b5db41a9a4cc1222816852f5"], "layout": "IPY_MODEL_82be5bb0aa4b4b9e89f10ba0133887d4"}}, "496ed24561bc491f8709c04c88e7e35c": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "5cc09e65caff49e2846725989b38c90c": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "5e4cc803b5e8419fbc29afee0dfe08d4": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "60b4448b21004a2e8adc58d233f19fd2": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": ""}}, "63e9c4cd6e6244ba94c7ca134d3410f4": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_c7f0837a85004ada9d89c3684517ebfc", "IPY_MODEL_87b72524d7d54fea8855e4ff15957178", "IPY_MODEL_03f97d8a4381448ba64353e80c4353ac"], "layout": "IPY_MODEL_8e857046712b43f1a94cb4a56bba8843"}}, "65ba9f8859cf48628ecbc8c826765e88": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "68a6fd55da8d4753bb4a8d5d0d7f7c2e": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_5cc09e65caff49e2846725989b38c90c", "placeholder": "\u200b", "style": "IPY_MODEL_3bb05083d49d46bc97def13644356207", "value": " 96%"}}, "73dd77769dad4094bfcef7897f69f5e5": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "77d61b0e1c464cde8b971939de0b889c": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_7e4859b11c914563bf92cf2f1092dd16", "IPY_MODEL_cbaa303ed5e14448be2eaf1ef821cb2a", "IPY_MODEL_a5a4d31f7a5f4b8b898eba9cbc4c90a3"], "layout": "IPY_MODEL_28cd28d983eb490dad4c47e90c0a7c16"}}, "7a22126bdc614c3e8adbb44f2b5b69cb": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": ""}}, "7e4859b11c914563bf92cf2f1092dd16": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_0955a161c78f4c6294b7463dcde991c4", "placeholder": "\u200b", "style": "IPY_MODEL_b0de1c8c26ba486c8d508521bbfc3ba5", "value": "100%"}}, "81f34696967e4e7898742dee31119448": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_bc74b1c35a1e43729bcfbfd949a8bd65", "max": 28.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_c6bf0eebcf394a2cb9ee1dc7580d3afa", "value": 28.0}}, "82be5bb0aa4b4b9e89f10ba0133887d4": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "82dca1386ee94406bbc359ae1c7dafbe": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_9c490a56f7044a13a2c9f26baa7bf73d", "IPY_MODEL_8cd4997e9160446faca978292d97659e", "IPY_MODEL_3835ae31957543f6bb49eb72f0ab5f0d"], "layout": "IPY_MODEL_ea8681142aa04f1f825a66012f606b1f"}}, "84975bea9cb7440fa4c98c492c5103dd": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": ""}}, "87b72524d7d54fea8855e4ff15957178": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_39eff407318e44f4b83b2ed74b19e641", "max": 28.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_73dd77769dad4094bfcef7897f69f5e5", "value": 28.0}}, "89a6a010595c41e496c8c179fc504622": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "8cd4997e9160446faca978292d97659e": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_bfa110cb409241698ee7d931dd684f70", "max": 28.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_65ba9f8859cf48628ecbc8c826765e88", "value": 28.0}}, "8e857046712b43f1a94cb4a56bba8843": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "9b433b71d731463e8849250fb35e8f96": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "9c490a56f7044a13a2c9f26baa7bf73d": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_5e4cc803b5e8419fbc29afee0dfe08d4", "placeholder": "\u200b", "style": "IPY_MODEL_84975bea9cb7440fa4c98c492c5103dd", "value": "100%"}}, "a5a4d31f7a5f4b8b898eba9cbc4c90a3": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_f9d67c931c5a4c5a930b5363f52256c0", "placeholder": "\u200b", "style": "IPY_MODEL_1c42120556214fdbbfd902358ae01291", "value": " 64/64 [00:26<00:00, 1.63it/s]"}}, "b0de1c8c26ba486c8d508521bbfc3ba5": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": ""}}, "b1441fbe34fc42188b8b6365b9938e6a": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "bc74b1c35a1e43729bcfbfd949a8bd65": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "bfa110cb409241698ee7d931dd684f70": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "c6bf0eebcf394a2cb9ee1dc7580d3afa": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "c74e75c66bfa4828a2227cf46d861496": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "c7f0837a85004ada9d89c3684517ebfc": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_e150742d5d4348c4ac877b3c34c8f916", "placeholder": "\u200b", "style": "IPY_MODEL_60b4448b21004a2e8adc58d233f19fd2", "value": "100%"}}, "cbaa303ed5e14448be2eaf1ef821cb2a": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_3e0bbd9c186c4fc3926f23ea8748c1b2", "max": 64.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_b1441fbe34fc42188b8b6365b9938e6a", "value": 64.0}}, "ce1ad59ac40645fc95ceed341a73818b": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_db22141e14264129bbffc39c1f1e5eae", "placeholder": "\u200b", "style": "IPY_MODEL_480e436a71444d538a13d939c22ba256", "value": " 27/28 [00:02<00:00, 8.49it/s]"}}, "cee09c9876fc4cc3b8b513adc415b4e8": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": ""}}, "db22141e14264129bbffc39c1f1e5eae": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "e150742d5d4348c4ac877b3c34c8f916": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "e72b83c7fe884736b4de3dca82aac052": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "ea8681142aa04f1f825a66012f606b1f": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "ed3634d2b5db41a9a4cc1222816852f5": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_496ed24561bc491f8709c04c88e7e35c", "placeholder": "\u200b", "style": "IPY_MODEL_cee09c9876fc4cc3b8b513adc415b4e8", "value": " 27/28 [00:02<00:00, 10.45it/s]"}}, "f9d67c931c5a4c5a930b5363f52256c0": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}}, "version_major": 2, "version_minor": 0}}}, "nbformat": 4, "nbformat_minor": 5}