{"cells": [{"cell_type": "markdown", "id": "10249518", "metadata": {"papermill": {"duration": 0.025611, "end_time": "2021-09-16T12:42:18.560094", "exception": false, "start_time": "2021-09-16T12:42:18.534483", "status": "completed"}, "tags": []}, "source": ["\n", "# Tutorial 10: Autoregressive Image Modeling\n", "\n", "* **Author:** Phillip Lippe\n", "* **License:** CC BY-SA\n", "* **Generated:** 2021-09-16T14:32:36.593971\n", "\n", "In this tutorial, we implement an autoregressive likelihood model for the task of image modeling.\n", "Autoregressive models are naturally strong generative models that constitute one of the current\n", "state-of-the-art architectures on likelihood-based image modeling,\n", "and are also the basis for large language generation models such as GPT3.\n", "We will focus on the PixelCNN architecture in this tutorial, and apply it to MNIST modeling.\n", "This notebook is part of a lecture series on Deep Learning at the University of Amsterdam.\n", "The full list of tutorials can be found at https://uvadlc-notebooks.rtfd.io.\n", "\n", "\n", "---\n", "Open in [![Open In Colab](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAHUAAAAUCAYAAACzrHJDAAAIuUlEQVRoQ+1ZaVRURxb+qhdolmbTUVSURpZgmLhHbQVFZIlGQBEXcMvJhKiTEzfigjQg7oNEJ9GMGidnjnNMBs2czIzajksEFRE1xklCTKJiQLRFsUGkoUWw+82pamn79etGYoKek1B/4NW99/tu3e/dquJBAGD27NkHALxKf39WY39gyrOi+i3xqGtUoePJrFmznrmgtModorbTu8YRNZk5cybXTvCtwh7o6NR2KzuZMWNGh6jtVt7nA0ymT5/eJlF9POrh7PAQl6s8bGYa3PUum//htmebVtLRqW0q01M5keTk5FZFzU0oRle3+zxwg5Hgtb+PZiL/ZVohxCI+hL5JgjmfjPxZ26+33BG3dA+ealHPM4gQAo5rU59gsI8bRvl54t3Ca62mvHyUAhtOlLd5WSQpKcluBjumnoCLs1EARkVd9E8l3p9y2i7RbQ1B6pFwu/YDgW8KbHJHMTQrwnjz2oZm9M4pavOCfo5jWrgCaaMVcMs6/pNhDr0+AMN93XlxV7R6DNpyzi7W/OE+yIrsjU6rTrbKV5cd/pNyItOmTbMp6sbBB+EqaYJY4cWE3VUciNt1TpgfcRFv71Fi54xT5kSoyLvOBEJMOMxWXkFlBeBSX4u6Zkcs+3KszYRtiapbNRqF31UgetVuc8z9vBXIv1qD+F1f83B6uDlCUyfsZGepGPpmg01OB7EITQbhS9ribKy+DmP1DUiClLz4bnIHVOqa7BY+Z1wg5g3zgUvyehiNpnJKxSLc/ts76LKm0BzX3c0RNy1yXjDcB5lWoro4iNHQxM+f1kWeWQARAWQS++trISJTp061Kep25X/MycwtjuctSC5rxo7ppi7VNUox5+PhPHtrsS2O1qJ6yx1QujQUzm9sh6hbkBlvvGcN8hYnwjUjH6kjfZEd5c/jitz5Jc5U3ENnFynKl4eB7nyEgP2UZ+Yz3/rVEbyYr27qELrtC4FIC0J7sc7xWnmccdHfRRTs0VB+cA4lt+oFcRR/wUeH8FG5w2Mbx8FQ8TXEvv1xYf4wBP3O2WyL3/UVjpXWgIqaFeUPr+wTmDvUB7njH6/bOv+HRg4SqioAg5GDe1aB3ZeMTJkyRSBqkLsWqSEm0fZVBEN94zEZnYvrdx1JL5cxe+a+AbhSJecRRHW/ikTFRTa38dtQlNZ5CRKwFvUtZU/kvBoEF9Uxni/XqIM+dwKbTw3rhcxIf7gmr2M+H6SMwx8iBzJbw5oxeG3Lv5FX9B3AGaHPS8e8z77H7v9VMpvPG5ug1enh7eGK8h0LBTwUb+GInqzInlRUK65DmTPQu4c3+uQKjwKK77zwUxBX4Tq7yR1RuiwUsqlrABCM6esHdXoy47fk4+prYKy8ZF574x4V5BnHQBuf4g9Z9ld8U36L2aktZNNplNfw7zotwWTy5MkCUft4aLEopJj5/OPHl1BQqeAVOnHgNSQOqmBzq9V9cfEm/yx5ubMGKS9cYPZ3vx2OS/c6PVHUuUO7Y1Pci3BO/1zgq18byebfGemLtNF+6JRtOvMk926ibussZqM+1mNz4TWkH7rCbM5phwGRGDAaoF8fY5OHFnlldAA8sgoEXKnDukA1NgSeNjqkJT9brbN4pC9WRweYXyLugR73c+MYvyWfu0yC6+mjzN1Isfw3FKJS98CU/zI1IHFkFPR52cHL2FJk0sB6kMTERIGo9GzcPkLNfA0cwdwi/hfEYO86ZMd9w+y1egfM2T2Eh/vesMNwljSzuZRT420SW3eqy8N6aHMmwmnFUZ7/PGVPbIoNZvNU1BURdHs0bT2+HjL8sDSM2e6vi4Lj5NW8WOLVA6RTT2azxLV+bglaFNqLieqemS/gWkw7NyoAHo+2dEsiivengjKsPFoqWOvbSh/kxPaxyW/JRzH2Fl3EzD9/xjAefJqB3usKUFn/0Gb+S/d/jy3FN2yLOmnSJJtn6oehByEiHPSeXnDxFGPRnoFoaBJjcdQlbDwcjL1zTNuQpoxD7R0OG0uUTMi0fkVwdzBdYIwcwZunxrVJVLplNm54BZp7jfDfYLoNyqQi1K6KxIdHzmN+QQ2WjFIwUT2zTGdlRXo4NFXVUO4sgX5dFC7f0aP/ZlNeUjFBuL8Xjl6uRuP6aMjSjpjzsH62FDU7JhBuGccEXIvDfJFFBc/gHw80dklfCVYnRaDfpiJcutPA4F7qJsfJeUPQI+1fqMlNhFx1FM0GDqkjFVg7NojlQ0Vt4aM5ReSqcbpaCg8nCW5lRsBvbT4T1TLfFptsfh7gItzuKTdJSEiwKSrt1vcmnEXXrsLbYnWDA1bu+z2WKy9Arq+1KRqdfKsoBo0GcdtEpS/B1bO4v0cFiUhkjskvKcMrWwtAPHuwQq8Z+4LZ1vTQANfXt4J0DwZX9gWa9qh4XDM/voC9JXfwYEMMHJcfNtusn82ihvliVUwg5KrPGVf6GH94ZJpEZBen6EC4qYTHA1dXhW0JIex8txzv//c8lhzXIi/BFxOH9jGbQhZsRalTIBZZ8KkGyZAxeRQvXkFF1TWz/Hm46jNYUnjPbt3JxIkT7f6dSj8qfJJyVvBxgaIlblOyjtysNHWN9fjjqWi7glJfW3/S0Hlj2XnA8PhKT9w6g3Qx3XiXhvuxQsuT1proxBKI/AaZqY1Xz5muvY8G8XkRRCaHsfQsRAFDH/tZPbcYuHotOG0FRIqB4HR3wNVoIPLtz8ycTguu+jpEigE218vd1YCr5m+HpHMvEI9u4LTXwNWaLjl0iPwGAmIpeHx1VeCqTJdPs1/vweweQPO3HC24NhOhnTphwoQnfv6QSY2ICbkNmdSA4h87oaLaiYfn5diIEd4att2erOwJXbPUHp953p6orQVSUVWRAXBT8c/dJ5L9xhzaJGp71GR/wFP8P5V2z10NSC9T93QM2xUg8fHxT+zU9ijeU4naHon8CjFJXFzc8/kn+dN06q9QgF98SYSo2Xen2NjYZy5sR6f+4nLSK5Iam2PH/x87a1YN/t5sBgAAAABJRU5ErkJggg==){height=\"20px\" width=\"117px\"}](https://colab.research.google.com/github/PytorchLightning/lightning-tutorials/blob/publication/.notebooks/course_UvA-DL/10-autoregressive-image-modeling.ipynb)\n", "\n", "Give us a \u2b50 [on Github](https://www.github.com/PytorchLightning/pytorch-lightning/)\n", "| Check out [the documentation](https://pytorch-lightning.readthedocs.io/en/latest/)\n", "| Join us [on Slack](https://join.slack.com/t/pytorch-lightning/shared_invite/zt-pw5v393p-qRaDgEk24~EjiZNBpSQFgQ)"]}, {"cell_type": "markdown", "id": "3109481c", "metadata": {"papermill": {"duration": 0.024038, "end_time": "2021-09-16T12:42:18.608085", "exception": false, "start_time": "2021-09-16T12:42:18.584047", "status": "completed"}, "tags": []}, "source": ["## Setup\n", "This notebook requires some packages besides pytorch-lightning."]}, {"cell_type": "code", "execution_count": 1, "id": "86088e31", "metadata": {"colab": {}, "colab_type": "code", "execution": {"iopub.execute_input": "2021-09-16T12:42:18.659637Z", "iopub.status.busy": "2021-09-16T12:42:18.659166Z", "iopub.status.idle": "2021-09-16T12:42:18.661686Z", "shell.execute_reply": "2021-09-16T12:42:18.661233Z"}, "id": "LfrJLKPFyhsK", "lines_to_next_cell": 0, "papermill": {"duration": 0.029126, "end_time": "2021-09-16T12:42:18.661813", "exception": false, "start_time": "2021-09-16T12:42:18.632687", "status": "completed"}, "tags": []}, "outputs": [], "source": ["# ! pip install --quiet \"torch>=1.6, <1.9\" \"seaborn\" \"torchvision\" \"matplotlib\" \"torchmetrics>=0.3\" \"pytorch-lightning>=1.3\""]}, {"cell_type": "markdown", "id": "e1dc997d", "metadata": {"papermill": {"duration": 0.024049, "end_time": "2021-09-16T12:42:18.710135", "exception": false, "start_time": "2021-09-16T12:42:18.686086", "status": "completed"}, "tags": []}, "source": ["
\n", "\n", "Similar to the language generation you have seen in assignment 2, autoregressive models work on images by modeling the likelihood of a pixel given all previous ones.\n", "For instance, in the picture below, we model the pixel $x_i$ as a conditional probability distribution\n", "based on all previous (here blue) pixels (figure credit - [Aaron van den Oord et al. ](https://arxiv.org/abs/1601.06759)):\n", "\n", "
\n", "\n", "Generally, autoregressive model over high-dimensional data $\\mathbf{x}$ factor the joint distribution as the following product of conditionals:\n", "\n", "$$p(\\mathbf{x})=p(x_1, ..., x_n)=\\prod_{i=1}^{n} p(x_i|x_1,...,x_{i-1})$$\n", "\n", "Learning these conditionals is often much simpler than learning the joint distribution $p(\\mathbf{x})$ all together.\n", "However, disadvantages of autoregressive models include slow sampling, especially for large images,\n", "as we need height-times-width forward passes through the model.\n", "In addition, for some applications, we require a latent space as modeled in VAEs and Normalizing Flows.\n", "For instance, in autoregressive models, we cannot interpolate between two images because of the lack of a latent representation.\n", "We will explore and discuss these benefits and drawbacks alongside with our implementation.\n", "\n", "Our implementation will focus on the [PixelCNN](https://arxiv.org/pdf/1606.05328.pdf) [2] model which has been discussed in detail in the lecture.\n", "Most current SOTA models use PixelCNN as their fundamental architecture,\n", "and various additions have been proposed to improve the performance\n", "(e.g. [PixelCNN++](https://arxiv.org/pdf/1701.05517.pdf) and [PixelSNAIL](http://proceedings.mlr.press/v80/chen18h/chen18h.pdf)).\n", "Hence, implementing PixelCNN is a good starting point for our short tutorial.\n", "\n", "First of all, we need to import our standard libraries. Similarly as in\n", "the last couple of tutorials, we will use [PyTorch\n", "Lightning](https://pytorch-lightning.readthedocs.io/en/latest/) here as\n", "well."]}, {"cell_type": "code", "execution_count": 2, "id": "6c3a3524", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:42:18.766239Z", "iopub.status.busy": "2021-09-16T12:42:18.759431Z", "iopub.status.idle": "2021-09-16T12:42:20.494992Z", "shell.execute_reply": "2021-09-16T12:42:20.494567Z"}, "papermill": {"duration": 1.761266, "end_time": "2021-09-16T12:42:20.495107", "exception": false, "start_time": "2021-09-16T12:42:18.733841", "status": "completed"}, "tags": []}, "outputs": [{"name": "stderr", "output_type": "stream", "text": ["/tmp/ipykernel_3486/3450944711.py:26: DeprecationWarning: `set_matplotlib_formats` is deprecated since IPython 7.23, directly use `matplotlib_inline.backend_inline.set_matplotlib_formats()`\n", " set_matplotlib_formats(\"svg\", \"pdf\") # For export\n", "Global seed set to 42\n"]}, {"name": "stdout", "output_type": "stream", "text": ["Using device cuda:0\n"]}, {"data": {"text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["\n", "import math\n", "import os\n", "import urllib.request\n", "from urllib.error import HTTPError\n", "\n", "# Imports for plotting\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "import pytorch_lightning as pl\n", "import seaborn as sns\n", "import torch\n", "import torch.nn as nn\n", "import torch.nn.functional as F\n", "import torch.optim as optim\n", "import torch.utils.data as data\n", "import torchvision\n", "from IPython.display import set_matplotlib_formats\n", "from matplotlib.colors import to_rgb\n", "from pytorch_lightning.callbacks import LearningRateMonitor, ModelCheckpoint\n", "from torchvision import transforms\n", "from torchvision.datasets import MNIST\n", "from tqdm.notebook import tqdm\n", "\n", "plt.set_cmap(\"cividis\")\n", "# %matplotlib inline\n", "set_matplotlib_formats(\"svg\", \"pdf\") # For export\n", "\n", "# Path to the folder where the datasets are/should be downloaded (e.g. MNIST)\n", "DATASET_PATH = os.environ.get(\"PATH_DATASETS\", \"data\")\n", "# Path to the folder where the pretrained models are saved\n", "CHECKPOINT_PATH = os.environ.get(\"PATH_CHECKPOINT\", \"saved_models/tutorial12\")\n", "\n", "# Setting the seed\n", "pl.seed_everything(42)\n", "\n", "# Ensure that all operations are deterministic on GPU (if used) for reproducibility\n", "torch.backends.cudnn.determinstic = True\n", "torch.backends.cudnn.benchmark = False\n", "\n", "# Fetching the device that will be used throughout this notebook\n", "device = torch.device(\"cpu\") if not torch.cuda.is_available() else torch.device(\"cuda:0\")\n", "print(\"Using device\", device)"]}, {"cell_type": "markdown", "id": "9527732b", "metadata": {"papermill": {"duration": 0.024895, "end_time": "2021-09-16T12:42:20.545620", "exception": false, "start_time": "2021-09-16T12:42:20.520725", "status": "completed"}, "tags": []}, "source": ["We again provide a pretrained model, which is downloaded below:"]}, {"cell_type": "code", "execution_count": 3, "id": "9b4b631a", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:42:20.600149Z", "iopub.status.busy": "2021-09-16T12:42:20.597718Z", "iopub.status.idle": "2021-09-16T12:42:20.772102Z", "shell.execute_reply": "2021-09-16T12:42:20.771611Z"}, "papermill": {"duration": 0.201736, "end_time": "2021-09-16T12:42:20.772216", "exception": false, "start_time": "2021-09-16T12:42:20.570480", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Downloading https://raw.githubusercontent.com/phlippe/saved_models/main/tutorial12/PixelCNN.ckpt...\n"]}], "source": ["# Github URL where saved models are stored for this tutorial\n", "base_url = \"https://raw.githubusercontent.com/phlippe/saved_models/main/tutorial12/\"\n", "# Files to download\n", "pretrained_files = [\"PixelCNN.ckpt\"]\n", "# Create checkpoint path if it doesn't exist yet\n", "os.makedirs(CHECKPOINT_PATH, exist_ok=True)\n", "\n", "# For each file, check whether it already exists. If not, try downloading it.\n", "for file_name in pretrained_files:\n", " file_path = os.path.join(CHECKPOINT_PATH, file_name)\n", " if not os.path.isfile(file_path):\n", " file_url = base_url + file_name\n", " print(\"Downloading %s...\" % file_url)\n", " try:\n", " urllib.request.urlretrieve(file_url, file_path)\n", " except HTTPError as e:\n", " print(\n", " \"Something went wrong. Please try to download the file from the GDrive folder, or contact the author with the full output including the following error:\\n\",\n", " e,\n", " )"]}, {"cell_type": "markdown", "id": "a675f0ec", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.024998, "end_time": "2021-09-16T12:42:20.823035", "exception": false, "start_time": "2021-09-16T12:42:20.798037", "status": "completed"}, "tags": []}, "source": ["Similar to the Normalizing Flows in Tutorial 11, we will work on the\n", "MNIST dataset and use 8-bits per pixel (values between 0 and 255). The\n", "dataset is loaded below:"]}, {"cell_type": "code", "execution_count": 4, "id": "26f824ca", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:42:20.878881Z", "iopub.status.busy": "2021-09-16T12:42:20.878404Z", "iopub.status.idle": "2021-09-16T12:42:20.910958Z", "shell.execute_reply": "2021-09-16T12:42:20.910469Z"}, "papermill": {"duration": 0.062918, "end_time": "2021-09-16T12:42:20.911075", "exception": false, "start_time": "2021-09-16T12:42:20.848157", "status": "completed"}, "tags": []}, "outputs": [{"name": "stderr", "output_type": "stream", "text": ["Global seed set to 42\n"]}], "source": ["# Convert images from 0-1 to 0-255 (integers). We use the long datatype as we will use the images as labels as well\n", "def discretize(sample):\n", " return (sample * 255).to(torch.long)\n", "\n", "\n", "# Transformations applied on each image => only make them a tensor\n", "transform = transforms.Compose([transforms.ToTensor(), discretize])\n", "\n", "# Loading the training dataset. We need to split it into a training and validation part\n", "train_dataset = MNIST(root=DATASET_PATH, train=True, transform=transform, download=True)\n", "pl.seed_everything(42)\n", "train_set, val_set = torch.utils.data.random_split(train_dataset, [50000, 10000])\n", "\n", "# Loading the test set\n", "test_set = MNIST(root=DATASET_PATH, train=False, transform=transform, download=True)\n", "\n", "# We define a set of data loaders that we can use for various purposes later.\n", "train_loader = data.DataLoader(train_set, batch_size=128, shuffle=True, drop_last=True, pin_memory=True, num_workers=4)\n", "val_loader = data.DataLoader(val_set, batch_size=128, shuffle=False, drop_last=False, num_workers=4)\n", "test_loader = data.DataLoader(test_set, batch_size=128, shuffle=False, drop_last=False, num_workers=4)"]}, {"cell_type": "markdown", "id": "8392a407", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.025246, "end_time": "2021-09-16T12:42:20.962362", "exception": false, "start_time": "2021-09-16T12:42:20.937116", "status": "completed"}, "tags": []}, "source": ["A good practice is to always visualize some data examples to get an intuition of the data:"]}, {"cell_type": "code", "execution_count": 5, "id": "4c29da69", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:42:21.018762Z", "iopub.status.busy": "2021-09-16T12:42:21.018292Z", "iopub.status.idle": "2021-09-16T12:42:21.101884Z", "shell.execute_reply": "2021-09-16T12:42:21.102269Z"}, "papermill": {"duration": 0.114672, "end_time": "2021-09-16T12:42:21.102399", "exception": false, "start_time": "2021-09-16T12:42:20.987727", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUgovR3JvdXAgPDwgL0NTIC9EZXZpY2VSR0IgL1MgL1RyYW5zcGFyZW5jeSAvVHlwZSAvR3JvdXAgPj4KL01lZGlhQm94IFsgMCAwIDMzNS4yOTkzNTQ4Mzg3IDE3Ny40OCBdIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovVHlwZSAvUGFnZSA+PgplbmRvYmoKOSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEyIDAgUiA+PgpzdHJlYW0KeJxNjrEOwjAMRHd/xX1BYsdpk45FlSLGsvABVQVUtAgqwe/jMgDDSb6T7XuCiXwrOK1gTKYXBAW+G5+XYTyUHYaV2PKZVCsXmkaryuz130pKLmbL+DeeiRa6I7nwkQZ2eduOWXOC1Oo44zHiiAW+DRuAGIAYAKPYpQbBVix1/H4ZZvi9oLuhp57eJg8oJwplbmRzdHJlYW0KZW5kb2JqCjEyIDAgb2JqCjE0MgplbmRvYmoKMTAgMCBvYmoKWyBdCmVuZG9iagozIDAgb2JqCjw8ID4+CmVuZG9iago0IDAgb2JqCjw8IC9BMSA8PCAvQ0EgMCAvVHlwZSAvRXh0R1N0YXRlIC9jYSAxID4+Ci9BMiA8PCAvQ0EgMSAvVHlwZSAvRXh0R1N0YXRlIC9jYSAxID4+ID4+CmVuZG9iago1IDAgb2JqCjw8ID4+CmVuZG9iago2IDAgb2JqCjw8ID4+CmVuZG9iago3IDAgb2JqCjw8IC9JMSAxMyAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9CaXRzUGVyQ29tcG9uZW50IDgKL0NvbG9yU3BhY2UgWy9JbmRleGVkIC9EZXZpY2VSR0IgMjIyICj////+/v79/f38/Pz7+/v6+vr5+fn4+Pj39/f19fX09PTz8/Py8vLx8fHw8PDv7+/u7u7t7e3s7Ozr6+vq6urp6eno6Ojn5+fm5ubl5eXk5OTj4+Pi4uLh4eHg4ODf39/e3t7d3d3c3Nzb29va2trZ2dnY2NjX19fW1tbV1dXU1NTT09PR0dHQ0NDPz8/Ozs7Nzc3MzMzLy8vJycnHx8fGxsbFxcXExMTDw8PCwsLBwcHAwMC/v7++vr69vb28vLy7u7u6urq5ubm4uLi3t7e2tra1tbW0tLSzs7OysrKwsLCvr6+urq6tra2srKyqqqqpqamoqKinp6elpaWkpKSioqKhoaGgoKCenp6cnJyampqZmZmYmJiXl5eWlpaVlZWUlJSTk5ORkZGPj4+NjY2Li4uKioqJiYmIiIiHh4eGhoaFhYWEhISDg4OCgoKAgIB+fn59fX18fHx7e3t5eXl4eHh3d3d2dnZ1dXV0dHRzc3NycnJxcXFwcHBvb29ubm5qampnZ2dmZmZkZGRjY2NiYmJgYGBfX19eXl5dXV1cXFxcXFxbW1taWlpZWVlYWFhXV1dWVlZVVVVUVFRTU1NRUVFQUFBPT09MTExLS0tKSkpJSUlISEhHR0dGRkZFRUVERERDQ0NCQkJBQUFAQEA/Pz8+Pj48PDw7Ozs6Ojo5OTk4ODg3Nzc1NTU0NDQzMzMyMjIxMTEwMDAvLy8uLi4sLCwrKysqKipcKFwoXCgnJycmJiYlJSUkJCQjIyMiIiIgICAfHx8eHh4dHR0cHBwbGxsaGhoZGRkYGBgXFxcWFhYVFRUUFBQTExMSEhIREREQEBAPDw8ODg5cclxyXHIMDAwLCwtcblxuXG4JCQkICAgHBwcGBgYFBQUEBAQDAwMCAgIBAQEAAAApXQovRGVjb2RlUGFybXMgPDwgL0NvbG9ycyAxIC9Db2x1bW5zIDMyMSAvUHJlZGljdG9yIDEwID4+Ci9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9IZWlnaHQgMTY0IC9MZW5ndGggMTQgMCBSIC9TdWJ0eXBlIC9JbWFnZQovVHlwZSAvWE9iamVjdCAvV2lkdGggMzIxID4+CnN0cmVhbQp4nO2d+19M+R/HZ6ZRKWFGJRoUWdEWspVLqnHLJSqXZV1qI7llv+7azTJZwmZlKXeKWLnLZWklconUisxmhPlbvu85M6dpTufymflMzNr384c8zmVe8/48M5/z+ZxzOiP7H0KH7HMX8K8HDdKCBmlBg7SYDRqdxn8vDg3SxqFB2jg0SBuHBmnj0CBt3JdhsLa2dvTo0adPOymOFDRIi6MGHz16VFrapUsXuVwuk0VHR3d4jVI76fVarRaqmTHDKXHkoEFa0CAtDhnctGnTpEmTFK3ExMR0eI2ie9TV1cXHyxny8ujj7AIN0oIGabHDoMGg1+uLi4ujojw9PRVt6d69+86dHVyj6B5Tpkxh9Pn6+hoM9HF2gQZpITX45MmTtp9bW8g/xfn5+TKZLCODZ5PB0NLSIlyjaHWDBg1iDO60+UUKIB23Z8+ekBBoFyQmJUHJr15JxaFBW9CgOJ/FYE7O5MmThfwpFAUFBUQ1vnoVHh4O++/fb7P62rVry5cvV6kiIiKKi4VqFKnOYhDG94cOHRLdTzqusXH27NnQz7uZgFKZf8PCSktLRePQYCvONvjxY319/dq1a728eLx17Xr+/PmbJj58+EBU49On5lfu2mU06YTsy5cvL1ni7+/PZmZl2dtkBsZgfHy86E5Ecfn5YMzPz++XXzZv3swadHPz8vI6d+6ccBwabAUNEvBJDer1vP3e1KlTU1NTL160t8b9+82v79EjPT29Xz9u7MiRI/V6e5tsNG7ZskWpVMrlixcvFt6JKO7x48chIeDroqlpVVVVB0xotVpGYlBQ0MuXQnFo0Awa/FcYhMFzXt6bN28cqlGvT0pKgkGHNa5z585abUZGBrOQxdsLShisqxs+fDgznK6trRWtSjpOp9MpFMuWLbMZ2sPCli09evSAAtPShOLQoBmnGmxuboZVzNyGRa2OjIx89uyZdSf4X/+S73+2aJPhAJ6b+5UJOMiXlJQYjXPnzlWYDu0wc7SzyUBhodyC2WBR0UETR44cEXqFQFxlZaVarQ4IgEZxN715M2DAAHiH1FShODRoRIOuZxAmgjb9X3BwcFkZ9H+72gJd2sSJE3ftOnr0KHmTuZw4oVKp4B2Ep7SicVFRrMFJkyZNnz7dzY1ZgKktLPAenQXilixZwkyBeV5RVmaeJP/2m1AcGjSiQVc36OMDs8L8/OTkZO7ohgEmtgcPkjaZyzffMH2suqZGaA9Cgzz06lVRUUEYl5KSAp50Ou56mPTPnAlbhg4d+u6dUHVo0Oh8g+YhJItSqdFo2o6E29G1a3l5OVGTuTAGly5dKrwHoUH4PQa0xcfHRyaD3oc70uaNa2zs06cPeGr/Bps2bWJmdcW8Z3/RIAsadD2DUJGILz7OnDlD1GQbioqKunUbNWpUU1OT8E5EBmNjX79+bbOlrKzMzw+2VFdXS8c9f24+lWpzuG1oaMjMdHd3Z7aIVocG0eB/1CCMNLp166ZQzJC4a5LI4E8/cbfU19eHhvr5+XEnusKjGWgFO/WtqSksLJTJZJbGZfDeJGCNQ4PGT2QQpkrDhg0rLHz69Cn4io1tsykg4PLly0RNtrJjxw7mtYcPHxbdj8hgQoLNpxgW4uPj/fzOnz9PGLdq1Sr4rHp7j2Xw94cF+P2mpo4YMcLNDQ0SxKFBVzPIOwVWqVS5JlauXMnd9MMPpE1maWiIiIiAV44eree9QkcY9+237IgafhHM+V+YyJaXjx8/HlYNHkweZzAYFi5cqFabj8mengkJCcxFdn+TzN27d4tWhwaNaND1DHLPsIoyZMiQykrSJrNYOkGFRCcoFffqFXsTNRAXF5eeDh7Y5Zwc++OYy+wHzp41Lz9/Dv9t1OoHDx6IVocG28Q5y+CVK1c0Go20vMDAwLNnDZwb6IkMJibCy3v37l1VJbWnRFxzc/PAgQM551Zh2KDT6d6/tz/Olt9/h4+06O0kaFAcNGh/nC2OGwROnTol4TArK+v48eOO1PjixYuvv4aEcePGSbdCOg4Ow4mJrDx3d/e9e4uKihyPawMzbThwQKo6NCgIlUGgvJzXXN++fc+YeMd78YqkxmvXrjE9AM+lNEfi7II8rrq6WqWCTzEadDQODdLGOcGgg0jHzZo1S6GIjY11UpxdkMfdvn2bmeKhQUfj0CBtnOsa/PPPTp06ma7jK9PT09vf8GhvnJOrs4IGaeNc1+CtW8whvV+/fvfv36ePsxPyuLdv3373nZeX1/XrUnFokB80SBvnuga/vDg0SBuHBmnj0CBtHBqkjcOnytOCBmlBg7SgQVrQIC1okBYczdDGoUHaODRIG4cGaePQIG0cGqSNQ4O0cWiQNg4N0sahQdq4T2rw3r17+SZ8fRUKBfvXk9ynCZPH2fDs2bPw8K1bt5JXd/PmzWHD5HJ5cPC0adM2btz4449Qmk6ny8iYMmVKoum2Tihx+vRFixY1NjYKx6FBNGjhizdYUjJ27NjQ0FBlKwrm7hmlUq1Wjx27evXq5mY74tqzfft2uVylUhFWd+zYsbCwsJ9/3rt3L+8bf/z48fXrkpKSDRtGjBjh7e2dklJfX88bhwbRoAVXNnjhwgXrwn7Og/WFa2xDWVmZWt1Gmo1BdiEzkzSOl4SEBLncw8ODsLrm5mapR8paePfuXW5uLvP3o7xxaFAKaoNRUXBEsi5qNBrII6mxDS9fvuzTBzzFxcXFt8Is9O/fnzXo4cFzJO0wg+QYDIYFCxagQXvjrKBBx+KsUBusqbExCPpslklrPHiQ96/tKyoqVq1aZekU16xZQxrHQa/Xx8TEmB7b2Et0P/sN3rkTHR0Nw+7Zs3kefYwGCXC+QTgWO2RQGDhQW6Z4jhs8efIk87ed27ZtE92PvLq3b99WVq5fv97Dw93dHUarf/8tFIcG+UGDtNU50WBuLhx+bdY43yB1P+hUgydOzJ8/v2fPnnI5/EhLu3v3rmgcGmwHGhThsxiMirKZFxsZgw4+gpsXZxhkv+mFyuCJE2lpacxf/UVFRSUmJp458573gSGcODTYSkcZTE7mroFBtfAJms9jMCIigt5gZKT5gSEpKW2ery0dhwZbQYMuaTAzM7P9iRhY2V4rUY28zJkzxzUM/vPP1atXV6xYERCgVCqDgoJ0uoaGBuk4NNgKGnRJg+0HzzAv1phITk6GjRqNzEKyRapdBpuakpKSmMvukODjw/MlzkRxpaVdunQBf76+Dx8+FN2TKK6uDkyuXLnS2zswMHDbNqFnExnRoBDOMwiyYGhpXYBPr6wVWIBVNTX7rZDXyHLlCnutzsfHZ8wYoRqlYgoKoCAwuHy51J52VXf3bqTp8CxyJwkaFAcNOhrHQm8QeruampoLF1hz0NXl5jJrLgifnSGvsaKiIjiYNThv3jwH41paWlJSQB/0WcInUeyvjqG+vj4yEvpY5gsyheLQoAhokKI6BjqDmZlgMMp0vR1+cOfBjhi8c+fOunXr8vJgHjxnDvzw9fVl/YWF8dwYJRHHAt0UM5geOnSo6H6EcVwaGgICAqKjo4Xj0KA4jhqsYa7QmUbMGs7pVTOc8/5SNRYXF8+bN3DgQN57t8LCwoSfSy3d5NOnT3t5gUEo9a+/RPfkiSP58vLs7Gwo9Y8/hOLQoBRoUCyu4wwy8kRutkxO5l48EaqRXcnOfq1PFWYXBg8enJ8v9EZEHVfv3mBw5syZYl98xxO3dOlSsbvQWGDMAPGmryvhj0ODUqBBwbgONSh1tzQzspaukaG2tjbYMnQWvo9648aNhHFc9u3b5+4OTRQa9QrHTZgwQau9LvqIRhOXLl2C+LVrheLQYMcYdJz2cdXV1bzSbBYcn9WFh4czw+kXL17YWd3Vq1djYz09PaOjMzIycnJyfv21rq6O+4qqqpCQECjw4kWhODSIBr98gy0tLZmZmVZpAwYM+P77Bw8eFBQUxMQoLYNqx+fF7qZekOjbYnjimprOnTuXnR0UFARdXWgoTNXHmNBqtXkm4uLiAgOhvg0bNgjHoUHXN2hkhlT+/gsXLgSD2dk3btywNmA3A82sLikpyXGDFh4+fHj48OFHj27dupWVlWXztVkzZoh8OSsaZEGDX77BLzoODdLGoUHaODRIG4cGaePQIG0cGqSNQ4O0cWiQNg4N0sahQdo4fKo8LWiQFjRICxqkBQ3S8n+rQtglCmVuZHN0cmVhbQplbmRvYmoKMTQgMCBvYmoKMzEyNAplbmRvYmoKMiAwIG9iago8PCAvQ291bnQgMSAvS2lkcyBbIDExIDAgUiBdIC9UeXBlIC9QYWdlcyA+PgplbmRvYmoKMTUgMCBvYmoKPDwgL0NyZWF0aW9uRGF0ZSAoRDoyMDIxMDkxNjE0NDIyMSswMicwMCcpCi9DcmVhdG9yIChNYXRwbG90bGliIHYzLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjQuMykgPj4KZW5kb2JqCnhyZWYKMCAxNgowMDAwMDAwMDAwIDY1NTM1IGYgCjAwMDAwMDAwMTYgMDAwMDAgbiAKMDAwMDAwNDkyNiAwMDAwMCBuIAowMDAwMDAwNjU5IDAwMDAwIG4gCjAwMDAwMDA2ODAgMDAwMDAgbiAKMDAwMDAwMDc3OSAwMDAwMCBuIAowMDAwMDAwODAwIDAwMDAwIG4gCjAwMDAwMDA4MjEgMDAwMDAgbiAKMDAwMDAwMDA2NSAwMDAwMCBuIAowMDAwMDAwNDAyIDAwMDAwIG4gCjAwMDAwMDA2MzkgMDAwMDAgbiAKMDAwMDAwMDIwOCAwMDAwMCBuIAowMDAwMDAwNjE5IDAwMDAwIG4gCjAwMDAwMDA4NTMgMDAwMDAgbiAKMDAwMDAwNDkwNSAwMDAwMCBuIAowMDAwMDA0OTg2IDAwMDAwIG4gCnRyYWlsZXIKPDwgL0luZm8gMTUgMCBSIC9Sb290IDEgMCBSIC9TaXplIDE2ID4+CnN0YXJ0eHJlZgo1MTQzCiUlRU9GCg==\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2021-09-16T14:42:21.065617\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.4.3, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}], "source": ["def show_imgs(imgs):\n", " num_imgs = imgs.shape[0] if isinstance(imgs, torch.Tensor) else len(imgs)\n", " nrow = min(num_imgs, 4)\n", " ncol = int(math.ceil(num_imgs / nrow))\n", " imgs = torchvision.utils.make_grid(imgs, nrow=nrow, pad_value=128)\n", " imgs = imgs.clamp(min=0, max=255)\n", " np_imgs = imgs.cpu().numpy()\n", " plt.figure(figsize=(1.5 * nrow, 1.5 * ncol))\n", " plt.imshow(np.transpose(np_imgs, (1, 2, 0)), interpolation=\"nearest\")\n", " plt.axis(\"off\")\n", " plt.show()\n", " plt.close()\n", "\n", "\n", "show_imgs([train_set[i][0] for i in range(8)])"]}, {"cell_type": "markdown", "id": "353ee95b", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.026067, "end_time": "2021-09-16T12:42:21.155368", "exception": false, "start_time": "2021-09-16T12:42:21.129301", "status": "completed"}, "tags": []}, "source": ["## Masked autoregressive convolutions\n", "\n", "The core module of PixelCNN is its masked convolutions.\n", "In contrast to language models, we don't apply an LSTM on each pixel one-by-one.\n", "This would be inefficient because images are grids instead of sequences.\n", "Thus, it is better to rely on convolutions that have shown great success in deep CNN classification models.\n", "\n", "Nevertheless, we cannot just apply standard convolutions without any changes.\n", "Remember that during training of autoregressive models, we want to use teacher forcing which both helps the model training, and significantly reduces the time needed for training.\n", "For image modeling, teacher forcing is implemented by using a training image as input to the model, and we want to obtain as output the prediction for each pixel based on *only* its predecessors.\n", "Thus, we need to ensure that the prediction for a specific pixel can only be influenced by its predecessors and not by its own value or any \"future\" pixels.\n", "For this, we apply convolutions with a mask.\n", "\n", "Which mask we use depends on the ordering of pixels we decide on, i.e. which is the first pixel we predict,\n", "which is the second one, etc.\n", "The most commonly used ordering is to denote the upper left pixel as the start pixel,\n", "and sort the pixels row by row, as shown in the visualization at the top of the tutorial.\n", "Thus, the second pixel is on the right of the first one (first row, second column),\n", "and once we reach the end of the row, we start in the second row, first column.\n", "If we now want to apply this to our convolutions, we need to ensure that the prediction of pixel 1\n", "is not influenced by its own \"true\" input, and all pixels on its right and in any lower row.\n", "In convolutions, this means that we want to set those entries of the weight matrix to zero that take pixels on the right and below into account.\n", "As an example for a 5x5 kernel, see a mask below (figure credit - [Aaron van den Oord](https://arxiv.org/pdf/1606.05328.pdf)):\n", "\n", "
\n", "\n", "Before looking into the application of masked convolutions in PixelCNN\n", "in detail, let's first implement a module that allows us to apply an\n", "arbitrary mask to a convolution:"]}, {"cell_type": "code", "execution_count": 6, "id": "7f373c01", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:42:21.215016Z", "iopub.status.busy": "2021-09-16T12:42:21.214539Z", "iopub.status.idle": "2021-09-16T12:42:21.216213Z", "shell.execute_reply": "2021-09-16T12:42:21.216588Z"}, "lines_to_next_cell": 2, "papermill": {"duration": 0.033688, "end_time": "2021-09-16T12:42:21.216710", "exception": false, "start_time": "2021-09-16T12:42:21.183022", "status": "completed"}, "tags": []}, "outputs": [], "source": ["class MaskedConvolution(nn.Module):\n", " def __init__(self, c_in, c_out, mask, **kwargs):\n", " \"\"\"Implements a convolution with mask applied on its weights.\n", "\n", " Args:\n", " c_in: Number of input channels\n", " c_out: Number of output channels\n", " mask: Tensor of shape [kernel_size_H, kernel_size_W] with 0s where\n", " the convolution should be masked, and 1s otherwise.\n", " kwargs: Additional arguments for the convolution\n", " \"\"\"\n", " super().__init__()\n", " # For simplicity: calculate padding automatically\n", " kernel_size = (mask.shape[0], mask.shape[1])\n", " dilation = 1 if \"dilation\" not in kwargs else kwargs[\"dilation\"]\n", " padding = tuple(dilation * (kernel_size[i] - 1) // 2 for i in range(2))\n", " # Actual convolution\n", " self.conv = nn.Conv2d(c_in, c_out, kernel_size, padding=padding, **kwargs)\n", "\n", " # Mask as buffer => it is no parameter but still a tensor of the module\n", " # (must be moved with the devices)\n", " self.register_buffer(\"mask\", mask[None, None])\n", "\n", " def forward(self, x):\n", " self.conv.weight.data *= self.mask # Ensures zero's at masked positions\n", " return self.conv(x)"]}, {"cell_type": "markdown", "id": "ab31353f", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.026241, "end_time": "2021-09-16T12:42:21.269129", "exception": false, "start_time": "2021-09-16T12:42:21.242888", "status": "completed"}, "tags": []}, "source": ["### Vertical and horizontal convolution stacks\n", "\n", "To build our own autoregressive image model, we could simply stack a few masked convolutions on top of each other.\n", "This was actually the case for the original PixelCNN model, discussed in the paper\n", "[Pixel Recurrent Neural Networks](https://arxiv.org/pdf/1601.06759.pdf), but this leads to a considerable issue.\n", "When sequentially applying a couple of masked convolutions, the receptive field of a pixel\n", "show to have a \"blind spot\" on the right upper side, as shown in the figure below\n", "(figure credit - [Aaron van den Oord et al. ](https://arxiv.org/pdf/1606.05328.pdf)):\n", "\n", "
\n", "\n", "Although a pixel should be able to take into account all other pixels above and left of it,\n", "a stack of masked convolutions does not allow us to look to the upper pixels on the right.\n", "This is because the features of the pixels above, which we use for convolution,\n", "do not contain any information of the pixels on the right of the same row.\n", "If they would, we would be \"cheating\" and actually looking into the future.\n", "To overcome this issue, van den Oord et.\n", "al [2] proposed to split the convolutions into a vertical and a horizontal stack.\n", "The vertical stack looks at all pixels above the current one, while the horizontal takes into account all on the left.\n", "While keeping both of them separate, we can actually look at the pixels on the right with the vertical stack without breaking any of our assumptions.\n", "The two convolutions are also shown in the figure above.\n", "\n", "Let us implement them here as follows:"]}, {"cell_type": "code", "execution_count": 7, "id": "08032e0c", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:42:21.326679Z", "iopub.status.busy": "2021-09-16T12:42:21.323108Z", "iopub.status.idle": "2021-09-16T12:42:21.329089Z", "shell.execute_reply": "2021-09-16T12:42:21.328692Z"}, "papermill": {"duration": 0.033791, "end_time": "2021-09-16T12:42:21.329186", "exception": false, "start_time": "2021-09-16T12:42:21.295395", "status": "completed"}, "tags": []}, "outputs": [], "source": ["class VerticalStackConvolution(MaskedConvolution):\n", " def __init__(self, c_in, c_out, kernel_size=3, mask_center=False, **kwargs):\n", " # Mask out all pixels below. For efficiency, we could also reduce the kernel\n", " # size in height, but for simplicity, we stick with masking here.\n", " mask = torch.ones(kernel_size, kernel_size)\n", " mask[kernel_size // 2 + 1 :, :] = 0\n", "\n", " # For the very first convolution, we will also mask the center row\n", " if mask_center:\n", " mask[kernel_size // 2, :] = 0\n", "\n", " super().__init__(c_in, c_out, mask, **kwargs)\n", "\n", "\n", "class HorizontalStackConvolution(MaskedConvolution):\n", " def __init__(self, c_in, c_out, kernel_size=3, mask_center=False, **kwargs):\n", " # Mask out all pixels on the left. Note that our kernel has a size of 1\n", " # in height because we only look at the pixel in the same row.\n", " mask = torch.ones(1, kernel_size)\n", " mask[0, kernel_size // 2 + 1 :] = 0\n", "\n", " # For the very first convolution, we will also mask the center pixel\n", " if mask_center:\n", " mask[0, kernel_size // 2] = 0\n", "\n", " super().__init__(c_in, c_out, mask, **kwargs)"]}, {"cell_type": "markdown", "id": "4d152ec5", "metadata": {"papermill": {"duration": 0.026372, "end_time": "2021-09-16T12:42:21.381597", "exception": false, "start_time": "2021-09-16T12:42:21.355225", "status": "completed"}, "tags": []}, "source": ["Note that we have an input argument called `mask_center`. Remember that\n", "the input to the model is the actual input image. Hence, the very first\n", "convolution we apply cannot use the center pixel as input, but must be\n", "masked. All consecutive convolutions, however, should use the center\n", "pixel as we otherwise lose the features of the previous layer. Hence,\n", "the input argument `mask_center` is True for the very first\n", "convolutions, and False for all others."]}, {"cell_type": "markdown", "id": "cd8536aa", "metadata": {"papermill": {"duration": 0.026279, "end_time": "2021-09-16T12:42:21.434188", "exception": false, "start_time": "2021-09-16T12:42:21.407909", "status": "completed"}, "tags": []}, "source": ["### Visualizing the receptive field\n", "\n", "To validate our implementation of masked convolutions, we can visualize the receptive field we obtain with such convolutions.\n", "We should see that with increasing number of convolutional layers, the receptive field grows in both vertical and horizontal direction, without the issue of a blind spot.\n", "The receptive field can be empirically measured by backpropagating an arbitrary loss for the output features of a speicifc pixel with respect to the input.\n", "We implement this idea below, and visualize the receptive field below."]}, {"cell_type": "code", "execution_count": 8, "id": "52423559", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:42:21.493549Z", "iopub.status.busy": "2021-09-16T12:42:21.493069Z", "iopub.status.idle": "2021-09-16T12:42:21.641737Z", "shell.execute_reply": "2021-09-16T12:42:21.641316Z"}, "papermill": {"duration": 0.181442, "end_time": "2021-09-16T12:42:21.641845", "exception": false, "start_time": "2021-09-16T12:42:21.460403", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUgovR3JvdXAgPDwgL0NTIC9EZXZpY2VSR0IgL1MgL1RyYW5zcGFyZW5jeSAvVHlwZSAvR3JvdXAgPj4KL01lZGlhQm94IFsgMCAwIDM0OS4yIDE4MS43MDY4MTgxODE4IF0gL1BhcmVudCAyIDAgUiAvUmVzb3VyY2VzIDggMCBSCi9UeXBlIC9QYWdlID4+CmVuZG9iago5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTIgMCBSID4+CnN0cmVhbQp4nJ1RPW/CMBDd71e8EYY6dzYmzgiijdqNKhJD1aEKBoKANgFR9d/3QqHfQ6mt033fPT8LlpQMBPMtGEuVZwhyJKO4r8p4mw9Rbok1vibXy4xVa3W0JIhJuR8kaIi/uguiDdVItawV8dZo/ni/u03EBBskA9vCEIUhCoOR6wTxDu36Vp+mlWsk14LRI8Y0Rn1qZMw/N7c+1SSqL7Qfwoad46zvM84gfW/cB4RyTcMCyZWWWRSzw4uLKd2hM+nCB+Mzl3p0YjVf7OIUTReWTXCno5kyPu2qfcSsiqtpF/cobuiyoDeEEjLzvuzflNBPSn4ZfGDHnsuOZf09b0OPlR35Oz3DavPQvJxHCL0C1xOKfAplbmRzdHJlYW0KZW5kb2JqCjEyIDAgb2JqCjI3MgplbmRvYmoKMTAgMCBvYmoKWyBdCmVuZG9iagoxOSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDI2NCA+PgpzdHJlYW0KeJw9UrmRAzEMy7cKlsBfUj2+uXFg958ewD07MTFLEQBB925RORs/bSXLj/zYZWdJ5Jb3oG3yuqLqBqmbIHPJcckVYpbyuBIkFi1lJtZnqoPycQ1qFb7wEzMT0yFJxBJyUo8irI+vg9f1HNxfN+n8GhkfdGxQekuSq6BUw75ytBI7lupdg+yDppvS6jPTruyApfGGrNSkTn8d9b8jLMKk3khFByEWv9PLHbIspBzU27l+A+Fd7YJYT6087BBp3lZ6SxXM5swETBltO6yAtVljwlQJ8BbNIdRaiMwXOq2I+eTc0cE0VXkaIsNShYPtPaM1XOgaEkvD+UnGBOa/8PqsyG1//wBwaGe6CmVuZHN0cmVhbQplbmRvYmoKMjAgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA5MiA+PgpzdHJlYW0KeJw9jcENwDAIA/9MwQgQAsT7VFUf6f7fJhHqBx8G2RhgYbM14MHZwJfS2je9pEWT2ghWtUXdUJ67FKVYXUelTMJPmTt/UnQc7XAO29/W5ThN4+hf99D9AQ9KHgsKZW5kc3RyZWFtCmVuZG9iagoyMSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDMwNyA+PgpzdHJlYW0KeJw9kktuAzEMQ/c+hS4QwPrZnvOkKLqY3n/bJyXpihzZFkVqlrpMWVMekDSThH/p8HCxnfI7bM9mZuBaopeJ5ZTn0BVi7qJ82cxGXVknxeqEZjq36FE5Fwc2Taqfqyyl3S54Dtcmnlv2ET+80KAe1DUuCTd0V6NlKTRjqvt/0nv8jDLgakxdbFKrex88XkRV6OgHR4kiY5cX5+NBCelKwmhaiJV3RQNB7vK0ynsJ7tveasiyB6mYzjspZrDrdFIubheHIR7I8qjw5aPYa0LP+LArJfRI2IYzcifuaMbm1MjikP7ejQRLj65oIfPgr27WLmC8UzpFYmROcqxpi1VO91AU07nDvQwQ9WxFQylzkdXqX8POC2uWbBZ4SvoFHqPdJksOVtnbqE7vrTzZ0PcfWtd0HwplbmRzdHJlYW0KZW5kb2JqCjIyIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjMyID4+CnN0cmVhbQp4nDVRSW7EMAy7+xX8wADW7rwnxaCH9v/XUsoUCEAltrglYmMjAi8x+DmI3PiSNaMmfmdyV/wsT4VHwq3gSRSBl+FedoLLG8ZlPw4zH7yXVs6kxpMMyEU2PTwRMtglEDowuwZ12Gbaib4h4bMjUs1GltPXEvTSKgTKU7bf6YISbav6c/usC2372hNOdnvqSeUTiOeWrMBl4xWTxVgGPVG5SzF9kOpsoSehvCifg2w+aohElyhn4InBwSjQDuy57WfiVSFoXd2nbWOoRkrH078NTU2SCPlECWe2NO4W/n/Pvb7X+w9OIVQRCmVuZHN0cmVhbQplbmRvYmoKMjMgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMzEgPj4Kc3RyZWFtCnicNU85kgQhDMt5hT4wVRjbQL+np7Y22Pl/upKZTpDwIcnTEx2ZeJkjI7Bmx9taZCBm4FNMxb/2tA8TqvfgHiKUiwthhpFw1qzjbp6OF/92lc9YB+82+IpZXhDYwkzWVxZnLtsFY2mcxDnJboxdE7GNda2nU1hHMKEMhHS2w5Qgc1Sk9MmOMuboOJEnnovv9tssdjl+DusLNo0hFef4KnqCNoOi7HnvAhpyQf9d3fgeRbvoJSAbCRbWUWLunOWEX712dB61KBJzQppBLhMhzekqphCaUKyzo6BSUXCpPqforJ9/5V9cLQplbmRzdHJlYW0KZW5kb2JqCjI0IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjQ5ID4+CnN0cmVhbQp4nD1QO45EIQzrOYUv8CTyI3AeRqstZu/frgOaKVBMfrYzJNARgUcMMZSv4yWtoK6Bv4tC8W7i64PCIKtDUiDOeg+IdOymNpETOh2cMz9hN2OOwEUxBpzpdKY9ByY5+8IKhHMbZexWSCeJqiKO6jOOKZ4qe594FiztyDZbJ5I95CDhUlKJyaWflMo/bcqUCjpm0QQsErngZBNNOMu7SVKMGZQy6h6mdiJ9rDzIozroZE3OrCOZ2dNP25n4HHC3X9pkTpXHdB7M+Jy0zoM5Fbr344k2B02N2ujs9xNpKi9Sux1anX51EpXdGOcYEpdnfxnfZP/5B/6HWiIKZW5kc3RyZWFtCmVuZG9iagoyNSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEzNiA+PgpzdHJlYW0KeJxNj0EOAzEIA+95hZ9AIEB4z1ZVD9v/X0vYdtMLHsmAbFEGgSWHeIcb4dHbD99FNhVn45xfUiliIZhPcJ8wUxyNKXfyY4+AcZRqLKdoeF5Lzk3DFy13Ey2lrZeTGW+47pf3R5VtkQ1Fzy0LQtdskvkygQd8GJhHdeNppcfd9myv9vwAzmw0SQplbmRzdHJlYW0KZW5kb2JqCjI2IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzQxID4+CnN0cmVhbQp4nEVSS25EMQjbv1NwgUjhl5DztKq6mN5/W5tM1c3gCWBseMtTpmTKsLklIyTXlE99IkOspvw0ciQipvhJCQV2lY/Ha0usjeyRqBSf2vHjsfRGptkVWvXu0aXNolHNysg5yBChnhW6snvUDtnwelxIuu+UzSEcy/9QgSxl3XIKJUFb0HfsEd8PHa6CK4JhsGsug+1lMtT/+ocWXO9992LHLoAWrOe+wQ4AqKcTtAXIGdruNiloAFW6i0nCo/J6bnaibKNV6fkcADMOMHLAiCVbHb7R3gCWfV3oRY2K/StAUVlA/MjVdsHeMclIcBbmBo69cDzFmXBLOMYCQIq94hh68CXY5i9Xroia8Al1umQvvMKe2ubnQpMId60ADl5kw62ro6iW7ek8gvZnRXJGjNSLODohklrSOYLi0qAeWuNcN7HibSOxuVff7h/hnC9c9usXS+yExAplbmRzdHJlYW0KZW5kb2JqCjI3IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTY0ID4+CnN0cmVhbQp4nEWQx3EFMQxD76oCJTCACvWsx/MP6/6vhvTTQXoYQgxiT8KwXFdxYXTDj7ctMw1/RxnuxvoyY7zVWCAn6AMMkYmr0aT6dsUZqvTk1WKuo6JcLzoiEsyS46tAI3w6sseTtrYz/XReH+wh7xP/KirnbmEBLqruQPlSH/HUj9lR6pqhjyorax5q2leEXRFK2z4upzJO3b0DWuG9las92u8/HnY68gplbmRzdHJlYW0KZW5kb2JqCjI4IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNzIgPj4Kc3RyZWFtCnicMzK3UDBQsDQBEoYWJgrmZgYKKYZcQL6piblCLhdIDMTKAbMMgLQlnIKIZ4CYIG0QxSAWRLGZiRlEHZwBkcvgSgMAJdsWyQplbmRzdHJlYW0KZW5kb2JqCjI5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNDcgPj4Kc3RyZWFtCnicMzK3UDBQsDQBEoYWJgrmZgYKKYZclhBWLhdMLAfMAtGWcAoinsGVBgC5Zw0nCmVuZHN0cmVhbQplbmRvYmoKMzAgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNjMgPj4Kc3RyZWFtCnicRZA7EgMhDEN7TqEj+CMDPs9mMik2929j2GxSwNNYIIO7E4LU2oKJ6IKHtiXdBe+tBGdj/Ok2bjUS5AR1gFak42iUUn25xWmVdPFoNnMrC60THWYOepSjGaAQOhXe7aLkcqbuzvlDcPVf9b9i3TmbiYHJyh0IzepT3Pk2O6K6usn+pMfcrNd+K+xVYWlZS8sJt527ZkAJ3FM52qs9Px8KOvYKZW5kc3RyZWFtCmVuZG9iagozMSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIzOSA+PgpzdHJlYW0KeJxNUMltBDEM+7sKNTDA6By7HgeLPLL9f0PKCZKXaEviofKUW5bKZfcjOW/JuuVDh06VafJu0M2vsf6jDAJ2/1BUEK0lsUrMXNJusTRJL9nDOI2Xa7WO56l7hFmjePDj2NMpgek9MsFms705MKs9zg6QTrjGr+rTO5UkA4m6kPNCpQrrHtQloo8r25hSnU4t5RiXn+h7fI4APcXejdzRx8sXjEa1LajRapU4DzATU9GVcauRgZQTBkNnR1c0C6XIynpCNcKNOaGZvcNwYAPLs4Skpa1SvA9lAegCXdo64zRKgo4Awt8ojPX6Bqr8XjcKZW5kc3RyZWFtCmVuZG9iagozMiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE2MCA+PgpzdHJlYW0KeJxFkDkSAzEIBHO9gidIXIL3rMu1wfr/qQfWR6LpAjQcuhZNynoUaD7psUahutBr6CxKkkTBFpIdUKdjiDsoSExIY5JIth6DI5pYs12YmVQqs1LhtGnFwr/ZWtXIRI1wjfyJ6QZU/E/qXJTwTYOvkjH6GFS8O4OMSfheRdxaMe3+RDCxGfYJb0UmBYSJsanZvs9ghsz3Ctc4x/MNTII36wplbmRzdHJlYW0KZW5kb2JqCjMzIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTggPj4Kc3RyZWFtCnicMza0UDCAwxRDrjQAHeYDUgplbmRzdHJlYW0KZW5kb2JqCjM0IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTMzID4+CnN0cmVhbQp4nEWPSw4EIQhE95yijsDHH+dxMumFc//tgJ1uE2M9hVSBuYKhPS5rA50VHyEZtvG3qZaORVk+VHpSVg/J4Iesxssh3KAs8IJJKoYhUIuYGpEtZW63gNs2DbKylVOljrCLozCP9rRsFR5folsidZI/g8QqL9zjuh3Ipda73qKLvn+kATEJCmVuZHN0cmVhbQplbmRvYmoKMzUgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA3NSA+PgpzdHJlYW0KeJwztTRSMFAwNgASpmZGCqYm5gophlxAPoiVy2VoZApm5XAZWZopWFgAGSZm5lAhmIYcLmNTc6ABQEXGpmAaqj+HK4MrDQCVkBLvCmVuZHN0cmVhbQplbmRvYmoKMzYgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNDEgPj4Kc3RyZWFtCnicPY/BDsMwCEPv+Qr/QKTYKaF8T6dqh+7/ryNLuwt6AmOMhdDQG6qaw4Zgm+PF0iVUa/gUxUAlN8iZYA6lpNIdR5F6YjgYXB60G47isej6EbuSZn3QxkK6JWiAe6xTadymcRPEHTUF6inqnKO8ELmfqWfYNJLdNLOSc7gNv3vPU9f/p6u8y/kFvXcu/gplbmRzdHJlYW0KZW5kb2JqCjE3IDAgb2JqCjw8IC9CYXNlRm9udCAvRGVqYVZ1U2FucyAvQ2hhclByb2NzIDE4IDAgUgovRW5jb2RpbmcgPDwKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDY2IC9CIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAxMTAgL24gMTEyIC9wCjExNCAvciAxMTYgL3QgMTE4IC92IDEyMSAveSBdCi9UeXBlIC9FbmNvZGluZyA+PgovRmlyc3RDaGFyIDAgL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvRm9udERlc2NyaXB0b3IgMTYgMCBSCi9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdIC9MYXN0Q2hhciAyNTUgL05hbWUgL0RlamFWdVNhbnMKL1N1YnR5cGUgL1R5cGUzIC9UeXBlIC9Gb250IC9XaWR0aHMgMTUgMCBSID4+CmVuZG9iagoxNiAwIG9iago8PCAvQXNjZW50IDkyOSAvQ2FwSGVpZ2h0IDAgL0Rlc2NlbnQgLTIzNiAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvRm9udE5hbWUgL0RlamFWdVNhbnMgL0l0YWxpY0FuZ2xlIDAKL01heFdpZHRoIDEzNDIgL1N0ZW1WIDAgL1R5cGUgL0ZvbnREZXNjcmlwdG9yIC9YSGVpZ2h0IDAgPj4KZW5kb2JqCjE1IDAgb2JqClsgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAKNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCAzMTggNDAxIDQ2MCA4MzggNjM2Cjk1MCA3ODAgMjc1IDM5MCAzOTAgNTAwIDgzOCAzMTggMzYxIDMxOCAzMzcgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNgo2MzYgNjM2IDMzNyAzMzcgODM4IDgzOCA4MzggNTMxIDEwMDAgNjg0IDY4NiA2OTggNzcwIDYzMiA1NzUgNzc1IDc1MiAyOTUKMjk1IDY1NiA1NTcgODYzIDc0OCA3ODcgNjAzIDc4NyA2OTUgNjM1IDYxMSA3MzIgNjg0IDk4OSA2ODUgNjExIDY4NSAzOTAgMzM3CjM5MCA4MzggNTAwIDUwMCA2MTMgNjM1IDU1MCA2MzUgNjE1IDM1MiA2MzUgNjM0IDI3OCAyNzggNTc5IDI3OCA5NzQgNjM0IDYxMgo2MzUgNjM1IDQxMSA1MjEgMzkyIDYzNCA1OTIgODE4IDU5MiA1OTIgNTI1IDYzNiAzMzcgNjM2IDgzOCA2MDAgNjM2IDYwMCAzMTgKMzUyIDUxOCAxMDAwIDUwMCA1MDAgNTAwIDEzNDIgNjM1IDQwMCAxMDcwIDYwMCA2ODUgNjAwIDYwMCAzMTggMzE4IDUxOCA1MTgKNTkwIDUwMCAxMDAwIDUwMCAxMDAwIDUyMSA0MDAgMTAyMyA2MDAgNTI1IDYxMSAzMTggNDAxIDYzNiA2MzYgNjM2IDYzNiAzMzcKNTAwIDUwMCAxMDAwIDQ3MSA2MTIgODM4IDM2MSAxMDAwIDUwMCA1MDAgODM4IDQwMSA0MDEgNTAwIDYzNiA2MzYgMzE4IDUwMAo0MDEgNDcxIDYxMiA5NjkgOTY5IDk2OSA1MzEgNjg0IDY4NCA2ODQgNjg0IDY4NCA2ODQgOTc0IDY5OCA2MzIgNjMyIDYzMiA2MzIKMjk1IDI5NSAyOTUgMjk1IDc3NSA3NDggNzg3IDc4NyA3ODcgNzg3IDc4NyA4MzggNzg3IDczMiA3MzIgNzMyIDczMiA2MTEgNjA1CjYzMCA2MTMgNjEzIDYxMyA2MTMgNjEzIDYxMyA5ODIgNTUwIDYxNSA2MTUgNjE1IDYxNSAyNzggMjc4IDI3OCAyNzggNjEyIDYzNAo2MTIgNjEyIDYxMiA2MTIgNjEyIDgzOCA2MTIgNjM0IDYzNCA2MzQgNjM0IDU5MiA2MzUgNTkyIF0KZW5kb2JqCjE4IDAgb2JqCjw8IC9CIDE5IDAgUiAvVyAyMCAwIFIgL2EgMjEgMCBSIC9jIDIyIDAgUiAvZCAyMyAwIFIgL2UgMjQgMCBSIC9mIDI1IDAgUgovZyAyNiAwIFIgL2ggMjcgMCBSIC9pIDI4IDAgUiAvbCAyOSAwIFIgL24gMzAgMCBSIC9wIDMxIDAgUiAvciAzMiAwIFIKL3NwYWNlIDMzIDAgUiAvdCAzNCAwIFIgL3YgMzUgMCBSIC95IDM2IDAgUiA+PgplbmRvYmoKMyAwIG9iago8PCAvRjEgMTcgMCBSID4+CmVuZG9iago0IDAgb2JqCjw8IC9BMSA8PCAvQ0EgMCAvVHlwZSAvRXh0R1N0YXRlIC9jYSAxID4+Ci9BMiA8PCAvQ0EgMSAvVHlwZSAvRXh0R1N0YXRlIC9jYSAxID4+ID4+CmVuZG9iago1IDAgb2JqCjw8ID4+CmVuZG9iago2IDAgb2JqCjw8ID4+CmVuZG9iago3IDAgb2JqCjw8IC9JMSAxMyAwIFIgL0kyIDE0IDAgUiA+PgplbmRvYmoKMTMgMCBvYmoKPDwgL0JpdHNQZXJDb21wb25lbnQgOCAvQ29sb3JTcGFjZSBbL0luZGV4ZWQgL0RldmljZVJHQiAxICj95zcAIk0pXQovRGVjb2RlUGFybXMgPDwgL0NvbG9ycyAxIC9Db2x1bW5zIDE1MyAvUHJlZGljdG9yIDEwID4+Ci9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9IZWlnaHQgMTUzIC9MZW5ndGggMzcgMCBSIC9TdWJ0eXBlIC9JbWFnZQovVHlwZSAvWE9iamVjdCAvV2lkdGggMTUzID4+CnN0cmVhbQp4nO3OsQnAQAzAQGf/pdOmSSF4eBenAcTNs7W5DfiNrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWOySbb4eWhzZkfUPWN2R9Q9Y3ZH1DtiGyHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHllvr+wFBs9ayQplbmRzdHJlYW0KZW5kb2JqCjM3IDAgb2JqCjE4OAplbmRvYmoKMTQgMCBvYmoKPDwgL0JpdHNQZXJDb21wb25lbnQgOCAvQ29sb3JTcGFjZSBbL0luZGV4ZWQgL0RldmljZVJHQiAxICj95zcAIk0pXQovRGVjb2RlUGFybXMgPDwgL0NvbG9ycyAxIC9Db2x1bW5zIDE1MyAvUHJlZGljdG9yIDEwID4+Ci9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9IZWlnaHQgMTUzIC9MZW5ndGggMzggMCBSIC9TdWJ0eXBlIC9JbWFnZQovVHlwZSAvWE9iamVjdCAvV2lkdGggMTUzID4+CnN0cmVhbQp4nO3OsQnAQAzAQGf/pdOmSSF4eBenAcTNs7W5DfiNrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWOySbb4eWhzZkfUPWN2R9Q9Y3ZH1DtiGyHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHllvr+wFBs9ayQplbmRzdHJlYW0KZW5kb2JqCjM4IDAgb2JqCjE4OAplbmRvYmoKMiAwIG9iago8PCAvQ291bnQgMSAvS2lkcyBbIDExIDAgUiBdIC9UeXBlIC9QYWdlcyA+PgplbmRvYmoKMzkgMCBvYmoKPDwgL0NyZWF0aW9uRGF0ZSAoRDoyMDIxMDkxNjE0NDIyMSswMicwMCcpCi9DcmVhdG9yIChNYXRwbG90bGliIHYzLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjQuMykgPj4KZW5kb2JqCnhyZWYKMCA0MAowMDAwMDAwMDAwIDY1NTM1IGYgCjAwMDAwMDAwMTYgMDAwMDAgbiAKMDAwMDAwODE0OCAwMDAwMCBuIAowMDAwMDA3MDE0IDAwMDAwIG4gCjAwMDAwMDcwNDYgMDAwMDAgbiAKMDAwMDAwNzE0NSAwMDAwMCBuIAowMDAwMDA3MTY2IDAwMDAwIG4gCjAwMDAwMDcxODcgMDAwMDAgbiAKMDAwMDAwMDA2NSAwMDAwMCBuIAowMDAwMDAwNDAxIDAwMDAwIG4gCjAwMDAwMDA3NjggMDAwMDAgbiAKMDAwMDAwMDIwOCAwMDAwMCBuIAowMDAwMDAwNzQ4IDAwMDAwIG4gCjAwMDAwMDcyMzAgMDAwMDAgbiAKMDAwMDAwNzY4OSAwMDAwMCBuIAowMDAwMDA1NzU1IDAwMDAwIG4gCjAwMDAwMDU1NTUgMDAwMDAgbiAKMDAwMDAwNTE2MSAwMDAwMCBuIAowMDAwMDA2ODA4IDAwMDAwIG4gCjAwMDAwMDA3ODggMDAwMDAgbiAKMDAwMDAwMTEyNSAwMDAwMCBuIAowMDAwMDAxMjg5IDAwMDAwIG4gCjAwMDAwMDE2NjkgMDAwMDAgbiAKMDAwMDAwMTk3NCAwMDAwMCBuIAowMDAwMDAyMjc4IDAwMDAwIG4gCjAwMDAwMDI2MDAgMDAwMDAgbiAKMDAwMDAwMjgwOSAwMDAwMCBuIAowMDAwMDAzMjIzIDAwMDAwIG4gCjAwMDAwMDM0NjAgMDAwMDAgbiAKMDAwMDAwMzYwNCAwMDAwMCBuIAowMDAwMDAzNzIzIDAwMDAwIG4gCjAwMDAwMDM5NTkgMDAwMDAgbiAKMDAwMDAwNDI3MSAwMDAwMCBuIAowMDAwMDA0NTA0IDAwMDAwIG4gCjAwMDAwMDQ1OTQgMDAwMDAgbiAKMDAwMDAwNDgwMCAwMDAwMCBuIAowMDAwMDA0OTQ3IDAwMDAwIG4gCjAwMDAwMDc2NjkgMDAwMDAgbiAKMDAwMDAwODEyOCAwMDAwMCBuIAowMDAwMDA4MjA4IDAwMDAwIG4gCnRyYWlsZXIKPDwgL0luZm8gMzkgMCBSIC9Sb290IDEgMCBSIC9TaXplIDQwID4+CnN0YXJ0eHJlZgo4MzY1CiUlRU9GCg==\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2021-09-16T14:42:21.570580\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.4.3, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}], "source": ["inp_img = torch.zeros(1, 1, 11, 11)\n", "inp_img.requires_grad_()\n", "\n", "\n", "def show_center_recep_field(img, out):\n", " \"\"\"Calculates the gradients of the input with respect to the output center pixel, and visualizes the overall\n", " receptive field.\n", "\n", " Args:\n", " img: Input image for which we want to calculate the receptive field on.\n", " out: Output features/loss which is used for backpropagation, and should be\n", " the output of the network/computation graph.\n", " \"\"\"\n", " # Determine gradients\n", " loss = out[0, :, img.shape[2] // 2, img.shape[3] // 2].sum() # L1 loss for simplicity\n", " # Retain graph as we want to stack multiple layers and show the receptive field of all of them\n", " loss.backward(retain_graph=True)\n", " img_grads = img.grad.abs()\n", " img.grad.fill_(0) # Reset grads\n", "\n", " # Plot receptive field\n", " img = img_grads.squeeze().cpu().numpy()\n", " fig, ax = plt.subplots(1, 2)\n", " _ = ax[0].imshow(img)\n", " ax[1].imshow(img > 0)\n", " # Mark the center pixel in red if it doesn't have any gradients (should be\n", " # the case for standard autoregressive models)\n", " show_center = img[img.shape[0] // 2, img.shape[1] // 2] == 0\n", " if show_center:\n", " center_pixel = np.zeros(img.shape + (4,))\n", " center_pixel[center_pixel.shape[0] // 2, center_pixel.shape[1] // 2, :] = np.array([1.0, 0.0, 0.0, 1.0])\n", " for i in range(2):\n", " ax[i].axis(\"off\")\n", " if show_center:\n", " ax[i].imshow(center_pixel)\n", " ax[0].set_title(\"Weighted receptive field\")\n", " ax[1].set_title(\"Binary receptive field\")\n", " plt.show()\n", " plt.close()\n", "\n", "\n", "show_center_recep_field(inp_img, inp_img)"]}, {"cell_type": "markdown", "id": "e9b19d4a", "metadata": {"papermill": {"duration": 0.027207, "end_time": "2021-09-16T12:42:21.696690", "exception": false, "start_time": "2021-09-16T12:42:21.669483", "status": "completed"}, "tags": []}, "source": ["Let's first visualize the receptive field of a horizontal convolution\n", "without the center pixel. We use a small, arbitrary input image\n", "($11\\times 11$ pixels), and calculate the loss for the center pixel. For\n", "simplicity, we initialize all weights with 1 and the bias with 0, and\n", "use a single channel. This is sufficient for our visualization purposes."]}, {"cell_type": "code", "execution_count": 9, "id": "ad09d7c8", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:42:21.755609Z", "iopub.status.busy": "2021-09-16T12:42:21.755136Z", "iopub.status.idle": "2021-09-16T12:42:21.910953Z", "shell.execute_reply": "2021-09-16T12:42:21.910478Z"}, "papermill": {"duration": 0.18684, "end_time": "2021-09-16T12:42:21.911059", "exception": false, "start_time": "2021-09-16T12:42:21.724219", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUgovR3JvdXAgPDwgL0NTIC9EZXZpY2VSR0IgL1MgL1RyYW5zcGFyZW5jeSAvVHlwZSAvR3JvdXAgPj4KL01lZGlhQm94IFsgMCAwIDM0OS4yIDE4MS43MDY4MTgxODE4IF0gL1BhcmVudCAyIDAgUiAvUmVzb3VyY2VzIDggMCBSCi9UeXBlIC9QYWdlID4+CmVuZG9iago5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTIgMCBSID4+CnN0cmVhbQp4nJ1RS08CMRC+z6/4jnCwO9NStj1CUKI3zCYcjAezFFjCrvIIxn/vrILPi9jJZN6dr18FK8oGgsUOjJXqMwRjZKN0qMp0Ox6i3BFrvibXi8aqtz56EsTk3A8SNMXfwyVRQxvk2taqeGu0fpSf4TZhigbZwLYwRGGIwmCM9QbxDu361uYqZY3sWjB6xIQm2JyGGIuvg21MGxK1FzoLYcPOcez7yBHS98Z9ri9rGhbIrrTNopi/vbaY0R060y58MD663KOTqsVyn2bYdmHZBHc6WinT0746JMyrtJ51cY/ihi4LekcoIZqPZf+mg37TIZFPhNhzCbGsn+Vt6LESIn9nZFg1D9uX8zigVzdDhxgKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyNzAKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNjQgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggOTIgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzMDcgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIzMiA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjMxID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDI0OSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMzYgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDM0MSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE2NCA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDcyID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDQ3ID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTYzID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMzkgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNjAgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE4ID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEzMyA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNzUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTQxID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvQmFzZUZvbnQgL0RlamFWdVNhbnMgL0NoYXJQcm9jcyAxOCAwIFIKL0VuY29kaW5nIDw8Ci9EaWZmZXJlbmNlcyBbIDMyIC9zcGFjZSA2NiAvQiA4NyAvVyA5NyAvYSA5OSAvYyAvZCAvZSAvZiAvZyAvaCAvaSAxMDggL2wgMTEwIC9uIDExMiAvcAoxMTQgL3IgMTE2IC90IDExOCAvdiAxMjEgL3kgXQovVHlwZSAvRW5jb2RpbmcgPj4KL0ZpcnN0Q2hhciAwIC9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnREZXNjcmlwdG9yIDE2IDAgUgovRm9udE1hdHJpeCBbIDAuMDAxIDAgMCAwLjAwMSAwIDAgXSAvTGFzdENoYXIgMjU1IC9OYW1lIC9EZWphVnVTYW5zCi9TdWJ0eXBlIC9UeXBlMyAvVHlwZSAvRm9udCAvV2lkdGhzIDE1IDAgUiA+PgplbmRvYmoKMTYgMCBvYmoKPDwgL0FzY2VudCA5MjkgL0NhcEhlaWdodCAwIC9EZXNjZW50IC0yMzYgL0ZsYWdzIDMyCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnROYW1lIC9EZWphVnVTYW5zIC9JdGFsaWNBbmdsZSAwCi9NYXhXaWR0aCAxMzQyIC9TdGVtViAwIC9UeXBlIC9Gb250RGVzY3JpcHRvciAvWEhlaWdodCAwID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL0NBIDAgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PgovQTIgPDwgL0NBIDEgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9CaXRzUGVyQ29tcG9uZW50IDggL0NvbG9yU3BhY2UgWy9JbmRleGVkIC9EZXZpY2VSR0IgMiAo/ec3ACJN/wAAKV0KL0RlY29kZVBhcm1zIDw8IC9Db2xvcnMgMSAvQ29sdW1ucyAxNTMgL1ByZWRpY3RvciAxMCA+PgovRmlsdGVyIC9GbGF0ZURlY29kZSAvSGVpZ2h0IDE1MyAvTGVuZ3RoIDM3IDAgUiAvU3VidHlwZSAvSW1hZ2UKL1R5cGUgL1hPYmplY3QgL1dpZHRoIDE1MyA+PgpzdHJlYW0KeJztzrEJgFAQQLHT/Ye21cLiCeIXkgky26rm68Ats86sM+vMOrPOrDPrzDqzzqwz68w6s86sM+vMOrPOrDPrzDqzzqwz68w6s86sM+vMOrPOrDPrzDqzzqx7PJuL/czMzMzMzMzsN7PXmXVmnVln1pl1Zp1ZZ9aZdWadWWfWmXVmnVln1pl1Zp1ZZ9aZdWadWWfWmXVmnVln1pl1Zp1ZZ9aZdWbdurMDCeZbZQplbmRzdHJlYW0KZW5kb2JqCjM3IDAgb2JqCjE3MgplbmRvYmoKMTQgMCBvYmoKPDwgL0JpdHNQZXJDb21wb25lbnQgOCAvQ29sb3JTcGFjZSBbL0luZGV4ZWQgL0RldmljZVJHQiAyICj95zcAIk3/AAApXQovRGVjb2RlUGFybXMgPDwgL0NvbG9ycyAxIC9Db2x1bW5zIDE1MyAvUHJlZGljdG9yIDEwID4+Ci9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9IZWlnaHQgMTUzIC9MZW5ndGggMzggMCBSIC9TdWJ0eXBlIC9JbWFnZQovVHlwZSAvWE9iamVjdCAvV2lkdGggMTUzID4+CnN0cmVhbQp4nO3OsQmAUBBAsdP9h7bVwuIJ4heSCTLbqubrwC2zzqwz68w6s86sM+vMOrPOrDPrzDqzzqwz68w6s86sM+vMOrPOrDPrzDqzzqwz68w6s86sM+vMOrPOrHs8m4v9zMzMzMzMzOw3s9eZdWadWWfWmXVmnVln1pl1Zp1ZZ9aZdWadWWfWmXVmnVln1pl1Zp1ZZ9aZdWadWWfWmXVmnVln1pl1Zt26swMJ5ltlCmVuZHN0cmVhbQplbmRvYmoKMzggMCBvYmoKMTcyCmVuZG9iagoyIDAgb2JqCjw8IC9Db3VudCAxIC9LaWRzIFsgMTEgMCBSIF0gL1R5cGUgL1BhZ2VzID4+CmVuZG9iagozOSAwIG9iago8PCAvQ3JlYXRpb25EYXRlIChEOjIwMjEwOTE2MTQ0MjIxKzAyJzAwJykKL0NyZWF0b3IgKE1hdHBsb3RsaWIgdjMuNC4zLCBodHRwczovL21hdHBsb3RsaWIub3JnKQovUHJvZHVjZXIgKE1hdHBsb3RsaWIgcGRmIGJhY2tlbmQgdjMuNC4zKSA+PgplbmRvYmoKeHJlZgowIDQwCjAwMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAwMDAxNiAwMDAwMCBuIAowMDAwMDA4MTIwIDAwMDAwIG4gCjAwMDAwMDcwMTIgMDAwMDAgbiAKMDAwMDAwNzA0NCAwMDAwMCBuIAowMDAwMDA3MTQzIDAwMDAwIG4gCjAwMDAwMDcxNjQgMDAwMDAgbiAKMDAwMDAwNzE4NSAwMDAwMCBuIAowMDAwMDAwMDY1IDAwMDAwIG4gCjAwMDAwMDA0MDEgMDAwMDAgbiAKMDAwMDAwMDc2NiAwMDAwMCBuIAowMDAwMDAwMjA4IDAwMDAwIG4gCjAwMDAwMDA3NDYgMDAwMDAgbiAKMDAwMDAwNzIyOCAwMDAwMCBuIAowMDAwMDA3Njc0IDAwMDAwIG4gCjAwMDAwMDU3NTMgMDAwMDAgbiAKMDAwMDAwNTU1MyAwMDAwMCBuIAowMDAwMDA1MTU5IDAwMDAwIG4gCjAwMDAwMDY4MDYgMDAwMDAgbiAKMDAwMDAwMDc4NiAwMDAwMCBuIAowMDAwMDAxMTIzIDAwMDAwIG4gCjAwMDAwMDEyODcgMDAwMDAgbiAKMDAwMDAwMTY2NyAwMDAwMCBuIAowMDAwMDAxOTcyIDAwMDAwIG4gCjAwMDAwMDIyNzYgMDAwMDAgbiAKMDAwMDAwMjU5OCAwMDAwMCBuIAowMDAwMDAyODA3IDAwMDAwIG4gCjAwMDAwMDMyMjEgMDAwMDAgbiAKMDAwMDAwMzQ1OCAwMDAwMCBuIAowMDAwMDAzNjAyIDAwMDAwIG4gCjAwMDAwMDM3MjEgMDAwMDAgbiAKMDAwMDAwMzk1NyAwMDAwMCBuIAowMDAwMDA0MjY5IDAwMDAwIG4gCjAwMDAwMDQ1MDIgMDAwMDAgbiAKMDAwMDAwNDU5MiAwMDAwMCBuIAowMDAwMDA0Nzk4IDAwMDAwIG4gCjAwMDAwMDQ5NDUgMDAwMDAgbiAKMDAwMDAwNzY1NCAwMDAwMCBuIAowMDAwMDA4MTAwIDAwMDAwIG4gCjAwMDAwMDgxODAgMDAwMDAgbiAKdHJhaWxlcgo8PCAvSW5mbyAzOSAwIFIgL1Jvb3QgMSAwIFIgL1NpemUgNDAgPj4Kc3RhcnR4cmVmCjgzMzcKJSVFT0YK\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2021-09-16T14:42:21.840084\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.4.3, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}], "source": ["horiz_conv = HorizontalStackConvolution(c_in=1, c_out=1, kernel_size=3, mask_center=True)\n", "horiz_conv.conv.weight.data.fill_(1)\n", "horiz_conv.conv.bias.data.fill_(0)\n", "horiz_img = horiz_conv(inp_img)\n", "show_center_recep_field(inp_img, horiz_img)"]}, {"cell_type": "markdown", "id": "a4151321", "metadata": {"papermill": {"duration": 0.02825, "end_time": "2021-09-16T12:42:21.967945", "exception": false, "start_time": "2021-09-16T12:42:21.939695", "status": "completed"}, "tags": []}, "source": ["The receptive field is shown in yellow, the center pixel in red, and all other pixels outside of the receptive field are dark blue.\n", "As expected, the receptive field of a single horizontal convolution with the center pixel masked and a $3\\times3$ kernel is only the pixel on the left.\n", "If we use a larger kernel size, more pixels would be taken into account on the left.\n", "\n", "Next, let's take a look at the vertical convolution:"]}, {"cell_type": "code", "execution_count": 10, "id": "4d7db603", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:42:22.028223Z", "iopub.status.busy": "2021-09-16T12:42:22.026790Z", "iopub.status.idle": "2021-09-16T12:42:22.183299Z", "shell.execute_reply": "2021-09-16T12:42:22.182815Z"}, "papermill": {"duration": 0.186816, "end_time": "2021-09-16T12:42:22.183411", "exception": false, "start_time": "2021-09-16T12:42:21.996595", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUgovR3JvdXAgPDwgL0NTIC9EZXZpY2VSR0IgL1MgL1RyYW5zcGFyZW5jeSAvVHlwZSAvR3JvdXAgPj4KL01lZGlhQm94IFsgMCAwIDM0OS4yIDE4MS43MDY4MTgxODE4IF0gL1BhcmVudCAyIDAgUiAvUmVzb3VyY2VzIDggMCBSCi9UeXBlIC9QYWdlID4+CmVuZG9iago5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTIgMCBSID4+CnN0cmVhbQp4nJ1RS08CMRC+z6/4jnCwO9NStj1CUKI3zCYcjAezFFjCrvIIxn/vrILPi9jJZN6dr18FK8oGgsUOjJXqMwRjZKN0qMp0Ox6i3BFrvibXi8aqtz56EsTk3A8SNMXfwyVRQxvk2taqeGu0fpSf4TZhigbZwLYwRGGIwmCM9QbxDu361uYqZY3sWjB6xIQm2JyGGIuvg21MGxK1FzoLYcPOcez7yBHS98Z9ri9rGhbIrrTNopi/vbaY0R060y58MD663KOTqsVyn2bYdmHZBHc6WinT0746JMyrtJ51cY/ihi4LekcoIZqPZf+mg37TIZFPhNhzCbGsn+Vt6LESIn9nZFg1D9uX8zigVzdDhxgKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyNzAKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNjQgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggOTIgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzMDcgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIzMiA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjMxID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDI0OSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMzYgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDM0MSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE2NCA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDcyID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDQ3ID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTYzID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMzkgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNjAgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE4ID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEzMyA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNzUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTQxID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvQmFzZUZvbnQgL0RlamFWdVNhbnMgL0NoYXJQcm9jcyAxOCAwIFIKL0VuY29kaW5nIDw8Ci9EaWZmZXJlbmNlcyBbIDMyIC9zcGFjZSA2NiAvQiA4NyAvVyA5NyAvYSA5OSAvYyAvZCAvZSAvZiAvZyAvaCAvaSAxMDggL2wgMTEwIC9uIDExMiAvcAoxMTQgL3IgMTE2IC90IDExOCAvdiAxMjEgL3kgXQovVHlwZSAvRW5jb2RpbmcgPj4KL0ZpcnN0Q2hhciAwIC9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnREZXNjcmlwdG9yIDE2IDAgUgovRm9udE1hdHJpeCBbIDAuMDAxIDAgMCAwLjAwMSAwIDAgXSAvTGFzdENoYXIgMjU1IC9OYW1lIC9EZWphVnVTYW5zCi9TdWJ0eXBlIC9UeXBlMyAvVHlwZSAvRm9udCAvV2lkdGhzIDE1IDAgUiA+PgplbmRvYmoKMTYgMCBvYmoKPDwgL0FzY2VudCA5MjkgL0NhcEhlaWdodCAwIC9EZXNjZW50IC0yMzYgL0ZsYWdzIDMyCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnROYW1lIC9EZWphVnVTYW5zIC9JdGFsaWNBbmdsZSAwCi9NYXhXaWR0aCAxMzQyIC9TdGVtViAwIC9UeXBlIC9Gb250RGVzY3JpcHRvciAvWEhlaWdodCAwID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL0NBIDAgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PgovQTIgPDwgL0NBIDEgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9CaXRzUGVyQ29tcG9uZW50IDggL0NvbG9yU3BhY2UgWy9JbmRleGVkIC9EZXZpY2VSR0IgMiAo/ec3ACJN/wAAKV0KL0RlY29kZVBhcm1zIDw8IC9Db2xvcnMgMSAvQ29sdW1ucyAxNTMgL1ByZWRpY3RvciAxMCA+PgovRmlsdGVyIC9GbGF0ZURlY29kZSAvSGVpZ2h0IDE1MyAvTGVuZ3RoIDM3IDAgUiAvU3VidHlwZSAvSW1hZ2UKL1R5cGUgL1hPYmplY3QgL1dpZHRoIDE1MyA+PgpzdHJlYW0KeJzt2LEJgEAABMHX/os2NRHcV1FktoBj4hvLVxtvAw4j65H1yHpkPbIeWY+sR9Yj65H1yHpkPbIeWY+sR9Yj65H1yHpkPbIeWY+sR9Yj65H1yHrTsnE2MjIyMjIyMrLLrfvumSTrkfXIemQ9sh5Z74e/xuOR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6G2EeWd0KZW5kc3RyZWFtCmVuZG9iagozNyAwIG9iagoxODUKZW5kb2JqCjE0IDAgb2JqCjw8IC9CaXRzUGVyQ29tcG9uZW50IDggL0NvbG9yU3BhY2UgWy9JbmRleGVkIC9EZXZpY2VSR0IgMiAo/ec3ACJN/wAAKV0KL0RlY29kZVBhcm1zIDw8IC9Db2xvcnMgMSAvQ29sdW1ucyAxNTMgL1ByZWRpY3RvciAxMCA+PgovRmlsdGVyIC9GbGF0ZURlY29kZSAvSGVpZ2h0IDE1MyAvTGVuZ3RoIDM4IDAgUiAvU3VidHlwZSAvSW1hZ2UKL1R5cGUgL1hPYmplY3QgL1dpZHRoIDE1MyA+PgpzdHJlYW0KeJzt2LEJgEAABMHX/os2NRHcV1FktoBj4hvLVxtvAw4j65H1yHpkPbIeWY+sR9Yj65H1yHpkPbIeWY+sR9Yj65H1yHpkPbIeWY+sR9Yj65H1yHrTsnE2MjIyMjIyMrLLrfvumSTrkfXIemQ9sh5Z74e/xuOR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6G2EeWd0KZW5kc3RyZWFtCmVuZG9iagozOCAwIG9iagoxODUKZW5kb2JqCjIgMCBvYmoKPDwgL0NvdW50IDEgL0tpZHMgWyAxMSAwIFIgXSAvVHlwZSAvUGFnZXMgPj4KZW5kb2JqCjM5IDAgb2JqCjw8IC9DcmVhdGlvbkRhdGUgKEQ6MjAyMTA5MTYxNDQyMjIrMDInMDAnKQovQ3JlYXRvciAoTWF0cGxvdGxpYiB2My40LjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcpCi9Qcm9kdWNlciAoTWF0cGxvdGxpYiBwZGYgYmFja2VuZCB2My40LjMpID4+CmVuZG9iagp4cmVmCjAgNDAKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDgxNDYgMDAwMDAgbiAKMDAwMDAwNzAxMiAwMDAwMCBuIAowMDAwMDA3MDQ0IDAwMDAwIG4gCjAwMDAwMDcxNDMgMDAwMDAgbiAKMDAwMDAwNzE2NCAwMDAwMCBuIAowMDAwMDA3MTg1IDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDQwMSAwMDAwMCBuIAowMDAwMDAwNzY2IDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDc0NiAwMDAwMCBuIAowMDAwMDA3MjI4IDAwMDAwIG4gCjAwMDAwMDc2ODcgMDAwMDAgbiAKMDAwMDAwNTc1MyAwMDAwMCBuIAowMDAwMDA1NTUzIDAwMDAwIG4gCjAwMDAwMDUxNTkgMDAwMDAgbiAKMDAwMDAwNjgwNiAwMDAwMCBuIAowMDAwMDAwNzg2IDAwMDAwIG4gCjAwMDAwMDExMjMgMDAwMDAgbiAKMDAwMDAwMTI4NyAwMDAwMCBuIAowMDAwMDAxNjY3IDAwMDAwIG4gCjAwMDAwMDE5NzIgMDAwMDAgbiAKMDAwMDAwMjI3NiAwMDAwMCBuIAowMDAwMDAyNTk4IDAwMDAwIG4gCjAwMDAwMDI4MDcgMDAwMDAgbiAKMDAwMDAwMzIyMSAwMDAwMCBuIAowMDAwMDAzNDU4IDAwMDAwIG4gCjAwMDAwMDM2MDIgMDAwMDAgbiAKMDAwMDAwMzcyMSAwMDAwMCBuIAowMDAwMDAzOTU3IDAwMDAwIG4gCjAwMDAwMDQyNjkgMDAwMDAgbiAKMDAwMDAwNDUwMiAwMDAwMCBuIAowMDAwMDA0NTkyIDAwMDAwIG4gCjAwMDAwMDQ3OTggMDAwMDAgbiAKMDAwMDAwNDk0NSAwMDAwMCBuIAowMDAwMDA3NjY3IDAwMDAwIG4gCjAwMDAwMDgxMjYgMDAwMDAgbiAKMDAwMDAwODIwNiAwMDAwMCBuIAp0cmFpbGVyCjw8IC9JbmZvIDM5IDAgUiAvUm9vdCAxIDAgUiAvU2l6ZSA0MCA+PgpzdGFydHhyZWYKODM2MwolJUVPRgo=\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2021-09-16T14:42:22.111784\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.4.3, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}], "source": ["vert_conv = VerticalStackConvolution(c_in=1, c_out=1, kernel_size=3, mask_center=True)\n", "vert_conv.conv.weight.data.fill_(1)\n", "vert_conv.conv.bias.data.fill_(0)\n", "vert_img = vert_conv(inp_img)\n", "show_center_recep_field(inp_img, vert_img)"]}, {"cell_type": "markdown", "id": "f506aec8", "metadata": {"papermill": {"duration": 0.029554, "end_time": "2021-09-16T12:42:22.243059", "exception": false, "start_time": "2021-09-16T12:42:22.213505", "status": "completed"}, "tags": []}, "source": ["The vertical convolution takes all pixels above into account. Combining\n", "these two, we get the L-shaped receptive field of the original masked\n", "convolution:"]}, {"cell_type": "code", "execution_count": 11, "id": "63b459cc", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:42:22.305232Z", "iopub.status.busy": "2021-09-16T12:42:22.304151Z", "iopub.status.idle": "2021-09-16T12:42:22.458303Z", "shell.execute_reply": "2021-09-16T12:42:22.457833Z"}, "papermill": {"duration": 0.185539, "end_time": "2021-09-16T12:42:22.458405", "exception": false, "start_time": "2021-09-16T12:42:22.272866", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUgovR3JvdXAgPDwgL0NTIC9EZXZpY2VSR0IgL1MgL1RyYW5zcGFyZW5jeSAvVHlwZSAvR3JvdXAgPj4KL01lZGlhQm94IFsgMCAwIDM0OS4yIDE4MS43MDY4MTgxODE4IF0gL1BhcmVudCAyIDAgUiAvUmVzb3VyY2VzIDggMCBSCi9UeXBlIC9QYWdlID4+CmVuZG9iago5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTIgMCBSID4+CnN0cmVhbQp4nJ1RS08CMRC+z6/4jnCwO9NStj1CUKI3zCYcjAezFFjCrvIIxn/vrILPi9jJZN6dr18FK8oGgsUOjJXqMwRjZKN0qMp0Ox6i3BFrvibXi8aqtz56EsTk3A8SNMXfwyVRQxvk2taqeGu0fpSf4TZhigbZwLYwRGGIwmCM9QbxDu361uYqZY3sWjB6xIQm2JyGGIuvg21MGxK1FzoLYcPOcez7yBHS98Z9ri9rGhbIrrTNopi/vbaY0R060y58MD663KOTqsVyn2bYdmHZBHc6WinT0746JMyrtJ51cY/ihi4LekcoIZqPZf+mg37TIZFPhNhzCbGsn+Vt6LESIn9nZFg1D9uX8zigVzdDhxgKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyNzAKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNjQgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggOTIgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzMDcgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIzMiA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjMxID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDI0OSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMzYgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDM0MSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE2NCA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDcyID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDQ3ID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTYzID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMzkgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNjAgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE4ID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEzMyA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNzUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTQxID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvQmFzZUZvbnQgL0RlamFWdVNhbnMgL0NoYXJQcm9jcyAxOCAwIFIKL0VuY29kaW5nIDw8Ci9EaWZmZXJlbmNlcyBbIDMyIC9zcGFjZSA2NiAvQiA4NyAvVyA5NyAvYSA5OSAvYyAvZCAvZSAvZiAvZyAvaCAvaSAxMDggL2wgMTEwIC9uIDExMiAvcAoxMTQgL3IgMTE2IC90IDExOCAvdiAxMjEgL3kgXQovVHlwZSAvRW5jb2RpbmcgPj4KL0ZpcnN0Q2hhciAwIC9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnREZXNjcmlwdG9yIDE2IDAgUgovRm9udE1hdHJpeCBbIDAuMDAxIDAgMCAwLjAwMSAwIDAgXSAvTGFzdENoYXIgMjU1IC9OYW1lIC9EZWphVnVTYW5zCi9TdWJ0eXBlIC9UeXBlMyAvVHlwZSAvRm9udCAvV2lkdGhzIDE1IDAgUiA+PgplbmRvYmoKMTYgMCBvYmoKPDwgL0FzY2VudCA5MjkgL0NhcEhlaWdodCAwIC9EZXNjZW50IC0yMzYgL0ZsYWdzIDMyCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnROYW1lIC9EZWphVnVTYW5zIC9JdGFsaWNBbmdsZSAwCi9NYXhXaWR0aCAxMzQyIC9TdGVtViAwIC9UeXBlIC9Gb250RGVzY3JpcHRvciAvWEhlaWdodCAwID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL0NBIDAgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PgovQTIgPDwgL0NBIDEgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9CaXRzUGVyQ29tcG9uZW50IDggL0NvbG9yU3BhY2UgWy9JbmRleGVkIC9EZXZpY2VSR0IgMiAo/ec3ACJN/wAAKV0KL0RlY29kZVBhcm1zIDw8IC9Db2xvcnMgMSAvQ29sdW1ucyAxNTMgL1ByZWRpY3RvciAxMCA+PgovRmlsdGVyIC9GbGF0ZURlY29kZSAvSGVpZ2h0IDE1MyAvTGVuZ3RoIDM3IDAgUiAvU3VidHlwZSAvSW1hZ2UKL1R5cGUgL1hPYmplY3QgL1dpZHRoIDE1MyA+PgpzdHJlYW0KeJztzrENgDAAwLDC/0ezsiCRIqCDc0Dksa3a+BtwGVmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWm5aNu5GRkZGRkZGRPZbt52aXZGRkZGRkZJ/LXo+sR9Yj65H1yHpkPbIeWY+sR9Yj65H1yHpkPbIeWY+sR9Yj65H1yHpkPbIeWY+sR9Yj65H1yHpkPbIeWY+sR9ZbV3YAnghZJwplbmRzdHJlYW0KZW5kb2JqCjM3IDAgb2JqCjE4MQplbmRvYmoKMTQgMCBvYmoKPDwgL0JpdHNQZXJDb21wb25lbnQgOCAvQ29sb3JTcGFjZSBbL0luZGV4ZWQgL0RldmljZVJHQiAyICj95zcAIk3/AAApXQovRGVjb2RlUGFybXMgPDwgL0NvbG9ycyAxIC9Db2x1bW5zIDE1MyAvUHJlZGljdG9yIDEwID4+Ci9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9IZWlnaHQgMTUzIC9MZW5ndGggMzggMCBSIC9TdWJ0eXBlIC9JbWFnZQovVHlwZSAvWE9iamVjdCAvV2lkdGggMTUzID4+CnN0cmVhbQp4nO3OsQ2AMADAsML/R7OyIJEioINzQOSxrdr4G3AZWY+sR9Yj65H1yHpkPbIeWY+sR9Yj65H1yHpkPbIeWY+sR9Yj65H1yHpkPbIeWY+sR9ablo27kZGRkZGRkZE9lu3nZpdkZGRkZGRkn8tej6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1ltXdgCeCFknCmVuZHN0cmVhbQplbmRvYmoKMzggMCBvYmoKMTgxCmVuZG9iagoyIDAgb2JqCjw8IC9Db3VudCAxIC9LaWRzIFsgMTEgMCBSIF0gL1R5cGUgL1BhZ2VzID4+CmVuZG9iagozOSAwIG9iago8PCAvQ3JlYXRpb25EYXRlIChEOjIwMjEwOTE2MTQ0MjIyKzAyJzAwJykKL0NyZWF0b3IgKE1hdHBsb3RsaWIgdjMuNC4zLCBodHRwczovL21hdHBsb3RsaWIub3JnKQovUHJvZHVjZXIgKE1hdHBsb3RsaWIgcGRmIGJhY2tlbmQgdjMuNC4zKSA+PgplbmRvYmoKeHJlZgowIDQwCjAwMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAwMDAxNiAwMDAwMCBuIAowMDAwMDA4MTM4IDAwMDAwIG4gCjAwMDAwMDcwMTIgMDAwMDAgbiAKMDAwMDAwNzA0NCAwMDAwMCBuIAowMDAwMDA3MTQzIDAwMDAwIG4gCjAwMDAwMDcxNjQgMDAwMDAgbiAKMDAwMDAwNzE4NSAwMDAwMCBuIAowMDAwMDAwMDY1IDAwMDAwIG4gCjAwMDAwMDA0MDEgMDAwMDAgbiAKMDAwMDAwMDc2NiAwMDAwMCBuIAowMDAwMDAwMjA4IDAwMDAwIG4gCjAwMDAwMDA3NDYgMDAwMDAgbiAKMDAwMDAwNzIyOCAwMDAwMCBuIAowMDAwMDA3NjgzIDAwMDAwIG4gCjAwMDAwMDU3NTMgMDAwMDAgbiAKMDAwMDAwNTU1MyAwMDAwMCBuIAowMDAwMDA1MTU5IDAwMDAwIG4gCjAwMDAwMDY4MDYgMDAwMDAgbiAKMDAwMDAwMDc4NiAwMDAwMCBuIAowMDAwMDAxMTIzIDAwMDAwIG4gCjAwMDAwMDEyODcgMDAwMDAgbiAKMDAwMDAwMTY2NyAwMDAwMCBuIAowMDAwMDAxOTcyIDAwMDAwIG4gCjAwMDAwMDIyNzYgMDAwMDAgbiAKMDAwMDAwMjU5OCAwMDAwMCBuIAowMDAwMDAyODA3IDAwMDAwIG4gCjAwMDAwMDMyMjEgMDAwMDAgbiAKMDAwMDAwMzQ1OCAwMDAwMCBuIAowMDAwMDAzNjAyIDAwMDAwIG4gCjAwMDAwMDM3MjEgMDAwMDAgbiAKMDAwMDAwMzk1NyAwMDAwMCBuIAowMDAwMDA0MjY5IDAwMDAwIG4gCjAwMDAwMDQ1MDIgMDAwMDAgbiAKMDAwMDAwNDU5MiAwMDAwMCBuIAowMDAwMDA0Nzk4IDAwMDAwIG4gCjAwMDAwMDQ5NDUgMDAwMDAgbiAKMDAwMDAwNzY2MyAwMDAwMCBuIAowMDAwMDA4MTE4IDAwMDAwIG4gCjAwMDAwMDgxOTggMDAwMDAgbiAKdHJhaWxlcgo8PCAvSW5mbyAzOSAwIFIgL1Jvb3QgMSAwIFIgL1NpemUgNDAgPj4Kc3RhcnR4cmVmCjgzNTUKJSVFT0YK\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2021-09-16T14:42:22.388054\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.4.3, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}], "source": ["horiz_img = vert_img + horiz_img\n", "show_center_recep_field(inp_img, horiz_img)"]}, {"cell_type": "markdown", "id": "62da8fad", "metadata": {"papermill": {"duration": 0.030338, "end_time": "2021-09-16T12:42:22.519399", "exception": false, "start_time": "2021-09-16T12:42:22.489061", "status": "completed"}, "tags": []}, "source": ["If we stack multiple horizontal and vertical convolutions, we need to take two aspects into account:\n", "\n", "1.\n", "The center should not be masked anymore for the following convolutions as the features at the pixel's position are already independent of its actual value.\n", "If it is hard to imagine why we can do this, just change the value below to `mask_center=True` and see what happens.\n", "2.\n", "The vertical convolution is not allowed to work on features from the horizontal convolution.\n", "In the feature map of the horizontal convolutions, a pixel contains information about all of the \"true\" pixels on the left.\n", "If we apply a vertical convolution which also uses features from the right, we effectively expand our receptive field to the true input which we want to prevent.\n", "Thus, the feature maps can only be merged for the horizontal convolution.\n", "\n", "Using this, we can stack the convolutions in the following way. We have\n", "two feature streams: one for the vertical stack, and one for the\n", "horizontal stack. The horizontal convolutions can operate on the joint\n", "features of the previous horizontals and vertical convolutions, while\n", "the vertical stack only takes its own previous features as input. For a\n", "quick implementation, we can therefore sum the horizontal and vertical\n", "output features at each layer, and use those as final output features to\n", "calculate the loss on. An implementation of 4 consecutive layers is\n", "shown below. Note that we reuse the features from the other convolutions\n", "with `mask_center=True` from above."]}, {"cell_type": "code", "execution_count": 12, "id": "f6dcdf70", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:42:22.585305Z", "iopub.status.busy": "2021-09-16T12:42:22.584833Z", "iopub.status.idle": "2021-09-16T12:42:23.217060Z", "shell.execute_reply": "2021-09-16T12:42:23.216574Z"}, "papermill": {"duration": 0.666981, "end_time": "2021-09-16T12:42:23.217176", "exception": false, "start_time": "2021-09-16T12:42:22.550195", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Layer 2\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUgovR3JvdXAgPDwgL0NTIC9EZXZpY2VSR0IgL1MgL1RyYW5zcGFyZW5jeSAvVHlwZSAvR3JvdXAgPj4KL01lZGlhQm94IFsgMCAwIDM0OS4yIDE4MS43MDY4MTgxODE4IF0gL1BhcmVudCAyIDAgUiAvUmVzb3VyY2VzIDggMCBSCi9UeXBlIC9QYWdlID4+CmVuZG9iago5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTIgMCBSID4+CnN0cmVhbQp4nJ1RS08CMRC+z6/4jnCwO9NStj1CUKI3zCYcjAezFFjCrvIIxn/vrILPi9jJZN6dr18FK8oGgsUOjJXqMwRjZKN0qMp0Ox6i3BFrvibXi8aqtz56EsTk3A8SNMXfwyVRQxvk2taqeGu0fpSf4TZhigbZwLYwRGGIwmCM9QbxDu361uYqZY3sWjB6xIQm2JyGGIuvg21MGxK1FzoLYcPOcez7yBHS98Z9ri9rGhbIrrTNopi/vbaY0R060y58MD663KOTqsVyn2bYdmHZBHc6WinT0746JMyrtJ51cY/ihi4LekcoIZqPZf+mg37TIZFPhNhzCbGsn+Vt6LESIn9nZFg1D9uX8zigVzdDhxgKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyNzAKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNjQgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggOTIgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzMDcgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIzMiA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjMxID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDI0OSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMzYgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDM0MSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE2NCA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDcyID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDQ3ID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTYzID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMzkgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNjAgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE4ID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEzMyA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNzUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTQxID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvQmFzZUZvbnQgL0RlamFWdVNhbnMgL0NoYXJQcm9jcyAxOCAwIFIKL0VuY29kaW5nIDw8Ci9EaWZmZXJlbmNlcyBbIDMyIC9zcGFjZSA2NiAvQiA4NyAvVyA5NyAvYSA5OSAvYyAvZCAvZSAvZiAvZyAvaCAvaSAxMDggL2wgMTEwIC9uIDExMiAvcAoxMTQgL3IgMTE2IC90IDExOCAvdiAxMjEgL3kgXQovVHlwZSAvRW5jb2RpbmcgPj4KL0ZpcnN0Q2hhciAwIC9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnREZXNjcmlwdG9yIDE2IDAgUgovRm9udE1hdHJpeCBbIDAuMDAxIDAgMCAwLjAwMSAwIDAgXSAvTGFzdENoYXIgMjU1IC9OYW1lIC9EZWphVnVTYW5zCi9TdWJ0eXBlIC9UeXBlMyAvVHlwZSAvRm9udCAvV2lkdGhzIDE1IDAgUiA+PgplbmRvYmoKMTYgMCBvYmoKPDwgL0FzY2VudCA5MjkgL0NhcEhlaWdodCAwIC9EZXNjZW50IC0yMzYgL0ZsYWdzIDMyCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnROYW1lIC9EZWphVnVTYW5zIC9JdGFsaWNBbmdsZSAwCi9NYXhXaWR0aCAxMzQyIC9TdGVtViAwIC9UeXBlIC9Gb250RGVzY3JpcHRvciAvWEhlaWdodCAwID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL0NBIDAgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PgovQTIgPDwgL0NBIDEgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9CaXRzUGVyQ29tcG9uZW50IDgKL0NvbG9yU3BhY2UgWy9JbmRleGVkIC9EZXZpY2VSR0IgNiAo/ec3yLdllI53ZmlwNUVsACJN/wAAKV0KL0RlY29kZVBhcm1zIDw8IC9Db2xvcnMgMSAvQ29sdW1ucyAxNTMgL1ByZWRpY3RvciAxMCA+PgovRmlsdGVyIC9GbGF0ZURlY29kZSAvSGVpZ2h0IDE1MyAvTGVuZ3RoIDM3IDAgUiAvU3VidHlwZSAvSW1hZ2UKL1R5cGUgL1hPYmplY3QgL1dpZHRoIDE1MyA+PgpzdHJlYW0KeJztzjESggAMAEFE9P9PtiUFDtcAxV6bSbLL56ktdwMOI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZL0g20bv0bpvjuYaGRkZGRkZGdlB8+FrtOwbzJWMjIyMjIyM7FTbn777zp8kIyMjIyMju1x2cWQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9st5zZT9MVLaSCmVuZHN0cmVhbQplbmRvYmoKMzcgMCBvYmoKMTk1CmVuZG9iagoxNCAwIG9iago8PCAvQml0c1BlckNvbXBvbmVudCA4IC9Db2xvclNwYWNlIFsvSW5kZXhlZCAvRGV2aWNlUkdCIDIgKP3nNwAiTf8AACldCi9EZWNvZGVQYXJtcyA8PCAvQ29sb3JzIDEgL0NvbHVtbnMgMTUzIC9QcmVkaWN0b3IgMTAgPj4KL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0hlaWdodCAxNTMgL0xlbmd0aCAzOCAwIFIgL1N1YnR5cGUgL0ltYWdlCi9UeXBlIC9YT2JqZWN0IC9XaWR0aCAxNTMgPj4Kc3RyZWFtCnic7c7BCYBAEMDA0/6L9utHMCDnPiYFhFnH1NbfgMfIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5ZL8jWF5GRkZGRkZGRkZGRkZGRkZENk5333i/JyMjIyMjItss2R9Yj65H1yHpkPbIeWY+sR9Yj65H1yHpkPbIeWY+sR9Yj65H1yHpkPbIeWY+sR9Yj65H1yHpkPbIeWY+sR9Yj682VXcPNUyMKZW5kc3RyZWFtCmVuZG9iagozOCAwIG9iagoxNzUKZW5kb2JqCjIgMCBvYmoKPDwgL0NvdW50IDEgL0tpZHMgWyAxMSAwIFIgXSAvVHlwZSAvUGFnZXMgPj4KZW5kb2JqCjM5IDAgb2JqCjw8IC9DcmVhdGlvbkRhdGUgKEQ6MjAyMTA5MTYxNDQyMjIrMDInMDAnKQovQ3JlYXRvciAoTWF0cGxvdGxpYiB2My40LjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcpCi9Qcm9kdWNlciAoTWF0cGxvdGxpYiBwZGYgYmFja2VuZCB2My40LjMpID4+CmVuZG9iagp4cmVmCjAgNDAKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDgxNTggMDAwMDAgbiAKMDAwMDAwNzAxMiAwMDAwMCBuIAowMDAwMDA3MDQ0IDAwMDAwIG4gCjAwMDAwMDcxNDMgMDAwMDAgbiAKMDAwMDAwNzE2NCAwMDAwMCBuIAowMDAwMDA3MTg1IDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDQwMSAwMDAwMCBuIAowMDAwMDAwNzY2IDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDc0NiAwMDAwMCBuIAowMDAwMDA3MjI4IDAwMDAwIG4gCjAwMDAwMDc3MDkgMDAwMDAgbiAKMDAwMDAwNTc1MyAwMDAwMCBuIAowMDAwMDA1NTUzIDAwMDAwIG4gCjAwMDAwMDUxNTkgMDAwMDAgbiAKMDAwMDAwNjgwNiAwMDAwMCBuIAowMDAwMDAwNzg2IDAwMDAwIG4gCjAwMDAwMDExMjMgMDAwMDAgbiAKMDAwMDAwMTI4NyAwMDAwMCBuIAowMDAwMDAxNjY3IDAwMDAwIG4gCjAwMDAwMDE5NzIgMDAwMDAgbiAKMDAwMDAwMjI3NiAwMDAwMCBuIAowMDAwMDAyNTk4IDAwMDAwIG4gCjAwMDAwMDI4MDcgMDAwMDAgbiAKMDAwMDAwMzIyMSAwMDAwMCBuIAowMDAwMDAzNDU4IDAwMDAwIG4gCjAwMDAwMDM2MDIgMDAwMDAgbiAKMDAwMDAwMzcyMSAwMDAwMCBuIAowMDAwMDAzOTU3IDAwMDAwIG4gCjAwMDAwMDQyNjkgMDAwMDAgbiAKMDAwMDAwNDUwMiAwMDAwMCBuIAowMDAwMDA0NTkyIDAwMDAwIG4gCjAwMDAwMDQ3OTggMDAwMDAgbiAKMDAwMDAwNDk0NSAwMDAwMCBuIAowMDAwMDA3Njg5IDAwMDAwIG4gCjAwMDAwMDgxMzggMDAwMDAgbiAKMDAwMDAwODIxOCAwMDAwMCBuIAp0cmFpbGVyCjw8IC9JbmZvIDM5IDAgUiAvUm9vdCAxIDAgUiAvU2l6ZSA0MCA+PgpzdGFydHhyZWYKODM3NQolJUVPRgo=\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2021-09-16T14:42:22.673112\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.4.3, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}, {"name": "stdout", "output_type": "stream", "text": ["Layer 3\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUgovR3JvdXAgPDwgL0NTIC9EZXZpY2VSR0IgL1MgL1RyYW5zcGFyZW5jeSAvVHlwZSAvR3JvdXAgPj4KL01lZGlhQm94IFsgMCAwIDM0OS4yIDE4MS43MDY4MTgxODE4IF0gL1BhcmVudCAyIDAgUiAvUmVzb3VyY2VzIDggMCBSCi9UeXBlIC9QYWdlID4+CmVuZG9iago5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTIgMCBSID4+CnN0cmVhbQp4nJ1RS08CMRC+z6/4jnCwO9NStj1CUKI3zCYcjAezFFjCrvIIxn/vrILPi9jJZN6dr18FK8oGgsUOjJXqMwRjZKN0qMp0Ox6i3BFrvibXi8aqtz56EsTk3A8SNMXfwyVRQxvk2taqeGu0fpSf4TZhigbZwLYwRGGIwmCM9QbxDu361uYqZY3sWjB6xIQm2JyGGIuvg21MGxK1FzoLYcPOcez7yBHS98Z9ri9rGhbIrrTNopi/vbaY0R060y58MD663KOTqsVyn2bYdmHZBHc6WinT0746JMyrtJ51cY/ihi4LekcoIZqPZf+mg37TIZFPhNhzCbGsn+Vt6LESIn9nZFg1D9uX8zigVzdDhxgKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyNzAKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNjQgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggOTIgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzMDcgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIzMiA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjMxID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDI0OSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMzYgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDM0MSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE2NCA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDcyID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDQ3ID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTYzID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMzkgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNjAgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE4ID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEzMyA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNzUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTQxID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvQmFzZUZvbnQgL0RlamFWdVNhbnMgL0NoYXJQcm9jcyAxOCAwIFIKL0VuY29kaW5nIDw8Ci9EaWZmZXJlbmNlcyBbIDMyIC9zcGFjZSA2NiAvQiA4NyAvVyA5NyAvYSA5OSAvYyAvZCAvZSAvZiAvZyAvaCAvaSAxMDggL2wgMTEwIC9uIDExMiAvcAoxMTQgL3IgMTE2IC90IDExOCAvdiAxMjEgL3kgXQovVHlwZSAvRW5jb2RpbmcgPj4KL0ZpcnN0Q2hhciAwIC9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnREZXNjcmlwdG9yIDE2IDAgUgovRm9udE1hdHJpeCBbIDAuMDAxIDAgMCAwLjAwMSAwIDAgXSAvTGFzdENoYXIgMjU1IC9OYW1lIC9EZWphVnVTYW5zCi9TdWJ0eXBlIC9UeXBlMyAvVHlwZSAvRm9udCAvV2lkdGhzIDE1IDAgUiA+PgplbmRvYmoKMTYgMCBvYmoKPDwgL0FzY2VudCA5MjkgL0NhcEhlaWdodCAwIC9EZXNjZW50IC0yMzYgL0ZsYWdzIDMyCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnROYW1lIC9EZWphVnVTYW5zIC9JdGFsaWNBbmdsZSAwCi9NYXhXaWR0aCAxMzQyIC9TdGVtViAwIC9UeXBlIC9Gb250RGVzY3JpcHRvciAvWEhlaWdodCAwID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL0NBIDAgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PgovQTIgPDwgL0NBIDEgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9CaXRzUGVyQ29tcG9uZW50IDgKL0NvbG9yU3BhY2UgWy9JbmRleGVkIC9EZXZpY2VSR0IgMTIgKP3nN+PNUsa1Z4KAeHZ2dl9jblJZbDdGbCc9bQszcAArZAAiTf8AACldCi9EZWNvZGVQYXJtcyA8PCAvQ29sb3JzIDEgL0NvbHVtbnMgMTUzIC9QcmVkaWN0b3IgMTAgPj4KL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0hlaWdodCAxNTMgL0xlbmd0aCAzNyAwIFIgL1N1YnR5cGUgL0ltYWdlCi9UeXBlIC9YT2JqZWN0IC9XaWR0aCAxNTMgPj4Kc3RyZWFtCnic7dY7bsNAEAVB6mvJ9v3Pq5QTkHADgiXD1elgHyrc5etdW14N2IysR9Yj65H1yHpkPbIeWY+sR9Yj65H1yHq7ss/RbXQdXdbN03w2J8nIyMjIyMj+tWyOnkeH0bLuOBroy31ERkZGRkZGRrYhO253Gn2MfuEXREZGRkZGRvZXZHNmfmDm7Xvd3uTPIyMjIyMjI3uS7KWR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch67yt7ALevlsIKZW5kc3RyZWFtCmVuZG9iagozNyAwIG9iagoyMjMKZW5kb2JqCjE0IDAgb2JqCjw8IC9CaXRzUGVyQ29tcG9uZW50IDggL0NvbG9yU3BhY2UgWy9JbmRleGVkIC9EZXZpY2VSR0IgMiAo/ec3ACJN/wAAKV0KL0RlY29kZVBhcm1zIDw8IC9Db2xvcnMgMSAvQ29sdW1ucyAxNTMgL1ByZWRpY3RvciAxMCA+PgovRmlsdGVyIC9GbGF0ZURlY29kZSAvSGVpZ2h0IDE1MyAvTGVuZ3RoIDM4IDAgUiAvU3VidHlwZSAvSW1hZ2UKL1R5cGUgL1hPYmplY3QgL1dpZHRoIDE1MyA+PgpzdHJlYW0KeJztzsEJgDAAwMDq/kP79aMYUFrkMkC4sa3amA24jKxH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIerey8XVkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGS/lu3n7pbPIyMjIyMjI3tJNjWyHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHllvXdkBi+JKDwplbmRzdHJlYW0KZW5kb2JqCjM4IDAgb2JqCjE3MgplbmRvYmoKMiAwIG9iago8PCAvQ291bnQgMSAvS2lkcyBbIDExIDAgUiBdIC9UeXBlIC9QYWdlcyA+PgplbmRvYmoKMzkgMCBvYmoKPDwgL0NyZWF0aW9uRGF0ZSAoRDoyMDIxMDkxNjE0NDIyMiswMicwMCcpCi9DcmVhdG9yIChNYXRwbG90bGliIHYzLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjQuMykgPj4KZW5kb2JqCnhyZWYKMCA0MAowMDAwMDAwMDAwIDY1NTM1IGYgCjAwMDAwMDAwMTYgMDAwMDAgbiAKMDAwMDAwODIwMiAwMDAwMCBuIAowMDAwMDA3MDEyIDAwMDAwIG4gCjAwMDAwMDcwNDQgMDAwMDAgbiAKMDAwMDAwNzE0MyAwMDAwMCBuIAowMDAwMDA3MTY0IDAwMDAwIG4gCjAwMDAwMDcxODUgMDAwMDAgbiAKMDAwMDAwMDA2NSAwMDAwMCBuIAowMDAwMDAwNDAxIDAwMDAwIG4gCjAwMDAwMDA3NjYgMDAwMDAgbiAKMDAwMDAwMDIwOCAwMDAwMCBuIAowMDAwMDAwNzQ2IDAwMDAwIG4gCjAwMDAwMDcyMjggMDAwMDAgbiAKMDAwMDAwNzc1NiAwMDAwMCBuIAowMDAwMDA1NzUzIDAwMDAwIG4gCjAwMDAwMDU1NTMgMDAwMDAgbiAKMDAwMDAwNTE1OSAwMDAwMCBuIAowMDAwMDA2ODA2IDAwMDAwIG4gCjAwMDAwMDA3ODYgMDAwMDAgbiAKMDAwMDAwMTEyMyAwMDAwMCBuIAowMDAwMDAxMjg3IDAwMDAwIG4gCjAwMDAwMDE2NjcgMDAwMDAgbiAKMDAwMDAwMTk3MiAwMDAwMCBuIAowMDAwMDAyMjc2IDAwMDAwIG4gCjAwMDAwMDI1OTggMDAwMDAgbiAKMDAwMDAwMjgwNyAwMDAwMCBuIAowMDAwMDAzMjIxIDAwMDAwIG4gCjAwMDAwMDM0NTggMDAwMDAgbiAKMDAwMDAwMzYwMiAwMDAwMCBuIAowMDAwMDAzNzIxIDAwMDAwIG4gCjAwMDAwMDM5NTcgMDAwMDAgbiAKMDAwMDAwNDI2OSAwMDAwMCBuIAowMDAwMDA0NTAyIDAwMDAwIG4gCjAwMDAwMDQ1OTIgMDAwMDAgbiAKMDAwMDAwNDc5OCAwMDAwMCBuIAowMDAwMDA0OTQ1IDAwMDAwIG4gCjAwMDAwMDc3MzYgMDAwMDAgbiAKMDAwMDAwODE4MiAwMDAwMCBuIAowMDAwMDA4MjYyIDAwMDAwIG4gCnRyYWlsZXIKPDwgL0luZm8gMzkgMCBSIC9Sb290IDEgMCBSIC9TaXplIDQwID4+CnN0YXJ0eHJlZgo4NDE5CiUlRU9GCg==\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2021-09-16T14:42:22.838845\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.4.3, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}, {"name": "stdout", "output_type": "stream", "text": ["Layer 4\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUgovR3JvdXAgPDwgL0NTIC9EZXZpY2VSR0IgL1MgL1RyYW5zcGFyZW5jeSAvVHlwZSAvR3JvdXAgPj4KL01lZGlhQm94IFsgMCAwIDM0OS4yIDE4MS43MDY4MTgxODE4IF0gL1BhcmVudCAyIDAgUiAvUmVzb3VyY2VzIDggMCBSCi9UeXBlIC9QYWdlID4+CmVuZG9iago5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTIgMCBSID4+CnN0cmVhbQp4nJ1RS08CMRC+z6/4jnCwO9NStj1CUKI3zCYcjAezFFjCrvIIxn/vrILPi9jJZN6dr18FK8oGgsUOjJXqMwRjZKN0qMp0Ox6i3BFrvibXi8aqtz56EsTk3A8SNMXfwyVRQxvk2taqeGu0fpSf4TZhigbZwLYwRGGIwmCM9QbxDu361uYqZY3sWjB6xIQm2JyGGIuvg21MGxK1FzoLYcPOcez7yBHS98Z9ri9rGhbIrrTNopi/vbaY0R060y58MD663KOTqsVyn2bYdmHZBHc6WinT0746JMyrtJ51cY/ihi4LekcoIZqPZf+mg37TIZFPhNhzCbGsn+Vt6LESIn9nZFg1D9uX8zigVzdDhxgKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyNzAKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNjQgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggOTIgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzMDcgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIzMiA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjMxID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDI0OSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMzYgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDM0MSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE2NCA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDcyID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDQ3ID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTYzID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMzkgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNjAgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE4ID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEzMyA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNzUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTQxID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvQmFzZUZvbnQgL0RlamFWdVNhbnMgL0NoYXJQcm9jcyAxOCAwIFIKL0VuY29kaW5nIDw8Ci9EaWZmZXJlbmNlcyBbIDMyIC9zcGFjZSA2NiAvQiA4NyAvVyA5NyAvYSA5OSAvYyAvZCAvZSAvZiAvZyAvaCAvaSAxMDggL2wgMTEwIC9uIDExMiAvcAoxMTQgL3IgMTE2IC90IDExOCAvdiAxMjEgL3kgXQovVHlwZSAvRW5jb2RpbmcgPj4KL0ZpcnN0Q2hhciAwIC9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnREZXNjcmlwdG9yIDE2IDAgUgovRm9udE1hdHJpeCBbIDAuMDAxIDAgMCAwLjAwMSAwIDAgXSAvTGFzdENoYXIgMjU1IC9OYW1lIC9EZWphVnVTYW5zCi9TdWJ0eXBlIC9UeXBlMyAvVHlwZSAvRm9udCAvV2lkdGhzIDE1IDAgUiA+PgplbmRvYmoKMTYgMCBvYmoKPDwgL0FzY2VudCA5MjkgL0NhcEhlaWdodCAwIC9EZXNjZW50IC0yMzYgL0ZsYWdzIDMyCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnROYW1lIC9EZWphVnVTYW5zIC9JdGFsaWNBbmdsZSAwCi9NYXhXaWR0aCAxMzQyIC9TdGVtViAwIC9UeXBlIC9Gb250RGVzY3JpcHRvciAvWEhlaWdodCAwID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL0NBIDAgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PgovQTIgPDwgL0NBIDEgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9CaXRzUGVyQ29tcG9uZW50IDgKL0NvbG9yU3BhY2UgWy9JbmRleGVkIC9EZXZpY2VSR0IgMjQgKP3nN+vUS7+waqacc5uTdpiRdn18eHh4dmxtcmBkblZcXG1GUGs3RmwuQWwqP20lPW0hO24RNW8ALWkALGYAKmAAXChbACNQACJN/wAAKV0KL0RlY29kZVBhcm1zIDw8IC9Db2xvcnMgMSAvQ29sdW1ucyAxNTMgL1ByZWRpY3RvciAxMCA+PgovRmlsdGVyIC9GbGF0ZURlY29kZSAvSGVpZ2h0IDE1MyAvTGVuZ3RoIDM3IDAgUiAvU3VidHlwZSAvSW1hZ2UKL1R5cGUgL1hPYmplY3QgL1dpZHRoIDE1MyA+PgpzdHJlYW0KeJzt2ElOAzEARNEmTGGGQGjuf1K2LqREKhSJpvP+1tPzzvI0L7XprwEHI+sj6yPrI+sj6yPr+yH7jD6it+gpehzLoVyWW+ZxZGRkZGRkZKuX5cI8YhtdRpuxq+gueo32ERkZGRkZGdnqZbvoIbqOLqJpLJibvNBLREZGRkZGRnZmsmOvoHzO3ES3Y/fRc/QenejHhYyMjIyMjOy/yHLq/kg582tsPklkZGRkZGRkv5YtKLI+sj6yPrI+sj6yPrI+sj6yPrI+sj6yPrI+sj6yPrI+sj6yPrI+sj6yPrI+sj6yPrI+sj6yPrI+sj6yPrI+sj6yPrK+5cq+AdxOGLwKZW5kc3RyZWFtCmVuZG9iagozNyAwIG9iagoyNTQKZW5kb2JqCjE0IDAgb2JqCjw8IC9CaXRzUGVyQ29tcG9uZW50IDggL0NvbG9yU3BhY2UgWy9JbmRleGVkIC9EZXZpY2VSR0IgMiAo/ec3ACJN/wAAKV0KL0RlY29kZVBhcm1zIDw8IC9Db2xvcnMgMSAvQ29sdW1ucyAxNTMgL1ByZWRpY3RvciAxMCA+PgovRmlsdGVyIC9GbGF0ZURlY29kZSAvSGVpZ2h0IDE1MyAvTGVuZ3RoIDM4IDAgUiAvU3VidHlwZSAvSW1hZ2UKL1R5cGUgL1hPYmplY3QgL1dpZHRoIDE1MyA+PgpzdHJlYW0KeJztzsEJgDAAALHW/Yf2K4LgidA+kgky5q7G6sAjs86sM+vMOrPOrLvNxkpmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmavHVfzF2ZmZmZmZmafZxsx68w6s86sM+vMOrPOrDPrzDqzzqwz68w6s86sM+vMOrPOrDPrzDqzzqwz68w6s86sM+vMOrPOrDPrzLp9ZydD3j3rCmVuZHN0cmVhbQplbmRvYmoKMzggMCBvYmoKMTU5CmVuZG9iagoyIDAgb2JqCjw8IC9Db3VudCAxIC9LaWRzIFsgMTEgMCBSIF0gL1R5cGUgL1BhZ2VzID4+CmVuZG9iagozOSAwIG9iago8PCAvQ3JlYXRpb25EYXRlIChEOjIwMjEwOTE2MTQ0MjIzKzAyJzAwJykKL0NyZWF0b3IgKE1hdHBsb3RsaWIgdjMuNC4zLCBodHRwczovL21hdHBsb3RsaWIub3JnKQovUHJvZHVjZXIgKE1hdHBsb3RsaWIgcGRmIGJhY2tlbmQgdjMuNC4zKSA+PgplbmRvYmoKeHJlZgowIDQwCjAwMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAwMDAxNiAwMDAwMCBuIAowMDAwMDA4MjU4IDAwMDAwIG4gCjAwMDAwMDcwMTIgMDAwMDAgbiAKMDAwMDAwNzA0NCAwMDAwMCBuIAowMDAwMDA3MTQzIDAwMDAwIG4gCjAwMDAwMDcxNjQgMDAwMDAgbiAKMDAwMDAwNzE4NSAwMDAwMCBuIAowMDAwMDAwMDY1IDAwMDAwIG4gCjAwMDAwMDA0MDEgMDAwMDAgbiAKMDAwMDAwMDc2NiAwMDAwMCBuIAowMDAwMDAwMjA4IDAwMDAwIG4gCjAwMDAwMDA3NDYgMDAwMDAgbiAKMDAwMDAwNzIyOCAwMDAwMCBuIAowMDAwMDA3ODI1IDAwMDAwIG4gCjAwMDAwMDU3NTMgMDAwMDAgbiAKMDAwMDAwNTU1MyAwMDAwMCBuIAowMDAwMDA1MTU5IDAwMDAwIG4gCjAwMDAwMDY4MDYgMDAwMDAgbiAKMDAwMDAwMDc4NiAwMDAwMCBuIAowMDAwMDAxMTIzIDAwMDAwIG4gCjAwMDAwMDEyODcgMDAwMDAgbiAKMDAwMDAwMTY2NyAwMDAwMCBuIAowMDAwMDAxOTcyIDAwMDAwIG4gCjAwMDAwMDIyNzYgMDAwMDAgbiAKMDAwMDAwMjU5OCAwMDAwMCBuIAowMDAwMDAyODA3IDAwMDAwIG4gCjAwMDAwMDMyMjEgMDAwMDAgbiAKMDAwMDAwMzQ1OCAwMDAwMCBuIAowMDAwMDAzNjAyIDAwMDAwIG4gCjAwMDAwMDM3MjEgMDAwMDAgbiAKMDAwMDAwMzk1NyAwMDAwMCBuIAowMDAwMDA0MjY5IDAwMDAwIG4gCjAwMDAwMDQ1MDIgMDAwMDAgbiAKMDAwMDAwNDU5MiAwMDAwMCBuIAowMDAwMDA0Nzk4IDAwMDAwIG4gCjAwMDAwMDQ5NDUgMDAwMDAgbiAKMDAwMDAwNzgwNSAwMDAwMCBuIAowMDAwMDA4MjM4IDAwMDAwIG4gCjAwMDAwMDgzMTggMDAwMDAgbiAKdHJhaWxlcgo8PCAvSW5mbyAzOSAwIFIgL1Jvb3QgMSAwIFIgL1NpemUgNDAgPj4Kc3RhcnR4cmVmCjg0NzUKJSVFT0YK\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2021-09-16T14:42:22.991369\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.4.3, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}, {"name": "stdout", "output_type": "stream", "text": ["Layer 5\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUgovR3JvdXAgPDwgL0NTIC9EZXZpY2VSR0IgL1MgL1RyYW5zcGFyZW5jeSAvVHlwZSAvR3JvdXAgPj4KL01lZGlhQm94IFsgMCAwIDM0OS4yIDE4MS43MDY4MTgxODE4IF0gL1BhcmVudCAyIDAgUiAvUmVzb3VyY2VzIDggMCBSCi9UeXBlIC9QYWdlID4+CmVuZG9iago5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTIgMCBSID4+CnN0cmVhbQp4nJ1RS08CMRC+z6/4jnCwO9NStj1CUKI3zCYcjAezFFjCrvIIxn/vrILPi9jJZN6dr18FK8oGgsUOjJXqMwRjZKN0qMp0Ox6i3BFrvibXi8aqtz56EsTk3A8SNMXfwyVRQxvk2taqeGu0fpSf4TZhigbZwLYwRGGIwmCM9QbxDu361uYqZY3sWjB6xIQm2JyGGIuvg21MGxK1FzoLYcPOcez7yBHS98Z9ri9rGhbIrrTNopi/vbaY0R060y58MD663KOTqsVyn2bYdmHZBHc6WinT0746JMyrtJ51cY/ihi4LekcoIZqPZf+mg37TIZFPhNhzCbGsn+Vt6LESIn9nZFg1D9uX8zigVzdDhxgKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyNzAKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNjQgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggOTIgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzMDcgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIzMiA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjMxID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDI0OSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMzYgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDM0MSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE2NCA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDcyID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDQ3ID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTYzID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMzkgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNjAgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE4ID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEzMyA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNzUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTQxID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvQmFzZUZvbnQgL0RlamFWdVNhbnMgL0NoYXJQcm9jcyAxOCAwIFIKL0VuY29kaW5nIDw8Ci9EaWZmZXJlbmNlcyBbIDMyIC9zcGFjZSA2NiAvQiA4NyAvVyA5NyAvYSA5OSAvYyAvZCAvZSAvZiAvZyAvaCAvaSAxMDggL2wgMTEwIC9uIDExMiAvcAoxMTQgL3IgMTE2IC90IDExOCAvdiAxMjEgL3kgXQovVHlwZSAvRW5jb2RpbmcgPj4KL0ZpcnN0Q2hhciAwIC9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnREZXNjcmlwdG9yIDE2IDAgUgovRm9udE1hdHJpeCBbIDAuMDAxIDAgMCAwLjAwMSAwIDAgXSAvTGFzdENoYXIgMjU1IC9OYW1lIC9EZWphVnVTYW5zCi9TdWJ0eXBlIC9UeXBlMyAvVHlwZSAvRm9udCAvV2lkdGhzIDE1IDAgUiA+PgplbmRvYmoKMTYgMCBvYmoKPDwgL0FzY2VudCA5MjkgL0NhcEhlaWdodCAwIC9EZXNjZW50IC0yMzYgL0ZsYWdzIDMyCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnROYW1lIC9EZWphVnVTYW5zIC9JdGFsaWNBbmdsZSAwCi9NYXhXaWR0aCAxMzQyIC9TdGVtViAwIC9UeXBlIC9Gb250RGVzY3JpcHRvciAvWEhlaWdodCAwID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL0NBIDAgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PgovQTIgPDwgL0NBIDEgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9CaXRzUGVyQ29tcG9uZW50IDgKL0NvbG9yU3BhY2UgWy9JbmRleGVkIC9EZXZpY2VSR0IgMzcgKP3nN+zVStTBXtK/YM28YqWbc6CXdZSOd4iFeIKAeHt7d3Z2dmZpcFhdbVddbVRabFFYbExUbERPa0FNazdGbC5BbCs/bSQ8bhY2bxQ2bwszcAAvbwAuawAsZgArZAAqYgBcKFsAJlUAJFIAI1AAIk3/AAApXQovRGVjb2RlUGFybXMgPDwgL0NvbG9ycyAxIC9Db2x1bW5zIDE1MyAvUHJlZGljdG9yIDEwID4+Ci9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9IZWlnaHQgMTUzIC9MZW5ndGggMzcgMCBSIC9TdWJ0eXBlIC9JbWFnZQovVHlwZSAvWE9iamVjdCAvV2lkdGggMTUzID4+CnN0cmVhbQp4nO3Zt27DMBgAYaU71ek9cYrz/q+YlTdekMEW7jaCoPRpI35Na/SNPtArekaPY9ziMT6SryNlSpYsWbJkyZLNQcbNN3SPztEJWoydokv0gFaIlGTJkiVLlizZLGSf6AldoEO0g6axPXSMbtAL+kLJkiVLlixZstnLlugI7aPdsQN0hu7QOxK3oGTJkiVLlizZtsi45MSF45FrxEHK1dgt4sdyxsJ7j/i7kyxZsmTJkiXbFhmXdPIg93juZ2z9LyVLlixZsmTJ/izboJL5kvmS+ZL5kvmS+ZL5kvmS+ZL5kvmS+ZL5kvmS+ZL5kvmS+ZL5kvmS+ZL5kvmS+ZL5kvmS+ZL5kvmS+ZL5kvmS+ZL5Nlf2C5hfl6YKZW5kc3RyZWFtCmVuZG9iagozNyAwIG9iagoyNzkKZW5kb2JqCjE0IDAgb2JqCjw8IC9CaXRzUGVyQ29tcG9uZW50IDggL0NvbG9yU3BhY2UgWy9JbmRleGVkIC9EZXZpY2VSR0IgMiAo/ec3ACJN/wAAKV0KL0RlY29kZVBhcm1zIDw8IC9Db2xvcnMgMSAvQ29sdW1ucyAxNTMgL1ByZWRpY3RvciAxMCA+PgovRmlsdGVyIC9GbGF0ZURlY29kZSAvSGVpZ2h0IDE1MyAvTGVuZ3RoIDM4IDAgUiAvU3VidHlwZSAvSW1hZ2UKL1R5cGUgL1hPYmplY3QgL1dpZHRoIDE1MyA+PgpzdHJlYW0KeJztzrEJwDAQwEA7+w+d1ikFb0hxN4DQWgAAAAAAAAAAAADw8Zz2CGfOnDlz5sxZOJvJXOCsc9Y565x1zjpnnbPOWeesc9Y565x1zjpnnbPOWeesc9Y565x1zjpnnbPOWeesc9Y565x1zjpnnbPOWeesc9b99+wFOVgutwplbmRzdHJlYW0KZW5kb2JqCjM4IDAgb2JqCjEyNAplbmRvYmoKMiAwIG9iago8PCAvQ291bnQgMSAvS2lkcyBbIDExIDAgUiBdIC9UeXBlIC9QYWdlcyA+PgplbmRvYmoKMzkgMCBvYmoKPDwgL0NyZWF0aW9uRGF0ZSAoRDoyMDIxMDkxNjE0NDIyMyswMicwMCcpCi9DcmVhdG9yIChNYXRwbG90bGliIHYzLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjQuMykgPj4KZW5kb2JqCnhyZWYKMCA0MAowMDAwMDAwMDAwIDY1NTM1IGYgCjAwMDAwMDAwMTYgMDAwMDAgbiAKMDAwMDAwODI4NiAwMDAwMCBuIAowMDAwMDA3MDEyIDAwMDAwIG4gCjAwMDAwMDcwNDQgMDAwMDAgbiAKMDAwMDAwNzE0MyAwMDAwMCBuIAowMDAwMDA3MTY0IDAwMDAwIG4gCjAwMDAwMDcxODUgMDAwMDAgbiAKMDAwMDAwMDA2NSAwMDAwMCBuIAowMDAwMDAwNDAxIDAwMDAwIG4gCjAwMDAwMDA3NjYgMDAwMDAgbiAKMDAwMDAwMDIwOCAwMDAwMCBuIAowMDAwMDAwNzQ2IDAwMDAwIG4gCjAwMDAwMDcyMjggMDAwMDAgbiAKMDAwMDAwNzg4OCAwMDAwMCBuIAowMDAwMDA1NzUzIDAwMDAwIG4gCjAwMDAwMDU1NTMgMDAwMDAgbiAKMDAwMDAwNTE1OSAwMDAwMCBuIAowMDAwMDA2ODA2IDAwMDAwIG4gCjAwMDAwMDA3ODYgMDAwMDAgbiAKMDAwMDAwMTEyMyAwMDAwMCBuIAowMDAwMDAxMjg3IDAwMDAwIG4gCjAwMDAwMDE2NjcgMDAwMDAgbiAKMDAwMDAwMTk3MiAwMDAwMCBuIAowMDAwMDAyMjc2IDAwMDAwIG4gCjAwMDAwMDI1OTggMDAwMDAgbiAKMDAwMDAwMjgwNyAwMDAwMCBuIAowMDAwMDAzMjIxIDAwMDAwIG4gCjAwMDAwMDM0NTggMDAwMDAgbiAKMDAwMDAwMzYwMiAwMDAwMCBuIAowMDAwMDAzNzIxIDAwMDAwIG4gCjAwMDAwMDM5NTcgMDAwMDAgbiAKMDAwMDAwNDI2OSAwMDAwMCBuIAowMDAwMDA0NTAyIDAwMDAwIG4gCjAwMDAwMDQ1OTIgMDAwMDAgbiAKMDAwMDAwNDc5OCAwMDAwMCBuIAowMDAwMDA0OTQ1IDAwMDAwIG4gCjAwMDAwMDc4NjggMDAwMDAgbiAKMDAwMDAwODI2NiAwMDAwMCBuIAowMDAwMDA4MzQ2IDAwMDAwIG4gCnRyYWlsZXIKPDwgL0luZm8gMzkgMCBSIC9Sb290IDEgMCBSIC9TaXplIDQwID4+CnN0YXJ0eHJlZgo4NTAzCiUlRU9GCg==\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2021-09-16T14:42:23.145226\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.4.3, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}], "source": ["# Initialize convolutions with equal weight to all input pixels\n", "horiz_conv = HorizontalStackConvolution(c_in=1, c_out=1, kernel_size=3, mask_center=False)\n", "horiz_conv.conv.weight.data.fill_(1)\n", "horiz_conv.conv.bias.data.fill_(0)\n", "vert_conv = VerticalStackConvolution(c_in=1, c_out=1, kernel_size=3, mask_center=False)\n", "vert_conv.conv.weight.data.fill_(1)\n", "vert_conv.conv.bias.data.fill_(0)\n", "\n", "# We reuse our convolutions for the 4 layers here. Note that in a standard network,\n", "# we don't do that, and instead learn 4 separate convolution. As this cell is only for\n", "# visualization purposes, we reuse the convolutions for all layers.\n", "for l_idx in range(4):\n", " vert_img = vert_conv(vert_img)\n", " horiz_img = horiz_conv(horiz_img) + vert_img\n", " print(\"Layer %i\" % (l_idx + 2))\n", " show_center_recep_field(inp_img, horiz_img)"]}, {"cell_type": "markdown", "id": "0fb070de", "metadata": {"papermill": {"duration": 0.035948, "end_time": "2021-09-16T12:42:23.289214", "exception": false, "start_time": "2021-09-16T12:42:23.253266", "status": "completed"}, "tags": []}, "source": ["The receptive field above it visualized for the horizontal stack, which includes the features of the vertical convolutions.\n", "It grows over layers without any blind spot as we had before.\n", "The difference between \"weighted\" and \"binary\" receptive field is that for the latter, we check whether there are any gradients flowing back to this pixel.\n", "This indicates that the center pixel indeed can use information from this pixel.\n", "Nevertheless, due to the convolution weights, some pixels have a stronger effect on the prediction than others.\n", "This is visualized in the weighted receptive field by plotting the gradient magnitude for each pixel instead of a binary yes/no.\n", "\n", "\n", "Another receptive field we can check is the one for the vertical stack\n", "as the one above is for the horizontal stack. Let's visualize it below:"]}, {"cell_type": "code", "execution_count": 13, "id": "cd0487f2", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:42:23.363786Z", "iopub.status.busy": "2021-09-16T12:42:23.363266Z", "iopub.status.idle": "2021-09-16T12:42:23.517870Z", "shell.execute_reply": "2021-09-16T12:42:23.517438Z"}, "papermill": {"duration": 0.193071, "end_time": "2021-09-16T12:42:23.517983", "exception": false, "start_time": "2021-09-16T12:42:23.324912", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUgovR3JvdXAgPDwgL0NTIC9EZXZpY2VSR0IgL1MgL1RyYW5zcGFyZW5jeSAvVHlwZSAvR3JvdXAgPj4KL01lZGlhQm94IFsgMCAwIDM0OS4yIDE4MS43MDY4MTgxODE4IF0gL1BhcmVudCAyIDAgUiAvUmVzb3VyY2VzIDggMCBSCi9UeXBlIC9QYWdlID4+CmVuZG9iago5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTIgMCBSID4+CnN0cmVhbQp4nJ1RS08CMRC+z6/4jnCwO9NStj1CUKI3zCYcjAezFFjCrvIIxn/vrILPi9jJZN6dr18FK8oGgsUOjJXqMwRjZKN0qMp0Ox6i3BFrvibXi8aqtz56EsTk3A8SNMXfwyVRQxvk2taqeGu0fpSf4TZhigbZwLYwRGGIwmCM9QbxDu361uYqZY3sWjB6xIQm2JyGGIuvg21MGxK1FzoLYcPOcez7yBHS98Z9ri9rGhbIrrTNopi/vbaY0R060y58MD663KOTqsVyn2bYdmHZBHc6WinT0746JMyrtJ51cY/ihi4LekcoIZqPZf+mg37TIZFPhNhzCbGsn+Vt6LESIn9nZFg1D9uX8zigVzdDhxgKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyNzAKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNjQgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggOTIgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzMDcgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIzMiA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjMxID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDI0OSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMzYgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDM0MSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE2NCA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDcyID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDQ3ID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTYzID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMzkgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNjAgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE4ID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEzMyA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNzUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTQxID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvQmFzZUZvbnQgL0RlamFWdVNhbnMgL0NoYXJQcm9jcyAxOCAwIFIKL0VuY29kaW5nIDw8Ci9EaWZmZXJlbmNlcyBbIDMyIC9zcGFjZSA2NiAvQiA4NyAvVyA5NyAvYSA5OSAvYyAvZCAvZSAvZiAvZyAvaCAvaSAxMDggL2wgMTEwIC9uIDExMiAvcAoxMTQgL3IgMTE2IC90IDExOCAvdiAxMjEgL3kgXQovVHlwZSAvRW5jb2RpbmcgPj4KL0ZpcnN0Q2hhciAwIC9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnREZXNjcmlwdG9yIDE2IDAgUgovRm9udE1hdHJpeCBbIDAuMDAxIDAgMCAwLjAwMSAwIDAgXSAvTGFzdENoYXIgMjU1IC9OYW1lIC9EZWphVnVTYW5zCi9TdWJ0eXBlIC9UeXBlMyAvVHlwZSAvRm9udCAvV2lkdGhzIDE1IDAgUiA+PgplbmRvYmoKMTYgMCBvYmoKPDwgL0FzY2VudCA5MjkgL0NhcEhlaWdodCAwIC9EZXNjZW50IC0yMzYgL0ZsYWdzIDMyCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnROYW1lIC9EZWphVnVTYW5zIC9JdGFsaWNBbmdsZSAwCi9NYXhXaWR0aCAxMzQyIC9TdGVtViAwIC9UeXBlIC9Gb250RGVzY3JpcHRvciAvWEhlaWdodCAwID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL0NBIDAgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PgovQTIgPDwgL0NBIDEgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9CaXRzUGVyQ29tcG9uZW50IDgKL0NvbG9yU3BhY2UgWy9JbmRleGVkIC9EZXZpY2VSR0IgMTYgKP3nN9/KVqWbc5GMd2Rnb05WbDREbCo/bSI7bggzcAAtaQAqYgAmVQAlVAAkUgAiTf8AACldCi9EZWNvZGVQYXJtcyA8PCAvQ29sb3JzIDEgL0NvbHVtbnMgMTUzIC9QcmVkaWN0b3IgMTAgPj4KL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0hlaWdodCAxNTMgL0xlbmd0aCAzNyAwIFIgL1N1YnR5cGUgL0ltYWdlCi9UeXBlIC9YT2JqZWN0IC9XaWR0aCAxNTMgPj4Kc3RyZWFtCnic7dk5agRBEEXB0b4v9z+t3P7mExIMTTy3qMwwC+ryNX1Mb9PL9Dw9HdujvbYjd91SLmRkZGRkZGRnkH1Or9PjdD/dTbfH9miv7chdtxQyMjIyMjKyU8jep32yPEy78Ga6HNujvbYjd91SyMjIyMjIyE4hu95XEBkZGRkZGdkZZNf7u0NGRkZGRkZ2Ptlv+z72NyPJemQ9sh5Zj6xH1vsj2T9E1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrXa/sB+8QQIcKZW5kc3RyZWFtCmVuZG9iagozNyAwIG9iagoyMzUKZW5kb2JqCjE0IDAgb2JqCjw8IC9CaXRzUGVyQ29tcG9uZW50IDggL0NvbG9yU3BhY2UgWy9JbmRleGVkIC9EZXZpY2VSR0IgMiAo/ec3ACJN/wAAKV0KL0RlY29kZVBhcm1zIDw8IC9Db2xvcnMgMSAvQ29sdW1ucyAxNTMgL1ByZWRpY3RvciAxMCA+PgovRmlsdGVyIC9GbGF0ZURlY29kZSAvSGVpZ2h0IDE1MyAvTGVuZ3RoIDM4IDAgUiAvU3VidHlwZSAvSW1hZ2UKL1R5cGUgL1hPYmplY3QgL1dpZHRoIDE1MyA+PgpzdHJlYW0KeJztzsEJwDAQwLC0+w/db76GC5QgDWC8FgAAAAAAAAAAAADXeEa8u5mks85Z56xz1jnrnHVDZwc465x1zjpnnbPOWeesc9Y565x1zjpnnbPOWeesc9Y565x1zjpnnbPOWeesc9Y565x1zjpnnbPOWeesc9Y565x1/z37AGw/MkUKZW5kc3RyZWFtCmVuZG9iagozOCAwIG9iagoxMjgKZW5kb2JqCjIgMCBvYmoKPDwgL0NvdW50IDEgL0tpZHMgWyAxMSAwIFIgXSAvVHlwZSAvUGFnZXMgPj4KZW5kb2JqCjM5IDAgb2JqCjw8IC9DcmVhdGlvbkRhdGUgKEQ6MjAyMTA5MTYxNDQyMjMrMDInMDAnKQovQ3JlYXRvciAoTWF0cGxvdGxpYiB2My40LjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcpCi9Qcm9kdWNlciAoTWF0cGxvdGxpYiBwZGYgYmFja2VuZCB2My40LjMpID4+CmVuZG9iagp4cmVmCjAgNDAKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDgxODIgMDAwMDAgbiAKMDAwMDAwNzAxMiAwMDAwMCBuIAowMDAwMDA3MDQ0IDAwMDAwIG4gCjAwMDAwMDcxNDMgMDAwMDAgbiAKMDAwMDAwNzE2NCAwMDAwMCBuIAowMDAwMDA3MTg1IDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDQwMSAwMDAwMCBuIAowMDAwMDAwNzY2IDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDc0NiAwMDAwMCBuIAowMDAwMDA3MjI4IDAwMDAwIG4gCjAwMDAwMDc3ODAgMDAwMDAgbiAKMDAwMDAwNTc1MyAwMDAwMCBuIAowMDAwMDA1NTUzIDAwMDAwIG4gCjAwMDAwMDUxNTkgMDAwMDAgbiAKMDAwMDAwNjgwNiAwMDAwMCBuIAowMDAwMDAwNzg2IDAwMDAwIG4gCjAwMDAwMDExMjMgMDAwMDAgbiAKMDAwMDAwMTI4NyAwMDAwMCBuIAowMDAwMDAxNjY3IDAwMDAwIG4gCjAwMDAwMDE5NzIgMDAwMDAgbiAKMDAwMDAwMjI3NiAwMDAwMCBuIAowMDAwMDAyNTk4IDAwMDAwIG4gCjAwMDAwMDI4MDcgMDAwMDAgbiAKMDAwMDAwMzIyMSAwMDAwMCBuIAowMDAwMDAzNDU4IDAwMDAwIG4gCjAwMDAwMDM2MDIgMDAwMDAgbiAKMDAwMDAwMzcyMSAwMDAwMCBuIAowMDAwMDAzOTU3IDAwMDAwIG4gCjAwMDAwMDQyNjkgMDAwMDAgbiAKMDAwMDAwNDUwMiAwMDAwMCBuIAowMDAwMDA0NTkyIDAwMDAwIG4gCjAwMDAwMDQ3OTggMDAwMDAgbiAKMDAwMDAwNDk0NSAwMDAwMCBuIAowMDAwMDA3NzYwIDAwMDAwIG4gCjAwMDAwMDgxNjIgMDAwMDAgbiAKMDAwMDAwODI0MiAwMDAwMCBuIAp0cmFpbGVyCjw8IC9JbmZvIDM5IDAgUiAvUm9vdCAxIDAgUiAvU2l6ZSA0MCA+PgpzdGFydHhyZWYKODM5OQolJUVPRgo=\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2021-09-16T14:42:23.446589\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.4.3, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}], "source": ["show_center_recep_field(inp_img, vert_img)"]}, {"cell_type": "markdown", "id": "bbe7c460", "metadata": {"papermill": {"duration": 0.036988, "end_time": "2021-09-16T12:42:23.591980", "exception": false, "start_time": "2021-09-16T12:42:23.554992", "status": "completed"}, "tags": []}, "source": ["As we have discussed before, the vertical stack only looks at pixels above the one we want to predict.\n", "Hence, we can validate that our implementation works as we initially expected it to.\n", "As a final step, let's clean up the computation graph we still had kept\n", "in memory for the visualization of the receptive field:"]}, {"cell_type": "code", "execution_count": 14, "id": "e76aa938", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:42:23.668813Z", "iopub.status.busy": "2021-09-16T12:42:23.668347Z", "iopub.status.idle": "2021-09-16T12:42:23.670487Z", "shell.execute_reply": "2021-09-16T12:42:23.670021Z"}, "papermill": {"duration": 0.041448, "end_time": "2021-09-16T12:42:23.670586", "exception": false, "start_time": "2021-09-16T12:42:23.629138", "status": "completed"}, "tags": []}, "outputs": [], "source": ["del inp_img, horiz_conv, vert_conv"]}, {"cell_type": "markdown", "id": "193e3fb4", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.037077, "end_time": "2021-09-16T12:42:23.744338", "exception": false, "start_time": "2021-09-16T12:42:23.707261", "status": "completed"}, "tags": []}, "source": ["## Gated PixelCNN\n", "\n", "
\n", "\n", "In the next step, we will use the masked convolutions to build a full autoregressive model, called Gated PixelCNN.\n", "The difference between the original PixelCNN and Gated PixelCNN is the use of separate horizontal and vertical stacks.\n", "However, in literature, you often see that people refer to the Gated PixelCNN simply as \"PixelCNN\".\n", "Hence, in the following, if we say \"PixelCNN\", we usually mean the gated version.\n", "What \"Gated\" refers to in the model name is explained next.\n", "\n", "### Gated Convolutions\n", "\n", "For visualizing the receptive field, we assumed a very simplified stack of vertical and horizontal convolutions.\n", "Obviously, there are more sophisticated ways of doing it, and PixelCNN uses gated convolutions for this.\n", "Specifically, the Gated Convolution block in PixelCNN looks as follows\n", "(figure credit - [Aaron van den Oord et al. ](https://arxiv.org/pdf/1606.05328.pdf)):\n", "\n", "
\n", "\n", "The left path is the vertical stack (the $N\\times N$ convolution is masked correspondingly),\n", "and the right path is the horizontal stack.\n", "Gated convolutions are implemented by having a twice as large output channel size,\n", "and combine them by a element-wise multiplication of $\\tanh$ and a sigmoid.\n", "For a linear layer, we can express a gated activation unit as follows:\n", "\n", "$$\\mathbf{y} = \\tanh\\left(\\mathbf{W}_{f}\\mathbf{x}\\right)\\odot\\sigma\\left(\\mathbf{W}_{g}\\mathbf{x}\\right)$$\n", "\n", "For simplicity, biases have been neglected and the linear layer split into two part, $\\mathbf{W}_{f}$ and $\\mathbf{W}_{g}$.\n", "This concept resembles the input and modulation gate in an LSTM, and has been used in many other architectures as well.\n", "The main motivation behind this gated activation is that it might allow to model more complex interactions and simplifies learning.\n", "But as in any other architecture, this is mostly a design choice and can be considered a hyperparameters.\n", "\n", "Besides the gated convolutions, we also see that the horizontal stack uses a residual connection while the vertical stack does not.\n", "This is because we use the output of the horizontal stack for prediction.\n", "Each convolution in the vertical stack also receives a strong gradient signal\n", "as it is only two $1\\times 1$ convolutions away from the residual connection,\n", "and does not require another residual connection to all its earleri layers.\n", "\n", "The implementation in PyTorch is fairly straight forward for this block,\n", "because the visualization above gives us a computation graph to follow:"]}, {"cell_type": "code", "execution_count": 15, "id": "cc9d393c", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:42:23.919095Z", "iopub.status.busy": "2021-09-16T12:42:23.918572Z", "iopub.status.idle": "2021-09-16T12:42:23.920520Z", "shell.execute_reply": "2021-09-16T12:42:23.920095Z"}, "lines_to_next_cell": 2, "papermill": {"duration": 0.138888, "end_time": "2021-09-16T12:42:23.920638", "exception": false, "start_time": "2021-09-16T12:42:23.781750", "status": "completed"}, "tags": []}, "outputs": [], "source": ["class GatedMaskedConv(nn.Module):\n", " def __init__(self, c_in, **kwargs):\n", " \"\"\"Gated Convolution block implemented the computation graph shown above.\"\"\"\n", " super().__init__()\n", " self.conv_vert = VerticalStackConvolution(c_in, c_out=2 * c_in, **kwargs)\n", " self.conv_horiz = HorizontalStackConvolution(c_in, c_out=2 * c_in, **kwargs)\n", " self.conv_vert_to_horiz = nn.Conv2d(2 * c_in, 2 * c_in, kernel_size=1, padding=0)\n", " self.conv_horiz_1x1 = nn.Conv2d(c_in, c_in, kernel_size=1, padding=0)\n", "\n", " def forward(self, v_stack, h_stack):\n", " # Vertical stack (left)\n", " v_stack_feat = self.conv_vert(v_stack)\n", " v_val, v_gate = v_stack_feat.chunk(2, dim=1)\n", " v_stack_out = torch.tanh(v_val) * torch.sigmoid(v_gate)\n", "\n", " # Horizontal stack (right)\n", " h_stack_feat = self.conv_horiz(h_stack)\n", " h_stack_feat = h_stack_feat + self.conv_vert_to_horiz(v_stack_feat)\n", " h_val, h_gate = h_stack_feat.chunk(2, dim=1)\n", " h_stack_feat = torch.tanh(h_val) * torch.sigmoid(h_gate)\n", " h_stack_out = self.conv_horiz_1x1(h_stack_feat)\n", " h_stack_out = h_stack_out + h_stack\n", "\n", " return v_stack_out, h_stack_out"]}, {"cell_type": "markdown", "id": "2b8ce7cc", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.037081, "end_time": "2021-09-16T12:42:23.995828", "exception": false, "start_time": "2021-09-16T12:42:23.958747", "status": "completed"}, "tags": []}, "source": ["### Building the model\n", "\n", "Using the gated convolutions, we can now build our PixelCNN model.\n", "The architecture consists of multiple stacked GatedMaskedConv blocks, where we add an additional dilation factor to a few convolutions.\n", "This is used to increase the receptive field of the model and allows to take a larger context into accout during generation.\n", "As a reminder, dilation on a convolution works looks as follows\n", "(figure credit - [Vincent Dumoulin and Francesco Visin](https://arxiv.org/pdf/1603.07285.pdf)):\n", "\n", "
\n", "\n", "Note that the smaller output size is only because the animation assumes no padding.\n", "In our implementation, we will pad the input image correspondingly.\n", "Alternatively to dilated convolutions, we could downsample the input and use a encoder-decoder architecture as in PixelCNN++ [3].\n", "This is especially beneficial if we want to build a very deep autoregressive model.\n", "Nonetheless, as we seek to train a reasonably small model, dilated convolutions are the more efficient option to use here.\n", "\n", "Below, we implement the PixelCNN model as a PyTorch Lightning module.\n", "Besides the stack of gated convolutions, we also have the initial\n", "horizontal and vertical convolutions which mask the center pixel, and a\n", "final $1\\times 1$ convolution which maps the output features to class\n", "predictions. To determine the likelihood of a batch of images, we first\n", "create our initial features using the masked horizontal and vertical\n", "input convolution. Next, we forward the features through the stack of\n", "gated convolutions. Finally, we take the output features of the\n", "horizontal stack, and apply the $1\\times 1$ convolution for\n", "classification. We use the bits per dimension metric for the likelihood,\n", "similarly to Tutorial 11 and assignment 3."]}, {"cell_type": "code", "execution_count": 16, "id": "16fca558", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:42:24.082322Z", "iopub.status.busy": "2021-09-16T12:42:24.081826Z", "iopub.status.idle": "2021-09-16T12:42:24.084319Z", "shell.execute_reply": "2021-09-16T12:42:24.083832Z"}, "papermill": {"duration": 0.051889, "end_time": "2021-09-16T12:42:24.084419", "exception": false, "start_time": "2021-09-16T12:42:24.032530", "status": "completed"}, "tags": []}, "outputs": [], "source": ["class PixelCNN(pl.LightningModule):\n", " def __init__(self, c_in, c_hidden):\n", " super().__init__()\n", " self.save_hyperparameters()\n", "\n", " # Initial convolutions skipping the center pixel\n", " self.conv_vstack = VerticalStackConvolution(c_in, c_hidden, mask_center=True)\n", " self.conv_hstack = HorizontalStackConvolution(c_in, c_hidden, mask_center=True)\n", " # Convolution block of PixelCNN. We use dilation instead of downscaling\n", " self.conv_layers = nn.ModuleList(\n", " [\n", " GatedMaskedConv(c_hidden),\n", " GatedMaskedConv(c_hidden, dilation=2),\n", " GatedMaskedConv(c_hidden),\n", " GatedMaskedConv(c_hidden, dilation=4),\n", " GatedMaskedConv(c_hidden),\n", " GatedMaskedConv(c_hidden, dilation=2),\n", " GatedMaskedConv(c_hidden),\n", " ]\n", " )\n", " # Output classification convolution (1x1)\n", " self.conv_out = nn.Conv2d(c_hidden, c_in * 256, kernel_size=1, padding=0)\n", "\n", " self.example_input_array = train_set[0][0][None]\n", "\n", " def forward(self, x):\n", " \"\"\"Forward image through model and return logits for each pixel.\n", "\n", " Args:\n", " x: Image tensor with integer values between 0 and 255.\n", " \"\"\"\n", " # Scale input from 0 to 255 back to -1 to 1\n", " x = (x.float() / 255.0) * 2 - 1\n", "\n", " # Initial convolutions\n", " v_stack = self.conv_vstack(x)\n", " h_stack = self.conv_hstack(x)\n", " # Gated Convolutions\n", " for layer in self.conv_layers:\n", " v_stack, h_stack = layer(v_stack, h_stack)\n", " # 1x1 classification convolution\n", " # Apply ELU before 1x1 convolution for non-linearity on residual connection\n", " out = self.conv_out(F.elu(h_stack))\n", "\n", " # Output dimensions: [Batch, Classes, Channels, Height, Width]\n", " out = out.reshape(out.shape[0], 256, out.shape[1] // 256, out.shape[2], out.shape[3])\n", " return out\n", "\n", " def calc_likelihood(self, x):\n", " # Forward pass with bpd likelihood calculation\n", " pred = self.forward(x)\n", " nll = F.cross_entropy(pred, x, reduction=\"none\")\n", " bpd = nll.mean(dim=[1, 2, 3]) * np.log2(np.exp(1))\n", " return bpd.mean()\n", "\n", " @torch.no_grad()\n", " def sample(self, img_shape, img=None):\n", " \"\"\"Sampling function for the autoregressive model.\n", "\n", " Args:\n", " img_shape: Shape of the image to generate (B,C,H,W)\n", " img (optional): If given, this tensor will be used as\n", " a starting image. The pixels to fill\n", " should be -1 in the input tensor.\n", " \"\"\"\n", " # Create empty image\n", " if img is None:\n", " img = torch.zeros(img_shape, dtype=torch.long).to(device) - 1\n", " # Generation loop\n", " for h in tqdm(range(img_shape[2]), leave=False):\n", " for w in range(img_shape[3]):\n", " for c in range(img_shape[1]):\n", " # Skip if not to be filled (-1)\n", " if (img[:, c, h, w] != -1).all().item():\n", " continue\n", " # For efficiency, we only have to input the upper part of the image\n", " # as all other parts will be skipped by the masked convolutions anyways\n", " pred = self.forward(img[:, :, : h + 1, :])\n", " probs = F.softmax(pred[:, :, c, h, w], dim=-1)\n", " img[:, c, h, w] = torch.multinomial(probs, num_samples=1).squeeze(dim=-1)\n", " return img\n", "\n", " def configure_optimizers(self):\n", " optimizer = optim.Adam(self.parameters(), lr=1e-3)\n", " scheduler = optim.lr_scheduler.StepLR(optimizer, 1, gamma=0.99)\n", " return [optimizer], [scheduler]\n", "\n", " def training_step(self, batch, batch_idx):\n", " loss = self.calc_likelihood(batch[0])\n", " self.log(\"train_bpd\", loss)\n", " return loss\n", "\n", " def validation_step(self, batch, batch_idx):\n", " loss = self.calc_likelihood(batch[0])\n", " self.log(\"val_bpd\", loss)\n", "\n", " def test_step(self, batch, batch_idx):\n", " loss = self.calc_likelihood(batch[0])\n", " self.log(\"test_bpd\", loss)"]}, {"cell_type": "markdown", "id": "0a771ffa", "metadata": {"papermill": {"duration": 0.037059, "end_time": "2021-09-16T12:42:24.159136", "exception": false, "start_time": "2021-09-16T12:42:24.122077", "status": "completed"}, "tags": []}, "source": ["To sample from the autoregressive model, we need to iterate over all dimensions of the input.\n", "We start with an empty image, and fill the pixels one by one, starting from the upper left corner.\n", "Note that as for predicting $x_i$, all pixels below it have no influence on the prediction.\n", "Hence, we can cut the image in height without changing the prediction while increasing efficiency.\n", "Nevertheless, all the loops in the sampling function already show that it will take us quite some time to sample.\n", "A lot of computation could be reused across loop iterations as those the features on the already predicted pixels will not change over iterations.\n", "Nevertheless, this takes quite some effort to implement, and is often not done in implementations because in the end,\n", "autoregressive sampling remains sequential and slow.\n", "Hence, we settle with the default implementation here.\n", "\n", "Before training the model, we can check the full receptive field of the model on an MNIST image of size $28\\times 28$:"]}, {"cell_type": "code", "execution_count": 17, "id": "546c40c3", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:42:24.236461Z", "iopub.status.busy": "2021-09-16T12:42:24.236001Z", "iopub.status.idle": "2021-09-16T12:42:24.487057Z", "shell.execute_reply": "2021-09-16T12:42:24.486641Z"}, "papermill": {"duration": 0.291233, "end_time": "2021-09-16T12:42:24.487172", "exception": false, "start_time": "2021-09-16T12:42:24.195939", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUgovR3JvdXAgPDwgL0NTIC9EZXZpY2VSR0IgL1MgL1RyYW5zcGFyZW5jeSAvVHlwZSAvR3JvdXAgPj4KL01lZGlhQm94IFsgMCAwIDM0OS4yIDE4MS43MDY4MTgxODE4IF0gL1BhcmVudCAyIDAgUiAvUmVzb3VyY2VzIDggMCBSCi9UeXBlIC9QYWdlID4+CmVuZG9iago5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTIgMCBSID4+CnN0cmVhbQp4nJ1RS08CMRC+z6/4jnCwO9NStj1CUKI3zCYcjAezFFjCrvIIxn/vrILPi9jJZN6dr18FK8oGgsUOjJXqMwRjZKN0qMp0Ox6i3BFrvibXi8aqtz56EsTk3A8SNMXfwyVRQxvk2taqeGu0fpSf4TZhigbZwLYwRGGIwmCM9QbxDu361uYqZY3sWjB6xIQm2JyGGIuvg21MGxK1FzoLYcPOcez7yBHS98Z9ri9rGhbIrrTNopi/vbaY0R060y58MD663KOTqsVyn2bYdmHZBHc6WinT0746JMyrtJ51cY/ihi4LekcoIZqPZf+mg37TIZFPhNhzCbGsn+Vt6LESIn9nZFg1D9uX8zigVzdDhxgKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoyNzAKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNjQgPj4Kc3RyZWFtCnicPVK5kQMxDMu3CpbAX1I9vrlxYPefHsA9OzExSxEAQfduUTkbP20ly4/82GVnSeSW96Bt8rqi6gapmyBzyXHJFWKW8rgSJBYtZSbWZ6qD8nENahW+8BMzE9MhScQSclKPIqyPr4PX9RzcXzfp/BoZH3RsUHpLkqugVMO+crQSO5bqXYPsg6ab0uoz067sgKXxhqzUpE5/HfW/IyzCpN5IRQchFr/Tyx2yLKQc1Nu5fgPhXe2CWE+tPOwQad5WeksVzObMBEwZbTusgLVZY8JUCfAWzSHUWojMFzqtiPnk3NHBNFV5GiLDUoWD7T2jNVzoGhJLw/lJxgTmv/D6rMhtf/8AcGhnugplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggOTIgPj4Kc3RyZWFtCnicPY3BDcAwCAP/TMEIEALE+1RVH+n+3yYR6gcfBtkYYGGzNeDB2cCX0to3vaRFk9oIVrVF3VCeuxSlWF1HpUzCT5k7f1J0HO1wDtvf1uU4TePoX/fQ/QEPSh4LCmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzMDcgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIzMiA+PgpzdHJlYW0KeJw1UUluxDAMu/sV/MAA1u68J8Wgh/b/11LKFAhAJba4JWJjIwIvMfg5iNz4kjWjJn5nclf8LE+FR8Kt4EkUgZfhXnaCyxvGZT8OMx+8l1bOpMaTDMhFNj08ETLYJRA6MLsGddhm2om+IeGzI1LNRpbT1xL00ioEylO23+mCEm2r+nP7rAtt+9oTTnZ76knlE4jnlqzAZeMVk8VYBj1RuUsxfZDqbKEnobwon4NsPmqIRJcoZ+CJwcEo0A7sue1n4lUhaF3dp21jqEZKx9O/DU1Nkgj5RAlntjTuFv5/z72+1/sPTiFUEQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjMxID4+CnN0cmVhbQp4nDVPOZIEIQzLeYU+MFUY20C/p6e2Ntj5f7qSmU6Q8CHJ0xMdmXiZIyOwZsfbWmQgZuBTTMW/9rQPE6r34B4ilIsLYYaRcNas426ejhf/dpXPWAfvNviKWV4Q2MJM1lcWZy7bBWNpnMQ5yW6MXROxjXWtp1NYRzChDIR0tsOUIHNUpPTJjjLm6DiRJ56L7/bbLHY5fg7rCzaNIRXn+Cp6gjaDoux57wIackH/Xd34HkW76CUgGwkW1lFi7pzlhF+9dnQetSgSc0KaQS4TIc3pKqYQmlCss6OgUlFwqT6n6Kyff+VfXC0KZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDI0OSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMzYgPj4Kc3RyZWFtCnicTY9BDgMxCAPveYWfQCBAeM9WVQ/b/19L2HbTCx7JgGxRBoElh3iHG+HR2w/fRTYVZ+OcX1IpYiGYT3CfMFMcjSl38mOPgHGUaiynaHheS85NwxctdxMtpa2XkxlvuO6X90eVbZENRc8tC0LXbJL5MoEHfBiYR3XjaaXH3fZsr/b8AM5sNEkKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDM0MSA+PgpzdHJlYW0KeJxFUktuRDEI279TcIFI4ZeQ87Squpjef1ubTNXN4AlgbHjLU6ZkyrC5JSMk15RPfSJDrKb8NHIkIqb4SQkFdpWPx2tLrI3skagUn9rx47H0RqbZFVr17tGlzaJRzcrIOcgQoZ4VurJ71A7Z8HpcSLrvlM0hHMv/UIEsZd1yCiVBW9B37BHfDx2ugiuCYbBrLoPtZTLU//qHFlzvffdixy6AFqznvsEOAKinE7QFyBna7jYpaABVuotJwqPyem52omyjVen5HAAzDjBywIglWx2+0d4Aln1d6EWNiv0rQFFZQPzI1XbB3jHJSHAW5gaOvXA8xZlwSzjGAkCKveIYevAl2OYvV66ImvAJdbpkL7zCntrm50KTCHetAA5eZMOtq6Oolu3pPIL2Z0VyRozUizg6IZJa0jmC4tKgHlrjXDex4m0jsblX3+4f4ZwvXPbrF0vshMQKZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE2NCA+PgpzdHJlYW0KeJxFkMdxBTEMQ++qAiUwgAr1rMfzD+v+r4b000F6GEIMYk/CsFxXcWF0w4+3LTMNf0cZ7sb6MmO81VggJ+gDDJGJq9Gk+nbFGar05NVirqOiXC86IhLMkuOrQCN8OrLHk7a2M/10Xh/sIe8T/yoq525hAS6q7kD5Uh/x1I/ZUeqaoY8qK2seatpXhF0RSts+LqcyTt29A1rhvZWrPdrvPx52OvIKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDcyID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXEC+qYm5Qi4XSAzEygGzDIC0JZyCiGeAmCBtEMUgFkSxmYkZRB2cAZHL4EoDACXbFskKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDQ3ID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTYzID4+CnN0cmVhbQp4nEWQOxIDIQxDe06hI/gjAz7PZjIpNvdvY9hsUsDTWCCDuxOC1NqCieiCh7Yl3QXvrQRnY/zpNm41EuQEdYBWpONolFJ9ucVplXTxaDZzKwutEx1mDnqUoxmgEDoV3u2i5HKm7s75Q3D1X/W/Yt05m4mBycodCM3qU9z5NjuiurrJ/qTH3KzXfivsVWFpWUvLCbedu2ZACdxTOdqrPT8fCjr2CmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMzkgPj4Kc3RyZWFtCnicTVDJbQQxDPu7CjUwwOgcux4Hizyy/X9DygmSl2hL4qHylFuWymX3IzlvybrlQ4dOlWnybtDNr7H+owwCdv9QVBCtJbFKzFzSbrE0SS/ZwziNl2u1juepe4RZo3jw49jTKYHpPTLBZrO9OTCrPc4OkE64xq/q0zuVJAOJupDzQqUK6x7UJaKPK9uYUp1OLeUYl5/oe3yOAD3F3o3c0cfLF4xGtS2o0WqVOA8wE1PRlXGrkYGUEwZDZ0dXNAulyMp6QjXCjTmhmb3DcGADy7OEpKWtUrwPZQHoAl3aOuM0SoKOAMLfKIz1+gaq/F43CmVuZHN0cmVhbQplbmRvYmoKMzIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNjAgPj4Kc3RyZWFtCnicRZA5EgMxCARzvYInSFyC96zLtcH6/6kH1kei6QI0HLoWTcp6FGg+6bFGobrQa+gsSpJEwRaSHVCnY4g7KEhMSGOSSLYegyOaWLNdmJlUKrNS4bRpxcK/2VrVyESNcI38iekGVPxP6lyU8E2Dr5Ix+hhUvDuDjEn4XkXcWjHt/kQwsRn2CW9FJgWEibGp2b7PYIbM9wrXOMfzDUyCN+sKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE4ID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozNCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEzMyA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNzUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTQxID4+CnN0cmVhbQp4nD2PwQ7DMAhD7/kK/0Ck2CmhfE+naofu/68jS7sLegJjjIXQ0BuqmsOGYJvjxdIlVGv4FMVAJTfImWAOpaTSHUeRemI4GFwetBuO4rHo+hG7kmZ90MZCuiVogHusU2ncpnETxB01Beop6pyjvBC5n6ln2DSS3TSzknO4Db97z1PX/6ervMv5Bb13Lv4KZW5kc3RyZWFtCmVuZG9iagoxNyAwIG9iago8PCAvQmFzZUZvbnQgL0RlamFWdVNhbnMgL0NoYXJQcm9jcyAxOCAwIFIKL0VuY29kaW5nIDw8Ci9EaWZmZXJlbmNlcyBbIDMyIC9zcGFjZSA2NiAvQiA4NyAvVyA5NyAvYSA5OSAvYyAvZCAvZSAvZiAvZyAvaCAvaSAxMDggL2wgMTEwIC9uIDExMiAvcAoxMTQgL3IgMTE2IC90IDExOCAvdiAxMjEgL3kgXQovVHlwZSAvRW5jb2RpbmcgPj4KL0ZpcnN0Q2hhciAwIC9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnREZXNjcmlwdG9yIDE2IDAgUgovRm9udE1hdHJpeCBbIDAuMDAxIDAgMCAwLjAwMSAwIDAgXSAvTGFzdENoYXIgMjU1IC9OYW1lIC9EZWphVnVTYW5zCi9TdWJ0eXBlIC9UeXBlMyAvVHlwZSAvRm9udCAvV2lkdGhzIDE1IDAgUiA+PgplbmRvYmoKMTYgMCBvYmoKPDwgL0FzY2VudCA5MjkgL0NhcEhlaWdodCAwIC9EZXNjZW50IC0yMzYgL0ZsYWdzIDMyCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnROYW1lIC9EZWphVnVTYW5zIC9JdGFsaWNBbmdsZSAwCi9NYXhXaWR0aCAxMzQyIC9TdGVtViAwIC9UeXBlIC9Gb250RGVzY3JpcHRvciAvWEhlaWdodCAwID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAvQiAxOSAwIFIgL1cgMjAgMCBSIC9hIDIxIDAgUiAvYyAyMiAwIFIgL2QgMjMgMCBSIC9lIDI0IDAgUiAvZiAyNSAwIFIKL2cgMjYgMCBSIC9oIDI3IDAgUiAvaSAyOCAwIFIgL2wgMjkgMCBSIC9uIDMwIDAgUiAvcCAzMSAwIFIgL3IgMzIgMCBSCi9zcGFjZSAzMyAwIFIgL3QgMzQgMCBSIC92IDM1IDAgUiAveSAzNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE3IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL0NBIDAgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PgovQTIgPDwgL0NBIDEgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSIC9JMiAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9CaXRzUGVyQ29tcG9uZW50IDgKL0NvbG9yU3BhY2UgWy9JbmRleGVkIC9EZXZpY2VSR0IgMTUgKP3nNyE7bh05bgQycAAxcAAsZwArZABcKFxcAFwoWwAmVwAmVQAkUgAjUAAjTwAiTf8AACldCi9EZWNvZGVQYXJtcyA8PCAvQ29sb3JzIDEgL0NvbHVtbnMgMTUzIC9QcmVkaWN0b3IgMTAgPj4KL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0hlaWdodCAxNTMgL0xlbmd0aCAzNyAwIFIgL1N1YnR5cGUgL0ltYWdlCi9UeXBlIC9YT2JqZWN0IC9XaWR0aCAxNTMgPj4Kc3RyZWFtCnic7dTLSgNBEEDR+H5G//9vpSuLDATBi0hiOHfVTcH02dTsPi613bkB30bWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+uR9ch6ZD2yHlmPrEfWI+tdn+z92Nz3q839bZr7jPZkZFclO3L2m+ePhs1o4yUjI/sT2Wb956dwOD6vHlcHw8vqBE1G9v9l08m+jeRhdT/N6GnKHycjuyDZ4fnX1WG/5ng7jeRuulntVp/TbzhkZGQ/kJ03sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6xH1iPrkfXIemQ9sh5Zj6z3BSeE7iMKZW5kc3RyZWFtCmVuZG9iagozNyAwIG9iagoyNjIKZW5kb2JqCjE0IDAgb2JqCjw8IC9CaXRzUGVyQ29tcG9uZW50IDggL0NvbG9yU3BhY2UgWy9JbmRleGVkIC9EZXZpY2VSR0IgMiAo/ec3ACJN/wAAKV0KL0RlY29kZVBhcm1zIDw8IC9Db2xvcnMgMSAvQ29sdW1ucyAxNTMgL1ByZWRpY3RvciAxMCA+PgovRmlsdGVyIC9GbGF0ZURlY29kZSAvSGVpZ2h0IDE1MyAvTGVuZ3RoIDM4IDAgUiAvU3VidHlwZSAvSW1hZ2UKL1R5cGUgL1hPYmplY3QgL1dpZHRoIDE1MyA+PgpzdHJlYW0KeJztzrEJwDAQADE7+w8dSJ/iwOAvpAm09lTrduCXWTd9tiYy68w6s86sM+vMOrPOrDPrzDqzzqwz68w6s86sM+vMOrPOrDPrzDqzzqwz68w6s86sM+vMOrPOrDPrzDqz7uTs+ewzzMzMZjLrzDqzzqwz68w6s86sM+vMOrPOrDPrzDqzzqwz68w6s86sM+vMOrPOrDPrzDqzzqwz68w6s86sM+vMOrPuBTtEMOAKZW5kc3RyZWFtCmVuZG9iagozOCAwIG9iagoxNzMKZW5kb2JqCjIgMCBvYmoKPDwgL0NvdW50IDEgL0tpZHMgWyAxMSAwIFIgXSAvVHlwZSAvUGFnZXMgPj4KZW5kb2JqCjM5IDAgb2JqCjw8IC9DcmVhdGlvbkRhdGUgKEQ6MjAyMTA5MTYxNDQyMjQrMDInMDAnKQovQ3JlYXRvciAoTWF0cGxvdGxpYiB2My40LjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcpCi9Qcm9kdWNlciAoTWF0cGxvdGxpYiBwZGYgYmFja2VuZCB2My40LjMpID4+CmVuZG9iagp4cmVmCjAgNDAKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDgyNTQgMDAwMDAgbiAKMDAwMDAwNzAxMiAwMDAwMCBuIAowMDAwMDA3MDQ0IDAwMDAwIG4gCjAwMDAwMDcxNDMgMDAwMDAgbiAKMDAwMDAwNzE2NCAwMDAwMCBuIAowMDAwMDA3MTg1IDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDQwMSAwMDAwMCBuIAowMDAwMDAwNzY2IDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDc0NiAwMDAwMCBuIAowMDAwMDA3MjI4IDAwMDAwIG4gCjAwMDAwMDc4MDcgMDAwMDAgbiAKMDAwMDAwNTc1MyAwMDAwMCBuIAowMDAwMDA1NTUzIDAwMDAwIG4gCjAwMDAwMDUxNTkgMDAwMDAgbiAKMDAwMDAwNjgwNiAwMDAwMCBuIAowMDAwMDAwNzg2IDAwMDAwIG4gCjAwMDAwMDExMjMgMDAwMDAgbiAKMDAwMDAwMTI4NyAwMDAwMCBuIAowMDAwMDAxNjY3IDAwMDAwIG4gCjAwMDAwMDE5NzIgMDAwMDAgbiAKMDAwMDAwMjI3NiAwMDAwMCBuIAowMDAwMDAyNTk4IDAwMDAwIG4gCjAwMDAwMDI4MDcgMDAwMDAgbiAKMDAwMDAwMzIyMSAwMDAwMCBuIAowMDAwMDAzNDU4IDAwMDAwIG4gCjAwMDAwMDM2MDIgMDAwMDAgbiAKMDAwMDAwMzcyMSAwMDAwMCBuIAowMDAwMDAzOTU3IDAwMDAwIG4gCjAwMDAwMDQyNjkgMDAwMDAgbiAKMDAwMDAwNDUwMiAwMDAwMCBuIAowMDAwMDA0NTkyIDAwMDAwIG4gCjAwMDAwMDQ3OTggMDAwMDAgbiAKMDAwMDAwNDk0NSAwMDAwMCBuIAowMDAwMDA3Nzg3IDAwMDAwIG4gCjAwMDAwMDgyMzQgMDAwMDAgbiAKMDAwMDAwODMxNCAwMDAwMCBuIAp0cmFpbGVyCjw8IC9JbmZvIDM5IDAgUiAvUm9vdCAxIDAgUiAvU2l6ZSA0MCA+PgpzdGFydHhyZWYKODQ3MQolJUVPRgo=\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2021-09-16T14:42:24.416262\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.4.3, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}], "source": ["test_model = PixelCNN(c_in=1, c_hidden=64)\n", "inp = torch.zeros(1, 1, 28, 28)\n", "inp.requires_grad_()\n", "out = test_model(inp)\n", "show_center_recep_field(inp, out.squeeze(dim=2))\n", "del inp, out, test_model"]}, {"cell_type": "markdown", "id": "7b3910fe", "metadata": {"papermill": {"duration": 0.038084, "end_time": "2021-09-16T12:42:24.563987", "exception": false, "start_time": "2021-09-16T12:42:24.525903", "status": "completed"}, "tags": []}, "source": ["The visualization shows that for predicting any pixel, we can take almost half of the image into account.\n", "However, keep in mind that this is the \"theoretical\" receptive field and not necessarily\n", "the [effective receptive field](https://arxiv.org/pdf/1701.04128.pdf), which is usually much smaller.\n", "For a stronger model, we should therefore try to increase the receptive\n", "field even further. Especially, for the pixel on the bottom right, the\n", "very last pixel, we would be allowed to take into account the whole\n", "image. However, our current receptive field only spans across 1/4 of the\n", "image. An encoder-decoder architecture can help with this, but it also\n", "shows that we require a much deeper, more complex network in\n", "autoregressive models than in VAEs or energy-based models."]}, {"cell_type": "markdown", "id": "af3db43c", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.039876, "end_time": "2021-09-16T12:42:24.641944", "exception": false, "start_time": "2021-09-16T12:42:24.602068", "status": "completed"}, "tags": []}, "source": ["### Training loop\n", "\n", "To train the model, we again can rely on PyTorch Lightning and write a\n", "function below for loading the pretrained model if it exists. To reduce\n", "the computational cost, we have saved the validation and test score in\n", "the checkpoint already:"]}, {"cell_type": "code", "execution_count": 18, "id": "386d2b64", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:42:24.724459Z", "iopub.status.busy": "2021-09-16T12:42:24.723984Z", "iopub.status.idle": "2021-09-16T12:42:24.726084Z", "shell.execute_reply": "2021-09-16T12:42:24.725601Z"}, "papermill": {"duration": 0.046099, "end_time": "2021-09-16T12:42:24.726183", "exception": false, "start_time": "2021-09-16T12:42:24.680084", "status": "completed"}, "tags": []}, "outputs": [], "source": ["def train_model(**kwargs):\n", " # Create a PyTorch Lightning trainer with the generation callback\n", " trainer = pl.Trainer(\n", " default_root_dir=os.path.join(CHECKPOINT_PATH, \"PixelCNN\"),\n", " gpus=1 if str(device).startswith(\"cuda\") else 0,\n", " max_epochs=150,\n", " callbacks=[\n", " ModelCheckpoint(save_weights_only=True, mode=\"min\", monitor=\"val_bpd\"),\n", " LearningRateMonitor(\"epoch\"),\n", " ],\n", " )\n", " result = None\n", " # Check whether pretrained model exists. If yes, load it and skip training\n", " pretrained_filename = os.path.join(CHECKPOINT_PATH, \"PixelCNN.ckpt\")\n", " if os.path.isfile(pretrained_filename):\n", " print(\"Found pretrained model, loading...\")\n", " model = PixelCNN.load_from_checkpoint(pretrained_filename)\n", " ckpt = torch.load(pretrained_filename, map_location=device)\n", " result = ckpt.get(\"result\", None)\n", " else:\n", " model = PixelCNN(**kwargs)\n", " trainer.fit(model, train_loader, val_loader)\n", " model = model.to(device)\n", "\n", " if result is None:\n", " # Test best model on validation and test set\n", " val_result = trainer.test(model, test_dataloaders=val_loader, verbose=False)\n", " test_result = trainer.test(model, test_dataloaders=test_loader, verbose=False)\n", " result = {\"test\": test_result, \"val\": val_result}\n", " return model, result"]}, {"cell_type": "markdown", "id": "245b1ba8", "metadata": {"papermill": {"duration": 0.037966, "end_time": "2021-09-16T12:42:24.802271", "exception": false, "start_time": "2021-09-16T12:42:24.764305", "status": "completed"}, "tags": []}, "source": ["Training the model is time consuming and we recommend using the provided pre-trained model for going through this notebook.\n", "However, feel free to play around with the hyperparameter like number of layers etc.\n", "if you want to get a feeling for those.\n", "\n", "When calling the training function with a pre-trained model, we automatically load it and print its test performance:"]}, {"cell_type": "code", "execution_count": 19, "id": "4328bb62", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:42:24.884095Z", "iopub.status.busy": "2021-09-16T12:42:24.883627Z", "iopub.status.idle": "2021-09-16T12:42:27.511570Z", "shell.execute_reply": "2021-09-16T12:42:27.511143Z"}, "papermill": {"duration": 2.671446, "end_time": "2021-09-16T12:42:27.511686", "exception": false, "start_time": "2021-09-16T12:42:24.840240", "status": "completed"}, "tags": []}, "outputs": [{"name": "stderr", "output_type": "stream", "text": ["GPU available: True, used: True\n"]}, {"name": "stderr", "output_type": "stream", "text": ["TPU available: False, using: 0 TPU cores\n"]}, {"name": "stderr", "output_type": "stream", "text": ["IPU available: False, using: 0 IPUs\n"]}, {"name": "stdout", "output_type": "stream", "text": ["Found pretrained model, loading...\n"]}, {"name": "stdout", "output_type": "stream", "text": ["Test bits per dimension: 0.808bpd\n"]}], "source": ["model, result = train_model(c_in=1, c_hidden=64)\n", "test_res = result[\"test\"][0]\n", "print(\n", " \"Test bits per dimension: %4.3fbpd\" % (test_res[\"test_loss\"] if \"test_loss\" in test_res else test_res[\"test_bpd\"])\n", ")"]}, {"cell_type": "markdown", "id": "c4e1ed4e", "metadata": {"papermill": {"duration": 0.038901, "end_time": "2021-09-16T12:42:27.590698", "exception": false, "start_time": "2021-09-16T12:42:27.551797", "status": "completed"}, "tags": []}, "source": ["With a test performance of 0.809bpd, the PixelCNN significantly outperforms the normalizing flows we have seen in Tutorial 11.\n", "Considering image modeling as an autoregressive problem simplifies the learning process as predicting\n", "one pixel given the ground truth of all others is much easier than predicting all pixels at once.\n", "In addition, PixelCNN can explicitly predict the pixel values by a discrete softmax while\n", "Normalizing Flows have to learn transformations in continuous latent space.\n", "These two aspects allow the PixelCNN to achieve a notably better performance.\n", "\n", "To fully compare the models, let's also measure the number of parameters of the PixelCNN:"]}, {"cell_type": "code", "execution_count": 20, "id": "f8c9acf4", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:42:27.674832Z", "iopub.status.busy": "2021-09-16T12:42:27.674359Z", "iopub.status.idle": "2021-09-16T12:42:27.676436Z", "shell.execute_reply": "2021-09-16T12:42:27.676819Z"}, "papermill": {"duration": 0.046548, "end_time": "2021-09-16T12:42:27.676936", "exception": false, "start_time": "2021-09-16T12:42:27.630388", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Number of parameters: 852,160\n"]}], "source": ["num_params = sum(np.prod(param.shape) for param in model.parameters())\n", "print(f\"Number of parameters: {num_params:,}\")"]}, {"cell_type": "markdown", "id": "d3ee86a3", "metadata": {"papermill": {"duration": 0.039335, "end_time": "2021-09-16T12:42:27.755406", "exception": false, "start_time": "2021-09-16T12:42:27.716071", "status": "completed"}, "tags": []}, "source": ["Compared to the multi-scale normalizing flows, the PixelCNN has considerably less parameters.\n", "Of course, the number of parameters depend on our hyperparameter choices.\n", "Nevertheless, in general, it can be said that autoregressive models\n", "require considerably less parameters than normalizing flows to reach\n", "good performance, based on the reasons stated above. Still,\n", "autoregressive models are much slower in sampling than normalizing\n", "flows, which limits their possible applications."]}, {"cell_type": "markdown", "id": "77208025", "metadata": {"papermill": {"duration": 0.046301, "end_time": "2021-09-16T12:42:27.842873", "exception": false, "start_time": "2021-09-16T12:42:27.796572", "status": "completed"}, "tags": []}, "source": ["## Sampling\n", "\n", "One way of qualitatively analysing generative models is by looking at the actual samples.\n", "Let's therefore use our sampling function to generate a few digits:"]}, {"cell_type": "code", "execution_count": 21, "id": "27bb57d8", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:42:27.932413Z", "iopub.status.busy": "2021-09-16T12:42:27.931873Z", "iopub.status.idle": "2021-09-16T12:42:32.123905Z", "shell.execute_reply": "2021-09-16T12:42:32.123484Z"}, "papermill": {"duration": 4.238655, "end_time": "2021-09-16T12:42:32.124020", "exception": false, "start_time": "2021-09-16T12:42:27.885365", "status": "completed"}, "tags": []}, "outputs": [{"name": "stderr", "output_type": "stream", "text": ["Global seed set to 1\n"]}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "8961cfdd02c24f6dbc1c42cf6db817fd", "version_major": 2, "version_minor": 0}, "text/plain": [" 0%| | 0/28 [00:00\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2021-09-16T14:42:32.079950\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.4.3, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}], "source": ["pl.seed_everything(1)\n", "samples = model.sample(img_shape=(16, 1, 28, 28))\n", "show_imgs(samples.cpu())"]}, {"cell_type": "markdown", "id": "8fb41b9f", "metadata": {"papermill": {"duration": 0.041107, "end_time": "2021-09-16T12:42:32.207277", "exception": false, "start_time": "2021-09-16T12:42:32.166170", "status": "completed"}, "tags": []}, "source": ["Most of the samples can be identified as digits, and overall we achieve a better quality than we had in normalizing flows.\n", "This goes along with the lower likelihood we achieved with autoregressive models.\n", "Nevertheless, we also see that there is still place for improvement\n", "as a considerable amount of samples cannot be identified (for example the first row).\n", "Deeper autoregressive models are expected to achieve better quality,\n", "as they can take more context into account for generating the pixels.\n", "\n", "Note that on Google Colab, you might see different results, specifically with a white line at the top.\n", "After some debugging, it seemed that the difference occurs inside the dilated convolution,\n", "as it gives different results for different batch sizes.\n", "However, it is hard to debug this further as it might be a bug of the installed PyTorch version on Google Colab.\n", "\n", "The trained model itself is not restricted to any specific image size.\n", "However, what happens if we actually sample a larger image than we had\n", "seen in our training dataset? Let's try below to sample images of size\n", "$64\\times64$ instead of $28\\times28$:"]}, {"cell_type": "code", "execution_count": 22, "id": "f55e0ed4", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:42:32.292415Z", "iopub.status.busy": "2021-09-16T12:42:32.291942Z", "iopub.status.idle": "2021-09-16T12:43:10.492028Z", "shell.execute_reply": "2021-09-16T12:43:10.492425Z"}, "papermill": {"duration": 38.244567, "end_time": "2021-09-16T12:43:10.492570", "exception": false, "start_time": "2021-09-16T12:42:32.248003", "status": "completed"}, "tags": []}, "outputs": [{"name": "stderr", "output_type": "stream", "text": ["Global seed set to 1\n"]}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "a04db2d75a3f476487be88f18d66ddb1", "version_major": 2, "version_minor": 0}, "text/plain": [" 0%| | 0/64 [00:00\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2021-09-16T14:43:10.455579\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.4.3, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}], "source": ["pl.seed_everything(1)\n", "samples = model.sample(img_shape=(8, 1, 64, 64))\n", "show_imgs(samples.cpu())"]}, {"cell_type": "markdown", "id": "3876e6b8", "metadata": {"papermill": {"duration": 0.043073, "end_time": "2021-09-16T12:43:10.581847", "exception": false, "start_time": "2021-09-16T12:43:10.538774", "status": "completed"}, "tags": []}, "source": ["The larger images show that changing the size of the image during testing confuses the model\n", "and generates abstract figures (you can sometimes spot a digit in the upper left corner).\n", "In addition, sampling for images of 64x64 pixels take more than a minute on a GPU.\n", "Clearly, autoregressive models cannot be scaled to large images without changing the sampling procedure such as with [forecasting](https://arxiv.org/abs/2002.09928).\n", "Our implementation is also not the most efficient as many computations can be stored and reused throughout the sampling process.\n", "Nevertheless, the sampling procedure stays sequential which is\n", "inherently slower than parallel generation like done in normalizing\n", "flows."]}, {"cell_type": "markdown", "id": "5ef1ba66", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.042826, "end_time": "2021-09-16T12:43:10.667581", "exception": false, "start_time": "2021-09-16T12:43:10.624755", "status": "completed"}, "tags": []}, "source": ["### Autocompletion\n", "\n", "One common application done with autoregressive models is\n", "auto-completing an image. As autoregressive models predict pixels one by\n", "one, we can set the first $N$ pixels to predefined values and check how\n", "the model completes the image. For implementing this, we just need to\n", "skip the iterations in the sampling loop that already have a value\n", "unequals -1. See above in our PyTorch Lightning module for the specific\n", "implementation. In the cell below, we randomly take three images from\n", "the training set, mask about the lower half of the image, and let the\n", "model autocomplete it. To see the diversity of samples, we do this 12\n", "times for each image:"]}, {"cell_type": "code", "execution_count": 23, "id": "44b3d642", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:43:10.758938Z", "iopub.status.busy": "2021-09-16T12:43:10.758463Z", "iopub.status.idle": "2021-09-16T12:43:18.484127Z", "shell.execute_reply": "2021-09-16T12:43:18.483633Z"}, "papermill": {"duration": 7.773878, "end_time": "2021-09-16T12:43:18.484244", "exception": false, "start_time": "2021-09-16T12:43:10.710366", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Original image and input image to sampling:\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUgovR3JvdXAgPDwgL0NTIC9EZXZpY2VSR0IgL1MgL1RyYW5zcGFyZW5jeSAvVHlwZSAvR3JvdXAgPj4KL01lZGlhQm94IFsgMCAwIDE3Mi4zODM3NSA5NS45NCBdIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovVHlwZSAvUGFnZSA+PgplbmRvYmoKOSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEyIDAgUiA+PgpzdHJlYW0KeJxNjUEKwjAURPf/FHOCJD/tJ8myUggu68YDhKAWW6kFvb6/XRQXAzPDDI8xku0YtxUOo+oLRobt6+dR6iWfUFZy2k/EwZsmNkE0Pf9SEpNabdzh7kQzLQjG72IJJu3TyEZavCuumGE7v2FZsaxYh6wflogNF/1xLxPsmdG/MNBAP35RJfYKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoxMzQKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMyAwIG9iago8PCA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL0NBIDAgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PgovQTIgPDwgL0NBIDEgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSID4+CmVuZG9iagoxMyAwIG9iago8PCAvQml0c1BlckNvbXBvbmVudCA4Ci9Db2xvclNwYWNlIFsvSW5kZXhlZCAvRGV2aWNlUkdCIDcwICj////9/f37+/v6+vr19fXy8vLw8PDv7+/t7e3s7Ozr6+vo6Ojk5OTj4+Pf39/Y2NjR0dHQ0NDPz8/Hx8fDw8PCwsLBwcG/v7+6urq5ubm0tLSurq6srKylpaWgoKCVlZWTk5OPj4+Li4uKioqIiIiAgIB9fX1ycnJjY2NgYGBfX19WVlZRUVFQUFBPT09MTExHR0dEREQ7Ozs1NTUyMjIwMDAsLCwqKiojIyMiIiIdHR0bGxsaGhoZGRkUFBQTExMSEhIODg4MDAxcblxuXG4GBgYEBAQAAAApXQovRGVjb2RlUGFybXMgPDwgL0NvbG9ycyAxIC9Db2x1bW5zIDE1OCAvUHJlZGljdG9yIDEwID4+Ci9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9IZWlnaHQgODIgL0xlbmd0aCAxNCAwIFIgL1N1YnR5cGUgL0ltYWdlCi9UeXBlIC9YT2JqZWN0IC9XaWR0aCAxNTggPj4Kc3RyZWFtCnic7dfXcoJAGIZh1gY2Yu8K9oJdsDdEc/+XFCBMYkECs2qE+b9DGN55DnYUiPpbj/hvgPGAhzNb8D4fs/rDY8DDiQEPJwY8nBjwcGLAw4kBDycGPJzYfV673SYIgmVv7xyPx9PJGk9r3cbU1v0Y8J7BE8VoNIoQ4riLyzxfq9UoiorFer2eWZ6oxNTWRYxXYmorptMC3vN4mw1S1+mo/d1uN5mUy+VgEGmrVqtmeRst1lFiohKbKLHgecxZPI77fjAQYBiGptHVEglJkszyOC0WUGL0TSyhxID3Qp4k5fN5r9f7W/F4MpkMy94/LK/8z31znrrxeNxsfihrNBqjkXylVJJtPp9vvQaezXlXGw6HFCXz+v2+QRF4+ovH4wj5/f7lcgk8R/IqlYpxEXg25HW7JEkmk4fDAXgWeYIgkKT8i1wo/FkEnt14rVZLfQ0dDIBnlbffy1/yCKVSKb1vFuDZmvdz8AxPHvD0l8vJuFBoPp8Dz1m87XYbici8bNZcEXjn43keoXA4LAjAcxyvWCwilE6nzRaBZxvebOZ2uxFyuVwMs1qtgGeJN52qLys0TS8W5orAsw3PehF4OEXg4RSfwHvbAQ9nwMPZF28utk0KZW5kc3RyZWFtCmVuZG9iagoxNCAwIG9iago0NjMKZW5kb2JqCjIgMCBvYmoKPDwgL0NvdW50IDEgL0tpZHMgWyAxMSAwIFIgXSAvVHlwZSAvUGFnZXMgPj4KZW5kb2JqCjE1IDAgb2JqCjw8IC9DcmVhdGlvbkRhdGUgKEQ6MjAyMTA5MTYxNDQzMTArMDInMDAnKQovQ3JlYXRvciAoTWF0cGxvdGxpYiB2My40LjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcpCi9Qcm9kdWNlciAoTWF0cGxvdGxpYiBwZGYgYmFja2VuZCB2My40LjMpID4+CmVuZG9iagp4cmVmCjAgMTYKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDE3ODMgMDAwMDAgbiAKMDAwMDAwMDY0NSAwMDAwMCBuIAowMDAwMDAwNjY2IDAwMDAwIG4gCjAwMDAwMDA3NjUgMDAwMDAgbiAKMDAwMDAwMDc4NiAwMDAwMCBuIAowMDAwMDAwODA3IDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM5NiAwMDAwMCBuIAowMDAwMDAwNjI1IDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMDYwNSAwMDAwMCBuIAowMDAwMDAwODM5IDAwMDAwIG4gCjAwMDAwMDE3NjMgMDAwMDAgbiAKMDAwMDAwMTg0MyAwMDAwMCBuIAp0cmFpbGVyCjw8IC9JbmZvIDE1IDAgUiAvUm9vdCAxIDAgUiAvU2l6ZSAxNiA+PgpzdGFydHhyZWYKMjAwMAolJUVPRgo=\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2021-09-16T14:43:10.792399\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.4.3, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}, {"name": "stderr", "output_type": "stream", "text": ["Global seed set to 1\n"]}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "e76d7eb2829b4c6c9b26ff6703e0787d", "version_major": 2, "version_minor": 0}, "text/plain": [" 0%| | 0/28 [00:00\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2021-09-16T14:43:13.309899\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.4.3, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}, {"name": "stdout", "output_type": "stream", "text": ["Original image and input image to sampling:\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUgovR3JvdXAgPDwgL0NTIC9EZXZpY2VSR0IgL1MgL1RyYW5zcGFyZW5jeSAvVHlwZSAvR3JvdXAgPj4KL01lZGlhQm94IFsgMCAwIDE3Mi4zODM3NSA5NS45NCBdIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovVHlwZSAvUGFnZSA+PgplbmRvYmoKOSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEyIDAgUiA+PgpzdHJlYW0KeJxNjUEKwjAURPf/FHOCJD/tJ8myUggu68YDhKAWW6kFvb6/XRQXAzPDDI8xku0YtxUOo+oLRobt6+dR6iWfUFZy2k/EwZsmNkE0Pf9SEpNabdzh7kQzLQjG72IJJu3TyEZavCuumGE7v2FZsaxYh6wflogNF/1xLxPsmdG/MNBAP35RJfYKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoxMzQKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMyAwIG9iago8PCA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL0NBIDAgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PgovQTIgPDwgL0NBIDEgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSID4+CmVuZG9iagoxMyAwIG9iago8PCAvQml0c1BlckNvbXBvbmVudCA4Ci9Db2xvclNwYWNlIFsvSW5kZXhlZCAvRGV2aWNlUkdCIDY4ICj////+/v78/Pz7+/v5+fn39/fy8vLx8fHq6urp6eno6Ojm5ubl5eXd3d3Z2dnX19fW1tbV1dXMzMzGxsbBwcG/v7+9vb28vLy7u7u6urq4uLi1tbW0tLSysrKtra2srKyhoaGYmJiVlZWRkZGAgIB1dXVzc3NxcXFqampnZ2diYmJaWlpXV1dTU1NRUVFPT09MTExLS0tCQkJAQEA7Ozs6OjowMDAgICAbGxsYGBgXFxcVFRUTExMODg5cclxyXHJcblxuXG4ICAgGBgYDAwMCAgIAAAApXQovRGVjb2RlUGFybXMgPDwgL0NvbG9ycyAxIC9Db2x1bW5zIDE1OCAvUHJlZGljdG9yIDEwID4+Ci9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9IZWlnaHQgODIgL0xlbmd0aCAxNCAwIFIgL1N1YnR5cGUgL0ltYWdlCi9UeXBlIC9YT2JqZWN0IC9XaWR0aCAxNTggPj4Kc3RyZWFtCnic7dhXj4JAFIZhxo5rb9h717X33v7/b9oZgu7qEpnNIeq457tDkjfPBRBEKr/0pGcD7g95kAnBO5mzsukx5EFiyIPEOHnz+TwSGQwGHEWeWITFkPcSvO02kUgQksvl4LwtixEW+xe85XKpKISt0WhAeax1jiHv+bxMJqP2XK79fg/lXVouo9ab8Px+v5psNg16PLxLizOGvAfwHA5Ht2sWj7U4Y8h7AI8+TA173Dyu1lvwajWr1UpIqVSC82osRlgMec/n0ZeVUEh9jtKXeTjvbxOe1263CfnmdTqdT7Z+v4884XnhcFjjpVKpbNZisagHdrs9m9W7l5Gnz/s9j8czmSDvHXhut/vj55xOSZK83punNfIE5EWjm83m6sxwKMsyIdPpFHlGvGr19sxqFQgEZHk2myFPcF48fnXt0QNFodfeaKRXRJ62fD6vPfd6vd5iQX85HsfjcTKZJCQYDOoXkScGb71ea19b6WKxYrFYKJwPK5UK8oy+Eux2Pp/v5lWPvszX64fDAXmC804netOm02eZzdZqtej/3TtF5EGKpsaQB4khDxJDHiSGPEgMeZAY8iAx5EFiQvBedsiDDHmQfQGIbT5JCmVuZHN0cmVhbQplbmRvYmoKMTQgMCBvYmoKNDU1CmVuZG9iagoyIDAgb2JqCjw8IC9Db3VudCAxIC9LaWRzIFsgMTEgMCBSIF0gL1R5cGUgL1BhZ2VzID4+CmVuZG9iagoxNSAwIG9iago8PCAvQ3JlYXRpb25EYXRlIChEOjIwMjEwOTE2MTQ0MzEzKzAyJzAwJykKL0NyZWF0b3IgKE1hdHBsb3RsaWIgdjMuNC4zLCBodHRwczovL21hdHBsb3RsaWIub3JnKQovUHJvZHVjZXIgKE1hdHBsb3RsaWIgcGRmIGJhY2tlbmQgdjMuNC4zKSA+PgplbmRvYmoKeHJlZgowIDE2CjAwMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAwMDAxNiAwMDAwMCBuIAowMDAwMDAxNzcyIDAwMDAwIG4gCjAwMDAwMDA2NDUgMDAwMDAgbiAKMDAwMDAwMDY2NiAwMDAwMCBuIAowMDAwMDAwNzY1IDAwMDAwIG4gCjAwMDAwMDA3ODYgMDAwMDAgbiAKMDAwMDAwMDgwNyAwMDAwMCBuIAowMDAwMDAwMDY1IDAwMDAwIG4gCjAwMDAwMDAzOTYgMDAwMDAgbiAKMDAwMDAwMDYyNSAwMDAwMCBuIAowMDAwMDAwMjA4IDAwMDAwIG4gCjAwMDAwMDA2MDUgMDAwMDAgbiAKMDAwMDAwMDgzOSAwMDAwMCBuIAowMDAwMDAxNzUyIDAwMDAwIG4gCjAwMDAwMDE4MzIgMDAwMDAgbiAKdHJhaWxlcgo8PCAvSW5mbyAxNSAwIFIgL1Jvb3QgMSAwIFIgL1NpemUgMTYgPj4Kc3RhcnR4cmVmCjE5ODkKJSVFT0YK\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2021-09-16T14:43:13.375277\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.4.3, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}, {"name": "stderr", "output_type": "stream", "text": ["Global seed set to 1\n"]}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "5ec4fe6328c247b489d2701b79eb8048", "version_major": 2, "version_minor": 0}, "text/plain": [" 0%| | 0/28 [00:00\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2021-09-16T14:43:15.879705\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.4.3, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}, {"name": "stdout", "output_type": "stream", "text": ["Original image and input image to sampling:\n"]}, {"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUgovR3JvdXAgPDwgL0NTIC9EZXZpY2VSR0IgL1MgL1RyYW5zcGFyZW5jeSAvVHlwZSAvR3JvdXAgPj4KL01lZGlhQm94IFsgMCAwIDE3Mi4zODM3NSA5NS45NCBdIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovVHlwZSAvUGFnZSA+PgplbmRvYmoKOSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEyIDAgUiA+PgpzdHJlYW0KeJxNjUEKwjAURPf/FHOCJD/tJ8myUggu68YDhKAWW6kFvb6/XRQXAzPDDI8xku0YtxUOo+oLRobt6+dR6iWfUFZy2k/EwZsmNkE0Pf9SEpNabdzh7kQzLQjG72IJJu3TyEZavCuumGE7v2FZsaxYh6wflogNF/1xLxPsmdG/MNBAP35RJfYKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoxMzQKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMyAwIG9iago8PCA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL0NBIDAgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PgovQTIgPDwgL0NBIDEgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvSTEgMTMgMCBSID4+CmVuZG9iagoxMyAwIG9iago8PCAvQml0c1BlckNvbXBvbmVudCA4Ci9Db2xvclNwYWNlIFsvSW5kZXhlZCAvRGV2aWNlUkdCIDc4ICj////+/v79/f38/Pz5+fn39/f09PTz8/Pv7+/t7e3s7Ozr6+vp6eno6Ojm5ubh4eHe3t7b29va2trT09POzs7Jycm/v7++vr67u7u6urqurq6pqamoqKinp6ekpKScnJyYmJiWlpaTk5ORkZGPj4+NjY2JiYmFhYWAgIB3d3d2dnZxcXFqampkZGRgYGBeXl5dXV1ZWVlXV1dVVVVUVFRQUFBDQ0NAQEA/Pz8+Pj44ODgxMTErKysmJiYjIyMgICAfHx8dHR0bGxsWFhYSEhIQEBAPDw8ODg4LCwsHBwcGBgYFBQUEBAQCAgIAAAApXQovRGVjb2RlUGFybXMgPDwgL0NvbG9ycyAxIC9Db2x1bW5zIDE1OCAvUHJlZGljdG9yIDEwID4+Ci9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9IZWlnaHQgODIgL0xlbmd0aCAxNCAwIFIgL1N1YnR5cGUgL0ltYWdlCi9UeXBlIC9YT2JqZWN0IC9XaWR0aCAxNTggPj4Kc3RyZWFtCnic7ddpV4JAFIBhmQkMNXMpU3M3W9yX0tJSwxRL4v//nO6MtllqNHmSuveLAsf3PB8GBh2nGz2O3wYsH+SJjC145s/M6Y/HkCcSQ55IDHkiMeSJxJAnEkOeSAx5IrHVvGq1GggQQiQpGo2Wy7quC/BYK8BaEmuVl7SQt2beaDRKJGRZpmwI4R/BYLDZbFrn8VbipfUcW9hC3pp5sDoo9Xg8pVI+n58VYRRFubqyyuMtylol1iJvW4tjtuVpmgZ3GaXtdts0u93uOZtIZFr1+YbDoQXerEVZy3xpRWYt3yct5K2TVywWYYkcHxuG8XrSMAqFgtvtJiSdTlvgfWPszLu5cblcXq9X0+YujMdjv98vSbFYDHk25R0dTTewDxdarRbflWq1GvIW8Q4PAQE379zpx8d4PA5X9vYmkwny7MiDd7PdXUB8/M3Z2XRTu7hYWETeZvMGgwFHvHt43N3lcrmtrSlvSfHf80x258LryvPGdXtbr9cdDjKbTCaDPDvzTk5ghTmdzgM2OztwsL0N2v19+Ia8VbyHh1QqBa98/D6V5XA4zP+NcmilUkGenXl8dF3n/0gvL/khPK1VFcj9fh95q3lz02g0KM1ms8uLyLMlLxQKEQJLEXnWeb2eqqqUIu8v8q6v+QaHPOQh7/3c3yeTSUXpdDrI+3O8rxaRJ1JEnkhxDbyNHeSJDPJE5gkEVtsWCmVuZHN0cmVhbQplbmRvYmoKMTQgMCBvYmoKNTMwCmVuZG9iagoyIDAgb2JqCjw8IC9Db3VudCAxIC9LaWRzIFsgMTEgMCBSIF0gL1R5cGUgL1BhZ2VzID4+CmVuZG9iagoxNSAwIG9iago8PCAvQ3JlYXRpb25EYXRlIChEOjIwMjEwOTE2MTQ0MzE1KzAyJzAwJykKL0NyZWF0b3IgKE1hdHBsb3RsaWIgdjMuNC4zLCBodHRwczovL21hdHBsb3RsaWIub3JnKQovUHJvZHVjZXIgKE1hdHBsb3RsaWIgcGRmIGJhY2tlbmQgdjMuNC4zKSA+PgplbmRvYmoKeHJlZgowIDE2CjAwMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAwMDAxNiAwMDAwMCBuIAowMDAwMDAxODcxIDAwMDAwIG4gCjAwMDAwMDA2NDUgMDAwMDAgbiAKMDAwMDAwMDY2NiAwMDAwMCBuIAowMDAwMDAwNzY1IDAwMDAwIG4gCjAwMDAwMDA3ODYgMDAwMDAgbiAKMDAwMDAwMDgwNyAwMDAwMCBuIAowMDAwMDAwMDY1IDAwMDAwIG4gCjAwMDAwMDAzOTYgMDAwMDAgbiAKMDAwMDAwMDYyNSAwMDAwMCBuIAowMDAwMDAwMjA4IDAwMDAwIG4gCjAwMDAwMDA2MDUgMDAwMDAgbiAKMDAwMDAwMDgzOSAwMDAwMCBuIAowMDAwMDAxODUxIDAwMDAwIG4gCjAwMDAwMDE5MzEgMDAwMDAgbiAKdHJhaWxlcgo8PCAvSW5mbyAxNSAwIFIgL1Jvb3QgMSAwIFIgL1NpemUgMTYgPj4Kc3RhcnR4cmVmCjIwODgKJSVFT0YK\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2021-09-16T14:43:15.945419\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.4.3, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}, {"name": "stderr", "output_type": "stream", "text": ["Global seed set to 1\n"]}, {"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "ff6c8e20790c444e8f7c56fe0cf6f5bd", "version_major": 2, "version_minor": 0}, "text/plain": [" 0%| | 0/28 [00:00\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2021-09-16T14:43:18.448685\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.4.3, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}], "source": ["def autocomplete_image(img):\n", " # Remove lower half of the image\n", " img_init = img.clone()\n", " img_init[:, 10:, :] = -1\n", " print(\"Original image and input image to sampling:\")\n", " show_imgs([img, img_init])\n", " # Generate 12 example completions\n", " img_init = img_init.unsqueeze(dim=0).expand(12, -1, -1, -1).to(device)\n", " pl.seed_everything(1)\n", " img_generated = model.sample(img_init.shape, img_init)\n", " print(\"Autocompletion samples:\")\n", " show_imgs(img_generated)\n", "\n", "\n", "for i in range(1, 4):\n", " img = train_set[i][0]\n", " autocomplete_image(img)"]}, {"cell_type": "markdown", "id": "ba91d363", "metadata": {"papermill": {"duration": 0.051443, "end_time": "2021-09-16T12:43:18.586848", "exception": false, "start_time": "2021-09-16T12:43:18.535405", "status": "completed"}, "tags": []}, "source": ["For the first two digits (7 and 6), we see that the 12 samples all\n", "result in a shape which resemble the original digit. Nevertheless, there\n", "are some style difference in writing the 7, and some deformed sixes in\n", "the samples. When autocompleting the 9 below, we see that the model can\n", "fit multiple digits to it. We obtain diverse samples from 0, 3, 8 and 9.\n", "This shows that despite having no latent space, we can still obtain\n", "diverse samples from an autoregressive model."]}, {"cell_type": "markdown", "id": "ee095fbc", "metadata": {"papermill": {"duration": 0.04956, "end_time": "2021-09-16T12:43:18.685925", "exception": false, "start_time": "2021-09-16T12:43:18.636365", "status": "completed"}, "tags": []}, "source": ["### Visualization of the predictive distribution (softmax)\n", "\n", "Autoregressive models use a softmax over 256 values to predict the next pixel.\n", "This gives the model a large flexibility as the probabilities for each pixel value can be learned independently if necessary.\n", "However, the values are actually not independent because the values 32 and 33 are much closer than 32 and 255.\n", "In the following, we visualize the softmax distribution that the model predicts to gain insights how it has learned the relationships of close-by pixels.\n", "\n", "To do this, we first run the model on a batch of images and store the output softmax distributions:"]}, {"cell_type": "code", "execution_count": 24, "id": "fd594831", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:43:18.789379Z", "iopub.status.busy": "2021-09-16T12:43:18.788903Z", "iopub.status.idle": "2021-09-16T12:43:18.897568Z", "shell.execute_reply": "2021-09-16T12:43:18.897984Z"}, "papermill": {"duration": 0.162907, "end_time": "2021-09-16T12:43:18.898132", "exception": false, "start_time": "2021-09-16T12:43:18.735225", "status": "completed"}, "tags": []}, "outputs": [], "source": ["det_loader = data.DataLoader(train_set, batch_size=128, shuffle=False, drop_last=False)\n", "imgs, _ = next(iter(det_loader))\n", "imgs = imgs.to(device)\n", "with torch.no_grad():\n", " out = model(imgs)\n", " out = F.softmax(out, dim=1)\n", " mean_out = out.mean(dim=[0, 2, 3, 4]).cpu().numpy()\n", " out = out.cpu().numpy()"]}, {"cell_type": "markdown", "id": "460b20fc", "metadata": {"papermill": {"duration": 0.049128, "end_time": "2021-09-16T12:43:18.998216", "exception": false, "start_time": "2021-09-16T12:43:18.949088", "status": "completed"}, "tags": []}, "source": ["Before diving into the model, let's visualize the distribution of the pixel values in the whole dataset:"]}, {"cell_type": "code", "execution_count": 25, "id": "cf4f889d", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:43:19.102964Z", "iopub.status.busy": "2021-09-16T12:43:19.102489Z", "iopub.status.idle": "2021-09-16T12:43:20.698247Z", "shell.execute_reply": "2021-09-16T12:43:20.698640Z"}, "papermill": {"duration": 1.650907, "end_time": "2021-09-16T12:43:20.698784", "exception": false, "start_time": "2021-09-16T12:43:19.047877", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUgovR3JvdXAgPDwgL0NTIC9EZXZpY2VSR0IgL1MgL1RyYW5zcGFyZW5jeSAvVHlwZSAvR3JvdXAgPj4KL01lZGlhQm94IFsgMCAwIDM4NC43IDI1MS45ODA2MjUgXSAvUGFyZW50IDIgMCBSIC9SZXNvdXJjZXMgOCAwIFIKL1R5cGUgL1BhZ2UgPj4KZW5kb2JqCjkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMiAwIFIgPj4Kc3RyZWFtCnic3V1LjzTHcbzvr5ijdGCxMrOeRxG0CehGibAPhg8CTdkihjIoQtLfd2b1TlfWdgigRVFAl4RP2q2YjZ2Ine56Zgc9vn379Ff0+O8fHvo/j/j4Vv/9Vb/+wr5/i/rdd2/SUqj61fP9K84UeouFszbF9dv/eXv7/VsMnWpJNebWHh+/ST1SL7G2x5/sF35xecH5zduHV78ltt9eg6Tx2/Sd1Rqya3m+WpK+z9dbOn7Kt4w3+f3jA51ICu3BVENKjz998/j3xx8fn/6KD29+rf++1X+HN59+/s1f/vD1N7/54rPH1z+85apvulHj5b25Vv+733779uXj+xdxDJTV9xf3+PaL99a3799IDfokKpRTSEZ2EPaQGinb19+9ffbV49N/pQfR46vfjz/XV//19h+PX8RfPv7z8dWv3/7lq7cvx+/7R2h9U62kPyixprb+IXzzT1dLRm1spQj9CLUl/VxymWIovdROi1zf/NPlctS2wZaS9B+hl7j9bIJbDbGV2Msq2DX/AwTXEvJg41zqjxHc+ecSLEVCqjnF9S/sm3+6YMkc+mCLpecfIVh9+bkEDx69S6aWE/fzJjpb/l6RNfCjSiAuMUuVRIc+UxViaTWbtin2F/TLx1ffvpXQe8uSKRc+5NvNSwGSkPU/pZjeRwp6zXWZJFVVGIuMF6tL39uvidU6qvj6klKgyJLs55Sh9vE2vv7OPP/k82++/d2//fm3v/vjD59894c//vmHx+f/+/jy5zCbhFRKL3l2Wa7pp9htN0q9e6iw2lL+p/jNN/C75VBEWqPp92z6SX43UlUp6+hEUnN+Ux0X9M9hOP14w6X1Yd8/23CWHmpsFOsck82mv3sUIOo4S9b32KpQa/kn3lGqeiUttqw2/22/48vv/4dRcuhiHQZLZ8r6N1J8DGJVwJsNFt6HsWmMetWgv/7t1z+W1//mi7cf+UodIbtR5ydZ9EMaMtWexMajPRDp2Dp/QJ4OYVPXS2/WC7gBrG/WwfNnt3RGVcaqV8vFmRJ1tKmfp6szE8n6MWzUc3/362DyrXf1pdhri95Err6QDgkUKldfTqTYZ6cVpsOtdybfeltfTGMyvRdfOJTaMgNfTqTkcbPh9HJrMPnW2/piGqWVdPVFQitVf/bqy4lUDjkVZn65NZh86219MY1s3118ySHmIgX4ciL6VVOLIr3cGky+9ba+mEZS/OpLCZyyTi2vvpxI1XFBTETp5dZg8q239cU0jnd68UXfsiSO7erLiehXUhuPnmcy+dbb+qJvjHtOwJcWCgsx8OVE9Cudq9fj83Iy+dbb+qIaqaXysddRX3rQ2UdM/erLidjEnHqU/O7WweRbb+uLaozWjVx8qaSTpKjj1osvE1EHut1o+rtbB5Nvvasveo/svUj8eHdVXzhQs8H8xZYX4O+vk2eHu+6QmJmBK2L3zkZXV16AH71Nnh3GdENiovTxzqqu6Nel6vT7asuJ+LvrZNrhnjs0ik4Ar77kUHMpGfhyItovl5SEX2YNItd4W1dMIV1vIOpKCT3lXD/OsZ8OKUl/k/RUXl4NJt96W19MY+wR+NICSUod+HIiNurXsVurL7cGk2+9rS+qUV98nUjXHoQlEbDlRHLVSWLqlN/NGkS+8bauqESyNcyPrjT9pcQiH/16OsSG/KXpbebdq0HkG+/qikmM9TJVVlco6DyYM3DlRIp2ylL72Kg7iXzjbV0h227O/WNvo7ZwaPra+nHV4ekQfw+ZTDvcWUxjy5mALynEqjdQ4MuJ+PHsZNphlDs0jlHGxZds94hI17XuiWTRS4f5uIxOJt96W19Mo0gGvpSQcukCfDmRnALpl6W+3BpMvvW2vphGncxcu+imPW3K2qlcfTmRoveURj22l1uDybfe1hfTSNSBLy3o97UCX05Eex71h8ce/2Tyrbf1xTRGu0d+9KXHEFlKv67pTiTn0GuxXZKnY/Ktd/XFNNp6GvCFAhMXAr6cSNYZc9IPRn9362Dyrbf1RTVqp5uvg93OIWmfI8CXE8kl5GxXz7tbB5Nvva0vHOQ4I3PxRXvbHvX+evXlRGoNlVvq/XDrncm33tYXsXuk/pmvvqRQa5cKfDkRvyI3mXZYpxsaU6XrWnfXu2dp3K+LuhPxu0OTaYc9o6FR9L9XX2rQH2cCvpzI4svJtIUvplH7jut0ujfVUC4d+HMCqWi/U0Z3PGlc4209MYGkA7SrJz1kyeC4ywnkGKoU6vwyavD41tu6YhL1pdeVbooqjxW5LnU7yK/ITa4d1umGSB24E7KGQifW1wNrTshPhBzZDvOjoTIqdl3y1r5W+1tqfF3cdZDeZCMlyuVl2UHmm+/rjYTYK2Xkje12RB3WA29OSAcsqdtM8d2ydzLffF9vVGUrlxMuZo3Kq70UZM0JZdYOqdv+yNNx+db7OmMir+NeiiWU0vJlF+XpIT9XfDHtMH88JMpl/8x8qUHH9zkiX04o98C5VcqnXcblW+/rjInU+d51CZxiDzGVxNdBnoPWj8xJtsenxlRSLcAbUh7JkoA3E7IDzVn7aDotG2S++bbeDJWxgEIBIgr6yWBQKeCg1Owr26F9ejLffF9vVKU1IG84FNLhG/LmhFZvTrI9vFGVUW8S1yVxIgktMoEJlIO0k24qNdeXZQeZb76vNxJqt90y4E0KvVMENQMO8tMlR7bFLMpUtiKgbICoBP26g7oBB2mHxJRaTC/LDjLffF9vTKX1u8AbVVGsBgB4c0JLPzXJ9uinTGViUD5A1ELOtYL6AQf5Ei1HtkPl1qFSCJQQEHVbqKughsBB/miMI9vhxMyhkiMoIyCOoUsuoI7AQX5b35HtsNt/qIwdlBIQ69+eUwa1BA7SiVMtUuvp2OByrfd1hq0gooFiAmIJQpJANYGDUtYJVO+FXoYdZL75vt6oytgaKCjQj4WdDbqsAj4dsjpzUu3hTAqp1wqKCvR2qq8mcHx+Issa6KTaYg3URLZSQGEBcQn6CWBozQmt3pxke3ijKmvOoLiAuIWoHw5QXeCgpNOCGIVPyw4y33xfb0xlSqDEgHis2kVQY+CgZdQ3ybYY9Q2V+maBN2I/WjqoM3BQppCUXNpp2SDzzbf1Zqjky9F5s4ZCkdxBrYGD/Bm0ybXDybRDJJwqCIfGqYF6Awf507+Ta4czwYfIeDmZdzzULupMCNQcOGiZRJ1cW8yhTKRVIYH1c8mBIxdQd+CgxqHrp6Oll2MHmW++rzc56Ng+gtIDkhJEe2hQe+CgbAewyFa2np7MN9/XG1XZSgflB3pNBO2hM6g/cNCyDjrJtlgHNZXVDoADb5pOoPVr5M0JlWhLEYnqy7KDzDff1xtTmSooQyDpoecmoA7BQUm/6tG2Wp6ezDff1xtTKQWUIuhEKFDSKSPwZkLLkYlJtsWZiaGSr6cZzRsOYmfzwBrxhPxRcke2wwnzQyVlUJJASUdvnAnUJDhoGRJPsi3GxENlTKAsgeyZr5QiqEtw0LKXOcm22Ms0lcfzEa/e6N8+cge1CQ5a9usm2Rb7dbZo2RuD8gS92QYd4jZQn+CgZeVmkm2xcmMqW2VQoqB/9sAtVlCj4KD1c3OS7fG5URG1EChTIB2jpNIrqFNw0LIaOsm2WA0dKnPMYBqe9dX5+sDBp0OWkd+k2mLkN0RKBwULlEl7mooGxSey7GNOqi32MYdIbqhoIUuIUhIqWpjQshI6ybZYCR0q7fG9wJsUmMej7a/enJAvanFkO9S6HCpjRVULOq5NOpRDVQsTWtYnJtkW6xOmMtoTQoA3RccnwqhqYUJqAjHLePSvI/PN9/VGJ4S9Xca9Zk0NVSlQ1cKEdGbZpRyroSeVa7yvLyoRFTGTjtp6o4iKFiZUxR6azLG+3BpVzK7xvr40myRfrhc1plhkQIyoamFCy7G+k2uLU30mUkciqGqhUJDcOqpamFBpOjHoPaeXYweZb76vN6bSVqCAN/acbH0h8uaEltPnk2yL0+dDpaB4AypiIxOUb+CgJBZVZo/tfHoy33xfb0wlo4ADKil0zijhwEHLmYlJtsWZiaGSUMgBlRKIEko5cFCKOjlIcqz0TTLffF9vVGXsHVUt2DMQoqCkAwcts6hJtsUsSlWyneAEq+fa1yTtilDVwoSWXZdJtsWui6ls1b69etOtth0lHjhomWFOsi1mmKayloqqFqreM2pEqQcOWmaYk2yLGaapLBkFH1DlEHNHyQcOWla0JtkWK1pDZULhB6QzIrbYQOTNCS27dZNsi926oVJQAALVFJJUlIDgID/9nlw7zL+HRkYRCFRzUAkoA8FByz7mJNtiH3OoJBSDQLWERtczxk+HLPOFSbXFfGGIjCgKgarOo6OgxfMTWXrvSbVF711r6L2jOAT9/6DiUB6Cg5b93Um2xf6uqWwNRSJQizZTRJkIDvK1Y5Nrh9ox01gLikWw2NCsP4lqFia07GFOsi32ME1lySgagZpKyR1lIzhIdOrEYxnr6cl88329MZUJxCNQ024mNZSP4CA/rDmpdhjVDInwAFLLdrgeJSQ4aNmIOrm22IcaIhmEJFArQQGUkuAgvaMwx/x+Mb24fOt9nTGRhIISqFX9JqOkBAct86dJtsX8aaiMKCxBO+GgUySUluCg5ZTsJNvilKyqLLZqCdbOWw+tC0pMcNB6SZ1ke1xTqrI1FJpAnUJshFITHJSS9kWt9fKy7CDzzbf1xlRatTbovDsHrhElJzhomUVNsi1mUaayFBSeoG0h5Y7SExy0PJlukm3xZLqhMqMABeoplNRQgoKDbLlcaqvTskHmm+/rjalMKESBeg5Np0WoYmFCy17dJNtir26oZBSkQL2GyAUlKThoWTufZFusnQ+VMFGNup0ezyhNwUHLqs0k22LVZqiMKFCBeg8pJpSo4KBlfDPJthjfqErpHYUqcNTf2wWlKjhomWhOsh1mmkNlayhYgSOF2hglKzioWLhNaeXl2MHlWu/rjGqsFUUrcGSdLV779qdD/IXjqHa4nobIUlDAAkftZ/Rmeh32TcRPCxzVDrOFITJnFLKgf3RV0VDIgoP8hpwj22Gf7lCZUMoCxxJ07NJBvYKD/PFGR7bDqcdDpaCUBY6W9lRQyoKD/HTSke0wyzxUMkpZ4NhCVwZQr+AgP510ZDvMMg+VBFIWdPhmIacoZcFB6yX14triijKRwBUdl3RBGQsO6hyyXlXj7N7B41vu64gK1P71usjHxCE1RgkLDlq6ppNri57JRNrJRPSBsSoVQgkLDvITSEe2w7xyqCwFJSww6VsvESUsOMgXsziyHWpchkp7IiO6pEqIqaOEBQf5jX9HtsN5gENlQgkL9qZZGkpYcNB6TZ1ke1xTplJQwgKT9b0VJSw4yD8qwZHt8ASFQyWjhAWmHopOIUGtgoP8Awsd2Q7PMTxUEkpYYI5B//AoYcFBfhHYke2wNnyojChhQYf7ej8VlLDgoOVzM8m2+NyYSh2sgVoFZgnUGCUsOMgfAHBkO5wLGCprQ49N1RtHkEooYcFB6+fmJNvjc6MqS0UJCypMZ0QRJSw4aOnDJ9kWfbipzBklLDBbhG5HCQsO8oXMjmyH+uZDZUIZC6wq9H6LMhYctN5vTrI97jemUlDGAnMPxNdnljwd4ssSHNUO1QqHSEYZCywxCBVwJH8iy37LpNpiv2WIJJSxwELaz2SUseAgH0TsyHbIJz5URpSxwML66oQyFhy0XFGTbIsrylS2jjIWdMYYWhOUseAg/6hqR7bDE6yHSv0G1Cvo60NUfaBewUG+wNuR7VD3PVSWCjIWWErgElHGgoOS9dhlHKWeVK7xvr6oRFjWzGLvuaOEBQctyzYn1xarNkNkBgkLLC0UaShhwUHLFt3JtcUO3RCZUMICSw86v0YJCw7yhxod2Q5nHQ+VghIWOFGIVFDCgoOWufck22LuPVQSSlhglaJDWpSw4CB/E55cO9yFh8aI8hVYR23SE8pXcNAyU5hkW8wUTGXrKF+Bk36jvRCoVnDQco5kkm1xjsRU1obyFThlHdEyyldwkOrV31UzvSw7yHzzfb1RlbYvC1bP9ZbRC6F8BQctK32TbIuVPlOZC8pX4NR0tB9RvoKDll2XSbbFrouptOfPgCGx3k9FOspXcJAPB3VkO2SGHipheTPr/TRzRfkKDlqvqZNsi2tqqBSUr8CZQqWC8hUctEylJtkWc6mhklG+Amfta2JG+QoO8o9zdGQ7POXxUEkoX4H1fhp7QvkKDlq9Ocn28MZURpSvoHPowO16buDpED9fmEw7zBdMYm0oXYFzCakyGhKfiNiTHqW3062Dyjff1xkVWSpKV2B7DqGO4FC1woSW9eFJtsX6sKnMBaUrsPYyLUeUruCgcpTsjoejOjLffF9vVGXKKF2BSwxKgtIVHFRyoCiN08uyg8w339aboTKhdAW2ojhuKF3BQZXsdGg/juRPMt98X29MpYB0BS4cElWUruCgZbJwcm0xVxgi0caC3i9KLChfwUHLitaLaov1rKGRQL4CF+1kekb5Cg7yFbqTa4u63SEyonwF1vuoDkxQvoKDVmtOsj28sYc5dZSvwEX1VUb5Cg5a9nYn2RZ7u6bSRiLokmpBCqF8BQf5h747sh2eBT9U5oryFbj0kHNE+QoOWlbPJ9kWq+em0iowwOp5NX0d5Ss4aL3fnGRb3G+GytRRzYKO2zo3lK/gIIsp0S9yPi0bZL75vt6YSkH5ClwlENWEahYmVMbRGnti/tOT+eb7emMq2b69epOCxILyFRy0zL8n2Rbz76GSKqpZqDmknlG+goNytwLDSvm0bJD55vt6YyojPIhUSygtoXwFBy3zhUm2xYzBVOroH9Us1Kr3DEH5Cg5aJpmTbItZpqnUjwCqWag9qAaUr+Cg5bzNJNvivI2pzBUlLHBTnhxRwoKDlt26SbbFbp2pTAUlLHDTcb9cn4j+dMjyvJ9JtcXzfobIjBIWuHEo3NAC+oks19Ok2uJ6GiITSljgpmM3qihhwUHLDu8k22KHd6gUlLCg84EQY0EJCw7yIVGObIfsqEMloYwFbiVQzyhjwUEthUiplnJaNsh88329MZURZSzoiDdISyhjwUHrNXWS7XFN6VutHWQs6J02aF+MMhYcVKKdNU9UX44NLt96X2eaPbwIHUNqlsnCKGXBQf4BjyfVDs93NIm5gowF7jH0TChjwUHLBPPk2mJ+aSJt1xGsn3cOlCLKWHDQsp41ybZYzzKVklHGAncJonJRzcKElr3vSbbF3vdQmVDGAnf7pqGMBQdV5Wexat6nJ/PN9/XGVArKWOCeQ40FZSw4yOX9OK4NUoAOjYwSFriX0HpGCQsOWp4WMMm2eFrAUEkoYUE5QmwJJSw4aHnA4yTb4gmPQ2VECQvctReughIWHOQHNpNrh5GNaSwd5StI1B8tjPIVHOSy+hzXBgl+Q2OuKF1BIgWdW6N0BQf5Y5+ObIfToENlghMFieMpnyhdwUFFQmR73NrLsoPMN9/XG1UpGaUrSEyqr6N0BQf5IwCObIeTAYfKhNIVJGa9mTaUruAgXwjlyHaojzpUCkpXkFhCihWlKzjIb+U6sh12eA+VjPIVJNaQ+7Ua8ekQv0juqHZYOz9EEspXkNhCbRkctZmIL2ZxVDvUuBwiI8pXkNhDr4LyFRzkH3nkyHZ4EtJQqVNFULEgRIEKd1Cx4CB/FMuR7XBCa6jMDeUrCHGQTChfwUHL3WaSbXG3MZXa44KKBSHVpz0RqFhwUEn6y6Sn8rLsIPPN9/VGVUoB+QpC2gXrjBpULDio2NkaW/J8OTa4fOt9nTGRYDhMOXRqKGHBQcuw5p1pizHNkCggY0H0PVOsKGPBQXYOP7LUctplXL71vs6YSEYZC9rJKE9BGQsOWu/AJ9ked2BTSShjQaiH1DLKWHCQDwxwZDvkCBwqI8pYEI523gFlLDjIb+E6sh12dofK0lHGgg7XQiuCMhYc5NY6HdcGK6BDo+01XtfNhSXEzChhwUH+McyObIenMw+VqaKEBWEdsyVCCQsO8lvcjmyHne+h0lZYwMo566iNO0pYcJCPtXFkO6TdHCozSlgQLqFQQwkLDvLb/45sh1MBh8qEEhZEVbRYUcKCg6rF0rXU+2nZIPPN9/XGVAqqbhZuOkIpKGHBQes1dZLtcU2ZSkYJCyIxaB+NEhYctOy5TLIt9lyGyogSFkQoSE0oYcFBvlzXke1QxTtUlo4SFkQsq1BQwoKDij2sWT9C5WXZQeab7+uNqlQQVCuISKj5ugr4dIh/QIuj2uG5LUNkqihhwd56TwSO5E/EzxYm0w6zBZMoBeUriJRAElG+goNKD0StHCvnk8w339cbVckZ5SuIqArtf0GtgoOWJa1JtsWa1lCZUL6CSAs5NpSv4KBl53uSbbHzPVQKylcQ6frqgvIVHFS1U9IPSm+nZYPMN9/XG1PJIF9B7LHuLaN8BQeVFpLorGE6Zly+9bbODJFwqqBKYk0oYcFBvpJlcu1Q33KIjCBhQZLooE1QwoKDlgWtk2uL9SwTmTtKWJCUdNTGKGHBQcsm3STbYpfOVKaGEhYk5VASoYQFB42Hkdht52XZQeab7+uNqtR5D6hW0ClSaBJRwoKDqp0K1a7qtOwg88339UZVckYZC5L0nkEdZSw4SIe+VT8prb8sO8h88329aaOUFNQriD1BIzaUseAgvbHYemik07JB5pvv642pFJSxIDkG6RVlLDjIHvCjt9/WTssGmW++rTdDJaOMBckUcisoY8FBy9m1SbbF2bWhklDGgmSVUjPKWHCQdkgUk9VvPD2Zb76vN6YyoowFyRJ6SShjwUHLWt8k22Ktz1TmjjIWJOdAmVHGgoNqCswjVuHpyXzzfb1RlamhjAXthVQFoYwFB9UcIluA6Muyg8w339cbVSkVZSxIriFLRBkLDtK7SpBR1Pz0ZL75vt6oSu1ZUM2CJfJpV4RqFiZEOoeK2mXR6dnBtrTf1x3TmVHKguQetBsGKQsTIbuOIudje2FyLe339cZkCkpaEB2lxF7RwPhEKI1IzNfn5uRa2m/rzZDJKGtBCgfWDweqXJgQsRpCY4vq6dmW9tOdT3/FJp0fv36Qmqeq6PHF49PPv/nLH77+5jdffPb4+oe3Dw5+996gs3ydo5Wx7vH22zepegEvL3tv+fC6C9vHH3Svmj/6t9i+fPs/nGeU3wplbmRzdHJlYW0KZW5kb2JqCjEyIDAgb2JqCjY4NzAKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTcgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzOTUgPj4Kc3RyZWFtCnicPVJLbsVACNvnFFyg0vCbz3lSVd28+29rQ1KpKryJMcYwfcqQueVLXRJxhcm3Xq5bPKZ8LltamXmIu4uNJT623JfuIbZddC6xOB1H8gsynSpEqM2q0aH4QpaFB5BO8KELwn05/uMvgMHXsA244T0yQbAk5ilCxm5RGZoSQRFh55EVqKRQn1nC31Hu6/cyBWpvjKULYxz0CbQFQm1IxALqQABE7JRUrZCOZyQTvxXdZ2IcYOfRsgGuGVRElnvsx4ipzqiMvETEPk9N+iiWTC1Wxm5TGV/8lIzUfHQFKqk08pTy0FWz0AtYiXkS9jn8SPjn1mwhhjpu1vKJ5R8zxTISzmBLOWChl+NH4NtZdRGuHbm4znSBH5XWcEy0637I9U/+dNtazXW8cgiiQOVNQfC7Dq5GscTEMj6djSl6oiywGpq8RjPBYRAR1vfDyAMa/XK8EDSnayK0WCKbtWJEjYpscz29BNZM78U51sMTwmzvndahsjMzKiGC2rqGautAdrO+83C2nz8z6KJtCmVuZHN0cmVhbQplbmRvYmoKMTggMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNDkgPj4Kc3RyZWFtCnicTVFJigMwDLvnFfpAIV6TvKdDmUPn/9fKDoU5BAmvkpOWmFgLDzGEHyw9+JEhczf9G36i2btZepLJ2f+Y5yJTUfhSqC5iQl2IG8+hEfA9oWsSWbG98Tkso5lzvgcfhbgEM6EBY31JMrmo5pUhE04MdRwOWqTCuGtiw+Ja0TyN3G77RmZlJoQNj2RC3BiAiCDrArIYLJQ2NhMyWc4D7Q3JDVpg16kbUYuCK5TWCXSiVsSqzOCz5tZ2N0Mt8uCoffH6aFaXYIXRS/VYeF+FPpipmXbukkJ64U07IsweCqQyOy0rtXvE6m6B+j/LUvD9yff4Ha8PzfxcnAplbmRzdHJlYW0KZW5kb2JqCjE5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggOTQgPj4Kc3RyZWFtCnicRY3BEcAgCAT/VEEJCgraTyaTh/b/jRAyfGDnDu6EBQu2eUYfBZUmXhVYB0pj3FCPQL3hci3J3AUPcCd/2tBUnJbTd2mRSVUp3KQSef8OZyaQqHnRY533C2P7IzwKZW5kc3RyZWFtCmVuZG9iagoyMCAwIG9iago8PCAvQkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzOQovU3VidHlwZSAvRm9ybSAvVHlwZSAvWE9iamVjdCA+PgpzdHJlYW0KeJzjMjQwUzA2NVXI5TI3NgKzcsAsI3MjIAski2BBZDO40gAV8wp8CmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzMjIgPj4Kc3RyZWFtCnicNVG7bcUwDOw1BRcwIH4lzeMgSJG3f5s72qlI07wfVV4ypVwudckqWWHypUN1iqZ8nmam/A71kOOYHtkhulPWlnsYFpaJeUodsZos93ALNr4AmhJzC/H3CPArgFHARKBu8fcPulkSQBoU/BTomquWWGICDYuFrdkV4lbdKVi4q/h2JLkHCXIxWehTDkWKKbfAfBks2ZFanOtyWQr/bn0CGmGFOOyzi0TgecADTCT+ZIBszz5b7OrqRTZ2hjjp0ICLgJvNJAFBUzirPrhh+2q75ueZKCc4OdavojG+DU7mS1LeV7nHz6BB3vgzPGd3jlAOmlAI9N0CIIfdwEaEPrXPwC4Dtkm7d2NK+ZxkKb4ENgr2qFMdyvBi7MxWb9j8x+jKZlFskJX10ekOytygE2Ieb2ShW7K2+zcPs33/AV8Ze2QKZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDgzID4+CnN0cmVhbQp4nEWMuw3AMAhEe6ZgBH4m9j5RlMLevw0QJW64J909XB0JmSluM8NDBp4MLIZdcYH0ljALXEdQjp3so2HVvuoEjfWmUvPvD5Se7KzihusBAkIaZgplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzIwID4+CnN0cmVhbQp4nDVSS24FMQjbzym4QKXwT87zqqqLvvtvaxO9FUwwYOMpL1nSS77UJdulw+RbH/clsULej+2azFLF9xazFM8tr0fPEbctCgRREz1YmS8VItTP9Og6qHBKn4FXCLcUG7yDSQCDavgHHqUzIFDnQMa7YjJSA4Ik2HNpcQiJciaJf6S8nt8nraSh9D1Zmcvfk0ul0B1NTugBxcrFSaBdSfmgmZhKRJKX632xQvSGwJI8PkcxyYDsNoltogUm5x6lJczEFDqwxwK8ZprVVehgwh6HKYxXC7OoHmzyWxOVpB2t4xnZMN7LMFNioeGwBdTmYmWC7uXjNa/CiO1Rk13DcO6WzXcI0Wj+GxbK4GMVkoBHp7ESDWk4wIjAnl44xV7zEzkOwIhjnZosDGNoJqd6jonA0J6zpWHGxx5a9fMPVOl8hwplbmRzdHJlYW0KZW5kb2JqCjI0IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzQwID4+CnN0cmVhbQp4nDVSOW4EMQzr/Qp9IIBu2+/ZIEiR/L8NqdkUA3F0UpQ7WlR2y4eFVLXsdPm0ldoSN+R3ZYXECcmrEu1ShkiovFYh1e+ZMq+3NWcEyFKlwuSk5HHJgj/DpacLx/m2sa/lyB2PHlgVI6FEwDLFxOgals7usGZbfpZpwI94hJwr1i3HWAVSG9047Yr3oXktsgaIvZmWigodVokWfkHxoEeNffYYVFgg0e0cSXCMiVCRgHaB2kgMOXssdlEf9DMoMRPo2htF3EGBJZKYOcW6dPTf+NCxoP7YjDe/OirpW1pZY9I+G+2Uxiwy6XpY9HTz1seDCzTvovzn1QwSNGWNksYHrdo5hqKZUVZ4t0OTDc0xxyHzDp7DGQlK+jwUv48lEx2UyN8ODaF/Xx6jjJw23gLmoj9tFQcO4rPDXrmBFUoXa5L3AalM6IHp/6/xtb7X1x8d7YDGCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNTEgPj4Kc3RyZWFtCnicLVFJcgNBCLvPK/SEZqffY5crh+T/1wjKBwYNi0B0WuKgjJ8gLFe85ZGraMPfMzGC3wWHfivXbVjkQFQgSWNQNaF28Xr0HthxmAnMk9awDGasD/yMKdzoxeExGWe312XUEOxdrz2ZQcmsXMQlExdM1WEjZw4/mTIutHM9NyDnRliXYZBuVhozEo40hUghhaqbpM4EQRKMrkaNNnIU+6Uvj3SGVY2oMexzLW1fz004a9DsWKzy5JQeXXEuJxcvrBz09TYDF1FprPJASMD9bg/1c7KT33hL584W0+N7zcnywlRgxZvXbkA21eLfvIjj+4yv5+f5/ANfYFuICmVuZHN0cmVhbQplbmRvYmoKMjYgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMTUgPj4Kc3RyZWFtCnicNVE5DgMhDOz3Ff5AJIwveE+iKM3+v82M0VYewVyGtJQhmfJSk6gh5VM+epkunLrc18xqNOeWtC1zgLi2vC+tksCJZoiDwWmYuAGaPAFD19GoUUMXHtDUpVMosNwEPoq3bg/dY7WBl7Yh54kgYigZLEHNqUUTFm3PJ6Q1v16LG96X7d3IU6XGlhiBBgFWOBzX6NfwlT1PJtF0FTLUqzXLGAkTRSI8+Y6m1RPrWjTSMhLUxhGsagO8O/0wTgAAE3HLAmSfSpSz5MRvsfSzBlf6/gGfR1SWCmVuZHN0cmVhbQplbmRvYmoKMTUgMCBvYmoKPDwgL0Jhc2VGb250IC9EZWphVnVTYW5zIC9DaGFyUHJvY3MgMTYgMCBSCi9FbmNvZGluZyA8PAovRGlmZmVyZW5jZXMgWyA0OCAvemVybyAvb25lIC90d28gL3RocmVlIC9mb3VyIC9maXZlIC9zaXggNTYgL2VpZ2h0IC9uaW5lIF0KL1R5cGUgL0VuY29kaW5nID4+Ci9GaXJzdENoYXIgMCAvRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Gb250RGVzY3JpcHRvciAxNCAwIFIKL0ZvbnRNYXRyaXggWyAwLjAwMSAwIDAgMC4wMDEgMCAwIF0gL0xhc3RDaGFyIDI1NSAvTmFtZSAvRGVqYVZ1U2FucwovU3VidHlwZSAvVHlwZTMgL1R5cGUgL0ZvbnQgL1dpZHRocyAxMyAwIFIgPj4KZW5kb2JqCjE0IDAgb2JqCjw8IC9Bc2NlbnQgOTI5IC9DYXBIZWlnaHQgMCAvRGVzY2VudCAtMjM2IC9GbGFncyAzMgovRm9udEJCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9Gb250TmFtZSAvRGVqYVZ1U2FucyAvSXRhbGljQW5nbGUgMAovTWF4V2lkdGggMTM0MiAvU3RlbVYgMCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL1hIZWlnaHQgMCA+PgplbmRvYmoKMTMgMCBvYmoKWyA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMAo2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDMxOCA0MDEgNDYwIDgzOCA2MzYKOTUwIDc4MCAyNzUgMzkwIDM5MCA1MDAgODM4IDMxOCAzNjEgMzE4IDMzNyA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2CjYzNiA2MzYgMzM3IDMzNyA4MzggODM4IDgzOCA1MzEgMTAwMCA2ODQgNjg2IDY5OCA3NzAgNjMyIDU3NSA3NzUgNzUyIDI5NQoyOTUgNjU2IDU1NyA4NjMgNzQ4IDc4NyA2MDMgNzg3IDY5NSA2MzUgNjExIDczMiA2ODQgOTg5IDY4NSA2MTEgNjg1IDM5MCAzMzcKMzkwIDgzOCA1MDAgNTAwIDYxMyA2MzUgNTUwIDYzNSA2MTUgMzUyIDYzNSA2MzQgMjc4IDI3OCA1NzkgMjc4IDk3NCA2MzQgNjEyCjYzNSA2MzUgNDExIDUyMSAzOTIgNjM0IDU5MiA4MTggNTkyIDU5MiA1MjUgNjM2IDMzNyA2MzYgODM4IDYwMCA2MzYgNjAwIDMxOAozNTIgNTE4IDEwMDAgNTAwIDUwMCA1MDAgMTM0MiA2MzUgNDAwIDEwNzAgNjAwIDY4NSA2MDAgNjAwIDMxOCAzMTggNTE4IDUxOAo1OTAgNTAwIDEwMDAgNTAwIDEwMDAgNTIxIDQwMCAxMDIzIDYwMCA1MjUgNjExIDMxOCA0MDEgNjM2IDYzNiA2MzYgNjM2IDMzNwo1MDAgNTAwIDEwMDAgNDcxIDYxMiA4MzggMzYxIDEwMDAgNTAwIDUwMCA4MzggNDAxIDQwMSA1MDAgNjM2IDYzNiAzMTggNTAwCjQwMSA0NzEgNjEyIDk2OSA5NjkgOTY5IDUzMSA2ODQgNjg0IDY4NCA2ODQgNjg0IDY4NCA5NzQgNjk4IDYzMiA2MzIgNjMyIDYzMgoyOTUgMjk1IDI5NSAyOTUgNzc1IDc0OCA3ODcgNzg3IDc4NyA3ODcgNzg3IDgzOCA3ODcgNzMyIDczMiA3MzIgNzMyIDYxMSA2MDUKNjMwIDYxMyA2MTMgNjEzIDYxMyA2MTMgNjEzIDk4MiA1NTAgNjE1IDYxNSA2MTUgNjE1IDI3OCAyNzggMjc4IDI3OCA2MTIgNjM0CjYxMiA2MTIgNjEyIDYxMiA2MTIgODM4IDYxMiA2MzQgNjM0IDYzNCA2MzQgNTkyIDYzNSA1OTIgXQplbmRvYmoKMTYgMCBvYmoKPDwgL2VpZ2h0IDE3IDAgUiAvZml2ZSAxOCAwIFIgL2ZvdXIgMTkgMCBSIC9uaW5lIDIxIDAgUiAvb25lIDIyIDAgUgovc2l4IDIzIDAgUiAvdGhyZWUgMjQgMCBSIC90d28gMjUgMCBSIC96ZXJvIDI2IDAgUiA+PgplbmRvYmoKMyAwIG9iago8PCAvRjEgMTUgMCBSID4+CmVuZG9iago0IDAgb2JqCjw8IC9BMSA8PCAvQ0EgMCAvVHlwZSAvRXh0R1N0YXRlIC9jYSAxID4+Ci9BMiA8PCAvQ0EgMSAvVHlwZSAvRXh0R1N0YXRlIC9jYSAxID4+Ci9BMyA8PCAvQ0EgMSAvVHlwZSAvRXh0R1N0YXRlIC9jYSAwLjUgPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgL0YxLURlamFWdVNhbnMtbWludXMgMjAgMCBSID4+CmVuZG9iagoyIDAgb2JqCjw8IC9Db3VudCAxIC9LaWRzIFsgMTEgMCBSIF0gL1R5cGUgL1BhZ2VzID4+CmVuZG9iagoyNyAwIG9iago8PCAvQ3JlYXRpb25EYXRlIChEOjIwMjEwOTE2MTQ0MzIwKzAyJzAwJykKL0NyZWF0b3IgKE1hdHBsb3RsaWIgdjMuNC4zLCBodHRwczovL21hdHBsb3RsaWIub3JnKQovUHJvZHVjZXIgKE1hdHBsb3RsaWIgcGRmIGJhY2tlbmQgdjMuNC4zKSA+PgplbmRvYmoKeHJlZgowIDI4CjAwMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAwMDAxNiAwMDAwMCBuIAowMDAwMDEyNDg1IDAwMDAwIG4gCjAwMDAwMTIyMjIgMDAwMDAgbiAKMDAwMDAxMjI1NCAwMDAwMCBuIAowMDAwMDEyMzk0IDAwMDAwIG4gCjAwMDAwMTI0MTUgMDAwMDAgbiAKMDAwMDAxMjQzNiAwMDAwMCBuIAowMDAwMDAwMDY1IDAwMDAwIG4gCjAwMDAwMDAzOTcgMDAwMDAgbiAKMDAwMDAwNzM2MyAwMDAwMCBuIAowMDAwMDAwMjA4IDAwMDAwIG4gCjAwMDAwMDczNDIgMDAwMDAgbiAKMDAwMDAxMTAzMSAwMDAwMCBuIAowMDAwMDEwODMxIDAwMDAwIG4gCjAwMDAwMTA0NzkgMDAwMDAgbiAKMDAwMDAxMjA4NCAwMDAwMCBuIAowMDAwMDA3MzgzIDAwMDAwIG4gCjAwMDAwMDc4NTEgMDAwMDAgbiAKMDAwMDAwODE3MyAwMDAwMCBuIAowMDAwMDA4MzM5IDAwMDAwIG4gCjAwMDAwMDg1MTEgMDAwMDAgbiAKMDAwMDAwODkwNiAwMDAwMCBuIAowMDAwMDA5MDYxIDAwMDAwIG4gCjAwMDAwMDk0NTQgMDAwMDAgbiAKMDAwMDAwOTg2NyAwMDAwMCBuIAowMDAwMDEwMTkxIDAwMDAwIG4gCjAwMDAwMTI1NDUgMDAwMDAgbiAKdHJhaWxlcgo8PCAvSW5mbyAyNyAwIFIgL1Jvb3QgMSAwIFIgL1NpemUgMjggPj4Kc3RhcnR4cmVmCjEyNzAyCiUlRU9GCg==\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2021-09-16T14:43:19.922594\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.4.3, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["sns.set()\n", "plot_args = {\"color\": to_rgb(\"C0\") + (0.5,), \"edgecolor\": \"C0\", \"linewidth\": 0.5, \"width\": 1.0}\n", "plt.hist(imgs.view(-1).cpu().numpy(), bins=256, density=True, **plot_args)\n", "plt.yscale(\"log\")\n", "plt.xticks([0, 64, 128, 192, 256])\n", "plt.show()\n", "plt.close()"]}, {"cell_type": "markdown", "id": "b8f425d1", "metadata": {"papermill": {"duration": 0.054971, "end_time": "2021-09-16T12:43:20.809613", "exception": false, "start_time": "2021-09-16T12:43:20.754642", "status": "completed"}, "tags": []}, "source": ["As we would expect from the seen images, the pixel value 0 (black) is the dominant value, followed by a batch of values between 250 and 255.\n", "Note that we use a log scale on the y-axis due to the big imbalance in the dataset.\n", "Interestingly, the pixel values 64, 128 and 191 also stand out which is likely due to the quantization used during the creation of the dataset.\n", "For RGB images, we would also see two peaks around 0 and 255,\n", "but the values in between would be much more frequent than in MNIST\n", "(see Figure 1 in the [PixelCNN++](https://arxiv.org/pdf/1701.05517.pdf) for a visualization on CIFAR10).\n", "\n", "Next, we can visualize the distribution our model predicts (in average):"]}, {"cell_type": "code", "execution_count": 26, "id": "45d72339", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:43:20.943659Z", "iopub.status.busy": "2021-09-16T12:43:20.935876Z", "iopub.status.idle": "2021-09-16T12:43:22.354374Z", "shell.execute_reply": "2021-09-16T12:43:22.354760Z"}, "papermill": {"duration": 1.486584, "end_time": "2021-09-16T12:43:22.354905", "exception": false, "start_time": "2021-09-16T12:43:20.868321", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUgovR3JvdXAgPDwgL0NTIC9EZXZpY2VSR0IgL1MgL1RyYW5zcGFyZW5jeSAvVHlwZSAvR3JvdXAgPj4KL01lZGlhQm94IFsgMCAwIDM4NC43IDI1MS45ODA2MjUgXSAvUGFyZW50IDIgMCBSIC9SZXNvdXJjZXMgOCAwIFIKL1R5cGUgL1BhZ2UgPj4KZW5kb2JqCjkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMiAwIFIgPj4Kc3RyZWFtCnic1Z1LryTHcYX3/St6KS2YzEdEPpYkaA/AHSnCXhheCDRli+DIoAhJf98RVfdWnaqOAGhRFBBDDKf7nO7sG9/tyldlZpTn949PPyvP//7pKf975uf38vdv8viDPn9kefbx0SalIY9+eHtUuaQ1c68sUr4+/Z/H4w+PnFYZnUbmOZ/3J7RyWT2P+fyzfuCHlxccTx63Vz+o6qeP1Gj7NPnJxkgMyg/vCsnP+f4j7e9CZfshf3zeimuN0nzWMhLR88/fPf/9+afnp5/Vnc2X8vd7+buz+fSL7/76x2+/+/rD589vf3rwTFxqp3r52UDFz3787vHV88f3gnMqLNzfy96efnhTHz8+igD6JIvFnPJWWCbm50o0i5T27cfH5988P/3X8izl+c0ftl/XN//1+I/nb/Jvn//5/ObLx7988/hq+7x/RKwPibXIG3tuzOUSLMq/PNoiRe+lldbpZ4Tb6deKt5ac+iLql3BB/eXR1iyaFpZrHe1nRFvq/NXCnV3C4dbXNV6Q/wEBj57qVlors/6cgFf9tQJuvaY5epUK5lKxgPzLA25cU9tKo7zKzwi4cv+1At7KGTV1+Q1QOerQU/l7gxypPvtMta/SedY19/g0qpT7HKyxncH+pvz2+c33j57Wkq8CF+51D1/rLjFKSyx/etd4n5TkklvtLGRIFFpK214slH7Uj8lD26n8/rBQKrk20vdJCWNtP8a3H5X5J1989/3v/+0vv/v9n3765OMf//SXn55f/O/zq18DtlyuibL8/POgDdIvwV0qpUxSabTacvmn8K4BeEsnpQzuo5+8T+kX8R4rzdJbzvKXgXcZ2wX9awAvPx94m2vD988GXtvSjlqldnbJTunv7gQ0IV4bJ55r9plbXr/sGz6EVZt5smD2eed33v8PUG2PS+q83FYtLL8j8bc+rATw0L7CWy+Wtk6vAPqb//rn5fVff3j8zFdKB/nBQ/rPU1vVT+S71Vrq8kzeJd3RJeEP+bbcnB/AqRrd6msulY+iLrL0nT8PScaJ/+OjaynS+WovZE6Hs1xdXdwLL1SjcvGiFy5F+im0yOByOPJozpI7X2ihGpaLE71wka7SkN87vXI5nC6V1ySp26+0QA3LxYleuLQ0tz7uK5fD6SzdgjaJrrRADcvFiV64yNidi7zzlcvh6KO2eu13WocalosTvXCRUeAewwuXw5F398Frr1+Q1qGG5eJEL1zkR66LyeByOEKAyyqt3GkdalguTvTCZepjqT5fuRyOtM0yzs6lXGihGpaLE71wkUFJ1or0lcvhyDeD5MGkCy1Uw3Jxov/4GFlGAjJQGy9cTkcIyPXC0pe70TrUqFy86IVLTWVyrQaXwxECkyrxvNM61LBcnOiFS0ttUKH5yuVwpCZptdPW8pwloRqWixO9cKHE8uvvBpfD0ZZnSit0pYVqWC5O9MKF05DmVkbFL1wOh3timqvNCy1Uw3JxohcuPS3KKxtcDoelDcqcV7vSAjUsFyd64TJTkWJeh9OHISPoQmstvrBCNSwVO3aBIlVn0am8FyjvBktMg3TODkihGJaJHfrHx5QPzaO/ts+HwXLJyMBwzgsoEKMycUIXJkVe2nmVVyiH00sqfa5KiArFsFSc4AVLTXPqffJXLIcjvZM1VmG6wEI1LBcneuFCKQ+i9jrbfTqdUs0yKqQLLVTDcnGiFy6cam+NDS6HI3VrbVU+50IL1bBcnOiFS0/E2/KLFy6HozfxmXi0Ky1Qw3JxohcuI3UqZRlcDkd6s7WPVa+wTjEsFSd2oTLTbLmU15nu0+lVGqBSR72wQjUsFyf6j4+Vk7wyt1cupyPXy+L61hodJaEalYsXvXAp0qaMxa+93NNhqWl7p3WjBWpYLk70wkXqzn2NyAuXwxECNGm2O6x3MSwVJ3ah0hJPljhfqRzOqIJijckXVqiG5eJEL1wojUGjvM5znw5L/5blUb3QQjUsFyd64cJp9Sb/vXI5HOm9yZtnvcI6xbBUnNiFykiFK/PrKPp0ek5TV26VCytUw3JxohcuU2IoNAwuh0MttdnKdr/sLAnVsFyc6IXLStykkNdZ7tPR9T5t0SwXWqiG5eJE//FRck6j6ErtFzBgaYNc+1rzygvUqGTc+BVNSSvL215nusHilmqZeatOkBjIcdk4AJRNS9IVKWyxOSyeWtWuXm7IQI7LxgGgbCjV2bN1Rb07vJL8WwbdgIEcl4wdvoLRBQmv/d53nUja6EltXVChGpeJFbkS6al3KcJg8u5I77ZyHdsq1PdyUIvLxI5dqYw0uU56nfYGS2oRWmVuc1BAC9S4ZJz4Fc1KmcroFprDYl2gq7sFbsRAjsvGASBsipSjN+CNq+m0pGGek/fFYogM5LBsPADKpiTS11psDot1p2KvtG7IQI7LxgGgbGqShoaNHQJgCYRZM+39vAuyQ47LxgGgbFoaS28RGWwOi6beFyDiKzKU47JxACgb6ZnM3oydAmAxyxdk0jYNA4WhHJeNA0DZdN2RJxEabA5LKt1WuVS+IQM5LhsHgLKRKHSvv9ETPi25eCqN1soVGcpx2TgAlM1MohRj1wBY8gXRuZk7MpTjsnEAKJuVxnZEhMHmsHikUvPsfEMGclw2DgBhU3NaLS9j9wBYVHTXP+3zEWdhKIdl4wFQNjVJtNPYQQAW64qq7cCGCzKU47JxACgbaWvynMYuArBIQl1j8LoiQzkuGweAsiEZS49h7CQAi3QzhdS55YYM5LhsHADKhlOfvRu7CcDqchlliZKuyFCOy8YBoGy61KfMxo4CsPpMI/dGN2KnGpeME76SmSl3ImNPAVikk3u593UFhnJcNg4AZbMkPonDYnNYejNu9DlvyFCOy8YBoMdv6VtrMwbhhyMDg95auwNDOSwZJ3wFUyQ8c7RwONT1vnbfJ25OXKDG5WJHr1yqTksVo+U+HNZ9f5kWXWihGpeLHf1+jl3WXTfG9PlpMQmDlqlfeYEal4wTv6LhJN3abOw1AGvIMLIV7cRciKEcl40DQNlobdGXsd8ALOnyTtb9fxdioMYl44SvZEbiwdPYcQAWLSmy8LZAD4GBHJeNA0DZSGe20zB2HYAlDXTr0uMtN2Qgx2XjAFA2K63tiEiDzWERiUxlO2UMkYEcl40DQNhIT79Q7cb+A7BI90rmVviGDOSwbDwAyqamJtEZexDAYok3F/myXJGhHJeNA0DZtMT7CcKvbA5LviC6XbLMKzKU47JxACgbkt/9diDgK5vDopFGKXPbeIDIQI7LxgGgbKSLsmY1diSApYsAypAir8hQjsvGAaBsRspzVGNXAlgk5fdK+zzfWRjKcdk4AJTNTHX0YuxMAEsngnX93o3YqcYl44SvZJb85jmzMf4+Lbl06hzy5woM5bhsHADCRuLrrPdNXtmcFmmiAJr7yoCzMJTDsvEAKJuiJzBNY5cCWNQTZ9Kbc1dkIMdl4wBQNi3pjUhro8JpSQ2zdEy5bshAjsvGAaBsKNWah7VT4bSk0ZZu8KjzhgzkuGwcAMqGE2U9FMNgc1jS2essz/sNGchx2TgAlI3UGWuydUm9O9sCR35bv3YWhXJcMnb4Cka6/NbG5cOQDkxva456gYVqXCpm7AplpjU0480rlXenVz1QpNaGrFCMS8UOXrDoAV7SBlu7FU5Lrpxea+31ggvVsGS8+BVN0dNTqrVb4bT0mNLatvPPoCxQ45JxwlcyNTG115P8f0CrDWmItvdfgKEcl40DQNm0NFq1chmApVMQdfI+j3UWhnJcNg4AZUNpqWPMm58W6abutd9uAWKnGpeME76S6ankZeUzAKvJMFvGBbNcgaEcl40DQNkMLcfKaQAWzcRj7MfRQ2Eox2XjAFA2M9Ec+spXNoelg2sq+wFGiAzkuGwcAMpmpT76sHYqnJZu86G2nZCGxE41LhknfCEzZPzT2cpuAJZmzqy88g0YymHZeACUjfT3mawMB2BJQ03Uy7786CwM5bhsHADKpqVKzcpyABbpgeOllnZDBnJcNg4AZSO/+1atTAdgtZV49v24ZCgM5bhsHADKRkeKxcp2AJZcPIW47x2/szCU47JxACibnmbJVsYDsLbeb3tbcA7IQI7LxgGgbEZaa1lZD8CSbszgsfY1SIAM5LhsHADKZqUyp5X5AKxt2b10hOsNGchx2TgAhM3cVtRb2Q/AkgYp987tjgzksGw8AMqmJO5sZUAAS74gozfNMHxFBnJcNg4AZSOhMBlZEE6HWlpt5H3qHICBHJeMHb6CkfDIXH90OKQnE+oxuVdcoMblYkevXDiVVo1sCKdDWrO0t5XVJy1Q43Kxo1cuXTv7VkYEsCTcPjnvg6iTF6hxyTjxK5qRuGQrKwJYerSy1LO9XYmhHJeNA0DZTHn1sjIjgMVZGqH8troGkIEcl40DQNksPf/Kyo4AFnFqi8e++ecsDOW4bBwAwmaVlHVzstFwn5YMJJuWWG7IQA7LxgOgbGqqvVtZEsCimQbl/eh/RAZyXDYOAGXTEmliFWP2/LS4JhkezH32/CwM5bhsHADKhlInsvIlgNWLTgv3OzKU47JxACgbTrNVK2cCWBJv0QTJN2Qox2XjAFA2I+VqZk0Da0tMUyrzFRnKcdk4AJSN3qnNVvYEsHTIxDlviUyhMJTjsnEAKJslfZRlZVAASzoygiDv81mADOS4bBwAHx81y+fOaWVRAIs0q9Gae/8GkIEclY0LQNmUNMawcimApV+Q7QNuyECOy8YBoGxqWr1b+RTAEghEvW/Hdl+RHXJcNg4AZUOpiGTsVgBLB06N5rZZ94rskOOycQAoG4mPyEqrAJZULEVT+/QbMpDjsnEAKJueRLDyKoAljXUjrnndkIEcl40DQNmMNGq18iqApVs2mm5RuCEDOS4bB4CymWmVYuRVOB1a8mCWO69TjcvFDl6wlJxeZ0J3VfcSZhrbusazBFTD8jCiVhYl1TmNfAqnMzVDQKZMByVU4vKwI1cmNZEuybO+JIcl7XGtbT/L8iwL1bhknPgVjTyRDpqxPwEsGrr9dN/GjcRAjsvGAaBs5EfXgweta+mwpCrJeVaaV2Qox2XjAFA2XUc9VjYFsGjqkHo/Yg4KQzkuGweAstFM0s3KpgAWtTSy3sW9EjvVuGSc8JWMtDO1WrkUwNLOv3w9xh0YyHHZOACUzZInxcqlAJZ0cCetwfOGDOS4bBwAwqbmNHO2cimApVtPsxTXbshADsvGA6BsSlrSUzF2KIAlFw8tzts2S0QGclw2DgBl01LRTKcWm8PSRZ6F1movyA45LhsHgLKh1PqwcimApYepFbmQ+g0ZyHHZOACUDSeRrFwKYGn3d0njtG7IQI7LxgGgbOS6ILJyKYDVdel92/IeQVmgxiXjhK9kRlqtWZkUwNKBQatMN2Aox2XjAFA2KxXxjR0KYElTXWpTHBdkKMdl4wAQNi2nJnWpsUMBLP2CyAftdxHOwlAOy8YDoGxK0hvWxg4FsKTKnXPxttwTCkM5LhsHgLLRVa7LyqUA1rYUNq9t8TQiAzkuGweAsmmaYdnKpQCWfEHEpFxuyECOy8YBoGw45T6MXAqno1veSb4ffAMGclwydvgKpqdq7mg+nbeTlwtdcYEal4sdvXLRn5mNXAqno+fLlba21cFAC9S4XOzolYumFiErlwJYLF+SSnPvCx9loRqXjBO/ollJ3mTlUgBLqpPF5b3+PQpDOS4bB4CwoZJEt3IpgKW3stdsrd2QgRyWjQdA2dRUc7ayKYCl0w9Uy3acGiIDOS4bB4CyaanNZWVTAEtvrvQXXKEznruBKxNKPKaVRQEsPdpdSt9veF9QHXJcNg4AZcNp6M1IY9b8tCRe6mNPUY3IQI7LxgGgbLq0Md3KogCWXjgste28IQM5LhsHgLLRepStLApg6TQn17EdqYHIQI7LxgGgbJau2rR2MYOlHZhaqNMNGchx2TgAhI1myq3NyqIAli4lL6NtS8kRGchh2XgAlE1JclFYWRTA0gZJ/i03Yqcal4wTvpKpSV5s5VAAa0sJUOp+b/csDOW4bBwAyoZSXtnKogCWNkw6cKpXZCjHZeMAUDac6phWFgWwNBVJZh7tBdkhx2XjAFA2XXtvVhYFsKQjPEmGkuWGDOS4bBwAymakzt3KogCWrsPqtc36guyQ47JxACgbaWqIrSwKYPFKpTce64oM5bhsHADCpusdWrKyKIAlOMoaJd+RgRyWjQdA2RRdJW1kUTgdzWsuCOjKC9S4XOzgFYsMos3FR++Gnp5b+pZ45CwGxLhMzMgViTzJxcigcDqaCGp12g6MOEmBGJeKHbxiIbkOspVBASzp7QqLtZ3cc5aFalwyTvyKhtMay8qgAJYmM5cOHa0rMZTjsnEAKBtpV7TxtS6nw9KF01zmWi/IDjkuGweAsplJeiNWDgWwBEJv/e0uywXZIcdl4wBQNiux7jcwZs1PS2/eSq+X6w0ZyHHZOACEzcg6brayKICllW6Tvt0NGcph2XgAlI2e296sLApg0ZKOjPRg6IoM5bhsHADKpqVSqr7ylc1h6YQntTlekR1yXDYOAGWznY82rD0Kp6WH5u7p1K7Acuwca27wyoV136254ui09KCIPim3Gy6Q47JxACgbaYPHsnIogKWz5FzLlpoFCkM5LhsHgLIZacrv3tqfcFp6sHura5/7PAtDOS4bB4CykRqDh5VDASy9TUltlPKC7JDjsnEACJsp5VC3ciiAtWWmLv3tmjqRgRyWjQdA2RTpu7GVQwEs3TBXpchyQwZyXDYOAGVTU69k5VAAS88Y0Twt5YYM5LhsHADKpqVZmpVDASxpkFam0tcVGcpx2TgAlI2eqVesHApgcdPqdzW+IQM5LhsHgLLRM66ylUMBLKl+x8y072w5C0M5LhsHgLKRKMYyciicDhdNfPS2BvQsCuW4ZOzwFcxM3K1zUU9H6lsqo+Z5wYVqXC529MplaaIeI4fC6dAUFDT2qb6TFqhxudjRC5clDYzeaDMmz09L19MQ076R+SgL1bBkvPgVjZ6hzFYOBbBY4s1jy42FwE41LhknfCXTUtNjRYyZ89Piodss15aKD4GBHJeNA0DZUOLSrAwKYPWS5mi8bshQjsvGAaBsWC6LamVQAEuG2FW+H/u64bMwlOOycQAom57mKlYGBbAEQqfytnrkguyQ47JxACibmfLMVgYFsHSIPSbvpy8DMpDjsnEAKJulJ29bGRTAkqa6SZdmP5/vLAzluGwcAB8fLctbeVgZFMDinnjpCrUbMpCjsnEBKJsidYY5UgBLBwW1lm26EwpDOS4bB4CyqWnqQkWLzWGRbrncM2xcib2rcck44SsZ0l3JVv4EsDR5N+mujSswlOOycQAoG5YeSrPyJ4Alw8lOeW1H1UBhKMdl4wBQNtIK52rlTwBL97GMWl+AvYlxuTjBK5chrUyxcieApZtYVm2jX3GhHJeNA0DZTJ3DtHIngCUj7UplX1SOyECOy8YBoGxWWn1ZuRPA0kRHVRqj9YLskOOycQAIm1JS4WnlTgCLW6I513ZQDSIDOSwbD4Cykd+9NDjGHgWwmFIbZXa+IQM5LhsHgLJpiVs3ciecDnPisma9AwM5Lhk7fAVDaRgrkN513cUyNC/LFRWocZlYkSsR3o4tep3jOx2pSYZUKWsiKdDiMrFjVyojFT01+HXCHCwZV+fWSm9XWqDGJePEr2h0d0qxsiiAJQMkGSrlwTdiIMdl4wBQNkva3GxlUQBLx9W9ze0gaigM5bhsHADCpsqr+7KyKIClnd285zSCskANS8YLX8nIVcHTyqEAFktkMqreDjdCYCDHZeMAUDYtZRpWFgWwWBc10pqvyA45LhsHgLLRDGDdyqIAlnReRi7bDhYkdqpxyTjhKxlOVNnKoQCWdHU7SdVLN2Agx2XjAFA2XZ6QlUMBLNYkWXNuC+4RGchx2TgAlI3UGLlZ25jBGjm1mWu/IUM5LhsHgLKZaa1q5VAAS6rcXOZcV2KgxiXjhC9kWk5lZiuDAlg9y7hpvqM5CkM5LBsPgLIpqfVlZVAAS5e/jpy3JZ5QGMpx2TgAlE1NLO8zdiiAJc3RqrydD4vETjUuGSd8JdPSoGFlUACLl1S4vTDfgIEcl40DQNlQWq1bGRTA0mqllVLpigzluGwcAMqmJ2lvrAwKYHXt4L1PZwEykOOycQAoG12tSFYGBbB60enfsR09gshAjsvGAaBsZuLcrAwKYLEe2p175isylOOycQAomyWvrkYGhdPpVVO0zH2QeRaFclwydvgChnKaZr6105G2SK4fyldcqIbl4kSvXGrKIxsZFE6Hu2a6zHNeaYEal4sdvXJpqfKyMiiApR270TnfeIEal4wTv6Iheeu0MiiA1Vsqi/etCFAYynHZOACUDev5PFYGBbAEwtC93P0F2SHHZeMAUDY9zcpWBgWwdNJqcH9jcxSGclw2DgBlM1PWBURGw31a8gVh0izMV2Qox2XjAFA2K9XcrEwKYHXSpBK0rzkCZCDHZeMAEDacU9ND04y589PqOtzmth0CgMhADsvGA6BsZBwkj40dCmAJhCrDyPaK7JDjsnEAKBsJRZPCGv3h09ounrplcERipxqXjBO+kmlp8bLyKIAlX49Mb1OgwOtU45JxwlcynApNK4sCWNIYrTX27XIIDOS4bBwAyqan1oaVRQGsviUS4+0Ac0QGclw2DgBlMxJLpWrtTzitkkui3ujtmjpKu+hx6TgIlM5Mo7CVSQGs0paMDXqd9UYN9bh0HARKR8/oaVYuBbCKVMFl0XqrdE5qqMel4yAQOr1ojmUrmwJYRfOKUR08b9RQD0vHQ6B0quZgsfIpgFVqT6WzNFM3aqgfdD79rGro9fnls6SqUZXnh+enX3z31z9++93XHz5/fvvT40bw45sg4/wxN+WHx+N3jzbkIr687E25ve6ltPsb4VXnW73Svnr8H9H6gQkKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iago2Njk3CmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjE3IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzk1ID4+CnN0cmVhbQp4nD1SS27FQAjb5xRcoNLwm895UlXdvPtva0NSqSq8iTHGMH3KkLnlS10ScYXJt16uWzymfC5bWpl5iLuLjSU+ttyX7iG2XXQusTgdR/ILMp0qRKjNqtGh+EKWhQeQTvChC8J9Of7jL4DB17ANuOE9MkGwJOYpQsZuURmaEkERYeeRFaikUJ9Zwt9R7uv3MgVqb4ylC2Mc9Am0BUJtSMQC6kAAROyUVK2QjmckE78V3WdiHGDn0bIBrhlURJZ77MeIqc6ojLxExD5PTfoolkwtVsZuUxlf/JSM1Hx0BSqpNPKU8tBVs9ALWIl5EvY5/Ej459ZsIYY6btbyieUfM8UyEs5gSzlgoZfjR+DbWXURrh25uM50gR+V1nBMtOt+yPVP/nTbWs11vHIIokDlTUHwuw6uRrHExDI+nY0peqIssBqavEYzwWEQEdb3w8gDGv1yvBA0p2sitFgim7ViRI2KbHM9vQTWTO/FOdbDE8Js753WobIzMyohgtq6hmrrQHazvvNwtp8/M+iibQplbmRzdHJlYW0KZW5kb2JqCjE4IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjQ5ID4+CnN0cmVhbQp4nE1RSYoDMAy75xX6QCFek7ynQ5lD5//Xyg6FOQQJr5KTlphYCw8xhB8sPfiRIXM3/Rt+otm7WXqSydn/mOciU1H4UqguYkJdiBvPoRHwPaFrElmxvfE5LKOZc74HH4W4BDOhAWN9STK5qOaVIRNODHUcDlqkwrhrYsPiWtE8jdxu+0ZmZSaEDY9kQtwYgIgg6wKyGCyUNjYTMlnOA+0NyQ1aYNepG1GLgiuU1gl0olbEqszgs+bWdjdDLfLgqH3x+mhWl2CF0Uv1WHhfhT6YqZl27pJCeuFNOyLMHgqkMjstK7V7xOpugfo/y1Lw/cn3+B2vD838XJwKZW5kc3RyZWFtCmVuZG9iagoxOSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDk0ID4+CnN0cmVhbQp4nEWNwRHAIAgE/1RBCQoK2k8mk4f2/40QMnxg5w7uhAULtnlGHwWVJl4VWAdKY9xQj0C94XItydwFD3Anf9rQVJyW03dpkUlVKdykEnn/DmcmkKh50WOd9wtj+yM8CmVuZHN0cmVhbQplbmRvYmoKMjAgMCBvYmoKPDwgL0JCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzkKL1N1YnR5cGUgL0Zvcm0gL1R5cGUgL1hPYmplY3QgPj4Kc3RyZWFtCnic4zI0MFMwNjVVyOUyNzYCs3LALCNzIyALJItgQWQzuNIAFfMKfAplbmRzdHJlYW0KZW5kb2JqCjIxIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzIyID4+CnN0cmVhbQp4nDVRu23FMAzsNQUXMCB+Jc3jIEiRt3+bO9qpSNO8H1VeMqVcLnXJKllh8qVDdYqmfJ5mpvwO9ZDjmB7ZIbpT1pZ7GBaWiXlKHbGaLPdwCza+AJoScwvx9wjwK4BRwESgbvH3D7pZEkAaFPwU6JqrllhiAg2Lha3ZFeJW3SlYuKv4diS5BwlyMVnoUw5Fiim3wHwZLNmRWpzrclkK/259AhphhTjss4tE4HnAA0wk/mSAbM8+W+zq6kU2doY46dCAi4CbzSQBQVM4qz64Yftqu+bnmSgnODnWr6Ixvg1O5ktS3le5x8+gQd74Mzxnd45QDppQCPTdAiCH3cBGhD61z8AuA7ZJu3djSvmcZCm+BDYK9qhTHcrwYuzMVm/Y/MfoymZRbJCV9dHpDsrcoBNiHm9koVuytvs3D7N9/wFfGXtkCmVuZHN0cmVhbQplbmRvYmoKMjIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA4MyA+PgpzdHJlYW0KeJxFjLsNwDAIRHumYAR+JvY+UZTC3r8NECVuuCfdPVwdCZkpbjPDQwaeDCyGXXGB9JYwC1xHUI6d7KNh1b7qBI31plLz7w+Unuys4obrAQJCGmYKZW5kc3RyZWFtCmVuZG9iagoyMyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDMyMCA+PgpzdHJlYW0KeJw1UktuBTEI288puECl8E/O86qqi777b2sTvRVMMGDjKS9Z0ku+1CXbpcPkWx/3JbFC3o/tmsxSxfcWsxTPLa9HzxG3LQoEURM9WJkvFSLUz/ToOqhwSp+BVwi3FBu8g0kAg2r4Bx6lMyBQ50DGu2IyUgOCJNhzaXEIiXImiX+kvJ7fJ62kofQ9WZnL35NLpdAdTU7oAcXKxUmgXUn5oJmYSkSSl+t9sUL0hsCSPD5HMcmA7DaJbaIFJucepSXMxBQ6sMcCvGaa1VXoYMIehymMVwuzqB5s8lsTlaQdreMZ2TDeyzBTYqHhsAXU5mJlgu7l4zWvwojtUZNdw3Duls13CNFo/hsWyuBjFZKAR6exEg1pOMCIwJ5eOMVe8xM5DsCIY52aLAxjaCaneo6JwNCes6VhxsceWvXzD1TpfIcKZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDM0MCA+PgpzdHJlYW0KeJw1UjluBDEM6/0KfSCAbtvv2SBIkfy/DanZFANxdFKUO1pUdsuHhVS17HT5tJXaEjfkd2WFxAnJqxLtUoZIqLxWIdXvmTKvtzVnBMhSpcLkpORxyYI/w6WnC8f5trGv5cgdjx5YFSOhRMAyxcToGpbO7rBmW36WacCPeIScK9Ytx1gFUhvdOO2K96F5LbIGiL2ZlooKHVaJFn5B8aBHjX32GFRYINHtHElwjIlQkYB2gdpIDDl7LHZRH/QzKDET6NobRdxBgSWSmDnFunT03/jQsaD+2Iw3vzoq6VtaWWPSPhvtlMYsMul6WPR089bHgws076L859UMEjRljZLGB63aOYaimVFWeLdDkw3NMcch8w6ewxkJSvo8FL+PJRMdlMjfDg2hf18eo4ycNt4C5qI/bRUHDuKzw165gRVKF2uS9wGpTOiB6f+v8bW+19cfHe2AxgplbmRzdHJlYW0KZW5kb2JqCjI1IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjUxID4+CnN0cmVhbQp4nC1RSXIDQQi7zyv0hGan32OXK4fk/9cIygcGDYtAdFrioIyfICxXvOWRq2jD3zMxgt8Fh34r121Y5EBUIEljUDWhdvF69B7YcZgJzJPWsAxmrA/8jCnc6MXhMRlnt9dl1BDsXa89mUHJrFzEJRMXTNVhI2cOP5kyLrRzPTcg50ZYl2GQblYaMxKONIVIIYWqm6TOBEESjK5GjTZyFPulL490hlWNqDHscy1tX89NOGvQ7Fis8uSUHl1xLicXL6wc9PU2AxdRaazyQEjA/W4P9XOyk994S+fOFtPje83J8sJUYMWb125ANtXi37yI4/uMr+fn+fwDX2BbiAplbmRzdHJlYW0KZW5kb2JqCjI2IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjE1ID4+CnN0cmVhbQp4nDVROQ4DIQzs9xX+QCSML3hPoijN/r/NjNFWHsFchrSUIZnyUpOoIeVTPnqZLpy63NfMajTnlrQtc4C4trwvrZLAiWaIg8FpmLgBmjwBQ9fRqFFDFx7Q1KVTKLDcBD6Kt24P3WO1gZe2IeeJIGIoGSxBzalFExZtzyekNb9eixvel+3dyFOlxpYYgQYBVjgc1+jX8JU9TybRdBUy1Ks1yxgJE0UiPPmOptUT61o00jIS1MYRrGoDvDv9ME4AABNxywJkn0qUs+TEb7H0swZX+v4Bn0dUlgplbmRzdHJlYW0KZW5kb2JqCjE1IDAgb2JqCjw8IC9CYXNlRm9udCAvRGVqYVZ1U2FucyAvQ2hhclByb2NzIDE2IDAgUgovRW5jb2RpbmcgPDwKL0RpZmZlcmVuY2VzIFsgNDggL3plcm8gL29uZSAvdHdvIC90aHJlZSAvZm91ciAvZml2ZSAvc2l4IDU2IC9laWdodCAvbmluZSBdCi9UeXBlIC9FbmNvZGluZyA+PgovRmlyc3RDaGFyIDAgL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvRm9udERlc2NyaXB0b3IgMTQgMCBSCi9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdIC9MYXN0Q2hhciAyNTUgL05hbWUgL0RlamFWdVNhbnMKL1N1YnR5cGUgL1R5cGUzIC9UeXBlIC9Gb250IC9XaWR0aHMgMTMgMCBSID4+CmVuZG9iagoxNCAwIG9iago8PCAvQXNjZW50IDkyOSAvQ2FwSGVpZ2h0IDAgL0Rlc2NlbnQgLTIzNiAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvRm9udE5hbWUgL0RlamFWdVNhbnMgL0l0YWxpY0FuZ2xlIDAKL01heFdpZHRoIDEzNDIgL1N0ZW1WIDAgL1R5cGUgL0ZvbnREZXNjcmlwdG9yIC9YSGVpZ2h0IDAgPj4KZW5kb2JqCjEzIDAgb2JqClsgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAKNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCAzMTggNDAxIDQ2MCA4MzggNjM2Cjk1MCA3ODAgMjc1IDM5MCAzOTAgNTAwIDgzOCAzMTggMzYxIDMxOCAzMzcgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNgo2MzYgNjM2IDMzNyAzMzcgODM4IDgzOCA4MzggNTMxIDEwMDAgNjg0IDY4NiA2OTggNzcwIDYzMiA1NzUgNzc1IDc1MiAyOTUKMjk1IDY1NiA1NTcgODYzIDc0OCA3ODcgNjAzIDc4NyA2OTUgNjM1IDYxMSA3MzIgNjg0IDk4OSA2ODUgNjExIDY4NSAzOTAgMzM3CjM5MCA4MzggNTAwIDUwMCA2MTMgNjM1IDU1MCA2MzUgNjE1IDM1MiA2MzUgNjM0IDI3OCAyNzggNTc5IDI3OCA5NzQgNjM0IDYxMgo2MzUgNjM1IDQxMSA1MjEgMzkyIDYzNCA1OTIgODE4IDU5MiA1OTIgNTI1IDYzNiAzMzcgNjM2IDgzOCA2MDAgNjM2IDYwMCAzMTgKMzUyIDUxOCAxMDAwIDUwMCA1MDAgNTAwIDEzNDIgNjM1IDQwMCAxMDcwIDYwMCA2ODUgNjAwIDYwMCAzMTggMzE4IDUxOCA1MTgKNTkwIDUwMCAxMDAwIDUwMCAxMDAwIDUyMSA0MDAgMTAyMyA2MDAgNTI1IDYxMSAzMTggNDAxIDYzNiA2MzYgNjM2IDYzNiAzMzcKNTAwIDUwMCAxMDAwIDQ3MSA2MTIgODM4IDM2MSAxMDAwIDUwMCA1MDAgODM4IDQwMSA0MDEgNTAwIDYzNiA2MzYgMzE4IDUwMAo0MDEgNDcxIDYxMiA5NjkgOTY5IDk2OSA1MzEgNjg0IDY4NCA2ODQgNjg0IDY4NCA2ODQgOTc0IDY5OCA2MzIgNjMyIDYzMiA2MzIKMjk1IDI5NSAyOTUgMjk1IDc3NSA3NDggNzg3IDc4NyA3ODcgNzg3IDc4NyA4MzggNzg3IDczMiA3MzIgNzMyIDczMiA2MTEgNjA1CjYzMCA2MTMgNjEzIDYxMyA2MTMgNjEzIDYxMyA5ODIgNTUwIDYxNSA2MTUgNjE1IDYxNSAyNzggMjc4IDI3OCAyNzggNjEyIDYzNAo2MTIgNjEyIDYxMiA2MTIgNjEyIDgzOCA2MTIgNjM0IDYzNCA2MzQgNjM0IDU5MiA2MzUgNTkyIF0KZW5kb2JqCjE2IDAgb2JqCjw8IC9laWdodCAxNyAwIFIgL2ZpdmUgMTggMCBSIC9mb3VyIDE5IDAgUiAvbmluZSAyMSAwIFIgL29uZSAyMiAwIFIKL3NpeCAyMyAwIFIgL3RocmVlIDI0IDAgUiAvdHdvIDI1IDAgUiAvemVybyAyNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE1IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL0NBIDAgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PgovQTIgPDwgL0NBIDEgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PgovQTMgPDwgL0NBIDEgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMC41ID4+ID4+CmVuZG9iago1IDAgb2JqCjw8ID4+CmVuZG9iago2IDAgb2JqCjw8ID4+CmVuZG9iago3IDAgb2JqCjw8IC9GMS1EZWphVnVTYW5zLW1pbnVzIDIwIDAgUiA+PgplbmRvYmoKMiAwIG9iago8PCAvQ291bnQgMSAvS2lkcyBbIDExIDAgUiBdIC9UeXBlIC9QYWdlcyA+PgplbmRvYmoKMjcgMCBvYmoKPDwgL0NyZWF0aW9uRGF0ZSAoRDoyMDIxMDkxNjE0NDMyMiswMicwMCcpCi9DcmVhdG9yIChNYXRwbG90bGliIHYzLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjQuMykgPj4KZW5kb2JqCnhyZWYKMCAyOAowMDAwMDAwMDAwIDY1NTM1IGYgCjAwMDAwMDAwMTYgMDAwMDAgbiAKMDAwMDAxMjMxMiAwMDAwMCBuIAowMDAwMDEyMDQ5IDAwMDAwIG4gCjAwMDAwMTIwODEgMDAwMDAgbiAKMDAwMDAxMjIyMSAwMDAwMCBuIAowMDAwMDEyMjQyIDAwMDAwIG4gCjAwMDAwMTIyNjMgMDAwMDAgbiAKMDAwMDAwMDA2NSAwMDAwMCBuIAowMDAwMDAwMzk3IDAwMDAwIG4gCjAwMDAwMDcxOTAgMDAwMDAgbiAKMDAwMDAwMDIwOCAwMDAwMCBuIAowMDAwMDA3MTY5IDAwMDAwIG4gCjAwMDAwMTA4NTggMDAwMDAgbiAKMDAwMDAxMDY1OCAwMDAwMCBuIAowMDAwMDEwMzA2IDAwMDAwIG4gCjAwMDAwMTE5MTEgMDAwMDAgbiAKMDAwMDAwNzIxMCAwMDAwMCBuIAowMDAwMDA3Njc4IDAwMDAwIG4gCjAwMDAwMDgwMDAgMDAwMDAgbiAKMDAwMDAwODE2NiAwMDAwMCBuIAowMDAwMDA4MzM4IDAwMDAwIG4gCjAwMDAwMDg3MzMgMDAwMDAgbiAKMDAwMDAwODg4OCAwMDAwMCBuIAowMDAwMDA5MjgxIDAwMDAwIG4gCjAwMDAwMDk2OTQgMDAwMDAgbiAKMDAwMDAxMDAxOCAwMDAwMCBuIAowMDAwMDEyMzcyIDAwMDAwIG4gCnRyYWlsZXIKPDwgL0luZm8gMjcgMCBSIC9Sb290IDEgMCBSIC9TaXplIDI4ID4+CnN0YXJ0eHJlZgoxMjUyOQolJUVPRgo=\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2021-09-16T14:43:21.574138\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.4.3, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["plt.bar(np.arange(mean_out.shape[0]), mean_out, **plot_args)\n", "plt.yscale(\"log\")\n", "plt.xticks([0, 64, 128, 192, 256])\n", "plt.show()\n", "plt.close()"]}, {"cell_type": "markdown", "id": "329f3ea9", "metadata": {"papermill": {"duration": 0.05645, "end_time": "2021-09-16T12:43:22.469433", "exception": false, "start_time": "2021-09-16T12:43:22.412983", "status": "completed"}, "tags": []}, "source": ["This distribution is very close to the actual dataset distribution.\n", "This is in general a good sign, but we can see a slightly smoother histogram than above.\n", "\n", "Finally, to take a closer look at learned value relations, we can\n", "visualize the distribution for individual pixel predictions to get a\n", "better intuition. For this, we pick 4 random images and pixels, and\n", "visualize their distribution below:"]}, {"cell_type": "code", "execution_count": 27, "id": "17909488", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:43:22.589483Z", "iopub.status.busy": "2021-09-16T12:43:22.588638Z", "iopub.status.idle": "2021-09-16T12:43:27.526284Z", "shell.execute_reply": "2021-09-16T12:43:27.526674Z"}, "papermill": {"duration": 5.001478, "end_time": "2021-09-16T12:43:27.526816", "exception": false, "start_time": "2021-09-16T12:43:22.525338", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUgovR3JvdXAgPDwgL0NTIC9EZXZpY2VSR0IgL1MgL1RyYW5zcGFyZW5jeSAvVHlwZSAvR3JvdXAgPj4KL01lZGlhQm94IFsgMCAwIDYxMi45IDM2MC43MDA2MjUgXSAvUGFyZW50IDIgMCBSIC9SZXNvdXJjZXMgOCAwIFIKL1R5cGUgL1BhZ2UgPj4KZW5kb2JqCjkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMiAwIFIgPj4Kc3RyZWFtCnic3N1Nsyy3cSbgff+KXkoLgkh8YykG7RuhHaUbnoXDCwVNecTgpYPDkOfvT76oPoUsIEumyDuXcsoh+Z43z6lz6uluFFCFQtHz28fnv6Pnf/z45P95+ue3/N//y/9+h68fnr/68CgUXOd/fff6VyzeVe9LyBz565f/+/H488O7TrWk6nNrz/WL1D314mt7/h/8wnfbN5xfPJbvfjxSdfUZfHYhFd/4T4ueXIwlliTi72Qcc3T57a8bP39Jxt/7w/Oy4dpzyvkZ+PtKfP1felJqLnDh+L/n//nm+b+e3z8//1048H7P//32QS+8z7/85r/+8vU3f3j3xfPrHx+5u1J7qOnyt8/08hc9/vj46vnD24a9o8wvDG/7iW2PL9+90scP/Pv88zPPpVwcjY2FWvqTWnWxlerT2JOvPzy+eP/8/J/pSfR8/+fxsr7/98e/Pn/jf/v8t+f73z/+6f3jq/FrPwrE4/hjdwgi/rtC7D5cJET8ESiI/GtrsZafTlHSJ7aoyfXScrxSnOnHkCjJJWzMp5rDT6eg0D6tRYjBFYqU+wVDxB9BI4TgCFtruab0d2j08Ik1uucfrbVd2zoRfwyN2l3F1kqp8e/4mIRcPq3GsfVIrjXKlZZGf+Y/F6I6boxCdTkEHGl6i8feY5+5TW3EyLzr0+I39Nvn+28fxfXOH61MuYRDB60pF4hfG/5PKYlfoGdygXyP9dwIdmduJbnWW68M19NrK2kUWPcH/gO8r/UJ/dc/KTnihi1hg4lfsT7eBF9/wGv12ZfffPunf/nrH//0/Y+fffjL93/98fnlfz6/+oQvEjc3lRglLC/SzH/Ri8T9DO5rpFpTC128SL60+olfI//TXyP+oI2/7x/kNer8Ts+NYlpeo5n/7KNNwIvUo+OXunL3rdT8SV6k8j/1tYiR39C9JIa8vBYi/0WvRQzF+chNug/80fsUL0X4R3spIqPxMCL05mMPfJzgPyYdo4sQM39R3sYXaQxHXB6qN9//uHz/H97df+e65Qd6+yG0HsrzMz7mBj72ptoxuOIXL/QS4iX/TuQxFVcLH4nQfM7NXGIexnxhx0o3wUCU388+hpXqLQ6NHMXAm74AXmJjUprIgOqRPLVd6pWHErn3wF3ifAGUqTkphYSpgmuRKjegC9WZh4Y9r7HHq6CMjVmpJmwVXeXPEW0fwDMPjRulwK19WQhFbMxKNWGrxPsR+ei6UL3F3Ihzbzal0eGSgCI2JqWJMFR2Ocbm6yp15qFnNmm5xAugTI1JqSRMVRy/MbJPK9WZh9pcq732RVDGxqxUE7aq3J3OwW8fwDMPPXC/2ce0EMrYmJVqwlb8h8fc+N2xWJ15aNF5bsopXAllbMxKNWGrzn3JkntZrc48tOR86Km1hVDExqxUkw+P6jH+DH0d28w88OeuRsrHUXASytiWlW4yrHhE3fvaXs08tNEw+VKvhDI2Z6WYsBVxx7KVtg5vZo4hX0i+1bgQitiYlWrCVvxPHqO0dXwz81Cr85mPfuVKKGNjVqoJW/F+JO/b1l6dOU6TtpZpnHQRhDI2ZqWasFVyOeG6/EL1FuN8Zs+dUr4AXmJjUpoIQ2X+KUp1Hd/MPDTsmu9jdHxu5ZIak1JJmKq4mALVra0688DvpBhyi4ugjI1ZqSZsVXm3QqvruZiZY9CXMJcmLYQiNmalmrBVc5RiLuv4ZuboRFVcSqUroYyNWakmbNWdTymUrbk6c1zi66H0FK6EMjZmpZoMq55SL+v5mJljzkZNOY6BjCQUsTkrxeTDo3nXUi5lHd/MfFzfr7HmeCWUsS0r3YStiN8dJea1czXz6AOPaUL06UJ4iY1ZqSZsFdyYLrqej5k59wtcqKX2dCG8xMasVBO2ity3rPgkLVZnHjp8agnhSihjY1aqCVsl/jH+WK1Ub3FAR72XOi4qC0AZG5PSRBgqc8+SB3RbY3XmoWGCSqdSLoAyNSalkjBV4d3qypvqzCMmIvJW01XwEhuzUk3YquKsZk7r+GbmuABfY0u1XQgvsTEr1YStmvOZwtoNPWPucjoe8bXFT6TGnDSPwdQz9bgObWYePf9k9Z6S5Luk5qQUEqbqruVQ4vaWOvNQ0NVsPeeL4CU2ZqWafHhgLMdDuW1a2swx3PM5t3EaQRDK2JaVbsJW5EpOPqxnYmaOU3m8nRrSlVDGxqxUE7YKLvM/wzqymXno3JkKMVK9EsrYmJVqwlbR8f6k7RA4c1xSjq1hYvOFUMbGrFQTtkou5kJhPRMz89C5N+W5C0VXQhkbs1JN2Ip3K5e2TaSd+bgrCzPZw0IoYmNWqglbFe5Z1rzNpJ35MV2oxV6vhDI2ZqWasBWurbdA65mYmUfPW+bR8egczM1cYmNWqsmw6rltE2POOJTueq3FlwvgJTYntYswVOOeJb5tlTrzMbuYtx/LBVCmxqRUEqbqPPT10W9t1Znj7VNbaAugSI1JqSIfHsQNTin8v+vwRhRCxeWs2o++1SSUsS2tGxVwkeMeQNnm0ooC9wtwl40fV+Alo8yteaks8AoulYBj/+p1FqKPjkKglq6Ol9yal8oCr+h4J3An8up1FiJ513ERMC+OMrfmpbLAK3E/PNat+yAKuOutcYd9zAoVW7rk1rxUFnhlRyXhrPDqdRbCuF+QW6u8OorcmpfKAq/iPPfG23qWRhRw9TTUYwKDZBSxNS0V5dDqhX9sHfeIAnqjlbulK6KI7WkpKNCqrhWu7G3XWcAyLrzdXtqiKHNrXioLvBp3PWvYJiWLAma1x4IzfwvjjK1pqSjQ6tzxrH2bbSsKx5nk5ktcFGVuzUtlwSJl/FtLK3Vru2YBcxpaL2mc9ZOOMjfmpbPAi7jjuV9tnXkMzeWxns9V8ZJb09JQgBVcrOMmrlXrLOCGgF5TGpd15oYusTUtVQVckbtOvm4zb0XhaNIj+g5XRplb81JZ4JUcVeJx4OZ1FrBKV2+1HG2XcJS5NS+VBV7Z+Rqw8ODqdRZwL2HDcptxcZS5NS+V5fDqNbRtFq4o4Gwgjw1jXhhFbE9LQYFWca3GnPfW6yzgzpPWUml1UZS5NS+V5VgeljsDuAtn9ToLESueEW85XB0vuTUvlQVezZWa+jYlVxRCxdKqxI3V1fGSW/NSWeDVXa65pL31OguhZqwhRrWujiK35qWysFfwuIMwpm0kNAuYB1+rD3FzFLkxL50FXsSdz7ot2zPziJOonkeJ8ap4ya1paSjACjgrWuPWeM1CyOM2p0QXRJlas1JNgBWxRHGK2zmcWeDf4mLnf6cFUebWvFQWeCXud3ba5uqKAma5Ybgz7ku5OIrcmpfKcnj12vfT9WeOu8mjZ5ewKMrcntaOAqzMXXOfw3YOZxYwfySk2NKiKGNrWqoKuIrD6urbnF1RwD3lPtVjzCgURWxNS0WBVnWlUd9m7YpCiB6Pe6Dj4oZQlLk1L5UFXs3lFgptZ3BmASuyxVx8roujzK15qSzw6i61GGkbA80CVnHF9O+xgt3FUeTWvFQW9sIa7S35bQavKGA+RM0+x7I4ytyYl84CL3IBtw5s7dcsYOXNFHmLaXGUuTUvlQVewVHLyW/DoFnAfC4qEQvVXBwvuTUvlQVe3CVohbYZvaKAOREel37o6njJrXmpLIdXbwVnFnavV2HclxFzagujiO1pKSjQStz33GdKvMW45lMzHat2XwRFbs1KIYFU5m4n96O2QdAsjP4VN1C5XA1Fas1KRYFW4U5na31/Z50FHP34N+Vj9rNQlLk1L5UFXlgHo+e+nb+ZhRjGtcUX17khGVvTUlGg1VzqPuxzeWcBn7rAOx7DVfGSW/NSWeDVXey+t73tOguh4FJsw2JbV0eZW/NSWdgrYStU2nYGZxawylapkdrqKHNjXjoLvMhRD3GfzzsLWO2uVt+OY6NwlLk1L5UFXsH5HnnHNq+zEDoWjW/j1mHJKGJrWirKodV7xDN2dq1XAZPosfHxfCepKHN7XgoLvKJrnX90GwHNwpi3m2NIcXUUuTUvlQVeyfFO0XZwPHOoVMrU86545ta0NBRgZf7WzKPATessoIuVcMYmLYoitqalqoCrYNXEXLZx0CxgbVNq/CGMV8ZLbs1LZYFX5Y5nDft83lkIeL58HxeALo6X3JqXygKvMddhe2jPzHGvZ+ZOaV0QRWzNSiMBVcdGSt7b+bOAwQ5F7mqthiK2pqWqMFf23OfscZ/LOwto0UsNyeeFUebGvHQWeHEf3Xuft5ZrFuDSYm3jyZqL45lb81JZ4MWddO9r2sZAs4C18Vr2fax0Ix1lbs1LZTm8+FvTPpd3Fsb7KAcsAn51lLk9L4UFXvwe8YErm9dZwIq6HQ+iKaujyK15qSzw4n6653fKdg5nFuCCGc6a45lb81JZ4MUddR/zPp13FkKrLkcfj/UkpKPIrXmpLPAquPc1xL39Ogsh4w5Gn497/oWjzK15qSzw4r66Tz1uZ3FmAfdoRE/huFNPOMrcmpfKAi/My8VDk1eutxwzdwOVlvOqKHJrWhoKsLiv7kvcZ/TOAhop3GB2TK+fijK2pqWqMFfhrrofZ5EXrlnA/T9EvdLKKHNjXjoLvIi7nrWGbSw0C8dTtfqxRIJgFLE1LRXl0Oqe+wXbSGgWAg+uY01xPINMKsrcnpfCAq/AHc9O+wn7WYi41SzTsfDZ3JCMrWmpKNCK3O3sjbZx0CxgpY3MbyW6IsrYmpaKAq3kMBdwn807C7jqk1ssgVZFkVvzUlnglR2nwe9t11kIuKRRcu+ro8yteaks8CoucXfAb6OgWcA6LiX1WvPiKHNrXioLvHjkxwOZfTbvLMClR+6h5t3xzK15qSzwwlgm8uBv8zoLOBtYfc3H9UbhKHNrXioLvDr3zmPv21mcWRjTwvlXHTfrSUeRW/NSWdireucplX1e7yxgbwM683FxlLkxL53l8OqUY9/ar1nA/deR92xRnKk9K4UEVuQaDwW3rsSZ4zqs96Uf6z8LQZlb09JQgMX/5sZ7n9I7C2igamr9ODBORRlb01JVwMV7QrzXe7t1FnBPSyylHcMgyShya14qC7wSdzsbte0MziyM+VzcoMe4OorcmpfKAq/M3c7W9hm9sxA990f5t7R4dbzk1rxUFnjx0Y163uf0zgKeuFj4hylcHS+5NS+VBV7VheBD3c7izAKespFzOG5il4wztqalokCrOQyZ6zYKmgU8xwwPQziuBglFmVvzUlng1Xkkw0PnbRQ0C4HfUTXT66yEcJS5NS+V5fDqIXCnQPF6FULOrsWGgc/VUeb2vBQW9sJdBSH6/eLsLIRMPLDO4ThlLx1FbsxLZ4EXuRrifnA8c1xV7CVRWRBFbM1KIwEVdwZCSnlrumYB1zHwpM/x4HRhKGNrWqoKuMa0b8rbOGgWeNdcKy0e57sEo8yteaks8MJYJrd9Vu8sjCc4t3o85VM6ytyal8oCr+xiKMrb6y3HeS3fsz9uSBCKMrempaEAq3Cvs4a0ncOZBZzaKni+RrwqytialqoCrsqdztr3Gb2zEANu/azheJTE3NIlt+alssCrcaezlbS3XWcB18ha6S+uc0MytqalohxaPfQYtzM4s3A8J7x3HxZFmdvzUljg1Xkc4/0+n3cW8AiEHAMull0dZW7NS2Vhr+5djb7u83lnAc9ACLG3YwwkHGVuzEtngRfxnlCK2zmcWcD6ZsRfH2cIhaPMrXmpLPAK+ExR2AZCs4ArP7mH1/x64Shza14qC7yiS5E/WVtXdRZw6rQHX477g4SjzK15qSzwSi7y2C/s7ddZwH1AeBoCLYwitqalokCLdyymfQL0mWOt1BhS66uhzK1paSjAKo5i6vuE3llAkx6J9y9eFWVsTUtVAVfFMpWF9pbrLGDdde/z69G8glHm1rxUlsOr817Q/mE8C2jRi+/hOOclHUVuz0thgVfD3eh+n4gzC+G4HBtiXRxlbs1LZYFX545nrX5vvs4CHoldKaTjOTjCUebWvFSWD4/g+XuxAvZ6HkcUxoin+3KsAC0cZW7L64YFXsQdz07bnF5RCAXPE4/kV0eZW/NSWeCFVdV77Wv7JQq4i7H70Mf7SDrK3JqXygKv6CLuhV3P5IjCaKdC9Gl1lLk1L5UFXol3jGib0ysKuLelRn+suCEdZW7NS2WBV3aUaFusd+bjWYO4ySUvijK3pqWhAIsbbFwSW0/jiAIWyK7hdWFRKMrYmpaqcnB1nGZYx0KiEHi0GCkfa51dGEVuz0thgVd1jdvubV6vKGCMSIXquPdAOsrcmpfKAq/makrbxLiZB945ii1QWxRlbk1LQwFWdyVl7qlvWmchVMJasy3SVVHG1rRUFeYi/q3YsW0cNAuYLdgp+7QoitiYlo4CLeIfLLVuLdcs4F1Uak9tVZS5NS+VBV6BO501la3bNQsYHfKop4w7NaSjzK15qSzwirxjjbY5vaIQqfLejs7DxfGSW/NSWeCVuN/ZWtlbr7OA64pU6nF2XmzpklvzUlnglbnj2XPZzuLMAj53qb2e5SIdZW7NS2U5vHr2YZvYKwqYphR4oFjb4ihze14KC7zKWBM17+3XWRgrSITSYtsdz9yal8oCr4r7VkrezuLMAs4GJgp+zKeXjjK35qWywKu5gjVkN663fMxJTZ03vCjK3JqWhgKs7nKOPu2N11kIJbkeWhkPZ5SKIrampaow11iWK9a0vblmAUOekDv/poVR5sa8dBZ4kYs5pW1mryjgfZSxfCotjjK35qWywCvwn55pm9srCiFzH8sfU1Ilo4itaako0OLeec4tbqdxZoF3HA995iZrUZS5NS+VBV7J+cxHum0kNAvoYflcU1gYRWxNS0U5tHquYZvXKwpjfnhIx3OVLooit+elsMArc7ez4tkjq9dZwLnm2IjC6ihza14qC7xwPzon2zhoFsbzSYj3PiyOMrfmpbLAi8d+ucdtXq8oYMUN4v9JeXGUuTUvlQVezeXifdjbr7OAZ4Byl7S21VHm1rxUFnh1h/2ibSg0CyHx3nbWWR1lbs1LZWGv6F3kRmib3SsKYwWcXOO4+1M6ytyYl84CL3K830Rb+zULWBYhB0/HFW3hKHNrXioLvLjNLmF7VO/M+Xfwj5aS6qIoc2taGgqwIqZP5m1qryhgWkRqoY3lu4SijK1pqSoHVy8J87V2rlcBDx7s1DvFlVHk9rwUFnhhRQ08aH3zOgsYUUciPFbv6ihza14qC7yyqyXnfWbvLIw7NlqtPS6OMrfmpbLAqzhusMM+s3cWcIIrE+9zXRxlbs1LZYFX5a5n6X07kzMLoeDCRqQcF0eZW/NSWeDVuOtZS987E2ch4rI/1hevV8dLbs1LZYFX565ni/vk3lkIObiYa03t6njJrXmpLOyVeCv8zW1rv2YBw8WGB0rExVHmxrx0FngR9z57bdtYaBZwHEzslGlxlLk1L5UFXsH5qlylPXOcEcQTxdOqKHNrWhrKgdUrUdW0XgUMelqP7ThROBVlbE9LUQFXdK1Sq9tYaBbw5MHcow9lYZS5NS+VBV7J1RryPr93FgIT4JGgYXWUuTUvlQVe2ZW6P8Fx5pg1WHrpMS2KMrempaEAiwczNfaya50FLPPPKPF1YDwVZWxNS1UBV3WY2bzP7Z0FLOpCwfM2VkaRW/NSWeDVXOTBX9nbrrOAE6k9lHCcJhSOMrfmpbLAq7tQi8/beZxZwKzBlhu1vDjK3JqXysJe2Tuqpe5ze2cB3YeWKB3XhISjzI156SzwIu55Vv67N6+zgHXrM/+a4/MoHGVuzUtlObx4WEN5O48zCxj8YHXe47yEcJS5PS+FBV78maqt7QPHWcC1xdR7OuYwCUeZW/NSWeAVufOJvdq8zgLON/tAr7fXZBSxNS0VBVrJlcY9qO0szixgJgn5Xo6z0FJR5Na8VBZ4ZZeb30/an3moWP0511oWRZlb09JQgIUJulT2yb2zgCYKy5+VuCiK2JqWqgKu6iL/zH7byyxghMgjH0wouTLK3JqXygIv/tNb9HEbCc0CTjr7XHqti6PMrXmpLPDqjlqs++zeWRhnICKPE9viKHNrXioLexV+j7SU9tNes4CL2D3n9mq+pqPMjXnpLIdXb5nCdnCcBdyNnbhdj6ujzO15KSzwItdabvvs3lnAP73P/riZQzjK3JqXygIv/ncrmbb2axbGGZvqQ9kcRW7NS2WB13gmUKDtTM4sROL3UQu+XhllbE1LRYFW4r5n7fvc3lkYp1IjxUpXxUtuzUtlgVd2WDGC9tbrLOBUKndQ+3HbsXQUuTUvlQVehTufPfrtPM4s4C1Vyedj7CgcZW7NS2WBV+WteL/P750FuPDR8LUS7dXxzK15qSzwao66r/v83lnAD0fuQMS8OorcmpfKAq/ufKe9r/qKj3MR1F89r2koc2tWCskh1TuewKtYvQo4X+N5x4/r/6ehTO1ZKSisVb1rHY82W7VmAdNJ+L+dwqIoc2NeOgu8yOEd0rd2axbQkQ9YIiEujjK35qWywCvw92pLHM8C3lLcX6jH9WzpKHJrXioLvPC8KXSkNq+zcLgUer2/FsdXbs1LZYEXrhnm3Pb26yzgxDP/ou7b4ihza14qC7yyi127yXEWcPa0YlpXXhxlbs1LZYFX4a1gZvPmdRYwnq611bQ6ytyal8oCr8pdz1q2B7SLAvqksZTXquPCUebWvFQWeDXufLZYt/M4s4A7Z3Ol47ln0lHm1rxUlsOr977fdXzmWKy3N+4/xEVR5va0dhRg4eYCfrPsjddZwHw4onicUp2IIrVmpZowVuOOuvepbGdwZgHnnBv/plwXRJkb89JZ4MW9AU+0z+ydBcywTFT8eFq2dJS5NS+VBV7cUfe0d+zPHMuVUElU86Ioc2taGgqwuJfuuUOwtfOzgDtmueeQj4vZU1HG1rRUFXAlLMUY9lm9s4CZgyG2cFwNEowyt+alssCLe5p+3NG5ep0FLJDA+95en8bpKHNrXioLvLiTzi132s7izALeR4l/Oq2OMrfmpbLAizvpPse0N/ZnAfdRUXu7f1Y4ytyal8oCL+6m++L3eb2zgBupem/teC6hdBS5NS+V5fDCoz7T3n6dBVxg7KHE42q2cJS5PS+FBV7cVfc1xW0gNAtj1Q1f2nHX3tzSJT+9Pv9d4N1+hufvn8QqvEf0fPf8/Mtv/usvX3/zh3dfPL/+8XExbfxnjCBmHHp9CTh+PP74iJ5cPITl94p4/Yltu+om5reKn/+b2/38d4R98rxP3nWq5aDevkhv6uPlxIv5fHf5psff+onHI/IRwY831WUvCv+7XnbgSC5/pvjZS84vzJ/HCzO/4fIL/r63dHiwA+G1fX7L/9VeXP5xPC0Pk0IuL8WMV17xtvGO8vM/zq2PL9+90scPD2LRzzyXYsGDmnhrviSfn9Rw4btUn8bOfP3h8cX75+f/TE/+be///PD8Y+///fGvz9/43z7/7fn+949/ev/46pOqpECYZZSx3rV848/4I6gkSsfW2nii409WKelXY6mdP3SBytoenPHHYCl4djZv7Vjl5SezUGi/lkuOfCyqhbd7cRHxR3DJobiEraUe/x6WHn41ls5v8DBuTL2wzPhjsIx1WrA1H7nv9JNdQi6/los8cPCLWlrjP38eOWb0s9tbIpfFjoTgInc1S438oTpMIMF/BIZ/7DGBfkO/fb7/9jH6ohnXyEo4yNAYc4F7EZn/U7gh50pygXyPcyM4Ts+tJJxN7JUNe3ptJYwCk/+A3+8r94We/u2feB6W54EpNsibrn38fV9/wAv42ZfffPunf/nrH//0/Y+fffjL93/98fnlfz6/+hVfuVxcpSoP+Wfy81+3cnndXguCef4ze/kkr1s38/LwSBRvPOrz9ZnRx3qBWkBjU1LMOaRP8gIVKy8QKGtNnUdDby+QiD7SCxQJJ+hzblRSDp/kBYpmXqBUeEwZW8/zBZrRz36B+FgkXyBcQUkpU6/xFx6ZcCYlNt8yvyr3L49/e3k+omt8DICff+bgcf/9f9+ZA/47Xe4BNJ/1gqt5PGTh/ToGdNxxDFh5RFa+kxW8vrVQHc9xE9u65Oe5A3N2uhDsMBcmol3a7M4KDtYUuMEJV9NLbtdOF4Id5sWkgmsXq91ZGYtnxJR6W01FbtdOF4Id5sigrdvtzspY1ZO5xjVoaSpzu3a6EOwwY6b4pLR3Z2UsJutDo7CYytyunS502HWsa1A0u1cljLfaGFBeTWVu2U4Tgh1m09SEax2r3VkZM3b5Y7qSitiunO7DctW72hvtB4qzgDatji744ilzs3K6D+CwfmXDs503ubOCK/8xFD9uQhCiMrYrpwOBjkcjHY3+TndWMCsuJSo9L6Qyt2unC8FunLsNMex2ZwW3dRBh1b0rqYjtyuk+kMMsHB6e7nBvBZzBDRWrrF49L7ldOdUHcJiOQyXsA7FZCb06z72QYzBxbuoS25XTgUCHmTn8FlIOr2cFfbfKm05tIZW5XTtdCHbj3JIPSkt3VkIvmO/UqS2mMrdrpwvBDjN2+Ki5j8NmBWfmiQetYTWVuV07Xeiw6x5dDM3uVQmjQ8L/CYupzC3baUKww4yoTNpw4qzgpsCSzqOsMBW5XTtdiO0aDw/452g/7zQruPblK84wLaYyN2t3IwQ7/tj5kv0+GJsVrPxR0BVui6nM7drpQrDjAYKvwe/t3aygH4d43NMlTWVu104Xgl3kjm7tfh9UzEoMWPqvHis3iG1dcrt2uhDsEvd0m/Kue+W45au3nvuVU8Z21TQbmPF++c7b2dXOSvSFP5Op5ig1L6ldN90HcgULeHMrv8udFYwaeFvhdYQ4t3XJ7drpQrDjQTz50veR2KzgVvzeYz6uiAlTmdu104UOu04U2z4Sm5Xo+UjAQ65jFDu3dckt22lCsMOjJgMfI3e7s8JjfJcK+URX00tu104Xgh2W8wn4xs3urIyZdKFQXEhFbFdO92G5zt9MMbV9HDYrOCLknEOti6jMzdrdCMGOXKZEdW/tZmU8hbF2HnYtpjK3a6cLwQ6zJlOre494VgJzJeKDaVhMZW7XTheCXXSRcq77OGxWQsf+UkhhNRW5XTtdCHaYtllwC/9md1bGeU0e88ewmMrcrp0uBLvsiEov+3hsVnDZFc+MO+Z3SlOR27XThWDHgyqqpewjslkJHauqJtxiezWVuV07Xeiw69R4RzS7V4U/oq4XaildTS+5ZTtNCHaVO7pdOd15FiJ23odU0yIqc7tyqg/gGndze1UmZM8KDqadSqV0Eb3EduV0INB1V4LHXm10ZwXLCdSaw6utm6Qyt2unC314JM+/NhDl7eyTqOABApU/pS0tpjK3ancnBDvC5Zm2z8cWFVzvjzzmz2kxlbldO10IdjgLwn/9dqAQlejx9IWaPV1NL7ldO10IdtEFTMTZzj6JymjXuA/X6Wp6ye3a6UKww631PE7YxmOigs9m5mFE66upyO3a6UKwy477Z2WflC0qoeN5dTwEa4upzO3a6UKHXef9iUp7d1Ywc7373qkvpjK3bKcJwa64FoqP23hMVHBaHUfW13FWmIrcrp0uBLvqaij7qbtZwNE08q+idhW95HblVB/ANe7m1hSUxu6soAPMQ4eUrqKX2K6cDgS6zrvfKGzjMVHBRBPP/6lpIZW5XTtdiO2IfzLwj+3jsVnBleuYWh73X4ttXXKzdjdCsCM0WMpQ9iyg9xvxfMUrqIztuqk6YMMqwz7QduZJVHBKmEHGU08Fp0jtuuk8gONhaDyec7bCnZUw7hBrPdICKnO7droQ7JLzEU9E2u3OCpb6zDnmQoupzO3a6UKHXY/8PtrPO80KxgwtEh3jf2kqcst2mhDsMu8+Hyj3cdisYHZT7+24C1aaytyunS4Eu4KHTFWvtHdnJYyJnVwNi6nM7drpQrCrvCsp+f2806xgvj8V7szVxVTmdu10Idhxcx+z3+dli8q47tp96OFqesnt2ulCsOsuxVz3mdmignNzIZbq+9X0ktu104WwNLLHetxpv6ldVNAFbo3fX301FblZuxuhY1npoN0iOws4v9RqPE6yzw3J2K6bqgO2wJ3c2vZJ2aISPRYbrjHVxVPEduV0INBF7uO23PaWblbGo8qptpyvpJfcrp0udNh13kbbR2OzgoeqxV6Sj4upzC3baUKww4Cq931Wtqjg0ZCtHuc6peiZ2nXTdeCWXU2+VKWpOysYNdQQjoGY4BSxXTndB3IFt/LHuo/DZgV9t5Z9qH0RlbldO10IdhWP7fD7fGxRwbzrXloOC6mI7crpPpBr/JOhFqWdOys4M8ctWkhpEZW5XTtdCHZ8iEwRyyBudmcFI4Z8PG75aipzu3a6ENtF3gyWMNlHYbMSecCPO2IXUhmblbvxgRw5SqmVva2bFR4wuI6VytNV9JLbtdOFYMc7yvuU93NOs4Keb+ph3Hx9IZ2xXTnd55DrqYR9NraooANXuAN3zOwUojK3bKcJwS7yF0W7hD0reEAs7uk8LmJLU5HbtdOFYJe4j1v3G55mIfSO28OOe6+lqMztyqk+gOPPHL9t0j4Qm5VxS1j0eSw1KUVFbFdOBwJd4R4uVljb6c4K7h7uJdVjECtJRW7XTheCXeWf7DUpbd1ZwSwJH0JMtJjK3K6dLgS75mL2Ke5jsVkZI/3kw3FVR5jK3K6dLgQ7buwzkTITe1bQ+2014nrE1VTmdu10IbZLePoRcfO12c0Kur+90LHy2sVU5GbtboRgR85n3qF9NDYrY2ZTCHk8PFKaytyunS502PUcgzIZe1YwO4cyftViKnPLdpoQ7PiLzBvZ27tZCQ2Pyn5d2hGkIrYrp/tALuJp5CXsZ55mBSuw8U8XWkVlbtdOF4JdciXn/UEKs4BZTbnU+GrrpqjM7cqpPoDLLufiSWnqzsrohmB5nXYVlbFdOR0IdMUlfufQfgpgVnCPROK+rw9X0ktu104Xgl3Fc3mTMh97VvDJrLitqV9NL7ldO10IdvjblaXFZyF0HBi4F0erqMjtyqk+gOvcM2vN72eeZgWnNWvq5UV3isrYrpwOxHTZc/+2Z2Uu9qxg1FBj77SRitys3Y3QYcffGJS52LMSA/H+tnYVFallN00HbuRa8a3vZ51mBXOaqPU41koU27rkdu10IdhhOTrCM+83u7OCe19zaa+1E4SpzO3a6UKw413BVcF9HDYrYw2s1vNYuV6aytyunS4Eu8RvncBt/W53Vsb9JVjOOa+mIrdrpwvBLrvEe9H2odisYH2J0uKLbpKK2K6c7gO54ngfojIXe1Yw4uLWrR7Xdua2LrldO10IdtUFPP5Fae3OyljN2fO+t6vpJbdrpwvBruFGuap8YN8KWMOJ95ooL6Iytyun+gCu836WpEzGnpVIfFTlY0LJF9FLbFdOBzroeqlUlbburIwpYb6V3K6kl9yynSbEdoWb+lJb3c86zQqOBx1Dh8X0kpu1uxGCHXEft2VlPvashMq7XxuNlWHFti65XTtdCHbcOyvjYQib3VnBsZRHXrnF1VTkdu10IdhF7uT2XvYzT7MSxuOdU6Wwmorcrp0uBLvk+AeLMiN7VvD+isX34/YTaSpyu3a6EOyyi5WiMiN7VtCu9R7ycWFMmorcrp0uBLvCzT3v3372aVZwLQLPxh5Pm5SmMrdrpwvBrjqqAY/22+zOyniGna+vGxWlqcjt2ulCsGvO15j2rvFZCHioWCNPq6jM7cqpPgcczoAkpbE7K4FzfpO9hrJTVMaW5TQg0HXXamram+6sjNE++XrcfyJIZW7XThdiu+rxMISszMmeFcyn431PxxoKwlTmZu1uhGBHrtQS9m7xWcB1Vz4w1OMmCiEqc7tyqg/ggsu19LgPxmZlHEgpkE9XURnbldOBQBe5h1uLMh97VvC5bHxkPVYokqQit2unC8EucQ+3RWWK4qyEMnYw1dVU5nbtdCHY4fQRVlXb7c4Krh624EtLi6nM7drpQrAr3MftVZmPPSv4bGZP7ZilKE1FbtdOF4Jddb75pB1kzwrub/KxvkYU0lTkdu10ocOOB1R8sNTsXhVM+M8tthIWU5lbttOEYNdca9SUSdmzgrFDqHTc9iRJZ2xXTveBXHe1hazMyZ4V3EVcqL3WeRaiMrdrpwuxXfOYQ8JfbXazEo7B62uOpzCVuVm7GyHYjQZLmfZ0FtCm+UzhaOuEqMztyqk+gAuOh/NFmZM9K2Oo70s9LmdPURnbldOBQBddbBmTvja6szLG+pm7JLSQytyunS4Eu+T4SOn9PhqblVDwmA4f/Woqc7t2uhDssqNWijIve1bGXf/k+3GLpzCVuV07XQh2hfu4NXaluTsrYy2sWnxMi6nM7drpQoddb833fTQ2KwEXw0qjkhZTmVu204RgV7mX26rWQTkrWHDSt5hjXkxlbtdOF4Jd415uT01p784K7jfxNb5myEpTkdu104Vg18c3t/3s06zgcVjZ+9czi4SpzO3a6UJs1/nXdt+UmdmzEnDrSad+3P8kTGVu1u5GCHbk+L+57e3drGD8EBtGEIupzO3a6UKwC443EZRp7bOCvkgJ4bWOpzCVuV07XQh2kTcTujI7e1Yw4k/HBcWrqczt2ulCsEuOetQeHjsrr3uJyzEbQJqK3K6dLgS77LirplzTPgvoiURckKiLqMztyqk+B1zv2StTs2clZCYqXK5XURlbltOAQFdc67kqM7NnBSsn1pSPC4tCVMR25XQfyFXunJVU9rNPs4JbOlMlOm5/EqIyt2unC8Gu8TdXyvtobFZgFGvLx1TFq+mZ27XThWDHA6rO+7+PxmZlGFEux9NjFtO33K6dLvThkbFgU285b62dqKAHR92XQKupyK3a3QnBjgdUuB9iG42JCnpwVLg7V1dTkdu104VgF3gzve9Ts0UFc7Br9KmvpjK3a6cLwY7HBzyOT1t7Jyo4M1x9L7upyO3a6UKw4xGCp5i20ZioRI8nY4fY09X0ktu104Vgx0MEH/bFsmYBsxITHxvGcyfFli65XTnV54DrPtSoNHZnBWOH0noYN00IURlbltOAQMcjBM9voe3Mk6iEimH/OLm+kIrcrp0uBDseIfhE+9xsUYl8WE086kqL6SW3a6cLwY5HCD7tFylmAeuJ+d7o6ilSu2qqDdBwZYuPkNtZJ1HBQ2KxWEygq6aM7crpQExH/JM8ktrnZIvKOBbwLxqzTCSpzM3a3QjBjkcGvvR9TraoYIZOJlzDWUxlbtdOF4Idjwx8LfsjxkUl5IZnYtfxTCxpKnO7droQ7CL3b1ukfRw2K6EkFxlqXLGWpjK3a6cLwS5xB5e/ez/AzgrWdaLjfk4pOlO7brrO4dZ9r/sy2aISKtb+yxh4LZ4it2ynCcFuTNJMfh+GzQqOpY2IxjN2L6Yit2unC8GuOGagfVa2qIw+XMnx6BMLU5nbtdOFYMe7T8Tb2e3Oyjgf3NJY6lSSitiunO4Ducads6C85145WrQSWg990ZS5XTdNB2qdh+6R9tnYohJy4TeW7zFcPGVq1033YbngXaTIx8hNblZC4wNqp+jbIipzs3Y3QrAjFyjlvp9tmhVcMeRi6XU1FbldO10IdsER5bDPxRYVPNW5tppiXE1FbtdOF4JddJ5yb3trNyuYDZYT73VfTGVu104XOuw6FSwiodi9KrgvjHpLaTWVuWU7TQh2ifu3Ne5zsUUFRnjAbt1J32K7croP5LjB4l5GVVq7s4IZ19R7zrSIytyunS4Eu4Kb+WvdzznNCq57+dbTeNLJxVTkdu10IdhxY0+8+/s4bFbG0dS/jhNTdKZ23XQduDU81I+UA+xbAX3fXktJfdUUuV051Qdw3cXgW9lPOM0KRvjet9D7Iipiu3I6ENNF3gxvoewjsVkJvIPkQzrOEAtSmZu1uxGCHTkKIezzsEUF88AIT0+oi6nM7drpQrDjHQ1hf8DYLODcEn42p0VU5nblVJ8DrodY8n6UmJVxeimWSvEqKmPLchoQ6KJrIUVlDvas8M7h/oh8nFuXpCK3a6cLwS65GrLPSlt3VsYonwf8xzUdYSpzu3a6EOyyKyHXtI/DZgXXvbznAURcTUVu104Xgl1xPD5IyhzsWcGiddQjpbyYytyunS4Eu8qd3IpLqZvdWcE1Q+79pmPOiTCVuV07XQh2jXu5/H37WadZwVOdU4h9rL0mTWVu104Xgh3OurWsTMOeFYz0o+/h1bsTpiK3a6cLsV3y3M9Fn2Ozm5WxSCfvWkmrqcjN2t0IwY64o9t73M87zQrOzeUWQgqLqczt2ulCh12PvKsa3VHAmjCBDwnjTn8pKnPLcooP4PiLSFGZiz0r43713P246VqIytiunA4EuuhqDF6ZdzIrAV3hxr25tJDK3K6dLgS7xLvCb6B9PDYrGLO2kvLrbTdNZW7XTheCXXY8Ok3KXOxZwVmmWOOrsZukIrYrp/tArrjEBeUSxaxg3TX+WRrrrklRmdu104VgVx1DNNpHY7OC2eq9cT8uL6Yyt2unC8GO//aI513vdmcl4OEJRKWHxVTmdu10Idh1nPQNXmntzkoo0fXe6GU3TWVu104XYrvsnY/87ftobFZCxrKJrfq+mMrcrN2N0GHXI9Yb1uxelZDJJV/TWLPuYipyy3aaEOyI+7kt+r29mxW8v3KNqabVVOR27XQh2GHRksat1m53VsadnL6W2hdTmdu104Vgh13pRZmXPSs4y9Qw7WQ1lbldO10IdsnlxFtR2ruzEhJ/TBODhcVU5nbtdCHYZf5JUrooZ2Gcn8tY3nQRlbldOdUHcMXxCKEqk7JnBQtN+l4zLaIytiunA4Gu8p6F1JS27qyMJ4mVGvMiKmK7croP5JqjFKnt555mJYTs+Jjae15EZW7XTheCXXc+xabMyZ4VPDEBlxKPOWPCVOZ27XShw66nlJU52bOCJ7G3nsNmKnPLdpoQ2xXPX2Q+Wm52s4IzJZ1et7MLUhGblbvxgRxhreFe97HYrGCGE3Erd4zFpKjI7drpQrDj4VQqRZmYPSvo+8aEGZ2Lqczt2ulCsIvcx62x7K3drGDefygUQ1tMZW7XTheCXeKfbNrl7FnBuc0WfD1mjQlTmdu104Vgxx201JSD7FkI3MD1TsfyWBJ0xnbdVB2wFd6xnrLS1J0V3I9YYqrHDWPCU8R25XQg0OEg6fk4udOdFcxdb6UeT+uUpDK3a6cLwa45n31T5mXPCq4e1lReKzwJU5nbtdOFDrue1bfdWwHPJY4tt+NGCikqcstyig/gumOFkPazTrOCo0Hklu24J1uIitiunA7EdNW7mkNX5mTPCo4HPfEBoa+kIjdrdyMEO3Il817sbd2s8AAfT0xsdSEVsV053QdyweWcYtzPOs0KxgzFx9cdKEJU5nbtdCHYRce75JUZ2bOCO2ETJtctpCK2K6f7QC65mLm+j8JmBddv8HSdo60TojK3a6cLwY73jMdWcT/rNCtYobPxxkpfTUVu104Xgl3hHm6lsA/FZgWj/NoTpdVU5nbtdCHYVe7j1qbMyZ6VkJIjaulYhUKYytyunS502PXMfTWlvTsr4x7sxNtMi6nMLdtpQrBr3MvtynT2szBW1aG3ddmEqMztyqk+gOu4h6QrU7JnJeTMH8xAZRGVsV05HYjpGn9z8YX2tm5W8LkMNVZKK6nIzdrdCMEOMyAo0v6BnRUes7peeYu0mMrcrp0uBLvgUgleme40Kxjr89734x5PYSpzu3a6EOyiiyXgccOb3VnB8xIapXjMFROmMrdrpwvBDhcFY1IeLDYroeJ5iSnkupjK3K6dLgS7jN0nZUb2rIRIzvMbbTwfVprK3K6dLgS7wjuaalfau7MSUp732wlSEduV030OuV5yUpbJmhXeJv9478fJJyEqc8t2mhDsqmulkDIfe1ZCIYflnVJbTGVu104Xgl1ztRRlmeyzELA0VuShflhEZW5XTvUBXOdObuX6LndWApHrjYXiVVTGduV0IKbr/GtLC8p87FnBZRyqvh/r2gtSmZu1uxGCHXEft3VlRvas4Jp15q5IXUhFbFdO94Fc4Ia+KxMUzwJGDb11OtYqFp4ytyun+gAuOh4YxLoPxGYF7y0e5b8m7UxRGduV04FAl7ixIq/MxZ6VwP8MnvIxyU6Qyvy0+/x34ck7Hp6/fxK78D7R893z8y+/+a+/fP3NH9598fz6x8f0PW0b/zUijmycvS8B45THHx+Fv61evvlI1u+72fL64+v3is3cbfnz3xH2y/N+ecfvmHKAb1+kN/vxouIlfb67fNPjb/3E45Eq/tLqYnr9OXgsbzzeajP+Tsb89nW559Fujh+XAb8wfx4vzLLhn/guHi8m4cV8fsv//b+P/dXEzeW1H3PM5589U/nXMKR4k3jH49D/eNv08eW7V/r4gX+Vf37muYQLEmNboZb+7C41PLno6w+PL94/P/9nehI93//54fnb3//741+fv/G/ff7b8/3vH//0/vHV/999J+KNhoiRjdx5Ef/yvSfyr43FWn7C3pf06XZ/XBpvOV73/kw/ws5jUi+25VPN4SfsPYX2yXYfIwycC839sv8i/uUAYTy6hjfWeAyYfgpAD58OgDtvIddxI6IEmPFHABhXOnhj3MGOP+X9H3L5ZABjw8XjkMjvz2trPeOf/eYPLjxz5uMF8YCi4Ug7dhs7y4euxpvmXZ4Gv6HfPt9/+yhYKCXHTLmEQwUtIheID2n8n1LA8OSjN282zo1gX7CVPr6Z8X7Ar/F8qAfu65+UsNB+TPg53kLt48/4+gNeis++/ObbP/3LX//4p+9//OzDX77/64/PL//z+dUneQ16cbl2X8v1NZjxL3oNuL8YCNeJyKdP8xqU/4GvAXG711ot/BdeXgSR/6JXgZt2Hh+2yLueuWv7KV6G+D/xZRhP6Sr89ywvw8x/9svQ8DLwIblhcZxYzob4574MlQVj842HFfX+VfBvr8Iv4Yu8o49fMh67fP/jF43HsExuD+X5GVGP/NdX33wZQw8sAlswKeVS+U5UGjpE44Hi34ktyfQci/1Ph7rjYChyPvoYdqe3QkyOB+zcdbzgydQOk44xlHokT01jelVSxC+K4yL+uSEZWmJSNdhprOVaMbdkdTor2PvSyoI3QztMNxjMNJZtTaR86s5KDuOJh2Mu0tySTO1A3XAwFBZpjVgAcXV6K2TeuVCPx6MLPJHaYdIxWGmsx9owO2RlOit44/BujqmoE0+EdphuNNhprL2aMRNkdToruXJ3NuZxqVPoidQO1A0HQ2Gh1Ry88rk7Kzyabr3RGJQLPpHagbrhYKixqmrDLI8V6qxwW1RyxB2mFz6R2oG64WCosYRq7mtP/TtRQV+ptThGVXNLMrUDdcPx4VGxXmoNfR+4zAr3wlMjWvhkagbqjmNA8Qi3972NmhWshpfauKw2NyRCS0wqBjONdVBL20cus8L9Jh7LlbF829ySTO1A3XAwFBY97bHtQ5dZGa0S9yvDlU+kdqBuOBiK/528b0oLdVYyOgSecrzwydQO1A0HQ40FTavyyXsr4PmT3HSHBU+kdph0DFYaa5emug9eZgVvnJbzGKdMPBHaYbrRYCesVBrQs96czgpWCe89LXgztMN0g8FMY1XSVvdTK7OCXnih4xb+uSWZ2oG64WCosQRpLvvQZVZ4kOJ520cH6tySTO1A3XAwFNYbTaEoDdRZYRJuu2Na9d5CO0w3GIOpp4R72BSmV6VwV6BXP8Zyc0sytQSlcnx4tLGSaCn7wGVWuC3yvH/pqjdDM0x3GMyERUNLzHv3aVb4nYMZ4oWueCK1A3XDwVBYIZR/cD+1Mis449vCWO1H6M3QDtMNBjONxUBr3octs4LnI8ccWrriidQO1A0HQ43FP9PezzwLeHx56eM5PnMzIrSDpFOwEZb57LwDO9JZKYQlGfLROL1tSIZ2mG402Gms6am+mc4Kv3FSbCFd9WRqB+qGg6HGAp457cOWWSmZ0zBmXMwNidAO0w0GM2G1Tgp7H/MslO5iazSuqwg6kdph0jGGUs+EiWAK06vC75vW8rjPadrNzBKSasFKYw3OEpU301kp0RGW/aernUjtQN1wfHh0rLgZozJHbFZK4ZR6Tlc+kZqBuuNgKKyvmXzYT6nMSsVVuuhju/DJ1A7UDQdDjdU06zZv8ztRKVgBLNGY6TS3JFM7UDccDDXWzkzKAW9W+L2TqZZ45ZOpHagbDobCQpmFwn5SZVZqdpVKGStoCj6R2oG64WCosSpmU2ayzgr6lL6PO3nnhkRoh+kGg5nGAphZmcg6K9ypzM3n8SxggSdSO1A3HAyF1S5boP20yqzgEWe+jMmYc0MitMN0gzGY+LOjTFY5C6XiOXDHWfC5HZlaYtIwWGmsYlmUiayzgusGseC8nMCToR2mGw126g6PhfZK63RW0FPyacy1mBsSoR2mG4wPD/L8zYX/dx+5iFLl/Q+l9iufTM1Q3YLAaixHWZSprKIEldTTuPS7EL7FhrBuRIA11p+MfW+oRAlT6QoeabQYitgQ1o0IsLDgZPR9H8SIUg2OfD2WaJeGIjaEdSMCrLHCZFV6CqLEb6FI/MmjxVDEhrBuRIA11pRMTWuzzhLO/Wb+R70aytgQ1o0IsLCKZKa2n3QRpUbOJ8rjmfViYzI2hHUjcmD1wj+4D2tEqTYsfkVjhrTYmIxNYakiwBorReamtVlnqSbeZDhWQpSGIjaEdSMCLKwNWYMyLViU+LBXYuB302IoYkNYNyLAGqtBdmXSqyg177iHEHu4GsrYENaNCJbKGus/lqq0WbNUcEIv4baFi6GM7WDdiQALaz5ql0hnhT9tvVIfT74Rm5KxISrdA1LBxeq9Mv9VlMbiVHQ8O1MIitQQ1Q0IrMaajlWZBCtKmFnWCh2997kxGRvCuhEB1lgYkwd6CtZZwtOUWm6FroYyNoR1IwKs7HwNWAlvxzpL/Hnjt1BMYTEUsSGsG5EDq9fQlCmxosQ9z0qN/70YitgUlioCrOJajTlrbdZZwrN+g29pMZSxIawbkWM50loTd8EVrLOEW/lyTy1eDWVsCOtGBFiNv0hdmSUrSnhMY/c5tsVQxIawbkSA1V2uuSStzTpLGNOEWny+GsrYENaNCGMF/tHKRzRlqDNLOOylHF4N/LkxGdvBuhMBFnEXsyqr3sxKxQPIePyXF0ERG6LSPSAVuH9Za1QarFnC+I87V2kRFKkhqhsQWEXuXrYUlVMzs1S5fxDzmK0nBWdqiOrGA1RjrURSJtCKErfhPvjXqHBuTMaGsG5EDqxeu3bi/axU6KR49K/mpmRsikrzgBSm6/sclBMzs8QDZQr9WLlMCIrUENUNCKyKq42bIO0geJYqFnJJKaaFUMSGsG5EgFVdadSVybSiVPE0tRrSQjhTQ1Q3HqBqLreA1YJ3qrOEJQFyyEevXQiK2BDWjQiw8MiBGEkZ4sxSLS6E8DoMTsKZGqK68WCq6F1sySvTakWJUTL/quP680XwjO1g3YkAi1xoqSrLxIoS5qjhEctxMRSxIawbEWAFRy0nr4xxZqlF5ykTLYYyNoR1IwIs3mFck1EGObNUs+Pf9bpOODcmY0NYNyIHVm+FN6RivUpYlyr7Pp68dTU8Y1NYqgiwEp7kob2vXoXa+f/zj65+IjYEpWqAKXPPkrtLyhhnliouC5Yy7u6efiI0BHXjAarCHcvWuvaeOkstuBz4jdSugjI2hHUjAqzKPcsxAN6xzhJ30HNvmMhwMZSxIawbEWA1l7oP2jTbWarJRfLtONcnDEVsCOtGBFjdxe5701qss3TcdFrjQjhTQ1Q3HkyVsB0qTTkpM0u4tYv/f+6LoIjtYN2JAIsc9RC1SbazxE1TwjC5L4YiNoR1IwKs4HyPuG1yxzpLrTrMqg39aihjQ1g3IgdW56NZVU7LzBIP/TBNbay2Kw1FbApLFQFWdK3zDyuDnFlqmX3e7qIQhiI2hHUjAqzk6nhk6W71VmmYV0s950VQxIaodA9IZf7m3IpyVmaWWhqjwLIIitQQ1Q0IrAr3v0suykBnlvgNFCi8Jn8IQhEbwroRAVbl3mUN2iTbWWpjsb3XhS9hKGJDWDciwGrcvazKQ2pmpZPzFEqkq6CMDVHpHpDq2EzJWtN+lvh41yhTqBdBmRqiugFhq+y5a9mjNsF2lhoWBs3HM64loYjtYN2JAIv74d77rLRXs9S4iSL+R1oMRWwI60YEWNwP974mZZgzS/wW4q2FsBqK2BDWjciBxd+ctAm2s4Qu+niC5JVwpqaoVA9QcT/cB64pVGep4UEskd9Ri6CIDWHdiACLO+I+tKScmJmlhttSSzwu1QtDERvCuhEBFlaRi1mbYTtLDc8vD/m4mVAYitgQ1o0IsDCRI4WotVhniVV8S32sX3g1PGNDWDciwOK+uE+cKVhnCQ+J6L0eVwqFoYgNYd2IAIs74z4Xzeqt0qKjUl8TsYSgiA1R6R6Q4q64L1GbZDtL/GGLcTzz5yIoUkNUNyBsVTwWa/JBOTEzS9xJL7hpKSyEIraDdScCLOLeZeUOuIJ1lvjTlloMY8E+aShiQ1g3IgdW58OZNs12lqDS+/GclsXwLTaFpYoAK3D3spN22n2WsJp9peO2ekE4U0NUNx6gwuWrjtkJO9VZ4qaJUqipLoIiNoR1IwKs5DBXT5tmO0t4gkvnFj0thiI2hHUjAqzsOA9ea7HOEh5QUlKubTEUsSGsGxFgFZc488pAZ5ZYpfKYubfN8IwNYd2IAIt3i0LRptnOUkPHyvfxMFNpKGJDWDciwOI/nmL0Wpt1lvioF3MLx/05wlDEhrBuRIDVHVHsXTk5M0t49FTEIjyLoYgNYd2IMFb1eBJJ0abbzhLmElGiY7qtMBSxHaw7kQOrU45dabNmCX2EkspxH4UwFLEpLFUEWOQaD/aUnsNZ6axTUzuOhXNTMjZEpXtAKowJs9pc21nqgY95rR4Tk6egSA1R3YDAir8gfnNo7dVZwtk9H8rrfXVuTMaGsG5EgJW4e9moKSdnZgnn95LvpS2GIjaEdSMCrMzdy9a0ybazhCdTEeVSr4YyNoR1IwKswt3LnrXJtrPE/QPfqR33qs6NydgQ1o0IsKoLwYeqnJ6ZpY4mvSXfr4YyNoR1IwKs5jAgrspQZ5bwvOY+fv5iKGNDWDciwOrOBx4YK0OdWeIxjc+plLQYitgQ1o3IgdVDiEVrs84SN04RdxPSYihiU1iqCGPhWnKIXru2Okud2/RQak5XQxnbwboTARa5GqJ2MDwr3EOI3J63ugiK2BCV7gGp4EpIKSsN1izhmFd4zFyvgiI1RHUDAis+moVMWRnqzBJOK+RUj0XE5sZkbAjrRgRYiduc3LQJt7M0HhEe8zGTWxiK2BDWjQiw+N0RivrGeqvwpy0nLraroIwNUekekMKCTTUk5dTMLDUc+eKrg3VuS6aGqG5AYIXV+GrXptvOUsdbKdDRv5obk7EhrBsRYHGbE1pJWnt1lvhfuCRY69VQxoawbkQOrB44Uk7NzBLGM8WX3BZDEZvCUkWA1V2L3mvTbWepYdHtfqzcIAhnaojqxoOpunc1+qpNtp0l/nnuK/QSF0ER28G6EwEW8b5QisqJmVnCnQC8s8eJGWEoYkNYNyLACi7HQEEZ58wSv4VKKv2YNyMMRWwI60YEWNGlGJo24XaWcFKB+1MxXw1lbAjrRgRYx3TsoLVZZ4lVeo/9uAHsYnjGhrBuRICVseaxNjn5rOApvZ58OZ5vPLd1yQ1h6SKwKo5i6tp821niD17NVNPVUKaGqG5AYFWdj7mQ1mKdJaiE2o75WFfCt9gQ1o3IgdV5P0j7FJ4lnFiorR4z/YShiE1hqSLAatzBrF6bNjNLHc9eb8c5P0E4U0NUNx6g6typrNVrDdZZwhPrK7+D8kJ4yQ1x3Zh8eATP343VovfTM6IEltZrOm4tvDKeuRmuWxNwccvDXSVlyq0oMVBsuGH8qihjQ1g3IsDCEju99r3VEiX+8IXUaw2LoYgNYd2IACtit5KyPo8okcdvSyEtiJfcENeNCbgSvz8QKlxnid9EvSQ/piKLjcnYENaNCLCyo0TKErezgneQD72ORenEti65ISxdBFbFeVzW2s/RiBK/g2oKx60mc1syNUR1A3JYdZxE2Ac8ooQhMx65R1dCGZvCUkWAVV3j5lqZditK/BbKpYzZDGJbIjVEdeMBKnyQkjJ/bVbwWavluI569TtjQ1S6B6RwVjhzb1yhOkvc8Uwx9pSvgiI1RHUDwlbEvzfxtypDnVlildxyHDNrr4RnbAfrTgRYxD9a0P/esc4SzsiknsYCydJQxIawbkSAFXi3aipKB2uWenG1hzRuqZeGIjaEdSMCrMh9y0bKhFtRGgMabtPjgnjJDXHdmIArce+ytaK1WmcJZ7CKj4GuijI2hHUjAqyMlicX5fTMLPEnrlM6ViqXhiI2hHUjcmD17IMy61aU8BbKmT9wi6GITWGpIsAqvMOe913BOkusEvl3HZ2Hi+EZG8K6EQFWxZW/kpWTM7OEk8chdB45XxEvuSGuGxNwNVcyfmrXequMM+2+N8XwLTZEpXtAirvh3AtPWpN1lsbHrftGF0GZGqK6AWGrwD/KY5ekvK1maZze652O0+9za5fcDtedCbgI90YkZeqtKIGF4ngu04L4Sg1R3XiAKvAfn0mZeCtKaJo87/2YVnQhFLkhrhsTcEVHObeonKOZJfJYhju28TyKC6PIDXHdmIArOZ/54KYMeGYJLKXGXMLOeOaGuG5MDq6ea1Am4IoSH/x8K+Xoas2NydgUlioCrMy9zNqD1nKdpd55lJPruOtEGorYENaNCLAKdzNbCcqQZ5ZwGYd3Po6Ja2Jrl9wQ140JuPBovR6VKbiihKNfTq0d42nJKHJDXDcm4MI1Le+D1m6dpXH0i57ixihyQ1w3JuDqDifXSRn5zBI6oryBNu5ZvTCK3BDXjQlzRe8iVqFTBj+z9Bo9+95WRpHb4bozARe5UAKR0nbNEk6Q8odu3GAoNiZjQ1g3IsAKjkpQnlU7Kzj68U/7RFfCS24ISxeBVXS+xKzMxhUlqFRP9aUlEc/YENYNyaHVS8KirJrWq0SYyFZzGc/VuSiK3BSXagKu5FrB48UVrrNEnjtZtR63N10YRW6I68YEXNnVkrM2HXeWiAKEeqGFUeaGuG5MwFVcKSVoE3JnCR3RzIe/Y3abYJS5Ia4bE3BVzFHrXTltM0uYc8S90NSuijI2hHUjAqzGfc1autaHOEto0FtMJbYr4iU3xHVjAq7Onc0WtTm5s8S9KtdDrZ5WRpEb4roxYa7E2+Fvb0rLNUvoXfXu0zE5XjKK3A7XnQm4iDucHati7VxnCd0F7pbWHldGkRviujEBV3C+qpdgzwpQeISYYt8Rz9wQli5yWPVKVHWsVwkqqYac44Z4xqawVBJo8XGtUqvK8GeWyFdHWWidiiI3xHVjAq7EA5iQtSm6szTOZcXwOuEsGUVuiOvGBFzjyqDyrMNZGV2F7I/n7VwQRW4ISxeBVXG5xl40rLOEs8o1UT46XAJRxIawbkigVR1mJ2tTdGeJiFzsPR7TkoSizA1x3ZiAq+E8VSxau3WWyDc8wyK0vjDK3BDXjQm4ugu1+KyctpklouR8r/6YDyEYZW6I68aEubJ3VEvVJurOEj50rfVjMeoLo8jtcN2ZgIu4u1n5L1e4zhJO/oXgaazzcGEUuSGuG5ODi0cylJUTN7M02qjcwnEOVTDK3BSXagKuwP3N1rTB4iyBJaaWFMW32BDWjQiwIvc3e9Ym7M4SeZwrpXqcFJSIIjfEdWMCruRK8zykUbjOEpEfC8WHuDDK3BDXjQm4ssvNa6fnzwqa88b/rKvhjA1R6R6QKvgkFW2+7iwR8b7nkl6HxEkoYkNYNyTQ4t1qY6nNXessEWGNDB7ubIoiN8R1YwIu/uNb9FEZ/MwSeu0xcSWvjCI3xHVjAq6xnnLVJuzOEhGWPo95LCR8YRS5Ia4bE+Yq3vmWknaSa5YwyAmh5dBWRpHb4bozObh6yxSUg+IsoY3qnWJdFWdsCksVARaGxjx6UYY+szRUco7HaZsF8S03xHVjAq7gaiuZlJZrltC1opR9DyujyA1x3ZiAi79oNZBy2maW8C4KY/9WRpEb4roxAVfiDmft2oTdWSLCk1R6iXllFLkhrhsTcGWHRR5Ia7vOElF0sZRGeWUUuSGuGxNwFdwrHb1y2maWuOL4S//6MApGkRviujEBV+XteK9N250ljAwThTge4HBhFLkhrhsTcDVH3Vdt2u4s4V1UfTruZJGKMzaEdSMCrI7HImsd1FcBx74UfHh1IQSgyA1RqR4HVO94hq1K9Srhw1ZyqsfQZxKK1BSVKsJY1bvW8diwHWuW8FkbKxyshjO2g3UnAixytcfUlfZqlvAeyiEecyGk4YwNYd2IACvwd+trDc/SuCqdYsq0IorcENeNCbjwbHJ0nBSus4SmPAdsc2UUuSGuGxNwJZd6zk1rt84SPnM9jss8C6PIDXHdmIAru9j1WxVnCe05Jf+6Wi0ZRW6I68YEXDws7uMRazvXWcJJZeq9lY1R5Ia4bkzAVXmHa1EecC5KYOklHetLXRXfYkNYNyLAatzXbLEqJ2xmCSr4bccZiCvimRviujE5uHrv2m3DZ2VcaiVfO62IIjeFpYnAqnNvk98oWrN1lo5eu3/NGxGIIjaEdUPCWo375t6nopysmSW8hwirMm+KIrfDdWcCLsI9+KRN1p0lCt5hBYiQF0aZG+K6MQEXd849ad35s4LGnHsJOa2IMjeEpYvAinvmPuSsNPKzRCExUMuLoUgNUd2AwIq75T4GbZruLFGIDsfA8ajui6HIDXHdmICLu+W8FW2a7iyhdUqBW/ewMorcENeNCbi4X84NdlJO2swShexw+bD0lVHkhrhuTMDFnQCfY9Ja+bOEJor3+u3dJRhFbojrxgRc3DX3xWsTdWeJIv91PuZWF0aZG+K6MTm4ui9Y9UjjepWoYAH+Eo/rY4JR5qa4VBNwcefc1xSV0c8sUR5P8D6W9BSKIj6xPv9d4B1/hufvn8QuvE/0fPf8/Mtv/usvX3/zh3dfPL/+8bGAfjgCqlj0JRPsH398RE+89UEsv1XEyw9sW1W3ML91/vjf2urnvyPsj+f98a5TLQf09kV6Mx8vJl7K57vLNz3+1k88HjEHLEXF/3fZh+LzZa++eyXyrxQ/KmN+Sf48XpJZXzb/E9/J4wWlB7+gz2/5v9oryj+OJ9N1fi9cXoEZL6zireId5ed/nBsfX757pY8fHsSSn3kuxYJHJPHGfEk+P8cNG/xbvv7w+OL98/N/xjnE5/s/Pzx///t/f/zr8zf+t89/e77//eOf3j+++lQQKRDmCeXYr2/wGf9yiETp2FirsdefAFHSryHB4/4YuWezftTP+CNIoL3Gxo7FV/57CQrtV6DIsTiciuz5QiHiX06RA9a74I2lHn+SRA+/hkRPWMExhXiVmPFHkOCxCI2N+YgnhP23FCGXX4FCtNppvHQh97PBn8nPbi2puDz3IAVH3LtpKVQ6KADAfwBWs2CG6fIb+u3z/bcPfmv2lmMmLAY/pNCecoF4YMz/KQUgT+6Nk+9xbgRHVmyljG9m0B/wa7C+KcBf/8Rjo3yICT/HW6h9/Blff8DL89mX33z7p3/56x//9P2Pn334y/d//fH55X8+v/pVXhdczcOy0u18XWbykV6XigeF4FkFnnvp4oWhOt60/z9emfzTXxk88qWEf8BXZjz+Nsaj8/bqEc3oI702WN+f+7mJKp5O8klem2ThteExZ0IDPPuvIvpIrw3x4c5nrEvHfYD0SVq0aKBFo9xc7Lihfr42M/pYrw2PC32kiBlvvn2S1yb8Q742/zinAPjv5F5VgO5n/KHhYVngsb2n1xiN+4zoa1xL38kSt4Wl8XHqiM+Nyfg8BWCJ7cYGbJi7EjsuN2xsZynyJ4G75KVdNWVsku3GBmyYxJIKLjtsbGcpdVf4TRXSVVPGJtlubMCGK+doIBW2s4SWnKvjLkGxMRmbZLuxARvmtRSftLbtLOXueNdaWzBnahLtRuZA671UXIxQ0F6lglssiXy/WsrYKJtqAzbMc6kJFyU2trNUI+87riZfNWVsku3Ghtl4BFx7I+V4cFZ41E2lZJ8WSxFbRLuRgRnxN7eGqzYb2lliKHR1S71YytQk2g0N1AJ3W9G4K2pnqRbe91BCvWLK2CTbjQ3YMBOGx0/r0OE7WcJxk+KxcI/YmIxNst3YgA0zYniwqqi9VSpWlzwetim2JFKTZLoLxDApBqvBK2RnqVbXa85jwXghKVKTaDc0UMPcGH7faAfQs9SKw5TSsfKR2JiMTbLd2IANc2SiD1qrdpb4Xz5FCvmqKWOTbDc2YMNcGT4uKgOqWcKsXMx0qFdNGZtku7E52LpPiZQh1Sy15DqPncZzfKSmiI2yqTZgwxSkTOrg4Cy17Epp4dX1mJoiNsl2Y8NsjTv7/IOknCyapcZDzpD765AwNUVske3OBmzc2/d8NFRGVbPEb6vawrE8i9QUsUm2GxuwcW/f1+CVtm2WWnc+lOOZsFJTxCbZbmzAFrnvWrtXBgizxD6VFMtXaJLsxgVkiTuuTXufvQqd+KDpfapXRRmbJFNdAJa5y9p5QwrZWer80yUdq0ZMSRGaJLuRAVrhDmvvXTlNNEv8lko5v95ok3KmJtFuZIDGQyLypSvDqVnCU5Nq73gQyQXzkpuEu9E54DoRVvbU4F4lAKUaX33cK+iZG4VTdQDXXKPgm3K2aJbQjIUUaPGUsUm2GxuwofMQ8J0721nqhT+PIbw+p1NTxCbZbmyYrfN3U0xNGVLNEj6OPrVC/cp5yS3C3ekAjlzGFEqlfZslAEXPo6e4g565SbgbHcAFl3gPqtLdnSX+RHInrRyj0bkxGZtku7EBW3SRcq7KsGqW8HAc3rkQy5XzkpuEu9EBHCZXFtwesMOdpdHhwFTNtoKK3CTcjQ7gMm6t6EUZZM3SeBxtGMv0LaAiNwl3owO4wv3YWooy0JolfCR5SFXGrTAXUJGbhLvROeA6Nd4TFe5VGk8iz5lCW0FFbhRO1QFc5a5s185UnhV8IJvnd9jGKXKTbLoN1Bp3ZHvVZjrP0njMVeNGLV05ZWyS7QYHbt2V4FPW2rezhEVbU4/5GNpLT5GbhLvR+fBInn8vD87zfjJJlMbzdnwNxzk4CSpyg3C3OoDjXligpkx3FiW8s/A8rLHKwwVU5CbhbnQAF1wMIStTnkUJD/7gPi5vYQUVuUm4Gx3ARRcwJ2Y/nSRKeGdRTX5cVL6Aitwk3I0O4HCTOg8E9uGWKOEuR97FHvICKnOTcDc6gMvOh1SUuc+ihG5H9Yn/3wIqc5NwNzoHHL9hYtTauLOEtqzUcFzDuoCK3CicqgM47vmH4uM+3BKl8SxO32IrK6jITcLd6ACuuhqKcipuVrBsp08hvo6pglPkJtl0G6g17srWFLQG7ixhHbyeSinxyiljk2w3OHDr3JMdKzPsbmcJq1M2inWcNrp4itwk3I0OwxH/aOCfU4Zbs4RVULEcagsrqMgtwt3pAI64J9u1YepZoUBYpyJRXDlFbpJNt4Fa4GGmD7SfThIl9HCxmtVYWV1yitgk2w0O3KKjeDy9a3M7SzgARDyzOayeIjcJd6MDuLEcSSHtc3qW8M7iz2c+RgwSVOQm4W50Drge+Q2knE6aJQBRSTW1HfTMjcKpOoDLrsXolQsNooSPZOtUa1hBRW4S7kYHcNy8x8gjdAXuLKGv1qmVcQ/zBVTkJuFudABXeV9S8srppFmi0PBQS99WzxmbZLuxAVtzOWavTJAWJbRkrZd0dH0lp8hNwt3oAK67xN0JZZK0KGFsUCjjxNsCKnKTcDc6WAjYY6HppNwXLkr4ROacw5jAdQEVuUW4O51jBeWg3ng6K8S77kNpYeMUuUk23QZqgTuytSmzpEWJ+J+x12PVHskpYpNsNzhwi9yPbbkp7dssoaeW+R1WNk+Rm4S70TngOm+kKYOtWUI71rEAfllBRW4UTtUBHPf6Y+/KRGlRQkuWqeZQVlCRm4S70QFcdpV3q2pN3Fka76ySa+grqMhNwt3oAK64kgiPIt/hzhJFbtZaT7UtoDI3CXejA7iKB1F4Zaq0KFHojg8EdSwZJUFlbhLuRgdwPNDketHauLOExaLxEJ6yeIrYJNuNDdg671ZMRTmdNEvwocif0LJznrlJuBsdhuNdD1gMRBlszdIA4s31roC+5Rbh7nQAR44S7lhT4M4Surmph7KBytwk3I0O4HgnEw/RlRNKs4Rn/qWaOpUVVOQm4W50DrieeK+UAdcsjWMn8di0r6AiNwqn6gAuupaKehV6lvgAwG2ZL+MZphJU5ibhbnQAl7gvW5W7j2aF+LMZezlu3L1witwkm24Dtcwd2RaTMtqaJYyqiPPxvA/BKWOTbDc4cCvckcXSZIrbWRp93OJzz4unzE3C3egArvKPdkxc2OHOErFVyrGWDVTkJuFudADXXMw+RWWwNUtoybi/Fl5HVAEqcpNwNzqA6/zHE2kTpWcJR85YyKcNVOQm4W50GC7h0UDELdYON0s4cdRKp7Fs+wVU5Bbh7nQANyYD5qgMt2aJkXigUMJY5PgCKnKTcDc6B1zPMWhzpWfp6ORi+fsVVORG4VQdwPGHLfNWlDZuloiPByHiRoYVVOQm4W50ABddzfxpU04pzRI+kgkTCFfPGZtku7EBW3IlZ+UZArOCd1UMbwcGiSlyk2y6DdSyy7l40pq3s4R3VeVfVdPCKWKTbDc4cOPPWS6VlLH9LKHDkSr31drqKXKTcDc6gOPdyjVpU6VniSIeykhYvmYBFblJuBsdwOGP11bcnhUMqSoumYaVU+Qm2XQbqHXuxDb+UmE7S+im8bZiXTlFbJLtBofdsnc4GaRNk54l4kFp9q3FsHqK3CLcnc4Bx98ZtGnSs0TcY+sl9pYWUJkbhVN1AEc4AYlroTvcWcI7i0rwG6jMTcLd6AAuuIq5zspAa5ZwmrKGkI+zcBJU5CbhbnQAx/tSQtAmSs8S3ll4glHOK6jITcLd6AAOk09D71obd5b+X3d3k2s5bkMBeF6rqBU4ov4oDRMECJBZkiUEGWSQUYCsPzzWfRavTG6A3Wig6/C++8ofbFmyZYswTkiU1y1oBarzkHCODuDaVWUzhjHa2iUq8xqlf27RKFCdh4RzdADXsVhAsSZL7xKAsAhxKW/QJw8J5+gAjq+MdVCsNu4p3QP5wbnOE1TlIeEcHcChQ9HYOlJ/KmjJaHL99OI2p85Dstk2UJuyjfcCbG+2p7TOAPx5SvCL84lDsjk4y212Jrbat6eE0QHL8GC95Ex7qjwonKkjcB3TFtBGveF26QYabZigP3lEOE8HcCQ92dGsidK7RFXOBdLLHfMEVXlIOEcHcFl6sjN3o4nbpXvPknI5PXccks2xAVuRfuyc3bigtEs4BTDe5fviVHlIOEcHcPWSn+zWROldug/ILPUXqMpDwjk6gGtXYSrWROldoootTmW96UGDqjwknKMDuH5lzqkZF5R2CUC5kexcb9AnDwnn6ACOL5LmvRmDrV2iypd0dz87nPLccUg2xwZs40pcqtH3fSpox/Am5N5OTJWHZLNtlpqMMbFos8X2KcEnFZ5rfs0X5xMHZTNx4DavgbG5tbs9JXQ4cut53UnVnioPCefoCByni7k1a5r0LhHuAtaR18O8GlTlEeE8HcDR1RnrU7zdfirU5O8mHbb1bjjFqfOQbLYN1LKcD/ssxjhrl26fjqfFX5qfNCSaQwO1Ij1Y7tYE6V2SX4FFQOqaPag1VR4SztEBXJUe7CjW/MFdItyNGXW8QVUeEs7RAVyTLuxM2biQtEskFHjW414S6gtU5SHhHB3ASdeVJ1sTpHcJ581CI60phApU5yHhHB3A8ZVGquYJ9SnhkOx5tvUsgwLVeUg4R2fBzUGUjUtJu0RNxvKtrHuB2nPHQdlMG7ANLDA2rDnSuwSfWnKfb82fOCSbYwO2efHIzZoivUvU6ZqzVDo5dR4SztERuCGfHiWTcRlpl6jhEdSfEaoC1XlEOE8HcCS9sGJNS3oqOB5Tk19WT06Vh2SzbaCWLxmid2uK9C5Rk6E8caeTU8Uh2RwcuJWrjIa77m+3p4Qh1ZQWrZycOw7J5tiADX/oKRkDrV3CftVHav3FqfKQcI4O4NqFV49bE6R3iXq597F+guo8JJyjA7gunVgu02rgnhKOSGnXfk4MG1TnIeEcnQU3x0jTGGjt0t3haPXF+aRB0UwZoLH0YQebvZCnhO5tG73eq9lrTJ2HhHN0ADekDzvrsNq3p4R+Ws9S5QNU5yHhHB3AzfvTw7iQtEvUM5bnaet+vQLVeUg4R0fgJm6u4Dm1N9wuUccvKD9dEQWq8ohwng7g6JL/2jDauF0izOxNta1JcApU5yHhHB3A4X33OVvTyncJ586cJxc+QVUeEs7RAVyR75FtNwZbu3SPDirzene5BlV5SDhHB3B4FqGYS6TuEkYHuXLJ5QDVeUg4Rwdw7ZIemXX7+akQpr3JAGstIag5VR6SzbZZanO2ZM2O3iUcj5zQ6/jm1HFQNhMHbljoqbE1PXqX0FcjwsvKD0+dh4RzdADHePalduNy0i6hyzELp7UaiALVeUg4RwdwQz7NcswZcE+JOl+10xptKU8Vh2RzbMA2pR/Lw5ocvUs4b47SaK3npjlVHhLO0fnPr5bkR+do7d3CqdJ95sQaDKfnjgOyuTZgw3ugZ27voZYqYVJDlm293xatOXUeEs7RAVyW75nTmB+tSph/NKnQHCeoykPCOTqAkz6/nBHru4VTpfvG3/is9vwFqvKQcI4O4PCyTyr1PdRSJQCRnErfnJ80JJojAzTp/adsvGdqV9DbSGXMOg9KnYdks22W2kyZi9W4PSVcoZQwMR+cKg7KZuLATbr+Sfad92UkVSLJ5/hMofnyVHlIOEcHcNL3T5WM6dGqhHGo7FaYLniAqjwknKMDOOn8p2rcYtgV6uNedyzzyanykGy2DdSk659ay+/LSKp0nzQFqdVvTh2HZHNwxI3kR2WrjKnRqnTPBemZ2+mp84hwng7gpOuf+jSmRqvSfeZsuOd3gqo8JJyjAzjp+ifuxqrZqnTP5pW+R36BqjwknKMDuCL92FHIGGbtEs6cVf65H/P4AlV5SDhHB3BVOrLyceOUuks4JEsmzuMEVXlIOEdnwc002XiFtCrd9xK4GKAqDwpn6gCuXYNSTcZoa5cwOiBp29YoVYOqPCScowO4fgkBGVOkVQk3YeqUZq0foDoPCefoAI6vTiRfZMA9JRySTDwHH6A6Dwnn6ABuXHKatPa3TwE4fdQ6+xvzyUOimTIgm1gyhozp0aqEnhq3MteF8o2p0pBojo2wZelIUBnT2Nd2CW1YktPnaAenziPCeTqAoytTbdO4lLRLmOs2U14zj75AVR4SztEBHGZ5tGxMkFYlAPU2ufEb9MlDwjk6gCtXoobz4RvuKeFBjyFfUPoJqvKQcI7OgpvU5bRown1KuGwk394SH6A6Dwpn6gCu4rHIYkyQViWcBLr8eYwDVOch4RwdwDWsVJTYauOeErodmUr+tHEbVOch4RwdwEnzTkNCA+4p4eoHlZnWnS0NqvKQcI4O4Fj6srMaE6RVSQbwMjroXPIBqvOQcI4O4GR4nhNZJ9WfCuYf8ZBvnCenykOy2TZQm1fJaXTjWtIuwUf6uz/HqeZ84pBsDo64Ffke+YpuDLd26R4djDlaOT1VHhHO0wEcllnP2ZggrUr3npVSvt+j+gWq8pBwjg7gZCNzNpbV2hVsYM8/0wcVp85Dstk2S23KWKkZp4Vdul/C1Wu7H6pXnDoOymbiwK1cI9diTY/eJZItJtnBeJ6eKg8J5+gATroSuaVmtW9PCYP4gifs8wmq8pBwjg7g8MR842oMtXYJ7xlMg2trB6jOQ8I5OoDrlwwBqjVBepewiVPG8GumrwZVeUg4RwdwLB1Zpmq1cU8JQyoZz+N/v0F1HhLO0QHckJ6sfNC4nLRLVBuuIKV5eKo4JJtjA7Yp/Vjp9BtjrV2630wuQ/h+au44JJtjI2w1SWcCPYs32y5hCYHKua+hveZUeUQ4TwdwJB3ZiXUm33BPCZvYWi+r76tBVR4SztFZcLPIZppuq4INLDKkWl3fb84nD8pm2UAtX6NQsSZJ79K9AhTXxCenikOyOThwKxeXnKxZIrtEcgbFm7Y/h+n21HlIOEcHcDI8F4NsDLV2CUukNCp1HJ4qDsnm2ICtXTLkrNYU6V0iOTTlRPC5lKQ5VR4SztEBnHQl8Apoq4F7Slg4azLWPz1BVR4SztEBnGyWHGpkDLR2idD36Kn0cYKqPCScowM4+cuX1qwp0ruEU8AYEpygOg8J5+gATkZLpedktXFPCcuftiRHZDlAdR4SztERuJYuOdRmMgZbu0QyLq3Sb1uDVAWq84hwns6Cm4W7NUV6l6iUiyhzeYGqPCicqQM4kr7sKMlo43aJZERfeNLa4zSoykPCOTqAy9KZHZjs/IZ7SjgJcMK1txNU5SHhHB3AYVtmt6ZK7xIWjk2j5H54qjgkm2MDNhkxVfkaq4V7SjKkulKmvLq/ilPnIeEcHcA1+VGy+iJPhUrCm7jamvCgOVUeks22gVqXrSK2ZknvEgYHLSeu/ZtTxyHZHBy4MZawq8Nq3Z4Smv/a5/3iN82p4pBsjg3YxiVtO16F92Z7SoQ5DqPN/uJUeUg4Rwdw80q1DGuK9C6hHZss3dx5gqo8JJyjs+Bmrc2aIr1L2LPKpM+DbhpU5UHhTB2B63JOrC2zcTlpl9DhYBnCz3qCqjwinKcDOLq4NjneDLinBCCqjOcpX6BPHhLO0QFcvnrt3ZolvUt3W1bG51FeDarykHCODuCK9GS5dKON2yVc/RgpFXqBqjwknKMDuCo/Osx70LuEkwA1+XeeoCoPCefoAK7JZg3rpPpUcEByyrxe1KU5VR6SzbaBWpeu7MQ6T2+2p0QZl8il28bfnDoOyebgwI3lQEvUjMHWLhE2fuTPG0W0p8pDwjk6gBtXamlY06R3iXIVoLFWCfwCVXlIOEdnwc1m73A/FfCQbGstb84nD8pm2UBtXiKQq3ExaZeokOxW1Nfk8s2p45BsDo64Md4kmKc1RXqXCE/uyh7Gp6fOI8J5OoCjqzfZDKN92yUckPc+lk9QlYeEc3QAl7FQJy4IveGeEuGR58xYgvcAVXlIOEcHcOWqrSVrmvQuEc2rdNm1+gGq85Bwjg7gpDPR5APGUGuXKNNFo471xL0C1XlIOEcHcE3+8r0W43LSLt17VpbfVw5QnYeEc3QA16UnyzJuMuCeEkajnWehcoDqPCScowM4lr4sD2uq9C4R3tnIP1d+FajOQ8I5OgtuNumWWW3cUyLC+IB6ryeoyoPCmTqAwzvvpzXF/KlQuheQpXxy6jwkm20DtSld2TmtmdK7BJ/W+1jXy784nzgkm4MjbkM+3VMno33bJUryBbNSe3mqPCKcpwM4kn1G2ijjON0lIiz6LO0YH6A6Dwnn6AAuY42FZE1O2iXsWaX3WfkA1XlIOEcHcEU2PXMymrhdoiQDBUrts8cpUJWHhHN0ACd/6LJhxiWlXaIkO5mcDta7CDWoykPCOTqAaxd12WuM4dYuzYp7MZ+1e/aX6Tgkm2MDti4bWXlaLdxToiS/bcjQan5zfuUh4RydBTd7q9Ybp3YJO5aMrtYgdX+ZjoOymTZg4wt3pqxp0ruEtYt7nettLEpTxSHZHBuwDdn0br1M+qnMgYWyP49AK0sVh0SzZWA2pQvL8gED7SnJoZgqtzVDeluqNCSaQyNqU35vH9maIL1L3K+RufKBqeOIbJ4N2Eh6r2NaE6R3SXxqHWW9FeNL84lDsjk2YMvSd53WzMGnIt1bvM99nQz2V+k4JJotA7MivYdU2Bhb7ZKcK3sZtYwvS52GRHNooFYvYkrWtOhdkp8ffa0JpSif8CH7wx+zbPTv/Puvv0lMZHvo919+/+HP//rfv//5r7//5U+///nfX5tVkf5Hx8TtarMRduFf//jVU7v467MrOT7mfO/50+dn97c43/u3X/8Hbs1TrAplbmRzdHJlYW0KZW5kb2JqCjEyIDAgb2JqCjI3MDEyCmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjE3IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzk1ID4+CnN0cmVhbQp4nD1SS27FQAjb5xRcoNLwm895UlXdvPtva0NSqSq8iTHGMH3KkLnlS10ScYXJt16uWzymfC5bWpl5iLuLjSU+ttyX7iG2XXQusTgdR/ILMp0qRKjNqtGh+EKWhQeQTvChC8J9Of7jL4DB17ANuOE9MkGwJOYpQsZuURmaEkERYeeRFaikUJ9Zwt9R7uv3MgVqb4ylC2Mc9Am0BUJtSMQC6kAAROyUVK2QjmckE78V3WdiHGDn0bIBrhlURJZ77MeIqc6ojLxExD5PTfoolkwtVsZuUxlf/JSM1Hx0BSqpNPKU8tBVs9ALWIl5EvY5/Ej459ZsIYY6btbyieUfM8UyEs5gSzlgoZfjR+DbWXURrh25uM50gR+V1nBMtOt+yPVP/nTbWs11vHIIokDlTUHwuw6uRrHExDI+nY0peqIssBqavEYzwWEQEdb3w8gDGv1yvBA0p2sitFgim7ViRI2KbHM9vQTWTO/FOdbDE8Js753WobIzMyohgtq6hmrrQHazvvNwtp8/M+iibQplbmRzdHJlYW0KZW5kb2JqCjE4IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjQ5ID4+CnN0cmVhbQp4nE1RSYoDMAy75xX6QCFek7ynQ5lD5//Xyg6FOQQJr5KTlphYCw8xhB8sPfiRIXM3/Rt+otm7WXqSydn/mOciU1H4UqguYkJdiBvPoRHwPaFrElmxvfE5LKOZc74HH4W4BDOhAWN9STK5qOaVIRNODHUcDlqkwrhrYsPiWtE8jdxu+0ZmZSaEDY9kQtwYgIgg6wKyGCyUNjYTMlnOA+0NyQ1aYNepG1GLgiuU1gl0olbEqszgs+bWdjdDLfLgqH3x+mhWl2CF0Uv1WHhfhT6YqZl27pJCeuFNOyLMHgqkMjstK7V7xOpugfo/y1Lw/cn3+B2vD838XJwKZW5kc3RyZWFtCmVuZG9iagoxOSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDk0ID4+CnN0cmVhbQp4nEWNwRHAIAgE/1RBCQoK2k8mk4f2/40QMnxg5w7uhAULtnlGHwWVJl4VWAdKY9xQj0C94XItydwFD3Anf9rQVJyW03dpkUlVKdykEnn/DmcmkKh50WOd9wtj+yM8CmVuZHN0cmVhbQplbmRvYmoKMjAgMCBvYmoKPDwgL0JCb3ggWyAtMTAyMSAtNDYzIDE3OTQgMTIzMyBdIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzkKL1N1YnR5cGUgL0Zvcm0gL1R5cGUgL1hPYmplY3QgPj4Kc3RyZWFtCnic4zI0MFMwNjVVyOUyNzYCs3LALCNzIyALJItgQWQzuNIAFfMKfAplbmRzdHJlYW0KZW5kb2JqCjIxIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzIyID4+CnN0cmVhbQp4nDVRu23FMAzsNQUXMCB+Jc3jIEiRt3+bO9qpSNO8H1VeMqVcLnXJKllh8qVDdYqmfJ5mpvwO9ZDjmB7ZIbpT1pZ7GBaWiXlKHbGaLPdwCza+AJoScwvx9wjwK4BRwESgbvH3D7pZEkAaFPwU6JqrllhiAg2Lha3ZFeJW3SlYuKv4diS5BwlyMVnoUw5Fiim3wHwZLNmRWpzrclkK/259AhphhTjss4tE4HnAA0wk/mSAbM8+W+zq6kU2doY46dCAi4CbzSQBQVM4qz64Yftqu+bnmSgnODnWr6Ixvg1O5ktS3le5x8+gQd74Mzxnd45QDppQCPTdAiCH3cBGhD61z8AuA7ZJu3djSvmcZCm+BDYK9qhTHcrwYuzMVm/Y/MfoymZRbJCV9dHpDsrcoBNiHm9koVuytvs3D7N9/wFfGXtkCmVuZHN0cmVhbQplbmRvYmoKMjIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA4MyA+PgpzdHJlYW0KeJxFjLsNwDAIRHumYAR+JvY+UZTC3r8NECVuuCfdPVwdCZkpbjPDQwaeDCyGXXGB9JYwC1xHUI6d7KNh1b7qBI31plLz7w+Unuys4obrAQJCGmYKZW5kc3RyZWFtCmVuZG9iagoyMyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDMyMCA+PgpzdHJlYW0KeJw1UktuBTEI288puECl8E/O86qqi777b2sTvRVMMGDjKS9Z0ku+1CXbpcPkWx/3JbFC3o/tmsxSxfcWsxTPLa9HzxG3LQoEURM9WJkvFSLUz/ToOqhwSp+BVwi3FBu8g0kAg2r4Bx6lMyBQ50DGu2IyUgOCJNhzaXEIiXImiX+kvJ7fJ62kofQ9WZnL35NLpdAdTU7oAcXKxUmgXUn5oJmYSkSSl+t9sUL0hsCSPD5HMcmA7DaJbaIFJucepSXMxBQ6sMcCvGaa1VXoYMIehymMVwuzqB5s8lsTlaQdreMZ2TDeyzBTYqHhsAXU5mJlgu7l4zWvwojtUZNdw3Duls13CNFo/hsWyuBjFZKAR6exEg1pOMCIwJ5eOMVe8xM5DsCIY52aLAxjaCaneo6JwNCes6VhxsceWvXzD1TpfIcKZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDM0MCA+PgpzdHJlYW0KeJw1UjluBDEM6/0KfSCAbtvv2SBIkfy/DanZFANxdFKUO1pUdsuHhVS17HT5tJXaEjfkd2WFxAnJqxLtUoZIqLxWIdXvmTKvtzVnBMhSpcLkpORxyYI/w6WnC8f5trGv5cgdjx5YFSOhRMAyxcToGpbO7rBmW36WacCPeIScK9Ytx1gFUhvdOO2K96F5LbIGiL2ZlooKHVaJFn5B8aBHjX32GFRYINHtHElwjIlQkYB2gdpIDDl7LHZRH/QzKDET6NobRdxBgSWSmDnFunT03/jQsaD+2Iw3vzoq6VtaWWPSPhvtlMYsMul6WPR089bHgws076L859UMEjRljZLGB63aOYaimVFWeLdDkw3NMcch8w6ewxkJSvo8FL+PJRMdlMjfDg2hf18eo4ycNt4C5qI/bRUHDuKzw165gRVKF2uS9wGpTOiB6f+v8bW+19cfHe2AxgplbmRzdHJlYW0KZW5kb2JqCjI1IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjUxID4+CnN0cmVhbQp4nC1RSXIDQQi7zyv0hGan32OXK4fk/9cIygcGDYtAdFrioIyfICxXvOWRq2jD3zMxgt8Fh34r121Y5EBUIEljUDWhdvF69B7YcZgJzJPWsAxmrA/8jCnc6MXhMRlnt9dl1BDsXa89mUHJrFzEJRMXTNVhI2cOP5kyLrRzPTcg50ZYl2GQblYaMxKONIVIIYWqm6TOBEESjK5GjTZyFPulL490hlWNqDHscy1tX89NOGvQ7Fis8uSUHl1xLicXL6wc9PU2AxdRaazyQEjA/W4P9XOyk994S+fOFtPje83J8sJUYMWb125ANtXi37yI4/uMr+fn+fwDX2BbiAplbmRzdHJlYW0KZW5kb2JqCjI2IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjE1ID4+CnN0cmVhbQp4nDVROQ4DIQzs9xX+QCSML3hPoijN/r/NjNFWHsFchrSUIZnyUpOoIeVTPnqZLpy63NfMajTnlrQtc4C4trwvrZLAiWaIg8FpmLgBmjwBQ9fRqFFDFx7Q1KVTKLDcBD6Kt24P3WO1gZe2IeeJIGIoGSxBzalFExZtzyekNb9eixvel+3dyFOlxpYYgQYBVjgc1+jX8JU9TybRdBUy1Ks1yxgJE0UiPPmOptUT61o00jIS1MYRrGoDvDv9ME4AABNxywJkn0qUs+TEb7H0swZX+v4Bn0dUlgplbmRzdHJlYW0KZW5kb2JqCjE1IDAgb2JqCjw8IC9CYXNlRm9udCAvRGVqYVZ1U2FucyAvQ2hhclByb2NzIDE2IDAgUgovRW5jb2RpbmcgPDwKL0RpZmZlcmVuY2VzIFsgNDggL3plcm8gL29uZSAvdHdvIC90aHJlZSAvZm91ciAvZml2ZSAvc2l4IDU2IC9laWdodCAvbmluZSBdCi9UeXBlIC9FbmNvZGluZyA+PgovRmlyc3RDaGFyIDAgL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvRm9udERlc2NyaXB0b3IgMTQgMCBSCi9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdIC9MYXN0Q2hhciAyNTUgL05hbWUgL0RlamFWdVNhbnMKL1N1YnR5cGUgL1R5cGUzIC9UeXBlIC9Gb250IC9XaWR0aHMgMTMgMCBSID4+CmVuZG9iagoxNCAwIG9iago8PCAvQXNjZW50IDkyOSAvQ2FwSGVpZ2h0IDAgL0Rlc2NlbnQgLTIzNiAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvRm9udE5hbWUgL0RlamFWdVNhbnMgL0l0YWxpY0FuZ2xlIDAKL01heFdpZHRoIDEzNDIgL1N0ZW1WIDAgL1R5cGUgL0ZvbnREZXNjcmlwdG9yIC9YSGVpZ2h0IDAgPj4KZW5kb2JqCjEzIDAgb2JqClsgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAKNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCAzMTggNDAxIDQ2MCA4MzggNjM2Cjk1MCA3ODAgMjc1IDM5MCAzOTAgNTAwIDgzOCAzMTggMzYxIDMxOCAzMzcgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNgo2MzYgNjM2IDMzNyAzMzcgODM4IDgzOCA4MzggNTMxIDEwMDAgNjg0IDY4NiA2OTggNzcwIDYzMiA1NzUgNzc1IDc1MiAyOTUKMjk1IDY1NiA1NTcgODYzIDc0OCA3ODcgNjAzIDc4NyA2OTUgNjM1IDYxMSA3MzIgNjg0IDk4OSA2ODUgNjExIDY4NSAzOTAgMzM3CjM5MCA4MzggNTAwIDUwMCA2MTMgNjM1IDU1MCA2MzUgNjE1IDM1MiA2MzUgNjM0IDI3OCAyNzggNTc5IDI3OCA5NzQgNjM0IDYxMgo2MzUgNjM1IDQxMSA1MjEgMzkyIDYzNCA1OTIgODE4IDU5MiA1OTIgNTI1IDYzNiAzMzcgNjM2IDgzOCA2MDAgNjM2IDYwMCAzMTgKMzUyIDUxOCAxMDAwIDUwMCA1MDAgNTAwIDEzNDIgNjM1IDQwMCAxMDcwIDYwMCA2ODUgNjAwIDYwMCAzMTggMzE4IDUxOCA1MTgKNTkwIDUwMCAxMDAwIDUwMCAxMDAwIDUyMSA0MDAgMTAyMyA2MDAgNTI1IDYxMSAzMTggNDAxIDYzNiA2MzYgNjM2IDYzNiAzMzcKNTAwIDUwMCAxMDAwIDQ3MSA2MTIgODM4IDM2MSAxMDAwIDUwMCA1MDAgODM4IDQwMSA0MDEgNTAwIDYzNiA2MzYgMzE4IDUwMAo0MDEgNDcxIDYxMiA5NjkgOTY5IDk2OSA1MzEgNjg0IDY4NCA2ODQgNjg0IDY4NCA2ODQgOTc0IDY5OCA2MzIgNjMyIDYzMiA2MzIKMjk1IDI5NSAyOTUgMjk1IDc3NSA3NDggNzg3IDc4NyA3ODcgNzg3IDc4NyA4MzggNzg3IDczMiA3MzIgNzMyIDczMiA2MTEgNjA1CjYzMCA2MTMgNjEzIDYxMyA2MTMgNjEzIDYxMyA5ODIgNTUwIDYxNSA2MTUgNjE1IDYxNSAyNzggMjc4IDI3OCAyNzggNjEyIDYzNAo2MTIgNjEyIDYxMiA2MTIgNjEyIDgzOCA2MTIgNjM0IDYzNCA2MzQgNjM0IDU5MiA2MzUgNTkyIF0KZW5kb2JqCjE2IDAgb2JqCjw8IC9laWdodCAxNyAwIFIgL2ZpdmUgMTggMCBSIC9mb3VyIDE5IDAgUiAvbmluZSAyMSAwIFIgL29uZSAyMiAwIFIKL3NpeCAyMyAwIFIgL3RocmVlIDI0IDAgUiAvdHdvIDI1IDAgUiAvemVybyAyNiAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE1IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL0NBIDAgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PgovQTIgPDwgL0NBIDEgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PgovQTMgPDwgL0NBIDEgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMC41ID4+ID4+CmVuZG9iago1IDAgb2JqCjw8ID4+CmVuZG9iago2IDAgb2JqCjw8ID4+CmVuZG9iago3IDAgb2JqCjw8IC9GMS1EZWphVnVTYW5zLW1pbnVzIDIwIDAgUiA+PgplbmRvYmoKMiAwIG9iago8PCAvQ291bnQgMSAvS2lkcyBbIDExIDAgUiBdIC9UeXBlIC9QYWdlcyA+PgplbmRvYmoKMjcgMCBvYmoKPDwgL0NyZWF0aW9uRGF0ZSAoRDoyMDIxMDkxNjE0NDMyNiswMicwMCcpCi9DcmVhdG9yIChNYXRwbG90bGliIHYzLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjQuMykgPj4KZW5kb2JqCnhyZWYKMCAyOAowMDAwMDAwMDAwIDY1NTM1IGYgCjAwMDAwMDAwMTYgMDAwMDAgbiAKMDAwMDAzMjYyOCAwMDAwMCBuIAowMDAwMDMyMzY1IDAwMDAwIG4gCjAwMDAwMzIzOTcgMDAwMDAgbiAKMDAwMDAzMjUzNyAwMDAwMCBuIAowMDAwMDMyNTU4IDAwMDAwIG4gCjAwMDAwMzI1NzkgMDAwMDAgbiAKMDAwMDAwMDA2NSAwMDAwMCBuIAowMDAwMDAwMzk3IDAwMDAwIG4gCjAwMDAwMjc1MDYgMDAwMDAgbiAKMDAwMDAwMDIwOCAwMDAwMCBuIAowMDAwMDI3NDg0IDAwMDAwIG4gCjAwMDAwMzExNzQgMDAwMDAgbiAKMDAwMDAzMDk3NCAwMDAwMCBuIAowMDAwMDMwNjIyIDAwMDAwIG4gCjAwMDAwMzIyMjcgMDAwMDAgbiAKMDAwMDAyNzUyNiAwMDAwMCBuIAowMDAwMDI3OTk0IDAwMDAwIG4gCjAwMDAwMjgzMTYgMDAwMDAgbiAKMDAwMDAyODQ4MiAwMDAwMCBuIAowMDAwMDI4NjU0IDAwMDAwIG4gCjAwMDAwMjkwNDkgMDAwMDAgbiAKMDAwMDAyOTIwNCAwMDAwMCBuIAowMDAwMDI5NTk3IDAwMDAwIG4gCjAwMDAwMzAwMTAgMDAwMDAgbiAKMDAwMDAzMDMzNCAwMDAwMCBuIAowMDAwMDMyNjg4IDAwMDAwIG4gCnRyYWlsZXIKPDwgL0luZm8gMjcgMCBSIC9Sb290IDEgMCBSIC9TaXplIDI4ID4+CnN0YXJ0eHJlZgozMjg0NQolJUVPRgo=\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2021-09-16T14:43:24.825070\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.4.3, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["fig, ax = plt.subplots(2, 2, figsize=(10, 6))\n", "for i in range(4):\n", " ax_sub = ax[i // 2][i % 2]\n", " ax_sub.bar(np.arange(out.shape[1], dtype=np.int32), out[i + 4, :, 0, 14, 14], **plot_args)\n", " ax_sub.set_yscale(\"log\")\n", " ax_sub.set_xticks([0, 64, 128, 192, 256])\n", "plt.show()\n", "plt.close()"]}, {"cell_type": "markdown", "id": "d39faab7", "metadata": {"papermill": {"duration": 0.074666, "end_time": "2021-09-16T12:43:27.671433", "exception": false, "start_time": "2021-09-16T12:43:27.596767", "status": "completed"}, "tags": []}, "source": ["Overall we see a very diverse set of distributions, with a usual peak\n", "for 0 and close to 1. However, the distributions in the first row show a\n", "potentially undesirable behavior. For instance, the value 242 has a\n", "1000x lower likelihood than 243 although they are extremely close and\n", "can often not be distinguished. This shows that the model might have not\n", "generlized well over pixel values. The better solution to this problem\n", "is to use discrete logitics mixtures instead of a softmax distribution.\n", "A discrete logistic distribution can be imagined as discretized, binned\n", "Gaussians. Using a mixture of discrete logistics instead of a softmax\n", "introduces an inductive bias to the model to assign close-by values\n", "similar likelihoods. We can visualize a discrete logistic below:"]}, {"cell_type": "code", "execution_count": 28, "id": "41272be3", "metadata": {"execution": {"iopub.execute_input": "2021-09-16T12:43:27.818104Z", "iopub.status.busy": "2021-09-16T12:43:27.817603Z", "iopub.status.idle": "2021-09-16T12:43:28.944136Z", "shell.execute_reply": "2021-09-16T12:43:28.944525Z"}, "papermill": {"duration": 1.200149, "end_time": "2021-09-16T12:43:28.944659", "exception": false, "start_time": "2021-09-16T12:43:27.744510", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUgovR3JvdXAgPDwgL0NTIC9EZXZpY2VSR0IgL1MgL1RyYW5zcGFyZW5jeSAvVHlwZSAvR3JvdXAgPj4KL01lZGlhQm94IFsgMCAwIDQwOS4zMjUgMjI4LjM3MDYyNSBdIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovVHlwZSAvUGFnZSA+PgplbmRvYmoKOSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEyIDAgUiA+PgpzdHJlYW0KeJzV3U1zHMeRBuB7/4o+UgcV6/vjKIZsRui0khGxB4cOFgVpoeBYIVGW7H+/mTUzQE3nmzRDJA6pDXqJl0AD70Nguio50x32n7aXX4T9x3c7/c/u95/o1x/0+9f89ubprdOW/XApFvr928ffx9hdar7Sb9/Su928+X/b9sPm3Qit5uZL7/vxjTx8GNW3vv/Kn/S1eIfHN7bDe29bqa7Tp8nRjVr4E562NIKrN9nbxyz66ny+hJePvcnmV/vLLg6bUnZ9DzU5T5/4fv/f/Z/7yy/imeor+vUT/TpTvfzy/veHN/ffvH61vXm3teba7dd3Tm4+6/a37ev9l+sBvQuF+K/HnG++vqTbL1sgo889/VFtjr/84sKkfnPaXt3tL/8a9hD2ux/mX9fd99vf9xfB+8/2b/e7r7a/3G1fz0/16QqG5J2vh7+Dx/AT1AxhuHmAD2kanrNpj/RtfSh6yT5FzxZcbx9WMz5jzUgfWPuh52P4CYrGmF340KbpOZv2dluS3v4U/Vp1H1YuP2O5lOixORwfG6/hJ6iZ6FG+hw/8uSzP2VQ84J/wA/6f7NmD+9Ca9abm+sM9ovP8QDFcWw8Qbw/wP5/RV+x6uv63v3j4N0WDTntthEr/tf3F/dv993+8/df984l+4Fn1z/74z4On7lovg010UDqLPOM3zuVIdB4duUQflppP2UfWpK+q1VJCirGE9zeNz96UvglHzK3VpelT9pFNO/2F5uJ7H8XH9zfNz940BPoL9DmVuFRdwo/sGgL9tcZca66t/Jey9fnL0qF6TyP2texT+LFl+UfV55h99j69v2x//rL0o9lrKj6vZZ/Cjy1LX1jt9Jhf6TE/v7fsc641r2Xpp7Pn5NtYyz6FH1uWfmZrpUVB6MW//3E4xNszGx/lcz5eqHMLSF9TG3kuEd97eqN3H7H5fjm9/SpOeD9/94/vHt4+/PafP4mbzg5xdE/f/KE0eiOft48xFdpB1usGMs/9Ji3V/tDff795/29ebx/4nrw3/Zz+0I0hNkeUE9b51LeeTt+Tg+PMveorszKgKcuk5krsIR4FtByLWZZBTadMpKMc9mVv35dDMdMyoCnLxEYn2XL86dBzLGZZBjWdMoEeWrN8PFFzKGZaBjRlGToNVZ/k44mWYzHLMqjplAkutSAfT9QcipmWAU1ZxlcXsjgL6zkWsyyDmk4ZT/sqsG5RcyhmWgY0JZk0iqtNrlu0XBEzLAObThn+QLlu0XMoZloGNGWZXhx9vHg8UXMsZlkGNWWZNtzoct2i5ljMsgxqOmUy7YPkukXPoZhpGdCUZeqgfZBct6g5FrMsg5pOmUxHkesWPYdipmVAU5Ypw3kwh1FzLGZZBjWdMomqgPWMmkMx0zKgKcvk7gqYw6g5FrMsg5pOmeQSmMPoORQzLQOaskzqLoA5jJpjMcsyqOmUobfAHEbPoZhpGdCUZWKjfRBYt2g5FrMsg5pOmUgfCNYtag7FTMuApiwTmotgDqPmWMyyDGo6ZYIbaD6j5lDMtAxoyjK+uobmMFqOxSzLoKZTJriC5jBqDsVMy4CmJBNHdRHMYbRcETMsA5tOmUD7ILlu0XMoZloGNGWZXqiKXLeoORazLIOaThn6rGAOo+dQzLQMaMoyrbgE5jBqjsUsy6CmU4afqSXXLXoOxUzLgKYsU+kLB3MYNcdilmVQU5Ypw1Uwh1FzLGZZBjWdMtllMIfRcyhmWgY0ZZk8aB8E1i1ajsUsy6CmUybRPgisW9QcipmWAU1Zhp9GDuYwao7FLMugplMmuQLmMHoOxUzLgKYsE7uLYA6j5ljMsgxqOmXoSwZzGD2HYqZlQFOWCc11MIdRcyxmWQY1nTLRFTSfUXMoZloGNGUZ32gfBNYtWo7FLMugplMm0j4IrFvUHIqZlgFNSYbeifZBct2i5YqYYRnYdMoEV8EcRs+hmGkZ0JRlenUZzGHUHItZlkFNp0xwAcxh9ByKmZYBTVmmletbtwJajsUsy6CmU8a7BuYweg7FTMuApixTC+2D5LpFzbGYZRnUdMrQUcAcRs+hmGkZ0JRlSqF9EFjPaDkWsyyDmrJMHq6DOYyaYzHLMqjplMmugDmMnkMx0zKgKcvwlZbAHEbNsZhlGdR0ymQXwBxGz6GYaRnQlGX4ilpgDqPmWMyyDGo6ZRLtg8C6Rc2hmGkZ0JRlQqd9EFi3aDkWsyyDmk6ZRPsgsG5RcyhmWgY0ZRnf3EBzGC3HYpZlUNMpE11Dcxg1h2KmZUBTkhnNZTCGUWLFy7ALKsos0UUwg9FiqGVZBfQkld6cB/MXJcZYhlVQT1YJtPeRaxUthliWVUBPUuGL5YK5ixJjLMMqqCerBNrzyDWKFkMsyyqgJ6nU6gKYtygxxjKsgnqyincdzFq0GGJZVgE9SaUUV8GcRYkxlmEV1JNV6MPAjEWLIZZlFdCTL2BVXADzFSXGWIZVUM95iSba24DXWuMYYxlWQT3n5QxoZwNeZ63EEMuyCug5X+BGOxvwnBccYyzDKqjnfDK4i+j1SEoMsSyrgJ7z6UHOw+e6wBhjGVZBPec/i7iGJilKDLEsq4CevGd2BcxRcIqpDJuAmnwCcgldGwamEMqwCGh52gJ6ORoMsZJdD1DytDX0ukUYQiO7GqAjfXNE9AJXmVaEVE17oOoE0tBroWVaVSbDIqD7aZs3wxCrUZk26NRMi6DufCM0dI0FmQ7oNGyLgO584zR0PQ6RpoCcZmpXBHUnkY6u3SLShJ1sn2lQ99OWE7rOj0hzQE7Z9vcI6k4iHV0TSqQlIKdiXAR0P20lo+uHibR25DRTuyKoO4kMB641J9JRkdOwvR5B3U9bzei6hMe05QiczqldEdSdRAa6huUx7aEDp3NqWAR0p01eQdc7PaYjJ+B0Tu2KoO6nrXt0bdxDWgNgOod2PVBz8ijoKsqHtOYBkLLtFTxqftqGR9fbPqR00CCVLqldEdSdRCq6NvshbaM14HRODYuA7qct+ODAhfyP8bzBrZC6pHZNYHtGqei+D4e4RR+atLrGllVAfVIJAd0n5BC3GmoGWJfYsAqqzyoV3VfmEPckpbrtNQksztPniO5AdIhHPs8Wb49xSQ2boPJs0hy4X9UaU6MQ+1HqKbVsAsqTSYro7mZrzBcpSnne8/lAdY0Nq6D6rNLQ3fDWmBqNNGqSWNfYsgqoTyrUDtw9cY3pd62FGgTWY2xYBdVnlY7utrnGyaWQY4oC6zG2rALqk0pJ6O6sa5xcSWMUifUYG1ZB9Vmlo7v5rnFyI4TUo8B6jC2rgPqkUjO6+/MaZ17Ett4PWEtsWAXVZ5XhwN3C17hQ/dhTP2AtsWUVUJ+fRJDR3eXXuDn6FulzQL8eZIkNq6D6rDJoSyPXK0s8XA+tjnHAWmLLKqA+qfRMmxm5XnmKC/3QjN7m+GQ5yBobVkH1SWXQMcBU5Snml6T01mu7xVpjwyqoPqsUl8FU5SnmF3WN6Pu4xVpjyyqg/mmLnt5TTlWWmF8s6ntI+RZrje2qwPqsUmg1JtYrSxx8d62n0Q9aN7llFwBALiHQ3kasWJY4pOrog0s+eq25YRcEwC6VdjdizbLE81pvofa5Q1691tyyCwDgpz8G2t+IVcsShz74TDxCPnituWEXBMAuVETOWZY4DL6m+qAt4cFrzS27AABySdF5OWlZ4sv3RZ//Rrh6rblhFwTALs0lOWtZ4uvjSD9wLbFlFVCfVOiNImctS3w56+QSD1prbtgFAbBLc01OW5b4vErJ/vyc4sVrzS27AAByKfxPX3L18hRfFrXzSUzLQdbYsAqqzyp0RpHzliW+bIDOP0RPB1ljyyqgPqnQF5zlvGWJz5vlkest1hobVkH1WaXze4pHlqf4MlgJ9RZrjS2rgPqk0pIb4FU+S3wewp0Xc8tBltiwCqrPKsMF8EqfJeaB7fBzhLAeZIktq4D6/PKn7BKYtyzxHO63i8rTQZbYsAqqzyrDVTBvWeLzPwTlfsBaYssqoD6pDPqywbRliec/GuaLytNBltiwCqp/2pL3tLsR65U1Pv8D83xJyy3WY2xXBdZnlUKbG7FeWePzkxHm041vsR5jyyqgPqkE+pxyzrLG84krfj4t/RbrMTasguqzSnFNTlnWmJ/klP3c8NxiPcaWVUB9UomBdnlivbLG5yfEjSyxrrFhFVSfVaqLcsayxufnTsYksa6xZRVQn1+yHVyWE5ZDPHIM8YC1xIZVUH1WqfSeYr1yiHvyGWBdY8sqoD6p5EA7G7leuY1pFzifuHI4yDU2rILqs8rc5YnHldu4RT+fqH7EusSWVUB9UinRJTlfOcYjn79XDge5xIZVUH1Waa7K+cox7t43gHWJLauA+qRSo+tyvnKM6cEEYV1iwyqoPqt0+oLleuUQ1zJfui2wivFXdMP6pNJ4lyfXK4e4xhgQ1jk2rILqs0qnnY1crxzjcV7bHg8yrK9tUX2++E6inY1crxzj1jLCOseGVVB9Vhm0s5HrlWN8UcFYllVAfVIZ2UUwX1FibGhYBfVkleEymK8c48v3yvEg5r9XUP3Tln12Tc5XlFgxtKsCe7LKcEPOV9QYYllWAT1JJRQX5HxFizGWYRXUk1Sip52NvAqcEmMswyqoJ6sU2tnIK8FpMcSyrAJ68gUDPe1sxLlWizGWYRXUk1Wq83K+osYQy7IK6EkqObgk5ytajLEMq6CerFJdkfMVNYZYllVAT1IpgZ8hKh5AlBhjGVZBPVmlOS/nK2oMsSyrgJ6kUiPtbOTCRIkxlmEV1JNVGu1s5MJEiyGWZRXQk1RapJ2NXJgoMcYyrIJ6skpzQw5S1BhiWVYBPfnSz/zsHLkwUWKMZVgF9WSV7jKYr2gxxLKsAnqSykiugkGKEmMswyqoJ6t018EgRYshlmUV0PO0Fc+vUhDnWiVWsOyqwJ6sMmhnI861agyxLKuAnqQSMu1s5NX2lRhjGVZBPVlluCZnA2oMsSyrgJ6kEovzcjagxRjLsArqSSqJjiFnA1qMsQyroJ6sUlyWswE1hliWVUBPvrWJd03OBrQYYxlWQT1ZpdDORq5XtBhiWVYBPUmlBNrZyIWJEmMswyqoJ6tU2tnIhYkWQyzLKqAnqdTgKrhDkBJjLMMqqCerUBE5G1BjiGVZBfQklRZdkLMBLcZYhlVQT1Zp6L63agyxLKuAnqTSI7o7shZjLMMqqCerNHQXbTWGWJZVQE++SV1C91vXYoxlWAX1ZJVOOxu5MNFiiGVZBfQ8bdUnl+VsQIkVLLsqsCerdNfkbECNIZZlFdCTVEJyQ84GtBhjGVZBPVlluCBnA2oMsSyrgJ6kErPLcjagxRjLsArqySqDdjbiXKvGEMuyCuhJKvxly9mAFmMswyqoJ99Y19PORpxrtRhjGVZBPVmluCRnA2oMsSyrgJ6kUuhzgrswKzHGMqyCerJKcR3ciVmLIZZlFdCTVGpwXs4GtBhjGVZBPVmluihnA2oMsSyrgJ6k0gLtbOTCRIkxlmEV1JNVKu1s5MJEiyGWZRXQk1R6oJ2NXJgoMcYyrIJ6skpzAcxXtBhiWVYBPUllRJfBIEWJMZZhFdSTVZqrYJCixRDLsgroedqaj67L2YASK1h2VWBPVukuyNmAGkMsyyqgJ6mERDsbca7VYoxlWAX1ZJVOOxtxrlVjiGVZBfQklZhoZyPOtVqMsQyroJ6sMpyXswE1hliWVUBPUqEvO8rZgBZjLMMqqCerDFfkbECNIZZlFdCTVHJ2Tc4GtBhjGVZBPVllXN+4VVFiiGVZBfQklVJoZyMXJkqMsQyroJ6kUunD5GxAizGWYRXUk1UK7WzkwkSLIZZlFdCTVJp3Xc4GtBhjGVZBPVmluiBnA2oMsSyrgJ6k0oNLcjagxRjLsArqySrVFTBf0WKIZVkF9CSVEVwHgxQlxliGVVBPVmm0s5ELEy2GWJZVQM/T1n2knY041yqxgmVXBfZklUY7G3GuVWOIZVkF9CSVEF2TswEtxliGVVBPVmluyNmAGkMsyyqgJ6nE5KKcDWgxxjKsgnqySndZzgbUGGJZVgE9SSUlV+VsQIsxlmEV1JNVOu1sxLlWjSGWZRXQk1Rypp2NXK8oMcYyrIJ6ssqgnY1cmGgxxLKsAnqSSsmuyNmAFmMswyqoJ6sMKiIXJloMsSyrgJ6kUovzcjagxRjLsArqSSqNjiFnA1qMsQyroJ6sUlyRswE1hliWVUBPUumedjZyYaLEGMuwCurJKoV2NnJhosUQy7IK6EkqI9DORi5MlBhjPaq8/CJy4bh/tdPHc5ewv95ffnn/+8Ob+29ev9rfvNuE3OkaRV+dz5djbn/bEn0Rx+/ta3Z8X3BU+eHrey4H0I/qqYh3oew/UpGf6Ncf5zdfX9Ltly3sfv/c0x8FfpXQ2SaGeD3Mm9P26m5/+Vf687jf/UBH9Pvd99vf9xdfPrx78+tn9Ckd7Rgu/+0v7n+739/+/OPDu98e3uzf0//79eG7f/328PM/P9u/3e++2v5yt31N//f/HTO62AplbmRzdHJlYW0KZW5kb2JqCjEyIDAgb2JqCjQ4NzYKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTcgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNjQgPj4Kc3RyZWFtCnicPZDBEUMhCETvVrElgIBAPclkcvi//2tAk1xkHWD3qTuBkFGHM8Nn4smD07E0cG8VjGsIryP0CE0Ck8DEwZp4DAsBp2GRYy7fVZZVp5Wumo2e171jQdVplzUNbdqB8q2PP8I13qPwGuweQgexKHRuZVoLmVg8a5w7zKPM535O23c9GK2m1Kw3ctnXPTrL1FBeWvuEzmi0/SfXL7sxXh+FFDkICmVuZHN0cmVhbQplbmRvYmoKMTggMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNzAgPj4Kc3RyZWFtCnicPZBLEsMgDEP3nEJHAP+A87TT6YLcf1vLmXSDFGPLL0RXdOyVh8fGlI33aGNPhC1c5XQaTlMZj4u7Zl2gy2Ey02+8mrnAVGGR1eyi+hi8ofOsZoevVTMxhDeZEhpgKndyD/X1pzjt25KQbFdh0J0apLMwzJH8PRBTc9BziJH8I19ya2HQmeYXFy2rGa1lTNHsYapsLQzqjUF3yvXUeq7zMBHv8wPfQT5kCmVuZHN0cmVhbQplbmRvYmoKMTkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzMDcgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyMCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDI0NCA+PgpzdHJlYW0KeJxFkU1yBSEIhPeeoi/wquRXPc+kUllM7r8NzbwkK1qF5gPTAhNH8BJD7ImVEx8yfC/oMny3MjvwOtmZcE+4blzDZcMzYVvgOyrLO15Dd7ZSP52hqu8aOd4uUjV0ZWSfeqGaC8yQiK4RWXQrl3VA05TuUuEabFuCFPVKrCedoDToEcrwd5RrfHUTT6+x5FTNIVrNrRMairBseEHUySQRtQ2LJ5ZzIVH5qhurOi5gkyXi9IDcoJVmfHpSSREwg3ysyWjMAjbQk7tnF8aaSx5Fjlc0mLA7STXwgPfitr73NnGP8xf4hXff/ysOfdcCPn8AS/5dBgplbmRzdHJlYW0KZW5kb2JqCjIxIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjMyID4+CnN0cmVhbQp4nDVRSW7EMAy7+xX8wADW7rwnxaCH9v/XUsoUCEAltrglYmMjAi8x+DmI3PiSNaMmfmdyV/wsT4VHwq3gSRSBl+FedoLLG8ZlPw4zH7yXVs6kxpMMyEU2PTwRMtglEDowuwZ12Gbaib4h4bMjUs1GltPXEvTSKgTKU7bf6YISbav6c/usC2372hNOdnvqSeUTiOeWrMBl4xWTxVgGPVG5SzF9kOpsoSehvCifg2w+aohElyhn4InBwSjQDuy57WfiVSFoXd2nbWOoRkrH078NTU2SCPlECWe2NO4W/n/Pvb7X+w9OIVQRCmVuZHN0cmVhbQplbmRvYmoKMjIgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMzEgPj4Kc3RyZWFtCnicNU85kgQhDMt5hT4wVRjbQL+np7Y22Pl/upKZTpDwIcnTEx2ZeJkjI7Bmx9taZCBm4FNMxb/2tA8TqvfgHiKUiwthhpFw1qzjbp6OF/92lc9YB+82+IpZXhDYwkzWVxZnLtsFY2mcxDnJboxdE7GNda2nU1hHMKEMhHS2w5Qgc1Sk9MmOMuboOJEnnovv9tssdjl+DusLNo0hFef4KnqCNoOi7HnvAhpyQf9d3fgeRbvoJSAbCRbWUWLunOWEX712dB61KBJzQppBLhMhzekqphCaUKyzo6BSUXCpPqforJ9/5V9cLQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjQ5ID4+CnN0cmVhbQp4nD1QO45EIQzrOYUv8CTyI3AeRqstZu/frgOaKVBMfrYzJNARgUcMMZSv4yWtoK6Bv4tC8W7i64PCIKtDUiDOeg+IdOymNpETOh2cMz9hN2OOwEUxBpzpdKY9ByY5+8IKhHMbZexWSCeJqiKO6jOOKZ4qe594FiztyDZbJ5I95CDhUlKJyaWflMo/bcqUCjpm0QQsErngZBNNOMu7SVKMGZQy6h6mdiJ9rDzIozroZE3OrCOZ2dNP25n4HHC3X9pkTpXHdB7M+Jy0zoM5Fbr344k2B02N2ujs9xNpKi9Sux1anX51EpXdGOcYEpdnfxnfZP/5B/6HWiIKZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDM5NSA+PgpzdHJlYW0KeJw9UktuxUAI2+cUXKDS8JvPeVJV3bz7b2tDUqkqvIkxxjB9ypC55UtdEnGFybderls8pnwuW1qZeYi7i40lPrbcl+4htl10LrE4HUfyCzKdKkSozarRofhCloUHkE7woQvCfTn+4y+AwdewDbjhPTJBsCTmKULGblEZmhJBEWHnkRWopFCfWcLfUe7r9zIFam+MpQtjHPQJtAVCbUjEAupAAETslFStkI5nJBO/Fd1nYhxg59GyAa4ZVESWe+zHiKnOqIy8RMQ+T036KJZMLVbGblMZX/yUjNR8dAUqqTTylPLQVbPQC1iJeRL2OfxI+OfWbCGGOm7W8onlHzPFMhLOYEs5YKGX40fg21l1Ea4dubjOdIEfldZwTLTrfsj1T/5021rNdbxyCKJA5U1B8LsOrkaxxMQyPp2NKXqiLLAamrxGM8FhEBHW98PIAxr9crwQNKdrIrRYIpu1YkSNimxzPb0E1kzvxTnWwxPCbO+d1qGyMzMqIYLauoZq60B2s77zcLafPzPoom0KZW5kc3RyZWFtCmVuZG9iagoyNSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDI0OSA+PgpzdHJlYW0KeJxNUUmKAzAMu+cV+kAhXpO8p0OZQ+f/18oOhTkECa+Sk5aYWAsPMYQfLD34kSFzN/0bfqLZu1l6ksnZ/5jnIlNR+FKoLmJCXYgbz6ER8D2haxJZsb3xOSyjmXO+Bx+FuAQzoQFjfUkyuajmlSETTgx1HA5apMK4a2LD4lrRPI3cbvtGZmUmhA2PZELcGICIIOsCshgslDY2EzJZzgPtDckNWmDXqRtRi4IrlNYJdKJWxKrM4LPm1nY3Qy3y4Kh98fpoVpdghdFL9Vh4X4U+mKmZdu6SQnrhTTsizB4KpDI7LSu1e8TqboH6P8tS8P3J9/gdrw/N/FycCmVuZHN0cmVhbQplbmRvYmoKMjYgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA5NCA+PgpzdHJlYW0KeJxFjcERwCAIBP9UQQkKCtpPJpOH9v+NEDJ8YOcO7oQFC7Z5Rh8FlSZeFVgHSmPcUI9AveFyLcncBQ9wJ3/a0FScltN3aZFJVSncpBJ5/w5nJpCoedFjnfcLY/sjPAplbmRzdHJlYW0KZW5kb2JqCjI3IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzQxID4+CnN0cmVhbQp4nEVSS25EMQjbv1NwgUjhl5DztKq6mN5/W5tM1c3gCWBseMtTpmTKsLklIyTXlE99IkOspvw0ciQipvhJCQV2lY/Ha0usjeyRqBSf2vHjsfRGptkVWvXu0aXNolHNysg5yBChnhW6snvUDtnwelxIuu+UzSEcy/9QgSxl3XIKJUFb0HfsEd8PHa6CK4JhsGsug+1lMtT/+ocWXO9992LHLoAWrOe+wQ4AqKcTtAXIGdruNiloAFW6i0nCo/J6bnaibKNV6fkcADMOMHLAiCVbHb7R3gCWfV3oRY2K/StAUVlA/MjVdsHeMclIcBbmBo69cDzFmXBLOMYCQIq94hh68CXY5i9Xroia8Al1umQvvMKe2ubnQpMId60ADl5kw62ro6iW7ek8gvZnRXJGjNSLODohklrSOYLi0qAeWuNcN7HibSOxuVff7h/hnC9c9usXS+yExAplbmRzdHJlYW0KZW5kb2JqCjI4IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNzIgPj4Kc3RyZWFtCnicMzK3UDBQsDQBEoYWJgrmZgYKKYZcQL6piblCLhdIDMTKAbMMgLQlnIKIZ4CYIG0QxSAWRLGZiRlEHZwBkcvgSgMAJdsWyQplbmRzdHJlYW0KZW5kb2JqCjI5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNDcgPj4Kc3RyZWFtCnicMzK3UDBQsDQBEoYWJgrmZgYKKYZclhBWLhdMLAfMAtGWcAoinsGVBgC5Zw0nCmVuZHN0cmVhbQplbmRvYmoKMzAgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNjMgPj4Kc3RyZWFtCnicRZA7EgMhDEN7TqEj+CMDPs9mMik2929j2GxSwNNYIIO7E4LU2oKJ6IKHtiXdBe+tBGdj/Ok2bjUS5AR1gFak42iUUn25xWmVdPFoNnMrC60THWYOepSjGaAQOhXe7aLkcqbuzvlDcPVf9b9i3TmbiYHJyh0IzepT3Pk2O6K6usn+pMfcrNd+K+xVYWlZS8sJt527ZkAJ3FM52qs9Px8KOvYKZW5kc3RyZWFtCmVuZG9iagozMSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIxOCA+PgpzdHJlYW0KeJw9ULmNBDEMy12FGljAeu2pZxaLS6b/9Ej59iLRFkVSKjWZkikvdZQlWVPeOnyWxA55huVuZDYlKkUvk7Al99AK8X2J5hT33dWWs0M0l2g5fgszKqobHdNLNppwKhO6oNzDM/oNbXQDVocesVsg0KRg17YgcscPGAzBmROLIgxKTQb/rnKPn16LGz7D8UMUkZIO5jX/WP3ycw2vU48nkW5vvuJenKkOAxEckpq8I11YsS4SEWk1QU3PwFotgLu3Xv4btCO6DED2icRxmlKOob9rcKXPL+UnU9gKZW5kc3RyZWFtCmVuZG9iagozMiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDgzID4+CnN0cmVhbQp4nEWMuw3AMAhEe6ZgBH4m9j5RlMLevw0QJW64J909XB0JmSluM8NDBp4MLIZdcYH0ljALXEdQjp3so2HVvuoEjfWmUvPvD5Se7KzihusBAkIaZgplbmRzdHJlYW0KZW5kb2JqCjMzIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNTEgPj4Kc3RyZWFtCnicMza0UDBQMDQwB5JGhkCWkYlCiiEXSADEzOWCCeaAWQZAGqI4B64mhyuDKw0A4bQNmAplbmRzdHJlYW0KZW5kb2JqCjM0IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTYwID4+CnN0cmVhbQp4nEWQORIDMQgEc72CJ0hcgvesy7XB+v+pB9ZHoukCNBy6Fk3KehRoPumxRqG60GvoLEqSRMEWkh1Qp2OIOyhITEhjkki2HoMjmlizXZiZVCqzUuG0acXCv9la1chEjXCN/InpBlT8T+pclPBNg6+SMfoYVLw7g4xJ+F5F3Fox7f5EMLEZ9glvRSYFhImxqdm+z2CGzPcK1zjH8w1MgjfrCmVuZHN0cmVhbQplbmRvYmoKMzUgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzMzQgPj4Kc3RyZWFtCnicLVJLcsUgDNtzCl2gM/gH5DzpdLp4vf+2kpNFRg5g9DHlholKfFkgt6PWxLeNzECF4a+rzIXPSNvIOojLkIu4ki2Fe0Qs5DHEPMSC76vxHh75rMzJswfGL9l3Dyv21IRlIePFGdphFcdhFeRYsHUhqnt4U6TDqSTY44v/PsVzLQQtfEbQgF/kn6+O4PmSFmn3mG3TrnqwTDuqpLAcbE9zXiZfWme5Oh7PB8n2rtgRUrsCFIW5M85z4SjTVka0FnY2SGpcbG+O/VhK0IVuXEaKI5CfqSI8oKTJzCYK4o+cHnIqA2Hqmq50chtVcaeezDWbi7czSWbrvkixmcJ5XTiz/gxTZrV5J89yotSpCO+xZ0vQ0Dmunr2WWWh0mxO8pITPxk5PTr5XM+shORUJqWJaV8FpFJliCdsSX1NRU5p6Gf778u7xO37+ASxzfHMKZW5kc3RyZWFtCmVuZG9iagozNiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDMyMCA+PgpzdHJlYW0KeJw1UktuBTEI288puECl8E/O86qqi777b2sTvRVMMGDjKS9Z0ku+1CXbpcPkWx/3JbFC3o/tmsxSxfcWsxTPLa9HzxG3LQoEURM9WJkvFSLUz/ToOqhwSp+BVwi3FBu8g0kAg2r4Bx6lMyBQ50DGu2IyUgOCJNhzaXEIiXImiX+kvJ7fJ62kofQ9WZnL35NLpdAdTU7oAcXKxUmgXUn5oJmYSkSSl+t9sUL0hsCSPD5HMcmA7DaJbaIFJucepSXMxBQ6sMcCvGaa1VXoYMIehymMVwuzqB5s8lsTlaQdreMZ2TDeyzBTYqHhsAXU5mJlgu7l4zWvwojtUZNdw3Duls13CNFo/hsWyuBjFZKAR6exEg1pOMCIwJ5eOMVe8xM5DsCIY52aLAxjaCaneo6JwNCes6VhxsceWvXzD1TpfIcKZW5kc3RyZWFtCmVuZG9iagozNyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE4ID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iagozOCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEzMyA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjM5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzQwID4+CnN0cmVhbQp4nDVSOW4EMQzr/Qp9IIBu2+/ZIEiR/L8NqdkUA3F0UpQ7WlR2y4eFVLXsdPm0ldoSN+R3ZYXECcmrEu1ShkiovFYh1e+ZMq+3NWcEyFKlwuSk5HHJgj/DpacLx/m2sa/lyB2PHlgVI6FEwDLFxOgals7usGZbfpZpwI94hJwr1i3HWAVSG9047Yr3oXktsgaIvZmWigodVokWfkHxoEeNffYYVFgg0e0cSXCMiVCRgHaB2kgMOXssdlEf9DMoMRPo2htF3EGBJZKYOcW6dPTf+NCxoP7YjDe/OirpW1pZY9I+G+2Uxiwy6XpY9HTz1seDCzTvovzn1QwSNGWNksYHrdo5hqKZUVZ4t0OTDc0xxyHzDp7DGQlK+jwUv48lEx2UyN8ODaF/Xx6jjJw23gLmoj9tFQcO4rPDXrmBFUoXa5L3AalM6IHp/6/xtb7X1x8d7YDGCmVuZHN0cmVhbQplbmRvYmoKNDAgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNTEgPj4Kc3RyZWFtCnicLVFJcgNBCLvPK/SEZqffY5crh+T/1wjKBwYNi0B0WuKgjJ8gLFe85ZGraMPfMzGC3wWHfivXbVjkQFQgSWNQNaF28Xr0HthxmAnMk9awDGasD/yMKdzoxeExGWe312XUEOxdrz2ZQcmsXMQlExdM1WEjZw4/mTIutHM9NyDnRliXYZBuVhozEo40hUghhaqbpM4EQRKMrkaNNnIU+6Uvj3SGVY2oMexzLW1fz004a9DsWKzy5JQeXXEuJxcvrBz09TYDF1FprPJASMD9bg/1c7KT33hL584W0+N7zcnywlRgxZvXbkA21eLfvIjj+4yv5+f5/ANfYFuICmVuZHN0cmVhbQplbmRvYmoKNDEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNzQgPj4Kc3RyZWFtCnicTZBJDkMhDEP3nMIXqIQzwOc8v6q6aO+/rUMHdYH85CBwPDzQcSQudGTojI4rmxzjwLMgY+LROP/JuD7EMUHdoi1Yl3bH2cwSc8IyMQK2RsnZPKLAD8dcCBJklx++wCAiXY/5VvNZk/TPtzvdj7q0Zl89osCJ7AjFsAFXgP26x4FLwvle0+SXKiVjE4fygeoiUjY7oRC1VOxyqoqz3ZsrcBX0/NFD7u0FtSM83wplbmRzdHJlYW0KZW5kb2JqCjQyIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNzUgPj4Kc3RyZWFtCnicM7U0UjBQMDYAEqZmRgqmJuYKKYZcQD6IlctlaGQKZuVwGVmaKVhYABkmZuZQIZiGHC5jU3OgAUBFxqZgGqo/hyuDKw0AlZAS7wplbmRzdHJlYW0KZW5kb2JqCjQzIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggODkgPj4Kc3RyZWFtCnicNU25EYAwDOs9hUfAj0i8D8dRhP1b7IQ0lk6fEcoHa+QBguGNLyH4oi8ZhLULDyr7SHTYRA1nFSQTw68s8KqcFW1zJRPZWUyjs0HL9K3tb4Meuj/djhwKCmVuZHN0cmVhbQplbmRvYmoKNDQgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNDEgPj4Kc3RyZWFtCnicPY/BDsMwCEPv+Qr/QKTYKaF8T6dqh+7/ryNLuwt6AmOMhdDQG6qaw4Zgm+PF0iVUa/gUxUAlN8iZYA6lpNIdR5F6YjgYXB60G47isej6EbuSZn3QxkK6JWiAe6xTadymcRPEHTUF6inqnKO8ELmfqWfYNJLdNLOSc7gNv3vPU9f/p6u8y/kFvXcu/gplbmRzdHJlYW0KZW5kb2JqCjQ1IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjE1ID4+CnN0cmVhbQp4nDVROQ4DIQzs9xX+QCSML3hPoijN/r/NjNFWHsFchrSUIZnyUpOoIeVTPnqZLpy63NfMajTnlrQtc4C4trwvrZLAiWaIg8FpmLgBmjwBQ9fRqFFDFx7Q1KVTKLDcBD6Kt24P3WO1gZe2IeeJIGIoGSxBzalFExZtzyekNb9eixvel+3dyFOlxpYYgQYBVjgc1+jX8JU9TybRdBUy1Ks1yxgJE0UiPPmOptUT61o00jIS1MYRrGoDvDv9ME4AABNxywJkn0qUs+TEb7H0swZX+v4Bn0dUlgplbmRzdHJlYW0KZW5kb2JqCjE1IDAgb2JqCjw8IC9CYXNlRm9udCAvRGVqYVZ1U2FucyAvQ2hhclByb2NzIDE2IDAgUgovRW5jb2RpbmcgPDwKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDQ2IC9wZXJpb2QgNDggL3plcm8gL29uZSAvdHdvIC90aHJlZSAvZm91ciAvZml2ZSAvc2l4IDU2Ci9laWdodCA2OCAvRCA4MCAvUCA5NyAvYSAvYiAvYyAvZCAvZSAxMDMgL2cgMTA1IC9pIDEwOCAvbCAxMTAgL24gL28gMTE0IC9yCi9zIC90IC91IC92IDEyMCAveCAveSBdCi9UeXBlIC9FbmNvZGluZyA+PgovRmlyc3RDaGFyIDAgL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvRm9udERlc2NyaXB0b3IgMTQgMCBSCi9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdIC9MYXN0Q2hhciAyNTUgL05hbWUgL0RlamFWdVNhbnMKL1N1YnR5cGUgL1R5cGUzIC9UeXBlIC9Gb250IC9XaWR0aHMgMTMgMCBSID4+CmVuZG9iagoxNCAwIG9iago8PCAvQXNjZW50IDkyOSAvQ2FwSGVpZ2h0IDAgL0Rlc2NlbnQgLTIzNiAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvRm9udE5hbWUgL0RlamFWdVNhbnMgL0l0YWxpY0FuZ2xlIDAKL01heFdpZHRoIDEzNDIgL1N0ZW1WIDAgL1R5cGUgL0ZvbnREZXNjcmlwdG9yIC9YSGVpZ2h0IDAgPj4KZW5kb2JqCjEzIDAgb2JqClsgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAKNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCAzMTggNDAxIDQ2MCA4MzggNjM2Cjk1MCA3ODAgMjc1IDM5MCAzOTAgNTAwIDgzOCAzMTggMzYxIDMxOCAzMzcgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNgo2MzYgNjM2IDMzNyAzMzcgODM4IDgzOCA4MzggNTMxIDEwMDAgNjg0IDY4NiA2OTggNzcwIDYzMiA1NzUgNzc1IDc1MiAyOTUKMjk1IDY1NiA1NTcgODYzIDc0OCA3ODcgNjAzIDc4NyA2OTUgNjM1IDYxMSA3MzIgNjg0IDk4OSA2ODUgNjExIDY4NSAzOTAgMzM3CjM5MCA4MzggNTAwIDUwMCA2MTMgNjM1IDU1MCA2MzUgNjE1IDM1MiA2MzUgNjM0IDI3OCAyNzggNTc5IDI3OCA5NzQgNjM0IDYxMgo2MzUgNjM1IDQxMSA1MjEgMzkyIDYzNCA1OTIgODE4IDU5MiA1OTIgNTI1IDYzNiAzMzcgNjM2IDgzOCA2MDAgNjM2IDYwMCAzMTgKMzUyIDUxOCAxMDAwIDUwMCA1MDAgNTAwIDEzNDIgNjM1IDQwMCAxMDcwIDYwMCA2ODUgNjAwIDYwMCAzMTggMzE4IDUxOCA1MTgKNTkwIDUwMCAxMDAwIDUwMCAxMDAwIDUyMSA0MDAgMTAyMyA2MDAgNTI1IDYxMSAzMTggNDAxIDYzNiA2MzYgNjM2IDYzNiAzMzcKNTAwIDUwMCAxMDAwIDQ3MSA2MTIgODM4IDM2MSAxMDAwIDUwMCA1MDAgODM4IDQwMSA0MDEgNTAwIDYzNiA2MzYgMzE4IDUwMAo0MDEgNDcxIDYxMiA5NjkgOTY5IDk2OSA1MzEgNjg0IDY4NCA2ODQgNjg0IDY4NCA2ODQgOTc0IDY5OCA2MzIgNjMyIDYzMiA2MzIKMjk1IDI5NSAyOTUgMjk1IDc3NSA3NDggNzg3IDc4NyA3ODcgNzg3IDc4NyA4MzggNzg3IDczMiA3MzIgNzMyIDczMiA2MTEgNjA1CjYzMCA2MTMgNjEzIDYxMyA2MTMgNjEzIDYxMyA5ODIgNTUwIDYxNSA2MTUgNjE1IDYxNSAyNzggMjc4IDI3OCAyNzggNjEyIDYzNAo2MTIgNjEyIDYxMiA2MTIgNjEyIDgzOCA2MTIgNjM0IDYzNCA2MzQgNjM0IDU5MiA2MzUgNTkyIF0KZW5kb2JqCjE2IDAgb2JqCjw8IC9EIDE3IDAgUiAvUCAxOCAwIFIgL2EgMTkgMCBSIC9iIDIwIDAgUiAvYyAyMSAwIFIgL2QgMjIgMCBSIC9lIDIzIDAgUgovZWlnaHQgMjQgMCBSIC9maXZlIDI1IDAgUiAvZm91ciAyNiAwIFIgL2cgMjcgMCBSIC9pIDI4IDAgUiAvbCAyOSAwIFIKL24gMzAgMCBSIC9vIDMxIDAgUiAvb25lIDMyIDAgUiAvcGVyaW9kIDMzIDAgUiAvciAzNCAwIFIgL3MgMzUgMCBSCi9zaXggMzYgMCBSIC9zcGFjZSAzNyAwIFIgL3QgMzggMCBSIC90aHJlZSAzOSAwIFIgL3R3byA0MCAwIFIgL3UgNDEgMCBSCi92IDQyIDAgUiAveCA0MyAwIFIgL3kgNDQgMCBSIC96ZXJvIDQ1IDAgUiA+PgplbmRvYmoKMyAwIG9iago8PCAvRjEgMTUgMCBSID4+CmVuZG9iago0IDAgb2JqCjw8IC9BMSA8PCAvQ0EgMCAvVHlwZSAvRXh0R1N0YXRlIC9jYSAxID4+Ci9BMiA8PCAvQ0EgMSAvVHlwZSAvRXh0R1N0YXRlIC9jYSAxID4+Ci9BMyA8PCAvQ0EgMSAvVHlwZSAvRXh0R1N0YXRlIC9jYSAwLjUgPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgPj4KZW5kb2JqCjIgMCBvYmoKPDwgL0NvdW50IDEgL0tpZHMgWyAxMSAwIFIgXSAvVHlwZSAvUGFnZXMgPj4KZW5kb2JqCjQ2IDAgb2JqCjw8IC9DcmVhdGlvbkRhdGUgKEQ6MjAyMTA5MTYxNDQzMjgrMDInMDAnKQovQ3JlYXRvciAoTWF0cGxvdGxpYiB2My40LjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcpCi9Qcm9kdWNlciAoTWF0cGxvdGxpYiBwZGYgYmFja2VuZCB2My40LjMpID4+CmVuZG9iagp4cmVmCjAgNDcKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMTUzNDkgMDAwMDAgbiAKMDAwMDAxNTExNCAwMDAwMCBuIAowMDAwMDE1MTQ2IDAwMDAwIG4gCjAwMDAwMTUyODYgMDAwMDAgbiAKMDAwMDAxNTMwNyAwMDAwMCBuIAowMDAwMDE1MzI4IDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM5OSAwMDAwMCBuIAowMDAwMDA1MzcxIDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwNTM1MCAwMDAwMCBuIAowMDAwMDEzNzE3IDAwMDAwIG4gCjAwMDAwMTM1MTcgMDAwMDAgbiAKMDAwMDAxMzA2MCAwMDAwMCBuIAowMDAwMDE0NzcwIDAwMDAwIG4gCjAwMDAwMDUzOTEgMDAwMDAgbiAKMDAwMDAwNTYyOCAwMDAwMCBuIAowMDAwMDA1ODcxIDAwMDAwIG4gCjAwMDAwMDYyNTEgMDAwMDAgbiAKMDAwMDAwNjU2OCAwMDAwMCBuIAowMDAwMDA2ODczIDAwMDAwIG4gCjAwMDAwMDcxNzcgMDAwMDAgbiAKMDAwMDAwNzQ5OSAwMDAwMCBuIAowMDAwMDA3OTY3IDAwMDAwIG4gCjAwMDAwMDgyODkgMDAwMDAgbiAKMDAwMDAwODQ1NSAwMDAwMCBuIAowMDAwMDA4ODY5IDAwMDAwIG4gCjAwMDAwMDkwMTMgMDAwMDAgbiAKMDAwMDAwOTEzMiAwMDAwMCBuIAowMDAwMDA5MzY4IDAwMDAwIG4gCjAwMDAwMDk2NTkgMDAwMDAgbiAKMDAwMDAwOTgxNCAwMDAwMCBuIAowMDAwMDA5OTM3IDAwMDAwIG4gCjAwMDAwMTAxNzAgMDAwMDAgbiAKMDAwMDAxMDU3NyAwMDAwMCBuIAowMDAwMDEwOTcwIDAwMDAwIG4gCjAwMDAwMTEwNjAgMDAwMDAgbiAKMDAwMDAxMTI2NiAwMDAwMCBuIAowMDAwMDExNjc5IDAwMDAwIG4gCjAwMDAwMTIwMDMgMDAwMDAgbiAKMDAwMDAxMjI1MCAwMDAwMCBuIAowMDAwMDEyMzk3IDAwMDAwIG4gCjAwMDAwMTI1NTggMDAwMDAgbiAKMDAwMDAxMjc3MiAwMDAwMCBuIAowMDAwMDE1NDA5IDAwMDAwIG4gCnRyYWlsZXIKPDwgL0luZm8gNDYgMCBSIC9Sb290IDEgMCBSIC9TaXplIDQ3ID4+CnN0YXJ0eHJlZgoxNTU2NgolJUVPRgo=\n", "image/svg+xml": ["\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2021-09-16T14:43:28.378484\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.4.3, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n"], "text/plain": ["
"]}, "metadata": {}, "output_type": "display_data"}], "source": ["mu = torch.Tensor([128])\n", "sigma = torch.Tensor([2.0])\n", "\n", "\n", "def discrete_logistic(x, mu, sigma):\n", " return torch.sigmoid((x + 0.5 - mu) / sigma) - torch.sigmoid((x - 0.5 - mu) / sigma)\n", "\n", "\n", "x = torch.arange(256)\n", "p = discrete_logistic(x, mu, sigma)\n", "\n", "# Visualization\n", "plt.figure(figsize=(6, 3))\n", "plt.bar(x.numpy(), p.numpy(), **plot_args)\n", "plt.xlim(96, 160)\n", "plt.title(\"Discrete logistic distribution\")\n", "plt.xlabel(\"Pixel value\")\n", "plt.ylabel(\"Probability\")\n", "plt.show()\n", "plt.close()"]}, {"cell_type": "markdown", "id": "0ee82ec6", "metadata": {"papermill": {"duration": 0.072594, "end_time": "2021-09-16T12:43:29.091439", "exception": false, "start_time": "2021-09-16T12:43:29.018845", "status": "completed"}, "tags": []}, "source": ["Instead of the softmax, the model would output mean and standard\n", "deviations for the $K$ logistics we use in the mixture. This is one of\n", "the improvements in autoregressive models that PixelCNN++ [3] has\n", "introduced compared to the original PixelCNN."]}, {"cell_type": "markdown", "id": "d30f7d84", "metadata": {"papermill": {"duration": 0.071033, "end_time": "2021-09-16T12:43:29.238966", "exception": false, "start_time": "2021-09-16T12:43:29.167933", "status": "completed"}, "tags": []}, "source": ["## Conclusion\n", "\n", "In this tutorial, we have looked at autoregressive image modeling, and\n", "implemented the PixelCNN architecture. With the usage of masked\n", "convolutions, we are able to apply a convolutional network in which a\n", "pixel is only influenced by all its predecessors. Separating the masked\n", "convolution into a horizontal and vertical stack allowed us to remove\n", "the known blind spot on the right upper row of a pixel. In experiments,\n", "autoregressive models outperformed normalizing flows in terms of bits\n", "per dimension, but are much slower to sample from. Improvements, that we\n", "have not implemented ourselves here, are discrete logistic mixtures, a\n", "downsampling architecture, and changing the pixel order in a diagonal\n", "fashion (see PixelSNAIL). Overall, autoregressive models are another,\n", "strong family of generative models, which however are mostly used in\n", "sequence tasks because of their linear scaling in sampling time than\n", "quadratic as on images."]}, {"cell_type": "markdown", "id": "38f1dd4f", "metadata": {"papermill": {"duration": 0.071797, "end_time": "2021-09-16T12:43:29.381584", "exception": false, "start_time": "2021-09-16T12:43:29.309787", "status": "completed"}, "tags": []}, "source": ["## References\n", "[1] van den Oord, A., et al.\n", "\"Pixel Recurrent Neural Networks.\"\n", "arXiv preprint arXiv:1601.06759 (2016).\n", "[Link](https://arxiv.org/abs/1601.06759)\n", "\n", "[2] van den Oord, A., et al.\n", "\"Conditional Image Generation with PixelCNN Decoders.\"\n", "In Advances in Neural Information Processing Systems 29, pp.\n", "4790\u20134798 (2016).\n", "[Link](http://papers.nips.cc/paper/6527-conditional-image-generation-with-pixelcnn-decoders.pdf)\n", "\n", "[3] Salimans, Tim, et al.\n", "\"PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications.\"\n", "arXiv preprint arXiv:1701.05517 (2017).\n", "[Link](https://arxiv.org/abs/1701.05517)"]}, {"cell_type": "markdown", "id": "c99bfefb", "metadata": {"papermill": {"duration": 0.074235, "end_time": "2021-09-16T12:43:29.526938", "exception": false, "start_time": "2021-09-16T12:43:29.452703", "status": "completed"}, "tags": []}, "source": ["## Congratulations - Time to Join the Community!\n", "\n", "Congratulations on completing this notebook tutorial! If you enjoyed this and would like to join the Lightning\n", "movement, you can do so in the following ways!\n", "\n", "### Star [Lightning](https://github.com/PyTorchLightning/pytorch-lightning) on GitHub\n", "The easiest way to help our community is just by starring the GitHub repos! This helps raise awareness of the cool\n", "tools we're building.\n", "\n", "### Join our [Slack](https://join.slack.com/t/pytorch-lightning/shared_invite/zt-pw5v393p-qRaDgEk24~EjiZNBpSQFgQ)!\n", "The best way to keep up to date on the latest advancements is to join our community! Make sure to introduce yourself\n", "and share your interests in `#general` channel\n", "\n", "\n", "### Contributions !\n", "The best way to contribute to our community is to become a code contributor! At any time you can go to\n", "[Lightning](https://github.com/PyTorchLightning/pytorch-lightning) or [Bolt](https://github.com/PyTorchLightning/lightning-bolts)\n", "GitHub Issues page and filter for \"good first issue\".\n", "\n", "* [Lightning good first issue](https://github.com/PyTorchLightning/pytorch-lightning/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22)\n", "* [Bolt good first issue](https://github.com/PyTorchLightning/lightning-bolts/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22)\n", "* You can also contribute your own notebooks with useful examples !\n", "\n", "### Great thanks from the entire Pytorch Lightning Team for your interest !\n", "\n", "![Pytorch Lightning](data:image/png;base64,H4sIAAAAAAACA9ycyZKjyNqm93UV0bmlSeaprPJYCyQ0IAkECIQ2xxicScyj4MZ631fWKCKzKior8++qJvQvTphFhAbnlfujz7/BAf/ll8fPb/9jKQu6paxewiZN/vXLb49/L4mdBV8+gezTi2c3NuzmSV7Bae6BL5/stsm/vpxEQdjATQjS6fXXJ1/f8Ozq9u31x+NP//rl5eW3ENje48H0MAWN/eKGdlWD5suntvFh9q1NEmW3lwok03FZDRcV8EHjhp9ewunRl09h0xT1rwgSRE3YOp/f/tn1JFJ/dvP0n0jY3dTRqv6q0dagcvOsAVnzT4XeBGA3yVvvc018tlN7zDO7/8c9evQBjlI7AD/pFfK92qQ0vZ0Bt/m7gF7cKq/rvIqCKPvbYv8vVNPPN6V38pOlZHk2pHlbf3pJgRfZ0ytJ8uklmo4NqqgZvnyqQ5vCcNiXx8NWJdSTH906wtnTZd0KCI4F29JGk8vGjWQNkitdkNex0KZMhvSieSwLYWgWWMmZLbRsSkjZ3zVA46q+XSBpdzx9+fLpbWR1MySgDgH4u5iQt4eIX9kp6PPqVsOMC1zXxTHM8zAS81mbdHyGdh1g2w4OUOezW0/DRL6a9wwWMUGcj97O3TW3nU9owJeyrawf2up2zA/4YnfQeNs7G2YjhgpBu/uDn9BSs6Lp5RpSBSjz2Ei/6xy7rTdR1soXY8MevH4+CweEdhflVQ2zPuN5JEF5OOXitMehHOMzJAcY1+ZcluD+hOK7P3PIBJHHMAsTv0c34QRWGdJsq7TeC8nWxb1LpXgLhowcvVzjhwCctqeVme2tzVG7onVFnE5kcr0dUdwduCSPwkAfh2N3uX8Ama8OgMVxhnGAi1IO7dI4h+IkBjCc8YGH4w5n/47lMVlqt4qK5qcgvMlDVJO3ePz7EYoj8DOLlRlhzRy7QD/J5yWn007vHO8BNx4tXTQx/RZltLppz/q16xiNNCoO0xeNEfKWO/hifR7posmKw11VmoVOss0rimYoHj6+KJLItZsoz5B4mv5v3f30Ulfu38YCsi6q8iyd/ARMUIABDE18jutP//oNeZP7Oldmolg6SVlQq/rg7YyD194JbGfv617Y14BIr9XF1bt1z09tTbDDufWSQBrF84f7itMXldcBVb2G2UndQNxVdXyeWt4QjT59LAo3bLMb/M6boJ7D0bbNPIGHrZ16NPUGIk63NX4vlyil7RPnDLVXM2zoneS3F1xJJZe8mGlP7NpBI52twSuF44pLtBCXS5fL5aqk4w5V8bO7btxN8AweHci8KaugOZwkUA//C4sPmCc4F8ThGu3jkJST7rjyd+VtWbirTOnkLC+I43q3Gpv2ZLNHbx+YeOKdnJRaczWISrpQgtBIvcWBuJADvz6wq71u3Cvtg2H84Vg9x2NxgiV/RGI+C/68dKrL1ZG4U16dKx+V8pVzXexCPRshvjykGOqUV0Gie2rTD4xly+qJFMnlJdzfFobe2bvwGN7v4BKkwW6voDrP6PLf8BmvCeGUOrYJgCPvy6fPX79+2+vszAXeY7Rvjf65CX3TgFGKnOaU7T1hQlGNKWkVk4aLUQtJ7U5ZqhhIfuOt71K/kTJ5bfXUMrnHm/B2Ms6qJZ4J/kZtD/L6ILWLaKcnAiFfrANxH9fuEI/tSjz9DQfzU27ZlCM2AM4zeGqaJ8ksgN+LwYD2qCm7sZ9AskYVZXJDFxkTzCW6VBQDVTjCdiplYSWIKOvbzk9DSJCRi5ArxMpTOG7ICHHfHRtyf9gyu9Tk/XVkcztRuRZVfmYX1QwLdKd6Jo2qKq/mIPxDBXYIknB9/xnsnBCMLc0PA8Nsh9Aq8hNKM0TAXvOy4nmUIBzI3g3cyEQ4CV0Y7YDcepKEIPm01lRzGEvkdrrR7A3KVG5TnV31wp70v+HJfs7uW0VYz4P3uwxMA4xxCMd5Aj0G21xbTKd2ZqkeRa6vUvx4uS5Ky+jG+5VddnlFnakE2bcr3lkL/IVitGygUTWGyvP1uuhW/FpMjvWIMdxyleEbsQ+MxRx62VQCNtE8dG8aMPApzOOewk3BuTOaHNPwHBMUYVSCWdDu6IXH4tCIDHo3JJ9cT2gMScjLdXMv0HroFwcWIWkoEtCbJ0Zqd8CIFd8ts5LOB1Pto1kzNmuqyGkfB9RwXUTeVDIHlV2EM0n+RBUmfNqnyB8kJ/PZdkWD616Do40sqU2lmo1s2GqE93y+sP3kuD6r9L09BWcEtXr/3jn2WKICHoZS30YC69ALFiOCyzqFWD9Gj8Iqrx1rRlxx82KYR7EYYAdwDkOTz4jDduUnB9E1RmNh2QqeMSkT3BJFCM3aqIVloyp2hwsO7ZR7sPPZInDy2NwRqxpIl+1e2Jb+PUE62q54r71y2j4tDqvrDFv0QGIPU+qR5LYXZQEMEvCodeYg/IkkTNsOAygCfQJVOkZyTYRujgkNoDkq3ZZmSWNctFwxtgFtxHgIES2294RWDVUyztpoc/XBSUpqIru7OqIRGyjfNozYRMEKsUEeQ8GMuOJVdjD9yYtZHL+JwMAmfMCS7BPIGZrmQdYY4dFBg4rIsehwiZjeUNwXKy93KBFXD9K9ldzFRsNXhWjtFKBfcuPeoMdLgBKNH+PezcbTVbPRoC4uLM1ezrBH4EUNHOb5Da6BW4HmIwzyZ5owReEk5/ncE7iuqvIqJpNFjVtnn183Xe+X3DXOqTGHLGxBYmgGlOW6kyDT0A0FQ6Oj7uelJ8va9agNWNDspa253G4j2SDQpKLH806YEasfDGYzxHDH5jiMfoZfjMrDdr2+rnN2cTrEbr5NJeWg11fDUBM6bATNYvhCTYu1skYWImSWkILIMeI1oXmStzfOFY4HZWXXjIyPC5XOywxt5vBK8ziCi8i9TVH0I2zwB3qTR8Rt1sWf4RHLU7lOOAOTk7B0q4uoiqRzb04dxVR+6KTKQuba/eLSWYudZp2IqGNXTrDWK7zsJ69oEqlGWsgaNR2CyMotr6yzLXmeEZd9O0qcfBbCrxKwzaE2S3PPWHaqB1Tix4TYVC3oobzaYciKdIf2HN0bA201lNC0TWGpY7gAhQniNhiUmlj0m+IErVJqYx98DzsoI3IgofvKFp2TSwUzrNAHdtNWAHbtJMnbD3GGP5Gcqj7cQ230GVRDdq+We9/rPAlHbpoUsztroOOB9xJ96O97Mil7g5YN2SM0QVGD845EKkLaiKer0lo9umYWphnuLGSbR4217xamCKg5thglzeMkVVa081C+04FZxsU9zH5GjI74273ak5VPSpkEZC61jXsX3kRO93pFOi9RrNTw7YiE2g2J9zjXI41zrHhkLWJUYYbLprGvnjey9AKxw2t143eMvpxhlcEUQitQ5B9hjt9rwSyLcQRmu0/gyIuYf3XDpj0JK0zPi3Spaq686xRX2hyYmPe3w96uR0kO3dWOF2nfE+zbDU2j0ZExahAO0eUGkcZ2qR/GYJ1ytIzshBl2GGX142xzDZctqGZVLX9WglGS8iifpp5xoiLiyMEV66PT9LRFjXjtbijkUm2QlBV2ZUst1MU1lqsTHflMq+3D+5E9nPl86y2RqtZD7M7wPdL640nId4V19cbVjp3FsIua1+azVnDeycAowH2GdZ+RFXKccYw2K2tvIt41cc3t9WofFxdraY7a3T22Q7UgTrVfF8iFCLc3NhfEIYgSos28usZodc2XkJaqEpax1C6wqTTmr9iMmRy3aQE3+RxyXyVgn/Eo0mOfUqM0nt0fEFmrW8ijO5y6WvTZPdnIer1faI220uK69Fb9HQIDy66P1vp2wskc2oliTiYWxPT7g+oztlDt77cFpZUtnx9n2Fxit5kbvi6ZfoQL/IEcTNEeQ9nuMzJDVaXON0hacQdEqMu9mwYqEEKNuS7jlA3urpwtLdO8HTyg11VhIvdlstucVsHZb1cCvdld/EE5uiaU8/qmPGStO/HW59CMsikbCe1qHsTfVWCSwgBFcs84TVsYtb/wTmEa3ZBlMV7rUsdzRQrprbxsOfVmub3e+inEblFDZ0ZcNZo4XlfQ+TiW2MI8bPTqpJ1ZbMkOtKx7uj+ihTJj/qagqSJ3KmsT4D4O+ghz/JkmbFOUg/ko9oyV7fCKElgWRCJB9+vad/NhEEy97K17fIFKoO8bwWCEHbSVoX68jUKKqblKmKPpjr6Z9vtjWUGblrid5ODG2+uMzBpzhk1meRP5Xw+YTKtuYD9321kR5ieSMAAYzZCe/wSq66yJ177cHy718VGI3NNRJ1DaE2hiL6GsXFKbympHHlRbE5cRHhnZcuPEy20qnnnyOrbsrZMM7H7m+PTMCPZ+D45gxqpYUYE0alO4arMMVLNoficFYzbj0AT2jJjdX5IOOR1Iub5Yw7nVfDG+E5dSSzt/XzZxx9/pXkzWx3QzevTyllbY5qZfa6wsBEsX4mjosTGlZNUmZTIwF5ILuZIyY4VsGnoNMhfA3y5om4fxT1qwizEc7fvPWPlObVSWByKLURdzS/6O8jc33Yr1vnS1K3TYtrfTfViJU5w39fFarKHTIZBzU1lsriNJHGh/iBfnNRZvaBeh90oUSYowzOKYTwUcgIspef4Iv/kjvSnhsz0UEM84qxUGuSbt9WTPbomtWSH6Xj4KZ+hI3jyh8aqNUFWtRx6lMFCTfaX36OWshCVvL73rORK1cXlSeyQTT9itxw2gp2jAQcmMWPR1/B+BkKVRG8PxZ+SSquwUVd+JKh72ayVfQtZKWPDn1Cx753yXiMO5d1RLbp2Oijjs6sZrG4yrUlyyyvVK7/B7appXh0MiSLTMgSC8yoeCGZGmaJN6mn5tEz6unXk7XfcRxvhzWZgkAQ04+xkR5yiMuqrp3V2IWhe9VyDARgznq6QhfI2DzCaz1mfXWy2XxcWJETSqDDek9YpW48kh+IclsHdxT9zO/qrOtM1JO+WXcUbEqYDtpWAqT9yPgPpXNZggcdcjyKdU2uPoqaGeD1qr3jc3K9G3JunvLwQyWLSRGLcAAhU+SEN21pxoX+pqHEFNgnNnMfJdUTOcm3lsyjOkjDvKOS1Iww4Os1j6X5PCeVeavNeBUZ9wPY98xhkF3yIsixVp3TQ3mGllVFuL1NlkN/LIN1bskuesWVGLSORW/Ha7Ellvx4B0rB256WQ26RSBiqsjI1mcgB5jeSHeDHsxI95MGUveTDVKEhVOblfe7FPVPxSEGY/1PPYpF6lumWjLofgSW9A126bCEg8dkczRUDKxpCqOvrrDN0zb0753MFSE3XlNhnEOsUaWy61OJsVFPuv59aKnjjUFnoU15aT2LIusizyrow7AbeaB6lEDZnY3j+kPJWGccFiW9Z+xOq5NYYTWcHZXQ4g11p0VpAW0XXkCWjYHl/fp2gov+sqNMVtUhL0i63ImS+UoRsdhg+dHY+lBLXWKWFBa26O2bXF2L86ISG8p9RQz8raYlVz+SQgmOdp1GPCMDF0zPAmVaE7JTqQohF6R0BQqq1BXJJZ5KT3vfryt1lJ48unmBsKyk27iKqD2J+0aCIbU4I3vckJm7pJUEnpH16NiFc+wyxpUEajhxnbmpUPvdSZ+FENOyeQzViUXpX48rdBaOHJIW+Do2Pmee2x2tGgZbblWdDoufINVO0pTbe1SMZt7tRbQy+riXlG7SwS/tXZHlI8vEnPVOdQy0N0sfh14dPBxjsptE/sRKD4gfP8XsrBPojZHEM+I46GvgzE6rsIIq7YlQfKurEku5ldouyrPoeQYUrhDRIfsDt3ZI8OgVXTWPuqutO4cNK36tO54pjAMkO6OrLpV3Tw0ZsShaQxvFvXtku1ZTL8Xg1nKJz3ggieQbEDkVxsoofJtVmcx12bbcap/Lv2aH8+XeIFVtXA1kvNuu2sM3GnHKzk22kjYKh2up/juNAdlHO8ANKpDhl6b7v0rNKPiqRvbvQEPtqfEcP4K5l/VYIcCLOc/5VwY2maM5Wvk8sAzVy5VefR+2+/jo3hJDqaZefGIjP3V2J33dmny3R7BRstU9ukqk0k5xynpmq0J3+Uvd2TJqAsu2Y3dckbUeR19/fVc6uOi0CiAuwj0s4n+QBP2cMD5Dv2Mlcxdcdyt2eJ+Pm9wYkzjWMlXF6jzdisnqrQM3V8lJ7+emtsJNAFtiUTAa4YULQX84jQub2PMSbph10OJImvG2cQr3HfbOb70jcFUWQdTblN/FNM/6cE4zREkhz1jlSOTbOLsLXCZMQC5y3AysHkcHc7+rtOYPsk89sQKUGQu2QhvLvvAlkC6NG97AO7g5FU1ZGccKwoJ5cvRgO6gaMygZoadNmEFwOe3N+ZwfK8Dc65NOwz1DHtky0aXEyPi013RJsWOEBCIWTgEoTEHhWoWq62/ue6ZHZlCRBQ0xnqxIitmCAas4tUYcd01u22xQpC2Urfw15YxakUyh18065rI6XDYx22HIohn3J9gSqxx6YnEP58QQzXRMQkkJXQP53KDtnrGI9IIlLPiXUMhRTbF3b9cNpK+7mJKQGVwZ6DR2R7wlTCe906jCzrZBsQwY/Y2uV3PCiqvAjDF2j5GUc9Y8TloqsZqVpCJ3UnbSrFSskRTk+OpbDTGkq9TRrgkLz6qgQtPbpRmBDVqgeVCPAHhVtxbtoAEV8bPzUBWi0GxrfEkrOYQ66fwXude1KazuP0hAxMYTlK4/4zZ6XeuLFt2IdyV21I9mSNunsaq3NOVGl5YdtiYIsv4DnfJBlwsyaSKUPloG+KeuSw1PdKHlpT1AmLNfPAPRhGsz7ZtzKDXZh2IZt2K9aYAM8BnUEA8o9pr/GXetX5cFsuDcknLc93a9vEoy4h3iVhbFfST1gq7Vidxi79DZnvrAJlANo3yIdNzVnZgz+NZrBbKHrEgKI4Pi+WciPC64cEUFpu2huvWSeddZ/tXtSkL9FHAOs/IqF2pRaALE+iQaWzI+iI1aECsGcbxT74b9G2lys4B8hr2MiRKyoGVuYUQ25RCw3Zoo7barmu3FK9XkXQJL/6un6pDbwbLHjhBAvd2NStI/KECM6jtOP4PrvD+gNtyN1vqKrAUdL8BEB6DRq6ZPe0q1EYkXCyClFUodXRTHblDahNOes9psUgGylGCbBky6WINePsuCy1vHQ66xzNx50V/J4X+J0Ael9bV0VTpPhYSMJcFFId+n6bNLiXANQVcOmp8jq7qgTxuQc9qUm0BRhukxmR3bR+NWxJQcbPQ+kL3T0hm1ZsrD6G6rB01x4gZs11qneUuoWUMdtap+eDbk73I92vYwwBN08z3AH75tpVLZj+2enmktMVU1356+bpByJdPfeQ14RcPdJEL4Ncnn74ZURM1CfhXMUyM3fBtL5ksygIkyYP8c5EFL3bzkk5BGFQv/+d/vyiD/mi3/73ZXw58tFpHzaZ1fkPetN/tNfPWQQ+89XwC8q6PegheXlV68Pj77aNe+moCOH26n1cv4fQOPD2ZHqePW45fFtuXxylge2r4+UVz7QS8DHlbvTzuZUzq//mS5c1LMwk7eZSAqkjsBnx+gV8+brif3u818raDxWtvfvTl5wXI3t59RwC6p8m3rS7etfj8+vIrwC+f3nh+v++Mn7SRB0fug+IPt8p42M9ro0ebx+B+KPj6xUz10IT1MSl859epz/+OvHdfDUaiGMmyjyqJoCmcfXfg2zf6GOXkCZs2m6bp9OTdsdOzh4vESIYhGJpDqT/2Z3kn0PRRMzH/9XXjnV+nmfFO4duofvcFw3+9Pw/GsDROMziKMBzmcSSLPm78ADCGAQdmKZuBOZZhGQfFMAb1Ht35UU+mTwLvOvG/3j7rZ61du3oPrG7T1K6Gfyd2FYB/v/b2Z0e+fiPvDv1A0/zJJ/5HTb8/G9MfZpwHv37F/t9uR3/txK920vxHoP7BAB/z5N8PA3s3wK/u5ccHPPziu7a5E79udvXjtv8Nk+NPH9hWyQ8s5p0//f9W/o+bda/7jU05xJ+D0lu+8nf3bnrLQqbf/wsAAP//AGQCm/08bWV0YSBuYW1lPSJyZXF1ZXN0LWlkIiBjb250ZW50PSIwODAzOjU2MDk6NUJERjkyOjk3MTBDOTo2MTQzMzk2NCIgZGF0YS1wamF4LXRyYW5zaWVudD0idHJ1ZSIvPjxtZXRhIG5hbWU9Imh0bWwtc2FmZS1ub25jZSIgY29udGVudD0iNmE1NjI3OWJlZGE1NTgxMGU0NDI4MTA4MGYyNTFhY2JiM2Y2NjYwNmJhNzRlOWQxZDI5YTE3NWNhMzNmMTU0ZSIgZGF0YS1wamF4LXRyYW5zaWVudD0idHJ1ZSIvPjxtZXRhIG5hbWU9InZpc2l0b3ItcGF5bG9hZCIgY29udGVudD0iZXlKeVpXWmxjbkpsY2lJNklpSXNJbkpsY1hWbGMzUmZhV1FpT2lJd09EQXpPalUyTURrNk5VSkVSamt5T2prM01UQkRPVG8yTVRRek16azJOQ0lzSW5acGMybDBiM0pmYVdRaU9pSXlNVE16T1RRd09EUTRPVFE0TXpZeE5UY3lJaXdpY21WbmFXOXVYMlZrWjJVaU9pSnBZV1FpTENKeVpXZHBiMjVmY21WdVpHVnlJam9pYVdGa0luMD0iIGRhdGEtcGpheC10cmFuc2llbnQ9InRydWUiLz48bWV0YSBuYW1lPSJ2aXNpdG9yLWhtYWMiIGNvbnRlbnQ9IjViNjg3NGU5NTE5NjcwZDI4N2IyYTljYmYzYjY2MzYyZjU3ZTE0MjliMjhlNmVkOGJmNTExZTdhY2NmNzUwZDIiIGRhdGEtcGpheC10cmFuc2llbnQ9InRydWUiLz7sPWl32ziS3/MruOrt3t3XocT7SGLPcyfO0XGOiZ1Od8/M44IkJDGmCDYP28q8/u9bBZAiRcmmZMk5Zmc6Y5IggCoU6iYA3bsnwf8ezWhBpITM6MFgyi5oFpAslPPS/0iDQi7IZCAFLCloUhwMMpqyPCpYNn+g2o6lWbamDKSQFEROP5IruchIkkdQ9fDevXvLXU+iYlr68jmd+4wDmLKsCMoiX9v9/ZyVWUDlgIV0LYCDQZGV8Gp0iIC6wHIaA/I0lOMoOR9IFyQuqejeEx1fh3SXIBPGJjGVASkqA2micRSQImJJC+lAPS+fyC+15y9+eWrpZjxPg/zDW/bEOI9iZsofj7zLn57Oz4zy1W+Dwy5R+rt/eWZOcmeqXF4QMjl5efTLhz8c36evL5Pff3+pZuqvf53nv+pXZXby/jbd//5p+sv8+Omlb1/qVJHLN2dxMXN+zoNz7al5WvwSTZ8o9OpSG+dHt+n+2a+5+ZK9f3/++jEjR78n9uXb13KhqG/nl6n7Ss+PPyaFp3u/f3gbQPf32r2zoGDxvIiCXJ6yvGhTnMU4vSwbCqYiaToM2AyZ4ZoOoIYcha0uRMMbWtALqCeXWdxqNC2KNH8wGl0Df1SxOL0qaJaQeORn7DKnmcf7ugHWbaDUr2sJaHdNALjoOWYrMzL6IS4eloCWjJV/mBQPeQnKRlPix8wfgYRebiR7XfgsLaJZ9InGcxlbj6OYthD45w9/lKx4CAyTA2bi4YEkroa43K8eMxgjAxVRV/rbP+o3xTyl4VEZAjoBXfOeJCyZIw4v3tYvEef6dZox1G4vwmXwqmXrtm0pqq0sI3JBsoj48TpIY0qKMqNPYzJZ85ZepSAYMxh487IiQF5Aw3wZgXdlkkTJZBk4qYb5IlwDgGOGU7zS/w04R51xa4qhO5YO2ktbBg3aerkmziLMinj48750K1iO4Zq27to9sIoMSIukq6Dd1KXtapptaEZPlxOQyGLqTaY08FjCDRGQ2xOzhDRebh+TOc26TALQLNsxHNM1l2uDfIxBBR7FtdR1ZwSGEhXzNf2tJwhNwjfjdySZ0LqBqihKQ/Rru1s/l6vdmfvtztqsu00H6+y3O6Sd0vDQmIEXEP6yIj3//LOB+VVJqKk4mq6Zqv4ZJNTUdRAmxTF7YG0hoaaqmbrquOpmEpqWcexl9I+S5oVHAk4+D7T2LC02ElFTcx1N05QO095aRK+hyG1l6prJ/BfkWkM3bU03+jhpL1yrKo5t2eYe7YqpOopua2Yf+hXXRnle0tuxq2ZqhtaVjtuz63pS3Jpd18/iLuz6jy4XrvAZsEQYLTUT17/9XVxZ9vcanX9WdzNSBNO/19WrG3oFE7KoWt2go9qt+J/gt3phOZvNPVIUWeSXBe22Q7+z2y4o84LNrm/Do88bgLXf//mPZRJ3WbKNY0W45QbchV9q8qbxxp/RhGYE4mKpdp0lmCLpJxKcX4IrlEuPgWNhqvwoBj7oigcweZmukcQ8PH/ZFYgPZ4FFLpNn5W9Pwmeu8/i3dz5JutxzEWUsQUFcaQ6iE5ZBw/ELZqlpvMItXUKpFuh/wwbp7ZNdjIW8unkjCysd2qqhQ2igmX1OZp4SmJ/e/hzVNi1L0RW3pz+WTUgSfeIi5KVxTcebunZUXVFUsxvGrHQd5V7MJhMaelF/r65i6w5oRK0P4QllfZ1pEGOZhqXpZp9DUDkBNHxcZhmw7Apb+qx4GsUQci8M3ANpTOJ8EeyRIGBlsnmwx8P1FQZrIrm19nKVXVwVplg1jL4BTudhxrwgjoLzYUjyKY9KhgUlwRQYs2As9tmVFxSkS9NtMXJUzVBh1EqfC5mX/iwqhm2+yzH1kMa0oF4eTRKvTHfHxjAdV9OdPnOdMG9GQeyDIZjD4Bx4FVMCUUg9NvaaVMPu+NiObZqm5fTgI2YKiNOOIGGugvM4yoshCUMPkym74+MopqlrvfxTzVaTO/WiWcqyAqYMHR26B0QsFb2gzQgzpXE6jCnJEm/GMvCGfJgur4Udx2qh2XfCyzF1RTX6lFyBMQQX6mHIvIQVHqh8UHhe18rsgouruhAlG322QdCIgl8495CFeEoaZHvIk2tjEPiGUntAyrRUzXY2m7g1SAlG2idKmqq4hmm4fSakR8gi8B1AFc3ozKdZvgesVMUCUbs1oYSk7ZdQ4HEDh+9IqDINETHgdMzC7o6Vhj6Q3ZsY6MEqjPIZxEq7o6Mbjq5pjrW9QeNR2gW/XZi23fExNR1N2uYGtsUy+1LXmglmTDX1PrvaQkLIkcelqiLJvnBxIbTVrD4VvW6CKp6tWHhnVEzLAGfb0vpY9yKil8MgJnmeMYjqxJdEr43ZHlABiwoecB/bCilq4YKeF9gtXr47FjbMDkQzfXPTIUgTYO4E29AUCCY20yMN8ErP7nc2HEzvGeq2uNwJZziuYelAm9vRZVGwOyKuqms6ODS3Q2QcZeBz7RUdC3Sr1ucLd9GZgFeF+eOA7sHagOFTddXYVoNs5++Km5UvWjeg5eiqBaFCr9EpU6BGiOlJHg6vzSRsSxNXsQz4Z24oPKjAdo8cXcUG66ZbvZGjAIrTsRi0H8Ux+iIpmeysxQAPWzdNpzdCq2J74fR4gghL2eGdcHCBK119M9cV0+zg+Hg5WeTUd4ANNgyCHqU3y1TDnqUkW+a/PeBgqpar6OpmvNCVgT1kVVyQUdXV1d6sWOUTpzRZgM8vowKzPDvjYII11RWnVzXVEgEeZzMP+4kNXPAkXBvCqA1VAR96MxG4OuRqd3GwdUV1Hac3iVM5nFM2o6gKvD1pJnAlHF2xtM24cd/QYeCmrSrKhlZSmGronmZpFuXUK7KIrHyC2xILTQEn29Qdzd3QdVj16LyCkl1dBkADDJOpaxtmGqIkLYsVWnhjlu0BEwukU1F6066Nio6SknogINM9kcICt9KGkPDWMzLO6K4KAtBwLEu3LHMzU1WZyZocQ/xMs4fYA9AAX9Jw3A2zYl00KupUemtnZFSQFt22tvKfGmTojES7C6yqqhiPbcgeXRxS8HMvWbby/Wx7NFBmDUCmBw3uXoOQBnvy4wAymE8LM5ObGY19swBIBDCl27f0roJeuzApmSOQ/ago1UYa6Op2FNhj6KspmCYyTMvezGxkFNxJ6D2kIXckMS0iOBP97KAEP3vOysxLKA13dTIRN9swwKxtmFLja6Hn4FhFM5LNPf6BeT+iqukGRqG9+bReIhWAV2Pt9oAXfs22nZ0nby/2TtNdx7A1tXcxax82C1u8F/unmaBibIgVetDqIrQzYFzBCf65vtns1Ko1IHFQxgQXoKtVhnh3TEzLcl2rN1dxPSbmnj76iFW8gIm+mbe8Dhdlf7jYuJfG0DfjWOGbhmSeexnqlWZp376zSxwz3bStDTUO6l7cQuRVq+L3QBhXc2x3w5BmzSRVi7P2NlGOqiq2siED36BaSJrGc69e5eHThI6jYh/46Yrj6r3OxLX02lK+bsFQjqE5urvhl/LW58OC5Lt+9BDLxV1Lczbjp975y4syRCdsu/m7DdEsW9VtbbMvRh2ieT64a2GQlTP/zvDDPKTa7z8uZ+EKtrUPchvUcKciaLDNVCuJzxGv8u6m0jVUw1A2TF7XSePbZWlug51rQ1WzdyKrT+ahR+KYc9mdEcxUwOOFQLlXYSxQ6vD/3WEGHG8plr2pn0lCL2TBdth09k3ciA04ecDofdjUy9NpgivrN4zbb0Mcx3UUt3dtxjKfC9fGJ0myqbfZgtdeqn8TZrppgPO34cfEdWv6F+hu8xHnFoiatqLgKubbI5rToky9S5adj2N2eWes55rgFGkbpr2X9vbcBsEbMAE32lIUXd3Mu8jBS809nvvdHbKh6aqiK5tl8kKKORTQ5gKFpNxLeGVZmusqSv9umyrnfT0OCyAZxa9F3X26i3Vpf/JdwJL06D9kWTpjUprxhZYS5uWkcUzyKWjh+1IxpVKzVFf6+VTiiRmpYJJPpZiBNxBKUYL1xBb8KSjLQ6kgE3g/ZhnlPTx580rK0BvLckmWOdg8yKK0kIKM5TnLokmU4PZn3PvLynwghXRMs4MBvwwAQEEnWVTMDwb5lAB7yzYxsl989uPo5On79ER7Nc0/Pvvt0/vzX1j44VzV/8jJ89+OkqcfP72JX7OXz85+zt+w55QVx6Pgr1f+rz+RJ5f5h1+jy9J2Ez26SH4/vWLp5cHBQMKNIoAKeInVpvjRR3JBBLoDKc+CZqO32NBd7/LOQSL4auuRuB21dlPTkFDNN83hx3xw+GgkejsUG7DF2QXd4xzyAu9WNr7z7fKHK6cd8G3hMQMqtloMxFkO3brInPyIhT4gq4cqVJZHroLE9gkQP707ev34uff23Zuz48dnL9689t69Pzn2Phz/9PzNm5f3Xx29e3l89vbk6PGx9/b49ZMXr595L16fnh2dnBxh7dNBazM80lcGNRNdHAyuxBb2atN5C6DiQDTi+4GvanhjqYEb6qaqmGEYjlXXdyHgUwNNH49NzSCq4YahaxPbIgoEpo6htg5FWAMvyNM1MG1KLDClYagSh8AlMFxqjxWTgm84JiHofV8lZmhYjjGmTqhRywdrhSs9wrGtm34fzHwNTG1smJrt24Zt2sR1QzM0LUMlJHQU8K1MxQwoOFqBQQwH7btt+MQc65YPow+d0HVuhvlxHUjf11RLd2zbGAcaDcaur/lAQYhiAssIA6pr9pjothM4Ci51Jw5VxyoxTAJuFVBHDFNwz8rBE7JYmryW50Zv52csC6Yn0WRaYG5klM4LLJDjukSC2lJH/jZriWI6WDlxoXWYQ4gKGykhVxuLWjiaju5YrqnecBZEp3lXGLsIbtRTe2tAG53WES7b9ZJcshtwWiXatv2npQ9qswWiPm9iq6Hm+G3kvNUL3xW09Vhpgd6JlzFW7It87S5vQ8t7tVLF423AJMYHgwCtHpiaeCBNMzpesS6bcbc4+GNGcog2RxCr5CNxXM7Iw13FUTCKZmDY8xGwJRumiMxGZ/9UB6HI2Ml1x52QNBq2cMWvJxekoPUhKiPedlXy6q5plrHstn2Lxtd33jLB4DpB17uBa7rzRHfLCDSTChNxLkcBatW1k7riMqQRBE6hjAwIXscwv+CHN8UMfKDvFP4/occbECTmh9UUVBJw+LrFgwEodH6sz5hciHLhz/DpH/GJ3wyhqn1e33Cm6aCwOWAYz49Xs+tYvB840qNzyhD4lTM8Zgpo1JrJ71SqgXmCyu26vJacB9ikVZlLDwgC6ppRxfztOUyiMcXziwTSo7oAnDg+bnRd31SuK+h8GZxyzClCHJx3ka0dp3rhkJC5mPuXckySSYnS2fHdAJsRd6f5rc/CeU1rsfVTZmUh0eRCbrZDcfcdNFaeAvWiCyqeUTeA61rMY0AFP/nLlxlJH0iYXjyXseBh5VNW2imMLmpQXPcJNYjoQpcw0YgVSBf2AoGOVDWGhqSi1Xcg9Fkhs7FcjWjBJumVrEnpXDYEe8v+RI6SMbgFCfohtCrFJWXy5RT4ScLDjGSAP2ZBmSPw/DxK5YItej48hQIMSKqCRyOywCdPSSI0HS4+kDHTxJKWYaqRytgEiJYLfcijmgxBtR5ln2TS26qetKDKOLqCAOgyCoupPIbwuPbNOeCK4j4JznELdhIKdn0gfWe7vjMeP+TtHkjK9w8HG2BZQ5eBLDNpHc6IJI8vADpIgLgKUlQUWUQEi5Lq4ZGY0xrWczHDLA6larJbPAeUCYG3ozjnk0CiBBo+ESXSKsOMDZxwbSBlTJADc0RCl7QYbdGTfBVLoRxP5HFMryT8wwcMsHguVUpb/F2Tu9VP2Gr3scyLaDyXfbDblCarnS04l/Nu1cMsk40bjPFAwmMRQGp91BLPK4muJnBCZB6jHwz++4QTTAKC/Y8kyHlfmjBk1FoL3Oeq+wHaZC6FM9REh615eQR6D+iPaupgoGsV5GkUgp6pmQNZ5id2BQGRpEiqBf+gTLj04OkMwWXlXCba9zMZWCBESqquMuIki9GvyGZN/ZQUU2kcxbGclTjDmElgIXhc4cHglSMpj/WhCRf4r7pxAn5rSNpQcyVraOqSOTRsyR6a7tAYKnA15aEqLjq0gAdXHiqqPHQ0flWHhitrcDfUbbiauiwKLKjmGlAFrhoWOuIZmgEtoECDAhV6teBqir6g9dDS8VZSh4qDKKrQGNoMbQ1vsdyx4R/gq0NdCzDE1rwzeGEiPlCkDm0o0GSogiBdcaMPXRNHABV1rAhDBGw0joXC6w91CzvQ8BHKNU6qoQUwADRAQUywRxUR06G+BpjgXxgFTjq0hwFLmoQvVCQG9MaJiwOvr0PDgJfwH/DI0EVgEgID7IemhfVU7JHXV00+V3xIMGp9aJs4FlPCscCUDTUTZm9o8zmEqRBYIFE4DV1BTICL49AAJMwuzLIJc3nkwCtdEn9hnAqgA+wgG0MDCWc6Mv/vE6ox5Cuuxi4mLTlF9d6WkSXZB7UB7jSYEVCMwNdJgA5S+0EGwzShRa2yIZpMQTUdLivDdvcj6L8F8DpVc41W6djFkfgI+Rd4wCXsB6fw+GOZ/kBm6UMsi1lwIPTtj0Lf/gjqY/EWdcbB99rT7/XHi3MAv9ePRcniHEBRgjYfLmg9eXsRDlSd85hm0EFSaoYVJeAGcb8hOJcaoo7NVfOcMLnExB42kHxQYmhDhGXnD1A3A5com0vcAoIFAfNf+QCqBKpWxz/5TDZX8eHaip+yUCvVOr+KqUoPHcwqu1l9ICnBKQQHLKi2Qja77e6LS0rmaCirRlVn9RE9fGP5YlVi1SW6tJUFXOppKTKvWiRg/+83mKzid/ri2Wvv/duljkTuExBIJh4EJUv1P0skuITN0pEqfEB//jlYmQd5OiMBRCCEarZGNd8nNtXUUDOoGRg2pb5r2cFYoVbojDXFpoFvmdQhvqlS23RMkxiWqQRhd8K7UiNJKB4/JH6ePizTrkh11YBfFgXYrbZtPgMRioFFIYiYVKdp8tf0CvwiYMVFYkGEKqKHTYxkywGq1IlfJPzI2pa0zApZJBnvdXwtbtXXWPLa0GvGFlYdK29v1QtQRxS9xfx2Rh2Vu20ewf/hH6pxVcUbbYr2hCyKVRXtgvlc7VbmzT/NFGm1D3vjPmzs4xVvpGqtBgpv0O1GUeS6m1XjwvOVj0aCAw47ur9rBtomQDh2r2hSSs2t3HaWO6FCwVJZkTKcabj6DADO4EbMvbBHdQNgpIUP3XGIex1bUQJqiDXB2arlaqn2dq80CStVLgsEm1gtp3i6F6rzVNbb3vPXIX5r5G1radPAczL2Km1XFf34w4KCGwmaiS4o/iFLcqJYknICjqKqgHcbg0umyfinLTW8Ev45UXV+q/F6Uree3KoIHWLdWF5TUxE1ZVET4UKXHDVw4G2nXVHhlTaTshX/CthlEYihiICzoOMf4FRFmvmyiX/wYTkOewZWj8TL0VMZLzmGnOXxGAuZB+ac8VddtThqGrUcoEb2uMhg6mPxuCR4y4JTPUGlhaiDu8P/QJku0XBCeXtUD+tiZwTRAOvK/XUeqLQyLpT/KkJfUV21NEm1VOFK0JjMF88QcDcuc5Pl6PSflzNcNr+m//pNq4gbS5hWBR1BvaaTgv4kH+4S8dv+6FrYkvRhOpeeRcXz0v/L2vdcDaCEp1cDaV5dl62sITkD6WoWP8hTAq7yIMVhZxcg1yidBwPOL/XouCwHU3oBl5BdQqB8XkyaCSR+zuISfMd12vyaIdS6gBvY+2psDbX7KL331UaS1rZaDpCWXgjCr33ZNgcZS8Ug0JAtTEftsot5EtLoywaofhiMsYZb24NdUCCmY2zJLyg7xjXDb4Kk1uflKj83h4AhnmK2CTR4Dpa1jIuMLJjkBJhJlqttG8shydSUfipnKec3WeY/k7BNquZpjcphfVfl9irMmr7z+cxnMRCPkULm9l3iTlXCshmJ227WIiJKM7ShP2Qkyx5WybpW8rJDnUaZdVQYhmUwLXrrc//aDhrFtqR0gLcWhJ8xXxz5vg3ZG5dgifDjNYQ/fMUB3D0JH43i6Doh25QYNReOqpVZd0iWIwHhG6ML/sQHV5V3SZrHCyDfGHUA5/Pqo85d0eZtBeIbowwMscTFXHdImdMKxDdGGZQoGRfu0cvRHcuUJMB8YwTiS2Xzu6TNCw7hWyMLXx9J7tpMvWiB+VwUWu/SPBqV8eE1r653lCBwVFoRWcFSKeU+qX69R77BNPCPkbgKZFPKL1H67nzX0xqvQxEYLQq+LHdvQlKx+Y8vPGJZtIUR/UykfVzhJ9X4HXZLviCJhXCsK1+kMzvFIsxfSYWsgdCavK0SHevyEYvZxr3jixnu5giWpo3nCq5PDWwxg2cI8xD/rgm5PtvIm91hn3v8xw3kw+b+Wlr8O0v2L5ElO75KQcHQf2fImhd1hmwfKTLwJjTMxXyGZNmNXs5u/gwVTPIVZN8qdgUNJW6qDO+X9F9u8jynRrOYbRVUGxV+P2MJQ0cbJky7zi09wfP4JZKEfHWj+AmVR6Op8fm5AtCKgjuMLrax3QKVQ3H90rHaRv6s+M3Pu43PtvFfW/gcth6+/sigwH2EfDfFV0DGsxqZw/ruqydgvaYoJv5waVHrV0BOruyQiIBcpfrw6YR8UY2/FVnxlFix3mo4KaPwDr+lbEHXN4CUJLCSOFYg9Ktl/4+s6mOWJKDxJDypU2LFlGb5bY2qsotRXbPADw+bmN3lJ7izKZXeAZBXx7idAX9M+ZsRrhaZxE99fRXSdSxQORTXb5CYszK5068xW/klFTb484blDCV1qeCboS4Ny2oVcmtL/ddA4SqCOq7xWySHFyXfDI1xj1l+LX15GP5liXyKCIpNY5hsXCpNRemXN7rrym+TJ/4c6VLcjESLNCbBZ8+XvmqBPmw9/Dtj+q+dMX0rTlr8djOm30jK1KhSpne8tLA6OLMxFf6Xym1WjDU4fIs/p/IZVhV+9oxhRevvqmUKsvgdmShvHRrwhX1N/Fkbif+cDfc0m8dvxgdqPiC2sznVrwV9JWTmuEin+MtFPPRuHr+89/N1LCHZPGro82p3Wv+wTdDbxA9fReBwV2selvt9NEpIs1llZefxdQcWtHew8HFWL/g9lKL9Wt65srbnWZTIwlfiG2H0xUaY7u7le/faHVRbfnNKsmDaMsP1gS3Vi2t8xJzGYxCAjBZ141koCnknNR5Q2Hig8KBLecBSGta9C0hLRQD/I/JIwQb3uodBrCBTb5bC0wr/a/C/eIogv380QiKC3kbmuJqlh/wNP6iHIc/gj2wcPsK/rc1jrWHL+Ko+mUIULe8rOsUtkUIwOPqy2Kz2brH/d+llFC6dLtW8qUeNBw+BadxgG2+NDO8DvCicwjU9sWySr3a31LhM1jVcDJZ/+dkOJ/xdRlB7wRQiWVocDN6fPZXBz53RYsqAAhPasPMjTsea+EhtfmYIEFzix3fK+Uxa4tDFqTK4TxImK5hmbEZjfu4Jb9AcObK2Xc1Uaxj61keDcMDVNkXkt/bO5S1GJrqp8AMTQOOwJQVNwTJ/1sfetAt5zQhQxd9rx6OF2hiJ7dqsOKfzg/z+qPVGHEb0x0pl/KF1WfwsKp44BUpuCeV2fR70Tlkc4vGcp4Ifut0tGG5N5So3stLm+hbtqqhzApJGBYmjT3hg23jcfi3EGIymz3x2tdSQbwAlecrSMhVmfV2N7tbU7nsOvzquWvSyUqXiAdQ01bxCyAdGOl+pWWuY9USsW+cl2OCcf52VMVwDScVfR02nf8SjZ7Q4Fa9p+HqxzfYJVk+q1cCtbvOUxjH/Le91w1sa2RJdG0H4PwAAAP//ALcASP88aW5wdXQgdHlwZT0iaGlkZGVuIiBkYXRhLWNzcmY9InRydWUiIGNsYXNzPSJqcy1kYXRhLWp1bXAtdG8tc3VnZ2VzdGlvbnMtcGF0aC1jc3JmIiB2YWx1ZT0ialhGTWg1NXBrcHRnUFJpTDYyVWhoRjZqWmxucGxvS1lIK29wYzZaUDdpM0QybFZGbGNyZkhSL3hhbzIwRmtvajhIckFIdEQraWhLZlN3YWFUM0FYT2c9PSIgLz7sfdt23EaS4Du/Ird82tN92kDhfqFF9sryRZ6lZJ+WrNntF27iUiyYQKEaQBVJ+czr/NH80H7JRmQmgEwARVZJlCW5S0dEAXm/REZGREZGnBDx70m2Wm8a0tyt07PZMkuSdDUjcU7r+mz2a63VWZNqdUqreKlhGm2RpXkyIytaQHoMmZHzEyL9e1Jvr8htka+ggGXTrE/n85ubG/3G1svqam4ZhjGHFDNykyXN8mxmWTOyTLOrZQPvxozQKqMabweUX23SrjVFpZmQlCZp1bboOr3Taohdzs6frGmzJIssz89mq3IF2eqmKq+hkV+Efvg0fDYj5ZrGWXN3NtOdGUnOZi9s3dXdpWnFpu4Tg9jE1G142lvTjg348DUWoMHfUoNkGkun2RgMT3urYUIWykIw6VtoyhzborSoawNWa5p6QLyLgJiubi41PbwwDQxZmlJmHKTzkxNpaJ8k2bYdi2/KW7IuYXKycqXRqC7zTZOScptWi7y8EeNHEg1Hgvy6KdZaU2r15uoqrTFLTWBqJ4K1uFw1NFul1Uyd1JMnm7ytuy11uoQmLdY5BaBRizo5eZJnfQmLPL0l+IAyIOMCprGhVcODAOQKaEq6atKKrDWDLFyAt212RVl3MRprnwgaN2hGqjIHICjX7Aub8oSShkbZKklvz2aa2cHXqtQ2EFjl0GgiN5FumnKiZVO9Z1M+2ZAuSmp2uYZJWmu4Aqp0cTYD8KANZbVofEFKHeETIgNBW0kZNxmMtlxvG8RaXS+rbHUN4wgryCJNetu0XeBT2c01W7rtYjS9bo3iq6izLVj8alW6Lge1jJvBU4nKSJM1OCN/TxkAl9WdWPQ5jdJcDd9m6Q2A+tnMgOVpegQbAjBew3hAq3Sznd2suJIxgFZtMDTdpqsySfiis4ilu0/hD3+hNMN08HcZ6L5L4Q/+s1D+ujUhlRys8felpoSbJq5+QCGYRbOWWkBNYmIGAzKYDuIQuRBTN3wLAg1siRNiW/CJcRAMJW218G0B2MDVzDchFBdruu16gHN0L3QgswMoULeM4A02Q1RlQrnL4O0Ll0CjLXdrw4PCH/xnLdEd3cpN3XGx+sCXohC7GReB7iEm8tU8mm5ttWFREOji3xJiRhHwf4S/3hmu1lX5axo3D4NWm3AAXT/zYBW0usBHhCuTTe5T9mOKiTbY5sGB6JmBg2uZnu4HDquLpQRkj/CoZsNYmCz3DYYM4xw21s8x4C3UyqOUyRRzw0FXCUewGAaLKXs+LKad4zeiJtYCW1oLhlgLW19ZIga2ydhqg1BMjH9vCy3AWtTVxvO4g7UmSoLgty+C6brtyboh9KkUCnkfDyI5yfEwQIp0A3h8xUJVcGzDHhMacSB86gBaCQl/IpaCoQ/DAMBUjYDxZ+lhbvTAgmDfoR7xWBbAFR4iDC+3dcMh+JDxHosiLF7DOPaYHOwnc9iyzjkh8wS60I4v3cJeV/E9aeegSvunSN+OK80b3DDl8Xyd0gLIvirGiJa8DCTyMpi17ZjYRaWqkLjd0QoW1VMFQ4qLba55umgIUBM3FV2TuAaaqNqsYqCJlA8NenOVNrOJUZJaF5UVkCQAAEiaJID91aGKy7ystOgKyK6qyWh1R9a3kIgHs8Z0EUWOuT0yJt8imly1RL5EC6zpSmIEphLzGpJ0QTd5j4HlKclgSJZZTap+U+8Jyx9HkW3dc6z88JZc5WVE810NoXlOygX5IWuebyKpGU8hnAfeU/1utiTRshWSjFqUl/E1H+atRvPsaqUVkCEH+ur//dd/y4WOJnt36R9s/hONQfNWW5Txph6DA351g/Tv8EGacmpC3r37+ELPT57M8wzG4sl8k+PPfXwGrKkqrQHWdrIZDzAZguwd0/LAYdzH1jAiXTRe7rw0znUKLUkoAvjLkoisacK4BBg6IhrejwPvt9TtLOmRkUjdYv48q5uovO2gApmigmNKiZ3ohmKSNWlHTko1zcGNEn4MBi4G3igZ7qWfNyvHunSpoNkjN3fk5o7c3JGbO3JzR27uyM0dubkjN3fk5v4Q3NzvzS+UNys8FfzjcQ2sY5dH3uHIOxx5hyPvcOQdjrzDkXc48g6fMe9QVld0lb2lkk6LzD3I0Uf+4cg//E78A4eVPxTnwLt05BmOPMORZzjyDEee4cgzHHmGI8/wWfIMx/OGI79wP7/Q6lGR7l+XjH+wicTki7IqzrtoeSyVfF2IvJy6GzdVmgMZvk0R/9ikiDQHH/kVjKnaI+UaDXAPnFif5+VVtvpblTabanXZlGd4T6r+k/30T9b38P8qa5abSI/LAj5+vntdAoheIBJaZasrCFrfNRik5VIY1BbBT0FrmDJ4Scq4hp+63FRxCi+XwC7BDgNvWUEBHcELNKLU16urmXLTR3T1Obtn9SJdbaCa1fUAcGR+R83OuJDlXVKVWpxn8fXZ7Lcv/7kpm69xo2kukTXh36f8B/DgEiKymLE1OsvDY77iP2t6l5c0EZlEYTC0LP1ltrpcQ3eUIvHKmrgnRgrogFJcjy0us7bQ1SbPv+qbM27kqx9/eHn5y89KQWWVwRxCK1ZXl5sqV9Kz6Tydz/uJnA+ncT6axDlO4ZxP4Bynb84nby6mbs4nbt5Om9KaTZ1Waof+8z9no8nQlgWF7S22g4Vj+TE+TBoaaRrakbeIUurB0zYSxw9jNwqpH/lpFPiJFSRJEoa+b4deHHgTM35F2+n+80UJ211Cyk3zF8Kh6CvC4iDwFaxrkq2+YjvdaY2rPFspa4S0aZQlSfsk4xUrLywscrOGlbW4jBt6hmX9dbP+khbrrzEM4OaMQ8Zfc9bMv0Izu1iEpDNcgvYzHE+2W//J/o6HIOBIIWK91cvyhuXncyUKZxyrFHyJ32f7rGR1aPdcjQMkqggjxA4gMD376JG62BcQ61tkfaeZw+qPq/lfbTW/w5yvaH4HPap1luDh6Uby9QqmTJ0UXPWwUuXcNEaoUJKxFqE6dc3TE9jOCY1jAGS1ZrbfK1nbFX46/zLnQ8yW85dXzdcspFvgLIRNH67urwUuORUtbPHIqYDJvEN3fxSAMjwrNBybxmkQWKnvBbHpW77lGjQK7TheWKbl2YvESGPDiI3YWThRmlIaGIGZGm6QqAB1PgAvMYwq/u4RvES2SYSeoNX4kLekb7cRMBINteeZSFgrF0x1HWCxo4NxIpFiZdQqspk8K/vHysIS2HCsf6W3iL5z6D7XyK+1Bd4wV9ThWa3tFQSJueAp24gJoStLQHiyBUwDIb/9xuNeImsIs3J+ouZgyLm9Y/8k2jRNuVJKg7kra8aRSZ8zcaGfp1eZmW+zusjqmnNPRVrXADeqEHeCoZClHQ9LHCR5CBtVzAEDWKyBd1g1AyZlKCq5bcftXlmFrfsWwYcqEDI8lEkSTw+d3NYtC4WQlix66CQPFyG+BSwVGabqBRRQWCjkFKN0XCbLBR0XWCUJWJNQFhLIyQyWpBdwnCgSDj5J/KudBgRQAA0xPQAYErgrrAt+twDX8zO8rBNm3iHON0mqLSp6BftnI0HrqmyyhdirgZpI84U2TC3mL6I18MooGpnAXXIx9TxKGzpnhWFvh+Wdt2sOO3giUTh0vc7bphSwzAjHIqx2qKTIGm2J4pKYVkmtpSsa5WnSJ0myGlY2QuC9yQDsN6lGV4m2rqYTSoQmwyp4aoPKV+xNGMngRi3qeJkWlBm1eFUumhtapa8YRn5WJj1wt7DM+iTwCdtpGKLpcYqEffoLOm1jTnrQ6JDWiYL5ekKoz65xdNk1ZXnL9j8RDGw/WTfAvcI6Txku0uq4StMVcrLubIwNO5pbxk7irAuy2Iio2KNIgB2GX2CH3dkOTrqXgRXZSmPoAghZ/rvIGsZXD/iCJ0uzw4nyERuTk42P2G4iLapSeq3dANFLFjYXrq2A+6d5d5Xro+M6drTFiXMh/hW3yJhkcy9EeDya+l2PpnrUrQiykNUoxc3CGjFp3VRpg/J5hMp1Va7bNO2k0jYrkIdkgYZKcHMWaQRt1mKoVhlU1p0YJoFigAGGJPWYmuxStcfFoySz82GIIMqkC4tyf4vbTpoo9ZZMX4icy4U0Vbm6kkYFSW7Zxo81MYjdkHVY6Wz2xS5MuqOHY5J6dj4K6jvNmjnq9gWSUIQ9NWmxqiJLAoO5iWA7a7sNlKsp2/KZkDr2114RRyOK1jj7U4+YfSGALVq2Hy0d3SFaWNMkgT6cEmt9S4yvOe59gvJRUUkHck1Z5k22XgPj0r9qNYmaFf5pdaGSjP+n3JBiUzckShnnBflQNg+oa0lXQJ7IJACp0wbZnlpA9KpclHle3oz5jY8gTVAbuolgy8uYegeTLZAbCtD2/hKGi59+uPzx5R+FH0yi1HRdmvgpcH2RGxuRm7qOaUZOmvh2HFihnbimY6e2bSyMKIn9xKG+ufBd07WoZ892Sb33lG5/Ors1rIVc2ZNx/w2gFgqbsDhJDgPcjnw31g3T13TThocR+mKDsftdJsbIgLAdxtJN02bR8PG0LQxLnjInNkUFQFqgAqBYYnMqwEAqwN1awJX4sUF02/GgQsPSvQBrgk039HNsqWEj4eB6FBoc4J/Y9wzDhq9cx04ZhodlGJCd/znij+Xy8Y/vrhCCMZjLx5xL2PJDj39jkSam9/CvS6/xotRQB0M9Vjm++Wo1rHEato53AGkX16GKyoPJOqlBL9+o48JG6u0Lm7gUBojTGfDoR8pwoWT4C2E0HJgZO8z7YULFCtdkZfCRgonTnQCIHPu5hWF0GA9NtDxsou3lUmufSkSGTXys/M0kmUHIS5nBOum2KKEoJyF4Fcn3yJz93AB20ZigjJAd+H9PpI9iFkLlU9lPENOzRnLO+kGEbvqBZ3m+ZfwLYfWAJot0EfteEBrUiOzISNLYT2NqwV8cLyDQTTzPts0oMtw4Dt3Qju1oYflJahtO9EhY/ZPA6wxWWhqOEa9RCYBTIBNm7sWEBYQxFBK35fmAF8wAlnwQoNwoMF0C/ApEmIoECRKhoMnyQ64K4wGaCYG3MlHRxgxH0qYA+LrQROORgFEcHzP5HpYQBuOkwKuFue5bGhblXEAjUDRm+86wBYikghzbBwyZaV74um35uucFkkKUyTr5tjCQuXPcCw9SAgYdspOu50CJrDPQDwdZQcuBhxmotaJSmwdbA+SwUGxmBJjK90INWTpPHUxUP8ohFg1tQiTksTWWZVikxsrEKjWskqUKIdQZtxICcRwt5EC3uNNMY9+WA9gNVj36fQWAdCKo/k6o3WPkuowzmgskjMZVpe8D+JY5wusVfQsLoJW1y3jbdGGvJ4gMaoaqK8TyO9RoYJ+zr4UYkgrp4tS+ss+u8n6byqKsrj/5TYVrIqfAaeY1b/Fxh9l9MGmmkUdDz3DtMImShLpW6plJbFpGmga+ZZlWnDihm6QpJDFiYCWoa1PL88w4Muji0XaYT2SXYcIKBJo02WtPQZLVcgenEvgy1IDlKNm0LIpiMmJxspJL8ICw1QM/eCrHGAbLHzD5HrIIphWwrCbhT8zM8r4RaaiaHV80JpLTsPDd1U4KGh1FzRa2EK8r6W1hYzDsPnLeezpua6qY0zRY9ZIWr8HI/R2YHf99DzNyIp9A0gMw8SptbiD/vEiLCNHxFKaXzkMVNB24gcDSHCju0XWE3f26OyCVyH9F602VKklHAzWAZ51tU61I2WnKxJlDJ+NXDpDw38mTFd3uJXfbdQ3j4ZXSlog1MZvjsEDpVjrJwKoIS9X1pVNxwUxDRdpf2siXELn7XAKlbg+3Ti4MCMPkjqA5No0J3QY3YPJsj/I6dR0U7LXXinCm4jJBvd7oEBEmrw8ysdOPs1lm9KWwqDrN07iBjRgVh2p+UnTJUT6DtcukvFnhNshB75IftokP2IFTWqfiq6FX4i2q6CpetuFrGl/jniHKA1Aq7/Cwryb792BZNtfp3dnsisSzoUpZD0xfkZfdFSiuVPYVweM20vdXACmDJ6EGMA2p8QYoI5wc3OX3gVIFDtgFsuXtpQyGPLBV3d4Nrt3ts25uhjfT2oiZcigs//sE9jMENGWldffGOhl1XbQy6n02PIcf6VvjI312wG615/Wjg3/lvF53fNza7PH5O/I3ruYwJsbT8fRJ3jikIkybneUz3mpUl9HfXejKky+TiLq0vrJ7Nh+JtRDAejZj58fn+FRVyeUc4l7Iy7IhdAvUKB5f7zOHz3BDQhWWtmgOX7ip8B3lURAZO2avD0Rlc55rjNFMubjdOI2n4jiI7ULivcjytAYaPd0LHamt6LFSdiBW+pG35jC89CnhoE8a9XA9DrwDC2TTu6AgLrMJ8ayZ3YVraUeb9N9IO06685gWABk0IAEvBrr7lH+guNngF+4CCjiHeLpQQ7Lhpw9AgQ2EHIoqOJTNzvnvg+jC8p2DkASk/6B4Yo1aJ1X6T2h7czC6wMwT2MKaKHQ30mCFECBl4uv90INca48d1gdih59R9a9t4xFJfBgkcZXB4EnA8Ki0iq+bvo9SS9++ADTi27rn+9KZDp4p6YHrbIGoQSGufEPWsVjui7YM+JYTGKga47x9wVhhVSuHc7MDYgMvx1ricqzCiZs2FwtssWhvyOELpvqNC8y2pcoHGHeNwge8wItNWGrmG2dp9oo5xIRCvZ1iA0UfyeBlvC0glzGQaIz4et4o0XnTerjvB2JMZeWhnob0+SD+DN2D0GfoflDs2es8How7paxjDGqPCt6NP6Wke6HPccU9Er06EIl+K9V9RKEfiMUrkJGX1GsfFYmaDElM2yEIJs0QuJNWCITWnnzP30U+8gLP/AHHOM6bcMCltYYFALNNmS8Q5gO4RsHIPAO2GZvyzAASsjPOYAjjDMZzaws9c2yKZ1X8xMoXclMnwG/DBtxveIgPl7ZuBuHQWoNFgrYRwyjEoMRcagFrHyp5OOoISrqMUxqduhvrocdu6bO4wCFdDdtRfSY3HGFaz01nulOmJvUqRB15VNYMVCTfGgOwMB4fWzydCwfqqWw+TEVbU56NAzG9hB5m59LH58pUC3XAQ1G9yDZG845S4G4U36ohHl5Zj9rpgaj9qajziNY/CFpf5/Tu0RH5XvztA3wy6iaEBJa/JdPShgdBjoN0rGmq2tyQHBCwg5I4z2GKVaEcbyCxHVxIxR6KRQQkzs7Fy+eKPYSho8O5bJFvjD9ctch7eGyRjqzSm0s5hCgf79CaHsFEhzLgbZv6th8Rxx6IQ8zV4+KOowmsz8IE1p4MdrtEz9u3B1HmYVz1h2Wq6zTeVFlzdyiebPON8aSnFjmJJ9sk7CAbG05onlaAntYl4Ks70pTX6eqyjukKKyN48tZ/vUPzesRZH4g4X7UtPZJmH0YVdYn+1R9ZVOl4vm7a9kBBHjky4ANzhrhM3QuGrB/KAJ3gjc98obuep+k2noaagYUO0A0bOE8P3vUwwGMTJwyQAAuQpQ9MG5CPwZIHLh6yIIPot8jJBlIwRr7RQQ/rhoP6qYYToo686/OMGsv4zMQqgec1UcMej3VR1ZX4b7Bho+5YJhbneT7rEr4Gbwsd2WygAkOVRTZdaEOudX0fxPqwWdht120XEC+TxXpM09ZwUWobiA0JNWuZnBWlnLYDg2JDKlu3QiRVdX6VA+pzoM+olwrIPHQxgQtjB/kMkQXvJIbBMxMFFy4JeV0owAgglr+Jfg+bihdHfN4X3ukXIY5Ze4/SBBYdf4Ts1kK9KbZHqXtJp7hlT+0Lh24DLZ6Ynbdvk9vA6HY6u2R+CEabI8qkeadFS+M4XeMqS2+beVvsX5dNke+4j/4hdpEM0BI09F3OtOp0vIX4anm7Se0rQI3LTpFn1VRZtIE6apKk6xRQyiq+42lEAI3K5nKzhgJT2GuwcrJOy3WeEpT8bVaI6A9t994n5KJPRxL8sFMtNn2PzbybKlnZogIkefFKlAPEs217PAGT0jn6SOkFKWFB9BZIbKNqvWsPSm2tZZgGu/MED0DEwN37o3TEuGAmNwLVxkevfnPhE58LFR0fryT4o1R8b9N2XeC+D3+10Dk7b98+AufP1TtZRajNKa75brM6i7Ic1uYph8KvD14+rUzvvlXS6lSSzuohjeoy3zSAI9DqITy4RmXFNCpJhaOkGbNzMTStmvweaJSnZDWiZKr9hpalDRkZXeSItt4UBbdhlkt2bu65pCHNmmSIYoxVikozhNk07M1gvj/6+r9OI9gYlmWVvUXyeuo+avigUgtb8w8nK1Bmt7+GzHhlDW7P15VWrnKgCV6U1YRyW2fYRsxtZwppz1kVYMNucnOwwNd9BjmpyjXqwfK8ypdW33D/x0qv2nWpBLY0A8s2VsflIDPOx6iGvr1sB5I0ox9UEu3by7Pu1G799FWBDzDdMDGMhOkCM/w5GuG5ekv2nhlT1A0fe84GU/W7aTMerIU5Pb5cz+x9R3hKReujDfS7aoAdqqg2PaCK/sn7jutYceOjjep76YW8mx7L9ABLB7/vO7zqgelHG9p3Oo89/Lx4ejjFKdj7DuXg7OjjLf4PcTT1DodrO5CDiH7f0R5IoD/GaP+OAu53kNlPD38ruHrf4R9Ibz4asH9k4dChUrAdFIgYyz0nRVyQBM5CYg7OewuaPLQPgMGVLM6iXwGJV4yBAq4W2S1iC9niZJMVKTO23V9YvM13XTscGG3ceWIi6hwYdkQRKRqD5ZICyS4k+8H/T4RV0NayLe2Fla3njnWKRhDRnFgNfGQTb5rBsdBBq4jdJ48Cx3VMx3MSz4hiaqdGkDixayaLgPqRsfDcyLe9IN3nxvns/Oe2ga1RAujH/9A0glURAboEWnrKQhRgPt1a1mlIXTtK0nRhLJwkXixS0w78JIhMx04iJzS9hWOmie1gS33fMalheakfpFESxTGqK2vahDMV2egkNzUpuaDr7LOt0RTmwDBlgWPff7Ye7KK0uUGDm20axbNdB/8n93qgEBAlZC0DgQoXoAzFK0zAUUTwmDEw5BycwBlsgXQWt4SMRTL9IPugmfL8ICDoRo1r3RjdZGgksGMZy4ogI/kpmYLBSwO8ffvJTs3HUtJ/4yk2PE0DT7uee73ZTdS5N6eu6iuK/IPChRhXD2yPGea0CH9iFR7xl05XPpSy1VClf3AVIOyuAghN0Id7+sLWx7YLdmn632tzpUW8E46PpH1W2E/mNi3u8f7T7ZQxrdB1kupVphM6Ta27V2xxvGjXxnSUVpRJL5AT9uHHqYRZ3skmSsnYmpmdD9bMHBfMoI+qUXCpCGYK/BshGlUNgvMjpvLqKk/RJsU0FujXo7jW/4yZGudSNWWdLijb43tPY0eD4R/CYLhqK3xoED9brTfoUKvY3PJR5LwcjB6NJGMK3GbnqZT6ixq2zvRHDCBTaRMlMScJWeoeiJVsUztmXOabgnkZbUFZQ1BGmrjuLJiiMWgOOKfcgGlHx00vOJjrRlpK/Si0R6iIKvA4tmu9zmmPodML+MctMKy4g1Mm+czQSD+rAX7SPJnING4Srx69TLHxgBUxkY2tHcBEjMBbaMyeRDssE8m7oykoDXf1heAvkv2yo51t5vlgOgqXWZ42aF94sdjVWIEAvmfDoWKjiSzMmMiyzAE4987D8RMOvxwpgcA9jrlktMngiSMieMexme2GhnaKFDcVuxEqY+BURNrVJDBiy3q1BxTfiG6rS/e9Kjp/zfaAYXmTA9Tlgg0jzWdkGmb4AAlYf3jy7lnfE84N8Wsm+U1Wlys2ho9ZWak0I+89p8XGlCafzXqwuJHdnUjcIsIB1PGszF484Di5JV+wqO94zCAXorp2ltVMEPNKRAzysOXHBl3NwcK/z6q6uYC4F8yKkJJVbfI/NykwgsCbr8sMt8V9+DOobzAkykdMYV41RrRvjVPTs00/tC3b0y3HCx1r4ByOnyoJ/IgIEmboP75/m/zw5m4w7NytowCxXanQ9jc3IQfIEMHtl+f5L5H977/S727X/7CWRlTkm3/chTfp//77Ni5elhf/IYXfnA1mdA19RXvh6OqlKDd1ijCoIBNVONB5cFHwgzxBOivve+Cf0uS16tZFKmcHNShcq/DSBfD/9hthb8zVy7NykydoPJvgqVu3yBTXhaIKycfHsPY9erEqX6Bl69e7HdNMtPv8ZdksUf6GJl+X5U0nJJGaojYSefTXP33706nwLUNvxW5OMhSxxlgwsJPA9qUoTWqWqegzs6hEbsrVvzWkjgFn5TohN8u7vzFOfDTWuzuK5TzrhSjyKfC4o6zSnvboWntKbNtY3+6v3ronQOAk43gytxtrYMBVm1mYBgdKJJsQgDF6XDQ3Km+1OnvL7L23zYKwr7kJ/lOypdWfNU1yLbGuMuRv/vJ1R23bVk+m4/sUmY7UNBphXO2l0UJXWaFVZdPC2JM4q2IUGEKhAaS6Yz+wzn0c9qq8ho4InPIMW9qGauWaxlkDyfEKYBfaeuVF7gGnW0sXC3hh7dPqGEVGVxpPOyNz1oBOLcLEG0pok9oi/MmYal97qCV9nSIAJX0xXQPgofe8fZsyoRpByORCH7s33G+No0hKWeCdIbv3sBMJ6GpT5d+tUCCf/D1dCC9Ve4nxpohjJqJWhdwVQHs5MvbJlRjwVBZJKcSa9TM+PaQzOTmFUk+6dfKRb05j04ncdYwdfGs82X6SJuATA7QgrLCThnCi7aNsxh/YIG7V6Nit21C5lGuYEh+KysLA/XpoKNibZJB3eK6SJReM8DPvcVjd6ZOxyeR0JcDZM8zOpg7Cqw7CFNkMq6edSpYdtqZvheNoBIf7vXPI/kSYZFUQJ5JXEu5KeNfGBgkWZdlMSnN4BIzPIWYjO8L7/A0/C8ulzR+1/3ipqnKUvNrPdzsT2Un2M3Zb0oiaJvdXCaEolX1fIn/inOshWh9r/ePS7gdxLY9D6P9xaPtPnHLnQtn9qfZ3p9kfoe3TZP1e7T5SIkdK5EiJfERK5LHZ3yP3e+R+Pz3ud5DucWlvrnHQ0d18696P5p5sJixW+eyrPZqTCDxJo0jx4iuUj05OBpbblxbXy4BtT8MZ7AAMHZomcbUpIokE73GK8Hwqe1SVnJ6S4lazWherqHbMX4tEKobfejE7jRT+zfSGCly28IBQ+VaMgnlbO+4VjCQRfnTyBM+R1UTYKWgyv/HZXaKE/sJG1fltpehrbeDy8V6NqQMuD7AGTXl/7Dy3dK/8R7nFkq5pRRv05Dmfiv/4HZw3VZrKPj7aHuP7RCc/894J8q/tJP/6w3azpXLb7orPP3x/BVXfdpt/vUuvuQPalljMVjTnWPa85Rdk768cmS+t7v2QrQYKT0SPBrx2P+hsuOi62VSp0B1ANTvmB1d29cqmYUKggPl3q+E1Eqb+oWRuloAf7rpF+z2n1SDsNp6irJAQ4TceROxeV+l+j8uckkbixOW8odYS3jYkoisfS0fpM7i2ieP4zpcve//Fe9yllPLlWS9n5Kgm4eqyqwmJBx3VwTXgR4q5Ea1T7hqZL4f9nYb1+tb7OgzjjJhSyPc/vvz28vsfL767/OaX169/enmwe7DP2OtX6KaG5wRWmFiG4dAEXQf70WLhpyb1k9Q3o8A30thKDTeMncC1o9iPF2GS2L6fBka7uKatSCBCZSjsK9L6aTtlPtnaayWzIf6TNyohenkn9D3Fjg5lDkwGft7j2SmLAaOc3Fd6gazIpPP0CenR+evesbh6CWLiAkR7I0XR9dyJ4EdLTmgojYr4dVOsm5Kxg21mLckowM+kpul+t6/Zakavgqj73yL2+4fvnrU/ecXm3knDbLsm7RGm7WInQPT2HxADC8Wuhyd2OH6ArrOkF/xwJDqZdTTPsNjWUUmrBGZpfXcIyKgKwpiZ86xbmm+gEXvJZw+FkXhT1TDm7HCkW5yjYe0ag4OqdvDhwf2wI9RecOmG6T2w+qPevflAU/Eh1qw6kg+urYNAgN8bG90T41Ht3bCREhXQj2TXcWl3QaiINLu/3DMqQFwPIN8KgvzXDo4mRK73X0/i8qBdwppBq3JtZSLbgc+GPSN4qrqalJ+NSPet2vuG19la2WTFLgy7cozjzXcEpCNGUUDewG7M/CDOX6V09ZJW6WrexYv0QEZ3BFyfl91hY6FysiQDWpPd+D6bYQpG5J2ihKzb/LuKAD0CJ9AJcLeUuTZnPxpvLjNI1q5OHlPrfJViAsH2sgW7mXu+AUvN/lt95gRf0mL99fbM6RgHy+k5DnynObz8z74pZH4uuaHsIbf7liZbnHHsmnOcTlsWBKrzKN9pfPCUZEixU2ULRAEfucB5aEXss11A8lmAQ/cqc8eT6wf6es3QoLioM8Y8dEx8iotwTxPUsgB8Sxj+xesCkLa6JgX61Ks363VZNeTPX4SuGfylP1RiwyzRGPuTsvxQf25HthMHoW/YtpFGnr0AstuKkySwqOs4NLW8hW+GHuCnPVvIBC8dRHC3XDjIiLb6LzEdaVUJQulsxg/mGdHFDtDZGIrDVGCHw9AzAye0WnYA0Xwmbtuy/K8xA54XAdRt8cCFp2Pg8257KTd8Msch3wGwaMjjUhjr2AG4e5oDYbXIUP0+x+NdgbNzBjBiTj5f2PuLggankaG0D+9an7lmDXHXyNzkPdYi9zdFeWAH56jfvFn/LcmqlOs2jS1KTtuq3D0cg4MzGcOzrUDeKLo9f7xtKNeTU+C1p3ib1sTiwL+2QuD1gkvFEKGS+vwCLQs0hA/fpHk9OsFusdeiXJXkd4TJfjWdizwjGOUjgMO5BgIZBgF696JMskWGflyftAJPZi0Ai0vxBSgBwzI1I9RM57XpnVrBqWH+owOHbqRfpWtiOl8RTI2nklJZ51NLQ7InON6MDh6n+gDJ0UyBu9Yap0IhYGhnavDTs7C3zPA65N1+ujC659hoE9mxL9CPG3rqVn1XMNfshvMG9VjQlKZpB7ppWjwFWpu3oQhPcWph+j5zEndh6b6J/n0s01Wca2jQSdTAcdxANd/rhDazNBzohmGKJ/fK5mmu7gXO2xc+JncGnnfQILylh6GVW7phBXpgqpdWdZe5/LEhAdp41kzZgbsvnBWptuNNXtEB1gF3mi+V0/ODmed8ioYHNdO4kZABWa0c3kvn8e3bgCFjLr93Utr9QbywGRmVTVMWLXply5e0920OMdMw+NbYHp0vNaEyJKPmdl0BuU4bLU8XDXJy9qw7zVFMZFxG5W3PQQ5sPKikvMw6fvR1yY3P7LUsXWaAwKUWsYSdAofwd7RRAKvAIugygdsp4GvK1YOQWax10FOMa/jiiaBs6wYz7G0Ejnrl2wlgIdu4RHGlOWK9oWaNHo6sHThoK92yvafj4k0D7dwazOB6yB/cVDf3QmbKC1ZYXZDPc0xmLNzSA6dvH/N9Y3q651tb3XYVE7q660NCOzbRPnkQIobyLCgMrUO4zEOXPXR57dgOlOdAg6CL7MGqgPRouBya+hTQIBHeHu9Z9e0KtsZrt3uRgbVbp9KxlLyWhNC5z9tLYnDt9qtpLJZh6loLmuC5K1nABtcfr8oCu1+QFyU3y5Is6TbtG8e5F3bRDYX9feYD6Ma+n0wb76Btdg1UALBO00RfL0hSqTT1wjHDbs2qMyAtv3Ncwoz6DiX6Y4sQreSfX1THIDEvn9zm/q9jK0LZ/ybuni9tGQymhBh7Qr5S0dLezZTcy9B86sqdxR1K1MRW355VHzU+99T43M3N9qbZZIwhaUe28CFRiu0apqrgVpY1HSxs/AaoN/qhBY0fhOP6G4DksqzOWA96wm4s12ZKRe8h3DaAjAicQ4TbvEkAD6Q7qn7USfvdJMQfdOIk8fOHm7z3PJngE3giSRYk7C4difFjsf5kbHBXgSZFqjVlrIvWzWSCjakYjzTEWFnDAzP2NT40W9/BIK257twugVeRAMa+mTgna08uuUxRpqZ6kdPCkxg+5gZB0pu2BnrTpjSf7+znBu87sQt0Bwlhdru6sUI9IP/rm50crxg4GEkTH6juPVQ2n9YSF70H5v0KxrfNNbR1CIRfCmn4b5HMJuwsftOsfoC9ppcvHqRzWdGbg8hoZhCH3iBamb2L/iF2BjU221b3HmTg/7fCoL5Q7KDqcmk7PjhjlcpuqdemLPMmW68Z5dW+aqsb7r+jKJnvjjyta7zN26DxqcF1WS77E3FQ+E22Ssqb+quCxoOU6klEktbXTbnW+xOJQXKZF/hpDRPckYUEuKofsub5JiLf8lKm2jSt51ViSXh3lIgGTEiCPjorkaTbLEbtJ7mFe/i6BJ55l4tJ/zAPk8JX5MiDJXMMjsYan0/4wRTuy33uWdPsPWtawrOmtUQ2w481HT2nGQ6K/zAu8NHvm8VsSPquL/uJdHTuktNTvQLproceySzHhsJ8yK4HAQaYoQ1P3/R4eRorj9c64dDTaHs66dDTfPsCWCgzQB/lwCWh5B6ZJPbgY+V7fMxt9B43FYtSjPvZJ/UomNm5+bfZ/0XjNOwd7+vdwiaXohb6bbE+ZzFzrgfEbhZVxfkTfHYnpcJvGK5U0t5w38vifYqm2g7CcPyMC035VTXuwr+8/l4DrqRIgfhImEnZZnb+/wEAAP//AKMAXP88aW5wdXQgdHlwZT0iaGlkZGVuIiBkYXRhLWNzcmY9InRydWUiIG5hbWU9ImF1dGhlbnRpY2l0eV90b2tlbiIgdmFsdWU9ImppZWg3V01GRlVIUjUrSFlGN1Q5bS9yVkhFaXk2N1hzV3d4WG9QWGFndlhkNHl1NUw2TnluSU5hd0FpRzNnZGJnZEVuSzlDSEJML2R2dk9ZcERFUllnPT0iIC8+7BvLkty28a6vgHlwLgKHAN/S7rpsuWI5JVdUjqOqnBQQBIf08hWSs7ujlKvyG/m9fEm6AXKG5MxKK9uxveWsRkOw0QAajUY/gJ4nZPZ3keyGoamJLEXfX1rJUNNGDoUE0KxMU1FvVUeGpimHom1VOivS+tYiw75Vl1a/S6pisJ6QxZ/oCkFLkajy0vpbsyPVrh9IokhfbGvoqaihM1KJa0WajrRd0za9IjLHIXuLpGIQNC16kZSK3hZDfrXq/qK/2Zox8iJNVX1pDd1OWSRXxTYfLi0WWOSmULdfNHeXlkMcwgKiYarriwbQmc0sclukQ26Q9YjYgsqmapta1cPU58imiUUTe4ZO9LllCLtoxZCTrChL2u1KYIq6UXWTptDvpfVNYPuE2aEvbO7DB6hxGDwp/M+5vQbD542bU/cNNnlXedDYAQAHuAAIfDQmTMn2X3Ld7QxKEerrti98YoeRR3z97ZBAI+GIL5hjc+AI9MEMDjz1cN94thcHgBrM+9XdejG3mV/aga7+HNF1GxzX8W3PgVc/920WSzvGRnbgc2oHke7aY0hZGGB7Cu1nRI+dU5t5cUnH/udMAUZ5sc2BWxS7P6miWKdbYs/vrKuLDa7H1ZOLDYjJUnQuNkb4sTJruurKANPiBvHGgllTKE5rn9KsVHcEZE3RcmuedyVBIG26VHWUm5dt19xSx5rGvEjVIIqyP/QDgp42tzUZ4bRTvRoObw2IZyn2JKVFXRa1oknZyGvrOIOLfldVotuf2brWuB1E3zbtrp2Ed74PYY/1BewoIqBJU/fWnDW/gR11rRKR0LzpindNPYhysblwI0UkFkxvJiOVDnXJ8d1xHOK++wZfPoxWMZc4H8S6V5Tg1azEKCwativX60wrVe/I4o32oDn7YY9aQnPrGWGh395ZKzkti7XOA6A4GaAYVHWviibfo4RVIFZ0V5eq72lbigHFfq2u9Z9esQmjR/JqGKN/Wgl5Fj/vVHZp5cPQ9s82m1T110PT2ltQ17vEhkW/f5CtoLIs5PWl9a0CqSyGpts/JU2raoLanox9WacsIOTPB6wvDdYpmzZivemX3Dxh7j2MtcYpbl7vv2s6mb/CrVAX9XbT7gcE0PIA6cTtphL9oLpN2sh+0ze7TqrN234QIN2bohJg2TZls23stt6ezOxLGLNsRLqiezERM42ZDO7Kg6LZjDpk3DJGj03qbER6Mio1nBqaXNiO6m44bEfY2TRp0j1pqUNA8yT0thMgSZ1eM23tKTYgZFv0A6BqxYfK7qDtjhoTEamEPQ/NW+quVE0rjt6H6QaHgt1WVFvSd/JhLEcaP4Lnn8ESXY5asQRddFwL2M1A0YyXZ6zBkZVPnpxR7O/X58t38Ky6a4sUoNK+31Xt0FCt7CestBBA2qT8Jo2v903eDNdqD6QvFfufoBf0p7AbPZtJN5lVWfQ7W22CKz4zOtrGaTMmm3JX1UTURUUzAcYO/LUMGK1HQJrfN/7Exk8oJX+w/k4ovdLliw0KheiUABLvqvZK12yaFo3R1WiOL/B7ohHUFw5GJw7puoOYGoKt0ZpdWliSqgWxy0UHy3Bp/fW7P9LIIpUa8gaYvVXD3JYWdbsbpqGwa5h2PXTNaNfFDj3UDqzCNGlqWpxQVagynbxhs6NAiUqVNyVI9pI7n+aqLIv2+Xv4R3DgrJG7fkbr6LFrIZAl+MrjauKsF274A8wuOA2aDfD5qtHyfJ9TtBCccUdMaubJUbs8OVE4pn0linq5j+b1o06aK42saVBf4DJAU9AK4GNpG0mzXVmCHoHtBVPqixuYDSwUTBtxYYpFnTV6UvPutGUB0YBmYNSg0ULG0VPrFLovygDKLcJMGdXRAVo3x9fvIY4psv2k2iaUCZyo4VaBfQIpDEibgFtYYSkLYFJl02n9SXsFVKe4qUflB0ZsrB8BR4xJDRx9ixKVr3YggLB6OSdNJ/REGdcPoMz/KLorTXKCVTMvtiymwfV+gC9T/6ls2v1zwh3OyFfF8HKXPCVf19Je2tt7m4PVXbkQoMRn/sNG1RvztoGFVLRtwGfYjyBgJTgptMmAWd1NIdUo+vkeTPjkXfzz03/smuE5RmPDW9wn5v2ZeYhalHswFL2tEQzwqXm0Yo/meMQf+5FiUFtwVRa9/FEL7aKx0UgLrG2DG1zTvEDVGmCBiSLy7BQRvOJtUYMY19u3u27ZZOLfjHU/v/lcULMDpr8tJvbUsDt/+OF0AWgOruOlFfoR5zKVcZQGUZSxyGWBFEmaqYSFmefzIBOSqyDO0ohlQcjiLOFJJpXyoyyRICnfITvQG/pFJKvtihsh9xT5oMBnHx6DaI1EP0S4zqE+YvEKROhzJ+FB6Huhk/mey91IAshx/CiM3CAMlB8mjgyzVPnSzxwpZZQpT7ppkvHIunptGPJxAvbbFwmwIruuGB4kE2dxH7FQxF7AA5UyNxCBZEw5QigZpnGUecJT0kmjOAlk7LluKDI/kEkcezFPfHB2vZAlmbVSILNpTqyyrv4yln6KZrq9vR0VE7Jg1+shHoPCMeQ+SLZOMR+xZHmoYUTsuyJynTR0My9RYeqrlDMRcinTwE+YH4OVc0PHYzII0lTFXgTi5nMuMhAbzY+zQnNQLMcTErMmT4nhOsQP7VOi2folzHItpis7Z10h0nIkc2xgimIRhrxsKtUClyCkKAZ0ro1PZy3d88NdQTr5nyDZ+qCSVNqn9O7fO0eX8gOnjdz7iNNGRP7400aIkK+pIe1Bp/gRcV64th/h4SAZC5HURY9wm8cksH2X+LYXktD2Y9uzHXj61Gbm4UILeImpjefWEddPPPqmHEq2G8LTd6kBBIAWe4CCp9sIjMw7NIOpA4ADgPl4vk5t3/QFre3AxSJhthMhiQwaQxs75FhEeBTCB+h1ATcACrG17gwqfKQHQMwOAcApoOCQsSm4duzjDADR1af5MVLDNRWOxrfdADvg+ApwrlllBzAGDA2jICXYI0PCXMDnQAl+wyz0jYELFMWEE6xgyAzoTTMXJz49bc+DSvgHImHHOBjBwYB62w8Qj2GPGp/5eq30lGDWrh36OBef4FxgyfAmwffsUK8hLIWhApmieRgbZsK4OA8OQ8Lqwir7sJafR1DlEvON58VADogD9WwPGedHVP87PUI+HOr9qjHdx5iofte2TTfM9Mpng9j2l2kzQJkatfAYTJY+WJDDQ2zWOdRHbLSCIA64EE7mBDyOIQDjXLHUZ1JmUrh+mkTMz7xUqszxndB1Qh6xwIviMHMj0M8gFS8MQ8Y4/6d4PDMWQSQigQ+PQXZeG1IfIjvnUB+x7KRpoOKMxZ6MnFhlSZh4keckfhbziIcyZCLIMplFWej7fpy5iYd+kAtOt2SArXR8hQxZC82PDt8fg7yItniIrKzRHrGcMB4zEAvhoHJxuAxY4idxIF0vCzJ0l73M8yIQHRCZ2EtcnoZSepkKYs4j5XPr6vPXX/94GRmPAR+bnAydKOoHKpazuI9YYjIwMWmasczxZOoyx3EC1xfCcwLBElemCkIn7sRRKhiLmQyl56gQtApYM6ZcJqyr70aO/Az2KMFLt0cgMEjnQ4TlBO8RC0oUJk6QoPGJHZ75LPOkG2Wp57oQZTMGUsOyMEiV8DiE48J1Yskc7qTckwEPeWxdfQFE/LiIWyTNbhhDbl1+z9GQqb/6HB/nou755fJpetE5p75NaHC4Cp5dmy9vlABzKES3n19lr3MADrf/EMyK78UdVV0H7SvV9zrmHzs+rQK6RJ+bb1NjERO1T3T96plDogQGPCyKtznXaWveLI0MglqIKZ1XDMJuCE1dCLD5vBraQGga5RBkgle8qsGoN3p17PddhfEzxuGhE0oINmOKUTiGsR4PxzLEpFFAnBLiXQxYMfqPlil1jHmmzn8JgSiLxLIW42/PxROD0H8V2J6PAbPjhe++iaE7AdGrzu6jGLmaFwfjcSAOMw/9ZTKhzhokzo1ORjxCDRDHOJ8INd1Kmwto83I8LNLiou+q8cZ8JlZp0VdF3y+vwL80QGLk66HHRL+AaN09SKxcPGHBr3kyJiwIrPGriAR27JUuygp+zfIqDQp+vYqxFGksssaiR7SIIGJJz+A5Bk9/vcIhSaRJIp4dRuskUeeePMljKgC+YeauFPV//vXvgbSq0/kXQy6GMeeCQGnIi54MRaXs5U3/TL3B8oNBKBXtQaHgffxMoYxq5VZ09VhMRF2rg4rRZPxfz/y+9czc8J0XJmryymlRH2wTyu4x21wnDIq6GXJMaRcJpp2b/EabHDxC6+pbhR4bmm40/QDsFMjoHtwaMo5nz5PFPoIy4zickAZgJO9npuzgZUARk/1KcDq14de33WcSzX56Phspc+g4E7tymHsmbVfo9LX87m3XQ0CGOZ6rzDaTyDOZj7lReKGtx0Tm/zKvzZA3dTW+VU16SEI++dWETkOb/XKiwqTJ2fsh+UgkPYwK/O9QTQHW0LQQh6ys5r3TPs35mqeD/+oa8XdmHtcGcmnogCkNxHe0bwu0YaTa0wCdn6WkTtU4wHGbvj/bDnMnzTY+8+uE102Lwo4j5ViQokvpmBh3KoeH7HPww6DH/TOCVw7PURNh/uH4aqEqAi2kUHJOMuvGEQ8Rwuqd0qQBHlW0VNlwWlmKDtrgRjWKos8FaDsDXuXGu4HT3j23zsQzZ3RbjXnvA0ZxbdIYHrR7Ou6w9RTeqa6ZY44phx/ct+NyLQdZbl8AHPbLiwnvC6M9QEXgOq2aV2A/Wz3EMXu/AbGB5ZkUAE4lUypNBIaoMEah0k/GyrEZiE6nxmTY29+SD93iEcFq2lCCGY2qkz9Ii+gdyhb+hCnfuNIhtgeejOsGBgSf3Fu6GXj5h8jUXXeB/3MK6O8qBLtvwBXJ3TdeTl1wZLgdhR54bkG4bod3sTxeOVSO1nT+TWz7SFUcBOYXVgcc/Moj2181Yx56hBzog5ZLL0xfunroPDF//qMsJIFoEuY/IWM8sDkLkYDVT6WwnNPoFE4P4646wpvb8F5V+OtLlsyVvF5JFoKMYM1zbndSgv6ZkhYeKnKwImEEFmJpSPQlMViZEFkVrn6YR0e7xtEExTaP1jZvtECwgGj3QDo4Dc6axvsitKXyWOjGmZUY87IxVR5r8qHCtI//AgAA//8BAAD//7J6XNJVEQIA){height=\"60px\" width=\"240px\"}"]}, {"cell_type": "raw", "metadata": {"raw_mimetype": "text/restructuredtext"}, "source": [".. customcarditem::\n", " :header: Tutorial 10: Autoregressive Image Modeling\n", " :card_description: In this tutorial, we implement an autoregressive likelihood model for the task of image modeling. Autoregressive models are naturally strong generative models that constitute...\n", " :tags: Image,GPU/TPU,UvA-DL-Course\n", " :image: _static/images/course_UvA-DL/10-autoregressive-image-modeling.jpg"]}], "metadata": {"jupytext": {"cell_metadata_filter": "id,colab_type,colab,-all", "formats": "ipynb,py:percent", "main_language": "python"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.7"}, "papermill": {"default_parameters": {}, "duration": 73.044444, "end_time": "2021-09-16T12:43:30.306539", "environment_variables": {}, "exception": null, "input_path": "course_UvA-DL/10-autoregressive-image-modeling/Autoregressive_Image_Modeling.ipynb", "output_path": ".notebooks/course_UvA-DL/10-autoregressive-image-modeling.ipynb", "parameters": {}, "start_time": "2021-09-16T12:42:17.262095", "version": "2.3.3"}, "widgets": {"application/vnd.jupyter.widget-state+json": {"state": {"01769afc187444bfaf2aec03b4f08496": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "0640c19d24424bdd9409f8216335ba18": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "0bb5668432394fb5befb04fa0f5d4605": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "1006aae5c5444b3583ff98233ccb55d2": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "14f0fe5dfbfb47b5bdb2c070ef6a31ee": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": ""}}, "1c756dfde210415c809395bc6f9bc1f8": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "1f6f6bfc18304ecab2062b2bf6933f8f": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "1fa667b5112343df9c5e37e6d9ca6b1f": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_0bb5668432394fb5befb04fa0f5d4605", "max": 28.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_5dbc98f3633e4245acdc824506aebf18", "value": 28.0}}, "1ffdceee837a4a4d9b92dc9983c6feb7": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "21e083e4f6784332a4e4be80c9719372": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "288775f9620a426f835527c2a20267f5": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "2990971999c9429baf82e5715967d2e7": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_3542c1b421d74a9c9673ec4a04fceba9", "placeholder": "\u200b", "style": "IPY_MODEL_f3ca5bf492e2434080c0eb2ee7d333f0", "value": " 28/28 [00:02<00:00, 8.81it/s]"}}, "353657f372f84344b42e3792af9f53b5": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "3542c1b421d74a9c9673ec4a04fceba9": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "3e194c3fa88d4f7796e12faf5a612eaa": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": ""}}, "443488f69eb4484d8d014daa1df90910": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_01769afc187444bfaf2aec03b4f08496", "placeholder": "\u200b", "style": "IPY_MODEL_3e194c3fa88d4f7796e12faf5a612eaa", "value": " 28/28 [00:04<00:00, 5.26it/s]"}}, "4588a9d7ce4c4420b31d66495762c51d": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": ""}}, "47dbee09f7744686a6f2811eabac4a47": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": ""}}, "4a41e2c8c24f4501ad3f27256fe2dd00": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_afe9d9293fc54046b1a23a652de9156c", "max": 28.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_e5e0bffb840a4e6bb1718c6edc3478c0", "value": 28.0}}, "580c4ca4d0104b1fb3d44477534e266c": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_5df1358fcace453c90d53fc8bf0c6852", "placeholder": "\u200b", "style": "IPY_MODEL_14f0fe5dfbfb47b5bdb2c070ef6a31ee", "value": "100%"}}, "58c7fa2a2b544e4995676502aab755f1": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "5dbc98f3633e4245acdc824506aebf18": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "5df1358fcace453c90d53fc8bf0c6852": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "5ec4fe6328c247b489d2701b79eb8048": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_f06e46b2cceb47d6bce7e6fb0deece49", "IPY_MODEL_4a41e2c8c24f4501ad3f27256fe2dd00", "IPY_MODEL_7b6d55f1c3e94df5a3c638a6963f180f"], "layout": "IPY_MODEL_e0e996dd55e54974abbba367a08b200d"}}, "609d34d968c74b98bfde90b5c8853bb2": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_1c756dfde210415c809395bc6f9bc1f8", "placeholder": "\u200b", "style": "IPY_MODEL_47dbee09f7744686a6f2811eabac4a47", "value": "100%"}}, "6678217ddd654f39bdbc8fba2cd3788f": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": ""}}, "6ddeeb369e0b46dd9e9a13c20c943fe4": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "71addce685174d8c9ed888c0e5e10e93": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": ""}}, "7b6d55f1c3e94df5a3c638a6963f180f": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_0640c19d24424bdd9409f8216335ba18", "placeholder": "\u200b", "style": "IPY_MODEL_4588a9d7ce4c4420b31d66495762c51d", "value": " 28/28 [00:02<00:00, 8.84it/s]"}}, "8961cfdd02c24f6dbc1c42cf6db817fd": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_d80e197eb464480d9f8d2b14d7d47169", "IPY_MODEL_1fa667b5112343df9c5e37e6d9ca6b1f", "IPY_MODEL_443488f69eb4484d8d014daa1df90910"], "layout": "IPY_MODEL_aa387ed113584652801c05449d0576f4"}}, "9162e93ab9e348f48b2f46cce2c72155": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": ""}}, "a04db2d75a3f476487be88f18d66ddb1": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_609d34d968c74b98bfde90b5c8853bb2", "IPY_MODEL_a70e1a44ecb440e8a79c4735752761c2", "IPY_MODEL_a8f4af858d4d457b8d2c2c59506c257c"], "layout": "IPY_MODEL_c5d74f5e252d446b838c3a099f2534f4"}}, "a70e1a44ecb440e8a79c4735752761c2": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_b91e999fdf704ceeb051bf05e4d0728f", "max": 64.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_f63d7ff436734e8dbe1e7ea69988a2e1", "value": 64.0}}, "a8f4af858d4d457b8d2c2c59506c257c": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_353657f372f84344b42e3792af9f53b5", "placeholder": "\u200b", "style": "IPY_MODEL_c142ac0ce8e6411ea84868b82e3112d3", "value": " 64/64 [00:38<00:00, 1.02s/it]"}}, "aa387ed113584652801c05449d0576f4": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "aa4ad04083b642e4b289e7350fbef888": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_1006aae5c5444b3583ff98233ccb55d2", "max": 28.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_d320a03105ae4a43a4d3ee1ec4c2cf75", "value": 28.0}}, "ad3bbb05cc344a59a95a7a6ca63461f5": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "afe9d9293fc54046b1a23a652de9156c": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "b91e999fdf704ceeb051bf05e4d0728f": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "ba1f0b2b45fd4c3ca18dfe2954a6747f": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_1f6f6bfc18304ecab2062b2bf6933f8f", "placeholder": "\u200b", "style": "IPY_MODEL_6678217ddd654f39bdbc8fba2cd3788f", "value": "100%"}}, "c142ac0ce8e6411ea84868b82e3112d3": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": ""}}, "c1e3f5021d764025a1e5dfc1a7acb318": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": ""}}, "c5d74f5e252d446b838c3a099f2534f4": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "d320a03105ae4a43a4d3ee1ec4c2cf75": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "d80e197eb464480d9f8d2b14d7d47169": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_6ddeeb369e0b46dd9e9a13c20c943fe4", "placeholder": "\u200b", "style": "IPY_MODEL_71addce685174d8c9ed888c0e5e10e93", "value": "100%"}}, "e0e996dd55e54974abbba367a08b200d": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "e1cd7fe63e8646ccb242422e29b75572": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "e5e0bffb840a4e6bb1718c6edc3478c0": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "e76d7eb2829b4c6c9b26ff6703e0787d": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_ba1f0b2b45fd4c3ca18dfe2954a6747f", "IPY_MODEL_ebd8b932d6ce4c76b14940b50d2d2569", "IPY_MODEL_2990971999c9429baf82e5715967d2e7"], "layout": "IPY_MODEL_1ffdceee837a4a4d9b92dc9983c6feb7"}}, "e973aa8423374ae9895e0f38003d662f": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_ad3bbb05cc344a59a95a7a6ca63461f5", "placeholder": "\u200b", "style": "IPY_MODEL_9162e93ab9e348f48b2f46cce2c72155", "value": " 28/28 [00:02<00:00, 8.85it/s]"}}, "ebd8b932d6ce4c76b14940b50d2d2569": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_e1cd7fe63e8646ccb242422e29b75572", "max": 28.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_288775f9620a426f835527c2a20267f5", "value": 28.0}}, "f06e46b2cceb47d6bce7e6fb0deece49": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_58c7fa2a2b544e4995676502aab755f1", "placeholder": "\u200b", "style": "IPY_MODEL_c1e3f5021d764025a1e5dfc1a7acb318", "value": "100%"}}, "f3ca5bf492e2434080c0eb2ee7d333f0": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": ""}}, "f63d7ff436734e8dbe1e7ea69988a2e1": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "ff6c8e20790c444e8f7c56fe0cf6f5bd": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_580c4ca4d0104b1fb3d44477534e266c", "IPY_MODEL_aa4ad04083b642e4b289e7350fbef888", "IPY_MODEL_e973aa8423374ae9895e0f38003d662f"], "layout": "IPY_MODEL_21e083e4f6784332a4e4be80c9719372"}}}, "version_major": 2, "version_minor": 0}}}, "nbformat": 4, "nbformat_minor": 5}