{"cells": [{"cell_type": "markdown", "id": "c1ca2a69", "metadata": {"papermill": {"duration": 0.179789, "end_time": "2021-12-04T15:53:22.277549", "exception": false, "start_time": "2021-12-04T15:53:22.097760", "status": "completed"}, "tags": []}, "source": ["\n", "# Tutorial 1: Introduction to PyTorch\n", "\n", "* **Author:** Phillip Lippe\n", "* **License:** CC BY-SA\n", "* **Generated:** 2021-12-04T16:52:44.026092\n", "\n", "This tutorial will give a short introduction to PyTorch basics, and get you setup for writing your own neural networks.\n", "This notebook is part of a lecture series on Deep Learning at the University of Amsterdam.\n", "The full list of tutorials can be found at https://uvadlc-notebooks.rtfd.io.\n", "\n", "\n", "---\n", "Open in [![Open In Colab](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAHUAAAAUCAYAAACzrHJDAAAIuUlEQVRoQ+1ZaVRURxb+qhdolmbTUVSURpZgmLhHbQVFZIlGQBEXcMvJhKiTEzfigjQg7oNEJ9GMGidnjnNMBs2czIzajksEFRE1xklCTKJiQLRFsUGkoUWw+82pamn79etGYoKek1B/4NW99/tu3e/dquJBAGD27NkHALxKf39WY39gyrOi+i3xqGtUoePJrFmznrmgtModorbTu8YRNZk5cybXTvCtwh7o6NR2KzuZMWNGh6jtVt7nA0ymT5/eJlF9POrh7PAQl6s8bGYa3PUum//htmebVtLRqW0q01M5keTk5FZFzU0oRle3+zxwg5Hgtb+PZiL/ZVohxCI+hL5JgjmfjPxZ26+33BG3dA+ealHPM4gQAo5rU59gsI8bRvl54t3Ca62mvHyUAhtOlLd5WSQpKcluBjumnoCLs1EARkVd9E8l3p9y2i7RbQ1B6pFwu/YDgW8KbHJHMTQrwnjz2oZm9M4pavOCfo5jWrgCaaMVcMs6/pNhDr0+AMN93XlxV7R6DNpyzi7W/OE+yIrsjU6rTrbKV5cd/pNyItOmTbMp6sbBB+EqaYJY4cWE3VUciNt1TpgfcRFv71Fi54xT5kSoyLvOBEJMOMxWXkFlBeBSX4u6Zkcs+3KszYRtiapbNRqF31UgetVuc8z9vBXIv1qD+F1f83B6uDlCUyfsZGepGPpmg01OB7EITQbhS9ribKy+DmP1DUiClLz4bnIHVOqa7BY+Z1wg5g3zgUvyehiNpnJKxSLc/ts76LKm0BzX3c0RNy1yXjDcB5lWoro4iNHQxM+f1kWeWQARAWQS++trISJTp061Kep25X/MycwtjuctSC5rxo7ppi7VNUox5+PhPHtrsS2O1qJ6yx1QujQUzm9sh6hbkBlvvGcN8hYnwjUjH6kjfZEd5c/jitz5Jc5U3ENnFynKl4eB7nyEgP2UZ+Yz3/rVEbyYr27qELrtC4FIC0J7sc7xWnmccdHfRRTs0VB+cA4lt+oFcRR/wUeH8FG5w2Mbx8FQ8TXEvv1xYf4wBP3O2WyL3/UVjpXWgIqaFeUPr+wTmDvUB7njH6/bOv+HRg4SqioAg5GDe1aB3ZeMTJkyRSBqkLsWqSEm0fZVBEN94zEZnYvrdx1JL5cxe+a+AbhSJecRRHW/ikTFRTa38dtQlNZ5CRKwFvUtZU/kvBoEF9Uxni/XqIM+dwKbTw3rhcxIf7gmr2M+H6SMwx8iBzJbw5oxeG3Lv5FX9B3AGaHPS8e8z77H7v9VMpvPG5ug1enh7eGK8h0LBTwUb+GInqzInlRUK65DmTPQu4c3+uQKjwKK77zwUxBX4Tq7yR1RuiwUsqlrABCM6esHdXoy47fk4+prYKy8ZF574x4V5BnHQBuf4g9Z9ld8U36L2aktZNNplNfw7zotwWTy5MkCUft4aLEopJj5/OPHl1BQqeAVOnHgNSQOqmBzq9V9cfEm/yx5ubMGKS9cYPZ3vx2OS/c6PVHUuUO7Y1Pci3BO/1zgq18byebfGemLtNF+6JRtOvMk926ibussZqM+1mNz4TWkH7rCbM5phwGRGDAaoF8fY5OHFnlldAA8sgoEXKnDukA1NgSeNjqkJT9brbN4pC9WRweYXyLugR73c+MYvyWfu0yC6+mjzN1Isfw3FKJS98CU/zI1IHFkFPR52cHL2FJk0sB6kMTERIGo9GzcPkLNfA0cwdwi/hfEYO86ZMd9w+y1egfM2T2Eh/vesMNwljSzuZRT420SW3eqy8N6aHMmwmnFUZ7/PGVPbIoNZvNU1BURdHs0bT2+HjL8sDSM2e6vi4Lj5NW8WOLVA6RTT2azxLV+bglaFNqLieqemS/gWkw7NyoAHo+2dEsiivengjKsPFoqWOvbSh/kxPaxyW/JRzH2Fl3EzD9/xjAefJqB3usKUFn/0Gb+S/d/jy3FN2yLOmnSJJtn6oehByEiHPSeXnDxFGPRnoFoaBJjcdQlbDwcjL1zTNuQpoxD7R0OG0uUTMi0fkVwdzBdYIwcwZunxrVJVLplNm54BZp7jfDfYLoNyqQi1K6KxIdHzmN+QQ2WjFIwUT2zTGdlRXo4NFXVUO4sgX5dFC7f0aP/ZlNeUjFBuL8Xjl6uRuP6aMjSjpjzsH62FDU7JhBuGccEXIvDfJFFBc/gHw80dklfCVYnRaDfpiJcutPA4F7qJsfJeUPQI+1fqMlNhFx1FM0GDqkjFVg7NojlQ0Vt4aM5ReSqcbpaCg8nCW5lRsBvbT4T1TLfFptsfh7gItzuKTdJSEiwKSrt1vcmnEXXrsLbYnWDA1bu+z2WKy9Arq+1KRqdfKsoBo0GcdtEpS/B1bO4v0cFiUhkjskvKcMrWwtAPHuwQq8Z+4LZ1vTQANfXt4J0DwZX9gWa9qh4XDM/voC9JXfwYEMMHJcfNtusn82ihvliVUwg5KrPGVf6GH94ZJpEZBen6EC4qYTHA1dXhW0JIex8txzv//c8lhzXIi/BFxOH9jGbQhZsRalTIBZZ8KkGyZAxeRQvXkFF1TWz/Hm46jNYUnjPbt3JxIkT7f6dSj8qfJJyVvBxgaIlblOyjtysNHWN9fjjqWi7glJfW3/S0Hlj2XnA8PhKT9w6g3Qx3XiXhvuxQsuT1proxBKI/AaZqY1Xz5muvY8G8XkRRCaHsfQsRAFDH/tZPbcYuHotOG0FRIqB4HR3wNVoIPLtz8ycTguu+jpEigE218vd1YCr5m+HpHMvEI9u4LTXwNWaLjl0iPwGAmIpeHx1VeCqTJdPs1/vweweQPO3HC24NhOhnTphwoQnfv6QSY2ICbkNmdSA4h87oaLaiYfn5diIEd4att2erOwJXbPUHp953p6orQVSUVWRAXBT8c/dJ5L9xhzaJGp71GR/wFP8P5V2z10NSC9T93QM2xUg8fHxT+zU9ijeU4naHon8CjFJXFzc8/kn+dN06q9QgF98SYSo2Xen2NjYZy5sR6f+4nLSK5Iam2PH/x87a1YN/t5sBgAAAABJRU5ErkJggg==){height=\"20px\" width=\"117px\"}](https://colab.research.google.com/github/PytorchLightning/lightning-tutorials/blob/publication/.notebooks/course_UvA-DL/01-introduction-to-pytorch.ipynb)\n", "\n", "Give us a \u2b50 [on Github](https://www.github.com/PytorchLightning/pytorch-lightning/)\n", "| Check out [the documentation](https://pytorch-lightning.readthedocs.io/en/latest/)\n", "| Join us [on Slack](https://join.slack.com/t/pytorch-lightning/shared_invite/zt-pw5v393p-qRaDgEk24~EjiZNBpSQFgQ)"]}, {"cell_type": "markdown", "id": "8387d057", "metadata": {"papermill": {"duration": 0.177673, "end_time": "2021-12-04T15:53:22.631831", "exception": false, "start_time": "2021-12-04T15:53:22.454158", "status": "completed"}, "tags": []}, "source": ["## Setup\n", "This notebook requires some packages besides pytorch-lightning."]}, {"cell_type": "code", "execution_count": 1, "id": "9d8201a6", "metadata": {"colab": {}, "colab_type": "code", "execution": {"iopub.execute_input": "2021-12-04T15:53:22.994731Z", "iopub.status.busy": "2021-12-04T15:53:22.994231Z", "iopub.status.idle": "2021-12-04T15:53:25.459580Z", "shell.execute_reply": "2021-12-04T15:53:25.459996Z"}, "id": "LfrJLKPFyhsK", "lines_to_next_cell": 0, "papermill": {"duration": 2.651811, "end_time": "2021-12-04T15:53:25.460282", "exception": false, "start_time": "2021-12-04T15:53:22.808471", "status": "completed"}, "tags": []}, "outputs": [], "source": ["! pip install --quiet \"torchmetrics>=0.3\" \"matplotlib\" \"torch>=1.6, <1.9\" \"pytorch-lightning>=1.3\""]}, {"cell_type": "markdown", "id": "3ef762bf", "metadata": {"papermill": {"duration": 0.180076, "end_time": "2021-12-04T15:53:25.821045", "exception": false, "start_time": "2021-12-04T15:53:25.640969", "status": "completed"}, "tags": []}, "source": ["<div class=\"center-wrapper\"><div class=\"video-wrapper\"><iframe src=\"https://www.youtube.com/embed/wnKZZgFQY-E\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen></iframe></div></div>\n", "Welcome to our PyTorch tutorial for the Deep Learning course 2020 at the University of Amsterdam!\n", "The following notebook is meant to give a short introduction to PyTorch basics, and get you setup for writing your own neural networks.\n", "PyTorch is an open source machine learning framework that allows you to write your own neural networks and optimize them efficiently.\n", "However, PyTorch is not the only framework of its kind.\n", "Alternatives to PyTorch include [TensorFlow](https://www.tensorflow.org/), [JAX](https://github.com/google/jax) and [Caffe](http://caffe.berkeleyvision.org/).\n", "We choose to teach PyTorch at the University of Amsterdam because it is well established, has a huge developer community (originally developed by Facebook), is very flexible and especially used in research.\n", "Many current papers publish their code in PyTorch, and thus it is good to be familiar with PyTorch as well.\n", "Meanwhile, TensorFlow (developed by Google) is usually known for being a production-grade deep learning library.\n", "Still, if you know one machine learning framework in depth, it is very easy to learn another one because many of them use the same concepts and ideas.\n", "For instance, TensorFlow's version 2 was heavily inspired by the most popular features of PyTorch, making the frameworks even more similar.\n", "If you are already familiar with PyTorch and have created your own neural network projects, feel free to just skim this notebook.\n", "\n", "We are of course not the first ones to create a PyTorch tutorial.\n", "There are many great tutorials online, including the [\"60-min blitz\"](https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html) on the official [PyTorch website](https://pytorch.org/tutorials/).\n", "Yet, we choose to create our own tutorial which is designed to give you the basics particularly necessary for the practicals, but still understand how PyTorch works under the hood.\n", "Over the next few weeks, we will also keep exploring new PyTorch features in the series of Jupyter notebook tutorials about deep learning.\n", "\n", "We will use a set of standard libraries that are often used in machine learning projects.\n", "If you are running this notebook on Google Colab, all libraries should be pre-installed.\n", "If you are running this notebook locally, make sure you have installed our `dl2020` environment ([link](https://github.com/uvadlc/uvadlc_practicals_2020/blob/master/environment.yml)) and have activated it."]}, {"cell_type": "code", "execution_count": 2, "id": "fd4e9d73", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:26.189594Z", "iopub.status.busy": "2021-12-04T15:53:26.189116Z", "iopub.status.idle": "2021-12-04T15:53:27.107430Z", "shell.execute_reply": "2021-12-04T15:53:27.107817Z"}, "papermill": {"duration": 1.106304, "end_time": "2021-12-04T15:53:27.107981", "exception": false, "start_time": "2021-12-04T15:53:26.001677", "status": "completed"}, "tags": []}, "outputs": [{"name": "stderr", "output_type": "stream", "text": ["/tmp/ipykernel_739/4100430984.py:14: DeprecationWarning: `set_matplotlib_formats` is deprecated since IPython 7.23, directly use `matplotlib_inline.backend_inline.set_matplotlib_formats()`\n", "  set_matplotlib_formats(\"svg\", \"pdf\")\n"]}], "source": ["import time\n", "\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "import torch\n", "import torch.nn as nn\n", "import torch.utils.data as data\n", "\n", "%matplotlib inline\n", "from IPython.display import set_matplotlib_formats\n", "from matplotlib.colors import to_rgba\n", "from tqdm.notebook import tqdm  # Progress bar\n", "\n", "set_matplotlib_formats(\"svg\", \"pdf\")"]}, {"cell_type": "markdown", "id": "e6ab2b50", "metadata": {"papermill": {"duration": 0.177291, "end_time": "2021-12-04T15:53:27.467274", "exception": false, "start_time": "2021-12-04T15:53:27.289983", "status": "completed"}, "tags": []}, "source": ["## The Basics of PyTorch\n", "\n", "We will start with reviewing the very basic concepts of PyTorch.\n", "As a prerequisite, we recommend to be familiar with the `numpy` package as most machine learning frameworks are based on very similar concepts.\n", "If you are not familiar with numpy yet, don't worry: here is a [tutorial](https://numpy.org/devdocs/user/quickstart.html) to go through.\n", "\n", "So, let's start with importing PyTorch.\n", "The package is called `torch`, based on its original framework [Torch](http://torch.ch/).\n", "As a first step, we can check its version:"]}, {"cell_type": "code", "execution_count": 3, "id": "11531bd1", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:27.828703Z", "iopub.status.busy": "2021-12-04T15:53:27.828236Z", "iopub.status.idle": "2021-12-04T15:53:27.830237Z", "shell.execute_reply": "2021-12-04T15:53:27.830613Z"}, "papermill": {"duration": 0.186067, "end_time": "2021-12-04T15:53:27.830761", "exception": false, "start_time": "2021-12-04T15:53:27.644694", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Using torch 1.8.1+cu102\n"]}], "source": ["print(\"Using torch\", torch.__version__)"]}, {"cell_type": "markdown", "id": "09c6bc8c", "metadata": {"papermill": {"duration": 0.180813, "end_time": "2021-12-04T15:53:28.192377", "exception": false, "start_time": "2021-12-04T15:53:28.011564", "status": "completed"}, "tags": []}, "source": ["At the time of writing this tutorial (mid of August 2021), the current stable version is 1.9.\n", "You should therefore see the output `Using torch 1.9.0`, eventually with some extension for the CUDA version on Colab.\n", "In case you use the `dl2020` environment, you should see `Using torch 1.6.0` since the environment was provided in October 2020.\n", "It is recommended to update the PyTorch version to the newest one.\n", "If you see a lower version number than 1.6, make sure you have installed the correct the environment, or ask one of your TAs.\n", "In case PyTorch 1.10 or newer will be published during the time of the course, don't worry.\n", "The interface between PyTorch versions doesn't change too much, and hence all code should also be runnable with newer versions.\n", "\n", "As in every machine learning framework, PyTorch provides functions that are stochastic like generating random numbers.\n", "However, a very good practice is to setup your code to be reproducible with the exact same random numbers.\n", "This is why we set a seed below."]}, {"cell_type": "code", "execution_count": 4, "id": "541190c5", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:28.555309Z", "iopub.status.busy": "2021-12-04T15:53:28.554820Z", "iopub.status.idle": "2021-12-04T15:53:28.559934Z", "shell.execute_reply": "2021-12-04T15:53:28.560320Z"}, "papermill": {"duration": 0.188101, "end_time": "2021-12-04T15:53:28.560451", "exception": false, "start_time": "2021-12-04T15:53:28.372350", "status": "completed"}, "tags": []}, "outputs": [{"data": {"text/plain": ["<torch._C.Generator at 0x7f42c8d3ab10>"]}, "execution_count": 4, "metadata": {}, "output_type": "execute_result"}], "source": ["torch.manual_seed(42)  # Setting the seed"]}, {"cell_type": "markdown", "id": "07432939", "metadata": {"papermill": {"duration": 0.179782, "end_time": "2021-12-04T15:53:28.920061", "exception": false, "start_time": "2021-12-04T15:53:28.740279", "status": "completed"}, "tags": []}, "source": ["### Tensors\n", "\n", "Tensors are the PyTorch equivalent to Numpy arrays, with the addition to also have support for GPU acceleration (more on that later).\n", "The name \"tensor\" is a generalization of concepts you already know.\n", "For instance, a vector is a 1-D tensor, and a matrix a 2-D tensor.\n", "When working with neural networks, we will use tensors of various shapes and number of dimensions.\n", "\n", "Most common functions you know from numpy can be used on tensors as well.\n", "Actually, since numpy arrays are so similar to tensors, we can convert most tensors to numpy arrays (and back) but we don't need it too often.\n", "\n", "#### Initialization\n", "\n", "Let's first start by looking at different ways of creating a tensor.\n", "There are many possible options, the most simple one is to call\n", "`torch.Tensor` passing the desired shape as input argument:"]}, {"cell_type": "code", "execution_count": 5, "id": "9a7c7abb", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:29.290240Z", "iopub.status.busy": "2021-12-04T15:53:29.289779Z", "iopub.status.idle": "2021-12-04T15:53:29.293533Z", "shell.execute_reply": "2021-12-04T15:53:29.293096Z"}, "papermill": {"duration": 0.186675, "end_time": "2021-12-04T15:53:29.293645", "exception": false, "start_time": "2021-12-04T15:53:29.106970", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["tensor([[[-8.4362e+34,  4.5652e-41, -2.7143e-34, -4.6629e+13],\n", "         [-1.2059e+35,  4.5652e-41, -8.4184e+34,  4.5652e-41],\n", "         [ 9.5052e-38,  2.5041e-19, -1.1549e+35,  4.5652e-41]],\n", "\n", "        [[-8.4185e+34,  4.5652e-41,  1.1811e+06,  6.3794e+25],\n", "         [-1.2059e+35,  4.5652e-41, -8.4186e+34,  4.5652e-41],\n", "         [ 1.1561e+02,  3.9652e+37, -1.2049e+35,  4.5652e-41]]])\n"]}], "source": ["x = torch.Tensor(2, 3, 4)\n", "print(x)"]}, {"cell_type": "markdown", "id": "c3259cdd", "metadata": {"papermill": {"duration": 0.182347, "end_time": "2021-12-04T15:53:29.658142", "exception": false, "start_time": "2021-12-04T15:53:29.475795", "status": "completed"}, "tags": []}, "source": ["The function `torch.Tensor` allocates memory for the desired tensor, but reuses any values that have already been in the memory.\n", "To directly assign values to the tensor during initialization, there are many alternatives including:\n", "\n", "* `torch.zeros`: Creates a tensor filled with zeros\n", "* `torch.ones`: Creates a tensor filled with ones\n", "* `torch.rand`: Creates a tensor with random values uniformly sampled between 0 and 1\n", "* `torch.randn`: Creates a tensor with random values sampled from a normal distribution with mean 0 and variance 1\n", "* `torch.arange`: Creates a tensor containing the values $N,N+1,N+2,...,M$\n", "* `torch.Tensor` (input list): Creates a tensor from the list elements you provide"]}, {"cell_type": "code", "execution_count": 6, "id": "d7d674fc", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:30.029266Z", "iopub.status.busy": "2021-12-04T15:53:30.028795Z", "iopub.status.idle": "2021-12-04T15:53:30.031289Z", "shell.execute_reply": "2021-12-04T15:53:30.031670Z"}, "papermill": {"duration": 0.18988, "end_time": "2021-12-04T15:53:30.031801", "exception": false, "start_time": "2021-12-04T15:53:29.841921", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["tensor([[1., 2.],\n", "        [3., 4.]])\n"]}], "source": ["# Create a tensor from a (nested) list\n", "x = torch.Tensor([[1, 2], [3, 4]])\n", "print(x)"]}, {"cell_type": "code", "execution_count": 7, "id": "2267c81e", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:30.409461Z", "iopub.status.busy": "2021-12-04T15:53:30.408988Z", "iopub.status.idle": "2021-12-04T15:53:30.411852Z", "shell.execute_reply": "2021-12-04T15:53:30.412236Z"}, "papermill": {"duration": 0.192911, "end_time": "2021-12-04T15:53:30.412364", "exception": false, "start_time": "2021-12-04T15:53:30.219453", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["tensor([[[0.8823, 0.9150, 0.3829, 0.9593],\n", "         [0.3904, 0.6009, 0.2566, 0.7936],\n", "         [0.9408, 0.1332, 0.9346, 0.5936]],\n", "\n", "        [[0.8694, 0.5677, 0.7411, 0.4294],\n", "         [0.8854, 0.5739, 0.2666, 0.6274],\n", "         [0.2696, 0.4414, 0.2969, 0.8317]]])\n"]}], "source": ["# Create a tensor with random values between 0 and 1 with the shape [2, 3, 4]\n", "x = torch.rand(2, 3, 4)\n", "print(x)"]}, {"cell_type": "markdown", "id": "f11f5cfd", "metadata": {"papermill": {"duration": 0.184187, "end_time": "2021-12-04T15:53:30.778847", "exception": false, "start_time": "2021-12-04T15:53:30.594660", "status": "completed"}, "tags": []}, "source": ["You can obtain the shape of a tensor in the same way as in numpy (`x.shape`), or using the `.size` method:"]}, {"cell_type": "code", "execution_count": 8, "id": "217f784e", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:31.151326Z", "iopub.status.busy": "2021-12-04T15:53:31.150840Z", "iopub.status.idle": "2021-12-04T15:53:31.153313Z", "shell.execute_reply": "2021-12-04T15:53:31.153693Z"}, "papermill": {"duration": 0.191823, "end_time": "2021-12-04T15:53:31.153829", "exception": false, "start_time": "2021-12-04T15:53:30.962006", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Shape: torch.Size([2, 3, 4])\n", "Size: torch.Size([2, 3, 4])\n", "Size: 2 3 4\n"]}], "source": ["shape = x.shape\n", "print(\"Shape:\", x.shape)\n", "\n", "size = x.size()\n", "print(\"Size:\", size)\n", "\n", "dim1, dim2, dim3 = x.size()\n", "print(\"Size:\", dim1, dim2, dim3)"]}, {"cell_type": "markdown", "id": "692ea692", "metadata": {"papermill": {"duration": 0.18425, "end_time": "2021-12-04T15:53:31.521916", "exception": false, "start_time": "2021-12-04T15:53:31.337666", "status": "completed"}, "tags": []}, "source": ["#### Tensor to Numpy, and Numpy to Tensor\n", "\n", "Tensors can be converted to numpy arrays, and numpy arrays back to tensors.\n", "To transform a numpy array into a tensor, we can use the function `torch.from_numpy`:"]}, {"cell_type": "code", "execution_count": 9, "id": "52c87663", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:31.897321Z", "iopub.status.busy": "2021-12-04T15:53:31.896848Z", "iopub.status.idle": "2021-12-04T15:53:31.899783Z", "shell.execute_reply": "2021-12-04T15:53:31.899380Z"}, "papermill": {"duration": 0.193907, "end_time": "2021-12-04T15:53:31.899893", "exception": false, "start_time": "2021-12-04T15:53:31.705986", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Numpy array: [[1 2]\n", " [3 4]]\n", "PyTorch tensor: tensor([[1, 2],\n", "        [3, 4]])\n"]}], "source": ["np_arr = np.array([[1, 2], [3, 4]])\n", "tensor = torch.from_numpy(np_arr)\n", "\n", "print(\"Numpy array:\", np_arr)\n", "print(\"PyTorch tensor:\", tensor)"]}, {"cell_type": "markdown", "id": "fe3837cb", "metadata": {"papermill": {"duration": 0.183323, "end_time": "2021-12-04T15:53:32.270124", "exception": false, "start_time": "2021-12-04T15:53:32.086801", "status": "completed"}, "tags": []}, "source": ["To transform a PyTorch tensor back to a numpy array, we can use the function `.numpy()` on tensors:"]}, {"cell_type": "code", "execution_count": 10, "id": "4a036f09", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:32.645038Z", "iopub.status.busy": "2021-12-04T15:53:32.644564Z", "iopub.status.idle": "2021-12-04T15:53:32.647718Z", "shell.execute_reply": "2021-12-04T15:53:32.647247Z"}, "papermill": {"duration": 0.193847, "end_time": "2021-12-04T15:53:32.647829", "exception": false, "start_time": "2021-12-04T15:53:32.453982", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["PyTorch tensor: tensor([0, 1, 2, 3])\n", "Numpy array: [0 1 2 3]\n"]}], "source": ["tensor = torch.arange(4)\n", "np_arr = tensor.numpy()\n", "\n", "print(\"PyTorch tensor:\", tensor)\n", "print(\"Numpy array:\", np_arr)"]}, {"cell_type": "markdown", "id": "7ed5a7da", "metadata": {"papermill": {"duration": 0.184638, "end_time": "2021-12-04T15:53:33.016987", "exception": false, "start_time": "2021-12-04T15:53:32.832349", "status": "completed"}, "tags": []}, "source": ["The conversion of tensors to numpy require the tensor to be on the CPU, and not the GPU (more on GPU support in a later section).\n", "In case you have a tensor on GPU, you need to call `.cpu()` on the tensor beforehand.\n", "Hence, you get a line like `np_arr = tensor.cpu().numpy()`."]}, {"cell_type": "markdown", "id": "e8e882b6", "metadata": {"papermill": {"duration": 0.183883, "end_time": "2021-12-04T15:53:33.386141", "exception": false, "start_time": "2021-12-04T15:53:33.202258", "status": "completed"}, "tags": []}, "source": ["#### Operations\n", "\n", "Most operations that exist in numpy, also exist in PyTorch.\n", "A full list of operations can be found in the [PyTorch documentation](https://pytorch.org/docs/stable/tensors.html#), but we will review the most important ones here.\n", "\n", "The simplest operation is to add two tensors:"]}, {"cell_type": "code", "execution_count": 11, "id": "a66afde2", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:33.763575Z", "iopub.status.busy": "2021-12-04T15:53:33.763098Z", "iopub.status.idle": "2021-12-04T15:53:33.768208Z", "shell.execute_reply": "2021-12-04T15:53:33.767800Z"}, "papermill": {"duration": 0.195513, "end_time": "2021-12-04T15:53:33.768317", "exception": false, "start_time": "2021-12-04T15:53:33.572804", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["X1 tensor([[0.1053, 0.2695, 0.3588],\n", "        [0.1994, 0.5472, 0.0062]])\n", "X2 tensor([[0.9516, 0.0753, 0.8860],\n", "        [0.5832, 0.3376, 0.8090]])\n", "Y tensor([[1.0569, 0.3448, 1.2448],\n", "        [0.7826, 0.8848, 0.8151]])\n"]}], "source": ["x1 = torch.rand(2, 3)\n", "x2 = torch.rand(2, 3)\n", "y = x1 + x2\n", "\n", "print(\"X1\", x1)\n", "print(\"X2\", x2)\n", "print(\"Y\", y)"]}, {"cell_type": "markdown", "id": "3510743a", "metadata": {"papermill": {"duration": 0.188609, "end_time": "2021-12-04T15:53:34.145802", "exception": false, "start_time": "2021-12-04T15:53:33.957193", "status": "completed"}, "tags": []}, "source": ["Calling `x1 + x2` creates a new tensor containing the sum of the two inputs.\n", "However, we can also use in-place operations that are applied directly on the memory of a tensor.\n", "We therefore change the values of `x2` without the chance to re-accessing the values of `x2` before the operation.\n", "An example is shown below:"]}, {"cell_type": "code", "execution_count": 12, "id": "93e2d0e2", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:34.523641Z", "iopub.status.busy": "2021-12-04T15:53:34.523169Z", "iopub.status.idle": "2021-12-04T15:53:34.528393Z", "shell.execute_reply": "2021-12-04T15:53:34.528008Z"}, "papermill": {"duration": 0.195416, "end_time": "2021-12-04T15:53:34.528506", "exception": false, "start_time": "2021-12-04T15:53:34.333090", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["X1 (before) tensor([[0.5779, 0.9040, 0.5547],\n", "        [0.3423, 0.6343, 0.3644]])\n", "X2 (before) tensor([[0.7104, 0.9464, 0.7890],\n", "        [0.2814, 0.7886, 0.5895]])\n", "X1 (after) tensor([[0.5779, 0.9040, 0.5547],\n", "        [0.3423, 0.6343, 0.3644]])\n", "X2 (after) tensor([[1.2884, 1.8504, 1.3437],\n", "        [0.6237, 1.4230, 0.9539]])\n"]}], "source": ["x1 = torch.rand(2, 3)\n", "x2 = torch.rand(2, 3)\n", "print(\"X1 (before)\", x1)\n", "print(\"X2 (before)\", x2)\n", "\n", "x2.add_(x1)\n", "print(\"X1 (after)\", x1)\n", "print(\"X2 (after)\", x2)"]}, {"cell_type": "markdown", "id": "acae6209", "metadata": {"papermill": {"duration": 0.189916, "end_time": "2021-12-04T15:53:34.905098", "exception": false, "start_time": "2021-12-04T15:53:34.715182", "status": "completed"}, "tags": []}, "source": ["In-place operations are usually marked with a underscore postfix (e.g. \"add_\" instead of \"add\").\n", "\n", "Another common operation aims at changing the shape of a tensor.\n", "A tensor of size (2,3) can be re-organized to any other shape with the same number of elements (e.g. a tensor of size (6), or (3,2), ...).\n", "In PyTorch, this operation is called `view`:"]}, {"cell_type": "code", "execution_count": 13, "id": "12e785e2", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:35.283833Z", "iopub.status.busy": "2021-12-04T15:53:35.283366Z", "iopub.status.idle": "2021-12-04T15:53:35.285936Z", "shell.execute_reply": "2021-12-04T15:53:35.290804Z"}, "papermill": {"duration": 0.19887, "end_time": "2021-12-04T15:53:35.290937", "exception": false, "start_time": "2021-12-04T15:53:35.092067", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["X tensor([0, 1, 2, 3, 4, 5])\n"]}], "source": ["x = torch.arange(6)\n", "print(\"X\", x)"]}, {"cell_type": "code", "execution_count": 14, "id": "68681255", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:35.669862Z", "iopub.status.busy": "2021-12-04T15:53:35.669397Z", "iopub.status.idle": "2021-12-04T15:53:35.672118Z", "shell.execute_reply": "2021-12-04T15:53:35.671721Z"}, "papermill": {"duration": 0.193577, "end_time": "2021-12-04T15:53:35.672223", "exception": false, "start_time": "2021-12-04T15:53:35.478646", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["X tensor([[0, 1, 2],\n", "        [3, 4, 5]])\n"]}], "source": ["x = x.view(2, 3)\n", "print(\"X\", x)"]}, {"cell_type": "code", "execution_count": 15, "id": "b87c271d", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:36.053012Z", "iopub.status.busy": "2021-12-04T15:53:36.052549Z", "iopub.status.idle": "2021-12-04T15:53:36.055288Z", "shell.execute_reply": "2021-12-04T15:53:36.054885Z"}, "papermill": {"duration": 0.195401, "end_time": "2021-12-04T15:53:36.055396", "exception": false, "start_time": "2021-12-04T15:53:35.859995", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["X tensor([[0, 3],\n", "        [1, 4],\n", "        [2, 5]])\n"]}], "source": ["x = x.permute(1, 0)  # Swapping dimension 0 and 1\n", "print(\"X\", x)"]}, {"cell_type": "markdown", "id": "b45990ab", "metadata": {"papermill": {"duration": 0.191632, "end_time": "2021-12-04T15:53:36.435435", "exception": false, "start_time": "2021-12-04T15:53:36.243803", "status": "completed"}, "tags": []}, "source": ["Other commonly used operations include matrix multiplications, which are essential for neural networks.\n", "Quite often, we have an input vector $\\mathbf{x}$, which is transformed using a learned weight matrix $\\mathbf{W}$.\n", "There are multiple ways and functions to perform matrix multiplication, some of which we list below:\n", "\n", "* `torch.matmul`: Performs the matrix product over two tensors, where the specific behavior depends on the dimensions.\n", "If both inputs are matrices (2-dimensional tensors), it performs the standard matrix product.\n", "For higher dimensional inputs, the function supports broadcasting (for details see the [documentation](https://pytorch.org/docs/stable/generated/torch.matmul.html?highlight=matmul#torch.matmul)).\n", "Can also be written as `a @ b`, similar to numpy.\n", "* `torch.mm`: Performs the matrix product over two matrices, but doesn't support broadcasting (see [documentation](https://pytorch.org/docs/stable/generated/torch.mm.html?highlight=torch%20mm#torch.mm))\n", "* `torch.bmm`: Performs the matrix product with a support batch dimension.\n", "If the first tensor $T$ is of shape ($b\\times n\\times m$), and the second tensor $R$ ($b\\times m\\times p$), the output $O$ is of shape ($b\\times n\\times p$), and has been calculated by performing $b$ matrix multiplications of the submatrices of $T$ and $R$: $O_i = T_i @ R_i$\n", "* `torch.einsum`: Performs matrix multiplications and more (i.e. sums of products) using the Einstein summation convention.\n", "Explanation of the Einstein sum can be found in assignment 1.\n", "\n", "Usually, we use `torch.matmul` or `torch.bmm`. We can try a matrix multiplication with `torch.matmul` below."]}, {"cell_type": "code", "execution_count": 16, "id": "913092b1", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:36.816103Z", "iopub.status.busy": "2021-12-04T15:53:36.815637Z", "iopub.status.idle": "2021-12-04T15:53:36.818033Z", "shell.execute_reply": "2021-12-04T15:53:36.818409Z"}, "papermill": {"duration": 0.195078, "end_time": "2021-12-04T15:53:36.818538", "exception": false, "start_time": "2021-12-04T15:53:36.623460", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["X tensor([[0, 1, 2],\n", "        [3, 4, 5]])\n"]}], "source": ["x = torch.arange(6)\n", "x = x.view(2, 3)\n", "print(\"X\", x)"]}, {"cell_type": "code", "execution_count": 17, "id": "aed345cb", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:37.208721Z", "iopub.status.busy": "2021-12-04T15:53:37.208252Z", "iopub.status.idle": "2021-12-04T15:53:37.212000Z", "shell.execute_reply": "2021-12-04T15:53:37.211600Z"}, "papermill": {"duration": 0.199386, "end_time": "2021-12-04T15:53:37.212109", "exception": false, "start_time": "2021-12-04T15:53:37.012723", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["W tensor([[0, 1, 2],\n", "        [3, 4, 5],\n", "        [6, 7, 8]])\n"]}], "source": ["W = torch.arange(9).view(3, 3)  # We can also stack multiple operations in a single line\n", "print(\"W\", W)"]}, {"cell_type": "code", "execution_count": 18, "id": "ba15bb65", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:37.596703Z", "iopub.status.busy": "2021-12-04T15:53:37.596239Z", "iopub.status.idle": "2021-12-04T15:53:37.599133Z", "shell.execute_reply": "2021-12-04T15:53:37.598757Z"}, "papermill": {"duration": 0.197023, "end_time": "2021-12-04T15:53:37.599245", "exception": false, "start_time": "2021-12-04T15:53:37.402222", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["h tensor([[15, 18, 21],\n", "        [42, 54, 66]])\n"]}], "source": ["h = torch.matmul(x, W)  # Verify the result by calculating it by hand too!\n", "print(\"h\", h)"]}, {"cell_type": "markdown", "id": "e2969a0e", "metadata": {"papermill": {"duration": 0.191191, "end_time": "2021-12-04T15:53:37.983302", "exception": false, "start_time": "2021-12-04T15:53:37.792111", "status": "completed"}, "tags": []}, "source": ["#### Indexing\n", "\n", "We often have the situation where we need to select a part of a tensor.\n", "Indexing works just like in numpy, so let's try it:"]}, {"cell_type": "code", "execution_count": 19, "id": "9eabb311", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:38.371801Z", "iopub.status.busy": "2021-12-04T15:53:38.371341Z", "iopub.status.idle": "2021-12-04T15:53:38.373967Z", "shell.execute_reply": "2021-12-04T15:53:38.374339Z"}, "papermill": {"duration": 0.198334, "end_time": "2021-12-04T15:53:38.374469", "exception": false, "start_time": "2021-12-04T15:53:38.176135", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["X tensor([[ 0,  1,  2,  3],\n", "        [ 4,  5,  6,  7],\n", "        [ 8,  9, 10, 11]])\n"]}], "source": ["x = torch.arange(12).view(3, 4)\n", "print(\"X\", x)"]}, {"cell_type": "code", "execution_count": 20, "id": "9e961958", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:38.762589Z", "iopub.status.busy": "2021-12-04T15:53:38.762126Z", "iopub.status.idle": "2021-12-04T15:53:38.764774Z", "shell.execute_reply": "2021-12-04T15:53:38.764396Z"}, "papermill": {"duration": 0.199401, "end_time": "2021-12-04T15:53:38.764881", "exception": false, "start_time": "2021-12-04T15:53:38.565480", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["tensor([1, 5, 9])\n"]}], "source": ["print(x[:, 1])  # Second column"]}, {"cell_type": "code", "execution_count": 21, "id": "8df1b8d5", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:39.158549Z", "iopub.status.busy": "2021-12-04T15:53:39.158081Z", "iopub.status.idle": "2021-12-04T15:53:39.160695Z", "shell.execute_reply": "2021-12-04T15:53:39.160291Z"}, "papermill": {"duration": 0.199565, "end_time": "2021-12-04T15:53:39.160802", "exception": false, "start_time": "2021-12-04T15:53:38.961237", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["tensor([0, 1, 2, 3])\n"]}], "source": ["print(x[0])  # First row"]}, {"cell_type": "code", "execution_count": 22, "id": "91fda3e0", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:39.553851Z", "iopub.status.busy": "2021-12-04T15:53:39.553387Z", "iopub.status.idle": "2021-12-04T15:53:39.555977Z", "shell.execute_reply": "2021-12-04T15:53:39.555511Z"}, "papermill": {"duration": 0.201859, "end_time": "2021-12-04T15:53:39.556086", "exception": false, "start_time": "2021-12-04T15:53:39.354227", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["tensor([3, 7])\n"]}], "source": ["print(x[:2, -1])  # First two rows, last column"]}, {"cell_type": "code", "execution_count": 23, "id": "a9cf47ae", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:39.949800Z", "iopub.status.busy": "2021-12-04T15:53:39.949338Z", "iopub.status.idle": "2021-12-04T15:53:39.951698Z", "shell.execute_reply": "2021-12-04T15:53:39.952071Z"}, "papermill": {"duration": 0.201325, "end_time": "2021-12-04T15:53:39.952202", "exception": false, "start_time": "2021-12-04T15:53:39.750877", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["tensor([[ 4,  5,  6,  7],\n", "        [ 8,  9, 10, 11]])\n"]}], "source": ["print(x[1:3, :])  # Middle two rows"]}, {"cell_type": "markdown", "id": "9fea92c8", "metadata": {"papermill": {"duration": 0.198501, "end_time": "2021-12-04T15:53:40.352499", "exception": false, "start_time": "2021-12-04T15:53:40.153998", "status": "completed"}, "tags": []}, "source": ["### Dynamic Computation Graph and Backpropagation\n", "\n", "One of the main reasons for using PyTorch in Deep Learning projects is that we can automatically get **gradients/derivatives** of functions that we define.\n", "We will mainly use PyTorch for implementing neural networks, and they are just fancy functions.\n", "If we use weight matrices in our function that we want to learn, then those are called the **parameters** or simply the **weights**.\n", "\n", "If our neural network would output a single scalar value, we would talk about taking the **derivative**, but you will see that quite often we will have **multiple** output variables (\"values\"); in that case we talk about **gradients**.\n", "It's a more general term.\n", "\n", "Given an input $\\mathbf{x}$, we define our function by **manipulating** that input, usually by matrix-multiplications with weight matrices and additions with so-called bias vectors.\n", "As we manipulate our input, we are automatically creating a **computational graph**.\n", "This graph shows how to arrive at our output from our input.\n", "PyTorch is a **define-by-run** framework; this means that we can just do our manipulations, and PyTorch will keep track of that graph for us.\n", "Thus, we create a dynamic computation graph along the way.\n", "\n", "So, to recap: the only thing we have to do is to compute the **output**, and then we can ask PyTorch to automatically get the **gradients**.\n", "\n", "> **Note:  Why do we want gradients?\n", "** Consider that we have defined a function, a neural net, that is supposed to compute a certain output $y$ for an input vector $\\mathbf{x}$.\n", "We then define an **error measure** that tells us how wrong our network is; how bad it is in predicting output $y$ from input $\\mathbf{x}$.\n", "Based on this error measure, we can use the gradients to **update** the weights $\\mathbf{W}$ that were responsible for the output, so that the next time we present input $\\mathbf{x}$ to our network, the output will be closer to what we want.\n", "\n", "The first thing we have to do is to specify which tensors require gradients.\n", "By default, when we create a tensor, it does not require gradients."]}, {"cell_type": "code", "execution_count": 24, "id": "c2cc2ed7", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:40.750883Z", "iopub.status.busy": "2021-12-04T15:53:40.750396Z", "iopub.status.idle": "2021-12-04T15:53:40.752681Z", "shell.execute_reply": "2021-12-04T15:53:40.752209Z"}, "papermill": {"duration": 0.203425, "end_time": "2021-12-04T15:53:40.752790", "exception": false, "start_time": "2021-12-04T15:53:40.549365", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["False\n"]}], "source": ["x = torch.ones((3,))\n", "print(x.requires_grad)"]}, {"cell_type": "markdown", "id": "95fb29d4", "metadata": {"papermill": {"duration": 0.202898, "end_time": "2021-12-04T15:53:41.153550", "exception": false, "start_time": "2021-12-04T15:53:40.950652", "status": "completed"}, "tags": []}, "source": ["We can change this for an existing tensor using the function `requires_grad_()` (underscore indicating that this is a in-place operation).\n", "Alternatively, when creating a tensor, you can pass the argument\n", "`requires_grad=True` to most initializers we have seen above."]}, {"cell_type": "code", "execution_count": 25, "id": "7c78f25f", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:41.555039Z", "iopub.status.busy": "2021-12-04T15:53:41.554541Z", "iopub.status.idle": "2021-12-04T15:53:41.556872Z", "shell.execute_reply": "2021-12-04T15:53:41.556397Z"}, "papermill": {"duration": 0.20518, "end_time": "2021-12-04T15:53:41.556986", "exception": false, "start_time": "2021-12-04T15:53:41.351806", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["True\n"]}], "source": ["x.requires_grad_(True)\n", "print(x.requires_grad)"]}, {"cell_type": "markdown", "id": "20803bcc", "metadata": {"papermill": {"duration": 0.196387, "end_time": "2021-12-04T15:53:41.950177", "exception": false, "start_time": "2021-12-04T15:53:41.753790", "status": "completed"}, "tags": []}, "source": ["In order to get familiar with the concept of a computation graph, we will create one for the following function:\n", "\n", "$$y = \\frac{1}{|x|}\\sum_i \\left[(x_i + 2)^2 + 3\\right]$$\n", "\n", "You could imagine that $x$ are our parameters, and we want to optimize (either maximize or minimize) the output $y$.\n", "For this, we want to obtain the gradients $\\partial y / \\partial \\mathbf{x}$.\n", "For our example, we'll use $\\mathbf{x}=[0,1,2]$ as our input."]}, {"cell_type": "code", "execution_count": 26, "id": "1736b0d5", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:42.353546Z", "iopub.status.busy": "2021-12-04T15:53:42.353077Z", "iopub.status.idle": "2021-12-04T15:53:42.355812Z", "shell.execute_reply": "2021-12-04T15:53:42.356193Z"}, "papermill": {"duration": 0.206211, "end_time": "2021-12-04T15:53:42.356325", "exception": false, "start_time": "2021-12-04T15:53:42.150114", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["X tensor([0., 1., 2.], requires_grad=True)\n"]}], "source": ["x = torch.arange(3, dtype=torch.float32, requires_grad=True)  # Only float tensors can have gradients\n", "print(\"X\", x)"]}, {"cell_type": "markdown", "id": "cf511f52", "metadata": {"papermill": {"duration": 0.200601, "end_time": "2021-12-04T15:53:42.755512", "exception": false, "start_time": "2021-12-04T15:53:42.554911", "status": "completed"}, "tags": []}, "source": ["Now let's build the computation graph step by step.\n", "You can combine multiple operations in a single line, but we will\n", "separate them here to get a better understanding of how each operation\n", "is added to the computation graph."]}, {"cell_type": "code", "execution_count": 27, "id": "fbcee41c", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:43.160150Z", "iopub.status.busy": "2021-12-04T15:53:43.159591Z", "iopub.status.idle": "2021-12-04T15:53:43.163509Z", "shell.execute_reply": "2021-12-04T15:53:43.163027Z"}, "papermill": {"duration": 0.209982, "end_time": "2021-12-04T15:53:43.163620", "exception": false, "start_time": "2021-12-04T15:53:42.953638", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Y tensor(12.6667, grad_fn=<MeanBackward0>)\n"]}], "source": ["a = x + 2\n", "b = a ** 2\n", "c = b + 3\n", "y = c.mean()\n", "print(\"Y\", y)"]}, {"cell_type": "markdown", "id": "c012a822", "metadata": {"papermill": {"duration": 0.201708, "end_time": "2021-12-04T15:53:43.567654", "exception": false, "start_time": "2021-12-04T15:53:43.365946", "status": "completed"}, "tags": []}, "source": ["Using the statements above, we have created a computation graph that looks similar to the figure below:\n", "\n", "<center style=\"width: 100%\"><img src=\"https://github.com/PyTorchLightning/lightning-tutorials/raw/main/course_UvA-DL/01-introduction-to-pytorch/pytorch_computation_graph.svg\" width=\"200px\"></center>\n", "\n", "We calculate $a$ based on the inputs $x$ and the constant $2$, $b$ is $a$ squared, and so on.\n", "The visualization is an abstraction of the dependencies between inputs and outputs of the operations we have applied.\n", "Each node of the computation graph has automatically defined a function for calculating the gradients with respect to its inputs, `grad_fn`.\n", "You can see this when we printed the output tensor $y$.\n", "This is why the computation graph is usually visualized in the reverse direction (arrows point from the result to the inputs).\n", "We can perform backpropagation on the computation graph by calling the\n", "function `backward()` on the last output, which effectively calculates\n", "the gradients for each tensor that has the property\n", "`requires_grad=True`:"]}, {"cell_type": "code", "execution_count": 28, "id": "3afeb787", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:43.969958Z", "iopub.status.busy": "2021-12-04T15:53:43.969494Z", "iopub.status.idle": "2021-12-04T15:53:44.203064Z", "shell.execute_reply": "2021-12-04T15:53:44.203439Z"}, "papermill": {"duration": 0.436371, "end_time": "2021-12-04T15:53:44.203621", "exception": false, "start_time": "2021-12-04T15:53:43.767250", "status": "completed"}, "tags": []}, "outputs": [], "source": ["y.backward()"]}, {"cell_type": "markdown", "id": "6b048147", "metadata": {"papermill": {"duration": 0.198366, "end_time": "2021-12-04T15:53:44.603913", "exception": false, "start_time": "2021-12-04T15:53:44.405547", "status": "completed"}, "tags": []}, "source": ["`x.grad` will now contain the gradient $\\partial y/ \\partial \\mathcal{x}$, and this gradient indicates how a change in $\\mathbf{x}$ will affect output $y$ given the current input $\\mathbf{x}=[0,1,2]$:"]}, {"cell_type": "code", "execution_count": 29, "id": "2a36cc4c", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:45.011258Z", "iopub.status.busy": "2021-12-04T15:53:45.010791Z", "iopub.status.idle": "2021-12-04T15:53:45.013643Z", "shell.execute_reply": "2021-12-04T15:53:45.013258Z"}, "papermill": {"duration": 0.210613, "end_time": "2021-12-04T15:53:45.013754", "exception": false, "start_time": "2021-12-04T15:53:44.803141", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["tensor([1.3333, 2.0000, 2.6667])\n"]}], "source": ["print(x.grad)"]}, {"cell_type": "markdown", "id": "01673131", "metadata": {"papermill": {"duration": 0.200204, "end_time": "2021-12-04T15:53:45.415831", "exception": false, "start_time": "2021-12-04T15:53:45.215627", "status": "completed"}, "tags": []}, "source": ["We can also verify these gradients by hand.\n", "We will calculate the gradients using the chain rule, in the same way as PyTorch did it:\n", "\n", "$$\\frac{\\partial y}{\\partial x_i} = \\frac{\\partial y}{\\partial c_i}\\frac{\\partial c_i}{\\partial b_i}\\frac{\\partial b_i}{\\partial a_i}\\frac{\\partial a_i}{\\partial x_i}$$\n", "\n", "Note that we have simplified this equation to index notation, and by using the fact that all operation besides the mean do not combine the elements in the tensor.\n", "The partial derivatives are:\n", "\n", "$$\n", "\\frac{\\partial a_i}{\\partial x_i} = 1,\\hspace{1cm}\n", "\\frac{\\partial b_i}{\\partial a_i} = 2\\cdot a_i\\hspace{1cm}\n", "\\frac{\\partial c_i}{\\partial b_i} = 1\\hspace{1cm}\n", "\\frac{\\partial y}{\\partial c_i} = \\frac{1}{3}\n", "$$\n", "\n", "Hence, with the input being $\\mathbf{x}=[0,1,2]$, our gradients are $\\partial y/\\partial \\mathbf{x}=[4/3,2,8/3]$.\n", "The previous code cell should have printed the same result."]}, {"cell_type": "markdown", "id": "7d826bdb", "metadata": {"papermill": {"duration": 0.198481, "end_time": "2021-12-04T15:53:45.813968", "exception": false, "start_time": "2021-12-04T15:53:45.615487", "status": "completed"}, "tags": []}, "source": ["### GPU support\n", "\n", "A crucial feature of PyTorch is the support of GPUs, short for Graphics Processing Unit.\n", "A GPU can perform many thousands of small operations in parallel, making it very well suitable for performing large matrix operations in neural networks.\n", "When comparing GPUs to CPUs, we can list the following main differences (credit: [Kevin Krewell, 2009](https://blogs.nvidia.com/blog/2009/12/16/whats-the-difference-between-a-cpu-and-a-gpu/))\n", "\n", "<center style=\"width: 100%\"><img src=\"https://github.com/PyTorchLightning/lightning-tutorials/raw/main/course_UvA-DL/01-introduction-to-pytorch/comparison_CPU_GPU.png\" width=\"700px\"></center>\n", "\n", "CPUs and GPUs have both different advantages and disadvantages, which is why many computers contain both components and use them for different tasks.\n", "In case you are not familiar with GPUs, you can read up more details in this [NVIDIA blog post](https://blogs.nvidia.com/blog/2009/12/16/whats-the-difference-between-a-cpu-and-a-gpu/) or [here](https://www.intel.com/content/www/us/en/products/docs/processors/what-is-a-gpu.html).\n", "\n", "GPUs can accelerate the training of your network up to a factor of $100$ which is essential for large neural networks.\n", "PyTorch implements a lot of functionality for supporting GPUs (mostly those of NVIDIA due to the libraries [CUDA](https://developer.nvidia.com/cuda-zone) and [cuDNN](https://developer.nvidia.com/cudnn)).\n", "First, let's check whether you have a GPU available:"]}, {"cell_type": "code", "execution_count": 30, "id": "8fe50089", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:46.222636Z", "iopub.status.busy": "2021-12-04T15:53:46.222150Z", "iopub.status.idle": "2021-12-04T15:53:46.224494Z", "shell.execute_reply": "2021-12-04T15:53:46.224024Z"}, "papermill": {"duration": 0.209045, "end_time": "2021-12-04T15:53:46.224603", "exception": false, "start_time": "2021-12-04T15:53:46.015558", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Is the GPU available? True\n"]}], "source": ["gpu_avail = torch.cuda.is_available()\n", "print(f\"Is the GPU available? {gpu_avail}\")"]}, {"cell_type": "markdown", "id": "cc6dd1a9", "metadata": {"papermill": {"duration": 0.201657, "end_time": "2021-12-04T15:53:46.632585", "exception": false, "start_time": "2021-12-04T15:53:46.430928", "status": "completed"}, "tags": []}, "source": ["If you have a GPU on your computer but the command above returns False, make sure you have the correct CUDA-version installed.\n", "The `dl2020` environment comes with the CUDA-toolkit 10.1, which is selected for the Lisa supercomputer.\n", "Please change it if necessary (CUDA 10.2 is currently common).\n", "On Google Colab, make sure that you have selected a GPU in your runtime setup (in the menu, check under `Runtime -> Change runtime type`).\n", "\n", "By default, all tensors you create are stored on the CPU.\n", "We can push a tensor to the GPU by using the function `.to(...)`, or `.cuda()`.\n", "However, it is often a good practice to define a `device` object in your code which points to the GPU if you have one, and otherwise to the CPU.\n", "Then, you can write your code with respect to this device object, and it allows you to run the same code on both a CPU-only system, and one with a GPU.\n", "Let's try it below.\n", "We can specify the device as follows:"]}, {"cell_type": "code", "execution_count": 31, "id": "5a3d8e38", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:47.042732Z", "iopub.status.busy": "2021-12-04T15:53:47.042239Z", "iopub.status.idle": "2021-12-04T15:53:47.044024Z", "shell.execute_reply": "2021-12-04T15:53:47.044398Z"}, "papermill": {"duration": 0.208184, "end_time": "2021-12-04T15:53:47.044529", "exception": false, "start_time": "2021-12-04T15:53:46.836345", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Device cuda\n"]}], "source": ["device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device(\"cpu\")\n", "print(\"Device\", device)"]}, {"cell_type": "markdown", "id": "83f280ae", "metadata": {"papermill": {"duration": 0.203434, "end_time": "2021-12-04T15:53:47.452136", "exception": false, "start_time": "2021-12-04T15:53:47.248702", "status": "completed"}, "tags": []}, "source": ["Now let's create a tensor and push it to the device:"]}, {"cell_type": "code", "execution_count": 32, "id": "31cbb86a", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:47.862514Z", "iopub.status.busy": "2021-12-04T15:53:47.862045Z", "iopub.status.idle": "2021-12-04T15:53:51.511287Z", "shell.execute_reply": "2021-12-04T15:53:51.510793Z"}, "papermill": {"duration": 3.856813, "end_time": "2021-12-04T15:53:51.511416", "exception": false, "start_time": "2021-12-04T15:53:47.654603", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["X tensor([[0., 0., 0.],\n", "        [0., 0., 0.]], device='cuda:0')\n"]}], "source": ["x = torch.zeros(2, 3)\n", "x = x.to(device)\n", "print(\"X\", x)"]}, {"cell_type": "markdown", "id": "de20fb36", "metadata": {"papermill": {"duration": 0.205639, "end_time": "2021-12-04T15:53:51.921267", "exception": false, "start_time": "2021-12-04T15:53:51.715628", "status": "completed"}, "tags": []}, "source": ["In case you have a GPU, you should now see the attribute `device='cuda:0'` being printed next to your tensor.\n", "The zero next to cuda indicates that this is the zero-th GPU device on your computer.\n", "PyTorch also supports multi-GPU systems, but this you will only need once you have very big networks to train (if interested, see the [PyTorch documentation](https://pytorch.org/docs/stable/distributed.html#distributed-basics)).\n", "We can also compare the runtime of a large matrix multiplication on the CPU with a operation on the GPU:"]}, {"cell_type": "code", "execution_count": 33, "id": "bbf73c6d", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:52.333482Z", "iopub.status.busy": "2021-12-04T15:53:52.333010Z", "iopub.status.idle": "2021-12-04T15:53:52.904067Z", "shell.execute_reply": "2021-12-04T15:53:52.904461Z"}, "papermill": {"duration": 0.778998, "end_time": "2021-12-04T15:53:52.904626", "exception": false, "start_time": "2021-12-04T15:53:52.125628", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["CPU time: 0.27197s\n", "GPU time: 0.02201s\n"]}], "source": ["x = torch.randn(5000, 5000)\n", "\n", "# CPU version\n", "start_time = time.time()\n", "_ = torch.matmul(x, x)\n", "end_time = time.time()\n", "print(f\"CPU time: {(end_time - start_time):6.5f}s\")\n", "\n", "# GPU version\n", "if torch.cuda.is_available():\n", "    x = x.to(device)\n", "    # CUDA is asynchronous, so we need to use different timing functions\n", "    start = torch.cuda.Event(enable_timing=True)\n", "    end = torch.cuda.Event(enable_timing=True)\n", "    start.record()\n", "    _ = torch.matmul(x, x)\n", "    end.record()\n", "    torch.cuda.synchronize()  # Waits for everything to finish running on the GPU\n", "    print(f\"GPU time: {0.001 * start.elapsed_time(end):6.5f}s\")  # Milliseconds to seconds"]}, {"cell_type": "markdown", "id": "b6483a54", "metadata": {"papermill": {"duration": 0.208145, "end_time": "2021-12-04T15:53:53.320523", "exception": false, "start_time": "2021-12-04T15:53:53.112378", "status": "completed"}, "tags": []}, "source": ["Depending on the size of the operation and the CPU/GPU in your system, the speedup of this operation can be >50x.\n", "As `matmul` operations are very common in neural networks, we can already see the great benefit of training a NN on a GPU.\n", "The time estimate can be relatively noisy here because we haven't run it for multiple times.\n", "Feel free to extend this, but it also takes longer to run.\n", "\n", "When generating random numbers, the seed between CPU and GPU is not synchronized.\n", "Hence, we need to set the seed on the GPU separately to ensure a reproducible code.\n", "Note that due to different GPU architectures, running the same code on different GPUs does not guarantee the same random numbers.\n", "Still, we don't want that our code gives us a different output every time we run it on the exact same hardware.\n", "Hence, we also set the seed on the GPU:"]}, {"cell_type": "code", "execution_count": 34, "id": "119c0085", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:53.736717Z", "iopub.status.busy": "2021-12-04T15:53:53.736247Z", "iopub.status.idle": "2021-12-04T15:53:53.737755Z", "shell.execute_reply": "2021-12-04T15:53:53.738129Z"}, "papermill": {"duration": 0.215091, "end_time": "2021-12-04T15:53:53.738260", "exception": false, "start_time": "2021-12-04T15:53:53.523169", "status": "completed"}, "tags": []}, "outputs": [], "source": ["# GPU operations have a separate seed we also want to set\n", "if torch.cuda.is_available():\n", "    torch.cuda.manual_seed(42)\n", "    torch.cuda.manual_seed_all(42)\n", "\n", "# Additionally, some operations on a GPU are implemented stochastic for efficiency\n", "# We want to ensure that all operations are deterministic on GPU (if used) for reproducibility\n", "torch.backends.cudnn.determinstic = True\n", "torch.backends.cudnn.benchmark = False"]}, {"cell_type": "markdown", "id": "d3e51727", "metadata": {"papermill": {"duration": 0.201794, "end_time": "2021-12-04T15:53:54.143745", "exception": false, "start_time": "2021-12-04T15:53:53.941951", "status": "completed"}, "tags": []}, "source": ["## Learning by example: Continuous XOR\n", "<div class=\"center-wrapper\"><div class=\"video-wrapper\"><iframe src=\"https://www.youtube.com/embed/schbjeU5X2g\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen></iframe></div></div>\n", "\n", "If we want to build a neural network in PyTorch, we could specify all our parameters (weight matrices, bias vectors) using `Tensors` (with `requires_grad=True`), ask PyTorch to calculate the gradients and then adjust the parameters.\n", "But things can quickly get cumbersome if we have a lot of parameters.\n", "In PyTorch, there is a package called `torch.nn` that makes building neural networks more convenient.\n", "\n", "We will introduce the libraries and all additional parts you might need to train a neural network in PyTorch, using a simple example classifier on a simple yet well known example: XOR.\n", "Given two binary inputs $x_1$ and $x_2$, the label to predict is $1$ if either $x_1$ or $x_2$ is $1$ while the other is $0$, or the label is $0$ in all other cases.\n", "The example became famous by the fact that a single neuron, i.e. a linear classifier, cannot learn this simple function.\n", "Hence, we will learn how to build a small neural network that can learn this function.\n", "To make it a little bit more interesting, we move the XOR into continuous space and introduce some gaussian noise on the binary inputs.\n", "Our desired separation of an XOR dataset could look as follows:\n", "\n", "<center style=\"width: 100%\"><img src=\"https://github.com/PyTorchLightning/lightning-tutorials/raw/main/course_UvA-DL/01-introduction-to-pytorch/continuous_xor.svg\" width=\"350px\"></center>"]}, {"cell_type": "markdown", "id": "be79359e", "metadata": {"papermill": {"duration": 0.205221, "end_time": "2021-12-04T15:53:54.552887", "exception": false, "start_time": "2021-12-04T15:53:54.347666", "status": "completed"}, "tags": []}, "source": ["### The model\n", "\n", "The package `torch.nn` defines a series of useful classes like linear networks layers, activation functions, loss functions etc.\n", "A full list can be found [here](https://pytorch.org/docs/stable/nn.html).\n", "In case you need a certain network layer, check the documentation of the package first before writing the layer yourself as the package likely contains the code for it already.\n", "We import it below:"]}, {"cell_type": "code", "execution_count": null, "id": "43daa017", "metadata": {"lines_to_next_cell": 0, "papermill": {"duration": 0.203631, "end_time": "2021-12-04T15:53:54.958970", "exception": false, "start_time": "2021-12-04T15:53:54.755339", "status": "completed"}, "tags": []}, "outputs": [], "source": []}, {"cell_type": "code", "execution_count": null, "id": "961da892", "metadata": {"papermill": {"duration": 0.204014, "end_time": "2021-12-04T15:53:55.368098", "exception": false, "start_time": "2021-12-04T15:53:55.164084", "status": "completed"}, "tags": []}, "outputs": [], "source": []}, {"cell_type": "markdown", "id": "5b961e6a", "metadata": {"papermill": {"duration": 0.203803, "end_time": "2021-12-04T15:53:55.774983", "exception": false, "start_time": "2021-12-04T15:53:55.571180", "status": "completed"}, "tags": []}, "source": ["Additionally to `torch.nn`, there is also `torch.nn.functional`.\n", "It contains functions that are used in network layers.\n", "This is in contrast to `torch.nn` which defines them as `nn.Modules` (more on it below), and `torch.nn` actually uses a lot of functionalities from `torch.nn.functional`.\n", "Hence, the functional package is useful in many situations, and so we import it as well here."]}, {"cell_type": "markdown", "id": "e60690c4", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.205786, "end_time": "2021-12-04T15:53:56.185307", "exception": false, "start_time": "2021-12-04T15:53:55.979521", "status": "completed"}, "tags": []}, "source": ["#### nn.Module\n", "\n", "In PyTorch, a neural network is built up out of modules.\n", "Modules can contain other modules, and a neural network is considered to be a module itself as well.\n", "The basic template of a module is as follows:"]}, {"cell_type": "code", "execution_count": 35, "id": "71eaf6ae", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:56.605744Z", "iopub.status.busy": "2021-12-04T15:53:56.605272Z", "iopub.status.idle": "2021-12-04T15:53:56.606804Z", "shell.execute_reply": "2021-12-04T15:53:56.607182Z"}, "lines_to_next_cell": 2, "papermill": {"duration": 0.216416, "end_time": "2021-12-04T15:53:56.607313", "exception": false, "start_time": "2021-12-04T15:53:56.390897", "status": "completed"}, "tags": []}, "outputs": [], "source": ["class MyModule(nn.Module):\n", "    def __init__(self):\n", "        super().__init__()\n", "        # Some init for my module\n", "\n", "    def forward(self, x):\n", "        # Function for performing the calculation of the module.\n", "        pass"]}, {"cell_type": "markdown", "id": "63463f01", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.229323, "end_time": "2021-12-04T15:53:57.045874", "exception": false, "start_time": "2021-12-04T15:53:56.816551", "status": "completed"}, "tags": []}, "source": ["The forward function is where the computation of the module is taken place, and is executed when you call the module (`nn = MyModule(); nn(x)`).\n", "In the init function, we usually create the parameters of the module, using `nn.Parameter`, or defining other modules that are used in the forward function.\n", "The backward calculation is done automatically, but could be overwritten as well if wanted.\n", "\n", "#### Simple classifier\n", "We can now make use of the pre-defined modules in the `torch.nn` package, and define our own small neural network.\n", "We will use a minimal network with a input layer, one hidden layer with tanh as activation function, and a output layer.\n", "In other words, our networks should look something like this:\n", "\n", "<center width=\"100%\"><img src=\"https://github.com/PyTorchLightning/lightning-tutorials/raw/main/course_UvA-DL/01-introduction-to-pytorch/small_neural_network.svg\" width=\"300px\"></center>\n", "\n", "The input neurons are shown in blue, which represent the coordinates $x_1$ and $x_2$ of a data point.\n", "The hidden neurons including a tanh activation are shown in white, and the output neuron in red.\n", "In PyTorch, we can define this as follows:"]}, {"cell_type": "code", "execution_count": 36, "id": "6c0cc064", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:57.460540Z", "iopub.status.busy": "2021-12-04T15:53:57.460070Z", "iopub.status.idle": "2021-12-04T15:53:57.461979Z", "shell.execute_reply": "2021-12-04T15:53:57.461520Z"}, "papermill": {"duration": 0.213388, "end_time": "2021-12-04T15:53:57.462086", "exception": false, "start_time": "2021-12-04T15:53:57.248698", "status": "completed"}, "tags": []}, "outputs": [], "source": ["class SimpleClassifier(nn.Module):\n", "    def __init__(self, num_inputs, num_hidden, num_outputs):\n", "        super().__init__()\n", "        # Initialize the modules we need to build the network\n", "        self.linear1 = nn.Linear(num_inputs, num_hidden)\n", "        self.act_fn = nn.Tanh()\n", "        self.linear2 = nn.Linear(num_hidden, num_outputs)\n", "\n", "    def forward(self, x):\n", "        # Perform the calculation of the model to determine the prediction\n", "        x = self.linear1(x)\n", "        x = self.act_fn(x)\n", "        x = self.linear2(x)\n", "        return x"]}, {"cell_type": "markdown", "id": "84553ec2", "metadata": {"papermill": {"duration": 0.204402, "end_time": "2021-12-04T15:53:57.868446", "exception": false, "start_time": "2021-12-04T15:53:57.664044", "status": "completed"}, "tags": []}, "source": ["For the examples in this notebook, we will use a tiny neural network with two input neurons and four hidden neurons.\n", "As we perform binary classification, we will use a single output neuron.\n", "Note that we do not apply a sigmoid on the output yet.\n", "This is because other functions, especially the loss, are more efficient and precise to calculate on the original outputs instead of the sigmoid output.\n", "We will discuss the detailed reason later."]}, {"cell_type": "code", "execution_count": 37, "id": "4f9b83e1", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:58.283224Z", "iopub.status.busy": "2021-12-04T15:53:58.282754Z", "iopub.status.idle": "2021-12-04T15:53:58.285141Z", "shell.execute_reply": "2021-12-04T15:53:58.285538Z"}, "papermill": {"duration": 0.213062, "end_time": "2021-12-04T15:53:58.285669", "exception": false, "start_time": "2021-12-04T15:53:58.072607", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["SimpleClassifier(\n", "  (linear1): Linear(in_features=2, out_features=4, bias=True)\n", "  (act_fn): Tanh()\n", "  (linear2): Linear(in_features=4, out_features=1, bias=True)\n", ")\n"]}], "source": ["model = SimpleClassifier(num_inputs=2, num_hidden=4, num_outputs=1)\n", "# Printing a module shows all its submodules\n", "print(model)"]}, {"cell_type": "markdown", "id": "360961ed", "metadata": {"papermill": {"duration": 0.202451, "end_time": "2021-12-04T15:53:58.691398", "exception": false, "start_time": "2021-12-04T15:53:58.488947", "status": "completed"}, "tags": []}, "source": ["Printing the model lists all submodules it contains.\n", "The parameters of a module can be obtained by using its `parameters()` functions, or `named_parameters()` to get a name to each parameter object.\n", "For our small neural network, we have the following parameters:"]}, {"cell_type": "code", "execution_count": 38, "id": "780fcc13", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:53:59.102900Z", "iopub.status.busy": "2021-12-04T15:53:59.102421Z", "iopub.status.idle": "2021-12-04T15:53:59.104603Z", "shell.execute_reply": "2021-12-04T15:53:59.104977Z"}, "papermill": {"duration": 0.211481, "end_time": "2021-12-04T15:53:59.105106", "exception": false, "start_time": "2021-12-04T15:53:58.893625", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Parameter linear1.weight, shape torch.Size([4, 2])\n", "Parameter linear1.bias, shape torch.Size([4])\n", "Parameter linear2.weight, shape torch.Size([1, 4])\n", "Parameter linear2.bias, shape torch.Size([1])\n"]}], "source": ["for name, param in model.named_parameters():\n", "    print(f\"Parameter {name}, shape {param.shape}\")"]}, {"cell_type": "markdown", "id": "bc034c95", "metadata": {"papermill": {"duration": 0.203191, "end_time": "2021-12-04T15:53:59.512934", "exception": false, "start_time": "2021-12-04T15:53:59.309743", "status": "completed"}, "tags": []}, "source": ["Each linear layer has a weight matrix of the shape `[output, input]`, and a bias of the shape `[output]`.\n", "The tanh activation function does not have any parameters.\n", "Note that parameters are only registered for `nn.Module` objects that are direct object attributes, i.e. `self.a = ...`.\n", "If you define a list of modules, the parameters of those are not registered for the outer module and can cause some issues when you try to optimize your module.\n", "There are alternatives, like `nn.ModuleList`, `nn.ModuleDict` and `nn.Sequential`, that allow you to have different data structures of modules.\n", "We will use them in a few later tutorials and explain them there."]}, {"cell_type": "markdown", "id": "136035b4", "metadata": {"papermill": {"duration": 0.204985, "end_time": "2021-12-04T15:53:59.921347", "exception": false, "start_time": "2021-12-04T15:53:59.716362", "status": "completed"}, "tags": []}, "source": ["### The data\n", "\n", "PyTorch also provides a few functionalities to load the training and\n", "test data efficiently, summarized in the package `torch.utils.data`."]}, {"cell_type": "code", "execution_count": null, "id": "1cf8b77a", "metadata": {"papermill": {"duration": 0.204911, "end_time": "2021-12-04T15:54:00.331565", "exception": false, "start_time": "2021-12-04T15:54:00.126654", "status": "completed"}, "tags": []}, "outputs": [], "source": []}, {"cell_type": "markdown", "id": "056c9971", "metadata": {"papermill": {"duration": 0.203757, "end_time": "2021-12-04T15:54:00.743646", "exception": false, "start_time": "2021-12-04T15:54:00.539889", "status": "completed"}, "tags": []}, "source": ["The data package defines two classes which are the standard interface for handling data in PyTorch: `data.Dataset`, and `data.DataLoader`.\n", "The dataset class provides an uniform interface to access the\n", "training/test data, while the data loader makes sure to efficiently load\n", "and stack the data points from the dataset into batches during training."]}, {"cell_type": "markdown", "id": "92ffaea5", "metadata": {"papermill": {"duration": 0.20609, "end_time": "2021-12-04T15:54:01.153489", "exception": false, "start_time": "2021-12-04T15:54:00.947399", "status": "completed"}, "tags": []}, "source": ["#### The dataset class\n", "\n", "The dataset class summarizes the basic functionality of a dataset in a natural way.\n", "To define a dataset in PyTorch, we simply specify two functions: `__getitem__`, and `__len__`.\n", "The get-item function has to return the $i$-th data point in the dataset, while the len function returns the size of the dataset.\n", "For the XOR dataset, we can define the dataset class as follows:"]}, {"cell_type": "code", "execution_count": 39, "id": "9eadd4ba", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:54:01.569750Z", "iopub.status.busy": "2021-12-04T15:54:01.569274Z", "iopub.status.idle": "2021-12-04T15:54:01.571170Z", "shell.execute_reply": "2021-12-04T15:54:01.570790Z"}, "papermill": {"duration": 0.213645, "end_time": "2021-12-04T15:54:01.571284", "exception": false, "start_time": "2021-12-04T15:54:01.357639", "status": "completed"}, "tags": []}, "outputs": [], "source": ["\n", "\n", "class XORDataset(data.Dataset):\n", "    def __init__(self, size, std=0.1):\n", "        \"\"\"\n", "        Inputs:\n", "            size - Number of data points we want to generate\n", "            std - Standard deviation of the noise (see generate_continuous_xor function)\n", "        \"\"\"\n", "        super().__init__()\n", "        self.size = size\n", "        self.std = std\n", "        self.generate_continuous_xor()\n", "\n", "    def generate_continuous_xor(self):\n", "        # Each data point in the XOR dataset has two variables, x and y, that can be either 0 or 1\n", "        # The label is their XOR combination, i.e. 1 if only x or only y is 1 while the other is 0.\n", "        # If x=y, the label is 0.\n", "        data = torch.randint(low=0, high=2, size=(self.size, 2), dtype=torch.float32)\n", "        label = (data.sum(dim=1) == 1).to(torch.long)\n", "        # To make it slightly more challenging, we add a bit of gaussian noise to the data points.\n", "        data += self.std * torch.randn(data.shape)\n", "\n", "        self.data = data\n", "        self.label = label\n", "\n", "    def __len__(self):\n", "        # Number of data point we have. Alternatively self.data.shape[0], or self.label.shape[0]\n", "        return self.size\n", "\n", "    def __getitem__(self, idx):\n", "        # Return the idx-th data point of the dataset\n", "        # If we have multiple things to return (data point and label), we can return them as tuple\n", "        data_point = self.data[idx]\n", "        data_label = self.label[idx]\n", "        return data_point, data_label"]}, {"cell_type": "markdown", "id": "170ab9ba", "metadata": {"papermill": {"duration": 0.207351, "end_time": "2021-12-04T15:54:01.984743", "exception": false, "start_time": "2021-12-04T15:54:01.777392", "status": "completed"}, "tags": []}, "source": ["Let's try to create such a dataset and inspect it:"]}, {"cell_type": "code", "execution_count": 40, "id": "be2c0ec7", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:54:02.398061Z", "iopub.status.busy": "2021-12-04T15:54:02.397602Z", "iopub.status.idle": "2021-12-04T15:54:02.401563Z", "shell.execute_reply": "2021-12-04T15:54:02.401095Z"}, "papermill": {"duration": 0.211018, "end_time": "2021-12-04T15:54:02.401672", "exception": false, "start_time": "2021-12-04T15:54:02.190654", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Size of dataset: 200\n", "Data point 0: (tensor([0.9632, 0.1117]), tensor(1))\n"]}], "source": ["dataset = XORDataset(size=200)\n", "print(\"Size of dataset:\", len(dataset))\n", "print(\"Data point 0:\", dataset[0])"]}, {"cell_type": "markdown", "id": "86828a5c", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.206005, "end_time": "2021-12-04T15:54:02.812653", "exception": false, "start_time": "2021-12-04T15:54:02.606648", "status": "completed"}, "tags": []}, "source": ["To better relate to the dataset, we visualize the samples below."]}, {"cell_type": "code", "execution_count": 41, "id": "84457292", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:54:03.229327Z", "iopub.status.busy": "2021-12-04T15:54:03.228861Z", "iopub.status.idle": "2021-12-04T15:54:03.230785Z", "shell.execute_reply": "2021-12-04T15:54:03.230390Z"}, "papermill": {"duration": 0.213933, "end_time": "2021-12-04T15:54:03.230894", "exception": false, "start_time": "2021-12-04T15:54:03.016961", "status": "completed"}, "tags": []}, "outputs": [], "source": ["def visualize_samples(data, label):\n", "    if isinstance(data, torch.Tensor):\n", "        data = data.cpu().numpy()\n", "    if isinstance(label, torch.Tensor):\n", "        label = label.cpu().numpy()\n", "    data_0 = data[label == 0]\n", "    data_1 = data[label == 1]\n", "\n", "    plt.figure(figsize=(4, 4))\n", "    plt.scatter(data_0[:, 0], data_0[:, 1], edgecolor=\"#333\", label=\"Class 0\")\n", "    plt.scatter(data_1[:, 0], data_1[:, 1], edgecolor=\"#333\", label=\"Class 1\")\n", "    plt.title(\"Dataset samples\")\n", "    plt.ylabel(r\"$x_2$\")\n", "    plt.xlabel(r\"$x_1$\")\n", "    plt.legend()"]}, {"cell_type": "code", "execution_count": 42, "id": "7b382126", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:54:03.657738Z", "iopub.status.busy": "2021-12-04T15:54:03.645550Z", "iopub.status.idle": "2021-12-04T15:54:04.134432Z", "shell.execute_reply": "2021-12-04T15:54:04.134847Z"}, "papermill": {"duration": 0.697237, "end_time": "2021-12-04T15:54:04.135014", "exception": false, "start_time": "2021-12-04T15:54:03.437777", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUgovR3JvdXAgPDwgL0NTIC9EZXZpY2VSR0IgL1MgL1RyYW5zcGFyZW5jeSAvVHlwZSAvR3JvdXAgPj4KL01lZGlhQm94IFsgMCAwIDI4OC45Nzc0NDU1MTg0IDI3Ny4zMDg3NSBdIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovVHlwZSAvUGFnZSA+PgplbmRvYmoKOSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEyIDAgUiA+PgpzdHJlYW0KeJytm0uvHLcRhffzK3ppL26LxTeXVhQbMJCFEyFZBFkIiqxYkJTYsuP8/HyHPfdO91XNSAacIIEvzSaL9Th1qsix5c3pyVe2vP6whOUN//t1seWb5cmzV//94eWrP3/zdHn54RQYf3eKva+jtZwrf77d/xlbW1PorTAcDn/963R6f2J1vviGhV+fTiWuVpLFsqS2lpyYxtKtrOnR6Nv9aCx5HX0bvqywH2Wn708/Ls7yMaY1LtHamvPy06vlb8v75clXcR55tWil9spW/JHr+T/tFNYWSu8xlTiWn14j+xv+9yuTIie5+t3y6LvTj5xe6rNlVLSSem2jJVt6Qnm1pFBKqMvLd8uTP4Xl2b8fZlsqa8tFc3LkL1uthzjKYGNneqzr4IS92AhxubO1ppIt5TC8xe+shDXUGrBeroO/c1hrHGbVsjcfVWLPGizXstwxN6Wc6hi5e7NTWFNGA4kJabGy5h5LLz2V5p202YrQOEyKmM34mu/iqBayd9Q7C2uvvaeYQ6pLwxSlmulrd3bqa0gNPcaaddSSVrNgtTR2c8TJaeWkIYVRel2kqdF6TaOO7s1nA0ypuXmMlKTLujY27K1iRG8HfDSYjd4xUV+s9tVCDy3hT+4GOO5oiCO/2rTbG+qxYa5AZe38+2yZQy95Re2xjpyjaysjTIY8NofaJD0h1Rs2ybh38pcPKWJfYwNifdW3o2Ji17iVsyYkx2NwzJFXonG0kWr0zorqcw8F6UPGsqEi2VAUeILEvjYMma30MJaYV5yX8FKUuCfNeQ094vEjjX4fAzmg+JHdEJRjlhBTbTqqtbqWJjdLPbua6WuPMZpCHOkAQaFgaxbdmJKjt9hHTJG4uDOT3awmmdUVx1AeIYQrELibn+JG3WJwdXlnKAgnbHJc04GxRRYi1Mge3g58wFpWwwhpzB1iiynknq7ACLAxCO0YRgLF2BFn6q0n8G8M1x3S2nNp6IUIn7He0BfgQMB5O2QpkQVNkRuJKyYXAs11iLpG47hoEmGImm5gCg5VfGDIzEi9ICzYdMbANmK2kcz1ZVJLLYAqsSr3IzmlCvS3VvzQApqIogZmE0msHnoGYYsPVKawLiHXjDsS5UD4YKb+9GFqZe3RQA+LSkEFP6i1u4oxHMHAO2Zg16Jj4BUlZveYAzdH7MjJirwGNxsjBDKL65YBH0DnVhVXqD2uHBjrFozthfggHQB8QzEYlb9zIcqLH7NllcxZ2N05pebmyKGvRjjBB5ThsxOJUWIJHW+35lt0rCCR8gfaIcJI0dZIsoEvvCyb1kR6B2FIhUuKzCZmJgb58oDsKKOalKnT4DJ4P/Fdk+sDeHiAEZBsgW6QFnIAxHURho9n97VWshL6IIXLefPooE/0nRf/IvHgIZFwsFAkWR8s755UwZaETMwiawrmiXEYA5DmT68rzhdAlhL1dzVcMgOULOEeFYGJ1mpiBnlRGkqRPMvn7mHvRIFIs0hhecgvUX/JbaZxHy+BL8hTSKmMwZEraRTK0APZ6iog4BFsQWYem7mAktZQkvuFRMgDHtyTqI0JIHA+HCK4rm+GFjGBidos+J4R6rkhnY+uIgbsXtgiTZIGc2kRLgCv9U9cUBHBBxNlA80HcAbplyTnhSKQQ0YrKAhmLJTIUEixsO6boALHbBAjSahsJoG3WR8Ejasg4k/5h7RsYi5pDeAnkZ+ylx0gloUAr+QSEBXpQohRcNh8cVAP+QCODybrYxPp5gNzU08vIsUs3s1gpYlsSiaJGev5LBZaZ5AiyhmrSu5YFc8pgJ0HDaheaA//INRZPKAjSKnA3M0NsBs0H4Lyp2yLdDLWEFxdIVJQtI5KMiQ5xLXL0AP491GZcCIusvIw9J1Q5ssrsDapCDQRaox3jo05II14WnJJ8t0gVZJPOhAiK0exO2Knuma1tLYSUiZSq3jUSpZrAs3s6UYoj9ZKhOQvglhtQ53pohouLkBGc4iyiIp3mCmHFSX0YY2En2JXGGFjnKDnOl34SvUAfy2imA31qPYApyqFGbp0Y4oNVFUBsLIjahR+Q3b9JE6mVc5WRE8rQcFJKdXFWBmpEqkYSQBIEllLhI7KBVwfC/Jf8Eb5j616qqRwMvOVXGV4DFgGGpTpoKge9IRqdh/MqKOoqyjy4JiiW0H+2TJQcgU92ipSkZhC/En1nFY13mie+KLDgDcADPFeKOJw/c5ho1tnANZjbYq9TvE2tr9TA5gIMDeestYnI8JvKUxEHQbGCJgu+RuoICd7V4Iqb1hYRGIGYeDJP7AmboA9KQo7OAmKYWlzsSaCddP0uDq6RLPUb8L64PN0ajCVzNax55Y7G3kTLyr+fEzJciBrzUsFyAhWknNzXT4SIiAjTgZcUJHBYoDh4vcdZtmiLM5/SQdb9Ytexbtj8HyetKcOCUgUNjupJsGuFMy+avKEYZacxCIqwJvE8QsM3MrUACpEKkECmUWppMve/QLD4HRQAnUn6izDAnal4Metq9/dgCQQUYO0Q5hWGALAAA77rAXCR4VXEH3MGk+gT0KpYr+ez1TZiaq3TtSGzxEiRLwLHpgVB2udbGlboyIloAAk655h0QT+Cz2GrCxiuqRhgO9aPMFdqyW4t+qADQ+Aqa7egFtPqbTDhgU4RYAJxYNSnr9xhd38707fLb+1TaekALmlKMVv1ZsDkoGyIaD8qDd3nLw8mrxryEVlvYp1iCMAVf05uAC1Ws1nfdujKBoUbzWrLSd0jY1aZLB6d2bL1lBt8qIK2Il+jTqa3GP3nvJofo5asRFJKaUJr2ArCRnzJ0+cCr5nOAnBkZSZyAdRPZygzpC7Ax5CUgVTR5MLqDGWaxQH9Q9M8GBMShWAqG78uKECnSp5H9x1gV5KjZJUnL6Y8j6BNNT78Tao60hNjlgmpS9NsTeSnCx7G6TZAgA8SPl5UY2cMAAZLnjTwRYLZJeg9pPsRz4D+YIl32JAOs49O65tNkOA09DVbPMMgDapXChwprDKtrVGbXfP5u0xxM8uXwYQ2uzikctERYenSmKZ2WjOINZgWqagRTUpur4ThSsU0tSOhH5S36yraDBfK2Qjshkcss1+JcxS9SnMO15x5SY94zoNBrFE9WeDOm25eCcFxCqVnJymCK/VZyZ15+JaFOJGgEIHYflDWsftW+LcublxInxOYoO4rvokImgqwMj77gZqCRDoKlo4HCwefWPaFIp71L4WUA9HbJT04CuAqZrRd3fSMMU0BmV+Ua+KMgv4IxN78Ye3ptyHehHszbeQuYqlxvBi407dx8xKlFZJ7hgIb9yn4WP3bfRHeIDsQAeIPbKuAALBSDam9KIc9BEHS2JINe/EOsi06g8g0RU8wM3AMnIxbh631AnjYIMe+pWIogSHYOeeZy89qqkF2kNbmhdSsaoj2oQ44gZkLEpZm2WUKw8FYFVBh7ePDUGLlB9seBrCuFTbFIspyI2LKgFdeVwJWPVbwSb+dVY3mjJkpYKBE1HTueJQiyjl4IfQgU09KqF0ZZOuGCCpHwQmwCgAKPBKRVunDKjeDnfihWRXHKckhZaqKhH25jsE5hHzaBVCdr4IIaio6bqvn0nNRPuS7vRwH8LE4LsujGztEjJgEkyNzX0wNPotLninMLvJsyU0pYcciN740idiT8SKYIwqGcvoYhYt+J4ADyNSImR8HrVCcnNpaoY0X/UqD0g0oL0sR+pHLHmyuZ6flRk4F7ElxFTg48gsn31NqrEI/KkZl7saizMTUpUqQTgf6A4GQC0AZ9q6jORlEHr4YbWqqIGKxaiuCl4NB8zwUBcwqSY412xx6l5i9s1EttWw9ZkIlQ54Ogq5uwhx1QyouvSBu3hRG3RhlnQPlKbf4D7IXnQN4gG46lbKQDXzJoCDm1kF+UPd+BEmVLEgZZR56ZN0swZsNh83kb4TIORD4ntRI3JEWDdlW/Tk0a0sfqXbTAWt6oWhNg9A4kpPeuuDbFjDvGjVNY/hBsSxqxxllEAdqy6wboXW2bKB02df+qAsgqqpFGuft8Aix6CuL8+dPDMAsbr6mjWDwWiHUoy3PsYpZBExRLkObhfVf+/JzZ3K+5nCiZpX/UpdxouxUDT46VDXZVR2BFUfkwBj69461MIzrO4VhAdjXvWhEMkxhisI4az71FYsb02cjPyjm+9hABPlNsyS+mmcPUwt90a0ukpUNw4HMZVqaiOp11ZhcpBwN3Va62LSkHmSTdq6d4RTU6O++iFLbVNAXtJbV8VOZmuZqIp+CKoza1uuUlGtO5NEEqIQNm95dSoTLD7VyaJX5Vhcx0dW1AcVVSM8NjEFNUZHQJLG//t5X0U3OarJltuVMVmIKoi4unbaJEI6C3WdlvDALUq4Ig/KISp0X637cBMaqujtNbpOTMWeVPmC9Xm7FiRjDj0YcEOWvEdaY8E+CzHyVIYW4Q1XaIL6/mKsseGTbXb5FfFEguv3Jrc3tdq2mw+USxmsW8fhp54V6NA1bikCbyIAVVIbuq4siqM0SA6ftp2tHI7C19X7oKmLhZ0wflQLKuoyND/crh3mdnCywCbFcCbJJ6TIC3i9nxZm7xTH0ZVFnU2o3lQMBtXDjigdrwQ8gFI1//WwgOyWVZq7BJZqUSpOumKqFEAotdnEGzdCupKkGr8G91QrMkUVs747KmngAbqiVOsfnqPmFtF+hdjragBR8ffZSi94A0TEX37yLLydOpBC9vzABNYkh7+iyaoSTc8/SGV5tiKbOu+5tisRm9Xvq9J10JOOAS3GB6JbVefZh4HTRMGjoAYYmW+TvJOSXQO5TKXgJGVDyWZcgQ7doVH1CWqqal6VkmkS4hhcin6X9fhG3p3U+dfzArQ03d93MXIG4EVaaIJJgJz8kPXYxEf6UvEZ0qVum8+lLBmEgVwO1GlrOt0/+lq2lhH1u5pGes61vD7xqRhMzft3abtBsZTtpdnpKQL8em4k3U2Vq3UPdTS9eVqAy1nTn54+X558LQdann9/kr/2aUS+ev7P09+XLyDE5cvlH8vzb09/fH76Ua2q7WQP/6ATfG13z169efHXX/7y4v2Hu3c/vP/lA0daONJpnuA0tDtxFfeS7wZvSd4hzjg2Fqc69gUPO3lDuMh7v/28mM3E+X773eCt7ZWK+rxhbbqX+LQAe4U9CADbp8jVCfYS7EZvilD0Pi3B7vUo6TNEKI4OYjBSWFTU7F817kZvijB0Nz8aPAn0+wwRmqMF6oV1qHo/Pqzcjd4SIcb5UKzpNVL+tAjmegJVO4QxV6Dg8LbzMnpTBEoyShhKJOqm/BkiHH3hYFLo2dZdJgXObuS2kD5ft3/BIk++judVv/jfl8vzNydd4ZLa+rzbRuYZrHMi2WzOszlvd+RLHxvUT2JEOrJ6549H3SMHvbXTfpmzk16gDNuVz+dgx+8AHRfx+9a/LP0g/mXUFV9F7lRXm3UHTAai128jyMducxFCpWaZ92AHKXbDt8VQ9QSJzHrTMwuJG0ByU45U9dxWdPogx2X4E3LoaVBXt6/E/LE593Lkm3LoTqWrPXSU4zL8CTkIqaEbiqincDflqDflGLbOpyqP7HIZ/oQcfTb29XTCJk+4Lke/JUekPA/BUOpBjt3wbTnifE9GOic13XJTu+2mrAaHV5l7FOMy/AkxMhWF7mNj255u3sC4I8RtLEncB0cfQlI1cVvVP/0+CBcvCBeXb5fwwJOOvwhwX/H7D/NPf3Gf97+79ryf+b/hNwKH2Zdlbq0e5rl214bL60P2gDIDwkWqibpk6w/a3YwUj0Z69uLnFx9e/bx8ePHuP29ffdgb7MlXabu+fP2gyPOvIGIg9LfMpHcdFCN9k7r2e6l3428ZV99JXbD9uFLrbnbTU7p4HI33c1+e9uOcK6d5xLfHccObgC3I4G5H9UPSWSEX+S6jLw/nuYy/PdlQEz+3+QDvYZxC6TL7YcfDaLisvR+/nOftcfzh9Psdd5py9f1Svxp5evrUj0Ee/6jjXD08/iHI6eYPQU5RbauzCGVOPZtcvaEGAc378aiaL82qX6ON1ErFpdEy5v2MbldTpsSc6tdVZ50PwTWuR2Rpg7uHHaOeIukZ6mE03e+3rfEwrqsgdfOXw8qF+k2N5KMcRTWSjW2NndR1ez2pHXcnrJcT7vRR93KATD1ubxp2s/WcSR3748py3drDYzmCSvyslzl7qRl1Trgf3etjP37R3n7lnaZ3chzsspN6Z8XdCYOrj3CRY7poOP+E6hFW6QnD7pueN7e+mlD+8PbFBxx9j1G3n0t83puKg2/rndR47Njnwb1/iEyZ+vgHb2K06pcwj5w66QmjfjOy91L9dGWMNOw42udWR4dO8Ouhy9aDQ+cJfI/iSreiNt9nH9w5mxbj0Ad3ZnQ71+X856GjIz/Mu/jDfsGL9+x337naRdKdV+4OtR99OP9+8KKq3ao7rV4kOFjgIu7eXJdzPdj1I/t/vuNq5VvkdHNa2zvtd6f/A2CYYAEKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iago0ODQ0CmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjE5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggOTUgPj4Kc3RyZWFtCnicPYxBDsAgCATvvGI/0AQRFf/TND3Y/1+7RtsLTHZhSjcoDiucVRXFG84kHz6SvcNax5CimUdDnN3cFg5LjRSrWBYWnmERpLQ1zPi8KGtgSinqaWf1v7vlegH/nxwsCmVuZHN0cmVhbQplbmRvYmoKMTcgMCBvYmoKPDwgL0Jhc2VGb250IC9EZWphVnVTYW5zLU9ibGlxdWUgL0NoYXJQcm9jcyAxOCAwIFIKL0VuY29kaW5nIDw8IC9EaWZmZXJlbmNlcyBbIDEyMCAveCBdIC9UeXBlIC9FbmNvZGluZyA+PiAvRmlyc3RDaGFyIDAKL0ZvbnRCQm94IFsgLTEwMTYgLTM1MSAxNjYwIDEwNjggXSAvRm9udERlc2NyaXB0b3IgMTYgMCBSCi9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdIC9MYXN0Q2hhciAyNTUgL05hbWUgL0RlamFWdVNhbnMtT2JsaXF1ZQovU3VidHlwZSAvVHlwZTMgL1R5cGUgL0ZvbnQgL1dpZHRocyAxNSAwIFIgPj4KZW5kb2JqCjE2IDAgb2JqCjw8IC9Bc2NlbnQgOTI5IC9DYXBIZWlnaHQgMCAvRGVzY2VudCAtMjM2IC9GbGFncyA5NgovRm9udEJCb3ggWyAtMTAxNiAtMzUxIDE2NjAgMTA2OCBdIC9Gb250TmFtZSAvRGVqYVZ1U2Fucy1PYmxpcXVlCi9JdGFsaWNBbmdsZSAwIC9NYXhXaWR0aCAxMzUwIC9TdGVtViAwIC9UeXBlIC9Gb250RGVzY3JpcHRvciAvWEhlaWdodCAwID4+CmVuZG9iagoxNSAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzUwIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjggNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjE3IDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTcgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwOAo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTk1IDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOCAwIG9iago8PCAveCAxOSAwIFIgPj4KZW5kb2JqCjI0IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjM1ID4+CnN0cmVhbQp4nDVRSW4AMQi75xX+QKWwJ++Zquqh/f+1hlEvAwPY2CTvwUYkPsSQ7ihXfMrqNMvwO1nkxc9K4eS9iAqkKsIKaQfPclYzDJ4bmQKXM/FZZj6ZFjsWUE3EcXbkNINBiGlcR8vpMNM86Am5PhhxY6dZrmJI691Svb7X8p8qykfW3Sy3TtnUSt2iZ+xJXHZeT21pXxh1FDcFkQ4fO7wH+SLmLC46kW72mymHlaQhOC2AH4mhVM8OrxEmfmYkeMqeTu+jNLz2QdP1vXtBR24mZCq3UEYqnqw0xoyh+o1oJqnv/4Ge9b2+/gBDTVS5CmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNjQgPj4Kc3RyZWFtCnicPZDBEUMhCETvVrElgIBAPclkcvi//2tAk1xkHWD3qTuBkFGHM8Nn4smD07E0cG8VjGsIryP0CE0Ck8DEwZp4DAsBp2GRYy7fVZZVp5Wumo2e171jQdVplzUNbdqB8q2PP8I13qPwGuweQgexKHRuZVoLmVg8a5w7zKPM535O23c9GK2m1Kw3ctnXPTrL1FBeWvuEzmi0/SfXL7sxXh+FFDkICmVuZHN0cmVhbQplbmRvYmoKMjYgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzMDcgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDI0OSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjggMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzOTUgPj4Kc3RyZWFtCnicPVJLbsVACNvnFFyg0vCbz3lSVd28+29rQ1KpKryJMcYwfcqQueVLXRJxhcm3Xq5bPKZ8LltamXmIu4uNJT623JfuIbZddC6xOB1H8gsynSpEqM2q0aH4QpaFB5BO8KELwn05/uMvgMHXsA244T0yQbAk5ilCxm5RGZoSQRFh55EVqKRQn1nC31Hu6/cyBWpvjKULYxz0CbQFQm1IxALqQABE7JRUrZCOZyQTvxXdZ2IcYOfRsgGuGVRElnvsx4ipzqiMvETEPk9N+iiWTC1Wxm5TGV/8lIzUfHQFKqk08pTy0FWz0AtYiXkS9jn8SPjn1mwhhjpu1vKJ5R8zxTISzmBLOWChl+NH4NtZdRGuHbm4znSBH5XWcEy0637I9U/+dNtazXW8cgiiQOVNQfC7Dq5GscTEMj6djSl6oiywGpq8RjPBYRAR1vfDyAMa/XK8EDSnayK0WCKbtWJEjYpscz29BNZM78U51sMTwmzvndahsjMzKiGC2rqGautAdrO+83C2nz8z6KJtCmVuZHN0cmVhbQplbmRvYmoKMjkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNDkgPj4Kc3RyZWFtCnicTVFJigMwDLvnFfpAIV6TvKdDmUPn/9fKDoU5BAmvkpOWmFgLDzGEHyw9+JEhczf9G36i2btZepLJ2f+Y5yJTUfhSqC5iQl2IG8+hEfA9oWsSWbG98Tkso5lzvgcfhbgEM6EBY31JMrmo5pUhE04MdRwOWqTCuGtiw+Ja0TyN3G77RmZlJoQNj2RC3BiAiCDrArIYLJQ2NhMyWc4D7Q3JDVpg16kbUYuCK5TWCXSiVsSqzOCz5tZ2N0Mt8uCoffH6aFaXYIXRS/VYeF+FPpipmXbukkJ64U07IsweCqQyOy0rtXvE6m6B+j/LUvD9yff4Ha8PzfxcnAplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggOTQgPj4Kc3RyZWFtCnicRY3BEcAgCAT/VEEJCgraTyaTh/b/jRAyfGDnDu6EBQu2eUYfBZUmXhVYB0pj3FCPQL3hci3J3AUPcCd/2tBUnJbTd2mRSVUp3KQSef8OZyaQqHnRY533C2P7IzwKZW5kc3RyZWFtCmVuZG9iagozMSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDQ3ID4+CnN0cmVhbQp4nDMyt1AwULA0ARKGFiYK5mYGCimGXJYQVi4XTCwHzALRlnAKIp7BlQYAuWcNJwplbmRzdHJlYW0KZW5kb2JqCjMyIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjU4ID4+CnN0cmVhbQp4nEWRS3IEIAhE956CI4D85DyTSmUxuf82Dc5kNnaXqP2ESiOmEiznFHkwfcnyzWS26Xc5VjsbBRRFKJjJVeixAqs7U8SZa4lq62Nl5LjTOwbFG85dOalkcaOMdVR1KnBMz5X1Ud35dlmUfUcOZQrYrHMcbODKbcMYJ0abre4O94kgTydTR8XtINnwByeNfZWrK3CdbPbRSzAOBP1CE5jki0DrDIHGzVP05BLs4+N254Fgb3kRSNkQyJEhGB2Cdp1c/+LW+b3/cYY7z7UZrhzv4neY1nbHX2KSFXMBi9wpqOdrLlrXGTrekzPH5Kb7hs65YJe7g0zv+T/Wz/r+Ax4pZvoKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvQkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzOQovU3VidHlwZSAvRm9ybSAvVHlwZSAvWE9iamVjdCA+PgpzdHJlYW0KeJzjMjQwUzA2NVXI5TI3NgKzcsAsI3MjIAski2BBZDO40gAV8wp8CmVuZHN0cmVhbQplbmRvYmoKMzQgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA4MyA+PgpzdHJlYW0KeJxFjLsNwDAIRHumYAR+JvY+UZTC3r8NECVuuCfdPVwdCZkpbjPDQwaeDCyGXXGB9JYwC1xHUI6d7KNh1b7qBI31plLz7w+Unuys4obrAQJCGmYKZW5kc3RyZWFtCmVuZG9iagozNSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIzOSA+PgpzdHJlYW0KeJxNUMltBDEM+7sKNTDA6By7HgeLPLL9f0PKCZKXaEviofKUW5bKZfcjOW/JuuVDh06VafJu0M2vsf6jDAJ2/1BUEK0lsUrMXNJusTRJL9nDOI2Xa7WO56l7hFmjePDj2NMpgek9MsFms705MKs9zg6QTrjGr+rTO5UkA4m6kPNCpQrrHtQloo8r25hSnU4t5RiXn+h7fI4APcXejdzRx8sXjEa1LajRapU4DzATU9GVcauRgZQTBkNnR1c0C6XIynpCNcKNOaGZvcNwYAPLs4Skpa1SvA9lAegCXdo64zRKgo4Awt8ojPX6Bqr8XjcKZW5kc3RyZWFtCmVuZG9iagozNiAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDUxID4+CnN0cmVhbQp4nDM2tFAwUDA0MAeSRoZAlpGJQoohF0gAxMzlggnmgFkGQBqiOAeuJocrgysNAOG0DZgKZW5kc3RyZWFtCmVuZG9iagozNyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDMzNCA+PgpzdHJlYW0KeJwtUktyxSAM23MKXaAz+AfkPOl0uni9/7aSk0VGDmD0MeWGiUp8WSC3o9bEt43MQIXhr6vMhc9I28g6iMuQi7iSLYV7RCzkMcQ8xILvq/EeHvmszMmzB8Yv2XcPK/bUhGUh48UZ2mEVx2EV5FiwdSGqe3hTpMOpJNjji/8+xXMtBC18RtCAX+Sfr47g+ZIWafeYbdOuerBMO6qksBxsT3NeJl9aZ7k6Hs8Hyfau2BFSuwIUhbkzznPhKNNWRrQWdjZIalxsb479WErQhW5cRoojkJ+pIjygpMnMJgrij5wecioDYeqarnRyG1Vxp57MNZuLtzNJZuu+SLGZwnldOLP+DFNmtXknz3Ki1KkI77FnS9DQOa6evZZZaHSbE7ykhM/GTk9Ovlcz6yE5FQmpYlpXwWkUmWIJ2xJfU1FTmnoZ/vvy7vE7fv4BLHN8cwplbmRzdHJlYW0KZW5kb2JqCjM4IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNzAgPj4Kc3RyZWFtCnicMzM2UzBQsDACEqamhgrmRpYKKYZcQD6IlcsFE8sBs8wszIEsIwuQlhwuQwtjMG1ibKRgZmIGZFkgMSC6MrjSAJiaEwMKZW5kc3RyZWFtCmVuZG9iagozOSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDMyMCA+PgpzdHJlYW0KeJw1UktuBTEI288puECl8E/O86qqi777b2sTvRVMMGDjKS9Z0ku+1CXbpcPkWx/3JbFC3o/tmsxSxfcWsxTPLa9HzxG3LQoEURM9WJkvFSLUz/ToOqhwSp+BVwi3FBu8g0kAg2r4Bx6lMyBQ50DGu2IyUgOCJNhzaXEIiXImiX+kvJ7fJ62kofQ9WZnL35NLpdAdTU7oAcXKxUmgXUn5oJmYSkSSl+t9sUL0hsCSPD5HMcmA7DaJbaIFJucepSXMxBQ6sMcCvGaa1VXoYMIehymMVwuzqB5s8lsTlaQdreMZ2TDeyzBTYqHhsAXU5mJlgu7l4zWvwojtUZNdw3Duls13CNFo/hsWyuBjFZKAR6exEg1pOMCIwJ5eOMVe8xM5DsCIY52aLAxjaCaneo6JwNCes6VhxsceWvXzD1TpfIcKZW5kc3RyZWFtCmVuZG9iago0MCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDE4ID4+CnN0cmVhbQp4nDM2tFAwgMMUQ640AB3mA1IKZW5kc3RyZWFtCmVuZG9iago0MSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEzMyA+PgpzdHJlYW0KeJxFj0sOBCEIRPecoo7Axx/ncTLphXP/7YCdbhNjPYVUgbmCoT0uawOdFR8hGbbxt6mWjkVZPlR6UlYPyeCHrMbLIdygLPCCSSqGIVCLmBqRLWVut4DbNg2yspVTpY6wi6Mwj/a0bBUeX6JbInWSP4PEKi/c47odyKXWu96ii75/pAExCQplbmRzdHJlYW0KZW5kb2JqCjQyIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjUxID4+CnN0cmVhbQp4nC1RSXIDQQi7zyv0hGan32OXK4fk/9cIygcGDYtAdFrioIyfICxXvOWRq2jD3zMxgt8Fh34r121Y5EBUIEljUDWhdvF69B7YcZgJzJPWsAxmrA/8jCnc6MXhMRlnt9dl1BDsXa89mUHJrFzEJRMXTNVhI2cOP5kyLrRzPTcg50ZYl2GQblYaMxKONIVIIYWqm6TOBEESjK5GjTZyFPulL490hlWNqDHscy1tX89NOGvQ7Fis8uSUHl1xLicXL6wc9PU2AxdRaazyQEjA/W4P9XOyk994S+fOFtPje83J8sJUYMWb125ANtXi37yI4/uMr+fn+fwDX2BbiAplbmRzdHJlYW0KZW5kb2JqCjQzIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjE1ID4+CnN0cmVhbQp4nDVROQ4DIQzs9xX+QCSML3hPoijN/r/NjNFWHsFchrSUIZnyUpOoIeVTPnqZLpy63NfMajTnlrQtc4C4trwvrZLAiWaIg8FpmLgBmjwBQ9fRqFFDFx7Q1KVTKLDcBD6Kt24P3WO1gZe2IeeJIGIoGSxBzalFExZtzyekNb9eixvel+3dyFOlxpYYgQYBVjgc1+jX8JU9TybRdBUy1Ks1yxgJE0UiPPmOptUT61o00jIS1MYRrGoDvDv9ME4AABNxywJkn0qUs+TEb7H0swZX+v4Bn0dUlgplbmRzdHJlYW0KZW5kb2JqCjIyIDAgb2JqCjw8IC9CYXNlRm9udCAvRGVqYVZ1U2FucyAvQ2hhclByb2NzIDIzIDAgUgovRW5jb2RpbmcgPDwKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDQ2IC9wZXJpb2QgNDggL3plcm8gL29uZSAvdHdvIDUyIC9mb3VyIC9maXZlIC9zaXggL3NldmVuCi9laWdodCA2NyAvQyAvRCA5NyAvYSAxMDEgL2UgMTA4IC9sIC9tIDExMiAvcCAxMTUgL3MgL3QgXQovVHlwZSAvRW5jb2RpbmcgPj4KL0ZpcnN0Q2hhciAwIC9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnREZXNjcmlwdG9yIDIxIDAgUgovRm9udE1hdHJpeCBbIDAuMDAxIDAgMCAwLjAwMSAwIDAgXSAvTGFzdENoYXIgMjU1IC9OYW1lIC9EZWphVnVTYW5zCi9TdWJ0eXBlIC9UeXBlMyAvVHlwZSAvRm9udCAvV2lkdGhzIDIwIDAgUiA+PgplbmRvYmoKMjEgMCBvYmoKPDwgL0FzY2VudCA5MjkgL0NhcEhlaWdodCAwIC9EZXNjZW50IC0yMzYgL0ZsYWdzIDMyCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnROYW1lIC9EZWphVnVTYW5zIC9JdGFsaWNBbmdsZSAwCi9NYXhXaWR0aCAxMzQyIC9TdGVtViAwIC9UeXBlIC9Gb250RGVzY3JpcHRvciAvWEhlaWdodCAwID4+CmVuZG9iagoyMCAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzQyIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjMgNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjEyIDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTIgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwNQo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTgyIDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoyMyAwIG9iago8PCAvQyAyNCAwIFIgL0QgMjUgMCBSIC9hIDI2IDAgUiAvZSAyNyAwIFIgL2VpZ2h0IDI4IDAgUiAvZml2ZSAyOSAwIFIKL2ZvdXIgMzAgMCBSIC9sIDMxIDAgUiAvbSAzMiAwIFIgL29uZSAzNCAwIFIgL3AgMzUgMCBSIC9wZXJpb2QgMzYgMCBSCi9zIDM3IDAgUiAvc2V2ZW4gMzggMCBSIC9zaXggMzkgMCBSIC9zcGFjZSA0MCAwIFIgL3QgNDEgMCBSIC90d28gNDIgMCBSCi96ZXJvIDQzIDAgUiA+PgplbmRvYmoKMyAwIG9iago8PCAvRjEgMjIgMCBSIC9GMiAxNyAwIFIgPj4KZW5kb2JqCjQgMCBvYmoKPDwgL0ExIDw8IC9DQSAwIC9UeXBlIC9FeHRHU3RhdGUgL2NhIDEgPj4KL0EyIDw8IC9DQSAxIC9UeXBlIC9FeHRHU3RhdGUgL2NhIDEgPj4KL0EzIDw8IC9DQSAwLjggL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMC44ID4+ID4+CmVuZG9iago1IDAgb2JqCjw8ID4+CmVuZG9iago2IDAgb2JqCjw8ID4+CmVuZG9iago3IDAgb2JqCjw8IC9GMS1EZWphVnVTYW5zLW1pbnVzIDMzIDAgUiAvTTAgMTMgMCBSIC9NMSAxNCAwIFIgPj4KZW5kb2JqCjEzIDAgb2JqCjw8IC9CQm94IFsgLTggLTggOCA4IF0gL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMzEgL1N1YnR5cGUgL0Zvcm0KL1R5cGUgL1hPYmplY3QgPj4Kc3RyZWFtCnicbZBBDoQgDEX3PUUv8ElLRWXr0mu4mUzi/bcDcUBM3TTQvjx+Uf6S8E6lwPgkCUtOs+R605DSukyMGObVsijHoFEt1s51OKjP0HBjdIuxFKbU1uh4o5vpNt6TP/qwWSFGPxwOr4R7FkMmXCkxBoffCy/bw/8Rnl7UwB+ijX5jWkP9CmVuZHN0cmVhbQplbmRvYmoKMTQgMCBvYmoKPDwgL0JCb3ggWyAtOCAtOCA4IDggXSAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEzMSAvU3VidHlwZSAvRm9ybQovVHlwZSAvWE9iamVjdCA+PgpzdHJlYW0KeJxtkEEOhCAMRfc9RS/wSUtFZevSa7iZTOL9twNxQEzdNNC+PH5R/pLwTqXA+CQJS06z5HrTkNK6TIwY5tWyKMegUS3WznU4qM/QcGN0i7EUptTW6Hijm+k23pM/+rBZIUY/HA6vhHsWQyZcKTEGh98LL9vD/xGeXtTAH6KNfmNaQ/0KZW5kc3RyZWFtCmVuZG9iagoyIDAgb2JqCjw8IC9Db3VudCAxIC9LaWRzIFsgMTEgMCBSIF0gL1R5cGUgL1BhZ2VzID4+CmVuZG9iago0NCAwIG9iago8PCAvQ3JlYXRpb25EYXRlIChEOjIwMjExMjA0MTY1NDA0KzAyJzAwJykKL0NyZWF0b3IgKE1hdHBsb3RsaWIgdjMuNC4zLCBodHRwczovL21hdHBsb3RsaWIub3JnKQovUHJvZHVjZXIgKE1hdHBsb3RsaWIgcGRmIGJhY2tlbmQgdjMuNC4zKSA+PgplbmRvYmoKeHJlZgowIDQ1CjAwMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAwMDAxNiAwMDAwMCBuIAowMDAwMDE1MTI1IDAwMDAwIG4gCjAwMDAwMTQzMTkgMDAwMDAgbiAKMDAwMDAxNDM2MiAwMDAwMCBuIAowMDAwMDE0NTA0IDAwMDAwIG4gCjAwMDAwMTQ1MjUgMDAwMDAgbiAKMDAwMDAxNDU0NiAwMDAwMCBuIAowMDAwMDAwMDY1IDAwMDAwIG4gCjAwMDAwMDA0MDUgMDAwMDAgbiAKMDAwMDAwNTM0NSAwMDAwMCBuIAowMDAwMDAwMjA4IDAwMDAwIG4gCjAwMDAwMDUzMjQgMDAwMDAgbiAKMDAwMDAxNDYxNyAwMDAwMCBuIAowMDAwMDE0ODcxIDAwMDAwIG4gCjAwMDAwMDYwNTYgMDAwMDAgbiAKMDAwMDAwNTg0OCAwMDAwMCBuIAowMDAwMDA1NTMyIDAwMDAwIG4gCjAwMDAwMDcxMDkgMDAwMDAgbiAKMDAwMDAwNTM2NSAwMDAwMCBuIAowMDAwMDEzMDIyIDAwMDAwIG4gCjAwMDAwMTI4MjIgMDAwMDAgbiAKMDAwMDAxMjQwNiAwMDAwMCBuIAowMDAwMDE0MDc1IDAwMDAwIG4gCjAwMDAwMDcxNDEgMDAwMDAgbiAKMDAwMDAwNzQ0OSAwMDAwMCBuIAowMDAwMDA3Njg2IDAwMDAwIG4gCjAwMDAwMDgwNjYgMDAwMDAgbiAKMDAwMDAwODM4OCAwMDAwMCBuIAowMDAwMDA4ODU2IDAwMDAwIG4gCjAwMDAwMDkxNzggMDAwMDAgbiAKMDAwMDAwOTM0NCAwMDAwMCBuIAowMDAwMDA5NDYzIDAwMDAwIG4gCjAwMDAwMDk3OTQgMDAwMDAgbiAKMDAwMDAwOTk2NiAwMDAwMCBuIAowMDAwMDEwMTIxIDAwMDAwIG4gCjAwMDAwMTA0MzMgMDAwMDAgbiAKMDAwMDAxMDU1NiAwMDAwMCBuIAowMDAwMDEwOTYzIDAwMDAwIG4gCjAwMDAwMTExMDUgMDAwMDAgbiAKMDAwMDAxMTQ5OCAwMDAwMCBuIAowMDAwMDExNTg4IDAwMDAwIG4gCjAwMDAwMTE3OTQgMDAwMDAgbiAKMDAwMDAxMjExOCAwMDAwMCBuIAowMDAwMDE1MTg1IDAwMDAwIG4gCnRyYWlsZXIKPDwgL0luZm8gNDQgMCBSIC9Sb290IDEgMCBSIC9TaXplIDQ1ID4+CnN0YXJ0eHJlZgoxNTM0MgolJUVPRgo=\n", "image/svg+xml": ["<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n", "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n", "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n", "<svg height=\"277.314375pt\" version=\"1.1\" viewBox=\"0 0 288.985258 277.314375\" width=\"288.985258pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n", " <metadata>\n", "  <rdf:RDF xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n", "   <cc:Work>\n", "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n", "    <dc:date>2021-12-04T16:54:03.873238</dc:date>\n", "    <dc:format>image/svg+xml</dc:format>\n", "    <dc:creator>\n", "     <cc:Agent>\n", "      <dc:title>Matplotlib v3.4.3, https://matplotlib.org/</dc:title>\n", "     </cc:Agent>\n", "    </dc:creator>\n", "   </cc:Work>\n", "  </rdf:RDF>\n", " </metadata>\n", " <defs>\n", "  <style type=\"text/css\">*{stroke-linecap:butt;stroke-linejoin:round;}</style>\n", " </defs>\n", " <g id=\"figure_1\">\n", "  <g id=\"patch_1\">\n", "   <path d=\"M 0 277.314375 \n", "L 288.985258 277.314375 \n", "L 288.985258 0 \n", "L 0 0 \n", "z\n", "\" style=\"fill:none;\"/>\n", "  </g>\n", "  <g id=\"axes_1\">\n", "   <g id=\"patch_2\">\n", "    <path d=\"M 52.160938 239.758125 \n", "L 275.360937 239.758125 \n", "L 275.360937 22.318125 \n", "L 52.160938 22.318125 \n", "z\n", "\" style=\"fill:#ffffff;\"/>\n", "   </g>\n", "   <g id=\"PathCollection_1\">\n", "    <defs>\n", "     <path d=\"M 0 3 \n", "C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n", "C 2.683901 1.55874 3 0.795609 3 0 \n", "C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n", "C 1.55874 -2.683901 0.795609 -3 0 -3 \n", "C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n", "C -2.683901 -1.55874 -3 -0.795609 -3 0 \n", "C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n", "C -1.55874 2.683901 -0.795609 3 0 3 \n", "z\n", "\" id=\"m12efddd26b\" style=\"stroke:#333333;\"/>\n", "    </defs>\n", "    <g clip-path=\"url(#p7e9b149886)\">\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"96.31168\" xlink:href=\"#m12efddd26b\" y=\"193.325344\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"232.057657\" xlink:href=\"#m12efddd26b\" y=\"62.145049\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"258.979442\" xlink:href=\"#m12efddd26b\" y=\"63.780462\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"108.913368\" xlink:href=\"#m12efddd26b\" y=\"204.409578\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"101.325808\" xlink:href=\"#m12efddd26b\" y=\"205.043015\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"70.983454\" xlink:href=\"#m12efddd26b\" y=\"189.560429\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"242.885541\" xlink:href=\"#m12efddd26b\" y=\"58.566133\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"232.016709\" xlink:href=\"#m12efddd26b\" y=\"51.440521\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"93.978943\" xlink:href=\"#m12efddd26b\" y=\"204.550687\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"237.538974\" xlink:href=\"#m12efddd26b\" y=\"53.572047\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"103.543109\" xlink:href=\"#m12efddd26b\" y=\"200.309835\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"255.555098\" xlink:href=\"#m12efddd26b\" y=\"32.201761\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"238.058002\" xlink:href=\"#m12efddd26b\" y=\"62.589723\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"232.170061\" xlink:href=\"#m12efddd26b\" y=\"57.754453\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"94.264177\" xlink:href=\"#m12efddd26b\" y=\"212.626168\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"89.231835\" xlink:href=\"#m12efddd26b\" y=\"185.037671\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"251.267421\" xlink:href=\"#m12efddd26b\" y=\"90.813692\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"254.815479\" xlink:href=\"#m12efddd26b\" y=\"83.753102\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"226.105337\" xlink:href=\"#m12efddd26b\" y=\"59.138235\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"82.022892\" xlink:href=\"#m12efddd26b\" y=\"209.202243\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"253.973129\" xlink:href=\"#m12efddd26b\" y=\"32.62492\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"245.15091\" xlink:href=\"#m12efddd26b\" y=\"59.955798\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"114.421987\" xlink:href=\"#m12efddd26b\" y=\"175.83743\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"228.657784\" xlink:href=\"#m12efddd26b\" y=\"32.336249\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"99.781014\" xlink:href=\"#m12efddd26b\" y=\"194.777215\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"227.807175\" xlink:href=\"#m12efddd26b\" y=\"51.50491\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"86.881354\" xlink:href=\"#m12efddd26b\" y=\"179.492748\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"250.727063\" xlink:href=\"#m12efddd26b\" y=\"48.77409\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"246.39628\" xlink:href=\"#m12efddd26b\" y=\"61.512745\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"240.179519\" xlink:href=\"#m12efddd26b\" y=\"43.69401\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"92.34098\" xlink:href=\"#m12efddd26b\" y=\"184.373252\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"104.506115\" xlink:href=\"#m12efddd26b\" y=\"209.709719\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"87.360439\" xlink:href=\"#m12efddd26b\" y=\"169.625602\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"97.745486\" xlink:href=\"#m12efddd26b\" y=\"201.318829\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"112.202457\" xlink:href=\"#m12efddd26b\" y=\"200.867622\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"93.386735\" xlink:href=\"#m12efddd26b\" y=\"195.701852\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"112.962408\" xlink:href=\"#m12efddd26b\" y=\"212.211753\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"213.923524\" xlink:href=\"#m12efddd26b\" y=\"39.620323\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"253.408944\" xlink:href=\"#m12efddd26b\" y=\"64.604829\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"238.202599\" xlink:href=\"#m12efddd26b\" y=\"63.759786\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"93.334759\" xlink:href=\"#m12efddd26b\" y=\"205.410605\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"222.697015\" xlink:href=\"#m12efddd26b\" y=\"89.092646\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"246.067518\" xlink:href=\"#m12efddd26b\" y=\"56.781365\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"97.908356\" xlink:href=\"#m12efddd26b\" y=\"212.127081\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"110.608912\" xlink:href=\"#m12efddd26b\" y=\"217.677792\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"119.278606\" xlink:href=\"#m12efddd26b\" y=\"177.02797\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"229.431275\" xlink:href=\"#m12efddd26b\" y=\"71.868079\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"217.108023\" xlink:href=\"#m12efddd26b\" y=\"44.358713\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"90.421001\" xlink:href=\"#m12efddd26b\" y=\"205.813556\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"230.637614\" xlink:href=\"#m12efddd26b\" y=\"68.485687\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"98.642521\" xlink:href=\"#m12efddd26b\" y=\"201.033344\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"242.296554\" xlink:href=\"#m12efddd26b\" y=\"33.609264\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"95.167044\" xlink:href=\"#m12efddd26b\" y=\"188.760034\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"227.659933\" xlink:href=\"#m12efddd26b\" y=\"76.575953\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"239.162594\" xlink:href=\"#m12efddd26b\" y=\"47.436505\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"70.985535\" xlink:href=\"#m12efddd26b\" y=\"190.609003\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"216.984682\" xlink:href=\"#m12efddd26b\" y=\"47.197067\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"247.342435\" xlink:href=\"#m12efddd26b\" y=\"65.6149\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"82.942512\" xlink:href=\"#m12efddd26b\" y=\"196.823089\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"94.610207\" xlink:href=\"#m12efddd26b\" y=\"229.874489\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"94.065144\" xlink:href=\"#m12efddd26b\" y=\"199.872245\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"75.9265\" xlink:href=\"#m12efddd26b\" y=\"200.007537\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"261.562131\" xlink:href=\"#m12efddd26b\" y=\"65.752612\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"227.651459\" xlink:href=\"#m12efddd26b\" y=\"73.759575\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"253.566025\" xlink:href=\"#m12efddd26b\" y=\"39.679857\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"76.490724\" xlink:href=\"#m12efddd26b\" y=\"185.321057\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"238.897525\" xlink:href=\"#m12efddd26b\" y=\"82.429587\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"249.223089\" xlink:href=\"#m12efddd26b\" y=\"89.661589\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"118.387936\" xlink:href=\"#m12efddd26b\" y=\"218.303568\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"108.710707\" xlink:href=\"#m12efddd26b\" y=\"207.070547\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"94.960359\" xlink:href=\"#m12efddd26b\" y=\"190.023846\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"95.804305\" xlink:href=\"#m12efddd26b\" y=\"187.711556\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"239.509346\" xlink:href=\"#m12efddd26b\" y=\"53.869424\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"226.971018\" xlink:href=\"#m12efddd26b\" y=\"88.154106\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"94.916859\" xlink:href=\"#m12efddd26b\" y=\"195.521558\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"106.991269\" xlink:href=\"#m12efddd26b\" y=\"186.850471\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"108.567957\" xlink:href=\"#m12efddd26b\" y=\"194.117803\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"239.227411\" xlink:href=\"#m12efddd26b\" y=\"37.590473\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"238.317384\" xlink:href=\"#m12efddd26b\" y=\"35.754283\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"96.994499\" xlink:href=\"#m12efddd26b\" y=\"208.108344\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"250.032038\" xlink:href=\"#m12efddd26b\" y=\"37.701594\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"232.359401\" xlink:href=\"#m12efddd26b\" y=\"72.374698\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"248.199716\" xlink:href=\"#m12efddd26b\" y=\"33.573869\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"88.429047\" xlink:href=\"#m12efddd26b\" y=\"192.95304\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"234.27057\" xlink:href=\"#m12efddd26b\" y=\"47.995435\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"98.536004\" xlink:href=\"#m12efddd26b\" y=\"212.515425\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"107.800107\" xlink:href=\"#m12efddd26b\" y=\"204.310706\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"87.726796\" xlink:href=\"#m12efddd26b\" y=\"191.216589\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"212.816998\" xlink:href=\"#m12efddd26b\" y=\"53.841351\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"225.168168\" xlink:href=\"#m12efddd26b\" y=\"47.841136\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"202.642744\" xlink:href=\"#m12efddd26b\" y=\"44.693113\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"86.981531\" xlink:href=\"#m12efddd26b\" y=\"191.534849\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"116.687423\" xlink:href=\"#m12efddd26b\" y=\"206.975146\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"91.776781\" xlink:href=\"#m12efddd26b\" y=\"209.817871\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"225.892096\" xlink:href=\"#m12efddd26b\" y=\"90.705434\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"102.087238\" xlink:href=\"#m12efddd26b\" y=\"195.303274\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"94.905243\" xlink:href=\"#m12efddd26b\" y=\"189.148117\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"89.515988\" xlink:href=\"#m12efddd26b\" y=\"200.223394\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"99.167164\" xlink:href=\"#m12efddd26b\" y=\"208.206126\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"211.577945\" xlink:href=\"#m12efddd26b\" y=\"57.867977\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"220.751413\" xlink:href=\"#m12efddd26b\" y=\"41.469595\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"72.137778\" xlink:href=\"#m12efddd26b\" y=\"214.445482\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"235.916363\" xlink:href=\"#m12efddd26b\" y=\"80.464924\"/>\n", "    </g>\n", "   </g>\n", "   <g id=\"PathCollection_2\">\n", "    <defs>\n", "     <path d=\"M 0 3 \n", "C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n", "C 2.683901 1.55874 3 0.795609 3 0 \n", "C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n", "C 1.55874 -2.683901 0.795609 -3 0 -3 \n", "C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n", "C -2.683901 -1.55874 -3 -0.795609 -3 0 \n", "C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n", "C -1.55874 2.683901 -0.795609 3 0 3 \n", "z\n", "\" id=\"m897671b02c\" style=\"stroke:#333333;\"/>\n", "    </defs>\n", "    <g clip-path=\"url(#p7e9b149886)\">\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"230.877063\" xlink:href=\"#m897671b02c\" y=\"180.913557\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"243.842427\" xlink:href=\"#m897671b02c\" y=\"181.183986\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"232.017097\" xlink:href=\"#m897671b02c\" y=\"198.932324\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"89.746376\" xlink:href=\"#m897671b02c\" y=\"45.392998\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"250.686437\" xlink:href=\"#m897671b02c\" y=\"201.599728\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"100.400405\" xlink:href=\"#m897671b02c\" y=\"63.553489\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"219.862038\" xlink:href=\"#m897671b02c\" y=\"211.350014\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"130.59866\" xlink:href=\"#m897671b02c\" y=\"59.599027\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"247.536058\" xlink:href=\"#m897671b02c\" y=\"217.196966\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"214.797166\" xlink:href=\"#m897671b02c\" y=\"185.557909\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"229.906907\" xlink:href=\"#m897671b02c\" y=\"173.017888\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"65.731142\" xlink:href=\"#m897671b02c\" y=\"44.566804\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"116.585574\" xlink:href=\"#m897671b02c\" y=\"45.533014\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"96.4775\" xlink:href=\"#m897671b02c\" y=\"76.495825\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"96.788387\" xlink:href=\"#m897671b02c\" y=\"42.091466\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"75.708547\" xlink:href=\"#m897671b02c\" y=\"39.030573\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"233.922057\" xlink:href=\"#m897671b02c\" y=\"201.757218\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"216.28277\" xlink:href=\"#m897671b02c\" y=\"180.766731\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"225.945553\" xlink:href=\"#m897671b02c\" y=\"182.832732\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"107.888844\" xlink:href=\"#m897671b02c\" y=\"59.25904\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"212.266314\" xlink:href=\"#m897671b02c\" y=\"192.147409\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"257.089413\" xlink:href=\"#m897671b02c\" y=\"200.542537\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"248.528097\" xlink:href=\"#m897671b02c\" y=\"198.107913\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"223.275659\" xlink:href=\"#m897671b02c\" y=\"202.11576\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"254.62465\" xlink:href=\"#m897671b02c\" y=\"176.456154\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"83.200733\" xlink:href=\"#m897671b02c\" y=\"70.175377\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"251.724096\" xlink:href=\"#m897671b02c\" y=\"179.390064\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"110.819287\" xlink:href=\"#m897671b02c\" y=\"55.123203\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"232.126351\" xlink:href=\"#m897671b02c\" y=\"178.669064\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"68.497902\" xlink:href=\"#m897671b02c\" y=\"56.973626\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"95.226666\" xlink:href=\"#m897671b02c\" y=\"66.120244\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"223.392974\" xlink:href=\"#m897671b02c\" y=\"208.672634\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"214.945261\" xlink:href=\"#m897671b02c\" y=\"183.133722\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"100.205618\" xlink:href=\"#m897671b02c\" y=\"60.661631\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"244.127174\" xlink:href=\"#m897671b02c\" y=\"183.85959\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"100.782099\" xlink:href=\"#m897671b02c\" y=\"68.534007\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"97.685973\" xlink:href=\"#m897671b02c\" y=\"71.284393\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"261.24119\" xlink:href=\"#m897671b02c\" y=\"209.357586\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"231.536281\" xlink:href=\"#m897671b02c\" y=\"180.074569\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"84.432042\" xlink:href=\"#m897671b02c\" y=\"55.826098\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"114.704427\" xlink:href=\"#m897671b02c\" y=\"57.881371\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"118.139583\" xlink:href=\"#m897671b02c\" y=\"40.282525\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"244.331312\" xlink:href=\"#m897671b02c\" y=\"206.328325\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"84.937455\" xlink:href=\"#m897671b02c\" y=\"59.985039\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"229.111028\" xlink:href=\"#m897671b02c\" y=\"185.254639\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"99.865846\" xlink:href=\"#m897671b02c\" y=\"73.50443\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"107.774522\" xlink:href=\"#m897671b02c\" y=\"50.128195\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"106.681533\" xlink:href=\"#m897671b02c\" y=\"53.57626\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"220.300095\" xlink:href=\"#m897671b02c\" y=\"186.154142\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"84.93114\" xlink:href=\"#m897671b02c\" y=\"77.508109\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"105.491498\" xlink:href=\"#m897671b02c\" y=\"48.161599\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"212.017878\" xlink:href=\"#m897671b02c\" y=\"182.204423\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"238.780524\" xlink:href=\"#m897671b02c\" y=\"210.592034\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"102.920944\" xlink:href=\"#m897671b02c\" y=\"48.671279\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"109.227118\" xlink:href=\"#m897671b02c\" y=\"67.791241\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"218.916511\" xlink:href=\"#m897671b02c\" y=\"195.110735\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"210.412471\" xlink:href=\"#m897671b02c\" y=\"197.807546\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"106.409234\" xlink:href=\"#m897671b02c\" y=\"62.370039\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"62.306392\" xlink:href=\"#m897671b02c\" y=\"69.539331\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"91.834453\" xlink:href=\"#m897671b02c\" y=\"77.469177\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"109.483259\" xlink:href=\"#m897671b02c\" y=\"52.116365\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"87.52736\" xlink:href=\"#m897671b02c\" y=\"63.223243\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"94.034599\" xlink:href=\"#m897671b02c\" y=\"55.768785\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"110.113367\" xlink:href=\"#m897671b02c\" y=\"68.261083\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"257.318305\" xlink:href=\"#m897671b02c\" y=\"202.317721\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"79.762115\" xlink:href=\"#m897671b02c\" y=\"39.847979\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"258.048499\" xlink:href=\"#m897671b02c\" y=\"185.012766\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"244.789162\" xlink:href=\"#m897671b02c\" y=\"171.865302\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"214.47439\" xlink:href=\"#m897671b02c\" y=\"211.299351\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"218.387835\" xlink:href=\"#m897671b02c\" y=\"227.253664\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"98.247032\" xlink:href=\"#m897671b02c\" y=\"41.347367\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"221.09122\" xlink:href=\"#m897671b02c\" y=\"178.959826\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"207.783237\" xlink:href=\"#m897671b02c\" y=\"165.02484\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"84.486389\" xlink:href=\"#m897671b02c\" y=\"47.120304\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"108.320496\" xlink:href=\"#m897671b02c\" y=\"59.847104\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"247.02495\" xlink:href=\"#m897671b02c\" y=\"184.293941\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"92.024123\" xlink:href=\"#m897671b02c\" y=\"51.399993\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"103.976033\" xlink:href=\"#m897671b02c\" y=\"61.578541\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"102.375453\" xlink:href=\"#m897671b02c\" y=\"64.816787\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"230.527021\" xlink:href=\"#m897671b02c\" y=\"177.171327\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"238.484799\" xlink:href=\"#m897671b02c\" y=\"167.950176\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"247.340005\" xlink:href=\"#m897671b02c\" y=\"185.179964\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"134.8086\" xlink:href=\"#m897671b02c\" y=\"59.304711\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"213.726653\" xlink:href=\"#m897671b02c\" y=\"213.578161\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"265.215483\" xlink:href=\"#m897671b02c\" y=\"212.626448\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"226.839442\" xlink:href=\"#m897671b02c\" y=\"209.593202\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"88.454985\" xlink:href=\"#m897671b02c\" y=\"69.802283\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"86.200641\" xlink:href=\"#m897671b02c\" y=\"59.30147\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"249.708568\" xlink:href=\"#m897671b02c\" y=\"212.467168\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"86.639713\" xlink:href=\"#m897671b02c\" y=\"66.734504\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"72.549088\" xlink:href=\"#m897671b02c\" y=\"61.78785\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"116.327234\" xlink:href=\"#m897671b02c\" y=\"48.616483\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"89.624632\" xlink:href=\"#m897671b02c\" y=\"51.591563\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"257.369976\" xlink:href=\"#m897671b02c\" y=\"209.982902\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"209.334228\" xlink:href=\"#m897671b02c\" y=\"191.955712\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"98.138755\" xlink:href=\"#m897671b02c\" y=\"44.566278\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"254.619078\" xlink:href=\"#m897671b02c\" y=\"207.56031\"/>\n", "    </g>\n", "   </g>\n", "   <g id=\"matplotlib.axis_1\">\n", "    <g id=\"xtick_1\">\n", "     <g id=\"line2d_1\">\n", "      <defs>\n", "       <path d=\"M 0 0 \n", "L 0 3.5 \n", "\" id=\"m522a3f537e\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n", "      </defs>\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"62.596276\" xlink:href=\"#m522a3f537e\" y=\"239.758125\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_1\">\n", "      <!-- \u22120.25 -->\n", "      <g transform=\"translate(47.27362 254.356562)scale(0.1 -0.1)\">\n", "       <defs>\n", "        <path d=\"M 678 2272 \n", "L 4684 2272 \n", "L 4684 1741 \n", "L 678 1741 \n", "L 678 2272 \n", "z\n", "\" id=\"DejaVuSans-2212\" transform=\"scale(0.015625)\"/>\n", "        <path d=\"M 2034 4250 \n", "Q 1547 4250 1301 3770 \n", "Q 1056 3291 1056 2328 \n", "Q 1056 1369 1301 889 \n", "Q 1547 409 2034 409 \n", "Q 2525 409 2770 889 \n", "Q 3016 1369 3016 2328 \n", "Q 3016 3291 2770 3770 \n", "Q 2525 4250 2034 4250 \n", "z\n", "M 2034 4750 \n", "Q 2819 4750 3233 4129 \n", "Q 3647 3509 3647 2328 \n", "Q 3647 1150 3233 529 \n", "Q 2819 -91 2034 -91 \n", "Q 1250 -91 836 529 \n", "Q 422 1150 422 2328 \n", "Q 422 3509 836 4129 \n", "Q 1250 4750 2034 4750 \n", "z\n", "\" id=\"DejaVuSans-30\" transform=\"scale(0.015625)\"/>\n", "        <path d=\"M 684 794 \n", "L 1344 794 \n", "L 1344 0 \n", "L 684 0 \n", "L 684 794 \n", "z\n", "\" id=\"DejaVuSans-2e\" transform=\"scale(0.015625)\"/>\n", "        <path d=\"M 1228 531 \n", "L 3431 531 \n", "L 3431 0 \n", "L 469 0 \n", "L 469 531 \n", "Q 828 903 1448 1529 \n", "Q 2069 2156 2228 2338 \n", "Q 2531 2678 2651 2914 \n", "Q 2772 3150 2772 3378 \n", "Q 2772 3750 2511 3984 \n", "Q 2250 4219 1831 4219 \n", "Q 1534 4219 1204 4116 \n", "Q 875 4013 500 3803 \n", "L 500 4441 \n", "Q 881 4594 1212 4672 \n", "Q 1544 4750 1819 4750 \n", "Q 2544 4750 2975 4387 \n", "Q 3406 4025 3406 3419 \n", "Q 3406 3131 3298 2873 \n", "Q 3191 2616 2906 2266 \n", "Q 2828 2175 2409 1742 \n", "Q 1991 1309 1228 531 \n", "z\n", "\" id=\"DejaVuSans-32\" transform=\"scale(0.015625)\"/>\n", "        <path d=\"M 691 4666 \n", "L 3169 4666 \n", "L 3169 4134 \n", "L 1269 4134 \n", "L 1269 2991 \n", "Q 1406 3038 1543 3061 \n", "Q 1681 3084 1819 3084 \n", "Q 2600 3084 3056 2656 \n", "Q 3513 2228 3513 1497 \n", "Q 3513 744 3044 326 \n", "Q 2575 -91 1722 -91 \n", "Q 1428 -91 1123 -41 \n", "Q 819 9 494 109 \n", "L 494 744 \n", "Q 775 591 1075 516 \n", "Q 1375 441 1709 441 \n", "Q 2250 441 2565 725 \n", "Q 2881 1009 2881 1497 \n", "Q 2881 1984 2565 2268 \n", "Q 2250 2553 1709 2553 \n", "Q 1456 2553 1204 2497 \n", "Q 953 2441 691 2322 \n", "L 691 4666 \n", "z\n", "\" id=\"DejaVuSans-35\" transform=\"scale(0.015625)\"/>\n", "       </defs>\n", "       <use xlink:href=\"#DejaVuSans-2212\"/>\n", "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-30\"/>\n", "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-32\"/>\n", "       <use x=\"242.822266\" xlink:href=\"#DejaVuSans-35\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"xtick_2\">\n", "     <g id=\"line2d_2\">\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"97.272305\" xlink:href=\"#m522a3f537e\" y=\"239.758125\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_2\">\n", "      <!-- 0.00 -->\n", "      <g transform=\"translate(86.139492 254.356562)scale(0.1 -0.1)\">\n", "       <use xlink:href=\"#DejaVuSans-30\"/>\n", "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-30\"/>\n", "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-30\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"xtick_3\">\n", "     <g id=\"line2d_3\">\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"131.948333\" xlink:href=\"#m522a3f537e\" y=\"239.758125\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_3\">\n", "      <!-- 0.25 -->\n", "      <g transform=\"translate(120.81552 254.356562)scale(0.1 -0.1)\">\n", "       <use xlink:href=\"#DejaVuSans-30\"/>\n", "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-32\"/>\n", "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-35\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"xtick_4\">\n", "     <g id=\"line2d_4\">\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"166.624361\" xlink:href=\"#m522a3f537e\" y=\"239.758125\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_4\">\n", "      <!-- 0.50 -->\n", "      <g transform=\"translate(155.491548 254.356562)scale(0.1 -0.1)\">\n", "       <use xlink:href=\"#DejaVuSans-30\"/>\n", "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-35\"/>\n", "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-30\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"xtick_5\">\n", "     <g id=\"line2d_5\">\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"201.300389\" xlink:href=\"#m522a3f537e\" y=\"239.758125\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_5\">\n", "      <!-- 0.75 -->\n", "      <g transform=\"translate(190.167577 254.356562)scale(0.1 -0.1)\">\n", "       <defs>\n", "        <path d=\"M 525 4666 \n", "L 3525 4666 \n", "L 3525 4397 \n", "L 1831 0 \n", "L 1172 0 \n", "L 2766 4134 \n", "L 525 4134 \n", "L 525 4666 \n", "z\n", "\" id=\"DejaVuSans-37\" transform=\"scale(0.015625)\"/>\n", "       </defs>\n", "       <use xlink:href=\"#DejaVuSans-30\"/>\n", "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-37\"/>\n", "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-35\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"xtick_6\">\n", "     <g id=\"line2d_6\">\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"235.976417\" xlink:href=\"#m522a3f537e\" y=\"239.758125\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_6\">\n", "      <!-- 1.00 -->\n", "      <g transform=\"translate(224.843605 254.356562)scale(0.1 -0.1)\">\n", "       <defs>\n", "        <path d=\"M 794 531 \n", "L 1825 531 \n", "L 1825 4091 \n", "L 703 3866 \n", "L 703 4441 \n", "L 1819 4666 \n", "L 2450 4666 \n", "L 2450 531 \n", "L 3481 531 \n", "L 3481 0 \n", "L 794 0 \n", "L 794 531 \n", "z\n", "\" id=\"DejaVuSans-31\" transform=\"scale(0.015625)\"/>\n", "       </defs>\n", "       <use xlink:href=\"#DejaVuSans-31\"/>\n", "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-30\"/>\n", "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-30\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"xtick_7\">\n", "     <g id=\"line2d_7\">\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"270.652446\" xlink:href=\"#m522a3f537e\" y=\"239.758125\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_7\">\n", "      <!-- 1.25 -->\n", "      <g transform=\"translate(259.519633 254.356562)scale(0.1 -0.1)\">\n", "       <use xlink:href=\"#DejaVuSans-31\"/>\n", "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-32\"/>\n", "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-35\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"text_8\">\n", "     <!-- $x_1$ -->\n", "     <g transform=\"translate(158.410938 268.034687)scale(0.1 -0.1)\">\n", "      <defs>\n", "       <path d=\"M 3841 3500 \n", "L 2234 1784 \n", "L 3219 0 \n", "L 2559 0 \n", "L 1819 1388 \n", "L 531 0 \n", "L -166 0 \n", "L 1556 1844 \n", "L 641 3500 \n", "L 1300 3500 \n", "L 1972 2234 \n", "L 3144 3500 \n", "L 3841 3500 \n", "z\n", "\" id=\"DejaVuSans-Oblique-78\" transform=\"scale(0.015625)\"/>\n", "      </defs>\n", "      <use transform=\"translate(0 0.3125)\" xlink:href=\"#DejaVuSans-Oblique-78\"/>\n", "      <use transform=\"translate(59.179688 -16.09375)scale(0.7)\" xlink:href=\"#DejaVuSans-31\"/>\n", "     </g>\n", "    </g>\n", "   </g>\n", "   <g id=\"matplotlib.axis_2\">\n", "    <g id=\"ytick_1\">\n", "     <g id=\"line2d_8\">\n", "      <defs>\n", "       <path d=\"M 0 0 \n", "L -3.5 0 \n", "\" id=\"me1dcdbe128\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n", "      </defs>\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"52.160938\" xlink:href=\"#me1dcdbe128\" y=\"223.94267\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_9\">\n", "      <!-- \u22120.2 -->\n", "      <g transform=\"translate(20.878125 227.741889)scale(0.1 -0.1)\">\n", "       <use xlink:href=\"#DejaVuSans-2212\"/>\n", "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-30\"/>\n", "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-32\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"ytick_2\">\n", "     <g id=\"line2d_9\">\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"52.160938\" xlink:href=\"#me1dcdbe128\" y=\"196.335617\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_10\">\n", "      <!-- 0.0 -->\n", "      <g transform=\"translate(29.257812 200.134836)scale(0.1 -0.1)\">\n", "       <use xlink:href=\"#DejaVuSans-30\"/>\n", "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-30\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"ytick_3\">\n", "     <g id=\"line2d_10\">\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"52.160938\" xlink:href=\"#me1dcdbe128\" y=\"168.728564\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_11\">\n", "      <!-- 0.2 -->\n", "      <g transform=\"translate(29.257812 172.527783)scale(0.1 -0.1)\">\n", "       <use xlink:href=\"#DejaVuSans-30\"/>\n", "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-32\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"ytick_4\">\n", "     <g id=\"line2d_11\">\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"52.160938\" xlink:href=\"#me1dcdbe128\" y=\"141.12151\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_12\">\n", "      <!-- 0.4 -->\n", "      <g transform=\"translate(29.257812 144.920729)scale(0.1 -0.1)\">\n", "       <defs>\n", "        <path d=\"M 2419 4116 \n", "L 825 1625 \n", "L 2419 1625 \n", "L 2419 4116 \n", "z\n", "M 2253 4666 \n", "L 3047 4666 \n", "L 3047 1625 \n", "L 3713 1625 \n", "L 3713 1100 \n", "L 3047 1100 \n", "L 3047 0 \n", "L 2419 0 \n", "L 2419 1100 \n", "L 313 1100 \n", "L 313 1709 \n", "L 2253 4666 \n", "z\n", "\" id=\"DejaVuSans-34\" transform=\"scale(0.015625)\"/>\n", "       </defs>\n", "       <use xlink:href=\"#DejaVuSans-30\"/>\n", "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-34\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"ytick_5\">\n", "     <g id=\"line2d_12\">\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"52.160938\" xlink:href=\"#me1dcdbe128\" y=\"113.514457\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_13\">\n", "      <!-- 0.6 -->\n", "      <g transform=\"translate(29.257812 117.313676)scale(0.1 -0.1)\">\n", "       <defs>\n", "        <path d=\"M 2113 2584 \n", "Q 1688 2584 1439 2293 \n", "Q 1191 2003 1191 1497 \n", "Q 1191 994 1439 701 \n", "Q 1688 409 2113 409 \n", "Q 2538 409 2786 701 \n", "Q 3034 994 3034 1497 \n", "Q 3034 2003 2786 2293 \n", "Q 2538 2584 2113 2584 \n", "z\n", "M 3366 4563 \n", "L 3366 3988 \n", "Q 3128 4100 2886 4159 \n", "Q 2644 4219 2406 4219 \n", "Q 1781 4219 1451 3797 \n", "Q 1122 3375 1075 2522 \n", "Q 1259 2794 1537 2939 \n", "Q 1816 3084 2150 3084 \n", "Q 2853 3084 3261 2657 \n", "Q 3669 2231 3669 1497 \n", "Q 3669 778 3244 343 \n", "Q 2819 -91 2113 -91 \n", "Q 1303 -91 875 529 \n", "Q 447 1150 447 2328 \n", "Q 447 3434 972 4092 \n", "Q 1497 4750 2381 4750 \n", "Q 2619 4750 2861 4703 \n", "Q 3103 4656 3366 4563 \n", "z\n", "\" id=\"DejaVuSans-36\" transform=\"scale(0.015625)\"/>\n", "       </defs>\n", "       <use xlink:href=\"#DejaVuSans-30\"/>\n", "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-36\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"ytick_6\">\n", "     <g id=\"line2d_13\">\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"52.160938\" xlink:href=\"#me1dcdbe128\" y=\"85.907404\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_14\">\n", "      <!-- 0.8 -->\n", "      <g transform=\"translate(29.257812 89.706623)scale(0.1 -0.1)\">\n", "       <defs>\n", "        <path d=\"M 2034 2216 \n", "Q 1584 2216 1326 1975 \n", "Q 1069 1734 1069 1313 \n", "Q 1069 891 1326 650 \n", "Q 1584 409 2034 409 \n", "Q 2484 409 2743 651 \n", "Q 3003 894 3003 1313 \n", "Q 3003 1734 2745 1975 \n", "Q 2488 2216 2034 2216 \n", "z\n", "M 1403 2484 \n", "Q 997 2584 770 2862 \n", "Q 544 3141 544 3541 \n", "Q 544 4100 942 4425 \n", "Q 1341 4750 2034 4750 \n", "Q 2731 4750 3128 4425 \n", "Q 3525 4100 3525 3541 \n", "Q 3525 3141 3298 2862 \n", "Q 3072 2584 2669 2484 \n", "Q 3125 2378 3379 2068 \n", "Q 3634 1759 3634 1313 \n", "Q 3634 634 3220 271 \n", "Q 2806 -91 2034 -91 \n", "Q 1263 -91 848 271 \n", "Q 434 634 434 1313 \n", "Q 434 1759 690 2068 \n", "Q 947 2378 1403 2484 \n", "z\n", "M 1172 3481 \n", "Q 1172 3119 1398 2916 \n", "Q 1625 2713 2034 2713 \n", "Q 2441 2713 2670 2916 \n", "Q 2900 3119 2900 3481 \n", "Q 2900 3844 2670 4047 \n", "Q 2441 4250 2034 4250 \n", "Q 1625 4250 1398 4047 \n", "Q 1172 3844 1172 3481 \n", "z\n", "\" id=\"DejaVuSans-38\" transform=\"scale(0.015625)\"/>\n", "       </defs>\n", "       <use xlink:href=\"#DejaVuSans-30\"/>\n", "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-38\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"ytick_7\">\n", "     <g id=\"line2d_14\">\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"52.160938\" xlink:href=\"#me1dcdbe128\" y=\"58.300351\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_15\">\n", "      <!-- 1.0 -->\n", "      <g transform=\"translate(29.257812 62.099569)scale(0.1 -0.1)\">\n", "       <use xlink:href=\"#DejaVuSans-31\"/>\n", "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-30\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"ytick_8\">\n", "     <g id=\"line2d_15\">\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"52.160938\" xlink:href=\"#me1dcdbe128\" y=\"30.693297\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_16\">\n", "      <!-- 1.2 -->\n", "      <g transform=\"translate(29.257812 34.492516)scale(0.1 -0.1)\">\n", "       <use xlink:href=\"#DejaVuSans-31\"/>\n", "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-32\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"text_17\">\n", "     <!-- $x_2$ -->\n", "     <g transform=\"translate(14.798438 136.388125)rotate(-90)scale(0.1 -0.1)\">\n", "      <use transform=\"translate(0 0.3125)\" xlink:href=\"#DejaVuSans-Oblique-78\"/>\n", "      <use transform=\"translate(59.179688 -16.09375)scale(0.7)\" xlink:href=\"#DejaVuSans-32\"/>\n", "     </g>\n", "    </g>\n", "   </g>\n", "   <g id=\"patch_3\">\n", "    <path d=\"M 52.160938 239.758125 \n", "L 52.160938 22.318125 \n", "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n", "   </g>\n", "   <g id=\"patch_4\">\n", "    <path d=\"M 275.360937 239.758125 \n", "L 275.360937 22.318125 \n", "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n", "   </g>\n", "   <g id=\"patch_5\">\n", "    <path d=\"M 52.160937 239.758125 \n", "L 275.360937 239.758125 \n", "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n", "   </g>\n", "   <g id=\"patch_6\">\n", "    <path d=\"M 52.160937 22.318125 \n", "L 275.360937 22.318125 \n", "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n", "   </g>\n", "   <g id=\"text_18\">\n", "    <!-- Dataset samples -->\n", "    <g transform=\"translate(113.418125 16.318125)scale(0.12 -0.12)\">\n", "     <defs>\n", "      <path d=\"M 1259 4147 \n", "L 1259 519 \n", "L 2022 519 \n", "Q 2988 519 3436 956 \n", "Q 3884 1394 3884 2338 \n", "Q 3884 3275 3436 3711 \n", "Q 2988 4147 2022 4147 \n", "L 1259 4147 \n", "z\n", "M 628 4666 \n", "L 1925 4666 \n", "Q 3281 4666 3915 4102 \n", "Q 4550 3538 4550 2338 \n", "Q 4550 1131 3912 565 \n", "Q 3275 0 1925 0 \n", "L 628 0 \n", "L 628 4666 \n", "z\n", "\" id=\"DejaVuSans-44\" transform=\"scale(0.015625)\"/>\n", "      <path d=\"M 2194 1759 \n", "Q 1497 1759 1228 1600 \n", "Q 959 1441 959 1056 \n", "Q 959 750 1161 570 \n", "Q 1363 391 1709 391 \n", "Q 2188 391 2477 730 \n", "Q 2766 1069 2766 1631 \n", "L 2766 1759 \n", "L 2194 1759 \n", "z\n", "M 3341 1997 \n", "L 3341 0 \n", "L 2766 0 \n", "L 2766 531 \n", "Q 2569 213 2275 61 \n", "Q 1981 -91 1556 -91 \n", "Q 1019 -91 701 211 \n", "Q 384 513 384 1019 \n", "Q 384 1609 779 1909 \n", "Q 1175 2209 1959 2209 \n", "L 2766 2209 \n", "L 2766 2266 \n", "Q 2766 2663 2505 2880 \n", "Q 2244 3097 1772 3097 \n", "Q 1472 3097 1187 3025 \n", "Q 903 2953 641 2809 \n", "L 641 3341 \n", "Q 956 3463 1253 3523 \n", "Q 1550 3584 1831 3584 \n", "Q 2591 3584 2966 3190 \n", "Q 3341 2797 3341 1997 \n", "z\n", "\" id=\"DejaVuSans-61\" transform=\"scale(0.015625)\"/>\n", "      <path d=\"M 1172 4494 \n", "L 1172 3500 \n", "L 2356 3500 \n", "L 2356 3053 \n", "L 1172 3053 \n", "L 1172 1153 \n", "Q 1172 725 1289 603 \n", "Q 1406 481 1766 481 \n", "L 2356 481 \n", "L 2356 0 \n", "L 1766 0 \n", "Q 1100 0 847 248 \n", "Q 594 497 594 1153 \n", "L 594 3053 \n", "L 172 3053 \n", "L 172 3500 \n", "L 594 3500 \n", "L 594 4494 \n", "L 1172 4494 \n", "z\n", "\" id=\"DejaVuSans-74\" transform=\"scale(0.015625)\"/>\n", "      <path d=\"M 2834 3397 \n", "L 2834 2853 \n", "Q 2591 2978 2328 3040 \n", "Q 2066 3103 1784 3103 \n", "Q 1356 3103 1142 2972 \n", "Q 928 2841 928 2578 \n", "Q 928 2378 1081 2264 \n", "Q 1234 2150 1697 2047 \n", "L 1894 2003 \n", "Q 2506 1872 2764 1633 \n", "Q 3022 1394 3022 966 \n", "Q 3022 478 2636 193 \n", "Q 2250 -91 1575 -91 \n", "Q 1294 -91 989 -36 \n", "Q 684 19 347 128 \n", "L 347 722 \n", "Q 666 556 975 473 \n", "Q 1284 391 1588 391 \n", "Q 1994 391 2212 530 \n", "Q 2431 669 2431 922 \n", "Q 2431 1156 2273 1281 \n", "Q 2116 1406 1581 1522 \n", "L 1381 1569 \n", "Q 847 1681 609 1914 \n", "Q 372 2147 372 2553 \n", "Q 372 3047 722 3315 \n", "Q 1072 3584 1716 3584 \n", "Q 2034 3584 2315 3537 \n", "Q 2597 3491 2834 3397 \n", "z\n", "\" id=\"DejaVuSans-73\" transform=\"scale(0.015625)\"/>\n", "      <path d=\"M 3597 1894 \n", "L 3597 1613 \n", "L 953 1613 \n", "Q 991 1019 1311 708 \n", "Q 1631 397 2203 397 \n", "Q 2534 397 2845 478 \n", "Q 3156 559 3463 722 \n", "L 3463 178 \n", "Q 3153 47 2828 -22 \n", "Q 2503 -91 2169 -91 \n", "Q 1331 -91 842 396 \n", "Q 353 884 353 1716 \n", "Q 353 2575 817 3079 \n", "Q 1281 3584 2069 3584 \n", "Q 2775 3584 3186 3129 \n", "Q 3597 2675 3597 1894 \n", "z\n", "M 3022 2063 \n", "Q 3016 2534 2758 2815 \n", "Q 2500 3097 2075 3097 \n", "Q 1594 3097 1305 2825 \n", "Q 1016 2553 972 2059 \n", "L 3022 2063 \n", "z\n", "\" id=\"DejaVuSans-65\" transform=\"scale(0.015625)\"/>\n", "      <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n", "      <path d=\"M 3328 2828 \n", "Q 3544 3216 3844 3400 \n", "Q 4144 3584 4550 3584 \n", "Q 5097 3584 5394 3201 \n", "Q 5691 2819 5691 2113 \n", "L 5691 0 \n", "L 5113 0 \n", "L 5113 2094 \n", "Q 5113 2597 4934 2840 \n", "Q 4756 3084 4391 3084 \n", "Q 3944 3084 3684 2787 \n", "Q 3425 2491 3425 1978 \n", "L 3425 0 \n", "L 2847 0 \n", "L 2847 2094 \n", "Q 2847 2600 2669 2842 \n", "Q 2491 3084 2119 3084 \n", "Q 1678 3084 1418 2786 \n", "Q 1159 2488 1159 1978 \n", "L 1159 0 \n", "L 581 0 \n", "L 581 3500 \n", "L 1159 3500 \n", "L 1159 2956 \n", "Q 1356 3278 1631 3431 \n", "Q 1906 3584 2284 3584 \n", "Q 2666 3584 2933 3390 \n", "Q 3200 3197 3328 2828 \n", "z\n", "\" id=\"DejaVuSans-6d\" transform=\"scale(0.015625)\"/>\n", "      <path d=\"M 1159 525 \n", "L 1159 -1331 \n", "L 581 -1331 \n", "L 581 3500 \n", "L 1159 3500 \n", "L 1159 2969 \n", "Q 1341 3281 1617 3432 \n", "Q 1894 3584 2278 3584 \n", "Q 2916 3584 3314 3078 \n", "Q 3713 2572 3713 1747 \n", "Q 3713 922 3314 415 \n", "Q 2916 -91 2278 -91 \n", "Q 1894 -91 1617 61 \n", "Q 1341 213 1159 525 \n", "z\n", "M 3116 1747 \n", "Q 3116 2381 2855 2742 \n", "Q 2594 3103 2138 3103 \n", "Q 1681 3103 1420 2742 \n", "Q 1159 2381 1159 1747 \n", "Q 1159 1113 1420 752 \n", "Q 1681 391 2138 391 \n", "Q 2594 391 2855 752 \n", "Q 3116 1113 3116 1747 \n", "z\n", "\" id=\"DejaVuSans-70\" transform=\"scale(0.015625)\"/>\n", "      <path d=\"M 603 4863 \n", "L 1178 4863 \n", "L 1178 0 \n", "L 603 0 \n", "L 603 4863 \n", "z\n", "\" id=\"DejaVuSans-6c\" transform=\"scale(0.015625)\"/>\n", "     </defs>\n", "     <use xlink:href=\"#DejaVuSans-44\"/>\n", "     <use x=\"77.001953\" xlink:href=\"#DejaVuSans-61\"/>\n", "     <use x=\"138.28125\" xlink:href=\"#DejaVuSans-74\"/>\n", "     <use x=\"177.490234\" xlink:href=\"#DejaVuSans-61\"/>\n", "     <use x=\"238.769531\" xlink:href=\"#DejaVuSans-73\"/>\n", "     <use x=\"290.869141\" xlink:href=\"#DejaVuSans-65\"/>\n", "     <use x=\"352.392578\" xlink:href=\"#DejaVuSans-74\"/>\n", "     <use x=\"391.601562\" xlink:href=\"#DejaVuSans-20\"/>\n", "     <use x=\"423.388672\" xlink:href=\"#DejaVuSans-73\"/>\n", "     <use x=\"475.488281\" xlink:href=\"#DejaVuSans-61\"/>\n", "     <use x=\"536.767578\" xlink:href=\"#DejaVuSans-6d\"/>\n", "     <use x=\"634.179688\" xlink:href=\"#DejaVuSans-70\"/>\n", "     <use x=\"697.65625\" xlink:href=\"#DejaVuSans-6c\"/>\n", "     <use x=\"725.439453\" xlink:href=\"#DejaVuSans-65\"/>\n", "     <use x=\"786.962891\" xlink:href=\"#DejaVuSans-73\"/>\n", "    </g>\n", "   </g>\n", "   <g id=\"legend_1\">\n", "    <g id=\"patch_7\">\n", "     <path d=\"M 200.5125 147.21625 \n", "L 268.360937 147.21625 \n", "Q 270.360937 147.21625 270.360937 145.21625 \n", "L 270.360937 116.86 \n", "Q 270.360937 114.86 268.360937 114.86 \n", "L 200.5125 114.86 \n", "Q 198.5125 114.86 198.5125 116.86 \n", "L 198.5125 145.21625 \n", "Q 198.5125 147.21625 200.5125 147.21625 \n", "z\n", "\" style=\"fill:#ffffff;opacity:0.8;stroke:#cccccc;stroke-linejoin:miter;\"/>\n", "    </g>\n", "    <g id=\"PathCollection_3\">\n", "     <g>\n", "      <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"212.5125\" xlink:href=\"#m12efddd26b\" y=\"123.833437\"/>\n", "     </g>\n", "    </g>\n", "    <g id=\"text_19\">\n", "     <!-- Class 0 -->\n", "     <g transform=\"translate(230.5125 126.458437)scale(0.1 -0.1)\">\n", "      <defs>\n", "       <path d=\"M 4122 4306 \n", "L 4122 3641 \n", "Q 3803 3938 3442 4084 \n", "Q 3081 4231 2675 4231 \n", "Q 1875 4231 1450 3742 \n", "Q 1025 3253 1025 2328 \n", "Q 1025 1406 1450 917 \n", "Q 1875 428 2675 428 \n", "Q 3081 428 3442 575 \n", "Q 3803 722 4122 1019 \n", "L 4122 359 \n", "Q 3791 134 3420 21 \n", "Q 3050 -91 2638 -91 \n", "Q 1578 -91 968 557 \n", "Q 359 1206 359 2328 \n", "Q 359 3453 968 4101 \n", "Q 1578 4750 2638 4750 \n", "Q 3056 4750 3426 4639 \n", "Q 3797 4528 4122 4306 \n", "z\n", "\" id=\"DejaVuSans-43\" transform=\"scale(0.015625)\"/>\n", "      </defs>\n", "      <use xlink:href=\"#DejaVuSans-43\"/>\n", "      <use x=\"69.824219\" xlink:href=\"#DejaVuSans-6c\"/>\n", "      <use x=\"97.607422\" xlink:href=\"#DejaVuSans-61\"/>\n", "      <use x=\"158.886719\" xlink:href=\"#DejaVuSans-73\"/>\n", "      <use x=\"210.986328\" xlink:href=\"#DejaVuSans-73\"/>\n", "      <use x=\"263.085938\" xlink:href=\"#DejaVuSans-20\"/>\n", "      <use x=\"294.873047\" xlink:href=\"#DejaVuSans-30\"/>\n", "     </g>\n", "    </g>\n", "    <g id=\"PathCollection_4\">\n", "     <g>\n", "      <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"212.5125\" xlink:href=\"#m897671b02c\" y=\"138.511562\"/>\n", "     </g>\n", "    </g>\n", "    <g id=\"text_20\">\n", "     <!-- Class 1 -->\n", "     <g transform=\"translate(230.5125 141.136562)scale(0.1 -0.1)\">\n", "      <use xlink:href=\"#DejaVuSans-43\"/>\n", "      <use x=\"69.824219\" xlink:href=\"#DejaVuSans-6c\"/>\n", "      <use x=\"97.607422\" xlink:href=\"#DejaVuSans-61\"/>\n", "      <use x=\"158.886719\" xlink:href=\"#DejaVuSans-73\"/>\n", "      <use x=\"210.986328\" xlink:href=\"#DejaVuSans-73\"/>\n", "      <use x=\"263.085938\" xlink:href=\"#DejaVuSans-20\"/>\n", "      <use x=\"294.873047\" xlink:href=\"#DejaVuSans-31\"/>\n", "     </g>\n", "    </g>\n", "   </g>\n", "  </g>\n", " </g>\n", " <defs>\n", "  <clipPath id=\"p7e9b149886\">\n", "   <rect height=\"217.44\" width=\"223.2\" x=\"52.160938\" y=\"22.318125\"/>\n", "  </clipPath>\n", " </defs>\n", "</svg>\n"], "text/plain": ["<Figure size 288x288 with 1 Axes>"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}], "source": ["visualize_samples(dataset.data, dataset.label)\n", "plt.show()"]}, {"cell_type": "markdown", "id": "c4ff362d", "metadata": {"papermill": {"duration": 0.209873, "end_time": "2021-12-04T15:54:04.554262", "exception": false, "start_time": "2021-12-04T15:54:04.344389", "status": "completed"}, "tags": []}, "source": ["#### The data loader class\n", "\n", "The class `torch.utils.data.DataLoader` represents a Python iterable over a dataset with support for automatic batching, multi-process data loading and many more features.\n", "The data loader communicates with the dataset using the function `__getitem__`, and stacks its outputs as tensors over the first dimension to form a batch.\n", "In contrast to the dataset class, we usually don't have to define our own data loader class, but can just create an object of it with the dataset as input.\n", "Additionally, we can configure our data loader with the following input arguments (only a selection, see full list [here](https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader)):\n", "\n", "* `batch_size`: Number of samples to stack per batch\n", "* `shuffle`: If True, the data is returned in a random order.\n", "This is important during training for introducing stochasticity.\n", "* `num_workers`: Number of subprocesses to use for data loading.\n", "The default, 0, means that the data will be loaded in the main process which can slow down training for datasets where loading a data point takes a considerable amount of time (e.g. large images).\n", "More workers are recommended for those, but can cause issues on Windows computers.\n", "For tiny datasets as ours, 0 workers are usually faster.\n", "* `pin_memory`: If True, the data loader will copy Tensors into CUDA pinned memory before returning them.\n", "This can save some time for large data points on GPUs.\n", "Usually a good practice to use for a training set, but not necessarily for validation and test to save memory on the GPU.\n", "* `drop_last`: If True, the last batch is dropped in case it is smaller than the specified batch size.\n", "This occurs when the dataset size is not a multiple of the batch size.\n", "Only potentially helpful during training to keep a consistent batch size.\n", "\n", "Let's create a simple data loader below:"]}, {"cell_type": "code", "execution_count": 43, "id": "93c89ea5", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:54:04.976648Z", "iopub.status.busy": "2021-12-04T15:54:04.976186Z", "iopub.status.idle": "2021-12-04T15:54:04.977746Z", "shell.execute_reply": "2021-12-04T15:54:04.978119Z"}, "papermill": {"duration": 0.215753, "end_time": "2021-12-04T15:54:04.978250", "exception": false, "start_time": "2021-12-04T15:54:04.762497", "status": "completed"}, "tags": []}, "outputs": [], "source": ["data_loader = data.DataLoader(dataset, batch_size=8, shuffle=True)"]}, {"cell_type": "code", "execution_count": 44, "id": "b7a50dd3", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:54:05.404638Z", "iopub.status.busy": "2021-12-04T15:54:05.404165Z", "iopub.status.idle": "2021-12-04T15:54:05.408587Z", "shell.execute_reply": "2021-12-04T15:54:05.408963Z"}, "papermill": {"duration": 0.219528, "end_time": "2021-12-04T15:54:05.409092", "exception": false, "start_time": "2021-12-04T15:54:05.189564", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Data inputs torch.Size([8, 2]) \n", " tensor([[ 1.2108, -0.1180],\n", "        [-0.1895,  0.0415],\n", "        [ 1.1542, -0.0989],\n", "        [ 1.1135,  0.1228],\n", "        [-0.0280,  0.0046],\n", "        [-0.0378,  1.0500],\n", "        [-0.0636,  0.9167],\n", "        [-0.0392,  0.8611]])\n", "Data labels torch.Size([8]) \n", " tensor([1, 0, 1, 1, 0, 1, 1, 1])\n"]}], "source": ["# next(iter(...)) catches the first batch of the data loader\n", "# If shuffle is True, this will return a different batch every time we run this cell\n", "# For iterating over the whole dataset, we can simple use \"for batch in data_loader: ...\"\n", "data_inputs, data_labels = next(iter(data_loader))\n", "\n", "# The shape of the outputs are [batch_size, d_1,...,d_N] where d_1,...,d_N are the\n", "# dimensions of the data point returned from the dataset class\n", "print(\"Data inputs\", data_inputs.shape, \"\\n\", data_inputs)\n", "print(\"Data labels\", data_labels.shape, \"\\n\", data_labels)"]}, {"cell_type": "markdown", "id": "40090168", "metadata": {"papermill": {"duration": 0.213288, "end_time": "2021-12-04T15:54:05.833809", "exception": false, "start_time": "2021-12-04T15:54:05.620521", "status": "completed"}, "tags": []}, "source": ["### Optimization\n", "\n", "After defining the model and the dataset, it is time to prepare the optimization of the model.\n", "During training, we will perform the following steps:\n", "\n", "1. Get a batch from the data loader\n", "2. Obtain the predictions from the model for the batch\n", "3. Calculate the loss based on the difference between predictions and labels\n", "4. Backpropagation: calculate the gradients for every parameter with respect to the loss\n", "5. Update the parameters of the model in the direction of the gradients\n", "\n", "We have seen how we can do step 1, 2 and 4 in PyTorch. Now, we will look at step 3 and 5."]}, {"cell_type": "markdown", "id": "202634de", "metadata": {"papermill": {"duration": 0.216115, "end_time": "2021-12-04T15:54:06.271200", "exception": false, "start_time": "2021-12-04T15:54:06.055085", "status": "completed"}, "tags": []}, "source": ["#### Loss modules\n", "\n", "We can calculate the loss for a batch by simply performing a few tensor operations as those are automatically added to the computation graph.\n", "For instance, for binary classification, we can use Binary Cross Entropy (BCE) which is defined as follows:\n", "\n", "$$\\mathcal{L}_{BCE} = -\\sum_i \\left[ y_i \\log x_i + (1 - y_i) \\log (1 - x_i) \\right]$$\n", "\n", "where $y$ are our labels, and $x$ our predictions, both in the range of $[0,1]$.\n", "However, PyTorch already provides a list of predefined loss functions which we can use (see [here](https://pytorch.org/docs/stable/nn.html#loss-functions) for a full list).\n", "For instance, for BCE, PyTorch has two modules: `nn.BCELoss()`, `nn.BCEWithLogitsLoss()`.\n", "While `nn.BCELoss` expects the inputs $x$ to be in the range $[0,1]$, i.e. the output of a sigmoid, `nn.BCEWithLogitsLoss` combines a sigmoid layer and the BCE loss in a single class.\n", "This version is numerically more stable than using a plain Sigmoid followed by a BCE loss because of the logarithms applied in the loss function.\n", "Hence, it is adviced to use loss functions applied on \"logits\" where possible (remember to not apply a sigmoid on the output of the model in this case!).\n", "For our model defined above, we therefore use the module `nn.BCEWithLogitsLoss`."]}, {"cell_type": "code", "execution_count": 45, "id": "5427b133", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:54:06.708663Z", "iopub.status.busy": "2021-12-04T15:54:06.708193Z", "iopub.status.idle": "2021-12-04T15:54:06.710106Z", "shell.execute_reply": "2021-12-04T15:54:06.709723Z"}, "papermill": {"duration": 0.221156, "end_time": "2021-12-04T15:54:06.710216", "exception": false, "start_time": "2021-12-04T15:54:06.489060", "status": "completed"}, "tags": []}, "outputs": [], "source": ["loss_module = nn.BCEWithLogitsLoss()"]}, {"cell_type": "markdown", "id": "af76ed16", "metadata": {"papermill": {"duration": 0.21844, "end_time": "2021-12-04T15:54:07.146700", "exception": false, "start_time": "2021-12-04T15:54:06.928260", "status": "completed"}, "tags": []}, "source": ["#### Stochastic Gradient Descent\n", "\n", "For updating the parameters, PyTorch provides the package `torch.optim` that has most popular optimizers implemented.\n", "We will discuss the specific optimizers and their differences later in the course, but will for now use the simplest of them: `torch.optim.SGD`.\n", "Stochastic Gradient Descent updates parameters by multiplying the gradients with a small constant, called learning rate, and subtracting those from the parameters (hence minimizing the loss).\n", "Therefore, we slowly move towards the direction of minimizing the loss.\n", "A good default value of the learning rate for a small network as ours is 0.1."]}, {"cell_type": "code", "execution_count": 46, "id": "60eae78b", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:54:07.623913Z", "iopub.status.busy": "2021-12-04T15:54:07.623439Z", "iopub.status.idle": "2021-12-04T15:54:07.625388Z", "shell.execute_reply": "2021-12-04T15:54:07.625006Z"}, "papermill": {"duration": 0.22472, "end_time": "2021-12-04T15:54:07.625501", "exception": false, "start_time": "2021-12-04T15:54:07.400781", "status": "completed"}, "tags": []}, "outputs": [], "source": ["# Input to the optimizer are the parameters of the model: model.parameters()\n", "optimizer = torch.optim.SGD(model.parameters(), lr=0.1)"]}, {"cell_type": "markdown", "id": "cab67e22", "metadata": {"papermill": {"duration": 0.214969, "end_time": "2021-12-04T15:54:08.054953", "exception": false, "start_time": "2021-12-04T15:54:07.839984", "status": "completed"}, "tags": []}, "source": ["The optimizer provides two useful functions: `optimizer.step()`, and `optimizer.zero_grad()`.\n", "The step function updates the parameters based on the gradients as explained above.\n", "The function `optimizer.zero_grad()` sets the gradients of all parameters to zero.\n", "While this function seems less relevant at first, it is a crucial pre-step before performing backpropagation.\n", "If we would call the `backward` function on the loss while the parameter gradients are non-zero from the previous batch, the new gradients would actually be added to the previous ones instead of overwriting them.\n", "This is done because a parameter might occur multiple times in a computation graph, and we need to sum the gradients in this case instead of replacing them.\n", "Hence, remember to call `optimizer.zero_grad()` before calculating the gradients of a batch."]}, {"cell_type": "markdown", "id": "815da7f3", "metadata": {"papermill": {"duration": 0.214692, "end_time": "2021-12-04T15:54:08.483699", "exception": false, "start_time": "2021-12-04T15:54:08.269007", "status": "completed"}, "tags": []}, "source": ["### Training\n", "\n", "Finally, we are ready to train our model.\n", "As a first step, we create a slightly larger dataset and specify a data loader with a larger batch size."]}, {"cell_type": "code", "execution_count": 47, "id": "be9ac2d0", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:54:08.929206Z", "iopub.status.busy": "2021-12-04T15:54:08.928716Z", "iopub.status.idle": "2021-12-04T15:54:08.931250Z", "shell.execute_reply": "2021-12-04T15:54:08.930773Z"}, "papermill": {"duration": 0.229194, "end_time": "2021-12-04T15:54:08.931361", "exception": false, "start_time": "2021-12-04T15:54:08.702167", "status": "completed"}, "tags": []}, "outputs": [], "source": ["train_dataset = XORDataset(size=1000)\n", "train_data_loader = data.DataLoader(train_dataset, batch_size=128, shuffle=True)"]}, {"cell_type": "markdown", "id": "0271a836", "metadata": {"papermill": {"duration": 0.214622, "end_time": "2021-12-04T15:54:09.363363", "exception": false, "start_time": "2021-12-04T15:54:09.148741", "status": "completed"}, "tags": []}, "source": ["Now, we can write a small training function.\n", "Remember our five steps: load a batch, obtain the predictions, calculate the loss, backpropagate, and update.\n", "Additionally, we have to push all data and model parameters to the device of our choice (GPU if available).\n", "For the tiny neural network we have, communicating the data to the GPU actually takes much more time than we could save from running the operation on GPU.\n", "For large networks, the communication time is significantly smaller than the actual runtime making a GPU crucial in these cases.\n", "Still, to practice, we will push the data to GPU here."]}, {"cell_type": "code", "execution_count": 48, "id": "c7b13460", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:54:09.806935Z", "iopub.status.busy": "2021-12-04T15:54:09.806433Z", "iopub.status.idle": "2021-12-04T15:54:09.810018Z", "shell.execute_reply": "2021-12-04T15:54:09.809609Z"}, "papermill": {"duration": 0.222561, "end_time": "2021-12-04T15:54:09.810128", "exception": false, "start_time": "2021-12-04T15:54:09.587567", "status": "completed"}, "tags": []}, "outputs": [{"data": {"text/plain": ["SimpleClassifier(\n", "  (linear1): Linear(in_features=2, out_features=4, bias=True)\n", "  (act_fn): Tanh()\n", "  (linear2): Linear(in_features=4, out_features=1, bias=True)\n", ")"]}, "execution_count": 48, "metadata": {}, "output_type": "execute_result"}], "source": ["# Push model to device. Has to be only done once\n", "model.to(device)"]}, {"cell_type": "markdown", "id": "cadf5955", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.214526, "end_time": "2021-12-04T15:54:10.247009", "exception": false, "start_time": "2021-12-04T15:54:10.032483", "status": "completed"}, "tags": []}, "source": ["In addition, we set our model to training mode.\n", "This is done by calling `model.train()`.\n", "There exist certain modules that need to perform a different forward\n", "step during training than during testing (e.g. BatchNorm and Dropout),\n", "and we can switch between them using `model.train()` and `model.eval()`."]}, {"cell_type": "code", "execution_count": 49, "id": "b2e9a50f", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:54:10.679237Z", "iopub.status.busy": "2021-12-04T15:54:10.678761Z", "iopub.status.idle": "2021-12-04T15:54:10.680363Z", "shell.execute_reply": "2021-12-04T15:54:10.680739Z"}, "papermill": {"duration": 0.219695, "end_time": "2021-12-04T15:54:10.680870", "exception": false, "start_time": "2021-12-04T15:54:10.461175", "status": "completed"}, "tags": []}, "outputs": [], "source": ["def train_model(model, optimizer, data_loader, loss_module, num_epochs=100):\n", "    # Set model to train mode\n", "    model.train()\n", "\n", "    # Training loop\n", "    for epoch in tqdm(range(num_epochs)):\n", "        for data_inputs, data_labels in data_loader:\n", "\n", "            # Step 1: Move input data to device (only strictly necessary if we use GPU)\n", "            data_inputs = data_inputs.to(device)\n", "            data_labels = data_labels.to(device)\n", "\n", "            # Step 2: Run the model on the input data\n", "            preds = model(data_inputs)\n", "            preds = preds.squeeze(dim=1)  # Output is [Batch size, 1], but we want [Batch size]\n", "\n", "            # Step 3: Calculate the loss\n", "            loss = loss_module(preds, data_labels.float())\n", "\n", "            # Step 4: Perform backpropagation\n", "            # Before calculating the gradients, we need to ensure that they are all zero.\n", "            # The gradients would not be overwritten, but actually added to the existing ones.\n", "            optimizer.zero_grad()\n", "            # Perform backpropagation\n", "            loss.backward()\n", "\n", "            # Step 5: Update the parameters\n", "            optimizer.step()"]}, {"cell_type": "code", "execution_count": 50, "id": "6857c620", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:54:11.110005Z", "iopub.status.busy": "2021-12-04T15:54:11.109541Z", "iopub.status.idle": "2021-12-04T15:54:12.322199Z", "shell.execute_reply": "2021-12-04T15:54:12.322585Z"}, "papermill": {"duration": 1.429741, "end_time": "2021-12-04T15:54:12.322764", "exception": false, "start_time": "2021-12-04T15:54:10.893023", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "8222db0c1ce847c18b82712853539f5e", "version_major": 2, "version_minor": 0}, "text/plain": ["  0%|          | 0/100 [00:00<?, ?it/s]"]}, "metadata": {}, "output_type": "display_data"}], "source": ["train_model(model, optimizer, train_data_loader, loss_module)"]}, {"cell_type": "markdown", "id": "27b0e8fd", "metadata": {"papermill": {"duration": 0.215255, "end_time": "2021-12-04T15:54:12.752884", "exception": false, "start_time": "2021-12-04T15:54:12.537629", "status": "completed"}, "tags": []}, "source": ["#### Saving a model\n", "\n", "After finish training a model, we save the model to disk so that we can load the same weights at a later time.\n", "For this, we extract the so-called `state_dict` from the model which contains all learnable parameters.\n", "For our simple model, the state dict contains the following entries:"]}, {"cell_type": "code", "execution_count": 51, "id": "0caf1849", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:54:13.185095Z", "iopub.status.busy": "2021-12-04T15:54:13.184626Z", "iopub.status.idle": "2021-12-04T15:54:13.190584Z", "shell.execute_reply": "2021-12-04T15:54:13.190117Z"}, "papermill": {"duration": 0.223351, "end_time": "2021-12-04T15:54:13.190711", "exception": false, "start_time": "2021-12-04T15:54:12.967360", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["OrderedDict([('linear1.weight', tensor([[-2.0670, -2.2975],\n", "        [ 1.2786, -1.8345],\n", "        [-1.4951, -0.5205],\n", "        [-0.6117, -0.7444]], device='cuda:0')), ('linear1.bias', tensor([ 0.7173, -0.8690,  1.3082, -0.2268], device='cuda:0')), ('linear2.weight', tensor([[-2.6062,  1.9208,  2.1200, -0.2700]], device='cuda:0')), ('linear2.bias', tensor([-0.9445], device='cuda:0'))])\n"]}], "source": ["state_dict = model.state_dict()\n", "print(state_dict)"]}, {"cell_type": "markdown", "id": "f31a8545", "metadata": {"papermill": {"duration": 0.215207, "end_time": "2021-12-04T15:54:13.620489", "exception": false, "start_time": "2021-12-04T15:54:13.405282", "status": "completed"}, "tags": []}, "source": ["To save the state dictionary, we can use `torch.save`:"]}, {"cell_type": "code", "execution_count": 52, "id": "a599c505", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:54:14.053263Z", "iopub.status.busy": "2021-12-04T15:54:14.052796Z", "iopub.status.idle": "2021-12-04T15:54:14.055294Z", "shell.execute_reply": "2021-12-04T15:54:14.054843Z"}, "papermill": {"duration": 0.222184, "end_time": "2021-12-04T15:54:14.055407", "exception": false, "start_time": "2021-12-04T15:54:13.833223", "status": "completed"}, "tags": []}, "outputs": [], "source": ["# torch.save(object, filename). For the filename, any extension can be used\n", "torch.save(state_dict, \"our_model.tar\")"]}, {"cell_type": "markdown", "id": "8b49cd0d", "metadata": {"papermill": {"duration": 0.212443, "end_time": "2021-12-04T15:54:14.482021", "exception": false, "start_time": "2021-12-04T15:54:14.269578", "status": "completed"}, "tags": []}, "source": ["To load a model from a state dict, we use the function `torch.load` to\n", "load the state dict from the disk, and the module function\n", "`load_state_dict` to overwrite our parameters with the new values:"]}, {"cell_type": "code", "execution_count": 53, "id": "c46852a3", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:54:14.913680Z", "iopub.status.busy": "2021-12-04T15:54:14.913209Z", "iopub.status.idle": "2021-12-04T15:54:14.921781Z", "shell.execute_reply": "2021-12-04T15:54:14.921382Z"}, "papermill": {"duration": 0.226809, "end_time": "2021-12-04T15:54:14.921890", "exception": false, "start_time": "2021-12-04T15:54:14.695081", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Original model\n", " OrderedDict([('linear1.weight', tensor([[-2.0670, -2.2975],\n", "        [ 1.2786, -1.8345],\n", "        [-1.4951, -0.5205],\n", "        [-0.6117, -0.7444]], device='cuda:0')), ('linear1.bias', tensor([ 0.7173, -0.8690,  1.3082, -0.2268], device='cuda:0')), ('linear2.weight', tensor([[-2.6062,  1.9208,  2.1200, -0.2700]], device='cuda:0')), ('linear2.bias', tensor([-0.9445], device='cuda:0'))])\n", "\n", "Loaded model\n", " OrderedDict([('linear1.weight', tensor([[-2.0670, -2.2975],\n", "        [ 1.2786, -1.8345],\n", "        [-1.4951, -0.5205],\n", "        [-0.6117, -0.7444]])), ('linear1.bias', tensor([ 0.7173, -0.8690,  1.3082, -0.2268])), ('linear2.weight', tensor([[-2.6062,  1.9208,  2.1200, -0.2700]])), ('linear2.bias', tensor([-0.9445]))])\n"]}], "source": ["# Load state dict from the disk (make sure it is the same name as above)\n", "state_dict = torch.load(\"our_model.tar\")\n", "\n", "# Create a new model and load the state\n", "new_model = SimpleClassifier(num_inputs=2, num_hidden=4, num_outputs=1)\n", "new_model.load_state_dict(state_dict)\n", "\n", "# Verify that the parameters are the same\n", "print(\"Original model\\n\", model.state_dict())\n", "print(\"\\nLoaded model\\n\", new_model.state_dict())"]}, {"cell_type": "markdown", "id": "c3b78bcc", "metadata": {"papermill": {"duration": 0.214236, "end_time": "2021-12-04T15:54:15.352385", "exception": false, "start_time": "2021-12-04T15:54:15.138149", "status": "completed"}, "tags": []}, "source": ["A detailed tutorial on saving and loading models in PyTorch can be found\n", "[here](https://pytorch.org/tutorials/beginner/saving_loading_models.html)."]}, {"cell_type": "markdown", "id": "2f92de3a", "metadata": {"papermill": {"duration": 0.214006, "end_time": "2021-12-04T15:54:15.782014", "exception": false, "start_time": "2021-12-04T15:54:15.568008", "status": "completed"}, "tags": []}, "source": ["### Evaluation\n", "\n", "Once we have trained a model, it is time to evaluate it on a held-out test set.\n", "As our dataset consist of randomly generated data points, we need to\n", "first create a test set with a corresponding data loader."]}, {"cell_type": "code", "execution_count": 54, "id": "fa7f5a42", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:54:16.218034Z", "iopub.status.busy": "2021-12-04T15:54:16.217505Z", "iopub.status.idle": "2021-12-04T15:54:16.220128Z", "shell.execute_reply": "2021-12-04T15:54:16.219730Z"}, "papermill": {"duration": 0.22249, "end_time": "2021-12-04T15:54:16.220237", "exception": false, "start_time": "2021-12-04T15:54:15.997747", "status": "completed"}, "tags": []}, "outputs": [], "source": ["test_dataset = XORDataset(size=500)\n", "# drop_last -> Don't drop the last batch although it is smaller than 128\n", "test_data_loader = data.DataLoader(test_dataset, batch_size=128, shuffle=False, drop_last=False)"]}, {"cell_type": "markdown", "id": "9b0c0259", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.223015, "end_time": "2021-12-04T15:54:16.659749", "exception": false, "start_time": "2021-12-04T15:54:16.436734", "status": "completed"}, "tags": []}, "source": ["As metric, we will use accuracy which is calculated as follows:\n", "\n", "$$acc = \\frac{\\#\\text{correct predictions}}{\\#\\text{all predictions}} = \\frac{TP+TN}{TP+TN+FP+FN}$$\n", "\n", "where TP are the true positives, TN true negatives, FP false positives, and FN the fale negatives.\n", "\n", "When evaluating the model, we don't need to keep track of the computation graph as we don't intend to calculate the gradients.\n", "This reduces the required memory and speed up the model.\n", "In PyTorch, we can deactivate the computation graph using `with torch.no_grad(): ...`.\n", "Remember to additionally set the model to eval mode."]}, {"cell_type": "code", "execution_count": 55, "id": "81d5e627", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:54:17.100737Z", "iopub.status.busy": "2021-12-04T15:54:17.100266Z", "iopub.status.idle": "2021-12-04T15:54:17.102233Z", "shell.execute_reply": "2021-12-04T15:54:17.101833Z"}, "papermill": {"duration": 0.228058, "end_time": "2021-12-04T15:54:17.102341", "exception": false, "start_time": "2021-12-04T15:54:16.874283", "status": "completed"}, "tags": []}, "outputs": [], "source": ["def eval_model(model, data_loader):\n", "    model.eval()  # Set model to eval mode\n", "    true_preds, num_preds = 0.0, 0.0\n", "\n", "    with torch.no_grad():  # Deactivate gradients for the following code\n", "        for data_inputs, data_labels in data_loader:\n", "\n", "            # Determine prediction of model on dev set\n", "            data_inputs, data_labels = data_inputs.to(device), data_labels.to(device)\n", "            preds = model(data_inputs)\n", "            preds = preds.squeeze(dim=1)\n", "            preds = torch.sigmoid(preds)  # Sigmoid to map predictions between 0 and 1\n", "            pred_labels = (preds >= 0.5).long()  # Binarize predictions to 0 and 1\n", "\n", "            # Keep records of predictions for the accuracy metric (true_preds=TP+TN, num_preds=TP+TN+FP+FN)\n", "            true_preds += (pred_labels == data_labels).sum()\n", "            num_preds += data_labels.shape[0]\n", "\n", "    acc = true_preds / num_preds\n", "    print(f\"Accuracy of the model: {100.0*acc:4.2f}%\")"]}, {"cell_type": "code", "execution_count": 56, "id": "71db7937", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:54:17.540450Z", "iopub.status.busy": "2021-12-04T15:54:17.539985Z", "iopub.status.idle": "2021-12-04T15:54:17.546678Z", "shell.execute_reply": "2021-12-04T15:54:17.547057Z"}, "papermill": {"duration": 0.228671, "end_time": "2021-12-04T15:54:17.547187", "exception": false, "start_time": "2021-12-04T15:54:17.318516", "status": "completed"}, "tags": []}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Accuracy of the model: 100.00%\n"]}], "source": ["eval_model(model, test_data_loader)"]}, {"cell_type": "markdown", "id": "0b763086", "metadata": {"papermill": {"duration": 0.216167, "end_time": "2021-12-04T15:54:17.978220", "exception": false, "start_time": "2021-12-04T15:54:17.762053", "status": "completed"}, "tags": []}, "source": ["If we trained our model correctly, we should see a score close to 100% accuracy.\n", "However, this is only possible because of our simple task, and\n", "unfortunately, we usually don't get such high scores on test sets of\n", "more complex tasks."]}, {"cell_type": "markdown", "id": "5fc4b4e1", "metadata": {"lines_to_next_cell": 2, "papermill": {"duration": 0.215766, "end_time": "2021-12-04T15:54:18.415334", "exception": false, "start_time": "2021-12-04T15:54:18.199568", "status": "completed"}, "tags": []}, "source": ["#### Visualizing classification boundaries\n", "\n", "To visualize what our model has learned, we can perform a prediction for every data point in a range of $[-0.5, 1.5]$, and visualize the predicted class as in the sample figure at the beginning of this section.\n", "This shows where the model has created decision boundaries, and which points would be classified as $0$, and which as $1$.\n", "We therefore get a background image out of blue (class 0) and orange (class 1).\n", "The spots where the model is uncertain we will see a blurry overlap.\n", "The specific code is less relevant compared to the output figure which\n", "should hopefully show us a clear separation of classes:"]}, {"cell_type": "code", "execution_count": 57, "id": "9b2cdd8a", "metadata": {"execution": {"iopub.execute_input": "2021-12-04T15:54:18.862084Z", "iopub.status.busy": "2021-12-04T15:54:18.861126Z", "iopub.status.idle": "2021-12-04T15:54:19.252452Z", "shell.execute_reply": "2021-12-04T15:54:19.252841Z"}, "papermill": {"duration": 0.618628, "end_time": "2021-12-04T15:54:19.253007", "exception": false, "start_time": "2021-12-04T15:54:18.634379", "status": "completed"}, "tags": []}, "outputs": [{"data": {"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUgovR3JvdXAgPDwgL0NTIC9EZXZpY2VSR0IgL1MgL1RyYW5zcGFyZW5jeSAvVHlwZSAvR3JvdXAgPj4KL01lZGlhQm94IFsgMCAwIDI5MS4xMDU2MjUgMjc3LjMwODc1IF0gL1BhcmVudCAyIDAgUiAvUmVzb3VyY2VzIDggMCBSCi9UeXBlIC9QYWdlID4+CmVuZG9iago5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTIgMCBSID4+CnN0cmVhbQp4nK2bS5NcuXGF9/Ur7lJa9CWAxHM54/FMWBFejMWwFw4vGDRFa4IzNkXJ8s/3d3Cr6t6qzuqmFDNPFhoFJPJx8mQCHZefTm++icvHL0tYfuLfvy5x+WF5892H//3j+w//8sO3y/svp8D4z6c04hpDqanw8dPxY2pttdBbYTjcfPqv0+mXE6vzjR9Y+OPpVPpaIt+xtpZsTGLhVtZRjmOf9rFU8jr6Nnj57nGMHf5w+rw8WzbFtuZ8+d+fPiz/tvyyvPkm6aSRk0ZOGp6d9DNf6IvOq/8/W/T9z8ubf4rLd/+9/Hj6cfkbdw1rTLHUXjkbH3I9/9VOYW2h9J6spLH86eMu35qQ8OH3lrvvnT6jaskelxjTGuMo1mKMtoy8ppZbLNZLnaf458Ap9umhrjlEMytlDD7aain0UmIOxZmfZPoUYkWstDzFNfUeRop1dGf2U0Qd+AlTRm+aH5G8ptGD5eh9AdvnZpZGTkznzKMPZBwhebOTra1bLCHk2hedfGT+hIY8aaLlteWYK7MQXmeNLVtpobnzn2TjkXoKJY+lrDUmk2pGdo8aOts3FDEwEZ9TWMswq7mG6J1VlioJYXpro/KxryPmgvsRR/4OZQ1pIEwybRDLWpAfF0nV3A3GGou0hxw6fsLUNfQ+SvSthTpRD8tb2bQ7LBsHYRNvfkYlRAH2DLbY2gOR2WNpnqNN/YTBSojDdPQTCeXQQ461uRpl+5xzz6Wjfma3NKxnjOc6cmprCHhDjmG0peH30WK23KM3PeE4aCZ1qR49Vr5VcnCXfkoYKhhOklJeUKrhy00OPXyfT+iN+JCvFH2WY2S8r1n1DDv9so0UUjMME22soeHDGK26eq84SgkjWo+SLsqNcmuphgeax/C1d0t4Z16egO/UYsoNpPbk6QNMi+hSyIYXGV7HdgHfc8WJIa4htlCLhawgxxTEb7FcLLoHDmE1NAp64f0zEDqGxmAEm69SoZ+saehJOwa8rdrAgMV1/dTXnCooVQo6lwZGaQKiaq6OgL2BoxlAk7X+inRJHtLdSMkr1u8SAFeOmdkZdQYi1xe/MB9nb7luh+lsg7WKtzjaN/7uNUbNHkhOrh1CxQdBm20kQBzdGb4g02bw27PsCqyDdliq4zjk0ppIFEaMu/ix8pNW+Sco/5AiTOH9AP8Erj3VRvj1BZWAw2So5PokCsGWg3jIqc8zhGAoZJgLBQ0g48d4hyIoGlnCiD4gpHurW8D/UFjAY8pUIQkBmCndPScWIaDAXgLWBgfFbwAxDv0gp9lKfiRFESFDn1F6ncBfgpt3iA8jMwSFXF4IP1x5KCW67gJcAAN4L3AE4YFOjFgrJn3gWxXdEEKhhZkVACsjfG1EXM51L/wpGNk+VTky9icBlVa8dF/XNmT1SnolfZD3hdjmhVyvmDSQIwHWpRPRSV4GD3HFxiRlTBVWcSdUDqEZcEdX5QPMDThUStCNDWBkTcW/SwwA83UEBBfEidXg+ECFVWGID2EmgCGRARnSYsgyGRtyAteBlbyxYIcHC7Lj2msjaGEW/gbyeIK0JGC0bWbijwVUim4ylkSdKG4CAls6TsHqtYTiRsiQbZReKywuCe3IZ/hE9VOU5IW0sUHNW8bC35MSYvWpSgZkSNWE1BmwyZZtNJKCm11NfqPYrzNCMvmzkpnBMV87wDXHDJVNUtrMYSBDyqUObwPwIBdATv4FkFUAGOmT9e55sXhkakkph2yBcNVCwSLNHiEZNL7As2Bj+jKCwfqqHyIIotPFHiGHizgamKbs3Lu7OsKShsIgQG3mcuPYAjYXDCCRgLPyKNly4bsAQzDs6qcxKAGa54eE3iRZQmWZFS5YH9GmJDeGRGKbTiY3mA0R7/JzUTjSeg7CYcg61iFUkvT0gHgUvHD0OlreiAJ2Igj5vis/1VQvkBNAib16biBKSi6DU5JucDJCluy7KA1OyOwAvzNdqA5pgtwSoNP7WbY9KIrwb3I7HllR97RpxgCdKukBmIHkgGhVHn2aRNTyxLdHWFMCcc3ugFknOCoEThr3IztXNIxFTXjcKQCI0uwGEsQOlCgkShI25sHnh0HbXBSYtDAqrOGZ8gSjmMtdmDFcwdE3VIGj4SJsRaUFPI6QH+QlmQcKlvFBOSbSiPLNdPOgWJmlDf7d2uTEaWK8dPmA9+AqkDBQ2qbWTUGFp/lZVYeFYSABeQ+GUGrIFEbxAWcm8SYj8EhJEyL5zAciBZR18ZpIo97Iqhexuz7GBgJiaZejPKnyhrdZH/qvdhgTGMCO6AINgVpVH5KXEo4g7UPmqG/c1VG3GYZUBsPUyMZpWNglkYO4hpGAp2Fs6ZLcFFV7mScJyZvKAy4jMg7AJNToR9KT6n80TY3ObKouviLiCya4gUSUYnIq8pK3nMmXI0X0cGvKzWOpmiAZM6eS5eFDQWzGFQYNJ6KHkrLIZbAwCSwopNz4KCtBERs1dl0G7hblLkHlpSc7Ya8Mp6SksIaGdfEfKlD3rJwN/jMpTEGJUib1lmtMQoGwAN7Qg7oShHUknRYXF4FQ8gmhJoWTXihx+Rrmd0s9iBHn7BgTx1ClSuEHcYIEuTycQjZkAoDUr4iiDCXGqfp8dUNNM7urhG8iVYY0uOyganJZoWgABQAJRaxQ+ldhC9+uN9P/jg6c8D+r6zQSIp8o74AauEwlZT9ru91OXu4mH3pt2goGh4FhOmQVSCPxDjGycklv8dajREOrGCA+AZ6ShFAGaWM4s0krqbKvUsVE0FV8i0hK5i2+1SFRwUYCJSFCHQZmB47MlwY2QR0CLsBKJ8SZaBTBcCnz79dvcAsSZz9TalXAUB1Kd2f6EMtqQh8o3NkXiLQAYbgY83b5Cv5TBKhb2WY2ELgD2fXS27pdPgop+KF0KdmBW7JeLtATT5wn8BsCQLCZ+ncqfeNQydGbqxsYQoawqlNWwDtmdZvfdjWD31YVCih7bPVepArH0OpiPv+CjY3tVrUcZl7lE3YW0feWL5pA8ZgnzCG79aZWVzLvqLgVgNUxLSRKZGQQo41c6lu1StEAEelc+IxhrQj2XI/BwTENzLZPl0yqq1rhlNfq9G51Ux1I1oLtLNqK5Ctykr2DEkxAFQyEWFNruYqzI0xxDTrgllaDCl8sDvpawqCgqHmSQGwh6CAza7etvsN5xPKTtzpsBapTKAlLmiQd/cOh7dogvPNdkewCNxiEppoJZCBVJG5Y49oISeFSTda3lZhVs4/k4UxH5aUBNBAjyGUSZpCZCY3gmxPeRLhDPgceuXQKMViREmL35qsnzM+pAiiulbWopNKEuuTqZevEwXIqiEjgoXYqbEpHMVIvTMuqpjCGVKGoHGlqtlO6XUjU80giQ1MMyd/1dWUynKsNz6qCOAhWCmryUjwkMjAEqvlnDWoihtjhgLOJSAJLRNS5OnHNihwlZ5lSe0FIZvfDDeqnDoDhkJQjBNQyJkOjWpc6XYRJK2yrqNZpY6MP8BjKgJb9UIpq7DTM2QecD6rU1aLE533USKqHCL6x0WMClXiiinHtNCNZNxdUrbFszfqW1eKODxxtci9V1L1wPqUOii9qjetVwb30ZVV3GMKDNqffVFHISMh4hjVlF6p0EDJPLKBqIBuk5MYIdShe07K6GYIcopXpwc2TQ9cs4BWVy0QCSgHEaPr0QPFKFpQtitcqZk9lUNWNRBxP9tkwDwDC6EV3XmxIPBI01roPCCqLgQ9oDtWJOFmgdGAHX/Vk7SEe0OeFnRhbJG3W3FzF4yZoBugTQuMGDUoj8u6j5FjlsVL9bOSDkjnNzpk9wHeKmMkf8yxEum6kyB2qYNy0WhVDjSAscRo5qhWgms2bDXhleW/oaqrN6y9wLIP62UUPoYFSXe9NPjwbobgBHOEBodHdJ5rUF3QBR64kP3Xd9DTPzbJuNQYZJs78BLQV3fO0MbxkiS6QBu474izvQJ/EVPgWiPUAbah2BNYhz4CFK5Ofhu80Xbc+Im4F353XuFXJM1DSuCnK1GYAOTpFSZ/VR8YtIDTRgz6ZhvIP8mbqJIi3z6tGVxZFnKl73+YtrHjcRO3hZnmVhKKBIGtXU0hAAyOA12ZvcRIxBUmGORLbukaJqn2Kj6kAgbodGF0OA8vSxYgNi567TEiC1bV8bvniXsoNZEM/9UHbiJtEWGcgd95yUe2RPnrx3EXztXYGh7dyltRM3ufUzYUxdbRmv8Zm+MV545LUsQqeg83bbwiKwZeUQSzKIfUkwMcNQTQWSmqyzGKZY5M7os8inwaMug4IU+J0MRMqKQM1ez/8FlVVaII0VAOz40v+MPIqdVzw/VE8wjCBZQvyX4KlFNy3+O7O+jWTHyk50ljgfpQeUKh0LT1vVS8+HoE5MsH5Oo+4yOqburx2FjNgaJMsAmw0mkX3gy97igCHdV2HzFoIMkktASe/XqTdKgdKRhmmFm6Rw+t7TXN9TIUBifEkoeKTOqgiFLW7tcckNRm3JRtVwUKXZIk8OZrn9JgJgFGqx4zqKOEscA7z54In4J2eVEwqzwmr+mE+fnV1jXQvNgS/Qy16OJxyvOdfFUKOc5HT0fWmUl1Z8PdwqV5WNFDyZN1V6f2LmneUFr67kNSDzMOMpraTke1Z2ad5Mn+u21uDoIcY6viD1fMK37dQw95qaUHgOTWGV0Pf5e9bjYSYeG88PwuhMI6Qk0c5FWCBkDTdcPbZSITPqla53kXfU6s1YKIgMKp6hqGefJtPOBwQ2LqUXU3+2ZBQv4Uql0ByIUY9hCAS3Df6QLimXN0qSJdKLJwGoDWvXeFNrcw7PutuJjA1B/VwoOpaTx1TAIOoa67oT72tun2WJRV1kAm9NQNidQXnaZ50AL6IAU8Hnt0lvFHhcZRn6ydd3motWzsI3nR+J8aI93btOkRNfH66dvqWzf96bhA9Ta+t2/1q2e7k2PT07dvlzfdS/fL2D6cp0Qjby7q3/3n69+U3YS2/Xf5jefu70z++PX1W92k70PUPEvz7+PTdh5/e/etffv/uly9PP//xl798Wba3aacp+GmWF+1W4n3sJZGFZiMOeyhzOIgadlGvO9c2O+c3O1/HXty5DFGkr9u5PN85CUPG3QvD69hLO6eobtVX7Ry9Mz9/x3h82/jizmgGxPu6nW/OfFQcYNqkXumvx3ReRd+F0eozK7z5Pp2X/M3//XZ5+9OpqE8hslRmgZWnF86JcWlzXpzzDuf0wgAaHG9fcDqnDCJu2sngLHnu+XWhEH6FWLhIXXVVcSv1dehlqSswmL5e6vRrRPBF6hFX1ShHqa9DrtQ4waZhNfeDK/VNCDv+fNlbbdjQbzffx17efdbJ/dXtkxPI1+2BT0rt2+2vY69sn/Xo7dXty0unbxSgd26+j72yfR2qsF7bvr1wetjnWu/8dR97Zfuhm+JXto8v2j6Jhtw53j728vZJjeLXXC++aPv9pfdh+8Pr7xe3L3GNr7levLP959PGpsQd8rqloZjVm7f2awFq2gE1Lb9bwpVn3L5Ff4Ft3Kjg9y8nnru5X/n+/jDzaINHq4Z5jsMV2vLxJjXB0vRCfCY4wWi3W6ukW6t89+7P7758+PPy5d3P//Ppw5ejgd58Y9tV3ser4s6/WVDLWV7dNmfxDdKM9XYd+XSab2wTuGWXMb1IOM/peoNTzy8Lz2MK3+X9aR/J7XrswyClHQRWT0oOGzC6Td3luA6936W9jn06Vb0WGNCkw2i1fd55k32o7ctdxjaRPx0Gruc6LL8d/k5j7/XLFN+eXvt1hftfOzgT5ftfVTi9+KsKp9bOe5sqYflWk6A9hnEda3oCrT6MBiic9CyBsWrW0xxT/QrpP3U2HoSaWlZjzXoaszB21pJuS/TQ6jAU5/rzi9cRvfNl88NaJI/S9BZ13zSb3qCqu33apWNQ7z75yuEMOW9nuJzz/Jnv6Z6o9hH3OQoqo84/roRzDT0KvNkyg+ez43uQLT87wnHovGd+pozDWled7ZselLtLd7DCfoaLre7seXamcP7dnztAGGWfbeMFfP6HT+++4ItHBHj5Yv7rbu9390t6YzjB/MYD9+HdzHoySBWRjv6Q9Coj6qHVjReq6O3ZdPd28TA9HcpNxfk+VC+bHDxRr1pN7yWOnqguo+6synFr9StyzuXGF8l/Iekm4OiLDF5Ocz332Pc+WPMw82r2w5JX/7jZfPekg5z52Xn2ocO598Grfo7rXfV42Pqg8oOQu3EOp7Hn57Z976/yUPVX415OveCl8eilP57+H5bXH+wKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iago0NjgwCmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjIwIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggOTUgPj4Kc3RyZWFtCnicPYxBDsAgCATvvGI/0AQRFf/TND3Y/1+7RtsLTHZhSjcoDiucVRXFG84kHz6SvcNax5CimUdDnN3cFg5LjRSrWBYWnmERpLQ1zPi8KGtgSinqaWf1v7vlegH/nxwsCmVuZHN0cmVhbQplbmRvYmoKMTggMCBvYmoKPDwgL0Jhc2VGb250IC9EZWphVnVTYW5zLU9ibGlxdWUgL0NoYXJQcm9jcyAxOSAwIFIKL0VuY29kaW5nIDw8IC9EaWZmZXJlbmNlcyBbIDEyMCAveCBdIC9UeXBlIC9FbmNvZGluZyA+PiAvRmlyc3RDaGFyIDAKL0ZvbnRCQm94IFsgLTEwMTYgLTM1MSAxNjYwIDEwNjggXSAvRm9udERlc2NyaXB0b3IgMTcgMCBSCi9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdIC9MYXN0Q2hhciAyNTUgL05hbWUgL0RlamFWdVNhbnMtT2JsaXF1ZQovU3VidHlwZSAvVHlwZTMgL1R5cGUgL0ZvbnQgL1dpZHRocyAxNiAwIFIgPj4KZW5kb2JqCjE3IDAgb2JqCjw8IC9Bc2NlbnQgOTI5IC9DYXBIZWlnaHQgMCAvRGVzY2VudCAtMjM2IC9GbGFncyA5NgovRm9udEJCb3ggWyAtMTAxNiAtMzUxIDE2NjAgMTA2OCBdIC9Gb250TmFtZSAvRGVqYVZ1U2Fucy1PYmxpcXVlCi9JdGFsaWNBbmdsZSAwIC9NYXhXaWR0aCAxMzUwIC9TdGVtViAwIC9UeXBlIC9Gb250RGVzY3JpcHRvciAvWEhlaWdodCAwID4+CmVuZG9iagoxNiAwIG9iagpbIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwCjYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgMzE4IDQwMSA0NjAgODM4IDYzNgo5NTAgNzgwIDI3NSAzOTAgMzkwIDUwMCA4MzggMzE4IDM2MSAzMTggMzM3IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYKNjM2IDYzNiAzMzcgMzM3IDgzOCA4MzggODM4IDUzMSAxMDAwIDY4NCA2ODYgNjk4IDc3MCA2MzIgNTc1IDc3NSA3NTIgMjk1CjI5NSA2NTYgNTU3IDg2MyA3NDggNzg3IDYwMyA3ODcgNjk1IDYzNSA2MTEgNzMyIDY4NCA5ODkgNjg1IDYxMSA2ODUgMzkwIDMzNwozOTAgODM4IDUwMCA1MDAgNjEzIDYzNSA1NTAgNjM1IDYxNSAzNTIgNjM1IDYzNCAyNzggMjc4IDU3OSAyNzggOTc0IDYzNCA2MTIKNjM1IDYzNSA0MTEgNTIxIDM5MiA2MzQgNTkyIDgxOCA1OTIgNTkyIDUyNSA2MzYgMzM3IDYzNiA4MzggNjAwIDYzNiA2MDAgMzE4CjM1MiA1MTggMTAwMCA1MDAgNTAwIDUwMCAxMzUwIDYzNSA0MDAgMTA3MCA2MDAgNjg1IDYwMCA2MDAgMzE4IDMxOCA1MTggNTE4CjU5MCA1MDAgMTAwMCA1MDAgMTAwMCA1MjEgNDAwIDEwMjggNjAwIDUyNSA2MTEgMzE4IDQwMSA2MzYgNjM2IDYzNiA2MzYgMzM3CjUwMCA1MDAgMTAwMCA0NzEgNjE3IDgzOCAzNjEgMTAwMCA1MDAgNTAwIDgzOCA0MDEgNDAxIDUwMCA2MzYgNjM2IDMxOCA1MDAKNDAxIDQ3MSA2MTcgOTY5IDk2OSA5NjkgNTMxIDY4NCA2ODQgNjg0IDY4NCA2ODQgNjg0IDk3NCA2OTggNjMyIDYzMiA2MzIgNjMyCjI5NSAyOTUgMjk1IDI5NSA3NzUgNzQ4IDc4NyA3ODcgNzg3IDc4NyA3ODcgODM4IDc4NyA3MzIgNzMyIDczMiA3MzIgNjExIDYwOAo2MzAgNjEzIDYxMyA2MTMgNjEzIDYxMyA2MTMgOTk1IDU1MCA2MTUgNjE1IDYxNSA2MTUgMjc4IDI3OCAyNzggMjc4IDYxMiA2MzQKNjEyIDYxMiA2MTIgNjEyIDYxMiA4MzggNjEyIDYzNCA2MzQgNjM0IDYzNCA1OTIgNjM1IDU5MiBdCmVuZG9iagoxOSAwIG9iago8PCAveCAyMCAwIFIgPj4KZW5kb2JqCjI1IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjM1ID4+CnN0cmVhbQp4nDVRSW4AMQi75xX+QKWwJ++Zquqh/f+1hlEvAwPY2CTvwUYkPsSQ7ihXfMrqNMvwO1nkxc9K4eS9iAqkKsIKaQfPclYzDJ4bmQKXM/FZZj6ZFjsWUE3EcXbkNINBiGlcR8vpMNM86Am5PhhxY6dZrmJI691Svb7X8p8qykfW3Sy3TtnUSt2iZ+xJXHZeT21pXxh1FDcFkQ4fO7wH+SLmLC46kW72mymHlaQhOC2AH4mhVM8OrxEmfmYkeMqeTu+jNLz2QdP1vXtBR24mZCq3UEYqnqw0xoyh+o1oJqnv/4Ge9b2+/gBDTVS5CmVuZHN0cmVhbQplbmRvYmoKMjYgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxNjQgPj4Kc3RyZWFtCnicPZDBEUMhCETvVrElgIBAPclkcvi//2tAk1xkHWD3qTuBkFGHM8Nn4smD07E0cG8VjGsIryP0CE0Ck8DEwZp4DAsBp2GRYy7fVZZVp5Wumo2e171jQdVplzUNbdqB8q2PP8I13qPwGuweQgexKHRuZVoLmVg8a5w7zKPM535O23c9GK2m1Kw3ctnXPTrL1FBeWvuEzmi0/SfXL7sxXh+FFDkICmVuZHN0cmVhbQplbmRvYmoKMjcgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAzMDcgPj4Kc3RyZWFtCnicPZJLbgMxDEP3PoUuEMD62Z7zpCi6mN5/2ycl6Yoc2RZFapa6TFlTHpA0k4R/6fBwsZ3yO2zPZmbgWqKXieWU59AVYu6ifNnMRl1ZJ8XqhGY6t+hRORcHNk2qn6sspd0ueA7XJp5b9hE/vNCgHtQ1Lgk3dFejZSk0Y6r7f9J7/Iwy4GpMXWxSq3sfPF5EVejoB0eJImOXF+fjQQnpSsJoWoiVd0UDQe7ytMp7Ce7b3mrIsgepmM47KWaw63RSLm4XhyEeyPKo8OWj2GtCz/iwKyX0SNiGM3In7mjG5tTI4pD+3o0ES4+uaCHz4K9u1i5gvFM6RWJkTnKsaYtVTvdQFNO5w70MEPVsRUMpc5HV6l/DzgtrlmwWeEr6BR6j3SZLDlbZ26hO76082dD3H1rXdB8KZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDI0OSA+PgpzdHJlYW0KeJw9UDuORCEM6zmFL/Ak8iNwHkarLWbv364DmilQTH62MyTQEYFHDDGUr+MlraCugb+LQvFu4uuDwiCrQ1IgznoPiHTspjaREzodnDM/YTdjjsBFMQac6XSmPQcmOfvCCoRzG2XsVkgniaoijuozjimeKnufeBYs7cg2WyeSPeQg4VJSicmln5TKP23KlAo6ZtEELBK54GQTTTjLu0lSjBmUMuoepnYifaw8yKM66GRNzqwjmdnTT9uZ+Bxwt1/aZE6Vx3QezPictM6DORW69+OJNgdNjdro7PcTaSovUrsdWp1+dRKV3RjnGBKXZ38Z32T/+Qf+h1oiCmVuZHN0cmVhbQplbmRvYmoKMjkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNDkgPj4Kc3RyZWFtCnicTVFJigMwDLvnFfpAIV6TvKdDmUPn/9fKDoU5BAmvkpOWmFgLDzGEHyw9+JEhczf9G36i2btZepLJ2f+Y5yJTUfhSqC5iQl2IG8+hEfA9oWsSWbG98Tkso5lzvgcfhbgEM6EBY31JMrmo5pUhE04MdRwOWqTCuGtiw+Ja0TyN3G77RmZlJoQNj2RC3BiAiCDrArIYLJQ2NhMyWc4D7Q3JDVpg16kbUYuCK5TWCXSiVsSqzOCz5tZ2N0Mt8uCoffH6aFaXYIXRS/VYeF+FPpipmXbukkJ64U07IsweCqQyOy0rtXvE6m6B+j/LUvD9yff4Ha8PzfxcnAplbmRzdHJlYW0KZW5kb2JqCjMwIDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNDcgPj4Kc3RyZWFtCnicMzK3UDBQsDQBEoYWJgrmZgYKKYZclhBWLhdMLAfMAtGWcAoinsGVBgC5Zw0nCmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNTggPj4Kc3RyZWFtCnicRZFLcgQgCET3noIjgPzkPJNKZTG5/zYNzmQ2dpeo/YRKI6YSLOcUeTB9yfLNZLbpdzlWOxsFFEUomMlV6LECqztTxJlriWrrY2XkuNM7BsUbzl05qWRxo4x1VHUqcEzPlfVR3fl2WZR9Rw5lCtiscxxs4MptwxgnRput7g73iSBPJ1NHxe0g2fAHJ419lasrcJ1s9tFLMA4E/UITmOSLQOsMgcbNU/TkEuzj43bngWBveRFI2RDIkSEYHYJ2nVz/4tb5vf9xhjvPtRmuHO/id5jWdsdfYpIVcwGL3Cmo52suWtcZOt6TM8fkpvuGzrlgl7uDTO/5P9bP+v4DHilm+gplbmRzdHJlYW0KZW5kb2JqCjMyIDAgb2JqCjw8IC9CQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDM5Ci9TdWJ0eXBlIC9Gb3JtIC9UeXBlIC9YT2JqZWN0ID4+CnN0cmVhbQp4nOMyNDBTMDY1VcjlMjc2ArNywCwjcyMgCySLYEFkM7jSABXzCnwKZW5kc3RyZWFtCmVuZG9iagozMyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDgzID4+CnN0cmVhbQp4nEWMuw3AMAhEe6ZgBH4m9j5RlMLevw0QJW64J909XB0JmSluM8NDBp4MLIZdcYH0ljALXEdQjp3so2HVvuoEjfWmUvPvD5Se7KzihusBAkIaZgplbmRzdHJlYW0KZW5kb2JqCjM0IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMjM5ID4+CnN0cmVhbQp4nE1QyW0EMQz7uwo1MMDoHLseB4s8sv1/Q8oJkpdoS+Kh8pRblspl9yM5b8m65UOHTpVp8m7Qza+x/qMMAnb/UFQQrSWxSsxc0m6xNEkv2cM4jZdrtY7nqXuEWaN48OPY0ymB6T0ywWazvTkwqz3ODpBOuMav6tM7lSQDibqQ80KlCuse1CWijyvbmFKdTi3lGJef6Ht8jgA9xd6N3NHHyxeMRrUtqNFqlTgPMBNT0ZVxq5GBlBMGQ2dHVzQLpcjKekI1wo05oZm9w3BgA8uzhKSlrVK8D2UB6AJd2jrjNEqCjgDC3yiM9foGqvxeNwplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNTEgPj4Kc3RyZWFtCnicMza0UDBQMDQwB5JGhkCWkYlCiiEXSADEzOWCCeaAWQZAGqI4B64mhyuDKw0A4bQNmAplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMzM0ID4+CnN0cmVhbQp4nC1SS3LFIAzbcwpdoDP4B+Q86XS6eL3/tpKTRUYOYPQx5YaJSnxZILej1sS3jcxAheGvq8yFz0jbyDqIy5CLuJIthXtELOQxxDzEgu+r8R4e+azMybMHxi/Zdw8r9tSEZSHjxRnaYRXHYRXkWLB1Iap7eFOkw6kk2OOL/z7Fcy0ELXxG0IBf5J+vjuD5khZp95ht0656sEw7qqSwHGxPc14mX1pnuToezwfJ9q7YEVK7AhSFuTPOc+Eo01ZGtBZ2NkhqXGxvjv1YStCFblxGiiOQn6kiPKCkycwmCuKPnB5yKgNh6pqudHIbVXGnnsw1m4u3M0lm675IsZnCeV04s/4MU2a1eSfPcqLUqQjvsWdL0NA5rp69lllodJsTvKSEz8ZOT06+VzPrITkVCaliWlfBaRSZYgnbEl9TUVOaehn++/Lu8Tt+/gEsc3xzCmVuZHN0cmVhbQplbmRvYmoKMzcgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCA3MCA+PgpzdHJlYW0KeJwzMzZTMFCwMAISpqaGCuZGlgophlxAPoiVywUTywGzzCzMgSwjC5CWHC5DC2MwbWJspGBmYgZkWSAxILoyuNIAmJoTAwplbmRzdHJlYW0KZW5kb2JqCjM4IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTggPj4Kc3RyZWFtCnicMza0UDCAwxRDrjQAHeYDUgplbmRzdHJlYW0KZW5kb2JqCjM5IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggMTMzID4+CnN0cmVhbQp4nEWPSw4EIQhE95yijsDHH+dxMumFc//tgJ1uE2M9hVSBuYKhPS5rA50VHyEZtvG3qZaORVk+VHpSVg/J4Iesxssh3KAs8IJJKoYhUIuYGpEtZW63gNs2DbKylVOljrCLozCP9rRsFR5folsidZI/g8QqL9zjuh3Ipda73qKLvn+kATEJCmVuZHN0cmVhbQplbmRvYmoKNDAgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyNTEgPj4Kc3RyZWFtCnicLVFJcgNBCLvPK/SEZqffY5crh+T/1wjKBwYNi0B0WuKgjJ8gLFe85ZGraMPfMzGC3wWHfivXbVjkQFQgSWNQNaF28Xr0HthxmAnMk9awDGasD/yMKdzoxeExGWe312XUEOxdrz2ZQcmsXMQlExdM1WEjZw4/mTIutHM9NyDnRliXYZBuVhozEo40hUghhaqbpM4EQRKMrkaNNnIU+6Uvj3SGVY2oMexzLW1fz004a9DsWKzy5JQeXXEuJxcvrBz09TYDF1FprPJASMD9bg/1c7KT33hL584W0+N7zcnywlRgxZvXbkA21eLfvIjj+4yv5+f5/ANfYFuICmVuZHN0cmVhbQplbmRvYmoKNDEgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAyMTUgPj4Kc3RyZWFtCnicNVE5DgMhDOz3Ff5AJIwveE+iKM3+v82M0VYewVyGtJQhmfJSk6gh5VM+epkunLrc18xqNOeWtC1zgLi2vC+tksCJZoiDwWmYuAGaPAFD19GoUUMXHtDUpVMosNwEPoq3bg/dY7WBl7Yh54kgYigZLEHNqUUTFm3PJ6Q1v16LG96X7d3IU6XGlhiBBgFWOBzX6NfwlT1PJtF0FTLUqzXLGAkTRSI8+Y6m1RPrWjTSMhLUxhGsagO8O/0wTgAAE3HLAmSfSpSz5MRvsfSzBlf6/gGfR1SWCmVuZHN0cmVhbQplbmRvYmoKMjMgMCBvYmoKPDwgL0Jhc2VGb250IC9EZWphVnVTYW5zIC9DaGFyUHJvY3MgMjQgMCBSCi9FbmNvZGluZyA8PAovRGlmZmVyZW5jZXMgWyAzMiAvc3BhY2UgNDYgL3BlcmlvZCA0OCAvemVybyAvb25lIC90d28gNTMgL2ZpdmUgNTUgL3NldmVuIDY3IC9DIC9EIDk3Ci9hIDEwMSAvZSAxMDggL2wgL20gMTEyIC9wIDExNSAvcyAvdCBdCi9UeXBlIC9FbmNvZGluZyA+PgovRmlyc3RDaGFyIDAgL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvRm9udERlc2NyaXB0b3IgMjIgMCBSCi9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdIC9MYXN0Q2hhciAyNTUgL05hbWUgL0RlamFWdVNhbnMKL1N1YnR5cGUgL1R5cGUzIC9UeXBlIC9Gb250IC9XaWR0aHMgMjEgMCBSID4+CmVuZG9iagoyMiAwIG9iago8PCAvQXNjZW50IDkyOSAvQ2FwSGVpZ2h0IDAgL0Rlc2NlbnQgLTIzNiAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTEwMjEgLTQ2MyAxNzk0IDEyMzMgXSAvRm9udE5hbWUgL0RlamFWdVNhbnMgL0l0YWxpY0FuZ2xlIDAKL01heFdpZHRoIDEzNDIgL1N0ZW1WIDAgL1R5cGUgL0ZvbnREZXNjcmlwdG9yIC9YSGVpZ2h0IDAgPj4KZW5kb2JqCjIxIDAgb2JqClsgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAKNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCA2MDAgNjAwIDYwMCAzMTggNDAxIDQ2MCA4MzggNjM2Cjk1MCA3ODAgMjc1IDM5MCAzOTAgNTAwIDgzOCAzMTggMzYxIDMxOCAzMzcgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNgo2MzYgNjM2IDMzNyAzMzcgODM4IDgzOCA4MzggNTMxIDEwMDAgNjg0IDY4NiA2OTggNzcwIDYzMiA1NzUgNzc1IDc1MiAyOTUKMjk1IDY1NiA1NTcgODYzIDc0OCA3ODcgNjAzIDc4NyA2OTUgNjM1IDYxMSA3MzIgNjg0IDk4OSA2ODUgNjExIDY4NSAzOTAgMzM3CjM5MCA4MzggNTAwIDUwMCA2MTMgNjM1IDU1MCA2MzUgNjE1IDM1MiA2MzUgNjM0IDI3OCAyNzggNTc5IDI3OCA5NzQgNjM0IDYxMgo2MzUgNjM1IDQxMSA1MjEgMzkyIDYzNCA1OTIgODE4IDU5MiA1OTIgNTI1IDYzNiAzMzcgNjM2IDgzOCA2MDAgNjM2IDYwMCAzMTgKMzUyIDUxOCAxMDAwIDUwMCA1MDAgNTAwIDEzNDIgNjM1IDQwMCAxMDcwIDYwMCA2ODUgNjAwIDYwMCAzMTggMzE4IDUxOCA1MTgKNTkwIDUwMCAxMDAwIDUwMCAxMDAwIDUyMSA0MDAgMTAyMyA2MDAgNTI1IDYxMSAzMTggNDAxIDYzNiA2MzYgNjM2IDYzNiAzMzcKNTAwIDUwMCAxMDAwIDQ3MSA2MTIgODM4IDM2MSAxMDAwIDUwMCA1MDAgODM4IDQwMSA0MDEgNTAwIDYzNiA2MzYgMzE4IDUwMAo0MDEgNDcxIDYxMiA5NjkgOTY5IDk2OSA1MzEgNjg0IDY4NCA2ODQgNjg0IDY4NCA2ODQgOTc0IDY5OCA2MzIgNjMyIDYzMiA2MzIKMjk1IDI5NSAyOTUgMjk1IDc3NSA3NDggNzg3IDc4NyA3ODcgNzg3IDc4NyA4MzggNzg3IDczMiA3MzIgNzMyIDczMiA2MTEgNjA1CjYzMCA2MTMgNjEzIDYxMyA2MTMgNjEzIDYxMyA5ODIgNTUwIDYxNSA2MTUgNjE1IDYxNSAyNzggMjc4IDI3OCAyNzggNjEyIDYzNAo2MTIgNjEyIDYxMiA2MTIgNjEyIDgzOCA2MTIgNjM0IDYzNCA2MzQgNjM0IDU5MiA2MzUgNTkyIF0KZW5kb2JqCjI0IDAgb2JqCjw8IC9DIDI1IDAgUiAvRCAyNiAwIFIgL2EgMjcgMCBSIC9lIDI4IDAgUiAvZml2ZSAyOSAwIFIgL2wgMzAgMCBSCi9tIDMxIDAgUiAvb25lIDMzIDAgUiAvcCAzNCAwIFIgL3BlcmlvZCAzNSAwIFIgL3MgMzYgMCBSIC9zZXZlbiAzNyAwIFIKL3NwYWNlIDM4IDAgUiAvdCAzOSAwIFIgL3R3byA0MCAwIFIgL3plcm8gNDEgMCBSID4+CmVuZG9iagozIDAgb2JqCjw8IC9GMSAyMyAwIFIgL0YyIDE4IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL0NBIDAgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PgovQTIgPDwgL0NBIDEgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PgovQTMgPDwgL0NBIDAuOCAvVHlwZSAvRXh0R1N0YXRlIC9jYSAwLjggPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgL0YxLURlamFWdVNhbnMtbWludXMgMzIgMCBSIC9JMSAxMyAwIFIgL00wIDE0IDAgUiAvTTEgMTUgMCBSID4+CmVuZG9iagoxMyAwIG9iago8PCAvQml0c1BlckNvbXBvbmVudCA4IC9Db2xvclNwYWNlIC9EZXZpY2VSR0IKL0RlY29kZVBhcm1zIDw8IC9Db2xvcnMgMyAvQ29sdW1ucyAyMTggL1ByZWRpY3RvciAxMCA+PgovRmlsdGVyIC9GbGF0ZURlY29kZSAvSGVpZ2h0IDIxOCAvTGVuZ3RoIDQyIDAgUiAvU3VidHlwZSAvSW1hZ2UKL1R5cGUgL1hPYmplY3QgL1dpZHRoIDIxOCA+PgpzdHJlYW0KeJzVvVmS5DqsbQnIYiI10ZptDeMK9UEC2OgouUfkue/R0jwpimK7tMHOPfj/+3//HyIiEgLH9I2TJoxruMTkRXI0iwAeImKBQPFH2D7tVvTvSw3cIff2sBCL8O3R2G7dxELXrSE38U2X+OV1yw6ET3tkX26PXCmmNI9Y5H0X8sIiMRb+DnW0GoW6k0do2iq2IXZHavPUU1y7DKNFKhoSKgxXifAli8ODUsO5LRVE4xj+tSsP7xaRcguajFfnmcciyw6xflpdbg3KkAJFPqh+UmaI6jtDnjVpybFgLAAE7WKHymKtY2X9lQbmchwk6b9xDY6/cdyU/YFInitbnmoCPY9Pwi0Z2fKs/vggyADqhNNj/TohmAqOBEhM1jC1kpjUde+GVwRo1jIIxQIQZv3UPrmBfyUJTy5ldqUcZzak+/cqj5bIX7s+9xgaTAwFD4EQhgdjl6MCuWJFblr9M+wWH3mokJLVBC0yo2gVjXRPJ2AVuKaOxb3pnzGXF+4V1W/U8UDeyOVjYYtAIrI5wV+i3JY+CcxkqTUywsEElrp+4iNhPJdF0YUtKq4Vr5Ynl3NopLOlbtqk2ut/4abhnLmA42NsoqYZNNqZoc8E8k1JPnLZnHXptpa6EMYQmEaNBLYbAJWKLCWLjyxiIEHLtuU8WOpY/b6o+MrVBmkD/ymyZ3UMRry+jIXLZyLj3TdV6+N8rpf9EzhwRI9+2ugw6JbI2VJj/PYz2HSEWy11j6AUT6ma+3FQUZMaXDuq+e/cq6nMuWgRygc7OyVV7PWrrF+4NoGxhwCCcaSYRo11NkMe5xHWCmUqQJ7NvLDUIfAA3wB0uvvPucQMDjhKjX1OFEZ+km5hmrEVp+Z614zj7eex6zRexKqIR+6hNJ0jzzSu2jSW2vjjGcq8+NJqW7XUUIzUIPnZ/5Ocl+tBHT99NSaZPMinx4mZ/YO3MhiyswdAjNPeV5Y6rNp0n05PEMXOUstQyBa+duDYjW6D+weYzr18fMxx/M3MI+UHMvlcpiiQf9MyMZVRLWpIGjgmdQRLnYFLRLaBEoXT0qGjpTbljireVvB54Jhi/n3Tn9yBh33rj5fBIYNcgjcCWWJ6fO5vjU76f3FlWEJpDBHscpwLU+SGAn/JXj9YatDdzyw1iGKw1OTh7jm8eDHy6L5g9Dej/3+FIz0Q2QtktNdSY/5huVLHIIXVjH5oqc/2OiDuUAq9tNQQHuXwf29s+FEfDdMuoQOOfwLBbLUPLdfj+/RUfegw8azLyzn1JIc4UGtRSyxCRgdMGVCzYQDFdyNZ6gBorXi9C2Kcwl+5X2DQ5nDKlr9WxzwFPcbEcqDJPjz7D+YxkHHxlLvISpK0opfBgJJjESx1jml+tNR2ayqkNE1em6pZ2f5DN0iMdHH6yzZkJfKD6b2BgPu8+5GKRXhsICYpI/S/ddOgPhnHLFpxcJYstYVwlMA0oKRIarDUOyOhOJoMZSv+MHCc++01Bv+pO4F2HeuTXWUR8jipncbJmTUN1g0fu2J840LviocgHBiCGQrtfWoMj3qZChgXILt5N2TEEUE01tkEz5dhKNn66+VLNwz4zhEO464+wQ+M9cyiZ88DlJM4T+m8jnmOkdA+qEh9CUadmyALn5JsfZ7KpIzwMpmaZKCjRkKZ/6mFfuVO9rqNWeN8NHZ8WeMDkSeBbHdoOMb52pWBoOcQjXUIxPhJ2158trASWvwyUjwY66CgxTWjtT/i85c6Owlk/+w/Wuh5NNwec7rxhwOfmIeJTV1PDoFFh1rI8F8ePnalzlASDBxTvl3nnyYJ/5k+Tjb6cQZzTore4/g5Cg2Rk0DmaK8zexGxURTs1yiNCYiIVx44YhHSlIXSrKWdlddCTkIUB4v/0XTkq2ymeUWbZkPCRf1r9zKnh0feVamN9jev+mEQWoCoI7YQQaPlAWUjlpLhI49cB47tsJVVxXsDLWPPfw/rZExjNR7s9aNAnom8mpt/6drijgI5faurXL4qaRr/zdEaIKibr3gxicDsEgKXph1ShBCyqMPWflI1FfidjmT3haE7J3IeFD7NgAMM/3CTULP5Y8jfGOgpxMVP7BLGcPsyz2PMxRmJxynDxzxwnFZ5MOWneUwNbC7fu+/GfG/s2PD4ZLIJibz6JP90Zb+rbW7VeTL+pfl5lNmVdjuGo4hOV5aqmrl/01QaihFFUaiK4lDaPx9B1mYP2dYRMcY5COTA9zORV7JB/2Z2NqY6NWqdzbxQ2dMrJEVgio2uE2pQNQkSWMrerjjirTDcjJm2K02f1e61m9Joh0/9zGoSyBcm+xwi9B8Ya837nVqFmN+3//Akty98Mtk0TgjSjGS03XQKz5m+LP9vWPxCSPuVglKco1l/HkRWIh3HpwqfKsWxqx/dYeDyiWuKLE99iZscFfw0j4E0W7zCZ5nZUB0MQF7N4g5EqxOal26Usdfq1YtiSTkRWQXyLZH41Froae31W+vwkq0na/t+5fwPXKWk8wco1904I8FolIQzKWWM348Uo2R+0x7zM4Ee+BTuIrRJPRXoSyLxqbfGekLp80JjtJfAnwcurx/uhROVMryNT8kimg3QFcEXKTfD96/M9LnSHz3eC2cnkDV+cwnPNrlcuuRhDfEHIjVOUH6f9G+L0LjOZDfT6mHT71SAbu4SPDL1+2t3bOpGrt5MKw9mvbx2jckubtqfaxK/2hp9aq+rG5ppmr50gXxI5+HW2aVFR0wvqcr+J25MZXhpcVp9lKaHTcv3lag5PLbVwFm21+bvAH11jLC+Ay+JvCi+o0f+/rW62av2m3lkl2Lxl2jt8LGZr1hSML+phAVxfUHblw37oRWaho/Ng9GTCHs02d8T6cdvyyZYSa8NnV2vedn/ylx91V2PD3EL64tqvirm6+HbQznf1V3acyeDDW2KMwlk+wi/JjLpcUskDiV1V6aU8KW9/jMd+yfJe/Un2maTndlqd587lenf6kEjX8HGw7/iFgQwTQeh6j7tLuI1TrHbSqYUqogWIk8vw4UlfJzQ/MXXWf4xwNl9UJFu4Ji/flAWIFP8NkGOl28n78fSBc+jIZoexFsdf5h+NtkHIkFrG8PdfmVgPXVZI9YR5FetlvP4X3Rz+Ser+mYtxmNKR+HfNdrb97aYzsYDQjgJZMi6HSm+JzJFo6Y5eplcxjqc3qsJxhLllP9E7l6OkH6RfLZyZbZxzuPdSPhtai9uvnsqwKGZm72uo/NOIBuTHXUXeapE9uNI6oh8lEk7fktEVNYg377rH7XreTbz8sGP3LtpyiSi05DpHPjStSJblaMteBWn3tWRYjvsxVvkTD8SmeWZoS6Phpvi1/3sRE9z3uS9QP7f5Vo6O5Md7srrl5Oe3rCplVNHHorJGY6TvZ6LY6mFLA6TmDdEEhDclqTKpL2B+bzjQSBjnV6Z7H8zbfnDVLu+i3nYGvif5/3oeP43dfZgrw8CWU02ptmLpRZAviaSGiKJSdZU5pVAovtn8P2X3Z3zqwbyP3vHxteiLyCa9VMiwGKDXfmU4qFCZIJJOGSXy/NouOvYkSk2fd3FbgVyWiupQ6H/FdeP2J/dDEYeunw8YmlS/GBM3hS/s9eNQGrkPAtJnjSIpJ7IGr9ObmohJ5nMUNpCTxDI1z9p8J8NIv+Y7amfh2y+Jy8+2fbK+ZHqRE32abzYTlAeTfZh+vIhka3hrjKZk7rsrp2VMjpfCOQ37qunWqN0jjbkfh70f57gSzdlNljbPd808tATnQA6vUB2fV9N9t8S2RpuSi9ktd2XGlYO5Y0mO956Ntn/17l/PK7oX+EO0HaY0w43Nm1PuujxE3YxnXDrJZFlO1HSUJLKbzYlmaRSd18Gr3Oa+DdXQrL/jMj/XbjNGvzeBbUopjOBJTbkL6P7dshYe91yPAjkaLLpMyIRplrsLJPlkZPtviSfhnpjsrP7hMgQ86M5wSQajzFfhvyV6+rT5i8MJ3GqaZvsNabJAFlHf0tkUMojkU0EzJRikWaZPNjuAOVe6ElnBVqTLeHW8/c/Wvc7CD7Wzjm7c1Jfi3RSDkyr3wuhiFEaTnX+nXjkT0UREMTzy58SCYXECJjpG5l8tt0JyvA9azTZ0s+y0Zy1RJ468oDsV5gOw6Sv3EcFsH4abnmqyUxPMaFXUp95H7cCCZejyX5JJCtAFO5iBC+PFnuUyU+hZB07sqnjCq+DSIomm05ETlA2v173e3PZpXB+H/6Fc7NRO6z2H+XulK7PAn+x7EEgo8mO6rhCXhOJt9QzDSWfZVKrEBrhACWBsSY12QicDyL1r1e8IpI8Nfw3uT8Qtpr7JxE+LkDzqpW2xpjcXApjZ3O119lYDwKZTHbreU/k2XBLkcmwYVNkMkGJ6XtDJSgvI4YClBzatRlEZiI/FZ42fgnMxXiZzpN7TvZ9VgOFUtq9eaTKYYQPBTKPIKEjO1FM5D0Qifa60nbauU4yeYBSS57tAD54QRUt2mndZyKSviEjYfEsVI8Qv6P8e2fisRIulrQ3052cmMftNQ0qG9URx4tHM/0BkSFBKAYSiTk+2O4Wymi7/ZWDOsaxo6njSgou0+mKici3X4SNqdW7TETpL7Q9JiihJB88OxZwvIzZYAhjP5ldxvg4WOztNQXsggpaearJNslsQ14S2RpuhupEmUy2O7OFDTXNcmII7Mrgck/9ddedgAQVLETSG8M9dPVBGpt+P+bwhy4UNjZ3jlBNGyYCMhk6oOhK6EjosJ7Od+r4isiIZjLcoZAU40C0vPyU3jF6gvKKUw02vMogUn/iCKY1NBAJ5r75h4V5tLPVxWjV+rWXL2+dHJrp2MptxyT+BHsaH3Sl5JSmC2RSAKAzqMvgeUukR5jXgN7J5AcDygTlpfOYYK8tXwhpiKzjyARlFAIqlxhu5eoJ+/A3SQ4FGGO3MZYZ7ZIIHblCUifhso71NJjpKidGYTNYBATdIichbM30TGSNlqcvHGohMdMkpZ9Bmcz3eiot9LT2uq5EbiLJSfWSdMa6YpGjZQsuHlhV842OFhFtFj4fHTwgxZPvvtTFoo7kZjqfl5pE0fTJuvlgpgUeBLh5vWkeblVA4LSoCbWGM3glXkFJnq8/dUUKqfg3ami+JyJNJlNbVpdYiZqaHAonpnCKOef13tUhYMgDdTEoHLf8uS6S95wpZSuQWR1BFJ0wCqhlIlE+i4GWznB7XpH7gCx3nFGI9qiU4a22R35wP5r2w1z8wsREIsS8/5alaAWFmVa4psPWVU8opKy1vfH1m50++LHmHZOsqXkgVHu/Qux+WRWXHU3Aj+GkDRVOgovRz4aH5hDKIPr27h6BeyweyBpzpRUurRN3OZlFLBppLy8PMSRSCr8jENRLo5FaONZyr071Hsf4mrdUHFOvSGyR1ejGJTGxyCaSvLheDmhZdFxvJMGT8Inxe72sz37h2BsrJ6Lv4abN4DMQvUEoQrHDqes8y84bPL5m1oupXAE+AYSLx/sLUogJMhFtKEFZlNdd5ldQUohGFGIGKGP8XYiN451/eHTUSO0JweKK7CU3qLZ3Z1xTC7cgRIsYpDE92AhhHBU8Z9QlMOnrvoUSiLpob10iVS1I4FILJyBmHgilZcXccdGMTkTap3ZBe1f7a5BJKyCqXQTuYyipMQiCLyHG+bmJL5C02BkWMhHpYi5guMkrA409u6iLXkSURvHIISZptfHfpw60EGUv3aq6iCBaB1v/u70mlUa4xLGpSSZ5PwinLhANbYk0Cql8IoLxBckyKfY2CBlzHXDPUKZoBDEJGoGiWP5I6G+sv7WONYe10yYyVckMt+VqhZhcNtBQiCqNQOSkZ/Upe+QbZ7QVXTQtRO2k1l5TBFQvvZzaK1h+YC+IYh44tkQOjBLKJ0hJh10YUE4qeIKSumjkMYmiBV+XPz5T9l6weoZL0wbqBd+GkkTki7qIDrZudclMt9JoEQZkY4JPgdxfboqOuhi0cDVI0EUEeQcyXpL3CqKZzJSJBxJJ0RBbeKOIFoLwdU954obU+l/EecW7L6AM4legJBM4I0UQxzp8jCKUrJj1mQCRXgLRhj3PaLwgWsizjEn0V0yL/2MHb93Oh9nOIAdScfjoFR/MoakCqJ9zSW0gM8mbgSMdxovFRnv8IpNUIjS2W+/SL6C0mGTZrbx/hNhATCiIN4F3Rp1gQjiBR0vQcbmT8/yy5hVpxD309pGS+OB/6ZJFtizBulnr7FfRzHF4U72sOHAkDqm64oasyDisU2MqGumVjTJM1UZ3umi4dPMYsN0TlFCFRyhDTIJ3NSz0JIFMQgj+/QmDjySWRA6ltrjEVKOGoc4hixgzwYrxBaLVxJ+cTPHNFptlAOYmmcw04FMQ7k1aQ6AKNoj87VSmwNfLJKQcehNst9/1G1DmTilJByTnCTj93MIXLxBdNpcDOgORwF/i0kWe4msBjT1RaJUJ4XBZH8wh+O9DInM5jEINSUMXIhfCLXvsYJ0F0isNDYUtURSZPTcpRFb+7PNxKlNlMkGJ1FphH5WSMpREgYrwBqIF/5E1ztMFmpvkeiYy9EoMx47YeSQuO+cgErwSovWxu9i54o9kHW2TPeRdTbCFm+aRv2+p5z281UXw4OYHdgNBxHppREK5gie7oateTWUQynZAqaV7Nt9YRxTRgwXXsaPQzbzIu0mY2Kx2JTKOl/FW2DHTxgvbDFlhmqZsWGyFk0KE3rULSR+5uqwDLOLEBcN3OZNAlvZKCJI9qlxE4HBY/kIau0/MqL1sPB2UpE2r7+wIJVGo5sGCExGvZXC6iS65bpZLzR2OI28ihukO9LDXtlui8zEWQSEmh+gkFqMHTxjtyHWgCXm1ot7lT0SN+BHp5LrVRcpceruk7IHFgy6uMvgiuMZRW5eJ/Mi1FJ5YnD3kOuqz7xqBOrHstxBBHTeRdLOThxTa5dS3kEGjBcjl2ZkxSCyamRaPqTDoJT6SRJRe9B42KlbUyKuWmlQasShZBTVd4LLVRb+89ugcw7eH82vdCCEccwnAsTfvQS97jyKVRS68KvQ4rEw1DYNpCeqoRBIHEON0O/QCNHHTR67jeUU9gxllwC/BcE/hju8Ukh4pOTZu5q+3yC2m5H5JTcCeD5blWfD0rfbJzfmZTmInCtPlBwJJuwWs4T8VSzIuozouCRS6mAhALCvkRI0clju5a4hgZ6J12HYJPvHAdRlWIk0XIbHA6NlNnWqrBClmZNExpVjh2NZYrAkIbJ4nIPpCN2LZxTtDWcMLdmPBiNyCvxRL47KqIxER3R2RXP4dnZTGDW/N4EYVpDBwJAynEm6pxcCnIo8KYEvBTZwKKHjAGmjlrQ9mXTwLEkZ0HWe374eatRk9ZjfqourcvotqB53zgVg26jgRWZy/88p+00H1RNbcZJh0AFE9roIpHB8U1ctHUXzjoErljM0Di15C1lLHhYDG6pVLD4+SCPHjowBCL41x2fiQ40FBQ5wEpbZBFMJXq+j0c9N10Z3t9fIIMdHFeU6DbqWLZ4QG3NIdaaOVyTVRnKBAnN5wU/BQKU5DMJUYh95Ign9kUeKlNT11/VozfC6XAwoy2aUzovmVME9L8QR0NlWrUGJTiJ3o6e11J5Pc/Ztd10cktF/Q02NahwyljSNNLAWeapd7Xupx4ULUA6cbXeEDakM9WfPOJrujGov6zlLX8F4m2+CD8rWPPnumYaUJYRDuBejuMI+zxo7XTXejjiiTPJlsd3H/NAXqVTzcqoFWhxxEw5zGLilKIwCaiRRJyM71gBcGZ9OWKKA2sTifrk3qFjxjOIcncxMeZbKXRkjwfTFeeoiyBfcIZwtuY8frJllWO7OofynuZmJi1h9wRmXulDJAuUKSRkJBqwvyWSbXFlIXxl0+079Hp6LIof1iKSX708EIixBqCjXXln8gsitaE5RRc/EtiUWgD+5XUDazmRDeAOpWQyjNrC8muUhuEvP4pezeT1YbndEp+ZbTGfXy5JLsVRbJAwOdUGF68gfXdldowm1kLBVUPpDAmFc5Et/Og7uuIqawuNjw1hJtq+UdXI0+zxR+554L2HHJYWZdddEEx5YkmYhVJpVI83M7qCpofuCssZInqmPQP4HwD3MWimMJaEjBwkSRzwIZhZLUbFHub4YBas4wXFdMDKnz+8PMQvISSt6WNAxOmpbwPbZGI4sGt9FoOK4rP7c2bstignIzJ3Jx34LiUPIwhBEtBD5UHZd7eLSHJYS4TOJsRs10i+YJU62SWDwcLJKd0e4EMvYElUFk28u9FMXQDcqkYD0d9qTWQXKEkwry0IlD5Hqr1cU26+1fCz3bHBMJP3nW2PEWYmJmQV1kKDpnCsOBSAqqcyAjjSDJ+p7CqNEvqeeP23lMbL5QEnttMCFgFNORWAePq7p4RGjsxmDOa6y3LBYo20emzyHP/K2GKpbcl/+By59b3/uqiFU11a/wbdudDbdJY9V/+1zuDCN5HcxPlO21hYQJjUljTdD+PbrYeLBaxuQKGb8OhN3N7ofidxrJ5BN/KefgWqJl8HceOAhk2xVAx6PmvRTIobT1OEhbWCLiHxG+uUGQiatqgt/hYxGmOqDcIB5evuWsUK1DawWXPZFpum0ppO8oFk+bLUOkup8kQKHAf/mk/kBh6IytEN0mX5JJhO+UYrzZ9ry9A7EYQUG6fFpXFfH8SHuXaY0dr7CIsxHknQECitS6dd6XIkySoBRHk+pAhF7IpORLRg9Io7NoD9aln68ckkdecO3MdKBTc3zrDxhaGP6XH27uRuCcre7xKHIByuwGKamWeubrINld4I/QdZPY7CQa5YTg7ocMIq1Du0REa49aPNo+EF8OUdfZaChjCqEQcpBJI8+pBWlsxpRdL6RCZVFM/kEOk78lMrnQfbEvJzSJ4HTj01zdMc3p2O7FzNZ7jZzkuC1PCtsza9uhZj1+awNE8hmMCWcQyCyZanhsTBk1MnxSp4vFaHG8uwMRRPMUk42P+FNnp/mFosFyaRbIFyD2gRyGcOop2LzWxaYijSJOcIdD3UH8jq4pannqFam0vrpFuLItdDEjdhSkMYCYwI2z7P22LSgliCWWOBy3J6KOT8L7AFkGsTDq0pjoPDTx0AHN14DiO0XJnzdqg/9JP2AWPMtkGuG9zGP8fsPulzjXCbfC57QG1OSPxv3cJlsd17+LmEhun5r0MokCSZ1ewq01pmTaw8pcmalLJ1eO/BiIRMVwLxbBduMU56N18izljtrutwZKU7l3cmbKl4aSrUw+Jzp5HsOxREwk8nJa04cfxDW+ZjuXvUm47t4wL+b9a2SNTAJ2IkRXr45h4GgVW1/zYS0BQQdol1rHZJcAwssIIhGwGCOH1jlIS4egPyd+yytSoBQ679WWy9p5nGXSg79gsWbZCyQCGnvl0882z1bUl9sHzHCBBkWOxFdwijqyEilXZBTi+/6hieUq0ILdBk/RZCdbUR2KojWXe5BFPGahD3KbSJ+NZ2GArv/00r5w3Jvplsh1nRaK00oPdBsMU1cq37H4RiB7QOMSerl/CG9WnVMcNOI/69hEBdGR0hUcIVqLkQm7RWSVxgFN5FIxZSoDeDw817huf8VAdHttt+DT4id/lmMOCJpGbsLwUmHZMaUQ+aRWnS7GqN5YEiLI0VPT0Wl4KFnFpQUUh5WY2muBzLWrEXDsyAUpEuNvQ4mLOGtajaLYgMj6djFRvPTu1B8I8zVkBZTmGpEBBEJ6+sKXQDjFceScRcrMOyJeEscZgsTXqQAaeqKVyYqtiwmJ6NccHrUQmNuzsRRuMZEtiYEQx8UycvnZJ8VK2ecPdE+YF4PygSgqlFEXkc5lmAT5KyCSjhq9avoUbXl3DTBWKpdcb0gInM4B5RRb1cRU8fi6viXYL3hZG7lVxF4gGv5ateNdGQ0UHfmw/YgXLkay7pq2ophIPeIbY45cPqtm2xxLHXEVppE3g5IdSlpcXjFmmwI1IcFwU0DTEIyzSuXMVKfuRytPyNxJLym2yBMWu4TixTblWM2jP9q/sxA+tfwho+AfoWRFUFiO8YsK7uqoKlducmB6tuNSxHsGq+pc1uatrZPGjiRZHQmhzH4R2WKZbmVdBIG0EFZMiRo0ySTTKrAcGPEJx/1EC6LFcf2NkUuDPiBCbpoTkTuotnztHgR9ymgsRpnoVE9SuETbpOQHsez1v4hlbcmDjWD4fUc2pEAI88ym+Lm14O8EcnFJ5CEJzQ3QioydEf4HJ/mS460H292l580FWuL84deDdsmdyNDyEjuLm9585UZS40ZfJ4dBqwb4XonlM1ivB5cpAZ/KiApkWi/spiadQRdhoovX3ZNAKnPBcJNmRxFNbef1iAYFYrA+2SVFXNFQHS3ClAK2dengwKjTCUTG37B75Xi/J5b4SSxzwfTdPTJ3qNEgewdFPNKGXE41VQUkUXX08W8cRxJCSQOdYYgp6uccoYOYNM0slgRoarXDOavCSarmdun9Ew8P5BVAe2dSoU29Lyh4RiI7UfRemw4wRFWeQuJ7GaEkykm/ZnEUS2wQNOKHtoti2R2nw4UezJI1hTNMGr5DbFpDFKY7oIsosZh4mNmAZJJJC2v1sdqjmsH9VOk49SZgtAE0NvHuBm0ln8xGmWyIfGuvOmLSfc5Ck6H0FCKULXNF6U+RUyFf3prEso2z/3KCEukDoySWFEGMlxs4WAwHj6jycXOX/DKzGGfcBD+GZnSuhjApHZ1Ga0IKlDlm18RJD7ytyAdnm8jVMHGs+ZR819GVzjM3CUrWbzO/kclWFx8L+vja5fpAN9jdn1t4g8jezZNoJVXrSG0Hi7Km4UzrIMXS1AQiFUZ3Tytt7kE9UJikR0ldUpQY0u/WPItvlMO4sgOBWdEeBDKWoP/GzEGTajRW+4KyBHdFii62hJXJdVjEqYHP8sl6S2O4OmpB2bZeOrlyjDSwwkd04FLUzymaXaYlySyTGCJQyVivBqWu28P6ziCiZIQV/rD1nci4Sb0alZLhPn/D8BvayrO9tgULvmsRI7eB+fIj4aQG00i2Ns1edxR4EVlAh7jRSASROhyV1/EWcklEOL7Up04yicNKxcT9SFKrb60clVWhxsWOr0SmhrbBJaHhHtOLS4+xj3NgvJsvo7bZKwR3Wcsc5uBZ2GItel2cHw/AzbCGaP5Xt+yF1i4nIlE0mRQa6lmpzE3hnZSWeQ95BJPGE5qEr1MmLwfGu/mOAtI+QSCQCEEgrxzhOcgkRvOM6yryAb50CQiOe9AeCCPLwkpjxKnn8gzrBq7XxRi+vtia5gpE0KAGBAuQ0aPZi9wDixgymPKi0FbCZLixIgWiga9IofV+r5GRABSeOIl5kMlJ7FJGYU0ulcH4qOO5FEfha2J6IJO4WLbCFgILlwdYPYWTvTZ11MbBIbetQufh2g4RAJErWPmyZXG1QI9sZ8opCqRhSqubSWrfqg5pZUYIJ41s7+6UJiIpoNky9xbKWAiuX16p2BU5pGKvZzPKQkvJApdnKyyMWD9xSUcu99f+tWJsfaHtCAh2dHZoPg8rn3FEBXVTbo/U6Q4hmu03FTvSzoo5YZGQyUSSqnWVSYIWfg1lw+hEYRvhHL9XsleDy2SFD7D2WVcu7YAZVsZNHmmbNgLZDexYlDCGwJk2Ot5N+kqiIDIa7q2R63VC61aJNEuK3XYQpBWlu40CSYlI2iUhcihFk7JyWr4PUJaOTOtGgcMjcD0ch+Ubsz5lcPnBVOYAa5XnoI7Y2CaNWtez+TYmVKIEBpfdAvikkVeMQO73lc64tI5cEl4aCkpk0kozfB9rpL6uPZHabqxJ97b7EUre9qqVzFHwDjL5BGt7awNkTdnFH1cuH2FNubTqiJUxiSFtSjKjE5UyyGQICRPnlkvnT+JlikMhZC+tE7FtkWNDoEZGJ9D/u6hfOSeS1J6ogXaZJH2rwXZrLzxAeXCZzhT+DtaTJe2pykY8UdVI4Ee5OI5Te+tThK02KeUZzRdcng03F4/KYfiaoq1xSGqjtn4czPFbNFEUawjKJAFzE5SONGGJzDiNGqlB+VU88zwAmuqSqIpPqSbhOcujOj7o4orgxjqWltpqQ3FRGPbraAjSiGbL5YHF9R3FM5q7cVQs97XVobSOxJvWE7uCkkMeOnUg0mXS2uQAJWzSxN53bd0hEb7+XAwm9ObzMXIXQTFaNqtbIcIRy8scUR3TmD33BagjqS4SQe+WFUFyXQz+ics0ghRt9J5UJFL9SSy9nnOf4YIlIZeUQ8YuT0TudnkNpd0XjWO6ww6lS9FcF+vaRiwHRUwPHtA52twyE+/UMaTQsuu7MrXNU905xzFzEzQygUgqnI1GoidyWQ4H9bB2RIot9JgFt6NJgGalFOlBBB+4VKAErnI48p2g1EAkJCkIBB7FsmD24GYKP1q+jvMY1kB5ToEKl2as/SWuxcYmiQ2Ege3qDxWBbIeMgJ0Irz9fzAjijnMRwrrQDERqFjC4FC+QaZE1pb5OgmuuKpaJP5enCUrQrwexVBWkPJUhKsPK5He6q1iede41f01492BSuxhYuGxBj1zCd2WsSmI1DiGhMWJ/mL32Vi4CWe21e8gFz7m8Ra5ixCOI1ZpHEOPakL2+1hxARoAyWvCslBRUtgHC2gzRPIslqchFrnq/6mQw4q0w/oa/+tmdY0KqhNq9aeCSHriUvWdNoTJ54fcw+wbRce1kv+VcVhYx5FYP/srFTXSJXgYjLlc06O1vY0Qot4dsyyy7Hsq4cum9fCAS42nKZHxbdpACjhpp4C+F23XPZYr6ay6DjMFn+7goQrs4MnBZU7DvyljLsbcWNGl987DNsZWtyau9JpXJxGIQxTibvg01ESYRteDAJXWqSS+hBI0k9Gp/HGRybpKpmaJkWqPhtPoXXHo2/wGXGE5OVThHkhI52HFLYU1lmDZ/KJMPR1O7FvfGBFGkxWjBsWGRC1scVVBULIVv0EjmMBlH00+d+XYo4XWzNs+Xg0ymI7RvXZFMggVRibG+s+M5gcTlTOR3dvyD8SUlo1+4/Ll3r+2SNzKplXuhBqiUcUC5T+iQoam8NhpZcCSOaF4kInSxX+ps2uc6aL7xr9alEsvaBw9YWWq4xfwBkSaEUxMl6Dprbrn8hkv7E9yDqMZiveav/9REki72dGZYlctgrPGwGbRakslHRRBrR+2YuJGoaK7hIEUcL0qWl+p4cVMiwiQA5a72pX2o6ri6GfUy/NMyNq0XDdM0lMz9enxz0eKH5sUHNSOauDyzWAvVcplgSrV4RKrePUYLxn0aA4CxhqZJgEItQpMMrtVI4QQl0f4y1+aykcaWxWuLH8mWyT0BlyiNdnmln1FVy55LHRHThkN1bImsAvnCisT4mG1CE7mMLZz5491Zbbg/qFzqqBilFFbOSy457wocjZg24ZMG2zI4ftIBUPK2Octk1kj9XCHRWG8u7c94GXY0sChCBH5hkYtE9CesaH/eUSb52k0R/kCEl3jPqLHprOsaIlPHT+7RmpA/n9GcJLNmbXb8HZd50xmiPuzoHKp5eArppHkMsHHE1ceJS2gG9I/FM4EkbUcwzax6aSHIpZlgSuY7GOuKqYjsZ++LiPa30nwcSY6m/xV5EE7Rj9VkPj1tiTRdjMKaO0ndGdmQgAmz3Uc0z1w++jNVKhv/flHdLHUbTVi/K+M4RgqrRrr5tltzI7tAkvaz9jqiScaKz473z+UHFlEOl7EukikX3SJ08UUO4k3KX/1TdqaXseBWam/MI5GjXQtNlQNDlskYYwkQzW53EdP3ksQDFi2jrJY65DotquPDX3GZ5jGDOpKD6Dj+hUayNyqYaaqDSFXHOKemC36wL85jsrF2Fum6SGjL5ALxojjdhj/Y7X/KKTuRNeEWbytsOJzhriaymtauxO4+NlYuAYYDkCEkYXzikqMd59CVJy4HecucvSd1ejyMHZMc/rlGRhZXE7AGukzmeUxZAL/2suqN5hsHlOuShISdV5NJimJ5BZMtl79BqkQOm6AiSrwc3s2JxUYv4Xl8k600Fnn1SLDmMYGDujW3W84ws88X1c+8ikIlIHkkJD+ytRA1MqnjB8tAQ91J7QsrDW6pyUFc/r2tkjZacFlHEWSm26QxDShlKWUnk/B5F4Hk+NdMrPSZSOgMW570DmiawJ3UCCVablSwy+nuWSPTpe0zUVJE9p5NdsAzabmMFa+7MtOmTs/r3rNuNTKq47gGRF6Tg7MVn6SRAU2YrwSl9KkMKKWQXMRCNwjkvlxKqTJ5k1wokzrRcUDxD4NGwy1aXvVrXXSGGFbFc+f27dC7FC7NXcG7OGqchwoBL6o39HXqwjszDfPxVaTXS+UTl81CD0cQH9GkDwWSUCPnlUiJG3phBsO76MJ0G5SmiBwFUjZVQspllclNqnYkx7k2QPlMJL0QSH2qNMoA7pACFqIj9oVM8u5H6W8/ErkanLXTNw1fcxnCN47KXwURQ6hK42uNFL23uESBZAARRZGTNF7EaQFcSC5GA+0zblBHoiKTuiR5EclF9x2HkpShHIk088W5154ddzFPzRea0Z9IdCbhhFTxMvj12k15K5lJUHcgQ8av7HXlMk9lEn8OYhJFvLQCwBLFQSZtg0iIqPwpuEUeculE4tkIJhJYTVxTaNYFcNHZjLabEN1ClwrhtWSS+F5LkkQ3ERFddTHIoEQitVcFOnh3R5GTZsEcMUosFgpzQ3IXa7Un14gvNPKlm0w5Xg5cvlHK/Blw7KQRNdLj0LKCw1ybSo2hGsy0/myMqiM7hcYlEonrjgaiCZ6v7LAzJ0qh0qksLv7kIjZ1vInoousGmSQAESoiIqTHLUTrLqj34jJ57uPWfwbUI0TRS9bJ6cQVSnhiupz8FtRyaQOAg16+J1J+gMWA42i1gcK6oz31Q2wDM3ZEhH+lMFltRo3EhcPEoviWDKktXnmaEXaB3JIp10VbJslHkNdFcutExx/GWsDqj9Ig1uyumUOnzq5a4dHFaEGb433HS4v0bL6nRUrI/aSXaP0TlznSoI4UBbKx1AbiYUJT1LFp22jXrNpKIXCZQGQi1kfyircDel9r2ZFdGu0zsLgfvJdMLuoutdw63b4vMPmhVcWXfwiqsJ42gYSGDhMdTAkvWzk8A2pdkAAFOhvbLWGoMHIJyjdxORpuC9jjFtbWKAklIs1YJ1GMOGYQHcdOJkv7ePNhDYnybptzyUQKJUXbbalfIJPKoo4mL5aobqL9BCyuu05kmNNcdEm4JEKlFFAcvV4em99w6p3RVRYfKDRXbLG1agiQGc358g2XwVLXsjXYxXWi2jRp7JgHjmaa6yASPqlY6nFOY8qB1dtW20eKrJsxDF/dCiY7yuSiR1wFBYqmimjIRlKNSLI5jSrldRUivV5itcRNZO85CnLY9FHL3EfTbSwLxJQYHs6yIJqwr3PQyFSQwGVhzu2AxW7TqlxuY30HAx01UrL5phHHJJBjA+pdLHS2ZWaamdYfJl0WXOyvZuNqtg0fdX5DohoptH7CxyY01LAYibzougORnEw2DiW3Jfaxo8FUlyRPMpnErIQ/6mttbiZvz/gGfaaR9XIsQOUy3W3pTFziATPQSBDFdFer8rBzaPVvi64dRgAlNQK5S7stuJBc+6DueiRv+jX6B7ZYSfVPIooxhehmTeEiun3qbSDKTWT72tC4UA8YPnpQgKbvY87+ls4HJ/oRs8t0fo5m8KfNxjfP22XiEu04TmVUBYsoRmnMXJrHMm7nMbG9QpXSt1IIWARMiYlFFpHVZG8W4xo4bZlkJNWKkCywkNzE217b/EaiXl5q5lYdbuEr1BVb2EYmjIAyRI3NhJ16sOkPWmW2GB/phg2eVIdmuDxMt5OV68I7UWyJzFMZQY1MkkkRxF4mLdBbom8veyRsCZCbY0eQ4hRbhPHbCGqy9/Axmmz91DUgXY+8BUDEQJH74jS+TBb8wuVJ2ouRuz9ipdFkY69XOEKzFI0MDflOIyX5EqMwarTwRtTYe7PX8mEEiZAxlrxFUOWcKeDYi2K4JPgsFGZjPb3F8QUMr52dSNCypuNnG8r1GNPq3ispZSeEq8ppfkOSBdKmQWl8SduQd0SKNqakmsGQEWZsVRTHFmK4nB/Jd7EL4i18K0I54em0DETtYK8z01lZWmk8XK7Pn8SiCt5h4Phqcn2219g82En6mQeR5IGml8IkZrvJB5G+vrjgceVzw43oVMMd1iPxE6453YuCI7GnTSBxVTIf4LVejZ6GxfbBkkITR1kJAS7u5alSyMzfoRilVK82G38Si1ERu8XwDsSklMkPbeH1xvCOSNRIk8ldcvyCiymPmWxqNgnJARVkiXTPhoa5tm1iUyTyErqtUviJ1WWyjUQy/Sh1f+jXaLv7mIdXP/U6BkQ6K5pUOm1iscE0WWqIetps/AEW+5VwvIw4EmWlpGKmzZTnBkr2pRJZw2H/cF/61wxEbpXJYX2xWu26vN0SyWs9UojW94o0aY5JN4NIVcyglCCNZsonnkJfzqcu3tKcuqPQ+WaF8kEyqyn/iMu6K+NaKD2F7QKkh4DHK1CbSbx9mbouSTJJ4LeTPjfcxdFkZTHMVzZq9IJIXnPte0vj3qohIl9A0hm39WEdRKaeSy8bdZf4PPUsnmx9daDTGM3lA7NQ2U651OpQrd3kn7n0oGZXhubLlflgrJHCSmSbe0NgNdnaOhZ7pxC/FrgYveEdDyxe/dI3Ed3saTRnJ1jum9Fk28T7urZkhukP5UEk1Rk3CCRSuMJPSEGrSbxsXWjnNprNIzGmxCxQNaM1b7hUjj/i0sJ5rTsSQgn85cvBWDe6aIY7NQyU/mykuF6AOvq/a3+7ZZ1Au+5tuH3tUPaAsphsJ5IAQQ7IEvMebqbwlZYJJGO6xZI13aa+cJh8dgm+KnI1Tn8ryQTHuJVOMGK5CoDmb/XS7mVjnVg8LPRotsl2E+o/Nk5jWrAdwltIVtvEYnH+TaugYOIrM3rMtlpwMnO8/RrIqI4rVbmYb22HvQxpSglcEoVBJOFAJZ6jGSfXTSWDp7KYGHqVFD4baUIhCJvaiCZa847LdPnWn401ZaVsZFIrMK34eOljvQdzkVvJThQ6gYNA+swG7LXQ+uKBLFrCMduydrMKelcigzoSRZu+R4odl3u8AV3n9IDFCAJJec+GS+NlZ9YztV2Mc0oEzVdVWe0VH2bEQQWk4SHhMqLc8te4nwOLytyX85jDWg8UjUuU8T0/3Lt1hXwtU/NmY236uTriAlA4DjESSWbBWWhaHtcmCtPtoUP2XFtJbfZspuprrDOL3x9O02tOWZC3fJJDSiB2VTiw2NwKx29pZjHZ5cOiY1x9pNpipoel4ZE30YFLttpWbTyWu/xp9LdWGZm2eJUFIIpEinaRlQsUVE12uzye6LQ6wpuZesBPkmuLSazdK5dUrX3qMakESOoWXa7wuC/R7C4bFhmqLzizjvC1c2onr1XEJ3sdWibeyiZbQhPk8NjEXk+w12t7huCbNO1iJAl68upP8hPPGzZaqLh5I3QxUdwxg5ruYFzhjzPug1iG8GqXuYvWusKrD5Bqyoe1yVn5mstpp3E4YBas89v9GNNCyBm/Z3hwaViSBaIKhnEZ/7Haa1qrj5cuE+LkRo11VkHyWYgXmsMIktoNGybWRU2K05r1TccgmeZlHWNpzd34pmWgVi/ZPdKFx8B3eiviD8IT3iKFewfxKzTz+/ajoZXFaX1nHD5CKQKFrcnuXf6ah0D1OFw0D++6JXudJjfwvRlUvpXvWkMkIRtKmrNUKRJJtkhuAgnrkXQTX+KHKK2gpCDG1WamfunxIJM5XniX+lafkuI2/v7eZ8yko/MRzQfzzXVmfWARKDzNqQ3ENE15bE6GR9gaoUCZJKOVyWJK91cESdb3D5KRrWvgWxRh7Lg+18oR4RYiQzq8xZgJltpX/wjs1rQm2/z2CwIvvyaLDejaNrT9URWk8RVIK50Jzc6gv5XM/MXWRxYnez2B2Epjel9ioCCXSyx492WTDnfhaUCnnxeRwOQG2KpE7lTX+o5BbiF7C5F5rfUw7/1DtlVx2znUV9e/gOjt1HQVR4FJ1ct0GgTYBByb/51hCi4+Aia8oxNHWgJFIkczS2b7NR2pX2z9jkUrEsyvvVKHt7veYucbuMxQBoGExPyybrCY+b7ovoV0VTLFsktmuQyWRa1GJtxCXBs2XDKMXNL60yOTyU4dQ2W35t2audTLicVk1ru0wqOMd3o6RfyZjCZq5IHL+qMoL1kEIiWUfgCR6Vz70j3ldJZWZGhgtNppYnwgclTH5dE9m2iv3UAzXSTEfK8fgmNtMQXRuCQh7ky2Vybt2ViNUBEbQ1CbEdq+Rn65DzTFhNEUhPk7pHkHg26D/vbFC5c/OI7plnjcTzFk5wQsiqcDlXo99sGSsHPJ2htWKVFWJ5nEeQff/nt5N7SRTTiqhhK7h0huZuZN26VFDENMu3vHAYIJpOg6u5ls8kpZb7jMaMUPFGarXR3ny1P8Cm4bu677xMf1LWIPO1jzYsr5h07Lje6nzGIvighisRyvxjC2wJG4jD+Htf46UX062+sod/5DPERElI7iVo3c/PmU3qcsFoF1w4bQXjPsH67PrZFqsgn6W+B/9kdEGy3/lMDhvIXJE0Zo7XJMoWGPW2+MmoaJMbah2VvzzpR/8Rs96/MkigHElsJae1AG96oCMIplK5M6Cc0uDAbTLJtoJtK3qjdzcrGTkibdNxPvn3LZB3UZ5jHr01clbbiE9aHg94AqkMnzxZiSu8DPH+d0uxVOFD+K1ryko8Zam4y+ZvERxGS+5xoLRPIVhAild0eUyd5wU3cy4nL1aokkQnu9Ltf3doK9Vvm0ynOdG+l4Ncok6RJrYXH5dx31VW++F/vSDd+RbQM/cNKkEAZm3fyaOmseJNN/qv4Ni/qnX63/vQTtVAbK16KZHYdEtvHSlPUFo6UvgH6c4qw+461guweTTFIgctnAkUgrPcdAjbNz02kNocqSCniY4ghdZasmcumVJTXQ2j5oDd4rXCOKA4jH/pmfnU7y0jOaLpl7V0Y+ZDEaaIY2RZ68eC+bLY5LLP3dE6Cg0V65TGaBxL0UilCSE7nXIyuR7GuTmHPayN6fdjIS48CejQOK6z6+QanpQCN4GMPNargnV+w4Gp834X205qJsAEU6z2g6LI8/N6ohhcVkoFsQC4VtYIlBZLaMIpc2WjIrlk1Q16KmYMlk01urDU9HIi03yBbtNe7ZEPlqOdHar4TeEBhjYcUVQQO1X5LcL6IULnLxUgtJCXnrajpQ/lSKfPBnQhO/nECfspistt2lzFyeZR8c0sxaEw2ylch1dzd+WBsPy0CWZBbLD4m0pwORtiqeoFyDyLRmtOJvEInW5Obem5aUfnONoskm77kkkMlen813vXU6HBl0a4hSb6chQUdnc5JXtIL+F1vfsTga6I9AtMj1DnfRkEv8QZ9oznA0SZHI4Oos40gkzlcUQSDcdhHDduImEhfMOZ60IArTmnAOKDajQxkFUivaVnJwiEV98AWC6FqaQ48jnROaGo1JT/SQfMXiURTruiO9aDp8KpxI1Wq4+hmC7GvLQvoTP5HI3GufEEnNGbO4hUiKodZwLf2sCPvZW8PTUTSK05r0w24EFffkwXzjkmRUnTNV+e6HIJ5SLoNIhixoQlNQHYGnsKZD37D42QI4mqfofO4CyuBrkOl3zxRTNNxviRQ98dCtkJs64ho42REKIuK9CGQaudfJWfiGHW0mvhHasjYuoZ0tng2Uw51JIKfhY27c7G9Y/G5MmRJ6s1ROtF/tPXZES20srh+osAcOLCbDXbOfynqshntVCz1HjjIZmRPn8kik1UvR9KOK+izClE70rKa5iZlh9g02fS/96GKkY41zRyG6dVoDe4m1TUQTPwmk6aL6Xn5ZsRXL/tajg0LmJNL5SBT8Fetn2ZnGUov3FgXJnFg8bcbEOr1563AGg+njdHKSyUpkKMnxgMU+9LDHeXLp8Qg8s0NpC9HUKJpsgi/FEhPdezbDaaRg59Ds5a8mOy17rbDDns1799FvrZwTl9arha9xiuLAzJoiixCpYfFgoMGypOxfKSWn/6FvICP78WaG0eSBSENGyCa9RNQTybQt+Jr81h+rCOab1VabvdZ/nkky5TqINIHcg0jicMwCX8h0Ds3Ma7+pLfuvonQN3PVEDp8AfXBtzPgKZe1M47EfXKzZwAlefsriA4iP0t+Z7JWXYIhSRpT3aYQ+JHKl2+0fam77l3osFsHyjcdrpdHOWOCWN8yy9+caJOy//XmaZVM5YrLNchXIN0pZRpBU0JncoSd5vDjSKWtmLchT2AYMG1lfscjx8mMHw4vdyNDcjDKJhlsN3SsiH3a097LgzYxGltIaEMv6hpaLIqB5MzGrgupTd4hin66Rvs5pjRkVz9pmVxwEEiOlSXfbwI0r8WUIP6VZB2kHOjn/efW4rBMpbFiM/CXyDiA+M5re1DKEMvOdZBINd7fk6bvbz0Tq9snV6pmW0CQwreygsQYDtF8sX4/nve6T97W1hQVe+DSetldrtUMc0jT2mrE9ExapweNdKSFvXU2HIiEpWduz/p5FHCxiToWGNrB3pcShJglKkMlguBuTvQLFibTyVCLdhi4Wy/J4syUT+jktDBHvQZ6jDCd2IWU9o7QGsnVfWz1hbKmAWpUpF+e1Swy9TwHfjxfpN3Tqecd3LB4M9BnEr421ZU3AZYKyyqRFMSLBfGuiTAQ/hkL9FmKc53S/nOvVJi+H11y3vBnWfQDlkKeuAV0it/6sdBovhepBCzl+tp1oAhk2tT8DdD7n61lPt8bnpAZpUj+gef5Jf8biA4jT3bZKapHJoFQQW5kMxywaLjsiqdmwWWXpiUxHfrw5wJqm6vAeL6Rlo/WlWNa4TOsPJacfESBCk036TiaTDXqZ+PMXtWP7lVFGY/W5q1kEZn5oWO6ul+vh1yyOpjkNLsdS96UPUB5lMm/VzEuSJyIlf82g1citf+7R0+Po2Oy13/X1I338gnVyslVxPWZB2AWsIxZTRzPZK7AVyNSYczdkaXwE8Yzy8GTo4h/Bno/LOv6mZoF8ZnEG8aPBjCWbudwSM8sknvf5DZGkFvPGgGYLERprJbCn0mEX0bd29Gw5r8s9VSLVyDWI3NvZ+mIEpYwmO0Bp91uBFF+O+LAnPjbNzzFTirjQMy0xhk/qWHw00J+DmJxzaakMMmmDpDrRzkRumnVmkwpptcaF8XiAnHE/WqXR5zcLtRXIkDTYdNw/tD2b9RbtMX3dzn402eYXYooCGZu0juIO4pdvle78woJnE2LL4ONyN4ZED8GDlO7GMo4gHmowPMMVyiiTbrgfiVyZuEJaekQUTPaWKPINvevOW4hW5lu10r5haHVZucMOjX8jDNbMieALstcld9rOjqs/jcm2O8MuItfvIp7Fst6KIX03vpDD/GBYBv8di/UgT1+kly9RepGjCw2qMpkN93si4zEY7xqrPhz8sY3muoWYpHFtWK9vGDJqJLKbHrQJDeM5NCFbG388ZkFRLA8CiWYkBZ4dRJAh/M2zwa2EfqBWTB2L9HcsziDWQkq9iJGqTOKBSPoTIm0XG7dJxM+hrQfXL/Ws6yByZKobNrMlmPV9947CGX5B4DLbDYVRLSfsJsVQIGRTFwXSHvlYHFoW34B4diuFHzPTlnwkKQgn3v2Ixa7S5yrgXcH/CpSN4daYvyRyp09qrHF5HFclYXHR1BHXcfapH1sGJxVCNdl2bjIqa1gbt7+4bVzKFTrCv5aeqrD4jALZ2OsP3UsWX+K+0/jphLAa6xBIM4svRPGLRgjtW6AMMlkMdyZydc87IomGXwUg10tmott/yidSiJqncQiwgzrceCZI9TVuHuoBC2l6Bz0C7WP6x+uHT+OQJthre5lf9oplUJzMtya3H/mBWiWBDMxRU216ZPEM4mcvTvDidyy8JDByR8PtRFLwvCRyJ1qPRZIftrjsHO6KywFKL30F0SNsBbWQ7tfVVnmJqGzYoIcjmvYGaruhQDb8HYaP7FFC1bAIvxHdn8JiP315HC8mFg8gfvb2xVYGb6OUZbvWxkxmm/q97DOR3QGLuAxJdsw2n6UMLIq917YwFCP4sNJupS3ym8uvPsOOtr+NpQmzyY4N1Qrkp0r5S+tPZDNr+k9Y/BTE5CKXbpzj2+yjokIkUTDcf0skEd0Mf1UzPmd1EM7L7ex0hq+D4fDRbPd171+UJIL5tX5H23oN1Qsbam8O+L728dcsDjJJ+dYDiynyFOvnX7L4ZyAmB6K42zrKpBvuSCQRpaHkB0RaFeYtxEv/tAKBNBpMOo/RMuU2UgWFg7oUv6YdZNiyng7+KGXeNjjLThY8CmQF8VOlDE/G9js4+aHIIo5kMM2vWXxRivN7NSYATeZQouHGg9P8PZE7fRQ6G00SLIzLXjIknbXgeZ9w9sdO/URLbamLHtTFX+6zpR+bPPn6fHvwJzYcIKomG0ITbdleW8NOenkM/IBjP++InrjEOHEZ7n4uii95xWlFM+iG/Oy9t7bjOLmZFoDORFqlve6oQ3eAKh/8SWuQDNVmKLdVaO1rq0aGcxU21rzjtKZuZ5ffV2EkQ032Cl/V84ongTzb66lfyNvu5GrKP1jo9FYZbZ29rqTmLIayvFfNnE7cmQ0RkEgMiWP2cQHIRlGFSHIT5z8uQFTOjQOXTiTzPkNuyzfx1I9/XUHvwrZNAMGwTns208Ef7DWy9zCabDoq3/fWmY4snuHGhZ7wyEcsFkM/lvM3g0jLpda2HU0ao/UH3yOR67FMpJZWrAfj1rDbRD1j4XNe2Yp1wSDSpRE9BCFMCqkhG2fZec+mrI2T5o4NBC0VTPZqyLToU46iZcNd/G/dDKInFRZ6zFNr8msWeb71sZNBKVMjnomEcI7f12b78lc53Ep4iRsmcjyri3gZYTvFfMbC593s9cw/HLASEQpfPiSVRjXcQgEc1v8CXnXRJzbOaLUnO95KY4nZs2B/dYsIoLTWqOHfsfhXIKZ86yLFeyL1ljc4SKYTqUlRmmibpSPYQpTKJSyPR2nUcWF/xkI1EvYPCUBkl96lbvi1nnwgzQrMWtpdfqF20ecskBOFBxfjn4x43pWhAlyB7yMW/1IUq1vlSa9jNNzPROpn3teGwaKmUTZs4qIPwV/uwJORa3mcUeHMsSYf7TUsj+v+ofGqZyv2jDv8ABWNB3+UKQwgWDrIAxVyIz4K5Bt3YLGmZsYaoyNDXG/FhE8s/jsQ0bW2G8Xvicj+pAX5nIYORMIIMpz3CQtA4S8m5VM/q1i63CjIq5st9meRbJxlX5qdlFNI1HSQfZOBlErc90/8cWlbDMk2anAeZ8L6ByP8r7D4Mtr5vWxt90wk6YTmJZEUegcWI2k4XQHnfdTjf1rBnK/daHHlf3B5EusO75zdCgd/xH6AjSlPboScS7Y0xW760QqmaLKLQOLI+0/0Mri8SUjQnxGU9yy+AnGy7FNEOUayOGci7XGYYr8hkkw/lEilGsqmn9etf9IaiDTsfAsxIsV2OBwqKG7A93bOKk+y++kQmm4nxj0bMv6cRc1qWIakAFyVQMbeeeJS7JmD+0kDXvAgf3/J4kT54FKEepanJh5Gk0AkoUwORO4nEpFENCyPMxWBvBoiDZdVqLKXre5/vNoSzwTtkWKseDgWyXSvibYuPPV7NuAxRBYDWwHBZI8C+VIXX2vnfklsk5Ciaf4nLMLdLzQeH3yGEmUSIvMTkUTRTNsykIb0RK4YQCSD+XZcppORhG2603MF9QjwbrE/dethH8bVJbXa1J39CckKCa3vL8BYGSZzdsyCoS23R3wM+mwUh/7y4xc+dmxZwRx+w+JfgBjyWqkeoHw03J8QmUIqkW4Q7bDPOjFuay6yV8vryciAHROxCHNQweDfxyJFTbyfsMSlH2au0xqKrw0FiJp1HzTZFjl9luYN4W+MOEbwX6GA+npaBb7/E1hMqY1Q/o5IgpCtEHHDRrNVPUum8PY5DZVTtHgy8laxu3S58cKCBlKJmIRdYu3xdHpID3Nkq20gSjLf+naFdR+CZlEhrAIZWhVSexbLxCLpr9+29pqsKf6Cxb8FMeS+8mlfRBgAeUhHZEDTbDQPRKpfH40TzmQTJSyM75XCG05GKjh3WOKJC0omh4oTMUy08fz5ZnEfQtuLTbf+HWNSGSw4+rpP2gIo32dIS7YvyXvl+rGjV7Nk9Z7FfyaKreOJSNot6CXqiHyzHtkQSbvzKG5o1aEkwUY2qRRxPfVDS//iGQtADTQyb21fOt3hOE71Te16wKKomQ8isVUpCGQrh9iwb2y0tUJw+MVWTLfGZPqGxY9BTAl+8rzD8Wi4vyMSDZYd+dlQDmd1QSDX3TgFl1u/FIunftI3FoLfDHo6NFn/YggRc3wNpqNoHHocVre8guFtjId2vZU+tNfNXf+7Mm0spOrfsfhu3PkyxVEmD0Tas0ciV+zzhs0uY/dlhr0uDdMaYhfLfOpn+/MgUsh/X2UpWdjgwb8Ywt03GeajaIhRqAv5iDnrX1oVxwiHkIMLRyi0UMlM/ysWvxhxmAk5uo+JhC8dBz8FIsPlQKQ1jLdP9zUG0l/XbU795MORcMZiYfc/O6cLeLW19LLALmHRCac19SjaugLNPZjs1IxZIEv7P4olER6hoJnF4n7L4i+Hvi+g3Ib7N0RaTPA0W4i0bYsWaqcYfhJNQCDLVxCbUz/ZoyUAew2/QeU72gTDRx9E4he04ZCHG25clNeaPJtseyFngcwIPirlz5FF6i5/xeIvQaxJnaH8KyLJzXeYV9I2aaaKgkRahDJuiz+rQulvGHaVCj0ZWhF5JTD3i860Np5OwaXpdk76nck2vZwE8r29/gF/ZbGa6TOL/xGINdk5Y24nN58S2e2S2emKlTxOtDeRt4K2wmEl0v68F2xq718zs2XIopcC9huw03dg70BaIIwpw1lgoszlBTOzpCk6BPGWrCbbXs7UUNYy1PjrJTHR/w9isQZLCmVuZHN0cmVhbQplbmRvYmoKNDIgMCBvYmoKMTg2ODQKZW5kb2JqCjE0IDAgb2JqCjw8IC9CQm94IFsgLTggLTggOCA4IF0gL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMzEgL1N1YnR5cGUgL0Zvcm0KL1R5cGUgL1hPYmplY3QgPj4Kc3RyZWFtCnicbZBBDoQgDEX3PUUv8ElLRWXr0mu4mUzi/bcDcUBM3TTQvjx+Uf6S8E6lwPgkCUtOs+R605DSukyMGObVsijHoFEt1s51OKjP0HBjdIuxFKbU1uh4o5vpNt6TP/qwWSFGPxwOr4R7FkMmXCkxBoffCy/bw/8Rnl7UwB+ijX5jWkP9CmVuZHN0cmVhbQplbmRvYmoKMTUgMCBvYmoKPDwgL0JCb3ggWyAtOCAtOCA4IDggXSAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDEzMSAvU3VidHlwZSAvRm9ybQovVHlwZSAvWE9iamVjdCA+PgpzdHJlYW0KeJxtkEEOhCAMRfc9RS/wSUtFZevSa7iZTOL9twNxQEzdNNC+PH5R/pLwTqXA+CQJS06z5HrTkNK6TIwY5tWyKMegUS3WznU4qM/QcGN0i7EUptTW6Hijm+k23pM/+rBZIUY/HA6vhHsWQyZcKTEGh98LL9vD/xGeXtTAH6KNfmNaQ/0KZW5kc3RyZWFtCmVuZG9iagoyIDAgb2JqCjw8IC9Db3VudCAxIC9LaWRzIFsgMTEgMCBSIF0gL1R5cGUgL1BhZ2VzID4+CmVuZG9iago0MyAwIG9iago8PCAvQ3JlYXRpb25EYXRlIChEOjIwMjExMjA0MTY1NDE5KzAyJzAwJykKL0NyZWF0b3IgKE1hdHBsb3RsaWIgdjMuNC4zLCBodHRwczovL21hdHBsb3RsaWIub3JnKQovUHJvZHVjZXIgKE1hdHBsb3RsaWIgcGRmIGJhY2tlbmQgdjMuNC4zKSA+PgplbmRvYmoKeHJlZgowIDQ0CjAwMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAwMDAxNiAwMDAwMCBuIAowMDAwMDMyODIyIDAwMDAwIG4gCjAwMDAwMTMwNzAgMDAwMDAgbiAKMDAwMDAxMzExMyAwMDAwMCBuIAowMDAwMDEzMjU1IDAwMDAwIG4gCjAwMDAwMTMyNzYgMDAwMDAgbiAKMDAwMDAxMzI5NyAwMDAwMCBuIAowMDAwMDAwMDY1IDAwMDAwIG4gCjAwMDAwMDA0MDEgMDAwMDAgbiAKMDAwMDAwNTE3NyAwMDAwMCBuIAowMDAwMDAwMjA4IDAwMDAwIG4gCjAwMDAwMDUxNTYgMDAwMDAgbiAKMDAwMDAxMzM3OSAwMDAwMCBuIAowMDAwMDMyMzE0IDAwMDAwIG4gCjAwMDAwMzI1NjggMDAwMDAgbiAKMDAwMDAwNTg4OCAwMDAwMCBuIAowMDAwMDA1NjgwIDAwMDAwIG4gCjAwMDAwMDUzNjQgMDAwMDAgbiAKMDAwMDAwNjk0MSAwMDAwMCBuIAowMDAwMDA1MTk3IDAwMDAwIG4gCjAwMDAwMTE4MTIgMDAwMDAgbiAKMDAwMDAxMTYxMiAwMDAwMCBuIAowMDAwMDExMjExIDAwMDAwIG4gCjAwMDAwMTI4NjUgMDAwMDAgbiAKMDAwMDAwNjk3MyAwMDAwMCBuIAowMDAwMDA3MjgxIDAwMDAwIG4gCjAwMDAwMDc1MTggMDAwMDAgbiAKMDAwMDAwNzg5OCAwMDAwMCBuIAowMDAwMDA4MjIwIDAwMDAwIG4gCjAwMDAwMDg1NDIgMDAwMDAgbiAKMDAwMDAwODY2MSAwMDAwMCBuIAowMDAwMDA4OTkyIDAwMDAwIG4gCjAwMDAwMDkxNjQgMDAwMDAgbiAKMDAwMDAwOTMxOSAwMDAwMCBuIAowMDAwMDA5NjMxIDAwMDAwIG4gCjAwMDAwMDk3NTQgMDAwMDAgbiAKMDAwMDAxMDE2MSAwMDAwMCBuIAowMDAwMDEwMzAzIDAwMDAwIG4gCjAwMDAwMTAzOTMgMDAwMDAgbiAKMDAwMDAxMDU5OSAwMDAwMCBuIAowMDAwMDEwOTIzIDAwMDAwIG4gCjAwMDAwMzIyOTIgMDAwMDAgbiAKMDAwMDAzMjg4MiAwMDAwMCBuIAp0cmFpbGVyCjw8IC9JbmZvIDQzIDAgUiAvUm9vdCAxIDAgUiAvU2l6ZSA0NCA+PgpzdGFydHhyZWYKMzMwMzkKJSVFT0YK\n", "image/svg+xml": ["<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n", "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n", "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n", "<svg height=\"277.314375pt\" version=\"1.1\" viewBox=\"0 0 291.115 277.314375\" width=\"291.115pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n", " <metadata>\n", "  <rdf:RDF xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n", "   <cc:Work>\n", "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n", "    <dc:date>2021-12-04T16:54:18.983966</dc:date>\n", "    <dc:format>image/svg+xml</dc:format>\n", "    <dc:creator>\n", "     <cc:Agent>\n", "      <dc:title>Matplotlib v3.4.3, https://matplotlib.org/</dc:title>\n", "     </cc:Agent>\n", "    </dc:creator>\n", "   </cc:Work>\n", "  </rdf:RDF>\n", " </metadata>\n", " <defs>\n", "  <style type=\"text/css\">*{stroke-linecap:butt;stroke-linejoin:round;}</style>\n", " </defs>\n", " <g id=\"figure_1\">\n", "  <g id=\"patch_1\">\n", "   <path d=\"M 0 277.314375 \n", "L 291.115 277.314375 \n", "L 291.115 0 \n", "L 0 0 \n", "z\n", "\" style=\"fill:none;\"/>\n", "  </g>\n", "  <g id=\"axes_1\">\n", "   <g id=\"patch_2\">\n", "    <path d=\"M 58.523438 239.758125 \n", "L 275.963437 239.758125 \n", "L 275.963437 22.318125 \n", "L 58.523438 22.318125 \n", "z\n", "\" style=\"fill:#ffffff;\"/>\n", "   </g>\n", "   <g clip-path=\"url(#p2bbf45a42c)\">\n", "    <image height=\"218\" id=\"image47246d48e3\" transform=\"scale(1 -1)translate(0 -218)\" width=\"218\" x=\"58.523438\" xlink:href=\"data:image/png;base64,\n", "iVBORw0KGgoAAAANSUhEUgAAANoAAADaCAYAAADAHVzbAABDhElEQVR4nOWda5LrPMycSR+vJ9vLlrLJjPLDpgE0GhdK8sz7VVg1Y4k3USQfNkDL9vxf//v/HOMdjjGHHOswSdwr3sfZcKjyVajq+mbYaWGUl/WS7lFbTs6nOdfxh3ud8Dp03DzGHMd4qLjH+FFxr9fH+FHnPybu3/wZj/Ez/g15/fdOt3E/499cx/9X4j55/u/7+FD5SNo77tOG6dvE2jvHYe5L+uUYjynp0qe273S/2T6OxoKNP44pJ2KOMR426jCJGO8nmL8Qu8h4T4gqTHqN74b+NY807z2QDfXqJwBOlAqylRchk0m6JvL77wPZ8YZAg2kBXHnZ3z9dx1RlPuWx3hiyORAqvdBgvxyfxYeNm0Cg+1D3OR+LofLG51HcK/ahK8PJwibVedjihrD83wZu7xr5PV6FTMqw1dVOJKnDQ4aTjq/8h4XrM8l/UrCMIimFYn8PlS+C8YHAzRgujEOwzN9E8KzK+Pmux4TbdLzMXnjgRIhhyyC5H7ZV5m7gdgH7DcjssTdh7EQBoBRkfkIiYKgmepIHytWCh5SFfAKxBdzeAzcP0fRF9fr0yUTAbL9mJiMfVxm3Ok8eHqyiCjZ+ge/AtspdhW6/fO1dnoXM189MGT8h3Go+vYp5wH7cBJ8fvw3Ntfjv3zg+QP5b58SPE3X8kXIAorneRH+MLxLxn+5PazIO1Td2DPh8r9QM694JHx8NyfZOnr3Ib8Omy1fgeft9N1yHDOuzVgODTLe1A1ltJkYT9QF5OAhWyR7gay2V+oeqNaM6sV0/kDc2c/1CQnyzEyZjT83q+E4gPpo+j2C4E7Zre43XodJBbzDk16wgi5Qthixy1NEMOsBEyv+oqbhe51I8pmgENnLu/Tyezq6Tm7leuXRfmfgNk7HaAOltekQhnjUPm8wg2d0cierx5a6q272h90bE8gtsXFTHHmTMjzCADW4mdnwZC91BTEYESZmNoDwepsz0xLS32QrXzxXM/lm/bMdkrOdcJjydEM3kRz05bPEdO7bT0DvV7VyQ61YqxhQ+M6nvgsyYjVNPtJ6J6Hyz6QHTSmd2D9X7ahrCf58/vvnBIP60af5AW/3mRw6bHTvdX7nJyNXsN8KDX+yvYGNt+VawgO2aihIvaWxQ9XFkmjPIhjkXyLyPIn/c7wJVMZsP+J6ZhQPfAwtNx0l8NQMxqJlbACI15vfJ/TI7RpEpPtxxpnJn5iJ3uNQb1gjF3uYIy8PrjZr3W+rWB2x80u+EzMIUx8OqrnYY+eS0T4B4ZWPb+EcM0TxMPG73+zeu2Q6jtE2brEvFOorszUXrl42g/3B8o1c+ptl87wQPW+qjYXZ/LuXyBvVgW+U9cHdAtwfYyscWnHsh8yZPDFnuj1lFE5CsTyST/h8BzD5ahUrl31Pj/hpRUgNvR80i38z6Zdac9BbCMK/+uPLZspCXWC1+hQcza/4atlWHV9gueJh3H7BJrsMUtweZ9TtsvEyWMWrIOFw/I4LstZmBZt2Pg+xfcLyAzExDW5aDrh8Ry3cXuU82VZ95PxbHTfpWj1FkwnfCuaVeWvnwDcjBY5fdg+0ccLwN0V+nfHy9rqnYhyyO10B3lcz+RZB5M868qezi1w6jVzDtx31ANY9s8euJDwfQqTZy2KyJZ9KUXzZcP+JY8bFk8bumYVU7BuOjSRV7O5HRpePG78Gm69v926171x9bi4fk3YVMTyYNmZ9k3ESMIJONCu6Lkac7JlcwZ16mzzDi9f3Wf6RYj+Hv2W7l+wUNTcah+lePlfeazikbCx3YHtggOf4d2O662StBmyQ8bYwYMp33DGTWrBTIcnMKzS87gfU2fvReFwA0LYTRJoit83CqZh75mrpdfPOjNh11/6MPJmPE5nE8nvfsLmINWS0PNiG6DboDtrPqdkeIfDFJW2EXssgvYE57DzJmZumdPfdMI1ES5pvFCsVg9PVNV8ZCxRcC/gC09WUH6RerYmhC4rhw+M6HjiZGTox7qNhLLJ9Q9lJXYJNr/BZwmZk4hgeKxWWQ4W4XQmZNxQoyP+lwkjqlIuYiUzZ8UNiX0eBy/86ar6hmHriuepn7nzjvuMkYq9n5TRAM1ZLM8h/DmI6RnzHSYzm/DttvqFtmJkq6blMPMp+uFw8OmW8HA0xPZAub84c+kHk1MxseYC7azY9DbYKwfNHGh2r79IsA3guae943s2Mwhu43NlZ2bCo1q+ZZhmY8c/SfDQ/WOH+p3F/DfDbOxtfKdSTNPRf0QFbdtNqwC5n3y2LIpC4FXvDhTWZCMvMMzclYzeJNDg5TtqvI/DIC3tafNRG1yg/oTz0GlZrF5+dC5ZMheI9oIvmV4xxstmmYNwpi5Z6FzpbLuyUaJISqD5mfFBoyN6EMZB4sebUbCkzNnAmn/uw2Po93IE4EUvtnOfh+04MvIHa8LFTxeGC4ByBfY3em5i14ruzzc3tyPMfxmSpzjHG8Y/Vl5pjmAphvfnLK/2HqrILu6C5uvU6PBssvILhIxIsOwhRDps0srmK5L1aZjImCOXi0qah9vchcxM0Qu8uY7yp6I0ssDTAT5zB9x/pSj0NlLsYhnomvmWxncRaiHOoRLOuMShPyzZF9MxLr3+kYXD+ivzr49tSQ2UGWsuchs4BF5pY3xeDhXAWZNQXJe2L0y3I8cPajMX5X0W2EzAqwfZORzxM2vpFPF413XgNLs7q7H+ARLGb+5JOLn3dhOwvc+ZD5Yv6evHk4xp2QxRPvMXDCBspiVAff39I7jD/uqQ+vZnyThNVvdxjzJz4YdNZHZSajBQ/7PlIzG+41J/1s6c7a+TIdl4m4zD5uOuo8r8tas7E2IyUOm2/Th8tzLVT2PEvPfVTur12BjKkB3TbXkx0+V4bflWh2D+fhoJJHqsRX8zDyj9B8wAq+joBBhKajjAUzGa1KxWpmw45zYfPq2V6X3b2q297/LWXL1E3nOatycflcRefYgUw7878FGe7uWSioSaiexrcbIMx8jDZHeFvYm9Nexaw/xs1GbzLysfBzlY3tubC09v7wZJsfS3Xw9RXEPdRbG7vKNtSZrXuE+c6HSsFeeSr/LFuAvmEucr/oh7whjaoWb93jU/psk4Rv9TOTMVMz/jfMeWweajOyUrOuCi3bTG/N+bLratzXOxueaC4iYt6cFGj8sdwWg02a70Hy5qQtuR+6Al+rcASZjT8Dmf1UNL5SpUs+V8a29f/N/CHh+PEqNBPhb5K48o9ZMmocwGTcU7NzgcNmr36o/3tB6lCKZoHxKoSwSVWCj+4mD5vNX6nbCGIi8M6seDlkzES+BzK7kjNFs+9TMdMNt/DVR1jcpkYBJH28KlGz4E1p/eSKvtdh7jfoAzAZo/G5JzAbrVPmfICvmzvg9dUMZFvnsU3AjuEdla1GtV92BH87ddlJIPntvpLk1el3QBb7M3iMQLCHeD108YPC7plG+l4ZmquxeRhvgFi41jj5+DWmXMF0+n7IavFp3/DNVr2fD37qlWfFybmfbP5YqvawcTNuJreHA9YJvTJcxap7uRuybMKib/YyGWu4PiClDwrXoFrTFTY7JsLFn8SP4It2Gb2a2fGp0rqhuy2XL+F5flb2+TIaX1K6TEi+OeLjhzsWe9b7aMdgXZX5bj7fldBTV2Y6noFsfOrhkEVqZietmGrWjNS+FuwiBg8KU/8reViYvW8Xf8+/X0CYeqHS+XGwYxCNeXcuRCbhjuMSx0aBt+7zPprGKoZtuNeh8gwTv0rlmyS2eTVweyGujQPmy+xAZuprPiTsAWP+k/fPMrVKf27J7DCiokXAH2M6JatMxtUfdsNjV83iad6d/tYfw9IcuGuBtewxVd1sMtmike8Wm172XMplq5W26fcDqgi/RgSZLjPNBKgh+wDV/IUXv7MIGxA7zy8O/aAwV6nPe20k31TnVj2Xb8gV2KsYg4+d43jYMURX5myoFu4dMzErV9Xx2nWcc4yDKdfriMWty2mFipRN1yHNHKG6rfI6b9UJnQHpqljlm90FGVUwChn7dRZ4Y5n+UicxDam5yExH3m455kodmYy6nz99TNTsmrmoZ6C+0isNx5+FezZE/JWeFihrPL6KoEFp/bXzsK2yq1Sn6ecC62AGFbZED75VN+uTfc43IWOmG4MMj5kZGP6eWWu7X+DiO51cwXZMxh01q49XSfZGURwscKzO3dBB9xWeyxMb8xjHwTy0HLbxObbY4E1hTARct+FViGrqqRieJwq20hLImLnl4Eogc++PATz+SQ//GJV8wxUzLS1sS2ntz/XGf95ktP2jF6RPXydqds1c5KqmA7NpMObqHMRrmM+jnYHNmpZWzcbI1c3fkO7wvXWnyp35jZGKIWwdyGqTi5tnHDL/XYzRNwVHXyEX+3hotgL08Nkytkuaq5s3GWs1Oxe8qumlfr+uM9df143CEzG6AzY9dT18om4xcN3m58GWizdmJJ5BpgGC87nOJS7aLLAT1W84zEEmO91JxKfsdXy2w8hVLPpKhFyhvMkofcxNRhOn1GxAWjx+Nhdfvn3pu55ZzGyjOq/b3n9XAbBpPwzVi8GW+W1jYPeMoYGLGroTGGAYP128XYGd/wXx8lzePmSxX8Y3QJwagRmYfu8H+Yo4v/ni/TJ+L1axvMlo1cwuXpGl0VUemTVR4P4as3X2bKTqmlX5Y5g3rC1w741IFWe17Bxs0gCrZJKL3WLWLdW64tM7KtaDTOeJTKoYMpzYftPDApf9oATfzmdfoBop56fN5eaH/dPjYOOYid4Zt3g0cTFGVRsDZ1IUzmlbB6oo31Nz5xCacxzHK83vSXrY7P6lLqeb49VNGqeb6PPkoTI7mH9wDbKVFm1+RLuMcyjz0ZmATMWUeeh+fVO/Mr/Mq1m0IFgl8/6mVXCEy/q5XM0ys5GPaDb2FjZbL16hi1bdshoqnXelP+2mJ4dtHHJTGWzD5Xk1ZQ47zW2Kb76Hrhd4Zx4u3psrXchwAvrJGENGFMU9WqV2CRlw5Et1YpPx561UP6YtXs0yxRqqX7Q/ps1GNCf5eFzxj/Qo4TW4LZR5/DuBLdd5/ij9KU3TRqD2y470De0KtvHJb/1A3SxEIRoUrndRYIBJLbGK6fg9yGLoyNfE0Uer4l1CfOoj/gzaT/B4Vbzp8fjc3+u+apMRPZCemuHYxdMyC9xXy52P/dBpmZ9hrPwrz1Mbdtbws6biGMx4rGHzftu6lgQBThrGwhm4pFxHxSTeKFsTMrYFjmaj38YvAPtAWZmEPx9gmGKxcp+2l88xHqb/ELhKzT5xJ+Y/9804bCOMvSfsgIXxT0nOdGoOtu0fwbYaxWHTTUbgBuTyDe/fuq8PVQxho+bjSchkMgNA4VMfqFbWBMxNRsyrr639MvLjE/ST0txk1MDJCNVqtuPV+BGM1Kv3SNW5UGttBJZNk/SnBsPi0YVNa51FSBuPglsNnG3iXkeyMl0Vc2pGfwiw55f1IPMKZvw091NK/FlH/zQ+qhlXQrynrsnIznl/nwtWxbSd5PPZnBJbhz3DdRcsjH/a6MooZLAxRctMyQw4aWi/u6rbtfVP02XWHFpx8kZ0DllmIvYgY092LJ/Mgpd+6Q58I5YzJx1c8rvW0nau1HaX8Rh2erLpx0ZuZzT5ktt50qPO0QsZVDZd8kR3uGae8dHugG2QUuuC83NxC5xt6PnuYibKNOl9FYtAY6v+NchAmQCy7GHh/G0BVC/45HRiMuJfZCJaNbOqg2Zj5Z9xX0wHbRvpmHPBg9GBSrcjqpcr3tMmX4Mt2rscn5KvI42XYL5vyUciHqsZnoMvMQE6B1OuaBaytYXPIcPjDLLsyQ+rfLK7yE3Ud3vV5gdbOEYIoJ5GfMLvTH6ZVzqIqtmdRH8FmysLvVbxms6BZdp5wLOOr+Q92MaY4/2u9ieP+G1oSkozNNpj2JXKYxeFeHWbLs2rmQC24jRMsdkYmYz809EcsurxKYTMvjG9lBIhY/4XqG/za+LQZJS+lniMu7oxIXNkqpgIuJHGVtfhob6LGCzblsNUMpePpjEQuDqw6SdIclNyDD39tUlpmyhNj2447lo0X6SWUM2oqcj8lbEBWWwuashwl3G9V8bfI4v8sug61eYHMxFjk9Eq2l3ekFc1C5X12a5hrEPfejoLFtYrn0czCNwD29IpfEqEAcduqjOQ3vRggDEzUa6oJ1AEGv7aphzjByYzyCw07BPSVtWCHUn1Xhlrk/zBm9REzdBMtH2AfpgfF3bMxqQaRZzukUtxZpusW5JBZcsCoAAX1q/LPL1/tgqdge1djzElUd30LVny9wdI36auyVryXRXjGyDWP9M7d7gJUkMWP1qFJqJXNGYuSt3RA8sassyk1HCtfmMbIDJ9+CZIFI5jfOYHbteL0+LHny2j0fV2Z08N1ut6n7hCtVid696e1vVkH26pYXOmZ6Fu0gjN/J6VbycEj/OASVdohdN+SOSLeVMRdxvxMat6A0TABJiCbXrcRczNw9e9PMZh7jEGi/mkuZrZsDfNZcboOLR37riSrbuuMwLrlcauHcGlz932vtWeHmzyOoZWt7UruRqjgdPIaY9s1+7HW8SJM4YFLAMtnpA9yPjHXbhvth6tyn9DGv0+/HJT1s4fn+a++m7Q+7QhXsB439dBq5osvCSfit1Zfnvamt9FT7VsOb7tJnFv0HSS7gAOVw4baN17V1I+12aBG58aeQfpBmcrKaYhYKs8wuV9sshs3IGMK5r7jbHil1vwZ5jih5A5XJ/zwGRcpiEHD9WML2D94H2wCrYVrm67VEtDB6xXHFOtqIztIaVo1j/bhY2Vt6bk++IAHDaTDUUUIrjG8IDl5mKsZpmKybcIWyVhiobfNuWf+sDdRgBpquu9r52Ztt4EFLi82Wh9sQEjYQHDwIGTpdabiKJqenSuKSVvSQWV5OuAJdeI9R3h0nnJkyFyXMGG8Xim1SoCTjdq32yEWzUmoj22KhbBFSuavCrzjUBmNzM8ZBawSNXUcfAMo9/tXG1R9xCajFy9PIC6D894RqsGPY9kwiNwOn99vcLnCqDKaq5UC+P88hSbk08dbZ+xZ9IeGomunK8DgXvXd+zZ4TpIPXb9mNAdsaoxuIZTB5zYaCou2Pz7YwDh1EDyN6SN2QjPMFqo+SbM57VpMnpFq4y1c8BZF8ECN8Ygj2jVyy6HyZbPsuyB9YrPesraUPY6oeloj6s46Uh8jFiXEWgZdLbRceBmDaqXvopXslzFLGx+G3+WbxazjQ/9/lj8hjSHDDdA+M/ZipKxe+Imo1U0NBV9X1f+lKCUfRp6QkwFTRYiKLBtmeHbAwvrif00BHL6D356SHRlzO7mMOLntO0N6bqNBe+OspU1nxQcMG862vNo88Nuelifib9PhtB4yPhjWAwyDp1VVFFaNBeHOmYmI/Y/68s6+DkjMwBHTsrsuAtdFhlUeF1eZw6W5OeqFZU7RrDraI8ZgJhWKSIej08pPziSe5DjYeIjuKyJuF5zJYthW7t+aLohbB6yn4+J6X/M3ZqZn02REjJ9/f7fUPdpe3pAfN7nVbCLsB4BPv3Pe347Rm4MFsZxc5Dljc1IlseZjl6BvJkYp0Um5vLo9E34Lhb4fPBrjb2tDDSuZBa0B8TjpofdfMhNxx3I/EPCfahQzfARq2Hy616W+4z6sxeYSxBt2dursfFkoYLJ5vN5O3fYVS1mErLrszzpriM2RkPJVq7Kj+NluemIIV5jUNkywGIVC81GsulR+Wc9yA4o4yGWtjE/Ddo/MR7NRvTRhuunqJ87gfnftj7Mf04zo+WWp0dg2XK5oVzBVSmdA20V8tv6MUArf/YVPvqmmG/G/D4fvGnD4NK3ZldtDhpshw9QCdj0qBXNfpmOe9OZQrlUE7+lKoKLwDYRLFQ2HbwfthP0ONl5wnx3Ac7G1tfwwZeM8rF4rlpSbwQKlt2BS+d/YlRuJp6Lsw+RxqZjVA8bnNg/ywDzk9CbjW/1Ms8gMr+MmYyy8fHPpR80TtQvNxl3fDLpg1rNhum7TrBjw2CzYxZNcLboxiGD7wxYGFf7aT24ePqsTEcb5zc+4rz89hE6aWRuAMT+GfPJ1qsHTvtgsUro79Lgkz+HrNqN1Io23aNbCBi+Twbtdx990VNi/dl+9YsUpu8FhG0MP/rMZ2P1xCGGSuK7qvWK+y5cNp2ajitxQpzPV0FWwbeugAYG5pCm2zgL2o7JOMcYTjHMl9VkvhiJo1v4GWS4yfLzua5AHvuOFrJYvayaR/0ax+meF1uEv0dml0uf3gtMcVh7MlAj3yyeTV1TslIusRFsXc85x+DvFlYKx0IN2Rz6hg9aUpqI4YDJ4n2zGLDCbITv0ahhU69KlfqQef8PAXM+o74X45eJetlpmqlZbprb4Mc9gk2XOe+PsTw1WKwNtb/l83kgz8Gl016Ktn465pPxMP9tyNQM87yOM8XCUnl65JdxRWNmozMZJ5vQHLBpznPI/DOODLLqDejub5dZk9EqWux/5caYD1rVdO8z036vZiwTLbMrxKqXqRaeR3DVZmMPLl2vmI4ENhtiYCIouf71wdP1rGbrc1QvrmoWPOaLMeWot/EZMGwL3+8uZnXytnj19ZB5wIbqt3gq+GDNROtCIGy6dlsDy7dCrXV+JvF4dieRauH5WdByuHw+76O9yag0ip3ruG75QfJhuoTYbGRgobIZRSO+GE7oaoeR/wxusZ2/sbuo304w91D4ZTg1Kr8sDjJiFraFW1ZnXysjoHj5Giwpl6sWnmdqpI85UHE+r2jvwnO+u/FQcSPfrMg8vMhr0/mqUJmM6zUzGWsF89DVgHHlon4ZfKaM72JmkMV+mQSr6NhnVX9zJWPnemLtGoiVWWjzetPQ1xGBVasW5slUC9N9r2bq9sTswtfLlMyNyOnK1h6bhGyIInOx9svW7YHJWELW2ACZDA6matnuojUTPWTe75pjpD9GgT4a9g32Ozv3I/XqQV0Hs0h2dDIe71gBY7B4uY4JqM/z/LkBzvJF6vZ51lFUS3W2s++suiFcc6zBQQzPqhkzfQ76GiraZzdRQacmeG4yVv5Y45XC6f1AC9kY+MTKCAHzw5+bi33/TM69ob+rY9iiqPwuWB4Glm/HH+sqXFPd3pmejzcUmfnH1C0yGbEGBFMakZWyufpqtkwsnJSZmv24+GrzAiE5Bxnxvxxw1i9bfaEXkwF9YPuph4M3Cw+XvtL6+lVfPTJmd8B6pfdNQp7/O3ARRVuZ7S1qtN6/sOt6L1KxDMqOkmG+CbdEIZsAXDCBK5PxA1L6M0t9MzOCmbXF/X1+D0BMYt0ntYKRQRsIF6bJ6Nc19UIEla2TAfOKj1QLzyu4KhBz8LhZGMGly8JmiFQedul8pyY9LrD5x4rHYA9h6bLYxbrZr2O9encAW3FMxUL/jP6+NIfO5QMVZFBHkBngyt9n031jp0o/zPeYIAZ6JLIRsyEDyreuBxa2IFa8Hbiu+VwZXKwe+sFPpkBOnRRwmYoxc5HDbUNkLq5X/htmMXA9k/ENW7gN/2PKU9gCyNAvLP+m98O8IcWGGFXfA7Tyy/hEsO2MmA++TB+sV57KJPTnu8qljzlckHagTVXXo0zHWMGU8ThEoRSMU+U7bE14bBvvr2PP/bpVfdNwBZlXMf+bYRlMKWxUATnUpckICq17bap79n131rBbtfjN+o6SZUDxdK6R3CR8xVU+Vpy/B1dmPi649LVje6JpOkZg8TxwPJdhKLK6QEaFZMF1nwJLN3vC7WZmYqRqDoJ5lIAhSKhiWJbtbIZwrbbPdZ966Ox9siHtBlQtuxDiyGTvklX+ls/vzUFbT2wSfguuBBgKl85Tw6Vn6XPC7TnFGgJTbB6SbxVR46C/aDwO3rqf0E0erlzJZGIv+Ii5OPONCr4Joo4nK5OrGD/XgHEzWI8TmoXYV9izFq78geBJjqK8PESmoK0zA+uKit2rXFjmjKIp0IJ30sjx8c5twVtNFP2aplx35UX/QiadPo4no5/EWuEUKJOrTrybaOPnQCUDAAHeGDDV9snuxQ6b7cvzpqIsfb6Ofq35yPqlMzYHV5ui80yN8FrR5KdpN5iFnXLPNXg1WPZjnx4o1K3qWyMwoN/h1QxNx0rNqE9kPtRZg8ag8z9Ny994rgHrQWYVTfoJX+OgbQ7d47Y0q6cLXQ6VpEWgYB37JiGmzyDfNDvm34RLv37esF7NkeP6S+ZeFem1UZc+p2Ry7MFCs3GYCVyAVnwcJo7rqJgvHynrFcjiadzpYw5bVGMWogUUTUFfJyqS5I3MyK5q6bzUXDx4Hpb/HFx5+acuisBZ9Ypfr6lZrWQ9RRNfzMTBM46t7X2Ie4yfUMUiBeObMdC+ue4tg2yQ/uip2eHSxfqwsbYMT/F1Yz4Pam4eRmYkg6vKuwNXroA78PXS36DlkNm46DeqvYZ11cxbyB2zMVaIzwQHBbNAZKpE3swmKtYBjG12aMjWMESQ4RBGOsF7VZDyuGQmng8MIJ7GwGL5M/MQJ61NLyE8BZeOP5NelyG7jjacUbLuh9gxlwYMbwsn6xj+09JzDPrjDhlMaXpDxew1Bk1zoFHIvEr5aWvTVz/yYGHz5XyorsbScRQ5WBLHJ3yVfg0uvmzpPBUoueJ16jQ+mjQdofqukulzBKuraA/wc3LYsg9c6vrifNk1uOK+zye7H76wSL/01cwqmF0685Iewwi8nmqxvH24XvH7ZiHLfwaeOn1B1ivz1AMaKZdXMhzQHSXTzRifazMTycLmTSz8SD9uisQgJaagUzH+npgxUylwVn3vgaxWpVcJhK1Xhgeuq124vOLtwaWPES6Wvw+Xjr8GXwfI9BGsGDJRtWHSaiVb9dpmy3GpYvRBWz3JQekobMR0TH06rmCZgqKpiGaiXjz8sET92evfvZwYemDZa/wlXH044jxMnbpw9pTNbIbg6zBxzFAcgFwVuJLpY7rim89jVRO8A5t947pWwLhuCpY+n7bL8d740GG/+L6zvXgeqwgs3gJ9TXvVw+U7C9fYfEIDy++CYqG4p4wvm5qOqF3on9lL1yGaNH7yHerBWrkdZzoGMGSwfUBzgOUmpz8PwBpiTESmYmQmSx8x/eBBTPZqHGqbo1ItLH8WLn18Bq5Onj4oEr8D127Z52OqzZD3AVO0dfzCbQ1uLyCMOMk0WCt/aDqSyc1Bi/Pg841RPak5mIDPdhX9fe9AFqnZSvUlOgEhiuLvMAt1Oztw0fyn4NpLv242+vs7BjyCtXwJo2LHuhR2WK1kbkimTmuYjDBZ+WQP3qjG89BE5KZg64kOOB6jo2JsSM5DthNqsLBVvAUZPHX6vOG9rnvhWuk7atiBS9frTEcpNhV8clkNYRQiBZPjfJXHSZzBxlTHxCWbHLkqfg8wdv+6byT0zfIo9GrO4cI6ariIyv05XLx+X+aE6gVw6bjn4x2NAEUfbYk+ocRyTpPuV+9sYkaTe4xcbfQOYgZkX7F6gMX3E02nHmS7atZVLR8Xw9UzC/+7cJ0rU8Cq4OrUaxRNdxQfCtvRPI9Pi00nibeg5bB5MN5KFn73YbQ1z0zHqA3qfNp241DcARhP86HKza+q07lZ2IML02O4WP7vwFWozy/CpetVT4a8zEU2XIc6kknEAqqYvrRvlgZuQrfUgK1Jv6dQrac3IuACwPomIvYH7+kKsD5ckieCBa+aqQ3P6+HSx/tw6fhr8HUgWOkduHbqxbinTJShCtnAhpOtj76b5ZiBxtWsYbaVpmFWxw5k+no2brh7YsPUAaxvIt6pXhH+DMgQxMI05ODtg3YGvh1wXVzwBTxVvdm1nrxrXxlXHAs41JlpFJtXTB0yuDrAcHMwO9dtMnFkg8O+2j7L+sCGPly2hC9VqZdtYa1eOv/31Ws/fR+gJsxHXW8/zd/DezOEhSh+vKu0eWIzsQLLxlfKta9OVXm4/qzbae/N37cc2/7cAcuWXCGC7A7A9CQhaU31YjNgb2Lq+KYCZQBV9R45QFj/2UUg+AJVHzITiA1XFywd34NrFzTbDSzPGLl6edMQ+4HBdR4sW8sKfch2AGN59STEOu9Upwq0vExfUVwcUa8etOfb4T6PFgUEslIwfE2BaykXBwYfn7LXiKDTUA9TJ97PX8DlA8KAreBXP1yePcBsnh3zsAtQF84euGncRdPwajsS05GtgyvEqzyDzUEyiZIlcOXwZOcEztse8rX3m/VbHmKbAq94XsXOAYbpV8wqXqYLbKYorD57T/eDvAttqmh+YmWru56kdCKDaknec0qWw2avH32i2XddvHAwqPaAisIx/Lcs2ivYq9irdgGzEwHOC8D2AdLpGSQXAMpA33y/62r7w3aoOKdoOOQVVBhvwZLb9RPcK08G3g5o+vpesXw8uzcEi6HA1MyCV+eIQgYeh8zbH7laXAUsX8F3J3Zvsid1/oHftdP+J040OdbF7DE1s1KTjICwAQ6HE46nz2PbuuJZV+Haz/pC4nx8FNgDa+xDLQepMVMqP6D6+H7A2CTWZfoKdWe5zO/aXxzuh0uX/Tzr+EoaKmmoODIxS18nA6NKb8JGVYspbjR1uoBJv8RpPMd0aS+oPGyYz9dXAVROjBOAnVOo3QmaT2y2UHwDoHOqHF9Lpz+tyugknX24DYwIsEzJuKr1YVvtyNSRtZGBhfGDnkuo4dL5DpVmodOwxQFrYZDtqNg074NdA6wzSTuql4Pn0lp+11mAenB1lDcq83yMn/epzqrBkvjYHOOTvqVKBVz8S0azOLiPaOEw09ifSzwPHi5e0xz2y2QzJcN6sT47kB1FYir2PbU5V188eXf9rvsU2bdlpz7WV+ZZR61ar9fzcNVA5XDpLvD17qlYBBiqGY/3weuR7ux1LbnSCy5eSkPo6+qtluFkDVXMqwqbdG0g7obSqReb7P20WoE6qpf3Gy17SPzz4UzCCi5pWh5XwKSOP9dK3rgeEBerrFcxfqzDQeJ4eOGCue3XOwhwCJuvqRcyMyZIO+qVfQeaPcBOQhn4Xt9SY+y3XKWa13zDheU/isZuMVaQCBqdLuf6IywOyOJ3moerm7fNtl3f5goeJp0vD5ITy4h+LdyGAszCFsF10Nfp4luKFJqKvUnbg45f48zk730MpaNQHbhZuVolq7LD3AdPfz7obdVKVStYomYOrhrgWMHwVd+mByOe5himyeMRme84jZTk1rD5kmtCxdeMVtFoOcFJexcw1cT1o+dhoXVsmIcxeH2447T8/lJFO3zZqO7PrmMMWQ1Snu91/jJRd+AcLj/eVj71VvBKNlSaDTNJwyuIli3g8MoLKHzNgq3DLhMHPd6DbE+9YrXYVcF+OzfqInGdNF5nBaFVrwiqqPzzMbBrmbok6lRBVpiGlZrpdnlVY90zoHt0XBX4N6KIWulUq2UeKK1kXtX8Nfyx1mG84+7kPQvaLkhl/BHXb8vtKPCuemUw2372cBXpYfzrVSkam9y1UoXndGOjp2Z4fQGM3cKuqbjSeYpXHRuzoMMfYJyfNPvLqHGY7jVaPtjEiCDzS1cMjz3/EpBHXr42S7+hbE0wj7oMS2d5nq/voUfVqABLAJr734m4o2a8S/H2eNBA5hBIrfp3Tw9Th9Ytq2ha1+IHhnk7mLKtYzfJDztiXs3wfAeavmJdAWwHokyhaih76hXBxevO4dJ5numOYACU+0ap0jyUz41V1xif42iNwjXfm4kWNj7NZ5oD9WjplFzNKpdo2Ri1msVw8TvDwexChktZFMfP9wGrnt7IymcwXVGvDlzD9GeQnkCUxa9jajruAcagYkrWhwvjeBdZM1F/C3JHr0ww2T1kci54edhYbn7sAy4Va5VeJTUoGBerE1MormJnYUQFi6Hpxeu+yPOORloNV5F+ES6d59n9fnn27b8UPle+gpiZj2PY4VZTa+KtSPDTlYcDpv2cegIfnyTRKa9cCJvOLVhlhixrl1UzHjc/b0ZzePK4HJwcuOHSrgF2BhQGgY9L0gxcFVTX4NJ5wEeLgZFvACbgZWVc3QysCC4ByyoZqprcXh0EB4z/1LEuehw23kAmsEFpOOfA6eXD38l06Z/JeajjkQG12ushw1HI4nLA2DXGGJj3RPzqoxjGvnqd3dTA9DyvjXegxUBYMPBbgPPvUYzMxxgsD5dex9mt6sDiMHXlQMSmyWV0ac73j3zMsdQLYRNwRcVyM9G2S/+xuzJQHTp9jjH0JOxAVuXhcQZCssnRVard+Crv6qNQ9Q6fVpWN0zsAxmnOdHSAmG8CHiofg/KVp2OOWsje5+7zZbrpeDygG+KwDDsOmX7Hy6rXGMcYAFuQC2r0wcZFSqb/OlCdhYzDkipb8j5YVO48YHJ/MWAJAGpj46x6YfrZPOuY+GhvkCZTKX7cMxlj9bLTg93+gFvbMRkXQmguIi4v5PxGh4Vt5QLtU+erTZG+xm30d6YAOEDdCHCrfAeyECgGa9NM5PCdA2/1YgWjyU+f2DijXnG6z4t5eNpn1zH+qSOuYpWPFoP4vvzUw8Yg65iMKzDYZpG2kOBaJRi9IXrDNoY1JKsrVa3UbWLLiD+f4TmHikN2pGU6KsahQYDj8oPmW/1UASZw+TRerg9VnLcHHstnFI39Gkv9M7OxcjnYph0KNCHt7Ue37oOOP8jROsMnOSKwuL4NKKFL9YNVpGH+holjapYrGgcmhykyH3dUrFK2XfB4XA0YB6gCrVI/zJulRaCO8fz8cHqyoxjBVm2ImDjyxL6HjN023s5QeX2wIEQpuamIsH2ur1TNQnYMaZE+Zm2o7iLOU0PUVbUINA0Zg6EbF0HEoJRzn9/mHWNcNA97KnVG4XQay/dckGU/1Nf78fQAuPAjMcxclGZb+KTpNq4T0LBD4Oyxhk132/HJO12N14I2l3I106EDUe/81Qa5XiMPBSaHiMGZKZKbGa0vQj0LVa5QdVqd76kh6wDVNh8nxiFYDCqmbCvgeRUEB9EYQUiAGsPqmNc5Kf36J6q2urejZtguKbF6J8rrAZyQpiHYg0yPhEAW5Vt9mMcNlYbnFiTM0wdsT7Uq0DJw9iFkpubzkQDWMxnhIWLj51nYZEhquHSz2XkcIr1hSoYmpIVtfM6HiuNXG59c/eDzAjCHhpArn07D3m5Dd3CwGFQIiIO1BRiWgfhD8jMYK/Owgs/XEwPIoOH5rBQgiO8nQ2pVK39sPQFM3y6e+1e8TR2yaZwB9opH9fKbIlbjpCVWtebnmS0Pn8aZ3cG6U2ylmE6rjIbFguVh0ROXAVVBFisdVzEPFQMqBszW+YlPAfueetWKdQ4unfd5BrD9p/dtt07VPK5seAudwCBEmFY+DZY3KC1AK26YMhY+dl61EntI0peaiVpZqEw+Ah2HkcRRyDh0tYpxMDhgUOeB8V3APAAVdFG5PM+5fDpdPYK1B5jsVEZwZYCtNHbb2MS9sPRIztaVNUp+696XkNo4WDsBB2O4c91jzEyU+2A97aEbQfweZJViMZhyRTLnJwCL1ek+9aqg8SoX513HTtGWKVnCRt8OwGHwU2OYV9ske4v2NuqggVloRNsb8Gb0+2wMvglSq1cNYXYnDsBDL0WxmskxUzAWj5BxaKq08TlmcNVK9n3A7gINgYnriPKO8RpPpWg/fT8t/RQ1gwtB40LMlU0HfVvovenb1Fq18tpjhp7UYnWxakkV2B2x8gfEd9SM+3NsZMYmZFG6bkumbD7f+Fx/jBGWyeCr4KjhQaVheWz+MTL1yuDSeZ8RZCFsG+aiHEuTIlWTY3bOwmGOpqnt+JwzlJZyMUXT1z2gdn21+WGX70byYJeYMQb0EvaeXDFWMz1RPFiSb94AGYMjV7vPeaBiEXQ2/bqCYXyWp8qX5gXAVqA+WgeyTNH8mipNQ8D8+qJDPoUFp3W+6l5X4bq1OscC5TGNINsJqFxW072BfBw6b7x0RT2/6nPgGJ8sgyxLZ0Dlave6Jw9UDlgv7QxgtSrF6sXqe92f5LVlJX+4vV8DVinaukyuZHra2luqg85vIZIztgmCKqZR7PtlEqcN0ywcwTHGHUHeWtmypY8BWJ1HoI2BQO2pGIImx/h6HbDa7GPqZft9T71QHo7RUbQUsnq9HQPB8023sNluyYPkthr2OrIYDZMj+yIdKWE76xvhMK/Sc9wA7ygbAfDwo3YeshgcA/aGinHAfNx5wBqqlORzeRuAYd1k17GGLHqYeAw//BJnm6Kb6NecXsAfRGLA8UeF5Tr4eTLbEt2FmMYULmsrP161jzHBbPTHPWUDWA4/SndAFoIGKobqF4OWwVcD1E3XaSyfLaPyteHCul/nsaK1IPNr64C4MTRo3nSU46i5I0hdeqSVSm5co8W+emCY82HqyUzGrpmIbeXpS714OYRIl/FgeGXTdXDAImWrIUOABOoV31cxD0kXoL7C+TxZvlfaGOfUS4KU4Yo2bVwXMg4Xqhs2DZudB4RSoyKKplHCr4Njz4p4mFi87j5s8Z6fpntnwGtuKvIygbId8WjdpWzDXIup3bpfD1IG0V2A5eoVgHhRvXAeHIMpWvqRGYyTbotAQ8AyuHwT42BhsVo1xjFwt1HHDYjPjU8ESNDeU7ap/nh6BdJqVa5kywzlaX1lm5/fgk2BO1ge6c0MLgZPDtb9gN2jXhwuHayizR/nf+XQ2e6JVSwGjMFVmYy6nAYOfy7JH4u+6VYIrNqIZOYiV7g8CFwMoM/dHDxN9+AxJk1nyhb9nQEPTUENmU+/CpqPG1BuuHxxfN88vK5euk7MJ4o20RfzQ1J+D8iw3estb9Y83jQMHgupK4NNq49XIatyWdDX+cTNYTYwzgQ2oHbZYnDl5uXruz4KxRsZYHoEAzU7NIg+XeIG1GHPR5If06QPrgN2xffCedCB8vXlPO47G3cgq1SMAxY3jYcDjtAv07H2ASudR86mqU0rFVMtC2wn+AnB022a7rVBXm06UzN+jBBJXbmysfIT/DFdF4NP+iFTN+wzDlAMmM+LANxrHmbqZc9fcc+H+8ara5BVKuabz4Ju5nT5DvXfv0+m4Roj/pF2Hu9bwsA7H6I7x1VSt4opW1fNjmEnHo6krs9DRs4bkGVwVfFj+NmTQamP/w4wDpc+f2L3xl+MyiGLFQynR6Roqrl0rh1gnnk18hsfK1WblDavrh/f4rbtOgtZXS5aBbFHbdqk+aNjbkJqRYzjnaopyIapuwdZDNeuikXmIknbACwDhY1FB7CVbnYdc8AqyIRt3436duD2GvNY5zk+PadNOQ+bNSEPl99DGFwb0nfMxyjwQdPnVr2w3GHyddXMwoogDVUfVbXDK0sFWV/N9lSMLd1U6Q7Mf0bBrgAmZY2idX64wkKm023VHLZ3ngQuq1Yk/Z1Bvsy0N/EZMPY6d6oZD9gTqyWo2DY/UzFvSq5XH+dHcZj8OLIk7cA2XIOMq5mPk3uPYbNt/m3AuHoxe82ZjrWKnYWMA+abqm+J3/gY473jp2GIVE3frjclOzBlMGMay4cDxfLFg88H94By0U7jymNhZKPJ0nS7PVgZZLmC+XhzL0FcDNjqA9tHUT7WnyPMX5fBulnZZwaTNy4OUyVC5sDSzVfjFsGVhVVG17q213NwBMBVD0LrQw7gHebjapmutRrwg+bpKBuzRfhoHiZ9Kr+MQWbbfhaysypmZtvB87h80OZR5M/KYDnMq+OedrcxMhPZWtmD7PU14FEzsUl4EywvgvKCYn6O1mv+a5wWppWzvv5OsOWZXd9VNm8iodJpoBhYKw+Cx5fSZTIyFUPbpQ9ZDFdmRsbgjeEBC/N9UndNSl8Gy2H9vnz0IxcUqg5kAFuiYlEzbR5uamnYmAnJVC3CWZfK8u6pWG2O2hXPq4NO04PHTaZc2Y4xYPSGqcMvq8NsflhQNCyVCWnPRxFv7gn6wqUfNp7l0X3xDcAi9WL5Cx8thixek97VT95UznsURItwojMTMFI1SbMQVfDcZSIOqCeqs2PurPPMbBwm3ZqLK83bKx5Ku2DofDuQVcDVcbo/un7YnYDpfDZvNJs9oAFotZLp6qZqfmQq8tvJwwFH+B7YByinanUQiPbKnQtZ/bEJghMk6k07CpFZOdwI6j8z4qFfNuixbi/aO5nKcVOxUjFcXCIV2weMm4N9wLC8rgNAk3XLn2ujwQ5nBlkutHnwinIAbBEkuXIxNczri9pzNvgVD1thj20vmvTEbIzg8yPtr8kB8ybkgHgLNVMrjNf9kSiagyw2E5k5GeWVe9D3Hy9+VVl2rTE+u44IWKRi3mCpIesAhjrlz5iSmXxzmB3ICIpdWBjsEn9NCWN9jwZa0vgEW6+5WbmO7YiJmtnATMhM3ay5OFwaNwsrU9Ev60y5ODAxKFdMxAgwrmrPqbqLW+1SXA8fMxcj4yYSVB5WWjSJUa2yb63S6bla1e2RGqO2VKXpwBxepWzeqCfZpIrMRrZ0+lFe+a0jwRUtVjcLwAFp1i7qQNZTMb7813kxD+az574sK8/KPHXXSxYpqodIX+ocZBF4PsgvTs/P/0rVqsDVKYa0B+dZgHlNfMDl3E7ec8qm6zK2DFzQQ2Xbx9VNx+G12PFVyDIV86B4s/C7gK2gTMfe5ofp5pl1vW/CznSk5uEn7goA94FxPVRa75cvny/y9yL4IuBWXX7pxfqyDZFh4rp+2h5kXMXwns+oGAOMz9zYj2P1jvG0hoJVMV2l3/yImp3dwl54gVWDscxD/6TI3rVWQJjP1Ne7UhaH7SHxhx9Qb3/gMumX0VddfIJVUK1jr7bcVMxMyB3Iqg2P31KxCrAVnry7pDlofHy6Z9qm3A2ZDWzS7ylTBsuOWXovdDwwO6FWNknzqzwDbx1bAFd6NIEQKjQZpU6dt1K0d9wFyLhyVcDJvd0BWGRZHAPeR1tFuF+mijYhu8NA25/0e0p0Dzh3qd+OsuWr+AjiIuDWOdsoQXNRQ6nbgop3P2RXTEWuYruAYXl7HtdFtvex+dZkxM2PXcjipkW+htR5hzZ2ww4wPu/1tobmovqzLbDl2MRi3vcY3mzMgfKmJDMZYwXF4zOQ5b5Vblb6/JgP643KYx0rjY2d89HsulbdOmsShyxjX5djpb9lrsX19kD5DTPSX1FeM9MH47hnjX5brHI2zavZUPm8wiXqdgNkfRXrmol98zyrR5d/sqZpyIyiubnXhaw/HdfGRp7nXnPvm2p5pa1sVeV57BWZalTlrIJpZ0LqXfliNZPrtzZFTkJWmYpROt53pWJXAMPrwufRcH1Q1VG/7F7I9HXvVYrrINVt6l7jTqhxouR+CL76vGiWctORXd+rHjMnlaId2KbzkOXKta9idwK2whO38lcVdv3hJmM1Xc5AJu2wtb8m+rlJyiDpwBzl2Y3/doivmXnK00x4Vg9Xr1zNhkrXdbBrVLNsH7LaVOyrWOzs8PfVbDmMf+qikV9mP1fGLu8vfR6ynfA9k+/e8Grnd0FkkwqPJx0pibfmIaufBaZmHZNRx3u4Bk1naf7Y5/2WilWArfD0gOUmo8/zV5D998NfKZxcPT6PgNR+Wl4nrx99u87xKs8A8vX7PLuQ7atYZiZW/u8r/aHXr32Tka+Pd4R76sm8kW6ZPNgNonwqXb3WmVC1KPfX9BvS3Gzkd3BGzaSsvr5O/wvI2D1qYnj8SpM6H/ZUN53tMnbCf0nNfs+srBacvCXX+6xSi87VdpeJVTf313pq1jUZfx8yZmbrOmw98WLxCg9e1JuMPTWLh2eSv/8JITKW9tvfmcY9T+HKlU18u+r+suEnb65mmYL5a3Gf6juQ+RCpmG0f93M/2/urWLbXkoVs4n0Pqv8mruibMV+tWmx+27+rl4EMNnucm6P+2E/gyGT0+XuQRXDUb1bz9tUKhuHhs0Zqlm+AdIenm/ad8BtX3Juyc+6amb8fjsZfNREjNTtjMjIV6kBm2/p7kB1jjscZNdsxGf9/CJlJiUMUQeRN6r0+7cJZje6RtiK+O78R0PfZ6tb3HZb6ON70wNCFzPtoum2vPA+sqqNm3XBlZebmx7ka71wG9v20yh/LDK1uXWdD5jnzzYDKRtm5G8xTv6eGZXafbexBdnzisoUk86D9I2wPeynefFuFr5Jf6nthf7rd2RpuONmr8GUhb8UOYLF3U19nL9eZevbMxgrmyGTM1bUHmV9MrqpY9JbHQ3OL2/lX1KwT/qcYnB0rPC7zSj/Xh/qJHb6B8v2xOWtF9MzGyHHJfTffxo6J6yHDNvuyEWQ+RBbAKzzwlthHYKLwPd/sv7YdMMaVvmDDvtxx5pbbB6J4fVE6A4/rw5WxyzS0d4Veb8U9EINbKZsvj3nXeddUjN+4l3pk1zG4p8wNjcLfYvKNq8frZTzZs6HkCpi9Rdrz6SJl7StxFZjh7I3oqBVoI1Uqtl5jkzGyFbJe6UCGbWflKhXToD66vpm/xLWw75FUvsDeNfph39TTYX/zpHvdyly9ullzJae0IHtTPDdPexM4L8PmTO6T9f2xWsW0Er52HQPfLA9/7WGd0do7rsbPeZmO+sSKxUxMft2ewRSH+8by2uLnJ3yk4xjnj7nJ6MvXkLF21jaCvdYju/VvTeUrWpkq4dGpv5+nG+QtEVYz12OzwTGtMZV7BAq8yX25GkaW/2qPxNq9Mx65TkefMZHz2Cbb+5w/b0teT6bQj+wbrb4Rfsfcuyucb0007dgHbb3pfua6tQ+Ix9FXU1wNvTHOJ/PONWojescvy659DjKzGbITqm64z/WO/bM67K+qezX29sS4G7+ztqLZGJuj3NPZfWsh7vF7rJpznuRenVdMxr04W1/cjveTIZGF+zo/0xXVTg8v09/X/G8pX38NxTLRE3fZcxkILzM14/K5hlQG2NlQ11L5Z9xs7LT0DpMx9o5te6LrGkXrA9Vr+qH+zoQzsPZDfLd8beKl8EP8sck4hoUDNz2YvxVvmNi8drpF0LPFM9/b6xhl3VBbGN2rZWpVnVcm4x2QsZ47ZTp+I1Sueq/cvbuP9bZM5npLXOTO8zptOfnLQOyk87Zl4dyI7Ic7FtBoJDJ4r1hZGWQ+mKf3s+LfDnvOL00PMuwZvr3c0QM/DKJ8n0yZidODtdrE2og7jzxP5tuJGqO6dUbje3OkqjkDpaNmcZ3nFLWKf9VBP4+mi/4GdLEZseNl2LruMjP3V/V44kvZeOeRmYH+Ua1oD02rGgfLQTVX3v39v+40vh76j66fVbN8iWT5Oncvef7cdPwdlNe17sgdmGNTx9llQuLqHcDKTIzMSQYhf6M7/44ppmrZlIrruid0/PT+XioLe6rFylSQHYOA9rtmY8/+jaf8jt70DaO4tp39q2g9tRsiXLH0taKND65uHixbjkHPFoSs7T6ts6WwF67XkNkc1bX2/LLs2qtu9cHPnVu7A8czW/Q7z8jfGzpIc5OxuyHiwUL/yoP5/ps5qEPVZ9vKnzu3afJXmZF5+r2mZQVHb+ehr2Z9v4wvy6dMxztWm7zhexrS/zYndq1+7hw2b1KyTRI04Qw0M1v0uA+Hx2hOMgAzHzLfTmBAfdM32wv3tOSKVxrP2z/w0c6ubbVi7e0GnQvcJNNqlBm6lSsd+1f+j6VnW0F+owQ3RKL2ZT4duw7P+98Ildl41ges8l8A7Uz3dXYMeztDnc8BfWutZS1EcLj/Y9OwPNu42Hvlfh8DLNra72+IdLax4mn7u4b+HVs2fTVj4TRoe93lv6yE1xeXr+o/V66bu7v5i2Uy/4f7YSNIL19nB9DIT/PtYyOMT8FgOr3f/5KclYHbKzvlovwPP8j9UMPmPYQ89NTMxLuEfGS/aUbaLX6/4YDx2Cr9xjUHrzKOa0B1PtuWGLgJ4NV+G8ZlcN6jN1HYWdy7G21nXJSTzzraypkXYYenrmPvmn+9TEYm43Dx8somc+xfxWbfQc3CSBmHK+sXgDGsn1abjrEpie2Nw2941L8RenPxts2QA/52yr1CvP59A9Y7Q7VxYNMQOO5fDZWOy5gtp8CcDEBfTp9ro37PT+OmJN4Txv/lEpkvjz2z8dzyMGvQvjmB+7qQhY6s7w4vU4XsCvY9p3ijQZ975RomXqtLpEwZmOxV1+fV1ToD+j7svVWmJPZNlk8r+V0m5L1ztl46q3CMP3wEK96vi/KtczUlE//s9xQuWy5wslqTMlSuyczHzisHk6umtO/TpjloO6UtQ9VjQ2QyzjH+R2yIXJkvuR/4uvk/Aa0HWX90MhXzU20vMBPM+0Uq/9RXt8cI3YrzfliuUgwk3H2MQMT8A9Lk2rX56E1ZNFkjyyDfRusZcXG4g+sds7ETDGhkc/byBbD+2lyM20Om9eU2ZaE74J0J4xXC7wLqSS55PFwMpCyNb5pEgOl2dszH6I2b2Pxm6sf7ME7/e5HszF/J01a0q7BVarWrZuyxq9/dEIlbXB1bMLzfdM58rMxGr6K6/k/ajN6OyM1Hr242yOftMij9Bo+rh8R9L+yLQRQaoF3ze7yK9SCr1QxDZwj6w7Sv5XxDBI/5Kp2Zj17JhsqHatg1H/ERgivm4zDn3nysZlBW3//cYO9k20frTj07pfyFs/qqa9hfJ9kvfy7kLu8KPTXDSS11ePPR+2bebERzszYfpV175mNkSrIR7/TZ/vIY13lNEu4N2IImaP5Wo25l8X3Iony761ye//ow5DXoXbbKT4s2QPbNR652/pWpG7QlMR8xLjIrK+gqP6034rEf+F8LDrTOeoH5OVyrzB5kZ7trt91XAjONMmc9h62jMDlA2kerzcdXuQn1I6yR+dgxJSPzMfPTOotENJLZCP9XzNJN0zEGx+eJ8+16Puacmo19s/Rq6A5qx2ezk4BtUBxmckavWIeAp/8skMPEr/PafERTct989CrUXaQw7q/9uZ3F/cES6gk6k78zjbumZvshb6dP1RMiM1f0hPN1eUWw72d1fLf49TB1oKpZsPw15Vq6bK1ozHzUdWP/YDhnPu7OlB3fsRP2Ef+1N6xz8zLa1GBqVl8nisuKd3S6e+VlJplzUhef7JE5l212cID0qwUR1c2rnW63VkZUZK4uVi0j89ErWTxSu9D9pdqxFoWg3akulcR2IMvKeDf73pAZppmW20kfm4842aXse8I63yba3ODgsV1Knh9N0nhTxOexZqrtp7PmI18ksvyxmv5t+H/zCogB5w+lPgAAAABJRU5ErkJggg==\" y=\"-21.758125\"/>\n", "   </g>\n", "   <g id=\"PathCollection_1\">\n", "    <defs>\n", "     <path d=\"M 0 3 \n", "C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n", "C 2.683901 1.55874 3 0.795609 3 0 \n", "C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n", "C 1.55874 -2.683901 0.795609 -3 0 -3 \n", "C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n", "C -2.683901 -1.55874 -3 -0.795609 -3 0 \n", "C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n", "C -1.55874 2.683901 -0.795609 3 0 3 \n", "z\n", "\" id=\"mfc9ffdc8c3\" style=\"stroke:#333333;\"/>\n", "    </defs>\n", "    <g clip-path=\"url(#p2bbf45a42c)\">\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"112.130475\" xlink:href=\"#mfc9ffdc8c3\" y=\"183.02716\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"218.531808\" xlink:href=\"#mfc9ffdc8c3\" y=\"79.706304\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"239.633825\" xlink:href=\"#mfc9ffdc8c3\" y=\"80.994397\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"122.008015\" xlink:href=\"#mfc9ffdc8c3\" y=\"191.757377\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"116.060682\" xlink:href=\"#mfc9ffdc8c3\" y=\"192.256288\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"92.277532\" xlink:href=\"#mfc9ffdc8c3\" y=\"180.061819\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"227.018994\" xlink:href=\"#mfc9ffdc8c3\" y=\"76.887462\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"218.499712\" xlink:href=\"#mfc9ffdc8c3\" y=\"71.275153\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"110.302013\" xlink:href=\"#mfc9ffdc8c3\" y=\"191.868518\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"222.828211\" xlink:href=\"#mfc9ffdc8c3\" y=\"72.953996\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"117.798662\" xlink:href=\"#mfc9ffdc8c3\" y=\"188.528318\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"236.949732\" xlink:href=\"#mfc9ffdc8c3\" y=\"56.122229\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"223.235039\" xlink:href=\"#mfc9ffdc8c3\" y=\"80.056541\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"218.619913\" xlink:href=\"#mfc9ffdc8c3\" y=\"76.248163\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"110.525586\" xlink:href=\"#mfc9ffdc8c3\" y=\"198.228967\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"106.581102\" xlink:href=\"#mfc9ffdc8c3\" y=\"176.499583\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"233.588936\" xlink:href=\"#mfc9ffdc8c3\" y=\"102.286439\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"236.369999\" xlink:href=\"#mfc9ffdc8c3\" y=\"96.725344\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"213.86622\" xlink:href=\"#mfc9ffdc8c3\" y=\"77.338064\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"100.93054\" xlink:href=\"#mfc9ffdc8c3\" y=\"195.532199\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"235.709742\" xlink:href=\"#mfc9ffdc8c3\" y=\"56.455519\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"228.794651\" xlink:href=\"#mfc9ffdc8c3\" y=\"77.981996\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"126.325818\" xlink:href=\"#mfc9ffdc8c3\" y=\"169.253244\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"215.866897\" xlink:href=\"#mfc9ffdc8c3\" y=\"56.228154\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"114.849831\" xlink:href=\"#mfc9ffdc8c3\" y=\"184.170689\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"215.200167\" xlink:href=\"#mfc9ffdc8c3\" y=\"71.325868\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"104.738732\" xlink:href=\"#mfc9ffdc8c3\" y=\"172.132264\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"233.165389\" xlink:href=\"#mfc9ffdc8c3\" y=\"69.175006\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"229.770805\" xlink:href=\"#mfc9ffdc8c3\" y=\"79.208287\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"224.897941\" xlink:href=\"#mfc9ffdc8c3\" y=\"65.173809\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"109.018133\" xlink:href=\"#mfc9ffdc8c3\" y=\"175.976269\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"118.553492\" xlink:href=\"#mfc9ffdc8c3\" y=\"195.931899\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"105.114252\" xlink:href=\"#mfc9ffdc8c3\" y=\"164.360655\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"113.25433\" xlink:href=\"#mfc9ffdc8c3\" y=\"189.323026\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"124.586091\" xlink:href=\"#mfc9ffdc8c3\" y=\"188.967644\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"109.837824\" xlink:href=\"#mfc9ffdc8c3\" y=\"184.898956\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"125.181761\" xlink:href=\"#mfc9ffdc8c3\" y=\"197.902563\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"204.317789\" xlink:href=\"#mfc9ffdc8c3\" y=\"61.965272\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"235.267519\" xlink:href=\"#mfc9ffdc8c3\" y=\"81.643688\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"223.348378\" xlink:href=\"#mfc9ffdc8c3\" y=\"80.978112\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"109.797084\" xlink:href=\"#mfc9ffdc8c3\" y=\"192.545811\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"211.194686\" xlink:href=\"#mfc9ffdc8c3\" y=\"100.930901\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"229.513113\" xlink:href=\"#mfc9ffdc8c3\" y=\"75.481734\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"113.381992\" xlink:href=\"#mfc9ffdc8c3\" y=\"197.835874\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"123.337028\" xlink:href=\"#mfc9ffdc8c3\" y=\"202.207751\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"130.132567\" xlink:href=\"#mfc9ffdc8c3\" y=\"170.190943\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"216.473179\" xlink:href=\"#mfc9ffdc8c3\" y=\"87.364405\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"206.813885\" xlink:href=\"#mfc9ffdc8c3\" y=\"65.697345\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"107.513202\" xlink:href=\"#mfc9ffdc8c3\" y=\"192.863185\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"217.41874\" xlink:href=\"#mfc9ffdc8c3\" y=\"84.700348\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"113.95745\" xlink:href=\"#mfc9ffdc8c3\" y=\"189.098171\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"226.55733\" xlink:href=\"#mfc9ffdc8c3\" y=\"57.230812\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"111.233278\" xlink:href=\"#mfc9ffdc8c3\" y=\"179.431408\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"215.084754\" xlink:href=\"#mfc9ffdc8c3\" y=\"91.072443\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"224.100848\" xlink:href=\"#mfc9ffdc8c3\" y=\"68.121491\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"92.279163\" xlink:href=\"#mfc9ffdc8c3\" y=\"180.887702\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"206.717206\" xlink:href=\"#mfc9ffdc8c3\" y=\"67.932904\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"230.512427\" xlink:href=\"#mfc9ffdc8c3\" y=\"82.439245\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"101.651362\" xlink:href=\"#mfc9ffdc8c3\" y=\"185.78207\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"110.796814\" xlink:href=\"#mfc9ffdc8c3\" y=\"211.814172\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"110.369579\" xlink:href=\"#mfc9ffdc8c3\" y=\"188.183661\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"96.152024\" xlink:href=\"#mfc9ffdc8c3\" y=\"188.29022\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"241.658206\" xlink:href=\"#mfc9ffdc8c3\" y=\"82.547711\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"215.078112\" xlink:href=\"#mfc9ffdc8c3\" y=\"88.854193\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"235.390643\" xlink:href=\"#mfc9ffdc8c3\" y=\"62.012163\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"96.594278\" xlink:href=\"#mfc9ffdc8c3\" y=\"176.722784\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"223.89308\" xlink:href=\"#mfc9ffdc8c3\" y=\"95.68291\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"231.986534\" xlink:href=\"#mfc9ffdc8c3\" y=\"101.379014\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"129.434435\" xlink:href=\"#mfc9ffdc8c3\" y=\"202.700628\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"121.849164\" xlink:href=\"#mfc9ffdc8c3\" y=\"193.853222\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"111.071273\" xlink:href=\"#mfc9ffdc8c3\" y=\"180.426817\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"111.73278\" xlink:href=\"#mfc9ffdc8c3\" y=\"178.6056\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"224.372642\" xlink:href=\"#mfc9ffdc8c3\" y=\"73.188217\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"214.544764\" xlink:href=\"#mfc9ffdc8c3\" y=\"100.191683\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"111.037176\" xlink:href=\"#mfc9ffdc8c3\" y=\"184.756952\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"120.501423\" xlink:href=\"#mfc9ffdc8c3\" y=\"177.927389\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"121.737273\" xlink:href=\"#mfc9ffdc8c3\" y=\"183.651319\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"224.151653\" xlink:href=\"#mfc9ffdc8c3\" y=\"60.366512\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"223.43835\" xlink:href=\"#mfc9ffdc8c3\" y=\"58.920283\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"112.665686\" xlink:href=\"#mfc9ffdc8c3\" y=\"194.670616\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"232.62061\" xlink:href=\"#mfc9ffdc8c3\" y=\"60.454033\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"218.768323\" xlink:href=\"#mfc9ffdc8c3\" y=\"87.76343\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"231.184386\" xlink:href=\"#mfc9ffdc8c3\" y=\"57.202935\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"105.951856\" xlink:href=\"#mfc9ffdc8c3\" y=\"182.733923\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"220.266348\" xlink:href=\"#mfc9ffdc8c3\" y=\"68.561718\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"113.873959\" xlink:href=\"#mfc9ffdc8c3\" y=\"198.141743\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"121.135411\" xlink:href=\"#mfc9ffdc8c3\" y=\"191.679503\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"105.401412\" xlink:href=\"#mfc9ffdc8c3\" y=\"181.366251\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"203.450464\" xlink:href=\"#mfc9ffdc8c3\" y=\"73.166107\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"213.131642\" xlink:href=\"#mfc9ffdc8c3\" y=\"68.440188\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"195.475611\" xlink:href=\"#mfc9ffdc8c3\" y=\"65.960727\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"104.817254\" xlink:href=\"#mfc9ffdc8c3\" y=\"181.616921\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"128.101527\" xlink:href=\"#mfc9ffdc8c3\" y=\"193.778082\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"108.575899\" xlink:href=\"#mfc9ffdc8c3\" y=\"196.017082\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"213.699076\" xlink:href=\"#mfc9ffdc8c3\" y=\"102.201172\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"116.657511\" xlink:href=\"#mfc9ffdc8c3\" y=\"184.585026\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"111.028071\" xlink:href=\"#mfc9ffdc8c3\" y=\"179.737071\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"106.803829\" xlink:href=\"#mfc9ffdc8c3\" y=\"188.460235\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"114.368679\" xlink:href=\"#mfc9ffdc8c3\" y=\"194.747632\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"202.479261\" xlink:href=\"#mfc9ffdc8c3\" y=\"76.337577\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"209.669671\" xlink:href=\"#mfc9ffdc8c3\" y=\"63.421804\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"93.182322\" xlink:href=\"#mfc9ffdc8c3\" y=\"199.661904\"/>\n", "     <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"221.556365\" xlink:href=\"#mfc9ffdc8c3\" y=\"94.135493\"/>\n", "    </g>\n", "   </g>\n", "   <g id=\"PathCollection_2\">\n", "    <defs>\n", "     <path d=\"M 0 3 \n", "C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n", "C 2.683901 1.55874 3 0.795609 3 0 \n", "C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n", "C 1.55874 -2.683901 0.795609 -3 0 -3 \n", "C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n", "C -2.683901 -1.55874 -3 -0.795609 -3 0 \n", "C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n", "C -1.55874 2.683901 -0.795609 3 0 3 \n", "z\n", "\" id=\"m0bf92b68d0\" style=\"stroke:#333333;\"/>\n", "    </defs>\n", "    <g clip-path=\"url(#p2bbf45a42c)\">\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"217.606427\" xlink:href=\"#m0bf92b68d0\" y=\"173.251328\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"227.769027\" xlink:href=\"#m0bf92b68d0\" y=\"173.464324\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"218.500016\" xlink:href=\"#m0bf92b68d0\" y=\"187.443356\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"106.984413\" xlink:href=\"#m0bf92b68d0\" y=\"66.511974\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"233.133545\" xlink:href=\"#m0bf92b68d0\" y=\"189.544269\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"115.335326\" xlink:href=\"#m0bf92b68d0\" y=\"80.815627\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"208.972555\" xlink:href=\"#m0bf92b68d0\" y=\"197.223837\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"139.005527\" xlink:href=\"#m0bf92b68d0\" y=\"77.700995\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"230.664194\" xlink:href=\"#m0bf92b68d0\" y=\"201.829041\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"205.002573\" xlink:href=\"#m0bf92b68d0\" y=\"176.909335\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"216.845993\" xlink:href=\"#m0bf92b68d0\" y=\"167.032503\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"88.160629\" xlink:href=\"#m0bf92b68d0\" y=\"65.861243\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"128.021696\" xlink:href=\"#m0bf92b68d0\" y=\"66.622254\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"112.260448\" xlink:href=\"#m0bf92b68d0\" y=\"91.009332\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"112.50413\" xlink:href=\"#m0bf92b68d0\" y=\"63.911605\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"95.981186\" xlink:href=\"#m0bf92b68d0\" y=\"61.50077\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"219.993175\" xlink:href=\"#m0bf92b68d0\" y=\"189.668312\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"206.167029\" xlink:href=\"#m0bf92b68d0\" y=\"173.135684\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"213.740977\" xlink:href=\"#m0bf92b68d0\" y=\"174.762918\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"121.204965\" xlink:href=\"#m0bf92b68d0\" y=\"77.433213\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"203.018823\" xlink:href=\"#m0bf92b68d0\" y=\"182.099388\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"238.152369\" xlink:href=\"#m0bf92b68d0\" y=\"188.7116\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"231.441781\" xlink:href=\"#m0bf92b68d0\" y=\"186.794029\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"211.648243\" xlink:href=\"#m0bf92b68d0\" y=\"189.950709\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"236.220422\" xlink:href=\"#m0bf92b68d0\" y=\"169.740566\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"101.853762\" xlink:href=\"#m0bf92b68d0\" y=\"86.031191\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"233.94689\" xlink:href=\"#m0bf92b68d0\" y=\"172.051387\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"123.501925\" xlink:href=\"#m0bf92b68d0\" y=\"74.175724\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"218.585652\" xlink:href=\"#m0bf92b68d0\" y=\"171.48351\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"90.32929\" xlink:href=\"#m0bf92b68d0\" y=\"75.633164\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"111.280011\" xlink:href=\"#m0bf92b68d0\" y=\"82.837267\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"211.740197\" xlink:href=\"#m0bf92b68d0\" y=\"195.115065\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"205.118653\" xlink:href=\"#m0bf92b68d0\" y=\"174.999985\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"115.182647\" xlink:href=\"#m0bf92b68d0\" y=\"78.537928\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"227.992219\" xlink:href=\"#m0bf92b68d0\" y=\"175.571697\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"115.634509\" xlink:href=\"#m0bf92b68d0\" y=\"84.738407\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"113.207682\" xlink:href=\"#m0bf92b68d0\" y=\"86.904679\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"241.406644\" xlink:href=\"#m0bf92b68d0\" y=\"195.654551\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"218.12314\" xlink:href=\"#m0bf92b68d0\" y=\"172.59052\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"102.818896\" xlink:href=\"#m0bf92b68d0\" y=\"74.729342\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"126.547202\" xlink:href=\"#m0bf92b68d0\" y=\"76.348126\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"129.239769\" xlink:href=\"#m0bf92b68d0\" y=\"62.486839\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"228.152228\" xlink:href=\"#m0bf92b68d0\" y=\"193.26863\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"103.215052\" xlink:href=\"#m0bf92b68d0\" y=\"78.005027\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"216.222162\" xlink:href=\"#m0bf92b68d0\" y=\"176.670472\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"114.916325\" xlink:href=\"#m0bf92b68d0\" y=\"88.653235\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"121.115357\" xlink:href=\"#m0bf92b68d0\" y=\"70.241533\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"120.258643\" xlink:href=\"#m0bf92b68d0\" y=\"72.957314\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"209.315916\" xlink:href=\"#m0bf92b68d0\" y=\"177.378942\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"103.210102\" xlink:href=\"#m0bf92b68d0\" y=\"91.806632\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"119.325862\" xlink:href=\"#m0bf92b68d0\" y=\"68.692592\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"202.824093\" xlink:href=\"#m0bf92b68d0\" y=\"174.268046\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"223.801372\" xlink:href=\"#m0bf92b68d0\" y=\"196.626833\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"117.310993\" xlink:href=\"#m0bf92b68d0\" y=\"69.094029\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"122.253941\" xlink:href=\"#m0bf92b68d0\" y=\"84.153386\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"208.231426\" xlink:href=\"#m0bf92b68d0\" y=\"184.433377\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"201.565732\" xlink:href=\"#m0bf92b68d0\" y=\"186.557453\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"120.045207\" xlink:href=\"#m0bf92b68d0\" y=\"79.883512\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"85.476218\" xlink:href=\"#m0bf92b68d0\" y=\"85.530225\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"108.621104\" xlink:href=\"#m0bf92b68d0\" y=\"91.775968\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"122.454711\" xlink:href=\"#m0bf92b68d0\" y=\"71.807464\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"105.245089\" xlink:href=\"#m0bf92b68d0\" y=\"80.555517\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"110.345637\" xlink:href=\"#m0bf92b68d0\" y=\"74.684201\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"122.948606\" xlink:href=\"#m0bf92b68d0\" y=\"84.523445\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"238.331781\" xlink:href=\"#m0bf92b68d0\" y=\"190.109778\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"99.158482\" xlink:href=\"#m0bf92b68d0\" y=\"62.144579\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"238.904126\" xlink:href=\"#m0bf92b68d0\" y=\"176.479967\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"228.511103\" xlink:href=\"#m0bf92b68d0\" y=\"166.124698\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"204.749573\" xlink:href=\"#m0bf92b68d0\" y=\"197.183933\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"207.817035\" xlink:href=\"#m0bf92b68d0\" y=\"209.749946\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"113.647455\" xlink:href=\"#m0bf92b68d0\" y=\"63.325534\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"209.936021\" xlink:href=\"#m0bf92b68d0\" y=\"171.712521\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"199.504867\" xlink:href=\"#m0bf92b68d0\" y=\"160.73698\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"102.861494\" xlink:href=\"#m0bf92b68d0\" y=\"67.872443\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"121.543306\" xlink:href=\"#m0bf92b68d0\" y=\"77.896386\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"230.263574\" xlink:href=\"#m0bf92b68d0\" y=\"175.913803\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"108.769772\" xlink:href=\"#m0bf92b68d0\" y=\"71.243232\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"118.137999\" xlink:href=\"#m0bf92b68d0\" y=\"79.260109\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"116.883422\" xlink:href=\"#m0bf92b68d0\" y=\"81.810632\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"217.332054\" xlink:href=\"#m0bf92b68d0\" y=\"170.303854\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"223.569574\" xlink:href=\"#m0bf92b68d0\" y=\"163.041048\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"230.510522\" xlink:href=\"#m0bf92b68d0\" y=\"176.611656\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"142.305392\" xlink:href=\"#m0bf92b68d0\" y=\"77.469185\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"204.163476\" xlink:href=\"#m0bf92b68d0\" y=\"198.978781\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"244.521801\" xlink:href=\"#m0bf92b68d0\" y=\"198.229188\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"214.441631\" xlink:href=\"#m0bf92b68d0\" y=\"195.840128\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"105.972186\" xlink:href=\"#m0bf92b68d0\" y=\"85.737332\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"104.205171\" xlink:href=\"#m0bf92b68d0\" y=\"77.466631\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"232.367065\" xlink:href=\"#m0bf92b68d0\" y=\"198.103735\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"104.549328\" xlink:href=\"#m0bf92b68d0\" y=\"83.321073\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"93.504718\" xlink:href=\"#m0bf92b68d0\" y=\"79.424966\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"127.819202\" xlink:href=\"#m0bf92b68d0\" y=\"69.050871\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"106.888986\" xlink:href=\"#m0bf92b68d0\" y=\"71.394117\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"238.372282\" xlink:href=\"#m0bf92b68d0\" y=\"196.147065\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"200.720575\" xlink:href=\"#m0bf92b68d0\" y=\"181.948403\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"113.562585\" xlink:href=\"#m0bf92b68d0\" y=\"65.860828\"/>\n", "     <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"236.216055\" xlink:href=\"#m0bf92b68d0\" y=\"194.238972\"/>\n", "    </g>\n", "   </g>\n", "   <g id=\"matplotlib.axis_1\">\n", "    <g id=\"xtick_1\">\n", "     <g id=\"line2d_1\">\n", "      <defs>\n", "       <path d=\"M 0 0 \n", "L 0 3.5 \n", "\" id=\"m6a5f4a2251\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n", "      </defs>\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"58.523438\" xlink:href=\"#m6a5f4a2251\" y=\"239.758125\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_1\">\n", "      <!-- \u22120.5 -->\n", "      <g transform=\"translate(46.382031 254.356562)scale(0.1 -0.1)\">\n", "       <defs>\n", "        <path d=\"M 678 2272 \n", "L 4684 2272 \n", "L 4684 1741 \n", "L 678 1741 \n", "L 678 2272 \n", "z\n", "\" id=\"DejaVuSans-2212\" transform=\"scale(0.015625)\"/>\n", "        <path d=\"M 2034 4250 \n", "Q 1547 4250 1301 3770 \n", "Q 1056 3291 1056 2328 \n", "Q 1056 1369 1301 889 \n", "Q 1547 409 2034 409 \n", "Q 2525 409 2770 889 \n", "Q 3016 1369 3016 2328 \n", "Q 3016 3291 2770 3770 \n", "Q 2525 4250 2034 4250 \n", "z\n", "M 2034 4750 \n", "Q 2819 4750 3233 4129 \n", "Q 3647 3509 3647 2328 \n", "Q 3647 1150 3233 529 \n", "Q 2819 -91 2034 -91 \n", "Q 1250 -91 836 529 \n", "Q 422 1150 422 2328 \n", "Q 422 3509 836 4129 \n", "Q 1250 4750 2034 4750 \n", "z\n", "\" id=\"DejaVuSans-30\" transform=\"scale(0.015625)\"/>\n", "        <path d=\"M 684 794 \n", "L 1344 794 \n", "L 1344 0 \n", "L 684 0 \n", "L 684 794 \n", "z\n", "\" id=\"DejaVuSans-2e\" transform=\"scale(0.015625)\"/>\n", "        <path d=\"M 691 4666 \n", "L 3169 4666 \n", "L 3169 4134 \n", "L 1269 4134 \n", "L 1269 2991 \n", "Q 1406 3038 1543 3061 \n", "Q 1681 3084 1819 3084 \n", "Q 2600 3084 3056 2656 \n", "Q 3513 2228 3513 1497 \n", "Q 3513 744 3044 326 \n", "Q 2575 -91 1722 -91 \n", "Q 1428 -91 1123 -41 \n", "Q 819 9 494 109 \n", "L 494 744 \n", "Q 775 591 1075 516 \n", "Q 1375 441 1709 441 \n", "Q 2250 441 2565 725 \n", "Q 2881 1009 2881 1497 \n", "Q 2881 1984 2565 2268 \n", "Q 2250 2553 1709 2553 \n", "Q 1456 2553 1204 2497 \n", "Q 953 2441 691 2322 \n", "L 691 4666 \n", "z\n", "\" id=\"DejaVuSans-35\" transform=\"scale(0.015625)\"/>\n", "       </defs>\n", "       <use xlink:href=\"#DejaVuSans-2212\"/>\n", "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-30\"/>\n", "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-35\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"xtick_2\">\n", "     <g id=\"line2d_2\">\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"112.883437\" xlink:href=\"#m6a5f4a2251\" y=\"239.758125\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_2\">\n", "      <!-- 0.0 -->\n", "      <g transform=\"translate(104.931875 254.356562)scale(0.1 -0.1)\">\n", "       <use xlink:href=\"#DejaVuSans-30\"/>\n", "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-30\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"xtick_3\">\n", "     <g id=\"line2d_3\">\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"167.243437\" xlink:href=\"#m6a5f4a2251\" y=\"239.758125\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_3\">\n", "      <!-- 0.5 -->\n", "      <g transform=\"translate(159.291875 254.356562)scale(0.1 -0.1)\">\n", "       <use xlink:href=\"#DejaVuSans-30\"/>\n", "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-35\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"xtick_4\">\n", "     <g id=\"line2d_4\">\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"221.603437\" xlink:href=\"#m6a5f4a2251\" y=\"239.758125\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_4\">\n", "      <!-- 1.0 -->\n", "      <g transform=\"translate(213.651875 254.356562)scale(0.1 -0.1)\">\n", "       <defs>\n", "        <path d=\"M 794 531 \n", "L 1825 531 \n", "L 1825 4091 \n", "L 703 3866 \n", "L 703 4441 \n", "L 1819 4666 \n", "L 2450 4666 \n", "L 2450 531 \n", "L 3481 531 \n", "L 3481 0 \n", "L 794 0 \n", "L 794 531 \n", "z\n", "\" id=\"DejaVuSans-31\" transform=\"scale(0.015625)\"/>\n", "       </defs>\n", "       <use xlink:href=\"#DejaVuSans-31\"/>\n", "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-30\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"xtick_5\">\n", "     <g id=\"line2d_5\">\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"275.963437\" xlink:href=\"#m6a5f4a2251\" y=\"239.758125\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_5\">\n", "      <!-- 1.5 -->\n", "      <g transform=\"translate(268.011875 254.356562)scale(0.1 -0.1)\">\n", "       <use xlink:href=\"#DejaVuSans-31\"/>\n", "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-35\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"text_6\">\n", "     <!-- $x_1$ -->\n", "     <g transform=\"translate(161.893438 268.034687)scale(0.1 -0.1)\">\n", "      <defs>\n", "       <path d=\"M 3841 3500 \n", "L 2234 1784 \n", "L 3219 0 \n", "L 2559 0 \n", "L 1819 1388 \n", "L 531 0 \n", "L -166 0 \n", "L 1556 1844 \n", "L 641 3500 \n", "L 1300 3500 \n", "L 1972 2234 \n", "L 3144 3500 \n", "L 3841 3500 \n", "z\n", "\" id=\"DejaVuSans-Oblique-78\" transform=\"scale(0.015625)\"/>\n", "      </defs>\n", "      <use transform=\"translate(0 0.3125)\" xlink:href=\"#DejaVuSans-Oblique-78\"/>\n", "      <use transform=\"translate(59.179688 -16.09375)scale(0.7)\" xlink:href=\"#DejaVuSans-31\"/>\n", "     </g>\n", "    </g>\n", "   </g>\n", "   <g id=\"matplotlib.axis_2\">\n", "    <g id=\"ytick_1\">\n", "     <g id=\"line2d_6\">\n", "      <defs>\n", "       <path d=\"M 0 0 \n", "L -3.5 0 \n", "\" id=\"m2359068326\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n", "      </defs>\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"58.523438\" xlink:href=\"#m2359068326\" y=\"239.758125\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_7\">\n", "      <!-- \u22120.50 -->\n", "      <g transform=\"translate(20.878125 243.557344)scale(0.1 -0.1)\">\n", "       <use xlink:href=\"#DejaVuSans-2212\"/>\n", "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-30\"/>\n", "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-35\"/>\n", "       <use x=\"242.822266\" xlink:href=\"#DejaVuSans-30\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"ytick_2\">\n", "     <g id=\"line2d_7\">\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"58.523438\" xlink:href=\"#m2359068326\" y=\"212.578125\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_8\">\n", "      <!-- \u22120.25 -->\n", "      <g transform=\"translate(20.878125 216.377344)scale(0.1 -0.1)\">\n", "       <defs>\n", "        <path d=\"M 1228 531 \n", "L 3431 531 \n", "L 3431 0 \n", "L 469 0 \n", "L 469 531 \n", "Q 828 903 1448 1529 \n", "Q 2069 2156 2228 2338 \n", "Q 2531 2678 2651 2914 \n", "Q 2772 3150 2772 3378 \n", "Q 2772 3750 2511 3984 \n", "Q 2250 4219 1831 4219 \n", "Q 1534 4219 1204 4116 \n", "Q 875 4013 500 3803 \n", "L 500 4441 \n", "Q 881 4594 1212 4672 \n", "Q 1544 4750 1819 4750 \n", "Q 2544 4750 2975 4387 \n", "Q 3406 4025 3406 3419 \n", "Q 3406 3131 3298 2873 \n", "Q 3191 2616 2906 2266 \n", "Q 2828 2175 2409 1742 \n", "Q 1991 1309 1228 531 \n", "z\n", "\" id=\"DejaVuSans-32\" transform=\"scale(0.015625)\"/>\n", "       </defs>\n", "       <use xlink:href=\"#DejaVuSans-2212\"/>\n", "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-30\"/>\n", "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-32\"/>\n", "       <use x=\"242.822266\" xlink:href=\"#DejaVuSans-35\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"ytick_3\">\n", "     <g id=\"line2d_8\">\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"58.523438\" xlink:href=\"#m2359068326\" y=\"185.398125\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_9\">\n", "      <!-- 0.00 -->\n", "      <g transform=\"translate(29.257812 189.197344)scale(0.1 -0.1)\">\n", "       <use xlink:href=\"#DejaVuSans-30\"/>\n", "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-30\"/>\n", "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-30\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"ytick_4\">\n", "     <g id=\"line2d_9\">\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"58.523438\" xlink:href=\"#m2359068326\" y=\"158.218125\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_10\">\n", "      <!-- 0.25 -->\n", "      <g transform=\"translate(29.257812 162.017344)scale(0.1 -0.1)\">\n", "       <use xlink:href=\"#DejaVuSans-30\"/>\n", "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-32\"/>\n", "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-35\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"ytick_5\">\n", "     <g id=\"line2d_10\">\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"58.523438\" xlink:href=\"#m2359068326\" y=\"131.038125\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_11\">\n", "      <!-- 0.50 -->\n", "      <g transform=\"translate(29.257812 134.837344)scale(0.1 -0.1)\">\n", "       <use xlink:href=\"#DejaVuSans-30\"/>\n", "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-35\"/>\n", "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-30\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"ytick_6\">\n", "     <g id=\"line2d_11\">\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"58.523438\" xlink:href=\"#m2359068326\" y=\"103.858125\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_12\">\n", "      <!-- 0.75 -->\n", "      <g transform=\"translate(29.257812 107.657344)scale(0.1 -0.1)\">\n", "       <defs>\n", "        <path d=\"M 525 4666 \n", "L 3525 4666 \n", "L 3525 4397 \n", "L 1831 0 \n", "L 1172 0 \n", "L 2766 4134 \n", "L 525 4134 \n", "L 525 4666 \n", "z\n", "\" id=\"DejaVuSans-37\" transform=\"scale(0.015625)\"/>\n", "       </defs>\n", "       <use xlink:href=\"#DejaVuSans-30\"/>\n", "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-37\"/>\n", "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-35\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"ytick_7\">\n", "     <g id=\"line2d_12\">\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"58.523438\" xlink:href=\"#m2359068326\" y=\"76.678125\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_13\">\n", "      <!-- 1.00 -->\n", "      <g transform=\"translate(29.257812 80.477344)scale(0.1 -0.1)\">\n", "       <use xlink:href=\"#DejaVuSans-31\"/>\n", "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-30\"/>\n", "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-30\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"ytick_8\">\n", "     <g id=\"line2d_13\">\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"58.523438\" xlink:href=\"#m2359068326\" y=\"49.498125\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_14\">\n", "      <!-- 1.25 -->\n", "      <g transform=\"translate(29.257812 53.297344)scale(0.1 -0.1)\">\n", "       <use xlink:href=\"#DejaVuSans-31\"/>\n", "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-32\"/>\n", "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-35\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"ytick_9\">\n", "     <g id=\"line2d_14\">\n", "      <g>\n", "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"58.523438\" xlink:href=\"#m2359068326\" y=\"22.318125\"/>\n", "      </g>\n", "     </g>\n", "     <g id=\"text_15\">\n", "      <!-- 1.50 -->\n", "      <g transform=\"translate(29.257812 26.117344)scale(0.1 -0.1)\">\n", "       <use xlink:href=\"#DejaVuSans-31\"/>\n", "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-2e\"/>\n", "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-35\"/>\n", "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-30\"/>\n", "      </g>\n", "     </g>\n", "    </g>\n", "    <g id=\"text_16\">\n", "     <!-- $x_2$ -->\n", "     <g transform=\"translate(14.798437 136.388125)rotate(-90)scale(0.1 -0.1)\">\n", "      <use transform=\"translate(0 0.3125)\" xlink:href=\"#DejaVuSans-Oblique-78\"/>\n", "      <use transform=\"translate(59.179688 -16.09375)scale(0.7)\" xlink:href=\"#DejaVuSans-32\"/>\n", "     </g>\n", "    </g>\n", "   </g>\n", "   <g id=\"patch_3\">\n", "    <path d=\"M 58.523438 239.758125 \n", "L 58.523438 22.318125 \n", "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n", "   </g>\n", "   <g id=\"patch_4\">\n", "    <path d=\"M 275.963437 239.758125 \n", "L 275.963437 22.318125 \n", "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n", "   </g>\n", "   <g id=\"patch_5\">\n", "    <path d=\"M 58.523438 239.758125 \n", "L 275.963437 239.758125 \n", "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n", "   </g>\n", "   <g id=\"patch_6\">\n", "    <path d=\"M 58.523438 22.318125 \n", "L 275.963437 22.318125 \n", "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n", "   </g>\n", "   <g id=\"text_17\">\n", "    <!-- Dataset samples -->\n", "    <g transform=\"translate(116.900625 16.318125)scale(0.12 -0.12)\">\n", "     <defs>\n", "      <path d=\"M 1259 4147 \n", "L 1259 519 \n", "L 2022 519 \n", "Q 2988 519 3436 956 \n", "Q 3884 1394 3884 2338 \n", "Q 3884 3275 3436 3711 \n", "Q 2988 4147 2022 4147 \n", "L 1259 4147 \n", "z\n", "M 628 4666 \n", "L 1925 4666 \n", "Q 3281 4666 3915 4102 \n", "Q 4550 3538 4550 2338 \n", "Q 4550 1131 3912 565 \n", "Q 3275 0 1925 0 \n", "L 628 0 \n", "L 628 4666 \n", "z\n", "\" id=\"DejaVuSans-44\" transform=\"scale(0.015625)\"/>\n", "      <path d=\"M 2194 1759 \n", "Q 1497 1759 1228 1600 \n", "Q 959 1441 959 1056 \n", "Q 959 750 1161 570 \n", "Q 1363 391 1709 391 \n", "Q 2188 391 2477 730 \n", "Q 2766 1069 2766 1631 \n", "L 2766 1759 \n", "L 2194 1759 \n", "z\n", "M 3341 1997 \n", "L 3341 0 \n", "L 2766 0 \n", "L 2766 531 \n", "Q 2569 213 2275 61 \n", "Q 1981 -91 1556 -91 \n", "Q 1019 -91 701 211 \n", "Q 384 513 384 1019 \n", "Q 384 1609 779 1909 \n", "Q 1175 2209 1959 2209 \n", "L 2766 2209 \n", "L 2766 2266 \n", "Q 2766 2663 2505 2880 \n", "Q 2244 3097 1772 3097 \n", "Q 1472 3097 1187 3025 \n", "Q 903 2953 641 2809 \n", "L 641 3341 \n", "Q 956 3463 1253 3523 \n", "Q 1550 3584 1831 3584 \n", "Q 2591 3584 2966 3190 \n", "Q 3341 2797 3341 1997 \n", "z\n", "\" id=\"DejaVuSans-61\" transform=\"scale(0.015625)\"/>\n", "      <path d=\"M 1172 4494 \n", "L 1172 3500 \n", "L 2356 3500 \n", "L 2356 3053 \n", "L 1172 3053 \n", "L 1172 1153 \n", "Q 1172 725 1289 603 \n", "Q 1406 481 1766 481 \n", "L 2356 481 \n", "L 2356 0 \n", "L 1766 0 \n", "Q 1100 0 847 248 \n", "Q 594 497 594 1153 \n", "L 594 3053 \n", "L 172 3053 \n", "L 172 3500 \n", "L 594 3500 \n", "L 594 4494 \n", "L 1172 4494 \n", "z\n", "\" id=\"DejaVuSans-74\" transform=\"scale(0.015625)\"/>\n", "      <path d=\"M 2834 3397 \n", "L 2834 2853 \n", "Q 2591 2978 2328 3040 \n", "Q 2066 3103 1784 3103 \n", "Q 1356 3103 1142 2972 \n", "Q 928 2841 928 2578 \n", "Q 928 2378 1081 2264 \n", "Q 1234 2150 1697 2047 \n", "L 1894 2003 \n", "Q 2506 1872 2764 1633 \n", "Q 3022 1394 3022 966 \n", "Q 3022 478 2636 193 \n", "Q 2250 -91 1575 -91 \n", "Q 1294 -91 989 -36 \n", "Q 684 19 347 128 \n", "L 347 722 \n", "Q 666 556 975 473 \n", "Q 1284 391 1588 391 \n", "Q 1994 391 2212 530 \n", "Q 2431 669 2431 922 \n", "Q 2431 1156 2273 1281 \n", "Q 2116 1406 1581 1522 \n", "L 1381 1569 \n", "Q 847 1681 609 1914 \n", "Q 372 2147 372 2553 \n", "Q 372 3047 722 3315 \n", "Q 1072 3584 1716 3584 \n", "Q 2034 3584 2315 3537 \n", "Q 2597 3491 2834 3397 \n", "z\n", "\" id=\"DejaVuSans-73\" transform=\"scale(0.015625)\"/>\n", "      <path d=\"M 3597 1894 \n", "L 3597 1613 \n", "L 953 1613 \n", "Q 991 1019 1311 708 \n", "Q 1631 397 2203 397 \n", "Q 2534 397 2845 478 \n", "Q 3156 559 3463 722 \n", "L 3463 178 \n", "Q 3153 47 2828 -22 \n", "Q 2503 -91 2169 -91 \n", "Q 1331 -91 842 396 \n", "Q 353 884 353 1716 \n", "Q 353 2575 817 3079 \n", "Q 1281 3584 2069 3584 \n", "Q 2775 3584 3186 3129 \n", "Q 3597 2675 3597 1894 \n", "z\n", "M 3022 2063 \n", "Q 3016 2534 2758 2815 \n", "Q 2500 3097 2075 3097 \n", "Q 1594 3097 1305 2825 \n", "Q 1016 2553 972 2059 \n", "L 3022 2063 \n", "z\n", "\" id=\"DejaVuSans-65\" transform=\"scale(0.015625)\"/>\n", "      <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n", "      <path d=\"M 3328 2828 \n", "Q 3544 3216 3844 3400 \n", "Q 4144 3584 4550 3584 \n", "Q 5097 3584 5394 3201 \n", "Q 5691 2819 5691 2113 \n", "L 5691 0 \n", "L 5113 0 \n", "L 5113 2094 \n", "Q 5113 2597 4934 2840 \n", "Q 4756 3084 4391 3084 \n", "Q 3944 3084 3684 2787 \n", "Q 3425 2491 3425 1978 \n", "L 3425 0 \n", "L 2847 0 \n", "L 2847 2094 \n", "Q 2847 2600 2669 2842 \n", "Q 2491 3084 2119 3084 \n", "Q 1678 3084 1418 2786 \n", "Q 1159 2488 1159 1978 \n", "L 1159 0 \n", "L 581 0 \n", "L 581 3500 \n", "L 1159 3500 \n", "L 1159 2956 \n", "Q 1356 3278 1631 3431 \n", "Q 1906 3584 2284 3584 \n", "Q 2666 3584 2933 3390 \n", "Q 3200 3197 3328 2828 \n", "z\n", "\" id=\"DejaVuSans-6d\" transform=\"scale(0.015625)\"/>\n", "      <path d=\"M 1159 525 \n", "L 1159 -1331 \n", "L 581 -1331 \n", "L 581 3500 \n", "L 1159 3500 \n", "L 1159 2969 \n", "Q 1341 3281 1617 3432 \n", "Q 1894 3584 2278 3584 \n", "Q 2916 3584 3314 3078 \n", "Q 3713 2572 3713 1747 \n", "Q 3713 922 3314 415 \n", "Q 2916 -91 2278 -91 \n", "Q 1894 -91 1617 61 \n", "Q 1341 213 1159 525 \n", "z\n", "M 3116 1747 \n", "Q 3116 2381 2855 2742 \n", "Q 2594 3103 2138 3103 \n", "Q 1681 3103 1420 2742 \n", "Q 1159 2381 1159 1747 \n", "Q 1159 1113 1420 752 \n", "Q 1681 391 2138 391 \n", "Q 2594 391 2855 752 \n", "Q 3116 1113 3116 1747 \n", "z\n", "\" id=\"DejaVuSans-70\" transform=\"scale(0.015625)\"/>\n", "      <path d=\"M 603 4863 \n", "L 1178 4863 \n", "L 1178 0 \n", "L 603 0 \n", "L 603 4863 \n", "z\n", "\" id=\"DejaVuSans-6c\" transform=\"scale(0.015625)\"/>\n", "     </defs>\n", "     <use xlink:href=\"#DejaVuSans-44\"/>\n", "     <use x=\"77.001953\" xlink:href=\"#DejaVuSans-61\"/>\n", "     <use x=\"138.28125\" xlink:href=\"#DejaVuSans-74\"/>\n", "     <use x=\"177.490234\" xlink:href=\"#DejaVuSans-61\"/>\n", "     <use x=\"238.769531\" xlink:href=\"#DejaVuSans-73\"/>\n", "     <use x=\"290.869141\" xlink:href=\"#DejaVuSans-65\"/>\n", "     <use x=\"352.392578\" xlink:href=\"#DejaVuSans-74\"/>\n", "     <use x=\"391.601562\" xlink:href=\"#DejaVuSans-20\"/>\n", "     <use x=\"423.388672\" xlink:href=\"#DejaVuSans-73\"/>\n", "     <use x=\"475.488281\" xlink:href=\"#DejaVuSans-61\"/>\n", "     <use x=\"536.767578\" xlink:href=\"#DejaVuSans-6d\"/>\n", "     <use x=\"634.179688\" xlink:href=\"#DejaVuSans-70\"/>\n", "     <use x=\"697.65625\" xlink:href=\"#DejaVuSans-6c\"/>\n", "     <use x=\"725.439453\" xlink:href=\"#DejaVuSans-65\"/>\n", "     <use x=\"786.962891\" xlink:href=\"#DejaVuSans-73\"/>\n", "    </g>\n", "   </g>\n", "   <g id=\"legend_1\">\n", "    <g id=\"patch_7\">\n", "     <path d=\"M 65.523438 59.674375 \n", "L 133.371875 59.674375 \n", "Q 135.371875 59.674375 135.371875 57.674375 \n", "L 135.371875 29.318125 \n", "Q 135.371875 27.318125 133.371875 27.318125 \n", "L 65.523438 27.318125 \n", "Q 63.523438 27.318125 63.523438 29.318125 \n", "L 63.523438 57.674375 \n", "Q 63.523438 59.674375 65.523438 59.674375 \n", "z\n", "\" style=\"fill:#ffffff;opacity:0.8;stroke:#cccccc;stroke-linejoin:miter;\"/>\n", "    </g>\n", "    <g id=\"PathCollection_3\">\n", "     <g>\n", "      <use style=\"fill:#1f77b4;stroke:#333333;\" x=\"77.523438\" xlink:href=\"#mfc9ffdc8c3\" y=\"36.291562\"/>\n", "     </g>\n", "    </g>\n", "    <g id=\"text_18\">\n", "     <!-- Class 0 -->\n", "     <g transform=\"translate(95.523438 38.916562)scale(0.1 -0.1)\">\n", "      <defs>\n", "       <path d=\"M 4122 4306 \n", "L 4122 3641 \n", "Q 3803 3938 3442 4084 \n", "Q 3081 4231 2675 4231 \n", "Q 1875 4231 1450 3742 \n", "Q 1025 3253 1025 2328 \n", "Q 1025 1406 1450 917 \n", "Q 1875 428 2675 428 \n", "Q 3081 428 3442 575 \n", "Q 3803 722 4122 1019 \n", "L 4122 359 \n", "Q 3791 134 3420 21 \n", "Q 3050 -91 2638 -91 \n", "Q 1578 -91 968 557 \n", "Q 359 1206 359 2328 \n", "Q 359 3453 968 4101 \n", "Q 1578 4750 2638 4750 \n", "Q 3056 4750 3426 4639 \n", "Q 3797 4528 4122 4306 \n", "z\n", "\" id=\"DejaVuSans-43\" transform=\"scale(0.015625)\"/>\n", "      </defs>\n", "      <use xlink:href=\"#DejaVuSans-43\"/>\n", "      <use x=\"69.824219\" xlink:href=\"#DejaVuSans-6c\"/>\n", "      <use x=\"97.607422\" xlink:href=\"#DejaVuSans-61\"/>\n", "      <use x=\"158.886719\" xlink:href=\"#DejaVuSans-73\"/>\n", "      <use x=\"210.986328\" xlink:href=\"#DejaVuSans-73\"/>\n", "      <use x=\"263.085938\" xlink:href=\"#DejaVuSans-20\"/>\n", "      <use x=\"294.873047\" xlink:href=\"#DejaVuSans-30\"/>\n", "     </g>\n", "    </g>\n", "    <g id=\"PathCollection_4\">\n", "     <g>\n", "      <use style=\"fill:#ff7f0e;stroke:#333333;\" x=\"77.523438\" xlink:href=\"#m0bf92b68d0\" y=\"50.969687\"/>\n", "     </g>\n", "    </g>\n", "    <g id=\"text_19\">\n", "     <!-- Class 1 -->\n", "     <g transform=\"translate(95.523438 53.594687)scale(0.1 -0.1)\">\n", "      <use xlink:href=\"#DejaVuSans-43\"/>\n", "      <use x=\"69.824219\" xlink:href=\"#DejaVuSans-6c\"/>\n", "      <use x=\"97.607422\" xlink:href=\"#DejaVuSans-61\"/>\n", "      <use x=\"158.886719\" xlink:href=\"#DejaVuSans-73\"/>\n", "      <use x=\"210.986328\" xlink:href=\"#DejaVuSans-73\"/>\n", "      <use x=\"263.085938\" xlink:href=\"#DejaVuSans-20\"/>\n", "      <use x=\"294.873047\" xlink:href=\"#DejaVuSans-31\"/>\n", "     </g>\n", "    </g>\n", "   </g>\n", "  </g>\n", " </g>\n", " <defs>\n", "  <clipPath id=\"p2bbf45a42c\">\n", "   <rect height=\"217.44\" width=\"217.44\" x=\"58.523438\" y=\"22.318125\"/>\n", "  </clipPath>\n", " </defs>\n", "</svg>\n"], "text/plain": ["<Figure size 288x288 with 1 Axes>"]}, "metadata": {"needs_background": "light"}, "output_type": "display_data"}], "source": ["@torch.no_grad()  # Decorator, same effect as \"with torch.no_grad(): ...\" over the whole function.\n", "def visualize_classification(model, data, label):\n", "    if isinstance(data, torch.Tensor):\n", "        data = data.cpu().numpy()\n", "    if isinstance(label, torch.Tensor):\n", "        label = label.cpu().numpy()\n", "    data_0 = data[label == 0]\n", "    data_1 = data[label == 1]\n", "\n", "    plt.figure(figsize=(4, 4))\n", "    plt.scatter(data_0[:, 0], data_0[:, 1], edgecolor=\"#333\", label=\"Class 0\")\n", "    plt.scatter(data_1[:, 0], data_1[:, 1], edgecolor=\"#333\", label=\"Class 1\")\n", "    plt.title(\"Dataset samples\")\n", "    plt.ylabel(r\"$x_2$\")\n", "    plt.xlabel(r\"$x_1$\")\n", "    plt.legend()\n", "\n", "    # Let's make use of a lot of operations we have learned above\n", "    model.to(device)\n", "    c0 = torch.Tensor(to_rgba(\"C0\")).to(device)\n", "    c1 = torch.Tensor(to_rgba(\"C1\")).to(device)\n", "    x1 = torch.arange(-0.5, 1.5, step=0.01, device=device)\n", "    x2 = torch.arange(-0.5, 1.5, step=0.01, device=device)\n", "    xx1, xx2 = torch.meshgrid(x1, x2)  # Meshgrid function as in numpy\n", "    model_inputs = torch.stack([xx1, xx2], dim=-1)\n", "    preds = model(model_inputs)\n", "    preds = torch.sigmoid(preds)\n", "    # Specifying \"None\" in a dimension creates a new one\n", "    output_image = (1 - preds) * c0[None, None] + preds * c1[None, None]\n", "    output_image = (\n", "        output_image.cpu().numpy()\n", "    )  # Convert to numpy array. This only works for tensors on CPU, hence first push to CPU\n", "    plt.imshow(output_image, origin=\"lower\", extent=(-0.5, 1.5, -0.5, 1.5))\n", "    plt.grid(False)\n", "\n", "\n", "visualize_classification(model, dataset.data, dataset.label)\n", "plt.show()"]}, {"cell_type": "markdown", "id": "34a5111d", "metadata": {"papermill": {"duration": 0.21845, "end_time": "2021-12-04T15:54:19.694395", "exception": false, "start_time": "2021-12-04T15:54:19.475945", "status": "completed"}, "tags": []}, "source": ["The decision boundaries might not look exactly as in the figure in the preamble of this section which can be caused by running it on CPU or a different GPU architecture.\n", "Nevertheless, the result on the accuracy metric should be the approximately the same."]}, {"cell_type": "markdown", "id": "a501a3ba", "metadata": {"papermill": {"duration": 0.219567, "end_time": "2021-12-04T15:54:20.134648", "exception": false, "start_time": "2021-12-04T15:54:19.915081", "status": "completed"}, "tags": []}, "source": ["## Additional features we didn't get to discuss yet\n", "\n", "Finally, you are all set to start with your own PyTorch project!\n", "In summary, we have looked at how we can build neural networks in PyTorch, and train and test them on data.\n", "However, there is still much more to PyTorch we haven't discussed yet.\n", "In the comming series of Jupyter notebooks, we will discover more and more functionalities of PyTorch, so that you also get familiar to PyTorch concepts beyond the basics.\n", "If you are already interested in learning more of PyTorch, we recommend the official [tutorial website](https://pytorch.org/tutorials/) that contains many tutorials on various topics.\n", "Especially logging with Tensorboard ([tutorial\n", "here](https://pytorch.org/tutorials/intermediate/tensorboard_tutorial.html))\n", "is a good practice that we will explore from Tutorial 5 on."]}, {"cell_type": "markdown", "id": "96f1e6ba", "metadata": {"papermill": {"duration": 0.227432, "end_time": "2021-12-04T15:54:20.580697", "exception": false, "start_time": "2021-12-04T15:54:20.353265", "status": "completed"}, "tags": []}, "source": ["## Congratulations - Time to Join the Community!\n", "\n", "Congratulations on completing this notebook tutorial! If you enjoyed this and would like to join the Lightning\n", "movement, you can do so in the following ways!\n", "\n", "### Star [Lightning](https://github.com/PyTorchLightning/pytorch-lightning) on GitHub\n", "The easiest way to help our community is just by starring the GitHub repos! This helps raise awareness of the cool\n", "tools we're building.\n", "\n", "### Join our [Slack](https://join.slack.com/t/pytorch-lightning/shared_invite/zt-pw5v393p-qRaDgEk24~EjiZNBpSQFgQ)!\n", "The best way to keep up to date on the latest advancements is to join our community! Make sure to introduce yourself\n", "and share your interests in `#general` channel\n", "\n", "\n", "### Contributions !\n", "The best way to contribute to our community is to become a code contributor! At any time you can go to\n", "[Lightning](https://github.com/PyTorchLightning/pytorch-lightning) or [Bolt](https://github.com/PyTorchLightning/lightning-bolts)\n", "GitHub Issues page and filter for \"good first issue\".\n", "\n", "* [Lightning good first issue](https://github.com/PyTorchLightning/pytorch-lightning/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22)\n", "* [Bolt good first issue](https://github.com/PyTorchLightning/lightning-bolts/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22)\n", "* You can also contribute your own notebooks with useful examples !\n", "\n", "### Great thanks from the entire Pytorch Lightning Team for your interest !\n", "\n", "[![Pytorch Lightning](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAByMAAAE8CAYAAABjFR0gAAAACXBIWXMAAAsSAAALEgHS3X78AAAgAElEQVR4nOzdT2ydV3rn+eP3FQLYoVJsZFEgPU4pjYDCAO5QobxLai6dypKKlFQ2Mroj2inImwnEkrwZIl2kqgOtJJWcrGTUlKikoZpFqkr+s0zGZLuzk1xiV23kDiZyKkVNLYKRIkWFALqvB7/X5zBH9+Wfe8lz3r/fD0BI5qXFe+/73vc95zzneZ7nPvvsMwNsZ3F6Y9YYM2sfvmeMWb2wPnlvmx8HAAAAAAAAAAAANhGMRMHi9Ma4MWbBfn2h8APGvGuMWb6wPnmn8AgAAAAAAAAAAABgEYzEMxanN+YVaDTGfGmId+a6ApYX1icfFB4BAAAAAAAAAABA5xGMRG5xeuOIMeaKMaY34jvyUP/fhfXJ5cIjAAAAAAAAAAAA6DSCkR1nS7IqCHlqn+/Ep8aY+Qvrk6uFRwAAAAAAAAAAANBJBCM7bHF6Y3mHvpB7tWaDkve6/v4CAAAAAAAAAAB0HcHIDlqc3pg1xqwM2Rdyr95W70n6SQIAAAAAAAAAAHQXwcgOWZzeOGSDkKP2hdwr9ZNcuLA+udL19x4AAAAAAAAAAKCLCEZ2gO0LqZKsZyp6tes2KEk/SQAAAAAAAAAAgA4hGNlyi9MbCzYQGbIv5F69a4OS9JMEAAAAAAAAAADoAIKRLWX7Ql4xxkzX7BU+tM/rCv0kAQAAAAAAAAAA2o1gZMvYvpAK9h2v+Sv7VBmb9JMEAAAAAAAAAABoL4KRLWH7Qqok61LDXtGaLd16p/AIAAAAAAAAAAAAGo1gZAssTm/M276QX2rwq7lug5KUbgUAAAAAAAAAAGgJgpENZvtCKgjZa8lLemh7SS4XHgEAAAAAAAAAAEDjEIxsIFuSVX0hT7X0Jaqf5PyF9cnVwiMAAAAAAAAAAABoDIKRDbM4vbFse0N+oQMvd80GJe8VHgEAAAAAAAAAAEDtEYxsiMXpjRM2G7LJfSH36m2Vo6WfJAAAAAAAAAAAQLMQjKy5xemNIzYI2Za+kHv10AYkrzTz6QMAAAAAAAAAAHQPwciasn0hVZL1TNffiwHrKlNLP0kAAAAAAAAAAID6IxhZQ4vTGws2ENmFvpB79a4NStJPEgAAAAAAAAAAoKYIRtbI4vTGrDFmpaN9IffqvMrY0k8SAAAAAAAAAACgfghG1sDi9MYh2xfyeNffiz361PaTXGnkswcAAAAAAAAAAGgpgpEVsn0hVZJ1qbNvQlhrNihJP0kAAAAAAAAAAIAaIBhZkcXpjXmbDUlfyPCu236SlG4FAAAAAAAAAACoEMHIktm+kApCTnfqhZfvoe0ludy1Fw4AAAAAAAAAAFAXBCNLYvtCKjB2qhMvuD4+tVmSN7v+RgAAAAAAAAAAAJSNYGRkXl/IBUqyVmrNBiXvdPg9AAAAAAAAAAAAKBXByIhsX0hlQ36ptS+yed7WMaGfJAAAAAAAAAAAQHwEIyNYnN44YvtC9lr34trhoQ1IXun6GwEAAAAAAAAAABATwciAbEnWK/SFbAz1k5y/sD652vU3AgAAAAAAAAAAIAaCkYEsTm8s0xeysd61/STvdf2NAAAAAAAAAAAACIlg5D4tTm+csNmQ9IVsvvM6lvSTBAAAAAAAAAAACINg5B7RF7K1HtosyZWuvxEAAAAAAAAAAAD7RTByRLYvpEqynmnUE8eo1m1Qkn6SAAAAAAAAAAAAe0QwcgSL0xsLNhBJX8juuK5jTj9JAAAAAAAAAACA0RGMHMLi9MasMWaFvpCd9dCW5KWfJAAAAAAAAAAAwAgIRu5gcXrjkA1C0hcS8qnNkqSfJAAAAAAAAAAAwBAIRm6BvpDYxZrtJ3ln5x8DAAAAAAAAAADoNoKRAxanN+ZtSU76QmI3121QktKtAAAAAAAAAAAAWyAYadm+kApCThceBLb30PaSXN72JwAAAAAAAAAAADqq88FI2xdSQcjjhQeB4X1qsyRv8p4BAAAAAAAAAAB8rrPBSNsXcsEYs1R4ENg79ZOcv7A+eY/3EAAAAAAAAAAAdF0ng5H0hUQJ3jbGLNNPEgAAAAAAAAAAdFmngpG2L6R6+/UKDwLhPbQBySu8twAAAAAAAAAAoIs6EYy0fSEVhDxVeBCI71NbunWV9xoAAAAAAAAAAHRJq4ORXl/IBUqyogbe1blIP0kAAAAAAAAAANAVrQ1G2r6Qyob8UuFBoFrn1bOUfpIAAAAAAAAAAKDtWheMpC8kGuKhzZJc4YABAAAAAAAAAIC2ak0wkr6QaKh1G5SknyQAAAAAAAAAAGidVgQjF6c3lukLiYajnyQAAAAAAAAAAGidRgcjF6c3Tqj3Hn0h0RIP7flMP0kAAAAAAAAAANAKjQxGLk5vHLFBG/pCoo0+Vclh+kkCAAAAAAAAAICma1QwcnF6Y9wGIekLiS5Ys0FJ+kkCAAAAAAAAAIBGakwwkr6Q6LDrtp8kpVsBAAAAAAAAAECj1D4YSV9IIPfQ9pJc5u0AAAAAAAAAAABNUdtgJH0hgS19arMkb271IAAAAAAAAAAAQJ3ULhhJX0hgKGs2KHmHtwsAAAAAAAAAANRVrYKRi9Mb6gm5TF9IYGj0kwQAAAAAAAAAALVVi2Dk4vTGrDFmhb6QwJ6on+TyhfXJK7x9AAAAAAAAAACgTioNRi5ObxyyQUj6QgL7p36S8xfWJ1d5LwEAAAAAAAAAQB1UEoy0fSFVjvVM4UEA+7Vmg5L3eCcBAAAAAAAAAECVSg9G0hcSKM3btnwr/SQBAAAAAAAAAEAlSgtG0hcSqAT9JAEAAAAAAAAAQGWiByPpCwnUAv0kAQAAAAAAAABA6aIFI+kLCdTSu8aYBfpJAgAAAAAAAACAMkQJRtIXEqi988aYK/STBAAAAAAAAAAAMQUNRtIXEmiUhzZLcoXDBgAAAAAAAAAAYggSjKQvJNBo6zYoST9JAAAAAAAAAAAQ1L6CkfSFBFqFfpIAAAAAAAAAACCoPQcjF6c35tVzjr6QQOvQTxIAAAAAAAAAAAQxcjDS9oVUEHK68CCAtvhUWc/0kwQAAAAAAAAAAPsxdDDS9oVUEPJ44UEAbbVmg5L0kwQAAAAAAAAAACPbNRhp+0IuGGOWCg8C6IrrNihJP0kAAAAAAAAAADC0HYOR9IUE4Hlorwf0kwQAAAAAAAAAAEPZMhhJX0gAO6CfJAAAAAAAAAAAGMozwUj6QgIYgfpJLlxYn7zDmwYAAAAAAAAAALayGYxcnN44YYxZoSQrgBG9TpYkAAAAAAAAAADYSh6MtGVZP9zicQAYBgFJAAAAAAAAAABQ8Nz/8es/HTfGqMzilwqPAuVYpz9p4z00xhy6sD75oOtvBAAAAAAAAAAA+DeJMeYEgUhU6PqF9ckjxphPOQiN9gV7LQEAAAAAAAAAANiUEEBAhRSInF+c3jhEQLwVZrv+BgAAAAAAAAAAgGcpGDle+C4QXx6ItL/lCO93Kxzq+hsAAAAAAAAAAACepWBkr/BdIC4/EGnIqAMAAAAAAAAAAGinhOOKkg0GIg3BSAAAAAAAAAAAgHYiGIkybRWIlOnCdwAAAAAAAAAAANB4BCNRli0DkYvTG2RFAgAAAAAAAAAAtBTBSJRhy0CkRTASAAAAAAAAAACgpQhGIradApFypPAdAAAAAAAAAAAAtALBSMS0WyDSkBkJAAAAAAAAAADQXgQjEcuugcjF6Q1lRX6h8AAAAAAAAAAAAABagWAkYhgmI9JQohUAAAAAAAAAAKDdCEYitPUhA5GGEq0AAAAAAAAAAADtRjASIa2PGGAkGAkAAAAAAAAAANBiBCMRSh6IvLA++WCYf29xemPcGPOlwgMAAAAAAAAAAABoDYKRCGGkQKRFViQAAAAAAAAAAEDLEYzEfu0lEClHCt8BAAAAAAAAAABAqxCMxH7sNRBpyIwEAAAAAAAAAABoP4KR2Kv9BCKlV/gOAAAAAAAAAAAAWoVgJPZiX4HIxekNsiIBAAAAAAAAAAA6gGAkRrXfjEhDv0gAAAAAAAAAAIBuUDByjWONIYUIRBr6RQIAAAAAAAAAAHTDAY4zhhQqEGkIRgIAAAAAdnHIfsWyygEY2nKkf3fFGHOv8F10gSpmjUd6nXeMMSHWrgA0R8wxA9cUIBCCkRhGsEDk4vSGBpxfKDwAYC/0eTphA/y9hr6DO2XnuwWiB3bwd4/Fik6Yt19ongX7WQXQzWt3rAUgghXdpfNqKeKrf67wHWwn1nFY5fPdWVcizmFfZbMB0DkxxwxcU4BACEZiN58GzIg09IsEgjrhBlv379/Pv5pkbGzMTE1N7TQBLTyWZdnjJElu24HgKgPCVjq01bFHI8Ta3Q6g/uYjXrsJVnTcBx98YDY2NoK9CceOHTMTExOF72Nnn3zyibl06dKOPzPKMZibmyt8H93zzjvvBHvNr7zyipmZmSl8H0B3hBwzcE0BwiMYiZ08VLAjYCDSUKIVCO/NN980t2/fbtU7e/DgQQUq878fPnw4D1xqIDgxMTE2MTHRswuebtfbu8aYm/aL0hkt0cbzuq1Onz6dfwGArt2hEKyA8/777wcdE9gxZeH72Nnjx4+DHYejR48WvoduChmMFAIHQLeFHjNwTQHCIhiJ7Ty0GZGhy60RjASwq0ePHm0OIN2fbqI6OTmZL2C4r4mJiePGGH1dM8Zct+XcyJgEAKBkIRd/CFYAAAAAQHsQjMRWogQiF6c3VHrvS4UHAGAEKrmhL+14E2VQKntCX2NjY6eMMaf6/f6P0zS9ZAOTAAAAAAAAAICKJLzxGBArI9LQLxIILv9MNa1XZGiuf83s7Kw5f/68+fjjj02api8rUzLLsn+wvTUBAAAAAAAAABUgGAlfzECkoUQrENy4sZmC+JyyJdW7Tj2rFJRMkuQlY8wPbNlWNkQAAAAAAAAAQMkIRsKJHYg0BCMBlEU9q/ygpDGmZ4z5oTFmmYMAAAAAAAAAAOUhGAlTRiBycXpDGVzThQcAICIXlLx8+bJ5/PixftFSv9//EVmSAAAAAABUapy2KgDQHQQjUUZGpGHhH0CVbty4YY4dO2bW1tbyfpJZln1kjJnnoAAAAAAAUCrNxW8aY/4/Y8wCbz0AdAPByG4rKxBpKNEKoGqPHj0y586dy7MkkyQZM8ZcM8ascGAAAAAAAIhKGZArWZY9snPx47zdANAtBCO7q8xApCEYCaAulCX52muvubKtp+yOzHEOEAAAAAAAwahK2pV+v/+PxpgfaP795MmTsQ8++CCfkwMAuuUAx7uTyg5ESq/wHQCoyCeffJKXbb169aqZmpo63u/3P0rT9MvGmAccEwAAAAAA9uSQsiCzLDubJMlL+gfSNDUKQK6uruZfAIBuIhjZPaUHIhenN8iKBFA7Ktv65ptvuoDkywQkAQAAAADYk3lbijUvv5okiVlbW9sMQGr+DQDoNoKR3bNQckaksWUZAKB2BgOSxphVrlkAAAAAAOxKWZDzNgvyoH74/v375rvf/W4egNzY2OAdBABsIhjZLa9fWJ9cqeAVkxkJoLYGApLTaqpvd3UCAAAAAIBnjasXpHpAGpsFqTKs77//vrl9+3bhhwEAMAQjO6WqQKQhGAmg7gYCkppQ3TPGLHPgAAAAAADYNG5bnLzssiAVhKQMKwBgN8kuj6MdKgtELk5vqNzhFwoPAEDNaPK0vLxsHj9+rCe2xEYKAAAAAACesaBApPpBHjt2zNy4cYNAJABgKAQj26/KjEjDYj6AJvnkk0/ygKRkWfaeLT8DAAAAAEDnqT+kNvC6eTMAAMMiGNluVQci5UjhOwBQY2q0r1IztgF/1ddQAAAAAADqYFbzZM2ZyYYEAIyKnpHtVYdApCEzEkATXbp0yRw9elT9I48bY04YY25yIFGmqakpc/DgwUa855OTk4XvAQAAAGidPOHg9u3bHFnU0uHDh4M9Lea5QHgEI9upFoHIxemNQ8aYLxUeAFB76v0wMTGx7dNUkGS3QZ7+/+3+jY8//rjwvbt3727urnSTmyonOQpIXr16VWVo/iJJkl8xxjwo/BCiOXfunOvfWXs6d3W+hPTWW2+ZmZkZTjAAAFALGpfcunWLgwF0W97GZGNjo+vvA2rq7NmzHBqgxghGtk9dMiINWZFAcykYGSgQsjb4jX6//8szMzMvD35/u9+nPo4KSupL5WDKot+ncq0nT55UetqCMYamGCVSZiBy5xv0NtwrfAcAALRBYUwfCJv9AAAhxFwsYp4LBEIwsl3qFIg0BCOBVnh1hxexp8FemqaF71numjFuy78ou/rQ1NRUT4GpkydPmvv375t33nnHvP/++4X/OQb9LgVmn3/++bfSNF1hEFqK5YYFfj8rfCcsguAAAKBqzO0BAHW2GjkgCSCAhDexNeoWiDRMWIBWWN3hKzT3u27aAMy8vY78O2PM7xlj3lXZ16WlpTwYOTsb/xKjsrE3btxQAPUXCQoBAAAAAAAAwOgIRrZD7QKR9IsEENADG6A8YYz5VWPMdQUlL168aJaXl/P+lTGpVKt6F/b7/T9wPTIAAAAAAAAAAMMhGNl852uYEWnIigQQyT2bMflqlmU/mZubM1evXo0akBzIjlwo/AAAAAAAAAAAYFsEI5vt+oX1ybqWDSQYCSCm1SRJft0Ys6Z+krEDkh988EH+Z7/f/1rhQQAAAAAAAADAtghGNpcCkfM1fvYEIwHE9sBea667gGQsGxsbZm1tTdmRL9pysQAAAAAAAACAIRCMbKZaByLpFwmgZPMuQ/LcuXPRfvP777/v/kowEgAAAAAAAACGRDCyeeqeEWnIigRQgRPqIXny5Elz9OjRKL99dXXVPH782GRZ9vuFBwEAAAAAAAAAWyIY2SxNCEQagpEAKvAgSZI/1K9dXo7XSlcBySRJDnKdAwAAAAAAAIDhEIxsjqYEIg2L9AAqsqpr5cTEhDl9+nSUZ3D79m33V65zAAAAAAAAADAEgpHN0KRApKFfJIAK5WmRr732WpRnQDASAAAAAAAAAEZDMLL+mhaIBIAq3dN1c2xszBw7diz409jY2DD379/XX3uFBwEAAAAAAAAABQQj641AJACM7or+j9nZOMmLn3zyifvrkcKDAAAAAAAAAIBnEIysryYHIh8WvgMA5bmTZdlPer2eOXjwYPBfevfuXffX8cKDAAAAAIA2GbdtOga/2JwKAMAIDvBm1VLTMyJvGmNOFb4LACVJkuT7xpgzyo58//33g/5SLzNSE9DVwg8AKItLfx5Mg/b/e/AzurrN94FQZu2ipVugHN9msfKOMebBwN85L+M5Yr8OuWtEv9//5TRNXx74jev2WDywx8V93WvRewEgnnHvejPuXW9+LU3TF7f5re66c89+rQ7cI1AuF3A8kmXZTJIkL+3227Mse5wkye2B+8adwg8CALC73eaS/pyxceMFgpH1824LSrMuE4wEUDFtijgzNTUV/Fk8evSo8D0AUbkBeL44tMuC3qDB/q5L7i/9fv+naZresgP41ZYEgma9YOxWE5f9WNjHwpoLArnnc8h+bWdwUvXAXtfrFhDyz8tXRjgvzRbnZs47L1e9BWmMTufXCfu15Xudpqn5+OOPN/9b/aanpqamvR857v6SZdkju9Fp1Z6LBAnQJaHvJ757DQ/2H/LuA7+z3X3g5z//ub+hcdPAdcddq/KxSr/f/3Gapn/jXXdGeU473WP3o+nHaztagzuRZdlXkiQZcz+TJMnmfeLWrVuF//Xw4cN5JZ6JiYmxiYmJnn+/8e4bN0c8fjGEPCdinFtHShiHx+nhEs7KHj5bux3X7TZltnWzQ8xrX9Pes+XCd3Z3ZIfqX1td+1d3eAzDc+MIt5Fpy3nLgKXCd4xZs8fhjre+sVezEa+Z9whG1su6HQQ12oX1yXuL0xtvKxDQ9QMKoDL5jVcTxNAGMiMHaSDx94Xv7oNdmP47+y/cswGBNkwe9F4tZ1n2+0mShK+na8yvMihutG0DCW5BTyWTtTlAf/c3Cbj/1uKQvyFhcnJSi0X5n/qamZnRguGLNuDgBvTveot+TTx/ZreZnISw3eR0K7P22OnP6S0e381Wk7DxPU6sQzthFyyfuXa5wNbGxkb+5Z+Xg+eoHD16NP/Tnaf6U/esgfPSZVt8jyDYUHSOzPf7/T/yMx7v379vbt++vXnd0H/rGO3EXS90nHRcjh49enBsbOyU3XB5zV4rrpDNio7Q4tiHkV7q+Zpc20eRj1EGrzXuPqDrjK43g/eC3eh64+4Jr7zyiu4H+rf1dcYGtr5jrzu7jU8WIq7FNPF4bUfHcSHLsjfc/fxnP/tZfr9wX7vdKwZ594xn7hvevXylovvGfMTxYQhf2Gbs1yUxEjq23ZTpZfOueoGLpo8xY57nrzZszBf6fdjq8zn4O/xgGBsqd6Zx1bydTz6Tfa95ir78DTC6Hzn+GoebP9qNTW5DjH8tWbPzx1GPx4mI44g1gpH1oUDk7IX1yVYsMFxYn1xYnN4YJ0MSQFXUN3JmZmbXsjqj2mVB4Z5doDyuxYfHjx8XfmBUU1NTL46Njbld1r1+v380TdMvN3yyoLJHH2nn8ZMnT7bcKb4XWjjWlzcQRrNsG0hYXV3dDCYMuzCkz6o/cPf/7ijgYBeM8q+JiYnjNgj0LTs2u9LEANDly5fzxdCtXvOoTp8+nX8NobCoZ2wgbqtF2Z2emwvUGbup5OzZs4WfKdkR+9q+6jImdO1yi5WaLI56HfNfv85vnyaYbjFzdnZ2zFvM/POKFzPr6pBfmUUBAR2PDz74IH9vR11MFhdU9o+TjsuxY8d0TDavFRprJEnyDXtMgFZToG2r7LC90P13bm6uaW/XCRvoyxdl3bXGjVF2uq8Nw/3/+vfeeeedfJHR3gf0pcDWGbs4uGavedvdB/IsVv0boTT0eG3nmXuG7ud6z2/cuLHvOYk7D/RviT12z9zLbcbrpSruG7ov7uWeuBWNz0NQ4D0mfQ5mZmai/o4A8s/s2tra5rkzDLcpcyuDmzIHN79tlc3rBS5WmrzWEPI8txtDCt9vAp0bly5dGvqZ7rRZz507jn9+6T3aKhhmA95/42WHd31D5bjdyHTOrXW47HuNrUYZRwzOHR23IUbHxq5tbH7G7Zzl+/bzvVtgMr8mvfnmm4UH9uPq1av5/00wsh5aFYh0VG52cTq/kBGQBFC6JEn+H2NM8GDkEG66YOTyctiNw/r35ubmXrYDiBOFH2iGQy4QqclhyIUS+/4YG0BCcxyxi3vBAgnDcgEH11vWZSPoPLJl0655AaDlpgS5QwUizbOLTdv1yd1yUc8tzu6ygWNLoZ57APP+hNF/bdtNAkNwAVz/vFQQ7NixY35gkiDYwLmnDUBaxAu5COVzizr60uLya6+9ZuymJ10nvpkkyR9u8xkBWkGLZaHGbVoka1Bwa95+xvN5he6L3/3ud6OPUXT/dPcbLfzquqPNQXZx8UMbONi2hHrIMXbDjtd2XIWFPNvDHUfda2O14fDHC/Y+7jJer/X7/T9N0/RPyryP67XWaIyFf5NXHwk5fh/clGm2CGC4DQ9u04MXuNCmzOtN3fwW+jxvajBS4+JQ78NWQW93Pvn3GjeX1p92E4bb6HutyefUPo3bTa1ntWHXVVHQear3MOT9ZzCgqY1E+mzrM97r9V6y978zdlPM/7nbxoNY9wuCkYiKgCSACilo0NONt+RJ10qWZX+mHcwa4IccXGgBVAO7qamp4zsEBupuOUYg0u3YtmVtq+7LguHM2kWhfLeeFoU0KI8VSBiGCwIpoLFVAMhOohoTlAxhh2NRWNTTZzr0pKoimwvP/oTRBQfLtlMQrIrFzBrY8twr8/i4BWbde7QRxh6PD211hPmdJvYAGkMluf9M9wJlL7ispSqCObqvuvuQ5jbKVpiZmdH46YfGmLftNZHrzvZ0LP9CC8FaoNc9Y5QMtBDc8dP4Uvfxubm5F9nMgir5Gx40xnTzac1/JiYm3Nxnt0xsYJObSzsDVUUqzw6vwLILQrpNrbr/7DC/Dkq/R/c6fblNTd6mmG95Gw+u+Bub+v3+r6k1TixJyw5yU023+UOogKQdIANAmSoLFqini0pV6GYfkiYMXqmNJvZrURbLKbdwHJLbrZ2m6cVqXyKGcMhOaLVw39P5cP78+XxgXObgfDcuAKTnpednMwRP2b6wKyP2UGwblVr+HwoGaVHPHb+Y2QUlme33+z/S4qAWnxUY14KvMlGqCkQO0iRWz+d3f/d388B9mqZuMfMfGpwxP4oT9rUWzr0q6Hql46HzxC6+5KVbO3IsgLZy45QfuHuBrrnnzp2rRVaZnoOuO97Y5Iy9LrqJR2+rLJaOGrdVa3QsDyoTUveMsgORPlc9R+eUAtw24/ZD+zy7PLZExTSm0VzMzX20Gc9uGv3QXhOPcIwwCn8+/dZbb+XnlK04c83OucIumNXHrL0vL+ne4z5XuvZXtdbhNjW5eaSek20rdcpubFp1x0Pzy5jjCIKR9XF8cXqjzQFJlQ95vfAAALRTfj0/efJk8Bfn+ubZicGhwg/Umzan5CWRQtMOY4u+XfU1bnfd/f1gELIugZ6tuIH7YFDSBhwWtvhfWsULLrrrjT7HP3QLtHU/fkMat9eODzVB1kRZ1xRNGOtazkwT2S0WM39gJ5JNuzcMY8sF5bqcezpPdM5oYm/7iv6AkuFAIy27cYruBdpoUOXi4U50/dN1R9dD26v5Q3fdCdG33uctSjZp4XjWbpw6rrGbjqUWxeuyaUrnlALcel52bHncPl8CPqicC1p4m61cJnbXN2Rij9yGSp1TXlCybRsx3HrHh5qbaY7mAn912rDrNh4oeULrG95n/MOnT5/+98L/EBjByHo51fKA5AoBSQAdoRIHa65xdGheMK9R2ZH9fv9rxk5uQtKCtDJRbYkJSlTV0+x2mXRN4hb+NHh/8uSJTrpv2V2drV048hYgD9kFiGvuGGqBtgUlWV2mXZ61rV27mqg9p3AAACAASURBVCg3JavEX8x0E8ksy37UskC5O0a1XFD26dqg52cDAWfIdAEa45BdgFvS5/fy5cv5vaDu/fVc1ZSB606U39MwK24xWBunNHar67F0m1n0PO3Goh+6DZzojNpuInPn58CGzK5U40AELrt/YCPGP7QgS/JIv9//yK13aE6pOVodNzP53PqGCxIfOHDgN83OrVr2jWBk/RCQBIB2yK/lCriEpl1lGuBkWfbVBi1ynlC5B020Qy9oeO8xWSj1dMVfEGp6Jp3OXwUcNGj3dnX+sANZktoteUrXnjqVLd2nK36mnY6prq9NtEVm3rdaFAg74x+jugcH9Px0nXNlW+3CBAFJoL60YepHWoBzmfFVlvHci4HrTu0XP0vwJbfBqAkbp/T89DwV8LFB5WsNbcmBvcmDkXUe32yRif0DsiSxH27uMpDd39Tr3nyWZR9pXUDZkLofN21O6QeJNRYiGNk9BCQBYP/ynVUVZres9Pv9n6qXoZrBh+SazdsF56bsSsx3+IYOYCj7dGZmRlmXP/abbqMWDtmswXx3oCt11oJMupzrFafsCbtw1Kbgz5ZcILIFvajG7fVi89ysa6bdqFygfCAQ1ujMXRccaNIx0vN02apamCAgCdTWvN0wNabrp+5xTQ3k6brjMuy6HIz02wA0bTHYlca048ol2k+gTvxMbJcl2YZxJqqzRXb/UgPn0wqgXtM4QusCyoZs8pzSBSVjtFZyCEbWV1cCkg8LDwJAQKEHAl7Z1XuFBwekafptfUcBydC0YGI+L316rgHnyyFXXi/0jk/XKzJN00uFB1GlPMvA3x1Y92ymvVL2hBega0XwZ5BeW4sCkUdsyeBpvZY2npt6XZrU67Onz6B26jbxnNQ9w5VKbOJ5NxiQtIsrAOrjmdLjbmzddNr4pYBkF7WhAofuGV5A8hQlW1E3LqOt6eNM1Ic7pxpYVWTFlXfXmL9pVRV2EjOgSjCy3roQkJwlIAkgkp7dsRfLrsFIVzbUBcxC0o5nr0Rk3Qf/+SQ69O6qgwcP5oHeLMsesXO4Vp7JMmj67sBhuOCPFsHaOCnXtbRFgciP/B5SbT039br02bOlj8aaeE5qMbnpk3o/IGlLHXOvAuphpYWlxzd1NTOyLa97ICB5jf58qBs3zvTaA9DrFPui63fDqoo8M45o68brGAhG1l/bA5J3CEgCiCBfcI2xcH748GH312GCkQ+MMdfHxsai9I70Fk5q3auu3+9/zUQo0ere0yRJLhceRFXm7aJJq7IMhuH6/bhJuQ3+NL0Rf06vq0WByDEFIvWaukClj/RZbGpAsg1cQNKVNGOxDqjcMwuILbi/oYV0XirYI1mW/aXrKwjUieY9Gmda1xjjYD/cmNlt8K1xQDIfR+g67WV0YkgEI5uBgCQAjCYPAMTYnTQxMeH+Okww0rjsyJMnTxYe2C8F97wSPnXdNTafpumLGlCGzkDy3lMyTeph3pU706C8bVkGw3KTcrtL+MM2TMpbkD24GYjUselKINLRZ5GAZLVcBoH5fFH5z1lUBipDIHIPRmlTgXA0l3Wb3J4+ffpfeWtbK1+7aOr1yI0zrWtt2YyJargNvq4McA0DkpuBSAVOu9yjea8IRjYHAUkAGF4+AL5161bwt8zLjLxTeHBr+rm1qakpfyIfjBfwqWvAI39eoQNTs7OzLjD8LgsjtbAZiGRx79lJuQ08EPypzqEsy/6bFvLUf7CrQXK9br1+G5B8ryF9WFpF10WvnBmbaIDyzROI3DfG3CXTfUOtOQ4cOPCbda+Gg/1p8ua/gbnP+8x9sF+uMo9tS3SlJm/oghtHvPXWW61vRRMLwchmISAJALtTtsFxlUOLscigoGK/3/+pLcE6rPzaHaNUq+vDmGXZ2cKD1dOx6Ok4hM5S9fpw1mVg2mUEIrdANloYk5OT7t8Z5ZrrjPf7/feTJDmo7Oym9x/cL71+vQ/qmWmMudnsV9NMWlT2+kdSygwozwnGKmgqV9Ehy7L/wmYi1NXA3IeNb9iXgb7rqgRWdWkbjSO+5cYRZETuHcHI5iEgCQA7yxf3XJAuJAUi1f8xTdNRUy5XFMCcm5vzF9aD0CBIJSzs4nLdSqLku3dDHwu9hzMzM5qQ/8QYs1r4AZTpiOsRyeJekZ+N9tlnn60xKR+dVxp72Gx034p203apR+RuvP6fvRpM6jvJnYv9fv9PuSYApThke+61pQcyOkbzPS+znns3aktzH2/jG1UgsC+uZKttTbRU4XrXETeOUNsFxhGj89ZB7xGMbCYFJO8sTm+0cvJKQBLAPowrQ1CDlRil+F555RX315EDYGmaXtSfCkiGVtNSrToWb+hYrK6GjRe6XpFJknyj8CDKlJe/1O/TLlgG5Vtz2WjPPffcL9meFyiHFuuO67y8dOkSb7lHZYVqMKnvLJ2TuiaonzIl94D4bIb8mII5ocekQFm0udPeu8+wkQV15m36OM44B/ulc8nLDv+LCq5/qrTzl67lR+iKX13hbTAmGNlg01oMJyAJAM9YUDk+Lf7HqN/u9Xzcy0qGsiP/xSsvGowWVlSW1pavqMt94YSOhZ5byGNx8ODBvNxtlmWPKDNYLVf+UosjXe3DN6ya9ryoPX3WvWvmKGVaNYZcop/H1pRhUfGkvvMUFDGfX0e/1vX3Aojsiu696rnnPndAE2ks45WbJ8DTIv1+/9dsoLk1BkoLH+r6Mcb+aE1Jaw4247bs7PBljSNUjazrLT9COdCOl9FZLiA5e2F9ci99dGpNAUm9Nrvo/4WuH2wAuzriFp9jlGg1Nhipcqtpmu6lXOCDNE3/amxs7JQW2EMHb/TvqUymnZxWXr6n3++fS9M0+MLP7OxsXirXGPOdPfaQQxjLbnGvTllnKqWsgLWjIH1d+jmo54U+p2NjY2dsIJ30jG3oWqvPusuCNsa8O0KZVmVl/0WSJPlCSB2O/+B56VOQuopgqSb1mlT3er2X6nLfiEVlgbzduPn7XXUmt85LZUfOzc29aKsaUMoMCE9rCWc0N1BZtaro+q/7gNvUqP8+fPhw/nddj+7evZv/XdclfdGHqho6Rjouul/oz8H79q1bt/LMRB2vqjJjNMfVfE+VgJIkGfm+rTmot7l2aK7qUMzxiu7V3jwvGP/+X1eqlNC2CjN6PVoHOH36tCstTJ9s7IvOJ10jJiYmypxLb44jqmz54cYRujfpGtn0cQTByOYjIAkAtnSCgl8aJMSYKHmTo78uPDg8jWBOaRIZOhipRU39u8qySNO06kXlWReoCj0QsgFXQ3ZZpTYD/1Ut7mkArs+kG5Srh+hO9Fw1MNdCkv6sokybrkt6v65evarP6X+1WZKtDqhrYUmLXu44ub67I3p7xEDZsnbN6ppYxXHWa1RJb/3pvnaj81MLm3q+obPJd6L7pQ2QL9lg2L0dfrwR3GKmO+92WoT0rwt6/8teXNbualu6fYFgJBCevzGl7E0fuv64wM9u94Fer/fMf7sWB7omUXkiLh0fHadhgmCDY03Nc3R8yrxv6/fYjSyKlJ4YtUrMftqFaA4Wqz+8jsHS0lLh+x3R2uoUCh7p2E5MTJyy4xw2YpajleeUrn/aBH3x4kXz9OnTPz1w4MBvFX4osKaPI9xX3RCMbIeuBCRX7GsFgEErWtiPufisSaq1n9KgWuhdm5iY6GlQEXLhcyDLYuTJaWD5zsfQCyh6z+zC8lobFs2bKnbgfyduwWhwwG0zlv/OBvf8DLr8g/vCCy8cnZmZGfMXkpQV5haRyqLPvHa1nzx58kUbYGtdmS1XSlkLXttMnNYK3ylyx/HmCBmRxt+9WmbGrgJgyuK0u3UHH17b4rx0tFhwROdnr9cb03mtxThdy3Vuxg6O6fOrhaKzZ88au8HjROGHGkLnnI7BFuecO9/8D7o2VIy/8MILMzMzMwfddUHnjQKE+oyWcW1zu5enpqambQkz7mtAOPnGFN3ry7zP61qkgM3AvWDN3gPueJ9z96SO2HvBuP277glfmZubG9N9VJuYdD/QdYmMyXA0p1DFCnf9z7JMNTL/xh4Xd5z8a7I7PrpWz2ZZ9tszMzMv6f/XMSrz3qHz2QYVR5nv7TcQNKuNSy64FPJ1agylsY+OQZIk/3mb8VKbHWnzi9M5YwPNy/QpL01rzyld/7QRZGZm5jdLqCxSp3HE+sD9yXjX9EP2a8txhMtsr9M4gmBke3QpQ5KAJACfBiDHtaAXa/FZi+u6kduAx36DfFrw7akPWuiFZg0y7OR0vsJgpAZCp1QeM3Qw0usdR1ZkdTZ7JpQ1KNfnT0EGHX+3az3Lsp8kSfJ9V6JFwdGdaEejHaBrLHGi3+//Tq/Xe1HBH52rmiiXlX1QUYmZ6LSYpInTwK77de813omdCdrv9/+8zED54GKm9a59vavDLqjZ81OTx/l+v/8H2lSi91FBSd3XYr4WLaDq8zUxMXHcfj4adT5uMWFf9xZdd3z/7ft+yL5uLbTmlQt0rdH7UkZ/OR1jGww+wb0NCOZQv99/6+c//3lpG1O2uBat2WvRzV3uff51Kh+722uTrkvzL7zwwldPnjw5puu0FhJ1XaIP8t5pTKkxgrepLT9OSZLstqD9wLs/rvj37eeff/5rp0+f/kWdA/q3Y28k0vhbi8s6N5IkGbb05WBwdeRfq3G0Wg1oDBtyzOxK8idJ8scNqRLwWeE7AWgsqUoNo9AcRl+DXNnGOpRsdK1ktBmbjVdb0rX+w60e2K+9nFMK9G1F55POqyoqiQxSlSF9ZVn2zSGu3Xulth/nnjx50oRxhH99b8Q4gmBku7Q9IPmAgCQAz7i92fY0yNaCcKybqgYG5vN+Dn9VeHB0NxVIUZ8uLd6HnBxoYKgJiV1UrmqwHyUrUu+VFg5sEKrKrM8uq2RQrh3nXums61qwT5JkxyDDNh7Ya8ZNG7zMB+gqHaQduxr8l7GIVEWJmdgUvNH7p+OUZdmjJEkuV1D2c96Vh44dKNdips5LF3i116VvDDFZ3InO6YU0TZUtO68J9tzc3Eta9NN5GfM1NXHXuo6BPkdeIPi6ff6jnnP37Lm6YjOVF1544YVz6nHk3vuYfZx0XG0wcp5gJBCMNk79ojYVxF6E1/hU14kA16JB+aaWJEny65J6BJ48efKgC3jVsexa3SlzXvcNLfTa+/Yf7nMDjrtv63hf0XhSC+S6p8bezGKzI8dK3kSkMc6ZLSpA7Ivre1ZxVZ+RuDLvIWjsbKs6ZMaY/2mM+X+H/We/+MUvHp2YmNiyvrBfRUbP1VVnqSIwOTDOpHfkFrYLLO/FwDn1o2HnJlmW/XtlfBce2KZMta5DsXvJbkXnss2OfCniNVDrDWOMI+IgGNk+LiB54sL6ZOt2nBCQBGDN2p5rL8YORBpvx2aohUK7UP8t/buhgzqaZHgLm6X3jrQ9K/PnEZJb9LfvHapR9aB8IXBmnctec4tIx7WIpKzP2Jl1JZeYicbPMOj3+/9ijLmYJMmVKnph2t2xeZA3JmVD6rptA695EDLCrtwVu+liQWXR9JouX76cZ+vFMLBrvfbZkf6Ccr/f/3Gapv8pUFm3B7Ykk87hlampqeNaQIvVI8vYEuteqdbxtveRBUqQV+hQsCD0WHSQ27Cg+8HTp0//9sCBA/97hBKT/nVp855QRuZ8m+i+oeu53dx23S7OhrrePnBVabIs+0ttZnHj2Fi0GG/nRo2raLCDxtz/dN/W2CAEjSs1/zDGfDTqhjCb/bQVV1JYf57QGEOfAa09aP6h31dmdpvGmdrEZ7N5Q8/nWkHvUahNDN459T9HKdm6w/m0bZlqrTvpfqTnXmag22VH2vWB0NfA0sYR/sbrksYRy8pwd+OImPepnWx7pqHRNJlVWdNW1om2WZ+ztgwTgG45ZBftP1QgUoMDZeTEnIjbpuvGBkNCbfJYUV8M/dtazA/JZSQqKBj0Hx6OMpNe1MAm9DHRcbbBjkYGbVqgtEG5JlAKumiCo0CDMeY37CJPrInrPVsi8VWbtZz//i16zwVlJ1B5EC3qL4pE1y69BhuIVEDoZRvYrWKBYV49PXTtiZnFpmu2XrNdzHw7SZJfj3hNemDfT52XjzXZjzlh9BZAat3H1C0ou/tymqb/IdKkXdeEr+tY6/fFvB54C4L0UgL2L79Q6j4ec36gcakW8+z94LytchCz1527J/yGxioKROl+FHoe0UZ6j7xA5PmIY8qbSZJ8+bPPPvtnHZ9QwaqtcN9opZABFdd3fdkGkf6dMeZ1XTs0v3IZvAqal0VrFNrU2uT+5A30fwV6yq5M9Yqdc/2KMeZXdT1VRRxd79577708qFbWPclVBFO7JhssDSmfC8UeR+geoYxhe2/6eknjiAV/HKHXWMU4gmBke33BZkgSkATQBrN28PP3rh+hsiHLKBXpTSRDLjg/SJLkexp4aFd1SBowaUFeQcEKyqDkvy909o4CAHqvbJlcdlJWo5TFvYFgz/lIgYbtrNrg0nUFOsoIQGh3soJoTStZ5AKR9v1RQOjLVfaAcQHdmGXRdG5qwqjAoDHm9yJk6m5H5+WX9Xs1aXRlw0PTIpE3qT9Uwusa2TYLyjFpB/HrLiAZa7LuLSq3ct4GlEjl5L8ae+OUNoZog4i9H7xaciWSO3assqZ7MAHJ3blqBl7pu5juPPfccz2dG5pDhp7nOcpA0nmeZdnRwoNA0QNbdUNBpFe1iVBBSc3rYo0rB3ktXAhGtsM9m233SzbQ/UiZt2Vs6HW8cyrkfEDjiDfKGEfoHuGNI8ps1VD5OIJgZLsRkATQVEdc6cJ+v/+Ptql3HoQ8f/58Pmguo7SIlxW5FqH8Qz4ZjrFrNtLAbDdHXP/O0JlJocvkYmTjZWRFDgR7Xq+izLBXaivPiNKEKtZCkvGyI5sWjNQEygUiI2etDuOEAroqrxurPJB/biowWEFfozsuIKnnEeuc9D7ftcyOLHlB2dFGpLf1e2NtgPLumWS4APujjI2xmD2slBGpjSHe/aCKEpluHeQ6Acmd6f7tVdooa6yle3Ye4dF4Kdax0b3DZpqFzgpCu63azZ75XEfjyjJKNep8tQH0rxQeRNO5QPfbWjsrK8itDfhWyPGz5pUHY44jtP7X9XEEwcj2cwHJVu4+UUDywvqkFsDfLjwIIITVIb+W9/B1c/Df1o4qY8xnxpgfGmOuKQCiDD8tMr/11lv5oMYLtEWlm7FKTVgxJq/aTbamAZvKUobkla3olZjhki9ehw5W6b2xQY+1EjPk8KzNjNdYg/Itgj1Vl+O9YgOifuAtOH1WbSCi15SsKC3EutKsNQlabZbSiUHXoIFzs6rrkBY31Rsx2uKm7q92oeiNwoMVq2hB2VlwWQQxAsEuiN7v93+58CCAofX7/T8yEcaijj7/LiOy4vuBM+8vJOJZ/lzO9hYu06rbyBKrXOutW7fcX8mqx15orvMbrqxwGQFJ9cy3AXQ2X7WPKwH6uts8GTsgqfGzV9UliH6/n980Yo0j9J64jMi6jSPKqDrnEIzsBgUkf7A4vdGoXfejuLA+uWBTmwGE1Rvya2kPX8cH/+2f/exnYypbqOCjSqKpFOsrr7ySTyQ1eC2TBgmuL1jE8oP5qF+L+6GVnOGymTkXOljsDWLpFVmRLMvOmmd3Hwalwa8mLObzxvl1GJQ7K36Jxlh9VeqejebTwp6ujerfmqbpsRqUTdZmi54mojGy5fV63cQsSZI/rsG5edMtbsZYNNJmA7tQdLBu2bpuMbeCBWX3e3X8/U1KQWnsY/uuAtibI/oM6bMUI0teYwB33bVZb3UZq8xrs4TGUrGuT02lyipeNn0Vx2vZlS6M3JuvlqXV0QgqK/wlXUPKCEhSlr4TVvxqLrEDkt4aYYgA9yGNI7RRONY4wt2n6ziO0KbLmL2OfQcK30GbXVuc3lDgrnULuovTG+OU7wPCGeUmpMXavWQNafE4Vkm9/VImjCaOmkAmSRJzVK5s0J/0er2XNDgJ+X4oKKjj+MILL7yRJEnsIEeUzCSdW5oY9fv9n6ZpSjCyGrMxS2C6HnDW6zXMftV5d2RsbOzMxYsXo2wc0GdVE5MXXnjhq0mS1HrjmJ6nXdi7WGWPSE9e+SPW7lXv9b5dow0RytL7Sq/Xe1nnjj4/ITeB6Dqu6659b2vxmr2S6VUtKBub5fLuxMTEcWVHlb1BCsCu8vtnrAoqWqT3+tXW6gKgvs1Zlv3DyZMnD+raVEYriybwFsGrKPsvD5IkuawNu5pXhs468Y4zwUjsxwNdQz777LNP5+bmfknnVazr6N27d91fZ1m/bTXXXuKjpaWlMR330G18nIFWB/u9N+fzylgbsL1xxNfrOo44ffr0QV0DYo8jCEZ2T+sCkjYQqQ/ydOFBANEpk6JNk14FR7ydz38YO/MnSZJv6NocepLqMlzm5uZchku0636/3/9amqbBB26uV2Sapt8uPIiy5It7sRbevUF5nYI9gxRsPzI1NdVTgN8LngZjP6tjdhJUdj/CobjNAXaTRi0WEFSST9eeGOenNqXY1/uTyJtSRmazA29OTEx8SbuOXWZxYMdt1nvV2a9+3+Cqj4PO++Na4A59zmmhRjuSAy2mAJ2TZdnvJ0kS5X7gl4lO07RW9wPrgZ2z/EDjqjJ6ddWdNo14m1iq3Dyl+8aSjkmZJfCAET147rnnVKnqh9qIp7WdGJtQvcARAfT2u2OrylzTtU8bemO0e/EC3PvOto05r3TjCNt6qI6B+FLHEZRp7aZrbSnZSiASQGi6+XqT1zKCAje1uK8bfugeYF6mYsxr/rz6eioQGXrSQonW6vX7/d8xkTINtFBUs96DO5lXuRlNpGKU2vImPbXt8e0+j3aXf+UBqtildFQm3PzbhpE6vF7fHTvp1s7ad+3ENsZX5T199Hmz1RfWa5CN66oZBL9fx+rHC3TEIVVxUInWGJ+lqstED+mmzd6OUsWhaWo0h9D44bo23oXuP+6d65S8RAgaW349VisAxwYkWb/thhV3X/I2FgblBbjH9/nvRptXujYnVp3XPDbHEQQjEcsVG8hrLAKRAELT5L2C4Ih2IX1HA//QN30NpuwArRdxB2KUslgDZfnqUA6yi9R/6UWVaI3B9Uxwvdhq7l6SJJf0OY3RS0HBSPVcVWZH4cGasKU7TY02B0QrpaOsSJcFU+PNEA/sztoTNmgY46vyLF1tWrBqcRySJPm+secIgNrILxSxshlqUCZ6WPncpayeT3Vm53M/rUmmef4cvPtZEAEX4QHniusdF2uco/kOOkUbeh/pvhSrd6633rUf0cYRA5n6nR9HGIKRnfaFyJkyURGIBBCaFhvOnj2rYMBju/O5zEyYvFRDjB1jXi+1GMFVBTh7GgCGLtVLVmQt5MGeyIt77zaoJOGysqIUlIsxmdJnKEmSg3Xc4V6z7DQnnzTeunWr8MB+uetPmqbUVKuYl0lSl+tE/jxCZ7gM9LwBMJr8c+OVawvGW5CrY3nWQfdcFl6XS7W6IEqapn9deLAa+X3j8OHDNXk6wPbcJlFXISS0gb6RaD/XOzdagCtQgDs/H2O0n2riOCJ2diTByG6rbSmwnRCIBBCaFhVdllaSJMcq2LF0z5VECL1r1su4eiPCztl8QOUFPIPQ8fCykuidVZ0yBuV1L8/6DFuyM1p2pFW7ybm3O7o2/SyzLPuKrm1eECcI1xvTZlSwGaJiXuC/LjuJ8+fxyiuvFB7YD8q0AnuXZdlvmwjjFY1H7captQZV6cjH5rFK4jVBDccs+bljzyWg7vKS9JqLx9h8yXink64oO1LrXKHbHAzYcyWwfr+f3zhCjyN0P/I2YDOOsAhGdlvjassTiAQQmhYa3nnnHaNdxMaY1ysMfuXZkaH7vGjAryCHzbgKuQllPMuyryoYELpEq3sPyEqqVpZlR+/fvx+8b4I3KG/S4p6zEmsyVeedwt4CWl0CQkeSJBmLESh3G0LSNP2rwoMonbcxpS5YVAZqxvWLDK2hVTp0jVrX/CZWSby68zaL1KYcnu5joTPqgVhcJluXNzUgqM22RKE33ptnq+TsORjp+kWGxjhiawQj0RgEIgGEpsHQQCCyykFCtF2IN27cyP/s9/vnCg/u3byCAaEDkQrw6Lgo4ENWUqUO6fhGHpRfKTzYALEmU3qvFdzv9/thU64CqOHCXr6hLkZJvrr1KES+QPBPNXsb1iIGI+n9BYwmWolW735Qm6oAQ8rvXzEWfZvAzutMnTa81fA+Buwkv4bQHxsB1fm+lD+pGOsedawuNKSox4tgZLfVvXHqoBUCkQBCUUDk4sWLdQlE5mKVgNTASl/a8RUqKz7LsrMmQolWHRcdEwV8Cg+iTFGDPerN2sBBuRNtcG4/py/WLSBRw4W9fOdrjMzIXq/nSrQ2bZyMBvMWQBpXuQaoWH4/UCWHkLQx0KviUGYf+RDy8VVXg5E2A3Gt8ACAYT1wmVERy2qy+apb7mizeU0D3Pm5GLoalFfq/d3Cg/UXdRxBMLLbGlMabXF6Qwt/xwsPAMAeqD/k0tKSC4jUIhBp3YxVAtILGobo0XdCJbHW1taCD9q8cjCNzJprkXxBPPQOQQ3KbbD5bwoPNke0yZQX/K1VQMIu7K0XHqhOlB2s7pimafrXhQeBiOihBOxZHowMvXnKu8c3sXf5PW2qUbUVANij/NoXer7jbSRk81XHJEnyf2sdoIYlq/NzMfQm18OHD7u/NnGDa9RxBMHIbmvEwNoGIk8VHgCAEWmXs0qWKuClYEKSJF+uWSk+1dO/rEGaV8oyCJVTVQlI9XkMsBNx3njlX0NRELaBDb7bKl/cC71A7pX7bGpWZC7WZMp7v/fc8yKi2mSG9Pv9XzYRzk9v0tjExefW6Wq/MwAjiZJd45VibuT9IE3TvIkW11Gg3rys7roF5/JrH71OEVAelPPmW63GOGJ7BCO7rfYfCAKRAEJ57bXX8uCZK92TJMmv1HSXUh4cjdEw8VfO0wAAIABJREFUXgFJ9QFUZmPhweEpSHJcE6fQu8ea3kuwZaKUwfQmtE0vgZmPoUJPprz3u47ByNpQyekYfT1adH62QtMn8QBKETujoamb4/L7WMT+tgAC8KoM1a1sab4JkQ0NCCgfz9fwvhSl4o63CZtxxACCkd316YX1yVp/IAhEAghBi8vvvPOOOXv2rOt7dt4OOOra/0XX5uu66Yeu0e5Ktfb7/XOFB4eXl3nVexqSJjrq1ZZl2U9YeG4vb0Lb9GAPi3wVU6Z3aC06PwEA++C1S2jqImL+vLuSgQIguHw+TjASodX1nIrYMoFxxACCkd1V64VeApEA9kuDnOXl5Twb0tY6XzPG/IYxZrkBb26eHalszpC08/Ljjz/Os4r2WApmPMuyNxQEWF0NextxmaBJknyj8CBKl2XZUa9sUDD6LKr/QAuOaD4493Y8BuG953G6xbdDtKxRe37+uPAAUBJXghgA9ikfp9iNmJ3hLXLXddMpAHQVAe5miTaOIBjZXbXd8U0gEsB+KBNSQcj33nvPzM3NucDH63ZxvynZLqtaENfCeOg+DSrVai0UHtzdiSRJDurfCLlzTLvPVaI1y7LHTe8l2BYq5xsjGGk+L7H5d4VvNk+UHY5eqSZsLw9G3rp1a9sf2I80Tf+J9x5V8DYLAaiYxt/qL89xaBavYgUVDgAAlWnRJuzgCEZ2Vy0zIwlEAtgLF8xS6VBlQvpByDRN/xeXadgkaZpeMhGyIxVIVGZjlmVfHbU3RZZl3zReuddQVI5WO66SJPkeO5mBevFK1QEA0BlPnz79R442AAAYldbc+v3+P/PGFRGM7KaHF9Yna7dTjEAkgFG4AOSlS5fMhx9+aJaWlvwSe40NQnpWtCNbgdXQwQAFJJX5pkzHwoPbm02S5KW1tbXg2VunT592f21CCV2gU7zsbHq5AgA64ZNPPjG/8Au/8L9ytAEAwKgYR2zvwLaPoM1qt5hEIBLAbrQgrubJ+vPo0aPPlC9VFmSapn+lAF6apq0py5MkyWVjzJL6KSrrMxRlNurfVKZjkiTDBmzzsq5emdcgdCxtSaW1Bjf3BgAAAOpipOonAAAAnmjjCIKR3VSrYCSBSKC7lPE32BNRwSljG1vrS4Eqr/+Hb91ez1oVgByg6+OSK0EbijIb1ZtqZmbmJdtLc7f7gnq0HVcPwdXVsLcQvTbrSuFBAAAAoDrjDW0hcMTYzAwAAFCZRo8jbt++XXhgvwhGdlNtgpEEIoF6OnfuXJ6FuB319xsMIkbmsubu2WtYV8oF6vVen5iYOKWgXcisRP1bKmtrjJkf4v3MsyJD94pUsNn190zT9GbhB1Apfc5DU++E559//tfSNOXgYq/yydxO9ygAQLdoTBmyjcCtW7fcOPlIQ+cd2khoHj16VHgAAKribZCY5SCgJvK5JeOIgkOF7wRCMLJ7atMvkkAkUF9a5LU3zp2s7fDYKAZvzHe8nUP0KPs8OzJKMFJB57GxsVM22Ljdbq3xLMveePLkSfASrQpESpqmFwsPomprU1NTvdDPQRPQmZmZFwsPNE8+gb57927QJ65JkLXd5xGf3yOC99I19vyMcd4DAKLRXKGnKiohFxFVDcRq9CJijIwGANgrNkighjS3PM44ooDMSARTiw8AgUigMZ7jUFVu9enTp387MzPzm8pGDVlu6caNG+b06dPGZkduVyb1RJIkBxWIDD15eO2119xfh+1biYbTAN9udBimPHCdRck48EpSt7X0dK0pc9c6RA9bAGiO0BtUvM1GszuMkeusR4lWAACGwzjiGSotOx1rHJEUvoO2q3xxi0AkAIzmwIED3zbPBu+C+OCDD/J/Jsuys9v9e1mWfdNEKNGqTE9bBvQ6WWC1tFmuJCRvQHuk4e9PlJ2CMbL92kilnYfI3h+ZyulYTT8/AaAr8o1NodtHaLyiDSpZln2l8GD9nTD0iwQAYBiMI4ryKlCxqisQjOyeSrMQCEQCwJ6saPFdZU1DBoeUpba2tmaSJHlpm74Ns3rs448/DlqyQk6ePOn+2sTd5l2Qb17yMvWC8II9W51vjZFl2e+bCAN0bxJEZuQO0jT9u+0f3bsWBcsBoCui9RFeXV3VGHnMBfca5IR7/gAAYEeMI4ry50swEkFcWJ+sbERKIBIA9i5N0zw70vVZDMXrAzm/xT+pXpJ5OdeQFHCxQZc1gi61lZeoDD0ob/gOQeeQC9KHRs/IoeXXjaNHjwb9R70JV9MmjK3kHV/uEwC2k18fQmc0mGeDeY26J2jDlMZaBCMBANhVlE3YprnjiPEsy74acxxBMLJb1qp6tQQiAWDf8gzC0KVaNcCwzbVP2drwjnqmHddjoQch9IpshDwYGWtxz+4Q3CoA3gTzJlLGgReMZAVxZ1GC5eoBarMjp11fUNQCwXkAO1nTImLoUue6z9tewoNj5DqbV693ApEAAAxtTesejCNyJ7RWE3McQTCyWyoZkRKIBIAgtBh7XX0W1W8xJC87csH7Z/O/v/POO0F/lwZ4yu7MsuwRwchai9I7weyejVt7/X7/awOvIxj1Qez3+z9u4vtSsmiZMK6XLtmRANAYUbLlzbPVQRYKD9aQ6/UeuqoJAAAtxjjCcuOI0OuAPoKR3VJ6MJJAJAAElWdHev0Wg3CL7y7IYkszvBGjNIN77kmSXC48iFpRUCxGsEelMG02bq+B2WfzaZq+qM+MsuhCcpOfNE3jNGdol2jBcnfNy7LsbOFBAEAd5RfuGIuI3/3ud115+XMNyGqYd2XkvR7IAABgZ4wjPpePI7TWsbGxUXgwFIKRHVJ2v0gCkQAQ3B1XQiLkQEkDDQ04FGSx2WoqzXBQmV+hAy5eVidZkTWXpunf6BnOzs4Gf6LeTrvlwoM15nYKxsiK9EqOUlttOFHK6eh6qIVcTcR0+hd+AABQN/l9M8Z4ReNgZTXY8vJXCj9QH+P9fv9P9WwuXrzICYoY8pLpoUvkA0ANRAtGNmkckWXZn5nIWZGGYGSnrJf5YglEAkA0+QAmdKlWN+BQsMUFXLSLKyQtEtnG4NddzzfUWrRBuYJ5Xq/SpgR8FlzGgbI7Q/MWUQlGDifa4vPVq1fdXxsVLAeAjlKQ5F2NMb3ey8FoPNyAMcuyNhXquZIViUjyMoahN4F5jhS+AwDleOA2unZ4HHFFCQl6rjGzIg3ByE4pbWGLQCQARHUzy7KfqO9iyIGSnw2kr7W1teCDkNdee839lazIZogW7JFLly7lf/b7/T8vPFg/4y5Iv7wcPj6lhR31i9Rnm0D90G6aSOengs26HtpSwk0Jls8G/mJREECT5GMWjY9DU1aDu/dnWfZeDcusqarJGS10xs5mAEK7deuW+xfrXr4QQLtFm1s2ZBxxqqxxBMHI7iglGEkgEgDic/0WQ/eO9LKB/EbbQShwqmCL+hCS+dUYUTMN1JtPAZ80TV9uQAbainYKanAeY6egm/QkSfL9woPYzp0syx7FyNw13vUwy7K/KDxYH8t6D4wxnxljPgz8Nc+ZB6BB8o1uoSuHONqkomwBjQX6/f5HNVpIPJJlWb6pSwudodsrACUiGAmgSnkwMsamJrP1OKIuNscR586dK2UccaDwHbRV9IVfApEAUJqVLMv+y7Fjx8YUHAk1YHDZQAo+hS5Defr06fzPNE0vFR5EnWlQflyD8hi75LRwpsD32NjYkh2r1DFQvaD3QGXPYu0U9HZgkjU8AgVvx8bGTmnxOXQfT10DlSHe6/VessHyugXM83H3kydP8pJ8CpKHCJTrvbTltDkXATSJ20B1XJtUYpRTV0UH9cubmZl52bZNqHrThhYQP1IfqvPnz0d5zUBstnShsRUZbvKGA6jIPVuqtadyrTFKnmscoTHK1NTUy3auVatxRFll3glGdsP6hfXJBzFfKYFIACjVgyRJvqdFeAUxQi7Ch17QN7YEpZ6nMniSJGGS2Sw6XtcUoIgRiFPwRAHJixcv6vx4P0mSL7ueNDVxwhjzrcePH0cpz2ps1nCv18tLtCZJUqfX3gT5+DP0ddDRMde/W8Ngef66dV5qo0fIiaPKadtrNecigKbJN1BpzBIrMKesAWXOT01NubWPBRsILdvmAuIHH3wQ5R4IlOHu3bvut1AeHkDVNMfqaT4Ua+7/5ptvDo4jqgpIbo4jlLFZ5jiCMq3dEHUxYXF6Y4FAJACULh8duYzDUDQICT0Q0aLQ2NiYspi+U9GCDfZOx+u6MqVi9Y5UuVYFOjUQrlkPBQ3Q/9LYoFSsnYKuFIwrv4yRrCqIq2BujFLCW/T3OFT4oXId6ff7P4oViPSu1ZQLBtBEK/1+/6eh+6r7dF/QQqK99p6qqGTrCT8QGWvBFCiDPksa02RZ9tu84QAqpgpkjzSO0Ib6GLYYR/yo6nGEMjbLRDCyG6Lt4l6c3lAE/1uFBwAAseVlJBQkitUzLRSvt+UVzopGitqHSRSM1EA4SZKXsiz7HzXYHT3rlyxRwDQW7by0KIu5By6IG6u/h46919/j/QqD5fM6J9VjVeVj9XkMHSCnXDCApkvT9NsmQl91n1tIdH2vsyz7B1tJIbZxO5b+gRufEIhEGyiTWeMsDUU4oACq5OaWZYwjNJfzxhFlXP9qMY4gGNkNUVbQbCDyWuEBAEBZ8pGDF8yoHQVKbf+xd20AFc0TNfvM0UDYC0h+VOGChMY3H7oBesySJS4TTdmnZA3vmTJh/kXXwVg7WLVbVOemJos2C6bMDMlxGxy89uTJk7HLly/nZQJD9Qp2/HLBNe3d2nozMzOm3+//uOvvA7BPV5TVoPtrrHuCsQuJyk53m1W0sGfLxMa6P8zb68MZ9djTIialWVE2ZR6rl1po3rlMdB1A1a7EnlsaO47Q7/DGER+WNY5QNnqV4wgFI3uF76JNPr2wPhl88ZdAJADUQilBov0g66sdkiT5holQFniQApJ2QD5mB+RlLkpsBn00QI8diDTPvp8svuzdgzRN/0pB3Zg7WF2w3O5eLSt794gNfp7S4rPOlxs3bhR+KASvXPB3Snhd2Eaapv+09SMAhqS+6pd1T9DGjdi0WUULerpGq1+lMebv7Vgi1GLivM2YuJam6YsaI2lsHasnJrCTNE3/zm6iC0pVKGy1h15JWcYAsB3NLS/Gnls6ZY8jNJ+N2Vt7GGRGtl/wnc0EIgGgPsoKEu3FQKbNTU6bRoveh8nRgPytt97K+8cYY5ZsH4XYgZ8TdqfgZtAndiBSi4k2a/g6WcP7tpklHnMHqxcs1y/5YeQg8oJfllWvLVbfUvPsxhHKaQNouiuu51MZm/W0oKeFPZWct2OXU3Yx8Y6u5SMuKI7bYEzet0rrLqoYoZKwuk5rjBQ6Mx4YQV7FI8bnyvUss/3aq27XAKDb8nGE1gQaPo74Z38coaCn5rNVjyMOFL6DtrkT8vUQiASA2rmZZdmfzc7OHtQifJ0WKNxOMld3H82WpumfaAygAWzs4Ld2SGvRTb9rZmbmZRv4uW6DPyEDd7P23+ylaZoHmjQJiP050meVrMigdE5cHxsbO6Xrjo5hLFosU1BQGTdjY2NLWZa9YTeFhMr+PmIDgr0nT57kryVWNqRDuWB0WFdKEi+EXheouQf2uvytMsYsjq7XGkeo/67uRVNTU9N6DvrSomaSJB/b47DVdVa9qv+9FgzdN3QP0HhI94CYm1GAEej8Pa5SrRsbG0HfNy3GqyrJ0tLSWL/f/+923sEGqZbxyvyyERN1tjmO0JyvjEoLZvdxxOMkSW63YRxBMLL9gk2wCEQCQC1poPSdsbGxM1pQjr1oPQo9HztookRrO6w8ffr0azMzM7+pAbIGtjFpkUMLiDqP9OfExMQpu0twzQZ+bm4zEN/NuM2EPKesM/2sdgpevXq1tHIlej02+PM2k/FglEn41ddee21M5WdCL5L5lDWrc0UT016vpwnftSzLvmk3Xqzs8byctf1KdY7n5+TFixdLmTgSGEeHdaVlzXjhO+2nnk9/pA1NZYxZHG1m0j1CX8qm0O9W//SjR48eHBsb6+10ziVJkl/77969m99jynrOwAjyTQ0KKMU4P11VknPnzv3i2NjYt/r9/ltpmn7bjvm7tKGitbwKJsx/UHf5OKLX69VlHDHWlnEEwch2e3hhfTLIDZtAJADUmnaNntHuqboEI71Mm++RadMeBw4c0C7lDxWE0QC3jExcNxj3gpJuEH7NBiZX7de9bSa2h+zXrP3KB/DKhNRgXZ+ZMgfqmkzos2qzJAj+hKONGZeUrVjGDlYFO/U7dDxV8mZmZuYlt3PVGLPuLZw9GNjBOu6VH9P5eCTLst+2pV/zfiHaFRu7TLCjz1UDygV3MZCCiOpY2j4Gvc6uvNatpGn6n1RZQdmRutaVXT1E9wmNMdzYXIvwXlbQM7TxhPKraIB8ffGVV16JVoXCbfjStWtubu5FtWywX8aO+80OSRduvLXd4wAwtDRN/1hrH3UZRxi7lrAVzSFjbsYNiWBkuwW5AROIBIDay0sUKnOszF1bO/GafRNsaRedXG9PTEyc0SKB6+9SBheU1EKeJgM6173A5NKwT0EDdX1G9G+VXa5EC5GazJjPdy7+IYH64JZVNlXZimVdC92CmXau6rqnCaItqTNd+OFtqISO/h2dk2Vev+teLtj7fB6h7zCAPVBg4m1VD9G9t6wya9vRImZZFRiASO5lWfYTbcCK2R5EC+r6zCrg6bKCNP63436zU2aQ+bzvpDb8/S5BSQD7tFqncYSxc8+mIxjZbvvOiiQQCQCNodKAp9Rnr+pgpJsw2t2rlGBpn+V+v/8HJ0+efLGK8h8KUCgIqi8FgHSu6evw4cN+6Z9NWihRqRL9f/qqcsegzezUX98luBKHDfKWvoNV55ULzrvsF7dzVRkEvlu3buX/5c7Hqnp4KHha56xIsoQABLDQ7/e/ojJrGiPXqZ0B0ERJknxfFXkUJIxdxWGrrCCNr7Ya7xs7B9XcYG5u7mCWZe8lSfJLhR8CgNEsM44Ii2Bku+1rdZBAJAA0ymq/3/+xeuNoklZlg2oFACx6RbbTgzRN/6ML+GhQXlWAT79XX03oq6TPhYI//X7/p2mazhd+AKFUvoPVZb+4nauxSpnthxbrFBy3fX0XavcEASAQlWvNsuyjs2fPjmkzSJVjZKAFNL87o3FtWSXlfTt9fv2MIQUkbTl8siMB7McDxhFhJW16MXjWhfXJPd90CUQCQPOkaZqn5Sg4VBW7GzUPuBCMbDWNMc6rL+jFixe33aGMz2mDgAuKpWk6R3nW6LSD9ce9Xq/S62GdeeWC/zPnI4CWu5Mkifo+5ZtDtuvbCGAod+wG2HzeV0feJslDHFIAATCOCIhgZHut7fWVEYgEgMZaUY8MBQOrCg7pd5vPAy7fLjyItlE0410/0IYifRY1aVHg1hjzeogy+tiV28H6+OzZs0wYBygjUouIT58+/VtjzJXCDwBA+6zYrPl8MwabqIC9cxtgvb7TteJlSBKMBBAK44hACEa2156yIglEAkCzJUly2dheYFXwSrSywN0N89odrSC0y7TCv9Ek5erVqy4QeZ1s4VKxg3ULeh9cedYDBw78x+JPoApedglZqkA8Kkl9XddB3ZtZSAT2bEVVcDT+Z3wFoEMYRwRAMLK9Rt51TyASAFohDwJWUZpQgciJiQljgy4sqHaDMtC+7DJy67pDugouEGkXafSZoE9k+djB6nFZuubzjSsK1N4r/BCGkc+zjh49GuzNsvdOQ+Y0EN08C4mju3//vvt/jjToaSOiNE3/RP/6W2+9xdsMoEsYR+yBN444RDCyvUbKjCQQCQCtoSDgdS2+e1mKpfB+H9lf3fIgSZL/TZlWCkZSsrUQiHyXQGSl2MFazNJ9m+v0vrDZBmi2ZxYSyezandeDb7zGTxPlWlG5d5V9pz83gI5hHDEiv5cvwch2Wr+wPjn0JJlAJAC0Tp4dWWapVg3ANBlVyc69lgpHo6kkpjIkH+u863LJVpVbJCOydjq/g/XSpUt+cHyh8AMA0C2b9wVljIfMdAa6QuXe6c8NoKMYR+wRwch2GnoRmEAkALSSyrytaWBU1qDI7YhN0/RS4UF0hQtI/kQlWxX86FrQR5+5GzduDAYiyaKqh87uYNXmAG+zCMFxAPjcvCvlrftCW7K7NPby+tACMd3z+3Nz3gHomNaOI2Ku4xCMbKeheo0QiASAVsuzI8so1aqByuzsrFHfQEr/dZ4Ckr+uoEev1+tU0EeTDwUivTKYBH3qp1M7WHVt1jmpzQH6TKq/K8FxAHiGMsVfd9ldTQ+o6L7mrvtASVZci5CLFy/SPw1A1xTGEU2+Dmoc8f7770ddwyEY2U67ZkYSiASA1rvpMtRiL6oo4KkJaJIk3yk8iC56YIMem0GfsvuXlkmTDWWBavKhSYgmI5TBrDWNgc+3bQfrIL9cMIFIANjRiio76FqpLHIF85p2b9BYRD27dd2fmJgoPA5EtrnZy6sQAgBd8cw4QsG8Jo8jNE8+fPhw4WdCIRjZPp9eWJ+8t9OrIhAJAN2QJMllU0LvSO/fv1J4EF31wC5MfP3555//l6WlpVaWbVVGsCYbygLV5EOTELKDG2HZ38HatnNT56VfLphAJADs6k6apv/BbVbRvUHX0SZk0GvDl56rxuO2SgnlMlGFPCCpYLg2ImosAgAdUhhHNKUSj8YRWtPw1w1ttacoCEa2z44lWglEAkCn5EGRmJNB/dt2B/a76htS+AF03ZU0TX/LlW1t4i7BrWiRTwEslaNyZVltwGfHcRhqZXMHqzs3m57B63a0euflefqWAsBItFnlV13vdWUI1HUxUc9Jz00bvuxY/O0kSX7FEIxEdfKNiK5kq3pWU7YVQMdoHPEbGkcoS7Ip4wiv1cyr+guZkRjFtiVaCUQCQOdoAXpNCxSxFiW8xXuyIrGdzV2CypLULkEFfprYr08LKqdPnzbvvffeZjakHbAvEPBppGd2sGoipglZE8uLuWxILzPmVTsZBgCMRpvrtJPvVVdyTYuJusbWYdOKWzzUc9Jze/r06d8yFkGNaE74qmsX0paNiAAwgjt2HPF7uhb6Qck6jiO0ZuiNI/K4UsyNJAQj22fLYCSBSADorPy+EKN/jAKcCshogLXd/QfwLKdp+rKyaHU+1nmX4CAXhNSCiv60wZ6v20AW537z5TtYtaDr+oVpN38TMkvcZFIZCPY6f91mxnBeRkbmEdB6q/Y+/3suU1KbVlZXV/N7RJllKDUOUUBH4xC3eGg3RP3egQMHfotrPmpmNUmSX1eWjStX6MbQ3DsBdMhNOy973c0z3ThC1WzK3AC7wzjiVRs4LW0ccaDwHTTZwwvrk4XyYAQigdbKd77qpvLo0SOOMkrnasq73pTAEJRtcMIOeJdnZmZ6Ggzfv38/D6hoYF6n65kmCBq0a2e3qMegMeZSkiRXyD5onTt2QXc+y7Jvzs3NvaTj/sEHH+STttu3b9fq9SoI+eabb7rdrPlkMk3TP27TgrRXHqhun7Wbxpglt0EBQOvdtF9HlDXwwgsvfHVubm5M94jHjx/nYxfdI/S1sbER7L1Q0EbXegU9tfnPs2Y3eBGARJ09sFk2GjMvT0xMnNJ9U19ra2v550WfnZCfGQCoqZUDBw6s2DWQeY0jTp48Oab1NK2DuDFEV8YRBCPbhUAk0C26cRzXgF69y4AyKQiuEhMKziRJssKbjxGt2sG4vhYmJiaOa5egvhT80eKEvqqgQbsG7JocuIxiZUIq6E4QshNW7DXtmaDkJ598snluVrVwts25+ZMkSb6RpmnrrsNe1lHdFtzv/Ou//uvPJiYmvqj7IAHJ+GL2rWkzMpCC03rLfJIk83Zj1Qk/MClaVNT94u7du/mf2mA1zGYWbX7S2FqLhjrf9d9+VRN7rf+O7QdPj3b4n++6j0vv2V6Sy3Zs9Uav13tJC+PKmPQ/M/qs6DOj/waAFtKcRpnjC24c8cUvfvEroccRLghZ13EEwch2eSYYuTi9cYRAJNBquoksnzx58gu73aB0Q9tu8VQ3qsESnrZ5Mdph1p0DIWmR2J4n3+t6cOb/b+8OYuO6z8SA//2egMCKhAjoIRDdrBWgUC5pydC6tVvS2T0astLsxQK2ktvAuuzWiuUTDyuqB56sgEZPNnYbCwXknrqyrWsRCW1vllaD7iWCgUgbhEYOxcorI7l4xsU3+j9mxCEpinqPfDPv9wPGkvmGFN/jzPCb//f/vi+eP7EpoO6v2RE38u1YJCX7/f6fvfLKKy9U1QbVrum6dwmOqoL26rahXcrNkeTUxIhqU57ZelIybsePH1+IRbO4VYnJTz/9tPEFs3g8njhxYlidu8ljc7Uoimtjn7TP4jn1NG2HNsYvVUVyNQvtwIEDYxsu99s3vvGN/55SejOSkZu9Nm0Xd42q2lRLtm0vnnfsv6qqiaFruf3a2ZHNVYvf/va3Xzp69OihDRUI6+J3RsQ3Kb/WPeE9183878TC5U5fB4+MfaTl4ndpVzR0rq37HbmFWPxeLopiOVcZV8+ZP9nsORPPkyrGGr1u1eJ8kLhsXrxG1TlSYy/inUl8H1Qlj+rQtZgy3idlk9Qt4EFez433m2k0jjh69OhCrAVtFUfcvn17/e8NxRGNi2Tk/ZTSi235hngm65ntnIjUtgOm24O8m+YXFgbYQixILOx0UfRpVC1a8y7XTmsiGdlBEcOcL8tyfZfg888//6cLCwsvVIF4tSgRCxLVY3p0QWInNu4YjP/fJGlyc6Ql26RVH9zL339TX7uLPsi3SJif6vf7//H48ePfr5Ijo4/L+LN6XO5G9biM15R4Y73xDWZuxfo3bX5s1rnQGhXJBw4c+IuxA+2w3O/3/2R+fv77kv+N8n62Hs/6evHA75ZtVRurUl5UPJaTLXM5Fp+rPjk2tlR/7/f7v0kpfZb/90FOKt3Lf+520XD4b0VFRZ0idmpAkwujbXuEY2AnAAAWmUlEQVRcdelcd6J6jK9u8pwZPm8OHjz40vz8/DAIqlrSP0mV7H/C4vwzi++nK0n0iEUnKM5p8n1Qo5uvY9NhVRk37Zp4/uSqv7GPT5AbG2LeuZHXxfU4ot/v/7N471Hdqck4ou7ROSNrLg8O5G9SMnI6DB9oI4nIb3X9gkAHxHP9u7lyYztz2+yUvdP1yrYpFomd2lvIjVSP3dQqajjwm3oNE4FlWabRndPPP//8ifn5+Re2WpAY3SU4KhI7T6g07Y28Abgx4a+HVeKM+sVr3WpZlqtVYjIelwcPHvzh/Pz84c0el1s9JkfFgtkmCfGh/Abz02pHa1mWbX693fiGd6dJpMcW6kdcy5WpbX0+PijL8o9z/LVZfLVd3DXq3obfo5Jvj9u4OMP+uFN12mBHquf1tpXrOc5pRIOLiHU+H8+PfWR6delcd2PsOTOSWBiNEzbGDMfybWg02d+gppJdbaxwvTT2kfo08bt9Et8H3WvwOrcxfmrk+dOmqr+aVMnEx+KIjXFDQ3HE8P1Lg5ua7hzIJ7cXL9g0745EJPvIQsH+uac6jc30+/2fRIAS7QTrFG3pMkkPr31NW985nYPtIxuqDYaLoxt3CY6KuabRCTJ/6N6Gm58fuzFMTG6xo3+4MNbv9/9FJM93+LWrN+bV5qB4XN5refJxoy4utD7IjwOAthjGRVpXMkUebIjXt03yZ9X7hSbiqC5tzLDG1LyureXZ2NR+ww0fdY95GnUgv5C/OXaESfNFXvyQiGS/TNtOFJh0Z8uyfCESkXW2aI0WgtFCJCp2yrKUjGSvPdisUma7XYET3rKFyTC2o3+7xyQANGS4iFj3eIaRGWRd74jCZNiYwARgZxqJI0Zmot4pVnozNxosM2fvPJCIZB/dX+nN7GSHGrBHvvrqq5+kBlq0VrMMyrL867GDAADAvhgMBj+sZhnXaWQGn2QkAEypfr//p3Fmt27dqvUER9q0DmdGptxWRyJrspn7yX4yAwHaZfHAgQP/OuaV1R1EjLRo1ZoOAIA2GauGGgwGLxVFcehJ32O/3//7siz/39iBR1XvbZ5hW5kriuJw3bF/OHHiRPVXyUgAptXcdutcT4gntiv0q9rxtv136Fx0V7t5c7tT2Z2RDgvDmZFppTcTswYXJSSBXXhdVSS0ztnUQFVkJCKPHj0af70yAQsyAAB0yGYLhZ999lmKasEn2Wr2dEppYTAY/OeiKP79DufV7ZfhBuEbN8bysc/s+PHjwxncRVFIRgIwrWL82MJW5/bb3/5201mK0T3g+PHjW35ejiN+XBTFX+bNTW01XEdsMI54WBTFemXkaEIyLsrs2GcBPO5+vFDlVs9Ae8T84DMRJDWRjMzMigQAoFWKooiywIWTJ0/WMu8oYt8LFy7EQmP0F/vb2Ijb0jj4SCx0/u53v6s9/o958blNa/0llwDQIrlLwvdHOgI8s9OnT6e33norfpH+PFdHtnEdPeKI/xBxRN3JyJE44nb8pxg9GAnJld5MlKT+NKX0xdhnAzx6bbi00ps5JhEJrdRIVWTsZJqfnx8GZy0NngAA6LZhjBpxax0ino6E5Icffjj8aoPB4L/kNm5tcz4qQq9evVr7tzVyLcX/AEy1siyHG29eeuml2k4zfjdfunRp+PfBYPBJLiBom4gjDsf3+vDhw1q/tZFrOYwjirF7PEpKruYL8+7YQaDLojVjJCGXPQqgnQaDwVvRiqpaNKlL7OZKj4Kzy370AAC0ULRYq3URMRblLl++nH72s59F5eWhwWDwcVQQjN1x/xzr9/tvNxH/p8ev5Z2xgwAwXW6kmuOIlDc3vf/++8M4ot/vf9K2OCKldHEP4oitk5HpUULywUpvJnrOf/cJQziB6RevAT9Y6c1EW1Zz4qC9zsZupmirUOdupsOHD6fFxcVhj3ctWgEAaKlGFhFTrmy4fv16LCR+p9/v/6+2LCTGomZZlt+MRc66qxlCvAfIVEYCMO2Gv+tGfvfVJn5P3759Ozb4x4zq1bZcx5wcTU3FETtORlZWejP3Vnoz8RN4Oc+IA7ojnvM/iteAaOPs5w7t1u/3L6QcRNQp2lNFj/eiKP6rhwAAAC0VG2d70Vo0ZhTVbXl5eZiQjIXEliQkP4jvJRY3m2jRGtfw6NGjKW9OtikZgGl3L0YTRRwRm/LrFnOo7969G1/1TN7oP9VxRFzHHEd8VH3sicnISsyGixlx5klCJ4zOhbzmRw4TYbEKItbW1mr9fl977bXqr63ZvQUAAJsYdvFooqohbUhIDgaDf9jHGZJxnmc+//zz4eJmE1555ZXqq1oTAKATyrL8n6mhOCIqD8+dO7eekNznjU3DOCK+l6biiChsyNbjiB0nIysj8ySvjB0EpoG5kDCZzqbcQqpO0VJhZCfTPY8NAABabLjgNZJIq10kJGOuUoxHSCn9XXxoDy/HkXyOZ2K+UywgNtFWLW2xiAgAU67RTU1VQrJq2ToYDH6dUjo1dsfmHMktU4eJyPhemoojRq7h7pOR6Q/zJGPR8wfmScLUMBcSJtexamd0zIus0+nTp6uvZlYkAABtt95irYlWrZXLly+nS5cupUgIppQu9vv9/xvrbmN3rNfZXI35aiwgRrIwV1fUzoZEADrqTiQIFxYWGosjIvn3xhtvDEcsFUVxKKX0tzlBeGzszvWq4oiFphORG+KI9TzDrpKRlZghl+dJ/sg8SZhY5kLC5BtWRX7yySe1nkgEXhGA5Z1adkQDANB6ZVn+TWq4OjLl2Ds27lXVDSmlX+TFxLqTkme/+uqr/51S+nlUY0ZVZpMLiOnxqkgbEgHolKIofpb2II6IZORI29aFlNKv8u/dxuOIiF+ajCO2Kmx47uuvvx67824tza5Fa4rzKaVvNXYmQF1iLuSqdqww+QaDwT9FQPHyyy/XGkxE26c8L/Kn5kUCADAhogXZP0bVYlNt1jaKCoCInaMiMz2Kz39dFMX/yItwu9n0G9/4qX6//2dlWb4QH4ik53vvvZdu3bo1duc6xYbEjz/+uDqHP2r0HwOA9tnzOCI2AUW1ZK4mTNHlIW+uulFnHPHOO+801lWhsl0cUWsyMj1KSB7JC5Znxg4CbRFzIc9rxwpTIaoif379+vXh/Jo6RcvXgwcPflkUxXdG2yoAAEDLRRLwTLRSrbt7yHYiKRnVANFdpDIYDB4WRXE7Lyje26Lt6Vxe/FzM1RHrIs6Pc2g6CVmJ9xS5GuR1lZEAdNS+xBGRlIzb/Pz8+scGg0Gsy93KccSDLZKTizmOmGtzHFF7MrKyNLs2l5OSC2MHgf0ScyGXV3oz9Q6VA/ZNzKeJtlCx6FHn7qYIfi5evJjy5oWzY3cAAID2irlLv4qZ6iMtR/fM4cOHh9UUcYsE5aFDh3b8T8f3HAuGcYvNgU22Uduoqmbo9/u/Kcvyn4/dAQC6YV/jiPh9XMUQkxRHRIeIq1evbhlHNJaMrCzNrp3KSckXxw4Ce+V+TkLa1QjTJTb+/F20Woh2DnWK4CG3mfrBFruuAACgzYZVDTGTKW77KRYVo/Xa9773vU0XFGPhcG1tbbi5cC8XDTeK65SrMcaqGQCgY1oXR1R/bjQpcUTjyciKeZKwL77ImwFWtWSFqdRI24hqJ1Oupt6bBvkAAFCvYzFz6fe///03o4tILNKxtajAiFlSeU7Vv9zyjgDQDeKIp1B1WPvqq6/+z4EDB/7NZp9ZjH2kISu9meVc3nqlFVcHpl881+biuScRCVMpesGfiYHadfevjyArG9vFBAAAE+JeWZbvRCXihQsX/My2EW1lq/nzZVn++db3BIDOEEfsUFRsVtfowIEDf7HVZ+1ZMjI9Skg+WOnNnM0t326O3QGoQzy3Xo7n2kpvZrPB+MB0iG4DVQVjbWIhIgZNR393yUgAACbcclQ1LCwsjG64Y4PLly9X7WMvGdEAAOvEETsQG5p2EkfsaTKystKbubPSm4m2bz/Ks+yAZxfPpdfjubXSm7nhesJ06/f7P4kTvH79eq3n+dprrw3/LMvyr8cOAgDAhIlKv8Fg8OVbb71VzURnRFQyxHynWGyN9UTXBgD+oOoY8MYbb4gjNvE0ccS+JCMrK72Zayu9mWM5Y/rF2B2AnfgiP4eiJasqJuiGU2VZvhCJyLp71keP98zrCQAA0+BOURR/Gefx/vvvD1uJ8UjE/rEZcTAYPCzL8o9dFgAYE5V+r0flX8QR0VGMR542jtjXZGTFPEnYNXMhoZuGLVrrnhW5uLiYjh49mvJrizbPAABMi9hodyUWEt955x0LiXkB8eLFi7GA+GVRFP82pWRNAQA2tx5HvPfee+KIXcYRz3399ddjH9xPS7Nri7mcc6FV3xi0S8yFXNaOFTopNu/86vPPPx+tYqxF7PCK1goxdzal5PUFAIBpE4uJZ+7evZvefvvt2ruMTIpqATF7XVcUANiR9Tji3Llz6eHDh528ahviiBjFeG3sTptoXTKysjS7djYnJV8cOwjdFS1Zz2vHCp02DHwuXbpUa2VktKv6+OOPY0fTr4ui+KOxOwAAwHQYxtNffvnlcP5TLCh2iUQkADwTccQu44hWtGndTE62zJknCeviuXBMIhI67chgMPhxBDw3btRbuBg93kNRFH81dhAAAKbH2arV2tWrV9Pp06c786NdXl5eb6mWKxmsLwDA01mPI6LDWN1dy9rsWeOI1lZGjlqaXTuWqyTPjB2E6fdRroY0vw2IgOfnH374Ybp8+XJtFyN63UeV5cGDB6PP+3fMiwEAoAOGsXWc5s2bN4fx9bS2bY0uKDEr8/jx4+nrr7/+p+eeey5GI90ZuyMAsFOPxRGRqJvWtq11xRETkYysmCdJx/RyEtLcNmBoMBj8QyQLT548WetCyUiLhSs5mAIAgC6Y6/f7/60sy+9H95GolIyNf9O0mBiVn9FGLio4Yr00pXTK5kMAqMXcYDD4ONbqIo6ISsmIJaZJnXHERCUjK+ZJMuXMhQQ2ExtyfnH79u1hEFCnCJRid1NK6bspJVXYAAB0zXK/33+7LMtvfv7558OEZHQOmeSk5EsvvZQuXLhQxfnhpyml1bE7AgDP4kis5aeUhrv8I46IpGSMV5qyOOJSzsnt2kQmI9OjhGT1Q47bt8buAJMpntSrK70ZuxSBjYYDst9+++1a50VGcPHee++lvLtpcewOAADQDY+NCIoKh0hIRmJyktq3Rnx/7ty5ND8/X33oZu5+YtMhADRHHPEEE5uMrOR5krGz69WxgzA5zIUEthMbcP4xdlfVPRh7pCry5ZSSttAAAHTdcDFxMBj8uCiKYU+yu3fvpuvXr6dPP/10+Pc2WlxcHLZSqxYPB4PBr4ui+E8ppWtd/4ECwB56LCmZxBHrJj4ZWcnzJCMpOTt2ENrrfuwsMBcSeILhUOzYTXX58uXt7/kUYrj2K6+8klRFAgDAmCN5LtL50bWm2CAYnUriduvWrbFP2kuxqTA2K8YC4tGjR4f/cr/f//uyLC/nzioAwP6YyDgiJyH/qok4YmqSkZU8T3JV61ZaLuZCLq/0ZsxrAHbiTgQuJ0+eHLZ2iJYJOxVBzsZ2EKO7nQaDwcOiKP6Vtk0AALClY3nz3qnRzlzRgi0WEuO2F9UOMzMzw/cC1a1aOMyu5IVDm50BoF22jSMiMfnLX/5y6uOIqUtGpsfnSV4cOwj7792ciDQXEtiJCFh+1cSVyrum/zwnOwEAgCerKh0WB4PBvyuK4nD1GbGoGAuJkZiMv8fC4sOHD596cTEWC2OBsPrzxIkTw+qFQ4cObbzrR7l9WtysMQBA+z1VHBGetoJyszgi/tyQfEx7HUdMZTKyYp4kLXMzt2RVfQQ8jbP5NurOUwQJcznQGXVvJNgAAAB2by5XOyz2+/0TZVm+sN1XikXGWGDcaItk47rBYPBlURS3csXCDRWQADAV1uOIwWDww9Hk5GaiA1rcNpqEOGKqk5EV8yTZZ+ZCAgAAQDccyQuL1abAY/lWWdjsKuTxCbdHPnRj5M97xioAQCfsKo7o9/u/Kcvys5EPtS6O6EQysrI0uxatW5fNk2SPmAsJAAAAAAB0WqeSkekP8yQjIfnm2EGoTwx8PW8uJAAAAAAA0GWdS0ZW8jzJD7Yqa4VdupmTkHdcQAAAAAAAoOs6m4ys5HmSkZR8cewg7Nz9nIS85poBAAAAAAA80vlkZMU8SXYp5kKurvRmll1AAAAAAACAx0lGjsjzJFdTSmfGDsI4cyEBAAAAAAC2IRm5iaXZtbmclDRPks3EXMjlld7MjU2OAQAAAAAAkElGbmNpdu1UTkqaJ0nKcyEjCfmBqwEAAAAAAPBkkpE7sDS7FvMAz5sn2Vlf5KT0qpasAAAAAAAAOycZuUPmSXbWR3ku5L2uXwgAAAAAAICnJRn5lJZm1xajVad5klOvl5OQ5kICAAAAAADskmTkLi3Nrp3NSUnzJKfLFzkJaS4kAAAAAADAM5KMfAa5det58ySnxiVzIQEAAAAAAOojGVmDpdm1Y3me5KsTfzLddDOldNZcSAAAAAAAgHpJRtYoz5OMpOTs1JzUdLufk5DmQgIAAAAAADRAMrIBeZ7kqtatrRVzIZdXejOrXb8QAAAAAAAATSpc3fqt9GY+SCkdyzMIaZd342cjEQkAAAAAANA8lZENy/MkIzm5MNUn2n4xF/L8Sm/mTtcvBAAAAAAAwF6RjNwjeZ5kJCVf7MQJt8f9nIS81vULAQAAAAAAsNckI/fY0uza+ZhXaJ5k42Iu5OpKb2Z5ys8TAAAAAACgtSQj98HS7NqRnJB8s3Mnvzeu5GrIB104WQAAAAAAgLaSjNxHS7Nrc1G9Z55kbWIu5PJKb+bGlJwPAAAAAADARJOMbIGl2bVTOSlpnuTu3M9JyA8m8ZsHAAAAAACYVpKRLbI0uxatW8+bJ/lULuXZkFqyAgAAAAAAtIxkZMvkeZJRJXmm69fiCT7KcyHvbX83AAAAAAAA9otkZEstza4tRutR8yTH9HIS0lxIAAAAAACAlpOMbLml2bWzOSnZ9XmSX+S5kKtjRwAAAAAAAGglycgJkFu3xizJix29BO/mRKS5kAAAAAAAABNEMnKCLM2uHcvzJF/tyCnfTCmdNRcSAAAAAABgMklGTqA8TzKSkrNTeor381zIa2NHAAAAAAAAmBiSkRNsaXbtfJ4n+a0pOaWYC7m60ptZHjsCAAAAAADAxJGMnHB5nmQk796c8FO5kqshzYUEAAAAAACYEpKRUyLPk/wgpbQwYWcUcyGXV3ozN8aOAAAAAAAAMNEkI6dMnicZSckXW35m93MS8oOxIwAAAAAAAEwFycgptTS7Fq1bz7d0nuSlPBtSS1YAAAAAAIApJhk5xfI8ydWU0pmWnOVHeS7kvbEjAAAAAAAATB3JyA5Yml2by0nJ/Zon2ctJSHMhAQAAAAAAOkQyskOWZtfOxpzGPZwn+UWeC7k6dgQAAAAAAICpJxnZMbl16/k9mCf5bk5EmgsJAAAAAADQUZKRHbU0u3Yst259teYrcDOldNZcSAAAAAAAACQjO25pdm0xJyVnn/FK3M9zIa+NHQEAAAAAAKCTJCMZyvMkV3fRujXmQq6u9GaWx44AAAAAAADQaUXXLwCPrPRmPkgpHcuzHnfqSkppTiISAAAAAACAzaiMZEyeJxkJxlNbVEpeydWQd8aOAAAAAAAAQCYZybaWZtfmUkpHqvus9GZubHd/AAAAAAAAGEop/X/v3BJVuTshzwAAAABJRU5ErkJggg==){height=\"60px\" width=\"240px\"}](https://pytorchlightning.ai)"]}, {"cell_type": "raw", "metadata": {"raw_mimetype": "text/restructuredtext"}, "source": [".. customcarditem::\n", "   :header: Tutorial 1: Introduction to PyTorch\n", "   :card_description: This tutorial will give a short introduction to PyTorch basics, and get you setup for writing your own neural networks. This notebook is part of a lecture series on Deep...\n", "   :tags: GPU/TPU,UvA-DL-Course\n", "   :image: _static/images/course_UvA-DL/01-introduction-to-pytorch.jpg"]}], "metadata": {"jupytext": {"cell_metadata_filter": "colab,colab_type,id,-all", "formats": "ipynb,py:percent", "main_language": "python"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.7"}, "papermill": {"default_parameters": {}, "duration": 60.734817, "end_time": "2021-12-04T15:54:21.407766", "environment_variables": {}, "exception": null, "input_path": "course_UvA-DL/01-introduction-to-pytorch/Introduction_to_PyTorch.ipynb", "output_path": ".notebooks/course_UvA-DL/01-introduction-to-pytorch.ipynb", "parameters": {}, "start_time": "2021-12-04T15:53:20.672949", "version": "2.3.3"}, "widgets": {"application/vnd.jupyter.widget-state+json": {"state": {"1258f7118a0c424184d1474fa7c7c996": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_2a2ddc2695ca450cbb62218c40c78f0d", "placeholder": "\u200b", "style": "IPY_MODEL_c9b7c58a69a648deb8285855bed4a010", "value": "100%"}}, "2a2ddc2695ca450cbb62218c40c78f0d": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "3ae1c6ddf0434903b17b358d083da8db": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "4d30509a31df4d158d2bb33eb9986e6a": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": ""}}, "61c30cfd473447a38bffb83685bd5f10": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_cd512299253f4dca9324b2260bf7d58c", "placeholder": "\u200b", "style": "IPY_MODEL_eaacc50a6b044949945e0b3f82453cca", "value": " 100/100 [00:01&lt;00:00, 84.02it/s]"}}, "8222db0c1ce847c18b82712853539f5e": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": ["IPY_MODEL_1258f7118a0c424184d1474fa7c7c996", "IPY_MODEL_fa1ac1649a95469d9415b8ad881b0c26", "IPY_MODEL_61c30cfd473447a38bffb83685bd5f10"], "layout": "IPY_MODEL_3ae1c6ddf0434903b17b358d083da8db"}}, "9c98efcaa23c420fa86f3b36f45e2f83": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "c9b7c58a69a648deb8285855bed4a010": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": ""}}, "cd512299253f4dca9324b2260bf7d58c": {"model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null}}, "eaacc50a6b044949945e0b3f82453cca": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": {"_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": ""}}, "fa1ac1649a95469d9415b8ad881b0c26": {"model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_9c98efcaa23c420fa86f3b36f45e2f83", "max": 100.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_4d30509a31df4d158d2bb33eb9986e6a", "value": 100.0}}}, "version_major": 2, "version_minor": 0}}}, "nbformat": 4, "nbformat_minor": 5}