sessions/4_deep.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# Deep Networks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "In the last exercise we finished with a simple single layer MLP with softmax for multiclass classification. In this session we will design deep models and observe their performance. To do this, we will use the [Keras API](https://keras.io/) with tensorflow, which will allow us to design, train and evaluate simple models much faster and with fewer lines of code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt  \n",
    "%matplotlib inline\n",
    "from utils import plot_samples, plot_curves\n",
    "import time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "# force random seed for results to be reproducible\n",
    "SEED = 4242\n",
    "np.random.seed(SEED)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Before building deep networks, let's first train the same architecture as the one in the previous exercise, this time using keras. \n",
    "\n",
    "Let's begin by loading the MNIST dataset: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(60000, 28, 28)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAD8CAYAAABJsn7AAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXl8jNf+xz+TRWQnsQWtXFw0SSVFUc21xZaS2LdL+Kmg\nqrRa1WoFjaV1W/tSS8lFFbXETqUo0dpjX4vaao0sRhKJmfn+/pj7HJlMkMjMPM8z+b5fr/NqnmVm\nvj49z/c553vO+R4NEYFhGIaxPQ5yG8AwDFNcYQfMMAwjE+yAGYZhZIIdMMMwjEywA2YYhpEJdsAM\nwzAywQ6YYRhGJtgBMwzDyAQ7YIZhGJlwsuWPaTQaRS67IyKNXL/NmpjDmuQP62KO2jXhFjDDMIxM\nsANmGIaRCXbADMMwMqFKB/z2228jNTUVa9euxdq1a+U2h2EY5qXQ2DIdZVED5h4eHgCA3377DSEh\nIcjKygIAhIWF4dChQy/9vTyIYA5rYk5xGYRr0qQJfvvtN6xatQoA0KNHj+fez3XFHB6EYxiGUTg2\nnYZWVNq1awcACAkJAQC4ubkBAMqXLy+bTXLi6uqKzz77DADg7u6ONm3aICAgwOQeBwfjO9ZgMAAA\nzp49CwBo0aIF7t69a0NrGSXj6+uLuLg4AEDjxo1hMBjAmzVYH9U44JYtW+K///2vybm///4bAHDu\n3DkZLJIXT09PLFmyBJGRkQAAjUYDIjJ7aCTHK51/7bXXAAB9+vTBt99+a0OLGaUiOd933nnH5Py1\na9dkssg6lCtXToQxJf7xj38AMDbupGdIws3NDbt37wYArFy50io2qcYBjx49Gk5OpuZ++umnAIBL\nly7JYZKsvPXWW4iMjBQPyZ9//omtW7eavYyk3sLXX38NANi5cycAYMaMGTa01rb4+/sDAFq1aoWR\nI0eiWrVqAIwvoT179mDAgAEAime9yY9OnTqZOd/ly5fjq6++kskiy1KqVCkAwJEjR1CxYsVn3pfX\nAQMQDthacAyYYRhGJlTRAv7ggw/QoEEDk3MXL14Uo7TFkVu3biEtLQ3jx48HALPwDAD07dsXI0aM\nMDn3zTffAABycnKsbqOt8fT0xOTJk9GzZ08AgLe3NwDg8uXLAAC9Xo8mTZrgl19+AQC88847uHDh\ngjzGKoAmTZoAQL6hqD59+tjaHKsh9fqe1/qVDSluaIsCgApTKlWqRJUqVaKrV6+STqcTJTs7m2rV\nqlWo73pesaUGRdUkd/H19SUHBwdycHAwu9a3b1+6desW6fV60uv1dPv2bapfvz45OTmRk5OTXWni\n7e1N3t7etHPnTjIYDHT//n26f/8+zZ07lxo2bEguLi7k4uJCjo6O1KVLF9JqtaTVauncuXPk6Oio\ninpS1LryjH8PEZGoI2lpaZSWlkZNmjSxm+dn8ODBJr7jeUWv15scp6enU0BAAAUEBFjNp3AIgmEY\nRiYUG4Lw8vLC7NmzAQCVK1c2ubZs2TKcP3/+mZ91c3ND1apVxT06nc56hsrIgwcPTI6rV6+OSZMm\nAQA6d+6MzMxMXL16FQAQEREhpqDZE4GBgdi4cSMA44j22LFjMX36dACAVqs1u3/NmjV48uQJACA+\nPh5t2rTBli1bbGewAnBzc8P8+fPNZsh88sknAIA9e/bIZpulee+990yOz5w5A8C4qOvOnTsoXbo0\nACAzMxMJCQkICwsDANSpUwdr1661+jOjWAccEBCAiIgIk3OSM5FmP0iULVsWb731FgYPHgwA8PHx\nQd26dfHrr78CANq0aWN9g2WmZ8+eiI2NFdNqLly4gIEDB2Lfvn0yW2ZdduzYAUdHRwBA27ZtsX37\nduFQnkXNmjVtYZpiGTdunIiTS5w+fRrr16+XySLrkp2dDcDojOPj4wHk74CdnZ3RokUL8blNmzZZ\n3zglxWsAUFBQEAUFBdHdu3fNYjTDhw+n4cOHEwDq168fXb58mS5fvkx///13vjGdzMxMyszMpMDA\nQNXGsF5UmjZtSk2bNqUrV66QTqejQ4cO0aFDh8jf37+o8UFFa9K4cWNq3Lgx5eTkCA1e9BkPDw+a\nMmUK5eTkUE5ODul0OmrVqpUqNLFEXalfvz7Vr19fjKlIsd/ExERq1qyZXdaV0NBQqlChAlWoUOGF\n/46qVaua+A83Nzera8IxYIZhGJlQXAiiYcOGAIAyZcqYnE9OThaTosPCwjB16lQxwVqKZeXFxcUF\ngHFCvhT7sTd++uknAMbl2BcuXBAxrPzin/aEl5cXAOD69etISkp67r2VKlUCAGzduhWvv/66OH/0\n6FHs2LHDekYqjGnTpgEwH1O5efOm1RccyEVhQnA+Pj5WtCR/FOeApcrxv+6FYNGiRejfvz8AiFhv\n3kGEZ9G8eXNR+eyZqlWrCocSGxuLbdu2yWyR9fHy8hLO+OHDh2bXvb29xYqu3M4XAAYNGmR9AxVC\nt27dRONGQsoTsnfvXjlMUhy9evUSf2/btk3Ejq2J4tJRnjhxAoBxdDs3M2fOxLBhw/J+H4BnO2Bp\n9kOLFi2e+yYkO0inN3bsWAwZMgS+vr7iXLVq1cTAZWFRuibu7u4AgBs3bmDRokUAjHUkN9WqVUNU\nVBT69etncv7nn38GYFyoUpiHTE5NgKLVlZMnT5olavrzzz8BGPNrp6SkvLRdSq8rBSEkJAS7d++G\np6cnAODRo0dITk42acSUKVMGEyZMAIAX9qgLqgnHgBmGYeRCSSOWAOjEiRN04sQJk9HIpKQkGj9+\nfL4rV/KuXpFKRkYGxcTEUExMjKpHcQtTatSoIVYz6XQ6Wrx4sV2ObOcuw4cPp+TkZEpOTiaDwVCg\nEhgY+MKZMUrTpCh1pWLFinT79m2zZ6RKlSpUpUqVItc7NWoCgGrVqkUbNmygDRs2kF6vJ6KnqwKf\nVZYsWUJLliyxmCaKiwHnx5kzZ0ScryA8evQI/fr1E3P+igsXL14U3e24uDj06dNHdKFWr14tp2lW\nY9q0aSLuXa9ePfz73/8W84IB46KdjIwMAMbQw/bt24tdFrQtW7agbNmyZuftLd1kYXB0dMS3336L\n8PBwAMaGaEFyIAcHB1vUDsU5YCk+lzsVXqtWrURilYIwduzYYud8JQ4ePAgASEtLg4eHB6pUqSKz\nRdZHisedOXMGS5YsMblWrVo1zJ07FwDw5MkTfPzxxzYZXFEStWvXNnMsGzZskMkaZeDq6iqcb0HR\n6XQYO3asRe3gGDDDMIxMKK4FnN82OblH9p+HlBNg/vz5FrVJLZQrVw5jxowB8HQ63+HDh+U0SXbe\neecdMTd60qRJz80hYm9I6SYdHBxM5sprtVqzGSPFjZycHOzevRt16tQB8DR16enTpwHALNXtP//5\nT3h5eVm+56C0gHnNmjWpZs2adOPGjQKlj9Pr9ZSenk7Hjx8nV1dXcnV1LTaDCLlL9+7d6dSpUyYD\nBuPGjSt2Ayu5S9OmTclgMND58+fp/Pnzqh5sKqwu4eHhYkA270B1//79LaKvEnQpqu2RkZEUGRkp\n/ElERARFRETYTBPFtYClBNnh4eHYunUrgGcnUpaSje/evRvHjx+3jYFWxtnZWWSl2rx5s3gj56Vq\n1apo0KCBSDTUu3dvEJFYATdu3DjMmzfPNkYrDGmT1r59+wKAWVL64sD//d//me1/JtWNK1euyGES\nkw8cA2YYhpEJxbWAJU6fPi22od+2bRt8fHwwceJEAMCPP/4IAC+9ykvJ1KlTR6y2OXr0KFJSUhAU\nFCSu527x5l67npOTg9WrV4tcuC/Kj2DPREdHAzC2gBMTE8UWRMWZzMxMDB8+HID1N5pUE1JdAYwz\nh6TVgbZCsQ4YMC6fBJ4mUykOdO7cWfw9d+5cODs7iwG1vLu2bt68WeQ83rVrl10mXC8svr6+iIyM\nBGB8Qffq1UskYC/ObNmyJd99A4szJUuWNJkfvWbNGpsP0nIIgmEYRiYUl4xHDkhByUTKly8vWrWv\nvfaayb1TpkxBcnIyAGDJkiVISUmx2nZLStKkMKxatQpdu3YFALz//vsWHYiUUxOAn5/8KIomQUFB\nYvD+r7/+QvPmzXHjxg2L2FVQTdgBQ70VyJqoUZOAgAAcO3ZMvKT+9a9/WXTEnx1w/qixrgCAn58f\nli9fDgB49913LTqmxA64EKi1AlkTNWpSrlw5XLt2De+//z4AYz4MS8IOOH/UWFesTUE14RgwwzCM\nTHALGPwGzw/WxBxuAecP1xVzFBmCYBiGYZ7CIQiGYRiZYAfMMAwjE+yAGYZhZIIdMMMwjEywA2YY\nhpEJdsAMwzAyYdNsaGqfs2cNWBNzWJP8YV3MUbsm3AJmGIaRCXbADMMwMsEOmGGYYoOzszOcnZ3x\nww8/gIig0+mg0+kQGhoqiz2K3hGDYRjGkvznP/8BAPTr1w8GgwF//fUXAKB69erYt2+fze1hB2wn\nhIaG4sMPP0RISAgA4/ZF1apVw8aNGwEAAwYMwL179+Q0kWFkxcPDw6SlGxcXh/feew8ArLaxwYvg\nEATDMIxcEJHNCgBSYrGlBpbWxNPTkzw9PSk5OZkMBsMzy7Fjxyg8PLxYaFJQzYYPH05ERHq93qys\nW7dOUZrw81N0Tbp37y7+/yYkJFDJkiVl14RDECrG19cXv//+OwCYbFEPAE+ePEFOTg6cnZ0BAMHB\nwfj555+xdOlSAMDHH3+M7Oxs2xosE926dQMAjB49Gt7e3nBwMHb8/Pz8YDAYpAfZBGlnZXvE2dkZ\nb7zxBjp16gQAGDJkCDw8PPDw4UMAQOvWrXHgwAE5TbQKXbp0EX/fuXMHjx8/ltGa/6HUt1V+JTAw\nkAIDAykuLo6IiI4ePUpHjx6lrl27Fos3eO7i7e1Nu3btMmvpLl26lJYuXUr+/v4EgKpVq0bVqlWj\nVatWmdx37do1KlGiBJUoUcJuNMlbatSoQV9//bVo9eh0OtLpdHT27Fk6e/YsJSYmUmJiImVmZlJm\nZibpdDrKyMigjIwMGjlypKI0KYouPj4+FBwcTAsWLKAFCxbQhg0b8m31S+X+/fsUGBhoN8+Pv78/\n+fv7k1arFf/G0NBQq7R8C6sJx4AZhmFkQhUhCE9PTwwcOBCjR48W5xYtWgQvLy8AwOLFixEREYH5\n8+cDgOiW2zOTJ09G06ZNTc79/fff6N+/PwBjCAIALl++DACIiorCqVOn4OHhAQBwcXER99gTbm5u\nGDduHABj6KFy5crQarUAgNTUVFy7dg29e/cGANy8eROenp44e/YsAGNIQpopIk1XUiu+vr749ttv\nAQBvvvkmAgICxLVHjx7ho48+ws2bN8U5FxcXTJs2DYBxc9NKlSrhzJkztjXaSjg5Gd2cm5sbfv31\nVwDAH3/8IadJAkU7YMlZxMfHo1mzZrhx4wYAICwsTDgWAOjYsSMmT54sKtmwYcMUI7Clady4MQDj\nNtp5ycrKeqZTzcnJwYQJE6xqmxJo27YtPv74Y5Nz7du3BwDs2bPH7P6NGzfCz8/P5NgeWLx4Mdq1\na2dy7tSpUwCAiRMnYvXq1WafefvttwFA7Cptj+j1egCAwWCQ2RIjHIJgGIaRCcW2gAMCAjBr1iwA\nQNOmTbFr1y4xait1KSXi4+Nx+/ZtLFq0CAAwY8YMvPnmm7Y12EbUrl0bwNNuVW4GDRpka3MUR+PG\njaHRGBNRbdmyBVOmTDFr+UohiCVLlsDBwUG0hr755ht8+eWXtjXYSrz66qsmx4mJieL5SUlJMbu/\nXr166N69u01sY3KhtBFLLy8v8vLyosTERMrOzqbs7Gxat24dubu7v/Czvr6+5OvrSydPnqSAgAC7\nGcWViqOjI/3666/066+/5jvXt3r16jYfxZVbk7zl2rVrYrZDly5dTK61a9eO9u7dS1qtlrRaLel0\nOtLr9fTVV1/RV1999cwZIUrQpLC6LFq0SIz4T5s2jTw9PZ97f2xsrLj/7t27VKtWLbupK9WrV6fq\n1auTXq+nrVu30tatWy32nBRVE8W1gKXBj0aNGolBtYLEpEqUKIHZs2cDAF577TVUqlRJDK7YCy4u\nLmjevLk4TklJQWZmJgCgcuXKcpmlWD755BPcvHkTU6dOBQCEhISgRIkS4vrVq1fRu3dvJCUlATDG\nye2FGTNmoFKlSgCMce28vcbclChRAsOHDxfHq1atwvnz561uoxyUK1cOAODt7Y309HSZreEYMMMw\njHworbuwatUqWrVqFen1eoqMjKTIyMgXfkZaaCB1oVasWGGX3e0OHTqYhBz27t1rcjxv3jzy8/Mj\nPz8/m3Wh5NYkb4mNjRUhCKlI9eLhw4f0119/iZBDYUM2cmpSVF3yK66uruTq6kp3794lvV5PZ86c\noTNnzthdXckdgpBKcHCwRbV8WU0UF4KQlkB26dIFX331FQBj11uagibh5+cnBhW6du0KZ2dnZGRk\nAICYB2pv5J4uBcAsh+nAgQPFQNyaNWsQFRWljOWWNkSaapUfgwYNwooVK2xojXJp2bIlxowZAwAo\nU6YM1q1bZzLP3t4ZMmQI5s6di2bNmgEAmjdvjnr16j3z/sTEROzYsQM//PCDZQ1R2tvK3d2d3N3d\naf369ZSVlUVZWVn5LpfMzMykq1ev0tWrVykzM5P0ej2NGTOGxowZY3dvcKm89957z024k7e8jBZq\n0ySPzfnWlZf9PiVpUhRdAFCJEiVo0qRJNGnSJNqwYQMlJyebaFSYQTcl6VIQ+zQaDWk0GlqxYoXJ\nsvQnT56Y+ZT09PR8i3TP9evX6fr16xQdHU2Ojo7k6OhYJE04BswwDCMTigtBSGGEDh06oH79+gAA\nd3d3NGzYUHSnjx8/jtTUVBw/fhyAcelxw4YNsXbtWnmMlonHjx/jv//9LwBgx44dyMzMRK9evQAY\nlx5/9tlnYmXc119/LZeZViU8PBzz5s0DgGdmNiuu+Pv7AzDO/hg1atRzu9i9evVCTEyMjSyzLVKd\nmDx5slgV6eLiAo1Gg2vXrgEAvv32W+zatQsXLlzI9zvee+89vPPOO2jbti0AYP78+ahSpQoAIDY2\n9uWX9Sutu1CYMmrUKBo1ahQZDAb6/PPP7bYLJZW8IYhjx449897du3eTwWAQYZz69evbjSa+vr7U\no0cP6tGjB927d89ksE3K9rVhwwZx3lL1TU5NXub5SUpKoqSkJNF9TklJoZSUFBo/fjzVqlVLlHv3\n7lFmZqY4VpMuhbV1xYoVJqEIKVNaQT7r7u5O69ato3Xr1pmELqpVq/bSmiiuBVxQqlWrhs8++wyA\nMbfnzJkzZbbI9kj5W/MjIiICy5YtE2/8n376CcHBwaKHoWY6deqE77//3uy8VqvFmDFjRMtPaq0U\nV9avXw8A2L17NzIyMjBjxgwAwIMHD0zu0+v1cHFxEQNS9joHGACGDh0KwDgfuGnTpti+fTsA46Bk\n3oH+vGRkZODcuXMAjPlF7t+/D8CYg+Vl4RgwwzCMTKi2BdyqVSt4enoCME47k1aE2TMnT57ExYsX\nAQA1atQwSTGYl0ePHuHYsWOiBVy1alWMGTNG9BrUzLOmS7Vv3x537twRLeDiTmxsbIHuO3DggF3v\nAJKb5ORkAMDYsWMxY8YMsYltYmIizp49i8WLFz/zsy1atEDnzp3F8YABAwAAt27deml7NP+Lo9gE\njUZjsR+7cuWK2G4nJCTErFtVGIhIYym7CkthNZF2cZ07dy5ycnIwatQoAMDMmTNFqj0JDw8PMfg2\nZMgQHD9+HHXq1CnQ7yhVk7p16yIhIQHe3t4AjGGH3Okma9SoIbqV0iCJo6OjReySUxPAss8P8DTd\nq5TaNSwsDABw+vTpQn2PUuvKi/D29hbPR2ETWW3dulWEM65evWp2vaCaqLIFPHjwYPj7+4v160Vx\nvmpj4cKFAIAGDRqgb9++mDJlCgCY5frVaDTGIH8+WdPUTEJCAry8vKQBGHzyyScm2c6ioqJEJrCM\njAzOEPcMvL298eOPPwIwLsQYO3ZsoR2v2klPT8eQIUMAAF9++SV8fX3FhgYS0syRpKQkGAwGkbB/\n9uzZFtnQgGPADMMwMqGq5pG7uzuAp9nRpHnAxQkpzPDFF1/Ay8sLbdq0AQC4urq+8LPPW6arFry9\nvUFEorV25swZNGzYEAAwdepUEdMDjHE+XnpsTqlSpTBkyBC88847AIAbN27k240uDkg9qdTUVKSm\npoqQnq1QVQzYzc0NALB//36UL18egYGBAIoeglBrDAswbjcPGKfX5I51tm7dGvfu3YOLiwsA40DL\nxIkTcenSpQJ9r1I1mTlzJgYNGoS0tDQAxilAUipOjUaDjIwMkV6yd+/eJvueFRU1x4BLliwptijq\n06cP2rZtix07dgAAhg8fXqSpZ0qtK3JSUE04BMEwDCMTqmoBd+3aFQCwcuVKREdHIy4uziJ28Rvc\nHCVrcu3aNZFsPDcjRozAhQsXsG3bNqvYpaYWcPXq1VG+fHkAxg0KvvjiCzEr5O7du5g2bZpYmFHU\nRPRKrityYZezIGrUqCH+VsqupoztkRwJY440lWz9+vUiJHXkyBHExcWJHWN0Ot1zd8hgbIeqHHB8\nfDwA4yKM3NvSMwxjZOfOnQAgFikxyoZjwAzDMDKhqhiwteAYljmsiTlqigHbEq4r5hRUE5s6YIZh\nGOYpHIJgGIaRCXbADMMwMsEOmGEYRibYATMMw8gEO2CGYRiZYAfMMAwjEzZdCaf2OXvWgDUxhzXJ\nH9bFHLVrwi1ghmEYmWAHzDAMIxPsgBmGYWSCHTDDMIxMKNoBz5kzB3PmzAERwWAwiHL8+HH4+vrK\nbR7DqJqwsDCEhYVBr9cjPDxcbnOKJYp2wAzDMPaMYhOyz5gxA9HR0QDMd794/fXXceDAAdy7d0+c\nmzlzJlatWmVTG5VGkyZNEBAQAAAYPXo0/Pz8TK6npqZi9OjR4jglJQUAVKVbcHAwduzYgR9++AEA\n8OWXX770d7Vv3x5Xr17FiRMnLGWeKiEijB492mpbOTHPgYhsVgDQi0qJEiWoRIkSlJiYSDk5OZST\nk0MjRoyg8uXL05w5c2jOnDl04MABunDhgigpKSmUk5NDr7zyCr3yyisv/I28xZYavIwmzyvlypWj\ncuXKUXx8PN2/f58MBgMZDAbS6/UvLBkZGZSRkUGjRo1SjSZDhw4lnU5HWVlZlJWVRVFRUYXSy9/f\nnz755BP65JNP6OHDh3Tnzh1V1BNL1JW8JSwsjMLCwkin09GtW7de+nvUqImTkxONGzfO5HkgIvH3\nuHHjyMnJyeqaKK4F/OqrrwIAGjVqhAULFgAAvvvuOwDAkCFD8v1Mw4YNsWHDBqxfvx4A0LZtW9y5\nc8cG1spLjx49hCaNGjUq1GefPHmC7OxsABDbuquJ+/fvAwD27t1bqM9FRERg8uTJAIybUaamplrc\nNrWQu85ERUXJaIntqFixIgBg4MCBGD16tOTEkZSUBI1GgzfeeAOAsQf5ww8/4ObNm1a1h2PADMMw\nMqG4FnDfvn3F3xcvXizQZw4cOIC2bduK1tDAgQMRGxtrFfuURO3atV/Y8v3jjz8AGFuKXbt2RbVq\n1QAAAwYMwLJly6xuo7XIyMgAYNyi/mW5du0aXnvtNUuZpCpKliyJFi1aiOP9+/fLaI3tWLduHQDg\nzTffBBFh0KBBAIBly5ZBo9HgnXfeAQDMnz8fMTExmDJlCoCC+6JCo7R4zXfffUffffcd6fV6KlOm\nDJUpU6bAcZfU1FRKTU2lW7dukY+Pj13HsCpVqkRpaWlmcd1NmzbRpk2bqEGDBvTKK69QqVKlqFSp\nUgSAypYtK+LkDg4OqtREr9eTTqejc+fO0blz5wqtmxRDlr6jMJ+VU5Oi1JX8Sps2bYQOOp2O3Nzc\nrB7vlFuTMWPGUHZ2NmVnZ1N6ejo1a9ZMjDnlvff3338nnU5H169fp+vXr1ONGjWsogmHIBiGYWRC\ncSGI48ePi7979+4NAJg+fXqhvqN8+fLw8vIS06zsCVdXVwDAyJEj4enpaXb9gw8+AADcvn0bOTk5\nJtekgSuGad26tfj7xIkT0Ol0MlpjG9q0aQMnJ6PL27hxI3bv3v3Me/v164fJkyejffv2AIzP1bBh\nwyxuk+JawBs2bMCGDRtw5swZeHh4wMPDo0Cf69KlC1xdXYWDatOmjTXNlA1p9ZLkaPPy119/4a+/\n/sLIkSNtbJltsORLpGzZsmjcuLHFvk9N5F5J+ssvv5i9rO2R3F3/pKSk59771ltv4dq1a+L+vHPq\nrWKUkuI13333Hd25c4fu3LlDH374oYjVlChRgqKiomjYsGGijB07lrRarYiDXrx4kby9ve0uhgUY\n5y86OTnRtm3b8p3bK5GTk0O3b9+mkJAQCgkJsZu4nhTD5Rhw0YperxfP16uvvlqk71KLJlJcV6fT\n0cmTJ6ly5covvFer1ZJWq6WGDRtaRRPFtYAZhmGKC4qLAUuMGDFCLMqYOnUqpk6dWuDPTpgwAenp\n6dYyTVakWN3EiRPxxhtvoGzZsibX/9cqgKOjI8qVK4f3338fgDGGZQ/dTI1GA41Gg9KlSwMwLk0u\nzFJi6fMA4OBg2v74+uuvRejmww8/xOzZsy1ktTI5efIkAOD69esyW2IbRo4ciS1btgAAAgMDkZSU\nhLi4OAAQC7fatm0LAGJBhjSOdODAAavYpFgHDBidDAB07ty5UJ+bPn06rly5gn379lnDLEWwb98+\ndOjQASEhIeJcjx49zGKa/fv3BwCMHz8eN27csKmN1kDqupUpUwYAEB0djaFDhxb684Bx1eWkSZPQ\npUsXAMArr7yCrVu3AoCq50i/iA8//BAODg7iRVRc+P3337FixQoARkdbunRpjBgxQlzXaDRITk4G\nYBxr8PPzs7pGinbA7777br7nL1++LJbR5j4nLTIICAjAtm3bEBkZCQDPHe1UMwcOHDB5M8+bN08s\nRgkNDTW5t3Hjxli+fLlN7bMG6enpuHXrllhSOnjwYMyYMQMAkJmZiVu3bj3zs2XLlhWOGwCcnZ0x\ncuRI0Vvatm0bBg4cKH7HXmnXrh0MBkOxafnmZvDgweLvhg0bmi3Dl56nmzdv4tq1a6LFbC04Bsww\nDCMTim3a8yvAAAAeNklEQVQBR0dH47333hPHx48fx/DhwwEAR44cQWZmptlnSpYsCQAYN24chg4d\nilGjRgGw3xZwfkhdrLffftvkfPfu3e2iBbx06VLcv38fmzZtEufOnz8PwLi0OPd5jUYjwg2AsRcQ\nHBxscg4AOnbsCKDwiX3Uyj/+8Q8AEMmriivPi+u+8sorcHV1RZ06daxrhFKnjCQkJIh0lMuWLaOS\nJUsWahpIbGys+HzVqlVVOY3GxcWFvL29n7lcMr9y6NAhOnTokNn0tI4dO9rV1KLPP/+cPv/88+em\n2ySi55578OABNW7cWBWaFPb5ya906dKFunTpQk+ePKFTp06Ro6MjOTo6Fuk75dalqLY/q/zyyy+k\n0+koPj6e4uPjraaJYlvAAHDw4EEAL5cqr2LFinB0dAQAVK1aFVeuXLGobbZg1apViIiIwNKlSwEY\nV+c8i3LlyuHdd99FvXr1AECqnCIZj731AqQkKatXrxYrJZs0aQI3Nzdxj8FgEDpInD59WiTguXfv\nXrFp9bq5ueHTTz8FYOwZ/PDDD9Dr9TJbpSycnJzEhgVSoqIJEyZY9Tc5BswwDCMTim4BS1NCCorU\n4o2JiTFJa5l3rqxaiIyMBBGJaXiTJ08W8U6JSpUqAQBWrlxplppSp9OJrXvS0tJsYLHtePLkCQDj\n7JeIiAgAxhivtBQdMG5TFRsba1KPzp8/b9fTE59FeHg46tatK47VtA2VrfDz8xMtYCLCli1bcPTo\nUav+pqIdsDSQVLZs2RfmAKhQoQK+//57ABDTz06dOgUAJgMzakIaRHJ3dwdgTCBy4cIFAMYHqHv3\n7sIBBwcHi88Axq72iBEjsGPHDhksl4e84YSaNWvme19+A7j2jLu7u0lukEGDBhWLHWMKS0xMjPj7\n5s2bWLt2rdV/U7EOOCkpCc2bNwdgdDxr1qwR186cOYPAwEBxXKpUKfTv3x/ly5cX57RarUhY8+jR\nIxtZbVnyxi+rVasm5jpLiaOf9Zm4uLhi5XwLg/SSKl26dKFX0qmRYcOGoW7dutBqtQCA7du3y2yR\n8qhbty769+8vnp/o6Gj8+uuvVv9djgEzDMPIhZKnjJw8eZJOnjxZoB1+c5dTp05RdHS06qfRHDx4\nsMD/5gcPHtCyZcvIy8uLvLy8ijy9SKmaWKJERUVRVFQU6XQ6mjVrlio0KYouhw8fptTUVAoPD6fw\n8HCL66lGTfKWMWPGkMFgoO3bt9P27dttponmf/8Im6DRaAr1YxUqVABg3Cfuiy++eGZu4N9//x3x\n8fHYuXMnAGNOXKm7VRCISLZF8c/TxM/PD4mJiWLiPPB00YE0CHX48GEAwJw5c0yS2RcVpWpiCaRB\n2c2bN6Nu3boi1+uLxhnk1AR4eV22bdsGPz8/k7whlkTNdSUoKAgAkJCQgFKlSonQXlGnbRZUEw5B\nMAzDyISiW8C2Qslv8G7dusHHx0cc//jjjwCsP7CoZE0sRWRkJD766CORlvGjjz567v1qbQFbGzXX\nld9//x0A0KBBA/z999+oUqVKgT/bs2dPAE+X/+emoJqwA4a6K5C1YE3MYQecP2quK7kdcPv27QuV\n/UzakzG/cCc74EKg5gpkLVgTc9gB5w/XFXM4BswwDKNw2AEzDMPIhE1DEAzDMMxTuAXMMAwjE+yA\nGYZhZIIdMMMwjEywA2YYhpEJdsAMwzAywQ6YYRhGJmyakF3tq1asAWtiDmuSP6yLOWrXhFvADMMw\nMsEOmGEYRibYATMMw8gEO2CGYRiZUOyuyLnx9/fH3r17sXDhQgDA+PHjTa6XKFEC33zzDYYPHw4A\n2LBhA7p3747s7Gyb22oNpMTPFSpUQGRkJACgadOmMBgM4p60tDScPHkSGzduBABcvHixULlNGYax\nPdwCZhiGkQs17GDav39/MhgMNHPmTJo5c6bZdR8fHzIYDCalXLlyqt/VdcCAARQdHU3nzp2jc+fO\nkU6nE0Wv15sc5z6v1+tJq9VSv379ivVOt5YucmpiaV0aNGhAJ06coBMnThARmTw7v/32G5UvX17c\n6+zsTD179qSePXvSsGHDFKWLJTXx8vKiHj16UI8ePWj58uV069YtoQkR0YIFC6hGjRpUo0YNi9UV\nVYi1ceNGMhgMlJmZSZmZmVSqVCmT62FhYSYVaP369YXall2pFahHjx6UmZn5TEd79+5dunHjBt24\ncYPu3r1r4oB1Oh0lJiaq0tlY04kWpcipiSV16dixI2m1WlFXDAYDabVaSk1NpdTUVMrKyqLu3buL\n+2fNmiXunT9/vqJ0sZQmb7/9Nu3atcusIZe3xMTEUExMjMXqiqJjwL6+vgCAZs2aAQBu3rwJAGax\n3TfffNPkePny5dDr9Taw0LqsXLkSWq0Wrq6u+V6/efMmsrKyAACurq6oXLky2rZtCwCIiopCjRo1\n0KZNGwDA9u3bbWM0o1jq1asHAJg9ezbc3Nzw66+/AgDmzJmDS5cuiS3Zhw4dioSEBAwdOhQA0KdP\nH/Ed33//vY2ttg4ODg6IjIxE586dARjHWRwcnkZkdTqd2KwVAEJCQkyuW8wOi38jwzAMUyAU3QKW\nWnPu7u4AgEuXLgGAaPX5+fkBAL744guTz23evNlWJlqdwsxkaN++Pfz9/cWxr6+vaLH84x//sLRp\niqd8+fIoWbIkbty4AQAms0YkfHx80LdvXwBA69at8eOPPwKA+K+94ObmhhEjRgAwzqb5888/MXjw\nYADAlStXAEBsyd6rVy+sXr0ab731FgDAyckJn332GQCYtArVTL169bBu3TqTc1evXhVbzMfFxeHS\npUuYNm0aAGML2Boo2gGXLVvW5Hj37t0mxy4uLgAADw8PABBhh//FhooF0hS1mjVrIiYmRnSTJGdj\njW6TkqlSpQpiYmIAAN27d4eHhwf+/e9/AwCuXbsGAKhcuTIAoGvXrmjevDl8fHzE548dO2Zji21D\nxYoV0apVK3H82WefCccLACVLlkSFChUAGENfHh4euHfvHgAgNjbWbkIPEq+//rrJ8fjx4zFp0iST\n8GbdunURFRUFwPgc7dmzBz///LNF7SheTyfDMIyCUHQLuFu3bibHd+/efe790iKEx48fW80mOZAG\nI5s3by7OaTQajB49GtWrVwdgXIxCRKLlK/UCYmNjbWytfNSpUwe//PILypQpY3L+p59+eu7nHj58\nCACYNGkSvv32W6vZJyeXLl3CqVOnAAD/+te/zK5PmDABH3/8sTheu3YtlixZAsC+QnoSO3fuNDme\nPXu2SevXy8sLcXFxonek1+sxdOhQXLhwwaJ2KNYBh4aGilFbADh06BCWL19uco/UZZI4cOCATWyz\nNS1btgRgGpfUaDQvDLVs3rwZ69evt6ptSqBmzZoAgISEBJNwgsSjR48AGGOZJUuWFMf37t3DTz/9\nhDFjxgCw/9CV5DxCQ0PRqVMn7NmzBwCwdOlStGzZUvz7jx07hj59+oixFnskb2Nux44daNWqFTIz\nMwEABw8eFPUKAAYMGIDTp09b3A7FOuDmzZtDo3maUvPatWtmU8ukQQXA+IaKj4+3mX22RJoOVBj2\n79+P/v3748GDB1awSFl88MEHACCcr9QL2LFjB2JiYnD79m0AgLOzM9zc3JCWlgYAuHXrlgzWysfW\nrVsBAP3790eHDh3EtDNfX18QEQ4ePAjAOBhpz84XAJ48eYKYmBjRQwwODsbff/8trjs5GV3jqFGj\nAED0BiwNx4AZhmFkQrEt4Nq1a5scHz9+3OS4Vq1aaNeunTg2GAximpq9IcW2GzRoUODP+Pr6Iigo\nSLT2Tpw4YRXblMDly5dNjl977TUAxoREzFPS09PF39LMIQDIyMjA5MmT8Z///AcAkJOTY3PbbI1O\np8PEiRPFTKrPP/9ctHolNm3aJKahWS08pbRlg7Vq1aJatWpRVlaWWP53//598vLyMrkvKCjIZIng\nsWPHnvmd7dq1oz59+lCfPn0Ut8S0IJr4+/uTv78/zZ8/XywJJSLxd+4iodfr6cqVKzRgwAAaMGCA\nqpbdFtbWOnXqUJ06dUz+7Xq9niZNmkRNmjSxyFJVuTV5GV3ylp9//pl+/vlnszoTFBSkWl2Kqomn\npyd5enpSWlqa2bLj1q1bW10Tzf/+ETahIPs3SRO+v/76a3Fu0aJFGDBggDh2dnbGyJEjTdJSHjx4\n0GyQztPTE4BxocaiRYsAAB999JHZb5JK9rRydHQ0GXiMiYmBt7e3ON64cSM+/fRTAE97EFLc/M6d\nO9i7dy8GDhwIAC+M8alFE+DpXOeBAwdiypQpcHNzE9eysrJED+K7777DkSNHXtouOTUBXn7/s/Ll\ny6NTp06YM2eOyXmphxAZGVmk3oKa6kpu3N3dsWHDBgCmM4wk8vqdwlBQTTgGzDAMIxdK6i44ODjQ\n4cOH6fDhw2bp8fr06UNbt26lrVu30v79+1+YtSh3SU9Pp5o1a1LNmjXtrguVt/j5+ZGfnx9t2rSJ\nUlNTzTKoLVy4kBYuXEienp522a10dXWl0NBQCg0NpR9++IG0Wi1JpKSkUKVKlVTZ1X4ZXXx9fcnX\n15dWrVpFBoOBkpOTKTk5mVq0aEG//fabCEHUrl27SHVOTZpIxd3dnRISEkz8xJUrV2js2LE0duxY\nysnJIb1eT82aNaNmzZpZTRNFidWrV69COdbnlU2bNlFcXBzFxcWZpa+0hwpUkNKsWTPq378/9e/f\nn1JTU01yCM+ZM6dYaNKgQQNKT0+n9PR0IiL68ssvVeloXkaX6dOn0/Tp00mv19OZM2coICCAAgIC\nCACtW7dOOOBevXoVSWM1aSKV6OhoE39x+/ZtKlOmjLiemJhIBoOBunXrRt26dbOaJooSq2PHjpSV\nlWUyAFeQsnjxYnJ1dTUp/4sN2W0FKmxp0qQJrVu3TjjgGzduUHBwMAUHB6tCk5IlS1LJkiVp1qxZ\nhcpzXKZMGUpJSaGUlBQiIpo2bZoqHU1B64qTkxM5OTnR5MmTSavVklarpQcPHpC3t7fJfTt37hQO\neOrUqUWqW0rXJHcJCgqioKAgunPnjnC8t2/fFi8mqUgOeMaMGTRjxgyracIxYIZhGJlQ1Dzg+Ph4\nTJ48GQAwZswY3LlzR/y9ceNGMVIpre1PSUkBAIwePdruV+4Axlkd0oj+xo0bsXDhQrGs9kXs2bMH\nderUQfv27QEYs2NJ3yWlIVQyUurA8PBwdOzYsUCfcXR0RKNGjVC6dGkAxhkhv/32m7VMVAT169cH\nYLpKNCkpyWQOMGDMHSJhL5vXvgg3NzeRbrJcuXIAgK+++goAcPbsWQAQmx+UKlUKALBv3z6r2qQo\nBww8nX62bNkyUWmSk5MBmOdzlXbIKC5LSkuVKiUSqTRu3Bgff/yxmSbS8u3Y2FiULl1aLNMFjA78\nf902APnnx1UqUiKZ8PBwTJkyRSxKmTZtmkibKCE9PG+88YbJ1MRdu3aJaUf2iuRYNBoN9u7dC8DY\nQMlLcnKyqCv2uoApL6+//joCAwPF8fHjx7FgwQJx7ObmJjQLDAwEEYlGnrVQnAOW3sZ5VzcBpm91\nAEIsPz8/sd7fnsnOzhb/zooVK4qE9LmRHqp58+bley23A1YTEyZMAGBcpVWvXj2RIHvGjBliDuuR\nI0dQu3ZtkSGuQoUK2Ldvn1jjP3v2bBksty2dOnUCABCR6EFKrThpfvQHH3yA0NBQkdxpzZo1Mlhq\ne3K39A0GA2JjY0Uj5O2338bChQtRq1Ytcc9vv/1mljXN0nAMmGEYRiYU1wJ+Fs2aNUPdunVNzknd\n6zfeeCPfHKf2xr1797Bw4UIAxoxWFSpUgKOjY6G+QwrX6PV6k+6X0tFqtQAgdrt4Fps2bbKFOYol\n98rI3HHf9u3bix5ko0aNoNVqMWnSJLP77Jnc4yV6vR7//Oc/sXbtWgBARESESS6IlJQUvP/++1a3\nSXFLkZ9F165dsWrVKpNzhw4dAmCMC6ampr60XaTSpZTR0dEmSVUiIiLQtGlTAMg31KDRaPDqq68C\ngEnqvfxQqybWRE5NgILpMmTIEADG5PLSEu2srCyUKVNG1AmtVotRo0ZZbJshtdQVV1dXkTM875ZE\ngOkg7eDBg4sUGy+oJhyCYBiGkQslT5q2VVHTRHLWpHhqUlhdPvjgA8rIyKCMjAzS6/VkMBgoJiaG\nYmJiqFatWnajS2FtlbItrlixwmQx16FDhyg8PNzmmqgmBGFNSCVdKFvCmpgjpyYA65IfateEQxAM\nwzAywQ6YYRhGJtgBMwzDyIRNY8AMwzDMU7gFzDAMIxPsgBmGYWSCHTDDMIxMsANmGIaRCXbADMMw\nMsEOmGEYRiZsmo5S7csGrQFrYg5rkj+sizlq14RbwAzDMDLBDphhGEYm2AEzDMPIhKK3JJJ2d+jY\nsSPq1KmDP/74AwDw/fff4+rVq/IZxiiahg0bAgDGjRuHli1bQqfTAQCaNGkidkRgGCWgOAcs7fu2\nZcsWlClTBgDg4OCAK1euoHXr1gCMu9327dtXNhsZ5RIaGiq2nvf29sajR4/E1uKZmZlymiY706dP\nBwB4eHjg3r17uH79OgCgXr168Pb2RufOnQEYt66Kjo7GokWLZLO1uMAhCIZhGLlQ2vYhK1eupJUr\nV9Ljx49p5syZNHPmTGrSpAl5enqKe5KSkqhs2bLFcksVWxU1atKkSRO6d+8e6XQ60ul0lJycTA0a\nNLALTYpaV8LCwignJ4dycnLEFkV6vV4Uaesi6dry5ctVoYutnodFixYREVGFChWoQoUKFtNEcSEI\naXfj3r17Y82aNfnec+bMGfTu3RvTpk2zpWmMQpF2hp4yZQp8fHzE+SlTpuDgwYNymaUoPDw84Ojo\nKI5/+eUXk12Rr1+/LnbMBoBdu3bZ1D6lUr9+fQBAhw4dkJ2dLTl9i6E4Bzx48OBnXpNiwm3btsXt\n27dtZZJiaNy4sdg2e/369Zg4cSIeP34srr/33nsv/I7q1asDALp3746HDx9axU5b8/XXXwMAQkJC\nAAADBgwAAPz000+y2aQk3N3dMX78eHGclpaGTz/9FKdPnza57/jx47Y2TdHUrl0bW7ZsAWDc0r5n\nz564e/euRX+DY8AMwzAyobgW8POQWjalSpVCUlKSzNbYni+//FJ0gSIjIxEZGWlyXaPRPLOLlPda\nq1atnhniURNeXl6imyiRmJgIAMjOzpbDJMURHByMwMBAcdy/f3+z1i9jSlBQEL755hsR0po7dy7i\n4+Mt/juqccB169bFp59+CgCIj48XU42KE5cuXUKLFi0s8l316tWzCwdctWpVMXURABISEnDjxg0Z\nLVIOHTt2BACsXbsWADBx4kQAxvAV83zef/99tG7dWkxdnDJlilV+h0MQDMMwMqGKFnDp0qWxfft2\nlCpVCgDw+PFjBAQE4OjRozJbZluGDBmC3bt3AwDat2+PihUrmlxPTU3FypUrxXFQUJDoaq5atcrk\n3ryftRdKly6NN998EwCwb98+ma2Rl88//xwAQESYPn06xo0bJ69BKqBx48YAIBalSGFPq628VcOc\nvbp161JycjLt2LGDduzYQX/99RcZDAZKSEighIQEqlu3rmrndxbF7oKUnj17Us+ePUmv15NOp6P9\n+/fT/v37KSQkxC40cXZ2FvVAmv+r1WpJq9XShg0baOjQoRbTUk5NXqauSM+LXq+nVatWUVhYGIWF\nhVm8jqlJk+cVd3d3Onz4MB0+fJj0ej1t3LiRvL29ydvb22qa2HRb+qLk7ixdurSYI+zp6Yl69eqh\nU6dOAIB27dph+PDhLx3bIjvNZ/rKK69g5syZAIzzGA0GAxYuXAjgxVPW1KSJNLUuNjYW3bp1M7su\nxfF27NiBo0ePisGU8+fPF8ouOTUBCq+LlBNj/fr1KFu2LPR6PQDjNLS4uDgx7WzFihVFsktNdeVZ\nuLu7Y86cOYiKigIAHD16FK1atUJaWtpLfV9BNeEYMMMwjFyotbuQu/Tr14+uX79Obdq0oTZt2hTb\nLlTuUrJkSZo3b57oluv1etq5cyd5eXmRl5eX3WpSq1YtWrFiBa1YsYLS09NJr9eTRO6lt3q9njp2\n7EiOjo6qqCdF0aVSpUo0ZswYysjIEEuODQYDPX78mB4/fkyJiYk0depUCg0NpdDQ0GL5/DRs2NCk\nbnz//fdF+r4C269GsfIrHTp0oCtXrtCVK1fo7bffLnYVKG+pUqUKXbhwQTjgs2fPkr+/f7HTZNCg\nQbR69WpavXo1PXz4UOghlcI4HDk1saQu7dq1o8WLFwuHbDAYTMqkSZPI1dVVFboUVQtPT0/y9PSk\n5cuXk16vpzVr1tCaNWteKu77MpqoSqwXlUaNGlGjRo3o8uXLVhFLDZqUK1eOypUrR3v27CGdTkfX\nr1+n69evW60CqUETqTRo0EDoIpWIiAhVaGINXfz9/cnf359q1qxJ06ZNEy1ig8FAvXr1UoUuRdVg\n+vTpNH36dNLr9ZScnExBQUEUFBRUZG0Laj/HgBmGYeRCTW+rFxV3d3dyd3en8+fPU+nSpYvFGzx3\n8fHxEdPMpBberFmzaNasWVZ7gytdk7wlIiLCpAUcHR2tCk1s8fz069eP+vXrRwaDge7evasKXYry\n761WrRolJydTcnIyPXjwgNq2bWsxLQtsv1rEKkwZNmwYRUVF2X0Fylt++uknk4GE8+fPk4+PD/n4\n+FitAildk9ylRIkSNGPGjGIfA35RMRgM9OTJkwIPaqtREw8PD1q9erV4VhYvXmxRDQtsvxrEKmyJ\nioqi//73v6p4sCz1bx4xYoRYbKHT6SgjI4M6duxo9QqkZE2k4uTkRE5OTjRr1iwT57t69eoCzQhR\ngia2eH7atWtH7dq1I4PBQA8ePCjwy1uNmtSoUUPEfZOTk1+4MMlazw/HgBmGYeRCqW+ryZMnU8OG\nDalhw4YF/oyLiwu5uLjQkSNHikUL2MHBgZYuXUpLly6l7OxssbVMRkYGdejQwSZvcLk1qVWrFq1f\nv57Wr19PZcqUyfeeuXPn0ty5c82moBV2zricmhS1rjyv+Pr60rJly8Q0NCIq1DxYNWoizXyQ4t6W\n1rSg9it2KXJKSorYaubvv//GmjVrxLb0Z86cwcWLF03uDw0NRc+ePQEYd9WIjIzE5s2bC/RbpNKl\nlJ999plIMQgAWVlZ6NOnDwAUOXepWjQJCQnBkSNHABjzAC9dulRcq1OnDlq1aiWWKkt1XVp6O2jQ\noELtlCynJsCLdQkPDxc7pOzbtw9PnjzJ975SpUqhadOmYil/06ZNUalSJXF9wYIF+PDDD5GTk1Mg\nu9RSVwAgIiICALB8+XJs3rwZ0dHRACy/Y3ZBNVGsA3711VeFOG+88QZatGgBFxcXAEZH8+eff+b+\nXrz22mvQaIz/5vv37xcq25eaKpBEXFwc/v3vf5vs89W5c2eL5UlWiyaVK1fGnj17AABVqlR51vcB\nAPbs2YM9e/Zg8uTJAGCynVNBULoDbtCggdiy6scff8SSJUsAAC4uLvDw8BANlLCwMPj6+iL3s//H\nH3/g22+/BQBs3LixUHappa4AELlQOnfujJYtW1oto2JBNeEYMMMwjEwotgWcl7p16yIoKAgAMGLE\nCJMtVjQaDVJSUjBhwgQAKPRuyUp/g9eoUQMAMHToUHzwwQcAAIPBAAC4desWAOPurZbcqFTpmuQm\ndza0119/HVeuXAFg3Lz1jz/+wM6dOwEAkyZNema3vCAovQUMQPQMq1atCp1OJ33OpKcEAHfu3BFb\nN8XHx2Pjxo3Iysp6KbvUUlfc3d2RkJAAAPDx8UGtWrWsZpfqQxC2RMkVqF69eliwYAEA4y6tUnea\niLBkyRIR8zxy5AgyMjIsZpeSNZELNTjg4OBgAMDMmTPFi9vFxQVpaWniRbRlyxbs37/fYjv8qqWu\n+Pn54ebNmwCAJUuW4N1337WaXRyCYBiGUTpqmDJi7aLUaTSBgYF07949k6lTOTk5lJOTQ/PmzSNf\nX99ip0lxrSesS9E18fT0pF27dtGuXbsoODhYEZqoYk+44krLli3FttgSo0aNAmC9XVoZxl7RarVo\n3ry53GaYwA5YwRARsrKyxFZMPXv2xO+//y6zVQzDWAqOATMMw8gEz4IAQCoZxbUlrIk5cmoCsC75\noXZNbOqAGYZhmKdwCIJhGEYm2AEzDMPIBDtghmEYmWAHzDAMIxPsgBmGYWSCHTDDMIxMsANmGIaR\nCXbADMMwMsEOmGEYRibYATMMw8gEO2CGYRiZYAfMMAwjE+yAGYZhZIIdMMMwjEywA2YYhpEJdsAM\nwzAywQ6YYRhGJtgBMwzDyAQ7YIZhGJlgB8wwDCMT7IAZhmFkgh0wwzCMTLADZhiGkYn/B/c9pd8Y\ngPbPAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fa77007d4d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from keras.datasets import mnist\n",
    "from keras.utils import np_utils\n",
    "\n",
    "# Load pre-shuffled MNIST data into train and test sets\n",
    "(X_train, y_train), (X_test, y_test) = mnist.load_data()\n",
    "# Display some of the samples\n",
    "plot_samples(X_train)\n",
    "X_train.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "We flatten and normalize images:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "X_train = X_train.reshape(60000, 784)\n",
    "X_test = X_test.reshape(10000, 784)\n",
    "X_train = X_train.astype('float32')\n",
    "X_test = X_test.astype('float32')\n",
    "X_train /= 255\n",
    "X_test /= 255"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Categories need to be converted to one-hot vectors for training:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([5, 0, 4, ..., 5, 6, 8], dtype=uint8),\n",
       " array([[ 0.,  0.,  0., ...,  0.,  0.,  0.],\n",
       "        [ 1.,  0.,  0., ...,  0.,  0.,  0.],\n",
       "        [ 0.,  0.,  0., ...,  0.,  0.,  0.],\n",
       "        ..., \n",
       "        [ 0.,  0.,  0., ...,  0.,  0.,  0.],\n",
       "        [ 0.,  0.,  0., ...,  0.,  0.,  0.],\n",
       "        [ 0.,  0.,  0., ...,  0.,  1.,  0.]]))"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nb_classes = 10\n",
    "Y_train = np_utils.to_categorical(y_train, nb_classes)\n",
    "Y_test = np_utils.to_categorical(y_test, nb_classes)\n",
    "y_train, Y_train"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Now let's write the model from the previous exercise in Keras:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "____________________________________________________________________________________________________\n",
      "Layer (type)                     Output Shape          Param #     Connected to                     \n",
      "====================================================================================================\n",
      "dense_1 (Dense)                  (None, 10)            7850        dense_input_1[0][0]              \n",
      "____________________________________________________________________________________________________\n",
      "activation_1 (Activation)        (None, 10)            0           dense_1[0][0]                    \n",
      "====================================================================================================\n",
      "Total params: 7,850\n",
      "Trainable params: 7,850\n",
      "Non-trainable params: 0\n",
      "____________________________________________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Activation\n",
    "\n",
    "model = Sequential()\n",
    "# in the first layer we need to specify the input shape\n",
    "model.add(Dense(10, input_shape=(784,)))\n",
    "model.add(Activation('softmax'))\n",
    "\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "**Exercise**: ```model.summary()``` gave us the total number of trainable parameters of our model. How is this number obtained? Manually calculate the parameters of this model. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7850"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "784*10+10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "We are now ready to train. Let's define the optimizer:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "from keras.optimizers import SGD\n",
    "lr = 0.01\n",
    "# For now we will not decrease the learning rate\n",
    "decay = 0\n",
    "\n",
    "optim = SGD(lr=lr, decay=decay, momentum=0.9, nesterov=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "In Keras, we need to compile the model to define the loss and the optimizer we want to use. Since we are dealing with a classification problem, we will use the cross entropy loss, which is already defined in keras. Additionally, we will incorporate the accuracy as an additional metric to compute at the end of each epoch:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "model.compile(loss='categorical_crossentropy',\n",
    "              optimizer=optim,\n",
    "              metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Now let's train the model. ```model.fit()``` will do the training loop for us. We just need to pass the training data ```X_train``` and labels ```Y_train``` as input, specify the ```batch_size``` and the number of epochs ```nb_epoch``` we want to do. We also pass the test set ```(X_test,Y_Test)``` as validation data, which will allow us to see how the model performs on the test data as training progresses. Let's run it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 60000 samples, validate on 10000 samples\n",
      "Epoch 1/20\n",
      "4s - loss: 0.5890 - acc: 0.8459 - val_loss: 0.3800 - val_acc: 0.8991\n",
      "Epoch 2/20\n",
      "1s - loss: 0.3746 - acc: 0.8966 - val_loss: 0.3359 - val_acc: 0.9095\n",
      "Epoch 3/20\n",
      "1s - loss: 0.3429 - acc: 0.9050 - val_loss: 0.3164 - val_acc: 0.9135\n",
      "Epoch 4/20\n",
      "1s - loss: 0.3264 - acc: 0.9091 - val_loss: 0.3056 - val_acc: 0.9159\n",
      "Epoch 5/20\n",
      "1s - loss: 0.3159 - acc: 0.9119 - val_loss: 0.2992 - val_acc: 0.9168\n",
      "Epoch 6/20\n",
      "1s - loss: 0.3079 - acc: 0.9136 - val_loss: 0.2942 - val_acc: 0.9180\n",
      "Epoch 7/20\n",
      "1s - loss: 0.3024 - acc: 0.9157 - val_loss: 0.2902 - val_acc: 0.9188\n",
      "Epoch 8/20\n",
      "1s - loss: 0.2974 - acc: 0.9170 - val_loss: 0.2881 - val_acc: 0.9194\n",
      "Epoch 9/20\n",
      "1s - loss: 0.2936 - acc: 0.9181 - val_loss: 0.2843 - val_acc: 0.9206\n",
      "Epoch 10/20\n",
      "1s - loss: 0.2905 - acc: 0.9189 - val_loss: 0.2824 - val_acc: 0.9215\n",
      "Epoch 11/20\n",
      "1s - loss: 0.2874 - acc: 0.9201 - val_loss: 0.2811 - val_acc: 0.9222\n",
      "Epoch 12/20\n",
      "1s - loss: 0.2851 - acc: 0.9202 - val_loss: 0.2794 - val_acc: 0.9220\n",
      "Epoch 13/20\n",
      "1s - loss: 0.2831 - acc: 0.9211 - val_loss: 0.2792 - val_acc: 0.9221\n",
      "Epoch 14/20\n",
      "1s - loss: 0.2812 - acc: 0.9216 - val_loss: 0.2772 - val_acc: 0.9239\n",
      "Epoch 15/20\n",
      "1s - loss: 0.2794 - acc: 0.9226 - val_loss: 0.2782 - val_acc: 0.9231\n",
      "Epoch 16/20\n",
      "1s - loss: 0.2777 - acc: 0.9229 - val_loss: 0.2751 - val_acc: 0.9223\n",
      "Epoch 17/20\n",
      "1s - loss: 0.2764 - acc: 0.9232 - val_loss: 0.2749 - val_acc: 0.9217\n",
      "Epoch 18/20\n",
      "1s - loss: 0.2750 - acc: 0.9236 - val_loss: 0.2757 - val_acc: 0.9221\n",
      "Epoch 19/20\n",
      "1s - loss: 0.2740 - acc: 0.9239 - val_loss: 0.2739 - val_acc: 0.9234\n",
      "Epoch 20/20\n",
      "1s - loss: 0.2727 - acc: 0.9251 - val_loss: 0.2713 - val_acc: 0.9231\n",
      "(29.988792896270752, 'seconds.')\n"
     ]
    }
   ],
   "source": [
    "batch_size = 128\n",
    "nb_epoch = 20\n",
    "verbose = 2\n",
    "\n",
    "t = time.time()\n",
    "# 30 seconds for 20 epochs on GeForce GTX 980\n",
    "history = model.fit(X_train, Y_train,\n",
    "                batch_size=batch_size, nb_epoch=nb_epoch,\n",
    "                verbose=verbose,validation_data=(X_test, Y_test))\n",
    "\n",
    "print (time.time() - t, \"seconds.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "We can plot the loss and accuracy curves with the ```history``` object returned by ```model.fit()```. The function ````plot_curves```, which is defined in ```utils.py``` will do this for us."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbMAAAEKCAYAAACYKLs6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xt4VNXV+PHvSgiEcCcBRC4hKlIQ5A5SiqAUiZcXBFtU\npBWrRlQsttWKP9Eqvr611lrbiii1eMEbeEFpRUEpaK2CRAXlTkgCBBC530MIrN8fe4ZMJpNkkszJ\n5LI+z3OemTlzzpnNAFnZ+6y9tqgqxhhjTHUWE+0GGGOMMRVlwcwYY0y1Z8HMGGNMtWfBzBhjTLVn\nwcwYY0y1Z8HMGGNMtWfBzBhjTLVnwcwYY0y1Z8HMGGNMtVcn2g2IlJiYGK1fv360m2GMMdXK0aNH\nVVVL7tiIpAJ/AWKB51B9NMQxY4AHAQVWojrWt/96YIrvqP9F9cVItb3Qx9eUclYNGjTQI0eORLsZ\nxhhTrYjIUVVtUMIBscAGYBiQAywHrkV1TcAxHYE5wMWo7kOkJarfI9IcSAf64ILcl0BvVPdF+s9h\nw4zGGGNK0g/IQDUT1TzgdWBk0DE3A9NOBynV7337hwMforrX996HQKoXjbRgZowxtVgS1EEkPWBL\nCzqkDbA14HWOb1+gc4FzEfkvIkt9w5LhnhsRNeaemTHGmLLbDfmo9qngZeoAHYEhQFvgE0S6VbRt\nZW1AjXXixAlycnLIzc2NdlOqvfj4eNq2bUtcXFy0m2KMqVzbgHYBr9v69gXKAZahegLIQmQDLrht\nwwW4wHOXeNHIGp0AkpWVRaNGjUhMTEREotSy6k9V2bNnD4cOHSIlJSXazTHGRFAYCSB1cAkgQ3HB\naTkwFtXVAcek4pJCrkckCfga6EFB0kcv35Ff4RJA9kb6z1Gj75nl5uZaIIsAESExMdF6uMbURqr5\nwERgAbAWmIPqakSmIjLCd9QCYA8ia4DFwN2o7vEFrYdxAXA5MNWLQAY1vGe2du1aOnfuHKUW1Tz2\nfRpT85TaM6smavQ9s7Dk58P330OTJtCg2v99GmNqiX374NtvYeVKqFsXbrkl2i2KLgtmANu3Q0yM\nBTNjymHHDrjmGpg9G844I9qtqXlOnoSMDBe0vvmm4HHLloJjBgywYFaj75mFpU4diI2FvDxPLr9/\n/36efvrpMp932WWXsX///jKfN378eN58880yn2dMeT38MHz6KUydGu2WVE87dsDgwfDdd7B/P3zy\nCfztb3DzzdCvHzRqBD/4AVx9NTz6KGRlwcCB7vn8+bBtG/z3v9H+U0Sf9czA9dGPH/fk0v5gdttt\ntxXan5+fT506xX/98+fP96Q9xkRK/foQmBM0fbrb4uPh2LHwr1NTenaq7vs4cgSOHi38WNLz+fNh\n1Sro2BEOHy64XmIidO8OEybA+ee75507u+830I4dMGRI9f/+Kqr2BLM774QVK0K/d+wYnDpV9mHG\nHj3gySdLPGTy5Mls2rSJHj16EBcXR3x8PM2aNWPdunVs2LCBK6+8kq1bt5Kbm8ukSZNIS3OT7zt0\n6EB6ejqHDx/m0ksv5Uc/+hGfffYZbdq04d133yWcosqLFi3irrvuIj8/n759+zJ9+nTq1avH5MmT\nmTdvHnXq1OGSSy7h8ccf54033uChhx4iNjaWJk2a8Mknn5TtuzC1TmYm3HUXvPOO++GckACjRsHj\nj5ftOoE9u3IMYkSMKhw65HpH+/a5LdznBw+676Ai+XT+QBYXB9nZ0Lo1hJOIXVW+v2irPcGsJDEx\nbmDaA48++iirVq1ixYoVLFmyhMsvv5xVq1adnq81c+ZMmjdvzrFjx+jbty9XXXUViYmJha6xceNG\nXnvtNf7+978zZswY3nrrLcaNG1fi5+bm5jJ+/HgWLVrEueeey89//nOmT5/Oz372M+bOncu6desQ\nkdNDmVOnTmXBggW0adOmXMObpvyqa8+kdWto3Nj1RuLj3WPjxuH/GSLVs9u+HcaMgaeectc8eLDs\n24EDLiiV9GNAxOWJNWvmtqZNoUsX99i4sftdOCHBPfq3wNfBzw8cgMmTQ/8yEM53GKnvr6aoPcGs\npB7Ud99BTg707Onun3moX79+hSYe//Wvf2Xu3LkAbN26lY0bNxYJZikpKfTo0QOA3r17k52dXern\nrF+/npSUFM4991wArr/+eqZNm8bEiROJj4/nxhtv5IorruCKK64AYODAgYwfP54xY8YwevToSPxR\nTZgq+pt1RYNhRc7fudMNg6WlwYwZ7lrh2rgRbr8dFixwo/xxcdC1K1x8Mfz2t+EP0x096q7Xs2fJ\nn1evngs6gVu7du7RH6SaNi0crAIfGzd2v/cGK+/3529DeX8ZiFTPuKaoPcGsJHXrusfjx92/CA81\nCBjKXLJkCR999BGff/45CQkJDBkyJOTE5Hr16p1+Hhsby7EK/NpVp04dvvjiCxYtWsSbb77JU089\nxb///W+eeeYZli1bxnvvvUfv3r358ssviwRVE1p5f5hF6jfrigbDipz/9tsFz6dNC33M4cOwfr3b\n1q0r2DZsKHyr+sQJ+Pprt8XHh+7NJCa6ANSgAbz6auieVFwcLFrkEif8AaNRIxfMQqnoLwMV+f4q\n8stARXvGNY0FMygIZnl5EQ9mjRo14tChQyHfO3DgAM2aNSMhIYF169axdOnSiH1up06dyM7OJiMj\ng3POOYdZs2YxePBgDh8+zNGjR7nssssYOHAgZ511FgCbNm2if//+9O/fn/fff5+tW7daMAtTeX+Y\nVfQ364oGw0gOU6m64b7AYOXfcnIKjouJgbPPhk6dYPhwF3Q6dIAbb4Q334Q9e2Du3PAGSP7wh+K/\nv7L8QC/v318kvr9wfhkoSUWCYU1jwQwKB7MIS0xMZODAgXTt2pX69evTqlWr0++lpqbyzDPP0Llz\nZzp16sQFF1wQsc+Nj4/n+eef56c//enpBJAJEyawd+9eRo4cSW5uLqrKE088AcDdd9/Nxo0bUVWG\nDh1K9+7dI9aWqi5aPauK/mZd0WBY2vlHj8Lu3eFtWVmFM/H86eQXXeQe/dvZZxffQ7r88vDa7Rft\ne3ZVYZivosGwRlFVzzbQVND1oBmgk0O8nwy6CPQb0CWgbX37e4B+Drra997VpX1WQkKCBluzZk2R\nfSGdOqWanq66dWt4x9dSYX+f1cytt6rGxLjHsti+XXXsWNWEBFVwj9ddp7pjR/jXGDVK9bbbVFes\ncI+jRpWtDRMmuLbHx4f/ZzhxQjUzU3XhQtVBg1zbY2LcY2Kiart2qvXru9ehNhHVpCTVTp1UBw5U\nHTlSdeJE1WnTVBctUt22zf2XqgwV+f4i8fdXnu+/qgGOqIdxoLI2z3pmIsQC0whYaluEeaqsCTjs\nceAlVV4U4WLg98DPgKPAz1XZKMKZwJciLFDFmzQ7EU/nmpmqKdo9K/BumCkvz6V3Z2QU3jZtcr2o\nEycKrhEbC2ee6c4RgaFDISmp+K1pU8/zpMJWke8vEn9/NsxXdXg5zNgPyFAlE0Dk9FLbgcGsC/Br\n3/PFwDsAqmzwH6DKdhG+B1qAR8EMXDDzqAqIF26//Xb+GzTtf9KkSdxwww1RalH0lHeYMBLDRJH4\nYZaX5+Y35ee7IHPiROnP/Y9jx7rHRYtcv+nwYTjrLNi82U2d9GvY0E3K7d4drrrKDfedc47bWrcO\nnaVXG1T078+G+aoOL4NZqOWy+wcdsxIYDfwFGAU0EiFRlT3+A0ToB9QFNnnYVjeQf+CApx8RSdPs\nf85p5b2BX9k9K1U3C2TlysJ19tati8w0x6ZNXcC64AIYN64gWJ1zDrRoEd4E3NrGglHNEe0EkLuA\np0QYD3yCW/jt9H9rEVoDs4DrVTkVfLIIaUAaFORwlFvduu5X3FOnau+vqVESzdR2r4aJjh+HNWsK\nF4ZdudIlS/i1a+d6SiNGuD93XJwrFRoXV/LzUO+1bAnNm0em7cZUR14Gs1KX2lZlO65nhggNgav8\n98VEaAy8B9ynSsicdVVmADMAGjSgYguzBWY0Bhc/M56KVmo7VPw381OnXKHX1asLB65169xQILh/\nTl27wsiRBTX2unWz4GNMJHkZzJYDHUVIwQWxa4CxgQeIkATs9fW67gVm+vbXBebikkMqpwS8BbNy\nq66p7eFQdfX3MjNd4oR/87/evLnwrdZ27VzAGjGiIHB17Fh1EiaMqak8C2aq5IucXmo7FpipymoR\npgLpqswDhgC/F0Fxw4y3+04fA1wIJPqGIAHGq1JMpeAI8HCuWU0XzZ5VJIYJc3MLB6jggHXwYOHj\nmzeHlBQXqEaNcs87d7beljHRJFqRMs9VSIMGDfTIkSOF9q1du5bOnTuHd4FTp+Crr9yv0HfeGbGq\nr/v37+fVV18tsgRMOJ588knS0tJIKKEqib+6flJSUkWaGZbg7zO4Z+VXlntWt97qgpA/mfSWW7yr\n/O1f5HDVKrdC77ffuucZGYUz/+LjXYBKSXGZgf7n/q1JE2/aZ0w0iMhRVa32KxNHOwGk6oiJcXfT\nH388ouspFLeeWTiefPJJxo0bV2Iwi4S8PNcLOfts9xWEq6r0rIL5SysFBqxvv4W1awuCr4jL8uva\n1S162KlTQfBq1coy/4ypbiyY+Xm0nkLgembDhg2jZcuWzJkzh+PHjzNq1Cgeeughjhw5wpgxY8jJ\nyeHkyZPcf//97Ny5k+3bt3PRRReRlJTE4sWLS/2sJ554gpkzZwJw0003ceedd4a89tVXX11oTbML\nLriEtLTH2b4dkpPD/7NVhUnDR464Zeq++aYgcK1a5e5zBbazWzdXWqlbN7d17ux5TWljTCWyYOaX\nmenGuD780P1UjlChtcD1zBYuXMibb77JF198gaoyYsQIPvnkE3bt2sWZZ57Je++9B7gCxE2aNOGJ\nJ55g8eLFpQ4h5uXBW299ycyZz7Ns2TJUlf79+zN48GAyMzOLXHvPnj3MnTuXV15ZBwiHDrm56Lt2\nuU0EevcO789XmRUQjh1z2YLp6fDll+5xzZqCIcLGjV1Pa8wYF7C6dnWb1Us2puazYObXurW7GXL8\nuGepcQsXLmThwoX09C28dPjwYTZu3MigQYP4zW9+wz333MMVV1zBoEGDynTdnTvhs88+ZejQUaeX\nmBk9ejT/+c9/SE1NLXLt/Px84uPjeeqpG+nf/wp69XJrmsXEuIm37dqV9GmFeTXp9Phx19NKTy/Y\nVq0qmFzcogX07QujR7vA26OHa7cNDxrjAZFUXHGLWOA5VB8Nen888EcKpl89hepzvvdOAt/69m9B\ndYQXTbRgFmjvXlfr55574PnnI97NUFXuvfdebrnlliLvffXVV8yfP58pU6YwdOhQHnjggVKv9+WX\nrle2x1cv5cgR90M/8Af6ueeeG/La/jXNXnjhTWbOfIpnnvk3p065/Jey3DeLhBMnXKAK7HF9801B\n/cDmzaFPH1dVvU8ft7Vta4HLmEohUqTOLiLzUF0TdORsVCeGuMIxVHt43UwLZoFeecWltv3gBxHr\nZgSuZzZ8+HDuv/9+rrvuOho2bMi2bduIi4sjPz+f5s2bM27cOJo2bcpzzz1X6Nzihhm7dXO9KRHo\n2XMQDz00nl/9ajJt2ihz585l1qxZbN++vci1A9c0a916IBdffBadO7shxsACtF46dAjef9+tXfXe\ne+41uM5xnz7w61+7HlefPm69KwtcxkRNPyAD1UwARELV2Y06C2aBPJhrFrie2aWXXsrYsWMZMGAA\nAA0bNuTll18mIyODu+++m5iYGOLi4pg+fToAaWlppKamcuaZZ4ZMAPE3VxU6d+7FFVeMZ/TofsTF\nuQSQnj17smDBgiLXPnToUKE1zZ588gkSEsqW/FEee/bAP//phiYXLnRDiS1auGzCoUNd4DrrLKsm\nZkwVE06dXYCrELkQ2AD8ClX/OfGIpAP5wKOovuNFI22eWaCTJ92a7W3bVpu1xzMy3LBgixYFPatz\nzvHms8r8feJKPb3zjgtgH3/svuL27d29rlGjYOBAq45hTDS1EMnbVXBPC2AGqjNOvxL5CZCK6k2+\n1z8D+hcaUhRJBA6jehyRW4CrUb3Y914bVLchchbwb2AoqhEvHG89s0CxsW6r5Cog5Z3nBYUDl9c9\nq3BlZLjgNXcuLPVV1ezc2d2KHD0aevWyYUNjqordkI9qnxIOKbXOLqp7Al49BzwW8N4232MmIkuA\nnniwCooFs2BRWKRzxw63DlVJ87z69+/P8aB2zZo1i27dulVCC0um6jIP337bbd/6fsfr3RseecT1\nwMrYoTPGVB3LgY6IFFtnF5HWqPoz5kYAa337mwFHfT22JGAggYEugmp8MFNVpCzdgEpcpPPLL10g\n8CtpnteyZcsqpU3FCR6OPnAAFi+GBQvclpXl2j1oEDz5JFx5ZdXpKRpjKkA1H5FCdXZRXY3IVCAd\n1XnALxEZgbsvthdO19TtDDyLyCkgBnfPzJPEkRp9zywrK4tGjRqRmJgYfkDbssVlKvjmgnkpLw9y\ncmD//oJl1PzzvCo7Pb4kqsru3XvIyTnEv/6VwsKF8Pnn7v5Xw4aussb//I+rFN+qVbRba4wpi5pS\nm7FGB7MTJ06Qk5NDbqhquMU5cMBFl3btKiWtbs8eN8Qo4nppDRtWnYoV+flu7vjRo7BmTTz33deW\n/fvj6NULhg932wUXRGBhVGNM1Fgwq2JCBbNymTPH5YqvXOkWpPLY6NGu+EhgOajAqhqV6ehR+OQT\nN2y4cKErFQWufcOHwyWXwI9/7DInjTE1gwWzKiZiwWzZMtfd+Oc/4YorKn69Km7vXpg1C/71L/jP\nf1zuS716cOGFBb2v886z7ENjaqqaEsxqfAJImfmzFjZvjm47PPbNN/C3v7miJ8eOuYB1++2u93Xh\nhW4RAWOMqS48DWYiFCpOqcqjQe8nAzOBFrgMmHGq5Pje+wC4APhUlcrrIrVs6W4C1cBglp/vJjD/\n7W9uOLF+fbjuOrjjjkoZUTXGGM94luEggr845aVAF+BaEboEHfY48JIq5wNTgd8HvPdH4Gdeta9Y\nMTGuREUNCma7dsH//Z9bfPKnP3UJm4895jIp//53C2TGmOrPy3S9fkCGKpmq5AH+4pSBuuDKmwAs\nDnxflUXAIQ/bV7zkZPcTP0w7dsDgwfDddx62qRy++gpuuMElZt53n1tN+Z13XIWOu+921eiNMaYm\n8DKYhSpO2SbomJXAaN/zUUAjEaKfmJ6cXKae2cMPw6efwtSpHrYpTCdOwOuvu5qHvXvDG2/AL34B\nq1fDRx/ByJFWC9EYU/NEOwHkLuApEcYDn+BKpZwM92QR0oA0iPBcp/btXXfLn9pXjPr13Twsv+nT\n3RYf75IqKtPOnfDss/DMM67pZ58Nf/4zjB/vJmIbY0xN5mXPrNTilKpsV2W0Kj2B+3z79of7AarM\nUKWPKn3qRDIs+zMac3JKPCwzE8aOhYQE9zohwSVUZGVFsC2l2LmzYCjxd79z97/+9S/YsAHuvNMC\nmTGmdvAymC0HOoqQIkJdXHHKeYEHiJAkcroN9+IyG6MvzPT81q2hcWPXO4uPd4+NG1fO6jGqbjHs\nzp3htddgwgRYtw4++MCtyGxrghljahPPfuSpkg/4i1OuBeaoslqEqSKM8B02BFgvwgagFfCI/3wR\n/gO8AQwVIUeE4V61tYgyzDXbudMFkqVL3WNlJIFs2gTDhrl7YV27umIlf/2rS/AwxpjayCqAhJKX\n57paDzwADz4YmWtGQH6+q0j/wAOuEPFjj8HNN1svzBhTflYBpCarW9eNIZYhPd9rK1bAjTe6dPuR\nI2HaNGgTnBtqjDG1lP1OX5wypud75dgxuPde6NMHtm1zqfZz51ogM8aYQBbMilMFqoB8/DF07w6P\nPgrXX++q2P/kJ1b01xhjglkwK05yMmzd6lbNrGT797slYYYMcQtgfvQR/OMfVrHDGGOKY8GsOMnJ\nLhFk585K/di5c6FLFxe87r4bvv0Whg6t1CYYY0y1Y8GsOJW8FMz27XDVVW6xzlat4IsvXLaif0K2\nMcaY4lkwK0779u6xEoLZ88+73tj8+e7+2BdfuLqKxhhjwmOp+cWppJ7Z7Nlu8vPgwW45lo4dPf04\nY4ypkSyYFadxY1fY0MO5Zhs2wE03wYAB8OGHbiK0McaYsrNhxpJ4ONfs2DGXZl+vnuudWSAzxlRZ\nIqmIrEckA5HJId4fj8guRFb4tpsC3rsekY2+7Xqvmmg9s5K0bw/Z2Z5ceuJEWLXK3Sdr1670440x\nJipEYoFpwDDcupTLEZmH6pqgI2ejOjHo3ObA74A+gAJf+s7dF+lmWs+sJB71zF54AWbOdKs/p6ZG\n/PLGGBNJ/YAMVDNRzQNeB0aGee5w4ENU9/oC2IeAJz/1LJiVJDkZDh6EAwcidslVq+C22+Cii6pU\nDWNjjClOG2BrwOsc375gVyHyDSJvIuIfbwr33AqzYFaSCGc0Hjrk7pM1aQKvvgqxsRG5rDHGlFsS\n1EEkPWBLK8dl/gl0QPV8XO/rxci2snR2z6wkgXPNzj+/QpdSdSWqNm6ERYsqZwFPY4wpzW7IR7VP\nCYdsAwLv7Lf17Suguifg1XPAYwHnDgk6d0k5m1oi65mVJII9s2efhddfh4cfdjUXjTGmmlgOdEQk\nBZG6wDXAvEJHiLQOeDUCtyAzuMWZL0GkGSLNgEt8+yLO02AmQqoI60XIEKFIOqcIySIsEuEbEZaI\n0DbgvetF2OjbPEvnLFHLli53voJzzb78EiZNgksvhclFk1qNMabqUs0HJuKC0FpgDqqrEZmKyAjf\nUb9EZDUiK4FfAuN95+4FHsYFxOXAVN++iPNspWkRYoENBKZzwrWqrAk45g3gX6q8KMLFwA2q/EyE\n5kA6gemc0FuVYtM5I7rSdKBzz4WePd1ksHLYvx969XKrRH/9NSQmRrh9xhhTATVlpWkve2b9gAxV\nMlUpLp2zC/Bv3/PFAe8PBz5UZa8vgHmWzlmqCqxrpgo33OBWkpk92wKZMcZ4xctgFk5K5kpgtO/5\nKKCRCIlhnhsxO3a42ojffRfizQrMNXvySXjnHVf9fsCAirXRGGNM8aKdAHIXMFiEr4HBuMyXk+Ge\nLEKaCOkipOfnl78RDz8Mn34KU6eGeDM52UW548fLdM3PPoPf/hZGjYI77yx/24wxxpTOy2BWajqn\nKttVGa1KT+A+37794ZzrO3aGKn1U6VOnHJMM6tcHEZg+3S0oPX26e12/fsBB/ozGrVtDXiOU3bvh\n6qvdCOXMme6axhhjvONlMFsOdBQhRYSQ6ZwiJImcbsO9wEzf8wXAJSI0E8GzdM7MTBg7tmABzIQE\nuO46yMoKOKiM65qdOgU/+xns2gVvvOEK7xtjjPGWZ8FMlSLpnKqsFmGqCP50ziHAehE2AK2AR3zn\nFknn9O2LqNat3UovubkQH+8eGzcOmtBcxrlmv/89fPAB/OUvLovRGGOM9zxLza9s5U3NHz3aBbW0\nNJgxwyWDvP12wAF5eS7SPfBAqcUUFy+GH/8YrrkGXn7ZhheNMVVfTUnNr/XBLCxt28KwYfD888Ue\n8t130KMHNGsGy5dDw4beNMUYYyKppgQzq80YjlLmmuXnw7XXugL7H31kgcwYYypbtFPzq4dS5po9\n+CAsWeKyIbt2rbRWGWOM8bFgFo7kZJeaf+pUkbf27XNJHz//OVwfnQqSxhhT61kwC0dyMpw4EbJE\nyKZNLsaNGhWFdhljjAEsmIWnhLlm2dnusUOHSmuNMcaYIBbMwlHCXDP/BGsLZsYYEz0WzMLh75mF\nWNcsO9tV+bBKH8YYEz0WzMLRuLGLVsX0zFJSotAmY4wxp1kwC1cx6fnZ2TbEaIwx0WbBLFwhgpmq\nC2bWMzPGmOiyYBau5OQi98y+/x6OHbOemTHGRJsFs3C1b+/qVe3ff3qXpeUbY0zVYMEsXCHS8/1p\n+TbMaIwx0WXBLFwhgpn1zIwxpmqwYBYufzALuG+WlQVJSVYl3xhTw4mkIrIekQxEJpdw3FWIKCJ9\nfK87IHIMkRW+7RmvmuhpMBMhVYT1ImSIUOQLEKG9CItF+FqEb0S4zLe/rgjPi/CtCCtFGOJlO8PS\nogXUq1ekZ2a9MmNMjSYSC0wDLgW6ANci0iXEcY2AScCyoHc2odrDt03wqpmeBTMRinwBIgR/AVOA\nOar0BK4BnvbtvxlAlW7AMOBPIlHuRcbEFFnXzCZMG2NqgX5ABqqZqOYBrwMjQxz3MPAHILcyG+fn\nZYDoB2SokqlKcV+AAo19z5sA233PuwD/BlDle2A/0MfDtoYnYK7ZqVPuqfXMjDE1XBtga8DrHN++\nAiK9gHaovhfi/BREvkbkY0QGedVIL4NZ6V8APAiMEyEHmA/c4du/EhghQh0RUoDeQDsP2xqegLlm\n330HeXnWMzPGVG9JUAeR9IAtrUwXEIkBngB+E+LdHUB7VHsCvwZeRaRxiOMqrI4XFy2Da4EXVPmT\nCAOAWSJ0BWYCnYF0YDPwGXAy+GQR0oA0gLp1K6G17du7KJabS1ZWPGA9M2NM9bYb8lEtaeRrG4U7\nE219+/waAV2BJYgAnAHMQ2QEqunAcQBUv0RkE3Au7md7RHkZzEr7AgBuBFIBVPlchHggyTe0+Cv/\nQSJ8BmwI/gBVZgAzABo0QCPa+lD8GY1bt5Kd3RGwYGaMqfGWAx0RScH9DL8GGHv6XdUDQNLp1yJL\ngLtQTUekBbAX1ZOInAV0BDK9aKSXw4zLgY4ipIhQF/cFzAs6ZgswFECEzkA8sEuEBBEa+PYPA/JV\nWeNhW8MTMNfM1jEzxtQKqvnARGABsBaYg+pqRKYiMqKUsy8EvkFkBfAmMAHVvV4007OemSr5Iqe/\ngFhgpiqrRZgKpKsyDzfG+ncRfoVLBhmviorQElggwincbwI/86qdZRIw1yw7G1q1gvr1o9oiY4zx\nnup8XF5D4L4Hijl2SMDzt4C3vGtYAU/vmalS5AtQ5YGA52uAgSHOywY6edm2cmnTBkRO98ws+cMY\nYyJMZBLwPHAIeA7oCUxGdWFJp4U1zCjCKBGaBLxuKsKVFWhu9VS3Lpx5JmzebBOmjTHGG79A9SBw\nCdAMNzKDpqlkAAAgAElEQVT3aGknhXvP7HeqHPC/UGU/8LvytLLaS05GszezZYv1zIwxxgPie7wM\nmIXq6oB9xQo3mIU6Ltpp/dGRnEx+1hby861nZowxHvgSkYW4YLbAVybrVGknhRvM0kV4QoSzfdsT\nwJcVaGz11b49sdu3IpyyYGaMMZF3IzAZ6IvqUSAOuKG0k8INZncAecBsXFmqXOD28rWzmktOJib/\nBK3ZYcOMxhgTeQOA9ajuR2QcrobvgVLOCW+oUJUjULTqfa3kS8/vwGbatw+uzmWMMaaCpgPdEemO\nm771HPASMLikk8LNZvxQhKYBr5uJsKACja2+fMGse7Mt1KsX5bYYY0zNk4+q4grTP4XqNFzJrBKF\nm8SR5MtgBECVfb6JzbVP+/YAdGu8uZQDjTHGlMMhRO7FpeQP8hUyjivtpHDvmZ0Sob3/hQgdoBJq\nIVZFjRqxP6YZHetZMDPGGA9cjStO/AtUv8PV9f1jaSeF2zO7D/hUhI9x+f6D8FWrr21OnICsU8m0\nO2XBzBhjIk71O0ReAfoicgXwBaovlXZaWD0zVT7ALY65HngNd1PuWAWaW23l5MBmkmlxbEu0m2KM\nMTWPyBjgC+CnwBhgGSI/Ke20sHpmItwETMJ191YAFwCfAxeXt73VVVYWbKE9jff9G1Txrd9jjDEm\nMu7DzTH7HsC3jMxHuKr7xQr3ntkkoC+wWZWLcIUf95d8Ss2Une16ZnWOHoL9tfIrMMYYL8WcDmTO\nHsKIVeHeM8tVJVcERKinyjqRKljVvhJkZcFWSXbpL1u2QLNm0W6SMcbUJB8gsgB3SwtcQsj8Eo4H\nwg9mOb55Zu8AH4qwD6iVGRDZ2ZDbsj3sBDZvhu7do90kY4ypOVTvRuQqCpYHm4Hq3NJOC7cCyCjf\n0wdFWAw0AT4oV0OruawsaHFWckEwM8YYE1nlWNQz3HtmAZ/Bx6rMUyWvtGNFSBVhvQgZIkXLYYnQ\nXoTFInwtwjciXObbHyfCiyJ8K8JaEe4tazu9kp0NTTq2hPh4C2bGGBMpIocQORhic/tL4dkyLiLE\nAtOAYUAOsFyEeb7Vpf2mAHNUmS5CF9y4aAdcSmY9VbqJkACsEeE13wrUUXP8OGzfDilniasEssXS\n840xJiJUSy1ZVZIy98zKoB+QoUqmrxf3Oq7WViAFGvueNwG2B+xvIEIdoD6uYn+pkdlrW7a4bPwO\nHXDBzHpmxhhTJXgZzNoAWwNe5/j2BXoQGCdCDq5Xdodv/5vAEWAHsAV4XJW9HrY1LNnZ7rFDB1zB\nYQtmxhhTJXgZzMJxLfCCKm3xLZEtQgyuV3cSOBNIAX4jwlnBJ4uQJkK6COn5+d43NivLPaak4ILZ\nzp2Qm+v9BxtjjCmRl8FsG9Au4HVb375ANwJzAFT5HIgHkoCxwAeqnFDle+C/uHJahagyQ5U+qvSp\n49ndvwLZ2VCnDrRpw+mlYNi6taRTjDGm+hNJRWQ9IhmIFL+2pchViCgifQL23es7bz0iw71qopfB\nbDnQUYQUEeoC1wDzgo7ZAgwFEKEzLpjt8u2/2Le/Aa581joP2xqWrCx3qyw2ltNLwdhQozGmRhPx\nJ/NdCnQBrkWkS4jjGuGqRS0L2NcF97P/PCAVeNp3vYjzLJipkg9MBBYAa3FZi6tFmCrCCN9hvwFu\nFmElbrb3eFUU98U1FGE1Lig+r8o3XrU1XNnZvvtlUNAzs2BmjKnZ+gEZqGaiWlwyH8DDwB+AwHsv\nI4HXUT2OahaQ4btexHk6OKfKfILKkKjyQMDzNRTM8g485jAuPb9Kyc6Gyy/3vWjbFmJiLJgZY2q6\nUMl8/QsdIdILaIfqe4jcHXTu0qBzgxMBI6IS7jTVDMeOwXffBfTM4uLgzDNtrpkxplpLgjqIpAfs\nmoHqjLAv4FaCfgIYH+GmlYkFszD5O2ApKQE7ba6ZMaaa2w35qBZJsAtQWjJfI6ArsMS3JNYZwDxE\nRoRxbsREOzW/2vCn5Z/umYHNNTPG1AbLgY6IpCBSNJlP9QCqSah2QLUDblhxBKrpvuOuQaQeIilA\nR9zCmxFnwSxMhSZM+yUnu9T8kyej0CJjjKkEqkWS+VBdjchUX++rpHNX46ZfrcEVp78dVU9+YNow\nY5iysqBuXWjdOmBncjLk57ubaW08uadpjDHRp1okmQ/VB4o5dkjQ60eAR7xpWAHrmYUpO9vFrpjA\nb8zmmhljTJVgwSxM2dlByR9gc82MMaaKsGAWpqysoPtlYMHMGGOqCAtmYTh8GHbvDtEza9gQmje3\nuWbGGBNlFswAduyAwYNdIkcIITMZ/WyumTHGRJ0FM4CHH4ZPP4WpU0O+XWIws7lmxhgTdbU7mNWv\nDyIwfTqcOuUeRdz+AIXWMQvmD2aq3rfXGGNMSLU7mGVmwtixkJDgXickwHXXFUQvn+xsF99atgxx\njeRkd1Nt/37Pm2uMMSa02h3MWreGxo3datHx8e6xcWM444xCh/kzGV3ZsSA218wYY6KudgczgJ07\nYcIEWLrUPYZIAim0jlkwS883xpios3JWb79d8HzatJCHZGfDgAHFnO+/kfb++zAy1Hp1xhhjvOZp\nz0yEVBHWi5AhwuQQ77cXYbEIX4vwjQiX+fZfJ8KKgO2UCD28bGtxDhyAfftK6JklJcGkSfDss/Di\ni5XZNGOMMT6eBTMRYoFpwKVAF+BaEboEHTYFmKNKT9yyAk8DqPKKKj1U6QH8DMhSZYVXbS2JPy0/\nZCaj3+OPw8UXwy23wLJlldEsY4wxAbzsmfUDMlTJVCUPeB0IHodToLHveRNge4jrXOs7NypCrmMW\nrE4dmDPHrTw9ahRsD/XHMMYY4xUvg1kbYGvA6xzfvkAPAuNEyMEtL3BHiOtcDbzmRQPDUeKE6UCJ\nifDuu3DwIIwe7TIjjTHGVIpoZzNeC7ygSlvgMmCWSEGbROgPHFVlVaiTRUgTIV2E9Px8bxqYleVK\nMCYmhnFwt27w0ktuqPHWW20itTHGVBIvg9k2oF3A67a+fYFuxK1CiiqfA/FAUsD711BCr0yVGar0\nUaVPHY/yMv1p+SHnmIUyejQ88AC88AL87W/eNMoYY0whXgaz5UBHEVJEqIsLTPOCjtkCDAUQoTMu\nmO3yvY4BxhDF+2VQzDpmpfnd71ya/q9/DYsWedEsY4wxATwLZqrkAxOBBcBaXNbiahGmijDCd9hv\ngJtFWInrgY1XxT82dyGwVZVMr9pYGtVi1jErTUwMzJoFP/gBjBnjymYZY4zxjGgNua/ToEEDPXLk\nSESvuXevu1f2xBPwq1+V4wKbNkHfvtCmDXz+ubv5ZowxVYiIHFXVBtFuR0VFOwGkSgsrLb8kZ58N\ns2fDmjVw/fWuMr8xxpiIs2BWgrDT8ksybJibVP322/DIIxFolTHGmGAWzEpQ4jpmZXHnnfDzn7ss\nx3ffrXC7jDGmUomkIrIekQxEipQmRGQCIt8isgKRTxHp4tvfAZFjvv0rEHnGsybaPbPiTZwIL78c\noaXKcnPhwgth7VpXof+88yJwUWOMqZhS75mJxAIbgGG44hfLgWtRXRNwTGNUD/qejwBuQzUVkQ7A\nv1Dt6lX7/axnVoJypeUXJz4e5s51SSAjR7rsEmOMqfr6ARmoZqIaujShP5A5DYBK7yVZMCtBudLy\nS9Kmjbt3tnUrXHMNeFW2xBhjIiec0oQgcjsim4DHgF8GvJOCyNeIfIzIIK8aacGsGKoR7pn5DRgA\nzzwDH34I99wT4YsbY0zZJEEdRNIDtrRyXUh1GqpnA/fgVkQB2AG0R7Un8GvgVUQaF3eJirDFOYux\naxccPRrhnpnfDTfAihVuAlv37i45xBhjomA35KPap4RDwilNGOh1YDoAqseB477nX/p6bucC6RVo\nckjWMytGWOuYVYR/DbS0NPjiC48+xBhjKmw50BGRFERClyYU6Rjw6nJgo29/C18CCYicBXQEb6o6\nWTArRoUnTJcmLq7wGmhbtnj0QcYYUwGqRUoToroakam+zEWAiYisRmQFbjjxet/+C4FvfPvfBCag\n6kn2m6XmF+MPf4DJk93yZI0aReyyRX37Lfzwh66e42OPwc03u+fGGFMJrJxVDZed7eoyehrIwK2B\n9vXX0Ls3TJgAF10E69d7/KHGGFOzWDArRpnS8nfsgMGD4bvvyvdh55zjlor5xz/gm29cUsgjj8CJ\nE+W7njHG1DIWzIpRprT8hx+GTz+FqVPL/4Ei8ItfuAohI0bAlCmut2bJIcYYUyoLZiGcOlWwwnSJ\n6td3QWj6dHfS9Onudf365f/wM85wiSHvvAN79rh5ab/+NUS4VJcxxtQkFsxC2LkTjh8Po2eWmQlj\nx0JCgnudkADXXVeQClkRI0e6pWNuuQX+/Gfo2hUWLKj4dY0xpgbyNJiJkCrCehEyRChSaVmE9iIs\nFuFrEb4R4bKA984X4XMRVovwrQjxXrY1UNhp+a1bQ+PGrohwfLx7bNzY9a4ioUkTePpp+OQTqFcP\nUlPdBOvduyNzfWOMqSE8C2YixALTgEuBLsC1InQJOmwKMEeVnriJeE/7zq0DvAxMUOU8YAhQadkQ\nZVrHbOdOl4W4dKl7LG8SSEkGDXIVQ6ZMgddeg86d4dVXXc0tY4wxnvbM+gEZqmSqErrSsqus7K/T\n1QTY7nt+CfCNKisBVNmjykkP21pImSZMv/02TJvmMhCnTXOvvRAf7xJNvvoKzjrLDWdefjls3uzN\n5xljTDXiZTALp9Lyg8A4EXKA+cAdvv3nAirCAhG+EuG3HraziOxsaNmy4FZYldKtG3z2GTz5JHz8\nsVsX7a9/hZOVFuuNMabKiXYCyLXAC6q0BS4DZokQgyuA/CPgOt/jKBGGBp8sQpoI6SKkR3I1FU+q\n5UdSbCxMmgSrV8OPfuSen302/N//eTPMaYwxVZyXwSycSss3AnMAVPkciAeScL24T1TZrcpRXK+t\nV/AHqDJDlT6q9KkTwfr/EV/HzCsdOsD777tFP88+G+67D9q1g6uvhsWL7Z6aMabW8DKYLQc6ipAi\nQuhKy7AFXI9LhM64YLYLV9CymwgJvmSQwcAaKsHJk67mb5XumQUSgSuvdBVE1q2DO+5wa6VdfDF0\n6QJ/+Qvs2xftVhpjjKc8C2aqFKm0rMpqEaaK4K+0/BvgZhFWAq8B41VRVfYBT+AC4grgK1Xe86qt\ngbZvd1WkqkXPLFinTm6NtG3b4IUXXGr/nXe6Fa5/8QtXTcR6a8aYGsiq5gf5z3/gwgvhgw9g+PAI\nNCwcO3bANdfA7NmRm6Pm9/XXbmXrV15xVUR69YJbb4Vrr4UG1b5QtjGmgmpK1fwaHcxOnDhBTk4O\nubm5YV/n8GFXRerMM92SY5Wh1dSpNJs9m31XX83OBx7w5DNiDh+myT//SdPXXyd+40ZONmzIgREj\n2Hf11eR17FjsefHx8bRt25a4yvoyjDGVyoJZFRMqmGVlZdGoUSMSExMRkbCus32723r1qoRlxerX\nd1VDgsXHw7Fj3nymqkvtnz4d3ngD8vLcpOxx41yB44CeoaqyZ88eDh06REq1uYlojCmLmhLMop2a\n76nc3NwyBTJwP9vj4ippfUwvazsWRwQGDoSXX3b31h57zA1z3nKLK881YAA8+iisW4eIkJiYWKae\nrTHGREONDmZAmQIZuALD9ep51JhgXtd2LE1SEtx9N2zY4NZRe/hhl/1y772uZFanTsg991D/669t\nUrYxpkqr8cGsrI4fh7p1K/EDK6O2Y2lEXGWRKVMgPR22bnWluZKT4c9/psN117mbiDfdBP/6l3dD\noMYYU041+p7Z2rVr6dy5c9jXUIUvv3QdpjbBhbfKaf/+/bz66qvcdtttZTrvsssu49VXX6Vp06aR\naUh5HTjAtueeo83y5TB/Phw65IZDhw9389suvxwSE6PbRmNMudk9sxooL889RrJntn//fp5++uki\n+/NLqb81f/788APZjh0weLA3vbomTTh42WXw+utu6ZkFC+D6692cteuvh1atYMgQt8r2woWwf3/k\n22CMMaWoNT2zO+90q6iU5ORJOHrUdTxiY0v/zB49XL3fklxzzTW8++67dOrUibi4OOLj42nWrBnr\n1q1jw4YNXHnllWzdupXc3FwmTZpEWloaAB06dCA9PZ3Dhw9z6aWX8qMf/YjPPvuMNm3a8O6771I/\ncDXr226DZ5+FW27h7z17MmPGDPLy8jjnnHOYNWsWCQkJ7Ny5kwkTJpCZmQnA9OnT+eEPf8hLL73E\n448/johw/vnnM2vWrCJ/hpA93FOnXDf23Xfhn/+Eb78tmJDdqRP07w8XXOAeu3WrvHkOxpgyqSk9\nMwtmAU6ccDkYDRqEl80YTjDLzs7miiuuYNWqVSxZsoTLL7+cVatWnU5137t3L82bN+fYsWP07duX\njz/+mMTExELB7JxzziE9PZ0ePXowZswYRowYwbhx40pN7Z8yZQqtWrXijjvu4Oqrr2bAgAHceeed\nnDx5ksOHD5OTk8OoUaP47LPPSEpKOt2WYGEN1x444O63LV0Ky5a57fvv3Xv167u5Dv7g1r+/qyFZ\nxuQcY0zkhRXMRFKBvwCxwHOoPhr0/gTgduAkcBhIQ3WN7717cXV4TwK/RHVBhP8IgKtOXyuUFnTA\nZarv2OHtHLN+/foVmrP117/+lblz5wKwdetWNm7cSGLQPaiUlBR69OgBQO/evcn2rx6amQl33QXv\nvHO6S7nzhz8k7eBBMrt14/Dhwwz3lTH597//zUsvvQRAbGwsTZo04aWXXuKnP/0pSUlJACEDWdia\nNIGhQ90Grpe2eXPh4PbUU/CnP7n3zzijcHDr3dtlchpjqhYR/0LLw3BF4JcjMu90sHJeRfUZ3/Ej\ncOUIUxHpgqvLex5wJvARIueiGvH06FoTzMKRl+ful3k5x6xBQAmpJUuW8NFHH/H555+TkJDAkCFD\nQs7pqhcwVyA2NpZj/mzCEKn9C5cuZeqnn9K9e3deeOEFlixZUnqjvCinJeIKXHbo4K4N7gteubIg\nuC1b5gKxX8eO7jcJ/9azpyWXGBN9/YAMVN09ChH/QssFwUz1YMDxDXALL+M77nVUjwNZiGT4rvd5\npBtpCSABvEjLb9SoEYcOHQr53oEDB2jWrBkJCQmsW7eOpUuXlv0DglL7mx0/TuvWrTlx4gSvvPLK\n6cOGDh3K9OnTATh58iQHDhzg4osv5o033iD3vvvg00/J/X//r1x/xrDVrQt9+8LEiTBrlpvftnu3\ny5L83/9199aWLYN77oFhw9w8uORkGDXKzYF77z0XeI0xlSmchZZB5HZENgGPAb8s07kRYD2zAHl5\n0KhRZK+ZmJjIwIED6dq1K/Xr16dVq1an30tNTeWZZ56hc+fOdOrUiQsuuKDsH/D22wXPp01ja9eu\n3NG/Py1atKB///6nA+lf/vIX0tLS+Mc//kFsbCzTp09nwMUXsyE3FzZuBCD++efh+ee9LacVLDER\nLr3UbX5797oCyV99VbAF9uDOOKNwD65XL2jf3u7BGVMOSVAHkfSAXTNQnVHmC6lOA6YhMhaYAlwf\noSaGpdYkgJTm1Cn3M/PMM91WK+zYUeSeG6NGweOPFxpuLPV79LLqv9/Bg26IMjDArVnj/uLAZe10\n7Ajnnus2//OOHW2o0pgSlJoAIjIAeBDV4b7X9wKg+vtijo8B9qHapMixIgt814r4MKP1zHy8mGNW\n5UWqnNbDD8Onn7q5ZiHm1EVE48auIPKgQQX7jh51UwK++sotTLpxo5su8NZbhctvNW9eNMD5Hxs2\n9Ka9xtQcy4GOiKQA23AJHWMLHSHSEdWNvleXA/7n84BXEXkClwDSEfjCi0ZaMPPxB7NKq8tYQbff\nfjv//e9/C+2bNGkSN9xwQ9ku5L/nlpYGM2aU7Z5U8NSA6dPdVtZhyvL27BISCrIhA+XluWLNGza4\nAOd/XLzY3asL1Lq1C2wpKW6oMjnZPbZv76YPBM7nM6Y2Us1HxL/QciwwE9XViEwF0lGdB0xE5MfA\nCWAf/iFGd9wcXLJIPnC7F5mM4PEwowiF5iao8mjQ++2BF4GmvmMmqzJfhA641anX+w5dqsqEkj6r\nosOMu3a5TPJu3apPQKssxX6PYQ5Tlipg0ne5enZlCYZHj0JGRtFAl53t1v4J/v/QsmVBcAsMdP7X\nSUl2r85UazVl0rRnPTMRisxNEGGeKoFzE6YAc1SZLkIXYD7QwffeJlV6eNW+YLVymLGiKjpMGame\nXVmGORMS4Pzz3RYsL89NNtyypWDbvNk9rl3rlh8/erTwOfHxBYEt1NamDdSxARBjvObl/7J+QIYq\nmQAiFJ2b4OYi+GfKNgG2e9ieEvmXfrFfssuoIsOUISZ9n+7ZhSNSwdCvbl033FjcQqSqLtMyOND5\nH//5T/d9BIqNdQGtuGDXvr0NZRoTAV4Gs1DzC4JubvAgsFCEO3AT7X4c8F6KCF8DB4EpqvzHw7ZW\n/tIvNUXQ1IAyqWjPrqLB0C/cYUoRlxmZmOgmdIdy7FhBgAvePvnE9fyC14Zr2dIFvFat3NayZcHz\nwH0tWoRXNNSYWija4x/XAi+o8icRBgCzROgK7ADaq7JHhN7AOyKcp0rgLHNESAPSoOKBKC/PVWSK\ntPIuAQPw5JNPkpaWRoJ/JeqaqCI9u6qSjRkcDDt1clso+fkuoAUHuh073HexerV79I97BxJx9+hC\nBb2WLd17/q1FC/cPulKWTDcm+jxLAPEFpwdVGe57fS+AKr8POGY1kKrqenAiZAIXqPJ90LWWAHep\nEjixr5CKJIAEzjETieyUqcBCw2XlLzbsr50YLWVdF65SjR7tglpgMAzsLZaklELNYYt0AouqK9y8\nc6cr1rxzZ+EteF/Qv/vTYmNdL9If3IKDXeBrf4+zYUMba69lLAGkdMuBjiIUPzcBtgBDgRdE6AzE\nA7tEaAHsVeWkCGfh5iZketXQwLT8+++P7JSpyZMns2nTJnr06MGwYcNo2bIlc+bM4fjx44waNYqH\nHnqII0eOMGbMGHJycjh58iT3338/O3fuZPv27Vx00UUkJSWxePHikNe/9dZbWb58OceOHeMnP/kJ\nDz30EADLly9n0qRJHDlyhHr16rFo0SISEhK45557+OCDD4iJieHmm2/mjjvuqPgfMpoqMsxZVe7Z\nBfcMRaBpU7cV18MLtGkTjB0Lvr97du92265dBc9373ZJLLt2wZ49BZPNg8XFFQS25s0Lnhe3z//a\nxuhNtKmqZxvoZaAbQDeB3ufbNxV0hO95F9D/gq4EXQF6iW//VaCrffu+Av2f0j4rISFBg61Zs6bI\nvlD271etW1fV/UpceIuPD+sSxcrKytLzzjtPVVUXLFigN998s546dUpPnjypl19+uX788cf65ptv\n6k033RTQnv2qqpqcnKy7du0q8fp79uxRVdX8/HwdPHiwrly5Uo8fP64pKSn6xRdfqKrqgQMH9MSJ\nE/r000/rVVddpSdOnCh0bmnC/R6rpQkTVGNi3F90TIzqrbeGf+727apjx6omJLh/LAkJqtddp7pj\nR3jnx8dH5h/drbeWre0nT6ru2aO6fr3qp5+qpqaqiqhecIHqPfeo3nST6ujRqoMHq3btqtq6dfH/\nQfxbTIxqmzaqPXqoXnyx6k9+onrLLar33qv6xz+q/uMfqnPnqn78seq336pu26Z67FjB93jhheF/\nbyaigCPqYRyorM3Te2aqzMel2wfueyDg+RpgYIjz3gLe8rJtgY4fd2tMvvgizJtXsVyCkixcuJCF\nCxfS05c8cPjwYTZu3MigQYP4zW9+wz333MMVV1zBoMAqF6WYM2cOM2bMID8/nx07drBmzRpEhNat\nW9O3b18AGvuWVvnoo4+YMGECdXyp4hVa8qWmiOY9u2j1DGNiXI+qTZvC5y9d6rZQ56u6Nu7ZU3jb\nu9fV81y+3A1Rtmvn9q1eXfB+SauqJyS4nuiRI67GZp8+Bb1S/9asWdF9TZu677oqJMRURjk3U6po\nJ4BUCXl5BffLK5pLUBJV5d577+WWW24p8t5XX33F/PnzmTJlCkOHDuWBBx4IcYXCsrKyePzxx1m+\nfDnNmjVj/PjxIZeQMSWoyDAlVO9gWJbzRVz9ywYN3HQCKBpM1693W2AwVIXDh11QC94mTiw8b2/H\nDje9QcT9ZzxwoOgk9mCNG7vq4Pv2QZcu7v5fo0YF+wOfBz/6nx8+DDfcUP5gFOkEIlMuFswoSMv/\n/vvy/1wqTuASMMOHD+f+++/nuuuuo2HDhmzbto24uDjy8/Np3rw548aNo2nTpjz33HOFzi0uAeTg\nwYM0aNCAJk2asHPnTt5//32GDBlCp06d2LFjB8uXL6dv374cOnSI+vXrM2zYMJ599lkuuugi6tSp\nU+zK0qYMqnMwrIxgKlIQOJKTC58/YkTJFWROnYJDh2D//sLbvn2FX7//fsHKuiKumsvBg+7cYpZf\nCqlDB/e5gUEvVCD0P//5z93y9H6VMenfFMuCGa5nVq9exX8uhRK4BMyll17K2LFjGTBgAAANGzbk\n5ZdfJiMjg7vvvpuYmBji4uJOrzuWlpZGamoqZ555ZsgEkO7du9OzZ09+8IMf0K5dOwYOdCO2devW\nZfbs2dxxxx0cO3aM+vXr89FHH3HTTTexYcMGzj//fOLi4rj55puZOHFiZP6gpnyiGQwrer7XwTQm\nxvXQmjQpGgihaM9w2za3BQaTU6fcEKY/uAUGuTFjCgej48fdNImYGBgyxA2TZmUVnHP4cHh/rtxc\n9wOlQQMXoIt7nD278JxDfzCMi3P3Oxo1ckO3/l8GGjZ0f7bgbFPr2QG2BAwAK1a4IfgOHTxqXDVX\npVPzTXRVZGpERc+vaG3Qsp5/8mRBYPQHuIcfdovL1qnj7g1ecAGkprrrHTlS8uPBg66XWdI9xWCx\nsUUD3LZtrq7orbeWq2dnqfk1xMmT7t+SFRc2phwq2rOMZgWZsp4fG1swzOhXt64LIoHBOIz73afd\nev4FrNgAAAf0SURBVKs7r25dN0R07bVupXV/TzDwMXjfiy8WnmJR0XJu1VytD2anTrnErgZV/PeS\n/v37c/z48UL7Zs2aRbdu3aLUImOqgGgOs4I3w8Th/p9+5JHIlHOrIWyY0ZTKvkdjqqjgnl05qtDU\nlGFGK9xmjDHVlb9nt3Spe/zuu2i3KGpq/DCjqiJWa67cakrP3ZgayYsU7GqqRvfM4uPj2bNnj/1A\nLidVZc+ePcTHx0e7KcYYU6Ia3TNr27YtOTk57Nq1K9pNqbbi4+Np27ZttJthjDElqtEJIMYYY0pm\nCSDGGGNMFWHBzBhjTLVnwcwYY0y1V2PumYnIKaAiNVzqAGUoklbprH0VY+2rGGtfxVTl9tVX1Wrf\nsakxwayiRCRdVftEux3FsfZVjLWvYqx9FVPV21cTVPtobIwxxlgwM8YYU+1ZMCswI9oNKIW1r2Ks\nfRVj7auYqt6+as/umRljjKn2rGdmjDGm2qtVwUxEUkVkvYhkiMjkEO/XE5HZvveXiUiHSmxbOxFZ\nLCJrRGS1iEwKccwQETkgIit8WxmWtI1YO7NF5Fvf56eHeF9E5K++7/AbEelViW3rFPDdrBCRgyJy\nZ9AxlfodishMEfleRFYF7GsuIh+KyEbfY7Nizr3ed8xGEbm+Etv3RxFZ5/v7mysiTYs5t8R/Cx62\n70ER2Rbwd3hZMeeW+P/dw/bNDmhbtoisKOZcz7+/WkVVa8UGxAKbgLOAusBKoEvQMbcBz/ieXwPM\nrsT2tQZ6+Z43AjaEaN8Q4F9R/h6zgaQS3r8MeB8Q4AJgWRT/vr8DkqP5HQIXAr2AVQH7HgMm+55P\nBv4Q4rzmQKbvsZnvebNKat8lQB3f8z+Eal84/xY8bN+DwF1h/P2X+P/dq/YFvf8n4IFofX+1aatN\nPbN+QIaqZqpqHvA6MDLomJHAi77nbwJDpZIWQ1PVHar6le/5IWAt0KYyPjvCRgIvqbMUaCoiraPQ\njqHAJlXdHIXPPk1VPwH2Bu0O/Hf2InBliFOHAx+q6l5V3Qd8CKRWRvtUdaGq+if4LgWitmxCMd9f\nOML5/15hJbXP97NjDPBapD/XFFWbglkbYGvA6xyKBovTx/j+Mx8AEiuldQF8w5s9gWUh3h4gIitF\n5H0ROa9SG+YosFBEvhSRtBDvh/M9V4ZrKP6HSLS/w1aqusP3/DugVYhjqsr3+AtcTzuU0v4teGmi\nbxh0ZjHDtFXh+xsE7FTVjcW8H83vr8apTcGsWhCRhsBbwJ2qejDo7a9ww2bdgb8B71R2+4AfqWov\n4FLgdhG5MAptKJGI1AVGAG+EeLsqfIenqRtvqpIpxSJyH64E0yvFHBKtfwvTgbOBHsAO3FBeVXQt\nJffKqvz/peqkNgWzbUC7gNdtfftCHiMidYAmwJ5KaZ37zDhcIHtFVd8Ofl9VD6rqYd/z+fz/9u7m\nRY4ijOP496cRXxKJCSq+HNSoBxUkYAiS6EkJKiIqK0piotFLQA+eFFER8gfoKWBQwag5iGJwEUHI\nCgs5hI0EjRpFF08R2b1IIIoim8dDPY3tZEfal+lOz/4+MMxsTU13dW01z0x18xScJenCttqX+/0h\nn+eBfZTpnLom/TxqdwKHI2Ju8I3ToQ+BuWrqNZ/nF6nTaT9KehS4G9iSAfcUDcbCSETEXEQsRMRJ\n4NUh++26/5YB9wPvDKvTVf+Nq6UUzA4B10q6Kr+5PwRMDtSZBKq7xiaAT4adyP+3nF9/Hfg6Il4a\nUueS6hqepPWU/1+bwXa5pPOr15QbBb4cqDYJbMu7Gm8Gjtem1Noy9Btx132Y6uPsEeCDRep8DGyS\ntCqn0TZl2chJugN4GrgnIn4ZUqfJWBhV++rXYO8bst8m5/so3Q58ExHHFnuzy/4bW13fgdLmg3Kn\n3beUu5yey7KdlJMW4BzK1NQsMAOsabFtt1Cmm44An+XjLmAHsCPrPAl8Rbkz6yCwoeX+W5P7/jzb\nUfVhvY0CdmUffwGsa7mNyynBaWWtrLM+pATVH4HfKddtHqdch50CvgP2A6uz7jrgtdpnH8uxOAts\nb7F9s5TrTdU4rO7wvQz46O/GQkvteyvH1hFKgLp0sH359ynnexvty/I3qjFXq9t6/y2lhzOAmJlZ\n7y2laUYzMxtTDmZmZtZ7DmZmZtZ7DmZmZtZ7DmZmZtZ7DmZmp4HM5v9h1+0w6ysHMzMz6z0HM7N/\nQNLDkmZyDardks6UdELSyyrr0E1JuijrrpV0sLYu2Kosv0bS/kx2fFjS1bn5FZLey7XE9ra1YoPZ\nOHAwM2tI0nXAg8DGiFgLLABbKFlHPo2IG4Bp4MX8yJvAMxFxIyVjRVW+F9gVJdnxBkoGCSgrJTwF\nXE/JELFx5AdlNiaWdd0Asx65DbgJOJQ/ms6lJAk+yZ8JZd8G3pe0ErggIqazfA/wbubjuzwi9gFE\nxK8Aub2ZyFx+uTrxlcCB0R+WWf85mJk1J2BPRDz7l0LphYF6/zZH3G+11wv4/DRrzNOMZs1NAROS\nLgaQtFrSFZTzaCLrbAYORMRx4CdJt2b5VmA6yirixyTdm9s4W9J5rR6F2RjyNz+zhiLiqKTnKasD\nn0HJlP4E8DOwPt+bp1xXg7K8yysZrL4Htmf5VmC3pJ25jQdaPAyzseSs+Wb/kaQTEbGi63aYLWWe\nZjQzs97zLzMzM+s9/zIzM7PeczAzM7PeczAzM7PeczAzM7PeczAzM7PeczAzM7Pe+wOpIB2KU7/4\n0wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fa76d27e9d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_curves(history,nb_epoch)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The curve trend indicates that the model may be able to improve if we train it for longer, but for now let's leave it here.\n",
    "\n",
    "Let's now evaluate our model. ```model.evaluate()``` will take all the test samples, forward them through the network and return the average loss, and any additional metrics we specified (in our case, the accuracy)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loss: 0.271295\n",
      "Accuracy: 0.923100\n"
     ]
    }
   ],
   "source": [
    "score = model.evaluate(X_test, Y_test, verbose=0)\n",
    "print (\"Loss: %f\"%(score[0]))\n",
    "print (\"Accuracy: %f\"%(score[1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "We reach an accuracy of 92%, which is similar to the one we obtained in the previous exercise."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Single hidden layer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "Let's try to train a model with a hidden layer between the input and the classifier. \n",
    "\n",
    "**Exercise**: Modify the previous architecture to include this layer with 128 neurons. Make sure to use a non-linearity between the two layers (e.g. ReLU). Also remember that, in keras, the ```input_shape``` must be passed to the first layer of the network. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "____________________________________________________________________________________________________\n",
      "Layer (type)                     Output Shape          Param #     Connected to                     \n",
      "====================================================================================================\n",
      "dense_2 (Dense)                  (None, 128)           100480      dense_input_2[0][0]              \n",
      "____________________________________________________________________________________________________\n",
      "activation_2 (Activation)        (None, 128)           0           dense_2[0][0]                    \n",
      "____________________________________________________________________________________________________\n",
      "dense_3 (Dense)                  (None, 10)            1290        activation_2[0][0]               \n",
      "____________________________________________________________________________________________________\n",
      "activation_3 (Activation)        (None, 10)            0           dense_3[0][0]                    \n",
      "====================================================================================================\n",
      "Total params: 101,770\n",
      "Trainable params: 101,770\n",
      "Non-trainable params: 0\n",
      "____________________________________________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "np.random.seed(SEED)\n",
    "\n",
    "# MODEL DEFINITION\n",
    "model = Sequential()\n",
    "model.add(Dense(128,input_shape=(784,)))\n",
    "model.add(Activation('relu'))\n",
    "model.add(Dense(10))\n",
    "model.add(Activation('softmax'))\n",
    "# TODO: add layers to this network as indicated in the exercise\n",
    "\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "**Exercise**: Compute the number of parameters and check if they match the ones given by ```model.summary()```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "101770"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "784*128+128+128*10+10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Let's train the model you just defined. Notice that the compilation and training code did not change at all:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 60000 samples, validate on 10000 samples\n",
      "Epoch 1/20\n",
      "1s - loss: 0.4914 - acc: 0.8648 - val_loss: 0.2769 - val_acc: 0.9223\n",
      "Epoch 2/20\n",
      "1s - loss: 0.2607 - acc: 0.9264 - val_loss: 0.2209 - val_acc: 0.9395\n",
      "Epoch 3/20\n",
      "1s - loss: 0.2096 - acc: 0.9417 - val_loss: 0.1839 - val_acc: 0.9471\n",
      "Epoch 4/20\n",
      "1s - loss: 0.1771 - acc: 0.9500 - val_loss: 0.1625 - val_acc: 0.9551\n",
      "Epoch 5/20\n",
      "1s - loss: 0.1540 - acc: 0.9564 - val_loss: 0.1436 - val_acc: 0.9599\n",
      "Epoch 6/20\n",
      "1s - loss: 0.1362 - acc: 0.9619 - val_loss: 0.1329 - val_acc: 0.9615\n",
      "Epoch 7/20\n",
      "1s - loss: 0.1226 - acc: 0.9660 - val_loss: 0.1244 - val_acc: 0.9645\n",
      "Epoch 8/20\n",
      "1s - loss: 0.1111 - acc: 0.9696 - val_loss: 0.1167 - val_acc: 0.9652\n",
      "Epoch 9/20\n",
      "1s - loss: 0.1020 - acc: 0.9719 - val_loss: 0.1104 - val_acc: 0.9669\n",
      "Epoch 10/20\n",
      "1s - loss: 0.0945 - acc: 0.9740 - val_loss: 0.1044 - val_acc: 0.9697\n",
      "Epoch 11/20\n",
      "1s - loss: 0.0879 - acc: 0.9762 - val_loss: 0.0989 - val_acc: 0.9699\n",
      "Epoch 12/20\n",
      "1s - loss: 0.0819 - acc: 0.9776 - val_loss: 0.0932 - val_acc: 0.9721\n",
      "Epoch 13/20\n",
      "1s - loss: 0.0765 - acc: 0.9794 - val_loss: 0.0930 - val_acc: 0.9722\n",
      "Epoch 14/20\n",
      "1s - loss: 0.0717 - acc: 0.9807 - val_loss: 0.0919 - val_acc: 0.9725\n",
      "Epoch 15/20\n",
      "1s - loss: 0.0678 - acc: 0.9817 - val_loss: 0.0866 - val_acc: 0.9737\n",
      "Epoch 16/20\n",
      "1s - loss: 0.0640 - acc: 0.9825 - val_loss: 0.0856 - val_acc: 0.9739\n",
      "Epoch 17/20\n",
      "1s - loss: 0.0606 - acc: 0.9838 - val_loss: 0.0831 - val_acc: 0.9751\n",
      "Epoch 18/20\n",
      "1s - loss: 0.0570 - acc: 0.9845 - val_loss: 0.0808 - val_acc: 0.9750\n",
      "Epoch 19/20\n",
      "1s - loss: 0.0545 - acc: 0.9854 - val_loss: 0.0816 - val_acc: 0.9741\n",
      "Epoch 20/20\n",
      "1s - loss: 0.0517 - acc: 0.9862 - val_loss: 0.0791 - val_acc: 0.9757\n",
      "(29.94141387939453, 'seconds.')\n",
      "----------\n",
      "Loss: 0.079135\n",
      "Accuracy: 0.975700\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAa0AAAEKCAYAAAChTwphAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4lNX1wPHvIQkkAWQH2RRECJsCsklRQfmpgFYFl7qD\nVaOiltYNqUsVbeuCrVIRRYsKal1ww4KCgLi0giwFZU9AlAAiqyxhScj5/XFnyCSZJDPJvJmZ5Hye\n531m5p33nbmZLCf3veeeK6qKMcYYEw+qRbsBxhhjTKgsaBljjIkbFrSMMcbEDQtaxhhj4oYFLWOM\nMXHDgpYxxpi4YUHLGGNM3LCgZYwxJm5Y0DLGGBM3EqPdgEipVq2apqSkRLsZxhgTV7Kzs1VV46YD\nU2mCVkpKCvv37492M4wxJq6IyIFotyEccRNdjTHGGAtaxhhjSiYyEJE1iGQicm+Q54cjsg2Rpb7t\nBq+aUmkuDxpjjPGASAIwHjgbyAIWIjIN1ZWFjnwL1du8bk6lDlo5OTlkZWVx8ODBaDcl7iUnJ9Oi\nRQuSkpKi3RRjTMXqBWSiuh4AkTeBC4HCQatCVOqglZWVRe3atWnVqhUiEu3mxC1VZceOHWRlZdG6\ndetoN8cYE0ENIRGRRQG7JqI6MeBxc2BjwOMsoHeQl7oYkTOAtcAfUN0Y5Jhyq9RB6+DBgxawIkBE\naNCgAdu2bYt2U4wxEbYdclHtUc6X+Qj4F6qHELkJeBU4q/ytK6rSJ2JYwIoM+xyNqbI2AS0DHrfw\n7cunugPVQ75HLwHdvWpMpe5phSQ3F37+GerUgZo1o90aY4wJShV274bt22HbtoJb/fpw002evfVC\noC0irXHB6nLgygJHiDRFdYvv0QXAKq8aY0FLBDZvdrcWtIwxFcQfhLZscf83Fw5E27YVDFDbt7v/\nsYPp08fDoKWai8htwEwgAZiE6gpExgCLUJ0G/A6RC4BcYCcw3KPWWNAiIQGSksCjDMPdu3fzxhtv\nMGLEiLDOGzx4MG+88QZ169YN67zhw4dz/vnnc8kll4R1njEmMvLyYMcOF4z82+bNBR/7t+L+7NSt\nC40aue2EE6BXr/zHgVvDhu42NdXjL0p1BjCj0L4HA+6PBkZ73ArAgpaTnOxp0HruueeKBK3c3FwS\nE4v/+GfMmFHsc8aYinH4MOza5XpEJd1u25YfnLZuhZycoq9Vpw40beq2Pn3y7zdrBk2a5AeiBg3c\n/9EmuKoTtH7/e1i6NPhzBw+6fnetWuG9Zteu8PTTJR5y7733sm7dOrp27UpSUhLJycnUq1eP1atX\ns3btWi666CI2btzIwYMHGTlyJOnp6QC0atWKRYsWsW/fPgYNGsRpp53Gf//7X5o3b86HH35IKMWB\n58yZw1133UVubi49e/ZkwoQJ1KhRg3vvvZdp06aRmJjIOeecw9ixY3nnnXd4+OGHSUhIoE6dOnzx\nxRfhfRbGxIncXPjhB1i3Ln/76afgAelAKVX5kpNdr6hhQxeAOnTID0aFN897Q1VE1QlaJalWzV1g\nVnVjWxH02GOPsXz5cpYuXcq8efM477zzWL58+dH5TpMmTaJ+/focOHCAnj17cvHFF9OgQYMCr5GR\nkcG//vUvXnzxRS677DLeffddrr766hLf9+DBgwwfPpw5c+bQrl07rr32WiZMmMA111zD+++/z+rV\nqxERdu/eDcCYMWOYOXMmzZs3P7rPmHi1fz+sX18wMPm3H34oODaUnOyCSr16bmva1AWievWC3wbe\nT06O3tdYVVWdoFVSj+iXXyAjA9LSoHZtT5vRq1evAhN0x40bx/vvvw/Axo0bycjIKBK0WrduTdeu\nXQHo3r07GzZsKPV91qxZQ+vWrWnXrh0Aw4YNY/z48dx2220kJydz/fXXc/7553P++ecD0LdvX4YP\nH85ll13G0KFDI/GlGhNxqi4gFU5U+PHHgoFpy5aC59WrB23aQI8e8JvfuPsnnuhumzZ1/7ea+FB1\nglZJ/P8uHTzoedCqGZChOG/ePGbPns3XX39Namoq/fv3D1pyqkaNGkfvJyQkcKC0axYlSExM5Jtv\nvmHOnDlMnTqVZ599lrlz5/L888+zYMECpk+fTvfu3Vm8eHGR4GlMpPkz6LZuLZotV9zj4oafW7Rw\nQWjQIHcbuNWrV7Ffl/GOp0FLhIHAM7g0yZdUeazQ88cDk4BGuDTJq1XJ8j33BHAebgL0p8BIVdST\nhlav7i4LepCMUbt2bfbu3Rv0uV9++YV69eqRmprK6tWrmT9/fsTeNy0tjQ0bNpCZmcmJJ57IlClT\n6NevH/v27SM7O5vBgwfTt29fTjjhBADWrVtH79696d27Nx9//DEbN260oGXK5fBh1+PZtKnotnlz\n/v3i/gerXTs/O65ZM+jSpWjWnH9r2hRsDdiqwbOgJUKRysAiTFMtUGRxLDBZlVdFOAv4K3CNCL8C\n+gIn+477CugHzPOqsV5lEDZo0IC+ffvSuXNnUlJSaNKkydHnBg4cyPPPP0+HDh1IS0vj1FNPjdj7\nJicn8/LLL3PppZceTcS4+eab2blzJxdeeCEHDx5EVfnb3/4GwN13301GRgaqyoABA+jSpUvE2mIq\nn5wcyMqCDRvctnFj0aD0889Fz6tRwwWg5s2he3e44AJ3PzB7zh+UbLzIBCOq3nReROgDPKTKub7H\nowFU+WvAMSuAgapsFEGAX1Q5xnfus8BpgABfANeoFj/LumbNmlp45eJVq1bRoUOH0Bq8bh1kZ8NJ\nJ4XxVVYtYX2eJq7l5hYMSv7t++/dbVaWm48UqGFDF4CCbf5A1aBBxHOdTDmJSLaqxk1lBS8vD4ZS\nGXgZMBR3CXEIUFuEBqp8LcJnwBZc0Ho2WMASIR1IB3eFr1ySk12ea16ejcqaKiEnB9asgWXLXB5S\nYHDKyoIjR/KPFXFBp1UrOOMMdxu4tWjhelHGeC3aiRh3Ac+KMBzXm9oEHBHhRKADrjAjwKcinK7K\nl4EnqzIRmAhQs2Y5x7v81yIOHYqLi+O33nor//nPfwrsGzlyJNddd12UWmRi2a5dLjgtW+amKy5b\nBitWuHEncEGpWTMXgE47DVq3LhiUWraMwD+GxkSAl0Gr1MrAqmzG9bQQoRZwsSq7RbgRmK/KPt9z\nHwN9oGDQiqjADMI4CFrjx4+PdhNMDMrLc1e6/QHKH6Q2BlzzaNzYJTWMHOluu3SBtm2tp2Tig5dB\nayHQVoRiKwOL0BDYqUoerm7VJN9TPwI3ivBX3OXBfkDJpSfKKzBoGRPjjhxxgSgjw23ffecC1Lff\nunlM4MpqpqW5nlPXrvkB6thjo9t2Y8rDs6ClSq4IBSoDq7JChDHAIlWmAf2Bv4qguMuDt/pOn4pb\nQOw7QIFPVPnIq7YCnhfONSZcqi4Lzx+Y1q7Nv79unbuS7VenjgtIv/2tu+3aFTp1sgy8qmDLFrj8\ncnjrrarxD4mnY1qqFKkMrMqDAfen4gJU4fOOAN6tDlMcDwvnGlOcXbtg5cqiwSkzM7/XBO7yXZs2\n7lLeeee5W//WrJll5ZVVvP/Rf+QR+OorGDMGnnsu2q3xXrQTMWJLcjLs3OlJDUJjwKWSL18O8+e7\n7euvXZDyS0x0SRBt20L//tCuXX5gatnSXRAwkVXeP/rRCnopKQX/x54wwW3JyaUX+o1nltsdKDnZ\n1ZPp18+VfY4A/9IkZfH000+TnZ1d4jGtWrVi+/btZXp9472ff4Zp02D0aDjzTFdktVs3uOUWmDED\n2reHv/wFpk93wSs7291On+7KZY4YAWef7TL4LGAFt2VL2X5lU1Lc/6YTJrgElgkT3ONw87ACg15Z\nlLX969fDlVfmV49PTYWrrnJz6So1Va0UW2pqqha2cuXKIvtKtHu36sUXq1arpnrLLeGdW4zvv/9e\nO3XqVKZzjz/+eN22bVu5j4mUsD/PKubQIdVvvlEdN071yitVW7f2Lx2gmpio2rOn6u23q77+uuq6\ndap5edFucWzYvFn1jDNUt2wp2/m33FK2X9nNm933KTXVfY9SU1Wvuir0diQn539/A7fk5Ippv6rq\nzTe7c5OTy/4awH6Ngb/hoW5Rb0CktnIHrUj9BBbym9/8RpOTk7VLly5611136RNPPKE9evTQk046\nSR988EFVVd23b58OHjxYTz75ZO3UqZO++eab+swzz2hSUpJ27txZ+/fvX+zrBwatp556Sjt16qSd\nOnXSv//978W+tqrqqFGjtEOHDnrSSSfpnXfeGdLXYkErX16e6oYNqm+/rXrnnap9+xb8EWrWzP3/\nM3as6ldfqWZnR7vFsausf7Qj8Stbnj/6sRD0hgxRHTFCdelSdztkSOjn+lnQitegtXmz6hVX5P8k\nhfsTWIzAntbMmTP1xhtv1Ly8PD1y5Iied955+vnnn+vUqVP1hhtuOHrO7t27VTW8ntaiRYu0c+fO\num/fPt27d6927NhRlyxZEvS1t2/fru3atdM837/6u3btCulrqcpBa9cu1U8/VX30UdVf/1q1SZP8\nPzI1aqj26aN6xx0uiG3cWLFtK29PJVrnl/ePdnmDhmr5/+hHM+hFSrwFLRvT8mva1OUNHzrk0rQO\nHoRjjonoyOqsWbOYNWsW3bp145RTTmH16tVkZGRw0kkn8emnnzJq1Ci+/PJL6tSpE/Zrf/XVVwwZ\nMoSaNWtSq1Ythg4dypdffhn0tevUqXN0Ta333nuPVFtStYDDh2HxYjcoP2yYW422Xj03tnT//S6z\n79xzYfx4WLgQ9uyB//4XnnoKLr3UlTSqSOUdU4nW+eUdk2na1P2KHjyYn/gb7q/se++572OXLu72\nvffC+xq2boWbb3ZJNTffHN64VCTaXxVZ9mCgrVvdb9FFF8FnnxVdSa6cVJXRo0dz001Fs/mXLFnC\njBkzuP/++xkwYAAPPvhgkFcIX7t27YK+drA1taoiVVdrb/58+OYbWLAAlizJnwPVuDH07g1XX+1u\ne/RwyRSxoLzZY9E+PxJ/tP1BIz0dJk6M+K9sqQKDXFmK1ES7/XEp2l29SG0RScRQVc3KUl24UPXI\nkfDPDWL79u163HHHqaq7PNirVy/du3ev762ydOvWrbpp0yY9cOCAqqp+9NFHeuGFF6qqaufOnXX9\n+vUlvr7/8uDixYv1pJNO0v379+u+ffu0U6dOumTJkqCvvXfvXt26dauqusuF9evXD+lrqQyXB3Ny\n3Lf36adVL7lEtWnT/MtSKSmqp5/uxqjeftuNWcVyskR5Ly9F+3zVyIzJmPIhzi4PWk+rsAgXzg1c\nT2vQoEFceeWV9OnTB4BatWrx2muvkZmZyd133021atVISkpiwoQJAKSnpzNw4ECaNWvGZ599VuL7\nnHLKKQwfPpxevXoBcMMNN9CtWzdmzpxZ5LX37t0bdE2tymjPHjcX6j//cdv8+S6tHFwa+VlnQd++\n0KcPdO7s5kmFq7zzdMp6fnl7KtE+H8rfUzFVj2fraVW0cq+n5Zed7coTnHAC1K8fwRbGv3hYT+vH\nH11w+uord/vdd/mrzXTt6urw9e3rtubNI/OeI0bACy/ATTeVbXJqec4fOtQFj8DLS+GMy0T7fBN9\n8baelgWtwo4cgf/9z9XFadYsgi2Mf7EYtHbuhKlT3RDkV1+5daAAatVyvae+fV2g6t3b7YukwmM6\nfmUdEwr3fGMiId6Cll0eLCwhwS0cFGM1CHv37s2hwAqpwJQpUzipCq60nJMDn3wCr74KH33ksv1a\ntMgPUH37ugWoy3KpLxzr18Ndd8EHH7gOemoqDBkCY8dWzPnGVEWVPmipKhJuHcEYLJy7YMGCqL5/\ntHvkqq4DPHkyvPEGbNsGjRq5S2vDhrmU5YouFxkLY0LGVDWVep5WcnIyO3bsCP8Prv8vSCW5dFpe\nqsqOHTtIjsI6F5s3w5NPwsknQ/fuLqW6f3/Xw9q0Cf7+dzdWVZ6AVdbab1C+eTqRON+YqqZSj2nl\n5OSQlZXFwXB7TXv3usGSFi2sSqlPcnIyLVq0ICkpyfP3ys52l8wmT4ZPP3WJFH36uB7VZZe5ib6R\nVN5ECmPiWbyNaVXqoFVms2e78gdz57rS3MZzeXkukeLVV+Gdd9z/DccdB9de67a2bYs/t6wp45YI\nYUz8Ba1KfXmwzNLS3O3q1dFtRxWwaRM8/DCceKK7RPf223DJJS4b8PvvXYmgkgIWRK+MkDGm4lX6\nRIwyad4cataENWui3ZJKSRXmzXOTST/4wM0yqFsXnn0Whg93H30oYqGMkDGmYllPK5hq1dySsdbT\niqg9e1xg6tTJVaL47DO44w7Xu9mzB1asCD1gQWR6SpYIYUx8sTGt4lx5pav/Y9eKyu2771yCw5Qp\nsH8/9OwJt97qgkR5x5RuucVVYqhe3c3XsmQKY8JjY1qVRVoa/PCDjciX0eHDLjHijDNcuvorr7hl\nO775xm3DhllPyRgTPhvTKk779m7wJSPD/dU1xQrM3svNdT2fF190AeSEE9w8q+uugwYNCp5nBVeN\nMeGynlZxLIMwZGPGwJdfwq9+5SqnP/qomwg8Y4aL+XfdVTRg+VlPyZg4IDIQkTWIZCJybwnHXYyI\nItLDs6Z4OaYlwkDgGSABeEmVxwo9fzwwCWgE7ASuViXL99xxwEtAS0CBwapsKO69Ij6mlZ3tsgLG\njIEHHojc61Yixc1z8i/8bIyJfaWOaYkkAGuBs4EsYCFwBaorCx1XG5gOVAduQ3WRF+31rKclQgIw\nHhgEdASuEKFjocPGApNVORkYA/w14LnJwJOqdAB6AT971dagUlPh+OOrRE+rLGWMvv3WJVRAfgml\nlBQ3JrVhQ8SbaIyJnl5AJqrrUT0MvAlcGOS4R4DHAU//ZfXy8mAvIFOV9aoU94V2BPzrvH/mf94X\n3BJV+RRAlX2qZHvY1uDS0qpE0Apncu6WLXDDDa7e34oV7pKgiBuTOnTI5jkZE28aQiIiiwK29EKH\nNAc2BjzO8u3LJ3IK0BLV6d621tugVfoXCsuAob77Q4DaIjQA2gG7RXhPhP+J8KSv51ax2rd3E4wr\nybSAwlJSXMCZMMGVUZowwT0OtmBzdnZ+dYrJk+EPf4DMTGjSxMakjIln2yEX1R4B28SwXkCkGvA3\n4E5PGlhItBMx7gL6ifA/oB+wCTiCy2o83fd8T+AEYHjhk0VIF2GRCItycz1oXVqam1i0aZMHLx59\noaSc5+W5INWuHTz4IAwcCKtWwVNPucK1773nsva6dHG3tmqtMZXOJlxugV8L3z6/2kBnYB4iG4BT\ngWleJWN4GbRK+0JRZbMqQ1XpBtzn27cb1ytb6ru0mAt8AJxS+A1UmahKD1V6eLLgX/v27raSlnMq\nLeV83jw3bjVsmFvE+Ysv3CrBbdpEtdnGmIq1EGiLSGtEqgOXA9OOPqv6C6oNUW2FaitgPnBB3CVi\n4PtCRWgtQtEvFBChocjRNozGZRL6z60rQiPf47OAgpkqFaEKpL0HSzlfuxYuusgVuN+2DV57zT1/\n+unRbq0xpsKp5gK3ATOBVcDbqK5AZAwiF1R0c7xOeR8MPI1LeZ+kyp9FGAMsUmWaCJfgMgYV+AK4\nVZVDvnPPBp4CBFgMpPsSOoKKeMo7uLGsY45xM2PHjYvsa8egHTtcMsZzz7me1x//CL//ffAxLmNM\n5RBvZZys9mBpevSA+vVh1qzIv3aMOHTIjUc98ogrXHvjjW65kCZNot0yY4zX4i1oRTsRI/b5Mwhj\nWHmWi587Fzp3hjvvhN69YdkyeP55C1jGmNhkQas0aWnw448uizBGlWURxN27XY9qwAD3+OOP4ZNP\nXAAzxphYZUGrNP4MwoyM6LYjiHDmWQX64APo2BEmTYJ77nHVLQYOrJg2G2NMeVjQKk0MZxCGu7TH\n1q1w2WUwZAg0agQLFsDjj1uihTEmfljQKk3btq77EoPjWqEu7aEKr74KHTrAhx+6KuyLFrkcE2OM\niScWtEqTkhLThXNLW9pjwwZ36W/4cHdJcNkyuO8+SEqKRmuNMaZ8LOU9FIMGueiwZIk3r++BI0fg\n2WddgBKBxx5zS9NXs39TjDEBLOW9MvKnveflRbslIVm5Ek47zU0MPuMMV4391lstYBlj4p/9GQtF\nWporc56VFe2WlOjwYZf23rWrS3acMgWmT4fjjot2y4wxJjK8KDNb+QQWzo3RCPDNN3D99bB8OVx+\nOTzzDDRuHO1WGWNMZFlPKxQxnPaekwN33w19+sCuXTBtGvzrXxawjDGVk/W0QnHssS6XPMbS3nft\ngksvhTlz4Kab3JyrOnWi3SpjjPGOBa1QiLjeVgz1tDIz4fzz3QTjV15xa14ZY0xlZ5cHQ+Vh4dxw\nC95+/rkrbrt9O8yebQHLGFN1WNAKVVqayx7cty/iLx1OwduXX4azz3ZjVgsWuJR2Y4ypKixohcqf\nQbh2bcReMpyCt3l5MGoU/Pa30L8/fP21LXtvjKl6LGiFyoMMwlAL3u7fDxdfDE884apaTJ8OdetG\nrBnGGBM3LBEjVCee6EpKRHBcK5SCt1lZcMEFrmbguHFw222uN2aMMVWR9bRClZwMrVpFPIOwpIK3\nixZBr14uU/Df/4bbb7eAZYyp2qxgbjjOOw82bYKlS719H2DqVLj2Wpdw8e9/24rCxhhvWMHcyiwt\nzSVieFg4VxX+8hc3abhrV1eeyQKWMcY4FrTC0b49HDgAGzd68vKHDrk5V/fd5xI05s61ckzGGBPI\nglY4/GnvHlTG2LYNBgxwldnHjIHXXnPDaMYYY/JZ0AqHP+09wpUxVq50FS4WL4a33oIHHrCEC2OM\nCcbToCXCQBHWiJApwr1Bnj9ehDkifCvCPBFaFHr+GBGyRHjWy3aGrHFjN0Eqgj2tjRuhb1+3XNfn\nn8Nll0XspY0xptLxLGiJkACMBwYBHYErROhY6LCxwGRVTgbGAH8t9PwjwBdetTFsxRTODbd2oJ8q\njBjhxrK+/NKltxtjjCmelz2tXkCmKutVOQy8CVxY6JiOwFzf/c8CnxehO9AEmOVhG8MXpHBuOLUD\nA739tktnf/RRaNs2gm00xphKysug1RwITLPL8u0LtAwY6rs/BKgtQgMRqgFPAXeV9AYipIuwSIRF\nubkRanVp0tJg82bYsyes2oGF7djhJgv36AG/+533zTbGmMog2okYdwH9RPgf0A/YBBwBRgAzVMkq\n6WRVJqrSQ5UeiRVVkCqgcG6otQODufNOt4jjSy9BhbXdGGPinJd/LjcBLQMet/DtO0qVzfh6WiLU\nAi5WZbcIfYDTRRgB1AKqi7BPtWgyR4ULKJzbtEePUmsHBvPpp/Dqq/DHP0KXLt432RhjKgsve1oL\ngbYitBahOnA5MC3wABEa+i4FAowGJgGocpUqx6nSCtcbmxwTAQvceiAJCUfHtUqqHRjM/v2Qng7t\n2rnUdmOMMaHzrKelSq4ItwEzgQRgkiorRBgDLFJlGtAf+KsIissSvNWr9kRMjRrQuvXRDML33st/\navz40k9/8EHYsMGlt9vkYWOMCY8VzC2LX/8afvgBvv02rNMWLoRTT4Ubb4Tnn/eobcYYEwYrmFsV\n+AvnHjkS8ik5OXDDDW686/HHPWybMcZUYha0yqJ9ezcj+McfQz7lySddx+y556BOHQ/bZowxlZgF\nrbIIyCAMxZo1buLxJZfAhYWnVxtjTKwTGYjIGkQyESmaFCdyMyLfIbIUka8QKVz9KGIsaJWFf65W\nCIVz8/LcGFZKCvzjHx63yxhjIk2kSEm+IEHpDVRPQrUr8ATwN6+aY0GrLBo2hPr1Q+ppvfiiqyv4\n1FOlz98yxpgY1AvIRHU9qsFL8qnuCXhUE/Asw89qMZSFv3BuKT2tTZvgnnvgrLPguusqqG3GGBOG\nhpCIyKKAXRNRnRjwOFhJvt5FXkjkVuAOoDpwlgdNBSxolV379vDxx8U+7a/gfvgwTJxo62MZY2LT\ndshFtUe5X0h1PDAekSuB+4Fh5X7NIOzyYFmlpbnyF7/8EvTpd9+FadNcAkabNhXcNmOMiZxSS/IV\n8iZwkVeNsaBVViUkY+zcCbfdBqecAn/4QwW3yxhjImsh0BaR1ogELcmHSODiSucBGV41xi4PllVg\n2nuh1Rvvvhu2b3dXD62CuzEmrqnmIlKgJB+qKxAZAyxCdRpwGyL/B+QAu/Do0iBY0Cq7Nm1cRCrU\n05ozByZNglGjoFu3KLXNGGMiSXUGMKPQvgcD7o+sqKbY5cGySkqCE04okPaene0quJ94IvzpT1Fs\nmzHGxDKRkYgcg4gg8k9EliByTiinhhS0RBgiQp2Ax3VFvBtoixvt2xfoaT30EKxf7+ZmhbJ6sTHG\nVFG/9c3tOgeoB1wDPBbKiaH2tP6kytE0OVV2A5WnL7FlC/TrV/piWIWlpUFGBhw5wuLFbgLxjTdC\n//6etNIYYyoL/ySgwcAUVFcE7CtRqEEr2HGVZzzskUfgq69cfno42reHw4fJydjADTdA48bwxBPe\nNNEYYyqRxYjMwgWtmYjUBvJCOTHUwLNIhL/h6k+BW6xxcdjNjDUpKXDwYP7jCRPclpwMBw6Ufr4v\ng/CDx1ezdGkb3n0X6tb1qK3GGFN5XA90Bdajmo1IfSCkukGh9rRuBw4Db+Emjh0kHlYZLs369XDl\nlZCa6h6npsJVV8H334d2vm+u1qLX1jBkCAwd6lE7jTGmcukDrEF1NyJX4ypoBK/UUEhIPS1V9gNF\ny9HHu6ZN4ZhjXG8rOdndHnNM6JVtGzQgp04D2vyymj7XettUY4ypRCYAXRDpAtwJvARMBvqVdmKo\n2YOfilA34HE9EWaWsbGxZetWuPlmmD/f3YaZjLGzcXvSWEOHDh61zxhjKp9cVBVXLf5ZX93C2qGc\nGOqYVkNfxiAAquwSoXH47YxB772Xf3/8+OKPK8aPKe1pz0c0sPqCxhgTqr2IjMalup+OSDUgKZQT\nQx3TyhPhOP8DEVrh4XopFa2sGe8AK3LTaMLPJO7dFfmGGWNM5fQb4BBuvtZPuCK8T4ZyYqhB6z7g\nKxGmiPAa8DkwuiwtjUVlzXgH+HpX6KsYG2OMAV+geh2og8j5wEFUJ4dyakhBS5VPgB7AGuBfuIGz\nEHLCY1uaxqAPAAAgAElEQVRKilvnasIEyMtztyKhV7M4dAg+/8lXOHfZMu8aaowxlYnIZcA3wKXA\nZcACRC4J5dRQEzFuAObggtVdwBTgoRDOGyjCGhEyRYpmH4pwvAhzRPhWhHkitPDt7yrC1yKs8D33\nm1DaGa7yZrxnZECGtmF3i07w8MOwY4cXzTTGmMrmPqAnqsNQvRboBTwQyomhXh4cCfQEflDlTKAb\n5CdmBCNCAm4y8iCgI3CFCB0LHTYWmKzKycAY4K++/dnAtap0AgYCTwdmL0ZKeTPeV62CPBL46Ykp\nbi2Sm25ySxYbY4wpSTVUfw54vIMQ41GoQeugKgcBRKihymogrZRzegGZqqxX5TBuUvKFhY7pCMz1\n3f/M/7wqa1XdImKqbAZ+BhqF2NawlCfjffVqdznxuAu7uQGxd9+FKVO8aKYxxlQmnyAyE5HhiAwH\nplN46ZNihJrynuXr6XwAfCrCLuCHUs5pDmwMfA2gd6FjlgFDgWeAIUBtERqocvQ6mwi9gOrAuhDb\nGpbyZLyvWgXHH++7vHj33TB9uluy+IwzoFWrSDbTGGMqD9W7EbkY6OvbMxHV90M5NdSKGEN8dx8S\n4TOgDvBJ2A0t6i7gWRGGA18Am4Aj/idFaIobPxumWrSYogjpQDpA9eoRaE2YVq06WskJEhJg8mTo\n0gWGDYO5c90+Y4wxRam+C7wb7mlhLwKpyueqTPNd8ivJJqBlwOMWvn2Br7VZlaGqdMMNzPmXPUGE\nY3BdxvtUmV9MWyaq0kOVHhW9rH1enstyL1AJo3VrGDcOvvgC/va3im2QMcbEOpG9iOwJsrn9IfBy\n5eKFQFsRWotQHbgcmBZ4gAgNRY62YTQwybe/OvA+LkljqodtLLMffnCF4IuUbxo2DIYMgfvuszR4\nY4wJpFob1WOCbG5/CDwLWqrkArcBM4FVwNuqrBBhjAgX+A7rD6wRYS3QBPizb/9lwBnAcBGW+rau\nXrW1LFatcrdFgpYITJwI9eu7/PnApU+MMcaUi2glSdGuWbOm7t+/v8Le76mn4K67XKZ7gwZBDpgx\nA847D+64wx1sjDExSESyVbVmtNsRKi8vD1Zqq1ZBo0bFBCyAwYPhllvc2NbcucUcZIwxJhzW0yqj\nvn0hMRE+/7yEg/bvh27d3ODXd9/ZssbGmJhjPa0qQNX1tEpdQ6tmTXjtNVdG/tb4X+jZGGOizYJW\nGWzbBrt2hRC0AHr1ggcfhDfegDff9LxtxhhTmVnQKoNiMweL88c/Qu/ebowrK8uzdhljTGVnQasM\n/EHraDWM0iQmupqEhw/D8OFuZrIxxpiwWdAqg1Wr3HBVS3+9j1CWPm7b1mUSzpkD//hHhbTTGGMq\nGwtaZeCvOSji2xHq0sfp6W7u1qhRsGKF5+00xpjKxlLey6BlS+jfH6ZMTQle8SI52aW5B/PTT3DS\nSdCiBSxYEJ1Kv8YY42Mp75Xc3r0ul6JDB8q29PGxx8JLL8HSpfCnP1VIm40xprKwoBWm1avdbYcO\nlH3p4wsvhOuvh8cfd5cVjTHGhMSCVpiKpLuXdenjv//dLWVyzTWwJ6SK/MYYU+XZmFaYRo+GsWMh\nOxuSksr5Yv/5j1vleNgwmDQpIu0zxphw2JhWJbd6tcteL3fAAlfA8N574eWXLWgZY0wILGiFKaSa\ng+H4059gwAA3xmVLmBhjTIksaIXh8GHIzIxw0KpeHaZPh0sucQt0jRrlKvIaY0ysEBmIyBpEMhG5\nN8jzdyCyEpFvEZmDyPFeNcWCVhgyM+HIkTDKN4WqRg1XTPeWW+CJJ1yvKzc3wm9ijDFlIJIAjAcG\nAR2BKxDpWOio/wE9UD0ZmAo84VVzEr164coo7EK54UhIgPHjoUkTeOghtyTym2/mzwEzxpjo6AVk\noroeAJE3gQuBlUePUP0s4Pj5wNVeNcZ6WmEIu1BuuETcGNdzz8G//w3nnOPWQDHGGI80hEREFgVs\n6YUOaQ5sDHic5dtXnOuBjyPdTj/raYVh1So47jhXLNdTt9wCDRu66hpnnAGffALNS/oZMcaYstkO\nuaj2iMiLiVwN9AD6ReT1grCeVhginjlYkksvhY8/hg0bXGr8mjUV9MbGGFPAJqBlwOMWvn0Fifwf\ncB9wAaqHvGqMBa0Q5eW5OVoVFrTApcLPm+dmMp92GixcWIFvbowxACwE2iLSGpHqwOXAtAJHiHQD\nXsAFrJ+9bIwFrRD9+KMr3F6hQQuge3dXOaNWLTjzTPj00wpugDGmSlPNBW4DZgKrgLdRXYHIGEQu\n8B31JFALeAeRpYhMK+bVys3KOIXok09g0CD44gs4/fQIv/iWLXD55fDWW8UX2928GQYOdN29yZPd\n8cYYU05WximACANFWCNCpghFJqSJcLwIc0T4VoR5IrQIeG6YCBm+bZiX7QyFp+nuoSwi2ayZi5in\nnuqWQ7HVj40xVZBnPS0REoC1wNm4FMmFwBWq+bn9IrwD/FuVV0U4C7hOlWtEqA8swmWhKLAY6K5K\nsfnfXve00tPh/fdh27YIvmhKGRaRPHAArrgCPvwQ7r/fBbqjSygbY0x4rKeVrxeQqcp6VQ4D/glp\ngToCc333Pwt4/lzgU1V2+gLVp8BAD9taqlWrPJifVZZFJFNSYOpU+O1v4dFH3XIoR45EuGHGGBOb\nvAxaoUxIWwYM9d0fAtQWoUGI51YoT9Ldy7qIZGKiW/149GiYONGlxwfrsRljTCUT7ezBu4B+IvwP\nNxltExByt0GEdBEWibDIy1J927bBjh0ejWeVdRFJEfjLX9xiku+/D6ecAl9/7UEDjTEmdnhZEaPU\nCWmqbMbX0xKhFnCxKrtF2AT0L3TuvMJvoMpEYCJAzZp4lgbpaRLGe+/l3x8/Pvzzf/97d90yPd1N\nQv7d7+DPf66Ash3GGFPxvOxpLQTaitBahKAT0kRoKHK0DaMB/0qIM4FzRKgnQj3gHN++qPA0aEXC\nwIGwfLkr//TMM9C5M8yeHe1WGWNMxHkWtFQpMiFNlRUijBHBPyGtP7BGhLVAE+DPvnN3Ao/gAt9C\nYIxvX1SsWuVyJFq2LP3YqDnmGNdT+/xzt6zy2We7JU527452y4wxJmJscnEIzj3XrRSyeLEnL19+\nhScnHzgADz8MY8dC48YwYQJcWDhx0xhjLOW9UqrQQrllUXhyckoKPPYYLFjggtZFF8FvfuOSPowx\nJo5Z0CrFvn2wcWOMBq2UFJdFOGGCq+g7YYJ7nJLinu/e3RXZffRR+OAD6NgRXnsNKknv2hhT9VjQ\nKoV/RZCYDFqhTE5OSoL77oP//Q/atYNrroHzznMVgI0xJs5Y0CpFTGcOhjM5uWNHdwnx6addskan\nTvk9NGOMiRMWtEqxahUkJECbNtFuSTHCmZyckAAjR7r0+FNPhREj3HInGRkV115jjCkHyx4sxdCh\nsHKlWxGkUlGFl1+GO+6AQ4fc7R13QIMG0W6ZMaYCWfZgJRPzmYNlJeKK7q5c6bIL//pXaNXKjX/t\n2BHt1hljTFAWtEqQkwOZmZU0aPk1awb/+hd8+y0MHpwfvP74Rzc5zRhjYogFrRJkZkJubiUPWuAm\nJ996qysB9d13Lrvwscdc8Bo92oKXMSZmWNAqQUxnDkZS4OTkTp3gzTddssavfw2PP+6C1733RngF\nTGOMCZ8FrRL4g1bEF3+MFSVNTu7Y0V02XL4cLrgAnngCWreGUaMseBljosaCVglWr3ZFcmvVinZL\nPBLK5OSOHeGNN2DFCle/8MknXc/LgpcxJgosaJWg0mYO+oUzOblDB3j9dRe8LrrIFeNt1QruuceN\ng/XrF/oClsYYU0YWtIqRl+d6WpU6aEH4KycHBq8hQ+Cpp6BbN/jiC5fMUUnm/RljYpNNLi7Gjz/C\n8ce7YZ6bb47Yy1YuKSmud1ZYYiJkZUGTJhXfJmNMWGxycSVRZTIHy6PwmFj16tCwoZsn0Ly5GwP7\n8EM34c0YYyLAglYxLGiFoPCYWG4uXHqp+/DuvBO++caNf7VoAXfd5S4pBrNli42JGWNCYkGrGKtW\nQf360KhRtFsS44KNibVv7+Z3bdwIH30Ep53mJi537gy9e8Pzz8Pu3fmvUXgRS2OMKYaNaRXjjDNc\nMsZXX0XsJau2bdvcApQvv+yyDZOT4fDh4EujJCfDgQMV30ZjqiAb06okKn26e0Vr1Aj+8AdYtsyt\npvzb3+ZPgBNxtzVquDGywHlixhgTwIJWENu3u82ClgdEoEcPGD/eXVr8v//LT5M/dAimTXPVN776\nKvQFKm1MzJgqw4JWEP61syxoeSw5GWrXdotRfv45nHWW632NHw+nn+4yEEeMgDlzXJJHcWxMzJgq\no1KPaeXk5JCVlcXBYHOJSrB3L+zc6f5mJiZGspXxJzk5mRYtWpCUlFRxb7pnD0yfDu+9BzNmQHa2\nW5zywgvh4othwAB3KbG4eWI2JmZMyOJtTMvToCXCQOAZIAF4SZXHCj1/HPAqUNd3zL2qzBAhCXgJ\nOAVIBCar8teS3itY0Pr++++pXbs2DRo0QPzjJiHYuNHlDXTrlj/cUhWpKjt27GDv3r20bt06Oo3I\nzoaZM+Hdd10m4p49Ls3+/PNdz2zWLPj3v91xqamuSsfYscFLURVnyxa4/HJ4663wzjOmEoi3oOXZ\n5UEREoDxwCCgI3CFCB0LHXY/8LYq3YDLged8+y8FaqhyEtAduEmEVuG24eDBg2EHLHD/pCcnV+2A\nBSAiNGjQIOyeakT5A9Frr8HPP7se2CWXuEB2ww0umGVnQ1KS+8YVVzuxJHZ50Zi44eWYVi8gU5X1\nqhwG3gQuLHSMAsf47tcBNgfsrylCIpACHAb2lKUR4QYsyJ8ra8r2+XmmRg23uvI//+mSLmbPhuOO\nc4EtJ8cldLz6qgtmr78OmzeX/HolLc0SDksEMabCeBm0mgMbAx5n+fYFegi4WoQsYAZwu2//VGA/\nsAX4ERiryk4P23rUkSNu+pAFrRiXmOjGttavh3373Lpf48bBOee43tfVV7tByQ4dXDLH1KlFV2AO\nZWmWUFhPzZgKE+3swSuAV1RpAQwGpohQDddLOwI0A1oDd4pwQuGTRUgXYZEIi0pKLguH/0pYpILW\n7t27ee6550o/sJDBgwezO7BqhCmeiFtx+fbb4f33XXBavNit/dW6NUyZ4spLNWoEXbu6+WIffeSC\nVKhLswQTqZ6aMbFOZCAiaxDJROTeIM+fgcgSRHIRucTLpngZtDYBLQMet/DtC3Q98DaAKl8DyUBD\n4ErgE1VyVPkZ+A/Qo/AbqDJRlR6q9IhUlp8/aEXq705xQSu3lCg7Y8YM6tatG5lGVDUJCXDKKa7e\n4YwZLhX0v/+FRx91BX2ff96txly/Prz5JnTp4uaGXXtteJf4ItVTs8uLJpaJFMlPQKRwfsKPwHDg\nDa+b42VC90KgrQitccHqclwwCvQjMAB4RYQOuKC1zbf/LFzPqyZwKvB0eRrz+9/D0qWlH3fokLs8\nWLt26cd27QpPl9Kqe++9l3Xr1tG1a1eSkpJITk6mXr16rF69mrVr13LRRRexceNGDh48yMiRI0lP\nTwegVatWLFq0iH379jFo0CBOO+00/vvf/9K8eXM+/PBDUoqJqi+++CITJ07k8OHDnHjiiUyZMoXU\n1FS2bt3KzTffzPr16wGYMGECv/rVr5g8eTJjx45FRDj55JOZMmVK6V94vElKgj593Hbffe4/k/nz\n4bPPYO5cd/93v3PHHnec65X16gU9e0L37sX/MISziGZJAi8vlqFXbozHegGZqLo/HiL+/ISVR49Q\n3eB7LsSKAGXnWU9LlVzgNmAmsAqXJbhChDEiXOA77E7gRhGWAf8ChquiuKheS4QVuOD3sirfetXW\nQHl5UC2Cn8pjjz1GmzZtWLp0KU8++SRLlizhmWeeYe3atQBMmjSJxYsXs2jRIsaNG8eOHTuKvEZG\nRga33norK1asoG7durz77rvFvt/QoUNZuHAhy5Yto0OHDvzzn/8E4He/+x39+vVj2bJlLFmyhE6d\nOrFixQoeffRR5s6dy7Jly3jmmWci94XHsuRk6N8fHn4YvvzSFe/98ku3oGWfPrBkiVuR+cwzoU4d\nV+j3uuvc5b/Fi91/NX7hLqIZKFYSQaynV6U1hEREFgVs6YUOCSU/ocJ4OnVWlRm4BIvAfQ8G3F8J\n9A1y3j5c2nvElNYj8lu+3P1NO/HESL57vl69ehWY8zRu3Djef/99ADZu3EhGRgYNGjQocE7r1q3p\n2rUrAN27d2fDhg3Fvv7y5cu5//772b17N/v27ePcc88FYO7cuUyePBmAhIQE6tSpw+TJk7n00ktp\n2LAhAPXr14/Y1xlXatZ0lehPOy1/3/btrkbiN9+42+nT4ZVX3HM1arhudq9eMHSo65G1besqeYRj\n/Xp3CfODD4rOMwtHeXtq5T3f5rnFte2Qi2qR4ZdYVcXrPRSk6i4PejmUVLNm/hy+efPmMXv2bL7+\n+mtSU1Pp379/0DlRNWrUOHo/ISGBAyVUexg+fDgffPABXbp04ZVXXmHevHkRbX+V0bAhDBrkNnA/\nHD/8kB/EvvnGpd7/4x/u+Vq14OSTXTDr2tWNk3XunD/eFUx5Ly8WrggyYYLbQq0IUt7z/ezyZmUX\nSn5ChYl29mBMOXTI/W2KZLp77dq12bt3b9DnfvnlF+rVq0dqaiqrV69m/vz55X6/vXv30rRpU3Jy\ncnj99deP7h8wYAATJkwA4MiRI/zyyy+cddZZvPPOO0cvSe7cWSGzCuKTCLRqBZdd5rISP/8cfvkF\nvv3WBa/hw9115SlTID3drRtWu7ZLub/iCnjsMfjkE9crCaxCU57Li+VNBCnv+bFyedN4bSHQFpHW\niFTH5SdMi1ZjrKcVwP/PZSQzlhs0aEDfvn3p3LkzKSkpNGnS5OhzAwcO5Pnnn6dDhw6kpaVx6qmn\nlvv9HnnkEXr37k2jRo3o3bv30YD5zDPPkJ6ezj//+U8SEhKYMGECffr04b777qNfv34kJCTQrVs3\nXvFfAjOlS0yEk05ym19eHmzY4JZgWbrU3X79tctS9Gvc2PXEunZ11T26dIF27cK/vFjenlp5z4+V\ny5t2edJbqrmI+PMTEoBJqK5AZAywCNVpiPQE3gfqAb9G5GFUO3nRnEpdMHfVqlV0CKNU+5YtsGmT\nqzmYkBDpFsavcD9HE8SuXa5XFhjMli/PT+pISoK0NDffrGNHd9upkxtcLWk+x9ChLvikp8PEie6H\n+L33Qm9Xec+/5RZ3XvXq7mu56abQA0+kCh6PGAEvvBDeeweKdtCL8vvHW+1BC1oBvv/e1WPt0iXS\nrYtvFrQ8kpMDa9a4ILZ8OaxYAStXuh9E/+9l9eoumAUGsk6doE2b2FiCoDxBb8uW4ntqofzxrixB\nr7zvX04WtKIkEkFr5Ur3d6Bdu0i3LvJuvfVW/vOf/xTYN3LkSK677rqIv5cFrQq2f79b1G3FivxA\ntmJFwbEmfzDzB7EOHdzjE0+Mrxpk5empxXvQi9T7lzNoxlvQioF/1WKDqvv58WV/x7zx4Y5/mPhR\ns6ab1Ny9e8H9+/fDqlUFA9n8+QXHy6pVcwkjaWn5W/v27vbYY2Nv6QJ/IkpgTy1U0R6TK2/2ZayM\nCcYZC1o+OTluDD2e/kk1VUzNmtCjh9sC7dsHa9e6S41r1rhe2po1LsMxOzv/uNq1iwaytDQ3xyxa\n9RIDLyWW5R+xeA560Z7yEKcsaPl4kTloTIWoVcvVWjzllIL78/JcZpE/iPm3L75wS7f4iUCLFnDC\nCW5r0yb//gknuMsPsdZD84vnoFfe949UTy3OWNDyiXR1d2Oirlo1aNnSbWefXfC5/fshIyO/Z7Zu\nnfsj6J9LFqhWrYJBLHBr1cpVCIlX0Qx65X3/SNW+jDMWtHwOHnRp7pFOyNq9ezdvvPEGI0aMCPvc\np59+mvT0dFJLqqpgTFnUrJlfvaOw7Gw312z9+oJbRoZbMTrw0pOIW7fs+ONdby3YduyxsZHp6IXy\nBr3yKm/QjEOWPeizZo1LxqhTJ7JTJjZs2MD555/P8uXLwz7XX+m9YZSzQyx70Byl6v5QBgazdetg\n40bIynK3hTPiqlVzvYIWLVyACxbYmjWL7x5bHLPswTh14ICrORjpRJzApUnOPvtsGjduzNtvv82h\nQ4cYMmQIDz/8MPv37+eyyy4jKyuLI0eO8MADD7B161Y2b97MmWeeScOGDfnss8+Cvv4tt9zCwoUL\nOXDgAJdccgkPP/wwAAsXLmTkyJHs37+fGjVqMGfOHFJTUxk1ahSffPIJ1apV48Ybb+T2228P+rrG\nBCXi/ps79lj41a+KPq/qJlJnZQXfVq2CWbNc8khhjRsHD2yBj2vGzd9W4xHraQG5ue53IXDFCb/y\nJuIE9rRmzZrF1KlTeeGFF1BVLrjgAu655x62bdvGJ598wosvvgi4moR16tQJqae1c+dO6tevz5Ej\nRxgwYADjxo2jffv2tG/fnrfeeouePXuyZ88eUlNTefHFF5kzZw5vvvkmiYmJR88tjfW0TMTt2ZMf\nyDZudAkjmzYVDHDBamHWrRs8sDVrlh9MGzeuvJcjPWA9rTh08CB8+KGrezpjhneJOLNmzWLWrFl0\n69YNgH379pGRkcHpp5/OnXfeyahRozj//PM5/fTTQ37Nt99+m4kTJ5Kbm8uWLVtYuXIlIkLTpk3p\n2bMnAMcccwwAs2fP5uabbybR9wtdZZciMdF3zDGuykfHwgvgBjhwoGAgKxzUli1zlyoL/+MtAo0a\nuQDWtGl+MCv8uGlTNw0gVjMjTVAWtHC/Gw0bQr163ibiqCqjR4/mpptuKvLckiVLmDFjBvfffz8D\nBgzgwQcfDPIKBX3//feMHTuWhQsXUq9ePYYPHx50aRNj4lJKiqvwUdLidjk5sHmzS0D46Se3Bd7/\n6Sc3Efunn9yxwd7DH8CaNnU9tmbN8u/7b+vVs+AWIyxo4QKUiFv3L9KJOIFLk5x77rk88MADXHXV\nVdSqVYtNmzaRlJREbm4u9evX5+qrr6Zu3bq89NJLBc4t7vLgnj17qFmzJnXq1GHr1q18/PHH9O/f\nn7S0NLZs2cLChQvp2bMne/fuJSUlhbPPPpsXXniBM888M6zLg8bErKQkl7l4/PElH+cfaysc1LZs\nyX+8YgXMnu2WnCmsRo2Cga3wbePGUL++22rWtADnIQta5PeuvMheDVyaZNCgQVx55ZX06dMHgFq1\navHaa6+RmZnJ3XffTbVq1UhKSjq67lV6ejoDBw6kWbNmQRMxunTpQrdu3Wjfvj0tW7akb1+3CHT1\n6tV56623uP322zlw4AApKSnMnj2bG264gbVr13LyySeTlJTEjTfeyG233RaZL9SYWCaSH1RKuiQJ\nbnxgy5b8Hlzh25UrYc4c2L07+PlJSe596tXLf0//Vty+Bg3ceF01W+KwNJaIgVsxwj9/0hRliRjG\nBHHgQH4w27bNJY7s2uVug227drkElOIkJLjg1bCh2xo1yr9f3OMIzOG0RIw4c+SIyxq0ShjGmLCk\npORXBglVTo7roRUOaDt2uPGJ7dtdANy+3U0P8O/Lyyu+DQ0buukHgYWTK7EqH7Ty8lwPvVataLek\nZL179+bQoUMF9k2ZMoWTAlfNNcbEtqQk10Nq1Cj0c/LyXKALDGiFA1zz5t61OcZU+aCVlBQflwUX\nLFgQ7SYYY6KhWrX88a94WOzPYzbqZ4wxJm5U+qBVWRJNosU+P2NMLPE0aIkwUIQ1ImSKcG+Q548T\n4TMR/ifCtyIMDnjuZBG+FmGFCN+JEHaqRHJyMjt27LA/vGWkquzYsYNky1IxxsQIz1LeRUgA1gJn\nA1nAQuAKVVYGHDMR+J8qE0ToCMxQpZUIicAS4BpVlonQANitypHi3i9YyntOTg5ZWVlWJaIckpOT\nadGiBUlJSdFuijHGA5bynq8XkKnKegAR3gQuhPygBShwjO9+HWCz7/45wLeqLANQZUdZGpCUlETr\n1q3LcqoxxpgY5OXlwebAxoDHWb59gR4CrhYhC5gB+NfJaAeoCDNFWCLCPR620xhjTJyIdiLGFcAr\nqrQABgNTRKiG6wGeBlzlux0iwoDCJ4uQLsIiERbl5lZks40xxkSDl0FrE9Ay4HEL375A1wNvA6jy\nNZAMNMT1yr5QZbsq2bhe2CmF30CViar0UKWHLZ9jjDGVn5d/6hcCbUVojQtWlwNXFjrmR2AA8IoI\nHXBBaxswE7hHhFTgMNAP+HtJb5adna0iUo7lGkkEYrm/Zu0rH2tf+Vj7yieW25cS7QaEw7OgpUqu\nCLfhAlACMEmVFSKMARapMg24E3hRhD/gkjKGq6LALhH+hgt8issqnF7y+2m5eo0iskhVe5TnNbxk\n7Ssfa1/5WPvKJ9bbF088vaimygzcpb3AfQ8G3F8J9C3m3NeA17xsnzHGmPgS7UQMY4wxJmQWtPJN\njHYDSmHtKx9rX/lY+8on1tsXNyrNIpDGGGMqP+tpGWOMiRtVKmiJyEARWSMimSISpICv1BCRt3zP\nLxCRVhXYtpYi8pmIrBSRFSIyMsgx/UXkFxFZ6tseDPZaHrdzg4h853v/RUGeFxEZ5/sMvxWRIvPr\nPGxbWsBns1RE9ojI7wsdU6GfoYhMEpGfRWR5wL76IvKpiGT4busVc+4w3zEZIjKsAtv3pIis9n3/\n3heRusWcW+LPgofte0hENgV8DwcXc26Jv+8etu+tgLZtEJGlxZzr+edXKalqldhwaffrgBOA6sAy\noGOhY0YAz/vuXw68VYHtawqc4rtfG1dsuHD7+gP/jvLnuAFoWMLzg4GPAQFOBRZE8fv9E3B8ND9D\n4AzcxPjlAfueAO713b8XeDzIefWB9b7ber779SqofecAib77jwdrXyg/Cx627yHgrhC+/yX+vnvV\nvkLPPwU8GK3PrzJuVamn5Svgq+tV9TAcLeAb6ELgVd/9qcAAEZGKaJyqblHVJb77e4FVFK3VGA8u\nBIyCsjcAAATsSURBVCarMx+oKyJNo9COAcA6Vf0hCu99lKp+AewstDvw5+xV4KIgp54LfKqqO1V1\nF/ApMLAi2qeqs1TVPxF2Pq6aTVQU8/mFIpTf93IrqX2+vx2XAf+K9PtWZVUpaIVSwPfoMb5f2l+A\nBhXSugC+y5LdgAVBnu4jIstE5GMR6VShDXMUmCUii0UkPcjzoXzOFeFyiv9jEe3PsImqbvHd/wlo\nEuSYWPkcf4vrOQdT2s+Cl27zXb6cVMzl1Vj4/E4HtqpqRjHPR/Pzi1tVKWjFBRGpBbwL/F5V9xR6\negnuclcX4B/ABxXdPuA0VT0FGATcKiJnRKENJRKR6sAFwDtBno6Fz/AoddeJYjKFV0Tuw5Ueer2Y\nQ6L1szABaAN0BbbgLsHFoisouZcV879LsagqBa1QCvgePUZEEnFrfJVpLa+yEJEkXMB6XVXfK/y8\nqu5R1X2++zOAJBFpWFHt873vJt/tz8D7uMswgUL5nL02CFiiqlsLPxELnyGw1X/J1Hf7c5Bjovo5\nishw4HzgKl9gLSKEnwVPqOpWVT2iqnnAi8W8b7Q/v0RgKPBWccdE6/OLd1UpaPkK+Epr33/ilwPT\nCh0zDfBnaV0CzC3uFzbSfNe//wmsUtW/FXPMsf4xNhHphfv+VWRQrSkitf33cQP2ywsdNg241pdF\neCrwS8ClsIpS7H+40f4MfQJ/zoYBHwY5ZiZwjojU813+Ose3z3MiMhC4B7hAVbOLOSaUnwWv2hc4\nRjqkmPcN5ffdS/8HrFbVrGBPRvPzi3vRzgSpyA2X2bYWl1V0n2/fGNwvJ7gq8+8AmcA3wAkV2LbT\ncJeJvgWW+rbBwM3Azb5jbgNW4DKh5gO/quDP7wTfey/ztcP/GQa2UYDxvs/4O6BHBbexJi4I1QnY\nF7XPEBc8twA5uHGV63HjpHOADGA2UN93bA/gpYBzf+v7WcwErqvA9mXixoP8P4f+jNpmwIySfhYq\nqH1TfD9b3+ICUdPC7fM9LvL7XhHt8+1/xf8zF3BshX9+lXGzihjGGGPiRlW6PGiMMSbOWdAyxhgT\nNyxoGWOMiRsWtIwxxsQNC1rGGGPihgUtY2KAr/r8v6PdDmNinQUtY4wxccOCljFhEJGrReQb3xpI\nL4hIgojsE5G/i1sHbY6INPId21VE5gesS1XPt/9EEZntK9q7RETa+F6+lohM9a1l9XpFrTBgTDyx\noGVMiESkA/AboK+qdgWOAFfhqnAsUtVOwOfAn3ynTAZGqerJuAoO/v2vA+PVFe39Fa6iArjK/r8H\nOuIqJvT1/IsyJs4kRrsBxsSRAUB3YKGvE5SCK3abR35h1NeA90SkDlBXVT/37X8VeMdXb665qr4P\noKoHAXyv9436atX5VrttBXzl/ZdlTPywoGVM6AR4VVVHF9gp8kCh48paG+1QwP0j2O+nMUXY5UFj\nQjcHuEREGgOISH0ROR73e3SJ75grga9U9Rdgl4ic7tt/DfC5ulWps0TkIt9r1BCR1Ar9KoyJY/af\nnDEhUtWVInI/brXZarjK3rcC+4Fevud+xo17gVt25HlfUFoPXOfbfw3wgoiM8b3GpRX4ZRgT16zK\nuzHlJCL7VLVWtNthTFVglweNMcbEDetpGWOMiRvW0zLGGBM3LGgZY4yJGxa0jDHGxA0LWsYYY+KG\nBS1jjDFxw4KWMcaYuPH/YNz/Y4Khb9kAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fa7603232d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# COMPILE & TRAIN\n",
    "model.compile(loss='categorical_crossentropy',\n",
    "              optimizer=optim,\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "t = time.time()\n",
    "history = model.fit(X_train, Y_train,\n",
    "                batch_size=batch_size, nb_epoch=nb_epoch,\n",
    "                verbose=2,validation_data=(X_test, Y_test))\n",
    "print (time.time() - t, \"seconds.\")\n",
    "\n",
    "score = model.evaluate(X_test, Y_test, verbose=0)\n",
    "print (\"-\"*10)\n",
    "print (\"Loss: %f\"%(score[0]))\n",
    "print (\"Accuracy: %f\"%(score[1]))\n",
    "plot_curves(history,nb_epoch)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "So, adding a hidden layer pushed the accuracy from 0.92 to almost 0.98. Of course the number of parameters has increased significantly. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Going deep"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Can we keep adding layers just like that and get better and better accuracy? \n",
    "\n",
    "**Exercise**: Design and train network with 6 hidden layers with 128 neurons + classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "____________________________________________________________________________________________________\n",
      "Layer (type)                     Output Shape          Param #     Connected to                     \n",
      "====================================================================================================\n",
      "dense_4 (Dense)                  (None, 128)           100480      dense_input_3[0][0]              \n",
      "____________________________________________________________________________________________________\n",
      "activation_4 (Activation)        (None, 128)           0           dense_4[0][0]                    \n",
      "____________________________________________________________________________________________________\n",
      "dense_5 (Dense)                  (None, 128)           16512       activation_4[0][0]               \n",
      "____________________________________________________________________________________________________\n",
      "activation_5 (Activation)        (None, 128)           0           dense_5[0][0]                    \n",
      "____________________________________________________________________________________________________\n",
      "dense_6 (Dense)                  (None, 128)           16512       activation_5[0][0]               \n",
      "____________________________________________________________________________________________________\n",
      "activation_6 (Activation)        (None, 128)           0           dense_6[0][0]                    \n",
      "____________________________________________________________________________________________________\n",
      "dense_7 (Dense)                  (None, 128)           16512       activation_6[0][0]               \n",
      "____________________________________________________________________________________________________\n",
      "activation_7 (Activation)        (None, 128)           0           dense_7[0][0]                    \n",
      "____________________________________________________________________________________________________\n",
      "dense_8 (Dense)                  (None, 128)           16512       activation_7[0][0]               \n",
      "____________________________________________________________________________________________________\n",
      "activation_8 (Activation)        (None, 128)           0           dense_8[0][0]                    \n",
      "____________________________________________________________________________________________________\n",
      "dense_9 (Dense)                  (None, 128)           16512       activation_8[0][0]               \n",
      "____________________________________________________________________________________________________\n",
      "activation_9 (Activation)        (None, 128)           0           dense_9[0][0]                    \n",
      "____________________________________________________________________________________________________\n",
      "dense_10 (Dense)                 (None, 10)            1290        activation_9[0][0]               \n",
      "____________________________________________________________________________________________________\n",
      "activation_10 (Activation)       (None, 10)            0           dense_10[0][0]                   \n",
      "====================================================================================================\n",
      "Total params: 184,330\n",
      "Trainable params: 184,330\n",
      "Non-trainable params: 0\n",
      "____________________________________________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "np.random.seed(SEED)\n",
    "\n",
    "model = Sequential()\n",
    "model.add(Dense(128,input_shape=(784,)))\n",
    "model.add(Activation('relu'))\n",
    "for i in range(5):\n",
    "    model.add(Dense(128))\n",
    "    model.add(Activation('relu'))\n",
    "    \n",
    "model.add(Dense(10))\n",
    "model.add(Activation('softmax'))\n",
    "# TODO: add layers as indicated\n",
    "\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 60000 samples, validate on 10000 samples\n",
      "Epoch 1/20\n",
      "2s - loss: 0.5982 - acc: 0.8105 - val_loss: 0.2160 - val_acc: 0.9323\n",
      "Epoch 2/20\n",
      "2s - loss: 0.1573 - acc: 0.9533 - val_loss: 0.1425 - val_acc: 0.9571\n",
      "Epoch 3/20\n",
      "2s - loss: 0.1112 - acc: 0.9669 - val_loss: 0.1150 - val_acc: 0.9643\n",
      "Epoch 4/20\n",
      "2s - loss: 0.0862 - acc: 0.9740 - val_loss: 0.0920 - val_acc: 0.9711\n",
      "Epoch 5/20\n",
      "2s - loss: 0.0702 - acc: 0.9786 - val_loss: 0.0932 - val_acc: 0.9688\n",
      "Epoch 6/20\n",
      "2s - loss: 0.0575 - acc: 0.9821 - val_loss: 0.0887 - val_acc: 0.9747\n",
      "Epoch 7/20\n",
      "2s - loss: 0.0481 - acc: 0.9850 - val_loss: 0.0839 - val_acc: 0.9747\n",
      "Epoch 8/20\n",
      "2s - loss: 0.0399 - acc: 0.9872 - val_loss: 0.0918 - val_acc: 0.9745\n",
      "Epoch 9/20\n",
      "2s - loss: 0.0331 - acc: 0.9896 - val_loss: 0.1007 - val_acc: 0.9697\n",
      "Epoch 10/20\n",
      "2s - loss: 0.0300 - acc: 0.9903 - val_loss: 0.0929 - val_acc: 0.9742\n",
      "Epoch 11/20\n",
      "2s - loss: 0.0255 - acc: 0.9916 - val_loss: 0.0801 - val_acc: 0.9772\n",
      "Epoch 12/20\n",
      "2s - loss: 0.0201 - acc: 0.9939 - val_loss: 0.1021 - val_acc: 0.9732\n",
      "Epoch 13/20\n",
      "2s - loss: 0.0190 - acc: 0.9938 - val_loss: 0.1124 - val_acc: 0.9728\n",
      "Epoch 14/20\n",
      "2s - loss: 0.0171 - acc: 0.9945 - val_loss: 0.0954 - val_acc: 0.9760\n",
      "Epoch 15/20\n",
      "2s - loss: 0.0166 - acc: 0.9942 - val_loss: 0.0881 - val_acc: 0.9776\n",
      "Epoch 16/20\n",
      "2s - loss: 0.0139 - acc: 0.9955 - val_loss: 0.1026 - val_acc: 0.9749\n",
      "Epoch 17/20\n",
      "2s - loss: 0.0112 - acc: 0.9965 - val_loss: 0.0999 - val_acc: 0.9776\n",
      "Epoch 18/20\n",
      "2s - loss: 0.0114 - acc: 0.9964 - val_loss: 0.0949 - val_acc: 0.9788\n",
      "Epoch 19/20\n",
      "2s - loss: 0.0125 - acc: 0.9959 - val_loss: 0.0998 - val_acc: 0.9780\n",
      "Epoch 20/20\n",
      "2s - loss: 0.0110 - acc: 0.9965 - val_loss: 0.0975 - val_acc: 0.9791\n",
      "(44.94973015785217, 'seconds.')\n",
      "----------\n",
      "Loss: 0.097489\n",
      "Accuracy: 0.979100\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbMAAAEKCAYAAACYKLs6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl8VOXZ+P/PlY0k7JsIBAEFkU1BWbS0irUoKFVRiwpa\n8bFSXFr6POpP/Wlti9/+altr1W8pipZabbVaRaWPWhAEd5SIEQFZQkBJ2MK+Bkhy/f64z5BJMpPM\nZObMZMj1fr3O65y5zzL3TCZzzX2fexFVxRhjjEllacnOgDHGGBMrC2bGGGNSngUzY4wxKc+CmTHG\nmJRnwcwYY0zKs2BmjDEm5VkwM8YYk/IsmBljjEl5FsyMMcakvIxkZyAR0tLSNCcnJ9nZMMaYlHLw\n4EFV1ZQo9DSJYJaTk8OBAweSnQ1jjEkpInIo2XmIVEpEXGOMMaYuvgYzEWaJsE2E5WH2iwiPi1Ao\nwjIRzgzad4MIa73lhqD0s0T40jvncRHEz9dgjDGm8fO7ZPYMMLqO/WOA3t4yGZgBIEI74BfAcGAY\n8AsR2nrnzABuDjqvrusbY4xpAny9Z6bKeyL0qOOQy4BnVVFgsQhtROgMjATeVmUngAhvA6NFWAS0\nUmWxl/4scDnwVrR5O3r0KMXFxZSVlUV7qqkhOzubvLw8MjMzk50VY0wTlewGIF2BjUGPi720utKL\nQ6RHrbi4mJYtW9KjRw9ErKayoVSVHTt2UFxcTM+ePZOdHWNMIomMBh4D0oGnUX0oxDHjgV8CCnyB\n6gQ/spLsYOYbESbjqi7Jyqq9v6yszAJZHIgI7du3p7S0NNlZMcYkkkg6MB0YhStYLEFkDqorg47p\nDdwLjEB1FyIn+JWdZLdmLAG6BT3O89LqSs8LkV6LKjNVGaLKkIwwIdsCWXzY+2hMkzQMKES1CNUj\nwD9xt46C3QxMR3UXAKrb/MpMsktmc4DbRfgnrrHHHlU2izAX+P+CGn1cCNyryk4R9opwNvAJ8EPg\n//qWux07oKICTvDtx4QxJskqK2HPHti9G3btqr7etw9E3JKWVn0dKq3mOj0dcnKqltzc6uvAdrgf\n3OHye/Ag7N8ffjlwoGr7Zz+D9u19eetC3Q4aXuOYUwEQ+RBXFflLVP/jR2Z8DWYivIBrzNFBhGJc\nC8VMAFWeAN4ELgYKgYPAjd6+nSI8CCzxLjUt0BgEuBXXSjIH1/Aj6sYfEdu1Cw4ftmBmTAooL3e/\nP0tLYds2t2zfXjtA1Vzv3Quqyc17RkboYJeZWTtwHTwY+XXT0uDaaxsezDpABiL5QUkzUZ0ZxSUy\ncK3OR+Jq0t5DZCCquxuWo7qfyDeqXFvPfgVuC7NvFjArRHo+MCAuGaxPVpb7aeaD3bt38/zzz3Pr\nrbdGdd7FF1/M888/T5s2baI6b9KkSYwdO5arrroqqvOMqY+q+4Ldt88t+/dXbZeVuS/qjAz3xRzt\nuqLCBaRAcAoOVDUf79gRPig1bw5t2kDbtm7drRsMHFg9LXg7sG7RwpWwKivdtVWrtiNJq6iAQ4eq\nloMHo9s+cgROPNHlo3lzt45kCRybne3y31DboRzVIWF2h7sdFKwY+ATVo8B6RNbggtsS4izZ1YyN\nW+C/qaLC1RfE0e7du/nzn/9cK5iVl5eTUUedw5tvvhnXfJjjhyps3gyrVrll9WrYtKmq2is93a0D\nSySPKytrB6iaj/fvT3zJpm1bV2Fywglw2mlw7rlVjzt2rNru0MEda71GfLEE6I1IT1wQuwao2VLx\nNeBa4K+IdMBVOxb5kRkLZuAqlQsKaqcfPep+WjZv7v67ozFoEDz6aNjd99xzD+vWrWPQoEFkZmaS\nnZ1N27ZtWbVqFWvWrOHyyy9n48aNlJWVMXXqVCZPngxAjx49yM/PZ//+/YwZM4Zvf/vbfPTRR3Tt\n2pXXX3+dSAZUXrBgAXfeeSfl5eUMHTqUGTNm0KxZM+655x7mzJlDRkYGF154IQ8//DD/+te/+NWv\nfkV6ejqtW7fmvffei+59MHFXVgZr17pgFQhagXVwRULz5q4EIuJ+j1VWVi2RPhaBli3dr/yWLd3S\nqROcckrV4+B9NR9nZ7trHT3qqgGjXUP14NSxowtQoVoomwRTLUfkdmAu7n7YLFRXIDINyEd1jrfv\nQkRWAhXAXaju8CM7FszqEghgPvzsfOihh1i+fDkFBQUsWrSISy65hOXLlx/rqzVr1izatWvHoUOH\nGDp0KFdeeSXta1R8r127lhdeeIGnnnqK8ePH88orr3DdddfV+bxlZWVMmjSJBQsWcOqpp/LDH/6Q\nGTNmcP311/Pqq6+yatUqRITdu12V9rRp05g7dy5du3Y9lmb8peqCUmkpFBdXBatAwFq/vvpH8qST\noE8fmDTJlVL69HHrLl1iq2Iypl6qb+LaPgSnPRC0rcD/eIuvLJhB+BJUWRksXw49e/rWHChg2LBh\n1TodP/7447z66qsAbNy4kbVr19YKZj179mTQoEEAnHXWWWzYsKHe51m9ejU9e/bk1FNdI6MbbriB\n6dOnc/vtt5Odnc1NN93E2LFjGTt2LAAjRoxg0qRJjB8/niuuuCIeL7XJqayEnTtdcCotdfeAAtuh\nlu3b3b2SYLm5cOqpMGwYXH+9C1annQa9e7sSmDFNnQWzugQq2mt+s/igedA30qJFi5g/fz4ff/wx\nubm5jBw5MuSwW82aNTu2nZ6ezqFDDZ+tISMjg08//ZQFCxbw8ssv86c//Yl33nmHJ554gk8++YQ3\n3niDs846i88++6xWUD1e7d/vSkZ1NYGu2Qy65rJnj2uYUFkZ+jlatXJVZx07uhLWWWdVPe7YETp3\ndiWtvLzoa7qNaUosmNUlPd0tR4/G/dItW7ZkX5iWknv27KFt27bk5uayatUqFi9eHLfn7dOnDxs2\nbKCwsJBevXrx3HPPcd5557F//34OHjzIxRdfzIgRIzj55JMBWLduHcOHD2f48OG89dZbbNy48bgK\nZqqwdaurwvvqq6rqvK++go0b6z8/Kyt0a7KTTnIlpuBgVXNp3x6Cfo8YY2Jgwaw+WVm+lMzat2/P\niBEjGDBgADk5OXTq1OnYvtGjR/PEE0/Qt29f+vTpw9lnnx23583Ozuavf/0rP/jBD441AJkyZQo7\nd+7ksssuo6ysDFXlkUceAeCuu+5i7dq1qCoXXHABZ5xxRtzykkjl5e5eU3DQCqyDbwW2aOGq70aO\ndOsePVxACtX0uXlza4hgTGMhmuzeggnQvHlzrTnT9FdffUXfvn3rP3nNGvdN2K+fT7k7PkT8fvps\n5073Jwssq1e7oLV2bfXfJJ07u2DVt2/1ddeu1mjCmAAROaiqKXFX1kpm9cnKcr0XTaNx4AAUFlYP\nWoFl586q4zIyXNudvn1h7NiqoNWnj+sQa4w5flgwq09WlrtnVlmZEnfgb7vtNj788MNqaVOnTuXG\nG29MUo4aRhW+/hpWrKgdsIqLqx+bl+da+o0f79aBpUcP6yxrTFNhwaw+gW/Do0dT4m799OnTk52F\nqJWXu+rAzz+vWgoK3Nh5Ae3bu2bo3/1u9YDVq5c1TTfGWDCrX+AOf4oEs8bu4EH48svqgevLL12X\nPnAjRpx+uitlDR7stk891fdufsaYFGfBrD6BYJaAvmbHm717IT+/euBataqqz1WbNi5g3XqrWw8e\n7O5nRTMdhjHGgAWz+gVXM5o6FRfDhx/CBx/A++/DsmVVwy517eqC1ZVXVgWu7t2t5aAxJj4smNUn\nMJS4lcyqUXVVg4HR00tKYNQot695czjnHPjFL+Dss+HMM10nYWOaks2b4Zpr4MUX3TQuTe35E63x\nN89LNhFXOjtyxH06zjsPtmyJ+bKBKWAa4tFHH+VgPTP09ejRg+3btzfo+qFUVrrAtXmz67NVUOBa\nGn79tQtmWVluiMv8fNcJ+e23XTC76CILZKZh4vjvlpTnf/BBV0sxbVpqPn/KUdXjfsnNzdWaVq5c\nWSstrFWrVL/6SvWWW1TT0tw6RuvXr9f+/fs36Nzu3btraWlpzMfUpaJCdc8e1Y0b3UvPz1ddssQt\nX36pun69ammp6qFDqpWVUb6fJiKbNqmee67q5s3JzklyxPHfLaHPn50dmJ6z+pKdnVrPr6oKHNBG\n8B0eyeLvxdHRoKtBC0HvCbG/O+gC0GWgi0DzvPTzQQuCljLQy719z4CuD9o3qL58xBzMmjWL36fD\nc/XVV2t2draeccYZeuedd+rvfvc7HTJkiA4cOFAfeOABVVXdv3+/XnzxxXr66adr//799Z///Kc+\n9thjmpmZqQMGDNCRI0eGvX5wMPvDH/6g/fv31/79++sf//jHsNc+elT1pz+9W3v16qu9eg3UiRPv\n0Px81ZUrVb/5RnXXLtUjR0I/nwWz+Ev2l3msGhqM4/ll3BCxPv+mTaoTJqjm5rrzcnNVJ06M/H1I\n9vMHs2Dmgk466DrQk0GzQL8A7VfjmH+B3uBtfxf0uRDXaQe6EzRXq4LZVdHkJeZglp+vOnp0fD4d\nnuCS2dy5c/Xmm2/WyspKraio0EsuuUTfffddffnll/VHP/rRsXN2796tqtGVzPLz83XAgAG6f/9+\n3bdvn/br10+XLl167NplZapbtqguWbJb58/frieddKouXVqp69erbtiwS8vLI3s9od7PWEsWTbVk\nkuwv83hpaDCO15dxQz8/8Xj+KVPca8/Ojv49SPbzB0ulYObnPbNhQKEqRaocAf4JXFbjmH7AO972\nwhD7Aa4C3lKl7ptEfurWzU0oVVbmOkKVlbnRZ+N0V3XevHnMmzePwYMHc+aZZ7Jq1SrWrl3LwIED\nefvtt7n77rt5//33ad26ddTX/uCDDxg3bhzNmzenRYsWjBt3BfPnv88JJwzkzTffZvLku5kz532y\ns1tzyimtadUqm8cfv4mlS2fTuXMu6ekNf12x1tkn+55DrBr6/EVFMGGC+8iBW0+c6AZKTgU5Oe5W\n84wZ7l7rjBnucQSToANu3MxWrWL/d2vo5ycez791K0yZAosXu3U0n4FkP3+q8jOYdQWCJ9Eo9tKC\nfQEEZnwcB7QUoWb32GuAF2qk/VqEZSL8UQT/ezJnZblB/266yZdPh6py7733UlBQQEFBAYWFhdx0\n002ceuqpLF26lIEDB3L//fczrYHf6qquUcaGDbBtm/ug5+ScyuzZSxk2bCDPPns/s2dPo0ePDPLz\nP+UHP7iK//3f/2X06NENer5Yv8xiPT8g2TfAk/llCrEH82QG41i+jOPx+Yk1GMyeDdOnwxlnuPXs\n2dGdn+znT0l+FflArwJ9Oujx9aB/qnFMF9DZoJ+DPgZaDNomaH9n0FLQzBppAtoM9G+gD4R5/smg\n+aD5WVkxVjPu3+9aPuzaFfk59di+fbuedNJJquqqGYcNG6b79u1TVdXi4mLdunWrlpSU6KFDh1RV\n9d///rdedtllqqo6YMAALSoqqvP63bt31zVrSvW11z7TXr0G6vvvH9APPtivvXv313feWapff137\n2vv27dOtW7eqqqvSbNeuXcSvJ/j9jLWaJNn3HGIVj+cfN0711ltVCwrcety46PMR6z23WM6PVzVX\nQ8TznlFTRwpVM/rZz6wE6Bb0OM9LCwqkbMIrmYnQArhSlaDZpRgPvKrK0aBzNnubh0X4K3BnqCdX\nZSYwE6B5c2Kb58aHUUCC5zMbM2YMEyZM4JxzzgGgRYsW/P3vf6ewsJC77rqLtLQ0MjMzmTFjBgCT\nJ09m9OjRdOnShYULF1a7bmWlm9n46FHXbD4v70yuvnoSkycPIy0Nbr31R5x//mDmzp3L2LHVr71v\n376Qc5pFK9aSRaznFxXBnXfCa6+54bNyc2HcOHj44Qa9nKjF4/mDf0lHO9xmTk7V8GDgSiYzZrj3\nMpIJIGI9H6pKFpMnw8yZrpSXKPEq2ZoU41eUBM0ALQLtGdQApH+NYzqApnnbvwadVmP/YtDza6R1\n9tYC+ijoQ/XlJeYGIJWVrhFIcXHk5yRYebn75VlQ4AqRK1aorl7ttjds8P/5a76fsZYsYj0/mSWD\nZD9/skvGjUE8SrbGSmZekKRchNuBuUA6MEuVFSJMA/JVmQOMBH4jggLvAbcFzhehB65k926NS/9D\nhI6AAAXAFL9ewzHBHacbmaNH3X2wbdugogJatnSj0Af3qS4tdYsInHVWYvIVS8kiHufHo2QQywgK\nqVwyOR5KNrF+fkzqsZmmI7VqlYsGffrEOXcNc/iwuyl8ySXDOXLkMBkZboDetDSYNes52rQZyO7d\nVdOwtWnjGmVGM7/XkSOuyuyUU+o/r7HMNB1Pt94KTz4JP/4xNHCwlqS54goXlIKDaTSNAGI93xwf\nUmmm6SYdzE477TQk0pFu161zNwwGDPAhh5E7eNAFsZ07XWxt3979Ys7Orn7c119XlcZU3ZBS3btH\n91yBa9R3rqqyatWq4yaY1bxnFBDNPSNjjgepFMya7EDD2dnZ7Nixg/bt20cW0LKyYM8eFxmSMNT7\nvn0uiO3Z40panTq5JdA2paajR10Q6tjRBaRoBv3/7DP3MgPqqqZUVXbs2EF2zWiawpLdgMQYE70m\nG8zy8vIoLi6mtLQ0shP27nVTH69c6aJJAqg3Mv2ePa5aMS3N3bto2dIN+rt/f93nB1o0Bnz1VWTP\n26yZe6kHD1bF7txcaNs29DWys7PJy8uL/IU1csfDPSNjEkJkNPAYrl3E06g+VGP/JOD3VLVk/xOq\nT/uRlSYbzDIzM+nZs2fkJ7z4omsNsHw5JKA6be9eN/fX/Pmuiu+uu+DGG6s6ovrtllvcvZKsLHfv\nLBXvG8UimQ04jEkJIunAdGAUblCMJYjMQXVljSNfRPV2v7PTZINZ1AIlj+Ji6N/f16fasgXGjIEv\nv4ReveCdd1zjjURq6l/m1hrOmHoNAwpRLQJAJDBkYc1glhAWzCLV1RuJq6Sk7uNitHatmwNs2zYY\nPRreegt+85vEl4rsy9wY0wEyEMkPSpqJ6kxvO9SQhcNDXOZKRM4F1gD/jerGEMfEzCbnjFSXLm5d\nXOzbUyxZAt/6lhvD7sABeOON2MYmNMaYWGyHclSHBC0z6z+rmn8DPVA9HXgb+Fv8c+lYMItUVpZr\nPuhTMPvPf+D8813jjg8+SO1R040xTUK9QxaiugPVw96jpwHfhm2wYBaNrl19qWZ87jn4/vehd2/4\n6CMYMcJa0xljGr0lQG9EeiKShZvhZE61I0Q6Bz26FIiwTXX0LJhFIy8vriUzVfj97+GHP4Rzz4V3\n360KWE1xPiJjTApRLYdjQxZ+BbyE6gpEpiFyqXfUTxFZgcgXwE+BSX5lp8mOANIgt93mmuhv3x7z\npSorXcfcP/4Rrr4a/vY317/LGGMai1QaAcRKZtHo2tXNrxLhmEbhJjc8fBiuu84FsqlT4fnnLZAZ\nY0wsLJhFI9DXLML7ZqFmGt67Fy65BF54AR56yAW0BA0oYowxxy3rZxaN4L5mvXqFPSzc5IbNmkG/\nfrBsGTzzDNxwg7/ZNcaYpsLKBNEIHgWkDkVFtZvWX3qpa9m/ejX8+98WyIwxJp6sZBaNQMmsnmBW\nc6DaQ4dg3jxo3twNTTU8VB95Y4wxDWYls2i0aAGtW0d0zyzQtP6RRyA93Y3g8eGHFsiMMcYPFsyi\nFWFfs9mz4cIL4ac/deMSr1vXaCapNsaY446vwUyE0SKsFqFQhHtC7O8uwgIRlomwSIS8oH0VIhR4\ny5yg9J4ifOJd80URwkxP6ZMoOk7/6U9u+pZ333VVj8YYY/zhWzATITDXzRigH3CtCP1qHPYw8Kwq\npwPTgN8E7TukyiBvuTQo/bfAH1XpBewCbvLrNYQUxZBWJSVwxhmuZtIYY4x//CyZDQMKVSlS5QgQ\nmOsmWD/gHW97YYj91YggwHeBl72kvwGXxy3HkcjLc72gy8vrPbSkpKrNiDHGGP/4GcxCzXVT86v9\nC+AKb3sc0FKE9t7jbBHyRVgscixgtQd2qxKIJKGu6a+uXd1YVPUMlrh/v+sgbcHMGGP8l+wGIHcC\n54nwOXAebvqACm9fd1WGABOAR0U4JZoLizDZC4b5ERSiIhdhX7NATaQFM2OM8Z+fwazeuW5U2aTK\nFaoMBu7z0nZ76xJvXQQsAgYDO4A2Isf6x9WeP6fq2jNVGaLKkIx49qazYGaMMY2On8FsCdDba30Y\ncq4bETqIHMvDvcAsL72tCM0CxwAjgJWqKO7e2lXeOTcAr/v4GmoLHtKqDhbMjDEmcXwLZt59rWpz\n3aiyQoRpIsdaJ44EVouwBugE/NpL7wvki/AFLng9pMpKb9/dwP+IUIi7h/YXv15DSO3auWE9rGRm\njDGNhq/DWanyJvBmjbQHgrZfpqplYvAxHwEDw1yzCNdSMjlEIuprVlICbdq4IayMMcb4K9kNQFJT\nBH3NrFm+McYkjgWzhoigZFZcbMHMGGMSxYJZQwRKZqphD7GSmTHGJI4Fs4bIy4MjR2D79pC7y8td\nn2oLZsYYkxgWzBqinr5mW7e6QUIsmBljTGJYMGuIevqaWbN8Y4xJLAtmDVFPycyCmTHGJJYFs4bo\n1MlNH23BzBhjGgULZg2Rnu5m26yjmjEzEzp2THC+jDEmkURGI7IakUJEak3AHHTclYgoIkP8yooF\ns4aqo69ZSYmLdWn27hpjjlcitSZgRqTmBMwg0hKYCnziZ3bs67ah6hgFxPqYGWOagGFAIapFqIab\ngBngQeC3QJmfmbFgVofNm+G888LMw1lPycyCmTHmOFf/BMwiZwLdUH3D78xYMKvDgw/CBx/AtGkh\ndublwb59bjrpGiyYGWOOBx0gA5H8oGVyxCeLpAGPAHf4lsEgvo6an6pycqAsqEA8Y4ZbsrPh0CEv\nMbivWatWx47duxf277dgZoxJfduhHNVwjTbqm4C5JTAAWIQIwInAHEQuRTU/3nm1klkIRUUwYQLk\n5rrHubkwcSKsXx90UJi+ZtYs3xjTRCwBeiPSE5HaEzCr7kG1A6o9UO0BLAZ8CWRgwSykzp1dYaus\nzJXGysrc4xNPDDrIgpkxpilTrTUBM6orEJmGyKV1nxx/Vs0YxtatMGUKTJ4MM2e6xiDVdOni1jVa\nNFowM8Y0Gaq1JmBG9YEwx470MysWzMKYPbtqe/r0EAc0a+Z6RVvJzBhjks7XakYRRouwWoRCEWr1\nDhehuwgLRFgmwiIR8rz0QSJ8LMIKb9/VQec8I8J6EQq8ZZCfr6FOIfqalZRA27auEYkxxpjE8C2Y\niVCrd7gINXuHPww8q8rpwDTgN176QeCHqvQHRgOPitAm6Ly7VBnkLQV+vYZ6hehrZs3yjTEm8fws\nmQ0DClUpUiVc7/B+wDve9sLAflXWqLLW294EbAMa30iHFsyMMaZR8DOY1d87HL4ArvC2xwEtRWgf\nfIAIw4AsYF1Q8q+96sc/itAs1JOLMFmEfBHyy8tjeRl16NrVzTYd1CnNgpkxxiRespvm3wmcJ8Ln\nwHm4DncVgZ0idAaeA25UpdJLvhc4DRgKtAPuDnVhVWaqMkSVIRl+NXMJNM/ftAmA8nLXCtKCmTHG\nJJafway+3uGoskmVK1QZDNznpe0GEKEV8AZwnyqLg87ZrIqqchj4K646Mzlq9DXbsgUqKy2YGWNM\novkZzJYAvUXoKULt3uGACB1EjuXhXmCWl54FvIprHPJyjXM6e2sBLgeW+/ga6hY8pBXWLN8YY5LF\nt2CmSq3e4aqsEGGaCIHe4SOB1SKsAToBv/bSxwPnApNCNMH/hwhfAl8CHYD/49drqFeNkpkFM2OM\nSQ5fO02rUqt3uCoPBG2/DNVLXl7634G/h7nmd+OczYZr2dItFsyMMSapkt0AJPXl5VWrZszMhA4d\nkpwnY4xpYiyYxSqor1lJiRuyMc3eVWOMSSj72o1V0JBW1sfMGGOSw4JZrPLy3JD65eUWzIwxJkks\nmMUqLw8qKtAtWy2YGWNMklgwi5UXvQ6sKeHAAQtmxhiTDBbMYuX1Ndv1pWsEYsHMGGMSz4JZrAIl\ns9UWzIwxJlksmMWqQwfIyuLoBtei0YKZMcYkngWzWIlAXh5S4kpmXbokOT/GGNMEWTCLh65dydpe\nQrt2kJOT7MwYY0wKE5mKSCtEBJG/ILIUkQvrOy2iYCbCOBFaBz1uI8LlseT3uJKXR8vdxVbFaIwx\nsfsvVPcCFwJtgeuBh+o7KdKS2S9U2RN44M059ouG5PK4lJdHu0PFdO2iyc6JMcakOvHWFwPPoboi\nKC2sSINZqON8HXE/pXTtSjM9zKkddiY7J8YYk+o+Q2QeLpjNRaQlUFnfSZEGpHwRHgGme49vAz5r\nUDaPQ+Un5pEB9GleDLRPdnaMMSaV3QQMAopQPYhIO+DG+k6KtGT2E+AI8CLwT6AMF9AMsCPb3Szr\nmVWc5JwYY0wCiYxGZDUihYjcE2L/FES+RKQAkQ8Q6RfBVc8BVqO6G5HrgPuh6jZXOBEFM1UOqHKP\nKkNUGarK/6vKgUjObQpKxI0C0pWSJOfEGGMSRCQdV1s3BugHXBsiWD2P6kBUBwG/Ax6J4MozgIOI\nnAHcAawDnq3vpEhbM74tQpugx21FmBvBeaNFWC1CoQi1orYI3UVYIMIyERaJkBe07wYR1nrLDUHp\nZ4nwpXfNx0XqvzHotw1lJ1JBGicctpKZMabJGAYUolqE6hFcrd1l1Y5wrRIDmgORtJIrR1W9a/0J\n1elAy/pOirSasYPXgtHLH7uAE+o6QYRaUVuEmlH7YeBZVU4HpgG/8c5th2stORz3hv1ChLbeOTOA\nm4He3jI6wtfgm42bM9jCibQ+YCUzY0yT0RXYGPS42EurTuQ2RNbhSmY/jeC6+xC5F9ck/w1E0oDM\n+k6KNJhVinBSVd7oQf0RdhhQqEqRKqGjtgty73jbC4P2XwS8rcpOL3C+DYwWoTPQSpXFqiiu6Jn0\n/m4lJa6qMXu7lcyMMcePDpCBSH7QMjnqi6hOR/UU4G7c/a/6XA0cxvU32wLkAb+v76RIWzPeB3wg\nwru49v7fAep7UaGi9vAax3wBXAE8BowDWorQPsy5Xb2lOER6UpWUwM6cPKR4VbKzYowxcbPdVfkN\nCbO7BOg/omQmAAAgAElEQVQW9DjPSwvnn7iatbqpbkHkH8BQRMYCn6Ian3tmqvwHGAKsBl7A3ZQ7\nFMm59bgTOE+Ez4HzcG9ERRyuiwiTRcgXIb+8PB5XDK+kBPa37uo2jDGmaVgC9EakJyJZwDXAnGpH\niPQOenQJsLbeq4qMBz4FfgCMBz5B5Kr6TouoZCbCj4CpuMhbAJwNfAx8t47T6o3aqmzClcwQoQVw\npSq7RSgBRtY4d5F3fl6N9JARRJWZwEyA5s0juunYYCUlcLhjHizbA/v2Qct671UaY0xqUy1H5HZg\nLpAOzEJ1BSLTgHxU5wC3I/I94CiwC6oa89XhPmAoqtsAEOkIzAderuukSO+ZTQWGAl+rcj4wGKoa\nhISxBOgtQk8RQkZtETqIHMvDvcAsb3sucKHXarItboyuuapsBvaKcLbXivGHwOsRvgZfqHoFssDA\njFY6M8Y0Fapvonoqqqeg+msv7QEvkIHqVFT7ozoI1fO9oanqk3YskDk7iCBWRRrMylQpAxChmSqr\ngD51naBKORCI2l8BL6myQoRpIlzqHTYSWC3CGqAT8Gvv3J3Ag7iAuASY5qUB3Ao8DRTi+h+8FeFr\n8MXu3XDoEGT29AqMFsyMMSYW/0FkLiKTEJkEvAG8Wd9JkTYAKfb6mb0GvC3CLuDr+k5S5c2amVDl\ngaDtlwlTdFRlFlUlteD0fGBAhPn2XSB25Z7qBbNia9FojDENpnoXIlcCI7yUmai+Wt9pEQUzVcZ5\nm78UYSHQGvhPgzJ6nAkEs3YDulRPMMYY0zCqrwCvRHNK1CPfq/JutOcczwKxq/PJOdC+vZXMjDGm\nIUT2Ebr/sgCKaqu6TrdpXGIUCGZdugB5eRbMjDGmIVRjagYeaQMQE0ZJCXToAM2a4Vo0WjWjMcYk\nnAWzGJWUVLXKt5KZMcYkhwWzGNUKZtu2weHDSc2TMcY0NRbMYlQtmAU2Nm9OWn6MMaYpsmAWgyNH\nXEGsWskMrKrRGGMSzIJZDAIFsFolMwtmxhiTUBbMYhBouFirZGYtGo0xJqEsmMWgVjBr1QpatLCS\nmTHGJJgFsxjUCmYi1tfMGGOSwIJZDEpKXGfpdu2CEq2vmTHGJJwFsxgEmuWLBCVaMDPGmISzYBaD\nan3MArp2dc0cKyqSkidjjGmKLJjFIGQwy8uD8nLXAc0YY0xCWDBrINU6SmZgVY3GGJNAFswaaNcu\nKCsLUzIDa9FojDEJ5GswE2G0CKtFKBThnhD7TxJhoQifi7BMhIu99IkiFAQtlSIM8vYt8q4Z2HeC\nby9g82Y47zzYsqXWrlrN8gNsSCtjjEk434KZCOnAdGAM0A+4VoR+NQ67H3hJlcHANcCfAVT5hyqD\nVBkEXA+sV6Ug6LyJgf2q+Hdz6sEH4YMPYNq0WrvCBrMOHSAz00pmxhiTQH6WzIYBhaoUqXIE+Cdw\nWY1jFAhMhd0a2BTiOtd65yZOTo5rbz9jBlRWurWIS/eEDWZpaS7RSmbGGJMwfgazrsDGoMfFXlqw\nXwLXiVAMvAn8JMR1rgZeqJH2V6+K8eciSIhzEGGyCPki5JeXR5nzoiKYMAFyc93j3FyYOBHWrz92\nSCCYdekS4nzra2aMMQmV7AYg1wLPqJIHXAw8J1KVJxGGAwdVWR50zkRVBgLf8ZbrQ11YlZmqDFFl\nSEZGlLnq3NmNs1hWBtnZbt2qFZx44rFDSkqgY0fIygpxvg1pZYwxCeVnMCsBugU9zvPSgt0EvASg\nysdANtAhaP811CiVqbprqLIPeB5XnRl/W7fClCmweLFb12gEErJZfkCgZKbqS9aMMaZREBmNyGpE\nChGp1cgPkf9BZCUiyxBZgEh3v7ISbZklGkuA3iL0xAWxa4AJNY75BrgAeEaEvrhgVgrgldDG40pf\neGkZQBtVtouQCYwF5vuS+9mzq7anT6+1u85g1rUrHDrk2u9XG7jRGGOOEyKBRn6jcLeRliAyB9WV\nQUd9DgxB9SAitwC/w906ijvfSmaqlAO3A3OBr3CtFleIME2ES73D7gBuFuELXAlskiqB4sy5wEZV\nioIu2wyYK8IyoAAXJJ/y6zXUpd6SWeAgY4w5Pg0DClEtQjV0Iz/Vhage9B4txtXQ+cLPkhmqvIlr\n2BGc9kDQ9kpgRJhzFwFn10g7AJwV94xG6fBhKC2NIJgVF8PAgQnLlzHGxFMHyEAkPyhpJqozve1Q\njfyG13G5m4C34pzFY3wNZserzZvdus5qRrCSmTEmpW2HclSHxHwhkeuAIcB5MV8rDAtmDRC2j1lA\n586uX5o1zzfGHL8iaeQHIt8D7gPOQ/WwX5lJdtP8lFRvMMvMdM34LZgZY45fS4DeiPREJAvXyG9O\ntSNEBgNPApei6utUIhbMGqDeYBbYadWMxpjjlWqtRn6orkBkGiKBRn6/B1oA/0KkAJE5Ya4WM6tm\nbICSEteXum3bOg7Ky4PCwoTlyRhjEk61ViM/VB8I2v5eorJiJbMGCDTLl5ADaXlsfEZjjEkYC2YN\nUGcfs4C8PNi9Gw4cSEiejDGmKbNg1gARB7PAwcYYY3xlwSxKqhEGs8ABVtVojDG+s2AWpR073Agg\nVjIzxpjGw4JZlCJqlh98gJXMjDHGdxbMohRxMMvNdW33rWRmjDG+s2AWpYiDGdiM08YYkyAWzKIU\nCGadO0dwcPfu8MknsHFj/ccaY4xpMAtmUSopgRNOgKysCA7++c/h4EE491xYv973vBljTFNlwSxK\nETXLDxg2DBYsgD17XEBbu9bXvBljTFNlwSxKJSVVre7rtXkz3HEH/Otfrj3/uefCypX1n2eMMSYq\nvgYzEUaLsFqEQhHuCbH/JBEWivC5CMtEuNhL7yHCIREKvOWJoHPOEuFL75qPi1DXCIlxF1XJ7MEH\n4YMP4JVXYNEil3beefDFF35lzxhjmiTfgpkI6cB0YAzQD7hWhH41DrsfeEmVwbi5cP4ctG+dKoO8\nZUpQ+gzgZqC3t4z26zXUVFbmOk3XG8xyctwoxDNmQGWlW/fvD7t2ueH2zz8f8vPruYgxxphI+Vky\nGwYUqlKkyhHgn8BlNY5RoJW33RrYVNcFRegMtFJlsSoKPAtcHt9sh7fJy129wayoCCZMcH3NwK0n\nToQNG+C996B1a7jgAvj4Yz+za4wxTYafwawrENwmvdhLC/ZL4DoRinFz4vwkaF9Pr/rxXRG+E3TN\n4I5boa7pm4j7mHXuDK1auaJcdrZbt2rlZp/u2dMFtBNOgFGj4N13fc+3McYc75LdAORa4BlV8oCL\ngedESAM2Ayd51Y//AzwvcqwEFxERJouQL0J+eXl8MhtVh+mtW2HKFFi82K23bKna162bC2gnnQRj\nxsC8efHJoDHGNFF+zjRdAnQLepznpQW7Ce+elyofi5ANdFBlG3DYS/9MhHXAqd75wW0JQ10T77yZ\nwEyA5s3RmF8NUQaz2bOrtqdPr72/c2fXKGTUKPj+910jkbFj45FNY4xpcvwsmS0BeovQU4QsXAOP\nOTWO+Qa4AECEvkA2UCpCR68BCSKcjGvoUaTKZmCvCGd7rRh/CLzu42uopqTE3f5q3TpOFzzhBFi4\nEAYOhCuuqB4AjTHGRMy3YKZKOXA7MBf4CtdqcYUI00S41DvsDuBmEb4AXgAmeQ07zgWWiVAAvAxM\nUWWnd86twNNAIbAOeMuv11BToFm+xLMzQLt2rmP1kCEwfjy88EIcL26MMU2DqMalBq5Ra968uR44\ncCDm63z725CZ6QpTcbdvn6tmfP99mDULJk1yna6vuQZefNE1HjHGmAQSkYOq2jzZ+YhEshuApJSo\nOkxHq2VLeOst12T/xhvhySerOl1Pm+bTkxpTh82bXSf/4MZLxjRSVjKLkKprZf+zn8FvfxunjIVS\nVgbNm7vO1jVlZ8OhQz4+uTFBbr3V/aj68Y/hz3+u/3hz3LGS2XFo+3Y4csTHkllAdrbrdB08AGRO\njut0bSPvm0QINYKNiEs3ppGyYBahqJrlx6p7d7j44qrHhw65OdHat0/Ak5vjRkOrCcONYJNqP6as\nmtR/IqMRWY1IISK1xt9F5FxEliJSjshVfmbFglmEEhrMAEpLXTXP009Dp06uk3XfvvD886GrIE38\npfqXYUPvudY1gk0qifWec7L//sl+/vqI1Bp/F5Ga4+9+A0wCnvc9P6p63C+5ubkaqyefVAXVb76J\n+VLRq6xU/fe/VU8/3WVi4EDVOXNcuvHPLbeopqW5dSrJznafk5pLdnbk1xg3TvXWW1ULCtx63Dj/\n8htv8Xj9qsn/+8f6/Js2qZ57rurmzQ3OAnBAw323wjkKc4Me36twb5hjn1G4Kuy14rAkPdAkYolH\nMHvgAVUR1SNHYr5Uw1VUqD7/vOopp7g/3TnnqC5c6N/zxeGfISXF68swWe/fpk2qEyao5ua6fOfm\nqk6cmJx8pOLrT/bfvxEF43qC2VUKTwc9vl7hT2GO9T2Y+TmcVaN29OhRiouLKSsri+j4UaNcP7PC\nQp8zVp9Bg2D2bNq8+iodZswg8/zz2T9iBKVTp1I2YEC1QzNKS+lyxx2U/OEPVHTsGPVTdZo2jbbv\nv8+u//5vtj7wQLV92dnZ5OXlkZmZGdPL8VVD++kVFcGdd8Jrr8HBg+6e0bhx8PDD0T1/cDVXIlsD\nNpZqwlR9/cn++8f6/Dk57jUHzJjhlga0hu4AGYgEz1c1E9WZUV0kUfyMlI1lCVUyKyoq0tLSUq2M\nsKpu9WrVFSsiOjRxDh5Uffhh1fbt3S+3K69UXbmyan9Df5nV88uwsrJSS0tLtaioqO7rJLtkF8sv\n0ylT3LnZ2dFfI16/rGORzGrCZJdsVGN//cn++8fy/HEsmWPVjI1rCRXMVq5cGXEgU1Vdvlx17dqI\nD0+sPXtUf/EL1RYt3Ac/PT22f6YI/hkqKyt1ZXDgDCVZdf7JvmfUWKr5kiVerz+Z96yS/fdPZjAO\nUk8wy1AoUuipkKXwhUL/MMdaMIvHEi6YRePzz1U3bIjqlMTbtk31f/5HNSvL3eDLyGj4P1ME/wxh\n38Nk1/k3hmASpy+TlJXskk2yJfvvH6eSeZ3BzAWpixXWKKxTuM9Lm6Zwqbc9VKFY4YDCDoUVdV4v\nhsWa5kegshLKyyErK9k5qUfHjvCHP8C6dXDaaS7T4Ordi4shI4pbpHXNx1afWPspxdpptzHcM4rl\n/Qto7E2z65LMz09jEI+/fyxmz3ZTT51xhlv7NSOH6puonorqKaj+2kt7ANU53vYSVPNQbY5qe1T7\n+5MRrGQWiUOHVJcsUS0tjfiUeu3atUunT58e9XljxozRXbt21X/guHGuNPKjH6m2auV+2aanq154\noerTT6vu2NGAXFdX53uY7Dr/VG5aHpDspuHJlOySjVHVCEpmjWhpsmMzfvXVV/Tt2zei8/ftg9Wr\n4dRT3Q/8eNiwYQNjx45l+fLl1dLLy8vJiKYEFQlVKCiAf/3LtewrKnKltO99z007c/nl0LZt1Jet\n8z284gpXQpo8GWbOdKWMaH4d3nKLOy8ry40j1pTGB6zZGi2gKY3NGevnx8RFKo3NaMEMN3hwQUH4\n848erRr/Ny3CitlBg+DRR8Pvv+aaa3j99dfp06cPmZmZZGdn07ZtW1atWsWaNWu4/PLL2bhxI2Vl\nZUydOpXJkycD0KNHD/Lz89m/fz9jxozh29/+Nh999BFdu3bl9ddfJydMVdxTTz3FzJkzOXL4MKM6\ndOA3gweT+eqrsH49R0X4uEUL5rVuzfeffprhF13Es88+y8MPP4yIcPrpp/Pcc8/VumY0Pwii1pS/\nzDZvDt80O9VG4TApLZWCWZPtZxaNQLyP56ScDz30EMuXL6egoIBFixZxySWXsHz5cnr27AnArFmz\naNeuHYcOHWLo0KFceeWVtK8xNuPatWt54YUXeOqppxg/fjyvvPIK1113Xcjnu+KKK7j55psBuP/+\n+3miUyd+sm4d9154IVdWVHBuURHnfv01+v3vs+9b3+LLVatY+NFHtD/5ZHbu3Bnymr4KDlzTpyf+\n+ZOpMdzzMybFWDCj7hIUuDF+S0th8OA4zzIdZNiwYccCGcDjjz/Oq6++6j3/RtauXVsrmPXs2ZNB\ngwYBcNZZZ7Fhw4aw11++fDn3338/u3fvZv/+/Vx00UUgwtMFBfyyuNhV5y1Zgrz0Esyaxe937XJj\nQX7ve7QbNcr1Gu/Xz783wFQXaEAQXDI1xoRlwSwCR46473o/v8ebN68qyS9atIj58+fz8ccfk5ub\ny8iRI0OOVNKsWbNj2+np6Ryq437KpEmTeO211zjjjDN45plnWLRoUfUDRGDYMBg2jGe6dSPz88+Z\n0rYtvPEGvPmmO6ZzZ3efLbAY/zTlkqkxDeBr03wRRouwWoRCEWpNDyDCSSIsFOFzEZaJcLGXPkqE\nz0T40lt/N+icRd41C7zlBD9fA7h7ZvEetally5bs27cv5L49e/bQtm1bcnNzWbVqFYsXL475+fbt\n20fnzp05evQo//jHP46lX3DBBcyYMQOAiooK9uzZw3e/9z0e+egjdtx/P6xZw+6CAjd6/7nnutmw\nb7gBunbl5O9/391wfOMN10rGGGOSxLeSmQiB6QFGAcXAEhHmqLIy6LD7gZdUmSFCP+BNoAewHfi+\nKptEGADMBYInX5moSvB4Yb46cgRatIjvNdu3b8+IESMYMGAAOTk5dOrU6di+0aNH88QTT9C3b1/6\n9OnD2WefHfPzPfjggwwfPpyOHTsyfPjwY4H0scceY/LkyfzlL38hPT2dGTNmcM4553Dfffdx3nnn\nkZ6ezuDBg3nmmWfgpptcv68vvoD58zn62ms0e/JJeOwx1zrynHNciW3UKBg6NLp+bcYYEwPfWjOK\ncA7wS1Uu8h7fC6DKb4KOeRIoUuW33vF/UOVbNa4jwA6gsyqHRVgE3BlNMIulab4qLF3qphQLnvzZ\neO9hz57w4Yfw9tswf757s1Rdg4Xzz4eRI+Gss9wNx3j/IjDG+MpaMzpdgY1Bj4uB4TWO+SUwT4Sf\nAM2BUDdirgSWqnI4KO2vIlQArwD/RxXf+heUl7vv5sY8OHxSZWfDBRe4BWD7dnjnHRfY3n4bXn/d\npYtAnz5w5pkuuJ15pgtwrVsnL+/GmONGsuuBrgWeUeUPXsnsOREGqFIJIEJ/4LfAhUHnTFSlRISW\nuGB2PfBszQuLMBmYDLENQ3X0qFs3+qGsPLfddhsffvhhtbSpU6dy4403JiYDHTq4jtjjx7vHmza5\n0tpnn7n1u++62bIDevWqCm6BdQM6cBtjmjY/g1kJ0C3ocZ6XFuwmYDSAKh+LkA10ALaJkAe8CvxQ\nlXWBE1TdNVTZJ8LzwDBCBDNVZgIzAZo3b3jJ7cgRt06Vktn0xtbyrUsXt4wdW5W2dasLbIEgt3ix\nG5kkoGfP6qW3gQPdNaxbgDEmDD+D2RKgtwg9cUHsGmBCjWO+AS4AnhGhL5ANlIrQBngDuEeVY8UM\nETKANqpsFyETGAvM9/E1pFzJLCV06gRjxrglYPt2+PzzqhLcZ5/Byy9X7W/dGgYMqL106JD4/Btj\nGh3fgpkq5SLcjmuJmA7MUmWFCNOAfFXmAHcAT4nw34ACk1RR77xewAMiBKY4vhA4AMz1Alk6LpA9\n5ddrgNQrmaWsDh1cK8hRo6rSdu2CZctgxQpYvtwtL74ITz5ZdUynTtC/f/UA179//AbRNMakBBub\nsR4bNsCePW4mBVOdr2MzhqPqRsMIBLfAsnIlBP+NTzrJBba+faF3b7f06uWapEY6wKYxTZy1ZjyO\nHDniT6ls9+7dPP/889x6661Rn/voo48yefJkcgPzPTUlIlX34S4MahdUWQlff107yC1YAIeDGsI2\nawannFIV3ILXFuiMSVlWMqvHihXu+69XL1cguOYaV9MV65iv4aaAiURg5PwOSb5flJSSWbQqKqCk\nBNauhcJCtwRvhwp0wUGue3do1w7atHGtLNu2tTpn02RYyew4Ejz6x4MPwgcfwLRpsU+tdc8997Bu\n3ToGDRrEqFGjOOGEE3jppZc4fPgw48aN41e/+hUHDhxg/PjxFBcXU1FRwc9//nO2bt3Kpk2bOP/8\n8+nQoQMLFy4Mef1bbrmFJUuWcOjQIa666ip+9atfAbBkyRKmTp3KgQMHaNasGQsWLCA3N5e7776b\n//znP6SlpXHzzTfzk5/8JLYX2Fikp7sqx5NOquoLF1BZWT3QBQe5efNCzykGbi6g4ODWtm34x+3b\nu18+J55YNXOyMSbuLJjVoaLCLX36VP8BP2OGW2KZKzF4Cph58+bx8ssv8+mnn6KqXHrppbz33nuU\nlpbSpUsX3njjDcCN2di6dWseeeQRFi5cWGfJ7Ne//jXt2rWjoqKCCy64gGXLlnHaaadx9dVX8+KL\nLzJ06FD27t1LTk4OM2fOZMOGDRQUFJCRkZGcKV+SIS0NunVzy3e/W31fZaXrI7dxo2uIElh27679\n+Ouv3YR4u3bVPUZly5YuqHXuXBXgQi0nnOCCsDEmYhbM6hBolp+fD7/5Tei5EuNh3rx5zJs3j8GD\nBwOwf/9+1q5dy3e+8x3uuOMO7r77bsaOHct3vvOdiK/50ksvMXPmTMrLy9m8eTMrV65EROjcuTND\nhw4FoJXX4m/+/PlMmTLl2AzX7dq1i88LS2Vpae4eWrRjmJWXuxZDgWC3fbvrV7dlS9WyebMLflu2\nwN69oZ+7Y8eqwBcIuN26uRJmt24uX2EmYjWmKbJgVodAMDvpJH/nSlRV7r33Xn784x/X2rd06VLe\nfPNN7r//fi644AIeeOCBEFeobv369Tz88MMsWbKEtm3bMmnSpJBTyBgfZGS4qsUac8+FdfBg7WAX\nCHhbtlSNoLJtW+1zO3YMHegCS5cuNtizaTLsk16H4D5m8Z4rMXgKmIsuuoif//znTJw4kRYtWlBS\nUkJmZibl5eW0a9eO6667jjZt2vD0009XOzdcNePevXtp3rw5rVu3ZuvWrbz11luMHDmSPn36sHnz\nZpYsWcLQoUPZt28fOTk5jBo1iieffJLzzz//WDWjlc4SJDfXjXgSNDFrSGVlUFzsqj03boRvvqna\nLiyEhQtrl/ICJbxmzVyv/8BS83G4tEB6bm7VkpMT2WNrJGMSzIJZHYJH/4j3XInBU8CMGTOGCRMm\ncM455wDQokUL/v73v1NYWMhdd91FWloamZmZx+Ydmzx5MqNHj6ZLly4hG4CcccYZDB48mNNOO41u\n3boxYsQI73Vk8eKLL/KTn/yEQ4cOkZOTw/z58/nRj37EmjVrOP3008nMzOTmm2/m9ttvj/1FmvjJ\nznatK3v1Cn/M3r21A922be5X2ZEj7sZvYPvIERcg9+6tnhbqmMrK6PObkeECW3a2u/8XWNLSqq/D\nbQfWmZmuGqR169pLuPTcXBv6rAmypvl1+OYbd8vjzDP9zF3qSomm+SY2qu5X3cGDrrXTwYPVl/rS\nysqqWlJVVtbeDpUWvH3kiAu4e/a4Zd8+l6e6pKdXD3SB7eB1fWmtWrlSabzew/Lyqh8KNdeh0gLr\n8nL3gyA7u+rHQbh1dnbcg7g1zT9OHD1qYzKaJk6kqsqxTZtk58YFuX37qge4mkvNfXv3uiravXur\n9gWqXerSrJnrhhGgWrVE+riysup+RSI0a1Y7yP37367/5HHOglkdcnPj9+PML8OHD+dwcL8B4Lnn\nnmPgwIFJypExPkpLqypxdetW//GhqLqSTyDQBQJc8DqwHajREakq9QS2g5dQ6YH8Bt+TDL43GWq7\nZlp6usvroUOulFvfOlSan61eRUYDj+HGyn0a1Ydq7G+Gm9XkLNwky1ejusGPrFgwq0PnzsnOQf0+\n+eSTZGfBmNQiUlUt16lTsnOTukTSgenAKNzky0sQmYPqyqCjbgJ2odoLkWtw81Ne7Ud2bCA6Y4wx\nDTEMKES1CNUjwD+By2occxnwN2/7ZeACxJ/WOU06mDWFxi9+sffOmCavK7Ax6HGxlxb6GNVyYA8Q\nYSfM6DTZYJadnc2OHTvsS7kBVJUdO3aQnZ2d7KwYY3zUATIQyQ9aJic7T+E02XtmeXl5FBcXU1pa\nmuyspKTs7Gzyoh3qyRiTUrZDOapDwuwuAYJb4eR5aaGOKUYkA2iNawgSd002mGVmZtKzvlEXjDHG\nhLME6I1IT1zQugaYUOOYOcANwMfAVcA7flWHNdlgZowxJgaq5YjcDszFNc2fheoKRKYB+ajOAf4C\nPIdIIbATF/B80WRHADHGGFO3VBoBpMk2ADHGGHP8aBIlMxGpBBo4jSYZQHkcsxNvlr/YWP5iY/mL\nTWPPX46qpkShp0kEs1iISL6Gb82TdJa/2Fj+YmP5i01jz18qSYmIa4wxxtTFgpkxxpiUZ8GsfjOT\nnYF6WP5iY/mLjeUvNo09fynD7pkZY4xJeVYyM8YYk/IsmHlEZLSIrBaRQhG5J8T+ZiLyorf/ExHp\nkcC8dRORhSKyUkRWiMjUEMeMFJE9IlLgLQ8kKn/e828QkS+9584PsV9E5HHv/VsmImcmMG99gt6X\nAhHZKyI/q3FMQt8/EZklIttEZHlQWjsReVtE1nrrtmHOvcE7Zq2I3JDA/P1eRFZ5f79XRSTk1NP1\nfRZ8zN8vRaQk6G94cZhz6/xf9zF/LwblbYOIFIQ51/f377ikqk1+wQ3Fsg44GcgCvgD61TjmVuAJ\nb/sa4MUE5q8zcKa33RJYEyJ/I4H/TeJ7uAHoUMf+i4G3AAHOBj5J4t96C9A9me8fcC5wJrA8KO13\nwD3e9j3Ab0Oc1w4o8tZtve22CcrfhUCGt/3bUPmL5LPgY/5+CdwZwd+/zv91v/JXY/8fgAeS9f4d\nj4uVzJxhQKGqFmkUk8yJT5PM1aSqm1V1qbe9D/iK2vMGNXaXAc+qsxhoIyLJmMv7AmCdqn6dhOc+\nRlXfw41VFyz4M/Y34PIQp14EvK2qO1V1F/A2MDoR+VPVeermpAJYjBslPSnCvH+RiOR/PWZ15c/7\n3jvBeywAAASOSURBVBgPvBDv523KLJg5UU0ypz5PMlcXr3pzMPBJiN3niMgXIvKWiPRPaMZAgXki\n8pmEnvMokvc4Ea4h/JdIMt8/gE6qutnb3gJ0CnFMY3kf/wtX0g6lvs+Cn273qkFnhammbQzv33eA\nraq6Nsz+ZL5/KcuCWQoRkRbAK8DPVHVvjd1LcVVnZwD/F3gtwdn7tqqeCYwBbhORcxP8/PUSkSzg\nUuBfIXYn+/2rRl19U6Nsaiwi9+GGYPpHmEOS9VmYAZwCDAI246ryGqNrqbtU1uj/lxojC2ZONJPM\nIT5PMheKiGTiAtk/VHV2zf2quldV93vbbwKZItIhUflT1RJvvQ14FVedEyyS99hvY4Clqrq15o5k\nv3+erYGqV2+9LcQxSX0fRWQSMBaY6AXcWiL4LPhCVbeqaoWqVgJPhXneZL9/GcAVwIvhjknW+5fq\nLJg5S4DeItLT+/V+DW5SuWCBSebAm2Qu3D9zvHl17H8BvlLVR8Icc2LgHp6IDMP9bRMSbEWkuYi0\nDGzjGgosr3HYHOCHXqvGs4E9QVVqiRL2F3Ey378gwZ+xG4DXQxwzF7hQRNp61WgXemm+E5HRwP8D\nXKqqB8McE8lnwa/8Bd+DHRfmeSP5X/fT94BVqlocamcy37+Ul+wWKI1lwbW2W4Nr6XSflzYN948L\nkI2rnioEPgVOTmDevo2rcloGFHjLxcAUYIp3zO3AClzrrMXAtxKYv5O95/3Cy0Pg/QvOnwDTvff3\nS2BIgv++zXHBqXVQWtLeP1xQ3Qwcxd23uQl3D3YBsBaYD7Tzjh0CPB107n95n8NC4MYE5q8Qd78p\n8BkMtO7tArxZ12chQfl7zvtsLcMFqM418+c9rvW/noj8eenPBD5zQccm/P07HhcbAcQYY0zKs2pG\nY4wxKc+CmTHGmJRnwcwYY0zKs2BmjDEm5VkwM8YYk/IsmBnTyHkj+v9vsvNhTGNmwcwYY0zKs2Bm\nTJyIyHUi8qk3D9WTIpIuIvtF5I/i5qFbICIdvWMHicjioLnB2nrpvURkvjfg8VIROcW7fAsRedmb\nT+wfiZqxwZhUYcHMmDgQkb7A1cAIVR0EVAATcSOP5Ktqf+Bd4BfeKc8Cd6vq6bhRKwLp/wCmqxvw\n+Fu4USTAzZTwM6AfbpSIEb6/KGNSSEayM2DMceIC4CxgiVdoysENFFxJ1aCyfwdmi0hroI2qvuul\n/w34lzcmX1dVfRVAVcsAvOt9qt54ft4MxT2AD/x/WcakBgtmxsSHAH9T1XurJYr8vMZxDR0/7nDQ\ndgX2v2tMNVbNaEx8LACuEpETAESknYh0x/2PXeUdMwH4QFX3ALtE5Dte+vXAu+pmES8Wkcu9azQT\nkdyEvgpjUpT9ujMmDlR1pYjcj5shOA03WvptwAFg2P/f3h3bMAwCUQD912c9ly6ySqawl/MAqdOT\nwqzgWBe9V4KEoPq6Q4I59855r5acX7xsM6yOJM85vibZq+o111h+eAxoy6v5cKGq+owxHnfvA/6d\nNiMA7anMAGhPZQZAe8IMgPaEGQDtCTMA2hNmALQnzABo7wvd3RHrW3PjVwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fa754749910>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model.compile(loss='categorical_crossentropy',\n",
    "              optimizer=optim,\n",
    "              metrics=['accuracy'])\n",
    "t = time.time()\n",
    "history = model.fit(X_train, Y_train,\n",
    "                batch_size=batch_size, nb_epoch=nb_epoch,\n",
    "                verbose=2,validation_data=(X_test, Y_test))\n",
    "print (time.time() - t, \"seconds.\")\n",
    "\n",
    "score = model.evaluate(X_test, Y_test, verbose=0)\n",
    "print (\"-\"*10)\n",
    "print (\"Loss: %f\"%(score[0]))\n",
    "print (\"Accuracy: %f\"%(score[1]))\n",
    "plot_curves(history,nb_epoch)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Turns out that adding more layers did not do much for us. Accuracy is pretty much the same as the one we obtained with a single hidden layer, and now our model is overfitting to training data. How can we fix this?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Dropout"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "**Exercise**: Add dropout layer(s) to the model and see how their effect in the training curves & accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "____________________________________________________________________________________________________\n",
      "Layer (type)                     Output Shape          Param #     Connected to                     \n",
      "====================================================================================================\n",
      "dense_11 (Dense)                 (None, 128)           100480      dense_input_4[0][0]              \n",
      "____________________________________________________________________________________________________\n",
      "activation_11 (Activation)       (None, 128)           0           dense_11[0][0]                   \n",
      "____________________________________________________________________________________________________\n",
      "dense_12 (Dense)                 (None, 128)           16512       activation_11[0][0]              \n",
      "____________________________________________________________________________________________________\n",
      "activation_12 (Activation)       (None, 128)           0           dense_12[0][0]                   \n",
      "____________________________________________________________________________________________________\n",
      "dropout_1 (Dropout)              (None, 128)           0           activation_12[0][0]              \n",
      "____________________________________________________________________________________________________\n",
      "dense_13 (Dense)                 (None, 128)           16512       dropout_1[0][0]                  \n",
      "____________________________________________________________________________________________________\n",
      "activation_13 (Activation)       (None, 128)           0           dense_13[0][0]                   \n",
      "____________________________________________________________________________________________________\n",
      "dense_14 (Dense)                 (None, 128)           16512       activation_13[0][0]              \n",
      "____________________________________________________________________________________________________\n",
      "activation_14 (Activation)       (None, 128)           0           dense_14[0][0]                   \n",
      "____________________________________________________________________________________________________\n",
      "dropout_2 (Dropout)              (None, 128)           0           activation_14[0][0]              \n",
      "____________________________________________________________________________________________________\n",
      "dense_15 (Dense)                 (None, 128)           16512       dropout_2[0][0]                  \n",
      "____________________________________________________________________________________________________\n",
      "activation_15 (Activation)       (None, 128)           0           dense_15[0][0]                   \n",
      "____________________________________________________________________________________________________\n",
      "dense_16 (Dense)                 (None, 128)           16512       activation_15[0][0]              \n",
      "____________________________________________________________________________________________________\n",
      "activation_16 (Activation)       (None, 128)           0           dense_16[0][0]                   \n",
      "____________________________________________________________________________________________________\n",
      "dropout_3 (Dropout)              (None, 128)           0           activation_16[0][0]              \n",
      "____________________________________________________________________________________________________\n",
      "dense_17 (Dense)                 (None, 10)            1290        dropout_3[0][0]                  \n",
      "____________________________________________________________________________________________________\n",
      "activation_17 (Activation)       (None, 10)            0           dense_17[0][0]                   \n",
      "====================================================================================================\n",
      "Total params: 184,330\n",
      "Trainable params: 184,330\n",
      "Non-trainable params: 0\n",
      "____________________________________________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "np.random.seed(SEED)\n",
    "\n",
    "from keras.layers import Dropout\n",
    "\n",
    "dratio = 0.2\n",
    "H_DIM = 128\n",
    "\n",
    "model = Sequential()\n",
    "model.add(Dense(128,input_shape=(784,)))\n",
    "model.add(Activation('relu'))\n",
    "for i in range(5):\n",
    "    model.add(Dense(128))\n",
    "    model.add(Activation('relu'))\n",
    "    if i%2==0:\n",
    "        model.add(Dropout(dratio))\n",
    "    \n",
    "model.add(Dense(10))\n",
    "model.add(Activation('softmax'))\n",
    "# TODO: add layers as indicated\n",
    "\n",
    "model.summary()\n",
    "# TODO: add layers to the network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 60000 samples, validate on 10000 samples\n",
      "Epoch 1/20\n",
      "2s - loss: 0.8322 - acc: 0.7210 - val_loss: 0.2155 - val_acc: 0.9331\n",
      "Epoch 2/20\n",
      "2s - loss: 0.2309 - acc: 0.9332 - val_loss: 0.1360 - val_acc: 0.9593\n",
      "Epoch 3/20\n",
      "2s - loss: 0.1608 - acc: 0.9526 - val_loss: 0.1220 - val_acc: 0.9631\n",
      "Epoch 4/20\n",
      "2s - loss: 0.1294 - acc: 0.9630 - val_loss: 0.1040 - val_acc: 0.9685\n",
      "Epoch 5/20\n",
      "2s - loss: 0.1083 - acc: 0.9685 - val_loss: 0.1055 - val_acc: 0.9697\n",
      "Epoch 6/20\n",
      "2s - loss: 0.0911 - acc: 0.9730 - val_loss: 0.0899 - val_acc: 0.9721\n",
      "Epoch 7/20\n",
      "2s - loss: 0.0806 - acc: 0.9764 - val_loss: 0.0919 - val_acc: 0.9727\n",
      "Epoch 8/20\n",
      "2s - loss: 0.0719 - acc: 0.9786 - val_loss: 0.0831 - val_acc: 0.9759\n",
      "Epoch 9/20\n",
      "2s - loss: 0.0640 - acc: 0.9815 - val_loss: 0.0875 - val_acc: 0.9770\n",
      "Epoch 10/20\n",
      "2s - loss: 0.0566 - acc: 0.9837 - val_loss: 0.0917 - val_acc: 0.9753\n",
      "Epoch 11/20\n",
      "2s - loss: 0.0514 - acc: 0.9844 - val_loss: 0.0854 - val_acc: 0.9760\n",
      "Epoch 12/20\n",
      "2s - loss: 0.0484 - acc: 0.9855 - val_loss: 0.0887 - val_acc: 0.9772\n",
      "Epoch 13/20\n",
      "2s - loss: 0.0444 - acc: 0.9866 - val_loss: 0.0834 - val_acc: 0.9791\n",
      "Epoch 14/20\n",
      "2s - loss: 0.0409 - acc: 0.9876 - val_loss: 0.0832 - val_acc: 0.9778\n",
      "Epoch 15/20\n",
      "2s - loss: 0.0375 - acc: 0.9886 - val_loss: 0.0978 - val_acc: 0.9741\n",
      "Epoch 16/20\n",
      "2s - loss: 0.0339 - acc: 0.9896 - val_loss: 0.0834 - val_acc: 0.9795\n",
      "Epoch 17/20\n",
      "2s - loss: 0.0329 - acc: 0.9896 - val_loss: 0.0920 - val_acc: 0.9774\n",
      "Epoch 18/20\n",
      "2s - loss: 0.0293 - acc: 0.9907 - val_loss: 0.0888 - val_acc: 0.9775\n",
      "Epoch 19/20\n",
      "2s - loss: 0.0268 - acc: 0.9918 - val_loss: 0.0865 - val_acc: 0.9801\n",
      "Epoch 20/20\n",
      "2s - loss: 0.0242 - acc: 0.9925 - val_loss: 0.0955 - val_acc: 0.9784\n",
      "(50.16580080986023, 'seconds.')\n",
      "----------\n",
      "Loss: 0.095479\n",
      "Accuracy: 0.978400\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAa0AAAEKCAYAAAChTwphAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8VPW9+P/XOwtkAcIqAkGIioiIgiJoqdvXqwbqF0Wt\nGy60KkXEn/1VrXjdKt7e2tZa23sRi15axaoo1ZYqFlyw1KtoooICsgREScJmZA2EkOT9/eNzhkwm\nM8kkmTOTSd7Px+M85sxZ3znJzDufcz6LqCrGGGNMMkhJdADGGGNMtCxpGWOMSRqWtIwxxiQNS1rG\nGGOShiUtY4wxScOSljHGmKRhScsYY0zSsKRljDEmaVjSMsYYkzTSEh1ArKSkpGhmZmaiwzDGmKSy\nf/9+VdWkKcD4lrREmANcBGxX5cQw6wX4HTAO2A9MUuUTb90NwH3epv+hyjONnS8zM5Py8vJYhW+M\nMe2CiBxIdAxN4Wd2/ROQ38D6scAgb5oMzAIQoTvwIDAaGAU8KEI3H+M0xhiTJHxLWqosBb5tYJOL\ngWdVUVWWAV1F6ANcCLypyreq7ATepOHkZ4wxpp1I5H3MfsDmoPfF3rJIy40xxrRzSV0RQ4TJuFuL\ndOhQf/2hQ4coLi6moqIizpG1PRkZGeTm5pKenp7oUIwx7Vgik1YJ0D/ofa63rAQ4J2T5u+EOoMps\nYDZAdjb1BgYrLi6mc+fODBw4EBGJTdTtkKpSVlZGcXExeXl5iQ7HGNOOJfL24ALgehFEhNOB3aps\nARYBF4jQzauAcYG3rMkqKiro0aOHJawWEhF69OhhJVZjTML5WeX9BVyJqacIxbgagekAqjwJLMRV\ndy/CVXn/gbfuWxEeBgq8Q81QbbBCRyNxWMKKBbuOxpjWwLekpcrVjaxX4NYI6+YAc/yIq56qKti+\nHXJyIDs7Lqc0xphgNTVw8CAcOAAVFW5qyvyRR8LkyYn+KeIjqStixExpKaSkWNIyxkRFFcrLYe9e\n2LPHvbZkaumd99NPt6TVfqSmuoR16JAvh9+1axfPP/88U6dObdJ+48aN4/nnn6dr165N2m/SpElc\ndNFFXH755U3az5jWqrraJYjycti3r+5rYL6qym0rUjs15f3Bg3UTUHAiijSv9ap+hdepE3TuXHc6\n6qi677OyIDPTTRkZta/Rzqemxvaat2aWtERcffnKSl8Ov2vXLp544ol6Sauqqoq0tMiXf+HChb7E\nY4yfqqtdEgl8sQfmwy0LXR8pMcW7/k9amkskXbrUJpVu3WoTTfDy4PlwU3a2+5/YxE77SVo//jEs\nXx5+3f797jUrq2nHHD4cHn+8wU2mT5/Ohg0bGD58OOnp6WRkZNCtWzfWrFnDunXruOSSS9i8eTMV\nFRXcfvvtTPbK+AMHDqSwsJB9+/YxduxYvvvd7/L+++/Tr18//va3vxFN58Bvv/02d955J1VVVZx2\n2mnMmjWLjh07Mn36dBYsWEBaWhoXXHABjz76KC+//DIPPfQQqamp5OTksHTp0qZdC5NUDh1ySSF4\nCi3BhCaRht4HpgNN6MUuO9t9sXfqVDt16wb9+7t12dluWUOvgalDB1fyCUzQtPcdO9YmoY4da0ti\npvVpP0mrISkp7l9EHzzyyCOsXLmS5cuX8+677/K9732PlStXHm7vNGfOHLp3786BAwc47bTTuOyy\ny+jRo0edY6xfv54XXniBp556iiuuuIK//OUvXHvttQ2et6KigkmTJvH2229z3HHHcf311zNr1iyu\nu+46Xn31VdasWYOIsGvXLgBmzJjBokWL6Nev3+FlpnVQrb19FekWVqC0Epgae3/wYPTnFwmfKDp3\ndhUAAusCCShQymhomZVAkoxIPq6D81TgaVQfCVl/FPAM0NXbZjqqvtwuaj9Jq6ESUUkJbNkCp57q\n+79Yo0aNqtNA9/e//z2vvvoqAJs3b2b9+vX1klZeXh7Dhw8H4NRTT2XTpk2Nnmft2rXk5eVx3HHH\nAXDDDTcwc+ZMpk2bRkZGBjfeeCMXXXQRF110EQBjxoxh0qRJXHHFFVx66aWx+FHbjUBCibamV7hl\ngcQS6flJtI9cMzJqSy3BSaJPn/qlmsD7QNIJTUqB9xkZVvJo10RSgZnA+bhu9QoQWYDq6qCt7gNe\nQnUWIifgmjQN9COc9pO0GhLomujQofD9QcVQdlANxXfffZe33nqLDz74gKysLM4555ywDXg7dux4\neD41NZUDTbkHEyItLY2PPvqIt99+m/nz5/Pf//3fvPPOOzz55JN8+OGHvP7665x66ql8/PHH9ZJn\ne1NZCVu3usqlpaXu/5rAfPCysrLmn6NjRzd16lT3+cgRR4R/bhKYD32W0qWLSzANPCY1rdSWLXDV\nVTBvniu5tkKjgCJUNwIg8iKuw/PgpKVAF28+Byj1Kxj7E4faROVD0urcuTN79+4Nu2737t1069aN\nrKws1qxZw7Jly2J23sGDB7Np0yaKioo49thjmTt3LmeffTb79u1j//79jBs3jjFjxnD00UcDsGHD\nBkaPHs3o0aN544032Lx5c5tOWpWV8NVXsHEjfPmlK2yHJqRvvqm/X1qa+2Lp2xeOPRbOOsuVYnJy\nml7jq2PH1nGLLAm+NBuU7PE//DC89x7MmAFPPJHoaMIK14n56JBtfgYsRuQ2IBv4N7+CsaQFtYmq\nsjLmbbV69OjBmDFjOPHEE8nMzKR3796H1+Xn5/Pkk08yZMgQBg8ezOmnnx6z82ZkZPDHP/6R73//\n+4crYkyZMoVvv/2Wiy++mIqKClSVxx57DIC77rqL9evXo6qcd955nHzyyTGLJRFqatyX2ZdfuimQ\nnALzJSV1qyynpkLv3i4ZDRwI3/mOm+/Tx70Gpp49W0eiCdbSL+2WfmkmOmkkOv7m7p+ZWbdm5KxZ\nbsrIaFqFlpbG3xPSECkMWjQb1dlNPMzVwJ9Q/Q0iZwBzETkR1ZqmR9QI9Qa0SvYpKytLQ61evbre\nsrAqK1ULClS3bo1u+3Yq6usZB/v3q27YoPree6qzZ6vm5alOmqSan686eLBqx47BdcNURVT79VM9\n80zV669XffBB1WeeUV26VLWw0C3fsiUxP0tpqepZZzX//LfcopqS4l6bIiOj7jUKTBkZ8Tl/S7WW\n+Ju7f2mp6jXXqGZlubizslQnTmz630FL4wfKtaHvVzhDYVHQ+3sU7gnZZpVC/6D3GxWOaPC4zZwS\nnmxiNbUoadXUuG+uzZuj276dikfS2rdPdf16l0zmzVP97W9Vf/pT1WuvVT3vPNUTTlDt2jX8l1WH\nDqqnnKJ62WWqd96p+sQTqm+8obp2rWpFReRztvRDn6xJp6VfmrFKGs2V6Phj8fNPmeJ+9xkZTf8b\niNX1jyJppXlJKE+hg8IKhaEh27yhMMmbH6JQqiANHteSVguSlqrqZ5+5f92TxNSpU/Xkk0+uM82Z\nM8fXc8YiaVVXq371lerixaq//73qrbeqnn++6vHHq3bpEjkZDRigevrpqpde6vb5j/9QTU9P/JeO\navImHdWWfWnGqqTQkqSfyPhj8fNPmKA6darq8uXudcKE+MUf0GjScolonMI6hQ0K93rLZiiM9+ZP\nUPhfL6EtV7ig0WM2c7JnWgHp6b71iuGHmTNnJjqEw8LdU9+3D9atg7VrYc0a9xqYgu/Xd+kCeXmu\n0sPll8OgQe45UuBZUp8+0L17+CrXP/wh3Hkn/PWvrn14VhZMmACPPhpd3Bs3tmz/lj6TaOn5+/Rx\n16+iwp2zosK9b8pzjW3bYMoU12/d7NnudxmtWJwfWvZMKpHxx+Lnf+WV2vmmfqRjdf2j4tpcLQxZ\n9kDQ/GpgjA9nrseSVkCHDq55v2mSnTvhRz+Cf/0Lzj/fVWZYuxaKi2u3SUlxlRsGD4Zzz3WvgwfD\n8ce77W+9Ff7wB1ebbvr06M+d6C+dZE860LIvzZaePxYVERIZfyz2b6lEnz8RxJUOk192draWhySd\nL774giFDhkR3gOJi9xdwyintriVlZaX7Aj7mmNoma+GsWvUFZWVDKCiAwkJXsgr355OaCg895JLS\n4MGuanhGRv3tQr+0AprypXXppe7LP/hDG/xF5vf+t9zi9gt0X/mjHzWttNDS8yezLVsiJ/1krLqe\nrERkv6omzRAXVtIKSE9338BVVQ1/c7dC0SadSLZscbfzSkthwAC3rKbGfZGUl9e+FhfD2LFu/VFH\nQX6+22fNGtcrRFO/dFpaUoGW/6ed6P/UW3r+ZBbX21umzbCkFRDcwDjJkla4pBONjz+uW1LascNN\nodLTXULJyYHXX3e9XQWam91yC3z+eeKeCSRae046sdAeb2+ZlmllzSQTKJC0vv4azj7b9d8TA4Gh\nSZrj8ccfZ3+gB/owPv4Y+vYdyPr1ruuGHTvcbbuPP274uDU1rj+73r3r52eR2r7qjjkGTjrJTYMG\nQdeuMG5cbcKC2i+dZcvca1MvW0v3N8ntlVdcsj/5ZPfaXm6NmuazZ1oBlZXw2Wfuk/Pss01/OBHB\npk2buOiii1i5cmWT9w0MT9KzZ8+w6ysr4eijB/LMM4Xk5PQkJcUllv796yajwK2+QCes5eVuGbhu\niaqqXLJShV69IpfWmnQ9jTFJwZ5pJaucnNj0qRIieDyt888/nyOOOIKXXnqJgwcPMmHCBB566CF2\n7iznkkuuYOfOYmpqqrn//vvZtm0bpaWlnHvuufTs2ZMlS5bUO3agcKjqks7cuY+xcOEc0tPhuutu\n4vrrf8y2beVMm3YF27cXU11dzS233M8VV1zJY49NZ/HiBdTUpHHWWRfw+OOPsmOHbwM4G2NMTFjS\nCti4EW68EZYsccmrObUCwggeT2vx4sXMnz+fjz76CFVl/PjxLF26lNWrd9CtW1+effZ1BgxwHenm\n5OTw2GOPsWTJkoglrYAePeDbbz/m9df/yDPPfEh1tXL99aPJzT2bHTs20r9/X/7+99fp3BnKy3dT\nVVXGG2/UHVMrK6tpz8OMMSYR7JlWQKBWwMGDTa4VUFnpatA1VkpZvHgxixcvZsSIEZxyyimsWLGG\nN99cT69ew/joozd54IG7eeqpf1FUlBNVyIHbert2wcKF73HWWRPo0iWbo47qxKWXXsr27f9i3Lhh\nfPDBmzzyyN188MG/yMnJIScn5/CYWq+88gpZTR2x2RhjEsSSVrCdO+GKK5pcKyC49l5DVJV77rmH\n5cuXs3z5cjZsKOIHP7iRvLzjmDv3EwYNGsbTT9/HggUzGjxOZSVs3uwewVVXu47pe/Z0FSSGDnXV\n0TMzXXup4447jk8++YRhw4Zx3333MWPGjMNjal1++eW89tpr5OfnR3uFjDEmoez2YLCnn3ZV8E46\nKar6y5GqjIu4auFQdzytCy+8kPvvv5+JEyfSqVMnduwoYefOdLZvryInpzv5+dfSt29XFix4us6+\ngduDFRUuj5aVufP26OEqXBx9NKSlncmkSZO4997pqCqvvvoqc+fOpbS0lO7du3PttdfStWtXnn76\n6YhjahljTGtnSStYerqrVlddHdUQsMOGuQa3u3a53YJr7wUEj6c1duxYrrnmGs444wwAOnXqxH/8\nx3Ps2FHEXXfdRU1NCmlp6cyZMwuAyZMnk5+fT+/effnjH5ewc6dLiL16uVJVx461nXeccsopTJo0\niVGjRgFw0003MWLECBYtWsRdd91FSkoK6enpzJo1i71794YdU8sYY1o7q/Ie7NtvXYWMoUPd/bUo\nfPVVbemqsSrjTaHqqqhv3eqqqaemuiHYjzgicW2frcq7MW2PVXlPZsEjGEeZtA4dcomqVy9iUmVc\n1ZXctm517anS0yE31x0/NbVlxzbGmGRnSStYoAjThCFKjj22dr4lJayaGlfQ27rVPbvq2NFVqAgM\n7z569GgOHjxYZ5+5c+cybNiw5p/UGGOSjK9JS4R84HdAKvC0Ko+ErB8AzAF6Ad8C16pS7K2rBj73\nNv1alfHNiUFVkWh7bQ8krTi2sFV1JbStW2sLeHl59ceQ+vDDD+MWUzht5TayMSa5+VblXYRUYCYw\nFjgBuFqEE0I2exR4VpWTgBnAL4LWHVBluDc1K2FlZGRQVlYW/RduSkpcB4OsqYFNm1x3hx06uFLb\nCSe4WoGtaXQUVaWsrIyMcOOLGGPaPpF8RNYiUoRI/VHvRH6LyHJvWofILt9C8es/aBHOAH6myoXe\n+3sAVGsTkwirgHxVNosgwG5Vunjr9qnSKdrzhauIcejQIYqLi6kIN2hTJFu21NZ68FFNjSthVVS4\nHqS6dvX1dC2WkZFBbm4u6UnWA74xpmGNVsQQSQXWAecDxUABcLU3WnG47W8DRqD6w9hH6+/twX7A\n5qD3xcDokG1WAJfibiFOADqL0EOVMiBDhEKgCnhElb+GnkCEycBkqK1DESw9PZ28vLymRX3PPbBh\ngxtvwyelpa639FWr4Kmn3LhUxhjTSo0CilDdCIDIi8DFQPikBVcDD/oVTKJ7xLgTOFuET4GzgRKg\n2ls3QJWRwDXA4yIcE7qzKrNVGanKyCiaVUUnN7fuWPExtmoVnH66y4uvvQaTJvl2KmOMiYVwBZB+\nYbcUGQDkAe/4FYyfJa0SIKiZLbnessNUKcWVtBChE3CZKru8dSXe60YR3gVGABt8jNeLMtfVOd+3\nDzpFfXcyKv/8J1xyievacOlSGDEipoc3xpgm6wlpiBQGLZqN6uxmHu4qYD6q1Y1u2Ux+lrQKgEEi\n5InQAffDLAjeQISeIodjuAdXkxARuonQMbANMIbIRdHYys11ryUlDW/XRPPmwQUXuH55ly2zhGWM\naR2+gSpURwZNoQmr0QJIkKuAF/yIM8C3pKVKFTANWAR8AbykyioRZogcrg14DrBWhHVAb+Dn3vIh\nQKEIK4AluGdaSZm0VOE3v4GrroLRo+G992wIEGNMUikABiGSh0jYAggAIscD3YAP/AzG13ZaqiwE\nFoYseyBofj4wP8x+7wOJaTUbSFoxeK5VXQ0/+Qn8/veu8/hnnnG3Bo0xJmmoViESKICkAnNQXYXI\nDKAQ1UACuwp40e9GndYjRqh+3vPFFiatAwdg4kR49VW44w741a9cMzBjjEk6qvUKIKg+EPL+Z/EI\nxZJWqMxM17q3BUnrm29g/Hj37Orxx+H222MYnzHGtGOWtMLp16/ZSWvjRhg71vX+/vLLcNllMY7N\nGGPaMUta4TSzrVZhIXzve1BVBW+/DWPG+BCbMca0Y/aUBddz09lnu05rgWYlrYUL3TGysuD99y1h\nGWOMHyxpAQ8/7Kqiz5jhLcjNdR0DhgwFEsnTT7tnWEOGwAcfwODB/sVqjDHtWZseubgxmZmuw9pQ\nN6f9kdlVP3QPqBrpu3DXLjfm1bnnupqCMe5EwxhjfJVsIxe365LWxo1wzTXulh6414kT4ZHnom+r\n9fXXrj3W5MmWsIwxxm/tOmn16QNdurjSVkaGe+3SBbqfFH3SKi11r/3Cdx9pjDEmhtp10gLYtg2m\nTHFtqqZM8SpjNKFXjEBvT5a0jDHGf+2+yvsrr9TOz5wZmOsMnTs3qaTVp0/MQzPGGBOi3Ze0Ioqy\n2ntJCfTqFX4QSmOMMbFlSSuSJiStvn3jEI8xxhhLWhFFmbRKS+15ljHGxIslrUhyc12tjKqqBjez\nkpYxxsSPJa1IcnOhpiaob6f6Dh2C7dutpGWMMfFiSSuSKKq9b93qRia2kpYxpk0TyUdkLSJFiEyP\nsM0ViKxGZBUiz/sVSruv8h5RFEnLGhYbY9o8kVRgJnA+UAwUILIA1dVB2wwC7gHGoLoTkSP8CsdK\nWpFEMYKxNSw2xrQDo4AiVDeiWgm8CFwcss3NwExUdwKgut2vYCxpRdK9u+vbKYqSlt0eNMa0Yf2A\nzUHvi71lwY4DjkPkfxFZhki+X8HY7cFIRBqt9l5SAunprpd3Y4xJRj0hDZHCoEWzUZ3dxMOkAYOA\nc4BcYCkiw1DdFaMw65zIRBJF0urTB1KsvGqMSVLfQBWqIxvYpAToH/Q+11sWrBj4ENVDwJeIrMMl\nsYKYBovdHmxYI0nLGhYbY9qBAmAQInmIdACuAhaEbPNXXCkLRHribhdu9CMYS1oNyc11mammJuxq\na1hsjGnzVKuAacAi4AvgJVRXITIDkfHeVouAMkRWA0uAu1At8yMcuz3YkNxc14J4xw7o3bve6tJS\nuOCCBMRljDHxpLoQWBiy7IGgeQV+4k2+spJWQxpoq7VvH+zZYyUtY4yJJ0taDWmgrZY1LDbGmPjz\nNWmJkC/CWhGKRKjX9YcIA0R4W4TPRHhXhNygdTeIsN6bbvAzzogaKGkFGhZbScsYY+LHt6QlQqDr\nj7HACcDVIpwQstmjwLOqnATMAH7h7dsdeBAYjWuN/aAI3fyKNaIjjoC0NCtpGWNMK+FnSWsUUKTK\nRlUidf1xAvCON78kaP2FwJuqfKvKTuBNwLcW1hGlpLisZCUtY4xpFfxMWtF0/bECuNSbnwB0FqFH\nlPvGR4S2WqWl0Lmzm4wxxsRHoiti3AmcLcKnwNm4VtbV0e4swmQRCkUobGSsxubLza0tVgUpKbFb\ng8YYE29+Jq1Gu/5QpVSVS1UZAdzrLdsVzb7etrNVGanKyDS/WpwFSlqqdRZbw2JjjIk/P5NWATBI\nhDwRwnb9IUJPkcMx3APM8eYXAReI0M2rgHGBtyz+cnPhwAHYubPOYuvCyRhj4s+3pKVKva4/VFkl\nwgwRAl1/nAOsFWEd0Bv4ubfvt8DDuMRXAMzwlsVfmGrvNTUuaVlJyxhj4svXbpxUqdf1hyoPBM3P\nB+ZH2HcOtSWvxAluYHzSSQCUlbnenaykZYwx8ZXoihitX5iSllV3N8aYxLCk1Zgjj3TttYKSljUs\nNsaYxLCk1Zj0dJe4rKRljDEJZ0krGiFttUpLQcSNWmyMMSZ+LGlFI6RXjJIS1y1henoCYzLGmHbI\nklY0wiQtuzVojGk3RPIRWYtIESL1RuxAZBIiOxBZ7k03+RWKjVwcjdxcN+Ljnj3QpQulpbWVCo0x\npk0TCYzYcT6uH9gCRBagujpky3moTvM7HCtpRSNQTdB7rmUlLWNMOzIKKEJ1I6qRRuyIG0ta0Qhq\nq1VZCTt2WHV3Y0y7Ee2oG5ch8hki8xHpH2Z9TFjSikZQ0tqyxc1aScsY0xb0hDRECoOmyc04zN+B\ngaiehBv/8JnYRlnLnmlFI5ChioutYbExpk35BqpQHdnAJo2PuqFaFvTuaeBXMQswhJW0opGRAb16\nQXGxNSw2xrQ3BcAgRPIQCTtiByLBrVbH4zpJ94WVtKLlNTC2kpYxpl1RrUIkMGJHKjAH1VWIzAAK\nUV0A/H+IjAeqgG+BSX6FY0krWrm58PXXlJRAhw7Qo0eiAzLGmDhRrTdiB6oPBM3fgxsT0Xd2ezBa\nXgPjwDhaIokOyBhj2h9LWtHq1w/Kytjx9QG7NWiMMQliSStaXrX36q9LrBKGMcYkiCWtaHlJK21r\nsZW0jDEmQSxpRctLWj0qiq2kZYwxCWJJK1pe8SoXK2kZY0yLiNyOSBdEBJH/QeQTRC6IZteokpYI\nE0TICXrfVYRLmhtvUurUiUOdutIPe6ZljDEt9ENU9wAXAN2A64BHotkx2pLWg6rsDrxRZRfwYFOj\nTHb7uuZaScsYY1ou0GhoHDAX1VVByxoUbdIKt127a5j8bZZLWlbSMsaYFvkYkcW4pLUIkc5ATTQ7\nRpt4CkV4DDcQGMCtwMdNDjPJbU/rR54sJzs70ZEYY0xSuxEYDmxEdT8i3YEfRLNjtCWt24BKYB5u\nALAKXOJqV76uyeUI3QaVlYkOxRhjktkZwFpUdyFyLXAf1D6CakhUJS1VyoHpzY+vbdhwMJcUFLZs\ngQEDEh2OMcYkq1nAyYicDNyBG87kWeDsxnaMtvbgmyJ0DXrfTYRFzQw2aX2xt3YwSGOMMc1WhaoC\nFwP/jepMoHM0O0Z7e7CnV2MQAFV2Akc0tpMI+SKsFaFIpH5JTYSjRFgiwqcifCbCOG/5QBEOiLDc\nm56MMk7f1NTA5zstaRljTAzsReQeXFX31xFJAdKj2THapFUjwlGBNyIMBLShHURIxVXcGAucAFwt\nwgkhm90HvKTKCNzAYk8ErdugynBvmhJlnL7ZsQM2VXtJq6Sk4Y2NMcY05ErgIK691lbcaMi/jmbH\naJPWvcB7IswV4TngnzQ+dsoooEiVjapU4ipwXByyjQJdvPkcoDTKeOKupAR2k0NVRraVtIwx7YtI\nPiJrESlCJHL9BpHLEFFERjZ4PJeo/gzkIHIRUIHqs9GEElXSUuUfwEhgLfAC7sHZgUZ26wdsDnpf\n7C0L9jPgWhGKcQOM3Ra0Ls+7bfhPEc6MJk4/uRGLhUO9cy1pGWPaD5F6d80QCb1rhtfW6nbgwyiO\neQXwEfB94ArgQ0QujyacqGoPinCTF0wusBw4HfgA+D/R7N+Aq4E/qfIbEc4A5opwIrAFOEqVMhFO\nBf4qwlBV9oTENRmYDG40YT8F7ghKf0taxph2ZRRQhOpGAEQCd81Wh2z3MPBL4K4ojnkvcBqq271j\n9gLeAuY3tmO0twdvB04DvlLlXGAE1FbMiKAE6B/0PtdbFuxG4CUAVT4AMnCVPg6qUuYt/xjYABwX\negJVZqsyUpWRaT73z1Fa6kYr7jCwnyUtY0x70vhdM5FTgP6ovh7lMVMOJyynjCjzUbRJq0KVChcb\nHVVZAwxuZJ8CYJAIeSJ0wFW0WBCyzdfAed5xh+CS1g4RenkVORDhaGAQsDHKWH1RUgK9e0PKUbku\ng1VXJzIcY4yJiZ6Qhkhh0DS5SQdwNf8ewz02itY/EFmEyCREJgGv4x4RNSra8kmx107rr8CbIuwE\nvmpoB1WqRJgGLAJSgTmqrBJhBlCoygLcD/mUCP8/rlLGJFVUhLOAGSIcwvVHNUWVb6OM1Relpd7o\nJLm5LmFt24Z1QmiMSXbfuDZTDVWcaOyuWWfgROBdRACOBBYgMh7VwrBHVL0LkcuAMd6S2ai+Gk28\n4tp3RU+Es3E1/f7h1QpsFbKzs7W8vNy34598MgwcCH+76e8wfjx8+CGMGuXb+YwxJh5EZL+qRu5R\nVSQNWIcQunAbAAAgAElEQVS7K1aCu4t2jdcze7jt3wXujJiwWqjJT4JU+acfgbR2JSXwne9weARj\na6tljGkXVKsQqXPXDNVViMwAClENfewTmchewrfxFUBR7RJmXR3tbniR5jh4EMrKgm4PglXGMMa0\nH6oLCX3mpPpAhG3PaeA4UXXV1JBoK2K0a6Vek+e+fYGePV39ektaxhgTd5a0ohBIWv364eq951pb\nLWOMSQRLWlEIPL46XFmwn7XVMsaYRLCkFYU6JS2wkpYxxiSIJa0olJRAx47QrZu3IJC0mthcwBhj\nTMtY0opCoGGxazeHS1qVlfDNNwmNyxhj2htLWlEoKQnp/MLaahljTEJY0opCSUnQ8yywtlrGGJMg\nlrQaoRrU72CAJS1jjEkIS1qN2L0b9u8PuT3YuzekplrSMsaYOLOk1Yh61d3BJaw+fSxpGWNMnFnS\nakS9hsUB1lbLGGPizpJWI8KWtMCSljHGJIAlrUY0WtKyBsbGGBM3lrQaUVrqesLIzAxZkZsL5eWu\npoYxxpi4sKTViHoNiwOsgbExxsSdJa1G1GujFWBttYwx7YVIPiJrESlCZHqY9VMQ+RyR5Yi8h8gJ\nfoViSasR9XrDCLCkZYxpD0RSgZnAWOAE4OowSel5VIehOhz4FfCYX+FY0mpAdTVs3Rrh9mCfPu7V\nkpYxpm0bBRShuhHVSuBF4OI6W6juCXqXDfhWQy3NrwO3Bdu3u8QVtqTVoYPrGcOSljEmifWENEQK\ngxbNRnV20Pt+wOag98XA6HoHErkV+AnQAfg/PoQKWNJqUMTq7gHWVssYk+S+gSpUR7b4QKozgZmI\nXAPcB9zQ4mOGYbcHGxCxYXGAJS1jTNtXAvQPep/rLYvkReASv4KxpNUAK2kZYwwFwCBE8hDpAFwF\nLKizhcigoHffA9b7FYzdHmxAaSmkpLhHV2Hl5sKuXa6RcXZ2XGMzxpi4UK1CZBqwCEgF5qC6CpEZ\nQCGqC4BpiPwbcAjYiU+3BsGSVoNKSuDII12n7mEFNzA+7ri4xWWMMXGluhBYGLLsgaD52+MVit0e\nbEDEhsUB1lbLGGPiytekJUK+CGtFKBKhXitqEY4SYYkIn4rwmQjjgtbd4+23VoQL/YwzkohdOAVY\n0jLGmLjyLWmJUK8VtQihrajvA15SZQTu4d4T3r4neO+HAvnAE97x4ipibxgBgZWWtIwxJi78LGmN\nAopU2ahK+FbUrtV0F28+B/AqmXMx8KIqB1X5Eijyjhc3Bw7Azp2NJK3MTOje3ZKWMcbEiZ9JK1wr\n6tAU8DPgWhGKcQ/5bmvCvogwWYRCEQqrqmIVthNoo9Xg7UGwau/GGBNHia6IcTXwJ1VygXHAXJHo\nY1JltiojVRmZFuN6kI02LA6wpGWMMXHjZ9KKphX1jcBLAKp8AGQAPaPc11eNNiwOyM21MbWMMSZO\n/ExaBcAgEfJECN+KGr4GzgMQYQguae3wtrtKhI4i5AGDgI98jLWeJpW0tm+Hgwd9j8kYY9o735KW\nKlVAoBX1F7hagqtEmCHCeG+zO4CbRVgBvABMUkVVWYUrga0G/gHcqkq1X7GGU1Li6lnk5DSyYaDa\neyDLGWOM8Y2vPWKoUq8VtSoPBM2vBsZE2PfnwM/9jK8hgYbFIo1sGNxWKy/P97iMMaY9S3RFjFar\n0YbFAdZWyxhj4saSVgSNduEUMGCA6yz30Udhz57GtzfGGNNslrQAtmyBs8+GrVsBUG1CSSs7G+bN\ng88+g//7f12rZGOMMb6wpAXw8MPw3nswYwbgesKoqIiypAXwve/Bs8/Cv/4F3/8+HDrkX6zGGNOO\nte+klZnpalrMmgU1Ne5VhK59MoEmJC2Aq6+GJ56A11+HSZPc8YwxxsRU+05aGzfCNddAVpZ7n5UF\nEyfyzz99CUR5ezDYlCnwn/8Jzz8P06a5+4zGGJPsRPIRWYtIESL1RuxA5CeIrEbkM0TeRmSAX6G0\n70Eg+/SBLl3cvcCMDPfapQubKo4EmljSCpg+3d1f/PWvoVs3+HnCau0bY0zLiQRG7Dgf1w9sASIL\nUF0dtNWnwEhU9yNyC/Ar4Eo/wmnfJS2AbdtcCWnZMve6devhXpn69GnG8UTgl7+Em292pa5HH41p\nuMYYE2ejgCJUN6IafsQO1SWo7vfeLcN1veeL9l3SAnjlldr5mTMBKJ0KPXq4wlezBJ6T7d4Nd90F\nXbvCTTe1PFZjjIm/cKNujG5g+xuBN/wKxpJWGFFXd29IairMnQt798Lkya4/qO9/PybxGWNMrPSE\nNEQKgxbNRnV2sw4mci0wEjg7FrGFY0krjKgbFjemQweYPx8uvBAmToTOnSE/PwYHNsaY2PgGqlAd\n2cAm0Y26IfJvwL3A2aj61oO4PdMKIyYlrYCsLPj732HoULj0Uvjf/43RgY0xJi4KgEGI5CESfsQO\nkRHAH4DxqG73MxhLWiGqqlzdjJiUtAK6doVFi6B/f9cQefnyGB7cGGN8pFpvxA5UVyEyA5HAiB2/\nBjoBLyOyHJHQYahiRrSNtCXKzs7W8vLyFh+npMR13P7kk/CjH8UgsGBffw3f/a4be+tf/4Ljjovx\nCYwxpmlEZL+qZic6jmhZSStE1CMWN8dRR8Gbb7pGx+efD5s3N76PMcaYwyxphYh6xOLmGjzY3Src\ntcslrh07fDqRMca0PZa0Qvha0goYMQJeew2++srVJty928eTGWNM22FJK0RpqWtidcQRPp/ozDPh\nL39xQ5qMHw8xeB5njDFtnSWtECUlrvumlHhcmXHj4LnnXKWMo4923T/ZQJLGGBORJa0QMWtYHK0r\nr3RJa/hw19nugAHwwANQVhbHIIwxJjlY0goR04bF0RozxlXOKCiAc891g1IOGAB33FFbM8QYY4wl\nrVBxL2kFGznSdeC7ciVMmAC/+x3k5cEtt8CXXyYoKGOMaT3adOPiQ4cOUVxcTEVFRVTHqKlxTae6\ndnX920YrbccO+t5xByW/+Q3VvXo1JewGpW/eTI//+R9yXn0Vqalh9/e+R9lNN1F57LExO0djMjIy\nyM3NJT09PW7nNMbET7I1Lm7TSevLL7+kc+fO9OjRAxFp9BgVFa6QM3Ag9OzZhJNPnQp/+IPrQuOJ\nJ5oWeDRKSuCxx1w3Hfv3uz4M//3f4dRTY3+uIKpKWVkZe/fuJS8vz9dzGWMSI9mSVpu+PVhRURF1\nwgI4dMi9dugQ5QkyM2vHzqqpca8ibnks9esHv/mNa9d1//3wzjvuVmJ+PixdGttzBRERevToEXVJ\n1Rhj/NamkxYQdcICqKx0r1HfCdu4Ea65xvXkDu514sSmP3/asgXOPhu2bm14u549YcYMl7weeQQ+\n/dTtd+aZbuyuXbuadt4oNOX6GWOM39p80mqKJpe0+vSBLl3cfcWMDPfapQsceWTTTvzww/Deey4h\nRaNLF7j7bpcc/+u/3IO46693LaLHjoWnnrLuoYwxbZKvSUuEfBHWilAkwvQw638rwnJvWifCrqB1\n1UHrfOvmPtihQ65RcZMaFm/bBlOmwLJl7jWktLRr1y6eiPScq4Hbi+PGjWNXYyWnrCyYNs2V+D74\nAG6/HdaudSMlH3mkqz7/X/9V2zeVMcYkOd8qYoiQCqwDzgeKcQOJXa3K6gjb3waMUOWH3vt9qnSK\n9nzhKmJ88cUXDBkyJOqYN2yAAwfgxBOj3qVRmzZt4qKLLmLlypV1lldVVZG2YwfceSf89a+ugkVW\nlqvq/uijTS+tBajCihWui6hXXoHV3uU+/XRXgeOyy1zvG03Q1OtojEkeyVYRI83HY48CilTZCCDC\ni8DFED5pAVcDD/oVzI9/3PjYi/v3u9fAI6rGDB8Ojz/e8DbTp09nw4YNDB8+nPT0dDIyMujWrRtr\n1qxh3bp1vPH++1y4fz+VInQ8cADxbi8OHDiQwsJC9u3bx9ixY/nud7/L+++/T79+/fjb3/5GZnBl\njy1b4KqrYN48nvr735k9ezaVlZUce/zxPPfcc2QuXMihl14i/ac/hZ/+lDUZGWRddx1H/fjHPFtY\nyJ9+8Qt+tXkzf8rP57/nz4/uhzfGmATw8/ZgPyB4wKhib1k9IgwA8oB3ghZniFAowjIRLvEvzFqq\nse9z8JFHHuGYY45h+fLl/PrXv+aTTz7hd7/7HevWrQPgvKFDSZk6FT74gBe7duXg11/XO8b69eu5\n9dZbWbVqFV27duUvf/lL3Q2CnoldeumlFBQUsGLFCoYMGcLT770H997Ltccfz5z77oNHH2XwiBEc\n9dRTMHQoY266iUXp6Yzcv59HO0VdsDXGtCci+YisRaQIkXqPehA5C5FPEKlC5HI/Q/GzpNUUVwHz\nVakOWjZAlRIRjgbeEeFzVTYE7yTCZGAyNF55orESkSp88gn07u1GLvbLqFGj6rR5+s+RI3n11Vfh\nvffYVF1N3n33cXrIPnl5eQwfPhyAU089lU2bNrkVmZmu8kfArFn0mDWLgyKMHDqUffv2ceGFFwLw\nzjvv8Oyzz0LHjojXPVR1//4cc+gQfP45ABnPPAPPPANpafDxx+4+aVx6DjbtWtCdgmbfFm/P/L5+\nIqnATIIf9YgsQDX4rtnXwCTgztgHUJef30glQP+g97nesnCuAl4IXqDqtvVuL74LjAjdSZXZqoxU\nZWRaC9NvVZVLXH53/JCdXXvr+N133+Wtt97igw8+YMWKFYwYMSJsm6iOHTsenk9NTaWqqsq9CVPl\n/tXsbIrefJPPP/+cBx98MHIbq759+dNDD7F86NDadmWpqdCpk7sYJ5/sPgBXXUXXl1925/JLtFX+\nW6tkjz/Rmlp7NtaS/ffn//UbBRShuhHVSjj8qKeW6iZUPwNq/AoiwM+kVQAMEiFPhA64xFSvFqAI\nxwPdgA+ClnUToaM33xMYQ+RnYTERqO4e66TVuXNn9u7dG3bd7t276datG1lZWaxZs4Zly5Y17eBh\nqtzvrKqi17BhHDp0iD//+c+HNz3vvPOYNWsWANXV1ezevZvTJ0xg9ebN6MGDkJGBqsJ118HXX8Mf\n/wgXXghLl9LnwQfhmGNcP4g33QQvvOBqTQa09EOf7F9ayR5/osSrcX5jWvr7S9T1j9H16wlpiBQG\nTZNDNon6UU88+Ja0VKkCpgGLgC+Al1RZJcIMEcYHbXoV8KIqwdUYhwCFIqwAlgCPRKp1GCtNbqMV\npR49ejBmzBhOPPFE7rrrrjrr8vPzqaqqYsiQIUyfPp3TTw+9MRiFkCr35xx/PKNHj2bMmDEcf/zx\nhzf73e9+x5IlSxg2bBinnnoqq1evZujQoYw59lhezMnh+/37886gQe6D178/TJrkGiyXlLDh7393\nVeeHD4f5813p7sgjYdgwV8Plxhvd8CoPPdS02JP9SytW8Sc6abb0/M3dP96N80PF6veXqOsfo+v3\nDVShOjJomt20QOJMVdvElJWVpaFWr15db1kk27erFhSoVlREvUu7Uec6VlWpfvSR6i9+oZqSouru\nqtadUlNVf/1r1ddeU92wQbW6OvyBS0tVr7lGNSvL7ZeVpTpxouqWLU0LsLRU9ayzmr5fRkb4+DMy\noj9vLOK/5RZ3LW+5Jb7xt/T8sdh/yhS3b0ZG84/R3PO39PfXGq5/DK4fUK4Nfb/CGQqLgt7fo3BP\nhG3/pHB5g8dr4ZTwZBOrqaVJq6TEJa1I36/tWcTrWFqqeuWVqh07uj+ltDTVI45Q7dGj/od4+HDV\nq69WnTFD9eWXVVeuVD14MLm/tFRbFn+ik2ZLzx+LL+0JE1SnTlVdvty9TpgQ/b6xOH9Lfn+Jvv6q\nLbt+niiSVprCRoU8hQ4KKxSGRtjWkla0U0uT1qZNqp9+GvXmCTd16lQ9+eST60xz5szx5VwNXsdI\nH/qyMtX33lN96inVn/xENT9fdcCAuh/O1FTVTp1UBw5UveEGV1r6zndU1693Ca0xif7SUm3Zl0ai\nk2ZLzx+rkmZzxeL8Lf3ST+T1j5FGk5ZLRuMU1ilsULjXWzZDYbw3f5pCsUK5QpnCqkaP2cyptVR5\nT7jKSv9rDsbSzJkzEx2CE3imNnkyzJ7t7s8DdO/uRmQeM6bu9uXlrqupL75w0+rV7vWFF2p7LB40\nyFW179/f9d4RburRw93Tj9SjSEvjj9Yrr9TON/V3Eou+K1sSf0vPH6u+N5srFudvye8PEnv940l1\nIbAwZNkDQfMFuBrivmvT42k1pfuh1atd0ho0yI/okltcunGqqXHDRm/cGH4Krq0I0LmzS167d8Om\nTa5tWXU1XH65+/Lo2tXfeGPl0kvdl1fwl17wF2lrP3+yx59orSD+ZOvGyZKWZ8UKN1rxwIE+BJfk\nWkXfg+XlrlZUaDJbutSVsqqr627fpQsMGOCmo46qnQ+8P/JIazhtDMmXtOz2IO6f/EOHYl/d3cRQ\ndrbroSNcb8Y1NW4olq++qj99/bWrjhzaY36HDu72YyCR9e9ff+rcOT4/mzEmapa0cB1AgD/PtHbt\n2sXzzz/P1KlTm7zv448/zuTJk8mKtgff9iolxfW/1bs3jBoVfps9e+omsuDE9o9/uDYyoXcdcnJc\nn17hElr//m6d/W6MiSu7PQjs2wdr1sCxx7qhSWLZjVekoUmiEejpvWfPni0PpAVaxe1Bv1VWumdq\nmzdDcbF7DZ3CDazZvbtLYEceCb16udGlI7127263JE2rY7cHk1Bwbxj//u+1jdsjjd3YFMFDk5x/\n/vkcccQRvPTSSxw8eJAJEybw0EMPUV5ezhVXXEFxcTHV1dXcf//9bNu2jdLSUs4991x69uzJkiVL\nwh7/lltuoaCggAMHDnD55ZfzkNcrRUFBAbfffjvl5eV07NiRt99+m6ysLO6++27+8Y9/kJKSws03\n38xtt93W8h+yLejQwT3QbOihZkWFG1AzOJEFEty2bbBunUts+/aF3z8lxSWucEktJ8f1+5idHf41\nMJ+dbYnPtGuWtHD/ZI8ZU1vjGlyPLrNmuZqoBw40/9iPPPIIK1euZPny5SxevJj58+fz0UcfoaqM\nHz+epUuXsmPHDvr27cvrr78OuD4Jc3JyeOyxx1iyZEmDJa2f//zndO/enerqas477zw+++wzjj/+\neK688krmzZvHaaedxp49e8jMzGT27Nls2rSJ5cuXk5aWxrffftv8H6w9yshwfTAec0zD21VUwDff\nuAQW+ho8v2aNey0rc8/lopWZWT+xdeniakx27eoSYPBruGWdOlnyM0nJkhaupLVggRuV429/a36T\nn8YsXryYxYsXM2KE67B+3759rF+/njPPPJM77riDu+++m4suuogzzzwz6mO+9NJLzJ49m6qqKrZs\n2cLq1asREfr06cNpp50GQJcuXQB46623mDJlCmlel/jdu3eP3Q9namVkuOdd0Y5xU1PjakeWl7tS\nWrjXhtbt2+ee2W3e7JoA7NrV+H9aKSm1iS4nx01dutROnTvXfR9pmZX8TJxZ0sIlrT593OfWz3Z+\nqso999zDj370o3rrPvnkExYuXMh9993HeeedxwMPPBDmCHV9+eWXPProoxQUFNCtWzcmTZoUeSgS\n03qlpLiEEMvaipWVtQks8Bo8H7xs1y7Yu9c901uzxiXAPXvqjtXWkEDswYktdFlj8126WOnPRMWS\nFrW9YbS0c4RwgocmufDCC7n//vuZOHEinTp1oqSkhPT0dKqqqujevTvXXnstXbt25emnn66zb6Tb\ng3v27CE7O5ucnBy2bdvGG2+8wTnnnMPgwYPZsmULBQUFnHbaaezdu5fMzEzOP/98/vCHP3Duuece\nvj1opa02qkMH96ysV6/mH+PQIZfMAklsz57674OXB9bt3Qvbt9ddFnhw3JjQ0lxoCTB0CqzPyHAN\nzFNT3Wuk+dBlKSmuZ3eTNCxp4T5PmZkt79ElnOChScaOHcs111zDGWecAUCnTp147rnnKCoq4q67\n7iIlJYX09PTD415NnjyZ/Px8+vbtG7Yixsknn8yIESM4/vjj6d+/P2O8LpM6dOjAvHnzuO222zhw\n4ACZmZm89dZb3HTTTaxbt46TTjqJ9PR0br75ZqZNmxabH9S0PenpruJILP6xOXiwblILng9Nhrt3\n102IxcW1yyOMTdciaWnuCyA0EQYnzMaWderkjpFMfcElKavyDnz6qevK7qij/IouubWLKu8mOdTU\n1D7DC05yFRWuV5SqKjcF5sMtC11/6JB7BhguaQbmI9UIDRVIgFlZTX/NynLPCAPzDb2PYXK0Ku9J\nprraTfYPkjFJIFCBxKtcFDfV1bWlwdCktnu3qxBz4ICrxbV/f+188Ovu3a4Re/Dy/ftdKbSp0tLq\nJrGRI+HFF2P/c7dC7T5p1dS4ux/Zrfz/jNGjR3Mw5I977ty5DBs2LEERGdOOpKbWNh+ItZqaukms\nvLx2PvR9pHXtqNPUdp+00tNdZ+Gt3YcffpjoEIwxfkhJqW04bhpl9UuNMcY0TCQfkbWIFCEyPcz6\njojM89Z/iMhAv0Jp80mrrVQ0SRS7fsa0cyKpwExgLHACcDUiJ4RsdSOwE9Vjgd8Cv/QrnDadtDIy\nMigrK7Mv3mZSVcrKysjIyEh0KMaYxBkFFKG6EdVK4EXg4pBtLgae8ebnA+ch/jSAa9PPtHJzcyku\nLmZHuN65TVQyMjLIjbY7ImNMW9QP2Bz0vhgYHXEb1SpEdgM9gG9iHUybTlrp6enk5eUlOgxjjGm1\nekIaIoVBi2ajOjthATWiTSctY4wxDfsGqlAd2cAmJUD/oPe53rJw2xQjkgbkAGUxDdTTpp9pGWOM\nabECYBAieYh0AK4CFoRsswC4wZu/HHjHr8oEVtIyxhgTmXtGNQ1YBKQCc1BdhcgMoBDVBcD/AHMR\nKQK+xSU2X7SZvgdFpAZowXCNpAFVMQrHDxZfy1h8LWPxtUxrji9TVZPmrlubSVotJSKF2vB93YSy\n+FrG4msZi69lWnt8ySRpsqsxxhhjScsYY0zSsKRVq9W2S/BYfC1j8bWMxdcyrT2+pGHPtIwxxiQN\nK2kZY4xJGu0qaYlIvoisFZEiCdO9voh0FJF53voPxcfu9cOcu7+ILBGR1SKySkRuD7PNOSKyW0SW\ne9MD8YovKIZNIvK5d/7CMOtFRH7vXcPPROSUOMY2OOjaLBeRPSLy45Bt4noNRWSOiGwXkZVBy7qL\nyJsist577RZh3xu8bdaLyA3htvEpvl+LyBrv9/eqiIQd+bCxvwUf4/uZiJQE/Q7HRdi3wc+7j/HN\nC4ptk4gsj7Cv79evTVLVdjHhGsVtAI4GOgArgBNCtpkKPOnNXwXMi2N8fYBTvPnOwLow8Z0DvJbg\n67gJ6NnA+nHAG4AApwMfJvD3vRUYkMhrCJwFnAKsDFr2K2C6Nz8d+GWY/boDG73Xbt58tzjFdwGQ\n5s3/Mlx80fwt+Bjfz4A7o/j9N/h59yu+kPW/AR5I1PVri1N7KmmNAopUdaM2oXt98al7/VCqukVV\nP/Hm9wJf4HpOTjYXA8+qswzoKiJ9EhDHecAGVf0qAec+TFWX4noICBb8d/YMcEmYXS8E3lTVb1V1\nJ/AmkB+P+FR1saoGGsIuw/U1lxARrl80ovm8t1hD8XnfHVcAL8T6vO1Ze0pa4brXD00Kh7fxPrSB\n7vXjyrstOQL4MMzqM0RkhYi8ISJD4xqYo8BiEflYRCaHWR/NdY6Hq4j8ZZHoa9hbVbd481uB3mG2\naS3X8Ye4knM4jf0t+Gmad/tyToTbq63h+p0JbFPV9RHWJ/L6Ja32lLSSgoh0Av4C/FhV94Ss/gR3\nu+tk4L+Av8Y7PuC7qnoKbhTTW0XkrATE0CBxnXqOB14Os7o1XMPD1N0napVVeEXkXlzXQ3+OsEmi\n/hZmAccAw4EtuFtwrdHVNFzKavWfpdaoPSWtpnSvj/jcvX44IpKOS1h/VtVXQter6h5V3efNLwTS\nRaRnvOLzzlvivW4HXsXdhgkWzXX221jgE1XdFrqiNVxDYFvglqn3uj3MNgm9jiIyCbgImOgl1nqi\n+FvwhapuU9VqVa0Bnopw3kRfvzTgUmBepG0Sdf2SXXtKWgXAIBHJkyZ0rx/pAxtr3v3v/wG+UNXH\nImxzZOAZm4iMwv3+4plUs0Wkc2Ae98B+ZchmC4DrvVqEpwO7g26FxUvE/3ATfQ09wX9nNwB/C7PN\nIuACEenm3f66wFvmOxHJB34KjFfV/RG2ieZvwa/4gp+RTohw3mg+7376N2CNqhaHW5nI65f0El0T\nJJ4TrmbbOlytonu9ZTNwH06ADNwtpSLgI+DoOMb2Xdxtos+A5d40DpgCTPG2mQaswtWEWgZ8J87X\n72jv3Cu8OALXMDhGAWZ61/hzYGScY8zGJaGcoGUJu4a45LkFOIR7rnIj7jnp28B64C2gu7ftSODp\noH1/6P0tFgE/iGN8RbjnQYG/w0CN2r7Awob+FuIU31zvb+szXCLqExqf977e5z0e8XnL/xT4mwva\nNu7Xry1O1iOGMcaYpNGebg8aY4xJcpa0jDHGJA1LWsYYY5KGJS1jjDFJw5KWMcaYpGFJy5hWwOt9\n/rVEx2FMa2dJyxhjTNKwpGVME4jItSLykTcG0h9EJFVE9onIb8WNg/a2iPTyth0uIsuCxqXq5i0/\nVkTe8jrt/UREjvEO30lE5ntjWf05XiMMGJNMLGkZEyURGQJcCYxR1eFANTAR1wtHoaoOBf4JPOjt\n8ixwt6qehOvBIbD8z8BMdZ32fgfXowK4nv1/DJyA6zFhjO8/lDFJJi3RARiTRM4DTgUKvEJQJq6z\n2xpqO0Z9DnhFRHKArqr6T2/5M8DLXn9z/VT1VQBVrQDwjveRen3VeaPdDgTe8//HMiZ5WNIyJnoC\nPKOq99RZKHJ/yHbN7RvtYNB8Nfb5NKYeuz1oTPTeBi4XkSMARKS7iAzAfY4u97a5BnhPVXcDO0Xk\nTG/5dcA/1Y1KXSwil3jH6CgiWXH9KYxJYvafnDFRUtXVInIfbrTZFFzP3rcC5cAob9123HMvcMOO\nPGtjYYoAAABmSURBVOklpY3AD7zl1wF/EJEZ3jG+H8cfw5ikZr28G9NCIrJPVTslOg5j2gO7PWiM\nMSZpWEnLGGNM0rCSljHGmKRhScsYY0zSsKRljDEmaVjSMsYYkzQsaRljjEkalrSMMcYkjf8Hgepf\nbHH7ERYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fa754753d50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model.compile(loss='categorical_crossentropy',\n",
    "              optimizer=optim,\n",
    "              metrics=['accuracy'])\n",
    "t = time.time()\n",
    "history = model.fit(X_train, Y_train,\n",
    "                batch_size=batch_size, nb_epoch=nb_epoch,\n",
    "                verbose=verbose,validation_data=(X_test, Y_test))\n",
    "print (time.time() - t, \"seconds.\")\n",
    "\n",
    "score = model.evaluate(X_test, Y_test, verbose=0)\n",
    "print (\"-\"*10)\n",
    "print (\"Loss: %f\"%(score[0]))\n",
    "print (\"Accuracy: %f\"%(score[1]))\n",
    "plot_curves(history,nb_epoch)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Additional exercises:\n",
    "\n",
    "- Use one of the networks to study the effect of the following components:\n",
    "    - learning rate\n",
    "    - optimizer (you can find a list of the different optimizers available in keras [here](https://keras.io/optimizers/))\n",
    "    - learning rate decay\n",
    "- Try adding weight regularization in the hidden layers of the network. You can check how to do this in the documentation for [Dense layers](https://keras.io/layers/core/) and [Regularizers](https://keras.io/regularizers/) in keras."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}