notebooks/06-data_preparation_evaluation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 06 - Data Preparation and Advanced Model Evaluation\n",
    "\n",
    "\n",
    "by [Alejandro Correa Bahnsen](albahnsen.com/)\n",
    "\n",
    "version 0.2, May 2016\n",
    "\n",
    "## Part of the class [Machine Learning for Risk Management](https://github.com/albahnsen/ML_RiskManagement)\n",
    "\n",
    "\n",
    "This notebook is licensed under a [Creative Commons Attribution-ShareAlike 3.0 Unported License](http://creativecommons.org/licenses/by-sa/3.0/deed.en_US). Special thanks goes to [Kevin Markham](https://github.com/justmarkham)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Handling missing values\n",
    "\n",
    "scikit-learn models expect that all values are **numeric** and **hold meaning**. Thus, missing values are not allowed by scikit-learn."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PassengerId</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             Survived  Pclass  \\\n",
       "PassengerId                     \n",
       "1                   0       3   \n",
       "2                   1       1   \n",
       "3                   1       3   \n",
       "4                   1       1   \n",
       "5                   0       3   \n",
       "\n",
       "                                                          Name     Sex   Age  \\\n",
       "PassengerId                                                                    \n",
       "1                                      Braund, Mr. Owen Harris    male  22.0   \n",
       "2            Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0   \n",
       "3                                       Heikkinen, Miss. Laina  female  26.0   \n",
       "4                 Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0   \n",
       "5                                     Allen, Mr. William Henry    male  35.0   \n",
       "\n",
       "             SibSp  Parch            Ticket     Fare Cabin Embarked  \n",
       "PassengerId                                                          \n",
       "1                1      0         A/5 21171   7.2500   NaN        S  \n",
       "2                1      0          PC 17599  71.2833   C85        C  \n",
       "3                0      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "4                1      0            113803  53.1000  C123        S  \n",
       "5                0      0            373450   8.0500   NaN        S  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import zipfile\n",
    "with zipfile.ZipFile('../datasets/titanic.csv.zip', 'r') as z:\n",
    "    f = z.open('titanic.csv')\n",
    "    titanic = pd.read_csv(f, sep=',', index_col=0)\n",
    "titanic.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Survived      0\n",
       "Pclass        0\n",
       "Name          0\n",
       "Sex           0\n",
       "Age         177\n",
       "SibSp         0\n",
       "Parch         0\n",
       "Ticket        0\n",
       "Fare          0\n",
       "Cabin       687\n",
       "Embarked      2\n",
       "dtype: int64"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check for missing values\n",
    "titanic.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One possible strategy is to **drop missing values**:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(183, 11)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# drop rows with any missing values\n",
    "titanic.dropna().shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(714, 11)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# drop rows where Age is missing\n",
    "titanic[titanic.Age.notnull()].shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes a better strategy is to **impute missing values**:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "29.69911764705882"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# mean Age\n",
    "titanic.Age.mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "28.0"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# median Age\n",
    "titanic.Age.median()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    24.0\n",
       "dtype: float64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# most frequent Age\n",
    "titanic.Age.mode()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# fill missing values for Age with the median age\n",
    "titanic.Age.fillna(titanic.Age.median(), inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another strategy would be to build a **KNN model** just to impute missing values. How would we do that?\n",
    "\n",
    "If values are missing from a categorical feature, we could treat the missing values as **another category**. Why might that make sense?\n",
    "\n",
    "How do we **choose** between all of these strategies?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Handling categorical features\n",
    "\n",
    "How do we include a categorical feature in our model?\n",
    "\n",
    "- **Ordered categories:** transform them to sensible numeric values (example: small=1, medium=2, large=3)\n",
    "- **Unordered categories:** use dummy encoding (0/1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PassengerId</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Moran, Mr. James</td>\n",
       "      <td>male</td>\n",
       "      <td>28.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>330877</td>\n",
       "      <td>8.4583</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>McCarthy, Mr. Timothy J</td>\n",
       "      <td>male</td>\n",
       "      <td>54.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>17463</td>\n",
       "      <td>51.8625</td>\n",
       "      <td>E46</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Palsson, Master. Gosta Leonard</td>\n",
       "      <td>male</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>349909</td>\n",
       "      <td>21.0750</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Johnson, Mrs. Oscar W (Elisabeth Vilhelmina Berg)</td>\n",
       "      <td>female</td>\n",
       "      <td>27.0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>347742</td>\n",
       "      <td>11.1333</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Nasser, Mrs. Nicholas (Adele Achem)</td>\n",
       "      <td>female</td>\n",
       "      <td>14.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>237736</td>\n",
       "      <td>30.0708</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             Survived  Pclass  \\\n",
       "PassengerId                     \n",
       "1                   0       3   \n",
       "2                   1       1   \n",
       "3                   1       3   \n",
       "4                   1       1   \n",
       "5                   0       3   \n",
       "6                   0       3   \n",
       "7                   0       1   \n",
       "8                   0       3   \n",
       "9                   1       3   \n",
       "10                  1       2   \n",
       "\n",
       "                                                          Name     Sex   Age  \\\n",
       "PassengerId                                                                    \n",
       "1                                      Braund, Mr. Owen Harris    male  22.0   \n",
       "2            Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0   \n",
       "3                                       Heikkinen, Miss. Laina  female  26.0   \n",
       "4                 Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0   \n",
       "5                                     Allen, Mr. William Henry    male  35.0   \n",
       "6                                             Moran, Mr. James    male  28.0   \n",
       "7                                      McCarthy, Mr. Timothy J    male  54.0   \n",
       "8                               Palsson, Master. Gosta Leonard    male   2.0   \n",
       "9            Johnson, Mrs. Oscar W (Elisabeth Vilhelmina Berg)  female  27.0   \n",
       "10                         Nasser, Mrs. Nicholas (Adele Achem)  female  14.0   \n",
       "\n",
       "             SibSp  Parch            Ticket     Fare Cabin Embarked  \n",
       "PassengerId                                                          \n",
       "1                1      0         A/5 21171   7.2500   NaN        S  \n",
       "2                1      0          PC 17599  71.2833   C85        C  \n",
       "3                0      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "4                1      0            113803  53.1000  C123        S  \n",
       "5                0      0            373450   8.0500   NaN        S  \n",
       "6                0      0            330877   8.4583   NaN        Q  \n",
       "7                0      0             17463  51.8625   E46        S  \n",
       "8                3      1            349909  21.0750   NaN        S  \n",
       "9                0      2            347742  11.1333   NaN        S  \n",
       "10               1      0            237736  30.0708   NaN        C  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "titanic.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# encode Sex_Female feature\n",
    "titanic['Sex_Female'] = titanic.Sex.map({'male':0, 'female':1})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# create a DataFrame of dummy variables for Embarked\n",
    "embarked_dummies = pd.get_dummies(titanic.Embarked, prefix='Embarked')\n",
    "embarked_dummies.drop(embarked_dummies.columns[0], axis=1, inplace=True)\n",
    "\n",
    "# concatenate the original DataFrame and the dummy DataFrame\n",
    "titanic = pd.concat([titanic, embarked_dummies], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>Sex_Female</th>\n",
       "      <th>Embarked_Q</th>\n",
       "      <th>Embarked_S</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PassengerId</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.25</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             Survived  Pclass                     Name   Sex   Age  SibSp  \\\n",
       "PassengerId                                                                 \n",
       "1                   0       3  Braund, Mr. Owen Harris  male  22.0      1   \n",
       "\n",
       "             Parch     Ticket  Fare Cabin Embarked  Sex_Female  Embarked_Q  \\\n",
       "PassengerId                                                                  \n",
       "1                0  A/5 21171  7.25   NaN        S           0         0.0   \n",
       "\n",
       "             Embarked_S  \n",
       "PassengerId              \n",
       "1                   1.0  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "titanic.head(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- How do we **interpret** the encoding for Embarked?\n",
    "- Why didn't we just encode Embarked using a **single feature** (C=0, Q=1, S=2)?\n",
    "- Does it matter which category we choose to define as the **baseline**?\n",
    "- Why do we only need **two dummy variables** for Embarked?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.793721973094\n"
     ]
    }
   ],
   "source": [
    "# define X and y\n",
    "feature_cols = ['Pclass', 'Parch', 'Age', 'Sex_Female', 'Embarked_Q', 'Embarked_S']\n",
    "X = titanic[feature_cols]\n",
    "y = titanic.Survived\n",
    "\n",
    "# train/test split\n",
    "from sklearn.cross_validation import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1)\n",
    "\n",
    "# train a logistic regression model\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "logreg = LogisticRegression(C=1e9)\n",
    "logreg.fit(X_train, y_train)\n",
    "\n",
    "# make predictions for testing set\n",
    "y_pred_class = logreg.predict(X_test)\n",
    "\n",
    "# calculate testing accuracy\n",
    "from sklearn import metrics\n",
    "print(metrics.accuracy_score(y_test, y_pred_class))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ROC curves and AUC"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# predict probability of survival\n",
    "y_pred_prob = logreg.predict_proba(X_test)[:, 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "plt.rcParams['figure.figsize'] = (8, 6)\n",
    "plt.rcParams['font.size'] = 14"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x7f85b7959630>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgUAAAGJCAYAAAAaBkAzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XeYXVW5+PHvS0joFgQCSEc60kFBkEgVLuAV/AEqIAIq\n5dIUrwWRWEHw2q6VKsVCsXFtFDWIFGmCIAhICwiEJiYhpL+/P9YecjJMOTNzypyZ7+d59jNn9/fs\nnJz9nrXWXisyE0mSpEXaHYAkSRoeTAokSRJgUiBJkiomBZIkCTApkCRJFZMCSZIEmBRIkqRKy5OC\niNghIn4REY9HxPyIOKSOfTaOiEkRMSMiHouIU1oRqyRJo0k7SgqWBu4CjgNm9LdxRCwDXA08CWwJ\nHA98NCJObGaQkiSNNtHOHg0jYhpwTGZe2Mc2RwGnAStk5uxq2cnAkZm5amsilSRp5OuENgVvBq7r\nSggqVwIrR8TqbYpJkqQRpxOSghWBKd2WTQGiWidJkhqgE5ICSZLUAou2O4A6PAWM77ZsPJDVuoVE\nhMM+SpJGncyMoR6jE0oKbgR2iIhxNct2A57IzEd72iEznZo8nXrqqW2PYaRPXmOv8UiZvM4DnyZP\nTlZZpf7tG6Ud/RQsFRGbRsRm1flXq+ZXrdafFhHX1OzyQ8qji9+PiI0iYl/gY8D/tDp2SZJGsnZU\nH2wF/IFS/A/wmWq6ADiM0nhwza6NM3NqROwKfAu4BfgXcGZmfq2VQUuSRp6zz4Zjjml3FK+UCeut\n1/rztjwpyMxr6aOEIjPf38OyvwETmhiWBmjChAntDmHE8xo3n9e4NYbzdX7mGTjxRPjc59odySuN\nGdP6c7a186JmiIgcae9JktQcX/wiTJ9e/nayiCAb0NCwE54+kCSpbg8/DL/5DczotyN9uO462Hzz\n5sfUKUwKJEkdb/JkuOwyuOQSeOQR2GsvWHbZ/vfbaCN4xzuaHl7HsPpAktSRHnsMLr8cLr0UHngA\n3vlOOOAAmDABFh1lP3kbVX1gUiBJ6hhPPLEgEbj33vIr/4ADYKedYOzYdkfXPiYFvTApkKSRY84c\n+POf4aqr4MorS4nAPvvA/vvDLrvAuHH9H2M0MCnohUmBJHW2hx5akAT84Q+w1lqw226w++6w3Xaw\n2GLtjnD4MSnohUmBJHWWadPKzb8rEZg2rSQBu+0Gu+4K47uPfqNXMCnohUmBJA1v8+fD7bcvSAJu\nvx3e9KYFpQFvfCMs0gkj8wwjJgW9MCmQpOHniSdKEnDVVXD11bDcciUB2H13eOtbYaml2h1hZzMp\n6IVJgSS130svlY6BukoD/vnP0jCwq1pgtdXaHeHIYlLQC5MCSWq9TLjnnpIAXHkl3HADbLLJgtKA\nrbZqT1/+o4VJQS9MCiSpNZ57rlQFdFULjB1bEoDddiv9BrzmNe2OcPQwKeiFSYEkNcecOXDTTaUk\n4Kqr4L77SnuArtKAN7wBYsi3JQ2GSUEvTAokqXEefHBBEjBpEqy99oLSgO22s/Og4cKkoBcmBZI0\nNDfcABdfXJKBGTMWJAG77grLL9/u6NQTk4JemBRI0tDssANsuSUcdljpM8AqgeGvUUnBKBtHSpJU\nj/32K08PaHSxzyhJkgSYFEiSakyfXnofHM3DEI9mJgWSJABefBH22gsmTIBttml3NGoHkwJJEjNm\nwN57w5prwtlnOyDRaOU/uySNci+9BPvsA6usAuecY0IwmvlPL0mj2EsvwTveASuuCOef7/gEo51J\ngSSNUjNnwjvfWYYxvuACEwKZFEjSqDRrFuy7bxm06MILTQhU2HmRJDXIAw/AF78I997b7kj699xz\nsPnmpTvjRb0TqGI3x5I0RA8+CJ/7HPzqV3DccbDLLsO/a+AxY0pSYEIwMtjNsSS12UMPwec/D1dc\nAcceW0oKXvOadkclDZ5tCiRpgB55BD7wgdLBzyqrlGTg1FNNCNT5TAokqU6TJ8ORR5YRBMePh/vv\nh89+Fl772nZHJjWGSYEk9ePxx+Hoo0sd/GtfC/fdV6oNll223ZFJjWVSIEm9eOKJ0lZg001hmWXg\n73+H004rz/VLI5ENDSV1pEMPLb/gm2X+fLjjDjjssPKI4QorNO9c0nDhI4mSOtIyy8APfgBLLtm8\nc7zxjaXtgDTcNeqRRJMCSR1pmWVK8f4yy7Q7Eqn97KdAUlvNng133gntysHnzWvPeaWRzKRA0oDN\nnFlG1nvkEXj1q9sTw7bbwuKLt+fc0khlUiBpQLpG1lt22dKtr93kSiOHjyRKqtusWbDffqUe/6KL\nTAikkcb/0tIodOaZcMMNA9/v0UdhrbVKq38TAmnk8ekDaRTabjvYe29Yf/2B7Td2LOy+e/krafjw\n6QNJQ7LjjiU5kKQuJgXSCJFZRuur51G9GTOaH4+kzmNSII0QN98MEybAGmv0v+2ii8KKKzY7Ikmd\nxqRAGiFmz4attoLrrmt3JJI6lY8kSpIkwJICqe0y4RvfgFtvHdpxpkyBGHLbY0mjmUmB1EaZcMop\ncMUVcNJJQ7+pr7deY+KSNDqZFEhtNHEi/OIX8Pvfw/LLtzsaSaOdSYFGlWefhZdeancUxbnnwuWX\nwx/+YEIgaXgwKdCoMWsWrLwyjB/f7kiK1VYrJQQrrNDuSCSpMCnQqDFvXnk+/7HH2h2JJA1PPpIo\nSZIASwrUZLNnwxe+AE8/3e5IYM6cdkcgScObSYGaZs4cOOCAUpe/997tjqbYY492RyBJw1ddSUFE\nbAfsCKwBLAE8A9wOXJOZUwZ60og4GjgJWAn4G3BCZv6pj+13B04FNgZmAdcDH83MBwZ6brXGnDlw\n4IGlHv/nP4dx49odkSSpP5GZPa+IGAccC/wXsApwN/AE8BKwLLAR8Frgl8DnMvMvdZ0w4gDgIuBI\nys39GOD9wAaZ+XgP268B3At8DTgbWBo4A1grM9ftYfvs7T1paGbOLFN/5s+HI48sI/H95Cew2GLN\nj02SRrOIIDOH3KdpX0nBI8BdwPeBX2fmK57ujoj1gXcDHwBOzszz+z1hxE3AHZl5ZM2y+4HLMvPk\nHrbfD/gxMK7rbh8RE4DfActn5vPdtjcpaJJNNoGHH4YxY/rf9m1vgx/9CBZfvPlxSdJo16ikoK/q\ng/0y87a+ds7MvwOnRsTplKqFPkXEWGBL4Mxuq64Ctutlt1uAOcAREXEusBRwKHBz94RAzfXii3DH\nHbD22u2ORJLUDL0+klibEET03SN7Zr6UmffWcb7lgDFA93YIU4AeR3fPzMnAbsBnKe0JXqBUXQyT\npmuSJI0M9T598HhEnA+cn5kPNjOg7iJiPHAucAHwI2AZSoJwGfC2VsYyGpx/PtzbS3r33HOtjUWS\n1Fr1JgWnUxoDfiIi/gicA/wkM+todraQZ4F5QPeOZscDT/WyzzHA9Mz8eNeCiDgYeCwitsvMG7rv\nMHHixJdfT5gwgQkTJgwwzNHrzDPLY3s9dQX8mc+UrnklSe01adIkJk2a1PDj9trQsMeNIzYHDqM0\nLhxD+eV+bn9tD7odo6eGhvdRGhp+qoftvwy8NTO3qVm2EvDPavmfum1vQ8Mh2HDDMkjPhhu2OxJJ\nUr0a1dBwQN0cZ+ZfMvNYYGVgIiVBuDki7oyIQ/tre1D5CnBoRBweEetHxNcp/RV8FyAiTouIa2q2\n/xWwRUScEhFviIgtgPOByUDdyYgkSerbgHo0jIgxlAZ+hwF7UG7K51KShC8BOwMH93WMzLw0IpYF\nTqYkA3cDe9T0UbAisGbN9n+IiPcA/w18FJgB3AS8vafHJCVJ0uDUVX0QERtSEoGDgLHAD4CzM/Ou\nmm02AW7KzCWbFGtdrD4YGqsPJKnztKKfglp3A9cCHwEuz8xZPWzzEPCLoQYkSZLao96kYL3+xhnI\nzOmUBogaZp59Fr73vTIOQX+eeab58UiShqd6Gxr+omoHsJCIeHVE3NPgmNRgZ54J114Lc+f2Px1/\nPKy5Zv/HlCSNPPW2KZgPrJiZT3dbPh6YnJnDZsgb2xQsbPZsWHVVuO46WPcVw0dJkkaClrQpiIg9\na2Z3joh/18yPAXahPBqoYeqKK2CDDUwIJEn9669NwS+rv0l54qBWAo8DJzQ6KDXO2WfDBz7Q7igk\nSZ2gz+qDiFgMCOBhYGugthna3Myso+laa1l9sMDDD8PWW8PjjzuEsSSNZC2pPqh59HCloZ5IrXfu\nuXDQQSYEkqT69FpSEBFHA+dl5szqda8y89vNCG4wLCko5s6F1VeHq66CjTZqdzSSpGZqVElBX0nB\nk8DGmflc9bo3mZkrDzWQRjEpKK64Ak4/HW54xRiSkqSRpunVB5m5Uk+v1RnOOgs++MF2RyFJ6iT1\n9lOwfmb+vQXxDJklBaVh4SabwGOPwVJLtTsaSVKztXro5Hsi4taIOL7qsEjD2HnnwYEHmhBIkgam\n3qRgM+B3wIeBxyLitxHx3oho64iIeqV58+Ccc+ybQJI0cHUlBZn518z8WGauDuxK6cXwG8CUiLio\nmQFqYK66CsaPh803b3ckkqROU1ebgh53jNgCOBfYJDPHNDSqIRjtbQr23Rfe/nYbGUrSaNLqNgVd\nJ319RJwUEX8BbgFmAP811CDUGE8+CX/4A7zbAawlSYPQ39gHAETE4cB7gbcC/6CMg7BvZj7cxNg0\nQN//PrzrXbDMMu2ORJLUiep9JPEp4BLg4sy8pelRDcForT6YPx/WWQd+9CPYZpt2RyNJaqWWjH1Q\n4/XDcfAjLXDzzWWMg623bnckkqRO1WtSEBEbAn/PzPnAehG9JyCZeU8TYtMATJ0Kr3899PHPJElS\nn/oqKbgbWBF4unqdlGGUa8vmu+aHzdMHkiRpcPpKCjYAnql5LUmSRrC+BkS6r2b2X5n5dE/bRcQK\nDY9KkiS1XL39FDzZ080/Il4H9DWssiRJ6hD1JgW9NV9bCpjZoFgkSVIb9flIYkScUb1M4NMRMaNm\n9RjgzcBdTYpN/fjwh+Gaa8rradNgww3bG48kqbP110/BDtXfoCQAc2rWzab0bnh6E+JSHW65BU48\nEbbcssyvvHJ745EkdbY+k4LM3BYgIn4EfCgzp7YkKtXtDW+ATTZpdxSSpJGgrh4NM9MhdiRJGuH6\n6tHwUuCIzJxave5VZu7f8MgkSVJL9VVSMI8FvRc67oEkSSNcX50Xvbun15IkaWSqd5TEhUTEOGAb\n4MHMtPOiJpo4Ea66qud1d90Fiw7qX1CSpFeq65YSEWcBt2Xm9yJiUeBGYHNgVkTsk5lXNzPI0ey6\n6+DAA2GrrV65bsyYnpdLkjQY9f7O/A/ge9XrfYAVgDWAQ4HPAiYFTbTRRrDddu2OQpI00tXbzfHr\ngCnV67cDl2fmZOBCYKNmBCZJklqr3qRgCrB+RCwC7A78rlq+FD6ZIEnSiFBv9cGFwCXA45QxD7qq\nC7YG7uttJ0mS1Dnq7dHwlIj4O7Aa8OPMnFWz/5ebFZwkSWqduh9oy8wf9LDsnMaGI0mS2qXupCAi\nxgNvoTx5sFBbhMz8doPjkiRJLVZvPwX/j9KuYAzwPAu6P6Z6bVIgSVKHq7ek4DTKjf8TmTm7ifFI\nkqQ2qfeRxJWAb5kQSJI0ctWbFFwJbNnMQCRJUnvVW31wBXBmRKwH3AXMqV2Zmb9udGCSJKm16k0K\nzqv+fraHdUlpgKgGePJJOP54mFf1E/nXv0JEe2OSJI0O9SYFSzQ1Cr3s4Yfhzjvhi18s8wcf7GBI\nkqTWqLdHw1n9b6VGed3rYL/92h2FJGm0qbehIRFxWETcFhHPR8Qa1bKTIuKdzQpOkiS1Tl1JQUQc\nQ+mr4FJKVULXfs8AxzcnNEmS1Er1lhQcA3wgM78EzK1ZfhuwccOjkiRJLVdvUrAmcGcPy2cBSzUu\nHEmS1C71JgWPAJv2sHx34N6GRSP+/e92RyBJGq3qfSTxq8A3I2IsEMAW1SBJnwKOalZwo83f/w5H\nHAFnntnuSCRJo1FdJQWZeRZwJvAtYElKg8OPAB/PzIsHetKIODoiHoqIlyLi1ojYvo59ToiIeyNi\nZkT8MyK+ONDzDmf33w+77AJf+AK85z3tjkaSNBpFZva/Ve0OEatQkonHcqA7l/0PAC4CjgSupzRi\nfD+wQWY+3ss+XwH2BE4C7gZeDayUmb/tYdvBhNVyU6bA3KrJ5lNPwX/+J3zmM3DYYe2NS5LUeSKC\nzBxy/7cDTgqqk29DaWB4S2ZOH+C+NwF3ZOaRNcvuBy7LzJN72L5rvIWNM/P+Oo4/7JOCv/0NNtsM\nVlihzC+yCEycCIcf3tawJEkdqlFJQZ9tCiLiQ8BrqkcRu5b9FHgHpW3B4xGxc2Y+UM/JqjYJW1Kq\nImpdBfTWme8+wIPAnhHxG0opxbXARzPzmXrOO9zMnAmbbAK33dbuSCRJWqC/NgWHAU91zUTEPpSb\n9AeB7YEpwCkDON9ylMGTpnRbPgVYsZd91gLWAA4ADgEOAtanjNwoSZIapL+nD95A6aCoy38Av8zM\ncwEi4uPAuU2KrcsiwDjgoMx8sDrvwcB9EbF1Zt7S5PPX7ec/h2uu6X+7p59ufiySJA1Uf0nBEsC0\nmvltWTCMMsADwPgBnO9ZYF4P+4ynpkSimyeBuV0JAUBmPhAR84DVgFckBRMnTnz59YQJE5gwYcIA\nQhy8H/4Qll4attii7+3WXx8++MGWhCRJGoEmTZrEpEmTGn7cPhsaRsS9wMmZ+dOIWI5y496269d5\nRGwNXJGZK9V9wp4bGt5HaWj4qR623xX4LfCGzHy4WrY2JSHZJjNv7bZ92xoa7r8/vOtd5a8kSa3S\nkoaGwMWUTovWB3YC/tGtuP7NwN8GeM6vABdGxC2URxKPAlYCvgsQEacBW2fmLtX21wC3A+dFxImU\nBo5fBW7snhBIkqTB6y8pOJ3SJ8DBlFKC7r+BdwYuH8gJM/PSiFgWOJmSDNwN7FHTR8GKlLEWurbP\niNgL+AblqYOXKE8rfGQg55UkSX0bVD8Fw5nVB5Kk0aZR1Qf1DogkSZJGuF6Tgoi4KyLeFRH9dXC0\nZkT8b0R8rPHhdZYRVugiSRpl+rrhnwScAXwnIq4CbgWeAGYCrwU2pHRgtBmlkeBZzQ11+HvgAVh1\n1XZHIUnS4PTbpiAidgLeDewArE7pSOhfwF+AK4ELhlN3w+1qU/D887D66uXv2LEtP70kaRRr1SOJ\nZObvgd/XnHj4jzjUBtddB9tua0IgSepcA25oaELQs2uvhR13bHcUkiQNnk8fNMikSdCi3pQlSWoK\n+ylogBdeKA0Mn3sOxo1r6aklSbKfguHkT3+CbbYxIZAkdTaTggaw6kCSNBLUnRRExNiI2Csijo+I\nV1XLVu16PZrZyFCSNBLU1aYgItYArgbGA0sC62bmQxHxNWCJzPxQM4MciFa3KZg6FVZeGZ59FhZf\nvGWnlSTpZa1uU/B1yjDHr6OMUtjlZ5SREketP/0Jtt7ahECS1Pn67byosj2wXWbOiVgoEXkUWLnh\nUXUQqw4kSSNFvSUFiwBjeli+CjCtceF0nmuvtZGhJGlkqLek4GrgWOCoaj4jYingVOC3zQhsuLrl\nFrjhhvI6E+6+G970pvbGJElSI9Tb0HA1YBIwHdgAuAlYl1JKsH1mPtXEGAek2Q0NDz4YpkyBDTYo\n8+uuC8cc07TTSZLUr5YNiASQmZMjYhPgYGBLSnXCJZQREkdd9cHBB5dJkqSRpK6kICK2AW7LzO90\nWz4mIrbJzJubEp0kSWqZehsa3kh5HLG711TrJElSh6s3KQigp4r61wIzGheOJElqlz6rDyLi0upl\nAudExKya1WOATSmNDiVJUofrr6RgXjUFML9mfh7lSYQfUBofSpKkDtdnSUFmvhsgIh4BPp+ZL7Yi\nKEmS1Hr1PpL4iWYHIkmS2qveHg2JiHcD7wZWA8bVrsvMDRsclyRJarG6nj6IiBOA7wIPAusDvwce\nowyGdHnTopMkSS1T7yOJRwEfzMwTgTnAVzJzd+AbwPLNCk6SJLVOvUnBqix49PAlYJnq9UXA/o0O\nSpIktV69bQqmAMsCjwKTgW2AO4HVKY8rjljz58MVV8CsqoeGRx5paziSJDVNvUnBH4C9gL8AFwBf\ni4h9gTcBv2hSbMPCY4/Be94De+9d5l//eth88/bGJElSM9Q7dPI4YNHMnFHNvw94C3A/8L+ZOauv\n/Vup0UMnP/IITJhgCYEkafhq9dDJs4HZNfMXUEoMJEnSCFFvQ8MeRcReEXF7o4KRJEnt029SEBEH\nR8RFEXFeRGxRLXtzRNwE/AS4q9lBSpKk5uszKYiI44HzgM0pvRleWy37NaXx4ZqZ+b6mRylJkpqu\nvzYFHwT+KzO/FxG7AlcC+wLrZuazTY+uTSZPhrvvLq+nTGlvLJIktUp/ScEawG8BMvPqiJgLfHwk\nJwQAn/sc3HQTrLpqmd9vv/bGI0lSK/SXFCxB6cGwyyxKR0Yj2vz5cMIJcPjh7Y5EkqTWqeeRxEMj\nYnrN9gdFxEIlBZn57YZHJkmSWqq/pOBp4MSa+RcogyPVSqCjk4J77oHjjoOZM8v8Aw/ADju0NyZJ\nklqtz6QgM1dsVSDtcu+9sMsu8MlPwmabLVi+xRbti0mSpHaod+yDEem++0pCcPrpcMgh7Y5GkqT2\nGlKPhp3sgQdg553hC18wIZAkCeocEKmT1Dsg0rHHwjLLwBe/2IKgJElqokYNiDRqSwrmzYNVVml3\nFJIkDR+jNimQJEkLqzspiIix1aiIx0fEq6plq3a9liRJna2upw8iYg3gamA8sCTwf8BU4COUXg8/\n1JzwJElSq9RbUvB14HrgdSzc7fHPgJ0bHZQkSWq9evsp2B7YLjPnRCzUuPFRYOWGRyVJklqu3pKC\nRYAxPSxfBZjWuHAkSVK71JsUXA0cWzOfEbEUcCrV0MqSJKmz1Vt9cBIwKSL+CiwOXAisSyklOLhJ\nsUmSpBaqKynIzMkRsQlwCLAFpYThEuCCzLT6QJKkEaDeRxJfnZn/psOHSJYkSb2rt03BUxFxeUS8\nIyLGDvWkEXF0RDwUES9FxK0RsX2d+60TEdMiYupQY5AkSQurNyk4AJgL/JCSIHw3It4ymBNGxAHA\n14DPA5sBNwC/iYg+RyKokpEfAZMGc15JktS3upKCzLwiMw+k9Gj4YWBNSsPDhyLicwM854nAeZl5\nXmbel5nHAU8CR/Wz3xnAncDlAzyfJEmqw4AGRMrM6Zl5QWbuDmwK/Bv4ZL37V7/2t6Q84ljrKmC7\nPvb7D2BPFn4sUpIkNdCAkoKIWCwi3hURPwNup3R7/OUBHGI5SidIU7otnwKs2Ms5VwbOAt6bmTMG\nEq8kSapfvU8f7Ay8F9i3WvQTYA9gUmZmk2LrchHw7cy8tSucwRxkzhw44QSYUaUVN94IG2/cmAAl\nSRoJ6u286NeUngs/AFyRmbMGeb5ngXmUtgm1xgNP9bLP24AdImJiNR/AIhExGzg6M8/pvsPEiRNf\nfj1hwgQmTJjA1Knw/e/DN79Zlr/1rbDnnoN8F5IktdGkSZOYNGlSw48b9fzQj4hlM/P5hpww4ibg\njsw8smbZfcBlmfmpHrbfsNui/6S0Y9gaeKLqP6F2+x4LL557DtZdt/yVJGkkiQgyc1Al6bV6LSmI\niCVr6vBnRsSSvW07wLr+rwAXRsQtlOGYjwJWAr5bnfc0YOvM3KU69j3d4toamJ+Z9w7gnJIkqR99\nVR9Mi4iVMvNpYDrQV5FCTyMo9igzL42IZYGTKcnA3cAemfl4tcmKlEceJUlSC/VafRARuwO/y8y5\nEfF2+kgKMvPKJsU3YFYfSJJGm6ZXH9Te6DPT4ZElSRrh6uqnICJmRMTyPSxfNiLsO0CSpBGg3s6L\nFqfn/gEWH8AxJEnSMNZnPwURcXT1MoFDI2J6zeoxwI7A/U2KTZIktVB/nRedUv0N4CPA/Jp1s4FH\ngKMZpubPX9CD4fTpfW8rSdJo12dSkJkrAUTEjcCemfmvlkTVIKeeCl/6EowbV+bXWae98UiSNJzV\nO3Tytp2WEABMmwZnnFFKCaZPh7/8pd0RSZI0fPXVo+EZwGcy88Xqda8y878bHpkkSWqpvqoPdgDG\n1rzuTbNHSZQkSS3QV+dF2/b0WpIkjUyD7mMgIlaJiHqHXpYkScNcvT0aToyIg2rmfwlMBp6KiK2a\nFZwkSWqdeksKDgUehJcHStoWmABcBpzejMAkSVJr1Vv8vyLQNbTxnsBlmfnHiHgSuLkpkUmSpJaq\nt6TgeWCV6vXuwO9q9h/T6KAkSVLr1VtS8HPg4oi4F1gB6BpKeVOqagVJktTZ6i0pOAE4D/gn8PbM\nnFYtXx04qxmBSZKk1qqrpCAzZwNf6GH5mQ2PSJIktUXd/QxExLLAkcCGlF4M/waclZnPNyk2SZLU\nQvX2U/AmStuBI4HFgMUpQyb/IyK2bl54kiSpVeotKfgfSmPDD2TmXICqN8NzgK8C2zcnPEmS1Cr1\nJgVbAkd0JQQAmTm3Gj3x1qZEJkmSWqrepw+mAav2sHyVap0kSepw9SYFlwLnRsR+EbFSNb0LOLta\nJ0mSOly91QcnAWOBH7MgkZhPaVPw0SbEJUmSWqzefgpmAh+KiI8B61SLH8jMF5oWmSRJaql+k4KI\nWBnYifIo4rWZeUvTo5IkSS3XZ1IQEdsBvwZeVS2aHREHZeblTY9MkiS1VH8NDT8P3ASsTXnS4IfA\nl5sdlCRJar3+qg82Bd6WmQ8DRMTxwAsR8RrbE0iSNLL0V1LwWuCprplqdMQZ1XJJkjSC1PP0wboR\nsVzNfADrRMQSXQsy856GRyZJklqqnqTg2m7zAfyWMlJiVH/HNDguSZLUYv0lBRu0JApJktR2fSYF\nmXlfqwKRJEntVe/YB5IkaYQzKZAkSYBJgSRJqpgUSJIkYIBJQUQsHRGbRsTYZgUkSZLao66kICKW\niogLganAbcCq1fJvRsTJTYxPkiS1SL0lBacB6wHbATNrll8F/L9GByVJklqvnh4NAd4B7J+Zf46I\nrFl+D7BW48OSJEmtVm9JwfLA0z0sX6qBsUiSpDaqNym4DdizZr6rtOAw4MaGRiRJktqi3uqDk4Ff\nR8T61T5jLlKEAAAVFklEQVTHRMRGwARgxybFJkmSWqiukoLM/CPl5r8C8E9gX+BF4C2ZeXPzwpMk\nSa1Sb0kBmXkbcEATY5EkSW1UV1IQEUv2tT4zZzQmHEmS1C71lhRMZ0Hjwp6MaUAskiSpjepNCvbo\nNj8W2Bw4AjiloRFJkqS2qCspyMwre1j8y4i4HzgIuLChUUmSpJYb6iiJtwI7NSIQSZLUXoNOCiJi\nHHAM5RFFSZLU4ep9+uAZFm5oGMBrgNnAIU2IS5IktVi9DQ0/1W1+PvAMcENm9jQmgiRJ6jD9JgUR\nsSgwB/h1Zj7ViJNGxNHAScBKwN+AEzLzT71suyNwIrAN8GrgH8DXMvP8RsQiSZKKftsUZOZc4JvA\nYo04YUQcAHwN+DywGXAD8JuIWKWXXbYD/grsB2wEfAc4KyIObEQ8kiSpqLeh4c3Apg0654nAeZl5\nXmbel5nHAU8CR/W0cWaelpmfzswbM/ORzPwu8FNKkiBJkhqk3jYF3wT+JyJWpgyj/GLtysy8p56D\nRMRYYEvgzG6rrqKUCNTrVcBjA9hekiT1o96k4NLq77erv11PIkT1ut5ujpertp3SbfkUYOd6DhAR\ne1H6RhhIEiFJkvpRb1KwQVOjqFNEvAX4AXBsNWqjJElqkD6Tgog4Dzg+M+9r0PmeBeYB47stHw/0\n+WRDRGwP/Ar4VGae1de2EydOBOCmm2Du3AnAhEEFK0nScDRp0iQmTZrU8ONGZu+DH0bEPGClRvZF\nEBE3AXdk5pE1y+4DLsvM7v0hdK1/K/BL4JTM/Ho/x8+u93TCCbDGGuWvJEkjVUSQmTHU4/RXfTDk\nE/TgK8CFEXELcD3lqYOVgO8CRMRpwNaZuUs1P4GSEHwL+HFEdJUyzMvMZ5sQnyRJo1I9bQp6L0oY\nhMy8NCKWBU6mJAN3A3tk5uPVJisCa9bs8j5gCUpnRyfVLH8UWKuRsUmSNJr1V30wnzqSgsys9+mD\nprP6QJI02rSq+gDgg8ALQz2RJEka3upJCv7PQY8kSRr5+uvmuKHtCSRJ0vDVX1LQjKcPJEnSMNRn\n9UFm1jtg0rDydFXZMWNGe+OQJKmT1NvNcUfZeOMFr3fbrX1xSJLUSfp8JLET1T6SKEnSaNCoRxI7\nsnpAkiQ1nkmBJEkCTAokSVLFpECSJAEmBZIkqWJSIEmSAJMCSZJUMSmQJEmASYEkSaqYFEiSJMCk\nQJIkVUwKJEkSYFIgSZIqJgWSJAkwKZAkSRWTAkmSBJgUSJKkikmBJEkCTAokSVLFpECSJAEmBZIk\nqWJSIEmSAJMCSZJUMSmQJEmASYEkSaqYFEiSJMCkQJIkVUwKJEkSYFIgSZIqJgWSJAkwKZAkSRWT\nAkmSBJgUSJKkikmBJEkCTAokSVLFpECSJAEmBZIkqWJSIEmSAJMCSZJUMSmQJEmASYEkSaqYFEiS\nJMCkQJIkVUwKJEkSYFIgSZIqJgWSJAkwKZAkSRWTAkmSBJgUSJKkSluSgog4OiIeioiXIuLWiNi+\nn+03johJETEjIh6LiFNaFaskSaNFy5OCiDgA+BrweWAz4AbgNxGxSi/bLwNcDTwJbAkcD3w0Ik5s\nTcSSJI0O7SgpOBE4LzPPy8z7MvM4yg3/qF62PwhYAnhfZt6bmT8FvgR8uDXhqieTJk1qdwgjnte4\n+bzGreF17hwtTQoiYizl1/7V3VZdBWzXy25vBq7LzNk1y64EVo6I1Rsfperhf/Lm8xo3n9e4NbzO\nnaPVJQXLAWOAKd2WTwFW7GWfFXvZPvrYR5IkDZBPH0iSJAAiM1t3slJ9MAM4MDN/UrP8m8BGmfm2\nHva5AFg2M/euWbYV8Gdgrcx8tNv2rXtDkiQNE5kZQz3Goo0IpF6ZOScibgN2BX5Ss2pX4LJedrsR\nOD0ixtW0K9gNeKJ7QlCdY8gXRZKk0agd1QdfAQ6NiMMjYv2I+DqwEvBdgIg4LSKuqdn+h5TShe9H\nxEYRsS/wMeB/Wh24JEkjWUtLCgAy89KIWBY4mZIM3A3skZmPV5usCKxZs/3UiNgV+BZwC/Av4MzM\n/FprI5ckaWRraZsCSZI0fHXc0wd2kdx8A7nGEbFjRPw8Ip6IiBcj4s6IeH8r4+1UA/0s1+y3TkRM\ni4ipzY6x0w3mGkfECRFxb0TMjIh/RsQXWxFrpxrEd/LuEXFDREyNiGeq7491WhVvp4mIHSLiFxHx\neETMj4hD6thn0Pe9jkoK7CK5+QZ6jSmdTv0V2A/YCPgOcFZEHNiCcDvWIK5z135jgR8Bk5odY6cb\nzDWOiK8ARwIfBdYH9gT+2PxoO9MgvpPXAH4OXFttvzOwOPCrFoTbqZYG7gKOo7Sv69OQ73uZ2TET\ncBPw3W7L7ge+0Mv2RwEvAONqlp0MPNbu9zJcp4Fe416OcQlwWbvfy3CeBnudga8C5wLvA6a2+30M\n52kQ3xfrAbOBddsde6dMg7jG+wFzqKquq2UTgHmUR8/b/p6G8wRMAw7pZ5sh3fc6pqTALpKbb5DX\nuCevojQIVQ8Ge50j4j8ov1yPbV50I8Mgr/E+wIPAnhHxYEQ8HBHfj4jlmxhqxxrkNb6FkhQcERGL\nVL9qDwVuzsznmxXrKDOk+17HJAXYRXIrDOYaLyQi9gJ2Ar7X2NBGlAFf54hYGTgLeG9m9luEqEF9\nltcC1gAOAA6hDMa2PnBFc0LseAO+xpk5mdLPzGeBWZRftBsBe/e0vQZlSPe9TkoKNMxFxFuAHwDH\nZuZt7Y5nhLkI+HZm3lrN20lX4y0CjAMOyszrM/N64GDgTRGxdXtDGxkiYjyl+usCYCtgR0qReG+d\n16nFOikpeJZS7zS+2/LxwFO97PNUL9tnH/uMZoO5xgBULY5/DXwqM89qTngjxmCu89uAUyNiTkTM\nAc4Blo6I2RFxRPNC7ViDucZPAnMz88GuBZn5QHWc1ZoRZIcbzDU+BpiemR/PzDsz80+UxGvHiBhI\nFaV6N6T7XsckBZk5B+jqIrnWrsD1vex2I7BDRIyrWdZrF8mj3SCvMRHxVkpC8OnM/N/mRTgyDPI6\nb0xprb1pNX2a0hJ5U/yV9QqDvMbXA4tGxMudp0XE2pQicr8vuhnkNV6SkkjUml/97Zj70TA3tPte\nu1tTDrDl5f7ATOBwSl3f14GpwCrV+tOAa2q2fxXwBKWr5I2AfYF/Aye0+70M12kQ13gCMB34EiUb\n7ZqWa/d7Gc7TQK9zD/v79EGDrzGlSuYW4A+UBGxzyqOf17f7vQzXaRDX+G3AXOAU4A3AFsBvgUeA\nJdr9fobjBCxFSf43A14EPlXNr9rLNR7Sfa/tb3gQF+hI4CHgpeo/8Ftq1p0PPNht+42q/9gzgH9S\nirfb/j6G8zSQa1zNz+theqjd72O4TwP9LHfb16SgCdeYktBeUn2JPgVcCCzf7vcxnKdBXOP9gVur\n5OEpSr8F67f7fQzXidLuYn4P37Hn9XGNB33fs5tjSZIEWIcjSZIqJgWSJAkwKZAkSRWTAkmSBJgU\nSJKkikmBJEkCTAokSVLFpEAdIyLGRMT8iNin3bEMVkSsXb2HTfrZ7qKI+Gmr4hpuIuLCiPh4u+No\nlZ4+2xGxYUTcGBEvRcT9A/38R8ThETHk4Ygj4qcRcdxQj6POYFKglomI86svtXnV367Xfd4gWyki\nPlcT19yIeDQivhcRyzboFA9Rhi+9uzrfztX5XtVtu6Mp48w3Tc25u/49no2IayLiTQM8TkOTtYjY\nDNgD+EbNsv0i4sqIeLo6V8MGz4mIt0XE76r3/2JE/KNKSpZs1Dn6k5nzKJ+L39Qs/gKlZ8V1gDf3\nsk1fLgbW7ZqpPtt/GUR4nwVOiYilBrGvOoxJgVrtasoXW9e0EtUNchi5mxLbqpRR3d4JnNeIA2fx\ndGZ2DQITlNHLott20zJzaiPO2V9IlBvHipRxLP4F/HqASVCjh3E+FrgsM2fULFsK+BPwEUrMDRER\nG1MG87oVeCule9gjKcP5jutj14arPhdzaha9AbguMx/PzOd72aav483KzGe7Lx5EXHcAjwPvGei+\n6kDt7tfZafRMlD66r+hj/R7AdZQb03OUL+t1a9aPofQBvk/NsomUwVRmUgYBObdmXQCfAB6k9AF+\nJ3BgPzF+Dri927JTgFnAotX8JsDvqmM+Sxkffpma7bvW/5vSv/vtwA7VurWr97BJzet5NX/Pqra7\nGPhp9foo4J89xHopcHnN/Dsoo9a9VL3nzwJj+3ivO1fnfFXNss2qWHavWbYNcBXwTPWe/ghsXbP+\nsZr3MB+4fwgxjamu2R69rB9fnWO7Bn0mP0I/43RU12k+sCdwR/VebgY267bd9tW1mVFdk28CS3f7\nPP43cH/1eX0U+Gz3z3bN69rPxSfp+fP/euBH1efwxepad33WjgD+Vb0+vIdjvge4APhZt/exCCUJ\n+K+aZZ8Bft+u7w6n1k2WFGg4WRL4MrAl5Vfri8AVETGmp40j4gDgeOCDlF9Ve1MGZOlyOnAQ8CFg\nA8pIjudERPehXvszi/KFPLYqQr2SkrRsRRmB7K3AWTXb/xiYXK3fjHIjnFmzvuvX2kOUwWGgFBGv\nBHy42zZQBuh5XUTs1LUgIpYB9gIuqub3BL4PfK16r4cDB1C+zPsT1TGWAt5fnbv21+gy1bHfQkkQ\n/gr8JiJeXa3fujrG+yglDm8eQkybU0oFbq0j7kZ4ClixGv67P2dQ/n22pNz0/y8iFoOXqzx+SxnG\nemNgv2q7s7vt/9+UxHMD4F2Um+9CckE1wYOUz/BKwFe7bxcRS1OS6JUpn/2NWPjaJgs+Rz+g/Dv8\njZJYrQRcXsW3Z0QsV7PfHsCylMS0y83AmyNi0V6ujUaKdmclTqNnopQUzKEUzXZNv+pj+1dRftFs\nU80v9EsJ+CilqH9MD/suTflF96Zuy/8X+Hkf51yopIDy5f0g8Mdq/ijKr7LFa7bp+iW5ejU/HXh3\nL8d/uaSgZt+Ffq1Xyy+iKimo5n/BwqUgh1ISk7HV/PXAx7odYz/ghT7ea1fcU6t/i65f+jcAi/Sx\nXwBPA/v39O9Ss91gYtoPmN3H+kaXFCxCSVzmURKEX1ASzdf1cJ3eVbNsGUqpySHV/A+A73Q79lbV\nfq+pPsszgff3EkdPpQD3Ap/sbZvqs/gC8Opejnk48Hxvn+2a5fcAH66Zvxz4YbdtNq+u0aqNuO5O\nw3eypECtdi2l6HzTajqia0VEvCEifhgRD0bEvylDfgKs1suxLqF82T4cEWdXjdHGVus2BhYDro6I\naV1Tdb61+olxk4iYGhEzgLsoScEh1br1gTszs/aX//XV3w2qv18BLoiIqyPiExGxTj/nq8fFwL4R\n0VXP/R5KvXvXL/otgU93e68XAktHxOv6OG5Sir03Bw6klF68Lxe0eSAiVoiIsyLivoh4gZJELEvv\n/y5dBhPTEpSSmSGJiDVrzjs1Ik7qabvMnJ+ZhwKrUKoSHgM+Dvw9Itat3RS4qWa/aZRf3RtWi7YE\nDu32XidRbuJrU37FjwV+P9T3VmMz4C+Z+e8hHuccSgkRVYnB3tWyWi9Vf5cY4rk0zFkUpFabkZkP\n97Lu15Qb8BGU9gHzKb+WemzwlZmTqxvuLpRfc18FPhUR27KgEe2e1bFqze4nxr9TvhjnA09knQ27\nqIpqM/PTEXFhde7dgYkRcURmXlTncXpyBaWod++IuB7YiYWLigM4FejpMcb+Hkt7JEujxn9URdI/\nj4hNshRjQ/kV/CrgOEq1yCxKctdfQ7zBxPQssGRELJqZc/s5fl8mU5LOLs/1tXFmPkl5nz+IiE8B\n/wBOolRN1WMR4HvA13llw8vHKUnDcHUh8IWI2IZSRfTPzOyevHQ1PH2mpZGp5UwKNCxExAqUdgHv\nz8zrq2Xb0M8TMpk5C/gV8KuI+DLlC/jNlMZ9sylF+n8aYDiz+0hc7gXeGxFLZGbXr6ftKQnBvTVx\n/YPySN03IuIsSlFuV1JQ216gK0Hpsd1EzfFmRcRPKG0kVgUmd12nyl+A9TLzoX7fXd++T2lYeRSl\noRyUG8UHMvNKgIhYiVLn3RXbvIiY18N7GExMXY/MbUhpuzAoVUIzqGuRmS9ExBRKFVSXoHyuLoeX\n23RsBHy3Wn87sGFvn5uI+Bul6mxnGvQkC+Va7R8Rr8nMF+rYfjY9fM4y89mI+AXlM/pmymegu42B\nRzPzX0OIVx3ApEDDxbOUX48fjIinKDe+Myj1mD2KiPdXL2+mNEp8L+WL7x+ZOTUivgp8tWocdR3l\n1+62wKzMHOwX80XApynVA58Blge+A1xSlVwsBZxGuXk8Qmkd/hZKUfLLode8frT6u1dE/AZ4KTNf\n7OXcF1OeUV8P+GG3dZ+h/MJ/nNLYbR7wRmDLzPxEvW8uM+dHxNeBT0bEOVU1yf3AwRFxG+UansHC\nDSeh/DLfOSJuoFzfFwYTU2ZOiYi7KInWy0lBRLyWUl3R1SBunYh4EXgyM5+u9/11FxFHUW54P6OU\nUi1JKUpfn1LKUevTEfEvStuDz1DaYVxSrTsNuCEivkkp0ZlOSWz2zMyjqs/jN4EzImIu5fHK5ShP\nMJzF4FxMaVfz84g4mVIitgmlHcF1PWz/CLBmRGxKSZ6nZWZXUnoO8EvKPWGvHvbdgdLAViNduxs1\nOI2eif4fSdyJUoff9fjgTtXr91Trx1BuLF0Nrd4J3EhJJqZS6nx373bM4yh1vzOBKZQW4m/rI4Ye\nG2N12+aNwDWURORZyk1g6WrdYpQb9sOUetjHgW8DS1br167ewyY1x/s05Qt9LgseSVyooWG1LCg3\n37nA+j3EtRsl+ZlOaYD2Z+DIPt5Hb40cl66u6UnV/GbVtZ1BSRAOpDROq20Etw9wH6Vq4f7BxlTt\nczRwQ7dltY/U1U6f7OtYdXwmt6A8lvdA9e/5DKWNyIE123Rdpz0piUpvjyRuVX2+uh5FvQM4pds2\nH6ckHzMpN+lTe/psV8u6X+OetlmFkpg8T0lSbgG2r7lmtQ0NF6ckq/+qjvOebrE9DPymh2u0RPV+\nNm/l94VTe6ao/tElaViIiMUp7ToOyMw/D4N4dqb00/DabE2HUi0XEUtQEtMPZObl3dYdB+yWmT2V\nIGiEsfpA0rCSmTMj4hAWVBWoSSIiKFVgH6aUcPTUKHQm5TFNjQImBZKGncz8Y7tjGCXWolSdTAYO\nzZpHUbvk4Ns8qANZfSBJkgAHRJIkSRWTAkmSBJgUSJKkikmBJEkCTAokSVLFpECSJAHw/wHNC2Wa\nf/CVJQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f85b793c358>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot ROC curve\n",
    "fpr, tpr, thresholds = metrics.roc_curve(y_test, y_pred_prob)\n",
    "plt.plot(fpr, tpr)\n",
    "plt.xlim([0.0, 1.0])\n",
    "plt.ylim([0.0, 1.0])\n",
    "plt.xlabel('False Positive Rate (1 - Specificity)')\n",
    "plt.ylabel('True Positive Rate (Sensitivity)')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.838692434211\n"
     ]
    }
   ],
   "source": [
    "# calculate AUC\n",
    "print(metrics.roc_auc_score(y_test, y_pred_prob))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Besides allowing you to calculate AUC, seeing the ROC curve can help you to choose a threshold that **balances sensitivity and specificity** in a way that makes sense for the particular context."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([<matplotlib.axes._subplots.AxesSubplot object at 0x7f85b78fe208>,\n",
       "       <matplotlib.axes._subplots.AxesSubplot object at 0x7f85b73c7a58>], dtype=object)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfcAAAF/CAYAAABUloCSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAG4ZJREFUeJzt3X+Q5HV95/Hnm+HHCrJIxMyAKA0o5pA64CQGEbDNFeRn\nnWcuiehZojlJoluBEKpEFI8fp0Khp8iRiwmYMnonl+SSwjMUdy6RlhTCAsF4GhAoYZBl2UEvBsgu\nLLvM+/74fic048xOz/R35tvzmeejqmtnvt/ub79mevbz6u+3v/3pyEwkSVI59mg7gCRJapblLklS\nYSx3SZIKY7lLklQYy12SpMJY7pIkFcZylySpMJb7GhURe0fE5RGxOSK2R8SmiDi97VySVl5E7BcR\nl0TEDRHxeERMR8QH2s6lpbPc164/Ac4F/jtwNvAscENEnNJqKkltOAj4CHAMcDfg7Gar3J5tB9DK\ni4jXA28DPpCZn6yXfRH4DvAJ4MQW40laeVuAQzJza0QcBjzUdiANxz33telXgeeAa2YWZOYO4HPA\nT0fEK9sKJmnlZebOzNzadg41x3Jfm44DvpeZT8xafgcQwPErH0mS1BTLfW06GHhsjuUzyw5ZwSyS\npIZZ7mvTi4Adcyx/pm+9JGmVstzXpqeBfeZYvq5vvSRplbLc16bHqA7NzzazbMsKZpEkNcxyX5v+\nDjgyIg6YtfxEqve3/t3KR5IkNcVyX5v+JzAG/ObMgojYG3g3cFdmPtxSLklSA5zEZg3KzDsi4s+B\nj0bEy4AHgDOBDvDeNrNJakdEbABeAhxYL/rZiNir/vqqzHyqnWRaishceJbBiJgALgd+Edgf+B7w\nvsz8m77rXAycRfWHsQnYkJn3LENmNaDeU78UeCfwE1Sz012YmV9tNZikVkTEQ8B8E1gdnpnfX8k8\nGs6C5V6/Lns3cAtwNfBD4AhgS2beV1/nfOBDVHt/9wMXAScDR2XmtmVLL0mSfswg5f5x4JTMnPcD\nRSJiC9Vhm8vr79cBjwPnZeY1891OkiQ1b5AT6t4CbIqI/xERUxHxzfq1GQAi4nBgAtg4sywzn6Ha\n0z+p6cCSJGn3Bin3I4D3U73OfjpwJXB5RLy/Xj9B9fapqVm3m6rXSZKkFTTI2fJ7AHdk5ofr778V\nEUcBG4D/umzJJEnSkgxS7o8B985adi9wdv31VqpPEhsHNvddZ7xe92MiYuFT9KU1LDOj7QwrxfFA\nWthix4RBDsvfCrxm1rLXAA/Xd/gQVYmfNrOyPqHulPq28wUd2ctFF13Uegbzrd18a1Hbv/PV/Pey\nGjKab7jLUgxS7p8GToyID0XEkRHxa8DvUL0tbsaVwPkR8daIOAb4PPAUcN2SUkmSpCVb8LB8Zt4V\nEf8WuAy4EPg+8OHM/Gzfda6o99av5vlJbE5P3+MuSdKKG2j62cy8EbhxgetcSjXj2arX7XbbjrBb\n5hvOqOfTaFkNfy+jntF8K2+g6Wcbv9OIbON+pdUgIsg1dkKd44E0v6WMCX4qnCRJhbHcJUkqjOUu\nSVJhLHdJkgpjuUuSVBjLXZKkwljukiQVxnKXJKkwlrskSYWx3CVJKozlLklSYSx3SZIKY7lLklQY\ny12SpMJY7pIkFcZylySpMJa7JEmFsdwlSSqM5S5JUmEsd0mSCmO5S5JUGMtdkqTCWO6SJBXGcpck\nqTCWuyRJhbHcJUkqjOUuSVJhLHdJkgpjuUuSVBjLXZKkwljukiQVxnKXJKkwlrskSYWx3CVJKozl\nLklSYSx3SZIKY7lLklQYy12SpMJY7pIkFcZylySpMJa7JEmF2bPtAAC33347Dz744NDbOfTQQzn1\n1FMbSCRJ0uoVmbn7K0RcBFw0a/HWzDyk7zoXA2cBBwKbgA2Zec9utpn997vvvi9hbOw0YK/F5n+B\nbdv+lJ07n2VsbGyo7UhtiggyM9rOsVJmjweSXmgpY8Kge+7fBd4EzGz8ub47PR84FzgTuJ/qicDG\niDgqM7cNsvHp6WmefvpzwPpBc88p4k+Hur0kSSUYtNx3ZeYP5ll3DnBZZl4PEBFnAo8D7wCuGT6i\nJElajEFPqDsiIh6NiAcj4rqIOByg/ncC2Dhzxcx8BrgFOKnxtJIkaUGDlPvtwLuBnwPeS1Xmt0bE\ngfXXCUzNus1UvU6SJK2wBQ/LZ+b/6f8+Im4HHqJ6jX3TMuWSJElLtOi3wmXm9oj4e+DVwJepTrIb\nBzb3XW0c2Lq77XS7XTqdDp1Oh+npXYuNIRWj1+vR6/WYnJxkcnKy7Tit6B8Put0u3W637UhSa5oY\nExZ8K9yP3SBiHfAg8PuZ+bGI2AJclZmX962fAs7LzGvn2cYL3vqybt16duzYzPBny4/5Vjiter4V\nTlK/ZXkrXER8AvgK8H2qPfKPAPsCX6ivciVwQUTcBzwAXAg8BVy3mCCSJKkZgxyWPxT4EnAQ8AOq\nE+xOzMxHADLzinpv/Wqen8Tm9EHf4y5Jkpq16MPyjdyph+WleXlYXlK/pYwJfnCMJEmFsdwlSSqM\n5S5JUmEsd0mSCmO5S5JUGMtdkqTCWO6SJBXGcpckqTCWuyRJhbHcJUkqjOUuSVJhLHdJkgpjuUuS\nVBjLXZKkwljukiQVxnKXJKkwlrskSYWx3CVJKozlLklSYSx3SZIKY7lLklQYy12SpMJY7pIkFcZy\nlySpMJa7JEmFsdwlSSqM5S5JUmEsd0mSCmO5S5JUGMtdkqTCWO6SJBXGcpckqTCWuyRJhbHcJUkq\njOUuSVJhLHdJkgpjuUuSVBjLXZKkwljukiQVxnKXJKkwlrskSYWx3CVJKozlLklSYRZd7hFxQURM\nR8RVs5ZfHBGPRsT2iLg5Io5uLqYkSRrUoso9Ik4EzgK+NWv5+cC5wAbgBOBxYGNE7NdQTkmSNKCB\nyz0iDgD+G/Ae4B9nrT4HuCwzr8/Me4Azgf2BdzQVVJIkDWYxe+5/BPxZZn69f2FEHA5MABtnlmXm\nM8AtwElNhJQkSYPbc5ArRcRZwBHA2+dYPQEkMDVr+RRwyFDpJEnSoi1Y7hFxFPAx4I2ZOb38kSRJ\n0jAG2XN/A/BS4J6ImFk2BpwaEb8NHAMEMA5s7rvdOLB1vo12u106nQ6dTofp6V1LyS4Vodfr0ev1\nmJycZHJysu04regfD7rdLt1ut+1IUmuaGBMiM3d/hYj1wKGzFn8euB/4WGbeGxFbgKsy8/L6Nuuo\nDsufl5nXzrHN7L/fdevWs2PHZmD9kn6I57c7xs6dzzI2NjbUdqQ2RQSZGQtfswyzxwNJL7SUMWHB\nPffMfBK4Z9YdbQP+ITPvrRddCVwQEfcBDwAXAk8B1y0mjCRJGt5AJ9TN4QVPszPzinpv/WrgQGAT\ncHpmbhsynyRJWqQFD8svy516WF6al4flJfVbypjg3PKSJBXGcpckqTCWuyRJhbHcJUkqjOUuSVJh\nLHdJkgpjuUuSVBjLXZKkwljukiQVxnKXJKkwlrskSYWx3CVJKozlLklSYSx3SZIKY7lLklQYy12S\npMJY7pIkFcZylySpMJa7JEmFsdwlSSqM5S5JUmEsd0mSCmO5S5JUGMtdkqTCWO6SJBXGcpckqTCW\nuyRJhbHcJUkqjOUuSVJhLHdJkgpjuUuSVBjLXZKkwljukiQVxnKXJKkwlrskSYWx3CVJKozlLklS\nYSx3SZIKY7lLklQYy12SpMJY7pIkFcZylySpMJa7JEmFWbDcI+L9EfGtiHiivnwjIn5x1nUujohH\nI2J7RNwcEUcvX2RJkrQ7g+y5PwJ8ADgeeB3wNeD6iDgGICLOB84FNgAnAI8DGyNiv2VJLEmSdisy\nc/E3ivh/wAcz85qI2AJclZmX1+vWURX8eZl5zTy3z/77XbduPTt2bAbWL+FH6N/uGDt3PsvY2NhQ\n25HaFBFkZrSdY6XMHg8kvdBSxoRFveYeEXtExBnAfsCtEXE4MAFsnLlOZj4D3AKctJhtS5KkZuw5\nyJXqQ/C3AeuAp4C3ZuY9EfEGIIGpWTeZAg5pMqgkSRrMQOUOfBc4FjgA+FXgCxHxpmVLJUmSlmyg\ncs/MXcCD9bffjIjXU51E93EggHFgc99NxoGtu9tmt9ul0+nQ6XSYnt616OBSKXq9Hr1ej8nJSSYn\nJ9uO04r+8aDb7dLtdtuOJLWmiTFhqSfU/TXwaGa+a54T6qaoTqi7dp7be0KdNA9PqJPUbyljwoJ7\n7hFxGXAD1Vvi9gf+PfAmYOa97lcCF0TEfcADwIVUr8tft5ggkiSpGYMclp8Avlj/+wTwf4Gfz8yb\nADLzinpv/WrgQGATcHpmblueyJIkaXeWdFh+6Dv1sLw0Lw/LS+q37O9zlyRJo89ylySpMJa7JEmF\nsdwlSSqM5S5JUmEsd0mSCmO5S5JUGMtdkqTCWO6SJBXGcpckqTCWuyRJhSmu3F/+8iOJiKEuExOd\ntn8MSZKWbJBPhVtVpqYeBob7EIqpqTXzmR2SpAIVt+cuSdJaZ7lLklQYy12SpMJY7pIkFcZylySp\nMJa7JEmFsdwlSSqM5S5JUmEsd0mSCmO5S5JUGMtdkqTCWO6SJBXGcpckqTCWuyRJhbHcJUkqjOUu\nSVJhLHdJkgpjuUuSVBjLXZKkwljukiQVxnKXJKkwlrskSYWx3CVJKozlLklSYSx3SZIKY7lLklQY\ny12SpMJY7pIkFcZylySpMJa7JEmFWbDcI+KCiLgjIp6IiMcj4n9FxGvnuN7FEfFoRGyPiJsj4ujl\niSxJ0miYmOgQEY1cJiY6jeUaZM/9VOBq4A3Am4FdwE0R8ZKZK0TE+cC5wAbgBOBxYGNE7NdYUkmS\nRszU1MNANnKpttWMyMzF3aAq7CeAt2TmDfWyLcBVmXl5/f06qoI/LzOvmWMb2X+/69atZ8eOzcD6\npf4c9XbHyJym+kUNtSUW+3uRmhIRZGa0nWOlzB4PpNUkIhi+c/55a3N2z1LGhKW85r6+vt2P6js9\nHJgANs5cITOfAW4BTlrC9iVJ0hCWUu6fAe4Gbqu/n6B62jI163pT9TpJkrSC9lzMlSPiU1R742/0\nOJokSaNp4HKPiE8Dvw50M7P/Vf+tQADjwOa+5eP1ujl1u106nQ6dTofp6V2LSy0VpNfr0ev1mJyc\nZHJysu04regfD7rdLt1ut+1IUmuaGBMGOqEuIj4D/BpVsd8/x/q5Tqibojqh7to5ru8JddI8PKFO\nWj1G9YS6BffcI+L3gXcCbwGeiIjxetU/Zea2+usrgQsi4j7gAeBC4CngusWEkSRJwxvksPz7qJ6W\n/PWs5ZcAlwJk5hX13vrVwIHAJuD0vvKXJEkrZNHvc2/kTj0sL83Lw/LS6jGqh+WdW35O+4zcVIKS\nJA1qUW+FWzt20MQzsampNbPzJUkaIe65S5JUGMtdkqTCWO6SJBXGcpckqTCWuyRJhbHcJUkqjOUu\nSVJhLHdJkgpjuUuSVBjLXZK05kxMdBqZZnxUOf2sJGnNmZp6mGY+8GU0C949d0mSCmO5S5JUGMtd\nkqTCWO6SJBXGcpckqTCWuyRJhbHcJUkqjOUuSVJhLHdJkgpjuUuSVBjLXZKkwljukiQVxnKXJKkw\nlrskSYWx3CVJKozlLklSYSx3SZIKY7lLklQYy12SpMJY7pIkFcZylySpMJa7JEmFsdwlSSqM5S5J\nUmH2bDuAJO3Ob/3W77Fp092NbOuMM97KBz94TiPbkkZZZObK32lE9t/vunXr2bFjM7B+yO2OkTkN\nDPszRQPbqLbTxu9Xq1tEkJnRdo6VMns8mO2AAw7mySevBl465D3dzfHH/xV33/21IbejEkQ0N843\ns51qW3P9X1jKmOCeu6RV4CTg4CG3MQ38VQNZpNHna+6SJBXGcpckqTCWuyRJhRmo3CPilIj4ckRs\njojpiHjXHNe5OCIejYjtEXFzRBzdfFxJkrSQQffcXwx8Gzgb2D57ZUScD5wLbABOAB4HNkbEfg3l\nlCRJAxqo3DPzxsy8MDP/krnP+T8HuCwzr8/Me4Azgf2BdzQXVZIkDWLo19wj4nBgAtg4sywznwFu\noXr/iiRJWkFNnFA3QbU3PzVr+VS9TpIkrSDPlpckqTBNzFC3lWr+vXFgc9/y8XrdnLrdLp1Oh06n\nw/T0rgZiSKtTr9ej1+sxOTnJ5ORk23Fa0T8edLtdut1u25HUkImJDlNTDzeyrfHxw9i6dbKRbY2y\nJsaERc8tHxFPARsy8wt9y7YAV2Xm5fX366gOy5+XmdfOsQ3nlpfm4dzyL1TNLX83w08/+zWOP/6j\nzi2/wpqbwx2aHFOdW77a8H7Aq6h+ij2AV0bEscA/ZOYjwJXABRFxH/AAcCHwFHDdYsJIkqThDXpY\n/gTgZp5/enJJffkT4Dcy84p6b/1q4EBgE3B6Zm5rOK8kSVrAQOWemV9ngZPvMvNS4NImQkmSpKXz\nbHlJKsTERIeIaOQyMdFp+8eZwz6N/Xyl8/PcJakQ1VnpzZzcNTU1igW4gyZPXiuZe+6SJBXGcpck\nqTCWuyRJhfE192W1TyMnbqyVWZkkjZJmxi+1w3JfVs2c/DGaJ7ZIKltTJ685frXBw/KSJBXGcpck\nqTCWuyRJhbHcJUkqjOWuRWtqisvRnN5SklY/z5bXojU1xaXvApCk5eGeuyRJhbHcJUkqjOUuSVJh\nLHdJkgpjuUuSVBjLXZKkwljukiQVxnKXJKkwlrskSYWx3FeFfZzuVZI0MKefXRV24HSvkqRBuecu\nSVJhLHdJkgpjuUuSVBhfc19TqhPzpLXq29++q7H/A+Pjh7F162Qj25KaZrmvKc2cmAc+QdDqtGvX\nUzTzf8ATVDXaPCwvSVJhLHdJkgpjuUuSVBjLXS1qZua9sbH9nMFPq9rERKeRv2FphifUqUXNnOA3\nPR2NbMcTpNSWqamH8WRXNck9d0mSCmO5S5JUGMtdkqTCWO6SJBXGE+okaUmczlmjy3KXpCVpajpn\n8Cx3Nc3D8pIkFcZylySpMJa7JEmFabTcI+L9EfFgRDwdEXdFxMlNbl9aXs1Mh+s0tpLa1li5R8Tb\ngCuBjwLHAd8AboyIQ5u6D2l5zZwgNdylmkpUktrT5J77ucAfZ+YfZ+Z9mXk28BjwvgbvQwD02g6w\ngF7bARbQazuAVpVe2wEG0Gs7wAJ6bQdYQK/tAI1rpNwjYi/gdcDGWau+CpzUxH2oX6/tAAvotR1g\nAb22A2hV6bUdYAC9tgMsoNd2gAX02g7QuKb23A8CxoCpWcungImG7kOSJA1gJCax+cmfPJitWw9h\n2IkcXvzil/GjH81+fiFpNTv44EPYvv1VRAy3LzI9vYvnnmsolDTiInP4GZbqw/LbgTMy8y/6ll8N\nvDYz3zzr+k1N6yQVKTPXzJRljgfSwhY7JjSy556ZOyPib4HTgL/oW3Ua8OdzXH/NDFySds/xQGpe\nk4flPwV8ISLuBG6lOkv+YOAPG7wPSZK0gMbKPTP/LCJ+AvgwVal/B/iFzHykqfuQJEkLa+Q1d0mS\nNDqcW16SpMKsWLlHxGER8TP15bCVut8SRcR7I+KAtnPMJSJ+OiL+XUSc0HaWGfXLRf3fnxQRJ0fE\ni9rKtNY5HjTH8WDx1sSYkJnLeqGalvYR4Dlgur48Vy/73eW+/wHynUF1Rv8fAa+bte4g4MG2M86R\n+VngX4xAjo8DP19//TLgtvrx3Vn/eyvwshbzvRq4v/57u61+PG/u+zucBH6q7d9jX97DgJ+pL4e1\nnWeZfkbHg+YzOx4MnnHVjAnDjgfLuuceER8BLgE+OxOwL/AfAJdExIXLmWGBfO8Bvkj1oL4C+EZE\n/GbfVcao8rYiIp6c60J1IuSdfd+35Uyqzw8A+ATVLESvycy9qP4T7QT+c0vZZjI9CPxr4D7gRqpP\nd3kF8HLgu8DlraWrRcS5EfEIVdbb6suDEfFIRPxuu+ma43gwHMeDRoz8mNDYeLDMzzweAX5lN+t/\nBXi0xWdG3wJ+u+/7XwaeAH6n/n4ceK7FfE8BX6H6TzNzeTewC/jQzLIW8z0DdOqvHwLeOGv98cDW\nFvP9APiX9df7Uz1bP7lv/b9qM1+d4SPAk1TvMjmBapB5Rf31h+q/xwvbzNjgz+p4MFw+x4PhM470\nmNDkeLDcQbezm8NFwGuB7S3+IrfN/DH2LXtj/Qs8ZwT+M78auAv4HLBv3/KdwNFt5erLcS/wy/XX\n35vjP/NxwBMt5nsSOKL+eo/693Zs3/pXAU+2/Dsc6cJr+Gd1PBgun+PB8BlHekxocjxY7hPq7gD+\nY0TsPXtFvezD9XXa8gTVe/L/WWbeCvwS8J+A32sjVF+WB4A3UP1BfnPUTkqhmqDokxFxFPBf6q+P\nBIiIw4FPA/+7xXzfAc6KiAD+A/BD4O19699O9fpbm15KNSjO5z7gwBXKstwcD4bgeNCIUR8TmhsP\nlvlZyDFUr8H8I9XhpGvry1fqZVuo5p5v61nS9cCl86w7FfgnWnymPivPLwGPUg2AzzICz9TrXJ+q\n89wLPE11mGtH/e+dwHiL2X6uzrSTaq/x5Po/x99Slcgu4Ndb/v31gOuAvedYtzfwJaDX9uPc0M/q\neNBcVseDpeUb6TGhyfFg2SexiYj9gXcCJ/L8x79upTpJ4EuZ2doJIBHxJuCkzLxsnvVdqtew3rOi\nweYREYdQnfDzZuCYzLyn5UgARMRrgH8DHEF1qOsxqjNjb8rl/gNbOFsHeB1wZ2Z+PyLGgQ3AvsAN\nmXlzi/GIiGOAjcCLgL/h+Y9NHgdOoRqATsvMv28nYbMcD5rjeLA0ozwmNDkeOEOd1LJRLjxJK6up\n8cBylySpMK1OPxsRN0XE99rMsDvmG475tBij/niYbzijng9WR8ZBNfmRr0txJ9Wp/6PKfMMx35Ai\n4ibg8Mw8su0sK2DUHw/zDWfU88GIZ1zMeOBheWmERcRlwMSonMQlqT2LGQ9W4mz5Q4H3ASfxwpMD\nbgU+m5mblzXAAsw3HPNpMUb98TDfcEY9H6yOjE1Y1nKPiJOp5u59DPgqLzyt/zSqCSN+IauJIlac\n+czXZr5BRMQrgEsy8zfazjKsUX88zFd2PlgdGXdnUePBMr8h/y7gqt2s/wzVew3bmjDAfOZrLd+A\nP8OxjMjEKaU/HuYrO99qybhA/oHHg+Xec38aOC4z75tn/U8B38zMVj5D13zDMd/wIuJdC1zllVTP\n1MdWIs9yGvXHw3zDGfV8dYaRztjkeLDcZ8s/RvXBC3P+Iut1j82zbiWYbzjmG97nqWadmu9Zdqtv\nV23YqD8e5hvOqOeD0c/4eRoaD5a73D8JfDYiXk81pd7s1zfeDbT5edXmG475hrcFODsz/3KulRFx\nHNW81yUY9cfDfMMZ9Xww+hmbGw9W4DWCtwG3U03UP11fdtbLWv3QDvOZbwTyfRn46G7WHwtMt51z\nDT0e5is436hnbHI8WLH3uUfEXsBB9bc/zMydK3LHAzLfcMy3NBFxCvDizLxxnvX7ASdk5tdXNtny\nGtXHY4b5hjPq+WA0MzY5HjiJjSRJhSnpZB1JkoTlLklScSx3SZIKY7lLklQYy12SpML8f1NnEfqI\n6qCyAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f85c5e6b128>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# histogram of predicted probabilities grouped by actual response value\n",
    "df = pd.DataFrame({'probability':y_pred_prob, 'actual':y_test})\n",
    "df.hist(column='probability', by='actual', sharex=True, sharey=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7f85b72639e8>]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfIAAAF2CAYAAACVsBoeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuQXOV55/HvIwlxkZCNkNDFSAIhIyGwkUFgAZIYECRr\nJ3al4nKgNo5jr/MHxkXibC1JasEBp5ywWe86UN4kNms7tlMJsXFcTmoTJ+YyoysyEncBRjIoQggk\nJIFAoLvm3T/OND0znpF6eqb7nNP9/VR1mTlzzvQzx6Cf3vO8/b6RUkKSJJXTqLwLkCRJ9TPIJUkq\nMYNckqQSM8glSSoxg1ySpBIzyCVJKjGDXJKkEqspyCNiSUT8U0S8FBHdEfHJGq65ICK6ImJfRGyN\niC8Mv1xJktRbrSPy8cBTwO8C+453ckScCtwHvAJcDPwecHNE/H6ddUqSpAHEUFd2i4i9wOdSSt89\nxjmfBe4AzkgpHeo5dgtwQ0ppxjDqlSRJvTSqR74IWFkJ8R7/DkyPiFkNek9JktpOo4J8KrCj37Ed\nQPR8T5IkjQBnrUuSVGJjGvRztwNT+h2bAqSe7/UREW7BJklqOymlGO7PaNSI/CFgSUSM7XXsl4CX\nU0pbBrogpeSrwa/bbrst9xpa/eU99h63ysv7XP/rtdcSP/xh4sYbE3PnJk4/PfHxjye+/vXEz3+e\n6O7OzhspNY3II2IcMIesxz0KmBkRFwKvpZS2RsQdwCUppWt6Lvl74I+Bb0fEnwJzgT8EbhuxyiVJ\nKoADB2DNGrj//uz17LNwxRVwzTVwzz1w4YUwqoGN7FofrS8EOskejQN8sef1HeC/kE1gO7tyckrp\nzYi4FvhLYB3wOvDllNKdI1S3JEm5OHoUHnssC+0HHoC1a+F974Nly+DLX4ZFi+DEE5tXT01BnlJa\nzjEew6eUPj3AsaeBjror04jr6OjIu4SW5z1uPO9xc3ifq1KCTZuy0L7/fujshGnTshH3TTfBD34A\n73pXfvUNeUGYhhQRkYpQhyRJANu3V4P7gQeyMF+2LAvvq6+G6dOH/x4RQRqByW4GuSSp7b35Jixf\nXg3vl1+Gjo4suJctg3PPhRh25PZlkEuSVKdDh7LedmWC2lNPwQc/WB11X3QRjB7d2BoMckmSatTd\nDU8+WX1Uvno1zJ2bhfY118Dll8PJJze3JoNckqRjeOGF6qPyBx+EiROrj8o7OrKv82SQS5LUy86d\nWWBXRt3791cflS9bBjMKtvemQS5Jamtvvw0rVlRH3Zs3w5VXVoN7/vyRn6A2kgxySVJbOXwY1q2r\njrgfeQQWLqyOuhcuhBNOyLvK2hnkkqSWlhI8/XQ1uFesgNmzqyPuJUtg3Li8q6yfQS5Jajkvvth3\nIZZx46rBfdVVMHly3hWOHINcklR6r72WLXlaCe89e7KV0yrhffbZx/8ZZWWQS5JKZ/9+WLWqGtwb\nN8LixdU+9/ve19idworEIJckFd7Ro9mktMqj8ocfhve/v7oQywc/CGPH5l1lPgxySVLhpATPPVcd\ncS9fDu95T/VR+dKlMGFC3lUWg0EuSSqEl1/uO0Ft1KjqiPvqq2Hq1LwrLCaDXJKUizfegK6uanhv\n3953gtqcOcVeiKUoDHJJUlMcPAgPPVQdcW/YAJddVp2gtmBB43cKa0UGuSSpIbq74fHHq8G9Zk22\n3Gnlcflll8FJJ+VdZfkZ5JKkEZESPP989VF5Z2e28ErvncLe/e68q2w9BrkkqW47dmQ7hVXC+/Dh\nvjuFvec9eVfY+gxySVLN9u7tu1PYiy9mI+1KcM+b5wS1ZjPIJUmDOnwYfvrTap/7scfg0kuro+6L\nL4YxY/Kusr0Z5JKkd3R3Z7PJKyPuVauyj4FVRtyLF8Mpp+RdpXozyCWpzW3ZkoX2/fdn/e4JE/ru\nFHb66XlXqGMxyCWpzeze3XeC2t69fSeozZqVd4UaCoNcklrcvn3ZI/LKqPv552HJkmp4X3CBE9TK\nzCCXpBZz5AisX1+doLZuHXzgA9WFWC69FE44Ie8qNVIMckkquZTg2Wf77hQ2a1bfncLGj8+7SjWK\nQS5JJfTSS1lwV8J77Ni+O4WdcUbeFapZDHJJKoE9e7Kdwip97l27+u4UNnu2fe52ZZBLUgEdOJBt\nMlLpcz/zDFxxRXWC2oUXZvt1Swa5JBXA0aPZqmmVR+Vr12azySsj7ssugxNPzLtKFZFBLkk5SAl+\n/vPqo/KuLpg6tTrivvJKeNe78q5SZWCQS1KTbN/ed4Jad3ffCWrTp+ddocrIIJekBnnzzWynsMqo\ne9u2bMnTyqj73HOdoKbhM8glaYQcOpT1tisT1J58Mlt8pTLqvugiGD067yrVagxySapTd3cW1pVH\n5atXw9y51QlqV1wBJ5+cd5VqdQa5JA3B5s19dwqbOLH6qLyjI/taaiaDXJKOYefOvjuF7dtXfVS+\nbBnMmJF3hWp3Brkk9fL227ByZbXP/cIL2UfBKqPu+fOdoKZiMcgltbXDh7PdwSrB/cgjcPHF1RH3\nJZe4U5iKzSCX1FZSypY7rfS5V6zI1imvBPeSJTBuXN5VSrUzyCW1vBdfrC7E8sADcMop1UflV10F\nkyfnXaFUP4NcUst57TXo7KxOUHv99Sy4K6/Zs/OuUBo5Brmk0tu/P/sMd6XP/dxz2We4K7PL3/c+\ndwpT6zLIJZXO0aPZpLTKiPvhh+H976/2uRctgrFj865Sag6DXFLhpZSNsivBvXw5vOc91T730qUw\nYULeVUr5MMglFdLLL/fdKWzUqL47hU2dmneFUjEY5JIK4Y03spF2pc/9yitZYFdG3XPmuBCLNBCD\nXFIuDh6Ehx6qjrg3bMh625VR94IF7hQm1cIgl9QU3d3w+OPV4H7oITjvvOoEtcsvh5NOyrtKqXwM\nckkNkVK2TnnlUfmDD2YLr/TeKezd7867Sqn8DHJJI+bVV7PArix/euhQ3wlqZ56Zd4VS6zHIJdXt\nrbeytcoro+4tW7KRdmXUPW+eE9SkRjPIJdXs8GH46U+rfe7HHst2B6v0uRcuhDFj8q5Sai8GuaRB\npZTNJq88Kl+1KvsYWCW4Fy/ONiCRlJ+mB3lE3Aj8N2Aa8DTw+ZTSqmOc/8vAbcAFwEFgNXBzSmnT\nAOca5NIwbdlSfVT+wAPZimm9dwo7/fS8K5TUW1ODPCKuA/4WuIEskD8HfBo4L6X00gDnnwU8C9wJ\n/F9gPPA/gdkppXMHON8gl4Zo9+5sp7DKqHvv3r47hZ11Vt4VSjqWZgf5WuDxlNINvY5tBO5NKd0y\nwPkfA/4BGFtJ6IjoAB4AJqeUXut3vkEuHce+fdkj8sqo++c/zx6RV2aXX3CBE9SkMhmpID/u9JaI\nOAG4GPhyv2/9BLh8kMvWAYeB34mIbwLjgE8BD/cPcUkDO3IE1q+vTlBbvz5bNe2aa+Cuu+DSS90p\nTFINQQ5MAkYDO/od3wEsG+iClNKLEfFLwL3AXwGjgEeBD9VfqtTaUoKf/aw64l6+HGbOzB6T33wz\nLFkCp56ad5WSiqYhHziJiCnAN4HvAPcApwJ/QhbsVzXiPaUy2ratOuK+//5shH3NNXD99XD33XDG\nGXlXKKnoagnyXcBRYEq/41OA7YNc8zngrZTSH1UORMRvAVsj4vKU0pr+F9x+++3v/HNHRwcdHR01\nlCaVy5490NVVHXXv3FndKey222D2bPvcUqvq6uqiq6trxH/ucCa7PUc22e3WAc7/X8DSlNKlvY5N\nA7b1HF/V73wnu6klHTgAa9ZUR93PPJNtMlL5PPeCBdl+3ZLaT7Nnrf8G8F2ykfZq4LNkHz+bn1J6\nKSLuAC5JKV3Tc/5VwH3AF8kerU8A/gyYR/aRtf39fr5BrpZw9Gi2U1jlUfnatdls8kpwX3YZnHhi\n3lVKKoKmzVoHSCl9PyImAreQLQizAfhQr8+QTwXO7nV+Z0T8Z+APgJuBfcBa4D/1D3GpzFLKPgZW\neVTe2QlTp2ahfdNN8IMfwLvelXeVklqZS7RKQ7R9e9+dwrq7qyPuZctg+vS8K5RUBq61LjXJ3r3Z\nR8Eqo+6XXsqWPK0sf3ruuU5QkzR0BrnUIIcOZb3tygS1J5/MFl+pjLovvhhGj867SkllZ5BLI6S7\nG556qvqofPVqmDu3OuK+4go4+eS8q5TUagxyaRg2b+67U9hpp1XXLO/ogIkT865QUqszyKUh2LWr\nOkHtgQfg7bf7TlCbOTPvCiW1G4NcOoa334aVK6vB/cILsHRpNbzPP98JapLyZZBLvRw5AuvWVfvc\njzySTUqrBPcll8AJJ+RdpSRVGeRqaylly51WRtwrVsDZZ1cnqC1ZAuPG5V2lJA3OIFfb2bq1+pGw\nBx7IZpJXRtxXXw2TJ+ddoSTVziBXy3v99WzJ00pwv/ZaFtiV8J49O+8KJal+Brlazv792We4K6Pu\n557LPsNdCe73v9+dwiS1DoNcpXf0KDz6aHWC2sMPZ2Fd6XMvWgRjx+ZdpSQ1hkGu0kkJNm6sPirv\n6so2GKksxLJ0KUyYkHeVktQcBrlK4ZVX+k5Qi+g7QW3atLwrlKR8GOQqpDfeyHYKq4T3K69kO4VV\nRt1z5rgQiySBQa6COHgQHnqoGtwbNmS97cqo+wMfcKcwSRqIQa5cdHfDE09UH5WvWQPnnVedoHb5\n5XDSSXlXKUnFZ5CrKVLK1imvjLgffBAmTeq7U9i73513lZJUPga5GubVV/vuFHbwYN+dws48M+8K\nJan8DHKNmLfeytYqr4y6t2yBK6+shvd55zlBTZJGmkGuuh0+nC2+UlmI5bHHst3BKsG9cCGMGZN3\nlZLU2gxy1SylbDZ55VH5ypXZx8AqE9QWL4ZTTsm7SklqLwa5jmnLlr4T1MaPr464r7oqm7AmScqP\nQa4+du/uu1PYm2/23SnsrLPyrlCS1JtB3ub27YNVq6qj7k2bYMmSanBfcIE7hUlSkRnkbebIEXjk\nkeqIe906WLCg2ue+9FJ3CpOkMjHIW1xK8LOfVUfcy5fDjBnVhViWLIFTT827SklSvQzyFrRtW9+d\nwsaMgWuvre4UNmVK3hVKkkaKQd4C9uzJ9uSuhPfOndWdwpYtg3POcSEWSWpVBnlJbdgA99yTBfcz\nz2SbjFSCe8ECJ6hJUrswyEvqve+Fj3wEPvpRuOwyOPHEvCuSJOXBIC+hrVvhootgxw5H3pLU7kYq\nyI2TJurszLb9NMQlSSPFSGmizs5s9rkkSSPFIG+SlLI1z6+6Ku9KJEmtxCBvks2b4dAhmDs370ok\nSa3EIG+SymN1PxcuSRpJBnmTdHb6WF2SNPIM8iZIySCXJDWGQd4EGzfC6NEwe3belUiSWo1B3gT2\nxyVJjWKQN4GP1SVJjeISrQ2WUrb96Pr1MHNm3tVIkorCJVpL4umn4dRTDXFJUmMY5A3msqySpEYy\nyBvM/rgkqZHskTdQdzdMngwbNsC0aXlXI0kqEnvkJfDEE1mQG+KSpEYxyBvI/rgkqdEM8gayPy5J\najR75A1y5AhMmgSbNmWP1yVJ6s0eecE9+ijMmGGIS5IayyBvEPvjkqRmMMgbxP64JKkZ7JE3wKFD\nWX98yxY47bS8q5EkFZE98gJbtw7mzDHEJUmNV3OQR8SNEfFCROyPiPURsbiGaz4fEc9GxIGI2BYR\nfza8csvB/rgkqVlqCvKIuA64E/gSsABYA/w4Is48xjVfAW4AbgbmAR8GVgy34DKwPy5JapaaeuQR\nsRZ4PKV0Q69jG4F7U0q3DHD+XOAp4IKU0sYafn7L9MgPHMg+crZtG0yYkHc1kqSialqPPCJOAC4G\n7uv3rZ8Alw9y2UeB54EPR8TzEbE5Ir4dES3/qeq1a2H+fENcktQctTxanwSMBnb0O74DmDrINbOB\ns4DrgE8CnyB7vP7PdVVZIvbHJUnN1KhZ66OAscAnUkqrU0qrgd8CPhgRlzToPQvB/rgkqZnG1HDO\nLuAoMKXf8SnA9kGueQU4klJ6vnIgpbQpIo4CM4F1/S+4/fbb3/nnjo4OOjo6aiitWPbty5ZmveKK\nvCuRJBVNV1cXXV1dI/5zhzPZ7TmyyW63DnD+tcC/AXNSSpt7jp0DbAIuTSmt73d+S0x2u+8++OIX\nYdWqvCuRJBVdsxeE+QrwqYj4TETMi4i7gGnA13qKuSMi7u91/v3Ao8C3ImJBRHwA+CbwUP8QbyX2\nxyVJzVZTkKeUvg98HrgFeIxstvqHUkov9ZwyFTi71/kJ+FXgVWA58GPgReDXRqzyArI/LklqNtda\nHyF798K0abBrF5x0Ut7VSJKKzrXWC2blSrjkEkNcktRcBvkIsT8uScqDQT5C7I9LkvJgj3wEvP46\nzJwJu3fD2LF5VyNJKgN75AWyYgVcdpkhLklqPoN8BNgflyTlxSAfAfbHJUl5sUc+TDt3wpw5WX98\nTC0r10uShD3ywli+HBYvNsQlSfkwyIfJ/rgkKU8G+TDZH5ck5ckgH4ZXXoHt2+HCC/OuRJLUrgzy\nYejqgqVLYfTovCuRJLUrg3wY7I9LkvJmkA+D/XFJUt4M8jpt3Qp79sD55+ddiSSpnRnkdershI4O\nGOUdlCTlyBiqk/1xSVIRGOR1SAkefND+uCQpfwZ5HTZvhkOHYO7cvCuRJLU7g7wOldnqMeyl7iVJ\nGh6DvA72xyVJRWGQD1FKfn5cklQcBvkQbdyYLck6e3belUiSZJAPmf1xSVKRGORDZH9cklQkkVLK\nuwYiIhWhjuNJCaZMgfXrYebMvKuRJJVZRJBSGvbzXUfkQ/D003DqqYa4JKk4DPIhcLa6JKloDPIh\nsD8uSSoae+Q16u6GyZNhwwaYNi3vaiRJZWePvMmeeCILckNcklQkBnmN7I9LkorIIK+R/XFJUhHZ\nI6/BkSMwaRJs2pQ9XpckabjskTfRo4/CjBmGuCSpeAzyGtgflyQVlUFeA/vjkqSiskd+HIcOZf3x\nLVvgtNPyrkaS1CrskTfJunUwZ44hLkkqJoP8OOyPS5KKzCA/DvvjkqQis0d+DAcOZB8527YNJkzI\nuxpJUiuxR94Ea9fC/PmGuCSpuAzyY7A/LkkqOoP8GOyPS5KKzh75IPbtgzPOgB07YNy4vKuRJLUa\ne+QNtno1LFhgiEuSis0gH4T9cUlSGRjkg7A/LkkqA3vkA9i7F6ZPh5074aST8q5GktSK7JE30MqV\nsHChIS5JKj6DfAD2xyVJZWGQD8D+uCSpLOyR9/P66zBrFuzaBWPH5l2NJKlV2SNvkBUrYNEiQ1yS\nVA41B3lE3BgRL0TE/ohYHxGLa7zuvRGxNyLerL/M5rE/Lkkqk5qCPCKuA+4EvgQsANYAP46IM49z\n3QnAPUDX8MpsHvvjkqQyqalHHhFrgcdTSjf0OrYRuDeldMsxrvsLYAKwAvhqSmnADUGL0iPftQvO\nOQd274YxY/KuRpLUyprWI+8ZVV8M3NfvWz8BLj/Gdb8CfBi4aTgFNlNXFyxebIhLksqjlkfrk4DR\nwI5+x3cAUwe6ICKmA3cDv5lS2jesCpvI/rgkqWwaNWv9b4G/Simt7/l62I8OmsH+uCSpbGp5iLwL\nOApM6Xd8CrB9kGuuApZExO09XwcwKiIOATemlL7R/4Lbb7/9nX/u6Oigo6OjhtJGzvbt2evCC5v6\ntpKkNtHV1UVXV9eI/9zhTHZ7jmyy260DnD+/36FfA/47cAnwckrpjX7n5z7Z7Z574Hvfgx/9KNcy\nJEltYqQmu9U6resrwHcjYh2wGvgsMA34Wk8xdwCXpJSuAUgpPdOv2EuA7pTSs8MtuFHsj0uSyqim\nIE8pfT8iJgK3kAX4BuBDKaWXek6ZCpzdmBKbo7MTbirN/HpJkjKutQ5s3QoXXQQ7dsAoF62VJDWB\na62PoM5O6OgwxCVJ5WN0YX9cklReBjl+flySVF5tH+SbN8PBgzB3bt6VSJI0dG0f5A8+mD1Wj1Ks\nPSdJUl9tH+T2xyVJZdbWQZ6S/XFJUrm1dZBv2gSjR8Ps2XlXIklSfdo6yO2PS5LKrq2D3P64JKns\n2naJ1pRg6lR4+GGYNaupby1Jkku0Dtczz8D48Ya4JKnc2jbIK/1xSZLKrG2D3P64JKkVtGWPvLsb\nJk+Gp56C6dOb9raSJL3DHvkwPPlkFuSGuCSp7NoyyO2PS5JaRVsGucuySpJaRdv1yI8cgUmTsuVZ\nJ09uyltKkvQL7JHX6dFHYcYMQ1yS1BraLsj92JkkqZW0ZZDbH5cktYq26pEfOpT1x7dsgdNOa/jb\nSZI0KHvkdVi3DubMMcQlSa2jrYLc/rgkqdW0XZDbH5cktZK26ZEfOJB95GzbNpgwoaFvJUnScdkj\nH6K1a2H+fENcktRa2ibI7Y9LklpRWwW5/XFJUqtpix75vn1wxhmwYweMG9ewt5EkqWb2yIdg9WpY\nsMAQlyS1nrYIcvvjkqRW1TZBbn9cktSKWr5HvncvTJ8OO3fCSSc15C0kSRoye+Q1WrkSFi40xCVJ\nranlg9z+uCSplbVFkNsflyS1qpbukb/+OsyaBbt2wdixI/7jJUmqmz3yGqxYAYsWGeKSpNbV0kFu\nf1yS1OpaPsjtj0uSWlnL9sh37YJzzoHdu2HMmBH90ZIkDZs98uPo6oLFiw1xSVJra9kgtz8uSWoH\nLR3k9sclSa2uJYN8+/bsdeGFeVciSVJjtWSQd3bC0qUwenTelUiS1FgtG+T2xyVJ7aBlg9z+uCSp\nHbRckG/dCnv2wPnn512JJEmN13JB3tkJHR0wquV+M0mSflHLxZ39cUlSO2nJILc/LklqFy0V5Js3\nw8GDMHdu3pVIktQcLRXkDz6YPVaPYS9BL0lSOdQc5BFxY0S8EBH7I2J9RCw+xrlXRsSPIuLliHg7\nIp6IiE+PTMmDsz8uSWo3NQV5RFwH3Al8CVgArAF+HBFnDnLJ5cCTwMeA84G/Bu6OiOuHXfEgUrI/\nLklqPzXtRx4Ra4HHU0o39Dq2Ebg3pXRLTW8U8T1gVErp4wN8b9j7kW/cCNdcA1u2+GhdklR8TduP\nPCJOAC4G7uv3rZ+QjbxrNQF4fQjnD4n9cUlSO6rl0fokYDSwo9/xHcDUWt4kIn4VuBr4+pCqGwL7\n45KkdtTwWesRcQXwd8BNKaVHGvEeKUFXl0EuSWo/Y2o4ZxdwFJjS7/gUYPuxLuyZ2f4vwK0ppbuP\nde7tt9/+zj93dHTQ0dFRQ2mZZ56B8eNh1qyaL5Ekqam6urro6uoa8Z87nMluz5FNdrt1kGuWAv8P\n+EJK6a7j/PxhTXb76lfhiSfgG9+o+0dIktRUTZvs1uMrwKci4jMRMS8i7gKmAV/rKeaOiLi/V3Ed\nwL+SfezsHyJiSs9r0nALHoj9cUlSu6ppRA4QETcAf0AW4BuAz6eUVvd872+ApSmlc3p9/ckBfsyW\nlNLsAX523SPy7m6YPBmeegqmT6/rR0iS1HQjNSKvOcgbaThB/vjjcP318LOfjXBRkiQ1ULMfrRdW\n5fPjkiS1o9IHuf1xSVI7K/Wj9SNHYNKkbHnWM85oQGGSJDWIj9aBxx6DGTMMcUlS+yp1kNsflyS1\nu1IHuf1xSVK7K22P/PBhOP10+I//gIkTG1OXJEmN0vY98nXrYM4cQ1yS1N5KG+T2xyVJKnGQ2x+X\nJKmkPfKDB7PPj2/bBhMmNLAwSZIapK175GvXwvz5hrgkSaUMcvvjkiRlShnk9sclScqUrke+b1+2\nJOuOHTBuXIMLkySpQdq2R75mDSxYYIhLkgQlDHL745IkVZUuyO2PS5JUVaoe+d69MH06vPoqnHxy\nEwqTJKlB2rJHvmoVLFxoiEuSVFGqILc/LklSX6UKcvvjkiT1VZoe+Z49MHMm7NoFY8c2qTBJkhqk\n7XrkK1bAokWGuCRJvZUmyO2PS5L0i0oT5PbHJUn6RaXoke/aBeecA7t3w5gxTSxMkqQGaase+fLl\nsHixIS5JUn+lCHL745IkDawUQW5/XJKkgRU+yLdvz14LFuRdiSRJxVP4IO/qgqVLYfTovCuRJKl4\nCh/k9sclSRpc4YPc/rgkSYMrdJC/9FK2xvoFF+RdiSRJxVToIO/shI4OGFXoKiVJyk+hI9L+uCRJ\nx1boILc/LknSsRU2yDdvhoMHYd68vCuRJKm4ChvklcfqMezl5CVJal2FDXIfq0uSdHyFDPKUDHJJ\nkmpRyCDftClbkvWcc/KuRJKkYitkkNsflySpNoUMch+rS5JUm0gp5V0DEZEqdaQEU6fCww/DrFk5\nFyZJUoNEBCmlYT97LtyI/JlnYPx4Q1ySpFoULshdllWSpNoVLsjtj0uSVLtC9ci7u2HyZHjqKZg+\nPe+qJElqnJbskT/5ZBbkhrgkSbUpVJDbH5ckaWgKFeT2xyVJGprC9MgPH05MmgQbN8IZZ+RdkSRJ\njdVyPfLHHoMZMwxxSZKGojBBbn9ckqShqznII+LGiHghIvZHxPqIWHyc8y+IiK6I2BcRWyPiC8c6\n3/64JElDV1OQR8R1wJ3Al4AFwBrgxxFx5iDnnwrcB7wCXAz8HnBzRPz+YO+xZg1ceeXQipckqd3V\nOiL/feBbKaVvpZSeSyn9LllIf3aQ8z8BnAz8dkrp2ZTSD4E/B/7rYG8wZw5MnDiEyjVkXV1deZfQ\n8rzHjec9bg7vc3kcN8gj4gSyUfV9/b71E+DyQS5bBKxMKR3qdezfgekRMeB2KD5Wbzz/w2w873Hj\neY+bw/tcHrWMyCcBo4Ed/Y7vAKYOcs3UQc6Pwa4xyCVJGrrCzFpfsiTvCiRJKp/jLgjT82h9H3B9\nSukfex3/P8D5KaVfGEtHxHeAiSmlj/Q6thD4KTA7pbSl3/n5r0ojSVKTjcSCMGNqeJPDEfEIcC3w\nj72+dS1w7yCXPQT8j4gY26tP/kvAy/1DvOc9hv2LSJLUjmp9tP4V4FMR8ZmImBcRdwHTgK8BRMQd\nEXF/r/P/nmwU/+2IOD8ifh34Q+B/j2DtkiS1veOOyAFSSt+PiInALWQBvgH4UErppZ5TpgJn9zr/\nzYi4FvhLYB3wOvDllNKdI1m8JEntrhCbpkiSpPo0ZdZ6o5d31dDucURcGRE/ioiXI+LtiHgiIj7d\nzHrLaqi1PX6nAAAEMElEQVT/Lve67r0RsTci3mx0jWVXzz2OiM9HxLMRcSAitkXEnzWj1rKq48/k\nX46INRHxZkTs7Pnz473NqrdsImJJRPxTRLwUEd0R8ckarqk79xoe5M1Y3rXdDfUeky3k8yTwMeB8\n4K+BuyPi+iaUW1p13OfKdScA9wBdja6x7Oq5xxHxFeAG4GZgHvBhYEXjqy2nOv5MPgv4EbC85/xl\nwEnAvzSh3LIaDzwF/C7ZfLFjGnbupZQa+gLWAl/rd2wj8KeDnP9ZYA8wttexW4Ctja61rK+h3uNB\nfsb3gHvz/l2K/Kr3PgN/AXwT+G3gzbx/jyK/6vjzYi5wCDg379rL8qrjHn8MOExPK7bnWAdwlOxj\nxrn/TkV+AXuBTx7nnGHlXkNH5M1a3rWd1XmPBzKBbFKiBlDvfY6IXyEbId7UuOpaQ533+KPA88CH\nI+L5iNgcEd+OiMkNLLW06rzH68iC/HciYlTP6PFTwMMppdcaVWubGVbuNfrRelOWd21z9dzjPiLi\nV4Grga+PbGktZcj3OSKmA3cDv5lSOu7jNdX17/Js4CzgOuCTZBs2zQP+uTEllt6Q73FK6UWydUD+\nBDhINnI8H/jIQOerLsPKvcIs0ap8RMQVwN8BN6WUHsm7nhbzt8BfpZTW93ztwkcjbxQwFvhESml1\nSmk18FvAByPiknxLaw0RMYWsNfQdYCFwJdnj4sEWBFOTNTrId5H1Uab0Oz4F2D7INdsHOT8d45p2\nVs89BqBnpuq/AremlO5uTHkto577fBVwW0QcjojDwDeA8RFxKCJ+p3GlllY99/gV4EhK6fnKgZTS\npp6fM7MRRZZcPff4c8BbKaU/Sik9kVJaRfaXpSsjYijtOw1uWLnX0CBPKR0GKsu79nYtsHqQyx4C\nlkTE2F7HBl3etd3VeY+JiKVkIf7HKaWvNq7C1lDnfb6AbJbvhT2vPyabwXohjmZ+QZ33eDUwJiLe\nWZAqIs4he3zsnxf91HmPTyEL/966e/7Xp7ojY3i514QZe78BHAA+Q9a7ugt4Eziz5/t3APf3On8C\n8DLZMq/nA78OvAF8Pu/Zh0V91XGPO4C3gD8n+1tf5TUp79+lyK+h3ucBrnfW+gjfY7J2xTqgk+wv\nTR8g+5jf6rx/l6K+6rjHVwFHgC8Ac4CLgH8D/gM4Oe/fp4gvYBzZX9gXAG8Dt/Z8PWOQezys3GvW\nL3UD8AKwv+c/uit6fe9vgOf7nX9+z3+M+4BtZI9+c/8/p8ivodzjnq+PDvB6Ie/fo+ivof673O9a\ng7wB95jsL6Hf6/mDbzvwXWBy3r9HkV913OPfANb3BP52ss+Vz8v79yjqi2weQfcAf8Z+6xj3uO7c\nc4lWSZJKzP6GJEklZpBLklRiBrkkSSVmkEuSVGIGuSRJJWaQS5JUYga5JEklZpBLklRiBrkkSSX2\n/wEL6NAs3OdKvwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f85f3084ac8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# ROC curve using y_pred_class - WRONG!\n",
    "fpr, tpr, thresholds = metrics.roc_curve(y_test, y_pred_class)\n",
    "plt.plot(fpr, tpr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.780962171053\n"
     ]
    }
   ],
   "source": [
    "# AUC using y_pred_class - WRONG!\n",
    "print(metrics.roc_auc_score(y_test, y_pred_class))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you use **y_pred_class**, it will interpret the zeros and ones as predicted probabilities of 0% and 100%."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Cross-validation\n",
    "\n",
    "## Review of model evaluation procedures\n",
    "\n",
    "**Motivation:** Need a way to choose between machine learning models\n",
    "\n",
    "- Goal is to estimate likely performance of a model on **out-of-sample data**\n",
    "\n",
    "**Initial idea:** Train and test on the same data\n",
    "\n",
    "- But, maximizing **training accuracy** rewards overly complex models which **overfit** the training data\n",
    "\n",
    "**Alternative idea:** Train/test split\n",
    "\n",
    "- Split the dataset into two pieces, so that the model can be trained and tested on **different data**\n",
    "- **Testing accuracy** is a better estimate than training accuracy of out-of-sample performance\n",
    "- But, it provides a **high variance** estimate since changing which observations happen to be in the testing set can significantly change testing accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.cross_validation import train_test_split\n",
    "from sklearn import metrics\n",
    "\n",
    "# define X and y\n",
    "feature_cols = ['Pclass', 'Parch', 'Age', 'Sex_Female', 'Embarked_Q', 'Embarked_S']\n",
    "X = titanic[feature_cols]\n",
    "y = titanic.Survived"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.793721973094\n"
     ]
    }
   ],
   "source": [
    "# train/test split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1)\n",
    "\n",
    "# train a logistic regression model\n",
    "logreg = LogisticRegression(C=1e9)\n",
    "logreg.fit(X_train, y_train)\n",
    "\n",
    "# make predictions for testing set\n",
    "y_pred_class = logreg.predict(X_test)\n",
    "\n",
    "# calculate testing accuracy\n",
    "print(metrics.accuracy_score(y_test, y_pred_class))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.780269058296\n"
     ]
    }
   ],
   "source": [
    "# train/test split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=2)\n",
    "\n",
    "# train a logistic regression model\n",
    "logreg = LogisticRegression(C=1e9)\n",
    "logreg.fit(X_train, y_train)\n",
    "\n",
    "# make predictions for testing set\n",
    "y_pred_class = logreg.predict(X_test)\n",
    "\n",
    "# calculate testing accuracy\n",
    "print(metrics.accuracy_score(y_test, y_pred_class))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.784753363229\n"
     ]
    }
   ],
   "source": [
    "# train/test split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=3)\n",
    "\n",
    "# train a logistic regression model\n",
    "logreg = LogisticRegression(C=1e9)\n",
    "logreg.fit(X_train, y_train)\n",
    "\n",
    "# make predictions for testing set\n",
    "y_pred_class = logreg.predict(X_test)\n",
    "\n",
    "# calculate testing accuracy\n",
    "print(metrics.accuracy_score(y_test, y_pred_class))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "train test spliting create bias due to the intrinsic randomness in the sets selection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# K-fold cross-validation\n",
    "\n",
    "1. Split the dataset into K **equal** partitions (or \"folds\").\n",
    "2. Use fold 1 as the **testing set** and the union of the other folds as the **training set**.\n",
    "3. Calculate **testing accuracy**.\n",
    "4. Repeat steps 2 and 3 K times, using a **different fold** as the testing set each time.\n",
    "5. Use the **average testing accuracy** as the estimate of out-of-sample accuracy.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Diagram of **5-fold cross-validation:**\n",
    "\n",
    "![5-fold cross-validation](https://raw.githubusercontent.com/justmarkham/DAT8/master/notebooks/images/cross_validation_diagram.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration                   Training set observations                   Testing set observations\n",
      "    1     [ 5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24]        [0 1 2 3 4]       \n",
      "    2     [ 0  1  2  3  4 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24]        [5 6 7 8 9]       \n",
      "    3     [ 0  1  2  3  4  5  6  7  8  9 15 16 17 18 19 20 21 22 23 24]     [10 11 12 13 14]     \n",
      "    4     [ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 20 21 22 23 24]     [15 16 17 18 19]     \n",
      "    5     [ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19]     [20 21 22 23 24]     \n"
     ]
    }
   ],
   "source": [
    "# simulate splitting a dataset of 25 observations into 5 folds\n",
    "from sklearn.cross_validation import KFold\n",
    "kf = KFold(25, n_folds=5, shuffle=False)\n",
    "\n",
    "# print the contents of each training and testing set\n",
    "print('{} {:^61} {}'.format('Iteration', 'Training set observations', 'Testing set observations'))\n",
    "for iteration, data in enumerate(kf, start=1):\n",
    "    print('{:^9} {} {:^25}'.format(str(iteration), str(data[0]), str(data[1])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Dataset contains **25 observations** (numbered 0 through 24)\n",
    "- 5-fold cross-validation, thus it runs for **5 iterations**\n",
    "- For each iteration, every observation is either in the training set or the testing set, **but not both**\n",
    "- Every observation is in the testing set **exactly once**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Create k-folds\n",
    "kf = KFold(X.shape[0], n_folds=10, random_state=0)\n",
    "\n",
    "results = []\n",
    "\n",
    "for train_index, test_index in kf:\n",
    "    X_train, X_test = X.iloc[train_index], X.iloc[test_index]\n",
    "    y_train, y_test = y.iloc[train_index], y.iloc[test_index]\n",
    "\n",
    "    # train a logistic regression model\n",
    "    logreg = LogisticRegression(C=1e9)\n",
    "    logreg.fit(X_train, y_train)\n",
    "\n",
    "    # make predictions for testing set\n",
    "    y_pred_class = logreg.predict(X_test)\n",
    "\n",
    "    # calculate testing accuracy\n",
    "    results.append(metrics.accuracy_score(y_test, y_pred_class))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    10.000000\n",
       "mean      0.794644\n",
       "std       0.030631\n",
       "min       0.764045\n",
       "25%       0.768820\n",
       "50%       0.780899\n",
       "75%       0.820225\n",
       "max       0.842697\n",
       "dtype: float64"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(results).describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.cross_validation import cross_val_score\n",
    "\n",
    "logreg = LogisticRegression(C=1e9)\n",
    "\n",
    "results = cross_val_score(logreg, X, y, cv=10, scoring='accuracy')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    10.000000\n",
       "mean      0.794665\n",
       "std       0.019263\n",
       "min       0.775281\n",
       "25%       0.779963\n",
       "50%       0.786517\n",
       "75%       0.806742\n",
       "max       0.829545\n",
       "dtype: float64"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(results).describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Comparing cross-validation to train/test split\n",
    "\n",
    "Advantages of **cross-validation:**\n",
    "\n",
    "- More accurate estimate of out-of-sample accuracy\n",
    "- More \"efficient\" use of data (every observation is used for both training and testing)\n",
    "\n",
    "Advantages of **train/test split:**\n",
    "\n",
    "- Runs K times faster than K-fold cross-validation\n",
    "- Simpler to examine the detailed results of the testing process\n",
    "\n",
    "## Cross-validation recommendations\n",
    "\n",
    "1. K can be any number, but **K=10** is generally recommended\n",
    "2. For classification problems, **stratified sampling** is recommended for creating the folds\n",
    "    - Each response class should be represented with equal proportions in each of the K folds\n",
    "    - scikit-learn's `cross_val_score` function does this by default"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Improvements to cross-validation\n",
    "\n",
    "**Repeated cross-validation**\n",
    "\n",
    "- Repeat cross-validation multiple times (with **different random splits** of the data) and average the results\n",
    "- More reliable estimate of out-of-sample performance by **reducing the variance** associated with a single trial of cross-validation\n",
    "\n",
    "**Creating a hold-out set**\n",
    "\n",
    "- \"Hold out\" a portion of the data **before** beginning the model building process\n",
    "- Locate the best model using cross-validation on the remaining data, and test it **using the hold-out set**\n",
    "- More reliable estimate of out-of-sample performance since hold-out set is **truly out-of-sample**\n",
    "\n",
    "**Feature engineering and selection within cross-validation iterations**\n",
    "\n",
    "- Normally, feature engineering and selection occurs **before** cross-validation\n",
    "- Instead, perform all feature engineering and selection **within each cross-validation iteration**\n",
    "- More reliable estimate of out-of-sample performance since it **better mimics** the application of the model to out-of-sample data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Overfitting, Underfitting and Model Selection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we've gone over the basics of validation, and cross-validation, it's time to go into even more depth regarding model selection.\n",
    "\n",
    "The issues associated with validation and \n",
    "cross-validation are some of the most important\n",
    "aspects of the practice of machine learning.  Selecting the optimal model\n",
    "for your data is vital, and is a piece of the problem that is not often\n",
    "appreciated by machine learning practitioners.\n",
    "\n",
    "Of core importance is the following question:\n",
    "\n",
    "**If our estimator is underperforming, how should we move forward?**\n",
    "\n",
    "- Use simpler or more complicated model?\n",
    "- Add more features to each observed data point?\n",
    "- Add more training samples?\n",
    "\n",
    "The answer is often counter-intuitive.  In particular, **Sometimes using a\n",
    "more complicated model will give _worse_ results.**  Also, **Sometimes adding\n",
    "training data will not improve your results.**  The ability to determine\n",
    "what steps will improve your model is what separates the successful machine\n",
    "learning practitioners from the unsuccessful."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Illustration of the Bias-Variance Tradeoff\n",
    "\n",
    "For this section, we'll work with a simple 1D regression problem.  This will help us to\n",
    "easily visualize the data and the model, and the results generalize easily to  higher-dimensional\n",
    "datasets.  We'll explore a simple **linear regression** problem.\n",
    "This can be accomplished within scikit-learn with the `sklearn.linear_model` module.\n",
    "\n",
    "We'll create a simple nonlinear function that we'd like to fit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "def test_func(x, err=0.5):\n",
    "    y = 10 - 1. / (x + 0.1)\n",
    "    if err > 0:\n",
    "        y = np.random.normal(y, err)\n",
    "    return y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's create a realization of this dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def make_data(N=40, error=1.0, random_seed=1):\n",
    "    # randomly sample the data\n",
    "    np.random.seed(1)\n",
    "    X = np.random.random(N)[:, np.newaxis]\n",
    "    y = test_func(X.ravel(), error)\n",
    "    \n",
    "    return X, y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEACAYAAABI5zaHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFpRJREFUeJzt3X+Q3HV9x/HXm+ZusgYvSruAgtz6o5jWaQyZRpmRKRsk\nGu1UGNsOhCkKnrZMqjLUWhA7wzlOFR07FLXpID0dcJojU2lL1KoYzWoZwZyQcBQTiNq9INRkcTSj\nnYMc8u4fuxePzd3e7vf73e+Pzz4fMzvZ/eZz3+/79nbf+9n35/P9fM3dBQAIy0lZBwAASB7JHQAC\nRHIHgACR3AEgQCR3AAgQyR0AAtR1cjezCTM7bGbTC7Z93Mz2m9k+M7vTzEb6EyYAoBe99Nw/J+mN\nbdvulvQqd18n6aCkDyQVGAAguq6Tu7vfI+lnbdt2ufuzrYf3STozwdgAABElWXN/h6SvJLg/AEBE\niSR3M/ugpDl3357E/gAA8ayIuwMzu0LSmyVdsEw7FrEBgAjc3Xr9mV577ta6NR+YbZb0fklvcfen\nl/thdy/s7YYbbsg8hkGNv8ixE3/2t6LHH1UvUyG3S/qOpLPN7JCZXSnpU5JOlvR1M3vAzLZFjgQA\nkJiuyzLuftkimz+XYCwAgIRwhmqXqtVq1iHEUuT4ixy7RPxZK3r8UVmcmk5PBzLztI4FAKEwM3kK\nA6oAgAIguQNAgEjuABAgkjsABIjkDgABIrkDQIBI7gAQIJI7AASI5A4AASK5AwXUaDQ0NTWlRqOR\ndSjIKZI7UDCTkzs0OrpGmzZdpdHRNZqc3JF1SMgh1pYBCqTRaGh0dI1mZ3dLWitpWqXSRs3MHFC5\nXM46PPQBa8sAA6Ber2t4uKJmYpektRoaGlW9Xs8uKOQSyR0okEqlomPH6pKmW1umNTc3o0qlkl1Q\nyCWSO1Ag5XJZExPbVCpt1MjIepVKGzUxsY2SDE5AzR0ooEajoXq9rkqlQmIPXNSaO8kdAHKMAVUA\nwHEkdwAIEMkdAAJEcgcQGcsg5FfXyd3MJszssJlNL9j2QjO728weMbOvmdnq/oQJIG9YBiHfup4t\nY2bnSfqlpNvdfW1r28ck/dTdP25m10p6obtft8TPM1sGCATLIKSn77Nl3P0eST9r23yRpNta92+T\ndHGvAQAoHpZByL+4NfdT3f2wJLn7TySdGj8kAHmX9TII1PqXtyLh/XWsu4yPjx+/X61WVa1WEz48\ngDTML4MwNrZRQ0OjmpubSW0ZhMnJHRob26rh4eYHzMTENm3Zcknfj5uWWq2mWq0Wez89naFqZqOS\nvrig5r5fUtXdD5vZ6ZJ2u/vvLPGz1NyBwKS9DMIg1vrTOkPVWrd5OyVd0br/dkl39RoAgOIql8va\nsGFDaomVWn/3epkKuV3SdySdbWaHzOxKSTdK2mRmj0h6fesxAPRF1rX+ImHhMACZiFrSma+5L6z1\nh1Rzb8eqkAAKI+6g6CAteUxyBwIVWiIbxEHROFjyFwhQiKf4MyiaDnruQAdZ9ppD7eGG+nv1Cz13\nIGFZ95pD7eFyHdh00HMHFpGH3mUeYkhS+7eg0MYS+oWeO5CgPPSaQ+rhLvYtKO0ToAYNPXdgEXnq\nNRe9h9toNHTWWWfrqaf+UdImSf9b6G8gaaPnDiQoT73mxXq4RVoV8ZZbbtVTTx2T9PeS1kjaH8TY\nQd7Rc0fhpNmTzWOvOc1VEeP+/ot9A5KqWrnSdejQo7l5TvMsas9d7p7KrXkoIJ7t2+/wUukUX716\nvZdKp/j27XdkHVKqjhw54qXSKS496JK79KCXSqf4kSNHEj9WEs/1nj17fPXq9a1Y528v9w9/+O8S\njzdUrdzZc86l547CyFMdPCtTU1PatOkqHT16//FtIyPrtWvXLdqwYUNix0nqueZvFh81dwQvDzNY\nspbWqohJPdd5GrsYNElfiQnom+cmtmYvcNCWey2Xy7rppht19dXna3i4omeeOdSXZJnkc71lyyW6\n8MILcjd2ETp67iiM+V7gypXna9WqV2rlyvMHrhc4OblD11xznYaHX6Jjx36km266sS+DqUn3uMvl\nsiqViur1eiFm+ISA5I7CMTtJUqn17+BoNBoaG9uq2dnd+sUvpvX00/+la665rm/JcsuWSzQzc0C7\ndt2imZkDsT5Esl7KYRAxoIrCGPTBuSQGU7OY2jnof7e4GFBFLvTz5JpBH1CNO5iaVe950P9umYky\nfzLKTcxzD16/56CnOcc7r+af45GRc3p6jrN87vp57CNHjviePXuCfg0o4jx3knvg0nrxp5U8oia3\nkET5my52MtHIyDm+Z8+ePkb6a/34uw3KCW0kd5wgzRd/msljEHprScvDt54k/255+H3SQnLHc6T9\n4h+kN1tRhfStJ+tvImmKmtw5iSlQ84NYs7MnDmL1Y4bC/LzosbGNGhoa1dzczMDNQc+7kE4m4oS2\n5SUyFdLMrpE0JulZSQ9JutLdj7W18SSOhe5kNf0sj6soDppB+RvMr465sDPRr9UxsxR1KmTs5G5m\nL5Z0j6Q17n7MzHZI+rK7397WjuSeskF58ePX0lwOOA8G4YMs6+R+r6R1kn4h6d8l3ezuu9rakdwz\nMAgv/kGz1N+Uk4XClNlJTO7+hJqXWDkk6XFJP29P7MhOiNepLNJViJLW6UQkThbCQrEHVM3sBZIu\nkjQq6aikL5jZZe6+vb3t+Pj48fvValXVajXu4TFgBq3ssNDCtWWaA+XTGhvbqAsvvOD4wlwMMhZf\nrVZTrVaLv6MoU2wW3iT9iaRbFzy+XNKnF2mX4OQg5FU/56AP+nTLbqb/hTTdEU2KOBUyibVlDkk6\n18xWmplJer2k/QnsFwXT77VLBr3s0M3aMkmu5IhiS2oq5A2SLpU0J2mvpHe6+1xbG0/iWMinNAbz\nGDBkBtQgijqgmshJTO7+IUkfSmJfKKY0TprK8kSp9hkqWc1CCulEJPQX67kjEWn2qtNOrO2DuGNj\nl2ti4vMDOaiL9GU2z73rA5HcgxdiyeDED62apDdLuk+DWhpCujItywBSeiWDNHvuJ5abVkl6iRYb\n1CW5I0+4EhMS1emkqSROPkr7akInzlD5P0mPKerVkIC0kNyRiqhJeeEHwsKTeI4evV+zs7s1Nra1\nr2eqzg/ilkobNTKyXqXSH+vd737XgscbWf0SuUTNHX0XdbC1fSDz+uvfp0984s5YF4iO8zvkYbYM\nBg8Dqsitqakpbdp0VU9JebEPhJUrz5fZSQM9zx2DJ7OFw4DldHNmZbvFzkYdHn6prr/+fZREgC7Q\nc0cqep0m2amUI4mSCAYGZRnkXq916hDnzQO9IrkjSAxcYtCR3AEgQAyoAgCOI7kDQIBI7gAQIJI7\nYhvkC1YDeUVyRyxpL+QFoDvMlkFkXPYO6D9myyB1g37BaiDPSO6ILMqaMQDSQXJHZCeudc5CXkBe\nUHNHbCwRAPQPyw8AQIAyHVA1s9Vm9q9mtt/MHjaz1yaxXwBANCsS2s/Nkv7T3f/UzFZIel5C+wUA\nRBC7LGNmI5L2uvvLl2lHWQYAepRlWealkp40s8+Z2QNm9hkzKyWwXwBAREmUZVZIWi/pL939e2b2\nD5Kuk3RDe8Px8fHj96vVqqrVagKHRztmrwDFVavVVKvVYu8nibLMaZLudfeXtR6fJ+lad/+jtnaU\nZVIwf2m64eHmCUZcmg4otkynQprZtyS9y90fNbMbJD3P3a9ta0NyT0CnXjlrvQDhyXptmfdK+hcz\n2yfp1ZI+ktB+scByKzCy1guAeZzEVBDd9Mo7tZFEHR4ooKx77uizbnrlS631smvXN1lzHRgw9NwL\nopd6+sK6vCTq8ECB0XMPXC8rMJbLZW3YsEHlcpk6PDCg6LlnKMp89F5/hhk0QLHRcy+YqNceXdgr\n77Y9a64Dg4eeeway6E1z1ipQTFF77kmtCokezNfBZ2dPrINHSbzdJO5yuUxSBwYIZZkMRLn2aKPR\n0NTUlBqNxnO2Ry3vAAgbyT0DvdbBl0rgjUZDY2NbNTu7W0eP3q/Z2d0aG9t6wgcAgMFDzT1D3ZRT\nOtXn6/W6Nm26SkeP3n+8/cjIeu3adYs2bNiQzi8BoK+YLVMg8yUWScvOfOk0Tz1KeQfAYCC5p6zX\nGnmnBM40RwBLoSyTosVKLCtXnq+77tqhc845p2PNfWxsq4aGRjU3N3PCGu1McwTClel67l0diOSu\nqampthr5Dknv0KpVv61nn32s44U1SODAYCK5F8Bze+4vkvRKSTWxLACApTCgWgALa+SrVp0n6TfF\ngl4A+oGeewYajYb27t2riy/ewoJeADqi514g5XJZb3jDG5jpAqBv6LlnjIFSAJ0woAoAAaIsAwA4\njuQOAAEiuQNAgBJL7mZ2kpk9YGY7k9onACCaJHvuV0v6foL7AwBElEhyN7MzJb1Z0j8nsT8AQDxJ\n9dxvkvR+Scx1BIAciH2BbDP7Q0mH3X2fmVUlLTkfc3x8/Pj9arWqarUa9/CZ4gQkAEmr1Wqq1Wqx\n9xP7JCYz+4ikP5P0jKSSpOdL+jd3f1tbu6BOYppfY314uHkxjU7L9QJAVLk4Q9XMzpf0Pnd/yyL/\nF0xy73RdU3rwAJLEGaop6nRdUwDIg0STu7t/a7Fee2i4MDWAvKPnHgEXpgaQd6wKGQOzZQD0Wy4G\nVDseKMDkDgD9xoAqAOA4kjsABIjkDgABIrkDQIBI7gAQIJI7AASI5A4AASK5A0CASO4AECCSOwAE\niOQOAAEiuQNAgEjuABAgkjsABIjknpBGo6GpqSk1Go2sQwEAknsSJid3aHR0jTZtukqjo2s0Obkj\n65AADDgu1hFTo9HQ6Ogazc7uVvOC2dMqlTZqZuYAV2cCEBsX68hIvV7X8HBFzcQuSWs1NDSqer2e\nXVAABh7JPaZKpaJjx+qSpltbpjU3N6NKpZJdUAAGHsk9pnK5rImJbSqVNmpkZL1KpY2amNhGSQZA\npmLX3M3sTEm3SzpN0rOSbnX3Ty7SLpiae6PRUL1eV6VSOZ7EF9sGAHFFrbknkdxPl3S6u+8zs5Ml\n3S/pInc/0NYuiOQ+OblDY2NbNTzcLMdMTGzTli2XZB0WgEBlltwXCeQ/JH3K3b/Rtr3wyZ2ZMQDS\nlovZMmZWkbRO0neT3G9eMDMGQFGsSGpHrZLMFyRd7e6/XKzN+Pj48fvValXVajWpw6fiuTNjmj13\nZsYASFKtVlOtVou9n0TKMma2QtKXJH3F3W9eok3hyzLSr2vuQ0OjmpuboeYOoK8yrbmb2e2SnnT3\nv+rQJojkLjEzBkB6spwt8zpJ35b0kCRv3a5396+2tQsmuQNAWnIzW2bJA5HcAaBnuZgtAwDIB5I7\nAASI5A4AASK5A0CASO4AECCSOwAEiOQOAAEiuQNAgEjuABAgkjsABIjkDgABIrkDQIBI7gAQIJI7\nAASI5A4AASK5A0CASO4AECCSOwAEiOQOAAEiuQNAgEjuABAgkjsABIjkDgABSiS5m9lmMztgZo+a\n2bVJ7BMAEJ25e7wdmJ0k6VFJr5f0hKQpSZe6+4G2dh73WAAwaMxM7m69/lwSPffXSDro7jPuPifp\nDkkXJbBfAEBESST3MyQ9tuDxj1vbgtFoNDQ1NaVGo5F1KADQlRVpHmx8fPz4/Wq1qmq1mubhI5mc\n3KGxsa0aHq7o2LG6Jia2acuWS7IOC0CgarWaarVa7P0kUXM/V9K4u29uPb5Okrv7x9raFa7m3mg0\nNDq6RrOzuyWtlTStUmmjZmYOqFwuZx0egAGQZc19StIrzGzUzIYlXSppZwL7zVy9XtfwcEXNxC5J\nazU0NKp6vZ5dUADQhdjJ3d1/Jendku6W9LCkO9x9f9z95kGl0izFSNOtLdOam5tRpVLJLigA6EIi\nNXd3/6qkVyaxrzwpl8uamNimsbGNGhoa1dzcjCYmtlGSAZB7sWvuXR+ogDX3eY1GQ/V6XZVKhcQO\nIFVRa+4kdwDIsSwHVAEAOUNyB4AAkdwBIEAkdwAIEMkdAAJEcgeAAJHcASBAJHcACBDJHQACRHIH\ngACR3AEgQCR3AAgQyR0AAkRyB4AAkdwBIEAkdwAIEMkdAAJEcgeAAJHcASBAJHcACFCs5G5mHzez\n/Wa2z8zuNLORpAIDAEQXt+d+t6RXufs6SQclfSB+SPlUq9WyDiGWIsdf5Ngl4s9a0eOPKlZyd/dd\n7v5s6+F9ks6MH1I+Ff0FUuT4ixy7RPxZK3r8USVZc3+HpK8kuD8AQEQrlmtgZl+XdNrCTZJc0gfd\n/YutNh+UNOfu2/sSJQCgJ+bu8XZgdoWkd0m6wN2f7tAu3oEAYEC5u/X6M8v23Dsxs82S3i/pDzol\ndilacACAaGL13M3soKRhST9tbbrP3bcmERgAILrYZRkAQP707QxVM3uhmd1tZo+Y2dfMbPUibc40\ns2+a2cNm9pCZvbdf8XTDzDab2QEze9TMrl2izSfN7GDrxK11acfYyXLxm9llZvZg63aPmf1eFnEu\npZvnv9Vug5nNmdlb04xvOV2+fqpmttfM/tvMdqcdYyddvH5GzGxn67X/UGu8LRfMbMLMDpvZdIc2\neX7vdow/0nvX3ftyk/QxSX/Tun+tpBsXaXO6pHWt+ydLekTSmn7FtEy8J0n6gaRRSUOS9rXHIulN\nkr7cuv9aNctQqccaI/5zJa1u3d9ctPgXtPuGpC9JemvWcff4/K+W9LCkM1qPfyvruHuM/wOSPjof\nu5rl2BVZx96K5zxJ6yRNL/H/uX3vdhl/z+/dfq4tc5Gk21r3b5N0cXsDd/+Ju+9r3f+lpP2Szuhj\nTJ28RtJBd59x9zlJd6j5Oyx0kaTbJcndvytptZmdpnxYNn53v8/dj7Ye3qfsnuvFdPP8S9J7JH1B\n0pE0g+tCN/FfJulOd39cktz9yZRj7KSb+F3S81v3ny/pp+7+TIoxLsnd75H0sw5N8vzeXTb+KO/d\nfib3U939cCuwn0g6tVNjM6uo+cn13T7G1MkZkh5b8PjHOvEJbG/z+CJtstJN/Au9U/k66WzZ+M3s\nxZIudvd/UvN8izzp5vk/W9IpZrbbzKbM7PLUolteN/F/WtLvmtkTkh6UdHVKsSUhz+/dXnX13o07\nFXKpE5z+dpHmS47cmtnJavbGrm714NFHZrZR0pVqfhUskn9Qs8Q3L28JfjkrJK2XdIGkVZLuNbN7\n3f0H2YbVtTdK2uvuF5jZyyV93czW8p5NTy/v3VjJ3d03dQjisJmd5u6Hzex0LfE12sxWqJnYP+/u\nd8WJJ6bHJZ214PGZrW3tbV6yTJusdBO/zGytpM9I2uzunb7Gpq2b+H9f0h1mZmrWfN9kZnPuvjOl\nGDvpJv4fS3rS3Z+S9JSZfVvSq9WsdWetm/ivlPRRSXL3H5rZ/0haI+l7qUQYT57fu13p9b3bz7LM\nTklXtO6/XdJSifuzkr7v7jf3MZZuTEl6hZmNmtmwpEvV/B0W2inpbZJkZudK+vl86SkHlo3fzM6S\ndKeky939hxnE2Mmy8bv7y1q3l6rZIdiak8Qudff6uUvSeWb2G2b2PDUH9vanHOdSuol/RtKFktSq\nV58t6UepRtmZaelvc3l+785bMv5I790+jv6eImmXmjNg7pb0gtb2F0n6Uuv+6yT9Ss2R+b2SHlDz\nUymrEevNrXgPSrqute0vJP35gjafVrOn9aCk9VnFGiV+SbeqOcPhgdbzvSfrmHt9/he0/axyNFum\nh9fPX6s5Y2Za0nuyjrnH18+LJH2tFfu0pC1Zx7wg9u2SnpD0tKRDan7LKNJ7t2P8Ud67nMQEAAHi\nMnsAECCSOwAEiOQOAAEiuQNAgEjuABAgkjsABIjkDgABIrkDQID+H/TcEf7q3CoXAAAAAElFTkSu\nQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f85b72b7be0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "X, y = make_data(40, error=1)\n",
    "plt.scatter(X.ravel(), y);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now say we want to perform a regression on this data.  Let's use the built-in linear regression function to compute a fit:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEKCAYAAADpfBXhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmUVOW19/HvRrqlRRDRFifSLblR4qxLo1GjjQpBjch1\nCGKiUdobFUVXco1JbmLkvl41cb2+iTIoYqvAlUHUXKdoELXjxQkcwYGIQzeKCuWEQhga2O8fz2lS\ntD1U11ynf5+1anXVqVPn7Kqu2vXUfp7zHHN3REQkXroVOgAREck+JXcRkRhSchcRiSEldxGRGFJy\nFxGJISV3EZEYUnIX6YCZbTKzAYWOQ6QzlNxFOlZSB4OYWZmZzTaz96IvpqM7WP8rM/syunxlZhvM\n7Mak+883syXR/X8xs11y/ywkU0ru0qWYWTrvecvSvrdKZVlnt9GG/wV+BHzU0Yru3svde7t7b2Bn\n4B/A3dH+aoBrgJOBvkADMKMzMUthKLnHUNRiu9zMXo1aYpPNbKeo1fWlmc0xs+2S1j/czJ42s8/N\n7GUzOybpvnPN7I3ocW+b2U+T7jvGzN43s5+b2XIzW2Zm57YT17lm9k60rXfMbGS0vJuZ/V8zS0T7\nGB21OLslPZ9jk7ZzlZlNS7p9t5l9FMVfb2Z7J913h5lNNLOHzewroMbMyqP9NUaPm2hmWyc95hdm\n9qGZfWBm59FOy93MepvZbdH675vZ1WZm0X0/MbN5Zvb/zOwT4Ko2lpmZ/dbMGszsYzO708x6R9uo\nil6LUWbWCDze/n8f3L3J3W9y92eATR2t38LpwAp3fzq6fRIw290Xu/sG4GrgaDPbo5PblTxTco+v\nU4HjgD2BYcBfgF8BOwJbAZcCmNluwEPA/3H37YHLgXvNbIdoO8uBE6NW3XnAH83swKT97Az0AnYF\nzgcmJH9xNDOzbYAbge9H2zoCeCW6+6fAicABwCGEBNNRKST5/r8A3wR2Al4C7mqx7kjganfvBTwN\n/AH4F2D/6O9uwO+iOIcCPye8dt8Cju8gjinAemAAcBAwmPA6NDsMeDuK7Zo2lp0HnAMcE22nFzC+\nxX6OBgYC34/ifNXMzuwgtnScA0xt5/7mnLFvDvYt2eTuusTsArwHjEy6fQ8wIen2JcB90fUrgCkt\nHv8ocHYb2/4zMCa6fgywGuiWdP9y4DutPG4b4DPgX4EeLe57HPhp0u3BwMbm7UbP59ik+68CprYR\nXx9Ca7VXdPsO4M4W66wC9ki6/V3g3eh6HXBt0n3fimIZ0Mq+dgLWAlsnLTsTeCK6/hOgocVjWls2\nF7gw6faehC+MbkBVtP+qNN8L7wNHp7huFdCUvC/Cl9xyQjKvACYBG4ARhX6f69L+RS33+FqedH1N\nK7e3ja5XAT80s8+iy+fAkcAuAGZ2gpk9a2afRvedQGj9N/vU3ZN/+v8jadubufs/gBHARcBHZvag\nme0Z3b0rIQk1a0z1SUYlnd9H5ZwvCF8E3iLG95PWryR80bzY/JyBR4DmXyqtxdJWzb0KKIueT/Nr\nd0tb+25n2a5s+Zwbge5Av6RlH7QRQzadDcxz982xuPvjwFjgPuDd6PJVnuKRDCi5y/uEVnDf6LK9\nhw62682snNDqvx6o9FC2eYQ0Oxjd/TF3H0Io5fwdmBzd9RHQP2nVqhYPXU1IyM12Trr+I0Jn37Hu\n3geojuJLjjG5hPMJ4Qton6Tn3Mfdm0tJrcXSVonofULLfYek166Pu+/fxr7bWvYhWz7n5hZ08hdy\nPkbsnA3c2XKhu9/s7nu6+y6EJN8deC0P8UgGlNzlv4GTzWxI1AruEXWU7gqUR5dP3H2TmZ0ADEln\nJ1GH7rCo9t5EKI00t/jvBi41s93MbHvgly0e/gpwppl1N7PmmnyzbYF1wOdm1hO4jnYSobs74Uvl\nT1Ernmi/zc/rbuBcM/t2FOvv2tnWx8AcQj9Er6hjdIB1MPSwFTOAn5lZtZltS6jDz0z6RdTpL9Oo\n07hHdHPr5A7jNtY/gvAL4p4Wy7c2s32i698AbgX+5O4rOxuT5JeSezy1TG7tJbsPgFOA/wAShJLA\n5YR69ypCx+vsqHxxJnB/J/fdrBuho3IZofV8NKFEAyHZ/hV4FXgBuLfFY68kdHx+Rqi3J3eYTgWW\nRtt9DXimg/ggfHm8DTwXlXLmEOrcuPujwJ+AJ4C36Hh0yjmEL8A3ovhms+Uvi1TcDkwDngLeIfyy\nuDTp/q+9pmb2WvNoozb8nfCLZ1dCH8o/ouSMmf3azB5u5Xnc6+6rWyzvAUyPRho9R+iQbvMLT4qH\nhYZMCiua1QE/AJY3/+w0s+sJP4nXEd6U57n7lzmKVboIM6si1HbLWtTzRSRFnWm530E0DCvJHELt\n8kBgCfDrbAUmXV5WDhwS6apSTu7uPg/4vMWyuUktq+eA3bMYm3RtJXXIv0ixyWbNfRRhJIVIRty9\n0d23UklGJH1ZSe5m9hugyd2nZ2N7IiKSme6ZbsDCXCInAsd2sJ5+ZouIpMHdO90H1dmW+xYHh0Tz\ncPwCGObu6zp6cKEPx83kctVVVxU8hq4afynHrvgLfyn1+NOVcnI3s+mEMcR7mtnSaLa8cYSDSB4z\ns5fMbGLakYiISNakXJZx97NaWXxHFmMREZEs0RGqKaqpqSl0CBkp5fhLOXZQ/IVW6vGnK+UjVDPe\nkZnna18iInFhZngeOlRFRKQEKLmLiMSQkruISAwpuYuIpOH55+FHP4Innih0JK1TchcRSdG6dTBt\nGnznOzByJBx8MBx0UKGjap1Gy4iIdODDD+GWW+DWW2G//WDMGDjpJNhqq9zvW6NlRESyyB2eeQbO\nPBP22Qc+/RSefBIeewyGDctPYs9ExhOHiYjEydq1MHMmjBsHK1fCJZeEVnufPoWOrHNUlhERAd5/\nH26+GW67LdTSx4yBE06AbgWub6gsI9KFJBIJFixYQCKRKHQoJc0d/vd/4Ywz4IADYNWqcPvRR0NN\nvdCJPRMlHLpI1zRjxiyqqgYyePCFVFUNZMaMWYUOqeSsWQN1dWGky/nnw9FHQ0MD3HQT7LVXoaPL\nDpVlREpIIpGgqmoga9Y8CewPLKSiYhCNjYuprKwsdHhFr7ERJk6E228PwxnHjIEhQ4q7ha6yjEgX\n0NDQQHl5NSGxA+xPWVkVDQ0NhQuqyLmHUS6nnhpq6evXw7PPwsMPw9ChxZ3YM6HRMiIlpLq6mvXr\nG4CFNLfcm5oaqa6uLmhcxWj1arjrrjDqZePGMOpl6lTYdttCR5YfSu4iJaSyspK6uonU1g6irKyK\npqZG6uomqiST5L33YMIEuPNOOOII+OMf4bjjwDpd2ChtqrmLlKBEIkFDQwPV1dVK7ITSy+OPh1b6\n00/DuefC6NEwYEChI8tcujV3JXcRKVmrVoW5XsaNC7XzMWPgxz+Gnj0LHVn2pJvcVZYRkZLz9tuh\n9DJ1KhxzTLheU9P1Si/tiWk/sYjEzaZN8Ne/wg9+AN/9LpSXw0svwX33waBBSuwtqeUuIkXtq69g\nypRQeunRI5Re7r4bttmm0JEVNyV3EUlbLjt233oLxo+H//7vMNpl8mT43vfUQk9VymUZM6szs+Vm\ntjBp2fZmNsfM/m5mfzWz7XITpogUm1xMg7BpE/zlL2HCrqOOCmPSX30VZs8OUwQosacu5dEyZnYU\nsAqY6u77R8v+AHzq7teb2S+B7d39V208XqNlRGIi29MgrFwZxqWPHw+9esGll4Z51Hv0yHbkpSfn\n0w+4+zzg8xaLTwGmRNenAMM7G4CIlJ5sTYPw5ptw8cVQXR2mBLjzTnjxxTBOXYk9M5mOltnJ3ZcD\nuPvHwE6ZhyQixW7LaRCgM9MgbNwIDz4YJuyqqYG+feG118IJMo48MrXSi6Y87li2O1TbrbuMHTt2\n8/WamhpqamqyvHsRyYd0pkH44oswG+P48bDDDqH08uCDsPXWndv3jBmzqK0dTXl5+IKpq5vIyJEj\nMnxGxaO+vp76+vqMt9OpI1TNrAp4MKnm/iZQ4+7LzWxn4El3/3Ybj1XNXSRmUhkt8/rrYRjjrFlw\n4olhKONhh6XXOdoVpzzO1xGqFl2aPQCcC/wB+Alwf2cDEJHSVVlZ2WpSbS69jBsHb7wBF1wQ/u6y\nS2b7a671r1nz9Vp/XJN7ulJO7mY2HagBdjCzpcBVwO+B2WY2CmgEfpiLIEWkNHz2WTjD0YQJsPPO\nofRy+unhaNJs0JTHqUs5ubv7WW3cdXyWYhGRErVwYWil33MPnHxyGJd+6KHtPyadA6A05XHqNCuk\niKRlwwa4//6Q1JcsgYsugn/7N+jXr+PHZtop2pWmPNaUvyIxVWyJ7JNPwlQAN98M3/hG6CA99VQo\nK0vt8V2xUzQTOoeqSAzl4hD/dL38MowaBd/6Vpj35X/+B+bNgxEjUk/soPPA5ota7iLtKGSruRha\nuE1N8Oc/h9JLQ0M4u9H550Mmuy+G51VK1HIXybJCt5oL2cJdsQL+679gjz3CyJfLLgvnJv31rzNL\n7PDPTtGKikH07n0wFRWD1CmaA2q5i7SiGFqXhYjhhRdCK/2BB+C000I9/YADsrPtlr+Ciq0voVip\n5S6SRcVQF85XC3f9epgxI5zd6LTTYO+9w2nsbrste4m9tV9BlZWVHHrooUrsOaKWu0griqHlnhxL\nLlq4H38MkyaFy8CBoZV+8snQPcszTiUSCb7xjT1Zu3YCMBj4SDX2TlDLXSSLiqku3FoLN5NZEZ9/\nHn78Y/j2t+HDD8N5SZ94Av71X7Of2AEmTZrM2rXrgRuAgcCbGh2TB2q5S8nJZ622GOvC6RwAtG5d\nOO/ouHGQSMAll4Rhjdtv3/6+Mn3+rf0Cghp69HCWLn2raF7TYpZuyx13z8sl7EokM9Onz/SKir6+\n3XYHe0VFX58+fWahQ8qrFStWeEVFX4dXHdzhVa+o6OsrVqxodf1ly9yvvNK9Xz/34493v/9+9w0b\nUttXNl7r+fPn+3bbHRzF2nz5pl999TWd3lZXFeXOzufcdB6U1o6U3CVDnU1scdRasuzd+yCfP3/+\n5nU2bXJ/+mn3M89079PH/aKL3F9/vXP7ydZrrf9Z5tJN7qq5S8kohhEshdbeGZDWrg2nqTvkEDjn\nnDBn+nvvwcSJYQRMZ2TrtS6mvouuRjV3KRnFNIKlkCZNmsxll11BeXk1GzYs5frr7+Cjj4Zx221w\n0EFh1MsJJ0C3DJpu2X6ti7HvolRotIzEXnMrsEePY+jZcy969Dimy7UCZ8yYxc9+9ivKyvqzdm0/\nvv3tBfzud8P48kt46il49FE46aTMEjtkv8VdWVlJdXU1DQ0NOu9pnuRg4JNIbpl1Ayqiv11HIpFg\n1Kifs3btK0B/YB2LFl3F4sWXM2DAjlnf38iRIzj++GOz0uKO+3lPi5HKMlIyunJZZulSuPLKj5g2\nrQfu/xy/2Lv3wcydO4lDOzozRqQQ5ZGu/H/LBpVlpChkcnBNR7pah6o71NeHudIPPBAqKrajvHww\nrXWmpqJQE6F1tf9b0UhniE06FzQUMvZyPQa9qwyrW73afdIk9333dR840H3CBPcvvwz3Nb/GvXsf\n1KnXuJCvXS73vWLFCp8/f37s3gPJ0Dh3aU2+3vz5Sh7pJrdS8O677pdf7r7DDu4nn+w+Z04Ys95S\nOv/TVMbH51Iu/m9d5YA2JXf5mny++fOZPOLUWtu0yX3uXPdhw0JS//d/d3/nnezvpxh+9WTz/1YM\nzydflNxlC/l+83elD1s2fPWV+8SJ7nvv7b7PPu633OK+alVu9xmnXz2F/iWST+kmdw2FjKnmTqw1\na77eiZWLEQrN46JrawdRVlZFU1NjlxuDnop33glnNpoyBY4+GsaPh5oasM5PC9Vp2RzaWGhbHqkb\nRuB0pnO5K8jKUEgz+xlQC2wCFgHnufv6Fut4NvYlqSnU8DMdifh1mzbB3Llw001hut1Ro+CiiyBX\neair/A+ax84nNybiOHa+YLNCArsC7wLl0e1ZwDmtrJe73y3Sqjj9DC9FX37pPm6c+157ue+/v/vk\nyWEkTC51lU7GZnHqf2kLaZZlMm65m9muwLPAgcBXwJ+BG919bov1PNN9Sed1lVZcMVmyJJRbpk2D\n444Lc71873vZK7209T/VwULxVLCDmNz9Q8IpVpYCy4AvWiZ2KZw4nqcylwdKpWvTJnjkkTBh15FH\nQs+e8OqrMHt2qK1nK7G3dyCSDhaSZBl3qJpZH+AUoApYCdxjZme5+/SW644dO3bz9ZqaGmpqajLd\nvXQxxTZHycqVYZrdCRNg221DK/2++6CiIvv7SiQS1NaOZs2aJ6OO8oXU1g7i+OOP3TwxlzoZS199\nfT319fWZbyidWo5vWUs/HZicdPtsYHwr6+WmICVFJZc10GIabvnGG+6jR7tvv737iBHu8+a1fsBR\nNqUy/E/9LPFDAU/WsRQ43Mx6mJkBxwFvZmG7UmJyPXdJocsOGzfCQw/BkCEwaBD07QuLFsHMmaEU\nk+vhjO2dqKPZyJEjaGxczNy5k2hsXBzL0SOSonS+EVpegKsICX0hMAUoa2WdHH+/SSHlo1VdqJb7\n55+733CD+4AB7occ4j51qvvatTndZZvUMu96KNRomVRptEy8LViwgMGDL2Tlyhc3L+vsdLSpyOfY\n5tdfD6NeZs6EY49dy0knvcsPflDJTjtVFnQUkkZAdS3pjpZRcpesyOcwvFwmt40b4cEHYdw4eOMN\nuOAC2Gmn+7n88lGbO3Fra8+mrm5a0XTqSrwpuUvBlfIRg599BnV14WTS/frBpZfC6afDypUtv7Tq\ngROB59BYcsmHdJO75paRrMnX3CXZbLkvWhRa6bNnw8knw913Q3IV6etz9PQknOIuP3P2iKRLZ2KS\nrGrvoKlsHHyUjRE5GzbAvfeGCbuGDoX+/WHxYpg6dcvEDq2NUFkNvE+6Z0MSyZt0emHTuaDRMl1a\nunOeJI+bz3S0TCLhft117v37ux9xhPvMme7r16cee/MIlUsuuVQjViRv0HzuUqzSTcotvxCuvvqa\ntObwfukl9/POc+/Tx/3cc91ffDG955B8cFZXmLBKikO6yV01d8m5dOaWb+1Q+2uuOQazbqRyeH1T\nE/z5z6Ge3tAQpth96y1ItyxeWVm5Rawtb4sUGyV3ybl05jxp7QuhvHwPfvGL07n22rZPCLJiBUye\nDDffDAMGwGWXwfDh0F3vdOliNBRS8qKzwyTbGzcPfG20zAsvhFb6Aw/AaaeFCbwOOCAPT0wkxzTO\nXYpeZ4cwdvSFsH59GPUybhwsWwajR8P558MOO+TyWYjkl5K7xFJrXwgffwy33gq33AIDB4ZW+skn\nq/Qi8aTkLrH3/POhlf7ww/DDH8Ill8B++xU6KpHcUnKXWFq3Lhw9Om5c6Cy9+OJwgum+fQsdmUh+\nKLlLrHz4IUyaFMov++4bSi8nnQRbbVXoyETyq2DnUBXJFnd45hkYORL22QcSCXj8cXjsMRg2TIld\npDPUBSUZy3Qir7VrYdasUHr54otQern5ZujTJwfBinQRarlLRjKZyOuDD+C3v4WqKpgxA/7zP8NR\npD/7mRK7SKZUc5e0pXOCDneYNy+00ufOhR/9KIx62WuvvIYuUjI0n7vkXWfmjFmzJrTOx42D1atD\nB+ltt0Hv3vmPW6QrUHKXtKUyZ8zSpeHsRrffHuZKv+46GDIEuqkgKJJT+ohJ2iorK6mrm0hFxSB6\n9z6YiopB1NVNZMcdK6mvD3O8HHRQ6DB9+ulw8NHQoUrsIvmgmrtkrHm0zE47VTNnTiXjxoUpd8eM\ngbPPhl69Ch2hSOlSzV0KZvXqSu6+u5I77oAjjoAbboDjjwfr9NtRRLIlKz+QzWw7M5ttZm+a2etm\ndlg2tivFyz0cYDR8OBxySLg9f36YcnfwYCV2kULLVsv9RuAv7n6GmXUHtsnSdqXIrF4N06aFUS9m\nofRy113Qs2ehIxORZBnX3M2sN/Cyu3+zg/VUcy9h77wDEybAlClw9NEhqQ8apBa6SK4Vcm6ZPYBP\nzOwOM3vJzG41s4osbFcKzB3mzAlzpR9+OJSVwYsvhnOTHnusErtIMctGWaY7cDBwsbu/YGZ/An4F\nXNVyxbFjx26+XlNTQ01NTRZ2Ly1lOtfLV1+FFvr48bD11qGVPmsWbKNim0jO1dfXU19fn/F2slGW\n6Qc86+4DottHAb9095NbrKeyTB40n5quvDwcYNTRuUqTLVkSEvq0aaFlPmZMKMGohS5SOAUry7j7\ncuB9M9szWnQc8Eam25XWJRIJFixYQCKRaPW+2trRrFnzJCtXvsiaNU9SWzu61XWbbdoEjzwCJ54I\nRx4ZOkZffRXuuQeOOUaJXaRUZetYwUuBu8zsFeAA4NosbVeSdDQDY/NcL2EqAEie66WllSvhxhvD\nOUh/8xs44wxobIRrr4X+/XP9TEQk13SEaolIZQbG9taBkPybmr7J9Ol9mT49jEe/9NJw4JFa6CLF\nSWdiirlUWuVtzfUyZ84T7L77pRxxxAaOPHIDy5a9zqJFoZP0yCOV2EXiSC33EtGZudObR8tsv/0e\nTJ/eg7FjE7j3IxxbtoiKipp251wXkeKhlnvMtdUqby1BJxKV3H77oRx66I4888w6ttlmLP88aHi/\nNuvwIhIfarkXUDrj0dt6zMaN8NBDYVqA11+HCy4Il+7dO3+2JBEpHpoVssSkOx69srJyi6T82WdQ\nVxdOiNGvXxibfsYZUF6++RHU1U2ktnYQZWVVNDU1ttniF5H4UMu9ANI592hLixaFVvrs2WF6gDFj\nwpmO2ttnJketikhhqOVeQjpz7tFkGzaEKXVvuikcTXrhhbB4MXTrFhJ3ItF24m7Z4heReFOHagFs\nee5RaO3co8k++QSuvHIV/fuv4/e/b+Kii6ChAa68Ep54ov0Dm0Ska1JyL4BUR7688grU1kJV1Xqu\nu+4BVq0axWuv7cymTbMoK0tvugER6RqU3Atk5MgRNDYuZu7cSTQ2Lt7cmdrUFOro3/teqKXvvPNq\nNm3al40b92XVqru2SOCdmW5ARLoWJfcCaJ78C+DQQw+lsrKSRAKuuQb22CN0lF52Gbz3Hgwf/gZb\nb92L1hJ4Z8s7ItJ1KLnnWcvJv665Zg7nngt77hmS+cMPw1NPwemnQ/fu7dfnO3Ngk4h0LRoKmUdf\nHwK5GviMUaPWcMUVfdlrrx1bfVzzmPjkcerJY+I1zFEkvtIdCqnknkePPvoyw4c/z7p1F0ZL5gEn\n0rPnADZter/dA5mUwEW6JiX3IjZ/fqijP/jgJlatmsrGjUcA2wN7AfVoWgARaYsmDisy69fDXXfB\nYYfBiBFwwAHw7rvdmDatgoqK79Kz51HADmiki4jkglruWfbRR3DLLXDrrbDPPuFkGCedBFtt9c91\nEokEL7/8MsOHj9SEXiLSLrXcC8gdnn0WzjoL9t4bEgl4/HGYOxeGDdsysUM4iGnIkCEa6SIiOaOW\newbWrg1nMxo3Dr74Ai6+GM47D/r0SX0b6igVkfaoQzWPli2Dm2+GyZPhoIPCjIwnnADd9DtIRLJM\nZZkcc4d580Ln6H77wcqV4WCjRx8NNXUldhEpJprytwNr1sCMGaH0sno1XHJJaLH37l3oyERE2qay\nTBuWLg2ll7q6cBKMMWNgyBC10EUkvwpeljGzbmb2kpk9kK1t5ps7/O1vcNppoZa+Zg08/XSY72Xo\nUCV2ESkd2SzLXAa8AZRcweIf/wgHHI0bF6bcHTMG7rwTevUqdGQiIunJSlvUzHYHTgRuy8b28qWh\nAa64Aqqq4MEH4YYb4I03YPRoJXYRKW3ZKjT8EfgFUPRFdXd44gkYPhwOOQQ2boTnnw/nJh08GKzT\nlS0RkeKTcVnGzE4Clrv7K2ZWA7SZHseOHbv5ek1NDTU1NZnuPmWrV8O0aTB+fLg9ZkwoxfTsmf42\ndQCSiGRbfX099fX1GW8n49EyZnYt8GNgA1AB9ALuc/dzWqxXkNEy774LEybAlCnh1HVjxsCgQZm3\n0JvnWC8vDyfTaG+6XhGRdBXFEapmdgzw7+4+rJX78pbc3eGxx0IH6XPPhSkBRo+GbJ197usn3dCk\nXyKSG+km91gdxPTVVzB1aii9lJeHVvqsWbDNNtndT/OJqdes+fp0vUruIlIMsprc3f1vwN+yuc1U\nLFkSSi/TpoWSyy23wNFH565zdMvzmoaWu05MLSLFJBYt96lTQ+v8lVegf//c76/5xNS1tYO2OK+p\nWu0iUiw0/UAGNFpGRHKtKDpU291RDJO7iEiuFXxuGRERKR5K7iIiMaTkLiISQ0ruIiIxpOQuIhJD\nSu4iIjGk5C4iEkNK7iIiMaTkLiISQ0ruIiIxpOQuIhJDSu4iIjGk5C4iEkNK7iIiMaTkniWJRIIF\nCxaQSCQKHYqIiJJ7NsyYMYuqqoEMHnwhVVUDmTFjVqFDEpEuTifryFAikaCqaiBr1jxJ8/lUKyoG\n0di4WGdnEpGM6WQdBdLQ0EB5eTUhsQPsT1lZFQ0NDYULSkS6PCX3DFVXV7N+fQOwMFqykKamRqqr\nqwsXlIh0eUruGaqsrKSubiIVFYPo3ftgKioGUVc3USUZESmojGvuZrY7MBXoB2wCJrv7Ta2sF5ua\neyKRoKGhgerq6s1JvLVlIiKZSrfmno3kvjOws7u/YmbbAi8Cp7j74hbrxSK5z5gxi9ra0ZSXh3JM\nXd1ERo4cUeiwRCSmCpbcWwnkf4Bx7v54i+Uln9w1MkZE8q0oRsuYWTVwIPB8NrdbLDQyRkRKRfds\nbSgqydwDXObuq1pbZ+zYsZuv19TUUFNTk63d58WWI2NCy10jY0Qkm+rr66mvr894O1kpy5hZd+Ah\n4BF3v7GNdUq+LAP/rLmXlVXR1NSomruI5FRBa+5mNhX4xN1/3s46sUjuoJExIpI/hRwtcyTwFLAI\n8OjyH+7+aIv1YpPcRUTypWhGy7S5IyV3EZFOK4rRMiIiUhyU3EVEYkjJXUQkhpTcRURiSMldRCSG\nlNxFRGJIyV1EJIaU3EVEYkjJXUQkhpTcRURiSMldRCSGlNxFRGJIyV1EJIaU3EVEYkjJXUQkhpTc\nRURiSMnF8vTNAAAG+klEQVRdRCSGlNxFRGJIyV1EJIaU3EVEYkjJXUQkhpTcRURiSMldRCSGspLc\nzWyomS02s7fM7JfZ2KaIiKTP3D2zDZh1A94CjgM+BBYAZ7r74hbreab7EhHpaswMd7fOPi4bLffv\nAEvcvdHdm4CZwClZ2K6IiKQpG8l9N+D9pNsfRMtiI5FIsGDBAhKJRKFDERFJSfd87mzs2LGbr9fU\n1FBTU5PP3adlxoxZ1NaOpry8mvXrG6irm8jIkSMKHZaIxFR9fT319fUZbycbNffDgbHuPjS6/SvA\n3f0PLdYruZp7IpGgqmoga9Y8CewPLKSiYhCNjYuprKwsdHgi0gUUsua+APgXM6sys3LgTOCBLGy3\n4BoaGigvryYkdoD9KSuroqGhoXBBiYikIOPk7u4bgUuAOcDrwEx3fzPT7RaD6upQioGF0ZKFNDU1\nUl1dXbigRERSkJWau7s/CuyVjW0Vk8rKSurqJlJbO4iysiqamhqpq5uokoyIFL2Ma+4p76gEa+7N\nEokEDQ0NVFdXK7GLSF6lW3NXchcRKWKF7FAVEZEio+QuIhJDSu4iIjGk5C4iEkNK7iIiMaTkLiIS\nQ0ruIiIxpOQuIhJDSu4iIjGk5C4iEkNK7iIiMaTkLiISQ0ruIiIxpOQuIhJDSu4iIjGk5C4iEkNK\n7iIiMaTkLiISQ0ruIiIxpOQuIhJDGSV3M7vezN40s1fM7F4z652twEREJH2ZttznAPu4+4HAEuDX\nmYdUnOrr6wsdQkZKOf5Sjh0Uf6GVevzpyii5u/tcd98U3XwO2D3zkIpTqb9BSjn+Uo4dFH+hlXr8\n6cpmzX0U8EgWtyciImnq3tEKZvYY0C95EeDAb9z9wWid3wBN7j49J1GKiEinmLtntgGzc4F/A451\n93XtrJfZjkREuih3t84+psOWe3vMbCjwC+Do9hI7pBeciIikJ6OWu5ktAcqBT6NFz7n76GwEJiIi\n6cu4LCMiIsUnZ0eomtn2ZjbHzP5uZn81s+1aWWd3M3vCzF43s0Vmdmmu4kmFmQ01s8Vm9paZ/bKN\ndW4ysyXRgVsH5jvG9nQUv5mdZWavRpd5ZrZfIeJsSyqvf7TeoWbWZGan5jO+jqT4/qkxs5fN7DUz\nezLfMbYnhfdPbzN7IHrvL4r624qCmdWZ2XIzW9jOOsX82W03/rQ+u+6ekwvwB+CK6Povgd+3ss7O\nwIHR9W2BvwMDcxVTB/F2A94GqoAy4JWWsQAnAA9H1w8jlKHyHmsG8R8ObBddH1pq8Set9zjwEHBq\noePu5Ou/HfA6sFt0e8dCx93J+H8NXNccO6Ec273QsUfxHAUcCCxs4/6i/eymGH+nP7u5nFvmFGBK\ndH0KMLzlCu7+sbu/El1fBbwJ7JbDmNrzHWCJuze6exMwk/Ackp0CTAVw9+eB7cysH8Whw/jd/Tl3\nXxndfI7CvdatSeX1BxgD3AOsyGdwKUgl/rOAe919GYC7f5LnGNuTSvwO9Iqu9wI+dfcNeYyxTe4+\nD/i8nVWK+bPbYfzpfHZzmdx3cvflUWAfAzu1t7KZVRO+uZ7PYUzt2Q14P+n2B3z9BWy5zrJW1imU\nVOJPdj7FddBZh/Gb2a7AcHe/mXC8RTFJ5fXfE+hrZk+a2QIzOztv0XUslfjHA3ub2YfAq8BleYot\nG4r5s9tZKX12Mx0K2dYBTr9tZfU2e27NbFtCa+yyqAUvOWRmg4DzCD8FS8mfCCW+ZsWW4DvSHTgY\nOBboCTxrZs+6+9uFDStl3wdedvdjzeybwGNmtr8+s/nTmc9uRsnd3Qe3E8RyM+vn7svNbGfa+Blt\nZt0JiX2au9+fSTwZWgZ8I+n27tGyluv072CdQkklfsxsf+BWYKi7t/czNt9Sif8QYKaZGaHme4KZ\nNbn7A3mKsT2pxP8B8Im7rwXWmtlTwAGEWnehpRL/ecB1AO7+jpm9BwwEXshLhJkp5s9uSjr72c1l\nWeYB4Nzo+k+AthL37cAb7n5jDmNJxQLgX8ysyszKgTMJzyHZA8A5AGZ2OPBFc+mpCHQYv5l9A7gX\nONvd3ylAjO3pMH53HxBd9iA0CEYXSWKH1N4/9wNHmdlWZrYNoWPvzTzH2ZZU4m8EjgeI6tV7Au/m\nNcr2GW3/mivmz26zNuNP67Obw97fvsBcwgiYOUCfaPkuwEPR9SOBjYSe+ZeBlwjfSoXqsR4axbsE\n+FW07ALgp0nrjCe0tF4FDi5UrOnED0wmjHB4KXq95xc65s6+/knr3k4RjZbpxPvncsKImYXAmELH\n3Mn3zy7AX6PYFwIjCx1zUuzTgQ+BdcBSwq+MUvrstht/Op9dHcQkIhJDOs2eiEgMKbmLiMSQkruI\nSAwpuYuIxJCSu4hIDCm5i4jEkJK7iEgMKbmLiMTQ/wc7LvGhZ9qr5wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f85b72bf518>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "X_test = np.linspace(-0.1, 1.1, 500)[:, None]\n",
    "\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.metrics import mean_squared_error\n",
    "model = LinearRegression()\n",
    "model.fit(X, y)\n",
    "y_test = model.predict(X_test)\n",
    "\n",
    "plt.scatter(X.ravel(), y)\n",
    "plt.plot(X_test.ravel(), y_test)\n",
    "plt.title(\"mean squared error: {0:.3g}\".format(mean_squared_error(model.predict(X), y)));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have fit a straight line to the data, but clearly this model is not a good choice.  We say that this model is **biased**, or that it **under-fits** the data.\n",
    "\n",
    "Let's try to improve this by creating a more complicated model.  We can do this by adding degrees of freedom, and computing a polynomial regression over the inputs. Scikit-learn makes this easy with the ``PolynomialFeatures`` preprocessor, which can be pipelined with a linear regression.\n",
    "\n",
    "Let's make a convenience routine to do this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "from sklearn.linear_model import LinearRegression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we'll use this to fit a quadratic curve to the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "X_poly = PolynomialFeatures(degree=2).fit_transform(X)\n",
    "X_test_poly = PolynomialFeatures(degree=2).fit_transform(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEKCAYAAADpfBXhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYVNW19/HvQmhpEIhDO4CxWzQENRLFYHy9SWhQIs5T\nouJVg6Iml9chiSY45BVMgrNXjciVYDvgAJiocYwDXjpGZRJFHEDF0I3iQIEGVKaGXu8fuxqLpofq\nmk5V9e/zPOehhl3nrCq61tm1zj77mLsjIiLFpUPUAYiISOYpuYuIFCEldxGRIqTkLiJShJTcRUSK\nkJK7iEgRUnIXaYWZ1ZtZ76jjEGkLJXeR1hXcySBmdoiZLTCzL83seTPbrYW2feNt/m1m75rZcQnP\nfd/MnjWzFWb2qZlNNbOdc/MuJB1K7tKumFkqf/OWoW1vlcxjbV1HE222Bx4CLge2A+YCU1tY36PA\nY8C2wM+B+8xsz3iTbYEJQHl8+RK4qy0xS0TcXUuRLcBi4GLgdeALYCKwI/AUsAp4FuiR0P4g4CXg\nc+A1YGDCc8OBt+OvWwScm/DcQOAD4NfAp8BSYHgLcQ0H3o+v631gWPzxDsANQCy+jZFAPdAh4f0M\nTljPaODehPsPAh/H468G9k547i5gPPBk/LMYDJTEt1cbf914YOuE1/wG+Aj4EDgT2Aj0buY9dQfu\niLf/APgDYPHnfga8CPw3sBz4fTOPGfA7oAb4BLgb6B5fR3n8szgrHm91Ev//5wAvJtzvAqwG+jTR\ndh9gVaPHngGubGbd+wMro/4b19L6op578ToBOAToAxxDSOyXADsAWwEXAJhZL+AJ4Pfuvi1hp/BQ\nvPcHIWkf4e7dCYnuJjPbL2E7OwPdgJ7A2cBtZtajcTBm1gW4BTgsvq6DgXnxp88FjgC+C3wP+Amt\nl0ISn38K2IOwA3sVuL9R22HAH9y9G2Endi2wJ9Av/m8v4Ip4nEMJO6tDgG8Bh7YSxz3AeqA3IfEN\nIXwODb5P2GHtCIxt5rEzgTMIO8vehM9zXKPt/AjoCxwWj/N1MzulmZj2IezYAXD31fHt7dPKe2lg\nwHeaeW4g8FaS65EoRb130ZL5hdDTHZZw/6/AbQn3zwMejt/+LXBPo9c/DZzezLofAc6P3x4IfEW8\nhx1/7FPgwCZe1wX4DDge6NzouefZ/BfBEEJvuaWe+6Rm4vsGoafbLX7/LuDuRm2+BHZPuP9/gH/F\nb1cBVyU89y2a6bkTkvNaNu/1nwL8b/z2z4CaRq9p6rFpwC8S7vch7DA6EHruG4HyNvz/35H4HuKP\nvQic0UTbjoTEf3H89o+BdcDfm2jbD1gBHBz137iW1hf13IvXpwm31zRxf5v47XLgJDP7LL58DvwH\nsAuAmR1uZjPiB9Q+Bw4n9P4brHD3+oT7qxPWvYmH3uPJwH8BH5vZ42bWJ/50T0JJo0Ftsm/SzDqY\n2TVmtsjM/k3YEXijGD9IaF9G2NHMbXjPwN+Bhl8qTcXSXM29HOgUfz8Nn93tzW27hcd6svl7riUk\n2p0SHvuwmRia8iWhXJSoB6EstRl33wAcBxxFKFH9ilCf32x78Rr8U4Qd+8ttiEUi0jHqACRyHxB6\nwT9v/ISZlRB6/acBj7p7vZk9QooHGN39OeA5M9uaUI6YSOj9fwx8M6FpeaOXfkVIyA0SR2v8J3A0\noWe/JF4S+rxRjIklnOWEHdA+7v5xE2E2FUtzJaIPCD337T3etW1CU483fuwjNn/P5UAdYYf8zWZe\n05K3CL8QADCzroSyVZPlFHd/E6hMaP8Soe7fcL8ceI5Qh3+gDXFIhNRzl/uAo83sx/FecGczG2hm\nPQkHHkuA5fHEfjjhZ3ubmdmOZnZMvPZeR+hdNvT4HwQuMLNeZrYtMKrRy+cBp5hZRzNrqMk32IZQ\nRvg8nsSupoVEGE/CE4Gb47144ttteF8PAsPNbK94rFe0sK5PCAenbzKzbhb0NrMfJfGRJJoM/MrM\nKsxsG8KOb0rCL6K27kwfAfYxs+PjO9LRwDx3f7epxma2r5ltbWZdzOxiws7z7vhzvQhls1vdfWIb\n45AIKbkXp8bJraVk9yFwLHAZYbRKLaH+2sHdvyQceP1LvHxxCmHYXFu23aAD4UDlUkLv+UeEEg2E\nZPsM4SDgK4RhfIn+H+HA52eERJV4wHQSsCS+3jeBZEoGowh15pnxUs6zhDo37v40cDPwv8C7hMTW\nkjMIO8C34/H9hc1/WSTjTuBe4AXCKKLVxA94x23xmZrZm2Y2rKmVufty4ETgqnhM3yP83zW89lIz\nezLhJacTfrF8AgwChrh7Xfy5EcDuwBgzW2VmX5jZqja+P4mANf9rslFDsypCXe5Td+8Xf+w6wk/i\ndYQ/yjPdXf/xkpZ4GeBfQKdG9XwRSVJbeu53ER+GleBZQu1yP+A94NJMBSbtXkZOHBJpr5JO7u7+\nIuFAVeJj0xJ6VjOBXTMYm7RvBXfKv0g+yWTN/SzCkDKRtLh7rbtvpZKMSOoyktzN7HKgTsOkRETy\nQ9rj3M1sOOHU8cGttNPPbBGRFLh7m49BtbXnbiQc6IrPw/Eb4Bh3X9fai6M+HTedZfTo0ZHH0F7j\nL+TYFX/0S6HHn6qkk7uZPUAYQ9zHzJaY2ZnArYSTSJ4zs1fNbHzKkYiISMYkXZZx91ObeFjzOouI\n5CGdoZqkysrKqENISyHHX8ixg+KPWqHHn6qkz1BNe0NmnqttiYgUCzPDc3BAVURECoCSu4hIEVJy\nFxEpQkruIiJFSMldRKQIKbmLiBQhJXcRkSKk5C4iUoSU3EVEipCSu0gBisVizJkzh1gsFnUokqeU\n3EUKzOTJUykv78uQIb+gvLwvkydPjTokyUOaW0akgMRiMcrL+7JmzXSgHzCf0tJB1NYupKysLOrw\nJAs0t4xIO1BTU0NJSQUhsQP0o1OncmpqaqILSvKSkrtIAamoqGD9+hpgfvyR+dTV1VJRURFdUJKX\nlNxFCkhZWRlVVeMpLR1E9+79KS0dRFXVeJVkZAuquYsUoFgsRk1NDRUVFUrsRS7VmruSu4hIHtMB\nVRER2UTJXUSkCCm5i4gUISV3EUmZpkHIX0kndzOrMrNPzWx+wmPbmtmzZvaOmT1jZj2yE6aI5BtN\ng5Dfkh4tY2Y/AL4EJrl7v/hj1wIr3P06MxsFbOvulzTzeo2WESkSmgYhd7I+WsbdXwQ+b/TwscA9\n8dv3AMe1NQARKTyaBiH/pVtz39HdPwVw90+AHdMPSUTyXdTTIKjW37qOGV5fi3WXMWPGbLpdWVlJ\nZWVlhjcvIrnQMA3CiBGD6NSpnLq62pxNgzB58lRGjBhJSUnYwVRVjWfYsJOzvt1cqa6uprq6Ou31\ntOkMVTMrBx5PqLkvACrd/VMz2xmY7u57NfNa1dxFikyup0Foj7X+XJ2havGlwWPA8PjtnwGPtjUA\nESlcZWVlDBgwIGeJVbX+5LVlKOQDwMtAHzNbYmZnAtcAQ8zsHeCQ+H0RkayIutZfSDRxmIhEItWS\nTkPNPbHWX0w198Y0K6SIFIx0D4q2pymPldxFilSxJbL2eFA0HZryV6QIFeMp/joomhvquYu0IMpe\nc7H2cIv1fWWLeu4iGRZ1r7lYe7i6DmxuqOcu0oR86F22FoM7rFkDq1d/vTTc37Dh6/VYQp+vc2fo\n2vXrpUsXKC2FDjno5jX+FVRsxxKyJdWee6anHxApCg295jVrtuw1ZzsRbdgAH38MS5aUMWLEE9x+\n+98wW0B9fQm9e7/NoYeWEYvB8uWw1VYhSZeWhkTdkKw7dQrrSuxP1dfDunXw1VdhWb06/Lt+PWy7\nLZSVwQ47hH/LymDnnWG33aCiAsrL4ZvfhJKS1N5Tc6NjlNSzRz13kSbkoue+YgUsWBCWhQu//vfD\nD0OSLS8PyXWHHVZTWhqjT59t6d27OzvsEJ7ffvuQyNO1YQN89hnEYpsvYQcDNTVQWwsffRSS/p57\nwt57w157hWXvvaFnz81/ISSKxWLstlsf1q69DRgCfKwaexuo5y6SQZmcGMsdli6FV1+FuXO/Xlav\n/jpB9u0LAweGf8vLYeutE9fQBSjfbJ2xWIw338xMSaNjR9hxx7C0ZMOGkODfew/efjvsjB55JNxe\nuxb22w8OOAC+972w7LlnKPdMmDCRtWvXAzcC5wPjc/YrqD1Tz10KTi5rtalsq64OXnsN/vnPsMyc\nGUoiBxwA/fuHfw84IPTKm+vttiSXsyIm+/5XrAjv+ZVXwjJ3bvg10K/fembOvIUNG04EehOmDaik\nc2dnyZJ3ldyTkGrPHXfPyRI2JZKeBx6Y4qWl23mPHv29tHQ7f+CBKVGH5Bs2uL/8svuVV7ofcoh7\nt27u++7rPnKk++TJ7rW17vX1mdnWsmXLvLR0O4fXPfwmeN1LS7fzZcuWZWYDCdL9rGMx95tvfse3\n3roqHmvDMsFPO22Kf/llxkMuSvHc2facm8qLUtqQkrukKZeJrTUffOB+xx3uP/2p+3bbhWR+0UXu\njz/u/tln2dvu7NmzvUeP/psly+7d9/fZs2dndDuZ+qy3XM8i79hxrB988DrfZhv3Qw91v/FG97fe\nytwOsNikmtw1zl0KRpTjvt1h3jy44gro1y/Ul597Do44At54A+bPhxtugKOOCiNPsiVXsyJm6rPe\nckz7gUyatAcvvVTC0qVw3nmhhn/EEeFYw8iRMG1aKG1JelRzl4KR67Hn9fUwYwY8/HBYOnSA448P\ny0EHhWGIUZgwYSIXXvhbSkoq2LBhSVZq7pn+rFur3bvDO+/Ao4+Gz/r99+Hoo+HEE+HQQ8P4/PZK\nNXdpFx54YIp37vwN79q1j3fu/I2M19zr693nzXO/+GL3nj3dv/Md99Gjw2P5UDZoqIN367avb711\nd7/99j9nfVvdu++fkeMby5Yt89mzZydV2lmyxP2WW9x/9CP3Hj3czzjD/dlnw/GN9gbV3KU9aEg4\nXbt+N6MHVJcscb/mmpDMy8vdL73U/c03M7LqjInimENbEnJL0jk4+/HH7jff7H7AAe677BKObcyb\nl1Y4BUXJXYpeppPbunXuf/mL+5Ah4aDouee6v/CC+8aNGQ48QzJxMDVTybotMvn/9vbb7pdd5r7b\nbuEg9i23ZPcAdj5INbnrgKpkVCwWY86cOcRisYyvO1MH+RYtgksuCePMb7sNzjwznGQ0YQL88Ie5\nmWclFekeTI1qIrRMHgjfay8YOxYWL4ZbbgnHRHbfHYYPD7d1WC9BKnuEVBbUcy962R6Dnk4PcMMG\n90cfDePQy8pCTX3hwoyGlxOp1sGjHEaazW0vW7bMn3nmVb/iii98zz1Db378eC+qMfSoLCNNydXP\n8Fwlj7Ymt1Wrwk/3PfZwHzDA/f773deuzWhIOZfK/2muxsc3J9MHZxPX2dCZuO++KT5tmvtxx7lv\nv737qFHhfIRCp+QuW8jl2Zy5TB7JJLfFi91//etQS//pT91feik/RrtEJR9OAMtkR6O197NokfsF\nF7hvu637qae6z5mT9iYjo+Qum8n1lzkfkod7ONPx9NNDUr/4YveampxuPq9lo/cclWQ7E59/7n7D\nDeEA7MCB7s89V3g7eSV32UwUP8OjTB6vvOJ+wgnuO+7oPnZs+FLLlqIYLZMNbe1M1NW5T5rk3rev\n+4EHuj/2WOEk+UiTO/Ar4E3CYfz7gZIm2mT5I5BEUfWkc508XnzR/bDD3Hv1cr/ppuI6kJaqYkng\nrUmlM7FxYxj+ut9+7v36uU+Zkv8nRkWW3IGewL8aEjowFTijiXZZ/giksWL6Gd7YK6+4Dx0aTjj6\n858L/yBppuTjrJnZlOqOrL7e/Ykn3A86yH2ffdwffjh/e/KpJve055Yxs57ADGA/4AvgEeAWd5/W\nqJ2nuy1pu2K7TuVbb4XJu2bOhMsug7PPbnxhi+LX3P9pPlz3tdC4w1NPweWXh0sIjh0b5rJJZZ79\nbEl1bpm0T9dw948Il1hZAiwF/t04sUt0ysrKGDBgQMF/uRcvhtNPh8GDYd99v2TKlLmcdFKs3SX2\nlk5EinLWzEJlBkceGa6SdfHFYZbKQYPg5Zejjix9mei5fwN4CPgpsBL4K/AXd3+gUTsfPXr0pvuV\nlZVUVlamtW0pfitXht7UnXfC+efDrrs+xPnnn5uTqxDlm9Z65uq5p2/DBrj3XhgzBgYMgGuvhT32\nyG0M1dXVVFdXb7p/5ZVXptRzz0TN/SfAxIT7pwPjmmiXjXKU5JlMHcxbv9593Dj3nXZyHzHC/aOP\n8me4ZVSSGQFVzMdZcmn1averrgonQ110UbSjr4hwbpklwEFm1tnMDDgEWJCB9UqBycTcJe7w5JPh\nghiPPALPPAN33AG77KKyQzJzywwbdjK1tQuZNm0CtbUL282vmkwrLYVLLw3HeL74Ar79bRg3rsAu\nIpLKHqHxAowmJPT5wD1ApybaZHn/JlHKRK964UL3H/84jEV+4oktRy+09567u3rmUZk/P8we2rdv\nOBEql9BJTBKldE6a+uqrMI3r9tuH62muX99826iSW+NyU5RjydvLOPZ8U18fJp/bffcwpcWSJbnZ\nrpK7RCqVXnV9vfvf/hbGqp98svuHHya/rVwmt8Zjx88778J2NZZcNrd6tfsVV4TOyNVXh+sCZJOS\nu0SuLb3q9993P/LI8DN32rQcBtlGW+60pjuUtuvSkASLFrkffbT7U09ldzupJveOua7xS/EaNuxk\nDj10cIsnTW3YADfdFIaYXXxxuBhySUnbtpPLE7MaDuKuWdNwELcr8E2aOqir4Ybtyx57wGOPRR1F\n8/L0mjNSqFo6aWr69M/Yd9+vePzx9cyeHa6G1NbEnuurCW05QuUr4ANSvRqSSK4ouUvWrVsHxx//\nFoMHb6S29gbmzNmFWbOSS8qJl+2LxWKMGDGSNWums3LlXNasmc6IESOzckm/BmVlZVRVjae0dBDd\nu/entPREzjvvnIT7g6iqGq9eu+SdtM9QTXpDmlumXZoxA4YP38CiRc9QX787sDfJnjk5efJURowY\nuels1Msuu4gbbniIlSvnbmrTvXt/pk2bwIABA7L6PhqXgoptzh7JX6nOLaPkLlmxdi387ndw//1w\nwQWLuOaak1m1Kvmk3NSp9J07D8Ssg06vl3YlsonDRBqbNy/My1FTA2+8AWef3YO6uhraUqdu6mzU\nkpLdueyyi1QSEUmCeu6SMRs3wvXXw403wn//N5x22tdTpzaUWDp1KqeurrbVCb9amgQLUElE2g2V\nZSRS//oXnHEGdOoEd98N5eVbtmlrnbqtOwSRYqTkLpFwh7vuglGjwkRLv/wldMhgsU8HLqW9U3KX\nnFu1Cn7xC5g/H6ZMge98J+qIRIqPDqhKTs2dCwccAN26wezZSuwi+UbJXdrEHW6+GYYOhT/+ESZM\ngC5doo5KRBrT3DKStBUr4Mwz4ZNPYNYs6N076ohEpDnquUtSZs2C/feHPn3gxRc3T+yJUwSISH5Q\ncpcWucPtt8PRR8Ott8INN2w+2VeuJ/ISkeRotIw0a/Vq+K//gldfDVPzfutbmz/f0olGGrYokhka\nLSMZ9f77cPDBYf71mTO3TOygC1aL5DMld9nCk0+GxH722XDffdC1a9PttpzrXHObi+QLjZaRTdxh\n7NhQY3/kkZDgW9Iw1/mIEYM2myJAJRmR6KnmLkCor591FixeDH/7G+yyS/Kv1RQBItmj6QckZR9+\nCMcdB3vtBRMnQufOUUckIg0iPaBqZj3M7C9mtsDM3jKz72divZJ9s2bBQQfBSSfBpElK7CLFIlM1\n91uAp9z9p2bWEdAJ6QXgvvvg17+GO++Eo46KOhoRyaS0yzJm1h14zd33aKWdyjJ5or4eLr8cHnwQ\nHnsM9tkn6ohEpDmplmUy0XPfHVhuZncB3wVeAS509zUZWLdk2Nq1MHx4qLPPmgU77BB1RCKSDZlI\n7h2B/sD/dfdXzOxm4BJgdOOGY8aM2XS7srKSysrKDGxeGmtu9MqKFXDssdCrF0ybpvq6SD6qrq6m\nuro67fVkoiyzEzDD3XvH7/8AGOXuRzdqp7JMDjRcmq6kJJxg1HBpuvffh8MPh+OPh6uvzuzVkkQk\neyIdCmlm/wDOcfd3zWw00MXdRzVqo+SeAS2NKW9urpe//nURI0Zsy+jR4cpJIlI4op5b5gLgfjOb\nR6i7X5Wh9UqC1mZgbGquFziT//zPbaiqUmIXaU90ElOBSGYGxi3bLAU6MG1aZ/r126CzSEUKUNQ9\nd8myZGZgbJjrpXPnQZSUTMBsHTfdNJdly57VnOsi7Yx67gUi2bnTN26Ec85Zw0sv1fPww2vYcUfX\nnOsiBUw99yLX0CsvLR1E9+79KS0dtMUMjOvWwamnwuLFpcye3ZV99tlBc66LtFPquUcoldkUm3vN\nF1/ACSdAt27wwANfj2HX1ZJECpt67gUm1WuPlpWVMWDAgM0ScywGgwdDRUWYUiDx5KRkevwiUnzU\nc49AJnvTS5fCIYeEXvvYsWDN7N8157pIYYpybhlpo4Y6+Jo1W9bB25J4a2pCYj/11C855pgFLF/e\nfOIuKytTUhdpR1SWiUAq1x6NxWLMmTOHWCwGwHvvwcCB8MMfvsqNN5ZrmKOIbEbJPQJtrYM3rs9f\nf/3fGTQILrzwCx58cAhr1kxn5cq5rFkznREjRm7aAYhI+6Wae4SSqYNvWZ9/F+jObbd1YcCAdxgy\n5BesXDl3U/vu3fszbdoEBgwYkJP3ICLZpdEyBaShxAJsMfKlsS3HqfehtPRGBgx4J6Xyjoi0D0ru\nOdbWIZBfJ/BF8UcWA3du6u1rmKOINEVlmRxqaghk584DefTRqey///7NJuUxY57nyiv70aXLaNyn\nbpqjPXG9GuYoUpwinc89qQ0puTNnzpxGNfKpwFl07fot6us/2CJpA8yYEa6eNG7cv9l99/eUwEXa\nGSX3ArB5z30X4NtANc2dyDRzJhxzDNx7Lxx2WGRhi0iEdEC1ACTWyLt2/QGwPc1N6DV7duix33OP\nEruItJ167hGIxWK89tprHHfcsCanIKitLePII6GqCo46KupoRSRKKssUoIaLWXfqVE5dXS1VVeP5\n9rdP5vDDYeLEUJIRkfZNyb1AJY50Wbq0jKFD4fbb4bjjoo5MRPKBknuBW7AgTNs7bhyceGLU0YhI\nvtAB1QK2eDH8+Mdw3XVK7CKSGUruEfvoIzj0ULj0Ujj99KijEZFioeQeoeXLYcgQOOccGDky6mhE\npJhkLLmbWQcze9XMHsvUOovZypUwdGgYEXPJJVFHIyLFJpM99wuBtzO4vqK1ejUcfTR8//tw1VVR\nRyMixSgjyd3MdgWOAO7IxPqK2bp14XqnFRVw663NX/NURCQdmeq53wT8BtBYxxZs3BgOmnbpAnfe\nCR10xENEsiTtC2Sb2ZHAp+4+z8wqgWb7omPGjNl0u7KyksrKynQ3H6m2TLXrDr/8JSxbBk8/DR11\naXIRaUJ1dTXV1dVpryftk5jM7CrgNGADUAp0Ax529zMatSuqk5gapg4oKQkX02hqut5EV10FU6fC\nCy9Ajx45DFREClpenKFqZgOBi9x9i1lRiim5N3XRjcbT9SaqqoI//hFeegl69sx5uCJSwHSGag5t\neV3TzafrTfT44/C738Ezzyixi0juZDS5u/s/muq1F5tkL0z98stw1lnw6KPQp0+OgxSRdk099xQk\nc2Hqt9+G448PV1E68MAIgxWRdkmzQqahudEyH34IBx8MY8dqvhgRSU9eHFBtcUNFmNybsmoV/OAH\ncNpp8NvfRh2NiBQ6Jfc8UFcXLovXuzeMH6+zT0UkfRotEzH3MLPjVltpWgERiZ7Ok8yQa6+FV14J\nJynp7FMRiZrSUAZMnRrKMDNmQLduUUcjIqLknrYXX4Tzz4dp06BXr6ijEREJVHNPw3vvwU9+Esay\n9+vXensRkVxRck/R8uVwxBHw+9/DYYdFHY2IyOY0FDIF69eHa58edFA4kCoiki0a554j7nDuuWFe\n9kce0QU3RCS7Uk3uOqDaRn/6E8yaFabvVWIXkXyl5N4GTz8N11yjIY8ikv+U3JO0cCGccQY8/HC4\nuLWISD5TYSEJn30GRx8N110XJgUTEcl3OqDairo6GDoU+veH66+POhoRaW80cViWXHghdOkSau0t\nicVizJkzh1gslpvARERaoOTegttvh3/8A+6/P8z22JzJk6dSXt6XIUN+QXl5XyZPnpq7IEVEmqCy\nTDNeeglOOCH8u+eezbeLxWKUl/dlzZrphAtmz6e0dBC1tQs3uzqTiEgqVJbJoI8+gpNPhrvvbjmx\nA9TU1FBSUkFI7AD96NSpnJqamqzGKCLSEiX3RtatgxNPDBfeOPzw1ttXVFSwfn0NMD/+yHzq6mqp\n0HhJEYmQknsjF1wAPXvCpZcm176srIyqqvGUlg6ie/f+lJYOoqpqvEoyIhKptGvuZrYrMAnYCagH\nJrr7n5pol/c19z//GW6+OUwv0NIZqLFYjJqaGioqKjYl8aYeExFJV2QTh5nZzsDO7j7PzLYB5gLH\nuvvCRu3yOrnPmAHHHhsuvtGnT/PtJk+eyogRIykpCeWYqqrxDBt2cu4CFZF2JW9mhTSzvwG3uvvz\njR7P2+T+8ccwYEAY+njUUc2308gYEcm1vBgtY2YVwH7ArEyuN5vWrw9XU/r5z1tO7KCRMSJSODI2\ncVi8JPNX4EJ3/7KpNmPGjNl0u7KyksrKykxtPmW/+hWUlcHll7fedvORMaHnrpExIpJJ1dXVVFdX\np72ejJRlzKwj8ATwd3e/pZk2eVeWuf9+uPJKmDMHevRI7jUNNfdOncqpq6tVzV1EsirSmruZTQKW\nu/uvW2iTV8n97bdh4EB4/vm2X9xaI2NEJFeiHC3zH8ALwBuAx5fL3P3pRu3yJrl/+WU4gDpqFAwf\nHnU0IiLNy5vRMs1uKE+Suzuceip07Qp33BF1NCIiLdM1VJM0fjwsWBDGtYuIFKt21XOfPTsMd3z5\n5dYnBBMRyQd5Mc49n61YASedBBMmKLGLSPFrFz33+vrQY997b7jhhkhCEBFJiXruLbj6ali1Kvwr\nItIeFP2BOA5HAAAHvUlEQVQB1enTYdw4eOUV6NQp6mhERHKjqHvusRicfnq4olKvXlFHIyKSO0Vb\nc2+os/frB9dck7PNiohklGrujdx0E3z+OfzhD1FHIiKSe0XZc28Yzz57NmjCRhEpZOq5x61cCaec\nAv/zP0rsItJ+FVXP3R1OPhl22CFMMyAiUug0twwwcSK88w5MmhR1JCIi0Sqanvsbb8DgwfDPf0Lf\nvlnbjIhITrXrmvtXX4VyzPXXK7GLiECR9NwvuCAMe5w0CazN+zcRkfzVri/WUVMD228P3bplZfUi\nIpFp18ldRKRYteuau4iIbE7JXUSkCCm5i4gUISV3EZEipOQuIlKEMpLczWyomS00s3fNbFQm1iki\nIqlLeyikmXUA3gUOAT4C5gCnuPvCRu00FFJEpI2iHAp5IPCeu9e6ex0wBTg2A+sVEZEUZSK59wI+\nSLj/YfyxohGLxZgzZw6xWCzqUEREkpLTKX/HjBmz6XZlZSWVlZW53HxKJk+eyogRIykpqWD9+hqq\nqsYzbNjJUYclIkWqurqa6urqtNeTiZr7QcAYdx8av38J4O5+baN2BVdzj8VilJf3Zc2a6UA/YD6l\npYOorV1IWVlZ1OGJSDsQZc19DrCnmZWbWQlwCvBYBtYbuZqaGkpKKgiJHaAfnTqVU1NTE11QIiJJ\nSDu5u/tG4DzgWeAtYIq7L0h3vfmgoiKUYmB+/JH51NXVUqGLs4pInstIzd3dnwa+nYl15ZOysjKq\nqsYzYsQgOnUqp66ulqqq8SrJiEje05S/SYjFYtTU1FBRUaHELiI5pfncRUSKkOZzFxGRTZTcRUSK\nkJK7iEgRUnIXESlCSu4iIkVIyV1EpAgpuYuIFCEldxGRIqTkLiJShJTcRUSKkJK7iEgRUnIXESlC\nSu4iIkVIyV1EpAgpuYuIFCEldxGRIqTkLiJShJTcRUSKkJK7iEgRUnIXESlCaSV3M7vOzBaY2Twz\ne8jMumcqMBERSV26PfdngX3cfT/gPeDS9EPKT9XV1VGHkJZCjr+QYwfFH7VCjz9VaSV3d5/m7vXx\nuzOBXdMPKT8V+h9IIcdfyLGD4o9aocefqkzW3M8C/p7B9YmISIo6ttbAzJ4Ddkp8CHDgcnd/PN7m\ncqDO3R/ISpQiItIm5u7prcBsOHAOMNjd17XQLr0NiYi0U+5ubX1Nqz33lpjZUOA3wI9aSuyQWnAi\nIpKatHruZvYeUAKsiD80091HZiIwERFJXdplGRERyT9ZO0PVzLY1s2fN7B0ze8bMejTRZlcz+18z\ne8vM3jCzC7IVTzLMbKiZLTSzd81sVDNt/mRm78VP3Nov1zG2pLX4zexUM3s9vrxoZvtGEWdzkvn8\n4+0GmFmdmZ2Qy/hak+TfT6WZvWZmb5rZ9FzH2JIk/n66m9lj8b/9N+LH2/KCmVWZ2admNr+FNvn8\n3W0x/pS+u+6elQW4Fvht/PYo4Jom2uwM7Be/vQ3wDtA3WzG1Em8HYBFQDnQC5jWOBTgceDJ++/uE\nMlTOY00j/oOAHvHbQwst/oR2zwNPACdEHXcbP/8ewFtAr/j9HaKOu43xXwpc3RA7oRzbMerY4/H8\nANgPmN/M83n73U0y/jZ/d7M5t8yxwD3x2/cAxzVu4O6fuPu8+O0vgQVAryzG1JIDgffcvdbd64Ap\nhPeQ6FhgEoC7zwJ6mNlO5IdW43f3me6+Mn53JtF91k1J5vMHOB/4K7Asl8ElIZn4TwUecvelAO6+\nPMcxtiSZ+B3oFr/dDVjh7htyGGOz3P1F4PMWmuTzd7fV+FP57mYzue/o7p/GA/sE2LGlxmZWQdhz\nzcpiTC3pBXyQcP9DtvwAG7dZ2kSbqCQTf6Kzya+TzlqN38x6Ase5+/8QzrfIJ8l8/n2A7cxsupnN\nMbPTcxZd65KJfxywt5l9BLwOXJij2DIhn7+7bZXUdzfdoZDNneD0uyaaN3vk1sy2IfTGLoz34CWL\nzGwQcCbhp2AhuZlQ4muQbwm+NR2B/sBgoCsww8xmuPuiaMNK2mHAa+4+2Mz2AJ4zs376zuZOW767\naSV3dx/SQhCfmtlO7v6pme1MMz+jzawjIbHf6+6PphNPmpYCuyXc3zX+WOM232ylTVSSiR8z6wf8\nGRjq7i39jM21ZOL/HjDFzIxQ8z3czOrc/bEcxdiSZOL/EFju7muBtWb2AvBdQq07asnEfyZwNYC7\nv29mi4G+wCs5iTA9+fzdTUpbv7vZLMs8BgyP3/4Z0FzivhN4291vyWIsyZgD7Glm5WZWApxCeA+J\nHgPOADCzg4B/N5Se8kCr8ZvZbsBDwOnu/n4EMbak1fjdvXd82Z3QIRiZJ4kdkvv7eRT4gZltZWZd\nCAf2FuQ4zuYkE38tcChAvF7dB/hXTqNsmdH8r7l8/u42aDb+lL67WTz6ux0wjTAC5lngG/HHdwGe\niN/+D2Aj4cj8a8CrhL1SVEesh8bjfQ+4JP7Yz4FzE9qMI/S0Xgf6RxVrKvEDEwkjHF6Nf96zo465\nrZ9/Qts7yaPRMm34+7mYMGJmPnB+1DG38e9nF+CZeOzzgWFRx5wQ+wPAR8A6YAnhV0YhfXdbjD+V\n765OYhIRKUK6zJ6ISBFSchcRKUJK7iIiRUjJXUSkCCm5i4gUISV3EZEipOQuIlKElNxFRIrQ/weT\n5RokWCGCNQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f85b71bd2b0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model = LinearRegression()\n",
    "model.fit(X_poly, y)\n",
    "y_test = model.predict(X_test_poly)\n",
    "\n",
    "plt.scatter(X.ravel(), y)\n",
    "plt.plot(X_test.ravel(), y_test)\n",
    "plt.title(\"mean squared error: {0:.3g}\".format(mean_squared_error(model.predict(X_poly), y)));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This reduces the mean squared error, and makes a much better fit.  What happens if we use an even higher-degree polynomial?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEKCAYAAADpfBXhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8VOXVwPHfSUggsqhIlE0TxQU3VCpKtUpAELTutcXY\notjUtlq3ulSrtmK1r9W61NdKX1TEFcSlVrSoSCUqagVFQEAUwQQFgaDIZoBAzvvHM0Mmycy9k5k7\nWzjfzyefzNy5c+fMJHPmmfM893lEVTHGGNO65GU6AGOMMcGz5G6MMa2QJXdjjGmFLLkbY0wrZMnd\nGGNaIUvuxhjTCllyN6YJEakXkX0yHYcxybDkbkxzOXfyh4icICIfi8gGEfmPiOzlse/jIvKViHwr\nIgtFpCLitnNFZL2IrAv9bAx92B0RsU9fEXkjtN9XInJpqp+faTlL7qZVE5FE/scloMfOj2dbS48R\nZZ/dgOeAG4DOwAfARI+73Absraq7AKcBt4aTt6qOV9WOqtpJVTsBFwOLVfXDiMd6GfgHsCuwLzCl\nJc/JpIcl91ZARD4XkatFZE6oNfWgiOwuIpNDra8pIrJzxP79ReRtEVkjIh+KyICI20aKyILQ/T4T\nkV9G3DZARL4QkStFZKWILBORkR5xjRSRxaFjLRaR8tD2PBG5U0RqQo9xcah1mBfxfAZFHOcmEXk8\n4vrToRbjGhGpFJGDIm4bJyKjReTfIrIeKBORwtDjVYfuN1pE2kbc5xoRWS4iX4rIBXi03EWkk4g8\nFNr/CxG5RUQkdNv5IjJdRO4WkdXATTG2iYjcKCJVIrJCRB4RkU6hY5SEXoufi0g18B/vvz4AZwHz\nVPWfqroFGAUcJiL7R9tZVReo6qbwUwo9314xjn0+8FjE9SuBV1T1KVXdqqobVfWTOGI0aWbJvfU4\nCzgB2B/XGpsMXAd0AfKBywBEpAfwEvAnVd0VuBp4LtQiA1gJnBxqtV0A3CMih0c8TlegI9Ad+AVw\nf+QHR5iI7ATcCwwNHesYYHbo5l8CJwOHAUcCZ+NfCom8fTIuGe0OzAKebLJvOXCLqnYE3gZux7Uw\n+4R+9wD+GIpzGC5hnQDsBwz2ieNRYAuwD3AEMAT3OoQdDXwWiu3PMbZdAJwHDAgdpyPw9yaPczzQ\nGxgainOOiJwTI6aDgTnhK6r6XejxDo71JETkfhHZCHwMLMe9pk33KQGOo3Fy7w+sCTUOVorICyKy\nZ6zHMRmkqvaT4z/A50B5xPVngfsjrl8C/DN0+XfAo03u/wowIsaxnwcuDV0eAGwE8iJuXwkcFeV+\nOwHfAGcC7Zrc9h/glxHXhwDbwscNPZ9BEbffBDwWI75dgHqgY+j6OOCRJvtswJUhwte/DywJXR4L\n/E/EbfuFYtknymPtDmwC2kZsOwd4PXT5fKCqyX2ibZsK/Dri+v64D4w8oCT0+CUt+Ps/FPkcQtum\nA+f53E9wH7rXA/lRbv9D+LlFbPsk9HftCxTiPsCnZ/o9YD/Nf6zl3nqsjLhcG+V6h9DlEuAnIvJN\n6GcNcCzQDUBEThKRd0Xk69BtJ+Fa/2Ffq2p9xPXvIo69nbrW43DgIuArEXkxokzQHfgiYvfqeJ9k\nqKTzl1A551vcB4E2ifGLiP2LcR80H4SfM65mHP6mEi2WWDX3EqAg9HzCr93/xXpsj23dafycq4E2\nwB4R276MEUM0G4BOTbbtDKz3upM67wB74v5OTY0AHmmyrRZ4XlVnqSsB3QwcIyIdWxCvSQNL7jue\nL3Ct4M6hn13VdaDdISKFuFb/HUCxurLNyyTYwaiqr6nqibhSzifAg6GbvsIllLCSJnfdiEvIYV0j\nLv8UOBXXst8FKA3FFxljZAlnNe4D6OCI57yLqoZLSdFiiVUi+gLXct8t4rXbRVX7xHjsWNuW0/g5\nlwB1NP5AbsmInfnA9tKZiLTHla3mx3n/NjSpuYtI+AP/uSb7zo0SW86NLtoRWHLf8TwBnCoiJ4Za\nwe1CHaXdcV+zC4HVqlovIicBJybyIOI6dE8L1d7rcK3LcIv/aeAyEekhIrsC1za5+2zgHBFpIyLh\nmnxYB2Azru7bHjfyI2ZyUVXFfaj8LdSKJ/S44ef1NDBSRA4MxfpHj2OtwI0MuUdEOoY6RvcRkePj\neEkiTQB+KyKlItIBV4d/KuIbUUs/TJ8HDhaRM0MdxTcBs1X106Y7ikixiAwXkfahv/9QXGlpapNd\nzweeU9WNTbaPA84UkT4iUoAr3UxXVc9vCSb9LLm3DnG3pFT1S+B0XJ21BlcSuBpX796A63h9JlS+\nOAd4oYWPHZaH66hchms9H0/DV/8HgVdxnYDv07x1+Adcx+c3uEQV2WH6GLA0dNx5wDs+8YH78PgM\n+G+olDMFV+dGVV8B/ga8DnyK/+iU83AfgAtC8T1D428W8XgYeBx4E1iM+2ZxWcTtzV5TEZknodFG\nTanqauBHwP+EYjoS97cL3/f3IvLviGNfhPsW8g3uW9rlqvrviP3b4j5QH4nyWNNw/zuTgRW4DuFz\n43jOJs3ENWzi2FFkLHAKsLLJ11BE5Crgr0AXVf0m8ChNqxYalbEEKGhSzzfGJKglLfdxhIZlRRKR\nnrjRDnF3ihkTRSAnDhljnLiTu6pOB9ZEueke4JrAIjI7KuuUMyZAbZK5s4icBnyhqh+FTtIzpsVU\ntRp3opUxJiAJJ3cRKcJ1rAyJ3Jx0RMYYY5KWTMu9F26M8ZzQ3Bo9cSeKHKWqq5ruLCL2tdsYYxKg\nqi1uOLd0KOT2k0VUdZ6qdlXVfVR1b9wZdUdES+wRAebUz6RJyg9/6C7fdNNNGY8nmZ9cjj+XY7f4\nvX/Ky5Unn2z5/YYNUyZPVj76SDn44Nb9+icq7uQuIuNxY4r3F5GlodnzGuVuWllZRgSSeG2NMT62\nboU2CdYP7L3pLe6XVVU9T1RQ1Va3co0ld2NSK9HkbuM3/NkZqh4ik3tZWVlGY0lWLsefy7GDxe8l\nmZZ7vHL99U+UJXcPltyzQy7HDha/l3SUZXL99U+UJXcPVpYxJrWsLJM6ltw95OVZcjcmlZJtudv7\nMzZL7h5EoN6msTImZYJouVsrPjpL7h6sLGNMatXVpb5DdUdlyd2DJXdjUmvrVigoSOy+9t70Zsnd\ngyV3Y1LLOlRTx5K7B0vuxqRWOsa576gsuXuw5G5Matn0A6ljyd2DDYU0JrWsLJM6ltw92FBIY1LL\nxrmnjiV3D1aWMSa1bJx76lhy92DJ3ZjUsg7V1LHk7sGSuzGplcxJTPbe9GbJ3YMld2NSyzpUU6cl\nKzGNFZGVIjI3YtsdIvKxiMwWkedEpFNqwswMS+7GpFYyZ6gaby1puY8DhjbZNgU4WFUPBxYBvw8q\nsGyQl2ejZYxJpa1bIT8/sftaw8tb3MldVacDa5psm6qq4fT3X6BngLFlnLXcjUmtujooLGz5/aws\n4y/ImvvPgZcDPF7G2UlMxqTOtm3udzItd3t/xhbIICQRuQGoU9XxXvuNGjVq++WysrKsX/7KyjLG\npM6WLYm12qF1j3OvrKyksrIy6eMkndxFZCRwMjDIb9/I5J4LLLkbkzrJJPfWrGnD9+abb07oOC0t\ny0jox10RGQZcA5ymqpsTiiCLWXI3puVU4f33YdMm7/2STe5WkvHWkqGQ44F3gP1FZKmIXADcB3QA\nXhORWSIyOkVxZoTNLWNMy911FwwcCCNHeu8XVFnGRBd3WUZVz42yeVyAsWQd61BtPdavh1degT32\ngOOOs+SQKps2wW23wTvvwIABsGwZ9OgRfV8ry6SWnaHqwcoyrcO8eXDQQTBuHPzqV/CjH8HmVldE\nzA6TJ8Nhh8Ghh8Ipp8ALL8Te18oyqWXJ3YMl99y3bp1L5rfc4hLPnDkuKVxxRaYja51efNG93gDH\nHw/vvht73yDKMpbgY7Pk7sFq7rnvhhtcGSZc/y0shEcfhUmTXOnABEcVXnsNhgxx1/v3h/fei71/\nUC13K7FFZ5NterCae25buhTGj4eFCxtv79QJ7rwTLrsMZs605BCUqir3ftlvP3f9wAPhyy9hwwbo\n0KH5/tahmlrWcvdgZZncdscdcOGFUFzc/Lbhw6G2Fl5/Pf1xtVYzZ0K/fg2JNz/fJfpPP42+v3Wo\nppYldw+W3HPXxo2u1X7ppdFvz8uDK690LXgTjPffd8k90gEHNP/mFGYdqqllyd2D1dxz17PPwjHH\nxB6GB/DTn8IHH8CSJemLqzWbOROOPLLxtt69U5PcrSzjz5K7B6u5566HH4aKCu992rWDc86Bxx5L\nT0xBqqmpYebMmdTU1GQ6FMA1gmbNgu99r/H2Xr1if3haWSa1LLl7sLJMblqxwg15PPlk/33PP98l\n91z6O0+YMJGSkt4MGfJrSkp6M2HCxEyHxKJFsNtu0KVL4+2lpVBdHf0+VpZJLUvuHiy556YXXnCJ\nvW1b/3379nUteK8he9mkpqaGioqLqa2dxtq1H1BbO42Kiosz3oKPVpIBl9yrqqLfZ/NmG+eeSpbc\nPVjNPTc991zDiTR+ROCss+D551MbU1CqqqooLCwF+oS29KGgoISqWBk0Td5/P3py79EDVq50rfSm\nbJx7ally92A199yzfr07K3LYsPjvc+aZLrnnwt+6tLSULVuqgPBSxnOpq6umtLQ0c0HRMAyyqTZt\noFs3N969KetQTS1L7h6sLJN7pk2Do4+G9u3jv0/fvi7RzJ+furiCUlxczNixoykqGkinTn0pKhrI\n2LGjKY42mD9Ntm51fRx9+0a/vXt3WL68+XbrUE0tO0PVgyX33DNlCgxtuoy7DxE44wzXej/kkNTE\nFaTy8uEMHjyIqqoqSktLM5rYAT7+2JVfdt45+u3dusFXXzXfbh2qqWUtdw+W3HPPq6/CiSe2/H5n\nngn/+lfw8aRKcXEx/fr1y3hih9glmbDu3YNP7laW8WfJ3YN1qOaWJUtczf3QQ1t+32OPhcWL3TBK\nE7+amhomT15F794bYu6Tqpa78daSlZjGishKEZkbsW1XEZkiIp+IyKsiEuOLWW6yDtXcMmWKa7Xn\nJdBkKSiAE05wxzDxCY+3/9e/arjllh/HHG9vZZnMaMnbYBzQtJp5HTBVVQ8AXgd+H1Rg2cDKMrkl\ncrrZRAwb5so6xl/DePtKtm07mC1b7og53j5Wcrdx7qkVd3JX1enAmiabTwceDV1+FDgjoLiygiX3\n3KEKb74JEYvGt9jQoa7lbn9zfw3j7cM1sENjjrfv1i36aJnaWigqSjwGG+fuLdma++6quhJAVVcA\nuycfUvawmnvuWLjQzRm+556JH2Ovvdz0wLNmBRdXa9Uw3j6ctWOPt4/Vck8muW/evIklSxazZk3T\n9qYJC3oopOeXpFGjRm2/XFZWRlkyzaw0sJp77njzTbesW7KGDnULaUc729I0CI+3HzHivxQUfIjI\n6Jjj7YuL3XKHTWvsiSb3CRMm8sILebz88hzq69+mW7d/Arsm/mSyTGVlJZWVlUkfR7QF2UtESoAX\nVbVP6PrHQJmqrhSRrsA0VT0wxn21JY+VDbZtc/+M27ZlOhLj52c/g4ED/WeC9PPqq2691enTg4mr\ntdtrr23cffd8Bgzo5jkss2dPt6zhXns1bDvtNPf3Ov30+B+vpqaGkpLe1NZ+hkvoixD5jpUru2fF\nsNBUEBFUtcXFp5aWZST0EzYJGBm6fD7gsdZ57rGae25QhTfecGulJuv4493Zlt9+m/yxWrtly2Dj\nxnx+9KM+vok1Wmlm7dotLF36SYsmPWuo9Ydb6vshUpjxuXWyUUuGQo4H3gH2F5GlInIB8BdgiIh8\nApwQut5qWI98bqiuhrq6hrU7k1FU5Bb5sOX3/E2dCoMGxdeh2TS5T5gwkenTZ/H739/RommLG2r9\n4Vr7IlS3ZHxunWzUktEy56pqd1Vtq6p7qeo4VV2jqoNV9QBVPVFVW117R8SSe7YL19uDGjUxbJir\nu2eLbFuYI2zqVBg8OL5999jDzQ4JDcMo6+v7sHHj2BZNWxyu9efnT6Vduz/Qtu2vKCnZq9WWZJJh\nZ6j6sNJM9guqMzVs6FBXe1fNfGLNxoU5wL02U6fGf15B164NZ/82lFZ2Ct3asmmLy8uHc+aZp3Ld\ndRfw0kvP0blz6+lMDZIldx+W3LPfW28Fm9wPPND9ze+6a3JGE2u2LswB8NFHroS1zz7x7d+1a0PL\nvaG0Ep7kveXTFrdr1469996HXXe1xB6LJXcfltyz24oVsGpVsLM5isCAAbVcf/1bGU2s2bowB8CE\nCXD22fHvv8ceUF29mZkzZwIwduxoYA0dOpyY0LTFduKSP0vuPuxEpuz21lvwgx8kNp+Ml4MPXobq\nD8lkYs3WhTm2bYMnnoDzzov/Pu+9N4mXX/6QE06ooKSkNwAdOxbzwgt/obp6IeXlw1MU7Y7LkrsP\nO5EpuwVdbw87++xd2Lr1MOCj0Jb0J9bi4mJGjx5DQcETtGkzjby8dfTvP4NPPy3OaF/A66+7E5Pi\n/bY0ZsyD3HnnDaj2YP36ZdTWXktFxcVs2iQce2zfhDtD7X3pzRbr8GFlmexTU1OzfaGKt94qZsyY\n4B9jv/26sO++q1m69Cbatauirq467SsevfEG/PGPZzN48Kkcd9wX9O7dhSVLduHMMzfwzTdv0L79\nvdTVLWDs2NEpa/lGvtbFxcWowm23wW9+E//9L7/8d8BbwJ7ANGAgeXk92brV5nNPJUvuPiy5Z5cJ\nEyZSUXExhYWlbN68BviEvn0LUvJYP/1pF2pqnmDkyPlpX/Hoqafg8svhkUfgpJPaAvsCLlneeOMR\nbNs2h3XrzgYWUFFxHIMHDwokPlX49FP45huYM+dfXHllBYWFrjw0duxoNm0aTk1N/CUZ12+wN5s3\nh5v5fYCe1NXV0K6dJelUsuTuw2ru2SNy9EhtbR/gc/Ly3ubbbw9OSeIdOhR+9auduP9+j2WGUmDy\nZPjtb91Qw6YLj1RVVdG27R5s2rRbaMtBtGlzCFVVVUm/BpMnw9VXuwVPunSpY/bsAcAyamvbAUs5\n77zn2XXXel5/PY+COD9PS0tL2bq1Gtdv0AdYCCziz38ew223JZfZVa0048Vq7j6s5p49mo8e2ZuC\ngvkp6+Ts1w++/DL6dLWpsmABjBwJ//xn9BWlmney1rBx4x107Vqa8GOqwo03ulLLPffA0qXwwAOz\n6dRpCNAutNde5OV1ZuzYj1o0MilyQe+8vDkUFFzB//3fvfzkJyOSmu43ssVvrf/oLLn7sLJM9mhI\nbJXATGA9qpNT1snZpo07AzNdqzNt3OjWcv3rX+H734++z9Spr7N16xbg+8C+FBTsz1FHdeO664oT\nboRcdx28/DLMmOG+rYi417qu7nMiR+rk519B//7dW3z88vLhVFcvpH//rlx77V8466wzWL8eOnZM\nLN4wa3R5s+Tuw5J79iguLqaiYgRwMvBrQKioOCCltfDw2arpcM010L8/nH9+9NvDZam6ureBKuBP\n5OfDxIlFfPYZ3Hqr/2M0HWUzerRbGHzKFDcCJiyyxd2pU9+ExqJHmjr1dd57bzJ33vkKJSW9efbZ\nqeycxKKc1lqPg6qm5cc9VO7ZfXfVr77KdBS5Y9WqVTpjxgxdtWpVSo5dVNRZYU6o2rpei4o6p+Sx\nwr74QnW33VS3bk3ZQ6iq6ssvq+61l+q338beZ8aMGbrzzn21odqs2qnTETpjxgz96it3/4kTY99/\n/PintKios+68c18tKuqsV131pnbrprp4cez7BPH3bPi7rQjFPUcLC8/WsrLNCR9z5EjVhx9WnTlT\n9XvfS/gwOSGUO1ucc63l7sNq7vFL9TwozWvuHVJ+YlHPnu7U+dCJlSnx3Xdw0UXw0EN4tma9Tmrq\n2hUmTXJ18xkzmt+3+VQG/+Wuuw5k3Lg1nlMIFBcX069fv6S+HTX83fYIbelDfn4v8vM3JHxMcO/L\nNWvWsHHjxqyYkiHbWHL3ketlmXSd7JKOeVCaJ7fFaTmx6PTTXekiVf7nf+Doo/0n4fIrlRx2GIwd\n6+r2S5c2vm/zD8b9KCr6E507fxb002mm4e9WFdoyl61bN9OtW+I9qiLw3nszOPXUs1m0qDqrJlXL\nGok09xP5IUfLMj17qi5dmukoEtP0a/j48U+l7LG8SgZBGj/+KW3Xbk+FjdquXfeUPqew999X3W8/\n1fr64I+9cKEr+yxbFv99/Eol996rWlKiOn9+4/u40sinob/P0pSXtCKNH/+UFhaeqHl5c7WoqLOW\nl3+oV12V+PHKy7/TgoJLIp7PnLQ+n3QiwbKMJXcfe+6pWlWV6Sharnl9OrX//Ol8vIkT1+jhh69L\n2xu5vt7Vs+fNC/64gwer3n13sMdVVX30UfehceONqrNmqX7wgeoppyxQqNGioqtS/mEfzcyZq7Vr\n1026atUqveEG1T/9KfFjnXrqKi0qujnljYlskGhyD6QsIyK/FZF5IjJXRJ4UkQRPKs4+uVpzT/eM\ngkGPrvDywQe7cNppHdN2xqiIK3X885/BHvfpp92MlpdeGuxxa2pqOPDAmbzyytesWePWlx0xAnr1\nOpA5c4Q33hiekcm6DjpoN9asaUuXLsWsXevdv+CnY8cObN26GlgU2pIdk6pllUQ+ESJ/gO7AEqAw\ndH0icF6U/VL8+ZYae+/tPZogW6W75R75uKkaLRPWp4/qu++m7PBRVVaqHnZYcMdbu1a1Rw/V6dNb\nfl+v1zidpbhEdO6sunKl6ogRquPGJX6cn/9c9cILZ2jbtidoXt68rHyuQSFTZZlQcq/GrVjbBngR\nGBxlv1S/BinRq5fqokWZjiIx4Td6p05HtJp//mXLXIJI9dDEprZudcn4o4+COd5vf6t6wQUtv59X\n8s7UB3pL9O2r+t57qqedpvr884kfp6JC9cEHVV999Rs98MANWfUcg5Zock+6LKOqy4G7gKXAMuBb\nVZ2a7HGzRS6PlgmfGTh16phWM2f2q6/C8cdvYtas9E53m5/vJst65JHkjzVnDjz5JNx+e8vu5zci\nKZsX9wjbe2/4/HM3rUOPHskfb9ddd6V9+/a2hmoUSU8cJiK7AKcDJcBa4FkROVdVxzfdd9SoUdsv\nl5WVUVZWluzDp1yuTxxWXFyc1n/88BSxu+9eyt13F/PMM245tl/+Eq68krgnnIplzJilzJp1B9Om\nvbt9psJ0fWiNHOnmjr/ttsSfR329G9N+662NzwiNRzh5u0nTIDJ5FxcXNxkq2odsrEPvvTdUVbmf\nZMPKxb6weFRWVlJZWZn8gRJp7mvjcsvZwIMR10cAf4+yXwq/uKRO796qCxZkOorc0FAGOk5FPtRj\njqnSxYtVZ89WHTJEdcAA1Q0bEj/+8uWrFL5RmJ+xssMxx6hOmpT4/R96SPXoo1W3bWv5feMpu2R7\nKe4f/1A9+2zVnXZKbmjpL36h+sADqjNmqB55ZHDxZSMyOFpmKdBfRNqJiAAnAB8HcNyskMtlmXSK\nLBmsW/cmqnsya1ZfOnas4bDD4JVXXKvtrLPcMm2JmDTpa/LyVgIHhbakr+wQPhns7LPX88ADiZ0c\ntno1XH89/OMfiS0LGM+IpGwvxfXvD88+CyUlyc8PEx4EaaJLuiyjqjNE5FngQ6Au9PuBZI+bLSy5\nx6d5yWA3CgsbSgZ5ee70+hNPhJtuim+Sq6ZmziwhL+9u6utPJZ1lh8YLhKxAZCF77nka7dptaVFp\n6PLL4dxz4YgjEo+lvHw4gwcParQ6UlPpLsW1RHga42QXNLcpf+OQSHM/kR9ytCxz6KGurGC8rVq1\nStu1666w2bNksnKlateuqu+807Ljb9um2q2b6p13/luLijpr+/Z90lJ2aF4KmaZwi8LXLSoNPfOM\n6v77q27cmNJwc8I776h+911yx7jwQtUxY9zIm379gokrW2ETh6VGrp7ElG7FxcX8+Mcvkpc3JWbJ\noKamhurqmdxyyzoqKmDz5viPP306dOkC3buvR7Ue2Bz6nVrNR6C0ByYBnUPX/UtDy5bBJZfAo4/C\nTjulMtrc8P3vk9RCHWH2vvRmy+z5sLJMfOrq4I03+vLyy3uz665jmpUMGpc2qjjkkDnccktP3/JM\nePTN/fcfyo9+tJWKiovZtOkNwmWZioqBga0fGk3zESgbgXnActwpHt6loU2b3Nmtl13m6s0mGFaK\n8Wctdx+W3OPzzDOwzz5w4om7Npsitun47E2bpjFv3omMGVPP7NmxjxmeQnjw4Gt4+unNbNr0YNrH\ncTfvxPwRl1xyIe3afY+8vEUUFDwYc5qF+no3BLS0FH7/+5SFaExUltx95Po493R5+GE3fjuaaCfX\nFBa24+KLqxg50rX6m2o8+qYSUP72tz/FnM88lZqOQLnvvntZunQuTz65iU6d7qWkpHlnal0dXHgh\nLFkC48ZZSzMVrCzjzZK7D6u5+/vyS5g1C049NfrtsRaZ+M1vOtKzp5vPvKnmHwi7UFi4N9dff1Va\nJidrqumiFcXFxZxzzqE8+WQeZ5zhRgJt3er2fe89GDAAVqxwQ0Dbt095eDsc+7D0Z8ndh5Vl/I0f\n78avx+okizU+e/fdixkzxo37fvPNxvdp+ED4KrTFfSD86lcXZtU47vAaq088AbvsAp07wznnuKkK\nXnwROnTIaHitmo1z92Ydqj4suXtThccfh7//3Xu/WOOze/SAxx5zCXHaNDjgALd/cXExN9wwgRtv\nLKBDh1PZtu2dRq30bBrHfcQRUFkJ69a5EUBduljLMtVsnLs/S+4+LLl7mzPHJbXjjvPfN9bJNSee\nCH/+MwwcCGPGwCmnwPvvw5gxJ/LAA+s4/PA/xjxhJ5t06pTpCIxpYMndh3WoenviCbcYRCKn00e6\n4AK3GPWVV0J5uVvI4e67YfjwTkC/QGI1rYuVZLxZcvdhHaqxbdvm6u2vvx7M8YYMgblz3TeBjh2T\n/8AwrZeVYvzZ28eHlWVi+89/XM18t91aPolWLCKu1W6J3Zjk2FvIhyX32B5/HA46aBYlJb0ZMuTX\nlJT0ZsKEiZkOy+wg7Bu1N0vuPnK55r5ggatf9+8PN9zgyh1B2bABJk2q5+mny2OuDGRMqlhZxp8l\ndx+5WnOvJ6A9AAAWYUlEQVSvrISyMujXD+64A5YuddeDyrvPPw+HHrqOtm07kM3LupnWy8a5e7Pk\n7iMXyzJLl8Lw4fDUU270yfHHu7HkQ4bAGWdEP92/pR5/HM4/Py8j0wEYY+Pc/QWS3EVkZxF5RkQ+\nFpH5InJ0EMfNBrmW3FXh/PNdUh80qGG7iFv7s2NHuPnm5B5j+XKYORN+9rNOvisDGWMyI6ihkPcC\nk1X1xyLSBmg1s1bnWs392WdhzRq4+urmt+XlwSOPuNVwfvYz6N07sccYP95NY1tUFN/KQMakgpVk\nvCWd3EWkE3Ccqo4EUNWtQIBdd5mVSzX3ujq49lo3iVV+fvR9unZ1089ecQW8/HLLv9KqulkO77+/\nYVs2L+tmWicrxfgLoiyzN7BaRMaJyCwReUBEAlhnJTvkUlnmqafcwsOR5ZhoLrkEqqvhpZda/hgz\nZ7r5UwYMSCxGY0x6BJHc2wB9gftVtS/wHXBdAMfNCrmS3Ovr4fbbXau8psb7pKLCQrjrLvjd7xqm\nqY3X2LFuqgBrOZlMy5Vv1JkSRM39S+ALVX0/dP1Z4NpoO44aNWr75bKyMsrKygJ4+NTKlZr7v//t\nkvbq1RMpKXHL2W3ZUsXYsaOjTot70kkuwT/0EPz61/E9xnffuRWX5s7139eYVAo3Llpjgq+srKSy\nsjLp44gG8OqIyBvAhar6qYjcBOykqtc22UeDeKx0+/GP4Sc/cb+zQXhN0cjOS1U49lioqFjLpZfu\nQ23tNMJrjBYVDaS6emHUmvisWfDDH8Knn7pRNH7GjXPJffLkYJ+TMS112WXQqxccdZQbGfbuu5mO\nKHVEBFVt8XfloMa5XwY8KSKzgcOAKGvr5KZsKsuE1xRteqr/W2+5k5MOPvjTmGuMRivV9O0LgwfD\nX//q/9iqcM89riPWmEyzsqC/QJK7qs5R1X6qeriqnqWqa4M4bjZIV3IfP96dQXrFFdGnCWi6yHTk\nqf633ebq5716RV/Obtas2THnf7n1VjfyZfly7/hee839HjIkgCdrjEk5O0PVRzqS+7/+5TpCr74a\n1q51i1d8913jfaItMl1QUMLkyauYN88t6xZtObt77vkLv/3tdTHnfykpgYoK+MMfYsdXXw833uhi\ntBaTyRY5WOVNK0vuPlLZoVpTU8M777zPJZds4/HH3QpEDz8M++/vknXkP2+sRaafempfrr0W2rZ1\nW8vLhzdaY7Rv38NjlmrCrr8epkyBqVOjxzl+vPs9PLPLlRqznTUy/Fly95Gqlnu4fj548L/46qu3\nWbbMlUpE4MEHoaoK7ruvYf9orfI//GE8c+a0paKi8bGLi4vp168fxcXFMT8UIud/2WUX96FywQWw\nbFnjY1VXw1VXwf/+r82xbkwusberj/x8t+JQkCLr57W1t1JfX9KoVNK2LUyc6Orh77/fcL/IVnlV\n1UIqK4dy7bWwYUPsce3RPhSizf8yZAhceqnrYA036pcvh9NPd/X8/v2DfQ2MSZaVZbzZMns+UpHc\nw/Xz2tpwqaRke6kknHR79YLRo90wzFmzoK6uYQhkv379ePZZ18ru3PlpSkou8hzXHu/8L7/7nRsr\n37cvHHAALFwI11zjhpoZk01a8zj3oFhy95GK5N5QKqkCSok1Ve7ZZ7t52YcO/YK5cw+nbVt3vz//\n+TFuv/2HPPTQGn7yk4tC3wDcuPaKioEMHjyoWQKPd/6XK65w9f75812C3333IJ6xMcELJ3arv0dn\nZRkfqUjuANdffxX5+f+hbdu7PafKvfbaGt5/fzmbNi0JjXaZwVVXHcRVV61njz0+8+0sTUTnznDc\ncZbYTfayhO7PkruPoJN7uCP1zjufo75+AD//+c5UVy+MOkUAwIoVVXTocDmwc2hLL9q1e4CysoVx\ndZYaY3ZMltx9BJncm56IpFrKww9fw4cffhhzkq/S0lLq6hYRmcDhge21c1ssw+yorN7uzZK7jyCT\ne/MTkVayefNmzjrrd83OHA3zS+BNx7XH+gZgTGtiZRl/1qHqIz8/uHHujcso+wKPAe+ycaN3Z6jf\naBdbLMMY05S13H3k5QXXco9sheflfQgsJt7O0MgTk4wxVpbxY8ndR9AdquXlw6mqWshOOx1Fu3bT\nsc5QY1rOxrn7s+TuIxVDITdsKGaXXQp4+OGbrTPUmATZOHdvVnP3kYrkPmsWfO978Z85aoxpzBK6\nP0vuPvLzYcuWYI85bx4ceqi7bJ2hxphUCKwsIyJ5IjJLRCYFdcxskIqW+4IFcNBBwR7TmB2N1du9\nBVlzvxxYEODxsoIld2Oyj5Vl/AWS3EWkJ3Ay8FAQx8smQSf3ujpYvNgtyGGMMakSVMv9HuAaoNV9\nUQo6uS9eDD17QlFRcMc0ZkdkZRlvSSd3EfkhsFJVZwMS+mk1vJJ7TU3sRTJisZKMMcmzce7+ghgt\ncyxwmoicDBQBHUXkMVU9r+mOo0aN2n65rKyMsrKyAB4+tWIl9wkTJlJRcbHnIhnRzJ9vyd2YILTW\nce6VlZVUVlYmfRzRAD/6RGQAcJWqnhblNg3ysdJlzBg3Ln3MmIZtNTU1lJT0prZ2Gm76gLkUFQ2k\nunqh77DG8nI4+WQYMSKlYRvTql13nVv79wc/cJenT890RKkjIqhqiz/C7AxVH9Hmlmk+u2P8i2RY\nWcYYkw6BJndVfSNaqz2XRSvLJLpIxtat8Omn0Lt3CgI1ZgeTg4WAtLKWu49oyT3RRTI+/xy6doX2\n7VMYsDE7gNZWZ08Fm37AR6wO1fLy4QwaNIiXXlrNSSd1oXt3/ykErCRjjEkXa7n78BoK+c47xfzi\nFwdy333xzQ1jyd2Y4KhaacaLJXcfXsn90UfhxhvhoYdcPd3PggVw4IHBxmfMjiiyLGMlmugsufvw\nSu4zZ0JFBXTrBh984H+s+fPh4IODjc+YHZW12r1ZcvcRK7l/8w2sXw8lJTBwIEyb5n2cbdtg4UIr\nyxgTBGut+7Pk7iNWcv/oIzjkEPdPdswx8N573seproYuXaBjx9TEaYwxkSy5+4iV3Jcsgf32c5eP\nOAI+/ND7ODbtgDHBsrKMN0vuPmIl9y+/dLM7Auy7L3z9NaxZE/s4CxZYvd2YoFhZxp8ldx/5+VBf\n33x7ZHLPy4PDDoPZs2Mfx4ZBGmPSyZK7j2hzy0Dj5A7+pRlL7sYEy8a5e7Pk7iOesgzA4YfHTu71\n9fDxx5bcjQmKjXP3Z8ndR6zk/tVXbnx7WGnpGt5557uoC3csXeqmJ9155xQGaswOxlrt3iy5+4iW\n3Ovr3Tj33XZz1ydMmMgppxzEkiV57LXX4UyYMLHR/laSMSZY1lr3Z8ndR7Tk/u23brx6QYFbuKOi\n4mI2bXoVaMemTZVUVFzcqAU/ezb06YMxxqSNJXcf0ZJ7TQ2EZ/dtvnDHfs0W7pg5E/r1S0OwxuxA\nrCzjLYgFsnuKyOsiMl9EPhKRy4IILFtES+6rV7uzTSHawh3fNFu4Y8YMS+7GBMnKMv6CaLlvBa5U\n1YOB7wO/EZFWs9ZQtOS+ePFa2rRZQ01NTaOFO9q3H47I140W7li+HDZvhr33zkDwxpgdVtKLdajq\nCmBF6PIGEfkY6AEsTPbY2aBpcp8wYSK/+MU7wEBKSioYO3Y05eXDGTx4EJ99VsWQIb0YNmy/7fuH\nSzLW0jAmWDbO3VugKzGJSClwOOAzjVbuiEzu4c7Turp5QDfq6vahomIggwcPori4mOLiYo480iX0\nE09093nvPSvJGBM0G+fuL7DkLiIdgGeBy1V1Q7R9Ro0atf1yWVkZZWVlQT18ykQm93DnaW1teIB7\nn+2dp+EyzDHHwBtvNCT3116DO+9Mf9zGtHattdVeWVlJZWVl0scRDeAVEpE2wEvAy6p6b4x9NIjH\nSrfPP3fztVdVuZZ7SUlvams/AboAcykqGkh19cLtyf3tt+Hii2HOHFixAnr3hlWroLAwk8/CmNbl\n5pvd+SaDBsEf/+gaVK2ViKCqLf5+EtRQyIeBBbESey7Ly2uYOCzceZqfP4127W6kqGhgo85TgP79\n3WiaOXPcMnxnn22J3RiTfkmXZUTkWOCnwEci8iGgwPWq+kqyx84GTTtUy8uHM27cZoYNO4wRIy5v\nlNjD+19xBVx0ESxeDFOmpDlgY3YQOVgISKsgRsu8DeQHEEtWijYUsra2LUceuT9N8vp2V17pWvsH\nHOCmAjbGBMs6Uf0FOlqmNYo1/YDXJGD5+XDttamNy5gdnQ2F9GbTD/iIltzXrnWzPBpjMsOGQvqz\n5O6jTRvYurXxtrVrbfpeY0x2s+Tuo6CgcXKvr4cNG9yskMaYzLGSjDdL7j4KCmDLlobr69dD+/au\nXGOMyQwrxfiz5O6jTRtXcw+PdffrTDXGmGxgyd2HiGu919W569aZakx2sLKMN0vucSgsbJzcreVu\nTGZZWcafJfc4RNbdLbkbkx1snLs3S+5xiGy5W83dmMyzce7+LLnHwVruxphcY8k9DlZzNyb7WEnG\nmyX3ODRtudtoGWMyy0ox/iy5x8Fa7saYXGPJPQ5Wczcm+1hZxpsl9zg0bbl36pTZeIzZ0VlZxl8g\nyV1EhonIQhH5VERa3UzmkS33deus5W5MNrBx7t6STu4ikgf8HRgKHAyUi0jvZI+bTazlbkx2sXHu\n/oJouR8FLFLValWtA54CTg/guFnDWu7GmFwTRHLvAXwRcf3L0LZWZDMLFiyipqbGWu7GZAkryXhL\n6xqqo0aN2n65rKyMsrKydD58QiZMmMhrr+3EW2+9xY03jmXz5ho6dbJ+aGMyqTWXYiorK6msrEz6\nOKJJfvyJSH9glKoOC12/DlBVvb3JfprsY6VbTU0NJSW9qa1dDOwCzANKWbWqluLi4gxHZ8yO6447\nYPVqGDYMbr0VXn890xGljoigqi3+OAuiCToT2FdESkSkEDgHmBTAcTOuqqqKwsJSXGIHOASRWqqq\nqjIXlDEGsLKMn6TLMqq6TUQuAabgPizGqurHSUeWBUpLS9mypQr4BugMfAIIpaWlmQzLmB1euCxj\nCT62QGruqvoKcEAQx8omxcXFjB07mhEjXqCgYAWqlfTsOZ7i4t0yHZoxO7xwYm/N9fdkpLVDNReV\nlw9n2rTv6NhxNUcddRFjx9qsYcZkmiV0f5bc49Cp007sscde5OXZMEhjTG6wMX1xCJ+hapOGGZM9\nrN7uzZJ7HMJnqK5bZy13Y7KBlWX8WXKPQ9u2sHmztdyNMbnDknscOnaE9ettXhljsomVZbxZh2oc\nOnVyib1NGyvLGJMNbJy7P0vucQi33PPzreVuTLawce7eLLnHIdxyt6GQxmQHS+j+LLnHIZzct22D\nzp0zHY0xxviz5B6HcHJfvx66ds10NMYYsHq7H0vucejY0Q2D/OYb2H33TEdjjLGyjD8bChmHTp1g\n5UqX5AsLMx2NMcb4s+Qeh/bt3W9bn8OY7GFlGW+W3OOQF3qV7KugMdnBxrn7s5p7nK65Brp1y3QU\nxpgwG+fuLankLiJ3AKcCm4HFwAWqui6IwLLNHXdkOgJjTJgldH/JlmWmAAer6uHAIuD3yYdkjDEm\nWUkld1Wdqqr1oav/BXomH5Ixxvizeru3IDtUfw68HODxjDEmKivL+POtuYvIa8AekZsABW5Q1RdD\n+9wA1KnqeK9jjRo1avvlsrIyysrKWh6xMca0YpWVlVRWViZ9HNEkv9uIyEjgQmCQqm722E+TfSxj\njAG47z745BM47TT461/htdcyHVHqiAiq2uLvKsmOlhkGXAMc75XYjTEmSJFlGSvRRJdszf0+oAPw\nmojMEpHRAcRkjDG+rBDgLamWu6ruF1QgxhgTL2ut+7PpB4wxphWy5G6MyUlWlvFmyd0Yk3OsLOPP\nkrsxxrRCltyNMTlJ1UozXiy5G2Nyjo1z92fJ3RiTk6zV7s2SuzEm51hr3Z8ld2OMaYUsuRtjcpKV\nZbxZcjfG5Bwry/iz5G6MMa2QJXdjTE6yce7eLLkbY3KOjXP3Z8ndGJOTrNXuLZDkLiJXiUi9iHQO\n4njGGOPFWuv+kk7uItITGAJUJx9O9gpiwdpMyuX4czl2sPgzLdfjT1QQLfd7cOuotmq5/g+Sy/Hn\ncuxg8adKvGWZbI0/1ZJK7iJyGvCFqn4UUDzGGOPLyjL+fNdQFZHXgD0iNwEK3AhcjyvJRN5mjDEp\nZx2q3kQTfIVE5BBgKvAdLqn3BJYBR6nqqij725/CGGMSoKotbjgnnNybHUjkc6Cvqq4J5IDGGGMS\nFuQ4d8XKMsYYkxUCa7kbY4zJHik7Q1VEdhWRKSLyiYi8KiI7R9mnp4i8LiLzReQjEbksVfHEQ0SG\nichCEflURK6Nsc//isgiEZktIoenO0YvfvGLyLkiMif0M11EDs1EnLHE8/qH9usnInUiclY64/MT\n5/9PmYh8KCLzRGRaumP0Esf/TycRmRT63/9IREZmIMyoRGSsiKwUkbke+2Tze9cz/oTeu6qakh/g\nduB3ocvXAn+Jsk9X4PDQ5Q7AJ0DvVMXkE28e8BlQAhQAs5vGApwE/Dt0+Wjgv5mINYn4+wM7hy4P\ny7X4I/b7D/AScFam427h678zMB/oEbreJdNxtzD+3wO3hWMHvgbaZDr2UDw/AA4H5sa4PWvfu3HG\n3+L3birnljkdeDR0+VHgjKY7qOoKVZ0durwB+BjokcKYvBwFLFLValWtA57CPYdIpwOPAajqe8DO\nIrIH2cE3flX9r6quDV39L5l7raOJ5/UHuBR4Fmg2IivD4on/XOA5VV0GoKqr0xyjl3jiV6Bj6HJH\n4GtV3ZrGGGNS1emA12CObH7v+safyHs3lcl9d1VdGQpsBbC7184iUor75HovhTF56QF8EXH9S5q/\ngE33WRZln0yJJ/5IvwBeTmlELeMbv4h0B85Q1X+QfZ338bz++wOdRWSaiMwUkRFpi85fPPH/HThI\nRJYDc4DL0xRbELL5vdtScb13fU9i8uJzglNTMXtuRaQDrjV2eagFb1JIRAYCF+C+CuaSv+FKfGHZ\nluD9tAH6AoOA9sC7IvKuqn6W2bDiNhT4UFUHiUgv4DUR6WPv2fRpyXs3qeSuqkNi3RbqHNhDVVeK\nSFdifI0WkTa4xP64qr6QTDxJWgbsFXE9fFJW03329NknU+KJHxHpAzwADNPsOichnviPBJ4SEcHV\nfE8SkTpVnZSmGL3EE/+XwGpV3QRsEpE3gcNwte5Miyf+C4DbAFR1cejclt7A+2mJMDnZ/N6NS0vf\nu6ksy0wCRoYunw/EStwPAwtU9d4UxhKPmcC+IlIiIoXAObjnEGkScB6AiPQHvg2XnrKAb/wishfw\nHDBCVRdnIEYvvvGr6j6hn71xDYKLsySxQ3z/Py8APxCRfBHZCdex93Ga44wlnvirgcEAoXr1/sCS\ntEbpTYj9bS6b37thMeNP6L2bwt7fzrjpCT4BpgC7hLZ3A14KXT4W2Ibrmf8QmIX7VMpUj/WwULyL\ngOtC234F/DJin7/jWlpzcGfkZrynPd74gQdxIxxmhV7vGZmOuaWvf8S+D5NFo2Va8P9zNW7EzFzg\n0kzH3ML/n27Aq6HY5wLlmY45IvbxwHJgM7AU9y0jl967nvEn8t61k5iMMaYVsmX2jDGmFbLkbowx\nrZAld2OMaYUsuRtjTCtkyd0YY1ohS+7GGNMKWXI3xphWyJK7Mca0Qv8PK9JuKWVhTugAAAAASUVO\nRK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f85b71bd2e8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "X_poly = PolynomialFeatures(degree=30).fit_transform(X)\n",
    "X_test_poly = PolynomialFeatures(degree=30).fit_transform(X_test)\n",
    "\n",
    "model = LinearRegression()\n",
    "model.fit(X_poly, y)\n",
    "y_test = model.predict(X_test_poly)\n",
    "\n",
    "plt.scatter(X.ravel(), y)\n",
    "plt.plot(X_test.ravel(), y_test)\n",
    "plt.title(\"mean squared error: {0:.3g}\".format(mean_squared_error(model.predict(X_poly), y)))\n",
    "plt.ylim(-4, 14);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we increase the degree to this extent, it's clear that the resulting fit is no longer reflecting the true underlying distribution, but is more sensitive to the noise in the training data. For this reason, we call it a **high-variance model**, and we say that it **over-fits** the data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Just for fun, let's use IPython's interact capability (only in IPython 2.0+) to explore this interactively:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEKCAYAAADpfBXhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmUVPWZ//H3w9LSCo1LWlzTLTEGXNBgUDMaKFRAiCyT\niYlksqg4iTFGT5xk1CQTyWQympk5Mf4mYWIclJiftCT6G3EXCZSJGgKi4IYrgqAIjUEisko/vz/u\nLSia6u5abtWtuv15nVOHWm7d+9yi71Pf+q7m7oiISLL0iDsAERGJnpK7iEgCKbmLiCSQkruISAIp\nuYuIJJCSu4hIAim5i3TBzNrMbGDccYgUQsldpGs1NRjEzE41szlm9o6ZrTWzWWZ2SCfbp81si5n9\n1czeM7NlWa81hV9umdf+ambfq8yZSCmU3KVbMbNi/uYtomP3zOe5QveRwwHATUBTeNsE3NrJ9g5c\n6u4N7t7P3QfneL1/+FqDu/+4kJglHkruCWRmr5vZt81saVjautnMDjazB8KS1xwz65+1/Wlm9riZ\nbTCzp81sRNZrF5jZC+H7XjWzr2a9NsLMVpnZlWEJ8U0zu6CTuC4ws9fCfb1mZpPD53uY2X+aWWt4\njEvD0mKPrPM5M2s/15rZb7Ie/9bM1oTxp83s2KzXbjWzaWZ2v5m9B6TMrC483srwfdPMbJ+s93zH\nzN4ys9VmdiGdlNzNrMHM/ifcfpWZ/cjMLHztK2b2mJn91MzWA9d28JyZ2ffNbIWZvW1mM8ysIdxH\npuR8kZmtBH7f+f8+uPtD7n6Xu29y963Az4G/6eJtnX2BGcoVNUf/Ycn1GeAs4BhgAvAAcDXwIaAn\ncDmAmR0O3Af8i7sfAHwbuMvMDgr3sxYY5+4NwIXADWZ2UtZxDgH6AYcBFwO/yP7iyDCzfYEbgTHh\nvv4GWBK+/FVgHHAi8Angs3RdFZL9+gPAR4CDgaeA29ttOxn4kbv3Ax4HfgIcDQwJ/z0c+EEY5znA\nlQSf3UeBs7uI49fAdmAg8HFgFMHnkHEq8GoY2487eO5C4MvAiHA//QgScrbhwCBgTBjnUjM7v4vY\nMkYAz3exzXVmts7M/pj95R5yYIWZvWFmt2T9bUg1c3fdEnYDXgcmZz2+E/hF1uPLgP8X3v8n4Nft\n3v8Q8KUO9v2/wDfD+yOA94EeWa+vBU7J8b59gb8Afwv0affa74GvZj0eBezM7Dc8nzOzXr8WuK2D\n+PYH2oB+4eNbgRntttkEHJX1+JPA8vD+dODfsl77aBjLwBzHOhjYCuyT9dz5wLzw/leAFe3ek+u5\nucAlWY+PIfjC6EFQrbITaCryb2EI8A7wN51sMwzYD+hN8CXz18znEz4/NIylEfgd8FDcf+O6dX1T\nyT251mbd35Ljcd/wfhPwOTP7S3jbAJwOHApgZmPN7E9h49wGYCxB6T/jHXdvy3q8OWvfu7j7ZuDz\nwNeBNWZ2r5kdE758GLAqa/OV+Z5kWKVzfVid8y7BF4G3i3FV1vaNBF80izPnDDwIZEqjuWLpqMqi\niSAhrsn67H7Z0bE7ee4w9jznlUAvYEDWc6s7iKFDZnY0wa+ab7r7Ex1t5+6L3P19d9/h7rcR/LoZ\nF772vrs/5e5t7t5KUDAYbWb7FRqPVFavuAOQ2K0iKAV/rf0LZlZHUOr/IjDb3dvM7H8psoHR3R8B\nHgnrt38M3ExQ+l8DHJm1aVO7t75PkJAzsnt+/D0wnqBk/0ZYJbShXYzZVTjrCb6AjnP3NTnCzBVL\nR1VEqwhK7gd5WMzNIdfz7Z97iz3PuQnYQfCFfGQH7+mUmTUBjwA/dPeZhbw3PFZn/8eOqnSrnv6D\n5P8C481sdFgK7hM2lB4G1IW39WFiHwuMLuYgFjToTgjr3ncQVI1kSvy/BS43s8PN7ADgqnZvXwKc\nb2a9zCxTJ5/RF9gGbAhLk9fRSSIMk/DNwM/CUjzhcTPn9VvgAjMbHMb6g0729TYwh6Adol/YMDrQ\nzIbn8ZFkawG+ZWbNZtaX4IvvjqxfRAV9mYbtKL8H/svdb+5i2/7h//0+ZtbTzP4e+BRB1RxmdoqZ\nHROe20EE7Sbz3f29wk5RKk3JPZnaJ7fOkt1qYCLwXaCVoErg2wT13ZsIGl5/F1ZfnA/MLvDYGT0I\nGirfJCg9DyeoooEg2T4MLAWeBO5q995/Jmj4/AtBfXt2g+ltwBvhfp8DOqx+yHIVQYPmgrAqZw5B\nPTfu/hDwM2Ae8DJd9075MsEX4AthfL9jz18W+bgF+A3wB+A1gl8Wl2e9vtdnambPWdjbKIcpwFHA\nVMvqn5713mvM7P7wYW/gX4F1BP//3wAmuvur4esDCRL9X4FnCH6pfKHA85MYWMe/JtttaDYdOBdY\n6+5D2r32j8B/AB9y979EHqV0K2GVwnKgd7v6fBHJUyEl91sJu2FlM7MjCHo35N0IJpKHSAYOiXRX\neSd3d3+MoKGqvRuA70QWkUigpob8i1SbknrLmNkEYJW7PxsOyhMpmbuvJBhoJSJFKjq5m1k9QSPc\nqOynS45IRERKVkrJ/SNAM7A0nEvjCIKBIae4+7r2G5uZfmaLiBTB3QsuOBfaFdLCG+7+nLsf4u4D\n3f0oghF0H8+V2LMCrNnbtddeG3sM3TX+Wo5d8cd/q/X4i5V3cjezmQR9iI8JJxC6sH3uRtUyIiJV\nIe9qGXfvdOCCu2ulGhGRKqERqnlKpVJxh1CSWo6/lmMHxR+3Wo+/WHmPUC35QGZeqWOJiCSFmeEV\naFAVEZEaoOQuIpJASu4iIgmk5C4ikkBK7iIiCaTkLiKSQEruIiIJpOQuIpJASu4iIgmk5C4ikkBK\n7iIiCaTkLiKSQEruIiIJpOQuIpJAhazENN3M1prZM1nP/buZLTOzJWZ2l5k1lCdMEREpRCEl91uB\nMe2emwMc5+4nAa8A10QVmIiIFC/v5O7ujwEb2j03193bwocLgCMijE1ERIoUZZ37RcCDEe5PRESK\nlPcC2Z0xs+8BO9x9ZmfbTZ06ddf9VCrVbdc2FBHpSDqdJp1Ol7yfgtZQNbMm4F53H5L13AXAPwBn\nuvu2Tt6rNVRFRApU7BqqhZbcLbxlDnoO8B1geGeJXUREKivvkruZzQRSwEHAWuBa4LtAHfBOuNkC\nd7+0g/er5C4iUqBiS+4FVcuUQsldRKRwxSZ3jVAVEUkgJXcRkQRSchcRSSAldxGRBFJyFxEpws6d\n8NhjsGZN3JHkpuQuIpKnzZvhnnvgoovg0EPhsstg+fK4o8pNXSFFakxraysrVqygubmZxsbGuMNJ\nvPXr4b774O67Yf58OPlkmDQJJkyA5ubyH1/93EUSKjuZz507jylTLqWurpnt21cwffo0Jk/+fNwh\nJs5rr8Hs2UFCX7oURo0KEvq4cXDggZWNRcldJIFaWmbtSubbti2nrc3Zvv0PwBDgGerrR7Jy5Ysq\nwZeorQ0WL96d0NevD0rmEyfCWWdBnz7xxabkLpIwra2tNDUNYsuW+QTJfCbBrB+v7NqmoWEoc+fe\nxLBhw2KKsnZt3x5Us8yeHdwaGoJkPnEinHoq9KiSFslKTRwmIhWyYsUK6uqa2bIlMwnrKIIJWJ8h\nU3LfsWMlzZWo+E2IjRvhgQeCZP7ww3DssUEynzcPPvaxuKOLlpK7SJVqbg7q1Xcn8zX07t2DXr1G\n0rt3Ezt2rGT69GmqkunCqlVBD5fZs2HBAhg+PKg/v/FGGDAg7ujKR9UyIlUsU+eencxPOmkICxcu\n5JRTTmHw4MFxh1h13OG554K689mz4fXX4dOfDhL66NHQt2/cERZGde4iCaXeMl374AN4/PHdCd09\nqG6ZNAnOOAN61XAdhZK7SMLt3cBaW71lou6f//77MGdOkNDvvx+amnYn9BNOACs4HVYnTfkrknCZ\nBtYgsQMMoXfvJlasWBFfUHlqaZlFU9MgRo26hKamQbS0zCpqP+vWwfTpMH58MEJ02jQYNgyeeiro\nyviDH8CQIclJ7KUoZCWm6cC5wNrMGqpmdgAwC2gCVgCfc/eNHbxfJXeREtRqyb3UuF9+eXf/8+ef\nhzFjghL6uHGw//5lDz92lSi53wqMaffc1cBcd/8YMA+4ptAARCQ/jY2NTJ8+jfr6kTQ0DKW+fmRN\n9JYp9BdHW1vQq+Waa2DwYBg5Mpi/5Z//GdauhVmz4Atf6B6JvRQF1bmbWRNwb1bJ/UVghLuvNbND\ngLS7D+rgvSq5i0Sg1uaW6ajkvnjxY2zatInm5mb69Wtk3ryghH7PPXDQQbsHFH3iE9UzoCgOcQ1i\nOtjd1wK4+9tmdnCJ+xORLmQSeqbkW+0JvrGxkRtuuJ4rrhhBXV0zH3zwBlOmfJGhQz+N2RfYtm0Y\nffqM5eST65g4Ef74Rzj66Lijrn1RdxDqtGg+derUXfdTqRSpVCriw4skX/Z8M7XQHbKlZRbf+tbV\n1NUdybZtOxg79k6mTetJW9tPgZ7AKtrajueuux6v+i+qSkin06TT6ZL3U2q1zDIglVUtM9/dc46q\nULWMSOlKbZysdJXOunWtfPjD57Ft2yxgAPABPXrcQV3dYrZuvWHXdpojp2OV6gpp4S3jHuCC8P5X\ngNmFBiAi+SulO2RU3RG7smNHMFfL5ZfDiSc2sGPHbwgSO0Av9t33J7jPIJhWATRHTnnkndzNbCbw\nBHCMmb1hZhcC1wOjzOwl4KzwsYiUyZ7zzUC+ibG1tZUpUy5ly5b5bNy4mC1b5jNlyqW0trZGEtd7\n78Gdd8IXvwiHHAJXXx3M2/Lb375PXd1Je8S7c+db3Hjjv5fU66e1tZVFixZFFn8iuXtFbsGhRKRU\nM2fe4fX1B3pDw8e9vv5Anznzji7fs3DhQu/ff6gHA/ODW0PDx33hwoVFx/HWW+433eQ+dqx7v37u\nY8a4T5vmvnp1fvGuW7fOFy5c6OvWrSvouJn99e8/NO/zr2Vh7iw452r6AZEaVGjdeRQDoNzhxRd3\nz3/+0ktwzjlBd8WxY4P50KOKt5znUWs0n7uIdCgzAGrKlMKmC965MxhQlEnomzcHyfxf/gVGjIC6\nuvyPH0Xy3XuO+91tDklN7sVSyV2kxpTSFTKfEvSWLTB3bpDM7703qEPPDCgaOrQ887bkW7LPp+Re\na4O8ulJsyV117hKpYutRa01c57lu3Tqvrz/QYWlYd77U6+sPLDmO9evdZ8xw/9u/dW9ocE+l3G+4\nwX358ogC70ShdeidtTkksT6eIuvcldwlMkm8sHKJ8zyjbBh97TX3n/7UfcSIIKF/5jPuv/51kOgr\npdgvq1xfruX64oubkrvEqlourHKXqOM+z1KO39bm/uST7t//vvsJJ7gffLD7xRe733uv++bNFQg+\nhyi/rMrRI6gaFJvcu/F0PBKlaphrvBKDdOI+z0Jnhty+HR55BL7xDfjwh4PZFLdvh1/+Et56C26+\nGc49F+rrKxL+Xortt1/ufSVCMd8IxdxQyT3RarlEW43HySeOjn6hbNzofscd7uef777//u6nneZ+\n/fXuy5ZVNMS8FdNvvxL7qhaoWkbiFueFVcmf5NWYQFavDgYQjR4dDCgaN879V79yX7Mm7sjyU2x1\nWkd170lq1C82uasrpEQqrm5olR7cEnd3O/dgVaLMCkXLlwcrE02cGKxU1K9fxUOquFqbHbNYWiBb\nur3MxZ49SCdJF/vOnfD447sT+s6du/uff+pT0Lt33BFWTncaqaoRqlL1yl3anTz585x99pmJGsCy\neTPMmRMk9PvugyOPDJL5XXfBiSd234WgNVK1ayq5S0V0l5/QUWhtDUaGzp4N8+fDKacECX3CBGhq\niju66qCSex7vU3KXcqvlC7FSdeuvvLJ7/pZnn4XRo4OEPm4cHHBAdMeJs60g6mMnvRouQ9MPSMXl\n2yuhVgeXlHMk6s6d7n/+s/s117gfe6z7IYe4f+1r7g884L51a2SH2UOcI2vLdeyk9YzJBXWFlIxK\n/MEXcrFG3Te8EudXjv7sW7e6P/hgkMQPPdR98OAguS9YECT7coqzf361jA2oVbEmd+BbwHMEQ8Nu\nB+pybFPmj0DcK1M6K+ZijapveKVKn1H92tiwwf32293PO8+9f3/30093/4//cH/55bKE3aE4fz3V\n6i+3ahFbcgcOA5ZnEjowC/hyju3K/BFIpUpIxV6spZa4K1kCLOVYK1e6/9d/uZ91VjCgaPx49+nT\n3deujTzMvKnkXruKTe5RzS3TE9jPzHoB+wJvRbRfKUCl5j0pdg6PxsZGhg0bVnRjWiXndSlkDhd3\nWLo0WMBi6FA4+WR48km47DJYswbuuQcuuggOPjjyMHPKtb5ooXPSRCnOY3drxXwjtL8BlwPvAWuB\n33SwTXm/3qSiJaQ4huDHUQLs6NfGjh3u8+a5X3GFe3Oz+8CB7t/6lvujjwavxaWraqs4GyC7Q+Nn\nORDX9ANmtj9wF3AesBG4E/idu89st51fe+21ux6nUilSqVRJx5a9VbJ7WCW61bU/RkvLLC666BJ6\n9jyYnTvXccstv6xY97dNm+Dhh4PuivffDwMHBt0VJ02C446Lf0BRObqcxj3NQneUTqdJp9O7Hv/w\nhz/E4+gKCXwWuDnr8ZeAn+fYrozfbZKtnCWkSpa+cpVCM8/tt9+JFfnFsGZNMAHXpz8d1J+PHu3+\ni1+4r1pV1sMWJeqGy+6y+Eq1I8YG1VOAZ4E+gAEzgG/k2K7cn4GUWSUv9lxVMH367F+Raplly4Ip\ncj/5yWDK3PPPd29pcX/33UgPE7koq63UCFo9ik3uJTeouvtCgqqYp4GlYYL/Van7lerS2trKlCmX\nsmXLfDZuXMyWLfOZMuVSWltbczbglSpX42nPngfTo8eRRN2g2tYGTzwBV10FgwbB2WfDG2/A1Kmw\ndi20tMBZZ7Xy8svRnmPUomy4zLfxuhz/9xKRYr4RirmhkntN6+gn/49+9OOyjTwsZ8l9y5ZgebmL\nL3YfMMD9+OOD5ecWLQqWo8tWa9UTUVSd5VNyr7XPpVahEapSTh1d7H367L9XAn744Ycj+fmeq0fO\nzJl3eJ8+B3h9/fHep88BBSWUd95xv+22YCHohgb34cODBaJfe63w8+4O1ROd9Yjqzp9LpSm5S9m1\nv9h/9KMftyvN3+Gwb6SNne1LoZdddoVDH4cPO/Txyy67vNP3v/66+89+5p5KBQl90iT3GTPcW1vz\nO353H13Z0a+A7v65VFKxyV2zQkpBMl3j+vbty6pVq5g48fNs3foocCjwMSBNuWZ+XLZsGcceezKw\nYNcx4DReeGExgwcPBoI08/TTuxe0WLMGxo8PuiuefXbhC0HX8oyW5aTPpXK0WIdURGNjI3Pnzts1\nN3tbm9O79+nU1R3G++8fRK4GuKgu9oULFwJ7NqjCETzxxCLefHPwrilz+/QJ+p9PmwannQY9exZ/\nzEwj5ZQpI/cYO9DdE5g+l+qnkrsUpKMS24wZ/80FF3y9rCW5vUvuK4E/0dBwHoMH99w1oGjQoOgH\nFGkwT276XMpPJXepiI6WNzvqqKPKXpLr338wqdQM0um3gYHAi6RSxu239+SwwyI7TE6NjY1KXjno\nc6leKrlLQbqqa42yJOcOL7ywe4WiV14JViY6+eTV1Nc/yogRQ3fVtUu0VCKvHlpmTzoV5cVazvlr\ndu4MBhRlEvr27UH9+cSJMHw49O4dyWGkE1rvtroouUuHynGxRvllsXkzPPJIkMzvuw8OP3x3Qj/p\npPgn5OpO1Aum+ii5S07VerG2tgaJfPZsmD8/mAN90iSYMAG6mBZeymjRokWMGnUJGzcu3vVcQ8NQ\n5s69iWHDhsUYWfelBlXJqaMG0Ci7KObr1Vd3V7csXQqjRsFnPwu33AIHHljRUKQDey7EEhQG8lmI\nRaqPknvCxXmxtrXB4sXBYKLZs2H9+qBkftVVcNZZQX90qS7qv54cqpbpBiq5gMf27UE1y913B8vL\nNTTsrj8/9VToEdXCjlJW6i1TPVTnLjllTxewadOmslys774LDz4YlM4feihYlSiT0D/2sUgPJdLt\nqM5d9pKrl0xUjWKrVgUl89mzYcECGDEiSOY33ggDBkRyCBEpQSQldzPrD/wPcDzQBlzk7n9ut41K\n7iUo9Gdy1L1k3OHZZ3c3iL7+Opx7bpDQR4+Gvn0LPycR6VrcJfcbgQfc/Twz6wXsG9F+heL6qUfR\nS+aDD+Cxx3YndPegu+J//ieccQb00u8+kapVcsndzBqAp939I11sp5J7EYotgRf7vvffh4cfDpL5\n/fdDU1OQ0CdOhBNO0IAikUqLs+R+FLDezG4FTgSeBK5w9y0R7LvbK7YEXkiXtrVr4d57g4T+6KNB\nr5ZJk+Bf/xWOPLJMJyYiZRVFyT0zB+sn3f1JM/sZsNHdr223nUruRSi17ryjuvqXX97d//z552HM\nmCChjx0L++9fvvMRkcLEWXJfDaxy9yfDx3cCV+XacOrUqbvup1IpUqlUBIdPtlIHlWSmZG1rC3q1\nZBL6X/8aDCj6wQ8glYJ99inveYhIftLpNOl0uuT9RNVb5lHgH9z9ZTO7FtjX3a9qt41K7iUoZlDJ\n1q0wb16QzO+5Bw46iF0LWpx8sgYUidSCWAcxmdmJBF0hewPLgQvdfWO7bZTcK2DDhqAhdPbsYKbF\nIUN2Dyg6+ui4oxORQmmEao2KYpj3U0+9w8yZm1i48FCWLKnjzDODZH7uuaCR4yK1Tcm9BuXTfz1X\n8neHJUuC0vmMGRtYuXInvXs/jtnvuOmmSVxwwWfjOB0RKQMl9xqTTy+Y7OS/bdtqrrzyDt57bySz\nZwcrEo0evZnp089j+/brOtyHiNS2YpO7mtRikum/HiRlyO6/DkHyv+iif2LLlqfYuHExW7e+yXXX\n9adfv/d54IFgPdELL3ye+vq3O9yHiHRfSu4x2XOedcjMs15fP5Bf/QomTerN1q3PA03h673o1+9i\nJk16geOOC0aKdrQPLawgIkruMcn0X+/TZyT77fd39Op1N4cf/jLDhx9EOg0XXtiDPn2Op7PEndlH\nff1IGhqGUl8/UgsriAigOvdY7NwZDChqadnM3Xc727b1ZMIEZ/LkeoYPh7q6YLt8F9nQwgoiyaUG\n1Sq3ZQvMnRv0cLn3XujT513eeutm+vRZwgcfPMQttyhxi8jelNyr0DvvwH33BQn997+HoUOD/ufH\nHfcq48efzLZts4EUhfRyUbIX6V7UW6ZKLF8ON9wQzNcycGAw7H/SpOD5+fNhwIBZjB8/jG3bDgH+\nDpgFHEqPHh/i6aef7nTfLS2zaGoaxKhRl9DUNIiWllkVOCMRqUUquZfIHRYv3r2gxbp1MH58UEI/\n6yyor9+9ba6+7XAG0BP4EPX1f+m0Xj3KlZVEpDbEvRJTt7J9O6TTuyfk2nffoHT+y18Gc6H37Jn7\nfXvPzX4osBN4DBjCli3PMGXKSM4++8y9EnYUKyuJSPeh5J6njRvhwQeDhP7QQzB4cFA6f+QRGDQo\nv33s2S99CPAIcBi5BiG1T9h7v1d92kWkY0runVi9OiiZz54Nf/oTfOpTQQn9hhvgkEMK31/7udm3\nb3+dtjZn+/auE3ap87qLSPeiOvcs7sGqRJkFLZYvh3HjgoQ+Zgz07RvNcbJ7vMydOy+vvuy53qvE\nLpJ86gpZpA8+gCee2J3Qd+7cvaDFGWcEE3SVmxK2iHREyb0AmzfDnDlBQr///mAR6ExCHzIkmLdF\nRKQaxJ7czawH8CSw2t0n5Hg91uS+bt3uAUXz58MppwQJfcIEaGrq+v0iInGohq6QVwAvAA0R7rMk\nr7yyu//5s8/C6NHwuc/BjBlwwAFxRyciUj6RJHczOwIYB/wYuDKKfRajrQ0WLdqd0DdsCErm3/0u\nnHkm7LNPXJGJiFRWVCX3G4DvAP0j2l9B/vhHuP32oNviAQcE1S233ALDhkEPTbAgIt1QycndzD4N\nrHX3JWaWAireHPncc3D00fDoo/DRj1b66CIi1SeKkvvpwAQzGwfUA/3M7DZ3/3L7DadOnbrrfiqV\nIpVKRXB4+PrXI9lNSdSdUUSikE6nSafTJe8n0q6QZjYC+Mdq7C1TTtkLWW/fvqLLgUgiIvmKvStk\nGES3S+6arVFEyqkq5nN390dzJfYky8zWmGvyLxGRuKgvSYn2nK0RNFujiFQDJfcSZWZrrK8fSUPD\nUOrrR2q2RhGJXbecW6Yc1FtGRMqhKhpUOz1QwpO7iEg5VEWDqoiIVAcldxGRBFJyFxFJICV3EZEE\nUnIXEUkgJXcRkQRSchcRSSAldxGRBFJyFxFJICX3ErS2trJo0SJaW1vjDkVEZA9K7kVqaZlFU9Mg\nRo26hKamQbS0zIo7JBGRXTS3TBG0QIeIVEpsc8uY2RFmNs/MnjezZ83s8lL3We20QIeIVLsoqmU+\nAK509+OATwLfMLNBEey3ammBDhGpdiUnd3d/292XhPc3AcuAw0vdbzXTAh0iUu2iXiC7GUgDx4eJ\nPvu1xNS5Z2iBDhEpt2Lr3HtFGEBf4E7givaJPWPq1Km77qdSKVKpVFSHj0VjY6OSuohEKp1Ok06n\nS95PJCV3M+sF3Ac86O43drBN4kruIiLlFusye2Z2G7De3a/sZBsldxGRAsWW3M3sdOAPwLOAh7fv\nuvtD7bZTchcRKZAWyBYRSSAtkC0iIrsouYuIJJCSu4hIAim5i4gkkJK7iEgCKbmLiCSQkruISAIp\nuYuIJJCSu4hIAim5i4gkkJK7iEgCKbmLiCSQkruISAIpuYuIJJCSu4hIAim5i4gkUCTJ3czOMbMX\nzexlM7sqin1Ws9bWVhYtWkRra2vcoYiI5FRycjezHsDPgTHAccBkMxtU6n6rVUvLLJqaBjFq1CU0\nNQ2ipWVW3CGJiOwlijVUTwOudfex4eOrAXf3n7TbruaX2WttbaWpaRBbtswHhgDPUF8/kpUrX6Sx\nsTHu8EQkgeJcZu9wYFXW49Xhc4mzYsUK6uqaCRI7wBB6925ixYoV8QUlIpJDr0oebOrUqbvup1Ip\nUqlUJQ9fsubmZrZvXwE8Q6bkvmPHSpqbm2ONS0SSI51Ok06nS95PVNUyU939nPBxYqtlIKhznzLl\nUnr3bmKzGNsxAAAG2UlEQVTHjpVMnz6NyZM/H3dYIpJQxVbLRJHcewIvAWcBa4CFwGR3X9Zuu0Qk\ndwjq3lesWEFzc7Pq2kWkrGJL7uHBzwFuJKjDn+7u1+fYJjHJXUSkUmJN7nkdSMldRKRgcfaWSTwN\nWhKRWqPk3gUNWhKRWqRqmU5o0JKIxE3VMmWgQUsiUquU3Dux56Al0KAlEakVSu6daGxsZPr0adTX\nj6ShYSj19SOZPn2aqmREpOqpzj0PGrQkInFRP3cRkQRSg6qIiOyi5C4ikkBK7iIiCaTkLiKSQEru\nIiIJpOQuIpJASu4iIgmk5C4ikkAlJXcz+3czW2ZmS8zsLjNriCowEREpXqkl9znAce5+EvAKcE3p\nIYmISKlKSu7uPtfd28KHC4AjSg9JRERKFWWd+0XAgxHuT0REitSrqw3M7BFgQPZTgAPfc/d7w22+\nB+xw95md7Wvq1Km77qdSKVKpVOERi4gkWDqdJp1Ol7yfkmeFNLMLgH8AznT3bZ1sp1khRUQKVOys\nkF2W3Ls46DnAd4DhnSV2ERGprJJK7mb2ClAHvBM+tcDdL+1gW5XcRUQKpMU6REQSSIt1iIjILkru\nIiIJpOQuIpJASu4iIgmk5C4ikkBK7iIiCaTkLiKSQEruIiIJpOQuIpJASu4iIgmk5C4ikkBK7iIi\nCaTkLiKSQEruIiIJpOQuIpJAkSR3M/tHM2szswOj2J+IiJSm5ORuZkcAo4CVpYdTvaJYsDZOtRx/\nLccOij9utR5/saIoud9AsI5qotX6H0gtx1/LsYPij1utx1+skpK7mU0AVrn7sxHFIyIiEejV1QZm\n9ggwIPspwIHvA98lqJLJfk1ERGJW9ALZZnY8MBfYTJDUjwDeBE5x93U5ttfq2CIiRShmgeyik/te\nOzJ7HRjq7hsi2aGIiBQtyn7ujqplRESqQmQldxERqR5lG6FqZgeY2Rwze8nMHjaz/jm2OcLM5pnZ\n82b2rJldXq548mFm55jZi2b2spld1cE2/8fMXjGzJWZ2UqVj7ExX8ZvZF8xsaXh7zMxOiCPOjuTz\n+YfbDTOzHWb2mUrG15U8/35SZva0mT1nZvMrHWNn8vj7aTCze8K//WfN7IIYwszJzKab2Voze6aT\nbar52u00/qKuXXcvyw34CfBP4f2rgOtzbHMIcFJ4vy/wEjCoXDF1EW8P4FWgCegNLGkfCzAWuD+8\nfyqwII5YS4j/NKB/eP+cWos/a7vfA/cBn4k77gI///7A88Dh4eMPxR13gfFfA1yXiR14B+gVd+xh\nPGcAJwHPdPB61V67ecZf8LVbzrllJgK/Du//GpjUfgN3f9vdl4T3NwHLgMPLGFNnTgFecfeV7r4D\nuIPgHLJNBG4DcPc/A/3NbADVocv43X2Bu28MHy4gvs86l3w+f4BvAncCe/XIilk+8X8BuMvd3wRw\n9/UVjrEz+cTvQL/wfj/gHXf/oIIxdsjdHwM668xRzddul/EXc+2WM7kf7O5rw8DeBg7ubGMzayb4\n5vpzGWPqzOHAqqzHq9n7A2y/zZs5tolLPvFnuxh4sKwRFabL+M3sMGCSu/831dd4n8/nfwxwoJnN\nN7NFZvalikXXtXzi/zlwrJm9BSwFrqhQbFGo5mu3UHldu10OYupMFwOc2uuw5dbM+hKUxq4IS/BS\nRmY2EriQ4KdgLfkZQRVfRrUl+K70AoYCZwL7AX8ysz+5+6vxhpW3McDT7n6mmX0EeMTMhuiarZxC\nrt2Skru7j+rotbBxYIC7rzWzQ+jgZ7SZ9SJI7L9x99mlxFOiN4EPZz3ODMpqv82RXWwTl3zix8yG\nAL8CzvHqGpOQT/yfAO4wMyOo8x1rZjvc/Z4KxdiZfOJfDax3963AVjP7A3AiQV133PKJ/0LgOgB3\nfy0c2zIIeLIiEZammq/dvBR67ZazWuYe4ILw/leAjhL3LcAL7n5jGWPJxyLgaDNrMrM64HyCc8h2\nD/BlADM7DXg3U/VUBbqM38w+DNwFfMndX4shxs50Gb+7DwxvRxEUCC6tksQO+f39zAbOMLOeZrYv\nQcPesgrH2ZF84l8JnA0Q1lcfAyyvaJSdMzr+NVfN125Gh/EXde2WsfX3QILpCV4C5gD7h88fCtwX\n3j8d2EnQMv808BTBt1JcLdbnhPG+AlwdPvc14KtZ2/ycoKS1lGBEbuwt7fnGD9xM0MPhqfDzXhh3\nzIV+/lnb3kIV9ZYp4O/n2wQ9Zp4Bvhl3zAX+/RwKPBzG/gwwOe6Ys2KfCbwFbAPeIPiVUUvXbqfx\nF3PtahCTiEgCaZk9EZEEUnIXEUkgJXcRkQRSchcRSSAldxGRBFJyFxFJICV3EZEEUnIXEUmg/w8F\njwbmL3sdPwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f85b70e2828>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython.html.widgets import interact\n",
    "\n",
    "def plot_fit(degree=1, Npts=50):\n",
    "    X, y = make_data(Npts, error=1)\n",
    "    X_test = np.linspace(-0.1, 1.1, 500)[:, None]\n",
    "    \n",
    "    X_poly = PolynomialFeatures(degree=degree).fit_transform(X)\n",
    "    X_test_poly = PolynomialFeatures(degree=degree).fit_transform(X_test)\n",
    "\n",
    "    model = LinearRegression()\n",
    "    model.fit(X_poly, y)\n",
    "    y_test = model.predict(X_test_poly)\n",
    "\n",
    "    plt.scatter(X.ravel(), y)\n",
    "    plt.plot(X_test.ravel(), y_test)\n",
    "    plt.title(\"mean squared error: {0:.3g}\".format(mean_squared_error(model.predict(X_poly), y)))\n",
    "    plt.ylim(-4, 14)\n",
    "\n",
    "    \n",
    "interact(plot_fit, degree=[1, 40], Npts=[2, 100]);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Detecting Over-fitting with Validation Curves\n",
    "\n",
    "Clearly, computing the error on the training data is not enough (we saw this previously). As above, we can use **cross-validation** to get a better handle on how the model fit is working.\n",
    "\n",
    "Let's do this here, again using the ``validation_curve`` utility. To make things more clear, we'll use a slightly larger dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXUAAAEACAYAAABMEua6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHyJJREFUeJzt3XuQXOV55/HvQzSXRqORoHZ8gzBjzBJYm4uUVWJXqEpD\nAJOkNqbY3bDjir3GsywuGUMRZwO2NyVRsR0TKuU4cWEBGRvbawmVL1Um2SQDs2jspdZkZCGQY0m+\nZgZ8gWl7WZWhBiFbz/7RPaPWTF/OOX3u8/tUdWl6dLrP02e6n/P2816OuTsiIlIOp2UdgIiIxEdJ\nXUSkRJTURURKREldRKRElNRFREpESV1EpES6JnUzmzSz58zsYNPv/tzMDpvZk2b2RTMbTjZMEREJ\nIkhL/VPAm1f87mHg9e5+KfAd4H1xByYiIuF1Teru/hjw/IrfTbv7icbdx4GzE4hNRERCiqOm/k7g\nH2J4HhER6VFPSd3MPgAcd/ddMcUjIiI9WBf1gWb2DuB3gCu6bKfFZUREInB3C/uYoC11a9zqd8yu\nAf4b8HvufixAYIW9bd++PfMY1mr8RY5d8Wd/K3r8UQUZ0rgL+D/A+Wb2tJndAPw1MAQ8YmZPmNk9\nkSMQEZHYdC2/uPtbW/z6UwnEIiIiPdKM0i6q1WrWIfSkyPEXOXZQ/FkrevxRWS+1m0A7MPOk9yEi\nUjZmhifYUSoiIgWgpC4iUiJK6iIiJaKkLiJSIkrqIiIloqQuIlIiSuoikrharca+ffuo1WpZh1J6\nSuoikqjdu/cwOnoBV131LkZHL2D37j1Zh1RqmnwkIomp1WqMjl7A4uJe4GLgIJXK5czPH2FkZCTr\n8HJNk49EJHfm5ubo7x+jntABLqavb5S5ubnsgio5JXURSczY2BgvvzwHLF23/iDHj88zNjaWXVAl\np6QuIokZGRlhcvIeKpXLGR7eQqVyOZOT9yyXXtSBGj/V1EUkcbVajbm5OcbGxpYT+u7de5iY2EZ/\nf701Pzl5D+Pj12ccaX5ErakrqYusEa0Sa5axqAO1M3WUikhbeRtWqA7U5KilLlJyeWwV5zGmvFFL\nXSRmZenEy2OruFsHqkSnlrpIC2XqxMtzqzhPdf68UUepSEzynASjWjpJ9fWNcvz4fKFPUmtF1KS+\nLolgRIpsqVyxuLi6XFHUpD4+fj1XXnmFWsVrgJK6yAqnzoKst9TLMAtyZGREyXwNUEepyArqxJMi\nU01dpA114kmW1FEqIlIiGqcuIrlUlvH+RaGkLiKhBU3UzcsTnHPO+Xzwgx9Wck9Y16RuZpNm9pyZ\nHWz63Rlm9rCZfcvMpsxsY7JhikheBF1HplarMTGxjcXFvRw9+se89JLxJ3/yqVysPVNmXWvqZnYZ\n8ALwGXe/uPG7u4Cfuvufm9ntwBnufkebx6umLhJCnjtow0zM2rdvH1dd9S6OHv1H4AKgPJO50pBY\nTd3dHwOeX/HrtwCfbvz8aeDasDsWkdW6tYKzrk+HWUfm5Hj/R4Bgj5HeRa2pv8LdnwNw92eBV8QX\nksjadGq5Yj+Li3uZmNi2nMDzkPDDXJ5uabz/4OC7gSOBHiMxcPeuN2AUONh0//+u+P+fdnisi0h3\ns7OzvnHjFgdfvg0Pb/bZ2VlfWFjwSuVMh6ca//eUVypn+sLCgru779r1oFcqZ/rGjVu8UjnTd+16\nMLE4l/Y1PLw50L4WFhb8T//0Q6EeI+6N3BkoRzffAo1TN7NR4G/9ZE39MFB19+fM7FXAXne/sM1j\nffv27cv3q9Uq1Wo1+llIUpPn2m4ZdapXz83NNerT+5e3Hx7ewvT0vYyNjXWtc8f9twz6fM3bAase\no/fYSTMzM8zMzCzfv/POOyPV1IO21MeAbzTdvwu4vfHz7cBHOjw2yZOZJCTNlp+c1K4V3Kml3qmF\n3/ycaf8tu+1X77HOiNhSD5LQdwE/Ao4BTwM3AGcA08C3gIeBTR0en84RkNh0+6ov0S0l4U7Hst02\nURJ+Vn/LbvvVe6y7xJJ6rzcl9eLp1vKTaOJomYZN+Fn9LbvtV++x7pTUJTZqRcUvjWPaKuEvLCx4\nf//GU/bb379RLfUCiJrUtUyArBJl6dmsx0/nXZzXCW13rEdGRti6deuqv5P7L4AqsAWoNu4nq9t7\nSMsbJyjKmSDMDbXUCytI/dddHV5BxNUyDXusT5Y5FhxmHRZSLXN0ew+1+3YR5H0XdZ9FgcovkgV9\njQ4u7PjulaIc66L9fXptIJSpgaGkLplQh1c4vbQip6amfP36S0If615PJmnp9QRUtBNYN1GTuq5R\nKj0p6/U8kxL1OqG7d+/hne98Fy+99DJhj3VRLjrd6wW/y3jB8CjUUSo9UYdXcpY6RA8fPszExDZe\neul91D+ybwLOA97Ez3/+MtPTj3Z9rnadqHkSZl2ZJB5fGlGa92FuqPyyJpSlcyovmmvDAwObfHDw\nHIel0sKCw+ccNjnsLXSJYaVeS0VFKTUFQZJrv/RC66lLnhRhrZFWa8DArwG/AjzVtOUW4F6Gh29i\nevpetm7d2vE58/66l/Qaa5Feaye6RqlIF0Gv2JO1ubk51q07i/rKHDXgYgYHXwt8h+bSAswDL3Yt\nMRTldS/ptVRUhFJToqI078PcUPlFcqBIIyN27rzPoeJwSaPkcpdXKmf63Xf/hQ8MbPL+/gsdKj44\nONa1xFCk1x3GWij3oRmlIu3FOaOzWdwzaWu1GrfddgfwOPAk9UvA7eCjH/0IZ511FmbQ13eCgYE+\nPvCBG5mfP8L4+PVt40nqdcchyrGr1Wp88IMf5pxzzi/MN4/URTkThLmhlrrkQBIt1iQmurQa979h\nw6U+NTXVNf5W8eS1pR7l2C09Bs5zOMPhwdy8niSgyUcincU5MiKpZNnueaempjpO8uoUT95GhJyM\ndW9j6YLOI3gWFhZ8amrKBwc3nfL66qWpdJc9SJOSukgAcdViT21R19dVGRp6Q9vkEma/rZJwt5NI\nt5m9eapBz87OeqVybiMpb3E40wcHx1oeu6VjUZ9Je3qjdb50zM93+IRa6krqIr07mWTvaiSnSxwq\nvnPnfau2jVJqaJWEO7W481pmaeXQoUONjuDmVnfFDx06dMp2rV5TvexyX+Pf1zlU/Oabb8nolSRL\nSV1KLU8tzSUnR6m0T6RxJNvm197pOOStzNLO1NSUDwy8/pRvFZXK6m85rb4NwS83Wuz5P3n1Skld\ncq2XpBykpZtF0p+dnfUNGza3LXksbdPLgmdhW/l5PPk127XrwUZtvHtibvdtqK/vnMjHs0iU1CW3\nehklEqSlm+Zyqytbze06NeO4wk+WJZUkTg6nvp4Hl0sorf5mS/u/++6/aFmqqXeyqqXe6qakLonq\nNTEF6QBMK/G1Onk0lzz6+zd6X9/QqpNL1LJIVssaJ3WSXP16Fnz9+vN9amqq7f4HBjb5wMDYqlLN\nwMBw7stMvVJSl1zqNTEdOnTIBwaG27bM0kp8nU4e7YbcrbwmZ9iWbxYt9ST3GeS5W3eOrm6ZHzp0\nKNdlpjgoqUsu9ZIkllpslcpFDhUfGBjzgYFNp4wwSTrxLSXjbuPEkzq5pN35mfRJstvrabX/tdIy\nX0lJXXIrSmJq12Jbv/7CVc+RVOJrLgMMDm7y/v6NHVvi9Zb657w+UiPeFm5zqzTJztA0vh10ir/d\n/tdCy3wlJXXJtbCJqFWLDS72+rC21l/b4/zQt0oufX1DbU8eu3Y92Ej65zmc7n19Q4m0KNPoFM56\naGTW+88LJXUJLc/D31q31M9stIKT7TBcWFjwBx54wDdsuGhVGWJqamrVMQvauu31eKdZY8/6vZH1\n/vNASV1CKcJV15di3LDh0kZn2V2JlwR27ryvsc/NgfcZpA4dx/FOo1NYyTQ/lNQlsCJMKV9KLku1\n1KVkm8RX8pMnj4u81ZjooaE3dNxnt+MZ1/FO+u9WhBP9WqKkLoFlNf7ZPVhLsF1yOXTokD/wwAP+\n2GOPxdaaPDVRznp91uLJ47Jhw6X+wAMPdN1XpzpwnMc7ar2523FfS6WdolBSl8CyaqkHne7fKral\nlnp9db+KVyoXxdKaXL2+SPTj0i5ZxX28wybFIMc9rRO9vg0El0lSB24D/pn6BRM/B/S32CbxFy/h\npT3CIGhia3eRiJMTkOI9Ga2O6y6Him/YcGkiwyPTHtERphM3jaGMeS/75UnqSR14DfD9pUQO7AHe\n3mK7xF+8RJPm1+CgLcFWH/yBgU2Neves19ffjrc1uTLh7tx5XyLHJYuyQ5gWeNInnizLfkWUVVKf\nB84A1gF/C1zZYrvkX72kJkhiarVNmFZaqyRbf2z8LfUwr6uIwraOiz6xqUyyKr/cAvwMeA74bJtt\nkn7tkpIg9dBO24RpCa5MLkuPHRwca9TUO49ISUsRTga9tMDjfn1Jfhsowt8ijKhJ3eqPDc/MNgFf\nBP4jcBT4AvB5d9+1Yjvfvn378v1qtUq1Wo20T8lOrVZjdPQCFhf3Ur8y/UEqlcuZnz/CyMhIx232\n73+MF154gbGxMaB+hfuxsbHlx4WJYW5ujqGhoeXnC/sccdq9ew8TE9vo7x/j5ZfnmJy8h/Hx6zOL\np5OlYxfmmCX1+qLE0k2R/hbtzMzMMDMzs3z/zjvvxN0t9BNFORM0TgT/Abi/6f7bgI+32C7Rs5mk\nI0g9tNNiTGUb7VD2UkKRXl+RYg2DiC3103o4sTwNvNHMBs3MgN8CDvfwfJJjY2P1FlB9oBPAQY4f\nn19ufbfbZnHxexw79mWOHt3P4uJeJia2UavV0gw9EXNzc/T3j1H/RgJwMX19o8zNzWUXVIyK9PqK\nFGsaIid1d5+lXnI5ADwFGHBfTHFJzoyMjDA5eQ+VyuUMD2+hUrmcj370I8zNzS0n6ZXbDAz8JpXK\nq4Bq41nK82ELcpIrsiK9viLFmooozfswN1R+KZWVa6S0Kqs0T/GP+rU4606vMDNfy7qaYJFeX5Fi\nDQrNKJW0RBmeODRUr603X+CinaxnHYbZf9Ynn6QV6fUVKdYglNQlNWEnkezceV9jAlGwoYxZdnpl\nvf8klS3plV3UpN5LR6msUWFqmLVajdtuu4Njx77Cz372RNfO0k6dXrVajX379iXa0VrWTrfdu/cw\nOnoBV131LkZHL2D37j1ZhyQJUVKX0Fp1mk5O3tNyzHHYJNnuhPHEE0+mkpTK2OlWq9WYmNjG4uLe\nQoxCSuPkXWpRmvdhbqj8UlpBlwwIW85ov0xAOiWRsnW6Zb3mSpiyT9b9KXmCauqSV1EvPL2UCLJI\nSmWqP2fZTxC207ms/RlRKKlLrvWSJPVh710W3z7C/t1mZ2ebVuNM/lq0eRc1qa/LrO4jhdLreh0j\nIyOR1/lYquFPTFxOX98ox4/Pt63hS2vj49dz5ZVXxL7mSidL/SmLi6v7U1rt/4knnuRnP/sucCPw\nDHB74fszshB5Qa/AOzDzpPexliWxONLK552efjQXiyUl9VqLLq/HJcgicJ22hTeyc+fHuOmmG9MP\nPgfMDE9zQa+gN1R+SUxSnUorn7evb0ilj5zKe8di0LJPuyterdXSi7tq6mtOUnXmVs8Lpy/XONd6\nnTNPitLXkNQoqbKLmtQ1Tr2gkpok0+p54dXAI4372Y3b1vjlUxVlotTIyAhbt27tWBoKM/dBOlNS\nL6h2k2Sef/75npJeq+ft7/8Jg4PvzvTD1m1G5FpM+GWbKDU+fj3z80eYnr6X+fkjhbvIRW5Ead6H\nuaHyS2Ka65X9/Ru9r28oltpqqzpoluO2u301z3tdOS6t/gZlmyglJ6Gaejl1S6YLCws+NTXlg4Ob\nYq1Hdtpv2gm+0+SjtVKL7XTiKtNEKTlJSb2EgrZA05xxmUWruFPiznoKfBrWyolLTqWkXjJhPshp\nfejzMN18ZZmhaAkvSqt6LZy4ZDUl9ZIJ+0FOo7aadXJplxCLUleO+i2naCcuiYeSeslE+SC3S3px\n1VzznFzyXlfu9dgV5cQl8VFSL6Gbb77FoeLwrx0qfvPNt4R+jrhr4Eou0cTxLSfvJy6JV9SkrrVf\ncurkWhhfBNYDL1Kp/PuW62Z0f47ua2+EjS2Pa43kWVJ/CymvqGu/aJXGnDq5wl11+XedVrjr/BzB\nVskLqpcVF9cqrTQpaVFLPafiaNmpdZg/+pYjQUVtqWuZgJyKYy0MraeRP0HWQRHphVrqOdepZRek\n1Ver1Thw4AAAmzdvVjIRKQi11EuqXcuu2wJXzdv8/u+/j2uvHWd6+tHE4lyLC2qJ5JFa6gUUpFae\nZj199+49ubgykkiZqKW+hgRZRzuttbZrtRoTE9tYXNzL0aP7WVzcy8TENrXYRTLSU1I3s41m9nkz\nO2xm3zSzX48rsLILW65o3j7IOtpprbVdlAs1iKwVvbbUPwb8vbtfCFwCHO49pPJrVQ/vlORXbj89\n/WjXUS1pjXwp24UaRIouck3dzIaBA+7+ui7bqabepFWtu6/vMtat62tZk25XG9+//zGeeeYZoPOo\nljTGRS/V1Jsn1aimLtKbqDX1XpL6JcB9wCHqrfSvA7e6++KK7ZTUm+zbt4+rrnoXR4/ub/ymBowB\nX6NVh+bq7WFw8Fzcn2dw8NzcdExqUo1IvLJYJmAdsAV4t7t/3cz+ErgD2L5ywx07diz/XK1WqVar\nPey22E4tV1xM/YLOr6FVTXpkZKTF9jO89NKzwOMcO1Y/CUxMXM6VV16RaTLV0gEivZmZmWFmZqbn\n5+mlpf5K4Gvufm7j/mXA7e7+71Zsp5b6Cs3lipdf/hdOnHBefvmrtBt62Lz9sWPf47TTRllcPLj8\nfMPDW5ievpetW7dm84JEJHapl18aO/0KcKO7f9vMtgOnu/vtK7ZRUm+huVwxPf1o15r00vZDQ0P8\n6q9epvVcREouq6R+CfA3QB/wfeAGdz+6Yhsl9QDC1KSDdEyqxi1SbJkk9UA7UFJPRKekXcYZnjpJ\nyVqjpF5yQZNaGZfbLeNJSqQbLRNQYkEW71pSthmeWoZAJBwl9ZwLm9TKNsOzbCcpkaQpqedc2KRW\ntgtjlO0kJZI01dRzLmqNvEwdi1qGQNYidZQWUNArF9177/186EN309//2jWb1Mp0khIJQkm9YIKM\n6Fi5zfvf/15uuulGJTWRNUBJvUDyduUiEckfDWkskDxduUhEykVJPQN5unKRiJSLknoGggw7LNvQ\nRBFJh2rqGQo6+kWjPkTWHnWUFoAStIgEpY7SnAuzfouISFRqqadAwxNFJCy11HNMwxNFJC1K6inQ\n8EQRSYuSegrCDE+s1Wrs27dveWndlfdFRDpRTT1F3Ua/rFzrZWLiD5ic/B+64o/IGqQhjQXXqjMV\n3gj8PVBFnasia4s6SguuVWcqnA2sX76vzlUR6UZJPSdadabCD4AXl++rc1VEulFSz4nmzlQ4j3rJ\n5UbgOuA8rf0iIoGoph6jOJYBePjhh7nuult48cX/DYwANdavv4wvfemvufrqq2ONV0TySzX1jMW1\nDMDmzZs5caIG/Ljxmx9z4sRP2Lx5c2yxikh5qaUeg7iXAdCFlkVELfUMtRq5srh4Bvfee3/L7btN\nKBofv575+SNMT9/L/PwRJXQRCUwt9Ri0HmNeZXDQefrpb5/SWg9ywWkRkcxa6mZ2mpk9YWYP9fpc\nRTUyMsL73/9e4E3AFuBy4BP097/2lHHltVqNiYltLC7u5ejR/Swu7mViYpuWABCR2MRRfrkVOBTD\n8xTaTTfdyOBgP/BHwBHgwlXjyrVao4gkraekbmZnA78D/E084RTXyMgIn/zkTiqV9zA8/OaW48q1\nWqOIJK2nmrqZfR74ELAReK+7/16LbUpfU28WdNEujWwRkU6i1tTX9bDD3wWec/cnzawKtN35jh07\nln+uVqtUq9Wou829kZGRjsMYx8ev58orr2Bubo6hoSFeeOEFarWaZoqKrHEzMzPMzMz0/DyRW+pm\n9mHgD4CfAxVgA/Ald3/7iu3WVEs9KI2CEZFOMl1618x+E5VfAtM1S0WkG00+KhCNghGRpMSS1N39\nK61a6dKaRsGISFLUUs9AmGuWioiEoWUCMhTHUr0iUk66RqmISImoozSHuq3GKCISNyX1hMR10QwR\nkTBUfkmAxqGLSK9UfskRjUMXkawoqSdA49BFJCtK6iEE7fjUOHQRyYpq6gFFWYBL49BFJCqNU0+Q\nOj5FJG3qKE2QOj5FpCiU1ANQx6eIFIWSegDq+BSRolBNPQR1fIpIWtRRKiJSIuooFRERJXURkTJR\nUhcRKREl9Qi0TrqI5JWSekhaJ11E8kyjX0LQcgEikhaNfknBgQMHOO20EeDVjd9ouQARyRcl9YB2\n797DtdeO8+KLvwB+BdiDlgsQkbxZl3UARVCr1ZiY2HZK2QXexOBgP5OTO1V6EZHcUFIPYGmVxsXF\nk6s0rl9/Hl/60t1cffXVmcYmItJM5ZcAWq3SeOLED9i8eXOGUYmIrKakHoBWaRSRoog8pNHMzgY+\nA7wSOAHc7+5/1WK7Ug1p1CqNIpKGqEMae6mp/xz4Q3d/0syGgP1m9rC7H+nhOUVEpAeRyy/u/qy7\nP9n4+QXgMHBWXIHljWaSikgRxDKj1MzGgBngDY0E3/x/hS+/aCapiKQtsxmljdLLF4BbVyb0stCF\np0WkKHoap25m66gn9M+6+5fbbbdjx47ln6vVKtVqtZfdpu7UIY31lrpmkopInGZmZpiZmen5eXoq\nv5jZZ4CfuPsfdtim8OUXqNfUJya20dc3yvHj80xO3sP4+PVZhyUiJZX6NUrN7DeArwLfALxxe7+7\n/+OK7UqR1EFDGkUkPbrwtIhIiWjpXRERUVIXESkTJfUe6XqlIpInSuo90CxTEckbdZRGpFmmIpIk\ndZSmTLNMRSSPlNQjanXhDM0yFZGsKal30KkTVBfOEJE8Uk29jaVlAfr76y3ydssCaJapiCRBM0pj\npE5QEcmaOkpjpE5QESkqJfUW1AkqIkWlpN6COkFFpKhUU+9AnaAikhXV1EVEREm9Ha3rIiJFpPJL\nCxrSKCJZU/klRhrSKCJFpaTegoY0ikhRKam3oCGNIlJUqql3oCGNIpIVrf0iIlIi6igVEREldRGR\nMlFSFxEpESV1EZESUVIXESkRJXURkRLpKamb2TVmdsTMvm1mt8cVlIiIRBM5qZvZacDHgTcDrwfG\nzeyCuALLi5mZmaxD6EmR4y9y7KD4s1b0+KPqpaX+a8B33H3e3Y8DDwJviSes/Cj6G6PI8Rc5dlD8\nWSt6/FH1ktTPAp5puv+Dxu9ERCQj6igVESmRyGu/mNkbgR3ufk3j/h2Au/tdK7bTwi8iIhGkuqCX\nmf0S8C3gt4AfA7PAuLsfjvSEIiLSs3VRH+juvzCzm4GHqZdxJpXQRUSylfjSuyIikp7YO0rN7Awz\ne9jMvmVmU2a2scU2Z5vZo2b2TTP7hpndEnccYQSZRGVmf2Vm3zGzJ83s0rRj7KRb/Gb2VjN7qnF7\nzMwuyiLOdoJOYjOzrWZ23MyuSzO+bgK+f6pmdsDM/tnM9qYdYycB3j/DZvZQ473/DTN7RwZhtmRm\nk2b2nJkd7LBNnj+7HeOP9Nl191hvwF3AHzd+vh34SIttXgVc2vh5iHpt/oK4YwkY72nAd4FRoA94\ncmUswG8D/7Px868Dj2cRaw/xvxHY2Pj5mqLF37Td/wL+Drgu67hDHv+NwDeBsxr3/1XWcYeM/33A\nny3FDvwUWJd17I14LgMuBQ62+f/cfnYDxh/6s5vEkMa3AJ9u/Pxp4NqVG7j7s+7+ZOPnF4DDZDfG\nPcgkqrcAnwFw938CNprZK9MNs62u8bv74+5+tHH3cfI1nyDoJLb3AF8AFtIMLoAg8b8V+KK7/xDA\n3X+ScoydBInfgQ2NnzcAP3X3n6cYY1vu/hjwfIdN8vzZ7Rp/lM9uEkn9Fe7+XCOgZ4FXdNrYzMao\nn6n+KYFYgggyiWrlNj9ssU1Wwk4C+y/APyQaUThd4zez1wDXuvsngNBDvBIW5PifD5xpZnvNbJ+Z\nvS216LoLEv/HgX9jZj8CngJuTSm2OOT5sxtWoM9upNEvZvYI0Hy2M+pn8//eYvO2PbFmNkS99XVr\no8UuCTKzy4EbqH/lK5K/pF7KW5K3xN7NOmALcAWwHviamX3N3b+bbViBvRk44O5XmNnrgEfM7GJ9\nZtMT5rMbKam7+1Uddv6cmb3S3Z8zs1fR5uuyma2jntA/6+5fjhJHTH4InNN0/+zG71Zu88tdtslK\nkPgxs4uB+4Br3L3T19W0BYn/3wIPmplRr+n+tpkdd/eHUoqxkyDx/wD4ibu/BLxkZl8FLqFey85a\nkPhvAP4MwN2/Z2b/AlwAfD2VCHuT589uIGE/u0mUXx4C3tH4+T8D7RL2J4FD7v6xBGIIYx9wnpmN\nmlk/8J+ov4ZmDwFvh+WZtP9vqcSUA13jN7NzgC8Cb3P372UQYydd43f3cxu311JvCGzLSUKHYO+f\nLwOXmdkvmdnp1Dvs8jKnI0j888CVAI169PnA91ONsjOj/be3PH92l7SNP9JnN4He3DOBaeojWh4G\nNjV+/2rg7xo//wbwC+o97QeAJ6ifhbLqgb6mEe93gDsav7sJ+K9N23ycesvqKWBLVrFGiR+4n/qI\nhScax3s265jDHv+mbT9Jjka/hHj//BH1ETAHgfdkHXPI98+rgalG7AepzxzPPO5GbLuAHwHHgKep\nf6so0me3Y/xRPruafCQiUiJapVFEpESU1EVESkRJXUSkRJTURURKREldRKRElNRFREpESV1EpESU\n1EVESuT/Aw8+Jy9UZqkVAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f85b7178710>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "X, y = make_data(120, error=1.0)\n",
    "plt.scatter(X, y);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.learning_curve import validation_curve"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def rms_error(model, X, y):\n",
    "    y_pred = model.predict(X)\n",
    "    return np.sqrt(np.mean((y - y_pred) ** 2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.pipeline import make_pipeline\n",
    "\n",
    "def PolynomialRegression(degree=2, **kwargs):\n",
    "    return make_pipeline(PolynomialFeatures(degree),\n",
    "                         LinearRegression(**kwargs))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "degree = np.arange(0, 18)\n",
    "val_train, val_test = validation_curve(PolynomialRegression(), X, y,\n",
    "                                       'polynomialfeatures__degree', degree, cv=7,\n",
    "                                       scoring=rms_error)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's plot the validation curves:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEKCAYAAAAW8vJGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4W9Wd//H30eJF8r7E2Rxn30lIAoSdQAoFylYKDEsI\n0M5Mh6VlKEMb2lJo+dFOO8x0ugMtS6ANpIRCoYXCEJrSliWkkD129sRx4n2RZcva7vn9ocWSLdvy\nKsn+vp5Hj3SvrqSvHed+dO659xyltUYIIYQIMSW6ACGEEMlFgkEIIUQUCQYhhBBRJBiEEEJEkWAQ\nQggRRYJBCCFEFEuiC4iXUkrOqxVCiAHQWqv+bJ9SLQatdVLdHnzwwYTXkCp1SU1S01ioKxlrGoiU\nCgYhhBDDT4JBCCFEFAmGQVixYkWiS4gpGeuSmuIjNcUvGetKxpoGQg30GNRIU0ppwzBQql99KEII\nMaYppdD97HxOmbOSADp8HWRaMxNdhhBDburUqRw5ciTRZYgUVlZWxuHDh4fkvVKqxdDQ3kBBZkGi\nSxFiyAW/1SW6DJHCevobGkiLIaX6GFxeV6JLEEKIUS+lgqHd257oEoQQYtRLqWBw+aTFIIQQwy2l\ngsHr9+I3/IkuQwjRT7fffjuPPPLIkG8rhkdKdT5vqdrCnKI5ZKVlJbocIYZUMnc+T5s2jSeffJIL\nLrgg0aWIXozZzmeQfgYhko3fL614GF2/h5QLBjkzSYiRs3r1ao4ePcrll19OTk4Ojz76KEeOHMFk\nMvHUU09RVlbGypUrAbjuuuuYMGEC+fn5rFixgt27d4ff57bbbuNb3/oWAH/5y18oLS3lf/7nfygp\nKWHSpEk888wzA9q2sbGRyy+/nNzcXJYvX84DDzzAOeecE/Nncbvd3HzzzRQVFZGfn8/y5cupq6sD\noKmpic9//vNMmjSJwsJCrr766vDrfvnLXzJr1iyKioq46qqrOHHiRPg5k8nEz3/+c2bPns3s2bMB\nKC8v56KLLqKwsJB58+bx4osvhrd//fXXWbBgATk5OeGfKxmlXjBIB7QQI+bZZ59lypQp/OEPf8Dh\ncPAf//Ef4efeffddysvLefPNNwG49NJLOXDgALW1tSxdupSbbrqpx/etrq6mtbWV48eP86tf/Yo7\n77yTlpaWfm97xx13kJ2dTW1tLc888wxr167tcXSEtWvX4nA4qKqqorGxkccee4zMzMAFs6tWrcLl\ncrFnzx5qa2u55557AHjnnXf4+te/zoYNGzhx4gRTpkzh+uuvj3rf3//+92zevJndu3fT3t7ORRdd\nxKpVq6ivr+eFF17gjjvuoLy8HIB//ud/5pe//CUOh4OdO3cm7+G5RA8J24+hY/WWqi36kxOfaCFG\nm8B/xd6eH5rbQEydOlVv3LgxvHz48GFtMpn04cOHe3xNU1OTVkpph8Ohtdb61ltv1Q888IDWWutN\nmzZpm82m/X5/ePtx48bpDz/8sF/b+v1+bbVa9b59+8LPffOb39TnnHNOzJqeeuopfdZZZ+nt27dH\nrT9x4oQ2m826paWl22u+8IUv6K997WvhZafTqa1Wqz5y5IjWWmullN60aVP4+fXr1+tzzz036j2+\n+MUv6u985ztaa63Lysr0E088Ef69DKWe/oaC6/u1v025FoPf8OP2uRNdhhAjaqiiYShNnjw5/Ngw\nDNasWcPMmTPJy8tj2rRpKKWor6+P+drCwkJMps7dj81mw+l09mvburo6/H5/VB2lpaU91rt69Wo+\n/elPc/311zN58mTWrFmD3++nsrKSgoICcnJyur3m+PHjlJWVhZftdjuFhYVUVVXF/D0cOXKEDz74\ngIKCAgoKCsjPz2fdunXU1NQA8NJLL/HHP/6RsrIyzj//fD744IMe602klAsGkMNJQoykng7NRK5f\nt24dr732Gu+88w7Nzc0cPnx4UBPFxKO4uBiLxcKxY8fC6yorK3vc3mw288ADD7Br1y7ee+89Xnvt\nNZ599llKS0tpbGzE4XB0e83EiROjxrBqa2ujoaEhKgwifw+lpaWsWLGCxsZGGhsbaWpqwuFw8NOf\n/hSAZcuW8corr1BXV8eVV17JddddN6jfwXBJzWCQDmghRsz48eM5ePBg1LquO/zW1lbS09PJz8+n\nra2N+++/f9hHQjaZTFx99dU89NBDuFwuysvLefbZZ3vcftOmTezcuRPDMMjKysJqtWI2mxk/fjyX\nXHIJd9xxB83Nzfh8Pv76178CcMMNN/D000+zfft23G43X//61zn99NN7bJlcdtll7N27l1//+tf4\nfD68Xi9btmyhvLwcr9fLunXrcDgcmM1msrOzMZvNw/K7GazUDAZpMQgxYtasWcPDDz9MQUFB+Cya\nrjv91atXM2XKFCZNmsTChQs588wz+/UZ/QmRyG1/8pOf0NzczIQJE7jlllu48cYbSU9Pj/m66upq\nrrnmGnJzc1mwYAHnn38+q1atAuC5557DYrEwd+5cSkpK+NGPfgTAypUrefjhh7n66quZNGkShw4d\n4oUXXuix7qysLN566y1eeOEFJk6cyMSJE1mzZg0ejyf8OdOmTSMvL48nnniCdevWxf1zj6SUu8AN\nIMOSwYJxCxJckRBDJ5kvcEsla9asoaamhqeffjrRpYy4MX2BG4Db75b/REIIKioq2LFjBwCbN2/m\nySefjLoGQQxMSk3UE6K1xuVzYbPaEl2KECKBWltbueGGGzhx4gQlJSXcd999XH755YkuK+Wl5KEk\ngKl5Uym0FSawIiGGjhxKEoM15g8lgXRACyHEcEndYJBTVoUQYlikbjBIi0EIIYZFygaD1+/FZ/gS\nXYYQQow6KRsMIIeThBBiOKRUMHi90ctyOEmI5BWaSyFk4cKFvPvuu3Ft218yHejQSqnrGNxusFo7\nl6XFIERyixwyYufOnXFv25u1a9fyq1/9KjyeEcAvfvGLgRUoYkqpFkN9XfQfjrQYhBh7tNbDPkDf\nSEnW6UBTKhg+2NoUtSwtBiGG1w9+8AOuvfbaqHV33303//7v/w7AM888w/z588nJyWHmzJk88cQT\nPb7XtGnTeOeddwDo6Ojg1ltvpaCggIULF/LRRx9Fbfv973+fmTNnkpOTw8KFC3nllVeAwLSZt99+\nO++//z7Z2dkUFBQA0dOBQt/TcT7++OPMnj2bgoIC7rrrrh5r/uijjzj11FPJzc1lwoQJUTPY/e1v\nf+Oss84iPz+fsrKy8MiuDoeD1atXM27cOKZNmxZ1iGvt2rWcffbZfOUrX6GoqIhvf/vbADz11FPM\nnz+fwsJCLrnkEo4ePRp+zT333ENJSQm5ubksXrw4asrUYdPfmX0SdQP0efc+prdUbYm6dXg74p/i\nSIgkxUCnVxtmR44c0Xa7XTudTq211n6/X0+YMEFv3rxZa63166+/rg8dOqS11vrdd9/VNptNf/JJ\nYJbFTZs26dLS0vB7Rc4E97WvfU2fe+65urm5WR87dkwvXLgwatsNGzbo6upqrbXWv/3tb7Xdbg8v\nP/PMM91maYuc9W3jxo26qKhIb926VXs8Hv2lL30palY1pZS+/PLLtcPh0EePHtXFxcX6zTffjPnz\nn3HGGfrXv/611lrrtra28Cxzhw8f1tnZ2Xr9+vXa5/PpxsZGvW3bNq211jfffLO+6qqrdFtbmz58\n+LCePXu2fuqpp8K1WywW/bOf/Uz7/X7d0dGhX3nlFT1r1ixdUVGh/X6/fuSRR/SZZ56ptdb6zTff\n1Kecckp4xrfy8vLw76Grnv6GGMAMbinVx3CgZQ8+/ylYIoYwd/lcpFtiD7MrxGihvj00h070g/0b\ndmPKlCksXbqUl19+mVWrVrFx40bsdjunnnoqAJdcckl423POOYeLLrqIv/71r5x88sm9vu+LL77I\nY489Rm5uLrm5uXz5y1/m4YcfDj//uc99Lvz42muv5bvf/S6bN2+OaxykdevW8YUvfIHFixcD8L3v\nfY/8/HyOHj3KlClTALj//vvJzs4mOzub888/n61bt3LRRRd1e6+0tDT2799PQ0MDhYWFnHbaaQA8\n//zzXHjhheGJdvLz88nPz8cwDNavX8/27dux2WyUlZVx77338txzz3HbbbcBMGnSJO644w4A0tPT\nefzxx7n//vuZPXs2EBgh9pFHHqGyshKr1Uprayu7d+/mtNNOY86cOX3+/EMhpYKhXu3E7QZLxNh5\nLq+LvIy8xBUlxAjo7w59KN1www08//zzrFq1iueff54bb7wx/Nwbb7zBd77zHfbu3YthGLhcLhYt\nWtTnex4/fjxqFrTI6TMBnn32WX74wx9y+PBhIDBzWk/ThMZ672XLloWXI6fjDAVDSUlJ+PnephV9\n8skneeCBB5g7dy7Tp0/nW9/6Fp/5zGeorKxkxowZ3bavr6/H5/OFPyf0s0VOBdr17KsjR45w9913\nc++99wKdfShVVVWcf/753HXXXdx5550cPXqUq6++mkcffZSsrKy4fhcDlVJ9DB35W2mO7maQDmgh\nhtm1117Lpk2bqKqq4uWXXw4Hg8fj4ZprruGrX/0qdXV1NDU1cckll8Q1GOCECROipuGMnD7z6NGj\n/Ou//is///nPaWpqoqmpiQULFoTft6+O53im44zXjBkzWLduHXV1dXz1q1/lmmuuweVyUVpayv79\n+7ttX1RUhNVqjfr8I0eOMGnSpPBy1/qnTJnC448/HjUdqNPp5PTTTwfgrrvuYsuWLezevZuKigr+\n67/+q98/R3+lVDCYsfDBNumAFmIkFRUVcd5553Hbbbcxffr08OEMj8eDx+OhqKgIk8nEG2+8wVtv\nvRXXe1533XV873vfo7m5mWPHjoXnRIbAjtxkMlFUVIRhGDz99NNRp7qWlJRw7NgxvF0vbArq73Sc\nvfnNb34Tbqnk5uailMJkMnHTTTexceNGNmzYgN/vp7GxkW3btmEymbjuuuv4xje+gdPp5MiRI/zw\nhz/k5ptv7vEzvvjFL/Ld73433Knc0tLChg0bANiyZQubN2/G5/ORmZlJRkYGJtPw77ZTKhhy3YvZ\n3CWl3X43hjYSVJEQY8ONN97Ixo0buemmm8LrsrKy+PGPf8y1115LQUEBL7zwAldeeWWP7xH5TfnB\nBx9kypQpTJs2jYsvvpjVq1eHn5s3bx733nsvp59+OuPHj2fXrl2cffbZ4ecvuOACFixYwPjx4xk3\nbly3z+nvdJy9tUD+9Kc/sWDBAnJycrjnnntYv3496enplJaW8vrrr/Poo49SUFDAkiVL2L59OwA/\n/vGPsdlsTJ8+nXPPPZdVq1aF+xdiueqqq1izZg3XX389eXl5LFq0iD/96U9A4Aynf/mXf6GgoIBp\n06ZRVFTEfffd1+N7DZWUmo/h5DX/TmOjZsO3bsYU8W85r3ieTNojUprMxyAGa8zOx7Bo0mzq1E7c\nHdHr5XCSEEIMnZQKhhULZ+DK+4RWR3QqSge0EEIMnZQKhumTCjHpND7cKR3QQggxXFIqGDIyINe1\niA8qojugpcUghBBDZ1iDQSk1WSn1jlJql1Jqh1Lqyz1s92Ol1D6l1FalVI+XTFrMUJoxn4rGciL7\nWGTSHiGEGDrD3WLwAV/RWi8AzgDuVErNjdxAKXUJMENrPQv4IvBYb2940sTZ1OqdeLrOzSCHk4QQ\nYkgM65AYWutqoDr42KmU2gNMAsojNrsSeDa4zYdKqVylVInWuibWe5530gx+U70Vp1OTXtB5BpbL\n5yI7PXu4fhQhhlVZWdmoGUpaJEbXYUUGY8TGSlJKTQVOBj7s8tQkoDJiuSq4LmYwzC4txGSk8dGu\nBi4+pyi8XloMIpWFxgQSIhmMSDAopbKADcDdWuvYo1XF4fH/fhzDAOvf7fyh9g0uPqfzMnPpgBZC\nCNi0aRObNm0a1HsM+5XPSikL8AfgDa31j2I8/xjwZ631+uByOXBe10NJSim9pWoLALf892/o8PhY\nf/8t4edNysSSCUuG7wcRQogUlKxXPj8F7I4VCkGvAqsBlFKnA8099S+EnDRhDtXGTiLH0DK0gdvn\nHpqKhRBiDBvWQ0lKqbOAm4AdSqlPAA18HSgjMKvQE1rr15VSlyql9gNtQM+jTQWdu3A6L9Rvo71d\nk5sb3QEtk/YIIcTgDPdZSX8HzHFs1/OkqzHMn16Aeiedj8vrOX95cXi9TNojhBCDl1JXPodkZEB2\n2yL+tvtA1HrpgBZCiMFLyWCwWGCSdQG768qj1sspq0IIMXgpGQwAC8bP4YRvFz5/5zqZtEcIIQYv\nZYPh3IUzcGYFOqBDtNZ0+Dp6eZUQQoi+pGwwnDQrD+VPZ/v+uqj1cjhJCCEGJ2WDITMTstoW8ddd\n0gEthBBDKWWDwWKBCZYF7KquiFovLQYhhBiclA0GgAXj5lLl3YURMaqHtBiEEGJwUjoYzl4Q6IDu\ncHUmg0zaI4QQg5PSwbBkXi740tl5qDZqvRxOEkKIgUvpYMjMBHvrYv6yQ+aAFkKIoZJSwWBS0eVa\nrTDevJAdJ/ZGrZcWgxBCDFxKBYPNauu2bl7xHI55dqGlA1oIIYZESgVDVlpWt3Vnz5tJq207He7O\nZJAWgxBCDFzKB8Oyk7LR3nR2H60Or5NJe4QQYuBSPhjsdrA5TubdHXIFtBBCDIWUCgazyUyGJSNq\nndUK400L2V4lHdBCCDEUUioYIHarYW7RHCo7dketa/e2j1RJQggxqoyKYDhz3kxaMrfj9kR0QMuh\nJCGEGJBREQynLgp0QO+tOhFe5/bJpD1CCDEQKRcM6ZZ0LCZL1LqcHE1myxI2bT8YtV4m7RFCiP5L\nuWCA7q0GqxVK1AK2HYvugJZ+BiGE6L9REQwAswvmccS1K2qdnJkkhBD9N2qC4cw5M2jJ2I7XJx3Q\nQggxGCkZDDarrduAeqctycLwZLKvurMDWloMQgjRfykZDEqpbgPq5eVrMppO5i/bO4fg9hk+vH7v\nSJcnhBApLSWDAbofTkqzwjgW8knlvqj1cjhJCCH6Z9QEA8DsgrkcaYu+AloOJwkhRP+kbDDY0+zd\n1i2fNZOmtO34DemAFkKIgUrZYLCYLN0G1Fu+NNABfaD2eHidtBiEEKJ/UjYYoPvhpKJig7TGJVFD\ncMvVz0II0T+jKhjSrFBsLOQfhzuvgDa0IeEghBD9MKqCAWBW/lwOOaUDWgghBiqlgyHWgHqnzZhJ\nY9p2DOmAFkKIAUnpYIDurYblS7PRbhsHG6QDWgghBqLXYFBKmZVSj45UMQPRNRjGT/RhqV/C33Z1\nXugmLQYhhIhfr8GgtfYDZ49QLQPSNRjS06DIOIkthzo7oGXSHiGEiJ+l7034RCn1KvAi0BZaqbX+\n3bBV1Q+hAfUid/wzc+dR7vhV1HYuryvmRXFCCCGixRMMGUADcEHEOg0kRTCEBtRzepzhdafNmMV7\ndTvQWqOUAgKHkyQYhBCib30Gg9b6tpEoZDCy0rKiguHUJXaMP9g43HicaYWTAOmAFkKIePV5VpJS\narJS6mWlVG3w9pJSavJIFBevrv0Mk6d4Mdcs4b1y6YAWQoj+iud01aeBV4GJwdtrwXVJo+shoox0\nKPSfxOYDnR3Q0mIQQoj4xBMMxVrrp7XWvuDtGaB4mOvql1gD6k3Pns/+lj3hZZm0Rwgh4hNPMDQo\npVYFr2kwK6VWEeiMTipdDyedMn0W9eZAB3SIHE4SQoi+xRMMnweuA6qBE8A1QNJ1SHcNhtOW2DHc\nNiqb5QpoIYTojz6vfAau1lpfobUu1lqP01pfpbU+OkL1xa1rMEyb6cFUvZT39lWE10mLQQgh+hbP\nlc83jFAtg9J1QL2MdCjwLuLDfRFnJkmLQQgh+hTPBW5/V0r9FFhP9JXPHw9bVQOUlZZFc0dzeHla\n9jz2Nf81vNzh64i66E0IIUR38QTDycH770Ss00RfCZ0UugbDsqmz+Ier8wpoQxu4/e5uZzAJIYTo\n1Fcfgwn4hdb6/C63pAsF6N7PcOrJdrTHRmVLVXidHE4SQoje9dXHYABfHaFaBi00oF7IjDluOL6M\nzREjrUoHtBBC9C6e01XfVkr9h1KqVClVELoNe2UDEBpQL8SWCXnuRXy4VzqghRAiXvH0MfxT8P7O\niHUamD705Qxe5IB6SsG0rHlUNPwl/LzT48TQRlTLQgghRKc+945a62kxbnGFglLqSaVUjVJqew/P\nn6eUalZKfRy8fbO/P0BXXfsZlpTNppad4SugfYaPRlfjYD9GCCFGrXhGV7Uppb6plHoiuDxLKXVZ\nnO//NPDpPrZ5V2u9NHj7f3G+b4+6Dqh36mI7hieTY47ODujattrBfowQQoxa8Y6u6gHODC5XAXHt\nwLXWfwOa+thsSC8q6Dqg3pwFLnTVKfzjSMQV0F4Xre7WofxYIYQYNeIJhhla6x8AXgCtdTtDuzM/\nQym1VSn1R6XU/KF4w8jDSXY75HYs4v2IDmiQVoMQQvQkns5nj1Iqk0CHM0qpGYB7iD7/H8AUrXW7\nUuoS4BVgdk8bP/TQQ+HHK1asYMWKFTG3y0rLor69nkC9MNU2n/L6v0Rt09zRjNvnJt2SPsgfQQgh\nksemTZvYtGnToN5DRQ5LHXMDpS4EvgnMB94CzgJu1VrH9clKqTLgNa31oji2PQQs01p36x1WSum+\nag3p8HWwq3ZXePmnPzX4tfki3v+3/4saDmOcfRyluaVxvacQQqQipRRa634d5YnnrKT/A64GbgWe\nB06JNxRCddHDoSelVEnE49MIBNWgTxnKsGREDah3yqIstNtGVWtV1HYNrgb8hn+wHyeEEKNKPIeS\n0Fo3AH/s75srpdYBK4BCpdRR4EEgLfCW+gngGqXU7QT6L1x0XjMxaJHjJs1b7MJ4bRlbj5UzeX7n\ndNV+w0+Dq4Fx9nFD9bFCCJHy4gqGgdJa39jH8z8DfjYcnx0ZDDk5muy2xbxfsZ/L5n8qarvatloJ\nBiGEiDBqL/+NvJ5BKZiSOZ/ddXu6bef2uWnpaBnJ0oQQIqnFc4HbDKVUevDxCqXUl5VSecNf2uDY\nrfaojubFk2dzwr+LWB3YcuqqEEJ0iqfF8BLgV0rNBJ4ASoF1w1rVEFBKYbd2thpOWZSF9nTvgAZw\nuB10+DpGsjwhhEha8QSDobX2AZ8FfqK1vg+YMLxlDY3IC90WLG7HqFrG9hPlMbeVVoMQQgTEEwxe\npdQNwC3AH4LrrMNX0tCJDIb8QgN7y8m8X7Ev5rYN7XLqqhBCQHzBcBtwBvCI1vqQUmoa8NzwljU0\nunZAT86Yz67a7h3QAIY2qGuvG6nShBCjiNYat2+oBoRIvD5PV9Va7wa+HLF8CPj+cBY1VEID6oX6\nDxZNms3Lvt3hOaC7qmuro8ReEvM5IYQA8Pq9uHwuXF5X+L7D14GhDYrtxZTmlKb8PqTPYAgOsf0w\nUBbcXhG4QC1nmGsbEllpWeFgWHZSFr/bZ+OYo4rS3MndtvX4PTR3NJOfmT/SZQohkoyhjaidf+je\nZ/h6fE1dWx0ev4fp+dNTejKweC5w+18CQ2LsiHuwoiQSOaDewpPbMd4+k037NnPzKd2DAQKd0BIM\nQowtHb6ObiEw0ENDLR0t7G3Yy8yCmVFD86SSeKquBHamYihAdD/DuPF+svbfwobd97Fq2WdjNvec\nHift3vaouaOFEKOLoQ1aOlpodDXicDswtDGk79/maaO8vpyZBTOj5odJFfEEw1eB15VSfyFiuG2t\n9f8MW1VDKDSgns/woRQszD2dHe0dbD66g+VlsQd8rW2rZWre1JEtVAgxrLTWONwOGl2NNHc0D3kY\ndOX2uamor2BGwYxuUw4nu3gOgj0CtAMZQHbELWVE/qNcdl0DbPk31u94scftG12NeP3ekShNCDGM\nQmFwpPkI22q2sb9xP42uxmEPhRCf4WNfw77wuG2pIp4Ww0St9cJhr2QYRQ6od8qZTnL+9yY+PPFd\nqh2NjM8p6La91pr69nomZKfEdXxCiC6cHidNriaaOpoS/iXP0AYHGg9QmluaMgN2xtNieF0pddGw\nVzKMIvsZbHa49DIftqNX8OK2V3t8TV17XcxxlYQQyand284xxzF21Oygor6C2rbahIdCpMqWSo45\njiW6jLjEM4NbK2An0L/gJUGnq/ZnBreutNZ8Uv1JeEe/t0LxL//mJe3mq/nj6pdJs5hjvm5q3lQK\nbYUDrlkIMbw6fB00uhppcjWlzHhn+Zn5TM2bOmKnsw75DG4qcNrOAq21SWudqbXO0Vpnp8o1DCFd\nB9QrmaC5cHkp/pYJ/F/533t8nYyfJETy8Rt+apw17K7bza7aXZxoPZEyoQDQ5GpiX8O+Xq+HSLRe\ngyH4Fb3fM7clo8gO6Jxs+PRVdXS8eycv7tpATw2Rdm87To9zhCoUQvTG6/dS5ahiR+0OjjmO4fK6\nEl3SgDk9TirqK5J2GI142jIfK6VOHfZKhlnXcZOmzfSxLPci9jVVUH6issfXSatBiMRy+9wcaT7C\njtodVDurR81glx2+DioaKmj3tie6lG7iCYblwPtKqQNKqe1KqR1Kqe3DXdhQ63oecV4+XPo5B2rb\nrbyw/aUeX9fc0YzH7xnu8oQQXbR52jjYdJCdtTupb68flSeDeP1eKuorkm4WyXg6n8tirddaHxmW\ninquY9AXX++q3RV1LPLECfjyHWmcuOQ8XrvhNfKzY1+hOD5rPJNyJg3qs4UQ8XG4HVQ7q2l1tya6\nlBGjlGJK7hSKbEXD8t5D2vkMgQCIdRt4mYnTtdWQXwCXX2zDdOI0Xt39Vo+vq2uvG7ELYoQYi7TW\nNLoa2V23m30N+8ZUKEDg5z/SfIQqR/cZJhMhdYf/G4DIfgaAjHQ4+8JmzB/fzku7X8Ljid0i8Rt+\nGtobRqJEIcYUQxvUttWys3Ynh5oOpXSH8lCodlZzqOlQwg+bjalgiDVeSXExXLp0CQ2tTj48vKvH\n10ontBBDx2f4ONF6gh01O6hsqZR+vAiNrkYaXIn9IjqmgiE0oF6krCy48IpGjM238/zWlzB6COoO\nXwcOt2MEqhRi9PL4PVS2VLKjZgfHW48n9bn8Y1lqDhY+CJHjJkHg1NXSMj9n5F3J+43f42hNM1PH\n58V8bW2Mf+wdAAAdNUlEQVRbLTnpKXVtnxDDzm/48Rm+Pm9+7afD15HwwySib2MuGOxp9m4jHebl\nwWVX+njv9St4ccer3Dd+dczXtnS04Pa5Sbekj0SpQgwprTWGNtAE73tYjrWut5297OhHnzEXDNlp\n3UcMN5tg/klupv7mNl4/eAv/6riJ3JzY4yfVttVSmls63GUK0Su/4cdrePH6vT3e+7U/vHPXaNmB\ni7iNuWCwp9nJsGR0G1slvwCuXjmJHx4rZuO+97l62dkxX9/gamBi9kTMptjBIcRAaa3xGb4+d/he\nwys7eTGsxlwwABTZiroNf5ueBsvPa8X+jS+yvmQdly48m4wYR4z8hp8GV0PKjKsuRl5oBx861BJ+\nHDwWH2udz/DJtTIiaYzJYCi0FVLVWtXtW1dhAVy+6Dyed32dPZXHWDJzcszX17bVSjCMUV6/F7ff\njcfvwe0L3vvdUTt62cGLVDcmg8FispCXkUeTqylqfVYWfOrSNl748S38ZtwrnDTtLmJN1eD2uWnu\naCYvI/bZSyJ1+Qxf1A6/awDIIRwxFozJYAAothV3CwaACRM1Z+ddy9/qP0NN/T8zqST2+Em1bbUS\nDClKa43T46Td294tAOTbvhBjOBiy07NJt6R3Gw89Nxc+e7mNd/94Gr/fsZHbx30GFWP4qVZ3Ky6v\ni0xr5ghVLAZKa027t51WTyut7lacHqcEgBC9GFNXPndVbCvuts5sgumzvMxwrub3e3+Ho5exvPY3\n7qfR1TiMFYqBcnld1LbVcqDxANtqtlFeX06VowqH2yGhIEQfxnQwFNoKUTGaAwUFcN15i2jqaOKD\n/T2Pn+TxezjUdIiK+uScbGMs8fg91LfXc6jpENtrtrO7bjeVLZU0dzSPmoldhBgpY/ZQEvTcCZ1m\nhaWndZD3539mXfarnDNvAbZejhg5PU721O2h0FbIpOxJWM3WYa5ceP3e8KGhVk9r0k6RKEQqGtPB\nAD13QhcWwRXzLuZZz39z+MTtzJ/ed0dzQ3sDzR3NjM8aT4m9JGZrZKzRWuM1vN2GYIi8Grev+8jX\n+LUfp8c55odnFmI4jflg6KkT2m6Diy4ys+7xy3j+ozd5oPSfSIujIeA3/FQ5qqhvr2dyzuRRe+ZS\nPEMyeA2vHMYRIgWN+WCAwJXQsWZOGjdOc1budfy59m7uaLyWCSXxd8m4fW4ONB4gJz2HyTmTU+7s\nJZfXRZu3LeYOX67SFWJ0k2AgEAzHW493u3gpJxeuu3gKm97M580dH3Fz8XLM/eyud7gd7KnfQ5Gt\niInZE7vNB5EsvH4vDrcDh9tBq6cVr9+b6JKEEAmSnHupEdZTJ7RJQdk0P7PaV7Nh18tcsWw5Bfn9\nf3+tNXVtdTS5mpiQPYFiW3HC+x8MbYQ7bh1uhxyzF0KEjenTVSPFuqYBID8PbjrrbGrUJ+w6cmJQ\nn+EzfFS2VLKnfk9CZoNr97ZT7axmb8NetlZvZX/jfmqcNRIKQogo0mII6qkT2mqFxUsU+X+9nrXv\n/4GTZ/wL2d2ndOgXl9fFvoZ95GXkMTln8rBN/OPxezoPD7lbZRpFIURcJBgi9NQJXVAAV8y+imfb\nb+BE3Wqys4dmR97c0UyLu4USe0nUlKGa6L6Orn0fvT2vCYwD5HA75Nx+IcSASDBE6KkT2pYJl6zI\nZ93ak1n//l+5b+KnyIg9tl6/aa2pdlZT7awemjcUQohBkj6GCKFO6FgKi+CMnOt5+8QGGrtfDyeE\nEKOGBEMXRbaimOtzc2DVpxbTrmr5y/b9+ORwvRAigs/wjZr5OuRQUhc56TkxO6GVgsmTFDPbVvHb\n7a/yqaVfoTj2iUxCiFHC4/fQ6GqkwdVAY3vw3tX9vtHViNPjZEruFFZOW8nKaSuZVTAr4aelD5RK\nlYRTSumRqrXaWR2zE9rngzffaeXBis/yk7NeYfmSLEyp+e8uxJjn9rkpbyjnUNMh6tvrwzv4Rlcj\n9a7Acoevg4LMgvCtMLOQwszCqOWCzAIKbYVkp2Wzp34PGw9tZOPBjZhN5nBIzC2a26+QKMsr6/Ho\nRX8ppdBa92tPJcEQg8/wsb1me8xmYdVxuPVX/8nM3Ll8+8arKC4i5kQ+QojkobXmmOMYO2p3sLN2\nJztrd3Kw+SBT86Yyq2AWRbai6B198D4nPWdA3/q11pTXlwdC4tBGDG1wwbQLWDltJQuKF/T5nhIM\ncRrJYAA42HQw5qirLhf88qW9/Kb6AX518fNkZpooLg70QQghkkOru5VddbuigiDDksHCcQs5adxJ\nLBy3kLlFc8mwDNHphb3QWrOvcR8bD23k7YNv0+HrCLQkpq/kpHEnYVLdu3olGOI00sHgcDvY17Av\n5nMHD2puevHzXFlyD1cuXwRAegaMK4asrBErUQhBoIW/v3F/OAB21u2ktq2WuUVzWVjcGQTF9sR3\nCmqtOdh0MNyScLgd4ZbE4pLFmE1mQIIhbiMdDAA7a3fGvEisxQEPrX2Lf7T8H9+/6P+RF3HBW6YN\nxo2j14l9hBADo7WmqrWKioaKcBBUNFRQYi9h4biF4RbB9PzpSTtgZaTDzYfDIdHQ3sD5U89n5bSV\nXDXvKsZnjR+Sz0i6YFBKPQlcBtRorRf1sM2PgUuANuBWrfXWHrYb8WDoqRNaa9i6o4Ovvf5dmjI/\nYq73Rr5wzmcoySkIb5OVBcXjIGN4RrsQYtTzGT4ONh1kb8NeKhoqqKivYG/jXuxWO7MLZ7OgeAEL\nxy1kQfECstMHOU5NEqhsqWTjoY28c+gdPIaHvXftHZKzmpIxGM4GnMCzsYJBKXUJcJfW+jNKqeXA\nj7TWp/fwXiMeDF6/lx21O2J2QtfXQ10dbDtwnOe2vExN4ctM81/ELadew4yC6eHtcnKhuJi4JvkR\nYqxq87Sxr3FfVAAcajrEhOwJzCmcw5zCOcwunM2cwjnkZw5giOMUk5uRy8yCmUPyXkkXDABKqTLg\ntR6C4THgz1rr9cHlPcAKrXVNjG1HPBig505onx/274fQfDUVB1w887c3qBr/FCWW2dyw8DqWjFuO\nUgqlIC8fiorAYh7hH0CIJNPQ3kBFQ0VUS6C2vZbp+dPDITCncA4zC2am3ARXQ2XU9zH0EQyvAd/T\nWr8XXH4b+KrW+uMY2yYkGHrrhK5vCLQcIiczO3jAzNNv/52jJY+Rnefhs7Ou59wJF2M1paFMUFgA\nBYX0e8IfIZKF1poOXwdOj5M2b1v0vaeNNm9b+N7pcUY9f7z1OB6/h9kFs5lT1BkCZXllKdEnMFIS\nHQwp9S/x0EMPhR+vWLGCFStWDPtn9nQlNEBRIeTlQkMjNDUFAmL6DD8PzzidQ3tXsPaP+/h11S9Y\nX/oLPl36OS6afA26Po+mZigshIJ8uQZCDD1DG7R72+nwdYRvbr87cO9z97oca3uXzxW1w2/3tmM1\nW7Fb7djT7GRZs8hKy8KeZu9cl5ZFfkY+k3MmY7cGlrPSsiixlzA+a3zKXhGcCjZt2sSmTZsG9R6J\nbjF0PZRUDpyXTIeSoOdO6Eg+PzQ1QmMTGP7O9QfLbaz7XQuHi55Az32JM8dfyCWTb2CibSpWa+Dw\nUm6uBIQIfBN3+920ebp/C3d6nd3WRz4Xfuxx4vK5yLBkkGnJJMOSQbolnQxz8N6SQbo5vXN9L8uh\nW7o5vduOX77dD69EtxhGIhimEgiGk2I8dylwZ7Dz+XTgf5Op8zmkt07orvxGoPXQ2Aj+iIH29u+2\ns/55M0cK1qKXPcacgvlcOvkm5uUuIz1DkZsLdhtkZEhIJCutNX7tx2f4om5evxeXz0W7tz18C33L\n7rbe66Ld1067p512X3A54nmzyRzYCQe/ZYfv03pfjrxlWjLD58OL1DSqg0EptQ5YARQCNcCDQBqg\ntdZPBLf5KXAxgdNVb4vVvxDcLmHBAD13QvfE0NDcDA31RI3EunennQ3P5XMs5yXM5/wPudkWLpl0\nE8uLLsRismIyB66BsNk7g0IMnM/wUd9eT01bDTXOGmraaqh2VlPjrMHhcUTv5P0Rj3Vghx/e+RuB\nxyZlwmKyYDVZsZgs4VumNROb1YbNYiPTmondau9cF7xlWjKxp9kD9zGet1lt8k1cAKM8GIZSooOh\nt07o3mgNLS2Bjmqvp3P9nm1ZbFg7nlrbn7Ff+F80p+9iacG5nFa0koV5y7GYAue3msyBgLDZA4Eh\nQdFJa01zR3N4Z1/trA4EQMTOv8HVQF5GXvjYdom9hJKsEkrsJeSm52I1R+/gQ7euO/7QdrGGLxBi\nqEkwxCnRwQA9XwkdD63B0Ro4i8nj7ly3e2s2v1s7gePNdZSsfJ72aS/TYt7H0sJzOK3oUyzMW47V\nlBZ+H7Ols0VhsyXnBXQ+w0eruzUw17SnlVZ3K26/G7/24zcCh2IiH3dd7vo4vI3hx+13U9tWG/72\nn2HJiNrZj88aH/W42FaM1SwXkYjUIsEQp2QIhng6oePhaA0cYuro6FxXeyKNTz7I5ZP389hf2Uzh\niufxzf4drel7WFp4diAk8k8nzRSdBGZLICBCrYr0NIaEoQ0aXY2BHbzHEd7RO9yO8JzSrZ7gzt/d\n2vnY04rb5yYrLYvs9Gxy0nLISs8i3Zwe/vZtVmbMJnO3x6Fli8mC2WSO+TjNnMY4+7jwt/+xep67\nGN0kGOKUDMHQn07oeDidgUNMrvbo9e1OEzu25AaCYlcb6Us2YDrpRdpsu1lSeBanFa9kUf4Z3UIC\nAkFhUkDwz0Cpzs5sRUTHdvjPRFPvrqbSeYDK1gNUth3kaOtBjrUdIsOcSZY1J3jLxm7NDj+OXJed\nFrEuLZtMiz18YV9/9Gf7WP8EMf9ZetlOmQIXHJrNgd+bOfg4tE5OAhCJIsEQp2QIBoADjQdo7mge\n0vdsaw+cyeR0Rl8sB+D3w/7dWXz8Xi5btnXQOvFV0pf+FlfOdk4uOJPl41ayOP9M0sy9dz5orWny\n1HKs/SBV7Qc41n6QY20HOO46RIbZxmTbDCbZpofvJ9mmY7OM7aFi+wqOyHWRIRIZxF3XEWu7yHXx\n1CWBNepJMMQpWYJhoJ3Q8fAb0NoKDge0t8X+BnziWDqfvJ/LRx97OGx9k4xl6/EUfMLC3DM5e+IF\nLMo/C5ffSVVwxx8Kgqr2g1hMacGdfygAZjDZPh27RSaTGK26hYjq/fluy/G+by8bdw1NkwqcVBF5\nr0yB0QCUqZf1kbdRHo4SDHFKlmCAwXVCx8vnDwSEoyUwOVAsToeZ7R/l8OEWLzvaNmJZtAF3yd+x\naBsFxmzy9WwKmUmRaRbF5hnkWAowWzRmi8YSvDebo5e7rlcKTKbAvTLp8H9W+dYqEq23Vli3x728\nLrLlpiK3UT0cilXdXxNabzJ1tiLNpohWpal/hyclGOKUTMEwVJ3Q8fJ4gyHhAHdH7G18PqjYkc22\nj9NwOzMxfCZ8foXfp/D5TPh9oceBe7+/y7JPBbb3qvDr/D6FBrSh0DpwbwTvIRAUiq6BoYPf+nQ4\nQELbdRXPt06ldLenQ59jMnV+tsmsw/+JQ49Nps5aQttGPp+eYZCT5yUnz0d2ro/sPF9gOfQ414fF\nmhx/c2J0iDw8aYo4NBl6HAqROePLmJArwdCnZAqGoe6E7o8Od2dLwusd8Y8PCwRF4EI+dCgwQGsV\nXK+Cy2AY3f8m4+kojvW3rHXne4Y+zzACy0bk5xsqXF/XbXVw2w6XmdYWC45mC45mK60tFlqbg8st\nVpwtFtIz/MHACIRHTq63czkvuJzrw2wJhk84GIPhZe4hLFX09qEgI+obqI76Ftv1G6wYvU6dXcac\nUhlEL6VYzVZy03OHvBM6HhnpkFEcmEa03RUICEdr9PAbI0EpUGYIXO4VuUdPjvAeCoYB7c7O8Ght\nsQZDxEJddRoHyu20NltodVjw+2OETy9hGdn60pqoxzoYtlpH/DZ196BUqjNIAstEhUmslpUyB1tY\nJt2vlheh90NHH2Yh8NqugWaK2kaHH1vTDDLtfjJtBja7P/jY3/k4Ytlm95OeaWCSawpHnATDABXb\nixMSDJFsmYFbSQm0tQWmHG1t7X5mkxgYkwmycvxk5fiZUDq8fUr90RkencHRLUx6aVlFhlTM5yNe\nF34+4jMM3fn+oLrVEw64cB0KguHn85pobzPjajPjajfR0mShujIdV7s5sL498FzosbvDREaG0S00\nMm1+TOaIUAsdMjR3CbvI++D68CFGU+dyrH62znsj9nproC+u894gPcPAmqZTvlUnwTBAOek5pJnT\n8Pg9fW88zJQKTCWalUX4P6RhEP7PGDrso7suE8dz4T1P8E5HLUavi2Pb/ujpSF14JxRRc2g5su7R\nGpA9H06K80KOFGL4weUy42ozBcOkMzQMf2d4GZrActdQ67I+Mvy8HhNaE+5v8/sVPm/EfXidqftz\nEfehvjqf14TbZUIDmfbOVk9mD/fdns/qXE406WMYBJfXRaOrkaaOpmE/S0kMTNewiAoUI9Bp7/MH\n732B60ai7kf4EJ1IfV6PCreK2p3mzscRN1eX+67blYzXVFUOzQi50vmcQC6vi+aOZpo7mmn3tvf9\nApESAt8oO8Oja3CEwiPqkA7dW09RzxFjXS/dNEn8Zy+GgdZw8tQyTpopnc8pL9OaSaY1kwnZE/D4\nPeGQcHqcCTl7SQwNpcBiCdxIkgELu/45df3rihVOkaEU9Xwfz0W9X281xBlsRjBojWDYGkYgWP1G\n5/qxTilIS/DfmgTDMAgN9DbOPg6f4aOlo4WmjiZa3a0Yo/XAtxgxfV6dnOIdn5Eh0VN4+I3OPqRu\ngRZcGU/ghTYP9cnJd7gACYZhZjFZKLQVUmgrxNAGLR0tNHc00+JuwS9fj4ToxhwcBoMEjZbe9YSN\nyJM54jmBw/BH32ujM8hC96HnkzWIJBhGkEmZyM/MJz8zH601rZ7W8CEnrz+BV6sJIcLCoxOPwPUT\nPYVJdvbwf3ZvpPM5SdS11VHVWiWtCCFEwsdKkmsKk0SxvZj5xfPJTk/wVwUhxJgnwZBE0sxpzC6c\nzZTcKZhNQ3MOsxBC9JcEQxKS1oMQIpEkGJKUtB6EEIkiwZDkpPUghBhpclZSCpEzl4QYnTIsGdjT\n7GSlZWG32sm0Zg7Ze8tYSWOAx+/hcPNhWt2tiS5FCDEAJmWKCgF7mh2LafguKZNgGEOk9SBEaki3\npIdDICstiwxLBmoEJ2yQYBhjpPUgRHIJtQZCITDcrYF4SDCMUYloPZiUKXxTSkUvE70c2gZAoQb1\nGECjMbSB1oF7QxsDWuczfDKoYR96+3eO9W/d19+CoQ18hg+f4cNreMOPfYYPrz+wPNT1W0wWrGZr\n4N5k7bYcOutPxRh9MNY3+962s5qsI9oaiIcEwxgWb+vBbDJjVmYsJgtmU/C+h+WuO4GuO/nRILSj\nCu2UIndWsdaN9N+gUiocjpH3PT0HnWEa+W9mVuboZZO51+dD2yRC199/rH8Xv/aH/06tZmvMHX7o\nb3qsk2AQNLoa8Rv+8H+KyJ2+WZlH1U49EXoKjcgddCg8e3sc2nHHehy5kxdisCQYhBBCRJFB9IQQ\nQgyaBIMQQogoEgxCCCGiSDAIIYSIIsEghBAiigSDEEKIKBIMQgghokgwCCGEiCLBIIQQIooEgxBC\niCgSDIOwadOmRJcQUzLWJTXFR2qKXzLWlYw1DYQEwyAk6x9BMtYlNcVHaopfMtaVjDUNhASDEEKI\nKBIMQgghoqTUsNuJrkEIIVLRqJ2PQQghxMiQQ0lCCCGiSDAIIYSIkhLBoJS6WClVrpTaq5T6WhLU\nM1kp9Y5SapdSaodS6suJrilEKWVSSn2slHo10bUAKKVylVIvKqX2BH9fy5OgpnuUUjuVUtuVUr9R\nSqUlqI4nlVI1SqntEevylVJvKaUqlFJvKqVyk6CmHwT//bYqpV5SSuUkuqaI5+5VShlKqYKRrKm3\nupRSXwr+vnYopf4z0TUppRYrpd5XSn2ilNqslDqlr/dJ+mBQSpmAnwKfBhYANyil5ia2KnzAV7TW\nC4AzgDuToKaQu4HdiS4iwo+A17XW84DFwJ5EFqOUmgh8CViqtV4EWIDrE1TO0wT+riOtAd7WWs8B\n3gHuT4Ka3gIWaK1PBvYlSU0opSYDFwJHRriekG51KaVWAJcDJ2mtTwIeTXRNwA+AB7XWS4AHgf/q\n602SPhiA04B9WusjWmsv8AJwZSIL0lpXa623Bh87CezsJiWyJgj/R7kU+FWiawEIfrM8R2v9NIDW\n2qe1diS4LAAzYFdKWQAbcDwRRWit/wY0dVl9JbA2+HgtcFWia9Jav621NoKLHwCTE11T0A+B+0ay\nlkg91HU78J9aa19wm/okqMkAQi3PPKCqr/dJhWCYBFRGLB8jCXbCIUqpqcDJwIeJrQTo/I+SLKea\nTQPqlVJPBw9vPaGUykxkQVrr48B/A0cJ/Adp1lq/nciauhinta6BwBcQYFyC6+nq88AbiS5CKXUF\nUKm13pHoWrqYDZyrlPpAKfXneA7bjIB7gEeVUkcJtB76bPGlQjAkLaVUFrABuDvYckhkLZ8BaoIt\nGRW8JZoFWAr8TGu9FGgncKgkYZRSeQS+lZcBE4EspdSNiaypD8kS8iilvgF4tdbrElxHJvB1AodF\nwqsTVE5XFiBfa3068FXgtwmuBwKtmLu11lMIhMRTfb0gFYKhCpgSsTyZOJpCwy14GGID8JzW+veJ\nrgc4C7hCKXUQeB44Xyn1bIJrOkbgW92W4PIGAkGRSJ8CDmqtG7XWfuB3wJkJrilSjVKqBEApNR6o\nTXA9ACilbiVwmDIZQnQGMBXYppQ6RGCf8A+lVDK0rioJ/E2htf4IMJRShYktiVu01q8Ea9pA4PB8\nr1IhGD4CZiqlyoJnj1wPJMMZN08Bu7XWP0p0IQBa669rradoracT+B29o7VeneCaaoBKpdTs4KqV\nJL5j/ChwulIqQymlgjUlskO8a+vuVeDW4ONbgER86YiqSSl1MYFDlFdord0JqCeqJq31Tq31eK31\ndK31NAJfQJZorRMRol3//V4BLgAI/t1btdYNCa6pSil1XrCmlcDePt9Ba530N+BioILAGRFrkqCe\nswA/sBX4BPgYuDjRdUXUdx7waqLrCNaymEC4byXwTSo3CWp6kEAYbCfQwWtNUB3rCHR8uwkE1m1A\nPvB28O/9LSAvCWraR+DMn4+Dt58nuqYuzx8ECpLk388CPAfsALYA5yVBTWcGa/kEeJ9AiPb6PjIk\nhhBCiCipcChJCCHECJJgEEIIEUWCQQghRBQJBiGEEFEkGIQQQkSRYBBCCBFFgkGICEqpB5VSX0l0\nHUIkkgSDEENMKWVOdA1CDIYEgxjzlFLfCE6M8y4wJ7huulLqDaXUR0qpv4SG9Qiuf18ptU0p9bBS\nqjW4/jyl1LtKqd8Du4LrblJKfRgcWfYXwSE4UEpdqJR6Tym1RSm1XillS8xPLkRsEgxiTFNKLQWu\nAxYBnwFODT71BHCX1vpUAuME/SK4/kfAD7XWiwmM0RM5dMAS4Eta67nBiZv+CThTB0aWNYCbggOq\nfRNYqbU+BfgHcO9w/oxC9Jcl0QUIkWDnAC/rwOBw7uA3/kwC48u8GPqWD1iD92fQOVHUOqJnw9qs\ntT4afLySwEiyHwXfIwOoAU4H5gN/D663Ehi/RoikIcEgRDRFoCXdFPym35Xusm2kti7PrdVafyPq\nzZW6DHhLa33TUBQrxHCQQ0lirHsXuEopla6UyiYwX28bcEgpdU1oI6XUouDDD4DQ+t7mit4IXKOU\nKg6+Pl8pNSX4+rOUUjOC621KqVlD+hMJMUgSDGJM01p/AqwnMAT3H4HNwaduAr6glNqqlNoJXBFc\nfw/wFaXUVgITxrT08L57CPQlvKWU2kZgCO3xOjAH8K3A88H17xHs8BYiWciw20L0g1IqU2vtCj7+\nJ+B6rfVnE1yWEENK+hiE6J9lSqmfEuhDaAI+n+B6hBhy0mIQQggRRfoYhBBCRJFgEEIIEUWCQQgh\nRBQJBiGEEFEkGIQQQkSRYBBCCBHl/wNPlKtdXzvROwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f85b709a470>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def plot_with_err(x, data, **kwargs):\n",
    "    mu, std = data.mean(1), data.std(1)\n",
    "    lines = plt.plot(x, mu, '-', **kwargs)\n",
    "    plt.fill_between(x, mu - std, mu + std, edgecolor='none',\n",
    "                     facecolor=lines[0].get_color(), alpha=0.2)\n",
    "\n",
    "plot_with_err(degree, val_train, label='training scores')\n",
    "plot_with_err(degree, val_test, label='validation scores')\n",
    "plt.xlabel('degree'); plt.ylabel('rms error')\n",
    "plt.legend();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice the trend here, which is common for this type of plot.\n",
    "\n",
    "1. For a small model complexity, the training error and validation error are very similar. This indicates that the model is **under-fitting** the data: it doesn't have enough complexity to represent the data. Another way of putting it is that this is a **high-bias** model.\n",
    "\n",
    "2. As the model complexity grows, the training and validation scores diverge. This indicates that the model is **over-fitting** the data: it has so much flexibility, that it fits the noise rather than the underlying trend. Another way of putting it is that this is a **high-variance** model.\n",
    "\n",
    "3. Note that the training score (nearly) always improves with model complexity. This is because a more complicated model can fit the noise better, so the model improves. The validation data generally has a sweet spot, which here is around 5 terms.\n",
    "\n",
    "Here's our best-fit model according to the cross-validation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEACAYAAABI5zaHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcVNWZ//HPoenuKuluFlMahdCNIkENiiiORkcbIhHN\naMxkoqLRiZJEo0ZHs7hlAr6IUTLjLyZjEMmAaCJC1IxLomFRWsUlEEXBgImoDYJKlwoYtJtu5Pn9\ncav32rdbVf19v1716lpO3ftUddVzTz333HOdmSEiIqWln98BiIhI9im5i4iUICV3EZESpOQuIlKC\nlNxFREqQkruISAlKOrk75+Y657Y659Z0ue9nzrn1zrmXnHMPOOdqchOmiIikIpWe+53AyT3uWwIc\namZjgdeAa7MVmIiIpC/p5G5mK4BtPe5bZmZ7IjefB4ZlMTYREUlTNmvuFwKPZXF5IiKSpqwkd+fc\n9UCbmS3IxvJERCQz/TNdgHPuG8CpwMQE7TSJjYhIGszMpfqcVHvuLnLxbjg3GfgBcLqZ7Ur0ZDMr\n2su0adN8j6Gvxl/MsSt+/y/FHn+6UhkKuQB4FhjlnNvknLsA+B+gCljqnHvROTcr7UhERCRrki7L\nmNk5Ue6+M4uxiIhIlugI1STV19f7HUJGijn+Yo4dFL/fij3+dLlMajoprcg5y9e6RERKhXMOy8MO\nVRERKQJK7iKSN+FwmFWrVhEOh/0OpeQpuYtIXtx77yJqa0czadLF1NaO5t57F/kdUklTzV1Eci4c\nDlNbO5rm5uXAYcAagsEJbNz4KqFQyO/wCppq7iJSsBobG6moqMNL7ACHUV5eS2NjI6ByTS4ouYtI\nztXV1dHa2gi0nw5iDW1tG6mrq1O5JkdUlhHpY8LhMI2NjdTV1eW1JHLvvYuYOvUSystraWvbyNy5\nszjppIkq1ySgsoyIJORnL3nKlLPYuPFVli27g40bX2XKlLMSlmskfeq5iyTgV0832wpxp2YhxlRo\n1HMXyYFSqgcXYi85FAoxd+4sgsEJ1NSMIxicwNy5s5TYs0A9d5EYSq1XWcivp1R+HeWCeu4iWVaI\nPd1MFHIvORQKMX78+IKIpVSo5y4SQyH3dDOhXnJxUc9dJMsKuaebiWz2klM5+EgHKuVX0j1359xc\n4F+ArWZ2WOS+wcAioBZoBM40sx0xnq+euxQl9XSjax+3XlHhHaA0d+4spkw5K2HbXbve4Prrf8BF\nF31L72cS0u25p5Lcjwd2And3Se4zgffN7GfOuauBwWZ2TYznK7mLpCDeRsXvDU4qJavubdcD3wH2\nJhj8IO4GQTw5L8uY2QpgW4+7vwzcFbl+F3BGqgGISG/xhmAWwvDMVHY2d7bdD7gEaABeo7l5OVOn\nXqIyTa6keBbuWmBNl9sf9Hj8gzjPNRFJrKmpyYLBIQYvG5jByxYMDrGmpqa4j3V9/sqVK7vdl88Y\nY7e9x2BcpL13qak5wlauXJmzOEtBJHemlKvNLPkTZCe7rYj34PTp0zuu19fX99lzGxYbv0sAfU17\nT7e5OXqvONZjoVAoYR08W//L9p3NU6dO6DZXTLRlhkIhfv7zm7n88otobd2DN3mYV8ppnzxMn7FO\nDQ0NNDQ0ZL6gVLYE9O65rwf2jVz/NLA+znNzvYGTHFiwYKEFg0Ns4MBxFgwOsQULFvodUkmI17tO\nt+eeqDfd/r8cMODwrP0vk/mV0L7e6uojrKxsgJWX11hNzREdMegzFh9p9txTTe51wNout2cCV0eu\nXw3cHOe5uX4PJMtS+ektyUsmmbW36ZoEEz22cuVKGzgwetmjqanJKioGdvtfVlQMzPn/MtpnKBAY\nZIsXL066zNTX5Ty5AwuAt4FdwCbgAmAwsAz4G7AEGBTn+fl4HySL4iWLWPJR7y1mqdaq4/XuFy9e\n3JEkEy178eLFBiO7/S/hQFu8eHFOX2+iz1CiDVImn6VS+SzmpeeeyUXJvfik2qvSz+vE0tlgRhPr\nvY7Vq/eS+17d/pewV86Te6LPUKzHZ8+ek9FnqZQ+i0rukhPxygNd6ed1crLxPq1bt84qKwfFTZg9\ne6xNTU1WXl5lMNjgCIPBVl5elZf/T6LPUM/H2xN7uu9RqX0WldwlZ5L5eZutHmlfkOwGs13X93/B\ngoVWWVljMKpXiWXGjBsTrjcQGGQDBoyyQGBQXnuziT5DXR/P9LNUap9FJXfxVan1lnIt2Xpwz/KC\n1/v+T4NgjxLLYAsEBiVcXjHUoTP9LJXaZ1HJXXyXao/UT8WS5AKBQeYd/NMUSVZBgyEGMyN/D4vc\nNyep3mkxvG6zzD9LxfRZTCTd5K4pfyWriuFglFQmvEpWLl73T37yU/7zP28ERuPNyzcL+B4wBO9A\noHDk/guBawkGvxt3OuJUXvcnn8DWrbBlC7zzDnzwAWzfDtu2eZft2+Ef/4DWVmhr8y67d3vPq6z0\nLoGAd6muhlDIu+yzD+y/Pxx4IAwbBv3iTICSzNw6VVVV7Ny5syDn38mWnE8clikldykEuZijPVcb\ni55xQj39+7eye/ce4Pku9x9LIFDBvHmzO9bbM7FFW15l5YXcddcy3ntvEK+/Dm++CW+/7SX0piYY\nMgSGDoX99oO994ZBg2Dw4M6/1dVQUQHl5d6lf38oK/MSfkuLd9m1C3bsgHC487J5M7z+urfBqK2F\nAw7YxfDhYU48sZqJEwey777Jvd8wlObmDQSDnwZ2lOwkZOkmd5VlpE/J9s62XNV3o8UJB9o111zX\nY9TLICsrC9q6des6nttZkhhngcDBdu21y+1732u0iorfdVuec5tt/PjtdtFFZj/7mdl995k9+6zZ\nxo1mra0ZhZ+Ujz4ymznzMSsvv8DKy+dYv37LbcCAFqurM5s61WzBArN33+1s3z5e3ytVdd3fMMRg\neVHX1eOhQOaWESlodXVe7zra/Cbp6D4PTBjYRVnZ/h1zvfSUbKkgWpzB4DYmTDiR22//Ezt2/Amv\nJFPHXnudTGPjLrZsgZUrd/LjHzfzySebaG4eAOxm5sznOfPMGsxeBI4CRgBrCQTq+eMfX8WvisVH\nH4X50Y/OpK2tP/Ay8Ca7drXxm99sZPXqISxaBJdcAoccAiNGrOb++79B//6OlpZWvJlPDsObafJT\nQGu3OXYE9dyluKQypC6WbO5s6+y5t+/gPNwgaLNnz4m53mQPrOkZ5+23z7EFC5ZbefnpBpsjPded\nBh/Y3nt/YieeaPbVr261QOCmqL9MCm0n46JFixIeWLVrl9nChdusrOxugzaDVoNnDT5tMCfyC2ak\nwaC8jdvPNzRaRopBJqM1EiXHVJJnNg9tnz17Tq+hicHgkKSnBohm61azJ54wu+mmD+0rX9lqI0e+\nY/CBObfNnFth/frNsUDgJquoON1uv/3/usWV6IjQVF53rkbXdI7XP6hX6alrcm9qarL58+dbdfUY\ng0UG5xo8bfCewW0G6zteZz7myvGDkrsUvEwOCU/3MPZcfNl7vo4ZM2606uojeiWpAQM+2/E6Y9X6\nlyx5wZ580mzWLLNLLzWrrzf71KfMBg82O/54s4su8hJ8RcXpBq/E3HhEiy/THnquDuHv/F8tj/za\niT6ZWdfZJL2N54AubdcZ/Mzgk6zsOylkSu5S0DJNvosXL7YBAz5r3nhv75NbXT02qQmoshX/ypUr\nbd26dVFnOex5n1cuaIo8PsRWrPibVVScYvBWpM2HBu9YVdUndswx3g7En//cbMkSsy1bzPbs6Vy3\nHxO45XJj2f31LIwk+IOssrLzqNlo6/dKOE3WdQMK7TuJW62s7P/sz39+L+P4Co2SuxS0TJJvew/O\nq60OjiQE74Ce9tp2LpNR1x5sZWWNBYNjer2OGTNutEDg0xYMfsXgbIMnDJojbbZbdXWbjRz5npWV\nLbDKyv9nFRVfs1/+8uFuSTyWWK9t3bp1OTsgKZcby96vZ7lVVtZ0G/ETa7SQd0BXZy8/EBhkNTVH\nWCAw1L72tTU2ZIjZVVeZffBBxmEWDCV3SVuhnZYt0fO8Htwgg5lRT0aRrR2G0YfePWneTtM3Ireb\nrF+/JVZbu9sqK/fY4MFhg0cM5htcbnCPBQKjbOvWpm7LjFVSiaXna7vssityOuthrstcif5X0dbf\nNZm3P6fnZ/edd8y+/W2zffYxu+suS2rjWeiU3CUt+ZwaNZ3k65VjDu/RgzvMYHHU3mSmG6q3326y\nBx98ya688hErL7/MysvvMLjXINyxfufeMOcet/Ly+61//2l21VVP2fr1Zps3R9sQdR85k+l+h1il\noWzvX2hqaor8GhmUs9E1if5X0T4vyf5/V60yGzvWbNIks9dfz2rYeafkLinzY4KlrgkqmSGLXq+5\n53C59np275/ziezZ4/Xunn3W7J57zH7yE6/ePXGiWSj0D4MWc+4d80ZkvB9Z5yaDU80boRG7JBKt\nlNB1v0C23u9MSiapDBXtusPYr1EomWysW1vNZs70dlDPm1e8vXhfkztwJfAK3hEX9wAVUdrk+C2Q\nVPk1NWoyvdfuiXBhJKEfaGVlVQYVBvsZBK2i4uBey9i+3Wz1arPf/97sllvMLrvM7EtfMjvkELNg\n0Puyjx9vduaZZldfbTZ7ttmiRdussvIogzUGKyOll66/Fg60AQNGZTSnfbbe73Q3Eqm/77nd4Odr\nErM1a8wOPdT7f2/bltNV5YRvyR3YH3ijPaEDi4Dzo7TL8VsgqfKr557MOnsnwibba69RVlFRZfBg\npDTzeuSxrVZW9nsbO7bV9t7brKrKbMwYs9NPN7viCrNbbzV76CHvS/7hh9Hj6r6+Jus5RC/e0MOu\n4pWesvl+pzMnfHrve242+Pk+U9LHH3sb+QMOMHv55ZyuKuv8Tu4b8c6n2h94BDgpSrtcvweShnR3\nQibT64rWJtnksXVrkwUCh3ZJ4GFz7nFzbpN5Y5vf7raMYPAamzfvr9bUlN7P797Jb6ZB0Kqrxyad\nPNtfa7z3JttHxybb8032fc/HyBw/51tfsMD75bZoUc5XlTV+l2UuB/4BbAV+E6NNbt8BSVuqP4+T\n6XVFa9M+UqT3OPHP2qJF22zmTLOvf93sqKO83vfAgR9bv35PW3n5fVZefp1deuljkdJJQ9SedabJ\nIdrp3tI5oUYqG4J8SSWh5npkTr6OSYj1/r74olltrdn11xdHHd7Pnvsg4HG8SabLgP8DzonSzqZN\nm9ZxWb58eY7fEsmFZJJEvGFs1dVftLKyc61fv1usrGyFwTs2YECLTZhgduWVZnfeafb885210Z5f\n1PbEEwjUGQQtGPxc1o+eTPXwfL96oalK5VdDLkfm5OuYhPj7R8yOPtrs/PPzMwNmKpYvX94tV/qZ\n3P8N+HWX2+cBt0Vpl9t3RPIi1oiQ+fPnx91xCA+bN8mVGeyw/v1vtOnTX7QXX3wv5d5TKiNucq3Y\nzteZ6sYrV68vF5OYpbrR2LnT7F/+xeyLX4y9L6YQ+JncjwbWAgHAAfOBS6O0y/V7IHkQ/aCioFVX\nj7FAYIjdcssf7OabP7R+/f5g3ix+Zt4Qwm+bN7SwyWClVVV9LmtD93Ip0fqLqeeejly+vmz/b9PZ\nELW1mX3zm2bHHuuNsipEftfcp+FNsLwGuAsoj9Imx2+B5EvnhE5jDarNG6poBrsM3razzmq2Sy55\nzgKB0R09M+8EE4mnxY22nnyNqEh3/YU2lW4i6e5jKfTXl+6GaM8eb9K2o48uzKGSOohJ8qalxWz+\n/O32+c9vMNjWo0TT2VPqmkRiTYubTo+4EKdL8PsXRrLS3WAW2+tLdUO0Z483bPbIIwtvXhold8mp\nPXvM/vxns+98x2zvvb2paW+66cPIcMXECXDlypW9psWN95M51k/sGTNuzEtvvthq6cnwu4SUzs7q\ndDYo6T6vPcF//vNePb5QKLlLTrz/vtl//ZfZ6NFmI0eazZhh1tjY+XiyPaV0esK9h0z2nlo3l0dP\nllot3c8NVqq/GGbPnmOVlTVWXT0mr6WgTz7xRtBMnuydBaoQKLlLVq1ZY/atb5kNGuR92J95xuvZ\nROsVJdtTSvUnc8/2M2bcmNfkVCy15mT5tcFKdb2dJbzDI/toZuZ1w9raanbaaWZnn+0le78pufdx\n2aqJNjSYfeELZvvv7/XS//rXcMdys7GDM5Of5n5Nl1AMtWaz/J8/Nlmp/GJoamqyyspB1n001pCk\nR1dly8cfe2fCuvrqvK0yJiX3PizTpLtnj3cGoH/+Z6/0Mm+e95O053K9ES/+lilKrTedLfk8f2yq\nUtkoR9s3A4dZZWVN3j9r4bA3F82dd+Z1tb0oufdRsb44yZ4M4rnnzI47zqup//a33rjfWMvteZoz\nv3Ywrlu3zubPn5/SVL+lrBj2D2SybyaZYbO5sn69d+KPJ5/0ZfVmpuTeZ8U6HVnXkzNH88YbZmed\nZTZ0qNcz2b07ueV2Pc2Znz33WD3UYiqjpKvnayyWkT2p7puprh5rlZWDfEvs7ZYuNdt33+4DCfJJ\nyb3ExfpiRO/pdJ6cuWcC3rnTqyMOGWJ2ww2xh3zFmx8m1Ym1siVRD9Xvg57yIdaEbIXec09VoW2k\nb7nFm9CupSX/61ZyL2GJklb74wMGHBYpnSyM2oN77DGzESPMpkwx27Il+fVGO83Z7Nlz8p5I4/VQ\niy3BpZO84r1G7YvIrT17zL76VbOLL87/upXcS1SySaupKdrJnL2269eHbcoUL7E/9ljq64829LHQ\nhtQVS2nCLP1fGIleY6H1dkvNjh1mo0aZ3X13fter5F6iUk1aPXtwP/xhg+2/v9lVV5k1Nmbny18I\nB8P07KEWS889kziL5TWWsrVrvZN9vPpq/tap5F6impqarKJiYK/ad7wvdFNTkz311F/sggs+tuHD\nzZYvz2492u8kE6uHWgyliUw3jMXwGkvdr37lzUGTryNYldxLVFNTU2R8+WCDIwwGW3l5VdxEun69\ndzLoc87xZrnLRTIu1CRT6KWJbPwvCv01lro9e7wTrl97bX7Wp+Reojp7et486NAUt6d3331moZDZ\nr38dbRnp9RZjUZJJT6FuGCV5W7ea7bdffsa/p5vcnffc3HPOWb7WVUrC4TC1taNpbl4OHAasIRic\nwMaNrxIKhTrabNjQyG9/eyiPProX998PRx6Z2jKyGW9jYyN1dXVZX3Yp0ftU/P74R7j0UlizBmpq\ncrce5xxm5lJ+YjpbhHQuqOeetng9vQULFlogMNz693/a+vVbbnfc8fuoz/dKO3sZHGgVFQNz0lvs\nC+PMRbq64gqzJ57I7Trws+funBsI/C/wOWAPcKGZ/blHG8vGukpNsj249nZVVVXs3LmTuro6AIYP\nn0hLy0ogSKxefWevfT9gKYHApWza9Pes9hjz+etApC9Jt+feL0vr/wXwqJkdDByOd8o9SeDeexdR\nWzuaSZMuprZ2NPfeu4hwOMyqVasIh8Pd2oZCITZseIMjjzy+o/306Q+xa9dSvMQOcBjl5bU0NjZ2\nPK+xsZGKijq8hBsCzqGiYkS3NtnQfT3RYxGRPEqnu2/dyy01wOtJtMvNb5YiFW3URHl5dcyyRu/2\nrxlstf79zzdvvpfo0w3ka9ii38MjRUoVaZZlstFzHwG855y70zn3onNujnMumPBZfVzvnu5+tLV9\nQnPzcnbseIHm5uVMnXpJRw++d/uRlJf/AOcexDs/eR3l5ccxd+6sbmWQUCjE3LmzCAYnUFMzjmBw\nQq822ZCv9YhIcjKuuTvnjgSeB441s784524FdpjZtB7tbNq0zrvq6+upr6/PaN3FrHeNegFekn6t\no01NzTiWLbuD8ePHd2n/F7zt6WrgOLy3PnGNO1+jMzQKRCQzDQ0NNDQ0dNy+4YYb0qq5ZyO57ws8\nZ2YHRG4fD1xtZqf1aGeZrqvU3HvvIqZOvYTy8lpaW99kzx6jtfUpYiXr73//aW65ZRQDBnyP3bsf\noV+/Wpqb13Qsr+vGoJ2SrUhx822HqpltBd5yzo2K3PUFYF2my+0Lpkw5i40bX2XZsjvYtOnvzJ9/\nR8yyxrJlcPfd/8yiRR/zq19N4vHHHwW2AO3JfQ1tbRs7RtFA9B22xS7WDmcR6SGdQn3PC94ImVXA\nS8DvgYFR2uRiX0PJiXbU59NPe0ed/vjHy7rtcL3ssstjjn8vxR2cGkcvfRE6QrU0vfACnHIK/OpX\n2/n3fz+w1zjyF15Y0THuvWvZZdWqVUyadDE7drzQcV+0sk2x0Dh66av8HucuOdDYCKedBnfcAXV1\nr0UdR75z507Gjx/fK8HV1dXR2tpIvLJNMdE4epHUKLkXgGh15G3b4NRT4Zpr4CtfgaqqKlpaNgAN\nkRbxk3WpDU0stY2VSM6lU8tJ54Jq7lFFqyO3tJjV15v9x390bxMMjjEIWiBQl3TNuZRmbtRsitIX\noZp78YlWRw4EJnDaaVvYvTvAfffBBx/0blNZeSKrVz/LwQcf7O8L8IGGdkpfk27NvX8ugpHktNeR\nm5s768hm17F27Se88AKUlUVvU1k5gp07d/oVtq9CoZCSukgSVHP3QXuNvaqqqkcd+XV27TqXe+5p\nYa+9vHtUaxaRdCi551nXA4uOPPJ4pk79OsHgBKqqTgUGcP316xk3bu+O9qW2Y1RE8kM19zyKNVb7\nmWee4ZxzPsPZZxvTplXFfK5qzSJ9T7o1dyX3PIp1YNGkSX+irGwfFi4El/rJtESkhOkgpiIQrX7e\n0vI5Vq3amzlzuif2nmPfNaeKiKRCyT2PetbPKyvPJRCYw6JFZQwc2Nmu54RfZ555dslNACYiuaWy\njA/C4TCvv97IVVcdzmmnVXDttd0f61mXh2OAR4F6NKeKSN+iskwBSVRCCYVCLF06nmCwgh/+sPtj\n0eZQgWHAgI7bmlNFRBJRcs+yZOZQX70a/ud/4O67vQOVuopWl4fNwEcdtzXOXUQSUVkmi5KZlra1\nFcaPh+9/H847L/py2s/Q1Nw8GPgAOA/4DTCEYHAbc+fOYsqUs/LymkTEX76XZZxz/SInyH44W8ss\nNtFKKs3Ng7njjl93tLnxRhg+HL7+9djlm/YzNM2YcSGBgFFT8zSBgDFjxoVs3PiqEruIJJS1nrtz\n7krgSKDGzE6P8nif7LlDPYGAsWnT39m8OcTJJ8NLL8GTT3q984oKrwwTqzeug5dE+jZfe+7OuWHA\nqcD/ZmN5xSoUCnHddd8DjgXGAROA26moGMFrr23kggvgv/8bysvDkbLLcnbseIHm5uVMnXpJ1B2w\noVAo6sk4RETiyVZZ5ufAD4DS7pon4aKLvkUgUAF8H3gVOJi2to08+uhohg716uw6q5CI5FrGyd05\n9yVgq5m9BLjIpc8KhULMmzebYPC71NScTDA4gZtvns/s2VXcdpt3FGrvETEN7Nr1OlVV0eeVERFJ\nVcY1d+fcT4GvA7uBIFAN/N7Mzu/RzqZNm9Zxu76+nvr6+ozWXci61sqnTg1xzDFw3XWdj7ePiDEb\nSEvLuwSDI4EtGgkj0sc1NDTQ0NDQcfuGG27wf+Iw59yJwPf66g7VaB5+GH74Q3j5Zais7P7Y+vXr\nOeKIz7Nr15PEGjopIn2b70MhpbePPoLLL4dZs3ondoCdO3cSCByAau8ikm1ZTe5m9mS0XntfdeON\ncNxxMHFi9Md1liURyRWdQzVH3nwT5syBNWtij1VvnyVy6tQJlJfX0ta2UWdZEpGs0PQDOXL22XDo\noTByZOKDlXSgkojEojMxFZDnnoMzz4QVK8IcfHD8uWZEROLRDtU8ijelrxlceSX89KfQ1KSDlUTE\nH0ruKUo0pe+iRbB7N5x7rnaYioh/VJZJQaIpfVtaYPRob572E07wntN+sFLXHaY6SElEkpVuWUaj\nZVLQPidMc3PvMksoFOL22+HwwzsTO3jT95500kTtMBWRvFLPPQXxeu7BYIiRI2HpUhgzxu9IRaRU\naIdqHrSPSw8GJ1BTM45gcELHuPRf/tI7WEmJXUQKgXruaeg5Ln37djjoIHjmGRg1yu/oRKSUaJy7\nj370I3jnHZg71+9IRKTUaIeqD8LhMC++uJlZsw5n9WpVuESkcCgjpSEcDvOTn/yU4cNHcfrpr/Dh\nh/N49tlFiZ8oIpInKsukqH3cenPzELyzCv4VeINg8HhNKyAiWafRMnkQDnee2BpeA54BFgKf0rQC\nIlJQVHNPQe+DmPYFHgDKNa2AiBSUbJwge5hz7gnn3F+dc2udc5dnI7BC1HuumG3A4wQCl2oedhEp\nKNkoy+wGrjKzQ4FjgUudc6OzsNyC034QU2XlZJwLU1HxJWbMuJ5Nm/6u+WJEpKBknNzN7F0zeyly\nfSewHhia6XIL2Z4959Ov31rKyv7GgQceqB67iBScrI6Wcc7VAQ3A5yKJvutjRT9aJhwOM3z4GFpa\nNgEV6OQbIpJrvo+Wcc5VAfcDV/RM7KWisbER576Jl9hBJ98QkUKVldEyzrn+eIn9N2b2UKx206dP\n77heX19PfX19NlafN7W1dbS0/BvwJjACnXxDRLKtoaGBhoaGjJeTlbKMc+5u4D0zuypOm6Ivyzz1\nFJx99ods23YAFRXDdfINEck53yYOc84dBzwFrMU7ZNOA68zsTz3aFX1yP+MMmDwZvvrVsE6+ISJ5\noVkhc2zDBjj2WNi4Efbaq/fjPacBFhHJBt93qJa6X/wCvv3t6Ik90UmzRUTyTT33JGzbBgceCK+8\nAvvv3/2xRCfNFhHJhHruOXTrrTs55pj3KC8P93qsfb4ZL7GDhkeKSCFQck/gnnsWMWPGezz11JVR\nSy6955vR8EgR8Z/KMnGEw2GGDbuU1tb5wF7EKrm0z/FeXl6r4ZEiklUaLZMDq1at4vOf/5jdu0/s\nuK+mZhzLlt3B+PHju7XVaBkRyQWdQzUHysoOYPduhzeEfwzxSi6hUEhJXUQKhmrucTzwwN5Mnvwe\ngcAJDBjwWQKBEzVvu4gUBSX3GFpbYe5c+MIXNuBcPyAY+SsiUvhUc49h4UKYNauVv/xlP41hFxHf\naJx7lt1+O5x66iaNYReRoqTkHsXf/uZdzj9/oMawi0hRUnKPYv58OO882H9/75ypweAEamrGEQxO\n0A5VESkUD7eEAAAKwUlEQVQKqrn3sHs3DB8Oy5bBIYd492kMu4j4RePcs2TxYqit7UzsoDHsIlJ8\nVJbpYd48uPBCv6MQEclMtk6zNxm4FW9jMdfMZkZpU/BlmXAYDjoINm2Cmhq/oxER8XEopPOO7LkN\nOBk4FJjinBud6XL98Nvfwpe/rMQuIsUvG2WZo4HXzGyjmbUBC4EvZ2G5eWWmkoyIlI5sJPehwFtd\nbm+O3FdUXngBmpvhhBP8jkREJHN5HS0zffr0juv19fXU19fnc/Vx3X03nH8+uJQrWyIi2dPQ0EBD\nQ0PGy8l4h6pz7hhguplNjty+BrCeO1ULeYdqWxsMGwbPPAMjR/odjYhIJz/nllkFjHTO1TrnKoCz\ngYezsNy8WbYMRoxQYheR0pFxWcbMPnHOXQYsoXMo5PqMI8uje+6Bc8/1OwoRkezp89MPfPQRDB0K\nf/877LOP39GIiHSnKX/T9NBDcOyxSuwiUlr6fHJXSUZESlGfLsu0TzeweTNUVfkdjYhIbyrLpOF3\nv4MvfUmJXURKT59O7gsWqCQjIqWpz5Zl3noLxo6Fd9+F8nK/oxERiU5lmRTdf783A6QSu4iUoj6b\n3O+7D772Nb+jEBHJjT5ZlmkvybzzDlRU+B2NiEhsKsuk4IEH4PTTldhFpHT1yeSukoyIlLo+V5bZ\nvBkOO8wbJaOeu4gUOpVlkqSSjIj0BX0uuaskIyJ9QZ8qy2zZAmPGqCQjIsVDZZkkPPigN5eMEruI\nlLqMkrtz7mfOufXOuZeccw8452qyFVguPPQQnHGG31GIiOReRmUZ59xJwBNmtsc5dzPeibGvjdHW\n17LM9u0wfDi8/bZmgRSR4uFLWcbMlpnZnsjN54FhmSwvlx57DE44QYldRPqGbNbcLwQey+Lysuqh\nh7yJwkRE+oL+iRo455YC+3a9CzDgejN7JNLmeqDNzBbEW9b06dM7rtfX11NfX596xGlobYXFi+HW\nW/OyOhGRtDU0NNDQ0JDxcjIeCumc+wbwLWCime2K0863mvuSJTBtGjz3nC+rFxFJW7o194Q99wQr\nnQz8ADghXmL324MPqiQjIn1LpqNlXgMqgPcjdz1vZpfEaOtLz90MPvMZWLYMRo/O++pFRDLiS8/d\nzA7K5Pn58MILMGCAEruI9C0lf4SqSjIi0heVfHJ/5BFvFkgRkb6kpCcOa5+7vakJ+mdUgBIR8Ycm\nDoviscfg5JOV2EWk7ynp5P7oo94skCIifU3JlmV27YJ99oENGyAUyttqRUSySmWZHlasgIMPVmIX\nkb6pZJP7o4/Cqaf6HYWIiD+U3EVESlBJJvc33oAPPoBx4/yORETEHyWZ3B99FE45BfqV5KsTEUms\nJNOfhkCKSF9XckMhP/4Y9t0X3noLBg3K+epERHJKQyEjnnoKxo5VYheRvq3kkvuSJfDFL/odhYiI\nv0ouuS9dquQuIpKV5O6c+55zbo9zbkg2lpeud96BLVvgqKP8jEJExH8ZJ3fn3DBgErAx83Ays3Qp\nTJwIZWV+RyIi4q9s9Nx/jneSbN8tXQqTJvkdhYiI/zJK7s6504G3zGxtluJJm5nq7SIi7RKexsI5\ntxTYt+tdgAE/Aq7DK8l0fSym6dOnd1yvr6+nvr4++UgTWLsWqqpgxIisLVJEJO8aGhpoaGjIeDlp\nH8TknPscsAz4GC+pDwO2AEebWVOU9jk9iOmWW7y522+/PWerEBHJu3QPYkr7BHRm9grw6S4BvAmM\nM7Nt6S4zE0uWwMUX+7FmEZHCk81x7kaCskyutLTAs8/ChAl+rF1EpPBk7dTRZnZAtpaVqhUrYMwY\nTTkgItKuJI5Q1RBIEZHustZz99MJJ8ABvv1uEBEpPCU35a+ISCnRlL8iItJByV1EpAQpuYuIlCAl\ndxGREqTkLiJSgpTcRURKkJK7iEgJUnIXESlBSu4iIiVIyV1EpAQpuYuIlCAldxGREpRxcnfOfdc5\nt945t9Y5d3M2ghIRkcxklNydc/XAacAYMxsD/Hc2gipE2ThhrZ+KOf5ijh0Uv9+KPf50Zdpz/w5w\ns5ntBjCz9zIPqTAV+wekmOMv5thB8fut2ONPV6bJfRRwgnPueefccufcUdkISkREMpPwTEzOuaXA\nvl3vwjsZ9o8izx9sZsc458YDvwN0TiQREZ9ldCYm59yjwEwzezJyewPwT2b2fpS2Og2TiEga0jkT\nU6bnUH0QmAg86ZwbBZRHS+zpBiciIunJNLnfCcxzzq0FdgHnZx6SiIhkKm8nyBYRkfzJ2RGqzrnB\nzrklzrm/OecWO+cGRmkzzDn3hHPur5GDoC7PVTzJcM5Nds696pz7u3Pu6hhtfumce80595Jzbmy+\nY4wnUfzOuXOccy9HLiucc2P8iDOWZN7/SLvxzrk259y/5jO+RJL8/NQ751Y7515xzi3Pd4zxJPH5\nqXHOPRz57K91zn3DhzCjcs7Ndc5tdc6tidOmkL+7ceNP67trZjm5ADOBH0auX403Hr5nm08DYyPX\nq4C/AaNzFVOCePsBG4BaoBx4qWcswCnAHyPX/wl43o9YM4j/GGBg5PrkYou/S7vHgT8A/+p33Cm+\n/wOBvwJDI7c/5XfcKcZ/LXBTe+zA+0B/v2OPxHM8MBZYE+Pxgv3uJhl/yt/dXM4t82Xgrsj1u4Az\nejYws3fN7KXI9Z3AemBoDmOK52jgNTPbaGZtwEK819DVl4G7Aczsz8BA59y+FIaE8ZvZ82a2I3Lz\nefx7r6NJ5v0H+C5wP9CUz+CSkEz85wAPmNkWKLiD/pKJ34DqyPVq4H2LHMDoNzNbAWyL06SQv7sJ\n40/nu5vL5L6PmW2NBPYusE+8xs65Orwt159zGFM8Q4G3utzeTO83sGebLVHa+CWZ+Lv6JvBYTiNK\nTcL4nXP7A2eY2e14x1sUkmTe/1HAkMgBf6ucc+flLbrEkon/NuAQ59zbwMvAFXmKLRsK+bubqqS+\nuxmNlklwgFNPMffcOueq8HpjV0R68JJDzrkJwAV4PwWLya14Jb52hZbgE+kPjMMbPjwAeM4595yZ\nbfA3rKSdDKw2s4nOuQOBpc65w/SdzZ9UvrsZJXczmxQniK3OuX3NbKtz7tPE+BntnOuPl9h/Y2YP\nZRJPhrYAw7vcHha5r2ebzyRo45dk4sc5dxgwB5hsZvF+xuZbMvEfBSx0zjm8mu8pzrk2M3s4TzHG\nk0z8m4H3zKwFaHHOPQUcjlfr9lsy8V8A3ARgZq87594ERgN/yUuEmSnk725SUv3u5rIs8zDwjcj1\nfwdiJe55wDoz+0UOY0nGKmCkc67WOVcBnI33Grp6mMhYfufcMcD29tJTAUgYv3NuOPAAcJ6Zve5D\njPEkjN/MDohcRuB1CC4pkMQOyX1+HgKOd86VOef2wtuxtz7PccaSTPwbgZMAIvXqUcAbeY0yPkfs\nX3OF/N1tFzP+tL67Odz7OwRYhjcCZgkwKHL/fsAfItePAz7B2zO/GngRb6vk1x7ryZF4XwOuidx3\nEfDtLm1uw+tpvQyM8yvWdOIHfo03wuHFyPu90u+YU33/u7SdRwGNlknh8/N9vBEza4Dv+h1zip+f\n/YDFkdjXAFP8jrlL7AuAt/EOptyE9yujmL67ceNP57urg5hEREqQTrMnIlKClNxFREqQkruISAlS\nchcRKUFK7iIiJUjJXUSkBCm5i4iUICV3EZES9P8BMBuYOrn7jLUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f85b70134a8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "model = PolynomialRegression(4).fit(X, y)\n",
    "plt.scatter(X, y)\n",
    "plt.plot(X_test, model.predict(X_test));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Detecting Data Sufficiency with Learning Curves\n",
    "\n",
    "As you might guess, the exact turning-point of the tradeoff between bias and variance is highly dependent on the number of training points used.  Here we'll illustrate the use of *learning curves*, which display this property.\n",
    "\n",
    "The idea is to plot the mean-squared-error for the training and test set as a function of *Number of Training Points*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.learning_curve import learning_curve\n",
    "\n",
    "def plot_learning_curve(degree=3):\n",
    "    train_sizes = np.linspace(0.05, 1, 20)\n",
    "    N_train, val_train, val_test = learning_curve(PolynomialRegression(degree),\n",
    "                                                  X, y, train_sizes, cv=5,\n",
    "                                                  scoring=rms_error)\n",
    "    plot_with_err(N_train, val_train, label='training scores')\n",
    "    plot_with_err(N_train, val_test, label='validation scores')\n",
    "    plt.xlabel('Training Set Size'); plt.ylabel('rms error')\n",
    "    plt.ylim(0, 3)\n",
    "    plt.xlim(5, 80)\n",
    "    plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see what the learning curves look like for a linear model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEPCAYAAABGP2P1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYHGW58P/v3d2zb5klmYTsIQkJCUsAWQQ0oCLBw0ER\nkEDYPfITeOUg8jOgLMJRjkc8HpH3CCiBBA2rSEBAUCACIoRAAoQQEAiTfZ197+V+/6jqnu6enmSm\np3u6O3N/rquurq56uuruWequep6q5xFVxRhjjAnzZDoAY4wx2cUSgzHGmBiWGIwxxsSwxGCMMSaG\nJQZjjDExLDEYY4yJkdbEICIFIvK6iKwSkXdF5MY+yt0uIv8UkdUicmg6YzLGGLNnvnRuXFW7ROQE\nVW0XES/wdxF5RlVXhMuIyDxgf1WdJiJHAXcCR6czLmOMMX1Le1WSqra7swU4iSj+ibrTgCVu2deB\nChGpTXdcxhhjEkt7YhARj4isArYBf1HVN+KKjAU2Rr3f7C4zxhiTAUNxxRBS1TnAOOAoETkw3fs0\nxhiTvLS2MURT1WYReRE4GVgbtWozMD7q/Th3WQwRsU6djDEmCaoqAymf7ruSakSkwp0vAr4ErIsr\n9gRwvlvmaKBRVbcn2t7KzSt7TW9ueZM129fwz93/ZEPjBna07qCps4lOfyehUAhV3et044039qtc\ntky5Fm8uxpxr8VrMFm9fUzLSfcUwBlgsIh6cJPSQqj4tIpcCqqp3u+9PEZGPgDbgooHsQFXpDHTS\nGejstU5EyPfmU+AtoNBXSIGvgJriGjxij28YY0xf0n276rvAYQmW3xX3/oo07Z+uQBddgS6au5oB\n8Hl8VBVVpWN3xhizTxh2p84NHQ29ls2dO3foAxmEXIsXci/mXIsXLOahkGvxJkuSrYMaaiKiKzev\nTMV2OKT2ELwebwqiMsaY7CYi6AAbn4fsrqRsoao0dDZQU1yT6VCMiZg0aRJ1dXWZDsPksIkTJ/Lp\np5+mZFvDLjGAU51kicFkk7q6uqTvIDEGnCuDVBl2bQwALd0t+IP+TIdhjDFZaVgmhnB1kjHGmN6G\nZWKAxHcnGWOMGcaJobW7le5gd6bDMGZY+Pa3v82Pf/zjlJc16THsbleNNrZ8LKNLR6d0m8Ykw72l\nMNNhJDR58mTuueceTjzxxEyHYvagr7+hZG5XHbZXDGDVScakQjAYzHQIWWFf+jkM68TQ7m9P2MeS\nMcZx/vnns2HDBk499VTKy8u57bbbqKurw+PxsGjRIiZOnMgXvvAFAM466yzGjBlDZWUlc+fOZe3a\nnk6UL7roIm644QYA/va3vzF+/Hj++7//m9raWsaOHct9992XVNn6+npOPfVUKioqOOqoo7j++us5\n/vjjE36Xrq4uzjvvPGpqaqisrOSoo45i586dADQ0NHDxxRczduxYqqurOf300yOf+81vfsO0adOo\nqanhq1/9Klu3bo2s83g8/O///i/Tp09n+vTpAKxbt46TTjqJ6upqZs6cySOPPBIp//TTTzNr1izK\ny8sj3ysbDevEAFDfUZ/pEIzJWkuWLGHChAn86U9/orm5me9973uRdS+99BLr1q3j2WefBeCUU07h\n448/ZseOHRx22GGce+65fW5327ZttLS0sGXLFn77299y+eWX09TUNOCyl112GWVlZezYsYP77ruP\nxYsX93k//+LFi2lubmbz5s3U19dz5513UlRUBMCCBQvo6Ojg/fffZ8eOHVx11VUAvPDCC1x33XU8\n+uijbN26lQkTJnD22WfHbHfZsmWsWLGCtWvX0t7ezkknncSCBQvYtWsXDz74IJdddhnr1jmdSn/z\nm9/kN7/5Dc3NzaxZsyZrq+eGfWKw6iSTC0RSMyUrvu5aRPjRj35EUVERBQUFAFx44YUUFxeTl5fH\nDTfcwNtvv01LS0vC7eXn53P99dfj9XqZN28epaWlfPDBBwMqGwqFeOyxx7j55pspKChg5syZXHDB\nBX1+h7y8PHbv3s2HH36IiDBnzhxKS0vZtm0bzz77LHfddRfl5eV4vd7IVcfSpUu55JJLOOSQQ8jL\ny+PWW2/lH//4Bxs2bIhs97rrrmPEiBEUFBTwpz/9icmTJ3P++ec73e8ccghf//rXI1cN+fn5vPfe\ne7S0tFBRUcGhhx7a/1/CEBr2iaEz0Em7v33vBY3JINXUTKk0bty4yHwoFGLhwoVMnTqVESNGMHny\nZESEXbt2JfxsdXU1Hk/P4ae4uJjW1tYBld25cyfBYDAmjvHjxyfaBOBUi335y1/m7LPPZty4cSxc\nuJBgMMjGjRupqqqivLy812e2bNnCxIkTI+9LSkqorq5m8+aescSi919XV8drr71GVVUVVVVVVFZW\nsnTpUrZvd4aY+cMf/sBTTz3FxIkTOeGEE3jttdf6jDeThn1iAKtOMmZP+qqaiV6+dOlSnnzySV54\n4QUaGxv59NNPBzVQTH+MHDkSn8/Hpk2bIss2btzYZ3mv18v111/Pe++9x6uvvsqTTz7JkiVLGD9+\nPPX19TQ3N/f6zH777RfTh1VbWxu7d++OSQbRP4fx48czd+5c6uvrqa+vp6GhgebmZu644w4ADj/8\ncB5//HF27tzJaaedxllnnTWon0G6WGLAqpOM2ZPRo0fzySefxCyLP+C3tLRQUFBAZWUlbW1tXHvt\ntSntuycRj8fD6aefzk033URHRwfr1q1jyZIlfZZfvnw5a9asIRQKUVpaSl5eHl6vl9GjRzNv3jwu\nu+wyGhsbCQQCvPzyywDMnz+fe++9l3feeYeuri6uu+46jj766D6vTP7lX/6FDz/8kN/97ncEAgH8\nfj8rV65k3bp1+P1+li5dSnNzM16vl7KyMrze7Ozl2RID0B3sprU78WWsMcPdwoULueWWW6iqqorc\nRRN/0D///POZMGECY8eOZfbs2Xz2s58d0D4GkkSiy/7qV7+isbGRMWPGcMEFF3DOOedE2jzibdu2\njTPOOIOKigpmzZrFCSecwIIFCwC4//778fl8zJgxg9raWn75y18C8IUvfIFbbrmF008/nbFjx7J+\n/XoefPDBPuMuLS3lueee48EHH2S//fZjv/32Y+HChXR3d0f2M3nyZEaMGMHdd9/N0qVL+/29h9Kw\nfsAt2siSkUyomJC27RuzJ9n8gFsuWbhwIdu3b+fee+/NdChDzh5wS4OGjgb7xzQmx3zwwQe8++67\nAKxYsYJ77rkn5hkEk5xhOR5DIoFQgJbuFsoLet+ZYIzJTi0tLcyfP5+tW7dSW1vLNddcw6mnnprp\nsHKeVSVFqS6uZtKISWndhzGJWFWSGSyrSkqTxs5G++c0xgx7lhiiBENBmroSP5ZvjDHDhSWGOPaw\nmzFmuLPEEKeps4mQhjIdhjHGZIwlhjghDdHY2ZjpMIwxJmMsMSRg1UnGDF54LIWw2bNn89JLL/Wr\n7EDZcKCpZc8xJNDc1UwwFMTryc5+TIzJFdFdRqxZs6bfZfdk8eLF/Pa3v430ZwTw61//OrkATUJp\nvWIQkXEi8oKIvCci74rIdxKU+byINIrIW+70w3TG1B+qSkOndaxnTDZS1bR30DdUsnU40HRXJQWA\n76rqLOAY4HIRmZGg3Euqepg7/UeaY+oXq04yBv7rv/6LM888M2bZlVdeyb//+78DcN9993HggQdS\nXl7O1KlTufvuu/vc1uTJk3nhhRcA6Ozs5MILL6SqqorZs2fzxhtvxJT96U9/ytSpUykvL2f27Nk8\n/vjjgDNs5re//W3+8Y9/UFZWRlVVFRA7HCjsfTjOu+66i+nTp1NVVcUVV1zRZ8xvvPEGn/nMZ6io\nqGDMmDExI9i98sorHHvssVRWVjJx4sRIz67Nzc2cf/75jBo1ismTJ8dUcS1evJjjjjuO7373u9TU\n1PCjH/0IgEWLFnHggQdSXV3NvHnzYgYCuuqqq6itraWiooJDDjkkZsjUdElrYlDVbaq62p1vBd4H\nxiYomnXpv6WrBX/Qn+kwjMmos88+m2eeeYa2tjbAGZDnkUceiQzbWVtby9NPP01zczP33nsvV111\nFatXr97rdm+66SbWr1/P+vXrefbZZ1m8eHHM+qlTp/L3v/+d5uZmbrzxRhYsWMD27duZMWMGd955\nJ8cccwwtLS3U1/c+gevPcJxPPfUUb775Jm+//TYPP/wwzz33XMI4w0mwqamJjz/+ODJ+Ql1dHaec\ncgpXXnklu3btYvXq1ZHR2K644gpaWlr49NNPWb58OUuWLInp1O/1119n6tSp7Nixgx/84AcsW7aM\n//zP/4yM03D88cczf/58AJ577jleeeUVPvroI5qamnj44Yeprq7e6893sIasjUFEJgGHAq8nWH2M\niKwGNgPXqGr6U2I/NHQ2MKpkVKbDMAb5UWrOnfTGgT3ZP2HCBA477DD++Mc/smDBAp5//nlKSkr4\nzGc+A8C8efMiZY8//nhOOukkXn755b0OWfnII49w5513UlFRQUVFBd/5zne45ZZbIuu//vWvR+bP\nPPNMfvKTn7BixYp+9YMUPRwnwK233kplZSUbNmxgwgSnB+Vrr72WsrIyysrKOOGEE1i9ejUnnXRS\nr23l5+fz0UcfsXv3bqqrqznyyCMBeOCBB/jSl74USRSVlZVUVlYSCoV46KGHeOeddyguLmbixIlc\nffXV3H///Vx00UUAjB07lssuuwyAgoIC7rrrLq699lqmT58OOD3E/vjHP2bjxo3k5eXR0tLC2rVr\nOfLIIznggAP2+v1TYUgSg4iUAo8CV7pXDtHeBCaoaruIzAMeB6YPRVx7U99Rb4nBZIWBHtBTaf78\n+TzwwAMsWLCABx54gHPOOSey7plnnuHmm2/mww8/JBQK0dHRwcEHH7zXbW7ZsiVmFLTo4TMBlixZ\nwi9+8Qs+/fRTwBk5ra9hQhNt+/DDD4+8jx6OM5wYamtrI+v3NKzoPffcw/XXX8+MGTOYMmUKN9xw\nA1/5ylfYuHEj+++/f6/yu3btIhAIRPYT/m7RQ4HG331VV1fHlVdeydVXXw30tKFs3ryZE044gSuu\nuILLL7+cDRs2cPrpp3PbbbdRWlrar59FstKeGETEh5MU7lfVZfHroxOFqj4jIv8rIlWq2usa8a6f\n3xWZP/yYwznis0ekKWpHW3cb3cFu8r35ad2PMdnszDPP5Hvf+x6bN2/mj3/8Y2Sc4u7ubs444wx+\n97vfcdppp+HxePja177Wr/7GxowZw8aNG5k5cyZAzPCZGzZs4Fvf+hYvvvgixxxzDABz5syJbHdv\nDc/9GY6zv/bff//IYDp/+MMfOOOMM6ivr2f8+PGsWLGiV/mamhry8vKoq6tjxowZke82dmxPDXp8\n/BMmTOCHP/xhpPoo3hVXXMEVV1zBrl27OPPMM/nZz34WaZtIZPny5SxfvnygXzXGUDzHsAhYq6q/\nTLRSRGqj5o/E6fE1YcvvpVdfGpnSnRTCrBHaDHc1NTV8/vOf56KLLmLKlCmR6ozu7m66u7upqanB\n4/HwzDPP9FlXH++ss87i1ltvpbGxkU2bNkXGRAbnQO7xeKipqSEUCnHvvffG3OpaW1vLpk2b8PsT\ntwEOdDjOPfn9738fuVKpqKhARPB4PJx77rk8//zzPProowSDQerr63n77bfxeDycddZZ/OAHP6C1\ntZW6ujp+8YtfcN555/W5j0svvZSf/OQnkUblpqYmHn30UQBWrlzJihUrCAQCFBUVUVhYiMez58P2\n3LlzuemmmyJTMtJ9u+qxwLnAiSKyyr0d9WQRuVREvuUWO0NE1ojIKuB/gG+kM6aBssRgDJxzzjk8\n//zzkUZncIaxvP322znzzDOpqqriwQcf5LTTTutzG9FnyjfeeCMTJkxg8uTJnHzyyZx//vmRdTNn\nzuTqq6/m6KOPZvTo0bz33nscd9xxkfUnnngis2bNYvTo0Ywa1buqd6DDce7pCuTPf/4zs2bNory8\nnKuuuoqHHnqIgoICxo8fz9NPP81tt91GVVUVc+bM4Z133gHg9ttvp7i4mClTpvC5z32OBQsWRNoX\nEvnqV7/KwoULOfvssxkxYgQHH3wwf/7znwHnDqd/+7d/o6qqismTJ1NTU8M111zT57ZSJafGY7hk\n2SWcM/scKgorhnTfs0bNotBXOKT7NMOLjcdgBmvYjsewq30Xpz98OneuvJPmruYh269dNRhjhpOc\nSgzXf+56Fn91MTvadvC1h742ZAnCEoMxZjjJqaqk6KE9NzVvYtGqRfyt7m+cNess5s+en9bxmmeO\nnElxXnHatm+GN6tKMoOVyqqknE0MYUOVIGpLaxlXPvDb3YzpD0sMZrAsMSSwqXkT96y6h5fqXuKs\nWWdxzuxzKCsoS9n+8735HFR7UMq2Z0w0SwxmsCwx7EE6E8QBNQdQmp/eJw7N8GSJwQyWJYZ+2Ni0\nkUWrF/FS3Ut8Y9Y3mD97/qATxMiSkUyomLD3gsYM0KRJk2Ke1jVmoCZOnBjpQiSaJYYEwgni5Q0v\nRxJEsmf9Po+Pg2sP3mf6gjfG7PssMezBxqaN/OqNXxEKhbjtpNuS3s606mlpvfvJGGNSaZ9/wG0w\nxleM5+a5N/Pujnf5qP6jpLdjzzQYY/Z1wyYxABT6CjnnoHNYtHpR0tto7Gy0RkJjzD5tWCUGgDNm\nnsGKzSuoa0yuoS8YCtLU1ZTiqIwxJnsMu8RQkl/CN2Z9g/vevi/pbVh1kjFmXzbsEgPAN2Z9g5fq\nXmJLy5akPt/U2URIQymOyhhjssOwTAzlBeV8bcbXWPL2kqQ+H9IQjZ2NKY7KGGOyw7BMDADnHHQO\nz33yHDvbdib1eatOMsbsq4ZtYqgqquIr077C7979XVKfb+5qJhgKpjgqY4zJvGGbGADOO/g8nvzw\nSRo6Ggb8WVWloXPgnzPGmGw3rBPDqJJRfHHyF1m6ZmlSn7fqJGPMvmhYJwaACw65gMfef4yWrpYB\nf7alqwV/0J+GqIwxJnOGfWIYWz6W4yccz0PvPZTU5606yRizrxn2iQHgwkMv5KH3HqLd3z7gz1p1\nkjFmX2OJAZg0YhJH7HcEf3j/DwP+bFt3G93B7jREZYwxmWGJwXXxoRfz+3d/T2egc8CftasGY8y+\nxBKDa1r1NGbWzOSJD54Y8GctMRhj9iWWGKJcMucSFr+9eMB3GnX4O5K60jDGmGzky3QA2WT2qNlM\nGjGJp/75FF+d8dUBfXbdrnV4pCfPxo/ZoPQewyHRuA7x5bziZUbNDAp8BQOKxxhjkmVXDHHCVw2B\nUGBAnwuGgviD/sgUCAVipmAo2GsKaajXpKoxUyAU4OOGj603V2PMkLHEEOewMYdRXVTNXz/5a6ZD\niejwdyQ9sJAxxgxUWhODiIwTkRdE5D0ReVdEvtNHudtF5J8islpEDk1nTP1xyZxLWLRqUVadpdd3\n1LOjbUemwzDGDAPpvmIIAN9V1VnAMcDlIjIjuoCIzAP2V9VpwKXAnWmOaa+OHnc0hb5Cln+6PNOh\nxNjUvInW7tZMh2GM2celNTGo6jZVXe3OtwLvA2Pjip0GLHHLvA5UiEhtOuPaGxHh4jkXs2j1ooQN\nxJmiqnzS8In1z2SMSasha2MQkUnAocDrcavGAhuj3m+md/IYcp+b+Dn8QT+vbno106HE8Af9fNLw\nSVYlLGPMvmVIblcVkVLgUeBK98ohKXf9/K7I/OHHHM4Rnz0iBdEl5hEPF8+5mHveuofPjvssIpK2\nfQ1Ua3crm5o3Mb5ifKZDMcZkmeXLl7N8+fJBbUPSfeYpIj7gT8AzqvrLBOvvBF5U1Yfc9+uAz6vq\n9rhyunLzyrTGGi8YCnLmI2dy3fHXccR+6UtCyZpcOZmqoqpMh2GMyWIigqoO6Mx2KKqSFgFrEyUF\n1xPA+QAicjTQGJ8UMsXr8XLRnItYtGpRpkNJqK6xjg5/R6bDMMbsY9J9u+qxwLnAiSKySkTeEpGT\nReRSEfkWgKo+DawXkY+Au4DL0hnTQM2bOo+NzRt5Z/s7mQ6ll5CG+LjhYxt72hiTUmmvSkqVTFQl\nhT269lFe2fAK/3Py/2Rk/3tTUVjB1KqpmQ7DGJOFsrUqKeedOv1UPtj9Aet2rct0KAk1dTaxtWVr\npsMwxuwjLDH0Q4GvgHMPOpd7V9+b6VD6tKVlC02dTZkOwxizD8ipxFCaX5qxfX995tdZtW0VnzR8\nkrEY9mZ943q6Al2ZDsMYk+NyKjFMr56esdszi/KKOHvW2Vl91RAMBfmk4ZOs6uPJGJN7cioxiAiT\nKyczpmxMRvZ/1qyzeHXjq2xq3pSR/fdHu7+dDU0bMh2GMSaH5VRiCNuvbD8mjZg05E8jl+aXcsaB\nZ7D47cVDut+B2t2+m51tOzMdhjEmR+VkYgCoLq5mWtU0vB7vkO53/uz5PL/+eba1bhvS/Q7UxuaN\ntHW3ZToMY0wOytnEAFBWUDbkw16OKBzBaQecxv3v3D9k+0yGqvJxw8fWE6sxZsByOjEAFPoKmVEz\ng5L8kiHb57kHncszHz3D7vbdQ7bPZFhPrMaYZOR8YgDweXxMr55OZVHlkOyvpriGL+//ZX7/7u+H\nZH+D0drdyuaWzZkOwxiTQ/aYGETEKyK3DVUwg+ERD1MqpzC6dPSQ7O+CQy5g2QfLcuKhsu2t26nv\nqM90GMaYHLHHxKCqQeC4IYolJcaWj2XiiIlpv2NpdOlo5k6ay4PvPZjW/aSK9cRqjOmvvXaiJyK/\nxhlR7REgcpuLqj6W3tB6xaEDqStv7mrmk4ZP0trz6MamjVy47EKWnb0so09l91eBr4CZNTOH/E4u\nY0zmpKsTvUJgN3AicKo7/cvAwxta5QXlHFB9APne/LTtY3zFeI4ZdwyPrn00bftIpa5AF582fprp\nMIwxWS6nut1OJlZ/0M9H9R/R7m9PQ1Twcf3HXPb0ZSw7exmFvsK07CPV9ivbL2NPjxtjhlYyVwz9\nqUoaB/wKONZd9DLO2M1D2i9EsokBnAFt1jesp7GzMcVROa75yzUcNuYw5s+ej6oS0hAhDRHUIKpK\nUIORZZF1oSCKEgzFrov/jEc8TK+ejkdSewPZtOpplBeUp3Sbxpjsk67E8BdgKRB+omsBcK6qfimp\nKJM0mMQQtrFpIzvadqQooh7rdq3jkicuoTvYjaJ4xIMgeD1ePOKJTF6Jfd9rnceDB4/z6i5v6Wph\nSuUUbjnhlpS2Y/g8PmqKa/B6vJG4wvPhuKPnjTG5KV2JYbWqHrq3ZemWisQAsKFpQ1r6EQqEAgji\nJIUU3hHlD/q57R+38dbWt/j5ST9nQsWElG27v0SkV6IIz3vFi8/jw+fxkefNI8+TR543L7LMGJNZ\n6UoMzwP3Ag+4i+YDF6nqF5KKMkmpSgyqyoe7P6S1uzUFUQ2dR9c+yt1v3c3Nc2/m6HFHZzqcfhER\n8jx5vZJGomV2VWJMeqQrMUzEaWM4BlDgVeA7qjqkfTunKjGAcxa+btc6uoPdKdneUFm1dRXXvnAt\n5x18HufMPmfIe5dNJ494eiUNrzi31SqKqqI4v/9E8+G/jf6UjSY4P8Pwz7I/7/e0rtBXSGl+KcV5\nxfvU78fkrpQnBhHx4iSBXww2uMFKZWIAaOtu44PdH+RcP0JbW7Zy9V+uZlrVNK477roh7UDQ9J9H\nPJTkl1CWX0Zpfikl+SV2VWQyIl1XDCtU9chBRZYCqU4M4IxbkIv39Xf4O7j5pZvZ0rKFn33pZ4wq\nGZXpkMxeiAjFecWU5pdGJmuDMUMhXYnhF0Ae8BCxTz6/lUyQyUpHYoD0NUanm6py39v38cjaR/jp\nF37KQbUHZTokM0BFeUUxiSKdD2Oa4StdieHFBItVVU8cyI4GK12JIVcbo8NeqnuJW166hSuPupJ/\nmZ71D6SbPcj35lNWUEZJXgk+jy9yN1i4XSN6PvoOuHTcDWf2HeloY/AAZ6jqw4MNbrDSlRggdxuj\nw9Y3rOfq567m2AnHcuVRV1oVxTCWKGnEJ5X4Bnlg0O+BmGdhwvPRz+8kej5mT+tMaqTrimGlqh4x\nqMhSIJ2JAXK3MTqsuauZ656/jhAhbj3xVioKKzIdkjGDEk4Y8RL9j8bfbdafcuEEGj2Fk+iepkSf\ni99GuAeE6LvkQhqKuYsu0bL4z/S1rK/tJVp3+H6HpyUx/Cewi95tDEPawX+6EwPkbmN0WCAU4I4V\nd7C8bjk//9LP2b9q/0yHZIzJsCPGHpGWxLA+wWJV1SkD2dFgDUVigNxtjI721D+f4n9e+x9+cPwP\nmDtpbqbDMcZkUDKJYa+V0ao6OdmAROQenC66t6vqwQnWfx5YBnziLnpMVf8j2f2lwvjy8XT4O3K2\nMRrgK9O+wsSKiXz/r9/no/qPuHjOxVZna4zpt70eLUSkWER+KCJ3u++niUh/b3+5F/jyXsq8pKqH\nuVNGkwI4DTVTKqfk/K2Ds0fNZvFXF/P3jX9n4fML09btuDFm39Of08h7gW7gs+77zUC/DuCq+grQ\nsJdiWXePXZ43jymVU3L+9r+a4hru/MqdlOSVcMkTl7ClZUumQzLG5ID+JIb9VfW/AD+AqraT2oP5\nMSKyWkSeEpEDU7jdQSnJL2FixcRMhzFoBb4CbvjcDfzrAf/KRcsuYuWWlZkOyRiT5fpzw3u3iBTh\ndKCHiOwPdKVo/28CE1S1XUTmAY8D0/sqfNNNN0Xm586dy9y5c1MURmLVxdW0+dtyvjFaRJg/ez5T\nKqdw3QvX8c053+TMA8/M+SsiY0xvK19dyZv/eHNQ2+jPXUlfAn4IHAg8hzOS24WqurxfO3B6Z30y\nUeNzgrLrgcMT3Qo7VHclxcv1J6PjbWrexHef+y6H1B7ClUddSZ4nL+ZBKCDm6VpLHsbktrTcrgog\nItXA0ThVSK+p6q5+70BkEk5i6NWZj4jUqup2d/5I4GFVndTHdjKSGMB5PuD9ne/n7JPR8dq627j5\npZt5ZcMrvR6OASIP1IRFJwqIfboWoLygnJElIxlVPIpRJc4U/74or2jov6gxJn2JIVkishSYC1QD\n24EbgXyc5yDuFpHLgW/jtF90AFep6ut9bCtjiQFy/8noZPX1ZGX0a1NXEzvbdrKjfQc72pwp+v3O\ntp3kefNeA36GAAAcoElEQVQiiWJkychIwhhV3PN+ROEIu63WmBTLusSQSplODJD7T0ZniqrS3NXM\nzvadbG/b7iSNth293ncEOqgpqokkisrCSgp8BeR788nz5EXmI5Mnn3xfPgXeAvK8eRR449bHTZZ0\nzHBkiWEI7AtPRmerzkAnu9p3Ra46Gjsb6Q52x0xdwS78QT9dwa6Y5fHLuoLddAeiPhvqwic+8jwF\nTs+lePGIIHgQ8SAQmfcgIILbK467Plyd1tNXTvwyEQ/FvmIqCiqpLKqksmgE1cUjqC4ZwciySioL\nR1BZVElZfpm13Zghk64uMfYHNqlql4jMBQ4GlqhqY9KRJiFbEsO+1hidi4Ih8Psh4Hde/QH31Q+B\ngLM8/k9FVQlqAH+oC7/6UbcdxWlPieqMzJ0PRS+LKgva5+dCGqIj2EaLv4EWfyPN/gZa/Y00+xtp\nDTTSEnCWdwU7KM8fwYjCEVQWjaCqqNKdr6SioIIRhSPweXxR2048dGmi10Rlopd5xIPP44tcheV5\n48bfjl4WNUZ3/LJsvfoKdzoX0hBBDRIMBfucT/TzCc9Hb6/f66OWR3qNjeotNvyzD88n6mU2Hd2n\npysxrAaOACYBT+N0YTFLVU9JMs6kZEtigH2vMTrTVJ2DfSgEoaA7774mOvhrKNMRD04g5KfF30hL\noJEWfwOtgUbaQg20BxtpDTpJRAkiHsEj7uTpafSPv2tsb3eURS8LaQh/yE8gFMAf9OMPOVMgGIjM\nh5fHLwuEnPfdwW684o1JKOHxI8I3KITF3LSQ4C63+PXhj0ff6BAKuQdzdQ/soT3Ph6/owgfh+K7A\now/C4X1E4usJIPF43/1Yr2gkpr3FHp+0FO0V32B1Xd+VlsTwlqoeJiLXAJ2q+isRWaWqcwYT7EBl\nU2IApzH67c0f0N6utHdAdxfk5UF+AeTnQV4+FOSDbx8fGiGkzll6/AE9FD3FrdNQbCLIol9rVhMB\njxe8XvB63FcfeDzg8/ZeF/0+lSehztVXkEAoEKnGC4QCvbq+Dl+1xM8PdH382XXkgB93xh19Zp6r\nIlc7oSBBVTwp+L0dN/G41HeiB/hFZD5wAXCquyxvoMHlulAI2tqgtdWZ2tpK6OiYyLb2TyNlOjuB\nltjPiQfy851ksS8kjZBCezt0tENbO3R22IF9qKhCMOBMA+WJSyZeN2n4fE5S6bW89zAIESKCT3z4\nPD4KfYXJfyET4fdDRyd0dnro7PTQ0eEjFHQSenySD58MRM8nOjEYzMlAfw5NFwH/H/BjVV0vIpOB\n+5PfZW7w+3uSQGsrdCQ4AI7Ir6Yz2EZDV9+N0RqCrk5nysWkEVLnu7e3WSLIZaGgM/n9/Ssfvjrx\nxSWL6AQSXhc+KIX/LlQBdV6VgS8PLwsvF/eKxyPOvMe5N6Bn3uMcHIXUXhmli9/vnER2doaTQd/J\nXt0rcpI4GRA3eSTD7kpydXTEJoLufjYfqCp1bR/SEUhtY3QkaeT3nNX5fFH/jD73bCHF/wiWCEwu\nk/4kD3fe4yY/j6dn8np6Dqgeb0+ZZA+wgYCbADp6kkEgiYP8YJz3xTSMx+B2sX0LMNEtLzgPqJUn\nFWUW6F0tBMFgctsSEcYV709d2wcENRDVQBX7lHBPw1vvxrhw+eiyAKICXdDW1kJ3sDPh/j3RVQJ7\nSCC+PpKIJYLkqUJXp4fOdg8dHV66Ojx4fUpJWZDikiAFhaGcOIPdl6iCBiEd9yfIXhJH9Bl6OBn0\nNwmoQsPuPDZ9UsTG9c7U0uyjpDRASVmQ0jLntaQswfvSIHn5qf2n7U/j80fA6cC7mWz9HcwVQ2en\nc/APT4mqhbJdZ7Cdpu56mv31BEL9rA+II57YBBIIDr9EEAhAe6uPjnYPne1eOju8UfMeOtxlne2e\nnnUdzrrOqHUdHV66Oj3k5YcoKgpRWByksDBEICC0tXppa/ERCkFJadBNFO4/cmmQYvefubjU/ccu\nCVJcFowcBIpLgxQVB/Hkbhuq2YP2Vg+bPi1i06c9SWDT+iI8XmX85A7GT+5g3OQOKioDtLd6aW1x\n/p7aol5bW3y0tfhobfHS3uLDmxeiNCpRlJb3JI6nHx6TlsbnjcCarLolaA+6u52Df3t7z2uyVwPZ\npNBbTGFRMbVF42gLtNDUvZsWfyMh7f+X0xD4u50pl3V3CW0tPtrbvM4/S6v7T+MekNtbnfn2Fl9k\nWZu7LOj3UFTiHHgLi4M9B/WiIIXFIWd5UYiyEQFGjumiqDjkrIuUC5dx5hOMVd8TZ7c4scTF2u7G\n2lSfx5a6QnddOFanTFenh4JCZ1+RqSTqfeQ7hGLex5crLA5mTXvVnoSCzhnzzm0F7NyWz46tBezc\nWsDuHfmogs+nePMUn0/x+UL4wvN5iten+PJCThlfuEx4XVRZd5nPF4psKy8/RF6e8+rLU/Ld1/Dy\nPf1+9ybgF7ZuLIg5+G9cX0Rrs4+xEzsZ5yaBI45tjCSCZKhCZ4cnLmn0JJFk9OeK4TM4VUl/I6q7\nbVX976T2mKREVwyBQE8CCCeB/jau7QtCGqLV30Szv57WQFPO9eOkCm0tXlqafDQ3+mhpynNfe6ZE\nB1UgcoZdXBp0z8oDkdfwsuLIsp6z8Vyp3gmFnGqqjjYvHe1e99UTNR+ePDHvO91y7VHL8vI0kihK\nygKUVwYoHxGgotJP+YgA5SP87jLnfVl5YFAHxERUobXZGznw79zmHPh3bnXmd+/Ip6Q8wMjR3Ywa\n08XI0V2MHN1NdW03Xq8S8AuBgDMF/R5nPmpZwO8hGLUsGLU88t4ftcyd9/s9+LudZf5uwd/twe93\nXgN+5/mRvLhkkeh99LKA38Om9UVs31JATW13JAGEp5Fjuob0ajCZNob+JIbngFbgXaKq7lT1R8kE\nmSwR0ZYWjbka6ErVqBD7gKAGafE30NRdT3ugZe8fSINQyDnQNzf2HODDB/yWRh/N4QN+o4/mpjxa\nm70UFDhn52UVAcorApS5B6qyCucAFV3FEj7w5xfkVgLMpOh2kPY2J7k2NfhobnB+R5HXxjxneWMe\nHW1eSsqiE0eA8vB8pZ+KqCRSUemP/D66OoVd23sO9ju3FbBja08SEFFGju5m5JguZxrdzcjRXYwa\n00VNbXfW/V6d24MlJlFEJ46+lnu9ytiJHew3oXPA30mk585Ej9e5kgoEnf+tYMCp/Rjo+V+6EsMa\nVZ09sFBST0R05crs+sPJVoGQnyZ/Pc3d9XQG0zPWcygEa1eVsfyZGjatL6K5yUdbs4+ikqBzkA+f\nfVa4B/0RAcoq/O6B300CFQF8efY7zTaBALQ2+WhqjEseDXk0NfZOKl6fklcQorPdS/Wo2DP+kWO6\n3PfdlJTtA3W6KRL9MGx+Qc8diHm+vd9yGwo/zxJyEkUo6LzGJBB3XTAIZx2XnsTwX8BfVfW5gWw4\n1SwxJKcr2Emzv56m7nr8ocFfYjU1+Hj52WpefLqGgsIQJ35lFwcc1ELZiACl5YGcqM82qaMKne0e\nuro8lI8IpLyKxOeLfUYi/LyDOxuJIXp9eFmmeX09B/x89yogz51P9W3me5KuvpJagBKc9gU/Gbpd\n1RLD4HUE2mjy76bZ30Aw1P+GrlAI3l9dwYtP1bDmzTKOPL6ZL53axPSZ3Xg9Hrd7hG4C6s+5dg6T\nXfILoLgIikuc17xB9rEQnUQ0PmkkSCLRf70xy2NWJN5+mMetDkr22YdUS3liEOfG+/GqumGwwQ2W\niOjkycrhh8NhhzlTTU2mo8pNqkpboJmuUCcePHjcTrsEj9vXjBcPHhobPDz1Jy/LHvdQVASnnw7z\n5kFpad/bDagff6g7MgXUee0OdeEPdaO53gOeSamCwthEkGtXnCKSsHfU+A7zMnnClK4rhncTDcs5\n1ERE779fefNNePNNWL0aqqqcBHH44c40cmSmo8x9oRCsXAmPPQavvQYnnABf/zrMmpWa7gYCoUAk\nWSRKIMk+o5EJ4X5sIs+G+Hq6bxCJ66JBBrZc1alL1pDzGgq6r6GeZZF1od7lNNS7OiXcC2imDlIi\nUFDQkwSKivfcJ9PAtx8/TkbPa/iAHb8seoruhK/XOkm8rr9dZIc7HtxTD6uJ3kd3nx7ufTVRd+uJ\n1oWlKzEsBu5Q1TcGsuFUi69KCgbho4+cJPHWW7BqFZSXE3NFMXp0BgPOMfX18OST8Pjjzj/v6afD\nKaf0fXWQLiENEQj56Q51EVA/QQ0QDAUI4fTmGdToKZiWK5CYBwH3NKXwoBbuNTQ8eT3emINX+KAX\ned+PdYIH1IOqgHqchskgdHWH6OwK0tkdpKs7SFcgSHd3iK7uIN2BYNSBq+c1ROyy/vzcRaCoyENZ\niZfSEg+lJR7yfd6EB+LoA2/88v4c5G3go97CScLr8aYlMawDpgJ1QBs9bQwHJxtwMvbWxhAKwccf\n9ySKt96CkpLYK4oxY4Yw4BwQvjr44x/hH/9wrg5OPx1mz87+zsjCZ9khDRHCSRwhDRAiQJCAOx+M\nJJEQzqsSJEig10E++sx/sHXD8Qf5/kzZdGALRI19ER78KPq9MyndgVBMovB6PZQWe6go91Je5qG8\nzGNPb2cBEUlLYpiYaLmq1g1kR4MlIlpXp+zs56iaqvDJJ06CCCeLggKYOhVGjICKCucKo7y8Z76i\nome+uDj7D479pQpNTbB1K2zZ4rxu3QqvvtpzdTBvHpSVDW1cnqiOAvPyYl/DnQeGEwBEVbmkeGyB\nXstIsKyf5XJ9PICBUO1JFLBv/c/sS9KSGLJF+Mnnpib49NOB91CoCnV1zmebmqC52ZkaG3vmm5p6\n1vn9scmjvNxJKH0lk7Iyp9qlpGToG9BUYfdu2LYt9sAfPfl8zhXTmDGw335ONdvBB6fv6sDr7fuA\nH573pvjpWmNMb8MiMYBz0K6rcw7i6dLd3ZMs4pNG/Pvwa7hrjvx8J0GUlDjJIpwwwq99zUeXLS7u\nOXAGg7BrV+ID/pYtsH27Uz584E80pautoLTUSZDxB3+rQjAmOwybxBC2cyds2uTUlWcLdbuxDieJ\n6K69w6/R6xKVC3f7UVDgHPCbm52rkvCZfvQBP7ysqGhov2dhIYwd61xFGWOy17BLDOB0qb1+vXMg\n3ZeEQu44Ce3OGXlBQaYjcvh8TjKqqbH6ZGNywbBMDOCcpW/Z4tSxm/TweKC21pmsbcCY3DFsE0NY\na6tz9dDfYTlN/9TUOFcJg+2ewBgz9IZ9YgCnoXbDBueBLTM4FRVOO8JQt18YY1LHEkOUhgbnzqV9\nYfS2oVZcDOPGDf1zDcaY1EsmMaT1pkIRuUdEtovIO3soc7uI/FNEVovIoanad2UlHHigHdwGIj8f\nJk2CmTPt52bMcJbuu83vBb7c10oRmQfsr6rTgEuBO1O58/x8mD7dOfu1O2j65vU6VUazZkF1daaj\nMcZkWlqf0VXVV/rqUsN1GrDELfu6iFSISK2qbk9lHLW1zi2f69c7t4Aah4jTI+2YMbnX3bExJn0y\nfTgYC2yMer/ZXZbSxABOA+qMGU7XEbt3Ow+RDVf5+c6DaaNGZc/zEcaY7JHpxDAgN910U2R+7ty5\nzJ07d0Cf93icM+SRI50H48JJwp87QwAkLS/PaXeprBz6rrSNMUNn+fLlLF++fFDbSPtdSW5V0pOJ\nuukWkTuBF1X1Iff9OuDziaqSBnpXUn+pQkuL0xdRU1N2da8xWJYMjDHJ3JU0FFcM4k6JPAFcDjwk\nIkcDjaluX9gbkZ4eU4NB5/mHXK5qystzqomqqiwZGGOSk9YrBhFZCswFqnHaDW4E8nEG+rnbLXMH\ncDLOIEAXqepbfWwrLVcMfQlXNdXXZ/+T1D5fz5WB3WZqjIlmD7ilSXOzkyQaG7Onqik6GZSW2u24\nxpjELDGkWTDoPFG9e7fTL9NQ8/mcaqLwlYElA2PM3lhiGEJdXT13NaW6qik8fKXH40zl5ZYMjDHJ\nscSQIS0tPaPJeTyxB/Xw/ECW2cHfGJMqlhiMMcbEyLpO9IwxxuQeSwzGGGNiWGIwxhgTwxKDMcaY\nGJYYjDHGxLDEYIwxJoYlBmOMMTEsMRhjjIlhicEYY0wMSwzGGGNiWGIwxhgTwxKDMcaYGJYYjDHG\nxLDEYIwxJoYlBmOMMTEsMRhjjIlhicEYY0wMSwzGGGNiWGIwxhgTwxKDMcaYGJYYjDHGxLDEYIwx\nJoYlBmOMMTEsMRhjjImR9sQgIieLyDoR+VBEvp9g/edFpFFE3nKnH6Y7JmOMMX3zpXPjIuIB7gC+\nAGwB3hCRZaq6Lq7oS6r6r+mMxRhjTP+k+4rhSOCfqlqnqn7gQeC0BOUkzXEYY4zpp3QnhrHAxqj3\nm9xl8Y4RkdUi8pSIHJjmmIwxxuxBWquS+ulNYIKqtovIPOBxYHqGYzLGmGEr3YlhMzAh6v04d1mE\nqrZGzT8jIv8rIlWqWh+/sZtuuikyP3fuXObOnZvqeI0xJqctX76c5cuXD2oboqqpiSbRxkW8wAc4\njc9bgRXAfFV9P6pMrapud+ePBB5W1UkJtqXpjNUYY/ZFIoKqDqgdN61XDKoaFJErgOdw2jPuUdX3\nReRSZ7XeDZwhIt8G/EAH8I10xmSMMWbP0nrFkEp2xWCMMQOXzBWDPflsjDEmhiUGY4wxMSwxGGOM\niWGJwRhjTAxLDMYYY2JYYjDGGBPDEoMxxpgYlhiMMcbEsMRgjDEmhiUGY4wxMSwxGGOMiWGJwRhj\nTAxLDMYYY2JYYjDGGBPDEoMxxpgYlhiMMcbEsMRgjDEmhiUGY4wxMSwxGGOMiWGJwRhjTAxLDMYY\nY2JYYjDGGBPDEoMxxpgYlhiMMcbEsMRgjDEmhiUGY4wxMSwxGGOMiWGJwRhjTIy0JwYROVlE1onI\nhyLy/T7K3C4i/xSR1SJyaLpjMsYY07e0JgYR8QB3AF8GZgHzRWRGXJl5wP6qOg24FLgznTElsnz5\n8qHe5aDkWryQezHnWrxgMQ+FXIs3Wem+YjgS+Keq1qmqH3gQOC2uzGnAEgBVfR2oEJHaNMcVI9d+\n2bkWL+RezLkWL1jMQyHX4k1WuhPDWGBj1PtN7rI9ldmcoIwxxpghYo3PxhhjYoiqpm/jIkcDN6nq\nye77hYCq6k+jytwJvKiqD7nv1wGfV9XtcdtKX6DGGLMPU1UZSHlfugJxvQFMFZGJwFbgbGB+XJkn\ngMuBh9xE0hifFGDgX8wYY0xy0poYVDUoIlcAz+FUW92jqu+LyKXOar1bVZ8WkVNE5COgDbgonTEZ\nY4zZs7RWJRljjMk9w6rxWUTuEZHtIvJO1LJKEXlORD4QkWdFpCKTMcYTkXEi8oKIvCci74rId9zl\nWRm3iBSIyOsissqN90Z3eVbGG01EPCLylog84b7P6phF5FMRedv9Wa9wl2VtzCJSISKPiMj77t/z\nUVke73T3Z/uW+9okIt/J8pivEpE1IvKOiPxeRPKTiXdYJQbgXpyH7aItBP6qqgcALwDXDnlUexYA\nvquqs4BjgMvdhwSzMm5V7QJOUNU5wKHAPBE5kiyNN86VwNqo99kecwiYq6pzVPVId1k2x/xL4GlV\nnQkcAqwji+NV1Q/dn+1hwOE4Vd1/JEtjFpH9gP8DHKaqB+M0FcwnmXhVdVhNwETgnaj364Bad340\nsC7TMe4l/seBL+ZC3EAxsBL4TLbHC4wD/gLMBZ7Ihb8NYD1QHbcsK2MGyoGPEyzPyngTxHkS8HI2\nxwzsB9QBlW5SeCLZY8Vwu2JIZJS6d0Gp6jZgVIbj6ZOITMI5C38N5xedlXG7VTKrgG3AX1T1DbI4\nXtcvgGuA6Ea3bI9Zgb+IyBsi8k13WbbGPBnYJSL3ulUzd4tIMdkbb7xvAEvd+ayMWVW3AD8HNuA8\nKNykqn8liXgtMfSWla3xIlIKPApcqaqt9I4za+JW1ZA6VUnjgCNFZBZZHK+IfAXYrqqrgT3dFp01\nMbuOVaea4xScKsbjyd6fsw84DPi/bsxtOFUc2RpvhIjkAf8KPOIuysqYRWQEThdDE3GuHkpE5FyS\niNcSA2wP980kIqOBHRmOpxcR8eEkhftVdZm7OOvjVtVmYDlwMtkd77HAv4rIJ8ADwIkicj+wLYtj\nRlW3uq87caoYjyR7f86bgI2qutJ9/wecRJGt8UabB7ypqrvc99ka8xeBT1S1XlWDOO0hnyWJeIdj\nYhBizwqfAC505y8AlsV/IAssAtaq6i+jlmVl3CJSE77rQUSKgC8B75Ol8QKo6nWqOkFVp+A8hPmC\nqp4HPEmWxiwixe5VJCJSglMH/i5Z+nN2qzI2ish0d9EXgPfI0njjzMc5YQjL1pg3AEeLSKGICM7P\neC3JxJvpBpMhbpxZCmwButwf4kU4DTV/BT7AeRBvRKbjjIv5WCAIrAZWAW/hnIFXZWPcwEFujKuB\nd4AfuMuzMt4E8X+ensbnrI0Zp84+/DfxLrAwB2I+BKc3hNXAY0BFNsfrxlwM7ATKopZlbczAjTgn\nYu8Ai4G8ZOK1B9yMMcbEGI5VScYYY/bAEoMxxpgYlhiMMcbEsMRgjDEmhiUGY4wxMSwxGGOMiWGJ\nwWQ1EamK6vp4q4hsinrfr4GmxOlufdpeylwmIvGjCyYb82lujKvdLpAv3kv5E9weaBOtGy0iT7nb\nek9EHneXjxORBxJ9xpjBsucYTM4QkRuAVlX97wTrRLPgj1lE8nF6PT1MVbe7/exMVNWP9vCZW4Bd\nGvtke3jdb3G6Y/i1+362qq5JU/jGAHbFYHJLpCsTEdnfPYP+nYisAUaLyF0iskKcAYJ+GFX2ZRE5\nWES8ItIgIre6Z+B/F5Eat8wt0jMI0stumdfFGVTmaHd5sYg86l4FPOL2anpwXIzhQVAaAVTVH04K\nIjJKRP7gxviaiBwpIlOAbwLfc6+Cjo7b3hicfoZwt7cm6vuvcucXSc9gMjtF5Fp3+ffd77A6+udh\nzN5YYjC57ADg56o6W50O5b6vzoA1hwIniTOgUbwK4EVVDXdf3mc1j6oeBfz/ON0MgDMIylZVnQ3c\n4u4n/jM7cbodqBNnBK2z3X5rAG4HfurG+A2cMdA/AX4L/ExVD1PV1+I2eQewRET+KiLXup2gRXbn\n7vNidXos/RpOB2mLRWQeMMH9DnOAYxMkHWMSssRgctnHqroq6v25IvImTl9NM4ADE3ymXVWfc+ff\nBCb1se3HospMdOePAx4EUNV3cDqB60VVL8Lp6fIN4PvAXe6qLwJ3umf6jwMVIlKwpy+oqs8AU4B7\n3O/zlohUxpcTZ2yDR4Bvq9Mv/0nAySLyFs7PY39gevznjEmkX413xmSptvCMiEwFvgMcoaotbrfZ\nhQk+0x01H6Tv/4GufpTpc+wGt8pnjdtAvBb4llv+M+p0idyzEdnTEBCgqg04vXs+ICLP4CSotXHF\n7gKWqupLUbH9h6reu8eNG5OAXTGYXBZ9RC0HmoFWERlD77G9E31moP6OUwWEiBwEzOy1cZEycQbM\nCZuDM9wiOEOH/p+osoe4sy1u/L2DFTlRRArd+XKcXlU3xJW5EvCp6i+iFj8LXOJeSSAiY0Wkup/f\n0wxzdsVgclnkLiRVfUtE3sfpcrgOeCVROfo32lZfZX6FU3+/BueMfS3QFFdGgGtF5G6gA2ilpx3j\nCuDXInIR4AVexEkUy4BHRORrwOVx7QyfAe4QkW6cE7n/q6pvi8j+UWWuBtrcKioF7lDVRW4by2vu\nFUkzcA6wux/f3wxzdruqMf0kIl6cM/Mut+rqWWCaqoYyHJoxKWVXDMb0XynwfNSDdd+ypGD2RXbF\nYIwxJoY1PhtjjIlhicEYY0wMSwzGGGNiWGIwxhgTwxKDMcaYGJYYjDHGxPh/DSvo1VGuLWMAAAAA\nSUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f85b70ae208>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_learning_curve(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This shows a typical learning curve: for very few training points, there is a large separation between the training and test error, which indicates **over-fitting**.  Given the same model, for a large number of training points, the training and testing errors converge, which indicates potential **under-fitting**.\n",
    "\n",
    "As you add more data points, the training error will never increase, and the testing error will never decrease (why do you think this is?)\n",
    "\n",
    "It is easy to see that, in this plot, if you'd like to reduce the MSE down to the nominal value of 1.0 (which is the magnitude of the scatter we put in when constructing the data), then adding more samples will *never* get you there.  For $d=1$, the two curves have converged and cannot move lower. What about for a larger value of $d$?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEPCAYAAABGP2P1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd8XNWZ8PHfM0VdGhXbcpeEC8Z26DWBYCAhmGxCQoBg\nMBCSXVjKG5YQNoZQDISQbEjYALuUhGaIgQAJLRRnAYcWMM2AsU2V5SZLltX7lPP+cWdGM6MZaSRr\nqp5vPvOZO/eeufNoTO4zp9xzxBiDUkopFWBLdQBKKaXSiyYGpZRSYTQxKKWUCqOJQSmlVBhNDEop\npcJoYlBKKRUmoYlBRHJF5E0ReU9EPhSRq2OUu1lEPhWRtSKybyJjUkopNTRHIk9ujOkTkaOMMd0i\nYgdeE5FnjTFrAmVEZDEwyxgzR0QOAW4HDk1kXEoppWJLeFOSMabbv5mLlYgi76g7AVjhL/sm4BKR\nykTHpZRSKrqEJwYRsYnIe8AO4O/GmLciikwDtoS83ubfp5RSKgWSUWPwGWP2A6YDh4jI/ER/plJK\nqdFLaB9DKGNMu4i8BBwHrA85tA2YEfJ6un9fGBHRSZ2UUmoUjDEykvIJTQwiMgFwG2PaRCQf+Drw\nq4hiTwIXAA+LyKFAqzGmIdr53t729pjGd/8H99PY1UjR60Wce8m5UctMLZ7KlOIpY/q5u2v58uUs\nX7481WGMSKbFnGnxgsacDJkWL4DIiHICkPimpCnASyKyFngTeN4Y84yInCsi5wAYY54BakXkM+AO\n4PwExxRU7apmU+umIcu4fe7kBKOUUmki0cNVPwT2j7L/jojXFyYyjliqS6upba1lIQtjlnF7NTEo\npcaXcX3n89TiqbT0tPClg74Us0w61hgWLVqU6hBGLNNizrR4QWNOhkyLd7QkUxbqEREz1n0MAN9/\n9Ptcs+ga5k2YF/W40+5k78q9x/xzlVIqGUQkvTqfM0F1qdXPECsxeHyeJEekxqPq6mrq6upSHYbK\nYFVVVWzatGlMzqWJoXToDmhjDG6vG6fdmbyg1LhTV1dHptTeVXoazeijWMZ1HwNATWmNjkxSSqkQ\n4z4xVJdWs6lt05BldGSSUmo8GfeJocpVxZa2LXh93phl+r39SYxIKaVSa9wnhnxnPmX5ZdR31scs\no01JSu2e8847j+uvv37My6rEGPedzzBwB/T0kulRj2tTkhrPampquOuuuzj66KNHfY7bbrstIWVV\nYoz7GgMMPzJJawxKxeb1xm6GHU+y6XvQxMDA1BixaB+DGq/OPPNMNm/ezLe+9S1KSkq48cYbqaur\nw2azcffdd1NVVcUxxxwDwCmnnMKUKVMoKytj0aJFrF8/MIny2WefzVVXXQXAP/7xD2bMmMHvfvc7\nKisrmTZtGvfee++oyjY3N/Otb30Ll8vFIYccwpVXXskRRxwR9W/p6+vjjDPOYMKECZSVlXHIIYew\nc+dOAFpaWvjhD3/ItGnTqKio4MQTTwy+7w9/+ANz5sxhwoQJfOc736G+fqDZ2Waz8b//+7/MnTuX\nuXPnArBx40aOPfZYKioq2GuvvXjkkUeC5Z955hkWLFhASUlJ8O9KR5oYiKPGoE1JapxasWIFM2fO\n5Omnn6a9vZ2f/vSnwWMvv/wyGzdu5Pnnnwfg+OOP5/PPP6exsZH999+f008/PeZ5d+zYQUdHB9u3\nb+ePf/wjF1xwAW1tbSMue/7551NcXExjYyP33nsv9913X8zx/Pfddx/t7e1s27aN5uZmbr/9dvLz\n8wFYunQpPT09bNiwgcbGRi6++GIAXnzxRS6//HIeffRR6uvrmTlzJqeeemrYeZ944gnWrFnD+vXr\n6e7u5thjj2Xp0qU0NTXx0EMPcf7557Nx40YA/vVf/5U//OEPtLe3s27dut1qnkskTQwMnxg8Po/e\nfKRSSmRsHqMV+d+/iHDNNdeQn59Pbm4uAD/4wQ8oKCjA6XRy1VVX8f7779PR0RH1fDk5OVx55ZXY\n7XYWL15MUVERH3/88YjK+nw+/vKXv3DttdeSm5vLXnvtxVlnnRXzb3A6nezatYtPPvkEEWG//faj\nqKiIHTt28Pzzz3PHHXdQUlKC3W4P1jpWrlzJj370I/bZZx+cTic33HAD//znP9m8eXPwvJdffjml\npaXk5uby9NNPU1NTw5lnnomIsM8++/C9730vWGvIycnho48+oqOjA5fLxb777hv/P0ISaWIAKvIr\n8Bovrb2tMctoP4NKJWPG5jGWpk8fGKzh8/lYtmwZs2fPprS0lJqaGkSEpqamqO+tqKjAZhu4/BQU\nFNDZ2Tmisjt37sTr9YbFMWPGjGinAKxmsW984xuceuqpTJ8+nWXLluH1etmyZQvl5eWUlJQMes/2\n7dupqqoKvi4sLKSiooJt2wbWEgv9/Lq6Ot544w3Ky8spLy+nrKyMlStX0tBgLTHz2GOP8be//Y2q\nqiqOOuoo3njjjZjxppImBqxfPzWlNdS2aD+DUpFiNc2E7l+5ciVPPfUUL774Iq2trWzatAljTEJr\n2hMnTsThcLB169bgvi1btsQsb7fbufLKK/noo494/fXXeeqpp1ixYgUzZsygubmZ9vb2Qe+ZOnVq\n2BxWXV1d7Nq1KywZhH4PM2bMYNGiRTQ3N9Pc3ExLSwvt7e3ceuutABxwwAE8/vjj7Ny5kxNOOIFT\nTjllt76DRNHE4DfcHdDaz6DGq8mTJ/PFF1+E7Yu84Hd0dJCbm0tZWRldXV1cdtllYzp3TzQ2m40T\nTzyR5cuX09PTw8aNG1mxYkXM8qtXr2bdunX4fD6KiopwOp3Y7XYmT57M4sWLOf/882ltbcXj8fDK\nK68AsGTJEu655x4++OAD+vr6uPzyyzn00ENj1kz+5V/+hU8++YQHHngAj8eD2+3m7bffZuPGjbjd\nblauXEl7ezt2u53i4mLsdntCvpvdpYnBr8pVpUNWlYpi2bJlXHfddZSXlwdH0URe9M8880xmzpzJ\ntGnTWLhwIV/+8pdH9BkjSSKhZW+55RZaW1uZMmUKZ511FqeddlqwzyPSjh07OOmkk3C5XCxYsICj\njjqKpUuXAnD//ffjcDiYN28elZWV/P73vwfgmGOO4brrruPEE09k2rRp1NbW8tBDD8WMu6ioiFWr\nVvHQQw8xdepUpk6dyrJly+jv7w9+Tk1NDaWlpdx5552sXLky7r87mcb9egwBqzet5i8b/8LNx90c\n9fjkoslMK5mWsM9X45t/zvxUh5Hxli1bRkNDA/fcc0+qQ0m6WP8NjWY9Bq0x+NWU1lDXGns+fO1j\nUCr9fPzxx3z44YcArFmzhrvuuivsHgQ1Ojolht+0kmns7N5Jr6eXPEfeoOPalKRU+uno6GDJkiXU\n19dTWVnJpZdeyre+9a1Uh5XxNDH4OWwOphVPY0vbFuZUzBl0XDuflUo/Bx54IJ9++mmqw8g62pQU\nYqipMbTGoJQaLzQxhBjqDmivz4vP+JIbkFJKpYAmhhDD3cugHdBKqfFAE0MInUxPKaU0MYSpdlWz\nuW1zzCYj7WdQSo0HmhhCFOYUUpJbwo7OHVGPa41BqfgF1lIIWLhwIS+//HJcZUdKlwMdWzpcNUJg\nmc+pxVMHHdM+BqVGJnTKiHXr1sVddij33Xcff/zjH4PzGYEuBzrWElpjEJHpIvKiiHwkIh+KyI+j\nlDlSRFpF5F3/44pExjQcHbKqVHozxiR8gr5kSdflQBPdlOQBfmKMWQAcBlwgIvOilHvZGLO///GL\nBMc0pKE6oLUpSY03//Vf/8XJJ58ctu+iiy7iP/7jPwC49957mT9/PiUlJcyePZs777wz5rlqamp4\n8cUXAejt7eUHP/gB5eXlLFy4kLfeeius7K9//Wtmz55NSUkJCxcu5PHHHwesZTPPO+88/vnPf1Jc\nXEx5eTkQvhwoDL8c5x133MHcuXMpLy/nwgsvjBnzW2+9xUEHHYTL5WLKlClhK9i9+uqrfOUrX6Gs\nrIyqqqrgzK7t7e2ceeaZTJo0iZqamrAmrvvuu4/DDz+cn/zkJ0yYMIFrrrkGgLvvvpv58+dTUVHB\n4sWLwxYCuvjii6msrMTlcrHPPvuELZmaKAlNDMaYHcaYtf7tTmADEG0murRJ/9Wl1THnTNIagxpv\nTj31VJ599lm6uroAa0GeRx55JLhsZ2VlJc888wzt7e3cc889XHzxxaxdu3bY8y5fvpza2lpqa2t5\n/vnnue+++8KOz549m9dee4329nauvvpqli5dSkNDA/PmzeP222/nsMMOo6Ojg+bm5kHnjmc5zr/9\n7W+88847vP/++/z5z39m1apVUeMMJMG2tjY+//zz4PoJdXV1HH/88Vx00UU0NTWxdu3a4GpsF154\nIR0dHWzatInVq1ezYsWKsEn93nzzTWbPnk1jYyM///nPeeKJJ/jVr34VXKfhiCOOYMmSJQCsWrWK\nV199lc8++4y2tjb+/Oc/U1FRMez3u7uS1scgItXAvsCbUQ4fJiJrgW3ApcaYxKfEGIa6l0H7GFSq\nyDVj89vJXD2yGVxnzpzJ/vvvz1//+leWLl3KCy+8QGFhIQcddBAAixcvDpY94ogjOPbYY3nllVeG\nXbLykUce4fbbb8flcuFyufjxj3/MddddFzz+ve99L7h98skn88tf/pI1a9bENQ9S6HKcADfccANl\nZWVs3ryZmTNnAnDZZZdRXFxMcXExRx11FGvXruXYY48ddK6cnBw+++wzdu3aRUVFBQcffDAADz74\nIF//+teDiaKsrIyysjJ8Ph8PP/wwH3zwAQUFBVRVVXHJJZdw//33c/bZZwMwbdo0zj//fAByc3O5\n4447uOyyy5g7dy5gzRB7/fXXs2XLFpxOJx0dHaxfv56DDz6YPffcc9i/fywkJTGISBHwKHCRv+YQ\n6h1gpjGmW0QWA48Dc5MRVzQTCybS5+mjrbcNV54r7JgxBo/Pg8OmffYquUZ6QR9LS5Ys4cEHH2Tp\n0qU8+OCDnHbaacFjzz77LNdeey2ffPIJPp+Pnp4e9t5772HPuX379rBV0EKXzwRYsWIFN910E5s2\nbQKsldNiLRMa7dwHHHBA8HXocpyBxFBZWRk8PtSyonfddRdXXnkl8+bNY4899uCqq67im9/8Jlu2\nbGHWrFmDyjc1NeHxeIKfE/jbQpcCjRx9VVdXx0UXXcQll1wCDPShbNu2jaOOOooLL7yQCy64gM2b\nN3PiiSdy4403UlRUFNd3MVoJv8KJiAMrKdxvjHki8nhoojDGPCsi/ysi5caYQXXEO357R3D7gMMO\n4MAvH5iIeKkqrWJT2yb2ydtn0HG3162JQY0rJ598Mj/96U/Ztm0bf/3rX4PrFPf393PSSSfxwAMP\ncMIJJ2Cz2fjud78b17oSU6ZMYcuWLey1114AYctnbt68mXPOOYeXXnqJww47DID99tsveN7hOp7j\nWY4zXrNmzQoupvPYY49x0kkn0dzczIwZM1izZs2g8hMmTMDpdFJXV8e8efOCf9u0aQMt6JHxz5w5\nkyuuuCLYfBTpwgsv5MILL6SpqYmTTz6Z3/zmN8G+iWhWr17N6tWrR/qnhknGfQx3A+uNMb+PdlBE\nKkO2D8ZaPGhwwyFw7iXnBh+JSAoBgSGr0Wg/gxpvJkyYwJFHHsnZZ5/NHnvsEWzO6O/vp7+/nwkT\nJmCz2Xj22WdjttVHOuWUU7jhhhtobW1l69atwTWRwbqQ22w2JkyYgM/n45577gkb6lpZWcnWrVtx\nu6P/f3Gky3EO5U9/+lOwpuJyuRARbDYbp59+Oi+88AKPPvooXq+X5uZm3n//fWw2G6eccgo///nP\n6ezspK6ujptuuokzzjgj5mece+65/PKXvwx2Kre1tfHoo48C8Pbbb7NmzRo8Hg/5+fnk5eVhsw19\n2V60aBHLly8PPkYj0cNVvwKcDhwtIu/5h6MeJyLnisg5/mInicg6EXkP+G/g+4mMKR41ZTUxE4P2\nM6jx6LTTTuOFF14IdjqDtYzlzTffzMknn0x5eTkPPfQQJ5xwQsxzhP5Svvrqq5k5cyY1NTUcd9xx\nnHnmmcFje+21F5dccgmHHnookydP5qOPPuLwww8PHj/66KNZsGABkydPZtKkSYM+Z6TLcQ5VA3nu\nuedYsGABJSUlXHzxxTz88MPk5uYyY8YMnnnmGW688UbKy8vZb7/9+OCDDwC4+eabKSgoYI899uCr\nX/0qS5cuDfYvRPOd73yHZcuWceqpp1JaWsree+/Nc889B1gjnP7t3/6N8vJyampqmDBhApdeemnM\nc40VXdozipdqX+LJT57kpm/cNOjY1OKpTCmekpQ41PihS3uq3aVLeybYkPcyaFOSUirLaWKIYnrJ\ndBq6GqI2G+lNbkqpbKeJIQqn3cmUoilsbts86Jj2MSilsp0mhhhiNSdpU5JSKttpYoghVmLw+Dza\nSaiUymqaGGKoKa2hrm3wnEmBu5+VUipb6S28MVSXVvPQRw9FPeb2uXHanUmOSGWzqqqqrJlKWqVG\n5LQiu0MTQwxVrirqWuvwGR82Ca9Y9Xv7KXAWpCgylY0CcwIplQ60KSmG4txiCpwFNHY1DjqmQ1aV\nUtlME8MQakqjT42hI5OUUtlME8MQYg5Z1RqDUiqLaWIYQqzEoDe5KaWymSaGIehNbkqp8UgTwxCq\nS6upba0dtF+bkpRS2UwTwxAmFU6i291NR19H2H69+1kplc00MQzBJjaqSqui3gGt/QxKqWyliWEY\nOmRVKTXeaGIYRpWrSvsZlFLjiiaGYejIJKXUeKOJYRh6L4NSarzRxDCMma6Z1HfWD5pqW5uSlFLZ\nShPDMHLsOVQWVrKlbUvYfm1KUkplK00McYjWnKQ1BqVUttLEEIfq0mo2tW0K26d9DEqpbKWJIQ7V\npdXUtoQPWfUZH16fN0URKaVU4mhiiEO0GgNoP4NSKjtpYohDtauauta6QfMjaT+DUiobaWKIgyvP\nRa4jl6buprD9WmNQSmUjTQxxqnYNnoJbO6CVUtkooYlBRKaLyIsi8pGIfCgiP45R7mYR+VRE1orI\nvomMabSqSqt0yKpSalxwJPj8HuAnxpi1IlIEvCMiq4wxGwMFRGQxMMsYM0dEDgFuBw5NRDA2sfKg\nz/hG/N5os6xqU5JSKhsltMZgjNlhjFnr3+4ENgDTIoqdAKzwl3kTcIlI5VjHIiLMKp+FK881qvdH\nG5mkNQalVDZKWh+DiFQD+wJvRhyaBoTON7GNwcljdz+bmtIaSnJLKM8vH9U5ot39rH0MSqlslOim\nJAD8zUiPAhf5aw6jcsdv7whuH3DYARz45QPjet9M10zK8ssAKMktwSa2ETcnTS6aTHtfO139XRTm\nFALalKSUSj+rV69m9erVu3UOSfTaxSLiAJ4GnjXG/D7K8duBl4wxD/tfbwSONMY0RJQzb297e8Sf\nP71kOpVF4S1TtS21NPc0j/hcp/3lNK444grmT5wf3LfP5H1w2JKSX5VSasREBGOMjOQ9yWhKuhtY\nHy0p+D0JnAkgIocCrZFJYbQmF00elBSAYO1hpKpLBw9Z1X4GpVS2SehPXRH5CnA68KGIvAcY4HKg\nCjDGmDuNMc+IyPEi8hnQBZw9Fp89sXAi00qid1W4cl3YbfYRz3VU7Yrez5DvzB9tmEoplXYSmhiM\nMa8B9jjKXTiWn1ueX85M18yYx0UEV65rxM1JNaU1rPpiVdg+7WdQSmWbrLvz2ZXnorq0ethyoxmd\nVF1qzZkUSpuSlFLZJqsSQ1FOEXuU7YHI8P0sJbkl2G3DVmbCzHDNYGvH1rBlPrXGoJTKNlmTGAqc\nBcwunx28u3k4IkJpXumIPiPPkcfEgolsa98W3Kf3Miilsk1WJIZcRy5zKuaMuAZQljfy0UmRd0Br\nU5JSKttkfGJw2p3MrZg7qnsJRtOcFHkHtDYlKaWyTUYnBofNwdyKueTYc0b1/tE0Jw1KDF73oAV8\nlFIqk2VsYrDb7MypmEOeI2+3zjPS5qRos6yGdkYrpVSmy8jEYBMbs8pmUeAs2O1zleSWjKgZKlBj\nCK0laAe0UiqbZFxiEBFqymoozi0es/ONpDmpNK8Um9jY1bMruE/7GZRS2STjEkOVq2rE/QLDGenc\nSdH6GRKh293N9o7tCTm3UkrFMmRiEBG7iNyYrGCGM8M1g4qCijE/b3FO8YiakyL7GcayxmCMobmn\nmY1NG9mwcwP1HfW097WP2fmVUmo4Q14NjTFeETk8WcEMZ1LhpIScN9Cc1NTdFFf56tJq6toGpsYY\niz4Gj8/Dzq6d7OzeOagGUt9RT0luyW5/hlJKxSOen8nviciTwCNYs58CYIz5S8KiSoGy/LIRJYY3\ntr4RfL07TUnd7m4auxpp7mmOOey1s7+Tjr6OMetXUUqpocSTGPKAXcDRIfsMkFWJIdCcFM/Q00F3\nP4+wKckYQ0tvC41djXT1dw3/BqC+s14Tg1IqKYZNDMaYMVkfId2NpDlpStEUWnpa6HH3kO/Mj7vG\n4Pa6aepuitpcNJyOvg46+zspyika0fuUUmqkhh2VJCLTReSvItLofzwmItOTEVyyxTsVt91mZ4Zr\nRrCfwePzDLmGdFd/F7UttXzY+CHbO7aPuumpvqN+VO9TSqmRiGe46j1Yy29O9T+e8u/LOkU5RXGP\nTopczS3yYh86umhj08Yh+xDi1d7XHnfTk1JKjVY8iWGiMeYeY4zH/7gXmJjguFJCROK+p6GmLPqQ\nVbfXTX1HPR82fkhtS+2YX8h3dO4Y0/MppVSkeBLDLhFZ6r+nwS4iS7E6o7NSvHMnVbmqwhJDe1/7\nmDQXDae1t5Ued09Czq2UUhBfYvghcAqwA6gHTgKytkO6OLcYp905bLnq0mpqW2uDr+s76sekuSge\n9Z3a16CUSpwhG9RFxA6caIz5dpLiSQuleaXs7No5ZJkqVxVb27fi9XlHvKbD7mrpaaG3uHe3Z5ZV\nSqlohqwxGGO8wJIkxZI24hmdlO/Mpzy/PGW/3nWEklIqUeJpSnpNRG4VkSNEZP/AI+GRpVBRTlHc\nzUmRazMkS0tvC32evpR8tlIqu8UzNnNf//O1IfsM4XdCZ52yvDIauxqHLBPoZzh8ZvKnkzLGsKNz\nB1WlVUn/bKVUdhuuj8EG3GaM+XOS4kkbZfnxJYb1O9cnKaLBdvXsYkrxlFEvbaqUUtEM18fgA/4z\nSbGklXiak1LZlAQDtQallBpL8fQx/J+I/FREZohIeeCR8MjSwHD3NETe/ZwKTd1NCbtnQik1PsXT\nx/B9//MFIfsMsMfYh5NehmtOKs8vx2d8tPS0jHgVuLESqDXMcM1IyecrpbLPsDUGY0xNlEdcSUFE\n7hKRBhH5IMbxI0WkVUTe9T+uGOkfkEhFOUVDtt+LyKDV3FKhqbsprunClVIqHvHMrlogIleIyJ3+\n13NE5F/iPP89wDeGKfOyMWZ//+MXcZ43aYarCVSVVoWtzZAKPuOjobMhpTEopbJHvLOr9gNf9r/e\nBsR1ATfGvAq0DFNM4jlXqgzbz1BaTW1L7ZBlkqGxq1FrDUqpMRFPYphljPkvwA1gjOlmbC/mh4nI\nWhH5m4jMH8PzjonCnMIhm5NqSmtSXmMAq9Yw3PBapZSKRzydz/0iko/V4YyIzALG6pbbd4CZxphu\nEVkMPA7MjVV4+fLlwe1FixaxaNGiMQpjaGX5ZTGbaqpLq6lrrUtKHMNp7GqksrAy6XM3KaXSx+rV\nq1m9evVunUOGmw1URL4OXAHMB1YBXwF+YIyJ65NFpAp4yhizdxxla4EDjDHNUY6ZZMxcGk1Xfxcb\nmzZGPebxeTjy3iN54cwX0mJSu6nFU5lSPCXVYSil0oSIYIwZUStPPKOS/g6cCPwAeBA4MN6kEIiL\nGE1PIlIZsn0wVqIalBRSbajmJIfNwbTiaWxu25zkqKJr7GoccplRpZQaTlzrWBpjdgF/G+nJRWQl\nsAioEJHNwNVAjnVKcydwkoich9V/0cPAPRNppzy/POZdxtVl1o1ucytitoIljcfnYWfXTiqLKocv\nrJRSUcS3wPEoGWNOG+b4/wD/k8gYxkpZflnsxDAGd0D7jI/2vnaae5rDHp39nZw0/yRK80rjPldD\nVwMTCydik3jGFiilVLiEJoZsUuAsINeRG3Wq6+rSal7d8uqg/W6vm5beFpp7mtnVs4uWnpZBz4Fj\nrb2tFOYUUp5fTnleufWcX05DVwPX/OMafnfs7xCJr5nQ7XXT1N3EpMJJu/13K6XGn3g6n2cBW40x\nfSKyCNgbWGGMaU1CfKFxpKzzOWBb+7aotYYNOzdw0fMXsd/k/QYu+r0tdPV3UZZfFrzQl+WXUZFf\nEbzohz7K8sqiTtrn9rr50ZM/4ptzv8n3F8Tf0pZjz2HhpIVxJxOlVHYaTedzPIlhLXAgUA08AzwB\nLDDGHD/KOEclHRJDt7ubDTs3DNrv8Xl46pOnKHQWhl3sS3JLxqQ5Z3PbZn745A+57fjbmFMxJ+73\nzXTNZGLhxN3+fKVU5kpUYnjXGLO/iFwK9BpjbhGR94wx++1OsCOVDokBYF3jupSsnPb0J09z3/v3\ncf937497WGyuI5cFExdorUGpcSwhw1UBt4gsAc4CnvbvG37dyyw13BQZifLNOd9kz4o9+d0/fxf3\ne/o8fTT3pN3oX6VUmosnMZwNHAZcb4ypFZEa4P7EhpW+yvNTsxSFiLDs8GW8ue1NXqx9Me731XfW\nJzAqpVQ2GrYpKV2kS1MSwEeNH9Hr6U3JZ69rXMdPVv2EFd9ZweSiyXG9p6asJmUJTSmVWglpShKR\nfxGR90SkWUTaRaRDRNpHH2bmS9WiPAALJy1kycIlXPXSVXh93rjeo8t/KqVGIp6mpP/G6l+oMMaU\nGGOKjTElCY4rraWqnyHgzL3PxCY27ll7T1zle9w9tPQMN/u5UkpZ4kkMW4B1adOOkwbynfkpnTDP\nbrNz7VHX8sj6R1i7Y21c79Fag1IqXvEMVz0IuA74ByHTbRtj4h8eMwbSqY8BYHvHduo7Utux+4+6\nf3Dj6zey8sSVFOcWD1t+dvlsXHmuJESmlEoXiRquej3QDeQBxSGPcS3VzUkAR1YdyeEzD+f6V64n\nnqSpI5TqbKrgAAAgAElEQVSUUvGIp8awzhizMEnxDBVHWtUYILWjkwJ6Pb2c9fhZLFm4hO/M+86w\n5edUzKEkd1x3ESk1riSqxvCMiBw7ypiyWjoMAc1z5PHLo3/JrW/dGtcMr6lu/lJKpb94EsN5wHMi\n0qPDVcOlcthqqFnls/j3A/6dy1+8nH5v/5BlO/s76ezvTFJkSqlMNGRiEGuSnQXGGJsxJl+Hq4bL\nc+SR78xPdRgAfG+v7zG1aCq3rLll2LJaa1BKDWXIxOBv1B/xym3jSTp0QoPVjnjlV6/kxdoXeW3L\na0OWbe9rp6u/K0mRKaUyTTxNSe/6h6yqKNKlOQnAlefi2qOu5dp/XEtTd9OQZXWEklIqlnhGJW0E\nZgN1QBcgWJWJvRMfXlgcaTcqKWD9zvX0uHtSHUbQbW/fxrrGddyy+JYh14PId+bjsDlw2pw47c7g\ntsPmwGl3Brd12m6lMlei1mOoirbfGFM3kg/aXemcGHZ07mBb+7ZUhxHk8Xk456lzOKrmKM7Y+4zd\nPp/D5ggmi2iJxGl3kmPPwWHTlWKVSjcJSQzpIp0TQ5+nj3WN61IdRpjtHds56/Gz+P1xv2f+xPlJ\n+Uyb2IJJIseeg9MWsq3JQ6mU0MSQQht2bqDb3Z3qMMKs+nwVt719Gw989wEKcwpTHQ4wOHlESyCZ\nlDyMMXiNF6/Pi8/4EBEcNgd2sY+rJjhjzLj6ezOJJoYU8vq8dPR30NHXQUd/R9r0OVz7j2vxGR/L\nFy1PdShxs4ktWLuwiS3qQ0RiHhtUFgl7X4DX58VrrAt6YHuoZ5/xRd0Xi91mDyaJQHNcYF/wdZRj\nqRL6d3l8HrzG/+zfH207UC6QGOxix26zYxPbbm2PxVrp44kxBp/xBf8bDWz7jA9XnksTQ7rw+Dx0\n9nemPFH0uHs4/a+nc87+53Dc7ONSEkM6EREEGfKCnmqBDn9BgokssD3UPsG/f4h9BhPzwp9O//8K\nJH672GOWMQwd71B/j8EEv5vA5wW3Q76zyH2h+4d6f+S/Ueh25L9J6HbkuaNd6AM/SEITwVB/64HT\nDtTEkK48Pk8wSXT0dSR1jqWNTRu58NkLufeEe5leMj1pn6uUSj1NDBkk2YniTx/+ib9/8Xf++K0/\nZlQbvlJq92hiyGBur9tqekpQovAZH//x3H+w54Q9ueCgC8b03Eqp9KWJIYu4vW46+jto72unuad5\nTNp/d3Xv4vS/ns51i67joGl6M7tS48FoEkNCu/5F5C4RaRCRD4Yoc7OIfCoia0Vk30TGk0mcdifl\n+eVUl1Yzt2IuOfac3T5nRUEFV3/1aq7+x9W09raOQZRKqWyU6DFh9wDfiHVQRBYDs4wxc4BzgdsT\nHE9GKsopYv7E+WOy/sNhMw7j2D2O5dqXr02rUShKqfSR8KYk/5QaT0WbW0lEbgdeMsY87H+9AVhk\njGmIUnZcNSXF0tzTzOa2zXh93lGfw+11c/aTZ1OSW0JFfgUFzgIKnAXkOfIocBaQ78gn35lPviPf\neh2yHSiT58jTseZKZYDRNCWlenjKNGBLyOtt/n2DEoOylOeXU5RTRG1L7agX3HHandxy3C28u+Nd\netw99Hh6gs9N3U1h+7o93YPKdLu76fP0kevIpcBZQHFOMeX55ZTmlVKeX05ZXhll+WWU5ZWFvXbl\nurDbYo9LV0qlh1QnhhFZvnx5cHvRokUsWrQoZbGkUo49hz0n7MmOzh1s79g+qiahsvwyjqk5ZtQx\n+IyPXk8v3e5u2vvaaeltoaWnheaeZlp6W6htqeXd3ndp6Wmhpdfa39nfSXFuMeV55ZTml1KeVx6W\nQCITS1FOUfDOZZ1uQan4vP3627zzz3d26xzp1pS0EThSm5Li1+3upralNqk3zI2Wx+ehrbdtIIn0\nNgcTR+hzc28zrb2tdPR1hN3dGpjaAgbujA3cIRqY7iL07tJA2bBjiP+e10A5EIlxZ3HoHasISJS7\nVyP259hzKHD4m9+c+QPbUZrlwrZD9uU58rLiXpPANA2Ggekaou0b6pgxZtAdzpHXgeHugI7nPSM5\n57DnwgwuE/J3BJ9DysQ67jVefD7/VCwh25HTX0RO7xJ6/KbjbkrLpiTxP6J5ErgAeFhEDgVaoyUF\nFVuBs4C9Ju7FlrYtwy7Ok2oOm4OKggoqCipG/N7Qi0bg/0Shz4FjQNgFxRhDv9vQ0eWjowN6ug2G\nwHmCZw/+TzCIHUQMNhuIzWDzP8QGNjGI3WAT65jYwre9pn9QE1y329pu72unoasheLzXbdW4Ipvr\nut3dGMygeYQCcwhF3Sd2bDZb2P5o+wLTgQQf+PD5Bj8HplkIe/ZPpxF4HnRh91/QQ/89QuepCp3j\nKnL+qlhzWwFR+7Ik8pIS5QoTWSay1jno+DDnjDUFRsxzSfixQT8o/J8R87gw8G9ps0f997aLPaxM\n2HH/vtFIaI1BRFYCi4AKrH6Dq4EcrIV+7vSXuRU4DmsRoLONMe/GOJfWGIbR2ttKXWsdHp8n1aGk\nXL8b2tuhswN6kjhNlQg4HCEPZ8RrBzidYB+m3z7yV2DoL8Fh98X6ZemzLtaBC0dg0rtYz/FMUGi3\n2Ye8+I+kCdAY6OuH3t6BR1+ftV8EKwH7f2aOxWsZuHYTDFMiXodsD1c2tNxwn2tLYsuo3uCmcHvd\nbGrdRHtfe6pDSbrePuhoh45O6EvzljWxRU8YodtOR/gFKpv4jP/CHyUJjCdiC09SIgP7AgLfSdh3\nE7HPxCprYOkxmTcqSY0xp93JnIo5NHY1sq19W1rPIjoWenqsRNDeDu7+VEcTP+Oz4h0uZrvDShBO\np1X7cDoHkkYgkaQ7j3dwLaC/L9VRpQfjG7iop5MM+M9KjcakwkkU5xRT21qbNmtDjAVjoLsHOjqs\nZiK3O9URJZbXYz16Y9SARAaaqwYlDWd8zVZjye0OTwK9veBJcsumMdDbbaOjzUFnu4OOdsfAdsg+\nr0dwOHw4cwwOp8Hh8FnPToPT6d92GBw5PpyBbWdEGYfBkWNt2x3GKuffb3eY4DnsDqvPKll/v9cj\neDzCaG930qakLGeMYWv7Vhq7GlMdyqgZA11dVjLo6LQulOnA4xZ2NuTQsC2Xhm15NGzPxW43lJa7\nKa1w4yobeC4q8aasWchmH0gYNvswzTVDHBvqfV6fVRPYjfsuY35m4CLf0T5wcQ9c4DsjLvpWGTtO\np6GoxEOxy0NRiSe4XezyUFTspajEgyPHh6ffhsdtXUTdbv+229r2egS3W6wyge2QMh6PzX/c2rbe\nZ5XxeiR4cfZ4rH0220ACsjv8CSOQcAKJxDGQkBDrAu/1ysD5QrZj7ff5BLvdOp/dbujpdmgfg4qu\nva+dTa2bcHvd+Az4/C1MNltyO8JiCcRkfNa28Vm/Pts7oLNz7C848XL3Czt35NCwPY+Gbbns2JZL\n47ZcGrbn0dzkpHyCm8qpvVRO72PSlD58PqGtxUHrLietzU7amq3n/j4brlIPrgo3pWWDE0fwudyD\nwzm+/jsPJNjG7bk0bs+lwf9o3J7Lzvpc7A5DkctDccgFPuy5xGsdD0kCOTnp9R0aA15v4Je8lUQC\nicPrGUg0wWTiFowheHEPJpLABd+/7QjZHtgf3kdxxte08zlrBP9D8j98vvBHtH1D7ff5oM/tYXtX\nHR3uwRPoiW0gSUjo8xD7bHbr2ZiBi3kw6ZjoF/to+1L9z9rfL+ysz7V++W/P9ScAqwbQustJxaR+\nKqf2WY/pvdbztD4mVvbHfRHv7xfaQhJFa7PD2t7lpLVlYH97q4P8Ah+uMjeucitZuMo8uMqthFIS\n2C53U1ziIVNuJO/tsdHo/44bt+cGtxvqre+4rMLNJP93HHiunNrLxCn95OVndz9ZomliSCORF3av\n12prHep16D5fAv+/0NrfxI6eLZgs75iO1NtjY9OnBdR+UsCOrXnBRNDW7L/4T+ujcpp14Z88zbr4\nV1T2JbWD1+eDjjYH7S0OWluctLf4ax4tjoHtZidtrQ66OxwUlXgoKbOShqvcvx2RUFxlbgqLvQlt\n4zYGOtvtwV/6DdvzwmoA3V12Jk2xLvqTpljfbaX/9YQRJNjREIGcXMj1P/L8zzb74B8r3hg/ZAKP\n0GOhZdPlx07oqCabf4jsqV/VxJASfX2wZQv094df5NNZv7ePdnfLmJzLEBgzbz28oc9Y28n+t/N6\nYWttPp9vLOSLjwv5YmMBDfW5zKzpoWZuN1NmBhJALxWV/dgz5Jd3KI8HOlqtpNHmr3UEtlubraQS\neN3bY6Ok1ENevs+6aJmIIY5Gwoc+mtAhj4KBsCGQ1msJnqS/z4aIoXJaXzABhP76L61wJ6Xz1eGA\nvDzIzRtIBLk5qRn2G6xJB77v4Pc2eF+glh1aBsIv8Dabf0hryIVfQmvzMf5GvY8hBVpbYdOm9E8E\nqRa8vd+fLLwRScS6ESv8mM/4Qqa9CL1pyhacpsImNowRGnfY+WR9Lh9vcLJxg4PPPnFQWelj3nwv\ne833sWChj1mzDDk5gbt/vf5pBjwRycwTFqfH54k7udkd1v9BPZ7UN49FcvdbfR99vXZEzMANV/7j\n1mszcP9uyM1YFhP+Ojju3vpDnTmGgqLkdbCLbeCXfyAJ5OUldwRWptDEkETGwLZt0KATeCRdayus\nXw8ffTTwsNth4UJYsMB6zJ8PRUVj+7mB5OXxJxOf8YLNS2GxhxKXl/xCrzXdhjF4vAa32+D2+J8D\nr90Gj2dgv8dr8PpC59YZmJ4DBubTCZ+PZ/BcO9nAJnbs4vBP8+Dwv7aTlyfBWkCePwHk5kY/x0i/\nk9D5sOJ5He97IufeivY8XNnA3xM6/Uu0fcM9TyycqIkhGdxu+OILa7SMSqzeXvjkk/Ak0NwMe+01\nkAQWLoRJk5IXk80GpaVQVgYu1+43UwT6ltxu6+HxDPQ1RW4HniP/rxA2IRuDk0lo4rHmPQrMhxSY\nMyl07iOvNWeSv9YWVjZif+T/J63pNByDLvKh206HnVyngxynnbwcB7lOO7lOOzk5EvUu8Gy9+ztZ\nREQTQ6J1dEBtbfbfWJUMHo/167+pCXbtGvzYssX6rmtqwpNAVRVJ7xMQsZJAWZmVFJJ1s1IsPl/0\nhDHcs88X+0Ibbf9wZQPJBDHWBT7HFnOKj8B2qr+78UYTQ4LV11uPDPnKUsIYaGuzftUHLvCxLvzt\n7dZFtqIi+mPKFJg712o6SAURKC6G8nIrzkzsoFZKE0OCeDxWB3NbW0o+Pm34fNYFffv2gSS5ffvg\nC39+fvQL/YQJ1kU28Lq0ND3n+ikqsuIsK0vP+JQaCU0MCdDVZfUn9GfQBG2j5fXCzp2wY4d1wd++\nfWC7vt7qaC8qsn7JT50Kkydb25MmWRf9igrrghqrYzCdFRZaiaC83Gr2UCpbaGIYYzt3Wu3cGfIV\nxa2tDd57Dz79NPzi39hotaMHLvpTp1oX/tBEkKpmnUTIzx+oGWRiMlMqHpoYxojPB3V1Vjt5Nmhv\ntxLB22/DO+9Yw2z33hvmzRu4+E+dCpWV2XmBDHR+5uRYz7m5VjNWNiU5pWLRxDAGenvh889jT3Oc\nCTo7wxPB5s3wpS/BgQfCAQdYY/yzoe3cbh+42Ic+IvfpcEc1nmli2E3NzVZNIZHzFCVCZyesXWsl\ngbfftv6GhQutJHDAAdYwz0xrN8/JsX7ZR7vQBx467FGp4WliGCVjrL6EnTsTcvox19UVnghqa62L\nf6BGsGCBdTFNdw7HwFQGeSHTGuTm6kVfqbGiiWEU+vutUUddXWN+6jHT3Q3vvz+QCD7/3LrzN5AI\nFi5M374Bmy38gh+aBLKhOUupdKeJYYTa2qz7E5K99OBwjLFGDL36Krz2mjUlxLx54Ykg3TpOg1Ma\nR/z6z4Sai1LZTBNDnIwZGKaZLnp6YM0aKxm8/rr1a/rww63HfvulXyIoKLCGtrpc1rZ28CqVnjQx\nxMHtttrkOzrGIKjdtHXrQK3g/fet0UKBZFBVlV4XW4cDSkqsh8ulzUBKZQpNDMPweq3ZOVM1AZ7H\nY3Uav/qq9ejogC9/2UoEhxwy9tNE765AraCkxLozOJ0SlVIqPpoYhhGY2yeZdu2ymoZefdVqKpox\nY6BWMG9eeo2+sdsHagQlJZk3xFUpNZgmhiH4fPDhh4nvaPb5YOPGgVrBli1w8MFWIvjyl635hNJJ\nfv5AX4HWCpTKPpoYhtDQYLXpJ4LXCy+/bD1ef92aqjlQK9hnn/T75e1wDEx8l26xKaXGliaGGHw+\nWLcuMX0L9fVw9dXQ1weLF1vJYPr0sf+csZCba82HVFGRXk1YSqnEGU1iSPjYEhE5DvhvwAbcZYz5\ndcTxI4EngC/8u/5ijPnFWMawa1diksJzz8Fvfwunnw5nnJG+C7kUFloJoaws1ZEopTJBQhODiNiA\nW4FjgO3AWyLyhDFmY0TRl40x305EDMaM/f0KHR3w619bfQm33GJ1Iqcjl8tKCMXFqY5EKZVJEt2g\ncDDwqTGmzhjjBh4CTohSLmFdns3NY7vIzjvvwGmnWRfbBx5Iv6QgYq0xMH8+zJ6tSUEpNXKJbkqa\nBmwJeb0VK1lEOkxE1gLbgEuNMevHKoCxqi243XDHHfD003DFFVZfQjqx2azO5MpKnYZCKbV70uH+\n1XeAmcaYbhFZDDwOzB2LE7e0jM26Cps2Wclg4kRYudL6RZ4uAiOMJk1K3z4OpVRmSXRi2AbMDHk9\n3b8vyBjTGbL9rIj8r4iUG2MGrZ+2fPny4PaiRYtYtGjRkB9eXz+qmENig8ceg9tvh/POgxNPTJ9x\n/jrCSCkVzerVq1m9evVunSOhw1VFxA58jNX5XA+sAZYYYzaElKk0xjT4tw8G/myMqY5yrhENV21r\ng88+G33su3bBddcNPFcPiig1dISRUmok0m64qjHGKyIXAqsYGK66QUTOtQ6bO4GTROQ8wA30AN8f\ni8/endrCK6/AL34B3/42/OY36XETWEkJTJ6snclKqcTLyhvcOjqsNQxGqrcX/vu/rdlOr73Wmu46\n1crLrYSQn5/qSJRSmSjtagypMprawoYNVgfz/Pnw4IOpn+nU5YJp0zQhKKWSL+sSQ1fXyNZa8Hph\nxQprtNFPfwrf+EbiYotHcbGVEAoLUxuHUmr8yrrEMJLaQmCeIxG4/36rySZVCguthKB9CEqpVMuq\nxNDdbY1GikdgnqOlS61Hqu4ByM+HqVOhtDQ1n6+UUpGyKjHEc5dzRwf86ldW53Qq5znKzbUSQjrd\nLKeUUpBFiaG317rTeSjr18PPfgZHHGE1HeXlJSe2UDk5MGWKdWNautwsp5RSobImMQxXW6ivh5/8\nxOpg/trXkhNTKIfD6sOYOFHvVFZKpbesSAz9/dYsqrF0d1tJYenS5CcFu926U1nnMlJKZYqsSAw7\ndljzGkXj88FVV8Fee1kL6iSLzWYlg8pKq7aglFKZIuMvWW43NDXFPn7bbdZIpRtuSE6bvog1/fWU\nKekxlYZSSo1UxieGhobYtYVnnoFVq+Dee5Nzka6osBJCbm7iP0sppRIloxODxwM7d0Y/9sEH8Lvf\nWVNmJ3om0qIimDlTp69QSmWHjE4MjY1WH0KkHTvgP//Tuqt59uzEfb7NZt2tPGlS4j5DKaWSLWMT\ng9drJYZIoSOQjjgicZ/vclm1BF1GUymVbTI2MezcaSWHUIERSHvumbgRSA4HzJihdywrpbJXRiYG\nn8/qdI50223Q2gq//GViRiCVl1tJQYefKqWyWUZe4pqarI7nUM8+OzACaaybd3JyrGYjl2tsz6uU\nUuko4xKDMYNrC+vWWSOQbrtt7EcgTZxodTDrXctKqfEi4xLDrl3WFBgBO3bApZdafQtjOQIpLw+q\nqlK/kptSSiVbRiUGY8Iny+vpsUYgLVkydiOQRKxpLKZO1dlPlVLjU0YlhpYW6OuztgMjkObOhTPO\nGJvzFxRAdbXeqKaUGt8yKjGE1hbuuMOaUfX663f/l73NZtUQJk3SWoJSSmVUYujpsZ6fe84ahTQW\nI5CKi62+BJ3fSCmlLBmVGMAagfTb31ojkHbnJjO7HaZPt2ZCVUopNSCjEkNgBNKVV+7eCKTSUuu+\nBJ0WWymlBhMTa87qNCMiZs89DcceC2edNbpzFBZay2uWlo5tbEopla5EBGPMiHpPM6rGMHs2nHnm\nyN5TVGTd9FZaqhPeKaVUPDKqxvD662bYi7uI1aEcSAY6r5FSajwbTY3BlqhgAkTkOBHZKCKfiMjP\nYpS5WUQ+FZG1IrJvrHPFSgo2m5UEqqthn31gzhyrU1mTglJKjVxCE4OI2IBbgW8AC4AlIjIvosxi\nYJYxZg5wLnB7POe2261RSXvsYSWDWbOspTVHM6fR6tWrR/6mFMq0eCHzYs60eEFjToZMi3e0El1j\nOBj41BhTZ4xxAw8BJ0SUOQFYAWCMeRNwiUhltJM5HNbFf/ZsKxnU1FhNRrbd/Csy7R870+KFzIs5\n0+IFjTkZMi3e0Up0Y8s0YEvI661YyWKoMtv8+watuLD33npnslJKJVrC+xjGkiYFpZRKvISOShKR\nQ4Hlxpjj/K+XAcYY8+uQMrcDLxljHva/3ggcaYxpiDhXZgyfUkqpNJNu9zG8BcwWkSqgHjgVWBJR\n5kngAuBhfyJpjUwKMPI/TCml1OgkNDEYY7wiciGwCqvZ6i5jzAYROdc6bO40xjwjIseLyGdAF3B2\nImNSSik1tIy5wU0ppVRyZFTn8+4SkbtEpEFEPgjZVyYiq0TkYxF5XkRcqYwxkohMF5EXReQjEflQ\nRH7s35+WcYtIroi8KSLv+eO92r8/LeMNJSI2EXlXRJ70v07rmEVkk4i87/+u1/j3pW3MIuISkUdE\nZIP/v+dD0jzeuf7v9l3/c5uI/DjNY75YRNaJyAci8icRyRlNvOMqMQD3YN1sF2oZ8H/GmD2BF4HL\nkh7V0DzAT4wxC4DDgAv8NwmmZdzGmD7gKGPMfsC+wGIROZg0jTfCRcD6kNfpHrMPWGSM2c8YExgG\nns4x/x54xhizF7APsJE0jtcY84n/u90fOACrqfuvpGnMIjIV+H/A/saYvbG6CpYwmniNMePqAVQB\nH4S83ghU+rcnAxtTHeMw8T8OfC0T4gYKgLeBg9I9XmA68HdgEfBkJvy3AdQCFRH70jJmoAT4PMr+\ntIw3SpzHAq+kc8zAVKAOKPMnhSdHe60YbzWGaCYZ/ygoY8wOYFKK44lJRKqxfoW/gfUPnZZx+5tk\n3gN2AH83xrxFGsfrdxNwKRDa6ZbuMRvg7yLyloj8q39fusZcAzSJyD3+ppk7RaSA9I030veBlf7t\ntIzZGLMd+C2wGetG4TZjzP8xing1MQyWlr3xIlIEPApcZIzpZHCcaRO3McZnrKak6cDBIrKANI5X\nRL4JNBhj1gJDDYtOm5j9vmKsZo7jsZoYjyB9v2cHsD/wP/6Yu7CaONI13iARcQLfBh7x70rLmEWk\nFGuKoSqs2kOhiJzOKOLVxAANgbmZRGQy0JjieAYREQdWUrjfGPOEf3fax22MaQdWA8eR3vF+Bfi2\niHwBPAgcLSL3AzvSOGaMMfX+551YTYwHk77f81ZgizHmbf/rx7ASRbrGG2ox8I4xpsn/Ol1j/hrw\nhTGm2RjjxeoP+TKjiHc8JgYh/Ffhk8AP/NtnAU9EviEN3A2sN8b8PmRfWsYtIhMCox5EJB/4OrCB\nNI0XwBhzuTFmpjFmD6ybMF80xpwBPEWaxiwiBf5aJCJSiNUG/iFp+j37mzK2iMhc/65jgI9I03gj\nLMH6wRCQrjFvBg4VkTwREazveD2jiTfVHSZJ7pxZCWwH+vxf4tlYHTX/B3yMdSNeaarjjIj5K4AX\nWAu8B7yL9Qu8PB3jBr7kj3Et8AHwc//+tIw3SvxHMtD5nLYxY7XZB/6b+BBYlgEx74M1G8Ja4C+A\nK53j9cdcAOwEikP2pW3MwNVYP8Q+AO4DnKOJV29wU0opFWY8NiUppZQagiYGpZRSYTQxKKWUCqOJ\nQSmlVBhNDEoppcJoYlBKKRVGE4NKayJSHjL1cb2IbA15HddCU2JNtz5nmDLni0jk6oKjjfkEf4xr\n/VMg/3CY8kf5Z6CNdmyyiPzNf66PRORx//7pIvJgtPcotbv0PgaVMUTkKqDTGPO7KMfEpMF/zCKS\ngzXr6f7GmAb/PDtVxpjPhnjPdUCTCb+zPXDsj1jTMdzmf73QGLMuQeErBWiNQWWW4FQmIjLL/wv6\nARFZB0wWkTtEZI1YCwRdEVL2FRHZW0TsItIiIjf4f4G/JiIT/GWuk4FFkF7xl3lTrEVlDvXvLxCR\nR/21gEf8s5ruHRFjYBGUVgBjjDuQFERkkog85o/xDRE5WET2AP4V+Km/FnRoxPmmYM0zhP9860L+\n/vf823fLwGIyO0XkMv/+n/n/hrWh34dSw9HEoDLZnsBvjTELjTWh3M+MtWDNvsCxYi1oFMkFvGSM\nCUxfHrOZxxhzCPCfWNMMgLUISr0xZiFwnf9zIt+zE2vagTqxVtA61T9vDcDNwK/9MX4faw30L4A/\nAr8xxuxvjHkj4pS3AitE5P9E5DL/JGjBj/N/5g+NNWPpd7EmSLtPRBYDM/1/w37AV6IkHaWi0sSg\nMtnnxpj3Ql6fLiLvYM3VNA+YH+U93caYVf7td4DqGOf+S0iZKv/24cBDAMaYD7AmgRvEGHM21kyX\nbwE/A+7wH/oacLv/l/7jgEtEcof6A40xzwJ7AHf5/553RaQsspxYaxs8ApxnrHn5jwWOE5F3sb6P\nWcDcyPcpFU1cnXdKpamuwIaIzAZ+DBxojOnwT5udF+U9/SHbXmL/f6AvjjIx127wN/ms83cQrwfO\n8Zc/yFhTIg+cRIZaAgKMMS1Ys3s+KCLPYiWo9RHF7gBWGmNeDontF8aYe4Y8uVJRaI1BZbLQK2oJ\n0A50isgUBq/tHe09I/UaVhMQIvIlYK9BJxcpFmvBnID9sJZbBGvp0P8XUnYf/2aHP/7BwYocLSJ5\n/hYQY80AAAD7SURBVO0SrFlVN0eUuQhwGGNuCtn9PPAjf00CEZkmIhVx/p1qnNMag8pkwVFIxph3\nRWQD1pTDdcCr0coR32pbscrcgtV+vw7rF/t6oC2ijACXicidQA/QyUA/xoXAbSJyNmAHXsJKFE8A\nj4jId4ELIvoZDgJuFZF+rB9y/2OMeV9EZoWUuQTo8jdRGeBWY8zd/j6WN/w1knbgNGBXHH+/Gud0\nuKpScRIRO9Yv8z5/09XzwBxjjC/FoSk1prTGoFT8ioAXQm6sO0eTgspGWmNQSikVRjuflVJKhdHE\noJRSKowmBqWUUmE0MSillAqjiUEppVQYTQxKKaXC/H9QsxZZClyLdQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f85b62bb0f0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_learning_curve(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we see that by adding more model complexity, we've managed to lower the level of convergence to an rms error of 1.0!\n",
    "\n",
    "What if we get even more complex?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEPCAYAAABGP2P1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd8W/XV+PHPkbzteGU4IYkzCQkJEHYgUMwoJTyljAZK\nIKSkTwtlP4w+DZSRlgK/tnTRxd6EvZowHwgpm7BCCJlAlrNjx3tKOr8/rmTLW7a1bJ13q5evrq6u\njhVzz/1uUVWMMcaYAFesAzDGGBNfLDEYY4xpwRKDMcaYFiwxGGOMacESgzHGmBYsMRhjjGkhoolB\nRFJF5CMR+VxEvhSRmzo47g4RWSciy0RkaiRjMsYY07mkSJ5cVetF5FhVrRERN/CeiLyiqksDx4jI\nDGCcqu4tIocDdwLTIhmXMcaYjkW8KklVa/ybqTiJqPWIulOBh/3HfgTkiEhBpOMyxhjTvognBhFx\nicjnwHbg/1T141aHDAc2Bz3f4t9njDEmBqJRYvCp6oHACOBwEdk30p9pjDGm5yLaxhBMVStE5C3g\nJGBl0EtbgJFBz0f497UgIjapkzHG9ICqSneOj3SvpEEikuPfTge+C6xuddi/gTn+Y6YBZaq6o73z\nqWpEHjfddFPEzh2v8Z7/wvnc8eEdfSrmvvYdW8z9L+a+Fq9qz+6nI11iGAY8JCIunCT0pKq+LCIX\nAqqqd/ufnywiXwPVwNwIx2SMMaYTke6u+iVwUDv772r1/NJIxmHap206iBljjI18BqCoqCjWIXRL\nOOIVulXl2GuJ+B1Hm8UceX0t3p6SntZBRZuIaF+JtS/4yYs/YerQqVx++OWxDsUYE0Eignaz8Tlq\nvZJMfIl2icF0bvTo0WzcuDHWYZg+bNSoUWzYsCEs57LEkMCsBBY/Nm7caP8epldEwnezZ20MCSqc\nf0TGmP7FEoMxxpgWLDEkMOuuaoxpjyWGBGWNzyaaLrroIm655ZawH2siwxqfjTGdGjNmDPfddx/H\nHXdcj8/xr3/9KyLHmsiwEoMxple8Xm+sQ4gL/el7sMSQwKx7pOnKnDlz2LRpE6eccgrZ2dncfvvt\nbNy4EZfLxf3338+oUaM4/vjjATjrrLMYNmwYeXl5FBUVsXJl8yTKc+fO5cYbbwTgP//5DyNHjuRP\nf/oTBQUFDB8+nAcffLBHx5aWlnLKKaeQk5PD4Ycfzg033MDRRx/d7u9SX1/Peeedx6BBg8jLy+Pw\nww9n165dAOzZs4ef/OQnDB8+nIEDB3LGGWc0ve+ee+5h7733ZtCgQZx22mls27at6TWXy8U///lP\nJkyYwIQJEwBYvXo1J554IgMHDmTSpEk8/fTTTce//PLLTJ48mezs7KbfKx5ZYkhQ1l3VhOLhhx+m\nsLCQRYsWUVFRwTXXXNP02ttvv83q1at57bXXADj55JP55ptv2LlzJwcddBDnnntuh+fdvn07lZWV\nbN26lXvvvZdLLrmE8vLybh978cUXM2DAAHbu3MmDDz7IQw891OHf9kMPPURFRQVbtmyhtLSUO++8\nk/T0dABmz55NbW0tq1atYufOnVx55ZUALF68mOuuu45nnnmGbdu2UVhYyNlnn93ivC+++CJLly5l\n5cqV1NTUcOKJJzJ79mx2797NE088wcUXX8zq1c6k0j/96U+55557qKioYMWKFb2qnoskSwzG9AEi\n4Xn0VOvSpYjw61//mvT0dFJTUwE4//zzycjIIDk5mRtvvJEvvviCysrKds+XkpLCDTfcgNvtZsaM\nGWRlZbFmzZpuHevz+Xjuuef4zW9+Q2pqKpMmTeLHP/5xh79DcnIyJSUlrF27FhHhwAMPJCsri+3b\nt/Paa69x1113kZ2djdvtbip1LFiwgP/+7//mgAMOIDk5mdtuu40PPviATZs2NZ33uuuuIzc3l9TU\nVBYtWsSYMWOYM2cOIsIBBxzAD3/4w6ZSQ0pKCl999RWVlZXk5OQwderU0P8RosgSQwKz7qp9h2p4\nHuE0YsSIpm2fz8e8efMYP348ubm5jBkzBhFh9+7d7b534MCBuFzNl5+MjAyqqqq6deyuXbvwer0t\n4hg5cmR7pwCcarHvfe97nH322YwYMYJ58+bh9XrZvHkz+fn5ZGdnt3nP1q1bGTVqVNPzzMxMBg4c\nyJYtzWuJBX/+xo0b+fDDD8nPzyc/P5+8vDwWLFjAjh3OEjPPPvssL730EqNGjeLYY4/lww8/7DDe\nWLLEkKCsKsmEqqO/leD9CxYsYOHChSxevJiysjI2bNjQq4ViQjF48GCSkpIoLi5u2rd58+YOj3e7\n3dxwww189dVXvP/++yxcuJCHH36YkSNHUlpaSkVFRZv37LXXXi3msKqurqakpKRFMgj+HkaOHElR\nURGlpaWUlpayZ88eKioq+Pvf/w7AwQcfzAsvvMCuXbs49dRTOeuss3r1HUSKJQZjTKeGDh3Kt99+\n22Jf6wt+ZWUlqamp5OXlUV1dzbXXXhvxmw+Xy8UZZ5zB/Pnzqa2tZfXq1Tz88MMdHr9kyRJWrFiB\nz+cjKyuL5ORk3G43Q4cOZcaMGVx88cWUlZXh8Xh45513AJg1axYPPPAAy5cvp76+nuuuu45p06Z1\nWDL5/ve/z9q1a3n00UfxeDw0NjbyySefsHr1ahobG1mwYAEVFRW43W4GDBiA2+2OyHfTW5YYEpQg\n1ivJhGTevHncfPPN5OfnN/WiaX3RnzNnDoWFhQwfPpwpU6Zw5JFHduszupNEgo/929/+RllZGcOG\nDePHP/4x55xzTlObR2vbt29n5syZ5OTkMHnyZI499lhmz54NwCOPPEJSUhITJ06koKCAv/71rwAc\nf/zx3HzzzZxxxhkMHz6c9evX88QTT3QYd1ZWFq+//jpPPPEEe+21F3vttRfz5s2joaGh6XPGjBlD\nbm4ud999NwsWLAj5944mW48hQf180c8Znz+ea468puuDTcT558yPdRh93rx589ixYwcPPPBArEOJ\nuo7+hnqyHoOVGIwxfdaaNWv48ssvAVi6dCn33XdfizEIpmdsSgxjTJ9VWVnJrFmz2LZtGwUFBfzi\nF7/glFNOiXVYfZ4lhgRmVRemrzvkkENYt25drMPod6wqKUG5xP7pjTHts6tDAmv0NcY6BGNMHLLE\nkMAavA2xDsEYE4csMSSwOk9drEMwxsQhSwwJShDqPfWxDsMYE4csMSQwn/qs1GAiJrCWQsCUKVN4\n++23Qzq2u2w50PCy7qoJTFGqGqpIS0qLdSimnwqeMmLFihUhH9uZhx56iHvvvbdpPiOw5UDDLaIl\nBhEZISKLReQrEflSRC5v55hjRKRMRD7zP66PZEzGEfiPsLK+/fnyjYlXqtpvZgeO1+VAI12V5AGu\nUtXJwBHAJSIysZ3j3lbVg/yP30Y4JhOkqqH9OfCNAfj973/PmWee2WLfFVdcwf/8z/8A8OCDD7Lv\nvvuSnZ3N+PHjufvuuzs815gxY1i8eDEAdXV1nH/++eTn5zNlyhQ+/vjjFsf+7ne/Y/z48WRnZzNl\nyhReeOEFwFk286KLLuKDDz5gwIAB5OfnAy2XA4Wul+O86667mDBhAvn5+Vx66aUdxvzxxx9z6KGH\nkpOTw7Bhw1qsYPfuu+8yffp08vLyGDVqVNPMrhUVFcyZM4chQ4YwZsyYFlVcDz30EEcddRRXXXUV\ngwYN4te//jUA999/P/vuuy8DBw5kxowZLRYCuvLKKykoKCAnJ4cDDjigxZKpEROYMz0aD+AF4PhW\n+44BFobwXjXhc8lLl+hlL1+mn2z5ROs99bEOJ+HF69/3xo0bNTMzU6uqqlRV1ev16rBhw3Tp0qWq\nqvryyy/r+vXrVVX17bff1oyMDP38889VVXXJkiU6cuTIpnONHj1a33zzTVVV/eUvf6nf+c53tKys\nTIuLi3XKlCktjn3mmWd0+/btqqr61FNPaWZmZtPzBx98UI8++ugWcZ5//vl6ww03qKrqm2++qYMG\nDdJly5ZpQ0ODXnbZZfqd73yn6VgR0VNOOUUrKip006ZNOnjwYH3ttdfa/f2POOIIffTRR1VVtbq6\nWj/66CNVVd2wYYMOGDBAn3zySfV4PFpaWqpffPGFqqqed955etppp2l1dbVu2LBBJ0yYoPfff39T\n7ElJSfqPf/xDvV6v1tXV6QsvvKB77723rlmzRr1er95yyy165JFHqqrqa6+9pocccohWVFSoqurq\n1aubvofWOvob8u/v1rU6am0MIjIamAp81M7LR4jIMmAL8AtVjUJKTGzB025XNVSRn54f44hMZ+TX\n4ak60Zu6Nw1KYWEhBx10EM8//zyzZ8/mzTffJDMzk0MPPRSAGTNmNB179NFHc+KJJ/LOO+90uWTl\n008/zZ133klOTg45OTlcfvnl3HzzzU2v//CHP2zaPvPMM7n11ltZunRpSPMgBS/HCXDbbbeRl5fH\npk2bKCwsBODaa69lwIABDBgwgGOPPZZly5Zx4okntjlXSkoKX3/9NSUlJQwcOJDDDjsMgMcff5zv\nfve7TQvt5OXlkZeXh8/n48knn2T58uVkZGQwatQorr76ah555BHmzp0LwPDhw7n44osBSE1N5a67\n7uLaa69lwoQJgDND7C233MLmzZtJTk6msrKSlStXcthhh7HPPvt0+fuHQ1QSg4hkAc8AV6hq67qL\nT4FCVa0RkRk4pYoJ0YgrkQVP0WuJIf5194IeTrNmzeLxxx9n9uzZPP7445xzzjlNr73yyiv85je/\nYe3atfh8Pmpra9l///27POfWrVtbrIIWvHwmwMMPP8yf//xnNmzYADgrp3W0TGh75z744IObngcv\nxxlIDAUFBU2vd7as6H333ccNN9zAxIkTGTt2LDfeeCP/9V//xebNmxk3blyb43fv3o3H42n6nMDv\nFrwUaOveVxs3buSKK67g6quvBprbULZs2cKxxx7LpZdeyiWXXMKmTZs444wzuP3228nKygrpu+ip\niCcGEUnCSQqPqOqLrV8PThSq+oqI/FNE8lW1tPWx8+fPb9ouKiqiqKgoIjEnGmuANp0588wzueaa\na9iyZQvPP/980zrFDQ0NzJw5k0cffZRTTz0Vl8vF6aefHtLkjMOGDWPz5s1MmjQJoMXymZs2beKC\nCy7grbfe4ogjjgDgwAMPbDpvVw3PoSzHGapx48Y1Labz7LPPMnPmTEpLSxk5ciRLly5tc/ygQYNI\nTk5m48aNTJw4sel3Gz58eNMxreMvLCzk+uuvZ9asWe3GcOmll3LppZeye/duzjzzTP7whz80tU20\nZ8mSJSxZsqS7v2oL0RjHcD+wUlX/2t6LIlIQtH0YzuJBbZICOIkh8LCkED51njo8Pk+swzBxatCg\nQRxzzDHMnTuXsWPHNlVnNDQ00NDQwKBBg3C5XLzyyiu8/vrrIZ3zrLPO4rbbbqOsrIzi4uKmNZHB\nuZC7XC4GDRqEz+fjgQceaNHVtaCggOLiYhob25/rq7vLcXbmscceayqp5OTkICK4XC7OPfdc3nzz\nTZ555hm8Xi+lpaV88cUXuFwuzjrrLH71q19RVVXFxo0b+fOf/8x5553X4WdceOGF3HrrrU2NyuXl\n5TzzzDMAfPLJJyxduhSPx0N6ejppaWm4XJ1ftouKilpcK3si0t1VpwPnAseJyOf+7qgniciFInKB\n/7CZIrJCRD4H/gL8KJIxmfZZ7yTTmXPOOYc333yTc889t2lfVlYWd9xxB2eeeSb5+fk88cQTnHrq\nqR2eI/hO+aabbqKwsJAxY8Zw0kknMWfOnKbXJk2axNVXX820adMYOnQoX331FUcddVTT68cddxyT\nJ09m6NChDBkypM3ndHc5zs5KIK+++iqTJ08mOzubK6+8kieffJLU1FRGjhzJyy+/zO23305+fj4H\nHnggy5cvB+COO+4gIyODsWPH8p3vfIfZs2c3tS+057TTTmPevHmcffbZ5Obmsv/++/Pqq68CTg+n\nn/3sZ+Tn5zNmzBgGDRrEL37xiw7PFS62tGeCuuKVK/Cpj/Onng9AQVYBI7K7X9Q24WFLe5resqU9\nTdhZO4MxJsASQwJTmu8uaj21+NQXw2iMMfHCEkOCal2vqqrWzmCMASwxmCCWGIwxYIkhobVuqLJ2\nBmMMWGJIWO110atprLGeMcYYW4/BNPOpj+rGarJSIjvc3rQ1atSofjOVtImN1tOK9IYlBtNCVUOV\nJYYYCMwJZEw8sKqkBCVIi+6qAdYAbYyxxJCghParLSwxGGMsMZgWvD4vNY01sQ7DGBNDlhgSWEc9\nkKzUYExis8SQoDrrAWOJwZjEZonBtGED3YxJbJYYTBsen4c6T12swzDGxIglBtMuq04yJnFZYkhQ\nHXVXDbDEYEzissRg2mXtDMYkLksMCay9kc8BDd4GGrwNUYzGGBMvLDEkqFAmbLPqJGMSkyUG0yFL\nDMYkJksMpkPWzmBMYrLEkKAE6XJRnjpPHR6fJ0oRGWPihSWGBBXqojBWnWRM4rHEYDplicGYxGOJ\nIYF11l01wBKDMYnHEkOC6mrkc0BNYw0+9UU4GmNMPLHEYDqlqlZqMCbBWGIwXbLEYExiiWhiEJER\nIrJYRL4SkS9F5PIOjrtDRNaJyDIRmRrJmEz3WWIwJrEkRfj8HuAqVV0mIlnApyLyuqquDhwgIjOA\ncaq6t4gcDtwJTItwXAkv1O6qANUN1ahqt95jjOm7IlpiUNXtqrrMv10FrAKGtzrsVOBh/zEfATki\nUhDJuEz3+NRHdWN1rMMwxkRJ1NoYRGQ0MBX4qNVLw4HNQc+30DZ5mDALZeRzMKtOMiZxRLoqCQB/\nNdIzwBX+kkOPzJ8/v2m7qKiIoqKiXseWqLpbLWSJwZi+YcmSJSxZsqRX55Du3DX26ANEkoBFwCuq\n+td2Xr8TeEtVn/Q/Xw0co6o7Wh2nkY41kVy/+Hp2VO3g54f8PKTj3S43U4davwBj+hoRQVW7dScY\njaqk+4GV7SUFv38DcwBEZBpQ1jopmNjz+rzUNNbEOgxjTBREtCpJRKYD5wJfisjngALXAaMAVdW7\nVfVlETlZRL4GqoG5kYzJNAtlSoxgVQ1VZCRnRCgaY0y8iGhiUNX3AHcIx10ayThMW6FOiRGsqqGK\nIZlDIhCNMSae2MhnEzJrgDYmMVhiSGDdnRyv0dtInacuQtEYY+KFJYYEdfCwg3lv03vdGssAVmow\nJhFYYkhQ3x33XTw+Dx9v/bhb77PEYEz/Z4khQYkI5+x3Do9++Wi33meJwZj+zxJDApsxfgZrdq/h\n2z3fhvyeek89jd7GCEZljIk1SwwJLDUplZn7zuSxLx/r1vsqGyojFJExJh5YYkhwMyfNZPH6xZTU\nlIT8HqtOMqZ/s8SQoAID3PLS8/ju2O/y9MqnQ36vJQZj+jdLDIZz9juHZ1c9G/IYhdrGWjw+T4Sj\nMsbESqeJQUTcInJ7tIIxsTE6dzRThkzhpXUvhfweKzUY0391mhhU1QscFaVYTAzN3n82j335WMij\noS0xGNN/hTKJ3uci8m/gaZzZTwFQ1eciFpWJuoOGHkRmcibvbnqX74z6TpfHW2Iwpv8KpY0hDSgB\njgNO8T++H8mgTOS1XsFNRDh3v3NDHvBW01jT7bmWjDF9Q5clBlW19RESxAljT+BvS//Gql2rmDR4\nUqfHqipVDVVkp2ZHKTpjTLR0WWIQkREi8ryI7PQ/nhWREdEIzkRXkiuJs6ecHXKpwaqTjOmfQqlK\negBn+c29/I+F/n2mHzp94ul8UPwB26u2d3lsX0wMPvWxs3pnrMMwJq6FkhgGq+oDqurxPx4EBkc4\nLhMjWSlZfH/v7/PEiie6PLa6obrb03bHktfnZV3JOjaXb6a8rjzW4RgTt0JJDCUiMts/psEtIrNx\nGqNNH9bZ0p6zpsxi4dqFXZYIfOqjurG602PihdfnZV3puqbfaVP5Jms8N6YDoSSGnwBnAduBbcBM\nwBqk+7FhA4Zx2PDDeHHNi10e2xeqkxq9jawpWUN1Q3MSa/A2sLVyawyjMiZ+dTnyGThDVX+gqoNV\ndYiqnqaqm6IUn4mRc/c7lydWPNHl1BfxnhgavA2sLVlLbWNtm9d2Vu+kprEmBlEZE99CGfk8K0qx\nmDgyZcgUCjILWLx+cafHVdZXUlpbGqWouqfeU8+a3Ws6nANKVdlYtrFPtZMYEw2hVCW9JyJ/F5Gj\nReSgwCPikZmIaj3ArT2z95/No18+2umF06c+1u9Zz4ayDXFVZ1/bWMuakjU0eBs6Pa6msYZdNbui\nFJUxfUMoiWEqMBn4DfBH/8Mm1ksARxceTWV9Jcu2L+vy2JKaElbuWhkXVTM1jTWsLVkb8kpzWyq2\ndJlAjEkkXbUxuIB/qeqxrR7HRSk+E0Nul7tb60LXe+pZvXs1O6p2RDiyjlU1VLG2ZG23pgX3qY/N\n5ZsjGJUxfUtXbQw+4H+jFIuJQ6dMOIUvdnzBpvLQ+huoKsUVxXxd+nXU12yoqK9gXck6vD5vt99b\nVldGWV1ZBKIypu8JpSrpDRG5RkRGikh+4BHxyExcSEtK44yJZ7DgywXdel95XTkrd62ksj4660OX\n1ZXxdenXvWrn2FS+qUdJxZj+RrrqkSEi69vZrao6NjIhdRiHWu+R8Gn0NrJ8x/KQjt1ds5sznz6T\n53/0PLlpud3+rKFZQ9lrwF4hNXj3RGltKRvKNoSld9HgzMEU5hSGISpj4oOIoKrd+o+vyxKDqo5p\n5xFSUhCR+0Rkh4i0ewUSkWNEpExEPvM/ru9O8CY6BmUMomh0Ec+uerZH799etT2kHkI9sbtmN+v3\nrA9bl9Nd1btaDIQzJhGFMrtqhohcLyJ3+5/vLSKhrsfwAPC9Lo55W1UP8j9+G+J5TZSdu9+5PPXV\nUz2+uFc3VLNy10r21O4JW0w7q3eysWxj2M4XsLHcxjaYxBbq7KoNwJH+51uAkC7gqvou0NWVIDL1\nCyasxuePZ+/8vXn161d7fA6vz8u3e75lY9nGXo952Fa5LWI9iWoba9lRHbueVcbEWiiJYZyq/h5o\nBFDVGsJ7MT9CRJaJyEsism8Yz2vCLLAudG/vpnfX7GbVrlXtTlMRiuKK4ojPc7Stchv1nvqIfoYx\n8SqUNZ8bRCQdUAARGQeE67+YT4FCVa0RkRnAC8CEjg6eP39+03ZRURFFRUVhCiPx9KQh+PDhhyMi\nfFj8IUeMPKJXn1/nqWPV7lWMyB7BkMwhIb9vU/kmdlVHfqSyT31sKt/E3gP3jvhnGRNOS5YsYcmS\nJb06Ryi9kr4LXA/sC7wOTAfOV9WQPllERgELVXX/EI5dDxysqm0m37FeSeHl8Xn4YvsX3X7fwrUL\nefXrV/nHyf8IWyy5abmMyh1Fkqvj+xRVZWP5Rkpqojvj+5i8MeSnW+9s03dFqlfS/wFnAOcDjwOH\nhJoUAnHRQdWTiBQEbR+Gk6jic0Y2A8D3xn2Pb/Z8w7qSdWE7Z1ldGat2repwzIOqsr5sfdSTAjjV\nVja2wSSaUNoYUNUSVX1JVRep6u5QTy4iC4D3gQkisklE5orIhSJygf+QmSKyQkQ+B/4C/Kjbv4GJ\nqhR3CmftexaPfflYWM8bmB57a+XWFm0YPvXxzZ5vwtqbqTsavY0UVxTH5LONiZUuq5LihVUlhVdP\nq5LAucM//cnTeWrmUwzODP8qr1kpWYzJG4Nb3Hyz55uojZ7uzD6D9iErJSvWYRjTbRGpSjL9U2dL\ne3YlNy2Xk8afxFMrnwpjRM2qGqpYuWsla0rWxEVSAGzdBpNQQhngNk5EUv3bRSJyuYh0f14E06+c\nM+Ucnl/9fI+7nHbF6/NG7Nw9UeepY3vV9liHYUxUhFJieBbwish44G5gJNC9GdVMvzMyZyRTC6ay\ncO3CWIcSNduqtnW4Gpwx/UkoicGnqh7gdOBvqvoLYFhkwzJ9wez9Z7NgxYKE6bWjqiFPP25MXxZK\nYmgUkVnAj4FF/n3JkQvJREM4Zjo9oOAAclJzuOvTu6K+9kKsVNZXxqTbrDHRFEpimAscAdyiqutF\nZAzwSGTDMn2BiPD/jv9/rC5ZzTnPnRPSEqD9QXFFccIkQpOYrLtqgvKpj8+3fR6Wc6kqb65/kz9+\n8EeOKjyKSw+9lJy0nLCcO14NzBjI6NzRsQ7DmC5FpLuqiHxfRD4XkVIRqRCRShGp6HmYpr8REU4Y\newJPn/k0Sa4kznrmLF75+pW47N5ZWlvKK1+/0us7/pKakrjpSmtMuIUyV9LXOFNifBnLW3YrMYRX\nOEsMra3YuYJb372V3LRc5k2fFxcrom2r3MYjyx/h1W9eZXDGYMbkjuG3x/220/mZupKalMq+g/fF\nJTYcyMSvSA1w2wyssKty/9KbAW5dmTJkCg+f9jBHjjiSuS/O5d7P7o3I6m2hWL9nPfOXzGf287NJ\nT07nqZlP8fBpD1PVUMX8JfN71aOq3lNvYxtMvxRKieFQ4GbgPwRNt62qf4psaG3isNwURqrKZ9s+\ni/jnbK/azu/f/z0byzZy3dHXcfCwgyP+mQArd63kgWUP8MWOL/jR5B9x1r5nMSB1QNPrdZ46rnzt\nSgoyC7jxmBt7fNcvIkwaNIn05PRwhW5MWPWkxBBKYngdqAK+BJqW3VLVX/ckyJ6yxBBe0UoMAUs2\nLOEP7/+Bw4YfxhWHX0FuWvgHz6sqn277lAeWPcCGsg2ct/95nDbxNNKS0to9vraxlitevYLCnEKu\nO/q6HieHrJQs9hm0T29CNyZiIpUYVqjqlF5FFgaWGMIr2okBnHWf7/r0Ll795lUuPfRSTplwSljG\nU/jUxzub3uHBZQ9SXl/O+Qecz4zxM0h2dz3cprqhmstevYwJ+RP45fRf9jieUbmjGJQxqEfvNSaS\nIpUYfg+8oaqv9ya43rLEEF6xSAwBq3ev5tZ3biUtOY3rjrqux90+PT4Pb3z7Bg8se4AkVxJzp87l\n2NHH4na5u3WeqoYqLn3lUvYbsh9XTbuqR8nB7XIzefDkkJKRMdEUqcRQCWTitC804iy6o6qa3dNA\ne8ISQ/h9uvXTmH221+fl6ZVPc+/n9zJz0kzmTp1LalJqSO+t99SzaN0iHv7iYQoyC5g7dS7TRkzr\nVemjsr7+W5dwAAAgAElEQVSSi16+iEP3OpTLD7u8R+cKZSU6Y6It7IlBnP86RqpqzCeIscQQfrFM\nDAE7q3fyxw/+yNqStcw7ah6HDz+8w2OrG6p5bvVzPPblY0wcOJHzp57P1KFTwxZLeV05F710EdML\np3PxIRf3ONGkJ6eTlZJFVkoWA1IGWCnCxFSkSgxfqup+vYosDCwxhF88JIaAdze9y+/f+z37F+zP\nldOuZGDGwKbXyurKePKrJ3l65dMcNvwwzj/gfCYMnBCROPbU7uHCly7khDEncMHBF3T9hhCkJqU2\nJYmslKyQS0bGhEOkEsNDwN9V9ePeBNdblhjCL54SAzi9hO757B4Wrl3IRYdcxJEjj2TBlwtYtG4R\nx40+jjkHzInKYLmSmhIufOlCTh5/Mj858CdhP3+yO7lForCuriaSIpUYVgPjgY1ANc1tDPv3NNCe\nsMQQfvGWGALWlazj1ndv5Zs933DaPqdx7n7nUpBVENUYdtfs5oKFF3D6pNM5b//zIvpZSa6kpqqn\nrJQsMpIzwtJbyxiIXGIY1d5+Vd3YnQ/qLUsM4ReviQGcLqiN3saYVrvsqNrBBYsu4OwpZzNryqyo\nfa5LXKS4U0hyJZHsTibZldzmZ+C1aFFVPD5P06PR19jiudfnbYopyZXkbAfFadOGxE5EEkO8sMQQ\nfhX1FZTXlVNeX069p77rNySgbZXbuHDRhZx3wHmcue+ZsQ6nBRFpugAHLsgdJZHWXXiDL/StL/KN\n3sY2SaC3izG5xNVp4mi9z0pM4WOJwfRYnaeuKVFUNlTG5cyosVJcUcyFiy7kZwf9jNMmnhbrcHrE\nJa6mEkbgDj+euV3uFkkiMLdXIGEI0mK79WuB5529Fvw8eF849ncUQ+u4u9ruSFf/fSrNr6cnp1ti\nML3nUx+V9ZWU15dTXlceswnw4smm8k38/KWfc/EhF/P9Cd+PdTjGhOyQ4Yd0OzHYSBzThktc5KTl\nOIvt5DiliUCVU1VDVUKWJgpzCvnnyf/kopcuwu1yM2P8jFiHZEzEWGIwXUpLSiMtK42CrAJ86muq\ncqqor0io0sTo3NH8fcbfufjli0l2JXPC2BNiHZIxEWGJwXSLS1zkpuU2zY5a21jLzuqdlNSWJERJ\nYlz+OP42429c9spluMXNsWOOjXVIxoSd9SEzvZKenM6o3FFMHjyZ/PT8WIcTFRMGTuCvJ/2V2967\njbc3vh3rcIwJO0sMJixSk1IZkzeGyUMmk5eeF+twIm7ioIn8+Xt/5ua3b+b9ze/HOhxjwiqiiUFE\n7hORHSKyvJNj7hCRdSKyTETCNyOaiYm0pDTG5o1l0uBJTuN1PzZ58GRuP/F2blpyEx9t+SjW4RgT\nNpEuMTwAfK+jF0VkBjBOVfcGLgTujHA8JkoykjMYnz+eiYMmkp0a1Rnao+qAggP43Qm/41eLf8Un\nWz+JdTjGhEXExzH4p9RY2N7cSiJyJ/CWqj7pf74KKFLVHe0ca+MY+rCqhiq2VGyhqqEq1qFExMdb\nPubaxdcyY/wMhg8YzojsEYzIHsFeA/YixZ0S6/BMAuuL4xiGA5uDnm/x72uTGEzfFlgXuaK+gq2V\nW6luqI51SGF16PBD+cfJ/+DjLR+zoWwD7256l+LKYnZU7WBgxkBGZo9kRPaIFkljRPYIslKyYh16\nXFBVahprKKsrY0/dHsrqyiirK6O8vhyXuEh1p5LiTiHVnUpqUtfbKe6Ubq/kZ5rFOjF0y/z585u2\ni4qKKCoqilkspmeyU7PJTs2mrK6MrZVbqW2sjXVIYbPPwH3YZ+A+LfZ5fB62V22nuKK46bFi5wqK\nK53ttKQ0J0kMaE4WwwcMZ0DqgKaLXVpSWtPFrq/MIdTobaS8vpw9tc0X+bL6subn9WWU1TbvL6sr\nI8mVRG6q0xU6Nz2X3NRcctJy8KmPBm8D9Z566r31zra3nnpP2+0GbwN1njoavA24Xe4WCSPFnUKa\nO42UpJSmSQrBPwWFfxoKQXD+L22mpmia0iJ4+oqgfYHnnWld6xE8dUXwdiCuwBxSrR+B6ULaexQv\nL2bz8s24xNXjyQvjrSppNXCMVSUljj21e9hauZU6T12sQ4k6VaWktqRF0iiuKGZr5VaqGquo99RT\n56lruvB5fB7n4paU5iQMt/MzNSm16QKY6nYSSSCZtH4tyZWEx+fBpz586sOrXrw+b9O2T314fd6m\n7dbPg4/1qhefz9lu8DZQXl/u3PHX7qHeW09Oag65abnkpeU1jX1p/chLz2tKAGlJaWH9bj0+T9sE\n4q2nweNse3yepmOb/ue/xgT2Ae3ub72v6Tj//o7mUWr3eQfJRFXxqKfFhIadTXbY5uF/7+IfL46/\nuZJEZDROYmizCpyInAxcoqr/JSLTgL+o6rQOzmOJoZ9SVUprS9lWtc1mee2E1+dtusDVeepaJI3A\n3XSb/d6Wr3l9XtwuN25x43a5cYkLt/h/+vcH7jSTXElN28GvtdgWNy6XixRXSosLflZKVp8p3fR3\ncdfGICILgCJgoIhsAm4CUnAW+rlbVV8WkZNF5GucRYDmRjIeE59EhIEZA8lPz6ektoRtldsSaqqN\nULldbtJd6bbim4k4m13VxB1VZVfNLkprS7v9vlD41JeQVVcmMcVdicGYnhARhmQOYUjmkIh9RmV9\nJTuqd1BeVx6xzzCmr7LEYBLSgNQBDEgdQJ2njh1VOxJmEsC+JvifxJososeqkozB6V65q2YXu6p3\nNfVW6Q6PF7xe50LmdoG4wOUCl13MQuLxQkMDNNRDfdDPxlZNTSI43Uml+bkE7W/aF+L+wL+RBP8M\n2m76t2znNZf0jWRlVUnG9FCyO5m9BuzF0Kyh7K7ZzdbynVTX1+P1X/C9HvB4nO1AEgg893aRR9q9\noLjaJpDAay6381pSErjd/p9JfT/JqEJDY9DF358AGhq7/g6Dz4FCvNwiSgdJxe12/s3cbv+/obt5\nn8vV/DxeE4slBpNQVKGiwrkoNTY6F/fAw3nuwuMZgupgGhrLKG3YQa2nd6O01QdeH/R2lWXxJ4sk\nd9uk0eK5/0IUK15f24t/4O6/vxX6VUG94Ovh+13tJA63/8YgMHA7kDtaJBFpu6+pFEXbfd1licEk\nhKoqKCmBPXucu/yuiAjZKXlkp+RR46mipH47VY2xbahWn3NxbQzhWAm6a00KunN1+wfCdnZ97uri\n3d7r6mtOBKHe/Yci2eXMM+VVL4ov5HagxgahqiKJqgo3lRVJVJYnURX0s6oiCZ9XSEnzkZLqIzXN\nS2qaj9TA81T/dvC+FtteklMUVy+nIfV5nUco/6bRZInB9Fv19U4yKC11tnsqIymLjKTx1HvrKK3f\nQXlj/DdUqzaXhPrSkMEUdxoZ7iwykgaQkZRFsiuFhgYoL4eyMthT5mPPHh97ypSycqWszNlfXg7l\nZUJ5uVBR7qKhXhiQ4yM7x8uAbA9Z2R6ycjwMyGlk4JBGRo2vwe1WGupd1Ne5qa9zUV/noroyyb/P\neQS22+5z09ggJKcoqWnelokjzUdmppf0TC8ZWc4jM9NHeqaXzCwvmVk+MrN8ZGT6yMzykpGppKS2\nPw1HYPS54vNv97bcGRprfDb9itfrJILSUqeUEAkeXyN7Gnaxp2EX3h40VJtmqe500t1ZNFZms3tb\nFluLk9i8GTZtgs2bobgYamshNxdycpyfnW0HHpmZnVejBE950VM+n1NCqq2FujrnUVvrPKqqoKpS\nqK4WKiud54GfgUdlZfM+txuyspzHgAHOz7S0jn4HJ3aRwNgdBQn+nRTE/9up8s5bGfE3JUa4WGIw\nHQm0G5SUOHeP0foz8amPsobdlNbvpNHXl+7Lo08VqiuTKN2WTenWHHZuzWD7lhS2FLvYtMm5AI4c\nCYWFzs/A9ogRzoU/Xhtpw0HVKdEGJ4/KSifRBB/T+j2dnS94+7rrxBKDSRw1Nc1VRZ4Y3LiXlcGq\nVbBqlbJiZSOr10BNtQuXS3G7nfpncSsul7PtDtp2udXfi6X5WOc1xeWG1DQvaRk+0jO8pKX7SMvw\nkpbuJT3D5/zMdPYHv56a5ut1nXdvVVW42b4llR1b0tixJY1d2zLYuSWdrcXJoFBYKIwY0ZwAAj+j\ncfFPSnLuwtPTnZ9ut3PX39nD6+34tb5yOTrkEEsMpp9raHASQUlJyzuqSCspcZLA6tXOY9Uq585u\n4kSYNMn5OXEiZGc7Fw2PR/H4fHi9Pmfbq3h9Pn93Vx9er7Ov5X7F599XVyfU1rioq3FTW+Ns19YK\n1dWu5uc1Qk0N1NQItTVCfT2kpkFGBmRmKpkZznZGppCZ4VSvJCWB16f4fOqfLVXxerV5ny+wHXwc\nzn4vTc/VB6riXCB9Qn2di51bU/F6heEjvIwsFMYUJlFYKE0JIDc3Onf+bndzAggkgfR0SE4O7+eo\ndp5AWu8Lft7Vdm+07ql00EGWGEw/5PM5vYlKSpwidqTt2tV88V+1CtasceqNJ02CffZxfk6aBMOH\nE/M79GBerxNndbVTmgr8DGxXVzvHSGBgl6vldnvPg/cFejqJKIgPxIeKD3H5SElRxhWmk58vUav2\ncbnaTwAp/WDBvNYlktbfaXvdVDsiYonB9BMeT3NVUVlZ7++i2qMKO3e2LQl4PG1LAsOH9+967ngm\n0n4CSE2NdWR9gyUG0yeo+gc+dfKIVCJYvRrefhu++spJAtAyAUyaBEOHWhKINrfbudCnpjp3/IHt\nwHP79+g5SwwmLni9HV/w6+udEcbRogorV8Ibb8Dixc4F5rjjYP/9nURQUGAXnWgQaXnBb33xd9vy\nzBFjicFEXGNj81QSge3WF/9QRhZHkqpTInjjDXjzTafR8fjj4YQTYMIESwSRkpTU8YU/Odm+91ix\nxGB6JDAhXOBCH/wI3u/xxG8XPZ8PVqxoTgZpaU4iOOEEGD8+8helrCyn102g2qPF7J/tPLp6Dfzz\n8GjL7Y72deeYzuLozvPg7UBDtYk/lhhMlxobncFg5eVO425jY2Tq86PB54Ply51EsHix0zXzhBOc\n0sG4cZFNBi6XM0I1MNI2ySaXMXHKEoNpV3W1fy4ZfzLoy3w++OKL5mSQldVcMhg7NrKf7XY3T7+Q\nk2N3yKZvsMRgAKdqqLy8uWQQi1HB4eT1Oskg0ICcm9tcMhgzJrKfnZLSXCrIyrJ6ctP3WGJIYLW1\nzaWC6ur4bQsIVUNDy2qigQOdRHD88TB6dGQ/Oz29ORlkZET2s4yJNEsMCcTnay4RBBae6Ytqa2HD\nBvj2W1i/vvmxfbvTTnDccU4yKCyMbByBxuPcXBs4ZfoXSwz9XF1dcyKorOxbpYLKSueC3zoBlJbC\nqFFOlVDgMXasM6tmuOe2CWaNxyZRWGLoZ7xeZ6K2QMmgN4vNRMuePW0v/t9+6zR6jx7d8uI/diwM\nGxa9wU1JSc2Nx9nZ1nhsEoMlhj7O42m5iEdtbXyXCrZuhQ8/dKaZCCQA1bZ3/2PGxG6EcUaGkwxy\ncpzZRY1JNJYY+hiPp+XCHLW1sY6oczU18MknTjL48EMn7sMPh/32cxLA6NFOI3Ese+64XE5pIJAM\nIlkdZUxfYIkhzjU2drxCUzzy+WDtWvjgAycRrFoFkyc7yeCII2DvveOjOibQpTQnx2k3sC6lxjSz\nxBBnGhpaJoK+0Eawe3dzieCjj5y77yOOgGnT4OCDna6csSbiVAsFSgXxEJMx8SouE4OInAT8BXAB\n96nq71q9fgzwIvCtf9dzqvrbds4T94mhvr5lIugLXUjr62HZsuZksH07HHpoczIYNizWETqSklpW\nEdlsnMaEJu4Sg4i4gLXA8cBW4GPgbFVdHXTMMcDVqvqDLs4V14lh9WpnYFlfUFUFr78OS5Y4I4rH\njWuuHtp33/jpupme3rLh2KqIjOm+niSGSF8CDgPWqepGABF5AjgVWN3quD79n3xJSfwnBa8XPv4Y\nFi2Cd991EsEPfgC//a1zJx5rKSmBtYqbf1qpwJjYiHRiGA5sDnpejJMsWjtCRJYBW4BfqOrKCMcV\nNj6f020zXm3a5CSDl16C/Hz4/vfhmmucxtpYSUpqvvgHEoH1HjImfsRDpcGnQKGq1ojIDOAFYEKM\nYwrZzp3x15ZQVeVMOLdwIWzeDDNmwF/+4vQiija327nwB5cGbMoJY+JbpBPDFiB4lpsR/n1NVLUq\naPsVEfmniOSramnrk82fP79pu6ioiKKionDH2y0ej9NYGw98Pvj0U/j3v+Gdd5wG5DlzYPr06LUZ\niLStDkpLi85nG2McS5YsYcmSJb06R6Qbn93AGpzG523AUmCWqq4KOqZAVXf4tw8DnlLV0e2cK+4a\nnzdvdkoMsVRc7FQTLVrk9OE/5RQ46STIy4vO5wc3DqenWwOxMfEm7hqfVdUrIpcCr9PcXXWViFzo\nvKx3AzNF5CKgEagFfhTJmMKlvh527YrNZ9fUONNRL1zoTEVx0klw++2wzz7R+XwRZ4RzQYGVCIzp\nj2yAWw99+60zYVy0+Hzw+edOMvjPf+DAA53SwVFHRa/h1u2GwYNhyBBrLDamr4i7cQzhFE+Jobra\nGbcQDVu2wMsvO1VFaWlOMpgxw7ljj5aUFCcZDBpkXUiN6WssMUTJmjVOz59I8Hickcjvvec89uxx\nlrH8wQ9g4sTo1uGnp8PQoU57hbUdGNM3WWKIgrIy+Oab8J5z9254/30nESxdCiNHOr2Jpk93RiJH\ne6K67Gyn/SAeBr4ZY3rHEkOEqcLKlb2fFdXrdWYqfe89ZxRycbEzEnn6dDjyyOhWEwWIOCWDggJb\n59iY/sQSQ4Tt2uWMJO6Jigpn+ur33nN+5uU5DcfTp8MBB8RufiKXy2k7KChw2hKMMf2LJYYI8nph\nxQqnDSAUqvD1106J4L33YN06OOig5iqiWM9ampzsNCgPHmwNysb0Z5YYImjrVti2rfNjamqcNoJA\nw3FyslMqOOooJynEw1QQqalOg3KsV1ozxkSHJYYIaWx0Sgs+X/uvb98Ot9wCy5c7K5xNn+4kg8LC\n+Ln4ZmY61UXRGhFtjIkPlhgiZONGp+dQeyoq4Kc/hRNPhFmz4m/B+ZwcJyEMGBDrSIwxsRB3U2L0\nB7W1znoL7amvh6uvdha4+elPoxtXZwI9jIYOtWUvjTHdZ4mhC1u2OA3JrXm9cP31TuPtFVdEP672\nWA8jY0w4WGLoRGUllJe33a/qTFpXWQl33BH9AWitJSU19zCKl2U5jTF9l11GOlFc3P7+hx5ypq24\n557Y3pmnpjqlg4EDY5+cjDH9hyWGDpSWOt1PW1u0CJ57Du67D7Kyoh8XOCOThw51lueMl15Pxpj+\nwxJDO1SdtoXW3n8f/vY3uOsup9om2mwOI2NMNFhiaEd76zivXAk33eS0LYweHb1YRJySwdChNoeR\nMSY6LDG04vW2HeFcXAxXXQW/+pUzr1E0ZGU5XU7z8mxRHGNMdFliaGXbNic5BJSWwmWXwc9+BkVF\nkf3szEzIz7dkYIyJLUsMQRoanGqkgJoa+J//ge99D374w8h8ZmZmc8nAxh4YY+KBJYYgwYPZPB6Y\nNw/23hsuvDC8n2PJwBgTzywx+NXUONVG4CSHW25xxgZce214uoRmZDQng3iYZdUYYzpiicEveDDb\nv/4F334Ld97Zu5HElgyMMX2RJQacaS8qK53tZ56BN95wBrD1ZAK69PTmBmRLBsaYvijhE0PwYLa3\n3nISwj33dG/dgpQUZ/K6vDxIS4tMnMYYEy0JnxhKSpyptZctg1tvdUY2jxgR2nuzs50R0Dk5NjWF\nMab/SOjE4PM5S3Z++y387//CzTfDxImdv8ftdkoHgwdbVZExpn9K6MSwY4dTjXTFFc54hWnTOj42\nI8NJBvn5NpOpMaZ/S9jE4PHA11/D5ZfDzJlw8sltj3G5nHaDwYPjb8lOY4yJlIjf+4rISSKyWkTW\nisgvOzjmDhFZJyLLRGRqpGMCWL8errwSDj4Y5sxp+VpqKgwfDvvt50yYZ0nBGJNIIpoYRMQF/B34\nHjAZmCUiE1sdMwMYp6p7AxcCd0YyJnAGs11yiTNr6VVXwaefLgGcRuTx42HKFGc203hdDW3JkiWx\nDqHb+lrMfS1esJijoa/F21ORLjEcBqxT1Y2q2gg8AZza6phTgYcBVPUjIEdECiIVkKqTFEpK4De/\ncUoHa9YsYcoUJynk5ETqk8OnL/5x9rWY+1q8YDFHQ1+Lt6cifU88HNgc9LwYJ1l0dswW/74dkQjo\nllvgnXdgwQIYN85pQ8jOth5GxhgTEKeVJe075ZTevb+xEb76ChYvdibHM8YY05ZoYDrRSJxcZBow\nX1VP8j+fB6iq/i7omDuBt1T1Sf/z1cAxqrqj1bkiF6gxxvRjqtqtIbiRLjF8DIwXkVHANuBsYFar\nY/4NXAI86U8kZa2TAnT/FzPGGNMzEU0MquoVkUuB13Eauu9T1VUicqHzst6tqi+LyMki8jVQDcyN\nZEzGGGM6F9GqJGOMMX1PQk3uICL3icgOEVketC9PRF4XkTUi8pqIxFWHVREZISKLReQrEflSRC73\n74/LuEUkVUQ+EpHP/fHe5N8fl/EGExGXiHwmIv/2P4/rmEVkg4h84f+ul/r3xW3MIpIjIk+LyCr/\n3/PhcR7vBP93+5n/Z7mIXB7nMV8pIitEZLmIPCYiKT2JN6ESA/AAzmC7YPOAN1R1H2AxcG3Uo+qc\nB7hKVScDRwCX+AcJxmXcqloPHKuqBwJTgRkichhxGm8rVwArg57He8w+oEhVD1TVQDfweI75r8DL\nqjoJOABYTRzHq6pr/d/tQcDBOFXdzxOnMYvIXsBlwEGquj9OU8EsehKvqibUAxgFLA96vhoo8G8P\nBVbHOsYu4n8BOKEvxA1kAJ8Ah8Z7vMAI4P+AIuDffeFvA1gPDGy1Ly5jBrKBb9rZH5fxthPnicA7\n8RwzsBewEcjzJ4V/9/RakWglhvYMUX8vKFXdDgyJcTwdEpHROHfhH+L8Q8dl3P4qmc+B7cD/qerH\nxHG8fn8GfgEEN7rFe8wK/J+IfCwiP/Xvi9eYxwC7ReQBf9XM3SKSQfzG29qPgAX+7biMWVW3An8E\nNuEMFC5X1TfoQbyWGNqKy9Z4EckCngGuUNUq2sYZN3Grqk+dqqQRwGEiMpk4jldE/gvYoarLgM66\nRcdNzH7T1anmOBmnivFo4vd7TgIOAv7hj7kap4ojXuNtIiLJwA+Ap/274jJmEcnFmWJoFE7pIVNE\nzqUH8VpigB2BuZlEZCiwM8bxtCEiSThJ4RFVfdG/O+7jVtUKYAlwEvEd73TgByLyLfA4cJyIPAJs\nj+OYUdVt/p+7cKoYDyN+v+diYLOqfuJ//ixOoojXeIPNAD5V1d3+5/Ea8wnAt6paqqpenPaQI+lB\nvImYGISWd4X/Bs73b/8YeLH1G+LA/cBKVf1r0L64jFtEBgV6PYhIOvBdYBVxGi+Aql6nqoWqOhZn\nEOZiVT0PWEicxiwiGf5SJCKSiVMH/iVx+j37qzI2i8gE/67jga+I03hbmYVzwxAQrzFvAqaJSJqI\nCM53vJKexBvrBpMoN84sALYC9f4vcS5OQ80bwBqcgXi5sY6zVczTAS+wDPgc+AznDjw/HuMG9vPH\nuAxYDvzKvz8u420n/mNobnyO25hx6uwDfxNfAvP6QMwH4MyGsAx4DsiJ53j9MWcAu4ABQfviNmbg\nJpwbseXAQ0ByT+K1AW7GGGNaSMSqJGOMMZ2wxGCMMaYFSwzGGGNasMRgjDGmBUsMxhhjWrDEYIwx\npgVLDCauiUh+0NTH20SkOOh5SAtNiTPdeqerfIvIxSLSenXBnsZ8qj/GZf4pkH/SxfHH+megbe+1\noSLykv9cX4nIC/79I0Tk8fbeY0xv2TgG02eIyI1Alar+qZ3XROPgj1lEUnBmPT1IVXf459kZpapf\nd/Kem4Hd2nJke+C1e3GmY/iX//kUVV0RofCNAazEYPqWpqlMRGSc/w76URFZAQwVkbtEZKk4CwRd\nH3TsOyKyv4i4RWSPiNzmvwN/T0QG+Y+5WZoXQXrHf8xH4iwqM82/P0NEnvGXAp72z2q6f6sYA4ug\nlAGoamMgKYjIEBF51h/jhyJymIiMBX4KXOMvBU1rdb5hOPMM4T/fiqDf/3P/9v3SvJjMLhG51r//\nl/7fYVnw92FMVywxmL5sH+CPqjpFnQnlfqnOgjVTgRPFWdCotRzgLVUNTF/eYTWPqh4O/C/ONAPg\nLIKyTVWnADf7P6f1e3bhTDuwUZwVtM72z1sDcAfwO3+MP8JZA/1b4F7gD6p6kKp+2OqUfwceFpE3\nRORa/yRoTR/n/8yfqDNj6ek4E6Q9JCIzgEL/73AgML2dpGNMuywxmL7sG1X9POj5uSLyKc5cTROB\nfdt5T42qvu7f/hQY3cG5nws6ZpR/+yjgCQBVXY4zCVwbqjoXZ6bLj4FfAnf5XzoBuNN/p/8CkCMi\nqZ39gqr6CjAWuM//+3wmInmtjxNnbYOngYvUmZf/ROAkEfkM5/sYB0xo/T5j2hNS450xcao6sCEi\n44HLgUNUtdI/bXZaO+9pCNr20vF/A/UhHNPh2g3+Kp8V/gbilcAF/uMPVWdK5OaTSGdLQICq7sGZ\n3fNxEXkFJ0GtbHXYXcACVX07KLbfquoDnZ7cmHZYicH0ZcFX1GygAqgSkWG0Xdu7vfd013s4VUCI\nyH7ApDYnFxkgzoI5AQfiLLcIztKhlwUde4B/s9Iff9tgRY4TkTT/djbOrKqbWh1zBZCkqn8O2v0a\n8N/+kgQiMlxEBob4e5oEZyUG05c19UJS1c9EZBXOlMMbgXfbO47QVtvq6Ji/4dTfr8C5Y18JlLc6\nRoBrReRuoBaoorkd41LgXyIyF3ADb+EkiheBp0XkdOCSVu0MhwJ/F5EGnBu5f6jqFyIyLuiYq4Fq\nfxWVAn9X1fv9bSwf+kskFcA5QEkIv79JcNZd1ZgQiYgb58683l919Rqwt6r6YhyaMWFlJQZjQpcF\nvN5ux1sAAAA+SURBVBk0sO4CSwqmP7ISgzHGmBas8dkYY0wLlhiMMca0YInBGGNMC5YYjDHGtGCJ\nwRhjTAuWGIwxxrTw/wGmFNGHVffUfgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f85b622d2e8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_learning_curve(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For an even more complex model, we still converge, but the convergence only happens for *large* amounts of training data.\n",
    "\n",
    "So we see the following:\n",
    "\n",
    "- you can **cause the lines to converge** by adding more points or by simplifying the model.\n",
    "- you can **bring the convergence error down** only by increasing the complexity of the model.\n",
    "\n",
    "Thus these curves can give you hints about how you might improve a sub-optimal model. If the curves are already close together, you need more model complexity. If the curves are far apart, you might also improve the model by adding more data.\n",
    "\n",
    "To make this more concrete, imagine some telescope data in which the results are not robust enough.  You must think about whether to spend your valuable telescope time observing *more objects* to get a larger training set, or *more attributes of each object* in order to improve the model.  The answer to this question has real consequences, and can be addressed using these metrics."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Recall, Precision and F1-Score\n",
    "\n",
    "Intuitively, [precision](http://en.wikipedia.org/wiki/Precision_and_recall#Precision) is the ability\n",
    "of the classifier not to label as positive a sample that is negative, and\n",
    "[recall](http://en.wikipedia.org/wiki/Precision_and_recall#Recall) is the\n",
    "ability of the classifier to find all the positive samples.\n",
    "\n",
    "The  [F-measure](http://en.wikipedia.org/wiki/F1_score>)\n",
    "($F_\\beta$ and $F_1$ measures) can be interpreted as a weighted\n",
    "harmonic mean of the precision and recall. A\n",
    "$F_\\beta$ measure reaches its best value at 1 and its worst score at 0.\n",
    "With $\\beta = 1$,  $F_\\beta$ and\n",
    "$F_1$  are equivalent, and the recall and the precision are equally important."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import zipfile\n",
    "with zipfile.ZipFile('../datasets/titanic.csv.zip', 'r') as z:\n",
    "    f = z.open('titanic.csv')\n",
    "    titanic = pd.read_csv(f, sep=',', index_col=0)\n",
    "titanic.head()\n",
    "\n",
    "# fill missing values for Age with the median age\n",
    "titanic.Age.fillna(titanic.Age.median(), inplace=True)\n",
    "\n",
    "# define X and y\n",
    "feature_cols = ['Pclass', 'Parch', 'Age']\n",
    "X = titanic[feature_cols]\n",
    "y = titanic.Survived\n",
    "\n",
    "# train/test split\n",
    "from sklearn.cross_validation import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1)\n",
    "\n",
    "# train a logistic regression model\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "logreg = LogisticRegression(C=1e9)\n",
    "logreg.fit(X_train, y_train)\n",
    "\n",
    "# make predictions for testing set\n",
    "y_pred_class = logreg.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[107,  21],\n",
       "       [ 52,  43]])"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.metrics import confusion_matrix\n",
    "confusion_matrix(y_test, y_pred_class)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![f1score](images/Precisionrecall.svg.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "precision_score  0.671875\n",
      "recall_score     0.452631578947\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import precision_score, recall_score, f1_score\n",
    "print('precision_score ', precision_score(y_test, y_pred_class))\n",
    "print('recall_score    ', recall_score(y_test, y_pred_class))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### F1Score\n",
    "\n",
    "The traditional F-measure or balanced F-score (F1 score) is the harmonic mean of precision and recall:\n",
    "\n",
    "$$F_1 = 2 \\cdot \\frac{\\mathrm{precision} \\cdot \\mathrm{recall}}{\\mathrm{precision} + \\mathrm{recall}}.$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "f1_score     0.540880503145\n"
     ]
    }
   ],
   "source": [
    "print('f1_score    ', f1_score(y_test, y_pred_class))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary\n",
    "\n",
    "We've gone over several useful tools for model validation\n",
    "\n",
    "- The **Training Score** shows how well a model fits the data it was trained on. This is not a good indication of model effectiveness\n",
    "- The **Validation Score** shows how well a model fits hold-out data. The most effective method is some form of cross-validation, where multiple hold-out sets are used.\n",
    "- **Validation Curves** are a plot of validation score and training score as a function of **model complexity**:\n",
    "  + when the two curves are close, it indicates *underfitting*\n",
    "  + when the two curves are separated, it indicates *overfitting*\n",
    "  + the \"sweet spot\" is in the middle\n",
    "- **Learning Curves** are a plot of the validation score and training score as a function of **Number of training samples**\n",
    "  + when the curves are close, it indicates *underfitting*, and adding more data will not generally improve the estimator.\n",
    "  + when the curves are far apart, it indicates *overfitting*, and adding more data may increase the effectiveness of the model.\n",
    "  \n",
    "These tools are powerful means of evaluating your model on your data."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  },
  "name": "_merged"
 },
 "nbformat": 4,
 "nbformat_minor": 0
}