4.2.2.6 Lab - Evaluating Fit Errors in Linear Regression - ILM.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Title](Images/cisco.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lab - Evaluating Fit Errors in Linear Regression <font color='red'>(Instructor Version)</font>\n",
    "<p>\n",
    "<font color='red'>**Instructor Note**: Red font color and completed code cells indicate that the information appears in the instructor copy only.</font></p>\n",
    "### Objectives\n",
    "In this lab, you will become familiar with the concepts of evaluating fit errors in linear regression. \n",
    "<li>**Part 1: Import the Libraries and Data  **</li>\n",
    "<li>**Part 2: Calculate the Errors **</li>\n",
    "### Scenario / Background\n",
    "In statistics, linear regression is a way to model a relationship between dependent variable $y$ and independent variable $x$. The goal of regression is to find a model that describes the data as accurately as possible.\n",
    "\n",
    "In this lab, you will use the sales data and result of the linear regression from a previous lab to evaluate the accuracy of the model.\n",
    "### Required Resources\n",
    "* 1 PC with Internet access\n",
    "* Python libraries: `pandas`, `numpy`, and `sklearn`\n",
    "* Datafiles: stores-dist.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 1: Import the Libraries and Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this part, you will import the libraries and the data from the file `stores-dist.csv`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Step 1: Import the libraries."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this step, you will import the following libraries:\n",
    "\n",
    "* `numpy` as np\n",
    "* `pandas` as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Code Cell 1\n",
    "\n",
    "# This lab produces some minor warnings that can be ignored.\n",
    "# These warnings appear because some libraries are updated more often than others\n",
    "# and the system is letting the user know that some function will be depricated soon\n",
    "# Use the following code to prevent the warnings from being displayed, or comment them out\n",
    "# to see the warnings\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn import model_selection\n",
    "from sklearn.linear_model import LinearRegression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Step 2: Import the data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this step, you will import the data from the file `stores-dist.csv`, change the column headings, and verify that the file is imported correctly.\n",
    "\n",
    "The column headings, `annual net sales` and `number of stores in district` are renamed to make it easier during data processing.\n",
    "\n",
    "* `annual net sales` to sales\n",
    "* `number of stores in district` to stores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>district</th>\n",
       "      <th>sales</th>\n",
       "      <th>stores</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>231.0</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>156.0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>10.0</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>519.0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>437.0</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   district  sales  stores\n",
       "0         1  231.0      12\n",
       "1         2  156.0      13\n",
       "2         3   10.0      16\n",
       "3         4  519.0       2\n",
       "4         5  437.0       6"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Code Cell 2\n",
    "\n",
    "# Import the file stores-dist.txt\n",
    "salesDist = pd.read_csv('./Data/stores-dist.txt')\n",
    "\n",
    "# Change the column headings\n",
    "salesDist.columns = ['district','sales','stores']\n",
    "\n",
    "# Verify the imported data\n",
    "# SOLUTION:\n",
    "salesDist.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The cdistrict column is not necessary for the evaluation of the linear regression fit; therefore, the column can be dropped."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sales</th>\n",
       "      <th>stores</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>231.0</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>156.0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10.0</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>519.0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>437.0</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   sales  stores\n",
       "0  231.0      12\n",
       "1  156.0      13\n",
       "2   10.0      16\n",
       "3  519.0       2\n",
       "4  437.0       6"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Code Cell 3\n",
    "# Drop the district column.\n",
    "sales = salesDist.drop('district',axis=1)\n",
    "\n",
    "# Verify that the district column has been dropped.\n",
    "sales.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 2: Calculating the Errors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this part, you will use numpy to generate a regression line for the analyzed data. You will also calculate the centroid for this dataset. The centrod is the mean for the dataset. The generated simple linear regression line must also pass through the centroid.\n",
    " \n",
    "You will also use sklearn.metrics to evaluate the linear regression model. You will calculate the $R^2$ score and mean square error (MSE)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Step 1: Assign the x and y variables.\n",
    "Assign the sales from the dataframe as dependent variable $y$, and stores from the dataframe as the independent variable for $x$ axis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Code Cell 4\n",
    "#dependent variable for y axis\n",
    "y = sales.sales \n",
    "#independent variable for x axis\n",
    "# SOLUTION:\n",
    "x = sales.stores "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2: Calculate the y values in the model\n",
    "In a previous lab, you calculated the components for the linear regression fit with a polynomial model using `np.polyfit` to calculate a vector of coefficients `p` that minimizes the squared error. By using `np.poly1d`, you can compute the corresponding value for each value of `x` in the estimated polynomial model. \n",
    "\n",
    "To recall the slope and y-intercept of the line, use the variable `p`. The array `p` displays the coefficent in a descending order. For a first order polynomial, the first coefficient is the slope (`m`) and the second coefficent is the y-intercept (`b`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The array p(x) stores the calculated y value from the polynomial model for each x value,\n",
      "\n",
      "[ 169.93468442  134.14759895   26.78634257  527.80553905  384.65719719\n",
      "  420.44428266  205.72176988  134.14759895   26.78634257  277.29594081\n",
      "  527.80553905  313.08302627  456.23136812   62.57342803  169.93468442\n",
      "  205.72176988  420.44428266   98.36051349  313.08302627  527.80553905\n",
      "  563.59262451   62.57342803  134.14759895  348.87011173  384.65719719\n",
      "  563.59262451  277.29594081].\n",
      "\n",
      "The vector of coefficients p describes this regression model:\n",
      " \n",
      "-35.79 x + 599.4\n",
      "\n",
      "The zeroth order term (y-intercept or b) is stored in p[0]: 599.3797099726614.\n",
      "\n",
      "The first order term (slope or m) is stored in p[1]: -35.787085462974005.\n"
     ]
    }
   ],
   "source": [
    "# Code Cell 5\n",
    "# compute the y values from the polynomial model for each x value\n",
    "order = 1\n",
    "p = np.poly1d(np.polyfit(x, y ,order))\n",
    "\n",
    "print('The array p(x) stores the calculated y value from the polynomial model for each x value,\\n\\n{}.'.format(p(x)))\n",
    "print('\\nThe vector of coefficients p describes this regression model:\\n{}'.format(p))\n",
    "print('\\nThe zeroth order term (y-intercept or b) is stored in p[0]: {}.'.format(p[0]))\n",
    "print('\\nThe first order term (slope or m) is stored in p[1]: {}.'.format(p[1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Step 3: Use different measures to evaluate models.\n",
    "In this step, you will use `sklearn` to evaluate models. `Sklean` offers a range of measures. You will calculate the $R^2$ score, mean squared error (MSE), and mean absolute error (MAE) using the functions in `sklearn`.\n",
    "\n",
    "To calculate the value for each measure, provide the values from `y`, which is the observed values from the imported csv file, `stores-dist.csv` as the first argument. As the second argument, use the values from `p(x)`, which were calculated from your first order polynomial model in the form of:\n",
    "\n",
    "$$y = mx + b$$\n",
    "\n",
    "where the m is `p[1]` and b is `p[0]` in the `poly1d` results."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The $R^2$ (coefficent of determination) regression score function gives some information about the amount of fit of the model. The best possible score for $R^2$ is 1.0. This score indicates how well the model is explaining the observed outcome."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.83217523508888003"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Code Cell 6\n",
    "from sklearn.metrics import r2_score\n",
    "r2 = r2_score(y, p(x))\n",
    "r2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The mean squared error (MSE) is a measure of how well the model can be used to make a prediction. This number is always non-negative. The better values are closer to zero."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5961.3864659411583"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Code Cell 7\n",
    "from sklearn.metrics import mean_squared_error\n",
    "mse = mean_squared_error(y, p(x))\n",
    "mse"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The mean absolute error (MAE) is a measure of how close predictions are to the eventual outcomes. The MAE is an average of the absolute errors between the prediction and the true value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "61.223261178687302"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Code Cell 8\n",
    "from sklearn.metrics import mean_absolute_error\n",
    "mae = mean_absolute_error(y, p(x))\n",
    "mae"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All these measures allow you to determine how well your model can make prediction. In this lab, you only evaluated one model, simple linear regression. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<font size='0.5'>&copy; 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.<font>"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}