notebooks/AnyGivenSunday.ipynb

{
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  },
  "name": "",
  "signature": "sha256:a56c5ea1f33923505d3509128e1e8e33ca643dbca946851f28836c0aa0f1ef30"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Any Given Sunday: Football and a Machine Learning Rookie"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "I love football more than engineers love coffee, all my Turi friends know that. Throughout the course of an NFL season I have fantasy teams, point-spread pools, survivor pools and non-stop pontification on the latest terrible play call, awful refs and amazing plays (GO HAWKS!). Despite hearing about all the neato, life-changing, world-saving, cutting-edge, inspirational things machine learning could be used for, my first thought was \"Huh. I should learn that and use it for football.\"\n",
      "\n",
      "Well, let's take a shot.\n",
      "\n",
      "I'm no data scientist and this post won't be terribly math-y. I'm a software engineer that's fond of BBQ and learning by doing. Thankfully, I work with a slew of smart folks kind enough to assist with these educational shenanigans and provide guidance when I run off the rails. We'll apply the simplest of machine learning concepts, linear regression, to football in the simplest way and see what happens. You get to learn from all of the glorious mistakes I made so that you can go on to make bigger, better mistakes. We'll learn, together, [inch by inch](https://youtu.be/2traDe9a_XE?t=141)."
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Setting up the data"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "My dataset is from [Armchair Analysis](http://armchairanalysis.com), which is a pretty rad source of well-curated, nicely documented, NFL stats. You can grab yourself a copy for pretty cheap."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": true,
     "input": [
      "#Fire up the GraphLab engine\n",
      "import graphlab as gl\n",
      "import graphlab.aggregate as agg"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 22
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The team SFrame has all of one team's stats per game.  The games SFrame has info about each game itself, which teams played in it, where, etc.  So first, let me narrow it down to what I think are some relevant columns and join some info about each game to the team table."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "team = gl.SFrame.read_csv('C:\\\\Users\\\\Susan\\\\Documents\\\\NFLData_2000-2014\\\\csv\\TEAM.csv', header=True)\n",
      "games = gl.SFrame.read_csv('C:\\\\Users\\\\Susan\\\\Documents\\\\NFLData_2000-2014\\\\csv\\GAME.csv', header=True)\n",
      "# TID=team id\n",
      "# GID=Game id\n",
      "# TNAME=team name\n",
      "# PTS points,\n",
      "# RY rushing yards,\n",
      "# PY passing yards,\n",
      "# RA rushing attempts\n",
      "# PA pass attempts,\n",
      "# PC pass completions\n",
      "# SK sacks against,\n",
      "# FUM fumbles lost,\n",
      "# INT int for defense,\n",
      "# TOP time of possession,\n",
      "# TD tds,\n",
      "# TDR td rushing,\n",
      "# TDP td passing,\n",
      "# PLO total plays offense,\n",
      "# PLD total plays defense,\n",
      "# DP points by defense\n",
      "team = team.select_columns(\n",
      "    ['TNAME', 'TID', 'GID', 'PTS', 'RY', 'RA', 'PY', 'PA', 'PC', 'PU',\n",
      "     'SK', 'FUM', 'INT', 'TOP', 'TD', 'TDR', 'TDP',  'DP'])\n",
      "team['TOP'] = team['TOP'].astype(float)\n",
      "\n",
      "team = team.join(games.select_columns(['GID', 'SEAS', 'WEEK']), 'GID')\n",
      "team['SEAS'] = team['SEAS'].astype(str)\n",
      "\n",
      "# restrict to regular season\n",
      "team = team[team['WEEK'] < 18]\n",
      "\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<pre>PROGRESS: Finished parsing file C:\\Users\\Susan\\Documents\\NFLData_2000-2014\\csv\\TEAM.csv</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Finished parsing file C:\\Users\\Susan\\Documents\\NFLData_2000-2014\\csv\\TEAM.csv"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Parsing completed. Parsed 100 lines in 0.120085 secs.</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Parsing completed. Parsed 100 lines in 0.120085 secs."
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Finished parsing file C:\\Users\\Susan\\Documents\\NFLData_2000-2014\\csv\\TEAM.csv</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Finished parsing file C:\\Users\\Susan\\Documents\\NFLData_2000-2014\\csv\\TEAM.csv"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Parsing completed. Parsed 7978 lines in 0.116081 secs.</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Parsing completed. Parsed 7978 lines in 0.116081 secs."
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Finished parsing file C:\\Users\\Susan\\Documents\\NFLData_2000-2014\\csv\\GAME.csv</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Finished parsing file C:\\Users\\Susan\\Documents\\NFLData_2000-2014\\csv\\GAME.csv"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Parsing completed. Parsed 100 lines in 0.044031 secs.</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Parsing completed. Parsed 100 lines in 0.044031 secs."
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "------------------------------------------------------\n",
        "Inferred types from first line of file as \n",
        "column_type_hints=[long,long,str,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,str,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,str,str,long,long,long,long,long,long,str,str,str,str,str,str,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long]\n",
        "If parsing fails due to incorrect types, you can correct\n",
        "the inferred type list above and pass it to read_csv in\n",
        "the column_type_hints argument\n",
        "------------------------------------------------------\n",
        "------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Finished parsing file C:\\Users\\Susan\\Documents\\NFLData_2000-2014\\csv\\GAME.csv</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Finished parsing file C:\\Users\\Susan\\Documents\\NFLData_2000-2014\\csv\\GAME.csv"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Parsing completed. Parsed 3989 lines in 0.048034 secs.</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Parsing completed. Parsed 3989 lines in 0.048034 secs."
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n",
        "Inferred types from first line of file as \n",
        "column_type_hints=[long,long,long,str,str,str,str,str,str,str,str,str,str,str,str,long,long]\n",
        "If parsing fails due to incorrect types, you can correct\n",
        "the inferred type list above and pass it to read_csv in\n",
        "the column_type_hints argument\n",
        "------------------------------------------------------\n"
       ]
      }
     ],
     "prompt_number": 23
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Here's what we've got so far"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "team.head(5)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\"><table frame=\"box\" rules=\"cols\">\n",
        "    <tr>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">TNAME</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">TID</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">GID</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">PTS</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">RY</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">RA</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">PY</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">PA</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">PC</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">PU</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">SK</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">FUM</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">INT</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">TOP</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">TD</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">TDR</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">TDP</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">DP</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">SEAS</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">WEEK</th>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">SF</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">28</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">92</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">24</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">247</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">36</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">23</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">5</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">27.5</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">4</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">3</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2000</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">ATL</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">36</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">95</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">32</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">264</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">31</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">16</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">31.5</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">3</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">7</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2000</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">JAC</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">3</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">27</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">119</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">40</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">279</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">34</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">24</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">3</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">4</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">37.0</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">3</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2000</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">CLE</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">4</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">7</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">96</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">16</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">153</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">27</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">19</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">5</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">23.0</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2000</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">PHI</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">5</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">3</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">41</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">306</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">46</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">119</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">29</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">16</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">3</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">39.5</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">5</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">3</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">7</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2000</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
        "    </tr>\n",
        "</table>\n",
        "[5 rows x 20 columns]<br/>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 24,
       "text": [
        "Columns:\n",
        "\tTNAME\tstr\n",
        "\tTID\tint\n",
        "\tGID\tint\n",
        "\tPTS\tint\n",
        "\tRY\tint\n",
        "\tRA\tint\n",
        "\tPY\tint\n",
        "\tPA\tint\n",
        "\tPC\tint\n",
        "\tPU\tint\n",
        "\tSK\tint\n",
        "\tFUM\tint\n",
        "\tINT\tint\n",
        "\tTOP\tfloat\n",
        "\tTD\tint\n",
        "\tTDR\tint\n",
        "\tTDP\tint\n",
        "\tDP\tint\n",
        "\tSEAS\tstr\n",
        "\tWEEK\tint\n",
        "\n",
        "Rows: 5\n",
        "\n",
        "Data:\n",
        "+-------+-----+-----+-----+-----+----+-----+----+----+----+----+-----+-----+\n",
        "| TNAME | TID | GID | PTS |  RY | RA |  PY | PA | PC | PU | SK | FUM | INT |\n",
        "+-------+-----+-----+-----+-----+----+-----+----+----+----+----+-----+-----+\n",
        "|   SF  |  1  |  1  |  28 |  92 | 24 | 247 | 36 | 23 | 5  | 1  |  0  |  1  |\n",
        "|  ATL  |  2  |  1  |  36 |  95 | 32 | 264 | 31 | 16 | 2  | 0  |  1  |  0  |\n",
        "|  JAC  |  3  |  2  |  27 | 119 | 40 | 279 | 34 | 24 | 3  | 4  |  0  |  0  |\n",
        "|  CLE  |  4  |  2  |  7  |  96 | 16 | 153 | 27 | 19 | 5  | 1  |  1  |  0  |\n",
        "|  PHI  |  5  |  3  |  41 | 306 | 46 | 119 | 29 | 16 | 2  | 1  |  0  |  3  |\n",
        "+-------+-----+-----+-----+-----+----+-----+----+----+----+----+-----+-----+\n",
        "+------+----+-----+-----+----+------+------+\n",
        "| TOP  | TD | TDR | TDP | DP | SEAS | WEEK |\n",
        "+------+----+-----+-----+----+------+------+\n",
        "| 27.5 | 4  |  1  |  3  | 0  | 2000 |  1   |\n",
        "| 31.5 | 3  |  0  |  2  | 7  | 2000 |  1   |\n",
        "| 37.0 | 3  |  2  |  1  | 0  | 2000 |  1   |\n",
        "| 23.0 | 1  |  0  |  1  | 0  | 2000 |  1   |\n",
        "| 39.5 | 5  |  3  |  1  | 7  | 2000 |  1   |\n",
        "+------+----+-----+-----+----+------+------+\n",
        "[5 rows x 20 columns]"
       ]
      }
     ],
     "prompt_number": 24
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "##Predict the total season wins for a team\n",
      "Ok.  Great.  I have data... how do I data science it?  I know from intro reading that linear regression is probably the simplest thing for me to try.  But ... what type of question can I even answer with linear regression?  Well, the way I understand it, we can predict a continuous variable Y that changes based on the values of a bunch of other variables, X1, X2... etc.  So as all the X's change, our predicted Y is going to change.  Here  is the better, math-happy, grown-up explanation to achieve next level enlightenment.  The data science \"hello world\"-foobar example everyone on the planet uses to explain linear regression is predicting the price of a house.  How much will a house cost (Y) based on a bunch of info we have (Xs) like square footage, number of rooms, etc?  Well, I have a bunch of info about team stats.  What do I want to predict?\n",
      "\n",
      "Let's try to predict the total season wins for a team given that team's stats for the year.  Right now I don't have a season summary for each team but I can make one.  I'll also need to generate season win column for each team.  I\u2019m also going to split my data into a train set, and a testing set.  I can feed my model the training set to learn on, then reserve the test set to see how well the model performs against data it hasn\u2019t seen yet.  What I don't need to do is code up the algorithm, GraphLab Create has a toolkit for that.  "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# add a column to indicate if the team won this game or not\n",
      "winners = team.groupby(key_columns='GID', operations={'WIN_TID': agg.ARGMAX('PTS', 'TID')})\n",
      "team = team.join(winners, 'GID')\n",
      "team['WIN'] = team['TID'] == team['WIN_TID']\n",
      "\n",
      "#create season summary\n",
      "of_interest= ['PTS', 'RY', 'RA', 'PY', 'PA', 'SK', 'FUM', 'INT', 'TOP', 'TD', 'TDR', 'TDP', 'DP', 'WIN']\n",
      "season = team.groupby(['SEAS', 'TNAME'], {'%s_SUM' % x : agg.SUM(x) for x in of_interest})\n",
      "season_sums = filter(lambda x: '_SUM' in x, season.column_names())\n",
      "season_sums.remove('WIN_SUM')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 25
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "I now have season summaries that look like this"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "season.head()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\"><table frame=\"box\" rules=\"cols\">\n",
        "    <tr>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">SEAS</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">TNAME</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">INT_SUM</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">DP_SUM</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">RY_SUM</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">SK_SUM</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">RA_SUM</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">TD_SUM</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">FUM_SUM</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">TDP_SUM</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">TOP_SUM</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">PY_SUM</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">PTS_SUM</th>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2005</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">TEN</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">14</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">30</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1533</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">31</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">389</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">33</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">12</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">20</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">499.0</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">3597</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">299</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2013</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">ARI</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">22</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">39</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1560</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">41</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">402</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">41</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">9</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">24</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">496.5</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">4002</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">379</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2014</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">NYJ</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">15</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2285</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">47</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">501</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">27</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">9</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">16</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">497.0</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2946</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">283</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2005</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">ATL</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">13</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">23</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2563</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">39</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">514</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">39</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">16</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">19</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">486.5</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2679</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">351</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2003</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">PHI</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">11</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">9</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2026</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">43</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">406</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">43</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">11</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">17</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">454.5</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">3020</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">374</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2011</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">NYJ</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">18</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">27</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1699</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">40</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">436</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">45</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">16</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">26</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">493.0</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">3297</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">377</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2007</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">PHI</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">15</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1988</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">49</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">407</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">38</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">12</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">24</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">496.0</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">3755</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">336</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2010</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">NYJ</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">14</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">25</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2382</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">28</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">526</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">39</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">7</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">20</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">522.0</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">3229</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">367</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2012</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">PHI</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">15</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1893</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">48</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">407</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">29</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">22</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">18</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">476.5</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">3781</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">280</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2000</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">ATL</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">20</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">9</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1217</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">61</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">347</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">25</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">14</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">14</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">470.5</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2780</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">252</td>\n",
        "    </tr>\n",
        "</table>\n",
        "<table frame=\"box\" rules=\"cols\">\n",
        "    <tr>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">TDR_SUM</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">PA_SUM</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">WIN_SUM</th>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">8</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">588</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">4</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">12</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">573</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">10</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">11</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">495</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">4</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">17</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">450</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">8</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">23</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">481</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">12</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">14</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">543</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">8</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">12</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">575</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">8</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">14</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">520</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">11</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">10</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">614</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">4</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">6</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">514</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">4</td>\n",
        "    </tr>\n",
        "</table>\n",
        "[10 rows x 16 columns]<br/>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 26,
       "text": [
        "Columns:\n",
        "\tSEAS\tstr\n",
        "\tTNAME\tstr\n",
        "\tINT_SUM\tint\n",
        "\tDP_SUM\tint\n",
        "\tRY_SUM\tint\n",
        "\tSK_SUM\tint\n",
        "\tRA_SUM\tint\n",
        "\tTD_SUM\tint\n",
        "\tFUM_SUM\tint\n",
        "\tTDP_SUM\tint\n",
        "\tTOP_SUM\tfloat\n",
        "\tPY_SUM\tint\n",
        "\tPTS_SUM\tint\n",
        "\tTDR_SUM\tint\n",
        "\tPA_SUM\tint\n",
        "\tWIN_SUM\tint\n",
        "\n",
        "Rows: 10\n",
        "\n",
        "Data:\n",
        "+------+-------+---------+--------+--------+--------+--------+--------+---------+\n",
        "| SEAS | TNAME | INT_SUM | DP_SUM | RY_SUM | SK_SUM | RA_SUM | TD_SUM | FUM_SUM |\n",
        "+------+-------+---------+--------+--------+--------+--------+--------+---------+\n",
        "| 2005 |  TEN  |    14   |   30   |  1533  |   31   |  389   |   33   |    12   |\n",
        "| 2013 |  ARI  |    22   |   39   |  1560  |   41   |  402   |   41   |    9    |\n",
        "| 2014 |  NYJ  |    15   |   2    |  2285  |   47   |  501   |   27   |    9    |\n",
        "| 2005 |  ATL  |    13   |   23   |  2563  |   39   |  514   |   39   |    16   |\n",
        "| 2003 |  PHI  |    11   |   9    |  2026  |   43   |  406   |   43   |    11   |\n",
        "| 2011 |  NYJ  |    18   |   27   |  1699  |   40   |  436   |   45   |    16   |\n",
        "| 2007 |  PHI  |    15   |   0    |  1988  |   49   |  407   |   38   |    12   |\n",
        "| 2010 |  NYJ  |    14   |   25   |  2382  |   28   |  526   |   39   |    7    |\n",
        "| 2012 |  PHI  |    15   |   0    |  1893  |   48   |  407   |   29   |    22   |\n",
        "| 2000 |  ATL  |    20   |   9    |  1217  |   61   |  347   |   25   |    14   |\n",
        "+------+-------+---------+--------+--------+--------+--------+--------+---------+\n",
        "+---------+---------+--------+---------+---------+--------+---------+\n",
        "| TDP_SUM | TOP_SUM | PY_SUM | PTS_SUM | TDR_SUM | PA_SUM | WIN_SUM |\n",
        "+---------+---------+--------+---------+---------+--------+---------+\n",
        "|    20   |  499.0  |  3597  |   299   |    8    |  588   |    4    |\n",
        "|    24   |  496.5  |  4002  |   379   |    12   |  573   |    10   |\n",
        "|    16   |  497.0  |  2946  |   283   |    11   |  495   |    4    |\n",
        "|    19   |  486.5  |  2679  |   351   |    17   |  450   |    8    |\n",
        "|    17   |  454.5  |  3020  |   374   |    23   |  481   |    12   |\n",
        "|    26   |  493.0  |  3297  |   377   |    14   |  543   |    8    |\n",
        "|    24   |  496.0  |  3755  |   336   |    12   |  575   |    8    |\n",
        "|    20   |  522.0  |  3229  |   367   |    14   |  520   |    11   |\n",
        "|    18   |  476.5  |  3781  |   280   |    10   |  614   |    4    |\n",
        "|    14   |  470.5  |  2780  |   252   |    6    |  514   |    4    |\n",
        "+---------+---------+--------+---------+---------+--------+---------+\n",
        "[10 rows x 16 columns]"
       ]
      }
     ],
     "prompt_number": 26
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Awesome.  Let's get to it."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# predict number of wins\n",
      "#split the data into train and test\n",
      "season_train, season_test = season.random_split(0.8, seed=0)\n",
      "lin_model = gl.linear_regression.create(season_train, target='WIN_SUM', features=season_sums)\n",
      "lin_pred = lin_model.predict(season_test)\n",
      "lin_result = lin_model.evaluate(season_test)\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<pre>PROGRESS: Linear regression:</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Linear regression:"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of examples          : 362</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of examples          : 362"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of features          : 13</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of features          : 13"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of unpacked features : 13</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of unpacked features : 13"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of coefficients    : 14</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of coefficients    : 14"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Starting Newton Method</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Starting Newton Method"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "PROGRESS: Creating a validation set from 5 percent of training data. This may take a while.\n",
        "          You can set ``validation_set=None`` to disable validation tracking.\n",
        "\n"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: +-----------+----------+--------------+--------------------+----------------------+---------------+-----------------+</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: +-----------+----------+--------------+--------------------+----------------------+---------------+-----------------+"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: +-----------+----------+--------------+--------------------+----------------------+---------------+-----------------+</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: +-----------+----------+--------------+--------------------+----------------------+---------------+-----------------+"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: | 1         | 2        | 0.001001     | 5.115316           | 2.863893             | 1.634377      | 1.284790        |</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: | 1         | 2        | 0.001001     | 5.115316           | 2.863893             | 1.634377      | 1.284790        |"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: +-----------+----------+--------------+--------------------+----------------------+---------------+-----------------+</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: +-----------+----------+--------------+--------------------+----------------------+---------------+-----------------+"
       ]
      }
     ],
     "prompt_number": 27
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "How'd we do? "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print lin_result"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "{'max_error': 5.340498547412093, 'rmse': 1.7816995360472347}\n"
       ]
      }
     ],
     "prompt_number": 28
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Printing out lin_result shows us our max_error was around 5, and our rmse (root mean squared error, a favorite measure of how wrong you are) is around 1.8.  That's ... not bad?  Just browsing the first few, the model is generally close but not great."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print season_test.head(5)['WIN_SUM']\n",
      "print lin_pred[0:5]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "[4L, 11L, 5L, 10L, 4L]\n",
        "[5.928937053986296, 11.790802796979062, 4.276084765261491, 8.579262126730567, 6.401094017887274]\n"
       ]
      }
     ],
     "prompt_number": 29
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The first five actual win totals of my test data are (4,11,5,10,4) and my first five predictions out of lin_pred are (5.9, 11.8, 4.2, 8.7).  I can print the lin_model to get a little more information about what's going on here, or you can use lin_model.show() to see it all pretty in your browser."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "gl.canvas.set_target('ipynb')\n",
      "lin_model.show()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "javascript": [
        "$(\"head\").append($(\"<link/>\").attr({\n",
        "  rel:  \"stylesheet\",\n",
        "  type: \"text/css\",\n",
        "  href: \"//cdnjs.cloudflare.com/ajax/libs/font-awesome/4.1.0/css/font-awesome.min.css\"\n",
        "}));\n",
        "$(\"head\").append($(\"<link/>\").attr({\n",
        "  rel:  \"stylesheet\",\n",
        "  type: \"text/css\",\n",
        "  href: \"//turi.com/files/canvas/1.6.1/css/canvas.css\"\n",
        "}));\n",
        "\n",
        "            (function(){\n",
        "\n",
        "                var e = null;\n",
        "                if (typeof element == 'undefined') {\n",
        "                    var scripts = document.getElementsByTagName('script');\n",
        "                    var thisScriptTag = scripts[scripts.length-1];\n",
        "                    var parentDiv = thisScriptTag.parentNode;\n",
        "                    e = document.createElement('div');\n",
        "                    parentDiv.appendChild(e);\n",
        "                } else {\n",
        "                    e = element[0];\n",
        "                }\n",
        "\n",
        "                if (typeof requirejs !== 'undefined') {\n",
        "                    // disable load timeout; ipython_app.js is large and can take a while to load.\n",
        "                    requirejs.config({waitSeconds: 0});\n",
        "                }\n",
        "\n",
        "                require(['//turi.com/files/canvas/1.6.1/js/ipython_app.js'], function(IPythonApp){\n",
        "                    var app = new IPythonApp();\n",
        "                    app.attachView('model','Summary', {\"comparison\": null, \"selected_variable\": {\"comparison\": null, \"name\": [\"lin_model\"], \"view_file\": \"model\", \"view_component\": \"Summary\", \"view_params\": {\"model_type\": \"regression\", \"view\": \"Summary\"}, \"view_components\": [\"Summary\", \"Evaluation\", \"Comparison\"], \"model_type\": \"regression\", \"attributes\": {\"section_titles\": [\"Schema\", \"Hyperparameters\", \"Training Summary\", \"Settings\", \"Highest Positive Coefficients\", \"Lowest Negative Coefficients\"], \"sections\": [[[\"Number of coefficients\", 14], [\"Number of examples\", 362], [\"Number of feature columns\", 13], [\"Number of unpacked features\", 13]], [[\"L1 penalty\", 0.0], [\"L2 penalty\", 0.01]], [[\"Solver\", \"auto\"], [\"Solver iterations\", 1], [\"Solver status\", \"SUCCESS: Optimal solution found.\"], [\"Training time (sec)\", 0.002]], [[\"Residual sum of squares\", 966.9699], [\"Training RMSE\", 1.6344]], [[\"TDP_SUM\", 0.0716], [\"TDR_SUM\", 0.0661], [\"PTS_SUM\", 0.0413], [\"TOP_SUM\", 0.0195], [\"DP_SUM\", 0.0159]], [[\"(intercept)\", -2.9457], [\"TD_SUM\", -0.1688], [\"INT_SUM\", -0.1235], [\"FUM_SUM\", -0.0393], [\"SK_SUM\", -0.0379]]]}, \"evaluations\": [[\"season_test\", {\"max_error\": 5.340498547412093, \"rmse\": 1.7816995360472347}]], \"type\": \"Model\"}, \"ipython\": true, \"view_params\": {\"model_type\": \"regression\", \"view\": \"Summary\"}, \"model_type\": \"regression\", \"attributes\": {\"section_titles\": [\"Schema\", \"Hyperparameters\", \"Training Summary\", \"Settings\", \"Highest Positive Coefficients\", \"Lowest Negative Coefficients\"], \"sections\": [[[\"Number of coefficients\", 14], [\"Number of examples\", 362], [\"Number of feature columns\", 13], [\"Number of unpacked features\", 13]], [[\"L1 penalty\", 0.0], [\"L2 penalty\", 0.01]], [[\"Solver\", \"auto\"], [\"Solver iterations\", 1], [\"Solver status\", \"SUCCESS: Optimal solution found.\"], [\"Training time (sec)\", 0.002]], [[\"Residual sum of squares\", 966.9699], [\"Training RMSE\", 1.6344]], [[\"TDP_SUM\", 0.0716], [\"TDR_SUM\", 0.0661], [\"PTS_SUM\", 0.0413], [\"TOP_SUM\", 0.0195], [\"DP_SUM\", 0.0159]], [[\"(intercept)\", -2.9457], [\"TD_SUM\", -0.1688], [\"INT_SUM\", -0.1235], [\"FUM_SUM\", -0.0393], [\"SK_SUM\", -0.0379]]]}, \"evaluations\": [[\"season_test\", {\"max_error\": 5.340498547412093, \"rmse\": 1.7816995360472347}]]}, e);\n",
        "                });\n",
        "            })();\n",
        "        "
       ],
       "metadata": {},
       "output_type": "display_data"
      }
     ],
     "prompt_number": 30
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Not surprisingly, Xs that are positive coefficients are things like the sum of passing touchdowns, total points, and time of possession.  Xs that drag the Y value down are things like interceptions, sacks, and fumbles.  For  the life of me, I can't figure out why the sum of touchdowns would bring the win sum prediction down.  I had to ask my data science sherpa Chris why this could happen."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "> Chris:  I suspect that several of your features are highly correlated. This makes it tricky for the linear regression  model to have parameter estimates that aren't noisy; it doesn't know whether or not to make two highly correlated features be large or small, since there are ways for the predictions to be the same in either case. (Check out wikipedia's page on \"multicollinearity\" to read more.)  \n",
      "\n",
      ">You have two options:\n",
      "1. hand select a subset of the features, and see if the coefficients from the model make more sense.  Then decide if the accuracy of the previous model is worth the decrease in interpretability.\n",
      "2. experiment with increasing l2_penalty and l1_penalty arguments: this encourages the optimization to find estimates where some features are either close to 0 or dropped entirely.\n",
      "\n",
      "Yeah, actually.  I do have multiple columns that are kinda correlated.  All of the points columns are related, like the sums of rushing and passing TDs to the sum of total points.  Alright, throwing every single column of data I had at this didn't give me what I wanted.  Given Chris' explanation, the simplest thing to try is removing the columns that are duplicating the effect of points from the season_sums list of features and training the model again.  After removing features like TDR_SUM (rushing touchdown sum) and TDP_SUM (passing tds) and total points, I end up with coefficients that look like this."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# Why does TD_SUM end up in a negative coefficient?\n",
      "# Points are represented by multiple features, they're correlated and it's confusing the model\n",
      "season_sums.remove('PTS_SUM')\n",
      "season_sums.remove('TDP_SUM')\n",
      "season_sums.remove('TDR_SUM')\n",
      "lin_model = gl.linear_regression.create(season_train, target='WIN_SUM', features=season_sums)\n",
      "lin_pred = lin_model.predict(season_test)\n",
      "lin_result = lin_model.evaluate(season_test)\n",
      "lin_model.show()\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<pre>PROGRESS: Linear regression:</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Linear regression:"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of examples          : 357</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of examples          : 357"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of features          : 10</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of features          : 10"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of unpacked features : 10</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of unpacked features : 10"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of coefficients    : 11</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of coefficients    : 11"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Starting Newton Method</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Starting Newton Method"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: +-----------+----------+--------------+--------------------+----------------------+---------------+-----------------+</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: +-----------+----------+--------------+--------------------+----------------------+---------------+-----------------+"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: | Iteration | Passes   | Elapsed Time | Training-max_error | Validation-max_error | Training-rmse | Validation-rmse |</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: | Iteration | Passes   | Elapsed Time | Training-max_error | Validation-max_error | Training-rmse | Validation-rmse |"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: +-----------+----------+--------------+--------------------+----------------------+---------------+-----------------+</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: +-----------+----------+--------------+--------------------+----------------------+---------------+-----------------+"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: | 1         | 2        | 0.001000     | 5.141604           | 3.519644             | 1.722067      | 1.745545        |</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: | 1         | 2        | 0.001000     | 5.141604           | 3.519644             | 1.722067      | 1.745545        |"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: +-----------+----------+--------------+--------------------+----------------------+---------------+-----------------+</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: +-----------+----------+--------------+--------------------+----------------------+---------------+-----------------+"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "PROGRESS: Creating a validation set from 5 percent of training data. This may take a while.\n",
        "          You can set ``validation_set=None`` to disable validation tracking.\n",
        "\n"
       ]
      },
      {
       "javascript": [
        "$(\"head\").append($(\"<link/>\").attr({\n",
        "  rel:  \"stylesheet\",\n",
        "  type: \"text/css\",\n",
        "  href: \"//cdnjs.cloudflare.com/ajax/libs/font-awesome/4.1.0/css/font-awesome.min.css\"\n",
        "}));\n",
        "$(\"head\").append($(\"<link/>\").attr({\n",
        "  rel:  \"stylesheet\",\n",
        "  type: \"text/css\",\n",
        "  href: \"//turi.com/files/canvas/1.6.1/css/canvas.css\"\n",
        "}));\n",
        "\n",
        "            (function(){\n",
        "\n",
        "                var e = null;\n",
        "                if (typeof element == 'undefined') {\n",
        "                    var scripts = document.getElementsByTagName('script');\n",
        "                    var thisScriptTag = scripts[scripts.length-1];\n",
        "                    var parentDiv = thisScriptTag.parentNode;\n",
        "                    e = document.createElement('div');\n",
        "                    parentDiv.appendChild(e);\n",
        "                } else {\n",
        "                    e = element[0];\n",
        "                }\n",
        "\n",
        "                if (typeof requirejs !== 'undefined') {\n",
        "                    // disable load timeout; ipython_app.js is large and can take a while to load.\n",
        "                    requirejs.config({waitSeconds: 0});\n",
        "                }\n",
        "\n",
        "                require(['//turi.com/files/canvas/1.6.1/js/ipython_app.js'], function(IPythonApp){\n",
        "                    var app = new IPythonApp();\n",
        "                    app.attachView('model','Summary', {\"comparison\": null, \"selected_variable\": {\"comparison\": null, \"name\": [\"lin_model\"], \"view_file\": \"model\", \"view_component\": \"Summary\", \"view_params\": {\"model_type\": \"regression\", \"view\": \"Summary\"}, \"view_components\": [\"Summary\", \"Evaluation\", \"Comparison\"], \"model_type\": \"regression\", \"attributes\": {\"section_titles\": [\"Schema\", \"Hyperparameters\", \"Training Summary\", \"Settings\", \"Highest Positive Coefficients\", \"Lowest Negative Coefficients\"], \"sections\": [[[\"Number of coefficients\", 11], [\"Number of examples\", 357], [\"Number of feature columns\", 10], [\"Number of unpacked features\", 10]], [[\"L1 penalty\", 0.0], [\"L2 penalty\", 0.01]], [[\"Solver\", \"auto\"], [\"Solver iterations\", 1], [\"Solver status\", \"SUCCESS: Optimal solution found.\"], [\"Training time (sec)\", 0.002]], [[\"Residual sum of squares\", 1058.6893], [\"Training RMSE\", 1.7221]], [[\"TD_SUM\", 0.1518], [\"TOP_SUM\", 0.0225], [\"RA_SUM\", 0.016], [\"DP_SUM\", 0.0159]], [[\"(intercept)\", -3.1647], [\"INT_SUM\", -0.1534], [\"FUM_SUM\", -0.0761], [\"SK_SUM\", -0.031], [\"PA_SUM\", -0.0081]]]}, \"evaluations\": [[\"season_test\", {\"max_error\": 4.890134040943327, \"rmse\": 1.8691112976359405}]], \"type\": \"Model\"}, \"ipython\": true, \"view_params\": {\"model_type\": \"regression\", \"view\": \"Summary\"}, \"model_type\": \"regression\", \"attributes\": {\"section_titles\": [\"Schema\", \"Hyperparameters\", \"Training Summary\", \"Settings\", \"Highest Positive Coefficients\", \"Lowest Negative Coefficients\"], \"sections\": [[[\"Number of coefficients\", 11], [\"Number of examples\", 357], [\"Number of feature columns\", 10], [\"Number of unpacked features\", 10]], [[\"L1 penalty\", 0.0], [\"L2 penalty\", 0.01]], [[\"Solver\", \"auto\"], [\"Solver iterations\", 1], [\"Solver status\", \"SUCCESS: Optimal solution found.\"], [\"Training time (sec)\", 0.002]], [[\"Residual sum of squares\", 1058.6893], [\"Training RMSE\", 1.7221]], [[\"TD_SUM\", 0.1518], [\"TOP_SUM\", 0.0225], [\"RA_SUM\", 0.016], [\"DP_SUM\", 0.0159]], [[\"(intercept)\", -3.1647], [\"INT_SUM\", -0.1534], [\"FUM_SUM\", -0.0761], [\"SK_SUM\", -0.031], [\"PA_SUM\", -0.0081]]]}, \"evaluations\": [[\"season_test\", {\"max_error\": 4.890134040943327, \"rmse\": 1.8691112976359405}]]}, e);\n",
        "                });\n",
        "            })();\n",
        "        "
       ],
       "metadata": {},
       "output_type": "display_data"
      }
     ],
     "prompt_number": 31
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "That's more what I expected.  Touchdowns and time of possession lead the list of positive coefficients while interceptions lead the list of negative.  It doesn\u2019t look like my predictions changed all that much as a result, but the coefficients make more sense to me now."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print lin_result\n",
      "print season_test.head(5)['WIN_SUM']\n",
      "print lin_pred[0:5]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "{'max_error': 4.890134040943327, 'rmse': 1.8691112976359405}\n",
        "[4L, 11L, 5L, 10L, 4L]\n",
        "[5.356049945089315, 11.550561219697165, 4.353967601094646, 7.642765711124727, 6.205331611073706]\n"
       ]
      }
     ],
     "prompt_number": 32
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "##Predict who won a game\n",
      "Now, what if I want to predict who won a specific game?  Rather than trying to predict a sum, I want to predict whether one column, specifically the WIN column, contains a 1 (they won) or a 0 (they lost) based on a team's total stats for just that game.  This sounds fun.  But first, Chris advised that I do a little data reformatting.  It'll be easier to try and predict a win by looking at the difference between the numbers of the two teams playing each other.  I'm going to express this as Home-Away in columns named <stat>_DIFF.\n",
      "\n",
      "One side note here.  Whereas earlier I was trying to predict a continuous number, what I'm really trying to do now is predict what whether it's a WIN or a LOSS. In other words, the WIN column contains two categories; zero or one. It's not going to be a 0.7 or 1.3.  Using a formula similar to linear regression, this is commonly called logistic regression, to classify the value Y.  Rather than code up this math by hand, I'm going to use GLC's toolkit, getting right to the fun stuff."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# OK, so after talking to chris refactor data into one row per game\n",
      "# With values being Home - Away\n",
      "# add info about who was home and away\n",
      "more = team.join(games.select_columns(['GID', 'V', 'H', 'PTSV', 'PTSH']), 'GID')\n",
      "\n",
      "\n",
      "home = more[more['TNAME'] == more['H']].remove_columns(['PTSH', 'PTSV'])\n",
      "visit = more[more['TNAME'] == more['V']].remove_columns(['PTSH', 'PTSV'])\n",
      "difftable = home.join(visit, 'GID')\n",
      "#the visitor's info is now always under blank.1 column names\n",
      "\n",
      "for thing in ['PTS', 'RY', 'RA', 'PY', 'PA', 'SK', 'FUM', 'INT', 'TOP', 'TD', 'TDR', 'TDP', 'DP']:\n",
      "    difftable['%s_DIFF' % thing] = difftable[thing] - difftable['%s.1' % thing]\n",
      "\n",
      "diff_train, diff_test = difftable.random_split(0.8, seed=20)\n",
      "\n",
      "\n",
      "diff_lgc_model = gl.logistic_classifier.create(diff_train, target='WIN',\n",
      "                                               features=[ i for i in difftable.column_names() if \"_DIFF\" in i])\n",
      "\n",
      "\n",
      "diff_result = diff_lgc_model.evaluate(diff_test)\n",
      "print diff_result\n",
      "diff_lgc_model.show()\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<pre>PROGRESS: Logistic regression:</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Logistic regression:"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of examples          : 2923</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of examples          : 2923"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of classes           : 2</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of classes           : 2"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of feature columns   : 13</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of feature columns   : 13"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of unpacked features : 13</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of unpacked features : 13"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of coefficients    : 14</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of coefficients    : 14"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Starting Newton Method</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Starting Newton Method"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: +-----------+----------+--------------+-------------------+---------------------+</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: +-----------+----------+--------------+-------------------+---------------------+"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: | Iteration | Passes   | Elapsed Time | Training-accuracy | Validation-accuracy |</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: | Iteration | Passes   | Elapsed Time | Training-accuracy | Validation-accuracy |"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: +-----------+----------+--------------+-------------------+---------------------+</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: +-----------+----------+--------------+-------------------+---------------------+"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: | 1         | 2        | 0.012009     | 0.941156          | 0.954023            |</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: | 1         | 2        | 0.012009     | 0.941156          | 0.954023            |"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: | 2         | 3        | 0.033024     | 0.969894          | 0.977011            |</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: | 2         | 3        | 0.033024     | 0.969894          | 0.977011            |"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: | 3         | 4        | 0.043031     | 0.988368          | 0.988506            |</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: | 3         | 4        | 0.043031     | 0.988368          | 0.988506            |"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: | 4         | 5        | 0.054040     | 0.998632          | 0.994253            |</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: | 4         | 5        | 0.054040     | 0.998632          | 0.994253            |"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: | 5         | 6        | 0.065046     | 0.998974          | 1.000000            |</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: | 5         | 6        | 0.065046     | 0.998974          | 1.000000            |"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: | 6         | 7        | 0.075054     | 0.998632          | 1.000000            |</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: | 6         | 7        | 0.075054     | 0.998632          | 1.000000            |"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: +-----------+----------+--------------+-------------------+---------------------+</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: +-----------+----------+--------------+-------------------+---------------------+"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "PROGRESS: Creating a validation set from 5 percent of training data. This may take a while.\n",
        "          You can set ``validation_set=None`` to disable validation tracking.\n",
        "\n",
        "{'confusion_matrix': Columns:\n",
        "\ttarget_label\tint\n",
        "\tpredicted_label\tint\n",
        "\tcount\tint\n",
        "\n",
        "Rows: 2\n",
        "\n",
        "Data:\n",
        "+--------------+-----------------+-------+\n",
        "| target_label | predicted_label | count |\n",
        "+--------------+-----------------+-------+\n",
        "|      0       |        0        |  294  |\n",
        "|      1       |        1        |  433  |\n",
        "+--------------+-----------------+-------+\n",
        "[2 rows x 3 columns]\n",
        ", 'accuracy': 1.0}"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n"
       ]
      },
      {
       "javascript": [
        "$(\"head\").append($(\"<link/>\").attr({\n",
        "  rel:  \"stylesheet\",\n",
        "  type: \"text/css\",\n",
        "  href: \"//cdnjs.cloudflare.com/ajax/libs/font-awesome/4.1.0/css/font-awesome.min.css\"\n",
        "}));\n",
        "$(\"head\").append($(\"<link/>\").attr({\n",
        "  rel:  \"stylesheet\",\n",
        "  type: \"text/css\",\n",
        "  href: \"//turi.com/files/canvas/1.6.1/css/canvas.css\"\n",
        "}));\n",
        "\n",
        "            (function(){\n",
        "\n",
        "                var e = null;\n",
        "                if (typeof element == 'undefined') {\n",
        "                    var scripts = document.getElementsByTagName('script');\n",
        "                    var thisScriptTag = scripts[scripts.length-1];\n",
        "                    var parentDiv = thisScriptTag.parentNode;\n",
        "                    e = document.createElement('div');\n",
        "                    parentDiv.appendChild(e);\n",
        "                } else {\n",
        "                    e = element[0];\n",
        "                }\n",
        "\n",
        "                if (typeof requirejs !== 'undefined') {\n",
        "                    // disable load timeout; ipython_app.js is large and can take a while to load.\n",
        "                    requirejs.config({waitSeconds: 0});\n",
        "                }\n",
        "\n",
        "                require(['//turi.com/files/canvas/1.6.1/js/ipython_app.js'], function(IPythonApp){\n",
        "                    var app = new IPythonApp();\n",
        "                    app.attachView('model','Summary', {\"comparison\": null, \"selected_variable\": {\"comparison\": null, \"name\": [\"diff_lgc_model\"], \"view_file\": \"model\", \"view_component\": \"Summary\", \"view_params\": {\"model_type\": \"regression\", \"view\": \"Summary\"}, \"view_components\": [\"Summary\", \"Evaluation\", \"Comparison\"], \"model_type\": \"regression\", \"attributes\": {\"section_titles\": [\"Schema\", \"Hyperparameters\", \"Training Summary\", \"Settings\", \"Highest Positive Coefficients\", \"Lowest Negative Coefficients\"], \"sections\": [[[\"Number of coefficients\", 14], [\"Number of examples\", 2923], [\"Number of classes\", 2], [\"Number of feature columns\", 13], [\"Number of unpacked features\", 13]], [[\"L1 penalty\", 0.0], [\"L2 penalty\", 0.01]], [[\"Solver\", \"auto\"], [\"Solver iterations\", 10], [\"Solver status\", \"TERMINATED: Iteration limit reached.\"], [\"Training time (sec)\", 0.1291]], [[\"Log-likelihood\", 12.2532]], [[\"PTS_DIFF\", 2.4017], [\"TDR_DIFF\", 0.473], [\"TDP_DIFF\", 0.4619], [\"TD_DIFF\", 0.1943], [\"DP_DIFF\", 0.1096]], [[\"(intercept)\", -0.1019], [\"RA_DIFF\", -0.0452], [\"INT_DIFF\", -0.0204], [\"SK_DIFF\", -0.0181], [\"PA_DIFF\", -0.0055]]]}, \"evaluations\": [[\"diff_test\", {\"confusion_matrix\": [{\"count\": 294, \"target_label\": 0, \"predicted_label\": 0}, {\"count\": 433, \"target_label\": 1, \"predicted_label\": 1}], \"accuracy\": 1.0}]], \"type\": \"Model\"}, \"ipython\": true, \"view_params\": {\"model_type\": \"regression\", \"view\": \"Summary\"}, \"model_type\": \"regression\", \"attributes\": {\"section_titles\": [\"Schema\", \"Hyperparameters\", \"Training Summary\", \"Settings\", \"Highest Positive Coefficients\", \"Lowest Negative Coefficients\"], \"sections\": [[[\"Number of coefficients\", 14], [\"Number of examples\", 2923], [\"Number of classes\", 2], [\"Number of feature columns\", 13], [\"Number of unpacked features\", 13]], [[\"L1 penalty\", 0.0], [\"L2 penalty\", 0.01]], [[\"Solver\", \"auto\"], [\"Solver iterations\", 10], [\"Solver status\", \"TERMINATED: Iteration limit reached.\"], [\"Training time (sec)\", 0.1291]], [[\"Log-likelihood\", 12.2532]], [[\"PTS_DIFF\", 2.4017], [\"TDR_DIFF\", 0.473], [\"TDP_DIFF\", 0.4619], [\"TD_DIFF\", 0.1943], [\"DP_DIFF\", 0.1096]], [[\"(intercept)\", -0.1019], [\"RA_DIFF\", -0.0452], [\"INT_DIFF\", -0.0204], [\"SK_DIFF\", -0.0181], [\"PA_DIFF\", -0.0055]]]}, \"evaluations\": [[\"diff_test\", {\"confusion_matrix\": [{\"count\": 294, \"target_label\": 0, \"predicted_label\": 0}, {\"count\": 433, \"target_label\": 1, \"predicted_label\": 1}], \"accuracy\": 1.0}]]}, e);\n",
        "                });\n",
        "            })();\n",
        "        "
       ],
       "metadata": {},
       "output_type": "display_data"
      }
     ],
     "prompt_number": 43
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Me:  HOLY GUACAMOLE!!  LOOK AT MY ACCURACY! IT'S ALMOST PERFECT!\n",
      "\n",
      "Chris: That can't be right.\n",
      "\n",
      "Me: I CAN PREDICT THE FUTURE!\n",
      "\n",
      "Chris: I don't really think that's how it w-\n",
      "\n",
      "Me: I'M QUITTING MY JOB AND GOING TO VEGAS! I AM AMAZING!!!  SEE YOU CLOWNS LATER AAAAAAHAHAHAHAHA!!!!!!!\n",
      "\n",
      "Chris:  Did you leave the difference in scores as a feature for the model to use?\n",
      "\n",
      "Me: AAAHAHAHA-... what?  Yes?  ... Yes, I did.\n",
      "\n",
      "Chris: So the model learned that with 100% accuracy that the final score predicts who won.\n",
      "\n",
      "Me: Ahem.  Oh.\n",
      "Me: ... sorry about the clowns thing.\n",
      "\n",
      "Chris: ...\n",
      "\n",
      "Me: ... I'm also un-quitting.\n",
      "\n",
      "Chris: ...\n",
      "\n",
      "Me: ... I'll go back to my desk now.\n",
      "\n",
      "\n",
      "\n",
      "Yeah.  Once again putting all the info I had into this black box wasn't quite the best idea.  I want to predict who won based on every other stat but I don't want to use the score.  The model picks up pretty quickly that some of my features like, oh, say, the point differential between teams, is a pretty darn good indicator of what is in the WIN column.  Clever.  Let\u2019s remove that stuff.  \n"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# CLEARLY I CAN PREDICT THE FUTURE\n",
      "# oh.  remove points\n",
      "\n",
      "better_features= [ i for i in difftable.column_names() if \"_DIFF\" in i]\n",
      "better_features.remove('PTS_DIFF')\n",
      "better_features.remove('TD_DIFF')\n",
      "better_features.remove('TDR_DIFF')\n",
      "better_features.remove('TDP_DIFF')\n",
      "better_features.remove('DP_DIFF')\n",
      "\n",
      "better_model = gl.logistic_classifier.create(diff_train, target='WIN', features=better_features)\n",
      "better_model.show()\n",
      "print better_model.evaluate(diff_test)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<pre>PROGRESS: Logistic regression:</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Logistic regression:"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of examples          : 2935</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of examples          : 2935"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of classes           : 2</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of classes           : 2"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of feature columns   : 8</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of feature columns   : 8"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of unpacked features : 8</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of unpacked features : 8"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of coefficients    : 9</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of coefficients    : 9"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Starting Newton Method</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Starting Newton Method"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: +-----------+----------+--------------+-------------------+---------------------+</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: +-----------+----------+--------------+-------------------+---------------------+"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: | Iteration | Passes   | Elapsed Time | Training-accuracy | Validation-accuracy |</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: | Iteration | Passes   | Elapsed Time | Training-accuracy | Validation-accuracy |"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: +-----------+----------+--------------+-------------------+---------------------+</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: +-----------+----------+--------------+-------------------+---------------------+"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: | 1         | 2        | 0.009008     | 0.884497          | 0.885350            |</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: | 1         | 2        | 0.009008     | 0.884497          | 0.885350            |"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: | 2         | 3        | 0.017012     | 0.885179          | 0.878981            |</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: | 2         | 3        | 0.017012     | 0.885179          | 0.878981            |"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: | 3         | 4        | 0.023016     | 0.883475          | 0.885350            |</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: | 3         | 4        | 0.023016     | 0.883475          | 0.885350            |"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: | 4         | 5        | 0.031022     | 0.883816          | 0.885350            |</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: | 4         | 5        | 0.031022     | 0.883816          | 0.885350            |"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: | 5         | 6        | 0.040029     | 0.883816          | 0.885350            |</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: | 5         | 6        | 0.040029     | 0.883816          | 0.885350            |"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: | 6         | 7        | 0.047033     | 0.883816          | 0.885350            |</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: | 6         | 7        | 0.047033     | 0.883816          | 0.885350            |"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: +-----------+----------+--------------+-------------------+---------------------+</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: +-----------+----------+--------------+-------------------+---------------------+"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "PROGRESS: Creating a validation set from 5 percent of training data. This may take a while.\n",
        "          You can set ``validation_set=None`` to disable validation tracking.\n",
        "\n"
       ]
      },
      {
       "javascript": [
        "$(\"head\").append($(\"<link/>\").attr({\n",
        "  rel:  \"stylesheet\",\n",
        "  type: \"text/css\",\n",
        "  href: \"//cdnjs.cloudflare.com/ajax/libs/font-awesome/4.1.0/css/font-awesome.min.css\"\n",
        "}));\n",
        "$(\"head\").append($(\"<link/>\").attr({\n",
        "  rel:  \"stylesheet\",\n",
        "  type: \"text/css\",\n",
        "  href: \"//turi.com/files/canvas/1.6.1/css/canvas.css\"\n",
        "}));\n",
        "\n",
        "            (function(){\n",
        "\n",
        "                var e = null;\n",
        "                if (typeof element == 'undefined') {\n",
        "                    var scripts = document.getElementsByTagName('script');\n",
        "                    var thisScriptTag = scripts[scripts.length-1];\n",
        "                    var parentDiv = thisScriptTag.parentNode;\n",
        "                    e = document.createElement('div');\n",
        "                    parentDiv.appendChild(e);\n",
        "                } else {\n",
        "                    e = element[0];\n",
        "                }\n",
        "\n",
        "                if (typeof requirejs !== 'undefined') {\n",
        "                    // disable load timeout; ipython_app.js is large and can take a while to load.\n",
        "                    requirejs.config({waitSeconds: 0});\n",
        "                }\n",
        "\n",
        "                require(['//turi.com/files/canvas/1.6.1/js/ipython_app.js'], function(IPythonApp){\n",
        "                    var app = new IPythonApp();\n",
        "                    app.attachView('model','Summary', {\"comparison\": null, \"selected_variable\": {\"comparison\": null, \"name\": [\"better_model\"], \"view_file\": \"model\", \"view_component\": \"Summary\", \"view_params\": {\"model_type\": \"regression\", \"view\": \"Summary\"}, \"view_components\": [\"Summary\", \"Evaluation\", \"Comparison\"], \"model_type\": \"regression\", \"attributes\": {\"section_titles\": [\"Schema\", \"Hyperparameters\", \"Training Summary\", \"Settings\", \"Highest Positive Coefficients\", \"Lowest Negative Coefficients\"], \"sections\": [[[\"Number of coefficients\", 9], [\"Number of examples\", 2935], [\"Number of classes\", 2], [\"Number of feature columns\", 8], [\"Number of unpacked features\", 8]], [[\"L1 penalty\", 0.0], [\"L2 penalty\", 0.01]], [[\"Solver\", \"auto\"], [\"Solver iterations\", 6], [\"Solver status\", \"SUCCESS: Optimal solution found.\"], [\"Training time (sec)\", 0.051]], [[\"Log-likelihood\", 827.6028]], [[\"(intercept)\", 0.3714], [\"TOP_DIFF\", 0.0679], [\"PY_DIFF\", 0.0174], [\"RY_DIFF\", 0.0094]], [[\"INT_DIFF\", -0.7502], [\"FUM_DIFF\", -0.7219], [\"SK_DIFF\", -0.2456], [\"PA_DIFF\", -0.1451], [\"RA_DIFF\", -0.0075]]]}, \"evaluations\": [], \"type\": \"Model\"}, \"ipython\": true, \"view_params\": {\"model_type\": \"regression\", \"view\": \"Summary\"}, \"model_type\": \"regression\", \"attributes\": {\"section_titles\": [\"Schema\", \"Hyperparameters\", \"Training Summary\", \"Settings\", \"Highest Positive Coefficients\", \"Lowest Negative Coefficients\"], \"sections\": [[[\"Number of coefficients\", 9], [\"Number of examples\", 2935], [\"Number of classes\", 2], [\"Number of feature columns\", 8], [\"Number of unpacked features\", 8]], [[\"L1 penalty\", 0.0], [\"L2 penalty\", 0.01]], [[\"Solver\", \"auto\"], [\"Solver iterations\", 6], [\"Solver status\", \"SUCCESS: Optimal solution found.\"], [\"Training time (sec)\", 0.051]], [[\"Log-likelihood\", 827.6028]], [[\"(intercept)\", 0.3714], [\"TOP_DIFF\", 0.0679], [\"PY_DIFF\", 0.0174], [\"RY_DIFF\", 0.0094]], [[\"INT_DIFF\", -0.7502], [\"FUM_DIFF\", -0.7219], [\"SK_DIFF\", -0.2456], [\"PA_DIFF\", -0.1451], [\"RA_DIFF\", -0.0075]]]}, \"evaluations\": []}, e);\n",
        "                });\n",
        "            })();\n",
        "        "
       ],
       "metadata": {},
       "output_type": "display_data"
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "{'confusion_matrix': Columns:\n",
        "\ttarget_label\tint\n",
        "\tpredicted_label\tint\n",
        "\tcount\tint\n",
        "\n",
        "Rows: 4\n",
        "\n",
        "Data:\n",
        "+--------------+-----------------+-------+\n",
        "| target_label | predicted_label | count |\n",
        "+--------------+-----------------+-------+\n",
        "|      0       |        1        |   60  |\n",
        "|      1       |        1        |  359  |\n",
        "|      1       |        0        |   43  |\n",
        "|      0       |        0        |  270  |\n",
        "+--------------+-----------------+-------+\n",
        "[4 rows x 3 columns]\n",
        ", 'accuracy': 0.8592896174863388}\n"
       ]
      }
     ],
     "prompt_number": 40
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      " That's not bad.  The confusion matrix here is an awesome little guide to how the model did guessing the outcome of games.\n",
      " \n",
      " ##(N)ewb (F)eature exp(L)oration\n",
      "While I\u2019m in the business of classifying things, I browsed the list of classifiers in the API searching for other high-powered tools to naively explore.  Chris pointed me to the boosted trees classifier as an interesting way to explore the effects of each of my features on the classification.  It\u2019s not entirely clear to me how a tree, regardless of whether I boosted it or purchased it legally, is going to help me out but it looks easy enough to try."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "#Exploring the trees\n",
      "btc_classifier = gl.boosted_trees_classifier.create(diff_train, target='WIN', features=better_features)\n",
      "btc_classifier.show(view=\"Tree\", tree_id=1)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<pre>PROGRESS: Boosted trees classifier:</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Boosted trees classifier:"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of examples          : 2924</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of examples          : 2924"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of classes           : 2</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of classes           : 2"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of feature columns   : 8</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of feature columns   : 8"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of unpacked features : 8</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of unpacked features : 8"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Starting Boosted Trees</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Starting Boosted Trees"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:   Iter      Accuracy          Elapsed time</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:   Iter      Accuracy          Elapsed time"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:         (training) (validation)</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:         (training) (validation)"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      0  8.683e-001  7.977e-001        0.00s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      0  8.683e-001  7.977e-001        0.00s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      1  8.830e-001  8.497e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      1  8.830e-001  8.497e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      2  8.916e-001  8.786e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      2  8.916e-001  8.786e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      3  8.984e-001  8.671e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      3  8.984e-001  8.671e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      4  9.053e-001  8.728e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      4  9.053e-001  8.728e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      5  9.124e-001  8.728e-001        0.03s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      5  9.124e-001  8.728e-001        0.03s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      6  9.142e-001  8.728e-001        0.03s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      6  9.142e-001  8.728e-001        0.03s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      7  9.193e-001  8.786e-001        0.04s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      7  9.193e-001  8.786e-001        0.04s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      8  9.254e-001  8.786e-001        0.04s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      8  9.254e-001  8.786e-001        0.04s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      9  9.302e-001  8.786e-001        0.05s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      9  9.302e-001  8.786e-001        0.05s"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "PROGRESS: Creating a validation set from 5 percent of training data. This may take a while.\n",
        "          You can set ``validation_set=None`` to disable validation tracking.\n",
        "\n"
       ]
      },
      {
       "javascript": [
        "$(\"head\").append($(\"<link/>\").attr({\n",
        "  rel:  \"stylesheet\",\n",
        "  type: \"text/css\",\n",
        "  href: \"//cdnjs.cloudflare.com/ajax/libs/font-awesome/4.1.0/css/font-awesome.min.css\"\n",
        "}));\n",
        "$(\"head\").append($(\"<link/>\").attr({\n",
        "  rel:  \"stylesheet\",\n",
        "  type: \"text/css\",\n",
        "  href: \"//turi.com/files/canvas/1.6.1/css/canvas.css\"\n",
        "}));\n",
        "\n",
        "            (function(){\n",
        "\n",
        "                var e = null;\n",
        "                if (typeof element == 'undefined') {\n",
        "                    var scripts = document.getElementsByTagName('script');\n",
        "                    var thisScriptTag = scripts[scripts.length-1];\n",
        "                    var parentDiv = thisScriptTag.parentNode;\n",
        "                    e = document.createElement('div');\n",
        "                    parentDiv.appendChild(e);\n",
        "                } else {\n",
        "                    e = element[0];\n",
        "                }\n",
        "\n",
        "                if (typeof requirejs !== 'undefined') {\n",
        "                    // disable load timeout; ipython_app.js is large and can take a while to load.\n",
        "                    requirejs.config({waitSeconds: 0});\n",
        "                }\n",
        "\n",
        "                require(['//turi.com/files/canvas/1.6.1/js/ipython_app.js'], function(IPythonApp){\n",
        "                    var app = new IPythonApp();\n",
        "                    app.attachView('sgraph','View', {\"edges_labels\": [\"yes\", \"no\", \"yes\", \"yes\", \"no\", \"yes\", \"no\", \"no\", \"yes\", \"yes\", \"no\", \"no\", \"no\", \"no\", \"yes\", \"no\", \"yes\", \"yes\", \"yes\", \"no\", \"yes\", \"no\", \"no\", \"yes\", \"yes\", \"yes\", \"no\", \"yes\", \"no\", \"yes\", \"no\", \"no\", \"yes\", \"yes\", \"no\", \"no\", \"no\", \"yes\", \"no\", \"yes\", \"no\", \"no\", \"yes\", \"yes\", \"no\", \"yes\", \"no\", \"yes\", \"yes\", \"yes\", \"yes\", \"no\", \"yes\", \"yes\", \"no\", \"no\", \"no\", \"no\", \"yes\", \"no\", \"no\", \"yes\", \"no\", \"yes\", \"no\", \"no\", \"yes\", \"no\", \"yes\", \"yes\", \"yes\", \"no\", \"yes\", \"no\", \"no\", \"no\", \"no\", \"yes\", \"yes\", \"yes\", \"yes\", \"no\", \"no\", \"no\", \"yes\", \"no\", \"yes\", \"yes\", \"no\", \"yes\", \"yes\", \"no\", \"yes\", \"no\", \"no\", \"no\", \"no\", \"yes\", \"yes\", \"yes\", \"yes\", \"yes\", \"no\", \"yes\", \"no\", \"yes\", \"yes\", \"no\", \"no\", \"no\"], \"selected_variable\": {\"name\": [\"<SGraph>\"], \"view_file\": \"sgraph\", \"view_component\": \"View\", \"view_params\": {\"elabel_hover\": false, \"vertex_positions\": null, \"h_offset\": 0.0, \"node_size\": 300, \"ecolor\": [0.37, 0.33, 0.33], \"elabel\": \"value\", \"arrows\": true, \"ewidth\": 1, \"vlabel\": \"__repr__\", \"highlight_color\": [0.69, 0.0, 0.498], \"vcolor\": [0.522, 0.741, 0.0], \"vlabel_hover\": false, \"highlight\": {\"0\": [0.69, 0.0, 0.48], \"27\": [1.0, 0.33, 0.0], \"36\": [1.0, 0.33, 0.0], \"42\": [0.039, 0.55, 0.77], \"46\": [0.039, 0.55, 0.77], \"56\": [0.039, 0.55, 0.77], \"59\": [0.039, 0.55, 0.77], \"61\": [0.039, 0.55, 0.77], \"62\": [1.0, 0.33, 0.0], \"63\": [1.0, 0.33, 0.0], \"64\": [1.0, 0.33, 0.0], \"65\": [1.0, 0.33, 0.0], \"66\": [0.039, 0.55, 0.77], \"67\": [0.039, 0.55, 0.77], \"68\": [1.0, 0.33, 0.0], \"69\": [1.0, 0.33, 0.0], \"70\": [0.039, 0.55, 0.77], \"71\": [0.039, 0.55, 0.77], \"72\": [1.0, 0.33, 0.0], \"73\": [0.039, 0.55, 0.77], \"74\": [0.039, 0.55, 0.77], \"75\": [1.0, 0.33, 0.0], \"76\": [0.039, 0.55, 0.77], \"77\": [0.039, 0.55, 0.77], \"78\": [1.0, 0.33, 0.0], \"79\": [0.039, 0.55, 0.77], \"80\": [1.0, 0.33, 0.0], \"81\": [1.0, 0.33, 0.0], \"82\": [0.039, 0.55, 0.77], \"83\": [0.039, 0.55, 0.77], \"84\": [0.039, 0.55, 0.77], \"85\": [1.0, 0.33, 0.0], \"86\": [0.039, 0.55, 0.77], \"87\": [1.0, 0.33, 0.0], \"88\": [1.0, 0.33, 0.0], \"89\": [1.0, 0.33, 0.0], \"90\": [0.039, 0.55, 0.77], \"91\": [0.039, 0.55, 0.77], \"92\": [1.0, 0.33, 0.0], \"93\": [0.039, 0.55, 0.77], \"94\": [1.0, 0.33, 0.0], \"95\": [1.0, 0.33, 0.0], \"96\": [1.0, 0.33, 0.0], \"97\": [1.0, 0.33, 0.0], \"98\": [1.0, 0.33, 0.0], \"99\": [0.039, 0.55, 0.77], \"100\": [1.0, 0.33, 0.0], \"101\": [0.039, 0.55, 0.77], \"102\": [1.0, 0.33, 0.0], \"103\": [0.039, 0.55, 0.77], \"104\": [1.0, 0.33, 0.0], \"105\": [1.0, 0.33, 0.0], \"106\": [0.039, 0.55, 0.77], \"107\": [0.039, 0.55, 0.77], \"108\": [1.0, 0.33, 0.0], \"109\": [1.0, 0.33, 0.0], \"110\": [0.039, 0.55, 0.77]}, \"v_offset\": 0.03}, \"view_components\": [\"View\"], \"type\": \"SGraph\", \"descriptives_links\": {\"edges\": \"edges\", \"vertices\": \"vertices\"}, \"descriptives\": {\"edges\": 110, \"vertices\": 111}}, \"positions\": null, \"error_type\": 0, \"vertices\": [10, 73, 110, 96, 7, 29, 87, 92, 33, 47, 68, 8, 27, 70, 5, 52, 50, 21, 76, 107, 109, 58, 63, 67, 34, 16, 30, 65, 36, 23, 89, 49, 56, 43, 78, 40, 83, 0, 15, 62, 106, 18, 103, 37, 105, 61, 38, 75, 108, 35, 31, 88, 26, 86, 59, 46, 48, 2, 53, 13, 32, 71, 9, 51, 28, 69, 91, 93, 72, 97, 74, 6, 80, 3, 54, 45, 98, 100, 85, 25, 94, 12, 14, 82, 81, 42, 64, 60, 20, 79, 102, 39, 19, 1, 104, 101, 41, 77, 66, 17, 90, 11, 84, 44, 24, 95, 4, 99, 22, 55, 57], \"vertices_labels\": [\"RY_DIFF<39.0\", \"-0.351716\", \"-0.339242\", \"0.423268\", \"PA_DIFF<3.0\", \"INT_DIFF<3.0\", \"0.332328\", \"0.445376\", \"FUM_DIFF<1.0\", \"INT_DIFF<0.0\", \"0.157093\", \"FUM_DIFF<0.0\", \"0.132223\", \"-0.243078\", \"RA_DIFF<7.0\", \"PY_DIFF<91.0\", \"PY_DIFF<72.0\", \"RA_DIFF<-4.0\", \"-0.354642\", \"-0.322647\", \"0.078147\", \"RA_DIFF<16.0\", \"0.406855\", \"-0.322899\", \"PY_DIFF<98.0\", \"SK_DIFF<-1.0\", \"PY_DIFF<16.0\", \"0.319956\", \"0.461148\", \"PA_DIFF<-9.0\", \"0.067576\", \"RY_DIFF<-64.0\", \"-0.416429\", \"FUM_DIFF<-1.0\", \"0.212231\", \"RY_DIFF<-146.0\", \"-0.046159\", \"RA_DIFF<0.0\", \"PY_DIFF<-58.0\", \"0.346232\", \"-0.136474\", \"INT_DIFF<-1.0\", \"-0.094371\", \"TOP_DIFF<-14.0\", \"0.351638\", \"-0.045591\", \"PY_DIFF<-43.0\", \"0.041327\", \"0.151732\", \"PY_DIFF<-43.0\", \"RA_DIFF<-5.0\", \"0.055501\", \"INT_DIFF<1.0\", \"-0.077946\", \"-0.379589\", \"-0.339151\", \"FUM_DIFF<0.0\", \"INT_DIFF<2.0\", \"RY_DIFF<28.0\", \"RY_DIFF<-104.0\", \"FUM_DIFF<1.0\", \"-0.192886\", \"RY_DIFF<-56.0\", \"PY_DIFF<-136.0\", \"FUM_DIFF<-1.0\", \"0.072061\", \"-0.142469\", \"-0.019862\", \"0.425692\", \"0.013833\", \"-0.152694\", \"RY_DIFF<42.0\", \"0.407673\", \"SK_DIFF<0.0\", \"PY_DIFF<6.0\", \"INT_DIFF<2.0\", \"0.436409\", \"0.259858\", \"0.271457\", \"PA_DIFF<5.0\", \"0.320991\", \"SK_DIFF<3.0\", \"PA_DIFF<3.0\", \"-0.431976\", \"0.085714\", \"-0.12\", \"0.100573\", \"RA_DIFF<16.0\", \"PA_DIFF<26.0\", \"-0.09926\", \"0.170893\", \"PA_DIFF<-5.0\", \"PY_DIFF<133.0\", \"INT_DIFF<1.0\", \"0.104838\", \"-0.270032\", \"PY_DIFF<-76.0\", \"-0.096347\", \"-0.074114\", \"RA_DIFF<-9.0\", \"-0.2523\", \"PY_DIFF<-28.0\", \"-0.345711\", \"PA_DIFF<-2.0\", \"PA_DIFF<0.0\", \"0.215599\", \"SK_DIFF<-1.0\", \"-0.094845\", \"SK_DIFF<2.0\", \"RY_DIFF<7.0\", \"RY_DIFF<238.0\"], \"edges\": [[47, 87], [7, 16], [33, 65], [10, 21], [29, 58], [7, 15], [8, 18], [47, 88], [52, 97], [50, 93], [52, 98], [5, 12], [50, 94], [33, 66], [8, 17], [10, 22], [29, 57], [5, 11], [16, 33], [34, 68], [23, 47], [49, 92], [16, 34], [34, 67], [58, 107], [21, 43], [58, 108], [30, 59], [23, 48], [49, 91], [30, 60], [21, 44], [38, 73], [18, 37], [18, 38], [40, 78], [0, 2], [15, 31], [15, 32], [37, 71], [37, 72], [38, 74], [0, 1], [43, 81], [43, 82], [40, 77], [35, 70], [13, 27], [2, 5], [48, 89], [31, 61], [31, 62], [26, 53], [35, 69], [13, 28], [2, 6], [26, 54], [53, 100], [53, 99], [48, 90], [51, 96], [32, 63], [28, 56], [6, 13], [6, 14], [32, 64], [9, 19], [9, 20], [28, 55], [51, 95], [3, 7], [3, 8], [14, 29], [25, 52], [14, 30], [45, 86], [12, 26], [25, 51], [45, 85], [12, 25], [54, 101], [54, 102], [60, 110], [39, 76], [60, 109], [19, 40], [39, 75], [1, 3], [41, 80], [19, 39], [20, 41], [20, 42], [41, 79], [1, 4], [4, 10], [24, 50], [17, 36], [11, 23], [24, 49], [44, 83], [55, 103], [57, 105], [57, 106], [17, 35], [22, 46], [4, 9], [22, 45], [55, 104], [44, 84], [11, 24]], \"ipython\": true, \"error_msg\": \"\"}, e);\n",
        "                });\n",
        "            })();\n",
        "        "
       ],
       "metadata": {},
       "output_type": "display_data"
      }
     ],
     "prompt_number": 45
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Um, what?  I have no idea what I just did and this visualization isn\u2019t helpful at all.  I\u2019ve created what looks like an insane decision shrub of a tree but I don\u2019t think that every single tiny decision and combination of features is really this important.  Let me try that again, this time using just one decision tree and limiting its depth."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "btc_class2 = gl.boosted_trees_classifier.create(diff_train, target='WIN', features=better_features, max_iterations=1,\n",
      "                                                    max_depth=3)\n",
      "btc_class2.show(view=\"Tree\", tree_id=0)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<pre>PROGRESS: Boosted trees classifier:</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Boosted trees classifier:"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of examples          : 2962</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of examples          : 2962"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of classes           : 2</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of classes           : 2"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of feature columns   : 8</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of feature columns   : 8"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of unpacked features : 8</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of unpacked features : 8"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Starting Boosted Trees</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Starting Boosted Trees"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:   Iter      Accuracy          Elapsed time</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:   Iter      Accuracy          Elapsed time"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:         (training) (validation)</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:         (training) (validation)"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      0  8.126e-001  7.852e-001        0.00s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      0  8.126e-001  7.852e-001        0.00s"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "PROGRESS: Creating a validation set from 5 percent of training data. This may take a while.\n",
        "          You can set ``validation_set=None`` to disable validation tracking.\n",
        "\n"
       ]
      },
      {
       "javascript": [
        "$(\"head\").append($(\"<link/>\").attr({\n",
        "  rel:  \"stylesheet\",\n",
        "  type: \"text/css\",\n",
        "  href: \"//cdnjs.cloudflare.com/ajax/libs/font-awesome/4.1.0/css/font-awesome.min.css\"\n",
        "}));\n",
        "$(\"head\").append($(\"<link/>\").attr({\n",
        "  rel:  \"stylesheet\",\n",
        "  type: \"text/css\",\n",
        "  href: \"//turi.com/files/canvas/1.6.1/css/canvas.css\"\n",
        "}));\n",
        "\n",
        "            (function(){\n",
        "\n",
        "                var e = null;\n",
        "                if (typeof element == 'undefined') {\n",
        "                    var scripts = document.getElementsByTagName('script');\n",
        "                    var thisScriptTag = scripts[scripts.length-1];\n",
        "                    var parentDiv = thisScriptTag.parentNode;\n",
        "                    e = document.createElement('div');\n",
        "                    parentDiv.appendChild(e);\n",
        "                } else {\n",
        "                    e = element[0];\n",
        "                }\n",
        "\n",
        "                if (typeof requirejs !== 'undefined') {\n",
        "                    // disable load timeout; ipython_app.js is large and can take a while to load.\n",
        "                    requirejs.config({waitSeconds: 0});\n",
        "                }\n",
        "\n",
        "                require(['//turi.com/files/canvas/1.6.1/js/ipython_app.js'], function(IPythonApp){\n",
        "                    var app = new IPythonApp();\n",
        "                    app.attachView('sgraph','View', {\"edges_labels\": [\"no\", \"yes\", \"no\", \"yes\", \"yes\", \"no\", \"yes\", \"no\", \"yes\", \"no\", \"yes\", \"no\", \"no\", \"yes\"], \"selected_variable\": {\"name\": [\"<SGraph>\"], \"view_file\": \"sgraph\", \"view_component\": \"View\", \"view_params\": {\"elabel_hover\": false, \"vertex_positions\": null, \"h_offset\": 0.0, \"node_size\": 300, \"ecolor\": [0.37, 0.33, 0.33], \"elabel\": \"value\", \"arrows\": true, \"ewidth\": 1, \"vlabel\": \"__repr__\", \"highlight_color\": [0.69, 0.0, 0.498], \"vcolor\": [0.522, 0.741, 0.0], \"vlabel_hover\": false, \"highlight\": {\"0\": [0.69, 0.0, 0.48], \"7\": [1.0, 0.33, 0.0], \"8\": [0.039, 0.55, 0.77], \"9\": [0.039, 0.55, 0.77], \"10\": [0.039, 0.55, 0.77], \"11\": [1.0, 0.33, 0.0], \"12\": [1.0, 0.33, 0.0], \"13\": [1.0, 0.33, 0.0], \"14\": [0.039, 0.55, 0.77]}, \"v_offset\": 0.03}, \"view_components\": [\"View\"], \"type\": \"SGraph\", \"descriptives_links\": {\"edges\": \"edges\", \"vertices\": \"vertices\"}, \"descriptives\": {\"edges\": 14, \"vertices\": 15}}, \"positions\": null, \"error_type\": 0, \"vertices\": [5, 10, 8, 7, 0, 13, 2, 6, 9, 14, 12, 3, 1, 11, 4], \"vertices_labels\": [\"RA_DIFF<7.0\", \"-0.457554\", \"-0.121387\", \"0.279389\", \"RA_DIFF<0.0\", \"0.186722\", \"INT_DIFF<1.0\", \"SK_DIFF<1.0\", \"-0.073381\", \"-0.225\", \"0.503182\", \"PA_DIFF<-3.0\", \"INT_DIFF<0.0\", \"0.313274\", \"SK_DIFF<0.0\"], \"edges\": [[5, 12], [5, 11], [0, 2], [0, 1], [2, 5], [2, 6], [6, 13], [6, 14], [3, 7], [3, 8], [1, 3], [1, 4], [4, 10], [4, 9]], \"ipython\": true, \"error_msg\": \"\"}, e);\n",
        "                });\n",
        "            })();\n",
        "        "
       ],
       "metadata": {},
       "output_type": "display_data"
      }
     ],
     "prompt_number": 46
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Ok, limiting the depth did wonders for my readability here.  This is interesting, the tree isn\u2019t making decisions around the things I thought are important.  The big first branch happens on rushing attempts.   I can view what the model thinks is important with a call to model.get_feature_importance().  This gives me a sum of the nodes in the tree that are branching on each feature."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "btc_class2.get_feature_importance()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\"><table frame=\"box\" rules=\"cols\">\n",
        "    <tr>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">feature</th>\n",
        "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">count</th>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">INT_DIFF</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">RA_DIFF</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">SK_DIFF</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">PA_DIFF</td>\n",
        "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
        "    </tr>\n",
        "</table>\n",
        "[4 rows x 2 columns]<br/>\n",
        "</div>"
       ],
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 47,
       "text": [
        "Columns:\n",
        "\tfeature\tstr\n",
        "\tcount\tint\n",
        "\n",
        "Rows: 4\n",
        "\n",
        "Data:\n",
        "+----------+-------+\n",
        "| feature  | count |\n",
        "+----------+-------+\n",
        "| INT_DIFF |   2   |\n",
        "| RA_DIFF  |   2   |\n",
        "| SK_DIFF  |   2   |\n",
        "| PA_DIFF  |   1   |\n",
        "+----------+-------+\n",
        "[4 rows x 2 columns]"
       ]
      }
     ],
     "prompt_number": 47
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Huh.  Not what I thought.  The number of rushing attempts is more important than interceptions and more important than rushing yards?  While I could maybe see that, it doesn\u2019t line up with the features that other folks on the internet have concluded are important.  I\u2019d expect to see something about offense, maybe passing, or total rush yards.  Maybe if I make the tree deeper it\u2019d be a little closer to what I\u2019ve read is important in a football game?"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "btc_bigger = gl.boosted_trees_classifier.create(diff_train, target='WIN', \n",
      "features=better_features, max_iterations=50, max_depth=5)\n",
      "btc_bigger.show(view=\"Tree\", tree_id=0) "
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<pre>PROGRESS: Boosted trees classifier:</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Boosted trees classifier:"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of examples          : 2939</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of examples          : 2939"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of classes           : 2</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of classes           : 2"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of feature columns   : 8</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of feature columns   : 8"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of unpacked features : 8</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of unpacked features : 8"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Starting Boosted Trees</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Starting Boosted Trees"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:   Iter      Accuracy          Elapsed time</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:   Iter      Accuracy          Elapsed time"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:         (training) (validation)</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:         (training) (validation)"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      0  8.425e-001  7.848e-001        0.00s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      0  8.425e-001  7.848e-001        0.00s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      1  8.636e-001  8.418e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      1  8.636e-001  8.418e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      2  8.738e-001  8.608e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      2  8.738e-001  8.608e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      3  8.799e-001  8.671e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      3  8.799e-001  8.671e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      4  8.901e-001  8.608e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      4  8.901e-001  8.608e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      5  8.925e-001  8.861e-001        0.03s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      5  8.925e-001  8.861e-001        0.03s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      6  8.962e-001  8.987e-001        0.03s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      6  8.962e-001  8.987e-001        0.03s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      7  8.989e-001  8.924e-001        0.03s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      7  8.989e-001  8.924e-001        0.03s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      8  9.034e-001  8.924e-001        0.04s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      8  9.034e-001  8.924e-001        0.04s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      9  9.078e-001  8.924e-001        0.04s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      9  9.078e-001  8.924e-001        0.04s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     10  9.064e-001  8.924e-001        0.05s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     10  9.064e-001  8.924e-001        0.05s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     11  9.081e-001  8.861e-001        0.05s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     11  9.081e-001  8.861e-001        0.05s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     12  9.115e-001  8.797e-001        0.05s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     12  9.115e-001  8.797e-001        0.05s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     13  9.143e-001  8.797e-001        0.06s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     13  9.143e-001  8.797e-001        0.06s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     14  9.156e-001  8.734e-001        0.06s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     14  9.156e-001  8.734e-001        0.06s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     15  9.187e-001  8.734e-001        0.07s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     15  9.187e-001  8.734e-001        0.07s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     16  9.207e-001  8.734e-001        0.08s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     16  9.207e-001  8.734e-001        0.08s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     17  9.234e-001  8.734e-001        0.09s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     17  9.234e-001  8.734e-001        0.09s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     18  9.238e-001  8.734e-001        0.10s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     18  9.238e-001  8.734e-001        0.10s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     19  9.258e-001  8.861e-001        0.10s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     19  9.258e-001  8.861e-001        0.10s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     20  9.279e-001  8.861e-001        0.11s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     20  9.279e-001  8.861e-001        0.11s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     21  9.296e-001  8.861e-001        0.11s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     21  9.296e-001  8.861e-001        0.11s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     22  9.299e-001  8.797e-001        0.12s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     22  9.299e-001  8.797e-001        0.12s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     23  9.340e-001  8.797e-001        0.12s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     23  9.340e-001  8.797e-001        0.12s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     24  9.364e-001  8.797e-001        0.13s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     24  9.364e-001  8.797e-001        0.13s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     25  9.364e-001  8.797e-001        0.13s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     25  9.364e-001  8.797e-001        0.13s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     26  9.381e-001  8.797e-001        0.14s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     26  9.381e-001  8.797e-001        0.14s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     27  9.384e-001  8.797e-001        0.15s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     27  9.384e-001  8.797e-001        0.15s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     28  9.384e-001  8.797e-001        0.15s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     28  9.384e-001  8.797e-001        0.15s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     29  9.388e-001  8.797e-001        0.16s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     29  9.388e-001  8.797e-001        0.16s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     30  9.401e-001  8.797e-001        0.17s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     30  9.401e-001  8.797e-001        0.17s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     31  9.401e-001  8.797e-001        0.17s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     31  9.401e-001  8.797e-001        0.17s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     32  9.408e-001  8.797e-001        0.18s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     32  9.408e-001  8.797e-001        0.18s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     33  9.435e-001  8.797e-001        0.18s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     33  9.435e-001  8.797e-001        0.18s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     34  9.449e-001  8.797e-001        0.19s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     34  9.449e-001  8.797e-001        0.19s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     35  9.486e-001  8.734e-001        0.20s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     35  9.486e-001  8.734e-001        0.20s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     36  9.483e-001  8.797e-001        0.20s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     36  9.483e-001  8.797e-001        0.20s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     37  9.486e-001  8.797e-001        0.21s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     37  9.486e-001  8.797e-001        0.21s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     38  9.479e-001  8.797e-001        0.22s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     38  9.479e-001  8.797e-001        0.22s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     39  9.496e-001  8.797e-001        0.22s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     39  9.496e-001  8.797e-001        0.22s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     40  9.513e-001  8.797e-001        0.23s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     40  9.513e-001  8.797e-001        0.23s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     41  9.517e-001  8.797e-001        0.23s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     41  9.517e-001  8.797e-001        0.23s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     42  9.541e-001  8.797e-001        0.24s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     42  9.541e-001  8.797e-001        0.24s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     43  9.568e-001  8.734e-001        0.24s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     43  9.568e-001  8.734e-001        0.24s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     44  9.592e-001  8.734e-001        0.25s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     44  9.592e-001  8.734e-001        0.25s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     45  9.612e-001  8.734e-001        0.26s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     45  9.612e-001  8.734e-001        0.26s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     46  9.605e-001  8.734e-001        0.26s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     46  9.605e-001  8.734e-001        0.26s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     47  9.619e-001  8.734e-001        0.27s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     47  9.619e-001  8.734e-001        0.27s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     48  9.622e-001  8.734e-001        0.27s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     48  9.622e-001  8.734e-001        0.27s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:     49  9.622e-001  8.734e-001        0.28s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:     49  9.622e-001  8.734e-001        0.28s"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "PROGRESS: Creating a validation set from 5 percent of training data. This may take a while.\n",
        "          You can set ``validation_set=None`` to disable validation tracking.\n",
        "\n"
       ]
      },
      {
       "javascript": [
        "$(\"head\").append($(\"<link/>\").attr({\n",
        "  rel:  \"stylesheet\",\n",
        "  type: \"text/css\",\n",
        "  href: \"//cdnjs.cloudflare.com/ajax/libs/font-awesome/4.1.0/css/font-awesome.min.css\"\n",
        "}));\n",
        "$(\"head\").append($(\"<link/>\").attr({\n",
        "  rel:  \"stylesheet\",\n",
        "  type: \"text/css\",\n",
        "  href: \"//turi.com/files/canvas/1.6.1/css/canvas.css\"\n",
        "}));\n",
        "\n",
        "            (function(){\n",
        "\n",
        "                var e = null;\n",
        "                if (typeof element == 'undefined') {\n",
        "                    var scripts = document.getElementsByTagName('script');\n",
        "                    var thisScriptTag = scripts[scripts.length-1];\n",
        "                    var parentDiv = thisScriptTag.parentNode;\n",
        "                    e = document.createElement('div');\n",
        "                    parentDiv.appendChild(e);\n",
        "                } else {\n",
        "                    e = element[0];\n",
        "                }\n",
        "\n",
        "                if (typeof requirejs !== 'undefined') {\n",
        "                    // disable load timeout; ipython_app.js is large and can take a while to load.\n",
        "                    requirejs.config({waitSeconds: 0});\n",
        "                }\n",
        "\n",
        "                require(['//turi.com/files/canvas/1.6.1/js/ipython_app.js'], function(IPythonApp){\n",
        "                    var app = new IPythonApp();\n",
        "                    app.attachView('sgraph','View', {\"edges_labels\": [\"no\", \"no\", \"yes\", \"yes\", \"no\", \"yes\", \"no\", \"no\", \"yes\", \"yes\", \"yes\", \"no\", \"yes\", \"yes\", \"no\", \"yes\", \"no\", \"yes\", \"no\", \"no\", \"yes\", \"no\", \"no\", \"yes\", \"no\", \"yes\", \"yes\", \"yes\", \"yes\", \"no\", \"no\", \"no\", \"no\", \"yes\", \"no\", \"yes\", \"no\", \"yes\", \"yes\", \"no\", \"no\", \"yes\", \"no\", \"no\", \"yes\", \"yes\", \"no\", \"yes\", \"yes\", \"yes\", \"no\", \"no\", \"no\", \"no\", \"yes\", \"yes\", \"no\", \"yes\", \"no\", \"yes\", \"yes\", \"no\"], \"selected_variable\": {\"name\": [\"<SGraph>\"], \"view_file\": \"sgraph\", \"view_component\": \"View\", \"view_params\": {\"elabel_hover\": false, \"vertex_positions\": null, \"h_offset\": 0.0, \"node_size\": 300, \"ecolor\": [0.37, 0.33, 0.33], \"elabel\": \"value\", \"arrows\": true, \"ewidth\": 1, \"vlabel\": \"__repr__\", \"highlight_color\": [0.69, 0.0, 0.498], \"vcolor\": [0.522, 0.741, 0.0], \"vlabel_hover\": false, \"highlight\": {\"0\": [0.69, 0.0, 0.48], \"31\": [0.039, 0.55, 0.77], \"32\": [1.0, 0.33, 0.0], \"33\": [0.039, 0.55, 0.77], \"34\": [1.0, 0.33, 0.0], \"35\": [0.039, 0.55, 0.77], \"36\": [1.0, 0.33, 0.0], \"37\": [1.0, 0.33, 0.0], \"38\": [0.039, 0.55, 0.77], \"39\": [1.0, 0.33, 0.0], \"40\": [0.039, 0.55, 0.77], \"41\": [0.039, 0.55, 0.77], \"42\": [1.0, 0.33, 0.0], \"43\": [0.039, 0.55, 0.77], \"44\": [0.039, 0.55, 0.77], \"45\": [0.039, 0.55, 0.77], \"46\": [0.039, 0.55, 0.77], \"47\": [0.039, 0.55, 0.77], \"48\": [1.0, 0.33, 0.0], \"49\": [0.039, 0.55, 0.77], \"50\": [1.0, 0.33, 0.0], \"51\": [1.0, 0.33, 0.0], \"52\": [0.039, 0.55, 0.77], \"53\": [0.039, 0.55, 0.77], \"54\": [1.0, 0.33, 0.0], \"55\": [1.0, 0.33, 0.0], \"56\": [1.0, 0.33, 0.0], \"57\": [0.039, 0.55, 0.77], \"58\": [1.0, 0.33, 0.0], \"59\": [1.0, 0.33, 0.0], \"60\": [0.039, 0.55, 0.77], \"61\": [0.039, 0.55, 0.77], \"62\": [0.039, 0.55, 0.77]}, \"v_offset\": 0.03}, \"view_components\": [\"View\"], \"type\": \"SGraph\", \"descriptives_links\": {\"edges\": \"edges\", \"vertices\": \"vertices\"}, \"descriptives\": {\"edges\": 62, \"vertices\": 63}}, \"positions\": null, \"error_type\": 0, \"vertices\": [27, 52, 50, 5, 8, 29, 33, 47, 7, 10, 58, 56, 49, 34, 30, 16, 23, 36, 21, 61, 62, 15, 0, 43, 40, 38, 18, 37, 13, 53, 48, 2, 59, 46, 35, 26, 31, 6, 28, 51, 32, 9, 25, 14, 12, 45, 54, 3, 1, 42, 41, 60, 20, 39, 19, 57, 55, 24, 11, 22, 44, 17, 4], \"vertices_labels\": [\"PY_DIFF<-15.0\", \"-0.105882\", \"0.3\", \"RA_DIFF<6.0\", \"PY_DIFF<96.0\", \"INT_DIFF<2.0\", \"-0.066667\", \"-0.085714\", \"SK_DIFF<0.0\", \"INT_DIFF<1.0\", \"0.046154\", \"0.4176\", \"-0.054545\", \"0.327273\", \"TOP_DIFF<8.5\", \"RA_DIFF<-4.0\", \"PY_DIFF<-139.0\", \"0.171429\", \"FUM_DIFF<0.0\", \"-0.464789\", \"-0.02069\", \"PY_DIFF<-223.0\", \"RA_DIFF<0.0\", \"-0.026866\", \"-0.408\", \"-0.046154\", \"PA_DIFF<12.0\", \"0.371429\", \"PA_DIFF<13.0\", \"-0.12\", \"0.413546\", \"INT_DIFF<1.0\", \"0.189474\", \"-0.364706\", \"-0.290722\", \"RY_DIFF<-92.0\", \"-0.12\", \"SK_DIFF<1.0\", \"PY_DIFF<117.0\", \"0.463704\", \"0.433846\", \"RY_DIFF<-49.0\", \"PA_DIFF<3.0\", \"FUM_DIFF<0.0\", \"PY_DIFF<1.0\", \"-0.546667\", \"0.549035\", \"PA_DIFF<0.0\", \"INT_DIFF<0.0\", \"0.253846\", \"-0.15\", \"-0.3\", \"PY_DIFF<-63.0\", \"0.018182\", \"INT_DIFF<1.0\", \"-0.408\", \"0.037975\", \"PY_DIFF<5.0\", \"SK_DIFF<1.0\", \"RA_DIFF<-3.0\", \"-0.419417\", \"RY_DIFF<-5.0\", \"SK_DIFF<0.0\"], \"edges\": [[7, 16], [27, 56], [10, 21], [7, 15], [8, 18], [29, 59], [5, 12], [29, 60], [8, 17], [5, 11], [27, 55], [10, 22], [16, 33], [23, 47], [16, 34], [30, 61], [30, 62], [21, 43], [23, 48], [21, 44], [18, 37], [18, 38], [0, 2], [15, 31], [15, 32], [0, 1], [2, 5], [13, 27], [26, 53], [2, 6], [13, 28], [26, 54], [28, 58], [6, 13], [6, 14], [9, 19], [9, 20], [28, 57], [3, 7], [3, 8], [25, 52], [14, 29], [14, 30], [12, 26], [25, 51], [12, 25], [19, 40], [1, 3], [19, 39], [20, 41], [20, 42], [1, 4], [24, 50], [4, 10], [11, 23], [24, 49], [17, 36], [17, 35], [22, 46], [4, 9], [22, 45], [11, 24]], \"ipython\": true, \"error_msg\": \"\"}, e);\n",
        "                });\n",
        "            })();\n",
        "        "
       ],
       "metadata": {},
       "output_type": "display_data"
      }
     ],
     "prompt_number": 48
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Nope.  \n",
      "\n",
      "I\u2019m going to try going back and adding some more features\u2026 maybe compute average yards per pass attempt, yards per rush attempt, average starting field position.  You know, just some good to know stuff."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "# OK so this isn't really what I expected.\n",
      "# The first branch is Rushing Attempts, then Interceptions difference.\n",
      "# should probably add more features\n",
      "# PEN = penalty yardage against\n",
      "# SRP = Succeessful rush plays\n",
      "# SPP = Successful pass plays\n",
      "# SFPY =  The Total Starting Field Position Yardage: Dividing by the # of Drives on Offense (DRV)\n",
      "#         produces the Average Starting Field Position.\n",
      "# PU = Punts\n",
      "# DRV = Drives on Offense\n",
      "\n",
      "# Create a pass yds / attempt feature, and an average starting field position (SFPY/DRV)\n",
      "# Total rush / rush attempts (AVG RUSH)\n",
      "extra = gl.SFrame.read_csv('C:\\\\Users\\\\Susan\\\\Documents\\\\NFLData_2000-2014\\\\csv\\TEAM.csv', header=True)\n",
      "# games = gl.SFrame.read_csv('C:\\\\Users\\\\Susan\\\\Documents\\\\NFLData_2000-2014\\\\csv\\GAME.csv', header=True)\n",
      "extra = extra.select_columns(\n",
      "    [ 'TID', 'GID','SRP',  'SPP',  'PLO', 'PLD', 'PEN', 'SFPY', 'DRV'])\n",
      "# team_more_info['TOP'] = team_more_info['TOP'].astype(float)\n",
      "# team_more_info = team_more_info.join(games.select_columns(['GID', 'SEAS', 'WEEK']), 'GID')\n",
      "# team_more_info = team_more_info[team['WEEK'] < 18]\n",
      "team = team.join(extra, on=['GID', 'TID'])\n",
      "\n",
      "#GID 364 has no data (CAR/MIA 2001) for DRV ... without the if x['blah'] !=0 you get a divide by zero\n",
      "# but ONLY at the point you try and materialize the sframe because the operations are lazy! gah.\n",
      "\n",
      "team['YD_PER_P_ATT'] = team.apply(lambda x: float(x['PY']) / float(x['PA']) if x['PA'] != 0 else 0)\n",
      "team['YD_PER_R_ATT'] = team.apply(lambda x: float(x['RY']) / float(x['RA']) if x['RA'] != 0 else 0)\n",
      "team['AVG_STFP']     = team.apply(lambda x: float(x['SFPY']) / float(x['DRV'])  if x['DRV'] != 0 else 0)\n",
      "\n",
      "#On second thought, I want to drop that game altogether since I don't trust that.  The score and other\n",
      "#references show they did have an off drive.  wth?\n",
      "team = team.filter_by( [364], 'GID', exclude=True)\n",
      "\n",
      "features = ['YD_PER_P_ATT', 'YD_PER_R_ATT', 'AVG_STFP', 'SRP', 'PY', 'RY', 'DRV', 'SPP', 'FUM', 'SK', 'INT', 'PLO',\n",
      "            'PLD', 'TOP', 'PEN', 'PU']\n",
      "\n",
      "# doing that joining stuff again\n",
      "more = team.join(games.select_columns(['GID', 'V', 'H', 'PTSV', 'PTSH']), 'GID')\n",
      "home = more[more['TNAME'] == more['H']].remove_columns(['PTSH', 'PTSV'])\n",
      "visit = more[more['TNAME'] == more['V']].remove_columns(['PTSH', 'PTSV'])\n",
      "difftable = home.join(visit, 'GID')\n",
      "#the visitor's info is now always under blank.1 column names\n",
      "#so let's express everything again as a difference\n",
      "for thing in features:\n",
      "    difftable['%s_DIFF' % thing] = difftable[thing] - difftable['%s.1' % thing]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<pre>PROGRESS: Finished parsing file C:\\Users\\Susan\\Documents\\NFLData_2000-2014\\csv\\TEAM.csv</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Finished parsing file C:\\Users\\Susan\\Documents\\NFLData_2000-2014\\csv\\TEAM.csv"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Parsing completed. Parsed 100 lines in 0.135093 secs.</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Parsing completed. Parsed 100 lines in 0.135093 secs."
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Finished parsing file C:\\Users\\Susan\\Documents\\NFLData_2000-2014\\csv\\TEAM.csv</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Finished parsing file C:\\Users\\Susan\\Documents\\NFLData_2000-2014\\csv\\TEAM.csv"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Parsing completed. Parsed 7978 lines in 0.122088 secs.</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Parsing completed. Parsed 7978 lines in 0.122088 secs."
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "------------------------------------------------------\n",
        "Inferred types from first line of file as \n",
        "column_type_hints=[long,long,str,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,str,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,str,str,long,long,long,long,long,long,str,str,str,str,str,str,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long,long]\n",
        "If parsing fails due to incorrect types, you can correct\n",
        "the inferred type list above and pass it to read_csv in\n",
        "the column_type_hints argument\n",
        "------------------------------------------------------\n"
       ]
      }
     ],
     "prompt_number": 49
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "diff_train, diff_test = difftable.random_split(0.8, seed=0)\n",
      "stuff = [ i for i in difftable.column_names() if \"_DIFF\" in i]\n",
      "btc_classifier = gl.boosted_trees_classifier.create(diff_train, target='WIN',\n",
      "                                                    features=stuff)\n",
      "btc_classifier.show(view=\"Tree\", tree_id=1)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<pre>PROGRESS: Boosted trees classifier:</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Boosted trees classifier:"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of examples          : 2915</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of examples          : 2915"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of classes           : 2</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of classes           : 2"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of feature columns   : 16</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of feature columns   : 16"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of unpacked features : 16</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of unpacked features : 16"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Starting Boosted Trees</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Starting Boosted Trees"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:   Iter      Accuracy          Elapsed time</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:   Iter      Accuracy          Elapsed time"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:         (training) (validation)</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:         (training) (validation)"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      0  8.954e-001  8.205e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      0  8.954e-001  8.205e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      1  9.005e-001  8.269e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      1  9.005e-001  8.269e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      2  9.129e-001  8.718e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      2  9.129e-001  8.718e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      3  9.177e-001  8.654e-001        0.03s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      3  9.177e-001  8.654e-001        0.03s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      4  9.259e-001  8.590e-001        0.03s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      4  9.259e-001  8.590e-001        0.03s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      5  9.369e-001  8.782e-001        0.04s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      5  9.369e-001  8.782e-001        0.04s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      6  9.358e-001  8.910e-001        0.06s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      6  9.358e-001  8.910e-001        0.06s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      7  9.400e-001  8.846e-001        0.06s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      7  9.400e-001  8.846e-001        0.06s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      8  9.444e-001  8.846e-001        0.07s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      8  9.444e-001  8.846e-001        0.07s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      9  9.458e-001  8.846e-001        0.08s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      9  9.458e-001  8.846e-001        0.08s"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "PROGRESS: Creating a validation set from 5 percent of training data. This may take a while.\n",
        "          You can set ``validation_set=None`` to disable validation tracking.\n",
        "\n"
       ]
      },
      {
       "javascript": [
        "$(\"head\").append($(\"<link/>\").attr({\n",
        "  rel:  \"stylesheet\",\n",
        "  type: \"text/css\",\n",
        "  href: \"//cdnjs.cloudflare.com/ajax/libs/font-awesome/4.1.0/css/font-awesome.min.css\"\n",
        "}));\n",
        "$(\"head\").append($(\"<link/>\").attr({\n",
        "  rel:  \"stylesheet\",\n",
        "  type: \"text/css\",\n",
        "  href: \"//turi.com/files/canvas/1.6.1/css/canvas.css\"\n",
        "}));\n",
        "\n",
        "            (function(){\n",
        "\n",
        "                var e = null;\n",
        "                if (typeof element == 'undefined') {\n",
        "                    var scripts = document.getElementsByTagName('script');\n",
        "                    var thisScriptTag = scripts[scripts.length-1];\n",
        "                    var parentDiv = thisScriptTag.parentNode;\n",
        "                    e = document.createElement('div');\n",
        "                    parentDiv.appendChild(e);\n",
        "                } else {\n",
        "                    e = element[0];\n",
        "                }\n",
        "\n",
        "                if (typeof requirejs !== 'undefined') {\n",
        "                    // disable load timeout; ipython_app.js is large and can take a while to load.\n",
        "                    requirejs.config({waitSeconds: 0});\n",
        "                }\n",
        "\n",
        "                require(['//turi.com/files/canvas/1.6.1/js/ipython_app.js'], function(IPythonApp){\n",
        "                    var app = new IPythonApp();\n",
        "                    app.attachView('sgraph','View', {\"edges_labels\": [\"no\", \"no\", \"yes\", \"no\", \"yes\", \"yes\", \"no\", \"yes\", \"yes\", \"yes\", \"no\", \"no\", \"no\", \"no\", \"no\", \"yes\", \"yes\", \"yes\", \"yes\", \"no\", \"yes\", \"no\", \"yes\", \"yes\", \"yes\", \"no\", \"yes\", \"no\", \"yes\", \"no\", \"yes\", \"no\", \"yes\", \"no\", \"yes\", \"no\", \"no\", \"no\", \"no\", \"yes\", \"no\", \"yes\", \"no\", \"no\", \"no\", \"yes\", \"yes\", \"yes\", \"no\", \"no\", \"no\", \"no\", \"yes\", \"yes\", \"yes\", \"yes\", \"yes\", \"yes\", \"no\", \"no\", \"yes\", \"no\", \"yes\", \"no\", \"yes\", \"yes\", \"no\", \"no\", \"no\", \"yes\", \"no\", \"no\", \"yes\", \"yes\", \"yes\", \"no\", \"yes\", \"no\", \"yes\", \"no\", \"yes\", \"no\", \"no\", \"yes\", \"yes\", \"no\", \"yes\", \"no\", \"no\", \"yes\", \"yes\", \"no\", \"no\", \"yes\", \"yes\", \"no\", \"no\", \"yes\", \"yes\", \"yes\", \"no\", \"yes\", \"yes\", \"no\", \"yes\", \"no\", \"no\", \"no\", \"no\", \"yes\", \"no\", \"no\", \"yes\", \"no\", \"yes\", \"yes\", \"yes\", \"yes\", \"no\", \"no\", \"no\", \"yes\", \"yes\", \"no\"], \"selected_variable\": {\"name\": [\"<SGraph>\"], \"view_file\": \"sgraph\", \"view_component\": \"View\", \"view_params\": {\"elabel_hover\": false, \"vertex_positions\": null, \"h_offset\": 0.0, \"node_size\": 300, \"ecolor\": [0.37, 0.33, 0.33], \"elabel\": \"value\", \"arrows\": true, \"ewidth\": 1, \"vlabel\": \"__repr__\", \"highlight_color\": [0.69, 0.0, 0.498], \"vcolor\": [0.522, 0.741, 0.0], \"vlabel_hover\": false, \"highlight\": {\"0\": [0.69, 0.0, 0.48], \"39\": [1.0, 0.33, 0.0], \"63\": [0.039, 0.55, 0.77], \"64\": [1.0, 0.33, 0.0], \"65\": [0.039, 0.55, 0.77], \"66\": [0.039, 0.55, 0.77], \"67\": [0.039, 0.55, 0.77], \"68\": [1.0, 0.33, 0.0], \"69\": [1.0, 0.33, 0.0], \"70\": [0.039, 0.55, 0.77], \"71\": [1.0, 0.33, 0.0], \"72\": [0.039, 0.55, 0.77], \"73\": [0.039, 0.55, 0.77], \"74\": [1.0, 0.33, 0.0], \"75\": [1.0, 0.33, 0.0], \"76\": [0.039, 0.55, 0.77], \"77\": [0.039, 0.55, 0.77], \"78\": [1.0, 0.33, 0.0], \"79\": [0.039, 0.55, 0.77], \"80\": [0.039, 0.55, 0.77], \"81\": [1.0, 0.33, 0.0], \"82\": [0.039, 0.55, 0.77], \"83\": [1.0, 0.33, 0.0], \"84\": [0.039, 0.55, 0.77], \"85\": [1.0, 0.33, 0.0], \"86\": [0.039, 0.55, 0.77], \"87\": [1.0, 0.33, 0.0], \"88\": [0.039, 0.55, 0.77], \"89\": [1.0, 0.33, 0.0], \"90\": [1.0, 0.33, 0.0], \"91\": [0.039, 0.55, 0.77], \"92\": [1.0, 0.33, 0.0], \"93\": [1.0, 0.33, 0.0], \"94\": [0.039, 0.55, 0.77], \"95\": [0.039, 0.55, 0.77], \"96\": [0.039, 0.55, 0.77], \"97\": [1.0, 0.33, 0.0], \"98\": [0.039, 0.55, 0.77], \"99\": [1.0, 0.33, 0.0], \"100\": [0.039, 0.55, 0.77], \"101\": [1.0, 0.33, 0.0], \"102\": [1.0, 0.33, 0.0], \"103\": [0.039, 0.55, 0.77], \"104\": [1.0, 0.33, 0.0], \"105\": [0.039, 0.55, 0.77], \"106\": [1.0, 0.33, 0.0], \"107\": [0.039, 0.55, 0.77], \"108\": [1.0, 0.33, 0.0], \"109\": [1.0, 0.33, 0.0], \"110\": [0.039, 0.55, 0.77], \"111\": [1.0, 0.33, 0.0], \"112\": [1.0, 0.33, 0.0], \"113\": [1.0, 0.33, 0.0], \"114\": [1.0, 0.33, 0.0], \"115\": [1.0, 0.33, 0.0], \"116\": [0.039, 0.55, 0.77], \"117\": [0.039, 0.55, 0.77], \"118\": [1.0, 0.33, 0.0], \"119\": [1.0, 0.33, 0.0], \"120\": [0.039, 0.55, 0.77], \"121\": [1.0, 0.33, 0.0], \"122\": [0.039, 0.55, 0.77], \"123\": [0.039, 0.55, 0.77], \"124\": [1.0, 0.33, 0.0]}, \"v_offset\": 0.03}, \"view_components\": [\"View\"], \"type\": \"SGraph\", \"descriptives_links\": {\"edges\": \"edges\", \"vertices\": \"vertices\"}, \"descriptives\": {\"edges\": 124, \"vertices\": 125}}, \"positions\": null, \"error_type\": 0, \"vertices\": [8, 7, 52, 50, 96, 70, 27, 33, 113, 110, 92, 68, 47, 73, 115, 10, 87, 29, 5, 76, 67, 56, 63, 109, 58, 34, 16, 118, 65, 30, 21, 36, 23, 89, 49, 107, 18, 15, 38, 61, 106, 62, 123, 78, 40, 43, 0, 83, 124, 103, 37, 105, 75, 35, 108, 119, 121, 53, 2, 13, 59, 46, 31, 26, 86, 88, 48, 72, 69, 71, 32, 111, 91, 93, 28, 74, 6, 114, 51, 116, 9, 97, 54, 25, 14, 120, 122, 3, 80, 85, 45, 98, 100, 12, 94, 64, 60, 20, 19, 39, 102, 79, 42, 1, 104, 101, 41, 81, 82, 17, 66, 95, 11, 117, 57, 112, 77, 24, 84, 44, 99, 4, 90, 55, 22], \"vertices_labels\": [\"INT_DIFF<1.0\", \"INT_DIFF<1.0\", \"YD_PER_P_ATT_DIFF<0.254078\", \"SK_DIFF<1.0\", \"-0.369404\", \"-0.440631\", \"YD_PER_P_ATT_DIFF<0.941266\", \"SRP_DIFF<-5.0\", \"0.140453\", \"-0.261762\", \"0.206024\", \"0.177999\", \"INT_DIFF<0.0\", \"-0.122494\", \"0.428703\", \"YD_PER_P_ATT_DIFF<-2.766667\", \"0.328351\", \"YD_PER_P_ATT_DIFF<1.829498\", \"AVG_STFP_DIFF<2.690909\", \"-0.282568\", \"-0.180713\", \"FUM_DIFF<1.0\", \"-0.17274\", \"0.279887\", \"FUM_DIFF<4.0\", \"DRV_DIFF<-1.0\", \"SK_DIFF<-2.0\", \"0.15026\", \"-0.122373\", \"PEN_DIFF<4.0\", \"YD_PER_R_ATT_DIFF<2.629419\", \"YD_PER_P_ATT_DIFF<-4.333885\", \"FUM_DIFF<0.0\", \"0.356429\", \"SK_DIFF<0.0\", \"-0.366969\", \"FUM_DIFF<0.0\", \"SK_DIFF<0.0\", \"SPP_DIFF<13.0\", \"PY_DIFF<154.0\", \"0.306348\", \"YD_PER_P_ATT_DIFF<4.088973\", \"-0.26394\", \"0.11289\", \"TOP_DIFF<-5.0\", \"FUM_DIFF<0.0\", \"YD_PER_P_ATT_DIFF<-0.511259\", \"0.202394\", \"0.26331\", \"-0.206225\", \"PU_DIFF<1.0\", \"-0.190151\", \"0.210058\", \"FUM_DIFF<0.0\", \"0.113599\", \"0.288413\", \"0.398025\", \"SRP_DIFF<-10.0\", \"SRP_DIFF<-1.0\", \"AVG_STFP_DIFF<-6.890967\", \"AVG_STFP_DIFF<12.208791\", \"PLO_DIFF<10.0\", \"YD_PER_P_ATT_DIFF<-1.274832\", \"PEN_DIFF<10.0\", \"-0.249151\", \"-0.098632\", \"PEN_DIFF<-45.0\", \"-0.194906\", \"0.052341\", \"0.282568\", \"FUM_DIFF<0.0\", \"0.425712\", \"-0.256401\", \"0.381522\", \"YD_PER_P_ATT_DIFF<0.455495\", \"0.246736\", \"INT_DIFF<2.0\", \"0.38204\", \"RY_DIFF<-120.0\", \"-0.154167\", \"YD_PER_P_ATT_DIFF<-1.832956\", \"0.190161\", \"YD_PER_P_ATT_DIFF<1.99718\", \"PLO_DIFF<3.0\", \"RY_DIFF<53.0\", \"-0.222976\", \"-0.096236\", \"AVG_STFP_DIFF<5.029412\", \"-0.435132\", \"0.095577\", \"PEN_DIFF<47.0\", \"-0.299176\", \"-0.000541\", \"INT_DIFF<0.0\", \"-0.110983\", \"0.163496\", \"FUM_DIFF<1.0\", \"PEN_DIFF<-10.0\", \"YD_PER_R_ATT_DIFF<-3.715629\", \"0.11289\", \"0.421703\", \"-0.087897\", \"SK_DIFF<0.0\", \"SRP_DIFF<3.0\", \"0.088953\", \"0.014205\", \"PY_DIFF<36.0\", \"0.308724\", \"-0.129671\", \"SRP_DIFF<-2.0\", \"-0.391611\", \"-0.041842\", \"YD_PER_P_ATT_DIFF<2.320305\", \"-0.387999\", \"SRP_DIFF<6.0\", \"0.09736\", \"-0.39454\", \"RY_DIFF<-68.0\", \"-0.373752\", \"YD_PER_R_ATT_DIFF<3.655629\", \"0.407514\", \"AVG_STFP_DIFF<-2.081629\", \"0.007369\", \"FUM_DIFF<0.0\", \"INT_DIFF<2.0\"], \"edges\": [[33, 68], [7, 16], [33, 67], [27, 56], [10, 21], [7, 15], [8, 18], [52, 103], [29, 59], [47, 93], [5, 12], [47, 94], [50, 100], [29, 60], [52, 104], [8, 17], [27, 55], [5, 11], [50, 99], [10, 22], [16, 33], [34, 70], [36, 73], [58, 115], [23, 47], [16, 34], [30, 61], [30, 62], [21, 43], [23, 48], [34, 69], [36, 74], [56, 111], [58, 116], [49, 97], [49, 98], [56, 112], [21, 44], [37, 76], [18, 37], [18, 38], [62, 123], [62, 124], [38, 78], [0, 2], [15, 31], [37, 75], [61, 121], [43, 86], [15, 32], [61, 122], [40, 80], [43, 85], [0, 1], [40, 79], [38, 77], [13, 27], [2, 5], [48, 96], [46, 92], [31, 63], [59, 118], [53, 105], [53, 106], [26, 53], [35, 71], [35, 72], [13, 28], [2, 6], [46, 91], [26, 54], [31, 64], [59, 117], [48, 95], [32, 65], [28, 58], [6, 13], [6, 14], [51, 101], [51, 102], [9, 19], [9, 20], [32, 66], [28, 57], [3, 7], [3, 8], [14, 29], [25, 52], [14, 30], [54, 107], [45, 89], [54, 108], [12, 26], [25, 51], [12, 25], [45, 90], [19, 40], [42, 83], [60, 119], [1, 3], [60, 120], [19, 39], [20, 41], [20, 42], [41, 81], [41, 82], [1, 4], [42, 84], [55, 110], [57, 113], [24, 50], [4, 10], [44, 87], [17, 36], [55, 109], [11, 23], [24, 49], [17, 35], [44, 88], [22, 46], [57, 114], [4, 9], [22, 45], [11, 24]], \"ipython\": true, \"error_msg\": \"\"}, e);\n",
        "                });\n",
        "            })();\n",
        "        "
       ],
       "metadata": {},
       "output_type": "display_data"
      }
     ],
     "prompt_number": 50
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Gah, look at that mess again."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "btc_classifier = gl.boosted_trees_classifier.create(diff_train, target='WIN',\n",
      "                                                    features=stuff, max_depth=3)\n",
      "\n",
      "all_f_results = btc_classifier.evaluate(diff_test)\n",
      "btc_classifier.show(view=\"Tree\")\n",
      "print all_f_results"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<pre>PROGRESS: Boosted trees classifier:</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Boosted trees classifier:"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of examples          : 2932</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of examples          : 2932"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of classes           : 2</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of classes           : 2"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of feature columns   : 16</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of feature columns   : 16"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of unpacked features : 16</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of unpacked features : 16"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Starting Boosted Trees</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Starting Boosted Trees"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:   Iter      Accuracy          Elapsed time</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:   Iter      Accuracy          Elapsed time"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:         (training) (validation)</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:         (training) (validation)"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      0  8.254e-001  8.489e-001        0.00s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      0  8.254e-001  8.489e-001        0.00s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      1  8.571e-001  8.273e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      1  8.571e-001  8.273e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      2  8.547e-001  8.705e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      2  8.547e-001  8.705e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      3  8.636e-001  8.561e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      3  8.636e-001  8.561e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      4  8.660e-001  8.705e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      4  8.660e-001  8.705e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      5  8.738e-001  8.849e-001        0.03s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      5  8.738e-001  8.849e-001        0.03s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      6  8.782e-001  8.705e-001        0.03s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      6  8.782e-001  8.705e-001        0.03s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      7  8.810e-001  8.921e-001        0.04s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      7  8.810e-001  8.921e-001        0.04s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      8  8.844e-001  8.993e-001        0.04s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      8  8.844e-001  8.993e-001        0.04s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      9  8.932e-001  8.921e-001        0.04s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      9  8.932e-001  8.921e-001        0.04s"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "PROGRESS: Creating a validation set from 5 percent of training data. This may take a while.\n",
        "          You can set ``validation_set=None`` to disable validation tracking.\n",
        "\n"
       ]
      },
      {
       "javascript": [
        "$(\"head\").append($(\"<link/>\").attr({\n",
        "  rel:  \"stylesheet\",\n",
        "  type: \"text/css\",\n",
        "  href: \"//cdnjs.cloudflare.com/ajax/libs/font-awesome/4.1.0/css/font-awesome.min.css\"\n",
        "}));\n",
        "$(\"head\").append($(\"<link/>\").attr({\n",
        "  rel:  \"stylesheet\",\n",
        "  type: \"text/css\",\n",
        "  href: \"//turi.com/files/canvas/1.6.1/css/canvas.css\"\n",
        "}));\n",
        "\n",
        "            (function(){\n",
        "\n",
        "                var e = null;\n",
        "                if (typeof element == 'undefined') {\n",
        "                    var scripts = document.getElementsByTagName('script');\n",
        "                    var thisScriptTag = scripts[scripts.length-1];\n",
        "                    var parentDiv = thisScriptTag.parentNode;\n",
        "                    e = document.createElement('div');\n",
        "                    parentDiv.appendChild(e);\n",
        "                } else {\n",
        "                    e = element[0];\n",
        "                }\n",
        "\n",
        "                if (typeof requirejs !== 'undefined') {\n",
        "                    // disable load timeout; ipython_app.js is large and can take a while to load.\n",
        "                    requirejs.config({waitSeconds: 0});\n",
        "                }\n",
        "\n",
        "                require(['//turi.com/files/canvas/1.6.1/js/ipython_app.js'], function(IPythonApp){\n",
        "                    var app = new IPythonApp();\n",
        "                    app.attachView('sgraph','View', {\"edges_labels\": [\"no\", \"yes\", \"no\", \"yes\", \"yes\", \"no\", \"yes\", \"no\", \"yes\", \"no\", \"yes\", \"no\", \"no\", \"yes\"], \"selected_variable\": {\"name\": [\"<SGraph>\"], \"view_file\": \"sgraph\", \"view_component\": \"View\", \"view_params\": {\"elabel_hover\": false, \"vertex_positions\": null, \"h_offset\": 0.0, \"node_size\": 300, \"ecolor\": [0.37, 0.33, 0.33], \"elabel\": \"value\", \"arrows\": true, \"ewidth\": 1, \"vlabel\": \"__repr__\", \"highlight_color\": [0.69, 0.0, 0.498], \"vcolor\": [0.522, 0.741, 0.0], \"vlabel_hover\": false, \"highlight\": {\"0\": [0.69, 0.0, 0.48], \"7\": [0.039, 0.55, 0.77], \"8\": [0.039, 0.55, 0.77], \"9\": [0.039, 0.55, 0.77], \"10\": [1.0, 0.33, 0.0], \"11\": [1.0, 0.33, 0.0], \"12\": [0.039, 0.55, 0.77], \"13\": [1.0, 0.33, 0.0], \"14\": [1.0, 0.33, 0.0]}, \"v_offset\": 0.03}, \"view_components\": [\"View\"], \"type\": \"SGraph\", \"descriptives_links\": {\"edges\": \"edges\", \"vertices\": \"vertices\"}, \"descriptives\": {\"edges\": 14, \"vertices\": 15}}, \"positions\": null, \"error_type\": 0, \"vertices\": [5, 10, 8, 7, 0, 13, 2, 6, 9, 14, 12, 3, 1, 11, 4], \"vertices_labels\": [\"INT_DIFF<1.0\", \"0.307826\", \"-0.086897\", \"-0.484615\", \"YD_PER_P_ATT_DIFF<-0.511364\", \"0.156069\", \"RY_DIFF<-12.0\", \"AVG_STFP_DIFF<-6.480653\", \"-0.250909\", \"0.50506\", \"-0.340704\", \"AVG_STFP_DIFF<5.654113\", \"SRP_DIFF<3.0\", \"0.181395\", \"AVG_STFP_DIFF<-1.286364\"], \"edges\": [[5, 12], [5, 11], [0, 2], [0, 1], [2, 5], [2, 6], [6, 13], [6, 14], [3, 7], [3, 8], [1, 3], [1, 4], [4, 10], [4, 9]], \"ipython\": true, \"error_msg\": \"\"}, e);\n",
        "                });\n",
        "            })();\n",
        "        "
       ],
       "metadata": {},
       "output_type": "display_data"
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "{'confusion_matrix': Columns:\n",
        "\ttarget_label\tint\n",
        "\tpredicted_label\tint\n",
        "\tcount\tint\n",
        "\n",
        "Rows: 4\n",
        "\n",
        "Data:\n",
        "+--------------+-----------------+-------+\n",
        "| target_label | predicted_label | count |\n",
        "+--------------+-----------------+-------+\n",
        "|      1       |        1        |  372  |\n",
        "|      0       |        0        |  280  |\n",
        "|      1       |        0        |   47  |\n",
        "|      0       |        1        |   53  |\n",
        "+--------------+-----------------+-------+\n",
        "[4 rows x 3 columns]\n",
        ", 'accuracy': 0.8670212765957447}\n"
       ]
      }
     ],
     "prompt_number": 55
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Hey! Not bad! "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print btc_classifier.get_feature_importance()\n",
      "important_f  = btc_classifier.get_feature_importance()['feature']"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "+-------------------+-------+\n",
        "|      feature      | count |\n",
        "+-------------------+-------+\n",
        "| YD_PER_P_ATT_DIFF |   15  |\n",
        "|      RY_DIFF      |   14  |\n",
        "|   AVG_STFP_DIFF   |   12  |\n",
        "|      INT_DIFF     |   8   |\n",
        "|      FUM_DIFF     |   6   |\n",
        "|      SRP_DIFF     |   6   |\n",
        "|      PEN_DIFF     |   5   |\n",
        "|      SK_DIFF      |   2   |\n",
        "|      DRV_DIFF     |   1   |\n",
        "|      PU_DIFF      |   1   |\n",
        "+-------------------+-------+\n",
        "[10 rows x 2 columns]\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 59
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Oh man, that looks \u2026 actually pretty good.  My most important features relate to the passing game, field position and ball control.\n",
      "\n",
      "Out of all the features I gave it, the model identified the most important ones.  Just for giggles, I can start making life hard for my model.  Yes, yards per pass attempt appears to be important.  What if I ONLY gave the model that feature?  How accurate am I then?  Thanks to Chris\u2019 suggestion, I can loop through the list of features here, add them one by one and see how accurate I am with just that subset of features."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "models_i = []\n",
      "accuracy_i = []\n",
      "f_imp = []\n",
      "for i in range(1, len(important_f )):\n",
      "    print\n",
      "    btc = gl.boosted_trees_classifier.create(diff_train, target='WIN',\n",
      "                                             features=important_f[:i],\n",
      "                                             max_depth=3)\n",
      "    models_i.append(btc)\n",
      "    accuracy_i.append(btc.evaluate(diff_test))\n",
      "    f_imp.append(btc.get_feature_importance())\n",
      "\n",
      "accurate = [ x['accuracy'] for x in accuracy_i]\n",
      "test_sf = gl.SFrame( {'feature': range(1, len(important_f)), 'accuracy': accurate} )\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "html": [
        "<pre>PROGRESS: Boosted trees classifier:</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Boosted trees classifier:"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of examples          : 2924</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of examples          : 2924"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of classes           : 2</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of classes           : 2"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of feature columns   : 1</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of feature columns   : 1"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of unpacked features : 1</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of unpacked features : 1"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Starting Boosted Trees</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Starting Boosted Trees"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:   Iter      Accuracy          Elapsed time</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:   Iter      Accuracy          Elapsed time"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:         (training) (validation)</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:         (training) (validation)"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      0  7.469e-001  7.279e-001        0.00s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      0  7.469e-001  7.279e-001        0.00s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      1  7.469e-001  7.279e-001        0.00s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      1  7.469e-001  7.279e-001        0.00s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      2  7.469e-001  7.279e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      2  7.469e-001  7.279e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      3  7.469e-001  7.279e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      3  7.469e-001  7.279e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      4  7.469e-001  7.279e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      4  7.469e-001  7.279e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      5  7.500e-001  7.279e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      5  7.500e-001  7.279e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      6  7.500e-001  7.279e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      6  7.500e-001  7.279e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      7  7.500e-001  7.279e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      7  7.500e-001  7.279e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      8  7.500e-001  7.279e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      8  7.500e-001  7.279e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      9  7.517e-001  7.211e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      9  7.517e-001  7.211e-001        0.02s"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n",
        "PROGRESS: Creating a validation set from 5 percent of training data. This may take a while.\n",
        "          You can set ``validation_set=None`` to disable validation tracking.\n",
        "\n",
        "\n"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Boosted trees classifier:</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Boosted trees classifier:"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of examples          : 2921</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of examples          : 2921"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of classes           : 2</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of classes           : 2"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of feature columns   : 2</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of feature columns   : 2"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of unpacked features : 2</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of unpacked features : 2"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Starting Boosted Trees</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Starting Boosted Trees"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:   Iter      Accuracy          Elapsed time</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:   Iter      Accuracy          Elapsed time"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:         (training) (validation)</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:         (training) (validation)"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      0  8.100e-001  8.133e-001        0.00s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      0  8.100e-001  8.133e-001        0.00s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      1  8.069e-001  8.133e-001        0.00s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      1  8.069e-001  8.133e-001        0.00s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      2  8.148e-001  8.133e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      2  8.148e-001  8.133e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      3  8.144e-001  8.133e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      3  8.144e-001  8.133e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      4  8.148e-001  8.133e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      4  8.148e-001  8.133e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      5  8.138e-001  8.133e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      5  8.138e-001  8.133e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      6  8.144e-001  8.133e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      6  8.144e-001  8.133e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      7  8.148e-001  8.133e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      7  8.148e-001  8.133e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      8  8.148e-001  8.133e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      8  8.148e-001  8.133e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      9  8.148e-001  8.133e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      9  8.148e-001  8.133e-001        0.02s"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "PROGRESS: Creating a validation set from 5 percent of training data. This may take a while.\n",
        "          You can set ``validation_set=None`` to disable validation tracking.\n",
        "\n",
        "\n"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Boosted trees classifier:</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Boosted trees classifier:"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of examples          : 2925</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of examples          : 2925"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of classes           : 2</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of classes           : 2"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of feature columns   : 3</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of feature columns   : 3"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of unpacked features : 3</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of unpacked features : 3"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Starting Boosted Trees</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Starting Boosted Trees"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:   Iter      Accuracy          Elapsed time</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:   Iter      Accuracy          Elapsed time"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:         (training) (validation)</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:         (training) (validation)"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      0  8.202e-001  7.671e-001        0.00s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      0  8.202e-001  7.671e-001        0.00s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      1  8.403e-001  7.329e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      1  8.403e-001  7.329e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      2  8.441e-001  7.329e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      2  8.441e-001  7.329e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      3  8.479e-001  7.329e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      3  8.479e-001  7.329e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      4  8.520e-001  7.534e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      4  8.520e-001  7.534e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      5  8.540e-001  7.740e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      5  8.540e-001  7.740e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      6  8.561e-001  7.740e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      6  8.561e-001  7.740e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      7  8.544e-001  7.740e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      7  8.544e-001  7.740e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      8  8.554e-001  7.671e-001        0.03s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      8  8.554e-001  7.671e-001        0.03s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      9  8.568e-001  7.671e-001        0.03s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      9  8.568e-001  7.671e-001        0.03s"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "PROGRESS: Creating a validation set from 5 percent of training data. This may take a while.\n",
        "          You can set ``validation_set=None`` to disable validation tracking.\n",
        "\n",
        "\n"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Boosted trees classifier:</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Boosted trees classifier:"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of examples          : 2940</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of examples          : 2940"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of classes           : 2</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of classes           : 2"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of feature columns   : 4</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of feature columns   : 4"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of unpacked features : 4</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of unpacked features : 4"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Starting Boosted Trees</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Starting Boosted Trees"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:   Iter      Accuracy          Elapsed time</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:   Iter      Accuracy          Elapsed time"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:         (training) (validation)</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:         (training) (validation)"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      0  8.207e-001  8.321e-001        0.00s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      0  8.207e-001  8.321e-001        0.00s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      1  8.364e-001  8.550e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      1  8.364e-001  8.550e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      2  8.497e-001  8.626e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      2  8.497e-001  8.626e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      3  8.514e-001  8.779e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      3  8.514e-001  8.779e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      4  8.544e-001  8.779e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      4  8.544e-001  8.779e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      5  8.619e-001  8.702e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      5  8.619e-001  8.702e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      6  8.636e-001  8.779e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      6  8.636e-001  8.779e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      7  8.653e-001  8.855e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      7  8.653e-001  8.855e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      8  8.673e-001  8.931e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      8  8.673e-001  8.931e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      9  8.670e-001  9.008e-001        0.03s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      9  8.670e-001  9.008e-001        0.03s"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "PROGRESS: Creating a validation set from 5 percent of training data. This may take a while.\n",
        "          You can set ``validation_set=None`` to disable validation tracking.\n",
        "\n",
        "\n"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Boosted trees classifier:</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Boosted trees classifier:"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of examples          : 2919</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of examples          : 2919"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of classes           : 2</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of classes           : 2"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of feature columns   : 5</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of feature columns   : 5"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of unpacked features : 5</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of unpacked features : 5"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Starting Boosted Trees</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Starting Boosted Trees"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:   Iter      Accuracy          Elapsed time</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:   Iter      Accuracy          Elapsed time"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:         (training) (validation)</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:         (training) (validation)"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      0  8.181e-001  8.750e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      0  8.181e-001  8.750e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      1  8.345e-001  8.355e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      1  8.345e-001  8.355e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      2  8.438e-001  8.684e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      2  8.438e-001  8.684e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      3  8.565e-001  8.224e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      3  8.565e-001  8.224e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      4  8.602e-001  8.289e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      4  8.602e-001  8.289e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      5  8.650e-001  8.421e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      5  8.650e-001  8.421e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      6  8.722e-001  8.224e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      6  8.722e-001  8.224e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      7  8.750e-001  8.355e-001        0.03s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      7  8.750e-001  8.355e-001        0.03s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      8  8.777e-001  8.158e-001        0.03s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      8  8.777e-001  8.158e-001        0.03s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      9  8.777e-001  8.158e-001        0.03s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      9  8.777e-001  8.158e-001        0.03s"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "PROGRESS: Creating a validation set from 5 percent of training data. This may take a while.\n",
        "          You can set ``validation_set=None`` to disable validation tracking.\n",
        "\n",
        "\n"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Boosted trees classifier:</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Boosted trees classifier:"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of examples          : 2904</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of examples          : 2904"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of classes           : 2</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of classes           : 2"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of feature columns   : 6</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of feature columns   : 6"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of unpacked features : 6</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of unpacked features : 6"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Starting Boosted Trees</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Starting Boosted Trees"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:   Iter      Accuracy          Elapsed time</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:   Iter      Accuracy          Elapsed time"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:         (training) (validation)</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:         (training) (validation)"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      0  8.340e-001  7.784e-001        0.00s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      0  8.340e-001  7.784e-001        0.00s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      1  8.464e-001  8.204e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      1  8.464e-001  8.204e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      2  8.567e-001  8.084e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      2  8.567e-001  8.084e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      3  8.629e-001  8.024e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      3  8.629e-001  8.024e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      4  8.657e-001  8.204e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      4  8.657e-001  8.204e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      5  8.705e-001  8.144e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      5  8.705e-001  8.144e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      6  8.743e-001  8.263e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      6  8.743e-001  8.263e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      7  8.760e-001  8.263e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      7  8.760e-001  8.263e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      8  8.798e-001  8.323e-001        0.03s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      8  8.798e-001  8.323e-001        0.03s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      9  8.822e-001  8.383e-001        0.03s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      9  8.822e-001  8.383e-001        0.03s"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "PROGRESS: Creating a validation set from 5 percent of training data. This may take a while.\n",
        "          You can set ``validation_set=None`` to disable validation tracking.\n",
        "\n",
        "\n"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Boosted trees classifier:</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Boosted trees classifier:"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of examples          : 2929</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of examples          : 2929"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of classes           : 2</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of classes           : 2"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of feature columns   : 7</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of feature columns   : 7"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of unpacked features : 7</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of unpacked features : 7"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Starting Boosted Trees</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Starting Boosted Trees"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:   Iter      Accuracy          Elapsed time</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:   Iter      Accuracy          Elapsed time"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:         (training) (validation)</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:         (training) (validation)"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      0  8.218e-001  8.380e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      0  8.218e-001  8.380e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      1  8.327e-001  8.521e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      1  8.327e-001  8.521e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      2  8.508e-001  8.592e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      2  8.508e-001  8.592e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      3  8.621e-001  8.662e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      3  8.621e-001  8.662e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      4  8.686e-001  8.803e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      4  8.686e-001  8.803e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      5  8.730e-001  8.944e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      5  8.730e-001  8.944e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      6  8.774e-001  9.014e-001        0.03s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      6  8.774e-001  9.014e-001        0.03s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      7  8.812e-001  9.014e-001        0.03s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      7  8.812e-001  9.014e-001        0.03s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      8  8.829e-001  8.944e-001        0.03s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      8  8.829e-001  8.944e-001        0.03s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      9  8.860e-001  9.085e-001        0.04s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      9  8.860e-001  9.085e-001        0.04s"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "PROGRESS: Creating a validation set from 5 percent of training data. This may take a while.\n",
        "          You can set ``validation_set=None`` to disable validation tracking.\n",
        "\n",
        "\n"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Boosted trees classifier:</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Boosted trees classifier:"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of examples          : 2919</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of examples          : 2919"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of classes           : 2</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of classes           : 2"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of feature columns   : 8</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of feature columns   : 8"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of unpacked features : 8</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of unpacked features : 8"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Starting Boosted Trees</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Starting Boosted Trees"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:   Iter      Accuracy          Elapsed time</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:   Iter      Accuracy          Elapsed time"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:         (training) (validation)</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:         (training) (validation)"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      0  8.232e-001  8.882e-001        0.00s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      0  8.232e-001  8.882e-001        0.00s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      1  8.338e-001  8.684e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      1  8.338e-001  8.684e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      2  8.527e-001  8.947e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      2  8.527e-001  8.947e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      3  8.619e-001  9.013e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      3  8.619e-001  9.013e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      4  8.702e-001  9.079e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      4  8.702e-001  9.079e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      5  8.691e-001  8.947e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      5  8.691e-001  8.947e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      6  8.763e-001  9.013e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      6  8.763e-001  9.013e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      7  8.811e-001  9.079e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      7  8.811e-001  9.079e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      8  8.866e-001  9.079e-001        0.03s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      8  8.866e-001  9.079e-001        0.03s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      9  8.911e-001  9.145e-001        0.03s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      9  8.911e-001  9.145e-001        0.03s"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "PROGRESS: Creating a validation set from 5 percent of training data. This may take a while.\n",
        "          You can set ``validation_set=None`` to disable validation tracking.\n",
        "\n",
        "\n"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Boosted trees classifier:</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Boosted trees classifier:"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of examples          : 2894</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of examples          : 2894"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of classes           : 2</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of classes           : 2"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of feature columns   : 9</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of feature columns   : 9"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Number of unpacked features : 9</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Number of unpacked features : 9"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: Starting Boosted Trees</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: Starting Boosted Trees"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS: --------------------------------------------------------</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS: --------------------------------------------------------"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:   Iter      Accuracy          Elapsed time</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:   Iter      Accuracy          Elapsed time"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:         (training) (validation)</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:         (training) (validation)"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      0  8.286e-001  7.627e-001        0.00s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      0  8.286e-001  7.627e-001        0.00s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      1  8.583e-001  8.192e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      1  8.583e-001  8.192e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      2  8.642e-001  8.475e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      2  8.642e-001  8.475e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      3  8.645e-001  8.475e-001        0.01s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      3  8.645e-001  8.475e-001        0.01s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      4  8.704e-001  8.701e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      4  8.704e-001  8.701e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      5  8.732e-001  8.757e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      5  8.732e-001  8.757e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      6  8.822e-001  8.701e-001        0.02s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      6  8.822e-001  8.701e-001        0.02s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      7  8.901e-001  8.814e-001        0.03s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      7  8.901e-001  8.814e-001        0.03s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      8  8.887e-001  8.757e-001        0.03s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      8  8.887e-001  8.757e-001        0.03s"
       ]
      },
      {
       "html": [
        "<pre>PROGRESS:      9  8.922e-001  8.870e-001        0.03s</pre>"
       ],
       "metadata": {},
       "output_type": "display_data",
       "text": [
        "PROGRESS:      9  8.922e-001  8.870e-001        0.03s"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "PROGRESS: Creating a validation set from 5 percent of training data. This may take a while.\n",
        "          You can set ``validation_set=None`` to disable validation tracking.\n",
        "\n"
       ]
      }
     ],
     "prompt_number": 62
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "test_sf.show(view=\"Line Chart\", x='feature', y='accuracy')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "javascript": [
        "$(\"head\").append($(\"<link/>\").attr({\n",
        "  rel:  \"stylesheet\",\n",
        "  type: \"text/css\",\n",
        "  href: \"//cdnjs.cloudflare.com/ajax/libs/font-awesome/4.1.0/css/font-awesome.min.css\"\n",
        "}));\n",
        "$(\"head\").append($(\"<link/>\").attr({\n",
        "  rel:  \"stylesheet\",\n",
        "  type: \"text/css\",\n",
        "  href: \"//turi.com/files/canvas/1.6.1/css/canvas.css\"\n",
        "}));\n",
        "\n",
        "            (function(){\n",
        "\n",
        "                var e = null;\n",
        "                if (typeof element == 'undefined') {\n",
        "                    var scripts = document.getElementsByTagName('script');\n",
        "                    var thisScriptTag = scripts[scripts.length-1];\n",
        "                    var parentDiv = thisScriptTag.parentNode;\n",
        "                    e = document.createElement('div');\n",
        "                    parentDiv.appendChild(e);\n",
        "                } else {\n",
        "                    e = element[0];\n",
        "                }\n",
        "\n",
        "                if (typeof requirejs !== 'undefined') {\n",
        "                    // disable load timeout; ipython_app.js is large and can take a while to load.\n",
        "                    requirejs.config({waitSeconds: 0});\n",
        "                }\n",
        "\n",
        "                require(['//turi.com/files/canvas/1.6.1/js/ipython_app.js'], function(IPythonApp){\n",
        "                    var app = new IPythonApp();\n",
        "                    app.attachView('sframe','Plots', {\"selected_variable\": {\"name\": [\"test_sf\"], \"descriptives\": {\"rows\": 9, \"columns\": 2}, \"view_component\": \"Plots\", \"view_file\": \"sframe\", \"view_params\": {\"y\": \"accuracy\", \"x\": \"feature\", \"columns\": [\"accuracy\", \"feature\"], \"view\": \"Line Chart\"}, \"view_components\": [\"Summary\", \"Table\", \"Bar Chart\", \"BoxWhisker Plot\", \"Line Chart\", \"Scatter Plot\", \"Heat Map\", \"Plots\"], \"type\": \"SFrame\", \"columns\": [{\"dtype\": \"float\", \"name\": \"accuracy\"}, {\"dtype\": \"int\", \"name\": \"feature\"}], \"column_identifiers\": [\"feature\", \"accuracy\"]}, \"complete\": 1, \"ipython\": true, \"progress\": 1.0, \"data\": [[1, 0.7579787234042553], [2, 0.8138297872340425], [3, 0.8563829787234043], [4, 0.8670212765957447], [5, 0.875], [6, 0.875], [7, 0.8816489361702128], [8, 0.8696808510638298], [9, 0.8723404255319149]], \"columns\": [{\"dtype\": \"float\", \"name\": \"accuracy\"}, {\"dtype\": \"int\", \"name\": \"feature\"}]}, e);\n",
        "                });\n",
        "            })();\n",
        "        "
       ],
       "metadata": {},
       "output_type": "display_data"
      }
     ],
     "prompt_number": 63
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Keeping in mind I\u2019m adding them in by order of importance, it looks like I get 75% accuracy from just the first feature, yards per pass attempt.  I get another big boost by including starting field position, interception and rushing yards\u2026 but after that, the rest of the features really aren\u2019t contributing much.  This sounds about right, from what I\u2019ve read about the current state of the pro football world.\n",
      "\n",
      "And there you have it.  This is what it feels like to get into machine learning.  Lots of \u201cI have no idea what I\u2019m doing\u201d followed by an occasional \u201chuh\u2026 that was pretty cool\u201d.  There\u2019s plenty of additional tuning that can (and should) be done, but the ability to create insights and get moving with the basics quickly make the topic a lot more approachable.  And once I figure out how to use machine learning to beat the odds, I\u2019m going to Vegas. \n",
      "\n",
      "I\u2019ll send Chris a postcard."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": []
    }
   ],
   "metadata": {}
  }
 ]
}