Day_022_HW.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 作業 : (Kaggle)鐵達尼生存預測\n",
    "https://www.kaggle.com/c/titanic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [作業目標]\n",
    "- 試著模仿範例寫法, 在鐵達尼生存預測中, 觀察標籤編碼與獨編碼熱的影響"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [作業重點]\n",
    "- 回答在範例中的觀察結果\n",
    "- 觀察標籤編碼與獨熱編碼, 在特徵數量 / 邏輯斯迴歸分數 / 邏輯斯迴歸時間上, 分別有什麼影響 (In[3], Out[3], In[4], Out[4]) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 作業1\n",
    "* 觀察範例，在房價預測中調整標籤編碼(Label Encoder) / 獨熱編碼 (One Hot Encoder) 方式，  \n",
    "對於線性迴歸以及梯度提升樹兩種模型，何者影響比較大?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Pclass                                               Name     Sex   Age  \\\n",
       "0       3                            Braund, Mr. Owen Harris    male  22.0   \n",
       "1       1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0   \n",
       "2       3                             Heikkinen, Miss. Laina  female  26.0   \n",
       "3       1       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0   \n",
       "4       3                           Allen, Mr. William Henry    male  35.0   \n",
       "\n",
       "   SibSp  Parch            Ticket     Fare Cabin Embarked  \n",
       "0      1      0         A/5 21171   7.2500   NaN        S  \n",
       "1      1      0          PC 17599  71.2833   C85        C  \n",
       "2      0      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "3      1      0            113803  53.1000  C123        S  \n",
       "4      0      0            373450   8.0500   NaN        S  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 做完特徵工程前的所有準備 (與前範例相同)\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import copy, time\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "from sklearn.model_selection import cross_val_score\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "from sklearn.ensemble import GradientBoostingRegressor\n",
    "\n",
    "\n",
    "data_path = 'data/'\n",
    "df_train = pd.read_csv(data_path + 'titanic_train.csv')\n",
    "df_test = pd.read_csv(data_path + 'titanic_test.csv')\n",
    "\n",
    "train_Y = df_train['Survived']\n",
    "ids = df_test['PassengerId']\n",
    "df_train = df_train.drop(['PassengerId', 'Survived'] , axis=1)\n",
    "df_test = df_test.drop(['PassengerId'] , axis=1)\n",
    "df = pd.concat([df_train,df_test])\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5 Numeric Features : ['Name', 'Sex', 'Ticket', 'Cabin', 'Embarked']\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>None</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>None</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>113803</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>373450</td>\n",
       "      <td>None</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                Name     Sex  \\\n",
       "0                            Braund, Mr. Owen Harris    male   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female   \n",
       "2                             Heikkinen, Miss. Laina  female   \n",
       "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female   \n",
       "4                           Allen, Mr. William Henry    male   \n",
       "\n",
       "             Ticket Cabin Embarked  \n",
       "0         A/5 21171  None        S  \n",
       "1          PC 17599   C85        C  \n",
       "2  STON/O2. 3101282  None        S  \n",
       "3            113803  C123        S  \n",
       "4            373450  None        S  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#只取類別值 (object) 型欄位, 存於 object_features 中\n",
    "object_features = []\n",
    "for dtype, feature in zip(df.dtypes, df.columns):\n",
    "    if dtype == 'object':\n",
    "        object_features.append(feature)\n",
    "print(f'{len(object_features)} Numeric Features : {object_features}\\n')\n",
    "\n",
    "# 只留類別型欄位\n",
    "df = df[object_features]\n",
    "df = df.fillna('None')\n",
    "train_num = train_Y.shape[0]\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 作業2\n",
    "* 鐵達尼號例題中，標籤編碼 / 獨熱編碼又分別對預測結果有何影響? (Hint : 參考今日範例)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "shape : (891, 5)\n",
      "score : 0.7789000729487724\n",
      "time : 0.13746213912963867 sec\n"
     ]
    }
   ],
   "source": [
    "# 標籤編碼 + 羅吉斯迴歸\n",
    "\n",
    "df_temp = pd.DataFrame()\n",
    "for c in df.columns:\n",
    "    df_temp[c] = LabelEncoder().fit_transform(df[c])\n",
    "train_X = df_temp[:train_num]\n",
    "estimator = LogisticRegression(solver = \"lbfgs\")\n",
    "start = time.time()\n",
    "print(f'shape : {train_X.shape}')\n",
    "print(f'score : {cross_val_score(estimator, train_X, train_Y, cv=5).mean()}')\n",
    "print(f'time : {time.time() - start} sec')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "shape : (891, 2429)\n",
      "score : 0.8013346043513216\n",
      "time : 0.3659799098968506 sec\n"
     ]
    }
   ],
   "source": [
    "# 獨熱編碼 + 羅吉斯迴歸\n",
    "\n",
    "df_temp = pd.get_dummies(df)\n",
    "train_X = df_temp[:train_num]\n",
    "estimator = LogisticRegression(solver = \"lbfgs\")\n",
    "start = time.time()\n",
    "print(f'shape : {train_X.shape}')\n",
    "print(f'score : {cross_val_score(estimator, train_X, train_Y, cv=5).mean()}')\n",
    "print(f'time : {time.time() - start} sec')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "shape : (891, 5)\n",
      "score : 0.3771748103412774\n",
      "time : 0.19727587699890137 sec\n"
     ]
    }
   ],
   "source": [
    "# 標籤編碼 + 梯度提升樹\n",
    "\n",
    "df_temp = pd.DataFrame()\n",
    "for c in df.columns:\n",
    "    df_temp[c] = LabelEncoder().fit_transform(df[c])\n",
    "train_X = df_temp[:train_num]\n",
    "estimator = GradientBoostingRegressor()\n",
    "start = time.time()\n",
    "print(f'shape : {train_X.shape}')\n",
    "print(f'score : {cross_val_score(estimator, train_X, train_Y, cv=5).mean()}')\n",
    "print(f'time : {time.time() - start} sec')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "shape : (891, 2429)\n",
      "score : 0.3722152186918969\n",
      "time : 5.2788331508636475 sec\n"
     ]
    }
   ],
   "source": [
    "# 獨熱編碼 + 梯度提升樹\n",
    "\n",
    "df_temp = pd.get_dummies(df)\n",
    "train_X = df_temp[:train_num]\n",
    "estimator = GradientBoostingRegressor()\n",
    "start = time.time()\n",
    "print(f'shape : {train_X.shape}')\n",
    "print(f'score : {cross_val_score(estimator, train_X, train_Y, cv=5).mean()}')\n",
    "print(f'time : {time.time() - start} sec')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}