14_imbalanced/handling_imbalanced_data.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3 style='color:blue' align='center'>Handling imbalanced data in customer churn prediction</h3>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Customer churn prediction is to measure why customers are leaving a business. In this tutorial we will be looking at customer churn in telecom business. We will build a deep learning model to predict the churn and use precision,recall, f1-score to measure performance of our model.\n",
    "We will then handle imbalance in data using various techniques and improve f1-score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from matplotlib import pyplot as plt\n",
    "import numpy as np\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Load the data**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>customerID</th>\n",
       "      <th>gender</th>\n",
       "      <th>SeniorCitizen</th>\n",
       "      <th>Partner</th>\n",
       "      <th>Dependents</th>\n",
       "      <th>tenure</th>\n",
       "      <th>PhoneService</th>\n",
       "      <th>MultipleLines</th>\n",
       "      <th>InternetService</th>\n",
       "      <th>OnlineSecurity</th>\n",
       "      <th>...</th>\n",
       "      <th>DeviceProtection</th>\n",
       "      <th>TechSupport</th>\n",
       "      <th>StreamingTV</th>\n",
       "      <th>StreamingMovies</th>\n",
       "      <th>Contract</th>\n",
       "      <th>PaperlessBilling</th>\n",
       "      <th>PaymentMethod</th>\n",
       "      <th>MonthlyCharges</th>\n",
       "      <th>TotalCharges</th>\n",
       "      <th>Churn</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>6264</th>\n",
       "      <td>2578-JQPHZ</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>44</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>Fiber optic</td>\n",
       "      <td>No</td>\n",
       "      <td>...</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Two year</td>\n",
       "      <td>No</td>\n",
       "      <td>Bank transfer (automatic)</td>\n",
       "      <td>100.10</td>\n",
       "      <td>4378.35</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2679</th>\n",
       "      <td>7341-LXCAF</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>4</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Fiber optic</td>\n",
       "      <td>No</td>\n",
       "      <td>...</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>Month-to-month</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Electronic check</td>\n",
       "      <td>74.65</td>\n",
       "      <td>301.4</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1328</th>\n",
       "      <td>6475-VHUIZ</td>\n",
       "      <td>Female</td>\n",
       "      <td>0</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>23</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>DSL</td>\n",
       "      <td>No</td>\n",
       "      <td>...</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>Month-to-month</td>\n",
       "      <td>No</td>\n",
       "      <td>Electronic check</td>\n",
       "      <td>54.25</td>\n",
       "      <td>1221.55</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6773</th>\n",
       "      <td>3990-QYKBE</td>\n",
       "      <td>Male</td>\n",
       "      <td>1</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>37</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Fiber optic</td>\n",
       "      <td>No</td>\n",
       "      <td>...</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Month-to-month</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Electronic check</td>\n",
       "      <td>92.50</td>\n",
       "      <td>3473.4</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>616</th>\n",
       "      <td>6464-KEXXH</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>8</td>\n",
       "      <td>No</td>\n",
       "      <td>No phone service</td>\n",
       "      <td>DSL</td>\n",
       "      <td>No</td>\n",
       "      <td>...</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>Month-to-month</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Credit card (automatic)</td>\n",
       "      <td>25.25</td>\n",
       "      <td>186.3</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 21 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      customerID  gender  SeniorCitizen Partner Dependents  tenure  \\\n",
       "6264  2578-JQPHZ    Male              0      No         No      44   \n",
       "2679  7341-LXCAF    Male              0     Yes         No       4   \n",
       "1328  6475-VHUIZ  Female              0     Yes         No      23   \n",
       "6773  3990-QYKBE    Male              1     Yes         No      37   \n",
       "616   6464-KEXXH    Male              0      No         No       8   \n",
       "\n",
       "     PhoneService     MultipleLines InternetService OnlineSecurity  ...  \\\n",
       "6264          Yes                No     Fiber optic             No  ...   \n",
       "2679          Yes               Yes     Fiber optic             No  ...   \n",
       "1328          Yes                No             DSL             No  ...   \n",
       "6773          Yes               Yes     Fiber optic             No  ...   \n",
       "616            No  No phone service             DSL             No  ...   \n",
       "\n",
       "     DeviceProtection TechSupport StreamingTV StreamingMovies        Contract  \\\n",
       "6264              Yes         Yes         Yes             Yes        Two year   \n",
       "2679               No          No          No              No  Month-to-month   \n",
       "1328              Yes         Yes          No              No  Month-to-month   \n",
       "6773               No          No         Yes             Yes  Month-to-month   \n",
       "616                No          No          No              No  Month-to-month   \n",
       "\n",
       "     PaperlessBilling              PaymentMethod MonthlyCharges  TotalCharges  \\\n",
       "6264               No  Bank transfer (automatic)         100.10       4378.35   \n",
       "2679              Yes           Electronic check          74.65         301.4   \n",
       "1328               No           Electronic check          54.25       1221.55   \n",
       "6773              Yes           Electronic check          92.50        3473.4   \n",
       "616               Yes    Credit card (automatic)          25.25         186.3   \n",
       "\n",
       "     Churn  \n",
       "6264    No  \n",
       "2679   Yes  \n",
       "1328    No  \n",
       "6773   Yes  \n",
       "616     No  \n",
       "\n",
       "[5 rows x 21 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(\"customer_churn.csv\")\n",
    "df.sample(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "No     5174\n",
       "Yes    1869\n",
       "Name: Churn, dtype: int64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.Churn.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "73.46301292063042"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "517400/df.shape[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**First of all, drop customerID column as it is of no use**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.drop('customerID',axis='columns',inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "gender               object\n",
       "SeniorCitizen         int64\n",
       "Partner              object\n",
       "Dependents           object\n",
       "tenure                int64\n",
       "PhoneService         object\n",
       "MultipleLines        object\n",
       "InternetService      object\n",
       "OnlineSecurity       object\n",
       "OnlineBackup         object\n",
       "DeviceProtection     object\n",
       "TechSupport          object\n",
       "StreamingTV          object\n",
       "StreamingMovies      object\n",
       "Contract             object\n",
       "PaperlessBilling     object\n",
       "PaymentMethod        object\n",
       "MonthlyCharges      float64\n",
       "TotalCharges         object\n",
       "Churn                object\n",
       "dtype: object"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Quick glance at above makes me realize that TotalCharges should be float but it is an object. Let's check what's going on with  this column**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['29.85', '1889.5', '108.15', ..., '346.45', '306.6', '6844.5'],\n",
       "      dtype=object)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.TotalCharges.values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Ahh... it is string. Lets convert it to numbers**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       False\n",
       "1       False\n",
       "2       False\n",
       "3       False\n",
       "4       False\n",
       "        ...  \n",
       "7038    False\n",
       "7039    False\n",
       "7040    False\n",
       "7041    False\n",
       "7042    False\n",
       "Name: TotalCharges, Length: 7043, dtype: bool"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.to_numeric(df.TotalCharges,errors='coerce').isnull()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>gender</th>\n",
       "      <th>SeniorCitizen</th>\n",
       "      <th>Partner</th>\n",
       "      <th>Dependents</th>\n",
       "      <th>tenure</th>\n",
       "      <th>PhoneService</th>\n",
       "      <th>MultipleLines</th>\n",
       "      <th>InternetService</th>\n",
       "      <th>OnlineSecurity</th>\n",
       "      <th>OnlineBackup</th>\n",
       "      <th>DeviceProtection</th>\n",
       "      <th>TechSupport</th>\n",
       "      <th>StreamingTV</th>\n",
       "      <th>StreamingMovies</th>\n",
       "      <th>Contract</th>\n",
       "      <th>PaperlessBilling</th>\n",
       "      <th>PaymentMethod</th>\n",
       "      <th>MonthlyCharges</th>\n",
       "      <th>TotalCharges</th>\n",
       "      <th>Churn</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>488</th>\n",
       "      <td>Female</td>\n",
       "      <td>0</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>0</td>\n",
       "      <td>No</td>\n",
       "      <td>No phone service</td>\n",
       "      <td>DSL</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>Two year</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Bank transfer (automatic)</td>\n",
       "      <td>52.55</td>\n",
       "      <td></td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>753</th>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>0</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>Two year</td>\n",
       "      <td>No</td>\n",
       "      <td>Mailed check</td>\n",
       "      <td>20.25</td>\n",
       "      <td></td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>936</th>\n",
       "      <td>Female</td>\n",
       "      <td>0</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>0</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>DSL</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Two year</td>\n",
       "      <td>No</td>\n",
       "      <td>Mailed check</td>\n",
       "      <td>80.85</td>\n",
       "      <td></td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1082</th>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>0</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>Two year</td>\n",
       "      <td>No</td>\n",
       "      <td>Mailed check</td>\n",
       "      <td>25.75</td>\n",
       "      <td></td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1340</th>\n",
       "      <td>Female</td>\n",
       "      <td>0</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>0</td>\n",
       "      <td>No</td>\n",
       "      <td>No phone service</td>\n",
       "      <td>DSL</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>Two year</td>\n",
       "      <td>No</td>\n",
       "      <td>Credit card (automatic)</td>\n",
       "      <td>56.05</td>\n",
       "      <td></td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3331</th>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>0</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>Two year</td>\n",
       "      <td>No</td>\n",
       "      <td>Mailed check</td>\n",
       "      <td>19.85</td>\n",
       "      <td></td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3826</th>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>0</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>Two year</td>\n",
       "      <td>No</td>\n",
       "      <td>Mailed check</td>\n",
       "      <td>25.35</td>\n",
       "      <td></td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4380</th>\n",
       "      <td>Female</td>\n",
       "      <td>0</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>0</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>Two year</td>\n",
       "      <td>No</td>\n",
       "      <td>Mailed check</td>\n",
       "      <td>20.00</td>\n",
       "      <td></td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5218</th>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>0</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>One year</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Mailed check</td>\n",
       "      <td>19.70</td>\n",
       "      <td></td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6670</th>\n",
       "      <td>Female</td>\n",
       "      <td>0</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>0</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>DSL</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>Two year</td>\n",
       "      <td>No</td>\n",
       "      <td>Mailed check</td>\n",
       "      <td>73.35</td>\n",
       "      <td></td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6754</th>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>0</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>DSL</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>Two year</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Bank transfer (automatic)</td>\n",
       "      <td>61.90</td>\n",
       "      <td></td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      gender  SeniorCitizen Partner Dependents  tenure PhoneService  \\\n",
       "488   Female              0     Yes        Yes       0           No   \n",
       "753     Male              0      No        Yes       0          Yes   \n",
       "936   Female              0     Yes        Yes       0          Yes   \n",
       "1082    Male              0     Yes        Yes       0          Yes   \n",
       "1340  Female              0     Yes        Yes       0           No   \n",
       "3331    Male              0     Yes        Yes       0          Yes   \n",
       "3826    Male              0     Yes        Yes       0          Yes   \n",
       "4380  Female              0     Yes        Yes       0          Yes   \n",
       "5218    Male              0     Yes        Yes       0          Yes   \n",
       "6670  Female              0     Yes        Yes       0          Yes   \n",
       "6754    Male              0      No        Yes       0          Yes   \n",
       "\n",
       "         MultipleLines InternetService       OnlineSecurity  \\\n",
       "488   No phone service             DSL                  Yes   \n",
       "753                 No              No  No internet service   \n",
       "936                 No             DSL                  Yes   \n",
       "1082               Yes              No  No internet service   \n",
       "1340  No phone service             DSL                  Yes   \n",
       "3331                No              No  No internet service   \n",
       "3826               Yes              No  No internet service   \n",
       "4380                No              No  No internet service   \n",
       "5218                No              No  No internet service   \n",
       "6670               Yes             DSL                   No   \n",
       "6754               Yes             DSL                  Yes   \n",
       "\n",
       "             OnlineBackup     DeviceProtection          TechSupport  \\\n",
       "488                    No                  Yes                  Yes   \n",
       "753   No internet service  No internet service  No internet service   \n",
       "936                   Yes                  Yes                   No   \n",
       "1082  No internet service  No internet service  No internet service   \n",
       "1340                  Yes                  Yes                  Yes   \n",
       "3331  No internet service  No internet service  No internet service   \n",
       "3826  No internet service  No internet service  No internet service   \n",
       "4380  No internet service  No internet service  No internet service   \n",
       "5218  No internet service  No internet service  No internet service   \n",
       "6670                  Yes                  Yes                  Yes   \n",
       "6754                  Yes                   No                  Yes   \n",
       "\n",
       "              StreamingTV      StreamingMovies  Contract PaperlessBilling  \\\n",
       "488                   Yes                   No  Two year              Yes   \n",
       "753   No internet service  No internet service  Two year               No   \n",
       "936                   Yes                  Yes  Two year               No   \n",
       "1082  No internet service  No internet service  Two year               No   \n",
       "1340                  Yes                   No  Two year               No   \n",
       "3331  No internet service  No internet service  Two year               No   \n",
       "3826  No internet service  No internet service  Two year               No   \n",
       "4380  No internet service  No internet service  Two year               No   \n",
       "5218  No internet service  No internet service  One year              Yes   \n",
       "6670                  Yes                   No  Two year               No   \n",
       "6754                   No                   No  Two year              Yes   \n",
       "\n",
       "                  PaymentMethod  MonthlyCharges TotalCharges Churn  \n",
       "488   Bank transfer (automatic)           52.55                 No  \n",
       "753                Mailed check           20.25                 No  \n",
       "936                Mailed check           80.85                 No  \n",
       "1082               Mailed check           25.75                 No  \n",
       "1340    Credit card (automatic)           56.05                 No  \n",
       "3331               Mailed check           19.85                 No  \n",
       "3826               Mailed check           25.35                 No  \n",
       "4380               Mailed check           20.00                 No  \n",
       "5218               Mailed check           19.70                 No  \n",
       "6670               Mailed check           73.35                 No  \n",
       "6754  Bank transfer (automatic)           61.90                 No  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[pd.to_numeric(df.TotalCharges,errors='coerce').isnull()]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(7043, 20)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "' '"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.iloc[488].TotalCharges"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(7032, 20)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[df.TotalCharges!=' '].shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Remove rows with space in TotalCharges**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(7032, 20)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1 = df[df.TotalCharges!=' ']\n",
    "df1.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "gender               object\n",
       "SeniorCitizen         int64\n",
       "Partner              object\n",
       "Dependents           object\n",
       "tenure                int64\n",
       "PhoneService         object\n",
       "MultipleLines        object\n",
       "InternetService      object\n",
       "OnlineSecurity       object\n",
       "OnlineBackup         object\n",
       "DeviceProtection     object\n",
       "TechSupport          object\n",
       "StreamingTV          object\n",
       "StreamingMovies      object\n",
       "Contract             object\n",
       "PaperlessBilling     object\n",
       "PaymentMethod        object\n",
       "MonthlyCharges      float64\n",
       "TotalCharges         object\n",
       "Churn                object\n",
       "dtype: object"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "df1.TotalCharges = pd.to_numeric(df1.TotalCharges)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  29.85, 1889.5 ,  108.15, ...,  346.45,  306.6 , 6844.5 ])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1.TotalCharges.values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>gender</th>\n",
       "      <th>SeniorCitizen</th>\n",
       "      <th>Partner</th>\n",
       "      <th>Dependents</th>\n",
       "      <th>tenure</th>\n",
       "      <th>PhoneService</th>\n",
       "      <th>MultipleLines</th>\n",
       "      <th>InternetService</th>\n",
       "      <th>OnlineSecurity</th>\n",
       "      <th>OnlineBackup</th>\n",
       "      <th>DeviceProtection</th>\n",
       "      <th>TechSupport</th>\n",
       "      <th>StreamingTV</th>\n",
       "      <th>StreamingMovies</th>\n",
       "      <th>Contract</th>\n",
       "      <th>PaperlessBilling</th>\n",
       "      <th>PaymentMethod</th>\n",
       "      <th>MonthlyCharges</th>\n",
       "      <th>TotalCharges</th>\n",
       "      <th>Churn</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Female</td>\n",
       "      <td>0</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>1</td>\n",
       "      <td>No</td>\n",
       "      <td>No phone service</td>\n",
       "      <td>DSL</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>Month-to-month</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Electronic check</td>\n",
       "      <td>29.85</td>\n",
       "      <td>29.85</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>34</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>DSL</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>One year</td>\n",
       "      <td>No</td>\n",
       "      <td>Mailed check</td>\n",
       "      <td>56.95</td>\n",
       "      <td>1889.50</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>45</td>\n",
       "      <td>No</td>\n",
       "      <td>No phone service</td>\n",
       "      <td>DSL</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>One year</td>\n",
       "      <td>No</td>\n",
       "      <td>Bank transfer (automatic)</td>\n",
       "      <td>42.30</td>\n",
       "      <td>1840.75</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>22</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Fiber optic</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>Month-to-month</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Credit card (automatic)</td>\n",
       "      <td>89.10</td>\n",
       "      <td>1949.40</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Female</td>\n",
       "      <td>0</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>10</td>\n",
       "      <td>No</td>\n",
       "      <td>No phone service</td>\n",
       "      <td>DSL</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>Month-to-month</td>\n",
       "      <td>No</td>\n",
       "      <td>Mailed check</td>\n",
       "      <td>29.75</td>\n",
       "      <td>301.90</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7037</th>\n",
       "      <td>Female</td>\n",
       "      <td>0</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>72</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>No internet service</td>\n",
       "      <td>Two year</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Bank transfer (automatic)</td>\n",
       "      <td>21.15</td>\n",
       "      <td>1419.40</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7038</th>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>24</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>DSL</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>One year</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Mailed check</td>\n",
       "      <td>84.80</td>\n",
       "      <td>1990.50</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7039</th>\n",
       "      <td>Female</td>\n",
       "      <td>0</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>72</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Fiber optic</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>One year</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Credit card (automatic)</td>\n",
       "      <td>103.20</td>\n",
       "      <td>7362.90</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7040</th>\n",
       "      <td>Female</td>\n",
       "      <td>0</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>11</td>\n",
       "      <td>No</td>\n",
       "      <td>No phone service</td>\n",
       "      <td>DSL</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>Month-to-month</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Electronic check</td>\n",
       "      <td>29.60</td>\n",
       "      <td>346.45</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7042</th>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>66</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>Fiber optic</td>\n",
       "      <td>Yes</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Two year</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Bank transfer (automatic)</td>\n",
       "      <td>105.65</td>\n",
       "      <td>6844.50</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5163 rows × 20 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      gender  SeniorCitizen Partner Dependents  tenure PhoneService  \\\n",
       "0     Female              0     Yes         No       1           No   \n",
       "1       Male              0      No         No      34          Yes   \n",
       "3       Male              0      No         No      45           No   \n",
       "6       Male              0      No        Yes      22          Yes   \n",
       "7     Female              0      No         No      10           No   \n",
       "...      ...            ...     ...        ...     ...          ...   \n",
       "7037  Female              0      No         No      72          Yes   \n",
       "7038    Male              0     Yes        Yes      24          Yes   \n",
       "7039  Female              0     Yes        Yes      72          Yes   \n",
       "7040  Female              0     Yes        Yes      11           No   \n",
       "7042    Male              0      No         No      66          Yes   \n",
       "\n",
       "         MultipleLines InternetService       OnlineSecurity  \\\n",
       "0     No phone service             DSL                   No   \n",
       "1                   No             DSL                  Yes   \n",
       "3     No phone service             DSL                  Yes   \n",
       "6                  Yes     Fiber optic                   No   \n",
       "7     No phone service             DSL                  Yes   \n",
       "...                ...             ...                  ...   \n",
       "7037                No              No  No internet service   \n",
       "7038               Yes             DSL                  Yes   \n",
       "7039               Yes     Fiber optic                   No   \n",
       "7040  No phone service             DSL                  Yes   \n",
       "7042                No     Fiber optic                  Yes   \n",
       "\n",
       "             OnlineBackup     DeviceProtection          TechSupport  \\\n",
       "0                     Yes                   No                   No   \n",
       "1                      No                  Yes                   No   \n",
       "3                      No                  Yes                  Yes   \n",
       "6                     Yes                   No                   No   \n",
       "7                      No                   No                   No   \n",
       "...                   ...                  ...                  ...   \n",
       "7037  No internet service  No internet service  No internet service   \n",
       "7038                   No                  Yes                  Yes   \n",
       "7039                  Yes                  Yes                   No   \n",
       "7040                   No                   No                   No   \n",
       "7042                   No                  Yes                  Yes   \n",
       "\n",
       "              StreamingTV      StreamingMovies        Contract  \\\n",
       "0                      No                   No  Month-to-month   \n",
       "1                      No                   No        One year   \n",
       "3                      No                   No        One year   \n",
       "6                     Yes                   No  Month-to-month   \n",
       "7                      No                   No  Month-to-month   \n",
       "...                   ...                  ...             ...   \n",
       "7037  No internet service  No internet service        Two year   \n",
       "7038                  Yes                  Yes        One year   \n",
       "7039                  Yes                  Yes        One year   \n",
       "7040                   No                   No  Month-to-month   \n",
       "7042                  Yes                  Yes        Two year   \n",
       "\n",
       "     PaperlessBilling              PaymentMethod  MonthlyCharges  \\\n",
       "0                 Yes           Electronic check           29.85   \n",
       "1                  No               Mailed check           56.95   \n",
       "3                  No  Bank transfer (automatic)           42.30   \n",
       "6                 Yes    Credit card (automatic)           89.10   \n",
       "7                  No               Mailed check           29.75   \n",
       "...               ...                        ...             ...   \n",
       "7037              Yes  Bank transfer (automatic)           21.15   \n",
       "7038              Yes               Mailed check           84.80   \n",
       "7039              Yes    Credit card (automatic)          103.20   \n",
       "7040              Yes           Electronic check           29.60   \n",
       "7042              Yes  Bank transfer (automatic)          105.65   \n",
       "\n",
       "      TotalCharges Churn  \n",
       "0            29.85    No  \n",
       "1          1889.50    No  \n",
       "3          1840.75    No  \n",
       "6          1949.40    No  \n",
       "7           301.90    No  \n",
       "...            ...   ...  \n",
       "7037       1419.40    No  \n",
       "7038       1990.50    No  \n",
       "7039       7362.90    No  \n",
       "7040        346.45    No  \n",
       "7042       6844.50    No  \n",
       "\n",
       "[5163 rows x 20 columns]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1[df1.Churn=='No']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Data Visualization**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x25057e46190>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEWCAYAAACXGLsWAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAAlDUlEQVR4nO3debxd873/8ddbnCZBNGYZcNAUiSGIqBqqiRtqCKp6o4Ygv+rgunLbKqFX4hpbSrlVt2mpFEWo6VLVyEVRlyREJcgVBIfIpIgpbZLP74/1PbFzcs7e+wx7OGe/n4/Hfpy1vnsNn732Pvuzv9/vWt+liMDMzCyftSodgJmZVT8nCzMzK8jJwszMCnKyMDOzgpwszMysICcLMzMryMnCOi1J9ZJC0tqVjqWjSTpR0mM58x9I2qYN2zlW0p86Nrqi9ruvpDkl3kdHHaP/kvTvHRtd1+Nk0UlI+oak6ekfYr6k+yXt085tTpB0Y0fFWAqSPi/pNkmLJb0n6a+SviepWxXEdr2kv6f35B1JUyRtX4p9RcR6EfFKgXjWSJ4RcVNEjOjIWCT1k7Rc0rbNPHenpMsi4tGI2K4j91tIkcdotQST1vt2RJxf2ug6PyeLTkDS94CfARcBmwFbAr8ADq9gWB2qudpB+jJ6EngD2CkiPgscDQwBepV6/0X6SUSsB/QHFgLXN7NtSeoy/2sR8SYwFTg+t1zShsDBwKRKxGUlFhF+VPED+CzwAXB0nmWuBy7Imd8faMiZPxN4E1gKzAGGAwcBfwf+kbb/bFq2L3AP8A4wF/hmznYmALcBN6ZtPQd8HhhH9kX5BjCiSezXAvPT/i8AuqXnTgQeB65I+7qgmdd1I3BfntddDwQwGngdWAyc04rjMi8dm78Cy4DP5dteEcf9EOCDNP0wcGF6jR+nbW8PTEmvdw7w9Zx1N0rH/X3gKeB84LGc5wP4XJruCfwUeA14D3gslb2elvsgPfZKxzl3O18EpqX1pgFfzHnu4bTfx9P7+ydg4xZe+zeAl5uUfRd4utjPYJHv0VnAy2m954Ejc55r+toiHee+OcfgA+Cj9NwOwCfAilT+bgsxfJPss/9Oek/6NtnHt4GXgL8BVwOq9PdEOR5d5tdOF7YX0AO4sy0rS9oO+Bdgj4joBRwIzIuIP5LVVG6NrPq+S1rlZqCB7B/ua8BFkobnbPIw4AZgA+AZ4AGyGmo/4D+AX+YsOwlYTvYPvCswAvh/Oc/vCbwCbEr2xdrUAcDtRbzMfYDtyJLguZJ2KGKdRseQfcn3TrG2aXuS1gOOJTsmjY4HTiGrBS0iSxS/I3u9xwC/kDQoLXs12RdZH+Dk9GjJZcDuZF/8GwI/BFYC+6Xne6f39IkmMW4I3AdcRZacLgfuk7RRzmLfAE5KMX4G+EELMdwJbNykKfR44LdNF2zpM5jn9eV6GdiX7IfHecCNkvrkWyEi3kqvf73Ian13ArdExAtkX/RPpOd6NxPrMOBi4Otk78VrwC1NFjsU2APYJS13YJGvpVNzsqh+GwGLI2J5wSWbtwLoDgyUVBcR8yLi5eYWlLQF2RflmRHxSUTMBH7N6s0Nj0bEAyme24BNgEsi4h9k/1T1knpL2gz4CjA2Ij6MiIVktYhROdt6KyL+MyKWR8THLbz2+UW8xvMi4uOIeBZ4luyfuFhXRcQbTfbfmu39QNK7ZL9E1yP7tdvo+oiYnY7VQWRJ+jfp9T4N/B74Wup/OQo4Nx2rWbTQlJOas04GTo+INyNiRUT8JSKWFfFaDwFeiogbUgw3Ay+S/QBo9JuI+L90PCYDg5vbUHr+NuCEFNcAsgT2u2YWL/oz2Mx+bktf/isj4layX/RDi1k3xXUmWY0uX/LNdSxwXUQ8nY7pOGAvSfU5y1wSEe9GxOvAQ7RwjLoaJ4vqt4TsF1yb2tQjYi4wlqwJaaGkWyT1bWHxvsA7EbE0p+w1slpDowU50x+TJbIVOfOQfWluBdQB8yW9m75Qf0n2i7XRGwXCX0L2666Qt3OmP0r7L1ZzMbRme5dFRO+I2DwiRjb5Eszd9lbAno3HIh2PY4HNyRLu2k2Wf62F/W1MVtMs6su2ib7NbLfp+9ua1z4J+LqkHmQ/KP6YfhSsppWfwdVIOkHSzJxjtiPZMShm3a8ApwNHtPBjpDmrHaOI+IDsc9jWY9RlOFlUvyfImieOyLPMh8A6OfOb5z4ZEb+LiH3IvrAC+HHjU0228xawoaTczuMtydqaW+sNsn6AjdOXae+IWD8iBuUsU2jI4wfJfnG3Vd7jUmQM7ZG77TeAR3KORWNT0XfImqiWA1vkLL9lC9tcTPZ5WONMJAq/lrfIPgO52vr+EhGPkn2RHg4cRzNNUDnLtvQZbPE9krQV8CuyJqyNUrPRLECFYktNX5PI+oVyk3CrjpGkdclquG06Rl2Jk0WVi4j3gHOBqyUdIWkdSXWSviLpJ2mxmcDBkjaUtDnZrzgg+6eRNExSd7IvmY/JmgUgqyXUN56pk/6p/gJcLKmHpJ2BMcBNbYh7PlkH6U8lrS9pLUnbSvpSKzYzHviipEvT60LS5yTdKKl3EevPpIXjUgH3Ap+XdHx6/+ok7SFph1QzuwOYkN7fgWSd7GuIiJXAdcDlkvpK6iZpr/T+LiLru2jpWoM/pBi+IWltSf8MDEyxtdVvyb74ewP/3dwCBT6DM2n5PVqX7Mt9UdrOSWQ1i7wkrQ/cDfwoIh5r8vQCoL+kz7Sw+u+AkyQNTvFeBDwZEfMK7berc7LoBCLicuB7wI/I/nHeIPu1dVda5AaytvV5ZF/Qt+as3h24hOwX6dtkzUBnp+duS3+XSHo6TR9DdpbRW2Qdg+MjYkobQz+BrJP0ebIzR26nuGYlAFKTzl4pntmS3iNr559OdnZMIfmOS1mlpr0RZH02b5G9Fz8me38gez/XS+XXA7/Js7kfkJ2JNo3sjJ0fA2tFxEekM7BSs80XmsSwhKxz9vtkNYIfAodGxOJ2vLTfktVObs3Tb5LvM9jiexQRz5Od9fUE2Zf8TmRnahWyG9kJCpena2A+kPRBeu5/gNnA25LWeN0RMRX4d7LP2XyyGtyopsvVIkX45kdmZpafaxZmZlaQk4WZmRXkZGFmZgU5WZiZWUFdbmjnRhtvvHHU19dXOgwzs05lxowZiyNik6blXTZZ1NfXM3369EqHYWbWqUhqdvQAN0OZmVlBThZmZlaQk4WZmRXUZfssmvOPf/yDhoYGPvnkk0qH0mX16NGD/v37U1dXV+lQzKwD1VSyaGhooFevXtTX1yMVHLjSWikiWLJkCQ0NDWy99daVDsfMOlBNNUN98sknbLTRRk4UJSKJjTbayDU3sy6oppIF4ERRYj6+Zl1TzSULMzNrvZrqs2hK53Xsr+AYX3i497fffpuxY8cybdo0unfvTn19PUcccQT33HMP997bnnvQtM3EiROZOnUqt96a3Ubg/fffZ9ddd+XBBx90v4OZreKaRRlFBEceeST7778/L7/8Ms8//zwXXXQRCxYsKLxyHsuXL2/zut/85jdpaGjgwQcfBODcc8/l5JNPdqIwqzSpbY8ScbIoo4ceeoi6ujq+/e1vryobPHgw++67Lx988AFf+9rX2H777Tn22GNpvClVfX09ixdnN/SaPn06+++/PwATJkzglFNOYcSIEZxwwglMmDCBk08+mf33359tttmGq666qqiYJHHNNdcwduxYpk+fztSpUznjjDO48cYbGTp0KIMHD+Zb3/oWK1asYMWKFZx44onsuOOO7LTTTlxxxRUde4DMrGrVdDNUuc2aNYvdd9+92eeeeeYZZs+eTd++fdl77715/PHH2WefffJub8aMGTz22GP07NmTCRMm8OKLL/LQQw+xdOlStttuO77zne9QV1fHvvvuy9Kla96F9LLLLuOAAw5g55135sADD2T48OHcddddvPzyy9x66608/vjj1NXV8d3vfpebbrqJQYMG8eabbzJr1iwA3n333XYfEzPrHJwsqsTQoUPp378/kNU25s2bVzBZjBw5kp49e66aP+SQQ+jevTvdu3dn0003ZcGCBfTv359HH3204P5PPfVU7r//fr785S/z85//nBkzZrDHHnsA8PHHH7Ppppty2GGH8corr3DaaadxyCGHMGLEiHa8YjPrTEqWLCRdR3Zz+IURsWMq25Dshuz1ZDdo/3pE/C09Nw4YA6wA/jUiHkjlu5PdwL4n8Afg9OikNw4fNGgQt99+e7PPde/efdV0t27dVvVDrL322qxcuRJgjesX1l133aK2UahmAbDWWmux1lpZq2REMHr0aC6++OI11nn22Wd54IEHuPrqq5k8eTLXXXdd/hdtZl1CKfssrgcOalJ2FjA1IgYAU9M8kgYCo4BBaZ1fSOqW1rkGOAUYkB5Nt9lpDBs2jGXLlvGrX/1qVdm0adN45JFHWlynvr6eGTNmAPD73/++Tft99NFHmTlz5hqPxkTR1PDhw7n99ttZuHAhAO+88w6vvfYaixcvZuXKlRx11FGcf/75PP30022Kx8w6n5LVLCLiz5LqmxQfDuyfpicBDwNnpvJbImIZ8KqkucBQSfOA9SPiCQBJvwWOAO7vkBiLONW1I0nizjvvZOzYsVxyySX06NFj1amzLRk/fjxjxozhoosuYs899yxLnAMHDuSCCy5gxIgRrFy5krq6Oq6++mp69uzJSSedtKqm01zNw8y6JpWyRScli3tzmqHejYjeOc//LSI2kPRz4H8j4sZUfi1ZQpgHXBIRB6TyfYEzI+LQFvZ3ClkthC233HL3115b/R4eL7zwAjvssEOHvkZbk4+zWQdo62mw7fxOlzQjIoY0La+WU2ebOyqRp7xZETExIoZExJBNNlnjroBmZtZG5U4WCyT1AUh/F6byBmCLnOX6A2+l8v7NlJuZWRmVO1ncA4xO06OBu3PKR0nqLmlrso7spyJiPrBU0heUjVB3Qs46ZmZWJqU8dfZmss7sjSU1AOOBS4DJksYArwNHA0TEbEmTgeeB5cCpEbEibeo7fHrq7P10UOe2mZkVr5RnQx3TwlPDW1j+QuDCZsqnAzt2YGhmZtZK1dLBbWZmVay2k0VbR3Vsx2iPb7/9NqNGjWLbbbdl4MCBHHzwwUycOJFDD232bOCyOPHEE+nXrx/Lli0DYPHixdTX11csHjOrPrWdLMqsGocob9StWzcP3WFmLXKyKKNqHKK80dixY7niiivWSDwRwRlnnLFqWPLGmySZWW3xqLNlVK1DlANsueWW7LPPPtxwww0cdthhq5a54447mDlzJs8++yyLFy9mjz32YL/99qNPnz7tOBJm1tk4WVSJSg9RDnD22WczcuRIDjnkkFVljz32GMcccwzdunVjs80240tf+hLTpk1j5MiRbXiVZtZZOVmUUTUPUQ7wuc99jsGDBzN58uRVZZ10NHgz62DusyijzjBE+TnnnMNll122an6//fbj1ltvZcWKFSxatIg///nPDB06tE1xmFnnVdvJIqJjHwU0DlE+ZcoUtt12WwYNGsSECRPo27dvi+uMHz+e008/nX333Zdu3bq1uFxHGTRoELvtttuq+SOPPJKdd96ZXXbZhWHDhvGTn/yEzTffvORxmFl1KekQ5ZU0ZMiQmD59+mplHjq7PHyczTqAhyg3M7POxsnCzMwKqrlk0VWb3aqFj69Z11RTyaJHjx4sWbLEX2glEhEsWbKEHj16VDoUM+tgNXWdRf/+/WloaGDRokWVDqXL6tGjx6qLC82s66ipZFFXV8fWW29d6TDMzDqdmmqGMjOztnGyMDOzgpwszMysoJrqsyiWzmvblZMx3mdZmVnX5JqFmZkV5GRhZmYFOVmYmVlBThZmZlaQk4WZmRXks6E6UlvHn4d2j0FvZlZKrlmYmVlBThZmZlaQk4WZmRXkZGFmZgU5WZiZWUEFk4WkvSWtm6aPk3S5pK3as1NJ/yZptqRZkm6W1EPShpKmSHop/d0gZ/lxkuZKmiPpwPbs28zMWq+YmsU1wEeSdgF+CLwG/LatO5TUD/hXYEhE7Ah0A0YBZwFTI2IAMDXNI2lgen4QcBDwC0nd2rp/MzNrvWKSxfLIblp9OHBlRFwJ9GrnftcGekpaG1gHeCttf1J6fhJwRJo+HLglIpZFxKvAXGBoO/dvZmatUEyyWCppHHAccF/6VV/X1h1GxJvAZcDrwHzgvYj4E7BZRMxPy8wHNk2r9APeyNlEQypbg6RTJE2XNN332TYz6zjFJIt/BpYBYyLibbIv6kvbusPUF3E4sDXQF1hX0nH5VmmmrNnLnSNiYkQMiYghm2yySVtDNDOzJvIO95FqETdGxAGNZRHxOu3oswAOAF6NiEVpH3cAXwQWSOoTEfMl9QEWpuUbgC1y1u9P1mxlZmZlkrdmEREryDq3P9uB+3wd+IKkdSQJGA68ANwDjE7LjAbuTtP3AKMkdZe0NTAAeKoD4zEzswKKGUjwE+A5SVOADxsLI+Jf27LDiHhS0u3A08By4BlgIrAeMFnSGLKEcnRafrakycDzaflTUxIzM7MyKSZZ3JceHSYixgPjmxQvI6tlNLf8hcCFHRmDmZkVr2CyiIhJknoCW0bEnDLEZGZmVaaYK7gPA2YCf0zzgyXdU+K4zMysihRz6uwEsovg3gWIiJlkp72amVmNKPYK7vealPm2bmZmNaSYDu5Zkr4BdJM0gGxcp7+UNiwzM6smxdQsTiMbxG8ZcDPwPjC2hDGZmVmVKeZsqI+Ac9LDzMxqUMFkIWkIcDZQn7t8ROxcurDMzKyaFNNncRNwBvAcsLK04ZiZWTUqJlksighfV2FmVsOKSRbjJf2a7O51yxoLI+KOkkVlZmZVpZhkcRKwPdkNjxqboQJwsjAzqxHFJItdImKnkkdiZmZVq5jrLP5X0sCSR2JmZlWrmJrFPsBoSa+S9VkICJ86a2ZWO4pJFgeVPAozM6tqBZuhIuI1oDdwWHr0TmVmZlYjirmfxelkF+Ztmh43Sjqt1IGZmVn1KKYZagywZ0R8CCDpx8ATwH+WMjAzM6sexZwNJWBFzvyKVGZmZjWimJrFb4AnJd2Z5o8AritZRGZmVnWKGaL8ckkPk51CK+CkiHim1IGZmVn1KGaI8hsi4njg6WbKzMysBhTTZzEod0ZSN2D30oRjZmbVqMVkIWmcpKXAzpLeT4+lwELg7rJFaGZmFddisoiIiyOiF3BpRKyfHr0iYqOIGFfGGM3MrMKKaYa6V9K6AJKOk3S5pK1KHJeZmVWRYpLFNcBHknYBfgi8Bvy2pFGZmVlVKSZZLI+IAA4HroyIK4FepQ3LzMyqSTEX5S2VNA44DtgvnQ1VV9qwzMysmhRTs/hnsvtYjImIt4F+wKUljcrMzKpKMUOUvx0Rl0fEo2n+9YhoV5+FpN6Sbpf0oqQXJO0laUNJUyS9lP5ukLP8OElzJc2RdGB79m1mZq1XzBDlS3Ous/hE0gpJ77Vzv1cCf4yI7YFdgBeAs4CpETEAmJrmSbd0HUV2ceBBwC9SU5iZmZVJMTWLXjnXWfQAjgKubusOJa0P7Adcm7b/94h4l6wDfVJabBLZgIWk8lsiYllEvArMBYa2df9mZtZ6xfRZrCYi7gKGtWOf2wCLgN9IekbSr9N1HJtFxPy0j/lkN1qCrI/kjZz1G1LZGiSdImm6pOmLFi1qR4hmZparmIEEv5ozuxYwBIh27nM34LSIeFLSlaQmp5ZCaKas2f1HxERgIsCQIUPaE6OZmeUo5tTZw3KmlwPzyJqG2qoBaIiIJ9P87WTJYoGkPhExX1IfsjGoGpffImf9/sBb7di/mZm1UjH3szipI3cYEW9LekPSdhExBxgOPJ8eo4FL0t/GwQrvAX4n6XKgLzAAeKojYzIzs/xaTBaSfgK8EhH/1aT834DNI+LMduz3NOAmSZ8BXgFOImvimixpDPA6cDRARMyWNJksmSwHTo2IFc1v1szMSiFfzeJQYMdmyq8E/gq0OVlExEyyvo+mhrew/IXAhW3dn5mZtU++s6EiIlY2U7iS5judzcysi8qXLD6SNKBpYSr7uHQhmZlZtcnXDHUucL+kC4AZqWwIMA4YW+K4zMysirSYLCLifklHAGeQdUgDzAKOiojnyhCbmZlVibynzkbELLLTWM3MrIa1ergPMzOrPU4WZmZWUIvJQtKP09+jyxeOmZlVo3w1i4Ml1ZGd/WRmZjUsXwf3H4HFwLqS3ie7EC8a/0bE+mWIz8zMqkCLNYuIOCMiPgvcl2581Cv3bxljNDOzCitmiPKxkg4lq1U8n+5WZ2ZmNSTfqLPrA78GdgeeJWt+2kXSDGBMRLxfnhDNzKzS8nVwX0U2LPiAiPhqRBwJbAs8B/y8HMGZmVl1yNcMtXdEnJhbEBEB/Iekl0oalZmZVZV8NQsPQ25mZkD+ZPG4pHMlrZY0JP078L+lDcvMzKpJvmao04BrgbmSZpKdDbUr8AwwpvShmZlZtcg3RPn7wNGStgUGkjVLnRkRL5crODMzqw4Fr7NIycEJotqpHV1MER0Xh5l1SR511szMCnKyMDOzgvImC0lrSZpVrmDMzKw65U0WEbESeFbSlmWKx8ysekhtf3QxxQwk2AeYLekp4MPGwogYWbKozMysqhSTLM4reRRmZlbVijl19hFJW5ENKPigpHWAbqUPzczMqkXBs6EkfRO4HfhlKuoH3FXCmMzMrMoUc+rsqcDewPsAEfESsGkpgzIzs+pSTJ/Fsoj4e+N4gpLWJhsnyqyyfNV67Wjre+33ucMUU7N4RNLZQE9J/wTcBvx3acMyM7NqUkyyOAtYRHaHvG8BfwB+1N4dS+om6RlJ96b5DSVNkfRS+rtBzrLjJM2VNEfSge3dt5mZtU4xZ0OtlDQJeJKs+WlOumNee50OvACsn+bPAqZGxCWSzkrzZ0oaCIwCBgF9gQclfT4iVnRADGZmVoRizoY6hGzU2avI7r09V9JX2rNTSf2BQ4Bf5xQfDkxK05OAI3LKb4mIZRHxKjAXGNqe/ZuZWesU08H9U+DLETEXIN3f4j7g/nbs92fAD4FeOWWbRcR8gIiYL6nxjKt+rH5nvoZUtgZJpwCnAGy5pUcoKRt3Ppp1ecX0WSxsTBTJK8DCtu5Q0qFpmzOKXaWZsma/ZSJiYkQMiYghm2yySVtDNDOzJlqsWUj6apqcLekPwGSyL+mjgWnt2OfewEhJBwM9gPUl3QgskNQn1Sr68GlCagC2yFm/P/BWO/ZvZmatlK9mcVh69AAWAF8C9ic7M2qDllfLLyLGRUT/iKgn67j+n4g4DrgHGJ0WGw3cnabvAUZJ6i5pa2AA8FRb929mZq2X7x7cJ5UzEOASYLKkMcDrZDUYImK2pMnA88By4FSfCWVWAb4IsqYV7OBOv+ZPA+pzl++IIcoj4mHg4TS9BBjewnIXAhe2d39mZtY2xZwNdRdwLdlV2ytLGo1ZZ+EzwKzGFJMsPomIq0oeiZmZVa1iksWVksYDfwKWNRZGxNMli8rMzKpKMcliJ+B4YBifNkNFmjczsxpQTLI4EtgmIv5e6mDMzKw6FZMsngV6046rts2sg/j0VauQYpLFZsCLkqaxep9Fu0+dNTOzzqGYZDG+5FGYmVlVK+Z+Fo+UIxAzM6texVzBvZRPR3n9DFAHfBgR67e8lpmZdSXF1Cxy7zmBpCPwzYfMzGpKMfezWE1E3IWvsTAzqynFNEN9NWd2LWAILdx8yMzMuqZizoY6LGd6OTCP7L7YZmZWI4rpsyj3fS3MzKzK5Lut6rl51ouIOL8E8ZiZWRXKV7P4sJmydYExwEaAk4WZWY3Id1vVnzZOS+oFnA6cBNwC/LSl9czMrOvJ22chaUPge8CxwCRgt4j4WzkCMzOz6pGvz+JS4KvARGCniPigbFGZmVlVyXdR3veBvsCPgLckvZ8eSyW9X57wzMysGuTrs2j11d1mZtY1FXNRnpWRzmvbzW18Sb2ZlZJrD2ZmVpCThZmZFeRkYWZmBTlZmJlZQU4WZmZWkJOFmZkV5GRhZmYFOVmYmVlBZU8WkraQ9JCkFyTNlnR6Kt9Q0hRJL6W/G+SsM07SXElzJB1Y7pjNzGpdJWoWy4HvR8QOwBeAUyUNBM4CpkbEAGBqmic9NwoYBBwE/EJStwrEbWZWs8qeLCJifkQ8naaXAi8A/cju6z0pLTYJOCJNHw7cEhHLIuJVYC4wtKxBm5nVuIr2WUiqB3YFngQ2i4j5kCUUYNO0WD/gjZzVGlJZc9s7RdJ0SdMXLVpUsrjNzGpNxZKFpPWA3wNjIyLfkOfNjazX7Lh5ETExIoZExJBNNtmkI8I0MzMqlCwk1ZElipsi4o5UvEBSn/R8H2BhKm8AtshZvT/wVrliNTOzypwNJeBa4IWIuDznqXuA0Wl6NHB3TvkoSd0lbQ0MAJ4qV7xmZlaZ+1nsDRwPPCdpZio7G7gEmCxpDPA6cDRARMyWNBl4nuxMqlMjYkXZozYzq2FlTxYR8RjN90MADG9hnQuBC0sWlJmZ5eUruM3MrCAnCzMzK8j34Dag7ff+Bt//26wWuGZhZmYFOVmYmVlBThZmZlaQk4WZmRXkZGFmZgU5WZiZWUFOFmZmVpCThZmZFeSL8qzi2npBoC8GNCsf1yzMzKwg1yysZnmIE7PiOVmYVYCb3qyzcTOUmZkV5GRhZmYFOVmYmVlBThZmZlaQO7jNaojPACuvrnS8nSzMrCy60hdnLXIzlJmZFeRkYWZmBTlZmJlZQU4WZmZWkDu4zazL8/Aq7eeahZmZFeRkYWZmBTlZmJlZQU4WZmZWkJOFmZkV1GmShaSDJM2RNFfSWZWOx8yslnSKZCGpG3A18BVgIHCMpIGVjcrMrHZ0imQBDAXmRsQrEfF34Bbg8ArHZGZWMzrLRXn9gDdy5huAPZsuJOkU4JQ0+4GkOUVse2NgcbsjBNo+piagotZuMdYy7Lvl1ZsvLnxcS7PfIldebe1WfwbavO/2v+a2f17Lf7w/jbVS73Xx+232uFbh/xUU+gy0c7/AVs0VdpZk0dyrX+PiyoiYCExs1Yal6RExpK2BlZNjLQ3HWhqOtTQqFWtnaYZqALbIme8PvFWhWMzMak5nSRbTgAGStpb0GWAUcE+FYzIzqxmdohkqIpZL+hfgAaAbcF1EzO6gzbeq2arCHGtpONbScKylUZFYFeFxFc3MLL/O0gxlZmYV5GRhZmYF1WyyqPbhQyRdJ2mhpFk5ZRtKmiLppfR3g0rGmGLaQtJDkl6QNFvS6VUcaw9JT0l6NsV6XrXG2khSN0nPSLo3zVdzrPMkPSdppqTpqawq45XUW9Ltkl5Mn929qjFWSdul49n4eF/S2ErEWpPJopMMH3I9cFCTsrOAqRExAJia5ittOfD9iNgB+AJwajqW1RjrMmBYROwCDAYOkvQFqjPWRqcDL+TMV3OsAF+OiME51wFUa7xXAn+MiO2BXciOcdXFGhFz0vEcDOwOfATcSSVijYiaewB7AQ/kzI8DxlU6rmbirAdm5czPAfqk6T7AnErH2EzMdwP/VO2xAusAT5ONBFCVsZJdTzQVGAbcW+2fAWAesHGTsqqLF1gfeJV0gk81x9okvhHA45WKtSZrFjQ/fEi/CsXSGptFxHyA9HfTCsezGkn1wK7Ak1RprKlZZyawEJgSEVUbK/Az4IfAypyyao0VslEV/iRpRhp6B6oz3m2ARcBvUhPfryWtS3XGmmsUcHOaLnustZosiho+xIonaT3g98DYiHi/0vG0JCJWRFal7w8MlbRjhUNqlqRDgYURMaPSsbTC3hGxG1nz7qmS9qt0QC1YG9gNuCYidgU+pAqanPJJFyOPBG6rVAy1miw66/AhCyT1AUh/F1Y4HgAk1ZElipsi4o5UXJWxNoqId4GHyfqFqjHWvYGRkuaRjbI8TNKNVGesAETEW+nvQrJ29aFUZ7wNQEOqVQLcTpY8qjHWRl8Bno6IBWm+7LHWarLorMOH3AOMTtOjyfoHKkqSgGuBFyLi8pynqjHWTST1TtM9gQOAF6nCWCNiXET0j4h6ss/n/0TEcVRhrACS1pXUq3GarH19FlUYb0S8DbwhabtUNBx4niqMNccxfNoEBZWItdKdNhXsLDoY+D/gZeCcSsfTTHw3A/OBf5D9EhoDbETW4flS+rthFcS5D1kT3l+BmelxcJXGujPwTIp1FnBuKq+6WJvEvT+fdnBXZaxk/QDPpsfsxv+pKo53MDA9fRbuAjao4ljXAZYAn80pK3usHu7DzMwKqtVmKDMzawUnCzMzK8jJwszMCnKyMDOzgpwszMysICcLs1ZKI5Z+t9JxmJWTk4VZ6/UGSposJHWKWx5b7XCyMGu9S4Bt0/0FLpV0hqRpkv6ac4+M+nSfhF+le2f8KV01jqSHJQ1J0xunIT2QdKKk2yT9N9mAfOsqu6/JtDTg3eEVer1mThZmbXAW8HJkAxJOAQaQjYM0GNg9ZwC9AcDVETEIeBc4qoht7wWMjohhwDlkw3zsAXwZuDQNpWFWdq7qmrXPiPR4Js2vR5YkXgdejYiZqXwG2f1JCpkSEe/kbHukpB+k+R7Alqx+MySzsnCyMGsfARdHxC9XK8zu7bEsp2gF0DNNL+fTWn2PJtv7sMm2j4qIOR0WrVkbuRnKrPWWAr3S9APAyel+HkjqJ6nQjWjmkd0iE+BreZZ7ADgtjeyLpF3bHLFZOzlZmLVSRCwBHpc0i+wWsr8DnpD0HNm9EXrlWx+4DPiOpL8AG+dZ7nygDvhr2tf57Q7erI086qyZmRXkmoWZmRXkZGFmZgU5WZiZWUFOFmZmVpCThZmZFeRkYWZmBTlZmJlZQf8fOIpfNKuVYfoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "tenure_churn_no = df1[df1.Churn=='No'].tenure\n",
    "tenure_churn_yes = df1[df1.Churn=='Yes'].tenure\n",
    "\n",
    "plt.xlabel(\"tenure\")\n",
    "plt.ylabel(\"Number Of Customers\")\n",
    "plt.title(\"Customer Churn Prediction Visualiztion\")\n",
    "\n",
    "blood_sugar_men = [113, 85, 90, 150, 149, 88, 93, 115, 135, 80, 77, 82, 129]\n",
    "blood_sugar_women = [67, 98, 89, 120, 133, 150, 84, 69, 89, 79, 120, 112, 100]\n",
    "\n",
    "plt.hist([tenure_churn_yes, tenure_churn_no], rwidth=0.95, color=['green','red'],label=['Churn=Yes','Churn=No'])\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x2505b006a30>"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEWCAYAAACXGLsWAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy86wFpkAAAACXBIWXMAAAsTAAALEwEAmpwYAAAppklEQVR4nO3de5yVVb3H8c9XJMBbpIJxkYaMVPDuiJaXSD1oooCVhWmicqLMk3JOmaKlWKaWpkdP6onMxEsqmhdO5pUsL8cLg4KClyMo6ggKaCp4IYHf+eNZg9thz96bmdl79sx836/Xfs1+1nNba+89+7fXWs+zliICMzOzQtZr6wyYmVn1c7AwM7OiHCzMzKwoBwszMyvKwcLMzIpysDAzs6IcLKzdklQjKSSt39Z5aW2Sjpb0YM7yckmfbcZxjpB0d+vmrqTz7i3puTKfo7Veo/+W9NPWzV3H42DRTkj6lqS69A+xSNIdkvZq4TEnSbqmtfJYDpI+L+lGSUslvS3pSUn/IalLFeTtSkn/TO/Jm5LukbRNOc4VERtFxAtF8rNW8IyIayNieGvmRVI/SSslbZVn3S2Szo+IByJi69Y8bzElvkYfCzBpv+9FxM/Lm7v2z8GiHZD0H8B/AmcDWwADgEuBUW2YrVaVr3aQvoweBV4Bto+ITwKHAbXAxuU+f4l+FREbAf2BxcCVeY4tSR3mfy0iXgWmA9/OTZe0KXAQMKUt8mVlFhF+VPED+CSwHDiswDZXAmflLA8D6nOWTwZeBZYBzwH7AQcC/wQ+TMefnbbtC0wD3gTmAd/JOc4k4EbgmnSsp4DPAxPJvihfAYY3yvvvgUXp/GcBXdK6o4GHgAvTuc7KU65rgNsLlLsGCGAs8DKwFDhtHV6XBem1eRJYAXyu0PFKeN1HAMvT878Bv0hlfD8dexvgnlTe54Bv5Oy7WXrd3wEeA34OPJizPoDPpec9gF8DLwFvAw+mtJfTdsvT4wvpdc49zheBGWm/GcAXc9b9LZ33ofT+3g1s3kTZvwXMb5T2feDxUj+DJb5HpwDz035PA4fmrGtctkivc9+c12A58F5aty3wAbAqpb/VRB6+Q/bZfzO9J30bneN7wPPAP4BLALX190QlHh3m104H9gWgO3BLc3aWtDXwb8BuEbExcACwICLuJKup3BBZ9X3HtMt1QD3ZP9zXgbMl7ZdzyEOAq4FPAU8Ad5HVUPsBPwN+m7PtFGAl2T/wzsBw4F9z1u8OvAD0JvtibWx/4KYSirkXsDVZEDxd0rYl7NPgcLIv+Z4pr806nqSNgCPIXpMG3wbGk9WClpAFij+Slfdw4FJJQ9K2l5B9kfUBjk2PppwP7Er2xb8p8GNgNbBPWt8zvacPN8rjpsDtwMVkwekC4HZJm+Vs9i3gmJTHTwA/aiIPtwCbN2oK/TZwVeMNm/oMFihfrvnA3mQ/PM4ErpHUp9AOEbEwlX+jyGp9twDXR8QzZF/0D6d1PfPkdV/gHOAbZO/FS8D1jTY7GNgN2DFtd0CJZWnXHCyq32bA0ohYWXTL/FYB3YDBkrpGxIKImJ9vQ0lbkn1RnhwRH0TELOByPt7c8EBE3JXycyPQCzg3Ij4k+6eqkdRT0hbAV4AJEfFuRCwmq0WMyTnWwoj4r4hYGRHvN1H2RSWU8cyIeD8iZgOzyf6JS3VxRLzS6PzrcrwfSXqL7JfoRmS/dhtcGRFz02t1IFmQ/kMq7+PAn4Cvp/6XrwGnp9dqDk005aTmrGOBEyPi1YhYFRH/GxErSijrCOD5iLg65eE64FmyHwAN/hAR/5dej6nATvkOlNbfCByV8jWILID9Mc/mJX8G85znxvTlvzoibiD7RT+0lH1Tvk4mq9EVCr65jgCuiIjH02s6EfiCpJqcbc6NiLci4mXgPpp4jToaB4vq9wbZL7hmtalHxDxgAlkT0mJJ10vq28TmfYE3I2JZTtpLZLWGBq/nPH+fLJCtylmG7EvzM0BXYJGkt9IX6m/JfrE2eKVI9t8g+3VXzGs5z99L5y9Vvjysy/HOj4ieEfHpiBjZ6Esw99ifAXZveC3S63EE8GmygLt+o+1fauJ8m5PVNEv6sm2kb57jNn5/16XsU4BvSOpO9oPizvSj4GPW8TP4MZKOkjQr5zXbjuw1KGXfrwAnAqOb+DGSz8deo4hYTvY5bO5r1GE4WFS/h8maJ0YX2OZdYIOc5U/nroyIP0bEXmRfWAH8smFVo+MsBDaVlNt5PICsrXldvULWD7B5+jLtGRGbRMSQnG2KDXl8L9kv7uYq+LqUmIeWyD32K8Dfc16Lhqai48iaqFYCW+ZsP6CJYy4l+zysdSUSxcuykOwzkKu57y8R8QDZF+ko4EjyNEHlbNvUZ7DJ90jSZ4DfkTVhbZaajeYAKpa31PQ1haxfKDcIr9NrJGlDshpus16jjsTBospFxNvA6cAlkkZL2kBSV0lfkfSrtNks4CBJm0r6NNmvOCD7p5G0r6RuZF8y75M1C0BWS6hpuFIn/VP9L3COpO6SdgDGAdc2I9+LyDpIfy1pE0nrSdpK0pfW4TBnAF+UdF4qF5I+J+kaST1L2H8WTbwubeDPwOclfTu9f10l7SZp21QzuxmYlN7fwWSd7GuJiNXAFcAFkvpK6iLpC+n9XULWd9HUvQZ/SXn4lqT1JX0TGJzy1lxXkX3x9wT+J98GRT6Ds2j6PdqQ7Mt9STrOMWQ1i4IkbQLcBvwkIh5stPp1oL+kTzSx+x+BYyTtlPJ7NvBoRCwodt6OzsGiHYiIC4D/AH5C9o/zCtmvrVvTJleTta0vIPuCviFn927AuWS/SF8jawY6Na27Mf19Q9Lj6fnhZFcZLSTrGDwjIu5pZtaPIuskfZrsypGbKK1ZCYDUpPOFlJ+5kt4ma+evI7s6pphCr0tFpaa94WR9NgvJ3otfkr0/kL2fG6X0K4E/FDjcj8iuRJtBdsXOL4H1IuI90hVYqdlmj0Z5eIOsc/aHZDWCHwMHR8TSFhTtKrLayQ0F+k0KfQabfI8i4mmyq74eJvuS357sSq1idiG7QOGCdA/McknL07q/AnOB1yStVe6ImA78lOxztoisBjem8XadkSI8+ZGZmRXmmoWZmRXlYGFmZkU5WJiZWVEOFmZmVlSHG9q5weabbx41NTVtnQ0zs3Zl5syZSyOiV+P0DhssampqqKura+tsmJm1K5Lyjh7gZigzMyvKwcLMzIpysDAzs6I6bJ+FmbVPH374IfX19XzwwQdtnZUOrXv37vTv35+uXbuWtL2DhZlVlfr6ejbeeGNqamqQig4wa80QEbzxxhvU19czcODAkvZxM5SZVZUPPviAzTbbzIGijCSx2WabrVPtzcHCzKqOA0X5retr7GBhZmZFuc/CzKqazmzdWkacUdq0DK+99hoTJkxgxowZdOvWjZqaGkaPHs20adP4859bMl9U80yePJnp06dzww3ZlB/vvPMOO++8M/fee2/J/Q4t4ZpFa5Ka/zCzqhERHHrooQwbNoz58+fz9NNPc/bZZ/P6668X37mAlStXNnvf73znO9TX13PvvfcCcPrpp3PsscdWJFCAg4WZ2Vruu+8+unbtyve+9701aTvttBN77703y5cv5+tf/zrbbLMNRxxxBA0TyNXU1LB0aTb5Xl1dHcOGDQNg0qRJjB8/nuHDh3PUUUcxadIkjj32WIYNG8ZnP/tZLr744pLyJInLLruMCRMmUFdXx/Tp0znppJO45pprGDp0KDvttBPf/e53WbVqFatWreLoo49mu+22Y/vtt+fCCy9s8WviZigzs0bmzJnDrrvumnfdE088wdy5c+nbty977rknDz30EHvttVfB482cOZMHH3yQHj16MGnSJJ599lnuu+8+li1bxtZbb81xxx1H165d2XvvvVm2bO0Zg88//3z2339/dthhBw444AD2228/br31VubPn88NN9zAQw89RNeuXfn+97/Ptddey5AhQ3j11VeZM2cOAG+99VaLXxMHCzOzdTB06FD69+8PZLWNBQsWFA0WI0eOpEePHmuWR4wYQbdu3ejWrRu9e/fm9ddfp3///jzwwANFz3/88cdzxx138OUvf5nf/OY3zJw5k9122w2A999/n969e3PIIYfwwgsv8IMf/IARI0YwfPjwFpQ4U7ZgIekKssnhF0fEdo3W/Qg4D+jVMFm8pInAOGAVcEJE3JXSdyWbwL4H8BfgxPDE4WZWRkOGDOGmm27Ku65bt25rnnfp0mVNP8T666/P6tWrAda6f2HDDTcs6RjFahYA6623Huutl/UgRARjx47lnHPOWWuf2bNnc9ddd3HJJZcwdepUrrjiisKFLqKcfRZXAgc2TpS0JfAvwMs5aYOBMcCQtM+lkrqk1ZcB44FB6bHWMc3MWtO+++7LihUr+N3vfrcmbcaMGfz9739vcp+amhpmzpwJwJ/+9KdmnfeBBx5g1qxZaz0aAkVj++23HzfddBOLFy8G4M033+Sll15i6dKlrF69mq997Wv8/Oc/5/HHH29WfnKVrWYREfdLqsmz6kLgx8BtOWmjgOsjYgXwoqR5wFBJC4BNIuJhAElXAaOBO8qVbzOrLqVe6tqaJHHLLbcwYcIEzj33XLp3777m0tmmnHHGGYwbN46zzz6b3XffvSL5HDx4MGeddRbDhw9n9erVdO3alUsuuYQePXpwzDHHrKnp5Kt5rCuVs0UnBYs/NzRDSRoJ7BcRJ6ZAUBsRSyX9BngkIq5J2/2eLCAsAM6NiP1T+t7AyRFxcBPnG09WC2HAgAG7vvRS3jk8yqcll8C6Zc0MgGeeeYZtt922rbPRKeR7rSXNjIjaxttW7NJZSRsApwGn51udJy0KpOcVEZMjojYianv1WmtWQDMza6ZKXg21FTAQmJ3GJOkPPC5pKFAPbJmzbX9gYUrvnyfdzMwqqGI1i4h4KiJ6R0RNRNSQBYJdIuI1YBowRlI3SQPJOrIfi4hFwDJJeyiLMEfx8b4OMzOrgLIFC0nXAQ8DW0uqlzSuqW0jYi4wFXgauBM4PiJWpdXHAZcD84D5uHPbzKziynk11OFF1tc0Wv4F8Is829UB2zVONzOzyvHYUGZmVpSDhZlVt5aM5tyCEZ5fe+01xowZw1ZbbcXgwYM56KCDmDx5MgcfnPfK/Yo4+uij6devHytWrABg6dKl1NTUVOTcDhZmZo1U4xDlDbp06dLioTuaw8HCzKyRahyivMGECRO48MIL1wo8EcFJJ520ZljyhkmSWotHnTUza6RahygHGDBgAHvttRdXX301hxxyyJptbr75ZmbNmsXs2bNZunQpu+22G/vssw99+vRpwSvxEQcLM7N10NZDlAOceuqpjBw5khEjRqxJe/DBBzn88MPp0qULW2yxBV/60peYMWMGI0eObEYp1+ZgYWbWSDUPUQ7wuc99jp122ompU6euSSv3zA3uszAza6Q9DFF+2mmncf75569Z3meffbjhhhtYtWoVS5Ys4f7772fo0KHNykc+DhZmVt0iWvdRgoYhyu+55x622morhgwZwqRJk+jbt2+T+5xxxhmceOKJ7L333nTp0qXJ7VrLkCFD2GWXXdYsH3rooeywww7suOOO7LvvvvzqV7/i05/+dKudr6xDlLel2traqKurq+xJPUS5WYt5iPLKqcohys3MrP1ysDAzs6IcLMys6nTU5vFqsq6vsYOFmVWV7t2788YbbzhglFFE8MYbb9C9e/eS9/F9FmZWVfr37099fT1Llixp66x0aN27d19zc2EpHCzMrKp07dqVgQMHtnU2rBE3Q5mZWVEOFmZmVpSDhZmZFVW2YCHpCkmLJc3JSTtP0rOSnpR0i6SeOesmSpon6TlJB+Sk7yrpqbTuYqklt0mbmVlzlLNmcSVwYKO0e4DtImIH4P+AiQCSBgNjgCFpn0slNQyuchkwHhiUHo2PaWZmZVa2YBER9wNvNkq7OyIapnd6BGi4bmsUcH1ErIiIF4F5wFBJfYBNIuLhyC66vgoYXa48m5lZfm3ZZ3EscEd63g94JWddfUrrl543Tjczswpqk2Ah6TRgJXBtQ1KezaJAelPHHS+pTlKdb+gxM2s9FQ8WksYCBwNHxEf389cDW+Zs1h9YmNL750nPKyImR0RtRNT26tWrdTNuZtaJVTRYSDoQOBkYGRHv5ayaBoyR1E3SQLKO7MciYhGwTNIe6Sqoo4DbKplnMzMr43Afkq4DhgGbS6oHziC7+qkbcE+6AvaRiPheRMyVNBV4mqx56viIWJUOdRzZlVU9yPo47sDMzCrKM+W1Js+UZ2btnGfKMzOzZnOwMDOzohwszMysqKLBQtKekjZMz4+UdIGkz5Q/a2ZmVi1KqVlcBrwnaUfgx8BLZMNumJlZJ1FKsFiZbp4bBVwUERcBG5c3W2ZmVk1Kuc9imaSJwJHAPmk02K7lzZaZmVWTUmoW3wRWAOMi4jWygfzOK2uuzMysqhSsWaRaxDURsX9DWkS8jPsszMw6lYI1izTkxnuSPlmh/JiZWRUqpc/iA+ApSfcA7zYkRsQJZcuVmZlVlVKCxe3pYWZmnVTRYBERUyT1AAZExHMVyJOZmVWZUu7gPgSYBdyZlneSNK3M+TIzsypSyqWzk4ChwFsAETELGFi2HJmZWdUp9Q7utxulefIFM7NOpJQO7jmSvgV0kTQIOAH43/Jmy8zMqkkpNYsfAEPI7uK+DngHmFDGPJmZWZUp5Wqo94DT0sPMzDqhosFCUi1wKlCTu31E7FC+bJmZWTUppRnqWuBK4GvAITmPgiRdIWmxpDk5aZtKukfS8+nvp3LWTZQ0T9Jzkg7ISd9V0lNp3cWStA7lMzOzVlBKsFgSEdMi4sWIeKnhUcJ+VwIHNko7BZgeEYOA6WkZSYOBMWR9IwcCl6ZBDCGbfGk8MCg9Gh/TzMzKrJSroc6QdDnZl/uKhsSIuLnQThFxv6SaRsmjgGHp+RTgb8DJKf36iFgBvChpHjBU0gJgk4h4GEDSVcBo4I4S8m1mZq2klGBxDLAN2YRHq1NaAAWDRRO2iIhFABGxSFLvlN4PeCRnu/qU9mF63jg9L0njyWohDBgwoBnZMzOzfEoJFjtGxPZlzke+fogokJ5XREwGJgPU1tb6xkEzs1ZSSp/FI6lPoTW8LqkPQPq7OKXXA1vmbNcfWJjS++dJNzOzCiolWOwFzEpXKT2Zrkx6spnnmwaMTc/HArflpI+R1E3SQLKO7MdSk9UySXukq6COytnHzMwqpJRmqGZdfSTpOrLO7M0l1QNnAOcCUyWNA14GDgOIiLmSpgJPAyuB49MsfQDHkV1Z1YOsY9ud22ZmFaaI4k37knYE9k6LD0TE7LLmqhXU1tZGXV1dZU/akltASngfzMzKTdLMiKhtnF7KfBYnkt2Y1zs9rpH0g9bPopmZVatSmqHGAbtHxLsAkn4JPAz8VzkzZmZm1aOUDm4Bq3KWV5H/klYzM+ugSqlZ/AF4VNItaXk0cEXZcmRmZlWnlCHKL5D0N7JLaAUcExFPlDtjZmZWPUoZovzqiPg28HieNDMz6wRK6bMYkruQRoPdtTzZMTOzatRksEjzSywDdpD0TnosIxuiw3dRm5l1Ik0Gi4g4JyI2Bs6LiE3SY+OI2CwiJlYwj2Zm1sZKaYb6s6QNASQdKekCSZ8pc77MzKyKlBIsLgPeS0N+/Bh4CbiqrLkyM7OqUkqwWBnZAFKjgIsi4iJg4/Jmy8zMqkkpN+UtkzQROBLYJ10N1bW82TIzs2pSSs3im2Rzb4+LiNfIpjU9r6y5MjOzqlLKHdyvARfkLL+M+yzMzDqVUu7gXsZH815/gqwJanlEfLKcGTMzs+pRSs3iY53ZkkYDQ8uVITMzqz6l9Fl8TETcCuzb+lkxM7NqVUoz1FdzFtcDavmoWcrMzDqBUi6dPSTn+UpgAdk9F2Zm1kmU0mdxTGufVNK/A/9KVkN5CjgG2AC4AaghC0jfiIh/pO0nkk3vugo4ISLuau08mZlZ0wqNOvsrSd/Lk/7vaR7uZpHUDzgBqI2I7YAuwBjgFGB6RAwCpqdlJA1O64cABwKXphsDzcysQgp1cB8MTM6TfhEwooXnXR/oIWl9shrFQrKmrSlp/RSy6VtJ6ddHxIqIeBGYh6/GMjOrqELBIiJidZ7E1WTTqzZLRLwKnA+8DCwC3o6Iu4EtImJR2mYR0Dvt0g94JecQ9SltLZLGS6qTVLdkyZLmZtHMzBopFCzekzSocWJKe7+5J5T0KbLawkCgL7ChpCML7ZInLe/VWBExOSJqI6K2V69ezc2imZk1UqiD+3TgDklnATNTWi0wEZjQgnPuD7wYEUsAJN0MfBF4XVKfiFgkqQ/ZjHyQ1SS2zNm/P1mzlZmZVUihmfLuIOs3+DJwZXoMA74WEX9pwTlfBvaQtIEkAfsBzwDTgLFpm7F8NHXrNGCMpG6SBgKDgMdacH4zM1tHBS+djYg5fPQF3ioi4lFJNwGPk9238QRZR/pGwFRJ48gCymFp+7mSpgJPp+2Pj4hVrZknMzMrTNm8Rh1PbW1t1NXVVfakana/P3TQ98HM2hdJMyOitnH6Oo8NZWZmnU+hm/J+mf4eVrnsmJlZNSpUszhIUleyq5/MzKwTK9TBfSewlOw+iHfI7neIhr8RsUkF8mdmZlWg0KWzJ6XZ8G6PiE0iYuPcvxXMo5mZtbFShiifIOlgslrF02l8JjMz60SaDBaSNgEuB3YFZpM1P+0oaSYwLiLeqUwWzcysrRXq4L6Y7Ea4QRHx1Yg4FNiKbP6J31Qic2ZmVh0KNUPtGRFH5yZEdgffzyQ9X9ZcmZlZVSlUs2jB7chmZtaRFAoWD0k6PQ32t4aknwKPlDdbZmZWTQo1Q/0A+D0wT9IssquhdiYb+G9c+bNmZmbVoslgka52OkzSVsBgsmapkyNifqUyZ2Zm1aHofRYpODhAmJl1Yh511szMinKwMDOzogoGC0nrSZpTqcyYmVl1KhgsImI1MFvSgArlx8zMqlApAwn2AeZKegx4tyExIkaWLVdmZlZVSgkWZ7b2SSX1JBukcDuy+zeOBZ4DbgBqgAXANyLiH2n7iWT3dqwCToiIu1o7T2Zm1rSiHdwR8XeyL++u6fkM4PEWnvci4M6I2AbYEXgGOAWYHhGDgOlpGUmDgTHAEOBA4FJJXVp4fjMzWwdFg4Wk7wA3Ab9NSf2AW5t7wjT0+T5kd4cTEf+MiLeAUcCUtNkUYHR6Pgq4PiJWpLk05gFDm3t+MzNbd6VcOns8sCfwDkBEPA/0bsE5PwssAf4g6QlJl0vaENgiIhalcyzKOUc/4JWc/etT2lokjZdUJ6luyZIlLciimZnlKiVYrIiIfzYsSFqfrJ+hudYHdgEui4idyTrNTymwfb7Rb/OePyImR0RtRNT26tWrBVk0s7VIzX9Yu1dKsPi7pFOBHpL+BbgR+J8WnLMeqI+IR9PyTWTB43VJfQDS38U522+Zs39/YGELzm9mZuuolGBxClmz0VPAd4G/AD9p7gkj4jXgFUlbp6T9yGbkmwaMTWljgdvS82nAGEndJA0EBgGPNff8Zma27koZSHC1pCnAo2TNP8+lGfNa4gfAtZI+AbwAHEMWuKZKGge8DByWzj9X0lSygLISOD4iVrXw/GZmtg6KBgtJI4D/Jht5VsBASd+NiDuae9KImAXU5lm1XxPb/wL4RXPPZ2ZmLVPKTXm/Br4cEfMA0vwWtwPNDhZmZta+lBIsFjcEiuQFPup8NjMrv+ZeUdXiFnNr0GSwkPTV9HSupL8AU8n6LA4ju4vbzCqtJZeh+ovTWqBQzeKQnOevA19Kz5cAnypbjszMrOoUmoP7mEpmxMzMqlcpV0MNJLvUtSZ3ew9RbmbWeZTSwX0r2aB//wOsLmtuzMysKpUSLD6IiIvLnhMzM6tapQSLiySdAdwNrGhIjIiWzmlhZmbtRCnBYnvg28C+fNQMFWnZzMw6gVKCxaHAZ3OHKTczs86llGAxG+iJ79q2auMb1Kzc/Blbo5RgsQXwrKQZfLzPwpfOmpl1EqUEizPKngszM6tqpcxn8fdKZMTMzKpXKXdwL+OjOa8/AXQF3o2ITcqZMTMzqx6l1Cw2zl2WNBoYWq4MmZlZ9SllDu6PiYhb8T0WZmadSinNUF/NWVyPbDrUjnVNmJmZFVTK1VC581qsBBYAo8qSGzMzq0ql9FmUZV4LSV2AOuDViDhY0qbADWRDoS8AvhER/0jbTgTGAauAEyLirnLkyaxknubTOplC06qeXmC/iIift/DcJwLPAA1XVZ0CTI+IcyWdkpZPljQYGAMMAfoC90r6fESsauH5zcysRIU6uN/N84DsF/7JLTmppP7ACODynORRwJT0fAowOif9+ohYEREvAvPw1VhmZhVVaFrVXzc8l7QxWU3gGOB64NdN7Vei/wR+DORelrtFRCxK514kqXdK7wc8krNdfUpbi6TxwHiAAQMGtDCLZmbWoOCls5I2lXQW8CRZYNklIk6OiGYPKijpYGBxRMwsdZc8aXkbfiNickTURkRtr169mptFMzNrpFCfxXnAV4HJwPYRsbyVzrknMFLSQUB3YBNJ1wCvS+qTahV9+GiU23pgy5z9+wMLWykvZmZWgkI1ix+SdSj/BFgo6Z30WCbpneaeMCImRkT/iKgh67j+a0QcCUwDxqbNxgK3pefTgDGSukkaCAwCHmvu+c3MbN0V6rNY57u7W+hcYKqkccDLwGEpH3MlTQWeJrvP43hfCWVmVlmKDnrdd21tbdTV1VX2pJ4opbLa8vVuq/ss2mOZ2/Lc7bnMbUTSzIiobZxeyh3cZtZB6Mzmf/m1z68+ay0OFmZtoLlf2v7CtrZS6X4JMzNrhxwszMysKAcLMzMryn0WZlYR7lxv31yzMDOzohwszMysKAcLMzMrysHCzMyKcrAwM7OiHCzMzKwoXzprZh2eh1dpOdcszMysKAcLMzMrys1QZmbVqK3m8GiCg4W1XJV9qM2s9bkZyszMinKwMDOzoioeLCRtKek+Sc9ImivpxJS+qaR7JD2f/n4qZ5+JkuZJek7SAZXOs5lZZ9cWNYuVwA8jYltgD+B4SYOBU4DpETEImJ6WSevGAEOAA4FLJXVpg3ybmXVaFQ8WEbEoIh5Pz5cBzwD9gFHAlLTZFGB0ej4KuD4iVkTEi8A8YGhFM21m1sm1aZ+FpBpgZ+BRYIuIWARZQAF6p836Aa/k7Faf0vIdb7ykOkl1S5YsKVu+zcw6mzYLFpI2Av4ETIiIdwptmict7zWXETE5ImojorZXr16tkU0zM6ONgoWkrmSB4tqIuDklvy6pT1rfB1ic0uuBLXN27w8srFRezcysba6GEvB74JmIuCBn1TRgbHo+FrgtJ32MpG6SBgKDgMcqlV8zM2ubO7j3BL4NPCVpVko7FTgXmCppHPAycBhARMyVNBV4muxKquMjYlXFc21m1olVPFhExIPk74cA2K+JfX4B/KJsmTIzs4J8B7eZmRXlgQStzXliGrPq52CRh7+8zMw+zs1QZmZWlGsWHUVz55QAzythZkW5ZmFmZkW5ZmFA8/tpwH01Zp2BaxZmZlaUg4WZmRXlYGFmZkU5WJiZWVEOFmZmVpSDhZmZFeVgYWZmRTlYmJlZUb4pr8p4EMPK8Y2IZqVzzcLMzIpyzcLMrEw6Uu3VNQszMyvKwcLMzIpqN8FC0oGSnpM0T9IpbZ0fM7POpF0EC0ldgEuArwCDgcMlDW7bXJmZdR7tIlgAQ4F5EfFCRPwTuB4Y1cZ5MjPrNNrL1VD9gFdyluuB3RtvJGk8MD4tLpf0XCudf3NgabGNWjCxacumRS3PuUsqc4vO7TKv++4t2rmNytz0uUvfvXXP6zIX9pl8ie0lWOQr/VpXlkXEZGByq59cqouI2tY+bjVzmTsHl7lzaI0yt5dmqHpgy5zl/sDCNsqLmVmn016CxQxgkKSBkj4BjAGmtXGezMw6jXbRDBURKyX9G3AX0AW4IiLmVjALrd601Q64zJ2Dy9w5tLjMiqi2m8rNzKzatJdmKDMza0MOFmZmVpSDRSOStpR0n6RnJM2VdGJK31TSPZKeT38/1dZ5bU2Sukh6QtKf03KHLi+ApJ6SbpL0bHq/v9CRyy3p39Nneo6k6yR174jllXSFpMWS5uSkNVlOSRPTMELPSTqgbXLdfE2U97z0uX5S0i2Seuasa1Z5HSzWthL4YURsC+wBHJ+GFjkFmB4Rg4DpabkjORF4Jme5o5cX4CLgzojYBtiRrPwdstyS+gEnALURsR3ZhSJj6JjlvRI4sFFa3nKm/+0xwJC0z6VpeKH25ErWLu89wHYRsQPwf8BEaFl5HSwaiYhFEfF4er6M7AukH9nwIlPSZlOA0W2SwTKQ1B8YAVyek9xhywsgaRNgH+D3ABHxz4h4i45d7vWBHpLWBzYgu1epw5U3Iu4H3myU3FQ5RwHXR8SKiHgRmEc2vFC7ka+8EXF3RKxMi4+Q3ZsGLSivg0UBkmqAnYFHgS0iYhFkAQXo3YZZa23/CfwYWJ2T1pHLC/BZYAnwh9T8drmkDemg5Y6IV4HzgZeBRcDbEXE3HbS8eTRVznxDCfWrcN7K7VjgjvS82eV1sGiCpI2APwETIuKdts5PuUg6GFgcETPbOi8Vtj6wC3BZROwMvEvHaILJK7XRjwIGAn2BDSUd2ba5qgolDSXUXkk6jaxp/dqGpDyblVReB4s8JHUlCxTXRsTNKfl1SX3S+j7A4rbKXyvbExgpaQHZaL77SrqGjlveBvVAfUQ8mpZvIgseHbXc+wMvRsSSiPgQuBn4Ih23vI01Vc4OO5SQpLHAwcAR8dENdc0ur4NFI5JE1o79TERckLNqGjA2PR8L3FbpvJVDREyMiP4RUUPW8fXXiDiSDlreBhHxGvCKpK1T0n7A03Tccr8M7CFpg/QZ34+sP66jlrexpso5DRgjqZukgcAg4LE2yF+rknQgcDIwMiLey1nV/PJGhB85D2AvsmrZk8Cs9DgI2IzsKorn099N2zqvZSj7MODP6XlnKO9OQF16r28FPtWRyw2cCTwLzAGuBrp1xPIC15H1y3xI9kt6XKFyAqcB84HngK+0df5bqbzzyPomGr7D/rul5fVwH2ZmVpSboczMrCgHCzMzK8rBwszMinKwMDOzohwszMysKAcL61AkhaSrc5bXl7SkYTTdZhyvp6Tv5ywPa+pYkv4mqXYdjr2RpN9Kmp9Gg71f0u6SanJHEDWrBg4W1tG8C2wnqUda/hfg1RYcryfw/WIbNdPlZAPADYqIIcDRwOYtPWgaKNCsVTlYWEd0B9kougCHk920BKyZ1+DWNM7/I5J2SOmT0rwAf5P0gqQT0i7nAltJmiXpvJS2Uc48GNemO6LJOcc4SRfmLH9H0gWNttkK2B34SUSsBoiIFyLi9rRJF0m/SzWOuxuCXzrWDEmzJf1J0gYp/UpJF0i6D/ilpK1S+WZI+pmk5TnnPimlPynpzJS2oaTb03HnSPpmc19865gcLKwjup5sSIPuwA5kowY3OBN4IrJx/k8FrspZtw1wANmQzWekMcJOAeZHxE4RcVLabmdgAjCYbPTaPfOcf2TaH+AY4A+NthkCzIqIVU2UYRBwSapxvAV8LaXfHBG7RUTD/Bvjcvb5PLB/RPyQbK6OiyJiN3LG/pE0PB17KNkd7LtK2odsboOFEbFjZPNd3NlEvqyTcrCwDicingRqyGoVf2m0ei+yoS6IiL8Cm0n6ZFp3e2Tj/C8lG2huiyZO8VhE1Kcawax0rtzzvwv8FThY0jZA14h4ah2L8WJEzErPZ+acYztJD0h6CjiCLOg0uDEn+HwBuDE9/2PONsPT4wngcbIAOQh4Cthf0i8l7R0Rb69jfq2Dc9umdVTTyOZvGEY2LlCDQkM0r8hJW0XT/x+lbHc5Wc3lWdauVQDMBXaUtF5DM1SRczT0wVwJjI6I2ZKOJitfg3ebyG8uAedExG/XWiHtSjYO2jmS7o6In5VwPOskXLOwjuoK4Gd5ftHfT/aLHEnDgKVReL6SZcDG63ryyIY+3xL4Fjl9Jjnr55MNYnhmQ5+HpEGSRhU59MbAotTEdUSB7R7ho6arMTnpdwHHpvlakNRPUm9JfYH3IuIasiC7S7EyWufimoV1SBFRT9Zu39gkstnxngTe46Nhq5s6zhuSHkqXst4B3F5o+0amAjtFxD+aWP+vwK+BeZLeA94ATmpi2wY/JeuDeYms6aipQDYBuEbSD1Oe34Zsuk1J2wIPpxi1HDgS+BxwnqTVZKOXHldKAa3z8KizZmWS7se4MCKmt8G5NwDej4iQNAY4PCKK1VrMmuSahVkrk9STbEKZ2W0RKJJdgd+kJq63yOZhNms21yzMzKwod3CbmVlRDhZmZlaUg4WZmRXlYGFmZkU5WJiZWVH/D2LMTlpgmjSqAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "mc_churn_no = df1[df1.Churn=='No'].MonthlyCharges      \n",
    "mc_churn_yes = df1[df1.Churn=='Yes'].MonthlyCharges      \n",
    "\n",
    "plt.xlabel(\"Monthly Charges\")\n",
    "plt.ylabel(\"Number Of Customers\")\n",
    "plt.title(\"Customer Churn Prediction Visualiztion\")\n",
    "\n",
    "blood_sugar_men = [113, 85, 90, 150, 149, 88, 93, 115, 135, 80, 77, 82, 129]\n",
    "blood_sugar_women = [67, 98, 89, 120, 133, 150, 84, 69, 89, 79, 120, 112, 100]\n",
    "\n",
    "plt.hist([mc_churn_yes, mc_churn_no], rwidth=0.95, color=['green','red'],label=['Churn=Yes','Churn=No'])\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Many of the columns are yes, no etc. Let's print unique values in object columns to see data values**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "def print_unique_col_values(df):\n",
    "       for column in df:\n",
    "            if df[column].dtypes=='object':\n",
    "                print(f'{column}: {df[column].unique()}') "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "gender: ['Female' 'Male']\n",
      "Partner: ['Yes' 'No']\n",
      "Dependents: ['No' 'Yes']\n",
      "PhoneService: ['No' 'Yes']\n",
      "MultipleLines: ['No phone service' 'No' 'Yes']\n",
      "InternetService: ['DSL' 'Fiber optic' 'No']\n",
      "OnlineSecurity: ['No' 'Yes' 'No internet service']\n",
      "OnlineBackup: ['Yes' 'No' 'No internet service']\n",
      "DeviceProtection: ['No' 'Yes' 'No internet service']\n",
      "TechSupport: ['No' 'Yes' 'No internet service']\n",
      "StreamingTV: ['No' 'Yes' 'No internet service']\n",
      "StreamingMovies: ['No' 'Yes' 'No internet service']\n",
      "Contract: ['Month-to-month' 'One year' 'Two year']\n",
      "PaperlessBilling: ['Yes' 'No']\n",
      "PaymentMethod: ['Electronic check' 'Mailed check' 'Bank transfer (automatic)'\n",
      " 'Credit card (automatic)']\n",
      "Churn: ['No' 'Yes']\n"
     ]
    }
   ],
   "source": [
    "print_unique_col_values(df1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Some of the columns have no internet service or no phone service, that can be replaced with a simple No**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "df1.replace('No internet service','No',inplace=True)\n",
    "df1.replace('No phone service','No',inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "gender: ['Female' 'Male']\n",
      "Partner: ['Yes' 'No']\n",
      "Dependents: ['No' 'Yes']\n",
      "PhoneService: ['No' 'Yes']\n",
      "MultipleLines: ['No' 'Yes']\n",
      "InternetService: ['DSL' 'Fiber optic' 'No']\n",
      "OnlineSecurity: ['No' 'Yes']\n",
      "OnlineBackup: ['Yes' 'No']\n",
      "DeviceProtection: ['No' 'Yes']\n",
      "TechSupport: ['No' 'Yes']\n",
      "StreamingTV: ['No' 'Yes']\n",
      "StreamingMovies: ['No' 'Yes']\n",
      "Contract: ['Month-to-month' 'One year' 'Two year']\n",
      "PaperlessBilling: ['Yes' 'No']\n",
      "PaymentMethod: ['Electronic check' 'Mailed check' 'Bank transfer (automatic)'\n",
      " 'Credit card (automatic)']\n",
      "Churn: ['No' 'Yes']\n"
     ]
    }
   ],
   "source": [
    "print_unique_col_values(df1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Convert Yes and No to 1 or 0**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "yes_no_columns = ['Partner','Dependents','PhoneService','MultipleLines','OnlineSecurity','OnlineBackup',\n",
    "                  'DeviceProtection','TechSupport','StreamingTV','StreamingMovies','PaperlessBilling','Churn']\n",
    "for col in yes_no_columns:\n",
    "    df1[col].replace({'Yes': 1,'No': 0},inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "gender: ['Female' 'Male']\n",
      "SeniorCitizen: [0 1]\n",
      "Partner: [1 0]\n",
      "Dependents: [0 1]\n",
      "tenure: [ 1 34  2 45  8 22 10 28 62 13 16 58 49 25 69 52 71 21 12 30 47 72 17 27\n",
      "  5 46 11 70 63 43 15 60 18 66  9  3 31 50 64 56  7 42 35 48 29 65 38 68\n",
      " 32 55 37 36 41  6  4 33 67 23 57 61 14 20 53 40 59 24 44 19 54 51 26 39]\n",
      "PhoneService: [0 1]\n",
      "MultipleLines: [0 1]\n",
      "InternetService: ['DSL' 'Fiber optic' 'No']\n",
      "OnlineSecurity: [0 1]\n",
      "OnlineBackup: [1 0]\n",
      "DeviceProtection: [0 1]\n",
      "TechSupport: [0 1]\n",
      "StreamingTV: [0 1]\n",
      "StreamingMovies: [0 1]\n",
      "Contract: ['Month-to-month' 'One year' 'Two year']\n",
      "PaperlessBilling: [1 0]\n",
      "PaymentMethod: ['Electronic check' 'Mailed check' 'Bank transfer (automatic)'\n",
      " 'Credit card (automatic)']\n",
      "MonthlyCharges: [29.85 56.95 53.85 ... 63.1  44.2  78.7 ]\n",
      "TotalCharges: [  29.85 1889.5   108.15 ...  346.45  306.6  6844.5 ]\n",
      "Churn: [0 1]\n"
     ]
    }
   ],
   "source": [
    "for col in df1:\n",
    "    print(f'{col}: {df1[col].unique()}') "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "df1['gender'].replace({'Female':1,'Male':0},inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 0], dtype=int64)"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1.gender.unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**One hot encoding for categorical columns**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['gender', 'SeniorCitizen', 'Partner', 'Dependents', 'tenure',\n",
       "       'PhoneService', 'MultipleLines', 'OnlineSecurity', 'OnlineBackup',\n",
       "       'DeviceProtection', 'TechSupport', 'StreamingTV', 'StreamingMovies',\n",
       "       'PaperlessBilling', 'MonthlyCharges', 'TotalCharges', 'Churn',\n",
       "       'InternetService_DSL', 'InternetService_Fiber optic',\n",
       "       'InternetService_No', 'Contract_Month-to-month', 'Contract_One year',\n",
       "       'Contract_Two year', 'PaymentMethod_Bank transfer (automatic)',\n",
       "       'PaymentMethod_Credit card (automatic)',\n",
       "       'PaymentMethod_Electronic check', 'PaymentMethod_Mailed check'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2 = pd.get_dummies(data=df1, columns=['InternetService','Contract','PaymentMethod'])\n",
    "df2.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>gender</th>\n",
       "      <th>SeniorCitizen</th>\n",
       "      <th>Partner</th>\n",
       "      <th>Dependents</th>\n",
       "      <th>tenure</th>\n",
       "      <th>PhoneService</th>\n",
       "      <th>MultipleLines</th>\n",
       "      <th>OnlineSecurity</th>\n",
       "      <th>OnlineBackup</th>\n",
       "      <th>DeviceProtection</th>\n",
       "      <th>...</th>\n",
       "      <th>InternetService_DSL</th>\n",
       "      <th>InternetService_Fiber optic</th>\n",
       "      <th>InternetService_No</th>\n",
       "      <th>Contract_Month-to-month</th>\n",
       "      <th>Contract_One year</th>\n",
       "      <th>Contract_Two year</th>\n",
       "      <th>PaymentMethod_Bank transfer (automatic)</th>\n",
       "      <th>PaymentMethod_Credit card (automatic)</th>\n",
       "      <th>PaymentMethod_Electronic check</th>\n",
       "      <th>PaymentMethod_Mailed check</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2548</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2047</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2499</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>41</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>951</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5893</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>12</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 27 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      gender  SeniorCitizen  Partner  Dependents  tenure  PhoneService  \\\n",
       "2548       0              1        0           0       8             1   \n",
       "2047       0              0        0           0       3             1   \n",
       "2499       0              0        0           1      41             1   \n",
       "951        1              0        0           0       8             1   \n",
       "5893       0              0        1           1      12             1   \n",
       "\n",
       "      MultipleLines  OnlineSecurity  OnlineBackup  DeviceProtection  ...  \\\n",
       "2548              0               0             0                 0  ...   \n",
       "2047              0               1             0                 1  ...   \n",
       "2499              0               1             1                 1  ...   \n",
       "951               0               0             1                 0  ...   \n",
       "5893              1               0             0                 0  ...   \n",
       "\n",
       "      InternetService_DSL  InternetService_Fiber optic  InternetService_No  \\\n",
       "2548                    0                            1                   0   \n",
       "2047                    1                            0                   0   \n",
       "2499                    1                            0                   0   \n",
       "951                     0                            1                   0   \n",
       "5893                    0                            0                   1   \n",
       "\n",
       "      Contract_Month-to-month  Contract_One year  Contract_Two year  \\\n",
       "2548                        1                  0                  0   \n",
       "2047                        1                  0                  0   \n",
       "2499                        0                  1                  0   \n",
       "951                         1                  0                  0   \n",
       "5893                        0                  0                  1   \n",
       "\n",
       "      PaymentMethod_Bank transfer (automatic)  \\\n",
       "2548                                        0   \n",
       "2047                                        0   \n",
       "2499                                        0   \n",
       "951                                         0   \n",
       "5893                                        1   \n",
       "\n",
       "      PaymentMethod_Credit card (automatic)  PaymentMethod_Electronic check  \\\n",
       "2548                                      1                               0   \n",
       "2047                                      0                               1   \n",
       "2499                                      0                               0   \n",
       "951                                       0                               1   \n",
       "5893                                      0                               0   \n",
       "\n",
       "      PaymentMethod_Mailed check  \n",
       "2548                           0  \n",
       "2047                           0  \n",
       "2499                           1  \n",
       "951                            0  \n",
       "5893                           0  \n",
       "\n",
       "[5 rows x 27 columns]"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2.sample(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "gender                                       int64\n",
       "SeniorCitizen                                int64\n",
       "Partner                                      int64\n",
       "Dependents                                   int64\n",
       "tenure                                       int64\n",
       "PhoneService                                 int64\n",
       "MultipleLines                                int64\n",
       "OnlineSecurity                               int64\n",
       "OnlineBackup                                 int64\n",
       "DeviceProtection                             int64\n",
       "TechSupport                                  int64\n",
       "StreamingTV                                  int64\n",
       "StreamingMovies                              int64\n",
       "PaperlessBilling                             int64\n",
       "MonthlyCharges                             float64\n",
       "TotalCharges                               float64\n",
       "Churn                                        int64\n",
       "InternetService_DSL                          uint8\n",
       "InternetService_Fiber optic                  uint8\n",
       "InternetService_No                           uint8\n",
       "Contract_Month-to-month                      uint8\n",
       "Contract_One year                            uint8\n",
       "Contract_Two year                            uint8\n",
       "PaymentMethod_Bank transfer (automatic)      uint8\n",
       "PaymentMethod_Credit card (automatic)        uint8\n",
       "PaymentMethod_Electronic check               uint8\n",
       "PaymentMethod_Mailed check                   uint8\n",
       "dtype: object"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "cols_to_scale = ['tenure','MonthlyCharges','TotalCharges']\n",
    "\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "scaler = MinMaxScaler()\n",
    "df2[cols_to_scale] = scaler.fit_transform(df2[cols_to_scale])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "gender: [1 0]\n",
      "SeniorCitizen: [0 1]\n",
      "Partner: [1 0]\n",
      "Dependents: [0 1]\n",
      "tenure: [0.         0.46478873 0.01408451 0.61971831 0.09859155 0.29577465\n",
      " 0.12676056 0.38028169 0.85915493 0.16901408 0.21126761 0.8028169\n",
      " 0.67605634 0.33802817 0.95774648 0.71830986 0.98591549 0.28169014\n",
      " 0.15492958 0.4084507  0.64788732 1.         0.22535211 0.36619718\n",
      " 0.05633803 0.63380282 0.14084507 0.97183099 0.87323944 0.5915493\n",
      " 0.1971831  0.83098592 0.23943662 0.91549296 0.11267606 0.02816901\n",
      " 0.42253521 0.69014085 0.88732394 0.77464789 0.08450704 0.57746479\n",
      " 0.47887324 0.66197183 0.3943662  0.90140845 0.52112676 0.94366197\n",
      " 0.43661972 0.76056338 0.50704225 0.49295775 0.56338028 0.07042254\n",
      " 0.04225352 0.45070423 0.92957746 0.30985915 0.78873239 0.84507042\n",
      " 0.18309859 0.26760563 0.73239437 0.54929577 0.81690141 0.32394366\n",
      " 0.6056338  0.25352113 0.74647887 0.70422535 0.35211268 0.53521127]\n",
      "PhoneService: [0 1]\n",
      "MultipleLines: [0 1]\n",
      "OnlineSecurity: [0 1]\n",
      "OnlineBackup: [1 0]\n",
      "DeviceProtection: [0 1]\n",
      "TechSupport: [0 1]\n",
      "StreamingTV: [0 1]\n",
      "StreamingMovies: [0 1]\n",
      "PaperlessBilling: [1 0]\n",
      "MonthlyCharges: [0.11542289 0.38507463 0.35422886 ... 0.44626866 0.25820896 0.60149254]\n",
      "TotalCharges: [0.0012751  0.21586661 0.01031041 ... 0.03780868 0.03321025 0.78764136]\n",
      "Churn: [0 1]\n",
      "InternetService_DSL: [1 0]\n",
      "InternetService_Fiber optic: [0 1]\n",
      "InternetService_No: [0 1]\n",
      "Contract_Month-to-month: [1 0]\n",
      "Contract_One year: [0 1]\n",
      "Contract_Two year: [0 1]\n",
      "PaymentMethod_Bank transfer (automatic): [0 1]\n",
      "PaymentMethod_Credit card (automatic): [0 1]\n",
      "PaymentMethod_Electronic check: [1 0]\n",
      "PaymentMethod_Mailed check: [0 1]\n"
     ]
    }
   ],
   "source": [
    "for col in df2:\n",
    "    print(f'{col}: {df2[col].unique()}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Train test split**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = df2.drop('Churn',axis='columns')\n",
    "y = testLabels = df2.Churn.astype(np.float32)\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=15, stratify=y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0    4130\n",
       "1.0    1495\n",
       "Name: Churn, dtype: int64"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0    5163\n",
       "1.0    1869\n",
       "Name: Churn, dtype: int64"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.7624398073836276"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "5163/1869"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0    1033\n",
       "1.0     374\n",
       "Name: Churn, dtype: int64"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_test.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(5625, 26)"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1407, 26)"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>gender</th>\n",
       "      <th>SeniorCitizen</th>\n",
       "      <th>Partner</th>\n",
       "      <th>Dependents</th>\n",
       "      <th>tenure</th>\n",
       "      <th>PhoneService</th>\n",
       "      <th>MultipleLines</th>\n",
       "      <th>OnlineSecurity</th>\n",
       "      <th>OnlineBackup</th>\n",
       "      <th>DeviceProtection</th>\n",
       "      <th>...</th>\n",
       "      <th>InternetService_DSL</th>\n",
       "      <th>InternetService_Fiber optic</th>\n",
       "      <th>InternetService_No</th>\n",
       "      <th>Contract_Month-to-month</th>\n",
       "      <th>Contract_One year</th>\n",
       "      <th>Contract_Two year</th>\n",
       "      <th>PaymentMethod_Bank transfer (automatic)</th>\n",
       "      <th>PaymentMethod_Credit card (automatic)</th>\n",
       "      <th>PaymentMethod_Electronic check</th>\n",
       "      <th>PaymentMethod_Mailed check</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>684</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2446</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.239437</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1680</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.774648</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2220</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2842</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.042254</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5131</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.605634</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4372</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4687</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.295775</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2385</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.028169</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6773</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.507042</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10 rows × 26 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      gender  SeniorCitizen  Partner  Dependents    tenure  PhoneService  \\\n",
       "684        1              0        0           0  0.000000             1   \n",
       "2446       1              0        0           0  0.239437             1   \n",
       "1680       0              0        1           1  0.774648             1   \n",
       "2220       0              0        1           0  1.000000             1   \n",
       "2842       1              0        0           0  0.042254             0   \n",
       "5131       1              0        1           0  0.605634             1   \n",
       "4372       0              0        1           1  0.000000             1   \n",
       "4687       0              0        1           0  0.295775             1   \n",
       "2385       1              0        0           1  0.028169             0   \n",
       "6773       0              1        1           0  0.507042             1   \n",
       "\n",
       "      MultipleLines  OnlineSecurity  OnlineBackup  DeviceProtection  ...  \\\n",
       "684               0               0             0                 0  ...   \n",
       "2446              1               0             1                 0  ...   \n",
       "1680              1               0             0                 0  ...   \n",
       "2220              0               1             1                 0  ...   \n",
       "2842              0               1             0                 1  ...   \n",
       "5131              1               0             0                 0  ...   \n",
       "4372              0               0             0                 0  ...   \n",
       "4687              0               0             0                 1  ...   \n",
       "2385              0               0             1                 0  ...   \n",
       "6773              1               0             0                 0  ...   \n",
       "\n",
       "      InternetService_DSL  InternetService_Fiber optic  InternetService_No  \\\n",
       "684                     0                            1                   0   \n",
       "2446                    0                            1                   0   \n",
       "1680                    0                            0                   1   \n",
       "2220                    1                            0                   0   \n",
       "2842                    1                            0                   0   \n",
       "5131                    0                            1                   0   \n",
       "4372                    0                            0                   1   \n",
       "4687                    0                            1                   0   \n",
       "2385                    1                            0                   0   \n",
       "6773                    0                            1                   0   \n",
       "\n",
       "      Contract_Month-to-month  Contract_One year  Contract_Two year  \\\n",
       "684                         1                  0                  0   \n",
       "2446                        1                  0                  0   \n",
       "1680                        0                  1                  0   \n",
       "2220                        0                  0                  1   \n",
       "2842                        1                  0                  0   \n",
       "5131                        1                  0                  0   \n",
       "4372                        1                  0                  0   \n",
       "4687                        1                  0                  0   \n",
       "2385                        1                  0                  0   \n",
       "6773                        1                  0                  0   \n",
       "\n",
       "      PaymentMethod_Bank transfer (automatic)  \\\n",
       "684                                         0   \n",
       "2446                                        0   \n",
       "1680                                        0   \n",
       "2220                                        1   \n",
       "2842                                        0   \n",
       "5131                                        0   \n",
       "4372                                        1   \n",
       "4687                                        0   \n",
       "2385                                        0   \n",
       "6773                                        0   \n",
       "\n",
       "      PaymentMethod_Credit card (automatic)  PaymentMethod_Electronic check  \\\n",
       "684                                       0                               0   \n",
       "2446                                      1                               0   \n",
       "1680                                      0                               0   \n",
       "2220                                      0                               0   \n",
       "2842                                      0                               0   \n",
       "5131                                      0                               1   \n",
       "4372                                      0                               0   \n",
       "4687                                      1                               0   \n",
       "2385                                      0                               0   \n",
       "6773                                      0                               1   \n",
       "\n",
       "      PaymentMethod_Mailed check  \n",
       "684                            1  \n",
       "2446                           0  \n",
       "1680                           1  \n",
       "2220                           0  \n",
       "2842                           1  \n",
       "5131                           0  \n",
       "4372                           0  \n",
       "4687                           0  \n",
       "2385                           1  \n",
       "6773                           0  \n",
       "\n",
       "[10 rows x 26 columns]"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "26"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(X_train.columns)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Build a model (ANN) in tensorflow/keras**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow_addons import losses"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "from sklearn.metrics import confusion_matrix , classification_report"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "def ANN(X_train, y_train, X_test, y_test, loss, weights):\n",
    "    model = keras.Sequential([\n",
    "        keras.layers.Dense(26, input_dim=26, activation='relu'),\n",
    "        keras.layers.Dense(15, activation='relu'),\n",
    "        keras.layers.Dense(1, activation='sigmoid')\n",
    "    ])\n",
    "\n",
    "    model.compile(optimizer='adam', loss=loss, metrics=['accuracy'])\n",
    "    \n",
    "    if weights == -1:\n",
    "        model.fit(X_train, y_train, epochs=100)\n",
    "    else:\n",
    "        model.fit(X_train, y_train, epochs=100, class_weight = weights)\n",
    "    \n",
    "    print(model.evaluate(X_test, y_test))\n",
    "    \n",
    "    y_preds = model.predict(X_test)\n",
    "    y_preds = np.round(y_preds)\n",
    "    \n",
    "    print(\"Classification Report: \\n\", classification_report(y_test, y_preds))\n",
    "    \n",
    "    return y_preds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.4940 - accuracy: 0.7593\n",
      "Epoch 2/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.4263 - accuracy: 0.7988\n",
      "Epoch 3/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.4201 - accuracy: 0.7984\n",
      "Epoch 4/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.4177 - accuracy: 0.7995\n",
      "Epoch 5/100\n",
      "176/176 [==============================] - 0s 2ms/step - loss: 0.4121 - accuracy: 0.8041\n",
      "Epoch 6/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.4113 - accuracy: 0.8060\n",
      "Epoch 7/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.4088 - accuracy: 0.8080\n",
      "Epoch 8/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.4075 - accuracy: 0.8064\n",
      "Epoch 9/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.4058 - accuracy: 0.8059\n",
      "Epoch 10/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.4047 - accuracy: 0.8089\n",
      "Epoch 11/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.4039 - accuracy: 0.8071\n",
      "Epoch 12/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.4023 - accuracy: 0.8105\n",
      "Epoch 13/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.4014 - accuracy: 0.8110\n",
      "Epoch 14/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3996 - accuracy: 0.8068\n",
      "Epoch 15/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3990 - accuracy: 0.8094\n",
      "Epoch 16/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3974 - accuracy: 0.8110\n",
      "Epoch 17/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3961 - accuracy: 0.8108\n",
      "Epoch 18/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3944 - accuracy: 0.8151\n",
      "Epoch 19/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3951 - accuracy: 0.8119\n",
      "Epoch 20/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3932 - accuracy: 0.8119\n",
      "Epoch 21/100\n",
      "176/176 [==============================] - 0s 2ms/step - loss: 0.3917 - accuracy: 0.8171\n",
      "Epoch 22/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3918 - accuracy: 0.8153\n",
      "Epoch 23/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3908 - accuracy: 0.8165\n",
      "Epoch 24/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3901 - accuracy: 0.8135\n",
      "Epoch 25/100\n",
      "176/176 [==============================] - 0s 2ms/step - loss: 0.3891 - accuracy: 0.8130\n",
      "Epoch 26/100\n",
      "176/176 [==============================] - 0s 2ms/step - loss: 0.3876 - accuracy: 0.8158\n",
      "Epoch 27/100\n",
      "176/176 [==============================] - 0s 2ms/step - loss: 0.3871 - accuracy: 0.8183\n",
      "Epoch 28/100\n",
      "176/176 [==============================] - 0s 2ms/step - loss: 0.3860 - accuracy: 0.8190\n",
      "Epoch 29/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3868 - accuracy: 0.8180\n",
      "Epoch 30/100\n",
      "176/176 [==============================] - 0s 2ms/step - loss: 0.3843 - accuracy: 0.8181\n",
      "Epoch 31/100\n",
      "176/176 [==============================] - 0s 2ms/step - loss: 0.3845 - accuracy: 0.8155\n",
      "Epoch 32/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3828 - accuracy: 0.8213\n",
      "Epoch 33/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3836 - accuracy: 0.8203\n",
      "Epoch 34/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3822 - accuracy: 0.8180\n",
      "Epoch 35/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3810 - accuracy: 0.8194\n",
      "Epoch 36/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3805 - accuracy: 0.8197\n",
      "Epoch 37/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3807 - accuracy: 0.8204\n",
      "Epoch 38/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3799 - accuracy: 0.8187\n",
      "Epoch 39/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3778 - accuracy: 0.8229\n",
      "Epoch 40/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3768 - accuracy: 0.8210\n",
      "Epoch 41/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3762 - accuracy: 0.8199\n",
      "Epoch 42/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3758 - accuracy: 0.8226\n",
      "Epoch 43/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3767 - accuracy: 0.8233\n",
      "Epoch 44/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3753 - accuracy: 0.8260\n",
      "Epoch 45/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3732 - accuracy: 0.8222\n",
      "Epoch 46/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3740 - accuracy: 0.8235\n",
      "Epoch 47/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3729 - accuracy: 0.8244\n",
      "Epoch 48/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3723 - accuracy: 0.8249\n",
      "Epoch 49/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3721 - accuracy: 0.8254\n",
      "Epoch 50/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3710 - accuracy: 0.8265\n",
      "Epoch 51/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3712 - accuracy: 0.8265\n",
      "Epoch 52/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3688 - accuracy: 0.8274\n",
      "Epoch 53/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3692 - accuracy: 0.8242\n",
      "Epoch 54/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3693 - accuracy: 0.8260\n",
      "Epoch 55/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3684 - accuracy: 0.8242\n",
      "Epoch 56/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3680 - accuracy: 0.8244\n",
      "Epoch 57/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3664 - accuracy: 0.8272\n",
      "Epoch 58/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3670 - accuracy: 0.8238\n",
      "Epoch 59/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3670 - accuracy: 0.8238\n",
      "Epoch 60/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3661 - accuracy: 0.8265\n",
      "Epoch 61/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3654 - accuracy: 0.8288\n",
      "Epoch 62/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3663 - accuracy: 0.8290\n",
      "Epoch 63/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3644 - accuracy: 0.8265\n",
      "Epoch 64/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3635 - accuracy: 0.8290\n",
      "Epoch 65/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3630 - accuracy: 0.8272\n",
      "Epoch 66/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3615 - accuracy: 0.8279\n",
      "Epoch 67/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3630 - accuracy: 0.8274\n",
      "Epoch 68/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3625 - accuracy: 0.8265\n",
      "Epoch 69/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3612 - accuracy: 0.8277\n",
      "Epoch 70/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3607 - accuracy: 0.8284\n",
      "Epoch 71/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3604 - accuracy: 0.8309\n",
      "Epoch 72/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3604 - accuracy: 0.8295\n",
      "Epoch 73/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3590 - accuracy: 0.8284\n",
      "Epoch 74/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3586 - accuracy: 0.8300\n",
      "Epoch 75/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3583 - accuracy: 0.8300\n",
      "Epoch 76/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3575 - accuracy: 0.8318\n",
      "Epoch 77/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3575 - accuracy: 0.8334\n",
      "Epoch 78/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3563 - accuracy: 0.8345\n",
      "Epoch 79/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3580 - accuracy: 0.8309\n",
      "Epoch 80/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3554 - accuracy: 0.8320\n",
      "Epoch 81/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3563 - accuracy: 0.8322\n",
      "Epoch 82/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3546 - accuracy: 0.8345\n",
      "Epoch 83/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3562 - accuracy: 0.8293\n",
      "Epoch 84/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3536 - accuracy: 0.8322\n",
      "Epoch 85/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3544 - accuracy: 0.8308\n",
      "Epoch 86/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3522 - accuracy: 0.8350\n",
      "Epoch 87/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3527 - accuracy: 0.8329\n",
      "Epoch 88/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3517 - accuracy: 0.8329\n",
      "Epoch 89/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3513 - accuracy: 0.8334\n",
      "Epoch 90/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3515 - accuracy: 0.8332\n",
      "Epoch 91/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3509 - accuracy: 0.8354\n",
      "Epoch 92/100\n",
      "176/176 [==============================] - 0s 2ms/step - loss: 0.3506 - accuracy: 0.8375\n",
      "Epoch 93/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3503 - accuracy: 0.8336\n",
      "Epoch 94/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3490 - accuracy: 0.8338\n",
      "Epoch 95/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3481 - accuracy: 0.8331\n",
      "Epoch 96/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3481 - accuracy: 0.8377\n",
      "Epoch 97/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3473 - accuracy: 0.8380\n",
      "Epoch 98/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3471 - accuracy: 0.8359\n",
      "Epoch 99/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3475 - accuracy: 0.8343\n",
      "Epoch 100/100\n",
      "176/176 [==============================] - 0s 1ms/step - loss: 0.3479 - accuracy: 0.8334\n",
      "44/44 [==============================] - 0s 1ms/step - loss: 0.4865 - accuracy: 0.7861\n",
      "[0.48652538657188416, 0.7860696315765381]\n",
      "Classification Report: \n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "         0.0       0.84      0.88      0.86      1033\n",
      "         1.0       0.61      0.54      0.57       374\n",
      "\n",
      "    accuracy                           0.79      1407\n",
      "   macro avg       0.73      0.71      0.71      1407\n",
      "weighted avg       0.78      0.79      0.78      1407\n",
      "\n"
     ]
    }
   ],
   "source": [
    "y_preds = ANN(X_train, y_train, X_test, y_test, 'binary_crossentropy', -1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Mitigating Skewdness of Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Method 1: Undersampling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "reference: https://www.kaggle.com/rafjaa/resampling-strategies-for-imbalanced-datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Class count\n",
    "count_class_0, count_class_1 = df1.Churn.value_counts()\n",
    "\n",
    "# Divide by class\n",
    "df_class_0 = df2[df2['Churn'] == 0]\n",
    "df_class_1 = df2[df2['Churn'] == 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Random under-sampling:\n",
      "1    1869\n",
      "0    1869\n",
      "Name: Churn, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "# Undersample 0-class and concat the DataFrames of both class\n",
    "df_class_0_under = df_class_0.sample(count_class_1)\n",
    "df_test_under = pd.concat([df_class_0_under, df_class_1], axis=0)\n",
    "\n",
    "print('Random under-sampling:')\n",
    "print(df_test_under.Churn.value_counts())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = df_test_under.drop('Churn',axis='columns')\n",
    "y = df_test_under['Churn']\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=15, stratify=y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1    1495\n",
       "0    1495\n",
       "Name: Churn, dtype: int64"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Number of classes in training Data\n",
    "y_train.value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Printing Classification in the last, Scroll down till the last epoch to watch the classification report**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.5709 - accuracy: 0.7375\n",
      "Epoch 2/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.5009 - accuracy: 0.7656\n",
      "Epoch 3/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4873 - accuracy: 0.7692\n",
      "Epoch 4/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4828 - accuracy: 0.7669\n",
      "Epoch 5/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4808 - accuracy: 0.7652\n",
      "Epoch 6/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4781 - accuracy: 0.7706\n",
      "Epoch 7/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4763 - accuracy: 0.7716\n",
      "Epoch 8/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4739 - accuracy: 0.7709\n",
      "Epoch 9/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4721 - accuracy: 0.7759\n",
      "Epoch 10/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4717 - accuracy: 0.7736\n",
      "Epoch 11/100\n",
      "94/94 [==============================] - 0s 2ms/step - loss: 0.4690 - accuracy: 0.7756\n",
      "Epoch 12/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4676 - accuracy: 0.7742\n",
      "Epoch 13/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4682 - accuracy: 0.7709\n",
      "Epoch 14/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4664 - accuracy: 0.7786\n",
      "Epoch 15/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4644 - accuracy: 0.7793\n",
      "Epoch 16/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4626 - accuracy: 0.7809\n",
      "Epoch 17/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4620 - accuracy: 0.7756\n",
      "Epoch 18/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4604 - accuracy: 0.7773\n",
      "Epoch 19/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4585 - accuracy: 0.7799\n",
      "Epoch 20/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4578 - accuracy: 0.7789\n",
      "Epoch 21/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4566 - accuracy: 0.7836\n",
      "Epoch 22/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4551 - accuracy: 0.7866\n",
      "Epoch 23/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4529 - accuracy: 0.7796\n",
      "Epoch 24/100\n",
      "94/94 [==============================] - 0s 2ms/step - loss: 0.4525 - accuracy: 0.7826\n",
      "Epoch 25/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4525 - accuracy: 0.7870\n",
      "Epoch 26/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4501 - accuracy: 0.7826\n",
      "Epoch 27/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4480 - accuracy: 0.7843\n",
      "Epoch 28/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4505 - accuracy: 0.7829\n",
      "Epoch 29/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4462 - accuracy: 0.7839\n",
      "Epoch 30/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4448 - accuracy: 0.7893\n",
      "Epoch 31/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4423 - accuracy: 0.7883\n",
      "Epoch 32/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4422 - accuracy: 0.7870\n",
      "Epoch 33/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4402 - accuracy: 0.7890\n",
      "Epoch 34/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4394 - accuracy: 0.7923\n",
      "Epoch 35/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4395 - accuracy: 0.7930\n",
      "Epoch 36/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4389 - accuracy: 0.7910\n",
      "Epoch 37/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4347 - accuracy: 0.7957\n",
      "Epoch 38/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4343 - accuracy: 0.7963\n",
      "Epoch 39/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4328 - accuracy: 0.7936\n",
      "Epoch 40/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4322 - accuracy: 0.7953\n",
      "Epoch 41/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4302 - accuracy: 0.7930\n",
      "Epoch 42/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4299 - accuracy: 0.7977\n",
      "Epoch 43/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4292 - accuracy: 0.7930\n",
      "Epoch 44/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4270 - accuracy: 0.7967\n",
      "Epoch 45/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4266 - accuracy: 0.8003\n",
      "Epoch 46/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4269 - accuracy: 0.7946\n",
      "Epoch 47/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4241 - accuracy: 0.7963\n",
      "Epoch 48/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4233 - accuracy: 0.8017\n",
      "Epoch 49/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4221 - accuracy: 0.7987\n",
      "Epoch 50/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4202 - accuracy: 0.7970\n",
      "Epoch 51/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4200 - accuracy: 0.7993\n",
      "Epoch 52/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4187 - accuracy: 0.7997\n",
      "Epoch 53/100\n",
      "94/94 [==============================] - 0s 2ms/step - loss: 0.4177 - accuracy: 0.8037\n",
      "Epoch 54/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4168 - accuracy: 0.8043\n",
      "Epoch 55/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4161 - accuracy: 0.8013\n",
      "Epoch 56/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4175 - accuracy: 0.8057\n",
      "Epoch 57/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4140 - accuracy: 0.8033\n",
      "Epoch 58/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4125 - accuracy: 0.8057\n",
      "Epoch 59/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4120 - accuracy: 0.8054\n",
      "Epoch 60/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4130 - accuracy: 0.8070\n",
      "Epoch 61/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4098 - accuracy: 0.8087\n",
      "Epoch 62/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4107 - accuracy: 0.8077\n",
      "Epoch 63/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4083 - accuracy: 0.8017\n",
      "Epoch 64/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4074 - accuracy: 0.8084\n",
      "Epoch 65/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4079 - accuracy: 0.8043\n",
      "Epoch 66/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4064 - accuracy: 0.8060\n",
      "Epoch 67/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4041 - accuracy: 0.8094\n",
      "Epoch 68/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4060 - accuracy: 0.8120\n",
      "Epoch 69/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4027 - accuracy: 0.8161\n",
      "Epoch 70/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4020 - accuracy: 0.8140\n",
      "Epoch 71/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4013 - accuracy: 0.8104\n",
      "Epoch 72/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4015 - accuracy: 0.8117\n",
      "Epoch 73/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4016 - accuracy: 0.8161\n",
      "Epoch 74/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3985 - accuracy: 0.8147\n",
      "Epoch 75/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3975 - accuracy: 0.8130\n",
      "Epoch 76/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3976 - accuracy: 0.8107\n",
      "Epoch 77/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3988 - accuracy: 0.8154\n",
      "Epoch 78/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3983 - accuracy: 0.8130\n",
      "Epoch 79/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3957 - accuracy: 0.8154\n",
      "Epoch 80/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3926 - accuracy: 0.8151\n",
      "Epoch 81/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3934 - accuracy: 0.8171\n",
      "Epoch 82/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3923 - accuracy: 0.8167\n",
      "Epoch 83/100\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3910 - accuracy: 0.8161\n",
      "Epoch 84/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3908 - accuracy: 0.8201\n",
      "Epoch 85/100\n",
      "94/94 [==============================] - 0s 2ms/step - loss: 0.3898 - accuracy: 0.8191\n",
      "Epoch 86/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3885 - accuracy: 0.8184\n",
      "Epoch 87/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3921 - accuracy: 0.8174\n",
      "Epoch 88/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3883 - accuracy: 0.8204\n",
      "Epoch 89/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3863 - accuracy: 0.8234\n",
      "Epoch 90/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3856 - accuracy: 0.8231\n",
      "Epoch 91/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3846 - accuracy: 0.8204\n",
      "Epoch 92/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3847 - accuracy: 0.8211\n",
      "Epoch 93/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3831 - accuracy: 0.8214\n",
      "Epoch 94/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3829 - accuracy: 0.8247\n",
      "Epoch 95/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3820 - accuracy: 0.8221\n",
      "Epoch 96/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3827 - accuracy: 0.8237\n",
      "Epoch 97/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3819 - accuracy: 0.8227\n",
      "Epoch 98/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3805 - accuracy: 0.8271\n",
      "Epoch 99/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3802 - accuracy: 0.8261\n",
      "Epoch 100/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3781 - accuracy: 0.8264\n",
      "24/24 [==============================] - 0s 1ms/step - loss: 0.5917 - accuracy: 0.7246\n",
      "[0.5916573405265808, 0.7245989441871643]\n",
      "Classification Report: \n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "           0       0.74      0.68      0.71       374\n",
      "           1       0.71      0.76      0.74       374\n",
      "\n",
      "    accuracy                           0.72       748\n",
      "   macro avg       0.73      0.72      0.72       748\n",
      "weighted avg       0.73      0.72      0.72       748\n",
      "\n"
     ]
    }
   ],
   "source": [
    "y_preds = ANN(X_train, y_train, X_test, y_test, 'binary_crossentropy', -1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check classification report above. f1-score for minority class 1 improved from **0.57 to 0.76**. Score for class 0 reduced to 0.75 from 0.85 but that's ok. We have more generalized classifier which classifies both classes with similar prediction score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Method2: Oversampling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Random over-sampling:\n",
      "1    5163\n",
      "0    5163\n",
      "Name: Churn, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "# Oversample 1-class and concat the DataFrames of both classes\n",
    "df_class_1_over = df_class_1.sample(count_class_0, replace=True)\n",
    "df_test_over = pd.concat([df_class_0, df_class_1_over], axis=0)\n",
    "\n",
    "print('Random over-sampling:')\n",
    "print(df_test_over.Churn.value_counts())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = df_test_over.drop('Churn',axis='columns')\n",
    "y = df_test_over['Churn']\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=15, stratify=y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1    4130\n",
       "0    4130\n",
       "Name: Churn, dtype: int64"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Number of classes in training Data\n",
    "y_train.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.5655 - accuracy: 0.7068\n",
      "Epoch 2/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4985 - accuracy: 0.7529\n",
      "Epoch 3/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4913 - accuracy: 0.7607\n",
      "Epoch 4/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4854 - accuracy: 0.7650\n",
      "Epoch 5/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4814 - accuracy: 0.7655\n",
      "Epoch 6/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4787 - accuracy: 0.7688\n",
      "Epoch 7/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4769 - accuracy: 0.7703\n",
      "Epoch 8/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4742 - accuracy: 0.7714\n",
      "Epoch 9/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4717 - accuracy: 0.7703\n",
      "Epoch 10/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4693 - accuracy: 0.7748\n",
      "Epoch 11/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4676 - accuracy: 0.7745\n",
      "Epoch 12/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4665 - accuracy: 0.7769\n",
      "Epoch 13/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4631 - accuracy: 0.7805\n",
      "Epoch 14/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4612 - accuracy: 0.7776\n",
      "Epoch 15/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4597 - accuracy: 0.7824\n",
      "Epoch 16/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4578 - accuracy: 0.7795\n",
      "Epoch 17/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4556 - accuracy: 0.7809\n",
      "Epoch 18/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4544 - accuracy: 0.7806\n",
      "Epoch 19/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4524 - accuracy: 0.7843\n",
      "Epoch 20/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4505 - accuracy: 0.7827\n",
      "Epoch 21/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4484 - accuracy: 0.7862\n",
      "Epoch 22/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4454 - accuracy: 0.7858\n",
      "Epoch 23/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4441 - accuracy: 0.7900\n",
      "Epoch 24/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4427 - accuracy: 0.7916\n",
      "Epoch 25/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4402 - accuracy: 0.7904\n",
      "Epoch 26/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4385 - accuracy: 0.7941\n",
      "Epoch 27/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4377 - accuracy: 0.7944\n",
      "Epoch 28/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4348 - accuracy: 0.7948\n",
      "Epoch 29/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4335 - accuracy: 0.7962\n",
      "Epoch 30/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4313 - accuracy: 0.7977\n",
      "Epoch 31/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4301 - accuracy: 0.7988\n",
      "Epoch 32/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4290 - accuracy: 0.7983\n",
      "Epoch 33/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4271 - accuracy: 0.8023\n",
      "Epoch 34/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4253 - accuracy: 0.8023\n",
      "Epoch 35/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4245 - accuracy: 0.8039\n",
      "Epoch 36/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4225 - accuracy: 0.8012\n",
      "Epoch 37/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4215 - accuracy: 0.8048\n",
      "Epoch 38/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4197 - accuracy: 0.8025\n",
      "Epoch 39/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4183 - accuracy: 0.8065\n",
      "Epoch 40/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4174 - accuracy: 0.8035\n",
      "Epoch 41/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4157 - accuracy: 0.8061\n",
      "Epoch 42/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4144 - accuracy: 0.8051\n",
      "Epoch 43/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4131 - accuracy: 0.8068\n",
      "Epoch 44/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4109 - accuracy: 0.8087\n",
      "Epoch 45/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4117 - accuracy: 0.8084\n",
      "Epoch 46/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4090 - accuracy: 0.8114\n",
      "Epoch 47/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4080 - accuracy: 0.8126\n",
      "Epoch 48/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4059 - accuracy: 0.8111\n",
      "Epoch 49/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4067 - accuracy: 0.8087\n",
      "Epoch 50/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4037 - accuracy: 0.8120\n",
      "Epoch 51/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4044 - accuracy: 0.8111\n",
      "Epoch 52/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4008 - accuracy: 0.8125\n",
      "Epoch 53/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3982 - accuracy: 0.8116\n",
      "Epoch 54/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3986 - accuracy: 0.8153\n",
      "Epoch 55/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3970 - accuracy: 0.8142\n",
      "Epoch 56/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3944 - accuracy: 0.8182\n",
      "Epoch 57/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3945 - accuracy: 0.8138\n",
      "Epoch 58/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3908 - accuracy: 0.8203\n",
      "Epoch 59/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3898 - accuracy: 0.8177\n",
      "Epoch 60/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3908 - accuracy: 0.8195\n",
      "Epoch 61/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3892 - accuracy: 0.8189\n",
      "Epoch 62/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3875 - accuracy: 0.8189\n",
      "Epoch 63/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3866 - accuracy: 0.8202\n",
      "Epoch 64/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3848 - accuracy: 0.8208\n",
      "Epoch 65/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3846 - accuracy: 0.8211\n",
      "Epoch 66/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3830 - accuracy: 0.8223\n",
      "Epoch 67/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3814 - accuracy: 0.8223\n",
      "Epoch 68/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3799 - accuracy: 0.8247\n",
      "Epoch 69/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3787 - accuracy: 0.8243\n",
      "Epoch 70/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3787 - accuracy: 0.8262\n",
      "Epoch 71/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3781 - accuracy: 0.8265\n",
      "Epoch 72/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3746 - accuracy: 0.8293\n",
      "Epoch 73/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3753 - accuracy: 0.8248\n",
      "Epoch 74/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3749 - accuracy: 0.8274\n",
      "Epoch 75/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3737 - accuracy: 0.8268\n",
      "Epoch 76/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3720 - accuracy: 0.8291\n",
      "Epoch 77/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3722 - accuracy: 0.8308\n",
      "Epoch 78/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3713 - accuracy: 0.8285\n",
      "Epoch 79/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3713 - accuracy: 0.8289\n",
      "Epoch 80/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3688 - accuracy: 0.8316\n",
      "Epoch 81/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3677 - accuracy: 0.8338\n",
      "Epoch 82/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3674 - accuracy: 0.8322\n",
      "Epoch 83/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3671 - accuracy: 0.8316\n",
      "Epoch 84/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3669 - accuracy: 0.8300\n",
      "Epoch 85/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3656 - accuracy: 0.8322\n",
      "Epoch 86/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3653 - accuracy: 0.8352\n",
      "Epoch 87/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3641 - accuracy: 0.8355\n",
      "Epoch 88/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3639 - accuracy: 0.8341\n",
      "Epoch 89/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3616 - accuracy: 0.8360\n",
      "Epoch 90/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3614 - accuracy: 0.8379\n",
      "Epoch 91/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3613 - accuracy: 0.8345\n",
      "Epoch 92/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3595 - accuracy: 0.8362\n",
      "Epoch 93/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3586 - accuracy: 0.8364\n",
      "Epoch 94/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3584 - accuracy: 0.8364\n",
      "Epoch 95/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3586 - accuracy: 0.8392\n",
      "Epoch 96/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3582 - accuracy: 0.8378\n",
      "Epoch 97/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3555 - accuracy: 0.8370\n",
      "Epoch 98/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3562 - accuracy: 0.8402\n",
      "Epoch 99/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3543 - accuracy: 0.8397\n",
      "Epoch 100/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3543 - accuracy: 0.8400\n",
      "65/65 [==============================] - 0s 1ms/step - loss: 0.4601 - accuracy: 0.7851\n",
      "[0.4600695073604584, 0.7850919365882874]\n",
      "Classification Report: \n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "           0       0.84      0.70      0.77      1033\n",
      "           1       0.74      0.87      0.80      1033\n",
      "\n",
      "    accuracy                           0.79      2066\n",
      "   macro avg       0.79      0.79      0.78      2066\n",
      "weighted avg       0.79      0.79      0.78      2066\n",
      "\n"
     ]
    }
   ],
   "source": [
    "loss = keras.losses.BinaryCrossentropy()\n",
    "weights = -1\n",
    "y_preds = ANN(X_train, y_train, X_test, y_test, 'binary_crossentropy', -1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check classification report above. f1-score for minority class 1 improved from **0.57 to 0.76**. Score for class 0 reduced to 0.75 from 0.85 but that's ok. We have more generalized classifier which classifies both classes with similar prediction score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Method3: SMOTE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To install imbalanced-learn library use **pip install imbalanced-learn** command"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = df2.drop('Churn',axis='columns')\n",
    "y = df2['Churn']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1    5163\n",
       "0    5163\n",
       "Name: Churn, dtype: int64"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from imblearn.over_sampling import SMOTE\n",
    "\n",
    "smote = SMOTE(sampling_strategy='minority')\n",
    "X_sm, y_sm = smote.fit_sample(X, y)\n",
    "\n",
    "y_sm.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X_sm, y_sm, test_size=0.2, random_state=15, stratify=y_sm)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1    4130\n",
       "0    4130\n",
       "Name: Churn, dtype: int64"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Number of classes in training Data\n",
    "y_train.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.5222 - accuracy: 0.7470\n",
      "Epoch 2/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4716 - accuracy: 0.7712\n",
      "Epoch 3/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4627 - accuracy: 0.7798\n",
      "Epoch 4/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4552 - accuracy: 0.7824\n",
      "Epoch 5/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4518 - accuracy: 0.7852\n",
      "Epoch 6/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4472 - accuracy: 0.7860\n",
      "Epoch 7/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4443 - accuracy: 0.7904\n",
      "Epoch 8/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4396 - accuracy: 0.7953\n",
      "Epoch 9/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4367 - accuracy: 0.7944\n",
      "Epoch 10/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4350 - accuracy: 0.7961\n",
      "Epoch 11/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4290 - accuracy: 0.7985\n",
      "Epoch 12/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4271 - accuracy: 0.8008\n",
      "Epoch 13/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4228 - accuracy: 0.8059\n",
      "Epoch 14/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4198 - accuracy: 0.8058\n",
      "Epoch 15/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4192 - accuracy: 0.8061\n",
      "Epoch 16/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4133 - accuracy: 0.8126\n",
      "Epoch 17/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4124 - accuracy: 0.8120\n",
      "Epoch 18/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4081 - accuracy: 0.8151\n",
      "Epoch 19/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4056 - accuracy: 0.8165\n",
      "Epoch 20/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4029 - accuracy: 0.8178\n",
      "Epoch 21/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.4013 - accuracy: 0.8194\n",
      "Epoch 22/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3991 - accuracy: 0.8203\n",
      "Epoch 23/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3975 - accuracy: 0.8224\n",
      "Epoch 24/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3943 - accuracy: 0.8185\n",
      "Epoch 25/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3933 - accuracy: 0.8229\n",
      "Epoch 26/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3937 - accuracy: 0.8219\n",
      "Epoch 27/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3907 - accuracy: 0.8236\n",
      "Epoch 28/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3883 - accuracy: 0.8224\n",
      "Epoch 29/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3866 - accuracy: 0.8259\n",
      "Epoch 30/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3846 - accuracy: 0.8301\n",
      "Epoch 31/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3825 - accuracy: 0.8285\n",
      "Epoch 32/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3822 - accuracy: 0.8308\n",
      "Epoch 33/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3812 - accuracy: 0.8301\n",
      "Epoch 34/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3801 - accuracy: 0.8306\n",
      "Epoch 35/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3761 - accuracy: 0.8341\n",
      "Epoch 36/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3765 - accuracy: 0.8333\n",
      "Epoch 37/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3766 - accuracy: 0.8326\n",
      "Epoch 38/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3750 - accuracy: 0.8327\n",
      "Epoch 39/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3739 - accuracy: 0.8358\n",
      "Epoch 40/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3717 - accuracy: 0.8368\n",
      "Epoch 41/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3716 - accuracy: 0.8368\n",
      "Epoch 42/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3711 - accuracy: 0.8361\n",
      "Epoch 43/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3689 - accuracy: 0.8387\n",
      "Epoch 44/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3673 - accuracy: 0.8377\n",
      "Epoch 45/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3695 - accuracy: 0.8370\n",
      "Epoch 46/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3676 - accuracy: 0.8374\n",
      "Epoch 47/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3660 - accuracy: 0.8391\n",
      "Epoch 48/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3646 - accuracy: 0.8427\n",
      "Epoch 49/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3627 - accuracy: 0.8392\n",
      "Epoch 50/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3634 - accuracy: 0.8374\n",
      "Epoch 51/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3616 - accuracy: 0.8402\n",
      "Epoch 52/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3614 - accuracy: 0.8400\n",
      "Epoch 53/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3608 - accuracy: 0.8412\n",
      "Epoch 54/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3598 - accuracy: 0.8404\n",
      "Epoch 55/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3576 - accuracy: 0.8431\n",
      "Epoch 56/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3621 - accuracy: 0.8400\n",
      "Epoch 57/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3570 - accuracy: 0.8421\n",
      "Epoch 58/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3606 - accuracy: 0.8401\n",
      "Epoch 59/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3551 - accuracy: 0.8452\n",
      "Epoch 60/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3561 - accuracy: 0.8413\n",
      "Epoch 61/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3559 - accuracy: 0.8444\n",
      "Epoch 62/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3540 - accuracy: 0.8419\n",
      "Epoch 63/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3543 - accuracy: 0.8454\n",
      "Epoch 64/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3548 - accuracy: 0.8410\n",
      "Epoch 65/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3542 - accuracy: 0.8415\n",
      "Epoch 66/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3553 - accuracy: 0.8439\n",
      "Epoch 67/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3500 - accuracy: 0.8472\n",
      "Epoch 68/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3506 - accuracy: 0.8444\n",
      "Epoch 69/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3506 - accuracy: 0.8494\n",
      "Epoch 70/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3528 - accuracy: 0.8465\n",
      "Epoch 71/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3497 - accuracy: 0.8470\n",
      "Epoch 72/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3502 - accuracy: 0.8449\n",
      "Epoch 73/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3490 - accuracy: 0.8492\n",
      "Epoch 74/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3478 - accuracy: 0.8466\n",
      "Epoch 75/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3464 - accuracy: 0.8489\n",
      "Epoch 76/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3480 - accuracy: 0.8467\n",
      "Epoch 77/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3465 - accuracy: 0.8492\n",
      "Epoch 78/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3469 - accuracy: 0.8502\n",
      "Epoch 79/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3470 - accuracy: 0.8492\n",
      "Epoch 80/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3460 - accuracy: 0.8515\n",
      "Epoch 81/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3432 - accuracy: 0.8523\n",
      "Epoch 82/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3458 - accuracy: 0.8471\n",
      "Epoch 83/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3453 - accuracy: 0.8487\n",
      "Epoch 84/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3439 - accuracy: 0.8519\n",
      "Epoch 85/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3423 - accuracy: 0.8472\n",
      "Epoch 86/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3437 - accuracy: 0.8483\n",
      "Epoch 87/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3411 - accuracy: 0.8500\n",
      "Epoch 88/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3436 - accuracy: 0.8512\n",
      "Epoch 89/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3416 - accuracy: 0.8515\n",
      "Epoch 90/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3405 - accuracy: 0.8539\n",
      "Epoch 91/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3417 - accuracy: 0.8531\n",
      "Epoch 92/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3403 - accuracy: 0.8518\n",
      "Epoch 93/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3422 - accuracy: 0.8513\n",
      "Epoch 94/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3377 - accuracy: 0.8542\n",
      "Epoch 95/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3394 - accuracy: 0.8522\n",
      "Epoch 96/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3396 - accuracy: 0.8561\n",
      "Epoch 97/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3357 - accuracy: 0.8521\n",
      "Epoch 98/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3370 - accuracy: 0.8511\n",
      "Epoch 99/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3393 - accuracy: 0.8524\n",
      "Epoch 100/100\n",
      "259/259 [==============================] - 0s 1ms/step - loss: 0.3389 - accuracy: 0.8544\n",
      "65/65 [==============================] - 0s 1ms/step - loss: 0.4252 - accuracy: 0.8088\n",
      "[0.425162136554718, 0.8088092803955078]\n",
      "Classification Report: \n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "           0       0.84      0.76      0.80      1033\n",
      "           1       0.78      0.85      0.82      1033\n",
      "\n",
      "    accuracy                           0.81      2066\n",
      "   macro avg       0.81      0.81      0.81      2066\n",
      "weighted avg       0.81      0.81      0.81      2066\n",
      "\n"
     ]
    }
   ],
   "source": [
    "y_preds = ANN(X_train, y_train, X_test, y_test, 'binary_crossentropy', -1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "SMOT Oversampling increases f1 score of minority class 1 from **0.57 to 0.81 (huge improvement)** Also over all accuracy improves from 0.78 to 0.80"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Method4: Use of Ensemble with undersampling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    5163\n",
       "1    1869\n",
       "Name: Churn, dtype: int64"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2.Churn.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Regain Original features and labels\n",
    "X = df2.drop('Churn',axis='columns')\n",
    "y = df2['Churn']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=15, stratify=y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    4130\n",
       "1    1495\n",
       "Name: Churn, dtype: int64"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train.value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "model1 --> class1(1495) + class0(0, 1495)\n",
    "\n",
    "model2 --> class1(1495) + class0(1496, 2990)\n",
    "\n",
    "model3 --> class1(1495) + class0(2990, 4130)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [],
   "source": [
    "df3 = X_train.copy()\n",
    "df3['Churn'] = y_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>gender</th>\n",
       "      <th>SeniorCitizen</th>\n",
       "      <th>Partner</th>\n",
       "      <th>Dependents</th>\n",
       "      <th>tenure</th>\n",
       "      <th>PhoneService</th>\n",
       "      <th>MultipleLines</th>\n",
       "      <th>OnlineSecurity</th>\n",
       "      <th>OnlineBackup</th>\n",
       "      <th>DeviceProtection</th>\n",
       "      <th>...</th>\n",
       "      <th>InternetService_Fiber optic</th>\n",
       "      <th>InternetService_No</th>\n",
       "      <th>Contract_Month-to-month</th>\n",
       "      <th>Contract_One year</th>\n",
       "      <th>Contract_Two year</th>\n",
       "      <th>PaymentMethod_Bank transfer (automatic)</th>\n",
       "      <th>PaymentMethod_Credit card (automatic)</th>\n",
       "      <th>PaymentMethod_Electronic check</th>\n",
       "      <th>PaymentMethod_Mailed check</th>\n",
       "      <th>Churn</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>684</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2446</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.239437</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1680</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.774648</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2220</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2842</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.042254</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 27 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      gender  SeniorCitizen  Partner  Dependents    tenure  PhoneService  \\\n",
       "684        1              0        0           0  0.000000             1   \n",
       "2446       1              0        0           0  0.239437             1   \n",
       "1680       0              0        1           1  0.774648             1   \n",
       "2220       0              0        1           0  1.000000             1   \n",
       "2842       1              0        0           0  0.042254             0   \n",
       "\n",
       "      MultipleLines  OnlineSecurity  OnlineBackup  DeviceProtection  ...  \\\n",
       "684               0               0             0                 0  ...   \n",
       "2446              1               0             1                 0  ...   \n",
       "1680              1               0             0                 0  ...   \n",
       "2220              0               1             1                 0  ...   \n",
       "2842              0               1             0                 1  ...   \n",
       "\n",
       "      InternetService_Fiber optic  InternetService_No  \\\n",
       "684                             1                   0   \n",
       "2446                            1                   0   \n",
       "1680                            0                   1   \n",
       "2220                            0                   0   \n",
       "2842                            0                   0   \n",
       "\n",
       "      Contract_Month-to-month  Contract_One year  Contract_Two year  \\\n",
       "684                         1                  0                  0   \n",
       "2446                        1                  0                  0   \n",
       "1680                        0                  1                  0   \n",
       "2220                        0                  0                  1   \n",
       "2842                        1                  0                  0   \n",
       "\n",
       "      PaymentMethod_Bank transfer (automatic)  \\\n",
       "684                                         0   \n",
       "2446                                        0   \n",
       "1680                                        0   \n",
       "2220                                        1   \n",
       "2842                                        0   \n",
       "\n",
       "      PaymentMethod_Credit card (automatic)  PaymentMethod_Electronic check  \\\n",
       "684                                       0                               0   \n",
       "2446                                      1                               0   \n",
       "1680                                      0                               0   \n",
       "2220                                      0                               0   \n",
       "2842                                      0                               0   \n",
       "\n",
       "      PaymentMethod_Mailed check  Churn  \n",
       "684                            1      0  \n",
       "2446                           0      1  \n",
       "1680                           1      0  \n",
       "2220                           0      0  \n",
       "2842                           1      0  \n",
       "\n",
       "[5 rows x 27 columns]"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df3.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [],
   "source": [
    "df3_class0 = df3[df3.Churn==0]\n",
    "df3_class1 = df3[df3.Churn==1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_train_batch(df_majority, df_minority, start, end):\n",
    "    df_train = pd.concat([df_majority[start:end], df_minority], axis=0)\n",
    "\n",
    "    X_train = df_train.drop('Churn', axis='columns')\n",
    "    y_train = df_train.Churn\n",
    "    return X_train, y_train    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.5840 - accuracy: 0.7164\n",
      "Epoch 2/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.5094 - accuracy: 0.7542\n",
      "Epoch 3/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4964 - accuracy: 0.7656\n",
      "Epoch 4/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4893 - accuracy: 0.7649\n",
      "Epoch 5/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4844 - accuracy: 0.7672\n",
      "Epoch 6/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4817 - accuracy: 0.7689\n",
      "Epoch 7/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4767 - accuracy: 0.7719\n",
      "Epoch 8/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4758 - accuracy: 0.7722\n",
      "Epoch 9/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4733 - accuracy: 0.7753\n",
      "Epoch 10/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4702 - accuracy: 0.7753\n",
      "Epoch 11/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4703 - accuracy: 0.7736\n",
      "Epoch 12/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4659 - accuracy: 0.7769\n",
      "Epoch 13/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4648 - accuracy: 0.7796\n",
      "Epoch 14/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4631 - accuracy: 0.7813\n",
      "Epoch 15/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4614 - accuracy: 0.7826\n",
      "Epoch 16/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4597 - accuracy: 0.7819\n",
      "Epoch 17/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4579 - accuracy: 0.7806\n",
      "Epoch 18/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4578 - accuracy: 0.7829\n",
      "Epoch 19/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4559 - accuracy: 0.7863\n",
      "Epoch 20/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4534 - accuracy: 0.7866\n",
      "Epoch 21/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4526 - accuracy: 0.7846\n",
      "Epoch 22/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4506 - accuracy: 0.7896\n",
      "Epoch 23/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4491 - accuracy: 0.7916\n",
      "Epoch 24/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4489 - accuracy: 0.7896\n",
      "Epoch 25/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4469 - accuracy: 0.7953\n",
      "Epoch 26/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4469 - accuracy: 0.7923\n",
      "Epoch 27/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4449 - accuracy: 0.7943\n",
      "Epoch 28/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4434 - accuracy: 0.7940\n",
      "Epoch 29/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4443 - accuracy: 0.7906\n",
      "Epoch 30/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4419 - accuracy: 0.7963\n",
      "Epoch 31/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4397 - accuracy: 0.7983\n",
      "Epoch 32/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4391 - accuracy: 0.7983\n",
      "Epoch 33/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4401 - accuracy: 0.7973\n",
      "Epoch 34/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4377 - accuracy: 0.7997\n",
      "Epoch 35/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4356 - accuracy: 0.8000\n",
      "Epoch 36/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4350 - accuracy: 0.7990\n",
      "Epoch 37/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4334 - accuracy: 0.8000\n",
      "Epoch 38/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4343 - accuracy: 0.7973\n",
      "Epoch 39/100\n",
      "94/94 [==============================] - 0s 2ms/step - loss: 0.4311 - accuracy: 0.7997\n",
      "Epoch 40/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4299 - accuracy: 0.8007\n",
      "Epoch 41/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4300 - accuracy: 0.8010\n",
      "Epoch 42/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4279 - accuracy: 0.8050\n",
      "Epoch 43/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4286 - accuracy: 0.8000\n",
      "Epoch 44/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4264 - accuracy: 0.8027\n",
      "Epoch 45/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4262 - accuracy: 0.8007\n",
      "Epoch 46/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4243 - accuracy: 0.8030\n",
      "Epoch 47/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4227 - accuracy: 0.8067\n",
      "Epoch 48/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4208 - accuracy: 0.8084\n",
      "Epoch 49/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4233 - accuracy: 0.8037\n",
      "Epoch 50/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4203 - accuracy: 0.8037\n",
      "Epoch 51/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4182 - accuracy: 0.8080\n",
      "Epoch 52/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4191 - accuracy: 0.8067\n",
      "Epoch 53/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4155 - accuracy: 0.8050\n",
      "Epoch 54/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4164 - accuracy: 0.8104\n",
      "Epoch 55/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4142 - accuracy: 0.8130\n",
      "Epoch 56/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4146 - accuracy: 0.8087\n",
      "Epoch 57/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4137 - accuracy: 0.8087\n",
      "Epoch 58/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4106 - accuracy: 0.8130\n",
      "Epoch 59/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4108 - accuracy: 0.8114\n",
      "Epoch 60/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4090 - accuracy: 0.8137\n",
      "Epoch 61/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4086 - accuracy: 0.8157\n",
      "Epoch 62/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4081 - accuracy: 0.8127\n",
      "Epoch 63/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4070 - accuracy: 0.8134\n",
      "Epoch 64/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4054 - accuracy: 0.8140\n",
      "Epoch 65/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4047 - accuracy: 0.8137\n",
      "Epoch 66/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4043 - accuracy: 0.8161\n",
      "Epoch 67/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4050 - accuracy: 0.8107\n",
      "Epoch 68/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4023 - accuracy: 0.8161\n",
      "Epoch 69/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4015 - accuracy: 0.8161\n",
      "Epoch 70/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4007 - accuracy: 0.8174\n",
      "Epoch 71/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4011 - accuracy: 0.8161\n",
      "Epoch 72/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3984 - accuracy: 0.8187\n",
      "Epoch 73/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3982 - accuracy: 0.8191\n",
      "Epoch 74/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3968 - accuracy: 0.8201\n",
      "Epoch 75/100\n",
      "94/94 [==============================] - 0s 2ms/step - loss: 0.3968 - accuracy: 0.8207\n",
      "Epoch 76/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3980 - accuracy: 0.8154\n",
      "Epoch 77/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3943 - accuracy: 0.8214\n",
      "Epoch 78/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3947 - accuracy: 0.8191\n",
      "Epoch 79/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3927 - accuracy: 0.8207\n",
      "Epoch 80/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3918 - accuracy: 0.8224\n",
      "Epoch 81/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3912 - accuracy: 0.8207\n",
      "Epoch 82/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3897 - accuracy: 0.8211\n",
      "Epoch 83/100\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3921 - accuracy: 0.8244\n",
      "Epoch 84/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3879 - accuracy: 0.8247\n",
      "Epoch 85/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3908 - accuracy: 0.8284\n",
      "Epoch 86/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3880 - accuracy: 0.8221\n",
      "Epoch 87/100\n",
      "94/94 [==============================] - 0s 2ms/step - loss: 0.3894 - accuracy: 0.8258\n",
      "Epoch 88/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3862 - accuracy: 0.8271\n",
      "Epoch 89/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3881 - accuracy: 0.8284\n",
      "Epoch 90/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3843 - accuracy: 0.8268\n",
      "Epoch 91/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3847 - accuracy: 0.8268\n",
      "Epoch 92/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3865 - accuracy: 0.8268\n",
      "Epoch 93/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3832 - accuracy: 0.8271\n",
      "Epoch 94/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3834 - accuracy: 0.8268\n",
      "Epoch 95/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3814 - accuracy: 0.8284\n",
      "Epoch 96/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3802 - accuracy: 0.8324\n",
      "Epoch 97/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3800 - accuracy: 0.8268\n",
      "Epoch 98/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3799 - accuracy: 0.8314\n",
      "Epoch 99/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3783 - accuracy: 0.8324\n",
      "Epoch 100/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3779 - accuracy: 0.8288\n",
      "44/44 [==============================] - 0s 955us/step - loss: 0.6137 - accuracy: 0.7114\n",
      "[0.6136508584022522, 0.711442768573761]\n",
      "Classification Report: \n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "           0       0.89      0.70      0.78      1033\n",
      "           1       0.47      0.76      0.58       374\n",
      "\n",
      "    accuracy                           0.71      1407\n",
      "   macro avg       0.68      0.73      0.68      1407\n",
      "weighted avg       0.78      0.71      0.73      1407\n",
      "\n"
     ]
    }
   ],
   "source": [
    "X_train, y_train = get_train_batch(df3_class0, df3_class1, 0, 1495)\n",
    "\n",
    "y_pred1 = ANN(X_train, y_train, X_test, y_test, 'binary_crossentropy', -1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.6486 - accuracy: 0.6328\n",
      "Epoch 2/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.5236 - accuracy: 0.7522\n",
      "Epoch 3/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.5005 - accuracy: 0.7572\n",
      "Epoch 4/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4935 - accuracy: 0.7592\n",
      "Epoch 5/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4883 - accuracy: 0.7562\n",
      "Epoch 6/100\n",
      "94/94 [==============================] - 0s 2ms/step - loss: 0.4845 - accuracy: 0.7605\n",
      "Epoch 7/100\n",
      "94/94 [==============================] - 0s 2ms/step - loss: 0.4795 - accuracy: 0.7625\n",
      "Epoch 8/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4778 - accuracy: 0.7642\n",
      "Epoch 9/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4747 - accuracy: 0.7659\n",
      "Epoch 10/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4719 - accuracy: 0.7692\n",
      "Epoch 11/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4682 - accuracy: 0.7696\n",
      "Epoch 12/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4659 - accuracy: 0.7689\n",
      "Epoch 13/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4679 - accuracy: 0.7659\n",
      "Epoch 14/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4625 - accuracy: 0.7712\n",
      "Epoch 15/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4596 - accuracy: 0.7776\n",
      "Epoch 16/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4590 - accuracy: 0.7722\n",
      "Epoch 17/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4556 - accuracy: 0.7799\n",
      "Epoch 18/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4539 - accuracy: 0.7783\n",
      "Epoch 19/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4519 - accuracy: 0.7806\n",
      "Epoch 20/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4499 - accuracy: 0.7819\n",
      "Epoch 21/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4497 - accuracy: 0.7873\n",
      "Epoch 22/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4473 - accuracy: 0.7803\n",
      "Epoch 23/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4466 - accuracy: 0.7836\n",
      "Epoch 24/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4439 - accuracy: 0.7860\n",
      "Epoch 25/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4429 - accuracy: 0.7876\n",
      "Epoch 26/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4435 - accuracy: 0.7843\n",
      "Epoch 27/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4405 - accuracy: 0.7843\n",
      "Epoch 28/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4381 - accuracy: 0.7906\n",
      "Epoch 29/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4364 - accuracy: 0.7896\n",
      "Epoch 30/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4377 - accuracy: 0.7870\n",
      "Epoch 31/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4347 - accuracy: 0.7946\n",
      "Epoch 32/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4325 - accuracy: 0.7963\n",
      "Epoch 33/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4315 - accuracy: 0.7946\n",
      "Epoch 34/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4293 - accuracy: 0.7946\n",
      "Epoch 35/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4299 - accuracy: 0.7943\n",
      "Epoch 36/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4280 - accuracy: 0.7963\n",
      "Epoch 37/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4253 - accuracy: 0.8003\n",
      "Epoch 38/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4271 - accuracy: 0.7953\n",
      "Epoch 39/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4233 - accuracy: 0.8010\n",
      "Epoch 40/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4234 - accuracy: 0.7993\n",
      "Epoch 41/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4198 - accuracy: 0.7993\n",
      "Epoch 42/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4186 - accuracy: 0.8007\n",
      "Epoch 43/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4188 - accuracy: 0.8020\n",
      "Epoch 44/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4167 - accuracy: 0.8050\n",
      "Epoch 45/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4152 - accuracy: 0.8057\n",
      "Epoch 46/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4139 - accuracy: 0.8033\n",
      "Epoch 47/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4134 - accuracy: 0.8060\n",
      "Epoch 48/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4112 - accuracy: 0.8090\n",
      "Epoch 49/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4093 - accuracy: 0.8057\n",
      "Epoch 50/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4099 - accuracy: 0.8067\n",
      "Epoch 51/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4090 - accuracy: 0.8057\n",
      "Epoch 52/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4065 - accuracy: 0.8054\n",
      "Epoch 53/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4050 - accuracy: 0.8084\n",
      "Epoch 54/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4031 - accuracy: 0.8064\n",
      "Epoch 55/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4024 - accuracy: 0.8140\n",
      "Epoch 56/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4005 - accuracy: 0.8151\n",
      "Epoch 57/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.4020 - accuracy: 0.8074\n",
      "Epoch 58/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3997 - accuracy: 0.8140\n",
      "Epoch 59/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3982 - accuracy: 0.8140\n",
      "Epoch 60/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3974 - accuracy: 0.8154\n",
      "Epoch 61/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3969 - accuracy: 0.8114\n",
      "Epoch 62/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3955 - accuracy: 0.8140\n",
      "Epoch 63/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3961 - accuracy: 0.8134\n",
      "Epoch 64/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3938 - accuracy: 0.8161\n",
      "Epoch 65/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3912 - accuracy: 0.8177\n",
      "Epoch 66/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3922 - accuracy: 0.8127\n",
      "Epoch 67/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3899 - accuracy: 0.8174\n",
      "Epoch 68/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3909 - accuracy: 0.8147\n",
      "Epoch 69/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3891 - accuracy: 0.8194\n",
      "Epoch 70/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3878 - accuracy: 0.8207\n",
      "Epoch 71/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3870 - accuracy: 0.8214\n",
      "Epoch 72/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3870 - accuracy: 0.8224\n",
      "Epoch 73/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3852 - accuracy: 0.8214\n",
      "Epoch 74/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3847 - accuracy: 0.8247\n",
      "Epoch 75/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3846 - accuracy: 0.8191\n",
      "Epoch 76/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3835 - accuracy: 0.8201\n",
      "Epoch 77/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3825 - accuracy: 0.8264\n",
      "Epoch 78/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3830 - accuracy: 0.8184\n",
      "Epoch 79/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3813 - accuracy: 0.8244\n",
      "Epoch 80/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3784 - accuracy: 0.8278\n",
      "Epoch 81/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3794 - accuracy: 0.8227\n",
      "Epoch 82/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3788 - accuracy: 0.8224\n",
      "Epoch 83/100\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3779 - accuracy: 0.8227\n",
      "Epoch 84/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3769 - accuracy: 0.8221\n",
      "Epoch 85/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3758 - accuracy: 0.8281\n",
      "Epoch 86/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3749 - accuracy: 0.8308\n",
      "Epoch 87/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3730 - accuracy: 0.8258\n",
      "Epoch 88/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3737 - accuracy: 0.8231\n",
      "Epoch 89/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3789 - accuracy: 0.8227\n",
      "Epoch 90/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3718 - accuracy: 0.8318\n",
      "Epoch 91/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3700 - accuracy: 0.8284\n",
      "Epoch 92/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3703 - accuracy: 0.8268\n",
      "Epoch 93/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3694 - accuracy: 0.8308\n",
      "Epoch 94/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3675 - accuracy: 0.8281\n",
      "Epoch 95/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3682 - accuracy: 0.8301\n",
      "Epoch 96/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3670 - accuracy: 0.8271\n",
      "Epoch 97/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3647 - accuracy: 0.8308\n",
      "Epoch 98/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3653 - accuracy: 0.8355\n",
      "Epoch 99/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3659 - accuracy: 0.8355\n",
      "Epoch 100/100\n",
      "94/94 [==============================] - 0s 1ms/step - loss: 0.3648 - accuracy: 0.8294\n",
      "44/44 [==============================] - 0s 978us/step - loss: 0.6379 - accuracy: 0.7079\n",
      "[0.6378723382949829, 0.7078891396522522]\n",
      "Classification Report: \n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "           0       0.88      0.70      0.78      1033\n",
      "           1       0.47      0.74      0.57       374\n",
      "\n",
      "    accuracy                           0.71      1407\n",
      "   macro avg       0.67      0.72      0.68      1407\n",
      "weighted avg       0.77      0.71      0.72      1407\n",
      "\n"
     ]
    }
   ],
   "source": [
    "X_train, y_train = get_train_batch(df3_class0, df3_class1, 1495, 2990)\n",
    "\n",
    "y_pred2 = ANN(X_train, y_train, X_test, y_test, 'binary_crossentropy', -1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.6392 - accuracy: 0.6676\n",
      "Epoch 2/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.5260 - accuracy: 0.7609\n",
      "Epoch 3/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4914 - accuracy: 0.7674\n",
      "Epoch 4/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4835 - accuracy: 0.7715\n",
      "Epoch 5/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4793 - accuracy: 0.7738\n",
      "Epoch 6/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4751 - accuracy: 0.7787\n",
      "Epoch 7/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4736 - accuracy: 0.7731\n",
      "Epoch 8/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4708 - accuracy: 0.7818\n",
      "Epoch 9/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4700 - accuracy: 0.7772\n",
      "Epoch 10/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4679 - accuracy: 0.7810\n",
      "Epoch 11/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4670 - accuracy: 0.7799\n",
      "Epoch 12/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4653 - accuracy: 0.7791\n",
      "Epoch 13/100\n",
      "83/83 [==============================] - 0s 2ms/step - loss: 0.4632 - accuracy: 0.7822\n",
      "Epoch 14/100\n",
      "83/83 [==============================] - 0s 2ms/step - loss: 0.4607 - accuracy: 0.7810\n",
      "Epoch 15/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4590 - accuracy: 0.7894\n",
      "Epoch 16/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4573 - accuracy: 0.7867\n",
      "Epoch 17/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4561 - accuracy: 0.7879\n",
      "Epoch 18/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4548 - accuracy: 0.7863\n",
      "Epoch 19/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4535 - accuracy: 0.7894\n",
      "Epoch 20/100\n",
      "83/83 [==============================] - 0s 2ms/step - loss: 0.4506 - accuracy: 0.7901\n",
      "Epoch 21/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4518 - accuracy: 0.7890\n",
      "Epoch 22/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4487 - accuracy: 0.7947\n",
      "Epoch 23/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4471 - accuracy: 0.7917\n",
      "Epoch 24/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4458 - accuracy: 0.7909\n",
      "Epoch 25/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4430 - accuracy: 0.7958\n",
      "Epoch 26/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4423 - accuracy: 0.7935\n",
      "Epoch 27/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4421 - accuracy: 0.7920\n",
      "Epoch 28/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4421 - accuracy: 0.7992\n",
      "Epoch 29/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4383 - accuracy: 0.7951\n",
      "Epoch 30/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4389 - accuracy: 0.7996\n",
      "Epoch 31/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4387 - accuracy: 0.7966\n",
      "Epoch 32/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4352 - accuracy: 0.7996\n",
      "Epoch 33/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4330 - accuracy: 0.8000\n",
      "Epoch 34/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4317 - accuracy: 0.8027\n",
      "Epoch 35/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4307 - accuracy: 0.8015\n",
      "Epoch 36/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4275 - accuracy: 0.8023\n",
      "Epoch 37/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4275 - accuracy: 0.8053\n",
      "Epoch 38/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4239 - accuracy: 0.8008\n",
      "Epoch 39/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4252 - accuracy: 0.8061\n",
      "Epoch 40/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4227 - accuracy: 0.8083\n",
      "Epoch 41/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4201 - accuracy: 0.8053\n",
      "Epoch 42/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4207 - accuracy: 0.8053\n",
      "Epoch 43/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4192 - accuracy: 0.8110\n",
      "Epoch 44/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4184 - accuracy: 0.8121\n",
      "Epoch 45/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4156 - accuracy: 0.8099\n",
      "Epoch 46/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4138 - accuracy: 0.8125\n",
      "Epoch 47/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4118 - accuracy: 0.8137\n",
      "Epoch 48/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4120 - accuracy: 0.8144\n",
      "Epoch 49/100\n",
      "83/83 [==============================] - 0s 2ms/step - loss: 0.4104 - accuracy: 0.8110\n",
      "Epoch 50/100\n",
      "83/83 [==============================] - 0s 2ms/step - loss: 0.4097 - accuracy: 0.8106\n",
      "Epoch 51/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4075 - accuracy: 0.8125\n",
      "Epoch 52/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4049 - accuracy: 0.8163\n",
      "Epoch 53/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4051 - accuracy: 0.8171\n",
      "Epoch 54/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4039 - accuracy: 0.8152\n",
      "Epoch 55/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4023 - accuracy: 0.8152\n",
      "Epoch 56/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4002 - accuracy: 0.8159\n",
      "Epoch 57/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.4014 - accuracy: 0.8171\n",
      "Epoch 58/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3987 - accuracy: 0.8171\n",
      "Epoch 59/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3957 - accuracy: 0.8216\n",
      "Epoch 60/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3991 - accuracy: 0.8220\n",
      "Epoch 61/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3950 - accuracy: 0.8209\n",
      "Epoch 62/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3934 - accuracy: 0.8258\n",
      "Epoch 63/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3938 - accuracy: 0.8239\n",
      "Epoch 64/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3892 - accuracy: 0.8228\n",
      "Epoch 65/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3902 - accuracy: 0.8254\n",
      "Epoch 66/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3891 - accuracy: 0.8247\n",
      "Epoch 67/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3864 - accuracy: 0.8288\n",
      "Epoch 68/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3850 - accuracy: 0.8296\n",
      "Epoch 69/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3872 - accuracy: 0.8235\n",
      "Epoch 70/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3834 - accuracy: 0.8315\n",
      "Epoch 71/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3826 - accuracy: 0.8300\n",
      "Epoch 72/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3809 - accuracy: 0.8304\n",
      "Epoch 73/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3823 - accuracy: 0.8323\n",
      "Epoch 74/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3792 - accuracy: 0.8330\n",
      "Epoch 75/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3771 - accuracy: 0.8342\n",
      "Epoch 76/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3757 - accuracy: 0.8406\n",
      "Epoch 77/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3753 - accuracy: 0.8383\n",
      "Epoch 78/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3751 - accuracy: 0.8357\n",
      "Epoch 79/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3746 - accuracy: 0.8338\n",
      "Epoch 80/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3718 - accuracy: 0.8372\n",
      "Epoch 81/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3725 - accuracy: 0.8387\n",
      "Epoch 82/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3690 - accuracy: 0.8406\n",
      "Epoch 83/100\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3709 - accuracy: 0.8357\n",
      "Epoch 84/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3704 - accuracy: 0.8387\n",
      "Epoch 85/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3687 - accuracy: 0.8391\n",
      "Epoch 86/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3675 - accuracy: 0.8444\n",
      "Epoch 87/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3649 - accuracy: 0.8410\n",
      "Epoch 88/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3669 - accuracy: 0.8440\n",
      "Epoch 89/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3677 - accuracy: 0.8387\n",
      "Epoch 90/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3633 - accuracy: 0.8459\n",
      "Epoch 91/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3626 - accuracy: 0.8463\n",
      "Epoch 92/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3614 - accuracy: 0.8433\n",
      "Epoch 93/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3595 - accuracy: 0.8452\n",
      "Epoch 94/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3614 - accuracy: 0.8448\n",
      "Epoch 95/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3615 - accuracy: 0.8436\n",
      "Epoch 96/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3600 - accuracy: 0.8448\n",
      "Epoch 97/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3566 - accuracy: 0.8459\n",
      "Epoch 98/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3600 - accuracy: 0.8467\n",
      "Epoch 99/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3575 - accuracy: 0.8478\n",
      "Epoch 100/100\n",
      "83/83 [==============================] - 0s 1ms/step - loss: 0.3571 - accuracy: 0.8482\n",
      "44/44 [==============================] - 0s 978us/step - loss: 0.6610 - accuracy: 0.6915\n",
      "[0.6610018014907837, 0.6915422677993774]\n",
      "Classification Report: \n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "           0       0.89      0.66      0.76      1033\n",
      "           1       0.45      0.78      0.57       374\n",
      "\n",
      "    accuracy                           0.69      1407\n",
      "   macro avg       0.67      0.72      0.67      1407\n",
      "weighted avg       0.78      0.69      0.71      1407\n",
      "\n"
     ]
    }
   ],
   "source": [
    "X_train, y_train = get_train_batch(df3_class0, df3_class1, 2990, 4130)\n",
    "\n",
    "y_pred3 = ANN(X_train, y_train, X_test, y_test, 'binary_crossentropy', -1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1407"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(y_pred1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred_final = y_pred1.copy()\n",
    "for i in range(len(y_pred1)):\n",
    "    n_ones = y_pred1[i] + y_pred2[i] + y_pred3[i]\n",
    "    if n_ones>1:\n",
    "        y_pred_final[i] = 1\n",
    "    else:\n",
    "        y_pred_final[i] = 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.88      0.69      0.77      1033\n",
      "           1       0.46      0.75      0.57       374\n",
      "\n",
      "    accuracy                           0.70      1407\n",
      "   macro avg       0.67      0.72      0.67      1407\n",
      "weighted avg       0.77      0.70      0.72      1407\n",
      "\n"
     ]
    }
   ],
   "source": [
    "cl_rep = classification_report(y_test, y_pred_final)\n",
    "print(cl_rep)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "f1-score for minority class 1 improved to 0.62 from 0.57. The score for majority class 0 is suffering and reduced to 0.80 from 0.85 but at least there is some balance in terms of prediction accuracy across two classes"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}