03-01 Pandas Basics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Pandas\n",
    "\n",
    "NumPy creates ndarrays that must contain values that are of the same data type.\n",
    "\n",
    "Pandas creates dataframes. Each column in a dataframe is an ndarray. This allows us to have traditional tables of data where each column can be a different data type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The basic data structure in pandas is the *series*. You can construct it in a similar fashion to making a numpy array.\n",
    "\n",
    "The command to make a Series object is\n",
    "\n",
    "`pd.Series(data, index=index)`\n",
    "\n",
    "the `index` argument is optional"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0    0.25\n",
      "1    0.50\n",
      "2    0.75\n",
      "3    1.00\n",
      "dtype: float64\n",
      "<class 'pandas.core.series.Series'>\n"
     ]
    }
   ],
   "source": [
    "data = pd.Series([0.25, 0.5, 0.75, 1.0])\n",
    "print(data)\n",
    "print(type(data))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The series is printed out in a table form.\n",
    "\n",
    "The type is a Pandas Series"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.25 0.5  0.75 1.  ]\n",
      "<class 'numpy.ndarray'>\n"
     ]
    }
   ],
   "source": [
    "print(data.values)\n",
    "print(type(data.values))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The values attribute of the series is a numpy array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RangeIndex(start=0, stop=4, step=1)\n",
      "<class 'pandas.core.indexes.range.RangeIndex'>\n"
     ]
    }
   ],
   "source": [
    "print(data.index)\n",
    "print(type(data.index))  # the row names are known as the index"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can subset a pandas series like other python objects"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.5\n",
      "<class 'numpy.float64'>\n"
     ]
    }
   ],
   "source": [
    "print(data[1])\n",
    "print(type(data[1]))  # when you select only one value, it simplifies the object"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1    0.50\n",
      "2    0.75\n",
      "dtype: float64\n",
      "<class 'pandas.core.series.Series'>\n"
     ]
    }
   ],
   "source": [
    "print(data[1:3])\n",
    "print(type(data[1:3]))  # selecting multiple values returns a series"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1    0.50\n",
      "0    0.25\n",
      "2    0.75\n",
      "dtype: float64\n"
     ]
    }
   ],
   "source": [
    "print(data[np.array([1,0,2])])  # fancy indexing using a numpy array"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a    0.25\n",
      "b    0.50\n",
      "c    0.75\n",
      "d    1.00\n",
      "dtype: float64\n"
     ]
    }
   ],
   "source": [
    "# specifying the index values\n",
    "data = pd.Series([0.25, 0.5, 0.75, 1.0],\n",
    "                 index=['a', 'b', 'c', 'd'])\n",
    "print(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[1]  # subset with index position"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.25"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[\"a\"]  # subset with index names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    0.25\n",
       "b    0.50\n",
       "c    0.75\n",
       "dtype: float64"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[\"a\":\"c\"] # using names includes the last value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    0.25\n",
       "b    0.50\n",
       "dtype: float64"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[0:2]  # slicing behavior is unchanged"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Archie      71\n",
       "Betty       66\n",
       "Cheryl      66\n",
       "Jughead     72\n",
       "Veronica    62\n",
       "dtype: int64"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# creating a series from a python dictionary\n",
    "# remember, dictionary construction uses curly braces {}\n",
    "samp_dict = {'Archie': 71,\n",
    "             'Betty': 66,\n",
    "             'Veronica': 62,\n",
    "             'Jughead': 72,\n",
    "             'Cheryl': 66}\n",
    "samp_series = pd.Series(samp_dict)\n",
    "samp_series # the series gets alphabetized by its index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Index(['Archie', 'Betty', 'Cheryl', 'Jughead', 'Veronica'], dtype='object')\n"
     ]
    }
   ],
   "source": [
    "print(samp_series.index)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'dict'>\n",
      "<class 'pandas.core.series.Series'>\n"
     ]
    }
   ],
   "source": [
    "print(type(samp_dict))\n",
    "print(type(samp_series))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Archie             KJ\n",
      "Betty            Lili\n",
      "Cheryl      Madelaine\n",
      "Jughead          Cole\n",
      "Veronica       Camila\n",
      "dtype: object\n"
     ]
    }
   ],
   "source": [
    "actor_dict = {'Archie': \"KJ\",\n",
    "              'Jughead': \"Cole\",\n",
    "              'Betty': \"Lili\",\n",
    "              'Veronica': \"Camila\",\n",
    "              'Cheryl': \"Madelaine\"}  # note that the dictionary order is not same here\n",
    "actor = pd.Series(actor_dict)  # still get alphabetized by index\n",
    "print(actor)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Creating a DataFrame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              actor  height\n",
      "Archie           KJ      71\n",
      "Betty          Lili      66\n",
      "Cheryl    Madelaine      66\n",
      "Jughead        Cole      72\n",
      "Veronica     Camila      62\n"
     ]
    }
   ],
   "source": [
    "# we create a dataframe by providing a dictionary of series objects\n",
    "riverdale = pd.DataFrame({'height': samp_series,\n",
    "                       'actor': actor})  \n",
    "\n",
    "print(riverdale)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n"
     ]
    }
   ],
   "source": [
    "print(type(riverdale))  # this is a DataFrame object"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'a': 0, 'b': 0}, {'a': 1, 'b': 2}, {'a': 2, 'b': 5}]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = [{'a': 0, 'b': 0}, {'a': 1, 'b': 2}, {'a': 2, 'b': 5}]  # data is a list of dictionaries\n",
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   a  b\n",
      "1  0  0\n",
      "2  1  2\n",
      "3  2  5\n"
     ]
    }
   ],
   "source": [
    "print(pd.DataFrame(data, index = [1,2,3]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'a': 0, 'b': 0}, {'a': 1, 'b': 2}, {'a': 2, 'c': 5}]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = [{'a': 0, 'b': 0}, {'a': 1, 'b': 2}, {'a': 2, 'c': 5}]  # data is a list of dictionaries\n",
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   a    b    c\n",
      "0  0  0.0  NaN\n",
      "1  1  2.0  NaN\n",
      "2  2  NaN  5.0\n"
     ]
    }
   ],
   "source": [
    "print(pd.DataFrame(data))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'a': 0, 'b': 0}, {'a': 1, 'b': 2}, {'a': 2, 'c': '5'}]"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = [{'a': 0, 'b': 0}, {'a': 1, 'b': 2}, {'a': 2, 'c': \"5\"}]  # data is a list of dictionaries\n",
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   a    b    c\n",
      "0  0  0.0  NaN\n",
      "1  1  2.0  NaN\n",
      "2  2  NaN    5\n"
     ]
    }
   ],
   "source": [
    "print(pd.DataFrame(data))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1 8]\n",
      " [0 1]\n",
      " [3 0]\n",
      " [1 2]\n",
      " [1 7]]\n"
     ]
    }
   ],
   "source": [
    "data = np.random.randint(10, size = 10).reshape([5,2])\n",
    "print(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   x  y\n",
      "a  1  8\n",
      "b  0  1\n",
      "c  3  0\n",
      "d  1  2\n",
      "e  1  7\n"
     ]
    }
   ],
   "source": [
    "print(pd.DataFrame(data, columns = [\"x\",\"y\"], index = ['a','b','c','d','e']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Subsetting the DataFrame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              actor  height\n",
      "Archie           KJ      71\n",
      "Betty          Lili      66\n",
      "Cheryl    Madelaine      66\n",
      "Jughead        Cole      72\n",
      "Veronica     Camila      62\n"
     ]
    }
   ],
   "source": [
    "print(riverdale)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Index(['actor', 'height'], dtype='object')\n"
     ]
    }
   ],
   "source": [
    "print(riverdale.keys())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Archie             KJ\n",
       "Betty            Lili\n",
       "Cheryl      Madelaine\n",
       "Jughead          Cole\n",
       "Veronica       Camila\n",
       "Name: actor, dtype: object"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "riverdale['actor']  # extracting the column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Archie             KJ\n",
       "Betty            Lili\n",
       "Cheryl      Madelaine\n",
       "Jughead          Cole\n",
       "Veronica       Camila\n",
       "Name: actor, dtype: object"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "riverdale.actor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Lili'"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "riverdale.actor[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Cole'"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "riverdale.actor['Jughead']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "       Archie Betty     Cheryl Jughead Veronica\n",
      "actor      KJ  Lili  Madelaine    Cole   Camila\n",
      "height     71    66         66      72       62\n"
     ]
    }
   ],
   "source": [
    "print(riverdale.T)  # prints a copy of the transpose"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "actor     Cole\n",
      "height      72\n",
      "Name: Jughead, dtype: object\n",
      "<class 'pandas.core.series.Series'>\n",
      "<class 'numpy.ndarray'>\n"
     ]
    }
   ],
   "source": [
    "print(riverdale.loc['Jughead']) # subset based on location to get a row\n",
    "print(type(riverdale.loc['Jughead']))\n",
    "print(type(riverdale.loc['Jughead'].values))  # the values are of mixed type but is still a numpy array. \n",
    "# this is possible because it is a structured numpy array. (covered in \"Python for Data Science\" chapter 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Archie      71\n",
      "Betty       66\n",
      "Cheryl      66\n",
      "Jughead     72\n",
      "Veronica    62\n",
      "Name: height, dtype: int64\n",
      "<class 'pandas.core.series.Series'>\n",
      "<class 'numpy.ndarray'>\n"
     ]
    }
   ],
   "source": [
    "print(riverdale.loc[:,'height']) # subset based on location to get a column\n",
    "print(type(riverdale.loc[:,'height']))  #the object is a pandas series\n",
    "print(type(riverdale.loc[:,'height'].values))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "71"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "riverdale.loc['Archie','height']  # you can provide a pair of 'coordinates' to get a particular value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "actor     Cole\n",
       "height      72\n",
       "Name: Jughead, dtype: object"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "riverdale.iloc[3,] # subset based on index location"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "71"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "riverdale.iloc[0, 1] # pair of coordinates"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}