Final-Assignment-Webscraping.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center>\n",
    "    <img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0220EN-SkillsNetwork/labs/project/Images/SN_logo.png\" width=\"300\" alt=\"cognitiveclass.ai logo\"  />\n",
    "</center>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>Extracting Stock Data Using a Web Scraping</h1>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Not all stock data is available via API in this assignment; you will use web-scraping to obtain financial data. You will be quizzed on your results.\\\n",
    "Using beautiful soup we will extract historical share data from a web-page.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Table of Contents</h2>\n",
    "<div class=\"alert alert-block alert-info\" style=\"margin-top: 20px\">\n",
    "    <ul>\n",
    "        <li>Downloading the Webpage Using Requests Library</li>\n",
    "        <li>Parsing Webpage HTML Using BeautifulSoup</li>\n",
    "        <li>Extracting Data and Building DataFrame</li>\n",
    "    </ul>\n",
    "<p>\n",
    "    Estimated Time Needed: <strong>30 min</strong></p>\n",
    "</div>\n",
    "\n",
    "<hr>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "                  __    __    __    __\n",
      "                 /  \\  /  \\  /  \\  /  \\\n",
      "                /    \\/    \\/    \\/    \\\n",
      "███████████████/  /██/  /██/  /██/  /████████████████████████\n",
      "              /  / \\   / \\   / \\   / \\  \\____\n",
      "             /  /   \\_/   \\_/   \\_/   \\    o \\__,\n",
      "            / _/                       \\_____/  `\n",
      "            |/\n",
      "        ███╗   ███╗ █████╗ ███╗   ███╗██████╗  █████╗\n",
      "        ████╗ ████║██╔══██╗████╗ ████║██╔══██╗██╔══██╗\n",
      "        ██╔████╔██║███████║██╔████╔██║██████╔╝███████║\n",
      "        ██║╚██╔╝██║██╔══██║██║╚██╔╝██║██╔══██╗██╔══██║\n",
      "        ██║ ╚═╝ ██║██║  ██║██║ ╚═╝ ██║██████╔╝██║  ██║\n",
      "        ╚═╝     ╚═╝╚═╝  ╚═╝╚═╝     ╚═╝╚═════╝ ╚═╝  ╚═╝\n",
      "\n",
      "        mamba (0.15.3) supported by @QuantStack\n",
      "\n",
      "        GitHub:  https://github.com/mamba-org/mamba\n",
      "        Twitter: https://twitter.com/QuantStack\n",
      "\n",
      "█████████████████████████████████████████████████████████████\n",
      "\n",
      "\n",
      "Looking for: ['bs4==4.10.0']\n",
      "\n",
      "pkgs/r/linux-64          [>                   ] (--:--) No change\n",
      "pkgs/r/linux-64          [====================] (00m:00s) No change\n",
      "pkgs/main/linux-64       [>                   ] (--:--) No change\n",
      "pkgs/main/linux-64       [====================] (00m:00s) No change\n",
      "pkgs/r/noarch            [>                   ] (--:--) No change\n",
      "pkgs/r/noarch            [====================] (00m:00s) No change\n",
      "pkgs/main/noarch         [>                   ] (--:--) No change\n",
      "pkgs/main/noarch         [====================] (00m:00s) No change\n",
      "\n",
      "Pinned packages:\n",
      "  - python 3.7.*\n",
      "\n",
      "\n",
      "Transaction\n",
      "\n",
      "  Prefix: /home/jupyterlab/conda/envs/python\n",
      "\n",
      "  All requested packages already installed\n",
      "\n",
      "\n",
      "                  __    __    __    __\n",
      "                 /  \\  /  \\  /  \\  /  \\\n",
      "                /    \\/    \\/    \\/    \\\n",
      "███████████████/  /██/  /██/  /██/  /████████████████████████\n",
      "              /  / \\   / \\   / \\   / \\  \\____\n",
      "             /  /   \\_/   \\_/   \\_/   \\    o \\__,\n",
      "            / _/                       \\_____/  `\n",
      "            |/\n",
      "        ███╗   ███╗ █████╗ ███╗   ███╗██████╗  █████╗\n",
      "        ████╗ ████║██╔══██╗████╗ ████║██╔══██╗██╔══██╗\n",
      "        ██╔████╔██║███████║██╔████╔██║██████╔╝███████║\n",
      "        ██║╚██╔╝██║██╔══██║██║╚██╔╝██║██╔══██╗██╔══██║\n",
      "        ██║ ╚═╝ ██║██║  ██║██║ ╚═╝ ██║██████╔╝██║  ██║\n",
      "        ╚═╝     ╚═╝╚═╝  ╚═╝╚═╝     ╚═╝╚═════╝ ╚═╝  ╚═╝\n",
      "\n",
      "        mamba (0.15.3) supported by @QuantStack\n",
      "\n",
      "        GitHub:  https://github.com/mamba-org/mamba\n",
      "        Twitter: https://twitter.com/QuantStack\n",
      "\n",
      "█████████████████████████████████████████████████████████████\n",
      "\n",
      "\n",
      "Looking for: ['html5lib==1.1']\n",
      "\n",
      "pkgs/main/linux-64       Using cache\n",
      "pkgs/main/noarch         Using cache\n",
      "pkgs/r/linux-64          Using cache\n",
      "pkgs/r/noarch            Using cache\n",
      "\n",
      "Pinned packages:\n",
      "  - python 3.7.*\n",
      "\n",
      "\n",
      "Transaction\n",
      "\n",
      "  Prefix: /home/jupyterlab/conda/envs/python\n",
      "\n",
      "  Updating specs:\n",
      "\n",
      "   - html5lib==1.1\n",
      "   - ca-certificates\n",
      "   - certifi\n",
      "   - openssl\n",
      "\n",
      "\n",
      "  Package         Version  Build         Channel                 Size\n",
      "───────────────────────────────────────────────────────────────────────\n",
      "  Install:\n",
      "───────────────────────────────────────────────────────────────────────\n",
      "\n",
      "\u001b[32m  + html5lib    \u001b[00m      1.1  pyhd3eb1b0_0  pkgs/main/noarch       91 KB\n",
      "\u001b[32m  + webencodings\u001b[00m    0.5.1  py37_1        pkgs/main/linux-64     19 KB\n",
      "\n",
      "  Summary:\n",
      "\n",
      "  Install: 2 packages\n",
      "\n",
      "  Total download: 110 KB\n",
      "\n",
      "───────────────────────────────────────────────────────────────────────\n",
      "\n",
      "Downloading  [======>                                  ] (00m:00s)   74.88 KB/s\n",
      "Extracting   [>                                                      ] (--:--) \n",
      "\u001b[2A\u001b[0KFinished webencodings                         (00m:00s)              19 KB     75 KB/s\n",
      "Downloading  [======>                                  ] (00m:00s)   74.88 KB/s\n",
      "Extracting   [>                                                      ] (--:--) \n",
      "Downloading  [=========================================] (00m:00s)  427.69 KB/s\n",
      "Extracting   [>                                                      ] (--:--) \n",
      "Downloading  [=========================================] (00m:00s)  427.69 KB/s\n",
      "Extracting   [>                                                      ] (--:--) \n",
      "Downloading  [=========================================] (00m:00s)  427.69 KB/s\n",
      "Extracting   [>                                                      ] (--:--) \n",
      "\u001b[2A\u001b[0KFinished html5lib                             (00m:00s)              91 KB    354 KB/s\n",
      "Downloading  [=========================================] (00m:00s)  427.69 KB/s\n",
      "Extracting   [>                                                      ] (--:--) \n",
      "Downloading  [=========================================] (00m:00s)  427.69 KB/s\n",
      "Extracting   [>                                                      ] (--:--) \n",
      "Downloading  [=========================================] (00m:00s)  427.69 KB/s\n",
      "Extracting   [====================>                    ] (00m:00s)        1 / 2\n",
      "Downloading  [=========================================] (00m:00s)  427.69 KB/s\n",
      "Extracting   [====================>                    ] (00m:00s)        1 / 2\n",
      "Downloading  [=========================================] (00m:00s)  427.69 KB/s\n",
      "Extracting   [=========================================] (00m:00s)        2 / 2\n",
      "Preparing transaction: done\n",
      "Verifying transaction: done\n",
      "Executing transaction: done\n",
      "Collecting lxml==4.6.4\n",
      "  Downloading lxml-4.6.4-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (6.3 MB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.3/6.3 MB\u001b[0m \u001b[31m85.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m00:01\u001b[0m\n",
      "\u001b[?25hInstalling collected packages: lxml\n",
      "  Attempting uninstall: lxml\n",
      "    Found existing installation: lxml 4.9.1\n",
      "    Uninstalling lxml-4.9.1:\n",
      "      Successfully uninstalled lxml-4.9.1\n",
      "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
      "yfinance 0.1.93 requires lxml>=4.9.1, but you have lxml 4.6.4 which is incompatible.\u001b[0m\u001b[31m\n",
      "\u001b[0mSuccessfully installed lxml-4.6.4\n"
     ]
    }
   ],
   "source": [
    "#!pip install pandas==1.3.3\n",
    "#!pip install requests==2.26.0\n",
    "!mamba install bs4==4.10.0 -y\n",
    "!mamba install html5lib==1.1 -y\n",
    "!pip install lxml==4.6.4\n",
    "#!pip install plotly==5.3.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import requests\n",
    "from bs4 import BeautifulSoup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using Webscraping to Extract Stock Data Example\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we must use the `request` library to downlaod the webpage, and extract the text. We will extract Netflix stock data <https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0220EN-SkillsNetwork/labs/project/netflix_data_webpage.html>.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "url = \"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0220EN-SkillsNetwork/labs/project/netflix_data_webpage.html\"\n",
    "\n",
    "data  = requests.get(url).text"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we must parse the text into html using `beautiful_soup`\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "soup = BeautifulSoup(data, 'html5lib')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can turn the html table into a pandas dataframe\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "netflix_data = pd.DataFrame(columns=[\"Date\", \"Open\", \"High\", \"Low\", \"Close\", \"Volume\"])\n",
    "\n",
    "# First we isolate the body of the table which contains all the information\n",
    "# Then we loop through each row and find all the column values for each row\n",
    "for row in soup.find(\"tbody\").find_all('tr'):\n",
    "    col = row.find_all(\"td\")\n",
    "    date = col[0].text\n",
    "    Open = col[1].text\n",
    "    high = col[2].text\n",
    "    low = col[3].text\n",
    "    close = col[4].text\n",
    "    adj_close = col[5].text\n",
    "    volume = col[6].text\n",
    "    \n",
    "    # Finally we append the data of each row to the table\n",
    "    netflix_data = netflix_data.append({\"Date\":date, \"Open\":Open, \"High\":high, \"Low\":low, \"Close\":close, \"Adj Close\":adj_close, \"Volume\":volume}, ignore_index=True)    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now print out the dataframe\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Open</th>\n",
       "      <th>High</th>\n",
       "      <th>Low</th>\n",
       "      <th>Close</th>\n",
       "      <th>Volume</th>\n",
       "      <th>Adj Close</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Jun 01, 2021</td>\n",
       "      <td>504.01</td>\n",
       "      <td>536.13</td>\n",
       "      <td>482.14</td>\n",
       "      <td>528.21</td>\n",
       "      <td>78,560,600</td>\n",
       "      <td>528.21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>May 01, 2021</td>\n",
       "      <td>512.65</td>\n",
       "      <td>518.95</td>\n",
       "      <td>478.54</td>\n",
       "      <td>502.81</td>\n",
       "      <td>66,927,600</td>\n",
       "      <td>502.81</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Apr 01, 2021</td>\n",
       "      <td>529.93</td>\n",
       "      <td>563.56</td>\n",
       "      <td>499.00</td>\n",
       "      <td>513.47</td>\n",
       "      <td>111,573,300</td>\n",
       "      <td>513.47</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Mar 01, 2021</td>\n",
       "      <td>545.57</td>\n",
       "      <td>556.99</td>\n",
       "      <td>492.85</td>\n",
       "      <td>521.66</td>\n",
       "      <td>90,183,900</td>\n",
       "      <td>521.66</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Feb 01, 2021</td>\n",
       "      <td>536.79</td>\n",
       "      <td>566.65</td>\n",
       "      <td>518.28</td>\n",
       "      <td>538.85</td>\n",
       "      <td>61,902,300</td>\n",
       "      <td>538.85</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           Date    Open    High     Low   Close       Volume Adj Close\n",
       "0  Jun 01, 2021  504.01  536.13  482.14  528.21   78,560,600    528.21\n",
       "1  May 01, 2021  512.65  518.95  478.54  502.81   66,927,600    502.81\n",
       "2  Apr 01, 2021  529.93  563.56  499.00  513.47  111,573,300    513.47\n",
       "3  Mar 01, 2021  545.57  556.99  492.85  521.66   90,183,900    521.66\n",
       "4  Feb 01, 2021  536.79  566.65  518.28  538.85   61,902,300    538.85"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "netflix_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also use the pandas `read_html` function using the url\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "read_html_pandas_data = pd.read_html(url)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or we can convert the BeautifulSoup object to a string\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "read_html_pandas_data = pd.read_html(str(soup))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Beacause there is only one table on the page, we just take the first table in the list returned\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Open</th>\n",
       "      <th>High</th>\n",
       "      <th>Low</th>\n",
       "      <th>Close*</th>\n",
       "      <th>Adj Close**</th>\n",
       "      <th>Volume</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Jun 01, 2021</td>\n",
       "      <td>504.01</td>\n",
       "      <td>536.13</td>\n",
       "      <td>482.14</td>\n",
       "      <td>528.21</td>\n",
       "      <td>528.21</td>\n",
       "      <td>78560600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>May 01, 2021</td>\n",
       "      <td>512.65</td>\n",
       "      <td>518.95</td>\n",
       "      <td>478.54</td>\n",
       "      <td>502.81</td>\n",
       "      <td>502.81</td>\n",
       "      <td>66927600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Apr 01, 2021</td>\n",
       "      <td>529.93</td>\n",
       "      <td>563.56</td>\n",
       "      <td>499.00</td>\n",
       "      <td>513.47</td>\n",
       "      <td>513.47</td>\n",
       "      <td>111573300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Mar 01, 2021</td>\n",
       "      <td>545.57</td>\n",
       "      <td>556.99</td>\n",
       "      <td>492.85</td>\n",
       "      <td>521.66</td>\n",
       "      <td>521.66</td>\n",
       "      <td>90183900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Feb 01, 2021</td>\n",
       "      <td>536.79</td>\n",
       "      <td>566.65</td>\n",
       "      <td>518.28</td>\n",
       "      <td>538.85</td>\n",
       "      <td>538.85</td>\n",
       "      <td>61902300</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           Date    Open    High     Low  Close* Adj Close**     Volume\n",
       "0  Jun 01, 2021  504.01  536.13  482.14  528.21      528.21   78560600\n",
       "1  May 01, 2021  512.65  518.95  478.54  502.81      502.81   66927600\n",
       "2  Apr 01, 2021  529.93  563.56  499.00  513.47      513.47  111573300\n",
       "3  Mar 01, 2021  545.57  556.99  492.85  521.66      521.66   90183900\n",
       "4  Feb 01, 2021  536.79  566.65  518.28  538.85      538.85   61902300"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "netflix_dataframe = read_html_pandas_data[0]\n",
    "\n",
    "netflix_dataframe.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using Webscraping to Extract Stock Data Exercise\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use the `requests` library to download the webpage <https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0220EN-SkillsNetwork/labs/project/amazon_data_webpage.html>. Save the text of the response as a variable named `html_data`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "new_url = \"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0220EN-SkillsNetwork/labs/project/amazon_data_webpage.html\"\n",
    "html_data = requests.get(new_url).text"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Parse the html data using `beautiful_soup`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "beautiful_soup=BeautifulSoup(html_data,'html5lib')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Question 1</b> What is the content of the title attribute:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The title is: <title>Amazon.com, Inc. (AMZN) Stock Historical Prices &amp; Data - Yahoo Finance</title>\n"
     ]
    }
   ],
   "source": [
    "soup_title=beautiful_soup.title\n",
    "print(\"The title is:\",soup_title)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using beautiful soup extract the table with historical share prices and store it into a dataframe named `amazon_data`. The dataframe should have columns Date, Open, High, Low, Close, Adj Close, and Volume. Fill in each variable with the correct data from the list `col`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "amazon_data = pd.DataFrame(columns=[\"Date\", \"Open\", \"High\", \"Low\", \"Close\", \"Volume\"])\n",
    "\n",
    "for row in soup.find(\"tbody\").find_all(\"tr\"):\n",
    "    col = row.find_all(\"td\")\n",
    "    date = col[0].text\n",
    "    Open = col[1].text\n",
    "    high = col[2].text\n",
    "    low = col[3].text\n",
    "    close = col[4].text\n",
    "    adj_close = col[5].text\n",
    "    volume = col[6].text\n",
    "    \n",
    "    amazon_data = amazon_data.append({\"Date\":date, \"Open\":Open, \"High\":high, \"Low\":low, \"Close\":close, \"Adj Close\":adj_close, \"Volume\":volume}, ignore_index=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Print out the first five rows of the `amazon_data` dataframe you created.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Open</th>\n",
       "      <th>High</th>\n",
       "      <th>Low</th>\n",
       "      <th>Close</th>\n",
       "      <th>Volume</th>\n",
       "      <th>Adj Close</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Jun 01, 2021</td>\n",
       "      <td>504.01</td>\n",
       "      <td>536.13</td>\n",
       "      <td>482.14</td>\n",
       "      <td>528.21</td>\n",
       "      <td>78,560,600</td>\n",
       "      <td>528.21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>May 01, 2021</td>\n",
       "      <td>512.65</td>\n",
       "      <td>518.95</td>\n",
       "      <td>478.54</td>\n",
       "      <td>502.81</td>\n",
       "      <td>66,927,600</td>\n",
       "      <td>502.81</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Apr 01, 2021</td>\n",
       "      <td>529.93</td>\n",
       "      <td>563.56</td>\n",
       "      <td>499.00</td>\n",
       "      <td>513.47</td>\n",
       "      <td>111,573,300</td>\n",
       "      <td>513.47</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Mar 01, 2021</td>\n",
       "      <td>545.57</td>\n",
       "      <td>556.99</td>\n",
       "      <td>492.85</td>\n",
       "      <td>521.66</td>\n",
       "      <td>90,183,900</td>\n",
       "      <td>521.66</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Feb 01, 2021</td>\n",
       "      <td>536.79</td>\n",
       "      <td>566.65</td>\n",
       "      <td>518.28</td>\n",
       "      <td>538.85</td>\n",
       "      <td>61,902,300</td>\n",
       "      <td>538.85</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           Date    Open    High     Low   Close       Volume Adj Close\n",
       "0  Jun 01, 2021  504.01  536.13  482.14  528.21   78,560,600    528.21\n",
       "1  May 01, 2021  512.65  518.95  478.54  502.81   66,927,600    502.81\n",
       "2  Apr 01, 2021  529.93  563.56  499.00  513.47  111,573,300    513.47\n",
       "3  Mar 01, 2021  545.57  556.99  492.85  521.66   90,183,900    521.66\n",
       "4  Feb 01, 2021  536.79  566.65  518.28  538.85   61,902,300    538.85"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "amazon_data.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Question 2</b> What is the name of the columns of the dataframe\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Index(['Date', 'Open', 'High', 'Low', 'Close', 'Volume', 'Adj Close'], dtype='object')\n"
     ]
    }
   ],
   "source": [
    "print(amazon_data.columns)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Question 3</b> What is the `Open` of the last row of the amazon_data dataframe?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Open</th>\n",
       "      <th>High</th>\n",
       "      <th>Low</th>\n",
       "      <th>Close</th>\n",
       "      <th>Volume</th>\n",
       "      <th>Adj Close</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>65</th>\n",
       "      <td>Jan 01, 2016</td>\n",
       "      <td>109.00</td>\n",
       "      <td>122.18</td>\n",
       "      <td>90.11</td>\n",
       "      <td>91.84</td>\n",
       "      <td>488,193,200</td>\n",
       "      <td>91.84</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>66</th>\n",
       "      <td>Dec 01, 2015</td>\n",
       "      <td>124.47</td>\n",
       "      <td>133.27</td>\n",
       "      <td>113.85</td>\n",
       "      <td>114.38</td>\n",
       "      <td>319,939,200</td>\n",
       "      <td>114.38</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>67</th>\n",
       "      <td>Nov 01, 2015</td>\n",
       "      <td>109.20</td>\n",
       "      <td>126.60</td>\n",
       "      <td>101.86</td>\n",
       "      <td>123.33</td>\n",
       "      <td>320,321,800</td>\n",
       "      <td>123.33</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>68</th>\n",
       "      <td>Oct 01, 2015</td>\n",
       "      <td>102.91</td>\n",
       "      <td>115.83</td>\n",
       "      <td>96.26</td>\n",
       "      <td>108.38</td>\n",
       "      <td>446,204,400</td>\n",
       "      <td>108.38</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>69</th>\n",
       "      <td>Sep 01, 2015</td>\n",
       "      <td>109.35</td>\n",
       "      <td>111.24</td>\n",
       "      <td>93.55</td>\n",
       "      <td>103.26</td>\n",
       "      <td>497,401,200</td>\n",
       "      <td>103.26</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            Date    Open    High     Low   Close       Volume Adj Close\n",
       "65  Jan 01, 2016  109.00  122.18   90.11   91.84  488,193,200     91.84\n",
       "66  Dec 01, 2015  124.47  133.27  113.85  114.38  319,939,200    114.38\n",
       "67  Nov 01, 2015  109.20  126.60  101.86  123.33  320,321,800    123.33\n",
       "68  Oct 01, 2015  102.91  115.83   96.26  108.38  446,204,400    108.38\n",
       "69  Sep 01, 2015  109.35  111.24   93.55  103.26  497,401,200    103.26"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "amazon_data.tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>About the Authors:</h2> \n",
    "\n",
    "<a href=\"https://www.linkedin.com/in/joseph-s-50398b136/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0220ENSkillsNetwork900-2022-01-01\">Joseph Santarcangelo</a> has a PhD in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD.\n",
    "\n",
    "Azim Hirjani\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Change Log\n",
    "\n",
    "| Date (YYYY-MM-DD) | Version | Changed By | Change Description |\n",
    "| ----------------- | ------- | ---------- | ------------------ |\n",
    "\n",
    "```\n",
    "| 2021-06-09       | 1.2     | Lakshmi Holla|Added URL in question 3 |\n",
    "```\n",
    "\n",
    "\\| 2020-11-10        | 1.1     | Malika Singla | Deleted the Optional part |\n",
    "\\| 2020-08-27        | 1.0     | Malika Singla | Added lab to GitLab       |\n",
    "\n",
    "<hr>\n",
    "\n",
    "## <h3 align=\"center\"> © IBM Corporation 2020. All rights reserved. <h3/>\n",
    "\n",
    "<p>\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python",
   "language": "python",
   "name": "conda-env-python-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}