2_data_wrangling_and_retrieval/2.5_apis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "35870f3e-1791-4fc3-ba77-6b01a78bca50",
   "metadata": {},
   "source": [
    "# APIS "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34841864-ea2a-436e-b85a-5cc871472849",
   "metadata": {},
   "source": [
    "![apis_gif](https://media.giphy.com/media/wepUQluC5smgEd4Qz4/giphy.gif)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a6795d4c-fdfe-4f6d-ad6b-2479d9b0826d",
   "metadata": {
    "toc": true
   },
   "source": [
    "<h1>Table of Contents<span class=\"tocSkip\"></span></h1>\n",
    "<div class=\"toc\"><ul class=\"toc-item\"><li><span><a href=\"#APIS\" data-toc-modified-id=\"APIS-1\"><span class=\"toc-item-num\">1&nbsp;&nbsp;</span>APIS</a></span><ul class=\"toc-item\"><li><span><a href=\"#Web-Communication-Fundamentals\" data-toc-modified-id=\"Web-Communication-Fundamentals-1.1\"><span class=\"toc-item-num\">1.1&nbsp;&nbsp;</span>Web Communication Fundamentals</a></span><ul class=\"toc-item\"><li><span><a href=\"#DNS,-IP\" data-toc-modified-id=\"DNS,-IP-1.1.1\"><span class=\"toc-item-num\">1.1.1&nbsp;&nbsp;</span>DNS, IP</a></span></li><li><span><a href=\"#HTTP\" data-toc-modified-id=\"HTTP-1.1.2\"><span class=\"toc-item-num\">1.1.2&nbsp;&nbsp;</span>HTTP</a></span></li><li><span><a href=\"#URL\" data-toc-modified-id=\"URL-1.1.3\"><span class=\"toc-item-num\">1.1.3&nbsp;&nbsp;</span>URL</a></span></li></ul></li><li><span><a href=\"#HTTP-Requests-and-Responses\" data-toc-modified-id=\"HTTP-Requests-and-Responses-1.2\"><span class=\"toc-item-num\">1.2&nbsp;&nbsp;</span>HTTP Requests and Responses</a></span><ul class=\"toc-item\"><li><span><a href=\"#Requests\" data-toc-modified-id=\"Requests-1.2.1\"><span class=\"toc-item-num\">1.2.1&nbsp;&nbsp;</span>Requests</a></span></li><li><span><a href=\"#Response\" data-toc-modified-id=\"Response-1.2.2\"><span class=\"toc-item-num\">1.2.2&nbsp;&nbsp;</span>Response</a></span></li></ul></li><li><span><a href=\"#APIs\" data-toc-modified-id=\"APIs-1.3\"><span class=\"toc-item-num\">1.3&nbsp;&nbsp;</span>APIs</a></span><ul class=\"toc-item\"><li><span><a href=\"#RESTful-APIs\" data-toc-modified-id=\"RESTful-APIs-1.3.1\"><span class=\"toc-item-num\">1.3.1&nbsp;&nbsp;</span>RESTful APIs</a></span></li></ul></li><li><span><a href=\"#Requests-in-Python\" data-toc-modified-id=\"Requests-in-Python-1.4\"><span class=\"toc-item-num\">1.4&nbsp;&nbsp;</span>Requests in Python</a></span><ul class=\"toc-item\"><li><span><a href=\"#GET\" data-toc-modified-id=\"GET-1.4.1\"><span class=\"toc-item-num\">1.4.1&nbsp;&nbsp;</span>GET</a></span><ul class=\"toc-item\"><li><span><a href=\"#Accessing-an-endpoint\" data-toc-modified-id=\"Accessing-an-endpoint-1.4.1.1\"><span class=\"toc-item-num\">1.4.1.1&nbsp;&nbsp;</span>Accessing an endpoint</a></span></li><li><span><a href=\"#Response-headers\" data-toc-modified-id=\"Response-headers-1.4.1.2\"><span class=\"toc-item-num\">1.4.1.2&nbsp;&nbsp;</span>Response headers</a></span></li><li><span><a href=\"#Parameters\" data-toc-modified-id=\"Parameters-1.4.1.3\"><span class=\"toc-item-num\">1.4.1.3&nbsp;&nbsp;</span>Parameters</a></span></li><li><span><a href=\"#Request-Headers\" data-toc-modified-id=\"Request-Headers-1.4.1.4\"><span class=\"toc-item-num\">1.4.1.4&nbsp;&nbsp;</span>Request Headers</a></span></li></ul></li></ul></li><li><span><a href=\"#APIs-examples\" data-toc-modified-id=\"APIs-examples-1.5\"><span class=\"toc-item-num\">1.5&nbsp;&nbsp;</span>APIs examples</a></span><ul class=\"toc-item\"><li><span><a href=\"#NewsAPI\" data-toc-modified-id=\"NewsAPI-1.5.1\"><span class=\"toc-item-num\">1.5.1&nbsp;&nbsp;</span>NewsAPI</a></span><ul class=\"toc-item\"><li><span><a href=\"#💡-Check-for-understanding\" data-toc-modified-id=\"💡-Check-for-understanding-1.5.1.1\"><span class=\"toc-item-num\">1.5.1.1&nbsp;&nbsp;</span>💡 Check for understanding</a></span></li></ul></li><li><span><a href=\"#Pokemon-API\" data-toc-modified-id=\"Pokemon-API-1.5.2\"><span class=\"toc-item-num\">1.5.2&nbsp;&nbsp;</span>Pokemon API</a></span><ul class=\"toc-item\"><li><span><a href=\"#json_normalize()\" data-toc-modified-id=\"json_normalize()-1.5.2.1\"><span class=\"toc-item-num\">1.5.2.1&nbsp;&nbsp;</span>json_normalize()</a></span></li></ul></li><li><span><a href=\"#Coincap-API\" data-toc-modified-id=\"Coincap-API-1.5.3\"><span class=\"toc-item-num\">1.5.3&nbsp;&nbsp;</span>Coincap API</a></span></li><li><span><a href=\"#Api-Jokes\" data-toc-modified-id=\"Api-Jokes-1.5.4\"><span class=\"toc-item-num\">1.5.4&nbsp;&nbsp;</span>Api Jokes</a></span></li><li><span><a href=\"#💡-Check-for-understanding\" data-toc-modified-id=\"💡-Check-for-understanding-1.5.5\"><span class=\"toc-item-num\">1.5.5&nbsp;&nbsp;</span>💡 Check for understanding</a></span></li></ul></li><li><span><a href=\"#API-Wrappers\" data-toc-modified-id=\"API-Wrappers-1.6\"><span class=\"toc-item-num\">1.6&nbsp;&nbsp;</span>API Wrappers</a></span></li><li><span><a href=\"#Summary\" data-toc-modified-id=\"Summary-1.7\"><span class=\"toc-item-num\">1.7&nbsp;&nbsp;</span>Summary</a></span></li><li><span><a href=\"#Glossary\" data-toc-modified-id=\"Glossary-1.8\"><span class=\"toc-item-num\">1.8&nbsp;&nbsp;</span>Glossary</a></span></li><li><span><a href=\"#Further-materials\" data-toc-modified-id=\"Further-materials-1.9\"><span class=\"toc-item-num\">1.9&nbsp;&nbsp;</span>Further materials</a></span></li></ul></li></ul></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78337d49-bd91-4990-b1d6-fd2d4c03e6df",
   "metadata": {},
   "source": [
    "## Web Communication Fundamentals"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f5600c7-4e6b-4b57-986e-740538741ca9",
   "metadata": {},
   "source": [
    "![enjuto](https://www.publico.es/files/article_main/uploads//2014/12/13/548bb18f1d9ef.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "582db4b0-9f2d-4847-b52c-f11eb6ae8933",
   "metadata": {},
   "source": [
    "### DNS, IP"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e7a9565-1961-4631-a6a1-71735dae6f16",
   "metadata": {},
   "source": [
    "[How do we connect to www.google.com?](https://www.youtube.com/watch?v=sUhEqT_HSBI&ab_channel=ProfeSang)\n",
    "* **DNS (Domain Name System)**: it's essentially the phonebook of the internet. Humans access information online through domain names, like \"google.com\". Web browsers, however, interact through Internet Protocol (IP) addresses. In this example, the DNS maps the internet address www.google.com to the server's IP: 216.58.222.196\n",
    " * **IP**: Server identification. A code that allows information to be sent and received by the correct parties\n",
    " * **Domain Providers**: Sell and purchase Internet domains"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d42f993-6f37-42c7-8f04-4c5526c0825e",
   "metadata": {},
   "source": [
    "*When making API calls to specific domains, the DNS translates the human-readable domain name into an IP address.*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "216e4441-45eb-4a0a-b578-f4f056a724b1",
   "metadata": {},
   "source": [
    "### HTTP"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6d120e37-ac79-44c9-8fc5-3fe07f4ad7f6",
   "metadata": {},
   "source": [
    "**H**yper **T**ext **T**ransfer **P**rotocol\n",
    "- HTTP is a communications protocol that provides a structure for requests between the client and the server on a network.\n",
    "- For example, the web browser on the user's computer (the client) uses the HTTP protocol to request information from a website on a server.\n",
    "\n",
    "*We will use APIs that employ the **HTTP** protocol as transport.*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b9f3b1b-af11-419a-bd55-771a937b72dc",
   "metadata": {},
   "source": [
    "### URL\n",
    "Contains information about the resource being requested from the **server**."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6a7522a-6679-4d2d-9587-6b583c41a55f",
   "metadata": {},
   "source": [
    "![](https://github.com/data-bootcamp-v4/lessons/blob/main/img/http.png?raw=true)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "246020a7-6a89-4857-98c4-8f29c1d3c827",
   "metadata": {},
   "source": [
    "Examples:\n",
    "\n",
    "- https://www.google.com/webhp?authuser=2\n",
    "- https://www.towardsdatascience.com\n",
    "- https://www.ironhack.com/\n",
    "    - Protocol: https (https == http is the same, but https is encrypted)\n",
    "    - Domain Name --> ironhack\n",
    "    - TLD (Top-Level Domain) --> .com"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd726622-8824-4e1f-a166-bb0d7e94fd08",
   "metadata": {},
   "source": [
    "*When calling an API, you specify the URL of the API endpoint, which tells the system where to send the request.*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5fd2c3c5-1e3e-44a2-923b-c6dc1bb19f7c",
   "metadata": {},
   "source": [
    "## HTTP Requests and Responses"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b658ec7-5230-44ab-b10b-f8cc1c65edab",
   "metadata": {},
   "source": [
    "![](https://github.com/data-bootcamp-v4/lessons/blob/main/img/request-response.png?raw=true)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4941c601-e2f2-41bd-a0f3-c5e33fc8e282",
   "metadata": {},
   "source": [
    "**Requests** and **responses** are fundamental components of the client-server communication model."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b75d6570-aae7-492a-bcd0-86da6d102545",
   "metadata": {},
   "source": [
    "### Requests"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6d39d707-2230-4f72-af04-8c49abeb413e",
   "metadata": {},
   "source": [
    "These are queries or calls **sent by the client** (such as a web browser or other software) **to the server** in order to receive information (a **response**). \n",
    "\n",
    "A request typically consists of:\n",
    "- A method (such as GET, POST, PUT, DELETE) that defines the action to be performed\n",
    "     * GET: read the information of the resource, without modifying it in any way. Accessing the website from the browser **gets** information.\n",
    "- The URL or endpoint specifying the resource\n",
    "- Optional additional information such as:\n",
    "    - Headers (metadata): User-Agent, Accept-Language\n",
    "    - Parameters\n",
    "    - Body content. \n",
    "    \n",
    "For example, a client might send a GET request to retrieve information from a web page or a POST request to submit form data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03152bb9-ab13-458c-94d2-f5cb53117056",
   "metadata": {},
   "source": [
    "### Response"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc180f91-d6bf-4cb9-bf07-443d47ba0c4f",
   "metadata": {},
   "source": [
    "These are the answers or data sent **by the server back to the client in reply to a request**. \n",
    "\n",
    "A response typically includes:\n",
    "- A status code that indicates the success or failure of the request\n",
    "- Headers with meta-information about the server's behavior\n",
    "- The actual content or data (if applicable), such as HTML, JSON (similar to a Python dictionary), images, or other media types.\n",
    "\n",
    "An important part of the **header** is the **status code**. This code is a numerical value that indicates the server's result. There are different status codes depending on whether the server has managed to carry out the request or has not managed to do anything. These are some groups of status codes:\n",
    "\n",
    "- **2xx successful**: the request was successfully received, understood, and accepted\n",
    "- **3xx redirection**: more actions are required to complete the request\n",
    "- **4xx client error**: the request contains incorrect syntax or cannot be fulfilled\n",
    "- **5xx server error**: the server has failed to complete an apparently valid request\n",
    "\n",
    "Complete list:\n",
    "https://en.wikipedia.org/wiki/List_of_HTTP_status_codes\n",
    "\n",
    "Much more fun:\n",
    "https://http.cat/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f1ad409-99e9-4701-b879-f81d13ece283",
   "metadata": {
    "tags": []
   },
   "source": [
    "## APIs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "492c462e-3d81-4330-828f-ff2b027989d8",
   "metadata": {},
   "source": [
    "**A**pplication **P**rogramming **I**nterface\n",
    "\n",
    "- APIs define a set of rules and protocols that allow different software applications to communicate with each other.\n",
    "- The client calls the server through an API, and the server responds.\n",
    "-  Within a project, the backend may want to share information with the frontend without access to the database."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0849403b-d633-4756-82d2-cb74ac4d254b",
   "metadata": {},
   "source": [
    "![](https://github.com/data-bootcamp-v4/lessons/blob/main/img/api.png?raw=true)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b86f18cb-8590-4a6e-896b-c5e465e7719f",
   "metadata": {},
   "source": [
    "### RESTful APIs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62891457-6b5c-4443-9fb5-77f901ea7571",
   "metadata": {},
   "source": [
    "As a data analyst, you often need to access data from various sources. RESTful APIs provide a **standardized way** to retrieve, update, or delete data from other systems."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c2bea7b-f629-45f3-94b3-6cb6eac1272e",
   "metadata": {},
   "source": [
    "- Usually, **we (the client) send a request, and they (the server) return a response, often in JSON - JavaScript Object Notation - format**. While JSON looks similar to the dictionaries we're used to in Python, it's worth noting that when represented as raw text (e.g., in an API response), the entire JSON structure is a *string*. However, within that structure, JSON can represent various data types like numbers, booleans, and arrays, not just strings.\n",
    "\n",
    "- APIs always need to provide documentation for their various services: **endpoints**.  Each endpoint is a different URL.  \n",
    "- Sometimes we have to pass **parameters** to an API endpoint, similar to when we pass parameters to a Python function.\n",
    "\n",
    "At this point in the Bootcamp, **reading documentation becomes essential**, as each API will \"work\" differently, and to use it, we will need to know what it requires.\n",
    "\n",
    "[Here](https://github.com/public-apis/public-apis) is a repository where you can find several free APIs if you feel like exploring a bit."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "33c6c206-715a-42b8-929c-691d9c66fe4d",
   "metadata": {},
   "source": [
    "## Requests in Python"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "515484d8-81ec-48c1-abad-c778232b583f",
   "metadata": {},
   "source": [
    "You can make requests to RESTful APIs using libraries like `requests`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "54d23d14-7425-45c6-8ba5-74e5db2c047a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# If you don't have it installed you can do use using pip or pip3\n",
    "\n",
    "#!pip install requests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9ddac917-8539-46e8-abc9-37111e2fa74c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da5aff9d-8a79-4171-96b1-b7f0d2daf1cd",
   "metadata": {},
   "source": [
    "### GET"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2018fa77-7bce-437e-a806-69844737210b",
   "metadata": {},
   "source": [
    "Making a `GET request` to the API is simply a call to a `URL` that returns information when provided with the appropriate `parameters`. We will only perform **GET** requests."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "718172e2-5654-4536-bd72-594d80b31f8c",
   "metadata": {},
   "source": [
    "Here is an example on how to make a request with Python:\n",
    "\n",
    "```python\n",
    "url = \"https://api.example.com/products\"\n",
    "response = requests.get(url)\n",
    "\n",
    "if response.status_code == 200:\n",
    "    products = response.json()\n",
    "    # Now you can analyze and work with the products data in Python\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "307395ef-cfd3-4175-99d5-a6ce6213adb1",
   "metadata": {},
   "source": [
    "#### Accessing an endpoint\n",
    "\n",
    "As we mentioned above, APIs always need to provide documentation for their various services: **endpoints**.  Each endpoint is a different URL.  \n",
    "\n",
    "**Example: ISS API**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "792140dd-3cda-4355-86bd-dd6732880803",
   "metadata": {},
   "source": [
    "Let's get information from ISS (International Space Station)! We'll start looking at the [ISS API documentation](https://wheretheiss.at/w/developer)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2be4164-137b-411d-a78c-38bdd94ce9ed",
   "metadata": {},
   "source": [
    "This API allows you to access various data related to the International Space Station (ISS), including its current, past, or future position, timezone information for specific coordinates, and more.\n",
    "\n",
    "**Key Features**\n",
    "- **Authentication**: No authentication is currently required, but future endpoints may include this.\n",
    "- **Rate Limiting**: Limited to approximately 1 request per second.\n",
    "- **Responses**: Default to JSON format, with optional parameters to modify response appearance.\n",
    "- **Endpoints**: Several endpoints provide different types of information:\n",
    "    - **satellites**: Information about satellites, including the ISS.\n",
    "    - **satellites/[id]**: Position, velocity, and related information for a satellite.\n",
    "    - **satellites/[id]/positions**: Position data for specific timestamps.\n",
    "    - **satellites/[id]/tles**: TLE (Two-Line Element Set) data in either JSON or text format.\n",
    "    - **coordinates/[lat,lon]**: Timezone information for specific coordinates.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f1272582-bc6d-4f43-9e39-47d39ffd1bfd",
   "metadata": {},
   "source": [
    "**Examples**\n",
    "- Satellite details: `https://api.wheretheiss.at/v1/satellites`\n",
    "- ISS position: `https://api.wheretheiss.at/v1/satellites/25544`\n",
    "- Coordinates information: `https://api.wheretheiss.at/v1/coordinates/37.795517,-122.393693`\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ca5b7da-eec0-41ca-924b-67687f794e23",
   "metadata": {},
   "source": [
    "**Endpoint satellites**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "05fad64c-5f20-4b7f-a865-9659b9cc8783",
   "metadata": {},
   "outputs": [],
   "source": [
    "# We'll use the endpoint satellites, we read it gives us information about satellites\n",
    "\n",
    "url = \"https://api.wheretheiss.at/v1/satellites\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "68f33b09-b943-486c-98e0-2d18b25640c5",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = requests.get(url) # We use get method to make the request and get information from the API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "67938f79-0981-471e-9201-89352e99775b",
   "metadata": {},
   "outputs": [],
   "source": [
    "type(response) # Lets look at the type of the response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "310b2c20-4a51-47de-9983-1e266496c951",
   "metadata": {},
   "outputs": [],
   "source": [
    "response.status_code # We can access status code. 200 is OK"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "488cbcbe-2d4d-42c2-b92a-ec76b70b705b",
   "metadata": {},
   "source": [
    "We can access the response content, which returns a string"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5196a512-6690-49c9-971a-bc9cb2383932",
   "metadata": {},
   "outputs": [],
   "source": [
    "response.content"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "820ad33e-4664-46b6-a51a-0b277ba04c4d",
   "metadata": {},
   "source": [
    "If response Content-Type is json, we can access it better with `.json()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7f44e27c-7866-4e9c-8965-a20ccaef0e43",
   "metadata": {},
   "outputs": [],
   "source": [
    "info = response.json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cd406dbc-74f9-490b-b8c6-73ef02e9b9d4",
   "metadata": {},
   "outputs": [],
   "source": [
    "info"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "992d8cf6-8e5c-448d-bb1c-e9416aefe428",
   "metadata": {},
   "outputs": [],
   "source": [
    "type(info) # It returns a list of dictionaries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1a7d7dc2-f2d7-44a7-9bda-e5cc0490dffe",
   "metadata": {},
   "outputs": [],
   "source": [
    "info[0][\"id\"] # So we can access the id like this"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "903f3024-0998-459d-bb20-60a639def27e",
   "metadata": {},
   "source": [
    "#### Response headers \n",
    "\n",
    "Response headers are part of the HTTP response that a server sends back to the client after a request has been made. These headers provide meta-information about the response and can affect how the client handles the response.\n",
    "\n",
    "Here are some common response headers and what they typically represent:\n",
    "\n",
    "1. **`Date`**: Represents the date and time at which the response was sent.\n",
    "\n",
    "2. **`Server`**: Provides information about the software used by the originating server.\n",
    "\n",
    "3. **`X-Rate-Limit`**: Sometimes used in APIs to inform the client about rate limiting policies, such as the number of allowed requests in a given time frame.\n",
    "\n",
    "4. **`Content-Type`**: Specifies the media type of the resource or data the server is sending back. For example, it could be `application/json` for a JSON object, `text/html` for an HTML page, or `image/png` for an image.\n",
    "\n",
    "And more."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a53d51bc-2346-4139-a23f-49a34dc3b475",
   "metadata": {},
   "outputs": [],
   "source": [
    "response.headers # We can also access the response headers"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a46967a4-763f-4d4e-bca9-c3a37742f65e",
   "metadata": {},
   "source": [
    "#### Parameters"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69017232-6063-4ee5-a13c-0eb40f182b1e",
   "metadata": {},
   "source": [
    "As we mentioned above, sometimes we can pass **parameters** to an API endpoint, similar to when we pass parameters to a Python function."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4dbf1646-5953-44b9-a2e3-a7dbee7f1a4e",
   "metadata": {},
   "source": [
    "In the example above, we didn't use any parameters in the endpoint `satellites` since the [documentation](\"https://wheretheiss.at/w/developer\") said *Parameters: None*."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed0d4cf7-fa4d-4c83-8d56-4a25accc3d01",
   "metadata": {},
   "source": [
    "API parameters are specific values that you include in a request to an API endpoint to filter, sort, or detail the data that you want to retrieve. They allow you to customize the request to get exactly the information you need."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dbf6baed-1967-46ea-908f-bff2fd59dc46",
   "metadata": {},
   "source": [
    "There are several types of parameters that can be used in API requests, such as Path Parameters, Query Parameters, Header Parameters and Request Body Parameters. Let's look at them with an example using another endpoint."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f854df58-1a0a-4759-9991-30816cf45c92",
   "metadata": {},
   "source": [
    "##### 1. **Path Parameters**\n",
    "\n",
    "These are embedded in the URL path and are used to identify a specific resource. For example, in the URL above `https://api.wheretheiss.at/v1/satellites/25544`, the number `25544` is a path parameter that identifies a specific sattelite."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df9a0f8c-b363-4ceb-bcc6-04128ec2bffc",
   "metadata": {
    "tags": []
   },
   "source": [
    "**Endpoint satellites/id**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f7f1892c-1908-438d-a072-a77fc2214e52",
   "metadata": {},
   "source": [
    "Lets use endpoint satellites/[id] with the id we got from the previous endpoint.\n",
    "Important, we need to provide the whole url for the endpoint as a `string`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "60e19b4e-1476-456c-8272-a3175708368e",
   "metadata": {},
   "outputs": [],
   "source": [
    "url_iss_position = \"https://api.wheretheiss.at/v1/satellites/\"+str(info[0][\"id\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "42cb7357-ff48-4f7d-81df-8060c7ed794f",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = requests.get(url_iss_position)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "340a190c-6010-4134-bfd5-f4acc5484e8b",
   "metadata": {},
   "outputs": [],
   "source": [
    "response.status_code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a9ddae28-476f-4133-8a25-4e35c3fac891",
   "metadata": {},
   "outputs": [],
   "source": [
    "response.json()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c21289e-ed73-4fdf-8ff2-8ec53d1bedbb",
   "metadata": {},
   "source": [
    "Every time we call the endpoint, the information changes since it gives us the current position of the satellite. Let's check that with a loop to gather information at different times."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c02e4902-90b6-4f31-8326-2dcee2478aa8",
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "\n",
    "positions = []\n",
    "\n",
    "for i in range(10):\n",
    "    response = requests.get(url_iss_position)\n",
    "    data = response.json()\n",
    "    \n",
    "    positions.append(data)\n",
    "    time.sleep(0.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3801506d-82c7-4c1b-b4ab-81fce6a73b36",
   "metadata": {},
   "outputs": [],
   "source": [
    "type(positions) # We created a list with responses"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5f3da8cc-7ad5-4313-8bbd-71e733359747",
   "metadata": {},
   "outputs": [],
   "source": [
    "type(positions[0]) # Is a list of dictionaries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8d24d4c8-860c-4a6f-a5ec-db44ed9d4c65",
   "metadata": {},
   "outputs": [],
   "source": [
    "positions[0] # So we can access each dictionary by its position in the list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b588249b-ef73-4cd0-8c3f-4a7a46c9156f",
   "metadata": {},
   "outputs": [],
   "source": [
    "positions[9]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57c55b8a-1b23-4e9e-8f7b-d24b7a1b375e",
   "metadata": {},
   "source": [
    "Let's extract 10 latitudes from the 10 dictionaries in the list, with 2 decimals."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3371418e-6637-4857-913f-01fd244bd687",
   "metadata": {},
   "outputs": [],
   "source": [
    "latitudes = [round(p[\"latitude\"], 2) for p in positions]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "256a7492-b9de-416b-9cad-3b4155f767e4",
   "metadata": {},
   "outputs": [],
   "source": [
    "latitudes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7b841d34-6a6f-4f31-9764-1d5ac4d2f35f",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "pd.DataFrame(positions) # We can create a dataframe from the API response to work with it better"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "204abde8-009b-4217-8295-51fbde187742",
   "metadata": {
    "tags": []
   },
   "source": [
    "\n",
    "##### 2. **Query Parameters**\n",
    "\n",
    "These are added to the end of the URL after a question mark (`?`) and are often used to filter or sort the response. \n",
    "\n",
    "For example, in the ISS documentation, it mentions that there is a parameter `units` that can take values `miles` or `kilometers`. So in the URL `https://api.wheretheiss.at/v1/satellites/25544?units=miles`, we added `units=miles` which is a query parameter that shows the data in \"miles\". \n",
    "\n",
    "In general, we add the parameters like this `?param1=value1&param2=value2...` at the end of the URL."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b252f4b2-f739-4d78-8a14-7933b946c309",
   "metadata": {},
   "outputs": [],
   "source": [
    "url_iss_position"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b8bfc174-f81f-41cc-92f4-eec321e1961a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# We saw in the documentation that we can add a parameter units to use miles or kilometers\n",
    "\n",
    "url_iss_position2 = 'https://api.wheretheiss.at/v1/satellites/25544?units=miles'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "47ead56f-1882-41f4-8850-67a6611fe36d",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = requests.get(url_iss_position2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5a032e2c-abab-4c49-97d1-341d74a22c68",
   "metadata": {},
   "outputs": [],
   "source": [
    "response.status_code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "395d4fed-6f83-4c96-8cf5-8523ce297e92",
   "metadata": {},
   "outputs": [],
   "source": [
    "response.json()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c50844d-2d2c-4768-8d8b-703a2cb1875c",
   "metadata": {},
   "source": [
    "We can also do it by passing to the argument `params` a dictionary with the parameters in the `get` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cdcdd6f7-628b-4978-8656-23f72ecd5cac",
   "metadata": {},
   "outputs": [],
   "source": [
    "parameters = {\"units\": \"miles\"}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "31d68523-2258-408f-ac24-0836af707cd9",
   "metadata": {},
   "outputs": [],
   "source": [
    "url_iss_position"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2688ca3e-e9ef-44cf-8e28-7c575f2d61be",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = requests.get(\n",
    "    url=url_iss_position,\n",
    "    params=parameters\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "87e3afb1-2e29-4c62-8162-e8b27756be0c",
   "metadata": {},
   "outputs": [],
   "source": [
    "response.json()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6aea75ed-be20-475d-98c1-86f4e9d2b1fd",
   "metadata": {},
   "source": [
    "\n",
    "##### 3. **Header Parameters**\n",
    "\n",
    "These are included in the request header and can be used for various purposes, such as authentication (e.g., sending an API key or token), content negotiation (e.g., defining the response format), or custom settings defined by the API. If the API requires authentication, you might include an `Authorization` header with your API key."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78131fa0-a203-4d64-8fc2-939ee0ebf064",
   "metadata": {},
   "source": [
    "ISS API mentions that *currently there is no authentication required*. So we'll look at an example with a different API of `authentication` in `header parameters`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69484a90-65de-44b0-87bb-b591274f27b5",
   "metadata": {},
   "source": [
    "#### Request Headers"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a5782c9a-84ad-434e-b4a5-5657f17592f1",
   "metadata": {},
   "source": [
    "Request headers are key-value pairs sent in an HTTP request to provide information about the request itself."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70c5dec7-204a-4c3b-a9e4-8c7bbf516b10",
   "metadata": {},
   "source": [
    "Some request headers are:\n",
    "\n",
    "1. **Content-Type**: Specifies the media type of the resource or data. Common examples include \"application/json\" for JSON data, \"text/html\" for HTML content, and \"application/xml\" for XML data.\n",
    "\n",
    "3. **Authorization**: Contains credentials for authenticating the client with the server, often used with tokens or other forms of authentication.\n",
    "\n",
    "4. **User-Agent**: Provides information about the client (browser or other client), including its version and operating system.\n",
    "\n",
    "\n",
    "In Python's `requests` library, you can include headers in a request by using the `headers` argument, like this:\n",
    "\n",
    "```python\n",
    "headers = {'Content-Type': 'application/json', 'Authorization': 'Bearer YOUR_TOKEN'}\n",
    "response = requests.get(url, headers=headers)\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a5311048-ef5e-4b04-9e4b-8ed700ab3076",
   "metadata": {},
   "source": [
    "⚠️🚨 ¡Careful with the authentication token! ⚠️🚨  A token is a personal credential for accessing an API, through which your request quota to the API is managed. Therefore, the ideal procedure is to ensure security by storing the token as a variable in an .env file. This way, you can call the environment variable without having to publicly display the token."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b3cf22bc-d4e1-4e51-876e-f4df41b70f05",
   "metadata": {},
   "source": [
    "**Example: News API**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a6b35b6f-4cb4-4b89-af23-3c9c65fb2686",
   "metadata": {},
   "source": [
    "The *News API* lets you locate articles and breaking news headlines from news sources and blogs across the web."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "58123f1a-b5e4-4b20-a02d-47077a4a3635",
   "metadata": {},
   "source": [
    "- Let's look at the [NewsAPI Authentication documentation](https://newsapi.org/docs/authentication). It mentions: \n",
    "```\n",
    "You can attach your API key to a request in one of three ways:\n",
    "\n",
    "- Via the apiKey querystring parameter.\n",
    "- Via the X-Api-Key HTTP header.\n",
    "- Via the Authorization HTTP header. Including Bearer is optional, and be sure not to base 64 encode it like you may have seen in other authentication tutorials.\n",
    "We strongly recommend the either of last two so that your API key isn't visible to others in logs or via request sniffing.\n",
    "\n",
    "If you don't append your API key correctly, or your API key is invalid, you will receive a 401 - Unauthorized HTTP error.\n",
    "```\n",
    "\n",
    "- Let's look at the documentation for the endpoints to see what **parameters** they accept: [NewsAPI endpoints](https://newsapi.org/docs/endpoints). We see we have the endpoint `/v2/top-headlines` – *returns breaking news headlines for countries, categories, and singular publishers. This is perfect for use with news tickers or anywhere you want to use live up-to-date news headlines.*\n",
    "If we look at the [documentation of that endpoint](https://newsapi.org/docs/endpoints/top-headlines) we see we have more parameters we can add such as `country` or `category`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e2853b7d-09f0-46e3-83df-c1a1b03f6b61",
   "metadata": {},
   "outputs": [],
   "source": [
    "url = \"http://newsapi.org/v2/top-headlines\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ab2d561c-97fc-4750-9334-f19ce984b2ae",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = requests.get(url)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1d44b74a-8282-4f74-bcbb-e030a2d0a26a",
   "metadata": {},
   "outputs": [],
   "source": [
    "response.status_code"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c01205a-e732-44bc-9fc0-cb3cefdfc76f",
   "metadata": {},
   "source": [
    "Let's try a made up key, using the apiKey querystring parameter. As we read in the documentation, this is not the recommended way."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "24d14d50-c5dc-4c20-ab4b-4f65d5e61a61",
   "metadata": {},
   "outputs": [],
   "source": [
    "url = f\"http://newsapi.org/v2/top-headlines?country=us&apiKey=manolonodoesnthavekey\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2d106abb-06ac-4432-b4ad-2c51fbd90e28",
   "metadata": {},
   "outputs": [],
   "source": [
    "url"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "304ff5dd-8d38-4641-9799-8cd7978434d0",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = requests.get(url)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bb9dc34b-21de-4d00-8544-fc9ea2c23f70",
   "metadata": {},
   "outputs": [],
   "source": [
    "response.status_code"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e829611f-a4a8-4b3d-90a2-31c1bb04fb2b",
   "metadata": {},
   "source": [
    "**4xx**: client error. This means, it is our error."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71322103-4234-4105-be5d-037cb6107e26",
   "metadata": {},
   "source": [
    "Lets do with a correct key. You should get your own API key through their website. Once we've saved our key, we'll send it via the X-Api-Key HTTP header."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a0ae416d-0215-4acd-9479-2ba49dab313b",
   "metadata": {},
   "source": [
    "##### Saving the API Key"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c509a90-72f3-4bba-9b29-bc4316bd3111",
   "metadata": {},
   "source": [
    "Storing an API key directly in your code can expose sensitive information, especially if your code is publicly available (e.g., on a public GitHub repository). The best practice for saving and loading an API key in your code involves the following steps:\n",
    "\n",
    "1. **Storing the API Key**:\n",
    "    - **Use Environment Variables**: Store your API key in an environment variable on your system. This keeps the key out of your codebase and allows you to change it without altering your code.\n",
    "\n",
    "\n",
    "    - **Create a .env File**: If you prefer, you can create a `.env` file (just call it `.env` nothing else before the `.`) in your project directory to store the API key. Inside this file, you would have something like:\n",
    "   \n",
    "       ```\n",
    "       API_KEY=your-api-key-here\n",
    "       ```\n",
    "\n",
    "        - **Add .env to .gitignore**: If you're using a version control system like Git, make sure to add the `.env` file to your `.gitignore` file. This prevents the `.env` file (and therefore your API key) from being uploaded to any public repositories. We'll talk about Git and .gitignore in more detail soon.\n",
    "\n",
    "2. **Load the Key in Your Code**: \n",
    "\n",
    "    You can use libraries like `python-dotenv` to load the key into your code. \n",
    "    You would need to install `python-dotenv` first.\n",
    "    ```python \n",
    "    !pip install python-dotenv\n",
    "\n",
    "    from dotenv import load_dotenv\n",
    "    import os\n",
    "\n",
    "    load_dotenv()\n",
    "    api_key = os.getenv(\"API_KEY\")\n",
    "    ```\n",
    "\n",
    "       \n",
    "    Now, `api_key` contains the value of your API key, and you can use it to authenticate your requests to the API.\n",
    "       \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98b7f771-e230-4223-a5a7-4353ee337934",
   "metadata": {},
   "source": [
    "Lets do the second approach. Lets save it in an env file and save it in a variable."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3766f8a4-3e0a-4b18-8b16-d5e0d64c4dbd",
   "metadata": {},
   "source": [
    "`Make sure your file is named .env and not .env.txt! Is one of the most common errors. If you need, look at the document properties to make sure it doesn't have .txt at the end even if its not showing the .txt when looking at your folder.`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "12fa7d1f-37c7-40a7-b60f-e4e264a43574",
   "metadata": {},
   "outputs": [],
   "source": [
    "#!pip install python-dotenv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "65104c73-28cb-4b84-949b-013d2f0fd1b0",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from dotenv import load_dotenv, find_dotenv\n",
    "\n",
    "load_dotenv()\n",
    "\n",
    "my_key = os.getenv(\"API_KEY\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2b8daee-8bbe-4dba-b302-4424bff542ee",
   "metadata": {},
   "source": [
    "If you get an error, make sure you saved the .env file in the same directory as your jupyter notebook. You can use `os.getcwd()` to do so."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2edcbffe-e4b5-464b-8e5b-9efd3a41ed59",
   "metadata": {},
   "outputs": [],
   "source": [
    "os.getcwd()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f1e87da8-be8c-4f58-94aa-354b879a7ed6",
   "metadata": {},
   "source": [
    "##### Using the API Key in the Header Parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c2c03dbc-e8b1-4649-9c71-18e61cc61123",
   "metadata": {},
   "outputs": [],
   "source": [
    "url_top_headlines = \"http://newsapi.org/v2/top-headlines\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "262b26fd-e3a8-4bc3-a7cb-38901f474817",
   "metadata": {},
   "outputs": [],
   "source": [
    "parameters = {\"country\": \"us\", \"category\": \"science\"}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d3f8f790-1a58-4655-99bb-729fccd803eb",
   "metadata": {},
   "outputs": [],
   "source": [
    "headers = {\"X-Api-Key\": my_key}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "85034bf8-bcc1-4e4e-bcb1-a442b2d65711",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = requests.get(\n",
    "    url=url_top_headlines, \n",
    "    params=parameters, \n",
    "    headers=headers\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ad8de9aa-736e-480a-8a4d-1de84dbade07",
   "metadata": {},
   "outputs": [],
   "source": [
    "response.status_code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "56058048-233a-41f5-a1da-edb8092730de",
   "metadata": {},
   "outputs": [],
   "source": [
    "data = response.json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eb07a572-50ca-41b0-b3b3-ec82d0cd3f5f",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6e5c398-c333-4831-b15c-230c63e7ac9c",
   "metadata": {},
   "source": [
    "## APIs examples"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99d3ab93-1f0f-4c3d-9a4d-ebdf7337641b",
   "metadata": {},
   "source": [
    "### NewsAPI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f9720542-bc2e-4bd5-af7d-395f6aa420a0",
   "metadata": {},
   "outputs": [],
   "source": [
    "type(data) # response.json() returns a dictionary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "541f1c98-0837-4e79-b3bc-8266486acf2f",
   "metadata": {},
   "outputs": [],
   "source": [
    "data.keys() # we can get the keys with method keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b536335a-008b-41d9-af92-6ff4466da5e9",
   "metadata": {},
   "outputs": [],
   "source": [
    "data.get(\"status\") # we can get a key's value by using get method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eee49271-cd78-42f5-972d-dfe9728ea730",
   "metadata": {},
   "outputs": [],
   "source": [
    "data[\"totalResults\"] # or by using dictionary notation to access values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f26b202a-2d95-4e36-87e4-9a52f32a4be9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# this is the same as above\n",
    "data.get(\"totalResults\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61e9315e-c94e-41f8-8794-f255eb5220cc",
   "metadata": {},
   "source": [
    "`dict[key]` raises error if the key does not exits  \n",
    "`dict.get(key)` does not raise an error if the key does not exist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cc234036-e6d5-450e-9183-76e960220a7f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# lets look at the articles\n",
    "articles = data.get(\"articles\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0fac4939-7f00-4fe8-b758-4dc13e2c6b52",
   "metadata": {},
   "outputs": [],
   "source": [
    "type(articles) # lets see what type it is to know how to access it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5988fdf2-5d2a-4236-927f-e8abd9c0fe93",
   "metadata": {},
   "outputs": [],
   "source": [
    "len(articles)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2353c5e3-c200-40b5-9529-a4f531dc4d77",
   "metadata": {},
   "outputs": [],
   "source": [
    "articles[0] # since articles is a list, we can access its elements by the index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a98e38bb-4364-4d3a-817d-6993049f9408",
   "metadata": {},
   "outputs": [],
   "source": [
    "type(articles[0]) # each element inside the list is a dictionary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "18eb15ba-18e3-49c5-997d-050815814e21",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "[art[\"title\"] for art in articles] # lets get all the titles using list comprehension"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e8422609-6ccf-4ddc-b044-b5a951f49ab6",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.DataFrame(articles) # lets create a dataframe with the articles"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9069dc15-b0e6-4db9-acba-f1b5c0c37a5b",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2dc2bab1-1cdc-4a3f-9956-b5d5eb832103",
   "metadata": {},
   "source": [
    "#### 💡 Check for understanding"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e21fbb4-6e2d-43ef-a61e-38da7eedc5fe",
   "metadata": {},
   "source": [
    "1. What data type is the column *source*?\n",
    "2. Since the column *source* is not very useful as it is, create a column called *name* that contains only the name inside the column *source*.\n",
    "3. How many times each unique name appears in the \"name\" column?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f906ed21-be98-436b-816b-c73f5dd5c066",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your answer goes here"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48ca730a-10fe-4a69-ad11-01bf8759a938",
   "metadata": {},
   "source": [
    "### Pokemon API"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be8f6d65-359d-408b-aeb4-64379614377f",
   "metadata": {},
   "source": [
    "[PokeAPI](https://pokeapi.co/) is a full RESTful API linked to an extensive database detailing everything about the Pokémon main game series.\n",
    "\n",
    "In the documentation we see that we can get lot's of data. Let's look at the endpoint *pokemon*: ```https://pokeapi.co/api/v2/pokemon/{id or name}/``` "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b59992ec-a9de-4c14-88ef-177df49393a4",
   "metadata": {},
   "outputs": [],
   "source": [
    "res=requests.get('https://pokeapi.co/api/v2/pokemon/25')\n",
    "res.status_code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b180faa9-b8b2-4ec5-804e-d46941f6132c",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "data=res.json()\n",
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "faa82a7d-35f8-441c-9c65-c4cf34c8ba99",
   "metadata": {},
   "outputs": [],
   "source": [
    "type(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a4745456-3fa4-4511-a7be-c6eea20db9df",
   "metadata": {},
   "outputs": [],
   "source": [
    "data.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9fa671b3-9a78-49e8-bf57-0781a1fa6b70",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Lets look at the following attributes\n",
    "print(data[\"name\"])\n",
    "print(data[\"weight\"])\n",
    "print(data[\"sprites\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b4ceecb0-9e17-4cfe-a09e-67ae3026652d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Lets get the pokemon image from sprites - front_default.\n",
    "# Remember we can read the documentation to understand how to get all the information\n",
    "data['sprites']['front_default']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5bdb9958-a049-42b2-8c5c-475c7b90601d",
   "metadata": {},
   "source": [
    "We will define a function, `get_pokemon`, that takes an ID number as input. This function fetches details about a Pokémon from the PokeAPI using the given ID, extracts its name, weight, and front-facing image, and then returns this data in a dictionary format. If the ID is invalid or there's an issue with the request, a ValueError is raised."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "04981f9a-157d-49ae-9e5d-0b382d5ab2fd",
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_pokemon(id_number):\n",
    "    res=requests.get('https://pokeapi.co/api/v2/pokemon/{}'.format(id_number))\n",
    "    \n",
    "    if(res.status_code==200):\n",
    "        data = res.json()\n",
    "        return {\"id\": id_number,'name':data['name'], \"weight\":data[\"weight\"],'image':data['sprites']['front_default']}\n",
    "    else:\n",
    "        raise ValueError(\"Cannot get pokemon, id does not exist\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "96dbff41-1e63-40ee-b409-aed7cce7e7a7",
   "metadata": {},
   "outputs": [],
   "source": [
    "get_pokemon(25) # Lets use the function for pokemon id 25"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ebf5de4c-1150-4cab-812a-7b6d9f1e807c",
   "metadata": {},
   "outputs": [],
   "source": [
    "[get_pokemon(i+1) for i in range(5)] # Lets use list comprehension to gather information for the first 5 pokemons"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d419697-cb8a-4737-8b0b-3a742615f16a",
   "metadata": {},
   "source": [
    "We will now define a function, `print_pokemon`, that takes in a Pokémon dictionary and displays its ID, name, weight, and its image. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "da875adb-e2d1-4e59-8b2e-ce3e00da5f59",
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import Image\n",
    "\n",
    "def print_pokemon(poke):\n",
    "    print(str(poke[\"id\"])+\" \" +poke[\"name\"]+ \" \"+str(poke[\"weight\"]) + \" kg\")\n",
    "    display(Image(url=poke['image']))\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d284e0ef-4d49-42a3-bc50-c1a859a7e41a",
   "metadata": {},
   "source": [
    "After defining this function, we will retrieve and display details for the first 10 Pokémon by iterating through their IDs and using both the `get_pokemon` and `print_pokemon` functions. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9e8a778b-bf09-4351-b9d6-6adc759566e5",
   "metadata": {},
   "outputs": [],
   "source": [
    "[print_pokemon(get_pokemon(i+1)) for i in range(10)]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ee586a7-a55f-4c14-b3cf-55745308fe84",
   "metadata": {},
   "source": [
    "#### json_normalize()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ca157cb-530d-49de-9286-f072b3e295e4",
   "metadata": {},
   "source": [
    "Want the data as a DataFrame instead of a dictionary?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9672696a-3e67-4af5-af41-10270e1209b1",
   "metadata": {},
   "source": [
    "The following line creates a DataFrame from the items of the dictionary data. Each key-value pair will be treated as a row, with two columns: the first for the key and the second for the value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5455a1bd-a158-4111-9841-750add5b2708",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.DataFrame(data.items())\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "175094a5-2fcd-4956-8fb4-e0d59f701ffe",
   "metadata": {},
   "source": [
    "That format is still very hard to work with. Lets use `json_normalize` instead. `json_normalize` is used to normalize semi-structured JSON data into a flat table (DataFrame). It's particularly useful when dealing with nested dictionaries or lists inside JSON."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "def72654-66aa-41cd-900b-d55fae54894d",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.json_normalize(data)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a879ab4b-1a2e-42f7-8a4b-4bf2c4e8687c",
   "metadata": {},
   "source": [
    "A lot better, even though we still have nested dictionaries or lists inside the dataframe.\n",
    "\n",
    "Let's see again the difference between creating a dataframe directly or using `json_normalize` by using `data[\"abilities\"]`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "96389cbb-ee4f-4495-ae45-91de38ccc1fa",
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.DataFrame(data[\"abilities\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "04997d49-0438-40e5-b0cf-6244405aa55e",
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.json_normalize(data[\"abilities\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4680fda5-270d-45e9-ab0f-abebcfe09088",
   "metadata": {},
   "source": [
    "### Coincap API"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d43ce6fe-baf5-42da-9e40-d31f2c8e3b27",
   "metadata": {},
   "source": [
    "CoinCap is a useful tool for real-time pricing and market activity for over 1,000 cryptocurrencies. Check the documentation at https://docs.coincap.io/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bdfda195-9de8-4e2c-8b01-63161db18060",
   "metadata": {},
   "outputs": [],
   "source": [
    "url = \"http://api.coincap.io/v2/assets\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8cdd990d-cf9b-4bdc-bbd9-cb038685efd5",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = requests.get(url)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1ea53c95-0868-4440-80ba-13f492e4da6f",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8873ba49-0849-40f9-903d-9e49451de9ee",
   "metadata": {},
   "outputs": [],
   "source": [
    "res = response.json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "17605693-433c-4581-813a-136209c382e3",
   "metadata": {},
   "outputs": [],
   "source": [
    "res # it returns a dictionary with all the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b56de457-e978-468c-b752-97000ce71baa",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Again, let's try to have it as a dataframe\n",
    "data = pd.DataFrame(res[\"data\"])\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2e264ce5-5f90-49f2-8888-78b64808a3ba",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Lets see if the result would differ in this case using json_normalize\n",
    "pd.json_normalize(res[\"data\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52dfaadb-dd5f-4b7d-a832-c101e4d7c80a",
   "metadata": {},
   "source": [
    "### Api Jokes\n",
    "\n",
    "JokeAPI is a REST API that serves uniformly and well formatted jokes.\n",
    "It can be used without any API token, membership, registration or payment.\n",
    "It supports a variety of filters that can be applied to get just the right jokes you need.\n",
    "Check the documentation here: https://jokeapi.dev/ "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e897b2d9-088c-47fd-9d58-fefec4332150",
   "metadata": {},
   "outputs": [],
   "source": [
    "url_random_joke = \"https://v2.jokeapi.dev/joke/dark?amount=3\"\n",
    "request = requests.get(url_random_joke).json()\n",
    "request"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd07c567-1d95-428a-8ac9-9f9909457c49",
   "metadata": {
    "lang": "en"
   },
   "source": [
    "### 💡 Check for understanding\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08d0de37-f0e9-4249-a1f2-3e9c091f83da",
   "metadata": {},
   "source": [
    "Choose one API from the [Public APIs list](https://github.com/public-apis/public-apis). Attempt to call your selected API, either with or without parameters, and retrieve some valuable information. Document your findings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c8f7e8a5-31b7-4444-a80f-8a94dd85e2eb",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your answer goes here"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10d38405-f1bd-464f-a319-4a5f29a4cdc2",
   "metadata": {},
   "source": [
    "## API Wrappers\n",
    "\n",
    "A Python wrapper is a Python library or module that provides a more convenient or more \"Pythonic\" interface to another software component, such as a library in another language, a system tool, or a web API. It \"wraps\" the functionality of that component in a way that abstracts away its complexities and makes it easier to use in a Python context. 🙌🏻\n",
    "\n",
    "One example is the `tweepy` library, which makes obtaining data from Twitter's API relatively straightforward."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "796f214d-01cc-46d1-a020-d4bb8d256799",
   "metadata": {},
   "source": [
    "## Summary\n",
    "- Import the `requests` library.\n",
    "- Store the necessary credentials, such as API key or OAuth tokens, if the API requires them.\n",
    "- Execute a `request.get` to the desired API endpoint (the API's documentation will provide details on available endpoints).\n",
    "- The API returns a JSON response.\n",
    "- This JSON can be converted into a dataframe, or you can further explore its elements (typically a list of dictionaries)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "092ff771-4d9a-4d84-98c6-db652fe4ff72",
   "metadata": {},
   "source": [
    "## Glossary"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32ca2f95-7c09-4481-8897-0b0533dba4c9",
   "metadata": {},
   "source": [
    "* DNS: domain name system.\n",
    "* HTTP: is the protocol used to transfer data over the web.\n",
    "* API: application programming interface.\n",
    "* REST: series of rules, architectural style."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12ba250f-82d7-450b-9038-0f32013dbaf5",
   "metadata": {},
   "source": [
    "* `requests`: Python library for interacting with APIs.\n",
    "* URL: server name you want to ask information for.\n",
    "* endpoint: server service you want to ask information for.\n",
    "* parameters: extra parameters to your query.\n",
    "* headers: metadata, invisible information."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df2580ce-5fa4-4716-a1f1-3d6c8edd9673",
   "metadata": {},
   "source": [
    "## Further materials"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cdae6b8f-bed4-47f8-b9ae-3d8e24e8aba6",
   "metadata": {},
   "source": [
    "[5 Simple-To-Use APIs for Beginners](https://dev.to/alanconstantino/5-simple-to-use-apis-for-beginners-2e0n)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e747420-7bb6-4ea7-a5d9-e05c90fafd32",
   "metadata": {},
   "source": [
    "[RapidAPI](https://rapidapi.com/category/Sports): access thousands of APIs."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": true,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}