examples/notebooks/saving_and_loading_models.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# Saving and loading models\n",
    "\n",
    "In this notebook, we'll be using the red wine quality data set. The purpose of this notebook is to show how to save a trained model and load it again in a fresh session.",
    "\n",
    "[![Video Explanation Thumbnail](http://img.youtube.com/vi/ktVYFIH1K1g/0.jpg)](http://www.youtube.com/watch?v=ktVYFIH1K1g)"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Import Packages\n",
    "\n",
    "Let us import dffml and other packages that we might need."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "source": [
    "from dffml import *\n",
    "import asyncio\n",
    "import nest_asyncio"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "To use asyncio in a notebook, we need to use `nest_asycio.apply()`"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "source": [
    "nest_asyncio.apply()"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Build our Dataset\n",
    "\n",
    "Dffml has a very convinient function `cached_download()` that can be used to download datasets and make sure you don't download them if you have already.\n",
    "\n",
    "\n",
    "The cached_download() has the following parameters:\n",
    "\n",
    "\n",
    "\n",
    "1.        `url` (str) – The URL to download\n",
    "\n",
    "2.        `target_path` (str, pathlib.Path) – Path on disk to store download\n",
    "\n",
    "3.        `expected_hash` (str) – SHA384 hash of the contents\n",
    "\n",
    "4.        `protocol_allowlist` (list, optional) – List of strings, one of which the URL must start with. \n",
    "          We won't be using this in our case.\n"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "source": [
    "data_path = await cached_download(\n",
    "    \"https://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-red.csv\",\n",
    "    \"wine_quality.csv\",\n",
    "    \"789e98688f9ff18d4bae35afb71b006116ec9c529c1b21563fdaf5e785aea8b3937a55a4919c91ca2b0acb671300072c\",\n",
    ")"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "In Dffml, we try to use asynchronicity where we can, to get that extra bit of performance. Let's use the async version of load() to load the dataset that we just downloaded into a source. We can easily achieve this by declaring a CSVSource with the data_path and the delimiter since the data we downloaded seems to have a non-comma delimiter.\n",
    "\n",
    "After that, we can just create an array of records by loading each one through the load() function.\n",
    "\n",
    "Feel free to also try out the no async version of load().\n"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "source": [
    "async def load_dataset(data_path):\n",
    "    data_source = CSVSource(filename=data_path, delimiter=\";\")\n",
    "    data = [record async for record in load(data_source)]\n",
    "    return data\n",
    "\n",
    "\n",
    "data = asyncio.run(load_dataset(data_path))"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Dffml lets you visualize a record in quite a neat fashion. Lets have a look."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "source": [
    "print(data[0], \"\\n\")\n",
    "print(len(data))"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "\n",
      "\tKey:\t0\n",
      "                           Record Features\n",
      "+----------------------------------------------------------------------+\n",
      "|  fixed acidity  |                        7.4                         |\n",
      "+----------------------------------------------------------------------+\n",
      "| volatile acidity|                        0.7                         |\n",
      "+----------------------------------------------------------------------+\n",
      "|   citric acid   |                         0                          |\n",
      "+----------------------------------------------------------------------+\n",
      "|  residual sugar |                        1.9                         |\n",
      "+----------------------------------------------------------------------+\n",
      "|    chlorides    |                       0.076                        |\n",
      "+----------------------------------------------------------------------+\n",
      "|free sulfur dioxi|                         11                         |\n",
      "+----------------------------------------------------------------------+\n",
      "|total sulfur diox|                         34                         |\n",
      "+----------------------------------------------------------------------+\n",
      "|     density     |                       0.9978                       |\n",
      "+----------------------------------------------------------------------+\n",
      "|        pH       |                        3.51                        |\n",
      "+----------------------------------------------------------------------+\n",
      "|    sulphates    |                        0.56                        |\n",
      "+----------------------------------------------------------------------+\n",
      "|     alcohol     |                        9.4                         |\n",
      "+----------------------------------------------------------------------+\n",
      "|     quality     |                         5                          |\n",
      "+----------------------------------------------------------------------+\n",
      "                                           Prediction:    Undetermined \n",
      "\n",
      "1599\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Lets split our dataset into train and test splits."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "source": [
    "train_data = data[320:]\n",
    "test_data = data[:320]\n",
    "print(len(data), len(train_data), len(test_data))"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "1599 1279 320\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Instantiate our Models with parameters\n",
    "Dffml makes it quite easy to load multiple models dynamically using the `Model.load()` function. After that, you just have to parameterize the loaded models and they are ready to train interchangably!\n",
    "\n",
    "For this example, we'll be demonstrating 2 models but you can feel free to try more than 2 models in a similar fashion."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "source": [
    "ScikitETCModel = Model.load(\"scikitetc\")\n",
    "\n",
    "features = Features(\n",
    "    Feature(\"fixed acidity\", int, 1),\n",
    "    Feature(\"volatile acidity\", int, 1),\n",
    "    Feature(\"citric acid\", int, 1),\n",
    "    Feature(\"residual sugar\", int, 1),\n",
    "    Feature(\"chlorides\", int, 1),\n",
    "    Feature(\"free sulfur dioxide\", int, 1),\n",
    "    Feature(\"total sulfur dioxide\", int, 1),\n",
    "    Feature(\"density\", int, 1),\n",
    "    Feature(\"pH\", int, 1),\n",
    "    Feature(\"sulphates\", int, 1),\n",
    "    Feature(\"alcohol\", int, 1),\n",
    ")\n",
    "\n",
    "predict_feature = Feature(\"quality\", int, 1)\n",
    "\n",
    "model = ScikitETCModel(\n",
    "    features=features,\n",
    "    predict=predict_feature,\n",
    "    location=\"scikitetc\",\n",
    "    n_estimators=150,\n",
    ")"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Train our Models\n",
    "Finally, our models are ready to be trained using the `high-level` API. Let's make sure to pass each record as a parameter by simply using the unpacking operator(*)."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "source": [
    "await train(model, *train_data)"
   ],
   "outputs": [],
   "metadata": {
    "scrolled": true
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "**Model Saved!**\n",
    "\n",
    "When we train a model in DFFML, it is automatically saved on the location provided while instantiating the model."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Test our Models\n",
    "To test our model, we'll use the `score()` function in the `high-level` API.\n",
    "\n",
    "We ask for the accuracy to be assessed using the Mean Squared Error method by passing \"mse\" to `AccuracyScorer.load()`."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "source": [
    "MeanSquaredErrorAccuracy = AccuracyScorer.load(\"mse\")\n",
    "\n",
    "scorer = MeanSquaredErrorAccuracy()\n",
    "\n",
    "Accuracy = await score(model, scorer, predict_feature, *test_data)\n",
    "print(\"Accuracy:\", Accuracy)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Accuracy: 0.46875\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Restart kernel and load the model\n",
    "Restart kernel and execute cells below to demonstrate loading a saved model."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "from IPython.core.display import HTML\n",
    "\n",
    "HTML(\n",
    "    \"\"\"\n",
    "    <script>\n",
    "        Jupyter.notebook.kernel.restart();\n",
    "    </script>\n",
    "    \"\"\"\n",
    ")"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/html": [
       "\n",
       "    <script>\n",
       "        Jupyter.notebook.kernel.restart();\n",
       "    </script>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "execution_count": 10
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "source": [
    "from IPython.core.display import HTML\n",
    "\n",
    "HTML(\"<script>Jupyter.notebook.execute_cells_below()</script>\")"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/html": [
       "<script>Jupyter.notebook.execute_cells_below()</script>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "execution_count": 1
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Import Packages"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "source": [
    "from dffml import *\n",
    "import asyncio\n",
    "import nest_asyncio"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "To use asyncio in a notebook, we need to use `nest_asycio.apply()`"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "source": [
    "nest_asyncio.apply()"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Build our Dataset\n"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "source": [
    "data_path = await cached_download(\n",
    "    \"https://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-red.csv\",\n",
    "    \"wine_quality.csv\",\n",
    "    \"789e98688f9ff18d4bae35afb71b006116ec9c529c1b21563fdaf5e785aea8b3937a55a4919c91ca2b0acb671300072c\",\n",
    ")"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "**Load and split data**\n"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "source": [
    "async def load_dataset(data_path):\n",
    "    data_source = CSVSource(filename=data_path, delimiter=\";\")\n",
    "    data = [record async for record in load(data_source)]\n",
    "    return data\n",
    "\n",
    "\n",
    "data = asyncio.run(load_dataset(data_path))"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "source": [
    "train_data = data[320:]\n",
    "test_data = data[:320]\n",
    "print(len(data), len(train_data), len(test_data))"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "1599 1279 320\n"
     ]
    }
   ],
   "metadata": {
    "scrolled": true
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Load our saved model\n",
    "\n",
    "In DFFML, we can load our previously trained model by instantiating it in the same location."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "source": [
    "ScikitETCModel = Model.load(\"scikitetc\")\n",
    "\n",
    "features = Features(\n",
    "    Feature(\"fixed acidity\", int, 1),\n",
    "    Feature(\"volatile acidity\", int, 1),\n",
    "    Feature(\"citric acid\", int, 1),\n",
    "    Feature(\"residual sugar\", int, 1),\n",
    "    Feature(\"chlorides\", int, 1),\n",
    "    Feature(\"free sulfur dioxide\", int, 1),\n",
    "    Feature(\"total sulfur dioxide\", int, 1),\n",
    "    Feature(\"density\", int, 1),\n",
    "    Feature(\"pH\", int, 1),\n",
    "    Feature(\"sulphates\", int, 1),\n",
    "    Feature(\"alcohol\", int, 1),\n",
    ")\n",
    "\n",
    "predict_feature = Feature(\"quality\", int, 1)\n",
    "\n",
    "model = ScikitETCModel(\n",
    "    features=features,\n",
    "    predict=predict_feature,\n",
    "    location=\"scikitetc\",\n",
    "    n_estimators=150,\n",
    ")"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Lets check if the model is already trained."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "source": [
    "MeanSquaredErrorAccuracy = AccuracyScorer.load(\"mse\")\n",
    "\n",
    "scorer = MeanSquaredErrorAccuracy()\n",
    "\n",
    "Accuracy = await score(model, scorer, predict_feature, *test_data)\n",
    "print(\"Accuracy:\", Accuracy)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Accuracy: 0.46875\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "We can see that DFFML loaded the previously trained model which has the exact same accuracy."
   ],
   "metadata": {}
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}