src a. Retrieval.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "284a6008",
   "metadata": {},
   "source": [
    "# Tensorflow Recommenders - An Implementation on Retail Data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef0c96e5",
   "metadata": {},
   "source": [
    "Objective: To demonstrate TensorFlow 2.0 TFRS recommenders library to build a recommendation system on a customer retail data.\n",
    "\n",
    "Data source: https://www.kaggle.com/olistbr/brazilian-ecommerce/home/\n",
    "\n",
    "Benefit: Flexible model, ability to add different features and specify and adjust model complexity easily.\n",
    "\n",
    "Two types of recommendation model-- Retrieval and Ranking.\n",
    "\n",
    "Retrieval: The retrieval stage is responsible for selecting an initial set of hundreds of candidates from all possible candidates. The main objective of this model is to efficiently weed out all candidates that the user is not interested in. Because the retrieval model may be dealing with millions of candidates, it has to be computationally efficient.\n",
    "Retrieval can be computationally more efficient because it only returns smaller set of items a user would strongly interested.\n",
    "\n",
    "Ranking: The ranking stage takes the outputs of the retrieval model and fine-tunes them to select the best possible handful of recommendations. Its task is to narrow down the set of items the user may be interested in to a shortlist of likely candidates."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3813f873",
   "metadata": {},
   "source": [
    "# Tensorflow recommenders"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2de571a7",
   "metadata": {},
   "source": [
    "Tensorflow recommders work akin to a hybrid recommendation system, where it combines the interactions in between users and items, alongside any features available of both, in a matrix dot product. The content of both sides of equation are embed in layers of what they called a tower model, where they stack these information, learn about them separately, and combine them for learning and loss propagation."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e476c5e",
   "metadata": {},
   "source": [
    "# Pros and cons"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1517ae95",
   "metadata": {},
   "source": [
    "Pros:\n",
    "- flexibility to add features\n",
    "- flexiblity to tune models\n",
    "- with decent knowledge of Tensorflow, users can expand the layers of deep learning to create a complex engine\n",
    "\n",
    "Cons:\n",
    "- Further developement needed to scale up towards sparse data\n",
    " - strict data requirement, not-user friendly\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a42234be",
   "metadata": {},
   "source": [
    "## Basic Retrieval Model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b6ce1d6",
   "metadata": {},
   "source": [
    "Load the data. It has been cleaned before, so here is the cleaned and merged table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "7b2d9c3d",
   "metadata": {},
   "outputs": [],
   "source": [
    "### Import necessary libraries\n",
    "\n",
    "from typing import Dict, Text\n",
    "\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "\n",
    "import tensorflow_recommenders as tfrs\n",
    "\n",
    "import os\n",
    "import pprint\n",
    "import tempfile\n",
    "\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "3a8f3e82",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>order_id</th>\n",
       "      <th>order_purchase_timestamp</th>\n",
       "      <th>user_id</th>\n",
       "      <th>customer_city</th>\n",
       "      <th>customer_state</th>\n",
       "      <th>product_category</th>\n",
       "      <th>quantity</th>\n",
       "      <th>price</th>\n",
       "      <th>review_score</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>product_code</th>\n",
       "      <th>product_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>e481f51cbdc54678b7cc49136f2d6af7</td>\n",
       "      <td>2017-10-02 10:56:33</td>\n",
       "      <td>9ef432eb6251297304e76186b10a928d</td>\n",
       "      <td>sao paulo</td>\n",
       "      <td>SP</td>\n",
       "      <td>housewares</td>\n",
       "      <td>1.0</td>\n",
       "      <td>29.99</td>\n",
       "      <td>4</td>\n",
       "      <td>1.506942e+09</td>\n",
       "      <td>87285b34884572647811a353c7ac498a</td>\n",
       "      <td>housewares SKU 0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>53cdb2fc8bc7dce0b6741e2150273451</td>\n",
       "      <td>2018-07-24 20:41:37</td>\n",
       "      <td>b0830fb4747a6c6d20dea0b8c802d7ef</td>\n",
       "      <td>barreiras</td>\n",
       "      <td>BA</td>\n",
       "      <td>perfumery</td>\n",
       "      <td>1.0</td>\n",
       "      <td>118.70</td>\n",
       "      <td>4</td>\n",
       "      <td>1.532465e+09</td>\n",
       "      <td>595fac2a385ac33a80bd5114aec74eb8</td>\n",
       "      <td>perfumery SKU 0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>47770eb9100c2d0c44946d9cf07ec65d</td>\n",
       "      <td>2018-08-08 08:38:49</td>\n",
       "      <td>41ce2a54c0b03bf3443c3d931a367089</td>\n",
       "      <td>vianopolis</td>\n",
       "      <td>GO</td>\n",
       "      <td>auto</td>\n",
       "      <td>1.0</td>\n",
       "      <td>159.90</td>\n",
       "      <td>5</td>\n",
       "      <td>1.533718e+09</td>\n",
       "      <td>aa4383b373c6aca5d8797843e5594415</td>\n",
       "      <td>auto SKU 0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0                          order_id order_purchase_timestamp  \\\n",
       "0           0  e481f51cbdc54678b7cc49136f2d6af7      2017-10-02 10:56:33   \n",
       "1           1  53cdb2fc8bc7dce0b6741e2150273451      2018-07-24 20:41:37   \n",
       "2           2  47770eb9100c2d0c44946d9cf07ec65d      2018-08-08 08:38:49   \n",
       "\n",
       "                            user_id customer_city customer_state  \\\n",
       "0  9ef432eb6251297304e76186b10a928d     sao paulo             SP   \n",
       "1  b0830fb4747a6c6d20dea0b8c802d7ef     barreiras             BA   \n",
       "2  41ce2a54c0b03bf3443c3d931a367089    vianopolis             GO   \n",
       "\n",
       "  product_category  quantity   price  review_score     timestamp  \\\n",
       "0       housewares       1.0   29.99             4  1.506942e+09   \n",
       "1        perfumery       1.0  118.70             4  1.532465e+09   \n",
       "2             auto       1.0  159.90             5  1.533718e+09   \n",
       "\n",
       "                       product_code        product_id  \n",
       "0  87285b34884572647811a353c7ac498a  housewares SKU 0  \n",
       "1  595fac2a385ac33a80bd5114aec74eb8   perfumery SKU 0  \n",
       "2  aa4383b373c6aca5d8797843e5594415        auto SKU 0  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "masterdf = pd.read_csv('../data_cleaned/brazildata_mod.csv')\n",
    "masterdf.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "8bc6701a",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>order_id</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>user_id</th>\n",
       "      <th>customer_city</th>\n",
       "      <th>product_category</th>\n",
       "      <th>product_id</th>\n",
       "      <th>quantity</th>\n",
       "      <th>price</th>\n",
       "      <th>review_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>e481f51cbdc54678b7cc49136f2d6af7</td>\n",
       "      <td>1.506942e+09</td>\n",
       "      <td>9ef432eb6251297304e76186b10a928d</td>\n",
       "      <td>sao paulo</td>\n",
       "      <td>housewares</td>\n",
       "      <td>housewares SKU 0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>29.99</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>53cdb2fc8bc7dce0b6741e2150273451</td>\n",
       "      <td>1.532465e+09</td>\n",
       "      <td>b0830fb4747a6c6d20dea0b8c802d7ef</td>\n",
       "      <td>barreiras</td>\n",
       "      <td>perfumery</td>\n",
       "      <td>perfumery SKU 0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>118.70</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>47770eb9100c2d0c44946d9cf07ec65d</td>\n",
       "      <td>1.533718e+09</td>\n",
       "      <td>41ce2a54c0b03bf3443c3d931a367089</td>\n",
       "      <td>vianopolis</td>\n",
       "      <td>auto</td>\n",
       "      <td>auto SKU 0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>159.90</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>949d5b44dbf5de918fe9c16f97b45f8a</td>\n",
       "      <td>1.511033e+09</td>\n",
       "      <td>f88197465ea7920adcdbec7375364d82</td>\n",
       "      <td>sao goncalo do amarante</td>\n",
       "      <td>pet_shop</td>\n",
       "      <td>pet_shop SKU 0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>45.00</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ad21c59c0840e6cb83a9ceb5573f8159</td>\n",
       "      <td>1.518557e+09</td>\n",
       "      <td>8ab97904e6daea8866dbdbc4fb7aad2c</td>\n",
       "      <td>santo andre</td>\n",
       "      <td>stationery</td>\n",
       "      <td>stationery SKU 0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>19.90</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>82592</th>\n",
       "      <td>aa04ef5214580b06b10e2a378300db44</td>\n",
       "      <td>1.485477e+09</td>\n",
       "      <td>f01a6bfcc730456317e4081fe0c9940e</td>\n",
       "      <td>divinopolis</td>\n",
       "      <td>health_beauty</td>\n",
       "      <td>health_beauty SKU 100</td>\n",
       "      <td>1.0</td>\n",
       "      <td>370.00</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>82593</th>\n",
       "      <td>63943bddc261676b46f01ca7ac2f7bd8</td>\n",
       "      <td>1.517922e+09</td>\n",
       "      <td>1fca14ff2861355f6e5f14306ff977a7</td>\n",
       "      <td>praia grande</td>\n",
       "      <td>baby</td>\n",
       "      <td>baby SKU 12</td>\n",
       "      <td>1.0</td>\n",
       "      <td>174.90</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>82594</th>\n",
       "      <td>83c1379a015df1e13d02aae0204711ab</td>\n",
       "      <td>1.503845e+09</td>\n",
       "      <td>1aa71eb042121263aafbe80c1b562c9c</td>\n",
       "      <td>nova vicosa</td>\n",
       "      <td>home_appliances_2</td>\n",
       "      <td>home_appliances_2 SKU 58</td>\n",
       "      <td>1.0</td>\n",
       "      <td>205.99</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>82595</th>\n",
       "      <td>11c177c8e97725db2631073c19f07b62</td>\n",
       "      <td>1.515447e+09</td>\n",
       "      <td>b331b74b18dc79bcdf6532d51e1637c1</td>\n",
       "      <td>japuiba</td>\n",
       "      <td>computers_accessories</td>\n",
       "      <td>computers_accessories SKU 2</td>\n",
       "      <td>1.0</td>\n",
       "      <td>179.99</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>82596</th>\n",
       "      <td>11c177c8e97725db2631073c19f07b62</td>\n",
       "      <td>1.515447e+09</td>\n",
       "      <td>b331b74b18dc79bcdf6532d51e1637c1</td>\n",
       "      <td>japuiba</td>\n",
       "      <td>computers_accessories</td>\n",
       "      <td>computers_accessories SKU 2</td>\n",
       "      <td>2.0</td>\n",
       "      <td>179.99</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>82597 rows × 9 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                               order_id     timestamp  \\\n",
       "0      e481f51cbdc54678b7cc49136f2d6af7  1.506942e+09   \n",
       "1      53cdb2fc8bc7dce0b6741e2150273451  1.532465e+09   \n",
       "2      47770eb9100c2d0c44946d9cf07ec65d  1.533718e+09   \n",
       "3      949d5b44dbf5de918fe9c16f97b45f8a  1.511033e+09   \n",
       "4      ad21c59c0840e6cb83a9ceb5573f8159  1.518557e+09   \n",
       "...                                 ...           ...   \n",
       "82592  aa04ef5214580b06b10e2a378300db44  1.485477e+09   \n",
       "82593  63943bddc261676b46f01ca7ac2f7bd8  1.517922e+09   \n",
       "82594  83c1379a015df1e13d02aae0204711ab  1.503845e+09   \n",
       "82595  11c177c8e97725db2631073c19f07b62  1.515447e+09   \n",
       "82596  11c177c8e97725db2631073c19f07b62  1.515447e+09   \n",
       "\n",
       "                                user_id            customer_city  \\\n",
       "0      9ef432eb6251297304e76186b10a928d                sao paulo   \n",
       "1      b0830fb4747a6c6d20dea0b8c802d7ef                barreiras   \n",
       "2      41ce2a54c0b03bf3443c3d931a367089               vianopolis   \n",
       "3      f88197465ea7920adcdbec7375364d82  sao goncalo do amarante   \n",
       "4      8ab97904e6daea8866dbdbc4fb7aad2c              santo andre   \n",
       "...                                 ...                      ...   \n",
       "82592  f01a6bfcc730456317e4081fe0c9940e              divinopolis   \n",
       "82593  1fca14ff2861355f6e5f14306ff977a7             praia grande   \n",
       "82594  1aa71eb042121263aafbe80c1b562c9c              nova vicosa   \n",
       "82595  b331b74b18dc79bcdf6532d51e1637c1                  japuiba   \n",
       "82596  b331b74b18dc79bcdf6532d51e1637c1                  japuiba   \n",
       "\n",
       "            product_category                   product_id  quantity   price  \\\n",
       "0                 housewares             housewares SKU 0       1.0   29.99   \n",
       "1                  perfumery              perfumery SKU 0       1.0  118.70   \n",
       "2                       auto                   auto SKU 0       1.0  159.90   \n",
       "3                   pet_shop               pet_shop SKU 0       1.0   45.00   \n",
       "4                 stationery             stationery SKU 0       1.0   19.90   \n",
       "...                      ...                          ...       ...     ...   \n",
       "82592          health_beauty        health_beauty SKU 100       1.0  370.00   \n",
       "82593                   baby                  baby SKU 12       1.0  174.90   \n",
       "82594      home_appliances_2     home_appliances_2 SKU 58       1.0  205.99   \n",
       "82595  computers_accessories  computers_accessories SKU 2       1.0  179.99   \n",
       "82596  computers_accessories  computers_accessories SKU 2       2.0  179.99   \n",
       "\n",
       "       review_score  \n",
       "0                 4  \n",
       "1                 4  \n",
       "2                 5  \n",
       "3                 5  \n",
       "4                 5  \n",
       "...             ...  \n",
       "82592             5  \n",
       "82593             4  \n",
       "82594             5  \n",
       "82595             2  \n",
       "82596             2  \n",
       "\n",
       "[82597 rows x 9 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "masterdf[['order_id', 'timestamp', 'user_id',\n",
    "       'customer_city', 'product_category','product_id', 'quantity',\n",
    "       'price', 'review_score' ]]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ffd63c56",
   "metadata": {},
   "source": [
    "# Idenfity features that we need, and prepare the dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "862a6c52",
   "metadata": {},
   "source": [
    "The data contains many usable features, notably customer geographical area and product category. Users who are familar with content-based recommendation or hybrid recommendation system know that additional features of users and items help to improve the accuracy of the prediction.\n",
    "\n",
    "Let us start with user id, product id, and quantity. These three features are the bare minimum of what the engine needs. Think of quantity as equivalent to ratings in movie recommendation system, where the quantity tells us how much the consumer likes to buy the item, the higher the quantity, the higher his/her preference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "2493a9c7",
   "metadata": {},
   "outputs": [],
   "source": [
    "### standardize item data types, especially string, float, and integer\n",
    "\n",
    "masterdf[['user_id',      \n",
    "          'product_id',  \n",
    "         ]] = masterdf[['user_id','product_id']].astype(str)\n",
    "\n",
    "# we will play around with the data type of the quantity, \n",
    "# which you shall see later it affects the accuracy of the prediction.\n",
    "\n",
    "masterdf['quantity'] = masterdf['quantity'].astype(float)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44c5fe64",
   "metadata": {},
   "source": [
    "#### Data Preparation - Query and candidate table\n",
    "\n",
    "We only need to prepare two set of reference table-- an item table, which will be the ultimate reference table for items to be recommended, and an interaction/query table (or a ranking table, in the case of IMDB movie example) where it shows user's previous purchase history."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "07fd7a5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "### define interactions data and user data\n",
    "\n",
    "### interactions \n",
    "### here we create a reference table of the user , item, and quantity purchased\n",
    "interactions_dict = masterdf.groupby(['user_id', 'product_id'])[ 'quantity'].sum().reset_index()\n",
    "\n",
    "## we tansform the table inta a dictionary , which then we feed into tensor slices\n",
    "# this step is crucial as this will be the type of data fed into the embedding layers\n",
    "interactions_dict = {name: np.array(value) for name, value in interactions_dict.items()}\n",
    "interactions = tf.data.Dataset.from_tensor_slices(interactions_dict)\n",
    "\n",
    "## we do similar step for item, where this is the reference table for items to be recommended\n",
    "items_dict = masterdf[['product_id']].drop_duplicates()\n",
    "items_dict = {name: np.array(value) for name, value in items_dict.items()}\n",
    "items = tf.data.Dataset.from_tensor_slices(items_dict)\n",
    "\n",
    "## map the features in interactions and items to an identifier that we will use throught the embedding layers\n",
    "## do it for all the items in interaction and item table\n",
    "## you may often get itemtype error, so that is why here i am casting the quantity type as float to ensure consistency\n",
    "interactions = interactions.map(lambda x: {\n",
    "    'user_id' : x['user_id'], \n",
    "    'product_id' : x['product_id'], \n",
    "    'quantity' : float(x['quantity']),\n",
    "\n",
    "})\n",
    "\n",
    "items = items.map(lambda x: x['product_id'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "63f8c5ad",
   "metadata": {},
   "source": [
    "#### Basic Housekeeping\n",
    "\n",
    "Let's also figure out unique user ids and movie titles present in the data.\n",
    "\n",
    "We need a reference map of the raw values of categorical features to embedding vectors in the retrieval models. To do that, we need a vocabulary that maps a raw feature value to an integer in a contiguous range, which then will be mapped to the corresponding embeddings in the embedding tables inside the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "821d7a24",
   "metadata": {},
   "outputs": [],
   "source": [
    "unique_item_titles = np.unique(np.concatenate(list(items.batch(1000))))\n",
    "unique_user_ids = np.unique(np.concatenate(list(interactions.batch(1_000).map(lambda x: x[\"user_id\"]))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "8dda9b49",
   "metadata": {},
   "outputs": [],
   "source": [
    "### get unique item and user id's as a lookup table\n",
    "unique_item_titles = np.unique(np.concatenate(list(items.batch(1000))))\n",
    "unique_user_ids = np.unique(np.concatenate(list(interactions.batch(1_000).map(lambda x: x[\"user_id\"]))))\n",
    "\n",
    "# Randomly shuffle data and split between train and test.\n",
    "tf.random.set_seed(42)\n",
    "shuffled = interactions.shuffle(100_000, seed=42, reshuffle_each_iteration=False)\n",
    "\n",
    "train = shuffled.take(60_000)\n",
    "test = shuffled.skip(60_000).take(20_000)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "106e5bbe",
   "metadata": {},
   "source": [
    "# Ranking and Retrieval"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46d25caa",
   "metadata": {},
   "source": [
    "#### Simple Retrieval Model\n",
    "\n",
    "There are five important component of the query and candicate tower: candidate model (item_model), querty model (user_model), metrics, task, and compute loss. I will explain each of them in the model below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "22c57e7f",
   "metadata": {},
   "outputs": [],
   "source": [
    "class RetailModel(tfrs.Model):\n",
    "\n",
    "    def __init__(self, user_model, item_model):\n",
    "        super().__init__()\n",
    "        \n",
    "        ### Candidate model (item)\n",
    "        ### This is Keras preprocessing layers to first convert user ids to integers, \n",
    "        ### and then convert those to user embeddings via an Embedding layer. \n",
    "        ### We use the list of unique user ids we computed earlier as a vocabulary:\n",
    "        item_model = tf.keras.Sequential([\n",
    "                                        tf.keras.layers.experimental.preprocessing.StringLookup(\n",
    "                                        vocabulary=unique_item_titles, mask_token=None),\n",
    "                                        tf.keras.layers.Embedding(len(unique_item_titles) + 1, embedding_dimension)\n",
    "                                        ])\n",
    "        ### we pass the embedding layer into item model\n",
    "        self.item_model: tf.keras.Model = item_model\n",
    "            \n",
    "        ### Query model (users)    \n",
    "        user_model = tf.keras.Sequential([\n",
    "                                        tf.keras.layers.experimental.preprocessing.StringLookup(\n",
    "                                        vocabulary=unique_user_ids, mask_token=None),\n",
    "                                        # We add an additional embedding to account for unknown tokens.\n",
    "                                        tf.keras.layers.Embedding(len(unique_user_ids) + 1, embedding_dimension)\n",
    "                                        ])\n",
    "        self.user_model: tf.keras.Model = user_model\n",
    "        \n",
    "        ### for retrieval model. we take top-k accuracy as metrics\n",
    "        metrics = tfrs.metrics.FactorizedTopK(\n",
    "                                            candidates=items.batch(128).map(item_model)\n",
    "        \n",
    "        # define the task, which is retrieval                                    )    \n",
    "        task = tfrs.tasks.Retrieval(\n",
    "                                    metrics=metrics\n",
    "                                    )\n",
    "       \n",
    "        self.task: tf.keras.layers.Layer = task\n",
    "\n",
    "    def compute_loss(self, features: Dict[Text, tf.Tensor], training=False) -> tf.Tensor:\n",
    "        # We pick out the user features and pass them into the user model.\n",
    "        user_embeddings = self.user_model(features[\"user_id\"])\n",
    "        # And pick out the movie features and pass them into the movie model,\n",
    "        # getting embeddings back.\n",
    "        positive_movie_embeddings = self.item_model(features[\"product_id\"])\n",
    "\n",
    "        # The task computes the loss and the metrics.\n",
    "        return self.task(user_embeddings, positive_movie_embeddings)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9483ed0",
   "metadata": {},
   "source": [
    "#### Fitting and evaluating"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "6bd1be4a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "8/8 [==============================] - 13s 1s/step - factorized_top_k/top_1_categorical_accuracy: 0.0013 - factorized_top_k/top_5_categorical_accuracy: 0.0047 - factorized_top_k/top_10_categorical_accuracy: 0.0074 - factorized_top_k/top_50_categorical_accuracy: 0.0192 - factorized_top_k/top_100_categorical_accuracy: 0.0291 - loss: 62067.3594 - regularization_loss: 0.0000e+00 - total_loss: 62067.3594\n",
      "Epoch 2/10\n",
      "8/8 [==============================] - 11s 1s/step - factorized_top_k/top_1_categorical_accuracy: 0.0873 - factorized_top_k/top_5_categorical_accuracy: 0.1983 - factorized_top_k/top_10_categorical_accuracy: 0.2584 - factorized_top_k/top_50_categorical_accuracy: 0.4776 - factorized_top_k/top_100_categorical_accuracy: 0.6039 - loss: 61768.0087 - regularization_loss: 0.0000e+00 - total_loss: 61768.0087\n",
      "Epoch 3/10\n",
      "8/8 [==============================] - 11s 1s/step - factorized_top_k/top_1_categorical_accuracy: 0.2200 - factorized_top_k/top_5_categorical_accuracy: 0.3801 - factorized_top_k/top_10_categorical_accuracy: 0.4406 - factorized_top_k/top_50_categorical_accuracy: 0.6642 - factorized_top_k/top_100_categorical_accuracy: 0.8036 - loss: 60099.1068 - regularization_loss: 0.0000e+00 - total_loss: 60099.1068\n",
      "Epoch 4/10\n",
      "8/8 [==============================] - 12s 1s/step - factorized_top_k/top_1_categorical_accuracy: 0.2785 - factorized_top_k/top_5_categorical_accuracy: 0.4599 - factorized_top_k/top_10_categorical_accuracy: 0.5180 - factorized_top_k/top_50_categorical_accuracy: 0.7009 - factorized_top_k/top_100_categorical_accuracy: 0.8098 - loss: 55294.3611 - regularization_loss: 0.0000e+00 - total_loss: 55294.3611\n",
      "Epoch 5/10\n",
      "8/8 [==============================] - 13s 2s/step - factorized_top_k/top_1_categorical_accuracy: 0.3065 - factorized_top_k/top_5_categorical_accuracy: 0.5276 - factorized_top_k/top_10_categorical_accuracy: 0.5828 - factorized_top_k/top_50_categorical_accuracy: 0.7376 - factorized_top_k/top_100_categorical_accuracy: 0.8273 - loss: 47883.4540 - regularization_loss: 0.0000e+00 - total_loss: 47883.4540\n",
      "Epoch 6/10\n",
      "8/8 [==============================] - 15s 2s/step - factorized_top_k/top_1_categorical_accuracy: 0.3571 - factorized_top_k/top_5_categorical_accuracy: 0.6018 - factorized_top_k/top_10_categorical_accuracy: 0.6493 - factorized_top_k/top_50_categorical_accuracy: 0.7834 - factorized_top_k/top_100_categorical_accuracy: 0.8602 - loss: 40391.6057 - regularization_loss: 0.0000e+00 - total_loss: 40391.6057\n",
      "Epoch 7/10\n",
      "8/8 [==============================] - 17s 2s/step - factorized_top_k/top_1_categorical_accuracy: 0.3963 - factorized_top_k/top_5_categorical_accuracy: 0.6771 - factorized_top_k/top_10_categorical_accuracy: 0.7188 - factorized_top_k/top_50_categorical_accuracy: 0.8341 - factorized_top_k/top_100_categorical_accuracy: 0.8985 - loss: 33843.2250 - regularization_loss: 0.0000e+00 - total_loss: 33843.2250\n",
      "Epoch 8/10\n",
      "8/8 [==============================] - 19s 2s/step - factorized_top_k/top_1_categorical_accuracy: 0.4230 - factorized_top_k/top_5_categorical_accuracy: 0.7477 - factorized_top_k/top_10_categorical_accuracy: 0.7853 - factorized_top_k/top_50_categorical_accuracy: 0.8878 - factorized_top_k/top_100_categorical_accuracy: 0.9287 - loss: 28254.5170 - regularization_loss: 0.0000e+00 - total_loss: 28254.5170\n",
      "Epoch 9/10\n",
      "8/8 [==============================] - 18s 2s/step - factorized_top_k/top_1_categorical_accuracy: 0.4504 - factorized_top_k/top_5_categorical_accuracy: 0.8098 - factorized_top_k/top_10_categorical_accuracy: 0.8448 - factorized_top_k/top_50_categorical_accuracy: 0.9247 - factorized_top_k/top_100_categorical_accuracy: 0.9504 - loss: 23588.9036 - regularization_loss: 0.0000e+00 - total_loss: 23588.9036\n",
      "Epoch 10/10\n",
      "8/8 [==============================] - 18s 2s/step - factorized_top_k/top_1_categorical_accuracy: 0.4745 - factorized_top_k/top_5_categorical_accuracy: 0.8694 - factorized_top_k/top_10_categorical_accuracy: 0.8987 - factorized_top_k/top_50_categorical_accuracy: 0.9479 - factorized_top_k/top_100_categorical_accuracy: 0.9737 - loss: 19782.7146 - regularization_loss: 0.0000e+00 - total_loss: 19782.7146\n"
     ]
    }
   ],
   "source": [
    "### Fitting and evaluating\n",
    "\n",
    "### we choose the dimensionality of the query and candicate representation.\n",
    "embedding_dimension = 32\n",
    "\n",
    "## we pass the model, which is the same model we created in the query and candidate tower, into the model\n",
    "item_model = tf.keras.Sequential([\n",
    "                                tf.keras.layers.experimental.preprocessing.StringLookup(\n",
    "                                vocabulary=unique_item_titles, mask_token=None),\n",
    "                                tf.keras.layers.Embedding(len(unique_item_titles) + 1, embedding_dimension)\n",
    "                                ])\n",
    "\n",
    "user_model = tf.keras.Sequential([\n",
    "                                tf.keras.layers.experimental.preprocessing.StringLookup(\n",
    "                                vocabulary=unique_user_ids, mask_token=None),\n",
    "                                # We add an additional embedding to account for unknown tokens.\n",
    "                                tf.keras.layers.Embedding(len(unique_user_ids) + 1, embedding_dimension)\n",
    "                                ])\n",
    "\n",
    "model = RetailModel(user_model, item_model)\n",
    "\n",
    "# a smaller learning rate may make the model move slower and prone to overfitting, so we stick to 0.1\n",
    "# other optimizers, such as SGD and Adam, are listed here https://www.tensorflow.org/api_docs/python/tf/keras/optimizers\n",
    "model.compile(optimizer=tf.keras.optimizers.Adagrad(learning_rate=0.1))\n",
    "\n",
    "cached_train = train.shuffle(100_000).batch(8192).cache()\n",
    "cached_test = test.batch(4096).cache()\n",
    "\n",
    "## fit the model with ten epochs\n",
    "model_hist = model.fit(cached_train, epochs=10)\n",
    "\n",
    "#evaluate the model\n",
    "model.evaluate(cached_test, return_dict=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "894b18a5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4/4 [==============================] - 6s 940ms/step - factorized_top_k/top_1_categorical_accuracy: 4.6207e-04 - factorized_top_k/top_5_categorical_accuracy: 0.0041 - factorized_top_k/top_10_categorical_accuracy: 0.0065 - factorized_top_k/top_50_categorical_accuracy: 0.0208 - factorized_top_k/top_100_categorical_accuracy: 0.0337 - loss: 22440.4807 - regularization_loss: 0.0000e+00 - total_loss: 22440.4807\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'factorized_top_k/top_1_categorical_accuracy': 0.0004620716208592057,\n",
       " 'factorized_top_k/top_5_categorical_accuracy': 0.004081632476300001,\n",
       " 'factorized_top_k/top_10_categorical_accuracy': 0.006469002924859524,\n",
       " 'factorized_top_k/top_50_categorical_accuracy': 0.02079322375357151,\n",
       " 'factorized_top_k/top_100_categorical_accuracy': 0.033654216676950455,\n",
       " 'loss': 4584.71142578125,\n",
       " 'regularization_loss': 0,\n",
       " 'total_loss': 4584.71142578125}"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.evaluate(cached_test, return_dict=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "9be01ecc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x244a538a580>"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAr7UlEQVR4nO3deXxddZ3/8dcnS5ulSZqtW7qk0MUWKNCGgiBQN0RHRHFUFBBRqPx+oOhv5gEOo4KjM+Ms+HMchNpxAAtoHRGU4ceA4Eiqw9aFvaWhdE1py83SpE2TZvv8/jgn6W2aJreltyf33vfz8biP3LPccz85bb6fc77f7/l+zd0REZHMlRV1ACIiEi0lAhGRDKdEICKS4ZQIREQynBKBiEiGUyIQEclwSgQi0s/Mqs3MzSwn6ljk+FEikEiY2VNm1mxmo6OORSTTKRHIcWdm1cC5gAMfO87frStdkQGUCCQKnweeBe4BrozfYGZTzOxBM4uZWaOZ3R637RozW2dme8xsrZnND9e7mc2I2+8eM/te+H6RmdWb2U1mthO428xKzeyR8Duaw/eT4z5fZmZ3m9lb4fbfhOtfNbOL4vbLNbMGMztt4C8YxvnRuOWccN/5ZpZnZveFv99uM1tpZuMHO1FmNsnMfh3GusnMvhq37VYze8DMfhmekzVmdmrc9jnhndduM3vNzD4Wty3fzG4zsy1m1mJmfzKz/LivvszMtoYx//VgsUn6UCKQKHweuD98faivEDSzbOARYAtQDVQBy8NtnwJuDT9bTHAn0Zjg900AyoBpwGKC//d3h8tTgXbg9rj97wUKgJOAccD/DdcvAy6P2+8jwA53f3GQ7/wF8Nm45Q8BDe6+hiD5lQBTgHLg2jCGg5hZFvCfwEsE5+L9wNfM7ENxu10M/Cr8/X4O/CZMULnhZ38X/g5fAe43s9nh5/4ZWACcHX72RqA37rjvAWaH3/ltM5szyO8o6cLd9dLruL0ICpguoCJcfh34evj+3UAMyBnkc48DNxzmmA7MiFu+B/he+H4R0AnkDRHTaUBz+H4iQYFYOsh+k4A9QHG4/ABw42GOOSPctyBcvh/4dvj+i8DTwLxhztWZwNYB6/4KuDt8fyvwbNy2LGAHQbXbucBOICtu+y/Cz2QRJJ5TB/nO6vB8To5b9zxwadT/d/RK3kt3BHK8XQn8zt0bwuWfc6B6aAqwxd27B/ncFODNo/zOmLt39C2YWYGZ/SSsFmkFVgBjwzuSKUCTuzcPPIi7vwX8D/BJMxsLfJiggD+Eu28A1gEXmVkBwR3Mz8PN9xIktuVh9dM/hlfwA00DJoVVO7vNbDdwMxBfjbQt7jt7gXqChDUJ2Bau67OF4M6iAshj6PO5M+79PmDMEPtKilPDmRw3YR30p4HssL4eYDRBIXwqQaE21cxyBkkG24ATD3PofQRVOX0mEBSIfQYOsfsXBNUeZ7r7zrCO/wXAwu8pM7Ox7r57kO/6GXA1wd/OM+6+/XC/Lweqh7KAtWFywN27gO8A3wkbzh8F1gP/PuDz24BN7j5ziO+Y0vcmrEqaDLzVt83MsuKSwVSgDmgAOgjO50tDHFsyhO4I5Hj6ONADzCWojjkNmAP8kaDu/3mCqo3vm1lh2Kh6TvjZnwJ/aWYLLDDDzKaF214EPmdm2WZ2IXD+MHEUEVSN7DazMuCWvg3uvgP4L+COsFE518zOi/vsb4D5wA0EbQZDWQ5cAPwvDtwNYGbvNbNTwjuQVoKqsp5BPv880Bo2dOeHv9/JZnZG3D4LzOwSC3pDfQ3YT9AQ/xzQBtwY/g6LgIuA5WFiuAv4QdgYnW1m7zZ15c1YSgRyPF1JUL+91d139r0IGmovI7giv4igfn0rwVX9ZwDc/VfA3xIUqHsICuSy8Lg3hJ/bHR7nN8PE8UMgn+DK+FngsQHbryAonF8H3iYoYAnjaAd+DUwHHhzqS8Kk8gxBg+wv4zZNIGhfaCWoPqoF7hvk8z3h73UasCmM96cEDc19fktwjprDuC9x9y537ySojvpw+Lk7gM+7++vh5/4SeAVYCTQB/4DKg4xl7pqYRuRImNm3gVnufvmwOyc3jlsJGskjjUNSn9oIRI5AWJX0JYKrb5G0oFtBkQSZ2TUEDbj/5e4roo5H5FhR1ZCISIbTHYGISIZLuTaCiooKr66ujjoMEZGUsnr16gZ3rxxsW8olgurqalatWhV1GCIiKcXMthxuW9KqhszsLjN728xePcx2M7MfmdkGM3vZwpEkRUTk+EpmG8E9wIVDbP8wMDN8LQbuTGIsIiJyGElLBGH3uqYhdrkYWOaBZwnGm5mYrHhERGRwUbYRVBE3ciLBcAJVBGPNHMTMFhPcNTB16tRDDtTV1UV9fT0dHR2HbJPh5eXlMXnyZHJzBxsAU0TSXZSJwAZZN+hDDe6+FFgKUFNTc8g+9fX1FBUVUV1djdlgh5XDcXcaGxupr69n+vTpUYcjIhGI8jmCeuKG0OXg4XOPSEdHB+Xl5UoCR8HMKC8v192USAaLMhE8DHw+7D10FtASjtZ4VJQEjp7OnUhmS1rVkJn9gmCawAozqycY8z0XwN2XEEzG8RFgA8HEIlclKxYRkVTUtr+bzY1tbG7Yx+bGNuZNLuHcmYM+E/aOJC0RuPtnh9nuwHXJ+n4RkVTQ3tnDlqY2Nje0salhX/CzMVh+e8/+g/a99vwTUysRSHJ0d3eTk6N/NpFU0tHVw7amfWxqaGNz44ECf3NjGztaDm6fqxgziuryQs6bVcn0ikKqywupriiguryQwtHJ+dtXiXIMffzjH2fbtm10dHRwww03sHjxYh577DFuvvlmenp6qKio4Pe//z179+7lK1/5CqtWrcLMuOWWW/jkJz/JmDFj2Lt3LwAPPPAAjzzyCPfccw9f+MIXKCsr44UXXmD+/Pl85jOf4Wtf+xrt7e3k5+dz9913M3v2bHp6erjpppt4/PHHMTOuueYa5s6dy+23385DDz0EwBNPPMGdd97Jgw8OObmWiByhzu5etjYdKOD7Cv3NDft4q6Wd+IGeywpHUV1ewLtPKKe6opDqikKmhwV+Ud7x78addongO//5Gmvfaj2mx5w7qZhbLjpp2P3uuusuysrKaG9v54wzzuDiiy/mmmuuYcWKFUyfPp2mpuD5uu9+97uUlJTwyiuvANDc3Dzssevq6njyySfJzs6mtbWVFStWkJOTw5NPPsnNN9/Mr3/9a5YuXcqmTZt44YUXyMnJoampidLSUq677jpisRiVlZXcfffdXHWVmmNEjkZXTy/1ze1hNc7BBf725nZ64wr7kvxcqisKOaO6lOqKyQeu7ssLKSkYWc/spF0iiNKPfvSj/ivvbdu2sXTpUs4777z+/vllZcEUu08++STLly/v/1xpaemwx/7Upz5FdnY2AC0tLVx55ZW88cYbmBldXV39x7322mv7q476vu+KK67gvvvu46qrruKZZ55h2bLh5lwXyVx91ThbGvextSl4bQ7r7Oub2+mOK+2LRudQXVHIaVNK+cRpVQdd3ZcWjorwtzgyaZcIErlyT4annnqKJ598kmeeeYaCggIWLVrEqaeeyvr16w/Z190H7bIZv25gv/7CwsL+99/61rd473vfy0MPPcTmzZtZtGjRkMe96qqruOiii8jLy+NTn/qU2hgko7k7DXs72drUFhT0je1saWrrL/wHNtAWjspmWnkhJ00q4c/mTaS6vDC4uq8opLxwVFp0v1aJcIy0tLRQWlpKQUEBr7/+Os8++yz79++ntraWTZs29VcNlZWVccEFF3D77bfzwx/+EAiqhkpLSxk/fjzr1q1j9uzZPPTQQxQVFR32u6qqqgC45557+tdfcMEFLFmyhEWLFvVXDZWVlTFp0iQmTZrE9773PZ544olknwqRyO3v7qG+uZ2tTfsOvroPf7Z39fTvawYTivOYWlbA+bMqmVpWwNTyAqaWFTCtvJDSgty0KOyHokRwjFx44YUsWbKEefPmMXv2bM466ywqKytZunQpl1xyCb29vYwbN44nnniCb37zm1x33XWcfPLJZGdnc8stt3DJJZfw/e9/n49+9KNMmTKFk08+ub/heKAbb7yRK6+8kh/84Ae8733v619/9dVXU1dXx7x588jNzeWaa67h+uuvB+Cyyy4jFosxd+7c43I+RJLJ3Wne19VfdbO1Mbi639IYFPw7WjsOapzNz81malkBU8oKOGdGBdPCgn5qeQFVY/PJy82O7pcZAVJuzuKamhofODHNunXrmDNnTkQRpYbrr7+e008/nS996UuDbtc5lJGmp9epb953UAEf/37P/u6D9h9XNDoo3OOu6PveV44ZnfZX9cMxs9XuXjPYNt0RZIAFCxZQWFjIbbfdFnUoIoNq29/N6ztbWftWK2t3BD/X79pDR1dv/z6jcrKYUprP1LICFk4vY0pZX/VNAVNKC8gfldlX9e+EEkEGWL16ddQhiABBlc6u1v2s23GgwF+7o5XNjW39VTljC3KZO7GYy8+cxqzxRUE1TnkB44vyyMrK7Kv6ZEmbRHC4HjMyvFSrHpTU0N3Ty8aGtoOu8tfuaKWprbN/n2nlBcydWMwlp1cxZ2IxcycVM7EkT3/Lx1laJIK8vDwaGxs1FPVR6JuPIC8vL+pQJIXt6eji9Z17gsL+rVbW7Wzl9Z176OwOqnZG5WQxe3wRH5wznrmTggL/XROKInmKVg6VFolg8uTJ1NfXE4vFog4lJfXNUCYyHHdnR0tHULXTd6W/o5Utjfv69ykrHMXcicV84exq5oZX+SdUFJKTHeWo9zKUtEgEubm5ml1L5Bjr6unlzdje/qv8vkJ/976u/n2mVxRy8qQSPl0zhTkTi5g7sYTxxeqhk2rSIhGIyDu3o6Wd5zc1sWpzMy9sa6Zu5146e4KqndE5WbxrYjEfPnkicycWMXdSMbMnFDMmSaNhyvGlf0WRDOTubHh7Lys3N7NycxMrNzdR39wOBEMqnD61lKveE1TtnDSpmOpyVe2kMyUCkQzQ2d3La2+1sHJzE89vamb1liaawyqeijGjWTi9lC+eM52F08t414QiFfoZRolAJA3t3d/NC1ubWbmpiZVhVU/fw1nTKwr5wJzxnDG9jIXVZUwrL1CdfoZTIhBJA7E9+1m1uam/qmftjlZ6ep0sg5MmlfDZhVNZWF3GgupSxhWpq7AcTIlAJMW4O1sa9/XX7a/c3MymhjYgaNQ9fepYrlt0IjXVZcyfVqoGXRmW/oeIjHA9vc66Ha0HFfyxcMz8sQW51Ewr49IzpnDG9DJOnlTCqBzV78uRUSIQGWE6unp4cdvuoH5/SzNrtjSzNxxps2psPuecWN5fv39i5RiNvyPvmBKBSIR6e52NDW28sn03L9e38NK23byyvYWunmD8p9nji/j46ZM4o7qMM6rLmDQ2P+KIJR0pEYgcJ311+y9vb+GV+qDgf3V7C22dwWxZ+bnZnDSpmC++ZzoLq8uomVY24iY5l/SkRCCSBO7O9t3tvFzfwsv1LbyyfTev1LfQ2hFU8YzKyWLuxGI+uWAyp1SVMG/yWGaMG0O2qnkkAkoEIu9Q3xj7L9cH1TpBwd/SP9xybrbxrgnFfPTUScyrKuGUySXMGl9Erh7akhFCiUDkCMX27O+v03+lvoWXt7f09+LJzjJmjhvDB+aMY97kscybXMLsCUWMztHsWTJyKRGIDKG5rZNXtgdX+H0NuTtaOgAwgxmVYzh3ZkV4pT+WuROLNWWipBwlApFQS3sXr21vCRtzW3h5+262NbX3bz+hopCF08v66/RPmlRMoR7WkjSg/8WSEXp6nca9+9nR0sHO1g52tnQE71va2dHSwVst7QcV+lPK8plXNZbLzpzGvKoSTqoqoSRfPXgkPSkRSMrr6ullV2sHu1r7CvcDP/sK/V2tHXT3Hjw3c262Mb44j4kleZw2pZRLz5jKKVUlnFJVQmnhqIh+G5HjT4lARrSOrp4DBXtrcPW+q385+Nmwdz9+cBlPfm42E0vymFCSx5knlDEhLPAnlOT3ry8rGKWnckVIciIwswuBfwGygZ+6+/cHbC8B7gOmhrH8s7vfncyYJDruzv7uXjq6emjv6qG9M/jZuLdz0Oqana0dB02L2Kc4L4eJJflMKMljzoRiJpTk9RfuE0vymVCcR3F+joZWFklQ0hKBmWUDPwY+CNQDK83sYXdfG7fbdcBad7/IzCqB9WZ2v7t3JisuOVRv74ACOiykO7p66Ojq7V/XERbc8fsdWNd7YFvcfv2f7+yho7vnkCv3gSrGjGJCSR6TS/OpqS7tL9gnluQxviSPCcV5aqAVOcaS+Re1ENjg7hsBzGw5cDEQnwgcKLLg0m0M0AR0JzEmIbgy/+4j63johfr+QvxojM7JIn9UNnk52cHP3Gzyc4N1Jfm55I3KJj83eOXlZgU/D1oXrC8rHM3EkjzGFY9Wf3uRCCQzEVQB2+KW64EzB+xzO/Aw8BZQBHzG3Y+uVJKE/fufNnHX/2zigrnjmV5RGBTgo7LJ6yvY4wrq/FEHv8+LK/xVvy6SHpKZCAYrJQZWDHwIeBF4H3Ai8ISZ/dHdWw86kNliYDHA1KlTj32kGeRPbzTwd4+u48KTJnDHZfNVmIsIyRzspB6YErc8meDKP95VwIMe2ABsAt418EDuvtTda9y9prKyMmkBp7stjW1c9/M1zBxXxG2fPlVJQESA5CaClcBMM5tuZqOASwmqgeJtBd4PYGbjgdnAxiTGlLHa9nezeNlqzODfPl+jBlcR6Ze00sDdu83seuBxgu6jd7n7a2Z2bbh9CfBd4B4ze4WgKukmd29IVkyZqrfX+Yv/eIk33t7Dsi+eydTygqhDEpERJKmXhe7+KPDogHVL4t6/BVyQzBgEbv/DBh57bSff/LM5vGdmRdThiMgIowHR09wTa3fxgyfquOT0Kr70nulRhyMiI5ASQRp7Y9cevv7LF5k3uYS/u+QUPWkrIoNSIkhTLfu6uGbZKvJys/nJFQvIy9WDWiIyOCWCNNTT63x1+Qts393OksvnM7EkP+qQRGQEUx/CNPSPj79ObV2Mv7/kFGqqy6IOR0RGON0RpJnfvridn9Ru5PKzpvLZhXoKW0SGp0SQRl7d3sJNv36ZhdVlfPujJ0UdjoikCCWCNNGwdz+Ll62irGAUd1w+n1E5+qcVkcSojSANdPX08r/vX0NjWycPXHs2FWNGRx2SiKQQJYI08Df/uZbnNzXxL5eeximTS6IOR0RSjOoPUtzy57dy77Nb+PJ5J3DxaVVRhyMiKUiJIIWt3tLEt377KufNquTGCw8ZvVtEJCFKBClqR0s7X753DVVj8/nXS08nW3MLiMhRUhtBCuro6uHae1fT3tnNz685k5KC3KhDEpEUpkSQYtydmx96hZfqW1h6xQJmjS+KOiQRSXHDVg2ZmcYoGEHu+p/NPLhmO1/7wEwuOGlC1OGISBpIpI3gOTP7lZl9xDSOcaT6Jp7/0Enj+er7ZkYdjoikiUQSwSxgKXAFsMHM/s7MZiU3LBloa+M+rv/FGk6sLOS2T5+miedF5JgZNhF44Al3/yxwNXAl8LyZ1ZrZu5MeodC2v5trlq3CPZh4fowmnheRY2jYEsXMyoHLCe4IdgFfAR4GTgN+BWj+wyRyd/7yV8HE8z/74kKmlRdGHZKIpJlELi2fAe4FPu7u9XHrV5nZksN8Ro6R2/97A//16k7++iNzOHdmZdThiEgaSiQRzHZ3H2yDu//DMY5H4jyxdhe3PVHHJ06v4upzdeMlIsmRSGPx78xsbN+CmZWa2ePJC0kANrwdTDx/SlUJf6+J50UkiRJJBJXuvrtvwd2bgXFJi0hoae/immWrycvN0sTzIpJ0iSSCHjPrn/PQzKYBg1YVyTvX0+t89RcvUN+8jzsvX8CksZp4XkSSK5E2gr8G/mRmteHyecDi5IWU2f7p8fXU1sX420+czBmaeF5EjoNhE4G7P2Zm84GzAAO+7u4NSY8sA/32xe0sqX2Tz505lcvOnBZ1OCKSIRJ9MqkHeBvIA+aaGe6+InlhZZ6+iefPqC7l1os08byIHD+JPFB2NXADMBl4keDO4BngfUmNLIM07N3Pl+9dTWnBKO64bIEmnheR4yqREucG4Axgi7u/FzgdiCU1qgzSN/F8w979LL2ihsoiTTwvIsdXIomgw907AMxstLu/DsxObliZ47uPBBPP/8Mn52nieRGJRCJtBPXhA2W/AZ4ws2bgrWQGlSmWP7+VZc9s4Zpzp/Px0zXxvIhEI5FeQ58I395qZn8ASoDHkhpVBuibeP7cmRXcpInnRSRCQ1YNmVmWmb3at+zute7+sLt3JnJwM7vQzNab2QYz+8Zh9llkZi+a2WtxzyqktZ0tHVx73xomluTzr589nZxsNQ6LSHSGvCNw914ze8nMprr71iM5sJllAz8GPgjUAyvN7GF3Xxu3z1jgDuBCd99qZmk/dEVHVw9fvncV+/Z3c//VZzK2YFTUIYlIhkukjWAi8JqZPQ+09a10948N87mFwAZ33whgZsuBi4G1cft8DniwL8m4+9tHEHtK+rcVG3mpvoWfaOJ5ERkhEkkE3znKY1cB2+KW64EzB+wzC8g1s6eAIuBf3H3ZwAOZ2WLCYS2mTp06cHNKeXLdLhZMK+VDmnheREaIRBqLj7befrBxkwcOVpcDLADeD+QDz5jZs+5eNyCGpQTzJlNTU5OyA9417t3Py9tb+PoHNOWziIwciTxZvIcDBfgoIBdoc/fiYT5aD0yJW57Mod1O64EGd28D2sxsBXAqUEca+tOGBtzh/FmaaUxERo5EJq8vcvfi8JUHfBK4PYFjrwRmmtl0MxsFXEow13G83wLnmlmOmRUQVB2tO7JfIXXU1sUoKxzFKVV6cExERo5EB53r5+6/OVxX0AH7dZvZ9cDjQDZwl7u/ZmbXhtuXuPs6M3sMeBnoBX7q7q8e/qipq7fXWVHXwLkzK8jK0mxjIjJyJFI1dEncYhZQQ4IT07j7o8CjA9YtGbD8T8A/JXK8VLZ2RysNe/dzniagF5ERJpE7govi3ncDmwm6gcoRqK0Lxuk7d1ZFxJGIiBwskV5DVx2PQNJdbV2MkyYVM64oL+pQREQOMmxjsZn9LHwCuG+51MzuSmpUaaa1o4s1W5rVW0hERqREBrmZ5+67+xbcvZlgTgJJ0NMbGunudSUCERmREkkEWWZW2rdgZmUcRW+jTFZbF2PM6BzmTysdfmcRkeMskQL9NuBpM3uAoLfQp4G/TWpUacTdWVEX45wZ5eRqlFERGYESeaBsGcFDZLsIpqi8xN3vTXZg6eLN2F62727n/FlpP7CqiKSoRJ4jOAt4zd1vD5eLzOxMd38u6dGlgafWB91Gz1O3UREZoRKpq7gT2Bu33BaukwTU1sWYMW4Mk0sLog5FRGRQiSQCc/f+J4ndvRc1FiekvbOH5zY1qbeQiIxoiSSCjWb2VTPLDV83ABuTHVg6eHZTI53dvUoEIjKiJZIIrgXOBrZzYHKZxckMKl3Uro+Rl5vFwullUYciInJYiQwx8TbBENJyhFbUxTjrhHLycrOjDkVE5LAS6TWUB3wJOAnoHyjH3b+YxLhS3tbGfWxsaOOKd0+LOhQRkSElUjV0LzAB+BBQSzDT2J5kBpUOat8Iuo2qfUBERrpEEsEMd/8WwfSUPwP+DDgluWGlvtr1MaaU5TO9ojDqUEREhpRIIugKf+42s5OBEqA6aRGlgc7uXp5+s4HzZ1ViptnIRGRkS+R5gKXhoHPfJJhzeAzwraRGleJWbWliX2ePhpUQkZSQSK+hn4ZvVwAnJDec9FBbFyM323j3ieVRhyIiMiwNh5kEtetj1EwrY8xoPYAtIiOfEsExtqu1g9d37uH82eotJCKpQYngGOubpF7dRkUkVQxZd2FmJcCFQBXBpDRvAY/HT10pB6utizGuaDTvmlAUdSgiIgk57B2BmX0eWAMsAgqAQuC9wOpwmwzQ3dPLn95Qt1ERSS1D3RH8NbBg4NV/2JX0OWBZEuNKSS/Vt9DS3qX2ARFJKUO1ERhBddBAveE2GaC2LkaWwXtmaDYyEUkdQ90R/C2wxsx+B2wL100FPgh8N9mBpaLauhinTRnL2IJRUYciIpKww94RhOMK1RAMNLcf6ASeAmrc/Z7jEVwqaWrr5OX63XqaWERSzpC9hty9GVhuZmXBojcfn7BSzx/fiOGO2gdEJOUM1WtoqpktN7O3CRqHV5rZ2+G66uMWYYqorYtRWpDLKVUlUYciInJEhmos/iXwEDDR3We6+wxgIvAbYPlxiC1l9PY6K+oaOHdmJdlZakcXkdQyVCKocPdfuntP3wp373H35YBGU4uzdkcrDXv362liEUlJQ7URrDazO4CfcaDX0BTgSuCFZAeWSvqGlTh3lrqNikjqGeqO4PPAK8B3gMeB3wG3Aq8CVyRycDO70MzWm9kGM/vGEPudYWY9ZvbnCUc+gtTWxThpUjHjivKG31lEZIQ57B2Bu3cCd4avI2Zm2cCPCZ47qCdobH7Y3dcOst8/ECSblNPa0cWaLc0sPk9TNYhIajqq0UfN7NsJ7LYQ2ODuG8Okshy4eJD9vgL8Gnj7aGKJ2tMbGunudbUPiEjKOtphqK9OYJ8qDrQtQHBXUBW/g5lVAZ8Algx1IDNbbGarzGxVLBY70liTqrYuxpjROcyfVhp1KCIiR+WwVUNm1nq4TUB+AscerB/lwLGLfgjc5O49Q43W6e5LgaUANTU1g41/FAl3Z0VdjHNmlJObrakdRCQ1DdVraDdwhrvvGrjBzLYduvsh6gl6GfWZTDCfQbwagieXASqAj5hZt7v/JoHjR+7N2F62727nuvfOiDoUEZGjNlQiWAZMAw5JBMDPEzj2SmCmmU0HtgOXAp+L38Hdp/e9N7N7gEdSJQkAPLU+qKY6T91GRSSFDdVr6JtDbLtpuAO7e7eZXU/QGygbuMvdXzOza8PtQ7YLpILauhgzxo1hcmlB1KGIiBy1IQedG8jMbnX3WxPd390fBR4dsG7QBODuXziSWKLW3tnDc5uauOKsaVGHIiLyjhxpC+fHkhJFCnp2UyOd3b3qNioiKe9IE4FGVAvVro+Rl5vFwullUYciIvKOHGkimJ+UKFLQiroYZ51QTl5udtShiIi8I8MmAjM7wcz+08wagF1m9lszy+jxFLY27mNjQ5uqhUQkLSRyR/Bz4D+ACcAk4FfAL5IZ1EhX+0Zft1ElAhFJfYkkAnP3e929O3zdx6FPCGeU2vUxJpfmc0JFYdShiIi8Y4kkgj+Y2TfMrNrMppnZjcD/M7OycC7jjNLZ3cvTbzZw/qxKhhoWQ0QkVSTyHMFnwp9fHrD+iwR3BhnVXrBqSxP7OnvUPiAiaWPYRBA/DITAiroGcrKMs2doWAkRSQ/DJgIzywX+F3BeuOop4Cfu3pXEuEas2roYNdWljBl9RA9li4iMWIm0EdwJLADuCF8LOMpZy1LdrtYO1u1o5fxZ46IORUTkmBlqPoIcd+8mGIr61LhN/21mLyU/tJFnRThJvdoHRCSdDHVH8Hz4s8fMTuxbGT5M1pPUqEao2roYlUWjmTOxKOpQRESOmaEquvv6Rv4lQRfSjeFyNXBVMoMaiXp6nT++0cAH545Xt1ERSStDJYJKM/s/4fufEMwp0AbkAacDf0hybCPKS/W7aWnvUrWQiKSdoRJBNjCGg0ccHRP+zLi6kdr1MbIM3qNuoyKSZoZKBDvc/W+OWyQjXG1djFOnjKW0cFTUoYiIHFNDNRarIjzU3NbJS/W7VS0kImlpqETw/uMWxQj3xw0NuKvbqIikp8MmAndvOp6BjGS162OMLchl3uSxUYciInLMHekMZRmnt9eprYtx7sxKsrNUWyYi6UeJYBjrdrbSsHe/qoVEJG0pEQyjNhxW4ryZ6jYqIulJiWAYtetjzJ1YzLjivKhDERFJCiWCIezp6GL1lmbOn61qIRFJX0oEQ3j6zUa6e13tAyKS1pQIhlBbF2PM6BzmTy2NOhQRkaRRIjgMd6d2fYyzTyxnVI5Ok4ikL5Vwh/FmrI3tu9vVPiAiaU+J4DAOdBtVIhCR9KZEcBi1dTFOrCxkSllB1KGIiCSVEsEgOrp6eG5joyapF5GMoEQwiGc3NrK/u1ftAyKSEZKaCMzsQjNbb2YbzOwbg2y/zMxeDl9Pm9mpyYwnUbV1MUbnZHHm9LKoQxERSbqkJQIzywZ+DHwYmAt81szmDthtE3C+u88DvgssTVY8R6K2LsZZJ5STl5sddSgiIkmXzDuChcAGd9/o7p3AcuDi+B3c/Wl3bw4XnwUmJzGehGxr2sfGWJueJhaRjJHMRFAFbItbrg/XHc6XgP8abIOZLTazVWa2KhaLHcMQD9XXbVTtAyKSKZKZCAabxcUH3dHsvQSJ4KbBtrv7UnevcfeaysrkFtC1dTEml+ZzQkVhUr9HRGSkSGYiqAemxC1PBt4auJOZzQN+Clzs7o1JjGdYnd29PL2hgfNnVWKm2chEJDMkMxGsBGaa2XQzGwVcCjwcv4OZTQUeBK5w97okxpKQ1VuaaevsUfuAiGSUnGQd2N27zex64HEgG7jL3V8zs2vD7UuAbwPlwB3hFXi3u9ckK6bh1NbFyMkyzp6h2chEJHMkLREAuPujwKMD1i2Je381cHUyYzgStXUxaqpLGTM6qadFRGRE0ZPFoV2tHazb0aphJUQk4ygRhFb0dRtV+4CIZBglglBtXYzKotHMmVgUdSgiIseVEgHQ0+v88Q11GxWRzKREALxUv5uW9i5VC4lIRlIiAGrXx8gyeI+6jYpIBlIiIGgfOHXKWEoLR0UdiojIcZfxiaC5rZOX6nerWkhEMlbGJ4I/bmjAHc5TIhCRDJXxiaB2fYyS/FxOnTw26lBERCKR0Ymgt9eprYtx7swKsrPUbVREMlNGJ4J1O1tp2Ltf7QMiktEyOhHUalgJEZEMTwTrY8yZWMy44ryoQxERiUzGJoI9HV2s3tKsuwERyXgZmwiefrOR7l5XIhCRjJexiaC2LkbhqGwWTCuNOhQRkUhlZCJwd2rXxzh7RgWjcjLyFIiI9MvIUvDNWBvbd7erWkhEhAxNBJqNTETkgIxMBLV1MU6oLGRKWUHUoYiIRC7jEkFHVw/PbmzU3YCISCjjEsFzm5rY392rRCAiEsq4RFC7PsbonCzOOqE86lBEREaEzEsEdW9z5gnl5OVmRx2KiMiIkFGJYFvTPt6MtalaSEQkTkYlghVvqNuoiMhAGZUIatfHqBqbz4mVhVGHIiIyYmRMIujs7uXpNxs5f3YlZpqNTESkT8YkgjVbm9m7v1vVQiIiA2RMIsjJMhbNruTsE9VtVEQkXk7UARwvNdVl3HPVwqjDEBEZcTLmjkBERAaX1ERgZhea2Xoz22Bm3xhku5nZj8LtL5vZ/GTGIyIih0paIjCzbODHwIeBucBnzWzugN0+DMwMX4uBO5MVj4iIDC6ZdwQLgQ3uvtHdO4HlwMUD9rkYWOaBZ4GxZjYxiTGJiMgAyUwEVcC2uOX6cN2R7oOZLTazVWa2KhaLHfNARUQyWTITwWBPbflR7IO7L3X3GnevqazUcwAiIsdSMhNBPTAlbnky8NZR7CMiIkmUzESwEphpZtPNbBRwKfDwgH0eBj4f9h46C2hx9x1JjElERAZI2gNl7t5tZtcDjwPZwF3u/pqZXRtuXwI8CnwE2ADsA64a7rirV69uMLMtRxlWBdBwlJ9NRzofB9P5OEDn4mDpcD6mHW6DuR9SJZ+2zGyVu9dEHcdIofNxMJ2PA3QuDpbu50NPFouIZDglAhGRDJdpiWBp1AGMMDofB9P5OEDn4mBpfT4yqo1AREQOlWl3BCIiMoASgYhIhsuYRDDckNiZxMymmNkfzGydmb1mZjdEHVPUzCzbzF4ws0eijiVqZjbWzB4ws9fD/yPvjjqmqJjZ18O/kVfN7Bdmlhd1TMmQEYkgwSGxM0k38BfuPgc4C7guw88HwA3AuqiDGCH+BXjM3d8FnEqGnhczqwK+CtS4+8kED8ZeGm1UyZERiYDEhsTOGO6+w93XhO/3EPyhHzLqa6Yws8nAnwE/jTqWqJlZMXAe8O8A7t7p7rsjDSpaOUC+meUABaTpWGiZkggSGu46E5lZNXA68FzEoUTph8CNQG/EcYwEJwAx4O6wquynZlYYdVBRcPftwD8DW4EdBGOh/S7aqJIjUxJBQsNdZxozGwP8Gviau7dGHU8UzOyjwNvuvjrqWEaIHGA+cKe7nw60ARnZpmZmpQQ1B9OBSUChmV0ebVTJkSmJQMNdD2BmuQRJ4H53fzDqeCJ0DvAxM9tMUGX4PjO7L9qQIlUP1Lt73x3iAwSJIRN9ANjk7jF37wIeBM6OOKakyJREkMiQ2BnDzIygDnidu/8g6nii5O5/5e6T3b2a4P/Ff7t7Wl71JcLddwLbzGx2uOr9wNoIQ4rSVuAsMysI/2beT5o2nCdtGOqR5HBDYkccVpTOAa4AXjGzF8N1N7v7o9GFJCPIV4D7w4umjSQwPHw6cvfnzOwBYA1BT7sXSNOhJjTEhIhIhsuUqiERETkMJQIRkQynRCAikuGUCEREMpwSgYhIhlMiEDmOzGyRRjiVkUaJQEQkwykRiAzCzC43s+fN7EUz+0k4X8FeM7vNzNaY2e/NrDLc9zQze9bMXjazh8IxajCzGWb2pJm9FH7mxPDwY+LG+78/fGpVJDJKBCIDmNkc4DPAOe5+GtADXAYUAmvcfT5QC9wSfmQZcJO7zwNeiVt/P/Bjdz+VYIyaHeH604GvEcyNcQLBk94ikcmIISZEjtD7gQXAyvBiPR94m2CY6l+G+9wHPGhmJcBYd68N1/8M+JWZFQFV7v4QgLt3AITHe97d68PlF4Fq4E9J/61EDkOJQORQBvzM3f/qoJVm3xqw31DjswxV3bM/7n0P+juUiKlqSORQvwf+3MzGAZhZmZlNI/h7+fNwn88Bf3L3FqDZzM4N118B1IbzO9Sb2cfDY4w2s4Lj+UuIJEpXIiIDuPtaM/sm8DszywK6gOsIJmk5ycxWAy0E7QgAVwJLwoI+frTOK4CfmNnfhMf41HH8NUQSptFHRRJkZnvdfUzUcYgca6oaEhHJcLojEBHJcLojEBHJcEoEIiIZTolARCTDKRGIiGQ4JQIRkQz3/wEZAHav0GSooAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# num_validation_runs = len(one_layer_history.history[\"val_factorized_top_k/top_100_categorical_accuracy\"])\n",
    "epochs = [i for i in range(10)]\n",
    "\n",
    "plt.plot(epochs, model_hist.history[\"factorized_top_k/top_100_categorical_accuracy\"], label=\"accuracy\")\n",
    "plt.title(\"Accuracy vs epoch\")\n",
    "plt.xlabel(\"epoch\")\n",
    "plt.ylabel(\"Top-100 accuracy\");\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7eba79e",
   "metadata": {},
   "source": [
    "Notice that accuracy is pretty bad, but if we plot the number of accuracy vs epoch, we can see that it improves higher as more iterations go through."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4a52f0d5",
   "metadata": {},
   "source": [
    "To predict top recommended items for specific users, say, here, user 40, we use brute force layers take considers all available items for them. To speed up the calculation, refer to efficient serving: https://www.tensorflow.org/recommenders/examples/efficient_serving"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "38716fe4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Recommendations for user 40: [b'pet_shop SKU 25' b'cool_stuff SKU 25' b'bed_bath_table SKU 520'\n",
      " b'costruction_tools_tools SKU 1' b'bed_bath_table SKU 252'\n",
      " b'bed_bath_table SKU 230' b'computers_accessories SKU 127'\n",
      " b'furniture_decor SKU 87' b'watches_gifts SKU 265' b'electronics SKU 68']\n"
     ]
    }
   ],
   "source": [
    "# Create a model that takes in raw query features, and\n",
    "index = tfrs.layers.factorized_top_k.BruteForce(model.user_model)\n",
    "# recommends movies out of the entire movies dataset.\n",
    "index.index(items.batch(100).map(model.item_model), items)\n",
    "\n",
    "# Get recommendations.\n",
    "j = str(40)\n",
    "_, titles = index(tf.constant([j]))\n",
    "print(f\"Recommendations for user %s: {titles[0]}\" %(j))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4981023",
   "metadata": {},
   "source": [
    "There you are, our first simple yet effective recommendation engine using retrieval task. But what about ranking? can we rank all the items for best to worst, only then run retrieval task to retrieve selected items from the short list? Now we can explore another type of recommendation task: ranking."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python (tensorflow 2.0)",
   "language": "python",
   "name": "tensorflow2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}