In lesson 5 we put all the pieces of training together to understand exactly what is going on when we talk about back propagation. We'll use this knowledge to create and train a simple neural network from scratch.
We'll also see how we can look inside the weights of an embedding layer, to find out what our model has learned about our categorical variables. This will let us get some insights into which movies we should probably avoid at all costs…
Although embeddings are most widely known in the context of word embeddings for NLP, they are at least as important for categorical variables in general, such as for tabular data or collaborative filtering. They can even be used with non-neural models with great success.
- Lesson notes - thanks to @PoonamV
- Detailed lesson notes - thanks to @hiromi
- Notebooks:
- Excel spreadsheets:
- collab_filter.xlsx; Google Sheets full version; To run solver, please use Google Sheets short-cut version and follow instruction by @Moody
- graddesc: Excel version ; Google sheets version
- entropy_example.xlsx
- Lesson 5 in-class discussion thread
- Lesson 5 advanced discussion
- Links to different parts in video by @melonkernel
- NY Times Article - Finally, a Machine That Can Finish Your Sentence
- Netflix and Chill: Building a Recommendation System in Excel - Latent Factor Visualization in Excel blog post
- An overview of gradient descent optimization algorithms - Sebastian Ruder