Kamil Krzyk - My Road to AI

About

This is a repository that I have created to track my progress in AI/Data Science related topics in order to organise my knowledge and goals. Purpose of doing this is self-motivation, open source/study material for others, portfolio and TODO list.

Table of contents

• Kamil Krzyk - My Road to AI
• About
• Table of Contents
• AI Related Presentations
• AI Implementations
  • Machine Learning
  • Deep Learning
  • Tutorials
  • Based on Research Papers
• Algorithm Implementations
  • Divide & Conquer
• Books
• Courses & Certificates
• Sources
• Contact

AI Related Presentations

Presentation	Where	Date	Slides
Welcome to MOOC era! - My experiences with Deep Learning Foundations Nanodegree at Udacity	Speaker - GDG & Women Techmakers - Machine Learning #3	18.10.2017	Link
Soft introduction into MSE based Linear Regression (part 2 of 'What this Machine Learning is all about?' talk)	Azimo Lunch&Learn	16.11.2017	Link

AI Implementations

In this section I want to show off my knowledge about various AI related algorithms, frameworks, programming languages, libraries and more. Priority is to show how the algorithm works - not to solve complex and ambitious problems.

Machine Learning

Algorithm	Description	Implementation	Dataset	Creation Date	Last Update
Linear Regression	-	Python (raw)	Generated Numbers	18.04.2017	15.09.2017
	-	Python (sklearn)	Generated Numbers	18.04.2017	15.09.2017
	-	Python (Tensorflow)	Generated Numbers	23.09.2017	23.09.2017
Ridge Regression	Compared result with Linear Regression	Python (raw)	Generated Numbers	23.09.2017	23.09.2017
Polynomial Regression	Approximating Polynomial of degree 2	Python (raw)	Generated Numbers	08.06.2017	15.09.2017
	Approximating Polynomial of degree 2	Python (sklearn)	Generated Numbers	10.06.2017	15.09.2017
	Approximating Polynomial of degree 3	Python (raw)	Generated Numbers	10.06.2017	15.09.2017
	Approximating Polynomial of degree 3	Python (sklearn)	Generated Numbers	10.06.2017	15.09.2017
Logistic Regression	Data Analysis, Kaggle Competition	Python (raw)	Titanic Disaster	19.10.2017	24.10.2017
KNN	Manhattan, Euclidean Similarity	Python (raw)	iris	21.07.2017	24.09.2017
	Euclidean Similarity	Python (sklearn)	iris	22.07.2017	24.09.2017
PCA	-	Python (raw)	Generated Numbers	01.04.2017	23.09.2017
K-Means Clusters	3-dimensional data	Python (raw)	Video Game Sales from Kaggle	01.10.2017	05.10.2017
Naive Bayes	Gaussian Distribution	Python (raw)	Pima Indian Diabetes	02.11.2017	03.11.2017
Lasso Regression	-	-	-	-	-
SVM	-	-	-	-	-
Decision Tree	-	-	-	-	-
Random Forest	-	-	-	-	-

Deep Learning

Multilayer Perceptron

Problem	Description	Implementation	Dataset	Creation Date	Last Update
Digit Classification	2-layers, mini-batch	Python (raw)	MNIST	19.06.2017	14.08.2017
Digit Classification	2-layers, mini-batch, dropout-regularization	Python (Tensorflow)	MNIST	29.06.2017	18.07.2017
Digit Classification	2-layers, mini-batch	Python (Tensorflow + Keras)	MNIST	08.07.2017	18.07.2017
Digit Classification	2-layers, mini-batch	Python (tflearn)	MNIST	21.06.2017	21.06.2017
Digit Classification	2-layers, mini-batch	Python (Keras)	MNIST	18.07.2017	18.07.2017
Prediction of Bike Shop Clients Number	1-layer, mini-batch	Python (numpy, matplotlib)	Bike-Sharing	13.08.2017	13.08.2017
Encrypting data with Autoencoder	1-layer Encoder, 1-layer Decoder, mini-batch	Python (Tensorflow)	MNIST	13.07.2017	13.07.2017
Detecting Text Sentiment	-	-	IMDb	-	-

Convolutional Neural Net

Problem	Description	Implementation	Dataset	Creation Date	Last Update
Digit Classification	tf.layer module, dropout regularization, batch normalization	Python (Tensorflow)	MNIST	16.08.2017	23.08.2017
10 Classes Color Images Classification	tf.nn module, dropout regularization	Python (Tensorflow)	CIFAR-10	16.08.2017	07.09.2017
10 Classes Color Images Classification	tf.layer module, dropout regularization	Python (Tensorflow)	CIFAR-10	16.08.2017	09.09.2017
10 Classes Color Images Classification	tf.layer module, dropout regularization, batch normalization	Python (Tensorflow)	CIFAR-10	19.08.2017	10.09.2017

Recurrent Neural Network

Problem	Description	Implementation	Dataset	Creation Date	Last Update
Simple Language Translator	In form of my DLFND project for now	Python (Tensorflow)	Small part of French-English corpus	05.05.2017	24.05.2017
"The Simpsons" Script Generation	In form of my DLFND project for now	Python (Tensorflow)	"The Simpsons" script	06.06.2017	14.07.2017

Generative Adversarial Neural Network

Problem	Description	Implementation	Dataset	Creation Date	Last Update
Generating Human Face Miniatures	DCGAN	Python (Tensorflow)	CelebA	11.09.2017	13.09.2017

Tutorials

Teaching others is best way of teaching yourself. I will try to create tutorials with various implementations of ML&DL models and more. Idea of my tutorials is to build models with small steps, with many comments, ideally including math and links to sources that I use to create them.

Tutorial	Creation Date	Last Update
Implementing KNN with comments and basic math	21.07.2017	21.07.2017
Implementing PCA with comments and basic math	01.04.2017	01.04.2017

Based on Research Papers

In this section I will do my best to provide implementations of models based on research papers. My target framework will be Keras or/and PyTorch.

Convolutional Neural Network

Paper	Year	Implementation	Dataset	Creation Date	Last Update
Deep Photo Style Transfer	2017	-	-	-	-
Spatial Transformer Networks - STN	2016	-	-	-	-
You Only Look Once: Unified, Real-Time Object Detection - YOLO	2016	-	-	-	-
Semantic Style Transfer and Turning Two-Bit Doodles into Fine Artwork	2016	-	-	-	-
Colorful Image Colorization	2016	-	-	-	-
Deep Residual Learning for Image Recognition - Microsoft-ResNet	2015	-	-	-	-
Image Super-Resolution Using Deep Convolutional Networks	2015	-	-	-	-
A Neural Algorithm of Artistic Style - GATYS	2015	-	-	-	-

Recurrent Neural Network

Paper	Year	Implementation	Dataset	Creation Date	Last Update
Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation	2016	-	-	-	-
Deep Speech 2: End-to-End Speech Recognition in English and Mandarin	2015	-	-	-	-
A Neural Conversational Model	2015	-	-	-	-
Skip-Thought Vectors	2015	-	-	-	-
Sequence to Sequence Learning with Neural Networks	2014	-	-	-	-
Generating Sequences With Recurrent Neural Networks	2013	-	-	-	-

Generative Adversarial Neural Network

Paper	Year	Implementation	Dataset	Creation Date	Last Update
Generative Adversarial Text to Image Synthesis	2016	-	-	-	-
Deep Convolutional GAN: DCGAN	2015	-	-	-	-

Algorithm Implementations

Divide & Conquer

• Gauss's Integer Multiplication
• Karatsuba Multiplication
• Inversion Counting
• Merge Sort
• Quick Sort

Books

Usually I prefer online sources for studying, but I believe in the power of books and try to fit them into my daily agenda.

Programming related:

Book	Author	Started	Finished
Dive Into Python 3	Mark Pilgrim	Aug 2017	Sep 2017
Automate the Boring Stuff with Python	Al Sweigart	Sept 2017	Oct 2017

Machine Learning related:

Book	Author	Started	Finished
Python Data Science Handbook	Jake VanderPlas	Nov 2017	-

Deep Learning related:

Book	Author	Started	Finished
Grokking Deep Learning	Andrew Trask	Nov 2017	-

Courses & Certificates

When I was younger I played a lot of computer games. I still tend to play today a little as a form of relax and to spend time with friends that live far from me. One thing that I have very enjoyed about gaming was gathering trophies. You made an effort to complete list of challenges or get a great score and then looked at list of your achievements with satisfaction. My current self have inherited this habit and as I study on daily basis I like to gather proves that I have done something - to make it more like a game where each topic is a boss that you have to clear on hard mode. Of course what's in your head is most important but if it helps to motivate you, then why not?

• Programming languages:

  • Programming for Everybody (Getting Started with Python) (Feb 2017) (Coursera - University of Michigan - Charles Severance)
  • Python Data Structures (Feb 2017) (Coursera - University of Michigan - Charles Severance)
  • Using Python to Access Web Data (Feb 2017) (Coursera - University of Michigan - Charles Severance)
  • Using Databases with Python (Feb 2017) (Coursera - University of Michigan - Charles Severance)

• Algorithms:

  • Divide and Conquer, Sorting and Searching, and Randomized Algorithms (Sep 2017) (Coursera - University of Stanford)

• AI related:

  • Machine Learning (Nov 2016 - Feb 2017) (Coursera - Stanford - Andrew Ng)
  • Deep Learning Nanodegree (Mar 2017 - Aug 2017) (Udacity - Siraj Raval, Mat Leonard, Brok Bucholtz + guest lessions by: Ian Goodfellow, Andrew Trask)
  • Neural Networks and Deep Learning (Oct 2017) (Coursera - deeplearning.ai - Andrew Ng)
  • Practical Machine Learning (Nov - Dec 2017) (DataWorkshop - Vladimir Alekseichenko)

Sources

There is a list of sources that I have used (and found helpful in some way) or keep using in order to produce my repo content.

• Courses

  • https://www.coursera.org/learn/machine-learning
  • https://www.udacity.com/course/deep-learning-nanodegree-foundation--nd101
  • https://www.coursera.org/specializations/deep-learning
  • https://www.coursera.org/specializations/algorithms
  • https://www.coursera.org/learn/python
  • http://www.dataworkshop.eu/ (PL only)

• Online Lectures

  • CS231n (Winter 2016 with Andrej Karpathy)
  • CS231n (Spring 2017)
  • Natural Language Processing with Deep Learning (Winter 2017)
  • Deep RL Bootcamp, 26-27 August 2017, Berkeley CA

• YouTube Channels

  • Siraj Raval
  • sentdex
  • Alena Kruchkova - Machine Learning Book Club

• Videos Worth Watching

  • Learn Tensorflow and Deep Learning without PhD
  • Anatomy of Matplotlib
  • Sequence to Sequence Deep Learning (Quoc Le, Google)
    • Visualising and Understanding Neural Networks by Matt Zeiler
    • Deploying Machine Learning using sklearn pipelines

• Podcasts

  • http://biznesmysli.pl/
  • https://dataskeptic.com/podcast

• Online Books

  • Neural Networks - Deep Learning online book
    • Part of book online - Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville

• Blogs

  • http://kldavenport.com/
  • https://medium.com/@karpathy
  • https://machinelearningmastery.com/
  • https://colah.github.io/
  • https://distill.pub/

• Articles

  • Neural Networks - RNN
    • Understanding-LSTMs

• Cheatsheets

  • https://becominghuman.ai/cheat-sheets-for-ai-neural-networks-machine-learning-deep-learning-big-data-678c51b4b463

• Repositories

  • https://github.com/terryum/awesome-deep-learning-papers
  • https://github.com/songrotek/Deep-Learning-Papers-Reading-Roadmap
  • https://github.com/fchollet/deep-learning-with-python-notebooks
  • https://github.com/ZuzooVn/machine-learning-for-softwaxre-engineers
  • https://github.com/eriklindernoren/ML-From-Scratch
  • https://github.com/junyanz/CycleGAN
  • https://github.com/llSourcell

• Frameworks

  • Comparing Top Deep Learning Frameworks: Deeplearning4j, PyTorch, TensorFlow, Caffe, Keras, MxNet, Gluon & CNTK
  • http://scikit-learn.org/stable/
  • https://www.tensorflow.org/
  • https://keras.io/
  • http://pytorch.org/
  • https://docs.microsoft.com/en-us/cognitive-toolkit/
  • http://caffe.berkeleyvision.org/

• Cloud Computing

  • Google Colaboratory
  • Amazon Web Service
  • FloydHub

• Other

  • https://www.kaggle.com/
  • https://www.tensorflow.org/
  • https://www.quora.com/How-can-beginners-in-machine-learning-who-have-finished-their-MOOCs-in-machine-learning-and-deep-learning-take-it-to-the-next-level-and-get-to-the-point-of-being-able-to-read-research-papers-productively-contribute-in-an-industry/answer/Andrew-Ng?share=c26bd326

Contact

• Twitter: @F1sherKK
• Medium: @krzyk.kamil
• E-mail: [email protected]
• Android related open-source contributions: AzimoLabs
• LinkedIn