Project Overview

Operationalize a Machine Learning Microservice API.

Use a pre-trained, sklearn model that has been trained to predict housing prices in Boston according to several features, such as average rooms in a home and data about highway access, teacher-to-pupil ratios, and so on. You can read more about the data, which was initially taken from Kaggle, on the data source site.

Project Tasks

The project goal is to operationalize a working, machine learning microservice using kubernetes, which is an open-source system for automating the management of containerized applications.

• Test project code using linting
• Complete Dockerfile to containerize this application
• Deploy containerized application using Docker and make a prediction
• Improve the log statements in the source code for this application
• Configure Kubernetes and create a Kubernetes cluster
• Deploy a container using Kubernetes and make a prediction
• Upload a complete Github repo with CircleCI to indicate that the code has been tested

Setup and Run

You should have Python 3.7 available on your host.
Here is a tutorial to get it installed on Amazon Linux
Check the Python path using which python3

• Run python -m venv .devops to create a virtualenv
• Run source .devops/bin/activate to activate the virtualenv
• Run make install to install the necessary dependencies

You should also install other libraries and tools used for the project. Checkout the tutorials linked or official pages for the libraries and tools

• hadolint
  
• Docker
• kubectl
• minikube

• Run make lint to check the Dockerfile and app.py for possible issues and errors using hadolint and pylint

Running app.py

Verify the tools and libraries are installed

1 - Standalone: python app.py

2 - Run in Docker: ./run_docker.sh

• Run sudo systemctl status docker.service to verify docker is up and running
• Run ./run_docker.sh to build and start the app
• Run ./make_prediction.sh to make a prdiction

3 - Run in Kubernetes: ./run_kubernetes.sh

• Run kubectl version --client to verify kubectl is installed
• Run ./run_docker.sh if you haven't built the image
• Run ./upload_docker.sh to upload the image to your docker repository. You might be required to provide an access token for authentication
• Run minikube start
• Run minikube status to check the status of the cluster
• Run ./run_kubernetes.sh to deploy the app and port forward. In case you get an error when trying to port forward (the container might still be in the process of being created), wait a few minites and re-run the script
• Run ./make_prediction.sh to make a prdiction