Merge pull request graviraja#11 from graviraja/week7

Week7

Author: graviraja

.dvc/config

@@ -1,4 +1,6 @@
 [core]
-    remote = storage
+    remote = model-store
 ['remote "storage"']
     url = gdrive://19JK5AFbqOBlrFVwDHjTrf9uvQFtS0954
+['remote "model-store"']
+    url = s3://models-dvc/trained_models/

.github/workflows/build_docker_image.yaml

@@ -7,14 +7,29 @@ jobs:
     runs-on: ubuntu-latest
     defaults:
       run:
-        working-directory: ./week_6_github_actions
+        working-directory: ./week_7_ecr
     steps:
     - name: Checkout
       uses: actions/checkout@v2
       with:
         ref: ${{ github.ref }}
+    - name: Configure AWS Credentials
+      uses: aws-actions/configure-aws-credentials@v1
+      with:
+        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
+        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+        aws-region: us-west-2
     - name: Build container
       run: |
-        docker network create data
-        docker build --tag inference:latest .
-        docker run -d -p 8000:8000 --network data --name inference_container inference:latest
+        docker build --build-arg AWS_ACCOUNT_ID=${{ secrets.AWS_ACCOUNT_ID }} \
+                     --build-arg AWS_ACCESS_KEY_ID=${{ secrets.AWS_ACCESS_KEY_ID }} \
+                     --build-arg AWS_SECRET_ACCESS_KEY=${{ secrets.AWS_SECRET_ACCESS_KEY }} \
+                     --tag mlops-basics .
+    - name: Push2ECR
+      id: ecr
+      uses: jwalton/gh-ecr-push@v1
+      with:
+        access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
+        secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+        region: us-west-2
+        image: mlops-basics:latest

README.md

@@ -208,3 +208,28 @@ References
 - [Configuring service account](https://dvc.org/doc/user-guide/setup-google-drive-remote)
 
 - [Github actions](https://docs.github.com/en/actions/quickstart)
+
+
+## Week 7: Container Registry - AWS ECR
+
+<img src="https://img.shields.io/static/v1.svg?style=for-the-badge&label=difficulty&message=medium&color=orange"/>
+
+Refer to the [Blog Post here](https://www.ravirajag.dev/blog/mlops-container-registry)
+
+A container registry is a place to store container images. A container image is a file comprised of multiple layers which can execute applications in a single instance. Hosting all the images in one stored location allows users to commit, identify and pull images when needed.
+
+Amazon Simple Storage Service (S3) is a storage for the internet. It is designed for large-capacity, low-cost storage provision across multiple geographical regions.
+
+In this week, I will be going through the following topics:
+
+- `Basics of S3`
+
+- `Programmatic access to S3`
+
+- `Configuring AWS S3 as remote storage in DVC`
+
+- `Basics of ECR`
+
+- `Configuring GitHub Actions to use S3, ECR`
+
+![Docker](images/ecr_flow.png)

images/ecr_flow.png

@@ -0,0 +1,33 @@
+FROM huggingface/transformers-pytorch-cpu:latest
+
+COPY ./ /app
+WORKDIR /app
+
+ARG AWS_ACCESS_KEY_ID
+ARG AWS_SECRET_ACCESS_KEY
+
+
+#this envs are experimental
+ENV AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID \
+    AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
+
+
+# install requirements
+RUN pip install "dvc[s3]"
+RUN pip install -r requirements_inference.txt
+
+# initialise dvc
+RUN dvc init --no-scm
+# configuring remote server in dvc
+RUN dvc remote add -d model-store s3://models-dvc/trained_models/
+
+RUN cat .dvc/config
+# pulling the trained model
+RUN dvc pull dvcfiles/trained_model.dvc
+
+ENV LC_ALL=C.UTF-8
+ENV LANG=C.UTF-8
+
+# running the application
+EXPOSE 8000
+CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]

week_7_ecr/Dockerfile

@@ -0,0 +1,169 @@
+
+**Note: The purpose of the project to explore the libraries and learn how to use them. Not to build a SOTA model.**
+
+## Requirements:
+
+This project uses Python 3.8
+
+Create a virtual env with the following command:
+
+```
+conda create --name project-setup python=3.8
+conda activate project-setup
+```
+
+Install the requirements:
+
+```
+pip install -r requirements.txt
+```
+
+## Running
+
+### Training
+
+After installing the requirements, in order to train the model simply run:
+
+```
+python train.py
+```
+
+### Monitoring
+
+Once the training is completed in the end of the logs you will see something like:
+
+```
+wandb: Synced 5 W&B file(s), 4 media file(s), 3 artifact file(s) and 0 other file(s)
+wandb:
+wandb: Synced proud-mountain-77: https://wandb.ai/raviraja/MLOps%20Basics/runs/3vp1twdc
+```
+
+Follow the link to see the wandb dashboard which contains all the plots.
+
+### Versioning data
+
+Refer to the blog: [DVC Configuration](https://www.ravirajag.dev/blog/mlops-dvc)
+
+### Exporting model to ONNX
+
+Once the model is trained, convert the model using the following command:
+
+```
+python convert_model_to_onnx.py
+```
+
+### Inference
+
+#### Inference using standard pytorch
+
+```
+python inference.py
+```
+
+#### Inference using ONNX Runtime
+
+```
+python inference_onnx.py
+```
+
+## S3 & ECR
+
+Follow the instructions mentioned in the [blog post](https://www.ravirajag.dev/blog/mlops-container-registry) for creating S3 bucket and ECR repository. 
+
+### Configuring dvc
+
+```
+dvc init (this has to be done at root folder)
+dvc remote add -d model-store s3://models-dvc/trained_models/
+```
+
+### AWS credentials
+
+Create the credentials as mentioned in the [blog post](https://www.ravirajag.dev/blog/mlops-container-registry)
+
+**Do not share the secrets with others**
+
+Set the ACCESS key and id values in environment variables.
+
+```
+export AWS_ACCESS_KEY_ID=<ACCESS KEY ID>
+export AWS_SECRET_ACCESS_KEY=<ACCESS SECRET>
+```
+
+### Trained model in DVC
+
+Sdd the trained model(onnx) to dvc using the following command:
+
+```shell
+cd dvcfiles
+dvc add ../models/model.onnx --file trained_model.dvc
+```
+
+Push the model to remote storage
+
+```shell
+dvc push trained_model.dvc
+```
+
+### Docker
+
+Install the docker using the [instructions here](https://docs.docker.com/engine/install/)
+
+Build the image using the command
+
+```shell
+docker build -t mlops-basics:latest .
+```
+
+Then run the container using the command
+
+```shell
+docker run -p 8000:8000 --name inference_container mlops-basics:latest
+```
+
+(or)
+
+Build and run the container using the command
+
+```shell
+docker-compose up
+```
+
+### Pushing the image to ECR
+
+Follow the instructions mentioned in [blog post](https://www.ravirajag.dev/blog/mlops-container-registry) for creating ECR repository.
+
+- Authenticating docker client to ECR
+
+```
+aws ecr get-login-password --region us-west-2 | docker login --username AWS --password-stdin 246113150184.dkr.ecr.us-west-2.amazonaws.com
+```
+
+- Tagging the image
+
+```
+docker tag mlops-basics:latest 246113150184.dkr.ecr.us-west-2.amazonaws.com/mlops-basics:latest
+```
+
+- Pushing the image
+
+```
+docker push 246113150184.dkr.ecr.us-west-2.amazonaws.com/mlops-basics:latest
+```
+
+Refer to `.github/workflows/build_docker_image.yaml` file for automatically creating the docker image with trained model and pushing it to ECR.
+
+
+### Running notebooks
+
+I am using [Jupyter lab](https://jupyter.org/install) to run the notebooks.
+
+Since I am using a virtualenv, when I run the command `jupyter lab` it might or might not use the virtualenv.
+
+To make sure to use the virutalenv, run the following commands before running `jupyter lab`
+
+```
+conda install ipykernel
+python -m ipykernel install --user --name project-setup
+pip install ipywidgets
+```

week_7_ecr/README.md

@@ -0,0 +1,15 @@
+from fastapi import FastAPI
+from inference_onnx import ColaONNXPredictor
+app = FastAPI(title="MLOps Basics App")
+
+predictor = ColaONNXPredictor("./models/model.onnx")
+
+@app.get("/")
+async def home_page():
+    return "<h2>Sample prediction API</h2>"
+
+
+@app.get("/predict")
+async def get_prediction(text: str):
+    result =  predictor.predict(text)
+    return result

week_7_ecr/app.py

@@ -0,0 +1,6 @@
+defaults:
+  - model: default
+  - processing: default
+  - training: default
+  - override hydra/job_logging: colorlog
+  - override hydra/hydra_logging: colorlog

week_7_ecr/configs/config.yaml

@@ -0,0 +1,2 @@
+name: google/bert_uncased_L-2_H-128_A-2             # model used for training the classifier
+tokenizer: google/bert_uncased_L-2_H-128_A-2        # tokenizer used for processing the data

week_7_ecr/configs/model/default.yaml

@@ -0,0 +1,2 @@
+batch_size: 64
+max_length: 128

week_7_ecr/configs/processing/default.yaml

@@ -0,0 +1,5 @@
+max_epochs: 1
+log_every_n_steps: 10
+deterministic: true
+limit_train_batches: 0.25
+limit_val_batches: ${training.limit_train_batches}

week_7_ecr/configs/training/default.yaml

@@ -0,0 +1,57 @@
+import torch
+import hydra
+import logging
+
+from omegaconf.omegaconf import OmegaConf
+
+from model import ColaModel
+from data import DataModule
+
+logger = logging.getLogger(__name__)
+
+
+@hydra.main(config_path="./configs", config_name="config")
+def convert_model(cfg):
+    root_dir = hydra.utils.get_original_cwd()
+    model_path = f"{root_dir}/models/best-checkpoint.ckpt"
+    logger.info(f"Loading pre-trained model from: {model_path}")
+    cola_model = ColaModel.load_from_checkpoint(model_path)
+
+    data_model = DataModule(
+        cfg.model.tokenizer, cfg.processing.batch_size, cfg.processing.max_length
+    )
+    data_model.prepare_data()
+    data_model.setup()
+    input_batch = next(iter(data_model.train_dataloader()))
+    input_sample = {
+        "input_ids": input_batch["input_ids"][0].unsqueeze(0),
+        "attention_mask": input_batch["attention_mask"][0].unsqueeze(0),
+    }
+
+    # Export the model
+    logger.info(f"Converting the model into ONNX format")
+    torch.onnx.export(
+        cola_model,  # model being run
+        (
+            input_sample["input_ids"],
+            input_sample["attention_mask"],
+        ),  # model input (or a tuple for multiple inputs)
+        f"{root_dir}/models/model.onnx",  # where to save the model (can be a file or file-like object)
+        export_params=True,
+        opset_version=10,
+        input_names=["input_ids", "attention_mask"],  # the model's input names
+        output_names=["output"],  # the model's output names
+        dynamic_axes={
+            "input_ids": {0: "batch_size"},  # variable length axes
+            "attention_mask": {0: "batch_size"},
+            "output": {0: "batch_size"},
+        },
+    )
+
+    logger.info(
+        f"Model converted successfully. ONNX format model is at: {root_dir}/models/model.onnx"
+    )
+
+
+if __name__ == "__main__":
+    convert_model()