Skip to content

Commit

Permalink
updated custom_service doc and added index to examples README
Browse files Browse the repository at this point in the history
  • Loading branch information
harshbafna committed Feb 27, 2020
1 parent dddf4d4 commit dfaef5c
Show file tree
Hide file tree
Showing 2 changed files with 23 additions and 10 deletions.
5 changes: 3 additions & 2 deletions docs/custom_service.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,8 +94,6 @@ class ModelHandler(object):
return ["OK"] * self._batch_size

def handle(self, data, context):
ts

model_input = self.preprocess(data)
model_out = self.inference(model_input)
return self.postprocess(model_out)
Expand All @@ -119,6 +117,9 @@ Here the ``` handle()``` method is our entry point that will be invoked by Torch

This entry point is engaged in two cases: (1) when TorchServe is asked to scale a model up, to increase the number of backend workers (it is done either via a ```PUT /models/{model_name}``` request or a ```POST /models``` request with `initial-workers` option or during TorchServe startup when you use `--models` option (```torchserve --start --models {model_name=model.mar}```), ie., you provide model(s) to load) or (2) when TorchServe gets a ```POST /predictions/{model_name}``` request. (1) is used to scale-up or scale-down workers for a model. (2) is used as a standard way to run inference against a model. (1) is also known as model load time, and that is where you would normally want to put code for model initialization. You can find out more about these and other TorchServe APIs in [TorchServe Management API](./management_api.md) and [TorchServe Inference API](./inference_api.md)

** For a working example of a custom service handler refer [mnist digit classifier handler](../examples/image_classifier/mnist/mnist_handler.py) **


## Creating model archive with entry point

TorchServe, identifies the entry point to the custom service, from the manifest file. Thus file creating the model archive, one needs to mention the entry point using the ```--handler``` option.
Expand Down
28 changes: 20 additions & 8 deletions examples/README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,18 @@
# Contents of this Document
* [Creating mar file for an eager mode model](#creating-mar-file-for-eager-mode-model)
* [Creating mar file for an eager mode model](#creating-mar-file-for-torchscript-mode-model)
* [Serving torchvision image classification models in TorchServe](#serving-torchvision-image-classification-models-in-torchserve)
* [Serving Densenet161 model](#serving-torchvision-image-classification-models)
* [Serving Resnet18 model](#example-to-serve-resnet18-image-classification-model)
* [Serving Text Classification model](#example-to-serve-text-classification-model)
* [Serving Object Detection model](#example-to-serve-object-detection-model)
* [Serving Image Segmentation model](#example-to-serve-image-segmentation-model)

# TorchServe Examples

The following are examples on how to create and serve model archives with TorchServe.

#### Eager Mode
## Creating mar file for eager mode model

Following are the steps to create a torch-model-archive (.mar) to execute an eager mode torch model in TorchServe :

Expand All @@ -18,7 +28,7 @@ Following are the steps to create a torch-model-archive (.mar) to execute an eag
torch-model-archiver --model-name <model_name> --version <model_version_number> --model-file <path_to_model_architecture_file> --serialized-file <path_to_state_dict_file> --handler <path_to_custom_handler_or_default_handler_name> --extra-files <path_to_index_to_name_json_file>
```

#### TorchScript Mode
## Creating mar file for torchscript mode model

Following are the steps to create a torch-model-archive (.mar) to execute an eager mode torch model in TorchServe :

Expand All @@ -34,7 +44,9 @@ Following are the steps to create a torch-model-archive (.mar) to execute an eag
torch-model-archiver --model-name <model_name> --version <model_version_number> --serialized-file <path_to_executable_script_module> --extra-files <path_to_index_to_name_json_file> --handler <path_to_custom_handler_or_default_handler_name>
```
#### Eager Mode example using torchvision image classifiers:
## Serving torchvision image classification models
Following example demonstrates serving an image classification models available in torchvision using TorchServe's default image_classifier handler

* TorchVision Image Classification Models : Download a pre-trained model state_dict for computer vision model that classifies images from the following :

Expand Down Expand Up @@ -99,30 +111,30 @@ traced_script_module.save("dense161.pt")
curl -X POST http://127.0.0.1:8080/predictions/densenet161 -T examples/image_classifier/kitten.jpg
```

#### TorchScript example using Resnet18 image classifier:
## Example to serve Resnet18 image classification model
Following example demonstrates how to create a Resnet18 image classifier model archive, serve it on TorchServe and run image prediction using TorchServe's default image_classifier handler :
* [Image classification using Resnet18](image_classifier/resnet_18)
#### TorchScript example using custom model and custom handler:
## Example to serve a Custom Model with Custom Service Handler
Following example demonstrates how to create and serve a custom NN model with custom handler archives in TorchServe :
* [Digit recognition with MNIST](image_classifier/mnist)
#### Text Classification Example
## Example to serve Text classification model
Following example demonstrates how to create and serve a custom text_classification NN model with default text_classifer handler provided by TorchServe :
* [Text classification example](text_classification)
#### Object Detection Example
## Example to serve Object Detection model
Following example demonstrates how to create and serve a pretrained fast-rcnn NN model with default object_detector handler provided by TorchServe :
* [Object detection example](object_detector)
#### Image Segmentation Example
## Example to serve Image Segmentation model
Following example demonstrates how to create and serve a pretrained fcn NN model with default image_segmenter handler provided by TorchServe :
Expand Down

0 comments on commit dfaef5c

Please sign in to comment.