The models need to be placed and mounted in a particular directory structure and according to the following rules:

models/
├── model1
│   ├── 1
│   │   ├── ir_model.bin
│   │   └── ir_model.xml
│   └── 2
│       ├── ir_model.bin
│       └── ir_model.xml
├── model2
│   └── 1
│       ├── ir_model.bin
│       ├── ir_model.xml
│       └── mapping_config.json
├── model3
│    └── 1
│        └── model.onnx
├── model4
│      └── 1
│        ├── model.pdiparams
│        └── model.pdmodel
└── model5
       └── 1
         └── TF_fronzen_model.pb

• Each model should be stored in a dedicated directory, e.g. model1 and model2.

• Each model directory should include a sub-folder for each of its versions (1,2, etc). The versions and their folder names should be positive integer values.

• Note: In execution, the versions are enabled according to a pre-defined version policy. If the client does not specify the version number in parameters, by default, the latest version is served.

• Every version folder must include model files, that is, .bin and .xml for OpenVINO IR, .onnx for ONNX, .pdiparams and .pdmodel for Paddle Paddle, and .pb for TensorFlow. The file name can be arbitrary.

The required Model Server parameters are listed below. For additional configuration options, see the Model Server Parameters section.

–rm	remove the container when exiting the Docker container
-d	runs the container in the background
-v	defines how to mount the model folder in the Docker container
-p	exposes the model serving port outside the Docker container
openvino/model_server:latest	represents the image name; the ovms binary is the Docker entry point varies by tag and build process - see tags: https://hub.docker.com/r/openvino/model_server/tags/ for a full tag list.
–model_path	model location, which can be: a Docker container path that is mounted during start-up a Google Cloud Storage path gs://<bucket>/<model_path> an AWS S3 path s3://<bucket>/<model_path> an Azure blob path az://<container>/<model_path>
–model_name	the name of the model in the model_path
–port	the gRPC server port
–rest_port	the REST server port

If the serving port 9000 is already in use, please switch it to another avaiable port on your system.