You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardexpand all lines: demos/README.md
+32-33
Original file line number
Diff line number
Diff line change
@@ -1,39 +1,38 @@
1
1
# Demos {#ovms_docs_demos}
2
2
3
-
@sphinxdirective
3
+
```{toctree}
4
+
---
5
+
maxdepth: 1
6
+
hidden:
7
+
---
4
8
5
-
.. toctree::
6
-
:maxdepth: 1
7
-
:hidden:
8
-
9
-
ovms_demo_age_gender_guide
10
-
ovms_demo_horizontal_text_detection
11
-
ovms_demo_optical_character_recognition
12
-
ovms_demo_face_detection
13
-
ovms_demo_face_blur_pipeline
14
-
ovms_demo_capi_inference_demo
15
-
ovms_demo_single_face_analysis_pipeline
16
-
ovms_demo_multi_faces_analysis_pipeline
17
-
ovms_docs_demo_ensemble
18
-
ovms_docs_demo_mediapipe_image_classification
19
-
ovms_docs_demo_mediapipe_multi_model
20
-
ovms_docs_demo_mediapipe_object_detection
21
-
ovms_docs_demo_mediapipe_holistic
22
-
ovms_docs_image_classification
23
-
ovms_demo_using_onnx_model
24
-
ovms_demo_tf_classification
25
-
ovms_demo_person_vehicle_bike_detection
26
-
ovms_demo_vehicle_analysis_pipeline
27
-
ovms_demo_real_time_stream_analysis
28
-
ovms_demo_bert
29
-
ovms_demo_gptj_causal_lm
30
-
ovms_demo_llama_2_chat
31
-
ovms_demo_stable_diffusion
32
-
ovms_demo_universal-sentence-encoder
33
-
ovms_demo_speech_recognition
34
-
ovms_demo_benchmark_client
35
-
36
-
@endsphinxdirective
9
+
ovms_demo_age_gender_guide
10
+
ovms_demo_horizontal_text_detection
11
+
ovms_demo_optical_character_recognition
12
+
ovms_demo_face_detection
13
+
ovms_demo_face_blur_pipeline
14
+
ovms_demo_capi_inference_demo
15
+
ovms_demo_single_face_analysis_pipeline
16
+
ovms_demo_multi_faces_analysis_pipeline
17
+
ovms_docs_demo_ensemble
18
+
ovms_docs_demo_mediapipe_image_classification
19
+
ovms_docs_demo_mediapipe_multi_model
20
+
ovms_docs_demo_mediapipe_object_detection
21
+
ovms_docs_demo_mediapipe_holistic
22
+
ovms_docs_image_classification
23
+
ovms_demo_using_onnx_model
24
+
ovms_demo_tf_classification
25
+
ovms_demo_person_vehicle_bike_detection
26
+
ovms_demo_vehicle_analysis_pipeline
27
+
ovms_demo_real_time_stream_analysis
28
+
ovms_demo_bert
29
+
ovms_demo_gptj_causal_lm
30
+
ovms_demo_llama_2_chat
31
+
ovms_demo_stable_diffusion
32
+
ovms_demo_universal-sentence-encoder
33
+
ovms_demo_speech_recognition
34
+
ovms_demo_benchmark_client
35
+
```
37
36
38
37
OpenVINO Model Server demos have been created to showcase the usage of the model server as well as demonstrate it’s capabilities. Check out the list below to see complete step-by-step examples of using OpenVINO Model Server with real world use cases:
Copy file name to clipboardexpand all lines: docs/accelerators.md
+39-57
Original file line number
Diff line number
Diff line change
@@ -26,35 +26,26 @@ Before using GPU as OpenVINO Model Server target device, you need to:
26
26
27
27
Running inference on GPU requires the model server process security context account to have correct permissions. It must belong to the render group identified by the command:
28
28
29
-
@sphinxdirective
30
-
.. code-block:: sh
31
-
32
-
stat -c "group_name=%G group_id=%g" /dev/dri/render*
33
-
34
-
@endsphinxdirective
29
+
```bash
30
+
stat -c "group_name=%G group_id=%g" /dev/dri/render*
31
+
```
35
32
36
33
The default account in the docker image is preconfigured. If you change the security context, use the following command to start the model server container:
37
34
38
-
@sphinxdirective
39
-
.. code-block:: sh
40
-
41
-
docker run --rm -it --device=/dev/dri --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1) -u $(id -u):$(id -g) \
GPU device can be used also on Windows hosts with Windows Subsystem for Linux 2 (WSL2). In such scenario, there are needed extra docker parameters. See the command below.
48
42
Use device `/dev/dxg` instead of `/dev/dri` and mount the volume `/usr/lib/wsl`:
The `Auto Device` plugin can also use the [PERFORMANCE_HINT](performance_tuning.md) plugin config property that enables you to specify a performance mode for the plugin.
150
138
@@ -154,29 +142,23 @@ To enable Performance Hints for your application, use the following command:
154
142
155
143
LATENCY
156
144
157
-
@sphinxdirective
158
-
.. code-block:: sh
159
-
160
-
docker run --rm -d --device=/dev/dri --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1) -u $(id -u):$(id -g) \
Check also [building from sources](https://github.com/openvinotoolkit/model_server/blob/main/docs/build_from_source.md).
195
177
196
178
Example command to run container with NVIDIA support:
197
179
198
180
```bash
199
-
docker run -it --gpus all -p 9000:9000 -v ${PWD}/models/public/resnet-50-tf:/opt/model openvino/model_server:latest-cuda --model_path /opt/model --model_name resnet --port 9000 --target_device NVIDIA
181
+
docker run -it --gpus all -p 9000:9000 -v ${PWD}/models/public/resnet-50-tf:/opt/model openvino/model_server:latest-cuda --model_path /opt/model --model_name resnet --port 9000 --target_device NVIDIA
200
182
```
201
183
202
184
For models with layers not supported on NVIDIA plugin, you can use a virtual plugin `HETERO` which can use multiple devices listed after the colon:
203
185
```bash
204
-
docker run -it --gpus all -p 9000:9000 -v ${PWD}/models/public/resnet-50-tf:/opt/model openvino/model_server:latest-cuda --model_path /opt/model --model_name resnet --port 9000 --target_device HETERO:NVIDIA,CPU
186
+
docker run -it --gpus all -p 9000:9000 -v ${PWD}/models/public/resnet-50-tf:/opt/model openvino/model_server:latest-cuda --model_path /opt/model --model_name resnet --port 9000 --target_device HETERO:NVIDIA,CPU
205
187
```
206
188
207
189
Check the supported [configuration parameters](https://github.com/openvinotoolkit/openvino_contrib/tree/master/modules/nvidia_plugin#supported-configuration-parameters) and [supported layers](https://github.com/openvinotoolkit/openvino_contrib/tree/master/modules/nvidia_plugin#supported-layers-and-limitations)
Copy file name to clipboardexpand all lines: docs/binary_input.md
+10-11
Original file line number
Diff line number
Diff line change
@@ -1,17 +1,16 @@
1
1
# Support for Binary Encoded Image Input Data {#ovms_docs_binary_input}
2
2
3
-
@sphinxdirective
3
+
```{toctree}
4
+
---
5
+
maxdepth: 1
6
+
hidden:
7
+
---
4
8
5
-
.. toctree::
6
-
:maxdepth: 1
7
-
:hidden:
8
-
9
-
ovms_docs_binary_input_layout_and_shape
10
-
ovms_docs_binary_input_tfs
11
-
ovms_docs_binary_input_kfs
12
-
ovms_docs_demo_tensorflow_conversion
13
-
14
-
@endsphinxdirective
9
+
ovms_docs_binary_input_layout_and_shape
10
+
ovms_docs_binary_input_tfs
11
+
ovms_docs_binary_input_kfs
12
+
ovms_docs_demo_tensorflow_conversion
13
+
```
15
14
16
15
While OpenVINO models don't have the ability to process images directly in their binary format, the model server can accept them and convert
17
16
automatically from JPEG/PNG to OpenVINO friendly format using built-in [OpenCV](https://opencv.org/) library. To take advantage of this feature, there are two requirements:
0 commit comments