KServe

This doc is for kserve beginners who just start to learn so it would have a detail explanation. Each topic will be presented in 10mins. The presentation will be the "simple question & answer" style.

Topic

• Overview KServe
  • What is KServe?
    • https://devocean.sk.com/blog/techBoardDetail.do?ID=163645
  • Why we need KServe?
  • What kind of components are in the KServe?
  • Why KServe need Knative?
  • What network strategies you can choos for KNative?
• ModelMesh Overview
  • What is ModelMesh?
  • What we need ModelMesh?

Installation

Openshift cluster

• CRC

Upstream KServe

• RawDeployment mode
  • KServe(rawDeployment mode) + CertManager(serving certificate)
• KServe + ModelMesh
• ISTIO + BookInfo
• Serverless mode
  • KServe + Serverless(Knative) + ISTIO + CertManager(serving certificate)
  • KServe + Serverless(Knative) + Courier + CertManager(serving certificate)

Reference

Source

• kserve contants.go
• kserve beta api Deployment ways
• RawDeployment
  • https://kserve.github.io/website/admin/kubernetes_deployment/
• Serverless(Knative) + ISTIO
  • https://docs.openshift.com/container-platform/4.11/serverless/admin_guide/serverless-ossm-setup.html
• Serverless(knative) + kourier
  • https://docs.openshift.com/container-platform/4.11/serverless/admin_guide/serverless-ossm-setup.html
• ModelMesh
  • https://kserve.github.io/website/0.9/modelserving/mms/modelmesh/overview/#model-serving-runtimes

Serving Type (https://velog.io/@ifelifelse/Model-Serving)

• Online Serving
• Batch Serving

Examples https://github.com/kserve/kserve/tree/master/docs/samples

SDK를 이용하여 배포하기 https://kangwoo.kr/category/machine-learning/kubeflow/serving/kfserving/

http api https://github.com/kserve/kserve/blob/master/docs/predict-api/v2/required_api.md#httprest

openvino https://blog.openvino.ai/blog-posts/kserve-api