Skip to content

Generative AI Examples is a collection of GenAI examples such as ChatQnA, Copilot, which illustrate the pipeline capabilities of the Open Platform for Enterprise AI (OPEA) project.

License

Notifications You must be signed in to change notification settings

sharanshirodkar7/GenAIExamples

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Generative AI Examples

version license


Introduction

GenAIComps-based Generative AI examples offer streamlined deployment, testing, and scalability. All examples are fully compatible with Docker and Kubernetes, supporting a wide range of hardware platforms such as Gaudi, Xeon, and other hardwares.

Architecture

GenAIComps is a service-based tool that includes microservice components such as llm, embedding, reranking, and so on. Using these components, various examples in GenAIExample can be constructed, including ChatQnA, DocSum, etc.

GenAIInfra, part of the OPEA containerization and cloud-native suite, enables quick and efficient deployment of GenAIExamples in the cloud.

GenAIEval measures service performance metrics such as throughput, latency, and accuracy for GenAIExamples. This feature helps users compare performance across various hardware configurations easily.

Getting Started

GenAIExamples offers flexible deployment options that cater to different user needs, enabling efficient use and deployment in various environments. Here’s a brief overview of the three primary methods: Python startup, Docker Compose, and Kubernetes.

Users can choose the most suitable approach based on ease of setup, scalability needs, and the environment in which they are operating.

Deployment Guide

Deployment are based on released docker images by default, check docker image list for detailed information. You can also build your own images following instructions.

Prerequisite

  • For Docker Compose based deployment, you should have docker compose installed. Refer to docker compose install.
  • For Kubernetes based deployment, we provide 3 ways from the easiest manifests to powerful GMC based deployment.
    • You should have a kubernetes cluster ready for use. If not, you can refer to k8s install to deploy one.
    • (Optional) You should have GMC installed to your kubernetes cluster if you want to try with GMC. Refer to GMC install for more information.
    • (Optional) You should have Helm (version >= 3.15) installed if you want to deploy with Helm Charts. Refer to the Helm Installation Guide for more information.

Deploy Examples

Use Case Docker Compose
Deployment on Xeon
Docker Compose
Deployment on Gaudi
Kubernetes with GMC Kubernetes with Manifests Kubernetes with Helm Charts
ChatQnA Xeon Instructions Gaudi Instructions ChatQnA with GMC ChatQnA with Manifests ChatQnA with Helm Charts
CodeGen Xeon Instructions Gaudi Instructions CodeGen with GMC CodeGen with Manifests CodeGen with Helm Charts
CodeTrans Xeon Instructions Gaudi Instructions CodeTrans with GMC CodeTrans with Manifests CodeTrans with Helm Charts
DocSum Xeon Instructions Gaudi Instructions DocSum with GMC DocSum with Manifests DocSum with Helm Charts
SearchQnA Xeon Instructions Gaudi Instructions SearchQnA with GMC Not Supported Not Supported
FaqGen Xeon Instructions Gaudi Instructions FaqGen with GMC FaqGen with Manifests Not Supported
Translation Xeon Instructions Gaudi Instructions Translation with GMC Not Supported Not Supported
AudioQnA Xeon Instructions Gaudi Instructions AudioQnA with GMC AudioQnA with Manifests Not Supported
VisualQnA Xeon Instructions Gaudi Instructions VisualQnA with GMC VisualQnA with Manifests Not Supported

Supported Examples

Check here for detailed information of supported examples, models, hardwares, etc.

Additional Content

About

Generative AI Examples is a collection of GenAI examples such as ChatQnA, Copilot, which illustrate the pipeline capabilities of the Open Platform for Enterprise AI (OPEA) project.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Svelte 31.6%
  • Shell 29.1%
  • TypeScript 23.0%
  • SCSS 4.4%
  • Python 4.2%
  • JavaScript 3.8%
  • Other 3.9%