Table of Contents
This project is a reference pattern for deploying an Enterprise Search Assistant based on Retrieval-Augmented Generation (RAG) with Large Language Models (LLMs), built on top of Red Hat OpenShift AI. By integrating a dynamic retrieval mechanism, it enables the chosen LLM to pull relevant information from extensive data sources stored in a vector databse, enhancing response accuracy and relevance. This approach not only broadens the model's knowledge base beyond its initial training but also ensures that its outputs are informed by the most current data available as the vector database contains a specialised index of enterprise documents which provide 'context' to the LLM allowing more accurate results but also allowing users to refer to source documents.
This project uses validated patterns to allow consistent deployment across multiple platforms. The project itself was initially forked off of the core mutlicloud-gitops pattern.
This is an example of how to list things you need to use the software and how to install them.
-
An OpenShift Cluster with
- Cluster admin rights logged in with
oc
where the repository is cloned out - Connectivity through to GitHub (or an equivalent git platform)
- GPUs on at least one node within the cluster.
- Cluster admin rights logged in with
-
The current demonstration relies on
flash-attention
to decrease memory consumption for the LLM models. In order to run end-to-end demonstrations on this pattern, it is therefore recommended to have access to specific Nvidia GPUs which provide this capability, including:- Nvidia L40S
- Nvidia A100
- Nvidia H100/H200
The deployment method for validated pattenrs is described here.
Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature
) - Commit your Changes (
git commit -m 'Add some AmazingFeature'
) - Push to the Branch (
git push origin feature/AmazingFeature
) - Open a Pull Request
Distributed under the Apache-2.0 License. See LICENSE
for more information.