SambaNova AI Starter Kits are a collection of open-source examples and guides designed to facilitate the deployment of AI-driven use cases for both developers and enterprises.
To run these examples, you can obtain a free API key using SambaNova Cloud. Alternatively, if you are a current SambaNova customer, you can deploy your models using SambaStudio. Most of the code examples are written in Python, although the concepts can be applied to any programming language.
Questions? Just message us on SambaNova Community or create an issue in GitHub. We're happy to help live!
The table belows lists the available kits, which are grouped into four categories: 1) Data Ingestion & Preparation, 2) Model Development & Optimization, 3) Intelligent Information Retrieval, and 4) Advanced AI Capabilities.
Note: For each kit, we specify whether it is compatible with SambaNova Cloud, SambaStudio, or both.
Name | Kit Description | Compatible APIs | Category |
---|---|---|---|
Data Extraction | Series of notebooks that demonstrate methods for extracting text from documents in different input formats. | SambaNova Cloud, SambaStudio | Data Ingestion & Preparation |
YoDA: Your Data Your model | Sample training recipe to train a Language Model (LLM) using a customer's private data. | SambaStudio | Data Ingestion & Preparation |
Fine tuning embeddings | Example workflow for fine-tuning embeddings from unstructured data, leveraging Large Language Models (LLMs) and open-source embedding models to enhance NLP task performance. | SambaStudio | Model Development & Optimization |
Fine tuning SQL | Example workflow for fine-tuning an SQL model for Question-Answering purposes, leveraging Large Language Models (LLMs) and open-source embedding models to enhance SQL generation task performance. | SambaStudio | Model Development & Optimization |
Prompt Engineering | Starting point demo for prompt engineering using SambaNova's API to experiment with different use case templates. Provides useful resources to improve prompt crafting, making it an ideal entry point for those new to this AISK. | SambaNova Cloud, SambaStudio | Model Development & Optimization |
Enterprise Knowledge Retrieval | Sample implementation of the semantic search workflow using the SambaNova platform to get answers to questions about your documents. Includes a runnable demo. | SambaNova Cloud, SambaStudio | Intelligent Information Retrieval |
Image Search | This example workflow shows a simple approach to image search by image description or image similarity. All workflows are built using the SambaNova platform. | SambaStudio | Intelligent Information Retrieval |
Multimodal Knowledge Retriever | Sample implementation of the semantic search workflow leveraging the SambaNova platform to get answers using text, tables, and images to questions about your documents. Includes a runnable demo. | SambaNova Cloud, SambaStudio | Intelligent Information Retrieval |
Post Call Analysis | Example workflow that shows a systematic approach to post-call analysis including Automatic Speech Recognition (ASR), diarization, large language model analysis, and retrieval augmented generation (RAG) workflows. All workflows are built using the SambaNova platform. | SambaNova Cloud, SambaStudio | Intelligent Information Retrieval |
RAG Evaluation Kit | A tool for evaluating the performance of LLM APIs using the RAG Evaluation methodology. | SambaStudio | Intelligent Information Retrieval |
Search Assistant | Sample implementation of the semantic search workflow built using the SambaNova platform to get answers to your questions using search engine snippets, and website crawled information as the source. Includes a runnable demo. | SambaNova Cloud, SambaStudio | Intelligent Information Retrieval |
Web Crawled Data Retrieval | Sample implementation of a semantic search workflow built using the SambaNova platform to get answers to your questions using website crawled information as the source. Includes a runnable demo. | SambaNova Cloud, SambaStudio | Intelligent Information Retrieval |
Benchmarking | This kit evaluates the performance of multiple LLM models hosted in SambaStudio. It offers various performance metrics and configuration options. Users can also see these metrics within a chat interface. | SambaNova Cloud, SambaStudio | Advanced AI Capabilities |
Code Copilot | This example guide shows a simple integration with Continue VSCode and JetBrains extension using SambaNova platforms, to use Sambanova's hosted models as your custom coding assistant. | SambaStudio | Advanced AI Capabilities |
Bundle jump start | This kit demonstrates how to call SambaNova Bundle models using the Langchain framework. The script offers different approaches for calling Bundle models, including using SambaStudio with a named expert, and using SambaStudio with routing. | SambaStudio | Advanced AI Capabilities |
Financial Assistant | This app demonstrates the capabilities of LLMs in extracting and analyzing financial data using function calling, web scraping, and RAG. | SambaNova Cloud, SambaStudio | Advanced AI Capabilities |
Function Calling | Example of tools calling implementation and a generic function calling module that can be used inside your application workflows. | SambaNova Cloud, SambaStudio | Advanced AI Capabilities |
SambaNova Scribe | Example implementation of a transcription and summarization workflow. | SambaNova Cloud | Advanced AI Capabilities |
SambaCloud - Google Integration | App Scripts intended for those with SambaCloud API keys to integrate LLMs into Google Workspaces. | SambaNova Cloud | Advanced AI Capabilities |
Currently, there are two ways to obtain an API key from SambaNova. You can get a free API key using SambaNova Cloud. Alternatively, if you are a current SambaNova customer, you can deploy your models using SambaStudio.
For more information and to obtain your API key, visit the SambaNova Cloud webpage.
To integrate SambaNova Cloud LLMs with this AI starter kit, update the API information by configuring the environment variables in the ai-starter-kit/.env
file:
- Create the
.env
file atai-starter-kit/.env
if the file does not exist. - Enter the SambaNova Cloud API key in the
.env
file, for example:
SAMBANOVA_API_KEY = "456789abcdef0123456789abcdef0123"
Begin by deploying your LLM of choice (e.g., Llama 3 8B) to an endpoint for inference in SambaStudio. Use either the GUI or CLI, as described in the SambaStudio endpoint documentation.
To integrate your LLM deployed on SambaStudio with this AI starter kit, update the API information by configuring the environment variables in the ai-starter-kit/.env
file:
- Create the
.env
file atai-starter-kit/.env
if the file does not exist. - Set your SambaStudio variables. For example, an endpoint with the URL
"https://api-stage.sambanova.net/api/predict/nlp/12345678-9abc-def0-1234-56789abcdef0/456789ab-cdef-0123-4567-89abcdef0123"
is entered in the
.env
file as:
SAMBASTUDIO_URL="https://api-stage.sambanova.net/api/predict/nlp/12345678-9abc-def0-1234-56789abcdef0/456789ab-cdef-0123-4567-89abcdef0123"
SAMBASTUDIO_API_KEY="89abcdef-0123-4567-89ab-cdef01234567"
Currently, you can set your embedding models on CPU or SambaStudio. Note that embedding models are not available yet through SambaNova Cloud, but they will be in future releases.
You can run the Hugging Face embedding models locally on CPU. In this case, no information is needed in the .env
file.
Alternatively, you can use SambaStudio embedding model endpoints instead of the CPU-based HugginFace embeddings to increase inference speed. Please follow this guide to deploy your SambaStudio embedding model.
To integrate your embedding model deployed on SambaStudio with this AI starter kit, update the API information by configuring the environment variables in the ai-starter-kit/.env
file:
- Create the
.env
file atai-starter-kit/.env
if the file does not exist. - Set your SambaStudio variables. For example, an endpoint with the URL
"https://api-stage.sambanova.net/api/predict/generic/12345678-9abc-def0-1234-56789abcdef0/456789ab-cdef-0123-4567-89abcdef0123"
is entered in the.env
file as:
SAMBASTUDIO_EMBEDDINGS_BASE_URL="https://api-stage.sambanova.net"
SAMBASTUDIO_EMBEDDINGS_BASE_URI="api/predict/generic"
SAMBASTUDIO_EMBEDDINGS_PROJECT_ID="12345678-9abc-def0-1234-56789abcdef0"
SAMBASTUDIO_EMBEDDINGS_ENDPOINT_ID="456789ab-cdef-0123-4567-89abcdef0123"
SAMBASTUDIO_EMBEDDINGS_API_KEY="89abcdef-0123-4567-89ab-cdef01234567"
Go to the README.md
of the starter kit you want to use and follow the instructions. See Available AI Starter Kits.
Use Sambanova's LLMs and Langchain wrappers
Set your environment as shown in integrate your model.
- Import the SambaStudio langchain community wrapper in your project and define your *SambaStudio LLM:
- If using a Bundle endpoint:
from langchain_community.llms.sambanova import SambaStudio
load_dotenv('.env')
llm = SambaStudio(
model_kwargs={
"do_sample": False,
"max_tokens_to_generate": 512,
"temperature": 0.0,
"select_expert": "Meta-Llama-3-8B-Instruct",
"process_prompt": "False"
},
)
- If using a single model endpoint
from langchain_community.llms.sambanova import SambaStudio
load_dotenv('.env')
llm = SambaStudio(
model_kwargs={
"do_sample": False,
"max_tokens_to_generate": 512,
"temperature": 0.0,
"process_prompt": "False"
},
)
- Use the model
llm.invoke("your prompt")
See utils/usage.ipynb for an example.
- Import our SambaNovaCloud langchain internal wrapper in your project and define your SambaNovaCloud LLM:
from util..model_wrappers.llms.langchain_llms import SambaNovaCloud
load_dotenv('.env')
llm = SambaNovaCloud(model='Meta-Llama-3.1-70B-Instruct')
- Use the model
llm.invoke("your prompt")
See utils/usage.ipynb for an example.
- Import the SambaStudioEmbedding langchain community wrapper in your project and define your SambaStudioEmbeddings embedding:
- If using a Bundle endpoint
from langchain_community.embeddings import SambaStudioEmbeddings
load_dotenv('.env')
embedding = SambaStudioEmbeddings(
batch_size=1,
model_kwargs = {
"select_expert":e5-mistral-7b-instruct
}
)
- If using a single embedding model endpoint
from langchain_community.embeddings import SambaStudioEmbeddings
load_dotenv('.env')
embedding = SambaStudioEmbeddings(batch_size=32)
Note that using different embedding models (cpu or sambastudio) may change the results, and change the way they are set and their parameters
- Use your embedding model in your langchain pipeline
See utils/usage.ipynb for an example.
-
Before running the code, ensure that you have Node.js installed on your system. You can download the latest version from the official Node.js website.
-
Set Up the Environment. To set up the environment, run the following commands in your terminal:
npm init -y
npm install @langchain/openai @langchain/core
These commands will create a new package.json file and install the required dependencies.
- Create a new file named
app.js
and add the following code:
import { ChatOpenAI } from "@langchain/openai";
const SambaNovaCloudBaseURL = "https://api.sambanova.ai/v1";
const apiKey = "your-api-key";
const SambaNovaCloudChatModel = new ChatOpenAI({
temperature: 0.9,
model: "Meta-Llama-3.1-70B-Instruct",
configuration: {
baseURL: SambaNovaCloudBaseURL,
apiKey: apiKey,
},
});
const response = await SambaNovaCloudChatModel.invoke("Hi there, tell me a joke!");
console.log(response.content);
- To run the app, execute the following command in your terminal:
node app.js
Setting up your virtual environment
There are two approaches to setting up your virtual environment for the AI Starter Kits:
- Individual Kit Setup (Traditional Method)
- Base Environment Setup (WIP)
Each starter kit has its own README.md
and requirements.txt
file. You can set up a separate virtual environment for each kit by following the instructions in their respective directories. This method is suitable if you're only interested in running a single kit or prefer isolated environments for each project.
To use this method:
- Navigate to the specific kit's directory
- Create a virtual environment
- Install the requirements
- Follow the kit-specific instructions
For users who plan to work with multiple kits or prefer a unified development environment, we recommend setting up a base environment. This approach uses a Makefile to automate the setup of a consistent Python environment that works across all kits.
Benefits of the base environment approach:
- Consistent Python version across all kits
- Centralized dependency management
- Simplified setup process
- Easier switching between different kits
- pyenv: The Makefile will attempt to install pyenv if it's not already installed.
- Docker: (Optional) If you want to use the Docker-based setup, ensure Docker is installed on your system.
- Installs pyenv and Poetry if they are not already installed.
- Sets up a Python virtual environment using a specified Python version (default is 3.11.3).
- Installs all necessary dependencies for the base environment.
- Sets up the parsing service required by some kits.
- Installs system dependencies like Tesseract OCR and Poppler.
- Provides Docker-based setup options for consistent environments across different systems.
- Install and Set Up the Base Environment:
make all
This command will set up the base ai-starter-kit environment, including installing all necessary tools and dependencies.
- Activate the Base Environment:
source .venv/bin/activate
- Navigate to Your Chosen Starter Kit:
cd path/to/starter_kit
Within the starter kit there will be instructions on how to start the kit. You can skip the virtual environment creation part in the kits README.md as we've done it here.
For certain kits, we utilise a standard parsing service. By Default it's started automatically with the base environment. To work with this service in isolation, following the steps in this section.
- Start Parsing Service:
make start-parsing-service
- Stop Parsing Service:
make stop-parsing-service
- Check Parsing Service Status:
make parsing-status
- View Parsing Service Logs:
make parsing-log
To use the Docker-based setup:
- Ensure Docker is installed on your system.
- Build the Docker image:
make docker-build
- Run a specific kit in the Docker container:
make docker-run-kit KIT=<kit_name>
Replace <kit_name>
with the name of the starter kit you want to run (e.g., function_calling
).
- To open a shell in the Docker container:
make docker-shell
To clean up all virtual environments created by the makefile and stop parsing services run the following command:
make clean
This command removes all virtual environments created with the makefile, stops the parsing service, and cleans up any temporary files.
Troubleshooting
If you encounter issues while setting up or running the AI Starter Kit, here are some common problems and their solutions:
If you're having problems with Python versions:
- Ensure you have pyenv installed:
make ensure-pyenv
- Install the required Python versions:
make install-python-versions
- If issues persist, check your system's Python installation and PATH settings.
If you're experiencing dependency conflicts:
- Try cleaning your environment:
make clean
- Update the lock file:
poetry lock --no-update
- Reinstall dependencies:
make install
If you encounter an error while installing pikepdf
, such as:
ERROR: Failed building wheel for pikepdf
Failed to build pikepdf
This is likely due to missing qpdf
dependency. The Makefile should automatically install qpdf
for you, but if you're still encountering issues:
- Ensure you have proper permissions to install system packages.
- If you're on macOS, you can manually install
qpdf
using Homebrew:brew install qpdf
- On Linux, you can install it using your package manager, e.g., on Ubuntu:
sudo apt-get update && sudo apt-get install -y qpdf
- After installing
qpdf
, try runningmake install
again.
If you continue to face issues, please ensure your system meets all the requirements for building pikepdf
and consider checking the pikepdf documentation for more detailed installation instructions.
If the parsing service isn't starting or is behaving unexpectedly:
- Check its status:
make parsing-status
- View its logs:
make parsing-log
- Try stopping and restarting it:
make stop-parsing-service
followed bymake start-parsing-service
If you encounter issues related to Tesseract OCR or Poppler:
- Ensure the Makefile has successfully installed these dependencies.
- On macOS, you can manually install them using Homebrew:
brew install tesseract poppler
- On Linux (Ubuntu/Debian), you can install them manually:
sudo apt-get update && sudo apt-get install -y tesseract-ocr poppler-utils
- On Windows, you may need to install these dependencies manually and ensure they are in your system PATH.
If you're using the Docker-based setup and encounter issues:
- Ensure Docker is properly installed and running on your system.
- Try rebuilding the Docker image:
make docker-build
- Check Docker logs for any error messages.
- Ensure your firewall or antivirus is not blocking Docker operations.
- Ensure all prerequisites (Python, pyenv, Poetry) are correctly installed.
- Try cleaning and rebuilding the environment:
make clean all
- Check for any error messages in the console output and address them specifically.
- Ensure your
.env
file is correctly set up in the ai-starter-kit root with all necessary environment variables.
If you continue to experience issues, please open an issue with details about your environment, the full error message, and steps to reproduce the problem.
- Ensure you have sufficient permissions to install software on your system.
- The setup process may take several minutes, especially when installing Python versions or large dependencies.
- If you encounter any issues during setup, check the error messages and ensure your system meets all prerequisites.
- Always activate the base environment before navigating to and running a specific starter kit.
- Some kits may require additional setup steps. Always refer to the specific README of the kit you're using.
Note: These AI Starter Kit code samples are provided "as-is," and are not production-ready or supported code. Bugfix/support will be on a best-effort basis only. Code may use third-party open-source software. You are responsible for performing due diligence per your organization policies for use in your applications.