Skip to content

Commit

Permalink
Docs added
Browse files Browse the repository at this point in the history
  • Loading branch information
snova-jorgep committed May 29, 2024
1 parent 856cd3d commit e6f445b
Showing 1 changed file with 161 additions and 0 deletions.
161 changes: 161 additions & 0 deletions code_copilot/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@

<a href="https://sambanova.ai/">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="../images/SambaNova-light-logo-1.png" height="60">
<img alt="SambaNova logo" src="../images/SambaNova-dark-logo-1.png" height="60">
</picture>
</a>

Code Copilot
======================

<!-- TOC -->

- [Code Copilot](#code-copilot)
- [Overview](#overview)
- [Before you begin](#before-you-begin)
- [Set up the account and config file for the LLM](#set-up-the-account-and-config-file-for-the-llm)
- [Setup for SambaStudio users](#setup-for-sambastudio-users)
- [Setup for Sambaverse users](#setup-for-sambaverse-users)
- [Get Continue Extension/plugin](#get-continue-extensionplugin)
- [Setting up](#setting-up)
- [Setup Continue](#setup-continue)
- [Set custom Sambanova integration](#set-custom-sambanova-integration)
- [Usage](#usage)
- [Ask question to the LLM in your IDE](#ask-question-to-the-llm-in-your-ide)
- [Ask about selected code](#ask-about-selected-code)
- [Edit code](#edit-code)
- [Understand terminal errors](#understand-terminal-errors)
- [custom commands](#custom-commands)
- [Customizing the connector](#customizing-the-connector)
- [modify the model to use](#modify-the-model-to-use)
- [add custom commands](#add-custom-commands)
- [Acknowledgments](#acknowledgments)

<!-- /TOC -->

# Overview

This AI Starter Kit is a demonstration of how to use Sambanova's models as coding assistant leveraging the [Continue](https://www.continue.dev/) extension for VSCode and JetBrains. The Kit includes:

- A configurable SambaStudio - Continue connector. The connector generates answers from a deployed LLM.

- A configurable Sambaverse - Continue connector. The connector generates answers from a sambaverse hosted model.

- An installation, setup, and usage guide.

# Before you begin

In this starter kit, you need access to an Sambaverse account or a SambaStudio Environment.
> You can use a LLM of your choice, either from Sambaverse or from SambaStudio, but is highly recommended to use Meta-Llama-3-8B-Instruct
## Set up the account and config file for the LLM

The next step sets you up to use one of the models available from SambaNova. It depends on whether you're a SambaNova customer who uses SambaStudio or you want to use the publicly available Sambaverse.

### Setup for SambaStudio users

To perform this setup, you must be a SambaNova customer with a SambaStudio account.

1. Log in to SambaStudio and get your API authorization key. The steps for getting this key are described [here](https://docs.sambanova.ai/sambastudio/latest/cli-setup.html#_acquire_the_api_key).

2. Select the LLM to use (e,g. CoE1.1 with Llama 3 8B instruct) and deploy an endpoint for inference. See the [SambaStudio endpoint documentation](https://docs.sambanova.ai/sambastudio/latest/endpoints.html).

### Setup for Sambaverse users

1. Create a Sambaverse account at [Sambaverse](sambaverse.sambanova.net) and select your model.

2. Get your [Sambaverse API key](https://docs.sambanova.ai/sambaverse/latest/use-sambaverse.html#_your_api_key) (from the user button).

## Get Continue Extension/plugin

you will need to install the [Continue](https://www.continue.dev/) [VSCode](https://marketplace.visualstudio.com/items?itemName=Continue.continue) or [JetBrains](https://plugins.jetbrains.com/plugin/22707-continue) Extension. *this can be done directly from the extensions/plugins section of you IDE searching **continue***

# Setting up

## Setup Continue
After installing Continue you will need to do the basic setup

- First you will be prompted to select a model to use you can skip this step and close the ***Continue** extension

## Set custom Sambanova integration

After the basic installation it is needed to set the custom SambaStudio and Sambaverse ***Continue*** connectors

First you should modify the ***Continue*** config.ts file

- Open the config.ts file in `~/.continue/` folder and replace the contents with the contents of the [config.ts](config.ts) kit provided file.

- If you are using Sambaverse: replace the sambaverse_api_key variable with your previously generated Sambaverse api key

For example, for an api key `123456ab-cdef-0123-4567-890abcdef` update the first section in the config.ts file as:

```ts
//Sambaverse usage
const sambaverse_api_key = "123456ab-cdef-0123-4567-890abcdef"
```

- If you are using SambaStudio: replace the sambastudio_base_url, sambastudio_project_id, sambastudio_endpoint_id and sambaStudio_api_key variables with your SambaStudio endpoint keys:

For example, for an endpoint with the URL `https://api-stage.sambanova.net/api/predict/generic/stream/12345678-9abc-def0-1234-56789abcdef0/456789ab-cdef-0123-4567-89abcdef012` with api key `123456ab-cdef-0123-4567-890abcdef` update the first section in the config.ts file as:

```ts
//SambaStudio usage
const sambastudio_base_url = "https://api-stage.sambanova.net";
const sambastudio_project_id = "12345678-9abc-def0-1234-56789abcdef0";
const sambastudio_endpoint_id = "456789ab-cdef-0123-4567-89abcdef012";
const sambastudio_api_key = "123456ab-cdef-0123-4567-890abcdef"
```

Ten you should set the ***Continue*** config.json file. press+l this will open a new ***Continue*** session, Click in the gearbottom right button, the the json file will open, replace the contents of this file with the contents of the [config.json](config.json) kit provided file.

# Usage

## Ask question to the LLM in your IDE

You can interact with the LLM directly using+l this will open a ***Continue*** session in a new extension window then ask anything to the to the model!

## Ask about selected code

You can ask information of selected code to the LLM directly selecting your text and then pressing+l this will open a ***Continue*** session in a new extension window with the code snippet as context, then ask anything related with your code snippet!

## Edit code

You can ask your LLM to modify your code, ad functionalities, documentation etc, over a selected code snippet, for this first select the code snipped you want the model to modify, then press+i, this will open an input bar in the top of your IDE, then write your desired changes and press enter, this will generate the modified code for you and you can edit it, accept or reject the proposed changes.

## Understand terminal errors

you can ask the model to inspect your terminal error outputs to explain you the error and give you some suggestions, for this after getting an error in your terminal only press+shit+r, this will open a ***Continue*** session in a new extension window with the error explanation!

## Custom commands

you can execute your custom commands/prompts selecting a code snippet an then pressing+l to open a new ***Continue*** session then write `/<yourCommand>` to generate, see how to create your custom commands [here](#add-custom-commands)

> See more about ***Continue*** extension usage [here](https://docs.continue.dev/how-to-use-continue)

# Customizing the connector

The example template can be further customized based on the use case.

## Modify the parameters and the model to use

you can change the default Sambaverse model or the default model used with SambaStudio CoE models, and some model parameters modifying the `options` and the `body` params of the Sambaverse model and Sambastudio model in the `SambastudioModel` an `SambaverseModel` definitions of [config.ts file](config.ts)

## Add custom commands

You can add your custom commands adding them to the [config.json file](config.json)

A custom command should have the following structure

```json
{
"name": "yourCommand",
"prompt": "{{{ input }}} \n\n custom prompt",
"description": "Write unit tests for highlighted code "
}
```

# Acknowledgments

This kit work aims to show the integration of SambaNova Systems RDU acceleration models, The work here has been leveraged and adapted from the [Continue](https://docs.continue.dev/) documentation. The original docs can be found here:
[https://docs.continue.dev/](https://docs.continue.dev/)

0 comments on commit e6f445b

Please sign in to comment.