forked from julep-ai/julep
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
6c10cf8
commit ee76924
Showing
21 changed files
with
3,365 additions
and
2,813 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,73 @@ | ||
--- | ||
description: Documents to be added for Retrieval Augmented Generation | ||
--- | ||
|
||
# 📖 Documents | ||
|
||
A typical RAG application has the following components: | ||
|
||
1. Chunking | ||
2. Storing | ||
3. Retrieval | ||
4. Generation | ||
|
||
Julep offers a pre-built RAG pipeline out of the box. You can specify data sources scoped to an agent or a user. | ||
|
||
## Adding Documents | ||
|
||
### While creating agents/users | ||
|
||
{% code overflow="wrap" %} | ||
```python | ||
docs = [ | ||
{ | ||
"title": "Computer Scientists Invent an Efficient New Way to Count", | ||
"content": """In a recent paper, computer scientists have described a new way to approximate the number of distinct entries in a long list...""", | ||
"metadata": {"page": 1}, | ||
}, | ||
{ | ||
"title": "Computer Scientists Invent an Efficient New Way to Count", | ||
"metadata": {"page": 2}, | ||
"content": """The CVM algorithm, named for its creators — Sourav Chakraborty of the Indian Statistical Institute, Vinodchandran Variyam of the University of Nebraska, Lincoln, and Kuldeep Meel of the University of Toronto ...""", | ||
}, | ||
] | ||
``` | ||
{% endcode %} | ||
|
||
Docs can be scoped to agents or users directly. | ||
|
||
```python | ||
client.agents.create( | ||
name="Computer Scientist", | ||
model="gpt-4-turbo", | ||
docs=docs | ||
) | ||
``` | ||
|
||
> Useful for scenarios where an agent needs to have more context about private data or specific topic that needs to be available to all users. | ||
```python | ||
client.users.create( | ||
name="Anon", | ||
docs=docs | ||
) | ||
``` | ||
|
||
> Useful for scenarios where each user has a different persona, documentation. | ||
### Using \`docs.create\` | ||
|
||
Docs can also be added to an agent/user ad-hoc. | ||
|
||
```python | ||
client.docs.create( | ||
agent_id=agent.id, | ||
# user_id=user.id, | ||
doc={ | ||
"title": "Good and Bad Procrastination", | ||
"metadata": {"chunk": 1}, | ||
"content": """The most impressive people I know are all terrible procrastinators. So could it be that procrastination isn't always bad? Most people who write about procrastination write about how to cure it. But this is, strictly speaking, impossible. There are an infinite number of things you could be doing. No matter what you work on, you're not working on everything else. So the question is not how to avoid procrastination, but how to procrastinate well.""", | ||
}, | ||
) | ||
``` | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.