Promptmeteo is a Promt Engineer Python library build over LangChain that simplfies the use of LLMs for different tasks with a low-code interface. For doing so, Promptmeteo is able to use different LLMs model and mcreate the prompts dinamically for concrete task underneath, given just some configuration parameters.
TL;DR: Industrialize projects powered by LLMs easily.
LLMs are able to solve many tasks given a concrete instruction as input (prompt) and they can be used as a "reasoning engine" to build applications. However, this applications are very difficult to deploy and insdustrialize because two reasons. First because prompts usually include application logic in their definition, however, they are actually treated as a input argument. The latter mean that a bad prompt input can break the application.
Secondly, writing the concrete prompt for each task is not only a tedious work but also difficult. Slights changes in the input prompt can become in different results which make them very error-prone. Moreover when writting the prompt we do not only should take into consideration the task, but also the LLM that is going to use it, the model capacity...
TL;DR: Treating prompts an code equally!!!
Prompmeteo try to solve the problems mentioned before by sepaarating the prompt definition in two parts: the task-logic (which is coded in prompt templates) and the concrete-problem (which is included as argument variables). Prompmeteo include high level objects for different tasks which are programmed with .py and .prompt files.
The project includes high-level objects to solve different NLP tasks such as: text classification, Named Entity Recognition, code generation... This object only require configuration parameters to run and return the expected output from the task (i.e. we do not require to parse the output from the LLM).
The modules from Promptmeteo follow a similar model interface as a Scikit-Learn. Defining a interface with independant methods for training and predicting as well as saving and loading the model, allows Promptmeteo to be trained in and independant pipeline from predicting. This allows to reuse the conventional ML pipeline for LLM projects.
LLMs can improve the results by including examples in their prompt. Promptmeteo is able to be trained with examples and to ensure reproducibility, the training process can be stored as a binary model artifact. This allows to store the training results and reuse it many times in new data. The training process store the embeddings from the input text in a vector database such as FAISS.
Prompmeteo include the integration of different LLMs throught LangChain. This includes models that can be executed locally as well as remote API calls from providers such as OpenAI and HuggingFace.
Defining a concrete format when creating the prompts in Promptmeteo (.prompt), does not only allow to use it easily in a programatic way, but it also allows to versionate the prompts, understand where is the change when something happends and also define code test oriented to prompt testing. This testing includes aspects such as: validate the use of the language, that the size of the prompt is appropiate for the model,...
TEMPLATE:
"I need you to help me with a text classification task.
{__PROMPT_DOMAIN__}
{__PROMPT_LABELS__}
{__CHAIN_THOUGHT__}
{__ANSWER_FORMAT__}"
PROMPT_DOMAIN:
"The texts you will be processing are from the {__DOMAIN__} domain."
PROMPT_LABELS:
"I want you to classify the texts into one of the following categories:
{__LABELS__}."
PROMPT_DETAIL:
""
CHAIN_THOUGHT:
"Please provide a step-by-step argument for your answer, explain why you
believe your final choice is justified."
ANSWER_FORMAT:
"In your response, include only the name of the class as a single word, in
lowercase, without punctuation, and without adding any other statements or
words."
First of all, do not forget to configure providers credentials. Refer to the configure credentials section.
You can make a prediccion directly indanciating the model and calling the method predict().
from promptmeteo import DocumentClassifier
clf = DocumentClassifier(
language = 'en',
model_provider_name = 'hf_pipeline',
model_name = 'google/flan-t5-small',
prompt_labels = ['positive', 'neutral', 'negative']
)
clf.predict(['so cool!!'])[['positive']]Buy you can also include examples to improve the results by calling the method train()
clf = clf.train(
examples = ['i am happy', 'doesnt matter', 'I hate it'],
annotations = ['positive', 'neutral', 'negative'],
)
clf.predict(['so cool!!'])[['positive']]One the model is trained it can be save locally
clf.save_model("hello_world.prompt")and loaded again to make new predictions
from promptmeteo import DocumentClassifier
clf = DocumentClassifier(
language = 'en',
model_provider_name = 'hf_pipeline',
model_name = 'google/flan-t5-small',
).load_model("hello_world.prompt")
clf.predict(['so cool!!'])[['positive']]More examples can be seen in the directory examples.
Create a .env with the following variables depending on the LLM provider
First you should create a Service Account with the role: Vertex AI User.
Once created, generate a key, store it locally and reference the path in the .env file:
GOOGLE_CLOUD_PROJECT_ID="MY_GOOGLE_LLM_PROJECT_ID"
GOOGLE_APPLICATION_CREDENTIALS="PATH_TO_SERVICE_ACCOUNT_KEY_FILE.json"Create your Secret API key in your User settings page.
Indicate the value of the key in your .env file:
OPENAI_API_KEY="MY_OPENAI_API_KEY"You can also pass openai_api_key as a named parameter.
Create Access Token in your User settings page.
HUGGINGFACEHUB_API_TOKEN="MY_HF_API_KEY"You can also pass huggingfacehub_api_token as a named parameter.
make setupdocker build -t promptmeteo:latest .
docker run --rm -i -t -v .:/home promptmeteo:latestmake test
The current available tasks in Promptmeteo are:
| task_type | description |
|---|---|
DocumentQA |
Document-level question answering |
DocumentClassifier |
Document-level classification |
CodeGenerator |
Code generation |
The current available model_name and language values are:
| model_provider | model_name | language |
|---|---|---|
| openai | text-davinci-003 | es |
| openai | text-davinci-003 | en |
| hf_hub_api | google/flan-t5-xxl | es |
| hf_hub_api | google/flan-t5-xxl | en |
| hf_pipeline | google/flan-t5-small | es |
| hf_pipeline | google/flan-t5-small | en |
| text-bison | es | |
| text-bison | en | |
| text-bison@001 | es | |
| text-bison@001 | en | |
| text-bison-32k | es | |
| text-bison-32k | en |