Documentation | Website | Slack
Agenta is a LLM developer platform that helps teams quickly build and refine reliable LLM applications.
Agenta is end-to-end, it provides all the tools around the LLMOps worfklow: From building (LLM playground, automatic and human evaluation) to deploying (prompt and configuration management) up to monitoring (LLM Observability and tracing)
- Prompt Playground
- Custom Workflows
- LLM evaluation
- Human evaluation
- Prompt Management
- LLM Tracing
- LLM Monitoring
- Strong focus on enabling collaboration between developers and subject matter experts. Subject matter experts are first class citizens. This means a strong playground for prompt engineering
- Strong focus on the prompt engineering workflow: we are working to enable the best playground to iterate quickly on the prompts
- Strong focus on evaluation: Our evalution workflow is best in class. It comes with many evaluators out of the box and strong comparison views
- Open-telemetry native Observability SDK: means ...
The easiest way to get started is through Agenta Cloud. It is free to signup, does not require credit card, and comes with a generous free-tier.
mkdir agenta && cd agenta
curl -L https://raw.githubusercontent.com/agenta-ai/agenta/main/docker-compose.gh.yml -o docker-compose.gh.yml
docker compose -f docker-compose.gh.yml up -d
- 🧪 Experiment and compare prompts on any LLM workflow (chain-of-prompts, Retrieval Augmented Generation (RAG), LLM agents...)
- ✍️ Collect and annotate golden test sets for evaluation
- 📈 Evaluate your application with pre-existing or custom evaluators
- 🔍 Annotate and A/B test your applications with human feedback
- 🤝 Collaborate with product teams for prompt engineering and evaluation
- 🚀 Deploy your application in one-click in the UI, through CLI, or through github workflows.
| Playground | Evaluation |
|---|---|
Compare and version prompts for any LLM app, from single prompt to agents. playground_1024_30.07.2023.mp4 |
Define test sets, then evaluate manually or programmatically your different variants.new_eval_demo.mp4 |
| Human annotation | Deployment |
Use Human annotator to A/B test and score your LLM apps.  |
When you are ready, deploy your LLM applications as APIs in one click. |
Contact us here for enterprise support and early access to agenta self-managed enterprise with Kubernetes support.
By default, Agenta automatically reports anonymized basic usage statistics. This helps us understand how Agenta is used and track its overall usage and growth. This data does not include any sensitive information.
To disable anonymized telemetry, follow these steps:
- For web: Set
TELEMETRY_TRACKING_ENABLEDtofalsein youragenta-web/.envfile. - For CLI: Set
telemetry_tracking_enabledtofalsein your~/.agenta/config.tomlfile.
After making this change, restart Agenta Compose.
We warmly welcome contributions to Agenta. Feel free to submit issues, fork the repository, and send pull requests.
We are usually hanging in our Slack. Feel free to join our Slack and ask us anything
Check out our Contributing Guide for more information.
Thanks goes to these wonderful people (emoji key):
This project follows the all-contributors specification. Contributions of any kind are welcome!
Attribution: Testing icons created by Freepik - Flaticon