Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/Agenta-AI/agenta
The all-in-one LLM developer platform: prompt management, evaluation, human feedback, and deployment all in one place.
https://github.com/Agenta-AI/agenta
human-annotation langchain large-language-models llama-index llm llm-evaluation llm-framework llm-tools llmops llms prompt-engineering prompt-management prompt-toolkit rag rag-evaluation
Last synced: 3 months ago
JSON representation
The all-in-one LLM developer platform: prompt management, evaluation, human feedback, and deployment all in one place.
- Host: GitHub
- URL: https://github.com/Agenta-AI/agenta
- Owner: Agenta-AI
- License: mit
- Created: 2023-04-26T09:54:28.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-04-14T12:08:37.000Z (10 months ago)
- Last Synced: 2024-04-14T12:19:35.971Z (10 months ago)
- Topics: human-annotation, langchain, large-language-models, llama-index, llm, llm-evaluation, llm-framework, llm-tools, llmops, llms, prompt-engineering, prompt-management, prompt-toolkit, rag, rag-evaluation
- Language: Python
- Homepage: http://www.agenta.ai
- Size: 56.8 MB
- Stars: 799
- Watchers: 15
- Forks: 142
- Open Issues: 91
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Security: SECURITY.md
Awesome Lists containing this project
- awesome-llmops - agenta - AI/agenta.svg?style=flat-square) | (LLMOps / Observability)
- awesome-llm-eval - agenta - AI/agenta.svg?style=social) | 用于构建强大LLM应用的LLMOps平台。轻松尝试和评估不同提示、模型和工作流，以构建稳健的应用程序。 | (LLMOps / Popular-LLM)
- jimsghstars - Agenta-AI/agenta - The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM Observability all in one place. (Python)
- awesome - Agenta-AI/agenta - The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM Observability all in one place. (Python)
- awesome-generative-ai-data-scientist - Agenta - source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM Observability all in one place. [Documentation](https://docs.agenta.ai/) (LLMOps)
README
Home Page |
Slack |
Documentation
Collaborate on prompts, evaluate, and deploy LLM applications with confidence
The open-source LLM developer platform for prompt-engineering, evaluation, human feedback, and deployment of complex LLM apps.
---
Quick Start •
Features •
Documentation •
Enterprise •
Roadmap •
Join Our Slack •
Contributing
---
# ⭐️ Why Agenta?
Agenta is an end-to-end LLM developer platform. It provides the tools for **prompt engineering and management**, ⚖️ **evaluation**, **human annotation**, and :rocket: **deployment**. All without imposing any restrictions on your choice of framework, library, or model.
Agenta allows developers and product teams to collaborate in building production-grade LLM-powered applications in less time.
### With Agenta, you can:
- [🧪 **Experiment** and **compare** prompts](https://docs.agenta.ai/prompt_management/prompt_engineering) on [any LLM workflow](https://docs.agenta.ai/prompt_management/custom_applications) (chain-of-prompts, Retrieval Augmented Generation (RAG), LLM agents...)
- ✍️ Collect and [**annotate golden test sets**](https://docs.agenta.ai/evaluation/test_sets) for evaluation
- 📈 [**Evaluate** your application](https://docs.agenta.ai/evaluation/automatic_evaluation) with pre-existing or [**custom evaluators**](https://docs.agenta.ai/evaluation/custom_evaluator)
- [🔍 **Annotate** and **A/B test**](https://docs.agenta.ai/evaluation/human_evaluation) your applications with **human feedback**
- [🤝 **Collaborate with product teams**](https://docs.agenta.ai/misc/team_management) for prompt engineering and evaluation
- [🚀 **Deploy your application**](https://docs.agenta.ai/prompt_management/deployment) in one-click in the UI, through CLI, or through github workflows.
### Works with any LLM app workflow
Agenta enables prompt engineering and evaluation on any LLM app architecture:
- Chain of prompts
- RAG
- Agents
It works with any framework such as [Langchain](https://langchain.com), [LlamaIndex](https://www.llamaindex.ai/) and any LLM provider (openAI, Cohere, Mistral).
# Quick Start
### [Get started for free](https://cloud.agenta.ai?utm_source=github&utm_medium=readme&utm_campaign=github)
### [Explore the Docs](https://docs.agenta.ai/getting_started/introduction)
### [Create your first application in one-minute](https://docs.agenta.ai/getting_started/quick-start)
### [Create an application using Langchain](https://docs.agenta.ai/guides/tutorials/first-app-with-langchain)
### [Self-host agenta](https://docs.agenta.ai/self-host/host-locally)
### [Check the Cookbook](https://docs.agenta.ai/guides/evaluation_from_sdk)
# Features
| Playground | Evaluation |
| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Compare and version prompts for any LLM app, from single prompt to agents.
| Define test sets, then evaluate manually or programmatically your different variants.
|
| Human annotation | Deployment |
| Use Human annotator to A/B test and score your LLM apps.
| When you are ready, deploy your LLM applications as APIs in one click.
 |
# Enterprise Support
Contact us here for enterprise support and early access to agenta self-managed enterprise with Kubernetes support.
# Disabling Anonymized Tracking
By default, Agenta automatically reports anonymized basic usage statistics. This helps us understand how Agenta is used and track its overall usage and growth. This data does not include any sensitive information.
To disable anonymized telemetry, follow these steps:
- For web: Set `TELEMETRY_TRACKING_ENABLED` to `false` in your `agenta-web/.env` file.
- For CLI: Set `telemetry_tracking_enabled` to `false` in your `~/.agenta/config.toml` file.
After making this change, restart Agenta Compose.
# ⭐️ Join Our Team
- [Founding Product Engineer Frontend](https://agentaai.notion.site/Founding-Product-Engineer-Frontend-b6d26a3e9b254be6b6c2bfffbf0b53c5)
# Contributing
We warmly welcome contributions to Agenta. Feel free to submit issues, fork the repository, and send pull requests.
We are usually hanging in our Slack. Feel free to [join our Slack and ask us anything](https://join.slack.com/t/agenta-hq/shared_invite/zt-1zsafop5i-Y7~ZySbhRZvKVPV5DO_7IA)
Check out our [Contributing Guide](https://docs.agenta.ai/misc/contributing/getting-started) for more information.
## Contributors ✨
[](#contributors-)
Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):
Sameh Methnani
💻 📖
Suad Suljovic
💻 🎨 🧑🏫 👀
burtenshaw
💻
Abram
💻 📖
Israel Abebe
🐛 🎨 💻
Master X
💻
corinthian
💻 🎨
Pavle Janjusevic
🚇
Kaosi Ezealigo
🐛 💻
Alberto Nunes
🐛
Maaz Bin Khawar
💻 👀 🧑🏫
Nehemiah Onyekachukwu Emmanuel
💻 💡 📖
Philip Okiokio
📖
Abhinav Pandey
💻
Ramchandra Warang
💻 🐛
Biswarghya Biswas
💻
Uddeepta Raaj Kashyap
💻
Nayeem Abdullah
💻
Kang Suhyun
💻
Yoon
💻
Kirthi Bagrecha Jain
💻
Navdeep
💻
Rhythm Sharma
💻
Osinachi Chukwujama
💻
莫尔索
📖
Agunbiade Adedeji
💻
Emmanuel Oloyede
💻 📖
Dhaneshwarguiyan
💻
Priyanshu Prajapati
📖
Raviteja
💻
Arijit
💻
Yachika9925
📖
Aldrin
⚠️
seungduk.kim.2304
💻
Andrei Dragomir
💻
diego
💻
brockWith
💻
Dennis Zelada
💻
Romain Brucker
💻
Heon Heo
💻
Drew Reisner
💻
Ikko Eltociear Ashimine
📖
Vishal Vanpariya
💻
Youcef Boumar
📖
LucasTrg
💻 🐛
Ashraf Chowdury
🐛 💻
jp-agenta
💻 🐛
This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind are welcome!
**Attribution**: Testing icons created by [Freepik - Flaticon](https://www.flaticon.com/free-icons/testing)