https://github.com/agenta-ai/agenta

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.
https://github.com/agenta-ai/agenta

llm-as-a-judge llm-evaluation llm-framework llm-monitoring llm-observability llm-platform llm-playground llm-tools llmops-platform prompt-engineering prompt-management rag-evaluation

Last synced: 12 days ago
JSON representation

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

Host: GitHub
URL: https://github.com/agenta-ai/agenta
Owner: Agenta-AI
License: mit
Created: 2023-04-26T09:54:28.000Z (about 2 years ago)
Default Branch: main
Last Pushed: 2025-05-09T16:59:41.000Z (15 days ago)
Last Synced: 2025-05-12T15:23:31.238Z (12 days ago)
Topics: llm-as-a-judge, llm-evaluation, llm-framework, llm-monitoring, llm-observability, llm-platform, llm-playground, llm-tools, llmops-platform, prompt-engineering, prompt-management, rag-evaluation
Language: Python
Homepage: http://www.agenta.ai
Size: 166 MB
Stars: 2,698
Watchers: 28
Forks: 314
Open Issues: 21
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Security: SECURITY.md

Awesome Lists containing this project

Awesome_Multimodel_LLM - Agenta - Easily build, version, evaluate and deploy your LLM-powered apps. (Tools for deploying LLM)
Awesome-Prompt-Engineering - [Github
Awesome-LLM - Agenta - Easily build, version, evaluate and deploy your LLM-powered apps. (LLM Deployment)
StarryDivineSky - agenta-ai/agenta
awesome-generative-ai - agenta - An open-source end-to-end LLMOps platform for prompt engineering, evaluation, and deployment. #opensource (Coding / Developer tools)

README

The Open-source LLMOps Platform

Build reliable LLM applications faster with integrated prompt management, evaluation, and observability.

---

Documentation •
Website •
Agenta Cloud

---

Contributors

Last Commit

---

## What is Agenta?

Agenta is a platform for building production-grade LLM applications. It helps **engineering** and **product teams** create reliable LLM apps faster through integrated prompt management, evaluation, and observability.

## Core Features

### 🧪 Prompt Engineering & Management
Collaborate with Subject Matter Experts (SMEs) on prompt engineering and make sure nothing breaks in production.

- **Interactive Playground**: Compare prompts side by side against your test cases
- **Multi-Model Support**: Experiment with 50+ LLM models or [bring-your-own models](https://docs.agenta.ai/prompt-engineering/playground/adding-custom-providers?utm_source=github&utm_medium=referral&utm_campaign=readme)
- **Version Control**: Version prompts and configurations with branching and environments
- **Complex Configurations**: Enable SMEs to collaborate on [complex configuration schemas](https://docs.agenta.ai/custom-workflows/overview?utm_source=github&utm_medium=referral&utm_campaign=readme) beyond simple prompts

[Explore prompt management →](https://docs.agenta.ai/prompt-engineering/overview?utm_source=github&utm_medium=referral&utm_campaign=readme)

### 📊 Evaluation & Testing
Evaluate your LLM applications systematically with both human and automated feedback.
- **Flexible Test Sets**: Create test cases from production data, playground experiments, or upload CSVs
- **Pre-built and Custom Evaluators**: Use LLM-as-judge, one of our 20+ pre-built evaluators, or you custom evaluators
- **UI and API Access**: Run evaluations via UI (for SMEs) or programmatically (for engineers)
- **Human Feedback Integration**: Collect and incorporate expert annotations

[Explore evaluation frameworks →](https://docs.agenta.ai/evaluation/overview?utm_source=github&utm_medium=referral&utm_campaign=readme)

### 📡 Observability & Monitoring
Get visibility into your LLM applications in production.
- **Cost & Performance Tracking**: Monitor spending, latency, and usage patterns
- **Tracing**: Debug complex workflows with detailed traces
- **Open Standards**: OpenTelemetry native tracing compatible with OpenLLMetry, and OpenInference
- **Integrations**: Comes with pre-built integrations for most models and frameworks

[Learn about observability →](https://docs.agenta.ai/observability/overview?utm_source=github&utm_medium=referral&utm_campaign=readme)

## 📸 Screenshots

Playground
Prompt Management
Evaluation
Observability

## 🚀 Getting Started

### Agenta Cloud (Recommended):
The easiest way to get started is through Agenta Cloud. Free tier available with no credit card required.

### Self-hosting Agenta

1. Clone Agenta:
```bash
git clone https://github.com/Agenta-AI/agenta && cd agenta
```

2. Start Agenta services:
```bash
docker compose -f hosting/docker-compose/oss/docker-compose.gh.yml --env-file hosting/docker-compose/oss/.env.oss.gh --profile with-web up -d
```

3. Access Agenta at `http://localhost`.

For deploying on a remote host, or using different ports refers to our [self-hosting](https://docs.agenta.ai/self-host/host-locally?utm_source=github&utm_medium=referral&utm_campaign=readme) and [remote deployment documentation](https://docs.agenta.ai/self-host/host-remotely?utm_source=github&utm_medium=referral&utm_campaign=readme).

## 💬 Community

Find help, explore resources, or get involved:

### 🧰 Support

- **📚 [Documentation](https://docs.agenta.ai?utm_source=github&utm_medium=referral&utm_campaign=readme)** – Full guides and API reference
- **📋 [Changelog](https://docs.agenta.ai/changelog/main?utm_source=github&utm_medium=referral&utm_campaign=readme)** – Track recent updates
- **💬 [Slack Community](https://join.slack.com/t/agenta-hq/shared_invite/zt-2yewk6o2b-DmhyA4h_lkKwecDtIsj1AQ)** – Ask questions and get support

### 🤝 Contribute

We welcome contributions of all kinds — from filing issues and sharing ideas to improving the codebase.

- **🐛 [Report bugs](https://github.com/Agenta-AI/agenta/issues)** – Help us by reporting problems you encounter
- **💡 [Share ideas and feedback](https://github.com/Agenta-AI/agenta/discussions)** – Suggest features or vote on ideas
- **🔧 [Contribute to the codebase](https://docs.agenta.ai/misc/contributing/getting-started?utm_source=github&utm_medium=referral&utm_campaign=readme)** – Read the guide and open a pull request

## ⭐ Star Agenta

**Consider giving us a star!** It helps us grow our community and gets Agenta in front of more developers.

Star us

## Contributors ✨

[![All Contributors](https://img.shields.io/badge/all_contributors-49-orange.svg?style=flat-square)](#contributors-)

Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):

_{Sameh Methnani}
💻 📖

_{Suad Suljovic}
💻 🎨 🧑‍🏫 👀

_burtenshaw
💻

_Abram
💻 📖

_{Israel Abebe}
🐛 🎨 💻

_{Master X}
💻

_corinthian
💻 🎨

_{Pavle Janjusevic}
🚇

_{Kaosi Ezealigo}
🐛 💻

_{Alberto Nunes}
🐛

_{Maaz Bin Khawar}
💻 👀 🧑‍🏫

_{Nehemiah Onyekachukwu Emmanuel}
💻 💡 📖

_{Philip Okiokio}
📖

_{Abhinav Pandey}
💻

_{Ramchandra Warang}
💻 🐛

_{Biswarghya Biswas}
💻

_{Uddeepta Raaj Kashyap}
💻

_{Nayeem Abdullah}
💻

_{Kang Suhyun}
💻

_Yoon
💻

_{Kirthi Bagrecha Jain}
💻

_Navdeep
💻

_{Rhythm Sharma}
💻

_{Osinachi Chukwujama}
💻

_莫尔索
 📖

_{Agunbiade Adedeji}
💻

_{Emmanuel Oloyede}
💻 📖

_{Dhaneshwarguiyan}
💻

_{Priyanshu Prajapati}
📖

_Raviteja
💻

_Arijit
💻

_Yachika9925
📖

_Aldrin
⚠️

_{seungduk.kim.2304}
💻

_{Andrei Dragomir}
💻

_diego
💻

_brockWith
💻

_{Dennis Zelada}
💻

_{Romain Brucker}
💻

_{Heon Heo}
💻

_{Drew Reisner}
💻

_{Ikko Eltociear Ashimine}
📖

_{Vishal Vanpariya}
💻

_{Youcef Boumar}
📖

_LucasTrg
💻 🐛

_{Ashraf Chowdury}
🐛 💻

_jp-agenta
💻 🐛

_{Mr Unhappy}
🐛 🚇

_{Moreno Bonaventura}
🐛

This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind are welcome!

## Disabling Anonymized Tracking

By default, Agenta automatically reports anonymized basic usage statistics. This helps us understand how Agenta is used and track its overall usage and growth. This data does not include any sensitive information. To disable anonymized telemetry set `TELEMETRY_ENABLED` to `false` in your `.env` file.

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

https://github.com/agenta-ai/agenta

Awesome Lists containing this project

README

The Open-source LLMOps Platform

Documentation •
Website •
Agenta Cloud

https://github.com/agenta-ai/agenta

Awesome Lists containing this project

README

The Open-source LLMOps Platform

Documentation • Website • Agenta Cloud

Documentation •
Website •
Agenta Cloud