https://github.com/chirpz-ai/pandaprobe
open source agent engineering platform: traces, evals, and metrics to debug and improve your AI agents. Integrates with LangGraph, CrewAI, Claude Agent SDK, and more.
https://github.com/chirpz-ai/pandaprobe
agent-engineering agent-evaluation agent-observability agentic-ai claude-agent-sdk crewai langgraph monitoring open-source openai-agents-sdk self-hosted tracing
Last synced: 15 days ago
JSON representation
open source agent engineering platform: traces, evals, and metrics to debug and improve your AI agents. Integrates with LangGraph, CrewAI, Claude Agent SDK, and more.
- Host: GitHub
- URL: https://github.com/chirpz-ai/pandaprobe
- Owner: chirpz-ai
- License: apache-2.0
- Created: 2025-12-16T05:45:30.000Z (6 months ago)
- Default Branch: main
- Last Pushed: 2026-06-03T22:07:48.000Z (15 days ago)
- Last Synced: 2026-06-03T23:14:20.507Z (15 days ago)
- Topics: agent-engineering, agent-evaluation, agent-observability, agentic-ai, claude-agent-sdk, crewai, langgraph, monitoring, open-source, openai-agents-sdk, self-hosted, tracing
- Language: Python
- Homepage: https://pandaprobe.com
- Size: 14.3 MB
- Stars: 256
- Watchers: 1
- Forks: 16
- Open Issues: 15
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Citation: CITATION.cff
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
README
## What is PandaProbe?
PandaProbe is an open source agent engineering platform. It helps teams collaboratively trace, evaluate, monitor, and debug AI agents. You can use PandaProbe cloud or self host the service.
## Documentation
Visit our client library documentation for quickstart and explore advance integrations.
## PandaProbe Cloud
Managed deployment by the PandaProbe team, generous free-tier, no credit card required.
## Self-host PandaProbe
> **Prerequisites:** [Docker](https://docs.docker.com/get-docker/) must be installed and running.
```bash
git clone https://github.com/chirpz-ai/pandaprobe.git
cd pandaprobe
./start.sh
```
Once running, open:
- **Dashboard** — http://localhost:3000
- **API reference** — http://localhost:8000/scalar
## Architecture
```mermaid
sequenceDiagram
participant Client as 📡 SDK / HTTP Client
participant API as ⚡ FastAPI
participant Auth as 🔐 Auth Service
participant IdP as 🌐 Supabase / Firebase
participant Identity as 👥 Identity Service
participant Trace as Trace Service
participant Eval as 🧪 Eval Service
participant DB as 🗄️ PostgreSQL
participant Redis as 📮 Redis
participant Worker as ⚙️ Celery Worker
participant LLM as 🤖 LLM Engine (LiteLLM)
Note over Client,API: Management Plane (Bearer token)
Client->>API: Authorization: Bearer
API->>Auth: Verify token
Auth->>IdP: Validate with provider
IdP-->>Auth: User identity
Auth-->>API: Authenticated user
API->>Identity: /user, /organizations, /projects
Identity->>DB: Read / write
DB-->>Identity: Result
Identity-->>Client: Response
Note over Client,API: Data Plane (API key)
Client->>API: X-API-Key + X-Project-Name
API->>Identity: Resolve org & project
Identity-->>API: Project context
API->>Trace: POST /traces
Trace->>Redis: Enqueue ingestion job
Redis-->>Client: 202 Accepted
Redis->>Worker: Pick up job
Worker->>DB: Persist trace + spans
API->>Trace: GET /traces, /sessions
Trace->>DB: Query with filters
DB-->>Trace: Rows
Trace-->>Client: Paginated response
API->>Eval: POST /evaluations
Eval->>Redis: Enqueue eval job
Redis-->>Client: 202 Accepted
Redis->>Worker: Pick up job
Worker->>LLM: LLM-as-a-judge call
LLM-->>Worker: Verdict + score
Worker->>DB: Persist evaluation result
```
## Services
| Service | Description | Port |
|---|---|---|
| **frontend** | Next.js dashboard | 3000 |
| **app** | FastAPI application server | 8000 |
| **worker** | Celery background worker | — |
| **beat** | Celery Beat scheduler | — |
| **postgres** | PostgreSQL 16 | 5432 |
| **redis** | Redis 7 (broker + cache) | 6379 |
## Contributing
We welcome contributions! Please read the [Contributing Guide](CONTRIBUTING.md) for instructions on setting up your development environment, building from source, running tests, and submitting pull requests.
## Authors
Built by the [Chirpz AI](https://pandaprobe.com/about) team. Contact sina@pandaprobe.com for enquiries.
## License
PandaProbe is licensed under Apache 2.0 — see [LICENSE](LICENSE) for details.