https://github.com/AgentOps-AI/agentops
Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including OpenAI Agents SDK, CrewAI, Langchain, Autogen, AG2, and CamelAI
https://github.com/AgentOps-AI/agentops
agent agentops agents-sdk ai anthropic autogen cost-estimation crewai evals evaluation-metrics groq langchain llm mistral ollama openai openai-agents
Last synced: about 1 year ago
JSON representation
Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including OpenAI Agents SDK, CrewAI, Langchain, Autogen, AG2, and CamelAI
- Host: GitHub
- URL: https://github.com/AgentOps-AI/agentops
- Owner: AgentOps-AI
- License: mit
- Created: 2023-08-15T23:26:23.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2025-03-25T19:19:13.000Z (about 1 year ago)
- Last Synced: 2025-03-25T20:01:35.481Z (about 1 year ago)
- Topics: agent, agentops, agents-sdk, ai, anthropic, autogen, cost-estimation, crewai, evals, evaluation-metrics, groq, langchain, llm, mistral, ollama, openai, openai-agents
- Language: Python
- Homepage: https://agentops.ai
- Size: 143 MB
- Stars: 4,107
- Watchers: 43
- Forks: 367
- Open Issues: 87
-
Metadata Files:
- Readme: README.md
- Contributing: .github/CONTRIBUTING.md
- License: LICENSE
- Code of conduct: .github/CODE_OF_CONDUCT.md
Awesome Lists containing this project
- awesome-production-agentic-systems - AgentOps - AI/agentops.svg?cacheSeconds=86400) - AgentOps helps developers build, evaluate, and monitor AI agents from prototype to production. (Agent Observability)
- awesome-openclaw-money-maker - **AgentOps** - Python SDK for AI agent monitoring, LLM cost tracking, benchmarking. Integrates with CrewAI, Langchain, Autogen. (AI Agent Frameworks / General Agent Frameworks)
- StarryDivineSky - AgentOps-AI/agentops
- awesome-opensource-ai - AgentOps - Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and evaluation. Integrates with CrewAI, Agno, OpenAI Agents SDK, LangChain, Autogen, AG2, and CamelAI. MIT licensed.  (10. AI Safety, Alignment & Interpretability)
- Awesome-AI-Agents - agentops - Python SDK for agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks like CrewAI, Langchain, and Autogen  (Benchmark/Evaluator / Advanced Components)
- awesome-harness-engineering - AgentOps - source agent engineering platform (YC W24) with session replay, cost tracking, and failure detection across 10+ frameworks including CrewAI, LangGraph, and OpenAI Agents SDK. The step-by-step execution graph and cross-session metrics make it the most practical debugging layer for multi-agent systems in production.  (Design Primitives / Debugging & Developer Experience)
- awesome-agents - AgentOps - Toolkit for agent monitoring, testing, and replay debugging with session recordings. (Monitoring and Observability)
- awesome-production-machine-learning - AgentOps - AI/agentops.svg?style=social) - AgentOps helps developers build, evaluate, and monitor AI agents from prototype to production. (Agentic Framework)
- awesome_ai_agents - Agentops - Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks like CrewAI… (Building / Benchmarks)
- awesome-ai-engineering - Agent Ops
- awesome-nlp-llm-spanish-espa-ol - Agentops
- awesome-production-llm - AgentOps - AI`)  Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks like CrewAI, Langchain, and Autogen (LLM Application / RAG)
- awesome-agent-cortex - AgentOps - Monitoring, cost tracking, and benchmarking for agent workflows. (Agent Observability and Testing / Benchmark Reality Check (real-world tool use))
- awesome-agent-collaborate-tools - AgentOps - Agent-specific observability and testing. Session replays, cost tracking, failure analysis, and compliance monitoring. Native integrations with CrewAI, AutoGen, LangChain. (Observability & Debugging)
- awesome-ai-agents - AgentOps - AI/agentops) | Monitoring, cost tracking, and benchmarking SDK for agents | (🌟 Core Frameworks)
- jimsghstars - AgentOps-AI/agentops - Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Langchain, Autogen, AG2, and CamelAI (Python)
- awesome-ai-agents - AgentOps-AI/agentops - AgentOps is a Python SDK providing observability, cost tracking, benchmarking, and security tools for AI agents and large language models, integrating with major agent frameworks to support developers from prototype to production. (Agent Integration & Deployment Tools / MCP Server Framework)
- awesome-open-source-ai-tools - AgentOps-AI/agentops - Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent fr... (Chatbots & Virtual Companions)