Projects in Awesome Lists tagged with llm-observability
A curated list of projects in awesome lists tagged with llm-observability .
https://github.com/langfuse/langfuse
🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23
analytics autogen evaluation langchain large-language-models llama-index llm llm-evaluation llm-observability llmops monitoring observability open-source openai playground prompt-engineering prompt-management self-hosted ycombinator
Last synced: 13 May 2025
https://github.com/comet-ml/opik
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
langchain llama-index llm llm-evaluation llm-observability llmops open-source openai playground prompt-engineering
Last synced: 13 May 2025
https://github.com/helicone/helicone
🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓
agent-monitoring analytics evaluation gpt langchain large-language-models llama-index llm llm-cost llm-evaluation llm-observability llmops monitoring open-source openai playground prompt-engineering prompt-management ycombinator
Last synced: 08 May 2025
https://github.com/agenta-ai/agenta
The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.
llm-as-a-judge llm-evaluation llm-framework llm-monitoring llm-observability llm-platform llm-playground llm-tools llmops-platform prompt-engineering prompt-management rag-evaluation
Last synced: 12 May 2025
https://github.com/Helicone/helicone
🧊 Open source LLM-Observability Platform for Developers. One-line integration for monitoring, metrics, evals, agent tracing, prompt management, playground, etc. Supports OpenAI SDK, Vercel AI SDK, Anthropic SDK, LiteLLM, LLamaIndex, LangChain, and more. 🍓 YC W23
agent-monitoring analytics evaluation gpt langchain large-language-models llama-index llm llm-cost llm-evaluation llm-observability llmops monitoring open-source openai playground prompt-engineering prompt-management ycombinator
Last synced: 31 Mar 2025
https://github.com/lmnr-ai/lmnr
Laminar - open-source all-in-one platform for engineering AI products. Crate data flywheel for you AI app. Traces, Evals, Datasets, Labels. YC S24.
agents ai ai-observability aiops analytics developer-tools evals evaluation llm-evaluation llm-observability llm-workflow llmops monitoring observability open-source pipeline-builder rag rust-lang self-hosted
Last synced: 14 May 2025
https://github.com/voltagent/voltagent
Open Source TypeScript AI Agent Framework
agents ai autogen chatbots framework javascript llm llm-observability mcp nodejs observability tts typescript
Last synced: 15 May 2025
https://github.com/teilomillet/hapax
The reliability layer between your code and LLM providers.
anthropic infrastructure infrastructure-as-code llm llm-gateway llm-observability llmops monitoring observability open-source openai
Last synced: 09 Apr 2025
