https://github.com/Helicone/helicone

🧊 Open source LLM-Observability Platform for Developers. One-line integration for monitoring, metrics, evals, agent tracing, prompt management, playground, etc. Supports OpenAI SDK, Vercel AI SDK, Anthropic SDK, LiteLLM, LLamaIndex, LangChain, and more. 🍓 YC W23
https://github.com/Helicone/helicone

agent-monitoring analytics evaluation gpt langchain large-language-models llama-index llm llm-cost llm-evaluation llm-observability llmops monitoring open-source openai playground prompt-engineering prompt-management ycombinator

Last synced: about 1 year ago
JSON representation

🧊 Open source LLM-Observability Platform for Developers. One-line integration for monitoring, metrics, evals, agent tracing, prompt management, playground, etc. Supports OpenAI SDK, Vercel AI SDK, Anthropic SDK, LiteLLM, LLamaIndex, LangChain, and more. 🍓 YC W23

• Host: GitHub
• URL: https://github.com/Helicone/helicone
• Owner: Helicone
• License: apache-2.0
• Created: 2023-01-31T22:34:44.000Z (over 3 years ago)
• Default Branch: main
• Last Pushed: 2024-10-29T05:21:58.000Z (over 1 year ago)
• Last Synced: 2024-10-29T15:37:49.109Z (over 1 year ago)
• Topics: agent-monitoring, analytics, evaluation, gpt, langchain, large-language-models, llama-index, llm, llm-cost, llm-evaluation, llm-observability, llmops, monitoring, open-source, openai, playground, prompt-engineering, prompt-management, ycombinator
• Language: TypeScript
• Homepage: https://www.helicone.ai
• Size: 285 MB
• Stars: 1,861
• Watchers: 8
• Forks: 204
• Open Issues: 58
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING_GUIDELINES.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-x-ops - Helicone - source observability platform for LLM usage, latency, cost, caching, and request logs. (LLM and Agent Observability)
• awesome-ai-sdks - GitHub
• awesome-ai - Helicone AI - Open-source LLM observability platform for logging, monitoring, and debugging AI applications. (Development / LLM Ops)
• awesome-ai-coding-agent-tools - Helicone - Open-source LLM observability platform for tracing and monitoring. (Supporting Infrastructure / Observability)
• awesome-gpt4 - helicone - GPT4 Monitoring and Observability. (Tools / Open-source projects)
• StarryDivineSky - Helicone/helicone
• awesome-production-machine-learning - Helicone - Helicone is the all-in-one, open-source LLM developer platform. (Evaluation and Monitoring)
• awesome-ai-testing - Helicone - Open source LLM observability and prompt evaluation platform. (LLM-as-Judge Evaluation)
• awesome-open-source-ai-tools - Helicone/helicone - 🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓 (Coding & Development)
• Awesome-RAG-Production - Helicone - host / Cloud | OpenAI + Anthropic | Built-in | Fine-grained token analytics | Cost optimization and prompt versioning | (LLM Gateways & Routing / Tools)
• awesome-ai-ml-testing - Helicone - Open-source LLM observability platform. (📡 Monitoring & Observability)
• awesome-llmops - Helicone - source LLM observability platform for logging, monitoring, and debugging AI applications. Simple 1-line integration to get started. |  | (LLMOps / Observability)
• awesome-harness-engineering - Helicone - source LLM observability proxy (YC W23) with the largest open-source pricing database (300+ models). One-line proxy integration provides cost tracking, token monitoring, session tracing, and prompt versioning across providers. The AI Gateway component handles request routing and caching with zero-code changes. SOC 2 and GDPR compliant, self-hostable via Docker — the natural complement to execution-tracing tools when cost attribution is a primary concern.  (Design Primitives / Observability & Tracing)
• Awesome-LLMOps - Helicone AI - source LangSmith alternative for logging, monitoring, and debugging AI applications.| (Observation)
• awesome-ai-agents - Helicone/helicone - Helicone is an open-source LLM observability platform that enables developers to monitor, evaluate, and experiment with large language model applications through easy integration and comprehensive tools. (Agent Integration & Deployment Tools / MCP Server Framework)
• awesome-llm-cost - Helicone - Proxy first observability with cost tracking, caching, and rate limits, no SDK install required. Self hostable. (Cost Tracking, Observability, and Budgets / Open source)
• awesome-agents - Helicone - Open-source LLM observability platform for logging, monitoring, and improving AI applications. (Monitoring and Observability)
• awesome-opensource-ai - Helicone - Open-source LLM observability platform with prompt management, versioning, and experimentation. One-line integration, YC W23 company. Apache 2.0 licensed.  (13. Developer Tools & Integrations)
• awesome-ai-agents-2026 - Helicone - line integration. | (🔍 Observability and Evaluation / Tracing and Monitoring)
• AiTreasureBox - Helicone/helicone - 11-03_4657_0](https://img.shields.io/github/stars/Helicone/helicone.svg)|🧊 Open source LLM-Observability Platform for Developers. One-line integration for monitoring, metrics, evals, agent tracing, prompt management, playground, etc. Supports OpenAI SDK, Vercel AI SDK, Anthropic SDK, LiteLLM, LLamaIndex, LangChain, and more. 🍓 YC W23| (Repos)
• awesome-llmops - Helicone AI -  - Open-source LLM observability platform for logging, monitoring, and debugging AI applications. Simple 1-line integration to get started. (Tools (GitHub) / LLMOps vs MLOps)
• awesome - Helicone/helicone - 🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓 (TypeScript)
• awesome-llm-tools - HeliconeAI
• awesome-ai-eval - **Helicone** - Open-source LLM observability platform with cost tracking, caching, and evaluation tools. (Platforms / Open Source Platforms)
• llmops - Helicone - source LLM observability |  | (Observability & Monitoring / Resources)
• awesome-agent-cortex - Helicone - Open-source LLM observability and monitoring platform. (Agent Observability and Testing / Benchmark Reality Check (real-world tool use))
• awesome-agent-collaborate-tools - Helicone - Open-source LLM observability and caching proxy. Drop-in for OpenAI/Anthropic/Gemini. Per-request latency, cost, and error tracking with user-level attribution. (Observability & Debugging)
• awesome-rag - Helicone
• awesome-observability - Helicone - Open source LLM observability platform and AI Gateway. One line of code to monitor, evaluate, and experiment across 100+ providers. (10. LLM & AI Observability / Platforms)
• jimsghstars - Helicone/helicone - 🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓 (TypeScript)