An open API service indexing awesome lists of open source software.

https://github.com/arize-ai/openinference

OpenTelemetry Instrumentation for AI Observability
https://github.com/arize-ai/openinference

aiops gemini hacktoberfest haystack langchain langraph llamaindex llmops llms mcp openai openai-agents opentelemetry pydantic-ai smolagents telemetry tracing vercel vertex

Last synced: 10 days ago
JSON representation

OpenTelemetry Instrumentation for AI Observability

Awesome Lists containing this project

README

          


OpenInference









OpenInference is a set of conventions and plugins that is complimentary to [OpenTelemetry](https://opentelemetry.io/) to
enable tracing of AI applications. OpenInference is natively supported
by [arize-phoenix](https://github.com/Arize-ai/phoenix), but can be used with any OpenTelemetry-compatible backend as
well.

## Specification

The OpenInference specification is edited in markdown files found in the [spec directory](./spec/). It's designed to
provide insight into the invocation of LLMs and the surrounding application context such as retrieval from vector stores
and the usage of external tools such as search engines or APIs. The specification is transport and file-format agnostic,
and is intended to be used in conjunction with other specifications such as JSON, ProtoBuf, and DataFrames.

## Instrumentation

OpenInference provides a set of instrumentations for popular machine learning SDKs and frameworks in a variety of
languages.

## Python

### Libraries

| Package | Description | Version |
|-------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [`openinference-semantic-conventions`](./python/openinference-semantic-conventions) | Semantic conventions for tracing of LLM Apps. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-semantic-conventions.svg)](https://pypi.python.org/pypi/openinference-semantic-conventions) |
| [`openinference-instrumentation`](./python/openinference-instrumentation) | Reusable utilities, decorators, configurations, and helpers for instrumentation. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation.svg)](https://pypi.python.org/pypi/openinference-instrumentation) |
| [`openinference-instrumentation-agno`](./python/instrumentation/openinference-instrumentation-agno) | OpenInference Instrumentation for Agno Agents. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-agno.svg)](https://pypi.python.org/pypi/openinference-instrumentation-agno) |
| [`openinference-instrumentation-openai`](./python/instrumentation/openinference-instrumentation-openai) | OpenInference Instrumentation for OpenAI SDK. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-openai.svg)](https://pypi.python.org/pypi/openinference-instrumentation-openai) |
| [`openinference-instrumentation-openai-agents`](./python/instrumentation/openinference-instrumentation-openai-agents) | OpenInference Instrumentation for OpenAI Agents SDK. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-openai-agents.svg)](https://pypi.python.org/pypi/openinference-instrumentation-openai-agents) |
| [`openinference-instrumentation-claude-agent-sdk`](./python/instrumentation/openinference-instrumentation-claude-agent-sdk) | OpenInference Instrumentation for Claude Agent SDK. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-claude-agent-sdk.svg)](https://pypi.python.org/pypi/openinference-instrumentation-claude-agent-sdk) |
| [`openinference-instrumentation-llama-index`](./python/instrumentation/openinference-instrumentation-llama-index) | OpenInference Instrumentation for LlamaIndex. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-llama-index.svg)](https://pypi.python.org/pypi/openinference-instrumentation-llama-index) |
| [`openinference-instrumentation-dspy`](./python/instrumentation/openinference-instrumentation-dspy) | OpenInference Instrumentation for DSPy. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-dspy.svg)](https://pypi.python.org/pypi/openinference-instrumentation-dspy) |
| [`openinference-instrumentation-bedrock`](./python/instrumentation/openinference-instrumentation-bedrock) | OpenInference Instrumentation for AWS Bedrock. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-bedrock.svg)](https://pypi.python.org/pypi/openinference-instrumentation-bedrock) |
| [`openinference-instrumentation-langchain`](./python/instrumentation/openinference-instrumentation-langchain) | OpenInference Instrumentation for LangChain. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-langchain.svg)](https://pypi.python.org/pypi/openinference-instrumentation-langchain) |
| [`openinference-instrumentation-mcp`](./python/instrumentation/openinference-instrumentation-mcp) | OpenInference Instrumentation for MCP. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-mcp.svg)](https://pypi.python.org/pypi/openinference-instrumentation-mcp) |
| [`openinference-instrumentation-mistralai`](./python/instrumentation/openinference-instrumentation-mistralai) | OpenInference Instrumentation for MistralAI. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-mistralai.svg)](https://pypi.python.org/pypi/openinference-instrumentation-mistralai) |
| [`openinference-instrumentation-portkey`](./python/instrumentation/openinference-instrumentation-portkey) | OpenInference Instrumentation for Portkey. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-portkey.svg)](https://pypi.python.org/pypi/openinference-instrumentation-portkey) |
| [`openinference-instrumentation-guardrails`](./python/instrumentation/openinference-instrumentation-guardrails) | OpenInference Instrumentation for Guardrails. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-guardrails.svg)](https://pypi.python.org/pypi/openinference-instrumentation-guardrails) |
| [`openinference-instrumentation-vertexai`](./python/instrumentation/openinference-instrumentation-vertexai) | OpenInference Instrumentation for VertexAI. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-vertexai.svg)](https://pypi.python.org/pypi/openinference-instrumentation-vertexai) |
| [`openinference-instrumentation-crewai`](./python/instrumentation/openinference-instrumentation-crewai) | OpenInference Instrumentation for CrewAI. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-crewai.svg)](https://pypi.python.org/pypi/openinference-instrumentation-crewai) |
| [`openinference-instrumentation-haystack`](./python/instrumentation/openinference-instrumentation-haystack) | OpenInference Instrumentation for Haystack. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-haystack.svg)](https://pypi.python.org/pypi/openinference-instrumentation-haystack) |
| [`openinference-instrumentation-litellm`](./python/instrumentation/openinference-instrumentation-litellm) | OpenInference Instrumentation for liteLLM. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-litellm.svg)](https://pypi.python.org/pypi/openinference-instrumentation-litellm) |
| [`openinference-instrumentation-groq`](./python/instrumentation/openinference-instrumentation-groq) | OpenInference Instrumentation for Groq. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-groq.svg)](https://pypi.python.org/pypi/openinference-instrumentation-groq) |
| [`openinference-instrumentation-instructor`](./python/instrumentation/openinference-instrumentation-instructor) | OpenInference Instrumentation for Instructor. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-instructor.svg)](https://pypi.python.org/pypi/openinference-instrumentation-instructor) |
| [`openinference-instrumentation-anthropic`](./python/instrumentation/openinference-instrumentation-anthropic) | OpenInference Instrumentation for Anthropic. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-anthropic.svg)](https://pypi.python.org/pypi/openinference-instrumentation-anthropic) |
| [`openinference-instrumentation-beeai`](./python/instrumentation/openinference-instrumentation-beeai) | OpenInference Instrumentation for BeeAI. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-beeai.svg)](https://pypi.python.org/pypi/openinference-instrumentation-beeai) |
| [`openinference-instrumentation-google-genai`](./python/instrumentation/openinference-instrumentation-google-genai) | OpenInference Instrumentation for Google GenAI. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-google-genai.svg)](https://pypi.python.org/pypi/openinference-instrumentation-google-genai) |
| [`openinference-instrumentation-google-adk`](./python/instrumentation/openinference-instrumentation-google-adk) | OpenInference Instrumentation for Google ADK. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-google-adk.svg)](https://pypi.python.org/pypi/openinference-instrumentation-google-adk) |
| [`openinference-instrumentation-autogen-agentchat`](./python/instrumentation/openinference-instrumentation-autogen-agentchat) | OpenInference Instrumentation for Microsoft Autogen AgentChat. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-autogen-agentchat.svg)](https://pypi.python.org/pypi/openinference-instrumentation-autogen-agentchat) |
| [`openinference-instrumentation-pydantic-ai`](./python/instrumentation/openinference-instrumentation-pydantic-ai) | OpenInference Instrumentation for PydanticAI. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-pydantic-ai.svg)](https://pypi.python.org/pypi/openinference-instrumentation-pydantic-ai) |
| [`openinference-instrumentation-smolagents`](./python/instrumentation/openinference-instrumentation-smolagents) | OpenInference Instrumentation for smolagents. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-smolagents.svg)](https://pypi.python.org/pypi/openinference-instrumentation-smolagents) |
| [`openinference-instrumentation-pipecat`](./python/instrumentation/openinference-instrumentation-pipecat) | OpenInference Instrumentation for Pipecat. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-pipecat.svg)](https://pypi.python.org/pypi/openinference-instrumentation-pipecat) |
| [`openinference-instrumentation-agentspec`](./python/instrumentation/openinference-instrumentation-agentspec) | OpenInference Instrumentation for Open Agent Specification. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-agentspec.svg)](https://pypi.python.org/pypi/openinference-instrumentation-agentspec) |
| [`openinference-instrumentation-strands-agents`](./python/instrumentation/openinference-instrumentation-strands-agents) | OpenInference Instrumentation for Strands Agents. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-strands-agents.svg)](https://pypi.python.org/pypi/openinference-instrumentation-strands-agents) |

## Span Processors

Normalize and convert data across other instrumentation libraries by adding span processors that unify data.

| Package | Description | Version |
| ----------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [`openinference-instrumentation-openlit`](./python/instrumentation/openinference-instrumentation-openlit) | OpenInference Span Processor for OpenLIT traces. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-openlit.svg)](https://pypi.python.org/pypi/openinference-instrumentation-openlit) |
| [`openinference-instrumentation-openllmetry`](./python/instrumentation/openinference-instrumentation-openllmetry) | OpenInference Span Processor for OpenLLMetry (Traceloop) traces. | [![PyPI Version](https://img.shields.io/pypi/v/openinference-instrumentation-openllmetry.svg)](https://pypi.python.org/pypi/openinference-instrumentation-openllmetry) |

### Examples

| Name | Description | Complexity Level |
| ----------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- | ---------------- |
| [Agno](python/instrumentation/openinference-instrumentation-agno/examples/) | Agno agent examples | Beginner |
| [OpenAI SDK](python/instrumentation/openinference-instrumentation-openai/examples/) | OpenAI Python SDK, including chat completions and embeddings | Beginner |
| [Claude Agent SDK](python/instrumentation/openinference-instrumentation-claude-agent-sdk/examples/) | Claude Agent SDK | Beginner |
| [MistralAI SDK](python/instrumentation/openinference-instrumentation-mistralai/examples/) | MistralAI Python SDK | Beginner |
| [VertexAI SDK](python/instrumentation/openinference-instrumentation-vertexai/examples/) | VertexAI Python SDK | Beginner |
| [LlamaIndex](python/instrumentation/openinference-instrumentation-llama-index/examples/) | LlamaIndex query engines | Beginner |
| [DSPy](python/instrumentation/openinference-instrumentation-dspy/examples/) | DSPy primitives and custom RAG modules | Beginner |
| [Boto3 Bedrock Client](python/instrumentation/openinference-instrumentation-bedrock/examples/) | Boto3 Bedrock client | Beginner |
| [LangChain](python/instrumentation/openinference-instrumentation-langchain/examples/) | LangChain primitives and simple chains | Beginner |
| [LiteLLM](python/instrumentation/openinference-instrumentation-litellm/) | A lightweight LiteLLM framework | Beginner |
| [LiteLLM Proxy](python/instrumentation/openinference-instrumentation-litellm/examples/litellm-proxy/) | LiteLLM Proxy to log OpenAI, Azure, Vertex, Bedrock | Beginner |
| [Groq](python/instrumentation/openinference-instrumentation-groq/examples/) | Groq and AsyncGroq chat completions | Beginner |
| [Anthropic](python/instrumentation/openinference-instrumentation-anthropic/examples/) | Anthropic Messages client | Beginner |
| [BeeAI](python/instrumentation/openinference-instrumentation-beeai/examples/) | Agentic instrumentation in the BeeAI framework | Beginner |
| [Haystack](python/instrumentation/openinference-instrumentation-haystack/examples/) | A Haystack QA RAG application | Intermediate |
| [OpenAI Agents](python/instrumentation/openinference-instrumentation-openai-agents/examples/) | OpenAI Agents with handoffs | Intermediate |
| [Autogen AgentChat](python/instrumentation/openinference-instrumentation-autogen-agentchat/examples/) | Microsoft Autogen Assistant Agent and Team Chat | Intermediate |
| [PydanticAI](python/instrumentation/openinference-instrumentation-pydantic-ai/examples/) | PydanticAI agent examples | Intermediate |
| [Pipecat](python/instrumentation/openinference-instrumentation-pipecat/examples/) | Pipecat application examples | Intermediate |

## JavaScript



### Libraries

| Package | Description | Version |
| ----------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [`@arizeai/openinference-semantic-conventions`](./js/packages/openinference-semantic-conventions) | Semantic conventions for tracing of LLM Apps. | [![NPM Version](https://img.shields.io/npm/v/@arizeai/openinference-semantic-conventions.svg)](https://www.npmjs.com/package/@arizeai/openinference-semantic-conventions) |
| [`@arizeai/openinference-core`](./js/packages/openinference-core) | Reusable utilities, configuration, and helpers for instrumentation. | [![NPM Version](https://img.shields.io/npm/v/@arizeai/openinference-core.svg)](https://www.npmjs.com/package/@arizeai/openinference-core) |
| [`@arizeai/openinference-instrumentation-bedrock`](./js/packages/openinference-instrumentation-bedrock) | OpenInference Instrumentation for AWS Bedrock. | [![NPM Version](https://img.shields.io/npm/v/@arizeai/openinference-instrumentation-bedrock)](https://www.npmjs.com/package/@arizeai/openinference-instrumentation-bedrock) |
| [`@arizeai/openinference-instrumentation-bedrock-agent-runtime`](./js/packages/openinference-instrumentation-bedrock-agent-runtime) | OpenInference Instrumentation for AWS Bedrock Agent Runtime. | [![NPM Version](https://img.shields.io/npm/v/@arizeai/openinference-instrumentation-bedrock-agent-runtime)](https://www.npmjs.com/package/@arizeai/openinference-instrumentation-bedrock-agent-runtime) |
| [`@arizeai/openinference-instrumentation-beeai`](./js/packages/openinference-instrumentation-beeai) | OpenInference Instrumentation for BeeAI. | [![NPM Version](https://img.shields.io/npm/v/@arizeai/openinference-instrumentation-beeai)](https://www.npmjs.com/package/@arizeai/openinference-instrumentation-beeai) |
| [`@arizeai/openinference-instrumentation-langchain`](./js/packages/openinference-instrumentation-langchain) | OpenInference Instrumentation for LangChain.js. | [![NPM Version](https://img.shields.io/npm/v/@arizeai/openinference-instrumentation-langchain)](https://www.npmjs.com/package/@arizeai/openinference-instrumentation-langchain) |
| [`@arizeai/openinference-instrumentation-mcp`](./js/packages/openinference-instrumentation-mcp) | OpenInference Instrumentation for MCP. | [![NPM Version](https://img.shields.io/npm/v/@arizeai/openinference-instrumentation-mcp)](https://www.npmjs.com/package/@arizeai/openinference-instrumentation-mcp) |
| [`@arizeai/openinference-instrumentation-openai`](./js/packages/openinference-instrumentation-openai) | OpenInference Instrumentation for OpenAI SDK. | [![NPM Version](https://img.shields.io/npm/v/@arizeai/openinference-instrumentation-openai)](https://www.npmjs.com/package/@arizeai/openinference-instrumentation-openai) |
| [`@arizeai/openinference-instrumentation-anthropic`](./js/packages/openinference-instrumentation-anthropic) | OpenInference Instrumentation for the Anthropic SDK. | [![NPM Version](https://img.shields.io/npm/v/@arizeai/openinference-instrumentation-anthropic)](https://www.npmjs.com/package/@arizeai/openinference-instrumentation-anthropic) |
| [`@arizeai/openinference-instrumentation-claude-agent-sdk`](./js/packages/openinference-instrumentation-claude-agent-sdk) | OpenInference Instrumentation for Claude Agent SDK. | [![NPM Version](https://img.shields.io/npm/v/@arizeai/openinference-instrumentation-claude-agent-sdk)](https://www.npmjs.com/package/@arizeai/openinference-instrumentation-claude-agent-sdk) |
| [`@arizeai/openinference-vercel`](./js/packages/openinference-vercel) | OpenInference Support for Vercel AI SDK. | [![NPM Version](https://img.shields.io/npm/v/@arizeai/openinference-vercel)](https://www.npmjs.com/package/@arizeai/openinference-vercel) |
| [`@arizeai/openinference-tanstack-ai`](./js/packages/openinference-tanstack-ai) | OpenInference middleware for TanStack AI. | [![NPM Version](https://img.shields.io/npm/v/@arizeai/openinference-tanstack-ai)](https://www.npmjs.com/package/@arizeai/openinference-tanstack-ai) |
| [`@arizeai/openinference-genai`](./js/packages/openinference-genai) | OpenInference Support for GenAI conventions | [![NPM Version](https://img.shields.io/npm/v/@arizeai/openinference-genai)](https://www.npmjs.com/package/@arizeai/openinference-genai) |

## Java

### Libraries

| Package | Description | Version |
| --------------------------------------------------------------------------------------------------------------- | ---------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [`openinference-semantic-conventions`](./java/openinference-semantic-conventions) | Semantic conventions for tracing of LLM Apps. | [![Maven Central Version](https://img.shields.io/maven-central/v/com.arize/openinference-semantic-conventions)](https://central.sonatype.com/artifact/com.arize/openinference-semantic-conventions) |
| [`openinference-instrumentation`](./java/openinference-instrumentation) | Base instrumentation utilities. | [![Maven Central Version](https://img.shields.io/maven-central/v/com.arize/openinference-instrumentation)](https://central.sonatype.com/artifact/com.arize/openinference-instrumentation) |
| [`openinference-instrumentation-langchain4j`](./java/instrumentation/openinference-instrumentation-langchain4j) | OpenInference Instrumentation for LangChain4j. | [![Maven Central Version](https://img.shields.io/maven-central/v/com.arize/openinference-instrumentation-langchain4j)](https://central.sonatype.com/artifact/com.arize/openinference-instrumentation-langchain4j) |
| [`openinference-instrumentation-springAI`](./java/instrumentation/openinference-instrumentation-springAI) | OpenInference Instrumentation for Spring AI. | [![Maven Central Version](https://img.shields.io/maven-central/v/com.arize/openinference-instrumentation-springAI)](https://central.sonatype.com/artifact/com.arize/openinference-instrumentation-springAI) |
| [`openinference-instrumentation-annotation`](./java/instrumentation/openinference-instrumentation-annotation) | Annotation-based manual tracing with ByteBuddy. | [![Maven Central Version](https://img.shields.io/maven-central/v/com.arize/openinference-instrumentation-annotation)](https://central.sonatype.com/artifact/com.arize/openinference-instrumentation-annotation) |

### Examples

| Name | Description | Complexity Level |
| -------------------------------------------------------- | ---------------------------------------------- | ---------------- |
| [LangChain4j Example](java/examples/langchain4j-example) | Simple example using LangChain4j with OpenAI | Beginner |
| [Spring AI Example](java/examples/spring-ai-example) | Spring AI example with OpenAI and tool calling | Beginner |
| [Annotation Example](java/examples/annotation-example) | Annotation-based tracing with `@Chain`, `@LLM`, `@Tool`, `@Agent` | Beginner |
| [Programmatic Example](java/examples/programmatic-example) | Manual tracing with typed span classes (`LLMSpan`, `AgentSpan`, etc.) | Beginner |

## Supported Destinations

OpenInference supports the following destinations as span collectors.

- βœ… [Arize Phoenix](https://github.com/Arize-ai/phoenix)
- βœ… [Arize AX](https://arize.com/)
- βœ… Any OTEL-compatible collector

## Community

Join our community to connect with thousands of machine learning practitioners and LLM observability enthusiasts!

- 🌍 Join our [Slack community](https://join.slack.com/t/arize-ai/shared_invite/zt-3lqwr2oc3-7rhdyYEh82zJL_UhPKrb0A).
- πŸ’‘ Ask questions and provide feedback in the _#phoenix-support_ channel.
- 🌟 Leave a star on our [GitHub](https://github.com/Arize-ai/openinference).
- 🐞 Report bugs with [GitHub Issues](https://github.com/Arize-ai/openinference/issues).
- 𝕏 Follow us on [X](https://twitter.com/ArizePhoenix).
- πŸ—ΊοΈ Check out our [roadmap](https://github.com/orgs/Arize-ai/projects/45) to see where we're heading next.