{"id":33902237,"url":"https://github.com/lostbean/agent_obs","last_synced_at":"2026-03-17T16:38:36.579Z","repository":{"id":320442697,"uuid":"1081758708","full_name":"lostbean/agent_obs","owner":"lostbean","description":"An Elixir library for LLM agent observability","archived":false,"fork":false,"pushed_at":"2026-03-11T16:56:56.000Z","size":292,"stargazers_count":9,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-03-11T21:58:05.421Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lostbean.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-23T08:44:45.000Z","updated_at":"2026-03-11T16:56:45.000Z","dependencies_parsed_at":"2026-02-25T22:03:54.574Z","dependency_job_id":null,"html_url":"https://github.com/lostbean/agent_obs","commit_stats":null,"previous_names":["lostbean/agent_obs"],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/lostbean/agent_obs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lostbean%2Fagent_obs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lostbean%2Fagent_obs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lostbean%2Fagent_obs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lostbean%2Fagent_obs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lostbean","download_url":"https://codeload.github.com/lostbean/agent_obs/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lostbean%2Fagent_obs/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30627235,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-17T14:16:03.965Z","status":"ssl_error","status_checked_at":"2026-03-17T14:16:03.380Z","response_time":56,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-12-12T00:15:29.243Z","updated_at":"2026-03-17T16:38:36.572Z","avatar_url":"https://github.com/lostbean.png","language":"Elixir","funding_links":[],"categories":["Generative AI"],"sub_categories":["Development Tools"],"readme":"# AgentObs\n\n[![Hex.pm](https://img.shields.io/hexpm/v/agent_obs.svg)](https://hex.pm/packages/agent_obs)\n[![Documentation](https://img.shields.io/badge/docs-hexdocs-blue.svg)](https://hexdocs.pm/agent_obs)\n[![CI](https://github.com/lostbean/agent_obs/workflows/CI/badge.svg)](https://github.com/lostbean/agent_obs/actions)\n\n**An Elixir library for LLM agent observability.**\n\nAgentObs provides a simple, powerful, and idiomatic interface for instrumenting\nLLM agentic applications with telemetry events. It supports multiple\nobservability backends through a pluggable handler architecture.\n\n## Features\n\n- 🎯 **High-level instrumentation helpers** - `trace_agent/3`, `trace_tool/3`,\n  `trace_llm/3`, `trace_prompt/3`\n- 🤖 **ReqLLM integration helpers (optional)** - Automatic instrumentation for\n  ReqLLM with token tracking and streaming support\n- 🔌 **Pluggable backend architecture** - Support for multiple observability\n  platforms\n- 🌟 **OpenInference support** - Full semantic conventions for Arize Phoenix\n- 📊 **Rich metadata tracking** - Token usage, costs, tool calls, and more\n- 🚀 **Built on OTP** - Supervised handlers with fault tolerance\n- 🔗 **Jido integration (optional)** - Zero-code tracing for Jido composer\n  workflows\n- 🧪 **Backend-agnostic** - Standardized event schema independent of backends\n\n## Architecture\n\nAgentObs uses a two-layer architecture:\n\n**Layer 1: Core Telemetry API (Backend-Agnostic)**\n\n- Leverages Elixir's native `:telemetry` ecosystem\n- Provides high-level helpers for instrumenting agent operations\n- Defines standardized event schemas\n\n**Layer 2: Pluggable Backend Handlers**\n\n- Phoenix handler with OpenInference semantic conventions\n- Generic OpenTelemetry handler\n- Extensible to other platforms (Langfuse, Datadog, etc.)\n\n## Installation\n\nAdd `agent_obs` to your list of dependencies in `mix.exs`:\n\n```elixir\ndef deps do\n  [\n    {:agent_obs, \"~\u003e 0.1.0\"}\n  ]\nend\n```\n\n## Quick Start\n\n### 1. Configure AgentObs\n\n```elixir\n# config/config.exs\nconfig :agent_obs,\n  enabled: true,\n  handlers: [AgentObs.Handlers.Phoenix]\n\n# config/runtime.exs (for Arize Phoenix)\nconfig :opentelemetry,\n  span_processor: :batch,\n  resource: [service: [name: \"my_llm_agent\"]]\n\nconfig :opentelemetry_exporter,\n  otlp_protocol: :http_protobuf,\n  otlp_endpoint: System.get_env(\"ARIZE_PHOENIX_OTLP_ENDPOINT\", \"http://localhost:6006\"),\n  otlp_headers: []\n# Note: /v1/traces is automatically appended by the exporter\n```\n\n### 2. Instrument Your Agent\n\n```elixir\ndefmodule MyApp.WeatherAgent do\n  def get_forecast(city) do\n    AgentObs.trace_agent(\"weather_forecast\", %{input: \"What's the weather in #{city}?\"}, fn -\u003e\n      # Call LLM to determine tool to use\n      {:ok, tool_call, _metadata} = call_llm_for_planning(city)\n\n      # Execute the tool\n      {:ok, weather_data} = AgentObs.trace_tool(\"get_weather_api\", %{\n        arguments: %{city: city}\n      }, fn -\u003e\n        {:ok, %{temp: 72, condition: \"sunny\"}}\n      end)\n\n      # Return final result\n      {:ok, \"The weather in #{city} is #{weather_data.condition}\", %{\n        tools_used: [\"get_weather_api\"],\n        iterations: 1\n      }}\n    end)\n  end\n\n  defp call_llm_for_planning(city) do\n    AgentObs.trace_llm(\"gpt-4o\", %{\n      input_messages: [%{role: \"user\", content: \"Get weather for #{city}\"}]\n    }, fn -\u003e\n      # Simulate LLM API call\n      response = call_openai(...)\n\n      {:ok, response, %{\n        output_messages: [%{role: \"assistant\", content: response}],\n        tokens: %{prompt: 50, completion: 25, total: 75},\n        cost: 0.00012\n      }}\n    end)\n  end\nend\n```\n\n### 3. View Traces in Arize Phoenix\n\nStart a local Phoenix instance:\n\n```bash\ndocker run -p 6006:6006 -p 4317:4317 arizephoenix/phoenix:latest\n```\n\nNavigate to `http://localhost:6006` to view your traces with:\n\n- Rich chat message visualization\n- Token usage and cost tracking\n- Tool call inspection\n- Nested span relationships\n\n## Handlers\n\n### Phoenix Handler (OpenInference)\n\nTranslates events to OpenInference semantic conventions for Arize Phoenix:\n\n```elixir\nconfig :agent_obs,\n  handlers: [AgentObs.Handlers.Phoenix]\n```\n\n### Generic Handler (Basic OpenTelemetry)\n\nCreates basic OpenTelemetry spans without OpenInference:\n\n```elixir\nconfig :agent_obs,\n  handlers: [AgentObs.Handlers.Generic]\n```\n\n### Multiple Handlers\n\nUse multiple backends simultaneously:\n\n```elixir\nconfig :agent_obs,\n  handlers: [\n    AgentObs.Handlers.Phoenix,  # For detailed LLM observability\n    AgentObs.Handlers.Generic   # For APM integration\n  ]\n```\n\n## ReqLLM Integration (Optional)\n\nFor applications using [ReqLLM](https://hexdocs.pm/req_llm), AgentObs provides\nhigh-level helpers that automatically instrument LLM calls with full\nobservability:\n\n```elixir\n# Add to your deps\n{:req_llm, \"~\u003e 1.0.0-rc.7\"}\n\n# Non-streaming text generation\n{:ok, response} =\n  AgentObs.ReqLLM.trace_generate_text(\n    \"anthropic:claude-3-5-sonnet\",\n    [%{role: \"user\", content: \"Hello!\"}]\n  )\n\ntext = ReqLLM.Response.text(response)\n\n# Streaming text generation\n{:ok, stream_response} =\n  AgentObs.ReqLLM.trace_stream_text(\n    \"anthropic:claude-3-5-sonnet\",\n    [%{role: \"user\", content: \"Tell me a story\"}]\n  )\n\nstream_response.stream\n|\u003e Stream.filter(\u0026(\u00261.type == :content))\n|\u003e Stream.each(\u0026IO.write(\u00261.text))\n|\u003e Stream.run()\n\n# Structured data generation\nschema = [name: [type: :string, required: true], age: [type: :pos_integer]]\n\n{:ok, response} =\n  AgentObs.ReqLLM.trace_generate_object(\n    \"anthropic:claude-3-5-sonnet\",\n    [%{role: \"user\", content: \"Generate a person\"}],\n    schema\n  )\n\nobject = ReqLLM.Response.object(response)\n#=\u003e %{name: \"Alice\", age: 30}\n```\n\n**Benefits:**\n\n- Automatic token usage extraction\n- Automatic tool call parsing\n- Works across all ReqLLM providers (Anthropic, OpenAI, Google, etc.)\n- Supports both streaming and non-streaming\n- Structured data generation with schema validation\n- Bang variants (`!`) for convenience\n\nSee the [demo agent](demo/lib/demo/agent.ex) and\n[ReqLLM integration guide](guides/req_llm_integration.md) for complete examples.\n\n## Jido Integration (Optional)\n\nFor applications using [Jido](https://hexdocs.pm/jido), AgentObs provides\n`AgentObs.JidoTracer` — a drop-in `Jido.Observe.Tracer` implementation that\nautomatically instruments all composer events with OpenTelemetry spans.\n\n```elixir\n# Add to your deps\n{:jido, \"~\u003e 2.0\"}\n\n# Configure Jido to use the tracer\nconfig :jido, :observability,\n  tracer: AgentObs.JidoTracer\n```\n\nThat's it. All `[:jido, :composer, :agent|:llm|:tool]` events are automatically\nmapped to AgentObs event types and traced with OpenInference semantic conventions.\nParent-child span nesting is preserved, so you get a full trace tree in Phoenix:\n\n```\nweather_assistant (agent)\n  ├── gpt-4o #1 (llm)\n  ├── get_weather (tool)\n  └── gpt-4o #2 (llm)\n```\n\nSee the [Jido integration guide](guides/jido_integration.md) for details on\nevent mapping, metadata translation, and advanced usage.\n\n## API Reference\n\n### High-Level Instrumentation\n\n- **`trace_agent/3`** - Instruments agent loops or invocations\n- **`trace_tool/3`** - Instruments tool calls\n- **`trace_llm/3`** - Instruments LLM API calls\n- **`trace_prompt/3`** - Instruments prompt template rendering\n\n### ReqLLM Helpers (Optional)\n\n**Text Generation:**\n\n- **`AgentObs.ReqLLM.trace_generate_text/3`** - Non-streaming text generation\n- **`AgentObs.ReqLLM.trace_generate_text!/3`** - Non-streaming (bang variant)\n- **`AgentObs.ReqLLM.trace_stream_text/3`** - Streaming text generation\n\n**Structured Data Generation:**\n\n- **`AgentObs.ReqLLM.trace_generate_object/4`** - Non-streaming structured data\n- **`AgentObs.ReqLLM.trace_generate_object!/4`** - Non-streaming (bang variant)\n- **`AgentObs.ReqLLM.trace_stream_object/4`** - Streaming structured data\n\n**Tool Execution:**\n\n- **`AgentObs.ReqLLM.trace_tool_execution/3`** - Instrumented tool execution\n\n**Stream Helpers:**\n\n- **`AgentObs.ReqLLM.collect_stream/1`** - Collect text stream with metadata\n- **`AgentObs.ReqLLM.collect_stream_object/1`** - Collect object stream with\n  metadata\n\n### Low-Level API\n\n- **`emit/2`** - Emits custom telemetry events\n- **`configure/1`** - Runtime configuration updates\n\nSee the [full documentation](https://hexdocs.pm/agent_obs) for detailed API\nreference and examples.\n\n## Testing\n\n### Running Tests\n\n```bash\n# Run all tests (unit tests only, 99 tests)\nmix test\n\n# Include integration tests (requires API keys)\nmix test --include integration\n\n# Run only integration tests\nmix test --only integration\n```\n\n### ReqLLM Integration Tests\n\nThe ReqLLM module includes comprehensive test coverage with 193 tests:\n\n**Unit Tests (185 tests)** - Run by default, use mocked streams:\n\n- Stream text and object collection\n- Tool call extraction and argument parsing\n- Token usage extraction\n- Function signature validation\n- Error handling (malformed JSON, missing data)\n- Edge cases (nil values, partial data, multiple fragments)\n- All generate_text, generate_object, and stream_object variants\n\n**Integration Tests (8 tests)** - Excluded by default, require real LLM API\ncalls:\n\n- Real LLM streaming with telemetry verification\n- Real non-streaming text generation\n- Real structured data generation (objects)\n- Real streaming object generation\n- Real tool execution with instrumentation\n- Full agent loop with streaming and tools\n- Bang variants (`!`) with real API calls\n\nTo run integration tests, set one of these environment variables:\n\n```bash\nexport ANTHROPIC_API_KEY=your_key  # Uses claude-3-5-haiku-latest\n# OR\nexport OPENAI_API_KEY=your_key     # Uses gpt-4o-mini\n# OR\nexport GOOGLE_API_KEY=your_key     # Uses gemini-2.0-flash-exp\n\nmix test --include integration\n```\n\nIf no API key is configured, integration tests gracefully skip without failing.\n\n## Development\n\n### Quick Commands\n\n```bash\n# Install dependencies\nmix deps.get\n\n# Run pre-commit checks (format, test, credo)\nmix precommit\n\n# Run CI checks (format check, test, credo)\nmix ci\n```\n\n### Individual Commands\n\n```bash\n# Run tests\nmix test\n\n# Format code\nmix format\n\n# Check if code is formatted\nmix format --check-formatted\n\n# Run Credo (code quality)\nmix credo\n\n# Run Credo in strict mode\nmix credo --strict\n\n# Generate documentation\nmix docs\n\n# Run Dialyzer (type checking)\nmix dialyzer\n```\n\n### Pre-commit Hook\n\nFor automatic code quality checks before commits, you can run:\n\n```bash\nmix precommit\n```\n\nThis will:\n\n1. Format your code\n2. Run all tests\n3. Run Credo in strict mode\n\n### CI Pipeline\n\nThe `mix ci` command is designed for continuous integration and will:\n\n1. Check that code is properly formatted (fails if not)\n2. Run all tests\n3. Run Credo in strict mode\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n## License\n\nMIT License - see [LICENSE](LICENSE) file for details.\n\nCopyright (c) 2025 Edgar Gomes\n\n## References\n\n- [OpenInference Specification](https://arize-ai.github.io/openinference/spec/semantic_conventions.html)\n- [Arize Phoenix Documentation](https://arize.com/docs/phoenix/)\n- [OpenTelemetry Elixir](https://hexdocs.pm/opentelemetry/)\n- [Elixir Telemetry](https://hexdocs.pm/telemetry/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flostbean%2Fagent_obs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flostbean%2Fagent_obs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flostbean%2Fagent_obs/lists"}