{"id":30438390,"url":"https://github.com/azmaveth/ex_llm","last_synced_at":"2026-02-28T10:06:10.729Z","repository":{"id":295424779,"uuid":"989917979","full_name":"azmaveth/ex_llm","owner":"azmaveth","description":"All-in-one LLM library for Elixir with unified provider interface and structured outputs","archived":false,"fork":false,"pushed_at":"2025-07-07T03:11:37.000Z","size":4868,"stargazers_count":49,"open_issues_count":9,"forks_count":11,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-11-09T16:00:52.254Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/azmaveth.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-25T05:35:25.000Z","updated_at":"2025-11-03T00:09:25.000Z","dependencies_parsed_at":"2025-05-25T13:54:18.752Z","dependency_job_id":"dc66b9bf-0298-4118-9ba6-22593e42341c","html_url":"https://github.com/azmaveth/ex_llm","commit_stats":null,"previous_names":["azmaveth/ex_llm"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/azmaveth/ex_llm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/azmaveth%2Fex_llm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/azmaveth%2Fex_llm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/azmaveth%2Fex_llm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/azmaveth%2Fex_llm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/azmaveth","download_url":"https://codeload.github.com/azmaveth/ex_llm/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/azmaveth%2Fex_llm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29929641,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-28T09:58:13.507Z","status":"ssl_error","status_checked_at":"2026-02-28T09:57:57.047Z","response_time":90,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-23T04:02:31.598Z","updated_at":"2026-02-28T10:06:10.709Z","avatar_url":"https://github.com/azmaveth.png","language":"Elixir","funding_links":[],"categories":["LLM Clients and APIs","LLM Clients \u0026 SDKs","Generative AI"],"sub_categories":["How to Join","LLM Tools"],"readme":"# ExLLM\n\n\u003e **⚠️ DEPRECATED - NO LONGER MAINTAINED**\n\u003e\n\u003e **This library is no longer actively maintained.**\n\u003e\n\u003e **Please use [req_llm](https://github.com/agentjido/req_llm) by agentjido instead.**\n\u003e\n\u003e `req_llm` provides a modern, actively maintained alternative with similar functionality and better integration with the Elixir ecosystem.\n\n[![Hex.pm](https://img.shields.io/hexpm/v/ex_llm.svg)](https://hex.pm/packages/ex_llm)\n[![Hex Docs](https://img.shields.io/badge/hex-docs-lightgreen.svg)](https://hexdocs.pm/ex_llm/)\n[![License](https://img.shields.io/hexpm/l/ex_llm.svg)](https://github.com/azmaveth/ex_llm/blob/main/LICENSE)\n[![CI](https://github.com/azmaveth/ex_llm/actions/workflows/ci.yml/badge.svg)](https://github.com/azmaveth/ex_llm/actions/workflows/ci.yml)\n\n**A unified Elixir client for interfacing with multiple Large Language Model (LLM) providers.**\n\n`ExLLM` provides a single, consistent API to interact with a growing list of LLM providers. It abstracts away the complexities of provider-specific request formats, authentication, and error handling, allowing you to focus on building features.\n\n## Key Features\n\n- **Unified API:** Use a single `ExLLM.chat/2` interface for all supported providers, dramatically reducing boilerplate code\n- **Broad Provider Support:** Seamlessly switch between models from 14+ major providers\n- **Streaming Support:** Handle real-time responses for chat completions using Elixir's native streaming\n- **Standardized Error Handling:** Get predictable `{:error, reason}` tuples for common failure modes\n- **Session Management:** Built-in conversation state tracking and persistence\n- **Function Calling:** Unified tool use interface across providers that support it\n- **Multimodal Support:** Vision, audio, and document processing capabilities where available\n- **Minimal Overhead:** Designed as a thin, efficient client layer with focus on performance\n- **Extensible Architecture:** Adding new providers is straightforward through clean delegation patterns\n\n## Feature Status\n\n✅ **Production Ready:** Core chat, streaming, sessions, providers, function calling, cost tracking  \n🚧 **Under Development:** Context management, model capabilities API, configuration validation  \n\nSee [FEATURE_STATUS.md](FEATURE_STATUS.md) for detailed testing results and API status.\n\n## Supported Providers\n\nExLLM supports **14 providers** with access to hundreds of models:\n\n- **Anthropic Claude** - Claude 4, 3.7, 3.5, and 3 series models\n- **OpenAI** - GPT-4.1, o1 reasoning models, GPT-4o, and GPT-3.5 series\n- **AWS Bedrock** - Multi-provider access (Anthropic, Amazon Nova, Meta Llama, etc.)\n- **Google Gemini** - Gemini 2.5, 2.0, and 1.5 series with multimodal support\n- **OpenRouter** - Access to hundreds of models from multiple providers\n- **Groq** - Ultra-fast inference with Llama 4, DeepSeek R1, and more\n- **X.AI** - Grok models with web search and reasoning capabilities\n- **Mistral AI** - Mistral Large, Pixtral, and specialized code models\n- **Perplexity** - Search-enhanced language models\n- **Ollama** - Local model runner (any model in your installation)\n- **LM Studio** - Local model server with OpenAI-compatible API\n- **Bumblebee** - Local model inference with Elixir/Nx (optional dependency)\n- **Mock Adapter** - For testing and development\n\n## Installation\n\nAdd `ex_llm` to your list of dependencies in `mix.exs`:\n\n```elixir\ndef deps do\n  [\n    {:ex_llm, \"~\u003e 1.0.0-rc1\"},\n    \n    # Optional: For local model inference via Bumblebee\n    {:bumblebee, \"~\u003e 0.6.2\", optional: true},\n    {:nx, \"~\u003e 0.7\", optional: true},\n    \n    # Optional hardware acceleration backends (choose one):\n    {:exla, \"~\u003e 0.7\", optional: true},\n    \n    # Optional: For Apple Silicon Metal acceleration\n    {:emlx, github: \"elixir-nx/emlx\", branch: \"main\", optional: true}\n  ]\nend\n```\n\n## Quick Start\n\n### 1. Configuration\n\nSet your API keys as environment variables:\n\n```bash\nexport ANTHROPIC_API_KEY=\"your-anthropic-key\"\nexport OPENAI_API_KEY=\"your-openai-key\"\nexport GROQ_API_KEY=\"your-groq-key\"\n# ... other provider keys as needed\n```\n\n### 2. Basic Usage\n\n```elixir\n# Single completion\n{:ok, response} = ExLLM.chat(:anthropic, [\n  %{role: \"user\", content: \"Explain quantum computing in simple terms\"}\n])\n\nIO.puts(response.content)\n# Cost automatically tracked: response.cost\n\n# Streaming response\nExLLM.chat_stream(:openai, [\n  %{role: \"user\", content: \"Write a short story\"}\n], fn chunk -\u003e\n  IO.write(chunk.delta)\nend)\n\n# With session management\n{:ok, session} = ExLLM.Session.new(:groq)\n{:ok, session, response} = ExLLM.Session.chat(session, \"Hello!\")\n{:ok, session, response} = ExLLM.Session.chat(session, \"How are you?\")\n\n# Multimodal with vision\n{:ok, response} = ExLLM.chat(:gemini, [\n  %{role: \"user\", content: [\n    %{type: \"text\", text: \"What's in this image?\"},\n    %{type: \"image\", image: %{data: base64_image, media_type: \"image/jpeg\"}}\n  ]}\n])\n```\n\n## Configuration\n\nYou can configure providers in your `config/config.exs`:\n\n```elixir\nimport Config\n\nconfig :ex_llm,\n  default_provider: :openai,\n  providers: [\n    openai: [api_key: System.get_env(\"OPENAI_API_KEY\")],\n    anthropic: [api_key: System.get_env(\"ANTHROPIC_API_KEY\")],\n    gemini: [api_key: System.get_env(\"GEMINI_API_KEY\")]\n  ]\n```\n\n## Testing\n\nThe test suite includes both unit tests and integration tests. Integration tests that make live API calls are tagged and excluded by default.\n\nTo run unit tests only:\n```bash\nmix test\n```\n\nTo run integration tests (requires API keys):\n```bash\nmix test --include integration\n```\n\nTo run tests with intelligent caching for faster development:\n```bash\nmix test.live  # Runs with test response caching enabled\n```\n\n## Architecture\n\nExLLM uses a clean, modular architecture that separates concerns while maintaining a unified API:\n\n### Core Modules\n\n- **`ExLLM`** - Main entry point with unified API\n- **`ExLLM.API.Delegator`** - Central delegation engine for provider routing\n- **`ExLLM.API.Capabilities`** - Provider capability registry\n- **`ExLLM.Pipeline`** - Phoenix-style pipeline for request processing\n\n### Specialized Modules\n\n- **`ExLLM.Embeddings`** - Vector operations and similarity calculations\n- **`ExLLM.Assistants`** - OpenAI Assistants API for stateful agents\n- **`ExLLM.KnowledgeBase`** - Document management and semantic search\n- **`ExLLM.Builder`** - Fluent interface for chat construction\n- **`ExLLM.Session`** - Conversation state management\n\n### Benefits\n\n- **Clean Separation**: Each module has a single, focused responsibility\n- **Easy Extension**: Adding providers requires changes in just 1-2 files\n- **Performance**: Delegation adds minimal overhead\n- **Maintainability**: Clear boundaries between components\n\n## Documentation\n\n📚 **[Quick Start Guide](docs/QUICKSTART.md)** - Get up and running in 5 minutes  \n📖 **[User Guide](docs/USER_GUIDE.md)** - Comprehensive documentation of all features  \n🏗️ **[Architecture Guide](docs/ARCHITECTURE.md)** - Clean layered architecture and namespace organization  \n🔌 **[Pipeline Architecture](docs/PIPELINE_ARCHITECTURE.md)** - Phoenix-style plug system and extensibility  \n🔧 **[Logger Guide](docs/LOGGER.md)** - Debug logging and troubleshooting  \n⚡ **[Provider Capabilities](docs/PROVIDER_CAPABILITIES.md)** - Feature comparison across providers  \n🧪 **[Testing Guide](docs/TESTING.md)** - Comprehensive testing system with semantic tagging and caching\n\n### Key Topics Covered in the User Guide\n\n- **Configuration**: Environment variables, config files, and provider setup\n- **Chat Completions**: Messages, parameters, and response handling\n- **Streaming**: Real-time responses with error recovery and coordinator\n- **Session Management**: Conversation state and persistence\n- **Function Calling**: Tool use and structured interactions across providers\n- **Vision \u0026 Multimodal**: Image, audio, and document processing\n- **Cost Tracking**: Automatic cost calculation and token estimation\n- **Error Handling**: Retry logic and error recovery strategies\n- **Test Caching**: Intelligent response caching for faster development\n- **Model Discovery**: Query available models and capabilities\n- **OAuth2 Integration**: Complete OAuth2 flow for Gemini and other providers\n\n### Additional Documentation\n\n- 📋 **[Unified API Guide](docs/UNIFIED_API_GUIDE.md)** - Complete unified API documentation\n- 🔄 **[Migration Guide](MIGRATION_GUIDE_V1.md)** - Upgrading to v1.0.0\n- ✅ **[Release Checklist](RELEASE_CHECKLIST.md)** - Automated release process\n- 📚 **[API Reference](https://hexdocs.pm/ex_llm)** - Detailed API documentation on HexDocs\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## Support\n\n- 📖 **Documentation**: [User Guide](docs/USER_GUIDE.md)\n- 🐛 **Issues**: [GitHub Issues](https://github.com/azmaveth/ex_llm/issues)\n- 💬 **Discussions**: [GitHub Discussions](https://github.com/azmaveth/ex_llm/discussions)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fazmaveth%2Fex_llm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fazmaveth%2Fex_llm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fazmaveth%2Fex_llm/lists"}