https://github.com/doofinder/llm_composer
An Elixir library for integrating large language models (LLMs) via HTTP, supporting OpenAI, Ollama, and other future backends.
https://github.com/doofinder/llm_composer
Last synced: about 1 month ago
JSON representation
An Elixir library for integrating large language models (LLMs) via HTTP, supporting OpenAI, Ollama, and other future backends.
- Host: GitHub
- URL: https://github.com/doofinder/llm_composer
- Owner: doofinder
- License: mit
- Created: 2024-09-19T13:45:22.000Z (over 1 year ago)
- Default Branch: master
- Last Pushed: 2026-04-29T14:47:43.000Z (about 1 month ago)
- Last Synced: 2026-04-29T16:31:44.168Z (about 1 month ago)
- Language: Elixir
- Homepage: https://hex.pm/packages/llm_composer
- Size: 281 KB
- Stars: 38
- Watchers: 6
- Forks: 2
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
- Agents: AGENTS.md
Awesome Lists containing this project
- awesome-elixir-ai - LLM Composer - An Elixir library for integrating and managing multiple LLM backends including OpenAI, Ollama, and Google Gemini. (LLM Clients & SDKs / How to Join)
- awesome-elixir-llm-genai - LLMComposer - A streamlined way to build OpenAI and Ollama applications with auto-execution of functions. (LLM Clients and APIs)
- awesome-ml-gen-ai-elixir - LLM Composer - Multi-provider LLM library with routing, fallback, streaming, and cost tracking for OpenAI, Anthropic, Gemini, and more. (Generative AI / LLM Tools)
README
# LlmComposer
**LlmComposer** is an Elixir library that simplifies interaction with large language models (LLMs). It provides a unified interface to OpenAI (Chat Completions and Responses API), OpenRouter, Ollama, AWS Bedrock, and Google (Gemini), with support for streaming, function calls, structured outputs, cost tracking, and multi-provider failover routing.
## Table of Contents
- [Installation](#installation)
- [Tesla Configuration](#tesla-configuration)
- [Provider Compatibility](#provider-compatibility)
- [Usage](#usage)
- [Simple Chat](#simple-chat)
- [Using Message History](#using-message-history)
- [Providers](#providers)
- [Full Documentation](#full-documentation)
## Installation
Add `llm_composer` to your dependencies in `mix.exs`:
```elixir
def deps do
[
{:llm_composer, "~> 0.19"}
]
end
```
## Tesla Configuration
LlmComposer uses Tesla for HTTP. The default adapter is `Tesla.Adapter.Mint`, which supports
streaming out of the box. Finch is also supported if you prefer its connection pooling:
```elixir
# config/config.exs
config :llm_composer, :tesla_adapter, {Tesla.Adapter.Finch, name: MyApp.Finch}
# application.ex supervision tree
{Finch, name: MyApp.Finch}
```
You can also customize the JSON engine (defaults to `JSON`, falls back to `Jason`):
```elixir
config :llm_composer, :json_engine, Jason
```
## Provider Compatibility
| Feature | OpenAI | OpenRouter | Ollama | Bedrock | Google |
|---|---|---|---|---|---|
| Basic Chat | ✅ | ✅ | ✅ | ✅ | ✅ |
| Streaming | ✅ | ✅ | ✅ | ✅ | ✅ |
| Function Calls | ✅ | ✅ | ⚠️¹ | ✅ | ✅ |
| Structured Outputs | ✅ | ✅ | ⚠️¹ | ✅ | ✅ |
| Cost Tracking | ✅ | ✅ | ❌ | ✅ | ✅ |
| Fallback Models | ❌ | ✅ | ❌ | ❌ | ❌ |
| Provider Routing | ❌ | ✅ | ❌ | ❌ | ❌ |
¹ Via Ollama's OpenAI-compatible endpoint — see the [Providers guide](https://hexdocs.pm/llm_composer/providers.html).
## Usage
### Simple Chat
```elixir
Application.put_env(:llm_composer, :open_ai, api_key: "")
settings = %LlmComposer.Settings{
providers: [
{LlmComposer.Providers.OpenAI, [model: "gpt-4.1-mini"]}
],
system_prompt: "You are a helpful assistant."
}
{:ok, res} = LlmComposer.simple_chat(settings, "hi")
IO.inspect(res.main_response)
```
### Using Message History
For multi-turn conversations, use `run_completion/2` with an explicit message list:
```elixir
messages = [
LlmComposer.Message.new(:user, "What is the Roman Empire?"),
LlmComposer.Message.new(:assistant, "The Roman Empire was a period of ancient Roman civilization."),
LlmComposer.Message.new(:user, "When did it begin?")
]
{:ok, res} = LlmComposer.run_completion(settings, messages)
IO.inspect(res.main_response)
```
### Providers
All five providers share the same interface. Quick references:
| Provider | Setup |
|---|---|
| OpenAI | `Application.put_env(:llm_composer, :open_ai, api_key: "...")` |
| OpenRouter | `Application.put_env(:llm_composer, :open_router, api_key: "...")` |
| Ollama | No API key — start Ollama server locally |
| AWS Bedrock | Configure via ExAws |
| Google | `Application.put_env(:llm_composer, :google, api_key: "...")` or Goth/Vertex |
See the [Providers guide](https://hexdocs.pm/llm_composer/providers.html) for full examples,
Vertex AI setup, OpenAI-compatible servers, and provider-specific options.
## Full Documentation
Complete reference documentation is available on [HexDocs](https://hexdocs.pm/llm_composer):
- [Providers](https://hexdocs.pm/llm_composer/providers.html) — per-provider setup, Vertex AI, OpenAI-compatible servers, structured outputs, custom request params
- [Streaming](https://hexdocs.pm/llm_composer/streaming.html) — Finch setup, StreamChunk fields, token tracking
- [Function Calls](https://hexdocs.pm/llm_composer/function_calls.html) — 3-step manual function call workflow, FunctionExecutor API
- [Provider Router](https://hexdocs.pm/llm_composer/provider_router.html) — multi-provider failover with exponential backoff
- [Cost Tracking](https://hexdocs.pm/llm_composer/cost_tracking.html) — automatic and manual pricing, ETS cache setup
- [Custom Providers](https://hexdocs.pm/llm_composer/custom_provider.html) — implementing the Provider behaviour
- [Configuration](https://hexdocs.pm/llm_composer/configuration.html) — all global options, retry configuration