{"id":15175662,"url":"https://github.com/lebrunel/ollama-ex","last_synced_at":"2025-04-12T16:38:20.219Z","repository":{"id":217007319,"uuid":"742939667","full_name":"lebrunel/ollama-ex","owner":"lebrunel","description":"A nifty little library for working with Ollama in Elixir.","archived":false,"fork":false,"pushed_at":"2025-01-08T14:13:31.000Z","size":125,"stargazers_count":105,"open_issues_count":0,"forks_count":6,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-03T22:07:11.825Z","etag":null,"topics":["ai","chatgpt-api","elixir","llms","localllm","ollama"],"latest_commit_sha":null,"homepage":"https://hexdocs.pm/ollama","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lebrunel.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-13T20:21:14.000Z","updated_at":"2025-03-27T00:20:10.000Z","dependencies_parsed_at":"2024-03-24T10:28:18.973Z","dependency_job_id":"6442caaf-e3fb-494e-a0b6-e1548e82a8f1","html_url":"https://github.com/lebrunel/ollama-ex","commit_stats":null,"previous_names":["lebrunel/ollama-ex"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lebrunel%2Follama-ex","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lebrunel%2Follama-ex/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lebrunel%2Follama-ex/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lebrunel%2Follama-ex/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lebrunel","download_url":"https://codeload.github.com/lebrunel/ollama-ex/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248597743,"owners_count":21130936,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","chatgpt-api","elixir","llms","localllm","ollama"],"created_at":"2024-09-27T12:39:50.862Z","updated_at":"2025-04-12T16:38:20.199Z","avatar_url":"https://github.com/lebrunel.png","language":"Elixir","funding_links":[],"categories":["LLM Clients and APIs","LLM Clients \u0026 SDKs","Generative AI"],"sub_categories":["How to Join","LLM Tools"],"readme":"# Ollama\n\n![Ollama-ex](https://raw.githubusercontent.com/lebrunel/ollama-ex/main/media/poster.webp)\n\n![Hex.pm](https://img.shields.io/hexpm/v/ollama?color=informational)\n![License](https://img.shields.io/github/license/lebrunel/ollama-ex?color=informational)\n![Build Status](https://img.shields.io/github/actions/workflow/status/lebrunel/ollama-ex/elixir.yml?branch=main)\n\n[Ollama](https://ollama.ai) is a powerful tool for running large language models locally or on your own infrastructure. This library provides an interface for working with Ollama in Elixir.\n\n- 🦙 Full implementation of the Ollama API\n- 🧰 Tool use (function calling)\n- 🧱 Structured outputs\n- 🛜 Streaming requests\n  - Stream to an Enumerable\n  - Or stream messages to any Elixir process\n\n## Installation\n\nThe package can be installed by adding `ollama` to your list of dependencies in `mix.exs`.\n\n```elixir\ndef deps do\n  [\n    {:ollama, \"~\u003e 0.8\"}\n  ]\nend\n```\n\n## Quickstart\n\nFor more examples, refer to the [Ollama documentation](https://hexdocs.pm/ollama).\n\n### 1. Generate a completion\n\n```elixir\nclient = Ollama.init()\n\nOllama.completion(client, [\n  model: \"llama2\",\n  prompt: \"Why is the sky blue?\",\n])\n# {:ok, %{\"response\" =\u003e \"The sky is blue because it is the color of the sky.\", ...}}\n```\n\n### 2. Generate the next message in a chat\n\n```elixir\nOllama.chat(client, [\n  model: \"llama2\",\n  messages: [\n    %{role: \"system\", content: \"You are a helpful assistant.\"},\n    %{role: \"user\", content: \"Why is the sky blue?\"},\n    %{role: \"assistant\", content: \"Due to rayleigh scattering.\"},\n    %{role: \"user\", content: \"How is that different than mie scattering?\"},\n  ]\n])\n# {:ok, %{\"message\" =\u003e %{\n#   \"role\" =\u003e \"assistant\",\n#   \"content\" =\u003e \"Mie scattering affects all wavelengths similarly, while Rayleigh favors shorter ones.\"\n# }, ...}}\n```\n\n### 3. Generate structured data\n\nThe `:format` option can be used with both `completion/2` and `chat/2`.\n\n```elixir\nOllama.completion(client, [\n  model: \"llama3.1\",\n  prompt: \"Tell me about Canada\",\n  format: %{\n    type: \"object\",\n    properties: %{\n      name: %{type: \"string\"},\n      capital: %{type: \"string\"},\n      languages: %{type: \"array\", items: %{type: \"string\"}},\n    },\n    required: [\"name\", \"capital\", \"languages\"]\n  }\n])\n# {:ok, %{\"response\" =\u003e \"{ \\\"name\\\": \\\"Canada\\\" ,\\\"capital\\\": \\\"Ottawa\\\" ,\\\"languages\\\": [\\\"English\\\", \\\"French\\\"] }\", ...}}\n```\n\n## Streaming\n\nStreaming is supported on certain endpoints by setting the `:stream` option to `true` or a `t:pid/0`.\n\nWhen `:stream` is set to `true`, a lazy `t:Enumerable.t/0` is returned, which can be used with any `Stream` functions.\n\n```elixir\n{:ok, stream} = Ollama.completion(client, [\n  model: \"llama2\",\n  prompt: \"Why is the sky blue?\",\n  stream: true,\n])\n\nstream\n|\u003e Stream.each(\u0026 Process.send(pid, \u00261, [])\n|\u003e Stream.run()\n# :ok\n```\n\nThis approach above builds the `t:Enumerable.t/0` by calling `receive`, which may cause issues in `GenServer` callbacks. As an alternative, you can set the `:stream` option to a `t:pid/0`. This returns a `t:Task.t/0` that sends messages to the specified process.\n\nThe following example demonstrates a streaming request in a LiveView event, sending each streaming message back to the same LiveView process:\n\n```elixir\ndefmodule MyApp.ChatLive do\n  use Phoenix.LiveView\n\n  # When the client invokes the \"prompt\" event, create a streaming request and\n  # asynchronously send messages back to self.\n  def handle_event(\"prompt\", %{\"message\" =\u003e prompt}, socket) do\n    {:ok, task} = Ollama.completion(Ollama.init(), [\n      model: \"llama2\",\n      prompt: prompt,\n      stream: self(),\n    ])\n\n    {:noreply, assign(socket, current_request: task)}\n  end\n\n  # The streaming request sends messages back to the LiveView process.\n  def handle_info({_request_pid, {:data, _data}} = message, socket) do\n    pid = socket.assigns.current_request.pid\n    case message do\n      {^pid, {:data, %{\"done\" =\u003e false} = data}} -\u003e\n        # handle each streaming chunk\n\n      {^pid, {:data, %{\"done\" =\u003e true} = data}} -\u003e\n        # handle the final streaming chunk\n\n      {_pid, _data} -\u003e\n        # this message was not expected!\n    end\n  end\n\n  # Tidy up when the request is finished\n  def handle_info({ref, {:ok, %Req.Response{status: 200}}}, socket) do\n    Process.demonitor(ref, [:flush])\n    {:noreply, assign(socket, current_request: nil)}\n  end\nend\n```\n\nRegardless of the streaming approach used, each streaming message is a plain `t:map/0`. For the message schema, refer to the [Ollama API docs](https://github.com/ollama/ollama/blob/main/docs/api.md).\n\n## Function calling\n\nOllama 0.3 and later versions support tool use and function calling on compatible models. Note that Ollama currently doesn't support tool use with streaming requests, so avoid setting `:stream` to `true`.\n\nUsing tools typically involves at least two round-trip requests to the model. Begin by defining one or more tools using a schema similar to ChatGPT's. Provide clear and concise descriptions for the tool and each argument.\n\n```elixir\nstock_price_tool = %{\n  type: \"function\",\n  function: %{\n    name: \"get_stock_price\",\n    description: \"Fetches the live stock price for the given ticker.\",\n    parameters: %{\n      type: \"object\",\n      properties: %{\n        ticker: %{\n          type: \"string\",\n          description: \"The ticker symbol of a specific stock.\"\n        }\n      },\n      required: [\"ticker\"]\n    }\n  }\n}\n```\n\nThe first round-trip involves sending a prompt in a chat with the tool definitions. The model should respond with a message containing a list of tool calls.\n\n```elixir\nOllama.chat(client, [\n  model: \"mistral-nemo\",\n  messages: [\n    %{role: \"user\", content: \"What is the current stock price for Apple?\"}\n  ],\n  tools: [stock_price_tool],\n])\n# {:ok, %{\"message\" =\u003e %{\n#   \"role\" =\u003e \"assistant\",\n#   \"content\" =\u003e \"\",\n#   \"tool_calls\" =\u003e [\n#     %{\"function\" =\u003e %{\n#       \"name\" =\u003e \"get_stock_price\",\n#       \"arguments\" =\u003e %{\"ticker\" =\u003e \"AAPL\"}\n#     }}\n#   ]\n# }, ...}}\n```\n\nYour implementation must intercept these tool calls and execute a corresponding function in your codebase with the specified arguments. The next round-trip involves passing the function's result back to the model as a message with a `:role` of `\"tool\"`.\n\n```elixir\nOllama.chat(client, [\n  model: \"mistral-nemo\",\n  messages: [\n    %{role: \"user\", content: \"What is the current stock price for Apple?\"},\n    %{role: \"assistant\", content: \"\", tool_calls: [%{\"function\" =\u003e %{\"name\" =\u003e \"get_stock_price\", \"arguments\" =\u003e %{\"ticker\" =\u003e \"AAPL\"}}}]},\n    %{role: \"tool\", content: \"$217.96\"},\n  ],\n  tools: [stock_price_tool],\n])\n# {:ok, %{\"message\" =\u003e %{\n#   \"role\" =\u003e \"assistant\",\n#   \"content\" =\u003e \"The current stock price for Apple (AAPL) is approximately $217.96.\",\n# }, ...}}\n```\n\nAfter receiving the function tool's value, the model will respond to the user's original prompt, incorporating the function result into its response.\n\n## License\n\nThis package is open source and released under the [Apache-2 License](https://github.com/lebrunel/ollama/blob/master/LICENSE).\n\n© Copyright 2024 [Push Code Ltd](https://www.pushcode.com/).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flebrunel%2Follama-ex","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flebrunel%2Follama-ex","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flebrunel%2Follama-ex/lists"}