{"id":51227881,"url":"https://github.com/r-uby-dev/llm","last_synced_at":"2026-07-16T23:01:20.375Z","repository":{"id":258010735,"uuid":"867137291","full_name":"r-uby-dev/llm","owner":"r-uby-dev","description":"Ruby's capable AI runtime","archived":false,"fork":false,"pushed_at":"2026-07-14T23:58:37.000Z","size":139996,"stargazers_count":136,"open_issues_count":0,"forks_count":6,"subscribers_count":7,"default_branch":"main","last_synced_at":"2026-07-15T00:28:34.070Z","etag":null,"topics":["a2a","a2a-client","agent","agent2agent","agents","ai","ai-runtime","llm","llm-agents","llm-framework","llm-frameworks","llms","mcp","mcp-client","rag","ruby","ruby-lib","ruby-library"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/r-uby-dev.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-10-03T14:12:21.000Z","updated_at":"2026-07-14T23:58:39.000Z","dependencies_parsed_at":"2025-05-09T02:22:09.728Z","dependency_job_id":"ba77c54a-d162-4f0d-b4fc-92e39b2ca46f","html_url":"https://github.com/r-uby-dev/llm","commit_stats":null,"previous_names":["antaz/llm","llmrb/llm","llmrb/llm.rb","r-uby-dev/llm.rb","r-uby-dev/llm"],"tags_count":95,"template":false,"template_full_name":null,"purl":"pkg:github/r-uby-dev/llm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/r-uby-dev%2Fllm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/r-uby-dev%2Fllm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/r-uby-dev%2Fllm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/r-uby-dev%2Fllm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/r-uby-dev","download_url":"https://codeload.github.com/r-uby-dev/llm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/r-uby-dev%2Fllm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35560452,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-16T02:00:06.687Z","response_time":83,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["a2a","a2a-client","agent","agent2agent","agents","ai","ai-runtime","llm","llm-agents","llm-framework","llm-frameworks","llms","mcp","mcp-client","rag","ruby","ruby-lib","ruby-library"],"created_at":"2026-06-28T13:00:35.994Z","updated_at":"2026-07-16T23:01:20.369Z","avatar_url":"https://github.com/r-uby-dev.png","language":"Ruby","funding_links":[],"categories":["Ruby"],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://r.uby.dev\"\u003e\n    \u003cimg\n      src=\"https://github.com/r-uby-dev/llm.rb/raw/main/rubydev.svg\"\n      width=\"400\"\n      height=\"200\"\n      border=\"0\"\n      alt=\"a r.uby.dev project\"\n     \u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n\u003e A [r.uby.dev](https://r.uby.dev/llm) project.\n\nWelcome to the canonical llm.rb repository.\n\nllm.rb is an advanced runtime for building capable AI applications\non CRuby. By default it has zero runtime dependencies although certain\nfunctionality \u0026ndash; such as ActiveRecord support \u0026ndash; require\noptional dependencies that are opt-in.\n\n## Features\n\nThe runtime supports OpenAI, OpenAI-compatible endpoints, Anthropic, Google\nGemini, Mistral, DeepSeek, DeepInfra, xAI, Z.ai, AWS Bedrock, Ollama, and llama.cpp.\nIt has first-class support for streaming, tool calls,  MCP\nand A2A, embeddings, vector stores and the RAG pattern.\n\nThere are multiple HTTP backends to choose from, tools can be run concurrently\nor in parallel via threads, async tasks, fibers, ractors, and fork, and it is\nalso possible to make a tool call while the model is still streaming.\n\nThe runtime builds on top of three core concepts: providers, contexts, and agents,\nso once you learn the fundamentals, everything else falls into place naturally. And once\nyou learn llm.rb, you will also be able to use \u003ca href=\"https://r.uby.dev/mruby-llm\"\u003emruby-llm\u003c/a\u003e and\n\u003ca href=\"https://r.uby.dev/wasm-llm\"\u003ewasm-llm\u003c/a\u003e because the API is pretty much identical.\n\n## Install\n\n```bash\ngem install llm.rb\n```\n\n## Quick start\n\n#### LLM::Agent\n\nThe [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html) class is the default high-level interface,\nand it is recommended for most use-cases. It manages tool execution\nautomatically, guards against infinite loops, manages conversation\nstate, and much more.\n\n```ruby\nrequire \"llm\"\n\nllm = LLM.deepseek(key: ENV[\"KEY\"])\nagent = LLM::Agent.new(llm, stream: $stdout)\nagent.talk \"Hello world\"\n```\n\n#### LLM::Context\n\nThe [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html) class is at the heart of the runtime\nand it is what [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html) uses under the hood.\nIt requires that the tool call loop be managed manually -\nsometimes that can be useful, but usually for advanced use-cases.\nIf you're new to llm.rb, try [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html) first.\n\n```ruby\nrequire \"llm\"\n\nllm = LLM.deepseek(key: ENV[\"KEY\"])\nctx = LLM::Context.new(llm, stream: $stdout)\nctx.talk \"Hello world\"\n```\n\n#### LLM::Tool\n\nSubclasses of [`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) are plain Ruby classes with\nan optional set of typed parameters. \u003cbr\u003e The model can choose to\ncall them on your behalf, and they're one of the most powerful features\nfor extending the feature set or abilities of a model.\n\n```ruby\nclass ReadFile \u003c LLM::Tool\n  name \"read-file\"\n  description \"Read a file\"\n  parameter :path, String, \"The filename or path\"\n  required %i[path]\n\n  def call(path:)\n    {contents: File.read(path)}\n  end\nend\n```\n\n#### LLM::Stream\n\nStreams can be simple IO objects or subclasses of\n[`LLM::Stream`](https://r.uby.dev/api-docs/llm.rb/LLM/Stream.html) with structured callbacks for content,\nreasoning, tool calls, tool returns, and compaction.\n\n```ruby\nclass MyStream \u003c LLM::Stream\n  def on_content(content)\n    print content\n  end\n\n  def on_reasoning_content(content)\n    warn content\n  end\nend\n\nllm = LLM.deepseek(key: ENV[\"KEY\"])\nagent = LLM::Agent.new(llm, stream: MyStream.new)\nagent.talk \"Explain Ruby fibers.\"\n```\n\n#### LLM::REPL\n\nThe [LLM::Agent#repl](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html#repl-instance_method)\nmethod allows an agent to spawn a read-eval-print loop\nthat can be useful while developing or operating agents.\nIt can be used to debug tool calls, confirm an\nagent has done what was expected, or improve an agent by\nasking questions about what it has done up to that point.\n\nThis feature requires that the [curses](https://github.com/ruby/curses)\nand [kramdown](https://github.com/gettalong/kramdown) libraries are\ninstalled and available to require.\n\nThe TUI displays a status line with a context-usage bar and cost\ncounter, a scrollable transcript with markdown rendering, and a\nmulti-line input area. The UI stays responsive while the model\nis generating a response.\n\n##### REPL: Agent\n\nA REPL session is started by calling `repl` on any agent\ninstance. The session inherits the agent's model, tools,\nskills, and instructions.\n\n```ruby\nrequire \"llm\"\n\nllm = LLM.deepseek(key: ENV[\"KEY\"])\nagent = LLM::Agent.new(llm)\nagent.repl\n```\n\n##### REPL: State\n\nThe `path:` option accepts a file path where runtime state\nis read from and written to. This lets you resume a\nconversation across REPL sessions.\n\n```ruby\nrequire \"llm\"\n\nllm = LLM.deepseek(key: ENV[\"KEY\"])\nagent = LLM::Agent.new(llm)\nagent.repl(path: \"session.json\")\n```\n\n##### REPL: Tools\n\nThe `tools` option lets you attach additional tools\nfor the duration of the session. This is in addition to\nany tools that might already be associated with an agent.\n\nA number of optional tools are distributed as part of\nllm.rb. They power the agents that can be found in the\n[agents/](agents/) directory.\n\n```ruby\nrequire \"llm\"\n\nllm = LLM.deepseek(key: ENV[\"KEY\"])\nagent = LLM::Agent.new(llm)\nagent.repl(tools: [Debugger])\n```\n\nThe following example starts a read-eval-print loop\nwith all of the builtin tools available.\n\n```ruby\nrequire \"llm\"\nrequire \"llm/tools\"\n\nllm = LLM.deepseek(key: ENV[\"KEY\"])\nagent = LLM::Agent.new(llm)\nagent.repl(tools: LLM::Tool.subclasses)\n```\n\n##### REPL: Skills\n\nThe `skills` option lets you load extra skill directories\nwithout attaching them to an agent permanently.\n\n```ruby\nrequire \"llm\"\n\nllm = LLM.deepseek(key: ENV[\"KEY\"])\nagent = LLM::Agent.new(llm)\nagent.repl(skills: [__dir__])\n```\n\n##### REPL: Tracer\n\nBy default the tracer is disabled for the duration of the\nsession. Setting `tracer: true` configures the REPL to use\nthe tracer associated with an instance of\n[`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).\n\n```ruby\nrequire \"llm\"\n\nllm    = LLM.deepseek(key: ENV[\"KEY\"])\ntracer = LLM.logger(llm, path: \"agent.log\")\nagent  = LLM::Agent.new(llm, tracer:)\nagent.repl(tracer: true, tools: [Debugger])\n```\n\n##### REPL: Commands\n\nCommands are recognized by a `/` prefix and are backed by the\n[`LLM::Repl::Command`](https://r.uby.dev/api-docs/llm.rb/LLM/Repl/Command.html)\nclass, which can be subclassed to add custom commands. Once you\ncreate a subclass, it is automatically added to the repl. A command\ncan have zero or more parameters, and all parameters are presumed\nto be a String (at least for now).\n\n```ruby\nrequire \"llm\"\nrequire \"llm/repl\"\n\nclass Greeter \u003c LLM::Command\n  name \"greet\"\n  description \"Greets the given name\"\n  parameter :name, String, \"The person's name\"\n  required %i[name]\n\n  def call(name:)\n    write(\"Welcome #{name}!\\n\")\n  end\nend\n```\n\n##### REPL: Input\n\nThe input area supports several keyboard shortcuts:\n\n| Key | Action |\n|---|---|\n| `Enter` | Submit the current prompt |\n| `Ctrl+A` | Jump to the start of the line |\n| `Ctrl+E` | Jump to the end of the line |\n| `Ctrl+F` | Move the cursor forward |\n| `Ctrl+K` | Erase from cursor to the end of the line |\n| `Ctrl+Y` | Paste previously killed text |\n| `Ctrl+D` | Delete the character at the cursor |\n| `Left / Right` | Move the cursor |\n| `Up / Down` | Scroll the transcript |\n| `/exit` | Leave the REPL |\n\n#### LLM::MCP\n\nThe Model Context Protocol (MCP) has first-class support\nin llm.rb. The stdio and http transports work out of the\nbox. MCP tools are translated into subclasses of\n[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) that can be used with [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)\nor [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).\n\n```ruby\nrequire \"llm\"\n\nllm   = LLM.deepseek(key: ENV[\"KEY\"])\nmcp   = LLM::MCP.stdio(argv: [\"ruby\", \"server.rb\"])\nagent = LLM::Agent.new(llm, stream: $stdout, tools: mcp.tools)\nagent.talk \"Run the tool\"\n```\n\n#### LLM::A2A\n\nThe Agent 2 Agent (A2A) protocol has first-class support\nin llm.rb. The http and jsonrpc transports work out of the\nbox. A2A skills are translated into subclasses of\n[`LLM::Tool`](https://r.uby.dev/api-docs/llm.rb/LLM/Tool.html) that can be used with [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html)\nor [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html).\n\n```ruby\nrequire \"llm\"\n\nllm   = LLM.deepseek(key: ENV[\"KEY\"])\na2a   = LLM::A2A.rest(url: \"https://remote-agent.example.com\")\nagent = LLM::Agent.new(llm, stream: $stdout, tools: a2a.skills)\nagent.talk \"Run the skill\"\n```\n\n#### RAG\n\nMost providers offer an embedding model that can be\nused for semantic search, or similarity search. An\nembedding model can generate embeddings that can then\nbe stored in a database that is optimized for storing\nand querying vectors, such as SQLite's [sqlite-vec](https://github.com/asg017/sqlite-vec)\nor PostgreSQL's [pg-vector](https://github.com/pgvector/pgvector).\n\nllm.rb also includes support for OpenAI's vector store API. It\nprovides a vector database as a HTTP service but we won't cover\nthat here.\n\n```ruby\nrequire \"llm\"\n\nllm  = LLM.openai(key: ENV[\"KEY\"])\nbody = \"llm.rb is Ruby's capable AI runtime.\"\nembedding = llm.embed([body]).embeddings.first\n\nDocument.create!(\n  title: \"llm.rb\",\n  body:,\n  embedding:,\n)\n```\n\n#### Concurrency\n\nThe runtime supports five different concurrency strategies that have\ndifferent attributes. The choice between all of them often depends\non the requirements of your application.\n\nIO-bound tools are a good fit for the `:task`, `:thread`,\nand `:fiber` strategies while true parallelism can be achieved\nwith the `:fork` and `:ractor` strategies. The\n`:fork` strategy also provides a separate process that offers\nisolation from its parent.\n\n```ruby\nrequire \"llm\"\n\nllm   = LLM.deepseek(key: ENV[\"KEY\"])\ntools = [FetchNews, FetchStocks, FetchFeeds]\nagent = LLM::Agent.new(llm, tools:, concurrency: :fork)\nagent.talk \"Run the tools in parallel\"\n```\n\n#### ORM\n\nBecause both [`LLM::Context`](https://r.uby.dev/api-docs/llm.rb/LLM/Context.html), and [`LLM::Agent`](https://r.uby.dev/api-docs/llm.rb/LLM/Agent.html)\ncan be serialized to JSON and stored in a simple string, both ActiveRecord\nand Sequel support can be implemented within a single column on a single row.\n\nThe runtime includes first-class support for both ActiveRecord *and* Sequel, and\nfor both Rack-based applications *and* Rails-based applications. On databases\nwhere it is supported, such as PostgreSQL, the column can be optimized by using\nthe `jsonb` type.\n\n```ruby\nrequire \"active_record\"\nrequire \"llm\"\nrequire \"llm/active_record\"\n\nclass Agent \u003c ApplicationRecord\n  acts_as_agent do |agent|\n    agent.model \"deepseek-v4-pro\"\n    agent.instructions \"solve the user's query\"\n    agent.tools [Research, FinalizeResearch, ActOnResearch]\n  end\n\n  private\n\n  # By convention, this method defines the provider for a model.\n  # If necessary, it can be renamed with: provider: :your_method.\n  def set_provider\n    LLM.deepseek(key: ENV[\"KEY\"])\n  end\n\n  # By convention, this method returns the context options given\n  # to LLM::Context or LLM::Agent.\n  def set_context\n    {}\n  end\nend\n\nagent = Agent.create!\nagent.talk \"perform research\"\n```\n\n## FAQ\n\n\u003cdetails\u003e\n\u003csummary\u003eWhat providers does llm.rb support?\u003c/summary\u003e\n\u003cbr\u003e\n\u003cp\u003e\n\n**Cloud**\n\nThe following cloud-based providers are available to choose from. \u003cbr\u003e\nIn no particular order:\n\n🇺🇸 OpenAI \u003cbr\u003e\n🇺🇸 DeepInfra \u003cbr\u003e\n🇺🇸 xAI \u003cbr\u003e\n🇺🇸 Google (Gemini) \u003cbr\u003e\n🇺🇸 AWS bedrock \u003cbr\u003e\n🇺🇸 Anthropic \u003cbr\u003e\n🇨🇳 DeepSeek \u003cbr\u003e\n🇨🇳 zAI \u003cbr\u003e\n🇪🇺 Mistral \u003cbr\u003e\n\n**Weights**\n\nThe following providers provide access to open-weight models. \u003cbr\u003e\nIn no particular order:\n\n🇺🇸 DeepInfra \u003cbr\u003e\n🇺🇸 AWS bedrock \u003cbr\u003e\n🇨🇳 DeepSeek \u003cbr\u003e\n🇨🇳 zAI \u003cbr\u003e\n🇪🇺 Mistral \u003cbr\u003e\n\n**Local**\n\nThe following providers can be run locally on your own hardware. \u003cbr\u003e\nIn no particular order:\n\n* Ollama\n* Llamacpp\n\u003c/p\u003e\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eI have a limited budget. What should I do?\u003c/summary\u003e\n\u003cbr\u003e\n\u003cp\u003e\nThere a few options. The first option is to host\nyour own model, and use the ollama or llamacpp\nproviders. This can be diffilcult though because\na capable model requires hardware that can\nmatch it. If you have the ability to self-host,\nthis would be my first option.\n\u003c/p\u003e\n\u003cp\u003e\nThe second option is DeepSeek. \u003cbr\u003e\nThe deepseek-v4-flash model costs pennies to use. \u003cbr\u003e\nAnd llm.rb has been optimized for deepseek. For example,\nDeepSeek does not have image generation capabilities\nbut on the llm.rb runtime it does (vector graphics only,\nthough).\n\u003c/p\u003e\n\u003cp\u003e\nThe same is true for structured outputs. DeepSeek does\nnot support structured outputs in the same way as OpenAI or\nGoogle, but the llm.rb runtime makes it appear as\nthough it does, through the `json_object` response\ntype.\n\u003c/p\u003e\nIf you're on a budget, DeepSeek is hard to beat.\n\u003c/details\u003e\n\u003cdetails\u003e\n\u003csummary\u003eCan I download llm.rb via a decentralized network?\u003c/summary\u003e\n\u003cbr\u003e\nYou can!\n\u003cbr\u003e\nWe are on the \u003ca href=\"https://radicle.network\"\u003eradicle.network\u003c/a\u003e\n\u003cbr\u003e\nEvery commit that lands on GitHub also lands on Radicle.\n\u003cbr\u003e\nOur repository ID is z2PtfQ6dYwyYaW2aGrztG1sMyDmCE.\n\u003cbr\u003e\nBrowse on \u003ca href=\"https://radicle.network/nodes/iris.radicle.network/z2PtfQ6dYwyYaW2aGrztG1sMyDmCE\"\u003ethe web\u003c/a\u003e.\n\u003c/details\u003e\n\n## Resources\n\nIf you like what you read so far, check out the [deepdive.md](https://r.uby.dev/llm/deepdive/)\nto learn more. Unfortunately it\nwasn't possible to cover every feature without the README becoming a small book.\nThe [r.uby.dev](https://r.uby.dev) homepage also includes more learning material\nand resources.\n\n## License\n\n[Business Source License 1.1](./LICENSE)\n\u003cbr\u003e\nCommercial production use requires a commercial license.\n\u003cbr\u003e\nEach version converts to the [BSD Zero Clause](https://choosealicense.com/licenses/0bsd/)\nfour years after its first public release.\n\u003cbr\u003e\nContact [robert@r.uby.dev](mailto:robert@r.uby.dev) for a commercial license.\n\n### Waivers\n\nWaivers are automatically granted for: \u003cbr\u003e\n\n  * Personal use\n  * Students\n  * Teachers\n  * Evaluation, development, and testing\n  * Non-profits and charities\n  * Companies with less than or equal to 50 employees\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fr-uby-dev%2Fllm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fr-uby-dev%2Fllm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fr-uby-dev%2Fllm/lists"}