{"id":35071226,"url":"https://github.com/barek2k2/local_llm","last_synced_at":"2026-04-19T06:45:08.105Z","repository":{"id":327177575,"uuid":"1108182585","full_name":"barek2k2/local_llm","owner":"barek2k2","description":"Lightweight Ruby gem for interacting with locally running Ollama LLMs with streaming, chat, and full offline privacy.","archived":false,"fork":false,"pushed_at":"2025-12-06T18:51:48.000Z","size":9278,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-04-19T06:44:08.526Z","etag":null,"topics":["ai","artificial-intelligence","chatbot","data-privacy-compliance","data-security","llm","local-ai","local-ai-development","local-ai-llm","machine-learning","offline-ai","privacy","private-chat","ruby","ruby-on-rails","security"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/barek2k2.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-02T06:10:34.000Z","updated_at":"2025-12-10T03:05:36.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/barek2k2/local_llm","commit_stats":null,"previous_names":["barek2k2/local_llm"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/barek2k2/local_llm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/barek2k2%2Flocal_llm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/barek2k2%2Flocal_llm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/barek2k2%2Flocal_llm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/barek2k2%2Flocal_llm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/barek2k2","download_url":"https://codeload.github.com/barek2k2/local_llm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/barek2k2%2Flocal_llm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31997804,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-18T20:23:30.271Z","status":"online","status_checked_at":"2026-04-19T02:00:07.110Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","artificial-intelligence","chatbot","data-privacy-compliance","data-security","llm","local-ai","local-ai-development","local-ai-llm","machine-learning","offline-ai","privacy","private-chat","ruby","ruby-on-rails","security"],"created_at":"2025-12-27T11:56:29.287Z","updated_at":"2026-04-19T06:45:08.098Z","avatar_url":"https://github.com/barek2k2.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Local Large Language Model(LLM) or Offline LLM for Ruby on Rails\n\n**`local_llm`** is a lightweight Ruby gem that lets you talk to **locally installed LLMs via Ollama** — with **zero cloud dependency**, full **developer control**, and **configurable defaults**, including **real-time streaming support**. Instead of sending sensitive data to cloud APIs, this gem allows you to interact with powerful AI models directly from your local machine or server. It is built for privacy, control, and simplicity, making it ideal for developers who want fast AI features without internet dependency, usage limits, or data exposure. LocalLLM works seamlessly with both plain Ruby and Ruby on Rails applications.\n\nIt supports:\n- Completely **OFFLINE!**\n- Any Ollama model (LLaMA, Mistral, CodeLLaMA, Qwen, Phi, Gemma, etc.)\n- Developer-configurable default models\n- Developer-configurable Ollama API endpoint\n- Developer-configurable **streaming or non-streaming**\n- One-shot Q\u0026A and multi-turn chat\n- Works in plain Ruby \u0026 Rails\n- Also safe for HIPAA, SOC2, and regulated workflows where data privacy is a big concern\n\n### Privacy \u0026 Security\n - **100% local inference**\n - **No cloud calls**\n - **No API keys**\n - **No data leaves your machine**\n\n---\n\n## 🎬 Live Demo\n![Rails local LLM Streaming](https://raw.githubusercontent.com/barek2k2/local_llm_demo/76a05e485bf92e0c31e922db6b9df6608086d1f8/app/assets/images/demo.gif)\n\nRuby on Rails Example at https://github.com/barek2k2/local_llm_demo\n\n\n## 🚀 Features\n\n- Use **any locally installed Ollama model**\n- Change **default models at runtime**\n- Enable or disable **real-time streaming**\n- Works with:\n  - `llama2`\n  - `mistral`\n  - `codellama`\n  - `qwen`\n  - `phi`\n  - Anything supported by Ollama\n- No API keys needed\n- No cloud calls\n- Full privacy\n- Works completely offline\n\n---\n\n## 📦 Installation\n\n### Install Ollama\n\nDownload from:\n\nhttps://ollama.com\n\nThen start it:\n\n```bash\nollama serve\n```\n\nOr simply install it by running `brew install ollama` command in your mac machine.\n\n### How to Install New LLMs\n```\nollama pull llama2:13b\nollama pull mistral:7b-instruct\nollama pull codellama:13b-instruct\nollama pull qwen2:7b\n```\n\n### Verify Installed Models\n```\nollama list\n```\n\n### Configuration\n```\nLocalLlm.configure do |c|\n  c.base_url = \"http://localhost:11434\"\n  c.default_general_model = \"llama2:13b\"\n  c.default_fast_model    = \"mistral:7b-instruct\"\n  c.default_code_model    = \"codellama:13b-instruct\"\n  c.default_stream = false   # true = stream by default, false = return full text\nend\n```\n\n### Basic Usage (Non-Streaming)\n```\nLocalLlm.ask(\"llama2:13b\", \"What is HIPAA?\")\nLocalLlm.ask(\"qwen2:7b\", \"Explain transformers in simple terms.\")\n\nLocalLlm.general(\"What is a Denial of Service attack?\")\nLocalLlm.fast(\"Summarize this paragraph in 3 bullet points.\")\nLocalLlm.code(\"Write a Ruby method that returns factorial of n.\")\n```\n\n### Constant Alias (LocalLlm vs LocalLLM)\n\nFor convenience and readability, `LocalLLM` is provided as a direct alias of `LocalLlm`.\n\nThis means **both constants work identically**:\n\n```\nLocalLlm.fast(\"Tell me About Bangladesh\")\nLocalLLM.fast(\"Explain HIPAA in simple terms.\") # alias of LocalLlm\n```\n\n### Streaming Usage (Live Output)\n```\nLocalLlm.configure do |c|\n  c.default_stream = true\nend\n\nLocalLlm.fast(\"Explain HIPAA in very simple words.\") do |chunk|\n  print chunk\nend\n```\n\n### Per-Call Streaming Override\n```\nLocalLlm.fast(\"Explain LLMs in one paragraph.\", stream: true) do |chunk|\n  print chunk\nend\n\nfull_text = LocalLlm.fast(\"Explain DoS attacks briefly.\", stream: false)\nputs full_text\n```\n\n\n### List Installed Ollama Models from Ruby\n```\nLocalLlm.models\n```\n\n### Switching to Qwen (or Any New Model)\n```\nollama pull qwen2:7b\n```\n\n```\nLocalLlm.ask(\"qwen2:7b\", \"Explain HIPAA in simple terms.\")\n```\n\n### Full Chat API (Multi-Turn)\n```\nLocalLlm.chat(\"qwen2:7b\", [\n  { \"role\" =\u003e \"system\", \"content\" =\u003e \"You are a helpful assistant.\" },\n  {\"role\" =\u003e \"user\", \"content\" =\u003e \"Explain Ruby shortly in one sentence\"},\n])\n```\n`Ruby is a dynamic, open-source programming language known for its simplicity and readability, designed for building web applications with the Ruby on Rails framework.`\n\n```\nLocalLlm.chat(\"qwen2:7b\", [\n  { \"role\" =\u003e \"system\", \"content\" =\u003e \"You are a helpful assistant.\" },\n  {\"role\" =\u003e \"user\", \"content\" =\u003e \"Explain Ruby shortly in one sentence\"},\n  { \"role\" =\u003e \"assistant\",   \"content\" =\u003e \"Ruby is an open-source, dynamic, object-oriented programming language that emphasizes simplicity and readability, making it popular for web development with the Rails framework\" },\n  { \"role\" =\u003e \"user\",   \"content\" =\u003e \"Tell me the year in number when it was created?\" }\n])\n```\n`Ruby was created in the year 1995.`\n\n```\nLocalLlm.chat(\"qwen2:7b\", [\n  { \"role\" =\u003e \"system\", \"content\" =\u003e \"You are a helpful assistant.\" },\n  {\"role\" =\u003e \"user\", \"content\" =\u003e \"Explain Ruby shortly in one sentence\"},\n  { \"role\" =\u003e \"assistant\",   \"content\" =\u003e \"Ruby is an open-source, dynamic, object-oriented programming language that emphasizes simplicity and readability, making it popular for web development with the Rails framework\" },\n  { \"role\" =\u003e \"user\",   \"content\" =\u003e \"Tell me the year in number when it was created?\" },\n  { \"role\" =\u003e \"assistant\",   \"content\" =\u003e \"Ruby was created in the year 1995.\" },\n  {\"role\" =\u003e \"user\", \"content\" =\u003e \"Thanks so much!\"}\n])\n```\n`You're welcome! If you have any other questions, feel free to ask. Happy coding!`\n\n\n### Make Qwen the Default\n```\nLocalLlm.configure do |c|\n  c.default_general_model = \"qwen2:7b\"\nend\n\nLocalLlm.general(\"Explain transformers.\")\n```\n\n### 🔌 Remote Ollama / Docker Support\n```\nLocalLlm.configure do |c|\n  c.base_url = \"http://192.168.1.100:11434\"\nend\n```\n\n### Troubleshooting\n`Connection refused - connect(2) for \"localhost\" port 11434 (Errno::ECONNREFUSED)`\n\n##### This means ollama is not installed or not running in your machine. So run the following commands below\n - `brew install ollama`\n - `brew services start ollama`\n\nAfter successfully starting this, it would run on port 11434 intom your machine. make sure it pulls installed LLM by running `ollama list`\n\n### How to use it in Ruby On Rails\n - In your Gemfile, add the gem as \n  `gem \"local_llm\", \"~\u003e 0.1.1\"`\n - Run `bundle install`\n - Create an initializer at `config/initializers/local_llm.rb`\n - Put the following code into it\n ```\n LocalLlm.configure do |c|\n  # Default Ollama endpoint\n  c.base_url = \"http://localhost:11434\"\n\n  # Choose your default models (must be installed in Ollama)\n  c.default_general_model = \"qwen2:7b\"\n  c.default_fast_model    = \"mistral:7b-instruct\"\n  c.default_code_model    = \"codellama:13b-instruct\"\n  c.default_stream = true # stream support by default\nend\n\n ```\n - Then from your any controller or model, run this\n ```\n question = \"What is Ruby?\"\n LocalLlm.fast(question) do |chunk|\n   print chunk\n end\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbarek2k2%2Flocal_llm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbarek2k2%2Flocal_llm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbarek2k2%2Flocal_llm/lists"}