{"id":17065918,"url":"https://github.com/nicieja/lammy","last_synced_at":"2025-07-24T02:36:43.126Z","repository":{"id":257825894,"uuid":"865624729","full_name":"nicieja/lammy","owner":"nicieja","description":"An LLM library for Ruby","archived":false,"fork":false,"pushed_at":"2025-01-02T01:38:06.000Z","size":90,"stargazers_count":28,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-12T21:10:30.761Z","etag":null,"topics":["claude","gem","gemini","gpt","llm","openai","rails","ruby"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nicieja.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-30T20:56:30.000Z","updated_at":"2025-01-02T01:38:09.000Z","dependencies_parsed_at":"2024-10-14T11:04:50.698Z","dependency_job_id":"d0279f95-1b5d-4003-be34-926583e39452","html_url":"https://github.com/nicieja/lammy","commit_stats":{"total_commits":27,"total_committers":2,"mean_commits":13.5,"dds":0.03703703703703709,"last_synced_commit":"02c7a47ee9543587d93bd704147d45bdc2bd7cd9"},"previous_names":["nicieja/lammy"],"tags_count":12,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nicieja%2Flammy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nicieja%2Flammy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nicieja%2Flammy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nicieja%2Flammy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nicieja","download_url":"https://codeload.github.com/nicieja/lammy/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":234715616,"owners_count":18875905,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["claude","gem","gemini","gpt","llm","openai","rails","ruby"],"created_at":"2024-10-14T11:01:35.154Z","updated_at":"2025-01-19T23:58:32.547Z","avatar_url":"https://github.com/nicieja.png","language":"Ruby","funding_links":[],"categories":["Ruby"],"sub_categories":[],"readme":"# Lammy\n\nLammy is a simple LLM library for Ruby. It doesn't treat prompts as just strings. They represent the entire code that generates the strings sent to a LLM. The abstraction also makes it easy to attach these methods directly to models, avoiding the need for boilerplate service code.\n\nThe approach is inspired by [Python's ell](https://github.com/MadcowD/ell). I haven't come across a Ruby port yet, so I decided to start experimenting on my own.\n\n## Why?\n\nI wanted to create a simple library that would let me use LLMs in my Ruby projects without dealing with a lot of boilerplate code.\n\nUsing something like `langchain` felt too complex for many of my needs. Another option would be to integrate a library directly with a framework like Ruby on Rails, leveraging its conventions. You could, for example, store prompts in the database or as views. But that seemed like overkill for what I needed, and it would add a dependency on the framework, making it harder to use in simple programs.\n\nPersonally, I don't think prompt engineering needs to be that complicated, which is why the `ell` approach—treating prompts like simple functions—really resonated with me. I wanted to bring something similar to Ruby. I don’t see why LLMs can't be treated like databases in Active Record, where all the complexity is abstracted away. You can query without needing to think much about the underlying SQL. With Lammy, the idea is similar: you just define your prompt in a method on a model and call it like any other method.\n\n## Installation\n\n### Bundler\n\nAdd this line to your application's Gemfile:\n\n```ruby\ngem \"lammy\"\n```\n\nAnd then execute:\n\n```bash\n$ bundle install\n```\n\nYou can find a basic example of how to use Lammy in Rails in the [lammy-rails-example](https://github.com/nicieja/lammy-rails-example) repository.\n\n### Gem install\n\nOr install with:\n\n```bash\n$ gem install lammy\n```\n\nand require with:\n\n```ruby\nrequire \"lammy\"\n```\n\n## Usage\n\nWe currently support OpenAI's models and Anthropic's Claude. You can use any model that supports the OpenAI API or Claude. Make sure to set the `OPENAI_ACCESS_TOKEN` environment variable for OpenAI models or the `ANTHROPIC_API_KEY` for Claude models.\n\n### Chat\n\nLammy allows you to interact with a chat model using the `llm` decorator. The `llm` decorator accepts a `model` argument, where you specify the name of the model you'd like to use.\n\n```ruby\nclass User\n  # To be able to make LLM calls, we first include `L` at the top of our class\n  include L\n\n  attr_reader :name\n\n  def initialize(name:)\n    @name = name\n  end\n\n  # Take a message as input and return a model-generated message as output\n  llm(model: \"gpt-4o\")\n  def welcome\n    # User message goes here\n    \"Say hello to #{name.reverse} with a poem.\"\n  end\nend\n\nuser = User.new(name: \"John Doe\")\nuser.welcome\n\n# =\u003e \"Hello eoD nhoJ, let's make a cheer,\\n\n# With a whimsical poem to bring you near.\\n\n# Though your name's in reverse, it’s clear and bright,\\n\n# Let's dance in verse on this delightful night!\"\n```\n\n### System message\n\nYou can provide a system message to the model through the `context` method. This is an optional approach that allows you to give the model additional context. We chose not to use the `system` method because it's a potentially risky Ruby method.\n\n```ruby\nclass User\n  include L\n\n  # (...)\n\n  llm(model: \"gpt-4o\")\n  def welcome\n    # An optional system message\n    context \"You are an AI that only writes in lower case.\"\n    # User message goes here\n    \"Say hello to #{name.reverse} with a poem.\"\n  end\nend\n\nuser = User.new(name: \"John Doe\")\nuser.welcome\n\n# =\u003e \"hello eod nhoj, let's make a cheer,\\n\n# with a whimsical poem to bring you near.\\n\n# though your name's in reverse, it’s clear and bright,\\n\n# let's dance in verse on this delightful night!\"\n```\n\n### Structured output for OpenAI's models\n\nYou can request OpenAI's models to return a structured JSON output by using the `schema` option in the decorator. This is an optional feature that allows you to define a structured output format for the model. To handle arrays of objects, use `L.to_a`, and for a single object, use `L.to_h`.\n\n```ruby\nclass User\n  include L\n\n  # (...)\n\n  # Define a structured output schema for Lammy to handle JSON responses.\n  # For a single object instead of an array, use `L.to_h`.\n  llm(model: \"gpt-4o-2024-08-06\", schema: L.to_a(name: :string, city: :string))\n  def friends\n    \"Hallucinate a list of friends for #{name}.\"\n  end\nend\n\nuser = User.new(name: \"John Doe\")\nuser.friends\n\n# =\u003e [{\"name\"=\u003e\"Alice Summers\", \"city\"=\u003e\"Austin\"},\n#   {\"name\"=\u003e\"Brian Thompson\", \"city\"=\u003e\"Denver\"},\n#   {\"name\"=\u003e\"Charlie Herrera\", \"city\"=\u003e\"Seattle\"},\n#   {\"name\"=\u003e\"Diana Flores\", \"city\"=\u003e\"San Francisco\"},\n#   {\"name\"=\u003e\"Eli Grant\", \"city\"=\u003e\"New York\"},\n#   {\"name\"=\u003e\"Fiona Collins\", \"city\"=\u003e\"Chicago\"},\n#   {\"name\"=\u003e\"George Baker\", \"city\"=\u003e\"Los Angeles\"},\n#   {\"name\"=\u003e\"Hannah Kim\", \"city\"=\u003e\"Miami\"},\n#   {\"name\"=\u003e\"Isaac Chen\", \"city\"=\u003e\"Boston\"},\n#   {\"name\"=\u003e\"Jessica Patel\", \"city\"=\u003e\"Houston\"}]\n```\n\n### Prefilling assistant responses for Claude\n\nAnthtopic decided to improve output consistency and implement JSON mode by allowing users to prefill the model's response. Lammy enables this feature through its array syntax, along with the `L.user` and `L.system` helper methods.\n\n```ruby\nclass User\n  include L\n\n  # (...)\n\n  llm(model: \"claude-3-5-sonnet-20240620\")\n  def welcome\n    # Provide a list of messages to the model for back-and-forth conversation\n    [\n      # User message goes here\n      L.user(\"Say hello to #{name.reverse} with a poem.\"),\n      # When using Claude, you have the ability to guide its responses by prefilling it\n      L.assistant(\"Here's a little poem for you:\")\n    ]\n  end\nend\n```\n\nAlthough only Claude models prefill responses, the array syntax can be applied to both OpenAI and Claude models. For OpenAI's models, this feature is used to continue the conversation from where the previous message left off, enabling multi-message conversations like the one in our upcoming example.\n\n### Streaming\n\nYou can use the `stream` method to stream responses from the LLM in real time, which can be much faster and help create a more engaging user experience. To receive chunks of the response as they come in, pass a lambda to the `stream` method.\n\n```ruby\nclass Bot\n  include L\n\n  llm(model: \"gpt-4o\")\n  def talk(message)\n    # Use the `stream` method to stream chunks of the response.\n    # In this case, we're just printing the chunks.\n    stream -\u003e(content) { puts content }\n    # Nothing fancy, simply transfer the message to the model\n    message\n  end\nend\n\nbot = Bot.new\nbot.talk(\"Hello, how are you?\")\n\n# =\u003e \"I'm here and ready to help. How can I assist you today?\"\n```\n\nThis is a simplified explanation of how you can use the `stream` method. For a complete example, refer to [this file](https://github.com/nicieja/lammy/blob/main/examples/streaming.rb). This implementation allows to hold an actual conversation with the model, which is the most common use case for chatbots, and does it using Lammy's array syntax.\n\n### Asynchronous generation\n\nYou can use the `async` option to run the method asynchronously. This is useful for generating long-running responses, such as chatbots, and can be combined with streaming. To use this feature, you need to configure Sidekiq and include the `Lammy::Job` module in your application.\n\n```ruby\n# config/initializers/lammy.rb\nrequire 'lammy/sidekiq' # Configure Sidekiq worker for processing async jobs\n\n# app/models/user.rb\nclass User \u003c ApplicationRecord\n  attribute :name, :string\n\n  # Use the `llm` method with `async: true` to run the method asynchronously\n  llm(model: 'gpt-4o', async: true)\n  def welcome\n    \"Say hello to #{name} with a poem.\"\n  end\nend\n\nuser = User.create!(name: 'John')\nputs user.welcome\n# =\u003e \"ea39ea2c55d568cf96032ce1\"\n```\n\nFor this feature to work, your model must be saved to the database and have a retrievable `id` attribute.\n\n### Vision\n\nYou can use a vision model to generate a description of an image this way:\n\n```ruby\nclass Image\n  include L\n\n  attr_accessor :file\n\n  llm(model: \"gpt-4o\")\n  def describe\n    L.user(\"Describe this image.\", image: file)\n  end\nend\n\nimage = Image.new\nimage.file = File.read(\"./examples/assets/ruby.jpg\")\nimage.describe\n\n# =\u003e \"The image is an illustration of a red gem, specifically a ruby.\n# The gem is depicted with facets that reflect light, giving it a shiny\n# and polished appearance. This image is often associated with\n# the Ruby programming language logo.\"\n```\n\nThe `L.user` helper method must be used to attach the image to the prompt.\n\n### Custom clients\n\nFor a more robust setup, you can configure the client directly and pass it to the decorator.\n\n```ruby\n# Helicone is an open-source LLM observability platform for developers\n# to monitor, debug, and optimize their apps\n$helicone = OpenAI::Client.new(\n  access_token: \"access_token_goes_here\",\n  uri_base: \"https://oai.hconeai.com/\",\n  request_timeout: 240,\n  extra_headers: {\n    \"X-Proxy-TTL\" =\u003e \"43200\",\n    \"X-Proxy-Refresh\": \"true\",\n    \"Helicone-Auth\": \"Bearer HELICONE_API_KEY\",\n    \"helicone-stream-force-format\" =\u003e \"true\",\n  }\n)\n\nclass User\n  include L\n\n  # (...)\n\n  # Pass the Helicone client to Lammy's decorator\n  llm(model: \"gpt-4o\", client: $helicone)\n  def description\n    \"Describe #{name} in a few sentences.\"\n  end\nend\n```\n\n### Embeddings\n\nYou can use the embeddings endpoint to obtain a vector of numbers that represents an input. These vectors can be compared across different inputs to efficiently determine their similarity. Currently, Lammy supports only OpenAI's embeddings endpoint.\n\n```ruby\nclass User\n  include L\n\n  # (...)\n\n  # Text embeddings measure the relatedness of text strings. The response\n  # will contain a list of floating point numbers, which you can extract,\n  # save in a vector database, and use for many different use cases.\n  v(model: \"text-embedding-3-large\", dimensions: 256)\n  def embeddings\n    %Q{\n      Hi, I'm #{name}. I'm a software engineer with a passion for Ruby\n      and open-source development.\n    }\n  end\nend\n\nuser = User.new(name: \"John Doe\")\nuser.embeddings\n\n# =\u003e [0.123, -0.456, 0.789, ...]\n# This will be the embedding vector returned by the model\n```\n\nNow you're able to store this vector in a vector database, such as `pgvector`, and use it to compare the similarity of different inputs. For example, you can use the [cosine similarity](https://en.wikipedia.org/wiki/Cosine_similarity) to determine the similarity between two vectors.\n\n## Configuration\n\nLammy allows you to configure global settings using a configuration block. This is useful for setting a default model or a custom client that will be used across your application.\n\n### Setting a global model\n\nYou can set a global LLM model that will be used by default in your application. This is done using the `configure` method:\n\n```ruby\nLammy.configure do |config|\n  config.model = \"gpt-4o\"\nend\n```\n\nWith a global model configured, you can now use the `llm` decorator without specifying the model:\n\n```ruby\nclass User\n  include L\n\n  # (...)\n\n  llm\n  def welcome\n    \"Say hello to #{name} with a poem.\"\n  end\nend\n```\n\n### Setting a custom client\n\nYou can also set a global custom client. This is useful if you want to use a specific client configuration throughout your application:\n\n```ruby\nLammy.configure do |config|\n  config.client = OpenAI::Client.new(\n    access_token: \"access_token_goes_here\",\n    uri_base: \"https://oai.hconeai.com/\",\n    request_timeout: 240,\n    extra_headers: {\n      \"X-Proxy-TTL\" =\u003e \"43200\",\n      \"X-Proxy-Refresh\": \"true\",\n      \"Helicone-Auth\": \"Bearer HELICONE_API_KEY\",\n      \"helicone-stream-force-format\" =\u003e \"true\",\n    }\n  )\nend\n```\n\n## Versioning\n\nSemantic versioning is used. For a version number `major.minor.patch`, unless `major` is 0:\n\n1. `major` version is incremented when incompatible API changes are made,\n2. `minor` version is incremented when functionality is added in a backwards-compatible manner,\n3. `patch` version is incremented when backwards-compatible bug fixes are made.\n\nMajor version \"zero\" (0.y.z) is for initial development. Anything may change at any time. The public API should not be considered stable. Furthermore, version \"double-zero\" (0.0.x) is not intended for public use, as even minimal functionality is not guaranteed to be implemented yet.\n\n## Contributing\n\n1. Fork the repository\n2. Create your feature branch: `git checkout -b my-new-feature`\n3. Commit your changes: `git commit -am 'Add some feature'`\n4. Push to the branch: `git push origin my-new-feature`\n5. Create a new pull request\n\n## License\n\nLammy is released under the MIT License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnicieja%2Flammy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnicieja%2Flammy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnicieja%2Flammy/lists"}