{"id":17061569,"url":"https://github.com/paxa/light_record","last_synced_at":"2025-04-05T23:07:46.580Z","repository":{"id":56881209,"uuid":"44088438","full_name":"Paxa/light_record","owner":"Paxa","description":"ActiveRecord extension to kick the speed of allocating ActiveRecord object","archived":false,"fork":false,"pushed_at":"2022-11-10T09:32:26.000Z","size":914,"stargazers_count":164,"open_issues_count":3,"forks_count":6,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-29T21:09:31.374Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Paxa.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-10-12T06:33:11.000Z","updated_at":"2025-02-14T07:12:26.000Z","dependencies_parsed_at":"2022-08-20T22:31:15.923Z","dependency_job_id":null,"html_url":"https://github.com/Paxa/light_record","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Paxa%2Flight_record","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Paxa%2Flight_record/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Paxa%2Flight_record/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Paxa%2Flight_record/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Paxa","download_url":"https://codeload.github.com/Paxa/light_record/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247411234,"owners_count":20934653,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-14T10:47:31.001Z","updated_at":"2025-04-05T23:07:46.550Z","avatar_url":"https://github.com/Paxa.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"LightRecord\n===========\n\n[![Build Status](https://travis-ci.org/Paxa/light_record.svg?branch=master)](https://travis-ci.org/Paxa/light_record)\n\nActiveRecord extension to kick the speed of allocating ActiveRecord object, and it allows fetching DB rows sequentially to reduce memory usage\n\n**Supports:** Rails 5, 6 and 7\n\n**Note:** it overrides some internal methods of active record, and in some rare cases it not work correctly,\nplease test it well in your project before using it in production\n\n### How it works\n\nIt provides functionality to load ActiveRecord records with patched attribute related methods.\nThis make AR objects as read-only but it makes up to 5 times less object allocations.\n\nEach time when you retrieve objects via `.light_records` it will create anonymous class to work with given set of attributes.\n\n```\n  LightRecord Extension Class\n              ↓\n        Your AR Model\n              ↓\n      ActiveRecord::Base\n```\n\n\n### Installation\n\n```ruby\ngem 'light_record', github: 'paxa/light_record'\n```\n\n#### `scope.light_records`\n\n```ruby\nrecords = User.limit(1_000_000).light_records\nrecords # =\u003e array of records. Very fast and very memory efficient\n```\n\nIdea is to skip all magic related to attributes and object initialization. This creates new class inherited from your model. That allows us to create only one extra object when we initialize new record.\n\n\nSimply it become something like this:\n\n```ruby\nclass User_light_record \u003c User\n  def initialize(attributes)\n    @attributes = attributes # hash of data \"as is\" from database library\n  end\n\n  def email\n    @attributes[:email]\n  end\nend\n```\n\n\n#### `scope.light_records_each`\n\n\nOther method: `.light_records_each`, it will utilize `stream: true` feature from mysql2 client. So it will initialize objects one by one for every interaction:\n\n```ruby\nUser.limit(1_000_000).light_records_each do |user|\n  user.do_something\nend\n```\n\nThis allow you to interate big amount of data without using `find_each` or `find_in_batches` because with `light_records_each` it will use very low memory. Or allow you to use `find_in_batches` with bigger batch size\n\n\\* Please note that time will be as a ruby [Time](https://ruby-doc.org/core-3.1.0/Time.html) object, instead of [TimeWithZone](http://api.rubyonrails.org/classes/ActiveSupport/TimeWithZone.html). To make it in correct time zone you can call it as:\n\n```ruby\nrecord.created_at.in_time_zone(Time.zone)\n```\n\n#### Benchmarks\n\nStill on a way,\nbut I try to use in some project and it gives 3-5 times improvement, and 2-3 times less memory usage\n\n\n---\n\nSometimes this can break functionality because it will override attribute methods and disable some of features in activerecord.\n\nThere is mechanism to override attribute methods created by LightRecord:\n\n```ruby\nclass User \u003c ActiveRecord::Base\n  # this module will be included in extending class when we use light_records and light_records_each\n  module LightRecord\n    def sometihng\n    end\n\n    def success\n      attributes[:success] == 1\n    end\n\n    def success_time\n      return attributes[:success_time] unless attributes[:success_time]\n      @success_time ||= attributes[:success_time].in_time_zone(Time.zone)\n    end\n\n    def success_time=(val)\n      @success_time = nil\n      super(val)\n    end\n  end\nend\n```\n\nNote: when you use LightRecord instances it will break type casting\n\n\nThis gem supports MySQL and PostgreSQL\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpaxa%2Flight_record","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpaxa%2Flight_record","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpaxa%2Flight_record/lists"}