{"id":23430622,"url":"https://github.com/joinhandshake/fast_inserter","last_synced_at":"2025-04-12T23:07:50.092Z","repository":{"id":46760161,"uuid":"51548070","full_name":"joinhandshake/fast_inserter","owner":"joinhandshake","description":"Quickly insert database records with confidence","archived":false,"fork":false,"pushed_at":"2023-04-25T13:56:13.000Z","size":84,"stargazers_count":40,"open_issues_count":3,"forks_count":7,"subscribers_count":89,"default_branch":"main","last_synced_at":"2025-04-12T23:07:24.482Z","etag":null,"topics":["insert-records","rails","ruby-gem","sql"],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/joinhandshake.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-02-11T21:24:35.000Z","updated_at":"2021-10-11T15:27:35.000Z","dependencies_parsed_at":"2024-06-21T15:35:19.877Z","dependency_job_id":"77265322-004d-4ce7-a264-604338ffd7dc","html_url":"https://github.com/joinhandshake/fast_inserter","commit_stats":null,"previous_names":["strydercorp/fast_inserter"],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joinhandshake%2Ffast_inserter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joinhandshake%2Ffast_inserter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joinhandshake%2Ffast_inserter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joinhandshake%2Ffast_inserter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/joinhandshake","download_url":"https://codeload.github.com/joinhandshake/fast_inserter/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248643004,"owners_count":21138355,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["insert-records","rails","ruby-gem","sql"],"created_at":"2024-12-23T09:39:04.959Z","updated_at":"2025-04-12T23:07:50.066Z","avatar_url":"https://github.com/joinhandshake.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# FastInserter\n\n[![Build Status](https://github.com/joinhandshake/fast_inserter/actions/workflows/ruby.yml/badge.svg)](https://github.com/joinhandshake/fast_inserter/actions)\n[![Gem Version](https://badge.fury.io/rb/fast_inserter.svg)](https://badge.fury.io/rb/fast_inserter)\n\nUse raw SQL to insert database records in bulk, fast. Supports uniqueness constraints, timestamps, and checking for existing records.\n\nThe motivation for this library comes from the fact that rails does validations on each and every inserted record in the join table. And, even if you pass validate: false, it still loads each record and inserts one by one. This is all good, but also means inserting a large number (thousands) of records is slow.\n\nThis library skips active record altogether and uses raw sql to insert records. However, using raw sql goes around all your business logic, so we provide ways to still have niceties like uniqueness constraints and timestamps.\n\n## Installation\n\nAdd this line to your application's Gemfile:\n\n```ruby\ngem 'fast_inserter'\n```\n\nAnd then execute:\n\n    $ bundle\n\nOr install it yourself as:\n\n    $ gem install fast_inserter\n\n## Runtime dependencies\n\n* activerecord: Fast inserter depends on active record for handling database connections, database configuration, executing the sql, and sql sanitization.\n\n## Usage\n\n*We wrote a longer post about this gem at https://joinhandshake.com/engineering/2016/01/26/quickly-inserting-thousands-of-records-in-rails.html*\n\nIn most cases, you probably don't want to use this library and instead should use active record. However, should you need to use this library, usage instructions are below.\n\nA basic usage for inserting multiple 'MassEmailsUser' records:\n\n```ruby\n@mass_email = MassEmail.find(params[:id])\nuser_ids = [1, 2, 3, 4] # ids to fast insert\nparams = {\n  table: 'mass_emails_users',\n  static_columns: {\n    mass_email_id: @mass_email.id\n  },\n  additional_columns: {\n    created_by_id: current_user.id\n  },\n  options: {\n    timestamps: true,\n    unique: true,\n    check_for_existing: true\n  },\n  group_size: 2_000,\n  variable_column: 'user_id',\n  values: user_ids\n}\ninserter = FastInserter::Base.new(params)\ninserter.fast_insert\n```\n\nLet's walkthrough the options.\n\n### table\n\nDefines the table name to insert into\n\n### static_columns\n\nThese are columns and the values for the columns which will not change for each insertion.\n\n### additional_columns\n\nThese are also static columns which will not changed, but these columns will not be used for uniqueness validation constraints.\n\n### timestamps\n\nIncludes created_at and updated_at timestamps to each record. Default is false.\n\n### unique\n\nEnsures that the 'values' parameter is a unique set of values. Default is false.\n\n### check_for_existing\n\nQueries the table for any values which already exist and removes them from the values to be inserted. This query uses 'static_columns' and 'variable_column' for determining uniqueness. Default is false.\n\n### group_size\n\nInsertions will be broken up into batches. This specifies the number of records you want to insert per batch. Default is 1,000.\n\n### variable_column\n\nThe name of the column which we will be dynamically inserting records for. This is the only column which changes per-record being inserted.\n\n### values\n\nThe large list of values to use for the 'variable_column' value when inserting the records.\n\n## Multiple Variable Columns\nRather than a single `variable_column`, you may pass an array of `variable_columns`, along with `values` as an array of arrays.\n\nExample:\n```\nvariable_columns: %w(user_id user_email)\nvalues: [[1, 'foo@example.com'], [2, 'bar@example.com']]\n```\n\n## Development\n\nAfter checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.\n\nTo install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).\n\n## Contributing\n\nBug reports and pull requests are welcome on GitHub at https://github.com/joinhandshake/fast_inserter. All code must run on sqlite, pg, and mysql (tests are set up CI already).\n\n\n## License\n\nThe gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoinhandshake%2Ffast_inserter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjoinhandshake%2Ffast_inserter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoinhandshake%2Ffast_inserter/lists"}