{"id":13484333,"url":"https://github.com/jamis/bulk_insert","last_synced_at":"2025-11-11T20:50:21.253Z","repository":{"id":1749221,"uuid":"44027165","full_name":"jamis/bulk_insert","owner":"jamis","description":"Efficient bulk inserts with ActiveRecord","archived":false,"fork":false,"pushed_at":"2022-01-10T23:16:43.000Z","size":135,"stargazers_count":817,"open_issues_count":12,"forks_count":81,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-04-13T18:44:39.068Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jamis.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"MIT-LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-10-10T20:30:37.000Z","updated_at":"2025-03-28T11:01:54.000Z","dependencies_parsed_at":"2022-07-20T06:47:05.648Z","dependency_job_id":null,"html_url":"https://github.com/jamis/bulk_insert","commit_stats":null,"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jamis%2Fbulk_insert","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jamis%2Fbulk_insert/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jamis%2Fbulk_insert/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jamis%2Fbulk_insert/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jamis","download_url":"https://codeload.github.com/jamis/bulk_insert/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254227581,"owners_count":22035664,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T17:01:22.668Z","updated_at":"2025-11-11T20:50:21.195Z","avatar_url":"https://github.com/jamis.png","language":"Ruby","readme":"# BulkInsert\n\nA little ActiveRecord extension for helping to insert lots of rows in a\nsingle insert statement.\n\n## Installation\n\nAdd it to your Gemfile:\n\n```ruby\ngem 'bulk_insert'\n```\n\n## Usage\n\nBulkInsert adds a new class method to your ActiveRecord models:\n\n```ruby\nclass Book \u003c ActiveRecord::Base\nend\n\nbook_attrs = ... # some array of hashes, for instance\nBook.bulk_insert do |worker|\n  book_attrs.each do |attrs|\n    worker.add(attrs)\n  end\nend\n```\n\nAll of those `#add` calls will be accumulated into a single SQL insert\nstatement, vastly improving the performance of multiple sequential\ninserts (think data imports and the like).\n\nIf you don't like using a block API, you can also simply pass an array\nof rows to be inserted:\n\n```ruby\nbook_attrs = ... # some array of hashes, for instance\nBook.bulk_insert values: book_attrs\n```\n\nBy default, the columns to be inserted will be all columns in the table,\nminus the `id` column, but if you want, you can explicitly enumerate\nthe columns:\n\n```ruby\nBook.bulk_insert(:title, :author) do |worker|\n  # specify a row as an array of values...\n  worker.add [\"Eye of the World\", \"Robert Jordan\"]\n\n  # or as a hash\n  worker.add title: \"Lord of Light\", author: \"Roger Zelazny\"\nend\n```\n\nIt will automatically set `created_at`/`updated_at` columns to the current\ndate, as well.\n\n```ruby\nBook.bulk_insert(:title, :author, :created_at, :updated_at) do |worker|\n  # specify created_at/updated_at explicitly...\n  worker.add [\"The Chosen\", \"Chaim Potok\", Time.now, Time.now]\n\n  # or let BulkInsert set them by default...\n  worker.add [\"Hello Ruby\", \"Linda Liukas\"]\nend\n```\n\nSimilarly, if a value is omitted, BulkInsert will use whatever default\nvalue is defined for that column in the database:\n\n```ruby\n# create_table :books do |t|\n#   ...\n#   t.string \"medium\", default: \"paper\"\n#   ...\n# end\n\nBook.bulk_insert(:title, :author, :medium) do |worker|\n  worker.add title: \"Ender's Game\", author: \"Orson Scott Card\"\nend\n\nBook.first.medium #-\u003e \"paper\"\n```\n\nBy default, the batch is always saved when the block finishes, but you\ncan explicitly save inside the block whenever you want, by calling\n`#save!` on the worker:\n\n```ruby\nBook.bulk_insert do |worker|\n  worker.add(...)\n  worker.add(...)\n\n  worker.save!\n\n  worker.add(...)\n  #...\nend\n```\n\nThat will save the batch as it has been defined to that point, and then\nempty the batch so that you can add more rows to it if you want. Note\nthat all records saved together will have the same created_at/updated_at\ntimestamp (unless one was explicitly set).\n\n### Batch Set Size\n\nBy default, the size of the insert is limited to 500 rows at a time.\nThis is called the _set size_. If you add another row that causes the\nset to exceed the set size, the insert statement is automatically built\nand executed, and the batch is reset.\n\nIf you want a larger (or smaller) set size, you can specify it in\ntwo ways:\n\n```ruby\n# specify set_size when initializing the bulk insert...\nBook.bulk_insert(set_size: 100) do |worker|\n  # ...\nend\n\n# specify it on the worker directly...\nBook.bulk_insert do |worker|\n  worker.set_size = 100\n  # ...\nend\n```\n\n### Insert Ignore\n\nBy default, when an insert fails the whole batch of inserts fail. The\n_ignore_ option ignores the inserts that would have failed (because of\nduplicate keys or a null in column with a not null constraint) and\ninserts the rest of the batch.\n\nThis is not the default because no errors are raised for the bad\ninserts in the batch.\n\n```ruby\ndestination_columns = [:title, :author]\n\n# Ignore bad inserts in the batch\nBook.bulk_insert(*destination_columns, ignore: true) do |worker|\n  worker.add(...)\n  worker.add(...)\n  # ...\nend\n```\n\n### Update Duplicates (MySQL, PostgreSQL)\n\nIf you don't want to ignore duplicate rows but instead want to update them\nthen you can use the _update_duplicates_ option. Set this option to true\n(MySQL) or list unique column names (PostgreSQL) and when a duplicate row\nis found the row will be updated with your new values.\nDefault value for this option is false.\n\n```ruby\ndestination_columns = [:title, :author]\n\n# Update duplicate rows (MySQL)\nBook.bulk_insert(*destination_columns, update_duplicates: true) do |worker|\n  worker.add(...)\n  worker.add(...)\n  # ...\nend\n\n# Update duplicate rows (PostgreSQL)\nBook.bulk_insert(*destination_columns, update_duplicates: %w[title]) do |worker|\n  worker.add(...)\n  # ...\nend\n```\n\n### Return Primary Keys (PostgreSQL, PostGIS)\n\nIf you want the worker to store primary keys of inserted records, then you can\nuse the _return_primary_keys_ option. The worker will store a `result_sets`\narray of `ActiveRecord::Result` objects. Each `ActiveRecord::Result` object\nwill contain the primary keys of a batch of inserted records.\n\n```ruby\nworker = Book.bulk_insert(*destination_columns, return_primary_keys: true) do\n|worker|\n  worker.add(...)\n  worker.add(...)\n  # ...\nend\n\nworker.result_sets\n```\n\n## Ruby and Rails Versions Supported\n\n\u003e :warning: The scope of this gem may be somehow covered natively by the `.insert_all` API\n\u003e introduced by [Rails 6](https://apidock.com/rails/v6.0.0/ActiveRecord/Persistence/ClassMethods/insert_all).\n\u003e This gem represents the state of art for rails version \u003c 6 and it is still open to\n\u003e further developments for more recent versions.\n\nThe current CI prevents regressions on the following versions:\n\nruby / rails | `~\u003e3` | `~\u003e4` | `~\u003e5` | `~\u003e6`\n:-----------:|-------|-------|-------|------\n2.2          |  yes  |  yes  |  no   |  no\n2.3          |  yes  |  yes  |  yes  |  no\n2.4          |  no   |  yes  |  yes  |  no\n2.5          |  no   |  no   |  yes  |  yes\n2.6          |  no   |  no   |  yes  |  yes\n2.7          |  no   |  no   |  yes  |  yes\n\nThe adapters covered in the CI are:\n* sqlite\n* mysql\n* postgresql\n\n\n## License\n\nBulkInsert is released under the MIT license (see MIT-LICENSE) by\nJamis Buck (jamis@jamisbuck.org).\n","funding_links":[],"categories":["Ruby","ORM/ODM Extensions"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjamis%2Fbulk_insert","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjamis%2Fbulk_insert","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjamis%2Fbulk_insert/lists"}