{"id":13878312,"url":"https://github.com/thoughtbot/art_vandelay","last_synced_at":"2025-04-10T03:58:20.205Z","repository":{"id":64497939,"uuid":"547819170","full_name":"thoughtbot/art_vandelay","owner":"thoughtbot","description":"Art Vandelay is an importer/exporter for Rails 7.0 and higher.","archived":false,"fork":false,"pushed_at":"2024-07-19T21:57:32.000Z","size":62,"stargazers_count":85,"open_issues_count":9,"forks_count":4,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-10T03:57:27.374Z","etag":null,"topics":["csv-export","csv-import","engine","exporter","gem","importer","rails"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thoughtbot.png","metadata":{"files":{"readme":"README.md","changelog":"NEWS.md","contributing":null,"funding":null,"license":"MIT-LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-10-08T10:59:38.000Z","updated_at":"2024-12-17T21:32:33.000Z","dependencies_parsed_at":"2024-06-26T23:01:35.808Z","dependency_job_id":"02065810-f13e-48cc-812d-0b14f22c0186","html_url":"https://github.com/thoughtbot/art_vandelay","commit_stats":{"total_commits":16,"total_committers":4,"mean_commits":4.0,"dds":0.25,"last_synced_commit":"738d9aef597e8ac202619ccb7b21df723b36a8bc"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thoughtbot%2Fart_vandelay","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thoughtbot%2Fart_vandelay/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thoughtbot%2Fart_vandelay/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thoughtbot%2Fart_vandelay/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thoughtbot","download_url":"https://codeload.github.com/thoughtbot/art_vandelay/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248154996,"owners_count":21056542,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["csv-export","csv-import","engine","exporter","gem","importer","rails"],"created_at":"2024-08-06T08:01:46.033Z","updated_at":"2025-04-10T03:58:20.183Z","avatar_url":"https://github.com/thoughtbot.png","language":"Ruby","funding_links":[],"categories":["Ruby"],"sub_categories":[],"readme":"# 🌐 Art Vandelay\n[![GitHub Actions\nDemo](https://github.com/thoughtbot/art_vandelay/actions/workflows/ci.yml/badge.svg)](https://github.com/thoughtbot/art_vandelay/actions/workflows/ci.yml)\n\nArt Vandelay is an importer/exporter for Rails 7.0 and higher.\n\nHave you ever been on a project where, out of nowhere, someone asks you to send them a CSV of data? You think to yourself, “Ok, cool. No big deal. Just gimme five minutes”, but then that five minutes turns into a few hours. Art Vandelay can help.\n\n**At a high level, here’s what Art Vandelay can do:**\n\n- 🕶 Automatically [filters out sensitive information](#%EF%B8%8F-configuration).\n- 🔁 Export data [in batches](#exporting-in-batches).\n- 📧 [Email](#artvandelayexportemail) exported data.\n- 📥 [Import data](#-importing) from a CSV or JSON file.\n\n## ✅ Installation\n\nAdd this line to your application's Gemfile:\n\n```ruby\ngem \"art_vandelay\"\n```\n\nAnd then execute:\n```bash\n$ bundle\n```\n\n## ⚙️ Configuration\n\n```ruby\n# config/initializers/art_vandelay.rb\nArtVandelay.setup do |config|\n  config.filtered_attributes = [:credit_card, :birthday]\n  config.from_address = \"no-reply-export@example.com\"\n  config.in_batches_of = 5000\nend\n```\n#### Default Values\n\n|Attribute|Value|Description|\n|---------|-----|-----------|\n|`filtered_attributes`|`[:passw, :secret, :token, :_key, :crypt, :salt, :certificate, :otp, :ssn]`|Attributes that will be automatically filtered when exported|\n|`from_address`|`nil`|The email address used when sending an email of exports|\n|`in_batches_of`|`10000`|The number of records that will be exported into each CSV|\n\n## 🧰 Usage\n\n### 📤 Exporting\n\nArt Vandelay supports exporting CSVs and JSON files.\n\n```ruby\nArtVandelay::Export.new(records, export_sensitive_data: false, attributes: [], in_batches_of: ArtVandelay.in_batches_of)\n```\n\n|Argument|Description|\n|--------|-----------|\n|`records`|An [Active Record Relation](https://api.rubyonrails.org/classes/ActiveRecord/Relation.html) or an instance of an Active Record. E.g. `User.all`, `User.first`, `User.where(...)`, `User.find_by`|\n|`export_sensitive_data`|Export sensitive data. Defaults to `false`. Can be configured with `ArtVandelay.filtered_attributes`.|\n|`attributes`|An array attributes to export. Default to all.|\n|`in_batches_of`|The number of records that will be exported into each file. Defaults to 10,000. Can be configured with `ArtVandelay.in_batches_of`|\n\n#### ArtVandelay::Export#csv\n\nReturns an instance of `ArtVandelay::Export::Result`.\n\n```ruby\nresult = ArtVandelay::Export.new(User.all).csv\n# =\u003e #\u003cArtVandelay::Export::Result\u003e\n\ncsv_exports = result.csv_exports\n# =\u003e [#\u003cCSV::Table\u003e, #\u003cCSV::Table\u003e, ...]\n\ncsv = csv_exports.first.to_a\n# =\u003e [[\"id\", \"email\", \"password\", \"created_at\", \"updated_at\"], [\"1\", \"user@example.com\", \"[FILTERED]\", \"2022-10-25 09:20:28 UTC\", \"2022-10-25 09:20:28 UTC\"]]\n```\n\n#### ArtVandelay::Export#json\n\nReturns an instance of `ArtVandelay::Export::Result`.\n\n```ruby\nresult = ArtVandelay::Export.new(User.all).json\n# =\u003e #\u003cArtVandelay::Export::Result\u003e\n\njson_exports = result.json_exports\n# =\u003e [#\u003cCSV::Table\u003e, #\u003cCSV::Table\u003e, ...]\n\njson = JSON.parse(json_exports.first)\n# =\u003e [{\"id\"=\u003e1, \"email\"=\u003e\"user@example.com\", \"password\"=\u003e\"[FILTERED]\", \"created_at\"=\u003e\"2022-10-25 09:20:28.123Z\", \"updated_at\"=\u003e\"2022-10-25 09:20:28.123Z\"}]\n```\n\n##### Exporting Sensitive Data\n\n```ruby\nresult = ArtVandelay::Export.new(User.all, export_sensitive_data: true).csv\n# =\u003e #\u003cArtVandelay::Export::Result\u003e\n\npassword = result.csv_exports.first[\"password\"]\n# =\u003e [\"bosco\"]\n```\n\n##### Exporting Specific Attributes\n\n```ruby\nresult = ArtVandelay::Export.new(User.all, attributes: [:email]).csv\n# =\u003e #\u003cArtVandelay::Export::Result\u003e\n\ncsv = result.csv_exports.first.to_a\n# =\u003e [[\"email\"], [\"george@vandelay_industries.com\"]]\n```\n\n##### Exporting in Batches\n\n```ruby\nresult = ArtVandelay::Export.new(User.all, in_batches_of: 100).csv\n# =\u003e #\u003cArtVandelay::Export::Result\u003e\n\ncsv_size = result.csv_exports.first.size\n# =\u003e 100\n```\n\n#### ArtVandelay::Export#email\n\nEmails the recipient(s) exports as attachments.\n\n```ruby\nemail(to:, from: ArtVandelay.from_address, subject: \"#{model_name} export\", body: \"#{model_name} export\")\n```\n\n|Argument|Description|\n|---------|-----|\n|`to`|An array of email addresses representing who should receive the email.|\n|`from`|The email address of the sender.|\n|`subject`|The email subject. Defaults to the following pattern: \"User export\"|\n|`body`|The email body. Defaults to the following pattern: \"User export\"|\n|`format`|The format of the export file. Either `:csv` or `:json`.|\n\n```ruby\nArtVandelay::Export\n  .new(User.where.not(confirmed: nil))\n  .email(\n    to: [\"george@vandelay_industries.com\", \"kel_varnsen@vandelay_industries.com\"],\n    from: \"noreply@vandelay_industries.com\",\n    subject: \"List of confirmed users\",\n    body: \"Here's an export of all confirmed users in our database.\",\n    format: :json\n  )\n# =\u003e ActionMailer::Base#mail: processed outbound mail in...\n```\n\n### 📥 Importing\n\n```ruby\nArtVandelay::Import.new(model_name, **options)\n```\n\n|Argument|Description|\n|--------|-----------|\n|`model_name`|The name of the model being imported. E.g. `:users`, `:user`, `\"users\"` or `\"user\"`|\n|`**options`|A hash of options. Available options are `rollback:`, `strip:`|\n\n#### Options\n\n|Option|Description|\n|------|-----------|\n|`rollback:`|Whether the import should rollback if any of the records fails to save.|\n|`strip:`|Strips leading and trailing whitespace from all values, including headers.|\n\n#### ArtVandelay::Import#csv\n\nImports records from the supplied CSV. Returns an instance of `ArtVandelay::Import::Result`.\n\n```ruby\ncsv_string = CSV.generate do |csv|\n  csv \u003c\u003c [\"email\", \"password\"]\n  csv \u003c\u003c [\"george@vandelay_industries.com\", \"bosco\"]\n  csv \u003c\u003c [\"kel_varnsen@vandelay_industries.com\", nil]\nend\n\nresult = ArtVandelay::Import.new(:users).csv(csv_string)\n# =\u003e #\u003cArtVandelay::Import::Result\u003e\n\nresult.rows_accepted\n# =\u003e [{:row=\u003e[\"george@vandelay_industries.com\", \"bosco\"], :id=\u003e1}]\n\nresult.rows_rejected\n# =\u003e [{:row=\u003e[\"kel_varnsen@vandelay_industries.com\", nil], :errors=\u003e{:password=\u003e[\"can't be blank\"]}}]\n```\n\n```ruby\ncsv(csv_string, **options)\n```\n\n|Argument|Description|\n|--------|-----------|\n|`csv_string`|Data in the form of a CSV string.|\n|`**options`|A hash of options. Available options are `headers:` and `attributes:`|\n\n##### Options\n\n|Option|Description|\n|------|-----------|\n|`headers:`|The CSV headers. Use when the supplied CSV string does not have headers.|\n|`attributes:`|The attributes the headers should map to. Useful if the headers do not match the model's attributes.|\n\n##### Setting headers\n\n```ruby\ncsv_string = CSV.generate do |csv|\n  csv \u003c\u003c [\"george@vandelay_industries.com\", \"bosco\"]\nend\n\nresult = ArtVandelay::Import.new(:users).csv(csv_string, headers: [:email, :password])\n# =\u003e #\u003cArtVandelay::Import::Result\u003e\n```\n\n#### ArtVandelay::Import#json\n\nImports records from the supplied JSON. Returns an instance of `ArtVandelay::Import::Result`.\n\n```ruby\njson_string = [\n  {\n    email: \"george@vandelay_industries.com\",\n    password: \"bosco\"\n  },\n  {\n    email: \"kel_varnsen@vanderlay_industries.com\",\n    password: nil\n  }\n].to_json\n\nresult = ArtVandelay::Import.new(:users).json(json_string)\n# =\u003e #\u003cArtVandelay::Import::Result\u003e\n\nresult.rows_accepted\n# =\u003e [{:row=\u003e[{\"email\"=\u003e\"george@vandelay_industries.com\", \"password\"=\u003e\"bosco\"}], :id=\u003e1}]\n\nresult.rows_rejected\n# =\u003e [{:row=\u003e[{\"email\"=\u003e\"kel_varnsen@vandelay_industries.com\", \"password\"=\u003enil}], :errors=\u003e{:password=\u003e[\"can't be blank\"]}}]\n```\n\n```ruby\njson(json_string, **options)\n```\n\n##### Options\n\n|Option|Description|\n|------|-----------|\n|`attributes:`|The attributes the JSON object keys should map to. Useful if the headers do not match the model's attributes.|\n\n#### Rolling back if a record fails to save\n\n`ArtVandelay::Import.new` supports a `:rollback` keyword argument. It imports all rows as a single transaction and does not persist any records if one record fails due to an exception.\n\n```ruby\ncsv_string = CSV.generate do |csv|\n  csv \u003c\u003c [\"email\", \"password\"]\n  csv \u003c\u003c [\"george@vandelay_industries.com\", \"bosco\"]\n  csv \u003c\u003c [\"kel_varnsen@vandelay_industries.com\", nil]\nend\n\nresult = ArtVandelay::Import.new(:users, rollback: true).csv(csv_string)\n# =\u003e rollback transaction\n```\n\n#### Mapping custom headers\n\nBoth `ArtVandelay::Import#csv` and `#json` support an `:attributes` keyword argument. This lets you map fields in the import document to your Active Record model's attributes.\n\n```ruby\ncsv_string = CSV.generate do |csv|\n  csv \u003c\u003c [\"email_address\", \"passcode\"]\n  csv \u003c\u003c [\"george@vandelay_industries.com\", \"bosco\"]\nend\n\nresult = ArtVandelay::Import.new(:users).csv(csv_string, attributes: {email_address: :email, passcode: :password})\n# =\u003e #\u003cArtVandelay::Import::Result\u003e\n```\n\n#### Stripping whitespace\n\n`ArtVandelay::Import.new` supports a `:strip` keyword argument to strip whitespace from values in the import document.\n\n```ruby\ncsv_string = CSV.generate do |csv|\n  csv \u003c\u003c [\"email_address  \", \" passcode  \"]\n  csv \u003c\u003c [\"  george@vandelay_industries.com  \", \"  bosco  \"]\nend\n\nresult = ArtVandelay::Import.new(:users, strip: true).csv(csv_string, attributes: {email_address: :email, passcode: :password})\n# =\u003e #\u003cArtVandelay::Import::Result\u003e\n\nresult.rows_accepted\n# =\u003e [{:row=\u003e[\"george@vandelay_industries.com\", \"bosco\"], :id=\u003e1}]\n```\n\n## 🙏 Contributing\n\n1. Run `./bin/setup`.\n2. Make your changes.\n3. Ensure `./bin/ci` passes.\n4. Create a [pull request](https://github.com/thoughtbot/art_vandelay/compare).\n\n## 📜 License\nThe gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthoughtbot%2Fart_vandelay","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthoughtbot%2Fart_vandelay","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthoughtbot%2Fart_vandelay/lists"}