{"id":13395011,"url":"https://github.com/IFTTT/polo","last_synced_at":"2025-03-13T20:31:51.765Z","repository":{"id":35590835,"uuid":"39863646","full_name":"IFTTT/polo","owner":"IFTTT","description":"Polo travels through your database and creates sample snapshots so you can work with real world data in development.","archived":false,"fork":false,"pushed_at":"2023-10-11T19:03:59.000Z","size":105,"stargazers_count":774,"open_issues_count":16,"forks_count":49,"subscribers_count":33,"default_branch":"main","last_synced_at":"2025-03-10T06:34:20.938Z","etag":null,"topics":["activerecord","database","productivity","snapshot"],"latest_commit_sha":null,"homepage":"http://ifttt.github.io","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/IFTTT.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2015-07-28T23:57:57.000Z","updated_at":"2025-02-11T21:32:01.000Z","dependencies_parsed_at":"2024-01-05T21:59:00.152Z","dependency_job_id":null,"html_url":"https://github.com/IFTTT/polo","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IFTTT%2Fpolo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IFTTT%2Fpolo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IFTTT%2Fpolo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IFTTT%2Fpolo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/IFTTT","download_url":"https://codeload.github.com/IFTTT/polo/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243478269,"owners_count":20297226,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["activerecord","database","productivity","snapshot"],"created_at":"2024-07-30T17:01:39.088Z","updated_at":"2025-03-13T20:31:51.119Z","avatar_url":"https://github.com/IFTTT.png","language":"Ruby","funding_links":[],"categories":["Ruby","Database Tools"],"sub_categories":[],"readme":"[![Open Source at IFTTT](http://ifttt.github.io/images/open-source-ifttt.svg)](http://ifttt.github.io)\n[![Build Status](https://travis-ci.org/IFTTT/polo.svg?branch=master)](https://travis-ci.org/IFTTT/polo)\n\n![Polo](https://raw.githubusercontent.com/IFTTT/polo/images/images/polo.png \"Polo\")\n\n# Polo\nPolo travels through your database and creates sample snapshots so you can work with real world data in any environment.\n\nPolo takes an `ActiveRecord::Base` seed object and traverses every whitelisted `ActiveRecord::Association` generating SQL `INSERTs` along the way.\n\nYou can then save those SQL `INSERTS` to .sql file and import the data to your favorite environment.\n\n## Motivation\nRead our [blog post](https://medium.com/engineering-at-ifttt/happier-rails-development-with-polo-9df6819136d3#.f8ll3azeq) or check out this [presentation](https://speakerdeck.com/nettofarah/polo-working-with-real-world-data-in-development).\n\n## Usage\nGiven the following data model:\n```ruby\nclass Chef \u003c ActiveRecord::Base\n  has_many :recipes\n  has_many :ingredients, through: :recipes\nend\n\nclass Recipe \u003c ActiveRecord::Base\n  has_many :recipes_ingredients\n  has_many :ingredients, through: :recipes_ingredients\nend\n\nclass Ingredient \u003c ActiveRecord::Base\nend\n\nclass RecipesIngredient \u003c ActiveRecord::Base\n  belongs_to :recipe\n  belongs_to :ingredient\nend\n```\n\n### Simple ActiveRecord Objects\n```ruby\ninserts = Polo.explore(Chef, 1)\n```\n```sql\nINSERT INTO `chefs` (`id`, `name`) VALUES (1, 'Netto')\n```\n\nWhere `Chef` is the seed object class, and `1` is the seed object id.\n\n### Simple Associations\n```ruby\ninserts = Polo.explore(Chef, 1, :recipes)\n```\n```sql\nINSERT INTO `chefs` (`id`, `name`) VALUES (1, 'Netto')\nINSERT INTO `recipes` (`id`, `title`, `num_steps`, `chef_id`) VALUES (1, 'Turkey Sandwich', NULL, 1)\nINSERT INTO `recipes` (`id`, `title`, `num_steps`, `chef_id`) VALUES (2, 'Cheese Burger', NULL, 1)\n```\n\n### Complex nested associations\n```ruby\ninserts = Polo.explore(Chef, 1, :recipes =\u003e :ingredients)\n```\n\n```sql\nINSERT INTO `chefs` (`id`, `name`) VALUES (1, 'Netto')\nINSERT INTO `recipes` (`id`, `title`, `num_steps`, `chef_id`) VALUES (1, 'Turkey Sandwich', NULL, 1)\nINSERT INTO `recipes` (`id`, `title`, `num_steps`, `chef_id`) VALUES (2, 'Cheese Burger', NULL, 1)\nINSERT INTO `recipes_ingredients` (`id`, `recipe_id`, `ingredient_id`) VALUES (1, 1, 1)\nINSERT INTO `recipes_ingredients` (`id`, `recipe_id`, `ingredient_id`) VALUES (2, 1, 2)\nINSERT INTO `recipes_ingredients` (`id`, `recipe_id`, `ingredient_id`) VALUES (3, 2, 3)\nINSERT INTO `recipes_ingredients` (`id`, `recipe_id`, `ingredient_id`) VALUES (4, 2, 4)\nINSERT INTO `ingredients` (`id`, `name`, `quantity`) VALUES (1, 'Turkey', 'a lot')\nINSERT INTO `ingredients` (`id`, `name`, `quantity`) VALUES (2, 'Cheese', '1 slice')\nINSERT INTO `ingredients` (`id`, `name`, `quantity`) VALUES (3, 'Patty', '1')\nINSERT INTO `ingredients` (`id`, `name`, `quantity`) VALUES (4, 'Cheese', '2 slices')\n```\n\n## Advanced Usage\n\nOccasionally, you might have a dataset that you want to refresh. A production database that has data that might be useful on your local copy of the database. Polo doesn't have an opinion about your data; if you try to import data with a key that's already in your local database, Polo doesn't necessarily know how you want to handle that conflict.\n\nAdvanced users will find the `on_duplicate` option to be helpful in this context. It gives Polo instructions on how to handle collisions.\n*Note: This feature is currently only supported for MySQL databases. (PRs for other databases are welcome!)*\n\nThere are two possible values for the `on_duplicate` key: `:ignore` and `:override`. Ignore keeps the old data. Override keeps the new data. If there's a collision and the on_duplicate param is not set, Polo will simpy stop importing the data.\n\n### Ignore\nA.K.A the Ostrich Approach: stick your head in the sand and pretend nothing happened.\n\n```ruby\nPolo.configure do\n  on_duplicate :ignore\nend\n\nPolo::Traveler.explore(Chef, 1, :recipes)\n```\n\n```sql\nINSERT IGNORE INTO `chefs` (`id`, `name`) VALUES (1, 'Netto')\nINSERT IGNORE INTO `recipes` (`id`, `title`, `num_steps`, `chef_id`) VALUES (1, 'Turkey Sandwich', NULL, 1)\nINSERT IGNORE INTO `recipes` (`id`, `title`, `num_steps`, `chef_id`) VALUES (2, 'Cheese Burger', NULL, 1)\n```\n\n### Override\nUse the option `on_duplicate: :override` to override your local data with new data from your Polo script.\n\n```ruby\nPolo.configure do\n  on_duplicate :override\nend\n\nPolo::Traveler.explore(Chef, 1, :recipes)\n```\n\n```sql\nINSERT INTO `chefs` (`id`, `name`) VALUES (1, 'Netto')\nON DUPLICATE KEY UPDATE id = VALUES(id), name = VALUES(name)\n...\n```\n\n### Sensitive Fields\nYou can use the `obfuscate` option to obfuscate sensitive fields like emails or\nuser logins.\n\n```ruby\nPolo.configure do\n  obfuscate :email, :credit_card\nend\n\nPolo::Traveler.explore(AR::Chef, 1)\n```\n\n```sql\nINSERT INTO `chefs` (`id`, `name`, `email`) VALUES (1, 'Netto', 'eahorctmaagfo.nitm@l')\n```\n\nWarning: This is not a security feature. Fields can still easily be rearranged back to their original format. Polo will simply scramble the order of strings so you don't accidentally end up causing side effects when using production data in development. It is not a good practice to use highly sensitive data in development.\n\n#### Advanced Obfuscation\n\nFor more advanced obfuscation, you can pass in a custom obfuscation strategy. Polo will take in a lambda that can be used to transform sensitive data.\n\nUsing a `:symbol` as an obfuscate key targets all columns of that name. Passing an SQL selector as a `String` will target columns within the specified table.\n\n````ruby\nPolo.configure do\n\n  email_strategy = lambda do |email|\n    first_part = email.split(\"@\")[0]\n    \"#{first_part}@test.com\"\n  end\n\n  credit_card_strategy = lambda do |credit_card|\n    \"4123 4567 8910 1112\"\n  end\n\n  # If you need the context of the record for its fields, it is accessible\n  # in the second argument of the strategy\n  social_security_strategy = lambda do |ssn, instance|\n    sprintf(\"%09d\", instance.id)\n  end\n\n  obfuscate({\n    'chefs.email' =\u003e email_strategy, # This only applies to the \"email\" column in the \"chefs\" table\n    :credit_card  =\u003e credit_card_strategy, # This applies to any column named \"credit_card\" across every table\n    :ssn_strategy =\u003e social_security_strategy\n  })\nend\n\nPolo::Traveler.explore(AR::Chef, 1)\n````\n\n```sql\nINSERT INTO `chefs` (`id`, `name`, `email`) VALUES (1, 'Netto', 'netto_test.@example.com')\n```\n\n## Installation\n\nAdd this line to your application's Gemfile:\n\n```ruby\ngem 'polo'\n```\n\nAnd then execute:\n```bash\n$ bundle\n```\n\nOr install it yourself as:\n```bash\n$ gem install polo\n```\n\n## Contributing\n\nBug reports and pull requests are welcome on GitHub at https://github.com/IFTTT/polo. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Code of Conduct](https://github.com/IFTTT/polo/blob/master/CODE_OF_CONDUCT.md).\n\nTo run the specs across all supported version of Rails, check out the repo and\nfollow these steps:\n\n```bash\n$ bundle install\n$ bundle exec appraisal install\n$ bundle exec appraisal rake\n```\n\n## License\n\nThe gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FIFTTT%2Fpolo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FIFTTT%2Fpolo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FIFTTT%2Fpolo/lists"}