{"id":33996449,"url":"https://github.com/carandclassic/private-dump","last_synced_at":"2025-12-13T08:05:01.738Z","repository":{"id":49452395,"uuid":"166579859","full_name":"carandclassic/private-dump","owner":"carandclassic","description":"Dump and anonymise your MySQL data","archived":false,"fork":false,"pushed_at":"2023-06-09T08:34:26.000Z","size":3873,"stargazers_count":6,"open_issues_count":6,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-10-21T06:29:11.324Z","etag":null,"topics":["anonymise","gdpr","mysql-backup","mysqldump","php"],"latest_commit_sha":null,"homepage":"","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/carandclassic.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-01-19T18:30:59.000Z","updated_at":"2025-09-17T13:46:55.000Z","dependencies_parsed_at":"2022-09-13T00:31:33.156Z","dependency_job_id":null,"html_url":"https://github.com/carandclassic/private-dump","commit_stats":null,"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/carandclassic/private-dump","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/carandclassic%2Fprivate-dump","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/carandclassic%2Fprivate-dump/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/carandclassic%2Fprivate-dump/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/carandclassic%2Fprivate-dump/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/carandclassic","download_url":"https://codeload.github.com/carandclassic/private-dump/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/carandclassic%2Fprivate-dump/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":27702409,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-13T02:00:09.769Z","response_time":147,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anonymise","gdpr","mysql-backup","mysqldump","php"],"created_at":"2025-12-13T08:05:01.168Z","updated_at":"2025-12-13T08:05:01.730Z","avatar_url":"https://github.com/carandclassic.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Private Dump\nPrivate Dump is a CLI tool which can create an anonymised dump of your MySQL database, usually for development or staging use.\n\nIt accomplishes this by reading a JSON configuration file which maps out which table columns should be modified and how.\n\n\n![travis-ci-build-status](https://travis-ci.com/carandclassic/private-dump.svg?branch=master) ![styleci-status](https://github.styleci.io/repos/166579859/shield)\n\nPrivate Dump requires PHP \u003e= 7.3\n\n# Table of Contents\n\n- [Installation](#installation)\n- [Usage](#usage)\n- [Configuration File](#configuration-file)\n  - [Key Value Tables](#key-value-tables)\n- [Replacements](#replacements)\n  - [Text](#text)\n  - [Dates](#dates)\n  - [Internet](#internet)\n  - [Random](#random)\n  - [User](#user)\n  - [Payment](#payment)\n  - [Company](#company)\n  - [Miscellaneous](#miscellaneous)\n  - [Barcodes](#barcodes)\n- [Transformers](#transformers)\n- [Notes](#notes)\n- [Dev Steps](#dev-steps)\n- [Release Process](#release-process)\n\n\n\n# Installation\n\nInstall with [Composer](https://getcomposer.org/)\n\n```\ncomposer require ashleyhindle/private-dump\n```\n\n\n\nInstall with [curl](https://curl.haxx.se/)\n\n```bash\ncurl -Lo private-dump https://github.com/cazana/private-dump/releases/download/v0.1.1/private-dump\nchmod a+x private-dump \n```\n\n\n\n# Usage\n\nWhen private-dump is ran with a valid configuration it will output the dump to stdout, allowing you to redirect it to a file, or pipe it to another program (for compression, transfer, encryption, etc..).\n\n\n\nFirst, create a [configuration file](#configuration-file) manually or from an [example config](configs/), then:\n\n\n\n**Composer**:\n\n- `vendor/bin/private-dump.phar -c private-dump.json \u003e /backups/mysql-backup-with-sensitive-data-overwritten.sql`\n\n**Curl**:\n- `private-dump -c private-dump.json \u003e /backups/mysql-backup-with-sensitive-data-overwritten.sql`\n\n\n\nYou can also override the MySQL username, password and hostname from the command line:\n\n**Composer**:\n```bash\nvendor/bin/private-dump.phar -c private-dump.json -u bigben -p bingbong -h rds-213121231-13gb.amazon.com \u003e /backups/mysql-backup-with-sensitive-data-overwritten.sql\n```\n\n**Curl**:\n```bash\nprivate-dump -c private-dump.json -u bigben -p bingbong -h rds-213121231-13gb.amazon.com \u003e /backups/mysql-backup-with-sensitive-data-overwritten.sql\n```\n\n\n\n*Note*: It's best not to pass the password on the command line as it can be seen in the process list, and will exist in the user's history. \n\n\n\n# Configuration File\n\nThe configuration file is a JSON dictionary containing:\n\n- database connection details\n- list of databases to dump\n- list of tables with replacements or transformers\n  - Seeder - one column can be used as a row-seeder to get repeatability in consecutive dumps\n  - keepers - some rows can be kept as is, if a specified column matches a regular expression\n  - columns with replacements or transformers\n  - options to restrict output\n    - `where` - Added to the query when retrieving data to dump: `... WHERE xxx...`\n    - `limit` - Added to the query when retrieving data to dump: `... LIMIT xxx...`\n\n\n\nAn example configuration, and configurations for popular applications, exist in the [configs](configs/) directory.\n\n\n\n**Annotated Example Configuration - private-dump.json**\n\n```\n{\n    \"connection\": { -- Database connection details\n        \"username\": \"bigben\",\n        \"password\": \"bingbong\",\n        \"hostname\": \"192.168.56.81\"\n    },\n    \"databases\": { -- Databases to dump - databases not present will not be dumped\n        \"databaseName\": {  -- All tables in this database are dumped\n            \"tableName\": { -- Referenced tables allow replacements\n                \"email_address\": \"@email\", -- Column and its replacement\n                \"full_name\": \"@fullName\",\n                \"is_active\": 0, --  Column with hardcoded value\n                \"$options\": { -- Special options array for limit/where\n                    \"where\": \"last_login \u003e NOW() - INTERVAL 1 WEEK\",\n                    \"limit\": 25\n                }\n            }\n        },\n        \"databaseTwo\": {}, -- Dump entire database and tables, with no replacements\n        \"databaseThree\": {\n            \"users\": {\n                \"$options\": { -- Only options to limit data, no replacements\n                    \"where\": \"is_active=1\"\n                }\n            }\n        }\n        \"databaseThree\": {\n            \"users\": {\n                \"@seed\": \"id\", -- Seed each row using user id, to get repeatability\n                \"@keepif\": { -- Keep all rows with emails using domain @ourdomain.com intact\n                    \"column\": \"email\",\n                    \"regex\": \"^.*@ourdomain.com$\"\n                }\n                \"first_name\": \"@user(current_user).firstName\",\n                \"last_name\": \"@user(current_user).lastName\",\n                \"email\": \"@user(current_user).email\", - email will be based on the first and last names\n            }\n        }\n    }\n}\n```\n\n\n\n#### Key Value Tables\n\nPrivate Dump supports replacing values in a key-value store, by using an array in the configuration file to link the `value` column with the `key` column as below:\n\n```json\n{\n    \"connection\": {...},\n    \"databases\": {\n        \"wordpress\": {\n            \"wp_options\": {\n                \"option_value\": {\n                    \"$link\": \"option_name\",\n                    \"$transformers\": {\n                        \"admin_email\": \"@email\",\n                        \"mailserver_pass\": \"@password\",\n                        \"autoload\": \"yes\"\n                    }\n                }\n            }\n        }\n    }\n}\n```\n\n\n\nThis is a bit more complicated than the standard replacements, but offers a lot of flexibility for anonymising all types of data.\n\n# Replacements\n\nThe vast majority of these are made possible by the amazing [Faker library](https://github.com/fzaninotto/Faker).  Most formatters listed in [Faker's documentation](https://github.com/fzaninotto/Faker#formatters) are supported in Private Dump's configuration file\n\nAll replacements below should be prefixed with an `@` as in the [example configuration files](configs).\n\nIf you need to use a hardcoded value (active=0, completed=1) you can do this by omitting the `@`: `\"active\": 0` in the configuration file.  \n\nYou can pass variables to commands as such `@numberBetween|100,1000`\n\n\n#### Text\n\n- `original` - The original value, useful to use with modifiers\n- `string` - Random length string up to 255 characters\n- `realText` - Quotes from books\n- `loremSentence` - 1 sentence of Lorem\n- `loremParagraph` - 3 sentences of Lorem\n- `loremParagraphs` - 3 paragraphs of Lorem\n\n#### Dates\n\n- `iso8601` - 2019-01-20\n- `iso8601Recent` - ISO 8601 date in the last 3 months\n\n#### Internet\n- `email` - bigben@example.com\n- `url` - https://www.parliament.uk/bigben\n- `ipv4`\n- `ipv6`\n- `userAgent`\n- `domainName` - bigben.net\n- `slug` - big-ben-bing-bong\n\n#### Random\n\n- `randomDigit` - singular digit\n- `randomNumber` - up to 8 digits\n- `randomLetter` \n- `randomString` - Random length string up to 255 characters\n\n#### User\n\n- `firstName`\n- `lastName`\n- `title` - Ms. Mr. Dr.\n- `fullName` - Brian May\n- `fullAddress` - One line: Building number, street, city, state/county, postcode/zip\n- `buildingNumber` - 368\n- `streetName` - Broadway\n- `streetAddress` - 368 Broadway\n- `city` - London\n- `postcode` - SW1A 0AA\n- `country` - England\n- `state` - Texas\n- `county` - London\n- `latitude` - 51.5008\n- `longitude` - `-.1246`\n- `phoneNumber`\n- `email` - bigben@example.com\n- `username` - BigBen\n- `password`\n- `url` - https://www.parliament.uk/bigben\n- `ipv4` - IPv4 Address\n- `ipv6` - IPv6 Address\n\n#### Payment\n\n- `creditCardType` - Mastercard\n- `creditCardNumber` - 4444 1111 2222 3333\n- `creditCardExpirationDate` - 04/22\n- `creditCardExpirationDateString` - '04/13'\n- `iban` - BI6B3N8497112740YZ575DJ28BP4\n- `swiftBicNumber` - BIGBEN22263\n\n#### Company\n\n- `company` - Company-Name\n- `jobTitle` - Croupier\n\n#### Miscellaneous\n\n- `boolean`\n- `md5`\n- `sha1`\n- `sha256`\n- `countryCode` - UK\n- `currencyCode` - EUR\n\n#### Barcodes\n\n- `barcodeEan13`\n- `barcodeEan8`\n- `barcodeIsbn13`\n- `barcodeIsbn10`\n\n\n\n# Transformers\n\n\n\n- `uppercase`\n- `lowercase`\n\n\n\n----\n\n\n\n*These notes are mainly for my own development use, feel free to ignore.*\n\n\n\n# Dev Steps\n\n1. Install [Box](https://box-project.github.io/box2/)\n2. Modify PHP configuration to set `phar.readonly = Off`\n3. `box build`\n4. `chmod a+x bin/private-dump.phar`\n\n\n# Release Process\n\n1. Build the PHAR: `box build`\n2. Rename the PHAR: `mv bin/private-dump.phar ./private-dump`\n3. Update the version in `README.md`'s installation instructions based on the next version from `git tag --list`\n4. Tag the next release: `git tag -a vx.x.x -m \"Release x.x.x\"`\n5. Push: `git push origin --tags`\n6. [Edit release on GitHub](https://github.com/cazana/private-dump/releases/) attaching the newly created `bin/private-dump` file\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcarandclassic%2Fprivate-dump","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcarandclassic%2Fprivate-dump","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcarandclassic%2Fprivate-dump/lists"}