{"id":14483128,"url":"https://github.com/rheinwerk-verlag/pganonymize","last_synced_at":"2025-07-05T23:08:04.398Z","repository":{"id":45436749,"uuid":"228773968","full_name":"rheinwerk-verlag/pganonymize","owner":"rheinwerk-verlag","description":"A commandline tool for anonymizing PostgreSQL databases","archived":false,"fork":false,"pushed_at":"2025-03-24T14:21:11.000Z","size":748,"stargazers_count":42,"open_issues_count":4,"forks_count":27,"subscribers_count":8,"default_branch":"development","last_synced_at":"2025-06-17T16:45:57.457Z","etag":null,"topics":["anonymization","anonymizer","cli","command-line","command-line-tool","database","developer-tools","dsgvo","gdpr","postgresql","privacy","python","python2","python27","python3"],"latest_commit_sha":null,"homepage":"http://pganonymize.readthedocs.io/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rheinwerk-verlag.png","metadata":{"files":{"readme":"README.rst","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.rst","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-12-18T06:32:40.000Z","updated_at":"2025-03-21T16:04:23.000Z","dependencies_parsed_at":"2024-05-31T15:14:40.426Z","dependency_job_id":"55ba1ef5-0091-4e51-b640-d49249f7733d","html_url":"https://github.com/rheinwerk-verlag/pganonymize","commit_stats":{"total_commits":454,"total_committers":15,"mean_commits":"30.266666666666666","dds":"0.20044052863436124","last_synced_commit":"84da6e9ddfb4d251a1f88f1d38389e265863eaae"},"previous_names":[],"tags_count":22,"template":false,"template_full_name":null,"purl":"pkg:github/rheinwerk-verlag/pganonymize","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rheinwerk-verlag%2Fpganonymize","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rheinwerk-verlag%2Fpganonymize/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rheinwerk-verlag%2Fpganonymize/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rheinwerk-verlag%2Fpganonymize/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rheinwerk-verlag","download_url":"https://codeload.github.com/rheinwerk-verlag/pganonymize/tar.gz/refs/heads/development","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rheinwerk-verlag%2Fpganonymize/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263819236,"owners_count":23516122,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anonymization","anonymizer","cli","command-line","command-line-tool","database","developer-tools","dsgvo","gdpr","postgresql","privacy","python","python2","python27","python3"],"created_at":"2024-09-03T00:01:32.384Z","updated_at":"2025-07-05T23:08:04.376Z","avatar_url":"https://github.com/rheinwerk-verlag.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"pganonymize\n===========\n\nA commandline tool to anonymize PostgreSQL databases for DSGVO/GDPR purposes.\n\nIt uses a YAML file to define which tables and fields should be anonymized and provides various methods of\nanonymization. The tool requires a direct PostgreSQL connection to perform the anonymization.\n\n.. class:: no-web no-pdf\n\n    |python| |license| |pypi| |downloads| |build| |codecov| |health|\n\n.. figure:: https://raw.githubusercontent.com/rheinwerk-verlag/pganonymize/main/docs/_static/demo.gif\n    :width: 100%\n\n.. contents::\n\nFeatures\n--------\n\n* Intentionally compatible with Python 2.7 (for old, productive platforms)\n* Anonymize PostgreSQL tables on data level entry with various providers (some examples in the table below)\n* Exclude data for anonymization depending on regular expressions or SQL ``WHERE`` clauses\n* Truncate entire tables for unwanted data\n\n+----------------+----------------------+-------------------------+----------------------------------+\n| Field          | Value                | Provider                | Output                           |\n+================+======================+=========================+==================================+\n| ``first_name`` | John                 | ``choice``              | (Bob|Larry|Lisa)                 |\n+----------------+----------------------+-------------------------+----------------------------------+\n| ``title``      | Dr.                  | ``clear``               |                                  |\n+----------------+----------------------+-------------------------+----------------------------------+\n| ``street``     | Irving St            | ``faker.street_name``   | Miller Station                   |\n+----------------+----------------------+-------------------------+----------------------------------+\n| ``password``   | dsf82hFxcM           | ``mask``                | XXXXXXXXXX                       |\n+----------------+----------------------+-------------------------+----------------------------------+\n| ``credit_card``| 1234-567-890         | ``partial_mask``        | 1??????????0                     |\n+----------------+----------------------+-------------------------+----------------------------------+\n| ``email``      | jane.doe@example.com | ``md5``                 | 0cba00ca3da1b283a57287bcceb17e35 |\n+----------------+----------------------+-------------------------+----------------------------------+\n| ``email``      | jane.doe@example.com | ``faker.unique.email``  | alex7@sample.com                 |\n+----------------+----------------------+-------------------------+----------------------------------+\n| ``phone_num``  | 65923473             | ``md5`` as_number: True | 3948293448                       |\n+----------------+----------------------+-------------------------+----------------------------------+\n| ``ip``         | 157.50.1.20          | ``set``                 | 127.0.0.1                        |\n+----------------+----------------------+-------------------------+----------------------------------+\n| ``uuid_col``   | 00010203-0405-...... | ``uuid4``               | f7c1bd87-4d....                  |\n+----------------+----------------------+-------------------------+----------------------------------+\n\n* Note: ``faker.unique.[provider]`` only supported on Python 3.6+ (Faker library min. supported python version)\n* Note: ``uuid4`` - only for (native `uuid4`_) columns\n\nSee the `documentation`_ for a more detailed description of the provided anonymization methods.\n\nInstallation\n------------\n\nThe default installation method is to use ``pip``:\n\n.. code-block::\n\n    $ pip install pganonymize\n\nUsage\n-----\n\n.. code-block::\n\n    usage: pganonymize [-h] [-v] [-l] [--schema SCHEMA] [--dbname DBNAME]\n                   [--user USER] [--password PASSWORD] [--host HOST]\n                   [--port PORT] [--dry-run] [--dump-file DUMP_FILE]\n\n    Anonymize data of a PostgreSQL database\n\n    optional arguments:\n    -h, --help            show this help message and exit\n    -v, --verbose         Increase verbosity\n    -l, --list-providers  Show a list of all available providers\n    --schema SCHEMA       A YAML schema file that contains the anonymization\n                            rules\n    --dbname DBNAME       Name of the database\n    --user USER           Name of the database user\n    --password PASSWORD   Password for the database user\n    --host HOST           Database hostname\n    --port PORT           Port of the database\n    --dry-run             Don't commit changes made on the database\n    --dump-file DUMP_FILE\n                          Create a database dump file with the given name\n    --dump-options DUMP_OPTIONS\n                          Options to pass to the pg_dump command\n    --init-sql INIT_SQL   SQL to run before starting anonymization\n    --parallel            Data anonymization is done in parallel\n\nDespite the database connection values, you will have to define a YAML schema file, that includes\nall anonymization rules for that database. Take a look at the `schema documentation`_ or the\n`YAML sample schema`_.\n\n\nExample calls:\n\n.. code-block::\n\n    $ pganonymize --schema=myschema.yml \\\n        --dbname=test_database \\\n        --user=username \\\n        --password=mysecret \\\n        --host=db.host.example.com \\\n        -v\n\n    $ pganonymize --schema=myschema.yml \\\n        --dbname=test_database \\\n        --user=username \\\n        --password=mysecret \\\n        --host=db.host.example.com \\\n        --init-sql \"set search_path to non_public_search_path; set work_mem to '1GB';\" \\\n        -v\n\nDatabase dump\n~~~~~~~~~~~~~\n\nWith the ``--dump-file`` argument it is possible to create a dump file after anonymizing the database. Please note,\nthat the ``pg_dump`` command from the ``postgresql-client-common`` library is necessary to create the dump file for the\ndatabase, e.g. under Linux:\n\n.. code-block::\n\n    $ sudo apt-get install postgresql-client-common\n\nExample call:\n\n.. code-block::\n\n    $ pganonymize --schema=myschema.yml \\\n        --dbname=test_database \\\n        --user=username \\\n        --password=mysecret \\\n        --host=db.host.example.com \\\n        --dump-file=/tmp/dump.gz \\\n        -v\n\nSo that the password for dumping does not have to be entered manually, it can also be entered as an environment var\n``PGPASSWORD``:\n\n.. code-block::\n\n    $ PGPASSWORD=password pganonymize --schema=myschema.yml \\\n        --dbname=test_database \\\n        --user=username \\\n        --password=mysecret \\\n        --host=db.host.example.com \\\n        --dump-file=/tmp/dump.gz \\\n        -v\n\n.. warning::\n\n    Currently only the ``dump-file`` operation supports environment variables.\n\nDocker\n~~~~~~\n\nIf you want to run the anonymizer within a Docker container you first have to build the image:\n\n.. code-block::\n\n    $ docker build -t pganonymize .\n\nAfter that you can pass a schema file to the container, using Docker volumes, and call the anonymizer:\n\n.. code-block::\n\n    $ docker run \\\n        -v \u003cpath to your schema\u003e:/schema.yml \\\n        -it pganonymize \\\n        /usr/local/bin/pganonymize \\\n        --schema=/schema.yml \\\n        --dbname=\u003cdatabase\u003e \\\n        --user=\u003cuser\u003e \\\n        --password=\u003cpassword\u003e \\\n        --host=\u003chost\u003e \\\n        -v\n\n\n.. _uuid4: https://www.postgresql.org/docs/current/datatype-uuid.html\n.. _documentation: https://pganonymize.readthedocs.io/en/latest/\n.. _schema documentation: https://pganonymize.readthedocs.io/en/latest/schema.html\n.. _YAML sample schema: https://github.com/rheinwerk-verlag/pganonymize/blob/master/sample_schema.yml\n\n.. |python| image:: https://img.shields.io/pypi/pyversions/pganonymize\n    :alt: PyPI - Python Version\n\n.. |license| image:: https://img.shields.io/badge/license-MIT-green.svg\n    :target: https://github.com/rheinwerk-verlag/pganonymize/blob/master/LICENSE.rst\n\n.. |pypi| image:: https://badge.fury.io/py/pganonymize.svg\n    :target: https://badge.fury.io/py/pganonymize\n\n.. |downloads| image:: https://static.pepy.tech/personalized-badge/pganonymize?period=total\u0026units=international_system\u0026left_color=blue\u0026right_color=black\u0026left_text=Downloads\n    :target: https://pepy.tech/project/pganonymize\n    :alt: Download count\n\n.. |build| image:: https://github.com/rheinwerk-verlag/pganonymize/actions/workflows/test.yml/badge.svg\n    :target: https://github.com/rheinwerk-verlag/pganonymize/actions\n\n.. |codecov| image:: https://codecov.io/gh/rheinwerk-verlag/pganonymize/branch/main/graph/badge.svg\n    :target: https://codecov.io/gh/rheinwerk-verlag/pganonymize\n\n.. |health| image:: https://snyk.io/advisor/python/pganonymize/badge.svg\n    :target: https://snyk.io/advisor/python/pganonymize\n    :alt: pganonymize\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frheinwerk-verlag%2Fpganonymize","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frheinwerk-verlag%2Fpganonymize","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frheinwerk-verlag%2Fpganonymize/lists"}