{"id":19856626,"url":"https://github.com/fgmacedo/datanonymizer","last_synced_at":"2025-05-02T01:31:40.683Z","repository":{"id":62566697,"uuid":"367097475","full_name":"fgmacedo/datanonymizer","owner":"fgmacedo","description":"Anonymizer tool for datasets such CSV files","archived":false,"fork":false,"pushed_at":"2024-10-07T06:31:02.000Z","size":22,"stargazers_count":9,"open_issues_count":2,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-23T00:37:40.468Z","etag":null,"topics":["data-anonymity","data-anonymization","data-anonymized","fake-data","fake-data-generator"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fgmacedo.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-05-13T15:47:08.000Z","updated_at":"2025-01-15T22:04:39.000Z","dependencies_parsed_at":"2022-11-03T16:15:56.279Z","dependency_job_id":null,"html_url":"https://github.com/fgmacedo/datanonymizer","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fgmacedo%2Fdatanonymizer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fgmacedo%2Fdatanonymizer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fgmacedo%2Fdatanonymizer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fgmacedo%2Fdatanonymizer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fgmacedo","download_url":"https://codeload.github.com/fgmacedo/datanonymizer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251969433,"owners_count":21673203,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-anonymity","data-anonymization","data-anonymized","fake-data","fake-data-generator"],"created_at":"2024-11-12T14:16:08.376Z","updated_at":"2025-05-02T01:31:40.413Z","avatar_url":"https://github.com/fgmacedo.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"=============\ndatanonymizer\n=============\n\nAnonymizer tool for datasets such CSV files.\n\nTo generate fake data, you can choose between two excelent generators:\n\n-  `Faker \u003chttps://faker.readthedocs.io/en/stable/index.html\u003e`_  (default).\n-  `mimesis \u003chttps://mimesis.name/index.html\u003e`_ via optional install.\n\n\nInstall\n=======\n\nUsing pip:\n\n.. code-block:: bash\n\n  pip install datanonymizer\n\n\nUsing mimesis instead of the default Faker:\n\n.. code-block:: bash\n\n  pip install datanonymizer[mimesis]\n\n\nOr from source:\n\n.. code-block:: bash\n\n  git clone https://github.com/fgmacedo/datanonymizer\n  cd datanonymizer\n  python setup.py install\n\n\nUsage\n=====\n\nPass your data through ``stdin`` and get it back anonymized on ``stdout``.\n\n.. note::\n\n    In this case, the output will be equal to the input as no conversions were applied.\n\n\n.. code-block:: bash\n\n  cat input_file.csv | datanonymizer \u003eoutput_file.csv\n\nUsing a config file to declare conversions and generators for the required fields:\n\n\n.. code-block:: bash\n\n  cat input_file.csv | datanonymizer --config ./dataset_anon_config.yml \u003eoutput_file.csv\n\n\nPlease see examples folder for a small demo:\n\n.. code-block:: bash\n\n  cat examples/small.csv | python -m datanonymizer -i --config examples/small_faker.yml --seed my_seed \u003eexamples/small_anonymized_using_faker.csv\n\n\n\nOptional arguments:\n\n.. code-block::\n\n  -h, --help            show this help message and exit\n  -l LANGUAGE, --language LANGUAGE\n                        Language used by the Generator\n  -di DELIMITER_INPUT, --delimiter_input DELIMITER_INPUT\n                        CSV delimiter\n  -do DELIMITER_OUTPUT, --delimiter_output DELIMITER_OUTPUT\n                        CSV delimiter\n  -i, --ignore_errors   Continue on errors\n  --head HEAD           Outputs only the first \u003cHEAD\u003e lines\n  -g {faker,mimesis}, --generator {faker,mimesis}\n                        Generator library to be used for fake data\n  --seed SEED           Seed for the pseudo random generator providers\n  --config CONFIG       Configuration file\n\n\nConfig file\n===========\n\nYou'l need a configuration file to setup transformations for each dataset.\n\nThis file is a simple `yaml \u003chttps://yaml.org/\u003e`_  where you can configure fields.\n\nField names should match the column name declared into the CSV input file.\n\n.. code-block:: yaml\n\n  ---\n  fields:\n    Task ID:\n      omit: true\n    Location:\n      conversions:\n        - fn: coords_to_h3\n          kwargs:\n            resolution: 8\n    Client Address:\n      conversions:\n        - fn: has_value\n      rename: has_address\n    Company Name:\n      generator:\n        provider: business.company\n      rename: company\n    Invoice ID:\n      generator:\n        provider: person.identifier\n        kwargs:\n          mask: \"#######\"\n      rename: invoice\n\n\nGenerators\n----------\n\nThe generatos clause depends of the library you choose to provide fake data.\n\n\nYou can use any generator available at the generic API from Faker or mimesis.\n\n\nFor example, if you wanna mimic data with company names:\n\n- Faker\n\n  .. code-block:: yaml\n\n    ---\n    fields:\n      Company Name:\n        generator:\n          provider: company\n\n- Mimesis\n\n  .. code-block:: yaml\n\n    ---\n    fields:\n      Company Name:\n        generator:\n          provider: business.company\n\nBut you can replace the real names by names of fruits (using Mimesis) or any other provider:\n\n.. code-block:: yaml\n\n  ---\n  fields:\n    Company Name:\n      generator:\n        provider: food.fruit\n\n\nOr generate random integers to replace real IDs:\n\n- Faker\n\n  .. code-block:: yaml\n\n    ---\n    fields:\n      ID:\n        generator:\n          provider: pyint\n          kwargs:\n            min_value: 1\n            max_value: 15_000_000\n\n- Mimesis\n\n  .. code-block:: yaml\n\n    ---\n    fields:\n      ID:\n        generator:\n          provider: person.identifier\n          kwargs:\n            mask: \"#######\"\n\n\nConversions\n-----------\n\nYou can apply any pre-configured conversion functions available.\n\n\n- coords_to_h3\n- has_value\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffgmacedo%2Fdatanonymizer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffgmacedo%2Fdatanonymizer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffgmacedo%2Fdatanonymizer/lists"}