{"id":26054719,"url":"https://github.com/ajschofield/gdpr-obfuscator","last_synced_at":"2026-05-08T00:32:19.055Z","repository":{"id":279649859,"uuid":"926769156","full_name":"ajschofield/gdpr-obfuscator","owner":"ajschofield","description":"A Python library designed to detect and remove Personally Identifiable Information (PII) from CSV files.","archived":false,"fork":false,"pushed_at":"2025-03-03T01:07:59.000Z","size":681,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-20T14:40:41.958Z","etag":null,"topics":["aws","csv","json","localstack","pytest","python","s3"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ajschofield.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-02-03T20:32:30.000Z","updated_at":"2025-03-04T12:05:33.000Z","dependencies_parsed_at":null,"dependency_job_id":"c124f8bf-cada-4633-93ee-4923b329b17c","html_url":"https://github.com/ajschofield/gdpr-obfuscator","commit_stats":null,"previous_names":["ajschofield/gdpr-obfuscator"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/ajschofield/gdpr-obfuscator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajschofield%2Fgdpr-obfuscator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajschofield%2Fgdpr-obfuscator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajschofield%2Fgdpr-obfuscator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajschofield%2Fgdpr-obfuscator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ajschofield","download_url":"https://codeload.github.com/ajschofield/gdpr-obfuscator/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajschofield%2Fgdpr-obfuscator/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32762002,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-07T02:14:30.463Z","status":"ssl_error","status_checked_at":"2026-05-07T02:14:29.405Z","response_time":62,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","csv","json","localstack","pytest","python","s3"],"created_at":"2025-03-08T09:13:18.228Z","updated_at":"2026-05-08T00:32:19.050Z","avatar_url":"https://github.com/ajschofield.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# GDPR Obfuscator - Launchpad Project\n\n## Overview\n\nA Python library designed to detect and remove Personally Identifiable Information (PII) from CSV files stored in an AWS S3 bucket.\n\n## Minimum Viable Product (MVP)\n\nThe MVP covers:\n1. Reading a JSON string containing the S3 location of the CSV file and the names of the fields that are required to be obfuscated\n2. Ingesting the CSV file containing data records (with a primary key) from an AWS S3 bucket\n3. Obfuscating chosen PII fields (e.g. `name`, `email_address`) by replacing their values with an obfuscated string (`***`)\n4. Returning the obfuscated data as a byte-stream that maintains the original structure but with sensitive fields changed\n\nThis meets the requirements under the General Data Protection Regulation [(GDPR)](https://ico.org.uk/media/for-organisations/guide-to-data-protection/guide-to-the-general-data-protection-regulation-gdpr-1-1.pdf) to ensure that all data containing information that can be used to identify an individual should be anonymised.\n\n## Setup\n\n### Prerequisites\n\n- Python \u003e= 3.13\n- Poetry \u003e= 2.0.1\n\n### Installation\n\nThere are two ways to install the package:\n\n#### Source\n\n```bash\ngit clone https://github.com/ajschofield/gdpr-obfuscator.git\ncd gdpr-obfuscator\npoetry install\n```\n\n#### Prebuilt Package\n\nDownload the latest release from [here](https://github.com/ajschofield/gdpr-obfuscator/releases/latest) and install using `pip`:\n\n```bash\n# Package name may be different to what is below\npip install gdpr_obfuscator-0.1.0-py3-none-any.whl\n```\n\n## Usage\n\nThe `Obfuscator` class can be imported directly into your Python code. Once instiantiated, you may call either the `process_s3` or `process_local` method. Each method takes a JSON string as the input, which must contain `file_path` and `pii_fields`.\n\n```json\n{\n    \"file_path\": \"s3://bucket-name/file-name.csv\",\n    \"pii_fields\": [\"name\", \"email_address\"]\n}\n```\n\nBoth methods return a byte-stream containing the obfuscated data which can be used with the put_object method in the boto3 library to upload the data reliably back to S3.\n\n```python\nfrom gdpr_obfuscator import Obfuscator\nimport json\n\ninput = json.dumps({\n    \"file_path\": \"s3://bucket-name/file-name.csv\",\n    \"pii_fields\": [\"name\", \"email_address\"]\n})\n\nobfuscator = Obfuscator()\nresult = obfuscator.process_s3(input)\n\nprint(result.decode(\"utf-8\"))\n```\n\nAlternatively, there is a command line interface available to use the package from the terminal. The CLI is not packaged with the library, so you will have to follow the steps in the [source installation](#source) section to use it.\n\n```bash\n❯❯ poetry run python cli.py --help\nusage: GDPR-Obfuscator [-h] (-l LOCAL | -s S3) -p PII [PII ...]\n\nObfuscate sensitive data stored locally or in an AWS environment\n\noptions:\n  -h, --help            show this help message and exit\n  -l, --local LOCAL     Local path to file\n  -s, --s3 S3           URI path to file stored in S3\n  -p, --pii PII [PII ...]\n                        List of PII fields to obfuscate, separated by spaces\n```\n\n```bash\n❯❯ poetry run python cli.py -l test/data/mock_data.csv -p name\nstudent_id,name,course,cohort,graduation_date,email_address\n1,***,UX/UI Design Bootcamp,2/29/2024,7/11/2024,jleger0@facebook.com\n2,***,Digital Marketing Bootcamp,2/24/2024,9/6/2024,cadrian1@gizmodo.com\n3,***,UX/UI Design Bootcamp,3/13/2024,10/24/2024,whugnin2@archive.org\n4,***,Artificial Intelligence Bootcamp,2/24/2024,9/14/2024,aspight3@4shared.com\n5,***,Artificial Intelligence Bootcamp,1/31/2024,9/4/2024,dcowpland4@dot.gov\n6,***,Digital Marketing Bootcamp,2/8/2024,,gkliement5@auda.org.au\n7,***,Internet of Things Bootcamp,2/21/2024,7/16/2024,\n8,***,Mobile App Development Bootcamp,2/17/2024,7/15/2024,smyrkus7@i2i.jp\n9,***,Game Development Bootcamp,3/1/2024,9/7/2024,nryal8@symantec.com\n...\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fajschofield%2Fgdpr-obfuscator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fajschofield%2Fgdpr-obfuscator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fajschofield%2Fgdpr-obfuscator/lists"}