{"id":16152923,"url":"https://github.com/deamoner/privyfilter","last_synced_at":"2025-07-26T04:32:39.223Z","repository":{"id":57454872,"uuid":"269399705","full_name":"Deamoner/privyfilter","owner":"Deamoner","description":"Privy filter is made to give the method for removing/accommodating for unethical bias in data.","archived":false,"fork":false,"pushed_at":"2020-07-07T14:22:26.000Z","size":74449,"stargazers_count":8,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-07-19T08:22:48.779Z","etag":null,"topics":["bias","deidentify","direct-identifiers","machine-learning","photo-privacy","privacy","racism"],"latest_commit_sha":null,"homepage":"https://privyfilter.herokuapp.com/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Deamoner.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-06-04T15:43:24.000Z","updated_at":"2025-04-19T18:18:20.000Z","dependencies_parsed_at":"2022-09-05T07:01:35.831Z","dependency_job_id":null,"html_url":"https://github.com/Deamoner/privyfilter","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Deamoner/privyfilter","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Deamoner%2Fprivyfilter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Deamoner%2Fprivyfilter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Deamoner%2Fprivyfilter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Deamoner%2Fprivyfilter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Deamoner","download_url":"https://codeload.github.com/Deamoner/privyfilter/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Deamoner%2Fprivyfilter/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267117278,"owners_count":24038641,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-26T02:00:08.937Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bias","deidentify","direct-identifiers","machine-learning","photo-privacy","privacy","racism"],"created_at":"2024-10-10T01:09:32.730Z","updated_at":"2025-07-26T04:32:39.175Z","avatar_url":"https://github.com/Deamoner.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Photo Filter for racial/ethical bias\n### Filtering photos for bias(racial, gender) for machine learning.\n### Privy filter is your photo dataset augmentor and analyzer for unethical bias(Race, Gender, Age)\n## Built with :heart: by [Matthew Davis](https://www.linkedin.com/in/tech-lead-matt-davis/) - [linkedin](https://www.linkedin.com/in/tech-lead-matt-davis/) - [github](https://github.com/Deamoner) - [Medium](https://medium.com/@mdavis_71283) - [Youtube](https://www.youtube.com/channel/UCJNZxBqs8ElqouPqAkZLlqg) - [Facebook](https://www.facebook.com/matthewjamesdavis/) - [Privy Filter Demo](https://privyfilter.herokuapp.com/)\nFiltering photos for bias(racial, gender) for machine learning.\n\n### Demo\n\nPrivy Filter: [Info and Demo Site](https://privyfilter.herokuapp.com/)\n\n### Use Cases\n- Train Machine Learning Models minimizing unethical bias inherent in underlying photo dataset - IE Race, Gender, Age Discrimination\n- Data Augmentation of Photo datasets to balance unethical data bias\n- Create and Identify Unit Test Photos for datasets to ensure continual testing of bias in classification type problems\n- Ethical Machine Learning - Remove Identity/Generalize Identinity/Generalize Race/Gender/Skincolor/Generalize\n\nPrivy filter is your photo dataset augmentor and analyzer for unethical bias(Race, Gender, Age)\n\n![privyfilter detect and blurr faces](https://github.com/Deamoner/privyfilter/raw/master/TestPics/privyfilter.jpg)\n![privyfilter detect and blurr faces](https://github.com/Deamoner/privyfilter/raw/master/Results/pipeline.png)\n\n\n\n### Latest Release:\nv0.2 - Sythetic Face Generation and Face Swapping\n\n### Roadmap\n\n#### v0.1 - Basic Direct Identifier Scrubbing - Done\n\n- Face Detect - Needs improvement\n- Face Scrub\n\n#### v0.1.6 - Basic Meta Data Scrubber - Done\n\n- RemoveMetaData(imgPath)\n\n#### v0.2.0 - Identify face and replace - Done\n\n- getRandomFakeFace : get Random Synthetic Face - done - needs tmp folder setting\n- peopleObject - getFaceInfo - done - face model not loading on\n- peopleObject - swapFace(img1, img2)\n\n#### v0.3.0 - ProcessAll and Use Case Direct Finishing\n\n- processOne() - review demographics of current photo - suggest additional photos\n- generatePhoto - from one photo and demographic info - generate those new photo demographic utilizing synthetic faces\n- processDir() - process all the photos in the directory - first calculate the statistics and distributions - and parameterize a directory to save all sythetic balanced photos.\n- Update DemoSite with Use Case direct notes and usability examples\n\n#### v0.4.0 - Identify other identifiers and bias features\n\n- Detect skin - Dataset: http://cs-chan.com/downloads_skin_dataset.html\n- AdjustSkin - Can adjust skincolor to generate new dataset photo\n- Update processone stats and generatephoto to utilize skincolor data and manipulation\n- Update demosite with video\n- Demo of CelebA Dataset\n\n#### v0.5 - Direct Aging and Gender\n\n- Test Face Aging Algo\n- Test Gender Changing Algo\n- Testing Race Direct Synthetic Style Transfer\n- Fixes for DetectSkin and AdjustSkin\n\n#### v0.6 - Improve/Increase Original Photoset data quality retention through improve synthetic matching\n\n- Face pose matching for synthetic face replacement\n- Review accuracy of underlying demographic model\n- New CelebA Demo\n\n#### v0.7 - Increase compatibility of pipeline for photosets\n\n- Support for multi face photo - Scoping - Document API updates\n- Support for people but no face in photos - recognize poses without faces\n\n#### v0.8 - Reliability Fixes\n\n- Back to baseline of usecase documentation\n- Use Case Automated Testing\n-\n\n### Methodology\n\nScrub all identifying and data possible for bias from the dataset keeping a featureset that still allows for useful machine learning.\n\n1. Identify Features - Faces, People\n2. Remove Unethical Bias Data - New Synthetic Faces, Adjust Skin, Gender and Age Transformation\n3. Create alternatives synthetic photos for balanced training\n\n## References Work\n\n- https://towardsdatascience.com/survey-d4f168791e57\n- https://brie.berkeley.edu/sites/default/files/brie_wp_2018-3.pdf\n- https://www.brookings.edu/research/algorithmic-bias-detection-and-mitigation-best-practices-and-policies-to-reduce-consumer-harms/\n- https://healthitanalytics.com/news/eliminating-racial-bias-in-algorithm-development\n\n### Process for Features\n\nWhat needs to be done for roadmap Contribution:\n1. Scoping\n2. Test Implementation for AI Model pieces - Colab Notebook of Models with Validation Data\n3. Library wrapping and Implementation\n4. Library Publish\n\n\nTasks:\n- [x] Face Detect - Colab Testing - Included in the Notebook folder in the repo.\n- [X] Facescrub - Colab Testing - Simplest De-identification - Simple blur - Commonly available\n- [X] Face Location Extraction\n- [X] Basic Library Object Implementation\n- [X] v0.1 Release for basic Direct Identifiers\n- [X] Pose Extraction Notebook\n- [X] Basic Skin Extraction\n- [X] Synthetic Face Generation\n- [X] Face Swap Testing\n- [X] Face Swap Integration into main library\n- [X] Fixes for model automated download\n- [X] Clean up API for people and privyfilter library to match specs.\n- [X] Replace Faces - Replace a face in a photo\n- [ ] Scope out API for calculation and methodology of race/gender face demographic balancing\n- [ ] Functions to process one photo for demographics and balanced demographics needed return\n- [ ] Function generateSytheticPhoto - create the photo required\n- [ ] Improved SkinExtraction Protocol\n- [ ] Skin Adjustment Methodology\n- [ ] Fixes for face detection to\n- [ ] v0.2 Release for George Floyd Implementation\n\n\n## Installation\n\nUse the package manager [pip](https://pip.pypa.io/en/stable/) to install Privy - Privacy Photo Filter library.\n\n```bash\npip install privyfilter\n```\n\n## Basic Usage\n\n```python\nfrom privyfilter.privyfilter import Privyfilter as pf\n\nfaces2, img = pf.faceScrub(\"../TestPics/1Person-Close.jpg\")\n\n```\n\n## Scrub Meta Data\n```python\nfrom privyfilter.privyfilter import Privyfilter as pf\nimg = pf.RemoveMetaData(\"../TestPics/1Person-Close.jpg\")\n```\n\n## Face Swap - Inputs imgPath\n```python\nswapimg = pf.swapFaces(\"../TestPics/1Person-Close.jpg\", \"../TestPics/2Person-Close.jpg\")\ncv2_imshow(swapimg)\n```\n\n## EndGoal API:\n   - privyfilter.processDir(srcPath, dstPath) - analyze all photos for demographics in srcdirect, if dstdirect - generate the balanced dataset)\n   - privyfilter.processOne(srcPath, dstPath) - analyze photo and suggest demographics of balanced photo dataset - if dstpath then generate\n   - privyfilter.generatePhoto(srcPhoto, demographic, dstPath) - generate a photo from the original with the demographic params specified  \n   - privyfilter.findFaces(imgpath, demographics=false) or privyfilter.findFaces(cv2 object)\n    - returns array of faces and can include demographics.\n   - privyfilter.scrubFaces(imgPath) or privyfilter.scrubFaces(cv2 object)\n    - returns faces, and scrubbed img\n   - privyfilter.swapFaces(img1Path, img2Path) or privyfilter.swapFaces(img1 cv2 object, img2 cv2 Object)\n    - replaces all faces in 2nd image with first. Returns new image as cv2 Object.\n   - privyfilter.getFakeFace()\n    - returns fakeface image object\n   - privyfilter.replaceFaces(img1)\n    - returns img of all faces randomly replaced with fake ones.\n\n## Deploying Module\n\n```\npython setup.py bdist_wheel --universal\ntwine upload dist/*\n```\n\n## Contributing\nPull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.\n\nPlease make sure to update the tests as appropriate.\n\n## License\n[MIT](https://choosealicense.com/licenses/mit/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeamoner%2Fprivyfilter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdeamoner%2Fprivyfilter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeamoner%2Fprivyfilter/lists"}