{"id":21743262,"url":"https://github.com/afoley587/deep-face-trafficking","last_synced_at":"2025-04-07T00:18:33.328Z","repository":{"id":246801669,"uuid":"716708563","full_name":"afoley587/deep-face-trafficking","owner":"afoley587","description":null,"archived":false,"fork":false,"pushed_at":"2024-08-22T13:18:42.000Z","size":14339,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-13T05:48:33.895Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/afoley587.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-09T17:48:14.000Z","updated_at":"2024-11-06T05:22:48.000Z","dependencies_parsed_at":null,"dependency_job_id":"ea328d80-57ee-4daa-8d13-1442f0b34b48","html_url":"https://github.com/afoley587/deep-face-trafficking","commit_stats":null,"previous_names":["afoley587/deep-face-trafficking"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/afoley587%2Fdeep-face-trafficking","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/afoley587%2Fdeep-face-trafficking/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/afoley587%2Fdeep-face-trafficking/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/afoley587%2Fdeep-face-trafficking/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/afoley587","download_url":"https://codeload.github.com/afoley587/deep-face-trafficking/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247569209,"owners_count":20959778,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-26T07:03:42.977Z","updated_at":"2025-04-07T00:18:33.307Z","avatar_url":"https://github.com/afoley587.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Can FOSS Computer Vision Combat Human Trafficking - Part I\n\nI am an engineer and I am passionate about a great many things:\n\n- The ocean\n- Good food\n- Camping / Nature\n\nBut, I also have a strong belief that we should try to make the world safer \nfor everyone. Unfortunately, I don't feel as though that's the case right\nnow. Everywhere I turn, there's another case of something heinous happening\nso, as an engineer and computer scientist, what can I do? In this post, \nI'd like to see if I can develop something that can detect possible human\ntrafficking threats using FOSS computer vision libraries. In this quest for justice, \nwe will leverage Python and the formidable combination of DeepFace and OpenCV to\nperform these analyses. We will then form some initial business logic about\npossible threats - what are their demographics/ages/etc.?\n\nThis intersection of artificial intelligence, computer vision, and human \ntrafficking may seem like an unlikely partnership at first glance, \nbut I believe the potential impact is profound. By leveraging cutting-edge technology, \nwe can not only identify potential victims but also search missing persons for matching\ncriteria, alert authorities, and geo-tag images in real time to hopefully bring\nus one step closer to eradicating this heinous crime. \n\nIn this blog post, we will explore how open-source computer vision libraries, \nDeepFace and OpenCV, can be used to detect subtle cues such as facial expressions \nand ages that may signify high probability victims. We will show preliminary results\nand in subsequent posts, I will expand upon these to add new features such as\nLangChain integrations for missing persons queries and GeoJson for geo-tagging.\n\nBut, let's get started!\n\n## System Overview\nI want to first give an overview of my envisioned system as a whole, which\nis subject to change as I develop this software. But anyway, let's\ntake a look at the below:\n\n![System Design](./images/high-level-design.png)\n\nIn the system design, we can see that we will use a camera as our IoT device. It\nwill stream frames from itself to our application. Our application will then send\nthe frames to the facial analysis module which will:\n\n- Find any faces in the frame\n- Detect their age, gender, emotion, and race\n- Annotate the frame\n- Return the frame and the results to the application\n\nThe application will then do a set of criteria checks on the results to see if this\nis a possible human trafficking case. Our initial attempt at business logic here will\nbe:\n\n- Children in danger: Are there any faces under 18 years old which are displaying\n    fear or disgust?\n- Woman in danger: Are there any faces which belong to women which are displaying\n    fear or disgust?\n\nThese are simple criterias, which will produce false postivies/negatives but it is\na start!\n\nIf we find any matching criteria, we will eventually forward them along to a\ngenerative AI module (think LangChain) which will perform a missing persons \ncheck. If there is a matching missing persons report, we will geo-tag the image\nand send our results to an operator. To be clear, today we will just be looking\nat a subset of the whole system:\n\n- The facial analysis module\n- The beginning portions of the application such as basic criteria checks, reading\n    from streams, and passing images onto the facial analysis module\n\nLet us look at the main libraries we will use to build these modules.\n\n## Main Libraries - DeepFace and OpenCV\n\n[DeepFace](https://pypi.org/project/deepface/) is an incredibly lightweight facial \nrecognition and analysis library that is openly available to python users. \nIt comes prebundled with some of the most popular facial detection libraries such as:\n\n- Retina Face\n- VGG-Face\n- Google Facenet\n- and more.\n\nA full list of detectors and models can be found in their [documentation](https://pypi.org/project/deepface/).\nIt wraps each model with a standard interface which means we can integrate\nthis into our code seamlessly and, if desired, change models without any\neffort.\n\nWith DeepFace, we get access to some amazing features:\n\n1. Face Verification - Comparing two faces to eachother and returning a probability\n                        of how similar they are. If the similarity is high enough,\n                        the model confidently reports that they are the same face.\n2. Face Recognition  - Recognizing a face from a known database of faces\n3. Face analysis     - Analyzing a facial image for gender, age, emotion, race,\n                       and other properties which can be derived from the face.\n\n[OpenCV](https://opencv.org/) is the open computer vision library. It's an open source\ncomputer vision and machine learning library. It comes prebundled with a slew of\nanalysis tools such as edge detectors, contour detectors, haar cascaders, and so \nmuch more.\n\n## Running\n\nTo start the python services and RabbitMQ:\n\n```shell\nprompt\u003e docker-compose up -d\n```\n\nThe golang streamer isn't in Docker because I need it to access my webcam and haven't\ngotten that working in docker just yet. You can just run \n```shell\nprompt\u003e go run main.go\nprompt\u003e curl -X POST http://localhost:8080/register/webcam\n```\n\nThe logs will indicate when it sends 100 frames to RabbitMQ\n\n```shell\n[GIN] 2024/06/30 - 06:22:28 | 200 |      76.333µs |             ::1 | POST     \"/register/webcam\"\n2024/06/30 06:22:33 File /tmp/streams_0.avi finished\n2024/06/30 06:22:33 Sending to Rabbit\n2024/06/30 06:22:33  [x] Sent /tmp/streams_0.avi\nopening new...\n2024/06/30 06:22:37 File /tmp/streams_1.avi finished\n2024/06/30 06:22:37 Sending to Rabbit\n2024/06/30 06:22:37  [x] Sent /tmp/streams_1.avi\nopening new...\nopening new...\n2024/06/30 06:22:40 File /tmp/streams_2.avi finished\n2024/06/30 06:22:40 Sending to Rabbit\n2024/06/30 06:22:40  [x] Sent /tmp/streams_2.avi\nopening new...\n2024/06/30 06:22:43 File /tmp/streams_3.avi finished\n2024/06/30 06:22:43 Sending to Rabbit\n2024/06/30 06:22:43  [x] Sent /tmp/streams_3.avi\n```\n\nAnd it'll start streaming from your webcam\n## Preliminary Results - Static Image Analysis\nThere is a lot of code, so I think it's best to show the preliminary\nresults as opposed to the whole code base. If you would like to access\nthe code, please reach out to me or respond to this article and I will \nhappily give you access.\n\nI tested with three images that showed people in some sort of fear. Let's take \na look at the results below:\n\n```shell\n2023-11-09 09:42:12.175 | INFO     | utils:analyze_image:81 - Found Possible Criteria Match - is_possible_trafficking\n2023-11-09 09:42:12.176 | INFO     | utils:analyze_image:99 - saving to ./trafficdetection/test-images/test3.processed.jpg\n2023-11-09 09:42:18.822 | INFO     | utils:analyze_image:99 - saving to ./trafficdetection/test-images/test2.processed.jpg\n2023-11-09 09:42:25.056 | INFO     | utils:analyze_image:81 - Found Possible Criteria Match - is_possible_trafficking\n2023-11-09 09:42:25.056 | INFO     | utils:analyze_image:99 - saving to ./trafficdetection/test-images/test.processed.jpg\n```\n\n![Test Image 1](./images/test.processed.jpg)\n![Test Image 2](./images/test2.processed.jpg)\n![Test Image 3](./images/test3.processed.jpg)\n\nOkay, so the results aren't too bad. We see that, if we have a woman\nand she is showcasing fear, we mark the image as `is_possible_trafficking`.\n\nIn the perfect world, if this mark gets added to an image, that's when we\nwould alert our generative AI module to start looking for missing persons\nwho may or may not fit the description (age ranges, genders, and races).\n\n## Preliminary Results - Live Video Stream\nLet's do the same thing with a video stream. I will first do one with \nmy webcam, processing frames of myself showcasing various facial expressions:\n\n![Test Webcam](./images/webcam.gif)\n\nWe can see that my expressions are captured in real time from my webcam. There\nis a slight delay as we buffer frames and then process them on a separate\nthread so that we get a smooth output video, but the result is pretty good.\nBecause I am not a child nor a woman, no trafficking alerts are sent out which\nis evident by our logs (above) and the output video.\n\nLet's try it with a video file:\n\n```shell\n2023-11-09 10:35:52.942 | INFO     | utils:analyze_video:66 - Found Possible Criteria Match - is_possible_trafficking\n2023-11-09 10:35:55.008 | INFO     | utils:analyze_video:66 - Found Possible Criteria Match - is_possible_trafficking\n2023-11-09 10:35:57.102 | INFO     | utils:analyze_video:66 - Found Possible Criteria Match - is_possible_trafficking\n2023-11-09 10:35:59.164 | INFO     | utils:analyze_video:66 - Found Possible Criteria Match - is_possible_trafficking\n2023-11-09 10:36:01.231 | INFO     | utils:analyze_video:66 - Found Possible Criteria Match - is_possible_trafficking\n2023-11-09 10:36:03.313 | INFO     | utils:analyze_video:66 - Found Possible Criteria Match - is_possible_trafficking\n2023-11-09 10:36:05.423 | INFO     | utils:analyze_video:66 - Found Possible Criteria Match - is_possible_trafficking\n2023-11-09 10:36:07.511 | INFO     | utils:analyze_video:66 - Found Possible Criteria Match - is_possible_trafficking\n2023-11-09 10:36:20.154 | INFO     | utils:analyze_video:66 - Found Possible Criteria Match - is_possible_trafficking\n2023-11-09 10:36:22.256 | INFO     | utils:analyze_video:66 - Found Possible Criteria Match - is_possible_trafficking\n```\n![Test Video File](./images/evil-dead.gif)\n\n## Preliminary Discussion\nNow, this does seem to work OK up to this point, but I have some concerns:\n\n* A lot of misclassification - The DeepFace analyses definitely work and,\n    for being open source with no tuning, work pretty dang well! I based\n    this project off of `RetinaFace` and I will likely play around with\n    other models. I see the biggest issues with ages and emotions, races\n    and genders seem to be a bit more robust.\n* Processing time - On my computer, the processing time was pretty high.\n    I am using a basic laptop and all computations take place on my CPU.\n    I implemented a buffered video reader so that I wouldn't miss frames,\n    but it would be fun to see what kind of performance we can achieve on\n    a GPU or more proper hardware.\n* Criterias - It's really hard to set a criteria for human trafficking.\n    I would assume it is a certain demographic showing fear, disgust, or\n    sadness. It might be worth implementing some scoring mechanism. An \n    example might be: This is a fearful woman accompanied by an angry male\n    which scores higher than just a fearful woman by herself. But these\n    criteria are extremely subjective and hard to define.\n\nAnyway, I think I have lots more work to do here, but I hope you enjoyed \nmy preliminary results and following along. If you would like to see the\ncode or join up in the project, please respond or reach out to me!\n\n# Follow Up - 11/15/2023\n\n```shell\n2023-11-15 06:13:20.025 | INFO     | utils:analyze_image:108 - [\n  {\n    \"age\": 35,\n    \"gender\": {\n      \"Woman\": 85.8,\n      \"Man\": 14.2\n    },\n    \"dominant_gender\": \"Woman\",\n    \"race\": {\n      \"white\": 94.8\n    },\n    \"dominant_race\": \"white\",\n    \"emotion\": {\n      \"sad\": 99\n    },\n    \"dominant_emotion\": \"sad\"\n  }\n]\n```\n\nIdentification and Verification:\nNext, we classify the individual as a potential victim of distress or trafficking, triggering a lookup in the National Missing and Unidentified Persons System (NAMUS). Redacted details maintain confidentiality, yet the critical information about age, gender, and race is unveiled.\n\n```shell\n2023-11-15 06:13:20.462 | INFO     | agents.namus:search:133 - {\n  \"count\": 500,\n  \"results\": [\n    {\n      \"idFormatted\": \"...\",\n      \"dateOfLastContact\": \"2023-10-24\",\n      \"gender\": \"Female\",\n      \"raceEthnicity\": \"White / Caucasian\",\n      \"currentAgeFrom\": 25,\n      \"currentAgeTo\": 25\n    }\n  ]\n}\n```\n\nVerification and Action:\nUpon finding a match, we download relevant images from NAMUS and conduct a facial comparison using DeepFace. The result? A confirmation that the distressed face captured aligns with a current missing person, bringing us one step closer to a resolution.\n\n```shell\n2023-11-15 06:13:21.927 | INFO     | agents.namus:_run_analysis:39 - Result is: {\"verified\": True, \"distance\": -4.44e-16, \"threshold\": 0.4, \"model\": \"VGG-Face\", \"time\": 0.86}\n```\n\nConclusion:\nIn the hypothetical scenario of a CCTV feed capturing a distressed face matching a missing person, the project culminates with geotagging, image preservation, and the dispatch of pertinent information to authorities. This amalgamation of technology and compassion offers a glimmer of hope in the fight against human trafficking.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fafoley587%2Fdeep-face-trafficking","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fafoley587%2Fdeep-face-trafficking","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fafoley587%2Fdeep-face-trafficking/lists"}