{"id":13679584,"url":"https://github.com/RuthGnz/SpyScrap","last_synced_at":"2025-04-29T19:31:38.249Z","repository":{"id":39010254,"uuid":"194836916","full_name":"RuthGnz/SpyScrap","owner":"RuthGnz","description":"CLI and GUI for OSINT.  Are you very exhibited on the Internet? Check it!  Twitter, Tinder, Facebook, Google, Yandex, BOE. It uses facial recognition to provide more accurate results.","archived":false,"fork":false,"pushed_at":"2023-11-26T11:59:06.000Z","size":16301,"stargazers_count":175,"open_issues_count":29,"forks_count":27,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-03-12T07:24:14.538Z","etag":null,"topics":["boe","cli","docker","facebook","facial-recognition","google","osint","reverse-image-search","scraping-websites","social-networks","socmint","tinder","twitter"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RuthGnz.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-07-02T09:53:47.000Z","updated_at":"2025-02-16T19:40:30.000Z","dependencies_parsed_at":"2024-01-22T05:49:47.904Z","dependency_job_id":null,"html_url":"https://github.com/RuthGnz/SpyScrap","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RuthGnz%2FSpyScrap","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RuthGnz%2FSpyScrap/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RuthGnz%2FSpyScrap/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RuthGnz%2FSpyScrap/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RuthGnz","download_url":"https://codeload.github.com/RuthGnz/SpyScrap/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251569549,"owners_count":21610575,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["boe","cli","docker","facebook","facial-recognition","google","osint","reverse-image-search","scraping-websites","social-networks","socmint","tinder","twitter"],"created_at":"2024-08-02T13:01:07.178Z","updated_at":"2025-04-29T19:31:33.238Z","avatar_url":"https://github.com/RuthGnz.png","language":"Python","funding_links":[],"categories":["Python","Pentesting"],"sub_categories":["OSINT - Open Source INTelligence"],"readme":"![Last update](https://img.shields.io/badge/last%20update-11%20FEB%202021-green.svg?style=flat-square)\n![OSINT](https://img.shields.io/badge/OSINT-brightgreen.svg?style=flat-square)\n![SOCMINT](https://img.shields.io/badge/SOCMINT-brightgreen.svg?style=flat-square)\n![TOOL](https://img.shields.io/badge/TOOL-brightgreen.svg?style=flat-square)\n![CYBERSECURITY](https://img.shields.io/badge/CYBERSECURITY-brightgreen.svg?style=flat-square)\n![SPYSCRAP](https://img.shields.io/badge/SPYSCRAP-brightgreen.svg?style=flat-square)\n\n# SpyScrap\n\n![alt text](./SpyScrap.png)\n\nThis is an [OSINT](https://en.wikipedia.org/wiki/Open-source_intelligence) tool. The main purpose is recolect information from different sources like Google, Tinder, Twitter and more. It combines facial recognition methods to filter the results and uses natural language processing for obtaining important entities from the website the user appears. The tool is able to calculate a final score which indicates the amount of public exposition an user has on the Internet.\nIt has two different modules that can work indepently: CLI and Web Interface.\nBoth modules are built using docker and are easy to deploy.\n\nIf you like the tool, give us a star! :star:\n\n# CLI\n\nCLI Module for web scraping:\n\n* Tinder\n* Instagram\n* Yandex\n* Google\n* Facebook\n* BOE\n* Twitter\n\n\n## Prerequisites\n\nDocker and docker-compose\n\n\n### Installation\n```bash\ndocker build -t spyscrap .\ndocker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap  [options]\n```\nYou must put the image you want to be used for facial recognition under the shared volume in docker as in the next example:\n```\ndocker run -ti -v /Users/ruthgnz/Documents/osint/SpyScrap/src/data:/spyscrap/data sp  -t twitter -n \"ruth gonzalez novillo\" -i ./data/descarga.jpeg\n```\n\n## Usage\n\n```bash\ndocker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap  [options]\n```\n\nGet Tinder users and store data in sqlite3 database. Tinder Token must be capturen when logging into Tinder App under Local Storage.\n```bash\ndocker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t tinder -k TOKEN\t\t\n```\n\nSearch in google.\nAdd -i to download images and do facial recognition\nAdd -p to only search in an specific site Ex: Linkedin\n\n```bash\ndocker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap --tag google -n \"\u003cname surname\u003e\"\ndocker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap --tag google -n \"\u003cname surname\u003e\" -i \u003cimagePath\u003e\ndocker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap --tag google -n \"\u003cname surname\u003e\" -i \u003cimagePath\u003e\t-p \"\u003cPlace\u003e\"\n```\n\nSearch twitter profiles\n```bash\ndocker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t twitter -n \"\u003cname surname\u003e\" -s \u003cnumber of twitter pages to search\u003e\t\t\n```\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\n\nSearch facebook profiles\nAdd -i to download images do facial recognition\t\t\n```bash\ndocker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t facebook -n \"\u003cname surname\u003e\"\ndocker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap --tag facebook -n \"\u003cname surname\u003e\"\t-i \u003cimagePath\u003e\t\t\t\n```\n\nSearch instagram profiles\nAdd -i to download instagram profile image and do facial recognition\n```bash\ndocker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t instagram -n \"\u003cname surname\u003e\"\ndocker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t instagram -n \"\u003cname surname\u003e\" -i \u003cimagePath\u003e\t\t\t\n```\n\nSearch DNI, Names and Surnames in BOE\n```bash\ndocker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t boe -n \"\u003ctext to search\u003e\" -s \u003cnumber of BOE pages to search\u003e\ndocker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t boe -n \"\u003ctext to search\u003e\" -s \u003cnumber of BOE pages to search\u003e\t-e \u003cboolean\u003e -d \u003cinit date\u003e -f \u003cfinal date\u003e\t\t\t\n```\n\nOTHER EXAMPLES:\n```  docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap [options]\n  docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t tinder -k TOKEN\t\t\t\n  docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap --tag google -n \"\u003cname surname\u003e\"\t\t\n  docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap --tag google -n \"\u003cname surname\u003e\" -i \u003cimagePath\u003e\t\t\t\t\t\t\t\t\n  docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap --tag google -n \"\u003cname surname\u003e\" -i \u003cimagePath\u003e\t-p \"\u003cPlace\u003e\"\t\t\t\t\t\t\t\t\n  docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t twitter -n \"\u003cname surname\u003e\" -s \u003cnumber of twitter pages to search\u003e\t\t\t\t\t\t\n  docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t facebook -n \"\u003cname surname\u003e\"\t\t\t\t\t\t\t\t\t\t\t\n  docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap --tag facebook -n \"\u003cname surname\u003e\"\t-i \u003cimagePath\u003e\t\t\t\t\t\t\t\t\n  docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t instagram -n \"\u003cname surname\u003e\"\t\t\t\t\t\t\t\t\t\t\t\t\n  docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t instagram -n \"\u003cname surname\u003e\" -i \u003cimagePath\u003e \t\t\t\t\t\t\t\t\t\t\t\n  docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t boe -n \"\u003ctext to search\u003e\" -s \u003cnumber of BOE pages to search\u003e\n  docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t boe -n \"\u003ctext to search\u003e\" -s \u003cnumber of BOE pages to search\u003e\t-e \u003cboolean\u003e -d \u003cinit date\u003e -f \u003cfinal date\u003e\n  docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap main.py -t yandex -k \u003cimgur id\u003e -i \u003cimagePath\u003e\n  docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap main.py -t yandex -i \u003cimgUrl\u003e\n  ```\nAll the results are stored in the docker shared volume you must have configured on your localhost when running the container. The first part is the path for your local folder and you can change it. The second part must be the one in the example (/spyscrap/data)\n```bash\n-v /PATH/TO/SpyScrap/src/data:/spyscrap/data\n ```\n# Web Interface\n\nThis is a wrapper for the CLI.\n\n![alt text](./GUI.png)\n\n## Prerequisites\n\nDocker and docker-compose\n\n### Installation\n```bash\ncd web\ndocker-compose up\n```\nOnce the images are built, open the browser:\n```\nhttp:\\\\localhost\n```\nFor searching in Tinder you must put the database.db file created using the CLI in the volume inside the folder:\n\n```\nSpyScrap\\web\\data\n```\nYou will also find in this folder the results of all your searches on the web interface.\n\n## DISCLAIMER ⚠️\nThis tool is for educational purposes only.\n\nPlease only use this tool on systems you have permission to access! Ethical use only.\n\nAny actions and or activities related to the tools we have created is solely your responsibility. The misuse of the tools we have created can result in criminal charges brought against the persons in question. We will not be held responsible in the event any criminal charges be brought against any individuals misusing the tools we have made to break the law.\n\n## Authors ✒️\n\n* **Ruth González** - [@RuthGnz](https://twitter.com/RuthGnz)\n* **Miguel Hernández** -  [@MiguelHzBz](https://twitter.com/MiguelHzBz)\n\n\n## Thanks 🎁\n\n* BBVA Next Technologies SecLab Team\n\n\u003cbr\u003e\nFeel free to collaborate!!\n\u003cbr\u003e\n\n\u003e NOTE: Facial recognition is slow. The tool doesn't implement threading, and depends on your computer power. Be patient when the dataset is huge and you are using images to filter the results, specially on the Tinder module.\n\n\n---\n⌨️ with ❤️ by [@RuthGnz](https://twitter.com/RuthGnz) \u0026 [@MiguelHzBz](https://twitter.com/MiguelHzBz)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FRuthGnz%2FSpyScrap","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FRuthGnz%2FSpyScrap","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FRuthGnz%2FSpyScrap/lists"}