Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/RuthGnz/SpyScrap
CLI and GUI for OSINT. Are you very exhibited on the Internet? Check it! Twitter, Tinder, Facebook, Google, Yandex, BOE. It uses facial recognition to provide more accurate results.
https://github.com/RuthGnz/SpyScrap
boe cli docker facebook facial-recognition google osint reverse-image-search scraping-websites social-networks socmint tinder twitter
Last synced: 2 months ago
JSON representation
CLI and GUI for OSINT. Are you very exhibited on the Internet? Check it! Twitter, Tinder, Facebook, Google, Yandex, BOE. It uses facial recognition to provide more accurate results.
- Host: GitHub
- URL: https://github.com/RuthGnz/SpyScrap
- Owner: RuthGnz
- License: mit
- Created: 2019-07-02T09:53:47.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2023-11-26T11:59:06.000Z (about 1 year ago)
- Last Synced: 2024-08-02T13:27:44.497Z (6 months ago)
- Topics: boe, cli, docker, facebook, facial-recognition, google, osint, reverse-image-search, scraping-websites, social-networks, socmint, tinder, twitter
- Language: Python
- Homepage:
- Size: 15.5 MB
- Stars: 159
- Watchers: 9
- Forks: 30
- Open Issues: 30
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# SpyScrap
This is an [OSINT](https://en.wikipedia.org/wiki/Open-source_intelligence) tool. The main purpose is recolect information from different sources like Google, Tinder, Twitter and more. It combines facial recognition methods to filter the results and uses natural language processing for obtaining important entities from the website the user appears. The tool is able to calculate a final score which indicates the amount of public exposition an user has on the Internet.
It has two different modules that can work indepently: CLI and Web Interface.
Both modules are built using docker and are easy to deploy.
If you like the tool, give us a star! :star:
# CLI
CLI Module for web scraping:
* Tinder
* Instagram
* Yandex
* Google
* Facebook
* BOE
* Twitter
## Prerequisites
Docker and docker-compose
### Installation
```bash
docker build -t spyscrap .
docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap [options]
```
You must put the image you want to be used for facial recognition under the shared volume in docker as in the next example:
```
docker run -ti -v /Users/ruthgnz/Documents/osint/SpyScrap/src/data:/spyscrap/data sp -t twitter -n "ruth gonzalez novillo" -i ./data/descarga.jpeg
```
## Usage
```bash
docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap [options]
```
Get Tinder users and store data in sqlite3 database. Tinder Token must be capturen when logging into Tinder App under Local Storage.
```bash
docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t tinder -k TOKEN
```
Search in google.
Add -i to download images and do facial recognition
Add -p to only search in an specific site Ex: Linkedin
```bash
docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap --tag google -n ""
docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap --tag google -n "" -i
docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap --tag google -n "" -i -p ""
```
Search twitter profiles
```bash
docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t twitter -n "" -s
```
Search facebook profiles
Add -i to download images do facial recognition
```bash
docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t facebook -n ""
docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap --tag facebook -n "" -i
```
Search instagram profiles
Add -i to download instagram profile image and do facial recognition
```bash
docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t instagram -n ""
docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t instagram -n "" -i
```
Search DNI, Names and Surnames in BOE
```bash
docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t boe -n "" -s
docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t boe -n "" -s -e -d -f
```
OTHER EXAMPLES:
``` docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap [options]
docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t tinder -k TOKEN
docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap --tag google -n ""
docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap --tag google -n "" -i
docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap --tag google -n "" -i -p ""
docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t twitter -n "" -s
docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t facebook -n ""
docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap --tag facebook -n "" -i
docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t instagram -n ""
docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t instagram -n "" -i
docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t boe -n "" -s
docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap -t boe -n "" -s -e -d -f
docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap main.py -t yandex -k -i
docker run -ti -v /PATH/TO/SpyScrap/src/data:/spyscrap/data spyscrap main.py -t yandex -i
```
All the results are stored in the docker shared volume you must have configured on your localhost when running the container. The first part is the path for your local folder and you can change it. The second part must be the one in the example (/spyscrap/data)
```bash
-v /PATH/TO/SpyScrap/src/data:/spyscrap/data
```
# Web Interface
This is a wrapper for the CLI.
## Prerequisites
Docker and docker-compose
### Installation
```bash
cd web
docker-compose up
```
Once the images are built, open the browser:
```
http:\\localhost
```
For searching in Tinder you must put the database.db file created using the CLI in the volume inside the folder:
```
SpyScrap\web\data
```
You will also find in this folder the results of all your searches on the web interface.
## DISCLAIMER ⚠️
This tool is for educational purposes only.
Please only use this tool on systems you have permission to access! Ethical use only.
Any actions and or activities related to the tools we have created is solely your responsibility. The misuse of the tools we have created can result in criminal charges brought against the persons in question. We will not be held responsible in the event any criminal charges be brought against any individuals misusing the tools we have made to break the law.
## Authors ✒️
* **Ruth González** - [@RuthGnz](https://twitter.com/RuthGnz)
* **Miguel Hernández** - [@MiguelHzBz](https://twitter.com/MiguelHzBz)
## Thanks 🎁
* BBVA Next Technologies SecLab Team
Feel free to collaborate!!
> NOTE: Facial recognition is slow. The tool doesn't implement threading, and depends on your computer power. Be patient when the dataset is huge and you are using images to filter the results, specially on the Tinder module.
---
⌨️ with ❤️ by [@RuthGnz](https://twitter.com/RuthGnz) & [@MiguelHzBz](https://twitter.com/MiguelHzBz)