Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/mirsamantajbakhsh/onionharvester-server
The web application for paralleling and distributing search operations among Onion Harvester clients.
https://github.com/mirsamantajbakhsh/onionharvester-server
Last synced: 2 months ago
JSON representation
The web application for paralleling and distributing search operations among Onion Harvester clients.
- Host: GitHub
- URL: https://github.com/mirsamantajbakhsh/onionharvester-server
- Owner: mirsamantajbakhsh
- License: gpl-3.0
- Created: 2017-12-14T23:36:26.000Z (about 7 years ago)
- Default Branch: master
- Last Pushed: 2018-03-31T13:42:57.000Z (almost 7 years ago)
- Last Synced: 2024-08-03T17:12:13.634Z (6 months ago)
- Language: Python
- Size: 78.1 KB
- Stars: 3
- Watchers: 2
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-network-stuff - **2**星
README
# OnionHarvester-Server
The Onion Router (TOR) project aims to hide users' IP and protect their privacy related to location and Internet usage.
Beside, users are able to create hidden services which are accessible through Onion addresses.
Unlike the normal Internet, in which list of all domains are reachable through TLDs (Top Level Domains), Onion addresses can not be listed, nor TOR does not want to list them.
Onion Harvester tries to find the updated list of all Onion addresses which are accessible through TOR.
This project uses volunteer clients all over the world to list the addresses. The project is made of two main sub-projects:
- Onion Harvester Server
- Onion Harvester Clients
The client for harvesting Onion addresses is available [here](https://github.com/mirsamantajbakhsh/OnionHarvester).
## Request
Clients can send a get request to ```http(s)://server/dispatcher/generate``` and catch search options include:
1. Unique ID
2. Start and End Onion addresses
3. Scanning ports
4. Timeout
Generated options will be in JSON format like:
```
{"id": "32 characters unique id", "start": "16 characters Onion address", "end": "16 characters Onion address", "ports": ["array of ports (default is 80, 443)"], "timeout": an integer of timeout (default is 30000 milliseconds)}
```
For example:
```
{"id": "d42ae54b9b3845b48868cde7a0ccd5bc", "start": "aaaaaaaaaaaaabea", "end": "aaaaaaaaaaaaabib", "ports": ["80", "443"], "timeout": 30000}
```
## Response
Clients after harvesting the range of addresses, can send results to ```http(s)://server/dispatcher/response``` as a JSON http post request.
It must be a HTTP post request with these parameters:
- id
- addresses
- complete
The id is a unique id that client received in request section and addresses is JSON format of valid Onion addresses. The complete parameter is used when the client has searched incompletely, so we will accept the response but that range will be kept for search again so it should have ```true``` or ```false``` value.
The addresses parameter should be an array of addresses, ports and scanned times like below. If scan hasn't any result so do'nt send addresses parameter:
```
[["address 1", port number, "date time (format: YYYY-MM-DD HH:MM)"], ["address 2", port number, "date time (format: YYYY-MM-DD HH:MM)"]]
```
For example:
Post ```id``` parameter:
```
d42ae54b9b3845b48868cde7a0ccd5bc
```
Post ```addresses``` parameter:
```
[["facebookcorewwwi", 80, "1991-06-22 22:22"], ["facebookcorewwwi", 443, "1991-06-22 22:22"]]
```
Post ```complete``` parameter:
```
true
```
## Install
The server has been written on Django framework and you can run below commands to install requirements:
```
git clone https://github.com/mohammadnassiri/OnionHarvester-Server.git
cd OnionHarvester-Server
pip3 install -r requirements.txt
python3 manage.py collectstatic
```
## Config
There is a ```.env ``` file contains configuration for server to be run. You can set debug mode, secret key, allowed hosts, database configurations and constants.
For generate new secret key, you can use this oneliner code:
```
python3 -c "import string,random; uni=string.ascii_letters+string.digits+string.punctuation; print(repr(''.join([random.SystemRandom().choice(uni) for i in range(random.randint(45,50))])))"
```
In constant section you have these options:
- TIMEOUT: The time that each client must response before end.
- RANGE_LIMIT: The range of Onion addresses that each client got in each request.
- PORTS: The ports that each client scans it on each Onion address
Please be noticed that each address can have about 3 seconds of scan time so for 2 ports it will be about 6 seconds, so you must give timeout a logical number related to range limit and ports config.
# Project
This project has been run on a server and you can use it now by see: [onionharvester.com](http://onionharvester.com)
# TODO
We did'nt consider any security principle on communications between clients and server, so we will work on some simple protocle to improve this approach.
Also template updates will be included periodically.
This project is in beta version and any we will happy to receive any suggestions.