https://github.com/ozansz/github-crawler
A basic utility for crawling users and e-mails of users
https://github.com/ozansz/github-crawler
crawler github python python3
Last synced: 4 months ago
JSON representation
A basic utility for crawling users and e-mails of users
- Host: GitHub
- URL: https://github.com/ozansz/github-crawler
- Owner: ozansz
- License: apache-2.0
- Created: 2017-12-28T22:21:34.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2018-01-04T23:20:28.000Z (over 7 years ago)
- Last Synced: 2025-02-02T00:27:53.899Z (5 months ago)
- Topics: crawler, github, python, python3
- Language: Python
- Size: 23.4 KB
- Stars: 3
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# github-crawler
A basic utility for crawling users and e-mails of users
## Usage Example
Just type
```
$ python crawl.py --config example/users.txt --proxies example/proxies.txt --uagents example/uagents.txt
```
The script will create text files filled with user names according to the user configuration file, and then create text files filled with user e-mails (or maybe more...) usşng these user names.
The script will save these files within the directory "GitHub_Crawler" by default. You can also change the name of this directory passing an argument more:
```
$ python crawl.py --config example/users.txt --proxies example/proxies.txt --uagents example/uagents.txt --dir CustomDir
```
### About User Configuration File
The syntax for the conf file is as following:
```
city_id::city_name::starting_page::ending_page
```