Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/ozansz/github-crawler
A basic utility for crawling users and e-mails of users
https://github.com/ozansz/github-crawler
crawler github python python3
Last synced: 28 days ago
JSON representation
A basic utility for crawling users and e-mails of users
- Host: GitHub
- URL: https://github.com/ozansz/github-crawler
- Owner: ozansz
- License: apache-2.0
- Created: 2017-12-28T22:21:34.000Z (about 7 years ago)
- Default Branch: master
- Last Pushed: 2018-01-04T23:20:28.000Z (almost 7 years ago)
- Last Synced: 2024-10-16T14:21:34.324Z (3 months ago)
- Topics: crawler, github, python, python3
- Language: Python
- Size: 23.4 KB
- Stars: 3
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# github-crawler
A basic utility for crawling users and e-mails of users
## Usage Example
Just type
```
$ python crawl.py --config example/users.txt --proxies example/proxies.txt --uagents example/uagents.txt
```
The script will create text files filled with user names according to the user configuration file, and then create text files filled with user e-mails (or maybe more...) usşng these user names.
The script will save these files within the directory "GitHub_Crawler" by default. You can also change the name of this directory passing an argument more:
```
$ python crawl.py --config example/users.txt --proxies example/proxies.txt --uagents example/uagents.txt --dir CustomDir
```
### About User Configuration File
The syntax for the conf file is as following:
```
city_id::city_name::starting_page::ending_page
```