Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/voidwalker7/advanced-github-scraper-with-switching-proxies
Advanced github scraper with switching proxies that will help ya find a keyword by searching the target's repos, folders, files.
https://github.com/voidwalker7/advanced-github-scraper-with-switching-proxies
firefox github-scraper proxies python scraper selenium-webdriver web-scraping
Last synced: about 2 months ago
JSON representation
Advanced github scraper with switching proxies that will help ya find a keyword by searching the target's repos, folders, files.
- Host: GitHub
- URL: https://github.com/voidwalker7/advanced-github-scraper-with-switching-proxies
- Owner: VoIDWALkER7
- Created: 2024-06-20T09:53:15.000Z (7 months ago)
- Default Branch: main
- Last Pushed: 2024-06-20T10:08:57.000Z (7 months ago)
- Last Synced: 2024-06-21T00:32:06.374Z (7 months ago)
- Topics: firefox, github-scraper, proxies, python, scraper, selenium-webdriver, web-scraping
- Language: Python
- Homepage:
- Size: 3.91 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# advanced-github-scraper-with-switching-proxies
Advanced github scraper with switching proxies that will help ya find a keyword by searching the target's repos, folders, files.
### it works!
# explanation
this is a github-scraper made with the help of python, and selenium library present in it.
this scrapes a repository that you enter, and looks through each folder and file, to search for the file that has the keyword that you are searching for. the output looks something just like this:
for proxies, you can visit the website: ```https://free-proxy-list.net``` and add the proxies in the list as follows:
```Python
[
"http://proxy1:port1",
"http://proxy2:port1",
#and so on
]
```
The proxies will be switched when accessing a new repo or a new file using the function switch_proxy()