https://github.com/xewdy444/proxy-scraper-checker

A command-line tool for scraping and checking HTTP and SOCKS5 proxies from the checkerproxy.net proxies archive
https://github.com/xewdy444/proxy-scraper-checker

checker http proxy scraper socks5

Last synced: 14 days ago
JSON representation

A command-line tool for scraping and checking HTTP and SOCKS5 proxies from the checkerproxy.net proxies archive

• Host: GitHub
• URL: https://github.com/xewdy444/proxy-scraper-checker
• Owner: Xewdy444
• License: mit
• Created: 2023-04-14T03:06:36.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2024-10-31T15:29:52.000Z (15 days ago)
• Last Synced: 2024-10-31T16:29:37.014Z (15 days ago)
• Topics: checker, http, proxy, scraper, socks5
• Language: Rust
• Homepage: https://crates.io/crates/proxy-scraper-checker
• Size: 652 KB
• Stars: 11
• Watchers: 1
• Forks: 6
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Proxy-Scraper-Checker

A command-line tool for scraping and checking HTTP and SOCKS5 proxies from the [checkerproxy.net proxies archive](https://checkerproxy.net/getAllProxy). By default, the working proxies are written to `proxies/http.txt` and `proxies/socks5.txt` according to the respective proxy type.

![image](https://github.com/Xewdy444/Proxy-Scraper-Checker/assets/95155966/91b55084-f82c-43d6-be29-6eaee1c8b23f)

## Docker

### Local

    $ docker build -t proxy-scraper-checker:latest .

    $ docker run -it -v ./proxies:/proxies --rm proxy-scraper-checker:latest

### GitHub Container Registry

    $ docker run -it -v ./proxies:/proxies --rm ghcr.io/xewdy444/proxy-scraper-checker:latest

## Installation

### Local

    $ cargo install --path .

### Rust Package Registry

    $ cargo install proxy-scraper-checker

## Usage

```

A command-line tool for scraping and checking HTTP and SOCKS5 proxies from the checkerproxy.net proxies archive

Usage: proxy-scraper-checker.exe [OPTIONS]

Options:

  -u, --url           The URL to check the proxies against [default: https://httpbin.org/ip]

      --tasks       The number of tasks to run concurrently for checking proxies [default: 512]

      --timeout   The proxy request timeout in seconds [default: 30]

  -f, --folder     The folder to save the working proxies to [default: proxies]

  -a, --anonymous          Only save anonymous proxies

      --http               Only save HTTP proxies

      --socks5             Only save SOCKS5 proxies

  -n, --no-set-limit       Do not set the open file limit (tasks * 2)

  -h, --help               Print help

```