{"id":26896888,"url":"https://github.com/rtulke/proxyreaper","last_synced_at":"2026-02-28T18:06:45.437Z","repository":{"id":280189775,"uuid":"941242594","full_name":"rtulke/proxyreaper","owner":"rtulke","description":"A powerful proxy checker that checks and evaluates proxies for speed, availability, and anonymity across multiple protocols (HTTP, HTTPS, SOCKS4, SOCKS5).","archived":false,"fork":false,"pushed_at":"2025-10-31T14:17:07.000Z","size":5553,"stargazers_count":12,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-31T15:26:25.810Z","etag":null,"topics":["anonymity","anonymous","hacking","hacking-tool","https","https-proxy","proxies","proxy","proxy-checker","proxy-list","proxy-server","socks","socks-checker","socks-proxy","socks4","socks4-proxy","socks5","socks5-proxy","speedx","vpn"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rtulke.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-03-01T20:30:20.000Z","updated_at":"2025-10-31T14:12:16.000Z","dependencies_parsed_at":"2025-04-01T04:12:37.522Z","dependency_job_id":null,"html_url":"https://github.com/rtulke/proxyreaper","commit_stats":null,"previous_names":["rtulke/proxyreaper"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/rtulke/proxyreaper","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rtulke%2Fproxyreaper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rtulke%2Fproxyreaper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rtulke%2Fproxyreaper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rtulke%2Fproxyreaper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rtulke","download_url":"https://codeload.github.com/rtulke/proxyreaper/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rtulke%2Fproxyreaper/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29946467,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-28T17:57:52.716Z","status":"ssl_error","status_checked_at":"2026-02-28T17:57:31.974Z","response_time":90,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anonymity","anonymous","hacking","hacking-tool","https","https-proxy","proxies","proxy","proxy-checker","proxy-list","proxy-server","socks","socks-checker","socks-proxy","socks4","socks4-proxy","socks5","socks5-proxy","speedx","vpn"],"created_at":"2025-04-01T04:02:21.729Z","updated_at":"2026-02-28T18:06:45.427Z","avatar_url":"https://github.com/rtulke.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"## Proxy Reaper - Documentation\n\n![Proxy Reaper Banner](https://raw.githubusercontent.com/rtulke/proxyreaper/main/demo/proxyreaper.gif)\n\nProxy Reaper is a powerful tool for checking proxy servers for availability, speed and anonymity. It supports various protocols such as HTTP, HTTPS, SOCKS4, and SOCKS5, and offers enhanced features for managing and testing proxies efficiently.\n\n## Table of Contents\n\n1. [Installation](#installation)\n2. [Installing OS wide (Debian based Distributions)](#installing-os-wide)\n3. [Basic Usage](#basic-usage)\n4. [Command Line Arguments](#command-line-arguments)\n5. [Configuration File](#configuration-file)\n6. [Proxy Formats and Sources](#proxy-formats-and-sources)\n7. [Speed Categories](#speed-categories)\n8. [Anonymity Levels](#anonymity-levels)\n9. [Filter Options](#filter-options)\n10. [Output Formats](#output-formats)\n11. [Advanced Features](#advanced-features)\n12. [Troubleshooting](#troubleshooting)\n13. [Examples](#examples)\n\n## Installation\n\n### Prerequisites\n\n- Python 3.6 or higher\n- Required Python packages:\n  - requests\n  - PySocks\n  - colorama\n\n\n### Download and Installation\n\n```bash\n# Clone the repository (if using Git)\ngit clone https://github.com/rtulke/proxyreaper.git\ncd proxyreaper\n\n# Or download the script directly\nwget https://raw.githubusercontent.com/rtulke/proxyreaper/main/proxyreaper.py\nchmod +x proxyreaper.py\n```\n\n\n### Installing Dependencies\n\n\n```bash\npip install requests PySocks colorama\n```\n\nor use the requirements.txt file:\n\n#### Linux / MacOS\n```bash\ncd proxyreaper\nchmod +x proxyreaper.py\npython -m venv venv\nsource venv/bin/activate\npip install -r requirements.txt\n```\n\n#### Windows\n\n```bash\ncd proxyreaper\npython -m venv venv\nvenv\\Scripts\\activate\npip install -r requirements.txt\n```\n\n## Installing OS wide\nThe system wide installation works for the most Debian based Distributions, like Debian, Ubuntu, Mint, Raspberry PI OS, Kali Linux...\n\n```bash\n# start as root or try: \"sudo su -\" or \"sudo -i\"\nsu - root\ncd ~\n\n# create dev directory for development stuff if needed\nmkdir dev\ncd dev\n\n# download via git\ngit clone https://github.com/rtulke/proxyreaper.git\ncd proxyreaper\nchmod +x proxyreaper.py\n\n# install dependencies\napt install python3-socks python3-colorama\n\n# copy proxyreaper script to `/usr/local/bin`\ncp proxyreaper.py /usr/local/bin/proxyreaper\n\n# install man page and updating mandb\ncp proxyreaper.1 /usr/local/share/man/man1/\nmandb\n\n# use the proxyreaper script from any directory\nproxyreaper -h\n\n# generate new config file\nproxyreaper --config\n\n# you can also try to edit the new generated configuration file\nvim ~/.proxyreaper.conf\n\n# try using the manual\nman proxyreaper\n```\n\n\n## Basic Usage\n\n```bash\n# Test a single proxy\npython proxyreaper.py https://www.google.com -p 1.2.3.4:8080\n\n# Test multiple proxies from a file\npython proxyreaper.py https://www.google.com -p proxies.txt\n\n# Test multiple files using glob patterns\npython proxyreaper.py https://www.google.com -p \"proxies/*.txt\"\n\n# Create a default configuration file\npython proxyreaper.py --config\n\n# Use automatic mode to download proxies from URLs defined in config [proxysources]\npython proxyreaper.py https://www.google.com -A\n\n# Filter and save only ultrafast proxies from Germany\npython proxyreaper.py https://www.google.com -p proxies.txt --filter-status ultrafast --filter-country de\n```\n\n## Command Line Arguments\n\n### Core Arguments\n\n| Argument | Short | Type | Description |\n|----------|-------|------|-------------|\n| `url` | - | positional | URL to test the proxies against |\n| `--proxy` | `-p` | string | Proxy, file with proxies, or glob pattern (e.g., `*.txt`, `proxies[1-5].txt`) |\n| `--timeout` | `-t` | integer | Timeout in seconds (default from config) |\n| `--output` | `-o` | choice | Save results format: `json`, `csv`, or `sqlite` (default: `csv`) |\n| `--response-time` | `-R` | float | Filter for fast proxies (maximum response time in milliseconds) |\n| `--concurrent` | `-c` | integer | Number of concurrent checks (default: 10) |\n| `--debug` | `-d` | flag | Enable detailed debug output |\n| `--version` | `-v` | flag | Display version information and exit |\n| `--automatic-mode` | `-A` | flag | Download proxy lists from configured URLs |\n| `--config` | `-C` | flag | Create default config file in `~/.proxyreaper.conf` |\n| `--reverse-lookup` | `-l` | flag | Enable reverse DNS lookup for proxy IPs (slower but shows hostnames) |\n\n### Filter Arguments\n\n| Argument | Type | Values | Description |\n|----------|------|--------|-------------|\n| `--filter-status` | multi | `ultrafast`, `fast`, `medium`, `slow` | Filter by speed category (can combine multiple) |\n| `--filter-anonymity` | multi | `highanonymous`, `anonymous`, `headerleak`, `transparent` | Filter by anonymity level (can combine multiple) |\n| `--filter-protocol` | multi | `http`, `https`, `socks4`, `socks5` | Filter by protocol type (can combine multiple) |\n| `--filter-country` | multi | ISO codes | Filter by country code, e.g., `de us uk fr` (can combine multiple) |\n| `--filter-tld` | multi | country codes | Filter by country TLD based on GeoIP, e.g., `de us uk` (can combine multiple) |\n\n## Configuration File\n\nProxy Reaper supports configuration files to store frequently used settings. The configuration files are searched in the following order:\n\n1. `~/.proxyreaper.conf` (user-specific configuration)\n2. `/etc/proxyreaper.conf` (system-wide configuration)\n3. Default values (if no configuration file is found)\n\n### Creating a Configuration File\n\nYou can create a default configuration file using:\n\n```bash\npython proxyreaper.py --config\n```\n\nThis will create a file at `~/.proxyreaper.conf` with default values.\n\n### Configuration File Format\n\nThe configuration file uses the INI format with the following sections:\n\n```ini\n[general]\ntimeout = 5\nconcurrent = 10\nresponse_time_filter = 1000\ntest_url = https://www.google.com\n\n[output]\nformat = csv\nsave_directory = results\n\n[proxysources]\nurls = https://raw.githubusercontent.com/username/proxy-list/main/proxies.txt, https://some-proxy-list.com/proxies.txt\n\n[advanced]\ndebug = false\nanonymity_check_url = https://httpbin.org/get\n```\n\n### Configuration Sections Explained\n\n#### [general]\n\n- `timeout`: Connection timeout in seconds (default: 5)\n- `concurrent`: Number of concurrent proxy checks (default: 10)\n- `response_time_filter`: Maximum response time in milliseconds for filtering (default: 1000)\n- `test_url`: URL to use for testing proxies (default: https://www.google.com)\n\n#### [output]\n\n- `format`: Default output format - `json`, `csv`, or `sqlite` (default: `csv`)\n- `save_directory`: Directory to save results (default: `results`)\n\n#### [proxysources]\n\n- `urls`: Comma-separated list of URLs to download proxy lists from when using automatic mode\n\n#### [advanced]\n\n- `debug`: Enable detailed debug output by default (true/false)\n- `anonymity_check_url`: URL to use for anonymity checks (default: https://httpbin.org/get)\n\n## Proxy Formats and Sources\n\n### Supported Proxy Formats\n\nProxy Reaper supports several proxy formats:\n\n- `host:port` (e.g., `127.0.0.1:8080`) - Defaults to HTTP protocol\n- `protocol://host:port` (e.g., `http://127.0.0.1:8080`)\n- `protocol://username:password@host:port` (e.g., `http://user:pass@127.0.0.1:8080`)\n\nSupported protocols:\n- HTTP\n- HTTPS\n- SOCKS4\n- SOCKS5\n\n### Proxy Input Methods\n\n#### 1. Single Proxy\nDirectly specify a proxy on the command line:\n```bash\npython proxyreaper.py https://www.google.com -p 127.0.0.1:8080\n```\n\n#### 2. Multiple Proxies (Comma-separated)\nUse comma-separated list:\n```bash\npython proxyreaper.py https://www.google.com -p \"127.0.0.1:8080,192.168.1.1:3128\"\n```\n\n#### 3. Single Text File\nProvide a file with one proxy per line:\n```bash\npython proxyreaper.py https://www.google.com -p proxies.txt\n```\n\n#### 4. Multiple Files (Glob Patterns)\nUse glob patterns to match multiple files:\n\n```bash\n# All .txt files in a directory\npython proxyreaper.py https://www.google.com -p \"proxies/*.txt\"\n\n# Files matching a specific pattern\npython proxyreaper.py https://www.google.com -p \"proxylist*.txt\"\n\n# Files with numbered ranges\npython proxyreaper.py https://www.google.com -p \"proxylist[1-5].txt\"\n\n# Complex patterns\npython proxyreaper.py https://www.google.com -p \"../sources/proxy_*.txt\"\n```\n\nSupported glob patterns:\n- `*` - Matches any characters\n- `?` - Matches single character\n- `[1-5]` - Matches range of characters\n- `[abc]` - Matches specific characters\n\n#### 5. Automatic Download\nUse automatic mode to download proxies from URLs specified in the configuration:\n```bash\npython proxyreaper.py https://www.google.com -A\n```\n\n### Example Proxy List File\n\nA proxy list file (e.g., `proxies.txt`) should contain one proxy per line:\n\n```\n# HTTP proxies\nhttp://192.168.1.1:8080\nhttp://user:pass@192.168.1.2:8080\n\n# HTTPS proxies\nhttps://192.168.1.3:443\n\n# SOCKS proxies\nsocks4://192.168.1.4:1080\nsocks5://192.168.1.5:1080\n\n# Without protocol (defaults to HTTP)\n192.168.1.6:8080\n```\n\nLines starting with `#` are treated as comments and ignored.\n\n## Speed Categories\n\nProxy Reaper categorizes proxies into four speed categories based on their response time:\n\n| Category | Response Time | Description | Use Case |\n|----------|---------------|-------------|----------|\n| **Ultrafast** | \u003c 100ms | Extremely fast proxies | Real-time applications, streaming, gaming |\n| **Fast** | 100-500ms | Fast proxies | Web browsing, API calls, general use |\n| **Medium** | 500-1000ms | Medium speed proxies | Background tasks, batch processing |\n| **Slow** | \u003e 1000ms | Slow proxies | Non-time-critical tasks |\n\n### Speed Category Output\n\nThe speed category is included in all output formats:\n\n**CSV Output:**\n```csv\nproxy,hostname,status,speed_category,response_time,country,city,anonymity,protocol,check_time\nhttp://1.2.3.4:8080,1.2.3.4,working,ultrafast,87.5,Germany,Berlin,High Anonymous,http,2025-10-31 14:30:45\n```\n\n**Console Output:**\n```\n[1/100] ULTRAFAST - http://1.2.3.4:8080 (Germany, Berlin, High Anonymous) - 87 ms\n[2/100] FAST - http://5.6.7.8:3128 (United States, New York, Anonymous) - 245 ms\n[3/100] SLOW - http://9.10.11.12:8080 (France, Paris, Transparent) - 1523 ms\n```\n\n## Anonymity Levels\n\nProxy Reaper categorizes proxies into different anonymity levels and uses color coding for easy identification:\n\n| Level | Color | Description | Headers Visible |\n|-------|-------|-------------|-----------------|\n| **High Anonymous** | Green (Bright) | Your IP address and proxy status are completely hidden | No proxy headers |\n| **Anonymous** | Blue (Bright) | Your IP address is hidden, but the server knows you're using a proxy | Minimal proxy headers |\n| **Anonymous (Header leak)** | Yellow | Your IP is hidden, but proxy headers are visible | Proxy headers visible |\n| **Transparent** | Red | Your original IP address is visible to the server | All headers visible |\n\n### How Anonymity is Determined\n\n1. **High Anonymous**: The proxy changes your IP and doesn't add proxy-related headers\n2. **Anonymous**: The proxy changes your IP but adds minimal headers revealing it's a proxy\n3. **Anonymous (Header leak)**: The proxy changes your IP but adds headers that reveal proxy details\n4. **Transparent**: The proxy doesn't hide your original IP address or forwards it in headers\n\n## Filter Options\n\nProxy Reaper provides powerful filtering capabilities to save only the proxies that meet your specific requirements. Filters are applied when saving results and create descriptive filenames.\n\n### Filter Behavior\n\n- **Multiple values within same filter**: OR logic (any match passes)\n- **Multiple different filters**: AND logic (all must match)\n- **Working proxies only**: Failed proxies are automatically excluded\n\n### Speed Status Filter\n\nFilter proxies by their speed category:\n\n```bash\n# Save only ultrafast proxies\npython proxyreaper.py https://www.google.com -p proxies.txt --filter-status ultrafast\n\n# Save ultrafast OR fast proxies\npython proxyreaper.py https://www.google.com -p proxies.txt --filter-status ultrafast fast\n\n# Save medium OR slow proxies\npython proxyreaper.py https://www.google.com -p proxies.txt --filter-status medium slow\n```\n\n**Available values:** `ultrafast`, `fast`, `medium`, `slow`\n\n### Anonymity Filter\n\nFilter proxies by their anonymity level:\n\n```bash\n# Save only high anonymous proxies\npython proxyreaper.py https://www.google.com -p proxies.txt --filter-anonymity highanonymous\n\n# Save high anonymous OR anonymous proxies\npython proxyreaper.py https://www.google.com -p proxies.txt --filter-anonymity highanonymous anonymous\n\n# Exclude transparent proxies (save all except transparent)\npython proxyreaper.py https://www.google.com -p proxies.txt --filter-anonymity highanonymous anonymous headerleak\n```\n\n**Available values:** `highanonymous`, `anonymous`, `headerleak`, `transparent`\n\n### Protocol Filter\n\nFilter proxies by their protocol:\n\n```bash\n# Save only HTTP proxies\npython proxyreaper.py https://www.google.com -p proxies.txt --filter-protocol http\n\n# Save HTTP OR HTTPS proxies\npython proxyreaper.py https://www.google.com -p proxies.txt --filter-protocol http https\n\n# Save only SOCKS proxies\npython proxyreaper.py https://www.google.com -p proxies.txt --filter-protocol socks4 socks5\n```\n\n**Available values:** `http`, `https`, `socks4`, `socks5`\n\n### Country Filter\n\nFilter proxies by their country location (based on GeoIP):\n\n```bash\n# Save only German proxies\npython proxyreaper.py https://www.google.com -p proxies.txt --filter-country de\n\n# Save German OR US proxies\npython proxyreaper.py https://www.google.com -p proxies.txt --filter-country de us\n\n# Save European proxies\npython proxyreaper.py https://www.google.com -p proxies.txt --filter-country de fr uk nl it es\n```\n\n**Supported country codes:** ISO 3166-1 alpha-2 codes (e.g., `de`, `us`, `uk`, `fr`, `jp`, `cn`, etc.)\n\nFull list of ~150 supported countries available in the code.\n\n### TLD Filter\n\nFilter proxies by country TLD based on GeoIP location:\n\n```bash\n# Save proxies from .de domains (German IPs)\npython proxyreaper.py https://www.google.com -p proxies.txt --filter-tld de\n\n# Save proxies from .de OR .us domains\npython proxyreaper.py https://www.google.com -p proxies.txt --filter-tld de us\n```\n\n**Note:** TLD filtering is based on GeoIP country mapping, not actual domain names.\n\n### Combining Filters\n\nCombine multiple filters for precise proxy selection:\n\n```bash\n# Ultrafast, high anonymous proxies from Germany\npython proxyreaper.py https://www.google.com -p proxies.txt \\\n  --filter-status ultrafast \\\n  --filter-anonymity highanonymous \\\n  --filter-country de\n\n# Fast HTTP/HTTPS proxies from US or UK\npython proxyreaper.py https://www.google.com -p proxies.txt \\\n  --filter-status ultrafast fast \\\n  --filter-protocol http https \\\n  --filter-country us uk\n\n# High anonymous SOCKS5 proxies from Europe (any speed)\npython proxyreaper.py https://www.google.com -p proxies.txt \\\n  --filter-anonymity highanonymous \\\n  --filter-protocol socks5 \\\n  --filter-country de fr uk nl it es\n```\n\n### Filter Output Filenames\n\nWhen filters are applied, the output filename includes filter information:\n\n```bash\n# Without filters:\nresults/proxy_results_20251031_143045.csv\n\n# With filters:\nresults/filtered_proxies_ultrafast_highanonymous_http_https_de_us_20251031_143045.csv\n```\n\nThis makes it easy to identify which filters were used for each result file.\n\n## Output Formats\n\nProxy Reaper supports multiple output formats. The default format is **CSV** if no `-o` option is specified.\n\n### CSV Output (Default)\n\n```bash\n# Uses CSV by default\npython proxyreaper.py https://www.google.com -p proxies.txt\n\n# Explicitly specify CSV\npython proxyreaper.py https://www.google.com -p proxies.txt -o csv\n```\n\nCreates a CSV file with detailed proxy information:\n\n```csv\nproxy,hostname,status,speed_category,response_time,country,city,anonymity,protocol,check_time\nhttp://1.2.3.4:8080,proxy.example.com,working,ultrafast,87.5,Germany,Berlin,High Anonymous,http,2025-10-31 14:30:45\nhttp://5.6.7.8:3128,5.6.7.8,working,fast,245.3,United States,New York,Anonymous,http,2025-10-31 14:30:46\n```\n\n### JSON Output\n\n```bash\npython proxyreaper.py https://www.google.com -p proxies.txt -o json\n```\n\nCreates a JSON file with detailed proxy information:\n\n```json\n[\n  {\n    \"proxy\": \"http://1.2.3.4:8080\",\n    \"hostname\": \"proxy.example.com\",\n    \"status\": \"working\",\n    \"speed_category\": \"ultrafast\",\n    \"response_time\": 87.5,\n    \"country\": \"Germany\",\n    \"city\": \"Berlin\",\n    \"anonymity\": \"High Anonymous\",\n    \"protocol\": \"http\",\n    \"check_time\": \"2025-10-31 14:30:45\"\n  }\n]\n```\n\n### SQLite Output\n\n```bash\npython proxyreaper.py https://www.google.com -p proxies.txt -o sqlite\n```\n\nCreates an SQLite database with a `proxies` table containing the proxy information. This is useful for more complex queries and data analysis.\n\n**Database schema:**\n```sql\nCREATE TABLE proxies (\n    proxy TEXT,\n    hostname TEXT,\n    status TEXT,\n    speed_category TEXT,\n    response_time REAL,\n    country TEXT,\n    city TEXT,\n    anonymity TEXT,\n    protocol TEXT,\n    check_time TEXT\n);\n```\n\n### Text Output\n\nIn addition to the specified format, Proxy Reaper **always creates a plain text file** with just the working proxies (one per line), which can be easily used in other applications:\n\n```\nhttp://1.2.3.4:8080\nhttp://5.6.7.8:3128\nsocks5://9.10.11.12:1080\n```\n\n## Advanced Features\n\n### Reverse DNS Lookup\n\nEnable reverse DNS lookup to resolve proxy IPs to hostnames:\n\n```bash\npython proxyreaper.py https://www.google.com -p proxies.txt -l\n```\n\n**Output without `-l`:**\n```\n[1/100] ULTRAFAST - http://1.2.3.4:8080 (Germany, Berlin, High Anonymous) - 87 ms\n```\n\n**Output with `-l`:**\n```\n[1/100] ULTRAFAST - proxy.example.com (Germany, Berlin, High Anonymous) - 87 ms\n```\n\nThe hostname is also saved in the output files:\n- CSV: `hostname` column\n- JSON: `hostname` field\n- Text: unchanged (still shows proxy URL)\n\n**Note:** Reverse DNS lookup is slower as it performs additional DNS queries. Use only when needed.\n\n### Glob Pattern Support\n\nLoad proxies from multiple files using glob patterns:\n\n```bash\n# All .txt files in directory\npython proxyreaper.py https://www.google.com -p \"proxies/*.txt\"\n\n# Specific pattern\npython proxyreaper.py https://www.google.com -p \"proxylist_*.txt\"\n\n# Numbered files\npython proxyreaper.py https://www.google.com -p \"proxylist[1-9].txt\"\n\n# Range\npython proxyreaper.py https://www.google.com -p \"proxy_[a-z].txt\"\n```\n\n**Console output:**\n```\nFound 15 file(s) matching pattern: proxies/*.txt\nLoaded 450 proxies from proxy_http.txt\nLoaded 230 proxies from proxy_socks.txt\n...\nTotal: Loaded 1250 proxies from 15 file(s)\n```\n\n### Automatic Saving During Execution\n\nProxy Reaper automatically saves intermediate results every 5 proxies checked. This ensures that even if the program is interrupted, you won't lose your progress.\n\nAutosave files are stored in the `results` directory:\n- `proxy_results_TIMESTAMP_final.json` - Complete autosave\n\n### GeoIP Caching\n\nTo improve performance and reduce API calls, Proxy Reaper caches geographical information for IP addresses. This significantly speeds up testing when multiple proxies are hosted on the same server or subnet.\n\n### Concurrent Testing\n\nProxy Reaper utilizes ThreadPoolExecutor for efficient concurrent proxy testing. You can control the number of concurrent connections with the `-c` or `--concurrent` parameter:\n\n```bash\n# Test with 20 concurrent workers\npython proxyreaper.py https://www.google.com -p proxies.txt -c 20\n\n# Test with 50 concurrent workers (for large lists)\npython proxyreaper.py https://www.google.com -p proxies.txt -c 50\n```\n\n**Performance recommendations:**\n- Default: 10 workers (balanced)\n- Small lists (\u003c100): 5-10 workers\n- Medium lists (100-1000): 10-20 workers\n- Large lists (\u003e1000): 20-50 workers\n\n### Automatic Proxy List Downloads\n\nThe automatic mode (`-A` or `--automatic-mode`) allows Proxy Reaper to download proxy lists from URLs specified in the configuration file:\n\n```bash\npython proxyreaper.py https://www.google.com -A\n```\n\nThis feature is useful for maintaining an up-to-date proxy list without manual intervention.\n\n**Configuration example:**\n```ini\n[proxysources]\nurls = https://raw.githubusercontent.com/user/proxy-list/main/http.txt,\n       https://www.proxy-list.download/api/v1/get?type=http,\n       https://api.proxyscrape.com/v2/?request=get\u0026protocol=http\n```\n\n### Detailed Debug Output\n\nEnable detailed debugging information with the `-d` or `--debug` flag:\n\n```bash\npython proxyreaper.py https://www.google.com -p proxies.txt -d\n```\n\nDebug mode shows:\n- HTTP headers and responses\n- Connection details\n- Error messages with stack traces\n- GeoIP API calls\n- File operations\n- Proxy validation details\n\n## Troubleshooting\n\n### Common Issues and Solutions\n\n#### 1. No valid proxies found\n\n**Symptoms:**\n```\nNo valid proxies found after validation.\nExiting.\n```\n\n**Solutions:**\n- Check that your proxy list file exists and has the correct format\n- Ensure that each proxy is on a separate line\n- Verify proxy format: `protocol://host:port` or `host:port`\n- If using glob patterns, check that the pattern matches existing files\n- If using automatic mode, check that the URLs in the configuration file are accessible\n\n#### 2. No files matched pattern\n\n**Symptoms:**\n```\nNo files matched pattern: proxies/*.txt\n```\n\n**Solutions:**\n- Verify the glob pattern is correct\n- Check that files exist in the specified directory\n- Use absolute paths or verify current working directory\n- Ensure file extensions match (e.g., `.txt` not `.TXT`)\n\n#### 3. Connection errors\n\n**Symptoms:**\n```\n[1/100] FAILED - http://1.2.3.4:8080\n```\n\n**Solutions:**\n- Increase the timeout value with `-t` or in the configuration file\n- Check your internet connection\n- Verify that the test URL is accessible from your location\n- Try a different test URL\n- Some proxies may be genuinely offline\n\n#### 4. Performance issues\n\n**Symptoms:**\n- Slow proxy checking\n- High CPU or memory usage\n- System slowdown\n\n**Solutions:**\n- Reduce the number of concurrent connections: `-c 5`\n- Use smaller proxy lists for testing\n- Increase timeout for slow networks\n- Disable reverse DNS lookup (remove `-l`)\n- Use debug mode to identify bottlenecks: `-d`\n\n#### 5. ImportError or ModuleNotFoundError\n\n**Symptoms:**\n```\nModuleNotFoundError: No module named 'requests'\n```\n\n**Solutions:**\n- Install required dependencies: `pip install requests PySocks colorama`\n- Activate virtual environment if using one: `source venv/bin/activate`\n- Verify Python version: `python --version` (3.6+ required)\n\n#### 6. Permission denied errors\n\n**Symptoms:**\n```\nPermissionError: [Errno 13] Permission denied: 'results/proxy_results.csv'\n```\n\n**Solutions:**\n- Check write permissions for the output directory\n- Create the output directory manually: `mkdir results`\n- Change output directory in config or use `-o` flag\n- Run with appropriate permissions\n\n## Examples\n\n### Basic Testing\n\n```bash\n# Test a single proxy against Google\npython proxyreaper.py https://www.google.com -p 1.2.3.4:8080\n\n# Test multiple proxies with increased timeout\npython proxyreaper.py https://www.google.com -p proxies.txt -t 10\n\n# Test proxies against a specific website\npython proxyreaper.py https://www.example.com -p proxies.txt\n\n# Test all proxy files in a directory\npython proxyreaper.py https://www.google.com -p \"proxy_sources/*.txt\"\n```\n\n### Speed and Performance\n\n```bash\n# Only consider proxies with response time under 500ms as \"FAST\"\npython proxyreaper.py https://www.google.com -p proxies.txt -R 500\n\n# Test with 30 concurrent workers for faster processing\npython proxyreaper.py https://www.google.com -p proxies.txt -c 30\n\n# Combine for maximum performance\npython proxyreaper.py https://www.google.com -p proxies.txt -c 50 -t 3\n```\n\n### Filtering Examples\n\n```bash\n# Save only ultrafast proxies\npython proxyreaper.py https://www.google.com -p proxies.txt --filter-status ultrafast\n\n# Save only high anonymous proxies\npython proxyreaper.py https://www.google.com -p proxies.txt --filter-anonymity highanonymous\n\n# Save only HTTP and HTTPS proxies\npython proxyreaper.py https://www.google.com -p proxies.txt --filter-protocol http https\n\n# Save only German proxies\npython proxyreaper.py https://www.google.com -p proxies.txt --filter-country de\n\n# Save ultrafast, high anonymous proxies from Germany or US\npython proxyreaper.py https://www.google.com -p proxies.txt \\\n  --filter-status ultrafast \\\n  --filter-anonymity highanonymous \\\n  --filter-country de us\n\n# Save fast HTTP proxies from Europe\npython proxyreaper.py https://www.google.com -p proxies.txt \\\n  --filter-status ultrafast fast \\\n  --filter-protocol http \\\n  --filter-country de fr uk nl it es pl\n```\n\n### Output Format Examples\n\n```bash\n# Save as CSV (default)\npython proxyreaper.py https://www.google.com -p proxies.txt\n\n# Save as JSON\npython proxyreaper.py https://www.google.com -p proxies.txt -o json\n\n# Save as SQLite database\npython proxyreaper.py https://www.google.com -p proxies.txt -o sqlite\n\n# Filter and save as JSON\npython proxyreaper.py https://www.google.com -p proxies.txt \\\n  --filter-status ultrafast \\\n  -o json\n```\n\n### Advanced Usage\n\n```bash\n# Enable debug mode for detailed information\npython proxyreaper.py https://www.google.com -p proxies.txt -d\n\n# Use automatic mode to download proxies from configured URLs\npython proxyreaper.py https://www.google.com -A\n\n# Use automatic mode with specific URLs\npython proxyreaper.py https://www.google.com -A -p \"https://example.com/proxies.txt,https://example2.com/list.txt\"\n\n# Run with reverse DNS lookup\npython proxyreaper.py https://www.google.com -p proxies.txt -l\n\n# Comprehensive test with all features\npython proxyreaper.py https://www.google.com \\\n  -p \"proxy_sources/*.txt\" \\\n  -c 30 \\\n  -t 10 \\\n  -l \\\n  --filter-status ultrafast fast \\\n  --filter-anonymity highanonymous \\\n  --filter-country de us uk \\\n  -o json \\\n  -d\n```\n\n### Practical Scenarios\n\n```bash\n# Find the fastest German proxies for web scraping\npython proxyreaper.py https://www.google.com -p proxies.txt \\\n  --filter-status ultrafast fast \\\n  --filter-country de \\\n  -o csv\n\n# Get high anonymous SOCKS5 proxies for secure browsing\npython proxyreaper.py https://www.google.com -p proxies.txt \\\n  --filter-anonymity highanonymous \\\n  --filter-protocol socks5 \\\n  -o json\n\n# Build a database of all working European proxies\npython proxyreaper.py https://www.google.com -p \"europe/*.txt\" \\\n  --filter-country de fr uk nl it es pl cz at ch \\\n  -o sqlite\n\n# Quick test of premium proxy list with hostname resolution\npython proxyreaper.py https://www.google.com \\\n  -p premium_proxies.txt \\\n  -l \\\n  -c 20 \\\n  --filter-status ultrafast \\\n  --filter-anonymity highanonymous\n\n# Daily automated proxy harvesting\npython proxyreaper.py https://www.google.com -A \\\n  -c 50 \\\n  --filter-status ultrafast fast medium \\\n  --filter-anonymity highanonymous anonymous \\\n  -o sqlite\n```\n\n## License\n\nMIT License\n\n## Author\n\n[Robert Tulke/rtulke]\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frtulke%2Fproxyreaper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frtulke%2Fproxyreaper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frtulke%2Fproxyreaper/lists"}