{"id":18450289,"url":"https://github.com/nand0san/proxy_find","last_synced_at":"2025-04-18T00:35:13.802Z","repository":{"id":254262638,"uuid":"845494586","full_name":"nand0san/proxy_find","owner":"nand0san","description":"Python scripts to find, extract, and test free HTTP proxies from online sources. Includes initial and advanced proxy validation.","archived":false,"fork":false,"pushed_at":"2024-10-10T09:46:12.000Z","size":13,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-15T23:12:51.001Z","etag":null,"topics":["hacking","proxy"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nand0san.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-21T11:09:19.000Z","updated_at":"2024-10-10T09:46:15.000Z","dependencies_parsed_at":null,"dependency_job_id":"046a06ef-5512-4b9e-87a4-1723d1b4109c","html_url":"https://github.com/nand0san/proxy_find","commit_stats":null,"previous_names":["nand0san/proxy_find"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nand0san%2Fproxy_find","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nand0san%2Fproxy_find/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nand0san%2Fproxy_find/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nand0san%2Fproxy_find/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nand0san","download_url":"https://codeload.github.com/nand0san/proxy_find/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249167448,"owners_count":21223506,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hacking","proxy"],"created_at":"2024-11-06T07:24:36.624Z","updated_at":"2025-04-15T23:13:13.815Z","avatar_url":"https://github.com/nand0san.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Proxy Finder and Tester\n\nThis repository contains two Python scripts for finding and testing HTTP proxies:\n\n1. `proxie_finder.py`: Extracts free proxies from a public list and tests alive proxies.\n2. `proxie_test.py`: Checks if a specific url can be reached through the found proxies. Useful to detect proxy blocking by WAF.\n\nRepository: [https://github.com/nand0san/proxy_find](https://github.com/nand0san/proxy_find)\n\n## Requirements\n\n- Python 3.7+\n- Required Python packages:\n  - requests\n  - beautifulsoup4\n  - aiohttp\n\n## Installation\n\n1. Clone this repository:\n   ```\n   git clone https://github.com/nand0san/proxy_find.git\n   cd proxy_find\n   ```\n\n2. Create a virtual environment and activate it:\n   ```\n   python -m venv venv\n   source venv/bin/activate  # On Windows use `venv\\Scripts\\activate`\n   ```\n\n3. Install the required packages:\n   ```\n   pip install requests beautifulsoup4 aiohttp\n   ```\n\n## Usage\n\n### Proxy Finder\n\nRun the proxy finder script to gather and initially test proxies:\n\n```\npython proxie_finder.py\n```\n\nThis script will:\n- Scrape free proxies from https://free-proxy-list.net/\n- Test if the proxies respond and allow web browsing.\n- Save working proxies to `proxies_working.csv`\n\n### Advanced Proxy Tester\n\nAfter running the finder, use the advanced tester to further verify the proxies. Specifically useful in case of testing proxy blocks by WAFs, firewalls, etc.\n\n1. For default tests:\n   ```\n   python proxie_test.py\n   ```\n\n2. For testing with a specific target URL:\n   ```\n   python proxie_test.py --target https://example.com\n   ```\n\nThis script will:\n- Read proxies from `proxies_working.csv`\n- Perform navigation test trough found proxies against the specified target URL.\n- Provide real-time feedback on the testing progress\n- Save verified proxies to `proxies_verified.csv`\n\n## Script Details\n\n### proxie_finder.py\n\n- Scrapes proxies from a free proxy list website\n- Tests each proxy asynchronously for basic connectivity\n- Saves proxies that pass initial tests to a CSV file\n- Includes information such as IP, port, country, anonymity level, and HTTPS support\n\n### proxie_test.py\n\n- Reads the proxies extracted by the first script\n- Performs more thorough tests, including:\n  - Checking multiple URLs (HTTP and HTTPS) or a specific target URL\n  - Validating responses\n  - Measuring response times\n- Provides real-time feedback on testing progress\n- Saves detailed results, including working status, detected IP, and average response time\n- Allows custom tests to be easily added\n\n## Output\n\nBoth scripts generate CSV files with the following information:\n\n- Host\n- Port\n- Country\n- Anonymity level\n- HTTPS support\n- Last checked time\n- Extraction date\n- Working status\n- Detected IP (may differ from the proxy's IP)\n- Average response time\n- Test results for each URL tested\n\n### Sample Output\n\n`proxies_working.csv`\n```\nhost,port,country,anonymity,https,last_checked,extraction_date,response_time,detected_ip\n20.111.54.16,8123,France,elite proxy,no,1 min ago,2024-08-22 11:33:14,0.35,20.111.54.16\n20.206.106.192,8123,Brazil,elite proxy,no,1 min ago,2024-08-22 11:33:14,0.53,20.206.106.192\n162.223.90.130,80,United States,elite proxy,no,1 min ago,2024-08-22 11:33:14,0.55,162.223.90.130\n...\n\n```\n\n`proxies_verified.csv`\n```\nhost,port,country,anonymity,https,last_checked,extraction_date,response_time,detected_ip,is_working,avg_response_time,test_http://httpbin.org/ip,test_https://api.ipify.org,test_https://www.google.com\n94.242.240.36,3128,Luxembourg,anonymous,yes,1 min ago,2024-08-22 11:33:14,0.80,94.242.240.36,Yes,0.38,Pass,Pass,Pass\n15.204.161.192,18080,United States,elite proxy,yes,20 mins ago,2024-08-22 11:33:14,1.44,77.111.247.46,Yes,0.52,Pass,Pass,Fail\n181.188.27.162,8080,Trinidad and Tobago,elite proxy,yes,20 mins ago,2024-08-22 11:33:14,1.62,181.188.27.162,Yes,0.59,Pass,Pass,Fail\n...\n```\n\n## Notes\n\n- The scripts use SSL verification disabling, which should be used cautiously in production environments.\n- Proxy availability and performance can change rapidly. Regular re-testing is recommended.\n- Using free proxies comes with risks. Use at your own discretion and avoid transmitting sensitive data.\n- The timeout for each proxy test is set to 5 seconds to balance between thoroughness and speed.\n\n## Contributing\n\nContributions, issues, and feature requests are welcome. Feel free to check [issues page](https://github.com/nand0san/proxy_find/issues) if you want to contribute.\n\n## License\n\nThis project is [MIT](https://choosealicense.com/licenses/mit/) licensed.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnand0san%2Fproxy_find","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnand0san%2Fproxy_find","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnand0san%2Fproxy_find/lists"}