{"id":31580805,"url":"https://github.com/forcefledgling/proxyhub","last_synced_at":"2025-10-05T21:20:24.745Z","repository":{"id":205610368,"uuid":"714641391","full_name":"ForceFledgling/proxyhub","owner":"ForceFledgling","description":"An advanced [Finder | Checker | Server] tool for proxy servers, supporting both HTTP(S) and SOCKS protocols. 🎭","archived":false,"fork":false,"pushed_at":"2023-11-12T09:54:04.000Z","size":1819,"stargazers_count":209,"open_issues_count":2,"forks_count":16,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-09-29T03:27:29.852Z","etag":null,"topics":["anonymity","anonymous","crawler","free-proxy","free-proxy-list","http-proxy","privacy","proxies","proxy","proxy-checker","proxy-grabber","proxy-list","proxy-scraper","proxy-scrapper","proxy-server","proxy-tool","proxypool","socks","socks4","socks5"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ForceFledgling.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-11-05T13:28:57.000Z","updated_at":"2025-09-18T07:41:57.000Z","dependencies_parsed_at":"2023-11-10T17:29:56.360Z","dependency_job_id":null,"html_url":"https://github.com/ForceFledgling/proxyhub","commit_stats":null,"previous_names":["forcefledgling/proxyhub"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/ForceFledgling/proxyhub","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ForceFledgling%2Fproxyhub","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ForceFledgling%2Fproxyhub/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ForceFledgling%2Fproxyhub/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ForceFledgling%2Fproxyhub/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ForceFledgling","download_url":"https://codeload.github.com/ForceFledgling/proxyhub/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ForceFledgling%2Fproxyhub/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278522090,"owners_count":26000787,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-05T02:00:06.059Z","response_time":54,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anonymity","anonymous","crawler","free-proxy","free-proxy-list","http-proxy","privacy","proxies","proxy","proxy-checker","proxy-grabber","proxy-list","proxy-scraper","proxy-scrapper","proxy-server","proxy-tool","proxypool","socks","socks4","socks5"],"created_at":"2025-10-05T21:20:22.517Z","updated_at":"2025-10-05T21:20:24.739Z","avatar_url":"https://github.com/ForceFledgling.png","language":"Python","readme":"*Porting to Python3.10+ is painful and the progress is moving slowly.*  \n*We need more volunteers to join. PRs welcome! :joy:*\n\nProxyHub\n===========\n\n![logo](https://github.com/ForceFledgling/proxyhub/blob/main/docs/img/logo_transparent.png)\n\nProxyHub is an open source tool that asynchronously finds public proxies from multiple sources and concurrently checks them.\n\n[![Documentation](https://img.shields.io/badge/documentation-yes-brightgreen.svg)](docs)\n[![License: Apache License 2.0](https://img.shields.io/badge/License-Apache2-brightgreen.svg)](https://choosealicense.com/licenses/apache-2.0/)\n[![pypi Version](https://img.shields.io/pypi/v/proxyhub.svg?style=flat-square\u0026logo=proxyhub\u0026logoColor=white)](https://pypi.org/project/proxyhub/)\n[![PyPi downloads](https://static.pepy.tech/personalized-badge/proxyhub?period=total\u0026units=international_system\u0026left_color=grey\u0026right_color=orange\u0026left_text=pip%20downloads)](https://pypi.org/project/proxyhub/)\n\nFeatures 🌟\n--------\n\n-   Finds more than 7000 working proxies from \\~50 sources.\n-   Support protocols: HTTP(S), SOCKS4/5. Also CONNECT method to ports 80 and 23 (SMTP).\n-   Proxies may be filtered by type, anonymity level, response time, country and status in DNSBL.\n-   Work as a proxy server that distributes incoming requests to external proxies. With automatic proxy rotation.\n-   All proxies are checked to support Cookies and Referer (and POST requests if required).\n-   Automatically removes duplicate proxies.\n-   Is asynchronous.\n\nRequirements 📋\n------------\n\n-   Python 3.8+\n-   [aiohttp](https://pypi.python.org/pypi/aiohttp)\n-   [aiodns](https://pypi.python.org/pypi/aiodns)\n-   [maxminddb](https://pypi.python.org/pypi/maxminddb)\n\nInstallation 🚀\n------------\n\n### Install locally\n\nTo install last stable release from pypi:\n\n``` {.sourceCode .bash}\n$ pip install proxyhub\n```\n\nTo install the latest development version from GitHub:\n\n``` {.sourceCode .bash}\n$ pip install -U git+https://github.com/ForceFledgling/proxyhub.git\n```\n\n### Use pre-built Docker image\n\n``` {.sourceCode .bash}\n$ docker pull ForceFledgling/proxyhub\n```\n\n### Build bundled one-file executable with pyinstaller\n\n#### Requirements\nSupported Operating System: Windows, Linux, MacOS\n\n*On UNIX-like systems (Linux / macOSX / BSD)*\n\nInstall these tools\n - upx\n - objdump (this tool is usually in the binutils package)\n``` {.sourceCode .bash}\n$ sudo apt install -y upx-ucl binutils # On Ubuntu / Debian\n```\n\n#### Build\n\n```\npip install pyinstaller \\\n\u0026\u0026 pip install . \\\n\u0026\u0026 mkdir -p build \\\n\u0026\u0026 cd build \\\n\u0026\u0026 pyinstaller --onefile --name proxyhub --add-data \"../proxyhub/data:data\" --workpath ./tmp --distpath . --clean ../py2exe_entrypoint.py \\\n\u0026\u0026 rm -rf tmp *.spec\n```\n\nThe executable is now in the build directory\n\nUsage 💡\n-----\n\n### CLI Examples\n\n#### Find\n\nFind and show 10 HTTP(S) proxies from United States with the high level of anonymity:\n\n``` {.sourceCode .bash}\n$ proxyhub find --types HTTP HTTPS --lvl High --countries US --strict -l 10\n```\n\n#### Grab\n\nFind and save to a file 10 US proxies (without a check):\n\n``` {.sourceCode .bash}\n$ proxyhub grab --countries US --limit 10 --outfile ./proxies.txt\n```\n\n#### Serve\n\nRun a local proxy server that distributes incoming requests to a pool of found HTTP(S) proxies with the high level of anonymity:\n\n``` {.sourceCode .bash}\n$ proxyhub serve --host 127.0.0.1 --port 8888 --types HTTP HTTPS --lvl High --min-queue 5\n```\n\nRun `proxyhub --help` for more information on the options available.\nRun `proxyhub \u003ccommand\u003e --help` for more information on a command.\n\n### Basic code example\n\nFind and show 10 working HTTP(S) proxies:\n\n``` {.sourceCode .python}\nimport asyncio\nfrom proxyhub import Broker\n\nasync def show(proxies):\n    while True:\n        proxy = await proxies.get()\n        if proxy is None: break\n        print('Found proxy: %s' % proxy)\n\nproxies = asyncio.Queue()\nbroker = Broker(proxies)\ntasks = asyncio.gather(\n    broker.find(types=['HTTP', 'HTTPS'], limit=10),\n    show(proxies))\n\nloop = asyncio.get_event_loop()\nloop.run_until_complete(tasks)\n```\n\n[More examples](https://proxyhub.readthedocs.io/en/latest/examples.html).\n\n### Proxy information per requests\n#### HTTP\nCheck `X-Proxy-Info` header in response.\n```\n$ http_proxy=http://127.0.0.1:8888 https_proxy=http://127.0.0.1:8888 curl -v http://httpbin.org/get\n*   Trying 127.0.0.1...\n* TCP_NODELAY set\n* Connected to 127.0.0.1 (127.0.0.1) port 8888 (#0)\n\u003e GET http://httpbin.org/get HTTP/1.1\n\u003e Host: httpbin.org\n\u003e User-Agent: curl/7.58.0\n\u003e Accept: */*\n\u003e Proxy-Connection: Keep-Alive\n\u003e\n\u003c HTTP/1.1 200 OK\n\u003c X-Proxy-Info: 174.138.42.112:8080\n\u003c Date: Mon, 04 May 2020 03:39:40 GMT\n\u003c Content-Type: application/json\n\u003c Content-Length: 304\n\u003c Server: gunicorn/19.9.0\n\u003c Access-Control-Allow-Origin: *\n\u003c Access-Control-Allow-Credentials: true\n\u003c X-Cache: MISS from ADM-MANAGER\n\u003c X-Cache-Lookup: MISS from ADM-MANAGER:880\n\u003c Connection: keep-alive\n\u003c\n{\n  \"args\": {},\n  \"headers\": {\n    \"Accept\": \"*/*\",\n    \"Cache-Control\": \"max-age=259200\",\n    \"Host\": \"httpbin.org\",\n    \"User-Agent\": \"curl/7.58.0\",\n    \"X-Amzn-Trace-Id\": \"Root=1-5eaf8e7c-6a1162a1387a1743a49063f4\"\n  },\n  \"origin\": \"...\",\n  \"url\": \"http://httpbin.org/get\"\n}\n* Connection #0 to host 127.0.0.1 left intact\n```\n\n#### HTTPS\nWe are not able to modify HTTPS traffic to inject custom header once they start being encrypted. A `X-Proxy-Info` will be sent to client after `HTTP/1.1 200 Connection established` but not sure how clients can read it.\n```\n(env) username@host:~/workspace/proxyhub2$ http_proxy=http://127.0.0.1:8888 https_proxy=http://127.0.0.1:8888 curl -v https://httpbin.org/get\n*   Trying 127.0.0.1...\n* TCP_NODELAY set\n* Connected to 127.0.0.1 (127.0.0.1) port 8888 (#0)\n* allocate connect buffer!\n* Establish HTTP proxy tunnel to httpbin.org:443\n\u003e CONNECT httpbin.org:443 HTTP/1.1\n\u003e Host: httpbin.org:443\n\u003e User-Agent: curl/7.58.0\n\u003e Proxy-Connection: Keep-Alive\n\u003e\n\u003c HTTP/1.1 200 Connection established\n\u003c X-Proxy-Info: 207.148.22.139:8080\n\u003c\n* Proxy replied 200 to CONNECT request\n* CONNECT phase completed!\n* ALPN, offering h2\n* ALPN, offering http/1.1\n* successfully set certificate verify locations:\n...\n*  SSL certificate verify ok.\n* Using HTTP2, server supports multi-use\n* Connection state changed (HTTP/2 confirmed)\n* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0\n* Using Stream ID: 1 (easy handle 0x5560b2e93580)\n\u003e GET /get HTTP/2\n\u003e Host: httpbin.org\n\u003e User-Agent: curl/7.58.0\n\u003e Accept: */*\n\u003e\n* Connection state changed (MAX_CONCURRENT_STREAMS updated)!\n\u003c HTTP/2 200\n\u003c date: Mon, 04 May 2020 03:39:35 GMT\n\u003c content-type: application/json\n\u003c content-length: 256\n\u003c server: gunicorn/19.9.0\n\u003c access-control-allow-origin: *\n\u003c access-control-allow-credentials: true\n\u003c\n{\n  \"args\": {},\n  \"headers\": {\n    \"Accept\": \"*/*\",\n    \"Host\": \"httpbin.org\",\n    \"User-Agent\": \"curl/7.58.0\",\n    \"X-Amzn-Trace-Id\": \"Root=1-5eaf8e77-efcb353b0983ad6a90f8bdcd\"\n  },\n  \"origin\": \"...\",\n  \"url\": \"https://httpbin.org/get\"\n}\n* Connection #0 to host 127.0.0.1 left intact\n```\n\n### HTTP API\n#### Get info of proxy been used for retrieving specific url\nFor HTTP, it's easy.\n```\n$ http_proxy=http://127.0.0.1:8888 https_proxy=http://127.0.0.1:8888 curl -v http://proxycontrol/api/history/url:http://httpbin.org/get\n*   Trying 127.0.0.1...\n* TCP_NODELAY set\n* Connected to 127.0.0.1 (127.0.0.1) port 8888 (#0)\n\u003e GET http://proxycontrol/api/history/url:http://httpbin.org/get HTTP/1.1\n\u003e Host: proxycontrol\n\u003e User-Agent: curl/7.58.0\n\u003e Accept: */*\n\u003e Proxy-Connection: Keep-Alive\n\u003e\n\u003c HTTP/1.1 200 OK\n\u003c Content-Type: application/json\n\u003c Content-Length: 34\n\u003c Access-Control-Allow-Origin: *\n\u003c Access-Control-Allow-Credentials: true\n\u003c\n{\"proxy\": \"...\"}\n```\n\nFor HTTPS, we're not able to know encrypted payload (request), so only hostname can be used.\n```\n$ http_proxy=http://127.0.0.1:8888 https_proxy=http://127.0.0.1:8888 curl -v http://proxycontrol/api/history/url:httpbin.org:443\n*   Trying 127.0.0.1...\n* TCP_NODELAY set\n* Connected to 127.0.0.1 (127.0.0.1) port 8888 (#0)\n\u003e GET http://proxycontrol/api/history/url:httpbin.org:443 HTTP/1.1\n\u003e Host: proxycontrol\n\u003e User-Agent: curl/7.58.0\n\u003e Accept: */*\n\u003e Proxy-Connection: Keep-Alive\n\u003e\n\u003c HTTP/1.1 200 OK\n\u003c Content-Type: application/json\n\u003c Content-Length: 34\n\u003c Access-Control-Allow-Origin: *\n\u003c Access-Control-Allow-Credentials: true\n\u003c\n{\"proxy\": \"...\"}\n* Connection #0 to host 127.0.0.1 left intact\n```\n\n#### Remove specific proxy from queue\n```\n$ http_proxy=http://127.0.0.1:8888 https_proxy=http://127.0.0.1:8888 curl -v http://proxycontrol/api/remove/PROXY_IP:PROXY_PORT\n*   Trying 127.0.0.1...\n* TCP_NODELAY set\n* Connected to 127.0.0.1 (127.0.0.1) port 8888 (#0)\n\u003e GET http://proxycontrol/api/remove/... HTTP/1.1\n\u003e Host: proxycontrol\n\u003e User-Agent: curl/7.58.0\n\u003e Accept: */*\n\u003e Proxy-Connection: Keep-Alive\n\u003e\n\u003c HTTP/1.1 204 No Content\n\u003c\n* Connection #0 to host 127.0.0.1 left intact\n```\n\nTODO 🛠️\n----\n\n-   Check the ping, response time and speed of data transfer\n-   Check site access (Google, Twitter, etc) and even your own custom URL's\n-   Information about uptime\n-   Checksum of data returned\n-   Support for proxy authentication\n-   Finding outgoing IP for cascading proxy\n-   The ability to specify the address of the proxy without port (try to connect on defaulted ports)\n\nContributing 🤝\n------------\n\n-   Fork it: \u003chttps://github.com/ForceFledgling/proxyhub/fork\u003e\n-   Create your feature branch: `git checkout -b my-new-feature`\n-   We use [Poetry](https://python-poetry.org/) to manage dependencies. If need, install dependencies: `poetry install`\n-   Commit your changes: `git commit -am 'Add some feature'`\n-   Push to the branch: `git push origin my-new-feature`\n-   Submit a pull request!\n-   [Contributor workflow](https://github.com/ForceFledgling/proxyhub/issues/)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fforcefledgling%2Fproxyhub","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fforcefledgling%2Fproxyhub","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fforcefledgling%2Fproxyhub/lists"}