{"id":13539392,"url":"https://github.com/constverum/proxybroker","last_synced_at":"2025-05-14T00:04:58.543Z","repository":{"id":38393090,"uuid":"44057019","full_name":"constverum/ProxyBroker","owner":"constverum","description":"Proxy [Finder | Checker | Server]. HTTP(S) \u0026 SOCKS :performing_arts:","archived":false,"fork":false,"pushed_at":"2024-03-18T18:41:12.000Z","size":7608,"stargazers_count":3949,"open_issues_count":103,"forks_count":1118,"subscribers_count":140,"default_branch":"master","last_synced_at":"2025-04-10T02:15:56.698Z","etag":null,"topics":["anonymity","anonymous","crawler","http-proxy","privacy","proxies","proxy","proxy-checker","proxy-list","proxy-server","proxypool","socks"],"latest_commit_sha":null,"homepage":"http://proxybroker.readthedocs.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/constverum.png","metadata":{"files":{"readme":"README.rst","changelog":"CHANGELOG.rst","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2015-10-11T15:13:37.000Z","updated_at":"2025-04-10T00:54:50.000Z","dependencies_parsed_at":"2023-01-31T18:45:26.336Z","dependency_job_id":"09e51cd3-583a-4e02-aba8-7de108403327","html_url":"https://github.com/constverum/ProxyBroker","commit_stats":{"total_commits":142,"total_committers":10,"mean_commits":14.2,"dds":0.5070422535211268,"last_synced_commit":"d21aae8575fc3a95493233ecfd2c7cf47b36b069"},"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/constverum%2FProxyBroker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/constverum%2FProxyBroker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/constverum%2FProxyBroker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/constverum%2FProxyBroker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/constverum","download_url":"https://codeload.github.com/constverum/ProxyBroker/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254043325,"owners_count":22004926,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anonymity","anonymous","crawler","http-proxy","privacy","proxies","proxy","proxy-checker","proxy-list","proxy-server","proxypool","socks"],"created_at":"2024-08-01T09:01:24.960Z","updated_at":"2025-05-14T00:04:58.492Z","avatar_url":"https://github.com/constverum.png","language":"Python","funding_links":[],"categories":["\u003ca id=\"1a9934198e37d6d06b881705b863afc8\"\u003e\u003c/a\u003e通信\u0026\u0026代理\u0026\u0026反向代理\u0026\u0026隧道","\u003ca id=\"d03d494700077f6a65092985c06bf8e8\"\u003e\u003c/a\u003e工具"],"sub_categories":["\u003ca id=\"56acb7c49c828d4715dce57410d490d1\"\u003e\u003c/a\u003e未分类-Proxy","\u003ca id=\"57b8e953d394bbed52df2a6976d98dfa\"\u003e\u003c/a\u003eSocks"],"readme":"ProxyBroker\n===========\n\n.. image:: https://img.shields.io/pypi/v/proxybroker.svg?style=flat-square\n    :target: https://pypi.python.org/pypi/proxybroker/\n.. image:: https://img.shields.io/travis/constverum/ProxyBroker.svg?style=flat-square\n    :target: https://travis-ci.org/constverum/ProxyBroker\n.. image:: https://img.shields.io/pypi/wheel/proxybroker.svg?style=flat-square\n    :target: https://pypi.python.org/pypi/proxybroker/\n.. image:: https://img.shields.io/pypi/pyversions/proxybroker.svg?style=flat-square\n    :target: https://pypi.python.org/pypi/proxybroker/\n.. image:: https://img.shields.io/pypi/l/proxybroker.svg?style=flat-square\n    :target: https://pypi.python.org/pypi/proxybroker/\n\nProxyBroker is an open source tool that asynchronously finds public proxies from multiple sources and concurrently checks them.\n\n.. image:: https://raw.githubusercontent.com/constverum/ProxyBroker/master/docs/source/_static/index_find_example.gif\n\n\nFeatures\n--------\n\n* Finds more than 7000 working proxies from ~50 sources.\n* Support protocols: HTTP(S), SOCKS4/5. Also CONNECT method to ports 80 and 23 (SMTP).\n* Proxies may be filtered by type, anonymity level, response time, country and status in DNSBL.\n* Work as a proxy server that distributes incoming requests to external proxies. With automatic proxy rotation.\n* All proxies are checked to support Cookies and Referer (and POST requests if required).\n* Automatically removes duplicate proxies.\n* Is asynchronous.\n\n.. * Save found proxies to a file in custom format.\n\nRequirements\n------------\n\n* Python **3.5** or higher\n* `aiohttp \u003chttps://pypi.python.org/pypi/aiohttp\u003e`_\n* `aiodns \u003chttps://pypi.python.org/pypi/aiodns\u003e`_\n* `maxminddb \u003chttps://pypi.python.org/pypi/maxminddb\u003e`_\n\n\nInstallation\n------------\n\nTo install last stable release from pypi:\n\n.. code-block:: bash\n\n    $ pip install proxybroker\n\nThe latest development version can be installed directly from GitHub:\n\n.. code-block:: bash\n\n    $ pip install -U git+https://github.com/constverum/ProxyBroker.git\n\n\nUsage\n-----\n\n\nCLI Examples\n~~~~~~~~~~~~\n\n\nFind\n\"\"\"\"\n\nFind and show 10 HTTP(S) proxies from United States with the high level of anonymity:\n\n.. code-block:: bash\n\n    $ proxybroker find --types HTTP HTTPS --lvl High --countries US --strict -l 10\n\n.. image:: https://raw.githubusercontent.com/constverum/ProxyBroker/master/docs/source/_static/cli_find_example.gif\n\n\nGrab\n\"\"\"\"\n\nFind and save to a file 10 US proxies (without a check):\n\n.. code-block:: bash\n\n    $ proxybroker grab --countries US --limit 10 --outfile ./proxies.txt\n\n.. image:: https://raw.githubusercontent.com/constverum/ProxyBroker/master/docs/source/_static/cli_grab_example.gif\n\n\nServe\n\"\"\"\"\"\n\nRun a local proxy server that distributes incoming requests to a pool\nof found HTTP(S) proxies with the high level of anonymity:\n\n.. code-block:: bash\n\n    $ proxybroker serve --host 127.0.0.1 --port 8888 --types HTTP HTTPS --lvl High\n\n\n.. image:: https://raw.githubusercontent.com/constverum/ProxyBroker/master/docs/source/_static/cli_serve_example.gif\n\n| Run ``proxybroker --help`` for more information on the options available.\n| Run ``proxybroker \u003ccommand\u003e --help`` for more information on a command.\n\n\nBasic code example\n~~~~~~~~~~~~~~~~~~\n\nFind and show 10 working HTTP(S) proxies:\n\n.. code-block:: python\n\n    import asyncio\n    from proxybroker import Broker\n\n    async def show(proxies):\n        while True:\n            proxy = await proxies.get()\n            if proxy is None: break\n            print('Found proxy: %s' % proxy)\n\n    proxies = asyncio.Queue()\n    broker = Broker(proxies)\n    tasks = asyncio.gather(\n        broker.find(types=['HTTP', 'HTTPS'], limit=10),\n        show(proxies))\n\n    loop = asyncio.get_event_loop()\n    loop.run_until_complete(tasks)\n\n`More examples \u003chttps://proxybroker.readthedocs.io/en/latest/examples.html\u003e`_.\n\n\nDocumentation\n-------------\n\nhttps://proxybroker.readthedocs.io/\n\n\nTODO\n----\n\n* Check the ping, response time and speed of data transfer\n* Check site access (Google, Twitter, etc) and even your own custom URL's\n* Information about uptime\n* Checksum of data returned\n* Support for proxy authentication\n* Finding outgoing IP for cascading proxy\n* The ability to specify the address of the proxy without port (try to connect on defaulted ports)\n\n\nContributing\n------------\n\n* Fork it: https://github.com/constverum/ProxyBroker/fork\n* Create your feature branch: ``git checkout -b my-new-feature``\n* Commit your changes: ``git commit -am 'Add some feature'``\n* Push to the branch: ``git push origin my-new-feature``\n* Submit a pull request!\n\n\nLicense\n-------\n\nLicensed under the Apache License, Version 2.0\n\n*This product includes GeoLite2 data created by MaxMind, available from* `http://www.maxmind.com \u003chttp://www.maxmind.com\u003e`_.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fconstverum%2Fproxybroker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fconstverum%2Fproxybroker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fconstverum%2Fproxybroker/lists"}