{"id":24866617,"url":"https://github.com/truongbo17/proxy-rotator","last_synced_at":"2025-08-01T20:04:18.316Z","repository":{"id":62598102,"uuid":"560936790","full_name":"truongbo17/proxy-rotator","owner":"truongbo17","description":"Proxy Rotator with Load Balancing Strategies","archived":false,"fork":false,"pushed_at":"2023-01-05T06:14:28.000Z","size":17200,"stargazers_count":8,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-09T05:13:47.866Z","etag":null,"topics":["crawl","guzzle","load-balancing","proxy","proxy-rotation","proxy-rotator","round-robin","strategy"],"latest_commit_sha":null,"homepage":"","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/truongbo17.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-11-02T15:31:45.000Z","updated_at":"2024-11-22T10:13:28.000Z","dependencies_parsed_at":"2023-02-03T15:15:29.175Z","dependency_job_id":null,"html_url":"https://github.com/truongbo17/proxy-rotator","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/truongbo17%2Fproxy-rotator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/truongbo17%2Fproxy-rotator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/truongbo17%2Fproxy-rotator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/truongbo17%2Fproxy-rotator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/truongbo17","download_url":"https://codeload.github.com/truongbo17/proxy-rotator/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":236608474,"owners_count":19176458,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crawl","guzzle","load-balancing","proxy","proxy-rotation","proxy-rotator","round-robin","strategy"],"created_at":"2025-02-01T01:42:53.822Z","updated_at":"2025-02-01T01:42:54.327Z","avatar_url":"https://github.com/truongbo17.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Proxy Rotation with Laravel Guzzle\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./img.png\"  alt=\"Proxy Rotation Logo\"/\u003e\n\u003c/p\u003e\n\n### Description\n* Implement Nginx simple load balancing strategies to build a Package Proxy Rotation to use with Guzzle or anymore...\n* Use only effective, consistent and simple strategies to rotate proxies\n------\n### Optional Features\n- [Installation](#installation)\n- [Quick Use](#quick-use)\n- [Sort Nodes](#sort-nodes)\n- [Max Use](#max-use)\n\n### Strategy\n- [Random](#random)\n- [Frequency](#frequency)\n- [Round Robin](#round-robin)\n- [Weighted Round Robin](#weighted-round-robin)\n- [Multiple Dynamic](#multiple-dynamic)\n\n### Multiple Endpoint\nAutomatically retry connect and switch between endpoint \n- [Introduction](MultipleEndpoint.md)\n\n## Next feature\n- ~~Use backup hosts (if the main host is not accessible, Guzzle will automatically connect to the backup hosts to get data...)~~ ✔️\n- ~~Configure strategies to run separately with hosts, smarter~~ ✔️\n- ~~Automatically retry connecting hosts when it fails to connect (configure number of retries, response code types)~~ ✔️\n- ~~Load balancing, apply separate strategies to each Cluster~~ ❌\n- Anymore...\n\n\n-------------------\n## Installation\nInstall the package:\n\n```php\ncomposer require truongbo/proxy-rotation\n```\nRun test:\n```php\n./vendor/bin/phpunit\n```\n\nPublish package files (config):\n\n```php\nphp artisan vendor:publish --provider=\"TruongBo\\ProxyRotation\\ProxyRotationServiceProvider\"\n```\n\n-------------------\n## Quick Use\n\nYou need to choose a rotation strategy and configure it with Rotation:\n```php\nuse TruongBo\\ProxyRotation\\Rotation;\nuse TruongBo\\ProxyRotation\\Strategy\\RoundRobin;\n\n$rotation = new Rotation(new RoundRobin(counter: 0));\n```\n\nInitialize a cluster consisting of proxy nodes:\n```php\nuse TruongBo\\ProxyRotation\\ProxyServer\\ProxyCluster;\nuse TruongBo\\ProxyRotation\\ProxyServer\\ProxyNode;\n\n$proxy_cluster = new ProxyCluster(\n        cluster_name: 'cluster1', \n        array_proxy_node: [\n            new ProxyNode(name: 'proxy-node1'),\n            new ProxyNode(name: 'proxy-node2'),\n            new ProxyNode(name: 'proxy-node3'),\n            new ProxyNode(name: 'proxy-node4'),\n        ]);\n```\n\nThen, we should connect ProxyMiddleware to Guzzle for the proxy balancing to work:\n```php\nuse GuzzleHttp\\HandlerStack;\nuse GuzzleHttp\\Client;\nuse TruongBo\\ProxyRotation\\Middleware\\ProxyMiddleware;\n\n$stack = HandlerStack::create();\n$stack-\u003epush(new ProxyMiddleware(rotation: $rotation,proxy_cluster: $proxy_cluster));\n\n$client = new Client([\n    'handler' =\u003e $stack,\n]);\n```\n\nFinished, check it out now:\n```php\nwhile (true) {\n    /** @var ResponseInterface $response */\n    $response = $client-\u003eget('https://httpbin.org/ip');\n\n    // ...\n}\n```\nResult: (With proxy-node as your proxy address)\n```text\n+-------------+\n| proxy-node1 |\n| proxy-node2 |\n| proxy-node3 |\n| proxy-node4 |\n| proxy-node1 |\n| proxy-node2 |\n| proxy-node3 |\n| proxy-node4 |\n| etc...      |\n+-------------+\n```\n\n-------------------\n## Sort Nodes\nSort to adjust the order of nodes in ascending or descending order:\n```php\nuse TruongBo\\ProxyRotation\\ProxyServer\\ProxyCluster;\nuse TruongBo\\ProxyRotation\\ProxyServer\\ProxyNode;\n\n$proxy_cluster = new ProxyCluster(\n        cluster_name: 'cluster1', \n        array_proxy_node: [\n            new ProxyNode(name: 'proxy-node1',weight: 1000),\n            new ProxyNode(name: 'proxy-node2',weight: 20),\n            new ProxyNode(name: 'proxy-node3',weight: 200),\n            new ProxyNode(name: 'proxy-node4',weight: 10),\n        ]);\n        \n$proxy_cluster-\u003esort(\"desc\");\n```\nTop proxy nodes will be used more and vice versa, for example:\n```text\n+-------------+-------------+\n| name        | weight      |\n+-------------+-------------+\n| proxy-node1 | 1000        |\n| proxy-node3 | 200         |\n| proxy-node2 | 20          |\n| proxy-node4 | 10          |\n+-------------+-------------+\n```\n\nSorting nodes can help you use the [Frequency](#frequency) strategy better.\nUse asc for reverse sort.\n\n-------------------\n\n## Max Use\n\n* Store the proxy usage count in the cache and count it. If it is used more than allowed, the proxy will be temporarily idle for the configured time period.\n* Use only with Random , Frequency and RoundRobin strategies\n\nExample: \n```php\n$cluster = new ProxyCluster('cluster1', [\n            new ProxyNode(name: 'node1',weight: 3,max_use: 4,max_use_wait: 10),\n            new ProxyNode(name: 'node2',weight: 1,max_use: 2,max_use_wait: 20),\n        ]);\n```\nEverything else runs automatically\n\n------------------\n# Random\n- $input_random : How are proxies random?\n```txt\n- both : All proxy nodes are random\n- has_weight : Only the weighted proxy node will be random\n- no_weight : Only proxy nodes without weights can be random\n```\nConfig:\n```php\nuse TruongBo\\ProxyRotation\\Rotation;\nuse TruongBo\\ProxyRotation\\Strategy\\Random;\nuse TruongBo\\ProxyRotation\\ProxyServer\\ProxyCluster;\nuse TruongBo\\ProxyRotation\\ProxyServer\\ProxyNode;\n\n$rotation = new Rotation(new Random(input_random: \"both\"));\n\n$proxy_cluster = new ProxyCluster(\n        cluster_name: 'cluster1', \n        array_proxy_node: [\n            new ProxyNode(name: 'proxy-node1',weight: 20),\n            new ProxyNode(name: 'proxy-node2'),\n            new ProxyNode(name: 'proxy-node3'),\n            new ProxyNode(name: 'proxy-node4',weight: 100),\n        ]);\n```\nOutput(both): \n```text\n+-------------+-------------+\n| name        | weight      |\n+-------------+-------------+\n| proxy-node3 | 0           |\n| proxy-node1 | 20          |\n| proxy-node2 | 0           |\n| proxy-node4 | 100         |\n+-------------+-------------+\n```\n----------\n\n## Frequency\n\nMore efficient using sort node\n```php\nuse TruongBo\\ProxyRotation\\ProxyServer\\ProxyCluster;\nuse TruongBo\\ProxyRotation\\ProxyServer\\ProxyNode;\nuse TruongBo\\ProxyRotation\\Strategy\\Frequency;\n\n$rotation = new Rotation(new Frequency(frequency: 0.8, depth: 0.2));\n\n$proxy_cluster = new ProxyCluster(\n        cluster_name: 'cluster1', \n        array_proxy_node: [\n            new ProxyNode(name: 'proxy-node1',weight: 2048),\n            new ProxyNode(name: 'proxy-node2',weight: 1024),\n            new ProxyNode(name: 'proxy-node3',weight: 512),\n            new ProxyNode(name: 'proxy-node4',weight: 256),\n            new ProxyNode(name: 'proxy-node5',weight: 256),\n            new ProxyNode(name: 'proxy-node6',weight: 64),\n            new ProxyNode(name: 'proxy-node7',weight: 32),\n            new ProxyNode(name: 'proxy-node8',weight: 16),\n            new ProxyNode(name: 'proxy-node9',weight: 8),\n            new ProxyNode(name: 'proxy-node10',weight: 4),\n        ]);\n        \n$cluster-\u003esort(\"desc\");\n```\nThe probability of choosing nodes for Frequency can be visualized as follows::\n```text\n+--------------+--------+\n| nodes        | chance |\n+--------------+--------+\n| proxy-node1  | 40%    |\n| proxy-node2  | 40%    |\n+--------------+--------+\n| proxy-node3  | 2.5%   |\n| proxy-node8  | 2.5%   |\n| proxy-node5  | 2.5%   |\n| proxy-node6  | 2.5%   |\n| proxy-node4  | 2.5%   |\n| proxy-node7  | 2.5%   |\n| proxy-node9  | 2.5%   |\n| proxy-node10 | 2.5%   |\n+-----------+-----------+\n```\n\n-------------\n\n## Round Robin\n\nThe proxies will be rotated in turn ($counter : start counting from somewhere)\n```php\nuse TruongBo\\ProxyRotation\\Rotation;\nuse TruongBo\\ProxyRotation\\Strategy\\RoundRobin;\nuse TruongBo\\ProxyRotation\\ProxyServer\\ProxyCluster;\nuse TruongBo\\ProxyRotation\\ProxyServer\\ProxyNode;\n\n$rotation = new Rotation(new RoundRobin(counter: 0));\n\n$proxy_cluster = new ProxyCluster(\n        cluster_name: 'cluster1', \n        array_proxy_node: [\n            new ProxyNode(name: 'proxy-node1'),\n            new ProxyNode(name: 'proxy-node2'),\n            new ProxyNode(name: 'proxy-node3'),\n            new ProxyNode(name: 'proxy-node4'),\n        ]);\n```\nOutput: \n```text\n+-------------+\n| proxy-node1 |\n| proxy-node2 |\n| proxy-node3 |\n| proxy-node4 |\n| proxy-node1 |\n| proxy-node2 |\n| proxy-node3 |\n| proxy-node4 |\n| etc...      |\n+-------------+\n```\n* You can interfere with proxy usage for a certain period of time if the proxy is restricted from use.Using [Max Use](#max-use)\n--------------------\n## Weighted Round Robin\n\nThe number of times this proxy node is called is the weight parameter passed in the initialization of the ProxyNode\n($counter : start counting from somewhere)\n```php\nuse TruongBo\\ProxyRotation\\Rotation;\nuse TruongBo\\ProxyRotation\\Strategy\\WeightedRoundRobin;\nuse TruongBo\\ProxyRotation\\ProxyServer\\ProxyCluster;\nuse TruongBo\\ProxyRotation\\ProxyServer\\ProxyNode;\n\n$rotation = new Rotation(new WeightedRoundRobin(counter: 0));\n\n$proxy_cluster = new ProxyCluster(\n        cluster_name: 'cluster1', \n        array_proxy_node: [\n            new ProxyNode(name: 'proxy-node1', weight: 3),\n            new ProxyNode(name: 'proxy-node2'),\n            new ProxyNode(name: 'proxy-node3', weight: 1),\n            new ProxyNode(name: 'proxy-node4', weight: 1),\n        ]);\n```\nOutput:\n```text\n+-------------+\n| proxy-node1 |\n| proxy-node1 |\n| proxy-node1 |\n| proxy-node3 |\n| proxy-node4 |\n| etc...      |\n+-------------+\n```\n* Proxy Node without weight will not be used\n\n-------------------\n\n## Multiple Dynamic\n\nDynamically change strategies according to the passed callable condition (Absolutely do not use if you do not know about it)\n\n```php\nuse TruongBo\\ProxyRotation\\Rotation;\nuse TruongBo\\ProxyRotation\\Strategy\\WeightedRoundRobin;\nuse TruongBo\\ProxyRotation\\ProxyServer\\ProxyCluster;\nuse TruongBo\\ProxyRotation\\ProxyServer\\ProxyNode;\n\n$rotation = new Rotation(new MultipleDynamic(\n            new RoundRobin(counter: 0),\n            new Random(input_random: \"has_weight\"),\n            new WeightedRoundRobin(counter: 0),\n        ));\n\n$proxy_cluster = new ProxyCluster(\n        cluster_name: 'cluster1', \n        array_proxy_node: [\n            new ProxyNode(name: 'proxy-node1', weight: 3),\n            new ProxyNode(name: 'proxy-node2'),\n            new ProxyNode(name: 'proxy-node3', weight: 1),\n            new ProxyNode(name: 'proxy-node4', weight: 1),\n        ]);\n\n        while (true) {\n            $node = $rotation-\u003epick($cluster, function (ProxyNode $proxy_node){\n                //condition switch between strategies in here.\n            });\n\n            echo $node?-\u003ename;\n        }\n```\n-------------------\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftruongbo17%2Fproxy-rotator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftruongbo17%2Fproxy-rotator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftruongbo17%2Fproxy-rotator/lists"}