{"id":13447595,"url":"https://github.com/Vladislao/proxy-supervisor","last_synced_at":"2025-03-22T01:31:07.584Z","repository":{"id":19869359,"uuid":"88137888","full_name":"Vladislao/proxy-supervisor","owner":"Vladislao","description":"Refresh, monitor and balance your proxies","archived":false,"fork":false,"pushed_at":"2024-04-27T07:42:13.000Z","size":722,"stargazers_count":15,"open_issues_count":10,"forks_count":7,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-10-09T21:34:17.095Z","etag":null,"topics":["proxies","proxy-api","proxy-balance","proxy-rotator","proxy-supervisor"],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Vladislao.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-04-13T07:33:33.000Z","updated_at":"2024-05-14T22:07:34.000Z","dependencies_parsed_at":"2024-04-27T08:30:47.980Z","dependency_job_id":"1112adb6-c32c-4c88-8e77-81719bc05362","html_url":"https://github.com/Vladislao/proxy-supervisor","commit_stats":{"total_commits":68,"total_committers":7,"mean_commits":9.714285714285714,"dds":"0.42647058823529416","last_synced_commit":"a0ff81b0bb1cf8cf8e8102bca7c72b25147992fa"},"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Vladislao%2Fproxy-supervisor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Vladislao%2Fproxy-supervisor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Vladislao%2Fproxy-supervisor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Vladislao%2Fproxy-supervisor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Vladislao","download_url":"https://codeload.github.com/Vladislao/proxy-supervisor/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244893417,"owners_count":20527585,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["proxies","proxy-api","proxy-balance","proxy-rotator","proxy-supervisor"],"created_at":"2024-07-31T05:01:21.960Z","updated_at":"2025-03-22T01:31:07.304Z","avatar_url":"https://github.com/Vladislao.png","language":"JavaScript","readme":"# proxy-supervisor\n\nRefresh, monitor and balance your proxies\n\n## Installation\n\n```bash\n$ npm install proxy-supervisor\n```\n\n## Features\n\n- Robust balancing\n- Monitoring, replenishment\n- HTTP, HTTPS, tunnels\n- Designed to support multiple proxy sources\n- High performance\n- High test coverage\n\n## How to\n\nFor a straightforward standalone proxy balancer accessible via command line, explore [proxy-supervisor-cli](https://github.com/vladislao/proxy-supervisor-cli) or [dockerized proxy-supervisor](https://hub.docker.com/r/vladislaosan/proxy-supervisor). \n\n## Usage\n\nStart by initializing a balancer and adding your proxies:\n\n```javascript\nconst http = require(\"http\");\nconst { balancer } = require(\"proxy-supervisor\");\n\nconst awesomeBalancer = balancer().add([\n  \"http://SOME_PROXY:38403\",\n  \"http://OTHER_PROXY:61637\"\n]);\n\n// Now, integrate it into your application. Below, we set up a basic HTTP server using the balancer as middleware.\n\nhttp\n  .createServer(awesomeBalancer.proxy())\n  .on(\"connect\", awesomeBalancer.connect())\n  .listen(3000);\n```\n\nGreat! The next step is to configure your balancing server as the proxy server in any application that needs to use proxies. This setup will channel requests through the specified proxies, forming a path like _(you) -\u003e (balancer) -\u003e (proxy) -\u003e (endpoint)_.\n\n#### Authentication\n\nIn scenarios where a proxy requires authorization, use the _formatHeaders_ function. This function enables you to embed proxy credentials in the URL (e.g., https://login:password@MY_PROXY:3123) and set the appropriate authorization header. Here's how to implement it:\n\n```javascript\nconst formatHeaders = (proxy, headers) =\u003e {\n  if (!proxy.url.auth) return headers;\n  return {\n    ...headers,\n    \"Auth-Proxy\":\n      \"Basic \" + Buffer.from(proxy.url.auth).toString(\"base64\"),\n  };\n};\n\nhttp\n  .createServer(balancer.proxy({ formatHeaders }))\n  .on(\"connect\", balancer.connect({ formatHeaders }))\n  .listen(3000);\n```\n\n## Design\n\n### Balancer\n\nA balancer is responsible for iterating over the list of proxies. Balancing across multiple proxy servers is a commonly used technique for minimizing the chance of blocking and increasing anonymity level.\n\nEach instance has its own list of proxies, which is controlled by sources. Balancer is not responsible for invalidating proxies.\n\n#### balancer.add(proxies)\n\n- **proxies** _\\\u003cArray\\\u003e | \\\u003cUrl\\\u003e | \\\u003cString\\\u003e_ List of proxy servers to be added.\n- Returns: _this_.\n\nAdds specified proxies to the list of the current balancer.\n\n#### balancer.remove(proxies)\n\n- **proxies** _\\\u003cArray\\\u003e | \\\u003cUrl\\\u003e | \\\u003cString\\\u003e_ List of proxy servers to be added.\n- Returns: _this_.\n\nRemoves specified proxies from the list of the current balancer.\n\n#### balancer.subscribe(source)\n\n- **source** _\\\u003cSource\\\u003e_ Source to listen.\n- Returns: _this_.\n\nSubscribes to the specified source.\n\n#### balancer.proxy([options])\n\n- **options** _\\\u003cObject\\\u003e_ Configuration details.\n\n  - **timeout** _\\\u003cInteger\\\u003e_ Sets the socket to timeout after timeout milliseconds of inactivity. Note that increasing the timeout beyond the OS-wide TCP connection timeout will not have any effect ([the default in Linux can be anywhere from 20-120 seconds](http://www.sekuda.com/overriding_the_default_linux_kernel_20_second_tcp_socket_connect_timeout)). Defaults to 30 seconds.\n \n  - **formatHeaders** _\\\u003cFunction\\\u003e_ This function is designed to modify headers before a request is sent through your proxy. It is commonly used for handling proxy authorization. The function signature is _(proxy, headers)_, and it must return an updated headers object.\n\n- Returns: _\\\u003cFunction\\\u003e_\n\nCreates a middleware function. Middleware has a signature of _(req, res, next)_. If _next_ function is provided, it will be called on response or error. Be aware that _res_ will be finished by then.\n\n#### balancer.connect([options])\n\n- **options** _\\\u003cObject\\\u003e_ Configuration details.\n\n  - **timeout** _\\\u003cInteger\\\u003e_ Sets the socket to timeout after timeout milliseconds of inactivity. Defaults to 30 seconds.\n\n  - **formatHeaders** _\\\u003cFunction\\\u003e_ This function is designed to modify headers before a request is sent through your proxy. It is commonly used for handling proxy authorization. The function signature is _(proxy, headers)_, and it must return an updated headers object.\n\n- Returns: _\\\u003cFunction\\\u003e_\n\nCreates a handler for HTTP CONNECT method. [It is used to open a tunnel between client and proxy server](https://tools.ietf.org/html/rfc2817#section-5.2).\n\n#### balancer.onNext(callback)\n\n- **callback** _\\\u003cFunction\\\u003e_ Callback function that returns a next proxy to be used.\n\nYou can specify your own balancing algorithm. Callback has a signature of _(proxies, url, req)_ and should return a single _\\\u003cUrl\\\u003e_ from a list.\n\n#### balancer.onAdd(callback)\n\n- **callback** _\\\u003cFunction\\\u003e_ Callback function that returns a new proxy.\n\nCallback will be called each time a new proxy is added to the list. Callback has a signature of _(proxy)_ and should return _\\\u003cObject\\\u003e_. A good place to set default parameters for a new proxy.\n\n#### balancer.onResponse(callback)\n\n- **callback** _\\\u003cFunction\\\u003e_ Callback function that handles response statuses.\n\nCallback has a signature of _(proxy, url, res, req)_ and will be called each time a request is completed. State of the proxy can be modified.\n\n#### balancer.onError(callback)\n\n- **callback** _\\\u003cFunction\\\u003e_ Callback function that handles request errors.\n\nCallback has a signature of _(proxy, url, err, req)_ and will be called each time a request resulted in an error. State of the proxy can be modified.\n\n### Source\n\nShould be used to modify the list of proxies for its listeners. The most common use case - collecting proxies from some site\nand adding them to listeners.\n\n#### source.addListener(listener)\n\n- **listener** _\\\u003cBalancer\\\u003e_ A balancer which will be added to the list of listeners.\n- Returns: _this_\n\nThis method simply attaches a balancer to the source.\n\n#### source.proxies()\n\n- Returns: _\\\u003cArray\\\u003e_ Returns list of unique proxy urls.\n\nHelper function to retrieve the list of proxies from all listeners. Proxies are unique across the array and represented as [_\\\u003cUrl\\\u003e_](https://nodejs.org/api/url.html#url_url_strings_and_url_objects).\n\n### Monitor\n\nParticular case of the _Source_. A monitor is responsible for filtering dead and slow proxies out from balancers.\n\n#### new Monitor([options])\n\n- **options** _\\\u003cObject\\\u003e_ Set of configurable options to set on the monitor. Can have the following fields:\n  - **target** _\\\u003cString\\\u003e_ specify path for the request to be done via proxies.\n  - **timeout** _\\\u003cInteger\\\u003e_ Sets the socket to timeout after timeout milliseconds of inactivity. Defaults to 3 seconds.\n  - **interval** _\\\u003cInteger\\\u003e_ Specifies how much time should pass after the last check is completed. Defaults to 5 minutes.\n\nMonitor is started automatically on creation, and will trigger for the first time after the specified **interval** is passed.\n\n#### monitor.start()\n\nStarts a monitor. Use only in case you have stopped monitor manually. Monitor is started automatically on the creation and can work with an empty list of listeners.\n\n#### monitor.stop()\n\nStops a monitor. It will clear current timer, but already running check will be not affected.\n\n#### monitor.check()\n\n- Returns: _\\\u003cPromise\\\u003e_ A promise, which resolves into an array of dead proxies. Those proxies are already removed from listeners.\n\nValidates proxies. This method will create parallel requests to the target location for each proxy. Timed out, unreachable or blocked proxies will be removed from all listeners. By default, valid status codes are _200, 201, 202_.\n\n#### monitor.onResponse(callback)\n\nYou can specify your own handler for proxies. Callback should have a signature of _(err, proxy, res, body)_ and return _true_ for valid proxy and _false_ otherwise.\n\n## Example\n\nTo run the example, clone this repo and install its dependencies:\n\n```bash\n$ git clone git@github.com:Vladislao/proxy-supervisor.git\n$ cd proxy-supervisor\n```\n\nDon't forget to modify your proxy.txt file. Grab any free proxies you can find.\n\nThen run the example with:\n\n```bash\n$ node example\n```\n\nHere is a simple curl command to check your proxy server:\n\n```bash\n$ curl http://google.com -x http://localhost:9999\n```\n\n## Tests\n\nTo run the test suite, execute the following commands:\n\n```bash\n$ npm install\n$ npm test\n```\n\n## License\n\n[MIT](LICENSE)\n","funding_links":[],"categories":["JavaScript"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FVladislao%2Fproxy-supervisor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FVladislao%2Fproxy-supervisor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FVladislao%2Fproxy-supervisor/lists"}