{"id":20710016,"url":"https://github.com/oxylabs/rotating-proxies-javascript","last_synced_at":"2025-04-23T04:51:56.054Z","repository":{"id":44763542,"uuid":"449200607","full_name":"oxylabs/rotating-proxies-javascript","owner":"oxylabs","description":"Learn how to rotate proxies using JavaScript.","archived":false,"fork":false,"pushed_at":"2025-02-11T12:44:23.000Z","size":37,"stargazers_count":4,"open_issues_count":1,"forks_count":3,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-03-29T22:21:05.432Z","etag":null,"topics":["axios","javascript","node-scraper","nodejs","proxy","puppeteer","python","rotating-proxy","web-proxies","web-proxy","web-scraping"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oxylabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-01-18T08:26:27.000Z","updated_at":"2025-02-11T12:44:27.000Z","dependencies_parsed_at":"2023-12-18T10:49:33.282Z","dependency_job_id":"f594c736-752f-40fd-9977-a0dc43f7ba58","html_url":"https://github.com/oxylabs/rotating-proxies-javascript","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxylabs%2Frotating-proxies-javascript","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxylabs%2Frotating-proxies-javascript/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxylabs%2Frotating-proxies-javascript/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxylabs%2Frotating-proxies-javascript/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oxylabs","download_url":"https://codeload.github.com/oxylabs/rotating-proxies-javascript/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250372947,"owners_count":21419722,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["axios","javascript","node-scraper","nodejs","proxy","puppeteer","python","rotating-proxy","web-proxies","web-proxy","web-scraping"],"created_at":"2024-11-17T02:09:33.103Z","updated_at":"2025-04-23T04:51:56.032Z","avatar_url":"https://github.com/oxylabs.png","language":"JavaScript","readme":"# Rotating-Proxies-with-JavaScript\n\n[![Oxylabs promo code](https://raw.githubusercontent.com/oxylabs/product-integrations/refs/heads/master/Affiliate-Universal-1090x275.png)](https://oxylabs.go2cloud.org/aff_c?offer_id=7\u0026aff_id=877\u0026url_id=112)\n\n[![](https://dcbadge.vercel.app/api/server/eWsVUJrnG5)](https://discord.gg/GbxmdGhZjq)\n\n[\u003cimg src=\"https://img.shields.io/static/v1?label=\u0026message=JavaScript\u0026color=brightgreen\" /\u003e](https://github.com/topics/javascript) [\u003cimg src=\"https://img.shields.io/static/v1?label=\u0026message=Web%20Scraping\u0026color=important\" /\u003e](https://github.com/topics/web-scraping) [\u003cimg src=\"https://img.shields.io/static/v1?label=\u0026message=Rotating%20Proxies\u0026color=blueviolet\" /\u003e](https://github.com/topics/rotating-proxies)\n\n- [Requirements](#requirements)\n- [Finding Current IP Aaddress](#finding-current-ip-aaddress)\n- [Using Proxy](#using-proxy)\n- [Rotating Multiple Proxies](#rotating-multiple-proxies)\n\n## Requirements\n\nIn this tutorial, we will be using [Axios](https://github.com/axios/axios) to make requests. If needed, the code can be easily modified for other libraries as well.\n\nOpen the terminal and run the following command to initiate a new Node project:\n\n```shell\nnpm init -y\n```\n\nNext step is to install Axios by running the following command:\n\n```sh\nnpm install axios\n```\n\n## Finding Current IP Address\n\nTo check if the proxy works properly, first, we need a basic code that prints the current IP address.\n\nThe website http://httpbin.org/ip is appropriate for this purpose as it returns IP addresses in a clean format.\n\nCreate a new JavaScript file and make changes as outlined below.\n\nThe first step would be to import `axios`.\n\n```JavaScript\nconst axios = require(\"axios\");\n```\nNext, call the `get()` method and send the URL of the target website.\n\n```javascript\nconst url = 'https://httpbin.org/ip';\nconst response = await axios.get(url);\n```\n\nTo see the data returned by the server, access `data` attribute of the `response` object:\n\n```JavaScript\nconsole.log(response.data);\n// Prints current IP\n```\n\nFor the complete implementation, see the [no_proxy.js](no_proxy.js) file.\n\n## Using a Proxy \n\nFor this example, we are going to use a proxy with IP 46.138.246.248 and port 8088. \n\nAxios can handle proxies directly. The proxy information needs to be sent as the second parameter of the `get()` method.\n\nThe proxy object should have a `host` and `port`. See an example:\n\n```JavaScript\nproxy_no_auth = {\n    host: '46.138.246.248',\n    port: 8088\n}\n```\n\nIf proxies need authentication, simply add an `auth` object with `username` and `password`.\n\n```javascript\nproxy_with_auth = {\n    host: '46.138.246.248',\n    port: 8088,\n    auth: {\n        username: 'USERNAME',\n        password: 'PASSWORD'\n    }\n}\n```\n\nThis `proxy_no_auth` or `proxy_with_auth` object can then be sent with the `get` method.\n\n```javascript\nconst response = await axios.get(url, {\n    proxy: proxy_no_auth\n});\n```\n\nRun this code from the terminal to see the effective IP address.\n\nYou will notice that now, instead of your original IP, the IP address of the proxy is printed.\n\n```sh\nnode single_proxy_axios.js\n// Prints {'origin': '46.138.246.248'}\n```\n\nSee the complete implementation in the [single_proxy_axios.js](single_proxy_axios.js) file.\n\nIf encountering an issue where you cannot target an HTTPS with an HTTP Proxy, use the [http_https_workaround.js](http_https_workaround.js) file.\n\nThis is a known issue, more info: https://github.com/axios/axios/issues/925\n\n## Rotating Multiple Proxies\n\nIf multiple proxies are available, it is possible to rotate proxies with JavaScript.\n\nSome websites allow downloading a list of proxies as CSV or similar format. \n\nIn this example, we will be working with a file downloaded from one of the free websites. \n\nThis file contains the proxies in this format. Note that proxy and port are separated by a comma.\n\n```\n20.94.229.106,80\n209.141.55.228,80\n103.149.162.194,80\n206.253.164.122,80\n200.98.114.237,8888\n193.164.131.202,7890\n98.12.195.129,44\n49.206.233.104,80\n```\n\nTo get a rotating IP proxy using this file, first, we need to read this CSV file in asynchronous code.\n\nTo read CSV file asynchronously, install the package [async-csv](https://www.npmjs.com/package/async-csv).\n\n```sh\nnpm install async-csv\n```\n\nWe will also need the `fs` package, which does not need a separate install.\n\nAfter the imports, use the following lines of code to read the CSV file.\n\n```javascript\n// Read file from disk:\nconst csvFile = await fs.readFile('proxy_list.csv');\n\n// Convert CSV string into rows:\nconst data = await csv.parse(csvFile);\n```\n\nThe data object is an `Array` that contains each row as `Array`.\n\nWe can loop over all these rows using the `map` function.\n\nNote that in the loop, we will call the get method of Axios to call the same URL, each time with a different proxy.\n\nThe `get` method of Axios is `async`. This means that we can not call the `map` function of `data` directly.\n\nInstead, we need to use the `Promise` object as follows:\n\n```JavaScript\nawait Promise.all(data.map(async (item) =\u003e {\n       // More async code here\n    }));\n```\n\nIt is time to create the `proxy` object. The structure will be as explained in the earlier section.\n\n```javascript\n// Create the Proxy object:\nproxy_no_auth = {\n  host: item[0],\n  port: item[1]\n};\n```\n\nAbove lines convert the data from `[ '20.94.229.106', '80' ]` format to `{ host: '20.94.229.106', port: '80' }`format.\n\nNext, call the `get` method and send the proxy object.\n\n```javascript\nconst url = 'https://httpbin.org/ip';\nconst response = await axios.get(url, {\n  proxy: proxy_no_auth\n});\n```\n\nFor the complete code, please see the [rotating_proxies.js](rotating_proxies.js) file.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foxylabs%2Frotating-proxies-javascript","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foxylabs%2Frotating-proxies-javascript","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foxylabs%2Frotating-proxies-javascript/lists"}