{"id":14973167,"url":"https://github.com/sunny9577/proxy-scraper","last_synced_at":"2025-05-15T23:03:27.283Z","repository":{"id":38416251,"uuid":"146194503","full_name":"sunny9577/proxy-scraper","owner":"sunny9577","description":"⭐️ A proxy scraper made using Protractor | Proxy list Updates every three hour 🔥","archived":false,"fork":false,"pushed_at":"2025-05-15T21:33:04.000Z","size":1810731,"stargazers_count":496,"open_issues_count":2,"forks_count":74,"subscribers_count":15,"default_branch":"master","last_synced_at":"2025-05-15T23:02:52.282Z","etag":null,"topics":["angularjs","freeproxy","getproxy","http-proxy","https-proxy","nodejs","protractor","proxies","proxy","proxy-list","proxy-scraper","proxylist","scraper","selenium","socks","socks5","socks5-proxy"],"latest_commit_sha":null,"homepage":"https://sunny9577.github.io/proxy-scraper/","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sunny9577.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-08-26T15:53:02.000Z","updated_at":"2025-05-15T21:33:07.000Z","dependencies_parsed_at":"2024-02-18T04:25:59.722Z","dependency_job_id":"ef668718-14f2-4588-afd5-fb1290f8eeba","html_url":"https://github.com/sunny9577/proxy-scraper","commit_stats":{"total_commits":13312,"total_committers":5,"mean_commits":2662.4,"dds":"0.13716947115384615","last_synced_commit":"d11dd5fea9a9f8f3144d2dec12db840dd5d6b77e"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sunny9577%2Fproxy-scraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sunny9577%2Fproxy-scraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sunny9577%2Fproxy-scraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sunny9577%2Fproxy-scraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sunny9577","download_url":"https://codeload.github.com/sunny9577/proxy-scraper/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254436944,"owners_count":22070946,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["angularjs","freeproxy","getproxy","http-proxy","https-proxy","nodejs","protractor","proxies","proxy","proxy-list","proxy-scraper","proxylist","scraper","selenium","socks","socks5","socks5-proxy"],"created_at":"2024-09-24T13:48:15.371Z","updated_at":"2025-05-15T23:03:27.247Z","avatar_url":"https://github.com/sunny9577.png","language":"JavaScript","readme":"# ⭐️ A proxy scraper made using Protractor | Proxy list - Updates every 3 hours 🔥\n\n[![Every 10 Minutes Update](https://github.com/sunny9577/proxy-scraper/actions/workflows/nodejs.yml/badge.svg?branch=master)](https://github.com/sunny9577/proxy-scraper/actions/workflows/nodejs.yml)\n![GitHub](https://img.shields.io/github/license/sunny9577/proxy-scraper)\n![GitHub last commit](https://img.shields.io/github/last-commit/sunny9577/proxy-scraper)\n[![Maintenance](https://img.shields.io/badge/Maintained%3F-yes-green.svg)](https://GitHub.com/sunny9577/proxy-scraper/graphs/commit-activity)\n[![Website sunny9577.github.io](https://img.shields.io/website-up-down-green-red/http/github.io.svg)](https://sunny9577.github.io/)\n[![made-with-nodejs](https://img.shields.io/badge/Made%20with-Nodejs-green.svg)](https://www.nodejs.org/)\n\n\u003c!-- dynamic-count-start --\u003e\n## Current Proxy Count: 1700+ 🚀\n\u003c!-- dynamic-count-end --\u003e\n\n\n## Quick Access - By Format\n\n\nTXT\n\n    curl \"https://sunny9577.github.io/proxy-scraper/proxies.txt\" \u003e proxies.txt\n\nJSON\n\n    curl \"https://sunny9577.github.io/proxy-scraper/proxies.json\" \u003e proxies.json\n\nCSV\n\n    curl \"https://sunny9577.github.io/proxy-scraper/proxies.csv\" \u003e proxies.csv\n\nYAML\n\n\n    curl \"https://sunny9577.github.io/proxy-scraper/proxies.yaml\" \u003e proxies.yaml\n\nXML\n\n\n    curl \"https://sunny9577.github.io/proxy-scraper/proxies.xml\" \u003e proxies.xml\n\n## Quick Access - By Type\n###### Supported formats: txt | json | csv | yaml | xml\n\nSOCKS5\n\n    curl \"https://sunny9577.github.io/proxy-scraper/generated/socks5_proxies.txt\" \u003e socks5_proxies.txt\n\n\nSOCKS4\n\n    curl \"https://sunny9577.github.io/proxy-scraper/generated/socks4_proxies.txt\" \u003e socks4_proxies.txt\n\nHTTP(S)\n\n    curl \"https://sunny9577.github.io/proxy-scraper/generated/http_proxies.txt\" \u003e http_proxies.txt\n\n\n## Setup Requirements\n1 - A Windows, Mac or Linux Computer\n\n2 - Google Chrome installed\n\n3 - Nodejs and NPM installed\n\n## Setup\nThis bot is based on [protractor](https://www.protractortest.org/#/) automated browser testing software.\n### 1 - Open a terminal and make sure you are in the project directory\n\tcd proxy-scraper\n\n### 2 - Make sure you have protractor installed\nTo check if you have protractor installed run:\n\n\tprotractor --version\n\nIf the output is something like this:\n\n\tVersion 7.0.0\n\nThis means protractor is already installed, otherwise run the command:\n\n\tnpm install -g protractor\n\n### 3 - Install Selenium Webdriver\nThis is where your program will run, to install it simply execute the command:\n\n\twebdriver-manager\n\n### 4 - Database Connectivity ( Only if u want to save output to database)\nYou can skip this step just set SAVE_TO_DB to false in config.js\n\nGo to the file config.js and change with your login information\n\n\tnpm install mysql2\n\t\nTable Structure (Table Name: proxies_tb)\n\t\t\n|Name|Type|Collation|Attributes|Null|Default|Comments|Extra|\n|----|----|---------|----------|----|-------|--------|-----|\n|1\t|idPrimary\t|int(11)\t|\t\t|No\t|None\t\t|AUTO_INCREMENT\n|2\t|proxyIndex\t|varchar(50)\t|latin1_swedish_ci\t\t|No\t|None\t\t\n|3\t|port\t|int(10)\t\t|\t|No\t|None\t\t\n|4\t|country\t|varchar(100)\t|latin1_swedish_ci\t\t|No\t|None\t\t\n|5\t|type\t|varchar(100)\t|latin1_swedish_ci\t\t|No\t|None\t\t\n|6\t|anonymity\t|varchar(100)\t|latin1_swedish_ci\t\t|No\t|None\t\n\n## Running the bot\n\n### 1 - Update the Selenium webserver and install dependencies\nOpen a terminal and make sure you are in the project directory:\n\t\n\tcd proxy-scraper\n\t\nUpdate webdrivers:\n\n\twebdriver-manager update\n\t\nInstall dependencies:\n\n\tnpm install\n\t\n### 2 - Run the bot\nIts time to start retrieving the proxies, make sure you are in the project directory:\n\t\n\tcd proxy-scraper\n\t\nRun the bot:\n\n\tprotractor conf.js\n\n## Additional Tools\n\n- #### socks5_test.py\n\t\n\tThis python script tests the SOCKS5 proxies against https://icanhazip.com and https://eset.com. Contributed by [@techroy23](https://www.github.com/techroy23)\n\n\tSteps to run:\n\t- pip3 install tabulate pysocks\n\t- python3 socks5_test.py\n\n## Disclaimer\nThis repository is intended solely for educational purposes. I neither endorse nor encourage any illegal activities or behavior.\n\n## License\n\n[MIT](LICENSE)\n\n[![Stargazers over time](https://starchart.cc/sunny9577/proxy-scraper.svg)](https://starchart.cc/Naereen/badges)\n","funding_links":[],"categories":["JavaScript"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsunny9577%2Fproxy-scraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsunny9577%2Fproxy-scraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsunny9577%2Fproxy-scraper/lists"}