{"id":20056860,"url":"https://github.com/makarasty/socks-scraper","last_synced_at":"2025-10-14T13:07:04.007Z","repository":{"id":210699572,"uuid":"727223919","full_name":"makarasty/socks-scraper","owner":"makarasty","description":"🔆 Library for Node.js for proxy collection and validation","archived":false,"fork":false,"pushed_at":"2024-07-14T19:08:04.000Z","size":44,"stargazers_count":2,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-19T22:53:00.651Z","etag":null,"topics":["jsdoc","nodejs","nodejs-modules","proxies","proxies-checker","proxies-list","proxies-scraper","proxy","proxy-checker","proxy-scraper"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/makarasty.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-04T12:44:55.000Z","updated_at":"2024-07-14T19:08:07.000Z","dependencies_parsed_at":"2025-03-11T03:44:34.588Z","dependency_job_id":null,"html_url":"https://github.com/makarasty/socks-scraper","commit_stats":null,"previous_names":["makarasty/proxyscraper","makarasty/socks-scraper"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/makarasty/socks-scraper","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/makarasty%2Fsocks-scraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/makarasty%2Fsocks-scraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/makarasty%2Fsocks-scraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/makarasty%2Fsocks-scraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/makarasty","download_url":"https://codeload.github.com/makarasty/socks-scraper/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/makarasty%2Fsocks-scraper/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265382503,"owners_count":23756375,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["jsdoc","nodejs","nodejs-modules","proxies","proxies-checker","proxies-list","proxies-scraper","proxy","proxy-checker","proxy-scraper"],"created_at":"2024-11-13T12:56:47.922Z","updated_at":"2025-10-14T13:06:58.977Z","avatar_url":"https://github.com/makarasty.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Proxy Scraper\n\nLibrary for Node.js for proxy collection and validation\n\n# Install\n- `npm i socks-scraper`\n\n# OS Dependencies\n- Node.js\n\n# Lib Dependencies\n- needle\n- proxy-agent\n\n# JSDoc\n```js\n/**\n * like { address, host, port, latency }\n * @typedef {Object} SocksScraper.IDefaultProxy\n * @property {string} address\n * @property {string} host\n * @property {number} port\n * @property {number} latency\n */\n```\n\n# Usage example\n```js\n// Initialize the scraper with a list of raw sites\n\tconst socksScraper = new SocksScraper([\n\t\t\"https://raw.githubusercontent.com/officialputuid/KangProxy/KangProxy/http/http.txt\",\n\t\t\"https://raw.githubusercontent.com/officialputuid/KangProxy/KangProxy/socks4/socks4.txt\",\n\t\t\"https://raw.githubusercontent.com/officialputuid/KangProxy/KangProxy/socks5/socks5.txt\",\n\n\t\t'https://api.proxyscrape.com/?request=displayproxies\u0026status=alive\u0026proxytype=socks4',\n\t\t'https://api.proxyscrape.com/?request=displayproxies\u0026status=alive\u0026proxytype=socks5',\n\t])\n\n\t// Add one more site to the list of sites on which free proxies are placed\n\tsocksScraper.addSites([\"https://api.proxyscrape.com/?request=displayproxies\u0026status=alive\"])\n\n\t// Timeout for checking the proxy in ms\n\tconst timeout = 10000\n\n\tconst chunkSize = 5000;\n\n\tconst retryCount = 5;\n\n\tconsole.log('Updating unchecked proxies...');\n\t// Gets proxies from all sites, VERY IMPORTANT: it must be called before the getWorkedSocksProxies()\n\tawait socksScraper.updateUncheckedProxies()\n\n\tconsole.log(`Done updating unchecked proxies! (${socksScraper.unCheckedProxies.size})`);\n\n\t// Get a list of proxies from all sites, check if they work and return the best ones\n\tconst wsp4 = await socksScraper.getWorkedSocksProxies('socks4', timeout, chunkSize, undefined, undefined, retryCount)\n\n\t// Sort the list by latency and take the fastest proxy\n\tconst bestWSP4 = SocksScraper.filterByLatency(wsp4)[0]\n\n\tconsole.log(`The best socks4 proxy is ${bestWSP4.address} with latency ${bestWSP4.latency}ms (${wsp4.length})`)\n\n\tconst wsp5 = await socksScraper.getWorkedSocksProxies('socks5', timeout, chunkSize, undefined, undefined, retryCount)\n\tconst bestWSP5 = SocksScraper.filterByLatency(wsp5)[0]\n\n\tconsole.log(`The best socks5 proxy is ${bestWSP5.address} with latency ${bestWSP5.latency}ms (${wsp5.length})`)\n\n\t/* only if you have VERY good internet...\n\n\tconst http = await socksScraper.getWorkedSocksProxies('http', timeout)\n\tconst bestHttp = SocksScraper.filterByLatency(http)[0]\n\n\tconsole.log(`The best http proxy is ${bestHttp.host}:${bestHttp.port} with latency ${bestHttp.latency}ms`)\n\t*/\n\n\t//  Check my socks5 proxy to see if it works at all\n\tconst mySocks4Proxy = await SocksScraper.isAliveProxy('socks4', '3.10.93.50:80', 10000)\n\tconst isAlive = Boolean(mySocks4Proxy)\n\n\tconsole.log(`My socks4 proxy is ${isAlive ? 'alive' : 'dead'}`)\n\tconsole.log(mySocks4Proxy);\n```\n```js\nMy socks5 proxy is alive!\n{\n\taddress: '3.122.84.99:3128',\n\thost: '3.122.84.99',\n\tport: 3128,\n\tlatency: 32\n}\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmakarasty%2Fsocks-scraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmakarasty%2Fsocks-scraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmakarasty%2Fsocks-scraper/lists"}