{"id":23071946,"url":"https://github.com/scrapingant/scrapingant-client-js","last_synced_at":"2025-08-15T14:32:39.446Z","repository":{"id":44735129,"uuid":"336855083","full_name":"ScrapingAnt/scrapingant-client-js","owner":"ScrapingAnt","description":"ScrapingAnt API client for JavaScript / Node.js.","archived":false,"fork":false,"pushed_at":"2022-05-06T20:33:12.000Z","size":33,"stargazers_count":12,"open_issues_count":2,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-12-05T22:08:47.702Z","etag":null,"topics":["crawler","scraper","scraping","scrapingant","webscraping"],"latest_commit_sha":null,"homepage":"https://www.npmjs.com/package/@scrapingant/scrapingant-client","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ScrapingAnt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-02-07T18:07:58.000Z","updated_at":"2024-10-10T01:07:53.000Z","dependencies_parsed_at":"2022-09-04T22:00:18.019Z","dependency_job_id":null,"html_url":"https://github.com/ScrapingAnt/scrapingant-client-js","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ScrapingAnt%2Fscrapingant-client-js","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ScrapingAnt%2Fscrapingant-client-js/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ScrapingAnt%2Fscrapingant-client-js/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ScrapingAnt%2Fscrapingant-client-js/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ScrapingAnt","download_url":"https://codeload.github.com/ScrapingAnt/scrapingant-client-js/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":229920847,"owners_count":18144863,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crawler","scraper","scraping","scrapingant","webscraping"],"created_at":"2024-12-16T07:18:17.351Z","updated_at":"2024-12-16T07:18:17.911Z","avatar_url":"https://github.com/ScrapingAnt.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ScrapingAnt API client for JavaScript\n`@scrapingant/scrapingant-client` is the official library to access [ScrapingAnt API](https://docs.scrapingant.com) from your\nJavaScript applications. It runs both in Node.js and browser and provides useful features like\nautomatic retries and parameters encoding to improve the ScrapingAnt usage experience.\n\n\u003c!-- toc --\u003e\n\n- [Quick Start](#quick-start)\n- [API key](#api-key)\n- [Retries with exponential backoff](#retries-with-exponential-backoff)\n- [API Reference](#api-reference)\n- [Examples](#examples)\n\n\u003c!-- tocstop --\u003e\n\n## Quick Start\n```js\nconst ScrapingAntClient = require('@scrapingant/scrapingant-client');\n\nconst client = new ScrapingAntClient({ apiKey: '\u003cYOUR-SCRAPINGANT-API-KEY\u003e' });\n\n// Scrape the example.com site.\nclient.scrape('https://example.com')\n    .then(res =\u003e console.log(res))\n    .catch(err =\u003e console.error(err.message));\n```\n\n## API key\nIn order to get API key you'll need to register at [ScrapingAnt Service](https://app.scrapingant.com)\n\n## Retries with exponential backoff\nNetwork communication sometimes fails, that's a given. The client will automatically retry requests that\nfailed due to a network error, an internal error of the ScrapingAnt API (HTTP 500+).\nBy default, it will retry up to 8 times. First retry will be attempted after ~500ms, second after ~1000ms\nand so on. You can configure those parameters using the `maxRetries` and `minDelayBetweenRetriesMillis`\noptions of the `ScrapingAntClient` constructor.\n\n## API Reference\nAll public classes, methods and their parameters can be inspected in this API reference.\n\n\u003ca name=\"ScrapingAntClient\"\u003e\u003c/a\u003e\n\n### [](#ScrapingAntClient) ScrapingAntClient\n\nScrapingAntClient is the official library to access [ScrapingAnt API](https://docs.scrapingant.com) from your\nJavaScript applications. It runs both in Node.js and browser.\n\n* [ScrapingAntClient](#ScrapingAntClient)\n    * [`new ScrapingAntClient(options)`](#new_ScrapingAntClient_new)\n    * [`.scrape(url, [params])`](#ScrapingAntClient+scrape) ⇒ [\u003ccode\u003eScrapingAnt API response\u003c/code\u003e](https://docs.scrapingant.com/request-response-format#response-structure)\n\n\n* * *\n\n\u003ca name=\"new_ScrapingAntClient_new\"\u003e\u003c/a\u003e\n\n#### [](#ScrapingAntClient) `new ScrapingAntClient(options)`\n\n\n| Param                                  | Type                | Default          |\n|----------------------------------------|---------------------|------------------|\n| [options]                              | \u003ccode\u003eobject\u003c/code\u003e |                  |\n| [options.maxRetries]                   | \u003ccode\u003enumber\u003c/code\u003e | \u003ccode\u003e8\u003c/code\u003e   |\n| [options.minDelayBetweenRetriesMillis] | \u003ccode\u003enumber\u003c/code\u003e | \u003ccode\u003e500\u003c/code\u003e |\n| [options.timeoutSecs]                  | \u003ccode\u003enumber\u003c/code\u003e | \u003ccode\u003e60\u003c/code\u003e  |\n| [options.apiKey]                       | \u003ccode\u003estring\u003c/code\u003e |                  |\n\n\n* * *\n\n\u003ca name=\"ScrapingAntClient+scrape\"\u003e\u003c/a\u003e\n\n#### [](#ScrapingAntClient+scrape) `scrapingAntClient.scrape(url, [parameters])` ⇒ [\u003ccode\u003eScrapingAnt API response\u003c/code\u003e](https://docs.scrapingant.com/request-response-format#response-structure)\n\nhttps://docs.scrapingant.com/request-response-format#available-parameters\n\n| Param                          | Type                 |\n|--------------------------------|----------------------|\n| url                            | \u003ccode\u003estring\u003c/code\u003e  |\n| [parameters]                   | \u003ccode\u003eobject\u003c/code\u003e  |\n| [parameters.browser]           | \u003ccode\u003eboolean\u003c/code\u003e |\n| [parameters.cookies]           | \u003ccode\u003estring\u003c/code\u003e  |\n| [parameters.headers]           | \u003ccode\u003eobject\u003c/code\u003e  |\n| [parameters.js_snippet]        | \u003ccode\u003estring\u003c/code\u003e  |\n| [parameters.proxy_type]        | \u003ccode\u003estring\u003c/code\u003e  |\n| [parameters.proxy_country]     | \u003ccode\u003estring\u003c/code\u003e  |\n| [parameters.wait_for_selector] | \u003ccode\u003estring\u003c/code\u003e  |\n| [parameters.return_text]       | \u003ccode\u003eboolean\u003c/code\u003e |\n\n**IMPORTANT NOTE:** \u003ccode\u003eparameters.js_snippet\u003c/code\u003e will be encoded to Base64 automatically by the ScrapingAnt JS client library.\n\n* * *\n\n\u003ca name=\"ScrapingAntApiError\"\u003e\u003c/a\u003e\n\n### [](#ScrapingAntApiError) ScrapingAntApiError\n\nAn `ScrapingAntApiError` is thrown for successful HTTP requests that reach the API,\nbut the API responds with an error response. Typically, those are internal errors,\nwhich are automatically retried, or validation errors, which are thrown immediately,\nbecause a correction by the user is needed.\n\n**Properties**\n\n| Name       | Type                | Description                        |\n|------------|---------------------|------------------------------------|\n| message    | \u003ccode\u003estring\u003c/code\u003e | Error message returned by the API. |\n| statusCode | \u003ccode\u003enumber\u003c/code\u003e | HTTP status code of the error.     |\n| httpMethod | \u003ccode\u003estring\u003c/code\u003e | HTTP method of the API call.       |\n\n\n* * *\n\n## Examples\n\n### Using residential proxy\n\n```js\nconst ScrapingAntClient = require('@scrapingant/scrapingant-client');\n\nconst client = new ScrapingAntClient({ apiKey: '\u003cYOUR-SCRAPINGANT-API-KEY\u003e' });\n\n// Get the residential IP info using httpbin.org\nclient.scrape('https://httpbin.org/ip', { proxy_type: 'residential' })\n    .then(res =\u003e console.log(res))\n    .catch(err =\u003e console.error(err.message));\n```\n\n### Sending custom cookies\n\n```js\nconst ScrapingAntClient = require('@scrapingant/scrapingant-client');\n\nconst client = new ScrapingAntClient({ apiKey: '\u003cYOUR-SCRAPINGANT-API-KEY\u003e' });\n\n// Scrape the httpbin.org site and get all the cookies sent before\nclient.scrape('https://httpbin.org/cookies', { cookies: 'cookieName1=cookieVal1;cookieName2=cookieVal2' })\n    .then(res =\u003e console.log(res))\n    .catch(err =\u003e console.error(err.message));\n```\n\n### Adding custom headers\n\n```js\nconst ScrapingAntClient = require('@scrapingant/scrapingant-client');\n\nconst client = new ScrapingAntClient({ apiKey: '\u003cYOUR-SCRAPINGANT-API-KEY\u003e' });\n\n// Scrape the httpbin.org site and get all the headers that would be sent before\nclient.scrape('https://httpbin.org/headers', { headers: { scraping: \"is cool!\" } })\n    .then(res =\u003e console.log(res))\n    .catch(err =\u003e console.error(err.message));\n```\n\n\n### Executing custom JS snippet\n\n```js\nconst ScrapingAntClient = require('@scrapingant/scrapingant-client');\n\nconst client = new ScrapingAntClient({ apiKey: '\u003cYOUR-SCRAPINGANT-API-KEY\u003e' });\n\n// Scrape the httpbin.org site and replace all the content with \"Hello, world\"\nconst customJsSnippet = \"var str = 'Hello, world!';\\n\" +\n    \"var htmlElement = document.getElementsByTagName('html')[0];\\n\" +\n    \"htmlElement.innerHTML = str;\"\nclient.scrape('https://httpbin.org/cookies', { js_snippet: customJsSnippet })\n    .then(res =\u003e console.log(res))\n    .catch(err =\u003e console.error(err.message));\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscrapingant%2Fscrapingant-client-js","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscrapingant%2Fscrapingant-client-js","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscrapingant%2Fscrapingant-client-js/lists"}