{"id":26742741,"url":"https://github.com/scrapfly/typescript-scrapfly","last_synced_at":"2025-06-13T23:33:38.672Z","repository":{"id":182987786,"uuid":"667873608","full_name":"scrapfly/typescript-scrapfly","owner":"scrapfly","description":"SDK for Scrapfly.io web scraping API","archived":false,"fork":false,"pushed_at":"2024-11-09T06:23:11.000Z","size":1475,"stargazers_count":10,"open_issues_count":0,"forks_count":5,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-04-14T17:49:17.924Z","etag":null,"topics":["api","saas","scraping","sdk","webscraping"],"latest_commit_sha":null,"homepage":"https://scrapfly.io/","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/scrapfly.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-07-18T13:48:33.000Z","updated_at":"2025-01-27T19:46:22.000Z","dependencies_parsed_at":"2024-06-22T07:27:26.987Z","dependency_job_id":null,"html_url":"https://github.com/scrapfly/typescript-scrapfly","commit_stats":null,"previous_names":["scrapfly/typescript-scrapfly"],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scrapfly%2Ftypescript-scrapfly","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scrapfly%2Ftypescript-scrapfly/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scrapfly%2Ftypescript-scrapfly/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scrapfly%2Ftypescript-scrapfly/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/scrapfly","download_url":"https://codeload.github.com/scrapfly/typescript-scrapfly/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248931050,"owners_count":21185104,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","saas","scraping","sdk","webscraping"],"created_at":"2025-03-28T06:19:51.950Z","updated_at":"2025-04-14T17:49:21.459Z","avatar_url":"https://github.com/scrapfly.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Scrapfly SDK\n\n`npm install scrapfly-sdk`  \n`deno add jsr:@scrapfly/scrapfly-sdk`  \n`bun jsr add @scrapfly/scrapfly-sdk`  \n\nTypescript/Javascript SDK for [Scrapfly.io](https://scrapfly.io/) web scraping API which allows to:\n\n-   Scrape the web without being blocked.\n-   Use headless browsers to access Javascript-powered page data.\n-   Scale up web scraping.\n-   ... and [much more](https://scrapfly.io/docs/scrape-api/getting-started)!\n\nFor web scraping guides see [our blog](https://scrapfly.io/blog/) and [#scrapeguide](https://scrapfly.io/blog/tag/scrapeguide/) tag for how to scrape specific targets.\n\nThe SDK is distributed through:\n- [npmjs.com/package/scrapfly-sdk](https://www.npmjs.com/package/scrapfly-sdk)\n- [jsr.io/@scrapfly/scrapfly-sdk](https://jsr.io/@scrapfly/scrapfly-sdk)\n\n## Quick Intro\n\n1. Register a [Scrapfly account for free](https://scrapfly.io/register)\n2. Get your API Key on [scrapfly.io/dashboard](https://scrapfly.io/dashboard)\n3. Start scraping: 🚀\n\n```javascript\n// node \nimport { ScrapflyClient, ScrapeConfig } from 'scrapfly-sdk';\n// bun\nimport { ScrapflyClient, ScrapeConfig} from '@scrapfly/scrapfly-sdk';\n// deno: \nimport { ScrapflyClient, ScrapeConfig } from 'jsr:@scrapfly/scrapfly-sdk';\n\nconst key = 'YOUR SCRAPFLY KEY';\nconst client = new ScrapflyClient({ key });\nconst apiResponse = await client.scrape(\n    new ScrapeConfig({\n        url: 'https://web-scraping.dev/product/1',\n        // optional parameters:\n        // enable javascript rendering\n        render_js: true,\n        // set proxy country\n        country: 'us',\n        // enable anti-scraping protection bypass\n        asp: true,\n        // set residential proxies\n        proxy_pool: 'public_residential_pool',\n        // etc.\n    }),\n);\nconsole.log(apiResponse.result.content); // html content\n// Parse HTML directly with SDK (through cheerio)\nconsole.log(apiResponse.result.selector('h3').text());\n```\n\nFor more see [/examples](/examples/) directory.  \nFor more on Scrapfly API see our [getting started documentation](https://scrapfly.io/docs/scrape-api/getting-started)\nFor Python see [Scrapfly Python SDK](https://github.com/scrapfly/python-scrapfly)\n\n## Debugging\n\nTo enable debug logs set Scrapfly's log level to `\"DEBUG\"`:\n\n```javascript\nimport { log } from 'scrapfly-sdk';\n\nlog.setLevel('DEBUG');\n```\n\nAdditionally, set `debug=true` in `ScrapeConfig` to access debug information in [Scrapfly web dashboard](https://scrapfly.io/dashboard):\n\n```typescript\nimport { ScrapflyClient } from 'scrapfly-sdk';\n\nnew ScrapeConfig({\n    url: 'https://web-scraping.dev/product/1',\n    debug: true,\n    // ^ enable debug information - this will show extra details on web dashboard\n});\n```\n\n## Development\n\nThis is a Deno Typescript project that builds to NPM through [DNT](https://github.com/denoland/dnt).\n\n- `/src` directory contains all of the source code with `main.ts` being the entry point.\n- `__tests__` directory contains tests for the source code.\n- `deno.json` contains meta information\n- `build.ts` is the build script that builds the project to nodejs ESM package.\n- `/npm` directory will be produced when `built.ts` is executed for building node package.\n\n```bash\n# make modifications and run tests\n$ deno task test\n# format\n$ deno fmt\n# lint\n$ deno lint\n# publish JSR:\n$ deno publish\n# build NPM package:\n$ deno task build-npm\n# publish NPM:\n$ cd npm \u0026\u0026 npm publish\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscrapfly%2Ftypescript-scrapfly","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscrapfly%2Ftypescript-scrapfly","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscrapfly%2Ftypescript-scrapfly/lists"}