{"id":13588673,"url":"https://github.com/ulixee/secret-agent","last_synced_at":"2025-04-01T13:03:43.431Z","repository":{"id":37181699,"uuid":"274739987","full_name":"ulixee/secret-agent","owner":"ulixee","description":"The web scraper that's nearly impossible to block - now called @ulixee/hero","archived":false,"fork":false,"pushed_at":"2023-03-07T03:00:38.000Z","size":48565,"stargazers_count":706,"open_issues_count":45,"forks_count":49,"subscribers_count":17,"default_branch":"main","last_synced_at":"2025-04-01T13:03:29.208Z","etag":null,"topics":["automated","browser","chromium","devtools","mitm","mitmproxy","playwright","proxy","puppeteer","scraping","secretagent","stealth"],"latest_commit_sha":null,"homepage":"https://secretagent.dev","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ulixee.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null},"funding":{"github":"ulixee"}},"created_at":"2020-06-24T18:21:38.000Z","updated_at":"2025-03-29T15:08:46.000Z","dependencies_parsed_at":"2024-01-14T04:38:53.016Z","dependency_job_id":"e9767532-75f2-4a2b-a891-4c4a1f619131","html_url":"https://github.com/ulixee/secret-agent","commit_stats":{"total_commits":607,"total_committers":6,"mean_commits":"101.16666666666667","dds":0.1334431630971994,"last_synced_commit":"d4b988597ca0f7ffd532994b5f82d983f4b13dca"},"previous_names":[],"tags_count":71,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ulixee%2Fsecret-agent","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ulixee%2Fsecret-agent/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ulixee%2Fsecret-agent/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ulixee%2Fsecret-agent/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ulixee","download_url":"https://codeload.github.com/ulixee/secret-agent/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246644047,"owners_count":20810686,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automated","browser","chromium","devtools","mitm","mitmproxy","playwright","proxy","puppeteer","scraping","secretagent","stealth"],"created_at":"2024-08-01T15:06:51.480Z","updated_at":"2025-04-01T13:03:43.384Z","avatar_url":"https://github.com/ulixee.png","language":"TypeScript","readme":"## SecretAgent 2.0 === Hero\n\n\u003e :bell: SecretAgent 2.0 is named \"[Hero](https://github.com/ulixee/hero)\" and currently in alpha testing. We're ready for developers to begin to switch over - it's a pretty easy transition ([migration guide](https://ulixee.org/docs/hero/help/migration-guide)). Follow along with development of Hero [here](https://github.com/ulixee/hero) or check out the latest npm packages `@ulixee/hero-playground`.\n\n---\n\n# SecretAgent\n\nSecretAgent is a web browser that's built for scraping.\n\n- [x] **Built for scraping** - it's the first modern headless browsers designed specifically for scraping instead of just automated testing.\n- [x] **Designed for web developers** - We've recreated a fully compliant DOM directly in NodeJS allowing you bypass the headaches of previous scraper tools.\n- [x] **Powered by Chrome** - The powerful Chrome engine sits under the hood, allowing for lightning fast rendering.\n- [x] **Emulates any modern browser** - BrowserEmulators make it easy to disguise your script as practically any browser.\n- [x] **Avoids detection along the entire stack** - Don't be blocked because of TLS fingerprints in your networking stack.\n\nCheck out our [website for more details](https://secretagent.dev).\n\n## Installation\n\n```shell script\nnpm i --save secret-agent\n```\n\nor\n\n```shell script\nyarn add secret-agent\n```\n\n## Usage\n\nSecretAgent provides access to the W3C DOM specification without the need for Puppeteer's complicated evaluate callbacks and multi-context switching:\n\n```js\nconst agent = require('secret-agent');\n\n(async () =\u003e {\n  await agent.goto('https://example.org');\n  const title = await agent.document.title;\n  const intro = await agent.document.querySelector('p').textContent;\n  await agent.close();\n})();\n```\n\nBrowse the [full API docs](https://secretagent.dev/docs).\n\n## Contributing\n\nWe'd love your help in making SecretAgent a better tool. Please don't hesitate to send a pull request.\n\n## License\n\n[MIT](LICENSE.md)\n","funding_links":["https://github.com/sponsors/ulixee"],"categories":["TypeScript","chromium"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fulixee%2Fsecret-agent","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fulixee%2Fsecret-agent","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fulixee%2Fsecret-agent/lists"}