{"id":13393398,"url":"https://github.com/lrlna/puppeteer-walker","last_synced_at":"2025-09-11T13:33:27.539Z","repository":{"id":65479843,"uuid":"110693630","full_name":"lrlna/puppeteer-walker","owner":"lrlna","description":"a puppeteer walker 🕷 🕸","archived":false,"fork":false,"pushed_at":"2020-09-03T02:57:23.000Z","size":82,"stargazers_count":79,"open_issues_count":5,"forks_count":11,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-04-25T19:01:03.141Z","etag":null,"topics":["chrome","crawler","headless","puppeteer","spider","walker"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lrlna.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-11-14T13:27:01.000Z","updated_at":"2023-06-29T04:50:24.000Z","dependencies_parsed_at":"2023-01-25T08:35:14.304Z","dependency_job_id":null,"html_url":"https://github.com/lrlna/puppeteer-walker","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lrlna%2Fpuppeteer-walker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lrlna%2Fpuppeteer-walker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lrlna%2Fpuppeteer-walker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lrlna%2Fpuppeteer-walker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lrlna","download_url":"https://codeload.github.com/lrlna/puppeteer-walker/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":233694820,"owners_count":18715508,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chrome","crawler","headless","puppeteer","spider","walker"],"created_at":"2024-07-30T17:00:51.817Z","updated_at":"2025-01-13T05:04:28.194Z","avatar_url":"https://github.com/lrlna.png","language":"JavaScript","funding_links":[],"categories":["Opensource projects","JavaScript"],"sub_categories":[],"readme":"# puppeteer-walker\n[![npm version][1]][2] [![build status][3]][4]\n[![downloads][5]][6] [![js-standard-style][7]][8]\n\nA crawler to go through your given site in a headless chrome using\n[puppeteer](https://github.com/GoogleChrome/puppeteer).  Returns an object\ncontaining host, current path, and current DOM object\n\n## Usage\n\n```js\nvar Walker = require('puppeteer-walker')\n\nvar walker = Walker()\n\nwalker.on('end', () =\u003e console.log('finished walking'))\nwalker.on('error', (err) =\u003e console.log('error', err))\nwalker.on('page', async (page) =\u003e {\n  var title = await page.title()\n  console.log(`title: ${title}`)\n})\n\nwalker.walk('https://avocado.choo.io')\n```\n\n## API\n### `walker = PuppeteerWalker()`\nCreate a new walker instance.\n\n### `walker.on('page', async cb(Page, push))`\nListen to a `page` event. Returns an instance of the puppeteer [Page\nClass](https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#class-page).\nThe `callback` has to be an Async Function.\n\nUse the `push(url)` method to add more pages into the internal walker queue.\nThis is useful for busting past login forms, and the like.\n\n### `walker.on('error', cb(err))`\nListen to `error` events.\n\n### `walker.on('end', cb)`\nListen to an `end` event.\n\n### `walker.walk(url)`\nStart walking the URL.\n\n## See Also\n- [GoogleChrome/puppeteer](https://github.com/GoogleChrome/puppeteer)\n\n## License\n[Apache-2.0](./LICENSE)\n\n[1]: https://img.shields.io/npm/v/puppeteer-walker.svg?style=flat-square\n[2]: https://npmjs.org/package/puppeteer-walker\n[3]: https://img.shields.io/travis/lrlna/puppeteer-walker/master.svg?style=flat-square\n[4]: https://travis-ci.org/lrlna/puppeteer-walker\n[5]: http://img.shields.io/npm/dm/puppeteer-walker.svg?style=flat-square\n[6]: https://npmjs.org/package/puppeteer-walker\n[7]: https://img.shields.io/badge/code%20style-standard-brightgreen.svg?style=flat-square\n[8]: https://github.com/feross/standard\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flrlna%2Fpuppeteer-walker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flrlna%2Fpuppeteer-walker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flrlna%2Fpuppeteer-walker/lists"}