{"id":15552221,"url":"https://github.com/bahmutov/cypress-crawl-example","last_synced_at":"2025-09-28T20:30:58.635Z","repository":{"id":37104009,"uuid":"444505380","full_name":"bahmutov/cypress-crawl-example","owner":"bahmutov","description":"Crawling all local pages using Cypress example","archived":false,"fork":false,"pushed_at":"2025-01-16T05:56:18.000Z","size":430,"stargazers_count":3,"open_issues_count":2,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-16T06:46:01.146Z","etag":null,"topics":["cypress-example"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bahmutov.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-01-04T17:22:06.000Z","updated_at":"2025-01-16T05:55:57.000Z","dependencies_parsed_at":"2024-03-14T03:26:49.150Z","dependency_job_id":"4e6c473f-5359-42f7-b99d-8bb93619e1f7","html_url":"https://github.com/bahmutov/cypress-crawl-example","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bahmutov%2Fcypress-crawl-example","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bahmutov%2Fcypress-crawl-example/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bahmutov%2Fcypress-crawl-example/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bahmutov%2Fcypress-crawl-example/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bahmutov","download_url":"https://codeload.github.com/bahmutov/cypress-crawl-example/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":234558803,"owners_count":18852283,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cypress-example"],"created_at":"2024-10-02T14:12:49.024Z","updated_at":"2025-09-28T20:30:58.122Z","avatar_url":"https://github.com/bahmutov.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# cypress-crawl-example [![renovate-app badge][renovate-badge]][renovate-app] ![cypress version](https://img.shields.io/badge/cypress-9.7.0-brightgreen) [![ci](https://github.com/bahmutov/cypress-crawl-example/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/bahmutov/cypress-crawl-example/actions/workflows/ci.yml)\n\n\u003e Crawling all local pages using Cypress example\n\nCypress is not built to be a fast crawler, but for simple tests it will do. In this example, we crawl all pages reachable from the root page. See [spec.js](./cypress/integration/spec.js)\n\n![Crawling local pages](./images/crawl.gif)\n\nIn [spec2.js](./cypress/integration/spec2.js) we resolve duplicate URLs by checking them using [cy.request](https://on.cypress.io/request) and taking the redirected URL.\n\n## Testing 404 page\n\nThere is automatic 404 page when trying to visit a non-existent page. The test in [404-spec.js](./cypress/integration/404-spec.js) implements the test.\n\n![404 spec](./images/404-spec.png)\n\n## Blog posts\n\n**Note:** please do not rename the spec files, as the blog post refers to them via GitHub links.\n\n- [Crawl Local Pages Using Cypress](https://glebbahmutov.com/blog/crawl-using-cypress/)\n\n## Videos\n\n- [Crawl Local Pages Using Cypress](https://youtu.be/FDNeiwKWdb4)\n- [Test 404 Page](https://youtu.be/0OfILKbqbsg)\n- [Crawl Pages With Resource Check](https://www.youtube.com/watch?v=h7taQNvj0Cs)\n\n## To run\n\n```\n$ npm install\n$ npm run dev\n```\n\n## Small print\n\nAuthor: Gleb Bahmutov \u0026lt;gleb.bahmutov@gmail.com\u0026gt; \u0026copy; 2022\n\n- [@bahmutov](https://twitter.com/bahmutov)\n- [glebbahmutov.com](https://glebbahmutov.com)\n- [blog](https://glebbahmutov.com/blog)\n- [videos](https://www.youtube.com/glebbahmutov)\n- [presentations](https://slides.com/bahmutov)\n- [cypress.tips](https://cypress.tips)\n- [Cypress Advent 2021](https://cypresstips.substack.com/)\n\nLicense: MIT - do anything with the code, but don't blame me if it does not work.\n\nSupport: if you find any problems with this module, email / tweet /\n[open issue](https://github.com/bahmutov/cypress-crawl-example/issues) on Github\n\n## MIT License\n\nCopyright (c) 2022 Gleb Bahmutov \u0026lt;gleb.bahmutov@gmail.com\u0026gt;\n\nPermission is hereby granted, free of charge, to any person\nobtaining a copy of this software and associated documentation\nfiles (the \"Software\"), to deal in the Software without\nrestriction, including without limitation the rights to use,\ncopy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the\nSoftware is furnished to do so, subject to the following\nconditions:\n\nThe above copyright notice and this permission notice shall be\nincluded in all copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,\nEXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES\nOF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\nNONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT\nHOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,\nWHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING\nFROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR\nOTHER DEALINGS IN THE SOFTWARE.\n\n[renovate-badge]: https://img.shields.io/badge/renovate-app-blue.svg\n[renovate-app]: https://renovateapp.com/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbahmutov%2Fcypress-crawl-example","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbahmutov%2Fcypress-crawl-example","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbahmutov%2Fcypress-crawl-example/lists"}