{"id":21575588,"url":"https://github.com/rangle/a11y-violations-crawler","last_synced_at":"2025-03-18T06:45:46.651Z","repository":{"id":54542550,"uuid":"329386797","full_name":"rangle/a11y-violations-crawler","owner":"rangle","description":"A performant webpage-crawler that will run each page through axe-core to display and/or store violations related to it for reporting.","archived":false,"fork":false,"pushed_at":"2021-02-11T19:47:19.000Z","size":1581,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-01-24T13:32:38.593Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"CSS","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rangle.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-01-13T17:44:52.000Z","updated_at":"2023-04-06T18:54:06.000Z","dependencies_parsed_at":"2022-08-13T19:10:38.892Z","dependency_job_id":null,"html_url":"https://github.com/rangle/a11y-violations-crawler","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rangle%2Fa11y-violations-crawler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rangle%2Fa11y-violations-crawler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rangle%2Fa11y-violations-crawler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rangle%2Fa11y-violations-crawler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rangle","download_url":"https://codeload.github.com/rangle/a11y-violations-crawler/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244173507,"owners_count":20410295,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-24T12:13:40.450Z","updated_at":"2025-03-18T06:45:46.624Z","avatar_url":"https://github.com/rangle.png","language":"CSS","funding_links":[],"categories":[],"sub_categories":[],"readme":"# POC - a11y crawler/scanner\nThe goal is to provide the client and/or developers a tool where they can access a website's accessibility issues so that they can be addressed.\nCreate a tool that involves:\n- crawling a website\n- gathering the interal urls\n- generating an accessibility report for each url\n- generating an accessibility summary report for the whole website \n\n## Installation\nPOC is using npm, expressjs\nCreated using node v12\n\nRun `yarn` from the root folder\n\n## Pre-Run Setup\nCopy the .env_sample file in crawler-scanner/ and rename it to .env.development\n\n## Running the crawler/checker from the browser (Via React Frontend)\n\n- navigate to localhost:3000\n- on the homepage, there will be a small form to launch a scan\n- enter the url and if you want to scan as well, check the box\n- press the submit button\n- refresh once the scanning is completed\n\n## Running Only the crawler/checker server\n`yarn workspace a11y-crawler start`\n## Running Only the React server\n`yarn workspace a11y-frontend-react start`\n## Running the crawler from the command line\n\nRunning this will launch the node-crawler and crawl the url you have provided.\nIt will generate a txt result file with all the valid urls found.\n\n- CD into the packages/crawler-scanner/src/lib folder\n\nnode crawler.js --siteUrl `\u003cURL\u003e` [--saveFile \u003cstring\u003e]\n\n- if no saveFile is provided, it will default to the url's hostname.\n- folder/file creation:  `./crawls/\u003chostname\u003e/\u003ctimestamp\u003e/\u003csaveFile\u003e.txt`\n\nExample call: `node crawler.js --siteUrl https://www.yahoo.ca`\n\n## Running the Puppeteer checker from the command line\n\nRunning this will launch the puppeteer axe checker. It will read the txt result\nfile generated in the crawl, and generate JSON result files listing out all the \naccessibility violations for each URL.\n\n- CD into the packages/crawler-scanner/src/lib folder\n\nnode checker.js --crawlFilePath `\u003cstring\u003e` --filePrefix `\u003cstring\u003e`\n\nExample: `node checker.js --crawlFilePath /Users/magalibautista/workspace/rangle/a11y-crawler-poc/src/crawls/www.yahoo.ca/2021-01-15T18-44-23.571Z/www.yahoo.ca.txt --filePrefix yahoo`\n\n- The scans folder will be created in ./src/public/ \n- A folder named with the hostname will be created in /scans\n\n## TODOs\n- allow more parameters to be passed to the axe core library for different kinds of scans\n- add an option to run Puppeteer headless (flag or dev environment var)\n- allow a user to upload a sitemap-type file to bypass crawling (FUTURE) **\n- make the front-end prettier\n- ensure there is no timeout when launching scan from the frontend (long polling?)\n- progress bar (via sockets) (or you can notify the user - via email)\n- potentially generate partial results right away\n\n## Notes\n- could become mini saas application\n- /public should be outside of /src\n- make sure ./scans folder exists (or create it) . (use makedirp module)\n- files should be stored outside of /src\n- how do we know when a page has loaded? \n- check why google.com keeps looping and if we can prevent that?\n- ** FRONT END MUST BE ACCESSIBLE **\n- could Cypress replace Puppeteer?","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frangle%2Fa11y-violations-crawler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frangle%2Fa11y-violations-crawler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frangle%2Fa11y-violations-crawler/lists"}