{"id":18605966,"url":"https://github.com/axllent/web-validator","last_synced_at":"2025-04-10T20:31:18.187Z","repository":{"id":145513131,"uuid":"239705324","full_name":"axllent/web-validator","owner":"axllent","description":"Validate website HTML \u0026 CSS, check links \u0026 resources","archived":false,"fork":false,"pushed_at":"2025-03-18T02:07:28.000Z","size":65,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"develop","last_synced_at":"2025-03-25T04:12:50.356Z","etag":null,"topics":["go","golang-application","validador"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/axllent.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-02-11T07:49:01.000Z","updated_at":"2025-03-18T02:07:32.000Z","dependencies_parsed_at":"2024-12-30T10:32:42.201Z","dependency_job_id":"fed39fe1-18fb-47d9-b18f-a9f6b4d139f4","html_url":"https://github.com/axllent/web-validator","commit_stats":null,"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/axllent%2Fweb-validator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/axllent%2Fweb-validator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/axllent%2Fweb-validator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/axllent%2Fweb-validator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/axllent","download_url":"https://codeload.github.com/axllent/web-validator/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248290033,"owners_count":21078923,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","golang-application","validador"],"created_at":"2024-11-07T02:23:44.892Z","updated_at":"2025-04-10T20:31:18.143Z","avatar_url":"https://github.com/axllent.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Validate website HTML \u0026 CSS, check links \u0026 resources\n\n[![Go Report Card](https://goreportcard.com/badge/github.com/axllent/web-validator)](https://goreportcard.com/report/github.com/axllent/web-validator)\n\nA command-line website validator for Linux, Mac \u0026 Windows, which can spider through a website, \nvalidating all HTML \u0026 CSS pages, check the existence of all assets (images, css, fonts etc), \nand verify outbound links.\n\n\n## Features\n\n- Check a single URL, to a certain depth, or an entire website\n- HTML \u0026 CSS validation using (default) the [Nu Html Checker](https://validator.w3.org/)\n- Detect \u0026 checks linked assets from HTML \u0026 linked CSS (fonts, favicons, images, videos, etc)\n- Detect mixed content (HTTPS =\u003e HTTP) for linked assets (fonts, images, CSS, JS etc)\n- Verify outbound links (to external websites)\n- Summary report or errors (\u0026 optionally HTML/CSS warnings)\n- Obeys `robots.txt` (can be ignored)\n\n\n## Usage options\n\n```shell\nUsage: web-validator [options] \u003curl\u003e\n\nOptions:\n  -a, --all                recursive, follow all internal links (default single URL)\n  -d, --depth int          crawl depth (\"-a\" will override this)\n  -o, --outbound           check outbound links (HEAD only)\n      --html               validate HTML\n      --css                validate CSS\n  -i, --ignore string      ignore URLs, comma-separated, wildcards allowed (*.jpg,example.com)\n  -n, --no-robots          ignore robots.txt (if exists)\n  -r, --redirects          treat redirects as errors\n  -w, --warnings           display validation warnings (default errors only)\n  -f, --full               full scan (same as \"-a -r -o --html --css\")\n  -t, --threads int        number of threads (default 5)\n      --timeout int        timeout in seconds (default 10)\n      --validator string   Nu Html validator (default \"https://validator.w3.org/nu/\")\n  -u, --update             update to latest release\n  -v, --version            show app version\n```\n\n\n## Examples\n\n- `web-validator https://example.com/` - scan URL, verify all direct assets \u0026 links\n- `web-validator https://example.com/ --css --html` - scan URL, verify all direct assets \u0026 links, validate HTML \u0026 CSS\n- `web-validator https://example.com/ -a` - scan entire site, verify assets \u0026 links\n- `web-validator https://example.com/ --css --html -d 2` - scan site to a depth of 2 internal links, verify assets \u0026 links, validate HTML and CSS\n- `web-validator https://example.com/ -a -o` - scan entire site, verify all assets, verify outbound links\n- `web-validator https://example.com/ -f` - scan entire site, verify all assets, verify outbound links, validate HTML \u0026 CSS\n\n\n## Installing\n\nDownload the [latest binary release](https://github.com/axllent/web-validator/releases/latest) for your system, \nor build from source `go get -u github.com/axllent/web-validator`(go \u003e= 1.11 required)\n\n\n## FAQ\n\n### When I scan a single page, web-validator scans many other pages too\n\nWhen scanning a page, the software will check all internal links from that single page, which include both pages and files. Only a HEAD request is done on linked pages (no validation etc) to check for a valid response.\n\n\n### Web-validator says some of my outbound links are broken, however they do work\n\nSome sites specifically block all HEAD requests, in which case web-validator will try a regular GET request. Some sites however go to extreme lengths to prevent any kind of scraping, such as LinkedIn, so these will always return an error response. LinkedIn (specifically) is now blacklisted in the application, so any linkedin links are completely ignored. If you come across another major site with similar issues, then let me know and I will add them to the list.\n\n\n### HTML/CSS validation\n\nValidation uses the [Nu Html validator](\"https://validator.w3.org/nu/\"), and by default uses the online public service (they [encourage this](https://github.com/validator/validator/wiki/Service-%C2%BB-Input-%C2%BB-POST-body)). You can however use your [own instance](https://validator.w3.org/docs/users.html) of the validator (open source), and use the `--validator \u003cyour-server\u003e` to specify your own.\n\n\n### Robots.txt\n\nBy default, web-validator obeys `Disallow` rules in `robots.txt` if it exists. You can optionally skip this by adding `-n` to your runtime flags. To add specific rules for just the validator, you can target it specifically with `User-agent: web-validator`, eg:\n\n```\nUser-agent: web-validator\nDisallow: /assets/Products/*\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faxllent%2Fweb-validator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faxllent%2Fweb-validator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faxllent%2Fweb-validator/lists"}