{"id":15649617,"url":"https://github.com/spk/validate-website","last_synced_at":"2025-04-23T13:45:45.798Z","repository":{"id":626951,"uuid":"235533","full_name":"spk/validate-website","owner":"spk","description":"Web crawler for checking the validity of your documents.","archived":false,"fork":false,"pushed_at":"2023-09-13T20:59:53.000Z","size":956,"stargazers_count":38,"open_issues_count":0,"forks_count":9,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-03-14T20:51:25.706Z","etag":null,"topics":["html","validator","web-crawler"],"latest_commit_sha":null,"homepage":"https://spk.github.com/validate-website/","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/spk.png","metadata":{"files":{"readme":"README.md","changelog":"History.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2009-06-24T21:12:59.000Z","updated_at":"2024-06-20T00:08:37.727Z","dependencies_parsed_at":"2024-06-20T00:08:32.690Z","dependency_job_id":"f8618670-cbc4-4f9e-a466-0b0dbd8a9269","html_url":"https://github.com/spk/validate-website","commit_stats":{"total_commits":374,"total_committers":10,"mean_commits":37.4,"dds":0.06951871657754005,"last_synced_commit":"473b40852393fd578ff295aaf27892b0db591fcf"},"previous_names":[],"tags_count":46,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spk%2Fvalidate-website","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spk%2Fvalidate-website/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spk%2Fvalidate-website/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spk%2Fvalidate-website/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/spk","download_url":"https://codeload.github.com/spk/validate-website/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250442212,"owners_count":21431282,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["html","validator","web-crawler"],"created_at":"2024-10-03T12:30:29.194Z","updated_at":"2025-04-23T13:45:45.783Z","avatar_url":"https://github.com/spk.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# validate-website\n\n## Description\n\nWeb crawler for checking the validity of your documents\n\n![validate website](https://raw.github.com/spk/validate-website/master/validate-website.png)\n\n## Installation\n\n### Debian\n\n```\napt install ruby-dev libxslt1-dev libxml2-dev\n```\n\nIf you want complete local validation look [tidy\npackages](https://binaries.html-tidy.org/)\n\n### RubyGems\n\n```\ngem install validate-website\n```\n\n## Synopsis\n\n```\nvalidate-website [OPTIONS]\nvalidate-website-static [OPTIONS]\n```\n\n## Examples\n\n```\nvalidate-website -v -s https://www.ruby-lang.org/\nvalidate-website -v -x tidy -s https://www.ruby-lang.org/\nvalidate-website -v -x nu -s https://www.ruby-lang.org/\nvalidate-website -h\n```\n\n## Description\n\nvalidate-website is a web crawler for checking the markup validity with XML\nSchema / DTD and not found urls (more info [doc/validate-website.adoc](https://github.com/spk/validate-website/blob/master/doc/validate-website.adoc)).\n\nvalidate-website-static checks the markup validity of your local documents with\nXML Schema / DTD (more info [doc/validate-website-static.adoc](https://github.com/spk/validate-website/blob/master/doc/validate-website-static.adoc)).\n\nHTML5 support with [libtidy5](http://www.html-tidy.org/) or [Validator.nu Web\nService](https://checker.html5.org/).\n\n## Exit status\n\n* 0: Markup is valid and no 404 found.\n* 64: Not valid markup found.\n* 65: There are pages not found.\n* 66: There are not valid markup and pages not found.\n\n## On your application\n\n``` ruby\nrequire 'validate_website/validator'\nbody = '\u003c!DOCTYPE html\u003e\u003chtml\u003e\u003c/html\u003e'\nv = ValidateWebsite::Validator.new(Nokogiri::HTML(body), body)\nv.valid? # =\u003e false\n```\n\n## Jekyll static site validation\n\nYou can add this Rake task to validate a\n[jekyll](https://github.com/jekyll/jekyll) site:\n\n``` ruby\ndesc 'validate _site with validate website'\ntask validate: :build do\n    Dir.chdir(\"_site\") do\n      system(\"validate-website-static\",\n               \"--verbose\",\n               \"--exclude\", \"examples\",\n               \"--site\", HTTP_URL)\n      exit($?.exitstatus)\n    end\n  end\nend\n```\n\n## More info\n\n### HTML5\n\n#### Tidy5\n\nIf the libtidy5 is found on your system this will be the default to validate\nyour html5 document. This does not depend on a tier service everything is done\nlocally.\n\n#### nokogiri\n\nnokogiri can validate html5 document without tier service but reports less\nerrors than tidy.\n\n#### Validator.nu web service\n\nWhen `--html5-validator nu` option is used HTML5 support is done by using the\nValidator.nu Web Service, so the content of your webpage is logged by a tier.\nIt's not the case for other validation because validate-website use the XML\nSchema or DTD stored on the data/ directory.\n\nPlease read \u003chttp://about.validator.nu/#tos\u003e for more info on the HTML5\nvalidation service.\n\n##### Use validator standalone web server locally\n\nYou can download [validator](https://github.com/validator/validator) jar and\nstart it with:\n\n```\njava -cp PATH_TO/vnu.jar nu.validator.servlet.Main 8888\n```\n\nThen you can use validate-website option:\n\n```\n--html5-validator-service-url http://localhost:8888/\n# or\nexport VALIDATOR_NU_URL=\"http://localhost:8888/\"\n```\n\nThis will prevent you to be blacklisted from validator webservice.\n\n## Tests\n\nWith standard environment:\n\n```\nbundle exec rake\n```\n\n## Credits\n\n* Thanks tenderlove for Nokogiri, this tool is inspired from markup_validity.\n* And Chris Kite for Anemone web-spider framework and postmodern for Spidr.\n\n## Contributors\n\nSee [GitHub](https://github.com/spk/validate-website/graphs/contributors).\n\n## License\n\nThe MIT License\n\nCopyright (c) 2009-2022 Laurent Arnoud \u003claurent@spkdev.net\u003e\n\n---\n[![Build](https://img.shields.io/gitlab/pipeline/spkdev/validate-website/master)](https://gitlab.com/spkdev/validate-website/-/commits/master)\n[![Coverage](https://gitlab.com/spkdev/validate-website/badges/master/coverage.svg)](https://gitlab.com/spkdev/validate-website/-/commits/master)\n[![Version](https://img.shields.io/gem/v/validate-website.svg)](https://rubygems.org/gems/validate-website)\n[![Documentation](https://img.shields.io/badge/doc-rubydoc-blue.svg)](http://www.rubydoc.info/gems/validate-website)\n[![License](https://img.shields.io/badge/license-MIT-blue.svg)](http://opensource.org/licenses/MIT \"MIT\")\n[![Inline docs](https://inch-ci.org/github/spk/validate-website.svg?branch=master)](http://inch-ci.org/github/spk/validate-website)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspk%2Fvalidate-website","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fspk%2Fvalidate-website","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspk%2Fvalidate-website/lists"}