{"id":24591959,"url":"https://github.com/4rlm/url_verifier","last_synced_at":"2026-04-22T21:37:34.750Z","repository":{"id":59158790,"uuid":"138944313","full_name":"4rlm/url_verifier","owner":"4rlm","description":"Ruby Gem: Format, Verify \u0026 Follow URL redirects with detailed reports.","archived":false,"fork":false,"pushed_at":"2018-07-11T19:33:33.000Z","size":96,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-01-18T08:46:26.085Z","etag":null,"topics":["4rlm","adam-booth","adam-john-booth","curation","curl","formatter","http","normalization","redirect","redirecting-urls","redirects","ruby-gem","status-codes","url","url-verifier","urlverifier","validation","verification","verifier","verify"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/4rlm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-06-27T23:46:13.000Z","updated_at":"2018-07-11T19:33:35.000Z","dependencies_parsed_at":"2022-09-13T20:11:36.818Z","dependency_job_id":null,"html_url":"https://github.com/4rlm/url_verifier","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/4rlm%2Furl_verifier","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/4rlm%2Furl_verifier/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/4rlm%2Furl_verifier/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/4rlm%2Furl_verifier/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/4rlm","download_url":"https://codeload.github.com/4rlm/url_verifier/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244147197,"owners_count":20405940,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["4rlm","adam-booth","adam-john-booth","curation","curl","formatter","http","normalization","redirect","redirecting-urls","redirects","ruby-gem","status-codes","url","url-verifier","urlverifier","validation","verification","verifier","verify"],"created_at":"2025-01-24T10:13:16.211Z","updated_at":"2026-04-22T21:37:34.712Z","avatar_url":"https://github.com/4rlm.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# UrlVerifier\n\n[![Build Status](https://travis-ci.org/4rlm/url_verifier.svg?branch=master)](https://travis-ci.org/4rlm/url_verifier)\n[![Gem Version](https://badge.fury.io/rb/url_verifier.svg)](https://badge.fury.io/rb/url_verifier)\n[![MIT License](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n\n\n### Format, Verify \u0026 Follow URL redirects with detailed reports.\n\nFollowing url redirects can sometimes take a few minutes and often creates various exceptions.  UrlVerifier is built with exceptional error handling, reformatting, and optional time limits you can set; default is set to 60 sec limit, but typically only takes 5-10 seconds per url.  UrlVerifier has been developed and improved upon for several years in an enterprise level app and is now available as an open source gem.  It is perfect for high-volume, yet smooth, uninterrupted url formatting and verification.\n\nExample:\n\n```\n{\n  url: 'blackwellford.com/staff',\n  verified_url: 'https://www.blackwellford.com',\n  url_path: '/staff',\n  response_code:'200',\n  url_redirected: true,\n  url_sts: 'Valid'\n}\n```\n\n## Installation\n\nAdd this line to your application's Gemfile:\n\n```ruby\ngem 'url_verifier'\n```\n\nAnd then execute:\n\n    $ bundle\n\nOr install it yourself as:\n\n    $ gem install url_verifier\n\n## Usage\n\n### Available Methods\n\n#### 1. Verify one URL as a String:\n\n```\nverifier = UrlVerifier::RunCurler.new\nverified_url_hash = verifier.verify_url('example.com')\n```\n\n#### 2. Verify Array of URL Strings:\n\n```\nverifier = UrlVerifier::RunCurler.new\nverified_url_hashes = verifier.verify_urls(array_of_urls)\n```\n\n\n### Usage Example\n\n```\narray_of_urls = %w[blackwellford.com/staff www.mccrea.subaru.com/inventory www.sofake.sofake https://www.century1chevy.com https://www.mccreasubaru.com]\n\nargs = { timeout_limit: 30 }\nverifier = UrlVerifier::RunCurler.new(args)\nverified_url_hashes = verifier.verify_urls(array_of_urls)\n```\nNote: `:timeout_limit` default is 60 seconds per url, but most urls take under 10 sec each.  You can override the default by passing in your desired limit.  If 60 seconds is fine, no need to pass any args.  You could simply instantiate like this:\n\n```\nverifier = UrlVerifier::RunCurler.new\nverified_url_hashes = verifier.verify_urls(array_of_urls)\n```\n\nReturns Data in Hash Format with detailed report:\n\nNotice the URLs in the input array above were NOT uniformly formatted.  UrlVerifier leverages the `utf8_sanitizer gem` and `crm_formatter gem` to pre-format the URLs before verifying, so the results will be uniformly formatted, like in the hashes below `:url_f`.  \n\n`:verified_url` is the final verified URL.  `:url_redirected` indicates that the verified URL is different than `:url_f`.  \n\nIf `url_sts: 'Invalid'`, `:wx_date` will be timestamped, which helps keep track of when it became invalid, incase you are running periodic database URL verifications and want to include these details in your reports.\n\n`:response_code` in the 200's is ideal.  If it has recently been forwarded it will be in the 300's, and 400's indicates an issue with the URL domain or server.  Some 400's could be run later when they resolve their issues, so don't always give up on them.  \n\nHere is a reference guide: [List of HTTP status codes](https://en.wikipedia.org/wiki/List_of_HTTP_status_codes)\n\n```\nverified_url_hashes = [\n  {\n    :web_status=\u003e\"formatted\",\n    :url=\u003e\"blackwellford.com/staff\",\n    :url_f=\u003e\"http://www.blackwellford.com\",\n    :url_path=\u003e\"/staff\",\n    :web_neg=\u003enil,\n    :verified_url=\u003e\"https://www.blackwellford.com\",\n    :url_redirected=\u003etrue,\n    :response_code=\u003e\"200\",\n    :url_sts=\u003e\"Valid\",\n    :url_date=\u003e2018-07-02 09:16:19 -0500,\n    :wx_date=\u003enil,\n    :timeout=\u003e0\n  },\n  {\n    :web_status=\u003e\"formatted\",\n    :url=\u003e\"www.mccrea.subaru.com/inventory\",\n    :url_f=\u003e\"http://www.mccrea.subaru.com\",\n    :url_path=\u003enil,\n    :web_neg=\u003enil,\n    :verified_url=\u003e\"https://www.mccreasubaru.com\",\n    :url_redirected=\u003etrue,\n    :response_code=\u003e\"200\",\n    :url_sts=\u003e\"Valid\",\n    :url_date=\u003e2018-07-02 09:16:38 -0500,\n    :wx_date=\u003enil,\n    :timeout=\u003e0\n  },\n  {\n    :web_status=\u003e\"invalid\",\n    :url=\u003e\"www.sofake.sofake\",\n    :url_f=\u003enil,\n    :url_path=\u003enil,\n    :web_neg=\u003e\"error: ext.invalid [sofake]\",\n    :verified_url=\u003enil,\n    :url_redirected=\u003efalse,\n    :response_code=\u003enil,\n    :url_sts=\u003e\"Invalid\",\n    :url_date=\u003e2018-07-02 09:16:58 -0500,\n    :wx_date=\u003e2018-07-02 09:16:58 -0500,\n    :timeout=\u003enil\n  },\n  {\n    :web_status=\u003e\"unchanged\",\n    :url=\u003e\"https://www.century1chevy.com\",\n    :url_f=\u003e\"https://www.century1chevy.com\",\n    :url_path=\u003enil,\n    :web_neg=\u003enil,\n    :verified_url=\u003e\"http://www.centurychevy.com\",\n    :url_redirected=\u003etrue,\n    :response_code=\u003e\"405\",\n    :url_sts=\u003e\"Valid\",\n    :url_date=\u003e2018-07-02 09:16:58 -0500,\n    :wx_date=\u003enil,\n    :timeout=\u003e0\n  },\n  {\n    :web_status=\u003e\"unchanged\",\n    :url=\u003e\"https://www.mccreasubaru.com\",\n    :url_f=\u003e\"https://www.mccreasubaru.com\",\n    :url_path=\u003enil,\n    :web_neg=\u003enil,\n    :verified_url=\u003e\"https://www.mccreasubaru.com\",\n    :url_redirected=\u003efalse,\n    :response_code=\u003e\"200\",\n    :url_sts=\u003e\"Valid\",\n    :url_date=\u003e2018-07-02 09:16:59 -0500,\n    :wx_date=\u003enil,\n    :timeout=\u003e0\n  }\n]\n```\n\n## Author\n\nAdam J Booth  - [4rlm](https://github.com/4rlm)\n\n\n## Development\n\nAfter checking out the repo, verify `bin/setup` to install dependencies. Then, verify `rake spec` to verify the tests. You can also verify `bin/console` for an interactive prompt that will allow you to experiment.\n\nTo install this gem onto your local machine, verify `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then verify `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).\n\n## Contributing\n\nBug reports and pull requests are welcome on GitHub at https://github.com/4rlm/url_verifier. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.\n\n## License\n\nThe gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).\n\n## Code of Conduct\n\nEveryone interacting in the UrlVerifier project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/4rlm/url_verifier/blob/master/CODE_OF_CONDUCT.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F4rlm%2Furl_verifier","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F4rlm%2Furl_verifier","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F4rlm%2Furl_verifier/lists"}