{"id":33916517,"url":"https://github.com/opensite-ai/domain_extractor","last_synced_at":"2025-12-12T07:22:41.177Z","repository":{"id":321736935,"uuid":"1086974688","full_name":"opensite-ai/domain_extractor","owner":"opensite-ai","description":"🔗 Lightweight Ruby library for parsing URLs and extracting domain components with accurate multi-part TLD support. Handles nested subdomains, query parameters, and URL normalization. Perfect for web scraping, analytics, and URL manipulation. Built on URI and public_suffix gem.","archived":false,"fork":false,"pushed_at":"2025-11-25T05:24:00.000Z","size":202,"stargazers_count":6,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-11-28T13:58:48.441Z","etag":null,"topics":["analytics","domain-analysis","domain-extraction","domain-parser","public-suffix","ruby","ruby-library","rubygem","subdomain-parser","tld-parser","url-manipulation","url-normalization","url-parser","url-parsing","web-scraping"],"latest_commit_sha":null,"homepage":"https://opensite.ai/developers","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/opensite-ai.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-31T07:22:49.000Z","updated_at":"2025-11-25T05:24:03.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/opensite-ai/domain_extractor","commit_stats":null,"previous_names":["opensite-ai/domain_extractor"],"tags_count":19,"template":false,"template_full_name":null,"purl":"pkg:github/opensite-ai/domain_extractor","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opensite-ai%2Fdomain_extractor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opensite-ai%2Fdomain_extractor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opensite-ai%2Fdomain_extractor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opensite-ai%2Fdomain_extractor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/opensite-ai","download_url":"https://codeload.github.com/opensite-ai/domain_extractor/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opensite-ai%2Fdomain_extractor/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":27678747,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-12T02:00:06.775Z","response_time":129,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["analytics","domain-analysis","domain-extraction","domain-parser","public-suffix","ruby","ruby-library","rubygem","subdomain-parser","tld-parser","url-manipulation","url-normalization","url-parser","url-parsing","web-scraping"],"created_at":"2025-12-12T07:22:40.082Z","updated_at":"2025-12-12T07:22:41.170Z","avatar_url":"https://github.com/opensite-ai.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"![Domain Extractor and Parsing Ruby Gem](https://github.com/user-attachments/assets/b3fbe605-3a3e-4d45-b0ed-b061a807d61b)\n\n# DomainExtractor\n\n[![Gem Version](https://badge.fury.io/rb/domain_extractor.svg?v=020)](https://badge.fury.io/rb/domain_extractor)\n[![CI](https://github.com/opensite-ai/domain_extractor/actions/workflows/ci.yml/badge.svg)](https://github.com/opensite-ai/domain_extractor/actions/workflows/ci.yml)\n\nA lightweight, robust Ruby library for url parsing and domain parsing with **accurate multi-part TLD support**. DomainExtractor delivers a high-throughput url parser and domain parser that excels at domain extraction tasks while staying friendly to analytics pipelines. Perfect for web scraping, analytics, url manipulation, query parameter parsing, and multi-environment domain analysis.\n\nUse **DomainExtractor** whenever you need a dependable tld parser for tricky multi-part tld registries or reliable subdomain extraction in production systems.\n\n## Why DomainExtractor?\n\n✅ **Accurate Multi-part TLD Parser** - Handles complex multi-part TLDs (co.uk, com.au, gov.br) using the [Public Suffix List](https://publicsuffix.org/)\n✅ **Nested Subdomain Extraction** - Correctly parses multi-level subdomains (api.staging.example.com)\n✅ **Smart URL Normalization** - Automatically handles URLs with or without schemes\n✅ **Powerful URL Formatting** - Transform and standardize URLs with flexible options\n✅ **Rails Integration** - Custom ActiveModel validator for declarative URL validation\n✅ **Query Parameter Parsing** - Parse query strings into structured hashes\n✅ **Batch Processing** - Parse multiple URLs efficiently\n✅ **IP Address Detection** - Identifies and handles IPv4 and IPv6 addresses\n✅ **Zero Configuration** - Works out of the box with sensible defaults\n✅ **Well-Tested** - Comprehensive test suite covering edge cases\n\n## Installation\n\nAdd this line to your application's Gemfile:\n\n```ruby\ngem 'domain_extractor'\n```\n\nAnd then execute:\n\n```bash\n$ bundle install\n```\n\nOr install it yourself:\n\n```bash\n$ gem install domain_extractor\n```\n\n## Quick Start\n\n```ruby\nrequire 'domain_extractor'\n\n# Parse a URL\nresult = DomainExtractor.parse('https://www.example.co.uk/path?query=value')\n\nresult[:subdomain]    # =\u003e 'www'\nresult[:domain]       # =\u003e 'example'\nresult[:tld]          # =\u003e 'co.uk'\nresult[:root_domain]  # =\u003e 'example.co.uk'\nresult[:host]         # =\u003e 'www.example.co.uk'\n\n# Guard a parse with the validity helper\nurl = 'https://www.example.co.uk/path?query=value'\nif DomainExtractor.valid?(url)\n  DomainExtractor.parse(url)\nelse\n  # handle invalid input\nend\n\n# New intuitive method-style access\nresult.subdomain      # =\u003e 'www'\nresult.domain         # =\u003e 'example'\nresult.host           # =\u003e 'www.example.co.uk'\n```\n\n## ParsedURL API - Intuitive Method Access\n\nDomainExtractor now returns a `ParsedURL` object that supports three accessor styles, making your intent clear and your code more robust:\n\n### Method Accessor Styles\n\n#### 1. Default Methods (Silent Nil)\n\nReturns the value or `nil` - perfect for exploratory code or when handling invalid data gracefully.\n\n```ruby\nresult = DomainExtractor.parse('https://api.example.com')\nresult.subdomain    # =\u003e 'api'\nresult.domain       # =\u003e 'example'\nresult.host         # =\u003e 'api.example.com'\n\n# Without subdomain\nresult = DomainExtractor.parse('https://example.com')\nresult.subdomain    # =\u003e nil (no error)\nresult.domain       # =\u003e 'example'\n```\n\n#### 2. Bang Methods (!) - Explicit Errors\n\nReturns the value or raises `InvalidURLError` - ideal for production code where missing data should fail fast.\n\n```ruby\nresult = DomainExtractor.parse('https://example.com')\nresult.domain!      # =\u003e 'example'\nresult.subdomain!   # raises InvalidURLError: \"subdomain not found or invalid\"\n```\n\n#### 3. Question Methods (?) - Boolean Checks\n\nAlways returns `true` or `false` - perfect for conditional logic without exceptions.\n\n```ruby\nDomainExtractor.parse('https://dashtrack.com').subdomain?        # =\u003e false\nDomainExtractor.parse('https://api.dashtrack.com').subdomain?   # =\u003e true\nDomainExtractor.parse('https://www.dashtrack.com').www_subdomain? # =\u003e true\n```\n\n### Quick Examples\n\n```ruby\nurl = 'https://api.staging.example.com/path'\nparsed = DomainExtractor.parse(url)\n\n# Method-style access\nparsed.host           # =\u003e 'api.staging.example.com'\nparsed.subdomain      # =\u003e 'api.staging'\nparsed.domain         # =\u003e 'example'\nparsed.root_domain    # =\u003e 'example.com'\nparsed.tld            # =\u003e 'com'\nparsed.path           # =\u003e '/path'\n\n# Question methods for conditionals\nif parsed.subdomain?\n  puts \"Has subdomain: #{parsed.subdomain}\"\nend\n\n# Bang methods when values are required\nbegin\n  subdomain = parsed.subdomain!  # Safe - has subdomain\n  domain = parsed.domain!        # Safe - has domain\nrescue DomainExtractor::InvalidURLError =\u003e e\n  puts \"Missing required component: #{e.message}\"\nend\n\n# Hash-style access still works (backward compatible)\nparsed[:subdomain]    # =\u003e 'api.staging'\nparsed[:host]         # =\u003e 'api.staging.example.com'\n```\n\n### Additional Examples\n\n#### Boolean Checks with Question Methods\n\n```ruby\n# Check for subdomain presence\nDomainExtractor.parse('https://dashtrack.com').subdomain?        # =\u003e false\nDomainExtractor.parse('https://api.dashtrack.com').subdomain?   # =\u003e true\n\n# Check for www subdomain specifically\nDomainExtractor.parse('https://www.dashtrack.com').www_subdomain? # =\u003e true\nDomainExtractor.parse('https://api.dashtrack.com').www_subdomain? # =\u003e false\n\n```\n\n#### Handling Unknown or Invalid Data\n\n```ruby\n# Default accessors fail silently with nil\nDomainExtractor.parse(nil).domain                 # =\u003e nil\nDomainExtractor.parse('').host                    # =\u003e nil\nDomainExtractor.parse('asdfasdfds').domain        # =\u003e nil\n\n# Boolean checks never raise\nDomainExtractor.parse(nil).subdomain?             # =\u003e false\nDomainExtractor.parse('').domain?                 # =\u003e false\nDomainExtractor.parse('https://dashtrack.com').subdomain? # =\u003e false\n\n# Bang methods raise when a component is missing\nDomainExtractor.parse('').host!                   # =\u003e raises DomainExtractor::InvalidURLError\nDomainExtractor.parse('asdfasdfds').domain!       # =\u003e raises DomainExtractor::InvalidURLError\n```\n\n#### Safe Batch Processing\n\n```ruby\nurls = [\n  'https://api.example.com',\n  'https://example.com',\n  'https://www.example.com'\n]\n\nurls.each do |url|\n  result = DomainExtractor.parse(url)\n\n  info = {\n    url: url,\n    has_subdomain: result.subdomain?,\n    is_www: result.www_subdomain?,\n    host: result.host\n  }\n\n  puts \"#{info[:url]} - subdomain: #{info[:has_subdomain]}, www: #{info[:is_www]}\"\nend\n```\n\n#### Production URL Validation\n\n```ruby\ndef validate_api_url(url)\n  result = DomainExtractor.parse(url)\n\n  # Ensure all required components exist\n  result.subdomain!  # Must have subdomain\n  result.domain!     # Must have domain\n\n  # Additional validation\n  return false unless result.subdomain.start_with?('api')\n\n  true\nrescue DomainExtractor::InvalidURLError =\u003e e\n  puts \"Validation failed: #{e.message}\"\n  false\nend\n\nvalidate_api_url('https://api.example.com/endpoint')  # =\u003e true\nvalidate_api_url('https://example.com/endpoint')      # =\u003e false (no subdomain)\nvalidate_api_url('https://www.example.com/endpoint')  # =\u003e false (not api subdomain)\n```\n\n#### Guard Clauses with Question Methods\n\n```ruby\ndef process_url(url)\n  result = DomainExtractor.parse(url)\n\n  return 'Invalid URL' unless result.valid?\n  return 'No subdomain present' unless result.subdomain?\n  return 'WWW redirect needed' if result.www_subdomain?\n\n  \"Processing subdomain: #{result.subdomain}\"\nend\n\nprocess_url('https://api.example.com')  # =\u003e \"Processing subdomain: api\"\nprocess_url('https://www.example.com')  # =\u003e \"WWW redirect needed\"\nprocess_url('https://example.com')      # =\u003e \"No subdomain present\"\n```\n\n#### Converting to Hash\n\n```ruby\nurl = 'https://api.example.com/path'\nresult = DomainExtractor.parse(url)\n\nhash = result.to_h\n# =\u003e {\n#   subdomain: \"api\",\n#   domain: \"example\",\n#   tld: \"com\",\n#   root_domain: \"example.com\",\n#   host: \"api.example.com\",\n#   path: \"/path\",\n#   query_params: {}\n# }\n```\n\n**[Comprehensive documentation and real-world examples of parsed URL quick start guide](https://github.com/opensite-ai/domain_extractor/blob/master/docs/PARSED_URL_QUICK_START.md)**\n\n## Usage Examples\n\n### Basic Domain Parsing\n\n```ruby\n# Parse a simple domain (fast domain extraction)\nDomainExtractor.parse('example.com')\n# =\u003e { subdomain: nil, domain: 'example', tld: 'com', ... }\n\n# Parse domain with subdomain\nDomainExtractor.parse('blog.example.com')\n# =\u003e { subdomain: 'blog', domain: 'example', tld: 'com', ... }\n```\n\n### Multi-Part TLD Support\n\n```ruby\n# UK domain\nDomainExtractor.parse('www.bbc.co.uk')\n# =\u003e { subdomain: 'www', domain: 'bbc', tld: 'co.uk', ... }\n\n# Australian domain\nDomainExtractor.parse('shop.example.com.au')\n# =\u003e { subdomain: 'shop', domain: 'example', tld: 'com.au', ... }\n```\n\n### Nested Subdomains\n\n```ruby\nDomainExtractor.parse('api.staging.example.com')\n# =\u003e { subdomain: 'api.staging', domain: 'example', tld: 'com', ... }\n```\n\n### Query Parameter Parsing\n\n```ruby\nparams = DomainExtractor.parse_query_params('?utm_source=google\u0026page=1')\n# =\u003e { 'utm_source' =\u003e 'google', 'page' =\u003e '1' }\n\n# Or via the shorter helper\nDomainExtractor.parse_query('?search=ruby\u0026flag')\n# =\u003e { 'search' =\u003e 'ruby', 'flag' =\u003e nil }\n```\n\n### Batch URL Processing\n\n```ruby\nurls = ['https://example.com', 'https://blog.example.org']\nresults = DomainExtractor.parse_batch(urls)\n```\n\n### Validation and Error Handling\n\n```ruby\nDomainExtractor.valid?('https://www.example.com') # =\u003e true\n\n# DomainExtractor.parse raises DomainExtractor::InvalidURLError on invalid input\nDomainExtractor.parse('not-a-url')\n# =\u003e raises DomainExtractor::InvalidURLError (message: \"Invalid URL Value\")\n```\n\n## API Reference\n\n```ruby\nDomainExtractor.parse(url_string)\n\n# =\u003e Parses a URL string and extracts domain components.\n\n# Returns: Hash with keys :subdomain, :domain, :tld, :root_domain, :host, :path\n# Raises: DomainExtractor::InvalidURLError when the URL fails validation\n```\n\n```ruby\nDomainExtractor.parse_batch(urls)\n\n# =\u003e Parses multiple URLs efficiently.\n\n# Returns: Array of parsed results\n```\n\n```ruby\nDomainExtractor.valid?(url_string)\n\n# =\u003e Checks if a URL can be parsed successfully without raising.\n\n# Returns: true or false\n```\n\n```ruby\nDomainExtractor.parse_query_params(query_string)\n\n# =\u003e Parses a query string into a hash of parameters.\n\n# Returns: Hash of query parameters\n```\n\n```ruby\nDomainExtractor.format(url_string, **options)\n\n# =\u003e Formats a URL according to the specified options.\n\n# Returns: Formatted URL string or nil if invalid\n# Options:\n#   :validation (:standard, :root_domain, :root_or_custom_subdomain)\n#   :use_protocol (true/false)\n#   :use_https (true/false)\n#   :use_trailing_slash (true/false)\n```\n\n## URL Formatting\n\nDomainExtractor provides powerful URL formatting capabilities to normalize, transform, and standardize URLs according to your application's requirements.\n\n### Basic Formatting\n\n```ruby\n# Remove trailing slash (default)\nDomainExtractor.format('https://example.com/')\n# =\u003e 'https://example.com'\n\n# Strip paths and query parameters\nDomainExtractor.format('https://example.com/path?query=value')\n# =\u003e 'https://example.com'\n\n# Normalize to HTTPS\nDomainExtractor.format('http://example.com')\n# =\u003e 'https://example.com'\n```\n\n### Validation Modes\n\n#### Standard Mode (Default)\n\nPreserves the full host as-is while normalizing protocol and trailing slashes.\n\n```ruby\nDomainExtractor.format('https://shop.example.com')\n# =\u003e 'https://shop.example.com'\n\nDomainExtractor.format('https://www.example.com/')\n# =\u003e 'https://www.example.com'\n\nDomainExtractor.format('https://api.staging.example.com')\n# =\u003e 'https://api.staging.example.com'\n```\n\n#### Root Domain Mode\n\nStrips all subdomains and returns only the root domain.\n\n```ruby\nDomainExtractor.format('https://shop.example.com', validation: :root_domain)\n# =\u003e 'https://example.com'\n\nDomainExtractor.format('https://www.example.com/', validation: :root_domain)\n# =\u003e 'https://example.com'\n\nDomainExtractor.format('https://api.staging.example.com', validation: :root_domain)\n# =\u003e 'https://example.com'\n\n# Works with multi-part TLDs\nDomainExtractor.format('https://shop.example.co.uk', validation: :root_domain)\n# =\u003e 'https://example.co.uk'\n```\n\n#### Root or Custom Subdomain Mode\n\nPreserves custom subdomains but specifically removes the 'www' subdomain.\n\n```ruby\nDomainExtractor.format('https://example.com', validation: :root_or_custom_subdomain)\n# =\u003e 'https://example.com'\n\nDomainExtractor.format('https://shop.example.com', validation: :root_or_custom_subdomain)\n# =\u003e 'https://shop.example.com'\n\n# Strips www subdomain\nDomainExtractor.format('https://www.example.com', validation: :root_or_custom_subdomain)\n# =\u003e 'https://example.com'\n\nDomainExtractor.format('https://api.example.com', validation: :root_or_custom_subdomain)\n# =\u003e 'https://api.example.com'\n```\n\n### Protocol Options\n\n#### Without Protocol\n\nRemove the protocol entirely from the output.\n\n```ruby\nDomainExtractor.format('https://example.com', use_protocol: false)\n# =\u003e 'example.com'\n\nDomainExtractor.format('https://shop.example.com', use_protocol: false)\n# =\u003e 'shop.example.com'\n\n# Combine with root_domain\nDomainExtractor.format('https://shop.example.com',\n                       validation: :root_domain,\n                       use_protocol: false)\n# =\u003e 'example.com'\n```\n\n#### HTTP vs HTTPS\n\nControl which protocol to use in the output.\n\n```ruby\n# Default: use HTTPS\nDomainExtractor.format('http://example.com')\n# =\u003e 'https://example.com'\n\n# Allow HTTP\nDomainExtractor.format('https://example.com', use_https: false)\n# =\u003e 'http://example.com'\n\nDomainExtractor.format('http://example.com', use_https: false)\n# =\u003e 'http://example.com'\n```\n\n### Trailing Slash Options\n\n```ruby\n# Remove trailing slash (default)\nDomainExtractor.format('https://example.com/')\n# =\u003e 'https://example.com'\n\n# Add trailing slash\nDomainExtractor.format('https://example.com', use_trailing_slash: true)\n# =\u003e 'https://example.com/'\n\nDomainExtractor.format('https://example.com/', use_trailing_slash: true)\n# =\u003e 'https://example.com/'\n\n# Works with other options\nDomainExtractor.format('https://shop.example.com',\n                       validation: :root_domain,\n                       use_trailing_slash: true)\n# =\u003e 'https://example.com/'\n```\n\n### Combined Options\n\nMix and match options for precise URL formatting:\n\n```ruby\n# Root domain, no protocol, with trailing slash\nDomainExtractor.format('https://shop.example.com/path',\n                       validation: :root_domain,\n                       use_protocol: false,\n                       use_trailing_slash: true)\n# =\u003e 'example.com/'\n\n# Strip www, use HTTP, with trailing slash\nDomainExtractor.format('https://www.example.com',\n                       validation: :root_or_custom_subdomain,\n                       use_https: false,\n                       use_trailing_slash: true)\n# =\u003e 'http://example.com/'\n\n# Standard mode, no protocol, with trailing slash\nDomainExtractor.format('https://api.example.com',\n                       use_protocol: false,\n                       use_trailing_slash: true)\n# =\u003e 'api.example.com/'\n```\n\n### Real-World Use Cases\n\n#### Canonical URL Generation\n\n```ruby\ndef canonical_url(url)\n  DomainExtractor.format(url,\n                         validation: :root_or_custom_subdomain,\n                         use_https: true,\n                         use_trailing_slash: false)\nend\n\ncanonical_url('http://www.example.com/')      # =\u003e 'https://example.com'\ncanonical_url('https://shop.example.com/')    # =\u003e 'https://shop.example.com'\n```\n\n#### Domain Normalization for Allowlists\n\n```ruby\ndef normalize_domain_for_allowlist(url)\n  DomainExtractor.format(url,\n                         validation: :root_domain,\n                         use_protocol: false)\nend\n\nnormalize_domain_for_allowlist('https://shop.example.com/path')  # =\u003e 'example.com'\nnormalize_domain_for_allowlist('http://www.example.com')         # =\u003e 'example.com'\n```\n\n#### Multi-Tenant URL Standardization\n\n```ruby\nclass Tenant \u003c ApplicationRecord\n  before_validation :normalize_custom_domain\n\n  private\n\n  def normalize_custom_domain\n    return if custom_domain.blank?\n\n    self.custom_domain = DomainExtractor.format(\n      custom_domain,\n      validation: :root_or_custom_subdomain,\n      use_https: true,\n      use_trailing_slash: false\n    )\n  end\nend\n```\n\n#### API Endpoint Formatting\n\n```ruby\ndef format_api_endpoint(url)\n  DomainExtractor.format(url,\n                         validation: :standard,\n                         use_https: true,\n                         use_trailing_slash: true)\nend\n\nformat_api_endpoint('http://api.example.com')  # =\u003e 'https://api.example.com/'\n```\n\n## Rails Integration\n\nDomainExtractor provides a custom ActiveModel validator for Rails applications, enabling declarative URL/domain validation with multiple modes and options.\n\n### Installation\n\nThe Rails validator is automatically available when using DomainExtractor in a Rails application (or any application with ActiveModel). No additional setup is required.\n\n### Basic Usage\n\n```ruby\nclass Website \u003c ApplicationRecord\n  # Standard validation - accepts any valid URL\n  validates :url, domain: { validation: :standard }\nend\n```\n\n### Validation Modes\n\n#### `:standard` - Accept Any Valid URL\n\nValidates that the URL is parseable and valid. This is the default mode.\n\n```ruby\nclass Website \u003c ApplicationRecord\n  validates :url, domain: { validation: :standard }\nend\n\n# Valid URLs\nwebsite = Website.new(url: 'https://mysite.com')        # ✅ Valid\nwebsite = Website.new(url: 'https://shop.mysite.com')   # ✅ Valid\nwebsite = Website.new(url: 'https://www.mysite.com')    # ✅ Valid\nwebsite = Website.new(url: 'https://api.staging.mysite.com') # ✅ Valid\n\n# Invalid URLs\nwebsite = Website.new(url: 'not-a-url')                 # ❌ Invalid\n```\n\n#### `:root_domain` - Root Domain Only\n\nOnly allows root domains without any subdomains.\n\n```ruby\nclass PrimaryDomain \u003c ApplicationRecord\n  validates :domain, domain: { validation: :root_domain }\nend\n\n# Valid URLs\ndomain = PrimaryDomain.new(domain: 'https://mysite.com')      # ✅ Valid\n\n# Invalid URLs\ndomain = PrimaryDomain.new(domain: 'https://shop.mysite.com') # ❌ Invalid (has subdomain)\ndomain = PrimaryDomain.new(domain: 'https://www.mysite.com')  # ❌ Invalid (has www subdomain)\n```\n\n#### `:root_or_custom_subdomain` - Root or Custom Subdomain (No WWW)\n\nAllows root domains or custom subdomains, but specifically excludes the 'www' subdomain.\n\n```ruby\nclass CustomDomain \u003c ApplicationRecord\n  validates :url, domain: { validation: :root_or_custom_subdomain }\nend\n\n# Valid URLs\ndomain = CustomDomain.new(url: 'https://mysite.com')       # ✅ Valid (root domain)\ndomain = CustomDomain.new(url: 'https://shop.mysite.com')  # ✅ Valid (custom subdomain)\ndomain = CustomDomain.new(url: 'https://api.mysite.com')   # ✅ Valid (custom subdomain)\n\n# Invalid URLs\ndomain = CustomDomain.new(url: 'https://www.mysite.com')   # ❌ Invalid (www not allowed)\n```\n\n### Protocol Options\n\n#### `use_protocol` (default: `true`)\n\nControls whether the protocol (http:// or https://) is required in the URL.\n\n```ruby\nclass Website \u003c ApplicationRecord\n  # Require protocol (default behavior)\n  validates :url, domain: { validation: :standard, use_protocol: true }\n\n  # Don't require protocol\n  validates :domain_without_protocol, domain: {\n    validation: :standard,\n    use_protocol: false\n  }\nend\n\n# With use_protocol: true (default)\nWebsite.new(url: 'https://mysite.com')  # ✅ Valid\nWebsite.new(url: 'mysite.com')          # ✅ Valid (auto-adds https://)\n\n# With use_protocol: false\nWebsite.new(domain_without_protocol: 'mysite.com')        # ✅ Valid\nWebsite.new(domain_without_protocol: 'https://mysite.com') # ✅ Valid (protocol stripped)\n```\n\n#### `use_https` (default: `true`)\n\nControls whether HTTPS is required. Only relevant when `use_protocol` is `true`.\n\n```ruby\nclass SecureWebsite \u003c ApplicationRecord\n  # Require HTTPS (default behavior)\n  validates :url, domain: { validation: :standard, use_https: true }\nend\n\nclass FlexibleWebsite \u003c ApplicationRecord\n  # Allow both HTTP and HTTPS\n  validates :url, domain: { validation: :standard, use_https: false }\nend\n\n# With use_https: true (default)\nSecureWebsite.new(url: 'https://mysite.com')  # ✅ Valid\nSecureWebsite.new(url: 'http://mysite.com')   # ❌ Invalid\n\n# With use_https: false\nFlexibleWebsite.new(url: 'https://mysite.com') # ✅ Valid\nFlexibleWebsite.new(url: 'http://mysite.com')  # ✅ Valid\n```\n\n### Real-World Examples\n\n#### Multi-Tenant Application with Custom Domains\n\n```ruby\nclass Tenant \u003c ApplicationRecord\n  # Allow custom subdomains but not www\n  validates :custom_domain, domain: {\n    validation: :root_or_custom_subdomain,\n    use_https: true\n  }\n\n  # Primary domain must be root only\n  validates :primary_domain, domain: {\n    validation: :root_domain,\n    use_protocol: false\n  }\nend\n\n# Valid configurations\ntenant = Tenant.create(\n  custom_domain: 'https://shop.example.com',    # ✅ Custom subdomain\n  primary_domain: 'example.com'                 # ✅ Root without protocol\n)\n\n# Invalid configurations\ntenant = Tenant.new(\n  custom_domain: 'https://www.example.com'      # ❌ www not allowed\n)\n```\n\n#### E-commerce Store Configuration\n\n```ruby\nclass Store \u003c ApplicationRecord\n  # Main storefront can be root or custom subdomain\n  validates :storefront_url, domain: {\n    validation: :root_or_custom_subdomain,\n    use_https: true\n  }\n\n  # Admin panel must be a subdomain (not root, not www)\n  validates :admin_url, domain: { validation: :standard }\n  validate :admin_must_have_subdomain\n\n  private\n\n  def admin_must_have_subdomain\n    parsed = DomainExtractor.parse(admin_url)\n    if parsed.valid? \u0026\u0026 !parsed.subdomain?\n      errors.add(:admin_url, 'must have a subdomain')\n    end\n  end\nend\n```\n\n#### API Service Registration\n\n```ruby\nclass ApiEndpoint \u003c ApplicationRecord\n  # API endpoints must use HTTPS\n  validates :url, domain: {\n    validation: :standard,\n    use_https: true\n  }\n\n  # Custom validation for API subdomain\n  validate :must_be_api_subdomain\n\n  private\n\n  def must_be_api_subdomain\n    return unless url.present?\n\n    parsed = DomainExtractor.parse(url)\n    if parsed.valid? \u0026\u0026 parsed.subdomain.present?\n      unless parsed.subdomain.start_with?('api')\n        errors.add(:url, 'must use an api subdomain')\n      end\n    end\n  end\nend\n```\n\n#### Domain Allowlist with Flexible Protocol\n\n```ruby\nclass AllowedDomain \u003c ApplicationRecord\n  # Accept domains with or without protocol\n  validates :domain, domain: {\n    validation: :root_domain,\n    use_protocol: false,\n    use_https: false\n  }\nend\n\n# All these are valid\nAllowedDomain.create(domain: 'example.com')\nAllowedDomain.create(domain: 'https://example.com')\nAllowedDomain.create(domain: 'http://example.com')\n```\n\n### Combining with Other Validators\n\nThe domain validator works seamlessly with other Rails validators:\n\n```ruby\nclass Website \u003c ApplicationRecord\n  validates :url, presence: true,\n                  domain: { validation: :standard },\n                  uniqueness: { case_sensitive: false }\n\n  validates :backup_url, domain: {\n    validation: :root_or_custom_subdomain,\n    use_https: true\n  }, allow_blank: true\nend\n```\n\n### Error Messages\n\nThe validator provides clear, specific error messages:\n\n```ruby\nwebsite = Website.new(url: 'not-a-url')\nwebsite.valid?\nwebsite.errors[:url]\n# =\u003e [\"is not a valid URL\"]\n\ndomain = PrimaryDomain.new(domain: 'https://shop.example.com')\ndomain.valid?\ndomain.errors[:domain]\n# =\u003e [\"must be a root domain (no subdomains allowed)\"]\n\ncustom = CustomDomain.new(url: 'https://www.example.com')\ncustom.valid?\ncustom.errors[:url]\n# =\u003e [\"cannot use www subdomain\"]\n\nsecure = SecureWebsite.new(url: 'http://example.com')\nsecure.valid?\nsecure.errors[:url]\n# =\u003e [\"must use https://\"]\n```\n\n## Use Cases\n\n**Web Scraping**\n\n```ruby\nurls = scrape_page_links(page)\ndomains = urls.map { |url| DomainExtractor.parse(url).root_domain }.compact.uniq\n```\n\n**Analytics \u0026 Tracking**\n\n```ruby\nreferrer = request.referrer\nparsed = DomainExtractor.parse(referrer)\ntrack_event('page_view', source_domain: parsed[:root_domain])\n```\n\n**Domain Validation**\n\n```ruby\ndef internal_link?(url, base_domain)\n  return false unless DomainExtractor.valid?(url)\n\n  DomainExtractor.parse(url).root_domain == base_domain\nend\n```\n\n## Performance\n\nOptimized for high-throughput production use:\n\n- **Single URL parsing**: 15-30μs per URL (50,000+ URLs/second)\n- **Batch processing**: 50,000+ URLs/second sustained throughput\n- **Memory efficient**: \u003c100KB overhead, ~200 bytes per parse\n- **Thread-safe**: Stateless modules, safe for concurrent use\n- **Zero-allocation hot paths**: Frozen constants, pre-compiled regex\n\nView [performance analysis](https://github.com/opensite-ai/domain_extractor/blob/master/docs/PERFORMANCE.md) for detailed benchmarks and optimization strategies and benchmark results along with a full set of enhancements made in order to meet the highly performance centric requirements of the OpenSite AI site rendering engine, showcased in the [optimization summary](https://github.com/opensite-ai/domain_extractor/blob/master/docs/OPTIMIZATION_SUMMARY.md)\n\n## Comparison with Alternatives\n\n| Feature                     | DomainExtractor | Addressable | URI (stdlib) |\n| --------------------------- | --------------- | ----------- | ------------ |\n| Multi-part TLD parser       | ✅              | ❌          | ❌           |\n| Subdomain extraction        | ✅              | ❌          | ❌           |\n| Domain component separation | ✅              | ❌          | ❌           |\n| Built-in url normalization  | ✅              | ❌          | ❌           |\n| Lightweight                 | ✅              | ❌          | ✅           |\n\n## Requirements\n\n- Ruby 3.2.0 or higher\n- public_suffix gem (~\u003e 6.0)\n\n## Contributing\n\nBug reports and pull requests are welcome on GitHub at https://github.com/opensite-ai/domain_extractor.\n\n## License\n\nThe gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).\n\n## Acknowledgments\n\n- Built on Ruby's standard [URI library](https://ruby-doc.org/stdlib/libdoc/uri/rdoc/URI.html)\n- Uses the [public_suffix gem](https://github.com/weppos/publicsuffix-ruby) for accurate TLD parsing\n\n---\n\nMade with ❤️ by [OpenSite AI](https://opensite.ai)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopensite-ai%2Fdomain_extractor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopensite-ai%2Fdomain_extractor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopensite-ai%2Fdomain_extractor/lists"}