{"id":15679738,"url":"https://github.com/skryukov/uri-idna","last_synced_at":"2025-05-07T09:41:58.390Z","repository":{"id":186335063,"uuid":"674994569","full_name":"skryukov/uri-idna","owner":"skryukov","description":"A IDNA2008, UTS46 and Punycode implementation in pure Ruby","archived":false,"fork":false,"pushed_at":"2025-04-21T20:19:53.000Z","size":387,"stargazers_count":12,"open_issues_count":0,"forks_count":1,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-21T21:25:53.310Z","etag":null,"topics":["hacktoberfest","idna","idna2008","ruby","uts46"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/skryukov.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-05T12:25:33.000Z","updated_at":"2025-04-21T20:19:54.000Z","dependencies_parsed_at":"2023-11-27T19:56:11.188Z","dependency_job_id":null,"html_url":"https://github.com/skryukov/uri-idna","commit_stats":{"total_commits":21,"total_committers":2,"mean_commits":10.5,"dds":0.2857142857142857,"last_synced_commit":"00df9c482a0f07cafc11496ed7ecdd5c61c66d00"},"previous_names":["skryukov/uri-idna"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skryukov%2Furi-idna","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skryukov%2Furi-idna/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skryukov%2Furi-idna/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skryukov%2Furi-idna/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/skryukov","download_url":"https://codeload.github.com/skryukov/uri-idna/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252852495,"owners_count":21814370,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hacktoberfest","idna","idna2008","ruby","uts46"],"created_at":"2024-10-03T16:35:12.851Z","updated_at":"2025-05-07T09:41:58.381Z","avatar_url":"https://github.com/skryukov.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# URI::IDNA\n\n[![Gem Version](https://badge.fury.io/rb/uri-idna.svg)](https://rubygems.org/gems/uri-idna)\n[![Ruby](https://github.com/skryukov/uri-idna/actions/workflows/main.yml/badge.svg)](https://github.com/skryukov/uri-idna/actions/workflows/main.yml)\n\nA IDNA2008, UTS46, IDNA from WHATWG URL Standard and Punycode implementation in pure Ruby.\n\nThis gem provides a number of functions for converting internationalized domain names (IDNs) between the Unicode and ASCII Compatible Encoding (ACE) forms.\n\n\u003ca href=\"https://evilmartians.com/?utm_source=uri-idna\u0026utm_campaign=project_page\"\u003e\n\u003cimg src=\"https://evilmartians.com/badges/sponsored-by-evil-martians.svg\" alt=\"Sponsored by Evil Martians\" width=\"236\" height=\"54\"\u003e\n\u003c/a\u003e\n\n## Installation\n\nAdd to your Gemfile:\n```ruby\ngem \"uri-idna\"\n```\n\nAnd then run `bundle install`.\n\n## Usage\n\nThere are plenty of ways to convert IDNs between Unicode and ACE forms.\n\n### IDNA2008\n\nThe [RFC 5891] defines two protocols for IDN conversion: [Registration](https://datatracker.ietf.org/doc/html/rfc5891#section-4) and [Domain Name Lookup](https://datatracker.ietf.org/doc/html/rfc5891#section-5).\n\n#### Registration protocol\n\n`URI::IDNA.register(alabel:, ulabel:, **options)`\n\n##### Options\n\n- `check_hyphens`: `true` – whether to check hyphens according to [Section 5.4](https://datatracker.ietf.org/doc/html/rfc5891#section-5.4).\n- `leading_combining`: `true` – whether to check leading combining marks according to [Section 5.4](https://datatracker.ietf.org/doc/html/rfc5891#section-5.4).\n- `check_joiners`: `true` – whether to check `CONTEXTJ` code points according to [Section 5.4](https://datatracker.ietf.org/doc/html/rfc5891#section-5.4).\n- `check_others`: `true` – whether to check `CONTEXTO` code points according to [Section 5.4](https://datatracker.ietf.org/doc/html/rfc5891#section-5.4).\n- `check_bidi`: `true` – whether to check bidirectional characters according to [Section 5.4](https://datatracker.ietf.org/doc/html/rfc5891#section-5.4).\n\n```ruby\nrequire \"uri/idna\"\n\nURI::IDNA.register(alabel: \"xn--gdkl8fhk5egc.jp\", ulabel: \"ハロー・ワールド.jp\")\n#=\u003e \"xn--gdkl8fhk5egc.jp\"\n\nURI::IDNA.register(ulabel: \"ハロー・ワールド.jp\")\n#=\u003e \"xn--gdkl8fhk5egc.jp\"\n\nURI::IDNA.register(alabel: \"xn--gdkl8fhk5egc.jp\")\n#=\u003e \"xn--gdkl8fhk5egc.jp\"\n\nURI::IDNA.register(ulabel: \"☕.us\")\n#\u003cURI::IDNA::InvalidCodepointError: Codepoint U+2615 at position 1 of \"☕\" not allowed\u003e\n```\n\n#### Domain Name Lookup Protocol\n\n`URI::IDNA.lookup(domain_name, **options)`\n\n##### Options\n\n- `check_hyphens`: `true` – whether to check hyphens according to [Section 4.2.3.1](https://datatracker.ietf.org/doc/html/rfc5891#section-4.2.3.1).\n- `leading_combining`: `true` – whether to check leading combining marks according to [Section 4.2.3.2](https://datatracker.ietf.org/doc/html/rfc5891#section-4.2.3.2).\n- `check_joiners`: `true` – whether to check CONTEXTJ code points according to [Section 4.2.3.3](https://datatracker.ietf.org/doc/html/rfc5891#section-4.2.3.3).\n- `check_others`: `true` – whether to check CONTEXTO code points according to [Section 4.2.3.3](https://datatracker.ietf.org/doc/html/rfc5891#section-4.2.3.3).\n- `check_bidi`: `true` – whether to check bidirectional characters according to [Section 4.2.3.4](https://datatracker.ietf.org/doc/html/rfc5891#section-4.2.3.4).\n- `verify_dns_length`: `true` – whether to check DNS length according to [Section 4.4](https://datatracker.ietf.org/doc/html/rfc5891#section-4.4).\n\n```ruby\nrequire \"uri/idna\"\n\nURI::IDNA.lookup(\"ハロー・ワールド.jp\")\n#=\u003e \"xn--pck0a1b0a6a2e.jp\"\n\nURI::IDNA.lookup(\"xn--pck0a1b0a6a2e.jp\")\n#=\u003e \"xn--pck0a1b0a6a2e.jp\"\n\nURI::IDNA.lookup(\"Ῠ.me\")\n#\u003cURI::IDNA::InvalidCodepointError: Codepoint U+1FE8 at position 1 of \"Ῠ\" not allowed\u003e\n```\n\n### Unicode UTS46 (TR46)\n\n_Current revision: 31_\n\nThe [UTS46] defines two IDN conversion functions: [ToASCII](https://www.unicode.org/reports/tr46/#ToASCII) and [ToUnicode](https://www.unicode.org/reports/tr46/#ToUnicode).\n\n#### ToASCII\n\n`URI::IDNA.to_ascii(domain_name, **options)`\n\n##### Options\n\n- `use_std3_ascii_rules`: `true` – whether to apply [STD3 rules](https://www.unicode.org/reports/tr46/#STD3_Rules) for both mapping and validation.\n- `check_hyphens`: `true` – whether to check hyphens according to [Section 4.2.3.1](https://datatracker.ietf.org/doc/html/rfc5891#section-4.2.3.1) of [RFC 5891].\n- `check_bidi`: `true` – whether to check bidirectional characters according to [Section 4.2.3.4](https://datatracker.ietf.org/doc/html/rfc5891#section-4.2.3.4) of [RFC 5891].\n- `check_joiners`: `true` – whether to check CONTEXTJ code points according to [Section 4.2.3.3](https://datatracker.ietf.org/doc/html/rfc5891#section-4.2.3.3) of [RFC 5891].\n- `transitional_processing`: `false` – (deprecated) whether to apply [transitional processing](https://www.unicode.org/reports/tr46/#ProcessingStepMap) for mapping.\n- `ignore_invalid_punycode`: `false` – whether to fast-path invalid Punycode labels according to [4th step of Processing](https://www.unicode.org/reports/tr46/#ProcessingStepPunycode).\n- `verify_dns_length`: `true` – whether to check DNS length according to [Section 4.4](https://datatracker.ietf.org/doc/html/rfc5891#section-4.4) of [RFC 5891].\n\n```ruby\nrequire \"uri/idna\"\n\nURI::IDNA.to_ascii(\"Bloß.de\")\n#=\u003e \"xn--blo-7ka.de\"\n\n# UTS46 transitional processing is disabled by default,\n# but can be enabled via option:\nURI::IDNA.to_ascii(\"Bloß.de\", transitional_processing: true)\n#=\u003e \"bloss.de\"\n\n# Note that UTS46 processing is not fully IDNA2008 compliant:\nURI::IDNA.to_ascii(\"☕.us\")\n#=\u003e \"xn--53h.us\"\n```\n\n#### ToUnicode\n\n`URI::IDNA.to_unicode(domain_name, **options)`\n\n##### Options\n\n- `use_std3_ascii_rules`: `true` – whether to apply [STD3 rules](https://www.unicode.org/reports/tr46/#STD3_Rules) for both mapping and validation.\n- `check_hyphens`: `true` – whether to check hyphens according to [Section 4.2.3.1](https://datatracker.ietf.org/doc/html/rfc5891#section-4.2.3.1) of [RFC 5891].\n- `check_bidi`: `true` – whether to check bidirectional characters according to [Section 4.2.3.4](https://datatracker.ietf.org/doc/html/rfc5891#section-4.2.3.4) of [RFC 5891].\n- `check_joiners`: `true` – whether to check CONTEXTJ code points according to [Section 4.2.3.3](https://datatracker.ietf.org/doc/html/rfc5891#section-4.2.3.3) of [RFC 5891].\n- `transitional_processing`: `false` – (deprecated) whether to apply [transitional processing](https://www.unicode.org/reports/tr46/#ProcessingStepMap) for mapping.\n- `ignore_invalid_punycode`: `false` – whether to fast-path invalid Punycode labels according to [4th step of Processing](https://www.unicode.org/reports/tr46/#ProcessingStepPunycode).\n\n```ruby\nrequire \"uri/idna\"\n\nURI::IDNA.to_unicode(\"xn--blo-7ka.de\")\n#=\u003e \"bloß.de\"\n```\n\n#### IDNA2008 compatibility\n\nIt's possible to use UTS46 mapping first and then apply IDNA2008, so the processing fully conforms IDNA2008:\n\n```ruby\nrequire \"uri/idna\"\n\n# For example we can use UTS46 mapping to downcase some characters\nchar = \"⼤\"\nchar.ord # \"\\u2F24\"\n#=\u003e 12068\n\n# just downcase doesn't work in this case\nchar.downcase.ord\n#=\u003e 12068\n\n# but UTS46 mapping does it's thing:\nURI::IDNA::UTS46::Mapping.call(char).ord\n#=\u003e 22823\n\n# so here is a full example:\ndomain = \"⼤.cn\" # \"\\u2F24.cn\"\nURI::IDNA.lookup(domain)\n# \u003cURI::IDNA::InvalidCodepointError: Codepoint U+2F24 at position 1 of \"⼤\" not allowed\u003e\n\nmapped_domain = URI::IDNA::UTS46::Mapping.call(domain)\nURI::IDNA.lookup(mapped_domain)\n#=\u003e \"xn--pss.cn\"\n```\n\n### WHATWG\n\nWHATWG's [URL Standard] uses UTS46 algorithm to define ToASCII and ToUnicode functions, it abstracts all available flags and provides only one—the `be_btrict` flag instead.\n\nNote that the `check_hyphens` UTS46 option is set to `false` in this algorithm.\n\n#### ToASCII\n\n`URI::IDNA.whatwg_to_ascii(domain_name, **options)`\n\n##### Options\n\n- `be_strict`: `true` – defines values of `use_std3_ascii_rules` and `verify_dns_length` UTS46 options.\n\n```ruby\nrequire \"uri/idna\"\n\nURI::IDNA.whatwg_to_ascii(\"Bloß.de\")\n#=\u003e \"xn--blo-7ka.de\"\n\n# The be_strict flag sets use_std3_ascii_rules and verify_dns_length UTS46 flags to its value\nURI::IDNA.whatwg_to_ascii(\"2003_rules.com\", be_strict: false)\n#=\u003e \"2003_rules.com\"\n\n# By default be_strict is set to true\nURI::IDNA.whatwg_to_ascii(\"2003_rules.com\")\n#\u003cURI::IDNA::InvalidCodepointError: Codepoint U+005F at position 5 of \"2003_rules\" not allowed\u003e\n```\n\n#### ToUnicode\n\n`URI::IDNA.whatwg_to_unicode(domain_name, **options)`\n\n##### Options\n\n- `be_strict`: `true` - defines value of `use_std3_ascii_rules` UTS46 option.\n\n```ruby\nrequire \"uri/idna\"\n\nURI::IDNA.whatwg_to_unicode(\"xn--blo-7ka.de\")\n#=\u003e \"bloß.de\"\n```\n\n### Punycode\n\nPunycode module performs conversion between Unicode and Punycode. Note that Punycode is not IDNA2008 compliant, it is only used for conversion, no validations performed.\n\n```ruby\nrequire \"uri/idna/punycode\"\n\nURI::IDNA::Punycode.encode(\"ハロー・ワールド\")\n#=\u003e \"gdkl8fhk5egc\"\n\nURI::IDNA::Punycode.decode(\"gdkl8fhk5egc\")\n#=\u003e \"ハロー・ワールド\"\n```\n\n## Full technical reference:\n\n### IDNA2008\n- [RFC 5890] – Definitions and Document Framework\n- [RFC 5891] – Protocol\n- [RFC 5892] – The Unicode Code Points\n- [RFC 5893] – Bidi rule\n\n### Punycode\n\n- [RFC 3492] – Punycode: A Bootstring encoding of Unicode\n\n### UTS46 (also referenced as TS46)\n\n- [Unicode IDNA Compatibility Processing][UTS46]\n\n## Development\n\nAfter checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.\n\nTo install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).\n\n### Generating Unicode data\n\nThis gem uses Unicode data files to perform IDN conversion. To generate new Unicode data files, run `bundle exec rake idna:generate`.\n\nTo specify Unicode version, use `VERSION` environment variable, e.g. `VERSION=15.1.0 bundle exec rake idna:generate`.\n\nBy default, used Unicode version is the one used by the Ruby version (`RbConfig::CONFIG[\"UNICODE_VERSION\"]`).\n\nTo set directory for generated files, use `DEST_DIR` environment variable, e.g. `DEST_DIR=lib/uri/idna/data bundle exec rake idna:generate`.\n\nUnicode data cached in the `tmp` directory by default, to change it, use `CACHE_DIR` environment variable, e.g. `CACHE_DIR=~/.cache/unicode_data bundle exec rake idna:generate`.\n\n_Note: `rake idna:generate` might generate different results on different versions of Ruby due to usage of built-in Unicode normalization methods._\n\n### Inspect Unicode data\n\nTo inspect Unicode data, run `bundle exec rake 'idna:inspect[\u003cHEX_CODE\u003e]'`.\n\nTo specify Unicode version, or cache directory, use `VERSION` or `CACHE_DIR` environment variables, e.g. `VERSION=15.1.0 bundle exec rake 'idna:inspect[1f495]'`.\n\n### Update UTS46 test suite data\n\nTo update UTS46 test suite data, run `bundle exec rake idna:update_uts46_test_suite`.\n\n## Contributing\n\nBug reports and pull requests are welcome on GitHub at https://github.com/skryukov/uri-idna.\n\n## License\n\nThe gem is available as open source under the terms of the [MIT License].\n\n[RFC 5890]: https://datatracker.ietf.org/doc/html/rfc5890\n[RFC 5891]: https://datatracker.ietf.org/doc/html/rfc5891\n[RFC 5892]: https://datatracker.ietf.org/doc/html/rfc5892\n[RFC 5893]: https://datatracker.ietf.org/doc/html/rfc5893\n[RFC 3492]: https://datatracker.ietf.org/doc/html/rfc3492\n[UTS46]: https://www.unicode.org/reports/tr46\n[URL Standard]: https://url.spec.whatwg.org/#idna\n[MIT License]: https://opensource.org/licenses/MIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fskryukov%2Furi-idna","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fskryukov%2Furi-idna","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fskryukov%2Furi-idna/lists"}