{"id":16073196,"url":"https://github.com/juanitofatas/active_normalizer","last_synced_at":"2025-03-23T20:22:18.761Z","repository":{"id":62552793,"uuid":"137454886","full_name":"JuanitoFatas/active_normalizer","owner":"JuanitoFatas","description":"Easily switch normalizer you want to use with Active Normalizer","archived":false,"fork":false,"pushed_at":"2018-06-17T05:52:10.000Z","size":12,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-14T22:53:48.614Z","etag":null,"topics":["normalize","normalizer","tool","unicode","utility"],"latest_commit_sha":null,"homepage":"https://github.com/JuanitoFatas/active_normalizer","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JuanitoFatas.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-06-15T07:22:56.000Z","updated_at":"2025-02-26T21:54:21.000Z","dependencies_parsed_at":"2022-11-03T04:15:16.937Z","dependency_job_id":null,"html_url":"https://github.com/JuanitoFatas/active_normalizer","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuanitoFatas%2Factive_normalizer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuanitoFatas%2Factive_normalizer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuanitoFatas%2Factive_normalizer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuanitoFatas%2Factive_normalizer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JuanitoFatas","download_url":"https://codeload.github.com/JuanitoFatas/active_normalizer/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245163167,"owners_count":20570875,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["normalize","normalizer","tool","unicode","utility"],"created_at":"2024-10-09T08:04:45.398Z","updated_at":"2025-03-23T20:22:18.689Z","avatar_url":"https://github.com/JuanitoFatas.png","language":"Ruby","readme":"# Active Normalizer\n\nNormalize weird Japanese characters, see [tests](/spec) for examples.\n\nNormalize fullwidth, halfwidth hiragana, katakana, symbols.\n\n## Usage\n\nEach normalizer class accepts option of `:nfc`, `:nfd`, `:nfkd`, `:nfkc` (See [Normalization Forms][unicode-nf] for more information).\nEach normalizer instance responds to `run`.\n\n```ruby\nrequire \"active_normalizer/normalizers/ruby\"\nnfkc_normalizer = ActiveNormalizer.new(\n  ActiveNormalizer::Normalizers::Ruby,\n  options: :nfkc\n)\nnfkc_normalizer.run(input)\n```\n\n## Benchmark\n\n```\nBenchmarking simple string: ８００ー1234５\nWarming up --------------------------------------\n                 UNF    92.981k i/100ms\n             Unicode    36.002k i/100ms\n                Ruby    17.044k i/100ms\n        UnicodeUtils    12.681k i/100ms\n       ActiveSupport     7.482k i/100ms\nCalculating -------------------------------------\n                 UNF      1.173M (±17.6%) i/s -      5.672M in   5.041037s\n             Unicode    404.502k (± 6.8%) i/s -      2.016M in   5.008748s\n                Ruby    191.562k (±30.3%) i/s -    835.156k in   5.106057s\n        UnicodeUtils    132.477k (± 5.3%) i/s -    672.093k in   5.088759s\n       ActiveSupport     75.011k (±34.9%) i/s -    329.208k in   5.058559s\n\nComparison:\n                 UNF:  1172663.8 i/s\n             Unicode:   404502.1 i/s - 2.90x  slower\n                Ruby:   191562.4 i/s - 6.12x  slower\n        UnicodeUtils:   132477.3 i/s - 8.85x  slower\n       ActiveSupport:    75010.6 i/s - 15.63x  slower\n\nWarming up --------------------------------------\n                 UNF    67.181k i/100ms\n             Unicode    31.572k i/100ms\n                Ruby    14.947k i/100ms\n        UnicodeUtils    12.443k i/100ms\n       ActiveSupport     5.561k i/100ms\nCalculating -------------------------------------\n                 UNF    997.098k (±25.2%) i/s -     27.477M in  30.052018s\n             Unicode    328.071k (±19.5%) i/s -      9.503M in  30.090451s\n                Ruby    177.045k (±32.8%) i/s -      4.529M in  30.071040s\n        UnicodeUtils    134.513k (± 6.7%) i/s -      4.019M in  30.059621s\n       ActiveSupport     68.063k (±44.7%) i/s -      1.668M in  30.131968s\n\nComparison:\n                 UNF:   997097.6 i/s\n             Unicode:   328070.8 i/s - 3.04x  slower\n                Ruby:   177044.6 i/s - 5.63x  slower\n        UnicodeUtils:   134512.7 i/s - 7.41x  slower\n       ActiveSupport:    68063.1 i/s - 14.65x  slower\n\n\nBenchmarking longer string: ㍻㍼㍽㍾㌀㌁㌂㌃㌄㌅㌆㌇㌈㌉㌊㌋㌌㌍㌎㌏㌐㌑㌒㌓㌔㌕㌖㌗㌘㌙㌚㌛㌜㌝㌞㌟㌠㌡㌢㌣㌤㌥㌦㌧㌨㌩㌪㌫㌬㌭㌮㌯㌰㌱㌲㌳㌴㌵㌶㌷㌸㌹㌺㌻㌼㌽㌾㌿㍀㍁㍂㍃㍄㍅㍆㍇㍈㍉㍊㍋㍌㍍㍎㍏㍐㍑㍒㍓㍔㍕㍖㍗\nWarming up --------------------------------------\n                 UNF     6.023k i/100ms\n             Unicode     1.238k i/100ms\n                Ruby     1.068k i/100ms\n        UnicodeUtils   319.000  i/100ms\n       ActiveSupport   258.000  i/100ms\nCalculating -------------------------------------\n                 UNF     59.891k (± 6.8%) i/s -    301.150k in   5.055411s\n             Unicode     11.740k (± 9.0%) i/s -     59.424k in   5.103353s\n                Ruby     10.655k (±10.9%) i/s -     53.400k in   5.091860s\n        UnicodeUtils      3.087k (± 8.9%) i/s -     15.312k in   5.004688s\n       ActiveSupport      2.533k (±11.1%) i/s -     12.642k in   5.064477s\n\nComparison:\n                 UNF:    59890.8 i/s\n             Unicode:    11740.2 i/s - 5.10x  slower\n                Ruby:    10655.0 i/s - 5.62x  slower\n        UnicodeUtils:     3087.4 i/s - 19.40x  slower\n       ActiveSupport:     2532.6 i/s - 23.65x  slower\n\nWarming up --------------------------------------\n                 UNF     5.739k i/100ms\n             Unicode     1.122k i/100ms\n                Ruby     1.113k i/100ms\n        UnicodeUtils   312.000  i/100ms\n       ActiveSupport   254.000  i/100ms\nCalculating -------------------------------------\n                 UNF     59.371k (± 4.4%) i/s -      1.779M in  30.026571s\n             Unicode     10.780k (±17.3%) i/s -    310.794k in  30.106556s\n                Ruby     11.144k (± 6.7%) i/s -    332.787k in  30.034689s\n        UnicodeUtils      3.164k (± 4.9%) i/s -     94.848k in  30.056928s\n       ActiveSupport      2.635k (± 8.8%) i/s -     78.486k in  30.075836s\n\nComparison:\n                 UNF:    59371.2 i/s\n                Ruby:    11143.9 i/s - 5.33x  slower\n             Unicode:    10779.6 i/s - 5.51x  slower\n        UnicodeUtils:     3163.5 i/s - 18.77x  slower\n       ActiveSupport:     2635.3 i/s - 22.53x  slower\n```\n\nBenchmark code can be found at [bin/benchmark](bin/benchmark).\n\n## Installation\n\nAdd this line to your application's Gemfile:\n\n```ruby\ngem \"active_normalizer\"\n```\n\nAnd then execute:\n\n    $ bundle\n\nOr install it yourself as:\n\n    $ gem install active_normalizer\n\n## Dependnecies\n\nActive Normalizer provides a handful of normalizers. Their dependencies are not bundled except for one that utilizes standard library. You must bundle the normalizer's gem dependency.\n\n#### ActiveNormalizer::Normalizers::Ruby\n\n```ruby\n# no dependency required, standard library\n\nrequire \"active_normalizer/normalizers/ruby\"\n```\n\n#### ActiveNormalizer::Normalizers::UNF - unf\n\n```ruby\ngem \"unf\"\n\nrequire \"active_normalizer/normalizers/unf\"\n```\n\n#### ActiveNormalizer::Normalizers::Unicode - unicode\n\n```ruby\ngem \"unicode\"\n\nrequire \"active_normalizer/normalizers/unicode\"\n```\n\n#### ActiveNormalizer::Normalizers::UnicodeUtils - unicode_utils\n\n```ruby\ngem \"unicode_utils\"\n\nrequire \"active_normalizer/normalizers/unicode_utils\"\n```\n\n#### ActiveNormalizer::Normalizers::ActiveSupportMultibyte - active_support\n\n```ruby\ngem \"active_support\"\n\nrequire \"active_normalizer/normalizers/active_support\"\n```\n\n## Development\n\nAfter checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/hack` for an interactive prompt that will allow you to experiment.\n\nTo install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).\n\n## Contributing\n\nBug reports and pull requests are welcome on GitHub at https://github.com/JuanitoFatas/active_normalizer.\n\n## License\n\nThe gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).\n\n[unicode-nf]: http://unicode.org/reports/tr15/#Norm_Forms\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjuanitofatas%2Factive_normalizer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjuanitofatas%2Factive_normalizer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjuanitofatas%2Factive_normalizer/lists"}