{"id":13419173,"url":"https://github.com/Tessil/hat-trie","last_synced_at":"2025-03-15T04:32:07.581Z","repository":{"id":42351868,"uuid":"88876180","full_name":"Tessil/hat-trie","owner":"Tessil","description":"C++ implementation of a fast and memory efficient HAT-trie","archived":false,"fork":false,"pushed_at":"2024-02-10T15:03:55.000Z","size":980,"stargazers_count":753,"open_issues_count":8,"forks_count":114,"subscribers_count":36,"default_branch":"master","last_synced_at":"2024-02-16T07:36:58.197Z","etag":null,"topics":["c-plus-plus","cpp","data-structures","hat-trie","header-only","trie"],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Tessil.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-04-20T14:38:53.000Z","updated_at":"2024-05-30T01:27:32.805Z","dependencies_parsed_at":"2024-02-16T07:46:40.345Z","dependency_job_id":null,"html_url":"https://github.com/Tessil/hat-trie","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tessil%2Fhat-trie","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tessil%2Fhat-trie/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tessil%2Fhat-trie/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tessil%2Fhat-trie/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Tessil","download_url":"https://codeload.github.com/Tessil/hat-trie/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243685506,"owners_count":20330980,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["c-plus-plus","cpp","data-structures","hat-trie","header-only","trie"],"created_at":"2024-07-30T22:01:12.310Z","updated_at":"2025-03-15T04:32:07.228Z","avatar_url":"https://github.com/Tessil.png","language":"C++","readme":"[![CI](https://github.com/Tessil/hat-trie/actions/workflows/ci.yml/badge.svg?branch=master)](https://github.com/Tessil/hat-trie/actions/workflows/ci.yml)\n\n## A C++ implementation of a fast and memory efficient HAT-trie\n\nTrie implementation based on the \"HAT-trie: A Cache-conscious Trie-based Data Structure for Strings.\" (Askitis Nikolas and  Sinha Ranjan, 2007) paper. For now, only the pure HAT-trie has been implemented, the hybrid version may arrive later. Details regarding the HAT-trie data structure can be found [here](https://tessil.github.io/2017/06/22/hat-trie.html).\n\nThe library provides an efficient and compact way to store a set or a map of strings by compressing the common prefixes. It also allows to search for keys that match a prefix. Note though that the default parameters of the structure are geared toward optimizing exact searches, if you do a lot of prefix searches you may want to reduce the burst threshold through the `burst_threshold` method.\n\nIt's a well adapted structure to store a large number of strings.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://tessil.github.io/images/hat-trie.png\" width=\"600px\" /\u003e\n\u003c/p\u003e\n\nFor the array hash part, the [array-hash](https://github.com/Tessil/array-hash) project is used and included in the repository.\n\nThe library provides two classes: `tsl::htrie_map` and `tsl::htrie_set`.\n\n### Overview\n\n- Header-only library, just add the [include](include/) directory to your include path and you are ready to go. If you use CMake, you can also use the `tsl::hat_trie` exported target from the [CMakeLists.txt](CMakeLists.txt).\n- Low memory usage while keeping reasonable performances (see [benchmark](#benchmark)).\n- Support prefix searches through `equal_prefix_range` (useful for autocompletion for example) and prefix erasures through `erase_prefix`.\n- Support longest matching prefix searches through `longest_prefix`.\n- Support for efficient serialization and deserialization (see [example](#serialization) and the `serialize/deserialize` methods in the [API](https://tessil.github.io/hat-trie/doc/html/classtsl_1_1htrie__map.html) for details).\n- Keys are not ordered as they are partially stored in a hash map.\n- All operations modifying the data structure (insert, emplace, erase, ...) invalidate the iterators. \n- Support null characters in the key (you can thus store binary data in the trie).\n- Support for any type of value as long at it's either copy-constructible or both nothrow move constructible and nothrow move assignable.\n- The balance between speed and memory usage can be modified through the `max_load_factor` method. A lower max load factor will increase the speed, a higher one will reduce the memory usage. Its default value is set to 8.0.\n- The default burst threshold, which is the maximum size of an array hash node before a burst occurs, is set to 16 384 which provides good performances for exact searches. If you mainly use prefix searches, you may want to reduce it to something like 1024 or lower for faster iteration on the results through the `burst_threshold` method.\n- By default the maximum allowed size for a key is set to 65 535. This can be raised through the `KeySizeT` template parameter.\n\nThread-safety and exception guarantees are similar to the STL containers.\n\n### Hash function\n\nThe default hash function used by the structure depends on the presence of `std::string_view`. If it is available, `std::hash\u003cstd::string_view\u003e` is used, otherwise a simple [FNV-1a](https://en.wikipedia.org/wiki/Fowler%E2%80%93Noll%E2%80%93Vo_hash_function#FNV-1a_hash) hash function is used to avoid any dependency.\n\nIf you can't use C++17 or later, we recommend to replace the hash function with something like [CityHash](https://github.com/google/cityhash), MurmurHash, [FarmHash](https://github.com/google/farmhash), ... for better performances. On the tests we did, CityHash64 offers a ~20% improvement on reads compared to FNV-1a.\n\n\n```c++\n#include \u003ccity.h\u003e\n\nstruct str_hash {\n    std::size_t operator()(const char* key, std::size_t key_size) const {\n        return CityHash64(key, key_size);\n    }\n};\n\ntsl::htrie_map\u003cchar, int, str_hash\u003e map;\n```\n\nThe `std::hash\u003cstd::string\u003e` can't be used efficiently as the structure doesn't store any `std::string` object. Any time a hash would be needed, a temporary `std::string` would have to be created.\n\n\n### Benchmark\n\n#### Wikipedia dataset\nThe benchmark consists in inserting all the titles from the main namespace of the Wikipedia archive into the data structure, check the used memory space after the insert (including potential memory fragmentation) and search for all the titles again in the data structure. The peak memory usage during the insert process is also measured with [time(1)](https://linux.die.net/man/1/time).\n\n* Dataset: [enwiki-20170320-all-titles-in-ns0.gz](https://dumps.wikimedia.org/enwiki/20170320/)\n* Size: 262.7 MiB\n* Number of keys: 13 099 148\n* Average key length: 19.90\n* Median key length: 17\n* Max key length: 251\n\nEach title is associated with an int (32 bits). All the hash based structures use [CityHash64](https://github.com/google/cityhash) as hash function. For the tests marked *with reserve*, the `reserve` function is called beforehand to avoid any rehash.\n\nNote that `tsl::hopscotch_map`, `std::unordered_map`, `google::dense_hash_map` and `spp::sparse_hash_map` use `std::string` as key which imposes a minimum size of 32 bytes (on x64) even if the key is only one character long. Other structures may be able to store one-character keys with 1 byte + 8 bytes for a pointer (on x64).\n\nThe benchmark was compiled with GCC 6.3 and ran on Debian Stretch x64 with an Intel i5-5200u and 8Go of RAM. Best of 20 runs was taken.\n\nThe code of the benchmark can be found on [Gist](https://gist.github.com/Tessil/72e11891fc155f5b2eb53de22cbc4053).\n\n##### Unsorted\n\nThe *enwiki-20170320-all-titles-in-ns0.gz* dataset is alphabetically sorted. For this benchmark, we first shuffle the dataset through [shuf(1)](https://linux.die.net/man/1/shuf) to avoid a biased sorted dataset.\n\n| Library | Data structure | Peak memory (MiB) | Memory (MiB) | Insert (ns/key) | Read (ns/key) |\n|---------|----------------|------------------:|-------------:|----------------:|--------------:|\n| [tsl::htrie_map](https://github.com/Tessil/hat-trie) | HAT-trie | **405.22** | **402.25** | 643.10 | 250.87 |\n| [tsl::htrie_map](https://github.com/Tessil/hat-trie) \u003cbr/\u003e max_load_factor=4 | HAT-trie |  471.85 | 468.50 | 638.66 | 212.90 |\n| [tsl::htrie_map](https://github.com/Tessil/hat-trie) \u003cbr/\u003e max_load_factor=2 | HAT-trie | 569.76 | 566.52 | 630.61 | 201.10 |\n| [tsl::htrie_map](https://github.com/Tessil/hat-trie) \u003cbr/\u003e max_load_factor=1 | HAT-trie | 713.44 | 709.81 | 645.76 | 190.87 |\n| [cedar::da](http://www.tkl.iis.u-tokyo.ac.jp/~ynaga/cedar/) | Double-array trie  | 1269.68 | 1254.41 | 1102.93 | 557.20 |\n| [cedar::da](http://www.tkl.iis.u-tokyo.ac.jp/~ynaga/cedar/) ORDERED=false | Double-array trie  | 1269.80 | 1254.41 | 1089.78 | 570.13 |\n| [cedar::da](http://www.tkl.iis.u-tokyo.ac.jp/~ynaga/cedar/) | Double-array reduced trie  | 1183.07 | 1167.79 | 1076.68 | 645.79 |\n| [cedar::da](http://www.tkl.iis.u-tokyo.ac.jp/~ynaga/cedar/) ORDERED=false | Double-array reduced trie  | 1183.14 | 1167.85 | 1065.43 | 641.98 |\n| [cedar::da](http://www.tkl.iis.u-tokyo.ac.jp/~ynaga/cedar/) | Double-array prefix trie | 498.69 | 496.54 | 1096.90 | 628.01 |\n| [cedar::da](http://www.tkl.iis.u-tokyo.ac.jp/~ynaga/cedar/) ORDERED=false | Double-array prefix trie  | 498.65 | 496.60 | 1048.40 | 628.94 |\n| [hat-trie](https://github.com/dcjones/hat-trie)\u003csup\u003e1\u003c/sup\u003e (C) | HAT-trie | 504.07 | 501.50 | 917.49 | 261.00 |\n| [qp trie](https://github.com/fanf2/qp) (C) | QP trie | 941.23 | 938.17 | 1349.25 | 1281.46 |\n| [crit-bit trie](https://github.com/fanf2/qp) (C) | Crit-bit trie | 1074.96 | 1071.98 | 2930.42 | 2869.74 |\n| [JudySL](http://judy.sourceforge.net/) (C) | Judy array | 631.09 | 628.37 | 884.29 | 803.58 |\n| [JudyHS](http://judy.sourceforge.net/) (C) | Judy array | 723.44 | 719.47 | 476.79 | 417.15 |\n| [tsl::array_map](https://github.com/Tessil/array-hash) | Array hash table | 823.54 | 678.73 | 603.94 |  138.24 |\n| [tsl::array_map](https://github.com/Tessil/array-hash) \u003cbr\u003ewith reserve | Array hash table | 564.26 | 555.91 | 249.52 | 128.28  |\n| [tsl::hopscotch_map](https://github.com/Tessil/hopscotch-map) | Hash table | 1325.83 | 1077.99 | 368.26 |  **119.49** |\n| [tsl::hopscotch_map](https://github.com/Tessil/hopscotch-map) \u003cbr\u003ewith reserve | Hash table | 1080.51 | 1077.98 | **240.58** | 119.91 |\n| [google::dense_hash_map](https://github.com/sparsehash/sparsehash) | Hash table | 2319.40 | 1677.11 | 466.60 | 138.87 |\n| [google::dense_hash_map](https://github.com/sparsehash/sparsehash) \u003cbr\u003ewith reserve | Hash table | 1592.51 | 1589.99 | 259.56 | 120.40 |\n| [spp::sparse_hash_map](https://github.com/greg7mdp/sparsepp) | Sparse hash table | 918.67 | 917.10 | 769.00 | 175.59 |\n| [spp::sparse_hash_map](https://github.com/greg7mdp/sparsepp) \u003cbr\u003ewith reserve | Sparse hash table | 913.35 | 910.65  | 427.22 | 159.08 |\n| [std::unordered_map](http://en.cppreference.com/w/cpp/container/unordered_map) | Hash table | 1249.05 | 1246.60 | 590.88 | 173.58 |\n| [std::unordered_map](http://en.cppreference.com/w/cpp/container/unordered_map) \u003cbr\u003ewith reserve | Hash table | 1212.23 | 1209.71 | 350.33 | 178.70 |\n\n1. As the hash function can't be passed in parameter, the code of the library itself is modified to use CityHash64.\n\n##### Sorted\n\nThe key are inserted and read in alphabetical order.\n\n| Library | Data structure | Peak memory (MiB) | Memory (MiB) | Insert (ns/key) | Read (ns/key) |\n|---------|----------------|------------------:|-------------:|----------------:|--------------:|\n| [tsl::htrie_map](https://github.com/Tessil/hat-trie) | HAT-trie | **396.10** | **393.22** | 255.76 | 68.08 |\n| [tsl::htrie_map](https://github.com/Tessil/hat-trie) \u003cbr/\u003e max_load_factor=4 | HAT-trie | 465.02 | 461.80 | 248.88 | 59.23 |\n| [tsl::htrie_map](https://github.com/Tessil/hat-trie) \u003cbr/\u003e max_load_factor=2 | HAT-trie | 543.99 | 541.21 | 230.13 | 53.50 |\n| [tsl::htrie_map](https://github.com/Tessil/hat-trie) \u003cbr/\u003e max_load_factor=1 | HAT-trie | 692.29 | 689.70 | 243.84 | **49.22** |\n| [cedar::da](http://www.tkl.iis.u-tokyo.ac.jp/~ynaga/cedar/) | Double-array trie  | 1269.58 | 1254.41 | 278.51 | 54.72 |\n| [cedar::da](http://www.tkl.iis.u-tokyo.ac.jp/~ynaga/cedar/) ORDERED=false | Double-array trie  | 1269.66 | 1254.41 | 264.43 | 56.02 |\n| [cedar::da](http://www.tkl.iis.u-tokyo.ac.jp/~ynaga/cedar/) | Double-array reduced trie  | 1183.01 | 1167.78 | 254.60 | 69.18 |\n| [cedar::da](http://www.tkl.iis.u-tokyo.ac.jp/~ynaga/cedar/) ORDERED=false | Double-array reduced trie  | 1183.03 | 1167.78 | 241.45 | 69.67 |\n| [cedar::da](http://www.tkl.iis.u-tokyo.ac.jp/~ynaga/cedar/) | Double-array prefix trie | 621.59 | 619.38 | 246.88 | 57.83 |\n| [cedar::da](http://www.tkl.iis.u-tokyo.ac.jp/~ynaga/cedar/) ORDERED=false | Double-array prefix trie | 621.59 | 619.38 | **187.98** | 58.56 |\n| [hat-trie](https://github.com/dcjones/hat-trie)\u003csup\u003e2\u003c/sup\u003e (C) | HAT-trie | 521.25 | 518.52 | 503.01 | 86.40 |\n| [qp trie](https://github.com/fanf2/qp) (C) | QP trie | 940.65 | 937.66 | 392.86 | 190.19 |\n| [crit-bit trie](https://github.com/fanf2/qp) (C) | Crit-bit trie | 1074.87 | 1071.98 | 430.04 | 347.60 |\n| [JudySL](http://judy.sourceforge.net/) (C) | Judy array | 616.95 | 614.27 | 279.07 | 114.47 |\n| [JudyHS](http://judy.sourceforge.net/) (C) | Judy array | 722.29 | 719.47 | 439.66 | 372.25 |\n| [tsl::array_map](https://github.com/Tessil/array-hash) | Array hash table | 826.98 | 682.99 | 612.31 | 139.16  |\n| [tsl::array_map](https://github.com/Tessil/array-hash) \u003cbr\u003ewith reserve | Array hash table | 565.37 | 555.35 | 246.55 | 126.32 |\n| [tsl::hopscotch_map](https://github.com/Tessil/hopscotch-map) | Hash table |  1331.87 | 1078.02 | 375.19 | 118.08 |\n| [tsl::hopscotch_map](https://github.com/Tessil/hopscotch-map) \u003cbr\u003ewith reserve | Hash table | 1080.51 | 1077.97 | 238.93 | 117.20 |\n| [google::dense_hash_map](https://github.com/sparsehash/sparsehash) | Hash table |  2325.27 | 1683.07 | 483.95 | 137.09 |\n| [google::dense_hash_map](https://github.com/sparsehash/sparsehash) \u003cbr\u003ewith reserve | Hash table | 1592.54 | 1589.99 | 257.22 | 113.71 |\n| [spp::sparse_hash_map](https://github.com/greg7mdp/sparsepp) | Sparse hash table | 920.96 | 918.70 | 772.03 | 176.64 |\n| [spp::sparse_hash_map](https://github.com/greg7mdp/sparsepp) \u003cbr\u003ewith reserve | Sparse hash table | 914.84 | 912.47 | 422.85 | 158.73 |\n| [std::unordered_map](http://en.cppreference.com/w/cpp/container/unordered_map) | Hash table | 1249.09 | 1246.65 | 594.85 | 173.54 |\n| [std::unordered_map](http://en.cppreference.com/w/cpp/container/unordered_map) \u003cbr\u003ewith reserve | Hash table |  1212.21 | 1209.71 | 347.40 | 176.49 |\n\n2. As the hash function can't be passed in parameter, the code of the library itself is modified to use CityHash64.\n\n\n#### Dr. Askitis dataset\n\nThe benchmark consists in inserting all the words from the \"Distinct Strings\" dataset of Dr. Askitis into the data structure, check the used memory space and search for all the words from the \"Skew String Set 1\" dataset (where a string can be present multiple times) in the data structure. Note that the strings in this dataset have a quite short average and median key length (which may not be a realistic use case compared to the Wikipedia dataset used above). It's similar to the one on the [cedar](http://www.tkl.iis.u-tokyo.ac.jp/~ynaga/cedar/) homepage.\n\n* Dataset: [distinct_1](http://web.archive.org/web/20120206015921/http://www.naskitis.com/) (write) / [skew1_1](http://web.archive.org/web/20120206015921/http://www.naskitis.com/) (read)\n* Size: 290.45 MiB / 1 029.46 MiB\n* Number of keys: 28 772 169 / 177 999 203\n* Average key length: 9.59 / 5.06\n* Median key length: 8 / 4\n* Max key length: 126 / 62\n\nThe benchmark protocol is the same as for the [Wikipedia dataset](https://github.com/Tessil/hat-trie#wikipedia-dataset).\n\n\n| Library | Data structure | Peak memory (MiB) | Memory (MiB) | Insert (ns/key) | Read (ns/key) |\n|---------|----------------|------------------:|-------------:|----------------:|--------------:|\n| [tsl::htrie_map](https://github.com/Tessil/hat-trie) | HAT-trie |  **604.76** | **601.79** | 485.45 | 77.80 |\n| [tsl::htrie_map](https://github.com/Tessil/hat-trie) \u003cbr/\u003e max_load_factor=4 | HAT-trie | 768.10 | 764.98 | 491.78 | 75.48 |\n| [tsl::htrie_map](https://github.com/Tessil/hat-trie) \u003cbr/\u003e max_load_factor=2 | HAT-trie | 1002.42 | 999.34 | 496.78 | 72.53 |\n| [tsl::htrie_map](https://github.com/Tessil/hat-trie) \u003cbr/\u003e max_load_factor=1 | HAT-trie | 1344.98 | 1341.97 | 520.66 | 72.45 |\n| [cedar::da](http://www.tkl.iis.u-tokyo.ac.jp/~ynaga/cedar/) | Double-array trie | 1105.45 | 1100.05 | 682.25 | 71.98 |\n| [cedar::da](http://www.tkl.iis.u-tokyo.ac.jp/~ynaga/cedar/) ORDERED=false | Double-array trie | 1105.47 | 1100.05 | 668.75 | 71.95 |\n| [cedar::da](http://www.tkl.iis.u-tokyo.ac.jp/~ynaga/cedar/) | Double-array reduced trie | 941.16 | 926.04 | 684.38 | 79.11 |\n| [cedar::da](http://www.tkl.iis.u-tokyo.ac.jp/~ynaga/cedar/) ORDERED=false | Double-array reduced trie | 941.16 | 925.98 | 672.14 | 79.02 |\n| [cedar::da](http://www.tkl.iis.u-tokyo.ac.jp/~ynaga/cedar/) | Double-array prefix trie | 714.58 | 712.59 | 831.71 | 75.83 |\n| [cedar::da](http://www.tkl.iis.u-tokyo.ac.jp/~ynaga/cedar/) ORDERED=false | Double-array prefix trie | 714.66 | 712.31 | 786.93 | 75.89 |\n| [hat-trie](https://github.com/dcjones/hat-trie)\u003csup\u003e3\u003c/sup\u003e (C) | HAT-trie | 786.93 | 784.32 | 743.34 | 93.58 |\n| [qp trie](https://github.com/fanf2/qp) (C) | QP trie | 1800.02 | 1797.21 | 987.95 | 428.51 |\n| [crit-bit trie](https://github.com/fanf2/qp) (C) | Crit-bit trie | 2210.52 | 2207.64 | 1986.19 | 1109.88 |\n| [JudySL](http://judy.sourceforge.net/) (C) | Judy array | 1025.59 | 1023.11 | 535.02 | 202.36 |\n| [JudyHS](http://judy.sourceforge.net/) (C) | Judy array | 1002.50 | 999.97 | 456.09 | 148.36 |\n| [tsl::array_map](https://github.com/Tessil/array-hash) | Array hash table | 1308.08 | 1031.67 | 545.82 | 46.41 |\n| [tsl::array_map](https://github.com/Tessil/array-hash) \u003cbr\u003ewith reserve | Array hash table | 979.44 | 921.363 | 244.19 | 45.74  |\n| [tsl::hopscotch_map](https://github.com/Tessil/hopscotch-map) | Hash table | 2336.39 | 1611.54 | 288.70 | 47.05 |\n| [tsl::hopscotch_map](https://github.com/Tessil/hopscotch-map) \u003cbr\u003ewith reserve | Hash table | 1614.22 | 1611.64 | **220.67** | 46.39 |\n| [google::dense_hash_map](https://github.com/sparsehash/sparsehash) | Hash table | 3913.64 | 2636.31 | 317.66 | 43.62 |\n| [google::dense_hash_map](https://github.com/sparsehash/sparsehash) \u003cbr\u003ewith reserve | Hash table | 2638.19 | 2635.68 | 227.58 | **43.09** |\n| [spp::sparse_hash_map](https://github.com/greg7mdp/sparsepp) | Sparse hash table | 1419.69 | 1417.61 | 586.26 | 56.00 |\n| [spp::sparse_hash_map](https://github.com/greg7mdp/sparsepp) \u003cbr\u003ewith reserve | Sparse hash table | 1424.21 | 1421.69 | 392.76 | 55.73 |\n| [std::unordered_map](http://en.cppreference.com/w/cpp/container/unordered_map) | Hash table | 2112.66 | 2110.19 | 554.02 | 105.05 |\n| [std::unordered_map](http://en.cppreference.com/w/cpp/container/unordered_map) \u003cbr\u003ewith reserve | Hash table | 2053.95 | 2051.67 | 309.06 | 109.89 |\n\n3. As the hash function can't be passed in parameter, the code of the library itself is modified to use CityHash64.\n\n### Installation\nTo use the library, just add the [include](include/) directory to your include path. It is a **header-only** library.\n\nIf you use CMake, you can also use the `tsl::hat_trie` exported target from the [CMakeLists.txt](CMakeLists.txt) with `target_link_libraries`. \n```cmake\n# Example where the hat-trie project is stored in a third-party directory\nadd_subdirectory(third-party/hat-trie)\ntarget_link_libraries(your_target PRIVATE tsl::hat_trie)  \n```\n\nThe code should work with any C++11 standard-compliant compiler and has been tested with GCC 4.8.4, Clang 3.5.0 and Visual Studio 2015.\n\nTo run the tests you will need the Boost Test library and CMake. \n\n```bash\ngit clone https://github.com/Tessil/hat-trie.git\ncd hat-trie/tests\nmkdir build\ncd build\ncmake ..\ncmake --build .\n./tsl_hat_trie_tests\n```\n\n### Usage\n\nThe API can be found [here](https://tessil.github.io/hat-trie/doc_without_string_view/html). If `std::string_view` is available, the API changes slightly and can be found [here](https://tessil.github.io/hat-trie/doc/html/).\n\n### Example\n\n```c++\n#include \u003ciostream\u003e\n#include \u003cstring\u003e\n#include \u003ctsl/htrie_map.h\u003e\n#include \u003ctsl/htrie_set.h\u003e\n\n\nint main() {\n    /*\n     * Map of strings to int having char as character type. \n     * There is no support for wchar_t, char16_t or char32_t yet, \n     * but UTF-8 strings will work fine.\n     */\n    tsl::htrie_map\u003cchar, int\u003e map = {{\"one\", 1}, {\"two\", 2}};\n    map[\"three\"] = 3;\n    map[\"four\"] = 4;\n    \n    map.insert(\"five\", 5);\n    map.insert_ks(\"six_with_extra_chars_we_ignore\", 3, 6);\n    \n    map.erase(\"two\");\n    \n    /*\n     * Due to the compression on the common prefixes, the letters of the string \n     * are not always stored contiguously. When we retrieve the key, we have to \n     * construct it.\n     * \n     * To avoid a heap-allocation at each iteration (when SSO doesn't occur), \n     * we reuse the key_buffer to construct the key.\n     */\n    std::string key_buffer;\n    for(auto it = map.begin(); it != map.end(); ++it) {\n        it.key(key_buffer);\n        std::cout \u003c\u003c \"{\" \u003c\u003c key_buffer \u003c\u003c \", \" \u003c\u003c it.value() \u003c\u003c \"}\" \u003c\u003c std::endl;\n    }\n    \n    /*\n     * If you don't care about the allocation.\n     */\n    for(auto it = map.begin(); it != map.end(); ++it) {\n        std::cout \u003c\u003c \"{\" \u003c\u003c it.key() \u003c\u003c \", \" \u003c\u003c *it \u003c\u003c \"}\" \u003c\u003c std::endl;\n    }\n    \n    \n    \n    \n    tsl::htrie_map\u003cchar, int\u003e map2 = {{\"apple\", 1}, {\"mango\", 2}, {\"apricot\", 3},\n                                      {\"mandarin\", 4}, {\"melon\", 5}, {\"macadamia\", 6}};\n    \n    // Prefix search\n    auto prefix_range = map2.equal_prefix_range(\"ma\");\n    \n    // {mandarin, 4} {mango, 2} {macadamia, 6}\n    for(auto it = prefix_range.first; it != prefix_range.second; ++it) {\n        std::cout \u003c\u003c \"{\" \u003c\u003c it.key() \u003c\u003c \", \" \u003c\u003c *it \u003c\u003c \"}\" \u003c\u003c std::endl;\n    }\n    \n    // Find longest match prefix.\n    auto longest_prefix = map2.longest_prefix(\"apple juice\");\n    if(longest_prefix != map2.end()) {\n        // {apple, 1}\n        std::cout \u003c\u003c \"{\" \u003c\u003c longest_prefix.key() \u003c\u003c \", \" \n                  \u003c\u003c *longest_prefix \u003c\u003c \"}\" \u003c\u003c std::endl;\n    }\n    \n    // Prefix erase\n    map2.erase_prefix(\"ma\");\n    \n    // {apricot, 3} {melon, 5} {apple, 1}\n    for(auto it = map2.begin(); it != map2.end(); ++it) {\n        std::cout \u003c\u003c \"{\" \u003c\u003c it.key() \u003c\u003c \", \" \u003c\u003c *it \u003c\u003c \"}\" \u003c\u003c std::endl;\n    }\n    \n    \n    \n    \n    tsl::htrie_set\u003cchar\u003e set = {\"one\", \"two\", \"three\"};\n    set.insert({\"four\", \"five\"});\n    \n    // {one} {two} {five} {four} {three}\n    for(auto it = set.begin(); it != set.end(); ++it) {\n        it.key(key_buffer);\n        std::cout \u003c\u003c \"{\" \u003c\u003c key_buffer \u003c\u003c \"}\" \u003c\u003c std::endl;\n    }\n} \n```\n\n#### Serialization\n\nThe library provides an efficient way to serialize and deserialize a map or a set so that it can be saved to a file or send through the network.\nTo do so, it requires the user to provide a function object for both serialization and deserialization.\n\n```c++\nstruct serializer {\n    // Must support the following types for U: std::uint64_t, float and T if a map is used.\n    template\u003ctypename U\u003e\n    void operator()(const U\u0026 value);\n    void operator()(const CharT* value, std::size_t value_size);\n};\n```\n\n```c++\nstruct deserializer {\n    // Must support the following types for U: std::uint64_t, float and T if a map is used.\n    template\u003ctypename U\u003e\n    U operator()();\n    void operator()(CharT* value_out, std::size_t value_size);\n};\n```\n\nNote that the implementation leaves binary compatibility (endianness, float binary representation, size of int, ...) of the types it serializes/deserializes in the hands of the provided function objects if compatibility is required.\n\nMore details regarding the `serialize` and `deserialize` methods can be found in the [API](https://tessil.github.io/hat-trie/doc/html/classtsl_1_1htrie__map.html).\n\n```c++\n#include \u003ccassert\u003e\n#include \u003ccstdint\u003e\n#include \u003cfstream\u003e\n#include \u003ctype_traits\u003e\n#include \u003ctsl/htrie_map.h\u003e\n\n\nclass serializer {\npublic:\n    serializer(const char* file_name) {\n        m_ostream.exceptions(m_ostream.badbit | m_ostream.failbit);\n        m_ostream.open(file_name);\n    }\n    \n    template\u003cclass T,\n             typename std::enable_if\u003cstd::is_arithmetic\u003cT\u003e::value\u003e::type* = nullptr\u003e\n    void operator()(const T\u0026 value) {\n        m_ostream.write(reinterpret_cast\u003cconst char*\u003e(\u0026value), sizeof(T));\n    }\n    \n    void operator()(const char* value, std::size_t value_size) {\n        m_ostream.write(value, value_size);\n    }\n\nprivate:\n    std::ofstream m_ostream;\n};\n\nclass deserializer {\npublic:\n    deserializer(const char* file_name) {\n        m_istream.exceptions(m_istream.badbit | m_istream.failbit | m_istream.eofbit);\n        m_istream.open(file_name);\n    }\n    \n    template\u003cclass T,\n             typename std::enable_if\u003cstd::is_arithmetic\u003cT\u003e::value\u003e::type* = nullptr\u003e\n    T operator()() {\n        T value;\n        m_istream.read(reinterpret_cast\u003cchar*\u003e(\u0026value), sizeof(T));\n        \n        return value;\n    }\n    \n    void operator()(char* value_out, std::size_t value_size) {\n        m_istream.read(value_out, value_size);\n    }\n\nprivate:\n    std::ifstream m_istream;\n};\n\n\nint main() {\n    const tsl::htrie_map\u003cchar, std::int64_t\u003e map = {{\"one\", 1}, {\"two\", 2}, \n                                                    {\"three\", 3}, {\"four\", 4}};\n    \n    \n    const char* file_name = \"htrie_map.data\";\n    {\n        serializer serial(file_name);\n        map.serialize(serial);\n    }\n    \n    {\n        deserializer dserial(file_name);\n        auto map_deserialized = tsl::htrie_map\u003cchar, std::int64_t\u003e::deserialize(dserial);\n        \n        assert(map == map_deserialized);\n    }\n    \n    {\n        deserializer dserial(file_name);\n        \n        /**\n         * If the serialized and deserialized map are hash compatibles (see conditions in API), \n         * setting the argument to true speed-up the deserialization process as we don't have \n         * to recalculate the hash of each key. We also know how much space each bucket needs.\n         */\n        const bool hash_compatible = true;\n        auto map_deserialized = \n            tsl::htrie_map\u003cchar, std::int64_t\u003e::deserialize(dserial, hash_compatible);\n        \n        assert(map == map_deserialized);\n    }\n}\n```\n\n##### Serialization with Boost Serialization and compression with zlib\n\nIt's possible to use a serialization library to avoid some of the boilerplate if the types to serialize are more complex.\n\nThe following example uses Boost Serialization with the Boost zlib compression stream to reduce the size of the resulting serialized file.\n\n\n```c++\n#include \u003cboost/archive/binary_iarchive.hpp\u003e\n#include \u003cboost/archive/binary_oarchive.hpp\u003e\n#include \u003cboost/iostreams/filter/zlib.hpp\u003e\n#include \u003cboost/iostreams/filtering_stream.hpp\u003e\n#include \u003cboost/serialization/split_free.hpp\u003e\n#include \u003cboost/serialization/utility.hpp\u003e\n#include \u003ccassert\u003e\n#include \u003ccstdint\u003e\n#include \u003cfstream\u003e\n#include \u003ctsl/htrie_map.h\u003e\n\n\ntemplate\u003ctypename Archive\u003e\nstruct serializer {\n    Archive\u0026 ar;\n    \n    template\u003ctypename T\u003e\n    void operator()(const T\u0026 val) { ar \u0026 val; }\n    \n    template\u003ctypename CharT\u003e\n    void operator()(const CharT* val, std::size_t val_size) {\n        ar.save_binary(reinterpret_cast\u003cconst void*\u003e(val), val_size*sizeof(CharT));\n    }   \n};\n\ntemplate\u003ctypename Archive\u003e\nstruct deserializer {\n    Archive\u0026 ar;\n    \n    template\u003ctypename T\u003e\n    T operator()() { T val; ar \u0026 val; return val; }\n    \n    template\u003ctypename CharT\u003e\n    void operator()(CharT* val_out, std::size_t val_size) {\n        ar.load_binary(reinterpret_cast\u003cvoid*\u003e(val_out), val_size*sizeof(CharT));\n    }  \n};\n\nnamespace boost { namespace serialization {\ntemplate\u003cclass Archive, class CharT, class T\u003e\nvoid serialize(Archive \u0026 ar, tsl::htrie_map\u003cCharT, T\u003e\u0026 map, const unsigned int version) {\n    split_free(ar, map, version); \n}\n\ntemplate\u003cclass Archive, class CharT, class T\u003e\nvoid save(Archive \u0026 ar, const tsl::htrie_map\u003cCharT, T\u003e\u0026 map, const unsigned int version) {\n    serializer\u003cArchive\u003e serial{ar};\n    map.serialize(serial);\n}\n\n\ntemplate\u003cclass Archive, class CharT, class T\u003e\nvoid load(Archive \u0026 ar, tsl::htrie_map\u003cCharT, T\u003e\u0026 map, const unsigned int version) {\n    deserializer\u003cArchive\u003e deserial{ar};\n    map = tsl::htrie_map\u003cCharT, T\u003e::deserialize(deserial);\n}\n}}\n\n\nint main() {\n    const tsl::htrie_map\u003cchar, std::int64_t\u003e map = {{\"one\", 1}, {\"two\", 2}, \n                                                    {\"three\", 3}, {\"four\", 4}};\n    \n    \n    const char* file_name = \"htrie_map.data\";\n    {\n        std::ofstream ofs;\n        ofs.exceptions(ofs.badbit | ofs.failbit);\n        ofs.open(file_name, std::ios::binary);\n        \n        boost::iostreams::filtering_ostream fo;\n        fo.push(boost::iostreams::zlib_compressor());\n        fo.push(ofs);\n        \n        boost::archive::binary_oarchive oa(fo);\n        \n        oa \u003c\u003c map;\n    }\n    \n    {\n        std::ifstream ifs;\n        ifs.exceptions(ifs.badbit | ifs.failbit | ifs.eofbit);\n        ifs.open(file_name, std::ios::binary);\n        \n        boost::iostreams::filtering_istream fi;\n        fi.push(boost::iostreams::zlib_decompressor());\n        fi.push(ifs);\n        \n        boost::archive::binary_iarchive ia(fi);\n     \n        tsl::htrie_map\u003cchar, std::int64_t\u003e map_deserialized;   \n        ia \u003e\u003e map_deserialized;\n        \n        assert(map == map_deserialized);\n    }\n}\n```\n\n### License\n\nThe code is licensed under the MIT license, see the [LICENSE file](LICENSE) for details.\n","funding_links":[],"categories":["TODO scan for Android support in followings","Containers","C++"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FTessil%2Fhat-trie","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FTessil%2Fhat-trie","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FTessil%2Fhat-trie/lists"}