{"id":17984355,"url":"https://github.com/kroggen/hash-table-tree","last_synced_at":"2026-01-18T22:07:38.849Z","repository":{"id":85583157,"uuid":"124174763","full_name":"kroggen/hash-table-tree","owner":"kroggen","description":"Fast unsorted key/value data structure for databases","archived":false,"fork":false,"pushed_at":"2018-03-08T16:02:52.000Z","size":31,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-02-09T13:42:22.459Z","etag":null,"topics":["data-structure","disk-based","hash-table","persistent","persistent-data-structure"],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kroggen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-03-07T03:39:51.000Z","updated_at":"2018-03-10T03:18:00.000Z","dependencies_parsed_at":"2023-03-19T10:22:32.340Z","dependency_job_id":null,"html_url":"https://github.com/kroggen/hash-table-tree","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kroggen%2Fhash-table-tree","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kroggen%2Fhash-table-tree/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kroggen%2Fhash-table-tree/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kroggen%2Fhash-table-tree/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kroggen","download_url":"https://codeload.github.com/kroggen/hash-table-tree/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247107827,"owners_count":20884797,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-structure","disk-based","hash-table","persistent","persistent-data-structure"],"created_at":"2024-10-29T18:20:53.325Z","updated_at":"2026-01-18T22:07:38.843Z","avatar_url":"https://github.com/kroggen.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# hash-table-tree\n\nIt is a data structure for fast disk-based unsorted key-value databases.\n\nIt implements a hash table on database pages.\n\nIt uses 2 types of database pages:\n\n1. Index page\n2. Data page\n\nThe keys and values are stored on the data pages with a data length before each.\n\n![img1](images/img1.png)\n\nWhen there is no more free space on a data page to store a new or updated key/value pair we:\n\n1. allocate a new index page\n2. choose a new hash salt and store it on the index page\n3. reshash all the keys in this data page with this new salt and store the pairs on new data pages\n\n![img2](images/img2.png)\n\nSo we don't need to rehash the entire database.\n\nBy using a different salt than in the previous index pages the hash function will generate different hashes for the same keys, so the keys on that new index page will be spreaded on separate pages.\n\nThis action of adding new index pages is limited by the total possible unique salts. Each salt is stored in the index page.\nIf it uses 1 byte we have 256 possible different salts for each path. This means the db can store at least nearly 1021^256 keys.\nThis number is way higher if we use 2 bytes to store the salt.\n\nNote that the same salt can be used in other paths and it must be unique only in the same path up to the root page.\n\n## Performance\n\nThe performance gain (at least on reads) comes from the amount of pointers on each index page and the consequence of requiring less index pages to be loaded (less disk I/O) when searching for a key.\n\nWe can have nearly 1021 pointers in a hash-table-tree 4kB page.\n\nA common B-tree with a 4 byte key has half that number of pointers. And with an 8 bytes key the number of pointers is 1/3.\n\n## Limitations\n\nAs in any hash table, it only supports unsorted data.\n\nIt uses A LOT of disk space if the key+value size is way lower than 4kB. It is as a trade-off for performance. But it is really good when used to store data nearly to 4090 bytes (key + value + length prefixes).\n\n## About the Code\n\nThis code is just a proof-of-concept, it does not implement transactions and it is not safe (yet).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkroggen%2Fhash-table-tree","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkroggen%2Fhash-table-tree","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkroggen%2Fhash-table-tree/lists"}