https://github.com/kroggen/hash-table-tree
Fast unsorted key/value data structure for databases
https://github.com/kroggen/hash-table-tree
data-structure disk-based hash-table persistent persistent-data-structure
Last synced: 9 months ago
JSON representation
Fast unsorted key/value data structure for databases
- Host: GitHub
- URL: https://github.com/kroggen/hash-table-tree
- Owner: kroggen
- License: mit
- Created: 2018-03-07T03:39:51.000Z (almost 8 years ago)
- Default Branch: master
- Last Pushed: 2018-03-08T16:02:52.000Z (almost 8 years ago)
- Last Synced: 2025-02-09T13:42:22.459Z (10 months ago)
- Topics: data-structure, disk-based, hash-table, persistent, persistent-data-structure
- Size: 30.3 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# hash-table-tree
This is a data structure that I created when I was thinking about enhancing data access speed on databases.
It implements a hash table on database pages.
It uses 2 types of database pages:
1. Index page
2. Data page
The keys and values are stored on the data pages with a data length before each.
When there is no more free space on a data page to store a new or updated key/value pair we:
1. allocate a new index page
2. choose a new hash salt and store it on the index page
3. reshash all the keys in this data page with this new salt and store the pairs on new data pages
So we don't need to rehash the entire database.
By using a different salt than in the previous index pages the hash function will generate different hashes for the same keys, so the keys on that new index page will be spreaded on separate pages.
This action of adding new index pages is limited by the total possible unique salts. Each salt is stored in the index page.
If it uses 1 byte we have 256 possible different salts for each path. This means the db can store at least nearly 1021^256 keys.
This number is way higher if we use 2 bytes to store the salt.
Note that the same salt can be used in other paths and it must be unique only in the same path up to the root page.
## Performance
The performance gain (at least on reads) comes from the amount of pointers on each index page and the consequence of requiring less index pages to be loaded (less disk I/O) when searching for a key.
We can have nearly 1021 pointers in a hash-table-tree 4kB page.
A common B-tree with a 4 byte key has half that number of pointers. And with an 8 bytes key the number of pointers is 1/3.
## Limitations
As in any hash table, it only supports unsorted data.
## About the Code
This code is just a proof-of-concept, it does not implement transactions and it is not safe (yet).
## Contact
Bernardo Ramos, Gensis Sistemas
contact 4T litereplica D0T io