https://github.com/chfoo/resdb
Read-only, key-value database using a resource embedding system for Haxe
https://github.com/chfoo/resdb
Last synced: 5 months ago
JSON representation
Read-only, key-value database using a resource embedding system for Haxe
- Host: GitHub
- URL: https://github.com/chfoo/resdb
- Owner: chfoo
- License: mit
- Created: 2018-07-25T03:52:43.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2018-09-05T19:20:54.000Z (over 7 years ago)
- Last Synced: 2023-10-20T18:49:00.450Z (over 2 years ago)
- Language: Haxe
- Size: 43.9 KB
- Stars: 2
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE.txt
Awesome Lists containing this project
README
Resdb
=====
Resdb is a read-only, key-value database using a resource embedding system for Haxe. It can be used for querying moderately sized datasets in a portable manner.
Getting started
---------------
Requires Haxe 3 or 4.
Install the library using haxelib:
haxelib install resdb
Alternatively, you may install the lastest git version:
haxelib git resdb
### Packing records into pages
Resdb stores the records into separate resources called pages. To prepare the records, use a page packer:
```haxe
var pagePacker = new PagePacker({name: "myDatabase"});
```
Then add your records:
```haxe
pagePacker.addRecord(Bytes.ofString("key1"), Bytes.ofString("value1"));
pagePacker.addRecord(Bytes.ofString("key2"), Bytes.ofString("value2"));
```
The database should be prepared outside the runtime of your program. You can do this during compile time with an initialization macro.
A helper function that uses Haxe's resource system is provided:
```haxe
class MyDatabase {
#if macro
public static function packAndEmbed() {
// ...
// Your page packer code
// ...
ResourceHelper.addResource(pagePacker);
}
#end
}
```
Then, add the following compile time option:
```
--macro MyDatabase.packAndEmbed()
```
### Compiler performance
If your IDE uses compiler completion services, you may want to use conditional compilation to skip embedding the resources to avoid slowing down the completion service:
```haxe
#if !embed_resources
return;
#else
// Your page packer code
```
Then, specify `-D embed_resources` argument when building to a target.
### Accessing the database
When you need to access your database in your program, you need to obtain the appropriate page store. When using Haxe's system, a helper function is provided:
```haxe
var database = ResourcePageStore.getDatabase({name: "myDatabase"});
```
You can then retrieve a record's value using a key:
```haxe
switch database.get(Bytes.ofString("key1")) {
case Some(value):
trace('Value = $value');
case None:
trace("Not found");
}
```
### Cursors
Cursors provide a way to navigate and query records by key in sorted order. You can create a cursor from your database instance:
```haxe
var cursor = database.cursor();
```
Then you can navigate in order, or to the first or last key:
```haxe
var firstKey = cursor.first();
var lastKey = cursor.last();
switch cursor.next() {
case Some(nextKey):
trace('Got next $nextKey');
case None:
trace("At last key");
}
switch cursor.previous() {
case Some(previousKey):
trace('Got next $previousKey');
case None:
trace("At last key");
}
```
You can get the current record that the cursor points to:
```haxe
var key = cursor.key();
var value = cursor.value();
```
To query for a key, use the `find()` method:
```haxe
var resultKey = cursor.find(Bytes.ofString("key5"));
```
The `find()` method will return the nearest key that is equal or less than the given key. The ordering is determined by `Bytes.compare`. In the example above, given a database containing keys `"key2", "key4", "key8"`, the resulting key will be `"key4"`.
### Adapters
While the underlying database uses `Bytes`, it may be more useful to use alternative types. By using `CursorAdapter`, and static extensions, you can bring your own wrappers and serializers.
The library includes `IntAdapter` that allows the use of integer `Int` as keys:
```haxe
using resdb.adapter.IntAdapter;
var database = ResourcePageStore.getDatabase("myDatabase");
var result = database.intGet(123);
trace(result);
var cursor = database.intCursor();
var findResult = cursor.find(456);
trace(findResult);
```
### Further details
* [API documentation](https://chfoo.github.io/resdb/api/)
OpenFL Assets
-------------
The OpenFL asset system support demonstrates how to use Resdb with another resource system.
Prepare the database pages by packing and saving them to your assets directory:
```haxe
var pagePacker = new PagePacker({ name: "myDatabase" });
// [do packing]
FilePageStore.saveDataPages("assets/", "myDatabase", pagePacker.packPages());
```
Then in your application, use:
```haxe
var database = OpenFLAssetPageStore.getDatabase({ name: "assets/myDatabase" });
```
Database format
---------------
The database stores records into pages. When the page packer processes the records, it sorts them by key, and fills pages with the serialized data. Records are not equally distributed and pages are filled up sequentially without splitting a record. As such, the maximum page size dictates how large a record may be.
Once all the records are packed and serialized into pages, the ranges of the records for each page are collected and serialized as the first page. The ranges serve as an index for looking up the appropriate page with the given key.
The maximum number of records & pages and the maximum length of keys & values is 2147483647 which is the limit of the integer type.
Compiler flags
--------------
To gain some performance, some compiler flags (`-D`) are provided to enable target specific optimizations.
| Flag | Target | Description |
|------|--------|-------------|
| `resdb-js-optimize-page-store` | JS (HTML) | In `ResourcePageStore`, uses native `atob()` for base64 decoding as a replacement for `haxe.Resource.getBytes()`. |