https://github.com/clivern/langmore
🔥 A KV Store Based On Write-Ahead Log.
https://github.com/clivern/langmore
caching kv-store memcached redis rust rust-lang write-ahead-log
Last synced: 6 months ago
JSON representation
🔥 A KV Store Based On Write-Ahead Log.
- Host: GitHub
- URL: https://github.com/clivern/langmore
- Owner: Clivern
- License: mit
- Created: 2021-01-09T20:34:20.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2024-11-07T13:56:02.000Z (11 months ago)
- Last Synced: 2024-11-07T14:36:24.407Z (11 months ago)
- Topics: caching, kv-store, memcached, redis, rust, rust-lang, write-ahead-log
- Language: Rust
- Homepage: https://crates.io/crates/langmore
- Size: 9.3 MB
- Stars: 1
- Watchers: 2
- Forks: 1
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- Funding: .github/FUNDING.yml
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
README
Langmore
A KV Store Based On Write-Ahead Log
## Design of Langmore
`Langmore` uses a lot of principles from log-structured file systems and draws inspiration from a number of designs that involve log file merging. It essentially is just a directory of append-only (log) files with a fixed structure and an in-memory index holding the keys mapped to a bunch of information necessary for point lookups - referring to the entry in the datafile.
### Datafiles
They re append-only log files that hold the KV pairs along with some meta-information. A single `Langmore` instance could have many datafiles, out of which just one will be active and opened for writing, while the others are considered immutable and are only used for reads.
Each entry in the datafile has a fixed structure illustrated above and it stores `CRC`, `timestamp`, `key_size`, `value_size`, `actual_key`, and the `actual_value`. All the write operations - create, update and delete - made on the engine translates into entries in this active datafile. When this active datafile meets a size threshold, it is closed and a new active datafile is created. when closed (intentionally or unintentionally), the datafile is considered immutable and is never opened for writing again.
### KeyDir
It is an in-memory hash table that stores all the keys present in the Langmore instance and maps it to the offset in the datafile where the log entry (value) resides; thus facilitating the point lookups. The mapped value in the Hash Table is a structure that holds `file_id`, `offset`, and some meta-information like timestamp, as illustrated below.
## Deployment
Build the project with the following command
```bash
$ make build
```
or install with cargo
```bash
$ cargo install langmore
```
Define the configs and run the `Langmore` binary.
```bash
export HOSTNAME=127.0.0.1:8080
export STORAGE_DIR=/etc/langmore
$ ./target/debug/langmore
```
## Usage
You can use `netcat` to interact with `Langmore`
```bash
$ nc 127.0.0.1 8080
```
## Versioning
For transparency into our release cycle and in striving to maintain backward compatibility, Langmore is maintained under the [Semantic Versioning guidelines](https://semver.org/) and release process is predictable and business-friendly.
See the [Releases section of our GitHub project](https://github.com/clivern/langmore/releases) for changelogs for each release version of Langmore. It contains summaries of the most noteworthy changes made in each release.
## Bug tracker
If you have any suggestions, bug reports, or annoyances please report them to our issue tracker at https://github.com/clivern/langmore/issues
## Security Issues
If you discover a security vulnerability within Langmore, please send an email to [hello@clivern.com](mailto:hello@clivern.com)
## Contributing
We are an open source, community-driven project so please feel free to join us. see the [contributing guidelines](CONTRIBUTING.md) for more details.
## License
© 2022, clivern. Released under [MIT License](https://opensource.org/licenses/mit-license.php).
**Langmore** is authored and maintained by [@Clivern](http://github.com/clivern).