https://github.com/a-poor/brickdb
A small basic proof-of-concept database written in Rust. I wouldn't recommend using this in production.
https://github.com/a-poor/brickdb
database rust storage
Last synced: 4 months ago
JSON representation
A small basic proof-of-concept database written in Rust. I wouldn't recommend using this in production.
- Host: GitHub
- URL: https://github.com/a-poor/brickdb
- Owner: a-poor
- License: mit
- Created: 2023-06-13T01:12:22.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2024-12-01T02:06:34.000Z (over 1 year ago)
- Last Synced: 2025-05-02T05:04:49.925Z (about 1 year ago)
- Topics: database, rust, storage
- Language: Rust
- Homepage: https://crates.io/crates/brickdb
- Size: 122 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# BrickDB
[](https://github.com/a-poor/brickdb/actions/workflows/rust-test.yml)
[](https://crates.io/crates/brickdb)
[](https://crates.io/crates/brickdb)
[](https://docs.rs/brickdb)
_created by Austin Poor_
A small basic proof-of-concept database written in Rust. I wouldn't recommend using this in production.
## Ideas for the Future
- Sharding
- Multi-node clusters (via Raft?)
- Schema validation (e.g )
## Notes
- [x] Update the `storage::level::Level::new` implementation to format the path using the `path` argument as a _parent directory path_
- [x] The LSMTree should be able to move data between levels (eg memtable -> level-1-sstable) without downtime (if possible). Can it craete an sstable from the memtable, write the memtable to disk, and then clear the memtable? Should there be a _frozen_ memtable that can be read from but not written to, while the on-disk operations are working?
- [x] Like the above, should also (maybe) work with SSTables? Or do they not need to, since they're read-only?
- Maybe they don't need to but need to be able to mark which table-ids are being compacted. So compaction can start on level 1 (say) and while it's happening, they can still be read from, until the new level-2 table is ready, at which point the level-1 tables can be removed. But since a new level one table might have been added (probably _shouldn't_ have been but _could_ have been), the compaction process should remember which tables it's including and not just say _all tables in level 1_.
- [x] Compress data written to disk with snappy compression? (try `snap`)
- [ ] There was an error with the snappy compression that I didn't *really* look into deeply. Go back and figure it out!
- [ ] Update `todo!()`s in tests (and the commented-out tests)
- [ ] Add the ability to encrypt data on disk (`aes` with [ring](https://docs.rs/ring/latest/ring)?)
- [ ] The `bloom` create hasn't had any updates in the past 7 years. Consider changing to a different implementation or writing it myself.
- [ ] Add metadata for the `storage::lsm::LSMTree` so it can be read back in.
- [ ] Create a separate reader/writer implementation for reading/writing data. It can simplify async writing of `bson` data, compression (share encoders/decoders?), and encryption.
- [ ] Implement the `WAL`
## Database Disk Structure
```
...
```
## To-Do
- [ ] Define on-disk structure for LSM
- [ ] Implement WAL
- [ ] Add compression when reading/writing from/to disk
- [ ] Add encryption when reading/writing from/to disk
- [ ] Add indexes