Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/akrylysov/pogreb
Embedded key-value store for read-heavy workloads written in Go
https://github.com/akrylysov/pogreb
go hash-table key-value key-value-store
Last synced: 30 days ago
JSON representation
Embedded key-value store for read-heavy workloads written in Go
- Host: GitHub
- URL: https://github.com/akrylysov/pogreb
- Owner: akrylysov
- License: apache-2.0
- Created: 2018-01-06T23:16:36.000Z (almost 7 years ago)
- Default Branch: main
- Last Pushed: 2024-08-03T13:41:27.000Z (3 months ago)
- Last Synced: 2024-09-27T03:41:05.994Z (about 1 month ago)
- Topics: go, hash-table, key-value, key-value-store
- Language: Go
- Homepage:
- Size: 105 KB
- Stars: 1,297
- Watchers: 34
- Forks: 91
- Open Issues: 17
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
- awesome-go - pogreb - Embedded key-value store for read-heavy workloads. (Database / Databases Implemented in Go)
- awesome-go - pogreb - Embedded key-value store for read-heavy workloads. Stars:`1.3K`. (Database / Databases Implemented in Go)
- awesome-golang-repositories - pogreb - value store for read-heavy workloads written in Go (Repositories)
- awesome-go-extra - pogreb - value store for read-heavy workloads written in Go|923|70|12|2018-01-06T23:16:36Z|2021-08-27T13:45:37Z| (Generators / Databases Implemented in Go)
- awesome-go-storage - pogreb - Embedded key-value store for read-heavy workloads. (Key-Value Store)
README
# Pogreb
[](https://pkg.go.dev/github.com/akrylysov/pogreb)
[](https://github.com/akrylysov/pogreb/actions)
[](https://goreportcard.com/report/github.com/akrylysov/pogreb)
[](https://codecov.io/gh/akrylysov/pogreb)
Pogreb is an embedded key-value store for read-heavy workloads written in Go.
## Key characteristics
- 100% Go.
- Optimized for fast random lookups and infrequent bulk inserts.
- Can store larger-than-memory data sets.
- Low memory usage.
- All DB methods are safe for concurrent use by multiple goroutines.
## Installation
```sh
$ go get -u github.com/akrylysov/pogreb
```
## Usage
### Opening a database
To open or create a new database, use the `pogreb.Open()` function:
```go
package main
import (
"log"
"github.com/akrylysov/pogreb"
)
func main() {
db, err := pogreb.Open("pogreb.test", nil)
if err != nil {
log.Fatal(err)
return
}
defer db.Close()
}
```
### Writing to a database
Use the `DB.Put()` function to insert a new key-value pair:
```go
err := db.Put([]byte("testKey"), []byte("testValue"))
if err != nil {
log.Fatal(err)
}
```
### Reading from a database
To retrieve the inserted value, use the `DB.Get()` function:
```go
val, err := db.Get([]byte("testKey"))
if err != nil {
log.Fatal(err)
}
log.Printf("%s", val)
```
### Deleting from a database
Use the `DB.Delete()` function to delete a key-value pair:
```go
err := db.Delete([]byte("testKey"))
if err != nil {
log.Fatal(err)
}
```
### Iterating over items
To iterate over items, use `ItemIterator` returned by `DB.Items()`:
```go
it := db.Items()
for {
key, val, err := it.Next()
if err == pogreb.ErrIterationDone {
break
}
if err != nil {
log.Fatal(err)
}
log.Printf("%s %s", key, val)
}
```
## Performance
The benchmarking code can be found in the [pogreb-bench](https://github.com/akrylysov/pogreb-bench) repository.
Results of read performance benchmark of pogreb, goleveldb, bolt and badgerdb
on DigitalOcean 8 CPUs / 16 GB RAM / 160 GB SSD + Ubuntu 16.04.3 (higher is better):
## Internals
[Design document](/docs/design.md).
## Limitations
The design choices made to optimize for point lookups bring limitations for other potential use-cases. For example, using a hash table for indexing makes range scans impossible. Additionally, having a single hash table shared across all WAL segments makes the recovery process require rebuilding the entire index, which may be impractical for large databases.