https://github.com/adrielcafe/pufferdb
:blowfish: An Android & JVM key-value storage powered by Protobuf and Coroutines
https://github.com/adrielcafe/pufferdb
android android-library coroutines database key-value-database key-value-store kotlin kotlin-android kotlin-library protobuf protocol-buffers rxjava2 storage
Last synced: 11 days ago
JSON representation
:blowfish: An Android & JVM key-value storage powered by Protobuf and Coroutines
- Host: GitHub
- URL: https://github.com/adrielcafe/pufferdb
- Owner: adrielcafe
- License: mit
- Created: 2019-04-02T19:49:35.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2021-03-02T22:05:16.000Z (about 4 years ago)
- Last Synced: 2024-11-07T13:38:48.548Z (6 months ago)
- Topics: android, android-library, coroutines, database, key-value-database, key-value-store, kotlin, kotlin-android, kotlin-library, protobuf, protocol-buffers, rxjava2, storage
- Language: Kotlin
- Homepage:
- Size: 332 KB
- Stars: 100
- Watchers: 6
- Forks: 5
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE.md
Awesome Lists containing this project
- awesome-list - adrielcafe/pufferdb - :blowfish: An Android & JVM key-value storage powered by Protobuf and Coroutines (Kotlin)
README
[](https://jitpack.io/#adrielcafe/pufferdb)
[](https://android-arsenal.com/api?level=14)
[](https://app.bitrise.io/app/9cf30678e9638eba)
[](https://codecov.io/gh/adrielcafe/pufferdb)
[](https://www.codacy.com/app/adriel_cafe/pufferdb)
[](https://kotlinlang.org/)
[](https://ktlint.github.io/)
[](https://opensource.org/licenses/MIT)
#  PufferDB
**PufferDB** is a :zap: key-value storage powered by **P**rotocol B**uffer**s (aka [Protobuf](https://developers.google.com/protocol-buffers/)) and [Coroutines](https://kotlinlang.org/docs/reference/coroutines-overview.html).
The purpose of this library is to provide an efficient, reliable and **Android independent** storage.
Why Android independent? The [SharedPreferences](https://developer.android.com/reference/android/content/SharedPreferences) and many great third-party libraries (like [Paper](https://github.com/pilgr/Paper/) and [MMKV](https://github.com/Tencent/MMKV/)) requires the Android Context to work. But if you are like me and want a **kotlin-only data module** (following the principles of [Clean Architecture](https://antonioleiva.com/clean-architecture-android/)), this library is for you!
This project started as a library module in one of my personal projects, but I decided to open source it and add more features for general use. Hope you like!
### About Protobuf
Protocol Buffers are a language-neutral, platform-neutral extensible mechanism for serializing structured data. Compared to JSON, Protobuf files are [smaller and faster](https://auth0.com/blog/beating-json-performance-with-protobuf/) to read/write because the data is stored in an [efficient binary format](https://developers.google.com/protocol-buffers/docs/encoding).
## Features
* [Fast](#benchmark)
* Works on [Android and JVM](#platform-compatibility)
* [Simple API](#core)
* [Thread-safe](#threading)
* Wrappers for [Coroutines](#coroutines) and [RxJava](#rxjava)
### Supported types
So far, PufferDB supports the following types:
- [x] `Double` and `List`
- [x] `Float` and `List`
- [x] `Int` and `List`
- [x] `Long` and `List`
- [x] `Boolean` and `List`
- [x] `String` and `List`
## Getting Started
### Import to your project
1. Add the JitPack repository in your root build.gradle at the end of repositories:
```gradle
allprojects {
repositories {
maven { url 'https://jitpack.io' }
}
}
```
2. Next, add the desired dependencies to your module:
```gradle
dependencies {
// Core library
implementation "com.github.adrielcafe.pufferdb:core:$currentVersion"
// Android helper
implementation "com.github.adrielcafe.pufferdb:android:$currentVersion"
// Coroutines wrapper
implementation "com.github.adrielcafe.pufferdb:coroutines:$currentVersion"
// RxJava wrapper
implementation "com.github.adrielcafe.pufferdb:rxjava:$currentVersion"
}
```
Current version: [](https://jitpack.io/#adrielcafe/pufferdb)
### Platform compatibility
| | `core` | `android` | `coroutines` | `rxjava` |
|---------|--------|-----------|--------------|----------|
| Android | ✓ | ✓ | ✓ | ✓ |
| JVM | ✓ | | ✓ | ✓ |
## Core
As the name suggests, Core is a standalone module and all other modules depends on it.
To create a new `Puffer` instance you must tell which file to use.
```kotlin
val pufferFile = File("path/to/puffer/file")
val puffer = PufferDB.with(pufferFile)
// or
val puffer = PufferDB.with(pufferFile, myCoroutineScope, myCoroutineDispatcher)
```
If you are on Android, I recommend to use the [Context.filesDir](https://developer.android.com/training/data-storage/files#WriteFileInternal) as the parent folder. If you want to save in the external storage remember to [ask for write permission](https://developer.android.com/training/data-storage/files#ExternalStoragePermissions) first.
Its API is similar to `SharedPreferences`:
```kotlin
puffer.apply {
val myValue = get("myKey")
val myValueWithDefault = get("myKey", "defaultValue")
put("myOtherKey", 123)
getKeys().forEach { key ->
// ...
}
if(contains("myKey")){
// ...
}
remove("myOtherKey")
removeAll()
}
```
But unlike `SharedPreferences`, there's no `apply()` or `commit()`. Changes are saved asynchronously every time a write operation (`put()`, `remove()` and `removeAll()`) happens.
### Threading
PufferDB uses a [ConcurrentHashMap](https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ConcurrentHashMap.html) to manage a thread-safe in-memory cache for fast read and write operations.
Changes are saved asynchronously with the help of a [StateFlow](https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/-state-flow/index.html) (to save the most recent state in a race condition) and [Mutex](https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.sync/-mutex/index.html) locker (to prevent simultaneous writes).
It is possible to run the API methods on the Android Main Thread, but you should *avoid that*. You can use one of the wrapper modules or built in extension functions for that (listed below).
## Android
The Android module contains an `AndroidPufferDB` helper class:
```kotlin
class MyApp : Application() {
override fun onCreate() {
super.onCreate()
// Init the PufferDB when your app starts
AndroidPufferDB.init(this)
}
}
// Now you can use it anywhere in your app
class MyActivity : AppCompatActivity() {
// Returns a default Puffer instance, the file is located on Context.filesDir
val corePuffer = AndroidPufferDB.withDefault()
// Returns a File that should be used to create a Puffer instance
val pufferFile = AndroidPufferDB.getInternalFile("my.db")
val coroutinePuffer = CoroutinePufferDB.with(pufferFile)
}
```
## Coroutines
The Coroutines module contains a `CoroutinePufferDB` wrapper class and some useful extension functions:
```kotlin
val pufferFile = File("path/to/puffer/file")
val puffer = CoroutinePufferDB.with(pufferFile)
// All methods are suspend functions that runs on Dispatchers.IO context
launch {
puffer.apply {
val myValue = get("myKey")
val myValueWithDefault = get("myKey", "defaultValue")
put("myOtherKey", 123)
getKeys().forEach { key ->
// ...
}
if(contains("myKey")){
// ...
}
remove("myOtherKey")
removeAll()
}
}
```
If you don't want to use this wrapper class, there's some built in extension functions that can be used with the Core module:
```kotlin
val pufferFile = File("path/to/puffer/file")
val puffer = PufferDB.with(pufferFile) // <- Note that we're using the Core PufferDB
launch {
puffer.apply {
val myValue = getSuspend("myKey")
val myValue = getAsync("myKey").await()
// You can use your own coroutine scope and dispatcher
putSuspend("myOtherKey", 123, myCoroutineScope, myCoroutineDispatcher)
putAsync("myOtherKey", 123, myActivityScope).await()
}
}
```
## RxJava
The RxJava module contains a `RxPufferDB` wrapper class and some useful extension functions:
```kotlin
val pufferFile = File("path/to/puffer/file")
val puffer = RxPufferDB.with(pufferFile)
puffer.apply {
// Some methods returns Single...
get("myKey") // OR get("myKey", "defaultValue")
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe { myValue ->
// ...
}
// ... And others returns Completable
put("myOtherKey", 123)
// ...
.subscribe {
// ...
}
getKeys()
// ...
.subscribe { keys ->
// ...
}
contains("myKey")
// ...
.subscribe { contains ->
// ...
}
remove("myOtherKey")
// ...
.subscribe {
// ...
}
removeAll()
// ...
.subscribe {
// ...
}
}
```
Like the Coroutines module, the RxJava module also provides some useful built in extension functions that can be used with the Core module:
```kotlin
val pufferFile = File("path/to/puffer/file")
val puffer = PufferDB.with(pufferFile) // <- Note that we're using the Core PufferDB
puffer.apply {
val myValue = getSingle("myKey").blockingGet()
putCompletable("myOtherKey", 123).blockingAwait()
getKeysObservable().blockingSubscribe { keys ->
// ...
}
}
```
## Benchmark
### Write & Read
| | Write 1k strings (ms) | Read 1k strings (ms) |
|-------------------|-----------------------|----------------------|
| **PufferDB** | **20** | **5** |
| [SharedPreferences](https://developer.android.com/training/data-storage/shared-preferences) | 278 | 7 |
| [MMKV](https://github.com/Tencent/MMKV) | 13 | 8 |
| [Paper](https://github.com/pilgr/Paper) | 818 | 169 |
| [Binary Prefs](https://github.com/yandextaxitech/binaryprefs) | 121 | 9 |
| [Hawk](https://github.com/orhanobut/hawk) | 15183 | 207 |
| | Write 100k strings (ms) | Read 100k strings (ms) |
|-------------------|-----------------------|----------------------|
| **PufferDB** | **259** | **32** |
| [SharedPreferences](https://developer.android.com/training/data-storage/shared-preferences) | 💥 | 💥 |
| [MMKV](https://github.com/Tencent/MMKV) | 871 | 516 |
| [Paper](https://github.com/pilgr/Paper) | 💥 | 💥 |
| [Binary Prefs](https://github.com/yandextaxitech/binaryprefs) | 1082 | 101 |
| [Hawk](https://github.com/orhanobut/hawk) | 💥 | 💥 |
### File size
| | 1k strings (kb) |
|-------------------|-----------------------|
| **PufferDB** | **25** |
| [SharedPreferences](https://developer.android.com/training/data-storage/shared-preferences) | 20 |
| [MMKV](https://github.com/Tencent/MMKV) | 40 |
| [Paper](https://github.com/pilgr/Paper) | 61 |
| [Binary Prefs](https://github.com/yandextaxitech/binaryprefs) | 53 |
| [Hawk](https://github.com/orhanobut/hawk) | 27 |
| | 100k strings (kb) |
|-------------------|-----------------------|
| **PufferDB** | **2.907** |
| [SharedPreferences](https://developer.android.com/training/data-storage/shared-preferences) | 💥 |
| [MMKV](https://github.com/Tencent/MMKV) | 4.104 |
| [Paper](https://github.com/pilgr/Paper) | 💥 |
| [Binary Prefs](https://github.com/yandextaxitech/binaryprefs) | 5.175 |
| [Hawk](https://github.com/orhanobut/hawk) | 💥 |
*Tested on Moto Z2 Plus*
You can run the [Benchmark](https://github.com/adrielcafe/PufferDB/blob/master/sample/src/main/kotlin/cafe/adriel/pufferdb/sample/Benchmark.kt) through the sample app.