{"id":13607987,"url":"https://github.com/adrielcafe/pufferdb","last_synced_at":"2025-04-23T16:34:03.048Z","repository":{"id":47986013,"uuid":"179148040","full_name":"adrielcafe/pufferdb","owner":"adrielcafe","description":":blowfish: An Android \u0026 JVM key-value storage powered by Protobuf and Coroutines","archived":false,"fork":false,"pushed_at":"2021-03-02T22:05:16.000Z","size":340,"stargazers_count":100,"open_issues_count":5,"forks_count":5,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-11-07T13:38:48.548Z","etag":null,"topics":["android","android-library","coroutines","database","key-value-database","key-value-store","kotlin","kotlin-android","kotlin-library","protobuf","protocol-buffers","rxjava2","storage"],"latest_commit_sha":null,"homepage":"","language":"Kotlin","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/adrielcafe.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null},"funding":{"ko_fi":"adrielcafe"}},"created_at":"2019-04-02T19:49:35.000Z","updated_at":"2024-05-20T20:45:52.000Z","dependencies_parsed_at":"2022-08-12T15:52:25.175Z","dependency_job_id":null,"html_url":"https://github.com/adrielcafe/pufferdb","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adrielcafe%2Fpufferdb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adrielcafe%2Fpufferdb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adrielcafe%2Fpufferdb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adrielcafe%2Fpufferdb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/adrielcafe","download_url":"https://codeload.github.com/adrielcafe/pufferdb/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223930294,"owners_count":17227046,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["android","android-library","coroutines","database","key-value-database","key-value-store","kotlin","kotlin-android","kotlin-library","protobuf","protocol-buffers","rxjava2","storage"],"created_at":"2024-08-01T19:01:23.372Z","updated_at":"2024-11-10T08:27:06.014Z","avatar_url":"https://github.com/adrielcafe.png","language":"Kotlin","readme":"[![JitPack](https://img.shields.io/jitpack/v/github/adrielcafe/pufferdb.svg?style=for-the-badge)](https://jitpack.io/#adrielcafe/pufferdb) \n[![Android API](https://img.shields.io/badge/api-14%2B-brightgreen.svg?style=for-the-badge)](https://android-arsenal.com/api?level=14) \n[![Bitrise](https://img.shields.io/bitrise/9cf30678e9638eba/master.svg?style=for-the-badge\u0026token=-KZNny2tfEouRiyRtPQW7A)](https://app.bitrise.io/app/9cf30678e9638eba) \n[![Codecov](https://img.shields.io/codecov/c/github/adrielcafe/pufferdb/master.svg?style=for-the-badge)](https://codecov.io/gh/adrielcafe/pufferdb) \n[![Codacy](https://img.shields.io/codacy/grade/a673b24ba23e4cd1bd1fdc9907aaafd2.svg?style=for-the-badge)](https://www.codacy.com/app/adriel_cafe/pufferdb) \n[![kotlin](https://img.shields.io/github/languages/top/adrielcafe/pufferdb.svg?style=for-the-badge)](https://kotlinlang.org/) \n[![ktlint](https://img.shields.io/badge/code%20style-%E2%9D%A4-FF4081.svg?style=for-the-badge)](https://ktlint.github.io/) \n[![License MIT](https://img.shields.io/github/license/adrielcafe/pufferdb.svg?style=for-the-badge\u0026color=yellow)](https://opensource.org/licenses/MIT) \n\n# ![logo](https://github.com/adrielcafe/pufferdb/blob/master/logo.png?raw=true) PufferDB\n\n**PufferDB** is a :zap: key-value storage powered by **P**rotocol B**uffer**s (aka [Protobuf](https://developers.google.com/protocol-buffers/)) and [Coroutines](https://kotlinlang.org/docs/reference/coroutines-overview.html).\n\nThe purpose of this library is to provide an efficient, reliable and **Android independent** storage. \n\nWhy Android independent? The [SharedPreferences](https://developer.android.com/reference/android/content/SharedPreferences) and many great third-party libraries (like [Paper](https://github.com/pilgr/Paper/) and [MMKV](https://github.com/Tencent/MMKV/)) requires the Android Context to work. But if you are like me and want a **kotlin-only data module** (following the principles of [Clean Architecture](https://antonioleiva.com/clean-architecture-android/)), this library is for you!\n\nThis project started as a library module in one of my personal projects, but I decided to open source it and add more features for general use. Hope you like!\n\n### About Protobuf\n\nProtocol Buffers are a language-neutral, platform-neutral extensible mechanism for serializing structured data. Compared to JSON, Protobuf files are [smaller and faster](https://auth0.com/blog/beating-json-performance-with-protobuf/) to read/write because the data is stored in an [efficient binary format](https://developers.google.com/protocol-buffers/docs/encoding).\n\n## Features\n* [Fast](#benchmark)\n* Works on [Android and JVM](#platform-compatibility)\n* [Simple API](#core)\n* [Thread-safe](#threading)\n* Wrappers for [Coroutines](#coroutines) and [RxJava](#rxjava)\n\n### Supported types\nSo far, PufferDB supports the following types:\n- [x] `Double` and `List\u003cDouble\u003e`\n- [x] `Float` and `List\u003cFloat\u003e`\n- [x] `Int` and `List\u003cInt\u003e`\n- [x] `Long` and `List\u003cLong\u003e`\n- [x] `Boolean` and `List\u003cBoolean\u003e`\n- [x] `String` and `List\u003cString\u003e`\n\n## Getting Started\n\n### Import to your project\n1. Add the JitPack repository in your root build.gradle at the end of repositories:\n```gradle\nallprojects {\n    repositories {\n        maven { url 'https://jitpack.io' }\n    }\n}\n```\n\n2. Next, add the desired dependencies to your module:\n```gradle\ndependencies {\n    // Core library\n    implementation \"com.github.adrielcafe.pufferdb:core:$currentVersion\"\n\n    // Android helper\n    implementation \"com.github.adrielcafe.pufferdb:android:$currentVersion\"\n\n    // Coroutines wrapper\n    implementation \"com.github.adrielcafe.pufferdb:coroutines:$currentVersion\"\n\n    // RxJava wrapper\n    implementation \"com.github.adrielcafe.pufferdb:rxjava:$currentVersion\"\n}\n```\nCurrent version: [![JitPack](https://img.shields.io/jitpack/v/github/adrielcafe/pufferdb.svg?style=flat-square)](https://jitpack.io/#adrielcafe/pufferdb)\n\n### Platform compatibility\n\n|         | `core` | `android` | `coroutines` | `rxjava` |\n|---------|--------|-----------|--------------|----------|\n| Android | ✓      | ✓        | ✓            | ✓       |\n| JVM     | ✓      |           | ✓           | ✓        |\n\n## Core\nAs the name suggests, Core is a standalone module and all other modules depends on it.\n\nTo create a new `Puffer` instance you must tell which file to use. \n```kotlin\nval pufferFile = File(\"path/to/puffer/file\")\nval puffer = PufferDB.with(pufferFile)\n// or\nval puffer = PufferDB.with(pufferFile, myCoroutineScope, myCoroutineDispatcher)\n```\n\nIf you are on Android, I recommend to use the [Context.filesDir](https://developer.android.com/training/data-storage/files#WriteFileInternal) as the parent folder. If you want to save in the external storage remember to [ask for write permission](https://developer.android.com/training/data-storage/files#ExternalStoragePermissions) first.\n\nIts API is similar to `SharedPreferences`:\n```kotlin\npuffer.apply {\n    val myValue = get\u003cString\u003e(\"myKey\")\n    val myValueWithDefault = get(\"myKey\", \"defaultValue\")\n    \n    put(\"myOtherKey\", 123)\n\n    getKeys().forEach { key -\u003e\n        // ...\n    }\n\n    if(contains(\"myKey\")){\n        // ...\n    }\n\n    remove(\"myOtherKey\")\n\n    removeAll()\n}\n```\n\nBut unlike `SharedPreferences`, there's no `apply()` or `commit()`. Changes are saved asynchronously every time a write operation (`put()`, `remove()` and `removeAll()`) happens.\n\n### Threading\nPufferDB uses a [ConcurrentHashMap](https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ConcurrentHashMap.html) to manage a thread-safe in-memory cache for fast read and write operations.\n\nChanges are saved asynchronously with the help of a [StateFlow](https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/-state-flow/index.html) (to save the most recent state in a race condition) and [Mutex](https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.sync/-mutex/index.html) locker (to prevent simultaneous writes).\n\nIt is possible to run the API methods on the Android Main Thread, but you should *avoid that*. You can use one of the wrapper modules or built in extension functions for that (listed below).\n\n## Android\nThe Android module contains an `AndroidPufferDB` helper class:\n```kotlin\nclass MyApp : Application() {\n\n    override fun onCreate() {\n        super.onCreate()\n        // Init the PufferDB when your app starts\n        AndroidPufferDB.init(this)\n    }\n}\n\n// Now you can use it anywhere in your app\nclass MyActivity : AppCompatActivity() {\n\n    // Returns a default Puffer instance, the file is located on Context.filesDir\n    val corePuffer = AndroidPufferDB.withDefault()\n\n    // Returns a File that should be used to create a Puffer instance\n    val pufferFile = AndroidPufferDB.getInternalFile(\"my.db\")\n    val coroutinePuffer = CoroutinePufferDB.with(pufferFile)\n}\n```\n\n## Coroutines\nThe Coroutines module contains a `CoroutinePufferDB` wrapper class and some useful extension functions:\n```kotlin\nval pufferFile = File(\"path/to/puffer/file\")\nval puffer = CoroutinePufferDB.with(pufferFile)\n\n// All methods are suspend functions that runs on Dispatchers.IO context\nlaunch {\n    puffer.apply {\n        val myValue = get\u003cString\u003e(\"myKey\")\n        val myValueWithDefault = get(\"myKey\", \"defaultValue\")\n        \n        put(\"myOtherKey\", 123)\n\n        getKeys().forEach { key -\u003e\n            // ...\n        }\n\n        if(contains(\"myKey\")){\n            // ...\n        }\n\n        remove(\"myOtherKey\")\n\n        removeAll()\n    }\n}\n```\n\nIf you don't want to use this wrapper class, there's some built in extension functions that can be used with the Core module:\n```kotlin\nval pufferFile = File(\"path/to/puffer/file\")\nval puffer = PufferDB.with(pufferFile) // \u003c- Note that we're using the Core PufferDB\n\nlaunch {\n    puffer.apply {\n        val myValue = getSuspend\u003cString\u003e(\"myKey\")\n\n        val myValue = getAsync\u003cString\u003e(\"myKey\").await()\n        \n        // You can use your own coroutine scope and dispatcher\n        putSuspend(\"myOtherKey\", 123, myCoroutineScope, myCoroutineDispatcher)\n\n        putAsync(\"myOtherKey\", 123, myActivityScope).await()\n    }\n}\n```\n\n## RxJava\nThe RxJava module contains a `RxPufferDB` wrapper class and some useful extension functions:\n```kotlin\nval pufferFile = File(\"path/to/puffer/file\")\nval puffer = RxPufferDB.with(pufferFile)\n\npuffer.apply {\n    // Some methods returns Single\u003cT\u003e...\n    get\u003cString\u003e(\"myKey\") // OR get(\"myKey\", \"defaultValue\")\n        .subscribeOn(Schedulers.io())\n        .observeOn(AndroidSchedulers.mainThread())\n        .subscribe { myValue -\u003e\n            // ...\n        }\n\n    // ... And others returns Completable\n    put(\"myOtherKey\", 123)\n        // ...\n        .subscribe {\n            // ...\n        }\n\n    getKeys()\n        // ...\n        .subscribe { keys -\u003e\n            // ...\n        }\n\n    contains(\"myKey\")\n        // ...\n        .subscribe { contains -\u003e\n            // ...\n        }\n\n    remove(\"myOtherKey\")\n        // ...\n        .subscribe {\n            // ...\n        }\n\n    removeAll()\n        // ...\n        .subscribe {\n            // ...\n        }\n}\n```\n\nLike the Coroutines module, the RxJava module also provides some useful built in extension functions that can be used with the Core module:\n```kotlin\nval pufferFile = File(\"path/to/puffer/file\")\nval puffer = PufferDB.with(pufferFile) // \u003c- Note that we're using the Core PufferDB\n\npuffer.apply {\n    val myValue = getSingle\u003cString\u003e(\"myKey\").blockingGet()\n\n    putCompletable(\"myOtherKey\", 123).blockingAwait()\n\n    getKeysObservable().blockingSubscribe { keys -\u003e\n        // ...\n    }\n}\n```\n\n## Benchmark\n\n### Write \u0026 Read\n|  | Write 1k strings (ms) | Read 1k strings (ms) |\n|-------------------|-----------------------|----------------------|\n| **PufferDB** | **20** | **5** |\n| [SharedPreferences](https://developer.android.com/training/data-storage/shared-preferences) | 278 | 7 |\n| [MMKV](https://github.com/Tencent/MMKV) | 13 | 8 |\n| [Paper](https://github.com/pilgr/Paper) | 818 | 169 |\n| [Binary Prefs](https://github.com/yandextaxitech/binaryprefs) | 121 | 9 |\n| [Hawk](https://github.com/orhanobut/hawk) | 15183 | 207 |\n\n|  | Write 100k strings (ms) | Read 100k strings (ms) |\n|-------------------|-----------------------|----------------------|\n| **PufferDB** | **259** | **32** |\n| [SharedPreferences](https://developer.android.com/training/data-storage/shared-preferences) | 💥 | 💥 |\n| [MMKV](https://github.com/Tencent/MMKV) | 871 | 516 |\n| [Paper](https://github.com/pilgr/Paper) | 💥 | 💥 |\n| [Binary Prefs](https://github.com/yandextaxitech/binaryprefs) | 1082 | 101 |\n| [Hawk](https://github.com/orhanobut/hawk) | 💥 | 💥 |\n\n### File size\n|  | 1k strings (kb) |\n|-------------------|-----------------------|\n| **PufferDB** | **25** |\n| [SharedPreferences](https://developer.android.com/training/data-storage/shared-preferences) | 20 |\n| [MMKV](https://github.com/Tencent/MMKV) | 40 |\n| [Paper](https://github.com/pilgr/Paper) | 61 |\n| [Binary Prefs](https://github.com/yandextaxitech/binaryprefs) | 53 |\n| [Hawk](https://github.com/orhanobut/hawk) | 27 |\n\n|  | 100k strings (kb) |\n|-------------------|-----------------------|\n| **PufferDB** | **2.907** |\n| [SharedPreferences](https://developer.android.com/training/data-storage/shared-preferences) | 💥 |\n| [MMKV](https://github.com/Tencent/MMKV) | 4.104 |\n| [Paper](https://github.com/pilgr/Paper) | 💥 |\n| [Binary Prefs](https://github.com/yandextaxitech/binaryprefs) | 5.175 |\n| [Hawk](https://github.com/orhanobut/hawk) | 💥 |\n\n*Tested on Moto Z2 Plus*\n\nYou can run the [Benchmark](https://github.com/adrielcafe/PufferDB/blob/master/sample/src/main/kotlin/cafe/adriel/pufferdb/sample/Benchmark.kt) through the sample app.","funding_links":["https://ko-fi.com/adrielcafe"],"categories":["数据库","Kotlin"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadrielcafe%2Fpufferdb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fadrielcafe%2Fpufferdb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadrielcafe%2Fpufferdb/lists"}