{"id":43988366,"url":"https://github.com/gasparian/pure-kv","last_synced_at":"2026-02-07T10:10:23.520Z","repository":{"id":57575946,"uuid":"347127713","full_name":"gasparian/pure-kv","owner":"gasparian","description":"Simple embedded in-memory key-value storage with ttl and RPC interface","archived":false,"fork":false,"pushed_at":"2022-09-12T14:24:55.000Z","size":170,"stargazers_count":4,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-21T20:00:18.923Z","etag":null,"topics":["bolt-database","concurrency","concurrent-map","database","go","go-datastore","go-key-value","go-rpc","golang","in-memory-database","key-value","key-value-database","key-value-store","pure-go","redis","rpc","rpc-client"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gasparian.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-03-12T16:24:35.000Z","updated_at":"2024-02-24T11:41:41.000Z","dependencies_parsed_at":"2022-08-28T18:01:22.319Z","dependency_job_id":null,"html_url":"https://github.com/gasparian/pure-kv","commit_stats":null,"previous_names":["gasparian/pure-kv-go"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/gasparian/pure-kv","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gasparian%2Fpure-kv","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gasparian%2Fpure-kv/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gasparian%2Fpure-kv/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gasparian%2Fpure-kv/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gasparian","download_url":"https://codeload.github.com/gasparian/pure-kv/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gasparian%2Fpure-kv/sbom","scorecard":{"id":419462,"data":{"date":"2025-08-11","repo":{"name":"github.com/gasparian/pure-kv","commit":"63beb8fdb8222c87f6f3ae561395a13c3367b4e6"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.4,"checks":[{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/test.yml:8: update your workflow using https://app.stepsecurity.io/secureworkflow/gasparian/pure-kv/test.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/test.yml:10: update your workflow using https://app.stepsecurity.io/secureworkflow/gasparian/pure-kv/test.yml/main?enable=pin","Info:   0 out of   2 GitHub-owned GitHubAction dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/test.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Code-Review","score":0,"reason":"Found 0/28 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'main'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 4 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-19T00:48:11.756Z","repository_id":57575946,"created_at":"2025-08-19T00:48:11.756Z","updated_at":"2025-08-19T00:48:11.756Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29192112,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-07T07:37:03.739Z","status":"ssl_error","status_checked_at":"2026-02-07T07:37:03.029Z","response_time":63,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bolt-database","concurrency","concurrent-map","database","go","go-datastore","go-key-value","go-rpc","golang","in-memory-database","key-value","key-value-database","key-value-store","pure-go","redis","rpc","rpc-client"],"created_at":"2026-02-07T10:10:21.009Z","updated_at":"2026-02-07T10:10:23.515Z","avatar_url":"https://github.com/gasparian.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"![main tests](https://github.com/gasparian/pure-kv/actions/workflows/test.yml/badge.svg?branch=main)\n\n# pure-kv  \nEasy-to-use embedded in-memory key-value storage with ttl and RPC interface.  \n\n\u003cp align=\"center\"\u003e \u003cimg src=\"./pics/logo.jpg\" height=300/\u003e \u003c/p\u003e  \n\nFeatures:  \n * doesn't depend on any third-party library;  \n * uses concurrent hash-map inside;  \n * time-to-live could be set to keys;  \n * could be used both as an embedded storage and as a remote RPC server (RPC client provided);  \n * could be dumped to disk and restored back with the full state;  \n\n## Install  \n```\ngo get github.com/gasparian/pure-kv\n```  \n\n## Run server  \n```\nmake\n./purekv --port 6666 \\\n         --shards 32 \\\n         --ttl_garbage_timeout 60000 \\\n```  \n\n## Usage  \nThe only thing you need to know about `pure-kv` - you can perform just set/get/del with time-to-live and keep byte arrays *only*. So you can serialize your data with `gob`, for example, and put the result in the store.  \n\n### Embedded  \nYou can use [store](https://github.com/gasparian/pure-kv/blob/main/pkg/purekv/store.go) directly in your go code.  \nHere is an example:  \n```go\n\nimport (\n    \"log\"\n    \"time\"\n    \"github.com/gasparian/pure-kv/pkg/purekv\"\n)\n\n// Create new store\nstore := purekv.NewStore(\n    32,    // number of shards for concurrent map\n    20000, // TTL garbage collector timeout in ms.\n           // setting 0 means turn off keys removal at all\n)         \ndefer store.Close() // Need to finalize background work\n// Set key providing ttl of 1 s\nerr = store.Set(\"someKey\", []byte{'a'}, 1000)\nif err != nil {\n    log.Fatal(err)\n}\n// Get current size of store\nstoreSize := store.Size()\nlog.Printf(\"Number of keys in store: %v\", storeSize)\n// Get value that has been set before\nval, err := Get(\"someKey\")\nif err != nil {\n    log.Fatal(err)\n}\nlog.Printf(\"Value got from store: %v\", val)\n// Check that key became stale\ntime.Sleep(time.Duration(1) * time.Second)\nok = store.Has(\"someKey\")\nif ok {\n    log.Fatal(\"Key should be already cleaned\")\n}\n// Setting a key with 0 ttl means that it \n// will live until you delete it manually\nstore.Set(\"anotherKey\", []byte{'a'}, 0)\nstore.Del(\"anotherKey\")\n// Check that it's not presented\nok = store.Has(\"anotherKey\")\nif ok {\n    log.Fatal(\"Key `anotherKey` should not be presented\")\n}\n// Dump store to disk\npath := \"/path/to/db/dump\"\nerr = store.Dump(path)\nif err != nil {\n    log.Fatal(err)\n}\n// Load store back\nstoreLoaded, err := purekv.Load(path)\ndefer storeLoaded.Close()\nif err != nil {\n    log.Fatal(err)\n}\nlog.Printf(\"Store loaded successfully, size: %v\", storeLoaded.Size())\n// Drop all entries in store and check it\nerr = store.DelAll()\nif err != nil {\n    log.Fatal(err)\n}\nstoreSize = storeLoaded.Size()\nif storeSize != 0 {\n    log.Fatal(\"Store should not contain entries after deletion\")\n}\n```  \n\n### RPC  \n\nRun [server:](https://github.com/gasparian/pure-kv/blob/main/internal/server/server.go)  \n```go\npackage main\n\nimport (\n    pkv \"github.com/gasparian/pure-kv/internal/server\"\n)\n\nfunc main() {\n    flag.Parse()\n    srv := pkv.InitServer(\n        6666,  // port\n        32,    // number of shards for concurrent map\n        60000, // TTL garbage collector timeout in ms.\n    )\n    srv.Start() // \u003c-- blocks\n}\n```  \n\n[Client](https://github.com/gasparian/pure-kv/blob/main/internal/client/client.go) usage example:  \n\n```go\npackage main\n\nimport (\n    \"log\"\n    pkv \"github.com/gasparian/pure-kv/internal/client\"\n)\n\nfunc main() {\n    // creates client instance by providing server address and timetout in ms. \n    cli := pkv.New(\"0.0.0.0:6668\", 500)\n    err := cli.Open()\n    defer cli.Close() \n    if err != nil {\n        log.Fatal(err)\n    }\n    // Creates new key-value pair with ttl of 1 sec.\n    err = cli.Set(\"someKey\", []byte{'a'}, 1000) \n    if err != nil {\n        log.Fatal(err)\n    }\n    // Returns set value\n    tmpVal, ok := cli.Get(\"someKey\") \n    if !ok {\n        log.Fatal(\"Can't get a value\")\n    }    \n    log.Println(val)\n    // Returns number of elements in store\n    size := cli.Size() \n    log.Println(size)\n    // Async. delete value from the bucket\n    err = cli.Del(\"someKey\")\n    if err != nil {\n        log.Fatal(err)\n    }\n    // Recreates the entire db, deleting everything\n    cli.DelAll() \n}\n```  \n\n### Tests  \n\nYou can run tests by package or left argument empty to run all the tests:  \n```\nmake test\n```  \n\nRun `store` benchmark:  \n```\nmake benchmark\n```  \nThe main idea of this benchmark is to see how concurrent access and the number of shards in store affects it's performance.  \n\n#### Benchmark results  \nI ran benchmark test on my laptop with the following configuration:  \n```\ngoos: darwin\ngoarch: amd64\ncpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz\ngomaxprocs: 12\n```  \nBelow you can find graphs for simulation of 100 serial/concurrent sets/gets.  \nFirst case is when `ttl` has been **turned off**:  \n\u003cp align=\"center\"\u003e \u003cimg src=\"./pics/100_no_ttl.svg\" width=600/\u003e \u003c/p\u003e  \n\nSecond case - when we use `ttl`:  \n\u003cp align=\"center\"\u003e \u003cimg src=\"./pics/100_ttl.svg\" width=600/\u003e \u003c/p\u003e  \n\nIn both cases you can see an improvement when using concurrent access, but the effect of adding more shards in our concurrent map quickly disappears, after ~4-8 shards.  \nAs you can see, the largest improvement belongs to concurrent sets: \u003e30% for experiment without keys ttl and \u003e200% for the experiment with ttl (max. 32 shards).  \nBut at the same time it looks like \"gets\" are more \"stable\", and just a single shard (which is effectively just a map with mutex lock) already does the job.  \nThe methodology of an experiment is simple and far from ideal, of course. Possibly we could see more improvement on a larger-scale experiments.  \n\n### Things to improve  \nNow, keys with expiry times are stored in a min heap, and this heap is being update with new entries through channels, which are unique for every shard. And on store start-up some amount of goroutines are spawned (number is equal to the number of shards), which may be not the best solution (see the code [here](https://github.com/gasparian/pure-kv/blob/main/pkg/purekv/store.go#L139)). On the other hand - these goroutines by default just waits for the new entry in the channel, which comes from the next `Set` operation, in case ttl has been set, but it could a problem when there are a lot of sets and these goroutines could take more and more CPU time.  \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgasparian%2Fpure-kv","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgasparian%2Fpure-kv","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgasparian%2Fpure-kv/lists"}