{"id":36578691,"url":"https://github.com/starskey-io/starskey","last_synced_at":"2026-01-12T07:40:13.673Z","repository":{"id":273309951,"uuid":"919276446","full_name":"starskey-io/starskey","owner":"starskey-io","description":"Package for fast persistent, transactional, and embedded key-value storage. LevelDB-WiscKey inspired.","archived":false,"fork":false,"pushed_at":"2025-03-19T21:09:28.000Z","size":665,"stargazers_count":122,"open_issues_count":0,"forks_count":11,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-19T22:22:02.530Z","etag":null,"topics":["database","key-value-store","lsm-tree","oltp","storage-engine"],"latest_commit_sha":null,"homepage":"https://starskey.io","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/starskey-io.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-20T05:08:00.000Z","updated_at":"2025-03-19T21:09:32.000Z","dependencies_parsed_at":null,"dependency_job_id":"2bad344e-c988-4e00-85a7-fa1602e23ac8","html_url":"https://github.com/starskey-io/starskey","commit_stats":null,"previous_names":["starskey-io/starskey"],"tags_count":19,"template":false,"template_full_name":null,"purl":"pkg:github/starskey-io/starskey","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/starskey-io%2Fstarskey","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/starskey-io%2Fstarskey/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/starskey-io%2Fstarskey/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/starskey-io%2Fstarskey/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/starskey-io","download_url":"https://codeload.github.com/starskey-io/starskey/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/starskey-io%2Fstarskey/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28336626,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-12T06:09:07.588Z","status":"ssl_error","status_checked_at":"2026-01-12T06:05:18.301Z","response_time":98,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","key-value-store","lsm-tree","oltp","storage-engine"],"created_at":"2026-01-12T07:40:12.986Z","updated_at":"2026-01-12T07:40:13.668Z","avatar_url":"https://github.com/starskey-io.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv\u003e\n    \u003ch1 align=\"left\"\u003e\u003cimg width=\"256\" src=\"artwork/starskey_logo.png\"\u003e\u003c/h1\u003e\n\u003c/div\u003e\n\n[![Go Reference](https://pkg.go.dev/badge/github.com/starskey-io/starskey.svg)](https://pkg.go.dev/github.com/starskey-io/starskey)\n\nStarskey is a fast embedded key-value store package for GO!  Starskey implements a multi-level, durable log structured merge tree.\n\n## Features\n- **Leveled partial merge compaction**  Compactions occur on writes. If any disk level reaches its maximum size, half of the sstables are merged into a new sstable and placed into the next level. This algorithm is recursive until the last level. At the last level, if full, we merge all sstables into a new sstable. During merge operations, tombstones (deleted keys) are removed when a key reaches the last level.\n- **Simple API** with Put, Get, Delete, Range, FilterKeys, Update (for txns), PrefixSearch, LongestPrefixSearch, DeleteByRange, DeleteByFilter, DeleteByPrefix.\n- **Acid transactions** You can group multiple operations into a single atomic transaction.  If any operation fails the entire transaction is rolled back.  Only committed operations within a transaction roll back.  These transactions would be considered fully serializable.  Transactions themselves are also thread safe so you can add operations to them safety.\n- **Configurable options** You can configure many options such as max levels, memtable threshold, bloom filter,succinct range filters, logging, compression and more.\n- **WAL with recovery** Starskey uses a write ahead log to ensure durability.  Memtable is replayed if a flush did not occur prior to shut down.  On sorted runs to disk the WAL is truncated.\n- **Key value separation** Keys and values are stored separately for sstables within a klog and vlog respectively.\n- **Bloom filters** Each sstable has an in memory bloom filter to reduce disk reads. Bloom filters are used to check if a key exists in an SST instead of scanning it entirely.\n- **Succinct Range Filters** If enabled, each sstable will use a SuRF instead of a bloom filter;  This will speed up range, prefix queries. Will use more memory than bloom filters. Only a bloom filter OR a SuRF filter can be enabled.\n- **Fast** up to 400k+ ops per second.\n- **Compression** S2, and Snappy compression is available.\n- **Logging** Logging to file is available.  Will write to standard out if not enabled.\n- **Thread safe** Starskey is thread safe.  Multiple goroutines can read and write to Starskey concurrently.  Starskey uses one global lock to keep things consistent.\n- **T-Tree memtable** the memory table is a balanced in-memory tree data structure, designed as an alternative to AVL trees and B-Trees for main-memory.\n- **Channel logging** You can log to a channel instead of a file or standard output.\n\n## Discord\nChat everything Starskey @ our [Discord Server](https://discord.gg/HVxkhyys3R)\n\n## Bench\nUse the benchmark program at [bench](https://github.com/starskey-io/bench) to compare Starskey with other popular key value stores/engines.\n\n## Basic Example\nBelow is a basic example of how to use Starskey.\n\n**Mind you examples use `skey` as the variable name for opened starskey instance(s).**\n\n```go\nimport (\n    \"fmt\"\n    \"github.com/starskey-io/starskey\"\n)\n\nfunc main() {\n    skey, err := starskey.Open(\u0026starskey.Config{\n        Permission:        0755,                   // Dir, file permission\n        Directory:         \"db_dir\",               // Directory to store data\n        FlushThreshold:    (1024 * 1024) * 24,     // 24mb Flush threshold in bytes, for production use 64mb or higher\n        MaxLevel:          3,                      // Max levels number of disk levels\n        SizeFactor:        10,                     // Size factor for each level.  Say 10 that's 10 * the FlushThreshold at each level. So level 1 is 10MB, level 2 is 100MB, level 3 is 1GB.\n        BloomFilter:       false,                  // If you want to use bloom filters\n        SuRF:              false,                  // If enabled will speed up range queries as we check if an sstable has the keys we are looking for.\n        Logging:           true,                   // Enable logging to file\n        Compression:       false,                  // Enable compression\n        CompressionOption: starskey.NoCompression, // Or SnappyCompression, S2Compression\n\n        // Internal options\n        // Optional: \u0026OptionalConfig{\n        //    BackgroundFSync:         .. If you don't want to fsync writes to disk (default is true)\n        //    BackgroundFSyncInterval: .. Interval for background fsync, if configured true (default is 256ms)\n        //    TTreeMin:                .. Minimum degree of the T-Tree\n        //    TTreeMax:                .. Maximum degree of the T-Tree\n        //    PageSize:                .. Page size for internal pagers\n        //    BloomFilterProbability:  .. Bloom filter probability\n        //    LogChannel              chan string   // Log channel, instead of log file or standard output we log to a channel\n        //    },\n        }) // Config cannot be nil**\n    if err != nil {\n        // ..handle error\n    }\n\n    key := []byte(\"some_key\")\n    value := []byte(\"some_value\")\n\n    // Write key-value pair\n    if err := skey.Put(key, value); err != nil {\n        // ..handle error\n    }\n\n    // Read key-value pair\n    v, err := skey.Get(key)\n    if err != nil {\n        // ..handle error\n    }\n\n    // Value is nil if key does not exist\n    if v == nil {\n        // ..handle error\n    }\n\n    fmt.Println(string(key), string(v))\n\n    // Close starskey\n    if err := skey.Close(); err != nil {\n        // ..handle error\n    }\n\n}\n\n```\n\n## Range Keys\nYou can provide a start and end key to retrieve a range of keys values.\n```go\nresults, err := skey.Range([]byte(\"key900\"), []byte(\"key980\"))\nif err != nil {\n    // ..handle error\n}\n\n// results is [][]byte where each element is a value\nfor _, value := range results {\n    fmt.Println(string(value))  // Each result is just the value\n}\n```\n\n## Filter Keys\nYou can filter keys to retrieve values based on a filter/compare method.\n```go\ncompareFunc := func(key []byte) bool {\n    // if has prefix \"c\" return true\n    return bytes.HasPrefix(key, []byte(\"c\"))\n}\n\nresults, err := skey.FilterKeys(compareFunc)\nif err != nil {\n    // ..handle error\n}\n```\n\n## Prefix Searches\nStarskey supports optimized prefix searches.\n\n### Longest Prefix Search\nYou can search for longest key with a common prefix.\n```go\nresult, n, err := skey.LongestPrefixSearch([]byte(\"common_prefix\"))\nif err != nil {\n    // ..handle error\n}\n```\n\n### Prefix Search\nYou can search for keys value's with a common prefix.\n```go\nresults, err := skey.PrefixSearch([]byte(\"ke\"))\nif err != nil {\n    // ..handle error\n}\n```\n\n## Acid Transactions\nUsing atomic transactions to group multiple operations into a single atomic transaction.  If any operation fails the entire transaction is rolled back.  Only committed operations roll back.\n```go\ntxn := skey.BeginTxn()\nif txn == nil {\n    // ..handle error\n}\n\ntxn.Put([]byte(\"key\"), []byte(\"value\"))\n// or txn.Delete([]byte(\"key\"))\n\nif err := txn.Commit(); err != nil {\n    // ..handle error\n}\n```\n\n**OR**\n\nYou should use `Update` if you want to say Get a value, modify it and then Put it back as an atomic operation.\nSay you want to increment a value, below is an example of how to do that.\n```go\nerr = skey.Update(func(txn *starskey.Txn) error {\n    value, err := txn.Get([]byte(\"key\"))\n    if err != nil {\n        return err\n    }\n\n    // Increment value\n    i, err := strconv.Atoi(string(value))\n    if err != nil {\n        return err\n    }\n\n    i++ // Increment value\n\n    txn.Put([]byte(\"key\"), []byte(strconv.Itoa(i))) // Put back incremented value, this update is atomic\n\n    return nil // Commit\n})\nif err != nil {\n    // ..handle error\n}\n```\n\n```go\nerr = skey.Update(func(txn *starskey.Txn) error {\n    txn.Put([]byte(\"key\"), []byte(\"value\")) // or txn.Delete, txn.Get\n    // ..\n    return nil // Commit\n})\nif err != nil {\n    // ..handle error\n}\n```\n\n## Delete\nDelete a key value pair from starskey.\n```go\nif err := skey.Delete([]byte(\"key\")); err != nil {\n    // ..handle error\n}\n```\n\n**Delete operations removing multiple key values return an `n` value which is the amount of keys deleted.**\n\n### Delete by range\nDelete key values within a range.\n```go\nif n, err := skey.DeleteByRange([]byte(\"startKey\"), []byte(\"endKey\")); err != nil {\n    // ..handle error\n}\n// n is amount of keys deleted\n```\n\n### Delete by filter\nDelete key values based on a filter/compare method.  You're comparing a key with a function and if it returns true the key is deleted.\n```go\ncompareFunc := func(key []byte) bool {\n    // if has prefix \"c\" return true\n    return bytes.HasPrefix(key, []byte(\"c\"))\n}\n\nif n, err := skey.DeleteByFilter(compareFunc); err != nil {\n    // ..handle error\n}\n// n is amount of keys deleted\n```\n\n### Delete by key prefix\nDelete key values with a common prefix.\n```go\nif n, err := skey.DeleteByPrefix([]byte(\"key\")); err != nil {\n    // ..handle error\n}\n// n is amount of keys deleted\n```\n\n## Ingesting log messages\n```go\npackage main\n\nimport (\n    \"fmt\"\n    \"github.com/starskey-io/starskey\"\n    \"sync\"\n    \"time\"\n)\n\nfunc main() {\n    // Create a buffered channel for logs\n    logChannel := make(chan string, 1000)\n\n    // Create Starskey instance with log channel configured\n    skey, err := starskey.Open(\u0026starskey.Config{\n        Permission:        0755,\n        Directory:         \"db_dir\",\n        FlushThreshold:    (1024 * 1024) * 24, // 24MB\n        MaxLevel:          3,\n        SizeFactor:        10,\n        BloomFilter:       false,\n        SuRF:              false,\n        Logging:           true,\n        Compression:       false,\n        CompressionOption: starskey.NoCompression,\n\n        // Configure the LogChannel in OptionalConfig\n        Optional: \u0026starskey.OptionalConfig{\n                LogChannel: logChannel,\n            },\n        })\n    if err != nil {\n        fmt.Printf(\"Failed to open Starskey: %v\\n\", err)\n        return\n    }\n    defer skey.Close()\n\n    // Start a goroutine to consume and process logs in real-time\n    var wg sync.WaitGroup\n\n    wg.Add(1) // Add one goroutine to wait for\n    go func() {\n        defer wg.Done()\n        for logMsg := range logChannel {\n            // Process log messages in real-time\n            timestamp := time.Now().Format(\"2006-01-02 15:04:05.000\")\n\n            fmt.Printf(\"[%s] %s\\n\", timestamp, logMsg)\n        }\n    }()\n\n    // Use Starskey for your normal operations\n    for i := 0; i \u003c 100; i++ {\n        key := []byte(fmt.Sprintf(\"key%d\", i))\n        value := []byte(fmt.Sprintf(\"value%d\", i))\n\n        if err := skey.Put(key, value); err != nil {\n            fmt.Printf(\"Failed to put key-value: %v\\n\", err)\n        }\n    }\n\n    // The log channel will keep receiving logs until Starskey is closed\n    time.Sleep(2 * time.Second) // Give some time for operations to complete\n\n    // Close starskey as we are done\n    skey.Close()\n\n    // close the channel as Starskey doesn't close it\n    close(logChannel)\n\n    // Wait for log processing to complete\n    wg.Wait()\n}\n```\n\n## Key Lifecycle\nA key once inserted will live in the memtable until it is flushed to disk.\nOnce flushed to disk it will live in a sstable at l1 until it is compacted.  Once compacted it will be merged into a new sstable at the next level.  This process is recursive until the last level.  At the last level if full we merge all sstables into a new sstable.\n\nIf a key is deleted it will live on the same way until it reaches last level at which point it will be removed entirely.\n\n## Memory and disk sorting\nThe sorting internally would be lexicographical (alphabetical), meaning it will sort based on the byte-by-byte comparisons of slices.\nWe use bytes.Compare to sort keys in memory and on disk.\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstarskey-io%2Fstarskey","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstarskey-io%2Fstarskey","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstarskey-io%2Fstarskey/lists"}