{"id":34139498,"url":"https://github.com/nedscode/memdb","last_synced_at":"2026-03-11T02:02:09.252Z","repository":{"id":46261569,"uuid":"97057413","full_name":"nedscode/memdb","owner":"nedscode","description":"The memdb library is an in-memory indexable store for go structs. Like a database, but in memory.","archived":false,"fork":false,"pushed_at":"2025-03-25T03:27:22.000Z","size":102,"stargazers_count":20,"open_issues_count":2,"forks_count":5,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-01-12T02:47:12.818Z","etag":null,"topics":["database","go","golang","memory"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"lgpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nedscode.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-07-12T22:34:05.000Z","updated_at":"2025-03-25T03:27:26.000Z","dependencies_parsed_at":"2022-08-27T19:22:04.620Z","dependency_job_id":null,"html_url":"https://github.com/nedscode/memdb","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/nedscode/memdb","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nedscode%2Fmemdb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nedscode%2Fmemdb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nedscode%2Fmemdb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nedscode%2Fmemdb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nedscode","download_url":"https://codeload.github.com/nedscode/memdb/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nedscode%2Fmemdb/sbom","scorecard":{"id":678374,"data":{"date":"2025-08-11","repo":{"name":"github.com/nedscode/memdb","commit":"a6fbc11fc823c7dda0442f831141399de77d8aac"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.7,"checks":[{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Code-Review","score":5,"reason":"Found 6/11 approved changesets -- score normalized to 5","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: GNU Lesser General Public License v3.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 27 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-21T22:16:37.870Z","repository_id":46261569,"created_at":"2025-08-21T22:16:37.870Z","updated_at":"2025-08-21T22:16:37.870Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30367799,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-10T21:41:54.280Z","status":"online","status_checked_at":"2026-03-11T02:00:07.027Z","response_time":84,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","go","golang","memory"],"created_at":"2025-12-15T02:22:46.222Z","updated_at":"2026-03-11T02:02:09.208Z","avatar_url":"https://github.com/nedscode.png","language":"Go","readme":"# memdb\n\nThe memdb library is a simple, light-weight, in-memory store for go structs that allows indexing and storage of items as well as configurable item expiry and collection at an interface level.\n\nPersistence to disk is optionally available in the [persistfile](persist/file) package, should it be required, but has an overhead of having to encode/save the items when inserted/updated and to load the items at creation time.\n\nImportantly this memdb library only stores pointers to your original struct, so all benefits and caveats of this fact apply. For example, your original struct remains mutable which can be useful (see caveats below). There is a very low overhead for storing items above that of creating the original struct.\n\nIf you're looking for a more full featured in-memory database with immutable storage and snapshots/transactions, you may be interested in looking into [Hashicorp's go-memdb](https://github.com/hashicorp/go-memdb) instead, it does have additional overheads, but less of the caveats.\n\n[![Build Status](https://travis-ci.org/nedscode/memdb.svg?branch=master)](https://travis-ci.org/nedscode/memdb)\n[![Go Report Card](https://goreportcard.com/badge/github.com/nedscode/memdb)](https://goreportcard.com/report/github.com/nedscode/memdb)\n[![Documentation](https://godoc.org/github.com/nedscode/memdb?status.svg)](http://godoc.org/github.com/nedscode/memdb)\n[![Coverage Status](https://coveralls.io/repos/github/nedscode/memdb/badge.svg?branch=master)](https://coveralls.io/github/nedscode/memdb?branch=master)\n[![GitHub issues](https://img.shields.io/github/issues/nedscode/memdb.svg)](https://github.com/nedscode/memdb/issues)\n\n[![license](https://img.shields.io/github/license/nedscode/memdb.svg?maxAge=2592000)](https://github.com/nedscode/memdb/LICENSE)\n\n## Important caveats\n\nAs you are working with in-memory objects, it can be easy to overlook that you're also indexing these items in a\ndatabase.\n\nJust like a real database, if you update an item such that it's index keys would change, you must Put it back in to\nupdate the items indexes in the database, and also to cause update notifications to be sent (and be persisted to disk if you're using this function).\n\nDO NOT under any circumstances update the PRIMARY KEYs (ie keys used to determine the output of the Less()\ncomparator) without first removing the existing item. Such an act would leave the item stranded in an unknown\nlocation within the index.\n\n## Including\n\nTo start using, get memdb `go get github.com/nedscode/memdb` and include it in your code:\n\n```golang\nimport \"github.com/nedscode/memdb\"\n```\n\n## Defining you storage struct\n\nThere are 2 ways to use use memdb, the first (and recommended way) is to rely on automatic field detection to index your objects.\n\n### Relying on reflection.\n\nYou can rely on reflection to implement your code more easily for most use-cases. This is now the recommended method of using memdb. See the next section \"Implementing Indexable\" for the manual way.\n\nDefine your struct:\n\n```golang\ntype car struct {\n    Make    string\n    Model   string\n    RRP     int\n}\n```\n\nTo create a storage instance initialise it and set the indexed fields.\n\nIndexed fields can only be set at the start before data gets stored. Attempt to set index fields after first use will cause a panic.\n\n```golang\n    mdb := memdb.NewStore().\n        PrimaryKey(\"make\", \"model\")\n```\n\n### Implementing Indexable. (alternative, older method)\n\nThis is the older and manual way of implementing storage and indexing of an item.\n\nBefore automatic field discovery was implemented, you needed to add extra methods to your objects so that the fields could be found and equality/sorting could be determined.\n \nWe no longer recommend you use this method unless you really have to as it has extra implementation overheads and makes changes harder than simply adding new a field.\n\nThis method may still be of use if you need to implement custom sorting/equality operations (perhaps using external lookup tables etc).\n\nDefine your struct as normal:\n\n```golang\ntype car struct {\n    Make    string\n    Model   string\n    RRP     int\n}\n```\n\nThen add the required methods to support storage in memdb as an Indexable interfaced object.\n\nWe need a comparator function `Less`. If `a.Less(b) == false \u0026\u0026 b.Less(a) == false`, then the item is determined to be\nequivalent and the same. Storage of multiple equivalent items will overwrite each other. If you can't figure out the\ncomparison yourself (eg unknown object type), call the Unsure function which will arbitrarily, but consistently\ndetermine the order. \n\n```golang\nfunc (i *car) Less(other memdb.Indexer) bool {\n    switch o := other.(type) {\n    case *car:\n        if i.Make \u003c o.Make {\n            return true\n        }\n        if i.Make \u003e o.Make {\n            return false\n        }\n        if i.Model \u003c o.Model {\n            return true\n        }\n        return false\n    }\n    return memdb.Unsure(i, other)\n}\n```\n\nFinally, we need a function that will return the string values of the indexed fields, all indexed fields are returned\nand stored as strings:\n\n```golang\nfunc (i *car) GetField(field string) string {\n    switch field {\n    case \"make\":\n        return i.Make\n    case \"model\":\n        return i.Model\n    default:\n        return \"\" // Indicates should not be indexed\n    }\n}\n```\n\nTo create a storage instance initialise it and set the indexed fields.\n\nIndexed fields can only be set at the start before data gets stored. Attempt to set index fields after use will cause a\npanic.\n\n```golang\n    mdb := memdb.NewStore()\n```\n\n### Adding indexes\n\nYou can add more ordinary indexes for the fields you want to search on.\n\n```golang\n    mdb.\n        CreateIndex(\"make\").\n        CreateIndex(\"model\")\n```\n\n### Compound indexes\n\nYou can also create compound indexes by supplying multiple fields:\n\n```golang\n    mdb.CreateIndex(\"make\", \"model\", \"rrp\")\n```\n\n### Unique indexes\n\nYou can create unique indexes by appending a `Unique()` to the definition.\n\nPutting an item with the same value as a unique index will cause the previous item to be \"Updated\".\n\n```golang\n    type car struct {\n        Make    string\n        Model   string\n        RRP     int\n        Vin     string\n    }\n\n    mdb.CreateIndex(\"vin\").Unique()\n```\n\n### Chaining it all together\n\nAll of the index creation can be chained together in the creation line, for example:\n\n```golang\n    mdb := memdb.NewStore().\n        PrimaryKey(\"make\", \"model\").\n        CreateIndex(\"make\").\n        CreateIndex(\"model\").\n        CreateIndex(\"vin\").Unique().\n        CreateIndex(\"make\", \"model\", \"rrp\")\n```\n\n### Adding items to the store.\n\nSaving items into the store is simple:\n\n```golang\n    mdb.Put(\u0026car{Make: \"Ford\", Model: \"Fiesta\", RRP: 27490})\n    mdb.Put(\u0026car{Make: \"Holden\", Model: \"Astra\", RRP: 24190})\n    mdb.Put(\u0026car{Make: \"Honda\", Model: \"Jazz\", RRP: 19790})\n```\n\n## Retrieving an item\n\nIn order to retrieve an item, you can either search in an index\n\n```golang\n    found := mdb.InPrimaryKey().One(\"Holden\", \"Astra\")\n    // OR\n    found := mdb.In(\"make\", \"model\").One(\"Holden\", \"Astra\")\n```\n\nOR supply an equivalent item as a search parameter to the `Get` method.\nYou only need to create enough fields in the search object to be deemed equivalent by your Less function:\n\n```golang\n    found := mdb.Get(\u0026car{Make: \"Holden\", Model: \"Astra\"})\n```\n\nOnce you have performed the search, you can cast it as a car and use it.\nThe cast will fail if the returned object is nil, or not a car.\n\n```golang\n    if vehicle, ok := found.(*car); ok {\n        fmt.Printf(\"Vehicle RRP is $%d\\n\", vehicle.RRP)\n    }\n```\n\n## Looking up items by indexed field\n\nThis is where it starts to get interesting, we can lookup items by any of our defined indexed fields:\n\n```golang\n    indexers := mdb.In(\"model\").Lookup(\"Astra\")\n    for _, indexer := range indexers {\n        vehicle := indexer.(*car)\n        fmt.Printf(\"%s %s ($%d rrp)\\n\", vehicle.Make, vehicle.Model, vehicle.RRP)\n    }\n```\n\nIf you have compound fields, you can search them like:\n\n```golang\n    indexers := mdb.In(\"make\", \"model\").Lookup(\"Holden\", \"Astra\")\n```\n\n## Index pathing\n\nIf you're using the simple method (with automatic fields), you can also index subfields with very little effort:\n\n```golang\n   type car struct {\n        Make    string\n        Model   string\n        Vin     string\n        Details map[string]string\n   }\n\n   mdb := memdb.NewStore()\n     .PrimaryKey(\"vin\")\n     .Index(\"make\", \"model\")\n     .Index(\"details.colour\")\n     .Index(\"details.style\")\n\n   mdb.Put(\u0026car{\n       Make: \"Honda\",\n       Model: \"Jazz\",\n       VIN:   \"abc123\",\n       Details: map[string]string{\n           \"color\": \"Metallic Blue\",\n           \"style\": \"Hatchback\",\n       }\n   })\n\n   // Now you can find all the hatchbacks you have in stock.\n   indexers := mdb.In(\"details.style\").Lookup(\"Hatchback\")\n```\n\n## Traversing the database\n\nIf you desire to walk the database, you can ascend or descend from the extremities or a certain point using one of the \nfollowing functions:\n\n * Ascend(iterator)\n * Descend(iterator)\n * AscendStarting(at, iterator)\n * DescendStarting(at, iterator)\n\nUse the functions by providing an iterator that returns false to stop traversal as follows:\n\n```golang\n    fmt.Println(\"Iterating over all cars, ascending:\\n\")\n    count := 0\n    mdb.Ascend(func(indexer memdb.Indexer) bool {\n        vehicle := indexer.(*car)\n        fmt.Printf(\"%s %s ($%d rrp)\\n\", vehicle.Make, vehicle.Model, vehicle.RRP)\n        count++\n        return true\n    })\n    fmt.Println(\"Found %d cars\\n\", count)\n```\n\nIf you wish to traverse your simple or compound indexed fields, you may also do this via:\n\n```golang\n    mdb.In(\"make\", \"model\").Each(func(indexer memdb.Indexer) bool {\n        vehicle := indexer.(*car)\n        fmt.Printf(\"%s %s ($%d rrp)\\n\", vehicle.Make, vehicle.Model, vehicle.RRP)\n    }, \"Holden\", \"Astra\")\n```\n\n## Notification\n\nItem notification can be performed via the On(event, callback) method:\n\n```golang\n    notify := func (event memdb.Event, old, new memdb.Indexer) {\n        fmt.Printf(\"Got %#v of %#v -\u003e %#v\", event, old, new)\n    }\n    \n    mdb.On(memdb.Insert, notify)\n    mdb.On(memdb.Update, notify)\n    mdb.On(memdb.Remove, notify)\n    mdb.On(memdb.Expiry, notify)\n```\n\n## Removal\n\nItems can be removed directly by calling the Delete function\n\n```golang\n    mdb.Delete(\u0026car{Make: \"Holden\", Model: \"Astra\"})\n```\n\n## Expiry\n\nItem expiry can be achieved by defining an expiry condition function and scheduling the expiry function.\n\nSay we expanded the car struct to have a sold time\n\n```golang\ntype car struct {\n    Make    string\n    Model   string\n    RRP     int\n    Sold    time.Time\n}\n```\n\nThen changed the IsExpired method like:\n\n```golang\nfunc (i *car) IsExpired() bool {\n    return i.Sold.Before(time.Now().Sub(24 * time.Hour))\n}\n```\n\nThen scheduled the Expire function:\n\n```golang\ngo func() {\n    tick := time.Tick(30 * time.Minute)\n    for range tick {\n        mdb.Expire()\n    }\n}\n```\n\nNow every 30 minutes, we will expire cars sold more than 24 hours ago from our listings.\n\n## Persistence\n\nSometimes you want to have your cake and eat it too. While this is specifically an in-memory\ndatabase, we also support an optional store-on-put and load-at-start style persistence model.\n\nThis is achieved by adding a Persister to the store after adding indexes and before beginning\nto use it.\n\nThere is currently a simple file-based Persister that you can use for simple use-cases, and as\na platform for developing your own more complicated solutions.\n\nThis is an example of using the built-in file Persister:\n\n```golang\nfunc indexerFactory(indexerType string) interface{} {\n    if (indexerType == \"*main.car\") {\n        return \u0026car{}\n    }\n    return nil\n}\n\n// …\n\n    p := filepersist.NewFileStorage(\"/tmp/mydata\", indexerFactory)\n    mdb := memdb.NewStore().\n        CreateIndex(\"make\").\n        CreateIndex(\"model\").\n        Persistent(p)\n```\n\n## License\n\n© 2017-2019, Neds International, code is released under GNU LGPL v3.0, see [LICENSE](LICENSE) file.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnedscode%2Fmemdb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnedscode%2Fmemdb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnedscode%2Fmemdb/lists"}