{"id":13504363,"url":"https://github.com/arriqaaq/flashdb","last_synced_at":"2025-04-05T21:09:04.130Z","repository":{"id":37734154,"uuid":"466890708","full_name":"arriqaaq/flashdb","owner":"arriqaaq","description":"FlashDB is an embeddable, in-memory key/value database in Go (with Redis like commands and super easy to read)","archived":false,"fork":false,"pushed_at":"2023-11-18T08:25:57.000Z","size":365,"stargazers_count":366,"open_issues_count":3,"forks_count":30,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-03-29T20:09:12.048Z","etag":null,"topics":["database","embedded","flashdb","go","golang","in-memory","in-memory-database","key-value","key-value-store","kv-store","redis"],"latest_commit_sha":null,"homepage":"https://aly.arriqaaq.com/building-a-database-in-go/","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/arriqaaq.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-03-07T00:19:52.000Z","updated_at":"2025-03-26T12:59:47.000Z","dependencies_parsed_at":"2024-01-13T19:20:18.347Z","dependency_job_id":"fbbe19c9-da70-42cb-92be-488277a59163","html_url":"https://github.com/arriqaaq/flashdb","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arriqaaq%2Fflashdb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arriqaaq%2Fflashdb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arriqaaq%2Fflashdb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arriqaaq%2Fflashdb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/arriqaaq","download_url":"https://codeload.github.com/arriqaaq/flashdb/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247399878,"owners_count":20932880,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","embedded","flashdb","go","golang","in-memory","in-memory-database","key-value","key-value-store","kv-store","redis"],"created_at":"2024-08-01T00:00:35.412Z","updated_at":"2025-04-05T21:09:04.104Z","avatar_url":"https://github.com/arriqaaq.png","language":"Go","funding_links":[],"categories":["Databases"],"sub_categories":["Key-Value Databases"],"readme":"\u003cp align=\"center\"\u003e\n\u003cimg\n    src=\"img/architecture.png\" alt=\"FlashDB\"\u003e\n\u003c/p\u003e\n\n# flashdb\n\nFlashDB is a simple, in-memory, key/value store in pure Go.\nIt persists to disk, is ACID compliant, and uses locking for multiple\nreaders and a single writer. It supports redis like operations for\ndata structures like SET, SORTED SET, HASH and STRING. \n\nFeatures\n========\n\n- In-memory database for [fast reads and writes](#performance)\n- Embeddable with a simple API\n- Supports Redis like operations for SET, SORTED SET, HASH and STRING\n- [Durable append-only file](#append-only-file) format for persistence\n- Option to evict old items with an [expiration](#data-expiration) TTL\n- ACID semantics with locking [transactions](#transactions) that support rollbacks\n\n\nArchitecture\n=============\n\nFlashDB is made of composable libraries that can be used independently and are easy to understand. The idea is to bridge the \nlearning for anyone new on how to build a simple ACID database.\n\n\n- [Set](https://github.com/arriqaaq/set)\n- [ZSet](https://github.com/arriqaaq/zset)\n- [String](https://github.com/arriqaaq/art)\n- [Hash](https://github.com/arriqaaq/hash)\n- [Append Only Log](https://github.com/arriqaaq/aol)\n\n\nGetting Started\n===============\n\n## Installing\n\nTo start using FlashDB, install Go and run `go get`:\n\n```sh\n$ go get -u github.com/arriqaaq/flashdb\n```\n\nThis will retrieve the library.\n\n\n## Opening a database\n\nThe primary object in FlashDB is a `DB`. To open or create your\ndatabase, use the `flashdb.New()` function:\n\n```go\npackage main\n\nimport (\n\t\"log\"\n\n\t\"github.com/arriqaaq/flashdb\"\n)\n\nfunc main() {\n\tconfig := \u0026flashdb.Config{Path:\"/tmp\", EvictionInterval: 10}\n\tdb, err := flashdb.New(config)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer db.Close()\n\n\t...\n}\n```\n\nIt's also possible to open a database that does not persist to disk by keeping the path in the config empty.\n\n```go\nconfig := \u0026flashdb.Config{Path:\"\", EvictionInterval: 10}\nflashdb.New(config)\n```\n\n## Transactions\nAll reads and writes must be performed from inside a transaction. FlashDB can have one write transaction opened at a time, but can have many concurrent read transactions. Each transaction maintains a stable view of the database. In other words, once a transaction has begun, the data for that transaction cannot be changed by other transactions.\n\nWhen a transaction fails, it will roll back, and revert all changes that occurred to the database during that transaction. When a read/write transaction succeeds all changes are persisted to disk.\n\n### Read-only Transactions\nA read-only transaction should be used when you don't need to make changes to the data. The advantage of a read-only transaction is that there can be many running concurrently.\n\n```go\nerr := db.View(func(tx *flashdb.Tx) error {\n\t...\n\treturn nil\n})\n```\n\n### Read/write Transactions\nA read/write transaction is used when you need to make changes to your data. There can only be one read/write transaction running at a time. So make sure you close it as soon as you are done with it.\n\n```go\nerr := db.Update(func(tx *flashdb.Tx) error {\n\t...\n\treturn nil\n})\n```\n\n### Setting and getting key/values\n\nTo set a value you must open a read/write transaction:\n\n```go\nerr := db.Update(func(tx *flashdb.Tx) error {\n\terr := tx.Set(\"mykey\", \"myvalue\")\n\treturn err\n})\n```\n\n\nTo get the value:\n\n```go\nerr := db.View(func(tx *flashdb.Tx) error {\n\tval, err := tx.Get(\"mykey\")\n\tif err != nil{\n\t\treturn err\n\t}\n\tfmt.Printf(\"value is %s\\n\", val)\n\treturn nil\n})\n```\n\nCommands\n========\n| String | Hash    | Set         | ZSet           |\n|--------|---------|-------------|----------------|\n| SET    | HSET    | SADD        | ZADD           |\n| GET    | HGET    | SISMEMBER   | ZSCORE         |\n| DELETE | HGETALL | SRANDMEMBER | ZCARD          |\n| EXPIRE | HDEL    | SREM        | ZRANK          |\n| TTL    | HEXISTS | SMOVE       | ZREVRANK       |\n|        | HLEN    | SCARD       | ZRANGE         |\n|        | HKEYS   | SMEMBERS    | ZREVRANGE      |\n|        | HVALS   | SUNION      | ZREM           |\n|        | HCLEAR  | SDIFF       | ZGETBYRANK     |\n|        |         | SCLEAR      | ZREVGETBYRANK  |\n|        |         |             | ZSCORERANGE    |\n|        |         |             | ZREVSCORERANGE |\n|        |         |             | ZCLEAR         |\n\nBenchmarks\n==========\n\n* Go Version : go1.11.4 darwin/amd64\n* OS: Mac OS X 10.13.6\n* Architecture: x86_64\n* 16 GB 2133 MHz LPDDR3\n* CPU: 3.1 GHz Intel Core i7\n\n```\nbadger 2022/03/09 14:04:44 INFO: All 0 tables opened in 0s\ngoos: darwin\ngoarch: amd64\npkg: github.com/arriqaaq/flashbench\ncpu: Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz\n\nBenchmarkBadgerDBPutValue64B-16     \t    9940\t    141844 ns/op\t    2208 B/op\t      68 allocs/op\nBenchmarkBadgerDBPutValue128B-16    \t    7701\t    192942 ns/op\t    2337 B/op\t      68 allocs/op\nBenchmarkBadgerDBPutValue256B-16    \t    7368\t    142600 ns/op\t    2637 B/op\t      69 allocs/op\nBenchmarkBadgerDBPutValue512B-16    \t    6980\t    148056 ns/op\t    3149 B/op\t      69 allocs/op\nBenchmarkBadgerDBGet-16             \t 1000000\t      1388 ns/op\t     408 B/op\t       9 allocs/op\n\nBenchmarkFlashDBPutValue64B-16     \t  204318\t      5129 ns/op\t    1385 B/op\t      19 allocs/op\nBenchmarkFlashDBPutValue128B-16    \t  231177\t      5318 ns/op\t    1976 B/op\t      16 allocs/op\nBenchmarkFlashDBPutValue256B-16    \t  189516\t      6202 ns/op\t    3263 B/op\t      15 allocs/op\nBenchmarkFlashDBPutValue512B-16    \t  165580\t      8110 ns/op\t    5866 B/op\t      16 allocs/op\nBenchmarkFlashDBGet-16             \t 4053836\t       295 ns/op\t      32 B/op\t       2 allocs/op\n\nPASS\nok  \tgithub.com/arriqaaq/flashbench\t28.947s\n```\n\n#### With fsync enabled for every update transaction\n\n```\nbadger 2022/03/09 14:04:44 INFO: All 0 tables opened in 0s\ngoos: darwin\ngoarch: amd64\npkg: github.com/arriqaaq/flashbench\ncpu: Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz\n\nBenchmarkNutsDBPutValue64B-16     \t      52\t  20301019 ns/op\t    1315 B/op\t      17 allocs/op\nBenchmarkNutsDBPutValue128B-16    \t      63\t  23496536 ns/op\t    1059 B/op\t      15 allocs/op\nBenchmarkNutsDBPutValue256B-16    \t      62\t  20037952 ns/op\t    1343 B/op\t      15 allocs/op\nBenchmarkNutsDBPutValue512B-16    \t      62\t  20090731 ns/op\t    1754 B/op\t      15 allocs/op\n\nBenchmarkFlashDBPutValue64B-16     \t      62\t  18364330 ns/op\t     692 B/op\t      15 allocs/op\nBenchmarkFlashDBPutValue128B-16    \t      64\t  18315903 ns/op\t    1015 B/op\t      15 allocs/op\nBenchmarkFlashDBPutValue256B-16    \t      64\t  19250441 ns/op\t    1694 B/op\t      15 allocs/op\nBenchmarkFlashDBPutValue512B-16    \t      61\t  18811900 ns/op\t    2976 B/op\t      15 allocs/op\nBenchmarkFlashDBGet-16\t\t\t    3599500\t     340.7 ns/op\t      32 B/op\t       2 allocs/op\n\nPASS\nok  \tgithub.com/arriqaaq/flashbench\t28.947s\n```\n\nThe benchmark code can be found here [flashdb-bench](https://github.com/arriqaaq/flashdb-bench).\n\n\n\nTODO\n====\n\nFlashDB is in early stages of development. A couple of to-do tasks listed:\n\n- Add more comprehensive unit test cases\n- Add explicit documentation on various usecases\n\n\nReferences\n==========\n\nFlashDB is inspired by NutsDB and BuntDB.\n\n\n## Contact\nFarhan Khan [@arriqaaq](http://twitter.com/arriqaaq)\n\n## License\nFlashDB source code is available under the MIT [License](/LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farriqaaq%2Fflashdb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Farriqaaq%2Fflashdb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farriqaaq%2Fflashdb/lists"}