{"id":37128089,"url":"https://github.com/v2e4lisp/badger","last_synced_at":"2026-01-14T14:54:50.885Z","repository":{"id":57593481,"uuid":"149008507","full_name":"v2e4lisp/badger","owner":"v2e4lisp","description":"Fast key-value DB in Go.","archived":false,"fork":true,"pushed_at":"2018-09-14T16:39:14.000Z","size":11168,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-06-20T15:01:12.979Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://open.dgraph.io/post/badger/","language":"Go","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"dgraph-io/badger","license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/v2e4lisp.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-09-16T15:24:28.000Z","updated_at":"2018-09-16T15:24:30.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/v2e4lisp/badger","commit_stats":null,"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"purl":"pkg:github/v2e4lisp/badger","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/v2e4lisp%2Fbadger","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/v2e4lisp%2Fbadger/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/v2e4lisp%2Fbadger/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/v2e4lisp%2Fbadger/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/v2e4lisp","download_url":"https://codeload.github.com/v2e4lisp/badger/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/v2e4lisp%2Fbadger/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28424016,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T13:30:50.153Z","status":"ssl_error","status_checked_at":"2026-01-14T13:29:08.907Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-14T14:54:49.776Z","updated_at":"2026-01-14T14:54:50.856Z","avatar_url":"https://github.com/v2e4lisp.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# BadgerDB [![GoDoc](https://godoc.org/github.com/dgraph-io/badger?status.svg)](https://godoc.org/github.com/dgraph-io/badger) [![Go Report Card](https://goreportcard.com/badge/github.com/dgraph-io/badger)](https://goreportcard.com/report/github.com/dgraph-io/badger) [![Build Status](https://teamcity.dgraph.io/guestAuth/app/rest/builds/buildType:(id:Badger_UnitTests)/statusIcon.svg)](https://teamcity.dgraph.io/viewLog.html?buildTypeId=Badger_UnitTests\u0026buildId=lastFinished\u0026guest=1) ![Appveyor](https://ci.appveyor.com/api/projects/status/github/dgraph-io/badger?branch=master\u0026svg=true) [![Coverage Status](https://coveralls.io/repos/github/dgraph-io/badger/badge.svg?branch=master)](https://coveralls.io/github/dgraph-io/badger?branch=master)\n\n![Badger mascot](images/diggy-shadow.png)\n\nBadgerDB is an embeddable, persistent, simple and fast key-value (KV) database\nwritten in pure Go. It's meant to be a performant alternative to non-Go-based\nkey-value stores like [RocksDB](https://github.com/facebook/rocksdb).\n\n## Project Status\nBadger v1.0 was released in Nov 2017. Check the [Changelog] for the full details.\n\n[Changelog]:https://github.com/dgraph-io/badger/blob/master/CHANGELOG.md\n\nWe introduced transactions in [v0.9.0] which involved a major API change. If you have a Badger\ndatastore prior to that, please use [v0.8.1], but we strongly urge you to upgrade. Upgrading from\nboth v0.8 and v0.9 will require you to [take backups](#database-backup) and restore using the new\nversion.\n\n[v1.0.1]: //github.com/dgraph-io/badger/tree/v1.0.1\n[v0.8.1]: //github.com/dgraph-io/badger/tree/v0.8.1\n[v0.9.0]: //github.com/dgraph-io/badger/tree/v0.9.0\n\n## Table of Contents\n * [Getting Started](#getting-started)\n    + [Installing](#installing)\n    + [Opening a database](#opening-a-database)\n    + [Transactions](#transactions)\n      - [Read-only transactions](#read-only-transactions)\n      - [Read-write transactions](#read-write-transactions)\n      - [Managing transactions manually](#managing-transactions-manually)\n    + [Using key/value pairs](#using-keyvalue-pairs)\n    + [Monotonically increasing integers](#monotonically-increasing-integers)\n    * [Merge Operations](#merge-operations)\n    + [Setting Time To Live(TTL) and User Metadata on Keys](#setting-time-to-livettl-and-user-metadata-on-keys)\n    + [Iterating over keys](#iterating-over-keys)\n      - [Prefix scans](#prefix-scans)\n      - [Key-only iteration](#key-only-iteration)\n    + [Garbage Collection](#garbage-collection)\n    + [Database backup](#database-backup)\n    + [Memory usage](#memory-usage)\n    + [Statistics](#statistics)\n  * [Resources](#resources)\n    + [Blog Posts](#blog-posts)\n  * [Contact](#contact)\n  * [Design](#design)\n    + [Comparisons](#comparisons)\n    + [Benchmarks](#benchmarks)\n  * [Other Projects Using Badger](#other-projects-using-badger)\n  * [Frequently Asked Questions](#frequently-asked-questions)\n\n## Getting Started\n\n### Installing\nTo start using Badger, install Go 1.8 or above and run `go get`:\n\n```sh\n$ go get github.com/dgraph-io/badger/...\n```\n\nThis will retrieve the library and install the `badger_info` command line\nutility into your `$GOBIN` path.\n\n\n### Opening a database\nThe top-level object in Badger is a `DB`. It represents multiple files on disk\nin specific directories, which contain the data for a single database.\n\nTo open your database, use the `badger.Open()` function, with the appropriate\noptions. The `Dir` and `ValueDir` options are mandatory and must be\nspecified by the client. They can be set to the same value to simplify things.\n\n```go\npackage main\n\nimport (\n\t\"log\"\n\n\t\"github.com/dgraph-io/badger\"\n)\n\nfunc main() {\n  // Open the Badger database located in the /tmp/badger directory.\n  // It will be created if it doesn't exist.\n  opts := badger.DefaultOptions\n  opts.Dir = \"/tmp/badger\"\n  opts.ValueDir = \"/tmp/badger\"\n  db, err := badger.Open(opts)\n  if err != nil {\n\t  log.Fatal(err)\n  }\n  defer db.Close()\n  // Your code here…\n}\n```\n\nPlease note that Badger obtains a lock on the directories so multiple processes\ncannot open the same database at the same time.\n\n### Transactions\n\n#### Read-only transactions\nTo start a read-only transaction, you can use the `DB.View()` method:\n\n```go\nerr := db.View(func(txn *badger.Txn) error {\n  // Your code here…\n  return nil\n})\n```\n\nYou cannot perform any writes or deletes within this transaction. Badger\nensures that you get a consistent view of the database within this closure. Any\nwrites that happen elsewhere after the transaction has started, will not be\nseen by calls made within the closure.\n\n#### Read-write transactions\nTo start a read-write transaction, you can use the `DB.Update()` method:\n\n```go\nerr := db.Update(func(txn *badger.Txn) error {\n  // Your code here…\n  return nil\n})\n```\n\nAll database operations are allowed inside a read-write transaction.\n\nAlways check the returned error value. If you return an error\nwithin your closure it will be passed through.\n\nAn `ErrConflict` error will be reported in case of a conflict. Depending on the state\nof your application, you have the option to retry the operation if you receive\nthis error.\n\nAn `ErrTxnTooBig` will be reported in case the number of pending writes/deletes in\nthe transaction exceed a certain limit. In that case, it is best to commit the\ntransaction and start a new transaction immediately. Here is an example (we are\nnot checking for errors in some places for simplicity):\n\n```go\nupdates := make(map[string]string)\ntxn := db.NewTransaction(true)\nfor k,v := range updates {\n  if err := txn.Set([]byte(k),[]byte(v)); err == ErrTxnTooBig {\n    _ = txn.Commit()\n    txn = db.NewTransaction(..)\n    _ = txn.Set([]byte(k),[]byte(v))\n  }\n}\n_ = txn.Commit()\n```\n\n#### Managing transactions manually\nThe `DB.View()` and `DB.Update()` methods are wrappers around the\n`DB.NewTransaction()` and `Txn.Commit()` methods (or `Txn.Discard()` in case of\nread-only transactions). These helper methods will start the transaction,\nexecute a function, and then safely discard your transaction if an error is\nreturned. This is the recommended way to use Badger transactions.\n\nHowever, sometimes you may want to manually create and commit your\ntransactions. You can use the `DB.NewTransaction()` function directly, which\ntakes in a boolean argument to specify whether a read-write transaction is\nrequired. For read-write transactions, it is necessary to call `Txn.Commit()`\nto ensure the transaction is committed. For read-only transactions, calling\n`Txn.Discard()` is sufficient. `Txn.Commit()` also calls `Txn.Discard()`\ninternally to cleanup the transaction, so just calling `Txn.Commit()` is\nsufficient for read-write transaction. However, if your code doesn’t call\n`Txn.Commit()` for some reason (for e.g it returns prematurely with an error),\nthen please make sure you call `Txn.Discard()` in a `defer` block. Refer to the\ncode below.\n\n```go\n// Start a writable transaction.\ntxn := db.NewTransaction(true)\ndefer txn.Discard()\n\n// Use the transaction...\nerr := txn.Set([]byte(\"answer\"), []byte(\"42\"))\nif err != nil {\n    return err\n}\n\n// Commit the transaction and check for error.\nif err := txn.Commit(nil); err != nil {\n    return err\n}\n```\n\nThe first argument to `DB.NewTransaction()` is a boolean stating if the transaction\nshould be writable.\n\nBadger allows an optional callback to the `Txn.Commit()` method. Normally, the\ncallback can be set to `nil`, and the method will return after all the writes\nhave succeeded. However, if this callback is provided, the `Txn.Commit()`\nmethod returns as soon as it has checked for any conflicts. The actual writing\nto the disk happens asynchronously, and the callback is invoked once the\nwriting has finished, or an error has occurred. This can improve the throughput\nof the application in some cases. But it also means that a transaction is not\ndurable until the callback has been invoked with a `nil` error value.\n\n### Using key/value pairs\nTo save a key/value pair, use the `Txn.Set()` method:\n\n```go\nerr := db.Update(func(txn *badger.Txn) error {\n  err := txn.Set([]byte(\"answer\"), []byte(\"42\"))\n  return err\n})\n```\n\nThis will set the value of the `\"answer\"` key to `\"42\"`. To retrieve this\nvalue, we can use the `Txn.Get()` method:\n\n```go\nerr := db.View(func(txn *badger.Txn) error {\n  item, err := txn.Get([]byte(\"answer\"))\n  if err != nil {\n    return err\n  }\n  val, err := item.Value()\n  if err != nil {\n    return err\n  }\n  fmt.Printf(\"The answer is: %s\\n\", val)\n  return nil\n})\n```\n\n`Txn.Get()` returns `ErrKeyNotFound` if the value is not found.\n\nPlease note that values returned from `Get()` are only valid while the\ntransaction is open. If you need to use a value outside of the transaction\nthen you must use `copy()` to copy it to another byte slice.\n\nUse the `Txn.Delete()` method to delete a key.\n\n### Monotonically increasing integers\n\nTo get unique monotonically increasing integers with strong durability, you can\nuse the `DB.GetSequence` method. This method returns a `Sequence` object, which\nis thread-safe and can be used concurrently via various goroutines.\n\nBadger would lease a range of integers to hand out from memory, with the\nbandwidth provided to `DB.GetSequence`. The frequency at which disk writes are\ndone is determined by this lease bandwidth and the frequency of `Next`\ninvocations. Setting a bandwith too low would do more disk writes, setting it\ntoo high would result in wasted integers if Badger is closed or crashes.\nTo avoid wasted integers, call `Release` before closing Badger.\n\n```go\nseq, err := db.GetSequence(key, 1000)\ndefer seq.Release()\nfor {\n  num, err := seq.Next()\n}\n```\n\n### Merge Operations\nBadger provides support for unordered merge operations. You can define a func\nof type `MergeFunc` which takes in an existing value, and a value to be\n_merged_ with it. It returns a new value which is the result of the _merge_\noperation. All values are specified in byte arrays. For e.g., here is a merge\nfunction (`add`) which adds a `uint64` value to an existing `uint64` value.\n\n```Go\nuint64ToBytes(i uint64) []byte {\n  var buf [8]byte\n  binary.BigEndian.PutUint64(buf[:], i)\n  return buf[:]\n}\n\nfunc bytesToUint64(b []byte) uint64 {\n  return binary.BigEndian.Uint64(b)\n}\n\n// Merge function to add two uint64 numbers\nfunc add(existing, new []byte) []byte {\n  return uint64ToBytes(bytesToUint64(existing) + bytesToUint64(new))\n}\n```\n\nThis function can then be passed to the `DB.GetMergeOperator()` method, along\nwith a key, and a duration value. The duration specifies how often the merge\nfunction is run on values that have been added using the `MergeOperator.Add()`\nmethod.\n\n`MergeOperator.Get()` method can be used to retrieve the cumulative value of the key\nassociated with the merge operation.\n\n```Go\nkey := []byte(\"merge\")\nm := db.GetMergeOperator(key, add, 200*time.Millisecond)\ndefer m.Stop()\n\nm.Add(uint64ToBytes(1))\nm.Add(uint64ToBytes(2))\nm.Add(uint64ToBytes(3))\n\nres, err := m.Get() // res should have value 6 encoded\nfmt.Println(bytesToUint64(res))\n```\n\n### Setting Time To Live(TTL) and User Metadata on Keys\nBadger allows setting an optional Time to Live (TTL) value on keys. Once the TTL has\nelapsed, the key will no longer be retrievable and will be eligible for garbage\ncollection. A TTL can be set as a `time.Duration` value using the `Txn.SetWithTTL()`\nAPI method.\n\nAn optional user metadata value can be set on each key. A user metadata value\nis represented by a single byte. It can be used to set certain bits along\nwith the key to aid in interpreting or decoding the key-value pair. User\nmetadata can be set using the `Txn.SetWithMeta()` API method.\n\n`Txn.SetEntry()` can be used to set the key, value, user metatadata and TTL,\nall at once.\n\n### Iterating over keys\nTo iterate over keys, we can use an `Iterator`, which can be obtained using the\n`Txn.NewIterator()` method. Iteration happens in byte-wise lexicographical sorting\norder.\n\n\n```go\nerr := db.View(func(txn *badger.Txn) error {\n  opts := badger.DefaultIteratorOptions\n  opts.PrefetchSize = 10\n  it := txn.NewIterator(opts)\n  defer it.Close()\n  for it.Rewind(); it.Valid(); it.Next() {\n    item := it.Item()\n    k := item.Key()\n    v, err := item.Value()\n    if err != nil {\n      return err\n    }\n    fmt.Printf(\"key=%s, value=%s\\n\", k, v)\n  }\n  return nil\n})\n```\n\nThe iterator allows you to move to a specific point in the list of keys and move\nforward or backward through the keys one at a time.\n\nBy default, Badger prefetches the values of the next 100 items. You can adjust\nthat with the `IteratorOptions.PrefetchSize` field. However, setting it to\na value higher than GOMAXPROCS (which we recommend to be 128 or higher)\nshouldn’t give any additional benefits. You can also turn off the fetching of\nvalues altogether. See section below on key-only iteration.\n\n#### Prefix scans\nTo iterate over a key prefix, you can combine `Seek()` and `ValidForPrefix()`:\n\n```go\ndb.View(func(txn *badger.Txn) error {\n  it := txn.NewIterator(badger.DefaultIteratorOptions)\n  defer it.Close()\n  prefix := []byte(\"1234\")\n  for it.Seek(prefix); it.ValidForPrefix(prefix); it.Next() {\n    item := it.Item()\n    k := item.Key()\n    v, err := item.Value()\n    if err != nil {\n      return err\n    }\n    fmt.Printf(\"key=%s, value=%s\\n\", k, v)\n  }\n  return nil\n})\n```\n\n#### Key-only iteration\nBadger supports a unique mode of iteration called _key-only_ iteration. It is\nseveral order of magnitudes faster than regular iteration, because it involves\naccess to the LSM-tree only, which is usually resident entirely in RAM. To\nenable key-only iteration, you need to set the `IteratorOptions.PrefetchValues`\nfield to `false`. This can also be used to do sparse reads for selected keys\nduring an iteration, by calling `item.Value()` only when required.\n\n```go\nerr := db.View(func(txn *badger.Txn) error {\n  opts := badger.DefaultIteratorOptions\n  opts.PrefetchValues = false\n  it := txn.NewIterator(opts)\n  defer it.Close()\n  for it.Rewind(); it.Valid(); it.Next() {\n    item := it.Item()\n    k := item.Key()\n    fmt.Printf(\"key=%s\\n\", k)\n  }\n  return nil\n})\n```\n\n### Garbage Collection\nBadger values need to be garbage collected, because of two reasons:\n\n* Badger keeps values separately from the LSM tree. This means that the compaction operations\nthat clean up the LSM tree do not touch the values at all. Values need to be cleaned up\nseparately.\n\n* Concurrent read/write transactions could leave behind multiple values for a single key, because they\nare stored with different versions. These could accumulate, and take up unneeded space beyond the\ntime these older versions are needed.\n\nBadger relies on the client to perform garbage collection at a time of their choosing. It provides\nthe following methods, which can be invoked at an appropriate time:\n\n* `DB.PurgeOlderVersions()`: Is no longer needed since v1.5.0. Badger's LSM tree automatically discards older/invalid versions of keys.\n* `DB.RunValueLogGC()`: This method is designed to do garbage collection while\n  Badger is online. Along with randomly picking a file, it uses statistics generated by the\n  LSM-tree compactions to pick files that are likely to lead to maximum space\n  reclamation.\n\nIt is recommended that this method be called during periods of low activity in\nyour system, or periodically. One call would only result in removal of at max\none log file. As an optimization, you could also immediately re-run it whenever\nit returns nil error (indicating a successful value log GC).\n\n```go\nticker := time.NewTicker(5 * time.Minute)\ndefer ticker.Stop()\nfor range ticker.C {\nagain:\n  err := db.RunValueLogGC(0.7)\n  if err == nil {\n    goto again\n  }\n}\n```\n\n### Database backup\nThere are two public API methods `DB.Backup()` and `DB.Load()` which can be\nused to do online backups and restores. Badger v0.9 provides a CLI tool\n`badger`, which can do offline backup/restore. Make sure you have `$GOPATH/bin`\nin your PATH to use this tool.\n\nThe command below will create a version-agnostic backup of the database, to a\nfile `badger.bak` in the current working directory\n\n```\nbadger backup --dir \u003cpath/to/badgerdb\u003e\n```\n\nTo restore `badger.bak` in the current working directory to a new database:\n\n```\nbadger restore --dir \u003cpath/to/badgerdb\u003e\n```\n\nSee `badger --help` for more details.\n\nIf you have a Badger database that was created using v0.8 (or below), you can\nuse the `badger_backup` tool provided in v0.8.1, and then restore it using the\ncommand above to upgrade your database to work with the latest version.\n\n```\nbadger_backup --dir \u003cpath/to/badgerdb\u003e --backup-file badger.bak\n```\n\n### Memory usage\nBadger's memory usage can be managed by tweaking several options available in\nthe `Options` struct that is passed in when opening the database using\n`DB.Open`.\n\n- `Options.ValueLogLoadingMode` can be set to `options.FileIO` (instead of the\n  default `options.MemoryMap`) to avoid memory-mapping log files. This can be\n  useful in environments with low RAM.\n- Number of memtables (`Options.NumMemtables`)\n  - If you modify `Options.NumMemtables`, also adjust `Options.NumLevelZeroTables` and\n   `Options.NumLevelZeroTablesStall` accordingly.\n- Number of concurrent compactions (`Options.NumCompactors`)\n- Mode in which LSM tree is loaded (`Options.TableLoadingMode`)\n- Size of table (`Options.MaxTableSize`)\n- Size of value log file (`Options.ValueLogFileSize`)\n\nIf you want to decrease the memory usage of Badger instance, tweak these\noptions (ideally one at a time) until you achieve the desired\nmemory usage.\n\n### Statistics\nBadger records metrics using the [expvar] package, which is included in the Go\nstandard library. All the metrics are documented in [y/metrics.go][metrics]\nfile.\n\n`expvar` package adds a handler in to the default HTTP server (which has to be\nstarted explicitly), and serves up the metrics at the `/debug/vars` endpoint.\nThese metrics can then be collected by a system like [Prometheus], to get\nbetter visibility into what Badger is doing.\n\n[expvar]: https://golang.org/pkg/expvar/\n[metrics]: https://github.com/dgraph-io/badger/blob/master/y/metrics.go\n[Prometheus]: https://prometheus.io/\n\n## Resources\n\n### Blog Posts\n1. [Introducing Badger: A fast key-value store written natively in\nGo](https://open.dgraph.io/post/badger/)\n2. [Make Badger crash resilient with ALICE](https://blog.dgraph.io/post/alice/)\n3. [Badger vs LMDB vs BoltDB: Benchmarking key-value databases in Go](https://blog.dgraph.io/post/badger-lmdb-boltdb/)\n4. [Concurrent ACID Transactions in Badger](https://blog.dgraph.io/post/badger-txn/)\n\n## Design\nBadger was written with these design goals in mind:\n\n- Write a key-value database in pure Go.\n- Use latest research to build the fastest KV database for data sets spanning terabytes.\n- Optimize for SSDs.\n\nBadger’s design is based on a paper titled _[WiscKey: Separating Keys from\nValues in SSD-conscious Storage][wisckey]_.\n\n[wisckey]: https://www.usenix.org/system/files/conference/fast16/fast16-papers-lu.pdf\n\n### Comparisons\n| Feature             | Badger                                       | RocksDB                       | BoltDB    |\n| -------             | ------                                       | -------                       | ------    |\n| Design              | LSM tree with value log                      | LSM tree only                 | B+ tree   |\n| High Read throughput | Yes                                          | No                           | Yes        |\n| High Write throughput | Yes                                          | Yes                           | No        |\n| Designed for SSDs   | Yes (with latest research \u003csup\u003e1\u003c/sup\u003e)      | Not specifically \u003csup\u003e2\u003c/sup\u003e | No        |\n| Embeddable          | Yes                                          | Yes                           | Yes       |\n| Sorted KV access    | Yes                                          | Yes                           | Yes       |\n| Pure Go (no Cgo)    | Yes                                          | No                            | Yes       |\n| Transactions        | Yes, ACID, concurrent with SSI\u003csup\u003e3\u003c/sup\u003e | Yes (but non-ACID)            | Yes, ACID |\n| Snapshots           | Yes                                           | Yes                           | Yes       |\n| TTL support         | Yes                                           | Yes                           | No       |\n\n\u003csup\u003e1\u003c/sup\u003e The [WISCKEY paper][wisckey] (on which Badger is based) saw big\nwins with separating values from keys, significantly reducing the write\namplification compared to a typical LSM tree.\n\n\u003csup\u003e2\u003c/sup\u003e RocksDB is an SSD optimized version of LevelDB, which was designed specifically for rotating disks.\nAs such RocksDB's design isn't aimed at SSDs.\n\n\u003csup\u003e3\u003c/sup\u003e SSI: Serializable Snapshot Isolation. For more details, see the blog post [Concurrent ACID Transactions in Badger](https://blog.dgraph.io/post/badger-txn/)\n\n### Benchmarks\nWe have run comprehensive benchmarks against RocksDB, Bolt and LMDB. The\nbenchmarking code, and the detailed logs for the benchmarks can be found in the\n[badger-bench] repo. More explanation, including graphs can be found the blog posts (linked\nabove).\n\n[badger-bench]: https://github.com/dgraph-io/badger-bench\n\n## Other Projects Using Badger\nBelow is a list of known projects that use Badger:\n\n* [0-stor](https://github.com/zero-os/0-stor) - Single device object store.\n* [Dgraph](https://github.com/dgraph-io/dgraph) - Distributed graph database.\n* [Sandglass](https://github.com/celrenheit/sandglass) - distributed, horizontally scalable, persistent, time sorted message queue.\n* [Usenet Express](https://usenetexpress.com/) - Serving over 300TB of data with Badger.\n* [go-ipfs](https://github.com/ipfs/go-ipfs) - Go client for the InterPlanetary File System (IPFS), a new hypermedia distribution protocol.\n* [gorush](https://github.com/appleboy/gorush) - A push notification server written in Go.\n* [emitter](https://github.com/emitter-io/emitter) - Scalable, low latency, distributed pub/sub broker with message storage, uses MQTT, gossip and badger.\n* [GarageMQ](https://github.com/valinurovam/garagemq) - AMQP server written in Go.\n\nIf you are using Badger in a project please send a pull request to add it to the list.\n\n## Frequently Asked Questions\n- **My writes are getting stuck. Why?**\n\nThis can happen if a long running iteration with `Prefetch` is set to false, but\na `Item::Value` call is made internally in the loop. That causes Badger to\nacquire read locks over the value log files to avoid value log GC removing the\nfile from underneath. As a side effect, this also blocks a new value log GC\nfile from being created, when the value log file boundary is hit.\n\nPlease see Github issues [#293](https://github.com/dgraph-io/badger/issues/293)\nand [#315](https://github.com/dgraph-io/badger/issues/315).\n\nThere are multiple workarounds during iteration:\n\n1. Use `Item::ValueCopy` instead of `Item::Value` when retrieving value.\n1. Set `Prefetch` to true. Badger would then copy over the value and release the\n   file lock immediately.\n1. When `Prefetch` is false, don't call `Item::Value` and do a pure key-only\n   iteration. This might be useful if you just want to delete a lot of keys.\n1. Do the writes in a separate transaction after the reads.\n\n- **My writes are really slow. Why?**\n\nAre you creating a new transaction for every single key update, and waiting for\nit to `Commit` fully before creating a new one? This will lead to very low\nthroughput. To get best write performance, batch up multiple writes inside a\ntransaction using single `DB.Update()` call. You could also have multiple such\n`DB.Update()` calls being made concurrently from multiple goroutines.\n\nThe way to achieve the highest write throughput via Badger, is to do serial\nwrites and use callbacks in `txn.Commit`, like so:\n\n```go\nche := make(chan error, 1)\nstoreErr := func(err error) {\n  if err == nil {\n    return\n  }\n  select {\n    case che \u003c- err:\n    default:\n  }\n}\n\ngetErr := func() error {\n  select {\n    case err := \u003c-che:\n      return err\n    default:\n      return nil\n  }\n}\n\nvar wg sync.WaitGroup\nfor _, kv := range kvs {\n  wg.Add(1)\n  txn := db.NewTransaction(true)\n  handle(txn.Set(kv.Key, kv.Value))\n  handle(txn.Commit(func(err error) {\n    storeErr(err)\n    wg.Done()\n  }))\n}\nwg.Wait()\nreturn getErr()\n```\n\nIn this code, we passed a callback function to `txn.Commit`, which can pick up\nand return the first error encountered, if any. Callbacks can be made to do more\nthings, like retrying commits etc.\n\n- **I don't see any disk write. Why?**\n\nIf you're using Badger with `SyncWrites=false`, then your writes might not be written to value log\nand won't get synced to disk immediately. Writes to LSM tree are done inmemory first, before they\nget compacted to disk. The compaction would only happen once `MaxTableSize` has been reached. So, if\nyou're doing a few writes and then checking, you might not see anything on disk. Once you `Close`\nthe database, you'll see these writes on disk.\n\n- **Reverse iteration doesn't give me the right results.**\n\nJust like forward iteration goes to the first key which is equal or greater than the SEEK key, reverse iteration goes to the first key which is equal or lesser than the SEEK key. Therefore, SEEK key would not be part of the results. You can typically add a `0xff` byte as a suffix to the SEEK key to include it in the results. See the following issues: [#436](https://github.com/dgraph-io/badger/issues/436) and [#347](https://github.com/dgraph-io/badger/issues/347).\n\n- **Which instances should I use for Badger?**\n\nWe recommend using instances which provide local SSD storage, without any limit\non the maximum IOPS. In AWS, these are storage optimized instances like i3. They\nprovide local SSDs which clock 100K IOPS over 4KB blocks easily.\n\n- **I'm getting a closed channel error. Why?**\n\n```\npanic: close of closed channel\npanic: send on closed channel\n```\n\nIf you're seeing panics like above, this would be because you're operating on a closed DB. This can happen, if you call `Close()` before sending a write, or multiple times. You should ensure that you only call `Close()` once, and all your read/write operations finish before closing.\n\n- **Are there any Go specific settings that I should use?**\n\nWe *highly* recommend setting a high number for GOMAXPROCS, which allows Go to\nobserve the full IOPS throughput provided by modern SSDs. In Dgraph, we have set\nit to 128. For more details, [see this\nthread](https://groups.google.com/d/topic/golang-nuts/jPb_h3TvlKE/discussion).\n\n- **Are there any linux specific settings that I should use?**\n\nWe recommend setting max file descriptors to a high number depending upon the expected size of you data.\n\n## Contact\n- Please use [discuss.dgraph.io](https://discuss.dgraph.io) for questions, feature requests and discussions.\n- Please use [Github issue tracker](https://github.com/dgraph-io/badger/issues) for filing bugs or feature requests.\n- Join [![Slack Status](http://slack.dgraph.io/badge.svg)](http://slack.dgraph.io).\n- Follow us on Twitter [@dgraphlabs](https://twitter.com/dgraphlabs).\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fv2e4lisp%2Fbadger","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fv2e4lisp%2Fbadger","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fv2e4lisp%2Fbadger/lists"}