{"id":13411869,"url":"https://github.com/claygod/coffer","last_synced_at":"2026-03-09T02:32:12.564Z","repository":{"id":82560104,"uuid":"186474679","full_name":"claygod/coffer","owner":"claygod","description":"Simply ACID* key-value database. At the medium or even low latency it tries to provide greater throughput without losing the ACID properties of the database. The database provides the ability to create record headers at own discretion and use them as transactions. The maximum size of stored data is limited by the size of the computer's RAM.","archived":false,"fork":false,"pushed_at":"2023-04-09T17:17:50.000Z","size":271,"stargazers_count":40,"open_issues_count":1,"forks_count":5,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-08-14T16:55:12.410Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/claygod.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-05-13T18:30:23.000Z","updated_at":"2024-12-31T20:30:18.000Z","dependencies_parsed_at":"2023-06-19T03:50:02.604Z","dependency_job_id":null,"html_url":"https://github.com/claygod/coffer","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/claygod/coffer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/claygod%2Fcoffer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/claygod%2Fcoffer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/claygod%2Fcoffer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/claygod%2Fcoffer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/claygod","download_url":"https://codeload.github.com/claygod/coffer/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/claygod%2Fcoffer/sbom","scorecard":{"id":285632,"data":{"date":"2025-08-11","repo":{"name":"github.com/claygod/coffer","commit":"5cbeb46a3fb8e417f63102a49f70469115ab0d42"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3,"checks":[{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Code-Review","score":0,"reason":"Found 0/30 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"SAST","score":0,"reason":"no SAST tool detected","details":["Warn: no pull requests merged into dev branch"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"License","score":9,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Warn: project license file does not contain an FSF or OSI license."],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-17T17:10:09.556Z","repository_id":82560104,"created_at":"2025-08-17T17:10:09.556Z","updated_at":"2025-08-17T17:10:09.556Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30280854,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-09T02:23:26.802Z","status":"ssl_error","status_checked_at":"2026-03-09T02:22:46.175Z","response_time":61,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-30T20:01:17.797Z","updated_at":"2026-03-09T02:32:12.554Z","avatar_url":"https://github.com/claygod.png","language":"Go","funding_links":[],"categories":["Database","数据库","Data Integration Frameworks","数据库  `go语言实现的数据库`","Generators","Uncategorized"],"sub_categories":["Databases Implemented in Go","Advanced Console UIs","标准 CLI","Go中实现的数据库"],"readme":"[![GoDoc](https://godoc.org/github.com/claygod/coffer?status.svg)](https://godoc.org/github.com/claygod/coffer) [![Mentioned in Awesome Go](https://awesome.re/mentioned-badge.svg)](https://github.com/avelino/awesome-go) [![Travis CI](https://travis-ci.org/claygod/coffer.svg?branch=master)](https://travis-ci.org/claygod/coffer) [![Go Report Card](https://goreportcard.com/badge/github.com/claygod/coffer)](https://goreportcard.com/report/github.com/claygod/coffer) [![codecov](https://codecov.io/gh/claygod/coffer/branch/master/graph/badge.svg)](https://codecov.io/gh/claygod/coffer)\n\n\n# Coffer\n\nSimply ACID* key-value database. At the medium or even low `latency` it tries to\nprovide greater `throughput` without losing the ACID properties of the database. The\ndatabase provides the ability to create record headers at own discretion and use them\nas transactions. The maximum size of stored data is limited by the size of the\ncomputer's RAM.\n\n*is a set of properties of database transactions intended to guarantee validity even in\nthe event of errors, power failures, etc.\n\nProperties:\n- high throughput\n- tolerated latency\n- high reliability\n\nACID:\n- good durabilty\n- compulsory isolation\n- atomic operations\n- consistent transactions\n\n## Table of Contents\n\n * [Usage](#Usage)\n * [Examples](#Examples)\n * [API](#api)\n      + [Methods](#methods)\n * [Config](#config)\n      + [Handler](#Handler)\n\t      - [Example of a handler without using an argument](#Example-of-a-handler-without-using-an-argument)\n\t      - [Example of a handler using an argument](#Example-of-a-handler-using-an-argument)\n * [Launch](#Launch)\n      - [Start](#Start)\n      - [Follow](#Follow)\n * [Data storage](#Data-storage)\n      - [Data loading after an incorrect shutdown](#Data-loading-after-an-incorrect-shutdown )\n * [Error codes](#Error-codes)\n      - [Code List](#Code-List)\n      - [Code checks through methods](#Code-checks-through-methods)\n * [Benchmark](#Benchmark)\n * [Dependencies](#Dependencies)\n * [ToDo](#TODO)\n\n## Usage\n\n```golang\npackage main\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/claygod/coffer\"\n)\n\nconst curDir = \"./\"\n\nfunc main() {\n\t// STEP init\n\tdb, err, wrn := coffer.Db(curDir).Create()\n\tswitch {\n\tcase err != nil:\n\t\tfmt.Println(\"Error:\", err)\n\t\treturn\n\tcase wrn != nil:\n\t\tfmt.Println(\"Warning:\", err)\n\t\treturn\n\t}\n\tif !db.Start() {\n\t\tfmt.Println(\"Error: not start\")\n\t\treturn\n\t}\n\tdefer db.Stop()\n\n\t// STEP write\n\tif rep := db.Write(\"foo\", []byte(\"bar\")); rep.IsCodeError() {\n\t\tfmt.Sprintf(\"Write error: code `%d` msg `%s`\", rep.Code, rep.Error)\n\t\treturn\n\t}\n\n\t// STEP read\n\trep := db.Read(\"foo\")\n\tif rep.IsCodeError() {\n\t\tfmt.Sprintf(\"Read error: code `%v` msg `%v`\", rep.Code, rep.Error)\n\t\treturn\n\t}\n\tfmt.Println(string(rep.Data))\n}\n```\n\n### Examples\n\nUse the following links to find many examples how use the transactions:\n\n- `Quick start` https://github.com/claygod/coffer/tree/master/examples/quick_start\n- `Finance` https://github.com/claygod/coffer/tree/master/examples/finance\n\n## API\n\nStarted DB returns reports after has performed an operation. Reports containing:\n- code (error codes here: `github.com/claygod/coffer/reports/codes`)\n- error\n- data\n- other details\nReporting structures read here:: `github.com/claygod/coffer/reports`\n\n### Methods\n\n* Start\n* Stop\n* StopHard\n* Save\n* Write\n* WriteList\n* WriteListUnsafe\n* Read\n* ReadList\n* ReadListUnsafe\n* Delete\n* DeleteListStrict\n* DeleteListOptional\n* Transaction\n* Count\n* CountUnsafe\n* RecordsList\n* RecordsListUnsafe\n* RecordsListWithPrefix\n* RecordsListWithSuffix\n\nPay attention!\n\nAll requests which names contain `Unsafe` can be usually executed in both cases: when the\ndatabase is running or stopped (not running). In the second case (when DB is stopped),\nshould not make requests in parallel, because in this case the consistency of DB can\nbe compromised and data lost.\n\nOther methods work only if the database is running.\n\n#### Start\n\nThe `Follow` interactor turns on while running a database. It controls the relevance of the\ncurrent checkpoint.\n\n#### Stop\n\nStop DB. If you want to periodically stop and start the database in your application, probably,\nyou may want to create a new client when the DB has been stopped.\n\n#### Write\n\nWrite a new record in a database specifying the key and value. Their length must satisfy the\nrequirements specified in configuration files.\n\n#### WriteList\n\nWrite several records to the database specifying corresponding `map` in the arguments.\n\nStrict mode (strictMode=true):\n\tThe operation performs if there are no records with these keys.\n\tA list of existed records is returned.\n\t\nOptional mode (strictMode=false):\n\tThe operation performs regardless of whether there are records with these keys or not.\n\tA list of existed records is returned.\n\t\nImportant: this argument is a reference argument; it cannot be changed in the called code!\n\n#### WriteListUnsafe\n\nWrite several records to the database specified the corresponding map in the arguments.\nThis method exists in order to fill the database faster before it starts.\nThe method is not for parallel use.\n\n#### Read\n\nRead one record from the database. In the received report there will be a result code.\nIf it is positive, that means that the value in the right data field.\n\n#### ReadList\n\nRead several records. There is a limit on the maximum number of readable records in the\nconfiguration. Except found records the list of not found records is returned.\n\n#### ReadListUnsafe\n\nRead several records. The method can be called when the database is stopped (not running).\nThe method is not for parallel use.\n\n#### Delete\n\nRemove a single record.\n\n#### DeleteList\n\nStrict mode (true):\nDelete several records. It is possible only if all records are in the database.\nIf at least there is a lack of one record, none of records will be deleted.\n\nOptional mode (false):\nDelete several records. All found records from the list in will be deleted in DB.\n\n#### Transaction\n\nMake a transaction. The transaction should be added in the database at the stage of creating\nand configuring. The user of the database is responsible for the consistency of the functionality of\ntransaction handlers between different runs of the database.\nThe transaction returns new values which are stored in the DB.\n\n#### Count\n\nGet the number of records in the database. A request can be made only when the database has started.\n\n#### CountUnsafe\n\nGet the number of records in the database. Requests to a stopped (or not running),\ndatabase cannot be made in parallel!\n\n#### RecordsList\n\nGet a list of all database keys. With a large number of records in the database, the request\nwill be slow. Use it only at great need to avoid problems.\nThe method works only when the database is running.\n\n#### RecordsListUnsafe\n\nGet a list of all database keys. With a large number of records in the database, the request\nwill be slow. Use it only at great need to avoid problems. The method is not for parallel use\nwhile using a request when database is stopped (or not running).\n\n#### RecordsListWithPrefix\n\nGet a list of all keys with prefix specified in the argument (prefix is the begging of record string).\n\n## Config\n\nIf you specify the path to the database directory all configuration parameters will be\nreset to the default:\n\n\tcof, err, wrn := Db(dirPath) . Create()\n\nDefault values can be found in the `/config.go` file. But each of the parameters can be\nconfigured:\n\n```golang\n\tDb(dirPath).\n\tBatchSize(batchSize).\n\tLimitRecordsPerLogfile(limitRecordsPerLogfile).\n\tFollowPause(100*time.Second).\n\tLogsByCheckpoint(1000).\n\tAllowStartupErrLoadLogs(true).\n\tMaxKeyLength(maxKeyLength).\n\tMaxValueLength(maxValueLength).\n\tMaxRecsPerOperation(1000000).\n\tRemoveUnlessLogs(true).\n\tLimitMemory(100 * 1000000).\n\tLimitDisk(1000 * 1000000).\n\tHandler(\"handler1\", \u0026handler1).\n\tHandler(\"handler2\", \u0026handler2).\n\tHandlers(map[string]*handler).\n\tCreate()\n```\n\t\n### Db\n\nSpecify the work directory where the database will store files. In case of a new\ndatabase the directory should not contain files with the “log”, “check”, “checkpoint”\nextensions.\n\n### BatchSize\n\nThe maximum number of records which database can add at a time (this applies to\nsetting up internal processes; this does not apply to the number of records added at a\ntime).\nDecreasing of this parameter slightly improves the `latency` (but not too much).\nIncreasing of this parameter slightly degrades the `latency`, but at the same time\nincreases the `throughput`.\n\n### LimitRecordsPerLogfile\n\nA number of operations which is going to be written to one log file. Small number\nforces the database creates new files very often, and it adversely affects the speed of\nthe database. A big number reduces the number of pauses while creating files, but\nthe size of files increases.\n\n### FollowPause\n\nThe size of the time interval for starting the `Follow` interactor, which analyzes old\nlogs and periodically creates new checkpoints.\n\n### LogsByCheckpoint\n\nThe option specifies after how many full log files it is necessary to create a new\ncheckpoint (the smaller number, the more often it should be created). For good\nproductivity, it’s better not to do it too often.\n\n### AllowStartupErrLoadLogs\n\nThe option allows the database works at startup, even if the last log file was\ncompleted incorrectly, i.e. the last record is corrupted (a typical situation for an\nabnormal shutdown). By default, the option is enabled.\n\n### MaxKeyLength\n\nThis is the maximum allowable key length.\n\n### MaxValueLength\n\nThis is the maximum size of the value length.\n\n### MaxRecsPerOperation\n\nThis is the maximum number of records that is possible per operation.\n\n### RemoveUnlessLogs\n\nThe option is for deleting old files. After `Follow` has created a new checkpoint, with the\npermission of this option, it removes unnecessary operations logs. If for some reason\nit’s needed to store the whole log of operations, this option can be disabled. But be\nready that this will increase the consumption of disk space.\n\n### LimitMemory\n\nThis is the minimum size of free RAM. When this limit reaches, the database\nterminates all operations and stops to avoid data loss.\n\n### LimitDisk\n\nThis is the minimum amount of free space on the hard drive. When this limit reaches,\nthe database terminates all operations and stops to avoid data loss.\n\n### Handler\n\nAdd a transaction handler. It is important that the name of the handler and the results\nof its work should be idempotent while running the same database at different time.\nOtherwise handlers will work differently and it will leads to a violation of data\nconsistency. If you intend to make changes to handlers time to time, adding a version\nnumber to the key helps streamline this process.\n\nConditions:\n- The argument passed to the handler must be a number (a slice of bytes).\n- If you need to transfer complex structures, it must be serialized into bytes.\n- The handler can only operate on existing records.\n- The handler cannot delete database records.\n- The handler should return the new values of all the requested records at the end of his work.\n- The number of records modified with the header is specified in the `MaxRecsPerOperation` configuration.\n\n#### Example of a handler without using an argument\n\n```golang\nfunc HandlerExchange(arg []byte, recs map[string][]byte) (map[string][]byte, error) {\n\tif arg != nil {\n\t\treturn nil, fmt.Errorf(\"Args not null.\")\n\t} else if len(recs) != 2 {\n\t\treturn nil, fmt.Errorf(\"Want 2 records, have %d\", len(recs))\n\t}\n\trecsKeys := make([]string, 0, 2)\n\trecsValues := make([][]byte, 0, 2)\n\tfor k, v := range recs {\n\t\trecsKeys = append(recsKeys, k)\n\t\trecsValues = append(recsValues, v)\n\t}\n\tout := make(map[string][]byte, 2)\n\tout[recsKeys[0]] = recsValues[1]\n\tout[recsKeys[1]] = recsValues[0]\n\treturn out, nil\n}\n```\n\n#### Example of a handler using an argument\n\n```golang\nfunc HandlerDebit(arg []byte, recs map[string][]byte) (map[string][]byte, error) {\n\tif arg == nil || len(arg) != 8 {\n\t\treturn nil, fmt.Errorf(\"Invalid Argument: %v.\", arg)\n\t} else if len(recs) != 1 {\n\t\treturn nil, fmt.Errorf(\"Want 1 record, have %d\", len(recs))\n\t}\n\tdelta := bytesToUint64(arg)\n\tvar recKey string\n\tvar recValue []byte\n\tfor k, v := range recs {\n\t\trecKey = k\n\t\trecValue = v\n\t}\n\tif len(recValue) != 8 {\n\t\treturn nil, fmt.Errorf(\"The length of the value in the record is %d bytes, but 8 bytes are needed\", len(recValue))\n\t}\n\tcurAmount := bytesToUint64(recValue)\n\tnewAmount := curAmount + delta\n\tif curAmount \u003e newAmount {\n\t\treturn nil, fmt.Errorf(\"Account overflow. There is %d, a debit of %d.\", curAmount, delta)\n\t}\n\treturn map[string][]byte{recKey: uint64ToBytes(newAmount)}, nil\n}\n```\n\n### Handlers\n\nAdd several handlers to the database at once. Important: handlers with matching keys\nare overwritten.\n\n### Create\n\nA mandatory command (must be the last one) finishes the configuration and creates\nthe database.\n\n## Launch\n\n### Start\n\nAt starting DB, the number in the end should be a checkpoint. If it is not, the database\nhas been stopped incorrectly. In this “error” case the last available checkpoint and all\nlogs after the checkpoint are loaded until it is possible.\n\nLoad the data until it is possible and finish the loading (there is must be a broken log\n(log with uncompleted data) or last log which has created before database has been\nstopped incorrectly, so you can load all available data). After all available data has\nloaded the database creates a new checkpoint. Only after it you can continue work\nwith code.\n\n### Follow\n\nAfter the database has been started, it writes all operations to a log. As a result, the\nlog file can greatly grow. If at the end of the application work the database is correctly\nstopped, a new checkpoint appears. At the next start of DB, the data will be taken\nfrom it.\n\nBut if the database is incorrectly stopped a new checkpoint will not be created. In this\ncase, at a new start of DB, the database loads the old checkpoint and re-performs all\noperations that has been completed and recorded in the log. This process can take\nmuch time, and as a result, the database will be loading for a long time (not always\nacceptable for applications).\n\nThat is why there is the follower mechanism in the database that methodically goes\nthrough the logs while working of the database and periodically creates checkpoints\nwhich are closer to the current moment. Also, the follower has a functionality to clean\nold logs and checkpoints in order to free up the space of hard drive.\n\n## Data storage\n\nYour data is stored as files in the directory that has been specified while creating the\ndatabase. Files with the log extension contain a description of completed operations.\nFiles with the `checkpoint` extension contain snapshots of the database state at a\ncertain point. Files with the `check` extension contain an incomplete snapshot of the\ndatabase state. Using the `RemoveUnlessLogs` configuration parameter, you can force\nthe database to delete old and unnecessary files in order to save the disk space.\n\nIf the database is stopped in the regular mode, the last file, which has been written to\nthe disk, is the `checkpoint` file. The number of the `checkpoint` will be the maximum\nnumber. If the database is stopped incorrectly, most likely that files with the `log` or\n`check` extensions will have the maximum number.\n\nAttention! Before the database has not been completely stopped, it is forbidden to\ncarry out any operations with database files.\n\nIf you want to copy the database to somewhere, you must copy all content of the\ndirectory. If you want to take a minimum of files while copying, then you need: to\ncopy the file with the `checkpoint` extension (which has the maximum number), and\nall files with the `log` extension (which have bigger number than copied file with the\n`checkpoint` extension).\n\n### Data loading after an incorrect shutdown\n\nIf the application work, which is using the database, is not completed correctly, then\nat the next staring the application, the database will try to find the last valid snapshot\nof the `checkpoint` state.\n\nWhen the file is found, the database will upload it, and then upload all the `log` files\nwith big numbers. We expect that the last `log` file might not be filled completely\nbecause during the recording the work could be interrupted.\n\nOnly undamaged part is uploaded from the damaged file and after that the database\nuploading is considering as completed.\n\nAt the end of the uploading, the database creates a new `checkpoint`. If system\ncrashes occur during the start (loading) of the database, it possible to get errors and\nviolation of data consistency.\n\n## Error Codes\n\nError codes are stored here: `github.com/claygod/coffer/reports/codes`\nIf the `Ok` code is received, the operation is finished completely. If the Code contains\n`Error` (the operation has not been completed or not fully completed, or completed\nwith an error), you can continue working with the database. If the code contains\n`Panic`, you cannot continue working with the database because of it stage.\n\n### Code List\n\n- Ok - done without comment\n- Error - not completed or not fully completed, but you can continue to work\n- ErrRecordLimitExceeded - record limit per operation exceeded\n- ErrExceedingMaxValueSize - value is too long\n- ErrExceedingMaxKeyLength - key is too long\n- ErrExceedingZeroKeyLength - key is too short\n- ErrHandlerNotFound - no handler found\n- ErrParseRequest – preparing of logging request was failed\n- ErrResources - not enough resources\n- ErrNotFound - no keys are found\n- ErrReadRecords - reading records error for a transaction (if there is a lack of at least one record, a transaction cannot be performed)\n- ErrHandlerReturn - found and uploaded handler returned an error\n- ErrHandlerResponse - handler returned incomplete reply\n- Panic - not finished, further work with the database is impossible\n- PanicStopped – application has been stopped\n- PanicWAL - an error occurred in the operation log\n\n### Code checks through methods\n\nIn order not to export data to an application (which works with a database), reports\nhave methods:\n\n- IsCodeOk - done without comment\n- IsCodeError - not completed or not fully completed, but you can continue to work\n- IsCodeErrRecordLimitExceeded - record limit for one operation is exceeded\n- IsCodeErrExceedingMaxValueSize - value is too long\n- IsCodeErrExceedingMaxKeyLength - key is too long\n- IsCodeErrExceedingZeroKeyLength - key is too short\n- IsCodeErrHandlerNotFound - no handler found\n- IsCodeErrParseRequest - preparing of logging request was failed\n- IsCodeErrResources - not enough resources\n- IsCodeErrNotFound - no keys are found\n- IsCodeErrReadRecords - reading records error for a transaction (if there is a lack of at least one record, a transaction cannot be performed)\n- IsCodeErrHandlerReturn – found and uploaded handler returned an error\n- IsCodeErrHandlerResponse - handler returned incomplete reply\n- IsCodePanic - not finished, further work with the database is impossible\n- IsCodePanicStopped – application has been stopped\n- IsCodePanicWAL - error occurred in the operation log\n\nIt is not very convenient to make large switches to check the received codes. You can\nlimit yourself to just three checks:\n\n- IsCodeOk - done without comment\n- IsCodeError - not completed or not fully completed, but you can continue to work (covers ALL errors)\n- IsCodePanic - not completed, further work with the database is not possible (covers ALL panics)\n\n## Benchmark\n\n- BenchmarkCofferWriteParallel32LowConcurent-4\t\t100000\t12933 ns/op\n- BenchmarkCofferTransactionSequence-4\t\t\t2000\t\t227928 ns/op\n- BenchmarkCofferTransactionPar32NotConcurent-4\t100000\t4132 ns/op\n- BenchmarkCofferTransactionPar32HalfConcurent-4\t100000\t4199 ns/op\n\n## Dependencies\n\n- github.com/shirou/gopsutil/disk\n- github.com/shirou/gopsutil/mem\n- github.com/sirupsen/logrus\n\n## TODO\n\n- [x] the log should start a new log at startup\n- [x] study out the names of checkpoints and logs (numbering logic)\n- [x] launch and work of follower\n- [x] cleaning unnecessary logs via follower\n- [ ] provide an opportunity not to delete old logs, add a test!\n- [x] loading from broken files to stop loading, but work must continue (AllowStartupErrLoadLogs)\n- [x] cyclic loading of checkpoints until they run out (with errors)\n- [x] returns not errors, but reports of work that’s been completed\n- [x] add DeleteOptional, and add it in Operations too\n- [x] test Count\n- [x] Write test\n- [x] Read test\n- [x] Delete test\n- [x] Transaction test\n- [x] test RecordsList\n- [x] test RecordsListUnsafe\n- [x] test RecordsListWithPrefix\n- [x] test RecordsListWithSuffix\n- [x] ReadListUnsafe test\n- [x] boot test with a broken log (last, the rest are ok)\n- [x] boot test with broken checkpoint\n- [x] boot test with a broken log and another the log which follow after\n- [x] transaction usage test\n- [x] for convenience of testing do WriteUnsafe\n- [x] ~~ what for WriteUnsafeRecord is need in Checkpoint ? (for recording at startup?) ~~ alternative to WriteListUnsafe (faster)\n- [x] benchmark of competitive and non-competitive records\n- [x] benchmark reading competitive\n- [ ] benchmark write and read in competitive mode\n- [x] benchmark competitive transactions in parallel mode\n- [ ] at boot - when files are broken, the “wrn” may returns, not “err”\n- [x] study out the log and the batch, why at fast record they get to the following log\n- [x] interception of panics at the root of the application and at the level of use cases\n- [ ] ~~ during transactions, you can delete some of the records from participating (! need for a question!) ~~ \n- [x] testing auxiliary helpers\n- [x] while creating a database immediately add a list of handlers because the uploading from logs happens instantly\n- [x] add a convenient configurator while creating a database\n- [x] translate comments into English\n- [x] clear code from old artifacts\n- [ ] create a directory for documentation\n- [x] create a directory for examples\n- [x] make a simple example with writing, transaction and reading\n- [x] make an example with financial transactions\n- [ ] error handling example\n- [ ] test the linter and eliminate all incorrectness in the code\n- [x] add Usage / Quick start text to readme\n- [x] description of error codes\n- [x] configuration description\n- [x] in the description specify third-party packages (as dependencies)\n- [x] add methods for reports in order to check for all errors like IsErrBlahBlahBlah\n- [x] transfer all imported packages to distribution\n- [x] switch from WriteUnsafeRecord to WriteListUnsafe\n- [x] add ReadListUnsafe for an ability to read when the database is stopped\n- [x] add RecordsListUnsafe, which can work with the stopped and running database\n- [x] get a list of keys with a condition of a prefix: RecordsListWithPrefix\n- [x] get a list of keys with a condition of a suffix: RecordsListWithSuffix\n- [x] remove the Save method\n- [x] returns new values in the report during a transaction\n- [x] check returned value in tests\n- [x] start numbering with big numbers, for example million or a billion (more convenient for sorting files)\n- [x] give a correct description-comment for all public methods\n- [ ] create description for error returns and warnings in the Create method\n- [ ] pause in the batcher - check its size, set the optimal size\n- [x] add in the description that the data is stored both on disk and in memory during the operation of the database\n- [ ] method for getting all log files and checkpoints\n- [ ] method for viewing a log file\n- [ ] method of viewing a checkpoint file\n- [ ] the method of strict adding record into the database (only if the record with such a key has not already existed)\n- [ ] add a method to view the status of the database\n\n### Copyright © 2019-2026 Eduard Sesigin. All rights reserved. Contacts: \u003cclaygod@yandex.ru\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclaygod%2Fcoffer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fclaygod%2Fcoffer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclaygod%2Fcoffer/lists"}