{"id":15493841,"url":"https://github.com/seriousme/durable-data","last_synced_at":"2026-04-25T21:31:47.572Z","repository":{"id":62422217,"uuid":"395772434","full_name":"seriousme/durable-data","owner":"seriousme","description":"Durable versions of Map and Set using an append log on disk.","archived":false,"fork":false,"pushed_at":"2023-11-01T18:13:28.000Z","size":27,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-04-09T00:14:14.617Z","etag":null,"topics":["deno","deno-module","ndjson"],"latest_commit_sha":null,"homepage":"https://deno.land/x/durable_data","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/seriousme.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-08-13T19:32:26.000Z","updated_at":"2024-06-30T15:46:49.000Z","dependencies_parsed_at":"2024-11-18T07:47:09.051Z","dependency_job_id":null,"html_url":"https://github.com/seriousme/durable-data","commit_stats":{"total_commits":21,"total_committers":2,"mean_commits":10.5,"dds":0.1428571428571429,"last_synced_commit":"ee4667d8f82b11e43783df936d211fdd44c6068d"},"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/seriousme/durable-data","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seriousme%2Fdurable-data","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seriousme%2Fdurable-data/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seriousme%2Fdurable-data/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seriousme%2Fdurable-data/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/seriousme","download_url":"https://codeload.github.com/seriousme/durable-data/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seriousme%2Fdurable-data/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32278249,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-25T18:29:39.964Z","status":"ssl_error","status_checked_at":"2026-04-25T18:29:32.149Z","response_time":59,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deno","deno-module","ndjson"],"created_at":"2024-10-02T08:09:39.134Z","updated_at":"2026-04-25T21:31:47.549Z","avatar_url":"https://github.com/seriousme.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n[![Deno CI](https://github.com/seriousme/durable-data/actions/workflows/deno-ci.yml/badge.svg)](https://github.com/seriousme/durable-data/actions/workflows/deno-ci.yml)\n[![CodeQL](https://github.com/seriousme/durable-data/actions/workflows/codeql-analysis.yml/badge.svg)](https://github.com/seriousme/durable-data/actions/workflows/codeql-analysis.yml)\n\n\n# durable_data\n\nThe durable_data module is made to provide durability to data structures across\nrestarts of code.\n\n## Usage\n\nThe following modules are exposed in `mod.ts`. This module persists data to the\nfilesystem so Deno should be run with\n`deno run \u003cscript\u003e --allow-read=\u003cdatabase directory\u003e --allow-write=\u003cdatabase directory\u003e`\n\n### DurableMap\n\nA DurableMap is a TypeScript Map with durability added. All normal Map operations work in memory. The  `set`, `delete` and `clear` operations are written to a log. \n\n```ts\nimport {\n  DurableMap\n} from \"https://deno.land/x/durable_data/mod.ts\";\n\nconst dm = new DurableMap('./dbdir/dataMap.db',{\n    // all options have defaults\n    mode: 0o660, // 0o600 (rw- --- ---), file mode to be used\n    separator: \"\\r\\n\", // \"\\n\", separator between records\n    decode: (line:string) =\u003e atob(line), // (line)=\u003eline ,line decoder\n    encode: (line:string) =\u003e btoa(line), // (line)=\u003eline ,line encoder\n});\nawait dm.load(); // load previously stored data\ndm.set(\"a\",\"value of a\");\nawait dm.compact(); // compact record log on disk\nawait dm.destroy(); // destroy data stored on disk, not in memory !\n```\n\n### DurableSet\n\nA DurableSet is a TypeScript Set with durability added. All normal Set operations work in memory. The  `add`, `delete` and `clear` operations are written to a log. \n\n```ts\nimport {\n  DurableSet\n} from \"https://deno.land/x/durable_data/mod.ts\";\n\nconst ds = new DurableSet('./dbdir/dataSet.db', {\n    // all options have defaults\n    mode: 0o660, // 0o600 (rw- --- ---), file mode to be used\n    separator: \"\\r\\n\", // \"\\n\", separator between records\n    decode: (line:string) =\u003e atob(line), // (line)=\u003eline ,line decoder\n    encode: (line:string) =\u003e btoa(line), // (line)=\u003eline ,line encoder\n});\nawait ds.load(); // load previously stored data\nds.add(\"value of a\");\nawait ds.compact(); // compact record log on disk\nawait ds.destroy(); // destroy data stored on disk, not in memory !\n```\n\n## API\n\nDurableMap and DurableSet have the same api apart from the native differences between Map and Set.\nuse `deno doc` to view the API.\n\n### encode/decode\n\nThe encode/decode combo can be used for all kinds of transformations including encrypt/decrypt. \nMake sure that decode is the inverse of encode.\n\n## Log format\n\nThe default log format is [Newline Delimited JSON](http://ndjson.org/)\n\nFor DurableMap, e.g.:\n```json\n{\"cmd\":\"set\",\"key\":\"a\",\"value\":{\"l0\":{\"l01\":1,\"l02\":2}}}\n{\"cmd\":\"set\",\"key\":\"b\",\"value\":\"b\"}\n{\"cmd\":\"clear\"}\n{\"cmd\":\"set\",\"key\":\"c\",\"value\":\"c\"}\n{\"cmd\":\"delete\",\"key\":\"c\"}\n{\"cmd\":\"set\",\"key\":\"d\",\"value\":\"d\"}\n``` \n\nFor DurableSet, e.g:\n```json\n{\"cmd\":\"add\",\"key\":\"a\"}\n{\"cmd\":\"add\",\"key\":\"b\"}\n{\"cmd\":\"clear\"}\n{\"cmd\":\"add\",\"key\":\"c\"}\n{\"cmd\":\"delete\",\"key\":\"c\"}\n{\"cmd\":\"add\",\"key\":\"d\"}\n```\n\nCompaction using `compact()` will reduce the volume by processing all updates and writing new `set` or `add` records.\n\n### Notes\n\n- `JSON.stringify` and `JSON.parse` are used in writing and reading data, so you can only persist data that can be handled by `JSON.stringify`! If you wish to alter this behaviour you can either prepare your data before `set` or `add` or overload `JSON.stringify` and `JSON.parse`.\n\n- Each database should be connected to a single process. Multiple concurrent readers/writers is *not* supported !\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fseriousme%2Fdurable-data","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fseriousme%2Fdurable-data","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fseriousme%2Fdurable-data/lists"}