{"id":20632144,"url":"https://github.com/vladimiry/fs-json-store","last_synced_at":"2025-04-15T18:57:43.986Z","repository":{"id":27177789,"uuid":"112803672","full_name":"vladimiry/fs-json-store","owner":"vladimiry","description":"Node.js module for storing JSON data on the file system","archived":false,"fork":false,"pushed_at":"2024-11-18T22:45:46.000Z","size":941,"stargazers_count":8,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-28T23:51:14.732Z","etag":null,"topics":["atomic","file-system","fs","json-store","storage","store","typescript"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vladimiry.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-12-02T01:44:02.000Z","updated_at":"2024-11-18T22:45:43.000Z","dependencies_parsed_at":"2024-06-16T11:24:52.568Z","dependency_job_id":"f9742d6d-fa08-4467-8a6d-e3c660da7340","html_url":"https://github.com/vladimiry/fs-json-store","commit_stats":{"total_commits":64,"total_committers":4,"mean_commits":16.0,"dds":0.21875,"last_synced_commit":"dcb890aa7087d4413f4359191f8071755f0bdf99"},"previous_names":[],"tags_count":27,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vladimiry%2Ffs-json-store","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vladimiry%2Ffs-json-store/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vladimiry%2Ffs-json-store/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vladimiry%2Ffs-json-store/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vladimiry","download_url":"https://codeload.github.com/vladimiry/fs-json-store/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249135790,"owners_count":21218365,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["atomic","file-system","fs","json-store","storage","store","typescript"],"created_at":"2024-11-16T14:15:05.658Z","updated_at":"2025-04-15T18:57:43.955Z","avatar_url":"https://github.com/vladimiry.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# fs-json-store\n\nis a module for Node.js for storing JSON data on the file system.\n\n[![GitHub Actions CI](https://github.com/vladimiry/fs-json-store/workflows/GitHub%20Actions%20CI/badge.svg?branch=master)](https://github.com/vladimiry/fs-json-store/actions)\n\nModule simply serializes/deserializes a file using `JSON.stringify / JSON.parse` functions, so it would not be a great idea to use it with a huge data sets, but it's ok for handling simple needs like storing app settings, etc.\n\n## Features\n\n- Atomic writing. Means data is not going to be corrupted (like getting half-written data file on abnormal program exit or power loss).\n- File system abstraction.\n- Custom adapters support. See [fs-json-store-encryption-adapter](https://github.com/vladimiry/fs-json-store-encryption-adapter) as an example.\n- Custom validation functions support.\n- Optimistic locking support (versioning).\n\n## Technical Notes\n- Module provides only `async` methods that return ES2015 Promises, `sync` methods set is not supported.\n- Module copes with [EPERM errors](https://github.com/search?q=EPERM\u0026type=Issues) using [fs-no-eperm-anymore](https://github.com/vladimiry/fs-no-eperm-anymore) module.\n- Module uses a custom atomic file writing implementation for the following reasons:\n  - atomic writing should be applied to all the fs abstractions (`StoreFs` implementations), not to just the node's `fs` module only, see [related issue](https://github.com/vladimiry/fs-json-store/issues/1) for details.\n  - [write-file-atomic](https://github.com/npm/write-file-atomic) module [doesn't yet properly handle](https://github.com/npm/write-file-atomic/issues/28) the [EPERM errors](https://github.com/isaacs/node-graceful-fs/pull/119) on Windows.  \n\n## Motivation\n\nI needed a simple to use module for storing JSON data on the file system with atomic writing, custom adapters, custom validators, optimistic locking features supported and TypeScript declarations provided. Besides store is supposed to cope with the EPERM errors pseudo-randomly happening on Windows. I didn't find an existing module that would meet the criteria, so a new one has been built.\n\n## Getting started\n\nUsing JavaScript and Promises:\n\n```javascript\nconst {Store} = require(\"fs-json-store\");\n\nconst store = new Store({file: \"data.json\"});\n\nstore.write([\"hello\"])\n    .then((data) =\u003e store.write([...data, \"world\"]))\n    .then(console.log) // prints \"[ 'hello', 'world' ]\"\n    .then(() =\u003e store.read())\n    .then(console.log); // prints \"[ 'hello', 'world' ]\"\n```\n\nUsing TypeScript and async/await:\n\n```typescript\nimport {Store} from \"fs-json-store\";\n\n(async () =\u003e {\n    const store = new Store({file: \"data.json\"});\n\n    console.log( // prints \"[ 'hello', 'world' ]\"\n        await store.write([...await store.write([\"hello\"]), \"world\",]),\n    );\n    console.log( // prints \"[ 'hello', 'world' ]\"\n        await store.read(),\n    );\n})();\n```\n\n## Store Signatures \n\n### `constructor(options)`\n\n- **`options`** `(object, required)`: an object with the flowing properties:\n    - **`file`** `(string, required)`: store file path.\n    - **`fs`** `(object, optional, defaults to the built-in node's \"fs\" wrapper)`: file system abstraction implementation. There is ony one built-in implementations which is a wrapped node's `fs` module. Custom abstractions can be added implementing the `StoreFs` interface.\n    - **`adapter`** `(object, optional)`: object or class instance with the `write(data: Buffer): Promise\u003cBuffer\u003e` and `read(data: Buffer): Promise\u003cBuffer` functions implemented. The custom adapter can be used for example for data encryption/archiving.\n    - **`optimisticLocking`** `(boolean, optional, defaults to false)`: flag property that toggles optimistic locking feature. With optimistic locking feature enabled stored data must be of the JSON `object` type, since the system `_rev` counter property needs be injected into the stored object.\n\n    - **`validators`** `(array, optional)`: array of the custom validation functions with the ```(data) =\u003e Promise\u003cstring | null\u003e``` signature, where `data` is the stored data. Store executes exposed `validate` method during both `read / write` operations.\n\n### `clone([options]): Store\u003cE\u003e`\n\nSynchronous method that returns a cloned store instance. See `options` parameter description in the `constructor` section, with the only difference is in that all the properties are optional, including the `file` property.\n\n### `readable(): Promise\u003cboolean\u003e`\n\nAsynchronous method that returns `true` if `file` associated with store is readable. It's basically a replacement for the `exists` method.\n\n### `readExisting([options]): Promise\u003cE\u003e`\n\nAsynchronous method that returns the stored data. Method throws an error if store is not `readable()`. Optional `options` argument is an object that can have the optional `adapter` property. Store uses the explicitly specified `adapter` overriding the instance's adapter just for the single `read` method execution (it might be useful for example in case if the data file initially was written using another adapter, so initial reading can be done using explicitly specified adapter).\n\n### `read([options]): Promise\u003cE | null\u003e`\n\nAsynchronous method that returns the stored data. Optional `options` argument is the same argument as in the `readExisting` method case.\n\n### `write(data, [options]): Promise\u003cE\u003e`\n\nAsynchronous method that writes data to `file` and returns the actual data. Optional `options.readAdapter` argument will be passed to the `read` method as the `options.adapter` argument (`read` method needs to be called during writing in case of the optimistic locking feature enabled).\n\n### `validate(data, messagePrefix?: string): Promise\u003cvoid\u003e`\n\nAsynchronous method that runs validation functions and throws an error in case of failed validation. Optional `messagePrefix` parameter will be added as a prefix to the error message.\n\n### `remove(): Promise\u003cvoid\u003e`\n\nAsynchronous method that removes the store associated `file`.\n\n## Usage Examples\n\n```typescript\nimport * as path from \"path\";\nimport * as pako from \"pako\";\nimport {Store, Model} from \"fs-json-store\";\n\nconst dataDirectory = path.join(process.cwd(), \"output\", String(Number(new Date())));\n\nconst examples = [\n    // basic\n    async () =\u003e {\n        const store = new Store({file: path.join(dataDirectory, \"basic.json\")});\n\n        await store.write([\n            ...await store.write([\"hello\"]),\n            \"world\",\n        ]);\n\n        console.log((await store.read()).join(\" \")); // prints `hello world`\n    },\n\n    // archiving adapter\n    async () =\u003e {\n        const store = new Store({\n            file: path.join(dataDirectory, \"archiving-adapter.bin\"),\n            adapter: {\n                async read(data) {\n                    return Buffer.from(pako.ungzip(data.toString(), {to: \"string\"}));\n                },\n                async write(data) {\n                    return Buffer.from(pako.gzip(data.toString(), {to: \"string\"}));\n                },\n            },\n        });\n\n        await store.write({data: \"archive data\"});\n\n        console.log(JSON.stringify(await store.read())); // prints `{\"data\":\"archive data\"}`\n    },\n\n    // validation\n    async () =\u003e {\n        interface DataModel extends Partial\u003cModel.StoreEntity\u003e {\n            numbers: number[];\n        }\n\n        const store = new Store\u003cDataModel\u003e({\n            file: path.join(dataDirectory, \"validation.json\"),\n            validators: [\n                async ({numbers}) =\u003e {\n                    if (!numbers || !numbers.length) {\n                        return `\"numbers\" array is not supposed to be empty`;\n                    }\n\n                    return null;\n                },\n            ],\n        });\n\n        try {\n            await store.write({numbers: []});\n        } catch (error) {\n            console.log(error); // prints error due to the failed validation\n        }\n\n        const storedData = await  store.write({numbers: [1]});\n        console.log(JSON.stringify(storedData)); // prints `{\"numbers\":[1]}`\n    },\n\n    // optimistic locking (versioning)\n    async () =\u003e {\n        const store = new Store({\n            file: path.join(dataDirectory, \"versioning.json\"),\n            optimisticLocking: true,\n        });\n        let storedData = await store.write({property: \"initial data\"});\n\n        console.log(storedData._rev); // prints `0`\n\n        try {\n            await store.write({newProperty: \"new data\"});\n        } catch (error) {\n            console.log(error); // prints error since `_rev` has not been specified\n        }\n\n        try {\n            await store.write({newProperty: \"new data\", _rev: 3});\n        } catch (error) {\n            console.log(error); // prints error since valid `_rev` has not been specified\n        }\n\n        storedData = await store.write({newProperty: \"new data\", _rev: storedData._rev});\n        console.log(storedData._rev); // prints `1`\n    },\n];\n\n(async () =\u003e {\n    for (const example of examples) {\n        await example();\n    }\n})();\n\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvladimiry%2Ffs-json-store","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvladimiry%2Ffs-json-store","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvladimiry%2Ffs-json-store/lists"}