Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/aneldev/dyna-disk-memory
Behaves like memory but it uses only the disk. Universal.
https://github.com/aneldev/dyna-disk-memory
disk memory nodejs service universal webworker
Last synced: 1 day ago
JSON representation
Behaves like memory but it uses only the disk. Universal.
- Host: GitHub
- URL: https://github.com/aneldev/dyna-disk-memory
- Owner: aneldev
- License: mit
- Created: 2017-08-04T20:53:18.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2022-12-03T05:57:09.000Z (almost 2 years ago)
- Last Synced: 2024-03-28T19:08:01.150Z (8 months ago)
- Topics: disk, memory, nodejs, service, universal, webworker
- Language: TypeScript
- Homepage:
- Size: 2.69 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 17
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# About
`dyna disk memory` is a simple data holder where it uses only the disk. Ideal for node.js applications and services that they don't want to use the expensive memory but their server's disk.
The performance is not the same as the physical memory, as it uses the disk but in nowadays the ssd disks are much faster and also this library splits the files in small folders so the system uses smaller indices to locate a file. This mechanism is implemented and used for the first time by this library.
So `dyna disk memory` is intended for a large amount of temporary or permanent data with a high performance according the disk.
The API is really simple and has nothing to do with files etc, all this _hard job_ is part of this library.
If your project is in Typescript then voila, `dyna disk memory` is too (you'll become good friends). But anyway it can be used by any javascript environment.
# Is Universal
Is universal. On the browser it uses the `localStorage` so on the browser the data limit is the limit browser's `localStorage`.
**But why universal support?**
Ideal for libraries that work in micro services (node.js) or web workers (browsers) that want to save a permanent data.
# Data structure
We have the `container` which is something like a folder and we drop `keys` with `data` ‘inside the container’. That's all!
You can delete a `container` or delete all `containers` at once to clean up your disk.
# Usage
```
// import
import {DynaDiskMemory} from 'dyna-disk-memory/dist/commonJs/node'; // For node
import {DynaDiskMemory} from 'dyna-disk-memory/dist/commonJs/browser'; // For browser
// create a memory
const memory = new DynaDiskMemory({
diskPath: './debug/diskMemory/',
});
// set a key with an object
memory.set(
'recipes', // container name
'84643545', // key
{ description: 'Cake with almonds', // data
flour: 3.14, sugar: 1.2 }
) // returns promise
.then(()=>{
console.log('data saved'); // that it is saved
})
.catch((error) => {
console.error('cannot write', error);
});
memory.get(
'recipes', // container name
'84643545', // key
) // returns promise
.then((data)=>{
console.log('data retrieved', data.description); // with our data
})
.catch((error) => {
console.error('cannot read', error); // or error if key doesn't exist
});
```
# Methods
## constructor(settings: ISettings)
### diskPath: string (required)
The path which will be created and used to save the data. Can be relative or root based.
### fragmentSize: number (optional)
This is optional, the default value is 13. It Is the size of the fragments to break the folder names. No need to touch it.
## set(container: string, key: string, data: any): Promise
Writes or overwrites the `data` into a `container` for this `key`.
Returns a Promise, `resolve: Function` and `reject: (error:any) => void`.
You might get only these errors:
- the data cannot be stringified
- on node.js, there is a disk error
- on browser, there is no free space in localStorage
## get(container: string, key: string): Promise
It returns the`data` from this `container` for this `key`.
It returns a Promise, `resolve: (data: any) => void` with the saved data.
If `key`is not found the `resolve` will be returned with `data: undefined`;
## del(container: string, key: string): Promise
It deletes a `key` from this container.
It returns a Promise, `resolve: Function` on completion of delete and `reject: (error:any) => void` on disk error.
## delContainer(container: string): Promise
It deletes a `container`, that means all the `keys` and `data` will be gone permanently.
It returns a Promise, `resolve: Function` when the deletion is completed and `reject: (error:any) => void` on disk error.
## delAll(): Promise
It deletes everything, deletes the `diskPath` which is defined in the settings.
It returns a Promise, `resolve: Function` when the deletion is completed and `reject: (error:any) => void` on disk error.
# FAQ
## When can we get a disk error
> Note that errors are raised only as rejected Promises.
### In node.js
- When something is wrong with your disk.
- When someone blocks instantly your files (like a test-watch)
## In browser
- On `set()` if there is not enough space in localStorage.
Every browser has different limit, you have to catch and handle the rejected Promise.
While each browser has a different limit it doesn’t worth to count the data you stored.
A roughly check of the used localStore size can be done with this script:
```
Object.keys(localStorage).reduce((total, key)=>{
let data = localStorage[key];
if (typeof data == 'string') total += data.length;
return total;
}, 0);
```
## Where is the `test-watch`?
It doesn’t exist in purpose! The demon is watching the files under the `temp/` and it instantly blocks the state of them and tests are failing.
# The future
There is no thoughts to extend this library. It is doing something very specific and it is doing this well. Other amazing libraries can be built using this, like: queue handlers, network buffers that they don't want to lose their data if they crashed.
# Troubleshooting
## Webpack cannot resolve `child_process`
Webpack builder for web environments might complain for missing `child_process`.
In this case add this to your Webpack config:
```
node: {
child_process: "empty",
},
```
# Debug, Fork, Dist, Liberté!
This project made with [dyna-ts-module-boilerplate](https://github.com/aneldev/dyna-ts-module-boilerplate), help of it is [here in readme.boilerplate](./readme.boilerplate.md).