Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/eduardsui/edwork
Decentralized, distributed read-write filesystem
https://github.com/eduardsui/edwork
decentralized-storage filesystem
Last synced: 3 months ago
JSON representation
Decentralized, distributed read-write filesystem
- Host: GitHub
- URL: https://github.com/eduardsui/edwork
- Owner: eduardsui
- License: mit
- Created: 2018-08-16T10:14:15.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2022-10-09T12:28:33.000Z (about 2 years ago)
- Last Synced: 2024-05-02T16:09:27.075Z (6 months ago)
- Topics: decentralized-storage, filesystem
- Language: C
- Homepage:
- Size: 4.19 MB
- Stars: 25
- Watchers: 7
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- AwesomeCppGameDev - edwork - write filesystem (C++)
README
# edwork
You may donate Bitcoin for this project at 14LqvMzFfaJ82C7wY5iavvTf9HPELYWsax
edwork is a lightweight decentralized, distributed read-write filesystem. It uses UDP and SCTP/UDP protocol for node sync. For now, it should not be used in production.
How it works
------------
After compiling you will get either an `edfs_mount`(if built with fuse or dokany) or `edfs_console`. This is a node. Every node is the same (no master). The difference is that some nodes may have a full key pair (public-private) and some have only the public key, that can be only used for checking signatures.
The nodes discover each other by using 3 strategies:
1. using the -use parameter to explicitly set the first node (eg.: `./edfs_mount -use someserver:4848`)
2. hardcoding a node (by defining EDFS_DEFAULT_HOST at compile time)
3. exchanging lists of addresses between them
All data traffic is encrypted using chacha20.
Every file has an unique 64-bit id, computed as:
```
file_id = xxhash(sha256(parent_id || filename))
```
There is some collision probability on deployments with lots of files. This could be mediated by using larger file ids. 64bit id's were used for compatibility with fuse's `ino_t` type.
After the file id is computed, the file content is split into 57k chunks (in order to fit in a UDP datagram). Each chunk is optionally compressed (compression is enabled by default).
Every user will have the full inode list, but not every file chunk. The inode list is sent when receiving a `ROOT` request.
File chunks are sent "on-needed" by broadcasting a ``WANT`` request. In order to avoid flooding the edwork nodes, a cost function (proof of work) function was added. It is closely related with hashcash original specification, but instead of sha-1, it uses sha3-256. For every chunk (57k or less) a node must compute a hash begining with 12 zero bits. A `ROOT` request needs a 14 zero bit proof of work.
The filesystem supports most of the I/O operations except rename. This is because the `file_id` is dependent of its name.
Compiling
------------
edwork may be compiled stand-alone (with no dependencies), with fuse or with dokany-fuse compatibility layer.
On Linux (fuse):
```
$ make
```
Linux, FreeBSD, OS X, without fuse:
```
$ make -f Makefile.console
```
Windows (dokany):
```
make -f Makefile.win32
```
Windows (console):
```
make -f Makefile.console.win32
```
Work in progress
------------
This is work in progress, protocol and API may change. For now, is not suitable for production use.