{"id":15486879,"url":"https://github.com/codenoid/sdfs","last_synced_at":"2025-04-22T17:28:07.922Z","repository":{"id":100303297,"uuid":"185814554","full_name":"codenoid/sdfs","owner":"codenoid","description":"a Connector |\u003e Distributed File System based on Symlinking (that fit author need)","archived":false,"fork":false,"pushed_at":"2019-10-23T12:53:00.000Z","size":6489,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-29T17:12:40.451Z","etag":null,"topics":["distributed-file-system","distributed-systems"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/codenoid.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-05-09T14:24:08.000Z","updated_at":"2023-09-08T17:53:37.000Z","dependencies_parsed_at":"2023-05-13T13:15:36.308Z","dependency_job_id":null,"html_url":"https://github.com/codenoid/sdfs","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codenoid%2Fsdfs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codenoid%2Fsdfs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codenoid%2Fsdfs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codenoid%2Fsdfs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/codenoid","download_url":"https://codeload.github.com/codenoid/sdfs/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250286702,"owners_count":21405487,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["distributed-file-system","distributed-systems"],"created_at":"2024-10-02T06:20:18.169Z","updated_at":"2025-04-22T17:28:07.845Z","avatar_url":"https://github.com/codenoid.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SDFS [![Go Report Card](https://goreportcard.com/badge/github.com/codenoid/sdfs)](https://goreportcard.com/report/github.com/codenoid/sdfs)\n\nSymlinked Distributed File System\n\n![Image](https://raw.githubusercontent.com/codenoid/sdfs/master/image/draw.png)\n\nSDFS use Symlink method to distribute file accross node (multi data center)\n\nCurrent important feature that missing in SDFS is : \n\n1. Replication (You can use Glusterfs in storage node)\n2. Disordered Process/Async NFS (non-queue), probably your data can lost when your node (that use \u003c92% disk usage) is crash\n3. Permission Validation\n4. Option \u0026 Customization\n5. Documentation\n6. Web Monitoring (Usage, Health Check, etc)\n7. etc\n\n## Usage\n\nSDFS Has 2 route API, `/api/connect` for combine new node and `/api/symlink` to tell the master what and where the file should i distribute accross available node, yes you can use FileSystem watcher, but what if you have thousand of directory ?, yeah more RAM (currently master-server only save symlink file)\n\n\u003e Tested \u0026 Used in Ubuntu 16.04\n\n### Master Server\n\n1. create a `/data` directory\n2. install `sudo apt-get install nfs-common` (make sure /etc/fstab are exist)\n3. Build master-server and run as sudo\n\n### Storage Server\n\n1. create a directory to receive data from Master-Server\n2. install `sudo apt-get install nfs-kernel-server` (make sure /etc/exports are exist)\n3. build storage-server and run `./storage-server connect 6.6.6.28 /path/to/receive/data storagenodeid` (6.6.6.28 is master ip address)\n\n### Example Usage\n\n1. Your application receive a file from user, and save that file in `/mnt/storage/application/image/image.jpg`\n2. After saving that file, tell master-server to distribute `/mnt/storage/application/image/image.jpg` (use /api/symlink)\n3. SDFS Will choose available storage-server and move the real file to storage-server then SDFS will create a symlink from storage-server to the first file path / saved file path\n4. You can easily use `connect` command to add new storage-server\n\n```\nPOST master-server.ip:2219/api/symlink\n# form url encoded, call this after you file has been saved\nurl: /path/to/saved/file.mp4\n```\n\n**or**\n\n1. use [File Mover](https://github.com/codenoid/sdfs/tree/master/services/file-mover)\n\n## Development Log\n\n* 13 May 2019\n\nSome code has not been tested (yet !)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodenoid%2Fsdfs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcodenoid%2Fsdfs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodenoid%2Fsdfs/lists"}