{"id":25305644,"url":"https://github.com/pedrokehl/distributed-file-server","last_synced_at":"2026-04-09T07:05:40.670Z","repository":{"id":225896557,"uuid":"86514163","full_name":"pedrokehl/distributed-file-server","owner":"pedrokehl","description":"Distributed file server using Python, NodeJS and Socket.IO","archived":false,"fork":false,"pushed_at":"2017-04-21T19:43:02.000Z","size":175,"stargazers_count":2,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-02-13T09:53:29.423Z","etag":null,"topics":["distributed-systems","flask","mongodb","nodejs","python","socket-io"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pedrokehl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2017-03-28T22:46:43.000Z","updated_at":"2024-03-07T06:06:07.000Z","dependencies_parsed_at":"2024-03-04T21:41:56.859Z","dependency_job_id":"a5392098-3fea-4819-abac-5b851c21f6ad","html_url":"https://github.com/pedrokehl/distributed-file-server","commit_stats":null,"previous_names":["pedrokehl/distributed-file-server"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pedrokehl%2Fdistributed-file-server","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pedrokehl%2Fdistributed-file-server/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pedrokehl%2Fdistributed-file-server/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pedrokehl%2Fdistributed-file-server/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pedrokehl","download_url":"https://codeload.github.com/pedrokehl/distributed-file-server/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247601482,"owners_count":20964863,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["distributed-systems","flask","mongodb","nodejs","python","socket-io"],"created_at":"2025-02-13T09:53:32.119Z","updated_at":"2025-12-30T23:05:54.365Z","avatar_url":"https://github.com/pedrokehl.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Distributed File Server\n=======================\n\n### Summary of features\n\n- A Python application that works like a manager to control the file-servers and get the requests from the clients.\n- A node.JS server that can have multiple instances in different environments that store the files\n- Balance the number of files per server, as file-servers connects and clients send files\n- Restrict 36 files per directory, if it exceeds, should create a new directory\n\n\n![alt text](https://github.com/pedrokehl/distributed-file-server/blob/master/docs/distributed-file-server-diagram.png \"Distributed File Server Diagram\")\n\n### Client\n\nNot implemented yet.\nResponsible for sending requests to the manager.\n\n### File Manager\nImplemented with Python using the Flask framework to provide a REST API, MongoDB to persist files and Socket.IO to communicate with File Servers\n\n- Receive client requests\n- Control which server should receive which file\n- Persist information about files\n- Balance servers\n\n\n![alt text](https://github.com/pedrokehl/distributed-file-server/blob/master/docs/manager-diagram.png \"File Manager Diagram\")\n\n\n### File Server\nImplemented with node.JS using socket.IO to communicate with the File Manager.\n\n- Connect with manager using Socket.IO\n- Receive manager requests like:\n    - delete-file\n    - upload-file\n    - download-file\n    - transfer-file\n- Control how many files per directory to do not exceed the number provided (36) then create another\n\n## Requirements\n\n- Can have many file-servers running simultaneously\n- A file send to the File-Manager should be saved in only one server\n- There's no replication between file-servers\n- A file server must have only 36 files/folder per directory\n- The client will send a base64 file, but the file-server must save it as a normal\n\n### Upload File\nStore a file in one of the file-servers\n\n##### Request\nPOST: [http://localhost:8000/file](http://localhost:8000/file)\n\nBody:\n\n| Field         | Description     | Required |\n| ------------- |-------------| :---------: |\n| filename | Name of the file, will be used to get the file later | X |\n| file | Content of the file encoded in base64 | X |\n##### Response\n\n- filename: Filename provided\n- id: Generated ID for the file saved\n\n### Download file\nGet the content of the file with the name provided\n\nGET: [http://localhost:8000/file/:filename](http://localhost:8000/file/:filename)\n\n### Delete file\nDelete the file of the file with the name provided\n\nDELETE: [http://localhost:8000/file/:filename](http://localhost:8000/file/:filename)\n\n\n### Response codes\n\nCode  | Content              | Description\n------- | ---------------------- | ------------\n1       | Servers unavailable  | When no server is available\n2       | File does not exist    | When the client request a file that does not exists\n3       | File name already exists | When the client tries to upload a file with a name that is already in use\n4       | File unavailable   | When the server of the file requests is unavailable\n\n\n### Instructions\n\n    # Clone de project\n    git clone --depth=1 https://github.com/pedrokehl/distributed-file-server.git\n    \n    # Run MongoDB\n    Execute mongod\n    \n    # Go to file-server-manager directory and install Python dependencies\n    pip install -r requirements.txt\n    \n    # Run the manager\n    python main.py\n    \n    # Open another terminal and go to the file-server-storage diretory    \n    # Install dependencies (use --production flag if you don't want devDependencies)\n    npm install\n    \n    # Start the server with the server id\n    node index.js 100\n\n### TODO\n\n- Implement the client\n- Change the manager to accept files directly instead of base64\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpedrokehl%2Fdistributed-file-server","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpedrokehl%2Fdistributed-file-server","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpedrokehl%2Fdistributed-file-server/lists"}