{"id":17132060,"url":"https://github.com/dayyass/pydfs","last_synced_at":"2025-04-13T07:55:41.115Z","repository":{"id":53025745,"uuid":"509189523","full_name":"dayyass/pydfs","owner":"dayyass","description":"Distributed File System written in Python","archived":false,"fork":false,"pushed_at":"2022-08-05T10:54:02.000Z","size":63,"stargazers_count":14,"open_issues_count":11,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-04-13T07:55:36.022Z","etag":null,"topics":["distributed-systems","filesystem","hadoop","hdfs","mapreduce","python"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/dfspy","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dayyass.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-06-30T18:21:15.000Z","updated_at":"2025-03-24T22:43:31.000Z","dependencies_parsed_at":"2022-09-04T19:02:02.675Z","dependency_job_id":null,"html_url":"https://github.com/dayyass/pydfs","commit_stats":null,"previous_names":["silkway-ai/pydfs"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dayyass%2Fpydfs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dayyass%2Fpydfs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dayyass%2Fpydfs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dayyass%2Fpydfs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dayyass","download_url":"https://codeload.github.com/dayyass/pydfs/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248681491,"owners_count":21144700,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["distributed-systems","filesystem","hadoop","hdfs","mapreduce","python"],"created_at":"2024-10-14T19:25:52.794Z","updated_at":"2025-04-13T07:55:41.093Z","avatar_url":"https://github.com/dayyass.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PyDFS\nDistributed File System written in Python.\n\n## Installation\n```\npip install dfspy\n```\n\n## Usage\nPyDFS is Centralized Distributed File System, which means there is a *master* and *slave* nodes.\nThe current implementation assumes that the system has only one master node and many slave nodes.\n\nPyDFS supports *command line interface* (*CLI*) to interact with it.\nThere are 2 groups of the commands (like Docker Management Commands):\n- `pydfs init` - to initialize, manage and sync nodes\n- `pydfs dfs` - to interact with DFS itself (put/get data to/from it)\n\nLet's take a closer look at these commands.\n\n### init commands\nWith `pydfs init` command you can initialize master and slave nodes - it's pretty simple:\n- `pydfs init master`\n- `pydfs init slave --master_ip [IP]`\n\n### dfs commands\n- `pydfs dfs put --path [PATH] --master_ip [IP]`\n- `pydfs dfs get --path [PATH] --master_ip [IP]`\n\n### other commands\n- `pydfs --version`\n- `pydfs --info`\n\n### Docker\nYou can also use *docker-compose* to run multi-container application with:\n- 1 master node\n- 2 slave nodes\n- 2 user nodes\n```\ndocker-compose up --build\n```\n\n## Requirements\nPython \u003e= 3.7\n\n## Citation\nIf you use **PyDFS** in a scientific publication, we would appreciate references to the following BibTex entry:\n```bibtex\n@misc{silkwayai2022pydfs,\n    author       = {Dani El-Ayyass and Artem Fomin},\n    title        = {Distributed File System written in Python},\n    howpublished = {\\url{https://github.com/silkway-ai/pydfs}},\n    year         = {2022}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdayyass%2Fpydfs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdayyass%2Fpydfs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdayyass%2Fpydfs/lists"}