{"id":21368473,"url":"https://github.com/bereket-g/google-file-system-implementation-with-python","last_synced_at":"2025-07-13T05:32:11.983Z","repository":{"id":94037657,"uuid":"120411828","full_name":"Bereket-G/Google-File-System-Implementation-with-Python","owner":"Bereket-G","description":null,"archived":false,"fork":false,"pushed_at":"2018-08-28T15:14:26.000Z","size":5,"stargazers_count":13,"open_issues_count":1,"forks_count":4,"subscribers_count":2,"default_branch":"master","last_synced_at":"2023-12-19T16:07:46.837Z","etag":null,"topics":["chunk","chunk-servers","filesystem","gfs","python"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Bereket-G.png","metadata":{"files":{"readme":"ReadMe.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2018-02-06T06:29:09.000Z","updated_at":"2022-06-10T19:24:39.000Z","dependencies_parsed_at":"2023-07-26T12:31:08.059Z","dependency_job_id":null,"html_url":"https://github.com/Bereket-G/Google-File-System-Implementation-with-Python","commit_stats":{"total_commits":3,"total_committers":1,"mean_commits":3.0,"dds":0.0,"last_synced_commit":"1e288c92618ba908916f48d41f5d86259ce0e322"},"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Bereket-G%2FGoogle-File-System-Implementation-with-Python","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Bereket-G%2FGoogle-File-System-Implementation-with-Python/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Bereket-G%2FGoogle-File-System-Implementation-with-Python/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Bereket-G%2FGoogle-File-System-Implementation-with-Python/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Bereket-G","download_url":"https://codeload.github.com/Bereket-G/Google-File-System-Implementation-with-Python/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225858622,"owners_count":17535447,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chunk","chunk-servers","filesystem","gfs","python"],"created_at":"2024-11-22T07:24:48.351Z","updated_at":"2024-11-22T07:24:48.954Z","avatar_url":"https://github.com/Bereket-G.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":" # Google File System Implementation with python \n\n GFS consists of five components: \n\n \t1. Master\n\n \t\t* It is a single server that holds all the metadata for the filesystem. By metadata we mean all information \n \t\t  about each file, it consititutes chunks, chunk server locations, and chunks indexes.\n\n \t2. Chunk servers\n\n \t\t* are where the actual data is stored and most connections takes place between client and chunk servers,\n \t\t  to avoid the master as a bottleneck.\n\n \t3. Primary back up Server\n\n \t\t* is a single server it holds the back-up for the information stored in the Master Server. If the master server\n \t\t accidentally crashes and later when it starts it can have all information backup from the primary backup server.\n\n \t\t*And to make this file system more tolerant we made the primary backup server to dump its information to disk, if\n \t\t it excperince unexpected failure.\n\n \t4. Client\n\n \t\t* is the only user-interactive part of the system. It mediates all requests between the client for the filesystem \n \t\taccess and the master and the chunkservers for data storage and retrieval.\n\n \t5. GFS.conf\n\n \t\t*It is a configruation file for the master server. It includes block size information to chunk and chunk servers \n \t\tlocation and port.\n\n\n## Requirement \n\n\t\n\t#rpyc :- is a python library for remote procedure calls.\n\n\n# Demonstration \n\t\n\t(launch the python files with separeate terminal)\n\n\t1. \u003e\u003e\u003e python primary_BackUp_server.py\n\t2. \u003e\u003e\u003e python Master_Server.py\n\t3. \u003e\u003e\u003e python Chunks.py\n\n\t4. \u003e\u003e\u003e python Client.py put ~/file_to_be_uploaded.txt myFileName \n\t5. \u003e\u003e\u003e python Client.py list\n\t\tlists all files uploaded.\n\n\t6. \u003e\u003e\u003e python Client.py get myFileName\n\t\t-Here it will get the metadata from the chunk and finally it will combine the chunks and it will print it here.\n\n\t7. \u003e\u003e\u003e python Client.py put ~/new/file oldFileName\n\t\t-It will overwrite the old file.\n\n\t7. \u003e\u003e\u003e python Client.py delete myFileName\n\t\t-It will delete the from the metadata and also delete all chunks from respective chunk servers.\n\n\n### Nb.\n\t\n\t*It will save the chunks in the home folder under directory \"gfs_root\" on all chunkServers.\n\t*It is not operating system dependent. The chunkServer can be Windows, Linux or Mac in the network.\t\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbereket-g%2Fgoogle-file-system-implementation-with-python","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbereket-g%2Fgoogle-file-system-implementation-with-python","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbereket-g%2Fgoogle-file-system-implementation-with-python/lists"}