https://github.com/akhilk2802/distributed-file-storage
File storage with consensus and fault tolerance
https://github.com/akhilk2802/distributed-file-storage
consensus-algorithm data-replication distributed-systems file-storage filesystem peer-to-peer
Last synced: 4 months ago
JSON representation
File storage with consensus and fault tolerance
- Host: GitHub
- URL: https://github.com/akhilk2802/distributed-file-storage
- Owner: akhilk2802
- Created: 2024-07-03T07:17:02.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-02-10T02:10:12.000Z (8 months ago)
- Last Synced: 2025-05-18T15:12:45.891Z (5 months ago)
- Topics: consensus-algorithm, data-replication, distributed-systems, file-storage, filesystem, peer-to-peer
- Language: Go
- Homepage:
- Size: 15 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Distributed File Storage
## 📌 Overview
This project is a **distributed file storage system** built to explore **distributed systems concepts**. It implements a **custom-built Peer-to-Peer (P2P) library** for efficient network communication and decentralized file management. The system supports **data replication** for fault tolerance and provides essential file operations such as **GET, READ, and WRITE**.
Currently, it is a work in progress, with planned enhancements and additional features being actively developed.
---
## 🚀 Features
### 🔹 Custom P2P Library
- Implements **peer-to-peer communication** for decentralized storage.
- Manages **peer discovery, network communication, and data consistency**.
### 🔹 Data Replication
- Ensures **data availability and fault tolerance** by replicating files across multiple nodes.
- Reduces the risk of **data loss due to node failures**.
### 🔹 Core File Operations
#### **1️⃣ GET Method**
- Retrieves **metadata or status information** about files in the distributed system.
#### **2️⃣ READ Method**
- Reads the **contents of a specific file** from the distributed storage.
#### **3️⃣ WRITE Method**
- Writes **data to a specific file** while ensuring **data consistency and replication**.
---
## Folder Structure
```plaintext
.
├── Makefile
├── README.md
├── bin
│ └── fs
├── go.mod
├── go.sum
├── main.go
├── p2p
│ ├── encoding.go
│ ├── handshake.go
│ ├── message.go
│ ├── tcp_transport.go
│ ├── tcp_transport_test.go
│ └── transport.go
├── package-lock.json
├── package.json
├── server.go
├── store.go
└── store_test.go
```
---
## 🔧 Future Enhancements
- **Sharding support** for improved scalability.
- **Consensus mechanism** for ensuring data integrity.
- **Dynamic node management** for auto-scaling and resilience.
- **Security improvements** including **encryption and authentication**.
---
🚀 **Distributed File Storage - A Peer-to-Peer Approach to Scalable Storage!**