https://github.com/tirthraj1605/yadfs

Yet Another Distributed File System
https://github.com/tirthraj1605/yadfs

hadoop python sql

Last synced: about 1 year ago
JSON representation

Yet Another Distributed File System

• Host: GitHub
• URL: https://github.com/tirthraj1605/yadfs
• Owner: Tirthraj1605
• Created: 2024-07-19T16:49:01.000Z (almost 2 years ago)
• Default Branch: main
• Last Pushed: 2024-11-15T08:43:14.000Z (over 1 year ago)
• Last Synced: 2024-11-15T09:35:26.599Z (over 1 year ago)
• Topics: hadoop, python, sql
• Language: Python
• Homepage:
• Size: 13.7 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# YADFS_BDProject

YADFS is a high-performance distributed file system designed for scalable and efficient data storage and retrieval in large-scale data environments. Built using big data technologies like Apache Spark and Hadoop, and enhanced with Kafka and Zookeeper for reliable data distribution, YADFS enables seamless management of massive datasets with minimal latency. This system is optimized for handling diverse workloads and ensures data consistency, fault tolerance, and fast access, making it ideal for data-intensive applications and real-time analytics. 

# Architecture 

![Alt text](https://github.com/Tirthraj1605/YADFS/blob/main/Architecture.png)