Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

Projects in Awesome Lists tagged with distributed-storage

A curated list of projects in awesome lists tagged with distributed-storage .

https://github.com/seaweedfs/seaweedfs

SeaweedFS is a distributed storage system for object storage (S3), file systems, and Iceberg tables, designed to handle billions of files with O(1) disk access and effortless horizontal scaling.

blob-storage cloud-drive distributed-file-system distributed-storage distributed-systems erasure-coding fuse hadoop-hdfs hdfs kubernetes object-storage posix replication s3 s3-storage seaweedfs tiered-file-system

Last synced: 15 Jun 2026

https://github.com/baidu/braft

An industrial-grade C++ implementation of RAFT consensus algorithm based on brpc, widely used inside Baidu to build highly-available distributed systems.

distributed-consensus distributed-storage raft raft-consensus-algorithm raft-cpp raft-cxx raft-protocol state-machine-replication

Last synced: 13 May 2025

https://github.com/brpc/braft

An industrial-grade C++ implementation of RAFT consensus algorithm based on brpc, widely used inside Baidu to build highly-available distributed systems.

distributed-consensus distributed-storage raft raft-consensus-algorithm raft-cpp raft-cxx raft-protocol state-machine-replication

Last synced: 20 Apr 2025

https://github.com/storj/storj

Ongoing Storj v3 development. Decentralized cloud object storage that is affordable, easy to use, private, and secure.

distributed distributed-storage distributed-systems golang object-storage open-source storage

Last synced: 16 Apr 2026

https://github.com/zuoyebang/bitalostored

Bitalostored is a high-performance distributed storage system, core engine based on bitalosdb(self-developed), compatible with Redis protocol.

bitalosdb database distributed-storage high-performance kvstore nosql redis storage-engine

Last synced: 03 Apr 2026

https://github.com/pravega/pravega

Pravega - Streaming as a new software defined storage primitive

data-ingestion distributed-storage real-time-data streaming streaming-data

Last synced: 13 May 2025

https://github.com/eclipse-zenoh/zenoh

zenoh unifies data in motion, data in-use, data at rest and computations. It carefully blends traditional pub/sub with geo-distributed storages, queries and computations, while retaining a level of time and space efficiency that is well beyond any of the mainstream stacks.

distributed-computing distributed-storage distributed-systems edge-computing embedded geo-distributed-storages iot messaging network-programming networking protocol robotics ros2 rust rust-lang storage zenoh

Last synced: 12 May 2025

https://github.com/moosefs/moosefs

MooseFS Distributed Storage – Open Source, Petabyte, Fault-Tolerant, Highly Performing, Scalable Network Distributed File System / Software-Defined Storage

backup block-storage distributed-computing distributed-file-storage distributed-file-system distributed-storage ditributed-systems erasure-coding file-storage file-system filesystem fuse high-availability media posix posix-compliant snapshots software-defined-storage storage tiering

Last synced: 14 May 2025

https://github.com/douban/gobeansdb

Distributed object storage server from Douban Inc.

distributed-storage kvstore memcached object-storage

Last synced: 04 Apr 2025

https://github.com/ArchiveBox/good-karma-kit

😇 A Docker Compose bundle to run on servers with spare CPU, RAM, disk, and bandwidth to help the world. Includes Tor, ArchiveWarrior, BOINC, and more...

archivebox archivewarrior boinc distributed-computing distributed-storage docker docker-compose foldingathome good-karma i2p internet-archiving ipfs kiwix pywb sia storj tor zimfarm

Last synced: 07 Apr 2025

https://github.com/archivebox/good-karma-kit

😇 A Docker Compose bundle to run on servers with spare CPU, RAM, disk, and bandwidth to help the world. Includes Tor, ArchiveWarrior, BOINC, and more...

archivebox archivewarrior boinc distributed-computing distributed-storage docker docker-compose foldingathome good-karma i2p internet-archiving ipfs kiwix pywb sia storj tor zimfarm

Last synced: 06 Apr 2025

https://github.com/chrislusf/vasto

A distributed key-value store. On Disk. Able to grow or shrink without service interruption.

distributed-storage key-value replication rocksdb

Last synced: 12 May 2025

https://github.com/matrixorigin/matrixcube

Fundamental Building Block for Elastic Storage With Strong Consistency and Reliability

cloud-native consensus distributed-storage distributed-systems go paxos raft

Last synced: 24 Apr 2025

https://github.com/zuoyebang/bitalosdb

Bitalosdb is a high-performance KV storage engine.

database distributed-storage high-performance io-architecture kvstore nosql storage-engine

Last synced: 13 Apr 2025

https://github.com/suse/deepsea

A collection of Salt files for deploying, managing and automating Ceph.

ceph deployment-system distributed-storage python salt software-defined-storage

Last synced: 13 Jul 2025

https://github.com/SUSE/DeepSea

A collection of Salt files for deploying, managing and automating Ceph.

ceph deployment-system distributed-storage python salt software-defined-storage

Last synced: 19 Jul 2025

https://github.com/leil-io/saunafs

SaunaFS is a free-and open source, distributed POSIX file system inspired by Google File System.

archiving distributed-file-system distributed-storage fault-tolerance high-availability high-performance high-performance-computing self-healing self-hosted

Last synced: 25 Feb 2026

https://github.com/freakmaxi/kertish-dos

Kertish-dos is a simple distributed object storage platform, implements object storage on a single distributed computer cluster, and provides interfaces for file/folder handling. Kertish-dos aims primarily for completely distributed operation without a single point of failure, scalable to the exabyte level.

data-farm dfs distributed-file-storage distributed-file-system distributed-object-storage distributed-object-store distributed-storage dos object-storage s3-storage scalable

Last synced: 11 Dec 2025

https://github.com/pitchpoint-solutions/sfs

The legacy distributed object storage server developed by PitchPoint Solutions can store billions of large and small files using minimal resources. Object data is stored in replicated volumes implemented like Facebooks Haystack Object Store. Object metadata which essentially maps an object name to a volume position is stored in an elasticsearch index. (Development by PitchPoint Solutions has been discontinued)

distributed-file-system distributed-storage distributed-systems elasticsearch java object-storage openstack-swift rxjava vertx

Last synced: 23 Oct 2025

https://github.com/oaklight/vesper-raft

A fault-tolerant distributed key-value store using the Raft consensus algorithm.

consensus-algorithm distributed-storage key-value python3 raft

Last synced: 12 Oct 2025

https://github.com/mulbc/gosbench

Distributed S3 benchmarking tool - Replacement of Cosbench

benchmark benchmarking ceph cosbench distributed-storage ocs prometheus-exporter s3

Last synced: 12 Apr 2025

https://github.com/loehnertz/tanglestash

IOTA meets BitTorrent: An algorithm to persist any file onto the tangle of IOTA

dag distributed-storage filesharing iota nodejs storage tangle torrent

Last synced: 23 Apr 2025

https://github.com/loehnertz/Tanglestash

IOTA meets BitTorrent: An algorithm to persist any file onto the tangle of IOTA

dag distributed-storage filesharing iota nodejs storage tangle torrent

Last synced: 02 Apr 2025

https://github.com/jd-opensource/dtc

High performance Distributed Table Cache system designed by JD.com, Offering hotspot data cache from databases in order to reduce pressure of database and improve QPS.

cache cache-storage distributed-storage distributed-systems storage

Last synced: 21 Aug 2025

https://github.com/skalenetwork/filestorage.js

Filestorage.js is client library for SKALE sidechain decentralized file storage. Used to transfer files between a browser and a SKALE sidechain.

blockchain distributed-storage filestorage javascript-library skale-chains

Last synced: 26 Jun 2025

https://github.com/uxff/flexdrive

A flexible extending drive, distributed cloud storage system using gRPC and distributed election algorithm.

distributed-storage election-algorithm golang grpc

Last synced: 17 Jan 2026

https://github.com/flowerinthenight/hedge

A distributed systems library for Kubernetes deployments built on top of spindle and Cloud Spanner.

cloud-spanner distributed-lock distributed-semaphore distributed-storage distributed-systems gcp golang key-value kubernetes spanner store

Last synced: 25 Apr 2025

https://github.com/rotationalio/honu

Adaptive consistency replication with reinforcement learning for large scale globally distributed storage.

adaptive-consistency distributed-storage distributed-systems gossip-protocol multi-armed-bandits

Last synced: 08 Apr 2025

https://github.com/cycneuramus/seaweedfs-docker-swarm

Performant, highly available distributed storage using SeaweedFS in Docker Swarm

distributed-storage docker docker-swarm persistent-storage seaweedfs swarm

Last synced: 08 Sep 2025

https://github.com/superjmn/seaweedfs-gui

A graphical user interface for Seaweed File System so you can manage your files easily.

avalonia avaloniaui distributed-storage file filesystem gui manager seaweedfs seaweedfs-client

Last synced: 26 Oct 2025

https://github.com/bsc-dom/dataclay

Python distributed data store that enables remotely access and method execution.

active-storage distributed-computing distributed-database distributed-storage parallel-computing python

Last synced: 14 Jul 2025

https://github.com/codenoid/Fido

E2E Solution for Distributed Cold-Data Storage

distributed-storage distributed-systems

Last synced: 13 Apr 2025

https://github.com/codenoid/fido

E2E Solution for Distributed Cold-Data Storage

distributed-storage distributed-systems

Last synced: 04 Aug 2025

https://github.com/arcana-network/storage

Arcana Storage SDK offers data privacy and enables dApp users to control any and all accesses to their data uploaded in the Arcana Store. Data owners can share, revoke, delete or transfer ownership to other users powered by blockchain.

data-privacy decentralized-storage distributed-storage private-nft web3

Last synced: 12 Jan 2026

https://github.com/saorsa-labs/communitas

Communitas: A decentralized collaboration platform built on the Saorsa P2P ecosystem. Features four-word addresses, end-to-end encryption, and distributed storage

collaboration decentralized desktop-app distributed-storage encryption four-word-addresses p2p rust tauri

Last synced: 08 May 2026

https://github.com/baggepinnen/lazywavfiles.jl

Lazily treat wav (audio) files as arrays. Arrays can be distributed over many wav files.

audio distributed distributed-storage lazy-loading wav

Last synced: 02 Jan 2026

https://github.com/teng231/dumbcache

Simple cache list, count data using redis for microservice.

cache-control distributed-storage golang redis

Last synced: 19 Jan 2026

https://github.com/sile/plumtuna

An implementation of Optuna storage layer based on Plumtree

distributed-storage optuna plumtree rust

Last synced: 12 Jul 2025

https://github.com/mrcolorr/supreme-pancake

Big Data Management project: The collection of data from a network of sensors was simulated (kafka), which then had to be processed (spark) and stored (cassandraDB) in a distributed and efficient way.

big-data bigdata cassandra cassandra-cluster cassandra-database cloud cloud-computing distributed-computing distributed-database distributed-storage distributed-systems hdfs kafka maven maven-pom spark zerotier zerotier-network zerotier-one

Last synced: 10 Aug 2025

https://github.com/akvize/reconcile-rs

A reconciliation service to sync a key-value map over multiple instances.

distributed-storage key-value-store reconciliation rust rust-lang sync

Last synced: 09 Apr 2025

https://github.com/ji-podhead/network-guides

in this series i will cover different network topics like DNS, DHCP, SSL (certmanager + letsencrypt). The guides derive from my personal research, work and will also be available on medium as blogs.

bind9 ceph dhcp distributed-storage dns dns-spoofing dynamic-dns firewall iphelper leases opnsense proxmox routing sharing subnetting zfs

Last synced: 19 Mar 2026

https://github.com/codiepp/elykseer-base

cryptographic data archive; written in F#; envisaged to stay another 10 years

archive cli cryptography data distributed-storage dotnet fsharp longterm-storage

Last synced: 19 May 2026

https://github.com/meysam81/distributed-storage

a partitioning of the key-value storages, quering the server

distributed-storage erlang erlang-developement erlang-otp key-value-database key-value-store

Last synced: 03 Jan 2026

https://github.com/alicepagano/mapd-b-project-distributed-computing-and-data-management

Final project of the course Management and Analysis of Physics Dataset - mod B

covid-19 distributed-computing distributed-storage

Last synced: 16 Mar 2025

https://github.com/murilobsd/docsdocs

:orange_book: Sharing Documents :lock:

distributed-storage networking p2p

Last synced: 30 Jul 2025

https://github.com/Vladimir-Novick/NetLogSystem

Log System to Distributed Computing Systems . DotNet Core Microservices

distributed-storage dotnet-core logger microservice mvc mvc-architecture networking web-application

Last synced: 04 May 2025

https://github.com/predatorray/candybox

CandyBox is a distributed object store for massive small files.

distributed-storage java

Last synced: 01 Mar 2026

https://github.com/hqbobo/dcache

distribute cache base on redis

distributed-storage golang redis

Last synced: 11 Jan 2026

https://github.com/aholstenson/dwaal

Distributed key-value storage for Node

crdt distributed-storage key-value persistence storage yjs

Last synced: 25 Jul 2025

https://github.com/reddec/vault

dummy distributed master-less object storage

backup-utility cli distributed-storage go golang masterless tools

Last synced: 25 Jul 2025

https://github.com/mrifat/d_boutique

A distributed in memory data store

distributed-storage educational-project elixir

Last synced: 17 Jun 2025

https://github.com/pumpkinseed/locker

A distributed lock service client for etcd

distributed-lock distributed-storage distributed-systems

Last synced: 14 Apr 2025

https://github.com/makcanca/textfiles

Textfiles from Golden Age of the internet. Uses IPFS distributed storage

bbs distributed-storage flutter ipfs

Last synced: 17 Apr 2026

https://github.com/JamieOw119/DistributedObjectStorage

A decentralized distributed object storage file system implemented by Golang

distributed-file-system distributed-storage object-storage

Last synced: 11 Jan 2026

https://github.com/aimanamri/raspberry-pi4-hadoop-spark-cluster

This is a self-documentation of learning distributed data storage, parallel processing, and Linux OS using Apache Hadoop, Apache Spark and Raspbian OS. In this project, 3-node cluster will be setup using Raspberry Pi 4, install HDFS and run Spark processing jobs via YARN.

big-data distributed-storage hadoop-cluster hdfs parallel-processing pyspark raspberry-pi-4 spark-cluster spark-shell yarn

Last synced: 21 Jan 2026

https://github.com/jguer/ist-clippy

Computer Networks and the Internet 17-18. Clippy is a fault resistant distributed clipboard. A public library is available in library/clipboard.h for usage in apps

clipboard distributed-storage

Last synced: 18 Mar 2025

https://github.com/guilherme-marcello/node-db

An efficient C-based key-value distributed storage service with data serialization and replication capabilities, inspired by Amazon's web service infrastructure.

amazon c distributed-database distributed-storage distributed-systems key-value serialization

Last synced: 13 Jun 2026

https://github.com/drostlab/hitparsr

Distributed Databasing of Alignment Hit Data

distributed-database distributed-file-system distributed-storage

Last synced: 05 Mar 2026

https://github.com/bkawk/ipfs-upload

Polymer2 Web Component to upload a file to IPFS

distributed-storage ipfs polymer2

Last synced: 05 Oct 2025

https://github.com/markoelez/basicfs

Simple distributed key-value store for small files

distributed-storage distributed-systems

Last synced: 17 Jan 2026

https://github.com/cqllum/cloudstore

Flask-based multi-cloud file storage system with automatic replication across Local, Mega, Dropbox, AWS S3, and Google Cloud. Features web UI, user management, and RESTful API.

aws distributed-storage distributed-storage-system distributed-systems dropbox flask gcp mega python s3 storage whitelabel

Last synced: 09 May 2026

https://github.com/akurilov/chuchundra

Storage mock for mongoose performance testing purposes

akka cluster distributed-storage mock-server s3 sbt scala swift

Last synced: 15 Apr 2026

https://github.com/storacha/storacha.ai

Source code for the storacha.ai microsite

ai distributed-storage

Last synced: 09 Oct 2025