Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

Projects in Awesome Lists tagged with distributed-storage

A curated list of projects in awesome lists tagged with distributed-storage .

https://github.com/seaweedfs/seaweedfs

SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.

blob-storage cloud-drive distributed-file-system distributed-storage distributed-systems erasure-coding fuse hadoop-hdfs hdfs kubernetes object-storage posix replication s3 s3-storage seaweedfs tiered-file-system

Last synced: 16 Dec 2024

https://github.com/baidu/braft

An industrial-grade C++ implementation of RAFT consensus algorithm based on brpc, widely used inside Baidu to build highly-available distributed systems.

distributed-consensus distributed-storage raft raft-consensus-algorithm raft-cpp raft-cxx raft-protocol state-machine-replication

Last synced: 17 Dec 2024

https://github.com/storj/storj

Ongoing Storj v3 development. Decentralized cloud object storage that is affordable, easy to use, private, and secure.

distributed distributed-storage distributed-systems golang object-storage open-source storage

Last synced: 17 Dec 2024

https://github.com/zuoyebang/bitalostored

Bitalostored is a high-performance distributed storage system, core engine based on bitalosdb(self-developed), compatible with Redis protocol.

bitalosdb database distributed-storage high-performance kvstore nosql redis storage-engine

Last synced: 19 Dec 2024

https://github.com/pravega/pravega

Pravega - Streaming as a new software defined storage primitive

data-ingestion distributed-storage real-time-data streaming streaming-data

Last synced: 17 Dec 2024

https://github.com/moosefs/moosefs

MooseFS Distributed Storage – Open Source, Petabyte, Fault-Tolerant, Highly Performing, Scalable Network Distributed File System / Software-Defined Storage

backup block-storage distributed-computing distributed-file-storage distributed-file-system distributed-storage ditributed-systems erasure-coding file-storage file-system filesystem fuse high-availability media posix posix-compliant snapshots software-defined-storage storage tiering

Last synced: 19 Dec 2024

https://github.com/eclipse-zenoh/zenoh

zenoh unifies data in motion, data in-use, data at rest and computations. It carefully blends traditional pub/sub with geo-distributed storages, queries and computations, while retaining a level of time and space efficiency that is well beyond any of the mainstream stacks.

distributed-computing distributed-storage distributed-systems edge-computing embedded geo-distributed-storages iot messaging network-programming networking protocol robotics ros2 rust rust-lang storage zenoh

Last synced: 17 Dec 2024

https://github.com/douban/gobeansdb

Distributed object storage server from Douban Inc.

distributed-storage kvstore memcached object-storage

Last synced: 21 Dec 2024

https://github.com/archivebox/good-karma-kit

😇 A Docker Compose bundle to run on servers with spare CPU, RAM, disk, and bandwidth to help the world. Includes Tor, ArchiveWarrior, BOINC, and more...

archivebox archivewarrior boinc distributed-computing distributed-storage docker docker-compose foldingathome good-karma i2p internet-archiving ipfs kiwix pywb sia storj tor zimfarm

Last synced: 21 Dec 2024

https://github.com/ArchiveBox/good-karma-kit

😇 A Docker Compose bundle to run on servers with spare CPU, RAM, disk, and bandwidth to help the world. Includes Tor, ArchiveWarrior, BOINC, and more...

archivebox archivewarrior boinc distributed-computing distributed-storage docker docker-compose foldingathome good-karma i2p internet-archiving ipfs kiwix pywb sia storj tor zimfarm

Last synced: 06 Nov 2024

https://github.com/chrislusf/vasto

A distributed key-value store. On Disk. Able to grow or shrink without service interruption.

distributed-storage key-value replication rocksdb

Last synced: 18 Nov 2024

https://github.com/matrixorigin/matrixcube

Fundamental Building Block for Elastic Storage With Strong Consistency and Reliability

cloud-native consensus distributed-storage distributed-systems go paxos raft

Last synced: 11 Nov 2024

https://github.com/zuoyebang/bitalosdb

Bitalosdb is a high-performance KV storage engine.

database distributed-storage high-performance io-architecture kvstore nosql storage-engine

Last synced: 21 Dec 2024

https://github.com/SUSE/DeepSea

A collection of Salt files for deploying, managing and automating Ceph.

ceph deployment-system distributed-storage python salt software-defined-storage

Last synced: 26 Nov 2024

https://github.com/suse/deepsea

A collection of Salt files for deploying, managing and automating Ceph.

ceph deployment-system distributed-storage python salt software-defined-storage

Last synced: 14 Dec 2024

https://github.com/freakmaxi/kertish-dfs

Kertish-dfs is a simple distributed storage platform, implements file storage on a single distributed computer cluster, and provides interfaces for file/folder handling. Kertish-dfs aims primarily for completely distributed operation without a single point of failure, scalable to the exabyte level.

data-farm dfs distributed-file-storage distributed-file-system distributed-storage scalable

Last synced: 01 Nov 2024

https://github.com/pitchpoint-solutions/sfs

The legacy distributed object storage server developed by PitchPoint Solutions can store billions of large and small files using minimal resources. Object data is stored in replicated volumes implemented like Facebooks Haystack Object Store. Object metadata which essentially maps an object name to a volume position is stored in an elasticsearch index. (Development by PitchPoint Solutions has been discontinued)

distributed-file-system distributed-storage distributed-systems elasticsearch java object-storage openstack-swift rxjava vertx

Last synced: 09 Oct 2024

https://github.com/loehnertz/Tanglestash

IOTA meets BitTorrent: An algorithm to persist any file onto the tangle of IOTA

dag distributed-storage filesharing iota nodejs storage tangle torrent

Last synced: 03 Nov 2024

https://github.com/loehnertz/tanglestash

IOTA meets BitTorrent: An algorithm to persist any file onto the tangle of IOTA

dag distributed-storage filesharing iota nodejs storage tangle torrent

Last synced: 20 Oct 2024

https://github.com/jd-opensource/dtc

High performance Distributed Table Cache system designed by JD.com, Offering hotspot data cache from databases in order to reduce pressure of database and improve QPS.

cache cache-storage distributed-storage distributed-systems storage

Last synced: 20 Dec 2024

https://github.com/flowerinthenight/hedge

A distributed systems library for Kubernetes deployments built on top of spindle and Cloud Spanner.

cloud-spanner distributed-lock distributed-semaphore distributed-storage distributed-systems gcp golang key-value kubernetes spanner store

Last synced: 10 Nov 2024

https://github.com/bsc-dom/dataclay

Python distributed data store that enables remotely access and method execution.

active-storage distributed-computing distributed-database distributed-storage parallel-computing python

Last synced: 23 Nov 2024

https://github.com/codenoid/fido

E2E Solution for Distributed Cold-Data Storage

distributed-storage distributed-systems

Last synced: 19 Oct 2024

https://github.com/codenoid/Fido

E2E Solution for Distributed Cold-Data Storage

distributed-storage distributed-systems

Last synced: 07 Nov 2024

https://github.com/superjmn/seaweedfs-gui

A graphical user interface for Seaweed File System so you can manage your files easily.

avalonia avaloniaui distributed-storage file filesystem gui manager seaweedfs seaweedfs-client

Last synced: 11 Oct 2024

https://github.com/baggepinnen/lazywavfiles.jl

Lazily treat wav (audio) files as arrays. Arrays can be distributed over many wav files.

audio distributed distributed-storage lazy-loading wav

Last synced: 21 Nov 2024

https://github.com/teng231/dumbcache

Simple cache list, count data using redis for microservice.

cache-control distributed-storage golang redis

Last synced: 22 Nov 2024

https://github.com/akvize/reconcile-rs

A reconciliation service to sync a key-value map over multiple instances.

distributed-storage key-value-store reconciliation rust rust-lang sync

Last synced: 12 Oct 2024

https://github.com/mrcolorr/supreme-pancake

Big Data Management project: The collection of data from a network of sensors was simulated (kafka), which then had to be processed (spark) and stored (cassandraDB) in a distributed and efficient way.

big-data bigdata cassandra cassandra-cluster cassandra-database cloud cloud-computing distributed-computing distributed-database distributed-storage distributed-systems hdfs kafka maven maven-pom spark zerotier zerotier-network zerotier-one

Last synced: 13 Nov 2024

https://github.com/sile/plumtuna

An implementation of Optuna storage layer based on Plumtree

distributed-storage optuna plumtree rust

Last synced: 07 Nov 2024

https://github.com/alicepagano/mapd-b-project-distributed-computing-and-data-management

Final project of the course Management and Analysis of Physics Dataset - mod B

covid-19 distributed-computing distributed-storage

Last synced: 22 Nov 2024

https://github.com/datahiv3/storage-layer

Protocol-agnostic, highly redundant storage infrastructure supporting multiple storage protocols and networks, with primary integration for 0G Protocol and IPFS. Provides unified storage interfaces while maintaining redundancy across various storage solutions.

0g-protocol blockchain-storage data-redundancy decentralized-storage distributed-storage ipfs legal-data protocol-agnostic redundant-storage storage-layer

Last synced: 21 Dec 2024

https://github.com/cycneuramus/seaweedfs-docker-swarm

Performant, highly available distributed storage using SeaweedFS in Docker Swarm

distributed-storage docker docker-swarm persistent-storage seaweedfs swarm

Last synced: 09 Nov 2024

https://github.com/Vladimir-Novick/NetLogSystem

Log System to Distributed Computing Systems . DotNet Core Microservices

distributed-storage dotnet-core logger microservice mvc mvc-architecture networking web-application

Last synced: 13 Nov 2024

https://github.com/meysam81/distributed-storage

a partitioning of the key-value storages, quering the server

distributed-storage erlang erlang-developement erlang-otp key-value-database key-value-store

Last synced: 22 Nov 2024

https://github.com/murilobsd/docsdocs

:orange_book: Sharing Documents :lock:

distributed-storage networking p2p

Last synced: 07 Nov 2024

https://github.com/mrifat/d_boutique

A distributed in memory data store

distributed-storage educational-project elixir

Last synced: 18 Dec 2024

https://github.com/pumpkinseed/locker

A distributed lock service client for etcd

distributed-lock distributed-storage distributed-systems

Last synced: 06 Nov 2024

https://github.com/jguer/ist-clippy

Computer Networks and the Internet 17-18. Clippy is a fault resistant distributed clipboard. A public library is available in library/clipboard.h for usage in apps

clipboard distributed-storage

Last synced: 24 Nov 2024

https://github.com/reddec/vault

dummy distributed master-less object storage

backup-utility cli distributed-storage go golang masterless tools

Last synced: 21 Nov 2024

https://github.com/aholstenson/dwaal

Distributed key-value storage for Node

crdt distributed-storage key-value persistence storage yjs

Last synced: 01 Dec 2024

https://github.com/ji-podhead/network-guides

in this series i will cover different network topics like DNS, DHCP, SSL (certmanager + letsencrypt). The guides derive from my personal research, work and will also be available on medium as blogs.

bind9 ceph dhcp distributed-storage dns dns-spoofing dynamic-dns firewall iphelper leases opnsense proxmox routing sharing subnetting zfs

Last synced: 28 Nov 2024

https://github.com/aimanamri/raspberry-pi4-hadoop-spark-cluster

This is a self-documentation of learning distributed data storage, parallel processing, and Linux OS using Apache Hadoop, Apache Spark and Raspbian OS. In this project, 3-node cluster will be setup using Raspberry Pi 4, install HDFS and run Spark processing jobs via YARN.

big-data distributed-storage hadoop-cluster hdfs parallel-processing pyspark raspberry-pi-4 spark-cluster spark-shell yarn

Last synced: 05 Nov 2024

https://github.com/akurilov/chuchundra

Storage mock for mongoose performance testing purposes

akka cluster distributed-storage mock-server s3 sbt scala swift

Last synced: 14 Nov 2024

https://github.com/guilherme-marcello/node-db

An efficient C-based key-value distributed storage service with network communication, data serialization and replication capabilities, inspired by Amazon's web service infrastructure.

amazon c distributed-database distributed-storage distributed-systems key-value serialization

Last synced: 10 Nov 2024

https://github.com/bkawk/ipfs-upload

Polymer2 Web Component to upload a file to IPFS

distributed-storage ipfs polymer2

Last synced: 07 Nov 2024

https://github.com/drostlab/hitparsr

Distributed Databasing of Alignment Hit Data

distributed-database distributed-file-system distributed-storage

Last synced: 12 Nov 2024