{"id":43926511,"url":"https://github.com/markosski/gossipgriddb","last_synced_at":"2026-02-06T23:07:54.859Z","repository":{"id":334484426,"uuid":"1141576719","full_name":"markosski/gossipgriddb","owner":"markosski","description":"GossipGridDB is a distributed Key-Value (KV) database designed for high availability and partition-aware compute ","archived":false,"fork":false,"pushed_at":"2026-02-02T04:00:20.000Z","size":250,"stargazers_count":1,"open_issues_count":8,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-02T11:04:03.595Z","etag":null,"topics":["distributed-database","key-value-database","rust-lang"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/markosski.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2026-01-25T03:10:20.000Z","updated_at":"2026-01-28T04:58:53.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/markosski/gossipgriddb","commit_stats":null,"previous_names":["markosski/gossipgriddb"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/markosski/gossipgriddb","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/markosski%2Fgossipgriddb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/markosski%2Fgossipgriddb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/markosski%2Fgossipgriddb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/markosski%2Fgossipgriddb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/markosski","download_url":"https://codeload.github.com/markosski/gossipgriddb/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/markosski%2Fgossipgriddb/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29179626,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-06T22:12:24.066Z","status":"ssl_error","status_checked_at":"2026-02-06T22:12:09.859Z","response_time":59,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["distributed-database","key-value-database","rust-lang"],"created_at":"2026-02-06T23:07:54.377Z","updated_at":"2026-02-06T23:07:54.854Z","avatar_url":"https://github.com/markosski.png","language":"Rust","readme":"# GossipGridDB 🕸️\n\n[![Rust](https://github.com/markosski/gossipgrid/actions/workflows/rust.yml/badge.svg)](https://github.com/markosski/gossipgrid/actions/workflows/rust.yml)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n\nGossipGridDB is a distributed Key-Value (KV) database designed for high availability and partition-aware compute (WIP). It uses a gossip protocol for cluster membership and a leader/replica model for data consistency.\n\n---\n\n## ⚠️ State of the Project\n*   **Heavily under development**: APIs and storage formats are subject to change.\n*   **Work in progress**: Partition-aware compute and advanced performance optimizations are still being implemented.\n*   **Not production-ready**: Use for experimental and learning purposes only.\n\n## 🚀 Features\n- [x] **Gossip Protocol**: Decentralized cluster membership management.\n- [x] **Leader/Replica Model**: High availability via data replication.\n- [x] **Durability**: Write-Ahead Log (WAL) ensures data is not lost on crash.\n- [x] **Partitioning**: Efficient data distribution for horizontal scaling.\n- [x] **Rich Querying**: Support for partition and range keys.\n- [x] **Embeddable**: Easily integrate into your Rust applications.\n- [x] **Pluggable Storage**: Interfaces for custom storage backends (In-memory provided).\n- [x] **Compute**: Execute Lua scripts on items within partition key range for server-side calculations.\n- [ ] **Partition-Aware Client**: Direct routing to the correct node for lower latency.\n- [ ] **Batch Writes**: Enhanced throughput for write-heavy workloads.\n- [ ] **WAL Compaction**: Reclaim disk space automatically, improved data recovery performance.\n\n## 🏗️ Architecture\n\n```mermaid\ngraph TD\n    subgraph Cluster\n        N1[Node 1: Leader]\n        N2[Node 2: Replica]\n        N3[Node 3: Replica]\n    end\n\n    Client[HTTP Client] --\u003e|Read/Write| N1\n    N1 \u003c--\u003e|Gossip Membership| N2\n    N1 \u003c--\u003e|Gossip Membership| N3\n    N2 \u003c--\u003e|Gossip Membership| N3\n    \n    N2 -.-\u003e|Pull Sync| N1\n    N3 -.-\u003e|Pull Sync| N1\n\n    subgraph \"Internal Node Components\"\n        G[Gossip Protocol]\n        S[Sync Server/Client]\n        W[WAL]\n        ST[Store]\n    end\n```\n\n## 📈 Performance Benchmarks\n*Tested on a 3-node cluster, replication=2, partitions=9, c6a.large EC2 instances (EBS GP3 3000 IOPS), test duration - 10s.*\n\n| Operation | Total Ops | Avg Latency | p90 | p95 |\n| :--- | :--- | :--- | :--- | :--- |\n| **Write (Replica Acknowledged)** | 9,409 | 3.07ms | 4.59ms | 4.94ms |\n| **Write (Eventual)** | 36,292 | 686.55µs | 969.7µs | 1.26ms |\n| **Read** | 56,416 | 433.91µs | 705.5µs | 810.91µs |\n\n## 🛠️ Getting Started\n\n### 1. Start a Local Cluster\nYou can spin up a 3-node cluster locally using the provided scripts:\n\n```bash\n# Start an ephemeral cluster\n./start_local_cluster_3n.sh\n\n# Or a named cluster (persists state)\n./start_local_cluster_3n_named.sh\n```\n\n### 2. Interact with the API\nInteract with the cluster using `curl`:\n\n**Store an item:**\n```bash\ncurl -H \"Content-Type: application/json\" -XPOST http://127.0.0.1:3001/items -d '{\"partition_key\": \"user_123\", \"message\": \"JSON payload\"}'\n```\n\n**Store items with range key:**\n```bash\ncurl -H \"Content-Type: application/json\" -XPOST http://127.0.0.1:3001/items -d '{\"partition_key\": \"user_123\", \"range_key\": \"transaction_10\", \"message\": \"{\\\"amount\\\": 100}\"}'\ncurl -H \"Content-Type: application/json\" -XPOST http://127.0.0.1:3001/items -d '{\"partition_key\": \"user_123\", \"range_key\": \"transaction_11\", \"message\": \"{\\\"amount\\\": 50}\"}'\n```\n\n**Retrieve all items by partition key:**\n```bash\ncurl -XGET http://127.0.0.1:3001/items/user_123\n```\n\n**Retrieve a specific item with range key:**\n```bash\ncurl -XGET http://127.0.0.1:3001/items/user_123/transaction_10\n```\n\n**Register and execute a compute function:**\n```bash\n# Register a function at node start, see start_local_cluster_3n.sh\n\n# Execute function on items\ncurl -XGET \"http://127.0.0.1:3001/items/user_123?fn=sum_amounts\"\n\n# List registered functions\ncurl -XGET http://127.0.0.1:3001/functions\n```\n\n## 🧪 Testing\n\n```bash\n# Unit tests\ncargo test --lib\n\n# Integration tests (requires single-threaded execution currently)\nRUST_LOG=info cargo test --test 'int_tests_*' -- --test-threads=4 --nocapture\n```\n\n## 📚 Documentation\nGenerate and open the rustdoc documentation:\n```bash\ncargo doc --no-deps --package gossipgrid --open\n```\n\n## 📄 License\nLicensed under [Apache-2.0](http://www.apache.org/licenses/LICENSE-2.0).\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarkosski%2Fgossipgriddb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarkosski%2Fgossipgriddb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarkosski%2Fgossipgriddb/lists"}