{"id":37091549,"url":"https://github.com/addityasingh/pickbox","last_synced_at":"2026-01-14T11:08:20.393Z","repository":{"id":300836093,"uuid":"1007336553","full_name":"addityasingh/pickbox","owner":"addityasingh","description":"A distributed storage system implemented in Go that provides file operations with replication and consistency guarantees.","archived":false,"fork":false,"pushed_at":"2025-07-12T10:35:12.000Z","size":288,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-12T11:46:11.351Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/addityasingh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-23T20:48:49.000Z","updated_at":"2025-07-12T10:32:12.000Z","dependencies_parsed_at":"2025-07-12T11:35:59.117Z","dependency_job_id":"d6bb1855-fad0-4302-b066-69b4198f21c5","html_url":"https://github.com/addityasingh/pickbox","commit_stats":null,"previous_names":["addityasingh/pickbox"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/addityasingh/pickbox","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/addityasingh%2Fpickbox","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/addityasingh%2Fpickbox/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/addityasingh%2Fpickbox/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/addityasingh%2Fpickbox/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/addityasingh","download_url":"https://codeload.github.com/addityasingh/pickbox/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/addityasingh%2Fpickbox/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28417917,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T10:47:48.104Z","status":"ssl_error","status_checked_at":"2026-01-14T10:46:19.031Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-14T11:08:19.868Z","updated_at":"2026-01-14T11:08:20.383Z","avatar_url":"https://github.com/addityasingh.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Pickbox - Distributed Storage System\n\n[![Pickbox CI/CD](https://github.com/addityasingh/pickbox/actions/workflows/go.yml/badge.svg)](https://github.com/addityasingh/pickbox/actions/workflows/go.yml)\n[![Go Report Card](https://goreportcard.com/badge/github.com/addityasingh/pickbox)](https://goreportcard.com/report/github.com/addityasingh/pickbox)\n[![codecov](https://codecov.io/gh/addityasingh/pickbox/branch/main/graph/badge.svg)](https://codecov.io/gh/addityasingh/pickbox)\n[![Go Version](https://img.shields.io/github/go-mod/go-version/addityasingh/pickbox)](https://golang.org/dl/)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n\nPickbox is a distributed storage system implemented in Go that provides file operations with replication and consistency guarantees.\n\n## Features\n\n- File operations (OPEN, READ, WRITE, CLOSE)\n- Distributed storage with multiple nodes\n- Chunk-based storage with replication\n- Vector clock-based conflict resolution\n- Concurrent request handling\n- Structured logging\n\n## Multi-Directional Replication Architecture\n\nThe current implementation (Step 3) provides advanced multi-directional file replication where any node can initiate changes that automatically propagate to all other nodes while maintaining strong consistency through Raft consensus.\n\n```mermaid\ngraph TB\n    subgraph \"Pickbox Multi-Directional Distributed Storage System\"\n        subgraph \"Node 1 (Leader)\"\n            N1[Node 1\u003cbr/\u003e127.0.0.1:8001]\n            FW1[File Watcher\u003cbr/\u003efsnotify]\n            FSM1[Enhanced FSM\u003cbr/\u003eContent Hash\u003cbr/\u003eDeduplication]\n            RF1[Raft Instance\u003cbr/\u003eLeader]\n            FS1[Local Storage\u003cbr/\u003edata/node1/]\n            ADM1[Admin Server\u003cbr/\u003e:9001\u003cbr/\u003eFORWARD Support]\n            STATE1[File State\u003cbr/\u003eSHA-256 Tracking]\n            \n            N1 --\u003e FW1\n            N1 --\u003e FSM1\n            N1 --\u003e ADM1\n            FW1 --\u003e|\"Detect Changes\"| FSM1\n            FSM1 --\u003e RF1\n            FSM1 --\u003e FS1\n            FSM1 \u003c--\u003e STATE1\n            FSM1 -.-\u003e|\"Pause During Apply\"| FW1\n        end\n        \n        subgraph \"Node 2 (Follower + Watcher)\"\n            N2[Node 2\u003cbr/\u003e127.0.0.1:8002]\n            FW2[File Watcher\u003cbr/\u003efsnotify]\n            FSM2[Enhanced FSM\u003cbr/\u003eContent Hash\u003cbr/\u003eDeduplication]\n            RF2[Raft Instance\u003cbr/\u003eFollower]\n            FS2[Local Storage\u003cbr/\u003edata/node2/]\n            ADM2[Admin Server\u003cbr/\u003e:9002\u003cbr/\u003eFORWARD Support]\n            STATE2[File State\u003cbr/\u003eSHA-256 Tracking]\n            \n            N2 --\u003e FW2\n            N2 --\u003e FSM2\n            N2 --\u003e ADM2\n            FW2 --\u003e|\"Detect Changes\"| FSM2\n            FSM2 --\u003e RF2\n            FSM2 --\u003e FS2\n            FSM2 \u003c--\u003e STATE2\n            FSM2 -.-\u003e|\"Pause During Apply\"| FW2\n        end\n        \n        subgraph \"Node 3 (Follower + Watcher)\"\n            N3[Node 3\u003cbr/\u003e127.0.0.1:8003]\n            FW3[File Watcher\u003cbr/\u003efsnotify]\n            FSM3[Enhanced FSM\u003cbr/\u003eContent Hash\u003cbr/\u003eDeduplication]\n            RF3[Raft Instance\u003cbr/\u003eFollower]\n            FS3[Local Storage\u003cbr/\u003edata/node3/]\n            ADM3[Admin Server\u003cbr/\u003e:9003\u003cbr/\u003eFORWARD Support]\n            STATE3[File State\u003cbr/\u003eSHA-256 Tracking]\n            \n            N3 --\u003e FW3\n            N3 --\u003e FSM3\n            N3 --\u003e ADM3\n            FW3 --\u003e|\"Detect Changes\"| FSM3\n            FSM3 --\u003e RF3\n            FSM3 --\u003e FS3\n            FSM3 \u003c--\u003e STATE3\n            FSM3 -.-\u003e|\"Pause During Apply\"| FW3\n        end\n        \n        subgraph \"Users \u0026 Applications\"\n            USER1[User/App A\u003cbr/\u003eEdits Node 1]\n            USER2[User/App B\u003cbr/\u003eEdits Node 2]\n            USER3[User/App C\u003cbr/\u003eEdits Node 3]\n            CLI[Admin CLI\u003cbr/\u003eCluster Mgmt]\n        end\n        \n        %% User Interactions\n        USER1 --\u003e|\"Create/Edit/Delete Files\"| FS1\n        USER2 --\u003e|\"Create/Edit/Delete Files\"| FS2\n        USER3 --\u003e|\"Create/Edit/Delete Files\"| FS3\n        CLI --\u003e ADM1\n        CLI --\u003e ADM2\n        CLI --\u003e ADM3\n        \n        %% Multi-Directional Replication Flow\n        %% Leader Direct Processing\n        FSM1 --\u003e|\"Direct Apply (Leader)\"| RF1\n        \n        %% Follower Forwarding to Leader\n        FSM2 --\u003e|\"TCP FORWARD Command\"| ADM1\n        FSM3 --\u003e|\"TCP FORWARD Command\"| ADM1\n        \n        %% Raft Consensus Distribution\n        RF1 --\u003e|\"Log Replication\"| RF2\n        RF1 --\u003e|\"Log Replication\"| RF3\n        RF2 -.-\u003e|\"Heartbeats/Votes\"| RF1\n        RF3 -.-\u003e|\"Heartbeats/Votes\"| RF1\n        \n        %% Apply Commands to All FSMs\n        RF1 --\u003e|\"Apply Log Entry\"| FSM1\n        RF1 --\u003e|\"Apply Log Entry\"| FSM2\n        RF1 --\u003e|\"Apply Log Entry\"| FSM3\n        \n        %% Smart File System Updates\n        FSM1 --\u003e|\"Hash-Verified Write\"| FS1\n        FSM2 --\u003e|\"Hash-Verified Write\"| FS2\n        FSM3 --\u003e|\"Hash-Verified Write\"| FS3\n        \n        %% File System Event Detection\n        FS1 -.-\u003e|\"inotify Events\"| FW1\n        FS2 -.-\u003e|\"inotify Events\"| FW2\n        FS3 -.-\u003e|\"inotify Events\"| FW3\n        \n        %% Result: Synchronized State\n        FS1 -.-\u003e|\"Identical Content\"| FS2\n        FS2 -.-\u003e|\"Identical Content\"| FS3\n        FS3 -.-\u003e|\"Identical Content\"| FS1\n        \n        %% Key Features Callouts\n        subgraph \"Key Features\"\n            FEAT1[✅ Any Node → All Nodes]\n            FEAT2[✅ Strong Consistency]\n            FEAT3[✅ Content Deduplication]\n            FEAT4[✅ Real-time Sync]\n            FEAT5[✅ Fault Tolerant]\n            FEAT6[✅ Concurrent Users]\n        end\n    end\n```\n\n### Key Architecture Features:\n\n- **🔄 Multi-Directional Replication**: Any node can initiate file changes that replicate to all others\n- **🛡️ Strong Consistency**: Raft consensus ensures all nodes maintain identical state\n- **⚡ Real-time Synchronization**: File changes detected and replicated within 1-4 seconds\n- **🔍 Content Deduplication**: SHA-256 hashing prevents infinite replication loops\n- **👥 Concurrent Users**: Multiple users can edit files simultaneously on different nodes\n- **🚀 High Performance**: Sub-second change detection with efficient consensus protocol\n\n## Project Structure\n\n```\n.\n├── cmd/                          # Application entry points\n│   ├── replication/             # Step 1: Basic Raft replication\n│   └── multi_replication/       # Multi-directional replication\n├── pkg/\n│   └── storage/\n│       ├── manager.go           # Storage manager implementation\n│       ├── raft_manager.go      # Raft consensus implementation\n│       └── raft_test.go         # Raft tests\n├── scripts/                     # Automation scripts\n│   ├── tests/                   # Test scripts\n│   │   ├── test_replication.sh\n│   │   └── test_multi_replication.sh\n│   ├── run_replication.sh       # Demo scripts\n│   ├── run_multi_replication.sh\n│   ├── cleanup_replication.sh   # Utility scripts\n│   └── add_nodes.go\n├── .cursor/debug/               # Architecture documentation\n│   ├── step1_basic_raft_replication.md\n\n│   ├── step3_multi_directional_replication.md\n│   └── architecture_evolution_overview.md\n├── go.mod                       # Go module definition\n├── go.sum                       # Go module checksums\n└── README.md                    # This file\n```\n\n## Building and Running\n\n### Prerequisites\n- Go 1.21 or later\n- Git for cloning the repository\n\n### Quick Start (Generic N-Node Clusters)\n\n1. **Clone the repository**:\n   ```bash\n   git clone \u003crepository-url\u003e\n   cd pickbox\n   ```\n\n2. **Setup development environment** (optional but recommended):\n   ```bash\n   make setup  # Install tools and pre-commit hooks\n   ```\n\n3. **Start a cluster (any size)**:\n   ```bash\n   # 3-node cluster (backward compatible)\n   ./scripts/cluster_manager.sh start -n 3\n   \n   # 5-node cluster\n   ./scripts/cluster_manager.sh start -n 5\n   \n   # 7-node cluster with custom ports\n   ./scripts/cluster_manager.sh start -n 7 -p 9000 -a 10000\n   \n   # Use configuration file\n   ./scripts/cluster_manager.sh start -c examples/cluster-configs/5-node-cluster.conf\n   ```\n\n4. **Test the system**:\n   ```bash\n   # Create files on any node - they replicate everywhere!\n   echo \"Hello from node1!\" \u003e data/node1/test1.txt\n   echo \"Hello from node2!\" \u003e data/node2/test2.txt\n   echo \"Hello from node3!\" \u003e data/node3/test3.txt\n   \n   # Verify replication (all nodes should have all files)\n   ls data/node*/\n   ```\n\n5. **Run comprehensive tests**:\n   ```bash\n   # Test specific cluster size\n   ./scripts/tests/test_n_replication.sh -n 5\n   \n   # Test with original scripts (3-node)\n   ./scripts/tests/test_multi_replication.sh\n   ```\n\n**Port Assignment Schema** (for N nodes starting at BASE_PORT=8001):\n- **node1**: Raft=8001, Admin=9001, Monitor=6001\n- **node2**: Raft=8002, Admin=9002, Monitor=6002  \n- **nodeN**: Raft=800N, Admin=900N, Monitor=600N\n- **Dashboard**: 8080 (shared across all nodes)\n\n## Cluster Management (N-Node Support)\n\nPickbox now supports **generic N-node clusters** with flexible configuration. You can run anywhere from 1 to 20+ nodes with automatic port assignment and cluster management.\n\n### Generic Cluster Manager\n\nThe new `cluster_manager.sh` provides comprehensive cluster lifecycle management:\n\n```bash\n# Start clusters of any size\n./scripts/cluster_manager.sh start -n 5                    # 5-node cluster\n./scripts/cluster_manager.sh start -n 10 -p 18000          # 10-node with high ports\n\n# Manage cluster lifecycle\n./scripts/cluster_manager.sh status -n 5                   # Check status\n./scripts/cluster_manager.sh logs -n 5                     # View logs\n./scripts/cluster_manager.sh restart -n 5                  # Restart cluster\n./scripts/cluster_manager.sh clean                         # Clean everything\n\n# Use configuration files\n./scripts/cluster_manager.sh start -c examples/cluster-configs/10-node-high-ports.conf\n```\n\n### Configuration Files\n\nPre-built configurations for common scenarios:\n\n- **`examples/cluster-configs/5-node-cluster.conf`** - Standard 5-node setup\n- **`examples/cluster-configs/7-node-cluster.conf`** - 7-node cluster  \n- **`examples/cluster-configs/10-node-high-ports.conf`** - 10-node with high ports\n\n**Example configuration:**\n```bash\nNODE_COUNT=5\nBASE_PORT=8001\nADMIN_BASE_PORT=9001\nMONITOR_BASE_PORT=6001\nDASHBOARD_PORT=8080\nHOST=127.0.0.1\nDATA_DIR=data\nBINARY=cmd/multi_replication/main.go\n```\n\n### Advanced Usage\n\n```bash\n# Multi-environment clusters\n./scripts/cluster_manager.sh start -n 3 -p 8001            # Development  \n./scripts/cluster_manager.sh start -n 5 -p 12001 --data-dir staging  # Staging\n./scripts/cluster_manager.sh start -n 7 -p 18001 --data-dir prod     # Production\n\n# Dynamic expansion\n./scripts/cluster_manager.sh start -n 3                    # Start with 3 nodes\ngo run scripts/add_nodes.go -nodes 2 -start 4             # Add node4, node5\n\n# Generic testing\n./scripts/tests/test_n_replication.sh -n 5 -v             # Test 5-node cluster\n./scripts/tests/test_n_replication.sh -n 10 -p 18001      # Test with custom ports\n```\n\n### Backward Compatibility\n\nAll existing 3-node scripts remain functional:\n\n```bash\n# Legacy scripts (still work)\n./scripts/run_multi_replication.sh                        # 3-node cluster\n./scripts/tests/test_multi_replication.sh                 # 3-node tests\n```\n\n## Usage\n\n### File Operations\n\nThe system automatically replicates file operations across all nodes. You can work with files directly through the file system:\n\n**Creating Files**:\n```bash\n# Create a file on any node\necho \"Hello World!\" \u003e data/node1/example.txt\necho \"Content from node2\" \u003e data/node2/another.txt\necho \"Data from node3\" \u003e data/node3/document.txt\n```\n\n**Reading Files**:\n```bash\n# Read files from any node (content is identical across all nodes)\ncat data/node1/example.txt\ncat data/node2/example.txt  # Same content as node1\ncat data/node3/example.txt  # Same content as node1\n```\n\n**Editing Files**:\n```bash\n# Edit files on any node using any editor\necho \"Updated content\" \u003e\u003e data/node2/example.txt\nnano data/node3/document.txt\nvim data/node1/another.txt\n```\n\n**Verifying Replication**:\n```bash\n# Check that all nodes have identical files\nfind data/ -name \"*.txt\" -exec echo \"=== {} ===\" \\; -exec cat {} \\;\n```\n\n### Admin Operations\n\n**Cluster Status**:\n```bash\n# Check cluster status via admin interface\necho \"STATUS\" | nc localhost 9001  # Node 1 admin port\necho \"STATUS\" | nc localhost 9002  # Node 2 admin port  \necho \"STATUS\" | nc localhost 9003  # Node 3 admin port\n```\n\n**Cleanup**:\n```bash\n# Clean up all processes and data\n./scripts/cleanup_replication.sh\n```\n\n## Implementation Details\n\n### Storage System\n\nThe storage system is implemented with the following components:\n\n1. **Storage Manager**: Manages multiple storage nodes and coordinates operations\n2. **Storage Node**: Handles chunk storage and replication\n3. **Vector Clock**: Implements vector clocks for conflict resolution\n\n### Concurrency\n\n- Each client connection is handled in a separate goroutine\n- Storage operations are protected by mutexes for thread safety\n- Vector clock operations are atomic\n\n### Logging\n\nThe system uses structured logging via `logrus` for better observability. Logs include:\n- Server startup and shutdown\n- Client connections and disconnections\n- File operations\n- Storage operations\n- Error conditions\n\n## Testing\n\nPickbox includes a comprehensive test suite covering unit tests, integration tests, and benchmarks. The system provides:\n\n- **Unit Tests**: Storage package, Raft manager, and multi-replication components *(active)*\n- **Integration Tests**: End-to-end 3-node cluster testing *(currently disabled for CI/CD stability)*\n- **Benchmark Tests**: Performance testing for critical operations *(active)*\n- **Test Scripts**: Automated testing for all replication modes *(manual execution only)*\n\n### Quick Test Commands\n\n```bash\n# Run all tests with coverage\n./scripts/run_tests.sh\n\n# Run integration tests\ncd test \u0026\u0026 go test -v .\n\n# Run unit tests\ngo test -v ./pkg/storage ./cmd/multi_replication\n```\n\n### Test Scripts\n\n- `scripts/tests/test_replication.sh` - Basic Raft replication tests\n- `scripts/tests/test_multi_replication.sh` - Multi-directional replication tests\n\n**📖 For comprehensive testing documentation, see [`test/README.md`](test/README.md)**\n\n## Code Quality \u0026 Linting\n\nPickbox enforces strict code quality standards through comprehensive linting and automated checks:\n\n### **Linting Tools**\n- **golangci-lint**: Comprehensive Go linter with 25+ enabled checks\n- **staticcheck**: Advanced static analysis for Go\n- **gosec**: Security vulnerability scanner\n- **pre-commit**: Automated quality checks on every commit\n\n### **Quality Checks**\n- ✅ **Unused Code Detection**: Catches unused variables, functions, and struct fields\n- ✅ **Security Scanning**: Detects potential security vulnerabilities\n- ✅ **Code Formatting**: Enforces consistent formatting with `gofmt` and `goimports`\n- ✅ **Performance Analysis**: Identifies inefficient code patterns\n- ✅ **Style Consistency**: Maintains consistent coding style across the project\n\n### **Development Workflow**\n\n```bash\n# Setup development environment\nmake setup                    # Install tools + pre-commit hooks\n\n# Code quality commands\nmake lint                     # Run all linters\nmake lint-fix                 # Auto-fix issues where possible\nmake check-unused             # Check for unused code specifically\nmake security                 # Run security analysis (go vet + gosec if available)\nmake security-install         # Install gosec and run full security analysis\nmake verify-all               # Run all checks (lint + test + security)\n\n# Pre-commit integration\ngit commit                    # Automatically runs quality checks\nmake pre-commit               # Run pre-commit hooks manually\n```\n\n### **CI Integration**\nAll quality checks run automatically in GitHub Actions:\n- **Pre-commit hooks** prevent bad code from being committed\n- **CI pipeline** runs comprehensive linting on every push/PR\n- **Security scanning** generates SARIF reports for GitHub Security tab\n- **Coverage enforcement** maintains quality thresholds\n\n## CI/CD Pipeline\n\nPickbox uses GitHub Actions for continuous integration and deployment:\n\n### Pipeline Features\n- **Multi-Go Version Testing**: Tests against Go 1.21 and 1.22\n- **Comprehensive Test Suite**: Unit tests, integration tests, and benchmarks\n- **Code Quality Checks**: `go vet`, `staticcheck`, and security scanning\n- **Cross-Platform Builds**: Linux, macOS, and Windows binaries\n- **Coverage Reporting**: Automated coverage reports via Codecov\n- **Security Scanning**: Gosec security analysis\n- **Automated Releases**: Binary releases on main branch pushes\n\n### Pipeline Jobs\n\n1. **Test Suite** (`test`) - Runs unit tests with coverage\n2. **Integration Tests** (`integration-test`) - End-to-end testing *(currently disabled - see Improvements section)*\n3. **Build** (`build`) - Cross-platform binary compilation\n4. **Security** (`security`) - Security vulnerability scanning\n5. **Release** (`release`) - Automated GitHub releases\n6. **Notify** (`notify`) - Pipeline status notifications\n\n### Artifacts Published\n- **Coverage Reports**: HTML and raw coverage data\n- **Binaries**: Cross-platform executables for all three modes\n- **Security Reports**: SARIF format security scan results\n- **Integration Logs**: Debug logs from failed integration tests\n\n### Monitoring\n- **Build Status**: [![Pickbox CI/CD](https://github.com/addityasingh/pickbox/actions/workflows/go.yml/badge.svg)](https://github.com/addityasingh/pickbox/actions/workflows/go.yml)\n- **Code Coverage**: [![codecov](https://codecov.io/gh/addityasingh/pickbox/branch/main/graph/badge.svg)](https://codecov.io/gh/addityasingh/pickbox)\n- **Code Quality**: [![Go Report Card](https://goreportcard.com/badge/github.com/addityasingh/pickbox)](https://goreportcard.com/report/github.com/addityasingh/pickbox)\n\n## Scripts Organization\n\n```\nscripts/\n├── tests/                    # Test scripts\n│   ├── README.md\n│   ├── test_replication.sh\n│   └── test_multi_replication.sh\n├── run_replication.sh        # Demo scripts\n├── run_multi_replication.sh\n├── cleanup_replication.sh    # Utility scripts\n└── add_nodes.go\n```\n\n## Architecture Documentation\n\nComprehensive architecture diagrams and documentation are available in `.cursor/debug/`:\n\n- **Step 1**: `step1_basic_raft_replication.md` - Basic Raft consensus replication\n  \n- **Step 3**: `step3_multi_directional_replication.md` - Multi-directional replication\n- **Overview**: `architecture_evolution_overview.md` - Complete evolution analysis\n\nEach document includes detailed Mermaid diagrams showing:\n- Node architecture and communication patterns\n- Data flow and command processing\n- Component relationships and dependencies\n- Evolution from basic consensus to advanced multi-directional replication\n\n## Improvements \n- [ ] Refactor code to be more readable\n- [x] Add tests for golang files\n- [x] Refactor test bash scripts from scripts folder\n- [x] Generate architecture diagram for each of the 3 versions (replication, multi_replication)\n- [x] Set up comprehensive CI/CD pipeline with GitHub Actions\n- [x] Add comprehensive linting with pre-commit hooks and unused field detection\n- [ ] Stabilize integration tests for reliable CI/CD execution (currently all disabled due to timing/resource issues)\n- [ ] Deploy and create client code for this setup to test end-to-end\n- [x] Make it a generalized solution for N nodes instead of hardcoded 3 nodes\n- [ ] Understand the RaftFSM\n\n## License\n\nMIT License\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faddityasingh%2Fpickbox","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faddityasingh%2Fpickbox","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faddityasingh%2Fpickbox/lists"}