{"id":29866596,"url":"https://github.com/namitkewat/statedb","last_synced_at":"2026-05-04T22:32:33.478Z","repository":{"id":306174115,"uuid":"1019519443","full_name":"namitkewat/statedb","owner":"namitkewat","description":"A high-performance, in-memory, Redis-like key-value database built from scratch in the Zig programming language","archived":false,"fork":false,"pushed_at":"2025-07-24T01:52:39.000Z","size":44,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-24T04:47:51.357Z","etag":null,"topics":["database","in-memory","key-value","redis","redis-like","resp","systems-programming","zig","ziglang"],"latest_commit_sha":null,"homepage":"","language":"Zig","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/namitkewat.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-14T13:01:07.000Z","updated_at":"2025-07-24T01:52:15.000Z","dependencies_parsed_at":"2025-07-30T19:01:51.814Z","dependency_job_id":null,"html_url":"https://github.com/namitkewat/statedb","commit_stats":null,"previous_names":["namitkewat/statedb"],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/namitkewat/statedb","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/namitkewat%2Fstatedb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/namitkewat%2Fstatedb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/namitkewat%2Fstatedb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/namitkewat%2Fstatedb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/namitkewat","download_url":"https://codeload.github.com/namitkewat/statedb/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/namitkewat%2Fstatedb/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32628205,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-04T10:08:07.713Z","status":"ssl_error","status_checked_at":"2026-05-04T10:08:02.005Z","response_time":58,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","in-memory","key-value","redis","redis-like","resp","systems-programming","zig","ziglang"],"created_at":"2025-07-30T13:00:55.410Z","updated_at":"2026-05-04T22:32:33.463Z","avatar_url":"https://github.com/namitkewat.png","language":"Zig","funding_links":[],"categories":[],"sub_categories":[],"readme":"# StateDB: A High-Performance, In-Memory, Redis-like Database in Zig\r\n\r\n## Project Overview and Goals\r\n\r\nStateDB is an in-memory, key-value database built from the ground up in Zig. It aims to be functionally similar to established databases like Redis while serving as a deep, practical exploration into modern systems programming.\r\n\r\nThe core hypothesis is that **Zig provides a superior balance of performance, simplicity, and safety for developing high-performance network services** compared to alternatives like C/C++ (which often lack memory safety) and Rust (which has a steeper learning curve).\r\n\r\n### Key Objectives:\r\n* **Implement a Minimalist, High-Performance Server:** Design and build a lean, low-latency database server leveraging Zig's direct control over memory and its compile-time features.\r\n* **Develop a Robust RESP Parser:** Create a parser for a significant subset of Redis-compatible commands, using Zig's strong type system and explicit error handling to ensure safety and correctness.\r\n* **Master Low-Level Systems Programming:** Gain practical experience with manual memory management, using different allocators (e.g., `ArenaAllocator` for request-scoped allocations) to optimize performance and prevent memory fragmentation.\r\n* **Utilize the Zig Ecosystem:** Become proficient with Zig's standard library for networking (`std.net`) and I/O, and master the Zig Build System for dependency management and cross-compilation.\r\n* **Create a Testable and Benchmarkable System:** Establish a clear testing protocol to measure key performance indicators like requests per second (RPS), latency, and memory footprint.\r\n\r\n## System Design\r\n\r\n```mermaid\r\n%%{init: {\r\n  'theme': 'neutral', \r\n  'look': 'handDrawn', \r\n  'themeVariables': {\r\n    'primaryColor': '#F2F2F2', \r\n    'primaryTextColor': '#1E1E1E', \r\n    'lineColor': '#4A4A4A', \r\n    'secondaryColor': '#E0E0E0', \r\n    'tertiaryColor': '#F8F8F8'\r\n}}}%%\r\ngraph TB\r\n\r\n    subgraph Client\r\n        A[Client Request \u003cbr\u003ee.g. redis-py or netcat]\r\n    end\r\n    \r\n    subgraph StateDBServer\r\n        B[\"1\\. TCP Listener\u003cbr\u003e(std.net.StreamServer)\"]\r\n        A --\u003e |Sends request| B\r\n\r\n        subgraph ClientHandler-Thread\r\n          style ClientHandler-Thread fill:#e6f3ff,stroke:#0066cc,stroke-width:2px,stroke-dasharray: 5 5\r\n          \r\n          topLabel2[\"One Thread\u003cbr\u003eper connection\"]\r\n\r\n          C{\"2\\. Read Loop\"} -- \"Raw Byte Stream\" --\u003e D[\"3\\. Command Parser \u003cbr\u003e (RESP Decoder)\"]\r\n          \r\n          subgraph CriticalSection\r\n                style CriticalSection fill:#ffebe6,stroke:#cc3300,stroke-width:2px\r\n                D -- \"Structured Command\" --\u003e E[\"4\\. Execution Engine \u003cbr\u003e (Command Dispatch)\"]\r\n                E -- \"Read/Write Data\" --\u003e F[\"5\\. In-Memory Storage \u003cbr\u003e (std.StringHashMap)\"]\r\n\r\n                topLabel1[\"(Mutex protected)\"]\r\n            end\r\n            E -- \"Response\" --\u003e C\r\n        end\r\n        B -- \"Accepts \u0026 Spawns Thread\" --\u003e C\r\n    end\r\n\r\n    C --\u003e|\"Sends Response\"| A\r\n\r\n    %% Styling\r\n    classDef fakeLabel fill:#FFF0,stroke:#FFF0,color:#444, font-weight:bold, font-size:14px, text-align:left;\r\n    \r\n    style A fill:#D1E8FF,stroke:#333,stroke-width:2px\r\n    style B fill:#C2F0C2,stroke:#333,stroke-width:2px\r\n    style C fill:#FFF2CC,stroke:#333,stroke-width:2px\r\n    style D fill:#FFD8B1,stroke:#333,stroke-width:2px\r\n    style E fill:#FFD8B1,stroke:#333,stroke-width:2px\r\n    style F fill:#F5C2E7,stroke:#333,stroke-width:2px\r\n  \r\n    style topLabel1 fill:#FFF,stroke:#FFF,color:#000,width:1px,height:1px,position:absolute,top:0,left:0,z-index:1;\r\n\r\n    style topLabel2 fill:#FFF,stroke:#FFF,color:#000,width:1px,height:1px,position:absolute,top:0,left:0,z-index:1;\r\n\r\n\r\n\r\n```\r\n\r\n## Technology Stack\r\n\r\n| Category | Technology | Description |\r\n| :--- | :--- | :--- |\r\n| **Core Language** | **Zig (v0.14.0+)** | Chosen for its simplicity, performance, explicit memory management, and powerful C interoperability. |\r\n| **Protocols** | TCP/IP, RESP | The server communicates over TCP and implements the Redis Serialization Protocol (RESP). |\r\n| **Libraries** | Zig Standard Library | Used for networking, I/O, memory management, and the built-in test runner. |\r\n| | `zig-clap` | A third-party library for parsing command-line arguments, managed via Git submodules. |\r\n| **Build System** | Zig Build System | Used to manage compilation, testing, dependencies, and cross-compilation to other platforms. |\r\n| **CI/CD** | GitHub Actions | Configured to automate building and testing across multiple platforms (Windows, macOS, Linux). |\r\n| **Testing** | `pytest`, `redis-py` | A comprehensive Python test suite for high-level compatibility and low-level protocol validation. |\r\n| | `ncat`, `telnet` | Used for manual end-to-end testing and ad-hoc validation. |\r\n| | `redis-benchmark` | The standard Redis tool used to measure performance metrics. |\r\n\r\n## Dependency Management: Using Git Submodules\r\n\r\nThis project uses `zig-clap` for command-line argument parsing. We use **Git submodules** to vendor this dependency directly into our repository, giving us precise control over the code.\r\n\r\n### Submodule Setup\r\n\r\nIf you are cloning this repository for the first time, you need to initialize the submodules:\r\n```bash\r\ngit submodule update --init --recursive\r\n````\r\n\r\n## Building and Running\r\n\r\nYou can build, run, and test the project using the standard Zig build system.\r\n\r\n### Build the Executable\r\n\r\nThis command compiles the project and places the executable in `zig-out/bin/statedb`.\r\n\r\n```bash\r\nzig build\r\n```\r\n\r\n### Run the Server\r\n\r\nThis command builds and immediately runs the server. By default, it listens on `localhost:8080`.\r\n\r\n```bash\r\nzig build run\r\n```\r\n\r\n### Run Unit Tests\r\n\r\nThis command compiles and runs all the tests defined within the Zig source files.\r\n\r\n```bash\r\nzig build test\r\n```\r\n\r\n## Continuous Integration \u0026 Release (CI/CD)\r\n\r\nThis project uses **GitHub Actions** to automate the build, test, and release process. The workflow is defined in `.github/workflows/release.yml`.\r\n\r\n### Automated Workflow Steps:\r\n\r\n1.  **Trigger:** The workflow is automatically triggered on every push to the `main` branch.\r\n2.  **Build Matrix:** It uses a build matrix to compile and test the project in parallel across multiple target platforms:\r\n      * `x86_64-windows`\r\n      * `x86_64-linux`\r\n      * `x86_64-macos`\r\n      * `aarch64-macos` (Apple Silicon)\r\n3.  **Create Release:** If all builds and tests succeed, the workflow automatically creates a new GitHub Release.\r\n4.  **Publish Artifacts:** The compiled binaries for all target platforms are compressed and uploaded to the release as downloadable artifacts, making it easy to access the latest version for any OS.\r\n\r\n## Testing the Server\r\n\r\nOnce the server is running, you can connect to it in several ways.\r\n\r\n### 1\\. Manual Testing with `ncat`\r\n\r\nYou can connect from another terminal using a tool like `ncat` or `telnet`.\r\n\r\n```bash\r\nncat localhost 8080\r\n```\r\n\r\n### 2\\. Automated Testing with Python\r\n\r\nThe project includes a comprehensive test suite written in Python to verify command correctness and compatibility.\r\n\r\n  * **`test.py`**: This script uses `pytest` to run a full suite of automated tests using both raw sockets and the `redis-py` client.\r\n    ```bash\r\n    # Install dependencies\r\n    pip install pytest redis\r\n\r\n    # Run the test suite\r\n    pytest -v test.py\r\n    ```\r\n  * **Jupyter Notebooks**: For interactive testing, two notebooks are provided in the `/notebooks` directory.\r\n      * **[redis-py-testing.ipynb](notebooks/redis-py-testing.ipynb)**: An interactive notebook for testing the server using the high-level `redis-py` client.\r\n      * **[raw-socket-testing.ipynb](notebooks/raw-socket-testing.ipynb)**: An interactive notebook for sending raw RESP commands to test the server's protocol parser directly.\r\n\r\n## Implemented Commands\r\n\r\nThis table tracks the implementation status of Redis commands.\r\n\r\n| Category | Command | Status |\r\n| :--- | :--- | :--- |\r\n| **Connection** | `PING` | [x] |\r\n| | `ECHO` | [x] |\r\n| | `CLIENT` | [x] |\r\n| **Keys** | `DEL` | [x] |\r\n| | `EXISTS` | [x] |\r\n| | `FLUSHDB` | [x] |\r\n| | `TYPE` | [x] |\r\n| **Strings** | `GET` | [x] |\r\n| | `SET` | [x] |\r\n| | `GETDEL` | [x] |\r\n| | `INCR` | [x] |\r\n| | `DECR` | [x] |\r\n| | `INCRBY` | [x] |\r\n| | `DECRBY` | [x] |\r\n| **Hashes** | `HSET` | [x] |\r\n| | `HGET` | [x] |\r\n| | `HGETALL` | [x] |\r\n\r\n## Roadmap\r\n\r\n### Current Implementation\r\n✅ **Completed Features**:\r\n- Thread-per-connection model with mutex protection\r\n- Full RESP2 protocol support\r\n- Core commands (SET/GET/HSET/HGET)\r\n- Python FFI prototype\r\n- CI/CD pipeline (multi-platform builds)\r\n\r\n### Potential Future Directions\r\n🔍 **Areas for Exploration** (when time/interest permits):\r\n- Async I/O improvements\r\n- RESP3 protocol support\r\n- Basic persistence options\r\n- Additional data structures\r\n- Security enhancements\r\n\r\n**Note**: These are not committed timelines but technical directions that could improve StateDB. Community contributions are especially welcome in these areas!\r\n\r\n### How to Contribute\r\n🌱 **Good First Issues**: [Explore beginner-friendly tasks](https://github.com/namitkewat/statedb/issues)\r\n\r\n💡 **Suggested Focus Areas**:\r\n- Protocol implementations\r\n- Performance optimizations\r\n- Client libraries\r\n- Documentation improvements\r\n\r\n## Project Status\r\n\r\nThis project is being developed as part of the final semester (Semester 4) project work for the **Master of Computer Applications (MCA)** program at **Manipal University Jaipur**.\r\n\r\nIt has a functional TCP server that can parse and handle a core set of Redis commands. The focus is on expanding the command set, ensuring correctness, and improving performance.\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnamitkewat%2Fstatedb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnamitkewat%2Fstatedb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnamitkewat%2Fstatedb/lists"}