{"id":24750843,"url":"https://github.com/srujayreddy/key-value-server","last_synced_at":"2025-10-06T13:42:04.529Z","repository":{"id":273339114,"uuid":"918553027","full_name":"SrujayReddy/Key-Value-Server","owner":"SrujayReddy","description":"This project implements a concurrent Key-Value (KV) Server that communicates with a multi-threaded client via a shared memory region.","archived":false,"fork":false,"pushed_at":"2025-01-20T09:20:13.000Z","size":31,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-23T03:21:35.073Z","etag":null,"topics":["c","concurrency","concurrent-hashmap","lockless-data-structures","memory-management","multithreading","operating-system"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SrujayReddy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-18T08:21:25.000Z","updated_at":"2025-01-20T10:05:19.000Z","dependencies_parsed_at":"2025-01-20T10:29:53.639Z","dependency_job_id":"48220c15-ee56-4df3-bc37-0932d880069d","html_url":"https://github.com/SrujayReddy/Key-Value-Server","commit_stats":null,"previous_names":["srujayreddy/key-value-server"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/SrujayReddy/Key-Value-Server","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SrujayReddy%2FKey-Value-Server","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SrujayReddy%2FKey-Value-Server/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SrujayReddy%2FKey-Value-Server/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SrujayReddy%2FKey-Value-Server/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SrujayReddy","download_url":"https://codeload.github.com/SrujayReddy/Key-Value-Server/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SrujayReddy%2FKey-Value-Server/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278621847,"owners_count":26017253,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-06T02:00:05.630Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["c","concurrency","concurrent-hashmap","lockless-data-structures","memory-management","multithreading","operating-system"],"created_at":"2025-01-28T09:09:06.351Z","updated_at":"2025-10-06T13:42:04.486Z","avatar_url":"https://github.com/SrujayReddy.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# Key-Value Server\n\n## Table of Contents\n1. [Overview](#overview)\n2. [Implementors](#implementors)\n3. [Project Architecture](#project-architecture)\n   - [Directory Structure](#directory-structure)\n   - [Ring Buffer](#ring-buffer)\n   - [KV Store](#kv-store)\n4. [Building the Project](#building-the-project)\n5. [Running the Server and Client](#running-the-server-and-client)\n   - [Command-Line Arguments](#command-line-arguments)\n   - [Examples](#examples)\n6. [Workload Generation](#workload-generation)\n7. [Resources](#resources)\n8. [License](#license)\n\n----------\n\n## Overview\n\nThis project implements a **concurrent Key-Value (KV) Server** that communicates with a **multi-threaded client** via a **shared memory** region. The shared memory region is divided into:\n\n-   A **Ring Buffer** (for request submission by the client and consumption by the server), and\n-   A **Request-status Board** (for the server to post results back to the client).\n\nThe key objectives are:\n\n-   Implementing a **thread-safe**, **lockless** (or partially lock-free) ring buffer using atomic operations or minimal locking.\n-   Building a **multi-threaded**, **thread-safe** KV Store with fine-grained locking or atomic operations for fast concurrent access.\n-   Demonstrating **interprocess communication** (IPC) using a file-backed `mmap`.\n-   Ensuring high concurrency and minimal blocking between client and server threads.\n\n----------\n\n## Implementors\n\n1.  **Dhruv Desai**\n    \n    -   **CS Login:** ddesai\n    -   **NetID:** ddesai7\n    -   **Email:** [ddesai7@wisc.edu](mailto:ddesai7@wisc.edu)\n2.  **Srujay Jakkidi**\n    \n    -   **CS Login:** srujay\n    -   **NetID:** jakkidi\n    -   **Email:** [jakkidi@wisc.edu](mailto:jakkidi@wisc.edu)\n\n----------\n\n## Project Architecture\n\n### Directory Structure\n```\nKey-Value-Server/\n├── README.md               # This README\n├── Makefile                # Builds client and server\n├── include/\n│   ├── common.h            # Contains shared typedefs and hash function (DO NOT MODIFY)\n│   └── ring_buffer.h       # Ring Buffer interface\n├── src/\n│   ├── client.c            # Client code (producer)\n│   ├── ring_buffer.c       # Ring Buffer implementation\n│   ├── kv_store.c          # KV Store + server main() implementation\n│   └── kv_store.h          # If a separate header is created for KV Store\n├── scripts/\n│   └── gen_workload.py     # Helper script to generate workloads\n├── tests/\n│   ├── workload.txt        # Example workload file\n│   └── resources.txt       # Document describing external resources used\n├── solution.txt            # Explanation of the implementation approach\n\n```\n\n**Note:**\n\n-   `kv_store.h` is optional; create it if you prefer a separate header for your KV Store data structures and prototypes.\n-   `solution.txt` can be used to provide further insights into your design.\n\n### Ring Buffer\n\n-   **Location**: `src/ring_buffer.c` / `include/ring_buffer.h`\n-   **Purpose**: Acts as a **lockless** (or minimally locked) producer-consumer queue between client and server processes.\n-   **Key Operations**:\n    -   `init_ring(struct ring *r)`: Initializes the ring buffer’s head and tail indices and any synchronization primitives (e.g., atomic counters).\n    -   `ring_submit(struct ring *r, struct buffer_descriptor *bd)`: Thread-safe enqueue operation.\n    -   `ring_get(struct ring *r, struct buffer_descriptor *bd)`: Thread-safe dequeue operation.\n\n### KV Store\n\n-   **Location**: `src/kv_store.c` (and optionally `src/kv_store.h`)\n-   **Purpose**: Maintains key-value mappings. Must handle concurrent `PUT` and `GET` operations from multiple server threads.\n-   **Features**:\n    -   **Hashtable** with open addressing (linear probing) or chaining.\n    -   **Fine-Grained Locking** or lock-free approach for concurrency.\n    -   **Rehashing / Migration** support (if implemented) to handle table growth.\n\n----------\n\n## Building the Project\n\nA `Makefile` is provided at the project root. By default, it produces two executables: `client` and `server`.\n\n1.  **Navigate** to the project directory.\n    \n2.  **Build** both client and server by running:\n    \n    ```bash\n    make\n    \n    ```\n    \n    This triggers the `all` target, compiling `client.c`, `kv_store.c`, and `ring_buffer.c`.\n    \n3.  **Clean** (optional):\n    \n    ```bash\n    make clean\n    \n    ```\n    \n    Removes object files and any existing `client`/`server` binaries.\n    \n\n----------\n\n## Running the Server and Client\n\n### Command-Line Arguments\n\n1.  **Server**\n    \n    ```\n    ./server -n \u003cnum_server_threads\u003e -s \u003cinitial_hashtable_size\u003e\n    \n    ```\n    \n    -   `-n`: Number of **server threads**.\n    -   `-s`: The **initial hashtable size** for the KV Store.\n2.  **Client**\n    \n    ```\n    ./client -n \u003cnum_client_threads\u003e -w \u003cwindows_per_client_thread\u003e\n    \n    ```\n    \n    -   `-n`: Number of **client threads**.\n    -   `-w`: Number of **Request-status Board windows** each client thread owns.\n\n\u003e **Note:**\n\u003e -   The **shared memory** is created in the client using a file named `shmem_file`.\n\u003e -   The **server** also maps the same file for interprocess communication.\n\n### Examples\n\n-   **Starting the Server** with 2 threads and an initial hashtable size of 5:\n    \n    ```bash\n    ./server -n 2 -s 5\n    \n    ```\n    \n-   **Starting the Client** with 3 threads and 2 windows per thread:\n    \n    ```bash\n    ./client -n 3 -w 2\n    \n    ```\n    \n-   **Automation**: You can run the client first (which might fork the server in some implementations) or manually start the server and then the client, depending on your design.\n\n----------\n\n## Workload Generation\n\nA script named `gen_workload.py` is provided under `scripts/`. Use it to generate input workloads (PUT/GET requests) for stress testing your KV store. The generated output can be redirected to `workload.txt` or any file of your choice.\n\nExample usage:\n\n```bash\npython3 scripts/gen_workload.py 1000 \u003e tests/workload.txt\n\n```\n\n\n## Resources\n\n-   **DPDK Ring** concepts: [DPDK Documentation](https://doc.dpdk.org/)\n-   **Concurrent Hash Tables**: “Concurrent Hash Tables: Fast and General (?)” by Jacobson et al.\n\n----------\n\n## License\n\nThis project was developed as part of the **CS 537: Introduction to Operating Systems** course at the University of Wisconsin–Madison. It is shared strictly for educational and learning purposes only.\n\n**Important Notes:**\n- Redistribution or reuse of this code for academic submissions is prohibited and may violate academic integrity policies.\n- The project is licensed under the [MIT License](https://opensource.org/licenses/MIT). Any usage outside academic purposes must include proper attribution.\n\n\n----------\n\n**Enjoy building and experimenting with your concurrent Key-Value Server!**\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsrujayreddy%2Fkey-value-server","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsrujayreddy%2Fkey-value-server","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsrujayreddy%2Fkey-value-server/lists"}