{"id":19176704,"url":"https://github.com/shikhar97/keyvaluestore","last_synced_at":"2026-04-09T22:49:58.641Z","repository":{"id":225756991,"uuid":"766612662","full_name":"Shikhar97/KeyValueStore","owner":"Shikhar97","description":"A simple key-value store using Kubernetes (k8s), FastAPI, and Huey as a REDIS queue","archived":false,"fork":false,"pushed_at":"2024-03-30T20:36:49.000Z","size":49,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-17T03:42:27.882Z","etag":null,"topics":["fastapi","huey","kubernetes","minikube","redis"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Shikhar97.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-03T18:32:36.000Z","updated_at":"2024-03-12T23:14:55.000Z","dependencies_parsed_at":"2024-03-30T21:29:31.668Z","dependency_job_id":"8a736cd3-7f1d-4d12-88c6-85fd76584ba7","html_url":"https://github.com/Shikhar97/KeyValueStore","commit_stats":null,"previous_names":["shikhar97/keyvaluestore"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Shikhar97%2FKeyValueStore","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Shikhar97%2FKeyValueStore/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Shikhar97%2FKeyValueStore/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Shikhar97%2FKeyValueStore/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Shikhar97","download_url":"https://codeload.github.com/Shikhar97/KeyValueStore/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240254288,"owners_count":19772392,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fastapi","huey","kubernetes","minikube","redis"],"created_at":"2024-11-09T10:29:36.813Z","updated_at":"2026-04-09T22:49:53.603Z","avatar_url":"https://github.com/Shikhar97.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Project Overview\n\nThis project implements a scalable key-value store using Kubernetes (k8s), FastAPI, and Huey as a REDIS queue. \nThe key-value store is designed to be highly scalable, robust, and efficient, capable of handling a large number of concurrent requests while maintaining high availability and fault tolerance.\n\n## Key Features\n\n1. **Kubernetes Deployment**: Utilizes Kubernetes for managing deployment, auto-scaling, and management of containerized applications.\n2. **FastAPI**: Modern, fast web framework for building APIs with Python 3.7+ based on standard Python type hints.\n3. **Huey as a REDIS Queue**: Lightweight task queue built on top of Redis, allows offloading time-consuming tasks to the background.\n4. **Scalability**: Designed to scale horizontally by adding more pods/deployments based on the metrics.\n\n\u003cdiv style=\"text-align:center\"\u003e\n  \u003cimg alt=\"Demo\" src=\"https://github.com/Shikhar97/KeyValueStore/blob/main/HLA.png\" /\u003e\n\u003c/div\u003e\n\n## Project Structure\n\n- `app/`: Contains the main application code\n  - `models/`: Data model for our server\n  - `Dockerfile`: Dockerfile to create an image for the server\n  - `huey_consumer.py`: To run the Huey consumer for the tasks\n  - `redis_huey.py`: It contains the functions to Add/Get/Delete key to/from database\n  - `requirements.txt`: Dependencies required for this project\n  - `main.py`: FastAPI application entry point\n- `deployments/`: Kubernetes deployment files\n  - `redis/`: To deploy redis on k8 cluster\n  - `kvserver/`: To deploy FastApi Server and consumer\n- `tests/`: Test cases\n- `README.md`: Project documentation\n\n\n## Prerequisites\n\nMake sure you have the following tools installed on your system:\n- Python 3.x\n- FastAPI\n- Huey\n- Redis\n- Docker\n- Kubernetes (minikube for local development)\n- kubectl\n\n\n## Usage\n### Local Setup\n\n**1. Clone the repository and Install dependencies**\n```bash\ngit clone https://github.com/Shikhar97/KeyValueStore.git\ncd KeyValueStore/app; pip install -r requirements.txt\n```\n\n**2. Run the redis server**\n```bash\nredis-server\n```\n\n**3. Run the FASTApi server**\n```bash\npython3 uvicorn main:app --host 0.0.0.0 --port 8000\n```\n\n**4. Run Huey consumer** \n```bash\npython3 huey_consumer.py redis_huey.huey\n```\n\n\nAccess FastAPI server at [http://localhost:8000/api/v1/](http://localhost:8000/api/v1) using Postman or any other tool. \n\n\n### Kubernetes Setup(Minikube)\n\n**1. Clone the repository**\n```bash\ngit clone https://github.com/Shikhar97/KeyValueStore.git\n\n```\n\n**2. Start minikube(_if not already running_) and Deploy the redis stateful-set**\n```bash\nminikube start\ncd KeyValueStore/deployment/redis\nkubectl -n kvstore apply -f redis-statefulSet.yml\n```\n\n**3. Deploy the FASTApi server**\n```bash\ncd KeyValueStore/deployment/kvserver\nkubectl -n kvstore apply -f kvserver-deployment.yml\n```\n\n**4. Enable port-forwarding** \n```bash\nkubectl -n kvstore port-forward svc/fastapi-service 8000:8000\n```\n\n**5. Cleanup**\n```bash\ncd KeyValueStore/deployment/redis\nkubectl -n kvstore delete -f redis-statefulSet.yml\n\ncd KeyValueStore/deployment/kvserver\nkubectl -n kvstore delete -f kvserver-deployment.yml\nminikube stop\nminikube delete\n```\n\nAccess FastAPI server at [http://localhost:8000/api/v1/](http://localhost:8000/api/v1) using Postman or any other tool.\n\n\n**_Docker images are built on arm64 architechture. Not supported on amd64._**\n\n## API Documentation\nCheck the available APIs at [http://localhost:8000/docs](http://localhost:8000/docs) \n\n## Testing\n### To run the test cases\n\n1. **Navigate to the root directory of your project:**\n\n    ```bash\n    cd KeyValueStore/app\n    ```\n\n2. **Install pytest:**\n\n    ```bash\n    pip install pytest\n    ```\n\n3. **Run the test cases using pytest:**\n\n    ```bash\n    pytest tests.py\n    ```\n\nThe test results will be displayed in the terminal, indicating whether each test passed or failed.\n\n\n## Acknowledgments\n\n- [Kubernetes](https://kubernetes.io/)\n- [FastAPI](https://fastapi.tiangolo.com/)\n- [Huey](https://huey.readthedocs.io/en/latest/)\n- [Postman](https://www.postman.com/)\n- [Redis](https://redis.io/docs/about/)\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshikhar97%2Fkeyvaluestore","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshikhar97%2Fkeyvaluestore","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshikhar97%2Fkeyvaluestore/lists"}