https://github.com/j3rrryy/cloud_storage

Cloud Storage app with microservice architecture. Written using Litestar and gRPC framework
https://github.com/j3rrryy/cloud_storage

apache-kafka cloud-storage docker grafana grpc litestar loki message-pack microservices minio nginx postgresql prometheus promtail redis

Last synced: 22 days ago
JSON representation

Cloud Storage app with microservice architecture. Written using Litestar and gRPC framework

• Host: GitHub
• URL: https://github.com/j3rrryy/cloud_storage
• Owner: j3rrryy
• License: mit
• Created: 2024-11-01T08:16:51.000Z (6 months ago)
• Default Branch: main
• Last Pushed: 2025-04-05T13:56:51.000Z (24 days ago)
• Last Synced: 2025-04-05T14:30:49.610Z (24 days ago)
• Topics: apache-kafka, cloud-storage, docker, grafana, grpc, litestar, loki, message-pack, microservices, minio, nginx, postgresql, prometheus, promtail, redis
• Language: Python
• Homepage:
• Size: 1.01 MB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Cloud Storage



  

    

  

  

    

  

  

    

  

  

    

  

  

    

  



## :book: Key features

- Microservice architecture

- gRPC between services

- Fast serialization with MessagePack

- Access and refresh JWT tokens

- Active sessions control

- Fast file upload/downolad

- Emails with new login info

- Main DB - PostgreSQL

- DB for cache - Redis

- S3 for files and logs - MinIO

- Message broker between Gateway and Mail service - Apache Kafka

- Monitoring - Prometheus & Grafana

- Log aggregation - Promtail & Loki & Grafana

![Architecture](https://github.com/j3rrryy/cloud_storage/blob/main/images/architecture.webp?raw=true)

> [!NOTE]

> API located at `/api`

>

> Docs located at `/api/docs`, but Swagger does not support MessagePack, so use another tool to send requests with this content type

>

> Grafana located at `/monitoring`

## :computer: Requirements

- Docker

## :hammer_and_wrench: Getting started

- Copy `.env` file from `examples/dev/` to `dev/` folder and fill it in

- **(For dev/prod)** Copy `redis.conf` file from `examples/` to `dev/` or `prod/` folder and fill it in

- **(For prod)** Copy `.env` file from `examples/prod/` to `prod/` folder and fill it in

- **(For prod)** Copy `nginx.conf` file from `examples/prod/` to `prod/` folder and fill it in

- **(For prod)** Copy `docker-compose.cert.yml` file from `examples/prod/` to `prod/` folder and fill it in

### :rocket: Start

- Run the **dev ver.**

  ```shell

  docker compose -f docker-compose.dev.yml up --build -d

  ```

- Run the **prod ver.** and get a SSL certificate

  - Create the directory on the server

    ```shell

    mkdir -p /cloud_storage/

    ```

  - Use SCP to copy the prod files to the server

    ```shell

    scp -r ./prod/* @:/cloud_storage/

    ```

  - Run the deploy script

    ```shell

    bash deploy.sh

    ```

    

### :x: Stop

```shell

docker compose -f docker-compose..yml stop

```