https://github.com/j3rrryy/cloud_storage
Cloud Storage app with microservice architecture. Written using Litestar and gRPC framework
https://github.com/j3rrryy/cloud_storage
apache-kafka cloud-storage docker grafana grpc litestar loki microservices minio nginx postgresql prometheus promtail redis
Last synced: 5 months ago
JSON representation
Cloud Storage app with microservice architecture. Written using Litestar and gRPC framework
- Host: GitHub
- URL: https://github.com/j3rrryy/cloud_storage
- Owner: j3rrryy
- License: mit
- Created: 2024-11-01T08:16:51.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2026-01-14T17:20:53.000Z (5 months ago)
- Last Synced: 2026-01-14T20:08:01.700Z (5 months ago)
- Topics: apache-kafka, cloud-storage, docker, grafana, grpc, litestar, loki, microservices, minio, nginx, postgresql, prometheus, promtail, redis
- Language: Python
- Homepage:
- Size: 5.49 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Cloud Storage
## :book: Key features
- Microservice architecture
- gRPC between services
- Fast serialization with MessagePack
- Access and refresh JWT tokens
- Active sessions control
- Multipart file upload & download
- Emails with new login info
- Main DB - PostgreSQL
- DB for cache - Redis
- S3 for files and logs - MinIO
- Message broker between Gateway and Mail service - Apache Kafka
- Monitoring - Prometheus & Grafana
- Log aggregation - Promtail & Loki & Grafana
> [!NOTE]
> API located at `/api`
>
> Docs located at `/api/docs`, but Swagger does not support MessagePack, so use another tool to send requests with this content type
>
> Grafana located at `/admin/grafana`
## :computer: Requirements
- Docker
## :hammer_and_wrench: Getting started
- Copy `.env` file from `examples/dev/` to `dev/` folder and fill it in
- **(For dev/prod)** Copy `redis.conf` file from `examples/` to `dev/` or `prod/` folder and fill it in
- **(For prod)** Copy `.env` file from `examples/prod/` to `prod/` folder and fill it in
- **(For prod)** Copy `nginx.conf` file from `examples/prod/` to `prod/` folder and fill it in
- **(For prod)** Copy `docker-compose.cert.yml` file from `examples/prod/` to `prod/` folder and fill it in
### :rocket: Start
- Run the **dev ver.**
- Only API
```shell
docker compose -f docker-compose.dev.yml --profile api up --build -d
```
- API + monitoring
```shell
docker compose -f docker-compose.dev.yml --profile all up --build -d
```
- Run the **prod ver.** and get a SSL certificate
- Create the directory on the server
```shell
mkdir -p /cloud_storage/
```
- Use SCP to copy the prod files to the server
```shell
scp -r ./prod/* @:/cloud_storage/
```
- Run the deploy script
```shell
bash deploy.sh
```
### :x: Stop
```shell
docker compose -f docker-compose..yml stop
```
### :chart_with_upwards_trend: Load testing
- Install Locust
```shell
pip install locust
```
- Run the script
```shell
locust --host localhost -f ./load_testing/load_test.py
```
- Open the [console](http://localhost:8089)