Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/musabbozkurt/live-data-service
Kafka, Grafana, Prometheus, Zipkin, Redis and Elasticsearch Integrations with Spring Boot 3, Java 21 and Docker. Integration Tests with Testcontainers for Controller. Unit Tests for Controller, Service, and Mapper with Junit 5, and Mockito
https://github.com/musabbozkurt/live-data-service
docker-compose elasticsearch flyway google-api-services-playintegrity grafana java-21 jsonplaceholder junit5 kafka kibana micrometer mockito prometheus redis redisson restclient spring-boot-3 testcontainers zipkin
Last synced: 26 days ago
JSON representation
Kafka, Grafana, Prometheus, Zipkin, Redis and Elasticsearch Integrations with Spring Boot 3, Java 21 and Docker. Integration Tests with Testcontainers for Controller. Unit Tests for Controller, Service, and Mapper with Junit 5, and Mockito
- Host: GitHub
- URL: https://github.com/musabbozkurt/live-data-service
- Owner: musabbozkurt
- Created: 2022-11-12T17:37:22.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-02-16T15:29:03.000Z (9 months ago)
- Last Synced: 2024-02-16T22:42:30.749Z (9 months ago)
- Topics: docker-compose, elasticsearch, flyway, google-api-services-playintegrity, grafana, java-21, jsonplaceholder, junit5, kafka, kibana, micrometer, mockito, prometheus, redis, redisson, restclient, spring-boot-3, testcontainers, zipkin
- Language: Java
- Homepage:
- Size: 350 KB
- Stars: 4
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
Table of Contents
- Summary
- Prerequisites and Installation
- Tech Stack
- How To Run And Test Application
- How To Run And Test Application with Dockerfile (OPTIONAL)
- How To Run And Test Application with docker-compose.yml (OPTIONAL)
- Redis Commands
- Google Play Integrity API
- References
-------
### Summary
#### live-data-service is providing CRUD operations to process live events
-------
### Prerequisites
- `Java 21+` needs to be installed `export JAVA_HOME=$(/usr/libexec/java_home -v 21)`
- `Maven` needs to be installed
- `Docker` needs to be installed
- Install any Java IDE (`Eclipse`, `STS`, `Intellij` etc..) and ensure you are able to launch
- Clone or checkout the project from version control system (`git`) and follow below steps
-------
### Tech_Stack
- `Java 21`
- `Spring Boot 3`
- `H2 Database` (Default values are provided below)
- `Default H2 Database Url`: http://localhost:8080/h2-console
- `username`: `sa`
- `password`: `sa`
- `url`: `jdbc:h2:mem:MB_TEST;DB_CLOSE_DELAY=-1`
- `Swagger`: http://localhost:8080/swagger-ui/index.html
- `Flyway` for database migration
- Centralize exception handling by `ControllerAdvice`
- `Mapstruct` to map different type of objects to each other
- `Micrometer` dependencies were added to track the logs easily
- `Testcontainers` dependencies were added for integration tests
- `docker-compose.yml` contains `Grafana`, `Prometheus` and `Zipkin` to track metrics, `Kafka` for event-driven
architecture, `Redis` for caching, `Elasticsearch` for search and analytics engine
- `Actuator`: http://localhost:8080/actuator
- `Kafka UI`: http://localhost:9091/
- `Grafana`
- Login Credentials
- Url: http://localhost:3000/
- Email or username: `admin`
- Password: `admin`
- Add `Prometheus` Datasource
- Url: http://host.docker.internal:9090
- Add Dashboards
- https://grafana.com/grafana/dashboards/4701-jvm-micrometer/
- https://grafana.com/grafana/dashboards/5373-micrometer-spring-throughput/
- `Prometheus`
- `Actuator`: http://localhost:8080/actuator/prometheus
- `Prometheus`: http://localhost:9090/graph
- `Zipkin UI`: http://localhost:9411
- `Elasticsearch`: http://localhost:9200/
- `Kibana`: http://localhost:5601/
- Test via `Postman` (OPTIONAL):
1. Import [Postman Collection](docs%2Funit_test_service.postman_collection.json)
2. Right-click the imported Postman Collection and click _**Run Collection**_ section.
3. On the right panel choose _**Functional**_ or _**Performance**_ section, edit _**Run configuration**_ and click
_**run**_ to test the application.
-------
### How_To_Run_And_Test_Application
- Please follow the following steps, if you want to build and run Spring Boot Application
```
*** Run the application by following these steps.
1 - cd live-data-service
2 - docker-compose up -d
3 - mvn clean install or mvn clean package
4 - mvn spring-boot:run
```
-------
### How_To_Run_And_Test_Dockerfile
- Please follow the following steps, if you want to build and run Dockerfile
```
1 - cd live-data-service
2 - docker-compose up -d
3 - mvn clean install or mvn clean package --------THIS IS MUST---------
4 - docker build -t mb/live-data-service .
5 - docker run -p 8080:8080 mb/live-data-service
```
-------
### How_To_Run_And_Test_Docker_Compose
- Please follow the following steps, if you want to build and run `docker-compose.yml`
- Remove `live-data-service` service comment in `services` section in `docker-compose.yml`
- `Docker` -> `Preferences` -> `Resources` -> `File sharing` -> click add button and select `prometheus` folder under
the `/src/main/resources` or just add `/etc/prometheus` path -> `Apply & Restart`
- 
```
1 - cd live-data-service
2 - mvn clean install or mvn clean package --------THIS IS MUST---------
3 - docker build -t mb/live-data-service .
4 - docker-compose up -d
```
-------
### Redis
- The following command returns all matched data by `'keyPattern:*'` pattern
- `redis-cli --scan --pattern 'keyPattern:*'`
- The following command deletes all matched data by `'keyPattern:*'` pattern
- `redis-cli KEYS 'keyPattern:*' | xargs redis-cli DEL`
- The following command finds `TYPE` in redis with `KEY`
- `TYPE key` -> `TYPE xxx:hashedIdOrSomethingElse`
- The following commands search by `TYPE`
- for `"string" TYPE`: `get key`
- for `"hash" TYPE`: `hgetall key`
- for `"list" TYPE`: `lrange key 0 -1`
- for `"set" TYPE`: `smembers key`
- for `"zset" TYPE`: `zrange key 0 -1 withScores`
- `RedisInsight`: http://localhost:8001/
-------
### Google_Play_Integrity_API
- [Setup Firebase](https://firebase.google.com/docs/admin/setup) or Just follow the following steps
to [Initialize the SDK, Create and Download Firebase Credentials JSON File](https://firebase.google.com/docs/admin/setup#initialize-sdk)
- After JSON file is downloaded, copy the content of downloaded file and replace `play-integrity-credentials.json`
content with that copied text
- Replace `application-name` value in application.yml file with `project_id` variable that is
in `play-integrity-credentials.json`
- [HELPFUL LINK BUT IT SHOULD NOT BE NEEDED](https://stackoverflow.com/a/40799378)
-------
### References
- [Metrics Made Easy Via Spring Actuator, Docker, Prometheus, and Grafana](https://www.youtube.com/watch?v=Utv7MWgNTvI)
- https://prometheus.io/docs/prometheus/latest/installation/#volumes-bind-mount
- [Spring Boot Rest Controller Unit Test with @WebMvcTest](https://www.bezkoder.com/spring-boot-webmvctest/)
- [Redis Commands](https://auth0.com/blog/introduction-to-redis-install-cli-commands-and-data-types/)
- [Running RedisInsight using Docker Compose](https://collabnix.com/running-redisinsight-using-docker-compose/)
- [Google Play Integrity API](https://developer.android.com/google/play/integrity)
- [A Guide to Querydsl with JPA](https://www.baeldung.com/querydsl-with-jpa-tutorial)