https://github.com/lreimer/data-services-javaee7
Data services: Processing big data the microservice way with Java EE 7
https://github.com/lreimer/data-services-javaee7
bigdata cockroachdb docker java javaee jaxrs jbatch jms jsonp kafka kubernetes microservice mosquitto mqtt payara postgresql rabbitmq restful
Last synced: 26 days ago
JSON representation
Data services: Processing big data the microservice way with Java EE 7
- Host: GitHub
- URL: https://github.com/lreimer/data-services-javaee7
- Owner: lreimer
- License: mit
- Created: 2018-02-21T15:37:55.000Z (about 8 years ago)
- Default Branch: master
- Last Pushed: 2018-09-26T07:29:10.000Z (over 7 years ago)
- Last Synced: 2025-07-08T23:40:21.361Z (9 months ago)
- Topics: bigdata, cockroachdb, docker, java, javaee, jaxrs, jbatch, jms, jsonp, kafka, kubernetes, microservice, mosquitto, mqtt, payara, postgresql, rabbitmq, restful
- Language: Java
- Size: 262 KB
- Stars: 4
- Watchers: 2
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Data services:
Processing big data the microservice way with Java EE 7
This repository contains the showcase for my talk at O'Reilly Architecture Conference NY 2018.
For details see: https://conferences.oreilly.com/software-architecture/sa-ny/public/schedule/detail/63967
The slides for this talk can be found on SpeakerDeck here: https://speakerdeck.com/lreimer/data-services-processing-big-data-the-microservice-way
# Usage
## Building the Data Services
The individual data services are all separate Gradle builds. Make sure to build these individually.
```
$ cd hazelcast/
$ ./gradlew clean build
```
## Running with Docker Compose
```
$ docker-compose up --build
```
## Setup Google Container Engine
- Install the gcloud SDK for Mac (or Windows)
- Make sure you have a project with billing activated as well as the container engine management API
- Make sure you have `kubectl` installed, either using `gcloud` or using `brew` et.al.
```
$ gcloud config list project
$ gcloud config set compute/zone europe-west1-b
$ gcloud config set container/use_client_certificate False
$ gcloud container clusters create data-services --num-nodes=5 --enable-autoscaling --min-nodes=5 --max-nodes=7
$ gcloud container clusters describe data-services
$ gcloud auth application-default login
$ kubectl cluster-info
```
Once you are done, remember to delete the cluster again!
```
$ gcloud container clusters delete data-services
```
## Administration
- **Rabbit MQ**
- URL: http://localhost:15672
- User: guest
- PWD: guest
- **Active MQ**
- URL: http://localhost:8161
- User: admin
- PWD: admin
- **Cockroach DB**
- URL: http://localhost:8080
- User: root
- PWD: root
## Cockroach DB Setup
Follow the instructions here: https://www.cockroachlabs.com/docs/stable/orchestrate-cockroachdb-with-kubernetes-insecure.html
Also, I have included the YAML files in `kubernetes/infrastructure/`. First apply the deployment.
Once everything has started, run the init job and finally create the services.
Test the cluster:
```
$ kubectl run cockroachdb -it --image=cockroachdb/cockroach --rm --restart=Never -- sql --insecure --host=cockroachdb-public
```
## REST endpoints
To test the different REST endpoints use the Postman collection: `data-services.postman_collection.json`
# References
- https://speakerdeck.com/lreimer/data-services-processing-big-data-the-microservice-way
- https://docs.payara.fish/documentation/payara-micro/payara-micro.html
- http://blog.payara.fish/kubernetes-native-discovery-with-payara-micro
- https://github.com/MeroRai/payara-hazelcast-kubernetes
- https://github.com/hazelcast/hazelcast-kubernetes
- https://hub.docker.com/r/hazelcast/hazelcast-kubernetes/
- http://blog.payara.fish/payara-micro-jca-adapters-mqtt
- http://blog.payara.fish/payara-micro-jca-adapters-apache-kafka
- http://blog.payara.fish/cloud-connectors-in-payara-micro
# Maintainer
M.-Leander Reimer (@lreimer),
# License
This software is provided under the MIT open source license, read the `LICENSE`
file for details.