https://github.com/stn1slv/meetup-schema-registry
Sources for Schema Registry meetup
https://github.com/stn1slv/meetup-schema-registry
apicurio json-schema kafka validation xml-schema
Last synced: 5 months ago
JSON representation
Sources for Schema Registry meetup
- Host: GitHub
- URL: https://github.com/stn1slv/meetup-schema-registry
- Owner: stn1slv
- License: apache-2.0
- Created: 2022-09-21T12:04:29.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2023-06-26T17:15:05.000Z (almost 3 years ago)
- Last Synced: 2025-12-22T03:47:45.152Z (6 months ago)
- Topics: apicurio, json-schema, kafka, validation, xml-schema
- Language: Java
- Homepage:
- Size: 6.51 MB
- Stars: 5
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Schema Registry Demo
The [recording](https://youtu.be/9aql6xu8kRc) and [slides](slides.pdf) from the meetup.
## Preparing
You have to add some entries to your ```/etc/hosts``` file:
```
127.0.0.1 keycloak
127.0.0.1 kafka
127.0.0.1 schema-registry
```
That's needed for host resolution because Kafka brokers and Kafka clients connecting to Keycloak have to use the same hostname to ensure the compatibility of generated access tokens. Also, when Kafka client connects to Kafka broker running inside docker image, the broker will redirect the client to ```kafka:9092```.
Preparing local environment:
MacOS
Required software:
- Docker engine & docker compose (Docker Desktop for Mac or [Rancher Desktop](https://github.com/rancher-sandbox/rancher-desktop/))
- [Git](https://github.com/git/git)
- [Apache Maven](https://github.com/apache/maven)
- [httpie](https://github.com/httpie/httpie) (or any other http client)
- [kcat](https://github.com/edenhill/kcat) (or any other kafka client)
You can install using following command:
```
brew install git httpie kcat maven
```
Linux
Required software:
- Docker engine & docker compose
- [Git](https://github.com/git/git)
- [Apache Maven](https://github.com/apache/maven)
- [httpie](https://github.com/httpie/httpie) (or any other http client)
- [kcat](https://github.com/edenhill/kcat) (or any other kafka client)
On Ubuntu you can install last three tools using the following command:
```
sudo apt update && sudo apt install git httpie kafkacat maven -y
```
Windows
Required software:
- Docker Desktop for Windows
- [Git](https://git-scm.com/download/win)
- [Apache Maven](https://github.com/apache/maven)
- Windows Subsystem for Linux (for httpie and kcat/kafkacat)
## Start environment
1. Clone this repository
```
git clone https://github.com/stn1slv/meetup-schema-registry
```
2. Use root directory of the repo
```
cd meetup-schema-registry
```
3. Cleanup
```
docker rm keycloak kafka zookeeper schema-registry sr-init kc-init
```
4. Start KeyCloak, Apicurio Registry and Kafka
```
docker-compose up
```
## Start demo applications
1. Start ServerA app
```
mvn clean spring-boot:run -f ServerA/pom.xml
```
2. Start ServerB app
```
mvn clean spring-boot:run -f ServerB/pom.xml
```
## Testing
#### JSON Schema
###### Send JSON message via http endpoint
Successful case
John made a purchase of item 20223 for Jane:
```
cat examples/purchaseOrderV1_Alice.json | http POST 'http://localhost:8085/doSomething' Content-Type:'application/json'
```
Alice sent a present to Bob:
```
cat examples/purchaseOrderV1_John.json | http POST 'http://localhost:8085/doSomething' Content-Type:'application/json'
```
Failed case
John made a purchase of item 20223 for Jane:
```
cat examples/purchaseOrderV1_Alice-invalid.json | http POST 'http://localhost:8085/doSomething' Content-Type:'application/json'
```
Alice sent a present to Bob:
```
cat examples/purchaseOrderV1_John-invalid.json | http POST 'http://localhost:8085/doSomething' Content-Type:'application/json'
```
#### XML Schema
###### Send XML message to Input topic in Kafka
Successful case
###### Send messages
John made a purchase of item 20223 for Jane:
```
kcat -P -b 127.0.0.1 -t input examples/purchaseOrderV1_Alice.xml
```
Alice sent a present to Bob:
```
kcat -P -b 127.0.0.1 -t input examples/purchaseOrderV1_John.xml
```
###### Monitor messages in Output topic
In case of successful validation the message will be moved to output topic:
```
kcat -b 127.0.0.1 -t output
```
Failed case
###### Send messages
John made a purchase of item 20223 for Jane:
```
kcat -P -b 127.0.0.1 -t input examples/purchaseOrderV1_Alice-invalid.xml
```
Alice sent a present to Bob:
```
kcat -P -b 127.0.0.1 -t input examples/purchaseOrderV1_John-invalid.xml
```
###### Monitor messages in DLQ topic
In case of validation failure the message will be moved to dlq topic:
```
kcat -C -b 127.0.0.1 -t dlq -f '\nKey: %k\t\nHeaders: %h \t\nValue: %s\\n--\n'
```
## URLs
#### Apicurio Registry
Endpoint: http://keycloak:8180/ui/artifacts
###### Users
| Username | Password | Role |
| -------- | -------- | ------------ |
| jcooper | jcooper | sr-admin |
| fkafka | fkafka | sr-developer |
| cdickens | cdickens | sr-readonly |