Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/drujensen/kscli
Kafka Schema Register CLI
https://github.com/drujensen/kscli
Last synced: 10 days ago
JSON representation
Kafka Schema Register CLI
- Host: GitHub
- URL: https://github.com/drujensen/kscli
- Owner: drujensen
- Created: 2022-09-24T21:42:26.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2022-11-06T00:17:40.000Z (about 2 years ago)
- Last Synced: 2024-10-25T01:35:43.832Z (about 2 months ago)
- Language: Rust
- Size: 48.8 KB
- Stars: 2
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Kafka Schema Registry CLI
An opinionated client for the [Confluent Schema Registry](https://docs.confluent.io/current/schema-registry/docs/index.html).
This cli uses a simple configuration file to manage schemas in the registry.
The topic name follows a strict naming convention:
`{purpose}.{service}.{resource}`
- purpose: One of the following - [request, reply, event, store, log]
- service: The name of the service or domain that owns this topic
- resource: The name of the resource or object this schema is about
The naming convention is used to generate the topic name and the schema name.
## Installation
To install using [brew on Mac or Linux](https://brew.sh/):
Install kscli:
```bash
```sh
brew tap drujensen/kscli
brew install kscli
```
## Usage
To start off, we need to create a config file. The default location of the config file is: `config/ksconfig.yml`
### Initialize config file
To create an example config file:
```sh
kscli init
```
If you need to specify a different location, you will need to use the `-c` flag:
```sh
kscli -c example.yml init
```
Here is what the config file looks like:
```yaml
---
service: blog
schema_path: "./schemas"
topics:
- purpose: request
resource: post
properties:
compatibility: BACKWARD
retry: true
dlt: true
- purpose: reply
resource: post
properties:
compatibility: FORWARD
retry: true
dlt: true
```
Sample avsc files will be created in the `schema_path` directory. The avsc files will be named using the naming convention.
The file naming convention is: `{schema_path}/{purpose}/{resource}.avsc`
A sample avsc file:
```json
{
"type": "record",
"name": "PostRequest",
"namespace": "com.example.blog",
"fields": [
{"name": "id", "type": "string"},
{"name": "title", "type": "string"},
{"name": "body", "type": "string"},
]
}
```
### Push to the Schema Registry
After setting up the config file and any changes to the schema, you can push the changes to the Schema Registry.
To push the changes:
```sh
kscli push
```
Push will create the topics and schemas in the registry. If you enabled retry and/or dlt, it will create another schema for each topic. These names are compatible with Java Spring Boot naming convention for RetryConfiguration.
This also changes the DLT compatibility to NONE so that the DLT topic can be used with any schema. If you don't set this to NONE, you could end up in an endless loop trying to process a message and getting a failure to register the new schema because its incompatible with the previous schema.
### Pull from the Schema Registry
If you need to pull the schemas from the Schema Registry, you can do so with the following command:
```sh
kscli pull
```
## Environment Variables
There are cases where you want to override some of the configuration settings. You can do this using environment variables.
The following environment variables are supported:
```sh
KAFKA_SCHEMA_CLI_CONFIG={path to the config file}
KAFKA_SCHEMA_REGISTRY_URL={url to the schema registry}
```
## Development
This was built using the Rust programming language. If you want to contribute, you will need to install Rust. You can find the installation instructions [here](https://www.rust-lang.org/tools/install).
To build the application:
```bash
cargo build
```
To run the application:
```bash
cargo run
```
To install the application:
```bash
cargo install --path .
```
## Contributing
1. Fork it ()
2. Create your feature branch (`git checkout -b my-new-feature`)
3. Commit your changes (`git commit -am 'Add some feature'`)
4. Push to the branch (`git push origin my-new-feature`)
5. Create a new Pull Request
## Contributors
- [Dru Jensen](https://github.com/drujensen) - creator and maintainer