https://github.com/ballerina-platform/module-ballerinax-confluent.cavroserdes
The Ballerina Avro Ser/Des package for Confluent provides APIs which will hide the complexities in serializing and deserializing data using Avro schemas along with the registration and retrieval of these schemas from the schema registry
https://github.com/ballerina-platform/module-ballerinax-confluent.cavroserdes
Last synced: 23 days ago
JSON representation
The Ballerina Avro Ser/Des package for Confluent provides APIs which will hide the complexities in serializing and deserializing data using Avro schemas along with the registration and retrieval of these schemas from the schema registry
- Host: GitHub
- URL: https://github.com/ballerina-platform/module-ballerinax-confluent.cavroserdes
- Owner: ballerina-platform
- License: apache-2.0
- Created: 2024-04-03T05:07:13.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-08-22T02:46:11.000Z (8 months ago)
- Last Synced: 2024-08-22T03:46:34.376Z (8 months ago)
- Language: Ballerina
- Size: 155 KB
- Stars: 1
- Watchers: 51
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
- awesome-kafka - ballerina-avro-kafka-serializer - Ballerina Confluent Avro Serializer/Deserializer (Libraries / Other Awesome Kafka)
README
# Ballerina Confluent Avro Serialization/Deserialization Connector
[](https://github.com/ballerina-platform/module-ballerinax-confluent.cavroserdes/actions/workflows/ci.yml)
[](https://github.com/ballerina-platform/module-ballerinax-confluent.cavroserdes/actions/workflows/trivy-scan.yml)
[](https://github.com/ballerina-platform/module-ballerinax-confluent.cavroserdes/actions/workflows/build-with-bal-test-graalvm.yml)
[](https://github.com/ballerina-platform/module-ballerinax-confluent.cavroserdes/commits/main)
[](https://github.com/ballerina-platform/ballerina-library/labels/module%2Fconfluent.cavroserdes)
[](https://opensource.org/licenses/Apache-2.0)
[Avro Serializer/Deserializer for Confluent Schema Registry](https://docs.confluent.io/platform/current/schema-registry/fundamentals/serdes-develop/serdes-avro.html) is an Avro serializer/deserializer designed to work with the Confluent Schema Registry. It is designed to not include the message schema in the message payload but instead includes the schema ID.
The Ballerina Avro Serializer/Deserializer for Confluent Schema Registry connector integrates with the Confluent Schema Registry for Avro serialization and deserialization.
## Quickstart
To use the Confluent schema registry connector in your Ballerina project, modify the `.bal` file as follows.
### Step 1: Import the module
Import the `ballerinax/confluent.cavroserdes` module into your Ballerina project.
```ballerina
import ballerinax/confluent.cavroserdes;
```
### Step 2: Invoke the connector operation
You can now utilize the operations available within the connector. To instantiate a `cregistry:Client` instance refer to the guidelines [here](https://central.ballerina.io/ballerinax/confluent.cregistry/latest).
```ballerina
public function main() returns error? {
cregistry:Client registry = ; // instantiate a schema registry client
string schema = string `
{
"type": "int",
"name" : "value",
"namespace": "data"
}`;
int value = 1;
byte[] bytes = check cavroserdes:serialize(registry, schema, value, "subject");
int number = check cavroserdes:deserialize(registry, bytes);
}
```
### Step 3: Run the Ballerina application
Use the following command to compile and run the Ballerina program.
```bash
bal run
```
## Examples
The Ballerina Avro Serializer/Deserializer connector for Confluent Schema Registry provides practical examples illustrating usage in various scenarios. Explore these [examples](https://github.com/ballerina-platform/module-ballerinax-confluent.cavroserdes/tree/main/examples).
1. [Kafka Avro producer](https://github.com/ballerina-platform/module-ballerinax-confluent.cavroserdes/tree/main/examples/kafka-avro-producer) - This example demonstrates how to publish Avro serialized data to a Kafka topic.
2. [Kafka Avro consumer](https://github.com/ballerina-platform/module-ballerinax-confluent.cavroserdes/tree/main/examples/kafka-avro-consumer) - This guide demonstrates how to consume data in the correct format according to the Avro schema from a Kafka topic.
## Issues and projects
The **Issues** and **Projects** tabs are disabled for this repository as this is part of the Ballerina library. To report bugs, request new features, start new discussions, view project boards, etc., visit the Ballerina library [parent repository](https://github.com/ballerina-platform/ballerina-library).
This repository only contains the source code for the package.
## Building from the source
### Prerequisites
1. Download and install Java SE Development Kit (JDK) version 21. You can download it from either of the following sources:
- [Oracle JDK](https://www.oracle.com/java/technologies/downloads/)
- [OpenJDK](https://adoptium.net/)
> **Note:** After installation, remember to set the `JAVA_HOME` environment variable to the directory where JDK was installed.
2. Download and install [Ballerina Swan Lake](https://ballerina.io/).
3. Download and install [Docker](https://www.docker.com/get-started).
> **Note**: Ensure that the Docker daemon is running before executing any tests.
4. Generate a Github access token with read package permissions, then set the following `env` variables:
```bash
export packageUser=
export packagePAT=
```
### Build options
Execute the commands below to build from the source.
1. To build the package:
```bash
./gradlew clean build
```
2. To run the tests:
```bash
./gradlew clean test
```
3. To build the without the tests:
```bash
./gradlew clean build -x test
```
4. To debug package with a remote debugger:
```bash
./gradlew clean build -Pdebug=
```
5. To debug with Ballerina language:
```bash
./gradlew clean build -PbalJavaDebug=
```
6. Publish the generated artifacts to the local Ballerina central repository:
```bash
./gradlew clean build -PpublishToLocalCentral=true
```
7. Publish the generated artifacts to the Ballerina central repository:
```bash
./gradlew clean build -PpublishToCentral=true
```
## Contributing to Ballerina
As an open source project, Ballerina welcomes contributions from the community.
For more information, go to the [contribution guidelines](https://github.com/ballerina-platform/ballerina-lang/blob/master/CONTRIBUTING.md).
## Code of conduct
All contributors are encouraged to read the [Ballerina Code of Conduct](https://ballerina.io/code-of-conduct).
## Useful links
- For more information go to the [confluent.cavroderdes](https://central.ballerina.io/ballerinax/confluent.cavroserdes/latest) library.
- Discuss code changes of the Ballerina project in [[email protected]](mailto:[email protected]).
- Chat live with us via our [Discord server](https://discord.gg/ballerinalang).
- Post all technical questions on Stack Overflow with the [#ballerina](https://stackoverflow.com/questions/tagged/ballerina) tag.