{"id":24117683,"url":"https://github.com/thake/avro4k-kafka-serializer","last_synced_at":"2025-10-05T01:56:18.989Z","repository":{"id":43323141,"uuid":"243688275","full_name":"thake/avro4k-kafka-serializer","owner":"thake","description":"A Kafka Serializer and SerDe implementation based on Avro4k serializers","archived":false,"fork":false,"pushed_at":"2024-08-06T23:52:52.000Z","size":166,"stargazers_count":21,"open_issues_count":2,"forks_count":9,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-09-18T06:47:07.137Z","etag":null,"topics":["avro","kafka-serializer","kotlin","schema-registry","serde"],"latest_commit_sha":null,"homepage":"","language":"Kotlin","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thake.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2020-02-28T05:48:52.000Z","updated_at":"2023-08-23T23:35:57.000Z","dependencies_parsed_at":"2025-09-18T06:33:12.171Z","dependency_job_id":"056c79cc-f34b-4bfe-a5a0-c14005135a7a","html_url":"https://github.com/thake/avro4k-kafka-serializer","commit_stats":null,"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"purl":"pkg:github/thake/avro4k-kafka-serializer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thake%2Favro4k-kafka-serializer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thake%2Favro4k-kafka-serializer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thake%2Favro4k-kafka-serializer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thake%2Favro4k-kafka-serializer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thake","download_url":"https://codeload.github.com/thake/avro4k-kafka-serializer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thake%2Favro4k-kafka-serializer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278399690,"owners_count":25980332,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-04T02:00:05.491Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["avro","kafka-serializer","kotlin","schema-registry","serde"],"created_at":"2025-01-11T07:55:33.866Z","updated_at":"2025-10-05T01:56:18.973Z","avatar_url":"https://github.com/thake.png","language":"Kotlin","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Kafka avro4k serializer / deserializer\n[![Maven Central](https://maven-badges.herokuapp.com/maven-central/com.github.thake.avro4k/avro4k-kafka-serializer/badge.svg)](https://maven-badges.herokuapp.com/maven-central/com.github.thake.avro4k/avro4k-kafka-serializer)\n\nThis project implements a Kafka serializer / deserializer that integrates with the confluent schema registry and\nleverages [avro4k](https://github.com/avro-kotlin/avro4k). It is based on confluent's [Kafka Serializer]((https://github.com/confluentinc/schema-registry/tree/master/avro-serializer)).\nAs such, this implementations can be used to in several projects (i.e. spring)\n\nThis SerDe supports retrying of failed calls to the schema registry (i.e. due to flaky network). Confluent's Serde does not implement this yet.\nSee https://github.com/confluentinc/schema-registry/issues/928.\n\n## Confluent Versions\n\nVersion 0.10.x is compatible with Apache Kafka 2.5.x / Confluent 5.5.x and avro4k \u003c 1.0\n\nVersion 0.11.x+ is compatible with Apache Kafka 2.6.x / Confluent 6.0.0 and avro4k \u003c 1.0\n\nVersion \u003e 0.13 is compatible with Apache Kafka 2.6.x/3.x (Confluent 6.x/7.x) and avro4k 1.x\n\n## Example usage\n\nYou can find an example configuration of a Kafka Consumer, Kafka Producer and Kafka Streams application in the [ConfluentIT](./src/integrationTest/kotlin/com/github/thake/kafka/avro4k/serializer/ConfluentIT.kt) integration test.\n\n### Spring Cloud Stream with Kafka\n\n```javascript\nspring:\n    application:\n        name: \u003cyour-application\u003e\n            kafka:\n            bootstrap-servers:\n            -\n            \u003cyour-kafka-bootstrap-server\u003e\n                cloud:\n                stream:\n                default:\n                consumer:\n                useNativeDecoding: true\n                producer:\n                useNativeEncoding: true\n                kafka:\n                streams:\n                binder:\n                        brokers: \u003cyour-broker\u003e\n                        configuration:\n                            schema.registry.url: \u003cyour-registry\u003e\n                            schema.registry.retry.attemps: 3\n                            schema.registry.retry.jitter.base: 10\n                            schema.registry.retry.jitter.max: 5000\n                            record.packages: \u003cpackages-of-avro4k-classes\u003e\n                            default.key.serde: com.github.thake.kafka.avro4k.serializer.Avro4kSerde\n                            default.value.serde: com.github.thake.kafka.avro4k.serializer.Avro4kSerde\n                    default:\n                        producer:\n                            keySerde: com.github.thake.kafka.avro4k.serializer.Avro4kSerde\n                            valueSerde: com.github.thake.kafka.avro4k.serializer.Avro4kSerde\n                        consumer:\n                            keySerde: com.github.thake.kafka.avro4k.serializer.Avro4kSerde\n                            valueSerde: com.github.thake.kafka.avro4k.serializer.Avro4kSerde\n...\n```\n\n### Maven\n\n```xml\n\u003cdependency\u003e\n    \u003cgroupId\u003ecom.github.thake.avro4k\u003c/groupId\u003e\n    \u003cartifactId\u003eavro4k-kafka-serializer\u003c/artifactId\u003e\n    \u003cversion\u003eVERSION\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\n### Configuration options\n\n- `schema.registry.url`  \n  Comma-separated list of URLs for schema registry instances that can\n  be used to register or look up schemas. If you wish to get a\n  connection to a mocked schema registry for testing, you can specify\n  a scope using the `mock://` pseudo-protocol. For example,\n  `mock://my-scope-name` corresponds to\n  `MockSchemaRegistry.getClientForScope(\"my-scope-name\")`.\n\n  - Type: list\n  - Importance: high\n\n- `record.packages`  \n  The packages in which record types annotated with `@AvroName`,\n  `@AvroAlias` and `@AvroNamespace` can be found. Packages are separated\n  by a comma `,`. Only needed for deserialization.\n\n  - Type: string\n  - Default: null\n  - Importance: high\n\n- `schema.registry.ssl.key.password`  \n  The password of the private key in the key store file or the PEM key\n  specified in `ssl.keystore.key`. This is required for clients only\n  if two-way authentication is configured.\n\n  - Type: password\n  - Default: null\n  - Importance: high\n\n- `schema.registry.ssl.keystore.certificate.chain`  \n  Certificate chain in the format specified by `ssl.keystore.type`.\n  Default SSL engine factory supports only PEM format with a list of\n  X.509 certificates\n\n  - Type: password\n  - Default: null\n  - Importance: high\n\n- `schema.registry.ssl.keystore.key`  \n  Private key in the format specified by `ssl.keystore.type`. Default\n  SSL engine factory supports only PEM format with PKCS\\#8 keys. If\n  the key is encrypted, key password must be specified using\n  `ssl.key.password`\n\n  - Type: password\n  - Default: null\n  - Importance: high\n\n- `schema.registry.ssl.keystore.location`  \n  The location of the key store file. This is optional for client and\n  can be used for two-way authentication for client.\n\n  - Type: string\n  - Default: null\n  - Importance: high\n\n- `schema.registry.ssl.keystore.password`  \n  The store password for the key store file. This is optional for\n  client and only needed if `ssl.keystore.location` is configured. Key\n  store password is not supported for PEM format.\n\n  - Type: password\n  - Default: null\n  - Importance: high\n\n- `schema.registry.ssl.truststore.certificates`  \n  Trusted certificates in the format specified by\n  `ssl.truststore.type`. Default SSL engine factory supports only PEM\n  format with X.509 certificates.\n\n  - Type: password\n  - Default: null\n  - Importance: high\n\n- `schema.registry.ssl.truststore.location`  \n  The location of the trust store file.\n\n  - Type: string\n  - Default: null\n  - Importance: high\n\n- `schema.registry.ssl.truststore.password`  \n  The password for the trust store file. If a password is not set,\n  trust store file configured will still be used, but integrity\n  checking is disabled. Trust store password is not supported for PEM\n  format.\n\n  - Type: password\n  - Default: null\n  - Importance: high\n\n- `auto.register.schemas`  \n  Specify if the Serializer should attempt to register the Schema with\n  Schema Registry\n\n  - Type: boolean\n  - Default: true\n  - Importance: medium\n\n- `basic.auth.credentials.source`  \n  Specify how to pick the credentials for Basic Auth header. The\n  supported values are URL, USER\\_INFO and SASL\\_INHERIT\n\n  - Type: string\n  - Default: URL\n  - Importance: medium\n\n- `basic.auth.user.info`  \n  Specify the user info for Basic Auth in the form of\n  {username}:{password}\n\n  - Type: password\n  - Default: \\[hidden\\]\n  - Importance: medium\n\n- `bearer.auth.credentials.source`  \n  Specify how to pick the credentials for Bearer Auth header.\n\n  - Type: string\n  - Default: STATIC\\_TOKEN\n  - Importance: medium\n\n- `bearer.auth.token`  \n  Specify the Bearer token to be used for authentication\n\n  - Type: password\n  - Default: \\[hidden\\]\n  - Importance: medium\n\n- `context.name.strategy`  \n  A class used to determine the schema registry context.\n\n  - Type: class\n  - Default:\n    io.confluent.kafka.serializers.context.NullContextNameStrategy\n  - Importance: medium\n\n- `key.subject.name.strategy`  \n  Determines how to construct the subject name under which the key\n  schema is registered with the schema registry. By default,\n  \\\u003ctopic\\\u003e-key is used as subject.\n\n  - Type: class\n  - Default:\n    io.confluent.kafka.serializers.subject.TopicNameStrategy\n  - Importance: medium\n\n- `normalize.schemas`  \n  Whether to normalize schemas, which generally ignores ordering when\n  it is not significant\n\n  - Type: boolean\n  - Default: false\n  - Importance: medium\n\n- `schema.registry.basic.auth.user.info`  \n  Specify the user info for Basic Auth in the form of\n  {username}:{password}\n\n  - Type: password\n  - Default: \\[hidden\\]\n  - Importance: medium\n\n- `schema.registry.ssl.enabled.protocols`  \n  The list of protocols enabled for SSL connections. The default is\n  `TLSv1.2,TLSv1.3` when running with Java 11 or newer, `TLSv1.2`\n  otherwise. With the default value for Java 11, clients and servers\n  will prefer TLSv1.3 if both support it and fallback to TLSv1.2\n  otherwise (assuming both support at least TLSv1.2). This default\n  should be fine for most cases. Also see the config documentation for\n  \u003cspan class=\"title-ref\"\u003essl.protocol\u003c/span\u003e.\n\n  - Type: list\n  - Default: TLSv1.2,TLSv1.3\n  - Importance: medium\n\n- `schema.registry.ssl.keystore.type`  \n  The file format of the key store file. This is optional for client.\n\n  - Type: string\n  - Default: JKS\n  - Importance: medium\n\n- `schema.registry.ssl.protocol`  \n  The SSL protocol used to generate the SSLContext. The default is\n  `TLSv1.3` when running with Java 11 or newer, `TLSv1.2` otherwise.\n  This value should be fine for most use cases. Allowed values in\n  recent JVMs are `TLSv1.2` and `TLSv1.3`. `TLS`, `TLSv1.1`, `SSL`,\n  `SSLv2` and `SSLv3` may be supported in older JVMs, but their usage\n  is discouraged due to known security vulnerabilities. With the\n  default value for this config and `ssl.enabled.protocols`, clients\n  will downgrade to `TLSv1.2` if the server does not support\n  `TLSv1.3`. If this config is set to `TLSv1.2`, clients will not use\n  `TLSv1.3` even if it is one of the values in ssl.enabled.protocols\n  and the server only supports `TLSv1.3`.\n\n  - Type: string\n  - Default: TLSv1.3\n  - Importance: medium\n\n- `schema.registry.ssl.provider`  \n  The name of the security provider used for SSL connections. Default\n  value is the default security provider of the JVM.\n\n  - Type: string\n  - Default: null\n  - Importance: medium\n\n- `schema.registry.ssl.truststore.type`  \n  The file format of the trust store file.\n\n  - Type: string\n  - Default: JKS\n  - Importance: medium\n\n- `value.subject.name.strategy`  \n  Determines how to construct the subject name under which the value\n  schema is registered with the schema registry. By default,\n  \\\u003ctopic\\\u003e-value is used as subject.\n\n  - Type: class\n  - Default:\n    io.confluent.kafka.serializers.subject.TopicNameStrategy\n  - Importance: medium\n\n- `id.compatibility.strict`  \n  Whether to check for backward compatibility between the schema with\n  the given ID and the schema of the object to be serialized\n\n  - Type: boolean\n  - Default: true\n  - Importance: low\n\n- `latest.compatibility.strict`  \n  Whether to check for backward compatibility between the latest\n  subject version and the schema of the object to be serialized\n\n  - Type: boolean\n  - Default: true\n  - Importance: low\n\n- `max.schemas.per.subject`  \n  Maximum number of schemas to create or cache locally.\n\n  - Type: int\n  - Default: 1000\n  - Importance: low\n\n- `proxy.host`  \n  The hostname, or address, of the proxy server that will be used to\n  connect to the schema registry instances.\n\n  - Type: string\n  - Default: \"\"\n  - Importance: low\n\n- `proxy.port`  \n  The port number of the proxy server that will be used to connect to\n  the schema registry instances.\n\n  - Type: int\n  - Default: -1\n  - Importance: low\n\n- `schema.reflection`  \n  If true, uses the reflection API when serializing/deserializing\n\n  - Type: boolean\n  - Default: false\n  - Importance: low\n\n- `schema.registry.retry.attempts`  \n  Number of retry attempts that will be made if the schema registry\n  seems to have a problem with requesting a schema.\n\n  - Type: int\n  - Default: 5\n  - Importance: low\n\n- `schema.registry.retry.jitter.base`  \n  Milliseconds that are used as a base for the jitter calculation\n  (sleep = random\\_between(0, min(max, base \\* 2 \\*\\* attempt)))\n\n  - Type: long\n  - Default: 10\n  - Importance: low\n\n- `schema.registry.retry.jitter.max`  \n  Milliseconds that are used as max for the jitter calculation (sleep\n  = random\\_between(0, min(max, base \\* 2 \\*\\* attempt)))\n\n  - Type: long\n  - Default: 5000\n  - Importance: low\n\n- `schema.registry.ssl.cipher.suites`  \n  A list of cipher suites. This is a named combination of\n  authentication, encryption, MAC and key exchange algorithm used to\n  negotiate the security settings for a network connection using TLS\n  or SSL network protocol. By default all the available cipher suites\n  are supported.\n\n  - Type: list\n  - Default: null\n  - Importance: low\n\n- `schema.registry.ssl.endpoint.identification.algorithm`  \n  The endpoint identification algorithm to validate server hostname\n  using server certificate.\n\n  - Type: string\n  - Default: https\n  - Importance: low\n\n- `schema.registry.ssl.engine.factory.class`  \n  The class of type\n  org.apache.kafka.common.security.auth.SslEngineFactory to provide\n  SSLEngine objects. Default value is\n  org.apache.kafka.common.security.ssl.DefaultSslEngineFactory\n\n  - Type: class\n  - Default: null\n  - Importance: low\n\n- `schema.registry.ssl.keymanager.algorithm`  \n  The algorithm used by key manager factory for SSL connections.\n  Default value is the key manager factory algorithm configured for\n  the Java Virtual Machine.\n\n  - Type: string\n  - Default: SunX509\n  - Importance: low\n\n- `schema.registry.ssl.secure.random.implementation`  \n  The SecureRandom PRNG implementation to use for SSL cryptography\n  operations.\n\n  - Type: string\n  - Default: null\n  - Importance: low\n\n- `schema.registry.ssl.trustmanager.algorithm`  \n  The algorithm used by trust manager factory for SSL connections.\n  Default value is the trust manager factory algorithm configured for\n  the Java Virtual Machine.\n\n  - Type: string\n  - Default: PKIX\n  - Importance: low\n\n- `use.latest.version`  \n  Specify if the Serializer should use the latest subject version for\n  serialization\n\n  - Type: boolean\n  - Default: false\n  - Importance: low\n\n- `use.schema.id`\n  Schema ID to use for serialization\n  - Type: int\n  - Default: -1\n  - Importance: low\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthake%2Favro4k-kafka-serializer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthake%2Favro4k-kafka-serializer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthake%2Favro4k-kafka-serializer/lists"}