{"id":15525118,"url":"https://github.com/domnikl/schema-registry-gitops","last_synced_at":"2025-04-06T02:09:55.493Z","repository":{"id":43951242,"uuid":"324245338","full_name":"domnikl/schema-registry-gitops","owner":"domnikl","description":"Manage Confluent Schema Registry subjects through Infrastructure as code","archived":false,"fork":false,"pushed_at":"2025-03-20T15:05:18.000Z","size":1083,"stargazers_count":90,"open_issues_count":7,"forks_count":16,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-03-30T01:09:44.954Z","etag":null,"topics":["ci-cd","confluent","devops","infrastructure-as-code","kafka","schema-registry","schema-validation","yaml"],"latest_commit_sha":null,"homepage":"","language":"Kotlin","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/domnikl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":null,"patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":"https://paypal.me/DominikLiebler"}},"created_at":"2020-12-24T22:22:27.000Z","updated_at":"2025-03-20T15:05:04.000Z","dependencies_parsed_at":"2024-01-14T09:33:35.832Z","dependency_job_id":"f4abe768-b437-4b69-abfb-42a506d8dbcd","html_url":"https://github.com/domnikl/schema-registry-gitops","commit_stats":{"total_commits":493,"total_committers":6,"mean_commits":82.16666666666667,"dds":"0.27180527383367137","last_synced_commit":"af3a2ff99cebdbf290e73379912f58cd92450e7f"},"previous_names":[],"tags_count":19,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/domnikl%2Fschema-registry-gitops","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/domnikl%2Fschema-registry-gitops/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/domnikl%2Fschema-registry-gitops/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/domnikl%2Fschema-registry-gitops/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/domnikl","download_url":"https://codeload.github.com/domnikl/schema-registry-gitops/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247423515,"owners_count":20936626,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ci-cd","confluent","devops","infrastructure-as-code","kafka","schema-registry","schema-validation","yaml"],"created_at":"2024-10-02T10:54:51.429Z","updated_at":"2025-04-06T02:09:55.469Z","avatar_url":"https://github.com/domnikl.png","language":"Kotlin","funding_links":["https://paypal.me/DominikLiebler"],"categories":["\u003ca name=\"Kotlin\"\u003e\u003c/a\u003eKotlin"],"sub_categories":[],"readme":"# schema-registry-gitops\n\n[![build](https://github.com/domnikl/schema-registry-gitops/workflows/build/badge.svg)](https://github.com/domnikl/schema-registry-gitops/actions)\n[![Docker Pulls](https://img.shields.io/docker/pulls/domnikl/schema-registry-gitops)](https://hub.docker.com/repository/docker/domnikl/schema-registry-gitops)\n\u003ca href=\"https://codeclimate.com/github/domnikl/schema-registry-gitops/maintainability\"\u003e\u003cimg src=\"https://api.codeclimate.com/v1/badges/2e87990ad7212a273b49/maintainability\" /\u003e\u003c/a\u003e\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)\n\nManages subjects, compatibility levels and schema registration for [Confluent Schema Registry](https://docs.confluent.io/platform/current/schema-registry/index.html) through applying a desired state file.\n\n## Overview\n\nSchema Registry GitOps is an Infrastructure as Code tool that applies a desired state configured through simple YAML and\nAvro/Protobuf/JSON Schema files to a schema registry. That way you can keep a version control history of your\nschemas and use all your favorite tools to validate, review, merge and evolve schemas in your CI/CD pipeline.\n\n![Yaml (+Avro, Protobuf, JSON) -\u003e CI/CD -\u003e Schema Registry](schema-registry-gitops.svg)\n\n## Usage\n\n```\nUsage: schema-registry-gitops [-hvV] [--properties=\u003cpropertiesFilePath\u003e]\n                              [-r=\u003cbaseUrl\u003e] [COMMAND]\nManages schema registries through Infrastructure as Code\n  -h, --help                 Show this help message and exit.\n      --properties=\u003cpropertiesFilePath\u003e\n                             a Java Properties file for client configuration\n                               (optional)\n  -r, --registry=\u003cbaseUrl\u003e   schema registry endpoint, overwrites 'schema.\n                               registry.url' from properties, can also be a\n                               list of urls separated by comma\n  -v, --verbose              enable verbose logging\n  -V, --version              Print version information and exit.\nCommands:\n  apply  applies the state to the given schema registry\n  dump   prints the current state\n  plan   validate and plan schema changes, can be used to see all pending\n           changes\n```\n\nIn order to get help for a specific command, try `schema-registry-gitops \u003ccommand\u003e -h`.\n\n## Running in Docker\n\n`schema-registry-gitops` is available through [Docker Hub](https://hub.docker.com/repository/docker/domnikl/schema-registry-gitops), so running it in a container is as easy as:\n\n```sh\ndocker run -v \"$(pwd)/examples\":/data domnikl/schema-registry-gitops plan --properties /data/client.properties /data/base.yml\n```\n\nPlease keep in mind that using a tagged release may be a good idea.\n\n## State files\n\nThe desired state is managed using the following YAML schema:\n\n```yaml\n# sets global compatibility level (optional)\ncompatibility: FULL_TRANSITIVE\n# sets normalize flag (optional)\n# setting to true enables clients to not have to pass the “normalize” query parameter to have normalization occur\nnormalize: true\nsubjects:\n  # a subject that links to a file for the schema definition\n  - name: my-new-subject-referencing-a-schema-file\n    # sets compatibility level for this subject (optional)\n    compatibility: BACKWARD\n    # file paths are always relative to the given (this) YAML file\n    file: my-actual-schema.avsc\n    # AVRO is the default type and can safely be omitted (only available for Schema Registry \u003e= 5.5)\n    type: AVRO\n    # (optional) list of references for this subject\n    # please note that these must be present in the registry before they can be referenced here\n    references:\n      # name including the namespace, should be the same as the `type` being used in AVRO\n      - name: dev.domnikl.schema-registry-gitops.User\n        # subject name this schema is registered with in the registry\n        subject: User-value\n        # version of the referenced schema\n        version: 1\n\n  # another example: instead of referencing a file, it is also possible\n  # to define the schema directly here, which is Protocol Buffers here (note explicit type here)\n  - name: my-new-inline-schema-subject\n    schema: 'syntax = \"proto3\";\n        package com.acme;\n\n        message OtherRecord {\n          int32 an_id = 1;\n        }'\n    type: PROTOBUF\n```\n\nWhen using multiple files, the state is the merge result of those. Keep in mind that later references of the same subject names will overwrite earlier definitions.\n\n### compatibility\n\nSupported `compatibility` values are:\n* `NONE`\n* `FORWARD`\n* `BACKWARD`\n* `FULL`\n* `FORWARD_TRANSITIVE`\n* `BACKWARD_TRANSITIVE`\n* `FULL_TRANSITIVE`\n\n### file, schema\n\nEither one of `file` (recommended) or `schema` must be set. The former contains a path to a schema file while the latter can be set\nto a string containing the schema.\n\n### type\n\nSupported `type` values are:\n\n* `AVRO`\n* `PROTOBUF`\n* `JSON`\n\n_Please note that `PROTOBUF` and `JSON` are only supported for Schema Registry \u003e= 5.5, versions prior to that only support `AVRO`._\n\n### references\n\n[References to other schemas](https://docs.confluent.io/platform/current/schema-registry/serdes-develop/index.html#referenced-schemas)\nare being configured here as an optional list of references. `name`, `subject` and `version` need to be configured for\nthis to work. Also note that referenced schemas need to be present in the schema registry by the time that\n`schema-registry-gitops` runs.\n\n## Configuration .properties\n\nConfiguration properties are being used to connect to the Schema Registry. The most common use case to use them\ninstead of just supplying `--registry` is to use SSL. The example below uses client certificates authentication.\n\n```properties\nschema.registry.url=https://localhost:8081\nsecurity.protocol=SSL\nschema.registry.ssl.truststore.location=truststore.jks\nschema.registry.ssl.truststore.password=\u003csecret\u003e\nschema.registry.ssl.keystore.location=keystore.jks\nschema.registry.ssl.keystore.password=\u003csecret\u003e\nschema.registry.ssl.key.password=\u003csecret\u003e\n```\n\n## Environment variables\n\nEnv variables prefixed with `SCHEMA_REGISTRY_GITOPS_` can be provided for configuration and will also be forwarded\nto configure the schema registry client being used. This example uses the same settings above.\n\n```sh\nSCHEMA_REGISTRY_GITOPS_SCHEMA_REGISTRY_URL=https://localhost:8081\nSCHEMA_REGISTRY_GITOPS_SECURITY_PROTOCOL=SSL\nSCHEMA_REGISTRY_GITOPS_SCHEMA_REGISTRY_SSL_TRUSTSTORE_LOCATION=truststore.jks\nSCHEMA_REGISTRY_GITOPS_SCHEMA_REGISTRY_SSL_TRUSTSTORE_PASSWORD=\u003csecret\u003e\nSCHEMA_REGISTRY_GITOPS_SCHEMA_REGISTRY_SSL_KEYSTORE_LOCATION=keystore.jks\nSCHEMA_REGISTRY_GITOPS_SCHEMA_REGISTRY_SSL_KEYSTORE_PASSWORD=\u003csecret\u003e\nSCHEMA_REGISTRY_GITOPS_SCHEMA_REGISTRY_SSL_KEY_PASSWORD=\u003csecret\u003e\n```\n\n## Deleting subjects ⚠️\n\nSubjects no longer listed in the State file but present in the registry will not be deleted by default. To enable full\nsync between the two, use either `-d` or `--enable-deletes` in `plan` and `apply` modes.\n\n## Development\n\nDocker is used to build and test `schema-registry-gitops` for development.\n\n```sh\n# test \u0026 build\ndocker build -t domnikl/schema-registry-gitops .\n\n# run it in Docker\ndocker run -v ./examples:/data domnikl/schema-registry-gitops plan --registry http://localhost:8081 /data/base.yml /data/with_references.yml\n```\n\n## Acknowledgement\n\nSchema Registry GitOps was born late in 2020 while being heavily inspired by [Shawn Seymour](https://github.com/devshawn) and his excellent [kafka-gitops](https://github.com/devshawn/kafka-gitops)! Much ❤ to [Confluent](https://www.confluent.io/) for building Schema Registry and an amazing client lib, I am really just standing on the shoulders of giants here.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdomnikl%2Fschema-registry-gitops","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdomnikl%2Fschema-registry-gitops","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdomnikl%2Fschema-registry-gitops/lists"}