{"id":18603592,"url":"https://github.com/cogini/confluent_schema_registry","last_synced_at":"2025-04-10T19:31:55.895Z","repository":{"id":36924933,"uuid":"214581995","full_name":"cogini/confluent_schema_registry","owner":"cogini","description":"Elixir client for the Confluent Schema Registry API https://www.confluent.io/confluent-schema-registry","archived":false,"fork":false,"pushed_at":"2024-07-22T21:42:47.000Z","size":53,"stargazers_count":9,"open_issues_count":0,"forks_count":0,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-04-07T05:44:15.329Z","etag":null,"topics":["avro","elixir","kafka","schema-registry-client","tesla"],"latest_commit_sha":null,"homepage":null,"language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cogini.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-10-12T05:13:15.000Z","updated_at":"2024-12-13T06:11:07.000Z","dependencies_parsed_at":"2022-08-08T18:30:19.098Z","dependency_job_id":null,"html_url":"https://github.com/cogini/confluent_schema_registry","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cogini%2Fconfluent_schema_registry","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cogini%2Fconfluent_schema_registry/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cogini%2Fconfluent_schema_registry/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cogini%2Fconfluent_schema_registry/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cogini","download_url":"https://codeload.github.com/cogini/confluent_schema_registry/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248281413,"owners_count":21077423,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["avro","elixir","kafka","schema-registry-client","tesla"],"created_at":"2024-11-07T02:14:50.968Z","updated_at":"2025-04-10T19:31:53.489Z","avatar_url":"https://github.com/cogini.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# confluent_schema_registry\n\nElixir client for the [Confluent® Schema Registry](https://www.confluent.io/confluent-schema-registry).\n\nIt implements the full [REST API](https://docs.confluent.io/current/schema-registry/develop/api.html) using\nthe [Tesla](https://github.com/teamon/tesla) HTTP client library. This gives it support for\nHTTP authentication, SSL/TLS and other configuration flexibility, e.g.\nselecting the underlying HTTP library (e.g. Hackney).\n\nThanks to [Schemex](https://hex.pm/packages/schemex) and [Avrora](https://github.com/Strech/avrora)\nfor inspiration.\n\n## Installation\n\nAdd `confluent_schema_registry` to your list of dependencies in `mix.exs`:\n\n```elixir\ndef deps do\n  [{:confluent_schema_registry, \"~\u003e 0.1.1\"}]\nend\n```\n\nThen run `mix deps.get` to fetch the new dependency.\n\nDocumentation is on [HexDocs](https://hexdocs.pm/confluent_schema_registry).\nTo generate a local copy, run `mix docs`.\n\n### Optional dependencies\n\nBy default, Tesla uses the [httpc](http://erlang.org/doc/man/httpc.html) HTTP client\nwhich comes with OTP. That library is pretty bare bones, and has issues with\ne.g. validating SSL certificates, so I recommend using [hackney](https://hex.pm/packages/hackney).\nConfigure it as documented in [Tesla.Adapter.Hackney](https://hexdocs.pm/tesla/Tesla.Adapter.Hackney.html).\n\n```elixir\nconfig :tesla, :adapter, Tesla.Adapter.Hackney\n```\n\n[AvroSchema](https://github.com/cogini/avro_schema) provides convenience functions\nto work  with Avro schemas, tag ids, encode/decode data, and cache results for performance\nand reliablilty.\n\n## Usage\n\nFirst create a client:\n\n```elixir\nclient = ConfluentSchemaRegistry.client(base_url: \"https://schemaregistry.example.com:8081/\")\n```\n\nIn the Kafka [wire format](https://docs.confluent.io/current/schema-registry/serializer-formatter.html#wire-format),\nthe data is prefixed by magic byte 0 to indicate that it is using the schema\nregistry, then four bytes for the schema id, then the data.\n\n### Producer\n\nOn startup, a producer looks up the version of the schema matching the\nsubject associated with the topic it is writing on. It encodes the data to\nbinary format using the schema, then appends the schema id to the binary and\nsends it to Kafka.\n\nThere are a couple of different options for how to get the schema, depending\non the policy and permissions for updating schemas.\n\n`is_registered/3` checks if a schema has already been registered under the\nspecified subject. If so, it returns the registration.\n\n```elixir\n schema = \"{\\\"type\\\":\\\"record\\\",\\\"name\\\":\\\"test\\\",\\\"fields\\\":[{\\\"name\\\":\\\"field1\\\",\\\"type\\\":\\\"string\\\"},{\\\"name\\\":\\\"field2\\\",\\\"type\\\":\\\"int\\\"}]}\"\ncase ConfluentSchemaRegistry.is_registered(client, \"test\", schema) do\n  {:ok, reg} -\u003e\n    # Found\n    schema = reg[\"schema\"]\n    schema_id = reg[\"id\"]\n  {:error, 404, %{\"error_code\" =\u003e 40401}} -\u003e\n    # Subject not found\n  {:error, 404, %{\"error_code\" =\u003e 40403}} -\u003e\n    # Schema not found\n  {:error, code, reason} -\u003e\n    # Other error\nend\n```\n\nIf the schema hasn't been registered, then the producer can attempt to\nregister it using `register_schema/3`. If it is successful, it returns the\nschema id. It can be called multiple times, and will return the same schema id.\n\n```elixir\ncase ConfluentSchemaRegistry.register_schema(client, \"test\", schema) do\n  {:ok, schema_id} -\u003e\n    # Already registered\n  {:error, 409, reason} -\u003e\n    # Conflict -- Incompatible Avro schema\n  {:error, 422, reason} -\u003e\n    # Unprocessable Entity, Invalid Avro schema\n  {:error, code, reason} -\u003e\n    # Other error\nend\n```\n\nAnother option is to use `get_schema/3` to read the latest registered schema\nfor the subject. You might do this when schema registrations are manually managed.\n\n```elixir\ncase ConfluentSchemaRegistry.get_schema(client, \"test\", \"latest\") do\n  {:ok, reg} -\u003e\n    # Already registered\n    schema = reg[\"schema\"]\n    schema_id = reg[\"id\"]\n  {:error, 404, %{\"error_code\" =\u003e 40401}} -\u003e\n    # Subject not found\n  {:error, 404, %{\"error_code\" =\u003e 40402}} -\u003e\n    # Version not found\n  {:error, 422, reason} -\u003e\n    # Unprocessable Entity, Invalid Avro version\n  {:error, code, reason} -\u003e\n    # Other error\nend\n```\n\n### Consumer\n\nA consumer receives data, gets the schema id from the prefix, and looks it up\nin the registry using `get_schema/1`, getting the schema which was used to write it.\nIt then decodes the binary using the schema.\n\n```elixir\n{:ok, schema} = ConfluentSchemaRegistry.get_schema(client, 21)\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcogini%2Fconfluent_schema_registry","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcogini%2Fconfluent_schema_registry","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcogini%2Fconfluent_schema_registry/lists"}