{"id":13692927,"url":"https://github.com/piotr-yuxuan/dove","last_synced_at":"2025-05-13T20:30:53.369Z","repository":{"id":57713640,"uuid":"171026117","full_name":"piotr-yuxuan/dove","owner":"piotr-yuxuan","description":"Infer specs from SpecificRecord Java classes of any Avro named type (record, fixed, or enum)","archived":false,"fork":false,"pushed_at":"2025-03-27T07:03:51.000Z","size":1035,"stargazers_count":27,"open_issues_count":2,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-27T08:23:17.820Z","etag":null,"topics":["avro","avro-format","avro-schema","clojure","clojure-spec","clojure-specs","spec","specific-schemas"],"latest_commit_sha":null,"homepage":"https://github.com/piotr-yuxuan/dove","language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/piotr-yuxuan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS.yml","security":".github/SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["piotr-yuxuan"]}},"created_at":"2019-02-16T16:38:46.000Z","updated_at":"2025-03-27T07:03:54.000Z","dependencies_parsed_at":"2024-03-12T20:27:08.926Z","dependency_job_id":"1b23527b-5dfa-495f-9112-dc53a5b4eeed","html_url":"https://github.com/piotr-yuxuan/dove","commit_stats":{"total_commits":181,"total_committers":4,"mean_commits":45.25,"dds":0.06629834254143652,"last_synced_commit":"20b625a57bcdf0aa7ce0700bd95cbb8040bb2c86"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/piotr-yuxuan%2Fdove","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/piotr-yuxuan%2Fdove/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/piotr-yuxuan%2Fdove/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/piotr-yuxuan%2Fdove/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/piotr-yuxuan","download_url":"https://codeload.github.com/piotr-yuxuan/dove/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254020902,"owners_count":22000805,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["avro","avro-format","avro-schema","clojure","clojure-spec","clojure-specs","spec","specific-schemas"],"created_at":"2024-08-02T17:01:03.572Z","updated_at":"2025-05-13T20:30:52.350Z","avatar_url":"https://github.com/piotr-yuxuan.png","language":"Clojure","funding_links":["https://github.com/sponsors/piotr-yuxuan"],"categories":["Avro \u0026 Confluent Schema Registry"],"sub_categories":[],"readme":"# 🕊 dove\n\n[![Build status](https://img.shields.io/github/workflow/status/piotr-yuxuan/dove/Walter%20CD)](https://github.com/piotr-yuxuan/dove/actions/workflows/walter-cd.yml)\n[![Clojars badge](https://img.shields.io/clojars/v/dove.svg)](https://clojars.org/piotr-yuxuan/dove)\n[![Clojars downloads](https://img.shields.io/clojars/dt/dove)](https://clojars.org/dove)\n[![cljdoc badge](https://cljdoc.org/badge/dove)](https://cljdoc.org/d/dove/dove/CURRENT)\n[![GitHub license](https://img.shields.io/github/license/piotr-yuxuan/dove)](https://github.com/piotr-yuxuan/dove/blob/main/LICENSE)\n[![GitHub issues](https://img.shields.io/github/issues/piotr-yuxuan/dove)](https://github.com/piotr-yuxuan/dove/issues)\n\n# What does it do?\n\nInfer specs from any Avro named type (record, fixed, or enum).\n\n# What can I use it for?\n\nGenerative testing and pre-serialisation validation.\n\n![avro aircraft](resources/avro.jpg)\n\nAvro was a British aircraft manufacturer. Nowadays it is a data\nserialization framework. A dove is smaller than an Avro aircraft, but\nit's softer and you can hold it in your hand, which makes it much more\nconvenient when dealing with data manually.\n\n# How to use it\n\nLet's take a couple of simple schemas and infer spec from them.\n\n``` avdl\n// Namespace as Java-like package name\n@namespace(\"com.bigCorp\")\n\nprotocol Messages {\n\n  /** Unique identifier. Will be cast to a UUID. */\n  fixed UID(16);\n\n  /** Category of a product. A product can only have zero of one category. */\n  enum Category {\n    FASHION, VEGETABLES, BEAUTY, SPORTSWEAR\n  }\n\n  /** Description of a product. Only fields `id` and `retailPrice` are required. */\n  record Product {\n    /** Identifier of this product. Will be cast to a UUID. */\n    UID id;\n\n    /** What a customer must pay to get one item. Gross profit for each sold product. */\n    decimal(8,1) retailPrice;\n\n    /** If known, count of available items of this product from our big warehouse. */\n    union { null, int } warehouseCount = null;\n\n    /** Ids of all the variants of this product family, including this very product id. */\n    array\u003cUID\u003e familyVariants = [];\n\n    /** Category of this product, if relevant */\n    union { null, Category } category = null;\n  }\n}\n```\n\nI hope comments make it quite easy to understand.\n\nDemo Avro schemas sources and classes are generated from IDL\nfiles with this Maven command:\n\n``` zsh\nmvn clean compile\n```\n\n``` clojure\n(ns dove.usage-demo\n  (:require [dove.core :as dove]\n            [clojure.spec.alpha :as s]\n            [clojure.spec.gen.alpha :as gen]\n            [clojure.test.check.generators :as test.g]\n            [clj-uuid :as uuid])\n  ;; These schemas are for the purpose of the demo.\n  (:import (com.bigCorp UID Category Product)\n           (java.nio ByteBuffer)\n           (java.util UUID)))\n```\n\nTurning a schema definition into a spec is straightforward with\n`dove/to-spec!`. It takes two arguments:\n\n- The schema which you want to recursively infer specs from.\n- Spec generation parameters. Defaults are `dove/convenient-args`,\n  which are equivalent to `{}`.\n\n``` clojure\n(dove/to-spec! (Category/getClassSchema) {})\n=\u003e :com.bigCorp/Category\n```\n\nThis returns the spec keyword.\n\n``` clojure\n(gen/generate (s/gen :com.bigCorp/Category))\n=\u003e \"VEGETABLES\"\n\n(gen/generate (s/gen :com.bigCorp/Category))\n=\u003e \"BEAUTY\"\n```\n\nClojure spec use a global registry. Dove mimics this and will ignore\nspecs it has already infered.\n\n``` clojure\n@dove/ignored-specs\n=\u003e #{:com.bigCorp/Category}\n```\n\nLet's infer a spec for a named, fixed type:\n\n``` clojure\n(dove/to-spec! (UID/getClassSchema) {})\n=\u003e :com.bigCorp/UID\n```\n\nThis spec matches the definition of a fixed type, so it's\nvalid. However it doesn't give useful sample.\n\n``` clojure\n(gen/generate (s/gen :com.bigCorp/UID))\n=\u003e #object[\"[B\" 0x5b37ba4c \"[B@5b37ba4c\"]\n```\n\nLet's redefine this spec to something more idiomatic.\n\n``` clojure\n(defn bytes-\u003euuid-v4\n  ^UUID [^bytes b]\n  (let [buffer ^ByteBuffer (ByteBuffer/wrap b)]\n    (uuid/v4 (.getLong buffer) (.getLong buffer))))\n\n(s/def :com.bigCorp/UID\n  (s/with-gen\n    uuid/uuid?\n    #(test.g/fmap bytes-\u003euuid-v4 (s/gen (dove/-\u003eavro-fixed? 16)))))\n\n(gen/generate (s/gen :com.bigCorp/UID))\n=\u003e #uuid\"d28df78a-8aa2-4b8c-b261-b0286581c865\"\n```\n\nAt this point two specs are known to have been infered and won't be\nfurther altered. You can also use that to instruct `dove` to leave\nsome specs untouched if you are willing to define them yourself.\n\n``` clojure\n@dove/ignored-specs\n=\u003e #{:com.bigCorp/Category :com.bigCorp/UID}\n```\n\nNow let's infer the spec for something bigger.\n\n``` clojure\n(dove/to-spec! (Product/getClassSchema) {})\n=\u003e :com.bigCorp/Product\n```\n\n``` clojure\n(gen/generate (s/gen :com.bigCorp/Product))\n=\u003e {:category \"BEAUTY\",\n    :warehouseCount 3\n    :familyVariants [#uuid\"de3ad2cd-551e-4b5d-a144-6f018ea38450\"],\n    :retailPrice 12.4M,\n    :id #uuid\"44857f13-f64c-4c16-9c33-bc83b6602213\"}\n```\n\nSome fields are not mandatory because of `union { null, … }`.\nLikewise, they can be missing from a sample:\n\n``` clojure\n(gen/generate (s/gen :com.bigCorp/Product))\n=\u003e {:familyVariants [],\n    :retailPrice 3476.3M,\n    :id #uuid\"9ab4e463-14fc-4226-ad26-ccdf22e77263\"}\n```\n\n# Further parameters\n\nThe previous examples focus on the ease of use and keep things\nsimple. Here are parameters you can tune to adapt `dove` to your\nneeds:\n\n``` clojure\n;; in `dove.core`\n(def convenient-args\n  \"These args are not meant to be your default choice, but they are\n  somehow convenient to use.\"\n  {:dry-run? false\n   :ns-keys? false\n   :enum-obj? false\n   :required-union-nil-value? false\n   :dove.spec/keyword dove-spec-keyword})\n```\n\n- When `:dry-run?` is `true`, `dove` will not define any spec but\n  instead print its name.\n- When `:ns-keys?` is `true`, keys of generated sample will be\n  namespaced in a Datomic-like way.\n- When `:enum-obj?` is `true`, generated sample of `enum` will be\n  actual `Enum`f instances and not mere strings.\n- When `:required-union-nil-value?` is `true`, all records fields will\n  be present in a sample. Fields which type is `union { null, … }`\n  could otherwise be missing.\n- When `:dove.spec/keyword` is filled, each generated record sample is\n  added its name under this key. This can be helpful to unambiguously\n  generate Avro record from maps.\n\n``` clojure\n(reset! dove/ignored-specs #{:com.bigCorp/UID})\n;; =\u003e #{:com.bigCorp/UID}\n\n(dove/to-spec! (Product/getClassSchema)\n               {:dry-run? false\n                :ns-keys? true\n                :enum-obj? true\n                :required-union-nil-value? true\n                :dove.spec/keyword dove/dove-spec-keyword})\n\n(gen/generate (s/gen :com.bigCorp/Product))\n=\u003e {:com.bigCorp.Product/category #object[com.bigCorp.Category 0x24246526 \"VEGETABLES\"],\n    :com.bigCorp.Product/familyVariants [#uuid\"182f170e-86d8-4c82-b067-999593756478\"\n                                         #uuid\"b7c292ee-a74e-48c4-8d5e-489d636d56d4\"\n                                         #uuid\"ea8284a0-c6e4-4690-8783-7dcd50d5e9bd\"],\n    :com.bigCorp.Product/warehouseCount 73,\n    :com.bigCorp.Product/retailPrice 0.2M,\n    :com.bigCorp.Product/id #uuid\"f3ef308b-2bce-4664-95b9-eb7a1aa78fd6\",\n    :dove.spec/name :com.bigCorp/Product}\n```\n\n# Troubleshoot\n\nAny behaviour different from [Avro 1.8.2\nspecification](https://avro.apache.org/docs/1.8.2/spec.html) should be\nconsidered a bug.\n\nThis library is currently under development. Feel free to give me any\nfeedback and I'll be more than happy to help you.\n\n# Related projects\n\nI've got a great deal of inspiration from these projects. They might\nsuit your needs better than dove. Each of them is pretty impressive!\n\n- [lancaster](https://github.com/deercreeklabs/lancaster) from [Chad\n  Harrington](https://github.com/chadharrington)\n- [spec-tools](https://github.com/metosin/spec-tools) from [Tommi\n  Reiman](https://github.com/ikitommi)\n- [spec-provider](https://github.com/stathissideris/spec-provider)\n  from [Stathis Sideris](https://github.com/stathissideris)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpiotr-yuxuan%2Fdove","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpiotr-yuxuan%2Fdove","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpiotr-yuxuan%2Fdove/lists"}