{"id":32183950,"url":"https://github.com/lab-79/dspec","last_synced_at":"2026-02-21T15:07:09.869Z","repository":{"id":62433258,"uuid":"67164954","full_name":"lab-79/dspec","owner":"lab-79","description":"Data specifications that compile to clojure.specs and Datomic schemas","archived":false,"fork":false,"pushed_at":"2017-01-31T19:22:37.000Z","size":250,"stargazers_count":22,"open_issues_count":6,"forks_count":3,"subscribers_count":6,"default_branch":"master","last_synced_at":"2026-02-16T07:18:12.852Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lab-79.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-09-01T20:43:29.000Z","updated_at":"2020-05-09T12:01:40.000Z","dependencies_parsed_at":"2022-11-01T21:01:30.335Z","dependency_job_id":null,"html_url":"https://github.com/lab-79/dspec","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"purl":"pkg:github/lab-79/dspec","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lab-79%2Fdspec","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lab-79%2Fdspec/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lab-79%2Fdspec/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lab-79%2Fdspec/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lab-79","download_url":"https://codeload.github.com/lab-79/dspec/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lab-79%2Fdspec/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29684106,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-21T14:31:22.911Z","status":"ssl_error","status_checked_at":"2026-02-21T14:31:22.570Z","response_time":107,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-21T23:25:14.114Z","updated_at":"2026-02-21T15:07:09.859Z","avatar_url":"https://github.com/lab-79.png","language":"Clojure","funding_links":[],"categories":[],"sub_categories":[],"readme":"# dspec\n\nA Clojure library designed to add useful semantics on top of Datomic.\n\n[![Build Status](https://travis-ci.org/lab-79/dspec.svg?branch=master)](https://travis-ci.org/lab-79/dspec)\n[![Clojars Project](https://img.shields.io/clojars/v/lab79/dspec.svg)](https://clojars.org/lab79/dspec)\n[![codecov](https://codecov.io/gh/lab-79/dspec/branch/master/graph/badge.svg)](https://codecov.io/gh/lab-79/dspec)\n\n## Purpose\n\nAdding additional semantics -- on top of those provided by Datomic --\nhelps us achieve the following:\n\n1. Documentation for engineers. Document the association of a collection\n   of attributes with a particular entity \"type\" (or \"types\") -- an association\n   that is captured implictly via Datomic queries that do not explicitly by\n   themselves make it easy for a developer to look in one place to understand\n   the different kinds of entities and data in the project.\n2. Eliminate duplication. From just one definition, we can:\n   1. Generate the datoms defining the Datomic schema.\n   2. Create corresponding `clojure.spec` definitions, that we can leverage for:\n      1. Validating data.\n      2. Generating sample or test data.\n\n## Goals\n\nThis library will help unify the following goals, from one central schema\nspecification.\n\n1. (Done) Provide a more readable way to read a Datomic schema and understand\n   how it is being used in an app.\n2. (Done) Generate schema edn for Datomic schema specifications.\n3. (Done) Generate `clojure.spec` spec definitions.\n4. Provide easy, extensible validation tagging (e.g., `:required`).\n5. (Done) Generate fake dev data or test data, by providing additional\n   semantics (e.g., what entity types can map to what other entity types via\n   Datomic `ref`s).\n6. Integrates with a nice way to add or retract from a Datomic db schema when\n   these semantic schema definitions change.\n7. (Done) Make it easy to extract specific *kinds* of entities.\n8. (Done) Conveniently detect what interfaces an entity satisfies.\n9. (Done) Support polymorphism.\n10. Small code base.\n11. Simple design.\n\n## Getting started\n\nAdd the following dependency to your `project.clj` file:\n\n[![Clojars Project](https://clojars.org/lab79/dspec/latest-version.svg)](https://clojars.org/lab79/dspec)\n\n\n## Usage\n\nWe adopt a declarative edn specification that we call \"interfaces\" for defining\nstronger semantics on top of Datomic schemas.\n\nAs a developer using this library, most of your application of this library\nwill be to convert some collection of data interfaces to:\n\n1. To Datomic schema attributes that will be added to the Datomic schema.\n2. To validation output that specifies if a vector of interfaces is invalid or\n   in what ways they are invalid.\n3. To `clojure.spec` definitions that we can use to check if a Datomic entity map\n   is invalid and in what ways it is invalid.\n\nArchitecturally, we convert all data interfaces into a single intermediate AST\nrepresenting our entire world of interfaces, enums, and their relationships\nwith each other (inheritance, references, etc.). Then our library converts this\nAST to either Datomic schemas, generated app data, or validation output that we\nmentioned above.\n\n### Defining dspec interfaces\n\nIn `dspec`, our semantic definitions are defined as data.\n\n```clojure\n(def user-spec\n  {:interface.def/name :interface/user\n   :interface.def/fields\n     {:user/username [:string \"A user's username\"\n                             :unique/identity ; can also be :unique/value\n                             :required]\n      :taggable/tags [[:string] \"Tags for an entity\"]\n      :user/registeredAt [:instant \"When a user registered\"]}\n   :interface.def/identify-via ['[?e :user/username]]})\n```\n\nThis defines a data interface named `:interface/user` such that an entity that\n\"satisfies\" the interface has the following attributes:\n\n- A string attribute named `:user/username` that is required and uniquely\nidentifies a given user.\n- An attribute named `:taggable/tags` that is a vector of strings. This\ncorresponds to a Datomic cardinality of `:db.cardinality/many` and is simply\nspecified by placing the member type inside a vector.\n- An attribute named `:user/registeredAt` that is of type instant.\n\nIt also specified how we can recognize this entity as implementing the\n`:interface/user` interface -- in this case because entities of this kind\nof interface have the attribute `:user/username` (indicated by\n`:interface.def/identify-via`).\n\nNotice how we can inline docstrings for each attribute. This helps us\nunderstand what each attribute is in a semantic definition. And as we will see\nlater, it also maps to the `:db/doc` field of a Datomic attribute.\n\n#### Primitive types\n\nThe possible types we can specify for an attribute are:\n\n- `:keyword` (or `[:keyword]`)\n- `:string` (or `[:string]`)\n- `:boolean` (or `[:boolean]`)\n- `:long` (or `[:long]`)\n- `:bigint` (or `[:bigint]`)\n- `:float` (or `[:float]`)\n- `:double` (or `[:double]`)\n- `:bigdec` (or `[:bigdec]`)\n- `:instant` (or `[:instant]`)\n- `:uuid` (or `[:uuid]`)\n- `:uri` (or `[:uri]`)\n- `:bytes` (or `[:bytes]`)\n\nA type signature in vector form (e.g., `[:string]`) implies a \"has many\"\nfield -- or in other words, a field that can have multiple members, i.e.,\n`:db.cardinality/many`.\n\n#### Enum Types\n\nAn attribute can also take on an enumerated set of values.\n\n```clojure\n{:interface.def/name :interface/person\n :interface.def/fields\n   {:person/gender [\"A person's gender\"\n                    :enum {:person.gender/male \"Male\"\n                           :person.gender/female  \"Female\"\n                           :person.gender/other \"Other\"}]}\n :interface.def/identify-via :datomic-spec/interfaces\n :interface.def/identifying-enum-part :db.part/user}\n\n; or\n{:interface.def/name :person\n :interface.def/fields\n   {:person/gender [:enum #{:person.gender/male\n                            :person.gender/female\n                            :person.gender/other]\n                            \"A person's gender\"}}\n :interface.def/identify-via :datomic-spec/interfaces\n :interface.def/identifying-enum-part :db.part/user}\n```\n\nThe type is specified by the keyword `:enum` immediately followed by either a\nset of enumerated values or a map of enumerated values and their corresponding\ndocstrings.\n\n#### Special Field Flags\n\nThere are a number of special flags that we can include in our field definitions.\nFor example, if we want to specify a field as one that must be required, we can add\nthe `:required` flag to the field definition:\n\n```clojure\n{:interface.def/name :interface/user\n :interface.def/fields\n   {:user/username [:string \"A user's username\" :required]\n    :user/password [:string \"A user's hashed password\" :required]}\n :interface.def/identify-via ['[?e :user/username]]}\n```\n\nThe field flags that come pre-packaged with `dspec` include:\n\n- `:required`\n  Means the field should be non-nil. It will generate a `clojure.spec` definition\n  that requires this particular attribute as a key in the map.\n- `:db.unique/value`\n  Results in Datomic schema generation of a Datomic attribute that has\n  `:db.unique/value` set to `true`.\n- `:db.unique/identity`\n  Results in Datomic schema generation of a Datomic attribute that has\n  `:db.unique/identity` set to `true`.\n- `:db/isComponent`\n  Results in Datomic schema generation of a Datomic attribute that has\n  `:db/isComponent` set to `true`.\n- `:db/index`\n  Results in Datomic schema generation of a Datomic attribute that has\n  `:db/index` set to `true`.\n- `:db/noHistory`\n  Results in Datomic schema generation of a Datomic attribute that has\n  `:db/noHistory` set to `true`.\n- `:db/fulltext`\n  Results in Datomic schema generation of a Datomic attribute that has\n  `:db/fulltext` set to `true`.\n- `:gen/should-generate`\n  Forces the field to be generated -- by, e.g. `clojure.spec.gen/generate`\n  or `clojure.spec.gen/sample` -- even if the field is not `:required`.\n\n#### Relationships between interfaces\n\nIn Datomic, a `:db.type/ref` entity can refer to any other entity. Usually, we want\nto define more constraints about the kinds of relationships an entity can have with\nother \"kinds\" of entities.\n\nWe offer a way to specify how entities map to each other.\n\nTake the example of a person having many names (e.g., maiden, married, etc.).\nHere is how we can represent that succinctly:\n\n```clojure\n[{:interface.def/name :interface/person\n  :interface.def/fields\n    {:person.id/ssn [:string \"A person's ssn number\"\n                             :db.unique/value]\n     :person/name [[:person/name] \"A person's names\"]}\n     ; The 2nd :person/name refers to the entity type defined below (see [XYZ])\n  :interface.def/identify-via ['[?e :person/name]]}\n\n\n ; Notice how we can define more than one schema type in a single edn map.\n ; Here we will define a :person/name type\n ; [XYZ] Can define more than one type in a given edn file. For convenience, we\n ; define :person/name here because it is used in the :person type above\n {:interface.def/name :person/name\n  :interface.def/fields\n    {:person.name/given [:string \"A given part of a person's name\"]\n     :person.name/family [:string \"The family part of a person's name\"]}\n  :interface.def/identify-via ['[?e :person.name/given]]}]\n```\n\nThe above interface definitions can generate the following Datomic schema:\n\n```clojure\n[{:db/id #db/id[:db.part/db]\n  :db/doc \"A person's names\"\n  :db/ident :person/name\n  :db/valueType :db.type/ref\n  :db/cardinality :db.cardinality/many\n  :db/isComponent true\n  :db.install/_attribute :db.part/db}\n {:db/id #db/id[:db.part/db]\n  :db/doc \"A given part of a person's name\"\n  :db/ident :person.name/given\n  :db/valueType :db.type/string\n  :db/cardinality :db.cardinality/one\n  :db.install/_attribute :db.part/db}\n {:db/id #db/id[:db.part/db]\n  :db/doc \"The family part of a person's name\"\n  :db/ident :person.name/family\n  :db/valueType :db.type/string\n  :db/cardinality :db.cardinality/one\n  :db.install/_attribute :db.part/db}]\n```\n\nNotice how we lose the semantics of the kinds of entities refs can map to.\nThis is expected, since Datomic does not provide that level of specificity.\n\nThe benefit of adding the additional specificity of ref constraints is the following:\n\n1. It self-documents how we intend to use specific `:db.type/ref` attributes.\n2. It helps us generate test data that maps to those intentions (see Section farther\n   down about data generation).\n3. It can be used to do some `clojure.spec` validation to enforce these constraints\n   before that information is added to Datomic, where these constraints are not enforced.\n\n\n#### Polymorphism\n\nWe chose the term `interface` because we wanted to signal how flexible we can be\nwhen we interpret data in Datomic. An entity can possess the interfaces of\n`:interface/person`, `:interface/mother`, and `:interface/daughter`.\n\nThis interface-based polymorphism is demonstrated in the following example, where\na patient and a practitioner are both persons as well.\n\n```clojure\n[{:interface.def/name :interface/person\n  :interface.def/fields\n    {:person/name [:string]}\n  :interface.def/identify-via :datomic-spec/interfaces\n  :interface.def/identifying-enum-part :db.part/user}\n\n {:interface.def/name :interface/patient\n  :interface.def/inherits [:interface/person]\n  :interface.def/fields\n    {:patient/physicians [[:interface/physician] \"A patient's physicians\"]}\n  :interface.def/identify-via :datomic-spec/interfaces\n  :interface.def/identifying-enum-part :db.part/user}\n\n {:interface.def/name :interface/physician\n  :interface.def/inherits #{:interface/person}\n  :interface.def/fields\n    {:physician/specialties [[:string] \"The physician's medical specialty or specialties.\"]}\n  :interface.def/identify-via :datomic-spec/interfaces\n  :interface.def/identifying-enum-part :db.part/user}]\n```\n\nThe above specifies that the `:patient` entity type and the `:practitioner`\nentity type will both inherit the type definition of `:person`.\n\nThe benefits of such an approach are that:\n\n1. We keep our interface definitions DRY.\n\n##### Detecting the interfaces an entity can represent\n\nIf we read an entity from a Datomic database, how can we determine what interfaces\nit implements? We will only be able to rely on the attributes of an entity to figure\nthis out. We can leverage attributes to identify entities in one of two ways.\n\n1. Via Datomic datalog query clauses that specify a precise way to detect an entity\n   via possession of an attribute (or attributes) that is (or are) always present\n   (i.e., `:required`).\n\n    ```clojure\n    {:interface.def/name :interface/automobile\n     :interface.def/fields\n       {:automobile/license-plate [:string :required]}\n     :interface.def/identify-via ['[?e :automobile/license-plate]]}\n    ```\n\n    Since we are using Datomic datalog syntax, we can use datalog itself to specify\n    more complex detection queries. In this example, an `:interface/automobile` is one\n    that has either a `:automobile/license-plate` or `:automobile/registration-id`:\n\n    ```clojure\n    {:interface.def/name :interface/automobile\n     :interface.def/fields\n       {:automobile/make [:string :required]\n        :automobile/model [:string :required]}\n     :interface.def/identify-via ['(or [?e :automobile/license-plate]\n                                       [?e :automobile/registration-id])]}\n    ```\n\n    And in this example, an `:interface/automobile` is one that has both\n    `:automobile/make` and `:automobile/model` attribute values:\n\n    ```clojure\n    {:interface.def/name :interface/automobile\n     :interface.def/fields\n       {:automobile/make [:string :required]\n        :automobile/model [:string :required]}\n     :interface.def/identify-via ['[?e :automobile/make]\n                                  '[?e :automobile/model]]}\n    ```\n\n2. Via a special attribute `:datomic-spec/interfaces` that is a many-cardinality\n   attribute of Datomic enums where an enum value names an interface that the entity\n   can be interpreted as.\n\n    ```clojure\n    {:interface.def/name :interface/automobile\n     :interface.def/fields {:automobile/license-plate [:string]}\n     :interface.def/identify-via :datomic-spec/interfaces\n     :interface.def/identifying-enum-part :db.part/autos}\n    ```\n\n    This comes in handy when we cannot rely on a particular value at an attribute for\n    identifying an entity. For example, we may want to specify someone as being an\n    `:interface/patient` in addition to an `:interface/person`, but the only attribute\n    the entity may have (besides `:db/id`) is `:person/name`. In this case, the only\n    way we could also identify a person as also being a patient is if we maintain an\n    attribute that the entity can use to self-label itself as an `:interface/patient`.\n    That special, reserved attribute is `:datomic-spec/interfaces`. It's Datomic schema\n    looks like:\n\n    ```clojure\n    {:db/id #db/id[:db.part/db]\n     :db/ident :datomic-spec/interfaces\n     :db/valueType :db.type/ref\n     :db/cardinality :db.cardinality/many\n     :db/index true\n     :db.install/_attribute :db.part/db}\n    ```\n\n    `:datomic-spec/interfaces` takes on a value that is a set of Datomic enums. For example,\n    in the `:interface/automobile` example, the Datomic schema will include:\n\n    ```clojure\n    {:db/id #db/id[:db.part/autos]\n     :db/ident :interface/automobile}\n    ```\n\n    The enum value is stored in the partition specified under the interface definition\n    key `:interface.def/identifying-enum-part`.\n\nWe choose to force you to think about how you will enforce this so that you do not lose\nany information about an entity's intended interfaces after you write it to the database.\nAn interface definition is not valid unless you specify `:interface.def/identify-via`.\n\nWe also provide convenience methods to determine the interfaces that a\nparticular entity or map loaded into memory satisfies:\n\n```clojure\n(let [entity {:db/id (datomic/tempid :db.part/user)\n              :automobile/make \"Toyota\"\n              :automobile/model \"Prius\"}]\n(lab79.dspec/entity-\u003einterfaces ast entity datomic.api/q)\n; =\u003e #{:interface/automobile}\n\n(let [entity {:db/id (datomic/tempid :db.part/user)}]\n(lab79.dspec/entity-\u003einterfaces ast entity datomic.api/q)\n; =\u003e #{}\n```\n\n`(lab79.dspec/satisfies-interface? ast interface-name entity datomic-q)`\n\n```clojure\n(let [entity {:db/id (datomic/tempid :db.part/user)\n              :automobile/make \"Toyota\"\n              :automobile/model \"Prius\"}]\n  (lab79.dspec/satisfies-interface? ast :interface/automobile entity datomic.api/q))\n; =\u003e true\n\n(let [entity {:db/id (datomic/tempid :db.part/user)}]\n  (lab79.dspec/satisfies-interface? ast :interface/automobile entity datomic.api/q))\n; =\u003e false\n```\n\n`(lab79.dspec/eid-satisfies-interface? ast interface-name entity-id datomic-q datomic-filter db)`\n\n```clojure\n(let [entity {:db/id (datomic/tempid :db.part/user)\n              :automobile/make \"Toyota\"\n              :automobile/model \"Prius\"}\n      tempid (:db/id entity)\n      {db :db-after\n      tempids :tempids} @(d/transact conn [entity])\n      eid (d/resolve-tempid db tempids tempid)]\n  (lab79.dspec/eid-satisfies-interface? ast :interface/automobile eid datomic.api/q datomic.api/filter db))\n; =\u003e true\n```\n\nIf you want to incorporate the query clauses to limit your Datomic query to a\ngiven set of interfaces, you can use `identify-via-clauses-for`, swapping in\nyour own custom entity id symbol (e.g., `?xx` in the following example):\n\n```clojure\n(datomic.api/q {:find '[?xx ...]\n                :where (conj (identify-via-clauses-for ast '?xx :interface/xx)\n                             ['?xx :other/attribute :x/y])})\n```\n\nThe prior example finds the collection of entity ids that satisfy interface\n`:interface/xx` and that also have an attribute `:other/attribute` with value\nequal to `:x/y`.\n\n\n### Converting Data Interfaces to Datomic Schemas\n\nOur declarative semantic definitions can be converted to Datomic schema maps.\n\nConsider the data interface we first encountered:\n\n```clojure\n(def user-interface\n  {:interface.def/name :interface/user\n   :interface.def/fields\n     {:user/username [:string \"A user's username\"\n                              :unique/identity ; can also be :unique/value\n                              :required]\n      :taggable/tags [[:string] \"Tags for an entity\"]\n      :user/registeredAt [:instant \"When a user registered\"]}\n   :interface.def/identify-via ['[?e :user/username]]})\n```\n\nIt will also be able to generate the following Datomic attributes.\n\n```clojure\n(require '[datomic.api :as d])\n\n(-\u003e user-interface\n    semantic-spec-\u003esemantic-ast\n    (semantic-ast-\u003edatomic-schemas d/tempid))\n```\n\nThis will generate the following Datomic attributes:\n\n```clojure\n[\n {:db/id #db/id[:db.part/db]\n  :db/ident :user/username\n  :db/doc \"A user's username\"\n  :db/valueType :db.type/string\n  :db/cardinality :db.cardinality/one\n  :db/unique :db.unique/identity\n  :db.install/_attribute :db.part/db}\n\n {:db/id #db/id[:db.part/db]\n  :db/ident :taggable/tags\n  :db/doc \"Tags for an entity\"\n  :db/valueType :db.type/string\n  :db/cardinality :db.cardinality/many\n  :db.install/_attribute :db.part/db}\n\n {:db/id #db/id[:db.part/db]\n  :db/ident :user/registeredAt\n  :db/doc \"When a user registered\"\n  :db/valueType :db.type/instant\n  :db/cardinality :db.cardinality/one\n  :db.install/_attribute :db.part/db}\n]\n```\n\nThese can be sent to a Datomic transactor to register these attributes\nas part of the Datomic schema.\n\nYou can expect Datomic enum entities to be generated for enum values that you\nspecify. For example, consider the following\n\n```clojure\n(def person-interface\n  {:interface.def/name :interface/person\n   :interface.def/fields\n     {:person/gender [\"A person's gender\"\n                      :enum {:person.gender/male \"Male\"\n                             :person.gender/female  \"Female\"\n                             :person.gender/other \"Other\"}]}})\n(-\u003e person-interface\n    semantic-spec-\u003esemantic-ast\n    (semantic-ast-\u003edatomic-schemas d/tempid))\n```\n\n\nThis will produce:\n\n```clojure\n[{:db/id #db/id[:db.part/db]\n  :db/doc \"A person's gender\"\n  :db/ident :person/gender\n  :db/valueType :db.type/ref\n  :db/cardinality :db.cardinality/one\n  :db.install/_attribute :db.part/db}\n\n ;; Gender enums\n\n {:db/id #db/id[:db.part/user]\n  :db/ident :person.gender/male\n  :db/doc \"Male\"}\n\n {:db/id #db/id[:db.part/user]\n  :db/ident :person.gender/female\n  :db/doc \"Female\"}\n\n {:db/id #db/id[:db.part/user]\n  :db/ident :person.gender/other\n  :db/doc \"Other\"}]\n```\n\n### Converting dspec interfaces to clojure.spec definitions\n\nOur declarative semantic definitions can be converted to `clojure.spec` specs.\n\n#### Data Generation\n\nBecause we leverage `clojure.spec`, we can also leverage its\n[abilities](http://clojure.org/guides/spec#_generators) to\ngenerate test or sample data.\n\n```clojure\n(require '[datomic-tools.core :refer [semantic-spec-coll-\u003esemantic-ast\n                                      register-specs-for-ast!\n                                      register-specs-for-ast-with-custom-generators!\n                                      ]])\n(require '[clojure.spec.gen :as gen])\n(require '[ccm-om-next.db.gen.util :refer [fn-\u003egen ensure-keys-gen]])\n(require '[faker.name :as fname])\n(require '[clojure.spec.gen :as gen])\n(require '[clojure.spec :as s])\n(import '[datomic.db DbId])\n\n(def edn-interfaces\n  [{:interface.def/name :interface/person\n    :interface.def/fields {:person.id/uuid [:uuid \"A uuid we assign to identify the person\" :db.unique/identity :required]\n                           :person.id/ssn [:string \"A person's social security number\" :db.unique/value]\n                           :person/name [[:interface.person/name] \"A person's names\"]\n                           :person/languages [[:string] \"Languages a person can speak\"]}\n    :interface.def/identify-via ['[?e :person.id/uuid]]}\n\n   {:interface.def/name :interface.person/name\n    :interface.def/fields {:person.name/given [:string \"A given part of a person's name\"]\n                           :person.name/family [:string \"The family part of a person's name\"]}\n    :interface.def/identify-via ['[?e :person.name/given]]}])\n\n(def generators\n  {:person.name/given (fn-\u003egen fname/first-name)\n                       ; (fn-\u003egen fn) calls fn to generate data\n   :person/name (fn [member-generator]\n                  (gen/set member-generator {:min-elements 1 :max-elements 1}))\n   :interface.person/name (ensure-keys-gen :person.name/family\n                                           :person.name/given)\n                           ; (ensure-keys-gen :key1 :key2) generates a map\n                           ; with at least :key1 and :key2\n   :person/languages #(s/gen #{\"English\" \"Spanish\"})})\n\n(def db-id? #(or (integer? %)\n                 (instance? datomic.db.DbId %)))\n\n(-\u003e edn-interfaces\n\n    ; Convert the semantic data interfaces to the intermediate semantic ast\n    ; representation\n    semantic-spec-coll-\u003esemantic-ast\n\n    ; Register `clojure.spec` specs based off the semantic ast\n    ; and based off of custom generators to over-write particular\n    ; `clojure.spec` keys.\n    (register-specs-for-ast-with-custom-generators! generators datomic.api/tempid db-id?))\n\n; Now that the specs are loaded into our Clojure environment, we\n; can generated sample data based off of them.\n\n; First create the generator\n(def generator (s/gen :interface/person))\n\n; Then generate data!\n\n(def num-people-to-generate 100)\n(gen/sample generator num-people-to-generate)\n```\n\nHere is an example of generating and loading sample data into Datomic\n\n```\n(require '[datomic.api :as d])\n(require '[clojure.core.async :as a :refer [\u003c!!]])\n\n;; Helper fns\n\n(defn vset-\u003evlist\n  [m]\n  (into {} (map\n             (fn [[k v]]\n               [k (cond (set? v) (into [] v)\n                        (map? v) (vset-\u003evlist v)\n                        :else v)])\n             m)))\n\n(defn elide-empty-str [m]\n  (into {}\n        (filter (fn [[_ v]]\n                  (or (not (string? v))\n                      (not (empty? v))))\n                m)))\n\n(defn elide-empty-set [m]\n  (into {}\n        (filter (fn [[_ v]]\n                  (or (not (set? v))\n                      (not (empty? v))))\n                m)))\n\n\n;; Data generation\n\n(def num-samples 100)\n(def generated-patients (gen/sample (s/gen :interface/patient) num-samples))\n\n;; Write data to Datomic\n\n(def conn (d/connect \"datomic:mem//some-db\"))\n(def concurrency 2)\n(def ch (a/to-chan (-\u003e\u003e generated-patients\n                        (map elide-empty-set)\n                        (map elide-empty-str)\n                        (map vset-\u003evlist)\n                        (map #(merge % {:db/id (d/tempid :db.part/user)}))\n                        (map #(vector %)))))\n(let [{:keys [result stop]} (tx-pipeline conn conc ch)]\n  (\u003c!! result)\n  (System/exit 0))\n```\n\n\n### Intermediate AST Representation\n\nAs we have seen, our declarative semantic definitions can be converted to one\nof two output formats:\n\n1. Datomic schema maps.\n2. `clojure.spec` declarations.\n\nBefore converting to either of these formats, we convert our human-optimized\nsemantic definitions into a machine-optimized intermediate representation. \n\nConsider our semantic definition for a user:\n\n```clojure\n(def user-interface\n  {:interface.def/name :interface/user\n   :interface.def/fields \n     #:user{:username [:string \"A user's username\"\n                               :db.unique/identity ; can also be :unique/value\n                               :required]\n            :taggable/tags [[:string] \"Tags for an entity\"]\n            :registeredAt [:instant \"When a user registered\"]\n            :taggable/tags [[:string] \"Tags for an entity\"]\n            :user/registeredAt [:instant \"When a user registered\"]}\n   :interface.def/identify-via ['[?e :user/username]]})\n```\n\nThen, we can convert this to the intermediate AST.\n\n```clojure\n(require '[lab79.dspec :refer [ds]])\n(ds/semantic-spec-coll-\u003esemantic-ast user-interface)\n```\n\nThis will generate the following AST.\n\n```clojure\n{:interface.ast/interfaces\n  {:interface/user\n    {:interface.ast.interface/name :interface/user\n     :interface.ast.interface/fields\n       {:user/username {:db/ident :user/username\n                        :db/valueType :db.type/string\n                        :interface.ast.field/type :string\n                        :db/cardinality :db.cardinality/one\n                        :db/doc \"A user's username\"\n                        :db/unique :db.unique/identity\n                        :interface.ast.field/required true}\n        :taggable/tags {:db/ident :taggable/tags\n                        :db/valueType :db.type/string\n                        :interface.ast.field/type :string\n                        :db/cardinality :db.cardinality/many\n                        :db/doc \"Tags for an entity\"}\n        :user/registeredAt {:db/ident :user/registeredAt\n                            :db/valueType :db.type/instant\n                            :interface.ast.field/type :instant\n                            :db/cardinality :db.cardinality/one\n                            :db/doc \"When a user registered\"}}\n     :interface.ast.interface/inherits #{}\n     :interface.ast.interface/identify-via ['[?e :user/username]]}}\n :interface.ast/enum-map {}}\n```\n\nLet's revisit another example.\n\n```clojure\n(def person-interface\n  {:interface.def/name :interface/person\n   :interface.def/fields\n     {:person/gender [\"A person's gender\"\n                      :enum {:person.gender/male \"Male\"\n                             :person.gender/female  \"Female\"\n                             :person.gender/other \"Other\"}]}\n   :interface.def/identify-via :datomic-spec/interfaces\n   :interface.def/identifying-enum-part :db.part/user})\n```\n\nThis semantic data interface generates the following AST.\n\n```clojure\n{:interface.ast/interfaces\n  {:interface/person\n    {:interface.ast.interface/name :interface/person\n     :interface.ast.interface/fields\n       {:person/gender {:db/ident :person/gender\n                        :db/valueType :db.type/ref\n                        :db/cardinality :db.cardinality/one\n                        :db/doc \"A person's gender\"\n                        :semantic/type :enum\n                        :interface.ast.field/enum-seq #{:gender/male\n                                                        :gender/female}}\n\n        :datomic-spec/interfaces {:db/ident :datomic-spec/interfaces\n                                  :db/valueType :db.type/ref\n                                  :db/index true\n                                  :interface.ast.field/type :enum\n                                  :interface.ast.field/possible-enum-vals #{:interface/refable}\n                                  :interface.ast.field/required true\n                                  :db/cardinality :db.cardinality/many}}\n     :interface.ast.interface/inherits #{}\n     :interface.ast.interface/identify-via ['[?e :datomic-spec/interfaces :interface/person]]}}\n :interface.ast/enum-map\n   {:gender/male {:db/ident :gender/male\n                  :db/doc \"Male\"}\n    :gender/female {:db/ident :gender/female\n                    :db/doc \"Female\"}\n    :interface/person {:db/ident :interface/person\n                       :db/part :db.part/user}}}\n```\n\nHere is how ref constraints are encapsulated in our AST:\n\n```clojure\n{:interface.ast/interfaces\n  {:interface/person\n     {:interface.ast.interface/name :interface/person\n      :interface.ast.interface/fields\n        {:person.id/ssn {:db/ident :person.id/ssn\n                         :db/valueType :db.type/string\n                         :interface.ast.field/type :string\n                         :db/cardinality :db.cardinality/one\n                         :db/doc \"A person's ssn number\"\n                         :db/unique :db.unique/value}\n         :person/name {:db/ident :person/name\n                       :db/valueType :db.type/ref\n                       :interface.ast.field/type :person/name\n                       :db/cardinality :db.cardinality/many\n                       :db/doc \"A person's names\"\n                       :semantic/type :person/name}}\n      :interface.ast.interface/inherits #{}\n      :interface.ast.interface/identify-via ['[?e :person/name]]}\n   :person/name\n     {:interface.ast.interface/name :person/name\n      :interface.ast.interface/fields\n        {:person.name/given {:db/ident :person.name/given\n                             :db/valueType :db.type/string\n                             :interface.ast.field/type :string\n                             :db/cardinality :db.cardinality/one\n                             :db/doc \"A given part of a person's name\"}\n         :person.name/family {:db/ident :person.name/family\n                              :db/valueType :db.type/string\n                              :interface.ast.field/type :string\n                              :db/cardinality :db.cardinality/one\n                              :db/doc \"The family part of a person's name\"}\n      :interface.ast.interface/inherits #{}\n      :interface.ast.interface/identify-via ['[?e :person.name/given]]}\n :interface.ast/enum-map {}}}}\n```\n\nAnd here is how interface-based polymorphism is encapsulated in our AST:\n\n```clojure\n{:interface.ast/interfaces\n  {:interface/person\n     {:interface.ast.interface/name :interface/person\n      :interface.ast/interface/fields\n        {:person/name {:db/ident :person/name\n                       :db/valueType :db.type/string\n                       :interface.ast.field/type :string\n                       :db/cardinality :db.cardinality/one}}\n      :interface.ast.interface/identify-via ['[?e :person/name]]}\n   :interface/patient\n     {:interface.ast.interface/name :interface/patient\n      :interface.ast.interface/fields\n        {:patient/physicians {:db/ident :patient/primary-care-providers\n                              :db/valueType :db.type/ref\n                              :interface.ast.field/type :practitioner\n                              :db/cardinality db.cardinality/many\n                              :db/doc \"A patient's physicians\"\n                              :semantic/type :interface/physician}}\n      :interface.ast.interface/inherits #{:interface/person}\n      :interface.ast.interface/identify-via ['[?e :datomic-spec/interfaces :interface/patient]]\n      :interface.ast.interface/identifying-enum-part :db.part/user}\n   :interface/physician\n     {:interface.ast.interface/name :interface/physician\n      :interface.ast.interface/fields\n        {:physician/specialties {:db/ident :physician/specialties\n                                 :db/valueType :db.type/string\n                                 :interface.ast.field/type :string\n                                 :db/cardinality :db.cardinality/many\n                                 :db/doc \"The physician's medical specialty or specialties\"}}\n      :interface.ast.interface/inherits #{:interface/person}\n      :interface.ast.interface/identify-via ['[?e :datomic-spec/interfaces :interface/physician]]\n      :interface.ast.interface/identifying-enum-part :db.part/user}\n   :interface.ast/enum-map\n     {:interface/patient {:db/ident :interface/patient\n                          :db/part :db.part/user}\n      :interface/physician {:db/ident :interface/physician\n                            :db/part :db.part/user}}\n```\n\n## Prior Art\n\nThere are several other Clojure libraries that add additional semantics on top of Datomic,\nmake it easier to work with Datomic queries and transactions, or help with .\nWe've drawn inspiration from these, but decided to work on our own library because none of\nthem quite fit our needs and desired style.\n\nStronger Datomic Semantics\n\n- https://github.com/yuppiechef/datomic-schema\n- https://github.com/cloojure/tupelo-datomic\n- http://docs.caudate.me/adi/\n- https://github.com/SparkFund/spec-tacular\n- https://github.com/facjure/atomic\n- https://github.com/cldwalker/datomico\n- https://github.com/ohlo/ginandtomic\n- https://github.com/devn/datomic-simple\n- https://github.com/eeng/datomic-qb\n- https://github.com/moquist/datomic-schematode\n- https://github.com/zololabs/demonic\n- https://github.com/appcanary/crustacean\n- https://github.com/vlacs/hatch\n- https://github.com/wkf/crudo\n\nEasier to work with Datomic queries and transactions:\n\n- https://github.com/flyingmachine/datomic-junk\n- https://github.com/juxt/datomic-extras\n- https://github.com/democracyworks/datomic-toolbox\n- https://github.com/magnars/datomic-snippets\n- https://github.com/webnf/webnf/tree/master/datomic\n- https://github.com/mysema/datomic-tools\n- https://github.com/evanspa/pe-datomic-utils\n- https://github.com/avodonosov/datomic-helpers\n- https://github.com/jonase/datomic-query-helpers\n- https://github.com/molst/hazel\n- https://github.com/rodnaph/attromic\n- https://github.com/totalperspective/fook\n- https://github.com/osbert/storable\n- https://github.com/mgaare/datomisc\n- https://github.com/halgari/fafnir\n- https://github.com/CareLogistics/wile\n- https://github.com/env/toolbox\n\nDatomic migrations:\n\n- https://github.com/juxt/joplin\n- https://github.com/Bijnagte/alchemist\n- https://github.com/rkneufeld/conformity\n- https://github.com/bitemyapp/brambling\n- https://github.com/RallySoftware/datomic-replication\n- https://github.com/guilespi/datomic-manage\n- https://github.com/ilshad/generations\n\nStoring arbitrary data in Datomic:\n\n- https://github.com/tailrecursion/monocopy\n- https://github.com/GoodGuide/datomizer\n- https://github.com/stathissideris/datomic-rtree\n\nDatomic and clojure.spec\n\n- https://github.com/alexanderkiel/datomic-spec\n\n## License\n\nThe MIT License (MIT)\nCopyright © 2016 Lab79, Inc.\n\nPermission is hereby granted, free of charge, to any person obtaining a copy of\nthis software and associated documentation files (the \"Software\"), to deal in\nthe Software without restriction, including without limitation the rights to\nuse, copy, modify, merge, publish, distribute, sublicense, and/or sell copies\nof the Software, and to permit persons to whom the Software is furnished to do\nso, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flab-79%2Fdspec","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flab-79%2Fdspec","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flab-79%2Fdspec/lists"}