{"id":21481951,"url":"https://github.com/jillesvangurp/json-dsl","last_synced_at":"2025-07-15T12:33:49.610Z","repository":{"id":222838350,"uuid":"758517986","full_name":"jillesvangurp/json-dsl","owner":"jillesvangurp","description":"Create Kotlin DSLs for any JSON/YAML. JsonDsl is a library that helps you build Kotlin DSLs for JSON and YAML dialects. The DSLs are easy to extend with custom fields by users via a MutableMap. This allows your DSL users to work around any features you don't have in your DSL.","archived":false,"fork":false,"pushed_at":"2024-11-16T12:32:04.000Z","size":237,"stargazers_count":17,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-11-16T13:25:33.589Z","etag":null,"topics":["domain-specific-language","json","kotlin-multiplatform","library","yaml"],"latest_commit_sha":null,"homepage":"","language":"Kotlin","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jillesvangurp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-16T13:51:44.000Z","updated_at":"2024-11-16T12:32:07.000Z","dependencies_parsed_at":"2024-07-26T10:23:40.723Z","dependency_job_id":"ebe59f8d-cd2c-4750-b604-a5cfb6c76c4c","html_url":"https://github.com/jillesvangurp/json-dsl","commit_stats":null,"previous_names":["jillesvangurp/json-dsl"],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jillesvangurp%2Fjson-dsl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jillesvangurp%2Fjson-dsl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jillesvangurp%2Fjson-dsl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jillesvangurp%2Fjson-dsl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jillesvangurp","download_url":"https://codeload.github.com/jillesvangurp/json-dsl/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":226042953,"owners_count":17564747,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["domain-specific-language","json","kotlin-multiplatform","library","yaml"],"created_at":"2024-11-23T12:29:15.409Z","updated_at":"2025-07-15T12:33:49.598Z","avatar_url":"https://github.com/jillesvangurp.png","language":"Kotlin","funding_links":[],"categories":[],"sub_categories":[],"readme":"# JsonDsl\n\n[![Process Pull Request](https://github.com/jillesvangurp/json-dsl/actions/workflows/pr_master.yaml/badge.svg)](https://github.com/jillesvangurp/json-dsl/actions/workflows/pr_master.yaml)\n\nJsonDsl is a multi platform kotlin library that helps you build Kotlin DSLs for JSON and YAML dialects. \nDSLs made with this library are easy to extend with custom fields by users via a MutableMap.\n\n# About DSLs\n\nA Domain Specific Language (DSL) is a language that is intended to allow users to program or specify things in language that closely matches their domain. They are popular for configuration files, for use with certain frameworks or tools, and there are a lot of niche tools, frameworks, and other software packages out there that implement them. \n\nSome general purpose languages have a syntax that makes it easy to (ab)use their features to do something similar. Lisp and Ruby are a good example of languages that have historically been used for this. Like those languages, Kotlin includes a few features that enable this. Which makes Kotlin very well suited for implementing all sorts of DSLs.\n\nThere are Kotlin DSLs for all sorts of things. A popular example is the HTML DSL that comes with things like Ktor and kotlin-js. Spring bundles a lot of Kotlin DSLs for it's Java framework that make using that a lot nicer than from Java. There are lots of examples.\n\n## Making easy to extend Kotlin DSLs for JSON/YAML dialects\n\nOf course, creating model classes for your JSON domain model and annotating thosre with annotations for e.g. `kotlinx.serialization`, `jackson`, etc. is a perfectly valid way to start creating a DSL for your JSON or YAML dialect of choice.\n\nHowever, using JSON frameworks like that have some limitations. What if your JSON dialect evolves and somebody adds some new features? Unless you change your model class, it would not be possible to access such new features via the Kotlin DSL. Or what if your JSON dialect is vast and complicated. Do you have to support all of its features? How do you decide what to allow and not allow in your Kotlin DSL.\n\nThis library started out as few classes in my [kt-search](https://github.com/jillesvangurp/kt-search) project. Kt-search implements a client library for Elasticsearch and Opensearch. Elasticsearch has several JSON dialects that are used for querying, defining index mappings, settings, and a few other things. Especially the query language has a large number of features and is constantly evolving. \n\nNot only do I have to worry about implementing each and every little feature these DSLs have and keeping up with upstream additions to OpenSearch and Elasticsearch. I also have to worry about supporting query and mapping features added via custom plugins. This is very challenging. And it was the main reason I created json-dsl: so I don't have to keep up.\n\n## Strongly typed and Flexible\n\nThe key feature in json-dsl is that it uses a `MutableMap` and property delegation for implementing DSL classes. This simple approach enables you to define classes with properties that delegate storing their value to this map. For anything that your\nclasses don't implement, the user can simply modify the underlying map directly using a simple `put`.\n\nThis simple approach gives users a nice fallback for things your DSL classes don't implement and it relieves Kotlin DSL creators from having to provide support for every new feature the upstream JSON dialect has or adds over time. You can provide a decent experience for your users with minimal effort. And you users can always work around whatever you did not implement.\n\nWith kt-search, I simply focus on supporting all the commonly used, and some less commonly used things in the Elastic DSLs. But for everything else, I just rely on letting the user modify the underlying map themselves. A lot of pull requests I get on this project are people adding features they need in the DSLs. So, over time, feature support has gotten more comprehensive.\n\n## Gradle\n\nThis library is published to our own maven repository. Simply add the repository like this:\n\n```kotlin\nrepositories {\n    mavenCentral()\n    maven(\"https://maven.tryformation.com/releases\") {\n        // optional but it speeds up the gradle dependency resolution\n        content {\n            includeGroup(\"com.jillesvangurp\")\n        }\n    }\n}\n```\n\nAnd then you can add the dependency:\n\n```kotlin\n    // check the latest release tag for the latest version\n    implementation(\"com.jillesvangurp:json-dsl:3.x.y\")\n```\n\nIf you were using json-dsl via kt-search before, you can update simply by bumping the version of json-dsl to 3.0. Previously, 2.x was released along with kt-search and has now been removed from that project.\n\n\n## Examples\n\nAll the examples in this README are implemented using\n    my [kotlin4example](https://github.com/jillesvangurp/kotlin4example) library. You can find \n    the source code that generates this README [here](https://github.com/jillesvangurp/json-dsl/blob/main/src/jvmTest/kotlin/com/jillesvangurp/jsondsl/readme/ReadmeGenerationTest.kt). \n    \nA more expanded version of these examples can be found in the form of a \nJupyter notebook [here](https://github.com/jillesvangurp/json-dsl-jupyter)\n\n### Hello World\n\nLet's start with a simple example.\n\n```kotlin\nclass MyDsl : JsonDsl() {\n  // adds a string property that the user can assign\n  var message by property\u003cString\u003e()\n}\n\n// a helper function to create MyDsl instances\nfun myDsl(block: MyDsl.() -\u003e Unit): MyDsl {\n  return MyDsl().apply(block)\n}\n\nval json = myDsl {\n  message = \"Hello world\"\n}.json(pretty = true)\n\nprintln(json)\n```\n\nThe json extension function uses a json serializer to produce \npretty printed json:\n\n```json\n{\n  \"message\": \"Hello world\"\n}\n```\n\nThere is also a YAML serializer. More on that below.\n\n### Common Kotlin Types\n\nJSON is a fairly simple data format. It has numbers, booleans, strings, lists and dictionaries. And null\nvalues.  Kotlin has a bit richer type system and mapping that to JSON is key to providing rich Kotlin DSL.                \n\nJsonDsl does a best effort to do map Kotlin types correctly to the intended JSON equivalent. \n           \nIt understands all the primitives, Maps and Lists. But also Arrays, Sets, Sequences, etc.                \nAnd of course other JsonDsl classes, so you can nest them.  And it falls back to using \n`toString()` for everything else.              \n\n```kotlin\nclass MyDsl : JsonDsl() {\n  var intVal by property\u003cInt\u003e()\n  var boolVal by property\u003cBoolean\u003e()\n  var doubleVal by property\u003cDouble\u003e()\n  var arrayVal by property\u003cArray\u003cString\u003e\u003e()\n  var listVal by property\u003cList\u003cString\u003e\u003e()\n  var mapVal by property\u003cMap\u003cString, String\u003e\u003e()\n  var idontknow by property\u003cAny\u003e()\n}\n\n// using kotlin's apply here, you can add helper functions of course\nMyDsl().apply {\n  intVal = 1\n  boolVal = true\n  doubleVal = PI\n  arrayVal = arrayOf(\"hello\", \"world\")\n  listVal = listOf(\"1\", \"2\")\n  mapVal = mapOf(\n    \"Key\" to \"Value\"\n  )\n\n  // The Any type is a bit of free for all\n  idontknow = mapOf(\n    \"arrays\" to arrayOf(\n      1, 2, \"3\", 4.0,\n      mapOf(\"this\" to \"is valid JSON\"), \"mixing types is allowed in JSON\"\n    ),\n    \"sequences\" to sequenceOf(1,\"2\",3.0)\n  )\n}\n```\n\nThis does the right things with all the used Kotlin types, including `Any`:\n\n```json\n{\n  \"int_val\": 1,\n  \"bool_val\": true,\n  \"double_val\": 3.141592653589793,\n  \"array_val\": [\n    \"hello\", \n    \"world\"\n  ],\n  \"list_val\": [\n    \"1\", \n    \"2\"\n  ],\n  \"map_val\": {\n    \"Key\": \"Value\"\n  },\n  \"idontknow\": {\n    \"arrays\": [\n      1, \n      2, \n      \"3\", \n      4.0, \n      {\n        \"this\": \"is valid JSON\"\n      }, \n      \"mixing types is allowed in JSON\"\n    ],\n    \"sequences\": [\n      1, \n      \"2\", \n      3.0\n    ]\n  }\n}\n```\n\n### Manipulating the Map directly\n\nAs mentioned, JsonDsl delegates the storing of properties to a `MutableMap\u003cString, Any?\u003e`. \n\nSo, all sub classes have direct access to that map. And you can put anything you want into it.\n\n```kotlin\nclass MyDsl : JsonDsl() {\n  var foo by property\u003cString\u003e()\n}\n\nMyDsl().apply {\n  // nicely typed.\n  foo = \"bar\"\n\n  // but we never defined a bar property\n  this[\"bar\"] = \"foo\"\n  // or this ...\n  this[\"whatever\"] = listOf(\n    MyDsl().apply {\n      this[\"you\"] = \"can add anything you want\"\n    },\n    42\n  )\n\n  // RawJson is a Kotlin value class\n  this[\"inline_json\"] = RawJson(\"\"\"\n    {\n      \"if\":\"you need to\",\n      \"you\":\"can even add json in string form\",               \n    }\n  \"\"\".trimIndent())\n}\n```\n\n```json\n{\n  \"foo\": \"bar\",\n  \"bar\": \"foo\",\n  \"whatever\": [\n    {\n      \"you\": \"can add anything you want\"\n    }, \n    42\n  ],\n  \"inline_json\": {\n    \"if\":\"you need to\",\n    \"you\":\"can even add json in string form\",                           \n}\n}\n```\n\n### snake_casing, custom names, defaults\n\nA lot of JSON dialects use snake cased field names. Kotlin of course uses \ncamel case for its identifiers and it has certain things that you can't redefine.\nLike the `size` property on `Map`, which is implemented by JsonDsl; or certain keywords.\n\n```kotlin\nclass MyDsl : JsonDsl(\n  // this will snake case all the names\n  namingConvention = PropertyNamingConvention.ConvertToSnakeCase\n) {\n  // -\u003e camel_case\n  var camelCase by property\u003cBoolean\u003e()\n  // unfortunately Map defines a size val already\n  var mySize by property\u003cInt\u003e(\n    customPropertyName = \"size\"\n  )\n  // val is a keyword in Kotlin\n  var myVal by property\u003cString\u003e(\n    customPropertyName = \"val\"\n  )\n  // Kotlin has default values, JSON does not.\n  var m by property(\n    customPropertyName = \"meaning_of_life\",\n    defaultValue = 42\n  )\n}\n\nMyDsl().apply {\n  camelCase = true\n  mySize = Int.MAX_VALUE\n  myVal = \"hello\"\n}\n```\n\n```json\n{\n  \"meaning_of_life\": 42,\n  \"camel_case\": true,\n  \"size\": 2147483647,\n  \"val\": \"hello\"\n}\n```\n\n### Custom values\n\nSometimes you want to have the serialized version of a value be different\nfrom the kotlin type that you are using. For this we have added the \nCustomValue interface.\n\nA simple use case for this could be Enums:                                \n\n```kotlin\nenum class Grades(override val value: Double) : CustomValue\u003cDouble\u003e {\n  Excellent(7.0),\n  Pass(5.51),\n  Fail(3.0),\n  ;\n}\n```\n\n```kotlin\nprintln(withJsonDsl {\n  this[\"grade\"] = Grades.Excellent\n}.json(true))\n```\n\nNote how the grade's Double value is used instead of the name.\n\nThe withJsonDsl function is a simple extension function that \ncreates a JsonDsl for you and applies the block to it.\n\n```json\n{\n  \"grade\": 7.0\n}\n```\n\nYou can also construct more complex ways to serialize your classes.\n\n```kotlin\ndata class Person(\n  val firstName: String,\n  val lastName: String): CustomValue\u003cList\u003cString\u003e\u003e {\n  override val value =\n    listOf(firstName, lastName)\n}\n\nwithJsonDsl {\n  this[\"person\"] = Person(\"Jane\", \"Doe\")\n}.json(true)\n```\n\nAnd of course your custom value can be a JsonDsl too.\n\n```kotlin\ndata class Person(\n  val firstName: String,\n  val lastName: String): CustomValue\u003cJsonDsl\u003e {\n  override val value = withJsonDsl {\n    this[\"person\"] = withJsonDsl {\n      this[\"fn\"] = firstName\n      this[\"ln\"] = lastName\n      this[\"full_name\"] = \"$firstName $lastName\"\n    }\n  }\n}\n```\n\nYou can also rely on the `toString()` function:\n\n```kotlin\ndata class FooBar(val foo:String=\"foo\", val bar: String=\"bar\")\nprintln(withJsonDsl {\n  this[\"foo\"]=FooBar()\n})\n```\n\nNote how it simply uses `toString()` on the data class\n\n```json\n{\n  \"foo\": \"FooBar(foo=foo, bar=bar)\"\n}\n```\n\nThis also works for things like enums, value classes, and other Kotlin language constructs.\n\n## YAML\n\nWhile initially written to support JSON, I also added a YAML serializer that you may use to \ncreate Kotlin DSLs for YAML based DSLs. So, you could use this to build Kotlin DSLs for things\nlike Github actions, Kubernetes, or other common things that use YAML.\n\n```kotlin\nclass YamlDSL : JsonDsl() {\n  var str by property\u003cString\u003e()\n  var map by property\u003cMap\u003cString,Any\u003e\u003e()\n  var list by property\u003cList\u003cAny\u003e\u003e()\n}\nval dsl = YamlDSL().apply {\n  str=\"\"\"\n    Multi line\n    Strings are \n       supported\n       and\n    preserve their\n      indentation!\n  \"\"\".trimIndent()\n  map = mapOf(\n    \"foo\" to \"bar\",\n    \"num\" to PI,\n    \"bool\" to true,\n    \"notABool\" to \"false\"\n  )\n}\n// default is true for including ---\nprint(dsl.yaml(includeYamlDocumentStart = false))\n```\n\nThis prints the YAML below:\n\n```yaml\nstr: |\n  Multi line\n  Strings are \n     supported\n       and\n  preserve their\n    indentation!\nmap: \n  foo: bar\n  num: 3.141592653589793\n  bool: true\n  notABool: \"false\"\n```\n\nThere are other tree like formats that might be supported in the future like TOML, properties, \nand other formats. I welcome pull requests for this provided they don't add any library dependencies.\n\n## A real life, complex example\n\nHere's a bit of the kt-search Kotlin DSL that I lifted\nfrom my kt-search library. It implements a minimal \nquery and only supports one of the (many) types of queries\nsupported by Elasticsearch.\n \nLike many real life\nJSON, the Elasticsearch DSL is quite complicated and challenging \nto model. This is why I created this library.\n\nThe code below is a good illustration of several things you can\ndo in Kotlin to make life nice for your DSL users.\n\n```kotlin\n// using DslMarkers is useful with\n// complicated DSLs\n@DslMarker\nannotation class SearchDSLMarker\n\ninterface QueryClauses\n\n// abbreviated version of the\n// Elasticsearch Query DSL in kt-search\nclass QueryDsl:\n  JsonDsl(namingConvention = PropertyNamingConvention.ConvertToSnakeCase),\n  // helper interface that we define\n  // extension functions on\n  QueryClauses\n{\n  // Elasticsearch often wraps objects in\n  // another object. So we use a custom\n  // setter here to hide that.\n  var query: ESQuery\n    get() {\n      val map =\n        this[\"query\"] as Map\u003cString, JsonDsl\u003e\n      val (name, details) = map.entries.first()\n      // reconstruct the ESQuery\n      return ESQuery(name, details)\n    }\n    set(value) {\n      // queries extend ESQuery\n      // which takes care of the wrapping\n      // via wrapWithName\n      this[\"query\"] = value.wrapWithName()\n    }}\n\n// easy way to create a query\nfun query(block: QueryDsl.()-\u003eUnit): QueryDsl {\n  return QueryDsl().apply(block)\n}\n\n@SearchDSLMarker\nopen class ESQuery(\n  val name: String,\n  val queryDetails: JsonDsl = JsonDsl()\n)  {\n\n  // Elasticsearch wraps everything in an outer object\n  // with the name as its only key\n  fun wrapWithName() = withJsonDsl() {\n    this[name] = queryDetails\n  }\n}\n\n// the dsl class for creating term queries\n// this is one of the most basic queries\n// in elasticsearch\n@SearchDSLMarker\nclass TermQuery(\n  field: String,\n  value: String,\n  termQueryConfig: TermQueryConfig = TermQueryConfig(),\n  block: (TermQueryConfig.() -\u003e Unit)? = null\n) : ESQuery(\"term\") {\n  // on init, apply the block to the configuration and\n  // assign it in the queryDetails from the parent\n  init {\n    queryDetails.put(field, termQueryConfig, PropertyNamingConvention.AsIs)\n    termQueryConfig.value = value\n    block?.invoke(termQueryConfig)\n  }\n}\n\n// configuration for term queries\n// this is a subset of the supported\n// properties.\nclass TermQueryConfig : JsonDsl() {\n  var value by property\u003cString\u003e()\n  var boost by property\u003cDouble\u003e()\n}\n\n// making this an extension function ensures it is available\n// on the receiver of the query {..} block\nfun QueryClauses.term(\n  field: String,\n  value: String,\n  block: (TermQueryConfig.() -\u003e Unit)? = null\n) =\n  TermQuery(field, value, block = block)\n\n// of course users of this DSL would\n// be storing json documents in elasticsearch\n// and they probably have model classes with\n// properties.\n// so supporting property references\n// for field names is a nice thing\nfun QueryClauses.term(\n  field: KProperty\u003c*\u003e,\n  value: String,\n  block: (TermQueryConfig.() -\u003e Unit)? = null\n) =\n  TermQuery(field.name, value, block = block)\n```\n\nAnd this is how your users would use this DSL.\n\n```kotlin\nclass MyModelClassInES(val myField: String)\nval q = query {\n  query = term(MyModelClassInES::myField, \"some value\")\n}\nval pretty = q.json(pretty = true)\nprintln(pretty)\n```\n\nSo, we created a query outer object with a query property.\nAnd we assigned a term query instance to that.\n\nIn JSON form this looks as follows:                \n\n```json\n{\n  \"query\": {\n    \"term\": {\n      \"myField\": {\n        \"value\": \"some value\"\n      }\n    }\n  }\n}\n```\n\nNote how it correctly wrapped the term query with an object. And how it correctly \nassigns the `TermConfiguration` in an object that has the field value as the key.\nAlso note how we use a property reference here to avoid having to use \na string literal. \n\nOf course, the Elasticsearch Query DSL support in \n[kt-search](https://github.com/jillesvangurp/kt-search) is a great \nreference for how to use JsonDsl.\n\n## Multi platform\n\nThis is a Kotlin multi platform library that should work on most  kotlin platforms (jvm, js, ios, wasm, linux/windows/mac native, android, etc).\n\nMy intention is to keep this code very portable and lightweight and not introduce any dependencies other than the Kotlin standard library. \n\n## Using JsonDsl with Javascript libraries\n\nIf you use kotlin-js, there are a lot of Javascript libraries out there that have functions that expect some sort of Javascript objects as a parameter. Integrating such libraries into Kotlin typically requires writing some type mappings to be able to call functions in these libraries and defining external class definitions for any models.\n\nWith JsonDsl you can skip a lot of these class definitions and simply create a Kotlin DSL instead. Or use simply use it in schemaless mode and rely on the convenient mappings included for the default Kotlin collection classes. You can use lists, enums, maps, etc. and it will do the right thing. You have the full flexibility of JsonDsl to make things as type safe as you need them to be.\n\nThere is a `toJsObject()` extension function that is available in kotlin-js that you can use to convert any JsonDsl instance to a javascript object.\n\n## Parsing\n\nThis library is not intended as a substitute for kotlinx.serialization or other JSON parsing frameworks. It is instead intended for writing Kotlin DSLs that you can use to generate valid JSON or YAML. JsonDsl does not support any form of parsing.\n\n## Development and stability\n\nBefore I extracted it from there, this library was part of [kt-search](https://github.com/jillesvangurp/kt-search), which has been out there for several years and  has a steadily growing user base. So, even though this library is relatively new, the code base has been stable and actively used for several years.\n\nOther than cleaning the code a bit up for public use, there were no compatibility breaking changes. I want to keep the API for json-dsl stable and will not make any major changes unless there is a really good reason. \n\nThis also means there won't be a lot of commits or updates since things are stable and pretty much working as intended. And because I have a few users of kt-search, I also don't want to burden them with compatibility breaking changes. Unless somebody finds a bug or asks for reasonable changes, the only changes likely to happen will be occasional dependency updates.\n\n## Libraries using Json Dsl\n\n- [kt-search](https://github.com/jillesvangurp/kt-search)\n\nIf you use this and find it useful, please add yourself to the list by creating a pull request on\n[outro.md](src/jvmTest/com/jillesvangurp/jsondsl/readme/outro.md) as this readme is generated using\nmy [kotlin4example](https://github.com/jillesvangurp/kotlin4example) library.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjillesvangurp%2Fjson-dsl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjillesvangurp%2Fjson-dsl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjillesvangurp%2Fjson-dsl/lists"}