{"id":13611035,"url":"https://github.com/nomemory/mapneat","last_synced_at":"2025-03-26T18:31:18.446Z","repository":{"id":45422045,"uuid":"306127792","full_name":"nomemory/mapneat","owner":"nomemory","description":"MapNeat is a JVM library written in Kotlin that provides an easy to use DSL (Domain Specific Language) for transforming JSON to JSON, XML to JSON, POJO to JSON in a declarative way.","archived":false,"fork":false,"pushed_at":"2022-04-21T07:50:10.000Z","size":302,"stargazers_count":62,"open_issues_count":1,"forks_count":5,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-22T06:32:36.642Z","etag":null,"topics":["dsl","java","json","jsonpath","jvm","kotlin","transform"],"latest_commit_sha":null,"homepage":"","language":"Kotlin","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nomemory.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-10-21T19:32:09.000Z","updated_at":"2025-01-13T14:47:57.000Z","dependencies_parsed_at":"2022-08-31T02:00:30.747Z","dependency_job_id":null,"html_url":"https://github.com/nomemory/mapneat","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nomemory%2Fmapneat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nomemory%2Fmapneat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nomemory%2Fmapneat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nomemory%2Fmapneat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nomemory","download_url":"https://codeload.github.com/nomemory/mapneat/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245712639,"owners_count":20660275,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dsl","java","json","jsonpath","jvm","kotlin","transform"],"created_at":"2024-08-01T19:01:51.101Z","updated_at":"2025-03-26T18:31:17.842Z","avatar_url":"https://github.com/nomemory.png","language":"Kotlin","readme":"**MapNeat** is a JVM library written in Kotlin that provides an easy to use DSL (*Domain Specific Language*) for transforming JSON to JSON, XML to JSON, POJO to JSON in a declarative way. \n\nNo intermediary POJOs are needed. Given's Kotlin high-interoperability **MapNeat** can be used in a Java project without any particular hassle. Check the documentation for examples on how to do that. \n\nUnder the hood **MapNeat** is using:\n* [jackson](https://github.com/FasterXML/jackson) and [json-path](https://github.com/json-path/JsonPath) for JSON querying and processing;\n* [JSON In Java](https://github.com/stleary/JSON-java) for converting from XML to JSON;\n* [JSONAssert](http://jsonassert.skyscreamer.org/) for making JSON assertions (testing purposes).\n\nBlog articles:\n* [Hello world, mapneat!](https://www.andreinc.net/2021/01/31/hello-world-mapneat)\n* [XML to JSON using mapneat](https://www.andreinc.net/2021/02/01/xml-to-json-using-mapneat) \n\n# Table of contents\n\n* [Getting started](#getting-started)\n* [How it works](#how-it-works)\n* [A typical transformation](#a-typical-transformation)\n* [Operations](#operations)\n    * [Assign](#assign-)\n    * [Shift](#shift-)\n    * [Copy](#copy-)\n    * [Move](#move-)\n    * [Delete](#delete--)  \n* [Using Mapneat from Java](#using-mapneat-from-java)\n* [Logging](#logging)\n* [Contributions and Roadmap](#contributing-and-roadmap)\n\n# Getting Started\n\nFor newver version (\u003e=0.9.5):\n\n```xml\n\u003cdependency\u003e\n  \u003cgroupId\u003enet.andreinc\u003c/groupId\u003e\n  \u003cartifactId\u003emapneat\u003c/artifactId\u003e\n  \u003cversion\u003e1.0.0\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\n```\nimplementation 'net.andreinc:mapneat:1.0.0`\n```\n\n\u003e For security issues, please update to `1.0.0`. \n\n# How it works\n\nThe library will transform any JSON, XML or POJO into another JSON, without the need of intermediary POJO classes.\n\nEvery operation applied to the source JSON (the input) is declarative in nature, and involves significantly less code than writing everything by hand.\n\nNormally, a transformation has the following structure:\n\n```kotlin\nval jsonValue : String = \"...\"\nval transformedJson = json(fromJson(jsonValue)) {\n    /* operation1 */\n    /* operation2 */\n    /* operation3 */\n    /* conditional block */\n        /* operation4 */\n}.getPrettyString() // Transformed output\n```\n\nIf the source is XML, `fromXML(xmlValue: String)`can be used. In this case the `xmlValue` is automatically converted to JSON using [JSON In Java](https://github.com/stleary/JSON-java).\n\nIf the source is a POJO, `fromObject(object)` can be used. In this case the `object` is automatically converted to JSON using jackson.\n\n# A typical transformation\n\nA typical transformation looks like this:\n\nJSON1:\n```json\n{\n  \"id\": 380557,\n  \"first_name\": \"Gary\",\n  \"last_name\": \"Young\",\n  \"photo\": \"http://srcimg.com/100/150\",\n  \"married\": false,\n  \"visits\" : [ \n    {\n        \"country\" : \"Romania\",\n        \"date\" : \"2020-10-10\"\n    },\n    {\n        \"country\" : \"Romania\",\n        \"date\" : \"2019-07-21\"\n    },\n    {\n        \"country\" : \"Italy\",\n        \"date\" : \"2019-12-21\"\n    },\n    {\n        \"country\" : \"France\",\n        \"date\" : \"2019-02-21\"\n    }\n  ]\n}\n```\n\nJSON2:\n```json\n{\n  \"citizenship\" : [ \"Romanian\", \"French\" ]\n}\n```\n\nWe write the **MapNeat** transformation like:\n\n```kotlin\nfun main() {\n    // JSON1 and JSON2 are both String variables \n    val transform = json(fromJson(JSON1)) {\n\n        \"person.id\"         /= 100\n        \"person.firstName\"  *= \"$.first_name\"\n        \"person.lastName\"   *= \"$.last_name\"\n\n        // We can using a nested json assignment instead of using the \".\" notation\n        \"person.meta\" /= json {\n            \"information1\" /= \"ABC\"\n            \"information2\" /= \"ABC2\"\n        }\n        \n        // We can assign a value from a lambda expression\n        \"person.maritalStatus\" /= {\n            if(sourceCtx().read(\"$.married\")) \n                \"married\" \n            else \n                \"unmarried\"\n        }\n\n        \"person.visited\" *= {\n            // We select only the country name from the visits array\n            expression = \"$.visits[*].country\"\n            processor = { countries -\u003e\n                // We don't allow duplications so we create a Set\n                (countries as List\u003cString\u003e).toMutableSet()\n            }\n        }\n\n        // We add a new country using the \"[+]\" notation\n        \"person.visited[+]\" /= \"Ireland\"\n\n        // We merge another array into the visited[] array\n        \"person.visited[++]\" /= mutableListOf(\"Israel\", \"Japan\")\n\n        // We look into a secondary json source - JSON2\n        // Assigning the citizenship array to a temporary path (person._tmp.array)\n        \"person._tmp\" /= json(fromJson(JSON2)) {\n            \"array\" *= \"$.citizenship\"\n        }\n\n        // We copy the content of temporary array into the path were we want to keep it\n        \"person._tmp.array\" % \"person.citizenships\"\n\n        // We remove the temporary path\n        - \"person._tmp\"\n\n        // We rename \"citizenships\" to \"citizenship\" because we don't like typos\n        \"person.citizenships\" %= \"person.citizenship\"\n    }\n\n    println(transform)\n}\n```\n\nAfter all the operations are performed step by step, the output looks like this:\n\n```json\n{\n  \"person\" : {\n    \"id\" : 100,\n    \"firstName\" : \"Gary\",\n    \"lastName\" : \"Young\",\n    \"meta\" : {\n      \"information1\" : \"ABC\",\n      \"information2\" : \"ABC2\"\n    },\n    \"maritalStatus\" : \"unmarried\",\n    \"visited\" : [ \"Romania\", \"Italy\", \"France\", \"Ireland\", \"Israel\", \"Japan\" ],\n    \"citizenship\" : [ \"Romanian\", \"French\" ]\n  }\n}\n```\n\n# Operations\n\nIn the previous example you might wonder what the operators `/=`, `*=`, `%`, `%=`, `-`  are doing. \n\nThose are actually shortcuts methods for the operations we are performing:\n\n| Operator  | Operation | Description |\n| :-------- | :-------- | :-------- |\n| `/=` | `assign`   | Assigns a given constant or a value computed in lambda expression to a certain path in the target JSON (the result). |\n| `*=` | `shift`    | Shifts a portion from the source JSON based on a JSON Path expression. |\n| `%`  | `copy`     | Copies a path from the target JSON to another another path. |\n| `%=` | `move`     | Moves a path from the target JSON to another path. |\n| `-`  | `delete`   | Deletes a path from the target JSON. |\n\nAdditionally, the paths from the target JSON can be \"decorated\" with \"array notation\":\n\n| Array Notation | Description |\n| :------- | :------- |\n| `path[]` | A `new` array will be created through the `assign` and `shift` operations. |\n| `path[+]` | An `append` will be performed through the `assign` and `shift` operations. |\n| `path[++]` | A `merge` will be performed through the `assign` and `shift` operations. |\n  \nIf you prefer, instead of using the operators you can use their equivalent methods.\n\nFor example:\n\n```\n\"person.name\" /= \"Andrei\"\n```\n\nCan be written as:\n\n```\n\"person.name\" assign \"Andrei\"\n```\n\nOr \n\n```\n\"person.name\" *= \"$.user.full_name\"\n```   \n\nCan be written as:\n\n```\n\"person.name\" shift \"$.user.full_name\"\n```\n\nPersonally, I prefer the operator notation (`/=`, `*=`, etc.), but some people consider the methods (`assign`, `shift`) more readable. \n\nFor the rest of the examples the operator notation will be used.\n\n## Assign (`/=`)\n\nThe **Assign** Operation is used to assign a value to a path in the resulting JSON (target).\n\nThe value can be a constant object, or a lambda (`()-\u003e Any`).\n\nExample:\n\n```kotlin\npackage net.andreinc.mapneat.examples\n\nimport net.andreinc.mapneat.dsl.json\n\nconst val A_SRC_1 = \"\"\"\n{\n    \"id\": 380557,\n    \"first_name\": \"Gary\",\n    \"last_name\": \"Young\"\n}    \n\"\"\"\n\nconst val A_SRC_2 = \"\"\"\n{\n    \"photo\": \"http://srcimg.com/100/150\",\n    \"married\": false\n}\n\"\"\"\n\nfun main() {\n    val transformed = json(A_SRC_1) {\n        // Assigning a constant\n        \"user.user_name\" /= \"neo2020\"\n\n        // Assigning value from a lambda expression\n        \"user.first_name\" /= { sourceCtx().read(\"$.first_name\") }\n\n        // Assigning value from another JSON source\n        \"more_info\" /= json(A_SRC_2) {\n            \"married\" /= { sourceCtx().read(\"$.married\") }\n        }\n\n        // Assigning an inner JSON with the same source as the parent\n        \"more_info2\" /= json {\n            \"last_name\" /= { sourceCtx().read(\"$.last_name\") }\n        }\n    }\n    println(transformed)\n}\n```\n\nOutput:\n```json\n{\n  \"user\" : {\n    \"user_name\" : \"neo2020\",\n    \"first_name\" : \"Gary\"\n  },\n  \"more_info\" : {\n    \"married\" : false\n  },\n  \"more_info2\" : {\n    \"last_name\" : \"Young\"\n  }\n}\n```\n\nIn the lambda method we pass to the `/=` operation we have access to:\n* `sourceCtx()` which represents the `ReadContext` of the source. We can use this to read JSON Paths just like in the example above;\n* `targetCtx()` which represents the `ReacContext` of the target. This is calculated each time we call the method. So, it contains only the changes that were made up until that point. In most cases this shouldn't be called.\n\nIn case we are using an inner JSON structure, we also have reference to the parent source and target contexts:\n* `parent.sourceCtx()`\n* `parent.targetCtx()`\n\n`parent()` returns a nullable value, so it needs to be used adding `!!` (double bang).\n\n```\n... {\n    \"something\" /= \"Something Value\"\n    \"person\" /= json {\n        \"innerSomething\" /= { parent()!!.targetCtx().read(\"$.something\") }\n    }\n}\n```\n\nFor more information about `ReadContext`s please check [json-path](https://github.com/json-path/JsonPath)'s documentation.\n\nThe **Assign** operation can also be used in conjunction with left-side array notations (`[]`, `[+]`, `[++]`):\n\n```kotlin\nfun main() {\n    val transformed = json(\"{}\") {\n        println(\"Simple array creation:\")\n        \"a\" /= 1\n        \"b\" /= 1\n        println(this)\n\n        println(\"Adds a new value in the array:\")\n        \"a[+]\" /= 2\n        \"b[+]\" /= true\n        println(this)\n\n        println(\"Merge in an existing array:\")\n        \"b[++]\" /= arrayOf(\"a\", \"b\", \"c\")\n        println(this)\n    }\n}\n```\n\nOutput:\n```\nSimple array creation:\n{\n  \"a\" : 1,\n  \"b\" : 1\n}\nAdds a new value in the array\n{\n  \"a\" : [ 1, 2 ],\n  \"b\" : [ 1, true ]\n}\nMerge in an existing array:\n{\n  \"a\" : [ 1, 2 ],\n  \"b\" : [ 1, true, \"a\", \"b\", \"c\" ]\n}\n```\n\n## Shift (`*=`)\n\nThe **Shift** operation is very similar to the *Assign* operation, but it provides an easier way to query the source JSON using [json-path](https://github.com/json-path/JsonPath).\n\nExample:\n\n```kotlin\npackage net.andreinc.mapneat.examples\n\nimport net.andreinc.mapneat.dsl.json\nimport java.time.LocalDate\nimport java.time.format.DateTimeFormatter\n\nval JSON_VAL = \"\"\"\n{\n  \"id\": 380557,\n  \"first_name\": \"Gary\",\n  \"last_name\": \"Young\",\n  \"photo\": \"http://srcimg.com/100/150\",\n  \"married\": false,\n  \"visits\" : [ \n    {\n        \"country\" : \"Romania\",\n        \"date\" : \"2020-10-10\"\n    },\n    {\n        \"country\" : \"Romania\",\n        \"date\" : \"2019-07-21\"\n    },\n    {\n        \"country\" : \"Italy\",\n        \"date\" : \"2019-12-21\"\n    },\n    {\n        \"country\" : \"France\",\n        \"date\" : \"2019-02-21\"\n    }\n  ]\n}\n\"\"\"\n\nfun main() {\n    val transformed = json(JSON_VAL) {\n        \"user.name.first\" *= \"$.first_name\"\n        // We use an additional processor to capitalise the last Name\n        \"user.name.last\" *= {\n            expression = \"$.last_name\"\n            processor = { (it as String).toUpperCase() }\n        }\n        // We add the photo directly into an array\n        \"user.photos[]\" *= \"$.photo\"\n        // We don't allow duplicates\n        \"user.visits.countries\" *= {\n            expression = \"$.visits[*].country\"\n            processor = { (it as MutableList\u003cString\u003e).toSet().toMutableList() }\n        }\n        // We keep only the last visit\n        \"user.visits.lastVisit\" *= {\n            expression = \"$.visits[*].date\"\n            processor = {\n                (it as MutableList\u003cString\u003e)\n                    .stream()\n                    .map { LocalDate.parse(it, DateTimeFormatter.ISO_DATE) }\n                    .max(LocalDate::compareTo)\n                    .get()\n                    .toString()\n            }\n        }\n    }\n\n    println(transformed)\n}\n```\n\nOutput:\n\n```json\n{\n  \"user\" : {\n    \"name\" : {\n      \"first\" : \"Gary\",\n      \"last\" : \"YOUNG\"\n    },\n    \"photos\" : [ \"http://srcimg.com/100/150\" ],\n    \"visits\" : {\n      \"countries\" : [ \"Romania\", \"Italy\", \"France\" ],\n      \"lastVisit\" : \"2020-10-10\"\n    }\n  }\n}\n```\n\nAs you can see in the above example, each expression can be accompanied with an additional processor method that allows developers to refine the results provided by the JSON path expression.\n\nSimilar to the **Assign** lambdas, `sourceCtx()`, `targetCtx()`, `parent!!.sourceCtx()`, `parent!!.targetCtx()` are also available to the method context and can be used.\n\nIf you want to `Shift` all the source JSON into the target you can use the following transformation:\n\n```\n\"\" *= \"$\n```\n\nOr call the `copySourceToTarget()` method directly.\n\nIn case a field is optional, and you don't want automatically fail the mapping, you can use the `leniency` property:\n\n```kotlin\n\"books\" *=  {\n                expression = \"$.store.broken.path\"\n                lenient = true\n            }\n```\n\n## Copy (`%`)\n\nThe **Copy** Operation moves a certain path from the target JSON to another path in the target JSON.\n\nExample:\n\n```kotlin\npackage net.andreinc.mapneat.examples\n\nimport net.andreinc.mapneat.dsl.json\n\nfun main() {\n    val transformed = json(\"{}\") {\n        \"some.long.path\" /= mutableListOf(\"A, B, C\")\n        \"some.long.path\" % \"copy\"\n        println(this)\n    }\n}\n``` \n\nOutput:\n\n```json\n{\n  \"some\" : {\n    \"long\" : {\n      \"path\" : [ \"A, B, C\" ]\n    }\n  },\n  \"copy\" : [ \"A, B, C\" ]\n}\n```\n\n## Move (`%=`)\n\nThe **Move** operation moves a certain path from the target JSON to a new path in the target JSON.\n\nExample:\n\n```kotlin\npackage net.andreinc.mapneat.examples\n\nimport net.andreinc.mapneat.dsl.json\n\nfun main() {\n    json(\"{}\") {\n        \"array\" /= intArrayOf(1,2,3)\n        \"array\" %= \"a.b.c.d\"\n        println(this)\n    }\n}\n```\n\nOutput:\n\n```json\n{\n  \"a\" : {\n    \"b\" : {\n      \"c\" : {\n        \"d\" : [ 1, 2, 3 ]\n      }\n    }\n  }\n}\n```\n\n## Delete (`-`)\n\nThe **Delete** operation deletes a certain path from the target JSON.\n\nExample:\n\n```kotlin\npackage net.andreinc.mapneat.examples\n\nimport net.andreinc.mapneat.dsl.json\n\nfun main() {\n    json(\"{}\") {\n        \"a.b.c\" /= mutableListOf(1,2,3,4,true)\n        \"a.b.d\" /= \"a\"\n        // deletes the array from \"a.b.c\"\n        - \"a.b.c\"\n        println(this)\n    }\n}\n```\n\nOutput:\n\n```json\n{\n  \"a\" : {\n    \"b\" : {\n      \"d\" : \"a\"\n    }\n  }\n}\n```\n\n# Using **MapNeat** from Java\n\nGiven Kotlin's high level of interoperability with Java, **MapNeat** can be used in any Java application.\n\nThe DSL file should remain kotlin, but it can be called from any Java program, as simple as:\n\n```kotlin\n@file : JvmName(\"Sample\")\n\npackage kotlinPrograms\n\nimport net.andreinc.mapneat.dsl.json\n\nfun personTransform(input: String) : String {\n    return json(input) {\n        \"person.name\" /= \"Andrei\"\n        \"person.age\" /= 13\n    }.getPrettyString()\n}\n```\n\nThe java file:\n\n```java\nimport static kotlinPrograms.Sample.personTransform;\n\npublic class Main {\n    public static void main(String[] args) {\n        // personTransform(String) is the method from Kotlin\n        String person = personTransform(\"{}\");\n        System.out.println(person);\n    }\n}\n```\n\nPS: Configuring the Java application to be Kotlin-enabled it's quite simple, usually IntelliJ is doing this automatically without amy developer intervention.\n\n# Logging\n\nThe library uses log4j2 for logging purposes. \n\nEach transformation gets logged by default to `SYSTEM_OUT`and to `logs/mapneat.log`.\n\nFor tracing and debugging purposes transformations have two IDs (id, parentId - if inner JSONs are used).\n\nE.g.:\n\n```\n19:05:06.204 [main] INFO  net.andreinc.mapneat.dsl.MapNeat - Transformation(id=a739ba94-dedd-4d5b-bd09-03b30693a1ae, parentId=null) INPUT = {\n  \"books\" : [\n    {\n      \"title\" : \"Cool dog\",\n      \"author\" : \"Mike Smith\"\n    },\n    {\n      \"title\": \"Feeble Cat\",\n      \"author\": \"John Cibble\"\n    },\n    {\n      \"title\": \"Morning Horse\",\n      \"author\": \"Kohn Gotcha\"\n    }\n  ],\n  \"address\" : {\n    \"country\" : \"RO\",\n    \"street_number\": 123,\n    \"city\": \"Bucharest\"\n  }\n}\n19:05:06.209 [main] INFO  net.andreinc.mapneat.dsl.MapNeat - Transformation(id=a739ba94-dedd-4d5b-bd09-03b30693a1ae, parentId=a739ba94-dedd-4d5b-bd09-03b30693a1ae) INPUT = INHERITED\n19:05:06.244 [main] INFO  net.andreinc.mapneat.operation.Assign - (transformationId=a739ba94-dedd-4d5b-bd09-03b30693a1ae) \"fullName\" ASSIGN(/=) \"John Cibble\"\n19:05:06.246 [main] INFO  net.andreinc.mapneat.operation.Assign - (transformationId=a739ba94-dedd-4d5b-bd09-03b30693a1ae) \"firstName\" ASSIGN(/=) \"John\"\n19:05:06.246 [main] INFO  net.andreinc.mapneat.operation.Assign - (transformationId=a739ba94-dedd-4d5b-bd09-03b30693a1ae) \"lastName\" ASSIGN(/=) \"Cibble\"\n19:05:06.248 [main] INFO  net.andreinc.mapneat.operation.Delete - (transformationId=a739ba94-dedd-4d5b-bd09-03b30693a1ae) DELETE(-) \"fullName\"\n```\n\n`(transformationId=a739ba94-dedd-4d5b-bd09-03b30693a1ae)` =\u003e represents the id\n`Transformation(id=a739ba94-dedd-4d5b-bd09-03b30693a1ae, parentId=a739ba94-dedd-4d5b-bd09-03b30693a1ae) INPUT = INHERITED` =\u003e marks the parentId\n   \n# Contributing and Roadmap\n\nThe highlevel roadmap for the library at this moment is:\n1. Make mapneat a command-line tool\n2. Create a mapneat-server to serve transformation sync / async\n\nAnyone if free to contribute. You know how github works:).\n\n\n----------------\n\nFor more code examples, please check: https://github.com/nomemory/mapneat-examples\n","funding_links":[],"categories":["Kotlin"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnomemory%2Fmapneat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnomemory%2Fmapneat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnomemory%2Fmapneat/lists"}