{"id":13485894,"url":"https://github.com/bazaarvoice/jolt","last_synced_at":"2025-05-13T22:05:51.319Z","repository":{"id":6960300,"uuid":"8212556","full_name":"bazaarvoice/jolt","owner":"bazaarvoice","description":"JSON to JSON transformation library written in Java.","archived":false,"fork":false,"pushed_at":"2024-07-26T14:57:21.000Z","size":1305,"stargazers_count":1606,"open_issues_count":403,"forks_count":334,"subscribers_count":70,"default_branch":"master","last_synced_at":"2025-04-29T13:45:55.343Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bazaarvoice.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"contributing.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2013-02-15T04:09:50.000Z","updated_at":"2025-04-28T14:32:23.000Z","dependencies_parsed_at":"2024-10-29T10:43:25.512Z","dependency_job_id":null,"html_url":"https://github.com/bazaarvoice/jolt","commit_stats":{"total_commits":303,"total_committers":42,"mean_commits":7.214285714285714,"dds":0.7425742574257426,"last_synced_commit":"468d67ec614bf6ee7a0daf13b491c97aeb41b461"},"previous_names":[],"tags_count":31,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bazaarvoice%2Fjolt","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bazaarvoice%2Fjolt/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bazaarvoice%2Fjolt/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bazaarvoice%2Fjolt/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bazaarvoice","download_url":"https://codeload.github.com/bazaarvoice/jolt/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254036817,"owners_count":22003653,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T18:00:33.610Z","updated_at":"2025-05-13T22:05:46.311Z","avatar_url":"https://github.com/bazaarvoice.png","language":"Java","readme":"Jolt\n========\n\nJSON to JSON transformation library written in Java where the \"specification\" for the transform is itself a JSON document.\n\n### Useful For\n\n1. Transforming JSON data from ElasticSearch, MongoDb, Cassandra, etc before sending it off to the world\n1. Extracting data from a large JSON documents for your own consumption\n\n## Table of Contents\n\n   1. [Overview](#Overview)\n   2. [Documentation](#Documentation)\n   3. [Shiftr Transform DSL](#Shiftr_Transform_DSL)\n   4. [Demo](#Demo)\n   5. [Getting Started](#Getting_Started)\n   6. [Getting Transform Help](#Getting_Transform_Help)\n   7. [Why Jolt Exists](#Why_Jolt_Exists)\n   8. [Alternatives](#Alternatives)\n   9. [Performance](#Performance)\n   10. [CLI](#CLI)\n   11. [Code Coverage](#Code_Coverage)\n   12. [Release Notes](#Release_Notes)\n\n## \u003ca name=\"Overview\"\u003e\u003c/a\u003e Overview\n\nJolt :\n\n* provides a set of transforms, that can be \"chained\" together to form the overall JSON to JSON transform.\n* focuses on transforming the *structure* of your JSON data, not manipulating specific values\n    * The idea being: use Jolt to get most of the structure right, then write code to fix values\n* consumes and produces \"hydrated\" JSON : in-memory tree of Maps, Lists, Strings, etc.\n    * use Jackson (or whatever) to serialize and deserialize the JSON text\n\n### Stock Transforms\n\nThe Stock transforms are:\n\n    shift       : copy data from the input tree and put it the output tree\n    default     : apply default values to the tree\n    remove      : remove data from the tree\n    sort        : sort the Map key values alphabetically ( for debugging and human readability )\n    cardinality : \"fix\" the cardinality of input data.  Eg, the \"urls\" element is usually a List, but if there is only one, then it is a String\n\nEach transform has its own DSL (Domain Specific Language) in order to facilitate its narrow job.\n\nCurrently, all the Stock transforms just effect the \"structure\" of the data.\nTo do data manipulation, you will need to write Java code.   If you write your Java \"data manipulation\" code to implement the Transform interface, then you can insert your code in the transform chain.\n\nThe out-of-the-box Jolt transforms should be able to do most of your structural transformation, with custom Java Transforms implementing your data manipulation.\n\n## \u003ca name=\"Documentation\"\u003e\u003c/a\u003e Documentation\n\nJolt [Slide Deck](https://docs.google.com/presentation/d/1sAiuiFC4Lzz4-064sg1p8EQt2ev0o442MfEbvrpD1ls/edit?usp=sharing) : covers motivation, development, and transforms.\n\nJavadoc explaining each transform DSL :\n\n* [shift](https://github.com/bazaarvoice/jolt/blob/master/jolt-core/src/main/java/com/bazaarvoice/jolt/Shiftr.java)\n* [default](https://github.com/bazaarvoice/jolt/blob/master/jolt-core/src/main/java/com/bazaarvoice/jolt/Defaultr.java)\n* [remove](https://github.com/bazaarvoice/jolt/blob/master/jolt-core/src/main/java/com/bazaarvoice/jolt/Removr.java)\n* [cardinality](https://github.com/bazaarvoice/jolt/blob/master/jolt-core/src/main/java/com/bazaarvoice/jolt/CardinalityTransform.java)\n* [sort](https://github.com/bazaarvoice/jolt/blob/master/jolt-core/src/main/java/com/bazaarvoice/jolt/Sortr.java)\n* full qualified Java ClassName : Class implements the Transform or ContextualTransform interfaces, and can optionally be SpecDriven (marker interface)\n    * [Transform](https://github.com/bazaarvoice/jolt/blob/master/jolt-core/src/main/java/com/bazaarvoice/jolt/Transform.java) interface\n    * [SpecDriven](https://github.com/bazaarvoice/jolt/blob/master/jolt-core/src/main/java/com/bazaarvoice/jolt/SpecDriven.java)\n        * where the \"input\" is \"hydrated\" Java version of your JSON Data\n\nRunning a Jolt transform means creating an instance of [Chainr](https://github.com/bazaarvoice/jolt/blob/master/jolt-core/src/main/java/com/bazaarvoice/jolt/Chainr.java)  with a list of transforms.\n\nThe JSON spec for Chainr looks like : [unit test](https://github.com/bazaarvoice/jolt/blob/master/jolt-core/src/test/resources/json/chainr/integration/firstSample.json).\n\nThe Java side looks like :\n\n``` java\nChainr chainr = JsonUtils.classpathToList( \"/path/to/chainr/spec.json\" );\n\nObject input = elasticSearchHit.getSource(); // ElasticSearch already returns hydrated JSon\n\nObject output = chainr.transform( input );\n\nreturn output;\n```\n\n### \u003ca name=\"Shiftr_Transform_DSL\"\u003e\u003c/a\u003e Shiftr Transform DSL\n\nThe Shiftr transform generally does most of the \"heavy lifting\" in the transform chain.\nTo see the Shiftr DSL in action, please look at our unit tests ([shiftr tests](https://github.com/bazaarvoice/jolt/tree/master/jolt-core/src/test/resources/json/shiftr)) for nice bite sized transform examples, and read the extensive Shiftr [javadoc](https://github.com/bazaarvoice/jolt/blob/master/jolt-core/src/main/java/com/bazaarvoice/jolt/Shiftr.java).\n\nOur unit tests follow the pattern :\n\n``` json\n{\n    \"input\": {\n        // sample input\n    },\n\n    \"spec\": {\n        // transform spec\n    },\n\n    \"expected\": {\n        // what the output of the transform looks like\n    }\n}\n```\n\nWe read in \"input\", apply the \"spec\", and [Diffy](https://github.com/bazaarvoice/jolt/blob/master/json-utils/src/main/java/com/bazaarvoice/jolt/Diffy.java) it against the \"expected\".\n\nTo learn the Shiftr DSL, examine \"input\" and \"output\" json, get an understanding of how data is moving, and *then* look at the transform spec to see how it facilitates the transform.\n\nFor reference, [this](https://github.com/bazaarvoice/jolt/blob/master/jolt-core/src/test/resources/json/shiftr/firstSample.json) was the very first test we wrote.\n\n\n## \u003ca name=\"Demo\"\u003e\u003c/a\u003e Demo\n\nThere is a demo available at [jolt-demo.appspot.com](http://jolt-demo.appspot.com/).\nYou can paste in JSON input data and a Spec, and it will post the data to server and run the transform.\n\nNote\n\n* it is hosted on a free Google App Engine instance, so it may take a minute to spin up.\n* it validates in input JSON and spec client side.\n\n## \u003ca name=\"Getting_Started\"\u003e\u003c/a\u003e Getting Started\n\nGetting started code wise has its [own doc](gettingStarted.md).\n\n## \u003ca name=\"Getting_Transform_Help\"\u003e\u003c/a\u003e Getting Transform Help\n\nIf you can't get a transform working and you need help, create and Issue in Jolt (for now).\n\nMake sure you include what your \"input\" is, and what you want your \"output\" to be.\n\n## \u003ca name=\"Why_Jolt_Exists\"\u003e\u003c/a\u003e Why Jolt Exists\n\nAside from writing your own custom code to do a transform, there are two general approaches to doing a JSON to JSON transforms in Java.\n\n1) JSON -\u003e XML -\u003e XSLT or STX -\u003e XML -\u003e JSON\n\nAside from being a Rube Goldberg approach, XSLT is more complicated than Jolt because it is trying to do the whole transform with a single DSL.\n\n2) Write a Template (Velocity, FreeMarker, etc) that take hydrated JSON input and write textual JSON output\n\nWith this approach you are working from the output format backwards to the input, which is complex for any non-trivial transform.\nEg, the structure of your template will be dictated by the output JSON format, and you will end up coding a parallel tree walk of the input data and the output format in your template.\nJolt works forward from the input data to the output format which is simpler, and it does the parallel tree walk for you.\n\n## \u003ca name=\"Alternatives\"\u003e\u003c/a\u003e Alternatives\n\nBeing in the Java JSON processing \"space\", here are some other interesting JSON manipulation tools to look at / consider :\n\n* [jq](https://stedolan.github.io/jq) - Awesome command line tool to extract data from JSON files (use it all the time, available via brew)\n* [JsonPath](https://github.com/jayway/JsonPath) - Java : Extract data from JSON using XPATH like syntax.\n* [JsonSurfer](https://github.com/jsurfer/JsonSurfer) - Java : Streaming JsonPath processor dedicated to processing big and complicated JSON data.\n\n## \u003ca name=\"Performance\"\u003e\u003c/a\u003e Performance\n\nThe primary goal of Jolt was to improve \"developer speed\" by providing the ability to have a declarative rather than imperative transforms.\nThat said, Jolt should have a better runtime than the alternatives listed above.\n\nWork has been done to make the stock Jolt transforms fast:\n\n1. Transforms can be initialized once with their spec, and re-used many times in a multi-threaded environment.\n    * We reuse initialized Jolt transforms to service multiple web requests from a DropWizard service.\n2. \"*\" wildcard logic was redone to reduce the use of Regex in the common case, which was a dramatic speed improvement.\n3. The parallel tree walk performed by Shiftr was optimized.\n\nTwo things to be aware of :\n\n1. Jolt is not \"stream\" based, so if you have a very large Json document to transform you need to have enough memory to hold it.\n2. The transform process will create and discard a lot of objects, so the garbage collector will have work to do.\n\n## \u003ca name=\"CLI\"\u003e\u003c/a\u003e Jolt CLI\n\nJolt Transforms and tools can be run from the command line. Command line interface doc [here](cli/README.md).\n\n## \u003ca name=\"Code_Coverage\"\u003e\u003c/a\u003e Code Coverage\n\n[![Build Status](https://secure.travis-ci.org/bazaarvoice/jolt.png)](http://travis-ci.org/bazaarvoice/jolt)\n\nFor the moment we have Cobertura configured in our poms.\n\n``` sh\nmvn cobertura:cobertura\nopen jolt-core/target/site/cobertura/index.html\n```\n\nCurrently, for the jolt-core artifact, code coverage is at 89% line, and 83% branch.\n\n## \u003ca name=\"Release_Notes\"\u003e\u003c/a\u003e Release Notes\n\n[Versions and Release Notes available here](https://github.com/bazaarvoice/jolt/releases).\n","funding_links":[],"categories":["Java","Recently Updated","Projects","项目","\u003ca name=\"Java\"\u003e\u003c/a\u003eJava","JSON Processing"],"sub_categories":["[Apr 09, 2025](/content/2025/04/09/README.md)","JSON","JSON Processing","Data Mapping Solution"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbazaarvoice%2Fjolt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbazaarvoice%2Fjolt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbazaarvoice%2Fjolt/lists"}