{"id":24558159,"url":"https://github.com/rse/extraction","last_synced_at":"2025-04-19T09:59:08.596Z","repository":{"id":58220926,"uuid":"48803320","full_name":"rse/extraction","owner":"rse","description":"Tree Extraction for JavaScript Object Graphs","archived":false,"fork":false,"pushed_at":"2023-08-29T09:24:53.000Z","size":8525,"stargazers_count":72,"open_issues_count":0,"forks_count":4,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-13T11:52:21.184Z","etag":null,"topics":["dsl","extraction","javascript","json","query","tree"],"latest_commit_sha":null,"homepage":"http://extraction.js.org/","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rse.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2015-12-30T14:07:00.000Z","updated_at":"2024-12-20T02:55:28.000Z","dependencies_parsed_at":"2023-12-15T20:10:06.934Z","dependency_job_id":null,"html_url":"https://github.com/rse/extraction","commit_stats":{"total_commits":142,"total_committers":1,"mean_commits":142.0,"dds":0.0,"last_synced_commit":"6d06b54bfda195ec92ca75ad52e79aa113989c55"},"previous_names":[],"tags_count":36,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rse%2Fextraction","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rse%2Fextraction/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rse%2Fextraction/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rse%2Fextraction/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rse","download_url":"https://codeload.github.com/rse/extraction/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249670113,"owners_count":21308674,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dsl","extraction","javascript","json","query","tree"],"created_at":"2025-01-23T05:47:32.591Z","updated_at":"2025-04-19T09:59:08.572Z","avatar_url":"https://github.com/rse.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n[Extraction](http://extraction.js.org/)\n==========\n\nTree Extraction for JavaScript Object Graphs\n\n[![github (author stars)](https://img.shields.io/github/stars/rse?logo=github\u0026label=author%20stars\u0026color=%233377aa)](https://github.com/rse)\n[![github (author followers)](https://img.shields.io/github/followers/rse?label=author%20followers\u0026logo=github\u0026color=%234477aa)](https://github.com/rse)\n\u003cbr/\u003e\n[![npm (project release)](https://img.shields.io/npm/v/extraction?logo=npm\u0026label=npm%20release\u0026color=%23cc3333)](https://npmjs.com/extraction)\n[![npm (project downloads)](https://img.shields.io/npm/dm/extraction?logo=npm\u0026label=npm%20downloads\u0026color=%23cc3333)](https://npmjs.com/extraction)\n\nAbout\n-----\n\nExtraction is a small JavaScript library for extracting object trees\nfrom arbitrary object graphs. Object graphs usually have cycles and\ncontain many information. Hence, the clue is that the extracted object\ntrees use links to break object reference cycles and can be just\npartial by leaving out non-requested information. The tree extraction\nis controlled with a custom JSON-style query language. The object tree\nis structurally derived from the object graph, but contains no\nreferences to the original objects and hence can be further mutated by\nthe caller.\n\nThe Extraction library is intended for two main use cases: primarily, to\nsupport the generation of responses in REST APIs based on object graphs\n(where the cycle problem and the partial information problem has to be\nresolved) and, secondarily, to support the persisting and restoring of\narbitrary in-memory object graph structures (where the cycle problem has\nto be resolved, too).\n\nNotice: this library intentionally does provide only a query language\nfor the tree extraction (starting at a certain tree root node) and not\nalso a query language for locating the tree root node. Locating nodes in\na graph is not within the scope of this library.\n\n[Play around with Extraction in the interactive demo!](http://extraction.js.org/)\n\nSneak Preview\n-------------\n\n\u003cimg src=\"smp/graph.png\" align=\"right\" alt=\"\"/\u003e\n\n```js\nimport { extract, reify } from \"./lib/extraction\"\nimport { expect } from \"chai\"\n\n/*  the graph  */\nvar Graph = {\n    Person: [\n        { id: 7,   name: \"God\",   tags: [ \"good\", \"nice\" ] },\n        { id: 666, name: \"Devil\", tags: [ \"bad\", \"cruel\" ] } ],\n    Location: [\n        { id: 0,   name: \"World\"  },\n        { id: 1,   name: \"Heaven\" },\n        { id: 999, name: \"Hell\"   } ] }\nGraph.Person[0].home    = Graph.Location[1]\nGraph.Person[1].home    = Graph.Location[2]\nGraph.Person[1].rival   = Graph.Person[0]\nGraph.Person[0].rival   = Graph.Person[1]\nGraph.Location[1].owner = Graph.Person[0]\nGraph.Location[2].owner = Graph.Person[1]\nGraph.Location[0].subs  = [ Graph.Location[1],\n                            Graph.Location[2] ]\n\n/*  use case 1: tree extraction  */\nlet tree = extract(Graph.Person[0],\n    \"{ name, rival: { home: { *, !owner, !subs } } }\")\nexpect(tree).to.be.deep.equal(\n    { name: \"God\", rival: { home: { id: 999, name: \"Hell\" } } })\n\n/*  use case 2: graph persistance  */\nlet storage = JSON.stringify(extract(Graph, \"{ -\u003e oo }\"))\nexpect(reify(JSON.parse(storage))).to.be.deep.equal(Graph)\n```\n\nInstallation\n------------\n\n```shell\n$ npm install extraction\n```\n\nUsage\n-----\n\nThe Extraction library exposes two API functions (signatures given in TypeScript notation):\n\n### `extract`\n\nThis is the main API method for extracting an object tree from an object\ngraph with the help of a tree extraction DSL.\n\n```\nextraction.extract(graph: object, spec: string, options?: object): object\n```\n\n- The `graph` argument has to be an Array of Object and be any start node in the graph.\n\n- The `spec` argument is the tree extraction specification Domain-Specific Language (DSL).\n  It has to follow the following PEG-style grammar:\n\n    RHS      |     | LHS\n    ---------|-----|---------------------------\n    spec     | ::= | object / array\n    object   | ::= | `\"{\"` content? `\"}\"`\n    array    | ::= | `\"[\"` content? `\"]\"`\n    content  | ::= | (`\"-\u003e\"` num) / (field (`\",\"` field)*)\n    field    | ::= | (property `\":\"` spec) / (`\"!\"`? property)\n    property | ::= | id / `\"*\"` / (num `\"..\"` num) / num\n    num      | ::= | (`\"-\"`? `[0-9]`+) / `\"-oo\"` / `\"oo\"`\n    id       | ::= | `[$a-zA-Z_][$a-zA-Z0-9_]`\\*\n\n  Hint: the matching of multiple `field` in `content` follows a last-match semantic!\n\n- The `options` argument is optional and can contain the following properties:\n\n    - `procValueBefore: (value: any, path: string) =\u003e any`:\u003cbr/\u003e\n       Pre-process a value (object or property value) at `path` before\n       it is taken into account. A caller could use this to convert the\n       value from a custom type into a standard JavaScript type.\n\n    - `procValueAfter: (value: any, path: string) =\u003e any`:\u003cbr/\u003e\n       Post-process a value (object or property value) at `path` after\n       it was taken into account. A caller could use this to convert the\n       value into an external representation like JSON or XML.\n\n    - `makeRefValue: (value: Object, pathNow: string, pathFirst: string) =\u003e any`:\u003cbr/\u003e\n       Make an object reference out of an object `value`, which is now found (again)\n       at path `pathNow` and the first-time found at `pathFirst`. The default\n       is to use `pathFirst` as the reference, but a caller could also use\n       a stub for `value` (usually based on just the OID of it) as the reference.\n\n    - `getKeysOfObject: (value: Object) =\u003e String[]`:\u003cbr/\u003e\n       Retrieve the keys of an object `value`. A caller could use this\n       to provide the keys of custom objects which are either\n       non-enumerable or perhaps are based on getter/setter on the\n       prototype chain.\n\n    - `debug: boolean`:\u003cbr/\u003e\n       Print debug information about internal processing.\n\n### `reify`\n\nThis is a utility API method to re-generate an object graph from an\nobject tree by reifying all self-references back to the referenced\nobjects.\n\n```\nextraction.reify(tree: object, options?: object): object\n```\n\n- The `tree` argument is the root of an object tree which should be traversed.\n\n- The `options` argument is optional and can contain the following properties:\n\n    - `procValueBefore: (value: any, path: string) =\u003e any`:\u003cbr/\u003e\n       Pre-process a value (object or property value) after it is taken into account.\n       A caller could use this to convert the value from an external representation\n       like JSON or XML.\n\n    - `procValueAfter: (value: any, path: string) =\u003e any`:\u003cbr/\u003e\n       Post-process a value (object or property value) after it was taken into account.\n       A caller could use this to convert the value from a standard type into a custom\n       JavaScript type.\n\n    -  `isReference: (value: any, path: string) =\u003e boolean`:\u003cbr/\u003e\n       Determine whether `value` is an object reference.\n\n    -  `getObject: (value: any, path: string) =\u003e any`:\u003cbr/\u003e\n       Fetch the underlying object from an object reference `value`, found at `path`.\n\n    -  `setObject: (value: any, path: string) =\u003e void`:\u003cbr/\u003e\n       Store an underlying object `value`, found at `path`.\n\n    - `debug: boolean`:\u003cbr/\u003e\n       Print debug information about internal processing.\n\nExample\n-------\n\nSuppose we have an object graph (aka \"business model\") based\non two entity definitions (in pseudo language):\n\n```\nPerson {\n    id:    number\n    name:  string\n    tags:  string+\n    home:  Location\n    rival: Person?\n}\nLocation {\n    id:     number\n    name:   string\n    owner:  Person?\n    subs:   Location*\n}\n```\n\nA possible JavaScript instanciation of this object graph definition then\ncould be:\n\n```js\nvar Graph = {\n    Person: [\n        { id: 7,   name: \"God\",   tags: [ \"good\", \"nice\" ] },\n        { id: 666, name: \"Devil\", tags: [ \"bad\", \"cruel\" ] }\n    ],\n    Location: [\n        { id: 0,   name: \"World\" },\n        { id: 1,   name: \"Heaven\" },\n        { id: 999, name: \"Hell\" }\n    ]\n}\n\nGraph.Person[0].home    = Graph.Location[1]\nGraph.Person[1].home    = Graph.Location[2]\n\nGraph.Person[1].rival   = Graph.Person[0]\nGraph.Person[0].rival   = Graph.Person[1]\n\nGraph.Location[0].subs  = [ Graph.Location[1], Graph.Location[2] ]\n\nGraph.Location[1].owner = Graph.Person[0]\nGraph.Location[2].owner = Graph.Person[1]\n```\n\nBecause of the relationship cycles in this graph, you cannot easily\nserialize this graph as JSON with plain `JSON.stringify()` as it\nwill detect but not handle the cycles correctly. With the Extraction library\nyou can serialize and deseralize this graph just fine:\n\n```js\n/*  import external requirements  */\nimport { extract, reify } from \"extraction\"\nimport { expect }         from \"chai\"\nimport { inspect }        from \"util\"\n\n/*  extract entire graph as a tree with self-references  */\nlet tree = extract(Graph, \"{ -\u003e oo }\")\nconsole.log(inspect(tree, { depth: null }))\n\n//  { Person:\n//     [ { id: 7,\n//         name: 'God',\n//         tags: [ 'good', 'nice' ],\n//         home: { id: 1, name: 'Heaven', owner: '@self.Person.0' },\n//         rival:\n//          { id: 666,\n//            name: 'Devil',\n//            tags: [ 'bad', 'cruel' ],\n//            home: { id: 999, name: 'Hell', owner: '@self.Person.0.rival' },\n//            rival: '@self.Person.0' } },\n//       '@self.Person.0.rival' ],\n//    Location:\n//     [ { id: 0,\n//         name: 'World',\n//         subs: [ '@self.Person.0.home', '@self.Person.0.rival.home' ] },\n//       '@self.Person.0.home',\n//       '@self.Person.0.rival.home' ] }\n\n/*  as the tree has no cycles, it can be serialized/unserialized just fine  */\ntree = JSON.parse(JSON.stringify(tree))\n\n/*  reify the object references to gain the original graph again  */\nlet GraphNew = reify(tree)\nexpect(GraphNew).to.be.deep.equal(Graph)\n```\n\nNow suppose we have a REST API where we want to let Persons\nwith their home Location be queried:\n\n```js\n/*  import external requirements  */\nimport HAPI        from \"hapi\"\nimport { extract } from \"./lib/extraction\"\n\n/*  import sample graph  */\nimport Graph       from \"./sample-graph\"\n\n/*  establish a new REST service  */\nvar server = new HAPI.Server()\nserver.connection({ address: \"0.0.0.0\", port: \"12345\" })\n\n/*  provide REST endpoints  */\nserver.route({\n    method: \"GET\",\n    path: \"/persons/{id}\",\n    handler: (request, reply) =\u003e {\n        let id = parseInt(request.params.id)\n        let person = Graph.Person.find((person) =\u003e person.id === id)\n        let response = JSON.stringify(extract(\n            person, \"{ id, name, home: { id, name } }\"\n        ))\n        reply(response)\n    }\n})\n\n/*  fire up REST service  */\nserver.start((err) =\u003e {\n    if (err)\n        console.log(err)\n})\n```\n\nQuerying the two Persons yields:\n\n```\n$ curl http://127.0.0.1:12345/persons/7\n{\"id\":7,\"name\":\"God\",\"home\":{\"id\":1,\"name\":\"Heaven\"}}\n\n$ curl http://127.0.0.1:12345/persons/6660\n{\"id\":666,\"name\":\"Devil\",\"home\":{\"id\":999,\"name\":\"Hell\"}}\n```\n\nFinally, instead of extracting a tree and then encoding it\nas JSON, you can immediately encode it during extraction:\n\n```js\nextraction.extract(Graph, \"{ -\u003e oo }\", {\n    procValueAfter: (value, path) =\u003e {\n        if (typeof value === \"object\" \u0026\u0026 value !== null) {\n            if (value instanceof Array)\n                value = \"[\" + value.join(\",\") + \"]\"\n            else\n                value = \"{\" + Object.keys(value).map(function (key) {\n                    return JSON.stringify(key) + \":\" + value[key]\n                }).join(\",\") + \"}\"\n        }\n        else\n            value = JSON.stringify(value)\n        return value\n    }\n}))\n// {\"Person\":[{\"id\":7,\"name\":\"God\",\"tags\":[\"good\",\"nice\"],\n// \"home\":{\"id\":1,\"name\":\"Heaven\",\"owner\":\"@self.Person.0\"},\n// \"rival\":{\"id\":666,\"name\":\"Devil\",\"tags\":[\"bad\",\"cruel\"],\n// \"home\":{\"id\":999,\"name\":\"Hell\",\"owner\":\"@self.Person.0.rival\"},\n// \"rival\":\"@self.Person.0\"}},\"@self.Person.0.rival\"],\n// \"Location\":[{\"id\":0,\"name\":\"World\",\"subs\":[\"@self.Person.0.home\",\n// \"@self.Person.0.rival.home\"]},\"@self.Person.0.home\",\n// \"@self.Person.0.rival.home\"]}\n```\n\nImplementation Notice\n---------------------\n\nAlthough the Extraction library is written in ECMAScript 6, it is\ntranspiled to ECMAScript 5 and this way runs in really all(!) current\n(as of 2017/Q1) JavaScript environments, of course.\n\nLicense\n-------\n\nCopyright \u0026copy; 2015-2023 Dr. Ralf S. Engelschall (http://engelschall.com/)\n\nPermission is hereby granted, free of charge, to any person obtaining\na copy of this software and associated documentation files (the\n\"Software\"), to deal in the Software without restriction, including\nwithout limitation the rights to use, copy, modify, merge, publish,\ndistribute, sublicense, and/or sell copies of the Software, and to\npermit persons to whom the Software is furnished to do so, subject to\nthe following conditions:\n\nThe above copyright notice and this permission notice shall be included\nin all copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,\nEXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\nMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.\nIN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY\nCLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,\nTORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE\nSOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frse%2Fextraction","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frse%2Fextraction","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frse%2Fextraction/lists"}