{"id":40685832,"url":"https://github.com/crs4/flatehr","last_synced_at":"2026-01-21T10:43:55.128Z","repository":{"id":60509207,"uuid":"371094819","full_name":"crs4/flatehr","owner":"crs4","description":"tool for generating openEHR compositions from other formats","archived":false,"fork":false,"pushed_at":"2025-02-20T09:12:36.000Z","size":1041,"stargazers_count":16,"open_issues_count":4,"forks_count":2,"subscribers_count":3,"default_branch":"develop","last_synced_at":"2025-11-27T15:21:10.860Z","etag":null,"topics":["data-conversion","etl","healthcare","interoperability","openehr"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/crs4.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-05-26T16:07:33.000Z","updated_at":"2025-03-20T22:32:22.000Z","dependencies_parsed_at":"2025-02-12T18:21:40.360Z","dependency_job_id":"6bfcb05a-6a00-4328-b301-b69dcdd07231","html_url":"https://github.com/crs4/flatehr","commit_stats":null,"previous_names":[],"tags_count":31,"template":false,"template_full_name":null,"purl":"pkg:github/crs4/flatehr","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/crs4%2Fflatehr","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/crs4%2Fflatehr/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/crs4%2Fflatehr/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/crs4%2Fflatehr/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/crs4","download_url":"https://codeload.github.com/crs4/flatehr/tar.gz/refs/heads/develop","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/crs4%2Fflatehr/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28631962,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-21T04:47:28.174Z","status":"ssl_error","status_checked_at":"2026-01-21T04:47:22.943Z","response_time":86,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-conversion","etl","healthcare","interoperability","openehr"],"created_at":"2026-01-21T10:43:55.046Z","updated_at":"2026-01-21T10:43:55.115Z","avatar_url":"https://github.com/crs4.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# FLATEHR\n![Python](https://img.shields.io/badge/python-3.8%20%7C%203.9%20%7C%203.10-blue)\n[![CI](https://github.com/crs4/flatehr/actions/workflows/main.yaml/badge.svg)](https://github.com/crs4/flatehr/actions/workflows/main.yaml)\n![test](https://github.com/crs4/flatehr/raw/master/docs/reports/tests-badge.svg)\n![coverage](https://github.com/crs4/flatehr/raw/master/docs/reports/coverage-badge.svg)\n![flake8](https://github.com/crs4/flatehr/raw/master/docs/reports/flake8-badge.svg)\n\n**FLATEHR** is a *low-code* Python tool for creating [openEHR](https://www.openehr.org/) **compositions** from different kind of sources. At the moment, **xml** and **json** sources are supported.\nGenerated compositions are formatted according to the [flat (simSDT) format](https://specifications.openehr.org/releases/ITS-REST/latest/simplified_data_template.html). \n\n\n**TL;DR:** it allows to populate compositions mapping key-values generated from a source (for example a XPATH and its relative values from an xml file) to simSDT paths. \nThe mapping is configured via a **yaml** file.\nSee the [examples section](#Examples) for more details.\n\n## Features\n * N:N mappings between source keys and flat paths\n * source keys order preserved, helpful for properly managed multiple cardinality RM objects\n * render values using jinja templates\n * use a value map to converts the source values\n * retrieve values from the web template using jq\n * set null flavour when some mapping is missing\n * set missing required values to the default value (defined in the web template)\n * automatic configuration skeleton generation\n * inspect a template\n * user defined functions\n\n## Installation\n### From pip\n\nRun:\n```\npip install flatehr\n\n```\n\n### From sources:\n\nDependencies:\n * [Poetry](https://python-poetry.org/): ```pip install poetry```\n\n For installing FLATEHR, first enable virtualenv:\n\n```\n$ poetry shell\n```\n\nThen run:\n\n```\n$ make install\n```\n\n### Docker\n\nYou can also use the docker image:\n```\ndocker run crs4/flatehr -h\n```\n\n\n\n## Architecture\n![architecture](https://github.com/crs4/flatehr/raw/master/docs/architecture.png)\nFLATEHR extracts an **ordered list of key-value tuples** from a source according to a [configuration file](#configuration). \nKeys are identified by XPATH or [JSONPATH](https://github.com/h2non/jsonpath-ng) strings, depending on the type of the source.\nA **flat composition** and optionally an **external ehr id** are then generated according to the configured mappings, \nand can be submitted to an openEHR server suppporting the simSDT syntax.\n\n\n## Configuration\n\nThe configuration for generating a composition (see [this](#generating-a-configuration-file) \nand [this](#generating-a-composition)) is written in YAML file.\nFor a quick overview, take a look at the [examples section](#examples).\n\nHere is the supported syntax:\n\n* [paths](#paths)\n* [paths.\\\u003cpath\\\u003e](#paths.\\\u003cpath\\\u003e)\n* [paths.\\\u003cpath\\\u003e.maps_to](#paths.\\\u003cpath\\\u003e.maps_to)\n* [paths.\\\u003cpath\\\u003e.suffixes](#paths.\\\u003cpath\\\u003e.suffixes)\n* [paths.\\\u003cpath\\\u003e.suffixes.\\\u003csuffix\\\u003e](#paths.\\\u003cpath\\\u003e.suffixes.\\\u003csuffix\\\u003e)\n* [paths.\\\u003cpath\\\u003e.suffixes.\\\u003csuffix\\\u003e.value](#paths.\\\u003cpath\\\u003e.suffixes.\\\u003csuffix\\\u003e.value)\n* [paths.\\\u003cpath\\\u003e.suffixes.\\\u003csuffix\\\u003e.jq](#paths.\\\u003cpath\\\u003e.suffixes.\\\u003csuffix\\\u003e.jq)\n* [paths.\\\u003cpath\\\u003e.value_map](#paths.\\\u003cpath\\\u003e.value_map)\n* [paths.\\\u003cpath\\\u003e.null_flavor](#paths.\\\u003cpath\\\u003e.null_flavor)\n* [set_missing_required_to_default](#set_missing_required_to_default)\n* [ehr_id](#ehr_id)\n* [ehr_id.maps_to](#ehr_id.maps_to)\n* [ehr_id.value](#ehr_id.value)\n\n#### paths\n---\nThe set of simSDT paths to be mapped.\n\n#### paths.\\\u003cpath\\\u003e\n---\nA valid simSDT path.\nFor example:\n```\npaths:\n  test/patient_data/gender/biological_sex:\n```\n\nCtx paths can be used too:\n\n```\npaths:\n  ctx/language:\n```\n\nValues can be a string (static mapping) or an object (dynamic mapping, see [suffixes](#paths.\\\u003cpath\\\u003e.suffixes.\\\u003csuffix\\\u003e)):\n\n```\npaths:\n  ctx/language: en\n```\n\nMultiple cardinality RM objects can be mapped, so that they are increased \neach time the source key appears (only one maps_to is possible in this case)\n\n```\npaths:\n  test/histopathology/result_group/laboratory_test_result/any_event:\n    maps_to:\n      - \"$..Event[?(@.@eventtype == 'Histopathology')]\"\n\n```\n\nIt is also possible to set all the children paths to a multiple cardinality object.\nIn this case no maps_to can be set.\n```\npaths:\n  test/histopathology/result_group/laboratory_test_result/any_event:*/test_name:\n    maps_to: []\n    suffixes:\n      \"\": \"Histopathology\"\n\n```\n\n\n\n#### paths.\\\u003cpath\\\u003e.maps_to\n---\nList of source keys that the given path maps to. It can be empty.\n```\npaths:\n  test/patient_data/gender/biological_sex:\n    maps_to: \n     - \"$..Dataelement_85_1.'#text'\"\n\n```\nFor XPath, you can use *ns* as placeholder for the namespace, as in *//ns:Dataelement_3_1/text()*\n\n#### paths.\\\u003cpath\\\u003e.suffixes\n---\nSuffixes to append to the given path.\n\n```\npaths:\n  test/patient_data/gender/biological_sex:\n    maps_to: \n     - \"$..Dataelement_85_1.'#text'\"\n    suffixes:\n      \"|value\" : \"a valid value\" \n```\n\n\n#### paths.\\\u003cpath\\\u003e.suffixes.\\\u003csuffix\\\u003e\n---\n\nA valid suffix to append to the given path, usually something like ```|code```. \nIf not suffix is expected, an empty string (\"\") can be used.\nThe value is a jinja template where some objects and methods are available,\nin particular  *maps_to* and *value_map*. \nSuffix can be also and object, see [value](#paths.\\\u003cpath\\\u003e.suffixes.\\\u003csuffix\\\u003e.value)\nand [jq](#paths.\\\u003cpath\\\u003e.suffixes.\\\u003csuffix\\\u003e.jq) .\n\n```\npaths:\n  ctx/territory:\n    maps_to:\n      - \"$..Location.@name\"\n    suffixes:\n      \"|code\": \"{{ value_map[maps_to[0]]  }}\"\n      \"|terminology\" : \"ISO_3166-1\"\n    value_map: # you can define a value_map that is available in the suffixes\n      test: IT\n      le test: FR\n\n```\n\n#### paths.\\\u003cpath\\\u003e.suffixes.\\\u003csuffix\\\u003e.value\n---\nThe value (jinja template, see [here](#paths.\\\u003cpath\\\u003e.suffixes.\\\u003csuffix\\\u003e))\nfor the given suffix.\n\n#### paths.\\\u003cpath\\\u003e.suffixes.\\\u003csuffix\\\u003e.jq\n---\nBoolean, if true the [suffix value](#paths.\\\u003cpath\\\u003e.suffixes.\\\u003csuffix\\\u003e.value) is \nused as parameter to *jq* (the input is the relative web template json for the given path).\nUseful for retrieving values from the template and use them for assigning the suffix.\n\n```\n\npaths:\n  test/patient_data/gender/biological_sex:\n    maps_to: #list of source key maps expressed as jsonpath\n     - \"$..Dataelement_85_1.'#text'\"\n    suffixes:\n      \"|code\":\n        value: '.inputs[0].list[] | select (.label == \"{{ maps_to[0] | upper }}\") | .value'\n        jq: true \n\n```\n\n#### paths.\\\u003cpath\\\u003e.value_map\n---\nCustom value mapping that can be used for the suffixes values \n(available in the jinja template rendering).\n```\npaths:\n  ctx/territory:\n    maps_to:\n      - \"$..Location.@name\"\n    suffixes:\n      \"|code\": \"{{ value_map[maps_to[0]]  }}\"\n      \"|terminology\" : \"ISO_3166-1\"\n    value_map:\n      test: IT\n      le test: FR\n```\n\n#### paths.\\\u003cpath\\\u003e.null_flavor\n---\nIf set and some *maps_to* key is missing, the given null flavor is used for the path.\n\n```\npaths:\n  test/patient_data/gender/biological_sex:\n    maps_to:\n     - \"$..Dataelement_85_1.'#text'\"\n    null_flavor:\n      value: unknown\n      code: 253\n      terminology: openehr\n\n```\n\n#### set_missing_required_to_default\n---\nBoolean, if set to true all the missing paths (after processing the source keys)\nthat are required are set to their default value (if specified in the template).\n\n#### ehr_id\n-----\nIt allows to specify the ehr id (external ref) for the composition.\n\n```\nehr_id:\n  maps_to: []\n  value : \"{{ random_ehr_id() }}\"\n```\n\n#### ehr_id.maps_to\n-------------------\n\nSee [here](#paths.\\\u003cpath\\\u003e.maps_to)\n\n#### ehr_id.value\n-------------------\nA jinja template, similiar to suffixes. A random_ehr_id function is available.\n\n### User defined functions\nIt is possible to use user defined functions inside the jinja templates. \nFunctions must be defined in module called ```flatehr_udf.py``` that has to be in the PYTHONPATH.\n\n\n## CLI\nMain command:\n\n```\n$ flatehr -h\nusage: flatehr [-h] [--version] {generate,inspect} ...\n\npositional arguments:\n  {generate,inspect}\n    inspect           Shows the template tree, with info about type, cardinality,\n                      requiredness and optionally aql path and expected inputs.\n\noptional arguments:\n  -h, --help          show this help message and exit\n  --version           show program's version number and exit\n\n```\n\n### Generating a Configuration File\n\nFor generating the configuration skeleton from a template, run:\n```\n$ flatehr generate skeleton -h\nusage: flatehr generate skeleton [-h] template_file\n\nGenerate a configuration skeleton for the given template.\n\npositional arguments:\n  template_file  the path to the web template (json)\n\noptional arguments:\n  -h, --help     show this help message and exit\n\n```\n\n### Generating a Composition\n\nFor generating a composition, use this subcommand:\n```\n$ flatehr generate from-file -h\nusage: flatehr generate from-file [-h] -t TEMPLATE_FILE -c CONF_FILE [-r RELATIVE_ROOT] [-s | --skip-ehr-id | --no-skip-ehr-id] input_file\n\nGenerates composition(s) from a file. xml and json sources supported.\nPrints on stdout an external ehr id (if flag --skip-ehr-id is not set) and the flat composition.\nIf --relative-root is set, as many compositions are generated as keys with the given value exists in the source.\n\npositional arguments:\n  input_file            source file\n\noptional arguments:\n  -h, --help            show this help message and exit\n  -t TEMPLATE_FILE, --template-file TEMPLATE_FILE\n                        web template path\n  -c CONF_FILE, --conf-file CONF_FILE\n                        yaml configuration path\n  -r RELATIVE_ROOT, --relative-root RELATIVE_ROOT\n                        id for the root(s) that maps 1:1 to composition\n                        (default: None)\n  -s, --skip-ehr-id, --no-skip-ehr-id\n                        if set, ehr_id is not printed\n                        (default: False)\n\n```\n\nFor an example, try:\n```bash\n$ flatehr generate from-file -t tests/resources/web_template.json -c tests/resources/xml_conf.yaml --skip-ehr-id tests/resources/source.xml\n```\n\n### Inspecting a template\n\nFor inspecting a template, run:\n```\n$ flatehr inspect -h\nusage: flatehr inspect [-h] [-a | --aql-path | --no-aql-path] [-i | --inputs | --no-inputs] template_file\n\nShows the template tree, with info about type, cardinality,\nrequiredness and optionally aql path and expected inputs.\n\npositional arguments:\n  template_file         path to the web template (json)\n\noptional arguments:\n  -h, --help            show this help message and exit\n  -a, --aql-path, --no-aql-path\n                        flag, if true shows the aql path for each node\n                        (default: False)\n  -i, --inputs, --no-inputs\n                        flag, if true shows the inputs for each node\n                        (default: False)\n\n```\n\n\n\n## Examples\nHere is an excerpt from *tests/resources/json_conf.yaml*. It maps the source *tests/resources/source.json* to a composition defined by the template *tests/resources/web_template.json* (see also *template.opt* and *template.zip* in the same directory) \nIt uses the jsonpath syntax adopted by [jsonpath-ng](https://github.com/h2non/jsonpath-ng) library.\n\n```yaml\npaths:\n  test/patient_data/gender/biological_sex: # destination path (completed by the suffixes below)\n    maps_to: #list of source key maps expressed as jsonpath\n     - \"$..Dataelement_85_1.'#text'\"\n    suffixes:\n      \"|value\" : \"{{ maps_to[0] | upper }}\" # suffixes are rendered as jinja templates, `maps_to` are available\n      \"|code\":\n        value: '.inputs[0].list[] | select (.label == \"{{ maps_to[0] | upper }}\") | .value'\n        jq: true # if jq is true, the rendered value is passed as argument jq. The json input is the relative web template\n      \"|terminology\" :\n        value: '.inputs[0].terminology'\n        jq: true\n    null_flavor: # if not all `maps_to` are available, null flavour can be used\n      value: unknown\n      code: 253\n      terminology: openehr\n\n  ctx/language: en #for static value it is as simple as that\n\n  ctx/territory:\n    maps_to:\n      - \"$..Location.@name\"\n    suffixes:\n      \"|code\": \"{{ value_map[maps_to[0]]  }}\"\n      \"|terminology\" : \"ISO_3166-1\"\n    value_map: # you can define a value_map that is available in the suffixes\n      test: IT\n      le test: FR\n\n  test/histopathology/result_group/laboratory_test_result/any_event: #non leaf path with multiple cardinality can be mapped, they are increased when mapping occurs\n    maps_to:\n      - \"$..Event[?(@.@eventtype == 'Histopathology')]\"\n\n\n```\n\nFor more exaustive examples, see *tests/test_cli.sh*.\n\n\n## Ingesting\nFrom parallel ingesting multiple sources to an openEHR istance, take a look at *scripts/ingest.sh*.\n\n# Acknowledgments\nThis work has been partially funded by the following sources:\n\n* The EOSC-Life European project (H2020, grant agreement N. 824087), within the EOSC-Life WP1 Demonstrator “Cloudification of BBMRI-ERIC CRC-Cohort and its Digital Pathology Imaging” (APPID 1228);\n* The “Piattaforma avanzata per Analisi massiva e Medicina digitale” project (grant by Sardegna Ricerche, ex art 9 L.R. 20/2015-year 2020);\n* The “Pathology in Automated Traceable Healthcare” (PATH) project (grant by the Italian Ministry of Education, University and Research. grant number PON04a2_0055);\n* The “Total Patient Management” (ToPMa) project (grant by the Sardinian Regional Authority, grant number RC_CRP_077);\n* The “Processing, Analysis, Exploration, and Sharing of Big and/or Complex Data” (XDATA) project (grant by the Sardinian Regional Authority);\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcrs4%2Fflatehr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcrs4%2Fflatehr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcrs4%2Fflatehr/lists"}