https://github.com/dhis2/dhis2-helix-etl

Data pipeline for Helix to DHIS 2 integration
https://github.com/dhis2/dhis2-helix-etl

Last synced: 10 months ago
JSON representation

Data pipeline for Helix to DHIS 2 integration

• Host: GitHub
• URL: https://github.com/dhis2/dhis2-helix-etl
• Owner: dhis2
• Created: 2020-01-15T13:50:31.000Z (over 6 years ago)
• Default Branch: master
• Last Pushed: 2021-01-18T08:20:44.000Z (over 5 years ago)
• Last Synced: 2025-07-30T03:59:36.784Z (11 months ago)
• Language: JavaScript
• Homepage:
• Size: 34.2 KB
• Stars: 2
• Watchers: 9
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# DHIS 2 Helix DXP Data Pipeline

## Requirements

The data pipeline is written in Node.js and requires a Node.js runtime.

## Configuration

First set up `config.json` with the required information, then run the script with Node:

    node index.js

This script will load data from Helix, transform it based on the profile mapping and push data to the DHIS2 instance defined in the configuration.

The `config.json` configuration file is in JSON format and looks like this:

```json

{

  "dhis2": {

    "url": "http://localhost/dhis",

    "username": "admin",

    "password": "district"

  },

  "profiles": [

    {

      "name": "DHIS2 Sample data",

      "url": "https://unicef.sdmxcloud.org/ws/public/sdmxapi/rest/data/UNICEF_TEST",

      "transformation": [

        {

          "dataElementId": "UNICEF_INDICATOR",

          "period": "TIME_PERIOD",

          "organisationUnit": "COUNTRY",

          "dataDimensions": [

            {

              "categoryOptionCombo": "OBS_VALUE",

              "value": "VALUE"

            },

            {

              "categoryOptionCombo": "LOWER_BOUND",

              "value": "LOWER_BOUND"

            },

            {

              "categoryOptionCombo": "UPPER_BOUND",

              "value": "UPPER_BOUND"

            }

          ]

        }

      ]

    }

  ]

}

```

The `dhis2` section contains information about the instance where data should be pushed. This includes the URL, username and password required to connect.

The `profiles` section represents each set of data to import into DHIS 2. Each profile refers to a helix endpoint and a `transformation` for the data.

The `transformation` consists of the key (`dataElementId`, `period`, `organisationUnit`) which relates directly to entities in DHIS 2, and a value. The value will be executed as javascript with variables based on the Helix payload. This means in our current example, we pick the certain properties that represents the diffent references to the entities in DHIS 2. We expect `dataElementId`, `organisationUnit` and `categoryOptionCombo` all refer to valid codes in DHIS 2. This is a property that can be configured for each entity in DHIS 2.

Each data dimension refer to each of the three values which are required (observed value, lower bound and upper bound) which are disaggregated into different category option combos. This means that each row of data from Helix will result in three DHIS 2 datavalues.

Additionally, the period must be in the correct format. If we are working solely with years, four digits is sufficient.