{"id":21660295,"url":"https://github.com/slub/ldjesmaterializer","last_synced_at":"2026-05-06T22:35:55.705Z","repository":{"id":41279303,"uuid":"176968010","full_name":"slub/ldjesmaterializer","owner":"slub","description":"a commandline command (Python3 program) that takes line-delimited JSON records and some arguments as input and materializes information from other records (stored in an elasticsearch index) into them","archived":false,"fork":false,"pushed_at":"2019-05-07T13:25:05.000Z","size":15,"stargazers_count":1,"open_issues_count":0,"forks_count":2,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-01-25T07:09:04.479Z","etag":null,"topics":["command-line-tool","elasticsearch","json","line-delimited-json","materialized","python"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/slub.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-03-21T14:59:05.000Z","updated_at":"2019-05-07T13:25:06.000Z","dependencies_parsed_at":"2022-09-21T01:10:50.559Z","dependency_job_id":null,"html_url":"https://github.com/slub/ldjesmaterializer","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slub%2Fldjesmaterializer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slub%2Fldjesmaterializer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slub%2Fldjesmaterializer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slub%2Fldjesmaterializer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/slub","download_url":"https://codeload.github.com/slub/ldjesmaterializer/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244555979,"owners_count":20471543,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["command-line-tool","elasticsearch","json","line-delimited-json","materialized","python"],"created_at":"2024-11-25T09:32:42.474Z","updated_at":"2026-05-06T22:35:55.678Z","avatar_url":"https://github.com/slub.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ldjesmaterializer - line-delimited JSON elasticsearch materializer\n\nldjesmaterializer is a commandline command (Python3 program) that takes line-delimited JSON records and some arguments as input and materializes information from other records (stored in an elasticsearch index) into them.\n\n## Usage\n\n```\nldjesmaterializer\n\nrequired arguments:\n  -elasticsearch-index-name ELASTICSEARCH_INDEX_NAME     the name of the elasticsearch index to use (default: None)\n  -elasticsearch-index-type ELASTICSEARCH_INDEX_TYPE     elasticsearch index (document) type to use (default: None)\n  -source-field SOURCE_FIELD                             the field name in the (source) record whose value should be used to request the records for materialization (default: None)\n  -seed-field SEED_FIELD                                 the field name in the records in the elasticsearch index that should be used to request the records with the seed value for materialization (default: None)\n  -materialization-field MATERIALIZATION_FIELD           the field name in the records in the elasticsearch index that contains the information for materialization (default: None)\n  -target-field TARGET_FIELD                             the field name in the (target) records that should be used to write the information retrieved from the materialization request to the elasticsearch index (default: None)\n\noptional arguments:\n  -h, --help                                                   show this help message and exit\n  -elasticsearch-instance-host ELASTICSEARCH_INSTANCE_HOST     hostname or IP address of the elasticsearch instance to use (default: localhost)\n  -elasticsearch-instance-port ELASTICSEARCH_INSTANCE_PORT     port of the elasticsearch instance to use (default: 9200)\n  -target-field-is-multivalued                                 indicates, whether the target field should be a JSON array (for multiple values) or a JSON object / simple value (for single values); default is false, i.e., the retrieved values will be written into a JSON array (default: False)\n  -materialization-value-count-only                            indicates, whether the only the materialization value count should be written into the target field (default: False)\n```\n\n* example:\n    ```\n    ldjesmaterializer -elasticsearch-instance-host [HOSTNAME OR IP OF YOUR ELASTICSEARCH INSTANCE] -elasticsearch-instance-port [PORT OF YOUR ELASTICSEARCH INSTANCE] -elasticsearch-index-name [YOUR ELASTICSEARCH INDEX NAME] -elasticsearch-index-type [DOCUMENT TYPE OF THE ELEASTICSEARCH INDEX] -source-field [SOURCE FIELD NAME] -seed-field [SEED FIELD NAME] -materialization-field [MATERIALIZATION FIELD NAME] -target-field [TARGET FIELD NAME]\n    ```\n\n## Requirements\n\n[elasticsearch-py](http://elasticsearch-py.rtfd.org/)\n\n### Install requirements\n\n#### Install requirements with pip\n\n1. (optionally) install [pip](https://pip.pypa.io/) for Python 3.x:\n\n    sudo apt-get install python3-pip\n\n2. install requirements with pip:\n\n    sudo -H pip3 install -r requirements.txt\n    \n#### Install requirements with OS package manager\n\ne.g.\n```\napt-get install python-elasticsearch\n\n```\n\n## Run\n\n* install elasticsearch-py\n* clone this git repo or just download the [ldjesmaterializer.py](ldjesmaterializer/ldjesmaterializer.py) file\n* run ./ldjesmaterializer.py\n* for a hackish way to use ldjesmaterializer system-wide, copy to /usr/local/bin\n\n### Install system-wide via pip\n\n* via pip:\n    ```\n    sudo -H pip3 install --upgrade [ABSOLUTE PATH TO YOUR LOCAL GIT REPOSITORY OF LDJESMATERIALIZER]\n    ```\n    (which provides you ```ldjesmaterializer``` as a system-wide commandline command)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fslub%2Fldjesmaterializer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fslub%2Fldjesmaterializer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fslub%2Fldjesmaterializer/lists"}