{"id":15687227,"url":"https://github.com/dermatologist/hephaestus","last_synced_at":"2025-09-09T01:36:54.847Z","repository":{"id":43175879,"uuid":"149873179","full_name":"dermatologist/hephaestus","owner":"dermatologist","description":" :stars: Hephaestus - ETL and ML tools for OHDSI - OMOP CDM","archived":false,"fork":false,"pushed_at":"2023-01-24T10:03:46.000Z","size":260,"stargazers_count":13,"open_issues_count":4,"forks_count":6,"subscribers_count":4,"default_branch":"develop","last_synced_at":"2025-04-20T00:38:22.279Z","etag":null,"topics":["datawarehouse","emr","etl","health-data","health-informatics"],"latest_commit_sha":null,"homepage":"http://nuchange.ca","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dermatologist.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.rst","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-09-22T12:04:44.000Z","updated_at":"2024-04-27T13:26:33.000Z","dependencies_parsed_at":"2023-02-13T20:01:57.882Z","dependency_job_id":null,"html_url":"https://github.com/dermatologist/hephaestus","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dermatologist%2Fhephaestus","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dermatologist%2Fhephaestus/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dermatologist%2Fhephaestus/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dermatologist%2Fhephaestus/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dermatologist","download_url":"https://codeload.github.com/dermatologist/hephaestus/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252931480,"owners_count":21827109,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["datawarehouse","emr","etl","health-data","health-informatics"],"created_at":"2024-10-03T17:45:32.894Z","updated_at":"2025-05-07T18:05:15.251Z","avatar_url":"https://github.com/dermatologist.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# :stars: Hephaestus - Data analytics tools for Digital Health!\n\n*Hephaestus was the god of fire, metalworking, stone masonry, forges and the art of sculpture.*\n\n\n# [Documentation](https://hephaestus.readthedocs.io/en/latest/)\n\n\n## About\n\nEfficient and effective health data warehousing and analysis require a common data model. \n\nThe [OHDSI - OMOP Common Data Model](https://www.ohdsi.org/) allows for the systematic analysis of disparate observational databases and EMRs. The data from disparate systems needs to be extracted, transformed and loaded on to a CDM database. Once a database has been converted to the OMOP CDM, evidence can be generated using standardized analytics tools.\n\nEach data source requires customized ETL tools for this conversion. :stars: Hephestus is a tool for this ETL process organized into modules to allow code reuse between various ETL tools for open-source EMR systems and data sources. :stars: Hephestus uses [*SqlAlchemy*](https://www.sqlalchemy.org/) for database connection and automapping tables to classes and [*bonobo*](https://www.bonobo-project.org/) for managing ETL. Hephaestus also aims to support common machine learning workflows such as model building with Apache spark. \n\n\n## Design principles\n\n* Support common functions such as creating OMOP table structure from the command line.\n* Use ORM (sqlalchemy) and ETL (bonobo) libraries to reduce boilerplate code and make code extensible and reusable.\n* Support ETL for common open-source EMRs such as OpenMRS and OSCAR EMR, and national level health databases such as Discharge Abstract Database (Canada) from the command line.\n* Create libraries to support common use cases such as vocabulary lookup and [Cui2Vec](http://cui2vec.dbmi.hms.harvard.edu/) based concept similarity search.\n* Support patient-level predictions.\n* Extend OMOP for public health use cases and support cohort-level predictions using MLlib (Spark's machine learning library).\n\n![Hephaestus](https://raw.github.com/dermatologist/hephaestus/develop/notes/hephaestus.png)\n\n## Features (Expected)\n\n### ETL tools for open source EMRs (OpenMRS and OSCAR EMR) and Discharge Anstract Database (Canada)\n* Work in Progress\n* Ref : https://github.com/maurya/openmrs-module-ohdsi\n\n### [Cui2Vec](http://cui2vec.dbmi.hms.harvard.edu/) based concept similarity search *hepahestus/vocabulary* folder\n* Work in progress\n* Create tables using instructions here: https://github.com/E-Health/OHDSIconceptid2cui\n* Cui.init_model() downloads and builds [Cui2Vec](https://arxiv.org/abs/1804.01486) model.\n* Find similar concepts, outliers and calculate distances.\n* Cui2Vec for noisy labeling and anchor learning.\n* See Also: https://github.com/OHDSI/Aphrodite\n\n### Spark ML based model building with tools for deploying models on serverless framework\n* Work in progress\n* See Also: https://github.com/OHDSI/PatientLevelPrediction\n\n## How to contribute and use:\n\n### Hephaestus is a work in progress. Please read CONTRIBUTING.md for more information on joining this project.\n\n\n## What it does\n\n### ETL\n* [x] Work in progress\n\n### ML\n* [x] Work in progress\n\n### Deployment artifacts\n* [x] Work in progress\n\n## How to install\n\n```text\n\nWork in progress\n\n\n```\n\n## How to Use\n\n* Use [OHDSIconceptid2cui](https://github.com/E-Health/OHDSIconceptid2cui) to create the mapping table ohdsi_to_cui in the vocabulary schema for all [cui2vec](https://arxiv.org/abs/1804.01486) functions of Hephaestus.\n* WIP\n\n\n### Command-line options\n\n```text\nwork in progress\n\n```\n\n| Command | Alternate | Description |\n| --- | --- | --- |\n| --inp | -i | Input file in the text format with \u003cbreak\u003e Topic \u003c/break\u003e |\n\n\n\n## Contributors and other projects\n\n* [Bell Eapen](https://nuchange.ca) (McMaster U) |  [Contact](https://nuchange.ca/contact)\n* This software is developed and tested using [Compute Canada](http://www.computecanada.ca) resources.\n* See also:  [:fire: The FHIRForm framework for managing healthcare eForms](https://github.com/E-Health/fhirform)\n* See also: [:eyes: Drishti | An mHealth sense-plan-act framework!](https://github.com/E-Health/drishti)\n* See also [:flashlight: Qualitative Research support tools in Python](https://github.com/dermatologist/nlp-qrmine)\n* See also [:hospital: Public Health Data Warehouse using FHIR and Kibana](https://github.com/E-Health/phis-dw)\n\n\n\n## Citation\n\n```\n\n@misc{eapenbr2018Hephaestus,\n  title={Hephaestus - Data warehouse and ETL tools for open source EMRs.},\n  author={Eapen, Bell Raj and contributors},\n  year={2018},\n  publisher={GitHub},\n  journal = {GitHub repository},\n  howpublished={\\url{https://github.com/dermatologist/hephaestus}}\n}\n\n```\n\n### Disclaimer\n\n*Parts of this material are based on the Canadian Institute for Health Information Discharge Abstract Database Research Analytic Files (sampled from fiscal years 2014-15). However the analysis, conclusions, opinions and statements expressed herein are those of the author(s) and not those of the Canadian Institute for Health Information.*\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdermatologist%2Fhephaestus","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdermatologist%2Fhephaestus","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdermatologist%2Fhephaestus/lists"}