{"id":14958184,"url":"https://github.com/dermatologist/fhiry","last_synced_at":"2025-04-06T23:17:41.390Z","repository":{"id":37788430,"uuid":"320945773","full_name":"dermatologist/fhiry","owner":"dermatologist","description":"FHIR to pandas dataframe for data analytics, AI and ML!","archived":false,"fork":false,"pushed_at":"2025-03-06T03:49:28.000Z","size":1080,"stargazers_count":34,"open_issues_count":20,"forks_count":7,"subscribers_count":4,"default_branch":"develop","last_synced_at":"2025-03-30T21:12:16.762Z","etag":null,"topics":["analytics","data-analysis","data-science","ehealth","fhir","hacktoberfest","jupyter-notebook","pandas","python","pytorch","tensorflow"],"latest_commit_sha":null,"homepage":"https://nuchange.ca","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dermatologist.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS.md","dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-12-12T23:34:10.000Z","updated_at":"2025-02-13T11:50:33.000Z","dependencies_parsed_at":"2023-11-12T00:21:38.296Z","dependency_job_id":"7906932d-e935-4b63-85cb-a3d98f1e24c5","html_url":"https://github.com/dermatologist/fhiry","commit_stats":{"total_commits":221,"total_committers":8,"mean_commits":27.625,"dds":0.6153846153846154,"last_synced_commit":"6c222078bc92f54072539e102852f71bc3dfcf99"},"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dermatologist%2Ffhiry","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dermatologist%2Ffhiry/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dermatologist%2Ffhiry/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dermatologist%2Ffhiry/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dermatologist","download_url":"https://codeload.github.com/dermatologist/fhiry/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247563941,"owners_count":20958971,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["analytics","data-analysis","data-science","ehealth","fhir","hacktoberfest","jupyter-notebook","pandas","python","pytorch","tensorflow"],"created_at":"2024-09-24T13:16:26.114Z","updated_at":"2025-04-06T23:17:41.352Z","avatar_url":"https://github.com/dermatologist.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# :fire: fhiry - FHIR to pandas dataframe for data analytics, AI and ML\nVirtual flattened view of *FHIR Bundle / ndjson / FHIR server / BigQuery!*\n\n![Libraries.io SourceRank](https://img.shields.io/librariesio/sourcerank/pypi/fhiry)\n[![PyPI download total](https://img.shields.io/pypi/dm/fhiry.svg)](https://pypi.python.org/pypi/fhiry/)\n![GitHub tag (latest by date)](https://img.shields.io/github/v/tag/dermatologist/fhiry)\n\n:fire: **FHIRy** is a [python](https://www.python.org/) package to facilitate health data analytics and machine learning by converting a folder of [FHIR bundles](https://www.hl7.org/fhir/bundle.html)/ndjson from [bulk data export](https://hl7.org/fhir/uv/bulkdata/export/index.html) into a [pandas](https://pandas.pydata.org/docs/user_guide/index.html) data frame for analysis. You can import the dataframe\ninto ML packages such as Tensorflow and PyTorch. **FHIRy also supports FHIR server search and FHIR tables on BigQuery.**\n\n\nTest this with the [synthea sample](https://synthea.mitre.org/downloads) or the downloaded ndjson from the [SMART Bulk data server](https://bulk-data.smarthealthit.org/). Use the 'Discussions' tab above for feature requests.\n\n:sparkles: Checkout [this template](https://github.com/dermatologist/kedro-multimodal) for Multimodal machine learning in healthcare!\n\n\n## UPDATE 1\nRecently added support for **LLM based natural language queries** of FHIR bundles/ndjson using [llama-index](examples/llm_example.py). Please install the llm extras as follows. Please be cognizant of the privacy issues with publically hosted LLMs. Any feedback will be highly appreciated. [See usage](examples/llm_example.py)!\n\n```\npip install fhiry[llm]\n```\n[See usage](examples/llm_example.py).\n\n## UPDATE 2\nAdded support for converting a FHIR Bundle to its textual representation for LLMs. You can also convert individual FHIR resources including *Patient, Condition, Observation, Procedure, Medication, AllergyIntolerance and DocumentReference*.\n\n```\nfrom fhiry import FlattenFhir\nbundle = json.load(jsonfile)\nflatten_fhir = FlattenFhir(bundle)\nprint(flatten_fhir.flattened)\n```\n\n## Installation\n\n### Stable\n```shell\npip install fhiry\n```\n\n### Latest dev version\n\n```\npip install git+https://github.com/dermatologist/fhiry.git\n```\n## Usage\n\n### 1. Import FHIR bundles (JSON) from folder to pandas dataframe\n\n```python\nimport fhiry.parallel as fp\ndf = fp.process('/path/to/fhir/resources')\nprint(df.info())\n```\n\nExample source data set: [Synthea](https://synthea.mitre.org/downloads)\n\nJupyter notebook example: [`notebooks/synthea.ipynb`](notebooks/synthea.ipynb)\n\n### 2. Import NDJSON from folder to pandas dataframe\n\n```python\nimport fhiry.parallel as fp\ndf = fp.ndjson('/path/to/fhir/ndjson/files')\nprint(df.info())\n```\n\nExample source data set: [SMART Bulk Data Server](https://bulk-data.smarthealthit.org/) Export\n\nJupyter notebook example: [`notebooks/ndjson.ipynb`](notebooks/ndjson.ipynb)\n\n### 3. Import FHIR Search results to pandas dataframe\n\nFetch and import resources from [FHIR Search API](https://www.hl7.org/fhir/search.html) results to pandas dataframe.\n\nDocumentation: [`fhir-search.md`](fhir-search.md)\n\n#### Example: Import all conditions with a certain code from FHIR Server\n\nFetch and import all condition resources with Snomed (Codesystem `http://snomed.info/sct`) Code `39065001` in the FHIR element `Condition.code` ([resource type specific FHIR search parameter `code`](https://www.hl7.org/fhir/condition.html#search)) to a pandas dataframe:\n\n```python\nfrom fhiry.fhirsearch import Fhirsearch\n\nfs = Fhirsearch(fhir_base_url = \"http://fhir-server:8080/fhir\")\n\nmy_fhir_search_parameters = {\n    \"code\": \"http://snomed.info/sct|39065001\",\n}\n\ndf = fs.search(resource_type = \"Condition\", search_parameters = my_fhir_search_parameters)\n\nprint(df.info())\n```\n\n### 4. Import [Google BigQuery](https://cloud.google.com/bigquery) [FHIR dataset](https://console.cloud.google.com/marketplace/details/mitre/synthea-fhir?q=synthea)\n\n```python\nfrom fhiry.bqsearch import BQsearch\nbqs = BQsearch()\n\ndf = bqs.search(\"SELECT * FROM `bigquery-public-data.fhir_synthea.patient` LIMIT 20\") # can be a path to .sql file\n\n```\n\n## Filters\n\nPass a config json to any of the constructors:\n* config_json can be a path to a json file.\n```\ndf = fp.process('/path/to/fhir/resources', config_json='{ \"REMOVE\": [\"resource.text.div\"], \"RENAME\": { \"resource.id\": \"id\" }  }')\n\nfs = Fhirsearch(fhir_base_url = \"http://fhir-server:8080/fhir\", config_json = '{ \"REMOVE\": [\"resource.text.div\"], \"RENAME\": { \"resource.id\": \"id\" }  }')\n\nbqs = BQsearch('{ \"REMOVE\": [\"resource.text.div\"], \"RENAME\": { \"resource.id\": \"id\" }  }')\n```\n\n## Columns\n* see df.columns\n\n```\npatientId\nfullUrl\nresource.resourceType\nresource.id\nresource.name\nresource.telecom\nresource.gender\n...\n...\n...\n```\n\n\n### [Documentation](https://dermatologist.github.io/fhiry/)\n\n## Give us a star ⭐️\nIf you find this project useful, give us a star. It helps others discover the project.\n\n## Contributors\n\n* [Bell Eapen](https://nuchange.ca) | [![Twitter Follow](https://img.shields.io/twitter/follow/beapen?style=social)](https://twitter.com/beapen)\n* [Markus Mandalka](https://github.com/Mandalka)\n* PR welcome, please see [CONTRIBUTING.md](/CONTRIBUTING.md)\n* [![forthebadge](https://forthebadge.com/images/badges/built-with-love.svg) using CC](https://alliancecan.ca/en/services/advanced-research-computing)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdermatologist%2Ffhiry","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdermatologist%2Ffhiry","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdermatologist%2Ffhiry/lists"}