{"id":18365324,"url":"https://github.com/mit-lcp/mimic-omop","last_synced_at":"2026-03-04T05:01:59.926Z","repository":{"id":28356963,"uuid":"102531192","full_name":"MIT-LCP/mimic-omop","owner":"MIT-LCP","description":"Mapping the MIMIC-III database to the OMOP schema","archived":false,"fork":false,"pushed_at":"2024-03-17T17:48:41.000Z","size":25868,"stargazers_count":132,"open_issues_count":46,"forks_count":49,"subscribers_count":29,"default_branch":"master","last_synced_at":"2025-04-10T17:00:04.649Z","etag":null,"topics":["data-model","mimic-iii","omop"],"latest_commit_sha":null,"homepage":null,"language":"PLpgSQL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MIT-LCP.png","metadata":{"files":{"readme":"README-run-etl.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-09-05T21:24:34.000Z","updated_at":"2025-03-04T08:06:46.000Z","dependencies_parsed_at":"2024-12-24T00:37:39.405Z","dependency_job_id":null,"html_url":"https://github.com/MIT-LCP/mimic-omop","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/MIT-LCP/mimic-omop","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MIT-LCP%2Fmimic-omop","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MIT-LCP%2Fmimic-omop/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MIT-LCP%2Fmimic-omop/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MIT-LCP%2Fmimic-omop/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MIT-LCP","download_url":"https://codeload.github.com/MIT-LCP/mimic-omop/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MIT-LCP%2Fmimic-omop/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30071895,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-04T03:25:38.285Z","status":"ssl_error","status_checked_at":"2026-03-04T03:25:05.086Z","response_time":59,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-model","mimic-iii","omop"],"created_at":"2024-11-05T23:13:10.843Z","updated_at":"2026-03-04T05:01:59.903Z","avatar_url":"https://github.com/MIT-LCP.png","language":"PLpgSQL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Running the ETL on PostgreSQL\n\nThis README overviews running the MIMIC-OMOP ETL from the ground up on a PostgreSQL server. You will need an installation of PostgreSQL 9.6+ in order to run the ETL. You will also need MIMIC-III installed on this instance of postgres, see here for details: https://mimic.physionet.org/gettingstarted/dbsetup/\n\nThis README will assume the following:\n\n* MIMIC-III v1.4 is available in the `mimic` database under the `mimiciii` schema\n* The standard concepts from Athena have been downloaded and are available somewhere (including running the extra script to download CPT code definitions)\n* The R software library with remotes (`install.packages(\"remotes\")`) and a GitHub package `remotes::install_github(\"r-dbi/RPostgres\")`\n* The computer has an active internet connection (needed to clone certain repositories throughout the build)\n\n## 0. Open up a terminal and define parameters\n\nTo simplify the reusability of these scripts, we define a few environment variables and reuse them throughout the rest of the guide.\n\nFirst is the connection string used to connect to the database. Note this also specifies the schema using `search_path`.\n\n```bash\nexport OMOP_SCHEMA='omop'\nexport OMOP='host=localhost dbname=postgres user=postgres options=--search_path='$OMOP_SCHEMA\nexport MIMIC='host=localhost dbname=postgres user=postgres options=--search_path=mimiciii'\n# or, e.g., export OMOP=\"dbname=mimic options=--search_path=omop\"\n```\n\nWe will later use these environmental variables to connect to postgres.\n\n**NOTE**: While ideally specifying the schemas would be completely configurable, much of the repository assumes the `omop` schema. The `mimiciii` schema should be configurable, but this is experimental.\n\n## 1. Build OMOP tables with standard concepts\n\nSee [the omop/build-omop/postgresql/README.md file](omop/build-omop/postgresql/README.md) to build OMOP on postgres.\n\n## 2. Create local MIMIC-III concepts\n\nWe need to create a `concept_id` for each MIMIC-III local code. OMOP reserves `concept_id` above 20,000,000+ for local codes, so we will use this range to insert ours.\n\n```sh\npsql \"$MIMIC\" -f mimic/build-mimic/postgres_create_mimic_id.sql\n```\n\nN.B. this script is called by `etl_sequence.sql`\n\nAfter this, every table in the MIMIC-III schema will have an additional column called `mimic_id`.\n\n## 3. Load the concepts from the CSV files\n\nFirst prepare a configuration file for the R script and save it as `mimic-omop.cfg` in the root folder of this repository. Here is an example of the file structure:\n\n```sh\ndbname=mimic\nuser=alistairewj\n```\n\nAfter that, run the R script from the root folder:\n\n```\nRscript etl/ConceptTables/loadTables.R mimiciii\n```\n\nThis will load various manual mappings to the database under the `mimiciii` schema.\n\n## 4. Run the ETL\n\nBe sure to run this from the *root* folder of the repository, or the relative path names will cause errors.\n\n```sh\npsql \"$MIMIC\" --set=OMOP_SCHEMA=\"$OMOP_SCHEMA\" -f \"etl/etl.sql\"\n```\n\n## 5. Check the ETL has run properly\n\nIn order to run the checks, you'll need [pgTap](http://pgtap.org/). pgTap is a testing framework for postgres.\nYou can install pgtap by either:\n\n* using a package manager, e.g. on Ubuntu using: `sudo apt-get install pgtap`.\n* from source, following the [install instructions here](https://pgxn.org/dist/pgtap/)\n\nIf building from source, pay careful attention to the make output. You may need to install additional perl modules in order to use functions such as pg_prove, using `cpan TAP::Parser::SourceHandler::pgTAP`.\nYou may also need to run the installation `make` files as the postgres user, who has superuser privileges to the postgres database.\n\nAfter you install it, be sure to enable the extension as follows:\n\n```sh\npsql \"$MIMIC\" -c \"CREATE EXTENSION pgtap;\"\n```\n\nNow the extension is available database-wide, and we can run the ETL.\n\n```sh\npsql \"$MIMIC\" -f \"etl/check_etl.sql\"\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmit-lcp%2Fmimic-omop","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmit-lcp%2Fmimic-omop","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmit-lcp%2Fmimic-omop/lists"}