{"id":28395103,"url":"https://github.com/databiosphere/terra-drs-hub","last_synced_at":"2025-06-27T01:31:20.220Z","repository":{"id":38334436,"uuid":"458309862","full_name":"DataBiosphere/terra-drs-hub","owner":"DataBiosphere","description":null,"archived":false,"fork":false,"pushed_at":"2025-06-17T13:08:51.000Z","size":622,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":21,"default_branch":"dev","last_synced_at":"2025-06-17T14:23:00.547Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DataBiosphere.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-02-11T19:09:12.000Z","updated_at":"2025-06-17T13:08:54.000Z","dependencies_parsed_at":"2023-12-21T20:16:11.607Z","dependency_job_id":"7602bf1a-fa36-448a-be4a-c8b1acc34c0f","html_url":"https://github.com/DataBiosphere/terra-drs-hub","commit_stats":null,"previous_names":[],"tags_count":136,"template":false,"template_full_name":null,"purl":"pkg:github/DataBiosphere/terra-drs-hub","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataBiosphere%2Fterra-drs-hub","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataBiosphere%2Fterra-drs-hub/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataBiosphere%2Fterra-drs-hub/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataBiosphere%2Fterra-drs-hub/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DataBiosphere","download_url":"https://codeload.github.com/DataBiosphere/terra-drs-hub/tar.gz/refs/heads/dev","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataBiosphere%2Fterra-drs-hub/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262172375,"owners_count":23269995,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-05-31T19:39:01.986Z","updated_at":"2025-06-27T01:31:20.206Z","avatar_url":"https://github.com/DataBiosphere.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# DrsHub (Also known as Dr. Martha Shub, MD)\n## Overview\nDrsHub is the [DRS](https://ga4gh.github.io/data-repository-service-schemas/preview/develop/docs/) resolution service for Terra. It is the hub through which DRS requests are routed, therefore: DRSHub.\n\nIt is a Java Spring Boot rewrite of the deprecated Cloud Function [Martha](https://github.com/broadinstitute/martha), specifically, its v3 API.\n\n### Background Info\n\n- [The adoption of DRS across data repositories](https://docs.google.com/document/d/1Wf4enSGOEXD5_AE-uzLoYqjIp5MnePbZ6kYTVFp1WoM/edit#heading=h.qiwlmit3m9)\n- [Global Alliance for Genomics and Health (GA4GH) DRS specifications](https://ga4gh.github.io/data-repository-service-schemas/preview/develop/docs/)\n\n## DRS Providers\n\nThis is the short name, full name, and auth type(s) for each provider\n\n- **AnVIL** (NHGRI Analysis Visualization and Informatics Lab-space)\n  - Ecm Provider Access Token\n- **BDC** (BioData Catalyst)\n  - Ecm Provider Access Token\n- **CRDC** (NCI Cancer Research/Proteomics Data Commons)\n  - Ecm Provider Access Token\n- **KidsFirst** (Gabriella Miller Kids First DRC)\n  - Ecm Provider Access Token\n- **Passport Test** (Passport Test Provider)\n  - Passport\n  - Ecm Provider Access Token\n- **TDR** (Terra Data Repo)\n  - Bearer Token\n- **Sage Bionetworks** (Synapse)\n  - Ecm Provider Access Token\n\n## Usage\nTo resolve a DRS URL, perform an HTTP `POST` to `/api/v4/drs/resolve`.\nThe content-type of your request should be `application/json` with the content/body of your request encoded accordingly.\n\nRequest bodies should look like\n```json\n{\n  \"url\": \"string\",\n  \"cloudPlatform\": [\"gs\", \"azure\",\"s3\", null],\n  \"fields\": [\"string\"]\n}\n```\nwhere `url` is the DRS URL to resolve and `fields` is any of\n```text\naccessUrl\nbondProvider\nbucket\ncontentType\nfileName\ngoogleServiceAccount\ngsUri\nhashes\nlocalizationPath\nname\nsize\ntimeCreated\ntimeUpdated\n```\n\nIf no `fields` are specified in the request, the response will include the following `fields` by default:\n```text\nbucket\ncontentType\nfileName\ngsUri\nhashes\nlocalizationPath\nname\nsize\ntimeCreated\ntimeUpdated\ngoogleServiceAccount\n```\n\nThe `cloudPlatform` field is optional and can be used to specify the preferred cloud platform to use for returning a signed URL.\nIf no option is found for the specified cloud platform, an attempt will be made to return a signed URL from a fall-back cloud platform.\n\n## Architecture\nDrsHub is a Java 17 Spring Boot application running in Kubernetes. As it simply resolves urls and doesn't have any state, it has no database. For developer convenience, a Swagger UI is provided.\n\nSome architecture diagrams can be found in [LucidChart](https://lucid.app/documents#/documents?folder_id=297026717)\n\n## Development\n\n### Setup\nInstall Java 17 SDK from your preferred provider. A common way to install and manage different JDK versions is to use [sdkman](https://sdkman.io/).\n\nIf developing in IntelliJ, you can configure the Project SDK to use Java 17.\nYou'll also need to set the Gradle JVM, located at `Preferences | Build, Execution, Deployment | Build Tools | Gradle`.\n\nYou must use [git-secrets](https://github.com/awslabs/git-secrets) to protect against committing passwords\nor other sensitive information ot this repository.  The linked repository gives instructions for\ninstalling and setting it up.\n\nDrsHub uses [Minnie Kenny](https://minnie-kenny.readthedocs.io/en/latest/), and is configured to run `minnie_kenny.sh` on `./gradlew test` tasks, ensuring that git-secrets is set up.\nYou can also run it manually to make sure `git-secrets` is set up without testing.\n\nBefore running anything, make sure to run `./render_config \u003cENV\u003e` to render secrets locally. The default `\u003cENV\u003e` is `dev`, which is what you should use for local running and testing.\n\nDrsHub uses Gradle as a build tool. Some common Gradle commands you may want to run are\n```shell\n./gradlew generateSwaggerCode # Generate Swagger code for models and Swagger UI\n./gradlew bootRun # Run DrsHub locally (Swagger UI at localhost:8080)\n./gradlew test # Run the unit tests\n./gradlew jib # Build the DrsHub Docker image\n```\n\n### Run Integration or Performance Tests\nDrsHub uses [TestRunner](https://github.com/DataBiosphere/terra-test-runner) to run its integration\nand performance tests.\n\nTo run the integration test suite, run\n```shell\n./gradlew runTest --args=\"suites/FullIntegration.json /tmp/test-results\"\n```\n\nTo run the performance test suite, run\n```shell\n./gradlew runTest --args=\"suites/FullPerf.json /tmp/test-results\"\n```\n\nAdding `--stacktrace` can give you more debugging information, if needed.\n\n### Run Pact Tests\nTo run the Pact tests, run the following:\n\n```shell\nexport PACT_BROKER_URL=\"pact-broker.dsp-eng-tools.broadinstitute.org\"\nexport PACT_PROVIDER_COMMIT=\"$(git rev-parse HEAD)\"\nexport PACT_PROVIDER_BRANCH=\"$(git rev-parse --abbrev-ref HEAD)\"\nexport PACT_BROKER_USERNAME=\"$(gcloud --project broad-dsp-eng-tools secrets versions access latest --secret pact-broker-users-read-write | jq -r .basic_auth_username)\"\nexport PACT_BROKER_PASSWORD=\"$(gcloud --project broad-dsp-eng-tools secrets versions access latest --secret pact-broker-users-read-write | jq -r .basic_auth_password)\"\n\n./gradlew verifyPacts\n```\n\n### Logging\nBy default, DrsHub will emit logs in the Stackdriver JSON format.\nTo disable this behavior for local development, add `DRSHUB_LOG_APPENDER=Console-Standard` to your environment when running DrsHub.\n\n## Deployment\nDrsHub runs in Kubernetes in GCP. Current deployments for each env can be found at:\n- Dev\n  - [Kubernetes Deployment](https://console.cloud.google.com/kubernetes/deployment/us-central1-a/terra-dev/terra-dev/drshub-deployment/overview?project=broad-dsde-dev)\n  - [Swagger UI](https://drshub.dsde-dev.broadinstitute.org/)\n- Alpha\n  - [Kubernetes Deployment](https://console.cloud.google.com/kubernetes/deployment/us-central1-a/terra-alpha/terra-alpha/drshub-deployment/overview?project=broad-dsde-alpha)\n  - [Swagger UI](https://drshub.dsde-alpha.broadinstitute.org/)\n- Staging\n  - [Kubernetes Deployment](https://console.cloud.google.com/kubernetes/deployment/us-central1-a/terra-staging/terra-staging/drshub-deployment/overview?project=broad-dsde-staging)\n  - [Swagger UI](https://drshub.dsde-staging.broadinstitute.org/)\n- Production\n  - [Kubernetes Deployment](https://console.cloud.google.com/kubernetes/deployment/us-central1-a/terra-prod/terra-prod/drshub-deployment/overview?project=broad-dsde-prod)\n  - [Swagger UI](https://drshub.dsde-prod.broadinstitute.org/)\n\n### DRS Provider Compact ID/URIs per Environment\n\nNote: there are a few tricky cases with the **compact IDs (CID)**:\n- BioDataCatalyst uses the CIB `dg.4503` in production, and `dg.712c` in non-prod environments\n- The AnVIL currently has two CIBs in use\n  - `dg.anv0` for old gen3 and TDR hosted data\n  - `drs.anv0` (note the **dg** vs **drs** prefix) for TDR hosted data, this will be used going forward\n- Most of these providers have only `prod` and `not prod` URIs, TDR is the only one that has specific URIs for each lower environment\n\n### Dev\n| Provider            | Compact Id (CIB)  | Host URI                                   |\n|---------------------|-------------------|--------------------------------------------|\n| AnVIL (TDR hosted)  | dg.anv0           | jade.datarepo-dev.broadinstitute.org       |\n| AnVIL (TDR hosted)  | drs.anv0          | jade.datarepo-dev.broadinstitute.org       |\n| BDC                 | dg.712c           | staging.gen3.biodatacatalyst.nhlbi.nih.gov |\n| CRDC                | dg.4dfc           | nci-crdc-staging.datacommons.io            |\n| KidsFirst           | dg.f82a1a         | gen3staging.kidsfirstdrc.org               |\n| Passport Test       | dg.test0          | ctds-test-env.planx-pla.net                |\n\n### Alpha\n| Provider            | Compact Id (CIB)  | Host URI                                   |\n|---------------------|-------------------|--------------------------------------------|\n| AnVIL (TDR hosted)  | dg.anv0           | data.alpha.envs-terra.bio                  |\n| AnVIL (TDR hosted)  | drs.anv0          | data.alpha.envs-terra.bio                  |\n| BDC                 | dg.712c           | staging.gen3.biodatacatalyst.nhlbi.nih.gov |\n| CRDC                | dg.4dfc           | nci-crdc-staging.datacommons.io            |\n| KidsFirst           | dg.f82a1a         | gen3staging.kidsfirstdrc.org               |\n| Passport Test       | dg.test0          | ctds-test-env.planx-pla.net                |\n\n### Staging\n\n| Provider            | Compact Id (CIB)  | Host URI                                   |\n|---------------------|-------------------|--------------------------------------------|\n| AnVIL (TDR hosted)  | dg.anv0           | data.staging.envs-terra.bio                |\n| AnVIL (TDR hosted)  | drs.anv0          | data.staging.envs-terra.bio                |\n| BDC                 | dg.712c           | staging.gen3.biodatacatalyst.nhlbi.nih.gov |\n| CRDC                | dg.4dfc           | nci-crdc-staging.datacommons.io            |\n| KidsFirst           | dg.f82a1a         | gen3staging.kidsfirstdrc.org               |\n| Passport Test       | dg.test0          | ctds-test-env.planx-pla.net                |\n\n### Prod\n\n| Provider             | Compact Id (CIB)  | Host URI                           |\n|----------------------|-------------------|------------------------------------|\n| AnVIL (TDR hosted)   | dg.anv0           | data.terra.bio                     |\n| AnVIL (TDR hosted)   | drs.anv0          | data.terra.bio                     |\n| BDC                  | dg.4503           | gen3.biodatacatalyst.nhlbi.nih.gov |\n| CRDC                 | dg.4dfc           | nci-crdc.datacommons.io            |\n| KidsFirst            | dg.f82a1a         | data.kidsfirstdrc.org              |\n| Passport Test        |                   |                                    |\n\n## SonarCloud Status\n[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=DataBiosphere_terra-drs-hub\u0026metric=alert_status)](https://sonarcloud.io/summary/new_code?id=DataBiosphere_terra-drs-hub)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatabiosphere%2Fterra-drs-hub","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatabiosphere%2Fterra-drs-hub","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatabiosphere%2Fterra-drs-hub/lists"}