{"id":28395141,"url":"https://github.com/databiosphere/terra-cloud-resource-lib","last_synced_at":"2025-06-27T01:31:29.326Z","repository":{"id":37897810,"uuid":"249730451","full_name":"DataBiosphere/terra-cloud-resource-lib","owner":"DataBiosphere","description":null,"archived":false,"fork":false,"pushed_at":"2025-06-23T12:27:46.000Z","size":1016,"stargazers_count":5,"open_issues_count":4,"forks_count":1,"subscribers_count":25,"default_branch":"master","last_synced_at":"2025-06-23T13:31:02.255Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DataBiosphere.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-03-24T14:29:43.000Z","updated_at":"2025-03-31T20:51:25.000Z","dependencies_parsed_at":"2023-01-31T06:00:59.456Z","dependency_job_id":"66888576-4e41-4a05-8e42-3ded99f0c573","html_url":"https://github.com/DataBiosphere/terra-cloud-resource-lib","commit_stats":null,"previous_names":[],"tags_count":75,"template":false,"template_full_name":null,"purl":"pkg:github/DataBiosphere/terra-cloud-resource-lib","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataBiosphere%2Fterra-cloud-resource-lib","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataBiosphere%2Fterra-cloud-resource-lib/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataBiosphere%2Fterra-cloud-resource-lib/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataBiosphere%2Fterra-cloud-resource-lib/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DataBiosphere","download_url":"https://codeload.github.com/DataBiosphere/terra-cloud-resource-lib/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataBiosphere%2Fterra-cloud-resource-lib/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262172426,"owners_count":23270009,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-05-31T19:39:17.347Z","updated_at":"2025-06-27T01:31:29.318Z","avatar_url":"https://github.com/DataBiosphere.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Terra Cloud Resource Library\n\nCloud Resource Library (CRL) wraps cloud API client libraries for Terra services. It enforces unified logging and\nallows central changes to how Terra services use Cloud APIs.\n\nTODO add more about library principles and Cloud Object Wrappers.\n\n## Cleanup Mode\nTerra manages cloud resources for users. To have confidence in our changes, we need integration test suites that\nexercises real cloud resource lifecycle operations. The expected use of tests is that tests will fail sometimes.\nNevertheless, we still want to (eventually) clean up cloud resources created by tests. Enter CRL cleanup mode.\n\nCRL can be configured to run in an integration test cleanup mode. In this mode, when a cloud resource is about to be\ncreated with a Cloud Object Wrapper method, the unique identifier for that cloud resource is persisted to an external\ndatabase. The resources tracked for cleanup can be later deleted, if they have not already been deleted. See\n[CRL Janitor](https://github.com/DataBiosphere/crl-janitor).\n\n# Development\n\n## Requirements\n\n- Java 17\n- [git-secret](https://git-secret.io/installation)\n- git config updated via `./minnie-kenny.sh -f`\n\n## Using the Gradle wrapper\nSet executable permissions:\n```\nchmod +x gradlew\n```\n\n### Dependencies\nWe use [Gradle's dependency locking](https://docs.gradle.org/current/userguide/dependency_locking.html)\nto ensure that builds use the same transitive dependencies, so they're reproducible. This means that\nadding or updating a dependency requires telling Gradle to save the change. If you're getting errors\nthat mention \"dependency lock state\" after changing a dep, you need to do this step.\n```\n./gradlew dependencies --write-locks\n```\n\n## Testing\n\n### Unit tests\nBuild and run unit tests:\n```\n./gradlew build test\n```\n\n### Integration Tests\nIntegration tests are run against cloud providers. Cloud resources required for integration tests are defined in\n[terraform-ap-modules/crl-test](https://github.com/broadinstitute/terraform-ap-modules/tree/master/crl-test).\n\nTo run them locally, you need account credentials. Today, two json files with the google service account credentials\nis expected at `common/src/testFixtures/resources/integration_service_account-admin.json`.\nAnd `common/src/testFixtures/resources/integration_service_account-user.json`\n\nBroad employees can get the credentials from Vault with:\n```\n./local-dev/render-test-config.sh\n```\nThen actually run the tests with:\n```\n./gradlew integrationTest\n```\n\nTODO: Make this easier, consider making credentials path configurable.\n\n## Linter\nAutomatically fix linting issues:\n```\n./gradlew spotlessApply\n```\n\n## Publishing an update\n\n# Quickstart\nThe current available packages are:  \n#### google-bigquery\nWraps [Google Cloud BigQuery API](https://cloud.google.com/bigquery/docs/apis).  \nArtifact Repository can be found [here](https://broadinstitute.jfrog.io/broadinstitute/webapp/#/artifacts/browse/tree/General/libs-snapshot-local/bio/terra/cloud-resource-lib/google-bigquery).\n#### google-billing\nWraps [Google Billing API](https://cloud.google.com/billing/docs/apis).  \nArtifact Repository can be found [here](https://broadinstitute.jfrog.io/broadinstitute/webapp/#/artifacts/browse/tree/General/libs-snapshot-local/bio/terra/cloud-resource-lib/google-billing).\n#### google-cloudresourcemanager\nWraps [Google Cloud ResourceManager Client API](https://cloud.google.com/resource-manager/docs/apis).\nTargets the Cloud Resource Manager v1 for Projects and Organizations. [github](https://github.com/googleapis/google-api-java-client-services/tree/master/clients/google-api-services-cloudresourcemanager/v1).\nArtifact Repository can be found [here](https://broadinstitute.jfrog.io/broadinstitute/webapp/#/artifacts/browse/tree/General/libs-snapshot-local/bio/terra/cloud-resource-lib/google-cloudresourcemanager).\n#### google-notebooks\nWraps [Google Cloud AI Notebooks Client API](https://cloud.google.com/ai-platform/notebooks/docs/reference/rest).\nArtifact Repository can be found [here](https://broadinstitute.jfrog.io/broadinstitute/webapp/#/artifacts/browse/tree/General/libs-snapshot-local/bio/terra/cloud-resource-lib/google-notebooks).\n#### google-dataproc\nWraps [Google Cloud Dataproc Client API](https://cloud.google.com/dataproc/docs/reference/rest).\nArtifact Repository can be found [here](https://broadinstitute.jfrog.io/broadinstitute/webapp/#/artifacts/browse/tree/General/libs-snapshot-local/bio/terra/cloud-resource-lib/google-dataproc).\n#### google-serviceusage\nWraps [Google Service Usage API](https://cloud.google.com/service-usage/docs/overview).  \nArtifact Repository can be found [here](https://broadinstitute.jfrog.io/broadinstitute/webapp/#/artifacts/browse/tree/General/libs-snapshot-local/bio/terra/cloud-resource-lib/google-serviceusage).\n#### google-storage\nWraps [Google Cloud Storage API](https://cloud.google.com/storage/docs/apis).  \nArtifact Repository can be found [here](https://broadinstitute.jfrog.io/broadinstitute/webapp/#/artifacts/browse/tree/General/libs-snapshot-local/bio/terra/cloud-resource-lib/google-storage).\n#### azure-resourcemanager-compute\nWraps [Azure Compute API](https://docs.microsoft.com/en-us/rest/api/compute/).\nArtifact Repository can be found [here](https://broadinstitute.jfrog.io/broadinstitute/webapp/#/artifacts/browse/tree/General/libs-snapshot-local/bio/terra/cloud-resource-lib/azure-resourcemanager-compute).\n#### azure-resourcemanager-containerinstance\nWraps [Azure Container Instances API](https://docs.microsoft.com/en-us/rest/api/container-instances/).\nArtifact Repository can be found [here](https://broadinstitute.jfrog.io/broadinstitute/webapp/#/artifacts/browse/tree/General/libs-snapshot-local/bio/terra/cloud-resource-lib/azure-resourcemanager-containerinstance).\n#### azure-resourcemanager-relay\nWraps [Azure Relay API](https://docs.microsoft.com/en-us/rest/api/relay/).\nArtifact Repository can be found [here](https://broadinstitute.jfrog.io/broadinstitute/webapp/#/artifacts/browse/tree/General/libs-snapshot-local/bio/terra/cloud-resource-lib/azure-resourcemanager-relay).\n#### azure-resourcemanager-batch-pool\nWraps [Azure Batch Pool API](https://learn.microsoft.com/en-us/rest/api/batchservice/pool).\nArtifact Repository can be found [here](https://broadinstitute.jfrog.io/broadinstitute/webapp/#/artifacts/browse/tree/General/libs-snapshot-local/bio/terra/cloud-resource-lib/azure-resourcemanager-batch).\n#### cloud-resource-schema\nThe general schema for how cloud resources are presented in CRL.   \nArtifact Repository can be found [here](https://broadinstitute.jfrog.io/broadinstitute/webapp/#/artifacts/browse/tree/General/libs-snapshot-local/bio/terra/cloud-resource-lib/cloud-resource-schema).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatabiosphere%2Fterra-cloud-resource-lib","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatabiosphere%2Fterra-cloud-resource-lib","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatabiosphere%2Fterra-cloud-resource-lib/lists"}