{"id":13905210,"url":"https://github.com/projectglow/glow","last_synced_at":"2025-10-21T19:51:47.106Z","repository":{"id":41820573,"uuid":"212904926","full_name":"projectglow/glow","owner":"projectglow","description":"An open-source toolkit for large-scale genomic analysis","archived":false,"fork":false,"pushed_at":"2024-11-10T00:12:18.000Z","size":101419,"stargazers_count":273,"open_issues_count":36,"forks_count":111,"subscribers_count":20,"default_branch":"main","last_synced_at":"2024-11-16T12:48:01.422Z","etag":null,"topics":["delta","genomics","gwas","machine-learning","population-genetics","regression","spark"],"latest_commit_sha":null,"homepage":"https://projectglow.io","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/projectglow.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.txt","code_of_conduct":"CODE-OF-CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-10-04T21:26:47.000Z","updated_at":"2024-11-16T07:42:37.000Z","dependencies_parsed_at":"2023-01-30T05:15:42.932Z","dependency_job_id":"d0da5508-8729-45ca-9a82-a8715e2fa67d","html_url":"https://github.com/projectglow/glow","commit_stats":{"total_commits":481,"total_committers":30,"mean_commits":"16.033333333333335","dds":0.6465696465696466,"last_synced_commit":"d63e99ea892ab82a9fd302467cc47aa3eb0a1571"},"previous_names":[],"tags_count":22,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/projectglow%2Fglow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/projectglow%2Fglow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/projectglow%2Fglow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/projectglow%2Fglow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/projectglow","download_url":"https://codeload.github.com/projectglow/glow/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":226336456,"owners_count":17608842,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["delta","genomics","gwas","machine-learning","population-genetics","regression","spark"],"created_at":"2024-08-06T23:01:12.005Z","updated_at":"2025-10-21T19:51:47.088Z","avatar_url":"https://github.com/projectglow.png","language":"Scala","funding_links":[],"categories":["医疗领域"],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"static/glow_logo_horiz_color.png\" width=\"300px\"/\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n\tAn open-source toolkit for large-scale genomic analyes\n  \u003cbr/\u003e\n  \u003ca href=\"https://glow.readthedocs.io/en/latest/?badge=latest\"\u003e\u003cstrong\u003eExplore the docs »\u003c/strong\u003e\u003c/a\u003e\n  \u003cbr/\u003e\n  \u003cbr/\u003e\n  \u003ca href=\"https://github.com/projectglow/glow/issues\"\u003eIssues\u003c/a\u003e\n\u003c/p\u003e\n\nGlow is an open-source toolkit to enable bioinformatics at biobank-scale and beyond.\n\n[![Tests](https://github.com/projectglow/glow/actions/workflows/tests.yml/badge.svg)](https://github.com/projectglow/glow/actions/workflows/tests.yml)\n[![Documentation\nStatus](https://readthedocs.org/projects/glow/badge/?version=latest)](https://glow.readthedocs.io/en/latest/?badge=latest)\n[![PyPI](https://img.shields.io/pypi/v/glow.py.svg)](https://pypi.org/project/glow.py/)\n[![Maven Central](https://img.shields.io/maven-central/v/io.projectglow/glow-spark3_2.12.svg)](https://mvnrepository.com/artifact/io.projectglow)\n[![Coverage Status](https://codecov.io/gh/projectglow/glow/branch/main/graph/badge.svg)](https://codecov.io/gh/projectglow/glow)\n[![Conda Version](https://img.shields.io/conda/vn/conda-forge/glow.svg)](https://anaconda.org/conda-forge/glow)\n[![DOI](https://zenodo.org/badge/212904926.svg)](https://zenodo.org/badge/latestdoi/212904926)\n\n# Easy to get started\nThe toolkit includes building blocks to perform common analyses right away:\n\n- Load VCF, BGEN, and Plink files into distributed DataFrames\n- Perform quality control and data manipulation with built-in functions\n- Variant normalization and liftOver\n- Perform genome-wide association studies\n- Integrate with Spark ML libraries for population stratification\n- Parallelize command line tools to scale existing workflows\n\n# Built to scale\nGlow makes genomic data work with Spark, the leading engine for working with large structured\ndatasets. It fits natively into the ecosystem of tools that have enabled thousands of organizations\nto scale their workflows. Glow bridges the gap between bioinformatics and the\nSpark ecosystem.\n\n# Flexible\nGlow works with datasets in common file formats like VCF, BGEN, and Plink as well as\nhigh-performance big data\nstandards. You can write queries using the native Spark SQL APIs in Python, SQL, R, Java, and Scala.\nThe same APIs allow you to bring your genomic data together with other datasets such as electronic\nhealth records, real world evidence, and medical images. Glow makes it easy to parallelize existing\ntools and libraries implemented as command line tools or Pandas functions.\n\n\n# Building and Testing\nThis project is built using [sbt](https://www.scala-sbt.org/1.0/docs/Setup.html) and Java 8.\n\nTo build and run Glow, you must [install conda](https://docs.conda.io/en/latest/miniconda.html) and\nactivate the environment in `python/environment.yml`.\n```\nconda env create -f python/environment.yml\nconda activate glow\n```\n\nWhen the environment file changes, you must update the environment:\n```\nconda env update -f python/environment.yml\n```\n\nStart an sbt shell using the `sbt` command.\n\n\u003e **FYI**: The following SBT projects are built on Spark 3.5.1/Scala 2.12.19 by default. To change the Spark version and\nScala version, set the environment variables `SPARK_VERSION` and `SCALA_VERSION`.\n\nTo compile the main code:\n```\ncompile\n```\n\nTo run all Scala tests:\n```\ncore/test\n```\n\nTo test a specific suite:\n```\ncore/testOnly *VCFDataSourceSuite\n```\n\nTo run all Python tests:\n```\npython/test\n```\nThese tests will run with the same Spark classpath as the Scala tests.\n\nTo test a specific Python test file:\n```\npython/pytest python/test_render_template.py\n```\n\nWhen using the `pytest` key, all arguments are passed directly to the\n[pytest runner](https://docs.pytest.org/en/latest/usage.html).\n\nTo run documentation tests:\n```\ndocs/test\n```\n\nTo run the Scala, Python and documentation tests:\n```\ntest\n```\n\nTo run Scala tests against the staged Maven artifact with the current stable version:\n```\nstagedRelease/test\n```\n\n## Testing code on a Databricks cluster\n\nYou can use the [build](https://github.com/projectglow/glow/blob/main/bin/build) script to create artifacts that you can install on a Databricks cluster.\n\nTo build Python and Scala artifacts:\n```\nbin/build --scala --python\n```\n\nTo build only Python (no sbt installation required):\n```\nbin/build --python\n```\n\nTo install the artifacts on a Databricks cluster after building:\n```\nbin/build --python --scala --install MY_CLUSTER_ID\n```\n\n## IntelliJ Tips\n\nIf you use IntelliJ, you'll want to:\n- Download library and SBT sources; use SBT shell for imports and build from [IntelliJ](https://www.jetbrains.com/help/idea/sbt.html)\n- Set up [scalafmt on save](https://scalameta.org/scalafmt/docs/installation.html)\n\nTo run Python unit tests from inside IntelliJ, you must:\n- Open the \"Terminal\" tab in IntelliJ\n- Activate the glow conda environment (`conda activate glow`)\n- Start an sbt shell from inside the terminal (`sbt`)\n\nThe \"sbt shell\" tab in IntelliJ will NOT work since it does not use the glow conda environment.\n\nTo test or testOnly in remote debug mode with IntelliJ IDEA set the remote debug configuration in IntelliJ to 'Attach to remote JVM' mode and a specific port number (here the default port number 5005 is used) and then modify the `testJavaOptions` in `build.sbt` to include:\n```\n\"-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005\"\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprojectglow%2Fglow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprojectglow%2Fglow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprojectglow%2Fglow/lists"}