{"id":47471154,"url":"https://github.com/NVIDIA/bionemo-framework","last_synced_at":"2026-04-08T10:00:33.362Z","repository":{"id":259334399,"uuid":"705428944","full_name":"NVIDIA/bionemo-framework","owner":"NVIDIA","description":"BioNeMo Framework: For building and adapting AI models in drug discovery at scale","archived":false,"fork":false,"pushed_at":"2026-04-04T05:42:30.000Z","size":447529,"stargazers_count":714,"open_issues_count":169,"forks_count":134,"subscribers_count":43,"default_branch":"main","last_synced_at":"2026-04-05T13:02:38.776Z","etag":null,"topics":["drug-discovery","gpu","machine-learning","pytorch"],"latest_commit_sha":null,"homepage":"https://nvidia.github.io/bionemo-framework/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NVIDIA.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE/license.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-10-16T01:31:06.000Z","updated_at":"2026-04-05T08:28:12.000Z","dependencies_parsed_at":"2026-01-20T11:13:45.373Z","dependency_job_id":"86510941-424b-451f-abba-2007b1d867df","html_url":"https://github.com/NVIDIA/bionemo-framework","commit_stats":null,"previous_names":["nvidia/bionemo-framework"],"tags_count":21,"template":false,"template_full_name":null,"purl":"pkg:github/NVIDIA/bionemo-framework","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NVIDIA%2Fbionemo-framework","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NVIDIA%2Fbionemo-framework/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NVIDIA%2Fbionemo-framework/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NVIDIA%2Fbionemo-framework/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NVIDIA","download_url":"https://codeload.github.com/NVIDIA/bionemo-framework/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NVIDIA%2Fbionemo-framework/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31549900,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-07T16:28:08.000Z","status":"online","status_checked_at":"2026-04-08T02:00:06.127Z","response_time":54,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["drug-discovery","gpu","machine-learning","pytorch"],"created_at":"2026-03-25T00:00:37.622Z","updated_at":"2026-04-08T10:00:33.354Z","avatar_url":"https://github.com/NVIDIA.png","language":"Jupyter Notebook","funding_links":[],"categories":["🤖 Foundation Models for Science","Biomedical Research \u0026 Drug Discovery"],"sub_categories":["Domain-Specific Models"],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003ch1\u003eBioNeMo Framework\u003c/h1\u003e\n  \u003ch4\u003eGPU-optimized recipes \u0026 toolkits for training transformer models at scale with biological data\u003c/h4\u003e\n\u003c/div\u003e\n\n\u003cdiv align=\"left\"\u003e\n\n[![Click here to deploy.](https://uohmivykqgnnbiouffke.supabase.co/storage/v1/object/public/landingpage/brevdeploynavy.svg)](https://console.brev.dev/launchable/deploy/now?launchableID=env-2pPDA4sJyTuFf3KsCv5KWRbuVlU)\n[![Docs Build](https://img.shields.io/github/actions/workflow/status/NVIDIA/bionemo-framework/pages/pages-build-deployment?label=docs-build)](https://nvidia.github.io/bionemo-framework)\n[![Test Status](https://github.com/NVIDIA/bionemo-framework/actions/workflows/unit-tests.yml/badge.svg)](https://github.com/NVIDIA/bionemo-framework/actions/workflows/unit-tests.yml)\n[![Latest Tag](https://img.shields.io/github/v/tag/NVIDIA/bionemo-framework?label=latest-version)](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/clara/containers/bionemo-framework/tags)\n[![codecov](https://codecov.io/gh/NVIDIA/bionemo-framework/branch/main/graph/badge.svg?token=XqhegdZRqB)](https://codecov.io/gh/NVIDIA/bionemo-framework)\n\n\u003cdiv align=\"left\"\u003e\n\nNVIDIA BioNeMo Framework is a comprehensive suite of programming tools, libraries, and models designed for digital biology. It accelerates the most time-consuming and costly stages of building and adapting biomolecular AI models by providing domain-specific, optimized model recipes and tooling that are easily integrated into GPU-based computational resources with state-of-the-art performance.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/docs/assets/images/esm2/esm2_low_precision/esm2_15b_grouped_bars.png\" width=\"600\"\u003e\n  \u003cbr\u003e\n  \u003cem\u003e Training benchmarks for ESM-2, a well known protein sequence model using the BERT architecture.\u003c/em\u003e\n\u003c/p\u003e\n\n## ⚡ Quick Start\n\n```bash\n# Try BioNeMo Recipes in Google Colab (Recommend A100, may be too slow or run out of memory on T4)\n# Copy paste into Google Colab cells\n\n!git clone https://github.com/NVIDIA/bionemo-framework.git\ncd bionemo-framework/bionemo-recipes/recipes/esm2_native_te/\n\n# Install transformer_engine[pytorch] from source, it takes a long time to install from PYPI\n!curl -L -o transformer_engine_torch-2.8.0-cp312-cp312-linux_x86_64.whl \"https://drive.google.com/uc?export=download\u0026id=1Oz6dkkIMahv3LN_fQhhQRolZ3m-sr9SF\"\n!pip install --no-build-isolation transformer-engine transformer_engine_torch-2.8.0-cp312-cp312-linux_x86_64.whl\n\n# Install dependencies\n!pip install -r requirements.txt\n\n# Run ESM2 Native Recipes with TE\n!python train_ddp.py\n```\n\n## Recent News\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/docs/assets/images/sae.png\" width=\"600\"\u003e\n  \u003cbr\u003e\n  \u003cem\u003e\u003ca href=\"https://research.nvidia.com/labs/dbr/blog/sae/\"\u003eSparse autoencoder feature dashboard for CodonFM 1B\u003c/a\u003e, showing learned latent features and their activations on protein sequences.\u003c/em\u003e\n\u003c/p\u003e\n\n- 03/13/2026 [Sparse Autoencoders for model interpretability](bionemo-recipes/interpretability/sparse_autoencoders/) — train and analyze SAEs on biological foundation models. Includes recipes for ESM2 and CodonFM with interactive feature dashboards.\n- 03/09/2026 [Qwen2.5 / Qwen3 model](bionemo-recipes/models/qwen/) with TE acceleration, FP8/MXFP8, KV-cache inference, and bidirectional HF checkpoint conversion.\n- 03/05/2026 [ESM2 NVFP4 and MXFP8](bionemo-recipes/recipes/esm2_native_te/README.md#low-precision-performance-benchmarks) low-precision training — up to **2,367 TFLOPS/GPU** on NVIDIA B300 at 15B scale with per-layer precision control.\n- 02/23/2026 [Mixtral MoE model](bionemo-recipes/models/mixtral/) with TE `GroupedLinear` for efficient parallel expert computation, FP8/FP4 support, and HF conversion.\n- 02/13/2026 [ESM2 PEFT recipe](bionemo-recipes/recipes/esm2_peft_te/) for LoRA fine-tuning with sequence packing support.\n- 01/14/2026 [Llama3 Context Parallelism](bionemo-recipes/recipes/llama3_native_te/README.md#performance-benchmarks) — scaling Llama 3 70B to 144K context on 36x GB300 NVL36 with ~65% MFU.\n- 10/27/2025 [CodonFM recipe](https://github.com/NVIDIA/bionemo-framework/tree/main/bionemo-recipes/recipes/codonfm_ptl_te) released! This is an accelerated version of the original [research codebase](https://github.com/NVIDIA-Digital-Bio/CodonFM) with [scientific preprint](https://research.nvidia.com/labs/dbr/assets/data/manuscripts/nv-codonfm-preprint.pdf).\n- 09/30/2025 Megatron/NeMo 5D parallel BioNeMo Framework image v2.7 [released on NGC](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/clara/containers/bionemo-framework) for both x86 and ARM CPUs.\n- 09/01/2025 [bionemo-recipes](https://github.com/NVIDIA/bionemo-framework/tree/main/bionemo-recipes) goes live! Lightweight and portable examples with state-of-the-art training performance you can riff on to meet your needs.\n\n## Code Overview\n\nA core use-case of the BioNeMo Framework is to help digital biology scientists accelerate and scale their model training onto a compute cluster. This repository contains 3 categories of modules for this use-case:\n\n1\\. Models using **fully-sharded-data-parallel (FSDP)**, which is possible with a number of different implementations including [PyTorch’s FSDP2/FSDP1](https://docs.pytorch.org/tutorials/intermediate/FSDP_tutorial.html) and [NVIDIA megatron-FSDP](https://github.com/NVIDIA/Megatron-LM/tree/main/megatron/core/distributed/fsdp/src). Sharding a model with FSDP typically requires only a few lines of code changes. You can find models and ready-to-run recipes parallelized with megatron-FSDP and accelerated with [NVIDIA TransformerEngine (TE)](https://github.com/NVIDIA/TransformerEngine) in [`bionemo-recipes`](./bionemo-recipes/).\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003e(Click to expand) \u003ccode\u003ebionemo-recipes\u003c/code\u003e support matrix \u003c/b\u003e\u003c/summary\u003e\n\u003csmall\u003e\n\n| Directory                                      | Description                                                                                                                   | Support Status | 5D Parallel | Megatron-FSDP | TE     | Sequence Packing | FP8    | Context Parallelism |\n| ---------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | -------------- | ----------- | ------------- | ------ | ---------------- | ------ | ------------------- |\n| `models/`\u003cbr\u003e`amplify`                         | TE accelerated protein BERT, pushed to HuggingFace                                                                            | ✅ Active      | ❌          | ✅            | ✅     | 🚧 WIP           | ✅     | 🚧 WIP              |\n| `models/`\u003cbr\u003e`esm2`                            | TE accelerated protein BERT, pushed to HuggingFace                                                                            | ✅ Active      | ❌          | ✅            | ✅     | ✅               | ✅     | ✅                  |\n| `models/`\u003cbr\u003e`llama3`                          | TE accelerated Llama 3                                                                                                        | ✅ Active      | ❌          | 🚧 WIP        | ✅     | ✅               | ✅     | ✅                  |\n| `models/`\u003cbr\u003e`geneformer`                      | TE accelerated single-cell BERT                                                                                               | 🚧 WIP         | ❌          | ✅            | 🚧 WIP | 🚧 WIP           | 🚧 WIP | 🚧 WIP              |\n| `recipes/`\u003cbr\u003e`codonfm_ptl_te`                 | Recipe for [CodonFM](https://research.nvidia.com/labs/dbr/assets/data/manuscripts/nv-codonfm-preprint.pdf)'s Encodon using TE | ✅ Active      | ❌          | 🚧 WIP        | ✅     | ✅               | 🚧 WIP | 🚧 WIP              |\n| `recipes/`\u003cbr\u003e`esm2_accelerate_te`             | Recipe for ESM2 TE + HF Accelerate                                                                                            | ✅ Active      | ❌          | 🚧 WIP        | ✅     | ❌               | ✅     | 🚧 WIP              |\n| `recipes/`\u003cbr\u003e`esm2_native_te`                 | Recipe for ESM2 TE + native PyTorch                                                                                           | ✅ Active      | ❌          | ✅            | ✅     | ✅               | ✅     | ✅                  |\n| `recipes/`\u003cbr\u003e`geneformer_native_te_mfsdp_fp8` | Recipe for Geneformer HF model                                                                                                | 🚧 WIP         | ❌          | ✅            | ✅     | ❌               | ✅     | 🚧 WIP              |\n| `recipes/`\u003cbr\u003e`llama3_native_te`               | Recipe for Llama 3 TE + native PyTorch                                                                                        | ✅ Active      | ❌          | 🚧 WIP        | ✅     | ✅               | ✅     | ✅                  |\n| `models/`\u003cbr\u003e`mixtral`                         | TE accelerated MoE model                                                                                                      | ✅ Active      | ❌          | 🚧 WIP        | ✅     | ✅               | ✅     | 🚧 WIP              |\n| `models/`\u003cbr\u003e`qwen`                            | TE accelerated Qwen2.5/Qwen3                                                                                                  | ✅ Active      | ❌          | 🚧 WIP        | ✅     | ✅               | ✅     | 🚧 WIP              |\n| `recipes/`\u003cbr\u003e`esm2_peft_te`                   | Recipe for ESM2 LoRA fine-tuning                                                                                              | ✅ Active      | ❌          | ❌            | ✅     | ✅               | 🚧 WIP | ❌                  |\n| `recipes/`\u003cbr\u003e`evo2_megatron`                  | Recipe for Evo2 via Megatron Bridge                                                                                           | 🚧 WIP         | ❌          | ❌            | ✅     | ❌               | ✅     | ❌                  |\n| `recipes/`\u003cbr\u003e`fp8_analysis`                   | FP8 training analyzer \u0026 heatmap tool                                                                                          | ✅ Active      | N/A         | N/A           | N/A    | N/A              | N/A    | N/A                 |\n| `recipes/`\u003cbr\u003e`vit`                            | Recipe for Vision Transformer                                                                                                 | 🚧 WIP         | ❌          | ✅            | ✅     | ❌               | ✅     | 🚧 WIP              |\n\n\u003c/small\u003e\n\n\u003c/details\u003e\n\n2\\. Models using explicit **5D parallelism** (tensor parallel, pipeline parallel, context parallel, etc.), for which NVIDIA provides accelerated support with [NeMo](https://github.com/NVIDIA-NeMo/NeMo) and [Megatron-Core](https://github.com/NVIDIA/Megatron-LM). 5D parallelism requires explicit modification of the model code to make it shardable along different dimensions. The models for this style of acceleration and parallelism can be found in the `sub-packages` directory. While it is possible to pip install the models, we strongly suggest using our [Docker image](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/clara/containers/bionemo-framework) that comes with NeMo and Megatron-Core pre-installed.\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003e(Click to expand) \u003ccode\u003esub-packages\u003c/code\u003e models support matrix\u003c/b\u003e\u003c/summary\u003e\n\u003csmall\u003e\n\n| Directory               | Description                      | Support        | 5D Parallel | Megatron-FSDP | TE  | Sequence Packing | FP8 | Context Parallel |\n| ----------------------- | -------------------------------- | -------------- | ----------- | ------------- | --- | ---------------- | --- | ---------------- |\n| `bionemo-core`          | Model Config/test data utils     | ✅ Active      | ✅          | N/A           | ✅  | ❌               | N/A | N/A              |\n| `bionemo-evo2`          | 5D parallel model                | ✅ Active      | ✅          | ❌            | ✅  | ❌               | ✅  | ✅               |\n| `bionemo-example_model` | Example 5D parallel model        | 🔧 Maintenance | ✅          | ❌            | ✅  | ❌               | ✅  | ✅               |\n| `bionemo-llm`           | 5D parallel base model (BioBert) | ✅ Active      | ✅          | ❌            | ✅  | ✅               | ✅  | ✅               |\n| `bionemo-testing`       | Testing Utilities                | ✅ Active      | ✅          | N/A           | N/A | N/A              | N/A | N/A              |\n\n\u003c/small\u003e\n\u003c/details\u003e\n\n3\\. Tooling for dataloading and in-the-training-loop processing, which are lightweight and individually pip installable. These are also in the `sub-packages` directory adjacent to the 5D parallel models.\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003e(Click to expand) \u003ccode\u003esub-packages\u003c/code\u003e tooling support matrix\u003c/b\u003e\u003c/summary\u003e\n\u003csmall\u003e\n\n| Directory                     | Description                                | Support        | 5D Parallel   | Megatron-FSDP | TE  | Sequence Packing | FP8 | Context Parallel |\n| ----------------------------- | ------------------------------------------ | -------------- | ------------- | ------------- | --- | ---------------- | --- | ---------------- |\n| `bionemo-moco`                | Molecular Co-design tools                  | ✅ Active      | ❌            | N/A           | N/A | N/A              | N/A | N/A              |\n| `bionemo-noodles`             | Python API to fast FASTA file I/O          | 🔧 Maintenance | ❌            | N/A           | N/A | N/A              | N/A | N/A              |\n| `bionemo-scspeedtest`         | Single Cell Dataloading benchmark tests    | ✅ Active      | N/A           | N/A           | N/A | N/A              | N/A | N/A              |\n| `bionemo-size-aware-batching` | Memory consumption aware batching          | 🔧 Maintenance | N/A           | N/A           | N/A | N/A              | N/A | N/A              |\n| `bionemo-scdl`                | Modular Single Cell Data Loader            | ✅ Active      | ✅ Compatible | N/A           | N/A | N/A              | N/A | N/A              |\n| `bionemo-webdatamodule`       | PyTorch Lightning module to use WebDataset | 🔧 Maintenance | N/A           | N/A           | N/A | N/A              | N/A | N/A              |\n\n\u003c/small\u003e\n\u003c/details\u003e\n\nBioNeMo Framework is part of a larger ecosystem of NVIDIA Biopharma products. Get notified of new releases, bug fixes, critical security updates, and more for biopharma. [Subscribe.](https://www.nvidia.com/en-us/clara/biopharma/product-updates/)\n\n## Documentation Resources\n\n- **Official Documentation:** Documentation for sub-packages, including user guides, API references, and troubleshooting, is available on our [official documentation](https://docs.nvidia.com/bionemo-framework/latest/). Nightly builds of this documentation is available on [BioNeMo Framework GitHub Pages](https://nvidia.github.io/bionemo-framework/)\n\n- **🚧 In-Progress Documentation 🚧:** `bionemo-recipes` documentation is currently work in progress, however the recipes are meant to be self-documented and easy to understand—we suggest you throw them into your favorite genai code assistant!\n\n## Getting Started with BioNeMo Framework - 5D Parallelism with NeMo/Megatron implementations\n\n:warning: **(This section is not relevant for bionemo-recipes)**\n\nFull documentation on using the BioNeMo Framework is provided in our documentation:\n\u003chttps://docs.nvidia.com/bionemo-framework/latest/user-guide/\u003e. To simplify the integration of optimized third-party dependencies, BioNeMo is primarily distributed as a containerized library. You can download the latest released container for the BioNeMo Framework from\n[NGC](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/clara/containers/bionemo-framework). To launch a pre-built container, you can use the brev.dev launchable [![ Click here to deploy.](https://uohmivykqgnnbiouffke.supabase.co/storage/v1/object/public/landingpage/brevdeploynavy.svg)](https://console.brev.dev/launchable/deploy/now?launchableID=env-2pPDA4sJyTuFf3KsCv5KWRbuVlU) or execute the following command:\n\n```bash\ndocker run --rm -it \\\n  --gpus=all --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 \\\n  nvcr.io/nvidia/clara/bionemo-framework:nightly \\\n  /bin/bash\n```\n\n### Setting up a local development environment\n\n#### Initializing 3rd-party dependencies as git submodules\n\nThe NeMo and Megatron-LM dependencies are included as git submodules in BioNeMo Framework. The pinned commits for these submodules represent the \"last-known-good\" versions of these packages\nthat are confirmed to be working with BioNeMo Framework (and those that are tested in CI).\n\nTo initialize these sub-modules when cloning the repo, add the `--recursive` flag to the git clone command:\n\n```bash\ngit clone --recursive git@github.com:NVIDIA/bionemo-framework.git\ncd bionemo-framework\n```\n\nTo download the pinned versions of these submodules within an existing git repository, run\n\n```bash\ngit submodule update --init --recursive\n```\n\nDifferent branches of the repo can have different pinned versions of these third-party submodules. Ensure submodules are automatically updated after switching branches or pulling updates by configuring git with:\n\n```bash\ngit config submodule.recurse true\n```\n\n**NOTE**: this setting will not download **new** or remove **old** submodules with the branch's changes.\nYou will have to run the full `git submodule update --init --recursive` command in these situations.\n\n#### Build the Docker Image Locally\n\nWith a locally cloned repository and initialized submodules, build the BioNeMo container using:\n\n```bash\ndocker buildx build . -t my-container-tag\n```\n\nIf you see an error message like `No file descriptors available (os error 24)`, add the option `--ulimit nofile=65535:65535` to the docker build command.\n\n#### VSCode Devcontainer for Interactive Debugging\n\nWe distribute a [development container](https://devcontainers.github.io/) configuration for vscode\n(`.devcontainer/devcontainer.json`) that simplifies the process of local testing and development. Opening the\nbionemo-framework folder with VSCode should prompt you to re-open the folder inside the devcontainer environment.\n\n\u003e [!NOTE]\n\u003e The first time you launch the devcontainer, it may take a long time to build the image. Building the image locally\n\u003e (using the command shown above) will ensure that most of the layers are present in the local docker cache.\n\n### Quick Start\n\nSee the [tutorials pages](https://docs.nvidia.com/bionemo-framework/latest/user-guide/examples/bionemo-esm2/pretrain/)\nfor example applications and getting started guides.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FNVIDIA%2Fbionemo-framework","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FNVIDIA%2Fbionemo-framework","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FNVIDIA%2Fbionemo-framework/lists"}