{"id":34630682,"url":"https://github.com/lucidrains/alphagenome","last_synced_at":"2026-02-04T22:18:43.648Z","repository":{"id":301555289,"uuid":"1009618138","full_name":"lucidrains/alphagenome","owner":"lucidrains","description":"Implementation of AlphaGenome, Deepmind's updated genomic attention model","archived":false,"fork":false,"pushed_at":"2026-02-01T13:23:22.000Z","size":17988,"stargazers_count":86,"open_issues_count":1,"forks_count":12,"subscribers_count":4,"default_branch":"main","last_synced_at":"2026-02-01T15:53:26.108Z","etag":null,"topics":["artificial-intelligence","attention-mechanisms","deep-learning","genomics","splicing","transformers"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lucidrains.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-06-27T12:30:07.000Z","updated_at":"2026-02-01T15:52:40.000Z","dependencies_parsed_at":"2025-07-24T05:13:23.168Z","dependency_job_id":"ace708b3-2b0f-438c-89d5-0085672fe7b7","html_url":"https://github.com/lucidrains/alphagenome","commit_stats":null,"previous_names":["lucidrains/alphagenome"],"tags_count":52,"template":false,"template_full_name":null,"purl":"pkg:github/lucidrains/alphagenome","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Falphagenome","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Falphagenome/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Falphagenome/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Falphagenome/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lucidrains","download_url":"https://codeload.github.com/lucidrains/alphagenome/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Falphagenome/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29097926,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-04T21:05:08.033Z","status":"ssl_error","status_checked_at":"2026-02-04T21:04:53.031Z","response_time":62,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","attention-mechanisms","deep-learning","genomics","splicing","transformers"],"created_at":"2025-12-24T16:35:31.160Z","updated_at":"2026-02-04T22:18:43.643Z","avatar_url":"https://github.com/lucidrains.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"./extended-figure-1.png\" width=\"450px\"\u003e\u003c/img\u003e\n\n## AlphaGenome\n\nImplementation of [AlphaGenome](https://deepmind.google/discover/blog/alphagenome-ai-for-better-understanding-the-genome/), Deepmind's updated genomic attention model\n\nThe [official code](https://github.com/google-deepmind/alphagenome_research) has been released!\n\n## Appreciation\n\n- [Miquel Anglada-Girotto](https://github.com/MiqG) for contributing the organism, output embedding, loss functions, and all the splicing prediction heads!\n\n- [Xinming Tu](https://github.com/XinmingTu) for aligning the architecture with the official JAX implementation, enabling pretrained weight compatibility, TFRecord dataset support, scoring and evaluation modules, and a test suite!\n\n## Install\n\n```bash\npip install alphagenome-pytorch\n```\n\nOr with [uv](https://docs.astral.sh/uv/) (recommended):\n\n```bash\nuv pip install alphagenome-pytorch\n```\n\n### Environment Setup\n\nTo use pretrained weights from HuggingFace, create a `.env` file with your credentials:\n\n```bash\ncp .env.template .env\n# Edit .env and add your HF_TOKEN\n```\n\nRequired environment variables:\n- `HF_TOKEN`: Your HuggingFace API token ([get one here](https://huggingface.co/settings/tokens))\n\n## Usage\n\nThe main unet transformer, without any heads\n\n```python\nimport torch\nfrom alphagenome_pytorch import AlphaGenome\n\nmodel = AlphaGenome()\n\ndna = torch.randint(0, 4, (2, 8192))\n\n# organism_index - 0 for human, 1 for mouse - can be changed with `num_organisms` on `AlphaGenome`\n\nembeds_1bp, embeds_128bp, embeds_pair = model(dna, organism_index = 0) # (2, 8192, 1536), (2, 64, 3072), (2, 4, 4, 128)\n```\n\nAdding all types of output heads (thanks to [@MiqG](https://github.com/MiqG))\n\n```python\nimport torch\nfrom alphagenome_pytorch import AlphaGenome, publication_heads_config\n\nmodel = AlphaGenome()\n\nmodel.add_heads(\n    'human',\n    num_tracks_1bp = 10,\n    num_tracks_128bp = 10,\n    num_tracks_contacts = 128,\n    num_splicing_contexts = 64, # 2 strands x num. CURIE conditions\n)\n\ndna = torch.randint(0, 4, (2, 8192))\n\norganism_index = torch.tensor([0, 1]) # the organism that each sequence belongs to\nsplice_donor_idx = torch.tensor([[10, 100, 34], [24, 546, 870]])\nsplice_acceptor_idx = torch.tensor([[15, 103, 87], [56, 653, 900]])\n\n# get sequence embeddings\n\nembeddings_1bp, embeddings_128bp, embeddings_pair = model(dna, organism_index, return_embeds = True) # (2, 8192, 1536), (2, 64, 3072), (2, 4, 4, 128)\n\n# get track predictions\n\nout = model(\n    dna,\n    organism_index,\n    splice_donor_idx = splice_donor_idx,\n    splice_acceptor_idx = splice_acceptor_idx\n)\n\nfor organism, outputs in out.items():\n    for out_head, out_values in outputs.items():\n        print(organism, out_head, out_values.shape)\n\n# human 1bp_tracks torch.Size([2, 8192, 10])\n# human 128bp_tracks torch.Size([2, 64, 10])\n# human contact_head torch.Size([2, 4, 4, 128])\n# human splice_logits torch.Size([2, 8192, 5])\n# human splice_usage torch.Size([2, 8192, 64])\n# human splice_juncs torch.Size([2, 3, 3, 64])\n\n# initialize published AlphaGenome for human and mouse\nmodel = AlphaGenome()\nmodel.add_heads(**publication_heads_config['human'])\nmodel.add_heads(**publication_heads_config['mouse'])\nmodel.total_parameters # 259,459,534 (vs ~450 million trainable parameters)\n```\n\n## Training\n\n### test minimal architecture\n```shell\n# loss quickly decreases and stabilizes at around 1349651\n# this minimal model (576,444 parameters) can be run with cpu\n\npython train_dummy.py --config_file=configs/dummy.yaml\n```\n\n## Loading Pretrained Weights\n\n### Installation requirements\n\nTo use the conversion utilities, you will need to install the `alphagenome-research` package manually from GitHub\n\n```bash\nuv pip install git+https://github.com/google-deepmind/alphagenome_research.git\n```\n\n### Loading and Converting\n\n```python\nfrom alphagenome_pytorch import AlphaGenome\n\n# Load AlphaGenome with official JAX weights\nmodel = AlphaGenome()\nmodel.add_reference_heads(\"human\")\nmodel.load_from_official_jax_model(\"all_folds\")\n\nmodel.eval()\n```\n\n## Contributing\n\n### Development Setup\n\n```bash\n# Clone and install with dev dependencies using uv\ngit clone https://github.com/lucidrains/alphagenome.git\ncd alphagenome\nuv pip install -e '.[test,convert]'\n\n# Set up environment variables\ncp .env.template .env\n# Edit .env and add your HF_TOKEN\n```\n\n### Running Tests\n\n```bash\n# Run unit tests\nuv run pytest tests/test_alphagenome.py -v\n\n# Run regression tests (requires HF access)\nALPHAGENOME_RUN_INTEGRATION_TESTS=1 uv run pytest tests/test_regression.py -v\n\n# Run full integration tests\nALPHAGENOME_RUN_INTEGRATION_TESTS=1 uv run pytest tests/test_integration_jax_torch.py -v\n```\n\n### Regenerating Regression Data\n\nIf you modify the model architecture, regenerate the reference tensors:\n\n```bash\nuv run python tests/generate_regression_tensors.py\n```\n\nThat's it. Vibe coding with some attention network is totally welcomed, if it works\n\n## Citations\n\n```bibtex\n@article {avsec2025alphagenome,\n    title   = {AlphaGenome: advancing regulatory variant effect prediction with a unified DNA sequence model},\n    author  = {Avsec, {\\v Z}iga and Latysheva, Natasha and Cheng, Jun and Novati, Guido and Taylor, Kyle R. and Ward, Tom and Bycroft, Clare and Nicolaisen, Lauren and Arvaniti, Eirini and Pan, Joshua and Thomas, Raina and Dutordoir, Vincent and Perino, Matteo and De, Soham and Karollus, Alexander and Gayoso, Adam and Sargeant, Toby and Mottram, Anne and Wong, Lai Hong and Drot{\\'a}r, Pavol and Kosiorek, Adam and Senior, Andrew and Tanburn, Richard and Applebaum, Taylor and Basu, Souradeep and Hassabis, Demis and Kohli, Pushmeet},\n    elocation-id = {2025.06.25.661532},\n    year    = {2025},\n    doi     = {10.1101/2025.06.25.661532},\n    publisher = {Cold Spring Harbor Laboratory},\n    URL     = {https://www.biorxiv.org/content/early/2025/06/27/2025.06.25.661532},\n    eprint  = {https://www.biorxiv.org/content/early/2025/06/27/2025.06.25.661532.full.pdf},\n    journal = {bioRxiv}\n}\n```\n\n```bibtex\n@misc{gopalakrishnan2025decouplingwhatwherepolar,\n    title   = {Decoupling the \"What\" and \"Where\" With Polar Coordinate Positional Embeddings}, \n    author  = {Anand Gopalakrishnan and Robert Csordás and Jürgen Schmidhuber and Michael C. Mozer},\n    year    = {2025},\n    eprint  = {2509.10534},\n    archivePrefix = {arXiv},\n    primaryClass = {cs.LG},\n    url     = {https://arxiv.org/abs/2509.10534}, \n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flucidrains%2Falphagenome","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flucidrains%2Falphagenome","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flucidrains%2Falphagenome/lists"}