{"id":13478329,"url":"https://github.com/atomicarchitects/equiformer_v2","last_synced_at":"2026-01-17T12:25:04.738Z","repository":{"id":177117724,"uuid":"656550520","full_name":"atomicarchitects/equiformer_v2","owner":"atomicarchitects","description":"[ICLR 2024] EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations","archived":false,"fork":false,"pushed_at":"2025-02-11T15:58:33.000Z","size":8009,"stargazers_count":287,"open_issues_count":17,"forks_count":39,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-09-09T12:44:40.688Z","etag":null,"topics":["ai-for-science","catalyst-design","computational-chemistry","computational-physics","deep-learning","drug-discovery","e3nn","equivariant-graph-neural-network","equivariant-neural-networks","force-fields","geometric-deep-learning","graph-neural-networks","interatomic-potentials","machine-learning","materials-science","molecular-dynamics","pytorch"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2306.12059","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/atomicarchitects.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-21T07:09:58.000Z","updated_at":"2025-09-03T16:23:54.000Z","dependencies_parsed_at":null,"dependency_job_id":"30cc1a5d-d217-44b6-b22d-03e4f27389a9","html_url":"https://github.com/atomicarchitects/equiformer_v2","commit_stats":null,"previous_names":["atomicarchitects/equiformer_v2"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/atomicarchitects/equiformer_v2","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/atomicarchitects%2Fequiformer_v2","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/atomicarchitects%2Fequiformer_v2/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/atomicarchitects%2Fequiformer_v2/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/atomicarchitects%2Fequiformer_v2/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/atomicarchitects","download_url":"https://codeload.github.com/atomicarchitects/equiformer_v2/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/atomicarchitects%2Fequiformer_v2/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28508464,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-17T11:50:55.898Z","status":"ssl_error","status_checked_at":"2026-01-17T11:50:55.569Z","response_time":85,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-for-science","catalyst-design","computational-chemistry","computational-physics","deep-learning","drug-discovery","e3nn","equivariant-graph-neural-network","equivariant-neural-networks","force-fields","geometric-deep-learning","graph-neural-networks","interatomic-potentials","machine-learning","materials-science","molecular-dynamics","pytorch"],"created_at":"2024-07-31T16:01:55.594Z","updated_at":"2026-01-17T12:25:04.716Z","avatar_url":"https://github.com/atomicarchitects.png","language":"Python","funding_links":[],"categories":["Python","Interatomic Potentials (ML-IAP)","🔬 Domain-Specific Applications"],"sub_categories":["🌌 Physics \u0026 Astronomy"],"readme":"# EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations\n\n**[Paper](https://arxiv.org/abs/2306.12059)** | **[OpenReview](https://openreview.net/forum?id=mCOBKZmrzD)** | **[Poster](docs/equiformer_v2_poster.pdf)**\n\nThis repository contains the official PyTorch implementation of the work \"EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations\" (ICLR 2024).\nWe provide the code for training the base model setting on the OC20 S2EF-2M and S2EF-All+MD datasets.\n\nAdditionally, EquiformerV2 has been incorporated into [OCP repository](https://github.com/FAIR-Chem/fairchem/tree/main/src/fairchem/core/models/equiformer_v2) and used in [Open Catalyst demo](https://open-catalyst.metademolab.com/).\n\nIn our subsequent [work](https://arxiv.org/abs/2403.09549), we find that we can generalize self-supervised learning similar to BERT, which we call **DeNS** (**De**noising **N**on-Equilibrium **S**tructures), to 3D atomistic systems to improve the performance of EquiformerV2 on energy and force predictions. \nPlease refer to the [paper](https://arxiv.org/abs/2403.09549) and the [code](https://github.com/atomicarchitects/DeNS) for further details.\n\n\n\u003cp align=\"center\"\u003e\n\t\u003cimg src=\"fig/equiformer_v2.png\" alt=\"photo not available\" width=\"98%\" height=\"98%\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n\t\u003cimg src=\"fig/equiformer_v2_speed_accuracy_tradeoffs.png\" alt=\"photo not available\" width=\"98%\" height=\"98%\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n\t\u003cimg src=\"fig/equiformer_v2_oc20_results.png\" alt=\"photo not available\" width=\"98%\" height=\"98%\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n\t\u003cimg src=\"fig/equiformer_v2_adsorbml_results.png\" alt=\"photo not available\" width=\"98%\" height=\"98%\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n\t\u003cimg src=\"fig/equiformer_v2_oc22_results.png\" alt=\"photo not available\" width=\"98%\" height=\"98%\"\u003e\n\u003c/p\u003e\n\n\n## Content ##\n0. [Environment Setup](#environment-setup)\n0. [Changelog](#changelog)\n0. [Training](#training)\n0. [File Structure](#file-structure)\n0. [Checkpoints](#checkpoints)\n0. [Citation](#citation)\n0. [Acknowledgement](#acknowledgement)\n\n\n\n## Environment Setup ##\n\n\n### Environment \n\nSee [here](docs/env_setup.md) for setting up the environment.\n\n\n### OC20\n\nThe OC20 S2EF dataset can be downloaded by following instructions in their [GitHub repository](https://github.com/Open-Catalyst-Project/ocp/blob/5a7738f9aa80b1a9a7e0ca15e33938b4d2557edd/DATASET.md#download-and-preprocess-the-dataset).\n\nFor example, we can download the OC20 S2EF-2M dataset by running:\n```\n    cd ocp\n    python scripts/download_data.py --task s2ef --split \"2M\" --num-workers 8 --ref-energy\n```\nWe also need to download the `\"val_id\"` data split to run training.\n\nAfter downloading, place the datasets under `datasets/oc20/` by using `ln -s`:\n```\n    cd datasets\n    mkdir oc20\n    cd oc20\n    ln -s ~/ocp/data/s2ef s2ef\n```\n\nTo train on different splits like All and All+MD, we can follow the same link above to download the datasets.\n\n\n## Changelog ##\n\n\nPlease refer to [here](docs/changelog.md).\n\n\n## Training ##\n\n\n### OC20\n\n1. We train EquiformerV2 on the OC20 **S2EF-2M** dataset by running:\n    \n    ```bash\n        sh scripts/train/oc20/s2ef/equiformer_v2/equiformer_v2_N@12_L@6_M@2_splits@2M_g@multi-nodes.sh\n    ```\n    The above script uses 2 nodes with 8 GPUs on each node.\n    \n    If there is an import error, it is possible that [`ocp/ocpmodels/common/utils.py`](https://github.com/Open-Catalyst-Project/ocp/blob/5a7738f9aa80b1a9a7e0ca15e33938b4d2557edd/ocpmodels/common/utils.py#L329) is not modified. \n    Please follow [here](docs/env_setup.md) for details.\n\n    We can also run training on 8 GPUs on 1 node:\n    ```bash\n        sh scripts/train/oc20/s2ef/equiformer_v2/equiformer_v2_N@12_L@6_M@2_splits@2M_g@8.sh\n    ```\n\n2. We train **EquiformerV2 (153M)** on OC20 **S2EF-All+MD** by running:\n    ```bash\n        sh scripts/train/oc20/s2ef/equiformer_v2/equiformer_v2_N@20_L@6_M@3_splits@all+md_g@multi-nodes.sh\n    ```\n    The above script uses 16 nodes with 8 GPUs on each node.\n\n3. We train **EquiformerV2 (31M)** on OC20 **S2EF-All+MD** by running:\n    ```bash\n        sh scripts/train/oc20/s2ef/equiformer_v2/equiformer_v2_N@8_L@4_M@2_splits@all+md_g@multi-nodes.sh\n    ```\n    The above script uses 8 nodes with 8 GPUs on each node.\n    \n4. We can train EquiformerV2 with **DeNS** (**De**noising **N**on-Equilibrium **S**tructures) as an auxiliary task to further improve the performance on energy and force predictions. Please refer to the [code](https://github.com/atomicarchitects/DeNS) for details.\n\n\n## File Structure ##\n\n1. [`nets`](nets) includes code of different network architectures for OC20.\n2. [`scripts`](scripts) includes scripts for training models on OC20.\n3. [`main_oc20.py`](main_oc20.py) is the code for training, evaluating and running relaxation.\n4. [`oc20/trainer`](oc20/trainer) contains code for the force trainer as well as some utility functions.\n5. [`oc20/configs`](oc20/configs) contains config files for S2EF.\n\n\n## Checkpoints ##\n\nWe provide the checkpoints of EquiformerV2 trained on S2EF-2M dataset for 30 epochs, EquiformerV2 (31M) trained on S2EF-All+MD, and EquiformerV2 (153M) trained on S2EF-All+MD.\n|Model\t|Split\t|Download\t|val force MAE (meV / Å) |val energy MAE (meV) |\n|---\t|---\t|---\t|---\t|---\t| \n|EquiformerV2\t|2M\t|[checkpoint](https://dl.fbaipublicfiles.com/opencatalystproject/models/2023_06/oc20/s2ef/eq2_83M_2M.pt) \\| [config](oc20/configs/s2ef/2M/equiformer_v2/equiformer_v2_N@12_L@6_M@2_epochs@30.yml)\t|19.4 | 278 |\n|EquiformerV2 (31M)|All+MD |[checkpoint](https://dl.fbaipublicfiles.com/opencatalystproject/models/2023_06/oc20/s2ef/eq2_31M_ec4_allmd.pt) \\| [config](oc20/configs/s2ef/all_md/equiformer_v2/equiformer_v2_N@8_L@4_M@2_31M.yml) |16.3 | 232 |\n|EquiformerV2 (153M) |All+MD | [checkpoint](https://dl.fbaipublicfiles.com/opencatalystproject/models/2023_06/oc20/s2ef/eq2_153M_ec4_allmd.pt) \\| [config](oc20/configs/s2ef/all_md/equiformer_v2/equiformer_v2_N@20_L@6_M@3_153M.yml) |15.0 | 227 |\n\n\n## Citation ##\n\nPlease consider citing the works below if this repository is helpful:\n\n- [EquiformerV2](https://arxiv.org/abs/2306.12059):\n    ```bibtex\n    @inproceedings{\n        equiformer_v2,\n        title={{EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations}}, \n        author={Yi-Lun Liao and Brandon Wood and Abhishek Das* and Tess Smidt*},\n        booktitle={International Conference on Learning Representations (ICLR)},\n        year={2024},\n        url={https://openreview.net/forum?id=mCOBKZmrzD}\n    }\n    ```\n\n- [eSCN](https://arxiv.org/abs/2302.03655):\n    ```bibtex\n    @inproceedings{\n        escn,\n        title={{Reducing SO(3) Convolutions to SO(2) for Efficient Equivariant GNNs}},\n        author={Passaro, Saro and Zitnick, C Lawrence},\n        booktitle={International Conference on Machine Learning (ICML)},\n        year={2023}\n    }\n    ```\n\n- [Equiformer](https://arxiv.org/abs/2206.11990):\n    ```bibtex\n    @inproceedings{\n        equiformer,\n        title={{Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs}},\n        author={Yi-Lun Liao and Tess Smidt},\n        booktitle={International Conference on Learning Representations (ICLR)},\n        year={2023},\n        url={https://openreview.net/forum?id=KwmPfARgOTD}\n    }\n    ```\n\n- [OC20 dataset](https://arxiv.org/abs/2010.09990):\n    ```bibtex\n    @article{\n        oc20,\n        author = {Chanussot*, Lowik and Das*, Abhishek and Goyal*, Siddharth and Lavril*, Thibaut and Shuaibi*, Muhammed and Riviere, Morgane and Tran, Kevin and Heras-Domingo, Javier and Ho, Caleb and Hu, Weihua and Palizhati, Aini and Sriram, Anuroop and Wood, Brandon and Yoon, Junwoong and Parikh, Devi and Zitnick, C. Lawrence and Ulissi, Zachary},\n        title = {{Open Catalyst 2020 (OC20) Dataset and Community Challenges}},\n        journal = {ACS Catalysis},\n        year = {2021},\n        doi = {10.1021/acscatal.0c04525},\n    }\n    ```\n\nPlease direct questions to Yi-Lun Liao (ylliao@mit.edu).\n\n\n## Acknowledgement ##\n\nOur implementation is based on [PyTorch](https://pytorch.org/), [PyG](https://pytorch-geometric.readthedocs.io/en/latest/index.html), [e3nn](https://github.com/e3nn/e3nn), [timm](https://github.com/huggingface/pytorch-image-models), [ocp](https://github.com/Open-Catalyst-Project/ocp), [Equiformer](https://github.com/atomicarchitects/equiformer).","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fatomicarchitects%2Fequiformer_v2","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fatomicarchitects%2Fequiformer_v2","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fatomicarchitects%2Fequiformer_v2/lists"}