{"id":19603888,"url":"https://github.com/divelab/mfa","last_synced_at":"2025-04-27T19:32:27.653Z","repository":{"id":241385021,"uuid":"805611715","full_name":"divelab/MFA","owner":"divelab","description":"Official code repository of paper Equivariance via Minimal Frame Averaging for More Symmetries and Efficiency.","archived":false,"fork":false,"pushed_at":"2025-01-18T03:00:38.000Z","size":196892,"stargazers_count":17,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-05T02:21:42.445Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/divelab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-25T01:19:40.000Z","updated_at":"2025-03-18T06:45:23.000Z","dependencies_parsed_at":"2024-05-28T04:23:25.529Z","dependency_job_id":"52de1c93-016e-4e03-b9a5-4bb09ad26c84","html_url":"https://github.com/divelab/MFA","commit_stats":null,"previous_names":["divelab/mfa"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/divelab%2FMFA","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/divelab%2FMFA/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/divelab%2FMFA/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/divelab%2FMFA/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/divelab","download_url":"https://codeload.github.com/divelab/MFA/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251195919,"owners_count":21550869,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T09:33:26.519Z","updated_at":"2025-04-27T19:32:27.361Z","avatar_url":"https://github.com/divelab.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Minimal Frame Averaging\r\n\r\n## Overview\r\n\r\nOfficial code repository of paper [Equivariance via Minimal Frame Averaging for More Symmetries and Efficiency](https://openreview.net/pdf?id=guFsTBXsov). In this repository, we have provided decorators to convert any non-equivariant/invariant neural network functions into group equivariant/invariant ones. This enables neural networks to handle transformations from various groups such as $O(d), SO(d), O(1,d-1)$, etc.\r\n\r\nIn this code base, we provide the equivariance error test for all the groups included in the paper, which suffices to show our idea and corresponding algorithms.\r\n\r\n\r\n\r\n\r\n## Installation\r\n\r\n**Note**: It is recommended to use a `conda` environment. First, you can install `torch` with the following command (if GPU is available):\r\n```sh\r\nconda install pytorch pytorch-cuda=11.8 -c pytorch -c nvidia\r\n```\r\n\r\nThen, install our package using:\r\n\r\n```sh\r\npip install -e .\r\n```\r\n\r\n\r\n## Usage\r\n\r\n### Example: $O(d)$-Equivariant/Invariant Decorator\r\n\r\nThe `od_equivariant_decorator` can be used to wrap any forward function to make it $O(d)$​-equivariant. An example model can be\r\n\r\n```python\r\nimport torch.nn as nn\r\n\r\nclass MyModel(nn.Module):\r\n    def __init__(self):\r\n        super(MyModel, self).__init__()\r\n        # Define your layers here\r\n\r\n    def forward(self, x):\r\n        # Your forward pass implementation\r\n        return x\r\n```\r\n\r\nTo apply this decorator to this neural network class’s forward function:\r\n\r\n```python\r\nimport torch.nn as nn\r\nfrom minimal_frame.group_decorator import od_equivariant_decorator\r\n\r\nclass MyModel(nn.Module):\r\n    def __init__(self):\r\n        super(MyModel, self).__init__()\r\n        # Define your layers here\r\n        \r\n    @od_equivariant_decorator\r\n    def forward(self, x):\r\n        # Your forward pass implementation\r\n        return x\r\n```\r\n\r\nTo apply this decorator to a neural network instance's forward function:\r\n\r\n```python\r\n# Instantiate your model\r\nmodel = MyModel()\r\n\r\n# Apply the O(d)-equivariant decorator\r\nmodel.forward = od_equivariant_decorator(model.forward)\r\n```\r\n\r\nSimilarly, the `od_invariant_decorator` can be used to make the network $O(d)$-invariant.\r\n\r\n\r\n\r\n## Supported Groups\r\n\r\nThis repository provides a collection of decorators to enforce equivariance or invariance properties in neural network models. Equivariance ensures that the output of the model transforms in the same way as the input under a given group of transformations. Invariance ensures that the output remains unchanged under such transformations.\r\n\r\nThese decorators in `group_decorator.py` are model-agnostic and can be applied to any model with input and output shapes of $n\\times d$ for equivariance or with input shape of $n\\times d$ and output shape of $1$ for invariance. A comprehensive equivariance analysis over various groups can be found [here](https://github.com/divelab/MFA/blob/main/tests/equivariance_test.ipynb). The groups and MFA decorators (if not specified) include \r\n\r\n- Orthogonal Group $O(d)$:\r\n  - `od_equivariant_decorator/od_invariant_decorator`: Our MFA method for $O(d)$\r\n  - `od_equivariant_puny_decorator/od_invariant_puny_decorator`: Puny's FA method [1] for $O(d)$\r\n  - `od_equivariant_puny_improve_decorator/od_invariant_puny_improve_decorator`: Improved [1]'s FA for degenerate eigenvalues, see our Appendix H.3\r\n  - `od_equivariant_sfa_decorator/od_invariant_sfa_decorator`: Stochastic FA method [2]\r\n- Special Orthogonal Group $SO(d)$: `sod_equivariant_decorator/sod_invariant_decorator`\r\n- Euclidean Group $E(d)$: `ed_equivariant_decorator/ed_invariant_decorator`\r\n- Special Euclidean Group $SE(d)$: `sed_equivariant_decorator/sed_invariant_decorator`\r\n- Unitary Group $U(d)$: `ud_equivariant_decorator/ud_invariant_decorator`\r\n- Special Unitary Group $SU(d)$: `sud_equivariant_decorator/sud_invariant_decorator`\r\n- Lorentz Group $O(1,d-1)$: `o1d_equivariant_decorator/o1d_invariant_decorator`\r\n- Proper Lorentz Group $SO(1,d-1)$: `so1d_equivariant_decorator/so1d_invariant_decorator`\r\n- General Linear Group $GL(d,\\mathbb{R})$ (_requiring full-column-rank input_): `gld_equivariant_decorator/gld_invariant_decorator`\r\n- Special Linear Group $SL(d,\\mathbb{R})$ (_requiring full-column-rank input_): `sld_equivariant_decorator/sld_invariant_decorator`\r\n- Affine Group $Aff(d,\\mathbb{R})$ (_requiring full-column-rank input_): `affd_equivariant_decorator/affd_invariant_decorator`\r\n- Permutation Group $S_n$: `sn_equivariant_decorator/sn_invariant_decorator`\r\n- Direct product between Permutation Group $S_n$ and other groups\r\n  - $S_n \\times O(d)$: `sn_od_equivariant_decorator/sn_od_invariant_decorator`\r\n  - $S_n \\times SO(d)$: `sn_sod_equivariant_decorator/sn_sod_invariant_decorator`\r\n  - $S_n \\times O(1,d-1)$: `sn_o1d_equivariant_decorator/sn_o1d_invariant_decorator`\r\n\r\nAdditionally, the $S_n$-equivariant/invariant decorators in `graph_group_decorator.py` for undirected graphs are provided for any model with undirected adjacency matrices as input. The corresponding equivariance analysis can be found [here](https://github.com/divelab/MFA/blob/main/tests/graph_equivariance_test.ipynb), including decorators for\r\n\r\n- Undirected unweighted adjacency matrix: `undirected_unweighted_sn_equivariant_decorator/undirected_unweighted_sn_invariant_decorator`\r\n- Undirected weighted adjacency matrix: `undirected_weighted_sn_equivariant_decorator/undirected_weighted_sn_invariant_decorator`\r\n\r\n\r\n\r\n\r\n\r\n## References\r\n\r\n[1] Puny, Omri, et al. \"Frame averaging for invariant and equivariant network design.\" *arXiv preprint arXiv:2110.03336* (2021).\r\n\r\n[2] Duval, Alexandre Agm, et al. \"Faenet: Frame averaging equivariant gnn for materials modeling.\" *International Conference on Machine Learning*. PMLR, 2023.\r\n\r\n[3] McKay, Brendan D. \"Practical graph isomorphism.\" (1981): 45-87.\r\n\r\n\r\n\r\n## Licence\r\n\r\nThis project is licensed under the MIT License. Please note that the `nauty` software is covered by its own licensing terms. All rights to the files mentioned in `nauty.py` are reserved by Brendan McKay under the Apache License 2.0.\r\n\r\n\r\n## Acknowledgements\r\n\r\nWe gratefully acknowledge the insightful discussions with Derek Lim and Hannah Lawrence. This work was supported in part by National Science Foundation grant IIS-2006861 and National Institutes of Health grant U01AG070112.\r\n\r\n## Citation\r\n\r\nIf you find this work useful, please consider citing:\r\n\r\n```bib\r\n@inproceedings{\r\nlin2024equivariance,\r\ntitle={Equivariance via Minimal Frame Averaging for More Symmetries and Efficiency},\r\nauthor={Yuchao Lin and Jacob Helwig and Shurui Gui and Shuiwang Ji},\r\nbooktitle={Forty-first International Conference on Machine Learning},\r\nyear={2024},\r\nurl={https://openreview.net/forum?id=guFsTBXsov}\r\n}\r\n```\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdivelab%2Fmfa","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdivelab%2Fmfa","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdivelab%2Fmfa/lists"}