https://github.com/atomind-ai/mlip-arena
Fair and transparent benchmark of machine-learned interatomic potentials (MLIPs), beyond basic error metrics
https://github.com/atomind-ai/mlip-arena
benchmark-framework interatomic-potentials machine-learning materials molecules quantum-chemistry
Last synced: about 1 month ago
JSON representation
Fair and transparent benchmark of machine-learned interatomic potentials (MLIPs), beyond basic error metrics
- Host: GitHub
- URL: https://github.com/atomind-ai/mlip-arena
- Owner: atomind-ai
- License: apache-2.0
- Created: 2024-03-24T20:36:55.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2025-03-04T06:30:42.000Z (about 1 month ago)
- Last Synced: 2025-03-04T07:32:05.134Z (about 1 month ago)
- Topics: benchmark-framework, interatomic-potentials, machine-learning, materials, molecules, quantum-chemistry
- Language: Python
- Homepage: https://huggingface.co/spaces/atomind/mlip-arena
- Size: 24.3 MB
- Stars: 56
- Watchers: 0
- Forks: 2
- Open Issues: 11
-
Metadata Files:
- Readme: .github/README.md
- License: LICENSE
- Citation: CITATION.cff
Awesome Lists containing this project
- best-of-atomistic-machine-learning - GitHub - 68% open · ⏱️ 19.03.2025): (Universal Potentials)
README
MLIP Arena
> [!CAUTION]
> MLIP Arena is currently in pre-alpha. The results are not stable. Please intepret them with care.
> [!NOTE]
> Contributions of new tasks are very welcome! If you're interested in joining the effort, please reach out to Yuan at [[email protected]](mailto:[email protected]). See [project page](https://github.com/orgs/atomind-ai/projects/1) for some outstanding tasks, or propose new one in [Discussion](https://github.com/atomind-ai/mlip-arena/discussions/new?category=ideas).
MLIP Arena is a unified platform for evaluating foundation machine learning interatomic potentials (MLIPs) beyond conventional error metrics. It focuses on revealing the physics and chemistry learned by these models and assessing their utilitarian performance agnostic to underlying model architecture. The platform's benchmarks are specifically designed to evaluate the readiness and reliability of open-source, open-weight models in accurately reproducing both qualitative and quantitative behaviors of atomic systems.
MLIP Arena leverages modern pythonic workflow orchestrator [Prefect](https://www.prefect.io/) to enable advanced task/flow chaining and caching.
## Installation
### From PyPI (without model running capability)
```bash
pip install mlip-arena
```
### From source
**Linux**
```bash
git clone https://github.com/atomind-ai/mlip-arena.git
cd mlip-arena
pip install torch==2.2.0
bash scripts/install-pyg.sh
bash scripts/install-dgl.sh
pip install -e .[test]
pip install -e .[mace]
# DeePMD
DP_ENABLE_TENSORFLOW=0 pip install -e .[deepmd]
```
**Mac**
```bash
# (Optional) Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh
source $HOME/.local/bin/env
# One script uv pip installation
bash scripts/install-macosx.sh
```
## Quickstart
### Molecular dynamics (MD)
Arena provides a unified interface to run all the compiled MLIPs. This can be achieved simply by looping through `MLIPEnum`:
```python
from mlip_arena.models import MLIPEnum
from mlip_arena.tasks.md import run as MD
# from mlip_arena.tasks import MD # for convenient import
from mlip_arena.tasks.utils import get_calculator
from ase import units
from ase.build import bulk
atoms = bulk("Cu", "fcc", a=3.6)
results = []
for model in MLIPEnum:
result = MD(
atoms=atoms,
calculator=get_calculator(
model,
calculator_kwargs=dict(), # passing into calculator
dispersion=True,
dispersion_kwargs=dict(damping='bj', xc='pbe', cutoff=40.0 * units.Bohr), # passing into TorchDFTD3Calculator
),
ensemble="nve",
dynamics="velocityverlet",
total_time=1e3, # 1 ps = 1e3 fs
time_step=2, # fs
)
results.append(result)
```
## Contribute
MLIP Arena is now in pre-alpha. If you're interested in joining the effort, please reach out to Yuan at [[email protected]](mailto:[email protected]).
### Development
```
git lfs fetch --all
git lfs pull
streamlit run serve/app.py
```
### Add new benchmark tasks (WIP)
> [!NOTE]
> Please reuse or extend the general tasks defined as Prefect / [Atomate2](https://github.com/materialsproject/atomate2) / [Quacc](https://github.com/Quantum-Accelerators/quacc) workflow.
> The following are some tasks implemented:
> - [Prefect structure optimization (OPT)](../mlip_arena/tasks/optimize.py)
> - [Prefect molecular dynamics (MD)](../mlip_arena/tasks/md.py)
> - [Prefect equation of states (EOS)](../mlip_arena/tasks/eos.py)
### Add new MLIP models
If you have pretrained MLIP models that you would like to contribute to the MLIP Arena and show benchmark in real-time, there are two ways:
#### External ASE Calculator (easy)
1. Implement new ASE Calculator class in [mlip_arena/models/externals](../mlip_arena/models/externals).
2. Name your class with awesome model name and add the same name to [registry](../mlip_arena/models/registry.yaml) with metadata.
> [!CAUTION]
> Remove unneccessary outputs under `results` class attributes to avoid error for MD simulations. Please refer to other class definition for example.
#### Hugging Face Model (recommended, difficult)
0. Inherit Hugging Face [ModelHubMixin](https://huggingface.co/docs/huggingface_hub/en/package_reference/mixins) class to your awesome model class definition. We recommend [PytorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/en/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin).
1. Create a new [Hugging Face Model](https://huggingface.co/new) repository and upload the model file using [push_to_hub function](https://huggingface.co/docs/huggingface_hub/en/package_reference/mixins#huggingface_hub.ModelHubMixin.push_to_hub).
2. Follow the template to code the I/O interface for your model [here](../mlip_arena/models/README.md).
3. Update model [registry](../mlip_arena/models/registry.yaml) with metadata