{"id":27979235,"url":"https://github.com/ml-jku/lam-slide","last_synced_at":"2025-05-08T02:32:30.845Z","repository":{"id":275755030,"uuid":"926787751","full_name":"ml-jku/LaM-SLidE","owner":"ml-jku","description":"Code for the paper LaM-SLidE - Latent Space Modeling of Spatial Dynamical Systems via Linked Entities","archived":false,"fork":false,"pushed_at":"2025-04-15T05:02:38.000Z","size":28411,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-15T06:19:50.285Z","etag":null,"topics":["dynamical-systems","flow-matching","latent-diffusion","latent-space","molecular-dynamics","molecular-dynamics-simulation","simulation"],"latest_commit_sha":null,"homepage":"https://ml-jku.github.io/LaM-SLidE/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ml-jku.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-02-03T21:21:01.000Z","updated_at":"2025-04-11T19:21:30.000Z","dependencies_parsed_at":"2025-02-04T12:33:00.373Z","dependency_job_id":"85d2c74e-7ea8-4c39-83ef-877440d52139","html_url":"https://github.com/ml-jku/LaM-SLidE","commit_stats":null,"previous_names":["ml-jku/lam-slide"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ml-jku%2FLaM-SLidE","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ml-jku%2FLaM-SLidE/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ml-jku%2FLaM-SLidE/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ml-jku%2FLaM-SLidE/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ml-jku","download_url":"https://codeload.github.com/ml-jku/LaM-SLidE/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252986885,"owners_count":21836248,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dynamical-systems","flow-matching","latent-diffusion","latent-space","molecular-dynamics","molecular-dynamics-simulation","simulation"],"created_at":"2025-05-08T02:31:57.192Z","updated_at":"2025-05-08T02:32:30.829Z","avatar_url":"https://github.com/ml-jku.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n# LaM-SLidE ({La}tent Space {M}odeling of Spatial Dynamical {S}ystems via {Li}nke{d} {E}ntities)\n\n\u003ca href=\"https://github.com/ashleve/lightning-hydra-template\"\u003e\u003cimg alt=\"Template\" src=\"https://img.shields.io/badge/-Lightning--Hydra--Template-017F2F?style=flat\u0026logo=github\u0026labelColor=gray\"\u003e\u003c/a\u003e\n\n\u003c/div\u003e\n\n\u003ccenter\u003e\n\n[![python](https://img.shields.io/badge/-Python_3.11-blue?logo=python\u0026logoColor=white)](https://www.python.org/downloads/release/python-3110/)\n[![pytorch](https://img.shields.io/badge/PyTorch_2.5-ee4c2c?logo=pytorch\u0026logoColor=white)](https://pytorch.org/docs/2.5/)\n[![lightning](https://img.shields.io/badge/-Lightning_2.4-792ee5?logo=pytorchlightning\u0026logoColor=white)](https://lightning.ai/docs/pytorch/stable/)\n[![hydra](https://img.shields.io/badge/Config-Hydra_1.3-89b8cd)](https://hydra.cc/)\n\n\u003c/center\u003e\n\n[**Project Page**](https://ml-jku.github.io/LaM-SLidE/) | [**Paper**](https://arxiv.org/abs/2502.12128/)\n\n\nImplementation of **LaM-SLidE** (Latent Space Modeling of Spatial Dynamical Systems via Linked Entities).\n\n**Note:** This repository is provided for research reproducibility only and is not intended for usage in application workflows.\n\n## News\n🔥***February 18, 2025***: *The training code and paper preprint are released.*\n\n## Setup\n\n### Installation\n\n```shell\nmamba env create -f environment.yaml\nmamba activate pyt25\n```\n\n### Env variables\n\nCreate an `.env` file and set the parameters for logging with [wandb](https://wandb.ai/). An example can be found [here](.env.example).\n\n## Setup\n\n### Data\n\nThe data for all experiment will be located in the `data` directory.\n\n```shell\nmkdir data\n```\n\n### Workflow\n\nBecause our methods reilies on two stage approach:\n\n1. First stage encoder/decoder\n2. Second stage latent model\n\nwe retrieve [wandb](\u003c%5Bwandb%5D(www.https://wandb.ai)\u003e) first stage model information direclty form the api, this simplyfies the workflow a lot, and for the second stage training we only have to provide the RunID of the first stage to the second stage training.\n\n## Experiments\n\n### MD17\n\n#### Data Preparation\n\nDownload the MD17 dataset in `.npz` format from [here](http://www.sgdml.org/#datasets). The dataset should be placed in `data/md17`.\n\n#### Training\n\n```python\n# First stage (Encoder-Decoder)\npython experiment=md17/first-stage\n\n# Second stage (Diffusion)\npython experiment=md17/second-stage first_stage_settings.run_id=[WB_RUN_ID] first_stage_settings.project=[WB_PROJECT]\n```\n\n### Pedestrian\n\n#### Data Preparation\n\nFollow the instructions [here](https://github.com/MediaBrain-SJTU/EqMotion?tab=readme-ov-file#data-preparation-3) to download and preprocess the data.\nThen move the preprocessed files in the folder `processed_data_diverse` into `data/pedestrian_eqmotion`.\n\n#### Training\n\n```python\n# First stage (Encoder-Decoder)\npython experiment=pedestrian/first-stage\n\n# Second stage (Diffusion)\npython experiment=pedestrian/second-stage first_stage_settings.run_id=[WB_RUN_ID] first_stage_settings.project=[WB_PROJECT]\n```\n\n### NBA\n\n#### Data preparation\n\nDownload the data from [here](https://github.com/xupei0610/SocialVAE/tree/main/data/nba)\n\nProcess the data with following commands.\n\n```python\n# Train\npython scripts/nba/process_4AA.py --data_dir data/social_vae_data/nba/score/train\npython scripts/nba/process_4AA.py --data_dir data/social_vae_data/nba/rebound/train\n\n# Val\npython scripts/nba/process_data.py --data_dir data/social_vae_data/nba/score/val\npython scripts/nba/process_4AA.py --data_dir data/social_vae_data/nba/rebound/val\n\n```\n\n#### Training\n\n```python\n# First stage (Encoder-Decoder)\npython experiment=nba/first-stage\n\n# Second stage (Diffusion)\npython experiment=nba/second-stage first_stage_settings.run_id=[WB_RUN_ID] first_stage_settings.project=[WB_PROJECT]\n```\n\n### Tetrapeptide - 4AA\n\nFollow the instructions [here](https://github.com/bjing2016/mdgen) to download the data.\n\n#### Data preparation\n\nProcess the data with the following commands.\n\n```python\n# Train\npython scripts/peptide/process_4AA.py --split data/mdgen/splits/4AA_train.csv --outdir data/mdgen/4AA_sims_processed/train --sim_dir data/mdgen/4AA_sims\n\n# Val\npython scripts/peptide/process_4AA.py --split data/mdgen/splits/4AA_val.csv --outdir data/mdgen/4AA_sims_processed/val --sim_dir data/mdgen/4AA_sims\n\n# Test\npython scripts/peptide/process_4AA.py --split data/mdgen/splits/4AA_test.csv --outdir data/mdgen/4AA_sims_processed/test --sim_dir data/mdgen/4AA_sims\n```\n\n#### Training\n\n```python\n# First stage (Encoder-Decoder)\npython experiment=peptide/first-stage\n\n# Second stage (Diffusion)\npython experiment=peptide/second-stage first_stage_settings.run_id=[WB_RUN_ID] first_stage_settings.project=[WB_PROJECT]\n```\n\n# Acknowledgments\n\nOur source code was inpired by previous work:\n\n- [mdgen](https://github.com/bjing2016/mdgen) - Latent space conditioning/masking.\n- [flux](https://github.com/black-forest-labs/flux) - Latent space model architecture.\n- [SiT](https://github.com/willisma/SiT) - Stochastic interpolants framework.\n- [UPT](https://github.com/ml-jku/UPT/) - Encoder - decoder architecture.\n\n# Citation\n\nIf you like our work, please consider giving it a star 🌟 and cite us\n\n```\n@misc{sestak2025lamslidelatentspacemodeling,\n      title={LaM-SLidE: Latent Space Modeling of Spatial Dynamical Systems via Linked Entities}, \n      author={Florian Sestak and Artur Toshev and Andreas Fürst and Günter Klambauer and Andreas Mayr and Johannes Brandstetter},\n      year={2025},\n      eprint={2502.12128},\n      archivePrefix={arXiv},\n      primaryClass={cs.LG},\n      url={https://arxiv.org/abs/2502.12128}, \n}\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fml-jku%2Flam-slide","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fml-jku%2Flam-slide","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fml-jku%2Flam-slide/lists"}