{"id":13870009,"url":"https://github.com/CUAI/Neural-Manifold-Ordinary-Differential-Equations","last_synced_at":"2025-07-15T20:31:01.126Z","repository":{"id":37648536,"uuid":"273325633","full_name":"CUAI/Neural-Manifold-Ordinary-Differential-Equations","owner":"CUAI","description":"[NeurIPS 2020] Neural Manifold Ordinary Differential Equations (https://arxiv.org/abs/2006.10254)","archived":false,"fork":false,"pushed_at":"2023-07-06T22:01:10.000Z","size":23,"stargazers_count":112,"open_issues_count":4,"forks_count":14,"subscribers_count":12,"default_branch":"master","last_synced_at":"2024-08-06T21:22:55.074Z","etag":null,"topics":["generative-models","machine-learning","neural-manifold-odes","neurips-2020"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CUAI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-06-18T19:41:41.000Z","updated_at":"2024-07-23T13:14:32.000Z","dependencies_parsed_at":"2022-09-04T15:12:15.003Z","dependency_job_id":null,"html_url":"https://github.com/CUAI/Neural-Manifold-Ordinary-Differential-Equations","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CUAI%2FNeural-Manifold-Ordinary-Differential-Equations","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CUAI%2FNeural-Manifold-Ordinary-Differential-Equations/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CUAI%2FNeural-Manifold-Ordinary-Differential-Equations/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CUAI%2FNeural-Manifold-Ordinary-Differential-Equations/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CUAI","download_url":"https://codeload.github.com/CUAI/Neural-Manifold-Ordinary-Differential-Equations/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":226068136,"owners_count":17568703,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["generative-models","machine-learning","neural-manifold-odes","neurips-2020"],"created_at":"2024-08-05T20:01:25.387Z","updated_at":"2024-11-23T16:30:56.630Z","avatar_url":"https://github.com/CUAI.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Neural Manifold Ordinary Differential Equations (ODEs)\n\nWe provide the code for [Neural Manifold ODEs](https://arxiv.org/abs/2006.10254) in this repository.\n\nSummary: We introduce Neural Manifold Ordinary Differential Equations, a manifold generalization of Neural ODEs, and construct Manifold Continuous Normalizing Flows (MCNFs). MCNFs require only local geometry (therefore generalizing to arbitrary manifolds) and compute probabilities with continuous change of variables (allowing for a simple and expressive flow construction). We find that leveraging continuous manifold dynamics produces a marked improvement for both density estimation and downstream tasks.\n\nThe multi-chart method from our paper (allowing generality) is showcased in the below figure.\n\n![Multi-chart approach](https://i.imgur.com/TuTFi2n.png)\n\nExample learned densities, together with baselines, are given below.\n\nHyperboloid             |  Sphere\n:-------------------------:|:-------------------------:\n![H^2](https://i.imgur.com/xcbMjnK.png)| ![S^2](https://i.imgur.com/JyQYdiL.png)\n\nBelow we have visualized how our Neural Manifold ODEs learn the `5gaussians` and `bigcheckerboard` densities on the hyperboloid (second and third rows in the hyperboloid figure above), as well as the `4wrapped` and `bigcheckerboard` densities on the sphere (second and third rows in the sphere figure above).\n\nHyperboloid Gaussians             |  Hyperboloid Checkerboard  | Sphere Wrapped Normals | Sphere Checkerboard\n:-------------------------:|:-------------------------:|:-------------------------:|:-------------------------:\n![H^2 Gaussians](https://media3.giphy.com/media/cItuTOsJxEleHPRL9v/giphy.gif)| ![H^2 Checkerboard](https://media1.giphy.com/media/j3oBJD5inJMSQq2ntI/giphy.gif)| ![S^2 Wrapped Normals](https://media3.giphy.com/media/jmw6pmg9gqClEo7GgY/giphy.gif)| ![S^2 Checkerboard](https://media3.giphy.com/media/ehJlhYPSNHjpEJWE9K/giphy.gif)\n\n\u003c!---hyperboloid gaussians https://media3.giphy.com/media/MF7pSvnQPKmQILKL2V/giphy.gif; hyperboloid checkerboard https://media3.giphy.com/media/cLZ7ImIllgNBvIEftp/giphy.gif ---\u003e\n\n## Software Requirements\nThis codebase requires Python 3, PyTorch 1.5+, torchdiffeq ([repo here](https://github.com/rtqichen/torchdiffeq), installed via `pip install torchdiffeq`), and Cartopy 0.18 (most easily installed via conda ```conda install -c conda-forge cartopy```).\n\n## Usage\n\n### Demo\n\nThe following command learns a generative flow model for the `5gaussians` density on the hyperboloid:\n\n```\npython main_density.py --epochs 300 --dev cuda --lr 1e-2 --dataset 5gaussians --M Hyperboloid --save\n```\n\nNote that only 200 samples are used per epoch. 5000 epochs were used for the results in the paper. The learned density on the two dimensional hyperboloid is given below (visualized on the Poincaré ball), in comparison with the ground truth density:\n\nGround Truth             |  Ours (100 epochs)         |    Ours (300 epochs)\n:-------------------------:|:-------------------------:|:---------------------:\n![H^2 GT](https://i.imgur.com/82Dsn4N.png)| ![H^2 100 epochs](https://i.imgur.com/AF3pPVd.png) | ![H^2 300 epochs](https://i.imgur.com/YmaMKHf.png)\n\nObserve that even after 300 epochs (only 60,000 samples), our model approaches the groundtruth. In general, we found that our method was substantially more sample efficient than the baselines (by nearly an order of magnitude). \n\n### Full Usage\n\nAll options are given below:\n\n```\nusage: main_density.py [-h] [--lr LR] [--weight_decay WEIGHT_DECAY]\n                       [--dataset DATASET] [--epochs EPOCHS]\n                       [--batch_size BATCH_SIZE] [--num_drops NUM_DROPS]\n                       [--flow_hidden_size FLOW_HIDDEN_SIZE] [--save]\n                       [--conc CONC] [--dev DEV] [--M {Hyperboloid,Sphere}]\n                       [--contsave] [--save_freq SAVE_FREQ]\n\noptional arguments:\n  -h, --help            show this help message and exit\n  --lr LR\n  --weight_decay WEIGHT_DECAY\n  --dataset DATASET     for hyperboloid: 1wrapped | 5gaussians |\n                        bigcheckerboard | mult_wrapped, for the sphere:\n                        1wrapped | 4wrapped | bigcheckerboard\n  --epochs EPOCHS\n  --batch_size BATCH_SIZE\n  --num_drops NUM_DROPS\n                        number of times to drop the learning rate\n  --flow_hidden_size FLOW_HIDDEN_SIZE\n                        Hidden layer size for flows.\n  --save                Save a visualization of the learned density\n  --conc CONC           Concentration of vMF\n  --dev DEV\n  --M {Hyperboloid,Sphere}\n  --contsave            Continuously save intermediate flow visualization in\n                        contsave/\n  --save_freq SAVE_FREQ\n                        frequency of continuous saving of intermediate flows\n```\n\n## Attribution\n\nIf you use this code or our results in your research, please cite:\n\n```\n@misc{lou2020neural,\n    title={Neural Manifold Ordinary Differential Equations},\n    author={Aaron Lou and Derek Lim and Isay Katsman and Leo Huang and Qingxuan Jiang and Ser-Nam Lim and Christopher De Sa},\n    year={2020},\n    eprint={2006.10254},\n    archivePrefix={arXiv},\n    primaryClass={stat.ML}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FCUAI%2FNeural-Manifold-Ordinary-Differential-Equations","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FCUAI%2FNeural-Manifold-Ordinary-Differential-Equations","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FCUAI%2FNeural-Manifold-Ordinary-Differential-Equations/lists"}