Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/MartinuzziFrancesco/awesome-scientific-machine-learning

A curated list of awesome Scientific Machine Learning (SciML) papers, resources and software
https://github.com/MartinuzziFrancesco/awesome-scientific-machine-learning

List: awesome-scientific-machine-learning

differential-equations neural-differential-equations physics-informed-learning physics-informed-neural-networks pinn scientific-machine-learning sciml

Last synced: 3 months ago
JSON representation

A curated list of awesome Scientific Machine Learning (SciML) papers, resources and software

Awesome Lists containing this project

README 

        
# Awesome Scientific Machine Learning

A curated list of awesome Scientific Machine Learning (SciML) papers, resources and software



## Contents

- [Papers](#papers)

  - [Neural Differential Equations](#papers-neural-des)

  - [Physics Informed NNs](#papers-physics-informed-nns)

  - [Neural Operators](#papers-neural-operators)

  - [Model Discovery](#papers-model-discovery)

- [Software](#software)

  - [Julia](#software-julia)

  - [Python](#software-python)

    - [Pytorch based](#software-python-pytorch)

    - [Jax based](#software-python-jax)

    - [Tensorflow based](#software-python-tensorflow)

    - [Other](#software-python-other)

- [Books](#books)

- [Videos](#videos)

  - [Channels](#videos-channels)

  - [Playlists](#videos-playlists)

- [Conferences and Workshops](#conferences)

  - [Future](#conferences-future)

  - [Past](#conferences-past)

- [Contributions Guidelines](#contributions)

  - [Papers](#contributions-papers)

  - [Software](#contributions-software)

  - [Videos](#contributions-videos)

  - [Conferences and Workshops](#contributions-conferences)





## Papers





### Neural Differential Equations

- Neural Ordinary Differential Equations [`pub`](https://papers.nips.cc/paper/2018/hash/69386f6bb1dfed68692a24c8686939b9-Abstract.html) [`arxiv`](https://arxiv.org/abs/1806.07366) 
 Chen, Ricky TQ, Yulia Rubanova, Jesse Bettencourt, and David K. Duvenaud 
 _Advances in neural information processing systems 31 (2018)_

- Universal Differential Equations for Scientific Machine Learning [`arxiv`](https://arxiv.org/abs/2001.04385) [`code`](https://github.com/ChrisRackauckas/universal_differential_equations) 
 Rackauckas, Christopher, Yingbo Ma, Julius Martensen, Collin Warner, Kirill Zubov, Rohit Supekar, Dominic Skinner, Ali Ramadhan, and Alan Edelman 
 _arXiv preprint arXiv:2001.04385 (2020)_

- Stiff neural ordinary differential equations [`pub`](https://aip.scitation.org/doi/full/10.1063/5.0060697) [`arxiv`](https://arxiv.org/abs/2103.15341) [`code`](https://github.com/DENG-MIT/StiffNeuralODE) 
 Kim, Suyong, Weiqi Ji, Sili Deng, Yingbo Ma, and Christopher Rackauckas 
 _Chaos: An Interdisciplinary Journal of Nonlinear Science 31, no. 9 (2021): 093122_

- Hamiltonian neural networks [`pub`](https://papers.nips.cc/paper/2019/hash/26cd8ecadce0d4efd6cc8a8725cbd1f8-Abstract.html) [`arxiv`](https://arxiv.org/abs/1906.01563) [`code`](https://github.com/greydanus/hamiltonian-nn) 
 Greydanus, Samuel, Misko Dzamba, and Jason Yosinski 
 _Advances in neural information processing systems 32 (2019)._

- Augmented neural odes [`pub`](https://papers.nips.cc/paper/2019/hash/21be9a4bd4f81549a9d1d241981cec3c-Abstract.html) [`arxiv`](https://arxiv.org/abs/1904.01681) [`code`](https://github.com/EmilienDupont/augmented-neural-odes) 
 Dupont, Emilien, Arnaud Doucet, and Yee Whye Teh 
 _Advances in Neural Information Processing Systems 32 (2019)._

- Fourier neural operator for parametric partial differential equations [`arxiv`](https://arxiv.org/abs/2010.08895) 
 Li, Zongyi, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar 
 _arXiv preprint arXiv:2010.08895 (2020)._

- GRU-ODE-Bayes: Continuous modeling of sporadically-observed time series [`pub`](https://proceedings.neurips.cc/paper/2019/hash/455cb2657aaa59e32fad80cb0b65b9dc-Abstract.html) [`arxiv`](https://arxiv.org/abs/1905.12374) [`code`](https://github.com/edebrouwer/gru_ode_bayes) 
 De Brouwer, Edward, Jaak Simm, Adam Arany, and Yves Moreau. 
 _Advances in neural information processing systems 32 (2019)._

- Learning long-term dependencies in irregularly-sampled time series [`arxiv`](https://arxiv.org/abs/2006.04418) [`code`](https://github.com/mlech26l/ode-lstms) 
 Lechner, Mathias, and Ramin Hasani 
 _arXiv preprint arXiv:2006.04418 (2020)._

- Neural controlled differential equations for irregular time series [`pub`](https://proceedings.neurips.cc/paper/2020/hash/4a5876b450b45371f6cfe5047ac8cd45-Abstract.html) [`arxiv`](https://arxiv.org/abs/2005.08926) [`code`](https://github.com/patrick-kidger/NeuralCDE) 
 Kidger, Patrick, James Morrill, James Foster, and Terry Lyons 
 _Advances in Neural Information Processing Systems 33 (2020): 6696-6707._

- Neural stochastic differential equations: Deep latent gaussian models in the diffusion limit [`arxiv`](https://arxiv.org/abs/1905.09883) 
 Tzen, Belinda, and Maxim Raginsky 
 arXiv preprint arXiv:1905.09883 (2019).





### Physics Informed NNs

- Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations [`pub`](https://www.sciencedirect.com/science/article/pii/S0021999118307125) [`code`](https://github.com/maziarraissi/PINNs) 
 Raissi, Maziar, Paris Perdikaris, and George E. Karniadakis 
 _Journal of Computational physics 378 (2019): 686-707_

- Artificial neural networks for solving ordinary and partial differential equations. [`pub`](https://ieeexplore.ieee.org/abstract/document/712178) [`arxiv`](https://arxiv.org/abs/physics/9705023) 
 Lagaris, Isaac E., Aristidis Likas, and Dimitrios I. Fotiadis 
 _IEEE transactions on neural networks 9, no. 5 (1998): 987-1000._

- fPINNs: Fractional physics-informed neural networks [`pub`](https://epubs.siam.org/doi/10.1137/18M1229845) [`arxiv`](https://arxiv.org/abs/1811.08967)  
 Pang, Guofei, Lu Lu, and George Em Karniadakis 
 _SIAM Journal on Scientific Computing 41, no. 4 (2019): A2603-A2626._

- Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems [`pub`](https://www.sciencedirect.com/science/article/pii/S0021999119305340?via%3Dihub) [`arxiv`](https://arxiv.org/abs/1809.08327v1) 
 Zhang, Dongkun, Lu Lu, Ling Guo, and George Em Karniadakis 
 _Journal of Computational Physics 397 (2019): 108850._

- Physics-informed neural networks with hard constraints for inverse design [`pub`](https://epubs.siam.org/doi/10.1137/21M1397908) [`arxiv`](https://arxiv.org/abs/2102.04626) [`code`](https://github.com/lululxvi/hpinn) 
 Lu, Lu, Raphael Pestourie, Wenjie Yao, Zhicheng Wang, Francesc Verdugo, and Steven G. Johnson 
 _SIAM Journal on Scientific Computing 43, no. 6 (2021): B1105-B1132._

- On the eigenvector bias of fourier feature networks: From regression to solving multi-scale pdes with physics-informed neural networks [`pub`](https://www.sciencedirect.com/science/article/abs/pii/S0045782521002759) [`arxiv`](https://arxiv.org/abs/2012.10047) [`code`](https://github.com/PredictiveIntelligenceLab/MultiscalePINNs) 
 Wang, Sifan, Hanwen Wang, and Paris Perdikaris 
 _Computer Methods in Applied Mechanics and Engineering 384 (2021): 113938._





### Neural Operators

- DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators [`arxiv`](https://arxiv.org/abs/1910.03193) 
 Lu, Lu, Pengzhan Jin, and George Em Karniadakis 
 _arXiv preprint arXiv:1910.03193 (2019)._

- Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators [`pub`](https://www.nature.com/articles/s42256-021-00302-5) [`code`](https://github.com/lululxvi/deeponet) 
 Lu, Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis 
 _Nature Machine Intelligence 3, no. 3 (2021): 218-229._

- Learning the solution operator of parametric partial differential equations with physics-informed DeepONets [`pub`](https://www.science.org/doi/full/10.1126/sciadv.abi8605) [`code`](https://github.com/PredictiveIntelligenceLab/Physics-informed-DeepONets) 
 Wang, Sifan, Hanwen Wang, and Paris Perdikaris 
 _Science advances 7, no. 40 (2021): eabi8605._

- Fourier neural operator for parametric partial differential equations [`arxiv`](https://arxiv.org/abs/2010.08895) 
 Li, Zongyi, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar 
 _arXiv preprint arXiv:2010.08895 (2020)._





### Model Discovery

- Discovering governing equations from data by sparse identification of nonlinear dynamical systems [`pub`](https://www.pnas.org/doi/full/10.1073/pnas.1517384113) [`arxiv`](https://arxiv.org/abs/1509.03580) 
 Brunton, Steven L., Joshua L. Proctor, and J. Nathan Kutz. 
 _Proceedings of the national academy of sciences 113, no. 15 (2016): 3932-3937._

- Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control [`pub`](https://doi.org/10.1098/rspa.2021.0904) [`arxiv`](https://arxiv.org/abs/2111.10992) 
 Fasel, Urban, J. Nathan Kutz, Bingni W. Brunton, and Steven L. Brunton. 
 _Proceedings of the Royal Society A 478, no. 2260 (2022): 20210904._





## Software





### Julia

Julia has an entire organization called [Scientific Machine Learning (Sciml)](https://sciml.ai/) which provides software to do SciML with. The following list is a non-comprehensive list of the packages provided, for a full description please check out their website.

- DifferentialEquations.jl [`code`](https://github.com/SciML/DifferentialEquations.jl) [`docs`](https://diffeq.sciml.ai/stable/) 
 Differential equations solvers.

- DiffEqFlux.jl [`code`](https://github.com/SciML/DiffEqFlux.jl) [`docs`](https://docs.sciml.ai/DiffEqFlux/stable/) 
 Interface to build all variaties of neural differential equations.

- NeuralPDEs.jl [`code`](https://github.com/SciML/NeuralPDE.jl) [`docs`](https://docs.sciml.ai/NeuralPDE/stable/) 
 Interface to build Physics Informed Neural Networks.





### Python





#### [Pytorch](https://pytorch.org/) based

- torchdiffeq [`code`](https://github.com/rtqichen/torchdiffeq) 
 Differential equations solvers.

- torchdyn [`code`](https://github.com/DiffEqML/torchdyn) [`docs`](https://torchdyn.readthedocs.io/en/latest/) 
 Library for neural differential equations and implicit models.

- NeuroMANCER [`code`](https://github.com/pnnl/neuromancer) [`docs`](https://pnnl.github.io/neuromancer/) 
 Neural Modules with Adaptive Nonlinear Constraints and Efficient Regularizations.

- IDRLnet [`code`](https://github.com/idrl-lab/idrlnet) [`docs`](https://idrlnet.readthedocs.io/en/latest/) 
 Machine learning library that solves both forward and inverse differential equations via physics-informed neural networks (PINN).





#### [Jax](https://jax.readthedocs.io/en/latest/) based

- diffrax [`code`](https://github.com/patrick-kidger/diffrax) [`docs`](https://docs.kidger.site/diffrax/) 
 Differential equations solvers.





#### [Tensorflow](https://www.tensorflow.org/) based

- SciANN [`code`](https://github.com/sciann/sciann) [`docs`](https://www.sciann.com/) 
 Neural Networks for Scientific Computations.

- NeuralUQ [`code`](https://github.com/Crunch-UQ4MI/neuraluq) 
 A comprehensive library for uncertainty quantification in neural differential equations and operators.





#### Other

- pysindy [`code`](https://github.com/dynamicslab/pysindy) [`docs`](https://pysindy.readthedocs.io/en/latest/) 
 Sparse Identification of Nonlinear Dynamical systems (SINDy).

- DeepXDE [`code`](https://github.com/lululxvi/deepxde) [`docs`](https://deepxde.readthedocs.io/en/latest/) 
 Multi-platform (Pytorch, Jax, Tensorflow) library for scientific machine learning and physics-informed learning.





## Books

- Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control. 2nd ed. [`pub`](https://doi.org/10.1017/9781009089517) 
 Brunton, Steven L., and J. Nathan Kutz. 
  _Cambridge: Cambridge University Press, 2022._

- Parallel Computing and Scientific Machine Learning (SciML): Methods and Applications [`site`](https://book.sciml.ai/) 
 Chris Rackauckas 




## Videos





### Channels

- Physics Informed Machine Learning [`profile`](https://www.youtube.com/c/PhysicsInformedMachineLearning) 
 From the channel description: _This channel hosts videos from workshops at UW on Data-Driven Science and Engineering, and Physics Informed Machine Learning._

- Parallel Computing and Scientific Machine Learning [`profile`](https://www.youtube.com/channel/UCDtsHjkOEMHYPGgpKX8VOPg) 
 Lecture videos for MIT's 18.337J/6.338J: Parallel Computing and Scientific Machine Learning course of Fall 2020 and Spring 2021.

- Steve Brunton [`profile`](https://www.youtube.com/c/Eigensteve) 
 Lectures ranging from Linear Algebra basis to SciML theory and applications.

- Nathan Kutz [`profile`](www.youtube.com/@NathanKutzAMATH) 
 From the channel description: _A YouTube channel for Applied and Computational Mathematics techniques, from full graduate level courses to tutorials on emerging methods._

- Crunch Group [`profile`](www.youtube.com/@CrunchGroup) 
 From the channel description: _This channel puts all the seminars that are weekly held at the CRUNCH Group, Division of Applied Mathematics, Brown University, USA. This group is the home of PINNs, DeepONet and much more!!!_





### Playlists

- SciMLCon 2022 [`list`](https://www.youtube.com/playlist?list=PLP8iPy9hna6QglWLQM02jcVjEBjaamzvw)

- Machine Learning for Physics and the Physics of Learning 2019 [`list`](https://youtube.com/playlist?list=PLHyI3Fbmv0SfQfS1rknFsr_UaaWpJ1EKA&si=phmipmwkZLYHxpsT)

- Data-driven Physical Simulations (DDPS) Seminar Series [`list`](https://youtube.com/playlist?list=PLy9rIbGDXrG2Ly0LPYNuNn1ohQTqO6mmp&si=2OjwWD2vt1XR1V47)

- ETH Zürich | Deep Learning in Scientific Computing 2023 [`list`](https://youtube.com/playlist?list=PLJkYEExhe7rYY5HjpIJbgo-tDZ3bIAqAm&si=6NjAezoZ69fUurNE)





### Conferences and Workshops





#### Future





#### Past

- The Symbiosis of Deep Learning and Differential Equations - NeurIPS 2022 Workshop [`page`](https://dlde-2022.github.io/) 
 Online 
 2022/12/14

- SciMLCon 2022 [`page`](https://scimlcon.org/2022/) 
 Online 
 2022/03/23

- The Symbiosis of Deep Learning and Differential Equations - NeurIPS 2021 Workshop [`page`](https://dl-de.github.io/) 
 Online 
 2021/12/14

- Scientific Machine Learning - Workshop [`page`](https://icerm.brown.edu/events/ht19-1-sml/) 
 ICERM, Brown University 
 2019/01/28 - 2019/01/30





## Contributions Guidelines

Contributions are very welcomed and encouraged! Please open a pull request with the indication of the type of contributions ([PAPER] for papers entries, [SOFTWARE] for software entries and so on), the entry itself and a few lines on why the contribution belongs to Awesome Scientific Machine Learning. Below are listed a few guidelines for each entry type. Please make sure to follow them to ensure a quick merge and an easier experience altogheter. Be aware that the guidelines themselves are subject to change, if you have ideas on how to improve the repo through them make sure to open a pr and tag it [GUIDELINES].



All of the guidelines given here are based on elements and resources already indexed, so if there is any confusion about them please make sure to check the raw version of the README.md before opening a pr.



### Papers

Papers should have title, author list and pubblication venue separated by break elements. The author's name are to be fully spelled out (following the Chicago entry on Google scholar) and the pubblication venue is to be written in _italic_.  Please provide the link of the source of the pubblication in [pub], and the [arxiv](https://arxiv.org/) abstract as [arxiv] when applicable. Additionally the code used for the paper can be linked in [code] (this only applies when the code is linked in the paper itself or written by the author(s). For third party implementations look at [Papers with Code](https://paperswithcode.com/)).





### Software

Software should have the full name of the library and they have to be indexed under the appropriate language (Julia, Pytho, R...). If the language index for the software you want to add to the list is not present please make sure to add it. Please provide the source of the code in [code] (it can be a GitHub link, GitLab link or you favourite version control host) and the link to the documentation in [docs]. In the next line please provide a couple of sentences with reference to what the software is doing. Please refrain from linking general deep learning libraries, in most cases they should be indexed already under their language.





### Videos

Videos should have the title, speaker and host separated by break elements. After the title please provide the link to the video in [video] and the slides in [slides] if applicable. Entire channels dedicated to SciML can be linked providing the source in [profile]. Playlists of SciML videos should be linked as [list]. Please try to minimize the intersection of playlists and channels.





### Conferences and Workshops

Events should be indexed in chronological order, with the latest on top. Link to the main page should be done in [page]. Please be mindful of the distinction between past events and future events. Nome of the event, location and date(s) should be separeted by break elements. Dates follow the format YYYY/MM/DD. If the event spanned more days please indicate starting day and ending day separated with the following symbol: -. Example: YYYY/MM/DD-YYYY/MM/DD.



An event can be added in the future section without date or location as long as there is a web page with the description of the event. The missing date or location line should be filled in as TBD. When the date is known please make sure to update the entry.