Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

https://github.com/usccolumbia/Awesome-Machine-learning-for-discovery-of-physical-laws

A curated list of awesome resources on using machine learning and data science for discovery of physical laws
https://github.com/usccolumbia/Awesome-Machine-learning-for-discovery-of-physical-laws

List: Awesome-Machine-learning-for-discovery-of-physical-laws

Last synced: 26 days ago
JSON representation

A curated list of awesome resources on using machine learning and data science for discovery of physical laws

Lists

README 

        
## Awesome Machine learning for discovery of physical laws (ML4PhysicsLAW)

A curated list of awesome resources on using machine learning and data science for discovery of physical laws,inspired by [awesome-computer-vision](https://github.com/jbhuang0604/awesome-computer-vision).



For a list people in ML4PhysicsLaw, please visit [here](https://github.com/usccolumbia/Awesome-Machine-learning-for-discovery-of-physical-laws/blob/master/people.md)



Edited by [Machine Learning and Evolution Laboratory at University of South Carolina](http://mleg.cse.sc.edu)



## Contributing

Please feel free to send me [pull requests](https://github.com/usccolumbia/Awesome-Machine-learning-for-discovery-of-physical-laws/pulls) or email Dr. Jianjun Hu at University of South Carolina([email protected]) to add links.








## Table of Contents



## Reading list to get inspired

- [Artificial Intelligence and Augmented Intelligence for Automated Investigations for Scientific Discovery network](http://www.ai3sd.org/aireact2020/abstracts)

- Nuclear Fusion and Artificial Intelligence: the Dream of Limitless Energy from [AI daily](https://aidaily.co.uk/articles/nuclear-fusion-and-artificial-intelligence-the-dream-of-limitless-energy)

- Is Fusion Really Close To Reality? Yes, Thanks To Machine Learning. [Forbes](https://www.forbes.com/sites/jeffmcmahon/2020/04/27/is-fusion-really-closer-to-reality-yes-thanks-to-machine-learning/#613f2150652b)

- Artificial Intelligence Accelerates Development of Limitless Fusion Energy. [article](https://scitechdaily.com/artificial-intelligence-accelerates-development-of-limitless-fusion-energy/)

- Scientists think we’ll finally solve nuclear fusion thanks to cutting-edge AI. [link](https://thenextweb.com/neural/2020/04/27/scientists-think-well-finally-solve-nuclear-fusion-thanks-to-cutting-edge-ai/)

- Containing the sun using Deep-learning AI. [Harvard link](https://news.harvard.edu/gazette/story/2019/04/harvard-princeton-scientists-make-ai-breakthrough-for-fusion-energy/)

- AI for global health. [Talk](https://aiinglobalhealth.org/)

- Blending physics with artificial intelligence. [](https://www.spiedigitallibrary.org/conference-proceedings-of-spie/11396/113960B/Blending-physics-with-artificial-intelligence/10.1117/12.2565099.short?SSO=1)

- [Advancing Fusion with Machine Learning workshop report](https://science.osti.gov/-/media/fes/pdf/workshop-reports/FES_ASCR_Machine_Learning_Report.pdf?hash=27C6DA2A9A92F884DC618FCB928A89F4C39BD764&la=en) identified several areas that ML can help fusion physics.

- Machine Learning Reveals Quantum Phases of Matter 

- AI helps unlock 'dark matter' of bizarre superconductors

- Can AI help crack the code of fusion power? [theverge](https://www.theverge.com/2019/1/29/18201220/nuclear-fusion-energy-artificial-intelligence-machine-learning-tae-google)

- [AI could be the perfect tool for exploring the Universe](https://www.theverge.com/2017/11/15/16654352/ai-astronomy-space-exploration-data)

- [Tackling Climate Change with Machine Learning ICLR2020 workshop papers](https://www.climatechange.ai/ICLR2020_workshop.html)

- [A machine-learning revolution for physics and materials science...](https://physicsworld.com/a/a-machine-learning-revolution/)

- [Machine learning meets quantum physics](https://physicstoday.scitation.org/doi/10.1063/PT.3.4164)

- [Machine learning versus physics-based modeling](https://www.kdnuggets.com/2019/05/physics-machine-learning-models.html)

- [AI Copernicus ‘discovers’ that Earth orbits the Sun](https://www.nature.com/articles/d41586-019-03332-7)

- [AI Teaches Itself Laws of Physics](https://www.unite.ai/ai-teaches-itself-laws-of-physics/)



## Exemplary research works



- "Distilling free-form natural laws from experimental data." Science 324, no. 5923 (2009): 81-85. [Link](https://science.sciencemag.org/content/324/5923/81?casa_token=Cbi15b-au3cAAAAA:bZryjDfd1k-r3SoZaW2xRhzhY7qRkaY8SXnauk93tQkEYDtDcDNTLXHbiwQ0rc_acbAx_8nU9HGFID8) and [Citations](https://scholar.google.com/scholar?hl=en&as_sdt=5,41&sciodt=0,41&cites=11583184257062107912&scipsc=&q=&scisbd=1) by Schmidt, Michael, and Hod Lipson. 

- "AI Feynman: A physics-inspired method for symbolic regression." [link](https://advances.sciencemag.org/content/6/16/eaay2631.abstract) Science Advances 6, no. 16 (2020): eaay2631. Udrescu, Silviu-Marian, and Max Tegmark. 

- Predicting disruptive instabilities in controlled fusion plasmas through deep learning. Nature 2020. [link](https://www.nature.com/articles/s41586-019-1116-4)

- K. T. Schütt, M. Gastegger, A. Tkatchenko, K.-R. Müller, R. J. Maurer. Unifying machine learning and quantum chemistry with a deep neural network for molecular wavefunctions. Nature Communications, 2019; 10 (1) DOI: 10.1038/s41467-019-12875-2

- [Discovering Physical Concepts with Neural Networks](https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.124.010508)



# Applications

- Finding Strong Gravitational Lenses in the Kilo Degree Survey with Convolutional Neural Networks. [arxiv](https://arxiv.org/abs/1702.07675)

- Physics-Guided Machine Learning for Scientific Discovery: An Application in Simulating Lake Temperature Profiles



## Survey/Review papers



- "Integrating physics-based modeling with machine learning: A survey." [PDF](https://arxiv.org/pdf/2003.04919) 2020 by Willard, Jared, Xiaowei Jia, Shaoming Xu, Michael Steinbach, and Vipin Kumar

- Achuta Kadambi "Blending physics with artificial intelligence", Proc. SPIE 11396, Computational Imaging V, 113960B (24 April 2020); https://doi.org/10.1117/12.2565099

- Machine learning and the physical sciences [arxiv](https://arxiv.org/abs/1903.10563)

- The power of machine learning. Nature physics. [link](https://www.nature.com/articles/s41567-019-0737-8)

- 

- Fast Differentiable Sorting and Ranking. [`arxiv`](https://arxiv.org/abs/2002.08871)

- Gradient Boosting Neural Networks: GrowNet. [`arxiv`](https://arxiv.org/abs/2002.07971)

- Learning with Differentiable Perturbed Optimizers. [`arxiv`](https://arxiv.org/abs/2002.08676)

- The Costs and Benefits of Goal-Directed Attention in Deep Convolutional Neural Networks. [`pdf`](https://arxiv.org/pdf/2002.02342.pdf)

- The Geometry of Sign Gradient Descent. [`arxiv`](https://arxiv.org/abs/2002.08056)

- The large learning rate phase of deep learning: the catapult mechanism. [`arxiv`](https://arxiv.org/abs/2003.02218)



## Symbolic regression for equation search/discovery



## Causal relationship learning/discovery

- Causal machine learning papers [link](https://github.com/logangraham/arXausality)

- Causal Discovery with Attention-Based Convolutional Neural Networks” by Meike Nauta * , Doina Bucur and Christin Seifert 

- https://aip.scitation.org/doi/full/10.1063/1.5025050

- https://academic.oup.com/bioinformatics/article/32/6/875/1744279

- https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0103812		



- https://github.com/khurramjaved96/online-causal-models

- https://github.com/capybaralet/REx_code_release

- https://github.com/reiinakano/invariant-risk-minimization

- https://github.com/IBM/IRM-games



- https://github.com/logangraham/arXausality   arxiv causal ML algorithms papers

- https://github.com/robertness/causalML

- https://github.com/FenTechSolutions/CausalDiscoveryToolbox

- https://github.com/microsoft/dowhy  causal inference



## ODE derivation, equation inference

-[Physics Based Deep Learning](https://github.com/thunil/Physics-Based-Deep-Learning)



## Bayesian networks



## Interpretable AI/ML, including Inpretable  deep neural networks



## Topological analysis



## feature/representation learning (DL, ISOMAP, ICA,ALAE)

- GLAD: Learning Sparse Graph Recovery



## Uncertainty quantification

- Physics-Guided Machine Learning for Scientific Discovery: An Application in Simulating Lake Temperature Profiles. [link](https://arxiv.org/abs/2001.11086)



## Graph neural networks, graph deep learning  



## Reinforcement learning for active learning and experiment planning/design



## ML/DL for design of experiments, active learning to do minimum no. of experiments/sampling



## Disentangling of features by deep learning



## Multi-scale issues



## Dimension reduction and visualization



## Active learning for experiments design

 active learning, sequential decision making, experimental design, reinforcement learning, interactive learning or generative learning

 - Workshop I: From Passive to Active: Generative and Reinforcement Learning with Physics [link](http://www.ipam.ucla.edu/programs/workshops/workshop-i-from-passive-to-active-generative-and-reinforcement-learning-with-physics/)  [talks](http://www.ipam.ucla.edu/programs/workshops/workshop-i-from-passive-to-active-generative-and-reinforcement-learning-with-physics/?tab=schedule)

 



## Extrapolation capability and model robustness

- building a model and try to break it to make it robust



## Deep learning resources'

- [github Awesome deep learning resources](https://github.com/endymecy/awesome-deeplearning-resources)



## AutoML



- When NAS Meets Robustness: In Search of Robust Architectures against Adversarial Attacks. [`arxiv`](https://arxiv.org/abs/1911.10695) [`code`](https://github.com/gmh14/RobNets)

- Gryffin: An algorithm for Bayesian optimization for categorical variables informed by physical intuition with applications to chemistry. [`arxiv`](https://arxiv.org/abs/2003.12127)

- Uncertainty Quantification for Bayesian Optimization. [`pdf`](https://arxiv.org/pdf/2002.01569.pdf)



## Dimension reduction



- An Idea From Physics Helps AI See in Higher Dimensions [Blog](https://www.quantamagazine.org/an-idea-from-physics-helps-ai-see-in-higher-dimensions-20200109/)

- Unbalanced GANs: Pre-training the Generator of Generative Adversarial Network using Variational Autoencoder. [`pdf`](https://arxiv.org/pdf/2002.02112.pdf)



## GNN



- Bridging the Gap Between Spectral and Spatial Domains in Graph Neural Networks. [`arxiv`](https://arxiv.org/abs/2003.11702) [`code`](https://github.com/balcilar/Spectral-Designed-Graph-Convolutions)

- Generalization and Representational Limits of Graph Neural Networks. [`arxiv`](https://arxiv.org/abs/2002.06157)

- SIGN: Scalable Inception Graph Neural Networks. [`arxiv`](https://arxiv.org/abs/2004.11198)

- StickyPillars: Robust feature matching on point clouds using Graph Neural Networks. [`arxiv`](https://arxiv.org/abs/2002.03983)

- Supervised Learning on Relational Databases with Graph Neural Networks. [`arxiv`](https://arxiv.org/abs/2002.02046) [`code`](https://github.com/mwcvitkovic/Supervised-Learning-on-Relational-Databases-with-GNNs)



## Meta Learning



- A Comprehensive Overview and Survey of Recent Advances in Meta-Learning. [`arxiv`](https://arxiv.org/abs/2004.11149)

- Meta-Learning in Neural Networks: A Survey. [`arxiv`](https://arxiv.org/pdf/2004.05439.pdf)

- Regularizing Meta-Learning via Gradient Dropout. [`arxiv`](https://arxiv.org/abs/2004.05859)



## Research opportunities of ML for physics



- 'Science Discovery with Machine Learning' involves bridging gaps in theoretical understanding via identification of missing effects using large datasets; the acceleration of hypothesis generation and testing and the optimisation of experimental planning. Essentially, machine learning is used to support and accelerate the scientific process itself.

- 'Machine Learning Boosted Diagnostics' is where machine learning methods are used to maximise the information extracted from measurements, systematically fuse multiple data sources and infer quantities that are not directly measured. Classifcation techniques, such as supervised learning, could be used on data that is extracted from the diagnostic measurements. 

- 'Model Extraction and Reduction' includes the construction of models of fusion systems and the acceleration of computational algorithms. Effective model reduction can result in shorten computation times and mean that simulations (for the tokamak fusion reactor for example) happen faster than real-time execution.

- 'Control Augmentation with Machine Learning'. Three broad areas of plasma control research would benefit significantly from machine learning: control-level models, real-time data analysis algorithms; optimisation of plasma discharge trajectories for control scenarios. Using AI to improve control mathematics could manage the uncertainty in calculations and ensure better operational performance.

- 'Extreme Data Algorithms' involves finding methods to manage the amount and speed of data that will be generated during the fusion models.

- 'Data-Enhanced Prediction' will help monitor the health of the plant system and predict any faults, such as disruptions which are essential to be mitigated.

- 'Fusion Data Machine Learning Platform' is a system that can manage, format, curate and enable the access to experimental and simulation data from fusion models for optimal usability when used by machine learning algorithms.



## Conferences and Workshops related

- UCLA IPAM Machine Learning for Physics and the Physics of Learning [Link](http://www.ipam.ucla.edu/programs/long-programs/machine-learning-for-physics-and-the-physics-of-learning/) [talks](http://www.ipam.ucla.edu/programs/long-programs/machine-learning-for-physics-and-the-physics-of-learning/?tab=seminar-series)

- UCLA IPAM Machine Learning for Physics and the Physics of Learning Tutorials [link](http://www.ipam.ucla.edu/programs/workshops/machine-learning-for-physics-and-the-physics-of-learning-tutorials/?tab=schedule)

- Workshop II: PDE and Inverse Problem Methods in Machine Learning [IPAM link](http://www.ipam.ucla.edu/programs/workshops/workshop-ii-pde-and-inverse-problem-methods-in-machine-learning/)

- Workshop IV: Using Physical Insights for Machine Learning [link](http://www.ipam.ucla.edu/programs/workshops/workshop-iv-using-physical-insights-for-machine-learning/)

- Workshop III: Validation and Guarantees in Learning Physical Models: from Patterns to Governing Equations to Laws of Nature [link](http://www.ipam.ucla.edu/programs/workshops/workshop-iii-validation-and-guarantees-in-learning-physical-models-from-patterns-to-governing-equations-to-laws-of-nature/)

- Workshop II: Interpretable Learning in Physical Sciences [link](http://www.ipam.ucla.edu/programs/workshops/workshop-ii-interpretable-learning-in-physical-sciences/) [talks](http://www.ipam.ucla.edu/programs/workshops/workshop-ii-interpretable-learning-in-physical-sciences/?tab=schedule)

- Workshop I: From Passive to Active: Generative and Reinforcement Learning with Physics [link](http://www.ipam.ucla.edu/programs/workshops/workshop-i-from-passive-to-active-generative-and-reinforcement-learning-with-physics/)

- Workshop IV: Deep Geometric Learning of Big Data and Applications. Non-Euclidean domain deep learning. [videos](http://www.ipam.ucla.edu/programs/workshops/workshop-iv-deep-geometric-learning-of-big-data-and-applications/?tab=schedule) [link](http://www.ipam.ucla.edu/programs/workshops/workshop-iv-deep-geometric-learning-of-big-data-and-applications/)

3D point clouds and 3D shapes in computer graphics, functional MRI signals on the brain structural connectivity network, the DNA of the gene regulatory network in genomics, drugs design in quantum chemistry, neutrino detection in high energy physics, and knowledge graph for common sense understanding of visual scenes. graphs and manifolds. Fundamental operations such as convolution, coarsening, multi-resolution, causality have been redefined through spectral and spatial approaches. 

- [ML4science workshop](https://www.ml4science.org/agenda-physics-in-ml)



## Journals

- [Machine Learning: Science and Technology](https://iopscience.iop.org/journal/2632-2153)



## software toolls



- [AI Feynman symbolic regression package](https://github.com/SJ001/AI-Feynman)

- [Eureka commercial](https://www.nutonian.com/products/eureqa/)

- [Integration of Neural Network-Based Symbolic Regression in Deep Learning for Scientific Discovery](https://arxiv.org/abs/1912.04825)

- [S. S. Sahoo, C. H. Lampert, and G. Martius, “Learning Equations for Extrapolation and Control,” jun 2018. [Online](https://arxiv.org/abs/1806.07259）

- [fast symbolic regression](https://github.com/cfusting/fast-symbolic-regression)



## Tutorials

- [Physics-Informed Learning Machines](https://users.wpi.edu/~zzhang7/NENAD19_Karniadakis.pdf)



## Resources for students



#### Resource link collection

 * [Resources for students](http://people.csail.mit.edu/fredo/student.html) - Frédo Durand (MIT)

 * [Advice for Graduate Students](http://www.dgp.toronto.edu/~hertzman/advice/) - Aaron Hertzmann (Adobe Research)

 * [Graduate Skills Seminars](http://www.dgp.toronto.edu/~hertzman/courses/gradSkills/2010/) - Yashar Ganjali, Aaron Hertzmann (University of Toronto)

 * [Research Skills](http://research.microsoft.com/en-us/um/people/simonpj/papers/giving-a-talk/giving-a-talk.htm) - Simon Peyton Jones (Microsoft Research)

 * [Resource collection](http://web.engr.illinois.edu/~taoxie/advice.htm) - Tao Xie (UIUC) and Yuan Xie (UCSB)



#### Writing

 * [Write Good Papers](http://people.csail.mit.edu/fredo/FredoGoodWriting.pdf) - Frédo Durand (MIT)

 * [Notes on writing](http://people.csail.mit.edu/fredo/PUBLI/writing.pdf) - Frédo Durand (MIT)

 * [How to Write a Bad Article](http://people.csail.mit.edu/fredo/FredoBadWriting.pdf) - Frédo Durand (MIT)

 * [How to write a good CVPR submission](http://billf.mit.edu/sites/default/files/documents/cvprPapers.pdf) - William T. Freeman (MIT)

 * [How to write a great research paper](https://www.youtube.com/watch?v=g3dkRsTqdDA) - Simon Peyton Jones (Microsoft Research)

 * [How to write a SIGGRAPH paper](http://www.slideshare.net/jdily/how-to-write-a-siggraph-paper) - SIGGRAPH ASIA 2011 Course

 * [Writing Research Papers](http://www.dgp.toronto.edu/~hertzman/advice/writing-technical-papers.pdf) - Aaron Hertzmann (Adobe Research)

 * [How to Write a Paper for SIGGRAPH](http://www.computer.org/csdl/mags/cg/1987/12/mcg1987120062.pdf) - Jim Blinn

 * [How to Get Your SIGGRAPH Paper Rejected](http://www.siggraph.org/sites/default/files/kajiya.pdf) - Jim Kajiya (Microsoft Research)

 * [How to write a SIGGRAPH paper](www.liyiwei.org/courses/how-siga11/liyiwei.pptx) - Li-Yi Wei (The University of Hong Kong)

 * [How to Write a Great Paper](http://www-hagen.informatik.uni-kl.de/~bertram/talks/getpublished.pdf) - Martin Martin Hering Hering--Bertram (Hochschule Bremen University of Applied Sciences)

 * [How to have a paper get into SIGGRAPH?](http://www-ui.is.s.u-tokyo.ac.jp/~takeo/writings/siggraph.html) - Takeo Igarashi (The University of Tokyo)

 * [Good Writing](http://www.cs.cmu.edu/~pausch/Randy/Randy/raibert.htm) - Marc H. Raibert (Boston Dynamics, Inc.)

 * [How to Write a Computer Vision Paper](http://web.engr.illinois.edu/~dhoiem/presentations/How%20to%20Write%20a%20Computer%20Vison%20Paper.ppt) - Derek Hoiem (UIUC)

 * [Common mistakes in technical writing](http://www.cs.dartmouth.edu/~wjarosz/writing.html) - Wojciech Jarosz (Dartmouth College)



#### Presentation

 * [Giving a Research Talk](http://people.csail.mit.edu/fredo/TalkAdvice.pdf) - Frédo Durand (MIT)

 * [How to give a good talk](http://www.dgp.toronto.edu/~hertzman/courses/gradSkills/2010/GivingGoodTalks.pdf) - David Fleet (University of Toronto) and Aaron Hertzmann (Adobe Research)

 * [Designing conference posters](http://colinpurrington.com/tips/poster-design) - Colin Purrington

 * [Physics-Based-Deep-Learning](https://github.com/thunil/Physics-Based-Deep-Learning)

#### Research

 * [How to do research](http://people.csail.mit.edu/billf/www/papers/doresearch.pdf) - William T. Freeman (MIT)

 * [You and Your Research](http://www.cs.virginia.edu/~robins/YouAndYourResearch.html) - Richard Hamming

 * [Warning Signs of Bogus Progress in Research in an Age of Rich Computation and Information](http://yima.csl.illinois.edu/psfile/bogus.pdf) - Yi Ma (UIUC)

 * [Seven Warning Signs of Bogus Science](http://www.quackwatch.com/01QuackeryRelatedTopics/signs.html) - Robert L. Park

 * [Five Principles for Choosing Research Problems in Computer Graphics](https://www.youtube.com/watch?v=v2Qaf8t8I6c) - Thomas Funkhouser (Cornell University)

 * [How To Do Research In the MIT AI Lab](http://www.cs.indiana.edu/mit.research.how.to.html) - David Chapman (MIT)

 * [Recent Advances in Computer Vision](http://www.slideshare.net/antiw/recent-advances-in-computer-vision) - Ming-Hsuan Yang (UC Merced)

 * [How to Come Up with Research Ideas in Computer Vision?](http://www.slideshare.net/jbhuang/how-to-come-up-with-new-research-ideas-4005840) - Jia-Bin Huang (UIUC)

 * [How to Read Academic Papers](http://www.slideshare.net/jbhuang/how-to-read-academic-papers) - Jia-Bin Huang (UIUC)



#### Time Management

 * [Time Management](https://www.youtube.com/watch?v=oTugjssqOT0) - Randy Pausch (CMU)



## Blogs

 * [Learn OpenCV](http://www.learnopencv.com/) - Satya Mallick

 * [Tombone's Computer Vision Blog](http://www.computervisionblog.com/) - Tomasz Malisiewicz

 * [Computer vision for dummies](http://www.visiondummy.com/) - Vincent Spruyt

 * [Andrej Karpathy blog](http://karpathy.github.io/) - Andrej Karpathy

 * [AI Shack](http://aishack.in/) - Utkarsh Sinha

 * [Computer Vision Talks](http://computer-vision-talks.com/) - Eugene Khvedchenya

 * [Computer Vision Basics with Python Keras and OpenCV](https://github.com/jrobchin/Computer-Vision-Basics-with-Python-Keras-and-OpenCV) - Jason Chin (University of Western Ontario)



## Links

* [The Computer Vision Industry](http://www.cs.ubc.ca/~lowe/vision.html) - David Lowe

* [German Computer Vision Research Groups & Companies](http://hci.iwr.uni-heidelberg.de/Links/German_Vision/)

* [awesome-deep-learning](https://github.com/ChristosChristofidis/awesome-deep-learning)

* [awesome-machine-learning](https://github.com/josephmisiti/awesome-machine-learning)

* [Cat Paper Collection](http://www.eecs.berkeley.edu/~junyanz/cat/cat_papers.html)

* [Computer Vision News](http://www.rsipvision.com/computer-vision-news/)

* [Most cited researchers on Google scholars](https://www.webometrics.info/en/hlargerthan100)



## Licenses

License



[![CC0](http://i.creativecommons.org/p/zero/1.0/88x31.png)](http://creativecommons.org/publicdomain/zero/1.0/)



## Acknowledgement

This awesome list is made possible by the NSF HDR Grant. Award #1940099 Collaborative Research: Integrating Physics and Generative Machine Learning Models for Inverse Materials Design. and the NSF HDR PI workshop April28-30, 2020.