https://github.com/openmdao/lectures
Lectures on various topics having to do with Optimization in general and OpenMDAO in particular
https://github.com/openmdao/lectures
Last synced: 7 months ago
JSON representation
Lectures on various topics having to do with Optimization in general and OpenMDAO in particular
- Host: GitHub
- URL: https://github.com/openmdao/lectures
- Owner: OpenMDAO
- Created: 2020-05-12T14:22:36.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2021-02-20T00:59:04.000Z (over 5 years ago)
- Last Synced: 2024-03-26T02:24:17.109Z (about 2 years ago)
- Language: Python
- Size: 33.3 MB
- Stars: 8
- Watchers: 8
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Gradient Based Optimization for Practitioners
A collection of notes and associated scripts for online lectures on OpenMDAO.
These lectures are the beginnings of a MDO training course that I am building.
I am just at the early stages of laying out the course and for now this is mostly a collection of scattered lectures on topics that I develop as I need to teach them.
The full collection of lectures can be found on the [OpenMDAO youtube channel](https://www.youtube.com/playlist?list=PLPusXFXT29sWLpp5A8yOt6AYiW7kus56q)
If you're new to OpenMDAO and looking for help getting started, check out the [User Guide](http://openmdao.org/twodocs/versions/latest/index.html) in the docs.
Thats the best place to get started.
# Course Outline
## Introduction
### Three Critical MDO skill sets
* Optimization Problem Formulation
* Model Building
* Model Differentiation
### Additional Important Skill sets
* Version control: Git and Github
* Coding best practices: test driven development
### Optimization Problem Formulation
* What is a optimizer and how do you use it?
* Basic optimization problem formulation: objective, ieq constraint, eq constraint
* Constrained optimization can be turned into a nonlinear solve (derive the KKT conditions)
* Getting optimization problem formulation right
* Basic scaling techniques for optimization
* Advanced optimization problem formulation
* bound constraint vs ieq constraints
* slack variables for improved stability
* eq constraints vs solver balance
* MDF vs IDF vs SAND
* Re-parameterizing a design space (need to develop good examples here)
* Dealing with multiple objectives: weighted sum vs epsilon-constraint
* Understanding what controls the compute cost for optimization
* Cost of a nonlinear solve at start of opt vs at end
* time to compute derivatives
* Weird Optimization Situations
* Singular objective function (solar tracking example including non-sun hours)
* Separable objectives
* constraint relaxation
* Nested optimization techniques for bumpy problems
* Why Optimization convergence is important --- so you get smooth trends
### Model Building
* Basic implicit and explicit functions
* Pseudo explicit functions: implicit functions hiding inside explicit ones
* Hierarchical model construction and visualization
* Implicit Model Structure: Cyclic Dependency -- example: 2D airfoils on translational and rotational spring
* Implicit Model Structure: Balance Equation -- example: modular truss solver
* Common Modeling Pattern: Node-resistance (circuit analysis, thermal analysis, truss solver)
* How to debug when a solver is not converging
* Basic solvers: fixed point iteration vs Newton-like methods
* Nested Solvers: when and how to use them
* adding curve fits into your model in a differentiable way -- Collocation example
* Using a newton solver to tackle nested optimizations -- L2 norm minimization
* (advanced topic) Advanced Solvers: Gradient Free (Bracketing, Brent, NLBGS, Jacobi) vs Gradient Based (Newton, Broyden)
* (advanced topic) Understanding Newton's method
* (advanced topic) What is a line search and why should you care about it?
* (advanced topic) Unbalanced nonlinearity and its relationship to nested solver structure
* (advanced topic) Matrix free linear solver methods for use with Newton solver
### Model Differentiation
* Total derivatives vs partials derivatives
* Derivatives of scalar functions:
* Computing derivatives of explicit functions: chain rule
* computing derivatives of implicit functions: analytic derivatives
* What to do with pseudo explicit functions?
* Basic Differentiation Techniques: FD + chain rule, FD + analytic
* Advanced Differentiation Techniques: CS, AD, CS + analytic, AD + analytic
* Differentiation of vector valued functions
* [diagonal](./simple_matrix_derivatives)
* [non-diagonal](./hard_matrix_derivatives)
* general sparse partials
* Computing derivatives as cheaply as possible
* Forward vs Reverse derivatives
* How to structure your code to be easily differentiable
* UDE
* (advanced topic) [Linear solver architectures for the UDE](./gs_chainrule)
* Common gotchas
* conditionals
* for loops
* singularities
* Derivatives of time-dependent functions