{"id":19387451,"url":"https://github.com/mitmath/18336","last_synced_at":"2025-08-04T18:39:55.810Z","repository":{"id":59700093,"uuid":"206361939","full_name":"mitmath/18336","owner":"mitmath","description":"18.336 - Fast Methods for Partial Differential and Integral Equations ","archived":false,"fork":false,"pushed_at":"2024-05-13T16:19:28.000Z","size":65518,"stargazers_count":183,"open_issues_count":0,"forks_count":17,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-01-07T08:28:01.288Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mitmath.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-09-04T16:16:50.000Z","updated_at":"2025-01-06T12:41:17.000Z","dependencies_parsed_at":"2024-05-13T17:53:57.927Z","dependency_job_id":null,"html_url":"https://github.com/mitmath/18336","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitmath%2F18336","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitmath%2F18336/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitmath%2F18336/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitmath%2F18336/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mitmath","download_url":"https://codeload.github.com/mitmath/18336/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240540551,"owners_count":19817791,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-10T10:09:16.096Z","updated_at":"2025-02-24T18:55:34.084Z","avatar_url":"https://github.com/mitmath.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch2 align=\"center\"\u003e\n  MIT 18.336J/6.7340J, Spring 2024 \u003cbr /\u003e\n  Fast Methods for Partial Differential and Integral Equations\n\u003c/h2\u003e\n\nCatalog description: *A unified introduction to the theory and practice of modern, near linear-time, numerical methods for large-scale partial differential and integral equations.\nTopics include: preconditioned iterative methods; generalized Fast Fourier Transform and other butterfly-based methods; multiresolution approaches including multigrid algorithms, hierarchical low-rank matrix decompositions, and low and high frequency Fast Multipole Methods.\nExample applications include: aircraft design, cardiovascular system modeling, electronic structure computation, and tomographic imaging.*\n\n## Syllabus\n\n**Lectures**: Monday/Wednesday 9:30-11:00 am in 2-136.\n\n**Office Hours**: Monday and Wednesday at 4-5pm in 2-238C.\n\n**Prerequisites**: This course covers advanced techniques for discretizing and solving PDEs.\nSome familiarity with ordinary differential equations, partial differential equations, Fourier transforms, linear algebra, and basic numerical methods for PDEs is assumed.\nIt is strongly recommended that you have taken a previous course on basic numerical methods, such as [2.096/6.336/16.910](http://student.mit.edu/catalog/m6b.html#6.336), [2.097/6.339/16.920](http://student.mit.edu/catalog/m16b.html#16.920), [18.085](http://student.mit.edu/catalog/m18a.html#18.085), or [6.337/18.335](http://student.mit.edu/catalog/m18a.html#18.335).\nProblem sets will involve extensive coding: they may be completed in the language of your choice. Solutions and class demos will be in **Julia**.\n\n**Textbooks \u0026 Other Reading**: Recommended reading will be posted as the class progresses.\nThere is no textbook for the course, but the following books may be useful:\n* Strauss \"Partial Differential Equations: An Introduction\". An advanced undergrad introduction to PDEs.\n* Boyd \"Chebyshev and Fourier Spectral Methods\". Very readable and [available online](http://depts.washington.edu/ph506/Boyd.pdf).\n* [Martinsson \"Fast Direct Solvers for Elliptic PDEs\"](https://doi.org/10.1137/1.9781611976045). Modern and concise.\n* LeVeque \"Finite difference methods for ordinary and partial differential equations\". (MIT access through SIAM.)\n\n**Grading**: 50% problem sets (approximately biweekly), 50% final project report and presentation.\nUnless previous arrangements are made, late problem sets will be accepted for 1 week after the initial due date with a 50% penalty.\n\n**Collaboration Policy**: Make a strong effort to solve problems on your own before discussing with any classmates.\nYou must write up your own code and solutions, and indicate your collaborators on your assignments.\n\n## Problem Sets\n\nProblem sets will be posted here (approximately biweekly) and should be submitted on [Gradescope](https://www.gradescope.com/).\n\n* [Problem Set 1](https://canvas.mit.edu/courses/24352/external_tools/369) is due on Friday February 23, 2024 at 6pm ([solutions](https://canvas.mit.edu/courses/24352/files/folder/Problem%20Sets/Solutions?)). \n* [Problem Set 2](https://canvas.mit.edu/courses/24352/files/folder/Problem%20Sets?) is due on Monday March 18, 2024 at 6pm ([solutions](https://canvas.mit.edu/courses/24352/files/folder/Problem%20Sets/Solutions)).\n* [Problem Set 3](https://canvas.mit.edu/courses/24352/files/folder/Problem%20Sets?) is due on Friday April 5, 2024 at 6pm ([solutions](https://canvas.mit.edu/courses/24352/files/folder/Problem%20Sets/Solutions)).\n* [Problem Set 4](https://canvas.mit.edu/courses/24352/files/folder/Problem%20Sets) is due on Friday April 19, 2024 at 6pm ([solutions](https://canvas.mit.edu/courses/24352/files/folder/Problem%20Sets/Solutions)).\n\n## Final Projects\n\nThe final project is a 10-12 page paper and a 15 minute presentation during the last week of classes.\n**The final project is broad in scope, but must include the implementation of a fast algorithm in Python or Julia along with performance and error analyses.**\nThe project can take the form of a \"literature review project\" discussing a published algorithm or a \"research project\" attempting to implement a new solver for a problem of your choice.\n\n### Lit. review projects\n\nOne possibility is to review and implement an algorithm that was mentioned briefly or not covered in the course.\nSuch a project should follow one or several published research papers describing the algorithm, along with a new implementation.\nPossible topics and suggested papers include:\n* Advanced optimizations for production-level FFT implementations.\n  * [Johnson \u0026 Frigo \"Implementing FFTs in Practice\"](https://github.com/mitmath/18336/blob/master/readings/johnson_frigo_implementing_ffts.pdf)\n* Hierarchical Poincare-Steklov schemes using Chebyshev polynomials.\n  * [Martinsson \"The Hierarchical Poincare-Steklov (HPS) solver for elliptic PDEs: A tutorial\"](https://github.com/mitmath/18336/blob/master/readings/martinsson_poincare_steklov.pdf)\n* Rectangular collocation with aliasing analysis.\n  * [Driscoll \u0026 Hale \"Rectangular spectral collocation\"](https://doi.org/10.1093/imanum/dru062)\n* Alternating direction implicit scheme for multidimensional Chebyshev Poisson solvers.\n  * [Fortunato \u0026 Townsend \"Fast Poisson solvers for spectral methods\"](https://arxiv.org/abs/1710.11259)\n* Bivariate orthogonal polynomials on triangles.\n  * [Olver et al. \"A Sparse Spectral Method on Triangles\"](https://doi.org/10.1137/19M1245888)\n* Non-uniform Fast Fourier Transforms.\n  * [Barnett et al. \"A Parallel Nonuniform Fast Fourier Transform Library Based on an 'Exponential of Semicircle' Kernel\"](https://doi.org/10.1137/18M120885X)\n* Sparse spectral elements.\n  * [Fortunato et al. \"The ultraspherical spectral element method\"](https://doi.org/10.1016/j.jcp.2020.110087)\n\n### Research projects\n\nAnother option is to use the methods covered in class to implement a fast solver for a research problem in your field.\nThis should include:\n* A discussion of the scientific problem and a brief derivation of the model PDE.\n* Mention of the current commonly used methods in the field for the problem.\n* A fast implementation of a new solver for the problem, or a related first-step.\n* Discussion of the prospects and limitations of fast/high-order techniques for this problem in the future.\n\nThe goal should be producing a functional solver matching or improving on existing techniques in certain cases.\nHowever, it is understood that this may not turn out to be feasible (or even possible), and that's what makes it research!\nIn that case, an implementation for a related toy model is expected, along with an analysis of the barriers to making a solver for the original problem.\n\n### Report \u0026 presentation format\n\nYour report should be written in the style a SIAM article, using the [SIAM article LaTeX templates](https://www.siam.org/publications/journals/about-siam-journals/information-for-authors#LaTex2e-Standard).\nYour report should be between 10 and 12 pages.\nThe SIAM layout has generous margins so this is not a lot of space.\nMake sure your presentation and notation is concise, but also has enough content and is not just padded out with images!\n\nAlong with your report, you must submit the code implementing your algorithms, preferably in the form of a Jupyter notebook.\nMake good use of headings, text, comments, and descriptive function/variables names.\nGood code is code that is easily understandable by others!\n\nYour report and presentation should both include:\n* Background information for your algorithm / physical application.\n* A concise mathematical description of the algorithm you're using.\n* Performance and error analysis for your implementation.\n\n## Lecture Material and Summaries\n\n### Lecture 1\n\n[Notes](https://github.com/mitmath/18336/blob/master/notes/lecture_01.pdf)\n\n* Why do we need fast PDE solvers? Modeling, simulation, design, and inference.\n* Solving PDEs on the computer: discretize, solve, and analyze.\n* Finite differences: epsilon accuracy, at what cost?\n\n### Lecture 2\n\n[Notes](https://github.com/mitmath/18336/blob/master/notes/lecture_02.pdf)\n\n* Symmetry and structure in a simple boundary value problem.\n* What structure appears in the discretization and how do we leverage it?\n* Locality and sparsity, translation invariance and Toeplitz matrices.\n* Fourier modes and diagonalization.\n\n**Further reading:** Finite difference approximation review, chapters 1 and 2 in LeVeque.\n\n### Lecture 3\n\n[Notes](https://github.com/mitmath/18336/blob/master/notes/lecture_03.pdf)\n\n* The Discrete Fourier Transform (DFT) matrix.\n* The Fast Fourier Transform (FFT) algorithm.\n* Radix-2 FFT: polynomial evaluation, even-odd decomposition, and roots of unity.\n\n**Further reading:** [Johnson \u0026 Frigo \"Implementing FFTs in Practice\"](https://github.com/mitmath/18336/blob/master/readings/johnson_frigo_implementing_ffts.pdf).\n\n### Lecture 4\n\n[Notes](https://github.com/mitmath/18336/blob/master/notes/lecture_04.pdf)\n\n* Fast Fourier Transform (powers other than 2).\n* Discrete Cosine and Sine Transforms (Type I).\n* Fast FFT-based Poisson solvers (1D).\n\n### Lecture 5\n\n[Notes](https://github.com/mitmath/18336/blob/master/notes/lecture_05.pdf)\n\n* Dirichlet boundary conditions and odd extensions.\n* Neumann boundary conditions and even extensions.\n* Circulant embedding for fast Toeplitz mat-vecs.\n\n**Further reading:** [Strang \"The Discrete Cosine Transform\"](https://epubs.siam.org/doi/10.1137/S0036144598336745).\n\n### Lecture 6\n\n[Notes](https://github.com/mitmath/18336/blob/master/notes/lecture_06.pdf)\n\n* Poisson's equation in a 2D separable domain.\n* Separable domains, vectorization, and Kronecker products.\n* Fast transforms and solvers on separable domains.\n\n### Lecture 7\n\n[Notes](https://github.com/mitmath/18336/blob/master/notes/lecture_07.pdf)\n\n* Complications: variable coefficients, time dependence, and nonlinearity.\n* The role of fast BVP solvers in time-steppers and nonlinear iterative solvers.\n* Iterative methods and preconditioning for BVPs.\n\n**Further Reading:** [Saad \"Five Key Concepts That Shaped Iterative Solution Methods for Linear Systems\"](https://sinews.siam.org/Details-Page/five-key-concepts-that-shaped-iterative-solution-methods-for-linear-systems).\n\n### Lecture 8\n\n[Notes](https://github.com/mitmath/18336/blob/master/notes/lecture_08.pdf)\n\n* Domain decomposition techniques.\n* Patching solutions for 1D and 2D Poisson problems.\n* Block structure and Schur complements.\n\n**Further Reading:** [Martinsson \"The Hierarchical Poincare-Steklov Solver for Elliptic PDEs\"](https://github.com/mitmath/18336/blob/master/readings/martinsson_poincare_steklov.pdf).\n\n### Lecture 9\n\n[Notes](https://github.com/mitmath/18336/blob/master/notes/lecture_09.pdf)\n\n* Hierarchical merging in domain decomposition.\n* Mapped elements and meshes.\n* Intro to spectral methods.\n\n### Lecture 10\n\n[Notes](https://github.com/mitmath/18336/blob/master/notes/lecture_10.pdf)\n\n* The Fourier spectral method in 1D.\n* Test space, trial space, and truncation errors.\n* Convergence rates for smooth functions.\n\n### Lecture 11\n\n[Notes](https://github.com/mitmath/18336/blob/master/notes/lecture_11.pdf)\n\n* Spectral methods with sine (cosine) basis.\n* Nonperiodic problems with definite parity.\n* 2D Fourier spectral methods.\n\n### Lecture 12\n\n[Notes](https://github.com/mitmath/18336/blob/master/notes/lecture_12.pdf)\n\n* Chebyshev polynomials and series.\n* Fast Chebyshev transforms.\n* Chebyshev differentiation matrices.\n\n### Lecture 13\n\n[Notes](https://github.com/mitmath/18336/blob/master/notes/lecture_13.pdf)\n\n* Polynomial interpolation.\n* Approximation properties of interpolants.\n* \"Good\" and \"bad\" grids: Chebyshev vs. equally spaced points.\n\n### Lecture 14\n\n[Notes](https://github.com/mitmath/18336/blob/master/notes/lecture_14.pdf)\n\n* Spectral collocation matrices.\n* Boundary bordering and rectangular projection.\n* Nonlinear problems: Newton's iteration and Frechet derivatives.\n\n**Further Reading:** [Driscoll and Hale \"Rectangular Spectral Collocation\"](https://doi.org/10.1093/imanum/dru062) and [Aurentz and Trefethen \"Block Operators and Spectral Discretizations\"](https://doi.org/10.1137/16M1065975).\n\n### Lecture 15\n\n[Notes](https://github.com/mitmath/18336/blob/master/notes/lecture_15.pdf)\n\n* Chebyshev \"T\" and \"U\" bases.\n* Sparse Chebyshev \"T\"-\u003e\"U\" differentiation matrix.\n* Fast solves with \"almost-banded\" discretizations.\n* Ultraspherical bases.\n\n**Further Reading:** [Olver and Townsend \"A Fast and Well-Conditioned Spectral Method\"](https://doi.org/10.1137/120865458).\n\n### Lecture 16\n\n* Multiplication matrices in ultraspherical bases.\n* When are multiplication matrices \"banded\"?\n* Pseudospectral methods for nonlinear IVPs.\n\n### Lecture 17\n\n[Notes](https://github.com/mitmath/18336/blob/master/notes/lecture_17.pdf)\n\n* Multidimensional sparse spectral methods.\n* Block structure for coupled ODE.\n* Block structure for PDE discretizations.\n\n**Further Reading:** [Vasil et. al. \"Tensor Calculus in Polar Coordinates Using Jacobi Polynomials\"](https://doi.org/10.1016/j.jcp.2016.08.013).\n\n### Lecture 18\n\n* The challenge of multiple nonperiodic variables.\n* Fast spectral methods for Poisson equation in separable domains.\n* ADI and factored ADI for Sylvester equations.\n\n**Further Reading:** [Fortunato and Townsend \"Fast Poisson Solvers for Spectral Methods\"](https://doi.org/10.1093/imanum/drz034) and [Townsend and Wilber \"On the Singular Values of Matrices with High Displacement Rank\"](https://doi.org/10.1016/j.laa.2018.02.025).\n\n### Lecture 19\n\n[Notes](https://github.com/mitmath/18336/blob/master/notes/lecture_19.pdf)\n\n* Low-rank matrices and functions.\n* Storage and computation with low-rank functions.\n* Integral operators with low-rank kernels.\n\n**Further Reading:** [Townsend and Trefethen \"Continuous Analogues of Matrix Factorizations\"](https://doi.org/10.1098/rspa.2014.0585).\n\n### Lecture 20\n\n* Green's functions for PDEs: Poisson equation.\n* Low-rank approximations: separation and smoothness.\n* Multipole expansions: projection and interpolation.\n\n### Lecture 21\n\n[Notes](https://github.com/mitmath/18336/blob/master/notes/lecture_21.pdf)\n\n* Hierarchical low-rank approximations.\n* Barnes-Hut for particle simulations.\n* The Fast Multipole Method (FMM).\n\n**Further Reading:** [Beatson and Greengard \"A Short Course on Fast Multipole Methods\"](https://github.com/mitmath/18336/blob/master/readings/beatson_greengard_fmm.pdf).\n\n### Lecture 22\n\n* Boundary Integral Equations\n* Single Layer Potentials\n* Double Layer Potentials\n\n**Further Reading:** [Martinsson's lectures from 2014 CBMS/NSF Conference on Fast Direct Solvers](https://amath.colorado.edu/faculty/martinss/2014_CBMS/lectures.html) and [Professor Demanet's 2014 lecture notes](https://github.com/mitmath/18336/blob/master/readings/demanet_18336_2014_notes.pdf).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmitmath%2F18336","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmitmath%2F18336","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmitmath%2F18336/lists"}