{"id":23280839,"url":"https://github.com/jloveric/neural-network-pdes","last_synced_at":"2026-02-28T23:03:03.328Z","repository":{"id":64932295,"uuid":"356466882","full_name":"jloveric/neural-network-pdes","owner":"jloveric","description":"Neural Network Implicit Representation of Partial Differential Equations","archived":false,"fork":false,"pushed_at":"2026-02-19T04:54:25.000Z","size":1596,"stargazers_count":6,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-02-19T10:46:50.953Z","etag":null,"topics":["euler-equations","fluid-dynamics","gas-dynamics","high-order-networks","implicit-representation","kan","neural-networks","pde","sirens","sod-shock-tube"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jloveric.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2021-04-10T04:14:06.000Z","updated_at":"2026-02-19T04:54:28.000Z","dependencies_parsed_at":"2023-12-19T14:58:05.334Z","dependency_job_id":"40284bfb-d1e6-4037-a6c4-b748038987aa","html_url":"https://github.com/jloveric/neural-network-pdes","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jloveric/neural-network-pdes","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jloveric%2Fneural-network-pdes","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jloveric%2Fneural-network-pdes/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jloveric%2Fneural-network-pdes/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jloveric%2Fneural-network-pdes/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jloveric","download_url":"https://codeload.github.com/jloveric/neural-network-pdes/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jloveric%2Fneural-network-pdes/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29954583,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-28T22:53:01.873Z","status":"ssl_error","status_checked_at":"2026-02-28T22:52:50.699Z","response_time":90,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["euler-equations","fluid-dynamics","gas-dynamics","high-order-networks","implicit-representation","kan","neural-networks","pde","sirens","sod-shock-tube"],"created_at":"2024-12-19T23:39:10.942Z","updated_at":"2026-02-28T23:03:03.315Z","avatar_url":"https://github.com/jloveric.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# neural-network-pdes\nNeural Network Implicit Representation of Partial Differential Equations.  The problems here are solved using a simple high order MLP where\nthe the input is (x,t) in 1D and the output is density, velocity and pressure.  The loss function is partial differential equation for the 1d euler equations of gas dynamics dotted with itself.  The resulting model contains the entire solution at every time point and every space point between start and end.\n\n## Installation\n```bash\nuv sync\n```\nTo include dev dependencies:\n```bash\nuv sync --group dev\n```\n\n## Quickstart\nRun an example:\n```bash\nuv run python examples/high_order_euler.py\n```\nRun a different example:\n```bash\nuv run python examples/high_order_euler_adaptive.py\n```\nRun the SIRENS-style example:\n```bash\nuv run python examples/sirens_euler.py\n```\nRun tests:\n```bash\nuv run pytest -q\n```\n\n\u003cimg src=\"images/euler1d-network.png\" width=100% height=100% style=\"display: block; margin: 0 auto\"\u003e\n\n## Euler equations\n\n$$ \n\\frac{\\partial}{\\partial t} \\left[ \\begin{array}{c} \n\\rho \\\\\n\\rho v \\\\\ne \\\\\n\\end{array}\\right]+\n\\frac{\\partial}{\\partial x} \\left[\n\\begin{array}{c}\n\\rho v \\\\\n\\rho v^{2}+p \\\\\n\\left(e+p\\right)v\n\\end{array}\\right]\n=r\n$$\n\nwhere r=0.  The PDE is used as the loss function and r is the residual so that $loss=r\\cdot r$.\n\n\n## Sod Shock\n\n\"Solutions\" to the Sod Shock problem in fluid dynamics, which is 1 dimensional ideal compressible gas dynamics (euler equations), a very classic problem.  Solutions need to be much better (and faster) before I try and extend to much more complicated systems. I'll try and improve on this as I have time.\n\n## Fourier layers\nThe solution at t=0 should be a step function.  The model need to learn the initial conditions as well as all\nthe boundary conditions and the interior solution.  You can see here the initial condition is not perfect.\n\n![Sod Shock Density](images/Density-fourier.png)\n![Sod Shock Velocity](images/Velocity-fourier.png)\n\n### Continuous piecewise polynomial layers\nThis is actually the 5th order polynomial (not 6th) using 12 layers and mesh refinement. Shocks aren't sharp enough to call it good.\n```\npython examples/high_order_euler.py mlp.hidden.width=20 max_epochs=10000 mlp.segments=2 mlp.n=2 mlp.hidden.layers=12 factor=0.025 mlp.layer_type=continuous optimizer.patience=200000 mlp.input.segments=10 batch_size=2048 form=primitive loss_weight.discontinuity=0.0 loss_weight.interior=1.0 optimizer=adam mlp.normalize=maxabs mlp.rotations=4 gradient_clip=0.0 loss_weight.boundary=10 loss_weight.initial=10 data_size=10000 mlp.resnet=False refinement.type=p_refine refinement.epochs=500 refinement.target_n=6 refinement.start_n=2\n```\n![Sod Shock Density](images/Density-continuous.png)\n![Sod Shock Pressure](images/Pressure-continuous.png)\n![Sod Shock Velocity](images/Velocity-continuous.png)\n\n### Discontinuous piecewise polynomial layers\n\nIn this case it gets\nthe initial condition almost exactly right and can produce genuine discontinuities, but it's clearly showing many wrong and entropy violating shocks (not to mention not conserving mass etc...).  I need to add some penalty function here to prevent bad shocks.  This one converged the fastest by far.  The wave reflects\nat the boundary as I'm using dirichlet bcs for now.  I believe this is the best long term approach if I can get rid of these various problems.\n\n![Sod Shock Density](images/Density-discontinuous.png)\n![Sod Shock Velocity](images/Velocity-discontinuous.png)\n\n### Following [SIRENS](https://arxiv.org/abs/2006.09661) style network\nSIRENS is just a standard MLP with a sinusoidal positional embedding at the input and sin wave activation functions.  In\nthis particular case the model has 8 hidden layers where each hidden layer is 100 units wide.\n\n![Sod Shock Density](images/Density-SIRENS.png)\n![Sod Shock Velocity](images/Velocity-SIRENS.png)\n\n## Good parameters - still no contact discontinuity\nUsing quadratic polynomial creates a sharper shock. Using a large number of input segments (20) and then\njust 2 for each following layer produces smooth results (no contact discontinuity though!). Primitive form\nof the equations seems to be working better than conservative form - conservative form pushes to the initial condition even when dropping loss weight at shocks (possible I've introduced another error in my code).\n```\npython examples/high_order_euler.py mlp.hidden.width=40 max_epochs=10000 mlp.segments=2 mlp.n=3 mlp.hidden.layers=2 factor=0.1 mlp.layer_type=continuous optimizer.patience=200 mlp.input.segments=20 batch_size=512 form=primitive loss_weight.discontinuity=1.0 loss_weight.interior=0.1 optimizer=adamw\n```\nand, adding additional rotations\n```\npython examples/high_order_euler.py mlp.hidden.width=40 max_epochs=10000 mlp.segments=2 mlp.n=3 mlp.hidden.layers=2 factor=0.1 mlp.layer_type=discontinuous optimizer.patience=200 mlp.input.segments=20 batch_size=512 form=primitive loss_weight.discontinuity=1.0 loss_weight.interior=0.1 optimizer=adamw mlp.normalize=maxabs mlp.rotations=4\n```\nbetter (starting to see a contact discontinuity - unfortunately it's showing up in pressure as well)\n```\npython examples/high_order_euler.py mlp.hidden.width=40 max_epochs=10000 mlp.segments=4 mlp.n=4 mlp.hidden.layers=4 factor=0.025 mlp.layer_type=continuous optimizer.patience=200 mlp.input.segments=20 batch_size=512 form=primitive loss_weight.discontinuity=0.0 loss_weight.interior=1.0 optimizer=adam mlp.normalize=maxabs mlp.rotations=4 gradient_clip=5.0 loss_weight.boundary=10 loss_weight.initial=10 data_size=10000\n```\nmore better - note that adamw seems to work faster as well as larger batch size batch_size=2048 \u003e 512 \u003e 128. Number of rotations doesn't seem\nto be a huge issue as long as its \u003e 1.\n```\npython examples/high_order_euler.py mlp.hidden.width=20 max_epochs=10000 mlp.segments=2 mlp.n=3 mlp.hidden.layers=3 factor=0.025 mlp.layer_type=continuous optimizer.patience=1000 mlp.input.segments=20 batch_size=2048 form=primitive loss_weight.discontinuity=0.0 loss_weight.interior=1.0e-1 optimizer=adamw mlp.normalize=maxabs mlp.rotations=6 gradient_clip=5.0 loss_weight.boundary=10 loss_weight.initial=10 data_size=10000 mlp.resnet=False\n```\nwith polynomial refinement\n```\npython examples/high_order_euler.py mlp.hidden.width=20 max_epochs=10000 mlp.segments=4 mlp.n=3 mlp.hidden.layers=8 factor=0.025 mlp.layer_type=continuous optimizer.patience=200 mlp.input.segments=20 batch_size=2048 form=primitive loss_weight.discontinuity=0.0 loss_weight.interior=1.0e-1 optimizer=adamw mlp.normalize=maxabs mlp.rotations=6 gradient_clip=5.0 loss_weight.boundary=10 loss_weight.initial=10 data_size=10000 mlp.resnet=False refinement.type=p_refine refinement.epochs=1000\n```\nthis one actually got the contact discontinuity, but the velocity is wrong\n```\npython examples/high_order_euler.py mlp.hidden.width=10 max_epochs=10000 mlp.segments=2 mlp.n=3 mlp.hidden.layers=8 factor=0.025 mlp.layer_type=continuous optimizer.patience=200 mlp.input.segments=20 batch_size=2048 form=primitive loss_weight.discontinuity=0.0 loss_weight.interior=1.0e-1 optimizer=adamw mlp.normalize=maxabs mlp.rotations=4 gradient_clip=5.0e-1 loss_weight.boundary=10 loss_weight.initial=10 data_size=10000 mlp.resnet=False refinement.type=p_refine refinement.epochs=1000\n```\nanother with contact discontinuity and correct velocity, 8 layers and 10th order polynomial using refinement for subsequent initialization of each model.\n```\npython examples/high_order_euler.py mlp.hidden.width=20 max_epochs=10000 mlp.segments=2 mlp.n=2 mlp.hidden.layers=8 factor=0.025 mlp.layer_type=continuous optimizer.patience=200 mlp.input.segments=10 batch_size=2048 form=primitive loss_weight.discontinuity=0.0 loss_weight.interior=1.0e-1 optimizer=adamw mlp.normalize=maxabs mlp.rotations=4 gradient_clip=5.0e-1 loss_weight.boundary=10 loss_weight.initial=10 data_size=10000 mlp.resnet=False refinement.type=p_refine refinement.epochs=500 refinement.target_n=11\n```\nThis one produced a pretty solid rarefaction wave, could be steeper (developed at 6th order) using \"waves\" approach.\n```\npython examples/high_order_euler.py mlp.hidden.width=20 max_epochs=10000 mlp.segments=2 mlp.n=2 mlp.hidden.layers=12 factor=0.025 mlp.layer_type=continuous optimizer.patience=200 mlp.input.segments=10 batch_size=2048 form=primitive loss_weight.discontinuity=0.0 loss_weight.interior=1.0e-1 optimizer=adamw mlp.normalize=maxabs mlp.rotations=4 gradient_clip=5.0e-1 loss_weight.boundary=10 loss_weight.initial=10 data_size=10000 mlp.resnet=False refinement.type=p_refine refinement.epochs=500 refinement.target_n=20 refinement.start_n=2\n```\nback to adam, which works fine with refinement at high order and with gradient clipping turned off.  May be problematic when doing high order without initializing from a lower order solution.  In this all the shock structure is there, shocks are just too smoothed out.\n```\npython examples/high_order_euler.py mlp.hidden.width=20 max_epochs=10000 mlp.segments=2 mlp.n=2 mlp.hidden.layers=8 factor=0.025 mlp.layer_type=continuous optimizer.patience=200000 mlp.input.segments=10 batch_size=2048 form=primitive loss_weight.discontinuity=0.0 loss_weight.interior=1.0e-1 optimizer=adam mlp.normalize=maxabs mlp.rotations=4 gradient_clip=0.0 loss_weight.boundary=10 loss_weight.initial=10 data_size=10000 mlp.resnet=False refinement.type=p_refine refinement.epochs=500 refinement.target_n=6 refinement.start_n=2\n```\n### With switch layer (gating)\nNothing to see below, runs but doesn't converge. Probably some different normalization.\n```\npython examples/high_order_euler.py mlp.hidden.width=20 max_epochs=10000 mlp.segments=2 mlp.n=5 mlp.hidden.layers=1 mlp.layer_type=switch_discontinuous optimizer.patience=200000 mlp.input.segments=10 batch_size=2048 form=primitive loss_weight.discontinuity=0.0 loss_weight.interior=1.0e-1 optimizer=adam mlp.normalize=maxabs mlp.rotations=4 gradient_clip=0.0 loss_weight.boundary=10 loss_weight.initial=10 data_size=10000 mlp.resnet=True\n```\n## Training\nHigh order MLP\n```\npython examples/high_order_euler.py\n```\nStandard MLP following [SIRENS](https://arxiv.org/abs/2006.09661) with sin waves and sin wave positional embedding.\n```\npython examples/sirens_euler.py\n```\n## Plotting\n```\npython examples/high_order_euler.py checkpoint=\\\"outputs/2021-04-25/18-54-45/lightning_logs/version_0/checkpoints/'epoch=0.ckpt'\\\" train=False\n```\n## Running tests\n```\npytest tests\n```\n\n## Papers of interest\nPapers for me to read solving similar problems\n\n[Solving Hydrodynamic Shock-Tube Problems Using Weighted Physics-Informed Neural Networks with Domain Extension](https://www.researchgate.net/publication/350239546_Solving_Hydrodynamic_Shock-Tube_Problems_Using_Weighted_Physics-Informed_Neural_Networks_with_Domain_Extension)\n\n[Discontinuity Computing using Physics Informed Neural Networks](https://www.researchgate.net/profile/Li-Liu-72/publication/359480166_Discontinuity_Computing_using_Physics-Informed_Neural_Networks/links/6312a7da5eed5e4bd1404c25/Discontinuity-Computing-using-Physics-Informed-Neural-Networks.pdf)\n\n[ACTIVE LEARNING BASED SAMPLING FOR HIGH-DIMENSIONAL NONLINEAR PARTIAL DIFFERENTIAL EQUATIONS](https://arxiv.org/pdf/2112.13988.pdf)\n\n[Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems]()\n\n[Flow Equivariant Recurrent Neural Networks](https://arxiv.org/html/2507.14793v2)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjloveric%2Fneural-network-pdes","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjloveric%2Fneural-network-pdes","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjloveric%2Fneural-network-pdes/lists"}