{"id":29193341,"url":"https://github.com/jktujq/pinn","last_synced_at":"2025-07-02T02:09:09.214Z","repository":{"id":259528163,"uuid":"860611627","full_name":"JktuJQ/PINN","owner":"JktuJQ","description":"This project attempts to solve Duffing equation using PINN and numerical methods.","archived":false,"fork":false,"pushed_at":"2025-04-03T21:40:48.000Z","size":3741,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-03T22:29:57.461Z","etag":null,"topics":["machine-learning","maths","neural-networks","python","simulation","torch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JktuJQ.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-20T19:09:51.000Z","updated_at":"2025-04-03T21:40:51.000Z","dependencies_parsed_at":"2024-10-26T08:14:43.153Z","dependency_job_id":"cbb9dcfa-6d14-4ffc-92dd-32afcc697edb","html_url":"https://github.com/JktuJQ/PINN","commit_stats":null,"previous_names":["jktujq/pinn"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/JktuJQ/PINN","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JktuJQ%2FPINN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JktuJQ%2FPINN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JktuJQ%2FPINN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JktuJQ%2FPINN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JktuJQ","download_url":"https://codeload.github.com/JktuJQ/PINN/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JktuJQ%2FPINN/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263061402,"owners_count":23407606,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["machine-learning","maths","neural-networks","python","simulation","torch"],"created_at":"2025-07-02T02:09:08.472Z","updated_at":"2025-07-02T02:09:09.191Z","avatar_url":"https://github.com/JktuJQ.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PINN (Physics-Informed Neural Networks)\n\nThis project attempts to solve *Duffing equation* using **PINN** and numerical methods.\n\n## Project goals\n\n* Implement [PINN](https://en.wikipedia.org/wiki/Physics-informed_neural_networks)\n  method for solving [Cauchy problem](https://en.wikipedia.org/wiki/Cauchy_problem)\n  of [Duffing equation](https://en.wikipedia.org/wiki/Duffing_equation)\n* Compare results with numerical methods\n* Research relation between behaviour of system and its parameters\n\n## Task formulation\n\n**[Duffing equation](https://en.wikipedia.org/wiki/Duffing_equation)**\\\n$` \\frac{d^2x}{dt^2}+\\delta \\frac{dx}{dt} + \\alpha x + \\beta x^3 = \\gamma \\cos(\\omega t), \\quad t\u003e0 `$\n\n*Cauchy data* of the problem - $` x(0) = x_0, \\frac{dx}{dt}(0) = v_0 `$\n\n*Parameters* of equation - $`\\alpha, \\beta, \\gamma, \\delta, \\omega`$\n\n## Numerical methods\n\n3 numerical methods were implemented:\n\n* [Euler method](https://en.wikipedia.org/wiki/Euler_method) (first-order numerical procedure)\n* [Trapezoidal method](https://en.wikipedia.org/wiki/Trapezoidal_rule) (second-order numerical procedure)\n* [Runge-Kutta method](https://en.wikipedia.org/wiki/Runge%E2%80%93Kutta_methods) (fourth-order numerical procedure)\n\nNumerical methods were tested on the following differential equation with analytic\nsolution - $` \\ddot x + \\alpha x = 0 \\left(x_0 = 1, v_0 = 0, t_0 = 0\\right) `$\n\nLog-log plot of numerical methods errors\n\n![Log-log plot of numerical methods errors](docs/images/log-log_plot_of_numerical_methods_errors.png)\n\n## PINN\n\nFor approximating equations whose solution is unknown,\nconventional types of neural networks are not applicable -\nit is impossible to calculate the error of prediction.\n\n[PINN](https://en.wikipedia.org/wiki/Physics-informed_neural_networks)\nis a type of neural network for approximating differential equations that has a special error function -\nit is calculated by substituting the predicted values into the original differential equation,\ntaking into account Cauchy data of the equation.\n\n$`\nLOSS = \\frac{1}{N} \\sum_{i=1}^{N}\n(x''_i + \\delta x'_i + \\alpha x + \\beta x_i^3 - \\gamma \\cos \\left( \\omega t_i \\right))) +\n\\lambda \\left(\\left(x(0) - f(0)\\right)^2 + \\left(x'(0) - f'(0)\\right)^2 \\right)\n`$\n\n### Neural network configurations\n\nThere were 3 configurations of neural network that were tested during research.\n\nTraining of models for tests was performed for 256 epochs.\n\n#### Configuration №1\n\n|                     | Layer 1 | Layer 2 | Layer 3 | Layer 4 | Layer 5 | Layer 6 |\n|:--------------------|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|\n| Neurons             |   32    |   32    |   32    |   32    |   32    |    1    |\n| Activation function |  `sin`  |  `sin`  | `tanh`  | `tanh`  |  `sin`  |  `id`   |\n\n| Max error |  MSE   |\n|:---------:|:------:|\n|   0.068   | 1.8e-7 |\n\n#### Configuration №2\n\n|                     | Layer 1 | Layer 2 |  Layer 3   | Layer 4 | Layer 5 | Layer 6 |\n|:--------------------|:-------:|:-------:|:----------:|:-------:|:-------:|:-------:|\n| Neurons             |   32    |   32    |     32     |   32    |   32    |    1    |\n| Activation function |  `sin`  | `tanh`  | `softplus` | `tanh`  |  `sin`  |  `id`   |\n\n| Max error |   MSE   |\n|:---------:|:-------:|\n|   0.059   | 1.39e-7 |\n\n#### Configuration №3\n\n|                     | Layer 1 | Layer 2 |  Layer 3   | Layer 4 | Layer 5 | Layer 6 |\n|:--------------------|:-------:|:-------:|:----------:|:-------:|:-------:|:-------:|\n| Neurons             |   64    |   64    |     64     |   64    |   64    |    1    |\n| Activation function |  `sin`  | `tanh`  | `softplus` | `tanh`  |  `sin`  |  `id`   |\n\n| Max error |   MSE   |\n|:---------:|:-------:|\n|   0.054   | 1.18e-7 |\n\n## Research of parameters\n\nBy setting some parameters of the Duffing equation to zero,\nit is transformed into the equations of already\nknown [oscillatory systems]((https://en.wikipedia.org/wiki/Harmonic_oscillator)).\n\nThus, it is possible to identify influence of the equation parameters\non the behavior of the oscillator using said method.\n\n### $` \\alpha `$ parameter\n\nFor given parameters $` \\beta = \\delta = \\gamma = \\omega = 0 `$,\nthe Duffing equation describes simple harmonic oscillations - $` \\ddot x + \\alpha x = 0 `$.\n\nWe can conclude that $` \\alpha `$ defines linear stiffness of oscillator.\n\n#### Plots and phase portraits\n\n|                 $` \\alpha = 1 `$ plot                 |                      $` \\alpha = 1 `$ phase portrait                      |\n|:-----------------------------------------------------:|:-------------------------------------------------------------------------:|\n| ![$` \\alpha = 1 `$ plot](docs/images/alpha1_plot.png) | ![$` \\alpha = 1 `$ phase portrait](docs/images/alpha1_phase_portrait.png) |\n\n|                 $` \\alpha = 0 `$ plot                 |\n|:-----------------------------------------------------:|\n| ![$` \\alpha = 0 `$ plot](docs/images/alpha0_plot.png) |\n\n|                 $` \\alpha = -3 `$ plot                  |                      $` \\alpha = -3 `$ phase portrait                       |\n|:-------------------------------------------------------:|:---------------------------------------------------------------------------:|\n| ![$` \\alpha = -3 `$ plot](docs/images/alpha-3_plot.png) | ![$` \\alpha = -3 `$ phase portrait](docs/images/alpha-3_phase_portrait.png) |\n\n### $` \\delta `$ parameter\n\nFor given parameter $` \\beta = 0 `$,\nthe Duffing equation describes damped oscillations - $` \\ddot x + \\delta \\dot x + \\alpha x = 0 `$.\n\nWe can conclude that $` \\delta `$ represents degree of oscillation damping.\n\n#### Plots\n\n|                  $` \\delta = 0.5 `$ plot                  |\n|:---------------------------------------------------------:|\n| ![$` \\delta = 0.5 `$ plot](docs/images/delta0.5_plot.png) |\n\n|                  $` \\delta = 0.03 `$ plot                   |\n|:-----------------------------------------------------------:|\n| ![$` \\delta = 0.03 `$ plot](docs/images/delta0.03_plot.png) |\n\n### $` \\beta `$ parameter\n\nFor given parameter $` \\gamma = 0 `$,\nthe Duffing equation describes non-linear dynamic oscillation system -\n$` \\ddot x + \\delta \\dot x + \\alpha x + \\beta x^3 = 0 `$.\n\nWe can conclude that $` \\beta `$ represents degree of non-linearity of driving force.\n\n#### Plots and phase portraits\n\nAfter some research, it was discovered that if $` \\beta `$ parameter is non-null,\nPINN has serious problems with approximating Duffing equation.\n\nThis is due to the fact that this coefficient has a decisive influence\non the possibility of the system transitioning to a state of dynamic chaos,\nand also affects the non-linear properties of the system.\n\n|                   $` \\beta \\neq 0 `$ plot                    |                        $` \\beta \\neq 0 `$ phase portrait                         |\n|:------------------------------------------------------------:|:--------------------------------------------------------------------------------:|\n| ![$` \\beta \\neq 0 `$ plot](docs/images/betanonnull_plot.png) | ![$` \\beta \\neq 0 `$ phase portrait](docs/images/betanonnull_phase_portrait.png) |\n\n$` \\beta `$ parameters also impose problems on numerical methods, but\nsince we can specify arbitrary precision and\nchoose different methods decent approximation is still possible.\n\n#### Stability of solutions\n\nAlthough $` \\beta `$ parameter severely complicates equation,\nwe can still research its behaviour using knowledge\nfrom [Stability theory](https://en.wikipedia.org/wiki/Stability_theory).\n\n[Hooke's law](https://en.wikipedia.org/wiki/Hooke%27s_law) for linear oscillation system -\n$` F = - \\alpha x `$.\nTransformed version of that law - $` F = - \\alpha x - \\beta x^3 = -x \\left(\\alpha - \\beta x^2 \\right) `$.\n\nNow it is possible to identify equilibrium points - points at which $` F = 0 `$.\n\nZero is always an equilibrium point; but if $` \\alpha \\lt 0 `$ or\n$` \\beta \\lt 0 `$ with big enough `x` value, then zero is unstable point,\notherwise it is asymptotically stable.\nIf there is a real solution for the $` \\left(\\alpha - \\beta x^2 \\right) = 0 `$ equation,\nthen that solution is an asymptotically stable point.\n\n|                              $` \\alpha \\lt 0 `$, $` \\beta \\gt 0 `$ phase portrait                               |\n|:---------------------------------------------------------------------------------------------------------------:|\n| ![$` \\alpha \\lt 0 `$, $` \\beta \\gt 0 `$ phase portrait](docs/images/alphaless0_betagreater0_phase_portrait.png) |\n\n|                                $` \\alpha \\gt 0 `$, $` \\beta \\gt 0 `$ phase portrait                                |\n|:------------------------------------------------------------------------------------------------------------------:|\n| ![$` \\alpha \\gt 0 `$, $` \\beta \\gt 0 `$ phase portrait](docs/images/alphagreater0_betagreater0_phase_portrait.png) |\n\n### $` \\gamma `$ and $` \\omega `$ parameters\n\nFor given parameter $` \\delta = 0 `$,\nthe Duffing equation describes driven oscillation system -\n$` \\ddot x + \\alpha x + \\beta x^3 = \\gamma \\cos \\left( \\omega t \\right) `$.\n\nWe can conclude that $` \\gamma `$ represents amplitude of periodic driving force and\n$` \\omega `$ represents angular frequency of a periodic driving force.\n\n#### Plots\n\n|                      $` \\gamma = 1.5 `$, $` \\omega = 1.2 `$ plot                       |\n|:--------------------------------------------------------------------------------------:|\n| ![$` \\gamma = 1.5 `$, $` \\omega = 1.2 `$ plot](docs/images/gamma1.5_omega1.2_plot.png) |\n\n|                      $` \\gamma = 0.5 `$, $` \\omega = 0.5 `$ plot                       |\n|:--------------------------------------------------------------------------------------:|\n| ![$` \\gamma = 0.5 `$, $` \\omega = 0.5 `$ plot](docs/images/gamma0.5_omega0.5_plot.png) |\n\n### General form of the equation\n\nWhen all parameters are non-null\n($` \\frac{d^2x}{dt^2}+\\delta \\frac{dx}{dt} + \\alpha x + \\beta x^3 = \\gamma \\cos(\\omega t), \\quad t\u003e0 `$),\nwe can research behaviour of Duffing equation by examining its\n[frequency response](https://en.wikipedia.org/wiki/Frequency_response).\n\nIf non-linearity is neglected,\nthe Duffing equation becomes a non-homogeneous linear differential equation,\nthe solution of which can be sought in the form\n$` x = a \\cos \\left( \\omega t \\right) + b \\sin \\left( \\omega t \\right) = A \\cos \\left( \\omega t - \\varphi_0 \\right) `$.\n\nSubstituting that solution into the original equation,\nwe find relation between coordinate amplitude and angular frequency of a periodic driving force -\n$` \\left[ \\left( \\omega^2 - \\alpha - \\frac{3}{4} \\beta A^2 \\right) + \\delta^2 \\omega^2 \\right] A^2 = \\gamma^2 `$.\n\n#### Plots\n\n|                      2D frequency response                      |\n|:---------------------------------------------------------------:|\n| ![2D frequency response](docs/images/2d_frequency_response.png) |\n\n|                      3D frequency response                      |\n|:---------------------------------------------------------------:|\n| ![3D frequency response](docs/images/3d_frequency_response.png) |\n\n#### Chaos state\n\nThe frequency response is directly related to\nthe [frequency hysteresis](https://en.wikipedia.org/wiki/Hysteresis)\nand bifurcations of the dynamic system.\n\nPrevious analysis assumed that the oscillator's natural frequency dominates\nand the non-linearity of the restoring force is negligible.\nIf the driving force is strong enough, the system's behavior will be chaotic.\n\n|                                  $` \\alpha = -1 `$, $` \\beta = 1 `$, $` \\gamma = 0.5 `$, $` \\delta = 0.3 `$, $` \\omega = 1.2 `$ plot                                  |                                      $` \\alpha = -1 `$, $` \\beta = 1 `$, $` \\gamma = 0.5 `$, $` \\delta = 0.3 `$, $` \\omega = 1.2 `$ phase portrait                                       |\n|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|\n| ![$` \\alpha = -1 `$, $` \\beta = 1 `$, $` \\gamma = 0.5 `$, $` \\delta = 0.3 `$, $` \\omega = 1.2 `$ plot](docs/images/alpha-1_beta1_gamma0.5_delta0.3_omega1.2_plot.png) | ![$` \\alpha = -1 `$, $` \\beta = 1 `$, $` \\gamma = 0.5 `$, $` \\delta = 0.3 `$, $` \\omega = 1.2 `$ phase portrait](docs/images/alpha-1_beta1_gamma0.5_delta0.3_omega1.2_phase_diagram.png) |\n\n## Conclusion\n\nAfter working and doing research with PINN I have come up with following conclusions:\n\n* Implementing PINN is much harder than implementing numerical methods\ndue to non-deterministic nature of neural networks.\nIt is hard to come up with reliable configuration that will fit the task,\nand there are no guarantees that\nthere is no corner case that neural network will fail to handle.\n\n* Training of neural networks takes more time than approximation with numerical methods even for small steps,\neven with GPU usage.\n\n* One advantage of neural networks over numerical methods is that function that they produce is continuous,\nand you are not constrained to the grid of coordinates.\nThat is not a big advantage, since you can just take mean value for point between grid points (basically trapezoid method).\n\nAll in all, although physics-informed neural networks are an interesting extension of common neural network architecture,\ntheir application is somewhat questionable due to several downsides of neural networks.\nMost of the time you are better off using simple and reliable numerical methods for fast prototyping and research.\n\n## Usage of project\n\nAlthough this project has no interactive component,\nyou can still use it to try out PINN and to do your own research of Duffing equation.\n\n[constants.py](constants.py) file contains definitions of equation parameters, which you can change.\n[main.py](main.py) initializes all systems and runs training of neural network.\nIt also tries to obtain true solution of your equation:\n\n```python\nreal_solution = duffing.harmonic_oscillator(t.to(DEVICE))\n```\n\nTrue solution is needed to record metrics of neural network and numerical solution.\n\nSince you do not know true solution for most of the configurations of parameters,\nyou might want to pass anything that resembles true solution and just ignore metrics like `MSE` and `max error`,\nbut if you know analytic solution you can substitute it\n([duffing.py](duffing.py) comes with some solutions for cases that were described in this research).\n\nAfter that, you are good to go with your research.\n\n## Contributions\n\nFeel free to star this repository if you liked my research or if you are interested in it;\nin case of latter you are also welcome to contact me with your suggestions or questions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjktujq%2Fpinn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjktujq%2Fpinn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjktujq%2Fpinn/lists"}