{"id":24809684,"url":"https://github.com/ilyaorson/control-neuralode","last_synced_at":"2025-10-08T11:35:35.270Z","repository":{"id":54598541,"uuid":"311203006","full_name":"IlyaOrson/Control-NeuralODE","owner":"IlyaOrson","description":"Neural ODEs as Feedback Policies for Nonlinear Optimal Control (IFAC 2023) https://doi.org/10.1016/j.ifacol.2023.10.1248","archived":false,"fork":false,"pushed_at":"2023-08-10T09:10:09.000Z","size":925,"stargazers_count":15,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-01-21T03:14:35.938Z","etag":null,"topics":["adjoint-sensitivities","automatic-differentiation","julia","neural-networks","neuralode","optimal-control","optimization","ordinary-differential-equations"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2210.11245","language":"Julia","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/IlyaOrson.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.bib","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-11-09T02:31:15.000Z","updated_at":"2025-01-11T21:02:28.000Z","dependencies_parsed_at":"2025-01-21T03:14:59.046Z","dependency_job_id":"98a35d38-c6e9-41d9-bdb4-92a1e6255211","html_url":"https://github.com/IlyaOrson/Control-NeuralODE","commit_stats":null,"previous_names":["ilyaorson/control_neuralode"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IlyaOrson%2FControl-NeuralODE","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IlyaOrson%2FControl-NeuralODE/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IlyaOrson%2FControl-NeuralODE/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IlyaOrson%2FControl-NeuralODE/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/IlyaOrson","download_url":"https://codeload.github.com/IlyaOrson/Control-NeuralODE/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":236337769,"owners_count":19132985,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adjoint-sensitivities","automatic-differentiation","julia","neural-networks","neuralode","optimal-control","optimization","ordinary-differential-equations"],"created_at":"2025-01-30T11:15:10.329Z","updated_at":"2025-10-08T11:35:30.227Z","avatar_url":"https://github.com/IlyaOrson.png","language":"Julia","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Feedback Control Policies with Neural ODEs\n\nThis code showcases how a state-feedback neural policy, as commonly used in reinforcement\nlearning, may be used similarly in an optimal control problem while enforcing state and control constraints.\n\n## Constrained Van der Pol problem\n$$\\begin{equation}\n    \\begin{aligned}\n        \\min_{\\theta} \\quad \u0026 J = \\int_0^5 x_1^2 + x_2^2 + u^2 \\,dt, \\\\\n                           \u0026 \\\\\n        \\textrm{s.t.} \\quad \u0026 \\dot x_1(t) = x_1(1-x_2^2) - x_2 + u, \\\\\n                            \u0026 \\dot x_2(t) = x_1, \\\\\n                            \u0026 u(t) = \\pi_\\theta^2(x_1, x_2) \\\\\n                            \u0026 \\\\\n                            \u0026 x(t_0) = (0, 1), \\\\\n                            \u0026 \\\\\n                            \u0026 x_1(t) + 0.4 \\geq 0, \\\\\n                            \u0026 -0.3 \\leq u(t) \\leq 1, \\\\\n    \\end{aligned}\n\\end{equation}$$\n\n### Phase Space with embedded policy (before and after optimization)\n\n![vdp_initial](https://github.com/IlyaOrson/control_neuralode/assets/12092488/1cbe6b23-71bd-4f5d-8f5a-7090ab4b4cd8)\n![vdp_constrained](https://github.com/IlyaOrson/control_neuralode/assets/12092488/3d86f7a7-3b92-4495-9940-7dbc4813037d)\n\n## JuliaCon 2021\nThe main ideas where presented in [this talk](https://www.youtube.com/watch?v=omS3ZngEygw) of JuliaCon2021.\n\n[![Watch the video](https://img.youtube.com/vi/omS3ZngEygw/maxresdefault.jpg)](https://www.youtube.com/watch?v=omS3ZngEygw)\n\n## Running the study cases\nThis code requires Julia 1.7.\n\nReproduce the environment with the required dependencies:\n```julia\njulia\u003e using Pkg; Pkg.activate(;temp=true)\njulia\u003e Pkg.add(url=\"https://github.com/IlyaOrson/ControlNeuralODE.jl\")\n```\n\nRun the test cases:\n\n```julia\njulia\u003e using ControlNeuralODE: van_der_pol, bioreactor, batch_reactor, semibatch_reactor\njulia\u003e van_der_pol(store_results=true)\n```\n\nThis will generate plots while the optimization runs and store result data in `data/`.\n\n## Methodology (Control Vector Iteration for the Neural Policy parameters)\nBy substituting the control function of the problem by the output of the policy, the\nweights of the controller become the new unconstrained controls of the system.\nThe problem becomes a parameter estimation problem where the Neural ODE adjoint method may be used\nto backpropagate sensitivities with respect to functional cost.\n\nThis method was originally called the _Kelley-Bryson_ gradient procedure (developed in the 60s);\nwhich is historically interesting due to being one of the earliest uses of backpropagation.\nIts continuous time extension is known as _Control Vector Iteration_ (CVI) in the optimal control\nliterature, where it shares the not so great reputation of indirect methods.\n\nIts modern implementation depends crucially on automatic differentiation to avoid the manual\nderivations; one of the features that made the original versions unattractive.\nThis is where `DiffEqFlux.jl` and similar software shine. See the publication for a clear explanation of the technical details.\n\n# Acknowledgements\nThe idea was inspired heavily by the trebuchet demo of Flux and the differentiable control\nexample of [DiffEqFlux](https://github.com/SciML/DiffEqFlux.jl). A similar idea was contrasted with reinforcement learning in [this work](https://github.com/samuela/ctpg). Chris Rackauckas advise was very useful.\n\n## Citation\n\nIf you find this work helpful please consider citing the following paper:\n```bibtex\n@misc{https://doi.org/10.48550/arxiv.2210.11245,\n  doi = {10.48550/ARXIV.2210.11245},\n  url = {https://arxiv.org/abs/2210.11245},\n  author = {Sandoval, Ilya Orson and Petsagkourakis, Panagiotis and del Rio-Chanona, Ehecatl Antonio},\n  keywords = {Optimization and Control (math.OC), Artificial Intelligence (cs.AI), Systems and Control (eess.SY), FOS: Mathematics, FOS: Mathematics, FOS: Computer and information sciences, FOS: Computer and information sciences, FOS: Electrical engineering, electronic engineering, information engineering, FOS: Electrical engineering, electronic engineering, information engineering},\n  title = {Neural ODEs as Feedback Policies for Nonlinear Optimal Control},\n  publisher = {arXiv},\n  year = {2022},\n  copyright = {arXiv.org perpetual, non-exclusive license}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Filyaorson%2Fcontrol-neuralode","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Filyaorson%2Fcontrol-neuralode","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Filyaorson%2Fcontrol-neuralode/lists"}