{"id":18519712,"url":"https://github.com/fedebotu/optimal-control-exercises","last_synced_at":"2025-04-09T08:32:19.183Z","repository":{"id":108871463,"uuid":"323290136","full_name":"fedebotu/optimal-control-exercises","owner":"fedebotu","description":"Exercise solutions for Daniel Liberzon's \"Calculus of Variations and Optimal Control Theory: A Concise Introduction\" book and other optimal control problems","archived":false,"fork":false,"pushed_at":"2021-10-14T06:41:48.000Z","size":1711,"stargazers_count":23,"open_issues_count":0,"forks_count":8,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-03-24T04:06:00.201Z","etag":null,"topics":["coursework","exercise-solutions","hamiltonian","inverted-pendulum","kaist","lagrangian","lqr","ode","optimal-control","optimal-control-exercises"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fedebotu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2020-12-21T09:34:10.000Z","updated_at":"2025-02-15T15:41:49.000Z","dependencies_parsed_at":"2023-03-15T00:17:09.921Z","dependency_job_id":null,"html_url":"https://github.com/fedebotu/optimal-control-exercises","commit_stats":{"total_commits":22,"total_committers":2,"mean_commits":11.0,"dds":"0.13636363636363635","last_synced_commit":"45e760704bcfa54bf0f2e855629b7d98d5d6bf62"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fedebotu%2Foptimal-control-exercises","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fedebotu%2Foptimal-control-exercises/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fedebotu%2Foptimal-control-exercises/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fedebotu%2Foptimal-control-exercises/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fedebotu","download_url":"https://codeload.github.com/fedebotu/optimal-control-exercises/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248003331,"owners_count":21031772,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["coursework","exercise-solutions","hamiltonian","inverted-pendulum","kaist","lagrangian","lqr","ode","optimal-control","optimal-control-exercises"],"created_at":"2024-11-06T17:17:09.504Z","updated_at":"2025-04-09T08:32:18.556Z","avatar_url":"https://github.com/fedebotu.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Optimal Control Exercises\n[![made-with-latex](https://img.shields.io/badge/Made%20with-LaTeX-1f425f.svg)](https://www.latex-project.org/)\n\n## 🗂 Contents\nThis repo contains some exercise solutions for Daniel Liberzon's _Calculus of Variations and Optimal Control Theory: A Concise Introduction_ book and other problems.\n\n- [Lecture 1](#lecture-1): _Introduction_: path optimization vs. point optimization; basic analysis; basic facts from finite-dimensional optimization\n- [Lecture 2](#lecture-2): _Calculus of variations_: examples of variational problems; Euler-Lagrange equation; Hamiltonial formalism and Legendre\ntransformation; mechanical interpretation; constraints; second variation and Legendre’s necessary condition; weak and strong extrema; conjugate points and sufficient conditions\n- [Lecture 3](#lecture-3): _From Calculus of Variations to Optimal Control_ : statement of the optimal control problem; variational argument and preview of the maximum principle\n- [Lecture 4](#lecture-4): _The Maximum Principle_: statement and proof of the maximum principle; relation to Lie brackets; bang-bang and singular optimal\ncontrols; dynamic programming; sufficient conditions for optimality\n- [Lecture 5](#lecture-5): _The Hamilton-Jacobi-Bellman equation_: HJB equations; viscosity solutions of the HJB equation\n- [Lecture 6](#lecture-6): _The Linear Quadratic Regulator_: LQR problems\n\n### Note\nThe book is part of coursework for the Optimal Control course from professor Donghwan Lee at KAIST. Many exercises are taken from Liberzon's book, but not all of them - and many are not in order as well. I could not find a solution manual at the time, but searching in Google with certain keywords helped for some of the exercises. If you would like to improve the repo, please read the [Feedback](#mailbox_with_mail-feedback) section!\n\nThe version of the text is the following:\n\n[1] D. Liberzon, Calculus of Variations and Optimal Control Theory: A Concise Introduction. Princeton ; Oxford: Princeton University Press, 2012.\n\n## 🖼️ Examples\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/Juju-botu/optimal-control-exercises/blob/main/images/example_screenshot0.png\" width=600 alt=\"Example screenshot0\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/Juju-botu/optimal-control-exercises/blob/main/images/example_screenshot.png\" width=600  alt=\"Example screenshot\"\u003e\n\u003c/p\u003e\n\n## 📜 Exercise List\nHere is the list of exercises divided by lectures, so they can be more easily searched. Equations are left in Latex and will be hopefully displayed by Github in the future.\n\n# Lecture 1\n\nExercise 1.13\n-------------\n\n*Prove the above lemma: \\\"If scalar sequence is monotonically\nnondecreasing and bounded above, then it converges. Similarly, if a\nscalar sequence is monotonically nonincreasing and bounded below, then\nit converges.\\\"*\n\n\nExercise 1.15\n-------------\n\n*Devise a set X whose supremum exists and find its supremum. What is the\nmaximum of the set? Do the same for the infimum.*\n\nExercise 1.17\n-------------\n\n*Devise a sequence $(x_k)_{k=1}^{\\infty}$ whose upper limit exists and\nfind its upper limit. Do the same for the infimum.*\\\n\nExercise 1.24\n-------------\n\n*Devise examples of an open set and a closed set.*\\\n\n\nExercise 1.68\n-------------\n\n*Prove the two following propositions.*\\\n\\\n**Proposition 1 (First-order necessary condition for constrained\noptimality):** Suppose that $f$ is a $\\mathcal{C}^1$ (continuously\ndifferentiable) function and $x^*$ is its local minimum. Then,\n$$\\langle \\nabla f (x^*), d  \\rangle \\geq 0$$ for all feasible\ndirections $d$.\\\n**Proposition 2 (Second-order necessary condition for constrained\noptimality):** Suppose that $f$ is a $\\mathcal{C}^2$ (continuously\ndifferentiable) function and $x^*$ is its local minimum. Then,\n$$d^T \\nabla ^2 f(x^*) d \\geq 0$$ for all feasible directions $d$ such\nthat $$\\langle \\nabla f (x^*), d \\rangle = 0$$\\\n\nExercise 1.69\n-------------\n\n*Suppose that $f$ is a $\\mathcal{C}^2$ function and $x^*$ us a point of\nits domain at which we have $\\langle\\nabla f(x^*),d \\rangle \\geq 0$ and\n$d^T \\nabla^2 f(x^*)d \u003e 0$ for every nonzero feasible direction d. Is\n$x^*$ necessarily a local minimum of f? Prove or give a\ncounterexample.*\\\n\nExercise 1.78\n-------------\n\n*Give an example where a local minimum $x^*$ is not a regular point and\nthe above necessary condition (namely first-order necessary condition\nfor constrained optimality) is false (justify both of these claims).*\\\n\nExercise 1.81\n-------------\n\n*Prove that the set of continuous functions\n$f:[a,b] \\to \\mathbb{R}, f \\in \\mathcal{C}^0$ is a vector space with\ntypical addition and scalar multiplication.*\\\n\nExercise 1.84\n-------------\n\n*For $y \\in \\mathcal{C}^0$, define:*\n$$\\mid\\mid y \\mid \\mid _{\\mathcal{L}_p} := \\left( \\int_a^b \\mid y(x) \\mid ^p dx \\right) ^{1/p}$$\n*where $p$ is a positive integer and $a, b \\in \\mathbb{R}$ with $a \u003c b$.\nProve that it is an inner product on the vector space $\\mathbb{C}^0$*\\\n\nExercise 1.85\n-------------\n\n*For $x, y \\in \\mathcal{C}^0$, define*\n$$\\langle x, y \\rangle = \\int_a^b x(t)y(t) dt$$ *where\n$a, b \\in \\mathbb{R}$ with $a \u003c b$. Prove that it is an inner product on\nthe vector space $\\mathcal{C}^0$.*\\\n\nExercise 1.96\n-------------\n\n*Consider the space $\\mathcal{C}^0([0,1]) \\to \\mathbb{R}$, let\n$g : \\mathbb{R} \\to \\mathbb{R}$ be a $\\mathcal{C}^1$ function, and\ndefine the functional $J$ on $V$ by $J(y) = \\int_0^1 g'(y(x))dx$. Show\nthat its first variation exists and it is given by the formula*\n$$\\delta J\\med_y (\\eta) = \\int_0^1 g'(y(x)) \\eta (x) dx$$\\\n\n# Lecture 2\n\nExercise 2.4\n------------\n\n*Find another example of a variational problem. Describe it verbally\nfirst, then formalize it by specifying admissible curves and giving an\nexpression for the functional to be minimized or maximized. You do not\nneed to solve it.*\\\n\nExercise 2.21\n-------------\n\n*Find an extremal for the problem*\n$$\\min\\limits_{y:[0,1] \\to \\mathbb{R}} J(y) = \\int_0^1 y(x)^2 (y'(x))^2 dx \\hspace{0.5cm} \\text{subject to}\\hspace{0.5cm} y(0) = 0, y(1) = 1$$\\\n\nExercise 2.22\n-------------\n\n*Find an extremal for the problem*\n$$\\min\\limits_{y:[0,1] \\to \\mathbb{R}} J(y) = \\int_0^1 \\{ (y'(x))^2 + 2y(x)e^x \\} dx \\hspace{0.5cm} \\text{subject to} \\hspace{0.5cm} y(0) = 0, y(1) = 1$$\\\n\nExercise 2.27 (The Brachistochrone Problem)\n-------------------------------------------\n\n*Use the no $x$ result to show that extremals for the brachistochrone\nproblem are indeed given by*\n$$x(\\theta) = a +c (\\theta - sin(\\theta), \\hspace{0.5cm} y(\\theta) = c(1 - cos(\\theta))$$\n*where the parameter $\\theta$ takes values between 0 and $2\\pi$ and\n$c \u003e 0$ is constant.*\n\nExercise 2.30\n-------------\n\n*Confirm directly from the equations (2.10), (2.11), 2.12) (Hamilton's\ncanonical equations) that in the \\\"no y\\\" case $p$ is constant along\nextremals and in the \\\"no x\\\" case $H$ is constant along extremals.*\\\n\nExercise 2.36\n-------------\n\n*Find the curve $y^*$ that minimizes*\n$$J(y) = \\frac{1}{2} \\int_0^1 y^' (x)^2 dx$$ *subject to the constraint*\n$$\\int_0^1 y(x)dx = \\frac{1}{6}$$\\\n\nExercise 2.37 (Dido's Problem)\n------------------------------\n\n*Show that optimal curves for Dido's problem are circular arcs.*\\\n\nExercise 2.38 (The Catenary Problem)\n------------------------------------\n\n*Show that the optimal curves for the catenary problem satisfy*\n$$y(x) = c \\cosh{x/c}, \\hspace{0.5cm} c\u003e0$$\\\n\nExercise 2.47\n-------------\n\n*Is Legendre's necessary condition satisfied by the admissible extremal\nfor the problem minimizing*\n$$J(y) = \\int_0^2 \\sqrt{1 + y(x)^2 y'(x)^2} dx$$ *subject to $y(0) = 1$\nand $y(2) = 3$? Find $L_{y'y'}(x,y(x),y'(x))$ explicitly.*\\\n\n# Lecture 3\n\nExercise 3.24\n-------------\n\n*Consider the problem*\n$$u^* := \\argmin_{u: [0,1] \\to \\mathbb{R}} J(u) := \\frac{1}{2} \\int_0^1 \\left( x(t)^2 + Pu(t)^2 \\right) dt$$\n\n*subject to*\n$$\\dot{x}(t) = u(t), \\hspace{0.5cm} x(0) = 1, \\hspace{0.5cm} x(1) = \\text{free}$$\n*where $P\u003e0$ us a weighting parameter. Find an optimal solution $u^*$\nand $x^*$ using the Euler-Lagrange condition.*\\\n\\\n*Discuss how the solution changes as $P \\to \\infty$ and $P \\to 0$.\nProvide an intuitive explanation of the phenomenon.*\\\n\n\n# Lecture 4\n\nExercise 4.10\n-------------\n\n*Solve the problem*\n\n$$\\begin{aligned}\n    \u0026u^* := \\argmin_{u:[0,t_f] \\to [-1, 1]} J := t_f \\\\\n    \u0026\\text{subject to} \\\\\n    \u0026\\begin{cases}\n        \u0026\\Dot{x}_1(t) = -4 x_1(t) + 2x_2(t) + 2u(t)\\\\\n        \u0026\\Dot{x}_2(t) = 3x_1(t) -3x_2(t)\n    \\end{cases}\n    \\smallspace\n    \\begin{cases}\n        \u0026x_1(0) = x_0 \\smallspace x_2(0) = 0 \\\\\n        \u0026x_1(t_f) = 0, \\smallspace x_2(t_f) = 0\n    \\end{cases}\\end{aligned}$$\\\n\n\nExercise 4.11 (Linear-quadratic problem)\n----------------------------------------\n\n*Solve the problem*\n\n$$\\begin{aligned}\n    \u0026u^* := \\argmin_{u:[0,T] \\to \\mathbb{R}} J(u) := \\frac{1}{2}q \\cdot x(T)^2 + \\frac{1}{2} \\int_0^T u(t) ^2 dt \\\\\n    \u0026\\text{subject to}\\\\\n    \u0026\\Dot{x}(t) = ax(t) + bu(t), \\smallspace x(0) = x_0\\end{aligned}$$\n\n*where a, b are given nonzero scalars, $x_0 \\in \\mathbb{R}$ is an\ninitial state, $T \u003e 0$ is a fixed final time, and $q \u003e 0$ is a given\npositive scalar.*\\\n\nExercise 4.12 (Control of production and consumption)\n-----------------------------------------------------\n\n*Suppose we own a factory whose output we can control. Let us begin to\nconstruct a mathematical model by setting*\n\n1.  *$x(t)$: amount of output produced at time $t \\in [0,T]$*\n\n2.  *$u(t)$: fraction of output reinvested at time $t \\in [0,T]$*\n\n*This will be our control, and is subject to the obvious constrain that*\n$$u(t) \\in [0,1]$$\n\n*Given such a control, the corresponding dynamics are provided by the\nODE*\n\n$$\\begin{aligned}\n    \u0026\\Dot{x}(t) = ku(t)x(t) \\\\\n    \u0026x(0) = x_0\\end{aligned}$$\n\n*where the constant $k \u003e0$ modeling the growth rate of our reinvestment\nand $u(t)x(t)$ is the amount of reinvested output. Let us take as a\npayoff functional*\n\n$$J(u) = \\int_0^T (1 - u(t))x(t)dt$$\n\n*where $(1-u(t))x(t)$ is our consumption. The meaning is that we want to\nmaximize our total consumption of the output.*\\\n*The overall problem is:* $$\\begin{aligned}\n    \u0026u^* := \\argmax_{u:[0,T] \\to [0,1]} J(u) = \\int_0^T (1-u(t))x(t)dt \\\\\n    \u0026\\text{subject to}\\\\\n    \u0026\\Dot{x}(t) = ku(t)x(t)\\\\\n    \u0026x(0)=x_0\\end{aligned}$$\n\n*Find an optimal control policy for this problem.*\\\n\nExercise 4.13\n-------------\n\n*A young investor has earned in the stock market a large amount of money\nS and plans to spend it so as to maximize his enjoyment through the rest\nof his life without working. He estimates that he will live exactly T\nmore years and that his capital $x(t)$ should be reduced to zero at time\nT, i.e., $x(T) = 0$. Also he models the evolution of his capital by the\ndifferential equation*\n\n$$\\Dot{x}(t) = \\alpha x(t) - u(t)$$\n\n*where $x(0) = S$ is his initial capital, $\\alpha \u003e 0$ is a given\ninterest rate, and $u(t) \\geq 0$ is his rate of expenditure. The total\nenjoyment he will obtain is given by*\n$$\\int_0^T e^{-\\beta t} \\sqrt{u(t)}dt$$\n\n*Here $\\beta$ is some positive scalar, which serves to discount the\nfuture enjoyment. Find the optimal control $u^* (t), t \\in [0, T]$.*\\\n\n*Solve the problem* $$\\begin{aligned}\n    \u0026u^* := \\argmin_{u:[0,T] \\to \\mathbb{R}} J(u) = \\int_0^T \\sqrt{1 + u(t)^2}dt \\\\\n    \u0026\\text{subject to}\\\\\n    \u0026\\Dot{x}(t) = u(t), \\smallspace x(0)=x_0\\end{aligned}$$ *where\n$x_0 \\in \\mathbb{R}$ is an initial state, $T\u003e0$ is a fixed final time.*\\\n\nExercise 4.14\n-------------\n\n*Solve the problem* $$\\begin{aligned}\n    \u0026u^* := \\argmin_{u:[0,T] \\to \\mathbb{R}} J(u) = \\int_0^T \\sqrt{1 + u(t)^2}dt \\\\\n    \u0026\\text{subject to}\\\\\n    \u0026\\Dot{x}(t) = u(t), \\smallspace x(0)=x_0\\end{aligned}$$ *where\n$x_0 \\in \\mathbb{R}$ is an initial state, $T\u003e0$ is a fixed final time.*\\\n\n\nExercise 4.15\n-------------\n\n*Solve the problem* $$\\begin{aligned}\n    \u0026u^* := \\argmin_{u:[0,T] \\to \\mathbb{R}} J(u) = \\int_0^T \\sqrt{1 + u(t)^2}dt \\\\\n    \u0026\\text{subject to}\\\\\n    \u0026\\Dot{x}(t) = u(t), \\smallspace x(0)=x_0, \\smallspace x(T) = x_1\\end{aligned}$$\n*where $x_0 \\in \\mathbb{R}$ is an initial state, $T\u003e0$ is a fixed final\ntime.*\\\n\n# Lecture 5\nExercise 5.4\n------------\n\n*Prove the lemma: if $a + \\varepsilon \\geq b$ for arbitrary\n$\\varepsilon \u003e 0$ with $a, b \\in \\mathbb{R}$, then $a \\geq b$.*\\\n\nExercise 5.5\n------------\n\n*Complete the proof of the principle of optimality by showing the\nreverse inequality $V(t,x) \\leq \\Bar{V}(t,x)$.*\\\n\nExercise 5.15 (Linear quadratic optimal tracking)\n-------------------------------------------------\n\n*For completeness, we consider a slightly more general form of the\nlinear quadratic regulator. The standard LQR derivation attempts to\ndrive the system to zero. Consider now the problem:*\n\n$$\\begin{aligned}\n    \u0026u^* := \\argmin_{u:[0,T] \\to \\mathbb{R}^m} J(u) \\\\\n    \u0026\\text{subject to}\\\\\n    \u0026\\Dot{x}(t) = Ax(t) + Bu(t), \\smallspace x(0)=x_0\\end{aligned}$$\n*where* $$\\begin{split}\n        J(u) := \u0026(x(T) - x_d (T)) ^T Q_f (x(T) - x_d (T) )  \\\\ \n        + \u0026\\int_0^T \\left( (x(t) - x_d(t)) ^T Q(x(t) - x_d(t)) + (u(t) - u_d(t)) ^T R(u(t) - u_d(t)) \\right) dt\n    \\end{split}$$\n\n*$Q_f = Q_f^T \\geq 0$ (positive semidefinite), $Q = Q^T \\geq 0$\n(positive semidefinite), $R = R^T \u003e0$ (positive definite).*\\\n*Find a formulation of an optimal policy and the corresponding HJB\nequation (similar to the Riccati equation) using the sufficient\ncondition.*\\\n*Hint: consider a candidate of value functions of the following form*\n$$\\hat{V}(x,t) = x^T S_2(t) x + s_1^T(t) x + s_0(t), \\smallspace S_2(t) = S_2(t) ^T \u003e0, \\smallspace s_1(t) \\in \\mathbb{R}^n, \\smallspace s_0(t) \\in \\mathbb{R}.$$\\\n\nExercise 5.16 (Nonlinear system)\n--------------------------------\n\n*Given the optimal control problem for a scalar nonlinear system:*\n\n$$\\begin{aligned}\n    \u0026u^* := \\argmin_{u:[0,1] \\to \\mathbb{R}} J(u) := \\int_0^1 [x(t)u(t)]^2 dt \\\\\n    \u0026\\text{subject to}\\\\\n    \u0026\\Dot{x}(t) = x(t)u(t), \\smallspace t \\in [0,1] \\\\\n    \u0026x(0) =1\\end{aligned}$$\n\n*Obtain the optimal feedback solution by solving the associated HJB\nequation.*\\\n*Hint: First show that the HJB partial differential equation admits a\nsolution that is quadratic in x.*\\\n\nExercise 5.19 (Linear quadratic regulator (LQR) problem)\n--------------------------------------------------------\n\n*Consider a cart with an inverted pendulum hinged on top of it. For simplicity, the cart and the pendulum are\nassumed to move in only one plane, and the friction, the mass of the\nstick, and the gust of wind are disregarded. The problem is to maintain\nthe pendulum at the vertical position. For example, if the inverted\npendulum is falling in the direction shown, the cart moves to the right\nand exerts a force, through the hinge, to push the pendulum back to the\nvertical position.*\\\n*Let's consider the linearized inverted pendulum system (assuming\n$\\theta(t)$ and $\\Dot{\\theta}(t)$ are very small)*\n\n$$\\begin{bmatrix}\n    \\Dot{x}_1(t) \\\\ \\Dot{x}_3(t) \\\\ \\Dot{x}_3(t) \\\\  \\Dot{x}_4(t) \\\\ \n\\end{bmatrix}=\n\\begin{bmatrix}\n    0 \u0026 1 \u0026 0 \u0026 0 \\\\\n    0 \u0026 0 \u0026 -\\frac{mg}{M} \u0026 0 \\\\\n    0 \u0026 0 \u0026 0 \u0026 1 \\\\\n    0 \u0026 0 \u0026 \\frac{(M+m)g}{Ml} \u0026 0 \n\\end{bmatrix}\n\\begin{bmatrix}\n    x_1(t) \\\\ x_2(t) \\\\ x_3(t) \\\\ x_4(t)\n\\end{bmatrix}\n+\n\\begin{bmatrix}\n    0 \\\\ \\frac{1}{M} \\\\ 0 \\\\ - \\frac{1}{Ml}\n\\end{bmatrix}\nu(t)$$ *where\n$x_1(t) = x(t), \\smallspace x_2(t) = \\Dot{x}(t), \\smallspace x_3(t) = \\theta(t), \\smallspace and \\smallspace x_4(t) = \\Dot{\\theta}(t)$.*\\\n*Consider the LQR problem*\n\n$$\\begin{aligned}\n    \u0026u^* := \\argmin_{u:[0,\\infty) \\to \\mathbb{R}^m} J(u) := \\int_0^\\infty \\left( x(t)^T Qx(t) + u(t)^T R u(t) \\right) dt \\\\\n    \u0026\\text{subject to}\\\\\n    \u0026\\Dot{x}(t) = Ax(t) + Bu(t), \\smallspace x(0) = x_0\\end{aligned}$$\n\n*where A and B are appropriate system matrices with m=1, M=2, l=3 and\ng=9.8.*\\\n*Tasks:*\n\n1.  *Appropriately choose the weighting $Q \\geq 0$ and $R \u003e 0$ to\n    maintain the pendulum at the vertical position, i.e.,\n    $\\theta(t) = 0$ and $\\Dot{\\theta}(t) = 0$ as much as possible.*\n\n2.  *Find an optimal policy for the LQR problem using Python or Matlab\n    functions.*\n\n3.  *Plot trajectories of the system x(t) and the control input u(t)\n    over certain time interval to demonstrate the performance of your\n    optimal control policy.*\n\n4.  *In the answer, please include your Python or Matlab codes.*\n\n5.  *Change the weight and see how the performance changes and discuss\n    about the results.*\n\n\n# Lecture 6\n\nExercise 6.15\n-------------\n\n*Consider the double integrator* $$\\begin{aligned}\n    \u0026\\Dot{x}_1(t) = x_2(t), \\\\\n    \u0026\\Dot{x}_2(t) = u(t)\\end{aligned}$$ *and the cost*\n$$J(u) = \\int_{t_0}^\\infty \\left( x_1(t)^2 + x_2(t)^2 + u(t) ^2 \\right) dt$$\n*Find P by solving the ARE. Verify (either analytically or numerically)\nthat it is indeed the steady-state solution of the RDE.*\\\n\n\n\nExercise 6.24\n-------------\n\n*Consider the infinite-horizon LQR problem* $$\\begin{aligned}\n    \u0026u^* := \\argmin_{u:[0,\\infty) \\to \\mathbb{R}} J(u) := \\int_0^\\infty \\left( qx(t)^2 + ru(t)^2  \\right) dt \\\\\n    \u0026\\text{subject to}\\\\\n    \u0026\\Dot{x}(t) = ax(t) + bu(t), \\smallspace t \\in [t_0, \\infty) \\\\\n    \u0026x(t_0) = x_0\\end{aligned}$$\n\n*where $a, q, r \u003e 0$ and $b \\in \\mathbb{R}$ is arbitrary. Find the\noptimal control law. Moreover, show that for $r \\to 0$, the eigenvalue\nof the optimal closed-loop system moves off to $- \\infty$, while for\n$r \\to \\infty$, the eigenvalue of the optimal closed-loop system tends\nto $−a$*\\\n\n\nExercise 6.25 (Boeing 747 Lateral Model)\n-------------\n\n*The complete lateral model of a Boeing 747 is* $$\\begin{aligned}\n    \u0026\\Dot{x}(t) = Ax(t) + Bu(t) \\\\\n    \u0026y(t) = Cx(t) + Du(t)\\end{aligned}$$ *where* $$A = \n    \\begin{bmatrix}\n        -10 \u0026 0 \u0026 0 \u0026 0 \u0026 0 \u0026 0 \\\\\n        0.0729 \u0026 -0.0558 \u0026 -0.997 \u0026 0.0802 \u0026 0.0415 \u0026 0 \\\\\n        -4.75 \u0026 0.598 \u0026 -0.115 \u0026 -0.0318 \u0026 0 \u0026 0 \\\\\n        1.53 \u0026 -3.05 \u0026 0.388 \u0026 -0.465 \u0026 0 \u0026 0 \\\\\n        0 \u0026 0 \u0026 0.0805 \u0026 1 \u0026 0 \u0026 0 \\\\\n        0 \u0026 0 \u0026 1 \u0026 0 \u0026 0 \u0026 -0.3333 \\\\\n    \\end{bmatrix}\n    , \\smallspace B =\n    \\begin{bmatrix}\n        1\\\\ 0\\\\ 0\\\\ 0\\\\ 0\\\\ 0\\\\\n    \\end{bmatrix}$$ *and* $$C =\n    \\begin{bmatrix}\n        0 \u0026 0 \u0026 1 \u0026 0 \u0026 0 \u0026 -0.3333\n    \\end{bmatrix}\n    , \\smallspace D=0$$\n\n*Minimize the sum of the energy of the output y and the energy of the\ncontrol u. The main effort is to minimize the energy of y which is\nsupposed to be zero in a steady state condition. So we put a weight\n$q = 9.527 \u003e 1$ on the energy of y. The problem now is as follows.*\n\n$$\\begin{aligned}\n    \u0026u^* := \\argmin_{u:[0,\\infty) \\to \\mathbb{R}} J(u) := \\int_0^\\infty [qy(t)^Ty(t) + u(t)^2] dt \\\\\n    \u0026\\text{subject to}\\\\\n    \u0026\\Dot{x}(t) = Ax(t) + Bu(t), \\smallspace t \\in [t_0, \\infty) \\\\\n    \u0026y(t) = Cx(t) + Du(t), \\smallspace t \\in [t_0, \\infty) \\\\\n    \u0026x(t_0) = x_0\\end{aligned}$$ *Tasks:*\n\n1.  *Find an optimal policy for the LQR problem by solving ARE using\n    Python or Matlab functions.*\n\n2.  *Plot trajectories of $y(t)$ and $u(t)$.*\n\n3.  *In the answer, please include your Python or Matlab codes.*\n\n\n\n## 📬 Feedback\nIf you would like to contribute to this repository by improving, correcting solutions or adding new exercises, feel free to raise an `Issue` or to contribute with a `Pull Request`!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffedebotu%2Foptimal-control-exercises","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffedebotu%2Foptimal-control-exercises","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffedebotu%2Foptimal-control-exercises/lists"}