Linear Quadratic Regulator (LQR)

$$$$

The linear quadratic regulator (LQR) controller is a well established and widespread optimal controller which acts on a linear system $\dot{\mathit{x}}=\mathit{A}\mathit{x}+\mathit{B}\mathit{u}$ and an objective which is specified by a quadratic, instantaneous cost function $J={\mathit{x}}^T\mathit{Q}\mathit{x}+{\mathit{u}}^T\mathit{R}\mathit{u}$ with the symmetric and positive definite matrices $\mathit{Q}={\mathit{Q}}^T⪰0$ and $\mathit{R}={\mathit{R}}^T\succ 0$. This allows for reducing the Hamilton-Jacobi-Bellman (HJB) equation, whose solution is the optimal cost-to-go, from which the optimal policy can be inferred, to the algebraic Riccati equation

$$\mathit{S}\mathit{A}+{\mathit{A}}^T\mathit{S}-\mathit{S}\mathit{B}{\mathit{R}}^{-1}{\mathit{B}}^T\mathit{S}+\mathit{Q}=0$$

for which good numerical solvers exist to find the optimal cost-to-go matrix $\mathit{S}$. The optimal policy obtained is

$$\mathit{u}(\mathit{x})=-{\mathit{R}}^{-1}{\mathit{B}}^T\mathit{S}\mathit{x}=-\mathit{K}\mathit{x}.$$

In order to use an LQR controller for stabilizing the double pendulum on the top, the dynamics have to be linearised around the top position ${\mathit{x}}^d=[\pi ,0,0,0]$ and ${\mathit{u}}^d=[0,0]$:

$$\mathit{A}={\frac{\partial \mathit{f}(\mathit{x},\mathit{u})}{\partial \mathit{x}}|}_{\mathit{x}={\mathit{x}}^d,\mathit{u}={\mathit{u}}^d},\mathit{B}={\frac{\partial \mathit{f}(\mathit{x},\mathit{u})}{\partial \mathit{u}}|}_{\mathit{x}={\mathit{x}}^d,\mathit{u}={\mathit{u}}^d}$$

and the state and actuation have to be expressed in relative coordinates $\tilde{\mathit{x}}=\mathit{x}-{\mathit{x}}^d$, $\tilde{\mathit{u}}=\mathit{u}-{\mathit{u}}^d$.

Region of Attraction (RoA)

For dynamical systems, the Region of Attraction (RoA) $\mathcal{B}$ around a fixed point ${\mathit{x}}^{\star }$ describes the set of initial states for which $\mathit{x}\to {\mathit{x}}^{\star }$ as $t\to \mathrm{\infty }$. Direct computation of this set is often not possible. However, it can be estimated by considering the sublevel set of a Lyapunov function $V(\mathit{x})$ [1]. When using LQR to stabilize the system around ${\mathit{x}}^{\star }$, the cost-to-go can serve as a quadratic Lyapunov function [2]. In this case, the estimated RoA can be written as:

$${\mathcal{B}}_{\text{est}}=\{\mathit{x}|{\mathit{x}}^T\mathit{S}\mathit{x}\le \rho \}$$

Where $\rho$ is a scalar that can be estimated using either probabilistic [3] or optimization based methods [4].

For further reading we refer to these lecture notes [2].

References

• [1] H. K. Khalil, Nonlinear Systems, 3rd ed. Upper Saddle River, N.J: Prentice Hall, 2002

• [2] R. Tedrake, Underactuated Robotics, 2022. (Online) url: http://underactuated.mit.edu

• [3] E. Najafi, R. Babuška, and G. A. D. Lopes, “A fast sampling method for estimating the domain of attraction,” Nonlinear Dynamics, vol. 86, no. 2, pp. 823–834, Oct. 2016. url: https://link.springer.com/article/10.1007/s11071-016-2926-7

• [4] P. Parrilo, “Structured semidefinite programs and semialgebraic ge- ometry methods in robustness and optimization,” Ph.D. dissertation, California Institute of Technology, Pasadena, California, 2000. url: https://www.proquest.com/openview/ff5fe1a4311720ae2dad28ddc1d22cf8/1?cbl=18750&diss=y&pq-origsite=gscholar&parentSessionId=MjXEze6vRVD%2BeSjkr1UEy6Zldtg74txylCbk173fanA%3D