Iterative Linear Quadratic Regulator (iLQR)

\[\newcommand{\vect}[1]{\boldsymbol{#1}} \newcommand{\dvect}[1]{\dot{\boldsymbol{#1}}} \newcommand{\ddvect}[1]{\ddot{\boldsymbol{#1}}} \newcommand{\mat}[1]{\boldsymbol{#1}}\]

Iterative LQR (iLQR) [1] is an extension of LQR to non- linear dynamics. The LQR uses the fixed point linearised dynamics for the entire state space and hence is only useful as long as the linearization error is small. In contrast to LQR, iLQR linearises the dynamics for every given state at each time step and can deal with nonlinear dynamics at the cost of being only able to optimize over a finite time horizon.

As a trajectory optimization method iLQR solves the following optimization problem for a sequence of \(N\) control inputs:

\[\begin{split}\min_{\vect{u}_0, \vect{u}_1, \ldots, \vect{u}_{N-1}} &\vect{x}_N^T \mat{Q}_{f} \vect{x}_N + \sum_{i=0}^{N-1} \vect{x}_i^T \mat{Q} \vect{x}_i + \vect{u}_i^T \mat{R} \vect{u}_i \label{eq:ilqr_opt}\\ \text{subject to}: & \hspace{1cm} \vect{x}_{i+1} = f_{discrete}(\vect{x}_{i}, \vect{u}_i)\end{split}\]

where a start state \(\vect{x}_0\) is set beforehand. \(\mat{Q}_f\), \(\mat{Q}\) and \(\mat{R}\) are cost matrices penalizing the final state, intermediate states and the control input respectively. \(f_{discrete}\) is the discretization of the system dynamics. \(\vect{x}\) and \(\vect{u}\) can also be expressed in relative coordinates \(\tilde{\vect{x}}\), \(\tilde{\vect{u}}\).

References