Lecture 13 Linear quadratic Lyapunov theory

1 EE363 Winter 2008-09 Lecture 13 Linear quadratic Lyapunov theory the Lyapunov equation Lyapunov stability conditions the Lyapunov operator and integral evaluating quadratic integrals analysis of ARE discrete-time results linearization theorem13 1 The Lyapunov equationtheLyapunov equationisATP+P A+Q= 0whereA, P, Q Rn n, andP, Qare symmetricinterpretation: for Linear system x=Ax, ifV(z) =zTP z, then V(z) = (Az)TP z+zTP(Az) = , ifzTP zis the (generalized)energy, thenzTQzis the associated(generalized)dissipationlinear - quadratic Lyapunov theory :lineardynamics,quadraticLyapunovf unctionLinear quadratic Lyapunov theory13 2we consider system x=Ax, with 1, .. , nthe eigenvalues ofAifP >0, then the sublevel sets are ellipsoids (and bounded) V(z) =zTP z= 0 z= 0boundedness condition:ifP >0,Q 0then all trajectories of x=Axare bounded(this means i 0, and if i= 0, then icorresponds to a Jordanblock of size one) the ellipsoids{z|zTP z a}are invariantLinear quadratic Lyapunov theory13 3 Stability conditionifP >0,Q >0then the system x=Axis (globally asymptotically)stable, , i<0to see this, note that V(z) = zTQz min(Q)zTz min(Q) max(P)zTP z= V(z)where = min(Q)/ max(P)>0 Linear quadratic Lyapunov theory13 4An extension based on observability(Lasalle s theorem for Linear dynamics, quadratic function)ifP >0,Q 0, and(Q, A)observable, then the system x=Axis(globally asymptotically) stableto see this, we first note that all eigenvalues satisfy i 0now suppose thatv6= 0,Av= v, = 0thenA v= v= v, so Q1/2v 2=v Qv= v (ATP+P A)v= v P v v P v= 0which impliesQ1/2v= 0, soQv= 0, contradicting observability (by PBH)interpretation.

2 Observability condition means no trajectory canstay in the zero dissipation set{z|zTQz= 0} Linear quadratic Lyapunov theory13 5An instability conditionifQ 0andP6 0, thenAis not stableto see this, note that V 0, soV(x(t)) V(x(0))sinceP6 0, there is awwithV(w)<0; trajectory starting atwdoes notconverge to zeroin this case, the sublevel sets{z|V(z) 0}(which are unbounded) areinvariantLinear quadratic Lyapunov theory13 6 The Lyapunov operatortheLyapunov operatoris given byL(P) =ATP+P Aspecial case of Sylvester operatorLis nonsingular if and only ifAand Ashare no common eigenvalues, ,Adoes not have pair of eigenvalues which are negatives of each other ifAis stable, Lyapunov operator is nonsingular ifAhas imaginary (nonzero,i -axis) eigenvalue, then Lyapunovoperator is singularthus ifAis stable, for anyQthere is exactly one solutionPof LyapunovequationATP+P A+Q= 0 Linear quadratic Lyapunov theory13 7 Solving the Lyapunov equationATP+P A+Q= 0we are givenAandQand want to findPif Lyapunov equation is solved as a set ofn(n+ 1)/2equations inn(n+ 1)/2variables, cost isO(n6)

3 Operationsfast methods, that exploit the special structure of the Linear equations, cansolve Lyapunov equation with costO(n3)based on first reducingAto Schur or upper Hessenberg formLinear quadratic Lyapunov theory13 8 The Lyapunov integralifAis stable there is an explicit formula for solution of Lyapunov equation:P= 0etATQetAdtto see this, we note thatATP+P A= 0(ATetATQetA+etATQetAA)dt= 0(ddtetATQetA)dt=etATQetA 0= QLinear quadratic Lyapunov theory13 9 Interpretation as cost-to-goifAis stable, andPis (unique) solution ofATP+P A+Q= 0, thenV(z) =zTP z=zT( 0etATQetAdt)z= 0x(t)TQx(t)dtwhere x=Ax,x(0) =zthusV(z)is cost-to-go from pointz(with no input) and integralquadratic cost function with matrixQLinear quadratic Lyapunov theory13 10ifAis stable andQ >0, then for eacht,etATQetA>0, soP= 0etATQetAdt >0meaning: ifAis stable, we can chooseanypositive definite quadratic formzTQzas thedissipation, , V=zTQz then solve a set of Linear equations to find the (unique) quadratic formV(z) =zTP z Vwill be positive definite, so it is a Lyapunov function that provesAisstablein particular:a Linear system is stable if and only if there is a quadraticLyapunov function that proves itLinear quadratic Lyapunov theory13 11generalization:ifAstable,Q 0, and(Q, A)observable, thenP >0to see this, the Lyapunov integral showsP 0ifP z= 0, then0 =zTP z=zT( 0etATQetAdt)z= 0 Q1/2etAz 2dtso we concludeQ1/2etAz= 0for allt 0this implies thatQz= 0,QAz= 0.

4 ,QAn 1z= 0, contradicting(Q, A)observableLinear quadratic Lyapunov theory13 12 Monotonicity of Lyapunov operator inversesupposeATPi+PiA+Qi= 0,i= 1,2ifQ1 Q2, then for allt,etATQ1etA etATQ2etAifAis stable, we haveP1= 0etATQ1etAdt 0etATQ2etAdt=P2in other words: ifAis stable thenQ1 Q2= L 1(Q1) L 1(Q2) , inverse Lyapunov operator is monotonic, or preserves matrixinequality, whenAis stable(question: isLmonotonic?) Linear quadratic Lyapunov theory13 13 Evaluating quadratic integralssuppose x=Axis stable, and defineJ= 0x(t)TQx(t)dtto findJ, we solve Lyapunov equationATP+P A+Q= 0forPthen,J=x(0)TP x(0)in other words: we can evaluate quadratic integral exactly, by solving a setof Linear equations, without even computing a matrix exponentialLinear quadratic Lyapunov theory13 14 Controllability and observability GrammiansforAstable, the controllability Grammian of(A, B)is defined asWc= 0etABBTetATdtand the observability Grammian of(C, A)isWo= 0etATCTCetAdtthese Grammians can be computed by solving the Lyapunov equationsAWc+WcAT+BBT= 0,ATWo+WoA+CTC= 0we always haveWc 0,Wo 0.

5 Wc>0if and only if(A, B)is controllable, andWo>0if and only if(C, A)is observableLinear quadratic Lyapunov theory13 15 Evaluating a state feedback gainconsider x=Ax+Bu,y=Cx,u=Kx,x(0) =x0with closed-loop system x= (A+BK)xstableto evaluate the quadratic integral performance measuresJu= 0u(t)Tu(t)dt,Jy= 0y(t)Ty(t)dtwe solve Lyapunov equations(A+BK)TPu+Pu(A+BK) +KTK= 0(A+BK)TPy+Py(A+BK) +CTC= 0then we haveJu=xT0 Pux0,Jy=xT0 Pyx0 Linear quadratic Lyapunov theory13 16 Lyapunov analysis of AREwrite ARE (withQ 0,R >0)ATP+P A+Q P BR 1 BTP= 0as(A+BK)TP+P(A+BK) + (Q+KTRK) = 0withK= R 1 BTPwe conclude: ifA+BKstable, thenP 0(sinceQ+KTRK 0) , any stabilizing solution of ARE is PSDif also(Q, A)is observable, then we concludeP >0to see this, we need to show that(Q+KTRK, A+BK)is observableif not, there isv6= (A+BK)v= v,(Q+KTRK)v= 0 Linear quadratic Lyapunov theory13 17which impliesv (Q+KTRK)v=v Qv+v KTRKv=kQ1/2vk2+kR1/2 Kvk2= 0soQv= 0,Kv= 0(A+BK)v=Av= v,Qv= 0which contradicts(Q, A)observablethe same argument shows that ifP >0and(Q, A)is observable, thenA+BKis stableLinear quadratic Lyapunov theory13 18 Monotonic norm convergencesuppose thatA+AT<0, , (symmetric part of)Ais negative definitecan express asATP+P A+Q= 0, withP=I,Q >0meaning:xTP x=kxk2decreases along every nonzero trajectory, , kx(t)kis alwaysdecreasing monotonicallyto0 x(t)is always moving towards originthis impliesAis stable, but the converse is false.

6 For a stable system, weneed not haveA+AT<0(for a stable system withA+AT6<0,kx(t)kconverges to zero, but notmonotonically) Linear quadratic Lyapunov theory13 19for a stable system we can always change coordinates so we havemonotonic norm convergenceletPdenote the solution ofATP+P A+I= 0takeT=P 1/2in new coordinatesAbecomes A=T 1AT, A+ AT=P1/2AP 1/2+P 1/2 ATP1/2=P 1/2(P A+ATP)P 1/2= P 1<0in new coordinates, convergence isobviousbecausekx(t)kis alwaysdecreasingLinear quadratic Lyapunov theory13 20 Discrete-time resultsall Linear quadratic Lyapunov results have discrete-time counterpartsthediscrete-timeLyapunov equation isATP A P+Q= 0meaning:ifxt+1=AxtandV(z) =zTP z, then V(z) = zTQz ifP >0andQ >0, thenAis (discrete-time) stable ( ,| i|<1) ifP >0andQ 0, then all trajectories are bounded( ,| i| 1;| i|= 1only for1 1 Jordan blocks) ifP >0,Q 0, and(Q, A)observable, thenAis stable ifP6>0andQ 0, thenAis not stableLinear quadratic Lyapunov theory13 21 Discrete-time Lyapunov operatorthe discrete-time Lyapunov operator is given byL(P) =ATP A PLis nonsingular if and only if, for alli, j, i j6= 1(roughly speaking, if and only ifAandA 1share no eigenvalues)ifAis stable, thenLis nonsingular.

7 In factP= t=0(AT)tQAtis the unique solution of Lyapunov equationATP A P+Q= 0the discrete-time Lyapunov equation can be solved quickly ( ,O(n3))and can be used to evaluate infinite sums of quadratic functions, quadratic Lyapunov theory13 22 Converse theoremssupposext+1=Axtis stable,ATP A P+Q= 0 ifQ >0thenP >0 ifQ 0and(Q, A)observable, thenP >0in particular, a discrete-time Linear system is stable if and onlyif there is aquadratic Lyapunov function that proves itLinear quadratic Lyapunov theory13 23 Monotonic norm convergencesupposeATP A P+Q= 0, withP=IandQ >0this meansATA < I, ,kAk<1meaning:kxtkdecreases on every nonzero trajectory; indeed,kxt+1k kAkkxtk<kxtkwhenkAk<1, stability is obvious, sincekxtk kAktkx0k system is calledcontractivesince norm is reduced at each stepthe converse is false: system can be stable withoutkAk<1 Linear quadratic Lyapunov theory13 24now supposeAis stable, and letPsatisfyATP A P+I= 0takeT=P 1/2in new coordinatesAbecomes A=T 1AT, so AT A=P 1/2 ATP AP 1/2=P 1/2(P I)P 1/2=I P 1< ,k Ak<1so for a stable system, we can change coordinates so the system iscontractiveLinear quadratic Lyapunov theory13 25 Lyapunov s linearization theoremwe consider nonlinear time-invariant system x=f(x), wheref:Rn Rnsupposexeis an equilibrium point, ,f(xe) = 0, and letA=Df(xe) Rn nthe linearized system, nearxe, is x=A xlinearization theorem: if the linearized system is stable, , i(A)<0fori= 1.

8 , n, thenthe nonlinear system is locally asymptotically stable if for somei, i(A)>0, then the nonlinear system is not locallyasymptotically stableLinear quadratic Lyapunov theory13 26stability of the linearized system determines the local stability of thenonlinear system,exceptwhen all eigenvalues are in the closed lefthalfplane, and at least one is on the imaginary axisexamples like x=x3(which is not LAS) and x= x3(which is LAS)show the theorem cannot, in general, be tightenedexamples:eigenvalues ofDf(xe)conclusion about x=f(x) 3, 4i, iLAS nearxe 3, 4i, inot LAS nearxe 3, 4i, ino conclusionLinear quadratic Lyapunov theory13 27 Proof of linearization theoremlet s assumexe= 0, and express the nonlinear differential equation as x=Ax+g(x)wherekg(x)k Kkxk2suppose thatAis stable, and letPbe unique solution of LyapunovequationATP+P A+I= 0the Lyapunov functionV(z) =zTP zproves stability of the linearizedsystem.

9 We ll use it to prove local asymptotic stability of the nonlinearsystemLinear quadratic Lyapunov theory13 28 V(z) = 2zTP(Az+g(z))=zT(ATP+P A)z+ 2zTP g(z)= zTz+ 2zTP g(z) kzk2+ 2kzkkPkkg(z)k kzk2+ 2 KkPkkzk3= kzk2(1 2 KkPkkzk)so forkzk 1/(4 KkPk), V(z) 12kzk2 12 max(P)zTP z= 12kPkzTP zLinear quadratic Lyapunov theory13 29finally, usingkzk2 1 min(P)zTP zwe haveV(z) min(P)16K2kPk2= kzk 14 KkPk= V(z) 12kPkV(z)and we re donecomments: proof actually constructs an ellipsoid inside basin of attractionofxe= 0, and a bound on exponential rate of convergence choice ofQ=Iwas arbitrary; can get better estimates using otherQs,better bounds ong, tighter bounding arguments .. Linear quadratic Lyapunov theory13 30 Integral quadratic performanceconsider x=f(x),x(0) =x0we are interested in the integral quadratic performance measureJ(x0) = 0x(t)TQx(t)dtfor any fixedx0we can find this (approximately) by simulation andnumerical integration(we ll assume the integral exists; we do not requireQ 0) Linear quadratic Lyapunov theory13 31 Lyapunov bounds on integral quadratic performancesuppose there is a functionV:Rn Rsuch that V(z) 0for allz V(z) zTQzfor allzthen we haveJ(x0) V(x0), , the Lyapunov functionVserves as anupper bound on the integral quadratic cost(sinceQneed not be PSD, we might not have V 0.)

10 So we cannotconclude that trajectories are bounded) Linear quadratic Lyapunov theory13 32to show this, we note thatV(x(T)) V(x(0)) = T0 V(x(t))dt T0x(t)TQx(t)dtand so T0x(t)TQx(t)dt V(x(0)) V(x(T)) V(x(0))since this holds for arbitraryT, we conclude 0x(t)TQx(t)dt V(x(0)) Linear quadratic Lyapunov theory13 33 Integral quadratic performance for Linear systemsfor a stable Linear system, withQ 0, the Lyapunov bound is sharp, ,there exists aVsuch that V(z) 0for allz V(z) zTQzfor allzand for whichV(x0) =J(x0)for allx0(takeV(z) =zTP z, wherePis solution ofATP+P A+Q= 0) Linear quadratic Lyapunov theory13 34