Hessian Examples - Home | IIT Hyderabad

1 The Hessian matrix: Eigenvalues, concavity, and curvatureBro. David E. Brown, BYU Idaho Dept. of Mathematics. All rights , of April 21, 2014 Contents1 Introduction12 The Hessian matrix and the local quadratic approximation23 The eigenvalues of the Hessian matrix34 Concavity and curvature65 Conclusion76 Answers to the exercises81 IntroductionStudents of courses in multivariable calculus are often taught the so-called D-test for optimizing functionsof two variables:Theorem (x,y)has continuous second partial derivatives .

2 LetD(x,y) =fxxfyy f2xy,and suppose(x0,y0)is a critical point IfD(x0,y0)>0anda. iffxx(x0,y0)>0, thenf(x0,y0)is a local minimum value off;b. iffxx(x0,y0)<0, thenf(x0,y0)is a local maximum value IfD(x0,y0)<0then(x0,y0)is a saddle point In all other cases, no conclusion can be drawn without further theorem is an application of a theorem upon which we can improve, namely:Theorem (x,y)has continuous second partial derivatives at(x0,y0), and letD(x,y) =fxxfyy IfD(x0,y0)>0anda. iffxx(x0,y0)>0, thenfis concave up at(x0,y0);b.

3 Iffxx(x0,y0)<0, thenfis concave down at(x0,y0).b. IfD(x0,y0)<0then the concavity offis inconsistent at(x0,y0).c. In all other cases, no conclusion can be drawn without further that Theorem says nothing about critical points. It s valid anywherefhas continuous secondpartial and are fine, as far as they go, but they don t go far enough for my tastes. In thisdocument, you will learn about the relationship between curvature, the concavity of a surface, and theeigenvalues of the Hessian matrix off.

4 We will begin with a look at the local quadratic approximation, tosee how the Hessian matrix can be The Hessian matrix and the local quadratic approximationRecall that the Hessian matrix ofz=f(x,y) is defined to beHf(x,y) =[fxxfxyfyxfyy],at any point at which all the second partial derivatives (x,y) = 3x2 5xy3, thenHf(x,y) =[6 15y2 15y2 30xy]. Note that the Hessian matrixis a function ofxandy. Also note thatfxy=fyxin this example . This is becausefis a polynomial, so itsmixed second partial derivatives are continuous, so they are of the Examples in this document will enjoy the property thatfxy=fyx, an assumption that is veryoften reasonable.

5 Therefore, we will assume the Hessian matrix offreduces toHf(x,y) =[fxxfxyfxyfyy].The quantityDof the D-test mentioned in the introduction is actually the determinant of the Hessianmatrix:det(Hf(x,y))= fxxfxyfxyfyy =fxxfyy fxyfxy=fxxfyy f2xy= now on, we ll callDtheHessian determinant off. What does the Hessian determinant offhave todo with optimizingf?Recall that the local quadratic approximation toz=f(x,y) at (x0,y0) isf(x,y) f(x0,y0) +# f(x0,y0) [x x0y y0]+12[x x0y y0]Hf(x0,y0)[x x0y y0],for (x,y) sufficiently close to (x0,y0).

6 If you ve read the documentIncrements, differentials, and localapproximations(click here), then you know that our excuse for using the Hessian matrix in this approximationis that second derivatives give information about concavity. We wanted to use information about concavityto improve on the locallinearapproximation, which only uses information about slope. Problem is,fhas four second partial derivatives four measures of s too many to keeptrack of, so it would be nice to have some way to combine them.

7 One way is to calculate the Hessiandeterminant, which is the D of the D-test. Another way is to calculate the so-called eigenvalues ofthe Hessian matrix, which are the subject of the next section. Until then, let the following exercise andtheorem amuse and amaze the mixed second partial derivatives are not continuous at some point, then they may or may not be equal , two of them are the same in this document, so there are only three different ones. I m going for drama here, OK?Page 2 Exercise points (0,0), ( /2, /2), and ( /2, /2) are critical points off(x,y) = Calculate the local quadratic approximation tofat each of the three given critical Graph the surfacez=f(x,y) together with the local quadratic approximations at the three given What does the concavity of the local quadratic approximation at a given point have to do with theconcavity of the surface at that point?

8 The results of the foregoing exercise are not co ncidental:Theorem (x,y)has continuous third partial If the local quadratic approximation tofis concave up at(x0,y0), so is the surfacez=f(x,y).b. If the local quadratic approximation tofis concave down at(x0,y0), so is the If the concavity of the local quadratic approximation tofat(x0,y0)is inconsistent, then so is the concavityof the , the Hessianmatrixsomehow knows whether the surface is concave up or down. However,the Hessiandeterminantmixes up the information inherent in the Hessian matrix in such a way as to notbe able to tell up from down: recall that ifD(x0,y0)>0, then additional information is needed, to be ableto tell whether the surface is concave up or down.

9 (We typically use the sign offxx(x0,y0), but the sign offyy(x0,y0) will serve just as well.)The eigenvalues ofHfwill tell us whether the surface is concave up, concave down, or a little of both,at any given point. To find out how, read The eigenvalues of the Hessian matrixIntroducing eigenvalues to students who have never heard of them is a bit problematic. There s no goodway for me to convince you that they re good for anything that is, until you ve seen some of their give information about a matrix; the Hessian matrix contains geometric information about thesurfacez=f(x,y).

10 We re going to use the eigenvalues of the Hessian matrix to get geometric informationabout the s the definition:Definition a square (that is,n n) matrix, and suppose there is a scalar and a vector# xfor whichA# x= # the ordered pair ( ,# x) is aneigenpair ofA,b. is aneigenvalue ofA, andc.# xis aneigenvector ofAassociated with .If you find the definition to be unenlightening, then think of it this way: If ( ,# x) is an eigenpair ofA,then multiplying thematrixAby the vector# xsimplifies down to multiplying thescalar by# x.