THE METHOD OF LAGRANGE MULTIPLIERS

1 THE METHOD OFLAGRANGE MULTIPLIERSW illiam F. TrenchAndrew G. Cowles Distinguished Professor EmeritusDepartment of MathematicsTrinity UniversitySan Antonio, Texas, is a supplement to the author sIntroduction to Real Analysis. It has beenjudged to meet the evaluation criteria set by the Editorial Board of the AmericanInstitute of Mathematics in connection with the Institute sOpen Textbook may be copied, modified, redistributed, translated, and built upon subject to theCreative CommonsAttribution-NonCommercial-ShareAl ike Unported License. Acomplete instructor s solution manual is available by email to verification of the requestor s faculty METHOD OF LAGRANGE MULTIPLIERSW illiam F. Trench1 ForewordThis is a revised and extended version of Section of myAdvanced Calculus(Harper& Row, 1978). It is a supplement to my textbookIntroduction to Real Analysis, whichis referenced via hypertext IntroductionTo avoid repetition, it is to be understood throughout thatfandg1,g2.

2 ,gmarecontinuously differentiable on an open thatm < (1)on a nonempty subsetD1ofD. IfX02D1and there is a neighborhoodNofX0suchthatf .X/ f .X0/(2)for everyXinN\D1, thenX0isa local maximum point offsubject to the constraints(1). However, we will usually say subject to rather than subject to the constraint(s). If (2) is replaced byf .X/ f .X0/;(3)then maximum is replaced by minimum. A local maximum or minimum offsubject to (1) is also called alocal extreme point offsubject to(1). More briefly, wealso speak ofconstrained local maximum, minimum, or extreme points. If (2) or (3)holds for allXinD1, we omit local. Recall ; x20; : : : ; xn0/is acritical pointof a differentiable ; x2; : : : ; ; x20; : : : ; xn0/D0; 1 i n:Therefore, every local extreme point ofLis a critical point ofL; however, a criticalpoint ofLis not necessarily a local extreme point ofL(pp. 334-5).Suppose that the system (1) of simultaneous equations can be solved forx1.

3 ,xmin terms of thexmC1, ..,xn; thus, ; : : : ; xn/; 1 j m:(4)Then a constrained extreme value offis an unconstrained extreme value off . ; : : : ; xn/; : : : ; ; : : : ; xn/; xmC1; : : : ; xn/:(5)2 However, it may be difficult or impossible to find explicit formulas forh1,h2, ..,hm,and, even if it is possible, the composite function (5) is almost always , there is a better way to to find constrained extrema, which also requiresthe solvability assumption, but does not require an explicit formula as indicated in (4).It is based on the following theorem. Since the proof is complicated, we consider twospecial cases 1 Suppose thatn > m:IfX0is a local extreme point offsubject @ @xr2 @ @ @xr2 @ @ @xr2 @xrm 0(6)for at least one choice ofr1< r2< < rminf1; 2; : : :; ng;then there are constants 1; 2;..; msuch thatX0is a critical point off 1g1 2g2 mgmIthat is;@f .X0/@xi @xi @xi @xiD0;1 i following implementation of this theorem is themethod ofLagrangemultipliers.

4 (a)Find the critical points off 1g1 2g2 mgm;treating 1, 2, .. mas unspecified constants.(b)Find 1, 2, .., mso that the critical points obtained in (a) satisfy the con-straints.(c)Determine which of the critical points are constrained extreme points off. Thiscan usually be done by physical or intuitive ,b2, ..,bmare nonzero constants andcis an arbitrary constant, then thelocal extreme points offsubject tog1Dg2D DgmD0are the same as the localextreme points ofaf csubject tob1g1Db2g2D DbmgmD0. Therefore, wecan replacef 1g1 2g2 mgmbyaf 1b1g1 2b2g2 mbmgm cto simplify computations. (Usually, the c indicates dropping additive constants.)We will denote the final form byL(forLagrangian).33 Extrema subject to one constraintHere is 2 Suppose thatn > 1:IfX0is a local extreme point offsubject 0for somer2 f1; 2; : : :; ng;then there is a constant such ;(7)1 i nIthus;X0is a critical point off g:ProofFor notational convenience, letrD1and ; x3; : : : ; x30; : : : xn0 0, the Implicit Function Theorem(Corollary , p.)

5 423)impliesthat there is a unique continuously differentiable ;defined on aneighborhoodN Rn 1ofU0;such ;U/2 Dfor allU2N, , ;U/D0;U2N:(8)Now define ;(9)which is permissible, 0. This implies (7) withiD1. Ifi > 1,differentiating (8) with respect @ @ @xiD0;U2N:(10)Also,@f . ;U//@xiD@f . ;U/@xiC@f . ;U/@ (10) implies @ @ @xiD0:(12)IfX0is a local extreme point offsubject , thenU0is an unconstrainedlocal extreme point off . ;U/; therefore, (11) implies that@f .X0/@xiC@f .X0/@ @xiD0:(13)Since a linear homogeneous system a bc d uv D 00 4has a nontrivial solution if and only if a bc d D0;(Theorem , p. 376), (12) and (13) imply that @f .X0/@xi@f .X0/@ @ @x1 D0;so @f .X0/@ @xi@f .X0/@ @x1 D0;since the determinants of a matrix and its transpose are equal. Therefore, the system26664@f .X0/@ @xi@f .X0/@ @x137775 uv D 00 has a nontrivial solution(Theorem , p. 376). 0,umust benonzero in a nontrivial solution.

6 Hence, we may assume thatuD1, so26664@f .X0/@ @xi@f .X0/@ @x137775 1v D 00 :(14)In particular,@f .X0/@ @x1D0;so :Now (9) implies that vD , and (14) becomes26664@f .X0/@ @xi@f .X0/@ @x137775 1 D 00 :Computing the topmost entry of the vector on the left yields (7).Example 1 Find the ; y0/on the lineaxCbyDdclosest to a given ; y1/.5 SolutionWe must x1 y1/2subject to the constraint. Thisis equivalent to x1 y1/2subject to the constraint, which issimpler. For, this we could x1 y1/2 .axCby d/Ihowever, x1 y1/22 .axCby/is better. SinceLxDx x1 aandLyDy y1 b;.x0; y0 a; y1C b/, where we must choose so ,ax0 Cby0 Dax1 Cby1C .a2Cb2/Dd;so Dd ax1 by1a2Cb2; ax1 by1/aa2Cb2; ax1 by1/ba2Cb2:The distance ; y1/to the line x1 y1/2 Djd ax1 by1jpa2Cb2:Example 2 Find the extreme values off .x; y/D2xCysubject tox2Cy2D4:SolutionLetLD2xCy xandLyD1 y; ; y0 ; 1= /. Sincex20Cy20D4, D p5=2. Hence, the constrainedmaximum is2p5, attained ; 2=p5/, and the constrained minimum is 2p5,attained at.

7 4=p5; 2=p5/.Example 3 Find the point in the plane3xC4yC D1(15)closest to. 1; 1; 1/.SolutionWe must minimizef .x; y; 1/2C. 1/26subject to (15). 1/2C. 1/22 .3xC4yC /IthenLxDxC1 3 ; LyDy 1 4 ;andL D 1 ;sox0D 1C3 ; y0D1C4 ; 0D1C :From (15),3. 1C3 / 1D1C26 D0;so D 1= ; y0; 0/D 2926;2226;2526 :The distance ; y0; 0/to. 1; 1; 1 1/2C. 0 1/2D1p26:Example 4 Assume thatn 2andxi 0,1 i n.(a)Find the extreme values ofnXiD1xisubject tonXiD1x2iD1.(b)Find the minimum value ofnXiD1x2isubject (a)LetLDnXiD1xi 2nXiD1x2iIthenLxiD1 xi;soxi0D1 ; 1 i n:Hence,nXiD1x2i0Dn= 2, so D ; x20; : : : ; xn0/D 1pn;1pn; : : : ;1pn :Therefore, the constrained maximum ispnand the constrained minimum is (b)LetLD12nXiD1x2i nXiD1xiI7thenLxiDxi ;soxi0D ; 1 i n:Hence,nXiD1xi0Dn D1, soxi0D D1=nand the constrained minimum isnXiD1x2i0D1nThere is no constrained maximum. (Why?)Example 5 Show thatx1=py1=q xpCyq; x; y 0;if1pC1qD1; p > 0;andq > 0:(16)SolutionWe first find the maximum off.

8 X; y/Dx1=py1=qsubject toxpCyqD ; x 0; y 0;(17)where is a fixed but arbitrary positive number. Sincefis continuous, it must assumea maximum at some ; y0/on the line segment (17), ; y0/cannot be anendpoint of the segment, sincef .p ; 0/Df .0; q /D0. Therefore,.x0; y0/is in theopen first xpCyq :ThenLxD1pxf .x; y/ pandLyD1qyf .x; y/ qD0;sox0Dy0Df .x0; y0/= . Now(16) and (17) imply thatx0Dy0D . Therefore,f .x; y/ f . ; /D 1=p 1=qD DxpCyq:This can be generalized (Exercise53). It can also be used to generalizeSchwarz sinequality (Exercise54).84 Constrained Extrema of Quadratic FormsIn this section it is convenient to writeXD26664x1x2:::xn37775:Aneigenvalueo f a square matrixAD aij ni;jD1is a number such that the systemAXD X;or, equivalently,.A I/XD0;has a solutionX 0. Such a solution is called aneigenvectorofA. You probablyknow from linear algebra that is an eigenvalue ofAif and only I/D0:Henceforth we assume thatAis ; 1 i; j n/. In this case, I/D.

9 1/n. 1/. 2/ . n/;where 1; 2; : : : ; nare real ;jD1aijxixjis aquadratic form. To find its maximum or minimum subject tonXiD1x2iD1, we formthe nXiD1x2i:ThenLxiD2nXjD1aijxj 2 xiD0; 1 i n;sonXjD1aijxj0D xi0; 1 i n:Therefore,X0is a constrained critical point ofQsubject tonXiD1x2iD1if and onlyifAX0D X0for some ; that is, if and only if is an eigenvalue andX0is an9associated unit eigenvector ofA. IfAX0DX0andnXix2i0D1, xi0/xi0D nXiD1x2i0D Itherefore, the largest and smallest eigenvalues ofAare the maximum and minimumvalues ofQsubject 6 Find the maximum and minimum 2 2xyC4x C4y subject to the constraintx2Cy2C 2D1:(18)SolutionThe matrix ofQisAD241 1 2 1 1 22 2 I/D 1 12 1 1 222 2 D . C2/. 2/. 4/;so 1D4; 2D2; 3D 2are the eigenvalues ofA. Hence, 1D4and 3D 2are the maximum and minimumvalues ofQsubject to (18).To find the ; y1; 1/whereQattains its constrained maximum, we firstfind an eigenvector ofAcorresponding to 1D4.

10 To do this, we find a nontrivialsolution of the 4I/24x1y1 135D24 3 1 2 1 3 22 2 23524x1y1 135D2400035:10 All such solutions are multiples of2411235:Normalizing this to satisfy (18) yieldsX1D1p624x1y1 135D 2411135:To find the ; y3; 3/whereQattains its constrained minimum, we firstfind an eigenvector ofAcorresponding to 3D 2. To do this, we find a nontrivialsolution of the 335D243 1 2 1 3 22 2 43524x3y3 335D2400035:All such solutions are multiples of2411 135:Normalizing this to satisfy (18) yieldsX3D24x2y2 235D 1p32411 135:As for the eigenvalue 2D2, we leave it you to verify that the only unit vectorsthat satisfyAX2D2X2areX2D 1p22411 135:For more on this subject, see Extrema subject to two constraintsHere is 3 Suppose thatn > 2:IfX0is a local extreme point offsubject @ @ @ @xs 0(19)for somerandsinf1; 2; : : :; ng;then there are constants and such that@f .X0/@xi @xi @xiD0;(20)1 i notational convenience, letrD1andsD2.