Lecture 6 Simplex method for linear programming

1 Simplex method for linear programmingslides credit: Weinan EExamples and standard formFundamental theoremSimplex algorithmOutlineExamples and standard formFundamental theoremSimplex algorithmExamples and standard formFundamental theoremSimplex algorithmExample: Transportation problemSchematics of transportation problemaaabbbb12m12n 1nCijDestinationOriginExamples and standard formFundamental theoremSimplex algorithmExample: Transportation problemIFormulation:mins=m i=1n j=1cijxijsubject to theconstraintm i=1xij bj, j= 1, .. , nn j=1xij ai, i= 1, .. , mxij 0, i= 1, .. , m;j= 1, .. , the supply of thei-th origin,bjis the demand of thej-thdestinations,xijis the amount of the shipment from sourceitodestinationjandcijis the unit transportation cost problem ( Simplex method )Examples and standard formFundamental theoremSimplex algorithmLinear programmingIDefinition:If the minimized (or maximized) function and the constraints are all inlinear forma1x1+a2x2+ +anxn+ type of optimization is called linear and standard formFundamental theoremSimplex algorithmGeneral form of constraints of linear programmingIThe minimized function will always beminxw=cTx(ormax)wherec,x are 3 kinds of constraints in general:IType I: type constraintai1x1+ai2x2+ +ainxn biIType II: = type constraintaj1x1+aj2x2+ +ajnxn=bjIType III.

2 Type constraintak1x1+ak2x2+ +aknxn bkExamples and standard formFundamental theoremSimplex algorithmExamples: general formIExample 1: (type III)minw= 100x1+ 300x2+ 400x3+ x1+ 5x2+ 10x3+ 10000xi 0, i= 1,2,3,4 IExample 2: (type I)maxw= 7x+ + 4y 3604x+ 5y 2003x+ 10y 300x 0, y 0 IExample 3: (type II)minw=x1+ 3x2+ x1+ 2x2+x3= 52x1+ 3x2+x3= 6x2 0, x3 0 Examples and standard formFundamental theoremSimplex algorithmStandard form of constraintsIStandard form a11x1+a12x2+ +a1nxn= +am2x2+ +amnxn=bmxi 0, i= 1, .. , nwherebi 0 (i= 1, .. , m).IIn matrix formminxw=cTx(ormax) ,x 0whereA Rm n,x Rn,b Rm,rank(A) =m n(This is notessential.) andbi 0 (i= 1, .. , m).Examples and standard formFundamental theoremSimplex algorithmExample 1: standard formIExample 1: (type III)minw= 100x1+ 300x2+ 400x3+ x1 + 5x2 + 10x3 + 10000xi 0, i = 1, 2, 3, 4 Introduce surplus variable x5 0, then the constraint becomes the standard formx1 + 5x2 + 10x3 + x5 = 10000xi 0, i = 1, 2, 3, 4, 5 Examples and standard formFundamental theoremSimplex algorithmExample 2: standard formIIExample 2:max w = 7x + 9x + 4y 3604x + 5y 2003x + 10y 300x 0, y 0 Introduce slack variable x1, x2, x3 0 and let x4 = x, x5 = y, then the constraint becomes the standard formmax w = 7x4 + 12x5x1+9x4+ 4x5= 360x2+4x4+ 5x5= 200x3+3x4+ 10x5= 300xi 0, i= 1,2.

3 ,5 Examples and standard formFundamental theoremSimplex algorithmExample 3: standard formIExample 3:minw=x1+ 3x2+ x1+ 2x2+x3= 52x1+ 3x2+x3= 6x2 0, x3 0 IDeal with thefree variablex1:Solvingx1from one equation andsubstitute it into 5 2x2 x3thenminw= 5 +x2+ x2+x3= 4x2 0, x3 0 Examples and standard formFundamental theoremSimplex algorithmRemarkIIf some ofbi<0in the primitive form, we can time 1to both sides atfirst and introduce the slack and surplus variables and standard formFundamental theoremSimplex algorithmOutlineExamples and standard formFundamental theoremSimplex algorithmExamples and standard formFundamental theoremSimplex algorithmDefinitionsIFor the standard form,nis calleddimension,mis calledorder, variablesxsatisfying ,x 0are calledfeasible (A) =m, and the firstmcolumns ofAare linearlyindependent, (a1,a2.)

4 ,am)is nonsingular, whereai= (a1i, a2i, , ami). Then linear systemBxB=bhas unique solutionxB=B 1b. Definex= (xB,0), thenxsatisfiesAx= called abasic solution(the otherxiare 0) with respect and standard formFundamental theoremSimplex algorithmDefinitionsIIf there is0amongxB, it is called adegenerate basic a basic solution is also a feasible solution, it is called abasic to column indices inBare calledbasic variable. Theothers are callednon-basic number of the basic feasible solutions is less thanCmn=n!m!(n m)!Examples and standard formFundamental theoremSimplex algorithmExampleILinear programmingmaxw= 10x1+ 11x23x1+ 4x2+x3= 95x1+ 2x2+x4= 8x1 2x2+x5= 1xi 0, i= 1,2,3,4,5 IChooseB= (a3,a4,a5) =I3 3, thenBis a basis,x= (0,0,9,8,1)is a non-degenerate basic solution.

5 It satisfies the constraint, thus is abasic feasible , x4, x5are basic and standard formFundamental theoremSimplex algorithmExampleILinear programmingmaxw= 10x1+ 11x23x1+ 4x2 172x1+ 5x2 16xi 0, i= 1,2 IThe set of all the feasible solutions are calledfeasibleregion. feasible regionI53 This feasible region is a colorred convex polyhedron spanned by points x1 = (0, 0), x2 = (0, 16 ), x3 = (3, 2) and x4 = ( 17 , 0).Examples and standard formFundamental theoremSimplex algorithmDefinitionsIAconvex setSmeans for anyx1,x2 Sand [0,1], thenx= x1+ (1 )x2 S. A non-convex set is shown , convex means any line segmentx1x2belongs toSifx1,x2 (0,0),x2= (0,165), ..are calledextreme pointsbecause there is noy1,y2 S,y16=y2and0< <1, such thatxi= y1+ (1 ) and standard formFundamental theoremSimplex algorithmFundamental theoremTheorem (Fundamental theorem)Optimizing a linear objective functionw=cTxis achieved at the extremepointsin the feasible region colorblue if the feasible solution set is not emptyand the optimum is finite.

6 =C*XZminZ=C*XmaxExamples and standard formFundamental theoremSimplex algorithmSome basic theoremsIThere are three cases for the feasible solutions of the standard formIEmpty set;IUnbounded set;unbounded convex IBounded convex point in the feasible solution set is a extreme pointif and only ifit is abasic feasible and standard formFundamental theoremSimplex algorithmOutlineExamples and standard formFundamental theoremSimplex algorithmExamples and standard formFundamental theoremSimplex algorithmSimplex methodISimplex method is first proposed by Dantzig in searching for all of the basic solution is not applicable because thewhole number idea of Simplex :Give a rule to transfer from one extreme point toanother such that the objective function is rule must beeasily and standard formFundamental theoremSimplex algorithmCanonical formIFirst suppose the standard form isAx=b,x 0 IOne canonical form is to transfer a coefficient submatrix intoImwithGaussian elimination.

7 For examplex= (x1, x2, x3)and(A,b) =(1 1 1 51 2 0 4) B=(0 1 1 11 2 0 4)then it is acanonical form forx1andx3. One extreme point isx= (4,0,1),x1andx3are basic and standard formFundamental theoremSimplex algorithmTransferINow supposeAis in canonical form as the last example, then we transferfrom one basic solution to enter the basis anda1leave the (0 1 1 11 2 0 4) (0 1 1 1 0 2) ( 0 1 1 0 2)IThe canonical form forx2andx3. The basic solution isx= (0,2,3). It isalso a extreme and standard formFundamental theoremSimplex algorithmTransferIThe transferred basic solution may be not feasible in to make the transferred basic solution feasible? to make the objective function decreasing after transfer?Examples and standard formFundamental theoremSimplex algorithmHow to make the transferred basic solution feasible?

8 IAssumption: All of the basic feasible solutions are ifx= (x1, x2, .. , xm,0, .. ,0)is a basic feasible solution, thenxi> the basis is{a1,a2, .. ,am}initially, and selectak(k > m)enter the basis. Supposeak=m i=1yikaithen for any >0 ak=m i=1 yikaiIxis a basic feasible solutionm i=1xiai=bExamples and standard formFundamental theoremSimplex algorithmHow to make the transferred basic solution feasible?IWe havem i=1(xi yik)ai+ ak=bIBecausexi>0, if >0is small enough, x= (x1 y1k, x2 y2k, .. , xm ymk,0, .. ,0, ,0, .. ,0)is a feasible make it a basic solution we choose = min1 i m{xiyik yik>0}=xryrkthen xis a basic feasible solution, and letarleave the 0fori= 1,2, .. , m, then for any >0, xis feasible, thus thefeasible region is unbounded in this and standard formFundamental theoremSimplex algorithmHow to make the transferred basic solution feasible?

9 ISupposen= 6and the constraints(A,b) = 1 0 0 2 4 6 40 1 0 1 2 3 30 0 1 1 2 1 1 IOne basisa1,a2,a3, the basic feasible solutionx= (4,3,1,0,0,0)We want to choosea4enter the basis. The problem is tochooseaileavethe (k= 4)i1 2 3xiyik23 so leta1enter into new basic solution forx2, x3, x4isx= (0,1,3,2,0,0).Examples and standard formFundamental theoremSimplex algorithmHow to make the objective function decreasing after transfer?IThe aim is to chooseksuch that the objective function decreasing afterakenter the the canonical form isxi+n j=m+1yijxj=yi0, i= 1,2, .. , mwhereyi0>0. The basic feasible solutionx= (y10, y20, .. , ym0,0, .. ,0)the value of objective functionz0=cTBxB=m k=1ckyk0 Examples and standard formFundamental theoremSimplex algorithmHow to make the objective function decreased after transfer?

10 IFor any feasible solutionx= (x1, .. , xm, xm+1, .. , xn), we havez=m k=1ck(yk0 n j=m+1ykjxj) +n j=m+1cjxj=m k=1ckyk0+n j=m+1cjxj n j=m+1(m k=1ckykj)xj=z0+n j=m+1(cj m k=1ckykj)xj=z0+n j=m+1(cj zj)xjwherezj=cTByj= mk= there existsj(m+ 1 j n) such thatrj=cj zj<0, then whenxjchange from0to positive, the objective function will be and standard formFundamental theoremSimplex algorithmSimplex strategyIOptimality criterion: Ifrj 0for allj, then it is a optimal criterion: If for somek(rk<0), we haveyjk 0 (j= 1,2, , m), thenminz= .IOtherwise: We can choose the vectorak(rk<0) to enter the basis andthe vectoraj(yj0yjk= miniyi0yik, yik>0) leave the and standard formFundamental theoremSimplex algorithmExampleIExampleminz= (3x1+x2+ 3x3) 2 1 11 2 32 2 1 x1x2x3 256 ,x 0 IStep 1: change into standard formminz= (3x1+x2+ 3x3) 2 1 1 1 0 01 2 3 0 1 02 2 1 0 0 1 x1x2x3x4x5x6 = 256 , xi 0, i= 1,2.