Lecture 2: Convex sets - University of Illinois Urbana ...

1 Lecture 2: Convex setsAugust 28, 2008 Lecture 2 Outline Review basic topology inRn Open Set and Interior Closed Set and Closure Dual Cone Convex set Cones Affine sets Half-Spaces, Hyperplanes, Polyhedra Ellipsoids and Norm Cones Convex , Conical, and Affine Hulls Simplex Verifying ConvexityConvex Optimization1 Lecture 2 Topology ReviewLet{xk}be a sequence of vectors sequence{xk} Rnconvergesto a vector x Rnwhen xk x tends to 0 ask Notation: When{xk}converges to a vector x, we writexk x The sequence{xk}converges x Rnif and only if for each componenti: thei-th components ofxkconverge to thei-th component of x|xik xi|tends to 0 ask Convex Optimization2 Lecture 2 Open Set and InteriorLetX Rnbe a nonempty setXisopenif for everyx Xthere is an open ballB(x, r)thatentirely lies in the setX, ,for eachx Xthere isr > for allzwith z x < r,we havez vectorx0is aninterior pointof the setX, if there is a ballB(x0, r)contained entirely in the the setXis the set of all interior points ofX, denotedby (X) How is (X)related toX?

2 ExampleX={x R2|x1 0, x2>0} (X) ={x R2|x1>0, x2>0} (S)of a probability simplexS={x Rn|x 0, e x= 1} a Convex setX, theinterior (X)is also convexConvex Optimization3 Lecture 2 Closed a given setX Rnis the set of all vectors that donot belong toX:the complement ofX={x Rn|x / X}=Rn\ setXisclosedif its complementRn\Xis open Examples:Rnand (both are open and closed){x R2|x1 0, x2>0}is open or closed?hyperplane, half-space, simplex, polyhedral sets? Theintersectionofanyfamily of closed set is closed Theunionof afinitefamily of closed set is closed Thesumof two closed sets isnot necessarily closed Example:C1={(x1, x2)|x1= 0, x2 R}C2={(x1, x2)|x1x2 1, x1 0}C1+C2is not closed! Fact:The sum of a compact set and a closed set is closedConvex Optimization4 Lecture 2 ClosureLetX Rnbe a nonempty vector xis aclosure pointof a setXif there exists a sequence{xk} Xsuch thatxk xClosure points ofX={( 1)n/n|n= 1,2.}

3 }, X={1 x|x X}? Notation: The set of closure points ofXis denoted bycl(X) What is relation betweenXandcl(X)? set is closed if and only if it contains its closure points, ,Xis closed iffcl(X) a Convex set, the closurecl(X)is convexConvex Optimization5 Lecture 2 BoundaryLetX Rnbe a nonempty ofthe setXis the set of closure points for both the setXand its complementRn\X, ,bd(X) =cl(X) cl(Rn\X) ExampleX={x Rn|g1(x) 0, .. , gm(x) 0}. IsXclosed?What constitutes the boundary ofX? Convex Optimization6 Lecture 2 Dual ConeLetKbe a nonempty cone coneofKis the setK defined byK ={z|z x 0for allx K} The dual coneK is aclosed Convex coneeven whenKis neither closednor Convex LetSbe a subspace. Then,S =S . LetCbe a closed Convex cone.

4 Then,(C ) =C. For an arbitrary coneK, we have(K ) =cl(conv(K)). Convex Optimization7 Lecture 2 Convex set Aline segmentdefined by vectorsxandyisthe set of points of the form x+ (1 )yfor [0,1] A setC Rnisconvexwhen, with any two vectorsxandythat belongto the setC, the line segment connectingxandyalso belongs toCConvex Optimization8 Lecture 2 ExamplesWhich of the following sets are Convex ? The spaceRn Alinethrough two given vectorsxandyl(x, y) ={z|z=x+t(y x), t R} Araydefined by a vectorx{z|z= x, 0} Thepositive orthant{x Rn|x 0}( componentwise inequality) The set{x R2|x1>0, x2 0} The set{x R2|x1x2= 0} Convex Optimization9 Lecture 2 ConeA setC Rnis aconewhen, with every vectorx C, the ray{ x| 0}belongs to the setC A cone may or may not be Convex Examples:{x Rn|x 0} {x R2|x1x2 0}For a two setsCandS, the sumC+Sis defined byC+S={z|z=x+y, x C, y S}(the order does nor matter) Convex Cone Lemma: A coneCis Convex if and only ifC+C CProof: Pick anyxandyinC, and any [0,1].

5 Then, xand(1 )ybelong . UsingC+C C, it follows that .. Reverse: LetCbe Convex cone, and pick anyx, y C. Consider1/2(x+y).. Convex Optimization10 Lecture 2 Affine SetA setC Rnis aaffinewhen, with every two distinct vectorsx, y C,the line{x+t(y x)|t R}belongs to the setC An affine set is always Convex Asubspaceis an affine setA setCis affine if and only ifCis a translated subspace, ,C=S+x0for some subspaceSand somex0 CDimension of an affine set Cis the dimension of the subspaceSConvex Optimization11 Lecture 2 Hyperplanes and Half-spacesHyperplaneis a set of the form{x|a x=b}for a nonzero vectoraHalf-spaceis a set of the form{x|a x b}with a nonzero vectoraThe vectorais referred to as thenormal vector A hyperplane inRndivides the space into two half-spaces{x|a x b}and{x|a x b} Half-spaces are Convex Hyperplanes are Convex and affineConvex Optimization12 Lecture 2 Polyhedral SetsApolyhedralset is given by finitely many linear inequalitiesC={x|Ax b}whereAis anm nmatrix Every polyhedral set is Convex Linear Problemminimizec xsubject toBx b, Dx=dThe constraint set{x|Bx b.}

6 Dx=d}is Optimization13 Lecture 2 EllipsoidsLetAbe a square (n n) matrix. Aispositive semidefinitewhenx Ax 0for allx Rn Aispositive definitewhenx Ax >0for allx Rn, x6= 0 Anellipsoidis a set of the formE={x|(x x0) P 1(x x0) 1}wherePis symmetric and positive definite x0is the center of the ellipsoidE A ball{x| x x0 r}is a special case of the ellipsoid (P=r2I) Ellipsoids are convexConvex Optimization14 Lecture 2 Norm ConesAnorm coneis the set of the formC={(x, t) Rn R| x t} The norm can be any norm inRn The norm cone for Euclidean norm is also known asice-cream cone Any norm cone is convexConvex Optimization15 Lecture 2 Convex and Conical HullsAconvex combinationof vectorsx1, .. , xmis a vector of the form 1x1+.

7 + mxm i 0for alliand mi=1 i= 1 Theconvex hullof a setXis the set of all Convex combinations of thevectors inX, denotedconv(X)Aconical combinationof vectorsx1, .. , xmis a vector of the form 1x1+..+ mxmwith i 0for alliTheconical hullof a setXis the set of all conical combinations of thevectors inX, denoted bycone(X) Convex Optimization16 Lecture 2 Affine HullAnaffine combinationof vectorsx1, .. , xmis a vector of the formt1x1+..+tmxmwith mi=1ti= 1,ti Rfor alliTheaffine hullof a setXis the set of all affine combinations of the vectorsinX, denotedaff(X)Thedimensionof a setXis the dimension of the affine hull ofXdim(X) =dim(aff(X)) Convex Optimization17 Lecture 2 SimplexAsimplexis a set given as a Convex combination of a finite collection ofvectorsv0, v1.

8 , vm:C=conv{v0, v1.. , vm}The dimension of the simplexCis equal to the maximum number of linearlyindependent vectors amongv1 v0, .. , vm Unit simplex{x Rn|x 0, e x 1}, e= (1, .. ,1),dim-? Probability simplex{x Rn|x 0, e x= 1},dim-? Convex Optimization18 Lecture 2 Practical Methods for Establishing Convexity of a SetEstablish the convexity of a given setX The set is one of the recognizable (simple) Convex sets such aspolyhedral, simplex, norm cone, etc Prove the convexity by directly applying the definitionFor everyx, y Xand (0,1), show that x+ (1 )yis also inX Show that the set is obtained from one of the simple Convex sets throughan operation that preserves convexityConvex Optimization19 Lecture 2 Operations Preserving ConvexityLetC Rn,C1 Rn,C2 Rn, andK Rmbe Convex sets.

9 Then, thefollowing sets are also Convex : TheintersectionC1 C2={x|x C1andx C2} ThesumC1+C2of two Convex sets ThetranslatedsetC+a ThescaledsettC={tx|x C}for anyt R TheCartesian productC1 C2={(x1, x2)|x1 C1, x2 C2} Thecoordinate projection{x1|(x1, x2) Cfor somex2} TheimageACunder a linear transformationA:Rn7 Rm:AC={y Rm|y=Axfor somex C} Theinverse imageA 1 Kunder a linear transformationA:Rn7 Rm:A 1K={x Rn|Ax K} Convex Optimization20