Linear Transformations - Mathematics

1 Linear TransformationsThe two basic vector operations are addition and scaling. From this perspec-tive, the nicest functions are those which preserve these operations:Def:Alinear transformationis a functionT:Rn Rmwhich satisfies:(1)T(x+y) =T(x) +T(y) for allx,y Rn(2)T(cx) =cT(x) for allx Rnandc :IfT:Rn Rmis a Linear transformation , thenT(0) = ve already met examples of Linear Transformations . Namely: ifAisanym nmatrix, then the functionT:Rn Rmwhich is matrix-vectormultiplicationT(x) =Axis a Linear transformation .(Wait: I thought matriceswerefunctions? Technically, no. matrices are lit-erally just arrays of numbers. However, matricesdefinefunctions by matrix-vector multiplication, and such functions are always Linear Transformations .)Question:Are these all the Linear Transformations there are? That is, doesevery Linear transformation come from matrix-vector multiplication? Yes:Prop :LetT:Rn Rmbe a Linear transformation .

2 Then the functionTis just matrix-vector multiplication:T(x) =Axfor some fact, them nmatrixAisA= T(e1) T(en) .Terminology:For Linear transformationsT:Rn Rm, we use the word kernel to mean nullspace. We also say image ofT to mean range ofT. So, for a Linear transformationT:Rn Rm:ker(T) ={x Rn|T(x) =0}=T 1({0})im(T) ={T(x)|x Rn}=T(Rn).Ways to Visualize functionsf:R R( :f(x) =x2)(1) Set-Theoretic Picture.(2) Graph off. (Thinking:y=f(x).)Thegraphoff:R Ris the subset ofR2given by:Graph(f) ={(x,y) R2|y=f(x)}.(3) Level sets off. (Thinking:f(x) =c.)Thelevel setsoff:R Rare the subsets ofRof the form{x R|f(x) =c},for constantsc to Visualize functionsf:R2 R( :f(x,y) =x2+y2)(1) Set-Theoretic Picture.(2) Graph off. (Thinking:z=f(x,y).)Thegraphoff:R2 Ris the subset ofR3given by:Graph(f) ={(x,y,z) R3|z=f(x,y)}.(3) Level sets off. (Thinking:f(x,y) =c.)Thelevel setsoff:R2 Rare the subsets ofR2of the form{(x,y) R2|f(x,y) =c},for constantsc to Visualize functionsf:R3 R( :f(x,y,z) =x2+y2+z2)(1) Set-Theoretic Picture.

3 (2) Graph off. (Thinking:w=f(x,y,z).)(3) Level sets off. (Thinking:f(x,y,z) =c.)Thelevel setsoff:R3 Rare the subsets ofR3of the form{(x,y,z) R3|f(x,y,z) =c},for constantsc inR2: Three descriptions(1)Graph of a functionf:R R. (That is:y=f(x))Such curves must pass the vertical line :When we talk about the curve y=x2, we actually mean tosay:the graph of the functionf(x) = is, we mean the set{(x,y) R2|y=x2}={(x,y) R2|y=f(x)}.(2)Level sets of a functionF:R2 R. (That is:F(x,y) =c)Example:When we talk about the curve x2+y2= 1, we actually meanto say:the level set of the functionF(x,y) =x2+y2at is, wemean the set{(x,y) R2|x2+y2= 1}={(x,y) R2|F(x,y) = 1}.(3)Parametrically:{x=f(t)y=g(t).Surfa ces inR3: Three descriptions(1)Graph of a functionf:R2 R. (That is:z=f(x,y).)Such surfaces must pass the vertical line :When we talk about the surface z=x2+y2, we actually meanto say:the graph of the functionf(x,y) =x2+ is, we mean the set{(x,y,z) R3|z=x2+y2}={(x,y,z) R3|z=f(x,y)}.}

4 (2)Level sets of a functionF:R3 R. (That is:F(x,y,z) =c.)Example:When we talk about the surface x2+y2+z2= 1, we actuallymean to say:the level set of the functionF(x,y,z) =x2+y2+z2at is, we mean the set{(x,y,z) R3|x2+y2+z2= 1}={(x,y,z) R3|F(x,y,z) = 1}.(3)Parametrically. (We ll discuss this another time, perhaps.)Two Examples of Linear Transformations (1)Diagonal matrices : Adiagonal matrixis a matrix of the formD= d10 00d2 dn .The Linear transformation defined byDhas the following effect: Vectors Stretched/contracted (possibly reflected) in thex1-direction byd1 Stretched/contracted (possibly reflected) in thex2-direction Stretched/contracted (possibly reflected) in thexn-direction bydn. Stretching in thexi-direction happens if|di|>1. Contracting in thexi-direction happens if|di|<1. Reflecting happens ifdiis negative.(2)Rotations inR2We writeRot :R2 R2for the Linear transformation which rotatesvectors inR2counter-clockwise through the angle.

5 Its matrix is:[cos sin sin cos ].The Multivariable Derivative: An ExampleExample:LetF:R2 R3be the functionF(x,y) = (x+ 2y,sin(x), ey) = (F1(x,y),F2(x,y),F3(x,y)).Itsderivativei s a Linear transformationDF(x,y) :R2 R3. The matrix ofthe Linear transformationDF(x,y) is:DF(x,y) = F1 x F1 y F2 x F2 y F3 x F3 y = 12cos(x) 00ey .Notice that (for example)DF(1,1) is a Linear transformation , as isDF(2,3),etc. That is, eachDF(x,y) is a Linear transformationR2 ApproximationSingle Variable SettingReview:In single-variable calc, we look at functionsf:R R. We writey=f(x), and at a point (a,f(a)) write: y , y=f(x) f(a), whiledy=f (a) x=f (a)(x a). So:f(x) f(a) f (a)(x a).Therefore:f(x) f(a) +f (a)(x a).The right-hand sidef(a) +f (a)(x a) can be interpreted as follows: It is the bestlinear approximationtof(x) atx=a. It is the1st Taylor polynomialtof(x) atx=a. The liney=f(a) +f (a)(x a) is the tangent line at (a,f(a)).Multivariable SettingNow consider functionsf:Rn Rm.

6 At a point (a,f(a)), we have exactlythe same thing:f(x) f(a) Df(a)(x a).That is:f(x) f(a) +Df(a)(x a).( )Note:The quantityDf(a) is amatrix, while (x a) is avector. That is,Df(a)(x a) is matrix-vector :Letf:R2 R. Let s writex= (x1,x2) anda= (a1,a2). Then( ) reads:f(x1,x2) f(a1,a2) +[ f x1(a1,a2) f x2(a1,a2)][x1 a1x2 a2]=f(a1,a2) + f x1(a1,a2)(x1 a1) + f x2(a1,a2)(x2 a2).Tangent Lines/Planes to GraphsFact:Suppose a curve inR2is given as a graphy=f(x). The equation ofthe tangent line at (a,f(a)) is:y=f(a) +f (a)(x a).Okay, you knew this from single-variable calculus. How does the multivari-able case work? Well:Fact:Suppose a surface inR3is given as a graphz=f(x,y). The equationof the tangent plane at (a,b,f(a,b)) is:z=f(a,b) + f x(a,b)(x a) + f y(a,b)(y b).Note the similarity between this and the Linear approximation tofat (a,b).Tangent Lines/Planes to Level SetsDef:For a functionF:Rn R, itsgradientis the vector inRngiven by: F=( F x1, F x2.)

7 , F xn).Theorem:Consider a level setF(x1,..,xn) =cof a functionF:Rn (a1,..,an) is a point on the level set, then F(a1,..,an) is normal tothe level 1:Suppose a curve inR2is given as a level curveF(x,y) = equation of the tangent line at a point (x0,y0) on the level curve is: F x(x0,y0)(x x0) + F y(x0,y0)(y y0) = 2:Suppose a surface inR3is given as a level surfaceF(x,y,z) = equation of the tangent plane at a point (x0,y0,z0) on the level surfaceis: F x(x0,y0,z0)(x x0) + F y(x0,y0,z0)(y y0) + F z(x0,y0,z0)(z z0) = :Do you see why Cor 1 and Cor 2 follow from the Theorem?Composition and Matrix MultiplicationRecall:Letf:X Yandg:Y Zbe functions. Theircompositionisthe functiong f:X Zdefined by(g f) =g(f(x)).Observations:(1) For this to make sense, we must have: co-domain(f) = domain(g).(2) Composition isnotgenerally commutative: that is,f gandg fareusually different.(3) Composition is always associative: (h g) f=h (g f).Fact:IfT:Rk RnandS:Rn Rmare both Linear Transformations , thenS Tis also a Linear :How can we describe the matrix of the Linear Transformations Tin terms of the matrices ofSandT?

8 Fact:LetT:Rn RnandS:Rn Rmbe Linear Transformations withmatricesBandA, respectively. Then the matrix ofS Tis the can multiply anm nmatrixAby ann kmatrixB. The result,AB, will be anm kmatrix:(m n)(n k) (m k).Notice thatnappears twice here to cancel out. That is, we need the numberof rows ofAto equal the number of columns ofB otherwise, the productABmakes no 1:LetAbe a (3 2)-matrix, and letBbe a (2 4)-matrix. TheproductABis then a (3 4) 2:LetAbe a (2 3)-matrix, and letBbe a (4 2)-matrix. ThenABis not defined. (But the productBAis defined: it is a (4 3)-matrix.)Two Model ExamplesExample 1A (Elliptic Paraboloid):Considerf:R2 Rgiven byf(x,y) =x2+ level sets offare curves inR2. The level sets are{(x,y)|x2+y2=c}.The graph offis a surface inR3. The graph is{(x,y,z)|z=x2+y2}.Notice that (0,0,0) is a local that f x(0,0) = f y(0,0) = 0. Also, 2f x2(0,0)>0 and 2f y2(0,0)> 1B (Elliptic Paraboloid):Considerf:R2 Rgiven byf(x,y) = x2 level sets offare curves inR2.

9 The level sets are{(x,y)| x2 y2=c}.The graph offis a surface inR3. The graph is{(x,y,z)|z= x2 y2}.Notice that (0,0,0) is a local that f x(0,0) = f y(0,0) = 0. Also, 2f x2(0,0)<0 and 2f y2(0,0)< 2 (Hyperbolic Paraboloid):Considerf:R2 Rgiven byf(x,y) =x2 level sets offare curves inR2. The level sets are{(x,y)|x2 y2=c}.The graph offis a surface inR3. The graph is{(x,y,z)|z=x2 y2}.Notice that (0,0,0) is a saddle pointof the graph that f x(0,0) = f y(0,0) = 0. Also, 2f x2(0,0)>0 while 2f y2(0,0)< Remark:In each case, the level sets offare obtained by slicingthe graph offby planesz=c. Try to visualize this in each RuleChain Rule (Matrix Form):Letf:Rn Rmandg:Rm Rpbe anydifferentiable functions. ThenD(g f)(x) =Dg(f(x)) Df(x).Here, the product on the right-hand side is a product of the case whereg:Rm Rhas codomainR, there is another way tostate the chain Rule:Letg=g(x1,..,xm) and suppose eachx1,..,xmis a functionof the variablest1.

10 ,tn. Then: g t1= g x1 x1 t1+ g x2 x2 t1+ + g xm xm t1,.. g tn= g x1 x1 tn+ g x2 x2 t1+ + g xm xm is a way to state this version of the chain rule in general that is,wheng:Rn Rphas codomainRp but let s keep things simple for 1:Letz=g(u,v), whereu=h(x,y) andv=k(x,y). Then thechain rule reads: z x= z u u x+ z v v xand z y= z u u y+ z v v 2:Letz=g(u,v,w), whereu=h(t),v=k(t),w=`(t). Thenthe chain rule reads: z t= z u u t+ z v v t+ z w w ,v,ware functions of just a single variablet, we can also write thisformula as: z t= z ududt+ z vdvdt+ z DerivativesDef:For a functionf:Rn R, itsdirectional derivativein the directionvat the pointx Rnis:Dvf(x) = f(x) , is the dot product of vectors. Therefore,Dvf(x) = f(x) v cos ,where =]( f(x),v).Usually, we assume thatvis a unit vector, meaning v = :Letf:R2 R. Letv=[ab]. Then:Dvf(x,y) = f(x,y) [ab]=[ f x f y] [ab]=a f x+b f particular, we have two important special cases:De1f(x,y) = f(x,y) [10]= f xDe2f(x,y) = f(x,y) [01]= f :Partial derivatives are themselves examples of directional derivatives!