Jeffrey R. Chasnov - Hong Kong University of Science and ...

1 Matrix Algebra for EngineersLecture notes forJeffrey R. ChasnovThe Hong Kong University of Science and TechnologyDepartment of MathematicsClear Water Bay, KowloonHong KongCopyrightc 2018, 2019 by Jeffrey Robert ChasnovThis work is licensed under the Creative Commons Attribution Hong Kong License. To view a copy of thislicense, visit or send a letter to Creative Commons, 171 SecondStreet, Suite 300, San Francisco, California, 94105, the promotional video on YouTubeThese are my lecture notes for my online Coursera course, Matrix Algebra for Engineers. I havedivided these notes into chapters called Lectures, with each lecture corresponding to a video onCoursera. I have also uploaded all my Coursera videos to YouTube, and links are placed at the top ofeach are problems at the end of each lecture chapter and I have tried to choose problems thatexemplify the main idea of the lecture . Students taking a formal University course in matrix or linearalgebra will usually be assigned many more additional problems, but here I follow the philosophythat less is more.

2 I give enough problems for students to solidify their understanding of the material,but not too many problems that students feel overwhelmed and drop out. I do encourage students toattempt the given problems, but if they get stuck, full solutions can be found in the are also additional problems at the end of coherent sections that are given as practice quizzeson the Coursera platform. Again, students should attempt these quizzes on the platform, but if astudent has trouble obtaining a correct answer, full solutions are also found in the mathematics in this matrix algebra course is at the level of an advanced high school student, buttypically students would take this course after completing a University -level single variable calculuscourse. There are no derivatives and integrals in this course, but student s are expected to have acertain level of mathematical maturity. Nevertheless, anyone who wants to learn the basics of matrixalgebra is welcome to ChasnovHong KongJuly 2018ivContentsI Matrices11 Definition of a matrix52 Addition and multiplication of matrices73 Special matrices9 Practice quiz: Matrix definitions114 Transpose matrix135 Inner and outer products156 Inverse matrix17 Practice quiz: Transpose and inverses197 Orthogonal matrices218 Rotation matrices239 Permutation matrices25 Practice quiz: Orthogonal matrices27II Systems of Linear Equations2910 Gaussian elimination3311 Reduced row echelon form3712 Computing inverses39 Practice quiz: Gaussian elimination4113 Elementary matrices4314 LU decomposition45vviCONTENTS15 Solving (LU)x = b47 Practice quiz: LU decomposition51 III Vector Spaces5316 Vector spaces5717 Linear independence5918 Span, basis and dimension61 Practice quiz: Vector space definitions6319 Gram-Schmidt process6520 Gram-Schmidt process example67 Practice quiz.

3 Gram-Schmidt process6921 Null space7122 Application of the null space7523 Column space7724 Row space, left null space and rank79 Practice quiz: Fundamental subspaces8125 Orthogonal projections8326 The least-squares problem8527 Solution of the least-squares problem87 Practice quiz: Orthogonal projections91IV Eigenvalues and Eigenvectors9328 Two-by-two and three-by-three determinants9729 Laplace expansion9930 Leibniz formula10331 Properties of a determinant105 Practice quiz: Determinants10732 The eigenvalue problem10933 Finding eigenvalues and eigenvectors (1)111 CONTENTSvii34 Finding eigenvalues and eigenvectors (2)113 Practice quiz: The eigenvalue problem11535 Matrix diagonalization11736 Matrix diagonalization example11937 Powers of a matrix12138 Powers of a matrix example123 Practice quiz: Matrix diagonalization125 Appendix127A Problem and practice quiz solutions129viiiCONTENTSWeek IMatrices13In this week s lectures, we learn about matrices.

4 Matrices are rectangular arrays of numbers orother mathematical objects and are fundamental to engineering mathematics. We will define matricesand how to add and multiply them, discuss some special matrices such as the identity and zero matrix,learn about transposes and inverses, and define orthogonal and permutation 1 Definition of a matrixView this lecture on YouTubeAnm-by-nmatrix is a rectangular array of numbers (or other mathematical objects) withmrowsandncolumns. For example, a two-by-two matrix A, with two rows and two columns, looks likeA=(a bc d).The first row has elementsaandb, the second row has elementscandd. The first column has elementsaandc; the second column has elementsbandd. As further examples, two-by-three and three-by-twomatrices look likeB=(a b cd e f),C= a db ec f .Of special importance are column matrices and row matrices. These matrices are also called column vector is in generaln-by-one and the row vector is one-by-n.

5 For example, whenn=3,we would write a column vector asx= abc ,and a row vector asy=(a b c).A useful notation for writing a generalm-by-nmatrix A isA= a11a12 a1na21a22 amn .Here, the matrix element of A in theith row and thejth column is denoted 1. DEFINITION OF A MATRIXP roblems for lecture diagonal of a matrix A are the entriesaijwherei= ) Write down the three-by-three matrix with ones on the diagonal and zeros ) Write down the three-by-four matrix with ones on the diagonal and zeros ) Write down the four-by-three matrix with ones on the diagonal and zeros to the ProblemsLecture 2 Addition and multiplication ofmatricesView this lecture on YouTubeMatrices can be added only if they have the same dimension. Addition proceeds element by example,(a bc d)+(e fg h)=(a+e b+fc+g d+h).Matrices can also be multiplied by a scalar. The rule is to just multiply every element of the example,k(a bc d)=(ka kbkc kd).Matrices (other than the scalar) can be multiplied only if the number of columns of the left matrixequals the number of rows of the right matrix.

6 In other words, anm-by-nmatrix on the left can onlybe multiplied by ann-by-kmatrix on the right. The resulting matrix will bem-by-k. Evidently, matrixmultiplication is generally not commutative. We illustrate multiplication using two 2-by-2 matrices:(a bc d)(e fg h)=(ae+bg a f+bhce+dg c f+dh),(e fg h)(a bc d)=(ae+c f be+d fag+ch bg+dh).First, the first row of the left matrix is multiplied against and summed with the first column of the rightmatrix to obtain the element in the first row and first column of the product matrix. Second, the firstrow is multiplied against and summed with the second column. Third, the second row is multipliedagainst and summed with the first column. And fourth, the second row is multiplied against andsummed with the second general, an element in the resulting product matrix, say in rowiand columnj, is obtained bymultiplying and summing the elements in rowiof the left matrix with the elements in columnjofthe right matrix.

7 We can formally write matrix multiplication in terms of the matrix elements. Let Abe anm-by-nmatrix with matrix elementsaijand let B be ann-by-pmatrix with matrix is anm-by-pmatrix, and its ij matrix element can be written ascij=n k= that the second index ofaand the first index ofbare summed 2. ADDITION AND MULTIPLICATION OF MATRICESP roblems for lecture the matricesA=(21 11 11), B=(4 212 4 2),C=(1 22 1),D=(3 44 3),E=(12).Compute if defined:B 2A,3C E,AC,CD, A=(1 22 4), B=(2 11 3)and C=(4 30 2). Verify that AB=AC and yet B = A= 1 1 11 2 31 3 4 and D= 2 0 00 3 00 0 4 . Compute AD and the associative law for matrix multiplication. That is, let A be anm-by-nmatrix, B ann-by-pmatrix, and C ap-by-qmatrix. Then prove that A(BC) = (AB) to the ProblemsLecture 3 Special matricesView this lecture on YouTubeThe zero matrix, denoted by 0, can be any size and is a matrix consisting of all zero elements.

8 Multi-plication by a zero matrix results in a zero matrix. The identity matrix, denoted by I, is a square matrix(number of rows equals number of columns) with ones down the main diagonal. If A and I are thesame sized square matrices, thenAI=IA=A,and multiplication by the identity matrix leaves the matrix unchanged. The zero and identity matricesplay the role of the numbers zero and one in matrix multiplication. For example, the two-by-two zeroand identity matrices are given by0=(0 00 0),I=(1 00 1).A diagonal matrix has its only nonzero elements on the diagonal. For example, a two-by-two diagonalmatrix is given byD=(d100d2).Usually, diagonal matrices refer to square matrices, but they can also be band (or banded) matrix has nonzero elements only on diagonal bands. For example, a three-by-three band matrix with nonzero diagonals one above and one below a nonzero main diagonal (calleda tridiagonal matrix) is given byB= d1a10b1d2a20b2d3 .An upper or lower triangular matrix is a square matrix that has zero elements below or above thediagonal.

9 For example, three-by-three upper and lower triangular matrices are given byU= a b c0d e00f ,L= a00b d0c e f .910 lecture 3. SPECIAL MATRICESP roblems for lecture ( 124 8).Construct a two-by-two matrix B such that AB is the zero matrix. Use two different nonzero columnsfor that(a100a2)(b100b2)=(a1b100a2b2).Prove in general that the product of two diagonal matrices is a diagonal matrix, with elements givenby the product of the diagonal that(a1a20a3)(b1b20b3)=(a1b1a1b2+a2b30a3 b3).Prove in general that the product of two upper triangular matrices is an upper triangular matrix, withthe diagonal elements of the product given by the product of the diagonal to the ProblemsPractice quiz: Matrix the two-by-two matrix with matrix elementsaij=i )(100 1)b)( 1 00 1)c)(0 1 1 0)d)(0 110) matrix product(1 1 11)( 111 1)is equal toa)( 222 2)b)(2 2 22)c)( 2 2 2 2)d)( 2 222) A and B ben-by-nmatrices with(AB)ij=n k=1aikbkj. If A and B are upper triangular matrices,thenaik=0 orbkj=0 < > < >ja) A and C onlyb) A and D onlyc) B and C onlyd) B and D onlySolutions to the Practice quiz1112 lecture 3.

10 SPECIAL MATRICESL ecture 4 Transpose matrixView this lecture on YouTubeThe transpose of a matrix A, denoted by ATand spoken as A-transpose, switches the rows andcolumns of A. That is,if A= a11a12 a1na21a22 amn ,then AT= a11a21 am1a12a22 amn .In other words, we writeaTij= , if A ism-by-nthen ATisn-by-m. As a simple example, view the following transpose pair: a db ec f T=(a b cd e f).The following are useful and easy to prove facts:(AT)T=A, and(A+B)T=AT+ less obvious fact is that the transpose of the product of matrices is equal to the product of thetransposes with the order of multiplication reversed, ,(AB)T= A is a square matrix, and AT=A, then we say that A issymmetric. If AT= A, then we say that Aisskew symmetric. For example, three-by-three symmetric and skew symmetric matrices look like a b cb d ec ef , 0b c b0e c e0 .Notice that the diagonal elements of a skew-symmetric matrix must be 4.