Linear Algebra, Theory And Applications

1 Linear algebra , Theory And Applications Kenneth Kuttler September 12, 2017. 2. Contents 1 Preliminaries 11. Sets And Set Notation .. 11. Functions .. 12. The Number Line And algebra Of The Real Numbers .. 12. Ordered fields .. 14. The Complex Numbers .. 15. The Fundamental Theorem Of algebra .. 18. Exercises .. 20. Completeness of R .. 21. Well Ordering And Archimedean Property .. 22. Division .. 24. Systems Of Equations .. 28. Exercises .. 33. Fn .. 34. algebra in Fn .. 34. Exercises .. 35. The Inner Product In Fn .. 35. What Is Linear algebra ? .. 38. Exercises .. 38. 2 Linear Transformations 39. Matrices .. 39. The ij th Entry Of A Product .. 44. Digraphs .. 46. Properties Of Matrix Multiplication .. 48. Finding The Inverse Of A Matrix .. 51. Exercises .. 54. Linear Transformations .. 56. Some Geometrically Defined Linear Transformations .. 58. The Null Space Of A Linear Transformation .. 61. Subspaces And Spans .. 62. An Application To Matrices.

2 67. Matrices And Calculus .. 68. The Coriolis Acceleration .. 69. The Coriolis Acceleration On The Rotating Earth .. 73. Exercises .. 78. 3. 4 CONTENTS. 3 Determinants 85. Basic Techniques And Properties .. 85. Exercises .. 89. The Mathematical Theory Of Determinants .. 91. The Function sgn .. 91. The Definition Of The Determinant .. 93. A Symmetric Definition .. 95. Basic Properties Of The Determinant .. 96. Expansion Using Cofactors .. 97. A Formula For The Inverse .. 99. Rank Of A Matrix .. 101. Summary Of Determinants .. 103. The Cayley Hamilton Theorem .. 104. Block Multiplication Of Matrices .. 106. Exercises .. 109. 4 Row Operations 113. Elementary Matrices .. 113. The Rank Of A Matrix .. 119. The Row Reduced Echelon Form .. 121. Rank And Existence Of Solutions To Linear Systems .. 124. Fredholm Alternative .. 125. Exercises .. 126. 5 Some Factorizations 131. LU Factorization .. 131. Finding An LU Factorization .. 131. Solving Linear Systems Using An LU Factorization.

3 133. The P LU Factorization .. 134. Justification For The Multiplier Method .. 136. Existence For The P LU Factorization .. 137. The QR Factorization .. 138. Exercises .. 141. 6 Spectral Theory 145. Eigenvalues And Eigenvectors Of A Matrix .. 145. Some Applications Of Eigenvalues And Eigenvectors .. 153. Exercises .. 156. Schur's Theorem .. 162. Trace And Determinant .. 169. Quadratic Forms .. 170. Second Derivative Test .. 171. The Estimation Of Eigenvalues .. 175. Advanced Theorems .. 176. Exercises .. 179. 7 Vector Spaces And Fields 189. Vector Space Axioms .. 189. Subspaces And Bases .. 190. Basic Definitions .. 190. A Fundamental Theorem .. 190. The Basis Of A Subspace .. 195. CONTENTS 5. Lots Of Fields .. 195. Irreducible Polynomials .. 195. Polynomials And Fields .. 199. The Algebraic Numbers .. 205. The Lindemannn Weierstrass Theorem And Vector Spaces .. 208. Exercises .. 209. 8 Linear Transformations 215. Matrix Multiplication As A Linear Transformation.

4 215. L (V, W ) As A Vector Space .. 215. The Matrix Of A Linear Transformation .. 217. Rotations About A Given Vector .. 224. The Euler Angles .. 226. Eigenvalues And Eigenvectors Of Linear Transformations .. 228. Exercises .. 229. 9 Canonical Forms 233. A Theorem Of Sylvester, Direct Sums .. 233. Direct Sums, Block Diagonal Matrices .. 236. Cyclic Sets .. 239. Nilpotent Transformations .. 243. The Jordan Canonical Form .. 246. Exercises .. 249. The Rational Canonical Form .. 254. Uniqueness .. 256. Exercises .. 261. 10 Markov Processes 263. Regular Markov Matrices .. 263. Migration Matrices .. 267. Absorbing States .. 267. Exercises .. 270. 11 Inner Product Spaces 273. General Theory .. 273. The Gram Schmidt Process .. 275. Riesz Representation Theorem .. 278. The Tensor Product Of Two Vectors .. 281. Least Squares .. 283. Fredholm Alternative Again .. 284. Exercises .. 285. The Determinant And Volume .. 289. Exercises .. 291. 12 Self Adjoint Operators 293.

5 Simultaneous Diagonalization .. 293. Schur's Theorem .. 296. Spectral Theory Of Self Adjoint Operators .. 298. Positive And Negative Linear Transformations .. 302. The Square Root .. 304. Fractional Powers .. 305. Square Roots And Polar Decompositions .. 306. 6 CONTENTS. An Application To Statistics .. 309. The Singular Value Decomposition .. 311. In The Frobenius Norm .. 313. Squares And Singular Value Decomposition .. 315. Moore Penrose Inverse .. 315.. 319. 13 Norms 323. The p Norms .. 329. The Condition Number .. 331. The Spectral Radius .. 333. Series And Sequences Of Linear Operators .. 335. Iterative Methods For Linear Systems .. 340. Theory Of Convergence .. 345. Exercises .. 348. 14 Numerical Methods, Eigenvalues 355. The Power Method For Eigenvalues .. 355. The Shifted Inverse Power Method .. 358. The Explicit Description Of The Method .. 359. Complex Eigenvalues .. 364. Rayleigh Quotients And Estimates for Eigenvalues .. 365.

6 The QR Algorithm .. 369. Basic Properties And Definition .. 369. The Case Of Real Eigenvalues .. 372. The QR Algorithm In The General Case .. 376. Exercises .. 383. A Matrix Calculator On The Web 385. Use Of Matrix Calculator On Web .. 385. B Positive Matrices 387. C Functions Of Matrices 395. D Di erential Equations 401. Theory Of Ordinary Di erential Equations .. 401. Linear Systems .. 402. Local Solutions .. 403. First Order Linear Systems .. 405. Geometric Theory Of Autonomous Systems .. 413. General Geometric Theory .. 416. The Stable Manifold .. 418. E Compactness And Completeness 423. The Nested Interval Lemma .. 423. Convergent Sequences, Sequential Compactness .. 424. CONTENTS 7. F Some Topics Flavored With Linear algebra 427. The Symmetric Polynomial Theorem .. 427. The Fundamental Theorem Of algebra .. 429. Transcendental Numbers .. 433. More On Algebraic Field Extensions .. 441. G Selected Exercises 455. Copyright . c 2012, 8 CONTENTS.

7 Preface This is a book on Linear algebra and matrix Theory . While it is self contained, it will work best for those who have already had some exposure to Linear algebra . It is also assumed that the reader has had calculus. Some optional topics require more analysis than this, however. I think that the subject of Linear algebra is likely the most significant topic discussed in undergraduate mathematics courses. Part of the reason for this is its usefulness in unifying so many di erent topics. Linear algebra is essential in analysis, applied math, and even in theoretical mathematics . This is the point of view of this book, more than a presentation of Linear algebra for its own sake. This is why there are numerous Applications , some fairly unusual. This book features an ugly, elementary, and complete treatment of determinants early in the book. Thus it might be considered as Linear algebra done wrong. I have done this because of the usefulness of determinants.

8 However, all major topics are also presented in an alternative manner which is independent of determinants. The book has an introduction to various numerical methods used in Linear algebra . This is done because of the interesting nature of these methods. The presentation here emphasizes the reasons why they work. It does not discuss many important numerical considerations necessary to use the methods e ectively. These considerations are found in numerical analysis texts. In the exercises, you may occasionally see at the beginning. This means you ought to have a look at the exercise above it. Some exercises develop a topic sequentially. There are also a few exercises which appear more than once in the book. I have done this deliberately because I think that these illustrate exceptionally important topics and because some people don't read the whole book from start to finish but instead jump in to the middle somewhere. There is one on a theorem of Sylvester which appears no fewer than 3 times.

9 Then it is also proved in the text. There are multiple proofs of the Cayley Hamilton theorem, some in the exercises. Some exercises also are included for the sake of emphasizing something which has been done in the preceding chapter. 9. 10 CONTENTS. Chapter 1. Preliminaries Sets And Set Notation A set is just a collection of things called elements. For example {1, 2, 3, 8} would be a set consisting of the elements 1,2,3, and 8. To indicate that 3 is an element of {1, 2, 3, 8} , it is customary to write 3 {1, 2, 3, 8} . 9 / {1, 2, 3, 8} means 9 is not an element of {1, 2, 3, 8} . Sometimes a rule specifies a set. For example you could specify a set as all integers larger than 2. This would be written as S = {x Z : x > 2} . This notation says: the set of all integers, x, such that x > 2. If A and B are sets with the property that every element of A is an element of B, then A is a subset of B. For example, {1, 2, 3, 8} is a subset of {1, 2, 3, 4, 5, 8} , in symbols, {1, 2, 3, 8}.

10 {1, 2, 3, 4, 5, 8} . It is sometimes said that A is contained in B or even B contains A . The same statement about the two sets may also be written as {1, 2, 3, 4, 5, 8} {1, 2, 3, 8}. The union of two sets is the set consisting of everything which is an element of at least one of the sets, A or B. As an example of the union of two sets {1, 2, 3, 8} {3, 4, 7, 8} =. {1, 2, 3, 4, 7, 8} because these numbers are those which are in at least one of the two sets. In general A B {x : x A or x B} . Be sure you understand that something which is in both A and B is in the union. It is not an exclusive or. The intersection of two sets, A and B consists of everything which is in both of the sets. Thus {1, 2, 3, 8} {3, 4, 7, 8} = {3, 8} because 3 and 8 are those elements the two sets have in common. In general, A B {x : x A and x B} . The symbol [a, b] where a and b are real numbers, denotes the set of real numbers x, such that a x b and [a, b) denotes the set of real numbers such that a x < b.]