Mathematics for Computer Science - MIT OpenCourseWare

1 Mcs 2015/5/18 1:43 page i #1 Mathematics for Computer Science revised Monday 18th May, 2015, 01:43 Eric Lehman Google Inc. F Thomson LeightonDepartment of Mathematics and the Computer Science and AI Laboratory, Massachussetts Institute of Technology; Akamai Technologies Albert R MeyerDepartment of Electrical Engineering and Computer Science and the Computer Science and AI Laboratory, Massachussetts Institute of Technology 2015, Eric Lehman, F Tom Leighton,Albert R Meyer. This work is available under the terms of theCreative CommonsAttribution-NonCommercial-ShareAl ike license. mcs 2015/5/18 1:43 page ii #2 mcs 2015/5/18 1:43 page iii #3 Contents I Proofs Introduction 3 References 4 1 What is a Proof? 5 Propositions 5 Predicates 8 The Axiomatic Method 8 Our Axioms 9 Proving an Implication 11 Proving an If and Only If 13 Proof by Cases 15 Proof by Contradiction 16 Good Proofs in Practice 17 References 19 2 The Well Ordering Principle 27 Well Ordering Proofs 27 Template for Well Ordering Proofs 28 Factoring into Primes 30 Well Ordered Sets 31 3 Logical Formulas 41 Propositions from Propositions 42 Propositional Logic in Computer Programs 45 Equivalence and Validity 48 The Algebra of Propositions 50 The SAT Problem 55 Predicate Formulas 56 References 61 4 Mathematical Data Types 81 Sets 81 Sequences 86 Functions 87 Binary Relations 89 Finite Cardinality 93 mcs 2015/5/18 1.

2 43 page iv #4 iv Contents 5 Induction 115 Ordinary Induction 115 Strong Induction 124 Strong Induction vs. Induction vs. Well Ordering 129 State Machines 130 6 Recursive Data Types 173 Recursive Definitions and Structural Induction 173 Strings of Matched Brackets 177 Recursive Functions on Nonnegative Integers 180 Arithmetic Expressions 183 Induction in Computer Science 188 7 Infinite Sets 205 Infinite Cardinality 206 The Halting Problem 215 The Logic of Sets 219 Does All This Really Work? 222 II Structures Introduction 241 8 Number Theory 243 Divisibility 243 The Greatest Common Divisor 248 Prime Mysteries 254 The Fundamental Theorem of Arithmetic 257 Alan Turing 259 Modular Arithmetic 263 Remainder Arithmetic 265 Turing s Code (Version ) 268 Multiplicative Inverses and Cancelling 270 Euler s Theorem 274 RSA Public Key Encryption 279 What has SAT got to do with it?

3 281 References 282 9 Directed graphs & Partial Orders 317 Vertex Degrees 319 Walks and Paths 320 mcs 2015/5/18 1:43 page v #5 v Contents Adjacency Matrices 323 Walk Relations 326 Directed Acyclic Graphs & Scheduling 327 Partial Orders 335 Representing Partial Orders by Set Containment 339 Linear Orders 340 Product Orders 340 Equivalence Relations 341 Summary of Relational Properties 343 10 Communication Networks 373 Complete Binary Tree 373 Routing Problems 373 Network Diameter 374 Switch Count 375 Network Latency 376 Congestion 376 2-D Array 377 Butterfly 379 Benes Network 381 11 Simple Graphs 393 Vertex Adjacency and Degrees 393 Sexual Demographics in America 395 Some Common Graphs 397 Isomorphism 399 Bipartite Graphs & Matchings 401 The Stable Marriage Problem 406 Coloring 413 Simple Walks 417

4 Connectivity 419 Forests & Trees 424 References 433 12 Planar Graphs 473 Drawing Graphs in the Plane 473 Definitions of Planar Graphs 473 Euler s Formula 484 Bounding the Number of Edges in a Planar Graph 485 Returning to K5 and K3;3 486 Coloring Planar Graphs 487 mcs 2015/5/18 1:43 page vi #6 vi Contents Classifying Polyhedra 489 Another Characterization for Planar Graphs 492 III Counting Introduction 501 References 502 13 Sums and Asymptotics 503 The Value of an Annuity 504 Sums of Powers 510 Approximating Sums 512 Hanging Out Over the Edge 516 Products 522 Double Trouble 525 Asymptotic Notation 528 14 Cardinality Rules 551 Counting One Thing by Counting Another 551 Counting Sequences 552 The Generalized Product Rule 555 The Division Rule 559 Counting Subsets 562 Sequences with Repetitions 564 Counting Practice.

5 Poker Hands 567 The Pigeonhole Principle 572 Inclusion-Exclusion 581 Combinatorial Proofs 587 References 591 15 Generating Functions 627 Infinite Series 627 Counting with Generating Functions 629 Partial Fractions 635 Solving Linear Recurrences 638 Formal Power Series 643 References 646 mcs 2015/5/18 1:43 page vii #7 vii Contents IV Probability Introduction 665 16 Events and Probability Spaces 667 Let s Make a Deal 667 The Four Step Method 668 Strange Dice 677 The Birthday Principle 684 Set Theory and Probability 686 References 690 17 Conditional Probability 697 Monty Hall Confusion 697 Definition and Notation 698 The Four-Step Method for Conditional Probability 700 Why Tree Diagrams Work 702 The Law of Total Probability 710 Simpson s Paradox 712 Independence 714 Mutual Independence 716 18 Random Variables 739 Random Variable Examples 739 Independence 741 Distribution Functions 742 Great Expectations 751 Linearity of Expectation 762 19 Deviation from the Mean 789 Markov s Theorem 789 Chebyshev s

6 Theorem 792 Properties of Variance 796 Estimation by Random Sampling 800 Confidence versus Probability 806 Sums of Random Variables 807 Really Great Expectations 816 20 Random Walks 839 Gambler s Ruin 839 Random Walks on Graphs 849 mcs 2015/5/18 1:43 page viii #8 viii Contents V Recurrences Introduction 865 21 Recurrences 867 The Towers of Hanoi 867 Merge Sort 870 Linear Recurrences 874 Divide-and-Conquer Recurrences 881 A Feel for Recurrences 888 Bibliography 895 Glossary of Symbols 899 Index 902 I mcs 2015/5/18 1:43 page 1 #9 Proofs mcs 2015/5/18 1:43 page 2 #10 mcs 2015/5/18 1:43 page 3 #11 Introduction This text explains how to use mathematical models and methods to analyze prob lems that arise in Computer Science .

7 Proofs play a central role in this work because the authors share a belief with most mathematicians that proofs are essential for genuine understanding. Proofs also play a growing role in Computer Science ; they are used to certify that software and hardware will always behave correctly, some thing that no amount of testing can do. Simply put, a proof is a method of establishing truth. Like beauty, truth some times depends on the eye of the beholder, and it should not be surprising that what constitutes a proof differs among fields. For example, in the judicial system, legal truth is decided by a jury based on the allowable evidence presented at trial. In the business world, authoritative truth is specified by a trusted person or organization, or maybe just your boss. In fields such as physics or biology, scientific truth is confirmed by In statistics, probable truth is established by statistical analysis of sample data.

8 Philosophical proof involves careful exposition and persuasion typically based on a series of small, plausible arguments. The best example begins with Cogito ergo sum, a Latin sentence that translates as I think, therefore I am. This phrase comes from the beginning of a 17th century essay by the mathematician/philosopher, Ren e Descartes, and it is one of the most famous quotes in the world: do a web search for it, and you will be flooded with hits. Deducing your existence from the fact that you re thinking about your existence is a pretty cool and persuasive-sounding idea. However, with just a few more lines 1 Actually, only scientific falsehood can be demonstrated by an experiment when the experiment fails to behave as predicted. But no amount of experiment can confirm that the next experiment won t fail.

9 For this reason, scientists rarely speak of truth, but rather of theories that accurately predict past, and anticipated future, experiments. 4 mcs 2015/5/18 1:43 page 4 #12 References of argument in this vein, Descartes goes on to conclude that there is an infinitely beneficent God. Whether or not you believe in an infinitely beneficent God, you ll probably agree that any very short proof of God s infinite beneficence is bound to be far-fetched. So even in masterful hands, this approach is not reliable. Mathematics has its own specific notion of proof. Definition. A mathematical proof of a proposition is a chain of logical deductions leading to the proposition from a base set of axioms. The three key ideas in this definition are highlighted: proposition, logical deduc tion, and axiom.

10 Chapter 1 examines these three ideas along with some basic ways of organizing proofs. Chapter 2 introduces the Well Ordering Principle, a basic method of proof; later, Chapter 5 introduces the closely related proof method of induction. If you re going to prove a proposition, you d better have a precise understand ing of what the proposition means. To avoid ambiguity and uncertain definitions in ordinary language, mathematicians use language very precisely, and they often express propositions using logical formulas; these are the subject of Chapter 3. The first three Chapters assume the reader is familiar with a few mathematical concepts like sets and functions. Chapters 4 and 7 offer a more careful look at such mathematical data types, examining in particular properties and methods for proving things about infinite sets.