Chapter 1

1 RS Chapter 1 Random Variables6/14/20191 Chapter 1 Probability theory : IntroductionBasic Probability General In a probability space ( , , P), the set is the set of all possible outcomesof a probability experiment . Mathematically, is just a set, with elements . It is called the sample space . An eventis the answer to a Yes/No question. Equivalently, an event is a subset of the probability space : A . Think of A as the set of outcomes where the answer is Yes , and Acis the complementary set where the answer is No . A -algebra is a mathematical model of a state of partial knowledge about the outcome. Informally, if is a -algebra and A , we say that A if we know whether A or Chapter 1 Random Variables6/14/20192 Definitions AlgebraDefinitions: Semiring(of sets)A collection of sets Fis called a semiringif it satisfies: F.

2 If A, B F, then A B F. If A, B F, then there exists a collection of sets C1, C2, .., Cn F, such that A \ B = .(A\B all elements of A not in B)Definitions: AlgebraA collection of sets F is called an algebraif it satisfies: F. If 1 F, then 1C F.(Fis closed under complementation) If 1 F & 2 F,then 1 2 F. (Fis closed under finite unions).Definitions: Sigma-algebraDefinition: Sigma-algebraA sigma-algebra( -algebra or -field) Fis a set of subsets of : F. If F, then C F.( C= complement of ) If 1, 2,.., n,.. F, then, F( 1i s are countable)Note: The set E = {{ },{a},{b},{c},{a,b},{b,c},{a,c},{a,b,c}} is an algebra and a -algebra. -algebras are a subset of algebras in the sense that all -algebras are algebras, but not vice versa. Algebras only require that they be closed under pairwise unions while -algebras must be closed under countablyinfinite Chapter 1 Random Variables6/14/20193 Theorem: All -algebras are algebras, and all algebras are , if we require a set to be a semiring, it is sufficient to show instead that it is a -algebra or algebra.

3 Sigma algebras can be generated from arbitrary sets. This will be useful in developing the probability :For some set X, the intersection of all -algebras, Ai, containingX that is, x X x Aifor all i is itself a -algebra, denoted (X). This is called the -algebra generatedby space , Definition:Sample SpaceThe sample space is the set of all possible unique outcomes of the experiment at : If we roll a die, = {1; 2; 3; 4; 5; 6}.In the probability space , the -algebra we use is ( ), the -algebra generated by . Thus, take the elements of and generate the "extended set" consisting of all unions, compliments, compliments of unions, unions of compliments, etc. Include ; with this "extended set" and the result is ( ), which we denote as .DefinitionThe -algebra generated by , denoted , is the collection of possible events from the experiment at Chapter 1 Random Variables6/14/20194 DefinitionThe -algebra generated by , denoted , is the collection of possible events from the experiment at : We have an experiment with = {1, 2}.

4 Then, = {{ },{1},{2},{1,2}}. Each of the elements of is an event. Think of events as descriptions of experiment outcomes ( : the nothing occurs event).Note that -algebras can be defined over the real line as well as over abstract sets. To develop this notion, we need the concept of a : There are many definitions of topology based on the concepts of neighborhoods, open sets, closed set, etc. We present the definition based on open sets.( , )Definition (via open sets): A topological space is an ordered pair (X, ), whereXis a set and is a collection of subsets ofX, satisfying:1. ; X 2. is closed under finite is closed under arbitrary element of a topology is known as an open set. The collection is called : We have an experiment with = {1, 2, 3}. Then, = {{ },{1,2,3}} is a (trivial) topology on.

5 = {{ },{1},{1,2,3}} is also a topology on . = {{ },{1,2},{2,3},{1,2,3}} is NOT a topology on . Topological SpaceRS Chapter 1 Random Variables6/14/20195 Definition: Borel -algebra (Emile Borel (1871-1956), France.)The Borel -algebra(or, Borel field) denoted B, of the topological space (X; ) is the -algebra generated by the family of open sets. Its elements are called Borel : Let C = {(a; b): a < b}. Then (C) = BRis the Borel field generated by the family of all open intervals do elements of BRlook like? Take all possible open intervals. Take their compliments. Take arbitrary unions. Include and R. BRcontains a wide range of intervals including open, closed, and half-open intervals. It also contains disjoint intervals such as {(2; 7] U (19; 32)}. It contains (nearly) every possible collection of intervals that are -algebraDefinition: measurable space A pair (X, ) is a measurable spaceif Xis a set and is a nonempty -algebra of subsets of measurable space allows us to define a function that assigns real-numbered values to the abstract elements of.)

6 Definition: measure Let (X, ) be a measurable space . Aset function defined on is called a measureiff it has the following 0 (A) for any A .2. ( ) = ( -additivity). For any sequence of pairwise disjoint sets {An} such that Un=1An , we have MeasuresRS Chapter 1 Random Variables6/14/20196 Intuition: A measure on a set, S, is a systematic way to assign a positive number to each suitable subset of that set, intuitively interpreted as its size. In this sense, it generalizes the concepts of length, area, Examples of measures:- Counting measure : (S) = number of elements in Lebesgue measure on R: (S) = conventional length of is, if S = [a,b] (S) = [a,b] = b Note: A measure may take as its value. Rules:(1) For any x R, + x= , x * = if x> 0, x * = if x< 0, and 0 * = 0;(2) + = ;(3) * a= for any a> 0;(4) or / are not : measure space A triplet (X, , ) is a measure space if (X, ) is a measurable space and : [0; ) is a measure .]

7 If (X) = 1, then is a probability measure , which we usually use notation P, and the measure space is a probability & measure SpaceRS Chapter 1 Random Variables6/14/20197 There is a unique measure on (R, BR) that satisfies ([a, b]) = b afor every finite interval [a, b], < a b< . This is called the Lebesgue measure . If we restrict to the measurable space ([0, 1], B[0,1]), then is a probability :-Any Cartesian product of the intervals [a,b] x [c,d] is Lebesgue measurable , and its Lebesgue measure is = (b a)*(d c).- ([set of rational numbers in an interval of R]) = 0. Note: Not all sets are Lebesgue measurable . See Vitali MeasureDefinition: measure ZeroA ( -) measurable set Eis said to have ( -) measure zeroif (E) = : The singleton points in Rn, and lines and curves in Rn, n 2. By countable additivity, any countable set in Rnhas measure zero.

8 A particular property is said to hold almost everywhereif the set of points for which the property fails to hold is a set of measure : a function vanishes almost everywhere ; f = galmost everywhere . Note: Integrating anything over a set of measure zero produces 0. Changing a function on a set of measure zero does not affect the value of its MeasureRS Chapter 1 Random Variables6/14/20198 Let ( , , ) be a measure space . Then, has the following properties:(i) (Monotonicity). If A B, then (A) (B).(ii) (Subadditivity). For any sequence A1,A2, .., (Ui=1Ai) i=1 (Ai)(iii) (Continuity). If A1 A2 A3 .. (or A1 A2 A3 .. and (A1) < ), then (limn An) = limn (An),wherelimn An= Ui=1Ai(or i=1Ai) measure : PropertiesA measure space ( , , ) is called finite if ( ) is a finite real number (not ).

9 A measure is called -finiteif can be decomposed into a countable union of measurable sets of finite measure . For example, the real numbers with the Lebesgue measure are -finite but not finite. Definition:Probability space A measure space is a probability space if ( )=1. In this case, is a probability measure , which we denote P. Let P be a probability measure . The cumulative distribution function( ) of P is defined as:F(x) = P (( , x]) , x RProbability SpaceRS Chapter 1 Random Variables6/14/20199 We have two measurable spaces ( 1, 1), and ( 2, 2). We want to define a measure on the (product) space 1 2, which reflects the structure of the original measure :Product spaceLet Гi, i I, be sets, where I = {1, .., k}, kis finite or Define the product spaceas i IГi= Г1 Гk= {(a1, .., ak) : ai Гi, i I}Example:R R= R2, R R R= R3 Definition:Let ( 1, 1) and ( 2, 2) be measurable spaces.)

10 A measurable rectangle is a set of the form A B, where A 1and B 2. Let Z = 1 2 andZ0= { i IAi Bi|Ai 1, Bi 2}Product SpaceLet Z be the -algebra generated by Z0. We write Z= 1 2. Generalizing the above definition, let ( i, i), i I, be measurable spaces. Then, i I iis not necessarily a -field (an algebra of power set ( i I i)). ( i I i) is called the product -fieldon the product space i I i. ( i I i, ( i I i)) is denoted by i I( i, i).Example: i=1,..k(R, B) = (Rk, Bk).Definition:Product measureLet ( 1, 1, 1) and ( 2, 2, 2) be measure spaces. A measure on (Z = 1 2 , Z= 1 2)is called a product measure if (A B) = 1(A) 2(B),where A 1and B 2. Product space & Product MeasureRS Chapter 1 Random Variables6/14/201910 Example: Let [a1,b1] [a2,b2] R2. The product measure of the usual area [a1,b1] [a2,b2]: = (b1 a1)(b2 a2) = ([a1, b1]) ([a2, b2]) A measure on ( , ) is said to be -finiteif and only if there exists a sequence A1, A2.