Statistical Modeling - Princeton University

1 Chapter 1 Statistical Statistical ModelsExample 1: (Sampling inspection). A lot containsNproducts with defective rate . Take a sample without replacement ofnproducts and getxdefective are the defective rates?Possible outcomes: GGDGGGDD , realization of do we connect the sample with the population?Modelling think of data as a realization of a the random 524: Statistical Modeling : Illustration of the sampling that a D = is large,a G = is Law: Under this physical experimentP(X=x) =(N x)(N N n x)(Nn),for max(0,n N(1 ))6x6min(n,N ).

2 Convention:(n0)= 1,(nm)= 0 ifm > example,X/n and n(X/n ) N(0, (1 )).ORF 524: Statistical Modeling : unknown, space : the possible value of : ={0/N,1/N, ,N/N}or[0,1].For this specific example, the model comes from physical experiment. Now sup-pose thatN= 10,000,n= 100 andx= 2. Our problem becomes an inverseproblem: What is the value of ?Logically, if = 1%, it is possible to getx= 2. If = 2%, it is also possibleto getx= 2. If = , it is also possible to getx= 2. So, givenx= 2,we can not tell exactly which it is. Our conclusion can not be drawn withoutuncertainty.

3 However, we do know some are more likely than the others and thedegree of uncertainty gets smaller, asngets large, : Statisticians think data as realizations from a stochastic model; this connectsORF 524: Statistical Modeling sample and parameters. Statistical conclusions can not be drawn without uncertainty, as we have only afinite sample. Probability is from a box to sample, while statistics is from a sample to a 2: A measurement model ( molecular weight, RNA/protein expres-sion level, fat-free weight). An object is weighedntimes, with outcomesx1, , be the true weight.

4 We think the observed data as realizations of randomvariablesX1, ,Xn, modeled asXi= + iwhere iis error of measurement ) iis independent of .ii) i,i= 1,2, ,nare 524: Statistical Modeling : Illustration of the idea of ) i,i= 1,2, ,nare identically ) the distribution of is continuous, withE( ) = 0; or specifically symmetricabout 0:f(y) =f( y) for , we assume further that i N(0, 2). Parameters in the model =( , 2), where 2is a nuisance a realizationx= (x1, ,xn) ofX= (X1, ,Xn), what is the value of ?ORF 524: Statistical Modeling , if = 100, it is possible to observex.

5 If = 1, it is also possible toobservex. So we can not absolutely tell what value of is. But from the square-rootlaw:var( X) =E( X )2= , xis likely close to whennis : Distributions of individual observation versus that of averageExample 3: Drug evaluation (Hypertension drug)Drug A m patiets Drug B n patietsORF 524: Statistical Modeling : blood eliminate confounding factors, use randomized controlled experiment. Hereare the hypothetical outcomes:Drug ADrug B150 110 160 187 153120 140 160 180 133 136x1x2x3x4x5y1y2y3y4y5y6To model the outcomes, a possible idealization is the following : Illustration of a two-sample problemDrug ADrug Brandom outcomesX1, ,XmY1 ,Ynrealizationsx1, ,xmy1, ,ynORF 524: Statistical Modeling , we might assume thatX1, , N( A, 2A)Y1, , N( B, 2B).

6 We sometimes assume further A= B= .Parameters in the model: = ( A, B, A, B).Parameters of interest: = A Band possibly .Connection sample with population: data are realizations from a population,whose distribution depends on .Model diagnostics: Statistical models are idealizations, postulated by statisti-cians needed to be verified. For example, the data histograms should look liketheoretical distributions. Two sample variances are about the same, formulationData:x= (x1, ,xn) are thought of the realization of a random vectorX=(X1, ,Xn).ORF 524: Statistical Modeling : The distribution ofXis assumed inP={P : }, is the : Inferences about.

7 In Example 1:P (x) =(N x)(N N n x)(Nn),where ={0,1/N, ,N/N}or [0,1]. In Example 2:P (x) = ni=1 1 (xi )where ( ) is the normal density, ={( , ), >0, >0}. In Example 3:P (x) = mi=1 1A (xi A A) ni=1 1B (yi B B),where ( ) is the normal density, ={( A, B, A, B) : A, B, A, B>0}.ORF 524: Statistical Modeling Dataxor its random variableXcan include bothx- parameter doesn t have to be inRk. In Example 2, without the normalityassumption,P (x) = ni=1f(xi ),assuming that{ i,i= 1, ,n}are random variables with densityf. Then, ={( ,f) : >0,fis symmetric}.

8 Since no form offhas been imposed, not been parameterized, theparameter space is called nonparametric or assumption: Throughout this class, we will assume that(i) Continuous variables: AllP are continuous with densitiesp(x, ) or(ii) Discrete variable:AllP are discrete with frequency functionsp(x, ). Further,there exists a set{x1,x2, ,}such that i=1p(xi, ) = 1,wherexiis independent of .ORF 524: Statistical Modeling convenience, we will callp(x, ) as density in both of parameters: There are sometimes more than one way ofparameterization.

9 In Example 3: writeX1, , N( + 1, 2)Y1, , N( + 2, 2). = ( , 1, 2, ). Hence,p (x,y, ) = mi=1 1 (xi 1 ) ni=1 1 (yi 2 ),If 1= (0,1,2,1) and 2= ( , , ,1), thenP 1=P 2. Thus, the parameters are not : The model{P , }is identifiable if 16= 2impliesP 16=P 4: (Regression Problem). Suppose a sample of data{(xi1, ,xip,yi)}ni=1are collected ,x1=age,x2= year of experience,x3= job grade,x4= gender,x5= PC 524: Statistical Modeling wish to study the association betweenYandX1, ,Xp. How to predictYbased onX? Any gender discrimination?

10 (Note: the dataxin the generalformulation now include all{(xi1, ,xip,yi)}ni=1). Model I: linear modelY= 0+ 1X1+ 2X2+ + 5X5+ , G,where is the part that can not be explained byX. Thus the parameter spaceis ={( 0, 1, , 5,G)}. Model II: semiparametric modelY= (X1,X2,X3) + 4X4+ 5X5+ .The parameter space is ={( ( ), 4, 5,G)}. Model III: nonparametric modelY= (X1, ,X5) + .ORF 524: Statistical Modeling parameter space is ={( ( ),G)}. Modeling : Data are thought of a realization from (Y,X1, ,X5) with the rela-tionship betweenXandYdescribed this example, the model is a convenient assumption made by data , Statistical models are frequently useful fictions.