Propensity Score Matching Regression …

1 Propensity Score MatchingRegression DiscontinuityLimited Dependent VariablesChristopher F BaumEC 823: Applied EconometricsBoston College, Spring 2013 Christopher F Baum (BC / DIW)PSM, RD, LDVB oston College, Spring 20131 / 99 Propensity Score matchingPropensity Score matchingPolicy evaluation seeks to determine the effectiveness of a particularintervention. In economic policy analysis, we rarely can work withexperimental data generated by purely random assignment of subjectsto the treatment and control groups. Random assignment, analogousto the randomized clinical trial in medicine, seeks to ensure thatparticipation in the intervention, or treatment, is the only differentiatingfactor between treatment and control non-experimental economic data, we observe whether subjectswere treated or not, but in the absence of random assignment, must beconcerned with differences between the treated and non-treated.

2 Forinstance, do those individuals with higher aptitude self-select into a jobtraining program? If so, they are not similar to correspondingindividuals along that dimension, even though they may be similar inother F Baum (BC / DIW)PSM, RD, LDVB oston College, Spring 20132 / 99 Propensity Score matchingThe key concern is that of similarity. How can we find individuals whoare similar on all observable characteristics in order to match treatedand non-treated individuals (or plants, or ) With a singlemeasure, we can readily compute a measure of distance between atreated unit and each candidate match.

3 With multiple measuresdefining similarity, how are we to balance similarity along each of thosedimensions?The method ofpropensity Score Matching (PSM) allows this matchingproblem to be reduced to a single dimension: that of the propensityscore. That Score is defined as the probability that a unit in the fullsample receives the treatment, given a set of observed variables. If allinformation relevant to participation and outcomes is observable to theresearcher, the Propensity Score will produce valid matches forestimating the impact of an intervention.

4 Thus, rather than Matching onall values of the variables, individual units can be compared on thebasis of their Propensity scores F Baum (BC / DIW)PSM, RD, LDVB oston College, Spring 20133 / 99 Propensity Score matchingAn important attribute of PSM methods is that they do not require thefunctional form to be correctly specified. If we used OLS methods suchasy=X +D + whereyis the outcome,Xare covariates andDis the treatmentindicator, we would be assuming that the effects of treatment areconstant across individuals. We need not make this assumption toemploy PSM.

5 As we will see, a crucial assumption is made on thecontents ofX, which should include all variables that can influence theprobability of F Baum (BC / DIW)PSM, RD, LDVB oston College, Spring 20134 / 99 Propensity Score matchingWhy use Matching methods?Why use Matching methods?The greatest challenge in evaluating a policy intervention is obtaining acredible estimate of thecounterfactual:what would have happened toparticipants (treated units) had they not participated? Without acredible answer, we cannot rule out that whatever successes haveoccurred among participants could have happened anyway.

6 Thisrelates to thefundamental problem of causal inference: it is impossibleto observe the outcomes of the same unit in both treatment conditionsat the same impact of a treatment on individuali, i, is the difference betweenpotential outcomes with and without treatment: i=Y1i Y0iwhere states 0 and 1 corrrespond to non-treatment and treatment, F Baum (BC / DIW)PSM, RD, LDVB oston College, Spring 20135 / 99 Propensity Score matchingWhy use Matching methods?To evaluate the impact of a program over the population, we maycompute the average treatment effect (ATE):ATE=E[ i] =E(Y1 Y0)Most often, we want to compute the average treatment effect on thetreated (ATT):ATT=E(Y1 Y0|D=1)whereD=1 refers to the F Baum (BC / DIW)PSM, RD, LDVB oston College, Spring 20136 / 99 Propensity Score matchingWhy use Matching methods?

7 The problem is that not all of these parameters are observable, as theyrely on counterfactual outcomes. For instance, we can rewrite ATT asATT=E(Y1|D=1) E(Y0|D=1)The second term is the average outcome of treated individuals hadthey not received the treatment. We cannot observe that, but we doobserve a corresponding quantity for the untreated, and can compute =E(Y1|D=1) E(Y0|D=0)The difference between ATT and can be defined as =ATT+SBwhere SB is the selection bias term: the difference between thecounterfactual for treated units and observed outcomes for F Baum (BC / DIW)PSM, RD, LDVB oston College, Spring 20137 / 99 Propensity Score matchingWhy use Matching methods?

8 For the computable quantity to be useful, the SB term must be selection bias in a non-experimental context is often sizable. Forinstance, those who voluntarily sign up for a teacher-training programmay be the more motivated teachers, who might be more likely to dowell (in terms of student test scores) even in the absence of other cases, the bias may not arise due to individuals self-selectinginto treatment, but being selected for treatment on the basis of aninterview or evaluation of their willingness to cooperate with theprogram.

9 This gives rise to administrative selection bias or programplacement in the case of a randomized experiment, participants selected fortreatment may choose not to be treated, or may not comply with allaspects of the treatment regime. In this sense, even a randomized trialmay involve bias in evaluating the effects of treatment, andnonexperimental methods may be required to adjust for that F Baum (BC / DIW)PSM, RD, LDVB oston College, Spring 20138 / 99 Propensity Score matchingRequirements for PSM validityRequirements for PSM validityTwo key assumptions underly the use of Matching methods, and PSMin particular:1 Conditional independence.

10 There exists a setXof observablecovariates such that after controlling for these covariates, thepotential outcomes are independent of treatment status:(Y1,Y0) D|X2 Common support: for each value foX, there is a positiveprobability of being both treated and untreated:0<P(D=1|X)<1 Christopher F Baum (BC / DIW)PSM, RD, LDVB oston College, Spring 20139 / 99 Propensity Score matchingRequirements for PSM validityThe conditional independence assumption(Y1,Y0) D|Ximplies that after controlling forX, the assignment of units to treatmentis as good as random.