Methods for Constructing and Assessing Propensity Scores

1 Methods for Constructing and AssessingPropensity ScoresMelissa M. Garrido, Amy S. Kelley, Julia Paris, Katherine Roza,Diane E. Meier, R. Sean Morrison, and Melissa D. model the steps involved in preparing for and carrying out propensityscore analyses by providing step-by-step guidance and Stata code applied to an empiri-cal , Stata code, and empirical examples are given to illustrate(1) the process of choosing variables to include in the Propensity score; (2) balance ofpropensity score across treatment and comparison groups; (3) balance of covariatesacross treatment and comparison groups within blocks of the Propensity score; (4)choice of matching and weighting strategies; (5) balance of covariates after matching orweighting the sample; and (6) interpretation of treatment effect use data from the Palliative Care for Cancer Patients(PC4C) study, a multisite observational study of the effect of inpatient palliative careon patient health outcomes and health services use, to illustrate the development anduse of a Propensity Scores are one useful tool for accounting for observed differ-ences between treated and comparison groups.

2 Careful testing of Propensity Scores isrequired before using them to estimate treatment data/quasi-experiments, administrative data uses,patient outcomes/functionRecent national initiatives for comparative effectiveness research recommendharnessing the power of existing data to evaluate health-related treatmenteffects (Patient-Centered Outcomes Research Institute 2012). A difficulty inusing observational data is that patient and provider characteristics may beassociated with both treatment selection and outcome, leading to different dis-tributions of covariates within treatment and comparison groups. Propensityscore analysis is a useful tool to account for imbalance in covariates betweentreated and comparison groups. A Propensity score is a single score that repre-sents the probability of receiving a treatment, conditional on a set of observedcovariates.

3 The goal of creating a Propensity score is to balance covariates Health Research and Educational TrustDOI: CORNER1701 Health Services Researchbetween individuals who did and did not receive a treatment, making it easierto isolate the effect of a the advantages and disadvantages of using Propensity Scores arewell known ( , Stuart 2010; Brooks and Ohsfeldt 2013), it is difficult tofindspecific guidance with accompanying statistical code for the steps involved increating and Assessing Propensity Scores . Other useful Stata references glossover Propensity score assessment (treatment effects manual, StataCorp. 2013a;Stata YouTube channel, ) or provide dis-jointed information ( ). Here, we synthesize informa-tion on creation and assessment of Propensity Scores within one article.

4 In thefollowing sections, we introduce situations in which Propensity Scores mightbe used in health services research and provide step-by-step instructions andStata 13 code and output to illustrate (1) choice of variables to include in thepropensity score; (2) balance of Propensity score across treatment and compar-ison groups; (3) balance of covariates across treatment and comparison groupswithin blocks of the Propensity score; (4) choice of matching and weightingstrategies; (5) balance of covariates after matching or weighting the sample bya Propensity score; and (6) interpretation of treatment effect TOCONSIDERPROPENSITYSCORESP ropensity Scores are useful when estimating a treatment s effect on an out-come using observational data and when selection bias due to nonrandomtreatment assignment is likely.

5 The classic experimental design for estimatingtreatment effects is a randomized controlled trial (RCT), where randomassignment to treatment balances individuals observed and unobservedAddress correspondence to Melissa M. Garrido, , GRECC, James J Peters VA Medical Cen-ter, Bronx, NY; Brookdale Department of Geriatrics and Palliative Medicine, Icahn School ofMedicine at Mount Sinai, New York, NY; 130 W. Kingsbridge Road, Room 4A-17, Bronx, NY10468;e-mail: Amy S. Kelley, , , and Melissa , , , are also with the Brookdale Department of Geriatrics and PalliativeMedicine, Icahn School of Medicine at Mount Sinai, New York, NY; GRECC, James J Peters VAMedical Center, Bronx, NY. Julia Paris, , and Katherine Roza, , are with the BrookdaleDepartment of Geriatrics and Palliative Medicine, Icahn School of Medicine at Mount Sinai, NewYork, NY; Diane E.

6 Meier, , is with the Center to Advance Palliative Care; BrookdaleDepartment of Geriatrics and Palliative Medicine, Icahn School of Medicine at Mount Sinai, NewYork, NY. R. Sean Morrison, , is with the National Palliative Care Research Center, Hertz-berg Palliative Care Institute; Brookdale Department of Geriatrics and Palliative Medicine, IcahnSchool of Medicine at Mount Sinai, New York, NY; GRECC, James J Peters VA Medical Center,Bronx, : Health Services Research49:5 (October 2014)characteristics across treatment and control groups. Because only one treat-ment state can be observed at a time for each individual, control individualsthat are similar to treated individuals in everything but treatment receipt areused as proxies for the counterfactual. In observational data, however, treat-ment assignment is not random.

7 This leads to selection bias, where measuredand unmeasured characteristics of individuals are associated with likelihoodof receiving treatment and with the outcome. Propensity Scores provide a wayto balancemeasuredcovariates across treatment and comparison groups andbetter approximate the counterfactual for treated Scores can be thought of as an advanced matching instance, if one were concerned that age might affect both treatment selec-tion and outcome, one strategy would be to compare individuals of similarage in both treatment and comparison groups. As variables are added to thematching process, however, it becomes more and more difficult tofind exactmatches for individuals ( , it is unlikely tofind individuals in both the treat-ment and comparison groups with identical gender, age, race, comorbiditylevel, and insurance status).

8 Propensity Scores solve this dimensionality prob-lem by compressing the relevant factors into a single score. Individuals withsimilar Propensity Scores are then compared across treatment and health services research, Propensity Scores are useful when ran-domization of treatments is impossible (Medicare demonstration projects) orunethical (end-of-life care). In addition, health services researchers are ofteninterested in a treatment s effect on multiple outcomes (such as cost and qual-ity), and a single Propensity score can be used to evaluate multiple outcomes(Wyss et al. 2013). Recently, health services researchers have used propensityscores to reduce confounding due to selection bias in evaluations of the effectsof physical health events on mental health service use (Yoon and Bernell2013), assertive community treatment on medical costs (Slade et al.)

9 2013), andpay-for-performance on Medicare costs (Kruse et al. 2012).The theory and principles behind Propensity Scores are described else-where (Rubin 1980; Rosenbaum and Rubin 1984, 1985; Imbens 2004; Hoet al. 2007; Stuart 2010; Brooks and Ohsfeldt 2013). This article is an introduc-tory how-to guide and focuses on the steps to create and assess propensityscores for a dichotomous treatment. More advanced readers may wish to usepropensity Scores with survey-weighted data (DuGoff, Schuler, and Stuart2014) or with multilevel categorical (Imbens 2000; Huang et al. 2005) or con-tinuous treatments (Jiang and Foster 2013). We use data from the PalliativeCare for Cancer Patients (PC4C) study, an observational study of inpatientMethods for Constructing and Assessing Propensity Scores1703palliative care s effect on multiple health outcomes for individuals with used Propensity Scores to account for the fact that patients baseline healthaffects both probability of receiving palliative care and experiencing adversehealth patients were hospitalized infive facilities with established palliativecare programs in New York, Ohio, Pennsylvania, Virginia, and approval was obtained from each study site.

10 Eligible patients were18 years of age or older, had an advanced cancer diagnosis, and spoke Eng-lish. Nonverbal patients, patients with dementia, and those who had previ-ously received palliative care were admitted for chemotherapy or had lengthsof stay less than 48 hours were excluded. Of the 3,227 eligible patients whoconsented to participate, 1,537 ( percent) had complete interview andmedical record data. Most patients with incomplete data were too medicallyill to continue study treatment variable was receipt of inpatient palliative care from aninterdisciplinary dedicated consultation team. Care consisted of symptomassessment and treatment, goals of care discussions, and care transition plan-ning. Data for the Propensity score come from medical record review com-pleted by trained project staff and patient baseline interviews and dailysymptom ANDOUTPUTS tata code fragments to accompany the steps listed below are detailed in thetechnical appendix.