Survival Data Analysis - Harvard University

1 Survival data AnalysisKosuke ImaiPrinceton UniversityPOL573 Quantitative Analysis IIIFall 2016 Kosuke Imai (Princeton) Survival DataPOL573 Fall 20151 / 39 Readings(Required) Freedman. (2008). Survival Analysis : A Primer TheAmerican Statistician, Vol. 62, pp. 110 119.(Required) Various articles on cabinet dissolution (referencesgiven in later slides)(Suggested) Wooldridge, Chapter 20.(Suggested) Box-Steffensmeier and Jones (1997). Time is of theEssence: Event History Models in Political Science, AmericanJournal of Political Science, Vol.

2 41, pp. 1414 1461.(Reference) Box-Steffensmeier and Jones. (2004).Event HistoryModeling: A Guide for Social Scientists. Cambridge UP.(Reference) Kalbfleisch and Prentice. (2002).The StatisticalAnalysis of Failure Time data . 2nd ed. Wiley & Imai (Princeton) Survival DataPOL573 Fall 20152 / 39 What is Survival data ?Duration data consisting of start time and end timeA running example: Cabinet durationOther examples: Congressional career, Peace agreement how long does something last?Estimating the causal effect of one factor on durationKey methodological issues: censoring, shape of hazard functionKosuke Imai (Princeton) Survival DataPOL573 Fall 20153 / 39 Survival FunctionYi [0, ): time to an event or failure timeSurvival function: Probability of surviving at least up to timeyS(y) Pr(Yi>y) =1 Pr(Yi y) CDF1 Monotonically decreasing2 Right-continuous3limy S(y) =0 and limy 0S(y) =1 Density function ofYi(if it exists).]

3 F(y) = ddyS(y)andS(y) = yf(t)dtFor sufficiently smallhand iff(y)is continuous aty,h f(y) Pr(y Yi<y+h) =S(y) S(y+h)Kosuke Imai (Princeton) Survival DataPOL573 Fall 20154 / 39 Hazard FunctionWhat is it?: The instantaneous rate at which events occur at timeygiven that the observation survived up to timey (y) limh 0Pr(y Yi<y+h|Yi y)h=f(y)S(y)= ddylogS(y)It s not probability!Hazard function completely determines the distribution ofYi:S(y) =exp( y0 (t)dt)Cumulative hazard function: (y) y0 (t)dt= logS(y)Kosuke Imai (Princeton) Survival DataPOL573 Fall 20155 / 39 Quantities of InterestSurvival curveHazard function is difficult to interpretExpected time to an event: (y) E(Yi y|Yi>y)= y(t y)f(t)S(y)dt=1S(y) yS(t)dtNote that (0) =E(Yi) = 0S(t)dt (y)also completely determines distribution ofYi.

4 S(y) = (0) (y)exp( y01 (t)dt)Kosuke Imai (Princeton) Survival DataPOL573 Fall 20156 / 39 CensoringRight-censoringYi (t, ): an observation does not experiencean event during the study periodIndependence assumption: (GivenXi) Hazard rates of those whoare censored do not systematically differ from those who are notlimh 0Pr(y Yi<y+h|Yi y,Xi)h=limh 0Pr(y Yi<y+h|Yi y,Ci y,Xi)hfor allywhereCibe a time to censoringImplied byYi Ci|Xi: Missing at random; censoring is ignorableOther types of censoring:1 Left-censoringYi [0,t]2 Interval-censoringYi (t1,t2)Left-truncation: units failing before the study periodKosuke Imai (Princeton) Survival DataPOL573 Fall 20157 / 39 Cabinet Duration and CensoringKing, Alt, Burns, and Laver (1990).

5 AJPSC ensoring: 12 months prior to the end of the constitutionalinter-election periodIs censoring independent?Kosuke Imai (Princeton) Survival DataPOL573 Fall 20158 / 39 Discrete Time ApproximationTime is continuous but we observe discrete time:t1<t2< Density function:f(tj) =Pr(Yi=tj) Survival function:S(y) = {j:tj>y}f(tj)Hazard function: (tj) =Pr(Yi=tj|Yi tj) =f(tj)limt tjS(t)=f(tj)S(tj 1)Cumulative hazard function: (y) = {j:tj y} (tj)Key relationships:S(tj) =j k=1(1 (tk))andf(tj) = (tj)j 1 k=1(1 (tk))Expected time to an event.

6 (tj) =1S(tj) k=j(tk+1 tk)S(tk)Kosuke Imai (Princeton) Survival DataPOL573 Fall 20159 / 39 Nonparametric ML Estimation of Survival FunctionIdea: Use one minus the empirical CDF to estimateS(y)Observed failure times:t1<t2< <tJTies are allowed, ,J n, anddjunits failed attjmjunits are right-censored in the interval[tj,tj+1)and fail in theinterval[tj+1, )Under the independence assumption,Pr(Yi=tj) =S(tj 1) S(tj) =f(tj)uncensoredPr(Yi>tj) =S(tj)censoredThe likelihood function:L( |Y) =J j=1f(tj)djS(tj)mj=J j=1 (tj)dj(1 (tj))nj djwherenj= Jk=j(dk+mk)is the number of units at risk Kosuke Imai (Princeton) Survival DataPOL573 Fall 201510 / 39 Kaplan-Meier EstimatorThe log-likelihood function:ln( |Y) =J j=1{djlog (tj) + (nj dj)log(1 (tj))}The Kaplan-Meier estimator: S(y) = {j:tj y}(1 (tj)) = {j:tj y}nj djnjThe Asymptotic variance estimator: V( S(y)) = S2(y) {j:tj y}djnj(nj dj)The Nelson-Aalen estimator: (y) = {j.]]}

7 Tj y}dj/njKosuke Imai (Princeton) Survival DataPOL573 Fall 201511 / 39 Compliance in International LawCommitment and Compliance in International Monetary Affairs December 2000 FIGURE 2. The Kaplan-Meier Survival Function Duration of Article XIV Status over Time Kaplan-Meier Survival estimate S(t) - 0 20 40 60 Number of Years since Joining the IMF hazard rate. data availability causes the sample size to change somewhat, but none of the conclusions dis- cussed below are altered substantially by analyzing a common sample.

8 The first two variables, Universality and Regional Norm, test the proposition that taking on an obligation is likely to be related to similar actions by others. Both of these have a large and positive influence on the acceptance rate. According to model 2, for example, every increase of one percentage point in the propor- tion of IMF members who accept Article VIII in- creases by the likelihood of acceptance by other members. A similar increase in the regional proportion of Article VIII adherents increases a country's likeli- hood of acceptance by This translates into an increase of 31% in the rate of acceptance for every increase of 10% in regional Clearly, as the number of countries who adhere to Article VIII in- creases, there is a much greater chance that an uncom- mitted government will do so.

9 Note that this effect is significant even when controlling for time (Year in model 4), which reduces the likelihood that the univer- sality and regional norms variables simply reflect an increase in adherents over time. There is also evidence that institutional incentives make some difference in Article VIII acceptance. Surveillance (a dummy variable with the value of 0 before 1977, 1 thereafter) has the expected negative 12 This is calculated by raising the estimated hazard ratio to the tenth power. 824 effect, although it is not robustly statistically significant for this sample of countries, many of which joined the IMF after surveillance became mandatory for all mem- Moreover, the coefficient is significantly weak- ened by including year as an explanatory variable.

10 This is understandable, since surveillance is a dichotomous dummy that distinguishes years before and after 1977. Flexible exchange. rates probably increase the likeli- hood of making an Article VIII commitment, since these minimize the foreign exchange reserves needed to defend a beleaguered currency, but the statistical significance varies with the model specification. Perhaps most interesting is the fairly strong and consistent negative effect of the use of fund credits on the Article VIII decision. This provides evidence that the IMF is not effectively using resources as leverage to pressure borrowers into making a legal In fact, Use of Fund Credits reduces the proportional 13 For countries that had joined the IMF by 1980, earlier research shows that the change in the surveillance regime had a very strong negative effect on the decision to commit to Article VIII, which indicates a rather perverse incentive to commit.