An Introduction to Splines

1 An Introduction to SplinesJames H. SteigerDepartment of Psychology and Human DevelopmentVanderbilt UniversityJames H. Steiger ( vanderbilt university )An Introduction to Splines1 / 23An Introduction to Splines1 Introduction2 Piecewise Regression RevisitedPiecewise Linear RegressionLinear spline Regression3 Cubic spline RegressionJames H. Steiger ( vanderbilt university )An Introduction to Splines2 / 23 IntroductionIntroductionWhen transformation won t linearize your model, the function is complicated, and youdon t have deep theoretical predictions about the nature of theX-Yregressionrelationship, but you do want to be able to characterize it, at least to the extent ofpredicting new values, you may want to consider a generalized additive model (GAM).

2 A generalized additive model representsE(Y|X=x) as a weight sum of smoothfunctions ll briefly discuss two examples, polynomial regression and spline H. Steiger ( vanderbilt university )An Introduction to Splines3 / 23 Piecewise Regression RevisitedPiecewise Linear RegressionPiecewise RegressionNonlinear relationships between a predictor and response can sometimes be difficult to fitwith a single parameter function or a polynomial of reasonable degree, say, between 2and example, you are already familiar with the UN data relating per capita GDP withinfant mortality rates per 1000.

3 We ve seen before that these data are difficult to analyzein their original form, but can be linearized by log-transforming both the predictor are the original data H. Steiger ( vanderbilt university )An Introduction to Splines4 / 23 Piecewise Regression RevisitedPiecewise Linear RegressionPiecewise Regression> data(UN)> attach(UN)> plot(gdp, ) H. Steiger ( vanderbilt university )An Introduction to Splines5 / 23 Piecewise Regression RevisitedPiecewise Linear RegressionPiecewise Regression> plot(log(gdp),log( ))4567891012345log(gdp)log( )James H.

4 Steiger ( vanderbilt university )An Introduction to Splines6 / 23 Piecewise Regression RevisitedPiecewise Linear RegressionPiecewise RegressionHere we fit the log-log model, then back-transform it to the original metric and plot the curve.> <- lm(I(log( )) ~ I(log(gdp)))> plot(gdp, )> curve(exp(coef( )[1] + coef( )[2]*log(x)),5,43000,add=T,col="red") H. Steiger ( vanderbilt university )An Introduction to Splines7 / 23 Piecewise Regression RevisitedPiecewise Linear RegressionPiecewise RegressionThis works quite a bit better than, say, fitting a polynomial of order 5, because polynomialscan be very unstable at their boundaries!

5 > <- lm( ~ gdp + I(gdp^2)+ + I(gdp^3) + I(gdp^4) + I(gdp^5))> plot(gdp, )> b0 <- coef( )[1]> b1 <- coef( )[2]> b2 <- coef( )[3]> b3 <- coef( )[4]> b4 <- coef( )[5]> b5 <- coef( )[6]> curve(b0+b1*x + b2*x^2 + b3*x^3 + b4*x^4 ++ b5 * x^5, 4,43000,add=T,col="red") H. Steiger ( vanderbilt university )An Introduction to Splines8 / 23 Piecewise Regression RevisitedPiecewise Linear RegressionPiecewise RegressionAnother approach is to fit more than one straight motivation to do this with the present data is substantive. We can see that there aremany countries jammed up against the left of the plot withgdpvalues below 2000, andthere is a steep decline of infant mortality as a function ofgdpwithin that area of theplot.

6 Once gdp exceeds around 2000, the decline is much less , for example, we could fit one straight line to the data wheregdpis less than or equalto 2000, and another for the data points where gdp exceeds already know how to do this!James H. Steiger ( vanderbilt university )An Introduction to Splines9 / 23 Piecewise Regression RevisitedPiecewise Linear RegressionPiecewise RegressionDefine an indicator variable, and then use it as a predictor, but also allow an interactionbetween this dummy predictor andgdpWe can express the model asE( |gdp) = 0+ 1gdp+ 2(gdp>2000)++ 3gdp(gdp>2000)+The dummy variable (gdp>2000)+takes on the value 1 whengdp>2000, zerootherwise.

7 You can see that for observations where gdp exceeds 2000, the model becomesE( |gdp) = ( 0+ 2) + ( 1+ 3)gdp(1)What is the model whengdp 2000? ( )James H. Steiger ( vanderbilt university )An Introduction to Splines10 / 23 Piecewise Regression RevisitedPiecewise Linear RegressionPiecewise RegressionThe point of separation in the piecewise regression system is called can have more than one can select the knota priori(say, at the median value of the predictor), or, as in thiscase, we can allow the data to H. Steiger ( vanderbilt university )An Introduction to Splines11 / 23 Piecewise Regression RevisitedLinear spline RegressionLinear spline RegressionThis system is straightforward to implement in , the lines need not join at the force the lines to join, eliminate several intercept-difference parameters and define thesystem follows:E(Y|X) = 0+ 1X+ 2(X a1)++ 3(X a2)++.

8 + k 1(X ak)+(2)We call thislinear spline terms of the form (u)+have the valueuifuis positive, and 0 s see how this is done in R with a knot at 1750. Notice that the second line segmentstarts at a height equal to that of the first line atX= H. Steiger ( vanderbilt university )An Introduction to Splines12 / 23 Piecewise Regression RevisitedLinear spline RegressionLinear spline Regression> <- lm( ~1 + gdp + I((gdp-1750)*(gdp>1750)))> summary( )Call:lm(formula = ~ 1 + gdp + I((gdp - 1750) * (gdp >1750)))Residuals:Min 1Q Median 3Q :Estimate Std.

9 Error t value Pr(>|t|)(Intercept) <2e-16 **gdp <2e-16 **I((gdp - 1750) * (gdp > 1750)) <2e-16 **---Signif. codes: 0'**' '**' '*' '.' ''1 Residual standard error: on 190 degrees of freedom(14 observations deleted due to missingness)Multiple R-squared: , Adjusted R-squared: : on 2 and 190 DF, p-value: < H. Steiger ( vanderbilt university )An Introduction to Splines13 / 23 Piecewise Regression RevisitedLinear spline RegressionLinear spline Regression> <- coef( )[1]> <- coef( )[2]> <- coef( )[3]> <- seq(0,1750,1)> <- seq(1750,42000,1)> <- + * > <- ( + * 1750 + ( + )* )> plot(gdp, )> lines( , , col="red")> lines( , , col="blue") H.

10 Steiger ( vanderbilt university )An Introduction to Splines14 / 23 Piecewise Regression RevisitedLinear spline RegressionLinear spline RegressionWe didn t do that well with only two could probably do much better with 3 or alternative is to fit different cubic functions that are connected at the discusscubic spline regressionin the next H. Steiger ( vanderbilt university )An Introduction to Splines15 / 23 Cubic spline RegressionCubic spline RegressionCubic spline regression fits cubic functions that are joined at a series functions will look really smooth if they have the same first and second derivativesat the a system follows the formE(Y|X) = 0+ 1X+ 2X2+ 3X3+ 4(X a1)3++ 5(X a2)3++.