Chapter 6: Curve Fitting - McMaster University

1 Chapter 6: Curve FittingTwo types of Curve Fitting Least square regressionGiven data for discrete values, derive a single Curve that represents the generaltrend of the data. When the given data exhibit a significant degree of error or noise. InterpolationGiven data for discrete values, fit a Curve or a series of curves that pass di-rectly through each of the points. When data are very I: Least Square Regression1 Simple Linear RegressionFitting a straight line to a set of paired observations(x1, y1),(x2, y2).

2 ,(xn, yn).Mathematical expression for the straight line (model)y=a0+a1xwherea0is the intercept, anda1is the ,measured yi,model=yi (a0+a1xi)Criterion for a best fit:minSr= mina0,a1n i=1e2i= mina0,a1n i=1(yi a0 a1xi)2 Finda0anda1:2 Sr a0= 2n i=1(yi a0 a1xi) = 0 (1) Sr a1= 2n i=1[(yi a0 a1xi)xi] = 0 (2)From (1), ni=1yi ni=1a0 ni=1a1xi= 0, orna0+n i=1xia1=n i=1yi(3)From (2), ni=1xiyi ni=1a0xi ni=1a1x2i= 0, orn i=1xia0+n i=1x2ia1=n i=1xiyi(4)(3) and (4) are called normal (3),a0=1nn i=1yi 1nn i=1xia1= y xa1where x=1n ni=1xi, y=1n ni= (4), ni=1xi(1n ni=1yi 1n ni=1xia1) + ni=1x2ia1= ni=1xiyi,a1= ni=1xiyi 1n ni=1xi ni=1yi ni=1x2i 1n( ni=1xi)2ora1=n ni=1xiyi ni=1xi ni=1yin ni=1x2i ( ni=1xi)2 Definitions.

3 Sr=n i=1e2i=n i=1(yi a0 a1xi)2 Standard error of the estimate:Sy/x= Srn 2 Spread around the regression lineStandard deviation of data pointsSy= Stn 1= ni=1(yi y)2n 14whereSt= ni=1(yi y)2. Spread around the mean value coefficient:r= St SrSt Improvement or error reduction due to describing the data in terms of astraight line rather than as an average criteria for regression (a)min ei, (b)min |ei|, and (c)min maxei5 Figure 1: (a) Spread of data around mean of dependent variable, (b) spread of data around the best-fit lineIllustration of linear regression with (a) small and (b) large residual errors6 xi= 1 + 2 +.

4 + 7 = 28 yi= + +..+ = 24 x2i= 12+ 22+..+ 72= 140 xiyi= 1 + 2 +..+ 7 = ni=1xiyi ni=1xi ni=1yin ni=1x2i ( ni=1xi)2=7 28 247 140 282= y xa1=1n yi a11n xi=17 24 17 28 = :y= + ni=1e2i,ei=yi (a0+a1xi)e1= 1 = 2 = 3 = ..e7= 7 = ( )2+ + ( )2+..+ ni=1(yi y)2= deviation of data points:Sy= Stn 1= error of the estimate:Sy/x= Srn 2= 2= < Sy,Sr< coefficientr= St SrSt= RegressionGiven data(xi, yi),i= 1,2, .. , n, fit a second order polynomialy=a0+a1x+a2x2ei=yi,measured yi,model=yi (a0+a1xi+a2x2i)Criterion for a best fit:minSr= mina0,a1,a2n i=1e2i= mina0,a1,a2n i=1(yi a0 a1xi a2x2i)2 Finda0,a1, anda2.

5 8 Sr a0= 2n i=1(yi a0 a1xi a2x2i) = 0 (1) Sr a1= 2n i=1[(yi a0 a1xi a2x2i)xi] = 0 (2) Sr a2= 2n i=1[(yi a0 a1xi a2x2i)x2i] = 0 (3)From (1), ni=1yi ni=1a0 ni=1a1xi ni=1a2x2i= 0, orna0+n i=1xia1+n i=1x2ia2=n i=1yi(1 )From (2), ni=1xiyi ni=1a0xi ni=1a1x2i ni=1x3ia2= 0, orn i=1xia0+n i=1x2ia1+n i=1x3ia2=n i=1xiyi(2 )From (3), ni=1x2iyi ni=1a0x2i ni=1a1x3i ni=1x4ia2= 0, orn i=1x2ia0+n i=1x3ia1+n i=1x4ia2=n i=1x2iyi(3 )9 Comments: The problem of determining a least-squares second order polynomial is equiv-alent to solving a system of 3 simultaneous linear equations.

6 In general, to fit anm-th order polynomialy=a0+a1x1+a2x2+..+amxmusing least-square regression is equivalent to solving a system of(m+ 1)simultaneous linear error:Sy/x= Srn (m+1)3 Multiple Linear RegressionMultiple linear regression is used whenyis a linear function of 2 or more inde-pendent :y=a0+a1x1+ data(x1i, x2i, yi),i= 1,2, .. , nei=yi,measured yi,modelSr= ni=1e2i= ni=1(yi a0 a1x1i a2x2i)2 Finda0,a1, anda2to Sr a0= 2n i=1(yi a0 a1x1i a2x2i) = 0 (1) Sr a1= 2n i=1[(yi a0 a1x1i a2x2i)x1i] = 0 (2) Sr a2= 2n i=1[(yi a0 a1x1i a2x2i)x2i] = 0 (3)From (1),na0+ ni=1x1ia1+ ni=1x2ia2= ni=1yi(1 )From (2), ni=1x1ia0+ ni=1x21ia1+ ni=1x1ix2ia2= ni=1x1iyi(2 )From (3), ni=1x2ia0+ ni=1x1ix2ia1+ ni=1x22ia2= ni=1x2iyi(3 ) n x1i x2i x1i x21i x1ix2i x2i x1ix2i x22i a0a1a2 = yi x1iyi x2iyi Standard error.

7 Sy/x= Srn (m+1)114 General Linear Least SquaresModel:y=a0Z0+a1Z1+a2Z2+..+amZmwhe reZ0, Z1, .. , Zmare(m+ 1)different cases: Simple linear LSR:Z0= 1,Z1=x,Zi= 0fori 2 Polynomial LSR:Zi=xi(Z0= 1,Z1=x,Z2=x2, ..) Multiple linear LSR:Z0= 1,Zi=xifori 1 Linear indicates the model s dependence on its parameters,ai s. The functionscan be highly ni=1e2i= ni=1(yi,measured yi,model)2 Given data(Z0i, Z1i, .. , Zmi, yi),i= 1,2, .. , n,Sr=n i=1(yi m j=0ajZji)2 Findaj,j= 0,1,2, .. , mto Sr ak= 2n i=1(yi m j=0ajZji) Zki= 0n i=1yiZki=n i=1m j=0 ZkiZjiaj, k= 0,1.

8 , mm j=0n i=1 ZkiZjiaj=n i=1yiZkiZTZA=ZTYwhereZ= Z01Z11.. Zm1Z02Z12.. Zm2..Z0nZ1n.. Zmn 13 PART II: Polynomial InterpolationGiven(n+ 1)data points,(xi, yi),i= 0,1,2, .. , n, there is one and only onepolynomial of ordernthat passes through all the s Divided-Difference Interpolating PolynomialsLinear InterpolationGiven(x0, y0)and(x1, y1)y1 y0x1 x0=f1(x) y0x x0f1(x) =y0+y1 y0x1 x0(x x0)f1(x): first order interpolationA smaller interval, ,|x1 x0|closer to zero, leads to better :Givenln 1 = 0,ln 6 = , use linear interpolation to findln :f1(2) = ln 2 = ln 1 +ln 6 ln 16 1 (2 1) = solution:ln 2 = t=|f1(2) ln 2ln 2| 100% =| | 100% = 2.

9 A smaller interval provides a better estimate16 Quadratic InterpolationGiven 3 data points,(x0, y0),(x1, y1), and(x2, y2), we can have a second orderpolynomialf2(x) =b0+b1(x x0) +b2(x x0)(x x1)f2(x0) =b0=y0f2(x1) =b0+b1(x1 x0) =y1, b1=y1 y0x1 x0f2(x2) =b0+b1(x2 x0) +b2(x2 x0)(x2 x1) =y2, b2=y2 y1x2 x1 y1 y0x1 x0x2 x0(*)Proof (*):b2=y2 b0 b1(x2 x0)(x2 x0)(x2 x1)=y2 y0 (y1 y0)(x2 x0)x1 x0(x2 x0)(x2 x1)=(y2 y0)(x1 x0) (y1 y0)(x2 x0)(x2 x0)(x2 x1)(x1 x0)=y2(x1 x0) y0x1+y0x0 (y1 y0)x2+y1x0 y0x0(x2 x0)(x2 x1)(x1 x0)=y2(x1 x0) y1x1+y1x0 (y1 y0)x2+y1x1 y0x1(x2 x0)(x2 x1)(x1 x0)=(y2 y1)(x1 x0) (y1 y0)(x2 x1)(x2 x0)(x2 x1)(x1 x0)17 Comments: In the expression off2(x), b0+b1(x x0)is linear interpolating from(x0, y0)and(x1, y1), and +b2(x x0)(x x1)introduces second order : Givenln 1 = 0,ln 4 = , andln 6 = , findln.

10 (x0, y0) = (1,0),(x1, y1) = (4, ),(x2, y2) = (6, )b0=y0= 0b1=y1 y0x1 x0= 04 1= y1x2 x1 y1 y0x1 x0x2 x0= 4 1= (x) = (x 1) (x 1)(x 4)f2(2) = t=|f2(2) ln 2ln 2| 100% = 3: Quadratic interpolation provides a better estimate than linear interpolation19 Straightforward Approachy=a0+a1x+a2x2a0+a1x0+a2x20=y0a0+ a1x1+a2x21=y1a0+a1x2+a2x22=y2or 1x0x201x1x211x2x22 a0a1a2 = y0y1y2 20 General Form of Newton s Interpolating PolynomialGiven(n+ 1)data points,(xi, yi),i= 0,1, .. , n, fit ann-th order polynomialfn(x) =b0+b1(x x0)+.