Chapter 6: Curve Fitting - McMaster University

1 Chapter 6: Curve FittingTwo types of Curve Fitting Least square regressionGiven data for discrete values, derive a single Curve that represents the generaltrend of the data. When the given data exhibit a significant degree of error or noise. InterpolationGiven data for discrete values, fit a Curve or a series of curves that pass di-rectly through each of the points. When data are very I: Least Square Regression1 Simple linear RegressionFitting a straight line to a set of paired observations(x1, y1),(x2, y2), .. ,(xn, yn).Mathematical expression for the straight line (model)y=a0+a1xwherea0is the intercept, anda1is the ,measured yi,model=yi (a0+a1xi)Criterion for a best fit:minSr= mina0,a1n i=1e2i= mina0,a1n i=1(yi a0 a1xi)2 Finda0anda1:2 Sr a0= 2n i=1(yi a0 a1xi) = 0 (1) Sr a1= 2n i=1[(yi a0 a1xi)xi] = 0 (2)From (1), ni=1yi ni=1a0 ni=1a1xi= 0, orna0+n i=1xia1=n i=1yi(3)From (2), ni=1xiyi ni=1a0xi ni=1a1x2i= 0, orn i=1xia0+n i=1x2ia1=n i=1xiyi(4)(3) and (4) are called normal (3),a0=1nn i=1yi 1nn i=1xia1= y xa1where x=1n ni=1xi, y=1n ni= (4), ni=1xi(1n ni=1yi 1n ni=1xia1) + ni=1x2ia1= ni=1xiyi,a1= ni=1xiyi 1n ni=1xi ni=1yi ni=1x2i 1n( ni=1xi)2ora1=n ni=1xiyi ni=1xi ni=1yin ni=1x2i ( ni=1xi)2 Definitions:Sr=n i=1e2i=n i=1(yi a0 a1xi)2 Standard error of the estimate:Sy/x= Srn 2 Spread around the regression lineStandard deviation of data pointsSy= Stn 1= ni=1(yi y)2n 14whereSt= ni=1(yi y)2.

2 Spread around the mean value coefficient:r= St SrSt Improvement or error reduction due to describing the data in terms of astraight line rather than as an average criteria for regression (a)min ei, (b)min |ei|, and (c)min maxei5 Figure 1: (a) Spread of data around mean of dependent variable, (b) spread of data around the best-fit lineIllustration of linear regression with (a) small and (b) large residual errors6 xi= 1 + 2 +..+ 7 = 28 yi= + +..+ = 24 x2i= 12+ 22+..+ 72= 140 xiyi= 1 + 2 +..+ 7 = ni=1xiyi ni=1xi ni=1yin ni=1x2i ( ni=1xi)2=7 28 247 140 282= y xa1=1n yi a11n xi=17 24 17 28 = :y= + ni=1e2i,ei=yi (a0+a1xi)e1= 1 = 2 = 3 = ..e7= 7 = ( )2+ + ( )2+..+ ni=1(yi y)2= deviation of data points:Sy= Stn 1= error of the estimate:Sy/x= Srn 2= 2= < Sy,Sr< coefficientr= St SrSt= RegressionGiven data(xi, yi),i= 1,2, .. , n, fit a second order polynomialy=a0+a1x+a2x2ei=yi,measured yi,model=yi (a0+a1xi+a2x2i)Criterion for a best fit:minSr= mina0,a1,a2n i=1e2i= mina0,a1,a2n i=1(yi a0 a1xi a2x2i)2 Finda0,a1, anda2:8 Sr a0= 2n i=1(yi a0 a1xi a2x2i) = 0 (1) Sr a1= 2n i=1[(yi a0 a1xi a2x2i)xi] = 0 (2) Sr a2= 2n i=1[(yi a0 a1xi a2x2i)x2i] = 0 (3)From (1), ni=1yi ni=1a0 ni=1a1xi ni=1a2x2i= 0, orna0+n i=1xia1+n i=1x2ia2=n i=1yi(1 )From (2), ni=1xiyi ni=1a0xi ni=1a1x2i ni=1x3ia2= 0, orn i=1xia0+n i=1x2ia1+n i=1x3ia2=n i=1xiyi(2 )From (3), ni=1x2iyi ni=1a0x2i ni=1a1x3i ni=1x4ia2= 0, orn i=1x2ia0+n i=1x3ia1+n i=1x4ia2=n i=1x2iyi(3 )9 Comments: The problem of determining a least-squares second order polynomial is equiv-alent to solving a system of 3 simultaneous linear equations.

3 In general, to fit anm-th order polynomialy=a0+a1x1+a2x2+..+amxmusing least-square regression is equivalent to solving a system of(m+ 1)simultaneous linear error:Sy/x= Srn (m+1)3 multiple linear RegressionMultiple linear regression is used whenyis a linear function of 2 or more inde-pendent :y=a0+a1x1+ data(x1i, x2i, yi),i= 1,2, .. , nei=yi,measured yi,modelSr= ni=1e2i= ni=1(yi a0 a1x1i a2x2i)2 Finda0,a1, anda2to Sr a0= 2n i=1(yi a0 a1x1i a2x2i) = 0 (1) Sr a1= 2n i=1[(yi a0 a1x1i a2x2i)x1i] = 0 (2) Sr a2= 2n i=1[(yi a0 a1x1i a2x2i)x2i] = 0 (3)From (1),na0+ ni=1x1ia1+ ni=1x2ia2= ni=1yi(1 )From (2), ni=1x1ia0+ ni=1x21ia1+ ni=1x1ix2ia2= ni=1x1iyi(2 )From (3), ni=1x2ia0+ ni=1x1ix2ia1+ ni=1x22ia2= ni=1x2iyi(3 ) n x1i x2i x1i x21i x1ix2i x2i x1ix2i x22i a0a1a2 = yi x1iyi x2iyi Standard error:Sy/x= Srn (m+1)114 General linear Least SquaresModel:y=a0Z0+a1Z1+a2Z2+..+amZmwhe reZ0, Z1, .. , Zmare(m+ 1)different cases: Simple linear LSR:Z0= 1,Z1=x,Zi= 0fori 2 Polynomial LSR:Zi=xi(Z0= 1,Z1=x,Z2=x2.)

4 multiple linear LSR:Z0= 1,Zi=xifori 1 linear indicates the model s dependence on its parameters,ai s. The functionscan be highly ni=1e2i= ni=1(yi,measured yi,model)2 Given data(Z0i, Z1i, .. , Zmi, yi),i= 1,2, .. , n,Sr=n i=1(yi m j=0ajZji)2 Findaj,j= 0,1,2, .. , mto Sr ak= 2n i=1(yi m j=0ajZji) Zki= 0n i=1yiZki=n i=1m j=0 ZkiZjiaj, k= 0,1, .. , mm j=0n i=1 ZkiZjiaj=n i=1yiZkiZTZA=ZTYwhereZ= Z01Z11.. Zm1Z02Z12.. Zm2..Z0nZ1n.. Zmn 13 PART II: Polynomial InterpolationGiven(n+ 1)data points,(xi, yi),i= 0,1,2, .. , n, there is one and only onepolynomial of ordernthat passes through all the s Divided-Difference Interpolating PolynomialsLinear InterpolationGiven(x0, y0)and(x1, y1)y1 y0x1 x0=f1(x) y0x x0f1(x) =y0+y1 y0x1 x0(x x0)f1(x): first order interpolationA smaller interval, ,|x1 x0|closer to zero, leads to better :Givenln 1 = 0,ln 6 = , use linear interpolation to findln :f1(2) = ln 2 = ln 1 +ln 6 ln 16 1 (2 1) = solution:ln 2 = t=|f1(2) ln 2ln 2| 100% =| | 100% = 2: A smaller interval provides a better estimate16 Quadratic InterpolationGiven 3 data points,(x0, y0),(x1, y1), and(x2, y2), we can have a second orderpolynomialf2(x) =b0+b1(x x0) +b2(x x0)(x x1)f2(x0) =b0=y0f2(x1) =b0+b1(x1 x0) =y1, b1=y1 y0x1 x0f2(x2) =b0+b1(x2 x0) +b2(x2 x0)(x2 x1) =y2, b2=y2 y1x2 x1 y1 y0x1 x0x2 x0(*)Proof (*).

5 B2=y2 b0 b1(x2 x0)(x2 x0)(x2 x1)=y2 y0 (y1 y0)(x2 x0)x1 x0(x2 x0)(x2 x1)=(y2 y0)(x1 x0) (y1 y0)(x2 x0)(x2 x0)(x2 x1)(x1 x0)=y2(x1 x0) y0x1+y0x0 (y1 y0)x2+y1x0 y0x0(x2 x0)(x2 x1)(x1 x0)=y2(x1 x0) y1x1+y1x0 (y1 y0)x2+y1x1 y0x1(x2 x0)(x2 x1)(x1 x0)=(y2 y1)(x1 x0) (y1 y0)(x2 x1)(x2 x0)(x2 x1)(x1 x0)17 Comments: In the expression off2(x), b0+b1(x x0)is linear interpolating from(x0, y0)and(x1, y1), and +b2(x x0)(x x1)introduces second order : Givenln 1 = 0,ln 4 = , andln 6 = , findln :(x0, y0) = (1,0),(x1, y1) = (4, ),(x2, y2) = (6, )b0=y0= 0b1=y1 y0x1 x0= 04 1= y1x2 x1 y1 y0x1 x0x2 x0= 4 1= (x) = (x 1) (x 1)(x 4)f2(2) = t=|f2(2) ln 2ln 2| 100% = 3: Quadratic interpolation provides a better estimate than linear interpolation19 Straightforward Approachy=a0+a1x+a2x2a0+a1x0+a2x20=y0a0+ a1x1+a2x21=y1a0+a1x2+a2x22=y2or 1x0x201x1x211x2x22 a0a1a2 = y0y1y2 20 General Form of Newton s Interpolating PolynomialGiven(n+ 1)data points,(xi, yi),i= 0,1, .. , n, fit ann-th order polynomialfn(x) =b0+b1(x x0)+.

6 +bn(x x0)(x x1)..(x xn) =n i=0bii 1 j=0(x xj)findb0, b1, .. , ,y0=b0orb0= ,y1=b0+b1(x1 x0), thenb1=y1 y0x1 x0 Defineb1=f[x1, x0] =y1 y0x1 ,y2=b0+b1(x2 x0) +b2(x2 x0)(x2 x1), thenb2=y2 y1x2 x1 y1 y0x1 x0x2 x0 Definef[x2, x1, x0] =f[x2,x1] f[x1,x0]x2 x0, thenb2=f[x2, x1, x0]..x=xn,bn=f[xn, xn 1, .. , x1, x0] =f[xn,xn 1,..,x1] f[xn 1,..,x1,x0]xn x0216 Lagrange Interpolating PolynomialsThe Lagrange interpolating polynomial is a reformulation of the Newton s in-terpolating polynomial that avoids the computation of divided differences. Thebasic format isfn(x) =n i=0Li(x)f(xi)whereLi(x) = nj=0,j6=ix xjxi xjLinear Interpolation (n= 1)f1(x) = 1i=0Li(x)f(xi) =L0(x)y0+L1(x)y1=x x1x0 x1y0+x x0x1 x0y1(f1(x) =y0+y1 y0x1 x0(x x0))Second Order Interpolation (n= 2)f2(x) = 2i=0Li(x)f(xi) =L0(x)y0+L1(x)y1+L2(x)y2=(x x1)(x x2)(x0 x1)(x0 x2)y0+(x x0)(x x2)(x1 x0)(x1 x2)y1+(x x0)(x x1)(x2 x0)(x2 x1)y222 Example: Givenln 1 = 0,ln 4 = , andln 6 = , findln :(x0, y0) = (1,0),(x1, y1) = (4, ),(x2, y2) = (6, )f1(x) =y0+y1 y0x1 x0(x x0) =x 41 4 0 +x 14 1 = (x) =(x x1)(x x2)(x0 x1)(x0 x2)y0+(x x0)(x x2)(x1 x0)(x1 x2)y1+(x x0)(x x1)(x2 x0)(x2 x1)y2=(x 4)(x 6)(1 4)(1 6) 0 +(x 1)(x 6)(4 1)(4 6) +(x 1)(x 4)(6 1)(6 4) =.

7 Findf( )by interpolating the following table of (1) Use Lagrange interpolationf2(x) = 3i=1Li(x)f(xi),Li(x) = 3j=1,j6=ix xjxi xjL1(x) =(x x2)(x x3)(x1 x2)(x1 x3)=( 2)( 3)(1 2)(1 3)= (x) =(x x1)(x x3)(x2 x1)(x2 x3)=( 1)( 3)(2 1)(2 3)= (x) =(x x1)(x x2)(x3 x1)(x3 x2)=( 1)( 2)(3 1)(3 2)= ( ) = + + = (2) use Newton s interpolationf2(x) =b0+b1(x x1) +b2(x x1)(x x2)b0=y1= y1x2 x1= 1= y1x2 x1 y1 y0x1 x0x2 x0= 2 1= ( ) = + ( 1) + ( 1)( 2) = (3) Use the straightforward methodf2(x) =a0+a1x+a2x2a0+a1+a2 12= +a1+a2 22= +a1+a2 32= 1 1 11 2 41 3 9 a0a1a2 = [a0a1a2] = [ ; ] 24f( ) = + :y=axbecxlny= lna+blnx+cx. LetY= lny,a0= lna,a1=b,x1= lnx,a2=c, andx2=x, then we haveY=a0+a1x1+ , , x1,i= , x2,i= 10, x21,i= , x22,i= 30, x1,ix2,i= , Yi= , x1,iYi= , x2,iYi= 4. 1 x1,i x2,i x1,i x21,i x2,ix1,i x2,i x1,ix2,i x22,i a0a1a2 = Yi x1,iYi x2,iYi 25 30 a0a1a2 = [a0a1a2] = [ ] a=ea0= ,b=a1= ,c=a2= , andy=axbecx= x 4: Linearization of nonlinear relationships27