Section 4 Roundoff and Truncation Error

1 MAE 4020/5020 Numerical Methods with MATLABSECTION 4: Roundoff AND Truncation ERRORSD efinitions of Error2K. Webb MAE 4020/50203 True ErrorK. Webb MAE 4020/5020 Absolute Error the difference between an approximation and the true value Relative Error the true Error as a percentage of the true value Both definitions require knowledge of the true value! If we had that, why would we be approximating?4 Approximating the ErrorK. Webb MAE 4020/5020 Since we don t know the true value, we can only approximate the Error Often, approximations are made iteratively Approximate the Error as the change in the approximate value from one iteration to the next 5 Relative Approximate ErrorK. Webb MAE 4020/5020 We don t know the true value, so we can t calculate the true Error approximate the Error Relative approximate Error an approximation of the Error relative to the approximation itself 6 Stopping CriterionK. Webb MAE 4020/5020 For iterative approximations, continue to iterate until the relative approximate Error magnitude is less than a specified stopping criterion For accuracy to at least significant figures set the stopping criterion to Roundoff Error7K.

2 Webb MAE 4020/50208 Roundoff ErrorsK. Webb MAE 4020/5020 Roundoff errors occur due to the way in which computers represent numerical values Computer representation of numerical values is limited in terms of: Magnitude there are upper and lower bounds on the magnitude of numbers that can be represented Precision not all numbers can be represented exactly Certain types of mathematical manipulations are more susceptible to Roundoff Error than others9 Number Systems Decimal K. Webb MAE 4020/5020 We are accustomed to the decimalnumber system A base 10number system Ten digits: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 Each digit represents an integer power of x 10 23 x 10 11 x 10 08 x 1012 x 1027 x 10310 Binary Number SystemK. Webb MAE 4020/5020 Computers represent numbers in binaryformat A base 2number system Two digits: 0, 1 Easy to store binary values in computer hardware an on or off switch a high or low voltage One digit is a bit eight bits is a byte Each bit represents an integer power of 2101101011 x 20= 10 x 21= 01 x 22= 40 x 23= 01 x 24= 161 x 25= 320 x 26= 01 x 27= 128= (181)1011 IEEE Double Precision FormatK.

3 Webb MAE 4020/5020 By default, MATLAB uses double precision floating point to store numeric values double 64 bit binary word52 bits11 bitsMantissa (f)Signed Exponent (e)Sign bit 12 IEEE Double Precision FormatK. Webb MAE 4020/5020 Mantissa Only the fractional portion of the mantissa stored Bit to the left of the binary point assumed to be 1 Normalized numbers Really a 53 bit value Exponent 11 bit signed integer value: 1022 .. 1023 Two special cases: e= 0x000 ( all zeros): zero if f= 0, subnormal # sif f 0 e= 0x7FF ( all ones): if f= 0, NaN if f 013 Normalized numbersK. Webb MAE 4020/5020 Leading zeros are removed Most significant digit (must be a 1 in binary) moved to the left of the binary point 53rdbit of the mantissa (always 1) needn t be stored Maximum Not stored 14 Subnormal NumbersK. Webb MAE 4020/5020 If , always, then the smallest number that could be represented is: 2 . If we allow for , then the most significant bit is somewhere to the right of the binary point Leading zeros not normalized.

4 Subnormal Allows for smaller numbers, filling in the hole around zero Subnormal numbers represented by setting the exponent to zero Smallest subnormal number:2 2 15 Doubles Range K. Webb MAE 4020/5020 Maximum value: 1 2 2 . Minimum normal value: 2 . Minimum subnormal value: 2 2 Precision machine epsilon . 16 Roundoff Error Mathematical OperationsK. Webb MAE 4020/5020 Certain types of mathematical operations are more susceptible to Roundoff errors: Subtractive cancellation subtracting of two nearly equal numbers results in a loss of significant digits Large computations even if the Roundoff Error from a single operation is small, the cumulative Error from many operations may be significant Adding large and small numbers as in an infinite series Inner products ( dot product) very common operation solution of linear systems of equationsTruncation Error17K. Webb MAE 4020/502018 Truncation ErrorsK.

5 Webb MAE 4020/5020 Errors that result from the use of an approximation in place of an exact mathematical procedure numerical integration, or the approximation derivatives with finite difference approximations To understand how Truncation errors arise, and to gain an understanding of their magnitudes, we ll make use of the Taylor Series19 Taylor SeriesK. Webb MAE 4020/5020 Taylor s Theorem any smooth ( , continuously differentiable) function can be approximated as a polynomial Taylor Series This infinite series is an equality An exactrepresentation of any smooth function as a polynomial An infinite order polynomial impractical20 Taylor Series ApproximationK. Webb MAE 4020/5020 Can approximate a function as a polynomial by truncatingthe Taylor series after a finite number of terms 2! ! where is the step sizeChapra21 Taylor Series Truncation ErrorK. Webb MAE 4020/5020 Can account for Error by lumping the and higher order terms into a single term, 2!

6 ! is the Error associated with truncating after terms 1! is some (unknown) value of between and 22 Derivative Mean Value TheoremK. Webb MAE 4020/5020 If and are continuous on , then there is a point on this interval, , where is the slope of the line joining and Chapra23 Truncation Error Dependence on Step SizeK. Webb MAE 4020/5020 1! We don t know , so we don t know We do know it s proportional to , where is the step size Error is on the order of If (first order approx.), halvingthe step size will quarterthe Error Discretizing equations Finite difference approximationsTruncation Errors in Practice24K. Webb MAE 4020/502025 Discretization of EquationsK. Webb MAE 4020/5020 As engineers, many of the mathematical expressions we are interested in are differential equations We know how to evaluate derivatives analytically Need an approximation for the derivative operation in order to solve numerically Discretization conversion of a continuous function, differentiation, to a discrete approximation for numerical evaluation26 Finite Difference ApproximationsK.

7 Webb MAE 4020/5020 Recall the definition of a derivative lim Remove the limit to approximate this numerically This is the forward difference approximation Uses value at and forward one step at to approximate the derivative at 27 Discretizing Equations Example K. Webb MAE 4020/5020 A free falling object (bungee jumper example from the text) can be modeled as where is velocity, is mass, is gravitational acceleration, and is a lumped drag coefficient This is a non linear ordinary differential equation (ODE), which can be solved analytically to yield tanh 28 Discretizing Equations Example K. Webb MAE 4020/5020 To solve numerically instead, approximate the derivative operation with a finite difference Solving for and using to denote the time step yields We ve transformed the differential equationto a difference equation An algebraicequation Can be solved iteratively using a loop29 Discretizing Equations Example K.

8 Webb MAE 4020/5020 The term in the square brackets is the original diff. eq., it is The difference equation is a first order Taylor series approximation Where we know that the Error is on the order of the step size squared Taylor series provides a relation between the step size and the accuracy of the numerical solution to the diff. Difference MethodsK. Webb MAE 4020/5020 The preceding example showed One method forward difference for numerically approximating a derivative Transformation of a differential equation to a difference equation How Taylor series can provide an understanding of the Error associated with an approximation Now we ll take a closer look at three finite difference methodsand how Taylor series can help us understand the Error associated with each31 Forward DifferenceK. Webb MAE 4020/5020 Can also derive the forward difference approximation from the Taylor Series Solving for We ve already seen that So, the Error term is The forward difference, including Error , is Error of the forward difference approximation is on the order of the step size32 Forward DifferenceK.

9 Webb MAE 4020/5020 Value of the function, at and forward one step at used to approximate the derivative at Chapra33 Backward DifferenceK. Webb MAE 4020/5020 Backward difference uses value of at and one step backward at to approximate the derivative at This can also be developed by expanding the Taylor series backward Then solving for Again the Error is on the order of the step size The backward difference expression, including Error , becomes Error of the backward difference approximation is on the order of the step size34 Backward DifferenceK. Webb MAE 4020/5020 Now use the value of at and backward one step at to approximate the derivative at Again, Error isChapra35 Central DifferenceK. Webb MAE 4020/5020 Central difference uses value of one step backward at and ones step ahead at to approximate the derivative at 2 This can also be developed by subtracting the backward Taylor series from the forward series Second order derivative terms cancel, leaving 2 Now, the remainder term is The central difference expression, including Error , becomes 2 Error of the central difference approximation is on the order of the step size squared Central difference method is more accurate than forward or backward Uses more information36 Central DifferenceK.

10 Webb MAE 4020/5020 Now use the value of backward one step at and forward one step at to approximate the derivative at Reduced Error : ChapraTotal Numerical Error37K. Webb MAE 4020/502038 Total Numerical ErrorK. Webb MAE 4020/5020 Total numerical Error is the sum of Roundoff and Truncation Error Roundoff Error is largely out of your control, and, with double precision arithmetic, it is not typically an issue Truncation Error can be a significant problem, but can be reduced by decreasing step size Reducing step size reduces Truncation Error , but may also result in subtractive cancellation, thereby increasing Roundoff Error Choose step size to minimize total Error Or, more typically, to reduce Truncation Error to an acceptable level39 Total Numerical ErrorK. Webb MAE 4020/5020 Reducing step size reduces Truncation Error , but may also result in subtractive cancellation, thereby increasing Roundoff Error Could choose step size to minimize total Error But, more typically, reduce step size just enough to reduce Truncation Error to an acceptable levelChapra40 Central Difference Error AnalysisK.