lowess — Lowess smoothing - Stata

1 Lowess smoothingDescriptionQuick startMenuSyntaxOptionsRemarks and examplesMethods and formulasAcknowledgmentReferencesAlso seeDescriptionlowesscarries out a locally weighted regression ofyvaronxvar, displays the graph, and optionallysaves the smoothed :lowessis computationally intensive and may therefore take a long time to run on aslow computer. Lowess calculations on 1,000 observations, for instance, require performing 1, startLocally weighted regression ofy1onxlowess y1 xAs above, but with a bandwidth of y1 x, bwidth(.4)With running-mean smoothinglowess y1 x, meanWithout the tricube weighting functionlowess y1 x, noweightGenerate a new variablevcontaining the smoothed values ofy1lowess y1 x, generate(v)Adjust the mean of the smoothed values to equal the mean of the unsmoothed valueslowess y1 x, adjustLowess smoothing of categorical variabley2onxin terms of the log of the odds ratiolowess y2 x, logitMenuStatistics>Nonparametric analysis> Lowess smoothing12 Lowess Lowess smoothingSyntaxlowessyvar xvar[if] [in] [,options]optionsDescriptionMainmeanrunn ing-mean smooth; default is running-line least squaresnoweightsuppress weighted regressions; default is tricube weighting functionbwidth(#)use#for the bandwidth.

2 Default isbwidth( )logittransform dependent variable to logitsadjustadjust smoothed mean to equal mean of dependent variablenographsuppress graphgenerate(newvar)createnewvarcontain ing smoothed values ofyvarPlotmarkeroptionschange look of markers (color, size, etc.)markerlabeloptionsadd marker labels; change look or positionSmoothed linelineopts(clineoptions)affect rendition of the smoothed lineAdd plotsaddplot(plot)add other plots to generated graphY axis, X axis, Titles, Legend, Overall, Bytwowayoptionsany of the options documented in [G-3]twowayoptionsyvarandxvarmay contain time-series operators; see[U] Time-series Main meanspecifies running-mean smoothing ; the default is running-line least-squares the use of Cleveland s (1979) tricube weighting function; the default is to use theweighting (#)specifies the bandwidth. Centered subsets ofbwidth() Nobservations are used forcalculating smoothed values for each point in the data except for the end points, where smaller,uncentered subsets are used.

3 The greater thebwidth(), the greater the smoothing . The default the smoothedyvarinto logits. Predicted values less than or greater are set to 1/Nand 1 1/N, respectively, before taking the mean of the smoothedyvarto equal the mean ofyvarby multiplying by anappropriate factor. This option is useful when smoothing binary (0/1) displaying the (newvar)createsnewvarcontaining the smoothed values Lowess smoothing 3 Plot markeroptionsaffect the rendition of markers drawn at the plotted points, including their shape,size, color, and outline; see [G-3] if and how the markers are to be labeled; see [G-3]markerlabeloptions. Smoothed line lineopts(clineoptions)affects the rendition of the Lowess -smoothed line; see [G-3]clineoptions. Add plots addplot(plot)provides a way to add other plots to the generated graph; see [G-3]addplotoption. Y axis, X axis, Titles, Legend, Overall, By twowayoptionsare any of the options documented in [G-3]twowayoptions.

4 These include op-tions for titling the graph (see [G-3]titleoptions), options for saving the graph to disk (see[G-3]savingoption), and theby()option (see [G-3]byoption).Remarks and default,lowessprovides locally weighted scatterplot smoothing . The basic idea is to createa new variable (newvar) that, for eachyvaryi, contains the corresponding smoothed value. Thesmoothed values are obtained by running a regression ofyvaronxvarby using only the data(xi, yi)and a few of the data near this point. Inlowess, the regression is weighted so that the central point(xi, yi)gets the highest weight and points that are farther away (based on the distance|xj xi|)receive less weight. The estimated regression line is then used to predict the smoothed value yiforyionly. The procedure is repeated to obtain the remaining smoothed values, which means that aseparate weighted regression is performed for every point in the is a desirable smoother because of its locality it tends to follow the data.

5 Polynomialsmoothing methods, for instance, are global in that what happens on the extreme left of a scatterplotcan affect the fitted values on the extreme 1 The amount of smoothing is affected bybwidth(#). You are warned to experiment with differentvalues. For instance,4 Lowess Lowess smoothing . use (Example data for Lowess ). Lowess h1 depth051015 Wet hole 10100200300400 Depthbandwidth = .8 Lowess smootherNow compare that with. Lowess h1 depth, bwidth(.4)051015 Wet hole 10100200300400 Depthbandwidth = .4 Lowess smootherIn the first case, the default bandwidth of is used, meaning that 80% of the data are usedin smoothing each point. In the second case, we explicitly specified a bandwidth of Smallerbandwidths follow the original data more 2 Twolowessoptions are especially useful with binary (0/1) the resulting curve (by multiplication) so that the mean of the smoothed values is equal tothe mean of the unsmoothed that the smoothed curve be in terms of the logof the odds ratio: Lowess Lowess smoothing 5.

6 Use (1978 automobile data). Lowess foreign mpg, ylabel(0 "Domestic" 1 "Foreign") jitter(5) adjustDomesticForeignCar origin10203040 Mileage (mpg)bandwidth = .8 Mean adjusted smoothLowess smoother. Lowess foreign mpg, logit yline(0)-4-2024 Car origin10203040 Mileage (mpg)bandwidth = .8 Logit transformed smoothLowess smootherWith binary data, if you do not use thelogitoption, it is a good idea to specifygraph sjitter()option; see [G-2]graph twoway scatter. Because the underlying data (whether the carwas manufactured outside the United States here) take on only two values, raw data points are morelikely to be on top of each other, thus making it impossible to tell how many points there sjitter()option adds some noise to the data to shift the points around. This noise affects only thelocation of points on the graph, not the Lowess you specify thelogitoption, the display of the raw data is Lowess Lowess smoothingTechnical notelowesscan be used for more than just Lowess smoothing .

7 Lowess can be usefully thought of asa combination of two smoothing concepts: the use of predicted values from regression (rather thanmeans) for imputing a smoothed value and the use of the tricube weighting function (rather than aconstant weighting function).lowessallows you to combine these concepts freely. You can use linesmoothing without weighting (specifynoweight), mean smoothing with tricube weighting (specifymean), or mean smoothing without weighting (specifymeanandnoweight).Methods and formulasLetyiandxibe the two variables, and assume that the data are ordered so thatxi xi+1fori=1, .. , N 1. For eachyi, a smoothed valueysiis subset used in calculatingysiis indicesi =max(1, i k)throughi+=min(i+k, N), wherek=b(N bwidth )/2c. The weights for each of the observations betweenj=i , .. , i+are either 1 (noweight) or the tricube (default),wj={1 (|xj xi| )3}3where = max(xi+ xi, xi xi ).

8 The smoothed valueysiis then the ( weighted ) mean orthe ( weighted ) regression prediction atxi. William Swain Cleveland (1943 ) studied mathematics and statistics at Princeton and Yale. Heworked for several years at Bell Labs in New Jersey and now teaches statistics and computerscience at Purdue. He has made key contributions in many areas of statistics, including graphicsand data visualization, time series, environmental applications, and analysis of Internet trafficdata. Acknowledgmentlowessis a modified version of a command originally written by Patrick Royston of theMRCC linical Trials Unit, London, and coauthor of the Stata Press bookFlexible Parametric SurvivalAnalysis Using Stata : Beyond the Cox , J. M., W. S. Cleveland, B. Kleiner, and P. A. Tukey. Methods for Data Analysis. Belmont,CA: , W. S. 1979. Robust locally weighted regression and smoothing of the AmericanStatistical Association74: 829 836.

9 Data. Summit, NJ: Elements of Graphing Data. Rev. ed. Summit, NJ: , N. J. 2005. Speaking Stata : smoothing in various Journal5: 574 Lowess smoothing 7 Goodall, C. 1990. A survey of smoothing techniques. InModern Methods of Data Analysis, ed. J. Fox and J. , 126 176. Newbury Park, CA: , C., and S. J. Sheather. 2010. Model fit assessment via marginal model Journal10: 215 , P., and N. J. Cox. 2005. A multivariable scatterplot Journal5: 405 see[R]lpoly Kernel- weighted local polynomial smoothing [R]smooth Robust nonlinear smoother[D]ipolate Linearly interpolate (extrapolate) values