Determining Confidence Intervals for the Z-score

1 White PaperProcess Capability MetricsDetermining Confidence Intervals for the Z-scoreSIX SIGMAQUALTECWHITE PAPERP rocess Capability Metrics1 2006 Six Sigma Qualtec. All Rights Capability and how to get them to startA White paper by Six Sigma Qualtec We re two Sigma. We re three Sigma. We re negative Sigma. Good or bad, I always see a look of satisfaction in the face of a new Black Belt the first time that they calculatethe Sigma level for their process. As the namesake for the Six Sigma initiative, new practitioners are alwaysdrawn to the Sigma level metric when studying capability analysis for the first time.

2 Prior to embarking on acapability study, students of Six Sigma should also be taught the importance of validating the integrity, relevance,accuracy and precision of the metrics and measurement system that they are using to draw conclusions and makerecommendations. So it is only fitting that the validity of the Sigma-level metric be understood as well. To accom-plish this we will determine the Confidence interval for the Sigma level as an assessment of its precision. As wewill see, a significant level of uncertainty can exist in your Sigma-level metric depending on how you carry outyour capability of statistics are all familiar with the concept of the Confidence interval for the sample mean and standarddeviation.

3 [1] Similarly, the Confidence interval concept can be applied to other statistics such as the Z-score . TheZ-score, often referred to as the Sigma Level, is a measure of process or product capability commonly linkedwith Six Sigma process improvement projects.[2] Given the importance of this statistic, it is critical for BlackBelts, process owners and management to understand the uncertainty associated with the determination of a following example illustrates the derivation of a Confidence interval for the Z-score . The derivation is basedon the Confidence Intervals for the mean and standard deviation, which are the two parameters found in the Z-scoreformula.

4 (See Equation 1.) The approach also makes use of Monte Carlo simulation, which will be described inmore detail later. (The example involves continuous time data. A similar outcome results from the analysis ofZ-scores derived from attribute data. However, that analysis is beyond the scope of this paper.)ExampleManagement at ABC Corporation is interested in Determining the capability of their order fulfillmentprocess vs. customer requirements. The key customer metric is shipment time the time that it takes anorder to be received by a customer after the order has been placed. Shipment time is of particular interestbecause a Black Belt team completed a project in this area six months ago and claimed significant improve-ment (Prior to the project the Sigma level was ).

5 Consequently, an audit team completed a capabilitystudy to see if the claimed improvements have been sustained. ABC Corporation is a low volume producerand only ships five orders per month. Therefore, delivery times for thirty shipments are available since theBlack Belt project was completed six months ago. In the interest of time a decision was made to base theaudit on this sample of 30 shipments. Based on customer feedback, delivery times of less than 12 days areacceptable. Results are shown next page in Figure 1:WHITE PAPERP rocess Capability Metrics2 2006 Six Sigma Qualtec. All Rights on this sample, average delivery time was days with a standard deviation of day.

6 Using thissample and a customer stated upper specification limit (USL) of 12 days capability analysis was completedand is shown in Figure 2:Based on this analysis Zusl = which corresponds to ppm. (For simplicity sake, let us ignoreany discussion about the Sigma shift [2]. Also note that for this example unbiasing constants were notused for the estimate of the process standard deviation and a subgroup size of 1 was used.[3])At this point most audit teams would stop here and report that the delivery process is two Sigma and theBlack Belt team would take their reward or punishment depending on how this number was that the Z-score before the Black Belt project was it does appear that an improvement wasmade.

7 But is a Z-score of significantly different than , statistically speaking?We must remember that the formula for Zusl (Equation 1) contains and s, which are only estimates of thepopulation mean and standard deviation. We must think about and s relative to their Confidence Intervals ,which are calculated using Equations 2 and 3.[1]Figure 1 Delivery TimeCapability StudyHistogramFigure 2 Process CapabilityAnalysis forDelivery TimeWHITE PAPERP rocess Capability Metrics3 2006 Six Sigma Qualtec. All Rights if the true population mean is closer to the USL than ? The Z-score will be lower reflecting poorercapability than reported.

8 On the other hand, if the population standard deviation is close to the sample slower Confidence bound, the Z-score will be higher than understand just how much higher or lower the true process capability might be ( , how much uncer-tainty exists in the calculation of the Z-score ) let us determine a Confidence interval for the Z-score basedon the Confidence Intervals for and s using a Monte Carlo simulation. The Monte Carlo approach is anumerical method that utilizes a sequence of random numbers to perform the simulation. [4] Random num-bers are selected in a series of trials from probability distribution functions (PDFs) that are assumed to rep-resent the underlying behavior of the system.

9 For example, many manufacturing process variables can bedescribed using a normal (or Gaussian) to our example, we begin by redefining ZUSL by combining equations 1-3 to give Equation equation 4, tn is a random observation selected from a t-distribution with n degrees of freedom. ?2 is arandom observation selected from a Chi-square distribution with n degrees of freedom. For the MonteCarlo simulation values of t and ?2 are randomly chosen and Z-scores are calculated for each of these distribution of possible Z-scores results. The resulting Z-scores can be transformed into defect rates inparts per million (ppm).

10 In practice we begin by creating a random sample of 1000 observations from a t-distribution with 29 degrees of freedom (n=30 for this example). The resulting histogram is shown inFigure 1 Equation 3 Equation 4 Equation 2 WHITE PAPERP rocess Capability Metrics4 2006 Six Sigma Qualtec. All Rights observation (an individual t-value) in this distribution is then inserted into the Equation 2 - the confi-dence interval for the mean. Each result is a member of the distribution of possible population means basedon , the standard deviation of the sample and the number of observations (n=30) in the capability study.