SUGI 23: Proving it Works: Using PROC COMPARE to Verify …

1 1 Proving it Works: Using PROC COMPARE to Verify an analysis Converted into SAS Softwareby Lauren Haworth, MA, Programmer,and Njeri Karanja, , , Nutrition ScientistKaiser Permanente Center for Health Research, Portland, OregonAbstractWhen SAS Software is used to replace legacy softwaresystems, programmers are often asked to recreate analy-ses run on the old system. The challenge for the pro-grammer is to make the results from the new systemmatch the results from the old system, so analyses will becomparable over paper explores the use of PROC COMPARE toprove that a new analysis system written in SAS producesresults that exactly match those produced by a legacysoftware system.

2 The example in this paper involves theanalysis of a survey form called the Food FrequencyQuestionnaire (FFQ), which is used in nutrition researchto capture information about a respondent s typical frequency with which food is consumed is one of avariety of methods for assessing the dietary intake ofgroups and The frequency method asksrespondents to report the frequency with which they con-sume each food, from a pre-selected list of foods, over aspecified time ,3 The information is then used todevelop a dietary profile for the individual respondent orthe are many food frequency instruments developed bydifferent groups to meet a variety of research needs.

3 Onesuch instrument is the Food Frequency Questionnairethat is part of the Health Habits and History Question-naire developed by the National Cancer Institute (NCI).4 The NCI FFQ has been calibrated and validated againstother dietary assessment methods in different ,6 The FFQ is analyzed Using special NCI-designed soft-ware called DietSys software was originallydeveloped in the 1980s as an analysis tool for the the FFQ and DietSys were again updated in estimates the intake of 33 nutrients and up to 20user-defined food groups.

4 The software allows users toenter and Verify data, standardize editing, and calculate avariety of health habits such as the frequency of restau-rant eating and vitamin FFQ was used to assess baseline food and nutrientintake in the Dietary Approaches to Stop Hypertension(DASH) The study tested the impact of dietarychange on blood problemThe NCI recently discontinued its policy of updating Di-etSys and providing direct technical support to users ofthe software. This leaves research scientists with a soft-ware package that is unable to support newer versions ofthe FFQ, resulting in a limited ability to assess changesin the food supply and eating habits.

5 Additionally, Diet-Sys has always had a few inherent operating glitches thatmake analysis inefficient. For example, DietSys does notprovide much flexibility for running custom has to run the program repeatedly to test differentanalysis options, and there is no support for automationor batch processing of food records. For these reasons, wedecided to develop a more flexible and upgradeable sys-tem Using our new program to be credible, we needed to dem-onstrate that results obtained Using our new SAS pro-gram (called CHRFFQ) were comparable to those thatwould be obtained with the DietSys program.

6 To ac-complish this we conducted a test run of the DASH base-line dietary data on CHRFFQ and DietSys and usedPROC COMPARE to validate the results. This paper willdescribe the development and testing of CHRFFQ,showing how SAS was an invaluable tool in the 1: Rewriting the program in SASOur first step in building CHRFFQ was to use the Diet-Sys documentation to write SAS code that we thoughtwould match the DietSys results. Thankfully, the DietSyssoftware is in the public domain, so we did not have todeal with any copyright analysis of FFQ forms involves computing nutrientintakes by multiplying the reported frequency for a givenfood with the amount of nutrient in a specified quantityof that food.

7 Total daily nutrient intake is the sum of theproduct of the frequency and nutrient amounts in all FFQ also asks more general questions such as howmany vegetables one eats, what fats one uses in cooking,favorite brands of cereal, and what vitamins one results are used to adjust nutrient calculations. ThePostersPosters2result is an extremely complex analysis that is not DietSys documentation outlined in great detail eachof the nutrient calculations and the effect of each of theadjustments. It also explained how each of the analysisoptions was implemented.

8 This documentation provedinvaluable to the development SAS code was written to use the same calculationsand adjustments. The resulting program had over threethousand lines of code. About half of the CHRFFQ de-velopment time was spent building the basic programand getting it to run 2: Setting up the testOnce the SAS program was running without errors, westarted the long process of getting it to match the resultsproduced by DietSys. Though we tried to copy the Diet-Sys algorithm, in many cases it was unclear from thedocumentation how DietSys was calculating certain re-sults.

9 The only way to see if we had gotten it right was totest CHRFFQ against set up a test dataset to use to COMPARE the two sys-tems. Over 400 FFQ forms from the DASH study wereentered into DietSys. This produced an ASCII file thatwas accepted by DietSys, and could be read into SASusing an INPUT the data file was ready, we ran it through the Diet-Sys analysis and the CHRFFQ analysis Using the sameoption settings. For the first test run, we left all of theoptions at their simplest setting (in most cases this meantturning the option off).

10 Next we converted the DietSys results into a SAS we were ready to COMPARE the results. Our goal wasto keep testing and revising CHRFFQ until we could getto the point where any differences between the CHRFFQand DietSys results were less than 1% of each 3: Running PROC COMPAREPROC COMPARE is the perfect tool for comparing twofiles. By specifying a common variable as an identifier, itconducts an observation by observation, variable by vari-able comparison to see if each of the data points is thesame. It then produces a detailed report outlining thedifferences.