Section 1.3, Data Collection and Experimental Design

1 Section , Data Collection and Experimental DesignAvariableis any characteristic that is recorded for subjects in a study. For example, gender, major, age, and GPA might be variables for a study about college Designing a studySteps for designing a study:1. Identify the variable (or variables) of interest and the population of the Develop a detailed plan for collecting the data. When using a sample, you need to make surethat the sample is representative of the Collect the Describe the data using descriptive Interpret the data and use inferential statistics to make decisions (or assumptions) about Identify any possible errors (some potential problems can be identified earlier in the process).

2 2 Collecting DataWe will focus on four methods for collecting data: Observational studies, experiments, simulations,and surveys. In anobservational study, a researcher measures and observes the variables of interestwithout changing existing conditions. In anexperiment, a researcher assigns atreatmentand observes theresponse. Sometimes,acontrol group(a group receiving no treatment or aplacebo) may be used to compare theeffectiveness of a treatment. Asimulationuses a mathematical, physical, or computer model to replicate the conditionsof a process or situation. This is frequently used when the actual situation is too expensive,dangerous, or impractical to replicate in real life.

3 Asurveyis used to investigate characteristics of a population. It is frequently used whenthe subjects are people, and questions are asked of them. When designing a survey, you mustbe very careful of wording (and sometimes ordering) the questions so that the results are which method for collecting data (observational study, an experiment, a simulation, or asurvey) is best in each of the following situations and explain your The effect a severe earthquake would have on the Salt Lake Valley. Simulation, since it is toodangerous to create an actual earthquake (Plus, we cannot control when nature will have anearthquake).2. Whether or not a certain coupon attached to the outside of a catalog makes recipients morelikely to order products from a mail-order company.

4 Experiment, since we are comparing twoscenarios and we can control Whether or not smoking has an effect on coronary heart disease. Observational study, sincewe will not be changing a person s behavior (there are ethical and health concerns for decidingwhether someone smokes or not)4. Determining the average household income of homes in Salt Lake City. Survey, since it can beanswered with a brief Experimental DesignDefinitions and Terminology Aconfounding variableoccurs when an experimenter cannot tell the difference between theeffects of different factors on a variable. Theplacebo effectoccurs when a subject (or Experimental unit ) reacts favorably to aplacebo when no medicated treatment has been given.

5 Blindingis a technique used to make the subjects blind to which treatment (or placebo)they are being given. Adouble-blindexperiment is one in which neither the experimenter nor the subjects knowwhich treatment is being given. Randomizationis a process of randomly assigning subjects to treatment groups. There areseveral different techniques for randomization: Acompletely randomized designassigns subjects to different treatment groups throughrandom assignment. Arandomized block designis sometimes used to make sure that subjects with cer-tain characteristics are assigned to each treatment. For example, when testing a certainmedication, you might first want to split subjects in groups according to either gender orage (or both), then randomly assign each of these groups to the different treatments.

6 Amatched pairs designpairs up subjects according to similarities. One subject in the pairreceives one treatment, while the other receives a different treatment. Sample sizeis the number of participants in the experiment. The larger the sample, themore representative of the population the results will be, but the costs of the experiment willalso be higher. Replicationis the ability to reproduce the experiment (and results) under similar For the following experiment, determine the Experimental units, treatments, and sample whether this experiment is blind, double-blind, and/or randomized. Also identify anypotential problems with the study with 233 low-income adult smokers evaluated the effectiveness of usual care (physicianadvice and follow-up) for smokers wishing to quit to the usual care enhanced by computer-assisted telephone counseling sessions.

7 Each subject was assigned randomly either to the usualcare or to the usual care plus counseling, and their smoking status (still smoking or quitsmoking) was observed after 3 months. The percentage who had quit smoking was higher forthe group receiving counseling. (fromJournal of Family Practice2000;50:138-144)The Experimental units are the 233 adult treatments are usual care and usual care plus counseling. The sample size is experiment is not blind or double-blind ( Experimental units will know if they receivecounseling or not), but it is randomized since the subjects were assigned How would you Design a placebo-controlled double-blind experiment with a randomized blockdesign for the following situation: A veterinarian wants to test a strain of antibiotic on calvesto determine their resistance to a common infection, and if their gender plays a role.

8 In apasture, there are 22 newborn calves (11 males and 11 females). There is enough vaccine for10 calves, but blood tests to determine their resistance to infection can be done on all the randomized block Design , we can choose 5 males and 5 females randomly to receivethe antibiotic, and the remaining 6 males and 6 females will receive a placebo (in this case,the placebo could be no treatment since the calves won t be telling anyone if they receiveda shot or not). In order to make the experiment double-blind, we need to make the calvesunaware of what treatment they are receiving (not difficult), and for the person carrying outthe blood tests to be unaware of the treatment (to make this possible, another person will beassigning which of the calves receive the antibiotic and which do not).

9 4 Sampling TechniquesIdeally, we would take acensus, that is, use every member of a population as a subject since thedescriptive statistics would be sufficient. However, this is often too costly and difficult. Instead,wesamplepart of the population. With sampling, we need to make sure that the sample isrepresentative of the population and large enough to be and Terminology Asampling erroris the difference between the results of the sample and those of the popu-lation. Even with the best sampling techniques, this is possible. Abiased sampleis one that is not representative of the entire population. We want to avoidbias. Arandom sampleis one in which every member of the population has an equal chance ofbeing chosen.

10 Asimple random sample(SRS) is a sample in which every possible sample of the samesize has the same chance of being collected. Normally, we will start by using a simple randomsample. Astratified sampleis used when it is important to have members from multiple segmentsof the population. First, the population is split into segments (called strata ), then a prede-termined number of subjects is chosen from each of the strata. Cluster samplingcan be used when the population naturally falls into subgroups with similarcharacteristics. First, determine the clusters, then select all the members of one or more ofthe clusters. Systematic samplingfirst involves assigning a number to each member of the populationand ordering them in some way.