RESEARCH METHODS EXPERIMENTAL DESIGN A set of …

1 RESEARCH METHODS . &. EXPERIMENTAL DESIGN . A set of notes suitable for seminar use by Robin Beaumont Last updated: Sunday, 26 July 2009. e-mail: This handout is part of a course. For details please see Section at: Introduction to Health Informatics RESEARCH METHODS Contents 1. Introduction _____ 3 2. Common Components of RESEARCH _____ 4 3. Approaches to RESEARCH _____ 5 Classification one - Qualitative / Quantitative _____ 5 Classification two - Deductive / Inductive _____ 6 Classification three - Mixed _____ 7 4. Experiments, Quasi Experiments and Non-Experiments _____ 8 Experiments _____ 8 Pretest Post-Test Control Group DESIGN _____ 9 Post-Test only Control Group DESIGN _____ 9 Solomon Four-Group DESIGN _____ 9 Factorial DESIGN _____ 10 Some factors to Consider _____ 10 Quasi-Experiments [Field METHODS ] _____ 11 Interrupted Time-Series DESIGN _____ 11 Multiple Time-Series Designs _____ 11 Non-equivalent Control Group DESIGN _____ 12 Separate-Sample Pretest-Posttest DESIGN _____ 12 Separate Sample Pretest-Posttest Control Group DESIGN _____ 12 Non- EXPERIMENTAL Designs _____ 13 One-Shot [Cross-sectional] Study [Case Study][Survey] _____ 13 One-Shot Correlational Study _____ 13 One-Group Pretest-Posttest DESIGN _____ 14 Static-Group Comparison DESIGN _____ 14 5.

2 Reliability _____ 15 Testing Reliability _____ 15 Validity (part 1) _____ 15 6. Operationalisation _____ 16 Concepts _____ 16 Dimensions of a Concept _____ 16 Measures_____ 17 Variables & Constants _____ 17 Scales _____ 18 Graphical Method of Representing Operationalisation_____ 19 7. Validity (part 2) _____ 20 Relationship with Reliability _____ 20 Types of Validity _____ 20 Face (logical), Content and Construct Validity of Measurements _____ 20 Importance of Good Operationalisation _____ 21 Context Specific _____ 21 Summary _____ 21 8. Hypotheses _____ 22 The RESEARCH Hypothesis _____ 22 The Statistical Hypotheses _____ 23 Null Hypotheses(h0) _____ 24 Testable Hypotheses _____ 24 Robin Beaumont 15/09/2009 e-mail: Source: C:\web_sites_mine\HIcourseweb new\chap16\s1\ Page 1. Introduction to Health Informatics RESEARCH METHODS Why 'Null' _____ 25 Relationship between a Inferential Statistic and a Null Hypothesis _____ 26 Alternative Hypothesis (h ) _____ 27 1. 9. Validity (part 3) _____ 28 Internal Validity (of Findings) _____ 28 Validity of Statistics_____ 30 Construct Validity of Measurements _____ 31 External Validity [Generalisations] _____ 32 Other types of Validity _____ 32 Resume _____ 33 10.

3 EXPERIMENTAL Control _____ 35 Manipulation of Variables _____ 35 Random Assignment [Allocation]_____ 36 Control / Placebo Groups _____ 37 Protocols _____ 37 Blinding _____ 38 Controlling / Measuring Inter-subject differences: _____ 38 Homogenous Subjects _____ 38 Blocking _____ 38 Matching _____ 38 Problems with Repeated Measures Designs _____38 Controlling Problems with Repeated Measures Designs _____. 39 Analysis of Covariance (ANCOVA) _____ 39 11. Evaluating RESEARCH Reports _____ 40 Robin Beaumont 1995 _____ 40 Phillips L R 1986 _____ 41 Oxman A D 1994 _____ 41 Hawthorn P J, 1983 _____ 42 Greenhalgh 1997 _____ 44 12. Questionnaire DESIGN an Introduction _____ 45 13. References _____ 45 14. Index _____ 47 Robin Beaumont 15/09/2009 e-mail: Source: C:\web_sites_mine\HIcourseweb new\chap16\s1\ Page 2. Introduction to Health Informatics RESEARCH METHODS 1. Introduction The main aims of this handout are: To provide a source of background material which will help you to take part in, and follow the seminar discussions.

4 Act as a reference for you during the rest of your course. Almost all of the material presented in this handout is DEBATABLE. Your views and criticisms, both whilst you are reading this handout, and during the seminars should be written in alongside the relevant material. Just because it has a reference does not mean it is holy! Many topics, such as validity, reappear several times each time discussing the concept in more depth, this is deliberate as I have found that teaching many of these topics requires us to re-visit and reflect many times before a detailed understanding is gained. I hope this approach also works for you. The symbols [] are used to provide equivalent or similar names man[male]. Robin Beaumont 15/09/2009 e-mail: Source: C:\web_sites_mine\HIcourseweb new\chap16\s1\ Page 3. Introduction to Health Informatics RESEARCH METHODS 2. Common Components of RESEARCH Nearly all RESEARCH situations consist of the following: 1 Experimenter / Investigator / Researcher 2 Subject (= Client / Volunteer / Cohort / Group) / Sample 3 Data (= Primary or Secondary data).

5 4 Instrument ( thermometer, ergometer, ECG, questionnaire). In addition the following may also be present: 5 Treatment (= Intervention / Manipulated variable / Test group). 6 Controls (= Non test group / placebo ). Exercises 1 Would a one-off survey possess components 5 and / or 6? What are the advantages and disadvantages of this? 2 Which components do you think a classic laboratory experiment possesses? 3 A single blind EXPERIMENTAL DESIGN is one where the subjects do not know if they are receiving the treatment or a placebo. What is a double blind EXPERIMENTAL DESIGN ? 4 What might be the advantages and disadvantages of 'blinding'? 5 Devise a diagram to try to demonstrate the possible relationships between the above components. 6 What might be the advantages and disadvantages of supplying a placebo treatment to a control group? During this course we will be looking closely at all the above aspects except data which is covered in the statistical notes. However if you want an introduction about what you should consider when collecting data see: Robin Beaumont 15/09/2009 e-mail: Source: C:\web_sites_mine\HIcourseweb new\chap16\s1\ Page 4.

6 Introduction to Health Informatics RESEARCH METHODS 3. Approaches to RESEARCH There are any different ways of classifying RESEARCH designs, none of which are entirely satisfactory. Classification one - Qualitative / Quantitative Qualitative Methodology Characteristics (a biased view of Qualitative methodologies?). Free to devise own method within basic philosophical approach Unstructured - often deliberately no hypothesis Grounded theory ('investigation /. Case Study'). Interpretation / Subjectivity considered important (reliability / validity ). Naturalistic (outside the laboratory' conditions of quantitative RESEARCH ). Aims to provide supporting information ('issues raised') which is individually focused rather than considering the individual as a representative of a sample. Causation, association and other possibly static universals are considered to be of less importance than the individuals' experience. someone spending time with a boxing club and writing up 'what it was like'' (Ethnographic) after getting to know the several of the staff and members 1.

7 Quantitative Methodology Rigorous clearly defined rules based upon previous results Statistical tests / and data analysis have 'rules'. Considers Objectivity as being of great importance Aims to reject a (null) Hypothesis in its most basic form someone spending time in a boxing club, with a specific aim to investigate a specific topic ( the level of health in participants) which probably will have a number of academic papers already available. The study might include using well developed questionnaires specific for this area, which would allow statistical comparison with other groups. Some people disagree with the above divisions because, they say: 1. Both METHODS are structured ('Structured analysis') its just different 2. 2. Often researchers use more qualitative techniques in initial RESEARCH designs and then go on to use more quantitative METHODS latter on once they have discovered what they want to investigate. Note: The above has nothing to do with the difference between Qualitative (=Nominal[binary][count][dichotomous]) data and Quantitative (= ordinal / Interval / ratio).

8 Data. Exercises 1) Consider which approach you might use for the following RESEARCH questions. a. Investigate the feelings people have towards the local sports club, or self help groups for weight reduction. b. Investigate the difference between several training regimes in terms of improving anaerobic performance. Or investigate the difference between several training regimes for increasing fitness after a Myocardial Infarction. c. Gain some insight into bodybuilders' or doctors perceptions of themselves d. Discover if participation in games at school subsequently reduces the chance of ant- social behaviour or incidence of obesity? 2) Can you think of situations where an initial qualitative method may be abandoned for a qualitative one eventually? For more details about the qualitative / quantitative continuum along with the philosophical assumptions each makes see: 1 For a good example of such an approach see John Sugden 'The exploitation of disadvantage: the occupational sub-culture of the boxer' in sport, leisure and social relations John Horne, David Jary, Alan eds.

9 Tomlinson pub. 1993 Routledge & Kegan Paul - 209. 2 See Nixon 1991? Sports sociology that matters: Imperatives and Challenges for the 1990s Sociology of sport 281-294. and Harris J C 1989? Suited up and stripped down: perspectives for sociocultural sport studies Sociology of sport jour. 6 335-347. Robin Beaumont 15/09/2009 e-mail: Source: C:\web_sites_mine\HIcourseweb new\chap16\s1\ Page 5. Introduction to Health Informatics RESEARCH METHODS Classification two - Deductive / Inductive This classification is based upon the relationship between one or more theories and the way the RESEARCH is carried out: Deductive approach: Deduce Hypothesis from theory Theory -> Hypothesis Observation Health Exercise Exercise / health measures is good is related to health Inductive approach Induce (means to cause) theory from observations Observations -> Generalisations Theory Exercise / health measures Exercise is related to health Peoples feelings /health Being healthy is related to feeling good.

10 -. -. -. -. -. -. Health is good The Office for Population Census Statistics (OPCS) collects mountains of data and their analysis of it has unearthed numerous associations/trends etc. Exercises 1 What may be missing from both the above approaches? Think of Homeostasis. 2 The famous 20th century philosopher, Wittgentein dismissed the need for 'theories' (see Monk R 1991 Ludwig Wittgenstein Vintage press , etc.) Do you always need to refer to, or attempt to develop, a theory? Discuss. 3 List below some common beliefs concerning health and/or sport and think how you might investigate them? Robin Beaumont 15/09/2009 e-mail: Source: C:\web_sites_mine\HIcourseweb new\chap16\s1\ Page 6. Introduction to Health Informatics RESEARCH METHODS Classification three - Mixed The way most people work 3. Select Problem area . Literature Review + find out what's going on . Select topic area . Formulate a Hypothesis . What do you intend to test? What is the relationship between the variables (difference, associated / causation etc.)