Developmental research methods: Creating knowledge from ...

1 Journal of Computing in Higher Education Spring 2005 Vol. 16(2), 23-38. Developmental research methods : Creating knowledge from Instructional Design and Development Practice Rita C. Richey Wayne State University James D. Klein Arizona State University ABSTRACT T HE PURPOSE OF THIS ARTICLE is to provide direction to those entertaining a Developmental research project. There are two categories of Developmental research , both of which are examined here. The two types vary in terms of the extent to which the conclusions resulting from the research are generalizable or con- textually specific. This article describes Developmental research in terms of the traditional stages of planning, conducting, and reporting a research project--problem definition, literature reviews, and research procedures.

2 Examples of adapting a literature review to a develop- mental study are identified. Problem definition procedures encompass focusing on and framing the problem, and identifying the limitations of the research . research procedures are adapted to Developmental studies, particularly with respect to identifying the study's participants, Creating a research design, and collecting and analyzing the data. Finally, issues with respect to reporting Developmental research are discussed. (Keywords: Developmental research , evaluation research instructional design, research methods , theory to practice) 23 Developmental research methods INTRODUCTION F OR MANY, the design, development and evaluation of instruc- tional products and programs are considered to be the heart of instructional technology.

3 It is also critical to those engaged in a broad range of computer applications. Consequently, it follows that a substantial part of our research should address design processes and tools. In other words, we should be studying design, development, and evaluation as well as doing it. Of course , this is not a new suggestion; some scholars have heeded it, even though the innovators often put more effort into doing rather than studying. The body of research literature that directly pertains to instruc- tional development is known as Developmental research : "the system- atic study of designing, developing and evaluating instructional pro- grams, processes and products that must meet the criteria of internal consistency and effectiveness" (Seels & Richey, 1994, p.)

4 127). De- velopmental research seeks to create knowledge grounded in data systematically derived from practice. It is a pragmatic type of research that offers a way to test "theory" that has been only hypothesized and to validate practice that has been perpetuated essentially through unchallenged tradition. In addition, it is a way to establish new pro- cedures, techniques, and tools based upon a methodical analysis of specific cases. As such, Developmental research can have a function of either Creating generalizable conclusions or statements of law, or producing context-specific knowledge that serves a problem solving function. There are two categories of Developmental research , referred to as Type 1 and Type 2 (Richey, Klein, & Nelson, 2004).

5 They vary in terms of the extent to which the conclusions resulting from the research are generalizable or contextually specific. Type 1 develop- mental studies focus upon a given instructional product, program, process, or tool. They reflect an interest in identifying either general development principles or situation-specific recommendations. Typi- cally Type 1 studies address not only product design and develop- 24 Richey and Klein ment, but evaluation as well. At times they may validate a particular design or development technique or tool. Type 2 studies, on the other hand, focus upon a given design, development, or evaluation model or process. They may involve constructing and validating unique design models and processes, as well as identifying those conditions that facilitate their successful use.

6 Developmental research is different from the design-based research that has been recently discussed. This research emphasizes the study of learning as a result of designing unique instructional interventions (The Design-Based research Collective, 2003). It is also different from traditional research in instructional psychology, media comparison, and message design. However, well-conceived Developmental research can offer a direct response to the many demands for research that is prescriptive and meets the pressing needs of practitioners (Richey, 1997). The aim of this article is to provide some direction to those entertaining a Developmental research project. In essence, it is a discussion of establishing the credibility of a given Developmental study by assuring authenticity and methodological precision.

7 Since many Developmental research topics can be addressed in a number of dif- ferent ways, this paper may also help the reader recognize the poten- tial of a given problem to be addressed as a Developmental topic. Below, we describe Developmental research in terms of the traditional stages of planning, conducting, and reporting a research project: 9 Problem definition; 9 Literature reviews; and 9 research procedures. DEFINING THE research PROBLEM The authenticity of a Developmental research project often de- pends upon the problem selected. Is the problem one that is common to many designers and developers? Is it one that is currently critical to the profession? Does the problem reflect realistic constraints and conditions typically faced by designers?

8 Does the problem pertain to 25 Developmental research methods cutting-edge technologies and processes? Answers to these questions not only predict interest in the project, but also whether the research is viewed as relevant. "Explorations of research relevance are typi- cally examinations of shared perceptions, the extent to which research - ers' notions of relevance are congruent with the perceptions and needs of practitioners" (Richey, 1998, p. 8). This is particularly true with Developmental research where the object of such research is clearly not simply knowledge , but knowledge that practitioners can use. Focusing the Problem. Given a relevant research topic, the project must first be given a " Developmental twist." This begins in the prob- lem definition stage.

9 It is done by focusing the research problem on a particular aspect of the design, development, or evaluation process, as opposed to focusing on a particular variable that impacts learning. The problem definition stage must also establish research param- eters. At minimum, this involves determining whether the research will be conducted as the design and development is occurring or whether retrospective data will be collected on a previously devel- oped program or set of materials. Then the scope of the study must be established. How much of the design and development process will be addressed? Will the research address: 9 All parts of the design of the instruction? 9 The development (or part of the development) of the instruc- tion? 9 The evaluation of the instruction?

10 If so, will formative, and summative, and confirmative evaluation be addressed? 9 The revision and retesting of the instruction? Developmental studies often are structured in phases. For example, comprehensive Type 1 studies may have an analysis phase, design phase, a development phase, and a try-out and evaluation phase. Another organization of a Type 1 study would include phases directed toward first analysis, then prototype development and testing, and finally prototype revision and retesting. A study by McKenney (2002) is an example of this type of research . McKenney examined the development of a computer program to support curriculum materials 26 Richey and Klein development in the context of secondary science and mathematics education in southern Africa.