Towards a Methodology for Developing Evidence-Informed ...

1 British Journal of management , Vol. 14. 207-222 (2003) Towards a Methodology for DevelopingEvidence- informed ManagementKnowledge by Means of SystematicReview*David Tranfield, David Denyer and Palminder SmartAdvanced management Research Centre (AMRC), Cranfield School of management ,Cranfield University, Cranfield, MK43 OAl, UKCorresponding author email: D. a review of the literature is an important part of any research project. Theresearcher both maps and assesses the relevant intellectual territory in order to specify aresearch question which will further develop the knowledge hase. However, traditional'narrative' reviews frequently lack thoroughness, and in many cases are not undertakenas genuine pieces of investigatory science.

2 Consequently they can lack a means formaking sense of what the collection of studies is saying. These reviews can he hiased bythe researcher and often lack rigour. Furthermore, the use of reviews of the availableevidence to provide insights and guidance for intervention into operational needs ofpractitioners and policymakers has largely been of secondary importance. Forpractitioners, making sense of a mass of often-contradictory evidence has hecomeprogressively harder. The quality of evidence underpinning decision-making and actionhas heen questioned, for inadequate or incomplete evidence seriously impedes policyformulation and implementation. In exploring ways in which evidence -informedmanagement reviews might be achieved, the authors evaluate the process of systematicreview used in the medical sciences.

3 Over the last fifteen years, medical science hasattempted to improve the review process hy synthesizing research in a systematic ,transparent, and reproducihie manner with the twin aims of enhancing the knowledge haseand informing policymaking and practice. This paper evaluates the extent to which theprocess of systematic review can be applied to the management field in order to produce areliable knowledge stock and enhanced practice by Developing context-sensitive paper highlights the challenges in Developing an appropriate : the need for an Evidence-Informed approachUndertaking a review of the literature to providethe best evidence for informing policy and*This paper results from research undertaken in Cran-field IMRC (EPSRC) grant no IMRC19, ' Developing amethodology for Evidence-Informed managementknowledge using systematic review ', Professor DavidTranfield and Dr David Denyer.

4 2003 British Academy of Managementpractice in any discipline, is a key researchobjective for the respective academic and practi-tioner post-World-War-II era witnessed a sharpfocus of attention by academics and practitionerson the discipline and profession of management (Blake and Mouton, 1976; Tisdall, 1982). Thepace of knowledge production in this field hasbeen accelerating ever since and has resulted in abody of knowledge that is increasingly fragmen-ted and transdisciplinary as well as being inter-dependent from advancements in the social208D. Tranfield, D. Denyer and P. Smartsciences (Friedman, Durkin, Phillips and Volt-singer, 2000).In management research, the literature reviewprocess is a key tool, used to manage the diversityof knowledge for a specific academic inquiry.

5 Theaim of conducting a literature review is often toenable the researcher both to map and to assessthe existing intellectual territory, and to specify aresearch question to develop the existing body ofknowledge further. management reviews areusually narrative and have been widely criticizedfor being singular descriptive accounts of thecontributions made by writers in the field, oftenselected for inclusion on the implicit biases of theresearcher (Fink, 1998; Hart, 1998). Not surpris-ingly they have also been condemned for lackingcritical assessment. The management -researchcommunity perpetuates this type of practice bynot actively commissioning infrastructural ar-rangements to ensure previous investments inliterature reviews are not lost.

6 This tolerance toloss of knowledge forms a high-risk strategy thatwill inevitably become unsustainable as organiza-tions endeavour further into the networked andknowledge-based of the available evidence in manage-ment to assimilate 'best evidence ' to provideinsights and guidance for intervention into theoperational needs of practitioners and policy-makers have largely become a secondary momentum from academics, practi-tioners, and government has stirred an urgentneed to re-evaluate the process by which manage-ment researchers conduct literature reviews. Overthe last fifteen years, medical science has at-tempted to improve the quality of the reviewprocess. This paper proposes the view thatapplying specific principles of the systematicreview Methodology used in the medical sciencesto management research will help in counteractingbias by making explicit the values and assump-tions underpinning a review .

7 By enhancing thelegitimacy and authority of the resultant evidence , systematic reviews could provide practitioners andpolicy-makers with a reliable basis to formulatedecisions and take action. This is particularlysobering if one considers the growing pressuresupon practitioners in today's global tradingenvironments to do this in shorter cycle paper will begin by discussing theevidence-based approach in medical sciencesthrough the effective use of systematic following sections will compare and contrastthe nature of reviews in medical science andmanagement research and evaluate the extent towhich the systematic review process can beapplied to the management field. Finally thispaper will present the challenges in designing anappropriate Methodology for management origins of the evidence -basedapproachSince the 1980s, the British central governmenthas placed increasing emphasis on ensuring thatpolicy and practice are informed through a morerigorous and challenging evidence base.

8 The'three E' initiatives (economy, efficiency andeffectiveness) have focused attention on thedelivery of public services and have led to thedevelopment of detailed guidance and best-practice manuals in many disciplines. Effective-ness in this context is concerned both withappropriateness and the validity of the methodsused by professionals in their day-to-day work toachieve their basic aims and also with the overallability of agencies to deliver the services they arerequired to provide (Davies, Nutley and Smith,2000). The concern for effective service deliveryhas attracted considerable attention, and hasfocused interest on basing policy and practice onthe best evidence available.

9 Consequently, anevidence-based movement has developed underNew Labour, and in May 1997 Tony Blairannounced that 'what counts is what works', theintention being to signal a new 'post-ideological'approach to public policy where evidence wouldtake centre stage in the decision-making process(Davies, Nutley and Smith, 2000).The evidence -based approach in medicalscience and healthcareThe evidence -based movement has had a majorimpact in certain disciplines. Pre-eminent havebeen applications in medical science, where thepace of knowledge production has meant thatmaking sense of an often-contradictory mass ofevidence has become increasingly difficult (Ohls-son, 1994). Specifically in the late 1980s, attentionDeveloping Evidence-Informed management Knowledge209was drawn to the comparative lack of rigour insecondary research (Mulrow, 1987).

10 Critics ar-gued that the preparation of reviews of secondarysources were dependent on implicit, idiosyncraticmethods of data collection and interpretation(Cook, Mulrow and Haynes, 1997; Greenhalgh,1997). In addition, practice based on poor-qualityevaluations of the literature sometimes had led toinappropriate recommendations (Cook, Green-gold, Ellrodt and Weingarten, 1997). In 1991,Smith questioned the overall wisdom of much ofmedical science, arguing that only 15-20% ofmedical interventions were supported by solidmedical evidence (Smith, 1991). The result, it wasargued, was that patients were being regularlysubjected to ineffective treatments and interven-tions, and for many practices there was little orno understanding of whether or not the benefitsoutweighed the potential harm (Davies, Nutleyand Smith, 1999).