Meta-analysis of Observational Studies in …

1 consensus STATEMENTMeta- analysis of Observational Studiesin EpidemiologyA Proposal for ReportingDonna F. Stroup, PhD, MScJesse A. Berlin, ScDSally C. Morton, PhDIngram Olkin, PhDG. David Williamson, PhDDrummond Rennie, MDDavid Moher, MScBetsy J. Becker, PhDTheresa Ann Sipe, PhDStephen B. Thacker, MD, MScfor the Meta-analysis OfObservational Studies inEpidemiology (MOOSE) GroupBECAUSE OF PRESSURE FOR TIMELYand informed decisions in pub-lic health and medicine and theexplosion of information in thescientific literature, research results mustbe synthesized to answer urgent of evidence-basedmethods to assess the effectiveness ofhealth care interventions and set policyare cited , asystematic approach to identifying, ap-praising, synthesizing, and (if appropri-ate) combining the results of relevantstudies to arrive at conclusions about abody of research, has been applied withincreasing frequency to randomized con-trolled trials (RCTs)

2 , which are consid-ered to provide the strongest evidenceregarding an ,7 However, in many situations random-ized controlled designs are not fea-sible, and only data from observationalstudies are , we define anobservational study as an etiologic or ef-fectiveness study using data from an ex-isting database, a cross-sectional study,a case series, a case-control design, a de-sign with historical controls, or a co-hort designs maylack the experimental element of a ran-dom allocation to an intervention andrely on Studies of association betweenchanges or differences in 1 characteris-tic (eg, an exposure or intervention) andchanges or differences in an outcome ofAuthor Affiliations:Centers for Disease Control andPrevention, Atlanta, Ga (Drs Stroup, Williamson, andThacker); University of Pennsylvania School of Medi-cine, Philadelphia (Dr Berlin); RAND Corporation, SantaMonica (Dr Morton), University of California, San Fran-cisco (Dr Rennie), Stanford University, Stanford (DrOlkin), Calif;JAMA, Chicago, Ill (Dr Rennie); ThomasC.

3 Chalmers Centre for Systematic Reviews, Chil-dren s Hospital of Eastern Ontario Research Insti-tute, Ottawa (Mr Moher); Michigan State Univer-sity, East Lansing (Dr Becker); and Georgia StateUniversity, Atlanta (Dr Sipe).A complete listof members of the MOOSE Group ap-pears at the end of this Author and Reprints:Donna F. Stroup,PhD, MSc, Centers for Disease Control and Preven-tion, 1600 Clifton Rd NE, Mail Stop C08, Atlanta, GA30333 (e-mail: of the pressure for timely, informed decisions in public health andclinical practice and the explosion of information in the scientific literature, researchresults must be synthesized. Meta-analyses are increasingly used to address this prob-lem, and they often evaluate Observational Studies . A workshop was held in Atlanta,Ga, in April 1997, to examine the reporting of meta-analyses of Observational studiesand to make recommendations to aid authors, reviewers, editors, and participants were selected by a steering committee, basedon expertise in clinical practice, trials, statistics, epidemiology, social sciences, and biomedi-cal editing.)

4 Deliberations of the workshop were open to other interested scientists. Fund-ing for this activity was provided by the Centers for Disease Control and conducted a systematic review of the published literature on the con-duct and reporting of meta-analyses in Observational Studies using MEDLINE, Educa-tional Research Information Center (ERIC), PsycLIT, and the Current Index to also examined reference lists of the 32 Studies retrieved and contacted experts inthe field. Participants were assigned to small-group discussions on the subjects of bias,searching and abstracting, heterogeneity, study categorization, and statistical ProcessFrom the material presented at the workshop, the authorsdeveloped a checklist summarizing recommendations for reporting meta-analyses of ob-servational Studies . The checklist and supporting evidence were circulated to all confer-ence attendees and additional experts. All suggestions for revisions were proposed checklist contains specifications for reporting of meta-analyses of Observational Studies in epidemiology, including background, search strat-egy, methods, results, discussion, and conclusion.

5 Use of the checklist should improvethe usefulness of meta-analyses for authors, reviewers, editors, readers, and decisionmakers. An evaluation plan is suggested and research areas are 2000;283 ,April 19, 2000 Vol 283, No. 15 2000 American Medical Association. All rights These designs have long beenused in the evaluation of educationalprograms10and exposures that mightcause disease or of riskfactors generally cannot be random-ized because they relate to inherent hu-man characteristics or practices, and ex-posing subjects to harmful risk factorsis times, clinical data maybe summarized in order to design a ran-domized may also be needed to assess theeffectiveness of an intervention in acommunity as opposed to the specialsetting of a controlled , aclear understanding of the advantagesand limitations of statistical synthesesof Observational data is Meta-analysis restricted toRCTs is usually preferred to meta-analyses of Observational Studies ,16-18thenumber of published meta-analysesconcerning Observational Studies inhealth has increased substantially dur-ing the past 4 decades (678 in 1955-1992, 525 in 1992-1995, and more than400 in 1996 alone).

6 19 While guidelines for meta-analyseshave been proposed, many are writtenfrom the meta-analyst s (author s) ratherthan from the reviewer s, editor s, orreader s perspective20and restrict at-tention to reporting of meta-analyses ,22 Meta-analyses of observa-tional Studies present particular chal-lenges because of inherent biases anddifferences in study designs23; yet, theymay provide a tool for helping to un-derstand and quantify sources of vari-ability in results across describe here the results of aworkshop held in Atlanta, Ga, in April1997, to examine concerns regarding thereporting of Meta-analysis Of Observa-tional Studies in Epidemiology(MOOSE). This article summarizes de-liberations of 27 participants (theMOOSE group) of evidence leading torecommendations regarding the report-ing of meta-analyses. Meta-analysis of in-dividual-level data from different stud-ies, sometimes called pooled analysis or Meta-analysis of individual patientdata, 25,26has unique challenges that wewill not address here.

7 We propose achecklist of items for reporting thatbuilds on similar activities for RCTs22and is intended for use by authors, re-viewers, editors, and readers of meta-analyses of Observational conducted a systematic review ofthe published literature on the con-duct and reporting of meta-analysesin Observational Studies . Databasessearched included MEDLINE, Educa-tional Resources Information Center,PsycLIT ( ), and the Current Index to Statis-tics. In addition, we examined refer-ence lists and contacted experts in thefield. We used the 32 articles retrievedto generate the conference agenda andset topics of bias, searching and ab-stracting, heterogeneity, study catego-rization, and statistical methods. We in-vited experts in Meta-analysis from thefields of clinical practice, trials, statis-tics, epidemiology, social sciences, andbiomedical workshop included an overviewof the quality of reporting of meta-analyses in education and the socialsciences.

8 Plenary talks were given on thetopics set by the conference agenda. Foreach of 2 sessions, workshop partici-pants were assigned to 1 of 5 small dis-cussion groups, organized around thetopic areas. For each group, 1 of theauthors served as facilitator, and arecorder summarized points of discus-sion for issues to be presented to allparticipants. Time was provided for the2 recorders and 2 facilitators for eachtopic to meet and prepare plenary pre-sentations given to the entire proposed a checklist for meta-analyses of Observational Studies basedon the deliberation of the independentgroups. Finally, we circulated the check-list for comment to all conference attend-ees and representatives of several con-stituencies who would use the checklist resulting from work-group deliberations is organizedaround recommendations for report-ing background, search strategy,methods, results, discussion, and con-clusions (TABLE).BackgroundReporting of the background shouldinclude the definition of the problemunder study, statement of hypothesis,description of the study outcome(s)considered, type of exposure or inter-vention used, type of study design used,and complete description of the studypopulation.

9 When combining observa-tional Studies , heterogeneity of popu-lations (eg, US vs international stud-ies), design (eg, case-control vs cohortstudies), and outcome (eg, differentstudies yielding different relative risksthat cannot be accounted for by sam-pling variation) is of the search strategy shouldinclude qualifications of the search-ers, specification of databases used,search strategy and index terms, use ofany special features (eg, explosion ),search software used, use of handsearching and contact with authors, useof materials in languages other than En-glish, use of unpublished material, andexclusion criteria used. Published re-search shows that use of electronic da-tabases may find only half of all rel-evant Studies , and contacting authorsmay be useful,27although this resultmay not be true for all topic example, a Meta-analysis of de-pression in elderly medical inpatients29used 2 databases for the search. Inaddition, bibliographies of retrievedpapers were searched.

10 However, the au-thors did not report their search strat-egy in enough detail to allow replica-tion. An example of a thorough rejectlog can be found in the report of a Meta-analysis of electrical and magnetic fieldexposure and ofa table characterizing Studies includedcan be found in Franceschi et al31andSaag et specification ofsearch strategy is not uniform; a reviewof 103 published meta-analyses in edu-cation showed that search procedureswere described inadequately in the ma-jority of the META-ANALYSES OF Observational Studies 2000 American Medical Association. All rights ,April 19, 2000 Vol 283, No. 152009 MethodsItems in this checklist section are con-cerned with the appropriateness of anyquantitative summary of the data; de-gree to which coding of data from the ar-ticles was specified and objective; assess-ment of confounding, study quality, andheterogeneity; use of statistical meth-ods; and display of results.