Improving the Reporting Quality of Nonrandomized ...

1 COMMENTARY . Improving the Reporting Quality of Nonrandomized Evaluations of Behavioral and Public Health Interventions: The TREND Statement Developing an evidence base | Don C. Des Jarlais, PhD, Cynthia Lyles, PhD, Nicole Crepaz, PhD, and the TREND Group for making public health deci . sions will require using data from evaluation studies with OVER THE PAST SEVERAL Also in this issue, Donner and Furthermore, evidence-based randomized and nonrandom decades, a strong movement to Klar,7 Murray et al.,8 and Var practice may often rely on meta- ized designs. Assessing indi . ward evidence-based medicine nell et provide overviews of analyses of large numbers of vidual studies and using stud . has 3 In the context of the benefits and pitfalls of the studies, some of which may re . ies in quantitative research syntheses require transparent evidence-based medicine, clinical group-randomized trial, which, port negative results.

2 Meta- Reporting of the study, with suf decisions are based on the best in some situations, may be a analysis requires full Reporting ficient detail and clarity to read available scientific data rather reasonable alternative to the of methods and outcomes to en . ily see differences and similar than on customary practices or RCT. There are also a wide vari able assessment of comparability ities among studies in the the personal beliefs of the health ety of Nonrandomized evalua of different studies. Inadequate, same area. The Consolidated care provider. There is now a tion designs that can contribute or nontransparent, Reporting Standards of Reporting Trials parallel movement toward evi important data on the efficacy may make it difficult to under . (CONSORT) statement provides dence-based public health prac or effectiveness of interventions, stand the variables that affect in . guidelines for transparent re ,5 The movement is in such as quasi-experimental de tervention outcomes and the porting of randomized clinical tended to utilize the best signs,10 Nonrandomized trials, central elements in intervention trials.

3 Available scientific knowledge as and natural experiments. Includ success or failure over multiple We present the initial ver . sion of the Transparent Report the foundation for public ing these types of designs in de studies. ing of Evaluations with Non- health related decisionmaking. veloping evidence-based recom In recent years, efforts have randomized Designs (TREND) In the context of evidence- mendations can provide a more been made to improve the qual . statement. These guidelines based medicine, the randomized integrated picture of the existing ity of Reporting of RCTs. The emphasize the Reporting of the controlled trial (RCT) is usually evidence and could help to Consolidated Standards of Re . ories used and descriptions of considered of greatest eviden strengthen public health prac porting Trials (CONSORT) state . intervention and comparison tiary value for assessing the effi tice.

4 Excluding data collected ment11 provides a 22-item check . conditions, research design, cacy of interventions. Indeed, under such designs would un list and subject flow chart for the and methods of adjusting for the preference for this design is doubtedly bias the evidence transparent Reporting of RCTs. possible biases in evaluation sufficiently strong that when em base toward interventions that This statement has been adopted studies that use nonrandom . pirical evidence from RCTs is are easier to evaluate but not as a framework for the Reporting ized designs. (Am J Public Health. 2004;94:361 366) available, weaker designs are necessarily more effective or of RCTs by a large number of often considered to be of little or cost-effective. medical, clinical, and psychologi . no evidentiary value. In this If Nonrandomized designs are cal journals (153, according to issue, Victora et make a to be systematically used in strong argument that evidence- building evidence-based public org, as of September 16, 2003).

5 Based public health will neces health practices, it will be neces Use of the CONSORT statement sarily involve the use of research sary to improve the Reporting has improved the Quality of RCT. designs other than RCTs. Most Quality of these types of studies. reports over the past several important, they argue that RCTs The transparency, or clarity, in There is yet, however, no are often not practical or not the Reporting of individual stud agreed-upon framework for the ethical for evaluating many pub ies is key. Sufficient detail and transparent Reporting of nonran . lic health interventions and dis clarity in the report allow readers domized research evaluations. cuss methods for drawing causal to understand the conduct and The HIV/AIDS Prevention inferences from Nonrandomized findings of the intervention study Research Synthesis (PRS) team evaluation designs ( plausibility and how the study was different of the Centers for Disease Con.)

6 And adequacy designs in their from or similar to other studies trol and Prevention (CDC) has terminology). in the field. been synthesizing evidence from March 2004, Vol 94, No. 3 | American Journal of Public Health Des Jarlais et al. | Peer Reviewed | Commentary | 361. COMMENTARY . HIV behavioral intervention adoption of a final set of report sion of information related to the It is important to note that the studies involving RCT and non- ing standards. design needed to assess possible TREND checklist is not intended randomized designs. The PRS Table 1 presents a proposed biases in the outcome data. Brief to serve as a criterion for evalu . team found that many study re checklist the Transparent Re comments may be helpful for a ating papers for publication. ports failed to include critical in porting of Evaluations with Non- few of the items included in the Rather, it is intended to improve formation ( , intervention tim randomized Designs (TREND) proposed TREND checklist.

7 The Quality of data Reporting in ing and dosage, effect size data) checklist for Reporting standards peer-reviewed publications so necessary for research synthe of behavioral and public health Use of theory (item 2). Behav that the conduct and the findings 17 To improve their ability intervention evaluations involv ioral and social science theories of research are transparent. As to synthesize HIV behavioral ing Nonrandomized designs. The provide a framework for gener the volume of public health liter . prevention research, the PRS TREND checklist is meant to be ating cumulative knowledge. ature is consistently expanding, team convened the CDC's Jour consistent with the CONSORT Thus, it would be very helpful to research synthesis becomes an nal Editors Meeting in Atlanta, checklist for the Reporting of include references to the theoret important tool for creating a cu . Ga, on July 24 25, 2003; this RCTs.

8 Items presented in bold ical bases of the intervention mulative body of knowledge and meeting was attended by editors face type in the table are particu being evaluated. This would per making evidence-based recom . and representatives of 18 jour larly relevant to behavioral and mit identification of theories that mendations of effective interven . nals that publish HIV behavioral public health intervention stud are useful in developing inter tions. Reporting standards will intervention studies (a complete ies, whether or not randomized ventions in different fields. Some help ensure that fewer interven . list of the journals is available designs are used. Thus, we would interventions, however, are tion trials with Nonrandomized from the authors and at http:// suggest that they be used to ex based on atheoretical needs as designs are missing information ). pand the information requested sessments or simply the experi critical for research synthesis and The main goals of the meeting by CONSORT for RCTs of be ence of the individuals who de that comparable information were to (1) communicate the havioral and public health inter signed the intervention.

9 In these across studies can be more easily usefulness and importance of ad ventions. Some of the items situations, a post hoc application consolidated and translated into equate Reporting standards, (8, 10, and 15) presented in the of a theory is not likely to be generalizable knowledge and (2) reach consensus on Reporting proposed TREND checklist are helpful. practice. standards for behavioral inter not relevant to RCTs and, thus, Description of the intervention We recognize several chal . ventions, (3) develop a checklist not included in the CONSORT condition and the services pro lenges in promoting and dissemi . of Reporting standards to guide checklist, but they are extremely vided in a comparison condition nating Reporting standards for authors and journal reviewers, relevant to Nonrandomized de (item 4). Although space is lim Nonrandomized intervention and (4) develop strategies to dis signs.

10 We also refer readers to ited in many journals, it is still evaluations. Most important, the seminate the resulting Reporting CONSORT elaboration re critical to provide sufficient de TREND checklist is only a sug . standards. ports11,18 that provide rationales tail so that a reader has an un gested set of guidelines and The discussions at the meet and examples for items in derstanding of the content and should be considered a work in ing broadened to include stan Table 1 that are shared with the delivery of both the experimen progress. It is highly likely that dardized Reporting of behavioral CONSORT checklist. tal intervention and the services improvements will be necessary;. and public health interventions The TREND checklist is pro in the comparison condition. For moreover, adaptations may be in general, rather than focusing posed for intervention evaluation example, usual care is not a needed to refine the standards only on HIV behavioral inter studies using Nonrandomized de helpful description of a compari for specific fields of intervention ventions.