The Newcastle-Ottawa Scale (NOS) for Assessing the Quality ...

1 The Newcastle-Ottawa Scale (NOS) for Assessing the Quality of nonrandomized studies in Meta- analysis G. Wells, B. Shea, D. O Connell, J. Robertson, J. Peterson, V. Welch, M. Losos, P. Tugwell Dr Daniel Pope Lecturer in EpidemiologyDr Nigel Bruce Reader in Public HealthUniversity of Liverpool, UKBackground Observational studies aetiological hypotheses (small RR but large PAF) Systematic review methodology (inc. meta- analysis ) attractive precise estimate when magnitude of risk is small Caution required (susceptibility to bias)!Bias and Confounding ..thorough consideration of sources of heterogeneity between observational study Egger et al, 2003 Newcastle-Ottowa Scale Easy, convenient tool for Quality assessment of non-randomised studies Newcastle-Ottowa ScaleCase-Control studies and Cohort StudiesStar system based on three domains:1)Selection of Study Groups2)Comparability of Groups3)Ascertainment of exposure/ outcomeDevelopment: Grouping Items Cohort studies Selection of cohorts Comparability of cohorts Assessment of outcome Case-Control studies Selection of case and controls Comparability of cases and controls Ascertainment of exposureDevelopment: Identifying Items Identify high Quality choices with a star A maximum of one star for each item within the Selection and Exposure/Outcome categories.

2 Maximum of two stars for Comparability Current Development: Validity Face/content validity Criterion validity Construct validity Inter and Intra-rater ReliabilityFuture Development: Scoring Identify threshold score distinguishing between good and poor Quality studiesN EW CAS TL E - O TTAW A Q UAL IT Y ASS ESS MENT SCA L ECAS E CON TRO L S T UD IESNote: A s tudy ca n be awa rded a ma ximum of one s tar for each numbered item within the Selection andExposure categor ies . A ma ximum of two stars c an be given for Compa lecti on1) Is the case definition adequate?a) yes , with independent valida tion b) yes , eg record linkage or bas ed on self reportsc) no description2) Representativeness of the cas esa) consecutive or obvious ly representative series of cas es b) potential for selection bias es or not s tated3) Selection of Controlsa) co mmunity controls b) hospital controlsc) no description4) Definition of Controlsa) no his tory of dis eas e (endpoint) b) no des cription of sourceCo mpa ra bilit y1) Compa rability of cas es a nd controls on the bas is of the des ign or analysisa) s tudy controls for _____ _____ (Selec t the most important fa ctor.

3 B) s tudy controls for a ny a dditional factor (This criteria could be modified to indica te s pecif ic control for a second important fa ctor.)Ex po su re1) Ascertainment of e xposurea) s ecure record (eg surgic al records ) b) s tru ctured interview where blind to cas e/control sta tus c) interview not blinded to case/control sta tusd) written self report or medica l record onlye) no description2) Sa me method of as certa in ment for cases a nd controlsa) yes b) no3) Non-Response ra tea) s a me ra te for both groups b) non respondents des cribedc) rate different and no designa tionNewcastle-Ottawa Quality Assessment Scale : Case-Control studies Selection (4) Comparability (1) Exposure (3) A study can be awarded a maximum of one star for each numbered item within the Selection and Exposure categories.

4 A maximum of two stars can be given for Comparability1. Is the case definition adequate?a) yes, with independent validation b) yes, eg record linkage or based on self reportsc) no description2. Representativeness of the casesa) consecutive or obviously representative series of cases b) potential for selection biases or not stated3. Selection of Controlsa) community controls b) hospital controlsc) no description4. Definition of Controlsa) no history of disease (endpoint) b) no description of sourceSelectionComparability1. Comparability of cases and controls on the basis of the design or analysisa) study controls for _____ (select the most important factor) b) study controls for any additional factor (This criteria could be modified to indicate specific control for a second important factor.

5 Exposure1. Ascertainment of exposurea) secure record (eg surgical records) b) structured interview where blind to case/control status c) interview not blinded to case/control statusd) written self report or medical record onlye) no description2. Same method of ascertainment for cases and controls a) yes b) no3. Non-Response Ratea) same rate for both groups b) non respondents describedc) rate different and no designationN EW CAS TL E - O TTAW A Q UAL IT Y ASS ESS MENT SCA L ECO HO R T S T UD IESNote: A s tudy ca n be awa rded a ma ximum of one star for each numbered item within the Selection andOu tcome c ategories . A ma xi mum of two s tars ca n be given for Compa ra bilityS electi on1) Representativeness of the e xposed cohorta ) truly representa tive of the average _____ _____ (describe) in the community b ) so mewha t representative of the average ___ _____ in the community c ) selected group of us ers eg nurses, volunteersd) no des cription of the derivation of the cohort2) Selection of the non expos ed cohorta ) drawn from the sa me co mmunity as the expos ed cohort b ) drawn from a different sourcec ) no description of the deriva tion of the non e xpos ed cohort3) Ascertainment of e xpos urea ) s ecure record (eg surgica l records ) b ) struc tured interview c ) written self reportd) no des cription4) Demonstra tion that outco me of interest was not present at sta rt of studya )

6 Yes b ) noCo m para bilit y1) Compa rability of cohorts on the basis of the design or ana lys isa ) s tudy controls for _____ __ (select the most important factor) b ) study controls for a ny a dditiona l factor (T his criteria could be modified to indicate s pecific control for a second i mportant factor.)O u tcome1) Assess ment of outco me a ) independent blind as sess ment b ) record linkage c ) self reportd) no des cription2) Was follow-up long enough for outcomes to occura ) yes (s elect an adequate follow up period for outcome of interest) b ) no3) Adequac y of follow up of cohortsa ) co mplete follow up - a ll subjects accounted for b ) sub jects lost to follow up unlikely to introduce bias - sma ll number lost - > ____ % (s elec t a n adequate % ) follow up, or description provided of those los t) c ) follow up ra te < ____% (selec t an adequa te % ) and no description of thos e lostd) no sta tementNewcastle-Ottawa Quality Assessment Scale .

7 Cohort studies Selection (4) Comparability (1) Outcome (3) A study can be awarded a maximum of one star for each numbered item within the Selection and outcome categories. A maximum of two stars can be given for ComparabilitySelection1. Representativeness of the exposed cohort a) truly representative of the average _____ (describe) in the community b) somewhat representative of the average _____ in the community c) selected group of users eg. nurses, volunteersd) no description of the derivation of the cohort2. Selection of the non exposed cohorta) drawn from the same community as the exposed cohort b) drawn from a different sourcec) no description of the derivation of the non exposed cohort3. Ascertainment of exposurea) secure record (eg .surgical records) b) structured interview c) written self reportd) no description4.

8 Demonstration that outcome of interest was not present at start of studya) yes b) noComparability1. Comparability of cohorts on the basis of the design or analysisa) study controls for _____ (select the most important factor) b) study controls for any additional factor (This criteria could be modified to indicate specific control for a second important factor.) Outcome1. Assessment of outcomea) independent blind assessment b) record linkage c) self reportd) no description2. Was follow up long enough for outcomes to occur a) yes (select an adequate follow up period for outcome of interest) b) no3. Adequacy of follow up of cohortsa) complete follow up - all subjects accounted for b) subjects lost to follow up unlikely to introduce bias - small number lost - > ___ % (select an adequate %) follow up, or description of those lost) c) follow up rate < ___% (select an adequate %) and no description of those lostd)

9 No statementRisk of Low Birth Weight and Stillbirth Associated With Indoor Air Pollution From Solid Fuel Use in Developing CountriesPope D, Epidemiologic Reviews, 2010 Clearly formulated question Comprehensive data search Unbiased selection and abstraction process Critical appraisal of data Synthesis of data Perform sensitivity and subgroup analyses if appropriate and possible Prepare a structured reportSteps of a Cochrane Systematic ReviewObjective Quantify the association between exposure to indoor air pollution and low birth weightInclusion Criteria Types of studies All study designs (intervention; observational) Population Live singleton births Exposure Any reporting of exposure to IAP (including solid fuel use etc) Outcomes studies reporting actual birth weight or LBW (<2500g) Clearly formulated question Comprehensive data search Unbiased selection and abstraction process Critical appraisal of data Synthesis of data Perform sensitivity and subgroup analyses if appropriate and possible Prepare a structured reportSteps of a Cochrane Systematic ReviewSearch Strategy Electronic Search of: MEDLINE EMBASE Cochrane Controlled Trials Register CINAHL LILACS Other Data Sources.

10 Grey literature (PASCAL, ICP) Contact with experts, review of references cited in retrieved articles Clearly formulated question Comprehensive data search Unbiased selection and abstraction process Critical appraisal of data Synthesis of data Perform sensitivity and subgroup analyses if appropriate and possible Prepare a structured reportSteps of a Cochrane Systematic ReviewData Extraction 2 independent reviewers selected studies 2 independent reviewers extracted data using pre-determined forms study design population characteristics Exposure (IAP) Outcomes (LBW) results differences resolved by consensusResults982 (from database search)29 (abstract review)7 (article review)4 (selected for review)6 (included in review)2 unpublished studies identified Clearly formulated question Comprehensive data search Unbiased selection and abstraction process Critical appraisal of data Synthesis of data Perform sensitivity and subgroup analyses if appropriate and possible Prepare a structured reportSteps of a Cochrane Systematic ReviewStudies included: 6 studies for data extraction (from 982) 2 cohort 2 cross-sectional 1 case-control1 intervention studyQuality assessment:Selection 4 stars:(representativeness; exposure assessment cohort/ cross-sectional; control selection case-control)Comparability 2 stars:(adjustment for main/ additional confounders eg.)