The Resource Developing and testing a tool for the classification of study designs in systematic reviews of interventions and exposures, prepared for Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services ; prepared by University of Alberta Evidence-based Practice Center ; investigators, Lisa Hartling ... [et al.], (electronic resource)

Developing and testing a tool for the classification of study designs in systematic reviews of interventions and exposures, prepared for Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services ; prepared by University of Alberta Evidence-based Practice Center ; investigators, Lisa Hartling ... [et al.], (electronic resource)

Label
Developing and testing a tool for the classification of study designs in systematic reviews of interventions and exposures
Title
Developing and testing a tool for the classification of study designs in systematic reviews of interventions and exposures
Statement of responsibility
prepared for Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services ; prepared by University of Alberta Evidence-based Practice Center ; investigators, Lisa Hartling ... [et al.]
Contributor
Subject
Language
  • eng
  • eng
Summary
BACKGROUND: Classification of study design can help provide a common language for researchers. Within a systematic review, definition of specific study designs can help guide inclusion, assess the risk of bias, pool studies, interpret results, and grade the body of evidence. However, recent research demonstrated poor reliability for an existing classification scheme. OBJECTIVES: To review tools used to classify study designs; to select a tool for evaluation; to develop instructions for application of the tool to intervention/exposure studies; and to test the tool for accuracy and interrater reliability. METHODS: We contacted representatives from all AHRQ Evidence-based Practice Centers (EPCs), other relevant organizations, and experts in the field to identify tools used to classify study designs. Twenty-three tools were identified; 10 were relevant to our objectives. The Steering Committee ranked the 10 tools using predefined criteria. The highest-ranked tool was a design algorithm for studies of health care interventions developed, but no longer advocated, by the Cochrane Non-Randomised Studies Methods Group. This tool was used as the basis for our classification tool and was revised to encompass more study designs and to incorporate elements of other tools. A sample of 30 studies was used to test the tool. Three members of the Steering Committee developed a reference standard (i.e., the "true" classification for each study); 6 testers applied the revised tool to the studies. Interrater reliability was measured using Fleiss' kappa (o) and accuracy of the testers' classification was assessed against the reference standard. Based on feedback from the testers and the reference standard committee, the tool was further revised and tested by another 6 testers using 15 studies randomly selected from the original sample. RESULTS: In the first round of testing the inter-rater reliability was fair among the testers (o = 0.26) and the reference standard committee (o = 0.33). Disagreements occurred at all decision points in the algorithm; revisions were made based on the feedback. The second round of testing showed improved interrater reliability (o = 0.45, moderate agreement) with improved, but still low, accuracy. The most common disagreements were whether the study was "experimental" (5/15 studies) and whether there was a comparison (4/15 studies). In both rounds of testing, the level of agreement for testers who had completed graduate-level training was higher than for testers who had not completed training. CONCLUSION: Potential reasons for the observed low reliability and accuracy include the lack of clarity and comprehensiveness of the tool, inadequate reporting of the studies, and variability in user characteristics. Application of a tool to classify study designs in the context of a systematic review should be accompanied by adequate training, pilot testing, and documented decision rules
Member of
Cataloging source
DNLM
Government publication
federal national government publication
Illustrations
illustrations
Index
no index present
Literary form
non fiction
Nature of contents
  • dictionaries
  • bibliography
NLM call number
WA 950
http://library.link/vocab/relatedWorkOrContributorName
  • Hartling, Lisa
  • United States
  • University of Alberta Evidence-based Practice Center
Series statement
  • Methods research report
  • AHRQ publication
Series volume
no. 11-EHC007-EF
http://library.link/vocab/subjectName
  • Meta-Analysis as Topic
  • Research Design
  • Classification
  • Review Literature as Topic
  • Reference Standards
Label
Developing and testing a tool for the classification of study designs in systematic reviews of interventions and exposures, prepared for Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services ; prepared by University of Alberta Evidence-based Practice Center ; investigators, Lisa Hartling ... [et al.], (electronic resource)
Instantiates
Publication
Note
  • Title from PDF title page
  • "Contract no. 290-02-0023."
  • "December 2010."
Bibliography note
Includes bibliographical references
Dimensions
unknown
Form of item
online
Specific material designation
remote
System control number
  • 1557917
  • (DNLM)BKSHLF:NBK52670
System details
Mode of access: Internet
Label
Developing and testing a tool for the classification of study designs in systematic reviews of interventions and exposures, prepared for Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services ; prepared by University of Alberta Evidence-based Practice Center ; investigators, Lisa Hartling ... [et al.], (electronic resource)
Publication
Note
  • Title from PDF title page
  • "Contract no. 290-02-0023."
  • "December 2010."
Bibliography note
Includes bibliographical references
Dimensions
unknown
Form of item
online
Specific material designation
remote
System control number
  • 1557917
  • (DNLM)BKSHLF:NBK52670
System details
Mode of access: Internet

Library Locations

  • African Studies LibraryBorrow it
    771 Commonwealth Avenue, 6th Floor, Boston, MA, 02215, US
    42.350723 -71.108227
  • Alumni Medical LibraryBorrow it
    72 East Concord Street, Boston, MA, 02118, US
    42.336388 -71.072393
  • Astronomy LibraryBorrow it
    725 Commonwealth Avenue, 6th Floor, Boston, MA, 02445, US
    42.350259 -71.105717
  • Fineman and Pappas Law LibrariesBorrow it
    765 Commonwealth Avenue, Boston, MA, 02215, US
    42.350979 -71.107023
  • Frederick S. Pardee Management LibraryBorrow it
    595 Commonwealth Avenue, Boston, MA, 02215, US
    42.349626 -71.099547
  • Howard Gotlieb Archival Research CenterBorrow it
    771 Commonwealth Avenue, 5th Floor, Boston, MA, 02215, US
    42.350723 -71.108227
  • Mugar Memorial LibraryBorrow it
    771 Commonwealth Avenue, Boston, MA, 02215, US
    42.350723 -71.108227
  • Music LibraryBorrow it
    771 Commonwealth Avenue, 2nd Floor, Boston, MA, 02215, US
    42.350723 -71.108227
  • Pikering Educational Resources LibraryBorrow it
    2 Silber Way, Boston, MA, 02215, US
    42.349804 -71.101425
  • School of Theology LibraryBorrow it
    745 Commonwealth Avenue, 2nd Floor, Boston, MA, 02215, US
    42.350494 -71.107235
  • Science & Engineering LibraryBorrow it
    38 Cummington Mall, Boston, MA, 02215, US
    42.348472 -71.102257
  • Stone Science LibraryBorrow it
    675 Commonwealth Avenue, Boston, MA, 02445, US
    42.350103 -71.103784
Processing Feedback ...