The Resource Peer review of search strategies, investigators, Rose Relevo, Robin Paynter, (electronic resource)

Peer review of search strategies, investigators, Rose Relevo, Robin Paynter, (electronic resource)

Label
Peer review of search strategies
Title
Peer review of search strategies
Statement of responsibility
investigators, Rose Relevo, Robin Paynter
Creator
Contributor
Subject
Genre
Language
  • eng
  • eng
Summary
BACKGROUND: Many steps in the preparation of effectiveness and comparative effectiveness reviews (CERs) by the Agency for Healthcare Research and Quality's Effective Health Care (EHC) Program involve outside review and input (e.g., by Key Informants, the Technical Expert Panel, Draft Report Peer Reviewers). However, development of bibliographic database search strategies is currently not consistently peer reviewed. An opportunity exists for the EHC Program to leverage its expert searchers across the 14 Evidence-based Practice Centers (EPCs) in the Program by implementing a process of peer review of search strategies. OBJECTIVE: Due to the number and frequency of observed errors in published search strategies uncovered in the research literature, an evaluation of the feasibility of instituting a peer review process is warranted. Specific Aims: 1. Evaluate whether the PRESS instrument or no-instrument (free-form' evaluations) is preferred by Technical Expert Peer Reviewers (TEPRs) of search strategies.2. Evaluate the usefulness of a peer review process for database search strategies, that is, do peer reviews change search strategies.3. Evaluate the costs of implementing a formal peer review of search strategies program as a part of the review process. METHODS: We identified current research protocol phase CER search strategies to review, TEPRs from across the EHC Program, and the original expert searchers who worked on developing each of the search strategies. Each TEPR was assigned to either the control group who only wrote "free-form" reviews or to the group who initially wrote a "free-form" review, then trained using the PRESS Instrument, and finally completed the last review using the PRESS instrument. Original expert searchers were asked to comment on the reviews of their searches. One study researcher administered the peer review process and log time required to perform it, while the other, blinded study researcher analyzed qualitative and quantitative data derived from the reviews, as well as demographic information about the TEPR and original expert searchers. RESULTS: As a whole, the group of people available to conduct and review search strategies in the EHC Program is professionally educated and very experienced. Of the 24 respondents to the demographic survey, 20 (83%) have master's degrees in library science. Most have more than 10 years' experience as a librarian or other information professional. Sixty-seven percent of respondents have more than 5 years experience contributing to systematic reviews, with 63 percent having contributed to more than 10 systematic reviews. These experienced searchers have a variety of relationships with the EPCs for whom they do searches. Of the 25 peer reviewers invited to participate, 24 completed the initial free-form round of reviews, and 15 completed either the PRESS review or a second free-form review as part of our control group. For the most part peer reviewers were positive about the review process, although many hesitated to incorporate the review process into their current workflow. All of the reviewers found the background material (systematic review protocol) helpful to the review. Of those who used the PRESS instrument, 82 percent (9) indicated that the instrument was helpful, 18 percent (2) reported that it was neither helpful nor limiting, and none of the reviewers indicated that the PRESS instrument was limiting. The PRESS instrument reviews contained more recommendations on the whole and in particular had more comments that could be termed error detection--specific comments about spelling or syntax indicating that a mistake had been made. In 97 percent of cases, the original searcher indicated that the comments did not cause them to alter their search strategies. CONCLUSIONS: While the results of this study suggest that if a formal peer review process is to be valuable then it would need to be both timely and timed for a window of opportunity immediately prior to the finalization of the protocol. Even if a formal peer review process is not implemented, the PRESS instrument could be useful in informal peer review or even self review. If review of search strategies is to take place, then these results suggest that the use of the PRESS instrument would cut down the time taken, increase the likelihood of response and be more effective in identifying actual errors in search strategies. Additionally, the content of the reviews indicates that there are several search tactics for which there is no consensus, and further research could help us to understand variation in practice around such issues as limits, searching for observational studies, and searching for outcomes and comparators. The process of reviewing other searchers' work can bring these issues to light, and a peer review-like process could be used to start investigations and discussions of what techniques work and why. Finally, many of the reviewers commented on the difficulty of reading the search strategies as currently presented. The EHC Program currently has no standards for reporting search strategies, and there is no recognized standard for reporting search strategies. Adopting standard of reporting designed to facilitate review may make it easier to review search strategies both internally and when reported to the public
Member of
Cataloging source
DNLM
http://library.link/vocab/creatorName
Relevo, Rose
Funding information
Prepared for: Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services, 540 Gaither Road, Rockville, MD 20850; www.ahrq.gov Contract No. 290-2007-100572, Prepared by: Oregon Evidence-based Practice Center, Portland, OR
Government publication
federal national government publication
Illustrations
illustrations
Index
no index present
Literary form
non fiction
Nature of contents
  • dictionaries
  • bibliography
NLM call number
WA 21
http://library.link/vocab/relatedWorkOrContributorName
  • Paynter, Robin
  • United States
  • Oregon Health & Science University
Series statement
  • Methods research reports
  • AHRQ publication
Series volume
no. 12-EHC068-EF
http://library.link/vocab/subjectName
  • Peer Review
  • Data Collection
Label
Peer review of search strategies, investigators, Rose Relevo, Robin Paynter, (electronic resource)
Instantiates
Publication
Note
"June 2012."
Bibliography note
Includes bibliographical references
Color
multicolored
Dimensions
unknown
Extent
1 online resource (PDF file (various pagings))
Form of item
online
Other physical details
ill.
Specific material designation
remote
System control number
  • 1592182
  • (DNLM)BKSHLF:NBK98353
Label
Peer review of search strategies, investigators, Rose Relevo, Robin Paynter, (electronic resource)
Publication
Note
"June 2012."
Bibliography note
Includes bibliographical references
Color
multicolored
Dimensions
unknown
Extent
1 online resource (PDF file (various pagings))
Form of item
online
Other physical details
ill.
Specific material designation
remote
System control number
  • 1592182
  • (DNLM)BKSHLF:NBK98353

Library Locations

  • African Studies LibraryBorrow it
    771 Commonwealth Avenue, 6th Floor, Boston, MA, 02215, US
    42.350723 -71.108227
  • Alumni Medical LibraryBorrow it
    72 East Concord Street, Boston, MA, 02118, US
    42.336388 -71.072393
  • Astronomy LibraryBorrow it
    725 Commonwealth Avenue, 6th Floor, Boston, MA, 02445, US
    42.350259 -71.105717
  • Fineman and Pappas Law LibrariesBorrow it
    765 Commonwealth Avenue, Boston, MA, 02215, US
    42.350979 -71.107023
  • Frederick S. Pardee Management LibraryBorrow it
    595 Commonwealth Avenue, Boston, MA, 02215, US
    42.349626 -71.099547
  • Howard Gotlieb Archival Research CenterBorrow it
    771 Commonwealth Avenue, 5th Floor, Boston, MA, 02215, US
    42.350723 -71.108227
  • Mugar Memorial LibraryBorrow it
    771 Commonwealth Avenue, Boston, MA, 02215, US
    42.350723 -71.108227
  • Music LibraryBorrow it
    771 Commonwealth Avenue, 2nd Floor, Boston, MA, 02215, US
    42.350723 -71.108227
  • Pikering Educational Resources LibraryBorrow it
    2 Silber Way, Boston, MA, 02215, US
    42.349804 -71.101425
  • School of Theology LibraryBorrow it
    745 Commonwealth Avenue, 2nd Floor, Boston, MA, 02215, US
    42.350494 -71.107235
  • Science & Engineering LibraryBorrow it
    38 Cummington Mall, Boston, MA, 02215, US
    42.348472 -71.102257
  • Stone Science LibraryBorrow it
    675 Commonwealth Avenue, Boston, MA, 02445, US
    42.350103 -71.103784
Processing Feedback ...