WHAT IS ALREADY KNOWN ON THIS TOPIC

Compared with systematic reviews, rapid reviews (RR) often abbreviate or limit the literature search in some way to accelerate review production. However, RR guidance rarely specifies how to select topic-appropriate search approaches.

WHAT THIS STUDY ADDS

This paper presents an overview of considerations and recommendations for RR searching, covering the complete search process from the planning stage to record management. We also provide extensive appendices with practical examples, useful sources and a glossary of terms.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICYIntroduction

This paper is part of a series from the Cochrane Rapid Reviews Methods Group (RRMG) providing methodological guidance for rapid reviews (RRs).1–3 While the RRMG’s guidance4 5 on Cochrane RR production includes brief advice on literature searching, we aim to provide in-depth recommendations for the entire search process.

Literature searching is the foundation for all reviews; therefore, it is important to understand the goals of a specific RR. The scope of RRs varies considerably (from focused questions to overviews of broad topics).6 As with conventional systematic reviews (SRs), there is not a one-size-fits-all approach for RR literature searches. We aim to support RR teams in choosing methods that best fit their project while understanding the limitations of modified search methods. Our recommendations derive from current systematic search guidance, evidence on modified search methods and practical experience conducting RRs.

This paper presents considerations and recommendations, described briefly in table 1. The table also includes a comparison to the SR search process based on common recommendations.7–10 We provide supplemental materials, including a list of additional resources, further details of our recommendations, practical examples, and a glossary (explaining the terms written in italics) in online supplemental appendices A–C.

Table 1

Recommendations for rapid review literature searching

Preparation and planning

Given that the results of systematic literature searches underpin a review, planning the searches is integral to the overall RR preparation. The RR search process follows the same steps as an SR search; therefore, RR teams must be familiar with the general standards of systematic searching. Templates (see online supplemental appendix B) and reporting guidance11 for SR searches can also be adapted to structure the RR search process.

Developing a plan for the literature search forms part of protocol development and should involve an information specialist (eg, librarian). Information specialists can assist in refining the research question, selecting appropriate search methods and resources, designing and executing search strategies, and reporting the search methods. At minimum, specialist input should include assessing information sources and methods and providing feedback on the primary database search strategy.

Two options exist for abbreviating the search process: (1) reducing time spent on conducting searches (eg, using automation tools, reusing existing search strategies, omitting planning or quality assurance steps) and (2) reducing the size of the search result (eg, limiting the number of information sources, increasing the precision of search strategies, using study design filters). Study selection (ie, screening search results) is usually more resource-intensive than searching,12 particularly for topics with complex or broad concepts or diffuse terminology; thus, the second option may be more efficient for the entire RR. Investing time upfront in optimising search sensitivity (ie, completeness) and precision (ie, positive predictive value) can save time in the long run by reducing the screening and selection workload.

Preliminary or scoping searches are critical to this process. They inform the choice of search methods and identify potentially relevant literature. Texts identified through preliminary searching serve as known relevant records that can be used throughout the search development process (see sections on database selection, development and validation of search strategies).

In addition to planning the search itself, the review team should factor in time for quality assurance steps (eg, search strategy peer review) and the management of search results (eg, deduplication, full-text retrieval).

Information sources and methods

To optimise the balance of search sensitivity and precision, RR teams should prioritise the most relevant information sources for the topic and the type of evidence required. These can include bibliographic databases (eg, MEDLINE/PubMed), grey literature sources and targeted supplementary search methods. Note that this approach differs from the Methodological Expectations of Cochrane Intervention Reviews Standards9 where the same core set of information sources is required for every review and further supplemented by additional topic-specific and evidence-specific sources.

Choosing bibliographic databases

For many review topics, most evidence is found in peer-reviewed journal articles, making bibliographic databases the main resource of systematic searching. Limiting the number of databases searched can be a viable option in RRs, but it is important to prioritise topic-appropriate databases.

MEDLINE has been found to have high coverage for studies included in SRs13 14 and is an appealing database choice because access is free via PubMed. However, coverage varies depending on topics and relevant study designs.15 16 Additionally, even if all eligible studies for a topic were available in MEDLINE, search strategies will usually miss some eligible studies because search sensitivity is lower than database coverage.13 17 This means searching MEDLINE alone is not a viable option, and additional information sources or search methods are required. Known relevant records can be used to help assess the coverage of selected databases (see also online supplemental appendix C).

Further information sources and search techniques

Supplementary systematic search methods have three main goals, to identify (1) grey literature, (2) published literature not covered by the selected bibliographic databases and (3) database-covered literature that was not retrieved by the database searches.

When RRs search only a small number of databases, supplementary searches can be particularly important to pick up eligible studies not identified via database searching. While supplementary methods might increase the time spent on searching, they sometimes better optimise search sensitivity and precision, saving time in the long run.18 Depending on the topic and relevant evidence, such methods can offer an alternative to adding additional specialised database searches. To decide if and what supplementary searches are helpful, it is important to evaluate what literature might be missed by the database searches and how this might affect the specific RR.

Study registries and other grey literature

Some studies indicate that the omission of grey literature searches rarely affects review conclusions.17 19 However, the relevance of study registries and other grey literature sources is topic-dependent.16 19–21 For example, randomised controlled trials (RCTs) on newly approved drugs are typically identified in ClinicalTrials.gov.20 For rapidly evolving topics such as COVID-19, preprints are an important source.21 For public health interventions, various types of grey literature may be important (eg, evaluations conducted by local public health agencies).22

Further supplementary search methods

Other supplementary techniques (eg, checking reference lists, reviewing specific websites or electronic table of contents, contacting experts) may identify additional studies not retrieved by database searches.23 One of the most common approaches involves checking reference lists of included studies and relevant reviews. This method may identify studies missed by limited database searches.12 Another promising citation-based approach is using the ‘similar articles’ option in PubMed, although research has focused on updating existing SRs.24 25

Considerations for RRs of RCTs

Databases and search methods to identify RCTs have been particularly well researched.17 20 24 26 27 For this reason, it is possible to give more precise recommendations for RRs based on RCTs than for other types of review. Table 2 provides an overview of the most important considerations; additional information can be found in online supplemental appendix C.

Table 2

Information sources for identification of randomised controlled trials (RCTs)

Search strategies

We define ‘search strategy’ as a Boolean search query in a specific database (eg, MEDLINE) using a specific interface (eg, Ovid). When several databases are searched, this query is usually developed in a primary database and interface (eg, Ovid MEDLINE) and translated to other databases.

Developing search strategies

Optimising search strategy precision while aiming for high sensitivity is critical in reducing the number of records retrieved. Preliminary searches provide crucial information to aid efficient search strategy development. Reviewing the abstracts and subject headings used in known relevant records will assist in identifying appropriate search terms. Text analysis tools can also be used to support this process,28 29 for example, to develop ‘objectively derived’ search strategies.30

Reusing or adapting complete search strategies (eg, from SRs identified by the preliminary searches) or selecting elements of search strategies for reuse can accelerate search strategy development. Additionally, validated search filters (eg, for study design) can be used to reduce the size of the search result without compromising the sensitivity of a search strategy.31 However, quality assurance measures are necessary whether the search strategy is purpose-built, reused or adapted (see the ‘Quality assurance’ section.)

Database-specific and interface-specific functionalities can also be used to improve searches’ precision and reduce the search result’s size. Some options are: restricting to records where subject terms have been assigned as the major focus of an article (eg, major descriptors in MeSH), using proximity operators (ie, terms adjacent or within a set number of words), frequency operators (ie, terms have to appear a minimum number of times in an abstract) or restricting search terms to the article title.32–34

Automated syntax translation can save time and reduce errors when translating a primary search strategy to different databases.35 36 However, manual adjustments will usually be necessary.

The time taken to learn how to use supporting technologies (eg, text analysis, syntax translation) proficiently should not be underestimated. The time investment is most likely to pay off for frequent searchers. A later paper in this series will address supporting software for the entire review process.

Limits and restrictions

Limits and restrictions (eg, publication dates, language) are another way to reduce the number of records retrieved but should be tailored to the topic and applied with caution. For example, if most studies about an intervention were published 10 years ago, then an arbitrary cut-off of ‘the last 5 years’ will miss many relevant studies.37 Similarly, limiting to ‘English only’ is acceptable for most cases, but early in the COVID-19 pandemic, a quarter of available research articles were written in Chinese.38 Depending on the RR topic, certain document types (eg, conference abstracts, dissertations) might be excluded if not considered relevant to the research question.

Note also that preset limiting functions in search interfaces (eg, limit to humans) often rely on subject headings (eg, MeSH) alone. They will miss eligible studies that lack or have incomplete subject indexing. Using (validated) search filters 31 is preferable.

Updating existing reviews

One approach to RR production involves updating an existing SR. In this case, preliminary searches should be used to check if new evidence is available. If the review team decide to update the review, they should assess the original search methods and adapt these as necessary.

One option is to identify the minimum set of databases required to retrieve all the original included studies.39 Any reused search strategies should be validated and peer-reviewed (see below) and optimised for precision and/or sensitivity.

Additionally, it is important to assess whether the topic terminology or the relevant databases have changed since the original SR search.

In some cases, designing a new search process may be more efficient than reproducing the original search.

Quality assurance and search strategy peer review

Errors in search strategies are common and can impact the sensitivity and comprehensiveness of the search result.40 If an RR search uses a small number of information sources, such errors could affect the identification of relevant studies.

Validation of search strategies

The primary database search strategy should be validated using known relevant records (if available). This means testing if the primary search strategy retrieves eligible studies found through preliminary searching. If some known studies are not identified, the searcher assesses the reasons and decides if revisions are necessary. Even a precision-focused systematic search should identify the majority—we suggest at least 80%–90%—of known studies. This is based on benchmarks for sensitivity-precision-maximising search filters41 and assumes that the set of known studies is representative of the whole of relevant studies.

Peer review of search strategies

Ideally, an information specialist should review the planned information sources and search methods and use the PRESS (Peer Review of Electronic Search Strategies) checklist42 to assess the primary search strategy. Turnaround time has to be factored into the process from the outset (eg, waiting for feedback, revising the search strategy). PRESS recommends a maximum turnaround time of five working days for feedback, but in-house peer review often takes only a few hours.

If the overall RR time plan does not allow for a full peer review of the search strategy, a review team member with search experience should check the search strategy for spelling errors and correct use of Boolean operators (AND, OR, NOT) at a minimum.

Reporting and record management

Record management requirements of RRs are largely identical to SRs and have to be factored into the time plan. Teams should develop a data management plan and review the relevant reporting standards at the project’s outset. PRISMA-S (Preferred Reporting Items for Systematic Reviews and Meta-Analyses literature search extension)11 is a reporting standard for SR searches that can be adapted for RRs.

Reference management software (eg, EndNote,43 Zotero44) should be used to track search results, including deduplication. Note that record management for database searches is less time-consuming than for many supplementary or grey literature searches, which often require manual entry into reference management software.12

Additionally, software platforms for SR production (eg, Covidence,45 EPPI-Reviewer,46 Systematic Review Data Repository Plus47) can provide a unified way to keep track of records throughout the whole review process, which can improve management and save time. These platforms and other dedicated tools (eg, SRA Deduplicator)48 also offer automated deduplication. However, the time and cost investment necessary to appropriately use these tools have to be considered.

Conclusion

Decisions about search methods for an RR need to consider where time can be most usefully invested and processes accelerated. The literature search should be considered in the context of the entire review process, for example, protocol development and literature screening: Findings of preliminary searches often affect the development and refinement of the research question and the review’s eligibility criteria. In turn, they affect the number of records retrieved by the searches and therefore the time needed for literature selection.

For this reason, focusing only on reducing time spent on designing and conducting searches can be a false economy when seeking to speed up review production. While some approaches (eg, text analysis, automated syntax translation) may save time without negatively affecting search validity, others (eg, skipping quality assurance steps, using convenient information sources without considering their topic appropriateness) may harm the entire review. Information specialists can provide crucial aid concerning the appropriate design of search strategies, choice of methods and information sources.

For this reason, we consider that investing time at the outset of the review to carefully choose a small number of highly appropriate search methods and optimise search sensitivity and precision likely leads to better and more manageable results.

Data availability statement

No data are available.

Ethics statementsPatient consent for publication

Not applicable.