How to Conduct a Systematic Review: Protocol, Search & Study Selection

A systematic review is a structured, transparent method for answering a focused research question by locating, appraising, and synthesizing all relevant studies. For students and early-career researchers, the process can feel daunting, but breaking it into three core stages—protocol, search, and study selection—makes it manageable. This guide explains each step in practical terms, with examples and tools you can use to conduct your own systematic review with confidence.

What Is a Systematic Review?

A systematic review is not just a literature review with a longer reading list. It is a rigorous research method that follows an explicit, pre-planned set of steps to minimize bias. Unlike a narrative review, which may select studies based on the author’s interests, a systematic review aims to capture all relevant evidence that meets clear eligibility criteria.

  • Starts with a focused research question, often using the PICO framework.
  • Uses a reproducible search strategy across multiple databases.
  • Applies pre-defined inclusion and exclusion criteria to select studies.
  • Includes an assessment of the risk of bias in included studies.
  • Presents a synthesis that reflects the body of evidence, not just a few convenient studies.

Prepare a Systematic Review Protocol

The protocol is your roadmap. Before you search for anything, you must write down exactly what you plan to do, why, and how. This does more than keep you organized; it helps others understand that your review was planned and not improvised along the way.

  • Define the research question with your population, intervention, comparator, and outcome.
  • List your eligible study designs, such as randomized controlled trials, cohort studies, or qualitative studies.
  • State your inclusion and exclusion criteria before you start screening.
  • Decide on the databases and other sources you will search.
  • Outline how you will extract data and assess study quality.
  • Explain your synthesis method, whether narrative, statistical meta-analysis, or both.

A helpful way to structure your question is the PICO framework. The table below shows how this works with a clinical example.

PICO element Example in a systematic review question
Population College students who report mild to moderate anxiety
Intervention A structured aerobic exercise program lasting at least 8 weeks
Comparator No intervention or usual campus counseling services
Outcome Change in self-reported anxiety scores from baseline to post-intervention

Develop a Comprehensive Search Strategy

Searching for evidence is not about typing a few keywords into Google Scholar. A high-quality systematic review depends on a balanced and sensitive search that finds all potentially relevant studies. You need to combine multiple databases, controlled vocabulary, and Boolean logic to build a reliable strategy.

  • Choose major databases relevant to your field, such as PubMed, Web of Science, Scopus, or ERIC.
  • Use both free-text keywords and subject headings, like MeSH in PubMed.
  • Combine synonyms with OR and different concepts with AND.
  • Use truncation and wildcard symbols carefully to capture spelling variations.
  • Save each search strategy so you can report it in the final review.

For example, a simple PubMed search string for the PICO question above might look like this:

"aerobic exercise" OR "physical activity" OR "exercise intervention" AND "college students" OR "university students" AND anxiety OR "anxiety symptoms"

Test your search strategy by checking whether it retrieves a few known relevant papers. If not, adjust your terms before proceeding.

Select Studies with Clear Eligibility Criteria

Study selection turns your search results into a final set of included studies. You will screen records in stages: first by title and abstract, then by full text. To keep this process fair and reproducible, your eligibility criteria must be specific and applied consistently.

  • Start with a title and abstract screen to remove obviously irrelevant records.
  • Retrieve full texts of all remaining records and re-check them against your criteria.
  • Record your reasons for excluding each full-text study.
  • Involve a second reviewer to reduce mistakes, if possible.
  • Resolve disagreements by discussion or a third reviewer.
Your eligibility criteria should be specific enough to apply consistently, but not so narrow that you miss relevant studies.

Use Screening Tools to Manage the Process

Handling hundreds or thousands of citations in spreadsheets is possible but painful. Fortunately, several free and paid online tools are designed for systematic review workflows. These tools let you import references, remove duplicates, and work through screening in a structured way.

  • Import citations from databases directly into a screening tool.
  • Use automated deduplication to remove duplicate records.
  • Create a labeling system for inclusion, exclusion, or unclear decisions.
  • Enable blinding between reviewers if you want an independent screening process.
  • Track your screening progress with simple statistics.

Popular tools include Rayyan and Covidence, but you can also use a shared spreadsheet if your review is small. The key is to keep a clean record of every decision you make.

Extract Data and Assess Study Quality

Once you have your final list of included studies, you need to collect their key characteristics and outcomes in a structured way. This data extraction stage is critical because your analysis will rely on complete and consistent data from every study.

  • Create a data extraction form that includes author, year, sample size, setting, intervention details, comparator, and outcomes.
  • Pilot the form on two or three studies to identify missing fields or unclear instructions.
  • Extract data from each study in a same format, using the same definitions.
  • Assess the risk of bias in each study with a recognized tool, such as the Cochrane Risk of Bias tool or ROBINS-I for non-randomized studies.
  • Report the quality assessment clearly so readers can judge the strength of your conclusions.

Avoid Common Systematic Review Pitfalls

Even with careful planning, small mistakes can weaken a systematic review. Knowing the common pitfalls in advance helps you stay on track and produce a more reliable final product.

  • Skipping the protocol or not registering it somewhere accessible.
  • Searching only one database and neglecting grey literature.
  • Using overly narrow search terms that miss important variations.
  • Applying eligibility criteria in a subjective or inconsistent way.
  • Not documenting why full-text studies were excluded.
  • Ignoring conflicts of interest or duplicative publications from the same dataset.
A systematic review is only as good as its search. If you miss key terms, you miss key studies.

Final Thoughts

Conducting a systematic review is a demanding but highly rewarding process. By preparing a clear protocol, building a careful search strategy, and selecting studies in a reproducible way, you can produce evidence that is genuinely useful to researchers and decision-makers. Remember that the process is not linear; you may need to revisit your search terms, refine your criteria, or adjust your extraction form as you learn more.

  • Start early and document every step, even the frustrating ones.
  • Ask a supervisor or librarian for help when building your search.
  • Use screening tools to make the process more efficient and transparent.
  • Keep your research question focused to avoid a massive workload.
  • Treat your protocol as a living document, but record any changes honestly.

Frequently Asked Questions

What is a systematic review protocol?

A systematic review protocol is a written document that explains the rationale and planned methods for the review. It includes the research question, search strategy, eligibility criteria, data extraction plan, and quality assessment methods. Registering the protocol in a public repository also helps prevent duplication and unplanned changes.

How long does a systematic review take?

It depends on the topic, the number of included studies, and your experience. A small review done by a student can take several months, while a large review with a team might take over a year. Most of the time is spent searching, screening, and extracting data, so good organization is essential.

How many databases should I search?

At least three databases are recommended for a traditional systematic review. For many health and social science topics, this means PubMed or Cochrane Library, Web of Science or Scopus, and a subject-specific database like CINAHL or PsycINFO. Searching grey literature sources is also recommended to avoid publication bias.

What if I cannot find many studies?

If your search returns very few studies, you may need to broaden your question or reconsider your inclusion criteria. It can also be useful to check how other researchers have defined similar concepts and to search for unpublished studies in clinical trial registries and academic theses. Reporting a thorough search in an empty review is still valuable, but you should explain the limited evidence clearly.

How do I create a Boolean search string?

Use AND to combine different concepts, OR to combine synonyms within a concept, and NOT only when necessary, because it can exclude useful records. Write down your question first, then break it into concepts. For each concept, list synonyms and variant spellings, and link them with OR. Finally, combine the concept groups with AND.

What is the difference between inclusion and exclusion criteria?

Inclusion criteria define the characteristics a study must have to be included in your review, such as participants, intervention, and outcome. Exclusion criteria are conditions that disqualify a study even if it seems relevant, such as wrong language, full text unavailable, or duplicate publication. Both should be based on your research question and stated clearly in your protocol.

Should I include non-English studies?

It depends on your research question and available resources. Excluding non-English studies can introduce language bias, but translating studies can be resource-intensive. Before deciding, think about whether the topic is likely to have important evidence in other languages. Many reviews restrict to English and state this limitation explicitly.

What is grey literature?

Grey literature refers to reports, dissertations, conference papers, government documents, and other materials that are not published in traditional peer-reviewed journals. Searching grey literature can reduce publication bias because studies with null or negative findings are sometimes never published in journals. Examples include trial registries, institutional repositories, and conference abstracts.

How do I screen studies efficiently?

Start with your title and abstract screen using a strict checklist derived from your eligibility criteria. Do not read the full text unless a record’s relevance is unclear. Work in batches, use a screening tool, and track every decision. Having a second reviewer helps, but even one reviewer can stay consistent by regularly reflecting on the criteria.

What is a PRISMA flow diagram?

A PRISMA flow diagram is a visual summary of the study selection process in a systematic review. It shows how many records were identified, duplicates removed, records screened, full texts sought, and studies included or excluded. It is a standard part of most systematic review reports and makes the review process transparent.

Still to read...