Systematic Searching

"A systematic review attempts to identify, appraise, and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a given research question. Researchers conducting systematic reviews use explicit methods aimed at minimizing bias in order to produce more reliable findings that can be used to inform decision making."

--Cochrane Collaboration

Learn how to conduct a systematic review. 

Meta-Analysis

Meta-analysis is "the quantitative, scientific synthesis of research results...used to synthesized evidence across studies to detect effects, estimate their magnitudes, and analyze the factors influencing those effects."

Gurevitch J, Koricheva J, Nakagawa S, Stewart G. Meta-analysis and the science of research synthesis. Nature. 2018 Mar;555(7695):175-82.

Network Meta-Analysis (NMA)

"Network meta-analysis is a technique for comparing multiple treatments simultaneously in a single analysis by combining direct and indirect evidence within a network of randomized controlled trials."

Rouse B, Chaimani A, Li T. Network meta-analysis: an introduction for clinicians. Internal and emergency medicine. 2017 Feb 1;12(1):103-11.

Individual Patient Data Meta-Analysis (IPD-MA)

"Meta‐analysis using individual participant data (IPD) obtains and synthesises the raw, participant‐level data from a set of relevant studies." 

Burke DL, Ensor J, Riley RD. Meta‐analysis using individual participant data: one‐stage and two‐stage approaches, and why they may differ. Statistics in medicine. 2017 Feb 28;36(5):855-75.

Literature / Narrative Review

Literature/narrative review is a "comprehensive study and interpretation of literature that addresses a specific topic."

🕮 Aveyard H. Doing a literature review in health and social care: A practical guide. McGraw-Hill Education (UK); 2014.

Scoping Review

"A scoping review or scoping study is a form of knowledge synthesis that addresses an exploratory research question aimed at mapping key concepts, types of evidence, and gaps in research related to a defined area or field by systematically searching, selecting, and synthesizing existing knowledge."

🕮 Colquhoun HL, Levac D, O'Brien KK, Straus S, Tricco AC, Perrier L, Kastner M, Moher D. Scoping reviews: time for clarity in definition, methods, and reporting. Journal of clinical epidemiology. 2014 Dec 1;67(12):1291-4.

➲ GUIDANCE for scoping reviews:

🕮 Peters MDJ, Godfrey C, McInerney P, Munn Z, Tricco AC, Khalil, H. Chapter 11: Scoping Reviews (2020 version). In: Aromataris E, Munn Z (Editors). Joanna Briggs Institute Manual for Evidence Synthesis, JBI, 2020. Available from https://synthesismanual.jbi.global. https://doi.org/10.46658/JBIMES-20-12

🕮 Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. International journal of evidence-based healthcare. 2015 Sep 1;13(3):141-6.doi: 10.1097/XEB.0000000000000050.PMID: 26134548

🕮 von Elm E, Schreiber G, Haupt CC. Methodische Anleitung für Scoping Reviews (JBI-Methodologie). Zeitschrift für Evidenz, Fortbildung und Qualität im Gesundheitswesen. 2019 Jun 1;143:1-7. German. doi: 10.1016/j.zefq.2019.05.004.PMID: 31296451.

🕮 PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. This reporting standard for scoping reviews contains 20 essential items and other resources such as a fillable checklist and flowchart.

Rapid Review

"Rapid reviews are a form of knowledge synthesis in which components of the systematic review process are simplified or omitted to produce information in a timely manner."

🕮 Khangura S, Konnyu K, Cushman R, Grimshaw J, Moher D. Evidence summaries: the evolution of a rapid review approach. Systematic reviews. 2012 Dec 1;1(1):10.

➲ GUIDANCE for rapid reviews:

🕮 Tricco AC, Langlois E, Straus SE, World Health Organization. Rapid reviews to strengthen health policy and systems: a practical guide. World Health Organization; 2017.

Realist Review / Synthesis

"A synthesis of complex social interventions to explore how contextual factors influence the link between intervention and outcome (summed up in the question "what works, how, for whom, in what circumstances and to what extent?")..."

🕮 Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R. RAMESES publication standards: realist syntheses. BMC Med. 2013 Jan 29;11:21.

Umbrella Reviews (Reviews of reviews)

"An umbrella review allows the findings of reviews relevant to a review question to be compared and contrasted. An umbrella review's most characteristic feature is that this type of evidence synthesis only considers for inclusion the highest level of evidence, namely other systematic reviews and meta-analyses."

🕮 Aromataris E, Fernandez R, Godfrey CM, Holly C, Khalil H, Tungpunkom P. Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach. International journal of evidence-based healthcare. 2015 Sep 1;13(3):132-40.

➲ GUIDANCE for umbrella reviews:

🕮 Aromataris E, Fernandez R, Godfrey C, Holly C, Khalil H, Tungpunkom P. Chapter 10: Umbrella Reviews. In: Aromataris E, Munn Z (Editors). Joanna Briggs Institute Manual for Evidence Synthesis, JBI, 2020. Available from https://synthesismanual.jbi.global. https://doi.org/10.46658/JBIMES-20-11

1.  Determine if a systematic review is an appropriate review methodology for your research question. 

2.  Check for recently published systematic reviews on your topic. (You do not want to duplicate work!)

3.  Gather experts for your systematic review team, including an information specialist experienced in systematic literature searching. 

4.  Write and register your systematic review protocol. 

5.  Carry out a thorough search of the literature.

6.  Screen and select studies based on your protocol.

7.  Perform data extraction.

8.  Assess the quality of the studies. 

9.  Synthesize the results. Perform meta-analysis, if a statistical component is included. 

10. Interpret the findings.

11. Write and publish your systematic review.

Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital Systematic Review and Network Meta-Analysis Process Diagram
https://knowledgetranslation.net/knowledge-syntheses/ks-research/systematic-reviews/

Systematic reviews are considered the highest levels of evidence, but only if they are done rigorously.

(c) Copyright 2006 - 2014. Trustees of Dartmouth College and Yale University. All Rights Reserved. Produced by Jan Glover, Dave Izzo, Karen Odato, and Lei Wang.

More information

Systematic reviews are executed in a collaborative manner and take a substantial amount of time to complete. Systematic reviews can take approximately 12-18 months, from protocol to publication. 

A proper systematic review cannot be conducted by one person. You will need:

• Subject experts with clinical and methodological expertise;
• Two people to review the results independently;
• A third person or tiebreaker to make decisions if there is disagreement about a study meeting the inclusion criteria;
• A statistician, if performing a meta-analysis;
• An information specialist/medical librarian trained to conduct a systematic literature search.

Information Specialist Co-Authorship

Information specialists can:  

• Design the search strategy, a substantive contribution to the study and acquisition of data
• Write the methodology section detailing search strategies used
• Edit the manuscript and provide approval of the methodology section
• Agree to be accountable to the rest of the team members

Authors who do not wish to engage the information specialists as co-authors must acknowledge them as contributors. 

Available through the University of Bern Library
(Link auf Listen in Swissbib, wird von der FBB hergestellt)

Available to borrow from the Library, click on the title for more information. 

Gough D, Oliver S, Thomas J, editors. An introduction to systematic reviews. Sage; 2017 Mar 28.

Higgins JP, Green S, editors. Cochrane handbook for systematic reviews of interventions. John Wiley & Sons; 2011 Aug 24.

Example

In patients with an acute migraine, what is the effect of sumatriptan compared with naproxen on pain relief?

P            patients with an acute migraine

I             sumatriptan

C            naproxen

O            pain relief

There are additional frameworks to help structure other types of research questions.

PEO

P = Phenomenon of Interest
E = Exposure
O = Outcome

__________

PICo

PI = Phenomenon of Interest
Co = Context

__________

SPIDER

S = Sample
PI = Phenomenon of Interest
D = Design
E = Evaluation
R = Research type

A protocol is a detailed plan for your study. It includes a rationale for conducting the project, research question, inclusion/exclusion criteria, literature search plan, a method for data abstraction/data management and to evaluate the quality of studies.

Support article: How to setup a protocol (SciFlow)

PROSPERO template | PRISMA template

The Cochrane Collaboration publishes protocols as does the journal Systematic Reviews

Many health care journals are publishing protocols now too, see for example the following protocols listed in PubMed.

If you have a specific research question you may need to search for scholarly literature via a citation database. 

What is a citation database? A citation database is an online collection of referenced articles, books, and other materials. Depending on your topic, you may need to search in interdisciplinary and/or subject-specific databases.

Below are examples of citation databases. There is also a "cheat sheet" for a quick overview of some of the popular databases you will use for literature searching.

A comprehensive guide to searching and databases is RefHunter - Manual zur Literaturrecherche in Fachdatenbanken (pdf, German).

What is it?

PubMed is the free interface for searching MEDLINE records. MEDLINE is the largest database of biomedical literature. PubMed also contains other resources and tools. 

Scope/content

PubMed contains more than 30 million citations from the biomedical, life sciences literature. 

Features

You can access other resources such as PubMed Central, NCBI Bookshelf, and search tools such as the MeSH database to find subject headings for your literature search. You can create an account to save references, search strategies, create search alerts, etc. 

Search techniques

You can conduct a basic or an advanced search. The basic search is useful to explore a topic. PubMed's advanced search builder is helpful for systematic searches (when you have identified your research question and key concepts). 

What is it? 

CINAHL stands for the Cumulative Index of Nursing and Allied Health Literature. It is a database via the EBSCOhost platform. 

Scope/content

Scope coverage in nursing, biomedicine, health sciences, alternative/complementary medicine, consumer health, etc. Other resources include health care books, nursing dissertations, selected conference proceedings, book chapters, and standards of practice materials. 

Features 

Access to evidence-based summaries of diseases, treatment options, etc. Nursing staff members can obtain continuing education units from the modules available. Basic and advanced search options are available. You can create an account to save your searches and references. Export search results from CINAHL to citation management tools. 

Search techniques

What is it?

Ovid MEDLINE® is produced by the U.S. National Library of Medicine (NLM). Ovid is the searching platform for MEDLINE, the largest biomedical database.

Scope/content

The database contains more than 23 million references. It covers topics such as biomedicine, allied health, veterinary sciences, biological and physical sciences, humanities, and information science dating back to 1946.

Features 

Basic and advanced search modes available. Other search features citation search, similar articles. For topic searches, you can access the MeSH database to identify subject headings. 

Search techniques

What is it?

Scopus is a license-based abstract and citation database by Elsevier.

Scope/content

It comprises over 77.8 million records in the fields of science, technology, medicine, social science, arts, and humanities from 1788 until today.

Features 

It provides analytical tools to create citation overviews, view author and affiliation profiles, citation scores, and examine article and journal metrics. You can conduct a basic and advanced search. In the advanced search mode, you can search in various search fields and use advanced searching techniques. 

Search techniques

What is it? 

Excerpta Medica Database, known as EMBASE, is a biomedical and pharmacological database. You can access this database via Ovid or Elsevier provider. This guide refers to the Elsevier provider version. 

Scope/Content

This database contains 30 million records including articles with strong coverage of drug and pharmaceutical research, pharmacology, and toxicology.

Features 

The PV Wizard search tool is helpful for drug and pharmaceutical searches. Basic, PICO, and advanced search modes are available. Access the Emtree Thesaurus to find subject headings for topic searches.  

Search techniques

What is it?

Dimensions is a license based platform for linked and enriched research data. A free version is available too.

Scope / content

It comprises records in the fields of medicine, life science and health science but also social sciences and other research area: Publications, clinical trials, grants, policy papers, patents and datasets.
The data is gathered from Crossref, PubMed, Europe PubMed Central, arXiv and from direct contacts with more than 130 publishers.

Stand 6.5.2021, © dimensions.ai

Features

• Linked and enriched data
  Dimensions links different data to each other. With a single search it displays publications, grants, data sets, clinical trials, patents, and policy documents at one glance.
   
• Analysis of abstracts
  Dimensions analyzes abstracts and suggests similar articles or search concepts. The latter can easily be added to a search string.
   
• Metrics
  Dimensions Analytics offers metrics: How is a publication or a researcher's work perceived within the scientific community. Results are displayed in different visual ways.

Search techniques

Grey Literature is literature that is not published in the traditional sense such as clinical trials data, conference abstracts and proceedings, poster presentations, technical reports, government reports, datasets, thesis/dissertations, newsletters, lab notes, registries. It is important to include grey literature in your review in order to broaden its scope, reduce publication bias, and include emerging research.
We are often asked what database to search for grey literature. The truth is that there isn't just one and it depends on the topic. Consider your topic and its stakeholders. Checking the publications written by organizations and associations relevant to your topic is a good way to start.
Here are some resources for grey literature searching:

Grey Literature Search Guide from the University of Toronto Libraries
From the University of Toronto Libraries' Systematic Reviews in the Sciences & Health Sciences Guide.

Grey Matters: A Practical Tool for Searching Health-Related Grey Literature
CADTH's Directory of grey literature producers, with a focus on health technology assessment and drug resources.

GreyNet (former OpenGrey)
Information on Grey Literature in Europe and access to 700.000 bibliographical references of grey literature (paper) produced in Europe and allows you to export records and locate the documents.

ClinicalTrials.gov
ClinicalTrials.gov is a registry and results database of publicly and privately supported clinical studies of human participants conducted around the world.

European Union Clinical Trials Register
The EU Clinical Trials Register contains information on interventional clinical trials on medicines conducted in the European Union (EU).

ISRCTN registry
The ISRCTN registry is a primary clinical trial registry recognised by WHO and ICMJE that accepts all clinical research studies (whether proposed, ongoing or completed), providing content validation and curation and the unique identification number necessary for publication.

WHO International Clinical Trials Registry Platform (ICTRP)
The main aim of the WHO ICTRP is to facilitate the prospective registration of the WHO Trial Registration Data Set on all clinical trials, and the public accessibility of that information.

Conference Proceedings Citation Index (part of Web of Science Core Collection)

Other databases such as Embase and PsycINFO include conference proceedings and papers. You can also search individual conference proceedings to locate relevant papers.

ProQuest Dissertations & Theses Global

DissOnline und Online-Dissertationen der Deutsche Nationalbibliothek

Networked Digital Library of Theses and Dissertations, NDLTD

CADTH (Canadian Agency for Drugs and Technologies in Health)
CADTH is a Canadian independent, not-for-profit organization responsible for providing health care decision-makers with objective evidence.

SuRe Info
Summarized Research Information Retrieval for health technology assessment.

European Medicines Agency
The European Medicines Agency (EMA) is a decentralised agency of the European Union (EU), located in London. It began operating in 1995. The Agency is responsible for the scientific evaluation, supervision, and safety monitoring of medicines in the EU. 

Devices@FDA 
Devices@FDA is a catalog of cleared and approved medical device information from the United States Food and Drug Administration (FDA). It includes links to the device summary information, manufacturer, approval date, user instructions, and other consumer information.

Drugs@FDA
Information about FDA-approved brand name and generic prescription and over-the-counter human drugs and biological therapeutic products. Drugs@FDA includes most of the drug products approved since 1939.

Researchers may decide to search Google Scholar for studies but may sometimes limit to a certain number of results such as the first 100 or 200. 

Google Scholar searches may retrieve grey literature not indexed in academic databases. Keep in mind that Google Scholar is a search engine not a bibliographic database. 

Google Scholar has an “Advanced Search” which can help focus on the vast number of results. Click on the menu icon in the top left corner to access.

Before you begin searching: breakdown your research question into key concepts. 

Research question example:

Are helmets effective in preventing head injury for cyclists? 

Ask yourself:

• What do I need to know?
• How can I determine if something is relevant to my study or project?
• Which search terms do I need to see in a relevant article?

You can also use a  framework or search concept tool to help identify your key concepts to your question. As you formulate your question, find out where to search. 

Keywords are considered to be your natural language; words you use to describe your concepts. 

Create a list of synonyms for each of your concepts. A good way to identify synonyms is to scan the titles and abstracts of key articles to see how your concept is described. In emerging fields usage may not be consistent. Consider different spellings and phrasing.  Searching in the database, keywords are searched in record’s title, abstract, author.

Example: Keyword search in Ovid Medline:

For example, “knowledge translation”, “translational medicine”, “knowledge diffusion”, “knowledge mobilization”, "bench to bedside” are all terms that may be used for similar concepts.

Another way to find synonyms is to check the "Used For" terms in a database that uses subject headings. Here are the “Used For” terms in the scope note (description) for the MeSH subject heading “Translational Medical Research”

Subject headings are pre-defined "controlled vocabulary" terms used to describe the content or topic of a publication. Subject headings are selected and indexed in a specific part of the record. 

Each database may contain its own subject headings. In Medline via PubMed, subject headings are called MeSH (Medical Subject Headings).

Boolean operators connect your search terms together to either narrow or broaden your set of results. The three basic Boolean operators are: AND, OR, and NOT. Boolean operators must often be capitalized.

OR

Use OR to search any search terms. OR combines terms similar in concept. Helps broaden your search

cyclists OR cycling

AND

Use AND to narrows your results. AND searches for all search terms. Helps narrow your search

bicycles AND helmets AND head injuries

NOT

Use NOT to exclude search terms. NOT should be used with caution.

cycling NOT motorcycles

Proximity operators allow you to determine how closely search terms should appear in the search results. This is useful when concepts are expressed in different ways. Proximity operator syntax/symbols can vary by database. 

Example:

Databases follow a search order when search terms are combined with more than one boolean operator (AND, OR, NOT).

Nesting (the use of parentheses) organises search terms and the search order.

If you do not nest your search terms the database will interpret the search for you.

For example, in PubMed, the AND operator is searched first, but it follows a search order from left to right. 

Searching: cyclists OR bicycles AND helmets OR “head protection”

PubMed will interpret: (cyclists OR bicycles AND helmets) OR "head protection"

Nesting your terms: (cyclists OR bicycles) AND (helmets OR “head protection”)

PubMed will search terms in parenthesis first then search with AND.

Wildcard symbols substitute one or more letters within a word. This is useful if a word is spelled differently. Certain databases may provide different options for wildcards. 

For example, in Ovid MEDLINE, you can use a # or ? <

Colo?r  will find:

• color (American spelling)
• colour (British spelling) 

Databases usually require phrases to be placed within double quotes, for example, “kidney transplant”. Without the double quotes, the database will search for the words individually anywhere in the record.

Almost all databases allow you to search specific fields in the bibliographic record such as title, abstract, author, journal title. Searching specific fields such as title and abstract only can allow you to do a more focused search. Check the Advanced Search options if field searching is not available in the basic search.

Search filters are pre-defined strategies that have been developed by experienced searchers to help answer specific clinical questions, limit to certain populations or study designs. Many of them are validated.

There are a wide variety of filters available. You would first run your search and then add the search filter to your strategy. If you decide to use a filter, be sure to cite it. Below is a selection of links to search filters.

BMJ Clinical Evidence Study Design Filters

CADTH's Search Filters to Identify Economic Evaluations
A group of search filters developed and tested using a gold standard set of economic evaluations and statistical analysis.

Clinical Queries in PubMed Search Page
Specialized PubMed searches intended for clinicians to limit retrieval to articles that report research conducted with specific methodologies. Clinical Queries in PubMed Tutorial

Cochrane Search Filters
The Cochrane Collaboration has developed highly sensitive search strategies for identifying randomized trials

InterTASC Information Specialists' Sub-Group Search Filter Resource
A collaborative venture to identify, assess, and test search filters designed to retrieve research by study design or focus.

McMaster University Health Information Research Unit (HiRU) Hedges
Search filters developed and validated by researchers in HiRU are available for use on PubMed’s Clinical Queries page and Health Services Research Queries page, Ovid’s Additional Limits page for MEDLINE, Embase and PsycINFO; EBSCOhost’s main searching page for CINAHL; and HiRU’s Nephrology filters and Knowledge Translation (KT) filters.

More PubMed Topic Specific Queries
The starting point for all the PubMed special queries, including comparative effectiveness research, health services research, AIDS, health disparities, and more.

Scottish Intercollegiate Guidelines Network (SIGN) Search Filters
SIGN has devised suitable strategies for search filters in Ovid Medline, Ovid Embase, and CINAHL. Filter available includes Systematic Reviews, Randomized Controlled Trials, Observational Studies, Diagnostic Studies, Economic Studies, Patient issues.

You may have also set out other limits in your systematic review protocol such as date limits or age limits. We recommend that carry these out at the end of your search.

Also known as forward and backward citation searching or “snowball” searching, you can check for articles listed in the bibliographies of key articles to see if there are additional relevant studies.

Use Web of Science, Scopus or Google Scholar to link to the articles that have cited key articles.

Once you have created a search strategy, you can save the search history. You can save it within the database or set up email alerts of new publications based on your search strategy. Register for a personal account within the respective database to save your search history. 

Saving the search history can help:

Keep track of your searching methods important for publication

Provide a base to refine or improve the search strategy

Determine searches with the most relevant results

The Canadian Agency for Drugs and Technologies in Health (CADTH) Peer Review Checklist for Search Strategies (PRESS) is a checklist of items to aid in the assessment of electronic database search strategies. This checklist is recommended for use by librarians undertaking the peer review of systematic review search strategies.

It is important to continuously monitor the literature for new studies on your topic. Once you have created a search strategy and saved it your personal account within the database, you can use it to run automatic updates or alerts to be sent to your email.

You can also set up table of contents alerts from specific journals.

Each database had a slightly different method for created accounts and alerts. See for example, the instructions for PubMed e-mail alerts.

Depending on how long it is taking you to carry out the systematic review, you will need to update your literature search to include the latest evidence. If you have saved your search strategies in the respective databases, then it should not be difficult to re-run the search with new date limits.

The literature search is also typically updated prior to being submitted for publication.

You may wish to check if any key articles have been retracted. Try searching Retraction Watch.

Keep track of all included and excluded records for reporting. There are free and subscription-based online tools to help with the screening progress: RevMan, Covidence, EPPI Reviewer, DistillerSR, or Rayyan. 

The Systematic Review Toolbox, is an online search catalogue for tools supporting systematic review processes. 

Your literature search methods should be transparent and reproducible. The reproducibility of your search strategy makes your work credible and reliable. 

Tools like Notepad, Google Docs, Microsoft Word, Microsoft Excel can help you keep a detailed record of your methods and search strategy. 

Reporting your search methods helps users to evaluate the validity of your project.

Atkinson, K. M., Koenka, A. C., Sanchez, C. E., Moshontz, H., & Cooper, H. (2015). Reporting standards for literature searches and report inclusion criteria: making research syntheses more transparent and easy to replicate. Research synthesis methods, 6(1), 87-95.

https://www.ncbi.nlm.nih.gov/pubmed/26035472

Rethlefsen ML, Kirtley S, Waffenschmidt S, Ayala AP, Moher D, Page MJ, Koffel JB. PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews. Systematic reviews. 2021 Dec;10(1):1-9.

https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-020-01542-z

Note: There are reporting standards and guidelines for study types (i.e. randomized control trial, case study, observational study, systematic reviews)

Page MJ, McKenzie J, Bossuyt P, Boutron I, Hoffmann T, Mulrow Cindy et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews 2020. doi:10.31222/osf.io/v7gm2.

The diagram is part of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 Statement, a reporting standard for systematic reviews. 

The PRISMA flow diagram template is an example of how to report your search results. 

Study selection is determined by the inclusion/exclusion criteria you have specified in your project. At a minimum, two reviewers will independently screen search results.

There are two levels of screening: title and abstract screening and full-text article review.

1. First level: screening of titles and abstracts of records to see if they potentially meet the criteria. Those meeting the criteria will move forward to the second level.
    
2. Second level: full-text article review. 

Track the number of excluded records and the reasons for exclusion, review the PRISMA 2020 Statement and Checklist. 

Once you have selected your studies, you can begin the data collection/data extraction process. Data extraction helps authors to summarize and appraise studies. 

What are the essential elements/characteristics you will need to know about your included studies?

There are data extraction tools and forms to help in this process. 

Below is an example of a data extraction form used for RCT studies in a systematic review. 
This form was created by the Cochrane Collaboration.

Critical appraisal is the careful analysis of studies to determine their relative value. The Health and Medical Division of the National Academy of Sciences includes Standard 3.6: “Critically appraise each study:

3.6.1 Systematically assess the risk of bias, using predefined criteria
3.6.2 Assess the relevance of the study’s populations, interventions, and outcome measures
3.6.3 Assess the fidelity of the implementation of interventions”

See also: What is Critical Appraisal?

AMSTAR Checklist (A Measurement Tool to Assess Systematic Reviews)
A valid, reliable and useable instrument that helps users differentiate between systematic reviews, focusing on their methodological quality and expert consensus.

CATMaker and EBM Calculators
Centre for Evidence-Based Medicine (Oxford, UK). CATmaker is a computer-assisted critical appraisal tool, now free. It provides guided appraisal and calculations for therapy, diagnosis, prognosis, etc.

Critical Appraisal Checklists by BMJ Best Practice

Critical Appraisal Skills Programme (CASP) Tools & Checklists

Critical Appraisal Tools from CEBM

GRADE Working Group
The working group has developed a common, sensible and transparent approach to grading quality of evidence and strength of recommendations.

International Centre for Allied Health Evidence, Critical Analysis Tools

The ROBINS-I tool (Risk Of Bias In Non-randomized Studies - of Interventions)
by the Cochrane Methods Bias Group

SIGN Critical Appraisal Notes and Checklists
Includes checklists for systematic reviews, RCTs, and other types of studies.

Available online from the UB Medical Library 

Elwood M. Critical appraisal of epidemiological studies and clinical trials. Oxford University Press; 2017 Mar 1.

Available for borrowing from the UB Medical Library

Bootland D, Coughlan E, Galloway R, Goubet S, McWhirter E. Critical Appraisal from Papers to Patient: A Practical Guide. CRC Press; 2016 Dec 19.

PubMed complains not having retrieved any records for the keyword {example term}, even though the keyword is an entry term from the MeSH database or a synonym taken from the Emtree thesaurus. What happened?

Explanation 1: There might be a spelling error or a special character that might cause trouble. Perhaps you chose an expression which is not used in natural language ("heart bypass, left" instead of "left heart bypass"). 

Explanation 2: Everything is fine. Not all keywords will return a result, even when they are entry terms or synonyms taken from a thesaurus. The purpose of entry terms and synonyms is to support "automatic term mapping". It might just be the case that the keyword doesn't appear in any record. 

PubMed meldet, es habe keinen Treffer für das Stichwort {Beispielstichwort} gefunden. Es ist sogar ein Entry Term aus der MeSH-Datenbank bzw. ein Synonym aus Emtree! Was ist das Problem? 

Erklärung 1: Es ist ein Schreibfehler, es gibt Probleme mit Sonderzeichen oder Sie verwenden eine Schreibweise, die in natürlicher Sprache nicht vorkommt ("heart bypass, left" statt "left heart bypass"). 

Erklärung 2: Es ist alles in Ordnung. Es muss nicht zwangsläufig Suchtreffer für jedes Stichwort geben, selbst wenn Sie "Entry terms" aus der MeSH-Datenbank in [TIAB] suchen. Sinn der Entry Terms oder Synonyme sind vor allem das "automatic term mapping". Falls alles richtig geschrieben ist, kann es einfach sein, dass der Begriff wirklich nirgendwo in der Datenbank auftaucht

Is it ok to use more than one index term/subject heading per concept? 

By all means, yes! Please use as many relevant index terms as you can find. Any number of index terms can be employed for each concept and connected with "OR" (e.g. "urinary tract infections"[MeSH] OR "urinary bladder diseases"[MeSH] OR "urethral diseases"[MeSH]). It could also be that there is no suitable index term -- in this case you should focus on a comprehensive textword search. 

Darf ich mehr als ein Schlagwort pro Konzept haben? 

Unbedingt! Sie sollten so viele Schlagworte benutzen wie Sie finden können und für das Konzept passend sind. Sie können beliebig viele ("0 bis n") Schlagworte für ein Konzept verwenden und mit "OR" aneinanderhängen. (z.B. "urinary tract infections"[MeSH] OR "urinary bladder diseases"[MeSH] OR "urethral diseases"[MeSH]). Es kann auch vorkommen, dass es gar kein Schlagwort gibt -- in dem Fall machen Sie eine ausgiebige Stichwortsuche. 

I use PICO to divide my research question, but I find more/less than four concepts within my question. What's wrong? 

Probably nothing. PICO is just an aid, not a requirement. It is very possible that there is only a population and an intervention in your question, or perhaps entirely different concepts. It is also possible that the question contains concepts which are not helpful in your search strategy. Terms like "benefit", "outcome" or "effectiveness" are imprecise and it is hard to build a precise search with them. That's why some concepts are better excluded from the search. 

Ich finde für meine Frage mehr/weniger Konzepte als PICO vorgibt. Was mache ich falsch? 

Wahrscheinlich nichts. PICO ist nur eine Hilfestellung, es muss nicht jeder Teil vorhanden sein oder verwendet werden. Es ist völlig in Ordnung, nur eine Population und eine Intervention in der Frage zu haben, oder ganz andere Konzepte. Es kann auch sein, dass die Fragestellung Konzepte aufweist, die man besser nicht in der Suchstrategie einbaut. Begriffe wie "benefit", "outcome" oder "effectiveness" sind schwammig und daher schwer zu suchen und würden als eigenes Konzept nur die Suche unnötig behindern. In dem Fall ist es besser, sie weg zu lassen. 

Should I apply a date limit in order to reduce the number of search results? 

In most cases: No. Reducing the number of search results artificially by looking at the most recent publications is arbitrary and may introduce bias. What if you miss an important reference just outside your set date range? There might be reasons with regard to contents to apply a date limit, for instance literature about COVID-19 can be limited to 2019 and later.

Sollte ich einen Datumsfilter anwenden, wenn ich zu viele Treffer habe? 

In den meisten Fällen: Nein. Die Suchtreffer künstlich zu verringern, indem man nur die neueste Literatur sucht, ist sehr willkürlich und kann für eine Verzerrung der Ergebnisse sorgen. Was, wenn Sie wichtige Literatur nur ein Jahr vor Ihrem Datumsfilter verpassen? Es kann sinnvoll sein, wenn die zeitliche Beschränkung inhaltlich begründet ist, z.B. Literatur zu COVID-19 brauche ich nicht vor 2019 suchen. 

Is it wrong to truncate the text term {example term} or search it as a phrase? Is it okay to use a subheading for the index term {example term} or search it as a major topic? 

Literature research is not an entirely exact science. Nobody can tell you beforehand whether your choice of terms or the techniques you apply is the best way to do it. You should decide whether you need to build a broad search (e.g. for a master's thesis, a PhD thesis or a review) or a focused search (for a quick answer to a very precise question).  

Ist es falsch, wenn ich den Suchbegriff {Beispiel} trunkiere/als Phrase suche? Ist es okay, beim Schlagwort {Beispiel} ein Subheading oder Major Topic zu verwenden? 

Die Literaturrecherche ist keine ganz exakte Wissenschaft. Die Entscheidung, welche Suchbegriffe Sie verwenden und wie Sie ihre Suche thematisch einschränken, kann ihnen niemand abnehmen. Es kommt darauf an, ob Sie eine breite Suche (z.B. für eine Masterarbeit, Doktorarbeit oder ein Review) oder eine sehr fokussierte Suche (für eine sehr konkrete Fragestellung) durchführen möchten.  

There are no PDFs in PubMed. How do I get full texts?  

Usually, there is a full-text link (or the DOI) in the records, which should allow access when you are within the university network or connected to the VPN. Either you are directly taken to the journal website where you can try to access the PDF, because the university has a subscription for the journal, or you sometimes have to choose "institutional access" and authenticate yourself using your SWITCH edu-ID. You could also try using the browser plugin from https://unpaywall.org, which provides legal access to OpenAccess full texts, if there are any. If that doesn't work you can always order a copy of the article (for a fee) via the university library. See also: https://www.ub.unibe.ch/recherche/fachinformationen/medizin/systematic_searching/help/index_ger.html#pane706441 

Es gibt in PubMed keine PDFs zum Herunterladen. Wie komme ich an die Volltexte? 

In den meisten Fällen gibt es einen Link zum Volltext (oder den DOI), über den man Zugang zum PDF bekommt, sofern man sich im VPN oder Universitätsnetzwerk befindet. Entweder führt der Link direkt zur Zeitschrift, bei der das PDF heruntergeladen werden kann, sofern die Universität die Zeitschrift abonniert hat, oder es ist nötig, sich über "institutional access" mit der SWITCH edu-ID anzumelden. Es gibt auch ein Browser-Plugin auf https://unpaywall.org, das nach legalen OpenAccess-Volltexten sucht und diese anbietet. Falls das alles nicht geht, bleibt immer noch die Möglichkeit, den Artikel über die Universitätsbibliothek gebührenpflichtig zu bestellen. Siehe auch: https://www.ub.unibe.ch/recherche/fachinformationen/medizin/systematic_searching/help/index_ger.html#pane706441

Apparently, PubMed and Ovid have problems with some quotation marks. 

Sometimes certain quotation marks cause problems when entering a query, especially when using Apple devices, but it's hard to reproduce. These problems could be an encoding issue, perhaps when you use a sophisticated word processor (like MS Word) which tries to make the text 'look good' in some way, perhaps due to a particular localization, like switching to swiss or french guillemets (see https://en.wikipedia.org/wiki/Quotation_mark). As a workaround you should try using a plain text editor (e.g. notepad++) and copying it from there into the search box. 

Offenbar haben PubMed oder Ovid ein Problem mit manchen Anführungszeichen. 

Manchmal kommt es vor, dass es wegen bestimmter Anführungszeichen Fehler beim Start einer Suchanfrage gibt (besonders bei Apple Computern und Tablets), aber der Fehler ist bisher nicht gut reproduzierbar. Es könnten Probleme mit der Zeichenkodierung sein, z.B. wenn ein Textverarbeitungsprogramm wie Word versucht, den Text 'schön' aussehen zu lassen, oder die Anführungszeichen in regionale Varianten wie Guillemets umsetzt (siehe https://de.wikipedia.org/wiki/Anf%C3%BChrungszeichen). Wird dann der Text kopiert, kann die Suchoberfläche mit dem von Word verwendeten Sonderzeichen nichts anfangen. Die Verwendung eines simplen Texteditors (wie z.B. Notepad++), in dem die Suchstrategie entwickelt und herauskopiert wird, kann die Lösung sein. 

There is no subject heading/MeSH term for the drug {example drug}. What should I do? 

That's a common scenario; there are not always index terms for everything or the closest subject heading is still too broad. Go to https://swissmedicinfo.ch, https://drugbase.de, Drugs@FDA (https://www.accessdata.fda.gov/scripts/cder/daf/) or a similar website to look for drug brand names in order to find the active ingredient or other brand names for the same drug. Use these for building a textword search. 

Für das Medikament {Beispiel} gibt es kein Schlagwort bzw. keinen MeSH term. Was mache ich jetzt? 

Das ist durchaus möglich; es gibt nicht immer passende Schlagworte in der Datenbank oder das am nächsten liegende Schlagwort ist immer noch zu breit. Auf Seiten wie https://swissmedicinfo.ch, https://drugbase.de oder Drugs@FDA (https://www.accessdata.fda.gov/scripts/cder/daf/) lassen sich viele Präparatenamen für Medikamente auf den unterschiedlichen Märkten suchen. Dort sind in den meisten Fällen auch deren Wirkstoffe und darüber weitere Handelsnamen zu finden. Verwenden Sie diese in der Stichwortsuche. 

What is the right amount of retrieved records? (How many are too many or too few?) 

Any very broad research question will generate a lot of search results, whereas a very specific question may yield only a few hits. What is best for you depends on the kind of paper you want to write and the amount of records you are able to screen. In case you just need a short answer, a few (systematic) reviews will be enough for you. If you are the one writing a (systematic) review, you are looking for primary studies and should be ready to screen a few hundred or thousand records. 

Was ist eine gute Anzahl an Treffern für meine Suche? (Wann sind es zu viele oder zu wenige?) 

Eine sehr breite Fragestellung wird immer viele Treffer in den Datenbanken generieren, während es zu einer sehr spezifischen Frage häufig nur sehr wenig Literatur gibt. Wieviele Treffer 'gut' sind, hängt von der Art der Arbeit ab, die Sie schreiben wollen und von der Menge an Referenzen, die Sie noch in der Lage sind, zu screenen. Falls eine kurze Antwort benötigt wird, können ein paar (Systematische) Reviews ausreichen. Falls ein (Systematisches) Review geschrieben werden soll, benötigt man Primärstudien und muss in der Lage sein ein paar Hundert oder Tausend Referenzen zu screenen. 

What is the right filter for me?  

It depends on the kind of paper you want to write. If you need to find primary studies for a review, choose a filter for RCTs, CCTs or perhaps observational studies. If you need a quick answer or you want to write a review of reviews, better use a (systematic) review filter. 

Welcher Suchfilter ist der Richtige für mich? 

Das hängt ganz von der Arbeit ab, die Sie schreiben möchten. Wenn Sie Primärstudien für ein Review brauchen, sollten Sie einen Filter für RCTs, CCTs oder Beobachtungsstudien nehmen. Für eine schnellere Antwort zu einer präzisen Frage oder z.B. ein Review of Reviews ist ein Filter für (Systematische) Reviews sinnvoll. 

The research support services team of the University of Bern Medical Library offers individual research consultations, lectures, and workshops for students, researchers and clinicians to support them in literature searching.

Advisory services and individual research consultation

• Consultation and support for developing search strategies
• Feedback on framing the research question and advice on how to translate into a search strategy customized for each database/source to search
• Feedback on initial search strategy / Peer Review of search strategies
• Advice on relevant sources to search (medical databases, trial registries, grey literature)
• Referral to relevant handbooks and guidance documents for conducting literature reviews
• Setting up search alerts for new publications

Librarian assisted search services

Librarian assisted searches are for the purposes but not limited to:

• topic exploration
• research support
• quality improvement projects
• teaching/training
• presentations
• personal knowledge

For students, the medical information specialist will not conduct a complete literature search needed to write their master thesis or dissertation. Students are encouraged to attend courses and/or individual research consultations offered by the library.

Comprehensive literature searching services

The medical library supports researchers, clinicians and external clients in the design and conduct of thorough and complex literature searches for systematic or scoping reviews, HTAs, clinical guidelines. Services include:

• Consultation and guidance in various steps in conducting a systematic review
• Search concepts and strategy construction for initial search strategy
• Database selection; grey literature / unpublished literature choices
• First search strategy draft submitted to the author
• Search strategy revision / final approval
• Running the searches in all agreed sources
• Search completion
• Peer review of search strategies
• Results delivered in an agreed format
• Search documentation of search strategies, databases, limits, numbers of results (to be included in the appendix of a manuscript and in a PRISMA flowchart)
• Writing the search  methodology for the manuscript

Courses

The offered courses in literature searching and evidence-based medicine include:

• Curricular courses for the faculties of medicine, dentistry, and veterinary
• Customized courses to meet the needs of your institution, department or group

For any course offers in literature searching the research support services team can be contacted by email: support_med.ub@unibe.ch

Academic Writing Courses

Toolkit: How to Write a Great Research Paper using Reporting Guidelines (EQUATOR Network)

Guidance on Scientific Writing (EQUATOR Network)

University Library Courses including Reference Management / Literaturverwaltung

In many cases, you can link directly to an article from a database while you are on the UniBern network.

If not: Start your search for an article in the swisscovery catalog. For online available resources choose "Unibe & PH plus" or search directly in e-Journals (via Browzine).

Interlibrary Loans and Document Delivery

Open Access Department
Open Access publications are freely available online. Your funders may require you to publish in an open access journal to making your work accessible to all.

Check your publisher's copyright and self-archiving policies using SHERPA/RoMEO or contact your publisher.

BORIS is the institutional repository of the University of Bern and the Bern University Hospital. 

Research Data Management Department

Tools und Programme für das Studium und die Forschung (scroll to the bottom of the page)