Screening is the process of deciding which studies are to be included in the systematic review. Criteria should be pre-determined, unambiguous, transparent, consistently applied and reported in the final study. Inclusion criteria typically include:
The screening process should be reported in the final publication, as described in the PRISMA statement.
Covidence is a program which greatly facilitates the screening process. It is available to RMH, RWH & PMCC staff - request access here (please use organisational email).
Visit Covidence Academy for guides and tips on using Covidence and other aspects of doing a systematic review..
Once screened for inclusion the full text of articles must be critically assessed. The assessment should be sufficiently rigorous to allow substantive and well supported descriptions of the quality of evidence to be made.
See our Critical Appraisal guide for more information.
The use of critical appraisal tools (checklists, etc) is recommended to help maintain a systematic approach. See this link for examples.
Meta-evidence blog - Assessing and addressing bias in systematic reviews
Critical appraisal is the process of systematically analysing research to assess methods, validity and usefulness.
The key questions in critical appraisal are:
Why was the study done?
A clearly focused question should address population, intervention and outcomes.
What type of study was done?
The study design must match the question asked. Intervention questions are best answered with randomised controlled trials.
What are the study characteristics?
Use the PICO question format to help you answer this question.
What was done to address bias?
a) Was the assignment of patients to treatments randomised?
b) Were patients, health workers and study personnel ‘blind’ to treatment allocation?
c) Were all of the patients who entered the trial properly accounted for at its conclusion? Look for follow-up tables and whether patients were analysed in the groups to which they were randomised.
d) Were the groups similar at the start of the study?
e) Aside from the experimental intervention, were the groups treated equally?
What are the results and are the results valid?
a) Are the outcome measures relevant?
b) How large was the treatment effect?
c) How precise was the estimate of the treatment effect? Look for confidence limits and p values.
What conclusions can you make?
a) Can the results be applied to my patients? Were all clinically important outcomes considered? Are the benefits worth the harms and costs?
b) Are the results relevant to my situation? Patient population / similar definitions/protocols / health system similarities