Meta-analysis and systematic reviews

Kamila Kholmatova
2013
What is a systematic review?
 A systematic review
• This is a systematic collection of the relevant
primary papers in human populations

• that deal with a focused question

• and includes a summary of the evidence from

the primary sources.
 Systematic reviews have
increasingly replaced traditional
reviews and expert
commentaries as a way of
summarising
research evidence
Systematic vs traditional reviews

• Traditional reviews may be called:

- literature reviews,
- narrative reviews,
- critical reviews,
- commentaries within the literature.

• Although often very useful background

reading, they differ from a systematic review
in that they are not led via a peer-reviewed
protocol and so it is not often possible to
replicate the findings.
 A systematic review
• This is in contrast to an overview which
often tackles a whole topic rather than a
focused question,
• may include opinion based articles
• and whose summary may be influenced by
clinical experience or evidence from other
sources such as animal experiments.
 Procedures of a high quality
systematic reviews
• Identify all relevant published and
unpublished evidence
• Select studies or reports for inclusion
• Assess the quality of each study or report
• Synthesise the findings from individual
studies or reports in an unbiased way
• Interpret the findings and present a
balanced and
• impartial summary of the findings with due
consideration of any flaws in the evidence.
 Systematic reviews
• may examine quantitative or
qualitative evidence;
• put simply, when the two or more types
of evidence are examined within
• one review it is called a mixed-method
systematic review.
What is a meta-analysis?
 Meta-analysis
A particular type of systematic review
that uses quantitative methods to
combine the results from a number of
studies
 Meta-analysis
Meta analysis is a statistical anlysis
which combines the results of several
independent studies considered by the
analyst to be combinable.

Huque 1988
 Meta-analysis
• Through such a procedure, effects
which are hard or impossible to discern
in the original studies because of a too
small sample size can be made visible,
• as the meta-analysis is (in the ideal
case) equivalent to a single study with
the combined size of all original studies
 Meta-analysis
• Since meta-analysis is a retrospective look at
data, it is important to make the process
rigorous and well defined to prevent
opportunities for bias to distort the results.
• Only in this way can it achieve the status of a
scientific discipline.
• This necessitates blinding the selection of
papers, extraction of data and quality
assessment in duplicate following an
established protocol at the start of the study.
Procedures of a high-quality MA
1. all studies included in a meta-analysis must fulfill
predetermined criteria (protocol);
2. All must have used essentially the same or closely
comparable methods and procedures;
3. the populations studied must be comparable;
4. the data must be complete and free of biases -
such as those due to selection or exclusion criteria.
5. the raw data from all studies is usually reanalyzed,
partly to verify the original findings from these
studies, and partly to provide a database for
summative analysis of the entire set of data.
6. All eligible studies must be included in the meta-
analysis
7. to increase a statistical power of data
Evaluation of the results in MA
– Look at the Relative Risk (RR) of the
main outcome in the two groups.
– In a meta analysis this is usually
presented as a diamond with the 95%
confidence intervals grouped about a line
indicating the null hypothesis.
– A number of outcomes may be
considered at once so more than one
diamond may be presented.
Meta-analysis
Critical appraisal of
the meta-analysis?
Critical Appraisal of a meta-
analysis methodology
• Selection bias
• Generalisability
• Combinability
• Consistency
• Statistics
• Sensitivity analysis
 Sensitivity analysis
• Can be done using quality score of trials
e.g. published vs unpublished,
observational and RCTs, details of
randomisation
• Publication bias - i.e. how many studies
showing no difference would have to
exist but not published to invalidate my
results?
What is Grey literature?
 What is Grey literature?
• a body of materials that cannot be found easily
through conventional channels such as publishers,
"but which is frequently original and usually recent1

• information produced on all levels of government,

academics, business and industry in electronic and
print formats not controlled by commercial publishing
i.e. where publishing is not the primary activity of the
producing body2

1.M.C. Debachere, 1995

2. Luxembourg, 1997
Grey and unpublished Literature
• A comprehensive overview should also look for grey and
unpublished literature. Studies with negative findings may
be difficult to get published.

• If only studies with positive results are published then a

meta-analysis of the published papers will give a positive
result.

• Suggest contacting researchers known to be active in the

field of interest. Also drug companies often have
unpublished studies which they are now releasing to
Cochrane and other interested parties.
http://library.brooklyn.cuny.edu/access/greyliter.htm
Thank you for your attention