META ANALYSIS

Partini Pudjiastuti
Child Health Department
Faculty of Medicine University of Indonesia

Background
Facts: INFORMATION EXPLOTION

For most clinical problems/public health issues there is

an overwhelming amount of existing information, as well

as new information produced every year

However, much of this information

isn't very good (= poor quality)
is derived from different methods & definitions (= poor
standardization)
is often contradictory (= heterogeneity)

So how should we go about summarizing medical

information?

. . . the mass of new information makes it difficult

for practicing physicians to follow the literature in

all areas that might be relevant to their practices.
New methods to synthesize and present
information from widely dispersed publications
are needed
. . . .
Jerome Kassirer. Clinical trials and meta-analysis: what do they
do for us? N Engl J Med 1992; 327:273-4.

How do we summarize
medical information?
Traditional Approach
Expert Opinion
Narrative review articles (Literature review)
Consensus statements (group expert opinion)

New Approach (Meta-analysis)

Explicit quantitative synthesis of ALL the evidence

Differences between narrative and systematic reviews

Feature

Narrative Review

Systematic Review

Topic Typically broad-scoped

Focused research question

Data sources and The search strategy and

search strategy databases that were used may
not be provided
Authorship A recognized expert(s) on the
topic
Article selection Typically not specified
criteria

A team of experts having methodologic

and clinical expertise
Consistently applied inclusion and
exclusion criteria

Searching May be extensive, intended to

locate literature on the topic
area in question
Appraisal of Indefinite, may be variable
included articles
Synthesis A qualitative summary is
usually provided

Extensive, intended to locate all

primary studies on a particular research
question
Critical appraisal is meticulous,
typically involving the use of
data extraction forms
A qualitative summary is provided,
quantitative when the data can be
pooled

Inferences Sometimes evidence-based

Evidence-based Chiropractic

The search strategy is explicit and

comprehensive with a list of all
databases that were utilized

Usually evidence-based

The three types of

literature review designs
Narrative Review

Selective review of the literature that broadly covers

a specific topic.
Does not follow strict systematic methods to locate
and synthesize articles.

Systematic Review

Utilizes exacting search strategies to make certain

that the maximum extent of relevant research has
been considered.
Original articles are methodologically appraised and
synthesized.

Meta-analysis

Quantitatively combines the results of studies that

are the result of a systematic literature review.
Capable of performing a statistical analysis of the
pooled results of relevant studies.

Meta-Analysis
Definition:
A specific subset of systematic reviews that

statistically combine data from many studies in

order to find a common effect.

Meta-analysis refers to the analysis of analyses

the statistical analysis of a large collection of

analysis results from individual studies for the
purpose of integrating findings (Glass, 1976).

Meta-Analysis
Definition:
Meta-analysis is a name that is given to any

review article in which the results of several

independent studies are combined statistically
to produce a single estimate of the effect of
a particular intervention or health care
situation (Jadad, 1998)
Considered as a true investigation:
observational - retrospective study

Review, Systematic review & Metaanalysis

Review
Systematic
review

Meta-analysis

Meta-Analysis
Initially developed in social sciences in mid-1960s
Adapted to medical studies in early 1980s
Initially applied to RCTs esp. when individual

studies were small and under-powered

Also applied to observational epidemiologic studies
often with little fore-thought which generated
much controversy
Explosion in the number of published metaanalyses in the last 10-15 years.

10

Goals & Objectives of a Metaanalysis

1. to summarize a large and complex body of literature on
2.
3.
4.
5.

a topic
to resolve conflicting research reports in the literature
to clarify or quantify the strengths and weaknesses of
studies on a topic
to document the need for a major clinical trial
to avoid the time and expense of conducting a clinical
trial

Goals & Objectives

6. to increase statistical power by combining many smaller studies:
Statistical power refers to the probability of detecting a finding (e.g

clinically important difference) of certain size if one truly exists

combining studies provides a larger sample increase in statistical
power
Increase the evidence for, or confidence in, a finding/conclusion

7. to improve the precision of an estimated treatment effect:

Precision refers to the degree of accuracy in the estimation, usually

measured by the reciprocal of the variance or standard deviation of the

estimate (i.e. Precision =1 / standard deviation)

Goals & Objectives

8. to detect smaller treatment effects that have been

reported
9. to investigate variations in treatment effects through
subgroup (or stratified) analysis
10. to investigate or improve the generalizability (external
validity) of known treatment effects

Hierarchy of studies

How to create a meta

analysis
Require careful planning, and adequate

resources (time = $$$)

Need to develop study protocol
INTRODUCTION
Background

/ Justification
Research question(s)
Objectives
Hypothesis
METHODS

Steps in meta-analysis

1. Identification:

Formulate question
Develop search strategy
Searching relevant studies

2. Selection:

Select eligible studies and assess quality

Choosing variables (comparison & outcomes)

3. Abstraction:

Extract data and calculate individual summary measures

4. Analysis:

Quantitative pooling
Sensitivity analysis
Investigate heterogeneity & biases

5. Write up:

Clear presentation

1. IDENTIFICATION
The research question must be clearly defined:
Defined patient population
Clear & consistent definitions of the disease,

interventions, and outcomes

A carefully defined search strategy must be

used to detect and prevent publication bias

The authors must cite where they looked and

should be exhaustive in looking for unpublished

studies

IDENTIFICATION - Sources
M-As use systematic, explicit search procedures (cf.

qualitative literature review)

MEDLINE

4100 journals
1966 - present

Web search at PubMed: http://www.ncbi.nlm.nih.gov/PubMed

other search engines: BRS Colleague, WinSPIRs, etc

EMBASE

similar to MEDLINE, European version

Expensive, not widely available in US

Identification - sources
Cochrane Collaboration Controlled Trials Register
Over 160,000 trials, including abstracts (+ translations)
by subscriptions.. MSU Electronic Library database
includes
MEDLINE,

EMBASE
non-English publications
non-indexed publications
hand-search of journals

Other MEDLARS

CancerLit, AIDSLINE, TOXLINE, Dissertation Abstracts Online

Index Medicus
important if searching before 1966
hand-search only

Identification - steps
Search own personal files
Search electronic databases
Review titles and on-line abstracts to eliminate irrelevant
Retrieve remaining articles, review, and determine if meet

inclusion/exclusion criteria

Review reference lists of articles for missed references

Consult experts/colleagues/companies
Conduct hand-searches of non-electronic databases and/or relevant journals
Consider consulting an expert (medical librarian) with training in MEDLINE

and use of MeSH terms.

Identification
find relevant study

SEARCH FOR EXISTING REVIEWS

FIND PUBLISHED PRIMARY STUDIES

Break down research question into components

Use appropriate synonyms
Use electronic databases, hand searching, etc
LOOK FOR UNPUBLISHED PRIMARY STUDIES

(write to experts, search registries for

completed/ongoing trials)

Other search issues

Non-English Studies
MEDLINE
Translation

of title usually provided but abstracts often not.

No a priori justification for excluding non-English studies

Quality

is often equivalent or even better!

Excluding non-English studies can effect conclusions
But including means you need a translation just to determine

eligibility!

Other search issues

Fugitive Literature
unpublished

studies ( why are they unpublished?)

dissertations
drug

company studies

book

chapters

non-indexed
conference

studies and abstracts

proceedings

government

reports

pre-MEDLINE

(1966)

2. SELECTION
TYPICAL INCLUSION CRITERIA:
study design (e.g., RCTs?, DBPC?, Cohort & CCS?)
setting (emergency department, outpatient, inpatient)
age (adults only, > 60 only, etc)
year of publication or conduct (esp. if technology or typical dosing

changes)
similarity of exposure or treatment (e.g., drug class, or dosage)
similarity of outcomes (case definitions)
minimum sample size or follow-up
languages?
complete vs incomplete (abstracts)
published vs fugitive?
pre-1966?

Selection Other
Issues.
Multiple publications from same study?
Include only one!
Selection process should be done independently by

at least 2 reviewers
Measure agreement (K) and resolve

discrepancies

Document excluded studies and reasons

for exclusion

Keep pertinent but excluded studies

Keep some, throw out

Cannot
others
include all studies
Keep the ones with
high levels of evidence
good quality
check with QUOROM guidelines

Usually, meta-analysis is done with RCTs

Case series, and case reports definitely out
Selection problems are major problems

Quality Control in MA:QUOROM Table

Detailed

Guidelines

A Good Checklist
Use it for reporting
Meta Analysis
Systematic reviews

Flow chart of Study retrieval

and selection
Non-steroidal anti-inflammatory drugs,
ncluding cyclo-oxygenase-2 inhibitors,
in osteoarthritic knee pain:
meta-analysis of randomised
placebo controlled trials

3. ABSTRACTION
Goal: to abstract reliable, valid and bias free information from all

written sources
Should expect a degree of unreliability
intra- and inter- rater reliability is rarely if ever 100%!!

Many sources of potential error:

Article may be wrong due to typographical or copyediting errors
Reported results can be misinterpreted
Errors in data entry during abstraction process

The weights are chosen to reflect the amount

of information that each trial contains.

Weighting

Give more weight to the more informative

studies. Weight by:

Size (sample size (n))

Event rate
Homogeneity (inverse of the variance)
Quality
Other factors

Abstraction
Typical process

2 independent reviewers
Practice with 2 or 3 articles to

calibrate
Use a 3rd reviewer or consensus
meeting to resolve conflicts
Measure agreement (K) and resolve
discrepancies

Abstraction investigator
bias
Abstractor may be biased in favor of (or against!)

a particular outcome (positive or negative

finding), or researcher/institution, or journal.

Prominent journals may be given greater weight

or authority (rightly or wrongly)

If this may be an issue, have research assistant

eliminate identifiers from articles (= blind

review)

Abstraction blind review

Remove study information that could affect inclusion or

quality of abstraction, like:

author, title, journal, institution, country

Berlin (97):
compared blinded vs non-blinded reviews
Found discrepancy in which studies to include but

little difference in summary effect sizes

Time consuming
Probably can avoid esp. if use well defined abstraction

procedures

Abstract Data:
Create a spreadsheet (Excel)
For each study, create the following columns:
name of the study
name of the author, year published
number of participants who received intervention
number of participants who were in control arm
number who developed outcomes in intervention
number who developed outcomes in control arm

Spreadsheet Data
We created seven
columns:
trial: trial identity code
trialname: name of trial
year: year of the study
pop1: study population
deaths1: deaths in study
pop0: control population
deaths0: deaths in control

4. ANALYSIS
Many techniques
Still controversial
Ask expert (a statistician) in M-A
Focus on variability (heterogeneity)

Analysis - heterogeneity
What is Heterogeneity?
No two studies are exactly the same! Heterogeneity (or Diversity) is
the extent to which individual studies vary from one another.
Types of Heterogeneity
1. Clinical (for example, studies with different age
criteria for enrollment)
2. Methodological (variability in design for example,
parallel group vs. crossover placebo controlled trial)
3. Statistical (observed differences in treatment effects among
studies are greater than those one would expect by chance alone)

Analysis - heterogeneity
Causes of heterogeneity
Differences between studies with respect to:
Patients: diagnosis, prognostic factors, in- and exclusion
criteria, etc.
Interventions: type, dose, duration, etc.
Outcomes: type, scale, cut-off points, duration of follow-up,

etc.
Quality and methodology: randomised or not, allocation

concealment, blinding, analysis, etc.

How to Deal With

Heterogeneity

Ignore it and use fixed effects model (not

recommended)
Test for it and do not pool results if studies
are significantly heterogeneous
Incorporate it:
assume that heterogeneity is due to
random differences among studies whose
sources can not be identified
use random effects model

Consequences of
heterogeneity
When the results of studies in a meta-analysis are

inconsistent, it reduces confidence in its conclusions

The meta-analysis may actually be worthless if too
dissimilar
For instance, combining studies that used different types
of comparison groups
Or outcomes that were dissimilar

ANALYSIS
Focus on variability (heterogeneity)
Fixed effects model
o only intra-study, ignore interstudy var
o narrower CI
Random effects model
o considers also inter-study var
o broader CI

Fixed Effects or Random Effects?

Fixed Effects Model

Random Effects Model

Conduct if it is reasonable to assume

Conduct if test of heterogeneity is

underlying Rx effect is SAME for all

studies
Pooling: Mantel Haenszel OR
Test: test of heterogeneity
If significant, go for random effects
model
Short 95% CI for summary
Smaller summary estimate
OR=0.77 [0.72,0.83]

significant (shows heterogeneity)

Assume that TRUE log odds ratio
comes from a normal distribution
Method: DerSimonian Lairs
method (DSL) of calculating Odds
Ratio
OR=0.78 [0.69,0.88]

Meta-analysis (Forest)
plot

Are the results similar across studies? 3 tests

1. Eyeball test do they look the same?
2. Test of Null hypothesis of no variation (p-value)
3. Proportion of variation not due to chance (I2)

Statistical measures of heterogeneity

The Chi2 test measures the amount of variation
in a set of trials, and tells us if it is more than
would be expected by chance
Small p values suggest that heterogeneity is
present
This test is not very good at detecting
heterogeneity. Often a cut-off of p<0.10 is used,
but lack of statistical significance does not mean
there is no heterogeneity
44

Statistical measures of heterogeneity (2)

I2 is the proportion of variation that is due to
heterogeneity rather than chance
Large values of I2 suggest heterogeneity
Roughly, I2 values of 25%, 50%, and 75% could
be interpreted as indicating low, moderate, and
high heterogeneity
(Higgins JPT et al. Measuring inconsistency in meta-analyses. BMJ .
2003;327:557-60.)

45

Test of sensitivity:
Was the result robust?
Random & fixed effects: similar
Exclusion of low weighted studies: similar
? Publication bias: larger sample size gave smaller
effect size
Exclusion of special cases, e.g. incomplete follow-up

Subgroup analysis
Meta-analyses typically include patients

with a variety of characteristics

e.g., age, gender, condition severity,
patient history, etc.
Patients in these subgroups may respond
to treatment differently
e.g., low back pain patients with leg pain
may respond to treatment differently
than low back pain only patients

Subgroup analysis
(cont.)

Carried out to identify variation between

patient groups regarding certain outcomes

or findings
The process helps readers to distinguish
the effects of a treatment between
subgroups
The statistical power of the subgroups will
decline as a result

Analyse Data
Statistically

Combine data to arrive at a summary, 3 measures

Effect Size (Odds Ratio)
Variance with 95% Confidence Interval
Test of Heterogeneity

Two Graphs
Forest Plot
Funnel Plot

Examine why the studies are heterogeneous, if they

are
Use Statistical Packages, several choices

Summary Estimates
Mantel Haenszel OR=0.77
95% Confidence Interval
[0.72, 0.83]

Test of Heterogeneity:
Chi-square (df=21) = 31.5
P Value = 0.07

The pooled Odds Ratio shows

that those receiving
streptokinase at AMI are about
77% at risk of death (23% less
likely to die)
That in 95 out of 100 such
meta analyses, the pooled
Odds Ratio would lie between
0.72 and 0.83, indicating a
statistically significant
protective effect
That these studies were not
significantly heterogeneous

Study Results: usually presented in graphs

Forrest Plot

Synonyms: metagraph, bobblogram

5. Write up study results

For numeric outcome
Presented as mean difference for individual

study with corresponding 95% CI

Represented by square, the area = relative
weighted value
Summarized as combined mean difference with
CI as diamond
Vertical line: mean diff = 0 (no effect)
Vertical dotted line: if CI crosses this line:
means homogenous; otherwise: heterogeneous

Meta-analysis: Example of graph presentation

of RCTs with numerical outcome (xe-xc)
Study I
Study II
Study III
Study IV
Study V
Study VI
Study VII1
Study VIII

1992
1994
1995
1995
1996
1997
1999
2000

Combined
-0.3

Favor drug

Xe-xc=0

+0.3

Favor placebo

Study results
For nominal outcomes:
Presented as OR for individual study with

corresponding 95% CI

Represented by square, the area = relative

weighted value

Summarized as combined OR with CI as diamond

Vertical line: OR = 1 (no effect)
Vertical dotted line: if CI crosses this line: means

homogenous; otherwise: heterogeneous

Meta-analysis: Example of graph presentation

of RCTs with nominal outcome
Study I
Study II
Study III
Study IV
Study V
Study VI
Study VII1
Study VIII

1992
1994
1995
1995
1996
1997
1999
2000

Combined
0.1

10
OR = 1

Favor drug

Favor placebo

Forest plot
The label tells you what the comparison and

outcome of interest are

Forest plot
Each study has an ID

Forest plot
The data for each trial are here, divided into

experimental and control groups

Forest plot
Results from each trial are also given

numerically

Forest plot
This is the % weight given to each study in

the pooled analysis

Forest plot
The size of the block for each study is

proportional to the % weight. The horizontal

line represents the confidence interval.

Forest plot
The label above the graph tells you which

statistic has been used

Forest plot

The horizontal line at the bottom is the

scale measuring the treatment effect.

Here, towards the left, the scale is less
than one, meaning risperidone has
made people less likely to leave the
study early

Forest plot
The vertical line in the middle is where

treatment and control have the same

effect there is no difference between
the two

Forest plot

Take care to read the labels things to

the left do not always mean the

treatment is better than the control

Forest plot
The pooled analysis is given a diamond

shape the widest bit is located on the

point estimate and the horizontal width
in the confidence interval. Does the
confidence interval cross the line of no
effect?

Forest plot
Results of the pooled analysis are also given

numerically

Forest plot
The result of a Z test with a p value are also

given for the overall effect

Forest plot
The forest plot displays the results of the Chi-

square test and the p value

Publication bias
Published studies are not representative of all studies
that have been performed
Articles with positive findings (P < 0.05) are more
likely to be published
Hence published studies are a biased sub-set
Publication bias = systematic error of M-A that results
from using only published studies

Evidence of Publication Bias

Easterbrook (1991): 285 analyzed studies reviewed by Oxford
Ethics Committee 1984-87

Study Status

% (P < 0.05)

Published

138

67%

Presented Only

69

55%

Neither

78

29%

Total

285

54%

Implications of Publication Bias

Simes (1986): Chemotherapy for Advanced Ovarian CA
Comparison of Published Trials vs Registered Trials

Results

Published

Registered

16

13

1.16

1.06

95% CI

1.06 1.27

0.97 1.15

P value

0.02

0.24

N
Median Survival
Ratio

Funnel Plots
Funnel plots are a device for checking for publication bias.
Each dot represents the overall effect
from one RCT.
As sample size increases, the width of
the confidence interval should
decrease.
Result should be located in a
symmetric, triangular area centered on
the overall effect for all studies.

Funnel Plots
Missing studies will manifest as an asymmetry in the
funnel plot.
Missing studies will appear
as a gap in the portion of the
funnel plot where you would
expect to find negative
studies.
The unopposed positive
studies will shift the
apparent treatment effect
(blue line) towards a larger
effect size than it really is.

Cumulative Meta-analysis

75

Case study: Is passive smoking

harmful?
A topic of great debate and controversy for many years
First few epidemiologic studies were published in 1918
Vigorously attacked by the tobacco industry
Too small an association
Potential bias
Potential confounding
Lack of biological proof

Evidence accumulated over the next 2 decades

It was not until about 10 years ago when several
official bodies/agencies concluded that passive
smoking is a cause of lung cancer
The tobacco industry continues to dispute this claim!!

Hackshaw AK et al. BMJ 1997;315:980-88.

Hackshaw AK. Stat Meth Med Res 1998;7:119-136.

Is passive smoking harmful?

Hackshaw et al. conducted a very
comprehensive systematic review in 1997:
They identified 37 published studies that reported
risk of lung cancer among lifelong non-smoking
women according to the husbands smoking
status
Their meta-analysis revealed that the overall risk
of lung cancer among lifelong non-smoking
women was 1.24 times higher when their
husbands smoked, as compared to those women
whose husbands did not smoke.
Hackshaw AK et al. BMJ 1997;315:980-88.
Hackshaw AK. Stat Meth Med Res 1998;7:119-136.

Is passive smoking harmful?

Passive
smoking and
lung cancer
review

Hackshaw AK et al. BMJ 1997;315:980-88.

Hackshaw AK. Stat Meth Med Res 1998;7:119-136.

Is passive smoking harmful?

Hackshaw AK et al. BMJ 1997;315:980-88.

Hackshaw AK. Stat Meth Med Res 1998;7:119-136.

Total 37 studies

Is passive
smoking
harmful?
Sub-group analysis

Hackshaw AK et al. BMJ 1997;315:980-88.

Hackshaw AK. Stat Meth Med Res 1998;7:119-136.

Criticism of Meta-analysis
The file drawer problem/Publication bias: There is always a high

likelihood for studies that did not achieve statistical significance not
to be published. This is called the file drawer problem because it is
assumed that this studies are filed away somewhere and not
accessible to the public. Therefore statistically significant studies
are more likely to be included in the meta-analysis which will result
in an overestimate of the treatment effect.
Potential solutions:
if possible contact all persons known to work in the field to
inquire about studies done, but not published
estimate how many studies it will take to reduce the treatment
effect to non-significance

Criticism of Meta-analysis
The Simpsons Paradox: A reversal in the direction of the

relationship that occurs when different data from different

sources are combined
Confounding variables: There is always potential for
differences across studies that may be confounded with
treatments used. For example, if studies in a meta-analysis
were done in different countries, cultural differences may be
confounded with treatment differences.
Subtle differences in treatment with same name

Criticism of Meta-analysis
Biased or flawed studies: If the studies in the meta-analysis are

flawed or biased, so are the results! GIGO

Statistical significance versus practical / clinical significance: Metaanalysis is likely to find statistical significance because of the
increase in power as the sample size increases. Must evaluate the
magnitude of the treatment effect to assess clinical significance
False findings of no difference: A statistically non-significant result
may also be found because one does not have enough data (or
power). But one must remember that no evidence of effect is NOT
evidence of no effect
Rule of Thumb: Always determine the sample size whenever no

difference or no relationship is found in a study.

Evaluating metaanalysis

Validity
Importance
Applicability

Critical Appraisal of a MetaanalysisGUIDE

COMMENTS
Did the meta-analysis address a
focused question?

Is the main question clear from

the title or abstract?

Were the criteria used to select

Specify the subjects, exposures,
articles for inclusion appropriate? and outcomes? Specify the
methodological standards used
to select studies?
Is it likely that important relevant
studies were missed?

Authors conducted a thorough

search? Identify unpublished
studies?

Critical Appraisal of a Metaanalysis

GUIDE

COMMENTS

Was the validity of the included

studies appraised?
Were the assessments of
studies reproducible?

Did two or more people

participate to select studies?

Were the results similar from

study to study?

Tests of Homogeneity?

What are the overall results of

the meta-analysis?

Weighted studies according to

their size? RR and the absolute
risk reduction? Likelihood
Ratios? ORs? Effect Size?

Critical Appraisal of a Metaanalysis

GUIDE
COMMENTS
How precise were the results?

Can the results be applied to my

patient care?
Were all clinically important
outcomes considered?
Are the benefits worth the harm
and costs?

Confidence interval around the

estimate?

Thank you

Funnel Plots
Treatment effects (for eg. lnOR, lnRR, RD) is plotted
against sample size (or other measures of precision such
as standard error or variance)
5000

10000

Sample size (log scale)

Sample size

4000
3000
2000
1000
0
0.01

1000

100

10

0.1

Relative risk (log scale)

10

0.01

0.1

Relative risk (log scale)

10

Funnel Plot
Plots the effect size against
the sample size of the study
To study a funnel plot, look at
its LOWER LEFT corner, thats
where negative or null studies
are located
If EMPTY, this indicates
PUBLICATION BIAS

Graphical approach: Funnel

Plots
Scatterplot of effect estimates against sample size
Used to detect publication bias
If no bias, expect symmetric, inverted funnel
x
x
x
x x x

x
x
x x x

If bias, expect asymmetric or skewed shape

x
x

x
x x x
x x x x

Graphical Approaches - Funnel plot

Missing studies = small effects size with
negative findings
X

X
Sample
Size
(precision)

X
X
X

X
X
X

X
X

X
X

Effect Size

X
X

X
X

Publication bias Approaches

1. Attempt to Retrieve all Studies
Required for Cochrane Publications
Difficult to identify unpublished studies and then to find out details about
them

Worst Case Adjustment

Number of unpublished negative studies to negate a positive metaanalysis:
X = [N x (ES) / 1.645]2 - N
where: N = number of studies in meta-analysis,
ES = effect size

Example:
If N = 25, and ES = 0.6 then X = 58.2
Almost 60 unpublished negative studies would be required to negate the
meta-analysis of 25 studies.

Growth of Meta-analysis
3000

Number of Publications

2500

2000

1500

1000

500

0
93-94

94-95

95-96

96-97

97-98

98-99

99-00

2000-1

2001-2

2002-3

2003-4

Year of Publications

meta-analysis as publication type from PubMed, from years 1993 through 2004

Other search issues

Fugitive Literature
Sometimes important sources of information
Hard to track down contact experts/colleagues
Need to decide whether to include or not - general
consensus is that you should.

Example
Let's say we want to know whether
streptokinase is protective for death
from acute myocardial infarction. How
should we set up a search strategy?

The Search
streptokinase[text word] OR acute
myocardial infarction[text word] produces ALL
articles that contain EITHER streptokinase OR
acute myocardial infarction anywhere in the
text inclusive, many
streptokinase [text word] AND acute
myocardial infarction [text word] will capture
only those subsets that have BOTH
streptokinase AND acute myocardial
infarction anywhere in the text restrictive,
few

Summary Estimates
Mantel Haenszel OR=0.77
95% Confidence Interval
[0.72, 0.83]

Test of Heterogeneity:
Chi-square (df=21) = 31.5
P Value = 0.07

The pooled Odds Ratio shows

that those receiving
streptokinase at AMI are about
77% at risk of death (23% less
likely to die)
That in 95 out of 100 such
meta analyses, the pooled
Odds Ratio would lie between
0.72 and 0.83, indicating a
statistically significant
protective effect
That these studies were not
significantly heterogeneous