You are on page 1of 33

HOW TO APPRAISE JURNAL

ON PROGNOSIS
DEDDY NW ACHADIONO
CLINICAL SCENARIO
• Man with a history of a stroke who is concerned
about his risk of seizure
Are the results of this study valid?
• Information about prognosis can come from a variety of
study types.
• Cohort studies are the best source of evidence about prognosis.
• Randomised control trials can also provide information about
prognosis although trial participants may not be representative of
the population with the disorder.
• Case-control studies are useful when the outcome of interest is
rare or when the required follow-up is long. The strength of
inference that can be drawn from a case-control study is limited.
• You see a 70 year old man in your outpatient clinic 3 months after he
was discharged from your service with an ischemic stroke.
• He is in sinus rhythm, has mild residual left-sided weakness but is
otherwise well.
• His only medication is ASA and he has no allergies.
• He recently saw an article on the BMJ website describing the risk of
seizure after a stroke and is concerned that this will happen to him.
• Our search of the literature to answer this question retrieved an
article from the BMJ (1997;315:1582-7).
Was a defined, representative sample of patients assembled at a
common (usually early) point in the course of their disease?

• We hope to find that the individuals included in the study are


representative of the underlying population (and reflect the spectrum
of illness).
• But, from what point in the target disorder should patients be
followed? Above, we state 'usually early' implying an inception cohort
(a group of people who are assembled at an early point in their
disease), but clinicians may want information about prognosis in later
stages of a target disorder.
• Thus, a study that assembled patients at a later point in the disease
may provide useful information. However, if observations are made at
different points in the course of disease for various people in the
cohort, the relative timing of outcome events would be difficult to
interpret.
• The ideal cohort is one in which participants are all at a similar stage
in the course of the same disease.
Table 1
Frequency of onset seizures and development of post stroke
seizures by type of first stroke

Number (%) of patients


No (%; 95% CI) of
with onset seizures who
Type of first stroke patients with onset
developed post stroke
seizures
seizures
Cerebral infarction (n=545) 10 (2; 0.9 to 3.4) 4 (40)
Primary intracerebral haemorhage
2 (3; 0.4 to 10.5) 1 (50)
(n=66)
Subarachnoid haemorrhage (n=33) 2 (6; 0.7 to 20.2) 0
Unknown (n=31) 0 (0; 0 to 11.2) 0
Total (n=675) 14 (2; 1.1 to 3.5) 5 (36)
• Returning to the paper we found, the study included patients who
were entered after their first stroke. Further details on entry
procedures aren't included in the study.
Was patient follow-up sufficiently long and
complete?
• Ideally, we'd like to see a follow-up period for a study that
lasts until every patient recovers or has one of the other
outcomes of interest, or until the elapsed time of
observation is of clinical interest to clinicians or patients.
• If follow-up is short, it may be that too few study patients
will have the outcome of interest, thus providing little
information of use to a patient.
• The more patients who are unavailable for follow-up, the
less accurate the estimate of the risk of the outcome.
• Losses may occur because patients are too ill (or too well) to
be followed or may have died, and the failure to document
these losses threatens the validity of the study.
• Sometimes, however, losses to follow-up are unavoidable
and unrelated to prognosis.
• Although an analysis showing that the baseline
demographics of these patients are similar to those followed
up provides some reassurance that certain types of
participants were not selectively lost, such an analysis is
limited by those characteristics that were measured at
baseline.
• Investigators cannot control for unmeasured traits that may
be important prognostically, and that may have been more
or less prevalent in the lost participants than in the followed-
up participants. most evidence-based journals of  require
at least 80% follow-up for a prognosis study to be considered
valid.
• In the study we retrieved, follow-up was sufficiently complete and
patients were followed from 2 to 6.5 years.
Were objective outcome criteria applied in a
"blind" fashion?
• We need to assess whether and how explicit criteria for each
outcome of interest were applied and if there is evidence that they
were applied without knowledge of the prognostic factors under
consideration.
• Blinding is crucial if any judgement is required to assess the outcome
because unblinded investigators may search more aggressively for
outcomes in people with the characteristic(s) felt to be of prognostic
importance than in other individuals.
• Blinding may be unnecessary if the assessments are
preplanned for all patients and/or are unequivocal, such as
total mortality.
• However, judging the underlying cause of death is difficult
and requires blinding to the presence of the risk factor to
ensure that it is unbiased.
• In the study we identified, patients were asked at follow-up if they had
a seizure and if they said "yes", a study neurologist subsequently
assessed them. It is unclear if the study neurologist was "blind".
Was there adjustment for important
prognostic factors
• We often want to know if patients with certain
characteristics will have a different prognosis.
• For example, are patients with an intracranial hemorrhage at
increased risk of seizure?
• Demographic, disease-specific or comorbid variables that are
associated with the outcome of interest are called prognostic
factors.
• They need not be causal but must be strongly enough associated
with the development of an outcome to predict its occurrence.
Was there validation in an independent, "test
set" of patients?
• The identification of a prognostic factor for the first time
could be the result of a chance difference in its distribution
between patients with different prognoses.
• Therefore, the initial patient group in which the variable was
identified as a prognostic factor may be considered to be a
training set or a hypothesis generation set.
• Indeed, if investigators were to search for multiple potential
prognostic factors in the same data set, a few would likely
emerge on the basis of chance alone.
• Ideally, therefore, data from a second independent patient
group, or a "test set" would be required to confirm the
importance of a prognostic factor.
• Although this degree of evidence has often not been
collected in the past, an increasing number of reports are
describing a second, independent study validating the
predictive power of prognostic factors.
• If a second, independent study validates these prognostic
factors, it can be called a clinical prediction guide.
Table 2 Number (% of cohort; 95% confidence interval) of patients with single and recurrent seizures
after first stroke (four patients who had seizures within a few hours of death as part of a terminal illness,
or seizures that started before the stroke, were excluded)
Single post stroke Recurrent post
Classification of first stroke Total
seizure stroke seizures
Cerebral infarction (n=545): 17 (3) 18 (3) 35 (6; 4 to 9)
Total anterior circulation
5 (5) 10 (11) 15 (16)
infarction (n=92)
Partial anterior circulation
5 (3) 3 (2) 8 (4)
infarction (n=185)
Lacunar infarction (n=137) 3 (2) 2 (1) 5 (3)
Posterior circulation infarction
4 (3) 3 (2) 7 (5)
(n=129)
Primary intracerebral
3 (5) 4 (6) 7 (11; 3 to 18)
haemorrhage (n=66)
Subarachnoid haemorrhage
3 (9) 3 (9) 6 (18; 5 to 31)
(n=33)
Unknown (n=31) 0 0 0
Total (n=675) 23 (3) 25 (4) 48 (7; 6 to 9)
• In the study we found, the investigators looked at patients with
different stroke types and identified that patients in these groups had
different risks of seizures. This was not tested in an independent group
of patients to see if it holds true.
Are the results of this study
important?
How likely are the outcomes over time?
• Typically, results of prognosis studies are reported in
one of three ways:
• as a percentage of the outcome of interest at a particular
point in time (e.g. 1 year survival rates),
• as median time to the outcome (e.g. the length of follow-
up by which 50% of patients have died) or
• as event curves (e.g. survival curves) that illustrate, at
each point in time, the proportion of the original study
sample who have not yet had a specified outcome.
• From the study we found, the risk of seizure after any type of stroke is
5.7% at 1 year.
How precise is this prognostic estimate?
• The precision of the estimate is best reflected by its 95% confidence
interval; the range of values within which we can be 95% sure that
the population value lies.
• The narrower the confidence interval, the more precise is the
estimate.
• If survival over time is the outcome of interest, earlier follow-up
periods usually include results from more patients than later periods,
so that survival curves are more precise (i.e. have narrower
confidence intervals) earlier in follow-up.
• To calculate the 95% confidence interval for the study we
identified, we can use the following equation:
• 95% Confidence Interval = p +/- 1.96 x SE
• Standard Error (SE)

• And 'p' is a proportion of people with the outcome of interest and 'n'
is the sample size.
From the study, n = 675 and p = 0.057
SE =

= 0.009
Therefore the 95% CI is:
0.057 +/- 1.96 x 0.009 = 3.9% to 7.5%
Child Health
Are children with middle ear disease at increased risk
of behavioural problems? Arch Dis Child 1999;80:28-35.
Critical Care Medicine
In patients without motor response three days after hypoxic
stroke, what is the probability of a good neurologic recovery
(cerebral performance category[CPC] 1 or 2)? Edgren E,
Hedstrand U, Sutton-Tyrell K, Safar P, and BRCTI stud
group. Assessment of neurological prognosis in comatose
Gastroenterology and Hepatology
In a patient with newly diagnosed Crohn's disease what are the chances of
dying prematurely? Loftus EV, Silverstein MD, Sandborn WJ, et al.
Crohn's Disease in Olmstead County Minnesota, 1940-1993:
Incidence, Prevalence, and Survival. Gastroenterology
1998;114:1161-1168.
General Practice
What is the risk of recurrence of ureteric calculus after a single episode in a
middle aged male?Kamihira O, et al. Long-term stone recurrence rate
after extracorporeal shock wave lithotripsy. J Urol 1996
Oct;156(4):1267-71.
Alhamdulillah

You might also like