VALIDITY & RELIABILITY

Ari Udiyono
 Biologic Variations
Biologic Variations of Human Populations

Unimodal Curve
 Biologic Variations
Biologic Variations of Human Populations

Bimodal Curve

Normal

Abnormal

A B C D E
VALIDITY OF SCREENIG TEST

Validity of a test is defined as the

ability of a test to distinguish
between who has a disease and
who does not
Sensitivity: the ability of the test to
identify correctly those who have the
disease
Specificity: the ability of the test to
identify correctly those who do not
have the disease
Results of a Dichotomous Test

Populations
Without
Test Result With Disease
Disease
True Positive (TP) False Positive (FP)
Positive Have disease and No disease but
have positive test have positive test

False Negative (FN) True Negative (TN)

Negative Have disease but No disease and
have negative test have negative test
Results of a Dichotomous Test

Populations
Without
Test Result With Disease
Disease
True Positive (TP) False Positive (FP)
Positive Have disease and No disease but
have positive test have positive test
False Negative (FN) True Negative (TN)
Negative Have disease but No disease and
haveTPnegative test have negative test
Sensitivity =
TP + FN
TN
Specificity =
TN + FP
 Exercise 1
Suppose there was a hypothetical
population of 1000 people of whom 100
have a certain disease and 900 do not
The test indicates that of the 100 people
with disease, 80 were correctly identified
as positive by the test
Of the 900 people who did not have the
disease, the test correctly identified 800
as negative
Very Important !!!
 False Positive
End up with a large group of people
who screened positive:
Really have the disease (true positive)
Do not have the disease (false positive)
All positives are brought back in for
more sophisticated and more
expensive test (financial aspect)
Being anxiety and worry about the
positive result (emotional states)
 False Negative
Person has the disease but is
erroneously informed that the test
result is negative
Be a big problem cause of a serious
disease for which effective
intervention is available
Failed to detect the disease earlier
of its natural history of disease
That might be missed at an early
treatable stage
Test of Continuous Variables

High

Blood
Sugar

Low

Non
Diabetics
Diabetics
Test of Continuous Variables

High Non
Diabetics
Diabetics

Test (+) 5 2
Blood
Sugar
Test (-) 15 18

20 20
Low

Non
Diabetics
Diabetics
Test of Continuous Variables

High Non
Diabetics
Diabetics

Test (+) 17 14
Blood
Sugar Test (-) 3 6

20 20
Low

Non
Diabetics
Diabetics
Test of Continuous Variables

High

Blood
Sugar

Low

Non
Diabetics
Diabetics
Test of Continuous Variables

High

Blood
Sugar

Low

Non
Diabetics
Diabetics
Test of Continuous Variables

High

Blood
Sugar

Low

Non
Diabetics
Diabetics
Test of Continuous Variables

High

Blood
Sugar

Low

Non
Diabetics
Diabetics
 Distribution of Blood Sugar

50
Number of Cases

40

30

20

10
5
40 80 120 160 200 240 280 320 360
Blood Sugar mg/100cc
 The Box Diagram

Non Non
Diabetics Diabetics
Diabetics Diabetics

a b a+b
Test (+) (true (False Test (+) (All people with
Positives) Positives) positives tests)

c d c+d
Test (-) (False (True Test (-) (All people with
Negatives) Negatives) negatives tests)
 Exercise 2
Assume Disease Prevalence = 5%
Population = 10,000
Test 1 (Blood Sugar)
Sensitivity = 70%
Specificity = 80%
Test 2 (Glucose Tolerance Test)
Sensitivity = 90%
Specificity = 90%
Calculate:
Net Sensitivity & Net Specificity
 Predictive Value of a Test
Positive Predictive Value: what
proportion of patients who test
positive actually have the disease in
question
Negative Predictive Value: what is
the probability that this patient does
not have the disease
Predictive Value of a Test

Test Disease No Total

Result Disease
Positive 80 100 180

Negative 20 800 820

Total 100 900 1,000

80
Positive Predictive Value =180 = 44%

80
Negative Predictive Value = = 44%
180
 Exercise 3
Assume a test with Sensitivity = 99% and
Specificity = 95% in a population of
10,000 people
Calculate the Predictive Value with
Disease Prevalence 1%
Disease Prevalence 5%
 Exercise 4
The head of a firemens union consulted a
university cardiologist because the fire
department physician had read an article in a
leading medical journal reporting that a
certain ECG finding was highly predictive of
serious, generally unrecognized, coronary
heart disease.
On the basis of this paper, the fire
department physician was disqualifying many
young able-bodied firemen from active duty.
The cardiologist read the paper and found
that the study had been carried out in
hospitalized patients.
 Table A

Disease
Prevalence = 50%
Test (+) (-)
Sensitivity = 50%
(+) 250 250 500 Specificity = 50%

(-) 250 250 500 250

PV = 500 = 50%

500 500 1000

 Table B

Disease
Prevalence = 20%
Test (+) (-)
Sensitivity = 50%
(+) 100 400 500 Specificity = 50%

(-) 100 400 500 100

PV = 500 = 20%

200 800 1000

 Table C

Disease
Prevalence = 20%
Test (+) (-)
Sensitivity = 90%
(+) 180 400 580 Specificity = 50%

(-) 20 400 420 180

PV = 580 = 31%

200 800 1000

 Table D

Disease
Prevalence = 50%
Test (+) (-)
Sensitivity = 50%
(+) 100 80 180 Specificity = 50%

(-) 100 720 820 250

PV = 500 = 50%

200 800 1000

 Reliability of Test
The value and usefulness of the test
are minimal, if the test result
cannot be reproduced
Factors contribute ro the variation:
Intrasubject variation
Interobserver variation
 Intrasubject Variation
The values obtained in measuring
many human characteristics often
vary over time, even during a short
period
Variability over time is considerable
The conditions under which certain
test are conducted can lead to
different results in the same
individual
 Interobserver Variation
Variation between observers
Two examiners often do not derive
the same results
The extent to which observers
agree or disagree is an important
issue, whether we are considering
physical examinations, laboratory
tests, or other means of assessing
human characteristics
 Percent Agreement

Observer 1

Observer 2 (+) (-)

a x 100%
Agreement = a+b+c
(+) a b

(-) c d
Ignore
 Validity & Reliability

Test
Results

True Value
Graph of hypothetical test result that are reliable but invalid
 Validity & Reliability

Test Results

True Value
Graph of hypothetical test result that are valid but not reliable
 Validity & Reliability

Test
Results

True Value
Graph of hypothetical test result that are both valid and reliable