You are on page 1of 69

ACTL2002/ACTL5101 Probability and Statistics: Week 7

ACTL2002/ACTL5101 Probability and Statistics


c Katja Ignatieva

School of Risk and Actuarial Studies
Australian School of Business
University of New South Wales
k.ignatieva@unsw.edu.au

Week 7
Week 2
Week 3
Week 4
Probability:
Week 6
Review
Estimation: Week 5
Week
8
Week
9
Hypothesis testing:
Week
10
Week
11
Week 12
Linear regression:
Week 2 VL
Week 3 VL
Week 4 VL
Video lectures: Week 1 VL
Week 1

Week 5 VL

ACTL2002/ACTL5101 Probability and Statistics: Week 7

2101/2155

ACTL2002/ACTL5101 Probability and Statistics: Week 7

Last six weeks


Introduction to probability;
Moments: (non)-central moments, mean, variance (standard
deviation), skewness & kurtosis;
Special univariate (parametric) distributions (discrete &
continue);
Joint distributions;
Convergence; with applications LLN & CLT;
Estimators (MME, MLE, and Bayesian);
Evaluation of estimators;
Interval estimation.
2102/2155

ACTL2002/ACTL5101 Probability and Statistics: Week 7

This & next week


Hypothesis testing:
- Null hypothesis:
Selection of the null hypothesis;
Simple v.s. composite.

- Statistical tests;
- Rejection region;
Neyman-Pearson Lemma;
Uniformly most powerful;
Likelihood ratio test.

- Value of test;
- Accept the null or reject the null hypothesis.

Properties of hypothesis testing.


- Type I and II errors and power;
- p-value.
2103/2155

Special tests and other testing methods.

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Introduction in Hypothesis testing
Introduction

Hypothesis testing
Introduction in Hypothesis testing
Introduction
Selection of null hypothesis
The first step
Example: testing proportions
Statistical test
Statistical test
Rejection region
Best critical region
Neyman-Pearson Lemma
Neyman-Pearson Lemma: examples and exercises
Uniformly most powerful (UMP)
Example: Uniformly most powerful (UMP)
Generalized likelihood ratio test
Asymptotic distribution for the GLR test
Example: generalized likelihood ratio test
Summary
Summary

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Introduction in Hypothesis testing
Introduction

Introduction
Formal means of distinguishing between probability
distributions using sample data.
Examples:
- Are the means of two normal distributions with the same
variance equal?
- Are the variances different?
- Is the parameter of the distribution equal to a constant?

Neyman-Pearson approach: null hypothesis H0 ; alternative


hypothesis HA or H1 .
We have different types of hypothesis:

2104/2155

- Simple hypotheses: completely specify the probability


distribution.
- Composite hypotheses: probability distribution not completely
specified.

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Introduction in Hypothesis testing
Introduction

Hypothesis testing
In hypothesis testing the choice is either reject or accept the
hypothesis. Assume that X1 , X2 , . . . , Xn are from a random
sample with density f (x; ).
A statistical hypothesis is defined to be a conjecture or an
assertion about the distribution of a random variable(s). If the
statistical hypothesis completely specifies the distribution, it is
called simple. Otherwise, it is called composite.
A statistical test of a statistical hypothesis, also called test
statistic, T is a rule or procedure for deciding whether to
reject T .

2105/2155

The rejection region, or more commonly called the critical


region, of a test T is the set of all (x1 , x2 , . . . , xn ) for which
you would reject the statistical hypothesis. This is usually
denoted by C ? .

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Introduction in Hypothesis testing
Introduction

Hypothesis testing procedure


Testing procedure: When testing a hypothesis use the following
steps:
i. Define a statistical hypothesis.
Note that this includes a confidence level ();
ii. Define the test statistic T (using past weeks knowledge);
iii. Determine the rejection region C ? ;
iv. Calculate the value of the statistical test, given observed data
(x1 , . . . , xn );
v. Accept or reject H0 .
Note: we assume that H0 is true when testing! (see Type I
and Type II errors)
2106/2155

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Selection of null hypothesis
The first step

Hypothesis testing
Introduction in Hypothesis testing
Introduction
Selection of null hypothesis
The first step
Example: testing proportions
Statistical test
Statistical test
Rejection region
Best critical region
Neyman-Pearson Lemma
Neyman-Pearson Lemma: examples and exercises
Uniformly most powerful (UMP)
Example: Uniformly most powerful (UMP)
Generalized likelihood ratio test
Asymptotic distribution for the GLR test
Example: generalized likelihood ratio test
Summary
Summary

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Selection of null hypothesis
The first step

Hypothesis testing
Null Hypothesis H0 : the hypothesis being tested.
Alternative Hypothesis Ha : the hypothesis accepted if H0 is
rejected.
Results of Decision Making
Hypothesis
Decision
True
False
Reject H0
Type I Error=
No Error
Do not reject H0

No Error

Type II Error=

Type I, II errors: see properties of hypothesis testing (next


lecture).

2107/2155

We can only fix , therefore the choice of the null is


important.

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Selection of null hypothesis
The first step

Illustration Type I and Type II errors


Consider the following test:
H0 :

no correlation between insurance claims Australia and US


versus

Ha :

the correlation is positive

If you reject and in fact the there is positive correlation, then


no error would be committed.
On the other hand, if you reject but there is no correlation,
then an error called a Type I error would have been committed
and the cost might be that the insurer must hold more capital.

2108/2155

If you do not reject the null and later found there is positive
correlation, this is called a Type II error and the cost would be
potential default due to too few reserves.

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Selection of null hypothesis
The first step

Selection of null hypothesis


How to select the Null hypothesis (step i.)?
- One way: select the simpler of the two hypotheses as the null
hypothesis;
- The economic method: The null hypothesis is the hypothesis
where the consequences of incorrectly rejecting the null
hypothesis are largest;
- The statistical method: select the null as something that you
wish to conclusively disprove.

The level of significance of the test can be determined from:


Pr (T C ? |H0 ) = ,

where T is a test statistic and C ? is the rejection region (see


later).

2109/2155

Confidence level is the probability that you reject H0 given


that H0 is true.

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Selection of null hypothesis
The first step

Simple and composite hypothesis

Recall from introduction, two kind of hypothesis:

If X f (x; ), a statistical hypothesis is a statement about


the distribution of X . If the hypothesis completely specifies
f (x; ), then it is referred to as a simple hypothesis otherwise
it is called composite.

2110/2155

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Selection of null hypothesis
The first step

Simple and composite hypothesis


We consider three cases:
- Simple null v.s. simple alternative;
Example:
H0 : = 0 v.s. H1 : = 1 , for given constants 0 and 1 .
- Simple null v.s. composite alternative;
Example:
H0 : = 0 v.s. H1 : 1 , for given constant 0 and set 1 .
Where usually the alternative is of the form:
- H1 : =
6 0 ;
- H1 : < 0 ;
- H1 : > 0 .

- Composite null v.s. composite alternative;


Example:
H0 : 0 v.s. H1 : 1 , for given sets 0 and 1 .
Where usually the null and alternative are of the form:
2111/2155

- H0 : 0 vs H1 : > 0 ;
- H0 : 0 vs H1 : < 0 .

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Selection of null hypothesis
Example: testing proportions

Hypothesis testing
Introduction in Hypothesis testing
Introduction
Selection of null hypothesis
The first step
Example: testing proportions
Statistical test
Statistical test
Rejection region
Best critical region
Neyman-Pearson Lemma
Neyman-Pearson Lemma: examples and exercises
Uniformly most powerful (UMP)
Example: Uniformly most powerful (UMP)
Generalized likelihood ratio test
Asymptotic distribution for the GLR test
Example: generalized likelihood ratio test
Summary
Summary

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Selection of null hypothesis
Example: testing proportions

Example: testing proportions


An insurer investigates whether climate change has already an
(positive or negative) effect on the probability that insured
lodge a claim.
Due to the Law of Large Numbers the insurer has a good
(almost certain) estimate of the probability of lodging a claim
before the climate change (say before T 5). He knows that
the probability was p0 = 0.1%.
The last five years the number of claims were 5 and the
number of insurance contracts 2500.
Preform the test the insurer is going to make, with a
significance level of 5%.
2112/2155

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Selection of null hypothesis
Example: testing proportions

Example: testing proportions


i. Consider the hypothesis test:
H0 : p = p0

v.s.

Ha : p 6= p0 ,

with a random sample X1 , X2 , . . . , Xn from a Bernoulli (p)


distribution.
ii. We know that the best unbiased point estimator of p is:
n
1X
b
Xk .
p=
n
k=1

From central limit theorem (see week 6), we know that:

2113/2155

b
b
p E [b
p]
pp
p
=r
,
Var (b
p)
p (1 p)
n
has an approximate standard normal distribution.

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Selection of null hypothesis
Example: testing proportions

f(x|H0)

np 0 z1/2

When

2114/2155

p
np 0 (1 p 0 )

np0

np 0 + z1/2

p
np 0 (1 p 0 )

Xi is in the blue shaded area likely that p 6= p0 reject H0 .

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Selection of null hypothesis
Example: testing proportions

Under the null hypothesis H0 : p = p0 , the observed test


statistic (know this from last week CI properties) is:
T =r

b
p p0

p0 (1 p0 )
n
where T is standard normal distributed.

iii. We then reject the null hypothesis if:


T < z1/2

or

T > z1/2

at a level of significance . (More on this in the remainder of


the lecture)
b = 0.002, n = 2500 thus T = 1.58193.
iv. We have p0 = 0.001, p
2115/2155

v. Do not reject H0 . Question: Comment on the test.


d
Test uses: Bin(n, p) N(np, np(1 p)) which is good if
np , but n p0 = 2.5!

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Statistical test
Statistical test

Hypothesis testing
Introduction in Hypothesis testing
Introduction
Selection of null hypothesis
The first step
Example: testing proportions
Statistical test
Statistical test
Rejection region
Best critical region
Neyman-Pearson Lemma
Neyman-Pearson Lemma: examples and exercises
Uniformly most powerful (UMP)
Example: Uniformly most powerful (UMP)
Generalized likelihood ratio test
Asymptotic distribution for the GLR test
Example: generalized likelihood ratio test
Summary
Summary

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Statistical test
Statistical test

Step ii. Define a statistical test.


For this we can use our previous obtained knowledge;
Recall from week 5: Central limit theorem.
Recall from week 5: Maximum likelihood estimator;
Recall from week 6: CI for Maximum likelihood estimates;
Recall from week 6: Interval estimation using confidence
intervals.

Assume null hypothesis is true, i.e., H0 defines the population


parameters, which are being tested.

2116/2155

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Best critical region

Hypothesis testing
Introduction in Hypothesis testing
Introduction
Selection of null hypothesis
The first step
Example: testing proportions
Statistical test
Statistical test
Rejection region
Best critical region
Neyman-Pearson Lemma
Neyman-Pearson Lemma: examples and exercises
Uniformly most powerful (UMP)
Example: Uniformly most powerful (UMP)
Generalized likelihood ratio test
Asymptotic distribution for the GLR test
Example: generalized likelihood ratio test
Summary
Summary

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Best critical region

The critical region for a test of hypothesis is the subset of the


sample space that corresponds to rejecting the null hypothesis.
Example: Consider the following region to reject the null
hypothesis:
- Let X Bin(n, p), with n = 8 and unknown p;
- Test H0 : p = 0.6943 v.s. H1 : p > 0.6943 at = 0.05.
- Possible rejection region: Reject H0 only if X = 3, i.e., we
have Pr(X = 3) = 0.05 = .
- Hence, acceptance region is {0, 1, 2, 4, 5, 6, 7, 8}

Question: Is this a correct rejection region?


Solution: Yes, but it is not the best.
Consider three cases:
1) Neyman-Pearson Lemma; 2) UMP; 3) GLR.
2117/2155

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Best critical region

Best critical region


f(x|H0)

2118/2155

f(x|H1)

How to
select blue
shaded area
(rejection
region) such
that purple
area (i.e.,
rejecting H0
given H1 is
true) is the
largest?
Note: blue
shaded area
is Type I
error = .

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Best critical region

Formally: A subset C ? of the sample space is called a best


critical region for testing the above hypothesis if for every
subset A of the sample space for which:
Pr ((X1 , X2 , . . . , Xn ) A |H0 ) = ,

the following two conditions hold:

i) Pr ((X1 , X2 , . . . , Xn ) C ? |H0 ) = ;
ii) Pr ((X1 , X2 , . . . , Xn ) C ? |H1 ) >
Pr ((X1 , X2 , . . . , Xn ) A |H1 ).

i) Implies that the probability of rejection null hypothesis, given


that the null hypothesis is true, is equal to ;
ii) The probability that the H0 is rejected given H1 is true is the
largest for region C ? .
Used on slide 2120 for NP and on slide 2132 for UMP.

2119/2155

Optimal tests: tests with higher power than any other test,
given a significance level .

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Neyman-Pearson Lemma

Hypothesis testing
Introduction in Hypothesis testing
Introduction
Selection of null hypothesis
The first step
Example: testing proportions
Statistical test
Statistical test
Rejection region
Best critical region
Neyman-Pearson Lemma
Neyman-Pearson Lemma: examples and exercises
Uniformly most powerful (UMP)
Example: Uniformly most powerful (UMP)
Generalized likelihood ratio test
Asymptotic distribution for the GLR test
Example: generalized likelihood ratio test
Summary
Summary

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Neyman-Pearson Lemma

Neyman-Pearson Lemma
Suppose X1 , X2 , . . . , Xn is a random sample from a distribution
whose density function is f (x; ). Let the joint density of
X1 , X2 , . . . , Xn be:
L (x1 , x2 , . . . , xn ; ) = f (x1 ; )f (x2 ; ). . .f (xn ; ) = Pr(X = x; ).
A subset C ? = {x; 0 , 1 } of the sample space is the best critical
region for testing the simple null/simple alternative test stated
above, if the following conditions are satisfied (see slide 2119):
i) Pr ((X1 , X2 , . . . , Xn ) C ? |H0 ) = ;
L (x1 , . . . , xn ; 0 )
6 k,
ii) (x; 0 , 1 ) =
L (x1 , . . . , xn ; 1 )
for every (x1 , . . . , xn ) C ? , where k 1 is a constant such
that: Pr ((X1 , . . . , Xn ) C ? |0 ) = .
Hence: when rejecting the null, when it is incorrect (i.e.,
(x1 , . . . , xn ) C ? ) the likelihood of the data coming from the
2120/2155
alternative is larger than from the null distribution.

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Neyman-Pearson Lemma

Proof Neyman-Pearson Lemma


Let us denote C as the complement of set C ,
X = [X1 , . . . , Xn ] and x = [x1 , . . . , xn ] and A is an
n-dimensional event, let:
Z
Z
Z
Pr (X A|) =
fX (x; ) = fX (x1 , . . . , xn ; )dx1 . . . dxn = L(x; ),
A

for any (continuous case).

Using condition ii) we obtain:


- in case A C ? :

L(x; 0 ) k L(x; 1 )
?

Pr (X A|0 ) k Pr (X A|1 )

- in case A C :
?

1 Pr(X C |0 ) k (1 Pr(X C |1 ))

2121/2155

Pr(X C |0 ) k Pr(X C |1 ) + (1 k) k Pr(X C |1 )


Pr (X A|0 ) k Pr (X A|1 ).

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Neyman-Pearson Lemma

For any critical region C we have (using LTP):


C ? = (C ? C ) (C ? C )

and

C = (C C ? ) (C C ).

Define for a set A and parameter : A () = Pr(X A|):


C ? () =Pr(X C ? C |) + Pr(X C ? C |)
?

C () =Pr(X C ? C |) + Pr(X C C |)
?

C ? () C () =Pr(X C ? C |) Pr(X C C |).

(1)

* using (1) with = 1 ** using previous slide *** using (1)


with = 0 **** using slide 2119 A (0 ) = Pr (X A |H0 ) = :

C ? (1 ) C (1 ) =Pr(X C ? C |) Pr(X C C |)



?
(1/k) Pr(X C ? C |0 ) Pr(X C C |0 )

= (1/k) (C ? (0 ) C (0 )) = ( )/k = 0

2122/2155

C ? (1 ) C (1 ).

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Neyman-Pearson Lemma

Interpretation Neyman-Pearson Lemma


Suppose H0 specifies density or frequency function L (x; 0 )
and alternative H1 specifies L (x; 1 );
Relative likelihoods of H0 versus H1 is the likelihood ratio:
L (x; 0 ) /L (x; 1 ) .
Question: What does a low ratio imply?
Solution: A low ratio implies that the data are more likely
under H1 than under H0 .
Thus: Likelihood ratio rejects for small values of the ratio.
Neyman-Pearson Lemma: if the likelihood ratio that rejects
H0 with significance level is:
L (x; 0 ) /L (x; 1 ) < c,
2123/2155

then any other test with significance level will have power
less than or equal to that of the likelihood ratio test.

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Neyman-Pearson Lemma: examples and exercises

Hypothesis testing
Introduction in Hypothesis testing
Introduction
Selection of null hypothesis
The first step
Example: testing proportions
Statistical test
Statistical test
Rejection region
Best critical region
Neyman-Pearson Lemma
Neyman-Pearson Lemma: examples and exercises
Uniformly most powerful (UMP)
Example: Uniformly most powerful (UMP)
Generalized likelihood ratio test
Asymptotic distribution for the GLR test
Example: generalized likelihood ratio test
Summary
Summary

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Neyman-Pearson Lemma: examples and exercises

Example: Neyman-Pearson Lemma


Insurer AB is in the process of taking over local (and thus
small) insurer BC.
In the process of controlling the books of insurer BC the
insurer observes that the normally distributed claim sizes of
insurer BC are larger than of insurer AB. This can be due to,
for example:
- statistical uncertainty, due to small sample size of insurer BC;
- adverse selection effects.

Due to the LLN insurer AB knows that his claims are


2 ) distributed.
N(AB , AB

2124/2155

Insurer AB want to know whether the mean claims of insurer


BC is x or AB . Assume that AB is a know constant and the
2 .
variances of both insurers are equal, known and equals AB

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Neyman-Pearson Lemma: examples and exercises

Example: Neyman-Pearson Lemma


2 , = x and =
Let 2 = AB
0
A
AB . X1 , X2 , . . . , Xn from a
normal distribution with known variance 2 . Hypotheses:

H0 : = 0

H1 : = 1

v.s.

with a (given) constant.


Question: What is the best critical region?
Solution: Use Neyman-Pearson lemma.
1. The likelihood ratio is:
(x; 0 , 1 ) =

2125/2155

Qn

1
i=1 2

exp

1
2 2

(xi 0 )2

f0 (x) L(x; 0 )


=
=Q
n
f1 (x) L(x; 1 )
1 exp 12 (xi 1 )2
i=1 2
2
 P

n
2
1
exp 2
2
i=1 (xi 0 )
 P
 < k.
=
n
2
1
exp 2
(x

)
1
i
2
i=1

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Neyman-Pearson Lemma: examples and exercises

Question: when to reject H0 ?


Solution: reject for small values of the ratio L (x; 0 ) /L (x; 1 ).
Thus, by using a log-transformation (monotonic increasing
transformation) we have that we reject H0 when:
n
X

i=1
n
X
i=1

(xi 1 )2

n
X
i=1

(xi 0 )2 < k1

= log(k) 2 2



xi2 2xi 1 + 21 xi2 2xi 0 + 20 < k1


2n x (0 1 ) + n 21 20 < k1
n
X
 

xi (0 1 ) < k2
= (k1 n 21 20 )/2

i=1

is small, * using
2126/2155

n
P

xi = n x.

i=1

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Neyman-Pearson Lemma: examples and exercises

Therefore, we have:
(a) if 0 > 1 (0 1 ) > 0 the likelihood ratio is small if x is
small;
(b) if 0 < 1 (0 1 ) < 0 the likelihood ratio is small if x is
large.

2. In the latter (former)


 case need to determine
 a value x0 such
that Pr X > x0 |H0 = (Pr X < x0 |H0 = ) if H0 is true.

We know: X |H0 N 0 , 2 /n calculate the value of x0 .


Pr X < x0 |H0 =

Pr (Z < z1 ) =

2127/2155


X 0
x
< 0 0 =
/ n
/ n

x0 0

z1 =
x0 = 0 z1 / n.
/ n

Pr

Best critical
region:
n
?
C = (x1 , . . . , xn ) : x 0 z1
n
C ? = (x1 , . . . , xn ) : x 0 + z1

o
o

in case of (a) and


in case of (b).

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Neyman-Pearson Lemma: examples and exercises

Time until a natural disaster claims. Consider a random sample of


size n with Xi EXP(). Test H0 : = 0 v.s. H1 : = 1 < 0 .
Exercise: Find the best critical region.
1. Solution: The Neyman-Pearson Lemma says to reject H0 if:
P
0n exp (0 ni=1 xi )
P
(x; 0 , 1 ) = n
k,
1 exp (1 ni=1 xi )

where k is a constant such that Pr((x; 0 , 1 ) k|H0 ) = .

n
X
i=1

xi (1 0 ) log((1 /0 ) k)

where k1 = (1 0 )1 log((1 /0 )n k).


|
{z
}
<0

Thus
2128/2155

C?

= {(x1 , . . . , xn ) :

n
P

xi k1 }.

i=1

Next step: Find k1 use Pr(X C ? |0 ) = Pr

Pn

i=1

n
X
i=1

xi k1 ,


Xi k1 |0 .

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Neyman-Pearson Lemma: examples and exercises

2. Using
we have:
Pn m.g.f. technique (see week 4) P
n
2
i=1 Xi |H0 Gamma(n, 0 ) 20
i=1 Xi |H0 (2n),
!
n
n
X
X
Pr
Xi k1 |H0 =
Pr (2 0
Xi 2 0 k1 |H0 ) =
| {z }
i=1
i=1
21 (2n)
|
{z
}
2 (2n)

2 (2n)
21 (2n) = 2 0 k1
k1 = 1
.
2 0


n
P
?
2
Thus: C = (x1 , . . . , xn ) :
xi 1 (2n)/(2 0 ) .
i=1

2129/2155

Exercise: Consider a random sample of size n, with


0 xi 1 for i = 1, . . . , n.
Test H0 : X UNIF(0, 1) with fX (xi ) = 1 v.s.
H1 : X EXP(1) with fX (xi ) = e xi .
Note: under H0 : E[Xi ] = 0.5 and Var (Xi ) = 1/12.

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Neyman-Pearson Lemma: examples and exercises

1. Solution: The Neyman-Pearson Lemma says to reject H0 if:


1
Pn
(x; 0 , 1 ) =
k,
exp ( i=1 xi )

where k is a constant
P such that Pr((x; 0 , 1 ) k|H0 ) = .
So we reject H0 if ni=1 xi k1 = log(k).
P
2. How to determine the distribution of ni=1 Xi |H0 , use CLT:
Pn
Pn
P
X n E[Xi ]
Xi E[ ni=1 Xi ]
i=1
p
p i
= i=1
Zn =
Pn
Var ( i=1 Xi )
n Var (Xi )
Pn
X n/2 d
p i
= i=1
Z N(0, 1)
n/12
!
n
X
p
n
Pr (Zn < z1 ) =
Pr
Xi z1 n/12 =
2
i=1



n
P
z1 n
n
?
Thus C = (x1 , . . . , xn ) :
xi 2 12
.
2130/2155

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Uniformly most powerful (UMP)

Hypothesis testing
Introduction in Hypothesis testing
Introduction
Selection of null hypothesis
The first step
Example: testing proportions
Statistical test
Statistical test
Rejection region
Best critical region
Neyman-Pearson Lemma
Neyman-Pearson Lemma: examples and exercises
Uniformly most powerful (UMP)
Example: Uniformly most powerful (UMP)
Generalized likelihood ratio test
Asymptotic distribution for the GLR test
Example: generalized likelihood ratio test
Summary
Summary

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Uniformly most powerful (UMP)

Uniformly most powerful


Consider simple null, composite alternative hypothesis, i.e.,
H0 : = 0 v.s. H1 : 1 .
Question: When applying Neyman-Pearson Lemma what
would be L (x1 , x2 , . . . , xn ; 1 )?
Answer: This would depend on the 1 1 which is true.
The critical region C is uniformly most powerful (UMP) of
size for testing a simple H0 against a composite H1 if C is
the best critical region of size for testing H0 against
every simple hypothesis in H1 . The corresponding test is
called the UMP test of size .
If H1 composite then a uniformly most powerful test is most
powerful for every simple alternative in H1 .
2131/2155

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Uniformly most powerful (UMP)

Uniformly most powerful


Let X1 , . . . , Xn have joint p.d.f. f (x1 , . . . , xn ; ) for .
Consider hypothesis of the form:
H 0 : 0

v.s.

H1 : 0 .

A critical region C ? and the associated test are said to be


uniformly most powerful (UMP) of size if the following two
conditions hold (using slide 2119):
i) max {Pr((X1 , . . . , Xn ) C ? |)} = ;
0

ii) for all 0 and all critical regions C of size we have:


Pr((X1 , . . . , Xn ) C ? |) Pr((X1 , . . . , Xn ) C |)

2132/2155

i.e., the probability of rejecting the null, when the null is


incorrect (i.e., 0 ) is the largest for critical region C ? .

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Uniformly most powerful (UMP)

A joint p.d.f. f (x; ) is said to have a monotone likelihood


ratio (MLR) in the statistic T = (X ) if for any two values of
the parameter 1 < 2 , the ratio f (x; 2 )/f (x; 1 ) depends on
X only through the function (x), and this ratio is a
nondecreasing function of (x).
Neyman-Pearson tests also provide UMP test for
corresponding one-sided composite alternatives.
If a joint p.d.f. f (x; ) has a MLR in T = (X ) then a UMP
test of size for H0 : 0 v.s. H1 : > 0 is to reject H0 if
(x) k, where Pr( (X ) k|0 ) = .

Example: Consider
Pnthe example from slide 2128.
We have (X ) = i=1 Xi and
!
P
 n
n
X
0n exp ( ni=1 xi 0 )
0
P
(x; 0 , 1 ) = n
=
exp
xi (0 0 )
1 exp ( ni=1 xi 1 )
1
i=1

2133/2155

is thus a nondecreasing function in (x) if 1 < 0 .

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Example: Uniformly most powerful (UMP)

Hypothesis testing
Introduction in Hypothesis testing
Introduction
Selection of null hypothesis
The first step
Example: testing proportions
Statistical test
Statistical test
Rejection region
Best critical region
Neyman-Pearson Lemma
Neyman-Pearson Lemma: examples and exercises
Uniformly most powerful (UMP)
Example: Uniformly most powerful (UMP)
Generalized likelihood ratio test
Asymptotic distribution for the GLR test
Example: generalized likelihood ratio test
Summary
Summary

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Example: Uniformly most powerful (UMP)

Example UMP: testing means from normal distribution


Consider sampling from the normal distribution to test the
mean parameter.
Same example, but now test whether the claims sizes of
insurer BC are higher.

Suppose X1 , X2 , . . . , Xn is a random sample from N , 2 .
We wish to test:
H0 : = 0

v.s.

H1 : > 0 ,

for some known constant 0 and .


If the variance 2 is known, we know from week 5:

2134/2155

X N(, 2 /n).

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Example: Uniformly most powerful (UMP)

Example UMP: testing means from normal distribution


1. The Likelihood ratio numerator f0 (x) = L(x; 0 ) is constant:
!
n

n
1 X
1
exp
f0 (x) =
(xi 0 )2 .
2 2
2
i=1

The Likelihood ratio denominator f1 (x) = L(x; 1 ) depends on


1 > 0 :
!

n
n
1
1 X
2
exp
f1 (x) =
(xi 1 ) .
2 2
2
i=1
for all 1 > 0 .
We have:
exp
(x; 0 , 1 ) =

2135/2155

exp

1
2 2
1
2 2


2
(x

)
0
i=1 i
 < k,
Pn
2
(x

)
1
i
i=1
Pn

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Example: Uniformly most powerful (UMP)

Reject H0 for small values of f0 (x)/f1 (x), i.e., (using a


log-transformation and simplifying) when:
x (0 1 ) < k1
is small (for all 1 > 0 ).
We have: 0 1 < 0 for all 1 > 0 , thus:


k1
?
x k
=
,
(0 1 )


2. Next, determine a value x0 such that Pr X > k ? |H0 = .


2
We know that X |H0 N 0 , n calculate the value of

k ? = 0 + z1 / n.

2136/2155

Best critical
region:
n
?
C = (x1 , . . . , xn ) : x 0 + z1

o
.

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Example: Uniformly most powerful (UMP)

Exercise UMP
Number of claim distribution. Consider a sample of size n
from a Poisson distribution. Xi Poi(), with joint p.d.f.:
Pn

exp(n) i=1 xi
fX (x; ) =
x1 ! . . . xn !
=(x1 ! . . . xn !)

exp log()

n
X
i=1

xi n

Exercise: use MLR to find the UMP of size for H0 : 0


v.s. H1 : > 0 .
1. Solution: We have:

!
n
X
f (x; 1 )
= exp (log(1 ) log(0 ))
xi n (1 0 )
f (x; 0 )
i=1

2137/2155

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Example: Uniformly most powerful (UMP)

P
P
(continued) () = ni=1 xi , reject H0 if T = ni=1 Xi k,
where Pr(T k|0 ) = .
P
2. We have T = ni=1 Xi POI(n0 ), thus

X
x
exp (n0 ) (n0 )

k2 = argmax

x!
k1
x=k1
P
Rejection region: C ? = {(x1 , . . . , xn ) | ni=1 xi k2 }.
Dental insurance: Consider a sample of size n from a twoparameter exponential distr, Xi EXP(1, ). The j.p.d.f. is:

P
exp ( ni=1 (xi )) , if < x(1) ;
fX (x, ) =
0,
if x(1) .
2138/2155

Example: use MLR to find the UMP of size for H0 : 0


v.s. H1 : > 0 .

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Example: Uniformly most powerful (UMP)

Distribution of minimum:
Pr(X(1) k) = Pr(X1 k, . . . , Xn k) = Pr(Xi k)n .
1. Solution: we have:

f (x, 1 )
0,
if 0 < x(1) 1 ;
=
exp (n (1 0 )), if 1 < x(1) .
f (x, 0 )

Note that f (x, 1 )/f (x, 0 ) is not defined for x(1) 0 , but
this is no problem because Pr(X(1) 0 |0 ) = 0.
T = () = X(1) , f (x, 1 )/f (x, 0 ) is nondecreasing function
in x(1) , hence we can use MLR.

2. Find rejection region by:


= Pr(T k|0 ) = Pr(X(1) k|0 ) = exp(n (k 0 ))

k =0 log()/n.

2139/2155



Rejection region: C ? = (x1 , . . . , xn ) : x(1) 0 log()/n .

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Generalized likelihood ratio test

Hypothesis testing
Introduction in Hypothesis testing
Introduction
Selection of null hypothesis
The first step
Example: testing proportions
Statistical test
Statistical test
Rejection region
Best critical region
Neyman-Pearson Lemma
Neyman-Pearson Lemma: examples and exercises
Uniformly most powerful (UMP)
Example: Uniformly most powerful (UMP)
Generalized likelihood ratio test
Asymptotic distribution for the GLR test
Example: generalized likelihood ratio test
Summary
Summary

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Generalized likelihood ratio test

Generalized likelihood ratio test


Summarizing the determination of the critical region we have:
- Neyman-Pearson: when simple null v.s. simple alternative;
- UMP: in case of simple null v.s. composite alternative
(example H1 : > 0 , but similarly we have H1 : < 0 );
- Can we use UMP for simple null v.s. composite alternative
with 1 6= 0 ?
No, C = (x0 , ) in case > 0 , but C = (, x0 ) in case
< 0 .

When there is no uniformly most powerful test for testing a


simple null against a composite alternative hypothesis. In this
case the generalized likelihood ratio test (LRT) is used.
The generalized LRT is also the basis for constructing a test of
a composite null against a composite alternative hypothesis.
2140/2155

Note: LRT tests do not always lead to the best critical region.

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Generalized likelihood ratio test

Best critical region, 2-sided tests


f(x|H0)

2141/2155
0 z1/2 n

How to
select blue
f(x|H1)
shaded area
(rejection
region) such
that purple
area (i.e.,
rejecting H0
given H1 is
true) is the
largest?
Note: blue
shaded area
is Type I
0 + z1/2 n error = .

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Generalized likelihood ratio test

Generalized likelihood ratio test


Suppose X1 , X2 , . . . , Xn is a random sample from f (x; ) so
that the joint density is:
L (x1 , x2 , . . . , xn ; ) = f (x1 ; ) f (x2 ; ) . . . f (xn ; ) .
Denote the test by:
H 0 : 0

v.s.

H 1 : 1 ,

for a given ,

where = 0 1 .
Define the likelihood functions:
L0 () = L (x1 , x2 , . . . , xn ; ) ,
L () = L (x1 , x2 , . . . , xn ; ) ,
2142/2155

with 0 ,
with .

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Generalized likelihood ratio test

Denote their maximums, if they exist, by max {L0 ()} and


0

max {L ()}, respectively.

Denote b0 the MLE under the restriction that H0 is true and b


under the restriction that H0 or H1 is true (typically the usual
MLE).
The ratio:
(x1 , x2 , . . . , xn ) =

max {L0 ()}

max {L ()}

L(x; b0 )
,
b
L(x; )

is called the likelihood ratio. Note: 0 (x) 1.

The LRT principle says that the null hypothesis above is


rejected if and only if:
(x1 , x2 , . . . , xn ) < k,
2143/2155

for some positive k.

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Asymptotic distribution for the GLR test

Hypothesis testing
Introduction in Hypothesis testing
Introduction
Selection of null hypothesis
The first step
Example: testing proportions
Statistical test
Statistical test
Rejection region
Best critical region
Neyman-Pearson Lemma
Neyman-Pearson Lemma: examples and exercises
Uniformly most powerful (UMP)
Example: Uniformly most powerful (UMP)
Generalized likelihood ratio test
Asymptotic distribution for the GLR test
Example: generalized likelihood ratio test
Summary
Summary

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Asymptotic distribution for the GLR test

GLR is similar to Neyman-Pearson principle.


GLR is not a complete generalization of Neyman-Pearson
Lemma, because the unrestricted estimate b could be possible
in 0 .

The function (X ) is a valid test statistic not a function


of unknown parameters:

- (X ) is free of parameters: exact critical value k can be


determined.
- (X ) under H0 depends on unknown parameters MLE are
asymptotically normally distributed (under regulatory
conditions) free of parameters. If X f (x; 1 , . . . k ) then
under H0 : (1 , . . . , r ) = (10 , . . . , r 0 ), r < k we have
approximately for large n:
2 log((X )) 2 (r )
An approximate size test to reject H0 if:
2144/2155

2 log((X )) 21 (r )

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Asymptotic distribution for the GLR test

Asymptotic Distribution of Likelihood Ratio


Recall from last week asymptotic MLE property:
p
d
nIf ? ()(b ) N(0, 1).

Under certain smoothness condition of the probability


density (or mass) functions involved, the distribution of the
test statistic:
T (X ) = 2 log () (= 2 (log (L(x; 0 )) log (L(x; ))))
has -under the null hypothesis- asymptotically a chi-squared
distribution with degrees of freedom dim () dim (0 ).
This asymptotic result often used for the (approximate)
distribution of the test statistic under the null hypothesis.
2145/2155

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Example: generalized likelihood ratio test

Hypothesis testing
Introduction in Hypothesis testing
Introduction
Selection of null hypothesis
The first step
Example: testing proportions
Statistical test
Statistical test
Rejection region
Best critical region
Neyman-Pearson Lemma
Neyman-Pearson Lemma: examples and exercises
Uniformly most powerful (UMP)
Example: Uniformly most powerful (UMP)
Generalized likelihood ratio test
Asymptotic distribution for the GLR test
Example: generalized likelihood ratio test
Summary
Summary

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Example: generalized likelihood ratio test

Example LRT: testing means from normal distribution


Consider sampling from the normal distribution to test the
mean parameter.

Suppose X1 , X2 , . . . , Xn is a random sample from N , 2 .
We wish to test:
H0 : = 0

v.s.

for some known constant 0 and .

H1 : 6= 0 ,

1. Define the sets:


thus

2146/2155

0 ={ = 0 }

and

={(, )}.

1 = { {(, 0 ) (0 , )}

2. Find the and 0 that maximizes the likelihood


function (see week 5 MLE):
b0 = 0
and
b = x.

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Example: generalized likelihood ratio test

Example LRT: testing means from normal distribution


3. Thus the Likelihood ratio numerator is:
!

n
n
1 X
1
2
exp
max {L0 (0 )} =
(xi 0 ) .
0
2 2
2
i=1
and the likelihood denumerator is:
max{L ()} =

2147/2155

n

exp

n
1 X
(xi x)2
2 2
i=1

Next slide: the likelihood ratio


(using exp(a)/ exp(b) P
= exp(a b));
and ** using nx 2x ni=1 xi = nx 2nx 2 = nx.

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Example: generalized likelihood ratio test

(x) = exp

n

1 X 
2
2
x)
(x

(x

0
i
i
2 2
i=1

<k

Rejecting for small values of is the same as rejecting for large


values of:
n 

X

(xi 0 )2 (xi x)2 > k1


(= 2 2 log (k))
{z
}
|
i=1

n
X
i=1

decreasing

xi2 + n20 20

n
X
i=1

= n (x 0 )2 > k1

xi

n
X
i=1

(x 0 )2 > k2
x 0 > |k3 |

equivalently: 0 k ? > x > 0 + k ? .

2148/2155

xi2 + nx 2 2x

n
X
i=1

xi

(= k1 /n)

p 
= k2

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Example: generalized likelihood ratio test

Example LRT: testing means from normal distribution


The rejection region is of the form:
C ? = {(x1 , . . . , xn ) : x {(, 0 k ? ) (0 + k ? , )}} .
where k ? is such that:


Pr 0 k ? X 0 + k ? H0 = .

4. If the variance 2 is known, we know from week 5 (or week 4


m.g.f. technique): X |H0 N(0 , 2 /n), thus:


k ?
X 0
k ?

H0 =
Pr
/ n
/ n
/ n


Pr z1/2 Z z1/2 =

k ? = z1/2 / n.

C =
2149/2155

Hence, the rejection region is:

(x1 , . . . , xn ) : x



, 0 z1/2
n



0 + z1/2 ,
.
n

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Example: generalized likelihood ratio test

Example LRT: testing means from normal distribution


Again, suppose
X1 , X2 , . . . , Xn is a random sample from

N , 2 , and x > 0 . We wish to test:
H0 : 0

v.s.

H1 : > 0 ,

for some known constant 0 and .


What is then the Likelihood ratio numerator?
Given L(, )/ < 0 if > x, L(, )/ = 0 if = x,
and L(, )/ > 0 if < x we have:

x, if x < 0 , i.e., the MLE;

b=
0 , if x 0 , i.e., bounding condition,
2150/2155

where
b is the 0 which maximizes L0 ().

Note: the Likelihood ratio denominator is the same as in the


previous example.

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Example: generalized likelihood ratio test

Exercise LRT
Testing equality in proportions of insured issuing claims.
Let X Bin(n1 , p1 ) and Y Bin(n2 , p2 ) with X and Y
independent.
Exercise: Test whether the proportions are equal, i.e.,
H0 : p1 = p2 = p v.s. H1 : p1 6= p2 , where p is unknown.
b1 = x/n1 and
Solution: Based on x and y the MLEs are p
b
p2 = y /n2 and the restricted MLE is b
p = (x + y )/(n1 + n2 ).
The GLR statistic is:

2151/2155

n1
b)n1 x
p x (1 p
x b

n1
b1 )n1 x
p1x (1 p
x b
b)n1 +n2 xy
p

f (x; b
p ) f (y ; b
p)
=
(x, y ) =
f (x; b
p1 ) f (y ; b
p2 )

b
p x+y (1
b1x (1 b
p
p1 )n1 x b
p2y (1 b
p2 )n2 y

n2
y

n2
y

b
p y (1 b
p )n2 y

b
p2y (1 b
p2 )n2 y

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Rejection region
Example: generalized likelihood ratio test

Exercise LRT
GLR test statistic under H0 depends on unknown parameter p.
Note rewrite H0 : = p2 p1 = 0 so H0 represents a
one-dimensional restriction in the parameters, i.e., r = 1 free
parameter.
An approximately size test is:
2 log((x, y )) 2 (1)
Hence, H0 is rejected if 2 log((x, y )) > 21
(1), i.e.,
C ? = (x1 , . . . , xn ) : 2 log((x, y )) > 21 (1)
2152/2155

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Summary
Summary

Hypothesis testing
Introduction in Hypothesis testing
Introduction
Selection of null hypothesis
The first step
Example: testing proportions
Statistical test
Statistical test
Rejection region
Best critical region
Neyman-Pearson Lemma
Neyman-Pearson Lemma: examples and exercises
Uniformly most powerful (UMP)
Example: Uniformly most powerful (UMP)
Generalized likelihood ratio test
Asymptotic distribution for the GLR test
Example: generalized likelihood ratio test
Summary
Summary

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Summary
Summary

Hypothesis tests
Testing procedure: When testing a hypothesis use the following
steps:
i. Define a statistical hypothesis.
Note that this includes a confidence level ();
ii. Define the test statistic T (using past weeks knowledge);
iii. Determine the rejection region C ? ;
iv. Calculate the value of the statistical test, given observed data
(x1 , . . . , xn );
v. Accept or reject H0 .
Note: we assume that H0 is true when testing! (see Type I
and Type II errors)
2153/2155

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Summary
Summary

Best critical region


Neyman-Pearson Lemma: simple null/simple alternative
test:
L(x; 0 )
< k s.t.: Pr (X C ? |H0 ) =
(x; 0 , 1 ) =
L(x; 1 )
Uniform most powerful test: simple/composite
null/composite alternative test:
i) max {Pr((X1 , . . . , Xn ) C ? |)} = ;
0

ii) for all 0 and all critical regions C of size we have:


Pr((X1 , . . . , Xn ) C ? |) Pr((X1 , . . . , Xn ) C |)

Generalized likelihood ratio test: simple/composite


null/composite alternative test:
(x; 0 , ) =
2154/2155

L(x; b0 )
<k
b
L(x; )

s.t.: Pr (X C ? |H0 ) = .

ACTL2002/ACTL5101 Probability and Statistics: Week 7


Summary
Summary

2155/2155

You might also like