F8: Heteroscedasticity

Feng Li
Department of Statistics, Stockholm University

What is so-called heteroscedasticity

In a linear regression model, we assume the error term has a normal
distribution with mean zero and variance of 2 , i.e.
Var(ui ) = 2
which is called homoscedasticity.
But when the error term does not have constant variance, i.e.
Var(ui ) = 2i
we call it heteroscedasticity.
See the differences between the two pictures for the model
Saving = + Income + ui

Feng Li (Stockholm University)

Econometrics

2/1

An OLS example

Recall the model Yi = 1 + 2 Xi + ui .

If the error term ui is homoscedastic with variance 2 , we know we have
BLUE estimators and

xi yi
2
2 = 2 , Var(

2 ) = 2 .
xi
xi
If the error term ui is homoscedastic with variance 2i , we have
2 2

x
xi yi
2
2 = 2 , Var(

2 ) = i2 i2 2 .
xi
( xi )
xi
why? see Appendix 11A.1.

2 is still linear and unbiased, why?

But it is not best anymore, i.e. will not grant the minimum variance.

Feng Li (Stockholm University)

Econometrics

3/1

Use GLS to take heteroscedasticity into account (1)

The OLS method treats every observation equally and does not take
heteroscedasticity into account.
The generalized least squares (GLS) will.

Consider the heteroscedasticity model

Yi = 1 + 2 X1i + ui , where Var(ui ) = 2i
?
If we transform the model by dividing 1/ wi where wi = 1/2i at both sides
(assume i is known),
Yi
1
Xi
ui
= 1
+ 2
+
i
i
i
i
which can be rewritten as

Yi = 1 X
0i + 2 X1i + ui ,

and u
i = ui /i is the new error term for the new model.
Var(ui /i ) = 1 is now a constant. why?

We call
1 2 as GLS estimators

Yi =
1 X0i + 2 X1i ,

Feng Li (Stockholm University)

Econometrics

4/1

Use GLS to take heteroscedasticity into account (2)

To obtain GLS estimators, we minimize

2 X )2
1 X
((
ui )2 ) = (Yi
1i
0i
which is done by the usual way we have done in OLS.
The GLS estimator of 2 is

wi Xi Yi wi Xi wi Yi
i
2 = w

wi wi X2i ( wi Xi )2
and the variance is
 
2 =
Var

wi

wi

wi X2i

wi Xi )2

where wi = 1/2i .
When wi = w = 1/2 , the GLS estimator will reduce to the OLS estimator.
Can you prove this?

= Y
X
= ( wi Xi )/ (wi ).
where Y = ( wi Yi )/ (wi ), X

1
2
At this particular setting wi = 1/2i , we call this is weighted least squares
(WLS) which is a special case of GLS.
is unbiased and Var(
) Var(
2 ).

2
2
Feng Li (Stockholm University)

Econometrics

5/1

Use GLS to take heteroscedasticity into account (2)*

It can be shown that

wi xi yi

2
wi xi

and
Var (2 ) =

1
wi xi

2

where
, y = Yi Y
xi = Xi X
i
See Exercise 11.5.

Feng Li (Stockholm University)

Econometrics

6/1

WLS example Example 11.7 (1)

Assume we want to make WLS regression with the give data.

What can you do then?

Option 1: Apply the general GLS formula in p.5 to obtain the estimators.
Option 2: Use OLS to regress Y/i with 1/i and Xi /i without intercept.

Can you obtain the same results?

Feng Li (Stockholm University)

Econometrics

7/1

WLS example Example 11.7 (2)

Compare the WLS results with OLS results.

How do the standard errors and t statistics change?

Feng Li (Stockholm University)

Econometrics

8/1

Consequences of using when heteroscedastic

Suppose there are heteroscedastic but we insist using OLS. What will go
wrong? whatever conclusions we draw may be misleading.
We could not establish confidence intervals and test hypotheses with usual t,
F tests.
The usual tests are likely to give larger variance than the true variance.
by OLS is a biased estimator of the true
The variance estimator of
variance.
2
i /(n 2) is biased.
The usual estimator of 2 which was u

Feng Li (Stockholm University)

Econometrics

9/1

Detection of heteroscedasticity (1)

2i against Yi
Plot u

Feng Li (Stockholm University)

Econometrics

10 / 1

Detection of heteroscedasticity (2)

2i against Xi
Plot u

Feng Li (Stockholm University)

Econometrics

11 / 1

Detection of heteroscedasticity (3)

QQ plot
If the residual is normally distributed, plot sample quantile for the residual
against the theoretical quantile from standard normal distribution should on
the 45 degree line.

Normal QQ Plot

Sample Quantiles

Theoretical Quantiles

Feng Li (Stockholm University)

Econometrics

12 / 1

Detection of heteroscedasticity (4)

Whites general heteroscedasticity test

H0 : No heteroscedasticity.
Consider Yi = 1 + 2 X2i + 3 X3i + ui , (other models are the same)
i.
step 1: Do the OLS to obtain the residuals u
step 2: Run the following model with the covariates and their crossproducts
21 = 1 + 2 X2i + 3 X3i + 4 X22i + 5 X23i + 6 X2i X3i + vi
u
and obtain R2 .
step 3: nR2 2 (k 1) where k is no. of unknown parameters in step 2.
step 4: If 2obs (k 1) 2crit (k 1), reject H0 .
Question: How do Whites test with Yi = 1 + 2 X2i + 3 X3i + 4 X4i + ui ?

Feng Li (Stockholm University)

Econometrics

13 / 1

Example of Whites test

Consider the following regression model with 41 observations,
lnYi = 1 + 2 lnX2i + 3 lnX3i + ui
where Y = ratio of trade taxes (import and export taxes) to total government
revenue, X2 = ratio of the sum of exports plus imports to GNP, and X3 =
GNP per capita.
By applying Whites heteroscedasticity test. We first obtain the residuals
from regression.
Then we do the following auxiliary regression

2i = 5.8+2.5 ln X2i +0.69 ln X3i 0.4(ln X2i )2 0.04(ln X3i )2 +0.002 ln X2i ln X3i
u
and R2 = 0.1148.
Can you compute the whites test statistic?
What is your conclusion of heteroscedasticity?

Feng Li (Stockholm University)

Econometrics

14 / 1

Detection of heteroscedasticity (5)

Goldfeld-Quandt test
It is popular to assume 2i is positively related to one of the covariates e.g.
2i = 2 X22i in a three covariates model.
The bigger Xi we have, the bigger 2i is.
H0 : Homoscedasticity
step 1: Sort covariates with the order of X2i
step 2: Delete the c central observations and dived the remaining parts into
two groups.
step 3: Fit the two groups separately with OLS and obtain RSS1 (for the
small values group) and RSS2 (for the large values group) with both
(n c)/2 k degrees of freedom. Why?
step 4: Compute the ratio
=

RSS2 /[(n c)/2 k]

F(((n c)/2 k), ((n c)/2 k))
RSS1 /[(n c)/2 k]

Reject H0 if Fcrit (((n c)/2 k), ((n c)/2 k)).

Feng Li (Stockholm University)

Econometrics

15 / 1

Detection of heteroscedasticity (6)

Breusch-Pagan-Godfrey test

Consider the model Yi = 1 + 2 X2i + ... + +k Xki + ui

Assume that 2i = 1 + 2 Z2i + ... + m Zmi where Zi are known variables
which can be Xi .
If there no heteroscedasticity, then 2 = ... = m = 0 and 2i = 1 .
1 , ..., u
n by the model.
step 1: Obtain u
2
2
= u
i /n.
step 2: Obtain
2i /
step 3: Construct variable pi = u
2
step 4: Regress pi = 1 + 2 Z2i + ... + m Zmi + vi
step 5: Obtain
ESS/2 2 (m 1)
Evidence of heteroscedasticity when ESS/2 2crit (m 1).

Feng Li (Stockholm University)

Econometrics

16 / 1

How to obtain estimators

with Yi = 1 + 2 Xi + ui when E(ui ) = 0 and Var(ui ) = 2i
When i is known: use WLS method to obtain BLUE estimators. pp. 45
When i is not known:

If V(ui ) = 2 X2i , do OLS with model

Yi
1
ui
= 2 + 1
+
Xi
Xi
Xi

and Var(ui /Xi ) = 2 . Why?

If Var(ui ) = 2 Xi (Xi 0), do OLS with model
1
u
Y
? i = 2 + 1 ? + ? i
Xi
Xi
Xi

?
and Var(ui / Xi ) = 2 . Why?
If Var(ui ) = 2 [E(Yi )]2 (Xi 0), do OLS with model
Yi
1
ui
= 2 + 1 +
Yi
Yi
Yi

and Var(ui /Yi ) Var(ui )/[EYi ]2 = Var(ui )/Yi 2 = 2 .

Do OLS with log transformed data lnYi = 1 + 2 lnXi + vi can also reduce
heteroscedasticity.

Feng Li (Stockholm University)

Econometrics

17 / 1